Max level shown:
1 starting January 15, 2025 at 05:18:52 AM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.2671339
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.020262236
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.021152548
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start ds: 8830
15 start access at:127.0.0.1:9998
16 Starting downstairs at port 8810
17 Starting downstairs at port 8830
18 Starting downstairs at port 8820
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Make output file at "/tmp/dsc/downstairs-8830.txt"
21 Make output file at "/tmp/dsc/downstairs-8810.txt"
22 [8810][0] initial start wait reports Starting
23 [8830][2] initial start wait reports Starting
24 [8820][1] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1192)
26 [8820][1] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1193)
28 [8810][0] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1194)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Jan 15 05:18:53.221 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Jan 15 05:18:58.311 INFO accepted connection, remote_addr: 127.0.0.1:57426, local_addr: 127.0.0.1:9998
36 Jan 15 05:18:58.313 INFO request completed, latency_us: 401, response_code: 200, uri: /regioninfo, method: GET, req_id: 9dcb1b2f-9ad6-4d70-b506-18b9a74d0217, remote_addr: 127.0.0.1:57426, local_addr: 127.0.0.1:9998
37 Jan 15 05:18:58.315 INFO request completed, latency_us: 122, response_code: 200, uri: /regioncount, method: GET, req_id: 505ff059-00e9-4468-9c3e-06517144379d, remote_addr: 127.0.0.1:57426, local_addr: 127.0.0.1:9998
382025-01-15T05:18:58.315ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392025-01-15T05:18:58.316ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Jan 15 05:18:58.317 INFO request completed, latency_us: 128, response_code: 200, uri: /port/cid/0, method: GET, req_id: 167071bb-ef99-42be-ae46-d3304e0ecaf4, remote_addr: 127.0.0.1:57426, local_addr: 127.0.0.1:9998
41 Jan 15 05:18:58.317 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/1, method: GET, req_id: 53d2ae82-4a96-4993-8d1b-87ea619e629d, remote_addr: 127.0.0.1:57426, local_addr: 127.0.0.1:9998
42 Jan 15 05:18:58.318 INFO request completed, latency_us: 94, response_code: 200, uri: /port/cid/2, method: GET, req_id: 424bea1c-ea2d-4511-b8e4-10cea9657501, remote_addr: 127.0.0.1:57426, local_addr: 127.0.0.1:9998
432025-01-15T05:18:58.318ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442025-01-15T05:18:58.319ZINFOcrucible: Upstairs starts
452025-01-15T05:18:58.319ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "008b9b7890522309c5657fa001c5655325f06c1f", git_commit_timestamp: "2025-01-15T05:03:03.000000000Z", git_branch: "renovate/http-1.x", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462025-01-15T05:18:58.319ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472025-01-15T05:18:58.319ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482025-01-15T05:18:58.321ZINFOcrucible: Crucible 78ec5640-41e6-4750-b3da-e748c8be8826 has session id: 3bdc822b-d016-4bce-93d8-f70cd4a05b55 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
492025-01-15T05:18:58.321ZINFOcrucible: Upstairs opts: Upstairs UUID: 78ec5640-41e6-4750-b3da-e748c8be8826, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
502025-01-15T05:18:58.321ZINFOcrucible: Crucible stats registered with UUID: 78ec5640-41e6-4750-b3da-e748c8be8826 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
512025-01-15T05:18:58.321ZWARNcrucible: could not find Downstairs address for Nexus session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
522025-01-15T05:18:58.323ZINFOcrucible: Downstairs in volume = 3 task = crutest
532025-01-15T05:18:58.323ZINFOcrucible: The guest has requested activation with gen:1
542025-01-15T05:18:58.323ZINFOcrucible: 78ec5640-41e6-4750-b3da-e748c8be8826 active request set session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
552025-01-15T05:18:58.323ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
562025-01-15T05:18:58.323ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
572025-01-15T05:18:58.323ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
582025-01-15T05:18:58.323ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
592025-01-15T05:18:58.323ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
602025-01-15T05:18:58.323ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
612025-01-15T05:18:58.323ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
622025-01-15T05:18:58.323ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
632025-01-15T05:18:58.323ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
642025-01-15T05:18:58.323ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62993) = io task client = 2 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
652025-01-15T05:18:58.323ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62594) = io task client = 0 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
662025-01-15T05:18:58.323ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42680) = io task client = 1 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
672025-01-15T05:18:58.325ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
682025-01-15T05:18:58.325ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
692025-01-15T05:18:58.325ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
702025-01-15T05:18:58.326ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
712025-01-15T05:18:58.326ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
722025-01-15T05:18:58.326ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
732025-01-15T05:18:58.327ZINFOcrucible: Waiting for more clients to be ready session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
742025-01-15T05:18:58.327ZINFOcrucible: Waiting for more clients to be ready session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
752025-01-15T05:18:58.327ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
762025-01-15T05:18:58.327ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
772025-01-15T05:18:58.327ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
782025-01-15T05:18:58.327ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
792025-01-15T05:18:58.327ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
802025-01-15T05:18:58.327ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
812025-01-15T05:18:58.328ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
822025-01-15T05:18:58.328ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
832025-01-15T05:18:58.328ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
842025-01-15T05:18:58.328ZINFOcrucible: Max found gen is 1 = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
852025-01-15T05:18:58.328ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
862025-01-15T05:18:58.328ZINFOcrucible: Next flush: 1 = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
872025-01-15T05:18:58.328ZINFOcrucible: All extents match = downstairs session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
882025-01-15T05:18:58.328ZINFOcrucible: No downstairs reconciliation required session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
892025-01-15T05:18:58.328ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
902025-01-15T05:18:58.328ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
912025-01-15T05:18:58.328ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
922025-01-15T05:18:58.328ZINFOcrucible: All required reconciliation work is completed session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
932025-01-15T05:18:58.328ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
94 {"msg":"78ec5640-41e6-4750-b3da-e748c8be8826 is now active with session: 3bdc822b-d016-4bce-93d8-f70cd4a05b55","v":0,"name":"crucible","level":30,"Activate successful
95 Wait for a query_work_queue command to finish before sending IO
96 time":"2025-01-15T05:18:58.328631303Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1198,"session_id":"3bdc822b-d016-4bce-93d8-f70cd4a05b55"}
972025-01-15T05:18:58.328ZINFOcrucible: Set Active after no reconciliation session_id = 3bdc822b-d016-4bce-93d8-f70cd4a05b55
982025-01-15T05:18:58.328ZINFOcrucible: The guest has finished waiting for activation with:1
99 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
100 Fill test
101 Read and Verify all blocks (0..5000 range:false)
102 Wrote out file "/tmp/test_replay_verify.log"
103 CLIENT: Tests done. All submitted work has been ACK'd
104 ----------------------------------------------------------------
105 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
106 States: Active Active Active
107 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
108 1021 Acked Read 256 Done Done Done false
109 1022 Acked Read 256 Done Done Done false
110 1023 Acked Read 256 Done Done Done false
111 1024 Acked Read 256 Done Done Done false
112 1025 Acked Read 256 Done Done Done false
113 1026 Acked Read 256 Done Done Done false
114 1027 Acked Read 256 Done Done Done false
115 1028 Acked Read 256 Done Done Done false
116 1029 Acked Read 256 Done Done Done false
117 1030 Acked Read 256 Done Done Done false
118 1031 Acked Read 256 Done Done Done false
119 1032 Acked Read 256 Done Done Done false
120 1033 Acked Read 256 Done Done Done false
121 1034 Acked Read 256 Done Done Done false
122 1035 Acked Read 256 Done Done Done false
123 1036 Acked Read 256 Done Done Done false
124 1037 Acked Read 256 Done Done Done false
125 1038 Acked Read 256 Done Done Done false
126 1039 Acked Read 136 Done Done Done false
127 1040 Acked Read 256 Done Done Done false
128 STATES DS:0 DS:1 DS:2 TOTAL
129 Sent 0 0 0 0
130 Done 20 20 20 60
131 Skipped 0 0 0 0
132 Error 0 0 0 0
133 Last Flush: 1020 1020 1020
134 Downstairs last five completed: 1020 1019 1018 1017 1016
135 Upstairs last five completed: 1040 1039 1038 1037 1036
136 CLIENT: Up:0 ds:20 act:3
137 ----------------------------------------------------------------
138 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
139 Downstairs last five completed: 1041 1040 1039 1038 1037
140 Upstairs last five completed: 1041 1040 1039 1038 1037
141 CLIENT: Up:0 ds:0 act:3
142 CLIENT: All crucible jobs finished, exiting program
1432025-01-15T05:19:02.897ZWARNcrucible: Guest handle has been dropped
144 Replay loop starts now January 15, 2025 at 05:19:02 AM UTC
145 Jan 15 05:19:02.966 INFO accepted connection, remote_addr: 127.0.0.1:51623, local_addr: 127.0.0.1:9998
146 Jan 15 05:19:02.966 INFO request completed, latency_us: 139, response_code: 200, uri: /regioninfo, method: GET, req_id: fa0d2af3-d54b-4300-94ec-dd6f8284a774, remote_addr: 127.0.0.1:51623, local_addr: 127.0.0.1:9998
1472025-01-15T05:19:02.967ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
148 Jan 15 05:19:02.967 INFO request completed, latency_us: 103, response_code: 200, uri: /regioncount, method: GET, req_id: fe59e71a-19e4-4d3d-b820-5bafc4ed668c, remote_addr: 127.0.0.1:51623, local_addr: 127.0.0.1:9998
1492025-01-15T05:19:02.968ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
150 Jan 15 05:19:02.968 INFO request completed, latency_us: 106, response_code: 200, uri: /port/cid/0, method: GET, req_id: e37d7484-1fc2-4f7b-b2c8-e152c13a8738, remote_addr: 127.0.0.1:51623, local_addr: 127.0.0.1:9998
151 Jan 15 05:19:02.968 INFO request completed, latency_us: 86, response_code: 200, uri: /port/cid/1, method: GET, req_id: 5853c1db-2ac6-41fc-948e-2935d670670e, remote_addr: 127.0.0.1:51623, local_addr: 127.0.0.1:9998
152 Jan 15 05:19:02.969 INFO request completed, latency_us: 107, response_code: 200, uri: /port/cid/2, method: GET, req_id: f081062f-917d-482e-95fc-004a95c65198, remote_addr: 127.0.0.1:51623, local_addr: 127.0.0.1:9998
1532025-01-15T05:19:02.969ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1542025-01-15T05:19:02.969ZINFOcrucible: Upstairs starts
1552025-01-15T05:19:02.969ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "008b9b7890522309c5657fa001c5655325f06c1f", git_commit_timestamp: "2025-01-15T05:03:03.000000000Z", git_branch: "renovate/http-1.x", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1562025-01-15T05:19:02.969ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1572025-01-15T05:19:02.969ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1582025-01-15T05:19:02.970ZINFOcrucible: Crucible 80f30d86-045b-4a67-aab5-3ac8e5d1f140 has session id: 91709b1d-df49-4651-b81a-5daddf421e6f session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1592025-01-15T05:19:02.970ZINFOcrucible: Upstairs opts: Upstairs UUID: 80f30d86-045b-4a67-aab5-3ac8e5d1f140, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1602025-01-15T05:19:02.970ZINFOcrucible: Crucible stats registered with UUID: 80f30d86-045b-4a67-aab5-3ac8e5d1f140 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1612025-01-15T05:19:02.970ZWARNcrucible: could not find Downstairs address for Nexus session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1622025-01-15T05:19:02.971ZINFOcrucible: Downstairs in volume = 3 task = crutest
1632025-01-15T05:19:02.971ZINFOcrucible: The guest has requested activation with gen:2
1642025-01-15T05:19:02.971ZINFOcrucible: 80f30d86-045b-4a67-aab5-3ac8e5d1f140 active request set session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1652025-01-15T05:19:02.971ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1662025-01-15T05:19:02.971ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1672025-01-15T05:19:02.971ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1682025-01-15T05:19:02.971ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1692025-01-15T05:19:02.971ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1702025-01-15T05:19:02.971ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1712025-01-15T05:19:02.971ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1722025-01-15T05:19:02.971ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1732025-01-15T05:19:02.971ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1742025-01-15T05:19:02.971ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59289) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1752025-01-15T05:19:02.971ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39562) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1762025-01-15T05:19:02.971ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51236) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1772025-01-15T05:19:02.972ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1782025-01-15T05:19:02.972ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1792025-01-15T05:19:02.972ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1802025-01-15T05:19:02.972ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1812025-01-15T05:19:02.973ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1822025-01-15T05:19:02.973ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1832025-01-15T05:19:02.973ZINFOcrucible: Waiting for more clients to be ready session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1842025-01-15T05:19:02.973ZINFOcrucible: Waiting for more clients to be ready session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1852025-01-15T05:19:02.973ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1862025-01-15T05:19:02.973ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1872025-01-15T05:19:02.973ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1882025-01-15T05:19:02.973ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1892025-01-15T05:19:02.973ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1902025-01-15T05:19:02.973ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1912025-01-15T05:19:02.974ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1922025-01-15T05:19:02.974ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1932025-01-15T05:19:02.974ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1942025-01-15T05:19:02.974ZINFOcrucible: Max found gen is 2 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1952025-01-15T05:19:02.974ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1962025-01-15T05:19:02.974ZINFOcrucible: Next flush: 2 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1972025-01-15T05:19:02.974ZINFOcrucible: All extents match = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198 {"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30Activate successful
199 Wait for a query_work_queue command to finish before sending IO
200 ,"time":"2025-01-15T05:19:02.974106712Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
2012025-01-15T05:19:02.974ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2022025-01-15T05:19:02.974ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2032025-01-15T05:19:02.974ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2042025-01-15T05:19:02.974ZINFOcrucible: All required reconciliation work is completed session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2052025-01-15T05:19:02.974ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2062025-01-15T05:19:02.974ZINFOcrucible: 80f30d86-045b-4a67-aab5-3ac8e5d1f140 is now active with session: 91709b1d-df49-4651-b81a-5daddf421e6f session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2072025-01-15T05:19:02.974ZINFOcrucible: Set Active after no reconciliation session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2082025-01-15T05:19:02.974ZINFOcrucible: The guest has finished waiting for activation with:2
209 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
210 Loading write count information from file "/tmp/test_replay_verify.log"
211 Jan 15 05:19:03.006 INFO accepted connection, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
212 Main task has work to do, go find it
213 got dsc Stop(2)
214 stop 2
215 Jan 15 05:19:03.007 INFO request completed, latency_us: 155, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 482a741a-082a-4258-bdb3-6d0f9f94fb9a, remote_addr: 127.0.0.[8830] Got stop action so:false kr:false
216 1:42402, local_addr: 127.0.0.1:9998
2172025-01-15T05:19:03.007ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2182025-01-15T05:19:03.007ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2192025-01-15T05:19:03.008ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2202025-01-15T05:19:03.008ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
221 Jan 15 05:19:03.008 INFO request completed, latency_us: 309, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9d1ec837-0eed-4fc6-87e0-bb7bdddaed8b, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
222 [8830] Exited with: ExitStatus(unix_wait_status(9))
223 [8830][2] reports Exit
224 Jan 15 05:19:07.010 INFO request completed, latency_us: 178, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8390d2ae-e8c9-4d44-9521-4143e24faa22, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
225 001/300 Read block 0 len 20480
226 002/300 Read block 162 len 28672
227 003/300 Write block 3795 len 24576 data: 2 2 2 2 2 2
228 004/300 Read block 4440 len 32768
229 005/300 Read block 2712 len 12288
230 006/300 Write block 4282 len 36864 data: 2 2 2 2 2 2 2 2 2
231 007/300 Read block 4493 len 32768
232 008/300 Write block 3139 len 20480 data: 2 2 2 2 2
233 009/300 Read block 982 len 40960
234 010/300 Write block 3181 len 12288 data: 2 2 2
235 011/300 Read block 833 len 8192
236 012/300 Write block 1369 len 20480 data: 2 2 2 2 2
237 013/300 Read block 2669 len 28672
238 014/300 Write block 2203 len 36864 data: 2 2 2 2 2 2 2 2 2
239 015/300 Write block 672 len 16384 data: 2 2 2 2
240 016/300 Read block 2489 len 40960
241 017/300 Read block 4199 len 36864
242 018/300 Write block 337 len 36864 data: 2 2 2 2 2 2 2 2 2
243 019/300 Read block 1374 len 24576
244 020/300 Write block 4900 len 40960 data: 2 2 2 2 2 2 2 2 2 2
245 021/300 Read block 4884 len 40960
246 022/300 Read block 3095 len 36864
247 023/300 Read block 439 len 4096
248 024/300 Write block 2302 len 8192 data: 2 2
249 025/300 Write block 4561 len 40960 data: 2 2 2 2 2 2 2 2 2 2
250 026/300 Write block 2758 len 32768 data: 2 2 2 2 2 2 2 2
251 027/300 Write block 4257 len 32768 data: 2 2 2 2 2 2 2 2
252 028/300 Read block 1586 len 4096
253 029/300 Write block 2936 len 16384 data: 2 2 2 2
254 030/300 Read block 4679 len 12288
255 031/300 Read block 4306 len 12288
256 032/300 Read block 1086 len 36864
257 033/300 Read block 2656 len 4096
258 034/300 Write block 1945 len 4096 data: 2
259 035/300 Read block 4052 len 8192
260 036/300 Read block 865 len 40960
261 037/300 Write block 1704 len 28672 data: 2 2 2 2 2 2 2
262 038/300 Write block 2646 len 40960 data: 2 2 2 2 2 2 2 2 2 2
263 039/300 Read block 2424 len 8192
264 040/300 Read block 3303 len 32768
265 041/300 Flush
266 042/300 Write block 2982 len 24576 data: 2 2 2 2 2 2
267 043/300 Read block 781 len 28672
268 044/300 Write block 3943 len 36864 data: 2 2 2 2 2 2 2 2 2
269 045/300 Write block 743 len 24576 data: 2 2 2 2 2 2
270 046/300 Write block 4909 len 36864 data: 3 2 2 2 2 2 2 2 2
271 047/300 Write block 4029 len 20480 data: 2 2 2 2 2
272 048/300 Write block 4239 len 32768 data: 2 2 2 2 2 2 2 2
273 049/300 Read block 4986 len 40960
274 050/300 Read block 959 len 24576
275 051/300 Read block 4713 len 4096
276 052/300 Flush
277 053/300 Write block 1108 len 20480 data: 2 2 2 2 2
278 054/300 Write block 3178 len 40960 data: 2 2 2 3 3 3 2 2 2 2
279 055/300 Write block 4158 len 20480 data: 2 2 2 2 2
280 056/300 Write block 2833 len 32768 data: 2 2 2 2 2 2 2 2
281 057/300 Write block 1537 len 36864 data: 2 2 2 2 2 2 2 2 2
282 058/300 Write block 3943 len 16384 data: 3 3 3 3
283 059/300 Read block 23 len 12288
284 060/300 Read block 1094 len 40960
285 061/300 Write block 1305 len 12288 data: 2 2 2
286 062/300 Read block 3109 len 20480
287 063/300 Read block 4799 len 32768
288 064/300 Write block 1649 len 4096 data: 2
289 065/300 Read block 918 len 16384
290 066/300 Write block 566 len 4096 data: 2
291 067/300 Read block 3313 len 8192
292 068/300 Read block 2796 len 40960
293 069/300 Read block 317 len 12288
294 070/300 Read block 170 len 36864
295 071/300 Write block 1010 len 16384 data: 2 2 2 2
296 072/300 Write block 3382 len 28672 data: 2 2 2 2 2 2 2
297 073/300 Write block 1713 len 12288 data: 2 2 2
298 074/300 Write block 836 len 16384 data: 2 2 2 2
299 075/300 Write block 643 len 40960 data: 2 2 2 2 2 2 2 2 2 2
300 076/300 Write block 4045 len 40960 data: 2 2 2 2 2 2 2 2 2 2
301 077/300 Write block 4368 len 8192 data: 2 2
302 078/300 Read block 4304 len 16384
303 079/300 Read block 2568 len 36864
304 080/300 Read block 1759 len 24576
305 081/300 Read block 2408 len 32768
306 082/300 Write block 2996 len 8192 data: 2 2
307 083/300 Read block 1807 len 4096
308 084/300 Write block 2109 len 20480 data: 2 2 2 2 2
309 085/300 Read block 656 len 28672
310 086/300 Read block 4175 len 40960
311 087/300 Flush
312 088/300 Write block 3462 len 16384 data: 2 2 2 2
313 089/300 Read block 736 len 36864
314 090/300 Write block 713 len 12288 data: 2 2 2
315 091/300 Write block 2961 len 20480 data: 2 2 2 2 2
316 092/300 Flush
317 093/300 Read block 2658 len 4096
318 094/300 Read block 3396 len 16384
319 095/300 Write block 3844 len 4096 data: 2
320 096/300 Write block 637 len 40960 data: 2 2 2 2 2 2 3 3 3 3
321 097/300 Read block 457 len 36864
322 098/300 Write block 574 len 4096 data: 2
323 099/300 Read block 1613 len 40960
324 100/300 Write block 4410 len 36864 data: 2 2 2 2 2 2 2 2 2
325 101/300 Read block 3427 len 28672
326 102/300 Flush
327 103/300 Write block 1533 len 4096 data: 2
328 104/300 Read block 1224 len 8192
329 105/300 Read block 2139 len 40960
330 106/300 Flush
331 107/300 Read block 3472 len 28672
332 108/300 Read block 4296 len 8192
333 109/300 Read block 1449 len 36864
334 110/300 Read block 949 len 8192
335 111/300 Read block 3704 len 32768
336 112/300 Read block 219 len 32768
337 113/300 Read block 3273 len 12288
338 114/300 Read block 69 len 40960
339 115/300 Write block 2366 len 40960 data: 2 2 2 2 2 2 2 2 2 2
340 116/300 Write block 2833 len 40960 data: 3 3 3 3 3 3 3 3 2 2
341 117/300 Write block 965 len 24576 data: 2 2 2 2 2 2
342 118/300 Write block 3694 len 20480 data: 2 2 2 2 2
343 119/300 Write block 3527 len 8192 data: 2 2
344 120/300 Read block 2432 len 32768
345 121/300 Read block 3629 len 36864
346 122/300 Read block 3909 len 36864
347 123/300 Write block 3938 len 36864 data: 2 2 2 2 2 4 4 4 4
348 124/300 Write block 3122 len 28672 data: 2 2 2 2 2 2 2
349 125/300 Write block 3180 len 20480 data: 3 4 4 4 3
350 126/300 Write block 4196 len 12288 data: 2 2 2
351 127/300 Write block 647 len 28672 data: 3 3 3 3 3 3 2
352 128/300 Read block 3239 len 32768
353 129/300 Read block 3274 len 8192
354 130/300 Write block 41 len 16384 data: 2 2 2 2
355 131/300 Read block 4013 len 40960
356 132/300 Write block 3299 len 20480 data: 2 2 2 2 2
357 133/300 Read block 4707 len 12288
358 134/300 Write block 411 len 12288 data: 2 2 2
359 135/300 Write block 2460 len 28672 data: 2 2 2 2 2 2 2
360 136/300 Flush
361 137/300 Read block 1653 len 28672
362 138/300 Write block 2009 len 12288 data: 2 2 2
363 139/300 Write block 2485 len 8192 data: 2 2
364 140/300 Write block 1924 len 4096 data: 2
365 141/300 Read block 4116 len 12288
366 142/300 Read block 3777 len 32768
367 143/300 Write block 1692 len 8192 data: 2 2
368 144/300 Write block 866 len 32768 data: 2 2 2 2 2 2 2 2
369 145/300 Flush
370 146/300 Read block 115 len 36864
371 147/300 Read block 3046 len 24576
372 148/300 Write block 768 len 12288 data: 2 2 2
373 149/300 Write block 295 len 8192 data: 2 2
374 150/300 Write block 4729 len 16384 data: 2 2 2 2
375 151/300 Write block 3057 len 8192 data: 2 2
376 152/300 Write block 481 len 8192 data: 2 2
377 153/300 Read block 4153 len 32768
378 154/300 Write block 4396 len 8192 data: 2 2
379 155/300 Read block 4468 len 36864
380 156/300 Read block 2245 len 24576
381 157/300 Read block 811 len 20480
382 158/300 Write block 3854 len 24576 data: 2 2 2 2 2 2
383 159/300 Read block 4577 len 36864
384 160/300 Read block 4910 len 28672
385 161/300 Read block 2177 len 24576
386 162/300 Write block 676 len 12288 data: 2 2 2
387 163/300 Write block 4131 len 20480 data: 2 2 2 2 2
388 164/300 Write block 4137 len 8192 data: 2 2
389 165/300 Write block 3492 len 28672 data: 2 2 2 2 2 2 2
390 166/300 Read block 4579 len 20480
391 167/300 Write block 2362 len 28672 data: 2 2 2 2 3 3 3
392 168/300 Write block 3382 len 4096 data: 3
393 169/300 Read block 214 len 16384
394 170/300 Read block 4153 len 32768
395 171/300 Read block 4828 len 8192
396 172/300 Write block 4018 len 20480 data: 2 2 2 2 2
397 173/300 Write block 1800 len 28672 data: 2 2 2 2 2 2 2
398 174/300 Read block 4649 len 36864
399 175/300 Read block 3438 len 28672
400 176/300 Write block 819 len 8192 data: 2 2
401 177/300 Write block 3575 len 40960 data: 2 2 2 2 2 2 2 2 2 2
402 178/300 Read block 851 len 12288
403 179/300 Read block 1142 len 32768
404 180/300 Write block 3999 len 28672 data: 2 2 2 2 2 2 2
405 181/300 Write block 4728 len 32768 data: 2 3 3 3 3 2 2 2
406 182/300 Write block 937 len 4096 data: 2
407 183/300 Write block 4263 len 12288 data: 3 3 2
408 184/300 Read block 2402 len 28672
409 185/300 Write block 3972 len 20480 data: 2 2 2 2 2
410 186/300 Read block 4869 len 24576
411 187/300 Read block 1038 len 20480
412 188/300 Read block 4888 len 36864
413 189/300 Read block 3690 len 36864
414 190/300 Write block 466 len 36864 data: 2 2 2 2 2 2 2 2 2
415 191/300 Read block 4989 len 16384
416 192/300 Write block 3156 len 24576 data: 2 2 2 2 2 2
417 193/300 Write block 3256 len 8192 data: 2 2
418 194/300 Read block 2877 len 32768
419 195/300 Flush
420 196/300 Read block 135 len 32768
421 197/300 Write block 3906 len 28672 data: 2 2 2 2 2 2 2
422 198/300 Flush
423 199/300 Write block 3264 len 24576 data: 2 2 2 2 2 2
424 200/300 Read block 2698 len 16384
425 201/300 Write block 2917 len 28672 data: 2 2 2 2 2 2 2
426 202/300 Read block 3728 len 32768
427 203/300 Read block 1923 len 40960
428 204/300 Write block 892 len 16384 data: 2 2 2 2
429 205/300 Write block 3155 len 28672 data: 2 3 3 3 3 3 3
430 206/300 Write block 2792 len 28672 data: 2 2 2 2 2 2 2
431 207/300 Read block 4317 len 40960
432 208/300 Read block 3451 len 8192
433 209/300 Read block 1302 len 4096
434 210/300 Write block 3232 len 28672 data: 2 2 2 2 2 2 2
435 211/300 Write block 357 len 40960 data: 2 2 2 2 2 2 2 2 2 2
436 212/300 Read block 2319 len 12288
437 213/300 Read block 3167 len 32768
438 214/300 Read block 3720 len 8192
439 215/300 Read block 683 len 20480
440 216/300 Read block 1909 len 40960
441 217/300 Read block 2166 len 28672
442 218/300 Read block 3713 len 8192
443 219/300 Read block 1825 len 12288
444 220/300 Read block 3209 len 12288
445 221/300 Read block 328 len 40960
446 222/300 Write block 1087 len 8192 data: 2 2
447 223/300 Read block 4056 len 40960
448 224/300 Write block 126 len 4096 data: 2
449 225/300 Flush
450 226/300 Read block 2356 len 8192
451 227/300 Write block 697 len 32768 data: 2 2 2 2 2 2 2 2
452 228/300 Write block 2459 len 20480 data: 2 3 3 3 3
453 229/300 Read block 3078 len 20480
454 230/300 Read block 4704 len 12288
455 231/300 Write block 3415 len 20480 data: 2 2 2 2 2
456 232/300 Read block 137 len 20480
457 233/300 Write block 4669 len 20480 data: 2 2 2 2 2
458 234/300 Read block 1798 len 4096
459 235/300 Write block 199 len 8192 data: 2 2
460 236/300 Read block 2755 len 4096
461 237/300 Read block 4182 len 20480
462 238/300 Write block 4721 len 4096 data: 2
463 239/300 Read block 1112 len 12288
464 240/300 Read block 2042 len 36864
465 241/300 Read block 27 len 24576
466 242/300 Read block 2578 len 36864
467 243/300 Write block 4450 len 4096 data: 2
468 244/300 Read block 1784 len 40960
469 245/300 Read block 1948 len 36864
470 246/300 Read block 2907 len 4096
471 247/300 Read block 344 len 28672
472 248/300 Read block 2076 len 32768
473 249/300 Read block 3224 len 32768
474 250/300 Read block 4480 len 20480
475 251/300 Write block 2965 len 4096 data: 3
476 252/300 Write block 310 len 8192 data: 2 2
477 253/300 Write block 3553 len 16384 data: 2 2 2 2
478 254/300 Write block 1416 len 36864 data: 2 2 2 2 2 2 2 2 2
479 255/300 Write block 4627 len 24576 data: 2 2 2 2 2 2
480 256/300 Write block 3982 len 4096 data: 2
481 257/300 Read block 2356 len 8192
482 258/300 Write block 2703 len 4096 data: 2
483 259/300 Flush
484 260/300 Read block 3769 len 24576
485 261/300 Write block 1461 len 20480 data: 2 2 2 2 2
486 262/300 Read block 1569 len 8192
487 263/300 Read block 538 len 16384
488 264/300 Write block 3924 len 12288 data: 2 2 2
489 265/300 Write block 2445 len 24576 data: 2 2 2 2 2 2
490 266/300 Read block 1208 len 8192
491 267/300 Read block 3978 len 24576
492 268/300 Read block 4898 len 12288
493 269/300 Write block 4493 len 36864 data: 2 2 2 2 2 2 2 2 2
494 270/300 Write block 1872 len 32768 data: 2 2 2 2 2 2 2 2
495 271/300 Read block 2576 len 40960
496 272/300 Flush
497 273/300 Write block 2353 len 8192 data: 2 2
498 274/300 Write block 4380 len 12288 data: 2 2 2
499 275/300 Write block 1596 len 12288 data: 2 2 2
500 276/300 Read block 2808 len 16384
501 277/300 Write block 1118 len 32768 data: 2 2 2 2 2 2 2 2
502 278/300 Read block 1560 len 4096
503 279/300 Flush
504 280/300 Flush
505 281/300 Read block 4771 len 28672
506 282/300 Flush
507 283/300 Read block 450 len 32768
508 284/300 Read block 505 len 32768
509 285/300 Write block 4545 len 32768 data: 2 2 2 2 2 2 2 2
510 286/300 Read block 1719 len 12288
511 287/300 Read block 3008 len 28672
512 288/300 Read block 460 len 20480
513 289/300 Write block 4377 len 36864 data: 2 2 2 3 3 3 2 2 2
514 290/300 Read block 2979 len 12288
515 291/300 Read block 2763 len 20480
516 292/300 Write block 1502 len 16384 data: 2 2 2 2
517 293/300 Write block 3310 len 8192 data: 2 2
518 294/300 Read block 2164 len 40960
519 295/300 Read block 1130 len 4096
520 296/300 Write block 2021 len 36864 data: 2 2 2 2 2 2 2 2 2
521 297/300 Read block 1806 len 20480
522 298/300 Read block 4313 len 28672
523 299/300 Write block 3708 len 40960 data: 2 2 2 2 2 2 2 2 2 2
524 300/300 Write block 3795 len 24576 data: 3 3 3 3 3 3
525 301/300 Write block 4127 len 24576 data: 2 2 2 2 3 3
526 Main task has work to do, go find it
527 got dsc Start(2)
528 start 2
529 Jan 15 05:19:07.250 INFO request completed, latency_us: 233, response_code: 204, uri: /start/cid/2, method: POST, req_id: 5800647c-3108-4f5d-8526-65593e895f3e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
530 Replay: started 2, returned:Ok(())
531 ----------------------------------------------------------------
532 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
533 States: Active Active Offline
534 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
535 1000 Acked Read 5 Done Done Sent false
536 1001 Acked Read 7 Done Done Sent false
537 1002 Acked Write 6 Done Done Sent false
538 1003 Acked Read 8 Done Done Sent false
539 1004 Acked Read 3 Done Done Sent false
540 1005 Acked Write 9 Done Done Sent false
541 1006 Acked Read 8 Done Done Sent false
542 1007 Acked Write 5 Done Done Sent false
543 1008 Acked Read 10 Done Done Sent false
544 1009 Acked Write 3 Done Done Sent false
545 1010 Acked Read 2 Done Done Sent false
546 1011 Acked Write 5 Done Done Sent false
547 1012 Acked Read 7 Done Done Sent false
548 1013 Acked Write 9 Done Done Sent false
549 1014 Acked Write 4 Done Done Sent false
550 1015 Acked Read 10 Done Done Sent false
551 1016 Acked Read 9 Done Done Sent false
552 1017 Acked Write 9 Done Done Sent false
553 1018 Acked Read 6 Done Done Sent false
554 1019 Acked Write 10 Done Done Sent false
555 1020 Acked Read 10 Done Done Sent false
556 1021 Acked Read 9 Done Done Sent false
557 1022 Acked Read 1 Done Done Sent false
558 1023 Acked Write 2 Done Done Sent false
559 1024 Acked Write 10 Done Done Sent false
560 1025 Acked Write 8 Done Done Sent false
561 1026 Acked Write 8 Done Done Sent false
562 1027 Acked Read 1 Done Done Sent false
563 1028 Acked Write 4 Done Done Sent false
564 1029 Acked Read 3 Done Done Sent false
565 1030 Acked Read 3 Done Done Sent false
566 1031 Acked Read 9 Done Done Sent false
567 1032 Acked Read 1 Done Done Sent false
568 1033 Acked Write 1 Done Done Sent false
569 1034 Acked Read 2 Done Done Sent false
570 1035 Acked Read 10 Done Done Sent false
571 1036 Acked Write 7 Done Done Sent false
572 1037 Acked Write 10 Done Done Sent false
573 1038 Acked Read 2 Done Done Sent false
574 1039 Acked Read 8 Done Done Sent false
575 1040 Acked Flush 0 Done Done Sent false
576 1041 Acked Write 6 Done Done Sent false
577 1042 Acked Read 7 Done Done Sent false
578 1043 Acked Write 9 Done Done Sent false
579 1044 Acked Write 6 Done Done Sent false
580 1045 Acked Write 9 Done Done Sent false
581 1046 Acked Write 5 Done Done Sent false
582 1047 Acked Write 8 Done Done Sent false
583 1048 Acked Read 10 Done Done Sent false
584 1049 Acked Read 6 Done Done Sent false
585 1050 Acked Read 1 Done Done Sent false
586 1051 Acked Flush 0 Done Done Sent false
587 1052 Acked Write 5 Done Done Sent false
588 1053 Acked Write 10 Done Done Sent false
589 1054 Acked Write 5 Done Done Sent false
590 1055 Acked Write 8 Done Done Sent false
591 1056 Acked Write 9 Done Done Sent false
592 1057 Acked Write 4 Done Done Sent false
593 1058 Acked Read 3 Done Done Sent false
594 1059 Acked Read 10 Done Done Sent false
595 1060 Acked Write 3 Done Done Sent false
596 1061 Acked Read 5 Done Done Sent false
597 1062 Acked Read 8 Done Done Sent false
598 1063 Acked Write 1 Done Done Sent false
599 1064 Acked Read 4 Done Done Sent false
600 1065 Acked Write 1 Done Done Sent false
601 1066 Acked Read 2 Done Done Sent false
602 1067 Acked Read 10 Done Done Sent false
603 1068 Acked Read 3 Done Done Sent false
604 1069 Acked Read 9 Done Done Sent false
605 1070 Acked Write 4 Done Done Sent false
606 1071 Acked Write 7 Done Done Sent false
607 1072 Acked Write 3 Done Done Sent false
608 1073 Acked Write 4 Done Done Sent false
609 1074 Acked Write 10 Done Done Sent false
610 1075 Acked Write 10 Done Done Sent false
611 1076 Acked Write 2 Done Done Sent false
612 1077 Acked Read 4 Done Done Sent false
613 1078 Acked Read 9 Done Done Sent false
614 1079 Acked Read 6 Done Done Sent false
615 1080 Acked Read 8 Done Done Sent false
616 1081 Acked Write 2 Done Done Sent false
617 1082 Acked Read 1 Done Done Sent false
618 1083 Acked Write 5 Done Done Sent false
619 1084 Acked Read 7 Done Done Sent false
620 1085 Acked Read 10 Done Done Sent false
621 1086 Acked Flush 0 Done Done Sent false
622 1087 Acked Write 4 Done Done Sent false
623 1088 Acked Read 9 Done Done Sent false
624 1089 Acked Write 3 Done Done Sent false
625 1090 Acked Write 5 Done Done Sent false
626 1091 Acked Flush 0 Done Done Sent false
627 1092 Acked Read 1 Done Done Sent false
628 1093 Acked Read 4 Done Done Sent false
629 1094 Acked Write 1 Done Done Sent false
630 1095 Acked Write 10 Done Done Sent false
631 1096 Acked Read 9 Done Done Sent false
632 1097 Acked Write 1 Done Done Sent false
633 1098 Acked Read 10 Done Done Sent false
634 1099 Acked Write 9 Done Done Sent false
635 1100 Acked Read 7 Done Done Sent false
636 1101 Acked Flush 0 Done Done Sent false
637 1102 Acked Write 1 Done Done Sent false
638 1103 Acked Read 2 Done Done Sent false
639 1104 Acked Read 10 Done Done Sent false
640 1105 Acked Flush 0 Done Done Sent false
641 1106 Acked Read 7 Done Done Sent false
642 1107 Acked Read 2 Done Done Sent false
643 1108 Acked Read 9 Done Done Sent false
644 1109 Acked Read 2 Done Done Sent false
645 1110 Acked Read 8 Done Done Sent false
646 1111 Acked Read 8 Done Done Sent false
647 1112 Acked Read 3 Done Done Sent false
648 1113 Acked Read 10 Done Done Sent false
649 1114 Acked Write 10 Done Done Sent false
650 1115 Acked Write 10 Done Done Sent false
651 1116 Acked Write 6 Done Done Sent false
652 1117 Acked Write 5 Done Done Sent false
653 1118 Acked Write 2 Done Done Sent false
654 1119 Acked Read 8 Done Done Sent false
655 1120 Acked Read 9 Done Done Sent false
656 1121 Acked Read 9 Done Done Sent false
657 1122 Acked Write 9 Done Done Sent false
658 1123 Acked Write 7 Done Done Sent false
659 1124 Acked Write 5 Done Done Sent false
660 1125 Acked Write 3 Done Done Sent false
661 1126 Acked Write 7 Done Done Sent false
662 1127 Acked Read 8 Done Done Sent false
663 1128 Acked Read 2 Done Done Sent false
664 1129 Acked Write 4 Done Done Sent false
665 1130 Acked Read 10 Done Done Sent false
666 1131 Acked Write 5 Done Done Sent false
667 1132 Acked Read 3 Done Done Sent false
668 1133 Acked Write 3 Done Done Sent false
669 1134 Acked Write 7 Done Done Sent false
670 1135 Acked Flush 0 Done Done Sent false
671 1136 Acked Read 7 Done Done Sent false
672 1137 Acked Write 3 Done Done Sent false
673 1138 Acked Write 2 Done Done Sent false
674 1139 Acked Write 1 Done Done Sent false
675 1140 Acked Read 3 Done Done Sent false
676 1141 Acked Read 8 Done Done Sent false
677 1142 Acked Write 2 Done Done Sent false
678 1143 Acked Write 8 Done Done Sent false
679 1144 Acked Flush 0 Done Done Sent false
680 1145 Acked Read 9 Done Done Sent false
681 1146 Acked Read 6 Done Done Sent false
682 1147 Acked Write 3 Done Done Sent false
683 1148 Acked Write 2 Done Done Sent false
684 1149 Acked Write 4 Done Done Sent false
685 1150 Acked Write 2 Done Done Sent false
686 1151 Acked Write 2 Done Done Sent false
687 1152 Acked Read 8 Done Done Sent false
688 1153 Acked Write 2 Done Done Sent false
689 1154 Acked Read 9 Done Done Sent false
690 1155 Acked Read 6 Done Done Sent false
691 1156 Acked Read 5 Done Done Sent false
692 1157 Acked Write 6 Done Done Sent false
693 1158 Acked Read 9 Done Done Sent false
694 1159 Acked Read 7 Done Done Sent false
695 1160 Acked Read 6 Done Done Sent false
696 1161 Acked Write 3 Done Done Sent false
697 1162 Acked Write 5 Done Done Sent false
698 1163 Acked Write 2 Done Done Sent false
699 1164 Acked Write 7 Done Done Sent false
700 1165 Acked Read 5 Done Done Sent false
701 1166 Acked Write 7 Done Done Sent false
702 1167 Acked Write 1 Done Done Sent false
703 1168 Acked Read 4 Done Done Sent false
704 1169 Acked Read 8 Done Done Sent false
705 1170 Acked Read 2 Done Done Sent false
706 1171 Acked Write 5 Done Done Sent false
707 1172 Acked Write 7 Done Done Sent false
708 1173 Acked Read 9 Done Done Sent false
709 1174 Acked Read 7 Done Done Sent false
710 1175 Acked Write 2 Done Done Sent false
711 1176 Acked Write 10 Done Done Sent false
712 1177 Acked Read 3 Done Done Sent false
713 1178 Acked Read 8 Done Done Sent false
714 1179 Acked Write 7 Done Done Sent false
715 1180 Acked Write 8 Done Done Sent false
716 1181 Acked Write 1 Done Done Sent false
717 1182 Acked Write 3 Done Done Sent false
718 1183 Acked Read 7 Done Done Sent false
719 1184 Acked Write 5 Done Done Sent false
720 1185 Acked Read 6 Done Done Sent false
721 1186 Acked Read 5 Done Done Sent false
722 1187 Acked Read 9 Done Done Sent false
723 1188 Acked Read 9 Done Done Sent false
724 1189 Acked Write 9 Done Done Sent false
725 1190 Acked Read 4 Done Done Sent false
726 1191 Acked Write 6 Done Done Sent false
727 1192 Acked Write 2 Done Done Sent false
728 1193 Acked Read 8 Done Done Sent false
729 1194 Acked Flush 0 Done Done Sent false
730 1195 Acked Read 8 Done Done Sent false
731 1196 Acked Write 7 Done Done Sent false
732 1197 Acked Flush 0 Done Done Sent false
733 1198 Acked Write 6 Done Done Sent false
734 1199 Acked Read 4 Done Done Sent false
735 1200 Acked Write 7 Done Done Sent false
736 1201 Acked Read 8 Done Done Sent false
737 1202 Acked Read 10 Done Done Sent false
738 1203 Acked Write 4 Done Done Sent false
739 1204 Acked Write 7 Done Done Sent false
740 1205 Acked Write 7 Done Done Sent false
741 1206 Acked Read 10 Done Done Sent false
742 1207 Acked Read 2 Done Done Sent false
743 1208 Acked Read 1 Done Done Sent false
744 1209 Acked Write 7 Done Done Sent false
745 1210 Acked Write 10 Done Done Sent false
746 1211 Acked Read 3 Done Done Sent false
747 1212 Acked Read 8 Done Done Sent false
748 1213 Acked Read 2 Done Done Sent false
749 1214 Acked Read 5 Done Done Sent false
750 1215 Acked Read 10 Done Done Sent false
751 1216 Acked Read 7 Done Done Sent false
752 1217 Acked Read 2 Done Done Sent false
753 1218 Acked Read 3 Done Done Sent false
754 1219 Acked Read 3 Done Done Sent false
755 1220 Acked Read 10 Done Done Sent false
756 1221 Acked Write 2 Done Done Sent false
757 1222 Acked Read 10 Done Done Sent false
758 1223 Acked Write 1 Done Done Sent false
759 1224 Acked Flush 0 Done Done Sent false
760 1225 Acked Read 2 Done Done Sent false
761 1226 Acked Write 8 Done Done Sent false
762 1227 Acked Write 5 Done Done Sent false
763 1228 Acked Read 5 Done Done Sent false
764 1229 Acked Read 3 Done Done Sent false
765 1230 Acked Write 5 Done Done Sent false
766 1231 Acked Read 5 Done Done Sent false
767 1232 Acked Write 5 Done Done Sent false
768 1233 Acked Read 1 Done Done Sent false
769 1234 Acked Write 2 Done Done Sent false
770 1235 Acked Read 1 Done Done Sent false
771 1236 Acked Read 5 Done Done Sent false
772 1237 Acked Write 1 Done Done Sent false
773 1238 Acked Read 3 Done Done Sent false
774 1239 Acked Read 9 Done Done Sent false
775 1240 Acked Read 6 Done Done Sent false
776 1241 Acked Read 9 Done Done Sent false
777 1242 Acked Write 1 Done Done Sent false
778 1243 Acked Read 10 Done Done Sent false
779 1244 Acked Read 9 Done Done Sent false
780 1245 Acked Read 1 Done Done Sent false
781 1246 Acked Read 7 Done Done Sent false
782 1247 Acked Read 8 Done Done Sent false
783 1248 Acked Read 8 Done Done Sent false
784 1249 Acked Read 5 Done Done Sent false
785 1250 Acked Write 1 Done Done Sent false
786 1251 Acked Write 2 Done Done Sent false
787 1252 Acked Write 4 Done Done Sent false
788 1253 Acked Write 9 Done Done Sent false
789 1254 Acked Write 6 Done Done Sent false
790 1255 Acked Write 1 Done Done Sent false
791 1256 Acked Read 2 Done Done Sent false
792 1257 Acked Write 1 Done Done Sent false
793 1258 Acked Flush 0 Done Done Sent false
794 1259 Acked Read 6 Done Done Sent false
795 1260 Acked Write 5 Done Done Sent false
796 1261 Acked Read 2 Done Done Sent false
797 1262 Acked Read 4 Done Done Sent false
798 1263 Acked Write 3 Done Done Sent false
799 1264 Acked Write 6 Done Done Sent false
800 1265 Acked Read 2 Done Done Sent false
801 1266 Acked Read 6 Done Done Sent false
802 1267 Acked Read 3 Done Done Sent false
803 1268 Acked Write 9 Done Done Sent false
804 1269 Acked Write 8 Done Done Sent false
805 1270 Acked Read 10 Done Done Sent false
806 1271 Acked Flush 0 Done Done Sent false
807 1272 Acked Write 2 Done Done Sent false
808 1273 Acked Write 3 Done Done Sent false
809 1274 Acked Write 3 Done Done Sent false
810 1275 Acked Read 4 Done Done Sent false
811 1276 Acked Write 8 Done Done Sent false
812 1277 Acked Read 1 Done Done Sent false
813 1278 Acked Flush 0 Done Done Sent false
814 1279 Acked Flush 0 Done Done Sent false
815 1280 Acked Read 7 Done Done Sent false
816 1281 Acked Flush 0 Done Done Sent false
817 1282 Acked Read 8 Done Done Sent false
818 1283 Acked Read 8 Done Done Sent false
819 1284 Acked Write 8 Done Done Sent false
820 1285 Acked Read 3 Done Done Sent false
821 1286 Acked Read 7 Done Done Sent false
822 1287 Acked Read 5 Done Done Sent false
823 1288 Acked Write 9 Done Done Sent false
824 1289 Acked Read 3 Done Done Sent false
825 1290 Acked Read 5 Done Done Sent false
826 1291 Acked Write 4 Done Done Sent false
827 1292 Acked Write 2 Done Done Sent false
828 1293 Acked Read 10 Done Done Sent false
829 1294 Acked Read 1 Done Done Sent false
830 1295 Acked Write 9 Done Done Sent false
831 1296 Acked Read 5 Done Done Sent false
832 1297 Acked Read 7 Done Done Sent false
833 1298 Acked Write 10 Done Done Sent false
834 1299 Acked Write 6 Done Done Sent false
835 1300 Acked Write 6 Done Done Sent false
836 STATES DS:0 DS:1 DS:2 TOTAL
837 Sent 0 0 301 301
838 Done 301 301 0 602
839 Skipped 0 0 0 0
840 Error 0 0 0 0
841 Last Flush: 1281 1281 0
842 Downstairs last five completed:
843 Upstairs last five completed: 1300 1299 1298 1297 1296
844 CLIENT: Up:0 ds:301 act:2
845 [8830] got start action
846 [8830] I am going to restart
847 Starting downstairs at port 8830
848 Make output file at "/tmp/dsc/downstairs-8830.txt"
849 [8830][2] reports Starting
850 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1203)
851 [8830][2] reports Running
852 ----------------------------------------------------------------
853 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
854 States: Active Active Offline
855 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
856 1000 Acked Read 5 Done Done Sent false
857 1001 Acked Read 7 Done Done Sent false
858 1002 Acked Write 6 Done Done Sent false
859 1003 Acked Read 8 Done Done Sent false
860 1004 Acked Read 3 Done Done Sent false
861 1005 Acked Write 9 Done Done Sent false
862 1006 Acked Read 8 Done Done Sent false
863 1007 Acked Write 5 Done Done Sent false
864 1008 Acked Read 10 Done Done Sent false
865 1009 Acked Write 3 Done Done Sent false
866 1010 Acked Read 2 Done Done Sent false
867 1011 Acked Write 5 Done Done Sent false
868 1012 Acked Read 7 Done Done Sent false
869 1013 Acked Write 9 Done Done Sent false
870 1014 Acked Write 4 Done Done Sent false
871 1015 Acked Read 10 Done Done Sent false
872 1016 Acked Read 9 Done Done Sent false
873 1017 Acked Write 9 Done Done Sent false
874 1018 Acked Read 6 Done Done Sent false
875 1019 Acked Write 10 Done Done Sent false
876 1020 Acked Read 10 Done Done Sent false
877 1021 Acked Read 9 Done Done Sent false
878 1022 Acked Read 1 Done Done Sent false
879 1023 Acked Write 2 Done Done Sent false
880 1024 Acked Write 10 Done Done Sent false
881 1025 Acked Write 8 Done Done Sent false
882 1026 Acked Write 8 Done Done Sent false
883 1027 Acked Read 1 Done Done Sent false
884 1028 Acked Write 4 Done Done Sent false
885 1029 Acked Read 3 Done Done Sent false
886 1030 Acked Read 3 Done Done Sent false
887 1031 Acked Read 9 Done Done Sent false
888 1032 Acked Read 1 Done Done Sent false
889 1033 Acked Write 1 Done Done Sent false
890 1034 Acked Read 2 Done Done Sent false
891 1035 Acked Read 10 Done Done Sent false
892 1036 Acked Write 7 Done Done Sent false
893 1037 Acked Write 10 Done Done Sent false
894 1038 Acked Read 2 Done Done Sent false
895 1039 Acked Read 8 Done Done Sent false
896 1040 Acked Flush 0 Done Done Sent false
897 1041 Acked Write 6 Done Done Sent false
898 1042 Acked Read 7 Done Done Sent false
899 1043 Acked Write 9 Done Done Sent false
900 1044 Acked Write 6 Done Done Sent false
901 1045 Acked Write 9 Done Done Sent false
902 1046 Acked Write 5 Done Done Sent false
903 1047 Acked Write 8 Done Done Sent false
904 1048 Acked Read 10 Done Done Sent false
905 1049 Acked Read 6 Done Done Sent false
906 1050 Acked Read 1 Done Done Sent false
907 1051 Acked Flush 0 Done Done Sent false
908 1052 Acked Write 5 Done Done Sent false
909 1053 Acked Write 10 Done Done Sent false
910 1054 Acked Write 5 Done Done Sent false
911 1055 Acked Write 8 Done Done Sent false
912 1056 Acked Write 9 Done Done Sent false
913 1057 Acked Write 4 Done Done Sent false
914 1058 Acked Read 3 Done Done Sent false
915 1059 Acked Read 10 Done Done Sent false
916 1060 Acked Write 3 Done Done Sent false
917 1061 Acked Read 5 Done Done Sent false
918 1062 Acked Read 8 Done Done Sent false
919 1063 Acked Write 1 Done Done Sent false
920 1064 Acked Read 4 Done Done Sent false
921 1065 Acked Write 1 Done Done Sent false
922 1066 Acked Read 2 Done Done Sent false
923 1067 Acked Read 10 Done Done Sent false
924 1068 Acked Read 3 Done Done Sent false
925 1069 Acked Read 9 Done Done Sent false
926 1070 Acked Write 4 Done Done Sent false
927 1071 Acked Write 7 Done Done Sent false
928 1072 Acked Write 3 Done Done Sent false
929 1073 Acked Write 4 Done Done Sent false
930 1074 Acked Write 10 Done Done Sent false
931 1075 Acked Write 10 Done Done Sent false
932 1076 Acked Write 2 Done Done Sent false
933 1077 Acked Read 4 Done Done Sent false
934 1078 Acked Read 9 Done Done Sent false
935 1079 Acked Read 6 Done Done Sent false
936 1080 Acked Read 8 Done Done Sent false
937 1081 Acked Write 2 Done Done Sent false
938 1082 Acked Read 1 Done Done Sent false
939 1083 Acked Write 5 Done Done Sent false
940 1084 Acked Read 7 Done Done Sent false
941 1085 Acked Read 10 Done Done Sent false
942 1086 Acked Flush 0 Done Done Sent false
943 1087 Acked Write 4 Done Done Sent false
944 1088 Acked Read 9 Done Done Sent false
945 1089 Acked Write 3 Done Done Sent false
946 1090 Acked Write 5 Done Done Sent false
947 1091 Acked Flush 0 Done Done Sent false
948 1092 Acked Read 1 Done Done Sent false
949 1093 Acked Read 4 Done Done Sent false
950 1094 Acked Write 1 Done Done Sent false
951 1095 Acked Write 10 Done Done Sent false
952 1096 Acked Read 9 Done Done Sent false
953 1097 Acked Write 1 Done Done Sent false
954 1098 Acked Read 10 Done Done Sent false
955 1099 Acked Write 9 Done Done Sent false
956 1100 Acked Read 7 Done Done Sent false
957 1101 Acked Flush 0 Done Done Sent false
958 1102 Acked Write 1 Done Done Sent false
959 1103 Acked Read 2 Done Done Sent false
960 1104 Acked Read 10 Done Done Sent false
961 1105 Acked Flush 0 Done Done Sent false
962 1106 Acked Read 7 Done Done Sent false
963 1107 Acked Read 2 Done Done Sent false
964 1108 Acked Read 9 Done Done Sent false
965 1109 Acked Read 2 Done Done Sent false
966 1110 Acked Read 8 Done Done Sent false
967 1111 Acked Read 8 Done Done Sent false
968 1112 Acked Read 3 Done Done Sent false
969 1113 Acked Read 10 Done Done Sent false
970 1114 Acked Write 10 Done Done Sent false
971 1115 Acked Write 10 Done Done Sent false
972 1116 Acked Write 6 Done Done Sent false
973 1117 Acked Write 5 Done Done Sent false
974 1118 Acked Write 2 Done Done Sent false
975 1119 Acked Read 8 Done Done Sent false
976 1120 Acked Read 9 Done Done Sent false
977 1121 Acked Read 9 Done Done Sent false
978 1122 Acked Write 9 Done Done Sent false
979 1123 Acked Write 7 Done Done Sent false
980 1124 Acked Write 5 Done Done Sent false
981 1125 Acked Write 3 Done Done Sent false
982 1126 Acked Write 7 Done Done Sent false
983 1127 Acked Read 8 Done Done Sent false
984 1128 Acked Read 2 Done Done Sent false
985 1129 Acked Write 4 Done Done Sent false
986 1130 Acked Read 10 Done Done Sent false
987 1131 Acked Write 5 Done Done Sent false
988 1132 Acked Read 3 Done Done Sent false
989 1133 Acked Write 3 Done Done Sent false
990 1134 Acked Write 7 Done Done Sent false
991 1135 Acked Flush 0 Done Done Sent false
992 1136 Acked Read 7 Done Done Sent false
993 1137 Acked Write 3 Done Done Sent false
994 1138 Acked Write 2 Done Done Sent false
995 1139 Acked Write 1 Done Done Sent false
996 1140 Acked Read 3 Done Done Sent false
997 1141 Acked Read 8 Done Done Sent false
998 1142 Acked Write 2 Done Done Sent false
999 1143 Acked Write 8 Done Done Sent false
1000 1144 Acked Flush 0 Done Done Sent false
1001 1145 Acked Read 9 Done Done Sent false
1002 1146 Acked Read 6 Done Done Sent false
1003 1147 Acked Write 3 Done Done Sent false
1004 1148 Acked Write 2 Done Done Sent false
1005 1149 Acked Write 4 Done Done Sent false
1006 1150 Acked Write 2 Done Done Sent false
1007 1151 Acked Write 2 Done Done Sent false
1008 1152 Acked Read 8 Done Done Sent false
1009 1153 Acked Write 2 Done Done Sent false
1010 1154 Acked Read 9 Done Done Sent false
1011 1155 Acked Read 6 Done Done Sent false
1012 1156 Acked Read 5 Done Done Sent false
1013 1157 Acked Write 6 Done Done Sent false
1014 1158 Acked Read 9 Done Done Sent false
1015 1159 Acked Read 7 Done Done Sent false
1016 1160 Acked Read 6 Done Done Sent false
1017 1161 Acked Write 3 Done Done Sent false
1018 1162 Acked Write 5 Done Done Sent false
1019 1163 Acked Write 2 Done Done Sent false
1020 1164 Acked Write 7 Done Done Sent false
1021 1165 Acked Read 5 Done Done Sent false
1022 1166 Acked Write 7 Done Done Sent false
1023 1167 Acked Write 1 Done Done Sent false
1024 1168 Acked Read 4 Done Done Sent false
1025 1169 Acked Read 8 Done Done Sent false
1026 1170 Acked Read 2 Done Done Sent false
1027 1171 Acked Write 5 Done Done Sent false
1028 1172 Acked Write 7 Done Done Sent false
1029 1173 Acked Read 9 Done Done Sent false
1030 1174 Acked Read 7 Done Done Sent false
1031 1175 Acked Write 2 Done Done Sent false
1032 1176 Acked Write 10 Done Done Sent false
1033 1177 Acked Read 3 Done Done Sent false
1034 1178 Acked Read 8 Done Done Sent false
1035 1179 Acked Write 7 Done Done Sent false
1036 1180 Acked Write 8 Done Done Sent false
1037 1181 Acked Write 1 Done Done Sent false
1038 1182 Acked Write 3 Done Done Sent false
1039 1183 Acked Read 7 Done Done Sent false
1040 1184 Acked Write 5 Done Done Sent false
1041 1185 Acked Read 6 Done Done Sent false
1042 1186 Acked Read 5 Done Done Sent false
1043 1187 Acked Read 9 Done Done Sent false
1044 1188 Acked Read 9 Done Done Sent false
1045 1189 Acked Write 9 Done Done Sent false
1046 1190 Acked Read 4 Done Done Sent false
1047 1191 Acked Write 6 Done Done Sent false
1048 1192 Acked Write 2 Done Done Sent false
1049 1193 Acked Read 8 Done Done Sent false
1050 1194 Acked Flush 0 Done Done Sent false
1051 1195 Acked Read 8 Done Done Sent false
1052 1196 Acked Write 7 Done Done Sent false
1053 1197 Acked Flush 0 Done Done Sent false
1054 1198 Acked Write 6 Done Done Sent false
1055 1199 Acked Read 4 Done Done Sent false
1056 1200 Acked Write 7 Done Done Sent false
1057 1201 Acked Read 8 Done Done Sent false
1058 1202 Acked Read 10 Done Done Sent false
1059 1203 Acked Write 4 Done Done Sent false
1060 1204 Acked Write 7 Done Done Sent false
1061 1205 Acked Write 7 Done Done Sent false
1062 1206 Acked Read 10 Done Done Sent false
1063 1207 Acked Read 2 Done Done Sent false
1064 1208 Acked Read 1 Done Done Sent false
1065 1209 Acked Write 7 Done Done Sent false
1066 1210 Acked Write 10 Done Done Sent false
1067 1211 Acked Read 3 Done Done Sent false
1068 1212 Acked Read 8 Done Done Sent false
1069 1213 Acked Read 2 Done Done Sent false
1070 1214 Acked Read 5 Done Done Sent false
1071 1215 Acked Read 10 Done Done Sent false
1072 1216 Acked Read 7 Done Done Sent false
1073 1217 Acked Read 2 Done Done Sent false
1074 1218 Acked Read 3 Done Done Sent false
1075 1219 Acked Read 3 Done Done Sent false
1076 1220 Acked Read 10 Done Done Sent false
1077 1221 Acked Write 2 Done Done Sent false
1078 1222 Acked Read 10 Done Done Sent false
1079 1223 Acked Write 1 Done Done Sent false
1080 1224 Acked Flush 0 Done Done Sent false
1081 1225 Acked Read 2 Done Done Sent false
1082 1226 Acked Write 8 Done Done Sent false
1083 1227 Acked Write 5 Done Done Sent false
1084 1228 Acked Read 5 Done Done Sent false
1085 1229 Acked Read 3 Done Done Sent false
1086 1230 Acked Write 5 Done Done Sent false
1087 1231 Acked Read 5 Done Done Sent false
1088 1232 Acked Write 5 Done Done Sent false
1089 1233 Acked Read 1 Done Done Sent false
1090 1234 Acked Write 2 Done Done Sent false
1091 1235 Acked Read 1 Done Done Sent false
1092 1236 Acked Read 5 Done Done Sent false
1093 1237 Acked Write 1 Done Done Sent false
1094 1238 Acked Read 3 Done Done Sent false
1095 1239 Acked Read 9 Done Done Sent false
1096 1240 Acked Read 6 Done Done Sent false
1097 1241 Acked Read 9 Done Done Sent false
1098 1242 Acked Write 1 Done Done Sent false
1099 1243 Acked Read 10 Done Done Sent false
1100 1244 Acked Read 9 Done Done Sent false
1101 1245 Acked Read 1 Done Done Sent false
1102 1246 Acked Read 7 Done Done Sent false
1103 1247 Acked Read 8 Done Done Sent false
1104 1248 Acked Read 8 Done Done Sent false
1105 1249 Acked Read 5 Done Done Sent false
1106 1250 Acked Write 1 Done Done Sent false
1107 1251 Acked Write 2 Done Done Sent false
1108 1252 Acked Write 4 Done Done Sent false
1109 1253 Acked Write 9 Done Done Sent false
1110 1254 Acked Write 6 Done Done Sent false
1111 1255 Acked Write 1 Done Done Sent false
1112 1256 Acked Read 2 Done Done Sent false
1113 1257 Acked Write 1 Done Done Sent false
1114 1258 Acked Flush 0 Done Done Sent false
1115 1259 Acked Read 6 Done Done Sent false
1116 1260 Acked Write 5 Done Done Sent false
1117 1261 Acked Read 2 Done Done Sent false
1118 1262 Acked Read 4 Done Done Sent false
1119 1263 Acked Write 3 Done Done Sent false
1120 1264 Acked Write 6 Done Done Sent false
1121 1265 Acked Read 2 Done Done Sent false
1122 1266 Acked Read 6 Done Done Sent false
1123 1267 Acked Read 3 Done Done Sent false
1124 1268 Acked Write 9 Done Done Sent false
1125 1269 Acked Write 8 Done Done Sent false
1126 1270 Acked Read 10 Done Done Sent false
1127 1271 Acked Flush 0 Done Done Sent false
1128 1272 Acked Write 2 Done Done Sent false
1129 1273 Acked Write 3 Done Done Sent false
1130 1274 Acked Write 3 Done Done Sent false
1131 1275 Acked Read 4 Done Done Sent false
1132 1276 Acked Write 8 Done Done Sent false
1133 1277 Acked Read 1 Done Done Sent false
1134 1278 Acked Flush 0 Done Done Sent false
1135 1279 Acked Flush 0 Done Done Sent false
1136 1280 Acked Read 7 Done Done Sent false
1137 1281 Acked Flush 0 Done Done Sent false
1138 1282 Acked Read 8 Done Done Sent false
1139 1283 Acked Read 8 Done Done Sent false
1140 1284 Acked Write 8 Done Done Sent false
1141 1285 Acked Read 3 Done Done Sent false
1142 1286 Acked Read 7 Done Done Sent false
1143 1287 Acked Read 5 Done Done Sent false
1144 1288 Acked Write 9 Done Done Sent false
1145 1289 Acked Read 3 Done Done Sent false
1146 1290 Acked Read 5 Done Done Sent false
1147 1291 Acked Write 4 Done Done Sent false
1148 1292 Acked Write 2 Done Done Sent false
1149 1293 Acked Read 10 Done Done Sent false
1150 1294 Acked Read 1 Done Done Sent false
1151 1295 Acked Write 9 Done Done Sent false
1152 1296 Acked Read 5 Done Done Sent false
1153 1297 Acked Read 7 Done Done Sent false
1154 1298 Acked Write 10 Done Done Sent false
1155 1299 Acked Write 6 Done Done Sent false
1156 1300 Acked Write 6 Done Done Sent false
1157 1301 Acked Flush 0 Done Done Sent false
1158 STATES DS:0 DS:1 DS:2 TOTAL
1159 Sent 0 0 302 302
1160 Done 302 302 0 604
1161 Skipped 0 0 0 0
1162 Error 0 0 0 0
1163 Last Flush: 1301 1301 0
1164 Downstairs last five completed:
1165 Upstairs last five completed: 1301 1300 1299 1298 1297
1166 CLIENT: Up:0 ds:302 act:2
11672025-01-15T05:19:13.008ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11682025-01-15T05:19:13.008ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54160) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11692025-01-15T05:19:13.009ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11702025-01-15T05:19:13.009ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11712025-01-15T05:19:13.009ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11722025-01-15T05:19:13.010ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11732025-01-15T05:19:13.010ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11742025-01-15T05:19:13.010ZINFOcrucible: [2] client re-new 302 jobs since flush 0 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11752025-01-15T05:19:13.010ZINFOcrucible: [2] Replayed 302 jobs since flush: 0 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1176 ----------------------------------------------------------------
1177 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1178 Downstairs last five completed: 1301 1300 1299 1298 1297
1179 Upstairs last five completed: 1301 1300 1299 1298 1297
1180 CLIENT: Up:0 ds:0 act:3
1181 Replay: All jobs finished
1182 Main task has work to do, go find it
1183 got dsc Stop(2)
1184 stop 2
1185 Jan 15 05:19:15.254 INFO request completed, latency_us: 329, response_code: 204, uri: /stop/cid/2, [8830] Got stop action so:false kr:false
1186 method: POST, req_id: 8b37ebc2-fd8b-477d-b2ec-1032f3d5cb2d, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
1187 Jan 15 05:19:15.254 INFO request completed, latency_us: 129, response_code: 200, uri: /state/cid/2, method: GET, req_id: de5e0ea4-a124-438a-8a74-7268603bd807, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
11882025-01-15T05:19:15.254ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11892025-01-15T05:19:15.255ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11902025-01-15T05:19:15.255ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11912025-01-15T05:19:15.255ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
1192 [8830] Exited with: ExitStatus(unix_wait_status(9))
1193 [8830][2] reports Exit
1194 Jan 15 05:19:19.257 INFO request completed, latency_us: 175, response_code: 200, uri: /state/cid/2, method: GET, req_id: 540f2448-56a2-4793-b93d-7c12b849dc90, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
1195 001/300 Read block 2125 len 16384
1196 002/300 Flush
1197 003/300 Flush
1198 004/300 Write block 4449 len 36864 data: 2 3 2 2 2 2 2 2 2
1199 005/300 Write block 1261 len 4096 data: 2
1200 006/300 Flush
1201 007/300 Write block 1385 len 8192 data: 2 2
1202 008/300 Write block 3381 len 36864 data: 2 4 3 3 3 3 3 3 2
1203 009/300 Flush
1204 010/300 Read block 226 len 4096
1205 011/300 Read block 500 len 24576
1206 012/300 Write block 434 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1207 013/300 Read block 4796 len 20480
1208 014/300 Write block 3669 len 32768 data: 2 2 2 2 2 2 2 2
1209 015/300 Write block 1179 len 8192 data: 2 2
1210 016/300 Read block 215 len 28672
1211 017/300 Write block 1013 len 16384 data: 3 2 2 2
1212 018/300 Flush
1213 019/300 Write block 2362 len 8192 data: 3 3
1214 020/300 Read block 1366 len 40960
1215 021/300 Read block 4550 len 12288
1216 022/300 Write block 2588 len 28672 data: 2 2 2 2 2 2 2
1217 023/300 Read block 110 len 40960
1218 024/300 Read block 4930 len 28672
1219 025/300 Write block 4909 len 36864 data: 4 3 3 3 3 3 3 3 3
1220 026/300 Write block 3601 len 32768 data: 2 2 2 2 2 2 2 2
1221 027/300 Write block 4114 len 28672 data: 2 2 2 2 2 2 2
1222 028/300 Write block 140 len 4096 data: 2
1223 029/300 Write block 1080 len 4096 data: 2
1224 030/300 Write block 894 len 4096 data: 3
1225 031/300 Write block 691 len 32768 data: 2 2 2 2 2 2 3 3
1226 032/300 Read block 2411 len 24576
1227 033/300 Write block 183 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1228 034/300 Read block 1113 len 40960
1229 035/300 Read block 20 len 28672
1230 036/300 Read block 1947 len 36864
1231 037/300 Write block 2455 len 32768 data: 2 2 2 2 3 4 4 4
1232 038/300 Flush
1233 039/300 Read block 4345 len 16384
1234 040/300 Read block 4142 len 36864
1235 041/300 Flush
1236 042/300 Read block 2214 len 12288
1237 043/300 Read block 4505 len 12288
1238 044/300 Write block 1214 len 20480 data: 2 2 2 2 2
1239 045/300 Write block 681 len 20480 data: 2 2 2 2 2
1240 046/300 Flush
1241 047/300 Read block 730 len 12288
1242 048/300 Write block 2899 len 32768 data: 2 2 2 2 2 2 2 2
1243 049/300 Write block 88 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1244 050/300 Read block 3604 len 4096
1245 051/300 Read block 169 len 32768
1246 052/300 Write block 518 len 32768 data: 2 2 2 2 2 2 2 2
1247 053/300 Flush
1248 054/300 Read block 626 len 20480
1249 055/300 Flush
1250 056/300 Read block 1328 len 40960
1251 057/300 Write block 2432 len 8192 data: 2 2
1252 058/300 Read block 2769 len 36864
1253 059/300 Flush
1254 060/300 Write block 3716 len 8192 data: 3 3
1255 061/300 Read block 3839 len 24576
1256 062/300 Write block 3438 len 24576 data: 2 2 2 2 2 2
1257 063/300 Read block 3524 len 28672
1258 064/300 Read block 4942 len 12288
1259 065/300 Write block 4165 len 8192 data: 2 2
1260 066/300 Write block 4501 len 12288 data: 3 2 2
1261 067/300 Flush
1262 068/300 Write block 1201 len 36864 data: 2 2 2 2 2 2 2 2 2
1263 069/300 Write block 2756 len 24576 data: 2 2 3 3 3 3
1264 070/300 Write block 395 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1265 071/300 Read block 462 len 16384
1266 072/300 Read block 2548 len 32768
1267 073/300 Write block 1218 len 20480 data: 3 2 2 2 2
1268 074/300 Flush
1269 075/300 Read block 4800 len 24576
1270 076/300 Read block 1487 len 32768
1271 077/300 Write block 1865 len 20480 data: 2 2 2 2 2
1272 078/300 Read block 1048 len 16384
1273 079/300 Write block 2566 len 16384 data: 2 2 2 2
1274 080/300 Write block 3887 len 8192 data: 2 2
1275 081/300 Read block 1862 len 16384
1276 082/300 Read block 2713 len 16384
1277 083/300 Write block 1997 len 36864 data: 2 2 2 2 2 2 2 2 2
1278 084/300 Write block 3411 len 32768 data: 2 2 2 2 3 3 3 3
1279 085/300 Read block 4125 len 32768
1280 086/300 Write block 2505 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1281 087/300 Read block 3409 len 28672
1282 088/300 Write block 2028 len 24576 data: 3 3 2 2 2 2
1283 089/300 Read block 1358 len 36864
1284 090/300 Read block 1738 len 4096
1285 091/300 Read block 2150 len 12288
1286 092/300 Read block 3581 len 32768
1287 093/300 Write block 1638 len 4096 data: 2
1288 094/300 Read block 4208 len 4096
1289 095/300 Read block 423 len 12288
1290 096/300 Read block 2272 len 16384
1291 097/300 Write block 1245 len 12288 data: 2 2 2
1292 098/300 Write block 3991 len 20480 data: 2 2 2 2 2
1293 099/300 Read block 3480 len 20480
1294 100/300 Read block 2184 len 40960
1295 101/300 Write block 1749 len 36864 data: 2 2 2 2 2 2 2 2 2
1296 102/300 Read block 1412 len 20480
1297 103/300 Write block 1326 len 16384 data: 2 2 2 2
1298 104/300 Write block 1585 len 12288 data: 2 2 2
1299 105/300 Write block 2223 len 4096 data: 2
1300 106/300 Read block 2209 len 8192
1301 107/300 Flush
1302 108/300 Read block 682 len 8192
1303 109/300 Read block 4180 len 12288
1304 110/300 Flush
1305 111/300 Flush
1306 112/300 Write block 1147 len 16384 data: 2 2 2 2
1307 113/300 Read block 3418 len 28672
1308 114/300 Read block 4528 len 4096
1309 115/300 Read block 1992 len 32768
1310 116/300 Read block 3964 len 36864
1311 117/300 Read block 3415 len 12288
1312 118/300 Read block 3337 len 20480
1313 119/300 Read block 354 len 28672
1314 120/300 Read block 2664 len 12288
1315 121/300 Write block 1153 len 8192 data: 2 2
1316 122/300 Read block 699 len 4096
1317 123/300 Read block 1602 len 24576
1318 124/300 Read block 4366 len 16384
1319 125/300 Read block 2067 len 20480
1320 126/300 Write block 1013 len 12288 data: 4 3 3
1321 127/300 Write block 3650 len 36864 data: 2 2 2 2 2 2 2 2 2
1322 128/300 Write block 1812 len 24576 data: 2 2 2 2 2 2
1323 129/300 Write block 3305 len 20480 data: 2 2 2 2 2
1324 130/300 Flush
1325 131/300 Read block 3522 len 20480
1326 132/300 Read block 1191 len 28672
1327 133/300 Flush
1328 134/300 Read block 1380 len 28672
1329 135/300 Write block 758 len 4096 data: 2
1330 136/300 Flush
1331 137/300 Read block 3191 len 16384
1332 138/300 Read block 1432 len 16384
1333 139/300 Write block 2071 len 24576 data: 2 2 2 2 2 2
1334 140/300 Write block 4896 len 36864 data: 2 2 2 2 3 3 3 3 3
1335 141/300 Write block 3479 len 20480 data: 2 2 2 2 2
1336 142/300 Read block 1263 len 24576
1337 143/300 Read block 377 len 40960
1338 144/300 Read block 3250 len 4096
1339 145/300 Read block 1734 len 20480
1340 146/300 Write block 599 len 16384 data: 2 2 2 2
1341 147/300 Write block 2023 len 36864 data: 3 3 3 3 3 4 4 3 3
1342 148/300 Read block 3519 len 4096
1343 149/300 Flush
1344 150/300 Read block 2667 len 16384
1345 151/300 Read block 3589 len 4096
1346 152/300 Read block 2471 len 20480
1347 153/300 Read block 1673 len 8192
1348 154/300 Flush
1349 155/300 Write block 3334 len 28672 data: 2 2 2 2 2 2 2
1350 156/300 Write block 678 len 32768 data: 3 2 2 3 3 3 3 3
1351 157/300 Flush
1352 158/300 Write block 4664 len 12288 data: 2 2 2
1353 159/300 Flush
1354 160/300 Read block 699 len 28672
1355 161/300 Write block 1603 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1356 162/300 Flush
1357 163/300 Read block 1352 len 36864
1358 164/300 Write block 1761 len 32768 data: 2 2 2 2 2 2 2 2
1359 165/300 Read block 3622 len 24576
1360 166/300 Write block 3466 len 36864 data: 2 2 2 2 2 2 2 2 2
1361 167/300 Read block 4778 len 20480
1362 168/300 Read block 1301 len 40960
1363 169/300 Read block 3501 len 28672
1364 170/300 Write block 3296 len 36864 data: 2 2 2 3 3 3 3 3 2
1365 171/300 Write block 2531 len 12288 data: 2 2 2
1366 172/300 Read block 3118 len 4096
1367 173/300 Write block 1078 len 32768 data: 2 2 3 2 2 2 2 2
1368 174/300 Read block 992 len 40960
1369 175/300 Read block 3833 len 4096
1370 176/300 Write block 3306 len 20480 data: 3 3 3 3 3
1371 177/300 Flush
1372 178/300 Read block 3954 len 24576
1373 179/300 Read block 4090 len 40960
1374 180/300 Write block 1893 len 20480 data: 2 2 2 2 2
1375 181/300 Read block 3197 len 4096
1376 182/300 Write block 3048 len 4096 data: 2
1377 183/300 Write block 2562 len 24576 data: 2 2 2 2 3 3
1378 184/300 Flush
1379 185/300 Read block 705 len 4096
1380 186/300 Read block 918 len 36864
1381 187/300 Write block 3937 len 32768 data: 2 3 3 3 3 3 5 5
1382 188/300 Flush
1383 189/300 Flush
1384 190/300 Read block 1944 len 32768
1385 191/300 Read block 60 len 32768
1386 192/300 Write block 1125 len 32768 data: 3 2 2 2 2 2 2 2
1387 193/300 Read block 4608 len 40960
1388 194/300 Write block 4520 len 4096 data: 2
1389 195/300 Write block 2348 len 12288 data: 2 2 2
1390 196/300 Read block 2669 len 16384
1391 197/300 Write block 295 len 40960 data: 3 3 2 2 2 2 2 2 2 2
1392 198/300 Read block 4617 len 20480
1393 199/300 Write block 2086 len 4096 data: 2
1394 200/300 Write block 1495 len 8192 data: 2 2
1395 201/300 Write block 2952 len 24576 data: 2 2 2 2 2 2
1396 202/300 Read block 2977 len 20480
1397 203/300 Flush
1398 204/300 Flush
1399 205/300 Read block 4294 len 36864
1400 206/300 Read block 2540 len 32768
1401 207/300 Read block 4198 len 32768
1402 208/300 Read block 4176 len 4096
1403 209/300 Read block 2018 len 36864
1404 210/300 Write block 4423 len 28672 data: 2 2 2 2 2 2 2
1405 211/300 Write block 2347 len 8192 data: 2 3
1406 212/300 Write block 95 len 4096 data: 3
1407 213/300 Flush
1408 214/300 Read block 2118 len 12288
1409 215/300 Flush
1410 216/300 Read block 2701 len 24576
1411 217/300 Write block 1861 len 24576 data: 2 2 2 2 3 3
1412 218/300 Read block 3072 len 32768
1413 219/300 Write block 2254 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1414 220/300 Flush
1415 221/300 Read block 4258 len 4096
1416 222/300 Write block 1842 len 16384 data: 2 2 2 2
1417 223/300 Write block 4872 len 12288 data: 2 2 2
1418 224/300 Read block 3378 len 24576
1419 225/300 Read block 47 len 16384
1420 226/300 Read block 814 len 24576
1421 227/300 Read block 4721 len 40960
1422 228/300 Read block 2316 len 40960
1423 229/300 Read block 2490 len 4096
1424 230/300 Read block 2592 len 20480
1425 231/300 Write block 253 len 24576 data: 2 2 2 2 2 2
1426 232/300 Write block 1629 len 28672 data: 2 2 2 2 2 2 2
1427 233/300 Write block 4504 len 28672 data: 2 2 2 2 2 2 2
1428 234/300 Write block 3124 len 4096 data: 3
1429 235/300 Write block 2446 len 36864 data: 3 3 3 3 3 2 2 2 2
1430 236/300 Write block 3523 len 4096 data: 2
1431 237/300 Read block 4778 len 40960
1432 238/300 Read block 735 len 8192
1433 239/300 Write block 1573 len 28672 data: 2 2 2 2 2 2 2
1434 240/300 Read block 2224 len 20480
1435 241/300 Flush
1436 242/300 Read block 4482 len 16384
1437 243/300 Write block 4817 len 4096 data: 2
1438 244/300 Write block 3816 len 4096 data: 2
1439 245/300 Read block 3696 len 16384
1440 246/300 Write block 3349 len 20480 data: 2 2 2 2 2
1441 247/300 Read block 1570 len 4096
1442 248/300 Read block 4182 len 40960
1443 249/300 Write block 3758 len 16384 data: 2 2 2 2
1444 250/300 Read block 3416 len 40960
1445 251/300 Read block 2992 len 4096
1446 252/300 Flush
1447 253/300 Write block 3019 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1448 254/300 Read block 4699 len 24576
1449 255/300 Write block 3953 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1450 256/300 Flush
1451 257/300 Read block 3014 len 12288
1452 258/300 Write block 3844 len 8192 data: 3 2
1453 259/300 Read block 4 len 20480
1454 260/300 Read block 3284 len 20480
1455 261/300 Write block 3067 len 4096 data: 2
1456 262/300 Read block 1990 len 40960
1457 263/300 Write block 1914 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1458 264/300 Read block 2260 len 40960
1459 265/300 Write block 351 len 36864 data: 2 2 2 2 2 2 3 3 3
1460 266/300 Read block 726 len 4096
1461 267/300 Read block 3435 len 4096
1462 268/300 Write block 3773 len 16384 data: 2 2 2 2
1463 269/300 Write block 2199 len 8192 data: 2 2
1464 270/300 Read block 2509 len 16384
1465 271/300 Write block 2243 len 20480 data: 2 2 2 2 2
1466 272/300 Read block 2703 len 4096
1467 273/300 Write block 2649 len 12288 data: 3 3 3
1468 274/300 Flush
1469 275/300 Read block 191 len 12288
1470 276/300 Read block 2598 len 12288
1471 277/300 Read block 3504 len 12288
1472 278/300 Read block 1935 len 40960
1473 279/300 Write block 320 len 4096 data: 2
1474 280/300 Write block 1426 len 8192 data: 2 2
1475 281/300 Write block 42 len 8192 data: 3 3
1476 282/300 Read block 488 len 16384
1477 283/300 Read block 2463 len 12288
1478 284/300 Read block 213 len 32768
1479 285/300 Write block 3041 len 32768 data: 2 2 2 2 2 2 2 3
1480 286/300 Read block 2992 len 28672
1481 287/300 Write block 934 len 16384 data: 2 2 2 3
1482 288/300 Flush
1483 289/300 Read block 415 len 24576
1484 290/300 Flush
1485 291/300 Read block 4894 len 24576
1486 292/300 Read block 4002 len 20480
1487 293/300 Read block 76 len 8192
1488 294/300 Write block 4546 len 32768 data: 3 3 3 3 3 3 3 2
1489 295/300 Write block 4897 len 8192 data: 3 3
1490 296/300 Read block 2937 len 28672
1491 297/300 Write block 103 len 8192 data: 2 2
1492 298/300 Flush
1493 299/300 Read block 97 len 32768
1494 300/300 Read block 4936 len 40960
1495 301/300 Read block 4651 len 16384
1496 Main task has work to do, go find it
1497 got dsc Start(2)
1498 start 2
1499 Jan 15 05:19:19.484 INFO request completed, latency_us: 189, response_code: 204, uri: /start/cid/2, method: POST, req_id: 4fa21879-0447-4a69-b291-d7e1f82cc0f5, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
1500 Replay: started 2, returned:Ok(())
1501 ----------------------------------------------------------------
1502 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1503 States: Active Active Offline
1504 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1505 1302 Acked Read 4 Done Done Sent false
1506 1303 Acked Flush 0 Done Done Sent false
1507 1304 Acked Flush 0 Done Done Sent false
1508 1305 Acked Write 9 Done Done Sent false
1509 1306 Acked Write 1 Done Done Sent false
1510 1307 Acked Flush 0 Done Done Sent false
1511 1308 Acked Write 2 Done Done Sent false
1512 1309 Acked Write 9 Done Done Sent false
1513 1310 Acked Flush 0 Done Done Sent false
1514 1311 Acked Read 1 Done Done Sent false
1515 1312 Acked Read 6 Done Done Sent false
1516 1313 Acked Write 10 Done Done Sent false
1517 1314 Acked Read 5 Done Done Sent false
1518 1315 Acked Write 8 Done Done Sent false
1519 1316 Acked Write 2 Done Done Sent false
1520 1317 Acked Read 7 Done Done Sent false
1521 1318 Acked Write 4 Done Done Sent false
1522 1319 Acked Flush 0 Done Done Sent false
1523 1320 Acked Write 2 Done Done Sent false
1524 1321 Acked Read 10 Done Done Sent false
1525 1322 Acked Read 3 Done Done Sent false
1526 1323 Acked Write 7 Done Done Sent false
1527 1324 Acked Read 10 Done Done Sent false
1528 1325 Acked Read 7 Done Done Sent false
1529 1326 Acked Write 9 Done Done Sent false
1530 1327 Acked Write 8 Done Done Sent false
1531 1328 Acked Write 7 Done Done Sent false
1532 1329 Acked Write 1 Done Done Sent false
1533 1330 Acked Write 1 Done Done Sent false
1534 1331 Acked Write 1 Done Done Sent false
1535 1332 Acked Write 8 Done Done Sent false
1536 1333 Acked Read 6 Done Done Sent false
1537 1334 Acked Write 10 Done Done Sent false
1538 1335 Acked Read 10 Done Done Sent false
1539 1336 Acked Read 7 Done Done Sent false
1540 1337 Acked Read 9 Done Done Sent false
1541 1338 Acked Write 8 Done Done Sent false
1542 1339 Acked Flush 0 Done Done Sent false
1543 1340 Acked Read 4 Done Done Sent false
1544 1341 Acked Read 9 Done Done Sent false
1545 1342 Acked Flush 0 Done Done Sent false
1546 1343 Acked Read 3 Done Done Sent false
1547 1344 Acked Read 3 Done Done Sent false
1548 1345 Acked Write 5 Done Done Sent false
1549 1346 Acked Write 5 Done Done Sent false
1550 1347 Acked Flush 0 Done Done Sent false
1551 1348 Acked Read 3 Done Done Sent false
1552 1349 Acked Write 8 Done Done Sent false
1553 1350 Acked Write 10 Done Done Sent false
1554 1351 Acked Read 1 Done Done Sent false
1555 1352 Acked Read 8 Done Done Sent false
1556 1353 Acked Write 8 Done Done Sent false
1557 1354 Acked Flush 0 Done Done Sent false
1558 1355 Acked Read 5 Done Done Sent false
1559 1356 Acked Flush 0 Done Done Sent false
1560 1357 Acked Read 10 Done Done Sent false
1561 1358 Acked Write 2 Done Done Sent false
1562 1359 Acked Read 9 Done Done Sent false
1563 1360 Acked Flush 0 Done Done Sent false
1564 1361 Acked Write 2 Done Done Sent false
1565 1362 Acked Read 6 Done Done Sent false
1566 1363 Acked Write 6 Done Done Sent false
1567 1364 Acked Read 7 Done Done Sent false
1568 1365 Acked Read 3 Done Done Sent false
1569 1366 Acked Write 2 Done Done Sent false
1570 1367 Acked Write 3 Done Done Sent false
1571 1368 Acked Flush 0 Done Done Sent false
1572 1369 Acked Write 9 Done Done Sent false
1573 1370 Acked Write 6 Done Done Sent false
1574 1371 Acked Write 10 Done Done Sent false
1575 1372 Acked Read 4 Done Done Sent false
1576 1373 Acked Read 8 Done Done Sent false
1577 1374 Acked Write 5 Done Done Sent false
1578 1375 Acked Flush 0 Done Done Sent false
1579 1376 Acked Read 6 Done Done Sent false
1580 1377 Acked Read 8 Done Done Sent false
1581 1378 Acked Write 5 Done Done Sent false
1582 1379 Acked Read 4 Done Done Sent false
1583 1380 Acked Write 4 Done Done Sent false
1584 1381 Acked Write 2 Done Done Sent false
1585 1382 Acked Read 4 Done Done Sent false
1586 1383 Acked Read 4 Done Done Sent false
1587 1384 Acked Write 9 Done Done Sent false
1588 1385 Acked Write 8 Done Done Sent false
1589 1386 Acked Read 8 Done Done Sent false
1590 1387 Acked Write 10 Done Done Sent false
1591 1388 Acked Read 7 Done Done Sent false
1592 1389 Acked Write 6 Done Done Sent false
1593 1390 Acked Read 9 Done Done Sent false
1594 1391 Acked Read 1 Done Done Sent false
1595 1392 Acked Read 3 Done Done Sent false
1596 1393 Acked Read 8 Done Done Sent false
1597 1394 Acked Write 1 Done Done Sent false
1598 1395 Acked Read 1 Done Done Sent false
1599 1396 Acked Read 3 Done Done Sent false
1600 1397 Acked Read 4 Done Done Sent false
1601 1398 Acked Write 3 Done Done Sent false
1602 1399 Acked Write 5 Done Done Sent false
1603 1400 Acked Read 5 Done Done Sent false
1604 1401 Acked Read 10 Done Done Sent false
1605 1402 Acked Write 9 Done Done Sent false
1606 1403 Acked Read 5 Done Done Sent false
1607 1404 Acked Write 4 Done Done Sent false
1608 1405 Acked Write 3 Done Done Sent false
1609 1406 Acked Write 1 Done Done Sent false
1610 1407 Acked Read 2 Done Done Sent false
1611 1408 Acked Flush 0 Done Done Sent false
1612 1409 Acked Read 2 Done Done Sent false
1613 1410 Acked Read 3 Done Done Sent false
1614 1411 Acked Flush 0 Done Done Sent false
1615 1412 Acked Flush 0 Done Done Sent false
1616 1413 Acked Write 4 Done Done Sent false
1617 1414 Acked Read 7 Done Done Sent false
1618 1415 Acked Read 1 Done Done Sent false
1619 1416 Acked Read 8 Done Done Sent false
1620 1417 Acked Read 9 Done Done Sent false
1621 1418 Acked Read 3 Done Done Sent false
1622 1419 Acked Read 5 Done Done Sent false
1623 1420 Acked Read 7 Done Done Sent false
1624 1421 Acked Read 3 Done Done Sent false
1625 1422 Acked Write 2 Done Done Sent false
1626 1423 Acked Read 1 Done Done Sent false
1627 1424 Acked Read 6 Done Done Sent false
1628 1425 Acked Read 4 Done Done Sent false
1629 1426 Acked Read 5 Done Done Sent false
1630 1427 Acked Write 3 Done Done Sent false
1631 1428 Acked Write 9 Done Done Sent false
1632 1429 Acked Write 6 Done Done Sent false
1633 1430 Acked Write 5 Done Done Sent false
1634 1431 Acked Flush 0 Done Done Sent false
1635 1432 Acked Read 5 Done Done Sent false
1636 1433 Acked Read 7 Done Done Sent false
1637 1434 Acked Flush 0 Done Done Sent false
1638 1435 Acked Read 7 Done Done Sent false
1639 1436 Acked Write 1 Done Done Sent false
1640 1437 Acked Flush 0 Done Done Sent false
1641 1438 Acked Read 4 Done Done Sent false
1642 1439 Acked Read 4 Done Done Sent false
1643 1440 Acked Write 6 Done Done Sent false
1644 1441 Acked Write 9 Done Done Sent false
1645 1442 Acked Write 5 Done Done Sent false
1646 1443 Acked Read 6 Done Done Sent false
1647 1444 Acked Read 10 Done Done Sent false
1648 1445 Acked Read 1 Done Done Sent false
1649 1446 Acked Read 5 Done Done Sent false
1650 1447 Acked Write 4 Done Done Sent false
1651 1448 Acked Write 9 Done Done Sent false
1652 1449 Acked Read 1 Done Done Sent false
1653 1450 Acked Flush 0 Done Done Sent false
1654 1451 Acked Read 4 Done Done Sent false
1655 1452 Acked Read 1 Done Done Sent false
1656 1453 Acked Read 5 Done Done Sent false
1657 1454 Acked Read 2 Done Done Sent false
1658 1455 Acked Flush 0 Done Done Sent false
1659 1456 Acked Write 7 Done Done Sent false
1660 1457 Acked Write 8 Done Done Sent false
1661 1458 Acked Flush 0 Done Done Sent false
1662 1459 Acked Write 3 Done Done Sent false
1663 1460 Acked Flush 0 Done Done Sent false
1664 1461 Acked Read 7 Done Done Sent false
1665 1462 Acked Write 10 Done Done Sent false
1666 1463 Acked Flush 0 Done Done Sent false
1667 1464 Acked Read 9 Done Done Sent false
1668 1465 Acked Write 8 Done Done Sent false
1669 1466 Acked Read 6 Done Done Sent false
1670 1467 Acked Write 9 Done Done Sent false
1671 1468 Acked Read 5 Done Done Sent false
1672 1469 Acked Read 10 Done Done Sent false
1673 1470 Acked Read 7 Done Done Sent false
1674 1471 Acked Write 9 Done Done Sent false
1675 1472 Acked Write 3 Done Done Sent false
1676 1473 Acked Read 1 Done Done Sent false
1677 1474 Acked Write 8 Done Done Sent false
1678 1475 Acked Read 10 Done Done Sent false
1679 1476 Acked Read 1 Done Done Sent false
1680 1477 Acked Write 5 Done Done Sent false
1681 1478 Acked Flush 0 Done Done Sent false
1682 1479 Acked Read 6 Done Done Sent false
1683 1480 Acked Read 10 Done Done Sent false
1684 1481 Acked Write 5 Done Done Sent false
1685 1482 Acked Read 1 Done Done Sent false
1686 1483 Acked Write 1 Done Done Sent false
1687 1484 Acked Write 6 Done Done Sent false
1688 1485 Acked Flush 0 Done Done Sent false
1689 1486 Acked Read 1 Done Done Sent false
1690 1487 Acked Read 9 Done Done Sent false
1691 1488 Acked Write 8 Done Done Sent false
1692 1489 Acked Flush 0 Done Done Sent false
1693 1490 Acked Flush 0 Done Done Sent false
1694 1491 Acked Read 8 Done Done Sent false
1695 1492 Acked Read 8 Done Done Sent false
1696 1493 Acked Write 8 Done Done Sent false
1697 1494 Acked Read 10 Done Done Sent false
1698 1495 Acked Write 1 Done Done Sent false
1699 1496 Acked Write 3 Done Done Sent false
1700 1497 Acked Read 4 Done Done Sent false
1701 1498 Acked Write 10 Done Done Sent false
1702 1499 Acked Read 5 Done Done Sent false
1703 1500 Acked Write 1 Done Done Sent false
1704 1501 Acked Write 2 Done Done Sent false
1705 1502 Acked Write 6 Done Done Sent false
1706 1503 Acked Read 5 Done Done Sent false
1707 1504 Acked Flush 0 Done Done Sent false
1708 1505 Acked Flush 0 Done Done Sent false
1709 1506 Acked Read 9 Done Done Sent false
1710 1507 Acked Read 8 Done Done Sent false
1711 1508 Acked Read 8 Done Done Sent false
1712 1509 Acked Read 1 Done Done Sent false
1713 1510 Acked Read 9 Done Done Sent false
1714 1511 Acked Write 7 Done Done Sent false
1715 1512 Acked Write 2 Done Done Sent false
1716 1513 Acked Write 1 Done Done Sent false
1717 1514 Acked Flush 0 Done Done Sent false
1718 1515 Acked Read 3 Done Done Sent false
1719 1516 Acked Flush 0 Done Done Sent false
1720 1517 Acked Read 6 Done Done Sent false
1721 1518 Acked Write 6 Done Done Sent false
1722 1519 Acked Read 8 Done Done Sent false
1723 1520 Acked Write 10 Done Done Sent false
1724 1521 Acked Flush 0 Done Done Sent false
1725 1522 Acked Read 1 Done Done Sent false
1726 1523 Acked Write 4 Done Done Sent false
1727 1524 Acked Write 3 Done Done Sent false
1728 1525 Acked Read 6 Done Done Sent false
1729 1526 Acked Read 4 Done Done Sent false
1730 1527 Acked Read 6 Done Done Sent false
1731 1528 Acked Read 10 Done Done Sent false
1732 1529 Acked Read 10 Done Done Sent false
1733 1530 Acked Read 1 Done Done Sent false
1734 1531 Acked Read 5 Done Done Sent false
1735 1532 Acked Write 6 Done Done Sent false
1736 1533 Acked Write 7 Done Done Sent false
1737 1534 Acked Write 7 Done Done Sent false
1738 1535 Acked Write 1 Done Done Sent false
1739 1536 Acked Write 9 Done Done Sent false
1740 1537 Acked Write 1 Done Done Sent false
1741 1538 Acked Read 10 Done Done Sent false
1742 1539 Acked Read 2 Done Done Sent false
1743 1540 Acked Write 7 Done Done Sent false
1744 1541 Acked Read 5 Done Done Sent false
1745 1542 Acked Flush 0 Done Done Sent false
1746 1543 Acked Read 4 Done Done Sent false
1747 1544 Acked Write 1 Done Done Sent false
1748 1545 Acked Write 1 Done Done Sent false
1749 1546 Acked Read 4 Done Done Sent false
1750 1547 Acked Write 5 Done Done Sent false
1751 1548 Acked Read 1 Done Done Sent false
1752 1549 Acked Read 10 Done Done Sent false
1753 1550 Acked Write 4 Done Done Sent false
1754 1551 Acked Read 10 Done Done Sent false
1755 1552 Acked Read 1 Done Done Sent false
1756 1553 Acked Flush 0 Done Done Sent false
1757 1554 Acked Write 10 Done Done Sent false
1758 1555 Acked Read 6 Done Done Sent false
1759 1556 Acked Write 10 Done Done Sent false
1760 1557 Acked Flush 0 Done Done Sent false
1761 1558 Acked Read 3 Done Done Sent false
1762 1559 Acked Write 2 Done Done Sent false
1763 1560 Acked Read 5 Done Done Sent false
1764 1561 Acked Read 5 Done Done Sent false
1765 1562 Acked Write 1 Done Done Sent false
1766 1563 Acked Read 10 Done Done Sent false
1767 1564 Acked Write 10 Done Done Sent false
1768 1565 Acked Read 10 Done Done Sent false
1769 1566 Acked Write 9 Done Done Sent false
1770 1567 Acked Read 1 Done Done Sent false
1771 1568 Acked Read 1 Done Done Sent false
1772 1569 Acked Write 4 Done Done Sent false
1773 1570 Acked Write 2 Done Done Sent false
1774 1571 Acked Read 4 Done Done Sent false
1775 1572 Acked Write 5 Done Done Sent false
1776 1573 Acked Read 1 Done Done Sent false
1777 1574 Acked Write 3 Done Done Sent false
1778 1575 Acked Flush 0 Done Done Sent false
1779 1576 Acked Read 3 Done Done Sent false
1780 1577 Acked Read 3 Done Done Sent false
1781 1578 Acked Read 3 Done Done Sent false
1782 1579 Acked Read 10 Done Done Sent false
1783 1580 Acked Write 1 Done Done Sent false
1784 1581 Acked Write 2 Done Done Sent false
1785 1582 Acked Write 2 Done Done Sent false
1786 1583 Acked Read 4 Done Done Sent false
1787 1584 Acked Read 3 Done Done Sent false
1788 1585 Acked Read 8 Done Done Sent false
1789 1586 Acked Write 8 Done Done Sent false
1790 1587 Acked Read 7 Done Done Sent false
1791 1588 Acked Write 4 Done Done Sent false
1792 1589 Acked Flush 0 Done Done Sent false
1793 1590 Acked Read 6 Done Done Sent false
1794 1591 Acked Flush 0 Done Done Sent false
1795 1592 Acked Read 6 Done Done Sent false
1796 1593 Acked Read 5 Done Done Sent false
1797 1594 Acked Read 2 Done Done Sent false
1798 1595 Acked Write 8 Done Done Sent false
1799 1596 Acked Write 2 Done Done Sent false
1800 1597 Acked Read 7 Done Done Sent false
1801 1598 Acked Write 2 Done Done Sent false
1802 1599 Acked Flush 0 Done Done Sent false
1803 1600 Acked Read 8 Done Done Sent false
1804 1601 Acked Read 10 Done Done Sent false
1805 1602 Acked Read 4 Done Done Sent false
1806 STATES DS:0 DS:1 DS:2 TOTAL
1807 Sent 0 0 301 301
1808 Done 301 301 0 602
1809 Skipped 0 0 0 0
1810 Error 0 0 0 0
1811 Last Flush: 1599 1599 1301
1812 Downstairs last five completed: 1301 1300 1299 1298 1297
1813 Upstairs last five completed: 1602 1601 1600 1599 1598
1814 CLIENT: Up:0 ds:301 act:2
1815 ----------------------------------------------------------------
1816 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1817 States: Active Active Offline
1818 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1819 1302 Acked Read 4 Done Done Sent false
1820 1303 Acked Flush 0 Done Done Sent false
1821 1304 Acked Flush 0 Done Done Sent false
1822 1305 Acked Write 9 Done Done Sent false
1823 1306 Acked Write 1 Done Done Sent false
1824 1307 Acked Flush 0 Done Done Sent false
1825 1308 Acked Write 2 Done Done Sent false
1826 1309 Acked Write 9 Done Done Sent false
1827 1310 Acked Flush 0 Done Done Sent false
1828 1311 Acked Read 1 Done Done Sent false
1829 1312 Acked Read 6 Done Done Sent false
1830 1313 Acked Write 10 Done Done Sent false
1831 1314 Acked Read 5 Done Done Sent false
1832 1315 Acked Write 8 Done Done Sent false
1833 1316 Acked Write 2 Done Done Sent false
1834 1317 Acked Read 7 Done Done Sent false
1835 1318 Acked Write 4 Done Done Sent false
1836 1319 Acked Flush 0 Done Done Sent false
1837 1320 Acked Write 2 Done Done Sent false
1838 1321 Acked Read 10 Done Done Sent false
1839 1322 Acked Read 3 Done Done Sent false
1840 1323 Acked Write 7 Done Done Sent false
1841 1324 Acked Read 10 Done Done Sent false
1842 1325 Acked Read 7 Done Done Sent false
1843 1326 Acked Write 9 Done Done Sent false
1844 1327 Acked Write 8 Done Done Sent false
1845 1328 Acked Write 7 Done Done Sent false
1846 1329 Acked Write 1 Done Done Sent false
1847 1330 Acked Write 1 Done Done Sent false
1848 1331 Acked Write 1 Done Done Sent false
1849 1332 Acked Write 8 Done Done Sent false
1850 1333 Acked Read 6 Done Done Sent false
1851 1334 Acked Write 10 Done Done Sent false
1852 1335 Acked Read 10 Done Done Sent false
1853 1336 Acked Read 7 Done Done Sent false
1854 1337 Acked Read 9 Done Done Sent false
1855 1338 Acked Write 8 Done Done Sent false
1856 1339 Acked Flush 0 Done Done Sent false
1857 1340 Acked Read 4 Done Done Sent false
1858 1341 Acked Read 9 Done Done Sent false
1859 1342 Acked Flush 0 Done Done Sent false
1860 1343 Acked Read 3 Done Done Sent false
1861 1344 Acked Read 3 Done Done Sent false
1862 1345 Acked Write 5 Done Done Sent false
1863 1346 Acked Write 5 Done Done Sent false
1864 1347 Acked Flush 0 Done Done Sent false
1865 1348 Acked Read 3 Done Done Sent false
1866 1349 Acked Write 8 Done Done Sent false
1867 1350 Acked Write 10 Done Done Sent false
1868 1351 Acked Read 1 Done Done Sent false
1869 1352 Acked Read 8 Done Done Sent false
1870 1353 Acked Write 8 Done Done Sent false
1871 1354 Acked Flush 0 Done Done Sent false
1872 1355 Acked Read 5 Done Done Sent false
1873 1356 Acked Flush 0 Done Done Sent false
1874 1357 Acked Read 10 Done Done Sent false
1875 1358 Acked Write 2 Done Done Sent false
1876 1359 Acked Read 9 Done Done Sent false
1877 1360 Acked Flush 0 Done Done Sent false
1878 1361 Acked Write 2 Done Done Sent false
1879 1362 Acked Read 6 Done Done Sent false
1880 1363 Acked Write 6 Done Done Sent false
1881 1364 Acked Read 7 Done Done Sent false
1882 1365 Acked Read 3 Done Done Sent false
1883 1366 Acked Write 2 Done Done Sent false
1884 1367 Acked Write 3 Done Done Sent false
1885 1368 Acked Flush 0 Done Done Sent false
1886 1369 Acked Write 9 Done Done Sent false
1887 1370 Acked Write 6 Done Done Sent false
1888 1371 Acked Write 10 Done Done Sent false
1889 1372 Acked Read 4 Done Done Sent false
1890 1373 Acked Read 8 Done Done Sent false
1891 1374 Acked Write 5 Done Done Sent false
1892 1375 Acked Flush 0 Done Done Sent false
1893 1376 Acked Read 6 Done Done Sent false
1894 1377 Acked Read 8 Done Done Sent false
1895 1378 Acked Write 5 Done Done Sent false
1896 1379 Acked Read 4 Done Done Sent false
1897 1380 Acked Write 4 Done Done Sent false
1898 1381 Acked Write 2 Done Done Sent false
1899 1382 Acked Read 4 Done Done Sent false
1900 1383 Acked Read 4 Done Done Sent false
1901 1384 Acked Write 9 Done Done Sent false
1902 1385 Acked Write 8 Done Done Sent false
1903 1386 Acked Read 8 Done Done Sent false
1904 1387 Acked Write 10 Done Done Sent false
1905 1388 Acked Read 7 Done Done Sent false
1906 1389 Acked Write 6 Done Done Sent false
1907 1390 Acked Read 9 Done Done Sent false
1908 1391 Acked Read 1 Done Done Sent false
1909 1392 Acked Read 3 Done Done Sent false
1910 1393 Acked Read 8 Done Done Sent false
1911 1394 Acked Write 1 Done Done Sent false
1912 1395 Acked Read 1 Done Done Sent false
1913 1396 Acked Read 3 Done Done Sent false
1914 1397 Acked Read 4 Done Done Sent false
1915 1398 Acked Write 3 Done Done Sent false
1916 1399 Acked Write 5 Done Done Sent false
1917 1400 Acked Read 5 Done Done Sent false
1918 1401 Acked Read 10 Done Done Sent false
1919 1402 Acked Write 9 Done Done Sent false
1920 1403 Acked Read 5 Done Done Sent false
1921 1404 Acked Write 4 Done Done Sent false
1922 1405 Acked Write 3 Done Done Sent false
1923 1406 Acked Write 1 Done Done Sent false
1924 1407 Acked Read 2 Done Done Sent false
1925 1408 Acked Flush 0 Done Done Sent false
1926 1409 Acked Read 2 Done Done Sent false
1927 1410 Acked Read 3 Done Done Sent false
1928 1411 Acked Flush 0 Done Done Sent false
1929 1412 Acked Flush 0 Done Done Sent false
1930 1413 Acked Write 4 Done Done Sent false
1931 1414 Acked Read 7 Done Done Sent false
1932 1415 Acked Read 1 Done Done Sent false
1933 1416 Acked Read 8 Done Done Sent false
1934 1417 Acked Read 9 Done Done Sent false
1935 1418 Acked Read 3 Done Done Sent false
1936 1419 Acked Read 5 Done Done Sent false
1937 1420 Acked Read 7 Done Done Sent false
1938 1421 Acked Read 3 Done Done Sent false
1939 1422 Acked Write 2 Done Done Sent false
1940 1423 Acked Read 1 Done Done Sent false
1941 1424 Acked Read 6 Done Done Sent false
1942 1425 Acked Read 4 Done Done Sent false
1943 1426 Acked Read 5 Done Done Sent false
1944 1427 Acked Write 3 Done Done Sent false
1945 1428 Acked Write 9 Done Done Sent false
1946 1429 Acked Write 6 Done Done Sent false
1947 1430 Acked Write 5 Done Done Sent false
1948 1431 Acked Flush 0 Done Done Sent false
1949 1432 Acked Read 5 Done Done Sent false
1950 1433 Acked Read 7 Done Done Sent false
1951 1434 Acked Flush 0 Done Done Sent false
1952 1435 Acked Read 7 Done Done Sent false
1953 1436 Acked Write 1 Done Done Sent false
1954 1437 Acked Flush 0 Done Done Sent false
1955 1438 Acked Read 4 Done Done Sent false
1956 1439 Acked Read 4 Done Done Sent false
1957 1440 Acked Write 6 Done Done Sent false
1958 1441 Acked Write 9 Done Done Sent false
1959 1442 Acked Write 5 Done Done Sent false
1960 1443 Acked Read 6 Done Done Sent false
1961 1444 Acked Read 10 Done Done Sent false
1962 1445 Acked Read 1 Done Done Sent false
1963 1446 Acked Read 5 Done Done Sent false
1964 1447 Acked Write 4 Done Done Sent false
1965 1448 Acked Write 9 Done Done Sent false
1966 1449 Acked Read 1 Done Done Sent false
1967 1450 Acked Flush 0 Done Done Sent false
1968 1451 Acked Read 4 Done Done Sent false
1969 1452 Acked Read 1 Done Done Sent false
1970 1453 Acked Read 5 Done Done Sent false
1971 1454 Acked Read 2 Done Done Sent false
1972 1455 Acked Flush 0 Done Done Sent false
1973 1456 Acked Write 7 Done Done Sent false
1974 1457 Acked Write 8 Done Done Sent false
1975 1458 Acked Flush 0 Done Done Sent false
1976 1459 Acked Write 3 Done Done Sent false
1977 1460 Acked Flush 0 Done Done Sent false
1978 1461 Acked Read 7 Done Done Sent false
1979 1462 Acked Write 10 Done Done Sent false
1980 1463 Acked Flush 0 Done Done Sent false
1981 1464 Acked Read 9 Done Done Sent false
1982 1465 Acked Write 8 Done Done Sent false
1983 1466 Acked Read 6 Done Done Sent false
1984 1467 Acked Write 9 Done Done Sent false
1985 1468 Acked Read 5 Done Done Sent false
1986 1469 Acked Read 10 Done Done Sent false
1987 1470 Acked Read 7 Done Done Sent false
1988 1471 Acked Write 9 Done Done Sent false
1989 1472 Acked Write 3 Done Done Sent false
1990 1473 Acked Read 1 Done Done Sent false
1991 1474 Acked Write 8 Done Done Sent false
1992 1475 Acked Read 10 Done Done Sent false
1993 1476 Acked Read 1 Done Done Sent false
1994 1477 Acked Write 5 Done Done Sent false
1995 1478 Acked Flush 0 Done Done Sent false
1996 1479 Acked Read 6 Done Done Sent false
1997 1480 Acked Read 10 Done Done Sent false
1998 1481 Acked Write 5 Done Done Sent false
1999 1482 Acked Read 1 Done Done Sent false
2000 1483 Acked Write 1 Done Done Sent false
2001 1484 Acked Write 6 Done Done Sent false
2002 1485 Acked Flush 0 Done Done Sent false
2003 1486 Acked Read 1 Done Done Sent false
2004 1487 Acked Read 9 Done Done Sent false
2005 1488 Acked Write 8 Done Done Sent false
2006 1489 Acked Flush 0 Done Done Sent false
2007 1490 Acked Flush 0 Done Done Sent false
2008 1491 Acked Read 8 Done Done Sent false
2009 1492 Acked Read 8 Done Done Sent false
2010 1493 Acked Write 8 Done Done Sent false
2011 1494 Acked Read 10 Done Done Sent false
2012 1495 Acked Write 1 Done Done Sent false
2013 1496 Acked Write 3 Done Done Sent false
2014 1497 Acked Read 4 Done Done Sent false
2015 1498 Acked Write 10 Done Done Sent false
2016 1499 Acked Read 5 Done Done Sent false
2017 1500 Acked Write 1 Done Done Sent false
2018 1501 Acked Write 2 Done Done Sent false
2019 1502 Acked Write 6 Done Done Sent false
2020 1503 Acked Read 5 Done Done Sent false
2021 1504 Acked Flush 0 Done Done Sent false
2022 1505 Acked Flush 0 Done Done Sent false
2023 1506 Acked Read 9 Done Done Sent false
2024 1507 Acked Read 8 Done Done Sent false
2025 1508 Acked Read 8 Done Done Sent false
2026 1509 Acked Read 1 Done Done Sent false
2027 1510 Acked Read 9 Done Done Sent false
2028 1511 Acked Write 7 Done Done Sent false
2029 1512 Acked Write 2 Done Done Sent false
2030 1513 Acked Write 1 Done Done Sent false
2031 1514 Acked Flush 0 Done Done Sent false
2032 1515 Acked Read 3 Done Done Sent false
2033 1516 Acked Flush 0 Done Done Sent false
2034 1517 Acked Read 6 Done Done Sent false
2035 1518 Acked Write 6 Done Done Sent false
2036 1519 Acked Read 8 Done Done Sent false
2037 1520 Acked Write 10 Done Done Sent false
2038 1521 Acked Flush 0 Done Done Sent false
2039 1522 Acked Read 1 Done Done Sent false
2040 1523 Acked Write 4 Done Done Sent false
2041 1524 Acked Write 3 Done Done Sent false
2042 1525 Acked Read 6 Done Done Sent false
2043 1526 Acked Read 4 Done Done Sent false
2044 1527 Acked Read 6 Done Done Sent false
2045 1528 Acked Read 10 Done Done Sent false
2046 1529 Acked Read 10 Done Done Sent false
2047 1530 Acked Read 1 Done Done Sent false
2048 1531 Acked Read 5 Done Done Sent false
2049 1532 Acked Write 6 Done Done Sent false
2050 1533 Acked Write 7 Done Done Sent false
2051 1534 Acked Write 7 Done Done Sent false
2052 1535 Acked Write 1 Done Done Sent false
2053 1536 Acked Write 9 Done Done Sent false
2054 1537 Acked Write 1 Done Done Sent false
2055 1538 Acked Read 10 Done Done Sent false
2056 1539 Acked Read 2 Done Done Sent false
2057 1540 Acked Write 7 Done Done Sent false
2058 1541 Acked Read 5 Done Done Sent false
2059 1542 Acked Flush 0 Done Done Sent false
2060 1543 Acked Read 4 Done Done Sent false
2061 1544 Acked Write 1 Done Done Sent false
2062 1545 Acked Write 1 Done Done Sent false
2063 1546 Acked Read 4 Done Done Sent false
2064 1547 Acked Write 5 Done Done Sent false
2065 1548 Acked Read 1 Done Done Sent false
2066 1549 Acked Read 10 Done Done Sent false
2067 1550 Acked Write 4 Done Done Sent false
2068 1551 Acked Read 10 Done Done Sent false
2069 1552 Acked Read 1 Done Done Sent false
2070 1553 Acked Flush 0 Done Done Sent false
2071 1554 Acked Write 10 Done Done Sent false
2072 1555 Acked Read 6 Done Done Sent false
2073 1556 Acked Write 10 Done Done Sent false
2074 1557 Acked Flush 0 Done Done Sent false
2075 1558 Acked Read 3 Done Done Sent false
2076 1559 Acked Write 2 Done Done Sent false
2077 1560 Acked Read 5 Done Done Sent false
2078 1561 Acked Read 5 Done Done Sent false
2079 1562 Acked Write 1 Done Done Sent false
2080 1563 Acked Read 10 Done Done Sent false
2081 1564 Acked Write 10 Done Done Sent false
2082 1565 Acked Read 10 Done Done Sent false
2083 1566 Acked Write 9 Done Done Sent false
2084 1567 Acked Read 1 Done Done Sent false
2085 1568 Acked Read 1 Done Done Sent false
2086 1569 Acked Write 4 Done Done Sent false
2087 1570 Acked Write 2 Done Done Sent false
2088 1571 Acked Read 4 Done Done Sent false
2089 1572 Acked Write 5 Done Done Sent false
2090 1573 Acked Read 1 Done Done Sent false
2091 1574 Acked Write 3 Done Done Sent false
2092 1575 Acked Flush 0 Done Done Sent false
2093 1576 Acked Read 3 Done Done Sent false
2094 1577 Acked Read 3 Done Done Sent false
2095 1578 Acked Read 3 Done Done Sent false
2096 1579 Acked Read 10 Done Done Sent false
2097 1580 Acked Write 1 Done Done Sent false
2098 1581 Acked Write 2 Done Done Sent false
2099 1582 Acked Write 2 Done Done Sent false
2100 1583 Acked Read 4 Done Done Sent false
2101 1584 Acked Read 3 Done Done Sent false
2102 1585 Acked Read 8 Done Done Sent false
2103 1586 Acked Write 8 Done Done Sent false
2104 1587 Acked Read 7 Done Done Sent false
2105 1588 Acked Write 4 Done Done Sent false
2106 1589 Acked Flush 0 Done Done Sent false
2107 1590 Acked Read 6 Done Done Sent false
2108 1591 Acked Flush 0 Done Done Sent false
2109 1592 Acked Read 6 Done Done Sent false
2110 1593 Acked Read 5 Done Done Sent false
2111 1594 Acked Read 2 Done Done Sent false
2112 1595 Acked Write 8 Done Done Sent false
2113 1596 Acked Write 2 Done Done Sent false
2114 1597 Acked Read 7 Done Done Sent false
2115 1598 Acked Write 2 Done Done Sent false
2116 1599 Acked Flush 0 Done Done Sent false
2117 1600 Acked Read 8 Done Done Sent false
2118 1601 Acked Read 10 Done Done Sent false
2119 1602 Acked Read 4 Done Done Sent false
2120 1603 Acked Flush 0 Done Done Sent false
2121 STATES DS:0 DS:1 DS:2 TOTAL
2122 Sent 0 0 302 302
2123 Done 302 302 0 604
2124 Skipped 0 0 0 0
2125 Error 0 0 0 0
2126 Last Flush: 1603 1603 1301
2127 Downstairs last five completed: 1301 1300 1299 1298 1297
2128 Upstairs last five completed: 1603 1602 1601 1600 1599
2129 CLIENT: Up:0 ds:302 act:2
21302025-01-15T05:19:25.256ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
21312025-01-15T05:19:25.257ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
21322025-01-15T05:19:25.257ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
21332025-01-15T05:19:25.257ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
21342025-01-15T05:19:25.257ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2135 ----------------------------------------------------------------
2136 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2137 States: Active Active Offline
2138 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2139 1302 Acked Read 4 Done Done Sent false
2140 1303 Acked Flush 0 Done Done Sent false
2141 1304 Acked Flush 0 Done Done Sent false
2142 1305 Acked Write 9 Done Done Sent false
2143 1306 Acked Write 1 Done Done Sent false
2144 1307 Acked Flush 0 Done Done Sent false
2145 1308 Acked Write 2 Done Done Sent false
2146 1309 Acked Write 9 Done Done Sent false
2147 1310 Acked Flush 0 Done Done Sent false
2148 1311 Acked Read 1 Done Done Sent false
2149 1312 Acked Read 6 Done Done Sent false
2150 1313 Acked Write 10 Done Done Sent false
2151 1314 Acked Read 5 Done Done Sent false
2152 1315 Acked Write 8 Done Done Sent false
2153 1316 Acked Write 2 Done Done Sent false
2154 1317 Acked Read 7 Done Done Sent false
2155 1318 Acked Write 4 Done Done Sent false
2156 1319 Acked Flush 0 Done Done Sent false
2157 1320 Acked Write 2 Done Done Sent false
2158 1321 Acked Read 10 Done Done Sent false
2159 1322 Acked Read 3 Done Done Sent false
2160 1323 Acked Write 7 Done Done Sent false
2161 1324 Acked Read 10 Done Done Sent false
2162 1325 Acked Read 7 Done Done Sent false
2163 1326 Acked Write 9 Done Done Sent false
2164 1327 Acked Write 8 Done Done Sent false
2165 1328 Acked Write 7 Done Done Sent false
2166 1329 Acked Write 1 Done Done Sent false
2167 1330 Acked Write 1 Done Done Sent false
2168 1331 Acked Write 1 Done Done Sent false
2169 1332 Acked Write 8 Done Done Sent false
2170 1333 Acked Read 6 Done Done Sent false
2171 1334 Acked Write 10 Done Done Sent false
2172 1335 Acked Read 10 Done Done Sent false
2173 1336 Acked Read 7 Done Done Sent false
2174 1337 Acked Read 9 Done Done Sent false
2175 1338 Acked Write 8 Done Done Sent false
2176 1339 Acked Flush 0 Done Done Sent false
2177 1340 Acked Read 4 Done Done Sent false
2178 1341 Acked Read 9 Done Done Sent false
2179 1342 Acked Flush 0 Done Done Sent false
2180 1343 Acked Read 3 Done Done Sent false
2181 1344 Acked Read 3 Done Done Sent false
2182 1345 Acked Write 5 Done Done Sent false
2183 1346 Acked Write 5 Done Done Sent false
2184 1347 Acked Flush 0 Done Done Sent false
2185 1348 Acked Read 3 Done Done Sent false
2186 1349 Acked Write 8 Done Done Sent false
2187 1350 Acked Write 10 Done Done Sent false
2188 1351 Acked Read 1 Done Done Sent false
2189 1352 Acked Read 8 Done Done Sent false
2190 1353 Acked Write 8 Done Done Sent false
2191 1354 Acked Flush 0 Done Done Sent false
2192 1355 Acked Read 5 Done Done Sent false
2193 1356 Acked Flush 0 Done Done Sent false
2194 1357 Acked Read 10 Done Done Sent false
2195 1358 Acked Write 2 Done Done Sent false
2196 1359 Acked Read 9 Done Done Sent false
2197 1360 Acked Flush 0 Done Done Sent false
2198 1361 Acked Write 2 Done Done Sent false
2199 1362 Acked Read 6 Done Done Sent false
2200 1363 Acked Write 6 Done Done Sent false
2201 1364 Acked Read 7 Done Done Sent false
2202 1365 Acked Read 3 Done Done Sent false
2203 1366 Acked Write 2 Done Done Sent false
2204 1367 Acked Write 3 Done Done Sent false
2205 1368 Acked Flush 0 Done Done Sent false
2206 1369 Acked Write 9 Done Done Sent false
2207 1370 Acked Write 6 Done Done Sent false
2208 1371 Acked Write 10 Done Done Sent false
2209 1372 Acked Read 4 Done Done Sent false
2210 1373 Acked Read 8 Done Done Sent false
2211 1374 Acked Write 5 Done Done Sent false
2212 1375 Acked Flush 0 Done Done Sent false
2213 1376 Acked Read 6 Done Done Sent false
2214 1377 Acked Read 8 Done Done Sent false
2215 1378 Acked Write 5 Done Done Sent false
2216 1379 Acked Read 4 Done Done Sent false
2217 1380 Acked Write 4 Done Done Sent false
2218 1381 Acked Write 2 Done Done Sent false
2219 1382 Acked Read 4 Done Done Sent false
2220 1383 Acked Read 4 Done Done Sent false
2221 1384 Acked Write 9 Done Done Sent false
2222 1385 Acked Write 8 Done Done Sent false
2223 1386 Acked Read 8 Done Done Sent false
2224 1387 Acked Write 10 Done Done Sent false
2225 1388 Acked Read 7 Done Done Sent false
2226 1389 Acked Write 6 Done Done Sent false
2227 1390 Acked Read 9 Done Done Sent false
2228 1391 Acked Read 1 Done Done Sent false
2229 1392 Acked Read 3 Done Done Sent false
2230 1393 Acked Read 8 Done Done Sent false
2231 1394 Acked Write 1 Done Done Sent false
2232 1395 Acked Read 1 Done Done Sent false
2233 1396 Acked Read 3 Done Done Sent false
2234 1397 Acked Read 4 Done Done Sent false
2235 1398 Acked Write 3 Done Done Sent false
2236 1399 Acked Write 5 Done Done Sent false
2237 1400 Acked Read 5 Done Done Sent false
2238 1401 Acked Read 10 Done Done Sent false
2239 1402 Acked Write 9 Done Done Sent false
2240 1403 Acked Read 5 Done Done Sent false
2241 1404 Acked Write 4 Done Done Sent false
2242 1405 Acked Write 3 Done Done Sent false
2243 1406 Acked Write 1 Done Done Sent false
2244 1407 Acked Read 2 Done Done Sent false
2245 1408 Acked Flush 0 Done Done Sent false
2246 1409 Acked Read 2 Done Done Sent false
2247 1410 Acked Read 3 Done Done Sent false
2248 1411 Acked Flush 0 Done Done Sent false
2249 1412 Acked Flush 0 Done Done Sent false
2250 1413 Acked Write 4 Done Done Sent false
2251 1414 Acked Read 7 Done Done Sent false
2252 1415 Acked Read 1 Done Done Sent false
2253 1416 Acked Read 8 Done Done Sent false
2254 1417 Acked Read 9 Done Done Sent false
2255 1418 Acked Read 3 Done Done Sent false
2256 1419 Acked Read 5 Done Done Sent false
2257 1420 Acked Read 7 Done Done Sent false
2258 1421 Acked Read 3 Done Done Sent false
2259 1422 Acked Write 2 Done Done Sent false
2260 1423 Acked Read 1 Done Done Sent false
2261 1424 Acked Read 6 Done Done Sent false
2262 1425 Acked Read 4 Done Done Sent false
2263 1426 Acked Read 5 Done Done Sent false
2264 1427 Acked Write 3 Done Done Sent false
2265 1428 Acked Write 9 Done Done Sent false
2266 1429 Acked Write 6 Done Done Sent false
2267 1430 Acked Write 5 Done Done Sent false
2268 1431 Acked Flush 0 Done Done Sent false
2269 1432 Acked Read 5 Done Done Sent false
2270 1433 Acked Read 7 Done Done Sent false
2271 1434 Acked Flush 0 Done Done Sent false
2272 1435 Acked Read 7 Done Done Sent false
2273 1436 Acked Write 1 Done Done Sent false
2274 1437 Acked Flush 0 Done Done Sent false
2275 1438 Acked Read 4 Done Done Sent false
2276 1439 Acked Read 4 Done Done Sent false
2277 1440 Acked Write 6 Done Done Sent false
2278 1441 Acked Write 9 Done Done Sent false
2279 1442 Acked Write 5 Done Done Sent false
2280 1443 Acked Read 6 Done Done Sent false
2281 1444 Acked Read 10 Done Done Sent false
2282 1445 Acked Read 1 Done Done Sent false
2283 1446 Acked Read 5 Done Done Sent false
2284 1447 Acked Write 4 Done Done Sent false
2285 1448 Acked Write 9 Done Done Sent false
2286 1449 Acked Read 1 Done Done Sent false
2287 1450 Acked Flush 0 Done Done Sent false
2288 1451 Acked Read 4 Done Done Sent false
2289 1452 Acked Read 1 Done Done Sent false
2290 1453 Acked Read 5 Done Done Sent false
2291 1454 Acked Read 2 Done Done Sent false
2292 1455 Acked Flush 0 Done Done Sent false
2293 1456 Acked Write 7 Done Done Sent false
2294 1457 Acked Write 8 Done Done Sent false
2295 1458 Acked Flush 0 Done Done Sent false
2296 1459 Acked Write 3 Done Done Sent false
2297 1460 Acked Flush 0 Done Done Sent false
2298 1461 Acked Read 7 Done Done Sent false
2299 1462 Acked Write 10 Done Done Sent false
2300 1463 Acked Flush 0 Done Done Sent false
2301 1464 Acked Read 9 Done Done Sent false
2302 1465 Acked Write 8 Done Done Sent false
2303 1466 Acked Read 6 Done Done Sent false
2304 1467 Acked Write 9 Done Done Sent false
2305 1468 Acked Read 5 Done Done Sent false
2306 1469 Acked Read 10 Done Done Sent false
2307 1470 Acked Read 7 Done Done Sent false
2308 1471 Acked Write 9 Done Done Sent false
2309 1472 Acked Write 3 Done Done Sent false
2310 1473 Acked Read 1 Done Done Sent false
2311 1474 Acked Write 8 Done Done Sent false
2312 1475 Acked Read 10 Done Done Sent false
2313 1476 Acked Read 1 Done Done Sent false
2314 1477 Acked Write 5 Done Done Sent false
2315 1478 Acked Flush 0 Done Done Sent false
2316 1479 Acked Read 6 Done Done Sent false
2317 1480 Acked Read 10 Done Done Sent false
2318 1481 Acked Write 5 Done Done Sent false
2319 1482 Acked Read 1 Done Done Sent false
2320 1483 Acked Write 1 Done Done Sent false
2321 1484 Acked Write 6 Done Done Sent false
2322 1485 Acked Flush 0 Done Done Sent false
2323 1486 Acked Read 1 Done Done Sent false
2324 1487 Acked Read 9 Done Done Sent false
2325 1488 Acked Write 8 Done Done Sent false
2326 1489 Acked Flush 0 Done Done Sent false
2327 1490 Acked Flush 0 Done Done Sent false
2328 1491 Acked Read 8 Done Done Sent false
2329 1492 Acked Read 8 Done Done Sent false
2330 1493 Acked Write 8 Done Done Sent false
2331 1494 Acked Read 10 Done Done Sent false
2332 1495 Acked Write 1 Done Done Sent false
2333 1496 Acked Write 3 Done Done Sent false
2334 1497 Acked Read 4 Done Done Sent false
2335 1498 Acked Write 10 Done Done Sent false
2336 1499 Acked Read 5 Done Done Sent false
2337 1500 Acked Write 1 Done Done Sent false
2338 1501 Acked Write 2 Done Done Sent false
2339 1502 Acked Write 6 Done Done Sent false
2340 1503 Acked Read 5 Done Done Sent false
2341 1504 Acked Flush 0 Done Done Sent false
2342 1505 Acked Flush 0 Done Done Sent false
2343 1506 Acked Read 9 Done Done Sent false
2344 1507 Acked Read 8 Done Done Sent false
2345 1508 Acked Read 8 Done Done Sent false
2346 1509 Acked Read 1 Done Done Sent false
2347 1510 Acked Read 9 Done Done Sent false
2348 1511 Acked Write 7 Done Done Sent false
2349 1512 Acked Write 2 Done Done Sent false
2350 1513 Acked Write 1 Done Done Sent false
2351 1514 Acked Flush 0 Done Done Sent false
2352 1515 Acked Read 3 Done Done Sent false
2353 1516 Acked Flush 0 Done Done Sent false
2354 1517 Acked Read 6 Done Done Sent false
2355 1518 Acked Write 6 Done Done Sent false
2356 1519 Acked Read 8 Done Done Sent false
2357 1520 Acked Write 10 Done Done Sent false
2358 1521 Acked Flush 0 Done Done Sent false
2359 1522 Acked Read 1 Done Done Sent false
2360 1523 Acked Write 4 Done Done Sent false
2361 1524 Acked Write 3 Done Done Sent false
2362 1525 Acked Read 6 Done Done Sent false
2363 1526 Acked Read 4 Done Done Sent false
2364 1527 Acked Read 6 Done Done Sent false
2365 1528 Acked Read 10 Done Done Sent false
2366 1529 Acked Read 10 Done Done Sent false
2367 1530 Acked Read 1 Done Done Sent false
2368 1531 Acked Read 5 Done Done Sent false
2369 1532 Acked Write 6 Done Done Sent false
2370 1533 Acked Write 7 Done Done Sent false
2371 1534 Acked Write 7 Done Done Sent false
2372 1535 Acked Write 1 Done Done Sent false
2373 1536 Acked Write 9 Done Done Sent false
2374 1537 Acked Write 1 Done Done Sent false
2375 1538 Acked Read 10 Done Done Sent false
2376 1539 Acked Read 2 Done Done Sent false
2377 1540 Acked Write 7 Done Done Sent false
2378 1541 Acked Read 5 Done Done Sent false
2379 1542 Acked Flush 0 Done Done Sent false
2380 1543 Acked Read 4 Done Done Sent false
2381 1544 Acked Write 1 Done Done Sent false
2382 1545 Acked Write 1 Done Done Sent false
2383 1546 Acked Read 4 Done Done Sent false
2384 1547 Acked Write 5 Done Done Sent false
2385 1548 Acked Read 1 Done Done Sent false
2386 1549 Acked Read 10 Done Done Sent false
2387 1550 Acked Write 4 Done Done Sent false
2388 1551 Acked Read 10 Done Done Sent false
2389 1552 Acked Read 1 Done Done Sent false
2390 1553 Acked Flush 0 Done Done Sent false
2391 1554 Acked Write 10 Done Done Sent false
2392 1555 Acked Read 6 Done Done Sent false
2393 1556 Acked Write 10 Done Done Sent false
2394 1557 Acked Flush 0 Done Done Sent false
2395 1558 Acked Read 3 Done Done Sent false
2396 1559 Acked Write 2 Done Done Sent false
2397 1560 Acked Read 5 Done Done Sent false
2398 1561 Acked Read 5 Done Done Sent false
2399 1562 Acked Write 1 Done Done Sent false
2400 1563 Acked Read 10 Done Done Sent false
2401 1564 Acked Write 10 Done Done Sent false
2402 1565 Acked Read 10 Done Done Sent false
2403 1566 Acked Write 9 Done Done Sent false
2404 1567 Acked Read 1 Done Done Sent false
2405 1568 Acked Read 1 Done Done Sent false
2406 1569 Acked Write 4 Done Done Sent false
2407 1570 Acked Write 2 Done Done Sent false
2408 1571 Acked Read 4 Done Done Sent false
2409 1572 Acked Write 5 Done Done Sent false
2410 1573 Acked Read 1 Done Done Sent false
2411 1574 Acked Write 3 Done Done Sent false
2412 1575 Acked Flush 0 Done Done Sent false
2413 1576 Acked Read 3 Done Done Sent false
2414 1577 Acked Read 3 Done Done Sent false
2415 1578 Acked Read 3 Done Done Sent false
2416 1579 Acked Read 10 Done Done Sent false
2417 1580 Acked Write 1 Done Done Sent false
2418 1581 Acked Write 2 Done Done Sent false
2419 1582 Acked Write 2 Done Done Sent false
2420 1583 Acked Read 4 Done Done Sent false
2421 1584 Acked Read 3 Done Done Sent false
2422 1585 Acked Read 8 Done Done Sent false
2423 1586 Acked Write 8 Done Done Sent false
2424 1587 Acked Read 7 Done Done Sent false
2425 1588 Acked Write 4 Done Done Sent false
2426 1589 Acked Flush 0 Done Done Sent false
2427 1590 Acked Read 6 Done Done Sent false
2428 1591 Acked Flush 0 Done Done Sent false
2429 1592 Acked Read 6 Done Done Sent false
2430 1593 Acked Read 5 Done Done Sent false
2431 1594 Acked Read 2 Done Done Sent false
2432 1595 Acked Write 8 Done Done Sent false
2433 1596 Acked Write 2 Done Done Sent false
2434 1597 Acked Read 7 Done Done Sent false
2435 1598 Acked Write 2 Done Done Sent false
2436 1599 Acked Flush 0 Done Done Sent false
2437 1600 Acked Read 8 Done Done Sent false
2438 1601 Acked Read 10 Done Done Sent false
2439 1602 Acked Read 4 Done Done Sent false
2440 1603 Acked Flush 0 Done Done Sent false
2441 STATES DS:0 DS:1 DS:2 TOTAL
2442 Sent 0 0 302 302
2443 Done 302 302 0 604
2444 Skipped 0 0 0 0
2445 Error 0 0 0 0
2446 Last Flush: 1603 1603 1301
2447 Downstairs last five completed: 1301 1300 1299 1298 1297
2448 Upstairs last five completed: 1603 1602 1601 1600 1599
2449 CLIENT: Up:0 ds:302 act:2
2450 [8830] got start action
2451 [8830] I am going to restart
2452 Starting downstairs at port 8830
2453 Make output file at "/tmp/dsc/downstairs-8830.txt"
2454 [8830][2] reports Starting
2455 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1204)
2456 [8830][2] reports Running
2457 ----------------------------------------------------------------
2458 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2459 States: Active Active Offline
2460 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2461 1302 Acked Read 4 Done Done Sent false
2462 1303 Acked Flush 0 Done Done Sent false
2463 1304 Acked Flush 0 Done Done Sent false
2464 1305 Acked Write 9 Done Done Sent false
2465 1306 Acked Write 1 Done Done Sent false
2466 1307 Acked Flush 0 Done Done Sent false
2467 1308 Acked Write 2 Done Done Sent false
2468 1309 Acked Write 9 Done Done Sent false
2469 1310 Acked Flush 0 Done Done Sent false
2470 1311 Acked Read 1 Done Done Sent false
2471 1312 Acked Read 6 Done Done Sent false
2472 1313 Acked Write 10 Done Done Sent false
2473 1314 Acked Read 5 Done Done Sent false
2474 1315 Acked Write 8 Done Done Sent false
2475 1316 Acked Write 2 Done Done Sent false
2476 1317 Acked Read 7 Done Done Sent false
2477 1318 Acked Write 4 Done Done Sent false
2478 1319 Acked Flush 0 Done Done Sent false
2479 1320 Acked Write 2 Done Done Sent false
2480 1321 Acked Read 10 Done Done Sent false
2481 1322 Acked Read 3 Done Done Sent false
2482 1323 Acked Write 7 Done Done Sent false
2483 1324 Acked Read 10 Done Done Sent false
2484 1325 Acked Read 7 Done Done Sent false
2485 1326 Acked Write 9 Done Done Sent false
2486 1327 Acked Write 8 Done Done Sent false
2487 1328 Acked Write 7 Done Done Sent false
2488 1329 Acked Write 1 Done Done Sent false
2489 1330 Acked Write 1 Done Done Sent false
2490 1331 Acked Write 1 Done Done Sent false
2491 1332 Acked Write 8 Done Done Sent false
2492 1333 Acked Read 6 Done Done Sent false
2493 1334 Acked Write 10 Done Done Sent false
2494 1335 Acked Read 10 Done Done Sent false
2495 1336 Acked Read 7 Done Done Sent false
2496 1337 Acked Read 9 Done Done Sent false
2497 1338 Acked Write 8 Done Done Sent false
2498 1339 Acked Flush 0 Done Done Sent false
2499 1340 Acked Read 4 Done Done Sent false
2500 1341 Acked Read 9 Done Done Sent false
2501 1342 Acked Flush 0 Done Done Sent false
2502 1343 Acked Read 3 Done Done Sent false
2503 1344 Acked Read 3 Done Done Sent false
2504 1345 Acked Write 5 Done Done Sent false
2505 1346 Acked Write 5 Done Done Sent false
2506 1347 Acked Flush 0 Done Done Sent false
2507 1348 Acked Read 3 Done Done Sent false
2508 1349 Acked Write 8 Done Done Sent false
2509 1350 Acked Write 10 Done Done Sent false
2510 1351 Acked Read 1 Done Done Sent false
2511 1352 Acked Read 8 Done Done Sent false
2512 1353 Acked Write 8 Done Done Sent false
2513 1354 Acked Flush 0 Done Done Sent false
2514 1355 Acked Read 5 Done Done Sent false
2515 1356 Acked Flush 0 Done Done Sent false
2516 1357 Acked Read 10 Done Done Sent false
2517 1358 Acked Write 2 Done Done Sent false
2518 1359 Acked Read 9 Done Done Sent false
2519 1360 Acked Flush 0 Done Done Sent false
2520 1361 Acked Write 2 Done Done Sent false
2521 1362 Acked Read 6 Done Done Sent false
2522 1363 Acked Write 6 Done Done Sent false
2523 1364 Acked Read 7 Done Done Sent false
2524 1365 Acked Read 3 Done Done Sent false
2525 1366 Acked Write 2 Done Done Sent false
2526 1367 Acked Write 3 Done Done Sent false
2527 1368 Acked Flush 0 Done Done Sent false
2528 1369 Acked Write 9 Done Done Sent false
2529 1370 Acked Write 6 Done Done Sent false
2530 1371 Acked Write 10 Done Done Sent false
2531 1372 Acked Read 4 Done Done Sent false
2532 1373 Acked Read 8 Done Done Sent false
2533 1374 Acked Write 5 Done Done Sent false
2534 1375 Acked Flush 0 Done Done Sent false
2535 1376 Acked Read 6 Done Done Sent false
2536 1377 Acked Read 8 Done Done Sent false
2537 1378 Acked Write 5 Done Done Sent false
2538 1379 Acked Read 4 Done Done Sent false
2539 1380 Acked Write 4 Done Done Sent false
2540 1381 Acked Write 2 Done Done Sent false
2541 1382 Acked Read 4 Done Done Sent false
2542 1383 Acked Read 4 Done Done Sent false
2543 1384 Acked Write 9 Done Done Sent false
2544 1385 Acked Write 8 Done Done Sent false
2545 1386 Acked Read 8 Done Done Sent false
2546 1387 Acked Write 10 Done Done Sent false
2547 1388 Acked Read 7 Done Done Sent false
2548 1389 Acked Write 6 Done Done Sent false
2549 1390 Acked Read 9 Done Done Sent false
2550 1391 Acked Read 1 Done Done Sent false
2551 1392 Acked Read 3 Done Done Sent false
2552 1393 Acked Read 8 Done Done Sent false
2553 1394 Acked Write 1 Done Done Sent false
2554 1395 Acked Read 1 Done Done Sent false
2555 1396 Acked Read 3 Done Done Sent false
2556 1397 Acked Read 4 Done Done Sent false
2557 1398 Acked Write 3 Done Done Sent false
2558 1399 Acked Write 5 Done Done Sent false
2559 1400 Acked Read 5 Done Done Sent false
2560 1401 Acked Read 10 Done Done Sent false
2561 1402 Acked Write 9 Done Done Sent false
2562 1403 Acked Read 5 Done Done Sent false
2563 1404 Acked Write 4 Done Done Sent false
2564 1405 Acked Write 3 Done Done Sent false
2565 1406 Acked Write 1 Done Done Sent false
2566 1407 Acked Read 2 Done Done Sent false
2567 1408 Acked Flush 0 Done Done Sent false
2568 1409 Acked Read 2 Done Done Sent false
2569 1410 Acked Read 3 Done Done Sent false
2570 1411 Acked Flush 0 Done Done Sent false
2571 1412 Acked Flush 0 Done Done Sent false
2572 1413 Acked Write 4 Done Done Sent false
2573 1414 Acked Read 7 Done Done Sent false
2574 1415 Acked Read 1 Done Done Sent false
2575 1416 Acked Read 8 Done Done Sent false
2576 1417 Acked Read 9 Done Done Sent false
2577 1418 Acked Read 3 Done Done Sent false
2578 1419 Acked Read 5 Done Done Sent false
2579 1420 Acked Read 7 Done Done Sent false
2580 1421 Acked Read 3 Done Done Sent false
2581 1422 Acked Write 2 Done Done Sent false
2582 1423 Acked Read 1 Done Done Sent false
2583 1424 Acked Read 6 Done Done Sent false
2584 1425 Acked Read 4 Done Done Sent false
2585 1426 Acked Read 5 Done Done Sent false
2586 1427 Acked Write 3 Done Done Sent false
2587 1428 Acked Write 9 Done Done Sent false
2588 1429 Acked Write 6 Done Done Sent false
2589 1430 Acked Write 5 Done Done Sent false
2590 1431 Acked Flush 0 Done Done Sent false
2591 1432 Acked Read 5 Done Done Sent false
2592 1433 Acked Read 7 Done Done Sent false
2593 1434 Acked Flush 0 Done Done Sent false
2594 1435 Acked Read 7 Done Done Sent false
2595 1436 Acked Write 1 Done Done Sent false
2596 1437 Acked Flush 0 Done Done Sent false
2597 1438 Acked Read 4 Done Done Sent false
2598 1439 Acked Read 4 Done Done Sent false
2599 1440 Acked Write 6 Done Done Sent false
2600 1441 Acked Write 9 Done Done Sent false
2601 1442 Acked Write 5 Done Done Sent false
2602 1443 Acked Read 6 Done Done Sent false
2603 1444 Acked Read 10 Done Done Sent false
2604 1445 Acked Read 1 Done Done Sent false
2605 1446 Acked Read 5 Done Done Sent false
2606 1447 Acked Write 4 Done Done Sent false
2607 1448 Acked Write 9 Done Done Sent false
2608 1449 Acked Read 1 Done Done Sent false
2609 1450 Acked Flush 0 Done Done Sent false
2610 1451 Acked Read 4 Done Done Sent false
2611 1452 Acked Read 1 Done Done Sent false
2612 1453 Acked Read 5 Done Done Sent false
2613 1454 Acked Read 2 Done Done Sent false
2614 1455 Acked Flush 0 Done Done Sent false
2615 1456 Acked Write 7 Done Done Sent false
2616 1457 Acked Write 8 Done Done Sent false
2617 1458 Acked Flush 0 Done Done Sent false
2618 1459 Acked Write 3 Done Done Sent false
2619 1460 Acked Flush 0 Done Done Sent false
2620 1461 Acked Read 7 Done Done Sent false
2621 1462 Acked Write 10 Done Done Sent false
2622 1463 Acked Flush 0 Done Done Sent false
2623 1464 Acked Read 9 Done Done Sent false
2624 1465 Acked Write 8 Done Done Sent false
2625 1466 Acked Read 6 Done Done Sent false
2626 1467 Acked Write 9 Done Done Sent false
2627 1468 Acked Read 5 Done Done Sent false
2628 1469 Acked Read 10 Done Done Sent false
2629 1470 Acked Read 7 Done Done Sent false
2630 1471 Acked Write 9 Done Done Sent false
2631 1472 Acked Write 3 Done Done Sent false
2632 1473 Acked Read 1 Done Done Sent false
2633 1474 Acked Write 8 Done Done Sent false
2634 1475 Acked Read 10 Done Done Sent false
2635 1476 Acked Read 1 Done Done Sent false
2636 1477 Acked Write 5 Done Done Sent false
2637 1478 Acked Flush 0 Done Done Sent false
2638 1479 Acked Read 6 Done Done Sent false
2639 1480 Acked Read 10 Done Done Sent false
2640 1481 Acked Write 5 Done Done Sent false
2641 1482 Acked Read 1 Done Done Sent false
2642 1483 Acked Write 1 Done Done Sent false
2643 1484 Acked Write 6 Done Done Sent false
2644 1485 Acked Flush 0 Done Done Sent false
2645 1486 Acked Read 1 Done Done Sent false
2646 1487 Acked Read 9 Done Done Sent false
2647 1488 Acked Write 8 Done Done Sent false
2648 1489 Acked Flush 0 Done Done Sent false
2649 1490 Acked Flush 0 Done Done Sent false
2650 1491 Acked Read 8 Done Done Sent false
2651 1492 Acked Read 8 Done Done Sent false
2652 1493 Acked Write 8 Done Done Sent false
2653 1494 Acked Read 10 Done Done Sent false
2654 1495 Acked Write 1 Done Done Sent false
2655 1496 Acked Write 3 Done Done Sent false
2656 1497 Acked Read 4 Done Done Sent false
2657 1498 Acked Write 10 Done Done Sent false
2658 1499 Acked Read 5 Done Done Sent false
2659 1500 Acked Write 1 Done Done Sent false
2660 1501 Acked Write 2 Done Done Sent false
2661 1502 Acked Write 6 Done Done Sent false
2662 1503 Acked Read 5 Done Done Sent false
2663 1504 Acked Flush 0 Done Done Sent false
2664 1505 Acked Flush 0 Done Done Sent false
2665 1506 Acked Read 9 Done Done Sent false
2666 1507 Acked Read 8 Done Done Sent false
2667 1508 Acked Read 8 Done Done Sent false
2668 1509 Acked Read 1 Done Done Sent false
2669 1510 Acked Read 9 Done Done Sent false
2670 1511 Acked Write 7 Done Done Sent false
2671 1512 Acked Write 2 Done Done Sent false
2672 1513 Acked Write 1 Done Done Sent false
2673 1514 Acked Flush 0 Done Done Sent false
2674 1515 Acked Read 3 Done Done Sent false
2675 1516 Acked Flush 0 Done Done Sent false
2676 1517 Acked Read 6 Done Done Sent false
2677 1518 Acked Write 6 Done Done Sent false
2678 1519 Acked Read 8 Done Done Sent false
2679 1520 Acked Write 10 Done Done Sent false
2680 1521 Acked Flush 0 Done Done Sent false
2681 1522 Acked Read 1 Done Done Sent false
2682 1523 Acked Write 4 Done Done Sent false
2683 1524 Acked Write 3 Done Done Sent false
2684 1525 Acked Read 6 Done Done Sent false
2685 1526 Acked Read 4 Done Done Sent false
2686 1527 Acked Read 6 Done Done Sent false
2687 1528 Acked Read 10 Done Done Sent false
2688 1529 Acked Read 10 Done Done Sent false
2689 1530 Acked Read 1 Done Done Sent false
2690 1531 Acked Read 5 Done Done Sent false
2691 1532 Acked Write 6 Done Done Sent false
2692 1533 Acked Write 7 Done Done Sent false
2693 1534 Acked Write 7 Done Done Sent false
2694 1535 Acked Write 1 Done Done Sent false
2695 1536 Acked Write 9 Done Done Sent false
2696 1537 Acked Write 1 Done Done Sent false
2697 1538 Acked Read 10 Done Done Sent false
2698 1539 Acked Read 2 Done Done Sent false
2699 1540 Acked Write 7 Done Done Sent false
2700 1541 Acked Read 5 Done Done Sent false
2701 1542 Acked Flush 0 Done Done Sent false
2702 1543 Acked Read 4 Done Done Sent false
2703 1544 Acked Write 1 Done Done Sent false
2704 1545 Acked Write 1 Done Done Sent false
2705 1546 Acked Read 4 Done Done Sent false
2706 1547 Acked Write 5 Done Done Sent false
2707 1548 Acked Read 1 Done Done Sent false
2708 1549 Acked Read 10 Done Done Sent false
2709 1550 Acked Write 4 Done Done Sent false
2710 1551 Acked Read 10 Done Done Sent false
2711 1552 Acked Read 1 Done Done Sent false
2712 1553 Acked Flush 0 Done Done Sent false
2713 1554 Acked Write 10 Done Done Sent false
2714 1555 Acked Read 6 Done Done Sent false
2715 1556 Acked Write 10 Done Done Sent false
2716 1557 Acked Flush 0 Done Done Sent false
2717 1558 Acked Read 3 Done Done Sent false
2718 1559 Acked Write 2 Done Done Sent false
2719 1560 Acked Read 5 Done Done Sent false
2720 1561 Acked Read 5 Done Done Sent false
2721 1562 Acked Write 1 Done Done Sent false
2722 1563 Acked Read 10 Done Done Sent false
2723 1564 Acked Write 10 Done Done Sent false
2724 1565 Acked Read 10 Done Done Sent false
2725 1566 Acked Write 9 Done Done Sent false
2726 1567 Acked Read 1 Done Done Sent false
2727 1568 Acked Read 1 Done Done Sent false
2728 1569 Acked Write 4 Done Done Sent false
2729 1570 Acked Write 2 Done Done Sent false
2730 1571 Acked Read 4 Done Done Sent false
2731 1572 Acked Write 5 Done Done Sent false
2732 1573 Acked Read 1 Done Done Sent false
2733 1574 Acked Write 3 Done Done Sent false
2734 1575 Acked Flush 0 Done Done Sent false
2735 1576 Acked Read 3 Done Done Sent false
2736 1577 Acked Read 3 Done Done Sent false
2737 1578 Acked Read 3 Done Done Sent false
2738 1579 Acked Read 10 Done Done Sent false
2739 1580 Acked Write 1 Done Done Sent false
2740 1581 Acked Write 2 Done Done Sent false
2741 1582 Acked Write 2 Done Done Sent false
2742 1583 Acked Read 4 Done Done Sent false
2743 1584 Acked Read 3 Done Done Sent false
2744 1585 Acked Read 8 Done Done Sent false
2745 1586 Acked Write 8 Done Done Sent false
2746 1587 Acked Read 7 Done Done Sent false
2747 1588 Acked Write 4 Done Done Sent false
2748 1589 Acked Flush 0 Done Done Sent false
2749 1590 Acked Read 6 Done Done Sent false
2750 1591 Acked Flush 0 Done Done Sent false
2751 1592 Acked Read 6 Done Done Sent false
2752 1593 Acked Read 5 Done Done Sent false
2753 1594 Acked Read 2 Done Done Sent false
2754 1595 Acked Write 8 Done Done Sent false
2755 1596 Acked Write 2 Done Done Sent false
2756 1597 Acked Read 7 Done Done Sent false
2757 1598 Acked Write 2 Done Done Sent false
2758 1599 Acked Flush 0 Done Done Sent false
2759 1600 Acked Read 8 Done Done Sent false
2760 1601 Acked Read 10 Done Done Sent false
2761 1602 Acked Read 4 Done Done Sent false
2762 1603 Acked Flush 0 Done Done Sent false
2763 STATES DS:0 DS:1 DS:2 TOTAL
2764 Sent 0 0 302 302
2765 Done 302 302 0 604
2766 Skipped 0 0 0 0
2767 Error 0 0 0 0
2768 Last Flush: 1603 1603 1301
2769 Downstairs last five completed: 1301 1300 1299 1298 1297
2770 Upstairs last five completed: 1603 1602 1601 1600 1599
2771 CLIENT: Up:0 ds:302 act:2
27722025-01-15T05:19:35.258ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27732025-01-15T05:19:35.258ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41542) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27742025-01-15T05:19:35.259ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27752025-01-15T05:19:35.260ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27762025-01-15T05:19:35.260ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27772025-01-15T05:19:35.260ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27782025-01-15T05:19:35.260ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27792025-01-15T05:19:35.260ZINFOcrucible: [2] client re-new 302 jobs since flush 1301 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27802025-01-15T05:19:35.260ZINFOcrucible: [2] Replayed 302 jobs since flush: 1301 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2781 ----------------------------------------------------------------
2782 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2783 Downstairs last five completed: 1603 1602 1601 1600 1599
2784 Upstairs last five completed: 1603 1602 1601 1600 1599
2785 CLIENT: Up:0 ds:0 act:3
2786 Replay: All jobs finished
2787 Main task has work to do, go find it
2788 got dsc Stop(2)
2789 stop 2
2790 Jan 15 05:19:35.493 INFO request completed, latency_us: 310, response_code: 204, uri: /stop/cid/2, method: POST, req_id: cd0ead96-1f86-42df-9f68-e0060b9dba47, remote_addr: 127.0.0[8830] Got stop action so:false kr:false
2791 .1:42402, local_addr: 127.0.0.1:9998
2792 Jan 15 05:19:35.494 INFO request completed, latency_us: 100, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6491a3f7-fdc8-4889-aae3-84b4cdfc9bae, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
27932025-01-15T05:19:35.494ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27942025-01-15T05:19:35.494ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27952025-01-15T05:19:35.494ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27962025-01-15T05:19:35.494ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
2797 [8830] Exited with: ExitStatus(unix_wait_status(9))
2798 [8830][2] reports Exit
2799 Jan 15 05:19:39.496 INFO request completed, latency_us: 178, response_code: 200, uri: /state/cid/2, method: GET, req_id: bda9f70c-70df-43c2-aee6-4e6241ba546d, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
2800 001/300 Write block 2821 len 8192 data: 2 2
2801 002/300 Write block 4814 len 8192 data: 2 2
2802 003/300 Read block 3839 len 16384
2803 004/300 Read block 2762 len 24576
2804 005/300 Write block 1786 len 28672 data: 2 2 2 2 2 2 2
2805 006/300 Write block 3570 len 20480 data: 2 2 2 2 2
2806 007/300 Read block 464 len 20480
2807 008/300 Read block 1101 len 40960
2808 009/300 Write block 3814 len 40960 data: 2 2 3 2 2 2 2 2 2 2
2809 010/300 Flush
2810 011/300 Write block 4136 len 4096 data: 2
2811 012/300 Read block 4112 len 8192
2812 013/300 Write block 3448 len 20480 data: 2 2 2 2 2
2813 014/300 Read block 4598 len 4096
2814 015/300 Write block 2508 len 24576 data: 3 3 3 3 3 3
2815 016/300 Read block 4578 len 16384
2816 017/300 Read block 2146 len 12288
2817 018/300 Read block 536 len 32768
2818 019/300 Write block 1049 len 16384 data: 2 2 2 2
2819 020/300 Write block 3522 len 36864 data: 2 3 2 2 2 3 3 2 2
2820 021/300 Flush
2821 022/300 Read block 2366 len 20480
2822 023/300 Flush
2823 024/300 Read block 102 len 32768
2824 025/300 Read block 1834 len 4096
2825 026/300 Read block 1849 len 20480
2826 027/300 Read block 2328 len 16384
2827 028/300 Write block 1167 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2828 029/300 Read block 3817 len 12288
2829 030/300 Read block 1395 len 36864
2830 031/300 Write block 2650 len 4096 data: 4
2831 032/300 Read block 3038 len 24576
2832 033/300 Read block 4431 len 24576
2833 034/300 Read block 2867 len 20480
2834 035/300 Flush
2835 036/300 Write block 3160 len 28672 data: 4 4 2 2 2 2 2
2836 037/300 Write block 3855 len 8192 data: 3 3
2837 038/300 Write block 1571 len 20480 data: 2 2 3 3 3
2838 039/300 Read block 4755 len 12288
2839 040/300 Read block 3024 len 16384
2840 041/300 Write block 4898 len 12288 data: 4 3 4
2841 042/300 Read block 3119 len 36864
2842 043/300 Read block 1711 len 4096
2843 044/300 Read block 2291 len 24576
2844 045/300 Read block 1872 len 32768
2845 046/300 Write block 4796 len 28672 data: 2 2 2 2 2 2 2
2846 047/300 Read block 1604 len 24576
2847 048/300 Write block 3678 len 24576 data: 2 2 2 2 2 2
2848 049/300 Read block 632 len 36864
2849 050/300 Write block 2799 len 24576 data: 2 2 2 2 2 2
2850 051/300 Write block 1748 len 16384 data: 2 3 3 3
2851 052/300 Write block 4844 len 28672 data: 2 2 2 2 2 2 2
2852 053/300 Read block 3644 len 16384
2853 054/300 Write block 2780 len 20480 data: 2 2 2 2 2
2854 055/300 Flush
2855 056/300 Write block 3380 len 28672 data: 2 3 5 4 4 4 4
2856 057/300 Write block 3430 len 28672 data: 2 2 2 2 2 2 2
2857 058/300 Flush
2858 059/300 Write block 2383 len 8192 data: 2 2
2859 060/300 Write block 4538 len 20480 data: 2 2 2 2 2
2860 061/300 Flush
2861 062/300 Read block 3231 len 20480
2862 063/300 Read block 1487 len 8192
2863 064/300 Write block 4607 len 28672 data: 2 2 2 2 2 2 2
2864 065/300 Write block 3836 len 20480 data: 2 2 2 2 2
2865 066/300 Read block 4305 len 40960
2866 067/300 Read block 4198 len 28672
2867 068/300 Write block 2565 len 24576 data: 3 4 4 3 3 2
2868 069/300 Read block 2173 len 28672
2869 070/300 Read block 1453 len 20480
2870 071/300 Write block 766 len 24576 data: 2 2 3 3 3 2
2871 072/300 Write block 3250 len 16384 data: 2 2 2 2
2872 073/300 Write block 4911 len 24576 data: 4 4 4 4 4 4
2873 074/300 Flush
2874 075/300 Read block 3454 len 12288
2875 076/300 Write block 3116 len 24576 data: 2 2 2 2 2 2
2876 077/300 Read block 2439 len 32768
2877 078/300 Read block 1982 len 36864
2878 079/300 Flush
2879 080/300 Write block 1935 len 4096 data: 2
2880 081/300 Read block 2284 len 36864
2881 082/300 Read block 1665 len 20480
2882 083/300 Write block 2531 len 32768 data: 3 3 3 2 2 2 2 2
2883 084/300 Read block 4812 len 8192
2884 085/300 Write block 2485 len 40960 data: 3 3 2 2 2 2 2 2 2 2
2885 086/300 Read block 1757 len 12288
2886 087/300 Read block 2932 len 32768
2887 088/300 Write block 678 len 40960 data: 4 3 3 4 4 4 4 4 2 2
2888 089/300 Read block 2626 len 40960
2889 090/300 Read block 1091 len 20480
2890 091/300 Read block 1872 len 20480
2891 092/300 Write block 199 len 32768 data: 3 3 2 2 2 2 2 2
2892 093/300 Read block 4092 len 28672
2893 094/300 Write block 4019 len 20480 data: 3 3 3 3 2
2894 095/300 Flush
2895 096/300 Read block 3116 len 32768
2896 097/300 Write block 1701 len 4096 data: 2
2897 098/300 Read block 1471 len 4096
2898 099/300 Write block 3129 len 16384 data: 2 2 2 2
2899 100/300 Write block 3757 len 16384 data: 2 3 3 3
2900 101/300 Read block 4661 len 40960
2901 102/300 Read block 1917 len 24576
2902 103/300 Write block 3650 len 8192 data: 3 3
2903 104/300 Flush
2904 105/300 Read block 3362 len 12288
2905 106/300 Read block 2485 len 32768
2906 107/300 Write block 529 len 16384 data: 2 2 2 2
2907 108/300 Read block 1628 len 16384
2908 109/300 Read block 455 len 36864
2909 110/300 Write block 1870 len 36864 data: 2 2 3 3 3 3 3 3 3
2910 111/300 Read block 626 len 24576
2911 112/300 Write block 3031 len 28672 data: 2 2 2 2 2 2 2
2912 113/300 Write block 2728 len 4096 data: 2
2913 114/300 Read block 1635 len 8192
2914 115/300 Read block 234 len 40960
2915 116/300 Read block 3736 len 20480
2916 117/300 Flush
2917 118/300 Flush
2918 119/300 Flush
2919 120/300 Read block 3495 len 24576
2920 121/300 Read block 4214 len 36864
2921 122/300 Read block 3133 len 28672
2922 123/300 Read block 887 len 8192
2923 124/300 Write block 3184 len 36864 data: 4 3 3 3 2 2 2 2 2
2924 125/300 Write block 2170 len 20480 data: 2 2 2 2 2
2925 126/300 Read block 4646 len 20480
2926 127/300 Write block 4723 len 28672 data: 2 2 2 2 2 3 4
2927 128/300 Write block 2167 len 24576 data: 2 2 2 3 3 3
2928 129/300 Flush
2929 130/300 Flush
2930 131/300 Read block 4456 len 40960
2931 132/300 Read block 2769 len 8192
2932 133/300 Write block 2741 len 16384 data: 2 2 2 2
2933 134/300 Read block 2503 len 32768
2934 135/300 Read block 1691 len 20480
2935 136/300 Write block 2566 len 4096 data: 5
2936 137/300 Write block 2209 len 4096 data: 3
2937 138/300 Read block 1001 len 8192
2938 139/300 Flush
2939 140/300 Flush
2940 141/300 Write block 1930 len 16384 data: 2 2 2 2
2941 142/300 Write block 4465 len 24576 data: 2 2 2 2 2 2
2942 143/300 Write block 4401 len 32768 data: 2 2 2 2 2 2 2 2
2943 144/300 Read block 3885 len 20480
2944 145/300 Write block 194 len 28672 data: 2 2 2 2 2 4 4
2945 146/300 Write block 2759 len 40960 data: 4 4 4 3 3 3 3 2 2 2
2946 147/300 Read block 873 len 16384
2947 148/300 Write block 2109 len 12288 data: 3 3 3
2948 149/300 Read block 3571 len 12288
2949 150/300 Read block 4266 len 8192
2950 151/300 Read block 3131 len 20480
2951 152/300 Flush
2952 153/300 Read block 1894 len 24576
2953 154/300 Write block 3915 len 32768 data: 2 2 2 2 2 2 2 2
2954 155/300 Read block 719 len 36864
2955 156/300 Write block 2895 len 28672 data: 2 2 2 2 3 3 3
2956 157/300 Flush
2957 158/300 Write block 1324 len 20480 data: 2 2 3 3 3
2958 159/300 Flush
2959 160/300 Write block 4142 len 32768 data: 2 2 2 2 2 2 2 2
2960 161/300 Write block 1681 len 20480 data: 2 2 2 2 2
2961 162/300 Read block 4140 len 40960
2962 163/300 Read block 2615 len 24576
2963 164/300 Write block 1293 len 12288 data: 2 2 2
2964 165/300 Read block 1262 len 40960
2965 166/300 Write block 4928 len 12288 data: 2 2 2
2966 167/300 Write block 4106 len 24576 data: 2 2 2 2 2 2
2967 168/300 Flush
2968 169/300 Read block 3375 len 4096
2969 170/300 Write block 2468 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2970 171/300 Read block 744 len 32768
2971 172/300 Write block 3585 len 36864 data: 2 2 2 2 2 2 2 2 2
2972 173/300 Read block 3858 len 12288
2973 174/300 Read block 3257 len 16384
2974 175/300 Read block 1079 len 40960
2975 176/300 Write block 1350 len 16384 data: 2 2 2 2
2976 177/300 Write block 1202 len 28672 data: 3 3 3 3 3 3 3
2977 178/300 Write block 958 len 28672 data: 2 2 2 2 2 2 2
2978 179/300 Read block 1853 len 28672
2979 180/300 Write block 4903 len 40960 data: 4 4 3 3 3 3 5 4 5 5
2980 181/300 Write block 3904 len 8192 data: 2 2
2981 182/300 Read block 2694 len 32768
2982 183/300 Read block 4551 len 32768
2983 184/300 Flush
2984 185/300 Read block 4051 len 8192
2985 186/300 Write block 112 len 36864 data: 2 2 2 2 2 2 2 2 2
2986 187/300 Read block 2301 len 12288
2987 188/300 Read block 2803 len 32768
2988 189/300 Read block 3172 len 40960
2989 190/300 Read block 1029 len 24576
2990 191/300 Flush
2991 192/300 Write block 1342 len 24576 data: 2 2 2 2 2 2
2992 193/300 Write block 3071 len 32768 data: 2 2 2 2 2 2 2 2
2993 194/300 Flush
2994 195/300 Write block 735 len 40960 data: 2 2 2 2 2 2 2 2 3 3
2995 196/300 Flush
2996 197/300 Write block 4187 len 32768 data: 2 2 2 2 2 2 2 2
2997 198/300 Read block 4446 len 12288
2998 199/300 Read block 1715 len 28672
2999 200/300 Read block 875 len 28672
3000 201/300 Read block 884 len 40960
3001 202/300 Read block 3316 len 4096
3002 203/300 Read block 102 len 8192
3003 204/300 Write block 4179 len 16384 data: 2 2 2 2
3004 205/300 Write block 1333 len 36864 data: 2 2 2 2 2 2 2 2 2
3005 206/300 Write block 2317 len 4096 data: 2
3006 207/300 Flush
3007 208/300 Flush
3008 209/300 Flush
3009 210/300 Read block 2751 len 8192
3010 211/300 Write block 3201 len 28672 data: 2 2 2 2 2 2 2
3011 212/300 Read block 613 len 36864
3012 213/300 Write block 2022 len 20480 data: 3 4 4 4 4
3013 214/300 Write block 4164 len 24576 data: 2 3 3 2 2 2
3014 215/300 Read block 3848 len 28672
3015 216/300 Read block 2032 len 16384
3016 217/300 Read block 1200 len 16384
3017 218/300 Write block 1247 len 12288 data: 3 2 2
3018 219/300 Write block 2346 len 20480 data: 2 3 4 3 3
3019 220/300 Write block 1977 len 24576 data: 2 2 2 2 2 2
3020 221/300 Read block 3093 len 16384
3021 222/300 Write block 2856 len 24576 data: 2 2 2 2 2 2
3022 223/300 Write block 4303 len 36864 data: 2 2 2 2 2 2 2 2 2
3023 224/300 Write block 789 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3024 225/300 Read block 4202 len 40960
3025 226/300 Write block 4439 len 12288 data: 2 2 2
3026 227/300 Write block 1863 len 12288 data: 3 3 4
3027 228/300 Read block 4665 len 40960
3028 229/300 Read block 3258 len 20480
3029 230/300 Read block 4839 len 12288
3030 231/300 Write block 4995 len 8192 data: 2 2
3031 232/300 Read block 1087 len 24576
3032 233/300 Read block 1638 len 36864
3033 234/300 Flush
3034 235/300 Read block 4461 len 40960
3035 236/300 Read block 1359 len 28672
3036 237/300 Write block 284 len 24576 data: 2 2 2 2 2 2
3037 238/300 Write block 408 len 12288 data: 2 2 2
3038 239/300 Read block 2701 len 40960
3039 240/300 Flush
3040 241/300 Write block 3605 len 12288 data: 3 3 3
3041 242/300 Write block 4976 len 16384 data: 2 2 2 2
3042 243/300 Read block 4711 len 36864
3043 244/300 Flush
3044 245/300 Write block 4425 len 40960 data: 3 3 3 3 3 2 2 2 2 2
3045 246/300 Read block 2041 len 16384
3046 247/300 Write block 1722 len 24576 data: 2 2 2 2 2 2
3047 248/300 Write block 2182 len 24576 data: 2 2 2 2 2 2
3048 249/300 Read block 4471 len 36864
3049 250/300 Read block 2867 len 20480
3050 251/300 Read block 2180 len 20480
3051 252/300 Read block 4834 len 16384
3052 253/300 Read block 1789 len 32768
3053 254/300 Read block 4579 len 32768
3054 255/300 Read block 3881 len 40960
3055 256/300 Write block 3943 len 32768 data: 6 6 5 5 3 3 3 3
3056 257/300 Write block 552 len 28672 data: 2 2 2 2 2 2 2
3057 258/300 Read block 2538 len 8192
3058 259/300 Read block 4978 len 4096
3059 260/300 Write block 3987 len 12288 data: 2 2 2
3060 261/300 Read block 2975 len 24576
3061 262/300 Write block 2052 len 36864 data: 2 2 2 2 2 2 2 2 2
3062 263/300 Read block 1372 len 24576
3063 264/300 Read block 529 len 12288
3064 265/300 Read block 3318 len 32768
3065 266/300 Write block 177 len 4096 data: 2
3066 267/300 Write block 1907 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3067 268/300 Read block 4051 len 16384
3068 269/300 Write block 4344 len 36864 data: 2 2 2 2 2 2 2 2 2
3069 270/300 Read block 4767 len 8192
3070 271/300 Read block 4573 len 20480
3071 272/300 Read block 433 len 8192
3072 273/300 Write block 662 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3073 274/300 Read block 466 len 16384
3074 275/300 Read block 340 len 4096
3075 276/300 Read block 3894 len 40960
3076 277/300 Write block 3422 len 12288 data: 2 2 2
3077 278/300 Read block 157 len 28672
3078 279/300 Write block 696 len 4096 data: 3
3079 280/300 Read block 4545 len 20480
3080 281/300 Read block 816 len 32768
3081 282/300 Read block 2986 len 16384
3082 283/300 Write block 2029 len 28672 data: 5 4 4 3 3 2 2
3083 284/300 Write block 3273 len 28672 data: 2 2 2 2 2 2 2
3084 285/300 Read block 3093 len 12288
3085 286/300 Read block 3852 len 16384
3086 287/300 Read block 778 len 4096
3087 288/300 Write block 3968 len 8192 data: 2 2
3088 289/300 Write block 685 len 32768 data: 5 3 3 2 2 2 3 3
3089 290/300 Read block 4114 len 40960
3090 291/300 Write block 1877 len 40960 data: 4 4 3 2 2 2 2 2 2 2
3091 292/300 Write block 1146 len 20480 data: 2 3 3 3 3
3092 293/300 Read block 1221 len 4096
3093 294/300 Write block 2325 len 8192 data: 2 2
3094 295/300 Write block 1365 len 32768 data: 2 2 2 2 3 3 3 3
3095 296/300 Read block 1880 len 8192
3096 297/300 Write block 166 len 8192 data: 2 2
3097 298/300 Write block 4448 len 4096 data: 2
3098 299/300 Read block 2229 len 40960
3099 300/300 Flush
3100 301/300 Read block 1347 len 4096
3101 Main task has work to do, go find it
3102 got dsc Start(2)
3103 start 2
3104 Jan 15 05:19:39.729 INFO request completed, latency_us: 218, response_code: 204, uri: /start/cid/2, method: POST, req_id: cf3b36e0-d3f3-4f5f-b7a9-2d5d4dac40f3, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
3105 Replay: started 2, returned:Ok(())
3106 ----------------------------------------------------------------
3107 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3108 States: Active Active Offline
3109 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3110 1604 Acked Write 2 Done Done Sent false
3111 1605 Acked Write 2 Done Done Sent false
3112 1606 Acked Read 4 Done Done Sent false
3113 1607 Acked Read 6 Done Done Sent false
3114 1608 Acked Write 7 Done Done Sent false
3115 1609 Acked Write 5 Done Done Sent false
3116 1610 Acked Read 5 Done Done Sent false
3117 1611 Acked Read 10 Done Done Sent false
3118 1612 Acked Write 10 Done Done Sent false
3119 1613 Acked Flush 0 Done Done Sent false
3120 1614 Acked Write 1 Done Done Sent false
3121 1615 Acked Read 2 Done Done Sent false
3122 1616 Acked Write 5 Done Done Sent false
3123 1617 Acked Read 1 Done Done Sent false
3124 1618 Acked Write 6 Done Done Sent false
3125 1619 Acked Read 4 Done Done Sent false
3126 1620 Acked Read 3 Done Done Sent false
3127 1621 Acked Read 8 Done Done Sent false
3128 1622 Acked Write 4 Done Done Sent false
3129 1623 Acked Write 9 Done Done Sent false
3130 1624 Acked Flush 0 Done Done Sent false
3131 1625 Acked Read 5 Done Done Sent false
3132 1626 Acked Flush 0 Done Done Sent false
3133 1627 Acked Read 8 Done Done Sent false
3134 1628 Acked Read 1 Done Done Sent false
3135 1629 Acked Read 5 Done Done Sent false
3136 1630 Acked Read 4 Done Done Sent false
3137 1631 Acked Write 10 Done Done Sent false
3138 1632 Acked Read 3 Done Done Sent false
3139 1633 Acked Read 9 Done Done Sent false
3140 1634 Acked Write 1 Done Done Sent false
3141 1635 Acked Read 6 Done Done Sent false
3142 1636 Acked Read 6 Done Done Sent false
3143 1637 Acked Read 5 Done Done Sent false
3144 1638 Acked Flush 0 Done Done Sent false
3145 1639 Acked Write 7 Done Done Sent false
3146 1640 Acked Write 2 Done Done Sent false
3147 1641 Acked Write 5 Done Done Sent false
3148 1642 Acked Read 3 Done Done Sent false
3149 1643 Acked Read 4 Done Done Sent false
3150 1644 Acked Write 3 Done Done Sent false
3151 1645 Acked Read 9 Done Done Sent false
3152 1646 Acked Read 1 Done Done Sent false
3153 1647 Acked Read 6 Done Done Sent false
3154 1648 Acked Read 8 Done Done Sent false
3155 1649 Acked Write 7 Done Done Sent false
3156 1650 Acked Read 6 Done Done Sent false
3157 1651 Acked Write 6 Done Done Sent false
3158 1652 Acked Read 9 Done Done Sent false
3159 1653 Acked Write 6 Done Done Sent false
3160 1654 Acked Write 4 Done Done Sent false
3161 1655 Acked Write 7 Done Done Sent false
3162 1656 Acked Read 4 Done Done Sent false
3163 1657 Acked Write 5 Done Done Sent false
3164 1658 Acked Flush 0 Done Done Sent false
3165 1659 Acked Write 7 Done Done Sent false
3166 1660 Acked Write 7 Done Done Sent false
3167 1661 Acked Flush 0 Done Done Sent false
3168 1662 Acked Write 2 Done Done Sent false
3169 1663 Acked Write 5 Done Done Sent false
3170 1664 Acked Flush 0 Done Done Sent false
3171 1665 Acked Read 5 Done Done Sent false
3172 1666 Acked Read 2 Done Done Sent false
3173 1667 Acked Write 7 Done Done Sent false
3174 1668 Acked Write 5 Done Done Sent false
3175 1669 Acked Read 10 Done Done Sent false
3176 1670 Acked Read 7 Done Done Sent false
3177 1671 Acked Write 6 Done Done Sent false
3178 1672 Acked Read 7 Done Done Sent false
3179 1673 Acked Read 5 Done Done Sent false
3180 1674 Acked Write 6 Done Done Sent false
3181 1675 Acked Write 4 Done Done Sent false
3182 1676 Acked Write 6 Done Done Sent false
3183 1677 Acked Flush 0 Done Done Sent false
3184 1678 Acked Read 3 Done Done Sent false
3185 1679 Acked Write 6 Done Done Sent false
3186 1680 Acked Read 8 Done Done Sent false
3187 1681 Acked Read 9 Done Done Sent false
3188 1682 Acked Flush 0 Done Done Sent false
3189 1683 Acked Write 1 Done Done Sent false
3190 1684 Acked Read 9 Done Done Sent false
3191 1685 Acked Read 5 Done Done Sent false
3192 1686 Acked Write 8 Done Done Sent false
3193 1687 Acked Read 2 Done Done Sent false
3194 1688 Acked Write 10 Done Done Sent false
3195 1689 Acked Read 3 Done Done Sent false
3196 1690 Acked Read 8 Done Done Sent false
3197 1691 Acked Write 10 Done Done Sent false
3198 1692 Acked Read 10 Done Done Sent false
3199 1693 Acked Read 5 Done Done Sent false
3200 1694 Acked Read 5 Done Done Sent false
3201 1695 Acked Write 8 Done Done Sent false
3202 1696 Acked Read 7 Done Done Sent false
3203 1697 Acked Write 5 Done Done Sent false
3204 1698 Acked Flush 0 Done Done Sent false
3205 1699 Acked Read 8 Done Done Sent false
3206 1700 Acked Write 1 Done Done Sent false
3207 1701 Acked Read 1 Done Done Sent false
3208 1702 Acked Write 4 Done Done Sent false
3209 1703 Acked Write 4 Done Done Sent false
3210 1704 Acked Read 10 Done Done Sent false
3211 1705 Acked Read 6 Done Done Sent false
3212 1706 Acked Write 2 Done Done Sent false
3213 1707 Acked Flush 0 Done Done Sent false
3214 1708 Acked Read 3 Done Done Sent false
3215 1709 Acked Read 8 Done Done Sent false
3216 1710 Acked Write 4 Done Done Sent false
3217 1711 Acked Read 4 Done Done Sent false
3218 1712 Acked Read 9 Done Done Sent false
3219 1713 Acked Write 9 Done Done Sent false
3220 1714 Acked Read 6 Done Done Sent false
3221 1715 Acked Write 7 Done Done Sent false
3222 1716 Acked Write 1 Done Done Sent false
3223 1717 Acked Read 2 Done Done Sent false
3224 1718 Acked Read 10 Done Done Sent false
3225 1719 Acked Read 5 Done Done Sent false
3226 1720 Acked Flush 0 Done Done Sent false
3227 1721 Acked Flush 0 Done Done Sent false
3228 1722 Acked Flush 0 Done Done Sent false
3229 1723 Acked Read 6 Done Done Sent false
3230 1724 Acked Read 9 Done Done Sent false
3231 1725 Acked Read 7 Done Done Sent false
3232 1726 Acked Read 2 Done Done Sent false
3233 1727 Acked Write 9 Done Done Sent false
3234 1728 Acked Write 5 Done Done Sent false
3235 1729 Acked Read 5 Done Done Sent false
3236 1730 Acked Write 7 Done Done Sent false
3237 1731 Acked Write 6 Done Done Sent false
3238 1732 Acked Flush 0 Done Done Sent false
3239 1733 Acked Flush 0 Done Done Sent false
3240 1734 Acked Read 10 Done Done Sent false
3241 1735 Acked Read 2 Done Done Sent false
3242 1736 Acked Write 4 Done Done Sent false
3243 1737 Acked Read 8 Done Done Sent false
3244 1738 Acked Read 5 Done Done Sent false
3245 1739 Acked Write 1 Done Done Sent false
3246 1740 Acked Write 1 Done Done Sent false
3247 1741 Acked Read 2 Done Done Sent false
3248 1742 Acked Flush 0 Done Done Sent false
3249 1743 Acked Flush 0 Done Done Sent false
3250 1744 Acked Write 4 Done Done Sent false
3251 1745 Acked Write 6 Done Done Sent false
3252 1746 Acked Write 8 Done Done Sent false
3253 1747 Acked Read 5 Done Done Sent false
3254 1748 Acked Write 7 Done Done Sent false
3255 1749 Acked Write 10 Done Done Sent false
3256 1750 Acked Read 4 Done Done Sent false
3257 1751 Acked Write 3 Done Done Sent false
3258 1752 Acked Read 3 Done Done Sent false
3259 1753 Acked Read 2 Done Done Sent false
3260 1754 Acked Read 5 Done Done Sent false
3261 1755 Acked Flush 0 Done Done Sent false
3262 1756 Acked Read 6 Done Done Sent false
3263 1757 Acked Write 8 Done Done Sent false
3264 1758 Acked Read 9 Done Done Sent false
3265 1759 Acked Write 7 Done Done Sent false
3266 1760 Acked Flush 0 Done Done Sent false
3267 1761 Acked Write 5 Done Done Sent false
3268 1762 Acked Flush 0 Done Done Sent false
3269 1763 Acked Write 8 Done Done Sent false
3270 1764 Acked Write 5 Done Done Sent false
3271 1765 Acked Read 10 Done Done Sent false
3272 1766 Acked Read 6 Done Done Sent false
3273 1767 Acked Write 3 Done Done Sent false
3274 1768 Acked Read 10 Done Done Sent false
3275 1769 Acked Write 3 Done Done Sent false
3276 1770 Acked Write 6 Done Done Sent false
3277 1771 Acked Flush 0 Done Done Sent false
3278 1772 Acked Read 1 Done Done Sent false
3279 1773 Acked Write 10 Done Done Sent false
3280 1774 Acked Read 8 Done Done Sent false
3281 1775 Acked Write 9 Done Done Sent false
3282 1776 Acked Read 3 Done Done Sent false
3283 1777 Acked Read 4 Done Done Sent false
3284 1778 Acked Read 10 Done Done Sent false
3285 1779 Acked Write 4 Done Done Sent false
3286 1780 Acked Write 7 Done Done Sent false
3287 1781 Acked Write 7 Done Done Sent false
3288 1782 Acked Read 7 Done Done Sent false
3289 1783 Acked Write 10 Done Done Sent false
3290 1784 Acked Write 2 Done Done Sent false
3291 1785 Acked Read 8 Done Done Sent false
3292 1786 Acked Read 8 Done Done Sent false
3293 1787 Acked Flush 0 Done Done Sent false
3294 1788 Acked Read 2 Done Done Sent false
3295 1789 Acked Write 9 Done Done Sent false
3296 1790 Acked Read 3 Done Done Sent false
3297 1791 Acked Read 8 Done Done Sent false
3298 1792 Acked Read 10 Done Done Sent false
3299 1793 Acked Read 6 Done Done Sent false
3300 1794 Acked Flush 0 Done Done Sent false
3301 1795 Acked Write 6 Done Done Sent false
3302 1796 Acked Write 8 Done Done Sent false
3303 1797 Acked Flush 0 Done Done Sent false
3304 1798 Acked Write 10 Done Done Sent false
3305 1799 Acked Flush 0 Done Done Sent false
3306 1800 Acked Write 8 Done Done Sent false
3307 1801 Acked Read 3 Done Done Sent false
3308 1802 Acked Read 7 Done Done Sent false
3309 1803 Acked Read 7 Done Done Sent false
3310 1804 Acked Read 10 Done Done Sent false
3311 1805 Acked Read 1 Done Done Sent false
3312 1806 Acked Read 2 Done Done Sent false
3313 1807 Acked Write 4 Done Done Sent false
3314 1808 Acked Write 9 Done Done Sent false
3315 1809 Acked Write 1 Done Done Sent false
3316 1810 Acked Flush 0 Done Done Sent false
3317 1811 Acked Flush 0 Done Done Sent false
3318 1812 Acked Flush 0 Done Done Sent false
3319 1813 Acked Read 2 Done Done Sent false
3320 1814 Acked Write 7 Done Done Sent false
3321 1815 Acked Read 9 Done Done Sent false
3322 1816 Acked Write 5 Done Done Sent false
3323 1817 Acked Write 6 Done Done Sent false
3324 1818 Acked Read 7 Done Done Sent false
3325 1819 Acked Read 4 Done Done Sent false
3326 1820 Acked Read 4 Done Done Sent false
3327 1821 Acked Write 3 Done Done Sent false
3328 1822 Acked Write 5 Done Done Sent false
3329 1823 Acked Write 6 Done Done Sent false
3330 1824 Acked Read 4 Done Done Sent false
3331 1825 Acked Write 6 Done Done Sent false
3332 1826 Acked Write 9 Done Done Sent false
3333 1827 Acked Write 10 Done Done Sent false
3334 1828 Acked Read 10 Done Done Sent false
3335 1829 Acked Write 3 Done Done Sent false
3336 1830 Acked Write 3 Done Done Sent false
3337 1831 Acked Read 10 Done Done Sent false
3338 1832 Acked Read 5 Done Done Sent false
3339 1833 Acked Read 3 Done Done Sent false
3340 1834 Acked Write 2 Done Done Sent false
3341 1835 Acked Read 6 Done Done Sent false
3342 1836 Acked Read 9 Done Done Sent false
3343 1837 Acked Flush 0 Done Done Sent false
3344 1838 Acked Read 10 Done Done Sent false
3345 1839 Acked Read 7 Done Done Sent false
3346 1840 Acked Write 6 Done Done Sent false
3347 1841 Acked Write 3 Done Done Sent false
3348 1842 Acked Read 10 Done Done Sent false
3349 1843 Acked Flush 0 Done Done Sent false
3350 1844 Acked Write 3 Done Done Sent false
3351 1845 Acked Write 4 Done Done Sent false
3352 1846 Acked Read 9 Done Done Sent false
3353 1847 Acked Flush 0 Done Done Sent false
3354 1848 Acked Write 10 Done Done Sent false
3355 1849 Acked Read 4 Done Done Sent false
3356 1850 Acked Write 6 Done Done Sent false
3357 1851 Acked Write 6 Done Done Sent false
3358 1852 Acked Read 9 Done Done Sent false
3359 1853 Acked Read 5 Done Done Sent false
3360 1854 Acked Read 5 Done Done Sent false
3361 1855 Acked Read 4 Done Done Sent false
3362 1856 Acked Read 8 Done Done Sent false
3363 1857 Acked Read 8 Done Done Sent false
3364 1858 Acked Read 10 Done Done Sent false
3365 1859 Acked Write 8 Done Done Sent false
3366 1860 Acked Write 7 Done Done Sent false
3367 1861 Acked Read 2 Done Done Sent false
3368 1862 Acked Read 1 Done Done Sent false
3369 1863 Acked Write 3 Done Done Sent false
3370 1864 Acked Read 6 Done Done Sent false
3371 1865 Acked Write 9 Done Done Sent false
3372 1866 Acked Read 6 Done Done Sent false
3373 1867 Acked Read 3 Done Done Sent false
3374 1868 Acked Read 8 Done Done Sent false
3375 1869 Acked Write 1 Done Done Sent false
3376 1870 Acked Write 10 Done Done Sent false
3377 1871 Acked Read 4 Done Done Sent false
3378 1872 Acked Write 9 Done Done Sent false
3379 1873 Acked Read 2 Done Done Sent false
3380 1874 Acked Read 5 Done Done Sent false
3381 1875 Acked Read 2 Done Done Sent false
3382 1876 Acked Write 10 Done Done Sent false
3383 1877 Acked Read 4 Done Done Sent false
3384 1878 Acked Read 1 Done Done Sent false
3385 1879 Acked Read 10 Done Done Sent false
3386 1880 Acked Write 3 Done Done Sent false
3387 1881 Acked Read 7 Done Done Sent false
3388 1882 Acked Write 1 Done Done Sent false
3389 1883 Acked Read 5 Done Done Sent false
3390 1884 Acked Read 8 Done Done Sent false
3391 1885 Acked Read 4 Done Done Sent false
3392 1886 Acked Write 7 Done Done Sent false
3393 1887 Acked Write 7 Done Done Sent false
3394 1888 Acked Read 3 Done Done Sent false
3395 1889 Acked Read 4 Done Done Sent false
3396 1890 Acked Read 1 Done Done Sent false
3397 1891 Acked Write 2 Done Done Sent false
3398 1892 Acked Write 8 Done Done Sent false
3399 1893 Acked Read 10 Done Done Sent false
3400 1894 Acked Write 10 Done Done Sent false
3401 1895 Acked Write 5 Done Done Sent false
3402 1896 Acked Read 1 Done Done Sent false
3403 1897 Acked Write 2 Done Done Sent false
3404 1898 Acked Write 8 Done Done Sent false
3405 1899 Acked Read 2 Done Done Sent false
3406 1900 Acked Write 2 Done Done Sent false
3407 1901 Acked Write 1 Done Done Sent false
3408 1902 Acked Read 10 Done Done Sent false
3409 1903 Acked Flush 0 Done Done Sent false
3410 1904 Acked Read 1 Done Done Sent false
3411 STATES DS:0 DS:1 DS:2 TOTAL
3412 Sent 0 0 301 301
3413 Done 301 301 0 602
3414 Skipped 0 0 0 0
3415 Error 0 0 0 0
3416 Last Flush: 1903 1903 1603
3417 Downstairs last five completed: 1603 1602 1601 1600 1599
3418 Upstairs last five completed: 1904 1903 1902 1901 1900
3419 CLIENT: Up:0 ds:301 act:2
3420 [8830] got start action
3421 [8830] I am going to restart
3422 Starting downstairs at port 8830
3423 Make output file at "/tmp/dsc/downstairs-8830.txt"
3424 [8830][2] reports Starting
3425 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1205)
3426 [8830][2] reports Running
3427 ----------------------------------------------------------------
3428 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3429 States: Active Active Offline
3430 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3431 1604 Acked Write 2 Done Done Sent false
3432 1605 Acked Write 2 Done Done Sent false
3433 1606 Acked Read 4 Done Done Sent false
3434 1607 Acked Read 6 Done Done Sent false
3435 1608 Acked Write 7 Done Done Sent false
3436 1609 Acked Write 5 Done Done Sent false
3437 1610 Acked Read 5 Done Done Sent false
3438 1611 Acked Read 10 Done Done Sent false
3439 1612 Acked Write 10 Done Done Sent false
3440 1613 Acked Flush 0 Done Done Sent false
3441 1614 Acked Write 1 Done Done Sent false
3442 1615 Acked Read 2 Done Done Sent false
3443 1616 Acked Write 5 Done Done Sent false
3444 1617 Acked Read 1 Done Done Sent false
3445 1618 Acked Write 6 Done Done Sent false
3446 1619 Acked Read 4 Done Done Sent false
3447 1620 Acked Read 3 Done Done Sent false
3448 1621 Acked Read 8 Done Done Sent false
3449 1622 Acked Write 4 Done Done Sent false
3450 1623 Acked Write 9 Done Done Sent false
3451 1624 Acked Flush 0 Done Done Sent false
3452 1625 Acked Read 5 Done Done Sent false
3453 1626 Acked Flush 0 Done Done Sent false
3454 1627 Acked Read 8 Done Done Sent false
3455 1628 Acked Read 1 Done Done Sent false
3456 1629 Acked Read 5 Done Done Sent false
3457 1630 Acked Read 4 Done Done Sent false
3458 1631 Acked Write 10 Done Done Sent false
3459 1632 Acked Read 3 Done Done Sent false
3460 1633 Acked Read 9 Done Done Sent false
3461 1634 Acked Write 1 Done Done Sent false
3462 1635 Acked Read 6 Done Done Sent false
3463 1636 Acked Read 6 Done Done Sent false
3464 1637 Acked Read 5 Done Done Sent false
3465 1638 Acked Flush 0 Done Done Sent false
3466 1639 Acked Write 7 Done Done Sent false
3467 1640 Acked Write 2 Done Done Sent false
3468 1641 Acked Write 5 Done Done Sent false
3469 1642 Acked Read 3 Done Done Sent false
3470 1643 Acked Read 4 Done Done Sent false
3471 1644 Acked Write 3 Done Done Sent false
3472 1645 Acked Read 9 Done Done Sent false
3473 1646 Acked Read 1 Done Done Sent false
3474 1647 Acked Read 6 Done Done Sent false
3475 1648 Acked Read 8 Done Done Sent false
3476 1649 Acked Write 7 Done Done Sent false
3477 1650 Acked Read 6 Done Done Sent false
3478 1651 Acked Write 6 Done Done Sent false
3479 1652 Acked Read 9 Done Done Sent false
3480 1653 Acked Write 6 Done Done Sent false
3481 1654 Acked Write 4 Done Done Sent false
3482 1655 Acked Write 7 Done Done Sent false
3483 1656 Acked Read 4 Done Done Sent false
3484 1657 Acked Write 5 Done Done Sent false
3485 1658 Acked Flush 0 Done Done Sent false
3486 1659 Acked Write 7 Done Done Sent false
3487 1660 Acked Write 7 Done Done Sent false
3488 1661 Acked Flush 0 Done Done Sent false
3489 1662 Acked Write 2 Done Done Sent false
3490 1663 Acked Write 5 Done Done Sent false
3491 1664 Acked Flush 0 Done Done Sent false
3492 1665 Acked Read 5 Done Done Sent false
3493 1666 Acked Read 2 Done Done Sent false
3494 1667 Acked Write 7 Done Done Sent false
3495 1668 Acked Write 5 Done Done Sent false
3496 1669 Acked Read 10 Done Done Sent false
3497 1670 Acked Read 7 Done Done Sent false
3498 1671 Acked Write 6 Done Done Sent false
3499 1672 Acked Read 7 Done Done Sent false
3500 1673 Acked Read 5 Done Done Sent false
3501 1674 Acked Write 6 Done Done Sent false
3502 1675 Acked Write 4 Done Done Sent false
3503 1676 Acked Write 6 Done Done Sent false
3504 1677 Acked Flush 0 Done Done Sent false
3505 1678 Acked Read 3 Done Done Sent false
3506 1679 Acked Write 6 Done Done Sent false
3507 1680 Acked Read 8 Done Done Sent false
3508 1681 Acked Read 9 Done Done Sent false
3509 1682 Acked Flush 0 Done Done Sent false
3510 1683 Acked Write 1 Done Done Sent false
3511 1684 Acked Read 9 Done Done Sent false
3512 1685 Acked Read 5 Done Done Sent false
3513 1686 Acked Write 8 Done Done Sent false
3514 1687 Acked Read 2 Done Done Sent false
3515 1688 Acked Write 10 Done Done Sent false
3516 1689 Acked Read 3 Done Done Sent false
3517 1690 Acked Read 8 Done Done Sent false
3518 1691 Acked Write 10 Done Done Sent false
3519 1692 Acked Read 10 Done Done Sent false
3520 1693 Acked Read 5 Done Done Sent false
3521 1694 Acked Read 5 Done Done Sent false
3522 1695 Acked Write 8 Done Done Sent false
3523 1696 Acked Read 7 Done Done Sent false
3524 1697 Acked Write 5 Done Done Sent false
3525 1698 Acked Flush 0 Done Done Sent false
3526 1699 Acked Read 8 Done Done Sent false
3527 1700 Acked Write 1 Done Done Sent false
3528 1701 Acked Read 1 Done Done Sent false
3529 1702 Acked Write 4 Done Done Sent false
3530 1703 Acked Write 4 Done Done Sent false
3531 1704 Acked Read 10 Done Done Sent false
3532 1705 Acked Read 6 Done Done Sent false
3533 1706 Acked Write 2 Done Done Sent false
3534 1707 Acked Flush 0 Done Done Sent false
3535 1708 Acked Read 3 Done Done Sent false
3536 1709 Acked Read 8 Done Done Sent false
3537 1710 Acked Write 4 Done Done Sent false
3538 1711 Acked Read 4 Done Done Sent false
3539 1712 Acked Read 9 Done Done Sent false
3540 1713 Acked Write 9 Done Done Sent false
3541 1714 Acked Read 6 Done Done Sent false
3542 1715 Acked Write 7 Done Done Sent false
3543 1716 Acked Write 1 Done Done Sent false
3544 1717 Acked Read 2 Done Done Sent false
3545 1718 Acked Read 10 Done Done Sent false
3546 1719 Acked Read 5 Done Done Sent false
3547 1720 Acked Flush 0 Done Done Sent false
3548 1721 Acked Flush 0 Done Done Sent false
3549 1722 Acked Flush 0 Done Done Sent false
3550 1723 Acked Read 6 Done Done Sent false
3551 1724 Acked Read 9 Done Done Sent false
3552 1725 Acked Read 7 Done Done Sent false
3553 1726 Acked Read 2 Done Done Sent false
3554 1727 Acked Write 9 Done Done Sent false
3555 1728 Acked Write 5 Done Done Sent false
3556 1729 Acked Read 5 Done Done Sent false
3557 1730 Acked Write 7 Done Done Sent false
3558 1731 Acked Write 6 Done Done Sent false
3559 1732 Acked Flush 0 Done Done Sent false
3560 1733 Acked Flush 0 Done Done Sent false
3561 1734 Acked Read 10 Done Done Sent false
3562 1735 Acked Read 2 Done Done Sent false
3563 1736 Acked Write 4 Done Done Sent false
3564 1737 Acked Read 8 Done Done Sent false
3565 1738 Acked Read 5 Done Done Sent false
3566 1739 Acked Write 1 Done Done Sent false
3567 1740 Acked Write 1 Done Done Sent false
3568 1741 Acked Read 2 Done Done Sent false
3569 1742 Acked Flush 0 Done Done Sent false
3570 1743 Acked Flush 0 Done Done Sent false
3571 1744 Acked Write 4 Done Done Sent false
3572 1745 Acked Write 6 Done Done Sent false
3573 1746 Acked Write 8 Done Done Sent false
3574 1747 Acked Read 5 Done Done Sent false
3575 1748 Acked Write 7 Done Done Sent false
3576 1749 Acked Write 10 Done Done Sent false
3577 1750 Acked Read 4 Done Done Sent false
3578 1751 Acked Write 3 Done Done Sent false
3579 1752 Acked Read 3 Done Done Sent false
3580 1753 Acked Read 2 Done Done Sent false
3581 1754 Acked Read 5 Done Done Sent false
3582 1755 Acked Flush 0 Done Done Sent false
3583 1756 Acked Read 6 Done Done Sent false
3584 1757 Acked Write 8 Done Done Sent false
3585 1758 Acked Read 9 Done Done Sent false
3586 1759 Acked Write 7 Done Done Sent false
3587 1760 Acked Flush 0 Done Done Sent false
3588 1761 Acked Write 5 Done Done Sent false
3589 1762 Acked Flush 0 Done Done Sent false
3590 1763 Acked Write 8 Done Done Sent false
3591 1764 Acked Write 5 Done Done Sent false
3592 1765 Acked Read 10 Done Done Sent false
3593 1766 Acked Read 6 Done Done Sent false
3594 1767 Acked Write 3 Done Done Sent false
3595 1768 Acked Read 10 Done Done Sent false
3596 1769 Acked Write 3 Done Done Sent false
3597 1770 Acked Write 6 Done Done Sent false
3598 1771 Acked Flush 0 Done Done Sent false
3599 1772 Acked Read 1 Done Done Sent false
3600 1773 Acked Write 10 Done Done Sent false
3601 1774 Acked Read 8 Done Done Sent false
3602 1775 Acked Write 9 Done Done Sent false
3603 1776 Acked Read 3 Done Done Sent false
3604 1777 Acked Read 4 Done Done Sent false
3605 1778 Acked Read 10 Done Done Sent false
3606 1779 Acked Write 4 Done Done Sent false
3607 1780 Acked Write 7 Done Done Sent false
3608 1781 Acked Write 7 Done Done Sent false
3609 1782 Acked Read 7 Done Done Sent false
3610 1783 Acked Write 10 Done Done Sent false
3611 1784 Acked Write 2 Done Done Sent false
3612 1785 Acked Read 8 Done Done Sent false
3613 1786 Acked Read 8 Done Done Sent false
3614 1787 Acked Flush 0 Done Done Sent false
3615 1788 Acked Read 2 Done Done Sent false
3616 1789 Acked Write 9 Done Done Sent false
3617 1790 Acked Read 3 Done Done Sent false
3618 1791 Acked Read 8 Done Done Sent false
3619 1792 Acked Read 10 Done Done Sent false
3620 1793 Acked Read 6 Done Done Sent false
3621 1794 Acked Flush 0 Done Done Sent false
3622 1795 Acked Write 6 Done Done Sent false
3623 1796 Acked Write 8 Done Done Sent false
3624 1797 Acked Flush 0 Done Done Sent false
3625 1798 Acked Write 10 Done Done Sent false
3626 1799 Acked Flush 0 Done Done Sent false
3627 1800 Acked Write 8 Done Done Sent false
3628 1801 Acked Read 3 Done Done Sent false
3629 1802 Acked Read 7 Done Done Sent false
3630 1803 Acked Read 7 Done Done Sent false
3631 1804 Acked Read 10 Done Done Sent false
3632 1805 Acked Read 1 Done Done Sent false
3633 1806 Acked Read 2 Done Done Sent false
3634 1807 Acked Write 4 Done Done Sent false
3635 1808 Acked Write 9 Done Done Sent false
3636 1809 Acked Write 1 Done Done Sent false
3637 1810 Acked Flush 0 Done Done Sent false
3638 1811 Acked Flush 0 Done Done Sent false
3639 1812 Acked Flush 0 Done Done Sent false
3640 1813 Acked Read 2 Done Done Sent false
3641 1814 Acked Write 7 Done Done Sent false
3642 1815 Acked Read 9 Done Done Sent false
3643 1816 Acked Write 5 Done Done Sent false
3644 1817 Acked Write 6 Done Done Sent false
3645 1818 Acked Read 7 Done Done Sent false
3646 1819 Acked Read 4 Done Done Sent false
3647 1820 Acked Read 4 Done Done Sent false
3648 1821 Acked Write 3 Done Done Sent false
3649 1822 Acked Write 5 Done Done Sent false
3650 1823 Acked Write 6 Done Done Sent false
3651 1824 Acked Read 4 Done Done Sent false
3652 1825 Acked Write 6 Done Done Sent false
3653 1826 Acked Write 9 Done Done Sent false
3654 1827 Acked Write 10 Done Done Sent false
3655 1828 Acked Read 10 Done Done Sent false
3656 1829 Acked Write 3 Done Done Sent false
3657 1830 Acked Write 3 Done Done Sent false
3658 1831 Acked Read 10 Done Done Sent false
3659 1832 Acked Read 5 Done Done Sent false
3660 1833 Acked Read 3 Done Done Sent false
3661 1834 Acked Write 2 Done Done Sent false
3662 1835 Acked Read 6 Done Done Sent false
3663 1836 Acked Read 9 Done Done Sent false
3664 1837 Acked Flush 0 Done Done Sent false
3665 1838 Acked Read 10 Done Done Sent false
3666 1839 Acked Read 7 Done Done Sent false
3667 1840 Acked Write 6 Done Done Sent false
3668 1841 Acked Write 3 Done Done Sent false
3669 1842 Acked Read 10 Done Done Sent false
3670 1843 Acked Flush 0 Done Done Sent false
3671 1844 Acked Write 3 Done Done Sent false
3672 1845 Acked Write 4 Done Done Sent false
3673 1846 Acked Read 9 Done Done Sent false
3674 1847 Acked Flush 0 Done Done Sent false
3675 1848 Acked Write 10 Done Done Sent false
3676 1849 Acked Read 4 Done Done Sent false
3677 1850 Acked Write 6 Done Done Sent false
3678 1851 Acked Write 6 Done Done Sent false
3679 1852 Acked Read 9 Done Done Sent false
3680 1853 Acked Read 5 Done Done Sent false
3681 1854 Acked Read 5 Done Done Sent false
3682 1855 Acked Read 4 Done Done Sent false
3683 1856 Acked Read 8 Done Done Sent false
3684 1857 Acked Read 8 Done Done Sent false
3685 1858 Acked Read 10 Done Done Sent false
3686 1859 Acked Write 8 Done Done Sent false
3687 1860 Acked Write 7 Done Done Sent false
3688 1861 Acked Read 2 Done Done Sent false
3689 1862 Acked Read 1 Done Done Sent false
3690 1863 Acked Write 3 Done Done Sent false
3691 1864 Acked Read 6 Done Done Sent false
3692 1865 Acked Write 9 Done Done Sent false
3693 1866 Acked Read 6 Done Done Sent false
3694 1867 Acked Read 3 Done Done Sent false
3695 1868 Acked Read 8 Done Done Sent false
3696 1869 Acked Write 1 Done Done Sent false
3697 1870 Acked Write 10 Done Done Sent false
3698 1871 Acked Read 4 Done Done Sent false
3699 1872 Acked Write 9 Done Done Sent false
3700 1873 Acked Read 2 Done Done Sent false
3701 1874 Acked Read 5 Done Done Sent false
3702 1875 Acked Read 2 Done Done Sent false
3703 1876 Acked Write 10 Done Done Sent false
3704 1877 Acked Read 4 Done Done Sent false
3705 1878 Acked Read 1 Done Done Sent false
3706 1879 Acked Read 10 Done Done Sent false
3707 1880 Acked Write 3 Done Done Sent false
3708 1881 Acked Read 7 Done Done Sent false
3709 1882 Acked Write 1 Done Done Sent false
3710 1883 Acked Read 5 Done Done Sent false
3711 1884 Acked Read 8 Done Done Sent false
3712 1885 Acked Read 4 Done Done Sent false
3713 1886 Acked Write 7 Done Done Sent false
3714 1887 Acked Write 7 Done Done Sent false
3715 1888 Acked Read 3 Done Done Sent false
3716 1889 Acked Read 4 Done Done Sent false
3717 1890 Acked Read 1 Done Done Sent false
3718 1891 Acked Write 2 Done Done Sent false
3719 1892 Acked Write 8 Done Done Sent false
3720 1893 Acked Read 10 Done Done Sent false
3721 1894 Acked Write 10 Done Done Sent false
3722 1895 Acked Write 5 Done Done Sent false
3723 1896 Acked Read 1 Done Done Sent false
3724 1897 Acked Write 2 Done Done Sent false
3725 1898 Acked Write 8 Done Done Sent false
3726 1899 Acked Read 2 Done Done Sent false
3727 1900 Acked Write 2 Done Done Sent false
3728 1901 Acked Write 1 Done Done Sent false
3729 1902 Acked Read 10 Done Done Sent false
3730 1903 Acked Flush 0 Done Done Sent false
3731 1904 Acked Read 1 Done Done Sent false
3732 1905 Acked Flush 0 Done Done Sent false
3733 STATES DS:0 DS:1 DS:2 TOTAL
3734 Sent 0 0 302 302
3735 Done 302 302 0 604
3736 Skipped 0 0 0 0
3737 Error 0 0 0 0
3738 Last Flush: 1905 1905 1603
3739 Downstairs last five completed: 1603 1602 1601 1600 1599
3740 Upstairs last five completed: 1905 1904 1903 1902 1901
3741 CLIENT: Up:0 ds:302 act:2
37422025-01-15T05:19:45.495ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37432025-01-15T05:19:45.495ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57197) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37442025-01-15T05:19:45.496ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37452025-01-15T05:19:45.496ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37462025-01-15T05:19:45.497ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37472025-01-15T05:19:45.497ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37482025-01-15T05:19:45.497ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37492025-01-15T05:19:45.497ZINFOcrucible: [2] client re-new 302 jobs since flush 1603 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37502025-01-15T05:19:45.497ZINFOcrucible: [2] Replayed 302 jobs since flush: 1603 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
3751 ----------------------------------------------------------------
3752 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3753 Downstairs last five completed: 1905 1904 1903 1902 1901
3754 Upstairs last five completed: 1905 1904 1903 1902 1901
3755 CLIENT: Up:0 ds:0 act:3
3756 Replay: All jobs finished
3757 Main task has work to do, go find it
3758 got dsc Stop(0)
3759 stop 0
3760 Jan 15 05:19:47.734 INFO request completed, latency_us: 306, response_code: 204, uri: /stop/cid/0, method:[8810] Got stop action so:false kr:false
3761 POST, req_id: 34936f18-696a-4dd6-be57-632328e7224b, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
37622025-01-15T05:19:47.735ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37632025-01-15T05:19:47.735ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
37642025-01-15T05:19:47.735ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
3765 Jan 15 05:19:47.735 INFO request completed, latency_us: 129, response_code: 200, uri: /state/cid/0, method: GET, req_id: b1d2615e-e65b-409a-b3eb-5fedc8cc0c48, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
37662025-01-15T05:19:47.735ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
3767 [8810] Exited with: ExitStatus(unix_wait_status(9))
3768 [8810][0] reports Exit
3769 Jan 15 05:19:51.737 INFO request completed, latency_us: 191, response_code: 200, uri: /state/cid/0, method: GET, req_id: 08ef0eb6-6bc1-45f4-bf1f-f99e20928f0a, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
3770 001/300 Read block 3326 len 20480
3771 002/300 Read block 3278 len 16384
3772 003/300 Read block 1655 len 16384
3773 004/300 Read block 1373 len 20480
3774 005/300 Flush
3775 006/300 Read block 1109 len 32768
3776 007/300 Write block 2281 len 16384 data: 2 2 2 2
3777 008/300 Flush
3778 009/300 Write block 339 len 20480 data: 3 3 3 3 3
3779 010/300 Write block 4125 len 12288 data: 2 2 3
3780 011/300 Write block 627 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3781 012/300 Read block 419 len 40960
3782 013/300 Read block 1691 len 8192
3783 014/300 Write block 2972 len 24576 data: 2 2 2 2 2 2
3784 015/300 Read block 4786 len 16384
3785 016/300 Write block 1411 len 40960 data: 2 2 2 2 2 3 3 3 3 3
3786 017/300 Flush
3787 018/300 Write block 1449 len 32768 data: 2 2 2 2 2 2 2 2
3788 019/300 Read block 2507 len 12288
3789 020/300 Read block 952 len 36864
3790 021/300 Read block 3543 len 36864
3791 022/300 Read block 3582 len 20480
3792 023/300 Read block 3989 len 16384
3793 024/300 Write block 3199 len 24576 data: 2 2 3 3 3 3
3794 025/300 Write block 4338 len 40960 data: 2 2 2 2 2 2 3 3 3 3
3795 026/300 Read block 3877 len 8192
3796 027/300 Read block 1708 len 8192
3797 028/300 Read block 657 len 4096
3798 029/300 Read block 4502 len 40960
3799 030/300 Write block 4519 len 28672 data: 2 3 2 2 2 2 2
3800 031/300 Write block 4104 len 36864 data: 2 2 3 3 3 3 3 3 2
3801 032/300 Read block 1835 len 20480
3802 033/300 Read block 1798 len 8192
3803 034/300 Write block 4844 len 28672 data: 3 3 3 3 3 3 3
3804 035/300 Read block 1344 len 8192
3805 036/300 Read block 3479 len 32768
3806 037/300 Read block 4764 len 4096
3807 038/300 Write block 1488 len 8192 data: 2 2
3808 039/300 Write block 3738 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3809 040/300 Write block 4853 len 4096 data: 2
3810 041/300 Write block 2697 len 8192 data: 2 2
3811 042/300 Write block 4732 len 12288 data: 4 3 3
3812 043/300 Read block 4258 len 24576
3813 044/300 Read block 3315 len 40960
3814 045/300 Flush
3815 046/300 Read block 2006 len 28672
3816 047/300 Write block 4328 len 28672 data: 2 2 2 2 2 2 2
3817 048/300 Flush
3818 049/300 Write block 429 len 4096 data: 2
3819 050/300 Write block 3967 len 4096 data: 2
3820 051/300 Read block 4208 len 20480
3821 052/300 Write block 594 len 8192 data: 2 2
3822 053/300 Read block 3678 len 40960
3823 054/300 Write block 1342 len 28672 data: 3 3 3 3 3 3 2
3824 055/300 Write block 3054 len 28672 data: 2 2 2 3 3 2 2
3825 056/300 Write block 1409 len 8192 data: 2 2
3826 057/300 Read block 912 len 8192
3827 058/300 Write block 3912 len 36864 data: 3 2 2 3 3 3 3 3 3
3828 059/300 Write block 3713 len 24576 data: 3 3 3 4 4 2
3829 060/300 Read block 4951 len 36864
3830 061/300 Read block 3119 len 28672
3831 062/300 Read block 384 len 40960
3832 063/300 Read block 314 len 8192
3833 064/300 Read block 449 len 40960
3834 065/300 Write block 2202 len 8192 data: 2 3
3835 066/300 Read block 3876 len 40960
3836 067/300 Write block 2105 len 28672 data: 2 2 2 2 4 4 4
3837 068/300 Write block 2780 len 4096 data: 3
3838 069/300 Write block 2067 len 36864 data: 2 2 2 2 3 3 3 3 3
3839 070/300 Write block 2625 len 20480 data: 2 2 2 2 2
3840 071/300 Write block 2982 len 24576 data: 3 3 3 3 3 3
3841 072/300 Read block 940 len 8192
3842 073/300 Read block 1951 len 12288
3843 074/300 Read block 74 len 28672
3844 075/300 Read block 2917 len 8192
3845 076/300 Write block 1733 len 8192 data: 2 2
3846 077/300 Read block 4602 len 28672
3847 078/300 Write block 4214 len 20480 data: 2 2 2 2 2
3848 079/300 Read block 2016 len 8192
3849 080/300 Read block 2386 len 20480
3850 081/300 Read block 1477 len 24576
3851 082/300 Write block 515 len 24576 data: 2 2 2 3 3 3
3852 083/300 Read block 1277 len 16384
3853 084/300 Read block 91 len 40960
3854 085/300 Write block 2850 len 12288 data: 2 2 2
3855 086/300 Read block 4906 len 36864
3856 087/300 Write block 469 len 40960 data: 3 3 3 3 3 3 2 2 2 2
3857 088/300 Write block 1430 len 24576 data: 2 2 2 2 2 2
3858 089/300 Write block 2981 len 24576 data: 2 4 4 4 4 4
3859 090/300 Write block 4888 len 36864 data: 2 2 2 2 2 2 2 2 3
3860 091/300 Read block 4314 len 16384
3861 092/300 Write block 2052 len 28672 data: 3 3 3 3 3 3 3
3862 093/300 Read block 400 len 12288
3863 094/300 Write block 2147 len 8192 data: 2 2
3864 095/300 Read block 154 len 8192
3865 096/300 Write block 2807 len 36864 data: 2 2 2 2 2 2 2 2 2
3866 097/300 Read block 3490 len 24576
3867 098/300 Read block 2414 len 32768
3868 099/300 Write block 2201 len 24576 data: 2 3 4 3 3 3
3869 100/300 Flush
3870 101/300 Write block 3780 len 24576 data: 2 2 2 2 2 2
3871 102/300 Write block 1555 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3872 103/300 Read block 4350 len 4096
3873 104/300 Write block 3244 len 8192 data: 2 2
3874 105/300 Read block 4021 len 28672
3875 106/300 Read block 2814 len 32768
3876 107/300 Read block 3417 len 4096
3877 108/300 Read block 17 len 20480
3878 109/300 Read block 572 len 12288
3879 110/300 Write block 2763 len 8192 data: 4 4
3880 111/300 Read block 3505 len 12288
3881 112/300 Read block 2611 len 16384
3882 113/300 Write block 2362 len 20480 data: 4 4 3 3 4
3883 114/300 Read block 1607 len 36864
3884 115/300 Write block 2345 len 32768 data: 2 3 4 5 4 4 2 2
3885 116/300 Read block 4255 len 36864
3886 117/300 Read block 4870 len 24576
3887 118/300 Write block 2741 len 12288 data: 3 3 3
3888 119/300 Read block 1873 len 24576
3889 120/300 Flush
3890 121/300 Read block 2424 len 40960
3891 122/300 Read block 1262 len 20480
3892 123/300 Flush
3893 124/300 Write block 2830 len 36864 data: 2 2 2 4 4 4 4 4 4
3894 125/300 Read block 4254 len 20480
3895 126/300 Flush
3896 127/300 Write block 1380 len 8192 data: 2 2
3897 128/300 Write block 1806 len 32768 data: 3 2 2 2 2 2 3 3
3898 129/300 Read block 1589 len 8192
3899 130/300 Flush
3900 131/300 Read block 4278 len 36864
3901 132/300 Read block 3041 len 24576
3902 133/300 Write block 1682 len 4096 data: 3
3903 134/300 Read block 4856 len 24576
3904 135/300 Read block 660 len 40960
3905 136/300 Write block 939 len 36864 data: 2 2 2 2 2 2 2 2 2
3906 137/300 Flush
3907 138/300 Flush
3908 139/300 Write block 4316 len 16384 data: 2 2 2 2
3909 140/300 Write block 1932 len 20480 data: 3 3 2 3 2
3910 141/300 Read block 4991 len 20480
3911 142/300 Write block 3674 len 36864 data: 3 3 3 2 3 3 3 3 3
3912 143/300 Write block 2239 len 24576 data: 2 2 2 2 3 3
3913 144/300 Read block 527 len 40960
3914 145/300 Flush
3915 146/300 Read block 1127 len 40960
3916 147/300 Read block 1304 len 4096
3917 148/300 Read block 3889 len 16384
3918 149/300 Write block 3351 len 12288 data: 3 3 3
3919 150/300 Read block 1687 len 20480
3920 151/300 Write block 2084 len 40960 data: 2 2 3 2 2 2 2 2 2 2
3921 152/300 Write block 4495 len 36864 data: 3 3 3 3 3 3 4 3 3
3922 153/300 Read block 277 len 12288
3923 154/300 Read block 1015 len 20480
3924 155/300 Write block 4751 len 24576 data: 2 2 2 2 2 2
3925 156/300 Read block 2255 len 16384
3926 157/300 Read block 4908 len 20480
3927 158/300 Write block 1186 len 36864 data: 2 2 2 2 2 2 2 2 2
3928 159/300 Read block 1369 len 4096
3929 160/300 Flush
3930 161/300 Write block 992 len 36864 data: 2 2 2 2 2 2 2 2 2
3931 162/300 Read block 2876 len 4096
3932 163/300 Read block 4116 len 28672
3933 164/300 Write block 4390 len 20480 data: 2 2 2 2 2
3934 165/300 Flush
3935 166/300 Read block 33 len 32768
3936 167/300 Write block 3889 len 28672 data: 2 2 2 2 2 2 2
3937 168/300 Write block 3950 len 8192 data: 4 3
3938 169/300 Write block 3892 len 12288 data: 3 3 3
3939 170/300 Read block 182 len 24576
3940 171/300 Read block 393 len 8192
3941 172/300 Write block 4542 len 8192 data: 3 2
3942 173/300 Write block 3865 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3943 174/300 Read block 935 len 32768
3944 175/300 Read block 167 len 24576
3945 176/300 Read block 3857 len 28672
3946 177/300 Write block 3127 len 8192 data: 3 3
3947 178/300 Write block 4884 len 12288 data: 2 2 2
3948 179/300 Write block 1424 len 40960 data: 3 2 3 3 2 2 3 3 3 3
3949 180/300 Write block 1636 len 40960 data: 2 2 3 2 2 2 2 2 2 2
3950 181/300 Write block 3760 len 16384 data: 4 3 2 2
3951 182/300 Write block 4875 len 32768 data: 2 2 2 2 2 2 2 2
3952 183/300 Flush
3953 184/300 Write block 4440 len 4096 data: 3
3954 185/300 Flush
3955 186/300 Read block 4824 len 20480
3956 187/300 Write block 3250 len 12288 data: 3 3 3
3957 188/300 Write block 2366 len 28672 data: 5 4 4 3 3 3 3
3958 189/300 Read block 2850 len 40960
3959 190/300 Read block 998 len 32768
3960 191/300 Read block 3033 len 28672
3961 192/300 Flush
3962 193/300 Flush
3963 194/300 Read block 2444 len 8192
3964 195/300 Read block 2523 len 16384
3965 196/300 Read block 317 len 20480
3966 197/300 Write block 1346 len 16384 data: 4 4 3 2
3967 198/300 Read block 2347 len 36864
3968 199/300 Read block 2793 len 32768
3969 200/300 Write block 3363 len 32768 data: 2 2 2 2 2 2 2 2
3970 201/300 Read block 3489 len 28672
3971 202/300 Read block 3166 len 24576
3972 203/300 Read block 3555 len 32768
3973 204/300 Read block 51 len 40960
3974 205/300 Write block 4245 len 28672 data: 3 3 2 2 2 2 2
3975 206/300 Read block 4487 len 8192
3976 207/300 Write block 1802 len 8192 data: 3 3
3977 208/300 Write block 3201 len 8192 data: 4 4
3978 209/300 Read block 596 len 36864
3979 210/300 Write block 3608 len 16384 data: 3 2 2 2
3980 211/300 Read block 2079 len 8192
3981 212/300 Write block 556 len 24576 data: 3 3 3 2 2 2
3982 213/300 Flush
3983 214/300 Read block 3924 len 32768
3984 215/300 Write block 2487 len 24576 data: 3 3 3 3 3 3
3985 216/300 Read block 4469 len 16384
3986 217/300 Write block 4857 len 36864 data: 2 2 2 2 2 2 2 2 2
3987 218/300 Read block 73 len 8192
3988 219/300 Flush
3989 220/300 Write block 486 len 24576 data: 2 2 2 2 2 2
3990 221/300 Read block 583 len 8192
3991 222/300 Write block 2861 len 36864 data: 3 2 2 2 2 2 2 2 2
3992 223/300 Write block 1291 len 16384 data: 2 2 3 3
3993 224/300 Write block 1694 len 28672 data: 2 2 2 2 2 2 2
3994 225/300 Flush
3995 226/300 Read block 1276 len 36864
3996 227/300 Write block 3663 len 40960 data: 2 2 2 2 2 2 3 3 3 3
3997 228/300 Write block 751 len 12288 data: 2 2 2
3998 229/300 Write block 4755 len 8192 data: 3 3
3999 230/300 Read block 743 len 20480
4000 231/300 Read block 3938 len 40960
4001 232/300 Read block 1995 len 16384
4002 233/300 Read block 4631 len 12288
4003 234/300 Write block 4611 len 16384 data: 3 3 3 2
4004 235/300 Write block 706 len 12288 data: 2 2 2
4005 236/300 Read block 67 len 12288
4006 237/300 Read block 2663 len 32768
4007 238/300 Write block 1304 len 32768 data: 2 3 3 3 2 2 2 2
4008 239/300 Write block 1570 len 40960 data: 2 3 3 4 4 4 3 3 3 3
4009 240/300 Flush
4010 241/300 Write block 3765 len 16384 data: 2 2 2 2
4011 242/300 Write block 499 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4012 243/300 Write block 3064 len 8192 data: 2 2
4013 244/300 Read block 3168 len 12288
4014 245/300 Write block 2760 len 12288 data: 5 5 4
4015 246/300 Write block 438 len 24576 data: 3 3 3 3 3 3
4016 247/300 Write block 3869 len 28672 data: 3 3 3 3 3 3 2
4017 248/300 Read block 4029 len 16384
4018 249/300 Read block 2038 len 16384
4019 250/300 Read block 1202 len 40960
4020 251/300 Write block 1951 len 4096 data: 2
4021 252/300 Write block 1153 len 12288 data: 3 3 2
4022 253/300 Read block 2419 len 36864
4023 254/300 Read block 3998 len 8192
4024 255/300 Read block 2874 len 40960
4025 256/300 Write block 3555 len 32768 data: 3 3 2 2 2 2 2 2
4026 257/300 Read block 3260 len 4096
4027 258/300 Write block 2226 len 36864 data: 2 2 2 2 2 2 2 2 2
4028 259/300 Read block 2991 len 28672
4029 260/300 Read block 1954 len 16384
4030 261/300 Flush
4031 262/300 Read block 2476 len 40960
4032 263/300 Flush
4033 264/300 Write block 3293 len 20480 data: 2 2 2 3 3
4034 265/300 Read block 1139 len 4096
4035 266/300 Write block 3770 len 16384 data: 2 2 2 3
4036 267/300 Read block 3182 len 32768
4037 268/300 Read block 1308 len 20480
4038 269/300 Read block 732 len 16384
4039 270/300 Flush
4040 271/300 Read block 4451 len 36864
4041 272/300 Read block 1445 len 16384
4042 273/300 Write block 4230 len 20480 data: 2 2 2 2 2
4043 274/300 Read block 743 len 24576
4044 275/300 Read block 4257 len 12288
4045 276/300 Write block 444 len 12288 data: 2 2 2
4046 277/300 Write block 1757 len 16384 data: 3 2 2 2
4047 278/300 Write block 3798 len 4096 data: 4
4048 279/300 Flush
4049 280/300 Write block 1485 len 36864 data: 2 2 2 3 3 2 2 2 2
4050 281/300 Read block 962 len 24576
4051 282/300 Read block 943 len 16384
4052 283/300 Write block 1821 len 12288 data: 2 2 2
4053 284/300 Read block 4582 len 12288
4054 285/300 Write block 1371 len 4096 data: 4
4055 286/300 Write block 2039 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4056 287/300 Write block 4279 len 16384 data: 2 2 2 3
4057 288/300 Write block 891 len 36864 data: 2 3 3 4 3 2 2 2 2
4058 289/300 Write block 1163 len 40960 data: 2 2 2 2 3 3 3 3 3 3
4059 290/300 Write block 4869 len 8192 data: 2 2
4060 291/300 Write block 3340 len 16384 data: 3 2 2 2
4061 292/300 Read block 2916 len 24576
4062 293/300 Read block 3389 len 20480
4063 294/300 Read block 164 len 4096
4064 295/300 Read block 1203 len 36864
4065 296/300 Read block 50 len 12288
4066 297/300 Write block 1761 len 32768 data: 3 3 3 3 3 3 3 3
4067 298/300 Flush
4068 299/300 Read block 3858 len 8192
4069 300/300 Read block 1029 len 4096
4070 301/300 Flush
4071 Main task has work to do, go find it
4072 got dsc Start(0)
4073 start 0
4074 Jan 15 05:19:51.966 INFO request completed, latency_us: 175, response_code: 204, uri: /start/cid/0, method: POST, req_id: 06d10583-1708-4d95-ae8e-637fab1ef977, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
4075 Replay: started 0, returned:Ok(())
4076 ----------------------------------------------------------------
4077 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4078 States: Offline Active Active
4079 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4080 1906 Acked Read 5 Sent Done Done false
4081 1907 Acked Read 4 Sent Done Done false
4082 1908 Acked Read 4 Sent Done Done false
4083 1909 Acked Read 5 Sent Done Done false
4084 1910 Acked Flush 0 Sent Done Done false
4085 1911 Acked Read 8 Sent Done Done false
4086 1912 Acked Write 4 Sent Done Done false
4087 1913 Acked Flush 0 Sent Done Done false
4088 1914 Acked Write 5 Sent Done Done false
4089 1915 Acked Write 3 Sent Done Done false
4090 1916 Acked Write 10 Sent Done Done false
4091 1917 Acked Read 10 Sent Done Done false
4092 1918 Acked Read 2 Sent Done Done false
4093 1919 Acked Write 6 Sent Done Done false
4094 1920 Acked Read 4 Sent Done Done false
4095 1921 Acked Write 10 Sent Done Done false
4096 1922 Acked Flush 0 Sent Done Done false
4097 1923 Acked Write 8 Sent Done Done false
4098 1924 Acked Read 3 Sent Done Done false
4099 1925 Acked Read 9 Sent Done Done false
4100 1926 Acked Read 9 Sent Done Done false
4101 1927 Acked Read 5 Sent Done Done false
4102 1928 Acked Read 4 Sent Done Done false
4103 1929 Acked Write 6 Sent Done Done false
4104 1930 Acked Write 10 Sent Done Done false
4105 1931 Acked Read 2 Sent Done Done false
4106 1932 Acked Read 2 Sent Done Done false
4107 1933 Acked Read 1 Sent Done Done false
4108 1934 Acked Read 10 Sent Done Done false
4109 1935 Acked Write 7 Sent Done Done false
4110 1936 Acked Write 9 Sent Done Done false
4111 1937 Acked Read 5 Sent Done Done false
4112 1938 Acked Read 2 Sent Done Done false
4113 1939 Acked Write 7 Sent Done Done false
4114 1940 Acked Read 2 Sent Done Done false
4115 1941 Acked Read 8 Sent Done Done false
4116 1942 Acked Read 1 Sent Done Done false
4117 1943 Acked Write 2 Sent Done Done false
4118 1944 Acked Write 10 Sent Done Done false
4119 1945 Acked Write 1 Sent Done Done false
4120 1946 Acked Write 2 Sent Done Done false
4121 1947 Acked Write 3 Sent Done Done false
4122 1948 Acked Read 6 Sent Done Done false
4123 1949 Acked Read 10 Sent Done Done false
4124 1950 Acked Flush 0 Sent Done Done false
4125 1951 Acked Read 7 Sent Done Done false
4126 1952 Acked Write 7 Sent Done Done false
4127 1953 Acked Flush 0 Sent Done Done false
4128 1954 Acked Write 1 Sent Done Done false
4129 1955 Acked Write 1 Sent Done Done false
4130 1956 Acked Read 5 Sent Done Done false
4131 1957 Acked Write 2 Sent Done Done false
4132 1958 Acked Read 10 Sent Done Done false
4133 1959 Acked Write 7 Sent Done Done false
4134 1960 Acked Write 7 Sent Done Done false
4135 1961 Acked Write 2 Sent Done Done false
4136 1962 Acked Read 2 Sent Done Done false
4137 1963 Acked Write 9 Sent Done Done false
4138 1964 Acked Write 6 Sent Done Done false
4139 1965 Acked Read 9 Sent Done Done false
4140 1966 Acked Read 7 Sent Done Done false
4141 1967 Acked Read 10 Sent Done Done false
4142 1968 Acked Read 2 Sent Done Done false
4143 1969 Acked Read 10 Sent Done Done false
4144 1970 Acked Write 2 Sent Done Done false
4145 1971 Acked Read 10 Sent Done Done false
4146 1972 Acked Write 7 Sent Done Done false
4147 1973 Acked Write 1 Sent Done Done false
4148 1974 Acked Write 9 Sent Done Done false
4149 1975 Acked Write 5 Sent Done Done false
4150 1976 Acked Write 6 Sent Done Done false
4151 1977 Acked Read 2 Sent Done Done false
4152 1978 Acked Read 3 Sent Done Done false
4153 1979 Acked Read 7 Sent Done Done false
4154 1980 Acked Read 2 Sent Done Done false
4155 1981 Acked Write 2 Sent Done Done false
4156 1982 Acked Read 7 Sent Done Done false
4157 1983 Acked Write 5 Sent Done Done false
4158 1984 Acked Read 2 Sent Done Done false
4159 1985 Acked Read 5 Sent Done Done false
4160 1986 Acked Read 6 Sent Done Done false
4161 1987 Acked Write 6 Sent Done Done false
4162 1988 Acked Read 4 Sent Done Done false
4163 1989 Acked Read 10 Sent Done Done false
4164 1990 Acked Write 3 Sent Done Done false
4165 1991 Acked Read 9 Sent Done Done false
4166 1992 Acked Write 10 Sent Done Done false
4167 1993 Acked Write 6 Sent Done Done false
4168 1994 Acked Write 6 Sent Done Done false
4169 1995 Acked Write 9 Sent Done Done false
4170 1996 Acked Read 4 Sent Done Done false
4171 1997 Acked Write 7 Sent Done Done false
4172 1998 Acked Read 3 Sent Done Done false
4173 1999 Acked Write 2 Sent Done Done false
4174 2000 Acked Read 2 Sent Done Done false
4175 2001 Acked Write 9 Sent Done Done false
4176 2002 Acked Read 6 Sent Done Done false
4177 2003 Acked Read 8 Sent Done Done false
4178 2004 Acked Write 6 Sent Done Done false
4179 2005 Acked Flush 0 Sent Done Done false
4180 2006 Acked Write 6 Sent Done Done false
4181 2007 Acked Write 10 Sent Done Done false
4182 2008 Acked Read 1 Sent Done Done false
4183 2009 Acked Write 2 Sent Done Done false
4184 2010 Acked Read 7 Sent Done Done false
4185 2011 Acked Read 8 Sent Done Done false
4186 2012 Acked Read 1 Sent Done Done false
4187 2013 Acked Read 5 Sent Done Done false
4188 2014 Acked Read 3 Sent Done Done false
4189 2015 Acked Write 2 Sent Done Done false
4190 2016 Acked Read 3 Sent Done Done false
4191 2017 Acked Read 4 Sent Done Done false
4192 2018 Acked Write 5 Sent Done Done false
4193 2019 Acked Read 9 Sent Done Done false
4194 2020 Acked Write 8 Sent Done Done false
4195 2021 Acked Read 9 Sent Done Done false
4196 2022 Acked Read 6 Sent Done Done false
4197 2023 Acked Write 3 Sent Done Done false
4198 2024 Acked Read 6 Sent Done Done false
4199 2025 Acked Flush 0 Sent Done Done false
4200 2026 Acked Read 10 Sent Done Done false
4201 2027 Acked Read 5 Sent Done Done false
4202 2028 Acked Flush 0 Sent Done Done false
4203 2029 Acked Write 9 Sent Done Done false
4204 2030 Acked Read 5 Sent Done Done false
4205 2031 Acked Flush 0 Sent Done Done false
4206 2032 Acked Write 2 Sent Done Done false
4207 2033 Acked Write 8 Sent Done Done false
4208 2034 Acked Read 2 Sent Done Done false
4209 2035 Acked Flush 0 Sent Done Done false
4210 2036 Acked Read 9 Sent Done Done false
4211 2037 Acked Read 6 Sent Done Done false
4212 2038 Acked Write 1 Sent Done Done false
4213 2039 Acked Read 6 Sent Done Done false
4214 2040 Acked Read 10 Sent Done Done false
4215 2041 Acked Write 9 Sent Done Done false
4216 2042 Acked Flush 0 Sent Done Done false
4217 2043 Acked Flush 0 Sent Done Done false
4218 2044 Acked Write 4 Sent Done Done false
4219 2045 Acked Write 5 Sent Done Done false
4220 2046 Acked Read 5 Sent Done Done false
4221 2047 Acked Write 9 Sent Done Done false
4222 2048 Acked Write 6 Sent Done Done false
4223 2049 Acked Read 10 Sent Done Done false
4224 2050 Acked Flush 0 Sent Done Done false
4225 2051 Acked Read 10 Sent Done Done false
4226 2052 Acked Read 1 Sent Done Done false
4227 2053 Acked Read 4 Sent Done Done false
4228 2054 Acked Write 3 Sent Done Done false
4229 2055 Acked Read 5 Sent Done Done false
4230 2056 Acked Write 10 Sent Done Done false
4231 2057 Acked Write 9 Sent Done Done false
4232 2058 Acked Read 3 Sent Done Done false
4233 2059 Acked Read 5 Sent Done Done false
4234 2060 Acked Write 6 Sent Done Done false
4235 2061 Acked Read 4 Sent Done Done false
4236 2062 Acked Read 5 Sent Done Done false
4237 2063 Acked Write 9 Sent Done Done false
4238 2064 Acked Read 1 Sent Done Done false
4239 2065 Acked Flush 0 Sent Done Done false
4240 2066 Acked Write 9 Sent Done Done false
4241 2067 Acked Read 1 Sent Done Done false
4242 2068 Acked Read 7 Sent Done Done false
4243 2069 Acked Write 5 Sent Done Done false
4244 2070 Acked Flush 0 Sent Done Done false
4245 2071 Acked Read 8 Sent Done Done false
4246 2072 Acked Write 7 Sent Done Done false
4247 2073 Acked Write 2 Sent Done Done false
4248 2074 Acked Write 3 Sent Done Done false
4249 2075 Acked Read 6 Sent Done Done false
4250 2076 Acked Read 2 Sent Done Done false
4251 2077 Acked Write 2 Sent Done Done false
4252 2078 Acked Write 10 Sent Done Done false
4253 2079 Acked Read 8 Sent Done Done false
4254 2080 Acked Read 6 Sent Done Done false
4255 2081 Acked Read 7 Sent Done Done false
4256 2082 Acked Write 2 Sent Done Done false
4257 2083 Acked Write 3 Sent Done Done false
4258 2084 Acked Write 10 Sent Done Done false
4259 2085 Acked Write 10 Sent Done Done false
4260 2086 Acked Write 4 Sent Done Done false
4261 2087 Acked Write 8 Sent Done Done false
4262 2088 Acked Flush 0 Sent Done Done false
4263 2089 Acked Write 1 Sent Done Done false
4264 2090 Acked Flush 0 Sent Done Done false
4265 2091 Acked Read 5 Sent Done Done false
4266 2092 Acked Write 3 Sent Done Done false
4267 2093 Acked Write 7 Sent Done Done false
4268 2094 Acked Read 10 Sent Done Done false
4269 2095 Acked Read 8 Sent Done Done false
4270 2096 Acked Read 7 Sent Done Done false
4271 2097 Acked Flush 0 Sent Done Done false
4272 2098 Acked Flush 0 Sent Done Done false
4273 2099 Acked Read 2 Sent Done Done false
4274 2100 Acked Read 4 Sent Done Done false
4275 2101 Acked Read 5 Sent Done Done false
4276 2102 Acked Write 4 Sent Done Done false
4277 2103 Acked Read 9 Sent Done Done false
4278 2104 Acked Read 8 Sent Done Done false
4279 2105 Acked Write 8 Sent Done Done false
4280 2106 Acked Read 7 Sent Done Done false
4281 2107 Acked Read 6 Sent Done Done false
4282 2108 Acked Read 8 Sent Done Done false
4283 2109 Acked Read 10 Sent Done Done false
4284 2110 Acked Write 7 Sent Done Done false
4285 2111 Acked Read 2 Sent Done Done false
4286 2112 Acked Write 2 Sent Done Done false
4287 2113 Acked Write 2 Sent Done Done false
4288 2114 Acked Read 9 Sent Done Done false
4289 2115 Acked Write 4 Sent Done Done false
4290 2116 Acked Read 2 Sent Done Done false
4291 2117 Acked Write 6 Sent Done Done false
4292 2118 Acked Flush 0 Sent Done Done false
4293 2119 Acked Read 8 Sent Done Done false
4294 2120 Acked Write 6 Sent Done Done false
4295 2121 Acked Read 4 Sent Done Done false
4296 2122 Acked Write 9 Sent Done Done false
4297 2123 Acked Read 2 Sent Done Done false
4298 2124 Acked Flush 0 Sent Done Done false
4299 2125 Acked Write 6 Sent Done Done false
4300 2126 Acked Read 2 Sent Done Done false
4301 2127 Acked Write 9 Sent Done Done false
4302 2128 Acked Write 4 Sent Done Done false
4303 2129 Acked Write 7 Sent Done Done false
4304 2130 Acked Flush 0 Sent Done Done false
4305 2131 Acked Read 9 Sent Done Done false
4306 2132 Acked Write 10 Sent Done Done false
4307 2133 Acked Write 3 Sent Done Done false
4308 2134 Acked Write 2 Sent Done Done false
4309 2135 Acked Read 5 Sent Done Done false
4310 2136 Acked Read 10 Sent Done Done false
4311 2137 Acked Read 4 Sent Done Done false
4312 2138 Acked Read 3 Sent Done Done false
4313 2139 Acked Write 4 Sent Done Done false
4314 2140 Acked Write 3 Sent Done Done false
4315 2141 Acked Read 3 Sent Done Done false
4316 2142 Acked Read 8 Sent Done Done false
4317 2143 Acked Write 8 Sent Done Done false
4318 2144 Acked Write 10 Sent Done Done false
4319 2145 Acked Flush 0 Sent Done Done false
4320 2146 Acked Write 4 Sent Done Done false
4321 2147 Acked Write 10 Sent Done Done false
4322 2148 Acked Write 2 Sent Done Done false
4323 2149 Acked Read 3 Sent Done Done false
4324 2150 Acked Write 3 Sent Done Done false
4325 2151 Acked Write 6 Sent Done Done false
4326 2152 Acked Write 7 Sent Done Done false
4327 2153 Acked Read 4 Sent Done Done false
4328 2154 Acked Read 4 Sent Done Done false
4329 2155 Acked Read 10 Sent Done Done false
4330 2156 Acked Write 1 Sent Done Done false
4331 2157 Acked Write 3 Sent Done Done false
4332 2158 Acked Read 9 Sent Done Done false
4333 2159 Acked Read 2 Sent Done Done false
4334 2160 Acked Read 10 Sent Done Done false
4335 2161 Acked Write 8 Sent Done Done false
4336 2162 Acked Read 1 Sent Done Done false
4337 2163 Acked Write 9 Sent Done Done false
4338 2164 Acked Read 7 Sent Done Done false
4339 2165 Acked Read 4 Sent Done Done false
4340 2166 Acked Flush 0 Sent Done Done false
4341 2167 Acked Read 10 Sent Done Done false
4342 2168 Acked Flush 0 Sent Done Done false
4343 2169 Acked Write 5 Sent Done Done false
4344 2170 Acked Read 1 Sent Done Done false
4345 2171 Acked Write 4 Sent Done Done false
4346 2172 Acked Read 8 Sent Done Done false
4347 2173 Acked Read 5 Sent Done Done false
4348 2174 Acked Read 4 Sent Done Done false
4349 2175 Acked Flush 0 Sent Done Done false
4350 2176 Acked Read 9 Sent Done Done false
4351 2177 Acked Read 4 Sent Done Done false
4352 2178 Acked Write 5 Sent Done Done false
4353 2179 Acked Read 6 Sent Done Done false
4354 2180 Acked Read 3 Sent Done Done false
4355 2181 Acked Write 3 Sent Done Done false
4356 2182 Acked Write 4 Sent Done Done false
4357 2183 Acked Write 1 Sent Done Done false
4358 2184 Acked Flush 0 Sent Done Done false
4359 2185 Acked Write 9 Sent Done Done false
4360 2186 Acked Read 6 Sent Done Done false
4361 2187 Acked Read 4 Sent Done Done false
4362 2188 Acked Write 3 Sent Done Done false
4363 2189 Acked Read 3 Sent Done Done false
4364 2190 Acked Write 1 Sent Done Done false
4365 2191 Acked Write 10 Sent Done Done false
4366 2192 Acked Write 4 Sent Done Done false
4367 2193 Acked Write 9 Sent Done Done false
4368 2194 Acked Write 10 Sent Done Done false
4369 2195 Acked Write 2 Sent Done Done false
4370 2196 Acked Write 4 Sent Done Done false
4371 2197 Acked Read 6 Sent Done Done false
4372 2198 Acked Read 5 Sent Done Done false
4373 2199 Acked Read 1 Sent Done Done false
4374 2200 Acked Read 9 Sent Done Done false
4375 2201 Acked Read 3 Sent Done Done false
4376 2202 Acked Write 8 Sent Done Done false
4377 2203 Acked Flush 0 Sent Done Done false
4378 2204 Acked Read 2 Sent Done Done false
4379 2205 Acked Read 1 Sent Done Done false
4380 2206 Acked Flush 0 Sent Done Done false
4381 STATES DS:0 DS:1 DS:2 TOTAL
4382 Sent 301 0 0 301
4383 Done 0 301 301 602
4384 Skipped 0 0 0 0
4385 Error 0 0 0 0
4386 Last Flush: 1905 2206 2206
4387 Downstairs last five completed: 1905 1904 1903 1902 1901
4388 Upstairs last five completed: 2206 2205 2204 2203 2202
4389 CLIENT: Up:0 ds:301 act:2
4390 [8810] got start action
4391 [8810] I am going to restart
4392 Starting downstairs at port 8810
4393 Make output file at "/tmp/dsc/downstairs-8810.txt"
4394 [8810][0] reports Starting
4395 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1206)
4396 [8810][0] reports Running
4397 ----------------------------------------------------------------
4398 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4399 States: Offline Active Active
4400 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4401 1906 Acked Read 5 Sent Done Done false
4402 1907 Acked Read 4 Sent Done Done false
4403 1908 Acked Read 4 Sent Done Done false
4404 1909 Acked Read 5 Sent Done Done false
4405 1910 Acked Flush 0 Sent Done Done false
4406 1911 Acked Read 8 Sent Done Done false
4407 1912 Acked Write 4 Sent Done Done false
4408 1913 Acked Flush 0 Sent Done Done false
4409 1914 Acked Write 5 Sent Done Done false
4410 1915 Acked Write 3 Sent Done Done false
4411 1916 Acked Write 10 Sent Done Done false
4412 1917 Acked Read 10 Sent Done Done false
4413 1918 Acked Read 2 Sent Done Done false
4414 1919 Acked Write 6 Sent Done Done false
4415 1920 Acked Read 4 Sent Done Done false
4416 1921 Acked Write 10 Sent Done Done false
4417 1922 Acked Flush 0 Sent Done Done false
4418 1923 Acked Write 8 Sent Done Done false
4419 1924 Acked Read 3 Sent Done Done false
4420 1925 Acked Read 9 Sent Done Done false
4421 1926 Acked Read 9 Sent Done Done false
4422 1927 Acked Read 5 Sent Done Done false
4423 1928 Acked Read 4 Sent Done Done false
4424 1929 Acked Write 6 Sent Done Done false
4425 1930 Acked Write 10 Sent Done Done false
4426 1931 Acked Read 2 Sent Done Done false
4427 1932 Acked Read 2 Sent Done Done false
4428 1933 Acked Read 1 Sent Done Done false
4429 1934 Acked Read 10 Sent Done Done false
4430 1935 Acked Write 7 Sent Done Done false
4431 1936 Acked Write 9 Sent Done Done false
4432 1937 Acked Read 5 Sent Done Done false
4433 1938 Acked Read 2 Sent Done Done false
4434 1939 Acked Write 7 Sent Done Done false
4435 1940 Acked Read 2 Sent Done Done false
4436 1941 Acked Read 8 Sent Done Done false
4437 1942 Acked Read 1 Sent Done Done false
4438 1943 Acked Write 2 Sent Done Done false
4439 1944 Acked Write 10 Sent Done Done false
4440 1945 Acked Write 1 Sent Done Done false
4441 1946 Acked Write 2 Sent Done Done false
4442 1947 Acked Write 3 Sent Done Done false
4443 1948 Acked Read 6 Sent Done Done false
4444 1949 Acked Read 10 Sent Done Done false
4445 1950 Acked Flush 0 Sent Done Done false
4446 1951 Acked Read 7 Sent Done Done false
4447 1952 Acked Write 7 Sent Done Done false
4448 1953 Acked Flush 0 Sent Done Done false
4449 1954 Acked Write 1 Sent Done Done false
4450 1955 Acked Write 1 Sent Done Done false
4451 1956 Acked Read 5 Sent Done Done false
4452 1957 Acked Write 2 Sent Done Done false
4453 1958 Acked Read 10 Sent Done Done false
4454 1959 Acked Write 7 Sent Done Done false
4455 1960 Acked Write 7 Sent Done Done false
4456 1961 Acked Write 2 Sent Done Done false
4457 1962 Acked Read 2 Sent Done Done false
4458 1963 Acked Write 9 Sent Done Done false
4459 1964 Acked Write 6 Sent Done Done false
4460 1965 Acked Read 9 Sent Done Done false
4461 1966 Acked Read 7 Sent Done Done false
4462 1967 Acked Read 10 Sent Done Done false
4463 1968 Acked Read 2 Sent Done Done false
4464 1969 Acked Read 10 Sent Done Done false
4465 1970 Acked Write 2 Sent Done Done false
4466 1971 Acked Read 10 Sent Done Done false
4467 1972 Acked Write 7 Sent Done Done false
4468 1973 Acked Write 1 Sent Done Done false
4469 1974 Acked Write 9 Sent Done Done false
4470 1975 Acked Write 5 Sent Done Done false
4471 1976 Acked Write 6 Sent Done Done false
4472 1977 Acked Read 2 Sent Done Done false
4473 1978 Acked Read 3 Sent Done Done false
4474 1979 Acked Read 7 Sent Done Done false
4475 1980 Acked Read 2 Sent Done Done false
4476 1981 Acked Write 2 Sent Done Done false
4477 1982 Acked Read 7 Sent Done Done false
4478 1983 Acked Write 5 Sent Done Done false
4479 1984 Acked Read 2 Sent Done Done false
4480 1985 Acked Read 5 Sent Done Done false
4481 1986 Acked Read 6 Sent Done Done false
4482 1987 Acked Write 6 Sent Done Done false
4483 1988 Acked Read 4 Sent Done Done false
4484 1989 Acked Read 10 Sent Done Done false
4485 1990 Acked Write 3 Sent Done Done false
4486 1991 Acked Read 9 Sent Done Done false
4487 1992 Acked Write 10 Sent Done Done false
4488 1993 Acked Write 6 Sent Done Done false
4489 1994 Acked Write 6 Sent Done Done false
4490 1995 Acked Write 9 Sent Done Done false
4491 1996 Acked Read 4 Sent Done Done false
4492 1997 Acked Write 7 Sent Done Done false
4493 1998 Acked Read 3 Sent Done Done false
4494 1999 Acked Write 2 Sent Done Done false
4495 2000 Acked Read 2 Sent Done Done false
4496 2001 Acked Write 9 Sent Done Done false
4497 2002 Acked Read 6 Sent Done Done false
4498 2003 Acked Read 8 Sent Done Done false
4499 2004 Acked Write 6 Sent Done Done false
4500 2005 Acked Flush 0 Sent Done Done false
4501 2006 Acked Write 6 Sent Done Done false
4502 2007 Acked Write 10 Sent Done Done false
4503 2008 Acked Read 1 Sent Done Done false
4504 2009 Acked Write 2 Sent Done Done false
4505 2010 Acked Read 7 Sent Done Done false
4506 2011 Acked Read 8 Sent Done Done false
4507 2012 Acked Read 1 Sent Done Done false
4508 2013 Acked Read 5 Sent Done Done false
4509 2014 Acked Read 3 Sent Done Done false
4510 2015 Acked Write 2 Sent Done Done false
4511 2016 Acked Read 3 Sent Done Done false
4512 2017 Acked Read 4 Sent Done Done false
4513 2018 Acked Write 5 Sent Done Done false
4514 2019 Acked Read 9 Sent Done Done false
4515 2020 Acked Write 8 Sent Done Done false
4516 2021 Acked Read 9 Sent Done Done false
4517 2022 Acked Read 6 Sent Done Done false
4518 2023 Acked Write 3 Sent Done Done false
4519 2024 Acked Read 6 Sent Done Done false
4520 2025 Acked Flush 0 Sent Done Done false
4521 2026 Acked Read 10 Sent Done Done false
4522 2027 Acked Read 5 Sent Done Done false
4523 2028 Acked Flush 0 Sent Done Done false
4524 2029 Acked Write 9 Sent Done Done false
4525 2030 Acked Read 5 Sent Done Done false
4526 2031 Acked Flush 0 Sent Done Done false
4527 2032 Acked Write 2 Sent Done Done false
4528 2033 Acked Write 8 Sent Done Done false
4529 2034 Acked Read 2 Sent Done Done false
4530 2035 Acked Flush 0 Sent Done Done false
4531 2036 Acked Read 9 Sent Done Done false
4532 2037 Acked Read 6 Sent Done Done false
4533 2038 Acked Write 1 Sent Done Done false
4534 2039 Acked Read 6 Sent Done Done false
4535 2040 Acked Read 10 Sent Done Done false
4536 2041 Acked Write 9 Sent Done Done false
4537 2042 Acked Flush 0 Sent Done Done false
4538 2043 Acked Flush 0 Sent Done Done false
4539 2044 Acked Write 4 Sent Done Done false
4540 2045 Acked Write 5 Sent Done Done false
4541 2046 Acked Read 5 Sent Done Done false
4542 2047 Acked Write 9 Sent Done Done false
4543 2048 Acked Write 6 Sent Done Done false
4544 2049 Acked Read 10 Sent Done Done false
4545 2050 Acked Flush 0 Sent Done Done false
4546 2051 Acked Read 10 Sent Done Done false
4547 2052 Acked Read 1 Sent Done Done false
4548 2053 Acked Read 4 Sent Done Done false
4549 2054 Acked Write 3 Sent Done Done false
4550 2055 Acked Read 5 Sent Done Done false
4551 2056 Acked Write 10 Sent Done Done false
4552 2057 Acked Write 9 Sent Done Done false
4553 2058 Acked Read 3 Sent Done Done false
4554 2059 Acked Read 5 Sent Done Done false
4555 2060 Acked Write 6 Sent Done Done false
4556 2061 Acked Read 4 Sent Done Done false
4557 2062 Acked Read 5 Sent Done Done false
4558 2063 Acked Write 9 Sent Done Done false
4559 2064 Acked Read 1 Sent Done Done false
4560 2065 Acked Flush 0 Sent Done Done false
4561 2066 Acked Write 9 Sent Done Done false
4562 2067 Acked Read 1 Sent Done Done false
4563 2068 Acked Read 7 Sent Done Done false
4564 2069 Acked Write 5 Sent Done Done false
4565 2070 Acked Flush 0 Sent Done Done false
4566 2071 Acked Read 8 Sent Done Done false
4567 2072 Acked Write 7 Sent Done Done false
4568 2073 Acked Write 2 Sent Done Done false
4569 2074 Acked Write 3 Sent Done Done false
4570 2075 Acked Read 6 Sent Done Done false
4571 2076 Acked Read 2 Sent Done Done false
4572 2077 Acked Write 2 Sent Done Done false
4573 2078 Acked Write 10 Sent Done Done false
4574 2079 Acked Read 8 Sent Done Done false
4575 2080 Acked Read 6 Sent Done Done false
4576 2081 Acked Read 7 Sent Done Done false
4577 2082 Acked Write 2 Sent Done Done false
4578 2083 Acked Write 3 Sent Done Done false
4579 2084 Acked Write 10 Sent Done Done false
4580 2085 Acked Write 10 Sent Done Done false
4581 2086 Acked Write 4 Sent Done Done false
4582 2087 Acked Write 8 Sent Done Done false
4583 2088 Acked Flush 0 Sent Done Done false
4584 2089 Acked Write 1 Sent Done Done false
4585 2090 Acked Flush 0 Sent Done Done false
4586 2091 Acked Read 5 Sent Done Done false
4587 2092 Acked Write 3 Sent Done Done false
4588 2093 Acked Write 7 Sent Done Done false
4589 2094 Acked Read 10 Sent Done Done false
4590 2095 Acked Read 8 Sent Done Done false
4591 2096 Acked Read 7 Sent Done Done false
4592 2097 Acked Flush 0 Sent Done Done false
4593 2098 Acked Flush 0 Sent Done Done false
4594 2099 Acked Read 2 Sent Done Done false
4595 2100 Acked Read 4 Sent Done Done false
4596 2101 Acked Read 5 Sent Done Done false
4597 2102 Acked Write 4 Sent Done Done false
4598 2103 Acked Read 9 Sent Done Done false
4599 2104 Acked Read 8 Sent Done Done false
4600 2105 Acked Write 8 Sent Done Done false
4601 2106 Acked Read 7 Sent Done Done false
4602 2107 Acked Read 6 Sent Done Done false
4603 2108 Acked Read 8 Sent Done Done false
4604 2109 Acked Read 10 Sent Done Done false
4605 2110 Acked Write 7 Sent Done Done false
4606 2111 Acked Read 2 Sent Done Done false
4607 2112 Acked Write 2 Sent Done Done false
4608 2113 Acked Write 2 Sent Done Done false
4609 2114 Acked Read 9 Sent Done Done false
4610 2115 Acked Write 4 Sent Done Done false
4611 2116 Acked Read 2 Sent Done Done false
4612 2117 Acked Write 6 Sent Done Done false
4613 2118 Acked Flush 0 Sent Done Done false
4614 2119 Acked Read 8 Sent Done Done false
4615 2120 Acked Write 6 Sent Done Done false
4616 2121 Acked Read 4 Sent Done Done false
4617 2122 Acked Write 9 Sent Done Done false
4618 2123 Acked Read 2 Sent Done Done false
4619 2124 Acked Flush 0 Sent Done Done false
4620 2125 Acked Write 6 Sent Done Done false
4621 2126 Acked Read 2 Sent Done Done false
4622 2127 Acked Write 9 Sent Done Done false
4623 2128 Acked Write 4 Sent Done Done false
4624 2129 Acked Write 7 Sent Done Done false
4625 2130 Acked Flush 0 Sent Done Done false
4626 2131 Acked Read 9 Sent Done Done false
4627 2132 Acked Write 10 Sent Done Done false
4628 2133 Acked Write 3 Sent Done Done false
4629 2134 Acked Write 2 Sent Done Done false
4630 2135 Acked Read 5 Sent Done Done false
4631 2136 Acked Read 10 Sent Done Done false
4632 2137 Acked Read 4 Sent Done Done false
4633 2138 Acked Read 3 Sent Done Done false
4634 2139 Acked Write 4 Sent Done Done false
4635 2140 Acked Write 3 Sent Done Done false
4636 2141 Acked Read 3 Sent Done Done false
4637 2142 Acked Read 8 Sent Done Done false
4638 2143 Acked Write 8 Sent Done Done false
4639 2144 Acked Write 10 Sent Done Done false
4640 2145 Acked Flush 0 Sent Done Done false
4641 2146 Acked Write 4 Sent Done Done false
4642 2147 Acked Write 10 Sent Done Done false
4643 2148 Acked Write 2 Sent Done Done false
4644 2149 Acked Read 3 Sent Done Done false
4645 2150 Acked Write 3 Sent Done Done false
4646 2151 Acked Write 6 Sent Done Done false
4647 2152 Acked Write 7 Sent Done Done false
4648 2153 Acked Read 4 Sent Done Done false
4649 2154 Acked Read 4 Sent Done Done false
4650 2155 Acked Read 10 Sent Done Done false
4651 2156 Acked Write 1 Sent Done Done false
4652 2157 Acked Write 3 Sent Done Done false
4653 2158 Acked Read 9 Sent Done Done false
4654 2159 Acked Read 2 Sent Done Done false
4655 2160 Acked Read 10 Sent Done Done false
4656 2161 Acked Write 8 Sent Done Done false
4657 2162 Acked Read 1 Sent Done Done false
4658 2163 Acked Write 9 Sent Done Done false
4659 2164 Acked Read 7 Sent Done Done false
4660 2165 Acked Read 4 Sent Done Done false
4661 2166 Acked Flush 0 Sent Done Done false
4662 2167 Acked Read 10 Sent Done Done false
4663 2168 Acked Flush 0 Sent Done Done false
4664 2169 Acked Write 5 Sent Done Done false
4665 2170 Acked Read 1 Sent Done Done false
4666 2171 Acked Write 4 Sent Done Done false
4667 2172 Acked Read 8 Sent Done Done false
4668 2173 Acked Read 5 Sent Done Done false
4669 2174 Acked Read 4 Sent Done Done false
4670 2175 Acked Flush 0 Sent Done Done false
4671 2176 Acked Read 9 Sent Done Done false
4672 2177 Acked Read 4 Sent Done Done false
4673 2178 Acked Write 5 Sent Done Done false
4674 2179 Acked Read 6 Sent Done Done false
4675 2180 Acked Read 3 Sent Done Done false
4676 2181 Acked Write 3 Sent Done Done false
4677 2182 Acked Write 4 Sent Done Done false
4678 2183 Acked Write 1 Sent Done Done false
4679 2184 Acked Flush 0 Sent Done Done false
4680 2185 Acked Write 9 Sent Done Done false
4681 2186 Acked Read 6 Sent Done Done false
4682 2187 Acked Read 4 Sent Done Done false
4683 2188 Acked Write 3 Sent Done Done false
4684 2189 Acked Read 3 Sent Done Done false
4685 2190 Acked Write 1 Sent Done Done false
4686 2191 Acked Write 10 Sent Done Done false
4687 2192 Acked Write 4 Sent Done Done false
4688 2193 Acked Write 9 Sent Done Done false
4689 2194 Acked Write 10 Sent Done Done false
4690 2195 Acked Write 2 Sent Done Done false
4691 2196 Acked Write 4 Sent Done Done false
4692 2197 Acked Read 6 Sent Done Done false
4693 2198 Acked Read 5 Sent Done Done false
4694 2199 Acked Read 1 Sent Done Done false
4695 2200 Acked Read 9 Sent Done Done false
4696 2201 Acked Read 3 Sent Done Done false
4697 2202 Acked Write 8 Sent Done Done false
4698 2203 Acked Flush 0 Sent Done Done false
4699 2204 Acked Read 2 Sent Done Done false
4700 2205 Acked Read 1 Sent Done Done false
4701 2206 Acked Flush 0 Sent Done Done false
4702 STATES DS:0 DS:1 DS:2 TOTAL
4703 Sent 301 0 0 301
4704 Done 0 301 301 602
4705 Skipped 0 0 0 0
4706 Error 0 0 0 0
4707 Last Flush: 1905 2206 2206
4708 Downstairs last five completed: 1905 1904 1903 1902 1901
4709 Upstairs last five completed: 2206 2205 2204 2203 2202
4710 CLIENT: Up:0 ds:301 act:2
47112025-01-15T05:19:57.736ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47122025-01-15T05:19:57.736ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33466) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47132025-01-15T05:19:57.737ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47142025-01-15T05:19:57.738ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47152025-01-15T05:19:57.738ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47162025-01-15T05:19:57.738ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47172025-01-15T05:19:57.738ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47182025-01-15T05:19:57.738ZINFOcrucible: [0] client re-new 301 jobs since flush 1905 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47192025-01-15T05:19:57.738ZINFOcrucible: [0] Replayed 301 jobs since flush: 1905 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
4720 ----------------------------------------------------------------
4721 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4722 Downstairs last five completed: 2206 2205 2204 2203 2202
4723 Upstairs last five completed: 2206 2205 2204 2203 2202
4724 CLIENT: Up:0 ds:0 act:3
4725 Replay: All jobs finished
4726 Main task has work to do, go find it
4727 got dsc Stop(1)
4728 stop 1
4729 Jan 15 05:19:59.971 INFO request completed, latency_us: 299[8820] Got stop action so:false kr:false
4730 , response_code: 204, uri: /stop/cid/1, method: POST, req_id: 3ccfcd32-c710-44c9-9f7e-47034e29daff, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
4731 Jan 15 05:19:59.972 INFO request completed, latency_us: 143, response_code: 200, uri: {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:19:59.972406043Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
4732 /state/cid/1, method: GET, req_id: 6221b387-4b74-4db3-93ed-5784df3b32c9, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
47332025-01-15T05:19:59.972ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47342025-01-15T05:19:59.972ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
47352025-01-15T05:19:59.972ZINFOcrucible: client task is exiting = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
4736 [8820] Exited with: ExitStatus(unix_wait_status(9))
4737 [8820][1] reports Exit
4738 Jan 15 05:20:03.974 INFO request completed, latency_us: 335, response_code: 200, uri: /state/cid/1, method: GET, req_id: 43dd79c8-3b58-422b-a19e-77d4d0ff1d18, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
4739 001/300 Write block 2558 len 28672 data: 2 2 2 2 3 3 3
4740 002/300 Write block 289 len 36864 data: 3 2 2 2 2 2 4 4 3
4741 003/300 Write block 2376 len 8192 data: 2 2
4742 004/300 Read block 3494 len 40960
4743 005/300 Read block 1547 len 24576
4744 006/300 Write block 1888 len 32768 data: 2 2 2 2 2 3 3 3
4745 007/300 Write block 1783 len 24576 data: 2 2 2 3 3 3
4746 008/300 Write block 240 len 4096 data: 2
4747 009/300 Read block 517 len 40960
4748 010/300 Flush
4749 011/300 Read block 2984 len 20480
4750 012/300 Read block 4054 len 40960
4751 013/300 Write block 2143 len 12288 data: 2 2 2
4752 014/300 Write block 3243 len 4096 data: 2
4753 015/300 Read block 3850 len 20480
4754 016/300 Read block 922 len 40960
4755 017/300 Flush
4756 018/300 Write block 2149 len 32768 data: 2 2 2 2 2 2 2 2
4757 019/300 Write block 2409 len 4096 data: 2
4758 020/300 Read block 2503 len 28672
4759 021/300 Read block 4862 len 16384
4760 022/300 Write block 3933 len 28672 data: 2 2 2 2 3 4 4
4761 023/300 Read block 1500 len 8192
4762 024/300 Write block 3035 len 32768 data: 3 3 3 2 2 2 3 3
4763 025/300 Write block 1368 len 36864 data: 3 4 4 5 4 3 2 2 2
4764 026/300 Write block 3657 len 24576 data: 3 3 2 2 2 2
4765 027/300 Write block 2642 len 12288 data: 2 2 2
4766 028/300 Write block 3939 len 4096 data: 5
4767 029/300 Write block 254 len 20480 data: 3 3 3 3 3
4768 030/300 Write block 1841 len 36864 data: 2 3 3 3 3 2 2 2 2
4769 031/300 Read block 3879 len 8192
4770 032/300 Read block 327 len 16384
4771 033/300 Write block 4869 len 40960 data: 3 3 2 3 3 3 3 3 3 3
4772 034/300 Read block 1348 len 28672
4773 035/300 Write block 1459 len 8192 data: 2 2
4774 036/300 Read block 1430 len 24576
4775 037/300 Write block 4517 len 16384 data: 2 2 3 4
4776 038/300 Write block 1597 len 8192 data: 3 3
4777 039/300 Write block 3762 len 20480 data: 3 3 2 3 3
4778 040/300 Read block 1210 len 20480
4779 041/300 Write block 3850 len 24576 data: 2 2 2 2 3 4
4780 042/300 Read block 2363 len 32768
4781 043/300 Read block 4529 len 24576
4782 044/300 Write block 2262 len 32768 data: 3 3 2 2 2 2 2 2
4783 045/300 Write block 74 len 28672 data: 2 2 2 2 2 2 2
4784 046/300 Read block 3350 len 4096
4785 047/300 Read block 4940 len 28672
4786 048/300 Read block 3349 len 28672
4787 049/300 Write block 2861 len 36864 data: 4 3 3 3 3 3 3 3 3
4788 050/300 Read block 2262 len 24576
4789 051/300 Read block 2867 len 24576
4790 052/300 Write block 1912 len 24576 data: 3 3 4 4 4 3
4791 053/300 Read block 1186 len 40960
4792 054/300 Read block 11 len 8192
4793 055/300 Write block 2166 len 28672 data: 2 3 3 3 4 4 4
4794 056/300 Read block 2324 len 4096
4795 057/300 Read block 3752 len 16384
4796 058/300 Read block 3838 len 28672
4797 059/300 Write block 3275 len 24576 data: 3 3 3 3 3 2
4798 060/300 Read block 3041 len 8192
4799 061/300 Flush
4800 062/300 Write block 4253 len 36864 data: 2 2 2 2 3 3 3 3 3
4801 063/300 Read block 1788 len 16384
4802 064/300 Write block 2020 len 12288 data: 2 3 4
4803 065/300 Write block 3334 len 20480 data: 3 3 3 3 3
4804 066/300 Read block 1843 len 28672
4805 067/300 Write block 1479 len 8192 data: 2 2
4806 068/300 Write block 3718 len 36864 data: 3 2 2 2 2 2 2 2 2
4807 069/300 Write block 4750 len 28672 data: 2 3 3 3 3 4 4
4808 070/300 Write block 2407 len 24576 data: 2 2 3 2 2 2
4809 071/300 Read block 1292 len 16384
4810 072/300 Write block 1347 len 40960 data: 5 4 3 3 3 3 3 2 2 2
4811 073/300 Read block 4204 len 32768
4812 074/300 Write block 3650 len 16384 data: 4 4 3 3
4813 075/300 Read block 755 len 40960
4814 076/300 Read block 3138 len 32768
4815 077/300 Write block 64 len 24576 data: 2 2 2 2 2 2
4816 078/300 Read block 4297 len 12288
4817 079/300 Write block 182 len 4096 data: 2
4818 080/300 Write block 2633 len 40960 data: 2 2 2 2 2 2 2 2 2 3
4819 081/300 Write block 869 len 28672 data: 3 3 3 3 3 2 2
4820 082/300 Read block 2753 len 24576
4821 083/300 Write block 1371 len 20480 data: 6 5 4 3 3
4822 084/300 Read block 1867 len 28672
4823 085/300 Read block 97 len 12288
4824 086/300 Write block 4959 len 4096 data: 2
4825 087/300 Flush
4826 088/300 Flush
4827 089/300 Read block 722 len 8192
4828 090/300 Write block 3545 len 28672 data: 2 2 2 2 2 2 2
4829 091/300 Read block 614 len 40960
4830 092/300 Write block 3245 len 36864 data: 3 2 2 2 2 4 4 4 3
4831 093/300 Read block 1757 len 12288
4832 094/300 Read block 447 len 16384
4833 095/300 Read block 2487 len 4096
4834 096/300 Write block 4798 len 4096 data: 3
4835 097/300 Write block 824 len 4096 data: 2
4836 098/300 Write block 2096 len 8192 data: 2 2
4837 099/300 Flush
4838 100/300 Write block 231 len 16384 data: 2 2 2 2
4839 101/300 Read block 2542 len 28672
4840 102/300 Read block 3807 len 16384
4841 103/300 Write block 2551 len 12288 data: 2 2 2
4842 104/300 Read block 1509 len 32768
4843 105/300 Write block 4359 len 28672 data: 2 2 2 2 2 2 2
4844 106/300 Write block 440 len 28672 data: 4 4 4 4 3 3 3
4845 107/300 Write block 2011 len 40960 data: 3 2 2 2 2 2 2 2 2 3
4846 108/300 Flush
4847 109/300 Write block 2808 len 4096 data: 3
4848 110/300 Write block 3918 len 4096 data: 4
4849 111/300 Read block 4753 len 40960
4850 112/300 Write block 2586 len 40960 data: 2 2 3 3 3 3 3 3 3 2
4851 113/300 Write block 251 len 8192 data: 2 2
4852 114/300 Read block 1470 len 4096
4853 115/300 Write block 2428 len 28672 data: 2 2 2 2 3 3 2
4854 116/300 Write block 3237 len 28672 data: 3 3 2 2 2 2 3
4855 117/300 Read block 190 len 20480
4856 118/300 Read block 1251 len 4096
4857 119/300 Write block 3428 len 4096 data: 2
4858 120/300 Read block 1016 len 24576
4859 121/300 Read block 1801 len 32768
4860 122/300 Read block 4619 len 4096
4861 123/300 Flush
4862 124/300 Read block 4885 len 20480
4863 125/300 Read block 1296 len 40960
4864 126/300 Flush
4865 127/300 Read block 2163 len 28672
4866 128/300 Write block 63 len 40960 data: 2 3 3 3 3 3 3 2 2 2
4867 129/300 Read block 1796 len 36864
4868 130/300 Read block 3857 len 12288
4869 131/300 Write block 1769 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4870 132/300 Write block 1861 len 8192 data: 3 3
4871 133/300 Write block 1033 len 32768 data: 2 2 2 2 2 2 2 2
4872 134/300 Read block 973 len 32768
4873 135/300 Flush
4874 136/300 Write block 4612 len 20480 data: 4 4 3 2 2
4875 137/300 Write block 2877 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4876 138/300 Read block 3828 len 12288
4877 139/300 Flush
4878 140/300 Write block 2674 len 28672 data: 2 2 2 2 2 2 2
4879 141/300 Read block 2342 len 20480
4880 142/300 Write block 3321 len 32768 data: 2 2 2 2 2 2 2 2
4881 143/300 Read block 3463 len 4096
4882 144/300 Write block 3707 len 4096 data: 2
4883 145/300 Write block 2111 len 36864 data: 5 3 3 2 2 2 2 2 2
4884 146/300 Read block 4056 len 16384
4885 147/300 Write block 1968 len 16384 data: 2 2 2 2
4886 148/300 Flush
4887 149/300 Read block 3634 len 12288
4888 150/300 Read block 2560 len 8192
4889 151/300 Read block 2890 len 12288
4890 152/300 Write block 2079 len 40960 data: 2 2 2 2 2 3 3 4 3 3
4891 153/300 Write block 234 len 40960 data: 3 2 2 2 2 2 3 2 2 2
4892 154/300 Read block 515 len 16384
4893 155/300 Write block 489 len 8192 data: 3 3
4894 156/300 Flush
4895 157/300 Read block 4446 len 32768
4896 158/300 Read block 400 len 12288
4897 159/300 Read block 2097 len 16384
4898 160/300 Read block 4642 len 40960
4899 161/300 Write block 1141 len 8192 data: 2 2
4900 162/300 Read block 4147 len 4096
4901 163/300 Write block 1399 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4902 164/300 Write block 594 len 28672 data: 3 3 2 2 2 3 3
4903 165/300 Write block 446 len 16384 data: 4 2 2 2
4904 166/300 Write block 2043 len 28672 data: 3 3 3 3 3 3 2
4905 167/300 Flush
4906 168/300 Read block 3074 len 12288
4907 169/300 Read block 105 len 8192
4908 170/300 Write block 822 len 20480 data: 2 2 3 2 2
4909 171/300 Flush
4910 172/300 Read block 418 len 24576
4911 173/300 Read block 1174 len 4096
4912 174/300 Write block 3243 len 4096 data: 4
4913 175/300 Write block 147 len 24576 data: 2 2 2 2 2 2
4914 176/300 Write block 4754 len 16384 data: 4 5 5 2
4915 177/300 Read block 1398 len 8192
4916 178/300 Write block 1564 len 12288 data: 3 2 2
4917 179/300 Read block 3843 len 8192
4918 180/300 Read block 527 len 28672
4919 181/300 Write block 441 len 8192 data: 5 5
4920 182/300 Read block 2168 len 36864
4921 183/300 Read block 1434 len 40960
4922 184/300 Write block 353 len 24576 data: 3 3 3 3 4 4
4923 185/300 Read block 3161 len 24576
4924 186/300 Write block 2342 len 16384 data: 2 2 2 3
4925 187/300 Read block 2414 len 28672
4926 188/300 Flush
4927 189/300 Read block 4794 len 4096
4928 190/300 Write block 2656 len 16384 data: 2 2 2 2
4929 191/300 Read block 4795 len 40960
4930 192/300 Read block 2212 len 16384
4931 193/300 Read block 3495 len 12288
4932 194/300 Write block 4005 len 16384 data: 3 2 2 2
4933 195/300 Read block 3277 len 28672
4934 196/300 Write block 4259 len 40960 data: 4 4 4 3 4 4 3 2 2 2
4935 197/300 Read block 4901 len 36864
4936 198/300 Read block 2422 len 36864
4937 199/300 Write block 258 len 32768 data: 4 2 2 2 2 2 2 2
4938 200/300 Read block 4461 len 20480
4939 201/300 Write block 4140 len 8192 data: 2 2
4940 202/300 Read block 3165 len 28672
4941 203/300 Read block 57 len 24576
4942 204/300 Write block 3954 len 12288 data: 3 3 3
4943 205/300 Write block 1637 len 32768 data: 3 4 3 3 3 3 3 3
4944 206/300 Read block 1662 len 24576
4945 207/300 Read block 2069 len 24576
4946 208/300 Write block 2550 len 8192 data: 2 3
4947 209/300 Read block 2447 len 40960
4948 210/300 Write block 4371 len 36864 data: 2 2 2 2 2 2 3 3 3
4949 211/300 Read block 2760 len 36864
4950 212/300 Write block 2782 len 32768 data: 3 3 3 2 2 2 2 2
4951 213/300 Read block 2682 len 40960
4952 214/300 Write block 3515 len 20480 data: 2 2 2 2 2
4953 215/300 Write block 3327 len 4096 data: 3
4954 216/300 Read block 3823 len 36864
4955 217/300 Read block 4565 len 24576
4956 218/300 Read block 3227 len 36864
4957 219/300 Write block 4097 len 8192 data: 2 2
4958 220/300 Write block 4277 len 40960 data: 2 2 3 3 3 4 3 3 3 3
4959 221/300 Write block 1874 len 32768 data: 4 4 4 5 5 4 3 3
4960 222/300 Read block 2536 len 36864
4961 223/300 Write block 4744 len 8192 data: 2 2
4962 224/300 Read block 749 len 16384
4963 225/300 Read block 3016 len 40960
4964 226/300 Read block 226 len 12288
4965 227/300 Flush
4966 228/300 Read block 1000 len 8192
4967 229/300 Read block 3641 len 24576
4968 230/300 Read block 3523 len 8192
4969 231/300 Read block 2395 len 12288
4970 232/300 Read block 3877 len 12288
4971 233/300 Read block 994 len 20480
4972 234/300 Flush
4973 235/300 Read block 4968 len 4096
4974 236/300 Write block 220 len 32768 data: 2 2 2 2 2 2 2 2
4975 237/300 Write block 4143 len 40960 data: 3 3 3 3 3 3 3 2 2 2
4976 238/300 Read block 2847 len 16384
4977 239/300 Read block 296 len 4096
4978 240/300 Read block 711 len 12288
4979 241/300 Read block 3793 len 4096
4980 242/300 Write block 4862 len 36864 data: 3 3 3 3 2 2 2 4 4
4981 243/300 Read block 4216 len 28672
4982 244/300 Read block 3676 len 40960
4983 245/300 Read block 2120 len 20480
4984 246/300 Read block 1186 len 36864
4985 247/300 Write block 615 len 32768 data: 2 2 2 2 2 2 2 2
4986 248/300 Read block 4346 len 32768
4987 249/300 Read block 3073 len 32768
4988 250/300 Write block 4932 len 16384 data: 2 2 2 2
4989 251/300 Read block 2128 len 40960
4990 252/300 Flush
4991 253/300 Read block 578 len 20480
4992 254/300 Read block 983 len 28672
4993 255/300 Write block 2643 len 20480 data: 3 3 2 3 3
4994 256/300 Write block 3856 len 28672 data: 4 3 3 3 2 2 2
4995 257/300 Read block 1038 len 32768
4996 258/300 Write block 4081 len 24576 data: 2 2 2 2 2 2
4997 259/300 Read block 1034 len 24576
4998 260/300 Write block 1545 len 12288 data: 3 2 2
4999 261/300 Read block 89 len 4096
5000 262/300 Write block 2562 len 12288 data: 4 4 4
5001 263/300 Flush
5002 264/300 Read block 4922 len 32768
5003 265/300 Read block 779 len 4096
5004 266/300 Read block 2431 len 16384
5005 267/300 Read block 3247 len 40960
5006 268/300 Read block 1459 len 40960
5007 269/300 Read block 1033 len 40960
5008 270/300 Write block 4026 len 20480 data: 2 2 2 3 3
5009 271/300 Read block 2424 len 4096
5010 272/300 Write block 2357 len 12288 data: 2 2 2
5011 273/300 Read block 3720 len 12288
5012 274/300 Write block 2984 len 12288 data: 5 5 5
5013 275/300 Write block 804 len 8192 data: 2 2
5014 276/300 Write block 4685 len 12288 data: 2 2 2
5015 277/300 Read block 2944 len 32768
5016 278/300 Read block 721 len 4096
5017 279/300 Write block 4408 len 20480 data: 3 2 3 3 3
5018 280/300 Flush
5019 281/300 Write block 4192 len 24576 data: 3 3 3 2 3 3
5020 282/300 Write block 2448 len 24576 data: 4 4 4 3 3 3
5021 283/300 Read block 3192 len 28672
5022 284/300 Read block 3786 len 8192
5023 285/300 Read block 114 len 16384
5024 286/300 Read block 4056 len 40960
5025 287/300 Write block 1870 len 20480 data: 3 3 4 4 5
5026 288/300 Write block 3472 len 36864 data: 3 3 3 2 2 2 2 3 3
5027 289/300 Read block 3456 len 36864
5028 290/300 Read block 4913 len 4096
5029 291/300 Read block 4749 len 4096
5030 292/300 Write block 575 len 32768 data: 2 2 2 2 2 2 2 2
5031 293/300 Read block 4602 len 8192
5032 294/300 Write block 3120 len 4096 data: 3
5033 295/300 Read block 3471 len 20480
5034 296/300 Read block 3175 len 8192
5035 297/300 Read block 261 len 20480
5036 298/300 Write block 4117 len 24576 data: 3 3 3 3 2 2
5037 299/300 Write block 1466 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5038 300/300 Read block 370 len 36864
5039 301/300 Read block 4397 len 32768
5040 Main task has work to do, go find it
5041 got dsc Start(1)
5042 start 1
5043 Jan 15 05:20:04.205 INFO request completed, latency_us: 183, response_code: 204, uri: /start/cid/1, method: POST, req_id: acfc46f1-30eb-4754-be4c-8d536adfe2b4, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
5044 Replay: started 1, returned:Ok(())
5045 ----------------------------------------------------------------
5046 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5047 States: Active Offline Active
5048 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5049 2207 Acked Write 7 Done Sent Done false
5050 2208 Acked Write 9 Done Sent Done false
5051 2209 Acked Write 2 Done Sent Done false
5052 2210 Acked Read 10 Done Sent Done false
5053 2211 Acked Read 6 Done Sent Done false
5054 2212 Acked Write 8 Done Sent Done false
5055 2213 Acked Write 6 Done Sent Done false
5056 2214 Acked Write 1 Done Sent Done false
5057 2215 Acked Read 10 Done Sent Done false
5058 2216 Acked Flush 0 Done Sent Done false
5059 2217 Acked Read 5 Done Sent Done false
5060 2218 Acked Read 10 Done Sent Done false
5061 2219 Acked Write 3 Done Sent Done false
5062 2220 Acked Write 1 Done Sent Done false
5063 2221 Acked Read 5 Done Sent Done false
5064 2222 Acked Read 10 Done Sent Done false
5065 2223 Acked Flush 0 Done Sent Done false
5066 2224 Acked Write 8 Done Sent Done false
5067 2225 Acked Write 1 Done Sent Done false
5068 2226 Acked Read 7 Done Sent Done false
5069 2227 Acked Read 4 Done Sent Done false
5070 2228 Acked Write 7 Done Sent Done false
5071 2229 Acked Read 2 Done Sent Done false
5072 2230 Acked Write 8 Done Sent Done false
5073 2231 Acked Write 9 Done Sent Done false
5074 2232 Acked Write 6 Done Sent Done false
5075 2233 Acked Write 3 Done Sent Done false
5076 2234 Acked Write 1 Done Sent Done false
5077 2235 Acked Write 5 Done Sent Done false
5078 2236 Acked Write 9 Done Sent Done false
5079 2237 Acked Read 2 Done Sent Done false
5080 2238 Acked Read 4 Done Sent Done false
5081 2239 Acked Write 10 Done Sent Done false
5082 2240 Acked Read 7 Done Sent Done false
5083 2241 Acked Write 2 Done Sent Done false
5084 2242 Acked Read 6 Done Sent Done false
5085 2243 Acked Write 4 Done Sent Done false
5086 2244 Acked Write 2 Done Sent Done false
5087 2245 Acked Write 5 Done Sent Done false
5088 2246 Acked Read 5 Done Sent Done false
5089 2247 Acked Write 6 Done Sent Done false
5090 2248 Acked Read 8 Done Sent Done false
5091 2249 Acked Read 6 Done Sent Done false
5092 2250 Acked Write 8 Done Sent Done false
5093 2251 Acked Write 7 Done Sent Done false
5094 2252 Acked Read 1 Done Sent Done false
5095 2253 Acked Read 7 Done Sent Done false
5096 2254 Acked Read 7 Done Sent Done false
5097 2255 Acked Write 9 Done Sent Done false
5098 2256 Acked Read 6 Done Sent Done false
5099 2257 Acked Read 6 Done Sent Done false
5100 2258 Acked Write 6 Done Sent Done false
5101 2259 Acked Read 10 Done Sent Done false
5102 2260 Acked Read 2 Done Sent Done false
5103 2261 Acked Write 7 Done Sent Done false
5104 2262 Acked Read 1 Done Sent Done false
5105 2263 Acked Read 4 Done Sent Done false
5106 2264 Acked Read 7 Done Sent Done false
5107 2265 Acked Write 6 Done Sent Done false
5108 2266 Acked Read 2 Done Sent Done false
5109 2267 Acked Flush 0 Done Sent Done false
5110 2268 Acked Write 9 Done Sent Done false
5111 2269 Acked Read 4 Done Sent Done false
5112 2270 Acked Write 3 Done Sent Done false
5113 2271 Acked Write 5 Done Sent Done false
5114 2272 Acked Read 7 Done Sent Done false
5115 2273 Acked Write 2 Done Sent Done false
5116 2274 Acked Write 9 Done Sent Done false
5117 2275 Acked Write 7 Done Sent Done false
5118 2276 Acked Write 6 Done Sent Done false
5119 2277 Acked Read 4 Done Sent Done false
5120 2278 Acked Write 10 Done Sent Done false
5121 2279 Acked Read 8 Done Sent Done false
5122 2280 Acked Write 4 Done Sent Done false
5123 2281 Acked Read 10 Done Sent Done false
5124 2282 Acked Read 8 Done Sent Done false
5125 2283 Acked Write 6 Done Sent Done false
5126 2284 Acked Read 3 Done Sent Done false
5127 2285 Acked Write 1 Done Sent Done false
5128 2286 Acked Write 10 Done Sent Done false
5129 2287 Acked Write 7 Done Sent Done false
5130 2288 Acked Read 6 Done Sent Done false
5131 2289 Acked Write 5 Done Sent Done false
5132 2290 Acked Read 7 Done Sent Done false
5133 2291 Acked Read 3 Done Sent Done false
5134 2292 Acked Write 1 Done Sent Done false
5135 2293 Acked Flush 0 Done Sent Done false
5136 2294 Acked Flush 0 Done Sent Done false
5137 2295 Acked Read 2 Done Sent Done false
5138 2296 Acked Write 7 Done Sent Done false
5139 2297 Acked Read 10 Done Sent Done false
5140 2298 Acked Write 9 Done Sent Done false
5141 2299 Acked Read 3 Done Sent Done false
5142 2300 Acked Read 4 Done Sent Done false
5143 2301 Acked Read 1 Done Sent Done false
5144 2302 Acked Write 1 Done Sent Done false
5145 2303 Acked Write 1 Done Sent Done false
5146 2304 Acked Write 2 Done Sent Done false
5147 2305 Acked Flush 0 Done Sent Done false
5148 2306 Acked Write 4 Done Sent Done false
5149 2307 Acked Read 7 Done Sent Done false
5150 2308 Acked Read 4 Done Sent Done false
5151 2309 Acked Write 3 Done Sent Done false
5152 2310 Acked Read 8 Done Sent Done false
5153 2311 Acked Write 7 Done Sent Done false
5154 2312 Acked Write 7 Done Sent Done false
5155 2313 Acked Write 10 Done Sent Done false
5156 2314 Acked Flush 0 Done Sent Done false
5157 2315 Acked Write 1 Done Sent Done false
5158 2316 Acked Write 1 Done Sent Done false
5159 2317 Acked Read 10 Done Sent Done false
5160 2318 Acked Write 10 Done Sent Done false
5161 2319 Acked Write 2 Done Sent Done false
5162 2320 Acked Read 1 Done Sent Done false
5163 2321 Acked Write 7 Done Sent Done false
5164 2322 Acked Write 7 Done Sent Done false
5165 2323 Acked Read 5 Done Sent Done false
5166 2324 Acked Read 1 Done Sent Done false
5167 2325 Acked Write 1 Done Sent Done false
5168 2326 Acked Read 6 Done Sent Done false
5169 2327 Acked Read 8 Done Sent Done false
5170 2328 Acked Read 1 Done Sent Done false
5171 2329 Acked Flush 0 Done Sent Done false
5172 2330 Acked Read 5 Done Sent Done false
5173 2331 Acked Read 10 Done Sent Done false
5174 2332 Acked Flush 0 Done Sent Done false
5175 2333 Acked Read 7 Done Sent Done false
5176 2334 Acked Write 10 Done Sent Done false
5177 2335 Acked Read 9 Done Sent Done false
5178 2336 Acked Read 3 Done Sent Done false
5179 2337 Acked Write 10 Done Sent Done false
5180 2338 Acked Write 2 Done Sent Done false
5181 2339 Acked Write 8 Done Sent Done false
5182 2340 Acked Read 8 Done Sent Done false
5183 2341 Acked Flush 0 Done Sent Done false
5184 2342 Acked Write 5 Done Sent Done false
5185 2343 Acked Write 10 Done Sent Done false
5186 2344 Acked Read 3 Done Sent Done false
5187 2345 Acked Flush 0 Done Sent Done false
5188 2346 Acked Write 7 Done Sent Done false
5189 2347 Acked Read 5 Done Sent Done false
5190 2348 Acked Write 8 Done Sent Done false
5191 2349 Acked Read 1 Done Sent Done false
5192 2350 Acked Write 1 Done Sent Done false
5193 2351 Acked Write 9 Done Sent Done false
5194 2352 Acked Read 4 Done Sent Done false
5195 2353 Acked Write 4 Done Sent Done false
5196 2354 Acked Flush 0 Done Sent Done false
5197 2355 Acked Read 3 Done Sent Done false
5198 2356 Acked Read 2 Done Sent Done false
5199 2357 Acked Read 3 Done Sent Done false
5200 2358 Acked Write 10 Done Sent Done false
5201 2359 Acked Write 10 Done Sent Done false
5202 2360 Acked Read 4 Done Sent Done false
5203 2361 Acked Write 2 Done Sent Done false
5204 2362 Acked Flush 0 Done Sent Done false
5205 2363 Acked Read 8 Done Sent Done false
5206 2364 Acked Read 3 Done Sent Done false
5207 2365 Acked Read 4 Done Sent Done false
5208 2366 Acked Read 10 Done Sent Done false
5209 2367 Acked Write 2 Done Sent Done false
5210 2368 Acked Read 1 Done Sent Done false
5211 2369 Acked Write 10 Done Sent Done false
5212 2370 Acked Write 7 Done Sent Done false
5213 2371 Acked Write 4 Done Sent Done false
5214 2372 Acked Write 7 Done Sent Done false
5215 2373 Acked Flush 0 Done Sent Done false
5216 2374 Acked Read 3 Done Sent Done false
5217 2375 Acked Read 2 Done Sent Done false
5218 2376 Acked Write 5 Done Sent Done false
5219 2377 Acked Flush 0 Done Sent Done false
5220 2378 Acked Read 6 Done Sent Done false
5221 2379 Acked Read 1 Done Sent Done false
5222 2380 Acked Write 1 Done Sent Done false
5223 2381 Acked Write 6 Done Sent Done false
5224 2382 Acked Write 4 Done Sent Done false
5225 2383 Acked Read 2 Done Sent Done false
5226 2384 Acked Write 3 Done Sent Done false
5227 2385 Acked Read 2 Done Sent Done false
5228 2386 Acked Read 7 Done Sent Done false
5229 2387 Acked Write 2 Done Sent Done false
5230 2388 Acked Read 9 Done Sent Done false
5231 2389 Acked Read 10 Done Sent Done false
5232 2390 Acked Write 6 Done Sent Done false
5233 2391 Acked Read 6 Done Sent Done false
5234 2392 Acked Write 4 Done Sent Done false
5235 2393 Acked Read 7 Done Sent Done false
5236 2394 Acked Flush 0 Done Sent Done false
5237 2395 Acked Read 1 Done Sent Done false
5238 2396 Acked Write 4 Done Sent Done false
5239 2397 Acked Read 10 Done Sent Done false
5240 2398 Acked Read 4 Done Sent Done false
5241 2399 Acked Read 3 Done Sent Done false
5242 2400 Acked Write 4 Done Sent Done false
5243 2401 Acked Read 7 Done Sent Done false
5244 2402 Acked Write 10 Done Sent Done false
5245 2403 Acked Read 9 Done Sent Done false
5246 2404 Acked Read 9 Done Sent Done false
5247 2405 Acked Write 8 Done Sent Done false
5248 2406 Acked Read 5 Done Sent Done false
5249 2407 Acked Write 2 Done Sent Done false
5250 2408 Acked Read 7 Done Sent Done false
5251 2409 Acked Read 6 Done Sent Done false
5252 2410 Acked Write 3 Done Sent Done false
5253 2411 Acked Write 8 Done Sent Done false
5254 2412 Acked Read 6 Done Sent Done false
5255 2413 Acked Read 6 Done Sent Done false
5256 2414 Acked Write 2 Done Sent Done false
5257 2415 Acked Read 10 Done Sent Done false
5258 2416 Acked Write 9 Done Sent Done false
5259 2417 Acked Read 9 Done Sent Done false
5260 2418 Acked Write 8 Done Sent Done false
5261 2419 Acked Read 10 Done Sent Done false
5262 2420 Acked Write 5 Done Sent Done false
5263 2421 Acked Write 1 Done Sent Done false
5264 2422 Acked Read 9 Done Sent Done false
5265 2423 Acked Read 6 Done Sent Done false
5266 2424 Acked Read 9 Done Sent Done false
5267 2425 Acked Write 2 Done Sent Done false
5268 2426 Acked Write 10 Done Sent Done false
5269 2427 Acked Write 8 Done Sent Done false
5270 2428 Acked Read 9 Done Sent Done false
5271 2429 Acked Write 2 Done Sent Done false
5272 2430 Acked Read 4 Done Sent Done false
5273 2431 Acked Read 10 Done Sent Done false
5274 2432 Acked Read 3 Done Sent Done false
5275 2433 Acked Flush 0 Done Sent Done false
5276 2434 Acked Read 2 Done Sent Done false
5277 2435 Acked Read 6 Done Sent Done false
5278 2436 Acked Read 2 Done Sent Done false
5279 2437 Acked Read 3 Done Sent Done false
5280 2438 Acked Read 3 Done Sent Done false
5281 2439 Acked Read 5 Done Sent Done false
5282 2440 Acked Flush 0 Done Sent Done false
5283 2441 Acked Read 1 Done Sent Done false
5284 2442 Acked Write 8 Done Sent Done false
5285 2443 Acked Write 10 Done Sent Done false
5286 2444 Acked Read 4 Done Sent Done false
5287 2445 Acked Read 1 Done Sent Done false
5288 2446 Acked Read 3 Done Sent Done false
5289 2447 Acked Read 1 Done Sent Done false
5290 2448 Acked Write 9 Done Sent Done false
5291 2449 Acked Read 7 Done Sent Done false
5292 2450 Acked Read 10 Done Sent Done false
5293 2451 Acked Read 5 Done Sent Done false
5294 2452 Acked Read 9 Done Sent Done false
5295 2453 Acked Write 8 Done Sent Done false
5296 2454 Acked Read 8 Done Sent Done false
5297 2455 Acked Read 8 Done Sent Done false
5298 2456 Acked Write 4 Done Sent Done false
5299 2457 Acked Read 10 Done Sent Done false
5300 2458 Acked Flush 0 Done Sent Done false
5301 2459 Acked Read 5 Done Sent Done false
5302 2460 Acked Read 7 Done Sent Done false
5303 2461 Acked Write 5 Done Sent Done false
5304 2462 Acked Write 7 Done Sent Done false
5305 2463 Acked Read 8 Done Sent Done false
5306 2464 Acked Write 6 Done Sent Done false
5307 2465 Acked Read 6 Done Sent Done false
5308 2466 Acked Write 3 Done Sent Done false
5309 2467 Acked Read 1 Done Sent Done false
5310 2468 Acked Write 3 Done Sent Done false
5311 2469 Acked Flush 0 Done Sent Done false
5312 2470 Acked Read 8 Done Sent Done false
5313 2471 Acked Read 1 Done Sent Done false
5314 2472 Acked Read 4 Done Sent Done false
5315 2473 Acked Read 10 Done Sent Done false
5316 2474 Acked Read 10 Done Sent Done false
5317 2475 Acked Read 10 Done Sent Done false
5318 2476 Acked Write 5 Done Sent Done false
5319 2477 Acked Read 1 Done Sent Done false
5320 2478 Acked Write 3 Done Sent Done false
5321 2479 Acked Read 3 Done Sent Done false
5322 2480 Acked Write 3 Done Sent Done false
5323 2481 Acked Write 2 Done Sent Done false
5324 2482 Acked Write 3 Done Sent Done false
5325 2483 Acked Read 8 Done Sent Done false
5326 2484 Acked Read 1 Done Sent Done false
5327 2485 Acked Write 5 Done Sent Done false
5328 2486 Acked Flush 0 Done Sent Done false
5329 2487 Acked Write 6 Done Sent Done false
5330 2488 Acked Write 6 Done Sent Done false
5331 2489 Acked Read 7 Done Sent Done false
5332 2490 Acked Read 2 Done Sent Done false
5333 2491 Acked Read 4 Done Sent Done false
5334 2492 Acked Read 10 Done Sent Done false
5335 2493 Acked Write 5 Done Sent Done false
5336 2494 Acked Write 9 Done Sent Done false
5337 2495 Acked Read 9 Done Sent Done false
5338 2496 Acked Read 1 Done Sent Done false
5339 2497 Acked Read 1 Done Sent Done false
5340 2498 Acked Write 8 Done Sent Done false
5341 2499 Acked Read 2 Done Sent Done false
5342 2500 Acked Write 1 Done Sent Done false
5343 2501 Acked Read 5 Done Sent Done false
5344 2502 Acked Read 2 Done Sent Done false
5345 2503 Acked Read 5 Done Sent Done false
5346 2504 Acked Write 6 Done Sent Done false
5347 2505 Acked Write 10 Done Sent Done false
5348 2506 Acked Read 9 Done Sent Done false
5349 2507 Acked Read 8 Done Sent Done false
5350 STATES DS:0 DS:1 DS:2 TOTAL
5351 Sent 0 301 0 301
5352 Done 301 0 301 602
5353 Skipped 0 0 0 0
5354 Error 0 0 0 0
5355 Last Flush: 2486 2206 2486
5356 Downstairs last five completed: 2206 2205 2204 2203 2202
5357 Upstairs last five completed: 2507 2506 2505 2504 2503
5358 CLIENT: Up:0 ds:301 act:2
5359 [8820] got start action
5360 [8820] I am going to restart
5361 Starting downstairs at port 8820
5362 Make output file at "/tmp/dsc/downstairs-8820.txt"
5363 [8820][1] reports Starting
5364 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1207)
5365 [8820][1] reports Running
5366 ----------------------------------------------------------------
5367 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5368 States: Active Offline Active
5369 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5370 2207 Acked Write 7 Done Sent Done false
5371 2208 Acked Write 9 Done Sent Done false
5372 2209 Acked Write 2 Done Sent Done false
5373 2210 Acked Read 10 Done Sent Done false
5374 2211 Acked Read 6 Done Sent Done false
5375 2212 Acked Write 8 Done Sent Done false
5376 2213 Acked Write 6 Done Sent Done false
5377 2214 Acked Write 1 Done Sent Done false
5378 2215 Acked Read 10 Done Sent Done false
5379 2216 Acked Flush 0 Done Sent Done false
5380 2217 Acked Read 5 Done Sent Done false
5381 2218 Acked Read 10 Done Sent Done false
5382 2219 Acked Write 3 Done Sent Done false
5383 2220 Acked Write 1 Done Sent Done false
5384 2221 Acked Read 5 Done Sent Done false
5385 2222 Acked Read 10 Done Sent Done false
5386 2223 Acked Flush 0 Done Sent Done false
5387 2224 Acked Write 8 Done Sent Done false
5388 2225 Acked Write 1 Done Sent Done false
5389 2226 Acked Read 7 Done Sent Done false
5390 2227 Acked Read 4 Done Sent Done false
5391 2228 Acked Write 7 Done Sent Done false
5392 2229 Acked Read 2 Done Sent Done false
5393 2230 Acked Write 8 Done Sent Done false
5394 2231 Acked Write 9 Done Sent Done false
5395 2232 Acked Write 6 Done Sent Done false
5396 2233 Acked Write 3 Done Sent Done false
5397 2234 Acked Write 1 Done Sent Done false
5398 2235 Acked Write 5 Done Sent Done false
5399 2236 Acked Write 9 Done Sent Done false
5400 2237 Acked Read 2 Done Sent Done false
5401 2238 Acked Read 4 Done Sent Done false
5402 2239 Acked Write 10 Done Sent Done false
5403 2240 Acked Read 7 Done Sent Done false
5404 2241 Acked Write 2 Done Sent Done false
5405 2242 Acked Read 6 Done Sent Done false
5406 2243 Acked Write 4 Done Sent Done false
5407 2244 Acked Write 2 Done Sent Done false
5408 2245 Acked Write 5 Done Sent Done false
5409 2246 Acked Read 5 Done Sent Done false
5410 2247 Acked Write 6 Done Sent Done false
5411 2248 Acked Read 8 Done Sent Done false
5412 2249 Acked Read 6 Done Sent Done false
5413 2250 Acked Write 8 Done Sent Done false
5414 2251 Acked Write 7 Done Sent Done false
5415 2252 Acked Read 1 Done Sent Done false
5416 2253 Acked Read 7 Done Sent Done false
5417 2254 Acked Read 7 Done Sent Done false
5418 2255 Acked Write 9 Done Sent Done false
5419 2256 Acked Read 6 Done Sent Done false
5420 2257 Acked Read 6 Done Sent Done false
5421 2258 Acked Write 6 Done Sent Done false
5422 2259 Acked Read 10 Done Sent Done false
5423 2260 Acked Read 2 Done Sent Done false
5424 2261 Acked Write 7 Done Sent Done false
5425 2262 Acked Read 1 Done Sent Done false
5426 2263 Acked Read 4 Done Sent Done false
5427 2264 Acked Read 7 Done Sent Done false
5428 2265 Acked Write 6 Done Sent Done false
5429 2266 Acked Read 2 Done Sent Done false
5430 2267 Acked Flush 0 Done Sent Done false
5431 2268 Acked Write 9 Done Sent Done false
5432 2269 Acked Read 4 Done Sent Done false
5433 2270 Acked Write 3 Done Sent Done false
5434 2271 Acked Write 5 Done Sent Done false
5435 2272 Acked Read 7 Done Sent Done false
5436 2273 Acked Write 2 Done Sent Done false
5437 2274 Acked Write 9 Done Sent Done false
5438 2275 Acked Write 7 Done Sent Done false
5439 2276 Acked Write 6 Done Sent Done false
5440 2277 Acked Read 4 Done Sent Done false
5441 2278 Acked Write 10 Done Sent Done false
5442 2279 Acked Read 8 Done Sent Done false
5443 2280 Acked Write 4 Done Sent Done false
5444 2281 Acked Read 10 Done Sent Done false
5445 2282 Acked Read 8 Done Sent Done false
5446 2283 Acked Write 6 Done Sent Done false
5447 2284 Acked Read 3 Done Sent Done false
5448 2285 Acked Write 1 Done Sent Done false
5449 2286 Acked Write 10 Done Sent Done false
5450 2287 Acked Write 7 Done Sent Done false
5451 2288 Acked Read 6 Done Sent Done false
5452 2289 Acked Write 5 Done Sent Done false
5453 2290 Acked Read 7 Done Sent Done false
5454 2291 Acked Read 3 Done Sent Done false
5455 2292 Acked Write 1 Done Sent Done false
5456 2293 Acked Flush 0 Done Sent Done false
5457 2294 Acked Flush 0 Done Sent Done false
5458 2295 Acked Read 2 Done Sent Done false
5459 2296 Acked Write 7 Done Sent Done false
5460 2297 Acked Read 10 Done Sent Done false
5461 2298 Acked Write 9 Done Sent Done false
5462 2299 Acked Read 3 Done Sent Done false
5463 2300 Acked Read 4 Done Sent Done false
5464 2301 Acked Read 1 Done Sent Done false
5465 2302 Acked Write 1 Done Sent Done false
5466 2303 Acked Write 1 Done Sent Done false
5467 2304 Acked Write 2 Done Sent Done false
5468 2305 Acked Flush 0 Done Sent Done false
5469 2306 Acked Write 4 Done Sent Done false
5470 2307 Acked Read 7 Done Sent Done false
5471 2308 Acked Read 4 Done Sent Done false
5472 2309 Acked Write 3 Done Sent Done false
5473 2310 Acked Read 8 Done Sent Done false
5474 2311 Acked Write 7 Done Sent Done false
5475 2312 Acked Write 7 Done Sent Done false
5476 2313 Acked Write 10 Done Sent Done false
5477 2314 Acked Flush 0 Done Sent Done false
5478 2315 Acked Write 1 Done Sent Done false
5479 2316 Acked Write 1 Done Sent Done false
5480 2317 Acked Read 10 Done Sent Done false
5481 2318 Acked Write 10 Done Sent Done false
5482 2319 Acked Write 2 Done Sent Done false
5483 2320 Acked Read 1 Done Sent Done false
5484 2321 Acked Write 7 Done Sent Done false
5485 2322 Acked Write 7 Done Sent Done false
5486 2323 Acked Read 5 Done Sent Done false
5487 2324 Acked Read 1 Done Sent Done false
5488 2325 Acked Write 1 Done Sent Done false
5489 2326 Acked Read 6 Done Sent Done false
5490 2327 Acked Read 8 Done Sent Done false
5491 2328 Acked Read 1 Done Sent Done false
5492 2329 Acked Flush 0 Done Sent Done false
5493 2330 Acked Read 5 Done Sent Done false
5494 2331 Acked Read 10 Done Sent Done false
5495 2332 Acked Flush 0 Done Sent Done false
5496 2333 Acked Read 7 Done Sent Done false
5497 2334 Acked Write 10 Done Sent Done false
5498 2335 Acked Read 9 Done Sent Done false
5499 2336 Acked Read 3 Done Sent Done false
5500 2337 Acked Write 10 Done Sent Done false
5501 2338 Acked Write 2 Done Sent Done false
5502 2339 Acked Write 8 Done Sent Done false
5503 2340 Acked Read 8 Done Sent Done false
5504 2341 Acked Flush 0 Done Sent Done false
5505 2342 Acked Write 5 Done Sent Done false
5506 2343 Acked Write 10 Done Sent Done false
5507 2344 Acked Read 3 Done Sent Done false
5508 2345 Acked Flush 0 Done Sent Done false
5509 2346 Acked Write 7 Done Sent Done false
5510 2347 Acked Read 5 Done Sent Done false
5511 2348 Acked Write 8 Done Sent Done false
5512 2349 Acked Read 1 Done Sent Done false
5513 2350 Acked Write 1 Done Sent Done false
5514 2351 Acked Write 9 Done Sent Done false
5515 2352 Acked Read 4 Done Sent Done false
5516 2353 Acked Write 4 Done Sent Done false
5517 2354 Acked Flush 0 Done Sent Done false
5518 2355 Acked Read 3 Done Sent Done false
5519 2356 Acked Read 2 Done Sent Done false
5520 2357 Acked Read 3 Done Sent Done false
5521 2358 Acked Write 10 Done Sent Done false
5522 2359 Acked Write 10 Done Sent Done false
5523 2360 Acked Read 4 Done Sent Done false
5524 2361 Acked Write 2 Done Sent Done false
5525 2362 Acked Flush 0 Done Sent Done false
5526 2363 Acked Read 8 Done Sent Done false
5527 2364 Acked Read 3 Done Sent Done false
5528 2365 Acked Read 4 Done Sent Done false
5529 2366 Acked Read 10 Done Sent Done false
5530 2367 Acked Write 2 Done Sent Done false
5531 2368 Acked Read 1 Done Sent Done false
5532 2369 Acked Write 10 Done Sent Done false
5533 2370 Acked Write 7 Done Sent Done false
5534 2371 Acked Write 4 Done Sent Done false
5535 2372 Acked Write 7 Done Sent Done false
5536 2373 Acked Flush 0 Done Sent Done false
5537 2374 Acked Read 3 Done Sent Done false
5538 2375 Acked Read 2 Done Sent Done false
5539 2376 Acked Write 5 Done Sent Done false
5540 2377 Acked Flush 0 Done Sent Done false
5541 2378 Acked Read 6 Done Sent Done false
5542 2379 Acked Read 1 Done Sent Done false
5543 2380 Acked Write 1 Done Sent Done false
5544 2381 Acked Write 6 Done Sent Done false
5545 2382 Acked Write 4 Done Sent Done false
5546 2383 Acked Read 2 Done Sent Done false
5547 2384 Acked Write 3 Done Sent Done false
5548 2385 Acked Read 2 Done Sent Done false
5549 2386 Acked Read 7 Done Sent Done false
5550 2387 Acked Write 2 Done Sent Done false
5551 2388 Acked Read 9 Done Sent Done false
5552 2389 Acked Read 10 Done Sent Done false
5553 2390 Acked Write 6 Done Sent Done false
5554 2391 Acked Read 6 Done Sent Done false
5555 2392 Acked Write 4 Done Sent Done false
5556 2393 Acked Read 7 Done Sent Done false
5557 2394 Acked Flush 0 Done Sent Done false
5558 2395 Acked Read 1 Done Sent Done false
5559 2396 Acked Write 4 Done Sent Done false
5560 2397 Acked Read 10 Done Sent Done false
5561 2398 Acked Read 4 Done Sent Done false
5562 2399 Acked Read 3 Done Sent Done false
5563 2400 Acked Write 4 Done Sent Done false
5564 2401 Acked Read 7 Done Sent Done false
5565 2402 Acked Write 10 Done Sent Done false
5566 2403 Acked Read 9 Done Sent Done false
5567 2404 Acked Read 9 Done Sent Done false
5568 2405 Acked Write 8 Done Sent Done false
5569 2406 Acked Read 5 Done Sent Done false
5570 2407 Acked Write 2 Done Sent Done false
5571 2408 Acked Read 7 Done Sent Done false
5572 2409 Acked Read 6 Done Sent Done false
5573 2410 Acked Write 3 Done Sent Done false
5574 2411 Acked Write 8 Done Sent Done false
5575 2412 Acked Read 6 Done Sent Done false
5576 2413 Acked Read 6 Done Sent Done false
5577 2414 Acked Write 2 Done Sent Done false
5578 2415 Acked Read 10 Done Sent Done false
5579 2416 Acked Write 9 Done Sent Done false
5580 2417 Acked Read 9 Done Sent Done false
5581 2418 Acked Write 8 Done Sent Done false
5582 2419 Acked Read 10 Done Sent Done false
5583 2420 Acked Write 5 Done Sent Done false
5584 2421 Acked Write 1 Done Sent Done false
5585 2422 Acked Read 9 Done Sent Done false
5586 2423 Acked Read 6 Done Sent Done false
5587 2424 Acked Read 9 Done Sent Done false
5588 2425 Acked Write 2 Done Sent Done false
5589 2426 Acked Write 10 Done Sent Done false
5590 2427 Acked Write 8 Done Sent Done false
5591 2428 Acked Read 9 Done Sent Done false
5592 2429 Acked Write 2 Done Sent Done false
5593 2430 Acked Read 4 Done Sent Done false
5594 2431 Acked Read 10 Done Sent Done false
5595 2432 Acked Read 3 Done Sent Done false
5596 2433 Acked Flush 0 Done Sent Done false
5597 2434 Acked Read 2 Done Sent Done false
5598 2435 Acked Read 6 Done Sent Done false
5599 2436 Acked Read 2 Done Sent Done false
5600 2437 Acked Read 3 Done Sent Done false
5601 2438 Acked Read 3 Done Sent Done false
5602 2439 Acked Read 5 Done Sent Done false
5603 2440 Acked Flush 0 Done Sent Done false
5604 2441 Acked Read 1 Done Sent Done false
5605 2442 Acked Write 8 Done Sent Done false
5606 2443 Acked Write 10 Done Sent Done false
5607 2444 Acked Read 4 Done Sent Done false
5608 2445 Acked Read 1 Done Sent Done false
5609 2446 Acked Read 3 Done Sent Done false
5610 2447 Acked Read 1 Done Sent Done false
5611 2448 Acked Write 9 Done Sent Done false
5612 2449 Acked Read 7 Done Sent Done false
5613 2450 Acked Read 10 Done Sent Done false
5614 2451 Acked Read 5 Done Sent Done false
5615 2452 Acked Read 9 Done Sent Done false
5616 2453 Acked Write 8 Done Sent Done false
5617 2454 Acked Read 8 Done Sent Done false
5618 2455 Acked Read 8 Done Sent Done false
5619 2456 Acked Write 4 Done Sent Done false
5620 2457 Acked Read 10 Done Sent Done false
5621 2458 Acked Flush 0 Done Sent Done false
5622 2459 Acked Read 5 Done Sent Done false
5623 2460 Acked Read 7 Done Sent Done false
5624 2461 Acked Write 5 Done Sent Done false
5625 2462 Acked Write 7 Done Sent Done false
5626 2463 Acked Read 8 Done Sent Done false
5627 2464 Acked Write 6 Done Sent Done false
5628 2465 Acked Read 6 Done Sent Done false
5629 2466 Acked Write 3 Done Sent Done false
5630 2467 Acked Read 1 Done Sent Done false
5631 2468 Acked Write 3 Done Sent Done false
5632 2469 Acked Flush 0 Done Sent Done false
5633 2470 Acked Read 8 Done Sent Done false
5634 2471 Acked Read 1 Done Sent Done false
5635 2472 Acked Read 4 Done Sent Done false
5636 2473 Acked Read 10 Done Sent Done false
5637 2474 Acked Read 10 Done Sent Done false
5638 2475 Acked Read 10 Done Sent Done false
5639 2476 Acked Write 5 Done Sent Done false
5640 2477 Acked Read 1 Done Sent Done false
5641 2478 Acked Write 3 Done Sent Done false
5642 2479 Acked Read 3 Done Sent Done false
5643 2480 Acked Write 3 Done Sent Done false
5644 2481 Acked Write 2 Done Sent Done false
5645 2482 Acked Write 3 Done Sent Done false
5646 2483 Acked Read 8 Done Sent Done false
5647 2484 Acked Read 1 Done Sent Done false
5648 2485 Acked Write 5 Done Sent Done false
5649 2486 Acked Flush 0 Done Sent Done false
5650 2487 Acked Write 6 Done Sent Done false
5651 2488 Acked Write 6 Done Sent Done false
5652 2489 Acked Read 7 Done Sent Done false
5653 2490 Acked Read 2 Done Sent Done false
5654 2491 Acked Read 4 Done Sent Done false
5655 2492 Acked Read 10 Done Sent Done false
5656 2493 Acked Write 5 Done Sent Done false
5657 2494 Acked Write 9 Done Sent Done false
5658 2495 Acked Read 9 Done Sent Done false
5659 2496 Acked Read 1 Done Sent Done false
5660 2497 Acked Read 1 Done Sent Done false
5661 2498 Acked Write 8 Done Sent Done false
5662 2499 Acked Read 2 Done Sent Done false
5663 2500 Acked Write 1 Done Sent Done false
5664 2501 Acked Read 5 Done Sent Done false
5665 2502 Acked Read 2 Done Sent Done false
5666 2503 Acked Read 5 Done Sent Done false
5667 2504 Acked Write 6 Done Sent Done false
5668 2505 Acked Write 10 Done Sent Done false
5669 2506 Acked Read 9 Done Sent Done false
5670 2507 Acked Read 8 Done Sent Done false
5671 2508 Acked Flush 0 Done Sent Done false
5672 STATES DS:0 DS:1 DS:2 TOTAL
5673 Sent 0 302 0 302
5674 Done 302 0 302 604
5675 Skipped 0 0 0 0
5676 Error 0 0 0 0
5677 Last Flush: 2508 2206 2508
5678 Downstairs last five completed: 2206 2205 2204 2203 2202
5679 Upstairs last five completed: 2508 2507 2506 2505 2504
5680 CLIENT: Up:0 ds:302 act:2
56812025-01-15T05:20:09.973ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
56822025-01-15T05:20:09.973ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64506) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
56832025-01-15T05:20:09.974ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
56842025-01-15T05:20:09.975ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
56852025-01-15T05:20:09.975ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
56862025-01-15T05:20:09.975ZINFOcrucible: send last flush ID to this DS: 2206 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
56872025-01-15T05:20:09.975ZINFOcrucible: Replied this last flush ID: 2206 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
56882025-01-15T05:20:09.975ZINFOcrucible: [1] client re-new 302 jobs since flush 2206 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
56892025-01-15T05:20:09.975ZINFOcrucible: [1] Replayed 302 jobs since flush: 2206 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
5690 ----------------------------------------------------------------
5691 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5692 Downstairs last five completed: 2508 2507 2506 2505 2504
5693 Upstairs last five completed: 2508 2507 2506 2505 2504
5694 CLIENT: Up:0 ds:0 act:3
5695 Replay: All jobs finished
5696 Main task has work to do, go find it
5697 got dsc Stop(1)
5698 stop 1
5699 Jan 15 05:20:12.209 INFO request completed, latency_us: 300, response_code: [8820] Got stop action so:false kr:false
5700 204, uri: /stop/cid/1, method: POST, req_id: 00d34bbb-757d-48c5-af64-a50465cc1e7e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
57012025-01-15T05:20:12.210ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
57022025-01-15T05:20:12.210ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
57032025-01-15T05:20:12.210ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
5704 Jan 15 05:20:12.210 INFO request completed, latency_us: 128, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3914353c-f9d6-402b-9ad4-aba4d5efe622, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:20:12.210488315Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
5705 .1:9998
5706 [8820] Exited with: ExitStatus(unix_wait_status(9))
5707 [8820][1] reports Exit
5708 Jan 15 05:20:16.212 INFO request completed, latency_us: 188, response_code: 200, uri: /state/cid/1, method: GET, req_id: 65214220-72f0-4b40-a8d1-e4d7fab7f413, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
5709 001/300 Write block 839 len 20480 data: 3 2 2 2 2
5710 002/300 Write block 1216 len 12288 data: 3 3 4
5711 003/300 Write block 4776 len 32768 data: 2 2 2 2 2 2 2 2
5712 004/300 Read block 1844 len 28672
5713 005/300 Flush
5714 006/300 Write block 3844 len 16384 data: 4 3 2 2
5715 007/300 Read block 4949 len 32768
5716 008/300 Flush
5717 009/300 Read block 1132 len 8192
5718 010/300 Write block 4763 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5719 011/300 Read block 3386 len 24576
5720 012/300 Write block 2533 len 12288 data: 4 3 3
5721 013/300 Write block 2113 len 12288 data: 4 3 3
5722 014/300 Read block 1284 len 4096
5723 015/300 Write block 3505 len 28672 data: 2 2 2 2 2 2 2
5724 016/300 Read block 3696 len 4096
5725 017/300 Write block 3112 len 12288 data: 2 2 2
5726 018/300 Write block 1223 len 20480 data: 2 2 2 2 2
5727 019/300 Write block 2121 len 4096 data: 2
5728 020/300 Read block 111 len 28672
5729 021/300 Read block 2930 len 28672
5730 022/300 Write block 4599 len 12288 data: 2 2 2
5731 023/300 Write block 1077 len 8192 data: 2 3
5732 024/300 Flush
5733 025/300 Write block 2318 len 4096 data: 2
5734 026/300 Read block 1301 len 8192
5735 027/300 Write block 4421 len 12288 data: 2 2 3
5736 028/300 Write block 2583 len 32768 data: 2 2 2 3 3 4 4 4
5737 029/300 Read block 678 len 16384
5738 030/300 Read block 2184 len 28672
5739 031/300 Read block 3941 len 20480
5740 032/300 Read block 4568 len 12288
5741 033/300 Write block 1259 len 36864 data: 2 2 3 2 2 2 2 2 2
5742 034/300 Read block 4561 len 36864
5743 035/300 Write block 2497 len 32768 data: 2 2 2 2 2 2 2 2
5744 036/300 Read block 2290 len 36864
5745 037/300 Write block 975 len 24576 data: 2 2 2 2 2 2
5746 038/300 Write block 4216 len 16384 data: 3 3 3 2
5747 039/300 Write block 1345 len 24576 data: 4 5 6 5 4 4
5748 040/300 Read block 3325 len 36864
5749 041/300 Write block 505 len 24576 data: 3 3 3 3 2 2
5750 042/300 Read block 4861 len 32768
5751 043/300 Write block 1814 len 28672 data: 3 3 3 3 2 2 2
5752 044/300 Read block 1783 len 32768
5753 045/300 Read block 1099 len 28672
5754 046/300 Flush
5755 047/300 Read block 1943 len 40960
5756 048/300 Write block 2333 len 12288 data: 2 2 2
5757 049/300 Flush
5758 050/300 Read block 3073 len 8192
5759 051/300 Write block 499 len 4096 data: 3
5760 052/300 Read block 3387 len 4096
5761 053/300 Write block 2247 len 40960 data: 3 2 2 2 2 2 2 3 3 3
5762 054/300 Read block 1477 len 32768
5763 055/300 Read block 2908 len 20480
5764 056/300 Read block 4638 len 24576
5765 057/300 Read block 4776 len 16384
5766 058/300 Write block 4552 len 24576 data: 4 3 2 2 2 2
5767 059/300 Write block 4973 len 28672 data: 2 2 2 3 3 3 3
5768 060/300 Read block 4898 len 36864
5769 061/300 Write block 4020 len 16384 data: 4 4 4 3
5770 062/300 Read block 2296 len 28672
5771 063/300 Write block 4360 len 8192 data: 3 3
5772 064/300 Read block 4443 len 4096
5773 065/300 Read block 1230 len 28672
5774 066/300 Flush
5775 067/300 Read block 2648 len 24576
5776 068/300 Read block 1323 len 20480
5777 069/300 Read block 4497 len 32768
5778 070/300 Write block 484 len 20480 data: 2 2 3 3 3
5779 071/300 Read block 2810 len 12288
5780 072/300 Write block 2904 len 32768 data: 3 3 3 2 2 2 2 2
5781 073/300 Flush
5782 074/300 Write block 321 len 32768 data: 2 2 2 2 2 2 2 2
5783 075/300 Read block 3937 len 20480
5784 076/300 Read block 4097 len 12288
5785 077/300 Write block 4049 len 20480 data: 3 3 3 3 3
5786 078/300 Read block 543 len 4096
5787 079/300 Write block 4979 len 16384 data: 4 2 2 2
5788 080/300 Read block 3348 len 36864
5789 081/300 Write block 3040 len 20480 data: 3 4 4 3 3
5790 082/300 Write block 3878 len 40960 data: 2 2 2 2 2 2 2 2 2 3
5791 083/300 Read block 4832 len 20480
5792 084/300 Write block 4447 len 20480 data: 2 3 3 4 3
5793 085/300 Write block 1109 len 20480 data: 3 3 3 3 2
5794 086/300 Write block 4444 len 8192 data: 2 2
5795 087/300 Write block 4739 len 28672 data: 2 2 2 2 2 3 3
5796 088/300 Write block 2712 len 24576 data: 2 2 2 2 2 2
5797 089/300 Write block 1145 len 16384 data: 2 3 4 4
5798 090/300 Write block 2822 len 24576 data: 3 2 2 2 2 2
5799 091/300 Read block 3161 len 36864
5800 092/300 Flush
5801 093/300 Read block 74 len 16384
5802 094/300 Read block 2053 len 12288
5803 095/300 Read block 254 len 8192
5804 096/300 Write block 4806 len 8192 data: 2 2
5805 097/300 Write block 3784 len 16384 data: 3 3 2 2
5806 098/300 Read block 1282 len 24576
5807 099/300 Write block 4288 len 32768 data: 3 3 3 2 2 2 2 2
5808 100/300 Read block 4394 len 36864
5809 101/300 Write block 214 len 24576 data: 2 2 2 2 2 2
5810 102/300 Read block 617 len 28672
5811 103/300 Write block 206 len 8192 data: 3 2
5812 104/300 Write block 1838 len 40960 data: 2 2 2 3 4 4 4 4 3 3
5813 105/300 Read block 3848 len 28672
5814 106/300 Flush
5815 107/300 Read block 109 len 16384
5816 108/300 Read block 2360 len 12288
5817 109/300 Read block 509 len 24576
5818 110/300 Read block 66 len 16384
5819 111/300 Write block 4048 len 16384 data: 3 4 4 4
5820 112/300 Read block 191 len 20480
5821 113/300 Read block 1385 len 24576
5822 114/300 Read block 4385 len 40960
5823 115/300 Read block 3183 len 36864
5824 116/300 Flush
5825 117/300 Read block 2574 len 40960
5826 118/300 Read block 413 len 12288
5827 119/300 Write block 1568 len 12288 data: 2 2 3
5828 120/300 Flush
5829 121/300 Read block 1511 len 24576
5830 122/300 Read block 3613 len 28672
5831 123/300 Read block 2489 len 4096
5832 124/300 Read block 4008 len 36864
5833 125/300 Read block 4922 len 8192
5834 126/300 Read block 4859 len 36864
5835 127/300 Read block 2243 len 20480
5836 128/300 Flush
5837 129/300 Write block 4059 len 4096 data: 2
5838 130/300 Read block 4959 len 36864
5839 131/300 Write block 3258 len 20480 data: 2 2 2 2 2
5840 132/300 Write block 427 len 24576 data: 2 2 3 2 2 2
5841 133/300 Read block 3108 len 4096
5842 134/300 Read block 2520 len 24576
5843 135/300 Read block 3532 len 4096
5844 136/300 Read block 1285 len 36864
5845 137/300 Flush
5846 138/300 Read block 2857 len 16384
5847 139/300 Read block 53 len 16384
5848 140/300 Read block 1378 len 4096
5849 141/300 Read block 4990 len 36864
5850 142/300 Flush
5851 143/300 Write block 876 len 20480 data: 2 2 2 2 2
5852 144/300 Flush
5853 145/300 Read block 1193 len 8192
5854 146/300 Write block 2363 len 32768 data: 5 4 4 6 5 5 4 4
5855 147/300 Read block 1365 len 16384
5856 148/300 Flush
5857 149/300 Flush
5858 150/300 Read block 3408 len 32768
5859 151/300 Read block 4897 len 8192
5860 152/300 Write block 4354 len 24576 data: 2 2 2 2 2 3
5861 153/300 Write block 4610 len 32768 data: 3 4 5 5 4 3 3 2
5862 154/300 Write block 3835 len 12288 data: 2 3 3
5863 155/300 Write block 1910 len 16384 data: 3 3 4 4
5864 156/300 Read block 3563 len 28672
5865 157/300 Write block 2525 len 20480 data: 2 2 2 2 2
5866 158/300 Write block 248 len 12288 data: 2 2 2
5867 159/300 Write block 465 len 36864 data: 2 3 3 3 4 4 4 4 4
5868 160/300 Read block 362 len 4096
5869 161/300 Write block 1626 len 4096 data: 2
5870 162/300 Write block 4468 len 4096 data: 3
5871 163/300 Read block 1654 len 20480
5872 164/300 Read block 4117 len 8192
5873 165/300 Flush
5874 166/300 Write block 4236 len 24576 data: 2 2 2 3 3 3
5875 167/300 Write block 1900 len 32768 data: 2 2 2 2 2 2 2 3
5876 168/300 Write block 3696 len 20480 data: 3 3 3 2 2
5877 169/300 Flush
5878 170/300 Write block 4573 len 12288 data: 2 2 2
5879 171/300 Read block 3642 len 20480
5880 172/300 Write block 1203 len 8192 data: 4 4
5881 173/300 Write block 72 len 36864 data: 3 2 3 3 3 3 3 3 3
5882 174/300 Read block 3336 len 36864
5883 175/300 Read block 3715 len 8192
5884 176/300 Read block 1366 len 32768
5885 177/300 Flush
5886 178/300 Write block 4656 len 8192 data: 2 2
5887 179/300 Write block 4078 len 8192 data: 2 2
5888 180/300 Read block 129 len 16384
5889 181/300 Write block 2678 len 12288 data: 3 3 3
5890 182/300 Read block 333 len 36864
5891 183/300 Write block 3060 len 36864 data: 3 2 2 2 3 3 2 3 2
5892 184/300 Read block 1029 len 12288
5893 185/300 Write block 4670 len 40960 data: 3 3 3 3 2 2 2 2 2 2
5894 186/300 Read block 3775 len 16384
5895 187/300 Write block 3054 len 20480 data: 3 3 3 4 4
5896 188/300 Write block 2094 len 12288 data: 2 2 3
5897 189/300 Read block 2310 len 40960
5898 190/300 Write block 1475 len 32768 data: 3 2 2 2 3 3 2 2
5899 191/300 Write block 3220 len 28672 data: 2 2 2 2 2 2 2
5900 192/300 Write block 2919 len 32768 data: 3 3 3 3 3 2 2 2
5901 193/300 Read block 3086 len 16384
5902 194/300 Write block 2440 len 12288 data: 2 2 2
5903 195/300 Read block 4728 len 24576
5904 196/300 Read block 347 len 32768
5905 197/300 Read block 2995 len 40960
5906 198/300 Write block 3716 len 4096 data: 5
5907 199/300 Read block 4443 len 8192
5908 200/300 Write block 2884 len 12288 data: 3 3 3
5909 201/300 Write block 3452 len 24576 data: 3 2 2 2 2 2
5910 202/300 Flush
5911 203/300 Write block 2590 len 24576 data: 5 4 4 4 4 3
5912 204/300 Flush
5913 205/300 Write block 3327 len 20480 data: 4 3 2 2 2
5914 206/300 Read block 4179 len 12288
5915 207/300 Write block 3619 len 4096 data: 2
5916 208/300 Write block 1851 len 12288 data: 2 2 2
5917 209/300 Read block 695 len 4096
5918 210/300 Write block 125 len 40960 data: 2 3 2 2 2 2 2 2 2 2
5919 211/300 Write block 2609 len 32768 data: 2 2 2 2 2 2 2 2
5920 212/300 Write block 4817 len 32768 data: 3 2 2 2 2 2 2 2
5921 213/300 Write block 1043 len 24576 data: 2 2 2 2 2 2
5922 214/300 Read block 3215 len 4096
5923 215/300 Read block 2745 len 24576
5924 216/300 Write block 3166 len 8192 data: 3 2
5925 217/300 Read block 966 len 32768
5926 218/300 Read block 3149 len 20480
5927 219/300 Read block 2468 len 4096
5928 220/300 Write block 3528 len 36864 data: 4 3 3 2 2 2 2 2 2
5929 221/300 Read block 3160 len 32768
5930 222/300 Write block 679 len 4096 data: 4
5931 223/300 Write block 127 len 20480 data: 3 3 3 3 3
5932 224/300 Write block 2871 len 4096 data: 2
5933 225/300 Read block 2273 len 12288
5934 226/300 Write block 859 len 24576 data: 2 2 2 2 2 2
5935 227/300 Write block 209 len 12288 data: 2 2 2
5936 228/300 Read block 1740 len 28672
5937 229/300 Write block 1971 len 12288 data: 3 2 2
5938 230/300 Read block 3944 len 12288
5939 231/300 Read block 3066 len 4096
5940 232/300 Read block 1362 len 28672
5941 233/300 Read block 3006 len 40960
5942 234/300 Read block 2662 len 8192
5943 235/300 Flush
5944 236/300 Write block 2604 len 8192 data: 2 2
5945 237/300 Read block 1031 len 4096
5946 238/300 Read block 545 len 12288
5947 239/300 Read block 3959 len 32768
5948 240/300 Read block 564 len 20480
5949 241/300 Write block 2152 len 4096 data: 3
5950 242/300 Flush
5951 243/300 Write block 2712 len 36864 data: 3 3 3 3 3 3 2 2 2
5952 244/300 Read block 2449 len 8192
5953 245/300 Write block 4339 len 4096 data: 3
5954 246/300 Write block 588 len 36864 data: 2 2 2 2 2 2 4 4 3
5955 247/300 Write block 2539 len 12288 data: 2 2 2
5956 248/300 Read block 1975 len 12288
5957 249/300 Write block 3599 len 8192 data: 2 2
5958 250/300 Read block 4601 len 12288
5959 251/300 Read block 650 len 36864
5960 252/300 Write block 577 len 12288 data: 3 3 3
5961 253/300 Read block 3312 len 32768
5962 254/300 Write block 442 len 32768 data: 6 5 4 4 5 3 3 3
5963 255/300 Write block 3373 len 8192 data: 2 2
5964 256/300 Write block 1010 len 36864 data: 3 3 3 5 4 4 3 2 2
5965 257/300 Write block 3651 len 28672 data: 5 4 4 3 3 3 4
5966 258/300 Read block 3810 len 24576
5967 259/300 Read block 2896 len 20480
5968 260/300 Read block 2379 len 8192
5969 261/300 Read block 3051 len 20480
5970 262/300 Write block 1569 len 28672 data: 3 4 4 4 5 5 5
5971 263/300 Read block 2974 len 4096
5972 264/300 Read block 4041 len 4096
5973 265/300 Read block 636 len 20480
5974 266/300 Flush
5975 267/300 Read block 3993 len 4096
5976 268/300 Write block 2496 len 16384 data: 2 3 3 3
5977 269/300 Read block 1724 len 16384
5978 270/300 Read block 3506 len 32768
5979 271/300 Write block 652 len 20480 data: 4 3 2 2 2
5980 272/300 Write block 3370 len 4096 data: 3
5981 273/300 Write block 645 len 8192 data: 4 4
5982 274/300 Read block 1254 len 28672
5983 275/300 Write block 2439 len 24576 data: 2 3 3 3 2 2
5984 276/300 Read block 2632 len 28672
5985 277/300 Read block 1520 len 40960
5986 278/300 Read block 4717 len 24576
5987 279/300 Read block 409 len 36864
5988 280/300 Read block 4571 len 16384
5989 281/300 Read block 3355 len 4096
5990 282/300 Flush
5991 283/300 Write block 1952 len 4096 data: 2
5992 284/300 Write block 4549 len 36864 data: 4 4 4 5 4 3 3 3 3
5993 285/300 Read block 2633 len 20480
5994 286/300 Read block 3539 len 36864
5995 287/300 Read block 3304 len 20480
5996 288/300 Read block 777 len 8192
5997 289/300 Read block 2389 len 12288
5998 290/300 Read block 214 len 40960
5999 291/300 Read block 250 len 16384
6000 292/300 Read block 1242 len 16384
6001 293/300 Read block 2775 len 20480
6002 294/300 Read block 1351 len 28672
6003 295/300 Write block 266 len 28672 data: 2 2 2 2 2 2 2
6004 296/300 Read block 336 len 8192
6005 297/300 Read block 4959 len 24576
6006 298/300 Read block 3678 len 36864
6007 299/300 Read block 667 len 12288
6008 300/300 Flush
6009 301/300 Write block 1623 len 32768 data: 2 2 2 3 2 2 3 3
6010 Main task has work to do, go find it
6011 got dsc Start(1)
6012 start 1
6013 Jan 15 05:20:16.436 INFO request completed, latency_us: 246, response_code: 204, uri: /start/cid/1, method: POST, req_id: f951dcdb-21ee-4ce2-924a-86342eb289df, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
6014 Replay: started 1, returned:Ok(())
6015 ----------------------------------------------------------------
6016 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6017 States: Active Offline Active
6018 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6019 2509 Acked Write 5 Done Sent Done false
6020 2510 Acked Write 3 Done Sent Done false
6021 2511 Acked Write 8 Done Sent Done false
6022 2512 Acked Read 7 Done Sent Done false
6023 2513 Acked Flush 0 Done Sent Done false
6024 2514 Acked Write 4 Done Sent Done false
6025 2515 Acked Read 8 Done Sent Done false
6026 2516 Acked Flush 0 Done Sent Done false
6027 2517 Acked Read 2 Done Sent Done false
6028 2518 Acked Write 10 Done Sent Done false
6029 2519 Acked Read 6 Done Sent Done false
6030 2520 Acked Write 3 Done Sent Done false
6031 2521 Acked Write 3 Done Sent Done false
6032 2522 Acked Read 1 Done Sent Done false
6033 2523 Acked Write 7 Done Sent Done false
6034 2524 Acked Read 1 Done Sent Done false
6035 2525 Acked Write 3 Done Sent Done false
6036 2526 Acked Write 5 Done Sent Done false
6037 2527 Acked Write 1 Done Sent Done false
6038 2528 Acked Read 7 Done Sent Done false
6039 2529 Acked Read 7 Done Sent Done false
6040 2530 Acked Write 3 Done Sent Done false
6041 2531 Acked Write 2 Done Sent Done false
6042 2532 Acked Flush 0 Done Sent Done false
6043 2533 Acked Write 1 Done Sent Done false
6044 2534 Acked Read 2 Done Sent Done false
6045 2535 Acked Write 3 Done Sent Done false
6046 2536 Acked Write 8 Done Sent Done false
6047 2537 Acked Read 4 Done Sent Done false
6048 2538 Acked Read 7 Done Sent Done false
6049 2539 Acked Read 5 Done Sent Done false
6050 2540 Acked Read 3 Done Sent Done false
6051 2541 Acked Write 9 Done Sent Done false
6052 2542 Acked Read 9 Done Sent Done false
6053 2543 Acked Write 8 Done Sent Done false
6054 2544 Acked Read 9 Done Sent Done false
6055 2545 Acked Write 6 Done Sent Done false
6056 2546 Acked Write 4 Done Sent Done false
6057 2547 Acked Write 6 Done Sent Done false
6058 2548 Acked Read 9 Done Sent Done false
6059 2549 Acked Write 6 Done Sent Done false
6060 2550 Acked Read 8 Done Sent Done false
6061 2551 Acked Write 7 Done Sent Done false
6062 2552 Acked Read 8 Done Sent Done false
6063 2553 Acked Read 7 Done Sent Done false
6064 2554 Acked Flush 0 Done Sent Done false
6065 2555 Acked Read 10 Done Sent Done false
6066 2556 Acked Write 3 Done Sent Done false
6067 2557 Acked Flush 0 Done Sent Done false
6068 2558 Acked Read 2 Done Sent Done false
6069 2559 Acked Write 1 Done Sent Done false
6070 2560 Acked Read 1 Done Sent Done false
6071 2561 Acked Write 10 Done Sent Done false
6072 2562 Acked Read 8 Done Sent Done false
6073 2563 Acked Read 5 Done Sent Done false
6074 2564 Acked Read 6 Done Sent Done false
6075 2565 Acked Read 4 Done Sent Done false
6076 2566 Acked Write 6 Done Sent Done false
6077 2567 Acked Write 7 Done Sent Done false
6078 2568 Acked Read 9 Done Sent Done false
6079 2569 Acked Write 4 Done Sent Done false
6080 2570 Acked Read 7 Done Sent Done false
6081 2571 Acked Write 2 Done Sent Done false
6082 2572 Acked Read 1 Done Sent Done false
6083 2573 Acked Read 7 Done Sent Done false
6084 2574 Acked Flush 0 Done Sent Done false
6085 2575 Acked Read 6 Done Sent Done false
6086 2576 Acked Read 5 Done Sent Done false
6087 2577 Acked Read 8 Done Sent Done false
6088 2578 Acked Write 5 Done Sent Done false
6089 2579 Acked Read 3 Done Sent Done false
6090 2580 Acked Write 8 Done Sent Done false
6091 2581 Acked Flush 0 Done Sent Done false
6092 2582 Acked Write 8 Done Sent Done false
6093 2583 Acked Read 5 Done Sent Done false
6094 2584 Acked Read 3 Done Sent Done false
6095 2585 Acked Write 5 Done Sent Done false
6096 2586 Acked Read 1 Done Sent Done false
6097 2587 Acked Write 4 Done Sent Done false
6098 2588 Acked Read 9 Done Sent Done false
6099 2589 Acked Write 5 Done Sent Done false
6100 2590 Acked Write 10 Done Sent Done false
6101 2591 Acked Read 5 Done Sent Done false
6102 2592 Acked Write 5 Done Sent Done false
6103 2593 Acked Write 5 Done Sent Done false
6104 2594 Acked Write 2 Done Sent Done false
6105 2595 Acked Write 7 Done Sent Done false
6106 2596 Acked Write 6 Done Sent Done false
6107 2597 Acked Write 4 Done Sent Done false
6108 2598 Acked Write 6 Done Sent Done false
6109 2599 Acked Read 9 Done Sent Done false
6110 2600 Acked Flush 0 Done Sent Done false
6111 2601 Acked Read 4 Done Sent Done false
6112 2602 Acked Read 3 Done Sent Done false
6113 2603 Acked Read 2 Done Sent Done false
6114 2604 Acked Write 2 Done Sent Done false
6115 2605 Acked Write 4 Done Sent Done false
6116 2606 Acked Read 6 Done Sent Done false
6117 2607 Acked Write 8 Done Sent Done false
6118 2608 Acked Read 9 Done Sent Done false
6119 2609 Acked Write 6 Done Sent Done false
6120 2610 Acked Read 7 Done Sent Done false
6121 2611 Acked Write 2 Done Sent Done false
6122 2612 Acked Write 10 Done Sent Done false
6123 2613 Acked Read 7 Done Sent Done false
6124 2614 Acked Flush 0 Done Sent Done false
6125 2615 Acked Read 4 Done Sent Done false
6126 2616 Acked Read 3 Done Sent Done false
6127 2617 Acked Read 6 Done Sent Done false
6128 2618 Acked Read 4 Done Sent Done false
6129 2619 Acked Write 4 Done Sent Done false
6130 2620 Acked Read 5 Done Sent Done false
6131 2621 Acked Read 6 Done Sent Done false
6132 2622 Acked Read 10 Done Sent Done false
6133 2623 Acked Read 9 Done Sent Done false
6134 2624 Acked Flush 0 Done Sent Done false
6135 2625 Acked Read 10 Done Sent Done false
6136 2626 Acked Read 3 Done Sent Done false
6137 2627 Acked Write 3 Done Sent Done false
6138 2628 Acked Flush 0 Done Sent Done false
6139 2629 Acked Read 6 Done Sent Done false
6140 2630 Acked Read 7 Done Sent Done false
6141 2631 Acked Read 1 Done Sent Done false
6142 2632 Acked Read 9 Done Sent Done false
6143 2633 Acked Read 2 Done Sent Done false
6144 2634 Acked Read 9 Done Sent Done false
6145 2635 Acked Read 5 Done Sent Done false
6146 2636 Acked Flush 0 Done Sent Done false
6147 2637 Acked Write 1 Done Sent Done false
6148 2638 Acked Read 9 Done Sent Done false
6149 2639 Acked Write 5 Done Sent Done false
6150 2640 Acked Write 6 Done Sent Done false
6151 2641 Acked Read 1 Done Sent Done false
6152 2642 Acked Read 6 Done Sent Done false
6153 2643 Acked Read 1 Done Sent Done false
6154 2644 Acked Read 9 Done Sent Done false
6155 2645 Acked Flush 0 Done Sent Done false
6156 2646 Acked Read 4 Done Sent Done false
6157 2647 Acked Read 4 Done Sent Done false
6158 2648 Acked Read 1 Done Sent Done false
6159 2649 Acked Read 9 Done Sent Done false
6160 2650 Acked Flush 0 Done Sent Done false
6161 2651 Acked Write 5 Done Sent Done false
6162 2652 Acked Flush 0 Done Sent Done false
6163 2653 Acked Read 2 Done Sent Done false
6164 2654 Acked Write 8 Done Sent Done false
6165 2655 Acked Read 4 Done Sent Done false
6166 2656 Acked Flush 0 Done Sent Done false
6167 2657 Acked Flush 0 Done Sent Done false
6168 2658 Acked Read 8 Done Sent Done false
6169 2659 Acked Read 2 Done Sent Done false
6170 2660 Acked Write 6 Done Sent Done false
6171 2661 Acked Write 8 Done Sent Done false
6172 2662 Acked Write 3 Done Sent Done false
6173 2663 Acked Write 4 Done Sent Done false
6174 2664 Acked Read 7 Done Sent Done false
6175 2665 Acked Write 5 Done Sent Done false
6176 2666 Acked Write 3 Done Sent Done false
6177 2667 Acked Write 9 Done Sent Done false
6178 2668 Acked Read 1 Done Sent Done false
6179 2669 Acked Write 1 Done Sent Done false
6180 2670 Acked Write 1 Done Sent Done false
6181 2671 Acked Read 5 Done Sent Done false
6182 2672 Acked Read 2 Done Sent Done false
6183 2673 Acked Flush 0 Done Sent Done false
6184 2674 Acked Write 6 Done Sent Done false
6185 2675 Acked Write 8 Done Sent Done false
6186 2676 Acked Write 5 Done Sent Done false
6187 2677 Acked Flush 0 Done Sent Done false
6188 2678 Acked Write 3 Done Sent Done false
6189 2679 Acked Read 5 Done Sent Done false
6190 2680 Acked Write 2 Done Sent Done false
6191 2681 Acked Write 9 Done Sent Done false
6192 2682 Acked Read 9 Done Sent Done false
6193 2683 Acked Read 2 Done Sent Done false
6194 2684 Acked Read 8 Done Sent Done false
6195 2685 Acked Flush 0 Done Sent Done false
6196 2686 Acked Write 2 Done Sent Done false
6197 2687 Acked Write 2 Done Sent Done false
6198 2688 Acked Read 4 Done Sent Done false
6199 2689 Acked Write 3 Done Sent Done false
6200 2690 Acked Read 9 Done Sent Done false
6201 2691 Acked Write 9 Done Sent Done false
6202 2692 Acked Read 3 Done Sent Done false
6203 2693 Acked Write 10 Done Sent Done false
6204 2694 Acked Read 4 Done Sent Done false
6205 2695 Acked Write 5 Done Sent Done false
6206 2696 Acked Write 3 Done Sent Done false
6207 2697 Acked Read 10 Done Sent Done false
6208 2698 Acked Write 8 Done Sent Done false
6209 2699 Acked Write 7 Done Sent Done false
6210 2700 Acked Write 8 Done Sent Done false
6211 2701 Acked Read 4 Done Sent Done false
6212 2702 Acked Write 3 Done Sent Done false
6213 2703 Acked Read 6 Done Sent Done false
6214 2704 Acked Read 8 Done Sent Done false
6215 2705 Acked Read 10 Done Sent Done false
6216 2706 Acked Write 1 Done Sent Done false
6217 2707 Acked Read 2 Done Sent Done false
6218 2708 Acked Write 3 Done Sent Done false
6219 2709 Acked Write 6 Done Sent Done false
6220 2710 Acked Flush 0 Done Sent Done false
6221 2711 Acked Write 6 Done Sent Done false
6222 2712 Acked Flush 0 Done Sent Done false
6223 2713 Acked Write 5 Done Sent Done false
6224 2714 Acked Read 3 Done Sent Done false
6225 2715 Acked Write 1 Done Sent Done false
6226 2716 Acked Write 3 Done Sent Done false
6227 2717 Acked Read 1 Done Sent Done false
6228 2718 Acked Write 10 Done Sent Done false
6229 2719 Acked Write 8 Done Sent Done false
6230 2720 Acked Write 8 Done Sent Done false
6231 2721 Acked Write 6 Done Sent Done false
6232 2722 Acked Read 1 Done Sent Done false
6233 2723 Acked Read 6 Done Sent Done false
6234 2724 Acked Write 2 Done Sent Done false
6235 2725 Acked Read 8 Done Sent Done false
6236 2726 Acked Read 5 Done Sent Done false
6237 2727 Acked Read 1 Done Sent Done false
6238 2728 Acked Write 9 Done Sent Done false
6239 2729 Acked Read 8 Done Sent Done false
6240 2730 Acked Write 1 Done Sent Done false
6241 2731 Acked Write 5 Done Sent Done false
6242 2732 Acked Write 1 Done Sent Done false
6243 2733 Acked Read 3 Done Sent Done false
6244 2734 Acked Write 6 Done Sent Done false
6245 2735 Acked Write 3 Done Sent Done false
6246 2736 Acked Read 7 Done Sent Done false
6247 2737 Acked Write 3 Done Sent Done false
6248 2738 Acked Read 3 Done Sent Done false
6249 2739 Acked Read 1 Done Sent Done false
6250 2740 Acked Read 7 Done Sent Done false
6251 2741 Acked Read 10 Done Sent Done false
6252 2742 Acked Read 2 Done Sent Done false
6253 2743 Acked Flush 0 Done Sent Done false
6254 2744 Acked Write 2 Done Sent Done false
6255 2745 Acked Read 1 Done Sent Done false
6256 2746 Acked Read 3 Done Sent Done false
6257 2747 Acked Read 8 Done Sent Done false
6258 2748 Acked Read 5 Done Sent Done false
6259 2749 Acked Write 1 Done Sent Done false
6260 2750 Acked Flush 0 Done Sent Done false
6261 2751 Acked Write 9 Done Sent Done false
6262 2752 Acked Read 2 Done Sent Done false
6263 2753 Acked Write 1 Done Sent Done false
6264 2754 Acked Write 9 Done Sent Done false
6265 2755 Acked Write 3 Done Sent Done false
6266 2756 Acked Read 3 Done Sent Done false
6267 2757 Acked Write 2 Done Sent Done false
6268 2758 Acked Read 3 Done Sent Done false
6269 2759 Acked Read 9 Done Sent Done false
6270 2760 Acked Write 3 Done Sent Done false
6271 2761 Acked Read 8 Done Sent Done false
6272 2762 Acked Write 8 Done Sent Done false
6273 2763 Acked Write 2 Done Sent Done false
6274 2764 Acked Write 9 Done Sent Done false
6275 2765 Acked Write 7 Done Sent Done false
6276 2766 Acked Read 6 Done Sent Done false
6277 2767 Acked Read 5 Done Sent Done false
6278 2768 Acked Read 2 Done Sent Done false
6279 2769 Acked Read 5 Done Sent Done false
6280 2770 Acked Write 7 Done Sent Done false
6281 2771 Acked Read 1 Done Sent Done false
6282 2772 Acked Read 1 Done Sent Done false
6283 2773 Acked Read 5 Done Sent Done false
6284 2774 Acked Flush 0 Done Sent Done false
6285 2775 Acked Read 1 Done Sent Done false
6286 2776 Acked Write 4 Done Sent Done false
6287 2777 Acked Read 4 Done Sent Done false
6288 2778 Acked Read 8 Done Sent Done false
6289 2779 Acked Write 5 Done Sent Done false
6290 2780 Acked Write 1 Done Sent Done false
6291 2781 Acked Write 2 Done Sent Done false
6292 2782 Acked Read 7 Done Sent Done false
6293 2783 Acked Write 6 Done Sent Done false
6294 2784 Acked Read 7 Done Sent Done false
6295 2785 Acked Read 10 Done Sent Done false
6296 2786 Acked Read 6 Done Sent Done false
6297 2787 Acked Read 9 Done Sent Done false
6298 2788 Acked Read 4 Done Sent Done false
6299 2789 Acked Read 1 Done Sent Done false
6300 2790 Acked Flush 0 Done Sent Done false
6301 2791 Acked Write 1 Done Sent Done false
6302 2792 Acked Write 9 Done Sent Done false
6303 2793 Acked Read 5 Done Sent Done false
6304 2794 Acked Read 9 Done Sent Done false
6305 2795 Acked Read 5 Done Sent Done false
6306 2796 Acked Read 2 Done Sent Done false
6307 2797 Acked Read 3 Done Sent Done false
6308 2798 Acked Read 10 Done Sent Done false
6309 2799 Acked Read 4 Done Sent Done false
6310 2800 Acked Read 4 Done Sent Done false
6311 2801 Acked Read 5 Done Sent Done false
6312 2802 Acked Read 7 Done Sent Done false
6313 2803 Acked Write 7 Done Sent Done false
6314 2804 Acked Read 2 Done Sent Done false
6315 2805 Acked Read 6 Done Sent Done false
6316 2806 Acked Read 9 Done Sent Done false
6317 2807 Acked Read 3 Done Sent Done false
6318 2808 Acked Flush 0 Done Sent Done false
6319 2809 Acked Write 8 Done Sent Done false
6320 STATES DS:0 DS:1 DS:2 TOTAL
6321 Sent 0 301 0 301
6322 Done 301 0 301 602
6323 Skipped 0 0 0 0
6324 Error 0 0 0 0
6325 Last Flush: 2808 2508 2808
6326 Downstairs last five completed: 2508 2507 2506 2505 2504
6327 Upstairs last five completed: 2809 2808 2807 2806 2805
6328 CLIENT: Up:0 ds:301 act:2
6329 [8820] got start action
6330 [8820] I am going to restart
6331 Starting downstairs at port 8820
6332 Make output file at "/tmp/dsc/downstairs-8820.txt"
6333 [8820][1] reports Starting
6334 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1208)
6335 [8820][1] reports Running
6336 ----------------------------------------------------------------
6337 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6338 States: Active Offline Active
6339 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6340 2509 Acked Write 5 Done Sent Done false
6341 2510 Acked Write 3 Done Sent Done false
6342 2511 Acked Write 8 Done Sent Done false
6343 2512 Acked Read 7 Done Sent Done false
6344 2513 Acked Flush 0 Done Sent Done false
6345 2514 Acked Write 4 Done Sent Done false
6346 2515 Acked Read 8 Done Sent Done false
6347 2516 Acked Flush 0 Done Sent Done false
6348 2517 Acked Read 2 Done Sent Done false
6349 2518 Acked Write 10 Done Sent Done false
6350 2519 Acked Read 6 Done Sent Done false
6351 2520 Acked Write 3 Done Sent Done false
6352 2521 Acked Write 3 Done Sent Done false
6353 2522 Acked Read 1 Done Sent Done false
6354 2523 Acked Write 7 Done Sent Done false
6355 2524 Acked Read 1 Done Sent Done false
6356 2525 Acked Write 3 Done Sent Done false
6357 2526 Acked Write 5 Done Sent Done false
6358 2527 Acked Write 1 Done Sent Done false
6359 2528 Acked Read 7 Done Sent Done false
6360 2529 Acked Read 7 Done Sent Done false
6361 2530 Acked Write 3 Done Sent Done false
6362 2531 Acked Write 2 Done Sent Done false
6363 2532 Acked Flush 0 Done Sent Done false
6364 2533 Acked Write 1 Done Sent Done false
6365 2534 Acked Read 2 Done Sent Done false
6366 2535 Acked Write 3 Done Sent Done false
6367 2536 Acked Write 8 Done Sent Done false
6368 2537 Acked Read 4 Done Sent Done false
6369 2538 Acked Read 7 Done Sent Done false
6370 2539 Acked Read 5 Done Sent Done false
6371 2540 Acked Read 3 Done Sent Done false
6372 2541 Acked Write 9 Done Sent Done false
6373 2542 Acked Read 9 Done Sent Done false
6374 2543 Acked Write 8 Done Sent Done false
6375 2544 Acked Read 9 Done Sent Done false
6376 2545 Acked Write 6 Done Sent Done false
6377 2546 Acked Write 4 Done Sent Done false
6378 2547 Acked Write 6 Done Sent Done false
6379 2548 Acked Read 9 Done Sent Done false
6380 2549 Acked Write 6 Done Sent Done false
6381 2550 Acked Read 8 Done Sent Done false
6382 2551 Acked Write 7 Done Sent Done false
6383 2552 Acked Read 8 Done Sent Done false
6384 2553 Acked Read 7 Done Sent Done false
6385 2554 Acked Flush 0 Done Sent Done false
6386 2555 Acked Read 10 Done Sent Done false
6387 2556 Acked Write 3 Done Sent Done false
6388 2557 Acked Flush 0 Done Sent Done false
6389 2558 Acked Read 2 Done Sent Done false
6390 2559 Acked Write 1 Done Sent Done false
6391 2560 Acked Read 1 Done Sent Done false
6392 2561 Acked Write 10 Done Sent Done false
6393 2562 Acked Read 8 Done Sent Done false
6394 2563 Acked Read 5 Done Sent Done false
6395 2564 Acked Read 6 Done Sent Done false
6396 2565 Acked Read 4 Done Sent Done false
6397 2566 Acked Write 6 Done Sent Done false
6398 2567 Acked Write 7 Done Sent Done false
6399 2568 Acked Read 9 Done Sent Done false
6400 2569 Acked Write 4 Done Sent Done false
6401 2570 Acked Read 7 Done Sent Done false
6402 2571 Acked Write 2 Done Sent Done false
6403 2572 Acked Read 1 Done Sent Done false
6404 2573 Acked Read 7 Done Sent Done false
6405 2574 Acked Flush 0 Done Sent Done false
6406 2575 Acked Read 6 Done Sent Done false
6407 2576 Acked Read 5 Done Sent Done false
6408 2577 Acked Read 8 Done Sent Done false
6409 2578 Acked Write 5 Done Sent Done false
6410 2579 Acked Read 3 Done Sent Done false
6411 2580 Acked Write 8 Done Sent Done false
6412 2581 Acked Flush 0 Done Sent Done false
6413 2582 Acked Write 8 Done Sent Done false
6414 2583 Acked Read 5 Done Sent Done false
6415 2584 Acked Read 3 Done Sent Done false
6416 2585 Acked Write 5 Done Sent Done false
6417 2586 Acked Read 1 Done Sent Done false
6418 2587 Acked Write 4 Done Sent Done false
6419 2588 Acked Read 9 Done Sent Done false
6420 2589 Acked Write 5 Done Sent Done false
6421 2590 Acked Write 10 Done Sent Done false
6422 2591 Acked Read 5 Done Sent Done false
6423 2592 Acked Write 5 Done Sent Done false
6424 2593 Acked Write 5 Done Sent Done false
6425 2594 Acked Write 2 Done Sent Done false
6426 2595 Acked Write 7 Done Sent Done false
6427 2596 Acked Write 6 Done Sent Done false
6428 2597 Acked Write 4 Done Sent Done false
6429 2598 Acked Write 6 Done Sent Done false
6430 2599 Acked Read 9 Done Sent Done false
6431 2600 Acked Flush 0 Done Sent Done false
6432 2601 Acked Read 4 Done Sent Done false
6433 2602 Acked Read 3 Done Sent Done false
6434 2603 Acked Read 2 Done Sent Done false
6435 2604 Acked Write 2 Done Sent Done false
6436 2605 Acked Write 4 Done Sent Done false
6437 2606 Acked Read 6 Done Sent Done false
6438 2607 Acked Write 8 Done Sent Done false
6439 2608 Acked Read 9 Done Sent Done false
6440 2609 Acked Write 6 Done Sent Done false
6441 2610 Acked Read 7 Done Sent Done false
6442 2611 Acked Write 2 Done Sent Done false
6443 2612 Acked Write 10 Done Sent Done false
6444 2613 Acked Read 7 Done Sent Done false
6445 2614 Acked Flush 0 Done Sent Done false
6446 2615 Acked Read 4 Done Sent Done false
6447 2616 Acked Read 3 Done Sent Done false
6448 2617 Acked Read 6 Done Sent Done false
6449 2618 Acked Read 4 Done Sent Done false
6450 2619 Acked Write 4 Done Sent Done false
6451 2620 Acked Read 5 Done Sent Done false
6452 2621 Acked Read 6 Done Sent Done false
6453 2622 Acked Read 10 Done Sent Done false
6454 2623 Acked Read 9 Done Sent Done false
6455 2624 Acked Flush 0 Done Sent Done false
6456 2625 Acked Read 10 Done Sent Done false
6457 2626 Acked Read 3 Done Sent Done false
6458 2627 Acked Write 3 Done Sent Done false
6459 2628 Acked Flush 0 Done Sent Done false
6460 2629 Acked Read 6 Done Sent Done false
6461 2630 Acked Read 7 Done Sent Done false
6462 2631 Acked Read 1 Done Sent Done false
6463 2632 Acked Read 9 Done Sent Done false
6464 2633 Acked Read 2 Done Sent Done false
6465 2634 Acked Read 9 Done Sent Done false
6466 2635 Acked Read 5 Done Sent Done false
6467 2636 Acked Flush 0 Done Sent Done false
6468 2637 Acked Write 1 Done Sent Done false
6469 2638 Acked Read 9 Done Sent Done false
6470 2639 Acked Write 5 Done Sent Done false
6471 2640 Acked Write 6 Done Sent Done false
6472 2641 Acked Read 1 Done Sent Done false
6473 2642 Acked Read 6 Done Sent Done false
6474 2643 Acked Read 1 Done Sent Done false
6475 2644 Acked Read 9 Done Sent Done false
6476 2645 Acked Flush 0 Done Sent Done false
6477 2646 Acked Read 4 Done Sent Done false
6478 2647 Acked Read 4 Done Sent Done false
6479 2648 Acked Read 1 Done Sent Done false
6480 2649 Acked Read 9 Done Sent Done false
6481 2650 Acked Flush 0 Done Sent Done false
6482 2651 Acked Write 5 Done Sent Done false
6483 2652 Acked Flush 0 Done Sent Done false
6484 2653 Acked Read 2 Done Sent Done false
6485 2654 Acked Write 8 Done Sent Done false
6486 2655 Acked Read 4 Done Sent Done false
6487 2656 Acked Flush 0 Done Sent Done false
6488 2657 Acked Flush 0 Done Sent Done false
6489 2658 Acked Read 8 Done Sent Done false
6490 2659 Acked Read 2 Done Sent Done false
6491 2660 Acked Write 6 Done Sent Done false
6492 2661 Acked Write 8 Done Sent Done false
6493 2662 Acked Write 3 Done Sent Done false
6494 2663 Acked Write 4 Done Sent Done false
6495 2664 Acked Read 7 Done Sent Done false
6496 2665 Acked Write 5 Done Sent Done false
6497 2666 Acked Write 3 Done Sent Done false
6498 2667 Acked Write 9 Done Sent Done false
6499 2668 Acked Read 1 Done Sent Done false
6500 2669 Acked Write 1 Done Sent Done false
6501 2670 Acked Write 1 Done Sent Done false
6502 2671 Acked Read 5 Done Sent Done false
6503 2672 Acked Read 2 Done Sent Done false
6504 2673 Acked Flush 0 Done Sent Done false
6505 2674 Acked Write 6 Done Sent Done false
6506 2675 Acked Write 8 Done Sent Done false
6507 2676 Acked Write 5 Done Sent Done false
6508 2677 Acked Flush 0 Done Sent Done false
6509 2678 Acked Write 3 Done Sent Done false
6510 2679 Acked Read 5 Done Sent Done false
6511 2680 Acked Write 2 Done Sent Done false
6512 2681 Acked Write 9 Done Sent Done false
6513 2682 Acked Read 9 Done Sent Done false
6514 2683 Acked Read 2 Done Sent Done false
6515 2684 Acked Read 8 Done Sent Done false
6516 2685 Acked Flush 0 Done Sent Done false
6517 2686 Acked Write 2 Done Sent Done false
6518 2687 Acked Write 2 Done Sent Done false
6519 2688 Acked Read 4 Done Sent Done false
6520 2689 Acked Write 3 Done Sent Done false
6521 2690 Acked Read 9 Done Sent Done false
6522 2691 Acked Write 9 Done Sent Done false
6523 2692 Acked Read 3 Done Sent Done false
6524 2693 Acked Write 10 Done Sent Done false
6525 2694 Acked Read 4 Done Sent Done false
6526 2695 Acked Write 5 Done Sent Done false
6527 2696 Acked Write 3 Done Sent Done false
6528 2697 Acked Read 10 Done Sent Done false
6529 2698 Acked Write 8 Done Sent Done false
6530 2699 Acked Write 7 Done Sent Done false
6531 2700 Acked Write 8 Done Sent Done false
6532 2701 Acked Read 4 Done Sent Done false
6533 2702 Acked Write 3 Done Sent Done false
6534 2703 Acked Read 6 Done Sent Done false
6535 2704 Acked Read 8 Done Sent Done false
6536 2705 Acked Read 10 Done Sent Done false
6537 2706 Acked Write 1 Done Sent Done false
6538 2707 Acked Read 2 Done Sent Done false
6539 2708 Acked Write 3 Done Sent Done false
6540 2709 Acked Write 6 Done Sent Done false
6541 2710 Acked Flush 0 Done Sent Done false
6542 2711 Acked Write 6 Done Sent Done false
6543 2712 Acked Flush 0 Done Sent Done false
6544 2713 Acked Write 5 Done Sent Done false
6545 2714 Acked Read 3 Done Sent Done false
6546 2715 Acked Write 1 Done Sent Done false
6547 2716 Acked Write 3 Done Sent Done false
6548 2717 Acked Read 1 Done Sent Done false
6549 2718 Acked Write 10 Done Sent Done false
6550 2719 Acked Write 8 Done Sent Done false
6551 2720 Acked Write 8 Done Sent Done false
6552 2721 Acked Write 6 Done Sent Done false
6553 2722 Acked Read 1 Done Sent Done false
6554 2723 Acked Read 6 Done Sent Done false
6555 2724 Acked Write 2 Done Sent Done false
6556 2725 Acked Read 8 Done Sent Done false
6557 2726 Acked Read 5 Done Sent Done false
6558 2727 Acked Read 1 Done Sent Done false
6559 2728 Acked Write 9 Done Sent Done false
6560 2729 Acked Read 8 Done Sent Done false
6561 2730 Acked Write 1 Done Sent Done false
6562 2731 Acked Write 5 Done Sent Done false
6563 2732 Acked Write 1 Done Sent Done false
6564 2733 Acked Read 3 Done Sent Done false
6565 2734 Acked Write 6 Done Sent Done false
6566 2735 Acked Write 3 Done Sent Done false
6567 2736 Acked Read 7 Done Sent Done false
6568 2737 Acked Write 3 Done Sent Done false
6569 2738 Acked Read 3 Done Sent Done false
6570 2739 Acked Read 1 Done Sent Done false
6571 2740 Acked Read 7 Done Sent Done false
6572 2741 Acked Read 10 Done Sent Done false
6573 2742 Acked Read 2 Done Sent Done false
6574 2743 Acked Flush 0 Done Sent Done false
6575 2744 Acked Write 2 Done Sent Done false
6576 2745 Acked Read 1 Done Sent Done false
6577 2746 Acked Read 3 Done Sent Done false
6578 2747 Acked Read 8 Done Sent Done false
6579 2748 Acked Read 5 Done Sent Done false
6580 2749 Acked Write 1 Done Sent Done false
6581 2750 Acked Flush 0 Done Sent Done false
6582 2751 Acked Write 9 Done Sent Done false
6583 2752 Acked Read 2 Done Sent Done false
6584 2753 Acked Write 1 Done Sent Done false
6585 2754 Acked Write 9 Done Sent Done false
6586 2755 Acked Write 3 Done Sent Done false
6587 2756 Acked Read 3 Done Sent Done false
6588 2757 Acked Write 2 Done Sent Done false
6589 2758 Acked Read 3 Done Sent Done false
6590 2759 Acked Read 9 Done Sent Done false
6591 2760 Acked Write 3 Done Sent Done false
6592 2761 Acked Read 8 Done Sent Done false
6593 2762 Acked Write 8 Done Sent Done false
6594 2763 Acked Write 2 Done Sent Done false
6595 2764 Acked Write 9 Done Sent Done false
6596 2765 Acked Write 7 Done Sent Done false
6597 2766 Acked Read 6 Done Sent Done false
6598 2767 Acked Read 5 Done Sent Done false
6599 2768 Acked Read 2 Done Sent Done false
6600 2769 Acked Read 5 Done Sent Done false
6601 2770 Acked Write 7 Done Sent Done false
6602 2771 Acked Read 1 Done Sent Done false
6603 2772 Acked Read 1 Done Sent Done false
6604 2773 Acked Read 5 Done Sent Done false
6605 2774 Acked Flush 0 Done Sent Done false
6606 2775 Acked Read 1 Done Sent Done false
6607 2776 Acked Write 4 Done Sent Done false
6608 2777 Acked Read 4 Done Sent Done false
6609 2778 Acked Read 8 Done Sent Done false
6610 2779 Acked Write 5 Done Sent Done false
6611 2780 Acked Write 1 Done Sent Done false
6612 2781 Acked Write 2 Done Sent Done false
6613 2782 Acked Read 7 Done Sent Done false
6614 2783 Acked Write 6 Done Sent Done false
6615 2784 Acked Read 7 Done Sent Done false
6616 2785 Acked Read 10 Done Sent Done false
6617 2786 Acked Read 6 Done Sent Done false
6618 2787 Acked Read 9 Done Sent Done false
6619 2788 Acked Read 4 Done Sent Done false
6620 2789 Acked Read 1 Done Sent Done false
6621 2790 Acked Flush 0 Done Sent Done false
6622 2791 Acked Write 1 Done Sent Done false
6623 2792 Acked Write 9 Done Sent Done false
6624 2793 Acked Read 5 Done Sent Done false
6625 2794 Acked Read 9 Done Sent Done false
6626 2795 Acked Read 5 Done Sent Done false
6627 2796 Acked Read 2 Done Sent Done false
6628 2797 Acked Read 3 Done Sent Done false
6629 2798 Acked Read 10 Done Sent Done false
6630 2799 Acked Read 4 Done Sent Done false
6631 2800 Acked Read 4 Done Sent Done false
6632 2801 Acked Read 5 Done Sent Done false
6633 2802 Acked Read 7 Done Sent Done false
6634 2803 Acked Write 7 Done Sent Done false
6635 2804 Acked Read 2 Done Sent Done false
6636 2805 Acked Read 6 Done Sent Done false
6637 2806 Acked Read 9 Done Sent Done false
6638 2807 Acked Read 3 Done Sent Done false
6639 2808 Acked Flush 0 Done Sent Done false
6640 2809 Acked Write 8 Done Sent Done false
6641 2810 Acked Flush 0 Done Sent Done false
6642 STATES DS:0 DS:1 DS:2 TOTAL
6643 Sent 0 302 0 302
6644 Done 302 0 302 604
6645 Skipped 0 0 0 0
6646 Error 0 0 0 0
6647 Last Flush: 2810 2508 2810
6648 Downstairs last five completed: 2508 2507 2506 2505 2504
6649 Upstairs last five completed: 2810 2809 2808 2807 2806
6650 CLIENT: Up:0 ds:302 act:2
66512025-01-15T05:20:22.211ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
66522025-01-15T05:20:22.211ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59039) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
66532025-01-15T05:20:22.212ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
66542025-01-15T05:20:22.213ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
66552025-01-15T05:20:22.213ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
66562025-01-15T05:20:22.213ZINFOcrucible: send last flush ID to this DS: 2508 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
66572025-01-15T05:20:22.213ZINFOcrucible: Replied this last flush ID: 2508 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
66582025-01-15T05:20:22.213ZINFOcrucible: [1] client re-new 302 jobs since flush 2508 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
66592025-01-15T05:20:22.213ZINFOcrucible: [1] Replayed 302 jobs since flush: 2508 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
6660 ----------------------------------------------------------------
6661 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6662 Downstairs last five completed: 2810 2809 2808 2807 2806
6663 Upstairs last five completed: 2810 2809 2808 2807 2806
6664 CLIENT: Up:0 ds:0 act:3
6665 Replay: All jobs finished
6666 Main task has work to do, go find it
6667 got dsc Stop(0)
6668 stop 0
6669 Jan 15 05:20:24.442 INFO request completed, latency_us: 340, response_code: 204, uri: /stop/cid/0, [8810] Got stop action so:false kr:false
6670 method: POST, req_id: b52471b0-c615-4f79-a2c9-b2c56920adc5, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
66712025-01-15T05:20:24.442ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
6672 Jan 15 05:20:24.442 INFO request completed, latency_us: 119, response_code: 200{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:20:24.442767429Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
6673 , uri: /state/cid/0, method: GET, req_id: 1a6b7a24-12e6-449a-9382-1412853e467a, remote_addr: 127.0{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:20:24.442796386Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
6674 .0.1:42402, local_addr: 127.0.0.1:9998
66752025-01-15T05:20:24.442ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
6676 [8810] Exited with: ExitStatus(unix_wait_status(9))
6677 [8810][0] reports Exit
6678 Jan 15 05:20:28.445 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/0, method: GET, req_id: 4c28b140-3d30-4038-bfab-7e959937cfab, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
6679 001/300 Flush
6680 002/300 Write block 101 len 8192 data: 2 2
6681 003/300 Flush
6682 004/300 Read block 2928 len 24576
6683 005/300 Write block 3468 len 12288 data: 3 3 3
6684 006/300 Write block 750 len 20480 data: 2 3 3 3 2
6685 007/300 Write block 526 len 16384 data: 2 2 2 3
6686 008/300 Read block 154 len 36864
6687 009/300 Write block 1421 len 40960 data: 3 3 3 4 3 4 4 3 3 4
6688 010/300 Flush
6689 011/300 Read block 798 len 32768
6690 012/300 Read block 2426 len 28672
6691 013/300 Read block 2718 len 36864
6692 014/300 Flush
6693 015/300 Write block 4347 len 28672 data: 4 3 3 3 3 3 2
6694 016/300 Write block 3036 len 36864 data: 4 4 3 3 4 5 5 4 4
6695 017/300 Read block 219 len 40960
6696 018/300 Flush
6697 019/300 Read block 914 len 36864
6698 020/300 Read block 2343 len 20480
6699 021/300 Write block 1981 len 16384 data: 3 3 2 2
6700 022/300 Write block 321 len 32768 data: 3 3 3 3 3 3 3 3
6701 023/300 Read block 2837 len 36864
6702 024/300 Read block 2390 len 32768
6703 025/300 Read block 2853 len 16384
6704 026/300 Read block 1829 len 40960
6705 027/300 Write block 4004 len 12288 data: 3 4 3
6706 028/300 Write block 375 len 16384 data: 2 2 2 2
6707 029/300 Write block 1528 len 20480 data: 2 2 2 2 2
6708 030/300 Read block 127 len 4096
6709 031/300 Read block 2730 len 24576
6710 032/300 Read block 4165 len 8192
6711 033/300 Read block 4785 len 16384
6712 034/300 Write block 1666 len 8192 data: 2 2
6713 035/300 Read block 111 len 40960
6714 036/300 Read block 511 len 12288
6715 037/300 Read block 3887 len 16384
6716 038/300 Read block 3091 len 12288
6717 039/300 Write block 3728 len 16384 data: 2 2 2 2
6718 040/300 Flush
6719 041/300 Flush
6720 042/300 Write block 88 len 24576 data: 3 3 3 3 3 3
6721 043/300 Read block 701 len 12288
6722 044/300 Write block 2454 len 4096 data: 3
6723 045/300 Write block 379 len 24576 data: 2 2 2 2 2 2
6724 046/300 Write block 3748 len 32768 data: 2 2 2 2 2 2 2 2
6725 047/300 Read block 3050 len 20480
6726 048/300 Write block 1582 len 32768 data: 2 2 2 3 3 3 2 2
6727 049/300 Read block 849 len 20480
6728 050/300 Read block 820 len 28672
6729 051/300 Read block 2857 len 40960
6730 052/300 Read block 455 len 8192
6731 053/300 Write block 4365 len 16384 data: 3 2 2 3
6732 054/300 Write block 3039 len 8192 data: 4 5
6733 055/300 Flush
6734 056/300 Write block 3544 len 12288 data: 2 3 3
6735 057/300 Write block 3655 len 24576 data: 4 4 5 4 3 3
6736 058/300 Read block 4345 len 32768
6737 059/300 Write block 4897 len 24576 data: 4 5 4 5 4 4
6738 060/300 Write block 1039 len 12288 data: 3 3 2
6739 061/300 Read block 2379 len 40960
6740 062/300 Write block 744 len 4096 data: 4
6741 063/300 Write block 530 len 16384 data: 3 3 3 2
6742 064/300 Read block 1412 len 32768
6743 065/300 Read block 1391 len 24576
6744 066/300 Flush
6745 067/300 Read block 4490 len 12288
6746 068/300 Write block 490 len 24576 data: 4 3 2 2 2 2
6747 069/300 Read block 3509 len 40960
6748 070/300 Write block 1992 len 4096 data: 2
6749 071/300 Write block 1786 len 36864 data: 4 4 4 3 3 3 3 2 2
6750 072/300 Read block 4752 len 24576
6751 073/300 Read block 3099 len 32768
6752 074/300 Read block 896 len 12288
6753 075/300 Write block 3279 len 40960 data: 4 3 2 2 2 2 2 2 2 2
6754 076/300 Read block 180 len 24576
6755 077/300 Write block 4168 len 4096 data: 3
6756 078/300 Write block 4356 len 32768 data: 3 3 3 4 4 4 3 3
6757 079/300 Write block 4413 len 40960 data: 3 3 3 3 3 3 2 2 3 3
6758 080/300 Read block 2471 len 20480
6759 081/300 Write block 3841 len 20480 data: 2 2 2 5 4
6760 082/300 Read block 669 len 4096
6761 083/300 Read block 4082 len 32768
6762 084/300 Read block 149 len 16384
6763 085/300 Write block 3425 len 36864 data: 2 2 2 3 2 3 3 3 3
6764 086/300 Read block 4094 len 12288
6765 087/300 Read block 315 len 16384
6766 088/300 Write block 2920 len 24576 data: 4 4 4 4 3 3
6767 089/300 Write block 3976 len 8192 data: 3 2
6768 090/300 Read block 2551 len 16384
6769 091/300 Read block 2196 len 36864
6770 092/300 Write block 967 len 16384 data: 3 3 3 3
6771 093/300 Read block 2689 len 8192
6772 094/300 Write block 3894 len 36864 data: 4 3 2 2 2 2 2 2 2
6773 095/300 Read block 792 len 16384
6774 096/300 Write block 1317 len 4096 data: 2
6775 097/300 Write block 2220 len 40960 data: 2 2 2 3 2 2 3 3 3 3
6776 098/300 Write block 496 len 4096 data: 2
6777 099/300 Write block 4423 len 24576 data: 4 3 4 4 4 4
6778 100/300 Write block 1203 len 28672 data: 5 5 4 4 4 4 3
6779 101/300 Write block 2221 len 8192 data: 3 3
6780 102/300 Write block 2232 len 16384 data: 3 3 3 2
6781 103/300 Write block 1252 len 28672 data: 2 2 2 2 2 2 2
6782 104/300 Read block 1300 len 28672
6783 105/300 Write block 2595 len 4096 data: 4
6784 106/300 Write block 2623 len 28672 data: 2 2 3 3 3 3 3
6785 107/300 Write block 2597 len 40960 data: 2 2 2 2 2 2 2 3 3 2
6786 108/300 Write block 4241 len 24576 data: 4 3 3 3 4 4
6787 109/300 Read block 3816 len 4096
6788 110/300 Write block 3684 len 28672 data: 2 2 2 2 2 2 2
6789 111/300 Read block 3062 len 36864
6790 112/300 Read block 4936 len 32768
6791 113/300 Read block 3527 len 12288
6792 114/300 Read block 4043 len 12288
6793 115/300 Flush
6794 116/300 Read block 635 len 20480
6795 117/300 Flush
6796 118/300 Flush
6797 119/300 Write block 1531 len 8192 data: 3 3
6798 120/300 Flush
6799 121/300 Write block 2455 len 8192 data: 3 3
6800 122/300 Write block 2755 len 12288 data: 2 3 3
6801 123/300 Flush
6802 124/300 Write block 1951 len 20480 data: 3 3 2 2 2
6803 125/300 Write block 1843 len 40960 data: 5 5 5 4 4 3 3 2 3 3
6804 126/300 Read block 284 len 32768
6805 127/300 Write block 1036 len 20480 data: 3 3 3 4 4
6806 128/300 Read block 4390 len 20480
6807 129/300 Read block 4784 len 12288
6808 130/300 Read block 3573 len 12288
6809 131/300 Flush
6810 132/300 Write block 1633 len 16384 data: 3 3 3 3
6811 133/300 Write block 4329 len 16384 data: 3 3 3 3
6812 134/300 Read block 2690 len 28672
6813 135/300 Read block 990 len 36864
6814 136/300 Write block 2775 len 4096 data: 2
6815 137/300 Write block 4228 len 24576 data: 2 2 3 3 3 3
6816 138/300 Write block 4027 len 28672 data: 3 3 4 4 3 3 3
6817 139/300 Read block 4460 len 40960
6818 140/300 Write block 2301 len 4096 data: 2
6819 141/300 Read block 2674 len 32768
6820 142/300 Read block 1896 len 8192
6821 143/300 Write block 255 len 24576 data: 4 4 4 5 3 3
6822 144/300 Read block 4809 len 8192
6823 145/300 Read block 29 len 28672
6824 146/300 Read block 4792 len 28672
6825 147/300 Read block 3409 len 16384
6826 148/300 Read block 2786 len 20480
6827 149/300 Read block 386 len 8192
6828 150/300 Read block 3953 len 40960
6829 151/300 Write block 3741 len 24576 data: 3 3 3 3 3 3
6830 152/300 Write block 1408 len 4096 data: 3
6831 153/300 Read block 448 len 4096
6832 154/300 Write block 1016 len 24576 data: 4 3 3 2 2 2
6833 155/300 Read block 739 len 40960
6834 156/300 Write block 1091 len 8192 data: 2 2
6835 157/300 Write block 4732 len 24576 data: 5 4 4 3 2 2
6836 158/300 Read block 3642 len 28672
6837 159/300 Write block 3191 len 12288 data: 3 3 2
6838 160/300 Read block 2648 len 28672
6839 161/300 Read block 4659 len 20480
6840 162/300 Read block 1672 len 4096
6841 163/300 Write block 740 len 40960 data: 3 3 3 4 5 3 3 3 3 2
6842 164/300 Read block 1185 len 32768
6843 165/300 Write block 4159 len 12288 data: 3 3 3
6844 166/300 Write block 1920 len 20480 data: 3 3 3 3 3
6845 167/300 Flush
6846 168/300 Read block 4882 len 16384
6847 169/300 Read block 1154 len 24576
6848 170/300 Write block 266 len 24576 data: 3 3 3 3 3 3
6849 171/300 Flush
6850 172/300 Read block 1900 len 20480
6851 173/300 Write block 856 len 8192 data: 2 2
6852 174/300 Write block 3069 len 16384 data: 2 2 3 3
6853 175/300 Write block 4484 len 32768 data: 2 2 2 2 2 2 2 2
6854 176/300 Read block 2108 len 12288
6855 177/300 Write block 4226 len 36864 data: 2 2 3 3 4 4 4 4 3
6856 178/300 Write block 3056 len 40960 data: 4 5 5 3 4 3 3 3 4 4
6857 179/300 Write block 3735 len 24576 data: 2 2 2 3 3 3
6858 180/300 Read block 555 len 28672
6859 181/300 Read block 4280 len 28672
6860 182/300 Write block 3128 len 40960 data: 4 3 3 3 3 2 2 2 2 2
6861 183/300 Flush
6862 184/300 Flush
6863 185/300 Write block 3637 len 28672 data: 2 2 2 2 2 2 2
6864 186/300 Write block 130 len 40960 data: 4 4 3 3 3 2 2 2 2 2
6865 187/300 Read block 2379 len 24576
6866 188/300 Write block 4706 len 4096 data: 2
6867 189/300 Read block 3771 len 24576
6868 190/300 Read block 3912 len 24576
6869 191/300 Read block 2130 len 32768
6870 192/300 Write block 3498 len 32768 data: 3 2 2 2 2 2 2 3
6871 193/300 Read block 1456 len 36864
6872 194/300 Flush
6873 195/300 Read block 3196 len 8192
6874 196/300 Write block 4696 len 4096 data: 2
6875 197/300 Read block 1405 len 4096
6876 198/300 Write block 2614 len 8192 data: 3 3
6877 199/300 Write block 4552 len 12288 data: 6 5 4
6878 200/300 Flush
6879 201/300 Flush
6880 202/300 Read block 47 len 4096
6881 203/300 Flush
6882 204/300 Read block 1005 len 20480
6883 205/300 Write block 4067 len 8192 data: 2 2
6884 206/300 Read block 3263 len 28672
6885 207/300 Read block 3405 len 16384
6886 208/300 Read block 3325 len 40960
6887 209/300 Read block 323 len 32768
6888 210/300 Write block 2384 len 16384 data: 3 2 2 2
6889 211/300 Write block 4573 len 36864 data: 3 3 3 2 2 2 2 2 2
6890 212/300 Read block 5 len 32768
6891 213/300 Read block 4118 len 36864
6892 214/300 Write block 1783 len 40960 data: 3 3 3 5 5 5 4 4 4 4
6893 215/300 Write block 2597 len 16384 data: 3 3 3 3
6894 216/300 Write block 4737 len 16384 data: 3 2 3 3
6895 217/300 Write block 4401 len 12288 data: 3 3 3
6896 218/300 Read block 3237 len 24576
6897 219/300 Flush
6898 220/300 Read block 3353 len 16384
6899 221/300 Read block 2810 len 12288
6900 222/300 Write block 1182 len 8192 data: 2 2
6901 223/300 Flush
6902 224/300 Read block 682 len 12288
6903 225/300 Write block 3178 len 24576 data: 3 3 4 5 5 5
6904 226/300 Read block 3197 len 40960
6905 227/300 Write block 4210 len 8192 data: 2 2
6906 228/300 Write block 1983 len 12288 data: 3 3 2
6907 229/300 Read block 1837 len 40960
6908 230/300 Write block 2564 len 24576 data: 5 4 6 5 4 4
6909 231/300 Write block 4753 len 20480 data: 4 5 6 6 3
6910 232/300 Write block 3311 len 36864 data: 3 2 2 2 2 2 2 2 2
6911 233/300 Read block 736 len 4096
6912 234/300 Read block 1287 len 28672
6913 235/300 Write block 1953 len 28672 data: 3 3 3 2 2 2 2
6914 236/300 Read block 3772 len 36864
6915 237/300 Write block 1716 len 16384 data: 2 2 2 2
6916 238/300 Write block 352 len 40960 data: 3 4 4 4 4 5 5 4 3 3
6917 239/300 Read block 2545 len 32768
6918 240/300 Read block 1871 len 8192
6919 241/300 Read block 1530 len 24576
6920 242/300 Read block 170 len 20480
6921 243/300 Read block 3426 len 40960
6922 244/300 Read block 4041 len 12288
6923 245/300 Read block 4035 len 16384
6924 246/300 Write block 3163 len 28672 data: 3 3 3 4 3 2 2
6925 247/300 Write block 3576 len 36864 data: 3 3 3 3 3 3 3 3 3
6926 248/300 Read block 4890 len 12288
6927 249/300 Read block 4032 len 16384
6928 250/300 Read block 3661 len 12288
6929 251/300 Write block 842 len 16384 data: 3 3 2 2
6930 252/300 Flush
6931 253/300 Read block 837 len 12288
6932 254/300 Flush
6933 255/300 Read block 4041 len 16384
6934 256/300 Read block 1197 len 32768
6935 257/300 Read block 2875 len 12288
6936 258/300 Flush
6937 259/300 Read block 1222 len 24576
6938 260/300 Write block 2586 len 8192 data: 4 4
6939 261/300 Write block 4223 len 32768 data: 2 2 2 3 3 4 4 5
6940 262/300 Read block 4187 len 20480
6941 263/300 Write block 4476 len 36864 data: 2 2 2 2 2 2 2 2 3
6942 264/300 Read block 4373 len 36864
6943 265/300 Write block 2084 len 12288 data: 4 4 5
6944 266/300 Read block 4713 len 12288
6945 267/300 Write block 4493 len 28672 data: 3 3 4 4 4 4 4
6946 268/300 Read block 1956 len 40960
6947 269/300 Write block 910 len 20480 data: 2 2 2 2 2
6948 270/300 Read block 3862 len 36864
6949 271/300 Read block 4561 len 24576
6950 272/300 Read block 4783 len 36864
6951 273/300 Read block 4319 len 32768
6952 274/300 Flush
6953 275/300 Write block 4498 len 28672 data: 5 5 4 5 4 4 3
6954 276/300 Read block 1078 len 20480
6955 277/300 Read block 4656 len 8192
6956 278/300 Flush
6957 279/300 Read block 1051 len 28672
6958 280/300 Read block 1433 len 28672
6959 281/300 Read block 2398 len 24576
6960 282/300 Write block 3430 len 20480 data: 4 4 4 4 3
6961 283/300 Write block 1888 len 28672 data: 3 3 3 3 3 4 4
6962 284/300 Write block 4874 len 28672 data: 4 4 4 4 4 3 3
6963 285/300 Write block 1230 len 12288 data: 2 2 2
6964 286/300 Read block 3168 len 16384
6965 287/300 Read block 184 len 12288
6966 288/300 Write block 4834 len 28672 data: 2 2 2 2 2 2 2
6967 289/300 Write block 3453 len 32768 data: 3 3 3 3 3 2 2 2
6968 290/300 Read block 2097 len 12288
6969 291/300 Write block 2865 len 12288 data: 4 4 4
6970 292/300 Read block 3587 len 28672
6971 293/300 Read block 2636 len 24576
6972 294/300 Read block 1933 len 24576
6973 295/300 Read block 1242 len 40960
6974 296/300 Read block 3533 len 4096
6975 297/300 Read block 3698 len 32768
6976 298/300 Flush
6977 299/300 Read block 3473 len 36864
6978 300/300 Write block 1582 len 28672 data: 3 3 3 4 4 4 3
6979 301/300 Write block 881 len 32768 data: 2 2 2 2 2 2 2 2
6980 Main task has work to do, go find it
6981 got dsc Start(0)
6982 start 0
6983 Jan 15 05:20:28.676 INFO request completed, latency_us: 257, response_code: 204, uri: /start/cid/0, method: POST, req_id: f3bcef8e-d3ed-4c8a-99c3-c3543f19ec5e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
6984 Replay: started 0, returned:Ok(())
6985 ----------------------------------------------------------------
6986 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6987 States: Offline Active Active
6988 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6989 2811 Acked Flush 0 Sent Done Done false
6990 2812 Acked Write 2 Sent Done Done false
6991 2813 Acked Flush 0 Sent Done Done false
6992 2814 Acked Read 6 Sent Done Done false
6993 2815 Acked Write 3 Sent Done Done false
6994 2816 Acked Write 5 Sent Done Done false
6995 2817 Acked Write 4 Sent Done Done false
6996 2818 Acked Read 9 Sent Done Done false
6997 2819 Acked Write 10 Sent Done Done false
6998 2820 Acked Flush 0 Sent Done Done false
6999 2821 Acked Read 8 Sent Done Done false
7000 2822 Acked Read 7 Sent Done Done false
7001 2823 Acked Read 9 Sent Done Done false
7002 2824 Acked Flush 0 Sent Done Done false
7003 2825 Acked Write 7 Sent Done Done false
7004 2826 Acked Write 9 Sent Done Done false
7005 2827 Acked Read 10 Sent Done Done false
7006 2828 Acked Flush 0 Sent Done Done false
7007 2829 Acked Read 9 Sent Done Done false
7008 2830 Acked Read 5 Sent Done Done false
7009 2831 Acked Write 4 Sent Done Done false
7010 2832 Acked Write 8 Sent Done Done false
7011 2833 Acked Read 9 Sent Done Done false
7012 2834 Acked Read 8 Sent Done Done false
7013 2835 Acked Read 4 Sent Done Done false
7014 2836 Acked Read 10 Sent Done Done false
7015 2837 Acked Write 3 Sent Done Done false
7016 2838 Acked Write 4 Sent Done Done false
7017 2839 Acked Write 5 Sent Done Done false
7018 2840 Acked Read 1 Sent Done Done false
7019 2841 Acked Read 6 Sent Done Done false
7020 2842 Acked Read 2 Sent Done Done false
7021 2843 Acked Read 4 Sent Done Done false
7022 2844 Acked Write 2 Sent Done Done false
7023 2845 Acked Read 10 Sent Done Done false
7024 2846 Acked Read 3 Sent Done Done false
7025 2847 Acked Read 4 Sent Done Done false
7026 2848 Acked Read 3 Sent Done Done false
7027 2849 Acked Write 4 Sent Done Done false
7028 2850 Acked Flush 0 Sent Done Done false
7029 2851 Acked Flush 0 Sent Done Done false
7030 2852 Acked Write 6 Sent Done Done false
7031 2853 Acked Read 3 Sent Done Done false
7032 2854 Acked Write 1 Sent Done Done false
7033 2855 Acked Write 6 Sent Done Done false
7034 2856 Acked Write 8 Sent Done Done false
7035 2857 Acked Read 5 Sent Done Done false
7036 2858 Acked Write 8 Sent Done Done false
7037 2859 Acked Read 5 Sent Done Done false
7038 2860 Acked Read 7 Sent Done Done false
7039 2861 Acked Read 10 Sent Done Done false
7040 2862 Acked Read 2 Sent Done Done false
7041 2863 Acked Write 4 Sent Done Done false
7042 2864 Acked Write 2 Sent Done Done false
7043 2865 Acked Flush 0 Sent Done Done false
7044 2866 Acked Write 3 Sent Done Done false
7045 2867 Acked Write 6 Sent Done Done false
7046 2868 Acked Read 8 Sent Done Done false
7047 2869 Acked Write 6 Sent Done Done false
7048 2870 Acked Write 3 Sent Done Done false
7049 2871 Acked Read 10 Sent Done Done false
7050 2872 Acked Write 1 Sent Done Done false
7051 2873 Acked Write 4 Sent Done Done false
7052 2874 Acked Read 8 Sent Done Done false
7053 2875 Acked Read 6 Sent Done Done false
7054 2876 Acked Flush 0 Sent Done Done false
7055 2877 Acked Read 3 Sent Done Done false
7056 2878 Acked Write 6 Sent Done Done false
7057 2879 Acked Read 10 Sent Done Done false
7058 2880 Acked Write 1 Sent Done Done false
7059 2881 Acked Write 9 Sent Done Done false
7060 2882 Acked Read 6 Sent Done Done false
7061 2883 Acked Read 8 Sent Done Done false
7062 2884 Acked Read 3 Sent Done Done false
7063 2885 Acked Write 10 Sent Done Done false
7064 2886 Acked Read 6 Sent Done Done false
7065 2887 Acked Write 1 Sent Done Done false
7066 2888 Acked Write 8 Sent Done Done false
7067 2889 Acked Write 10 Sent Done Done false
7068 2890 Acked Read 5 Sent Done Done false
7069 2891 Acked Write 5 Sent Done Done false
7070 2892 Acked Read 1 Sent Done Done false
7071 2893 Acked Read 8 Sent Done Done false
7072 2894 Acked Read 4 Sent Done Done false
7073 2895 Acked Write 9 Sent Done Done false
7074 2896 Acked Read 3 Sent Done Done false
7075 2897 Acked Read 4 Sent Done Done false
7076 2898 Acked Write 6 Sent Done Done false
7077 2899 Acked Write 2 Sent Done Done false
7078 2900 Acked Read 4 Sent Done Done false
7079 2901 Acked Read 9 Sent Done Done false
7080 2902 Acked Write 4 Sent Done Done false
7081 2903 Acked Read 2 Sent Done Done false
7082 2904 Acked Write 9 Sent Done Done false
7083 2905 Acked Read 4 Sent Done Done false
7084 2906 Acked Write 1 Sent Done Done false
7085 2907 Acked Write 10 Sent Done Done false
7086 2908 Acked Write 1 Sent Done Done false
7087 2909 Acked Write 6 Sent Done Done false
7088 2910 Acked Write 7 Sent Done Done false
7089 2911 Acked Write 2 Sent Done Done false
7090 2912 Acked Write 4 Sent Done Done false
7091 2913 Acked Write 7 Sent Done Done false
7092 2914 Acked Read 7 Sent Done Done false
7093 2915 Acked Write 1 Sent Done Done false
7094 2916 Acked Write 7 Sent Done Done false
7095 2917 Acked Write 10 Sent Done Done false
7096 2918 Acked Write 6 Sent Done Done false
7097 2919 Acked Read 1 Sent Done Done false
7098 2920 Acked Write 7 Sent Done Done false
7099 2921 Acked Read 9 Sent Done Done false
7100 2922 Acked Read 8 Sent Done Done false
7101 2923 Acked Read 3 Sent Done Done false
7102 2924 Acked Read 3 Sent Done Done false
7103 2925 Acked Flush 0 Sent Done Done false
7104 2926 Acked Read 5 Sent Done Done false
7105 2927 Acked Flush 0 Sent Done Done false
7106 2928 Acked Flush 0 Sent Done Done false
7107 2929 Acked Write 2 Sent Done Done false
7108 2930 Acked Flush 0 Sent Done Done false
7109 2931 Acked Write 2 Sent Done Done false
7110 2932 Acked Write 3 Sent Done Done false
7111 2933 Acked Flush 0 Sent Done Done false
7112 2934 Acked Write 5 Sent Done Done false
7113 2935 Acked Write 10 Sent Done Done false
7114 2936 Acked Read 8 Sent Done Done false
7115 2937 Acked Write 5 Sent Done Done false
7116 2938 Acked Read 5 Sent Done Done false
7117 2939 Acked Read 3 Sent Done Done false
7118 2940 Acked Read 3 Sent Done Done false
7119 2941 Acked Flush 0 Sent Done Done false
7120 2942 Acked Write 4 Sent Done Done false
7121 2943 Acked Write 4 Sent Done Done false
7122 2944 Acked Read 7 Sent Done Done false
7123 2945 Acked Read 9 Sent Done Done false
7124 2946 Acked Write 1 Sent Done Done false
7125 2947 Acked Write 6 Sent Done Done false
7126 2948 Acked Write 7 Sent Done Done false
7127 2949 Acked Read 10 Sent Done Done false
7128 2950 Acked Write 1 Sent Done Done false
7129 2951 Acked Read 8 Sent Done Done false
7130 2952 Acked Read 2 Sent Done Done false
7131 2953 Acked Write 6 Sent Done Done false
7132 2954 Acked Read 2 Sent Done Done false
7133 2955 Acked Read 7 Sent Done Done false
7134 2956 Acked Read 7 Sent Done Done false
7135 2957 Acked Read 4 Sent Done Done false
7136 2958 Acked Read 5 Sent Done Done false
7137 2959 Acked Read 2 Sent Done Done false
7138 2960 Acked Read 10 Sent Done Done false
7139 2961 Acked Write 6 Sent Done Done false
7140 2962 Acked Write 1 Sent Done Done false
7141 2963 Acked Read 1 Sent Done Done false
7142 2964 Acked Write 6 Sent Done Done false
7143 2965 Acked Read 10 Sent Done Done false
7144 2966 Acked Write 2 Sent Done Done false
7145 2967 Acked Write 6 Sent Done Done false
7146 2968 Acked Read 7 Sent Done Done false
7147 2969 Acked Write 3 Sent Done Done false
7148 2970 Acked Read 7 Sent Done Done false
7149 2971 Acked Read 5 Sent Done Done false
7150 2972 Acked Read 1 Sent Done Done false
7151 2973 Acked Write 10 Sent Done Done false
7152 2974 Acked Read 8 Sent Done Done false
7153 2975 Acked Write 3 Sent Done Done false
7154 2976 Acked Write 5 Sent Done Done false
7155 2977 Acked Flush 0 Sent Done Done false
7156 2978 Acked Read 4 Sent Done Done false
7157 2979 Acked Read 6 Sent Done Done false
7158 2980 Acked Write 6 Sent Done Done false
7159 2981 Acked Flush 0 Sent Done Done false
7160 2982 Acked Read 5 Sent Done Done false
7161 2983 Acked Write 2 Sent Done Done false
7162 2984 Acked Write 4 Sent Done Done false
7163 2985 Acked Write 8 Sent Done Done false
7164 2986 Acked Read 3 Sent Done Done false
7165 2987 Acked Write 9 Sent Done Done false
7166 2988 Acked Write 10 Sent Done Done false
7167 2989 Acked Write 6 Sent Done Done false
7168 2990 Acked Read 7 Sent Done Done false
7169 2991 Acked Read 7 Sent Done Done false
7170 2992 Acked Write 10 Sent Done Done false
7171 2993 Acked Flush 0 Sent Done Done false
7172 2994 Acked Flush 0 Sent Done Done false
7173 2995 Acked Write 7 Sent Done Done false
7174 2996 Acked Write 10 Sent Done Done false
7175 2997 Acked Read 6 Sent Done Done false
7176 2998 Acked Write 1 Sent Done Done false
7177 2999 Acked Read 6 Sent Done Done false
7178 3000 Acked Read 6 Sent Done Done false
7179 3001 Acked Read 8 Sent Done Done false
7180 3002 Acked Write 8 Sent Done Done false
7181 3003 Acked Read 9 Sent Done Done false
7182 3004 Acked Flush 0 Sent Done Done false
7183 3005 Acked Read 2 Sent Done Done false
7184 3006 Acked Write 1 Sent Done Done false
7185 3007 Acked Read 1 Sent Done Done false
7186 3008 Acked Write 2 Sent Done Done false
7187 3009 Acked Write 3 Sent Done Done false
7188 3010 Acked Flush 0 Sent Done Done false
7189 3011 Acked Flush 0 Sent Done Done false
7190 3012 Acked Read 1 Sent Done Done false
7191 3013 Acked Flush 0 Sent Done Done false
7192 3014 Acked Read 5 Sent Done Done false
7193 3015 Acked Write 2 Sent Done Done false
7194 3016 Acked Read 7 Sent Done Done false
7195 3017 Acked Read 4 Sent Done Done false
7196 3018 Acked Read 10 Sent Done Done false
7197 3019 Acked Read 8 Sent Done Done false
7198 3020 Acked Write 4 Sent Done Done false
7199 3021 Acked Write 9 Sent Done Done false
7200 3022 Acked Read 8 Sent Done Done false
7201 3023 Acked Read 9 Sent Done Done false
7202 3024 Acked Write 10 Sent Done Done false
7203 3025 Acked Write 4 Sent Done Done false
7204 3026 Acked Write 4 Sent Done Done false
7205 3027 Acked Write 3 Sent Done Done false
7206 3028 Acked Read 6 Sent Done Done false
7207 3029 Acked Flush 0 Sent Done Done false
7208 3030 Acked Read 4 Sent Done Done false
7209 3031 Acked Read 3 Sent Done Done false
7210 3032 Acked Write 2 Sent Done Done false
7211 3033 Acked Flush 0 Sent Done Done false
7212 3034 Acked Read 3 Sent Done Done false
7213 3035 Acked Write 6 Sent Done Done false
7214 3036 Acked Read 10 Sent Done Done false
7215 3037 Acked Write 2 Sent Done Done false
7216 3038 Acked Write 3 Sent Done Done false
7217 3039 Acked Read 10 Sent Done Done false
7218 3040 Acked Write 6 Sent Done Done false
7219 3041 Acked Write 5 Sent Done Done false
7220 3042 Acked Write 9 Sent Done Done false
7221 3043 Acked Read 1 Sent Done Done false
7222 3044 Acked Read 7 Sent Done Done false
7223 3045 Acked Write 7 Sent Done Done false
7224 3046 Acked Read 9 Sent Done Done false
7225 3047 Acked Write 4 Sent Done Done false
7226 3048 Acked Write 10 Sent Done Done false
7227 3049 Acked Read 8 Sent Done Done false
7228 3050 Acked Read 2 Sent Done Done false
7229 3051 Acked Read 6 Sent Done Done false
7230 3052 Acked Read 5 Sent Done Done false
7231 3053 Acked Read 10 Sent Done Done false
7232 3054 Acked Read 3 Sent Done Done false
7233 3055 Acked Read 4 Sent Done Done false
7234 3056 Acked Write 7 Sent Done Done false
7235 3057 Acked Write 9 Sent Done Done false
7236 3058 Acked Read 3 Sent Done Done false
7237 3059 Acked Read 4 Sent Done Done false
7238 3060 Acked Read 3 Sent Done Done false
7239 3061 Acked Write 4 Sent Done Done false
7240 3062 Acked Flush 0 Sent Done Done false
7241 3063 Acked Read 3 Sent Done Done false
7242 3064 Acked Flush 0 Sent Done Done false
7243 3065 Acked Read 4 Sent Done Done false
7244 3066 Acked Read 8 Sent Done Done false
7245 3067 Acked Read 3 Sent Done Done false
7246 3068 Acked Flush 0 Sent Done Done false
7247 3069 Acked Read 6 Sent Done Done false
7248 3070 Acked Write 2 Sent Done Done false
7249 3071 Acked Write 8 Sent Done Done false
7250 3072 Acked Read 5 Sent Done Done false
7251 3073 Acked Write 9 Sent Done Done false
7252 3074 Acked Read 9 Sent Done Done false
7253 3075 Acked Write 3 Sent Done Done false
7254 3076 Acked Read 3 Sent Done Done false
7255 3077 Acked Write 7 Sent Done Done false
7256 3078 Acked Read 10 Sent Done Done false
7257 3079 Acked Write 5 Sent Done Done false
7258 3080 Acked Read 9 Sent Done Done false
7259 3081 Acked Read 6 Sent Done Done false
7260 3082 Acked Read 9 Sent Done Done false
7261 3083 Acked Read 8 Sent Done Done false
7262 3084 Acked Flush 0 Sent Done Done false
7263 3085 Acked Write 7 Sent Done Done false
7264 3086 Acked Read 5 Sent Done Done false
7265 3087 Acked Read 2 Sent Done Done false
7266 3088 Acked Flush 0 Sent Done Done false
7267 3089 Acked Read 7 Sent Done Done false
7268 3090 Acked Read 7 Sent Done Done false
7269 3091 Acked Read 6 Sent Done Done false
7270 3092 Acked Write 5 Sent Done Done false
7271 3093 Acked Write 7 Sent Done Done false
7272 3094 Acked Write 7 Sent Done Done false
7273 3095 Acked Write 3 Sent Done Done false
7274 3096 Acked Read 4 Sent Done Done false
7275 3097 Acked Read 3 Sent Done Done false
7276 3098 Acked Write 7 Sent Done Done false
7277 3099 Acked Write 8 Sent Done Done false
7278 3100 Acked Read 3 Sent Done Done false
7279 3101 Acked Write 3 Sent Done Done false
7280 3102 Acked Read 7 Sent Done Done false
7281 3103 Acked Read 6 Sent Done Done false
7282 3104 Acked Read 6 Sent Done Done false
7283 3105 Acked Read 10 Sent Done Done false
7284 3106 Acked Read 1 Sent Done Done false
7285 3107 Acked Read 8 Sent Done Done false
7286 3108 Acked Flush 0 Sent Done Done false
7287 3109 Acked Read 9 Sent Done Done false
7288 3110 Acked Write 7 Sent Done Done false
7289 3111 Acked Write 8 Sent Done Done false
7290 STATES DS:0 DS:1 DS:2 TOTAL
7291 Sent 301 0 0 301
7292 Done 0 301 301 602
7293 Skipped 0 0 0 0
7294 Error 0 0 0 0
7295 Last Flush: 2810 3108 3108
7296 Downstairs last five completed: 2810 2809 2808 2807 2806
7297 Upstairs last five completed: 3111 3110 3109 3108 3107
7298 CLIENT: Up:0 ds:301 act:2
7299 ----------------------------------------------------------------
7300 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7301 States: Offline Active Active
7302 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7303 2811 Acked Flush 0 Sent Done Done false
7304 2812 Acked Write 2 Sent Done Done false
7305 2813 Acked Flush 0 Sent Done Done false
7306 2814 Acked Read 6 Sent Done Done false
7307 2815 Acked Write 3 Sent Done Done false
7308 2816 Acked Write 5 Sent Done Done false
7309 2817 Acked Write 4 Sent Done Done false
7310 2818 Acked Read 9 Sent Done Done false
7311 2819 Acked Write 10 Sent Done Done false
7312 2820 Acked Flush 0 Sent Done Done false
7313 2821 Acked Read 8 Sent Done Done false
7314 2822 Acked Read 7 Sent Done Done false
7315 2823 Acked Read 9 Sent Done Done false
7316 2824 Acked Flush 0 Sent Done Done false
7317 2825 Acked Write 7 Sent Done Done false
7318 2826 Acked Write 9 Sent Done Done false
7319 2827 Acked Read 10 Sent Done Done false
7320 2828 Acked Flush 0 Sent Done Done false
7321 2829 Acked Read 9 Sent Done Done false
7322 2830 Acked Read 5 Sent Done Done false
7323 2831 Acked Write 4 Sent Done Done false
7324 2832 Acked Write 8 Sent Done Done false
7325 2833 Acked Read 9 Sent Done Done false
7326 2834 Acked Read 8 Sent Done Done false
7327 2835 Acked Read 4 Sent Done Done false
7328 2836 Acked Read 10 Sent Done Done false
7329 2837 Acked Write 3 Sent Done Done false
7330 2838 Acked Write 4 Sent Done Done false
7331 2839 Acked Write 5 Sent Done Done false
7332 2840 Acked Read 1 Sent Done Done false
7333 2841 Acked Read 6 Sent Done Done false
7334 2842 Acked Read 2 Sent Done Done false
7335 2843 Acked Read 4 Sent Done Done false
7336 2844 Acked Write 2 Sent Done Done false
7337 2845 Acked Read 10 Sent Done Done false
7338 2846 Acked Read 3 Sent Done Done false
7339 2847 Acked Read 4 Sent Done Done false
7340 2848 Acked Read 3 Sent Done Done false
7341 2849 Acked Write 4 Sent Done Done false
7342 2850 Acked Flush 0 Sent Done Done false
7343 2851 Acked Flush 0 Sent Done Done false
7344 2852 Acked Write 6 Sent Done Done false
7345 2853 Acked Read 3 Sent Done Done false
7346 2854 Acked Write 1 Sent Done Done false
7347 2855 Acked Write 6 Sent Done Done false
7348 2856 Acked Write 8 Sent Done Done false
7349 2857 Acked Read 5 Sent Done Done false
7350 2858 Acked Write 8 Sent Done Done false
7351 2859 Acked Read 5 Sent Done Done false
7352 2860 Acked Read 7 Sent Done Done false
7353 2861 Acked Read 10 Sent Done Done false
7354 2862 Acked Read 2 Sent Done Done false
7355 2863 Acked Write 4 Sent Done Done false
7356 2864 Acked Write 2 Sent Done Done false
7357 2865 Acked Flush 0 Sent Done Done false
7358 2866 Acked Write 3 Sent Done Done false
7359 2867 Acked Write 6 Sent Done Done false
7360 2868 Acked Read 8 Sent Done Done false
7361 2869 Acked Write 6 Sent Done Done false
7362 2870 Acked Write 3 Sent Done Done false
7363 2871 Acked Read 10 Sent Done Done false
7364 2872 Acked Write 1 Sent Done Done false
7365 2873 Acked Write 4 Sent Done Done false
7366 2874 Acked Read 8 Sent Done Done false
7367 2875 Acked Read 6 Sent Done Done false
7368 2876 Acked Flush 0 Sent Done Done false
7369 2877 Acked Read 3 Sent Done Done false
7370 2878 Acked Write 6 Sent Done Done false
7371 2879 Acked Read 10 Sent Done Done false
7372 2880 Acked Write 1 Sent Done Done false
7373 2881 Acked Write 9 Sent Done Done false
7374 2882 Acked Read 6 Sent Done Done false
7375 2883 Acked Read 8 Sent Done Done false
7376 2884 Acked Read 3 Sent Done Done false
7377 2885 Acked Write 10 Sent Done Done false
7378 2886 Acked Read 6 Sent Done Done false
7379 2887 Acked Write 1 Sent Done Done false
7380 2888 Acked Write 8 Sent Done Done false
7381 2889 Acked Write 10 Sent Done Done false
7382 2890 Acked Read 5 Sent Done Done false
7383 2891 Acked Write 5 Sent Done Done false
7384 2892 Acked Read 1 Sent Done Done false
7385 2893 Acked Read 8 Sent Done Done false
7386 2894 Acked Read 4 Sent Done Done false
7387 2895 Acked Write 9 Sent Done Done false
7388 2896 Acked Read 3 Sent Done Done false
7389 2897 Acked Read 4 Sent Done Done false
7390 2898 Acked Write 6 Sent Done Done false
7391 2899 Acked Write 2 Sent Done Done false
7392 2900 Acked Read 4 Sent Done Done false
7393 2901 Acked Read 9 Sent Done Done false
7394 2902 Acked Write 4 Sent Done Done false
7395 2903 Acked Read 2 Sent Done Done false
7396 2904 Acked Write 9 Sent Done Done false
7397 2905 Acked Read 4 Sent Done Done false
7398 2906 Acked Write 1 Sent Done Done false
7399 2907 Acked Write 10 Sent Done Done false
7400 2908 Acked Write 1 Sent Done Done false
7401 2909 Acked Write 6 Sent Done Done false
7402 2910 Acked Write 7 Sent Done Done false
7403 2911 Acked Write 2 Sent Done Done false
7404 2912 Acked Write 4 Sent Done Done false
7405 2913 Acked Write 7 Sent Done Done false
7406 2914 Acked Read 7 Sent Done Done false
7407 2915 Acked Write 1 Sent Done Done false
7408 2916 Acked Write 7 Sent Done Done false
7409 2917 Acked Write 10 Sent Done Done false
7410 2918 Acked Write 6 Sent Done Done false
7411 2919 Acked Read 1 Sent Done Done false
7412 2920 Acked Write 7 Sent Done Done false
7413 2921 Acked Read 9 Sent Done Done false
7414 2922 Acked Read 8 Sent Done Done false
7415 2923 Acked Read 3 Sent Done Done false
7416 2924 Acked Read 3 Sent Done Done false
7417 2925 Acked Flush 0 Sent Done Done false
7418 2926 Acked Read 5 Sent Done Done false
7419 2927 Acked Flush 0 Sent Done Done false
7420 2928 Acked Flush 0 Sent Done Done false
7421 2929 Acked Write 2 Sent Done Done false
7422 2930 Acked Flush 0 Sent Done Done false
7423 2931 Acked Write 2 Sent Done Done false
7424 2932 Acked Write 3 Sent Done Done false
7425 2933 Acked Flush 0 Sent Done Done false
7426 2934 Acked Write 5 Sent Done Done false
7427 2935 Acked Write 10 Sent Done Done false
7428 2936 Acked Read 8 Sent Done Done false
7429 2937 Acked Write 5 Sent Done Done false
7430 2938 Acked Read 5 Sent Done Done false
7431 2939 Acked Read 3 Sent Done Done false
7432 2940 Acked Read 3 Sent Done Done false
7433 2941 Acked Flush 0 Sent Done Done false
7434 2942 Acked Write 4 Sent Done Done false
7435 2943 Acked Write 4 Sent Done Done false
7436 2944 Acked Read 7 Sent Done Done false
7437 2945 Acked Read 9 Sent Done Done false
7438 2946 Acked Write 1 Sent Done Done false
7439 2947 Acked Write 6 Sent Done Done false
7440 2948 Acked Write 7 Sent Done Done false
7441 2949 Acked Read 10 Sent Done Done false
7442 2950 Acked Write 1 Sent Done Done false
7443 2951 Acked Read 8 Sent Done Done false
7444 2952 Acked Read 2 Sent Done Done false
7445 2953 Acked Write 6 Sent Done Done false
7446 2954 Acked Read 2 Sent Done Done false
7447 2955 Acked Read 7 Sent Done Done false
7448 2956 Acked Read 7 Sent Done Done false
7449 2957 Acked Read 4 Sent Done Done false
7450 2958 Acked Read 5 Sent Done Done false
7451 2959 Acked Read 2 Sent Done Done false
7452 2960 Acked Read 10 Sent Done Done false
7453 2961 Acked Write 6 Sent Done Done false
7454 2962 Acked Write 1 Sent Done Done false
7455 2963 Acked Read 1 Sent Done Done false
7456 2964 Acked Write 6 Sent Done Done false
7457 2965 Acked Read 10 Sent Done Done false
7458 2966 Acked Write 2 Sent Done Done false
7459 2967 Acked Write 6 Sent Done Done false
7460 2968 Acked Read 7 Sent Done Done false
7461 2969 Acked Write 3 Sent Done Done false
7462 2970 Acked Read 7 Sent Done Done false
7463 2971 Acked Read 5 Sent Done Done false
7464 2972 Acked Read 1 Sent Done Done false
7465 2973 Acked Write 10 Sent Done Done false
7466 2974 Acked Read 8 Sent Done Done false
7467 2975 Acked Write 3 Sent Done Done false
7468 2976 Acked Write 5 Sent Done Done false
7469 2977 Acked Flush 0 Sent Done Done false
7470 2978 Acked Read 4 Sent Done Done false
7471 2979 Acked Read 6 Sent Done Done false
7472 2980 Acked Write 6 Sent Done Done false
7473 2981 Acked Flush 0 Sent Done Done false
7474 2982 Acked Read 5 Sent Done Done false
7475 2983 Acked Write 2 Sent Done Done false
7476 2984 Acked Write 4 Sent Done Done false
7477 2985 Acked Write 8 Sent Done Done false
7478 2986 Acked Read 3 Sent Done Done false
7479 2987 Acked Write 9 Sent Done Done false
7480 2988 Acked Write 10 Sent Done Done false
7481 2989 Acked Write 6 Sent Done Done false
7482 2990 Acked Read 7 Sent Done Done false
7483 2991 Acked Read 7 Sent Done Done false
7484 2992 Acked Write 10 Sent Done Done false
7485 2993 Acked Flush 0 Sent Done Done false
7486 2994 Acked Flush 0 Sent Done Done false
7487 2995 Acked Write 7 Sent Done Done false
7488 2996 Acked Write 10 Sent Done Done false
7489 2997 Acked Read 6 Sent Done Done false
7490 2998 Acked Write 1 Sent Done Done false
7491 2999 Acked Read 6 Sent Done Done false
7492 3000 Acked Read 6 Sent Done Done false
7493 3001 Acked Read 8 Sent Done Done false
7494 3002 Acked Write 8 Sent Done Done false
7495 3003 Acked Read 9 Sent Done Done false
7496 3004 Acked Flush 0 Sent Done Done false
7497 3005 Acked Read 2 Sent Done Done false
7498 3006 Acked Write 1 Sent Done Done false
7499 3007 Acked Read 1 Sent Done Done false
7500 3008 Acked Write 2 Sent Done Done false
7501 3009 Acked Write 3 Sent Done Done false
7502 3010 Acked Flush 0 Sent Done Done false
7503 3011 Acked Flush 0 Sent Done Done false
7504 3012 Acked Read 1 Sent Done Done false
7505 3013 Acked Flush 0 Sent Done Done false
7506 3014 Acked Read 5 Sent Done Done false
7507 3015 Acked Write 2 Sent Done Done false
7508 3016 Acked Read 7 Sent Done Done false
7509 3017 Acked Read 4 Sent Done Done false
7510 3018 Acked Read 10 Sent Done Done false
7511 3019 Acked Read 8 Sent Done Done false
7512 3020 Acked Write 4 Sent Done Done false
7513 3021 Acked Write 9 Sent Done Done false
7514 3022 Acked Read 8 Sent Done Done false
7515 3023 Acked Read 9 Sent Done Done false
7516 3024 Acked Write 10 Sent Done Done false
7517 3025 Acked Write 4 Sent Done Done false
7518 3026 Acked Write 4 Sent Done Done false
7519 3027 Acked Write 3 Sent Done Done false
7520 3028 Acked Read 6 Sent Done Done false
7521 3029 Acked Flush 0 Sent Done Done false
7522 3030 Acked Read 4 Sent Done Done false
7523 3031 Acked Read 3 Sent Done Done false
7524 3032 Acked Write 2 Sent Done Done false
7525 3033 Acked Flush 0 Sent Done Done false
7526 3034 Acked Read 3 Sent Done Done false
7527 3035 Acked Write 6 Sent Done Done false
7528 3036 Acked Read 10 Sent Done Done false
7529 3037 Acked Write 2 Sent Done Done false
7530 3038 Acked Write 3 Sent Done Done false
7531 3039 Acked Read 10 Sent Done Done false
7532 3040 Acked Write 6 Sent Done Done false
7533 3041 Acked Write 5 Sent Done Done false
7534 3042 Acked Write 9 Sent Done Done false
7535 3043 Acked Read 1 Sent Done Done false
7536 3044 Acked Read 7 Sent Done Done false
7537 3045 Acked Write 7 Sent Done Done false
7538 3046 Acked Read 9 Sent Done Done false
7539 3047 Acked Write 4 Sent Done Done false
7540 3048 Acked Write 10 Sent Done Done false
7541 3049 Acked Read 8 Sent Done Done false
7542 3050 Acked Read 2 Sent Done Done false
7543 3051 Acked Read 6 Sent Done Done false
7544 3052 Acked Read 5 Sent Done Done false
7545 3053 Acked Read 10 Sent Done Done false
7546 3054 Acked Read 3 Sent Done Done false
7547 3055 Acked Read 4 Sent Done Done false
7548 3056 Acked Write 7 Sent Done Done false
7549 3057 Acked Write 9 Sent Done Done false
7550 3058 Acked Read 3 Sent Done Done false
7551 3059 Acked Read 4 Sent Done Done false
7552 3060 Acked Read 3 Sent Done Done false
7553 3061 Acked Write 4 Sent Done Done false
7554 3062 Acked Flush 0 Sent Done Done false
7555 3063 Acked Read 3 Sent Done Done false
7556 3064 Acked Flush 0 Sent Done Done false
7557 3065 Acked Read 4 Sent Done Done false
7558 3066 Acked Read 8 Sent Done Done false
7559 3067 Acked Read 3 Sent Done Done false
7560 3068 Acked Flush 0 Sent Done Done false
7561 3069 Acked Read 6 Sent Done Done false
7562 3070 Acked Write 2 Sent Done Done false
7563 3071 Acked Write 8 Sent Done Done false
7564 3072 Acked Read 5 Sent Done Done false
7565 3073 Acked Write 9 Sent Done Done false
7566 3074 Acked Read 9 Sent Done Done false
7567 3075 Acked Write 3 Sent Done Done false
7568 3076 Acked Read 3 Sent Done Done false
7569 3077 Acked Write 7 Sent Done Done false
7570 3078 Acked Read 10 Sent Done Done false
7571 3079 Acked Write 5 Sent Done Done false
7572 3080 Acked Read 9 Sent Done Done false
7573 3081 Acked Read 6 Sent Done Done false
7574 3082 Acked Read 9 Sent Done Done false
7575 3083 Acked Read 8 Sent Done Done false
7576 3084 Acked Flush 0 Sent Done Done false
7577 3085 Acked Write 7 Sent Done Done false
7578 3086 Acked Read 5 Sent Done Done false
7579 3087 Acked Read 2 Sent Done Done false
7580 3088 Acked Flush 0 Sent Done Done false
7581 3089 Acked Read 7 Sent Done Done false
7582 3090 Acked Read 7 Sent Done Done false
7583 3091 Acked Read 6 Sent Done Done false
7584 3092 Acked Write 5 Sent Done Done false
7585 3093 Acked Write 7 Sent Done Done false
7586 3094 Acked Write 7 Sent Done Done false
7587 3095 Acked Write 3 Sent Done Done false
7588 3096 Acked Read 4 Sent Done Done false
7589 3097 Acked Read 3 Sent Done Done false
7590 3098 Acked Write 7 Sent Done Done false
7591 3099 Acked Write 8 Sent Done Done false
7592 3100 Acked Read 3 Sent Done Done false
7593 3101 Acked Write 3 Sent Done Done false
7594 3102 Acked Read 7 Sent Done Done false
7595 3103 Acked Read 6 Sent Done Done false
7596 3104 Acked Read 6 Sent Done Done false
7597 3105 Acked Read 10 Sent Done Done false
7598 3106 Acked Read 1 Sent Done Done false
7599 3107 Acked Read 8 Sent Done Done false
7600 3108 Acked Flush 0 Sent Done Done false
7601 3109 Acked Read 9 Sent Done Done false
7602 3110 Acked Write 7 Sent Done Done false
7603 3111 Acked Write 8 Sent Done Done false
7604 3112 Acked Flush 0 Sent Done Done false
7605 STATES DS:0 DS:1 DS:2 TOTAL
7606 Sent 302 0 0 302
7607 Done 0 302 302 604
7608 Skipped 0 0 0 0
7609 Error 0 0 0 0
7610 Last Flush: 2810 3112 3112
7611 Downstairs last five completed: 2810 2809 2808 2807 2806
7612 Upstairs last five completed: 3112 3111 3110 3109 3108
7613 CLIENT: Up:0 ds:302 act:2
76142025-01-15T05:20:34.443ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
76152025-01-15T05:20:34.443ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
76162025-01-15T05:20:34.443ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
76172025-01-15T05:20:34.443ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
76182025-01-15T05:20:34.443ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
7619 [8810] got start action
7620 [8810] I am going to restart
7621 Starting downstairs at port 8810
7622 Make output file at "/tmp/dsc/downstairs-8810.txt"
7623 [8810][0] reports Starting
7624 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1209)
7625 [8810][0] reports Running
7626 ----------------------------------------------------------------
7627 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7628 States: Offline Active Active
7629 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7630 2811 Acked Flush 0 Sent Done Done false
7631 2812 Acked Write 2 Sent Done Done false
7632 2813 Acked Flush 0 Sent Done Done false
7633 2814 Acked Read 6 Sent Done Done false
7634 2815 Acked Write 3 Sent Done Done false
7635 2816 Acked Write 5 Sent Done Done false
7636 2817 Acked Write 4 Sent Done Done false
7637 2818 Acked Read 9 Sent Done Done false
7638 2819 Acked Write 10 Sent Done Done false
7639 2820 Acked Flush 0 Sent Done Done false
7640 2821 Acked Read 8 Sent Done Done false
7641 2822 Acked Read 7 Sent Done Done false
7642 2823 Acked Read 9 Sent Done Done false
7643 2824 Acked Flush 0 Sent Done Done false
7644 2825 Acked Write 7 Sent Done Done false
7645 2826 Acked Write 9 Sent Done Done false
7646 2827 Acked Read 10 Sent Done Done false
7647 2828 Acked Flush 0 Sent Done Done false
7648 2829 Acked Read 9 Sent Done Done false
7649 2830 Acked Read 5 Sent Done Done false
7650 2831 Acked Write 4 Sent Done Done false
7651 2832 Acked Write 8 Sent Done Done false
7652 2833 Acked Read 9 Sent Done Done false
7653 2834 Acked Read 8 Sent Done Done false
7654 2835 Acked Read 4 Sent Done Done false
7655 2836 Acked Read 10 Sent Done Done false
7656 2837 Acked Write 3 Sent Done Done false
7657 2838 Acked Write 4 Sent Done Done false
7658 2839 Acked Write 5 Sent Done Done false
7659 2840 Acked Read 1 Sent Done Done false
7660 2841 Acked Read 6 Sent Done Done false
7661 2842 Acked Read 2 Sent Done Done false
7662 2843 Acked Read 4 Sent Done Done false
7663 2844 Acked Write 2 Sent Done Done false
7664 2845 Acked Read 10 Sent Done Done false
7665 2846 Acked Read 3 Sent Done Done false
7666 2847 Acked Read 4 Sent Done Done false
7667 2848 Acked Read 3 Sent Done Done false
7668 2849 Acked Write 4 Sent Done Done false
7669 2850 Acked Flush 0 Sent Done Done false
7670 2851 Acked Flush 0 Sent Done Done false
7671 2852 Acked Write 6 Sent Done Done false
7672 2853 Acked Read 3 Sent Done Done false
7673 2854 Acked Write 1 Sent Done Done false
7674 2855 Acked Write 6 Sent Done Done false
7675 2856 Acked Write 8 Sent Done Done false
7676 2857 Acked Read 5 Sent Done Done false
7677 2858 Acked Write 8 Sent Done Done false
7678 2859 Acked Read 5 Sent Done Done false
7679 2860 Acked Read 7 Sent Done Done false
7680 2861 Acked Read 10 Sent Done Done false
7681 2862 Acked Read 2 Sent Done Done false
7682 2863 Acked Write 4 Sent Done Done false
7683 2864 Acked Write 2 Sent Done Done false
7684 2865 Acked Flush 0 Sent Done Done false
7685 2866 Acked Write 3 Sent Done Done false
7686 2867 Acked Write 6 Sent Done Done false
7687 2868 Acked Read 8 Sent Done Done false
7688 2869 Acked Write 6 Sent Done Done false
7689 2870 Acked Write 3 Sent Done Done false
7690 2871 Acked Read 10 Sent Done Done false
7691 2872 Acked Write 1 Sent Done Done false
7692 2873 Acked Write 4 Sent Done Done false
7693 2874 Acked Read 8 Sent Done Done false
7694 2875 Acked Read 6 Sent Done Done false
7695 2876 Acked Flush 0 Sent Done Done false
7696 2877 Acked Read 3 Sent Done Done false
7697 2878 Acked Write 6 Sent Done Done false
7698 2879 Acked Read 10 Sent Done Done false
7699 2880 Acked Write 1 Sent Done Done false
7700 2881 Acked Write 9 Sent Done Done false
7701 2882 Acked Read 6 Sent Done Done false
7702 2883 Acked Read 8 Sent Done Done false
7703 2884 Acked Read 3 Sent Done Done false
7704 2885 Acked Write 10 Sent Done Done false
7705 2886 Acked Read 6 Sent Done Done false
7706 2887 Acked Write 1 Sent Done Done false
7707 2888 Acked Write 8 Sent Done Done false
7708 2889 Acked Write 10 Sent Done Done false
7709 2890 Acked Read 5 Sent Done Done false
7710 2891 Acked Write 5 Sent Done Done false
7711 2892 Acked Read 1 Sent Done Done false
7712 2893 Acked Read 8 Sent Done Done false
7713 2894 Acked Read 4 Sent Done Done false
7714 2895 Acked Write 9 Sent Done Done false
7715 2896 Acked Read 3 Sent Done Done false
7716 2897 Acked Read 4 Sent Done Done false
7717 2898 Acked Write 6 Sent Done Done false
7718 2899 Acked Write 2 Sent Done Done false
7719 2900 Acked Read 4 Sent Done Done false
7720 2901 Acked Read 9 Sent Done Done false
7721 2902 Acked Write 4 Sent Done Done false
7722 2903 Acked Read 2 Sent Done Done false
7723 2904 Acked Write 9 Sent Done Done false
7724 2905 Acked Read 4 Sent Done Done false
7725 2906 Acked Write 1 Sent Done Done false
7726 2907 Acked Write 10 Sent Done Done false
7727 2908 Acked Write 1 Sent Done Done false
7728 2909 Acked Write 6 Sent Done Done false
7729 2910 Acked Write 7 Sent Done Done false
7730 2911 Acked Write 2 Sent Done Done false
7731 2912 Acked Write 4 Sent Done Done false
7732 2913 Acked Write 7 Sent Done Done false
7733 2914 Acked Read 7 Sent Done Done false
7734 2915 Acked Write 1 Sent Done Done false
7735 2916 Acked Write 7 Sent Done Done false
7736 2917 Acked Write 10 Sent Done Done false
7737 2918 Acked Write 6 Sent Done Done false
7738 2919 Acked Read 1 Sent Done Done false
7739 2920 Acked Write 7 Sent Done Done false
7740 2921 Acked Read 9 Sent Done Done false
7741 2922 Acked Read 8 Sent Done Done false
7742 2923 Acked Read 3 Sent Done Done false
7743 2924 Acked Read 3 Sent Done Done false
7744 2925 Acked Flush 0 Sent Done Done false
7745 2926 Acked Read 5 Sent Done Done false
7746 2927 Acked Flush 0 Sent Done Done false
7747 2928 Acked Flush 0 Sent Done Done false
7748 2929 Acked Write 2 Sent Done Done false
7749 2930 Acked Flush 0 Sent Done Done false
7750 2931 Acked Write 2 Sent Done Done false
7751 2932 Acked Write 3 Sent Done Done false
7752 2933 Acked Flush 0 Sent Done Done false
7753 2934 Acked Write 5 Sent Done Done false
7754 2935 Acked Write 10 Sent Done Done false
7755 2936 Acked Read 8 Sent Done Done false
7756 2937 Acked Write 5 Sent Done Done false
7757 2938 Acked Read 5 Sent Done Done false
7758 2939 Acked Read 3 Sent Done Done false
7759 2940 Acked Read 3 Sent Done Done false
7760 2941 Acked Flush 0 Sent Done Done false
7761 2942 Acked Write 4 Sent Done Done false
7762 2943 Acked Write 4 Sent Done Done false
7763 2944 Acked Read 7 Sent Done Done false
7764 2945 Acked Read 9 Sent Done Done false
7765 2946 Acked Write 1 Sent Done Done false
7766 2947 Acked Write 6 Sent Done Done false
7767 2948 Acked Write 7 Sent Done Done false
7768 2949 Acked Read 10 Sent Done Done false
7769 2950 Acked Write 1 Sent Done Done false
7770 2951 Acked Read 8 Sent Done Done false
7771 2952 Acked Read 2 Sent Done Done false
7772 2953 Acked Write 6 Sent Done Done false
7773 2954 Acked Read 2 Sent Done Done false
7774 2955 Acked Read 7 Sent Done Done false
7775 2956 Acked Read 7 Sent Done Done false
7776 2957 Acked Read 4 Sent Done Done false
7777 2958 Acked Read 5 Sent Done Done false
7778 2959 Acked Read 2 Sent Done Done false
7779 2960 Acked Read 10 Sent Done Done false
7780 2961 Acked Write 6 Sent Done Done false
7781 2962 Acked Write 1 Sent Done Done false
7782 2963 Acked Read 1 Sent Done Done false
7783 2964 Acked Write 6 Sent Done Done false
7784 2965 Acked Read 10 Sent Done Done false
7785 2966 Acked Write 2 Sent Done Done false
7786 2967 Acked Write 6 Sent Done Done false
7787 2968 Acked Read 7 Sent Done Done false
7788 2969 Acked Write 3 Sent Done Done false
7789 2970 Acked Read 7 Sent Done Done false
7790 2971 Acked Read 5 Sent Done Done false
7791 2972 Acked Read 1 Sent Done Done false
7792 2973 Acked Write 10 Sent Done Done false
7793 2974 Acked Read 8 Sent Done Done false
7794 2975 Acked Write 3 Sent Done Done false
7795 2976 Acked Write 5 Sent Done Done false
7796 2977 Acked Flush 0 Sent Done Done false
7797 2978 Acked Read 4 Sent Done Done false
7798 2979 Acked Read 6 Sent Done Done false
7799 2980 Acked Write 6 Sent Done Done false
7800 2981 Acked Flush 0 Sent Done Done false
7801 2982 Acked Read 5 Sent Done Done false
7802 2983 Acked Write 2 Sent Done Done false
7803 2984 Acked Write 4 Sent Done Done false
7804 2985 Acked Write 8 Sent Done Done false
7805 2986 Acked Read 3 Sent Done Done false
7806 2987 Acked Write 9 Sent Done Done false
7807 2988 Acked Write 10 Sent Done Done false
7808 2989 Acked Write 6 Sent Done Done false
7809 2990 Acked Read 7 Sent Done Done false
7810 2991 Acked Read 7 Sent Done Done false
7811 2992 Acked Write 10 Sent Done Done false
7812 2993 Acked Flush 0 Sent Done Done false
7813 2994 Acked Flush 0 Sent Done Done false
7814 2995 Acked Write 7 Sent Done Done false
7815 2996 Acked Write 10 Sent Done Done false
7816 2997 Acked Read 6 Sent Done Done false
7817 2998 Acked Write 1 Sent Done Done false
7818 2999 Acked Read 6 Sent Done Done false
7819 3000 Acked Read 6 Sent Done Done false
7820 3001 Acked Read 8 Sent Done Done false
7821 3002 Acked Write 8 Sent Done Done false
7822 3003 Acked Read 9 Sent Done Done false
7823 3004 Acked Flush 0 Sent Done Done false
7824 3005 Acked Read 2 Sent Done Done false
7825 3006 Acked Write 1 Sent Done Done false
7826 3007 Acked Read 1 Sent Done Done false
7827 3008 Acked Write 2 Sent Done Done false
7828 3009 Acked Write 3 Sent Done Done false
7829 3010 Acked Flush 0 Sent Done Done false
7830 3011 Acked Flush 0 Sent Done Done false
7831 3012 Acked Read 1 Sent Done Done false
7832 3013 Acked Flush 0 Sent Done Done false
7833 3014 Acked Read 5 Sent Done Done false
7834 3015 Acked Write 2 Sent Done Done false
7835 3016 Acked Read 7 Sent Done Done false
7836 3017 Acked Read 4 Sent Done Done false
7837 3018 Acked Read 10 Sent Done Done false
7838 3019 Acked Read 8 Sent Done Done false
7839 3020 Acked Write 4 Sent Done Done false
7840 3021 Acked Write 9 Sent Done Done false
7841 3022 Acked Read 8 Sent Done Done false
7842 3023 Acked Read 9 Sent Done Done false
7843 3024 Acked Write 10 Sent Done Done false
7844 3025 Acked Write 4 Sent Done Done false
7845 3026 Acked Write 4 Sent Done Done false
7846 3027 Acked Write 3 Sent Done Done false
7847 3028 Acked Read 6 Sent Done Done false
7848 3029 Acked Flush 0 Sent Done Done false
7849 3030 Acked Read 4 Sent Done Done false
7850 3031 Acked Read 3 Sent Done Done false
7851 3032 Acked Write 2 Sent Done Done false
7852 3033 Acked Flush 0 Sent Done Done false
7853 3034 Acked Read 3 Sent Done Done false
7854 3035 Acked Write 6 Sent Done Done false
7855 3036 Acked Read 10 Sent Done Done false
7856 3037 Acked Write 2 Sent Done Done false
7857 3038 Acked Write 3 Sent Done Done false
7858 3039 Acked Read 10 Sent Done Done false
7859 3040 Acked Write 6 Sent Done Done false
7860 3041 Acked Write 5 Sent Done Done false
7861 3042 Acked Write 9 Sent Done Done false
7862 3043 Acked Read 1 Sent Done Done false
7863 3044 Acked Read 7 Sent Done Done false
7864 3045 Acked Write 7 Sent Done Done false
7865 3046 Acked Read 9 Sent Done Done false
7866 3047 Acked Write 4 Sent Done Done false
7867 3048 Acked Write 10 Sent Done Done false
7868 3049 Acked Read 8 Sent Done Done false
7869 3050 Acked Read 2 Sent Done Done false
7870 3051 Acked Read 6 Sent Done Done false
7871 3052 Acked Read 5 Sent Done Done false
7872 3053 Acked Read 10 Sent Done Done false
7873 3054 Acked Read 3 Sent Done Done false
7874 3055 Acked Read 4 Sent Done Done false
7875 3056 Acked Write 7 Sent Done Done false
7876 3057 Acked Write 9 Sent Done Done false
7877 3058 Acked Read 3 Sent Done Done false
7878 3059 Acked Read 4 Sent Done Done false
7879 3060 Acked Read 3 Sent Done Done false
7880 3061 Acked Write 4 Sent Done Done false
7881 3062 Acked Flush 0 Sent Done Done false
7882 3063 Acked Read 3 Sent Done Done false
7883 3064 Acked Flush 0 Sent Done Done false
7884 3065 Acked Read 4 Sent Done Done false
7885 3066 Acked Read 8 Sent Done Done false
7886 3067 Acked Read 3 Sent Done Done false
7887 3068 Acked Flush 0 Sent Done Done false
7888 3069 Acked Read 6 Sent Done Done false
7889 3070 Acked Write 2 Sent Done Done false
7890 3071 Acked Write 8 Sent Done Done false
7891 3072 Acked Read 5 Sent Done Done false
7892 3073 Acked Write 9 Sent Done Done false
7893 3074 Acked Read 9 Sent Done Done false
7894 3075 Acked Write 3 Sent Done Done false
7895 3076 Acked Read 3 Sent Done Done false
7896 3077 Acked Write 7 Sent Done Done false
7897 3078 Acked Read 10 Sent Done Done false
7898 3079 Acked Write 5 Sent Done Done false
7899 3080 Acked Read 9 Sent Done Done false
7900 3081 Acked Read 6 Sent Done Done false
7901 3082 Acked Read 9 Sent Done Done false
7902 3083 Acked Read 8 Sent Done Done false
7903 3084 Acked Flush 0 Sent Done Done false
7904 3085 Acked Write 7 Sent Done Done false
7905 3086 Acked Read 5 Sent Done Done false
7906 3087 Acked Read 2 Sent Done Done false
7907 3088 Acked Flush 0 Sent Done Done false
7908 3089 Acked Read 7 Sent Done Done false
7909 3090 Acked Read 7 Sent Done Done false
7910 3091 Acked Read 6 Sent Done Done false
7911 3092 Acked Write 5 Sent Done Done false
7912 3093 Acked Write 7 Sent Done Done false
7913 3094 Acked Write 7 Sent Done Done false
7914 3095 Acked Write 3 Sent Done Done false
7915 3096 Acked Read 4 Sent Done Done false
7916 3097 Acked Read 3 Sent Done Done false
7917 3098 Acked Write 7 Sent Done Done false
7918 3099 Acked Write 8 Sent Done Done false
7919 3100 Acked Read 3 Sent Done Done false
7920 3101 Acked Write 3 Sent Done Done false
7921 3102 Acked Read 7 Sent Done Done false
7922 3103 Acked Read 6 Sent Done Done false
7923 3104 Acked Read 6 Sent Done Done false
7924 3105 Acked Read 10 Sent Done Done false
7925 3106 Acked Read 1 Sent Done Done false
7926 3107 Acked Read 8 Sent Done Done false
7927 3108 Acked Flush 0 Sent Done Done false
7928 3109 Acked Read 9 Sent Done Done false
7929 3110 Acked Write 7 Sent Done Done false
7930 3111 Acked Write 8 Sent Done Done false
7931 3112 Acked Flush 0 Sent Done Done false
7932 STATES DS:0 DS:1 DS:2 TOTAL
7933 Sent 302 0 0 302
7934 Done 0 302 302 604
7935 Skipped 0 0 0 0
7936 Error 0 0 0 0
7937 Last Flush: 2810 3112 3112
7938 Downstairs last five completed: 2810 2809 2808 2807 2806
7939 Upstairs last five completed: 3112 3111 3110 3109 3108
7940 CLIENT: Up:0 ds:302 act:2
7941 ----------------------------------------------------------------
7942 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7943 States: Offline Active Active
7944 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7945 2811 Acked Flush 0 Sent Done Done false
7946 2812 Acked Write 2 Sent Done Done false
7947 2813 Acked Flush 0 Sent Done Done false
7948 2814 Acked Read 6 Sent Done Done false
7949 2815 Acked Write 3 Sent Done Done false
7950 2816 Acked Write 5 Sent Done Done false
7951 2817 Acked Write 4 Sent Done Done false
7952 2818 Acked Read 9 Sent Done Done false
7953 2819 Acked Write 10 Sent Done Done false
7954 2820 Acked Flush 0 Sent Done Done false
7955 2821 Acked Read 8 Sent Done Done false
7956 2822 Acked Read 7 Sent Done Done false
7957 2823 Acked Read 9 Sent Done Done false
7958 2824 Acked Flush 0 Sent Done Done false
7959 2825 Acked Write 7 Sent Done Done false
7960 2826 Acked Write 9 Sent Done Done false
7961 2827 Acked Read 10 Sent Done Done false
7962 2828 Acked Flush 0 Sent Done Done false
7963 2829 Acked Read 9 Sent Done Done false
7964 2830 Acked Read 5 Sent Done Done false
7965 2831 Acked Write 4 Sent Done Done false
7966 2832 Acked Write 8 Sent Done Done false
7967 2833 Acked Read 9 Sent Done Done false
7968 2834 Acked Read 8 Sent Done Done false
7969 2835 Acked Read 4 Sent Done Done false
7970 2836 Acked Read 10 Sent Done Done false
7971 2837 Acked Write 3 Sent Done Done false
7972 2838 Acked Write 4 Sent Done Done false
7973 2839 Acked Write 5 Sent Done Done false
7974 2840 Acked Read 1 Sent Done Done false
7975 2841 Acked Read 6 Sent Done Done false
7976 2842 Acked Read 2 Sent Done Done false
7977 2843 Acked Read 4 Sent Done Done false
7978 2844 Acked Write 2 Sent Done Done false
7979 2845 Acked Read 10 Sent Done Done false
7980 2846 Acked Read 3 Sent Done Done false
7981 2847 Acked Read 4 Sent Done Done false
7982 2848 Acked Read 3 Sent Done Done false
7983 2849 Acked Write 4 Sent Done Done false
7984 2850 Acked Flush 0 Sent Done Done false
7985 2851 Acked Flush 0 Sent Done Done false
7986 2852 Acked Write 6 Sent Done Done false
7987 2853 Acked Read 3 Sent Done Done false
7988 2854 Acked Write 1 Sent Done Done false
7989 2855 Acked Write 6 Sent Done Done false
7990 2856 Acked Write 8 Sent Done Done false
7991 2857 Acked Read 5 Sent Done Done false
7992 2858 Acked Write 8 Sent Done Done false
7993 2859 Acked Read 5 Sent Done Done false
7994 2860 Acked Read 7 Sent Done Done false
7995 2861 Acked Read 10 Sent Done Done false
7996 2862 Acked Read 2 Sent Done Done false
7997 2863 Acked Write 4 Sent Done Done false
7998 2864 Acked Write 2 Sent Done Done false
7999 2865 Acked Flush 0 Sent Done Done false
8000 2866 Acked Write 3 Sent Done Done false
8001 2867 Acked Write 6 Sent Done Done false
8002 2868 Acked Read 8 Sent Done Done false
8003 2869 Acked Write 6 Sent Done Done false
8004 2870 Acked Write 3 Sent Done Done false
8005 2871 Acked Read 10 Sent Done Done false
8006 2872 Acked Write 1 Sent Done Done false
8007 2873 Acked Write 4 Sent Done Done false
8008 2874 Acked Read 8 Sent Done Done false
8009 2875 Acked Read 6 Sent Done Done false
8010 2876 Acked Flush 0 Sent Done Done false
8011 2877 Acked Read 3 Sent Done Done false
8012 2878 Acked Write 6 Sent Done Done false
8013 2879 Acked Read 10 Sent Done Done false
8014 2880 Acked Write 1 Sent Done Done false
8015 2881 Acked Write 9 Sent Done Done false
8016 2882 Acked Read 6 Sent Done Done false
8017 2883 Acked Read 8 Sent Done Done false
8018 2884 Acked Read 3 Sent Done Done false
8019 2885 Acked Write 10 Sent Done Done false
8020 2886 Acked Read 6 Sent Done Done false
8021 2887 Acked Write 1 Sent Done Done false
8022 2888 Acked Write 8 Sent Done Done false
8023 2889 Acked Write 10 Sent Done Done false
8024 2890 Acked Read 5 Sent Done Done false
8025 2891 Acked Write 5 Sent Done Done false
8026 2892 Acked Read 1 Sent Done Done false
8027 2893 Acked Read 8 Sent Done Done false
8028 2894 Acked Read 4 Sent Done Done false
8029 2895 Acked Write 9 Sent Done Done false
8030 2896 Acked Read 3 Sent Done Done false
8031 2897 Acked Read 4 Sent Done Done false
8032 2898 Acked Write 6 Sent Done Done false
8033 2899 Acked Write 2 Sent Done Done false
8034 2900 Acked Read 4 Sent Done Done false
8035 2901 Acked Read 9 Sent Done Done false
8036 2902 Acked Write 4 Sent Done Done false
8037 2903 Acked Read 2 Sent Done Done false
8038 2904 Acked Write 9 Sent Done Done false
8039 2905 Acked Read 4 Sent Done Done false
8040 2906 Acked Write 1 Sent Done Done false
8041 2907 Acked Write 10 Sent Done Done false
8042 2908 Acked Write 1 Sent Done Done false
8043 2909 Acked Write 6 Sent Done Done false
8044 2910 Acked Write 7 Sent Done Done false
8045 2911 Acked Write 2 Sent Done Done false
8046 2912 Acked Write 4 Sent Done Done false
8047 2913 Acked Write 7 Sent Done Done false
8048 2914 Acked Read 7 Sent Done Done false
8049 2915 Acked Write 1 Sent Done Done false
8050 2916 Acked Write 7 Sent Done Done false
8051 2917 Acked Write 10 Sent Done Done false
8052 2918 Acked Write 6 Sent Done Done false
8053 2919 Acked Read 1 Sent Done Done false
8054 2920 Acked Write 7 Sent Done Done false
8055 2921 Acked Read 9 Sent Done Done false
8056 2922 Acked Read 8 Sent Done Done false
8057 2923 Acked Read 3 Sent Done Done false
8058 2924 Acked Read 3 Sent Done Done false
8059 2925 Acked Flush 0 Sent Done Done false
8060 2926 Acked Read 5 Sent Done Done false
8061 2927 Acked Flush 0 Sent Done Done false
8062 2928 Acked Flush 0 Sent Done Done false
8063 2929 Acked Write 2 Sent Done Done false
8064 2930 Acked Flush 0 Sent Done Done false
8065 2931 Acked Write 2 Sent Done Done false
8066 2932 Acked Write 3 Sent Done Done false
8067 2933 Acked Flush 0 Sent Done Done false
8068 2934 Acked Write 5 Sent Done Done false
8069 2935 Acked Write 10 Sent Done Done false
8070 2936 Acked Read 8 Sent Done Done false
8071 2937 Acked Write 5 Sent Done Done false
8072 2938 Acked Read 5 Sent Done Done false
8073 2939 Acked Read 3 Sent Done Done false
8074 2940 Acked Read 3 Sent Done Done false
8075 2941 Acked Flush 0 Sent Done Done false
8076 2942 Acked Write 4 Sent Done Done false
8077 2943 Acked Write 4 Sent Done Done false
8078 2944 Acked Read 7 Sent Done Done false
8079 2945 Acked Read 9 Sent Done Done false
8080 2946 Acked Write 1 Sent Done Done false
8081 2947 Acked Write 6 Sent Done Done false
8082 2948 Acked Write 7 Sent Done Done false
8083 2949 Acked Read 10 Sent Done Done false
8084 2950 Acked Write 1 Sent Done Done false
8085 2951 Acked Read 8 Sent Done Done false
8086 2952 Acked Read 2 Sent Done Done false
8087 2953 Acked Write 6 Sent Done Done false
8088 2954 Acked Read 2 Sent Done Done false
8089 2955 Acked Read 7 Sent Done Done false
8090 2956 Acked Read 7 Sent Done Done false
8091 2957 Acked Read 4 Sent Done Done false
8092 2958 Acked Read 5 Sent Done Done false
8093 2959 Acked Read 2 Sent Done Done false
8094 2960 Acked Read 10 Sent Done Done false
8095 2961 Acked Write 6 Sent Done Done false
8096 2962 Acked Write 1 Sent Done Done false
8097 2963 Acked Read 1 Sent Done Done false
8098 2964 Acked Write 6 Sent Done Done false
8099 2965 Acked Read 10 Sent Done Done false
8100 2966 Acked Write 2 Sent Done Done false
8101 2967 Acked Write 6 Sent Done Done false
8102 2968 Acked Read 7 Sent Done Done false
8103 2969 Acked Write 3 Sent Done Done false
8104 2970 Acked Read 7 Sent Done Done false
8105 2971 Acked Read 5 Sent Done Done false
8106 2972 Acked Read 1 Sent Done Done false
8107 2973 Acked Write 10 Sent Done Done false
8108 2974 Acked Read 8 Sent Done Done false
8109 2975 Acked Write 3 Sent Done Done false
8110 2976 Acked Write 5 Sent Done Done false
8111 2977 Acked Flush 0 Sent Done Done false
8112 2978 Acked Read 4 Sent Done Done false
8113 2979 Acked Read 6 Sent Done Done false
8114 2980 Acked Write 6 Sent Done Done false
8115 2981 Acked Flush 0 Sent Done Done false
8116 2982 Acked Read 5 Sent Done Done false
8117 2983 Acked Write 2 Sent Done Done false
8118 2984 Acked Write 4 Sent Done Done false
8119 2985 Acked Write 8 Sent Done Done false
8120 2986 Acked Read 3 Sent Done Done false
8121 2987 Acked Write 9 Sent Done Done false
8122 2988 Acked Write 10 Sent Done Done false
8123 2989 Acked Write 6 Sent Done Done false
8124 2990 Acked Read 7 Sent Done Done false
8125 2991 Acked Read 7 Sent Done Done false
8126 2992 Acked Write 10 Sent Done Done false
8127 2993 Acked Flush 0 Sent Done Done false
8128 2994 Acked Flush 0 Sent Done Done false
8129 2995 Acked Write 7 Sent Done Done false
8130 2996 Acked Write 10 Sent Done Done false
8131 2997 Acked Read 6 Sent Done Done false
8132 2998 Acked Write 1 Sent Done Done false
8133 2999 Acked Read 6 Sent Done Done false
8134 3000 Acked Read 6 Sent Done Done false
8135 3001 Acked Read 8 Sent Done Done false
8136 3002 Acked Write 8 Sent Done Done false
8137 3003 Acked Read 9 Sent Done Done false
8138 3004 Acked Flush 0 Sent Done Done false
8139 3005 Acked Read 2 Sent Done Done false
8140 3006 Acked Write 1 Sent Done Done false
8141 3007 Acked Read 1 Sent Done Done false
8142 3008 Acked Write 2 Sent Done Done false
8143 3009 Acked Write 3 Sent Done Done false
8144 3010 Acked Flush 0 Sent Done Done false
8145 3011 Acked Flush 0 Sent Done Done false
8146 3012 Acked Read 1 Sent Done Done false
8147 3013 Acked Flush 0 Sent Done Done false
8148 3014 Acked Read 5 Sent Done Done false
8149 3015 Acked Write 2 Sent Done Done false
8150 3016 Acked Read 7 Sent Done Done false
8151 3017 Acked Read 4 Sent Done Done false
8152 3018 Acked Read 10 Sent Done Done false
8153 3019 Acked Read 8 Sent Done Done false
8154 3020 Acked Write 4 Sent Done Done false
8155 3021 Acked Write 9 Sent Done Done false
8156 3022 Acked Read 8 Sent Done Done false
8157 3023 Acked Read 9 Sent Done Done false
8158 3024 Acked Write 10 Sent Done Done false
8159 3025 Acked Write 4 Sent Done Done false
8160 3026 Acked Write 4 Sent Done Done false
8161 3027 Acked Write 3 Sent Done Done false
8162 3028 Acked Read 6 Sent Done Done false
8163 3029 Acked Flush 0 Sent Done Done false
8164 3030 Acked Read 4 Sent Done Done false
8165 3031 Acked Read 3 Sent Done Done false
8166 3032 Acked Write 2 Sent Done Done false
8167 3033 Acked Flush 0 Sent Done Done false
8168 3034 Acked Read 3 Sent Done Done false
8169 3035 Acked Write 6 Sent Done Done false
8170 3036 Acked Read 10 Sent Done Done false
8171 3037 Acked Write 2 Sent Done Done false
8172 3038 Acked Write 3 Sent Done Done false
8173 3039 Acked Read 10 Sent Done Done false
8174 3040 Acked Write 6 Sent Done Done false
8175 3041 Acked Write 5 Sent Done Done false
8176 3042 Acked Write 9 Sent Done Done false
8177 3043 Acked Read 1 Sent Done Done false
8178 3044 Acked Read 7 Sent Done Done false
8179 3045 Acked Write 7 Sent Done Done false
8180 3046 Acked Read 9 Sent Done Done false
8181 3047 Acked Write 4 Sent Done Done false
8182 3048 Acked Write 10 Sent Done Done false
8183 3049 Acked Read 8 Sent Done Done false
8184 3050 Acked Read 2 Sent Done Done false
8185 3051 Acked Read 6 Sent Done Done false
8186 3052 Acked Read 5 Sent Done Done false
8187 3053 Acked Read 10 Sent Done Done false
8188 3054 Acked Read 3 Sent Done Done false
8189 3055 Acked Read 4 Sent Done Done false
8190 3056 Acked Write 7 Sent Done Done false
8191 3057 Acked Write 9 Sent Done Done false
8192 3058 Acked Read 3 Sent Done Done false
8193 3059 Acked Read 4 Sent Done Done false
8194 3060 Acked Read 3 Sent Done Done false
8195 3061 Acked Write 4 Sent Done Done false
8196 3062 Acked Flush 0 Sent Done Done false
8197 3063 Acked Read 3 Sent Done Done false
8198 3064 Acked Flush 0 Sent Done Done false
8199 3065 Acked Read 4 Sent Done Done false
8200 3066 Acked Read 8 Sent Done Done false
8201 3067 Acked Read 3 Sent Done Done false
8202 3068 Acked Flush 0 Sent Done Done false
8203 3069 Acked Read 6 Sent Done Done false
8204 3070 Acked Write 2 Sent Done Done false
8205 3071 Acked Write 8 Sent Done Done false
8206 3072 Acked Read 5 Sent Done Done false
8207 3073 Acked Write 9 Sent Done Done false
8208 3074 Acked Read 9 Sent Done Done false
8209 3075 Acked Write 3 Sent Done Done false
8210 3076 Acked Read 3 Sent Done Done false
8211 3077 Acked Write 7 Sent Done Done false
8212 3078 Acked Read 10 Sent Done Done false
8213 3079 Acked Write 5 Sent Done Done false
8214 3080 Acked Read 9 Sent Done Done false
8215 3081 Acked Read 6 Sent Done Done false
8216 3082 Acked Read 9 Sent Done Done false
8217 3083 Acked Read 8 Sent Done Done false
8218 3084 Acked Flush 0 Sent Done Done false
8219 3085 Acked Write 7 Sent Done Done false
8220 3086 Acked Read 5 Sent Done Done false
8221 3087 Acked Read 2 Sent Done Done false
8222 3088 Acked Flush 0 Sent Done Done false
8223 3089 Acked Read 7 Sent Done Done false
8224 3090 Acked Read 7 Sent Done Done false
8225 3091 Acked Read 6 Sent Done Done false
8226 3092 Acked Write 5 Sent Done Done false
8227 3093 Acked Write 7 Sent Done Done false
8228 3094 Acked Write 7 Sent Done Done false
8229 3095 Acked Write 3 Sent Done Done false
8230 3096 Acked Read 4 Sent Done Done false
8231 3097 Acked Read 3 Sent Done Done false
8232 3098 Acked Write 7 Sent Done Done false
8233 3099 Acked Write 8 Sent Done Done false
8234 3100 Acked Read 3 Sent Done Done false
8235 3101 Acked Write 3 Sent Done Done false
8236 3102 Acked Read 7 Sent Done Done false
8237 3103 Acked Read 6 Sent Done Done false
8238 3104 Acked Read 6 Sent Done Done false
8239 3105 Acked Read 10 Sent Done Done false
8240 3106 Acked Read 1 Sent Done Done false
8241 3107 Acked Read 8 Sent Done Done false
8242 3108 Acked Flush 0 Sent Done Done false
8243 3109 Acked Read 9 Sent Done Done false
8244 3110 Acked Write 7 Sent Done Done false
8245 3111 Acked Write 8 Sent Done Done false
8246 3112 Acked Flush 0 Sent Done Done false
8247 STATES DS:0 DS:1 DS:2 TOTAL
8248 Sent 302 0 0 302
8249 Done 0 302 302 604
8250 Skipped 0 0 0 0
8251 Error 0 0 0 0
8252 Last Flush: 2810 3112 3112
8253 Downstairs last five completed: 2810 2809 2808 2807 2806
8254 Upstairs last five completed: 3112 3111 3110 3109 3108
8255 CLIENT: Up:0 ds:302 act:2
82562025-01-15T05:20:44.444ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82572025-01-15T05:20:44.444ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38779) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82582025-01-15T05:20:44.445ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82592025-01-15T05:20:44.446ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82602025-01-15T05:20:44.446ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82612025-01-15T05:20:44.446ZINFOcrucible: send last flush ID to this DS: 2810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82622025-01-15T05:20:44.446ZINFOcrucible: Replied this last flush ID: 2810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82632025-01-15T05:20:44.446ZINFOcrucible: [0] client re-new 302 jobs since flush 2810 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82642025-01-15T05:20:44.447ZINFOcrucible: [0] Replayed 302 jobs since flush: 2810 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
8265 ----------------------------------------------------------------
8266 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8267 Downstairs last five completed: 3112 3111 3110 3109 3108
8268 Upstairs last five completed: 3112 3111 3110 3109 3108
8269 CLIENT: Up:0 ds:0 act:3
8270 Replay: All jobs finished
8271 Main task has work to do, go find it
8272 got dsc Stop(2)
8273 stop 2
8274 Jan 15 05:20:44.685 INFO request completed, latency_us: 347[8830] Got stop action so:false kr:false
8275 , response_code: 204, uri: /stop/cid/2, method: POST, req_id: 2c8030ab-6e8d-4eb7-a243-62502d30d8b7, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
82762025-01-15T05:20:44.686ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82772025-01-15T05:20:44.686ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82782025-01-15T05:20:44.686ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
82792025-01-15T05:20:44.686ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
8280 Jan 15 05:20:44.686 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/2, method: GET, req_id: f10fe511-ff02-4b2d-b826-90a08ec67b74, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
8281 [8830] Exited with: ExitStatus(unix_wait_status(9))
8282 [8830][2] reports Exit
8283 Jan 15 05:20:48.688 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6709d7e6-1143-48b5-89d2-2d1984cfe771, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
8284 001/300 Read block 4060 len 20480
8285 002/300 Write block 1500 len 16384 data: 2 2 3 3
8286 003/300 Read block 179 len 40960
8287 004/300 Write block 203 len 16384 data: 3 3 3 4
8288 005/300 Write block 3615 len 24576 data: 2 2 2 2 3 2
8289 006/300 Write block 3173 len 8192 data: 2 2
8290 007/300 Write block 4777 len 8192 data: 3 3
8291 008/300 Read block 4440 len 32768
8292 009/300 Read block 547 len 16384
8293 010/300 Read block 3471 len 32768
8294 011/300 Read block 3145 len 40960
8295 012/300 Flush
8296 013/300 Write block 120 len 12288 data: 3 2 2
8297 014/300 Read block 3598 len 24576
8298 015/300 Read block 2282 len 16384
8299 016/300 Write block 2033 len 12288 data: 4 3 3
8300 017/300 Write block 504 len 28672 data: 3 4 4 4 4 3 3
8301 018/300 Write block 851 len 36864 data: 2 2 2 2 2 3 3 2 3
8302 019/300 Read block 4994 len 16384
8303 020/300 Write block 4125 len 32768 data: 3 3 4 3 3 3 4 4
8304 021/300 Read block 231 len 20480
8305 022/300 Write block 3691 len 12288 data: 2 2 2
8306 023/300 Read block 4124 len 32768
8307 024/300 Read block 3585 len 16384
8308 025/300 Write block 1634 len 32768 data: 4 4 4 4 5 4 4 4
8309 026/300 Read block 4689 len 16384
8310 027/300 Flush
8311 028/300 Read block 2096 len 40960
8312 029/300 Write block 1977 len 8192 data: 3 3
8313 030/300 Flush
8314 031/300 Write block 326 len 4096 data: 4
8315 032/300 Write block 786 len 24576 data: 2 2 2 3 3 3
8316 033/300 Read block 2192 len 4096
8317 034/300 Write block 1825 len 28672 data: 2 2 2 2 2 2 2
8318 035/300 Flush
8319 036/300 Write block 6 len 36864 data: 2 2 2 2 2 2 2 2 2
8320 037/300 Write block 3870 len 40960 data: 4 4 4 4 4 3 2 2 3 3
8321 038/300 Write block 4113 len 40960 data: 2 3 3 3 4 4 4 4 3 3
8322 039/300 Read block 82 len 32768
8323 040/300 Read block 3583 len 32768
8324 041/300 Read block 584 len 36864
8325 042/300 Write block 2427 len 32768 data: 2 3 3 3 3 4 4 3
8326 043/300 Read block 3871 len 24576
8327 044/300 Write block 2976 len 20480 data: 3 3 2 2 2
8328 045/300 Read block 3563 len 40960
8329 046/300 Write block 3922 len 4096 data: 3
8330 047/300 Read block 4227 len 40960
8331 048/300 Read block 3532 len 28672
8332 049/300 Write block 2541 len 12288 data: 3 2 2
8333 050/300 Read block 1856 len 20480
8334 051/300 Write block 1700 len 20480 data: 3 3 2 2 3
8335 052/300 Read block 477 len 12288
8336 053/300 Read block 788 len 4096
8337 054/300 Read block 231 len 28672
8338 055/300 Read block 1125 len 16384
8339 056/300 Read block 237 len 28672
8340 057/300 Read block 2029 len 36864
8341 058/300 Write block 1102 len 20480 data: 2 2 2 2 2
8342 059/300 Write block 608 len 32768 data: 2 2 2 2 2 2 2 3
8343 060/300 Write block 876 len 40960 data: 3 3 3 3 3 3 3 3 3 3
8344 061/300 Write block 1319 len 12288 data: 2 2 2
8345 062/300 Read block 1193 len 40960
8346 063/300 Read block 2879 len 40960
8347 064/300 Read block 3545 len 24576
8348 065/300 Write block 3037 len 40960 data: 5 4 5 6 6 6 5 5 3 3
8349 066/300 Read block 3468 len 32768
8350 067/300 Read block 1141 len 24576
8351 068/300 Read block 2444 len 32768
8352 069/300 Write block 683 len 12288 data: 5 5 6
8353 070/300 Flush
8354 071/300 Write block 1639 len 36864 data: 5 5 5 4 4 4 3 2 2
8355 072/300 Read block 682 len 8192
8356 073/300 Flush
8357 074/300 Read block 1713 len 8192
8358 075/300 Read block 3661 len 32768
8359 076/300 Flush
8360 077/300 Write block 1198 len 32768 data: 2 2 2 3 4 6 6 5
8361 078/300 Read block 698 len 36864
8362 079/300 Read block 2408 len 8192
8363 080/300 Read block 4640 len 16384
8364 081/300 Write block 3390 len 12288 data: 2 2 2
8365 082/300 Read block 4049 len 8192
8366 083/300 Read block 4952 len 20480
8367 084/300 Read block 396 len 40960
8368 085/300 Read block 3057 len 20480
8369 086/300 Write block 4251 len 4096 data: 3
8370 087/300 Read block 2236 len 12288
8371 088/300 Write block 2570 len 36864 data: 3 2 2 2 2 2 2 2 2
8372 089/300 Write block 1582 len 40960 data: 4 4 4 5 5 5 4 3 2 2
8373 090/300 Read block 3348 len 16384
8374 091/300 Write block 2852 len 12288 data: 3 2 2
8375 092/300 Write block 319 len 32768 data: 2 3 4 4 4 4 4 5
8376 093/300 Flush
8377 094/300 Read block 4270 len 4096
8378 095/300 Write block 1868 len 12288 data: 3 3 4
8379 096/300 Read block 1603 len 24576
8380 097/300 Write block 929 len 20480 data: 2 2 2 2 2
8381 098/300 Read block 4714 len 8192
8382 099/300 Write block 3266 len 36864 data: 3 3 3 3 2 2 2 3 3
8383 100/300 Read block 3638 len 4096
8384 101/300 Read block 342 len 4096
8385 102/300 Read block 2646 len 40960
8386 103/300 Write block 3524 len 20480 data: 3 3 3 4 5
8387 104/300 Write block 2743 len 12288 data: 4 3 2
8388 105/300 Flush
8389 106/300 Flush
8390 107/300 Read block 4445 len 20480
8391 108/300 Write block 191 len 28672 data: 3 3 2 3 3 3 3
8392 109/300 Write block 34 len 20480 data: 2 2 2 2 2
8393 110/300 Write block 4363 len 20480 data: 4 3 4 3 3
8394 111/300 Read block 3610 len 32768
8395 112/300 Write block 4016 len 36864 data: 2 2 3 4 5 5 5 4 2
8396 113/300 Read block 1976 len 16384
8397 114/300 Read block 4107 len 12288
8398 115/300 Read block 4812 len 24576
8399 116/300 Write block 2892 len 36864 data: 2 2 2 3 3 3 3 4 4
8400 117/300 Read block 1128 len 4096
8401 118/300 Write block 3561 len 20480 data: 3 3 2 2 2
8402 119/300 Read block 2815 len 40960
8403 120/300 Write block 2868 len 12288 data: 4 4 2
8404 121/300 Write block 2344 len 4096 data: 3
8405 122/300 Read block 797 len 12288
8406 123/300 Read block 2465 len 12288
8407 124/300 Write block 2824 len 20480 data: 3 3 3 3 2
8408 125/300 Read block 701 len 32768
8409 126/300 Write block 4740 len 28672 data: 4 3 3 3 4 4 2
8410 127/300 Read block 1228 len 40960
8411 128/300 Read block 3833 len 4096
8412 129/300 Read block 1052 len 32768
8413 130/300 Flush
8414 131/300 Write block 2916 len 28672 data: 2 3 3 4 5 5 5
8415 132/300 Read block 4876 len 36864
8416 133/300 Write block 4432 len 40960 data: 3 3 3 2 2 2 2 3 4 3
8417 134/300 Read block 3316 len 16384
8418 135/300 Read block 3289 len 4096
8419 136/300 Read block 30 len 8192
8420 137/300 Read block 4245 len 40960
8421 138/300 Read block 3810 len 12288
8422 139/300 Write block 444 len 16384 data: 5 5 6 4
8423 140/300 Write block 3535 len 12288 data: 3 3 2
8424 141/300 Read block 4642 len 32768
8425 142/300 Write block 3362 len 20480 data: 2 3 3 3 3
8426 143/300 Write block 3335 len 20480 data: 4 4 4 4 3
8427 144/300 Read block 3009 len 4096
8428 145/300 Write block 4618 len 24576 data: 2 2 2 2 2 2
8429 146/300 Read block 1053 len 12288
8430 147/300 Read block 10 len 24576
8431 148/300 Read block 3905 len 24576
8432 149/300 Write block 1441 len 12288 data: 2 2 2
8433 150/300 Read block 328 len 12288
8434 151/300 Read block 250 len 36864
8435 152/300 Read block 973 len 32768
8436 153/300 Read block 3252 len 40960
8437 154/300 Write block 2438 len 8192 data: 2 3
8438 155/300 Write block 101 len 20480 data: 3 3 3 3 2
8439 156/300 Read block 1132 len 32768
8440 157/300 Read block 3260 len 24576
8441 158/300 Write block 1091 len 36864 data: 3 3 2 2 2 2 2 2 2
8442 159/300 Write block 3691 len 12288 data: 3 3 3
8443 160/300 Write block 1823 len 40960 data: 3 2 3 3 3 3 3 3 3 2
8444 161/300 Read block 1346 len 32768
8445 162/300 Flush
8446 163/300 Read block 2357 len 36864
8447 164/300 Read block 4125 len 32768
8448 165/300 Flush
8449 166/300 Read block 269 len 28672
8450 167/300 Flush
8451 168/300 Read block 2315 len 4096
8452 169/300 Write block 1711 len 32768 data: 2 2 3 3 3 3 3 3
8453 170/300 Write block 4209 len 40960 data: 2 3 3 2 2 3 3 4 4 4
8454 171/300 Flush
8455 172/300 Flush
8456 173/300 Write block 3353 len 20480 data: 4 2 2 2 2
8457 174/300 Write block 2669 len 20480 data: 2 2 2 2 2
8458 175/300 Write block 3445 len 40960 data: 2 2 2 3 3 3 3 4 4 4
8459 176/300 Write block 290 len 28672 data: 3 3 3 3 3 5 5
8460 177/300 Read block 4708 len 8192
8461 178/300 Write block 2149 len 36864 data: 3 3 3 4 3 3 3 3 2
8462 179/300 Flush
8463 180/300 Write block 733 len 36864 data: 2 2 3 3 3 3 3 4 4
8464 181/300 Flush
8465 182/300 Write block 1526 len 16384 data: 2 2 3 3
8466 183/300 Read block 2059 len 16384
8467 184/300 Flush
8468 185/300 Write block 1956 len 8192 data: 3 3
8469 186/300 Flush
8470 187/300 Read block 2374 len 8192
8471 188/300 Read block 734 len 16384
8472 189/300 Write block 4980 len 4096 data: 3
8473 190/300 Read block 4849 len 20480
8474 191/300 Read block 1836 len 32768
8475 192/300 Write block 1922 len 36864 data: 4 4 4 2 2 2 2 2 3
8476 193/300 Write block 2871 len 40960 data: 3 2 2 2 2 2 3 3 3 3
8477 194/300 Write block 3726 len 16384 data: 3 2 3 3
8478 195/300 Read block 4991 len 32768
8479 196/300 Flush
8480 197/300 Write block 2993 len 20480 data: 2 2 2 3 3
8481 198/300 Flush
8482 199/300 Read block 142 len 32768
8483 200/300 Write block 690 len 8192 data: 3 4
8484 201/300 Write block 3694 len 4096 data: 3
8485 202/300 Read block 3513 len 8192
8486 203/300 Read block 3684 len 28672
8487 204/300 Read block 1127 len 40960
8488 205/300 Read block 2750 len 12288
8489 206/300 Read block 2014 len 4096
8490 207/300 Read block 3697 len 36864
8491 208/300 Read block 4150 len 28672
8492 209/300 Read block 1166 len 24576
8493 210/300 Write block 1217 len 4096 data: 4
8494 211/300 Read block 2165 len 24576
8495 212/300 Write block 3680 len 16384 data: 4 4 4 3
8496 213/300 Write block 4852 len 20480 data: 2 3 2 2 2
8497 214/300 Read block 690 len 40960
8498 215/300 Read block 1547 len 4096
8499 216/300 Read block 142 len 20480
8500 217/300 Write block 3896 len 12288 data: 3 3 3
8501 218/300 Write block 4291 len 8192 data: 3 3
8502 219/300 Write block 4978 len 12288 data: 4 5 4
8503 220/300 Read block 1770 len 20480
8504 221/300 Write block 3728 len 24576 data: 4 4 3 3 2 2
8505 222/300 Flush
8506 223/300 Flush
8507 224/300 Write block 4373 len 16384 data: 3 3 3 3
8508 225/300 Read block 4691 len 24576
8509 226/300 Read block 3355 len 32768
8510 227/300 Read block 4722 len 12288
8511 228/300 Flush
8512 229/300 Write block 2413 len 4096 data: 2
8513 230/300 Read block 1884 len 28672
8514 231/300 Write block 1183 len 8192 data: 3 2
8515 232/300 Read block 389 len 40960
8516 233/300 Read block 652 len 20480
8517 234/300 Write block 3099 len 40960 data: 2 2 2 2 2 2 2 2 2 2
8518 235/300 Write block 3561 len 28672 data: 4 4 3 3 3 2 2
8519 236/300 Write block 4558 len 24576 data: 2 2 2 3 3 3
8520 237/300 Read block 2525 len 24576
8521 238/300 Flush
8522 239/300 Write block 4519 len 28672 data: 4 5 3 3 3 3 3
8523 240/300 Flush
8524 241/300 Write block 3862 len 12288 data: 3 2 2
8525 242/300 Read block 4050 len 36864
8526 243/300 Write block 851 len 28672 data: 3 3 3 3 3 4 4
8527 244/300 Read block 988 len 32768
8528 245/300 Write block 133 len 20480 data: 4 4 3 3 3
8529 246/300 Read block 2019 len 12288
8530 247/300 Read block 55 len 40960
8531 248/300 Write block 851 len 4096 data: 4
8532 249/300 Write block 3633 len 4096 data: 2
8533 250/300 Write block 2479 len 24576 data: 2 2 2 2 2 2
8534 251/300 Write block 3145 len 36864 data: 2 2 2 2 2 2 2 2 2
8535 252/300 Write block 673 len 8192 data: 3 3
8536 253/300 Read block 914 len 40960
8537 254/300 Write block 3532 len 8192 data: 3 3
8538 255/300 Read block 3573 len 20480
8539 256/300 Write block 3299 len 32768 data: 4 4 4 4 4 3 3 4
8540 257/300 Write block 1512 len 20480 data: 2 2 2 2 2
8541 258/300 Write block 3969 len 32768 data: 3 2 2 3 3 3 3 4
8542 259/300 Read block 4227 len 28672
8543 260/300 Write block 1568 len 12288 data: 3 4 5
8544 261/300 Read block 4691 len 32768
8545 262/300 Flush
8546 263/300 Read block 2781 len 16384
8547 264/300 Write block 1622 len 24576 data: 2 3 3 3 4 3
8548 265/300 Write block 4153 len 20480 data: 2 2 2 2 2
8549 266/300 Write block 3304 len 16384 data: 4 4 5 4
8550 267/300 Write block 201 len 40960 data: 3 3 4 4 4 5 3 2 3 3
8551 268/300 Flush
8552 269/300 Write block 4972 len 20480 data: 2 3 3 3 4
8553 270/300 Read block 181 len 12288
8554 271/300 Flush
8555 272/300 Read block 3270 len 4096
8556 273/300 Read block 790 len 40960
8557 274/300 Write block 1335 len 40960 data: 3 3 3 3 3 3 3 4 4 4
8558 275/300 Write block 2188 len 4096 data: 2
8559 276/300 Write block 1501 len 20480 data: 3 4 4 3 3
8560 277/300 Read block 2669 len 32768
8561 278/300 Read block 2760 len 36864
8562 279/300 Write block 443 len 24576 data: 6 6 6 7 5 4
8563 280/300 Read block 2719 len 24576
8564 281/300 Write block 657 len 28672 data: 2 2 2 2 2 3 3
8565 282/300 Read block 4542 len 20480
8566 283/300 Read block 2988 len 20480
8567 284/300 Flush
8568 285/300 Write block 2113 len 16384 data: 5 4 4 3
8569 286/300 Write block 647 len 28672 data: 4 4 4 4 4 5 4
8570 287/300 Write block 3171 len 4096 data: 2
8571 288/300 Read block 793 len 12288
8572 289/300 Write block 1628 len 24576 data: 3 4 4 3 3 4
8573 290/300 Write block 2801 len 20480 data: 3 3 3 3 2
8574 291/300 Read block 940 len 40960
8575 292/300 Read block 1932 len 24576
8576 293/300 Write block 2438 len 16384 data: 3 4 4 4
8577 294/300 Read block 4592 len 16384
8578 295/300 Read block 1492 len 8192
8579 296/300 Write block 568 len 36864 data: 2 2 2 2 2 2 3 3 3
8580 297/300 Read block 4994 len 16384
8581 298/300 Write block 1433 len 16384 data: 4 3 3 2
8582 299/300 Write block 3649 len 8192 data: 2 5
8583 300/300 Write block 1170 len 20480 data: 4 4 4 3 3
8584 301/300 Write block 3610 len 12288 data: 3 3 2
8585 Main task has work to do, go find it
8586 got dsc Start(2)
8587 start 2
8588 Jan 15 05:20:48.918 INFO request completed, latency_us: 179, response_code: 204, uri: /start/cid/2, method: POST, req_id: 1058ac72-6d04-4103-b334-5fe55943829d, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
8589 Replay: started 2, returned:Ok(())
8590 ----------------------------------------------------------------
8591 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8592 States: Active Active Offline
8593 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8594 3113 Acked Read 5 Done Done Sent false
8595 3114 Acked Write 4 Done Done Sent false
8596 3115 Acked Read 10 Done Done Sent false
8597 3116 Acked Write 4 Done Done Sent false
8598 3117 Acked Write 6 Done Done Sent false
8599 3118 Acked Write 2 Done Done Sent false
8600 3119 Acked Write 2 Done Done Sent false
8601 3120 Acked Read 8 Done Done Sent false
8602 3121 Acked Read 4 Done Done Sent false
8603 3122 Acked Read 8 Done Done Sent false
8604 3123 Acked Read 10 Done Done Sent false
8605 3124 Acked Flush 0 Done Done Sent false
8606 3125 Acked Write 3 Done Done Sent false
8607 3126 Acked Read 6 Done Done Sent false
8608 3127 Acked Read 4 Done Done Sent false
8609 3128 Acked Write 3 Done Done Sent false
8610 3129 Acked Write 7 Done Done Sent false
8611 3130 Acked Write 9 Done Done Sent false
8612 3131 Acked Read 4 Done Done Sent false
8613 3132 Acked Write 8 Done Done Sent false
8614 3133 Acked Read 5 Done Done Sent false
8615 3134 Acked Write 3 Done Done Sent false
8616 3135 Acked Read 8 Done Done Sent false
8617 3136 Acked Read 4 Done Done Sent false
8618 3137 Acked Write 8 Done Done Sent false
8619 3138 Acked Read 4 Done Done Sent false
8620 3139 Acked Flush 0 Done Done Sent false
8621 3140 Acked Read 10 Done Done Sent false
8622 3141 Acked Write 2 Done Done Sent false
8623 3142 Acked Flush 0 Done Done Sent false
8624 3143 Acked Write 1 Done Done Sent false
8625 3144 Acked Write 6 Done Done Sent false
8626 3145 Acked Read 1 Done Done Sent false
8627 3146 Acked Write 7 Done Done Sent false
8628 3147 Acked Flush 0 Done Done Sent false
8629 3148 Acked Write 9 Done Done Sent false
8630 3149 Acked Write 10 Done Done Sent false
8631 3150 Acked Write 10 Done Done Sent false
8632 3151 Acked Read 8 Done Done Sent false
8633 3152 Acked Read 8 Done Done Sent false
8634 3153 Acked Read 9 Done Done Sent false
8635 3154 Acked Write 8 Done Done Sent false
8636 3155 Acked Read 6 Done Done Sent false
8637 3156 Acked Write 5 Done Done Sent false
8638 3157 Acked Read 10 Done Done Sent false
8639 3158 Acked Write 1 Done Done Sent false
8640 3159 Acked Read 10 Done Done Sent false
8641 3160 Acked Read 7 Done Done Sent false
8642 3161 Acked Write 3 Done Done Sent false
8643 3162 Acked Read 5 Done Done Sent false
8644 3163 Acked Write 5 Done Done Sent false
8645 3164 Acked Read 3 Done Done Sent false
8646 3165 Acked Read 1 Done Done Sent false
8647 3166 Acked Read 7 Done Done Sent false
8648 3167 Acked Read 4 Done Done Sent false
8649 3168 Acked Read 7 Done Done Sent false
8650 3169 Acked Read 9 Done Done Sent false
8651 3170 Acked Write 5 Done Done Sent false
8652 3171 Acked Write 8 Done Done Sent false
8653 3172 Acked Write 10 Done Done Sent false
8654 3173 Acked Write 3 Done Done Sent false
8655 3174 Acked Read 10 Done Done Sent false
8656 3175 Acked Read 10 Done Done Sent false
8657 3176 Acked Read 6 Done Done Sent false
8658 3177 Acked Write 10 Done Done Sent false
8659 3178 Acked Read 8 Done Done Sent false
8660 3179 Acked Read 6 Done Done Sent false
8661 3180 Acked Read 8 Done Done Sent false
8662 3181 Acked Write 3 Done Done Sent false
8663 3182 Acked Flush 0 Done Done Sent false
8664 3183 Acked Write 9 Done Done Sent false
8665 3184 Acked Read 2 Done Done Sent false
8666 3185 Acked Flush 0 Done Done Sent false
8667 3186 Acked Read 2 Done Done Sent false
8668 3187 Acked Read 8 Done Done Sent false
8669 3188 Acked Flush 0 Done Done Sent false
8670 3189 Acked Write 8 Done Done Sent false
8671 3190 Acked Read 9 Done Done Sent false
8672 3191 Acked Read 2 Done Done Sent false
8673 3192 Acked Read 4 Done Done Sent false
8674 3193 Acked Write 3 Done Done Sent false
8675 3194 Acked Read 2 Done Done Sent false
8676 3195 Acked Read 5 Done Done Sent false
8677 3196 Acked Read 10 Done Done Sent false
8678 3197 Acked Read 5 Done Done Sent false
8679 3198 Acked Write 1 Done Done Sent false
8680 3199 Acked Read 3 Done Done Sent false
8681 3200 Acked Write 9 Done Done Sent false
8682 3201 Acked Write 10 Done Done Sent false
8683 3202 Acked Read 4 Done Done Sent false
8684 3203 Acked Write 3 Done Done Sent false
8685 3204 Acked Write 8 Done Done Sent false
8686 3205 Acked Flush 0 Done Done Sent false
8687 3206 Acked Read 1 Done Done Sent false
8688 3207 Acked Write 3 Done Done Sent false
8689 3208 Acked Read 6 Done Done Sent false
8690 3209 Acked Write 5 Done Done Sent false
8691 3210 Acked Read 2 Done Done Sent false
8692 3211 Acked Write 9 Done Done Sent false
8693 3212 Acked Read 1 Done Done Sent false
8694 3213 Acked Read 1 Done Done Sent false
8695 3214 Acked Read 10 Done Done Sent false
8696 3215 Acked Write 5 Done Done Sent false
8697 3216 Acked Write 3 Done Done Sent false
8698 3217 Acked Flush 0 Done Done Sent false
8699 3218 Acked Flush 0 Done Done Sent false
8700 3219 Acked Read 5 Done Done Sent false
8701 3220 Acked Write 7 Done Done Sent false
8702 3221 Acked Write 5 Done Done Sent false
8703 3222 Acked Write 5 Done Done Sent false
8704 3223 Acked Read 8 Done Done Sent false
8705 3224 Acked Write 9 Done Done Sent false
8706 3225 Acked Read 4 Done Done Sent false
8707 3226 Acked Read 3 Done Done Sent false
8708 3227 Acked Read 6 Done Done Sent false
8709 3228 Acked Write 9 Done Done Sent false
8710 3229 Acked Read 1 Done Done Sent false
8711 3230 Acked Write 5 Done Done Sent false
8712 3231 Acked Read 10 Done Done Sent false
8713 3232 Acked Write 3 Done Done Sent false
8714 3233 Acked Write 1 Done Done Sent false
8715 3234 Acked Read 3 Done Done Sent false
8716 3235 Acked Read 3 Done Done Sent false
8717 3236 Acked Write 5 Done Done Sent false
8718 3237 Acked Read 8 Done Done Sent false
8719 3238 Acked Write 7 Done Done Sent false
8720 3239 Acked Read 10 Done Done Sent false
8721 3240 Acked Read 1 Done Done Sent false
8722 3241 Acked Read 8 Done Done Sent false
8723 3242 Acked Flush 0 Done Done Sent false
8724 3243 Acked Write 7 Done Done Sent false
8725 3244 Acked Read 9 Done Done Sent false
8726 3245 Acked Write 10 Done Done Sent false
8727 3246 Acked Read 4 Done Done Sent false
8728 3247 Acked Read 1 Done Done Sent false
8729 3248 Acked Read 2 Done Done Sent false
8730 3249 Acked Read 10 Done Done Sent false
8731 3250 Acked Read 3 Done Done Sent false
8732 3251 Acked Write 4 Done Done Sent false
8733 3252 Acked Write 3 Done Done Sent false
8734 3253 Acked Read 8 Done Done Sent false
8735 3254 Acked Write 5 Done Done Sent false
8736 3255 Acked Write 5 Done Done Sent false
8737 3256 Acked Read 1 Done Done Sent false
8738 3257 Acked Write 6 Done Done Sent false
8739 3258 Acked Read 3 Done Done Sent false
8740 3259 Acked Read 6 Done Done Sent false
8741 3260 Acked Read 6 Done Done Sent false
8742 3261 Acked Write 3 Done Done Sent false
8743 3262 Acked Read 3 Done Done Sent false
8744 3263 Acked Read 9 Done Done Sent false
8745 3264 Acked Read 8 Done Done Sent false
8746 3265 Acked Read 10 Done Done Sent false
8747 3266 Acked Write 2 Done Done Sent false
8748 3267 Acked Write 5 Done Done Sent false
8749 3268 Acked Read 8 Done Done Sent false
8750 3269 Acked Read 6 Done Done Sent false
8751 3270 Acked Write 9 Done Done Sent false
8752 3271 Acked Write 3 Done Done Sent false
8753 3272 Acked Write 10 Done Done Sent false
8754 3273 Acked Read 8 Done Done Sent false
8755 3274 Acked Flush 0 Done Done Sent false
8756 3275 Acked Read 9 Done Done Sent false
8757 3276 Acked Read 8 Done Done Sent false
8758 3277 Acked Flush 0 Done Done Sent false
8759 3278 Acked Read 7 Done Done Sent false
8760 3279 Acked Flush 0 Done Done Sent false
8761 3280 Acked Read 1 Done Done Sent false
8762 3281 Acked Write 8 Done Done Sent false
8763 3282 Acked Write 10 Done Done Sent false
8764 3283 Acked Flush 0 Done Done Sent false
8765 3284 Acked Flush 0 Done Done Sent false
8766 3285 Acked Write 5 Done Done Sent false
8767 3286 Acked Write 5 Done Done Sent false
8768 3287 Acked Write 10 Done Done Sent false
8769 3288 Acked Write 7 Done Done Sent false
8770 3289 Acked Read 2 Done Done Sent false
8771 3290 Acked Write 9 Done Done Sent false
8772 3291 Acked Flush 0 Done Done Sent false
8773 3292 Acked Write 9 Done Done Sent false
8774 3293 Acked Flush 0 Done Done Sent false
8775 3294 Acked Write 4 Done Done Sent false
8776 3295 Acked Read 4 Done Done Sent false
8777 3296 Acked Flush 0 Done Done Sent false
8778 3297 Acked Write 2 Done Done Sent false
8779 3298 Acked Flush 0 Done Done Sent false
8780 3299 Acked Read 2 Done Done Sent false
8781 3300 Acked Read 4 Done Done Sent false
8782 3301 Acked Write 1 Done Done Sent false
8783 3302 Acked Read 5 Done Done Sent false
8784 3303 Acked Read 8 Done Done Sent false
8785 3304 Acked Write 9 Done Done Sent false
8786 3305 Acked Write 10 Done Done Sent false
8787 3306 Acked Write 4 Done Done Sent false
8788 3307 Acked Read 8 Done Done Sent false
8789 3308 Acked Flush 0 Done Done Sent false
8790 3309 Acked Write 5 Done Done Sent false
8791 3310 Acked Flush 0 Done Done Sent false
8792 3311 Acked Read 8 Done Done Sent false
8793 3312 Acked Write 2 Done Done Sent false
8794 3313 Acked Write 1 Done Done Sent false
8795 3314 Acked Read 2 Done Done Sent false
8796 3315 Acked Read 7 Done Done Sent false
8797 3316 Acked Read 10 Done Done Sent false
8798 3317 Acked Read 3 Done Done Sent false
8799 3318 Acked Read 1 Done Done Sent false
8800 3319 Acked Read 9 Done Done Sent false
8801 3320 Acked Read 7 Done Done Sent false
8802 3321 Acked Read 6 Done Done Sent false
8803 3322 Acked Write 1 Done Done Sent false
8804 3323 Acked Read 6 Done Done Sent false
8805 3324 Acked Write 4 Done Done Sent false
8806 3325 Acked Write 5 Done Done Sent false
8807 3326 Acked Read 10 Done Done Sent false
8808 3327 Acked Read 1 Done Done Sent false
8809 3328 Acked Read 5 Done Done Sent false
8810 3329 Acked Write 3 Done Done Sent false
8811 3330 Acked Write 2 Done Done Sent false
8812 3331 Acked Write 3 Done Done Sent false
8813 3332 Acked Read 5 Done Done Sent false
8814 3333 Acked Write 6 Done Done Sent false
8815 3334 Acked Flush 0 Done Done Sent false
8816 3335 Acked Flush 0 Done Done Sent false
8817 3336 Acked Write 4 Done Done Sent false
8818 3337 Acked Read 6 Done Done Sent false
8819 3338 Acked Read 8 Done Done Sent false
8820 3339 Acked Read 3 Done Done Sent false
8821 3340 Acked Flush 0 Done Done Sent false
8822 3341 Acked Write 1 Done Done Sent false
8823 3342 Acked Read 7 Done Done Sent false
8824 3343 Acked Write 2 Done Done Sent false
8825 3344 Acked Read 10 Done Done Sent false
8826 3345 Acked Read 5 Done Done Sent false
8827 3346 Acked Write 10 Done Done Sent false
8828 3347 Acked Write 7 Done Done Sent false
8829 3348 Acked Write 6 Done Done Sent false
8830 3349 Acked Read 6 Done Done Sent false
8831 3350 Acked Flush 0 Done Done Sent false
8832 3351 Acked Write 7 Done Done Sent false
8833 3352 Acked Flush 0 Done Done Sent false
8834 3353 Acked Write 3 Done Done Sent false
8835 3354 Acked Read 9 Done Done Sent false
8836 3355 Acked Write 7 Done Done Sent false
8837 3356 Acked Read 8 Done Done Sent false
8838 3357 Acked Write 5 Done Done Sent false
8839 3358 Acked Read 3 Done Done Sent false
8840 3359 Acked Read 10 Done Done Sent false
8841 3360 Acked Write 1 Done Done Sent false
8842 3361 Acked Write 1 Done Done Sent false
8843 3362 Acked Write 6 Done Done Sent false
8844 3363 Acked Write 9 Done Done Sent false
8845 3364 Acked Write 2 Done Done Sent false
8846 3365 Acked Read 10 Done Done Sent false
8847 3366 Acked Write 2 Done Done Sent false
8848 3367 Acked Read 5 Done Done Sent false
8849 3368 Acked Write 8 Done Done Sent false
8850 3369 Acked Write 5 Done Done Sent false
8851 3370 Acked Write 8 Done Done Sent false
8852 3371 Acked Read 7 Done Done Sent false
8853 3372 Acked Write 3 Done Done Sent false
8854 3373 Acked Read 8 Done Done Sent false
8855 3374 Acked Flush 0 Done Done Sent false
8856 3375 Acked Read 4 Done Done Sent false
8857 3376 Acked Write 6 Done Done Sent false
8858 3377 Acked Write 5 Done Done Sent false
8859 3378 Acked Write 4 Done Done Sent false
8860 3379 Acked Write 10 Done Done Sent false
8861 3380 Acked Flush 0 Done Done Sent false
8862 3381 Acked Write 5 Done Done Sent false
8863 3382 Acked Read 3 Done Done Sent false
8864 3383 Acked Flush 0 Done Done Sent false
8865 3384 Acked Read 1 Done Done Sent false
8866 3385 Acked Read 10 Done Done Sent false
8867 3386 Acked Write 10 Done Done Sent false
8868 3387 Acked Write 1 Done Done Sent false
8869 3388 Acked Write 5 Done Done Sent false
8870 3389 Acked Read 8 Done Done Sent false
8871 3390 Acked Read 9 Done Done Sent false
8872 3391 Acked Write 6 Done Done Sent false
8873 3392 Acked Read 6 Done Done Sent false
8874 3393 Acked Write 7 Done Done Sent false
8875 3394 Acked Read 5 Done Done Sent false
8876 3395 Acked Read 5 Done Done Sent false
8877 3396 Acked Flush 0 Done Done Sent false
8878 3397 Acked Write 4 Done Done Sent false
8879 3398 Acked Write 7 Done Done Sent false
8880 3399 Acked Write 1 Done Done Sent false
8881 3400 Acked Read 3 Done Done Sent false
8882 3401 Acked Write 6 Done Done Sent false
8883 3402 Acked Write 5 Done Done Sent false
8884 3403 Acked Read 10 Done Done Sent false
8885 3404 Acked Read 6 Done Done Sent false
8886 3405 Acked Write 4 Done Done Sent false
8887 3406 Acked Read 4 Done Done Sent false
8888 3407 Acked Read 2 Done Done Sent false
8889 3408 Acked Write 9 Done Done Sent false
8890 3409 Acked Read 4 Done Done Sent false
8891 3410 Acked Write 4 Done Done Sent false
8892 3411 Acked Write 2 Done Done Sent false
8893 3412 Acked Write 5 Done Done Sent false
8894 3413 Acked Write 3 Done Done Sent false
8895 STATES DS:0 DS:1 DS:2 TOTAL
8896 Sent 0 0 301 301
8897 Done 301 301 0 602
8898 Skipped 0 0 0 0
8899 Error 0 0 0 0
8900 Last Flush: 3396 3396 3112
8901 Downstairs last five completed: 3112 3111 3110 3109 3108
8902 Upstairs last five completed: 3413 3412 3411 3410 3409
8903 CLIENT: Up:0 ds:301 act:2
8904 [8830] got start action
8905 [8830] I am going to restart
8906 Starting downstairs at port 8830
8907 Make output file at "/tmp/dsc/downstairs-8830.txt"
8908 [8830][2] reports Starting
8909 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1210)
8910 [8830][2] reports Running
8911 ----------------------------------------------------------------
8912 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8913 States: Active Active Offline
8914 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8915 3113 Acked Read 5 Done Done Sent false
8916 3114 Acked Write 4 Done Done Sent false
8917 3115 Acked Read 10 Done Done Sent false
8918 3116 Acked Write 4 Done Done Sent false
8919 3117 Acked Write 6 Done Done Sent false
8920 3118 Acked Write 2 Done Done Sent false
8921 3119 Acked Write 2 Done Done Sent false
8922 3120 Acked Read 8 Done Done Sent false
8923 3121 Acked Read 4 Done Done Sent false
8924 3122 Acked Read 8 Done Done Sent false
8925 3123 Acked Read 10 Done Done Sent false
8926 3124 Acked Flush 0 Done Done Sent false
8927 3125 Acked Write 3 Done Done Sent false
8928 3126 Acked Read 6 Done Done Sent false
8929 3127 Acked Read 4 Done Done Sent false
8930 3128 Acked Write 3 Done Done Sent false
8931 3129 Acked Write 7 Done Done Sent false
8932 3130 Acked Write 9 Done Done Sent false
8933 3131 Acked Read 4 Done Done Sent false
8934 3132 Acked Write 8 Done Done Sent false
8935 3133 Acked Read 5 Done Done Sent false
8936 3134 Acked Write 3 Done Done Sent false
8937 3135 Acked Read 8 Done Done Sent false
8938 3136 Acked Read 4 Done Done Sent false
8939 3137 Acked Write 8 Done Done Sent false
8940 3138 Acked Read 4 Done Done Sent false
8941 3139 Acked Flush 0 Done Done Sent false
8942 3140 Acked Read 10 Done Done Sent false
8943 3141 Acked Write 2 Done Done Sent false
8944 3142 Acked Flush 0 Done Done Sent false
8945 3143 Acked Write 1 Done Done Sent false
8946 3144 Acked Write 6 Done Done Sent false
8947 3145 Acked Read 1 Done Done Sent false
8948 3146 Acked Write 7 Done Done Sent false
8949 3147 Acked Flush 0 Done Done Sent false
8950 3148 Acked Write 9 Done Done Sent false
8951 3149 Acked Write 10 Done Done Sent false
8952 3150 Acked Write 10 Done Done Sent false
8953 3151 Acked Read 8 Done Done Sent false
8954 3152 Acked Read 8 Done Done Sent false
8955 3153 Acked Read 9 Done Done Sent false
8956 3154 Acked Write 8 Done Done Sent false
8957 3155 Acked Read 6 Done Done Sent false
8958 3156 Acked Write 5 Done Done Sent false
8959 3157 Acked Read 10 Done Done Sent false
8960 3158 Acked Write 1 Done Done Sent false
8961 3159 Acked Read 10 Done Done Sent false
8962 3160 Acked Read 7 Done Done Sent false
8963 3161 Acked Write 3 Done Done Sent false
8964 3162 Acked Read 5 Done Done Sent false
8965 3163 Acked Write 5 Done Done Sent false
8966 3164 Acked Read 3 Done Done Sent false
8967 3165 Acked Read 1 Done Done Sent false
8968 3166 Acked Read 7 Done Done Sent false
8969 3167 Acked Read 4 Done Done Sent false
8970 3168 Acked Read 7 Done Done Sent false
8971 3169 Acked Read 9 Done Done Sent false
8972 3170 Acked Write 5 Done Done Sent false
8973 3171 Acked Write 8 Done Done Sent false
8974 3172 Acked Write 10 Done Done Sent false
8975 3173 Acked Write 3 Done Done Sent false
8976 3174 Acked Read 10 Done Done Sent false
8977 3175 Acked Read 10 Done Done Sent false
8978 3176 Acked Read 6 Done Done Sent false
8979 3177 Acked Write 10 Done Done Sent false
8980 3178 Acked Read 8 Done Done Sent false
8981 3179 Acked Read 6 Done Done Sent false
8982 3180 Acked Read 8 Done Done Sent false
8983 3181 Acked Write 3 Done Done Sent false
8984 3182 Acked Flush 0 Done Done Sent false
8985 3183 Acked Write 9 Done Done Sent false
8986 3184 Acked Read 2 Done Done Sent false
8987 3185 Acked Flush 0 Done Done Sent false
8988 3186 Acked Read 2 Done Done Sent false
8989 3187 Acked Read 8 Done Done Sent false
8990 3188 Acked Flush 0 Done Done Sent false
8991 3189 Acked Write 8 Done Done Sent false
8992 3190 Acked Read 9 Done Done Sent false
8993 3191 Acked Read 2 Done Done Sent false
8994 3192 Acked Read 4 Done Done Sent false
8995 3193 Acked Write 3 Done Done Sent false
8996 3194 Acked Read 2 Done Done Sent false
8997 3195 Acked Read 5 Done Done Sent false
8998 3196 Acked Read 10 Done Done Sent false
8999 3197 Acked Read 5 Done Done Sent false
9000 3198 Acked Write 1 Done Done Sent false
9001 3199 Acked Read 3 Done Done Sent false
9002 3200 Acked Write 9 Done Done Sent false
9003 3201 Acked Write 10 Done Done Sent false
9004 3202 Acked Read 4 Done Done Sent false
9005 3203 Acked Write 3 Done Done Sent false
9006 3204 Acked Write 8 Done Done Sent false
9007 3205 Acked Flush 0 Done Done Sent false
9008 3206 Acked Read 1 Done Done Sent false
9009 3207 Acked Write 3 Done Done Sent false
9010 3208 Acked Read 6 Done Done Sent false
9011 3209 Acked Write 5 Done Done Sent false
9012 3210 Acked Read 2 Done Done Sent false
9013 3211 Acked Write 9 Done Done Sent false
9014 3212 Acked Read 1 Done Done Sent false
9015 3213 Acked Read 1 Done Done Sent false
9016 3214 Acked Read 10 Done Done Sent false
9017 3215 Acked Write 5 Done Done Sent false
9018 3216 Acked Write 3 Done Done Sent false
9019 3217 Acked Flush 0 Done Done Sent false
9020 3218 Acked Flush 0 Done Done Sent false
9021 3219 Acked Read 5 Done Done Sent false
9022 3220 Acked Write 7 Done Done Sent false
9023 3221 Acked Write 5 Done Done Sent false
9024 3222 Acked Write 5 Done Done Sent false
9025 3223 Acked Read 8 Done Done Sent false
9026 3224 Acked Write 9 Done Done Sent false
9027 3225 Acked Read 4 Done Done Sent false
9028 3226 Acked Read 3 Done Done Sent false
9029 3227 Acked Read 6 Done Done Sent false
9030 3228 Acked Write 9 Done Done Sent false
9031 3229 Acked Read 1 Done Done Sent false
9032 3230 Acked Write 5 Done Done Sent false
9033 3231 Acked Read 10 Done Done Sent false
9034 3232 Acked Write 3 Done Done Sent false
9035 3233 Acked Write 1 Done Done Sent false
9036 3234 Acked Read 3 Done Done Sent false
9037 3235 Acked Read 3 Done Done Sent false
9038 3236 Acked Write 5 Done Done Sent false
9039 3237 Acked Read 8 Done Done Sent false
9040 3238 Acked Write 7 Done Done Sent false
9041 3239 Acked Read 10 Done Done Sent false
9042 3240 Acked Read 1 Done Done Sent false
9043 3241 Acked Read 8 Done Done Sent false
9044 3242 Acked Flush 0 Done Done Sent false
9045 3243 Acked Write 7 Done Done Sent false
9046 3244 Acked Read 9 Done Done Sent false
9047 3245 Acked Write 10 Done Done Sent false
9048 3246 Acked Read 4 Done Done Sent false
9049 3247 Acked Read 1 Done Done Sent false
9050 3248 Acked Read 2 Done Done Sent false
9051 3249 Acked Read 10 Done Done Sent false
9052 3250 Acked Read 3 Done Done Sent false
9053 3251 Acked Write 4 Done Done Sent false
9054 3252 Acked Write 3 Done Done Sent false
9055 3253 Acked Read 8 Done Done Sent false
9056 3254 Acked Write 5 Done Done Sent false
9057 3255 Acked Write 5 Done Done Sent false
9058 3256 Acked Read 1 Done Done Sent false
9059 3257 Acked Write 6 Done Done Sent false
9060 3258 Acked Read 3 Done Done Sent false
9061 3259 Acked Read 6 Done Done Sent false
9062 3260 Acked Read 6 Done Done Sent false
9063 3261 Acked Write 3 Done Done Sent false
9064 3262 Acked Read 3 Done Done Sent false
9065 3263 Acked Read 9 Done Done Sent false
9066 3264 Acked Read 8 Done Done Sent false
9067 3265 Acked Read 10 Done Done Sent false
9068 3266 Acked Write 2 Done Done Sent false
9069 3267 Acked Write 5 Done Done Sent false
9070 3268 Acked Read 8 Done Done Sent false
9071 3269 Acked Read 6 Done Done Sent false
9072 3270 Acked Write 9 Done Done Sent false
9073 3271 Acked Write 3 Done Done Sent false
9074 3272 Acked Write 10 Done Done Sent false
9075 3273 Acked Read 8 Done Done Sent false
9076 3274 Acked Flush 0 Done Done Sent false
9077 3275 Acked Read 9 Done Done Sent false
9078 3276 Acked Read 8 Done Done Sent false
9079 3277 Acked Flush 0 Done Done Sent false
9080 3278 Acked Read 7 Done Done Sent false
9081 3279 Acked Flush 0 Done Done Sent false
9082 3280 Acked Read 1 Done Done Sent false
9083 3281 Acked Write 8 Done Done Sent false
9084 3282 Acked Write 10 Done Done Sent false
9085 3283 Acked Flush 0 Done Done Sent false
9086 3284 Acked Flush 0 Done Done Sent false
9087 3285 Acked Write 5 Done Done Sent false
9088 3286 Acked Write 5 Done Done Sent false
9089 3287 Acked Write 10 Done Done Sent false
9090 3288 Acked Write 7 Done Done Sent false
9091 3289 Acked Read 2 Done Done Sent false
9092 3290 Acked Write 9 Done Done Sent false
9093 3291 Acked Flush 0 Done Done Sent false
9094 3292 Acked Write 9 Done Done Sent false
9095 3293 Acked Flush 0 Done Done Sent false
9096 3294 Acked Write 4 Done Done Sent false
9097 3295 Acked Read 4 Done Done Sent false
9098 3296 Acked Flush 0 Done Done Sent false
9099 3297 Acked Write 2 Done Done Sent false
9100 3298 Acked Flush 0 Done Done Sent false
9101 3299 Acked Read 2 Done Done Sent false
9102 3300 Acked Read 4 Done Done Sent false
9103 3301 Acked Write 1 Done Done Sent false
9104 3302 Acked Read 5 Done Done Sent false
9105 3303 Acked Read 8 Done Done Sent false
9106 3304 Acked Write 9 Done Done Sent false
9107 3305 Acked Write 10 Done Done Sent false
9108 3306 Acked Write 4 Done Done Sent false
9109 3307 Acked Read 8 Done Done Sent false
9110 3308 Acked Flush 0 Done Done Sent false
9111 3309 Acked Write 5 Done Done Sent false
9112 3310 Acked Flush 0 Done Done Sent false
9113 3311 Acked Read 8 Done Done Sent false
9114 3312 Acked Write 2 Done Done Sent false
9115 3313 Acked Write 1 Done Done Sent false
9116 3314 Acked Read 2 Done Done Sent false
9117 3315 Acked Read 7 Done Done Sent false
9118 3316 Acked Read 10 Done Done Sent false
9119 3317 Acked Read 3 Done Done Sent false
9120 3318 Acked Read 1 Done Done Sent false
9121 3319 Acked Read 9 Done Done Sent false
9122 3320 Acked Read 7 Done Done Sent false
9123 3321 Acked Read 6 Done Done Sent false
9124 3322 Acked Write 1 Done Done Sent false
9125 3323 Acked Read 6 Done Done Sent false
9126 3324 Acked Write 4 Done Done Sent false
9127 3325 Acked Write 5 Done Done Sent false
9128 3326 Acked Read 10 Done Done Sent false
9129 3327 Acked Read 1 Done Done Sent false
9130 3328 Acked Read 5 Done Done Sent false
9131 3329 Acked Write 3 Done Done Sent false
9132 3330 Acked Write 2 Done Done Sent false
9133 3331 Acked Write 3 Done Done Sent false
9134 3332 Acked Read 5 Done Done Sent false
9135 3333 Acked Write 6 Done Done Sent false
9136 3334 Acked Flush 0 Done Done Sent false
9137 3335 Acked Flush 0 Done Done Sent false
9138 3336 Acked Write 4 Done Done Sent false
9139 3337 Acked Read 6 Done Done Sent false
9140 3338 Acked Read 8 Done Done Sent false
9141 3339 Acked Read 3 Done Done Sent false
9142 3340 Acked Flush 0 Done Done Sent false
9143 3341 Acked Write 1 Done Done Sent false
9144 3342 Acked Read 7 Done Done Sent false
9145 3343 Acked Write 2 Done Done Sent false
9146 3344 Acked Read 10 Done Done Sent false
9147 3345 Acked Read 5 Done Done Sent false
9148 3346 Acked Write 10 Done Done Sent false
9149 3347 Acked Write 7 Done Done Sent false
9150 3348 Acked Write 6 Done Done Sent false
9151 3349 Acked Read 6 Done Done Sent false
9152 3350 Acked Flush 0 Done Done Sent false
9153 3351 Acked Write 7 Done Done Sent false
9154 3352 Acked Flush 0 Done Done Sent false
9155 3353 Acked Write 3 Done Done Sent false
9156 3354 Acked Read 9 Done Done Sent false
9157 3355 Acked Write 7 Done Done Sent false
9158 3356 Acked Read 8 Done Done Sent false
9159 3357 Acked Write 5 Done Done Sent false
9160 3358 Acked Read 3 Done Done Sent false
9161 3359 Acked Read 10 Done Done Sent false
9162 3360 Acked Write 1 Done Done Sent false
9163 3361 Acked Write 1 Done Done Sent false
9164 3362 Acked Write 6 Done Done Sent false
9165 3363 Acked Write 9 Done Done Sent false
9166 3364 Acked Write 2 Done Done Sent false
9167 3365 Acked Read 10 Done Done Sent false
9168 3366 Acked Write 2 Done Done Sent false
9169 3367 Acked Read 5 Done Done Sent false
9170 3368 Acked Write 8 Done Done Sent false
9171 3369 Acked Write 5 Done Done Sent false
9172 3370 Acked Write 8 Done Done Sent false
9173 3371 Acked Read 7 Done Done Sent false
9174 3372 Acked Write 3 Done Done Sent false
9175 3373 Acked Read 8 Done Done Sent false
9176 3374 Acked Flush 0 Done Done Sent false
9177 3375 Acked Read 4 Done Done Sent false
9178 3376 Acked Write 6 Done Done Sent false
9179 3377 Acked Write 5 Done Done Sent false
9180 3378 Acked Write 4 Done Done Sent false
9181 3379 Acked Write 10 Done Done Sent false
9182 3380 Acked Flush 0 Done Done Sent false
9183 3381 Acked Write 5 Done Done Sent false
9184 3382 Acked Read 3 Done Done Sent false
9185 3383 Acked Flush 0 Done Done Sent false
9186 3384 Acked Read 1 Done Done Sent false
9187 3385 Acked Read 10 Done Done Sent false
9188 3386 Acked Write 10 Done Done Sent false
9189 3387 Acked Write 1 Done Done Sent false
9190 3388 Acked Write 5 Done Done Sent false
9191 3389 Acked Read 8 Done Done Sent false
9192 3390 Acked Read 9 Done Done Sent false
9193 3391 Acked Write 6 Done Done Sent false
9194 3392 Acked Read 6 Done Done Sent false
9195 3393 Acked Write 7 Done Done Sent false
9196 3394 Acked Read 5 Done Done Sent false
9197 3395 Acked Read 5 Done Done Sent false
9198 3396 Acked Flush 0 Done Done Sent false
9199 3397 Acked Write 4 Done Done Sent false
9200 3398 Acked Write 7 Done Done Sent false
9201 3399 Acked Write 1 Done Done Sent false
9202 3400 Acked Read 3 Done Done Sent false
9203 3401 Acked Write 6 Done Done Sent false
9204 3402 Acked Write 5 Done Done Sent false
9205 3403 Acked Read 10 Done Done Sent false
9206 3404 Acked Read 6 Done Done Sent false
9207 3405 Acked Write 4 Done Done Sent false
9208 3406 Acked Read 4 Done Done Sent false
9209 3407 Acked Read 2 Done Done Sent false
9210 3408 Acked Write 9 Done Done Sent false
9211 3409 Acked Read 4 Done Done Sent false
9212 3410 Acked Write 4 Done Done Sent false
9213 3411 Acked Write 2 Done Done Sent false
9214 3412 Acked Write 5 Done Done Sent false
9215 3413 Acked Write 3 Done Done Sent false
9216 3414 Acked Flush 0 Done Done Sent false
9217 STATES DS:0 DS:1 DS:2 TOTAL
9218 Sent 0 0 302 302
9219 Done 302 302 0 604
9220 Skipped 0 0 0 0
9221 Error 0 0 0 0
9222 Last Flush: 3414 3414 3112
9223 Downstairs last five completed: 3112 3111 3110 3109 3108
9224 Upstairs last five completed: 3414 3413 3412 3411 3410
9225 CLIENT: Up:0 ds:302 act:2
92262025-01-15T05:20:54.687ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92272025-01-15T05:20:54.687ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38466) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92282025-01-15T05:20:54.688ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92292025-01-15T05:20:54.689ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92302025-01-15T05:20:54.689ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92312025-01-15T05:20:54.689ZINFOcrucible: send last flush ID to this DS: 3112 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92322025-01-15T05:20:54.689ZINFOcrucible: Replied this last flush ID: 3112 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92332025-01-15T05:20:54.689ZINFOcrucible: [2] client re-new 302 jobs since flush 3112 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92342025-01-15T05:20:54.690ZINFOcrucible: [2] Replayed 302 jobs since flush: 3112 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
9235 ----------------------------------------------------------------
9236 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9237 Downstairs last five completed: 3414 3413 3412 3411 3410
9238 Upstairs last five completed: 3414 3413 3412 3411 3410
9239 CLIENT: Up:0 ds:0 act:3
9240 Replay: All jobs finished
9241 Main task has work to do, go find it
9242 got dsc Stop(0)
9243 stop 0
9244 [8810] Got stop action so:false kr:false
9245 Jan 15 05:20:56.923 INFO request completed, latency_us: 328, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 97decdf7-3bd3-43ea-8ca4-0dd6be6b7ac2, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
9246 Jan 15 05:20:56.924 INFO request completed, latency_us: 139, response_code: 200, uri: /state/cid/0, method:{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:20:56.924420439Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
9247 GET, req_id: 19f9348d-ad83-4b23-b8ed-559b0efcecbd, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:20:56.924509061Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
9248 :9998
92492025-01-15T05:20:56.924ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
92502025-01-15T05:20:56.924ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
9251 [8810] Exited with: ExitStatus(unix_wait_status(9))
9252 [8810][0] reports Exit
9253 Jan 15 05:21:00.926 INFO request completed, latency_us: 194, response_code: 200, uri: /state/cid/0, method: GET, req_id: 042381f9-e41e-4f05-81a9-1cfd74ea6574, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
9254 001/300 Read block 3654 len 36864
9255 002/300 Read block 2302 len 16384
9256 003/300 Read block 4203 len 36864
9257 004/300 Read block 1631 len 20480
9258 005/300 Flush
9259 006/300 Flush
9260 007/300 Write block 4980 len 28672 data: 5 3 3 2 2 2 2
9261 008/300 Flush
9262 009/300 Write block 3671 len 28672 data: 4 4 3 4 4 4 3
9263 010/300 Read block 841 len 28672
9264 011/300 Read block 144 len 32768
9265 012/300 Write block 138 len 8192 data: 3 3
9266 013/300 Read block 3467 len 20480
9267 014/300 Read block 1678 len 40960
9268 015/300 Read block 138 len 8192
9269 016/300 Read block 1512 len 12288
9270 017/300 Write block 303 len 40960 data: 3 3 2 2 2 2 2 3 3 2
9271 018/300 Read block 2976 len 20480
9272 019/300 Write block 373 len 40960 data: 2 2 3 3 3 3 3 3 3 3
9273 020/300 Read block 4223 len 16384
9274 021/300 Write block 772 len 32768 data: 2 2 2 2 2 2 2 2
9275 022/300 Read block 4190 len 8192
9276 023/300 Read block 3606 len 36864
9277 024/300 Read block 1764 len 4096
9278 025/300 Read block 2201 len 24576
9279 026/300 Write block 1904 len 8192 data: 3 3
9280 027/300 Flush
9281 028/300 Read block 4093 len 24576
9282 029/300 Read block 2530 len 4096
9283 030/300 Write block 3078 len 12288 data: 3 2 2
9284 031/300 Read block 979 len 28672
9285 032/300 Write block 2407 len 32768 data: 3 3 4 3 3 3 3 2
9286 033/300 Write block 3101 len 24576 data: 3 3 3 3 3 3
9287 034/300 Write block 3779 len 8192 data: 2 3
9288 035/300 Read block 1710 len 16384
9289 036/300 Write block 3413 len 28672 data: 3 3 4 4 4 4 3
9290 037/300 Read block 2814 len 12288
9291 038/300 Read block 4091 len 20480
9292 039/300 Read block 3532 len 12288
9293 040/300 Flush
9294 041/300 Read block 261 len 32768
9295 042/300 Read block 986 len 16384
9296 043/300 Write block 4871 len 16384 data: 3 4 4 5
9297 044/300 Read block 2964 len 12288
9298 045/300 Read block 1944 len 12288
9299 046/300 Write block 4275 len 20480 data: 2 2 3 3 4
9300 047/300 Write block 1572 len 36864 data: 5 6 6 6 4 4 4 4 2
9301 048/300 Read block 3597 len 8192
9302 049/300 Read block 515 len 20480
9303 050/300 Write block 4408 len 8192 data: 4 3
9304 051/300 Write block 2456 len 32768 data: 4 3 3 4 5 5 5 4
9305 052/300 Flush
9306 053/300 Flush
9307 054/300 Write block 4695 len 24576 data: 2 3 2 2 2 2
9308 055/300 Write block 3931 len 20480 data: 2 2 3 3 3
9309 056/300 Read block 2269 len 20480
9310 057/300 Write block 1145 len 36864 data: 3 4 5 5 4 4 2 2 4
9311 058/300 Write block 1894 len 28672 data: 5 4 3 3 2 2 3
9312 059/300 Write block 2820 len 16384 data: 2 3 4 3
9313 060/300 Write block 1314 len 28672 data: 2 2 2 3 2 3 3
9314 061/300 Read block 3778 len 32768
9315 062/300 Write block 2244 len 32768 data: 4 3 3 4 3 3 3 3
9316 063/300 Write block 3262 len 16384 data: 3 2 3 3
9317 064/300 Read block 1424 len 4096
9318 065/300 Write block 636 len 8192 data: 3 3
9319 066/300 Flush
9320 067/300 Read block 1265 len 4096
9321 068/300 Write block 4499 len 16384 data: 6 5 6 5
9322 069/300 Read block 4631 len 16384
9323 070/300 Read block 2400 len 4096
9324 071/300 Write block 2881 len 40960 data: 3 3 3 4 4 4 2 2 2 2
9325 072/300 Read block 1200 len 28672
9326 073/300 Write block 263 len 16384 data: 3 3 3 4
9327 074/300 Read block 3756 len 36864
9328 075/300 Write block 1301 len 4096 data: 2
9329 076/300 Write block 1947 len 8192 data: 2 2
9330 077/300 Write block 2830 len 16384 data: 3 3 3 5
9331 078/300 Write block 2485 len 16384 data: 4 4 4 4
9332 079/300 Write block 3382 len 8192 data: 6 5
9333 080/300 Read block 2413 len 36864
9334 081/300 Read block 2664 len 36864
9335 082/300 Read block 3245 len 4096
9336 083/300 Flush
9337 084/300 Read block 3338 len 4096
9338 085/300 Write block 2632 len 24576 data: 2 3 3 3 3 3
9339 086/300 Write block 804 len 4096 data: 3
9340 087/300 Write block 35 len 24576 data: 3 3 3 3 2 2
9341 088/300 Read block 4969 len 32768
9342 089/300 Write block 4963 len 24576 data: 2 2 2 2 2 2
9343 090/300 Write block 100 len 28672 data: 2 4 4 4 4 3 2
9344 091/300 Write block 1563 len 12288 data: 3 4 3
9345 092/300 Write block 1847 len 4096 data: 5
9346 093/300 Read block 195 len 28672
9347 094/300 Read block 1297 len 16384
9348 095/300 Read block 2034 len 24576
9349 096/300 Read block 163 len 32768
9350 097/300 Write block 2594 len 28672 data: 5 5 2 4 4 4 4
9351 098/300 Read block 2349 len 16384
9352 099/300 Read block 1152 len 20480
9353 100/300 Write block 4051 len 4096 data: 5
9354 101/300 Write block 1996 len 24576 data: 2 3 3 3 3 3
9355 102/300 Read block 1117 len 16384
9356 103/300 Read block 181 len 12288
9357 104/300 Read block 4563 len 4096
9358 105/300 Write block 3600 len 32768 data: 3 3 3 3 3 4 4 4
9359 106/300 Write block 4834 len 28672 data: 3 3 3 3 3 3 3
9360 107/300 Read block 711 len 20480
9361 108/300 Read block 765 len 8192
9362 109/300 Read block 3188 len 12288
9363 110/300 Write block 3754 len 32768 data: 3 3 2 3 4 4 5 4
9364 111/300 Read block 1620 len 4096
9365 112/300 Write block 1640 len 8192 data: 6 6
9366 113/300 Read block 3736 len 32768
9367 114/300 Read block 4683 len 8192
9368 115/300 Read block 3563 len 24576
9369 116/300 Read block 2896 len 20480
9370 117/300 Read block 1670 len 24576
9371 118/300 Write block 2223 len 32768 data: 4 3 3 4 4 4 4 3
9372 119/300 Read block 3940 len 16384
9373 120/300 Flush
9374 121/300 Write block 752 len 28672 data: 4 4 3 2 2 2 3
9375 122/300 Read block 4050 len 16384
9376 123/300 Read block 2476 len 32768
9377 124/300 Read block 2630 len 20480
9378 125/300 Flush
9379 126/300 Read block 3337 len 28672
9380 127/300 Read block 1565 len 24576
9381 128/300 Read block 4651 len 12288
9382 129/300 Write block 2365 len 8192 data: 5 7
9383 130/300 Read block 3160 len 8192
9384 131/300 Read block 1506 len 40960
9385 132/300 Write block 2240 len 40960 data: 3 3 3 4 5 4 4 5 4 4
9386 133/300 Read block 3590 len 40960
9387 134/300 Write block 793 len 16384 data: 3 3 3 3
9388 135/300 Read block 485 len 40960
9389 136/300 Read block 2485 len 24576
9390 137/300 Write block 4452 len 40960 data: 3 3 3 3 3 3 2 2 2 2
9391 138/300 Read block 3676 len 4096
9392 139/300 Read block 4736 len 32768
9393 140/300 Read block 1478 len 40960
9394 141/300 Write block 450 len 28672 data: 2 2 2 2 2 2 2
9395 142/300 Read block 3385 len 24576
9396 143/300 Read block 1614 len 4096
9397 144/300 Write block 1848 len 40960 data: 4 4 3 4 4 3 2 2 2 2
9398 145/300 Read block 1139 len 32768
9399 146/300 Flush
9400 147/300 Read block 4580 len 24576
9401 148/300 Read block 2740 len 12288
9402 149/300 Write block 4651 len 36864 data: 2 2 2 2 2 3 3 2 2
9403 150/300 Write block 2540 len 4096 data: 3
9404 151/300 Write block 2700 len 8192 data: 2 2
9405 152/300 Flush
9406 153/300 Read block 1660 len 28672
9407 154/300 Read block 4825 len 8192
9408 155/300 Read block 748 len 28672
9409 156/300 Read block 1074 len 4096
9410 157/300 Write block 3777 len 28672 data: 2 2 3 4 3 3 3
9411 158/300 Read block 2195 len 32768
9412 159/300 Write block 759 len 16384 data: 2 2 2 2
9413 160/300 Read block 4155 len 36864
9414 161/300 Write block 2486 len 24576 data: 5 5 5 4 4 4
9415 162/300 Read block 4332 len 16384
9416 163/300 Read block 213 len 16384
9417 164/300 Flush
9418 165/300 Write block 2521 len 40960 data: 2 2 2 2 3 3 3 3 3 2
9419 166/300 Read block 642 len 4096
9420 167/300 Read block 1066 len 40960
9421 168/300 Read block 2311 len 36864
9422 169/300 Flush
9423 170/300 Write block 3872 len 8192 data: 5 5
9424 171/300 Read block 4486 len 32768
9425 172/300 Read block 4522 len 12288
9426 173/300 Write block 4132 len 12288 data: 5 3 3
9427 174/300 Flush
9428 175/300 Read block 369 len 36864
9429 176/300 Write block 3089 len 28672 data: 2 2 2 2 2 2 2
9430 177/300 Read block 2170 len 40960
9431 178/300 Read block 587 len 24576
9432 179/300 Read block 1204 len 12288
9433 180/300 Read block 1821 len 20480
9434 181/300 Read block 2529 len 32768
9435 182/300 Read block 2102 len 28672
9436 183/300 Write block 4651 len 12288 data: 3 3 3
9437 184/300 Write block 2588 len 12288 data: 5 5 6
9438 185/300 Read block 4904 len 16384
9439 186/300 Write block 2867 len 4096 data: 5
9440 187/300 Write block 4700 len 8192 data: 3 2
9441 188/300 Write block 563 len 28672 data: 2 2 2 3 2 3 3
9442 189/300 Read block 4368 len 32768
9443 190/300 Write block 4421 len 36864 data: 4 4 5 4 5 5 5 5 4
9444 191/300 Read block 2176 len 16384
9445 192/300 Read block 4156 len 16384
9446 193/300 Read block 358 len 4096
9447 194/300 Read block 1601 len 16384
9448 195/300 Write block 149 len 24576 data: 3 3 3 3 2 2
9449 196/300 Flush
9450 197/300 Read block 3951 len 32768
9451 198/300 Write block 2780 len 36864 data: 4 3 4 4 4 3 3 3 3
9452 199/300 Read block 1679 len 16384
9453 200/300 Flush
9454 201/300 Write block 4118 len 36864 data: 5 5 5 4 4 2 2 4 4
9455 202/300 Read block 3739 len 8192
9456 203/300 Write block 497 len 4096 data: 2
9457 204/300 Read block 1663 len 12288
9458 205/300 Flush
9459 206/300 Write block 4893 len 40960 data: 3 3 3 4 5 6 5 6 5 5
9460 207/300 Read block 1044 len 4096
9461 208/300 Read block 4948 len 28672
9462 209/300 Read block 4060 len 40960
9463 210/300 Flush
9464 211/300 Write block 4114 len 36864 data: 4 4 4 5 6 6 6 5 5
9465 212/300 Write block 805 len 28672 data: 3 2 2 2 2 2 2
9466 213/300 Read block 121 len 8192
9467 214/300 Read block 1724 len 24576
9468 215/300 Read block 2040 len 40960
9469 216/300 Read block 1241 len 24576
9470 217/300 Flush
9471 218/300 Read block 4666 len 28672
9472 219/300 Write block 4258 len 20480 data: 4 5 5 5 4
9473 220/300 Read block 2695 len 32768
9474 221/300 Write block 4193 len 12288 data: 4 4 3
9475 222/300 Read block 4854 len 40960
9476 223/300 Read block 4208 len 24576
9477 224/300 Read block 1323 len 24576
9478 225/300 Flush
9479 226/300 Read block 3150 len 20480
9480 227/300 Write block 1436 len 20480 data: 3 2 2 2 2
9481 228/300 Write block 3000 len 20480 data: 2 2 2 2 2
9482 229/300 Flush
9483 230/300 Read block 4805 len 4096
9484 231/300 Read block 3300 len 32768
9485 232/300 Read block 467 len 4096
9486 233/300 Flush
9487 234/300 Flush
9488 235/300 Read block 2987 len 12288
9489 236/300 Write block 3789 len 32768 data: 2 2 2 2 2 2 4 4
9490 237/300 Write block 634 len 4096 data: 3
9491 238/300 Read block 4682 len 4096
9492 239/300 Read block 1020 len 12288
9493 240/300 Write block 3763 len 24576 data: 4 3 4 4 3 3
9494 241/300 Flush
9495 242/300 Flush
9496 243/300 Read block 3626 len 20480
9497 244/300 Read block 1059 len 4096
9498 245/300 Read block 187 len 24576
9499 246/300 Read block 1381 len 40960
9500 247/300 Read block 2348 len 4096
9501 248/300 Read block 3497 len 40960
9502 249/300 Write block 4816 len 20480 data: 2 4 3 3 3
9503 250/300 Read block 2730 len 24576
9504 251/300 Read block 4730 len 36864
9505 252/300 Write block 870 len 12288 data: 4 4 4
9506 253/300 Flush
9507 254/300 Write block 1188 len 36864 data: 3 3 3 3 3 3 3 2 2
9508 255/300 Read block 1735 len 16384
9509 256/300 Write block 2029 len 8192 data: 6 5
9510 257/300 Read block 3488 len 8192
9511 258/300 Read block 3201 len 8192
9512 259/300 Read block 3134 len 20480
9513 260/300 Read block 2140 len 8192
9514 261/300 Flush
9515 262/300 Write block 1606 len 4096 data: 3
9516 263/300 Read block 2019 len 32768
9517 264/300 Read block 2694 len 8192
9518 265/300 Write block 4543 len 12288 data: 3 2 3
9519 266/300 Read block 3727 len 36864
9520 267/300 Read block 116 len 20480
9521 268/300 Read block 1838 len 32768
9522 269/300 Write block 4685 len 40960 data: 3 3 3 2 2 2 2 2 2 2
9523 270/300 Read block 4102 len 12288
9524 271/300 Read block 3803 len 36864
9525 272/300 Read block 793 len 4096
9526 273/300 Read block 2267 len 20480
9527 274/300 Write block 3676 len 32768 data: 5 4 4 4 5 5 5 4
9528 275/300 Write block 2755 len 40960 data: 3 4 4 4 5 6 6 5 5 5
9529 276/300 Flush
9530 277/300 Write block 1170 len 8192 data: 5 5
9531 278/300 Read block 3423 len 40960
9532 279/300 Write block 1678 len 40960 data: 2 2 2 3 4 3 3 3 2 2
9533 280/300 Write block 1828 len 8192 data: 4 4
9534 281/300 Write block 3030 len 4096 data: 2
9535 282/300 Write block 75 len 24576 data: 4 4 4 4 4 4
9536 283/300 Read block 3900 len 8192
9537 284/300 Write block 181 len 32768 data: 2 3 3 3 3 3 3 3
9538 285/300 Read block 1949 len 32768
9539 286/300 Write block 1242 len 16384 data: 2 2 2 3
9540 287/300 Flush
9541 288/300 Read block 3567 len 40960
9542 289/300 Write block 442 len 32768 data: 7 7 7 7 8 6 5 4
9543 290/300 Write block 2543 len 4096 data: 3
9544 291/300 Write block 4771 len 36864 data: 3 3 2 2 2 3 4 4 3
9545 292/300 Write block 4733 len 16384 data: 5 5 4 3
9546 293/300 Write block 4704 len 4096 data: 2
9547 294/300 Read block 855 len 16384
9548 295/300 Read block 2118 len 32768
9549 296/300 Read block 1455 len 12288
9550 297/300 Read block 4943 len 4096
9551 298/300 Read block 2937 len 20480
9552 299/300 Read block 2181 len 40960
9553 300/300 Write block 4743 len 20480 data: 4 5 5 3 2
9554 301/300 Read block 3845 len 32768
9555 Main task has work to do, go find it
9556 got dsc Start(0)
9557 start 0
9558 Jan 15 05:21:01.169 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/0, method: POST, req_id: 3a49999b-f2f2-4e29-8ca8-37f342cbad5b, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
9559 Replay: started 0, returned:Ok(())
9560 ----------------------------------------------------------------
9561 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9562 States: Offline Active Active
9563 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9564 3415 Acked Read 9 Sent Done Done false
9565 3416 Acked Read 4 Sent Done Done false
9566 3417 Acked Read 9 Sent Done Done false
9567 3418 Acked Read 5 Sent Done Done false
9568 3419 Acked Flush 0 Sent Done Done false
9569 3420 Acked Flush 0 Sent Done Done false
9570 3421 Acked Write 7 Sent Done Done false
9571 3422 Acked Flush 0 Sent Done Done false
9572 3423 Acked Write 7 Sent Done Done false
9573 3424 Acked Read 7 Sent Done Done false
9574 3425 Acked Read 8 Sent Done Done false
9575 3426 Acked Write 2 Sent Done Done false
9576 3427 Acked Read 5 Sent Done Done false
9577 3428 Acked Read 10 Sent Done Done false
9578 3429 Acked Read 2 Sent Done Done false
9579 3430 Acked Read 3 Sent Done Done false
9580 3431 Acked Write 10 Sent Done Done false
9581 3432 Acked Read 5 Sent Done Done false
9582 3433 Acked Write 10 Sent Done Done false
9583 3434 Acked Read 4 Sent Done Done false
9584 3435 Acked Write 8 Sent Done Done false
9585 3436 Acked Read 2 Sent Done Done false
9586 3437 Acked Read 9 Sent Done Done false
9587 3438 Acked Read 1 Sent Done Done false
9588 3439 Acked Read 6 Sent Done Done false
9589 3440 Acked Write 2 Sent Done Done false
9590 3441 Acked Flush 0 Sent Done Done false
9591 3442 Acked Read 6 Sent Done Done false
9592 3443 Acked Read 1 Sent Done Done false
9593 3444 Acked Write 3 Sent Done Done false
9594 3445 Acked Read 7 Sent Done Done false
9595 3446 Acked Write 8 Sent Done Done false
9596 3447 Acked Write 6 Sent Done Done false
9597 3448 Acked Write 2 Sent Done Done false
9598 3449 Acked Read 4 Sent Done Done false
9599 3450 Acked Write 7 Sent Done Done false
9600 3451 Acked Read 3 Sent Done Done false
9601 3452 Acked Read 5 Sent Done Done false
9602 3453 Acked Read 3 Sent Done Done false
9603 3454 Acked Flush 0 Sent Done Done false
9604 3455 Acked Read 8 Sent Done Done false
9605 3456 Acked Read 4 Sent Done Done false
9606 3457 Acked Write 4 Sent Done Done false
9607 3458 Acked Read 3 Sent Done Done false
9608 3459 Acked Read 3 Sent Done Done false
9609 3460 Acked Write 5 Sent Done Done false
9610 3461 Acked Write 9 Sent Done Done false
9611 3462 Acked Read 2 Sent Done Done false
9612 3463 Acked Read 5 Sent Done Done false
9613 3464 Acked Write 2 Sent Done Done false
9614 3465 Acked Write 8 Sent Done Done false
9615 3466 Acked Flush 0 Sent Done Done false
9616 3467 Acked Flush 0 Sent Done Done false
9617 3468 Acked Write 6 Sent Done Done false
9618 3469 Acked Write 5 Sent Done Done false
9619 3470 Acked Read 5 Sent Done Done false
9620 3471 Acked Write 9 Sent Done Done false
9621 3472 Acked Write 7 Sent Done Done false
9622 3473 Acked Write 4 Sent Done Done false
9623 3474 Acked Write 7 Sent Done Done false
9624 3475 Acked Read 8 Sent Done Done false
9625 3476 Acked Write 8 Sent Done Done false
9626 3477 Acked Write 4 Sent Done Done false
9627 3478 Acked Read 1 Sent Done Done false
9628 3479 Acked Write 2 Sent Done Done false
9629 3480 Acked Flush 0 Sent Done Done false
9630 3481 Acked Read 1 Sent Done Done false
9631 3482 Acked Write 4 Sent Done Done false
9632 3483 Acked Read 4 Sent Done Done false
9633 3484 Acked Read 1 Sent Done Done false
9634 3485 Acked Write 10 Sent Done Done false
9635 3486 Acked Read 7 Sent Done Done false
9636 3487 Acked Write 4 Sent Done Done false
9637 3488 Acked Read 9 Sent Done Done false
9638 3489 Acked Write 1 Sent Done Done false
9639 3490 Acked Write 2 Sent Done Done false
9640 3491 Acked Write 4 Sent Done Done false
9641 3492 Acked Write 4 Sent Done Done false
9642 3493 Acked Write 2 Sent Done Done false
9643 3494 Acked Read 9 Sent Done Done false
9644 3495 Acked Read 9 Sent Done Done false
9645 3496 Acked Read 1 Sent Done Done false
9646 3497 Acked Flush 0 Sent Done Done false
9647 3498 Acked Read 1 Sent Done Done false
9648 3499 Acked Write 6 Sent Done Done false
9649 3500 Acked Write 1 Sent Done Done false
9650 3501 Acked Write 6 Sent Done Done false
9651 3502 Acked Read 8 Sent Done Done false
9652 3503 Acked Write 6 Sent Done Done false
9653 3504 Acked Write 7 Sent Done Done false
9654 3505 Acked Write 3 Sent Done Done false
9655 3506 Acked Write 1 Sent Done Done false
9656 3507 Acked Read 7 Sent Done Done false
9657 3508 Acked Read 4 Sent Done Done false
9658 3509 Acked Read 6 Sent Done Done false
9659 3510 Acked Read 8 Sent Done Done false
9660 3511 Acked Write 7 Sent Done Done false
9661 3512 Acked Read 4 Sent Done Done false
9662 3513 Acked Read 5 Sent Done Done false
9663 3514 Acked Write 1 Sent Done Done false
9664 3515 Acked Write 6 Sent Done Done false
9665 3516 Acked Read 4 Sent Done Done false
9666 3517 Acked Read 3 Sent Done Done false
9667 3518 Acked Read 1 Sent Done Done false
9668 3519 Acked Write 8 Sent Done Done false
9669 3520 Acked Write 7 Sent Done Done false
9670 3521 Acked Read 5 Sent Done Done false
9671 3522 Acked Read 2 Sent Done Done false
9672 3523 Acked Read 3 Sent Done Done false
9673 3524 Acked Write 8 Sent Done Done false
9674 3525 Acked Read 1 Sent Done Done false
9675 3526 Acked Write 2 Sent Done Done false
9676 3527 Acked Read 8 Sent Done Done false
9677 3528 Acked Read 2 Sent Done Done false
9678 3529 Acked Read 6 Sent Done Done false
9679 3530 Acked Read 5 Sent Done Done false
9680 3531 Acked Read 6 Sent Done Done false
9681 3532 Acked Write 8 Sent Done Done false
9682 3533 Acked Read 4 Sent Done Done false
9683 3534 Acked Flush 0 Sent Done Done false
9684 3535 Acked Write 7 Sent Done Done false
9685 3536 Acked Read 4 Sent Done Done false
9686 3537 Acked Read 8 Sent Done Done false
9687 3538 Acked Read 5 Sent Done Done false
9688 3539 Acked Flush 0 Sent Done Done false
9689 3540 Acked Read 7 Sent Done Done false
9690 3541 Acked Read 6 Sent Done Done false
9691 3542 Acked Read 3 Sent Done Done false
9692 3543 Acked Write 2 Sent Done Done false
9693 3544 Acked Read 2 Sent Done Done false
9694 3545 Acked Read 10 Sent Done Done false
9695 3546 Acked Write 10 Sent Done Done false
9696 3547 Acked Read 10 Sent Done Done false
9697 3548 Acked Write 4 Sent Done Done false
9698 3549 Acked Read 10 Sent Done Done false
9699 3550 Acked Read 6 Sent Done Done false
9700 3551 Acked Write 10 Sent Done Done false
9701 3552 Acked Read 1 Sent Done Done false
9702 3553 Acked Read 8 Sent Done Done false
9703 3554 Acked Read 10 Sent Done Done false
9704 3555 Acked Write 7 Sent Done Done false
9705 3556 Acked Read 6 Sent Done Done false
9706 3557 Acked Read 1 Sent Done Done false
9707 3558 Acked Write 10 Sent Done Done false
9708 3559 Acked Read 8 Sent Done Done false
9709 3560 Acked Flush 0 Sent Done Done false
9710 3561 Acked Read 6 Sent Done Done false
9711 3562 Acked Read 3 Sent Done Done false
9712 3563 Acked Write 9 Sent Done Done false
9713 3564 Acked Write 1 Sent Done Done false
9714 3565 Acked Write 2 Sent Done Done false
9715 3566 Acked Flush 0 Sent Done Done false
9716 3567 Acked Read 7 Sent Done Done false
9717 3568 Acked Read 2 Sent Done Done false
9718 3569 Acked Read 7 Sent Done Done false
9719 3570 Acked Read 1 Sent Done Done false
9720 3571 Acked Write 7 Sent Done Done false
9721 3572 Acked Read 8 Sent Done Done false
9722 3573 Acked Write 4 Sent Done Done false
9723 3574 Acked Read 9 Sent Done Done false
9724 3575 Acked Write 6 Sent Done Done false
9725 3576 Acked Read 4 Sent Done Done false
9726 3577 Acked Read 4 Sent Done Done false
9727 3578 Acked Flush 0 Sent Done Done false
9728 3579 Acked Write 10 Sent Done Done false
9729 3580 Acked Read 1 Sent Done Done false
9730 3581 Acked Read 10 Sent Done Done false
9731 3582 Acked Read 9 Sent Done Done false
9732 3583 Acked Flush 0 Sent Done Done false
9733 3584 Acked Write 2 Sent Done Done false
9734 3585 Acked Read 8 Sent Done Done false
9735 3586 Acked Read 3 Sent Done Done false
9736 3587 Acked Write 3 Sent Done Done false
9737 3588 Acked Flush 0 Sent Done Done false
9738 3589 Acked Read 9 Sent Done Done false
9739 3590 Acked Write 7 Sent Done Done false
9740 3591 Acked Read 10 Sent Done Done false
9741 3592 Acked Read 6 Sent Done Done false
9742 3593 Acked Read 3 Sent Done Done false
9743 3594 Acked Read 5 Sent Done Done false
9744 3595 Acked Read 8 Sent Done Done false
9745 3596 Acked Read 7 Sent Done Done false
9746 3597 Acked Write 3 Sent Done Done false
9747 3598 Acked Write 3 Sent Done Done false
9748 3599 Acked Read 4 Sent Done Done false
9749 3600 Acked Write 1 Sent Done Done false
9750 3601 Acked Write 2 Sent Done Done false
9751 3602 Acked Write 7 Sent Done Done false
9752 3603 Acked Read 8 Sent Done Done false
9753 3604 Acked Write 9 Sent Done Done false
9754 3605 Acked Read 4 Sent Done Done false
9755 3606 Acked Read 4 Sent Done Done false
9756 3607 Acked Read 1 Sent Done Done false
9757 3608 Acked Read 4 Sent Done Done false
9758 3609 Acked Write 6 Sent Done Done false
9759 3610 Acked Flush 0 Sent Done Done false
9760 3611 Acked Read 8 Sent Done Done false
9761 3612 Acked Write 9 Sent Done Done false
9762 3613 Acked Read 4 Sent Done Done false
9763 3614 Acked Flush 0 Sent Done Done false
9764 3615 Acked Write 9 Sent Done Done false
9765 3616 Acked Read 2 Sent Done Done false
9766 3617 Acked Write 1 Sent Done Done false
9767 3618 Acked Read 3 Sent Done Done false
9768 3619 Acked Flush 0 Sent Done Done false
9769 3620 Acked Write 10 Sent Done Done false
9770 3621 Acked Read 1 Sent Done Done false
9771 3622 Acked Read 7 Sent Done Done false
9772 3623 Acked Read 10 Sent Done Done false
9773 3624 Acked Flush 0 Sent Done Done false
9774 3625 Acked Write 9 Sent Done Done false
9775 3626 Acked Write 7 Sent Done Done false
9776 3627 Acked Read 2 Sent Done Done false
9777 3628 Acked Read 6 Sent Done Done false
9778 3629 Acked Read 10 Sent Done Done false
9779 3630 Acked Read 6 Sent Done Done false
9780 3631 Acked Flush 0 Sent Done Done false
9781 3632 Acked Read 7 Sent Done Done false
9782 3633 Acked Write 5 Sent Done Done false
9783 3634 Acked Read 8 Sent Done Done false
9784 3635 Acked Write 3 Sent Done Done false
9785 3636 Acked Read 10 Sent Done Done false
9786 3637 Acked Read 6 Sent Done Done false
9787 3638 Acked Read 6 Sent Done Done false
9788 3639 Acked Flush 0 Sent Done Done false
9789 3640 Acked Read 5 Sent Done Done false
9790 3641 Acked Write 5 Sent Done Done false
9791 3642 Acked Write 5 Sent Done Done false
9792 3643 Acked Flush 0 Sent Done Done false
9793 3644 Acked Read 1 Sent Done Done false
9794 3645 Acked Read 8 Sent Done Done false
9795 3646 Acked Read 1 Sent Done Done false
9796 3647 Acked Flush 0 Sent Done Done false
9797 3648 Acked Flush 0 Sent Done Done false
9798 3649 Acked Read 3 Sent Done Done false
9799 3650 Acked Write 8 Sent Done Done false
9800 3651 Acked Write 1 Sent Done Done false
9801 3652 Acked Read 1 Sent Done Done false
9802 3653 Acked Read 3 Sent Done Done false
9803 3654 Acked Write 6 Sent Done Done false
9804 3655 Acked Flush 0 Sent Done Done false
9805 3656 Acked Flush 0 Sent Done Done false
9806 3657 Acked Read 5 Sent Done Done false
9807 3658 Acked Read 1 Sent Done Done false
9808 3659 Acked Read 6 Sent Done Done false
9809 3660 Acked Read 10 Sent Done Done false
9810 3661 Acked Read 1 Sent Done Done false
9811 3662 Acked Read 10 Sent Done Done false
9812 3663 Acked Write 5 Sent Done Done false
9813 3664 Acked Read 6 Sent Done Done false
9814 3665 Acked Read 9 Sent Done Done false
9815 3666 Acked Write 3 Sent Done Done false
9816 3667 Acked Flush 0 Sent Done Done false
9817 3668 Acked Write 9 Sent Done Done false
9818 3669 Acked Read 4 Sent Done Done false
9819 3670 Acked Write 2 Sent Done Done false
9820 3671 Acked Read 2 Sent Done Done false
9821 3672 Acked Read 2 Sent Done Done false
9822 3673 Acked Read 5 Sent Done Done false
9823 3674 Acked Read 2 Sent Done Done false
9824 3675 Acked Flush 0 Sent Done Done false
9825 3676 Acked Write 1 Sent Done Done false
9826 3677 Acked Read 8 Sent Done Done false
9827 3678 Acked Read 2 Sent Done Done false
9828 3679 Acked Write 3 Sent Done Done false
9829 3680 Acked Read 9 Sent Done Done false
9830 3681 Acked Read 5 Sent Done Done false
9831 3682 Acked Read 8 Sent Done Done false
9832 3683 Acked Write 10 Sent Done Done false
9833 3684 Acked Read 3 Sent Done Done false
9834 3685 Acked Read 9 Sent Done Done false
9835 3686 Acked Read 1 Sent Done Done false
9836 3687 Acked Read 5 Sent Done Done false
9837 3688 Acked Write 8 Sent Done Done false
9838 3689 Acked Write 10 Sent Done Done false
9839 3690 Acked Flush 0 Sent Done Done false
9840 3691 Acked Write 2 Sent Done Done false
9841 3692 Acked Read 10 Sent Done Done false
9842 3693 Acked Write 10 Sent Done Done false
9843 3694 Acked Write 2 Sent Done Done false
9844 3695 Acked Write 1 Sent Done Done false
9845 3696 Acked Write 6 Sent Done Done false
9846 3697 Acked Read 2 Sent Done Done false
9847 3698 Acked Write 8 Sent Done Done false
9848 3699 Acked Read 8 Sent Done Done false
9849 3700 Acked Write 4 Sent Done Done false
9850 3701 Acked Flush 0 Sent Done Done false
9851 3702 Acked Read 10 Sent Done Done false
9852 3703 Acked Write 8 Sent Done Done false
9853 3704 Acked Write 1 Sent Done Done false
9854 3705 Acked Write 9 Sent Done Done false
9855 3706 Acked Write 4 Sent Done Done false
9856 3707 Acked Write 1 Sent Done Done false
9857 3708 Acked Read 4 Sent Done Done false
9858 3709 Acked Read 8 Sent Done Done false
9859 3710 Acked Read 3 Sent Done Done false
9860 3711 Acked Read 1 Sent Done Done false
9861 3712 Acked Read 5 Sent Done Done false
9862 3713 Acked Read 10 Sent Done Done false
9863 3714 Acked Write 5 Sent Done Done false
9864 3715 Acked Read 8 Sent Done Done false
9865 STATES DS:0 DS:1 DS:2 TOTAL
9866 Sent 301 0 0 301
9867 Done 0 301 301 602
9868 Skipped 0 0 0 0
9869 Error 0 0 0 0
9870 Last Flush: 3414 3701 3701
9871 Downstairs last five completed: 3414 3413 3412 3411 3410
9872 Upstairs last five completed: 3715 3714 3713 3712 3711
9873 CLIENT: Up:0 ds:301 act:2
9874 [8810] got start action
9875 [8810] I am going to restart
9876 Starting downstairs at port 8810
9877 Make output file at "/tmp/dsc/downstairs-8810.txt"
9878 [8810][0] reports Starting
9879 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1211)
9880 [8810][0] reports Running
9881 ----------------------------------------------------------------
9882 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9883 States: Offline Active Active
9884 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9885 3415 Acked Read 9 Sent Done Done false
9886 3416 Acked Read 4 Sent Done Done false
9887 3417 Acked Read 9 Sent Done Done false
9888 3418 Acked Read 5 Sent Done Done false
9889 3419 Acked Flush 0 Sent Done Done false
9890 3420 Acked Flush 0 Sent Done Done false
9891 3421 Acked Write 7 Sent Done Done false
9892 3422 Acked Flush 0 Sent Done Done false
9893 3423 Acked Write 7 Sent Done Done false
9894 3424 Acked Read 7 Sent Done Done false
9895 3425 Acked Read 8 Sent Done Done false
9896 3426 Acked Write 2 Sent Done Done false
9897 3427 Acked Read 5 Sent Done Done false
9898 3428 Acked Read 10 Sent Done Done false
9899 3429 Acked Read 2 Sent Done Done false
9900 3430 Acked Read 3 Sent Done Done false
9901 3431 Acked Write 10 Sent Done Done false
9902 3432 Acked Read 5 Sent Done Done false
9903 3433 Acked Write 10 Sent Done Done false
9904 3434 Acked Read 4 Sent Done Done false
9905 3435 Acked Write 8 Sent Done Done false
9906 3436 Acked Read 2 Sent Done Done false
9907 3437 Acked Read 9 Sent Done Done false
9908 3438 Acked Read 1 Sent Done Done false
9909 3439 Acked Read 6 Sent Done Done false
9910 3440 Acked Write 2 Sent Done Done false
9911 3441 Acked Flush 0 Sent Done Done false
9912 3442 Acked Read 6 Sent Done Done false
9913 3443 Acked Read 1 Sent Done Done false
9914 3444 Acked Write 3 Sent Done Done false
9915 3445 Acked Read 7 Sent Done Done false
9916 3446 Acked Write 8 Sent Done Done false
9917 3447 Acked Write 6 Sent Done Done false
9918 3448 Acked Write 2 Sent Done Done false
9919 3449 Acked Read 4 Sent Done Done false
9920 3450 Acked Write 7 Sent Done Done false
9921 3451 Acked Read 3 Sent Done Done false
9922 3452 Acked Read 5 Sent Done Done false
9923 3453 Acked Read 3 Sent Done Done false
9924 3454 Acked Flush 0 Sent Done Done false
9925 3455 Acked Read 8 Sent Done Done false
9926 3456 Acked Read 4 Sent Done Done false
9927 3457 Acked Write 4 Sent Done Done false
9928 3458 Acked Read 3 Sent Done Done false
9929 3459 Acked Read 3 Sent Done Done false
9930 3460 Acked Write 5 Sent Done Done false
9931 3461 Acked Write 9 Sent Done Done false
9932 3462 Acked Read 2 Sent Done Done false
9933 3463 Acked Read 5 Sent Done Done false
9934 3464 Acked Write 2 Sent Done Done false
9935 3465 Acked Write 8 Sent Done Done false
9936 3466 Acked Flush 0 Sent Done Done false
9937 3467 Acked Flush 0 Sent Done Done false
9938 3468 Acked Write 6 Sent Done Done false
9939 3469 Acked Write 5 Sent Done Done false
9940 3470 Acked Read 5 Sent Done Done false
9941 3471 Acked Write 9 Sent Done Done false
9942 3472 Acked Write 7 Sent Done Done false
9943 3473 Acked Write 4 Sent Done Done false
9944 3474 Acked Write 7 Sent Done Done false
9945 3475 Acked Read 8 Sent Done Done false
9946 3476 Acked Write 8 Sent Done Done false
9947 3477 Acked Write 4 Sent Done Done false
9948 3478 Acked Read 1 Sent Done Done false
9949 3479 Acked Write 2 Sent Done Done false
9950 3480 Acked Flush 0 Sent Done Done false
9951 3481 Acked Read 1 Sent Done Done false
9952 3482 Acked Write 4 Sent Done Done false
9953 3483 Acked Read 4 Sent Done Done false
9954 3484 Acked Read 1 Sent Done Done false
9955 3485 Acked Write 10 Sent Done Done false
9956 3486 Acked Read 7 Sent Done Done false
9957 3487 Acked Write 4 Sent Done Done false
9958 3488 Acked Read 9 Sent Done Done false
9959 3489 Acked Write 1 Sent Done Done false
9960 3490 Acked Write 2 Sent Done Done false
9961 3491 Acked Write 4 Sent Done Done false
9962 3492 Acked Write 4 Sent Done Done false
9963 3493 Acked Write 2 Sent Done Done false
9964 3494 Acked Read 9 Sent Done Done false
9965 3495 Acked Read 9 Sent Done Done false
9966 3496 Acked Read 1 Sent Done Done false
9967 3497 Acked Flush 0 Sent Done Done false
9968 3498 Acked Read 1 Sent Done Done false
9969 3499 Acked Write 6 Sent Done Done false
9970 3500 Acked Write 1 Sent Done Done false
9971 3501 Acked Write 6 Sent Done Done false
9972 3502 Acked Read 8 Sent Done Done false
9973 3503 Acked Write 6 Sent Done Done false
9974 3504 Acked Write 7 Sent Done Done false
9975 3505 Acked Write 3 Sent Done Done false
9976 3506 Acked Write 1 Sent Done Done false
9977 3507 Acked Read 7 Sent Done Done false
9978 3508 Acked Read 4 Sent Done Done false
9979 3509 Acked Read 6 Sent Done Done false
9980 3510 Acked Read 8 Sent Done Done false
9981 3511 Acked Write 7 Sent Done Done false
9982 3512 Acked Read 4 Sent Done Done false
9983 3513 Acked Read 5 Sent Done Done false
9984 3514 Acked Write 1 Sent Done Done false
9985 3515 Acked Write 6 Sent Done Done false
9986 3516 Acked Read 4 Sent Done Done false
9987 3517 Acked Read 3 Sent Done Done false
9988 3518 Acked Read 1 Sent Done Done false
9989 3519 Acked Write 8 Sent Done Done false
9990 3520 Acked Write 7 Sent Done Done false
9991 3521 Acked Read 5 Sent Done Done false
9992 3522 Acked Read 2 Sent Done Done false
9993 3523 Acked Read 3 Sent Done Done false
9994 3524 Acked Write 8 Sent Done Done false
9995 3525 Acked Read 1 Sent Done Done false
9996 3526 Acked Write 2 Sent Done Done false
9997 3527 Acked Read 8 Sent Done Done false
9998 3528 Acked Read 2 Sent Done Done false
9999 3529 Acked Read 6 Sent Done Done false
10000 3530 Acked Read 5 Sent Done Done false
10001 3531 Acked Read 6 Sent Done Done false
10002 3532 Acked Write 8 Sent Done Done false
10003 3533 Acked Read 4 Sent Done Done false
10004 3534 Acked Flush 0 Sent Done Done false
10005 3535 Acked Write 7 Sent Done Done false
10006 3536 Acked Read 4 Sent Done Done false
10007 3537 Acked Read 8 Sent Done Done false
10008 3538 Acked Read 5 Sent Done Done false
10009 3539 Acked Flush 0 Sent Done Done false
10010 3540 Acked Read 7 Sent Done Done false
10011 3541 Acked Read 6 Sent Done Done false
10012 3542 Acked Read 3 Sent Done Done false
10013 3543 Acked Write 2 Sent Done Done false
10014 3544 Acked Read 2 Sent Done Done false
10015 3545 Acked Read 10 Sent Done Done false
10016 3546 Acked Write 10 Sent Done Done false
10017 3547 Acked Read 10 Sent Done Done false
10018 3548 Acked Write 4 Sent Done Done false
10019 3549 Acked Read 10 Sent Done Done false
10020 3550 Acked Read 6 Sent Done Done false
10021 3551 Acked Write 10 Sent Done Done false
10022 3552 Acked Read 1 Sent Done Done false
10023 3553 Acked Read 8 Sent Done Done false
10024 3554 Acked Read 10 Sent Done Done false
10025 3555 Acked Write 7 Sent Done Done false
10026 3556 Acked Read 6 Sent Done Done false
10027 3557 Acked Read 1 Sent Done Done false
10028 3558 Acked Write 10 Sent Done Done false
10029 3559 Acked Read 8 Sent Done Done false
10030 3560 Acked Flush 0 Sent Done Done false
10031 3561 Acked Read 6 Sent Done Done false
10032 3562 Acked Read 3 Sent Done Done false
10033 3563 Acked Write 9 Sent Done Done false
10034 3564 Acked Write 1 Sent Done Done false
10035 3565 Acked Write 2 Sent Done Done false
10036 3566 Acked Flush 0 Sent Done Done false
10037 3567 Acked Read 7 Sent Done Done false
10038 3568 Acked Read 2 Sent Done Done false
10039 3569 Acked Read 7 Sent Done Done false
10040 3570 Acked Read 1 Sent Done Done false
10041 3571 Acked Write 7 Sent Done Done false
10042 3572 Acked Read 8 Sent Done Done false
10043 3573 Acked Write 4 Sent Done Done false
10044 3574 Acked Read 9 Sent Done Done false
10045 3575 Acked Write 6 Sent Done Done false
10046 3576 Acked Read 4 Sent Done Done false
10047 3577 Acked Read 4 Sent Done Done false
10048 3578 Acked Flush 0 Sent Done Done false
10049 3579 Acked Write 10 Sent Done Done false
10050 3580 Acked Read 1 Sent Done Done false
10051 3581 Acked Read 10 Sent Done Done false
10052 3582 Acked Read 9 Sent Done Done false
10053 3583 Acked Flush 0 Sent Done Done false
10054 3584 Acked Write 2 Sent Done Done false
10055 3585 Acked Read 8 Sent Done Done false
10056 3586 Acked Read 3 Sent Done Done false
10057 3587 Acked Write 3 Sent Done Done false
10058 3588 Acked Flush 0 Sent Done Done false
10059 3589 Acked Read 9 Sent Done Done false
10060 3590 Acked Write 7 Sent Done Done false
10061 3591 Acked Read 10 Sent Done Done false
10062 3592 Acked Read 6 Sent Done Done false
10063 3593 Acked Read 3 Sent Done Done false
10064 3594 Acked Read 5 Sent Done Done false
10065 3595 Acked Read 8 Sent Done Done false
10066 3596 Acked Read 7 Sent Done Done false
10067 3597 Acked Write 3 Sent Done Done false
10068 3598 Acked Write 3 Sent Done Done false
10069 3599 Acked Read 4 Sent Done Done false
10070 3600 Acked Write 1 Sent Done Done false
10071 3601 Acked Write 2 Sent Done Done false
10072 3602 Acked Write 7 Sent Done Done false
10073 3603 Acked Read 8 Sent Done Done false
10074 3604 Acked Write 9 Sent Done Done false
10075 3605 Acked Read 4 Sent Done Done false
10076 3606 Acked Read 4 Sent Done Done false
10077 3607 Acked Read 1 Sent Done Done false
10078 3608 Acked Read 4 Sent Done Done false
10079 3609 Acked Write 6 Sent Done Done false
10080 3610 Acked Flush 0 Sent Done Done false
10081 3611 Acked Read 8 Sent Done Done false
10082 3612 Acked Write 9 Sent Done Done false
10083 3613 Acked Read 4 Sent Done Done false
10084 3614 Acked Flush 0 Sent Done Done false
10085 3615 Acked Write 9 Sent Done Done false
10086 3616 Acked Read 2 Sent Done Done false
10087 3617 Acked Write 1 Sent Done Done false
10088 3618 Acked Read 3 Sent Done Done false
10089 3619 Acked Flush 0 Sent Done Done false
10090 3620 Acked Write 10 Sent Done Done false
10091 3621 Acked Read 1 Sent Done Done false
10092 3622 Acked Read 7 Sent Done Done false
10093 3623 Acked Read 10 Sent Done Done false
10094 3624 Acked Flush 0 Sent Done Done false
10095 3625 Acked Write 9 Sent Done Done false
10096 3626 Acked Write 7 Sent Done Done false
10097 3627 Acked Read 2 Sent Done Done false
10098 3628 Acked Read 6 Sent Done Done false
10099 3629 Acked Read 10 Sent Done Done false
10100 3630 Acked Read 6 Sent Done Done false
10101 3631 Acked Flush 0 Sent Done Done false
10102 3632 Acked Read 7 Sent Done Done false
10103 3633 Acked Write 5 Sent Done Done false
10104 3634 Acked Read 8 Sent Done Done false
10105 3635 Acked Write 3 Sent Done Done false
10106 3636 Acked Read 10 Sent Done Done false
10107 3637 Acked Read 6 Sent Done Done false
10108 3638 Acked Read 6 Sent Done Done false
10109 3639 Acked Flush 0 Sent Done Done false
10110 3640 Acked Read 5 Sent Done Done false
10111 3641 Acked Write 5 Sent Done Done false
10112 3642 Acked Write 5 Sent Done Done false
10113 3643 Acked Flush 0 Sent Done Done false
10114 3644 Acked Read 1 Sent Done Done false
10115 3645 Acked Read 8 Sent Done Done false
10116 3646 Acked Read 1 Sent Done Done false
10117 3647 Acked Flush 0 Sent Done Done false
10118 3648 Acked Flush 0 Sent Done Done false
10119 3649 Acked Read 3 Sent Done Done false
10120 3650 Acked Write 8 Sent Done Done false
10121 3651 Acked Write 1 Sent Done Done false
10122 3652 Acked Read 1 Sent Done Done false
10123 3653 Acked Read 3 Sent Done Done false
10124 3654 Acked Write 6 Sent Done Done false
10125 3655 Acked Flush 0 Sent Done Done false
10126 3656 Acked Flush 0 Sent Done Done false
10127 3657 Acked Read 5 Sent Done Done false
10128 3658 Acked Read 1 Sent Done Done false
10129 3659 Acked Read 6 Sent Done Done false
10130 3660 Acked Read 10 Sent Done Done false
10131 3661 Acked Read 1 Sent Done Done false
10132 3662 Acked Read 10 Sent Done Done false
10133 3663 Acked Write 5 Sent Done Done false
10134 3664 Acked Read 6 Sent Done Done false
10135 3665 Acked Read 9 Sent Done Done false
10136 3666 Acked Write 3 Sent Done Done false
10137 3667 Acked Flush 0 Sent Done Done false
10138 3668 Acked Write 9 Sent Done Done false
10139 3669 Acked Read 4 Sent Done Done false
10140 3670 Acked Write 2 Sent Done Done false
10141 3671 Acked Read 2 Sent Done Done false
10142 3672 Acked Read 2 Sent Done Done false
10143 3673 Acked Read 5 Sent Done Done false
10144 3674 Acked Read 2 Sent Done Done false
10145 3675 Acked Flush 0 Sent Done Done false
10146 3676 Acked Write 1 Sent Done Done false
10147 3677 Acked Read 8 Sent Done Done false
10148 3678 Acked Read 2 Sent Done Done false
10149 3679 Acked Write 3 Sent Done Done false
10150 3680 Acked Read 9 Sent Done Done false
10151 3681 Acked Read 5 Sent Done Done false
10152 3682 Acked Read 8 Sent Done Done false
10153 3683 Acked Write 10 Sent Done Done false
10154 3684 Acked Read 3 Sent Done Done false
10155 3685 Acked Read 9 Sent Done Done false
10156 3686 Acked Read 1 Sent Done Done false
10157 3687 Acked Read 5 Sent Done Done false
10158 3688 Acked Write 8 Sent Done Done false
10159 3689 Acked Write 10 Sent Done Done false
10160 3690 Acked Flush 0 Sent Done Done false
10161 3691 Acked Write 2 Sent Done Done false
10162 3692 Acked Read 10 Sent Done Done false
10163 3693 Acked Write 10 Sent Done Done false
10164 3694 Acked Write 2 Sent Done Done false
10165 3695 Acked Write 1 Sent Done Done false
10166 3696 Acked Write 6 Sent Done Done false
10167 3697 Acked Read 2 Sent Done Done false
10168 3698 Acked Write 8 Sent Done Done false
10169 3699 Acked Read 8 Sent Done Done false
10170 3700 Acked Write 4 Sent Done Done false
10171 3701 Acked Flush 0 Sent Done Done false
10172 3702 Acked Read 10 Sent Done Done false
10173 3703 Acked Write 8 Sent Done Done false
10174 3704 Acked Write 1 Sent Done Done false
10175 3705 Acked Write 9 Sent Done Done false
10176 3706 Acked Write 4 Sent Done Done false
10177 3707 Acked Write 1 Sent Done Done false
10178 3708 Acked Read 4 Sent Done Done false
10179 3709 Acked Read 8 Sent Done Done false
10180 3710 Acked Read 3 Sent Done Done false
10181 3711 Acked Read 1 Sent Done Done false
10182 3712 Acked Read 5 Sent Done Done false
10183 3713 Acked Read 10 Sent Done Done false
10184 3714 Acked Write 5 Sent Done Done false
10185 3715 Acked Read 8 Sent Done Done false
10186 3716 Acked Flush 0 Sent Done Done false
10187 STATES DS:0 DS:1 DS:2 TOTAL
10188 Sent 302 0 0 302
10189 Done 0 302 302 604
10190 Skipped 0 0 0 0
10191 Error 0 0 0 0
10192 Last Flush: 3414 3716 3716
10193 Downstairs last five completed: 3414 3413 3412 3411 3410
10194 Upstairs last five completed: 3716 3715 3714 3713 3712
10195 CLIENT: Up:0 ds:302 act:2
101962025-01-15T05:21:06.925ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
101972025-01-15T05:21:06.926ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57484) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
101982025-01-15T05:21:06.926ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
101992025-01-15T05:21:06.927ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
102002025-01-15T05:21:06.927ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
102012025-01-15T05:21:06.927ZINFOcrucible: send last flush ID to this DS: 3414 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
102022025-01-15T05:21:06.927ZINFOcrucible: Replied this last flush ID: 3414 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
102032025-01-15T05:21:06.927ZINFOcrucible: [0] client re-new 302 jobs since flush 3414 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
102042025-01-15T05:21:06.928ZINFOcrucible: [0] Replayed 302 jobs since flush: 3414 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
10205 ----------------------------------------------------------------
10206 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10207 Downstairs last five completed: 3716 3715 3714 3713 3712
10208 Upstairs last five completed: 3716 3715 3714 3713 3712
10209 CLIENT: Up:0 ds:0 act:3
10210 Replay: All jobs finished
10211 Main task has work to do, go find it
10212 got dsc Stop(1)
10213 stop 1
10214 Jan 15 05:21:09.173 INFO [8820] Got stop action so:false kr:false
10215 request completed, latency_us: 343, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 97b387b8-daf9-4cd5-81f5-cdbb3f8d41c2, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
102162025-01-15T05:21:09.174ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
102172025-01-15T05:21:09.174ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
102182025-01-15T05:21:09.174ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
10219 Jan 15 05:21:09.174 INFO request completed, latency_us: 108, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4d1bf868-9104-4b48-a7bb-a96f57075071, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
102202025-01-15T05:21:09.174ZINFOcrucible: client task is exiting = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
10221 [8820] Exited with: ExitStatus(unix_wait_status(9))
10222 [8820][1] reports Exit
10223 Jan 15 05:21:13.176 INFO request completed, latency_us: 186, response_code: 200, uri: /state/cid/1, method: GET, req_id: 52ee0b6c-1a36-449d-b0be-f2486fab44a6, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
10224 001/300 Read block 2960 len 36864
10225 002/300 Write block 1295 len 8192 data: 3 2
10226 003/300 Write block 4579 len 16384 data: 3 3 3 2
10227 004/300 Read block 100 len 12288
10228 005/300 Write block 4251 len 8192 data: 4 2
10229 006/300 Write block 377 len 32768 data: 4 4 4 4 4 4 3 3
10230 007/300 Read block 2251 len 4096
10231 008/300 Write block 3770 len 20480 data: 3 3 3 4 3
10232 009/300 Read block 646 len 32768
10233 010/300 Flush
10234 011/300 Read block 1900 len 32768
10235 012/300 Write block 2332 len 24576 data: 2 3 3 3 2 2
10236 013/300 Read block 3077 len 12288
10237 014/300 Write block 2805 len 4096 data: 3
10238 015/300 Write block 2278 len 40960 data: 2 2 2 3 3 3 3 2 2 2
10239 016/300 Flush
10240 017/300 Read block 3133 len 32768
10241 018/300 Flush
10242 019/300 Write block 4248 len 24576 data: 3 3 3 5 3 3
10243 020/300 Write block 583 len 20480 data: 2 2 2 2 2
10244 021/300 Read block 3058 len 16384
10245 022/300 Write block 3212 len 32768 data: 2 2 2 2 2 2 2 2
10246 023/300 Flush
10247 024/300 Flush
10248 025/300 Write block 3856 len 12288 data: 5 4 4
10249 026/300 Read block 3761 len 24576
10250 027/300 Write block 4923 len 8192 data: 2 2
10251 028/300 Write block 2918 len 32768 data: 4 5 6 6 6 5 4 4
10252 029/300 Read block 17 len 32768
10253 030/300 Write block 4143 len 24576 data: 4 4 4 4 4 4
10254 031/300 Read block 4166 len 8192
10255 032/300 Read block 4286 len 40960
10256 033/300 Read block 1186 len 36864
10257 034/300 Read block 4663 len 20480
10258 035/300 Read block 1341 len 12288
10259 036/300 Write block 2491 len 12288 data: 5 4 3
10260 037/300 Read block 1324 len 24576
10261 038/300 Write block 879 len 8192 data: 4 4
10262 039/300 Write block 984 len 12288 data: 2 2 2
10263 040/300 Write block 4142 len 24576 data: 3 5 5 5 5 5
10264 041/300 Write block 1774 len 32768 data: 3 3 3 3 3 2 2 2
10265 042/300 Write block 4016 len 36864 data: 3 3 4 5 6 6 6 5 3
10266 043/300 Read block 547 len 20480
10267 044/300 Write block 4320 len 16384 data: 2 2 2 2
10268 045/300 Read block 554 len 36864
10269 046/300 Write block 4035 len 24576 data: 2 2 2 2 2 2
10270 047/300 Read block 775 len 16384
10271 048/300 Read block 2602 len 12288
10272 049/300 Read block 2580 len 20480
10273 050/300 Write block 3238 len 8192 data: 4 3
10274 051/300 Read block 1172 len 24576
10275 052/300 Read block 4618 len 16384
10276 053/300 Read block 2956 len 28672
10277 054/300 Read block 9 len 24576
10278 055/300 Write block 4644 len 8192 data: 2 2
10279 056/300 Read block 4398 len 12288
10280 057/300 Read block 4435 len 28672
10281 058/300 Read block 2766 len 16384
10282 059/300 Read block 3688 len 4096
10283 060/300 Read block 4071 len 4096
10284 061/300 Write block 1918 len 16384 data: 3 3 4 4
10285 062/300 Read block 2571 len 8192
10286 063/300 Read block 3662 len 16384
10287 064/300 Write block 1466 len 24576 data: 3 3 3 3 3 3
10288 065/300 Read block 4049 len 16384
10289 066/300 Flush
10290 067/300 Read block 4333 len 36864
10291 068/300 Read block 4799 len 16384
10292 069/300 Write block 2878 len 32768 data: 4 4 4 4 4 4 5 5
10293 070/300 Read block 1065 len 12288
10294 071/300 Read block 1316 len 24576
10295 072/300 Read block 3912 len 8192
10296 073/300 Write block 1757 len 28672 data: 4 3 3 3 4 4 4
10297 074/300 Flush
10298 075/300 Read block 3034 len 16384
10299 076/300 Read block 1995 len 32768
10300 077/300 Read block 3420 len 40960
10301 078/300 Write block 634 len 36864 data: 4 3 4 4 3 3 3 3 3
10302 079/300 Read block 836 len 8192
10303 080/300 Read block 787 len 28672
10304 081/300 Read block 1413 len 8192
10305 082/300 Read block 627 len 8192
10306 083/300 Write block 3872 len 4096 data: 6
10307 084/300 Read block 498 len 12288
10308 085/300 Write block 924 len 40960 data: 2 2 2 2 2 3 3 3 3 3
10309 086/300 Write block 4349 len 20480 data: 4 4 4 4 3
10310 087/300 Write block 3027 len 12288 data: 3 3 2
10311 088/300 Write block 3462 len 24576 data: 3 3 3 3 3 3
10312 089/300 Read block 4253 len 16384
10313 090/300 Read block 3981 len 16384
10314 091/300 Read block 1329 len 28672
10315 092/300 Read block 536 len 8192
10316 093/300 Read block 4779 len 8192
10317 094/300 Read block 4467 len 8192
10318 095/300 Read block 2290 len 12288
10319 096/300 Read block 1819 len 4096
10320 097/300 Write block 3975 len 24576 data: 4 5 3 2 2 2
10321 098/300 Read block 2936 len 36864
10322 099/300 Write block 2880 len 12288 data: 5 5 5
10323 100/300 Read block 4270 len 28672
10324 101/300 Write block 2414 len 8192 data: 3 2
10325 102/300 Write block 3592 len 12288 data: 3 3 2
10326 103/300 Read block 290 len 20480
10327 104/300 Write block 1868 len 16384 data: 4 4 5 4
10328 105/300 Read block 2428 len 32768
10329 106/300 Write block 3008 len 8192 data: 2 2
10330 107/300 Read block 2433 len 24576
10331 108/300 Write block 3288 len 40960 data: 3 2 2 2 2 3 3 3 4 4
10332 109/300 Write block 3950 len 12288 data: 5 4 2
10333 110/300 Write block 4892 len 28672 data: 3 4 4 4 5 6 7
10334 111/300 Write block 961 len 20480 data: 3 3 3 3 3
10335 112/300 Read block 1732 len 24576
10336 113/300 Read block 4775 len 40960
10337 114/300 Read block 4596 len 28672
10338 115/300 Flush
10339 116/300 Write block 3002 len 36864 data: 3 3 3 2 2 2 3 3 2
10340 117/300 Read block 4753 len 12288
10341 118/300 Read block 3785 len 36864
10342 119/300 Write block 2858 len 12288 data: 3 3 3
10343 120/300 Read block 4855 len 16384
10344 121/300 Read block 3125 len 32768
10345 122/300 Read block 4822 len 28672
10346 123/300 Write block 1300 len 28672 data: 2 3 2 2 3 4 4
10347 124/300 Write block 2411 len 40960 data: 4 4 4 4 3 2 2 2 2 2
10348 125/300 Write block 1543 len 20480 data: 3 3 4 3 3
10349 126/300 Write block 1154 len 16384 data: 4 3 2 2
10350 127/300 Write block 1940 len 40960 data: 2 2 2 2 2 3 2 3 3 2
10351 128/300 Write block 4547 len 4096 data: 4
10352 129/300 Write block 1612 len 28672 data: 3 2 2 2 2 2 2
10353 130/300 Read block 149 len 36864
10354 131/300 Read block 3563 len 24576
10355 132/300 Read block 316 len 36864
10356 133/300 Read block 161 len 4096
10357 134/300 Write block 3766 len 20480 data: 5 4 4 2 4
10358 135/300 Read block 3455 len 40960
10359 136/300 Read block 1284 len 8192
10360 137/300 Read block 278 len 36864
10361 138/300 Read block 4919 len 20480
10362 139/300 Write block 4205 len 20480 data: 2 2 2 2 3
10363 140/300 Flush
10364 141/300 Read block 3516 len 24576
10365 142/300 Read block 1345 len 8192
10366 143/300 Flush
10367 144/300 Read block 2564 len 12288
10368 145/300 Read block 2303 len 4096
10369 146/300 Write block 4580 len 40960 data: 4 4 3 2 2 2 2 2 2 2
10370 147/300 Read block 40 len 24576
10371 148/300 Read block 4142 len 36864
10372 149/300 Write block 1513 len 12288 data: 3 3 3
10373 150/300 Write block 1491 len 32768 data: 3 3 3 2 3 3 2 2
10374 151/300 Write block 4926 len 16384 data: 2 2 3 3
10375 152/300 Read block 1647 len 12288
10376 153/300 Write block 2934 len 4096 data: 2
10377 154/300 Write block 4725 len 32768 data: 3 3 3 4 5 4 4 6
10378 155/300 Flush
10379 156/300 Read block 2080 len 40960
10380 157/300 Read block 2741 len 20480
10381 158/300 Read block 1113 len 20480
10382 159/300 Write block 3801 len 8192 data: 2 2
10383 160/300 Read block 3960 len 4096
10384 161/300 Write block 1783 len 36864 data: 4 4 4 6 6 6 5 5 5
10385 162/300 Read block 1463 len 32768
10386 163/300 Read block 4709 len 36864
10387 164/300 Write block 1677 len 24576 data: 2 3 3 3 4 5
10388 165/300 Read block 3631 len 20480
10389 166/300 Read block 3144 len 32768
10390 167/300 Read block 4112 len 32768
10391 168/300 Read block 1143 len 16384
10392 169/300 Read block 2924 len 40960
10393 170/300 Read block 1098 len 12288
10394 171/300 Write block 4380 len 36864 data: 4 4 4 3 3 3 2 2 2
10395 172/300 Read block 840 len 36864
10396 173/300 Read block 1679 len 4096
10397 174/300 Write block 1035 len 8192 data: 3 4
10398 175/300 Read block 3748 len 8192
10399 176/300 Write block 4425 len 40960 data: 6 6 6 6 5 3 3 4 4 4
10400 177/300 Flush
10401 178/300 Read block 643 len 32768
10402 179/300 Write block 3029 len 28672 data: 3 3 3 3 3 3 4
10403 180/300 Flush
10404 181/300 Read block 3394 len 16384
10405 182/300 Write block 1507 len 8192 data: 2 2
10406 183/300 Read block 2671 len 8192
10407 184/300 Flush
10408 185/300 Read block 2710 len 40960
10409 186/300 Read block 998 len 36864
10410 187/300 Read block 3158 len 24576
10411 188/300 Write block 2327 len 32768 data: 2 2 2 2 2 3 4 4
10412 189/300 Read block 3433 len 16384
10413 190/300 Read block 4748 len 32768
10414 191/300 Write block 690 len 36864 data: 4 5 4 3 3 3 4 4 4
10415 192/300 Read block 11 len 28672
10416 193/300 Write block 2464 len 12288 data: 3 3 3
10417 194/300 Read block 4648 len 20480
10418 195/300 Flush
10419 196/300 Write block 325 len 40960 data: 5 6 4 4 2 2 2 2 2 2
10420 197/300 Read block 2629 len 16384
10421 198/300 Read block 1918 len 20480
10422 199/300 Read block 2687 len 20480
10423 200/300 Read block 3810 len 8192
10424 201/300 Write block 75 len 32768 data: 5 5 5 5 5 5 2 2
10425 202/300 Read block 4049 len 24576
10426 203/300 Read block 4189 len 8192
10427 204/300 Read block 1787 len 8192
10428 205/300 Read block 316 len 20480
10429 206/300 Write block 1456 len 40960 data: 3 2 2 3 3 3 3 3 3 3
10430 207/300 Read block 2318 len 32768
10431 208/300 Write block 4700 len 32768 data: 4 3 2 2 3 2 3 2
10432 209/300 Read block 2267 len 16384
10433 210/300 Read block 2939 len 4096
10434 211/300 Write block 2909 len 36864 data: 3 3 3 2 2 2 2 3 4
10435 212/300 Read block 4760 len 32768
10436 213/300 Write block 4444 len 40960 data: 3 3 2 3 4 4 5 4 4 4
10437 214/300 Write block 4601 len 40960 data: 3 2 2 2 2 2 3 3 3 4
10438 215/300 Write block 1586 len 24576 data: 6 6 5 4 3 3
10439 216/300 Read block 4793 len 4096
10440 217/300 Read block 2841 len 12288
10441 218/300 Read block 1483 len 16384
10442 219/300 Read block 4499 len 40960
10443 220/300 Write block 3558 len 12288 data: 3 3 3
10444 221/300 Read block 1523 len 40960
10445 222/300 Read block 4260 len 4096
10446 223/300 Write block 2850 len 4096 data: 3
10447 224/300 Write block 1826 len 40960 data: 4 4 5 5 4 4 3 2 2 2
10448 225/300 Write block 2116 len 8192 data: 4 3
10449 226/300 Read block 4229 len 4096
10450 227/300 Write block 948 len 16384 data: 2 2 2 2
10451 228/300 Read block 63 len 12288
10452 229/300 Read block 827 len 16384
10453 230/300 Read block 3298 len 4096
10454 231/300 Write block 115 len 4096 data: 3
10455 232/300 Read block 1212 len 4096
10456 233/300 Read block 1610 len 8192
10457 234/300 Read block 2616 len 8192
10458 235/300 Read block 4422 len 4096
10459 236/300 Read block 1857 len 24576
10460 237/300 Write block 2744 len 16384 data: 4 3 2 2
10461 238/300 Read block 380 len 12288
10462 239/300 Read block 4662 len 32768
10463 240/300 Read block 328 len 36864
10464 241/300 Write block 3990 len 12288 data: 2 3 3
10465 242/300 Write block 4455 len 20480 data: 4 4 4 3 3
10466 243/300 Flush
10467 244/300 Read block 116 len 40960
10468 245/300 Write block 407 len 36864 data: 2 3 3 3 3 3 3 2 2
10469 246/300 Read block 3839 len 24576
10470 247/300 Read block 1985 len 36864
10471 248/300 Read block 3967 len 32768
10472 249/300 Read block 4288 len 28672
10473 250/300 Write block 3538 len 28672 data: 2 2 2 2 2 2 3
10474 251/300 Read block 1873 len 24576
10475 252/300 Write block 455 len 32768 data: 3 3 2 2 2 2 2 2
10476 253/300 Read block 1592 len 8192
10477 254/300 Write block 4884 len 28672 data: 3 3 3 2 3 3 3
10478 255/300 Write block 2655 len 16384 data: 3 3 3 3
10479 256/300 Read block 3841 len 4096
10480 257/300 Read block 635 len 12288
10481 258/300 Read block 2665 len 16384
10482 259/300 Read block 405 len 36864
10483 260/300 Write block 369 len 36864 data: 2 2 2 2 3 3 4 4 5
10484 261/300 Write block 4057 len 12288 data: 2 2 3
10485 262/300 Read block 3293 len 20480
10486 263/300 Read block 4017 len 24576
10487 264/300 Read block 1688 len 12288
10488 265/300 Read block 677 len 28672
10489 266/300 Read block 2524 len 12288
10490 267/300 Read block 2451 len 32768
10491 268/300 Read block 209 len 28672
10492 269/300 Read block 3300 len 20480
10493 270/300 Write block 593 len 16384 data: 3 5 5 4
10494 271/300 Write block 3642 len 40960 data: 3 3 2 2 2 2 2 3 6 6
10495 272/300 Write block 3879 len 4096 data: 4
10496 273/300 Read block 2116 len 16384
10497 274/300 Write block 3529 len 36864 data: 4 4 3 4 4 3 4 4 3
10498 275/300 Write block 49 len 20480 data: 2 2 2 2 2
10499 276/300 Read block 2314 len 28672
10500 277/300 Write block 2822 len 40960 data: 5 4 4 4 4 4 3 2 4 4
10501 278/300 Write block 2830 len 12288 data: 5 5 4
10502 279/300 Write block 2108 len 8192 data: 3 5
10503 280/300 Read block 2780 len 32768
10504 281/300 Write block 445 len 24576 data: 8 9 7 6 5 3
10505 282/300 Read block 4232 len 28672
10506 283/300 Read block 1650 len 24576
10507 284/300 Read block 1542 len 20480
10508 285/300 Flush
10509 286/300 Read block 1221 len 24576
10510 287/300 Read block 2167 len 16384
10511 288/300 Read block 1156 len 20480
10512 289/300 Flush
10513 290/300 Write block 4832 len 32768 data: 2 2 4 4 4 4 4 4
10514 291/300 Write block 206 len 8192 data: 6 4
10515 292/300 Write block 2259 len 28672 data: 3 3 3 4 4 3 3
10516 293/300 Read block 3176 len 32768
10517 294/300 Read block 2885 len 40960
10518 295/300 Flush
10519 296/300 Read block 1485 len 32768
10520 297/300 Read block 853 len 4096
10521 298/300 Read block 3879 len 16384
10522 299/300 Flush
10523 300/300 Read block 921 len 40960
10524 301/300 Read block 2451 len 16384
10525 Main task has work to do, go find it
10526 got dsc Start(1)
10527 start 1
10528 Jan 15 05:21:13.419 INFO request completed, latency_us: 180, response_code: 204, uri: /start/cid/1, method: POST, req_id: 57a3ef5e-db95-4078-a35f-f91e900779f8, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
10529 Replay: started 1, returned:Ok(())
10530 ----------------------------------------------------------------
10531 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10532 States: Active Offline Active
10533 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10534 3717 Acked Read 9 Done Sent Done false
10535 3718 Acked Write 2 Done Sent Done false
10536 3719 Acked Write 4 Done Sent Done false
10537 3720 Acked Read 3 Done Sent Done false
10538 3721 Acked Write 2 Done Sent Done false
10539 3722 Acked Write 8 Done Sent Done false
10540 3723 Acked Read 1 Done Sent Done false
10541 3724 Acked Write 5 Done Sent Done false
10542 3725 Acked Read 8 Done Sent Done false
10543 3726 Acked Flush 0 Done Sent Done false
10544 3727 Acked Read 8 Done Sent Done false
10545 3728 Acked Write 6 Done Sent Done false
10546 3729 Acked Read 3 Done Sent Done false
10547 3730 Acked Write 1 Done Sent Done false
10548 3731 Acked Write 10 Done Sent Done false
10549 3732 Acked Flush 0 Done Sent Done false
10550 3733 Acked Read 8 Done Sent Done false
10551 3734 Acked Flush 0 Done Sent Done false
10552 3735 Acked Write 6 Done Sent Done false
10553 3736 Acked Write 5 Done Sent Done false
10554 3737 Acked Read 4 Done Sent Done false
10555 3738 Acked Write 8 Done Sent Done false
10556 3739 Acked Flush 0 Done Sent Done false
10557 3740 Acked Flush 0 Done Sent Done false
10558 3741 Acked Write 3 Done Sent Done false
10559 3742 Acked Read 6 Done Sent Done false
10560 3743 Acked Write 2 Done Sent Done false
10561 3744 Acked Write 8 Done Sent Done false
10562 3745 Acked Read 8 Done Sent Done false
10563 3746 Acked Write 6 Done Sent Done false
10564 3747 Acked Read 2 Done Sent Done false
10565 3748 Acked Read 10 Done Sent Done false
10566 3749 Acked Read 9 Done Sent Done false
10567 3750 Acked Read 5 Done Sent Done false
10568 3751 Acked Read 3 Done Sent Done false
10569 3752 Acked Write 3 Done Sent Done false
10570 3753 Acked Read 6 Done Sent Done false
10571 3754 Acked Write 2 Done Sent Done false
10572 3755 Acked Write 3 Done Sent Done false
10573 3756 Acked Write 6 Done Sent Done false
10574 3757 Acked Write 8 Done Sent Done false
10575 3758 Acked Write 9 Done Sent Done false
10576 3759 Acked Read 5 Done Sent Done false
10577 3760 Acked Write 4 Done Sent Done false
10578 3761 Acked Read 9 Done Sent Done false
10579 3762 Acked Write 6 Done Sent Done false
10580 3763 Acked Read 4 Done Sent Done false
10581 3764 Acked Read 3 Done Sent Done false
10582 3765 Acked Read 5 Done Sent Done false
10583 3766 Acked Write 2 Done Sent Done false
10584 3767 Acked Read 6 Done Sent Done false
10585 3768 Acked Read 4 Done Sent Done false
10586 3769 Acked Read 7 Done Sent Done false
10587 3770 Acked Read 6 Done Sent Done false
10588 3771 Acked Write 2 Done Sent Done false
10589 3772 Acked Read 3 Done Sent Done false
10590 3773 Acked Read 7 Done Sent Done false
10591 3774 Acked Read 4 Done Sent Done false
10592 3775 Acked Read 1 Done Sent Done false
10593 3776 Acked Read 1 Done Sent Done false
10594 3777 Acked Write 4 Done Sent Done false
10595 3778 Acked Read 2 Done Sent Done false
10596 3779 Acked Read 4 Done Sent Done false
10597 3780 Acked Write 6 Done Sent Done false
10598 3781 Acked Read 4 Done Sent Done false
10599 3782 Acked Flush 0 Done Sent Done false
10600 3783 Acked Read 9 Done Sent Done false
10601 3784 Acked Read 4 Done Sent Done false
10602 3785 Acked Write 8 Done Sent Done false
10603 3786 Acked Read 3 Done Sent Done false
10604 3787 Acked Read 6 Done Sent Done false
10605 3788 Acked Read 2 Done Sent Done false
10606 3789 Acked Write 7 Done Sent Done false
10607 3790 Acked Flush 0 Done Sent Done false
10608 3791 Acked Read 4 Done Sent Done false
10609 3792 Acked Read 8 Done Sent Done false
10610 3793 Acked Read 10 Done Sent Done false
10611 3794 Acked Write 9 Done Sent Done false
10612 3795 Acked Read 2 Done Sent Done false
10613 3796 Acked Read 7 Done Sent Done false
10614 3797 Acked Read 2 Done Sent Done false
10615 3798 Acked Read 2 Done Sent Done false
10616 3799 Acked Write 1 Done Sent Done false
10617 3800 Acked Read 3 Done Sent Done false
10618 3801 Acked Write 10 Done Sent Done false
10619 3802 Acked Write 5 Done Sent Done false
10620 3803 Acked Write 3 Done Sent Done false
10621 3804 Acked Write 6 Done Sent Done false
10622 3805 Acked Read 4 Done Sent Done false
10623 3806 Acked Read 4 Done Sent Done false
10624 3807 Acked Read 7 Done Sent Done false
10625 3808 Acked Read 2 Done Sent Done false
10626 3809 Acked Read 2 Done Sent Done false
10627 3810 Acked Read 2 Done Sent Done false
10628 3811 Acked Read 3 Done Sent Done false
10629 3812 Acked Read 1 Done Sent Done false
10630 3813 Acked Write 6 Done Sent Done false
10631 3814 Acked Read 9 Done Sent Done false
10632 3815 Acked Write 3 Done Sent Done false
10633 3816 Acked Read 7 Done Sent Done false
10634 3817 Acked Write 2 Done Sent Done false
10635 3818 Acked Write 3 Done Sent Done false
10636 3819 Acked Read 5 Done Sent Done false
10637 3820 Acked Write 4 Done Sent Done false
10638 3821 Acked Read 8 Done Sent Done false
10639 3822 Acked Write 2 Done Sent Done false
10640 3823 Acked Read 6 Done Sent Done false
10641 3824 Acked Write 10 Done Sent Done false
10642 3825 Acked Write 3 Done Sent Done false
10643 3826 Acked Write 7 Done Sent Done false
10644 3827 Acked Write 5 Done Sent Done false
10645 3828 Acked Read 6 Done Sent Done false
10646 3829 Acked Read 10 Done Sent Done false
10647 3830 Acked Read 7 Done Sent Done false
10648 3831 Acked Flush 0 Done Sent Done false
10649 3832 Acked Write 9 Done Sent Done false
10650 3833 Acked Read 3 Done Sent Done false
10651 3834 Acked Read 9 Done Sent Done false
10652 3835 Acked Write 3 Done Sent Done false
10653 3836 Acked Read 4 Done Sent Done false
10654 3837 Acked Read 8 Done Sent Done false
10655 3838 Acked Read 7 Done Sent Done false
10656 3839 Acked Write 7 Done Sent Done false
10657 3840 Acked Write 10 Done Sent Done false
10658 3841 Acked Write 5 Done Sent Done false
10659 3842 Acked Write 4 Done Sent Done false
10660 3843 Acked Write 10 Done Sent Done false
10661 3844 Acked Write 1 Done Sent Done false
10662 3845 Acked Write 7 Done Sent Done false
10663 3846 Acked Read 9 Done Sent Done false
10664 3847 Acked Read 6 Done Sent Done false
10665 3848 Acked Read 9 Done Sent Done false
10666 3849 Acked Read 1 Done Sent Done false
10667 3850 Acked Write 5 Done Sent Done false
10668 3851 Acked Read 10 Done Sent Done false
10669 3852 Acked Read 2 Done Sent Done false
10670 3853 Acked Read 9 Done Sent Done false
10671 3854 Acked Read 5 Done Sent Done false
10672 3855 Acked Write 5 Done Sent Done false
10673 3856 Acked Flush 0 Done Sent Done false
10674 3857 Acked Read 6 Done Sent Done false
10675 3858 Acked Read 2 Done Sent Done false
10676 3859 Acked Flush 0 Done Sent Done false
10677 3860 Acked Read 3 Done Sent Done false
10678 3861 Acked Read 1 Done Sent Done false
10679 3862 Acked Write 10 Done Sent Done false
10680 3863 Acked Read 6 Done Sent Done false
10681 3864 Acked Read 9 Done Sent Done false
10682 3865 Acked Write 3 Done Sent Done false
10683 3866 Acked Write 8 Done Sent Done false
10684 3867 Acked Write 4 Done Sent Done false
10685 3868 Acked Read 3 Done Sent Done false
10686 3869 Acked Write 1 Done Sent Done false
10687 3870 Acked Write 8 Done Sent Done false
10688 3871 Acked Flush 0 Done Sent Done false
10689 3872 Acked Read 10 Done Sent Done false
10690 3873 Acked Read 5 Done Sent Done false
10691 3874 Acked Read 5 Done Sent Done false
10692 3875 Acked Write 2 Done Sent Done false
10693 3876 Acked Read 1 Done Sent Done false
10694 3877 Acked Write 9 Done Sent Done false
10695 3878 Acked Read 8 Done Sent Done false
10696 3879 Acked Read 9 Done Sent Done false
10697 3880 Acked Write 6 Done Sent Done false
10698 3881 Acked Read 5 Done Sent Done false
10699 3882 Acked Read 8 Done Sent Done false
10700 3883 Acked Read 8 Done Sent Done false
10701 3884 Acked Read 4 Done Sent Done false
10702 3885 Acked Read 10 Done Sent Done false
10703 3886 Acked Read 3 Done Sent Done false
10704 3887 Acked Write 9 Done Sent Done false
10705 3888 Acked Read 9 Done Sent Done false
10706 3889 Acked Read 1 Done Sent Done false
10707 3890 Acked Write 2 Done Sent Done false
10708 3891 Acked Read 2 Done Sent Done false
10709 3892 Acked Write 10 Done Sent Done false
10710 3893 Acked Flush 0 Done Sent Done false
10711 3894 Acked Read 8 Done Sent Done false
10712 3895 Acked Write 7 Done Sent Done false
10713 3896 Acked Flush 0 Done Sent Done false
10714 3897 Acked Read 4 Done Sent Done false
10715 3898 Acked Write 2 Done Sent Done false
10716 3899 Acked Read 2 Done Sent Done false
10717 3900 Acked Flush 0 Done Sent Done false
10718 3901 Acked Read 10 Done Sent Done false
10719 3902 Acked Read 9 Done Sent Done false
10720 3903 Acked Read 6 Done Sent Done false
10721 3904 Acked Write 8 Done Sent Done false
10722 3905 Acked Read 4 Done Sent Done false
10723 3906 Acked Read 8 Done Sent Done false
10724 3907 Acked Write 9 Done Sent Done false
10725 3908 Acked Read 7 Done Sent Done false
10726 3909 Acked Write 3 Done Sent Done false
10727 3910 Acked Read 5 Done Sent Done false
10728 3911 Acked Flush 0 Done Sent Done false
10729 3912 Acked Write 10 Done Sent Done false
10730 3913 Acked Read 4 Done Sent Done false
10731 3914 Acked Read 5 Done Sent Done false
10732 3915 Acked Read 5 Done Sent Done false
10733 3916 Acked Read 2 Done Sent Done false
10734 3917 Acked Write 8 Done Sent Done false
10735 3918 Acked Read 6 Done Sent Done false
10736 3919 Acked Read 2 Done Sent Done false
10737 3920 Acked Read 2 Done Sent Done false
10738 3921 Acked Read 5 Done Sent Done false
10739 3922 Acked Write 10 Done Sent Done false
10740 3923 Acked Read 8 Done Sent Done false
10741 3924 Acked Write 8 Done Sent Done false
10742 3925 Acked Read 4 Done Sent Done false
10743 3926 Acked Read 1 Done Sent Done false
10744 3927 Acked Write 9 Done Sent Done false
10745 3928 Acked Read 8 Done Sent Done false
10746 3929 Acked Write 10 Done Sent Done false
10747 3930 Acked Write 10 Done Sent Done false
10748 3931 Acked Write 6 Done Sent Done false
10749 3932 Acked Read 1 Done Sent Done false
10750 3933 Acked Read 3 Done Sent Done false
10751 3934 Acked Read 4 Done Sent Done false
10752 3935 Acked Read 10 Done Sent Done false
10753 3936 Acked Write 3 Done Sent Done false
10754 3937 Acked Read 10 Done Sent Done false
10755 3938 Acked Read 1 Done Sent Done false
10756 3939 Acked Write 1 Done Sent Done false
10757 3940 Acked Write 10 Done Sent Done false
10758 3941 Acked Write 2 Done Sent Done false
10759 3942 Acked Read 1 Done Sent Done false
10760 3943 Acked Write 4 Done Sent Done false
10761 3944 Acked Read 3 Done Sent Done false
10762 3945 Acked Read 4 Done Sent Done false
10763 3946 Acked Read 1 Done Sent Done false
10764 3947 Acked Write 1 Done Sent Done false
10765 3948 Acked Read 1 Done Sent Done false
10766 3949 Acked Read 2 Done Sent Done false
10767 3950 Acked Read 2 Done Sent Done false
10768 3951 Acked Read 1 Done Sent Done false
10769 3952 Acked Read 6 Done Sent Done false
10770 3953 Acked Write 4 Done Sent Done false
10771 3954 Acked Read 3 Done Sent Done false
10772 3955 Acked Read 8 Done Sent Done false
10773 3956 Acked Read 9 Done Sent Done false
10774 3957 Acked Write 3 Done Sent Done false
10775 3958 Acked Write 5 Done Sent Done false
10776 3959 Acked Flush 0 Done Sent Done false
10777 3960 Acked Read 10 Done Sent Done false
10778 3961 Acked Write 9 Done Sent Done false
10779 3962 Acked Read 6 Done Sent Done false
10780 3963 Acked Read 9 Done Sent Done false
10781 3964 Acked Read 8 Done Sent Done false
10782 3965 Acked Read 7 Done Sent Done false
10783 3966 Acked Write 7 Done Sent Done false
10784 3967 Acked Read 6 Done Sent Done false
10785 3968 Acked Write 8 Done Sent Done false
10786 3969 Acked Read 2 Done Sent Done false
10787 3970 Acked Write 7 Done Sent Done false
10788 3971 Acked Write 4 Done Sent Done false
10789 3972 Acked Read 1 Done Sent Done false
10790 3973 Acked Read 3 Done Sent Done false
10791 3974 Acked Read 4 Done Sent Done false
10792 3975 Acked Read 9 Done Sent Done false
10793 3976 Acked Write 9 Done Sent Done false
10794 3977 Acked Write 3 Done Sent Done false
10795 3978 Acked Read 5 Done Sent Done false
10796 3979 Acked Read 6 Done Sent Done false
10797 3980 Acked Read 3 Done Sent Done false
10798 3981 Acked Read 7 Done Sent Done false
10799 3982 Acked Read 3 Done Sent Done false
10800 3983 Acked Read 8 Done Sent Done false
10801 3984 Acked Read 7 Done Sent Done false
10802 3985 Acked Read 5 Done Sent Done false
10803 3986 Acked Write 4 Done Sent Done false
10804 3987 Acked Write 10 Done Sent Done false
10805 3988 Acked Write 1 Done Sent Done false
10806 3989 Acked Read 4 Done Sent Done false
10807 3990 Acked Write 9 Done Sent Done false
10808 3991 Acked Write 5 Done Sent Done false
10809 3992 Acked Read 7 Done Sent Done false
10810 3993 Acked Write 10 Done Sent Done false
10811 3994 Acked Write 3 Done Sent Done false
10812 3995 Acked Write 2 Done Sent Done false
10813 3996 Acked Read 8 Done Sent Done false
10814 3997 Acked Write 6 Done Sent Done false
10815 3998 Acked Read 7 Done Sent Done false
10816 3999 Acked Read 6 Done Sent Done false
10817 4000 Acked Read 5 Done Sent Done false
10818 4001 Acked Flush 0 Done Sent Done false
10819 4002 Acked Read 6 Done Sent Done false
10820 4003 Acked Read 4 Done Sent Done false
10821 4004 Acked Read 5 Done Sent Done false
10822 4005 Acked Flush 0 Done Sent Done false
10823 4006 Acked Write 8 Done Sent Done false
10824 4007 Acked Write 2 Done Sent Done false
10825 4008 Acked Write 7 Done Sent Done false
10826 4009 Acked Read 8 Done Sent Done false
10827 4010 Acked Read 10 Done Sent Done false
10828 4011 Acked Flush 0 Done Sent Done false
10829 4012 Acked Read 8 Done Sent Done false
10830 4013 Acked Read 1 Done Sent Done false
10831 4014 Acked Read 4 Done Sent Done false
10832 4015 Acked Flush 0 Done Sent Done false
10833 4016 Acked Read 10 Done Sent Done false
10834 4017 Acked Read 4 Done Sent Done false
10835 STATES DS:0 DS:1 DS:2 TOTAL
10836 Sent 0 301 0 301
10837 Done 301 0 301 602
10838 Skipped 0 0 0 0
10839 Error 0 0 0 0
10840 Last Flush: 4015 3716 4015
10841 Downstairs last five completed: 3716 3715 3714 3713 3712
10842 Upstairs last five completed: 4017 4016 4015 4014 4013
10843 CLIENT: Up:0 ds:301 act:2
10844 [8820] got start action
10845 [8820] I am going to restart
10846 Starting downstairs at port 8820
10847 Make output file at "/tmp/dsc/downstairs-8820.txt"
10848 [8820][1] reports Starting
10849 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
10850 [8820][1] reports Running
10851 ----------------------------------------------------------------
10852 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10853 States: Active Offline Active
10854 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10855 3717 Acked Read 9 Done Sent Done false
10856 3718 Acked Write 2 Done Sent Done false
10857 3719 Acked Write 4 Done Sent Done false
10858 3720 Acked Read 3 Done Sent Done false
10859 3721 Acked Write 2 Done Sent Done false
10860 3722 Acked Write 8 Done Sent Done false
10861 3723 Acked Read 1 Done Sent Done false
10862 3724 Acked Write 5 Done Sent Done false
10863 3725 Acked Read 8 Done Sent Done false
10864 3726 Acked Flush 0 Done Sent Done false
10865 3727 Acked Read 8 Done Sent Done false
10866 3728 Acked Write 6 Done Sent Done false
10867 3729 Acked Read 3 Done Sent Done false
10868 3730 Acked Write 1 Done Sent Done false
10869 3731 Acked Write 10 Done Sent Done false
10870 3732 Acked Flush 0 Done Sent Done false
10871 3733 Acked Read 8 Done Sent Done false
10872 3734 Acked Flush 0 Done Sent Done false
10873 3735 Acked Write 6 Done Sent Done false
10874 3736 Acked Write 5 Done Sent Done false
10875 3737 Acked Read 4 Done Sent Done false
10876 3738 Acked Write 8 Done Sent Done false
10877 3739 Acked Flush 0 Done Sent Done false
10878 3740 Acked Flush 0 Done Sent Done false
10879 3741 Acked Write 3 Done Sent Done false
10880 3742 Acked Read 6 Done Sent Done false
10881 3743 Acked Write 2 Done Sent Done false
10882 3744 Acked Write 8 Done Sent Done false
10883 3745 Acked Read 8 Done Sent Done false
10884 3746 Acked Write 6 Done Sent Done false
10885 3747 Acked Read 2 Done Sent Done false
10886 3748 Acked Read 10 Done Sent Done false
10887 3749 Acked Read 9 Done Sent Done false
10888 3750 Acked Read 5 Done Sent Done false
10889 3751 Acked Read 3 Done Sent Done false
10890 3752 Acked Write 3 Done Sent Done false
10891 3753 Acked Read 6 Done Sent Done false
10892 3754 Acked Write 2 Done Sent Done false
10893 3755 Acked Write 3 Done Sent Done false
10894 3756 Acked Write 6 Done Sent Done false
10895 3757 Acked Write 8 Done Sent Done false
10896 3758 Acked Write 9 Done Sent Done false
10897 3759 Acked Read 5 Done Sent Done false
10898 3760 Acked Write 4 Done Sent Done false
10899 3761 Acked Read 9 Done Sent Done false
10900 3762 Acked Write 6 Done Sent Done false
10901 3763 Acked Read 4 Done Sent Done false
10902 3764 Acked Read 3 Done Sent Done false
10903 3765 Acked Read 5 Done Sent Done false
10904 3766 Acked Write 2 Done Sent Done false
10905 3767 Acked Read 6 Done Sent Done false
10906 3768 Acked Read 4 Done Sent Done false
10907 3769 Acked Read 7 Done Sent Done false
10908 3770 Acked Read 6 Done Sent Done false
10909 3771 Acked Write 2 Done Sent Done false
10910 3772 Acked Read 3 Done Sent Done false
10911 3773 Acked Read 7 Done Sent Done false
10912 3774 Acked Read 4 Done Sent Done false
10913 3775 Acked Read 1 Done Sent Done false
10914 3776 Acked Read 1 Done Sent Done false
10915 3777 Acked Write 4 Done Sent Done false
10916 3778 Acked Read 2 Done Sent Done false
10917 3779 Acked Read 4 Done Sent Done false
10918 3780 Acked Write 6 Done Sent Done false
10919 3781 Acked Read 4 Done Sent Done false
10920 3782 Acked Flush 0 Done Sent Done false
10921 3783 Acked Read 9 Done Sent Done false
10922 3784 Acked Read 4 Done Sent Done false
10923 3785 Acked Write 8 Done Sent Done false
10924 3786 Acked Read 3 Done Sent Done false
10925 3787 Acked Read 6 Done Sent Done false
10926 3788 Acked Read 2 Done Sent Done false
10927 3789 Acked Write 7 Done Sent Done false
10928 3790 Acked Flush 0 Done Sent Done false
10929 3791 Acked Read 4 Done Sent Done false
10930 3792 Acked Read 8 Done Sent Done false
10931 3793 Acked Read 10 Done Sent Done false
10932 3794 Acked Write 9 Done Sent Done false
10933 3795 Acked Read 2 Done Sent Done false
10934 3796 Acked Read 7 Done Sent Done false
10935 3797 Acked Read 2 Done Sent Done false
10936 3798 Acked Read 2 Done Sent Done false
10937 3799 Acked Write 1 Done Sent Done false
10938 3800 Acked Read 3 Done Sent Done false
10939 3801 Acked Write 10 Done Sent Done false
10940 3802 Acked Write 5 Done Sent Done false
10941 3803 Acked Write 3 Done Sent Done false
10942 3804 Acked Write 6 Done Sent Done false
10943 3805 Acked Read 4 Done Sent Done false
10944 3806 Acked Read 4 Done Sent Done false
10945 3807 Acked Read 7 Done Sent Done false
10946 3808 Acked Read 2 Done Sent Done false
10947 3809 Acked Read 2 Done Sent Done false
10948 3810 Acked Read 2 Done Sent Done false
10949 3811 Acked Read 3 Done Sent Done false
10950 3812 Acked Read 1 Done Sent Done false
10951 3813 Acked Write 6 Done Sent Done false
10952 3814 Acked Read 9 Done Sent Done false
10953 3815 Acked Write 3 Done Sent Done false
10954 3816 Acked Read 7 Done Sent Done false
10955 3817 Acked Write 2 Done Sent Done false
10956 3818 Acked Write 3 Done Sent Done false
10957 3819 Acked Read 5 Done Sent Done false
10958 3820 Acked Write 4 Done Sent Done false
10959 3821 Acked Read 8 Done Sent Done false
10960 3822 Acked Write 2 Done Sent Done false
10961 3823 Acked Read 6 Done Sent Done false
10962 3824 Acked Write 10 Done Sent Done false
10963 3825 Acked Write 3 Done Sent Done false
10964 3826 Acked Write 7 Done Sent Done false
10965 3827 Acked Write 5 Done Sent Done false
10966 3828 Acked Read 6 Done Sent Done false
10967 3829 Acked Read 10 Done Sent Done false
10968 3830 Acked Read 7 Done Sent Done false
10969 3831 Acked Flush 0 Done Sent Done false
10970 3832 Acked Write 9 Done Sent Done false
10971 3833 Acked Read 3 Done Sent Done false
10972 3834 Acked Read 9 Done Sent Done false
10973 3835 Acked Write 3 Done Sent Done false
10974 3836 Acked Read 4 Done Sent Done false
10975 3837 Acked Read 8 Done Sent Done false
10976 3838 Acked Read 7 Done Sent Done false
10977 3839 Acked Write 7 Done Sent Done false
10978 3840 Acked Write 10 Done Sent Done false
10979 3841 Acked Write 5 Done Sent Done false
10980 3842 Acked Write 4 Done Sent Done false
10981 3843 Acked Write 10 Done Sent Done false
10982 3844 Acked Write 1 Done Sent Done false
10983 3845 Acked Write 7 Done Sent Done false
10984 3846 Acked Read 9 Done Sent Done false
10985 3847 Acked Read 6 Done Sent Done false
10986 3848 Acked Read 9 Done Sent Done false
10987 3849 Acked Read 1 Done Sent Done false
10988 3850 Acked Write 5 Done Sent Done false
10989 3851 Acked Read 10 Done Sent Done false
10990 3852 Acked Read 2 Done Sent Done false
10991 3853 Acked Read 9 Done Sent Done false
10992 3854 Acked Read 5 Done Sent Done false
10993 3855 Acked Write 5 Done Sent Done false
10994 3856 Acked Flush 0 Done Sent Done false
10995 3857 Acked Read 6 Done Sent Done false
10996 3858 Acked Read 2 Done Sent Done false
10997 3859 Acked Flush 0 Done Sent Done false
10998 3860 Acked Read 3 Done Sent Done false
10999 3861 Acked Read 1 Done Sent Done false
11000 3862 Acked Write 10 Done Sent Done false
11001 3863 Acked Read 6 Done Sent Done false
11002 3864 Acked Read 9 Done Sent Done false
11003 3865 Acked Write 3 Done Sent Done false
11004 3866 Acked Write 8 Done Sent Done false
11005 3867 Acked Write 4 Done Sent Done false
11006 3868 Acked Read 3 Done Sent Done false
11007 3869 Acked Write 1 Done Sent Done false
11008 3870 Acked Write 8 Done Sent Done false
11009 3871 Acked Flush 0 Done Sent Done false
11010 3872 Acked Read 10 Done Sent Done false
11011 3873 Acked Read 5 Done Sent Done false
11012 3874 Acked Read 5 Done Sent Done false
11013 3875 Acked Write 2 Done Sent Done false
11014 3876 Acked Read 1 Done Sent Done false
11015 3877 Acked Write 9 Done Sent Done false
11016 3878 Acked Read 8 Done Sent Done false
11017 3879 Acked Read 9 Done Sent Done false
11018 3880 Acked Write 6 Done Sent Done false
11019 3881 Acked Read 5 Done Sent Done false
11020 3882 Acked Read 8 Done Sent Done false
11021 3883 Acked Read 8 Done Sent Done false
11022 3884 Acked Read 4 Done Sent Done false
11023 3885 Acked Read 10 Done Sent Done false
11024 3886 Acked Read 3 Done Sent Done false
11025 3887 Acked Write 9 Done Sent Done false
11026 3888 Acked Read 9 Done Sent Done false
11027 3889 Acked Read 1 Done Sent Done false
11028 3890 Acked Write 2 Done Sent Done false
11029 3891 Acked Read 2 Done Sent Done false
11030 3892 Acked Write 10 Done Sent Done false
11031 3893 Acked Flush 0 Done Sent Done false
11032 3894 Acked Read 8 Done Sent Done false
11033 3895 Acked Write 7 Done Sent Done false
11034 3896 Acked Flush 0 Done Sent Done false
11035 3897 Acked Read 4 Done Sent Done false
11036 3898 Acked Write 2 Done Sent Done false
11037 3899 Acked Read 2 Done Sent Done false
11038 3900 Acked Flush 0 Done Sent Done false
11039 3901 Acked Read 10 Done Sent Done false
11040 3902 Acked Read 9 Done Sent Done false
11041 3903 Acked Read 6 Done Sent Done false
11042 3904 Acked Write 8 Done Sent Done false
11043 3905 Acked Read 4 Done Sent Done false
11044 3906 Acked Read 8 Done Sent Done false
11045 3907 Acked Write 9 Done Sent Done false
11046 3908 Acked Read 7 Done Sent Done false
11047 3909 Acked Write 3 Done Sent Done false
11048 3910 Acked Read 5 Done Sent Done false
11049 3911 Acked Flush 0 Done Sent Done false
11050 3912 Acked Write 10 Done Sent Done false
11051 3913 Acked Read 4 Done Sent Done false
11052 3914 Acked Read 5 Done Sent Done false
11053 3915 Acked Read 5 Done Sent Done false
11054 3916 Acked Read 2 Done Sent Done false
11055 3917 Acked Write 8 Done Sent Done false
11056 3918 Acked Read 6 Done Sent Done false
11057 3919 Acked Read 2 Done Sent Done false
11058 3920 Acked Read 2 Done Sent Done false
11059 3921 Acked Read 5 Done Sent Done false
11060 3922 Acked Write 10 Done Sent Done false
11061 3923 Acked Read 8 Done Sent Done false
11062 3924 Acked Write 8 Done Sent Done false
11063 3925 Acked Read 4 Done Sent Done false
11064 3926 Acked Read 1 Done Sent Done false
11065 3927 Acked Write 9 Done Sent Done false
11066 3928 Acked Read 8 Done Sent Done false
11067 3929 Acked Write 10 Done Sent Done false
11068 3930 Acked Write 10 Done Sent Done false
11069 3931 Acked Write 6 Done Sent Done false
11070 3932 Acked Read 1 Done Sent Done false
11071 3933 Acked Read 3 Done Sent Done false
11072 3934 Acked Read 4 Done Sent Done false
11073 3935 Acked Read 10 Done Sent Done false
11074 3936 Acked Write 3 Done Sent Done false
11075 3937 Acked Read 10 Done Sent Done false
11076 3938 Acked Read 1 Done Sent Done false
11077 3939 Acked Write 1 Done Sent Done false
11078 3940 Acked Write 10 Done Sent Done false
11079 3941 Acked Write 2 Done Sent Done false
11080 3942 Acked Read 1 Done Sent Done false
11081 3943 Acked Write 4 Done Sent Done false
11082 3944 Acked Read 3 Done Sent Done false
11083 3945 Acked Read 4 Done Sent Done false
11084 3946 Acked Read 1 Done Sent Done false
11085 3947 Acked Write 1 Done Sent Done false
11086 3948 Acked Read 1 Done Sent Done false
11087 3949 Acked Read 2 Done Sent Done false
11088 3950 Acked Read 2 Done Sent Done false
11089 3951 Acked Read 1 Done Sent Done false
11090 3952 Acked Read 6 Done Sent Done false
11091 3953 Acked Write 4 Done Sent Done false
11092 3954 Acked Read 3 Done Sent Done false
11093 3955 Acked Read 8 Done Sent Done false
11094 3956 Acked Read 9 Done Sent Done false
11095 3957 Acked Write 3 Done Sent Done false
11096 3958 Acked Write 5 Done Sent Done false
11097 3959 Acked Flush 0 Done Sent Done false
11098 3960 Acked Read 10 Done Sent Done false
11099 3961 Acked Write 9 Done Sent Done false
11100 3962 Acked Read 6 Done Sent Done false
11101 3963 Acked Read 9 Done Sent Done false
11102 3964 Acked Read 8 Done Sent Done false
11103 3965 Acked Read 7 Done Sent Done false
11104 3966 Acked Write 7 Done Sent Done false
11105 3967 Acked Read 6 Done Sent Done false
11106 3968 Acked Write 8 Done Sent Done false
11107 3969 Acked Read 2 Done Sent Done false
11108 3970 Acked Write 7 Done Sent Done false
11109 3971 Acked Write 4 Done Sent Done false
11110 3972 Acked Read 1 Done Sent Done false
11111 3973 Acked Read 3 Done Sent Done false
11112 3974 Acked Read 4 Done Sent Done false
11113 3975 Acked Read 9 Done Sent Done false
11114 3976 Acked Write 9 Done Sent Done false
11115 3977 Acked Write 3 Done Sent Done false
11116 3978 Acked Read 5 Done Sent Done false
11117 3979 Acked Read 6 Done Sent Done false
11118 3980 Acked Read 3 Done Sent Done false
11119 3981 Acked Read 7 Done Sent Done false
11120 3982 Acked Read 3 Done Sent Done false
11121 3983 Acked Read 8 Done Sent Done false
11122 3984 Acked Read 7 Done Sent Done false
11123 3985 Acked Read 5 Done Sent Done false
11124 3986 Acked Write 4 Done Sent Done false
11125 3987 Acked Write 10 Done Sent Done false
11126 3988 Acked Write 1 Done Sent Done false
11127 3989 Acked Read 4 Done Sent Done false
11128 3990 Acked Write 9 Done Sent Done false
11129 3991 Acked Write 5 Done Sent Done false
11130 3992 Acked Read 7 Done Sent Done false
11131 3993 Acked Write 10 Done Sent Done false
11132 3994 Acked Write 3 Done Sent Done false
11133 3995 Acked Write 2 Done Sent Done false
11134 3996 Acked Read 8 Done Sent Done false
11135 3997 Acked Write 6 Done Sent Done false
11136 3998 Acked Read 7 Done Sent Done false
11137 3999 Acked Read 6 Done Sent Done false
11138 4000 Acked Read 5 Done Sent Done false
11139 4001 Acked Flush 0 Done Sent Done false
11140 4002 Acked Read 6 Done Sent Done false
11141 4003 Acked Read 4 Done Sent Done false
11142 4004 Acked Read 5 Done Sent Done false
11143 4005 Acked Flush 0 Done Sent Done false
11144 4006 Acked Write 8 Done Sent Done false
11145 4007 Acked Write 2 Done Sent Done false
11146 4008 Acked Write 7 Done Sent Done false
11147 4009 Acked Read 8 Done Sent Done false
11148 4010 Acked Read 10 Done Sent Done false
11149 4011 Acked Flush 0 Done Sent Done false
11150 4012 Acked Read 8 Done Sent Done false
11151 4013 Acked Read 1 Done Sent Done false
11152 4014 Acked Read 4 Done Sent Done false
11153 4015 Acked Flush 0 Done Sent Done false
11154 4016 Acked Read 10 Done Sent Done false
11155 4017 Acked Read 4 Done Sent Done false
11156 4018 Acked Flush 0 Done Sent Done false
11157 STATES DS:0 DS:1 DS:2 TOTAL
11158 Sent 0 302 0 302
11159 Done 302 0 302 604
11160 Skipped 0 0 0 0
11161 Error 0 0 0 0
11162 Last Flush: 4018 3716 4018
11163 Downstairs last five completed: 3716 3715 3714 3713 3712
11164 Upstairs last five completed: 4018 4017 4016 4015 4014
11165 CLIENT: Up:0 ds:302 act:2
111662025-01-15T05:21:19.175ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111672025-01-15T05:21:19.175ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46746) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111682025-01-15T05:21:19.177ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111692025-01-15T05:21:19.177ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111702025-01-15T05:21:19.177ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111712025-01-15T05:21:19.177ZINFOcrucible: send last flush ID to this DS: 3716 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111722025-01-15T05:21:19.177ZINFOcrucible: Replied this last flush ID: 3716 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111732025-01-15T05:21:19.177ZINFOcrucible: [1] client re-new 302 jobs since flush 3716 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111742025-01-15T05:21:19.178ZINFOcrucible: [1] Replayed 302 jobs since flush: 3716 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11175 ----------------------------------------------------------------
11176 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11177 Downstairs last five completed: 4018 4017 4016 4015 4014
11178 Upstairs last five completed: 4018 4017 4016 4015 4014
11179 CLIENT: Up:0 ds:0 act:3
11180 Replay: All jobs finished
11181 Main task has work to do, go find it
11182 got dsc Stop(0)
11183 stop 0
11184 Jan 15 05:21:21.424 INFO request completed, latency_us: 297, response_code: 204, uri: /stop/cid/0, method: [8810] Got stop action so:false kr:false
11185 POST, req_id: 592c38ae-f01c-4a30-a763-7ed3f4c46701, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
111862025-01-15T05:21:21.425ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111872025-01-15T05:21:21.425ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
111882025-01-15T05:21:21.425ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11189 Jan 15 05:21:21.425 INFO request completed, latency_us: 152, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8d662e32-ac35-4041-a57e-bc0e91233cf7, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
111902025-01-15T05:21:21.425ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
11191 [8810] Exited with: ExitStatus(unix_wait_status(9))
11192 [8810][0] reports Exit
11193 Jan 15 05:21:25.427 INFO request completed, latency_us: 181, response_code: 200, uri: /state/cid/0, method: GET, req_id: 397d34a3-4df4-4f50-9973-7da80b0ee86e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
11194 001/300 Read block 3660 len 12288
11195 002/300 Write block 2819 len 8192 data: 2 3
11196 003/300 Flush
11197 004/300 Read block 4155 len 16384
11198 005/300 Write block 4723 len 24576 data: 3 3 4 4 4 5
11199 006/300 Write block 1498 len 8192 data: 3 2
11200 007/300 Write block 2294 len 40960 data: 2 2 2 2 2 2 2 3 3 3
11201 008/300 Read block 3572 len 12288
11202 009/300 Write block 3098 len 16384 data: 2 3 3 4
11203 010/300 Write block 3020 len 40960 data: 3 3 3 3 3 3 3 4 4 4
11204 011/300 Flush
11205 012/300 Read block 4872 len 36864
11206 013/300 Read block 413 len 20480
11207 014/300 Flush
11208 015/300 Read block 2107 len 28672
11209 016/300 Write block 4631 len 8192 data: 3 3
11210 017/300 Read block 2626 len 20480
11211 018/300 Write block 4917 len 36864 data: 4 2 2 2 2 2 3 3 2
11212 019/300 Write block 4553 len 40960 data: 6 5 4 4 4 3 3 3 4 4
11213 020/300 Read block 2679 len 16384
11214 021/300 Flush
11215 022/300 Flush
11216 023/300 Flush
11217 024/300 Write block 2391 len 24576 data: 2 2 2 2 2 2
11218 025/300 Read block 941 len 8192
11219 026/300 Read block 773 len 8192
11220 027/300 Write block 3396 len 8192 data: 2 2
11221 028/300 Read block 1697 len 16384
11222 029/300 Read block 749 len 32768
11223 030/300 Read block 2608 len 28672
11224 031/300 Flush
11225 032/300 Write block 2657 len 28672 data: 4 4 3 2 2 2 2
11226 033/300 Read block 1626 len 4096
11227 034/300 Read block 3778 len 28672
11228 035/300 Write block 4827 len 12288 data: 2 2 2
11229 036/300 Read block 3364 len 24576
11230 037/300 Read block 91 len 36864
11231 038/300 Flush
11232 039/300 Read block 461 len 24576
11233 040/300 Write block 639 len 20480 data: 4 4 4 4 4
11234 041/300 Read block 2342 len 36864
11235 042/300 Read block 1258 len 4096
11236 043/300 Read block 2540 len 40960
11237 044/300 Write block 1156 len 36864 data: 3 3 2 2 2 2 2 3 3
11238 045/300 Write block 1419 len 8192 data: 4 4
11239 046/300 Read block 2132 len 20480
11240 047/300 Write block 4425 len 32768 data: 7 7 7 7 6 4 4 5
11241 048/300 Read block 3245 len 8192
11242 049/300 Write block 2163 len 8192 data: 2 2
11243 050/300 Read block 650 len 36864
11244 051/300 Read block 4715 len 8192
11245 052/300 Write block 4618 len 40960 data: 3 3 3 3 3 3 2 2 2 3
11246 053/300 Read block 2359 len 20480
11247 054/300 Read block 1975 len 36864
11248 055/300 Read block 1990 len 40960
11249 056/300 Read block 3386 len 8192
11250 057/300 Read block 2569 len 4096
11251 058/300 Read block 811 len 36864
11252 059/300 Read block 662 len 32768
11253 060/300 Read block 1633 len 28672
11254 061/300 Flush
11255 062/300 Read block 4285 len 28672
11256 063/300 Read block 4852 len 36864
11257 064/300 Read block 2119 len 28672
11258 065/300 Read block 3185 len 12288
11259 066/300 Read block 1337 len 8192
11260 067/300 Write block 1780 len 32768 data: 3 3 2 5 5 5 7 7
11261 068/300 Read block 4174 len 32768
11262 069/300 Write block 2022 len 40960 data: 5 5 5 5 5 4 5 7 6 5
11263 070/300 Read block 4067 len 20480
11264 071/300 Read block 2403 len 16384
11265 072/300 Read block 4058 len 40960
11266 073/300 Flush
11267 074/300 Write block 4710 len 20480 data: 2 2 2 2 2
11268 075/300 Write block 4335 len 24576 data: 2 2 2 3 4 3
11269 076/300 Write block 574 len 36864 data: 4 4 4 4 4 4 3 3 3
11270 077/300 Write block 3013 len 8192 data: 2 2
11271 078/300 Read block 1976 len 28672
11272 079/300 Read block 2803 len 32768
11273 080/300 Write block 1514 len 12288 data: 4 4 3
11274 081/300 Write block 3031 len 24576 data: 4 4 4 4 5 5
11275 082/300 Write block 4777 len 24576 data: 5 5 4 3 3 3
11276 083/300 Read block 1270 len 16384
11277 084/300 Read block 205 len 36864
11278 085/300 Flush
11279 086/300 Flush
11280 087/300 Write block 106 len 24576 data: 3 2 2 2 2 2
11281 088/300 Read block 492 len 20480
11282 089/300 Write block 2291 len 12288 data: 2 2 2
11283 090/300 Write block 4410 len 8192 data: 4 4
11284 091/300 Read block 3178 len 36864
11285 092/300 Read block 530 len 12288
11286 093/300 Read block 3758 len 4096
11287 094/300 Flush
11288 095/300 Write block 3892 len 40960 data: 4 4 5 4 4 4 4 3 3 3
11289 096/300 Read block 3740 len 8192
11290 097/300 Flush
11291 098/300 Read block 2087 len 16384
11292 099/300 Write block 4815 len 40960 data: 3 3 5 4 4 4 3 3 3 3
11293 100/300 Read block 3331 len 24576
11294 101/300 Flush
11295 102/300 Write block 3815 len 20480 data: 3 4 3 3 3
11296 103/300 Read block 2853 len 24576
11297 104/300 Read block 3968 len 20480
11298 105/300 Read block 3607 len 28672
11299 106/300 Read block 4484 len 4096
11300 107/300 Read block 992 len 16384
11301 108/300 Write block 3151 len 40960 data: 3 3 3 2 3 4 4 4 4 5
11302 109/300 Read block 1230 len 4096
11303 110/300 Write block 4378 len 40960 data: 4 4 5 5 5 4 4 4 3 3
11304 111/300 Flush
11305 112/300 Read block 1193 len 20480
11306 113/300 Read block 4897 len 16384
11307 114/300 Read block 2639 len 32768
11308 115/300 Read block 1414 len 16384
11309 116/300 Read block 2119 len 20480
11310 117/300 Read block 4613 len 16384
11311 118/300 Read block 4281 len 32768
11312 119/300 Read block 3283 len 28672
11313 120/300 Read block 1390 len 4096
11314 121/300 Read block 3438 len 36864
11315 122/300 Read block 4776 len 16384
11316 123/300 Flush
11317 124/300 Read block 1325 len 28672
11318 125/300 Flush
11319 126/300 Read block 3594 len 20480
11320 127/300 Read block 4585 len 12288
11321 128/300 Flush
11322 129/300 Write block 3857 len 36864 data: 5 5 4 3 3 4 3 3 3
11323 130/300 Write block 2746 len 36864 data: 3 3 2 2 2 2 2 2 2
11324 131/300 Flush
11325 132/300 Read block 1856 len 36864
11326 133/300 Write block 3680 len 12288 data: 6 6 6
11327 134/300 Write block 4276 len 4096 data: 3
11328 135/300 Write block 750 len 12288 data: 3 4 5
11329 136/300 Flush
11330 137/300 Flush
11331 138/300 Read block 3869 len 40960
11332 139/300 Flush
11333 140/300 Read block 584 len 20480
11334 141/300 Read block 4143 len 32768
11335 142/300 Write block 2940 len 36864 data: 2 2 2 2 2 2 2 2 2
11336 143/300 Read block 3219 len 20480
11337 144/300 Read block 4456 len 20480
11338 145/300 Write block 92 len 40960 data: 4 4 3 4 3 3 2 2 3 5
11339 146/300 Read block 3054 len 16384
11340 147/300 Write block 2233 len 40960 data: 4 4 3 2 2 2 3 4 4 4
11341 148/300 Read block 672 len 4096
11342 149/300 Read block 641 len 32768
11343 150/300 Flush
11344 151/300 Write block 88 len 16384 data: 4 4 4 4
11345 152/300 Read block 2241 len 28672
11346 153/300 Flush
11347 154/300 Read block 2329 len 40960
11348 155/300 Write block 84 len 20480 data: 2 2 2 2 5
11349 156/300 Write block 4525 len 4096 data: 4
11350 157/300 Write block 3914 len 36864 data: 3 4 4 4 5 4 4 3 4
11351 158/300 Read block 4176 len 32768
11352 159/300 Write block 2822 len 12288 data: 6 5 5
11353 160/300 Read block 609 len 32768
11354 161/300 Read block 1784 len 40960
11355 162/300 Write block 3736 len 12288 data: 3 3 4
11356 163/300 Read block 3473 len 28672
11357 164/300 Read block 4800 len 8192
11358 165/300 Write block 2270 len 28672 data: 2 2 2 2 2 2 2
11359 166/300 Read block 2873 len 24576
11360 167/300 Read block 1089 len 24576
11361 168/300 Read block 1714 len 36864
11362 169/300 Read block 2342 len 20480
11363 170/300 Read block 3562 len 20480
11364 171/300 Read block 2925 len 40960
11365 172/300 Write block 3420 len 16384 data: 2 2 3 3
11366 173/300 Flush
11367 174/300 Write block 1369 len 8192 data: 5 5
11368 175/300 Read block 2937 len 20480
11369 176/300 Read block 3237 len 8192
11370 177/300 Read block 546 len 28672
11371 178/300 Write block 3964 len 8192 data: 2 2
11372 179/300 Read block 4139 len 12288
11373 180/300 Flush
11374 181/300 Flush
11375 182/300 Read block 3520 len 8192
11376 183/300 Write block 3669 len 24576 data: 4 4 5 5 4 5
11377 184/300 Read block 2697 len 40960
11378 185/300 Write block 4483 len 36864 data: 3 4 3 3 3 3 3 3 3
11379 186/300 Read block 40 len 16384
11380 187/300 Write block 2702 len 20480 data: 2 3 2 2 2
11381 188/300 Write block 2178 len 24576 data: 2 2 2 2 3 3
11382 189/300 Write block 677 len 8192 data: 3 5
11383 190/300 Read block 3037 len 36864
11384 191/300 Flush
11385 192/300 Write block 3092 len 40960 data: 3 3 3 3 2 2 3 4 4 5
11386 193/300 Read block 2248 len 20480
11387 194/300 Read block 320 len 28672
11388 195/300 Read block 4028 len 28672
11389 196/300 Flush
11390 197/300 Read block 1423 len 40960
11391 198/300 Flush
11392 199/300 Read block 3564 len 24576
11393 200/300 Read block 1103 len 20480
11394 201/300 Write block 2759 len 32768 data: 6 7 7 6 6 6 4 3
11395 202/300 Write block 373 len 12288 data: 4 4 5
11396 203/300 Read block 45 len 4096
11397 204/300 Read block 4500 len 16384
11398 205/300 Flush
11399 206/300 Write block 2513 len 36864 data: 4 3 2 2 2 2 2 2 3
11400 207/300 Read block 4720 len 36864
11401 208/300 Write block 3537 len 32768 data: 4 3 3 3 3 3 3 4
11402 209/300 Read block 2760 len 16384
11403 210/300 Write block 2918 len 36864 data: 5 6 7 7 7 6 5 5 3
11404 211/300 Flush
11405 212/300 Read block 4490 len 20480
11406 213/300 Flush
11407 214/300 Read block 3261 len 36864
11408 215/300 Read block 3921 len 28672
11409 216/300 Read block 2700 len 12288
11410 217/300 Read block 1346 len 12288
11411 218/300 Read block 1235 len 8192
11412 219/300 Flush
11413 220/300 Read block 2839 len 36864
11414 221/300 Write block 1899 len 20480 data: 3 4 3 3 3
11415 222/300 Write block 4359 len 28672 data: 5 5 5 4 5 4 5
11416 223/300 Read block 4903 len 24576
11417 224/300 Flush
11418 225/300 Read block 4496 len 16384
11419 226/300 Write block 2182 len 40960 data: 4 4 3 3 3 3 3 2 2 2
11420 227/300 Write block 4083 len 4096 data: 3
11421 228/300 Read block 1247 len 12288
11422 229/300 Read block 3345 len 36864
11423 230/300 Write block 392 len 28672 data: 2 2 2 3 3 3 3
11424 231/300 Write block 2353 len 20480 data: 3 3 2 2 3
11425 232/300 Write block 707 len 8192 data: 3 3
11426 233/300 Write block 1214 len 4096 data: 3
11427 234/300 Read block 1084 len 40960
11428 235/300 Write block 2124 len 16384 data: 2 2 2 2
11429 236/300 Write block 4072 len 20480 data: 2 2 2 2 2
11430 237/300 Write block 2855 len 32768 data: 2 3 3 4 4 4 5 4
11431 238/300 Read block 2841 len 28672
11432 239/300 Read block 1240 len 4096
11433 240/300 Read block 2446 len 32768
11434 241/300 Read block 2035 len 24576
11435 242/300 Write block 2439 len 8192 data: 5 5
11436 243/300 Read block 3410 len 20480
11437 244/300 Write block 88 len 12288 data: 6 5 5
11438 245/300 Read block 1595 len 20480
11439 246/300 Write block 2338 len 36864 data: 2 2 2 2 3 3 4 4 4
11440 247/300 Write block 2347 len 24576 data: 5 6 5 5 3 3
11441 248/300 Write block 3162 len 20480 data: 3 4 4 4 5
11442 249/300 Read block 1852 len 12288
11443 250/300 Write block 454 len 12288 data: 3 4 4
11444 251/300 Flush
11445 252/300 Write block 689 len 8192 data: 3 5
11446 253/300 Write block 1462 len 20480 data: 4 4 4 4 4
11447 254/300 Write block 4034 len 36864 data: 2 3 3 3 3 3 3 2 2
11448 255/300 Read block 3794 len 40960
11449 256/300 Read block 1773 len 20480
11450 257/300 Flush
11451 258/300 Read block 3572 len 32768
11452 259/300 Write block 3554 len 24576 data: 3 4 4 3 4 4
11453 260/300 Write block 4344 len 24576 data: 4 4 4 5 4 5
11454 261/300 Read block 4514 len 12288
11455 262/300 Read block 3142 len 40960
11456 263/300 Read block 2316 len 32768
11457 264/300 Flush
11458 265/300 Read block 4605 len 32768
11459 266/300 Write block 1397 len 32768 data: 2 2 3 3 3 3 3 3
11460 267/300 Read block 3040 len 24576
11461 268/300 Write block 677 len 16384 data: 4 6 5 4
11462 269/300 Read block 608 len 40960
11463 270/300 Write block 3088 len 20480 data: 2 3 3 3 4
11464 271/300 Flush
11465 272/300 Read block 580 len 12288
11466 273/300 Write block 3647 len 24576 data: 3 3 4 7 7 5
11467 274/300 Flush
11468 275/300 Read block 4181 len 24576
11469 276/300 Flush
11470 277/300 Write block 4663 len 20480 data: 2 3 3 3 2
11471 278/300 Read block 301 len 32768
11472 279/300 Flush
11473 280/300 Write block 2552 len 4096 data: 3
11474 281/300 Write block 582 len 28672 data: 4 3 3 3 3 3 3
11475 282/300 Write block 3375 len 16384 data: 2 2 2 2
11476 283/300 Read block 4792 len 40960
11477 284/300 Write block 2503 len 24576 data: 3 3 3 3 3 4
11478 285/300 Read block 4390 len 24576
11479 286/300 Write block 3653 len 8192 data: 5 4
11480 287/300 Flush
11481 288/300 Write block 2016 len 28672 data: 3 3 3 3 4 4 6
11482 289/300 Write block 1877 len 12288 data: 6 6 5
11483 290/300 Flush
11484 291/300 Read block 725 len 40960
11485 292/300 Flush
11486 293/300 Flush
11487 294/300 Read block 4985 len 4096
11488 295/300 Write block 419 len 12288 data: 2 2 2
11489 296/300 Read block 1954 len 32768
11490 297/300 Write block 2182 len 24576 data: 5 5 4 4 4 4
11491 298/300 Write block 1081 len 28672 data: 3 3 3 3 3 2 3
11492 299/300 Read block 1779 len 4096
11493 300/300 Write block 3397 len 24576 data: 3 2 2 2 2 2
11494 301/300 Write block 3100 len 20480 data: 5 6 4 4 4
11495 Main task has work to do, go find it
11496 got dsc Start(0)
11497 start 0
11498 Jan 15 05:21:25.666 INFO request completed, latency_us: 199, response_code: 204, uri: /start/cid/0, method: POST, req_id: ebcaf4a9-c723-4c2b-a479-306d78784fcb, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
11499 Replay: started 0, returned:Ok(())
11500 ----------------------------------------------------------------
11501 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11502 States: Offline Active Active
11503 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11504 4019 Acked Read 3 Sent Done Done false
11505 4020 Acked Write 2 Sent Done Done false
11506 4021 Acked Flush 0 Sent Done Done false
11507 4022 Acked Read 4 Sent Done Done false
11508 4023 Acked Write 6 Sent Done Done false
11509 4024 Acked Write 2 Sent Done Done false
11510 4025 Acked Write 10 Sent Done Done false
11511 4026 Acked Read 3 Sent Done Done false
11512 4027 Acked Write 4 Sent Done Done false
11513 4028 Acked Write 10 Sent Done Done false
11514 4029 Acked Flush 0 Sent Done Done false
11515 4030 Acked Read 9 Sent Done Done false
11516 4031 Acked Read 5 Sent Done Done false
11517 4032 Acked Flush 0 Sent Done Done false
11518 4033 Acked Read 7 Sent Done Done false
11519 4034 Acked Write 2 Sent Done Done false
11520 4035 Acked Read 5 Sent Done Done false
11521 4036 Acked Write 9 Sent Done Done false
11522 4037 Acked Write 10 Sent Done Done false
11523 4038 Acked Read 4 Sent Done Done false
11524 4039 Acked Flush 0 Sent Done Done false
11525 4040 Acked Flush 0 Sent Done Done false
11526 4041 Acked Flush 0 Sent Done Done false
11527 4042 Acked Write 6 Sent Done Done false
11528 4043 Acked Read 2 Sent Done Done false
11529 4044 Acked Read 2 Sent Done Done false
11530 4045 Acked Write 2 Sent Done Done false
11531 4046 Acked Read 4 Sent Done Done false
11532 4047 Acked Read 8 Sent Done Done false
11533 4048 Acked Read 7 Sent Done Done false
11534 4049 Acked Flush 0 Sent Done Done false
11535 4050 Acked Write 7 Sent Done Done false
11536 4051 Acked Read 1 Sent Done Done false
11537 4052 Acked Read 7 Sent Done Done false
11538 4053 Acked Write 3 Sent Done Done false
11539 4054 Acked Read 6 Sent Done Done false
11540 4055 Acked Read 9 Sent Done Done false
11541 4056 Acked Flush 0 Sent Done Done false
11542 4057 Acked Read 6 Sent Done Done false
11543 4058 Acked Write 5 Sent Done Done false
11544 4059 Acked Read 9 Sent Done Done false
11545 4060 Acked Read 1 Sent Done Done false
11546 4061 Acked Read 10 Sent Done Done false
11547 4062 Acked Write 9 Sent Done Done false
11548 4063 Acked Write 2 Sent Done Done false
11549 4064 Acked Read 5 Sent Done Done false
11550 4065 Acked Write 8 Sent Done Done false
11551 4066 Acked Read 2 Sent Done Done false
11552 4067 Acked Write 2 Sent Done Done false
11553 4068 Acked Read 9 Sent Done Done false
11554 4069 Acked Read 2 Sent Done Done false
11555 4070 Acked Write 10 Sent Done Done false
11556 4071 Acked Read 5 Sent Done Done false
11557 4072 Acked Read 9 Sent Done Done false
11558 4073 Acked Read 10 Sent Done Done false
11559 4074 Acked Read 2 Sent Done Done false
11560 4075 Acked Read 1 Sent Done Done false
11561 4076 Acked Read 9 Sent Done Done false
11562 4077 Acked Read 8 Sent Done Done false
11563 4078 Acked Read 7 Sent Done Done false
11564 4079 Acked Flush 0 Sent Done Done false
11565 4080 Acked Read 7 Sent Done Done false
11566 4081 Acked Read 9 Sent Done Done false
11567 4082 Acked Read 7 Sent Done Done false
11568 4083 Acked Read 3 Sent Done Done false
11569 4084 Acked Read 2 Sent Done Done false
11570 4085 Acked Write 8 Sent Done Done false
11571 4086 Acked Read 8 Sent Done Done false
11572 4087 Acked Write 10 Sent Done Done false
11573 4088 Acked Read 5 Sent Done Done false
11574 4089 Acked Read 4 Sent Done Done false
11575 4090 Acked Read 10 Sent Done Done false
11576 4091 Acked Flush 0 Sent Done Done false
11577 4092 Acked Write 5 Sent Done Done false
11578 4093 Acked Write 6 Sent Done Done false
11579 4094 Acked Write 9 Sent Done Done false
11580 4095 Acked Write 2 Sent Done Done false
11581 4096 Acked Read 7 Sent Done Done false
11582 4097 Acked Read 8 Sent Done Done false
11583 4098 Acked Write 3 Sent Done Done false
11584 4099 Acked Write 6 Sent Done Done false
11585 4100 Acked Write 6 Sent Done Done false
11586 4101 Acked Read 4 Sent Done Done false
11587 4102 Acked Read 9 Sent Done Done false
11588 4103 Acked Flush 0 Sent Done Done false
11589 4104 Acked Flush 0 Sent Done Done false
11590 4105 Acked Write 6 Sent Done Done false
11591 4106 Acked Read 5 Sent Done Done false
11592 4107 Acked Write 3 Sent Done Done false
11593 4108 Acked Write 2 Sent Done Done false
11594 4109 Acked Read 9 Sent Done Done false
11595 4110 Acked Read 3 Sent Done Done false
11596 4111 Acked Read 1 Sent Done Done false
11597 4112 Acked Flush 0 Sent Done Done false
11598 4113 Acked Write 10 Sent Done Done false
11599 4114 Acked Read 2 Sent Done Done false
11600 4115 Acked Flush 0 Sent Done Done false
11601 4116 Acked Read 4 Sent Done Done false
11602 4117 Acked Write 10 Sent Done Done false
11603 4118 Acked Read 6 Sent Done Done false
11604 4119 Acked Flush 0 Sent Done Done false
11605 4120 Acked Write 5 Sent Done Done false
11606 4121 Acked Read 6 Sent Done Done false
11607 4122 Acked Read 5 Sent Done Done false
11608 4123 Acked Read 7 Sent Done Done false
11609 4124 Acked Read 1 Sent Done Done false
11610 4125 Acked Read 4 Sent Done Done false
11611 4126 Acked Write 10 Sent Done Done false
11612 4127 Acked Read 1 Sent Done Done false
11613 4128 Acked Write 10 Sent Done Done false
11614 4129 Acked Flush 0 Sent Done Done false
11615 4130 Acked Read 5 Sent Done Done false
11616 4131 Acked Read 4 Sent Done Done false
11617 4132 Acked Read 8 Sent Done Done false
11618 4133 Acked Read 4 Sent Done Done false
11619 4134 Acked Read 5 Sent Done Done false
11620 4135 Acked Read 4 Sent Done Done false
11621 4136 Acked Read 8 Sent Done Done false
11622 4137 Acked Read 7 Sent Done Done false
11623 4138 Acked Read 1 Sent Done Done false
11624 4139 Acked Read 9 Sent Done Done false
11625 4140 Acked Read 4 Sent Done Done false
11626 4141 Acked Flush 0 Sent Done Done false
11627 4142 Acked Read 7 Sent Done Done false
11628 4143 Acked Flush 0 Sent Done Done false
11629 4144 Acked Read 5 Sent Done Done false
11630 4145 Acked Read 3 Sent Done Done false
11631 4146 Acked Flush 0 Sent Done Done false
11632 4147 Acked Write 9 Sent Done Done false
11633 4148 Acked Write 9 Sent Done Done false
11634 4149 Acked Flush 0 Sent Done Done false
11635 4150 Acked Read 9 Sent Done Done false
11636 4151 Acked Write 3 Sent Done Done false
11637 4152 Acked Write 1 Sent Done Done false
11638 4153 Acked Write 3 Sent Done Done false
11639 4154 Acked Flush 0 Sent Done Done false
11640 4155 Acked Flush 0 Sent Done Done false
11641 4156 Acked Read 10 Sent Done Done false
11642 4157 Acked Flush 0 Sent Done Done false
11643 4158 Acked Read 5 Sent Done Done false
11644 4159 Acked Read 8 Sent Done Done false
11645 4160 Acked Write 9 Sent Done Done false
11646 4161 Acked Read 5 Sent Done Done false
11647 4162 Acked Read 5 Sent Done Done false
11648 4163 Acked Write 10 Sent Done Done false
11649 4164 Acked Read 4 Sent Done Done false
11650 4165 Acked Write 10 Sent Done Done false
11651 4166 Acked Read 1 Sent Done Done false
11652 4167 Acked Read 8 Sent Done Done false
11653 4168 Acked Flush 0 Sent Done Done false
11654 4169 Acked Write 4 Sent Done Done false
11655 4170 Acked Read 7 Sent Done Done false
11656 4171 Acked Flush 0 Sent Done Done false
11657 4172 Acked Read 10 Sent Done Done false
11658 4173 Acked Write 5 Sent Done Done false
11659 4174 Acked Write 1 Sent Done Done false
11660 4175 Acked Write 9 Sent Done Done false
11661 4176 Acked Read 8 Sent Done Done false
11662 4177 Acked Write 3 Sent Done Done false
11663 4178 Acked Read 8 Sent Done Done false
11664 4179 Acked Read 10 Sent Done Done false
11665 4180 Acked Write 3 Sent Done Done false
11666 4181 Acked Read 7 Sent Done Done false
11667 4182 Acked Read 2 Sent Done Done false
11668 4183 Acked Write 7 Sent Done Done false
11669 4184 Acked Read 6 Sent Done Done false
11670 4185 Acked Read 6 Sent Done Done false
11671 4186 Acked Read 9 Sent Done Done false
11672 4187 Acked Read 5 Sent Done Done false
11673 4188 Acked Read 5 Sent Done Done false
11674 4189 Acked Read 10 Sent Done Done false
11675 4190 Acked Write 4 Sent Done Done false
11676 4191 Acked Flush 0 Sent Done Done false
11677 4192 Acked Write 2 Sent Done Done false
11678 4193 Acked Read 5 Sent Done Done false
11679 4194 Acked Read 2 Sent Done Done false
11680 4195 Acked Read 7 Sent Done Done false
11681 4196 Acked Write 2 Sent Done Done false
11682 4197 Acked Read 3 Sent Done Done false
11683 4198 Acked Flush 0 Sent Done Done false
11684 4199 Acked Flush 0 Sent Done Done false
11685 4200 Acked Read 2 Sent Done Done false
11686 4201 Acked Write 6 Sent Done Done false
11687 4202 Acked Read 10 Sent Done Done false
11688 4203 Acked Write 9 Sent Done Done false
11689 4204 Acked Read 4 Sent Done Done false
11690 4205 Acked Write 5 Sent Done Done false
11691 4206 Acked Write 6 Sent Done Done false
11692 4207 Acked Write 2 Sent Done Done false
11693 4208 Acked Read 9 Sent Done Done false
11694 4209 Acked Flush 0 Sent Done Done false
11695 4210 Acked Write 10 Sent Done Done false
11696 4211 Acked Read 5 Sent Done Done false
11697 4212 Acked Read 7 Sent Done Done false
11698 4213 Acked Read 7 Sent Done Done false
11699 4214 Acked Flush 0 Sent Done Done false
11700 4215 Acked Read 10 Sent Done Done false
11701 4216 Acked Flush 0 Sent Done Done false
11702 4217 Acked Read 6 Sent Done Done false
11703 4218 Acked Read 5 Sent Done Done false
11704 4219 Acked Write 8 Sent Done Done false
11705 4220 Acked Write 3 Sent Done Done false
11706 4221 Acked Read 1 Sent Done Done false
11707 4222 Acked Read 4 Sent Done Done false
11708 4223 Acked Flush 0 Sent Done Done false
11709 4224 Acked Write 9 Sent Done Done false
11710 4225 Acked Read 9 Sent Done Done false
11711 4226 Acked Write 8 Sent Done Done false
11712 4227 Acked Read 4 Sent Done Done false
11713 4228 Acked Write 9 Sent Done Done false
11714 4229 Acked Flush 0 Sent Done Done false
11715 4230 Acked Read 5 Sent Done Done false
11716 4231 Acked Flush 0 Sent Done Done false
11717 4232 Acked Read 9 Sent Done Done false
11718 4233 Acked Read 7 Sent Done Done false
11719 4234 Acked Read 3 Sent Done Done false
11720 4235 Acked Read 3 Sent Done Done false
11721 4236 Acked Read 2 Sent Done Done false
11722 4237 Acked Flush 0 Sent Done Done false
11723 4238 Acked Read 9 Sent Done Done false
11724 4239 Acked Write 5 Sent Done Done false
11725 4240 Acked Write 7 Sent Done Done false
11726 4241 Acked Read 6 Sent Done Done false
11727 4242 Acked Flush 0 Sent Done Done false
11728 4243 Acked Read 4 Sent Done Done false
11729 4244 Acked Write 10 Sent Done Done false
11730 4245 Acked Write 1 Sent Done Done false
11731 4246 Acked Read 3 Sent Done Done false
11732 4247 Acked Read 9 Sent Done Done false
11733 4248 Acked Write 7 Sent Done Done false
11734 4249 Acked Write 5 Sent Done Done false
11735 4250 Acked Write 2 Sent Done Done false
11736 4251 Acked Write 1 Sent Done Done false
11737 4252 Acked Read 10 Sent Done Done false
11738 4253 Acked Write 4 Sent Done Done false
11739 4254 Acked Write 5 Sent Done Done false
11740 4255 Acked Write 8 Sent Done Done false
11741 4256 Acked Read 7 Sent Done Done false
11742 4257 Acked Read 1 Sent Done Done false
11743 4258 Acked Read 8 Sent Done Done false
11744 4259 Acked Read 6 Sent Done Done false
11745 4260 Acked Write 2 Sent Done Done false
11746 4261 Acked Read 5 Sent Done Done false
11747 4262 Acked Write 3 Sent Done Done false
11748 4263 Acked Read 5 Sent Done Done false
11749 4264 Acked Write 9 Sent Done Done false
11750 4265 Acked Write 6 Sent Done Done false
11751 4266 Acked Write 5 Sent Done Done false
11752 4267 Acked Read 3 Sent Done Done false
11753 4268 Acked Write 3 Sent Done Done false
11754 4269 Acked Flush 0 Sent Done Done false
11755 4270 Acked Write 2 Sent Done Done false
11756 4271 Acked Write 5 Sent Done Done false
11757 4272 Acked Write 9 Sent Done Done false
11758 4273 Acked Read 10 Sent Done Done false
11759 4274 Acked Read 5 Sent Done Done false
11760 4275 Acked Flush 0 Sent Done Done false
11761 4276 Acked Read 8 Sent Done Done false
11762 4277 Acked Write 6 Sent Done Done false
11763 4278 Acked Write 6 Sent Done Done false
11764 4279 Acked Read 3 Sent Done Done false
11765 4280 Acked Read 10 Sent Done Done false
11766 4281 Acked Read 8 Sent Done Done false
11767 4282 Acked Flush 0 Sent Done Done false
11768 4283 Acked Read 8 Sent Done Done false
11769 4284 Acked Write 8 Sent Done Done false
11770 4285 Acked Read 6 Sent Done Done false
11771 4286 Acked Write 4 Sent Done Done false
11772 4287 Acked Read 10 Sent Done Done false
11773 4288 Acked Write 5 Sent Done Done false
11774 4289 Acked Flush 0 Sent Done Done false
11775 4290 Acked Read 3 Sent Done Done false
11776 4291 Acked Write 6 Sent Done Done false
11777 4292 Acked Flush 0 Sent Done Done false
11778 4293 Acked Read 6 Sent Done Done false
11779 4294 Acked Flush 0 Sent Done Done false
11780 4295 Acked Write 5 Sent Done Done false
11781 4296 Acked Read 8 Sent Done Done false
11782 4297 Acked Flush 0 Sent Done Done false
11783 4298 Acked Write 1 Sent Done Done false
11784 4299 Acked Write 7 Sent Done Done false
11785 4300 Acked Write 4 Sent Done Done false
11786 4301 Acked Read 10 Sent Done Done false
11787 4302 Acked Write 6 Sent Done Done false
11788 4303 Acked Read 6 Sent Done Done false
11789 4304 Acked Write 2 Sent Done Done false
11790 4305 Acked Flush 0 Sent Done Done false
11791 4306 Acked Write 7 Sent Done Done false
11792 4307 Acked Write 3 Sent Done Done false
11793 4308 Acked Flush 0 Sent Done Done false
11794 4309 Acked Read 10 Sent Done Done false
11795 4310 Acked Flush 0 Sent Done Done false
11796 4311 Acked Flush 0 Sent Done Done false
11797 4312 Acked Read 1 Sent Done Done false
11798 4313 Acked Write 3 Sent Done Done false
11799 4314 Acked Read 8 Sent Done Done false
11800 4315 Acked Write 6 Sent Done Done false
11801 4316 Acked Write 7 Sent Done Done false
11802 4317 Acked Read 1 Sent Done Done false
11803 4318 Acked Write 6 Sent Done Done false
11804 4319 Acked Write 5 Sent Done Done false
11805 STATES DS:0 DS:1 DS:2 TOTAL
11806 Sent 301 0 0 301
11807 Done 0 301 301 602
11808 Skipped 0 0 0 0
11809 Error 0 0 0 0
11810 Last Flush: 4018 4311 4311
11811 Downstairs last five completed: 4018 4017 4016 4015 4014
11812 Upstairs last five completed: 4319 4318 4317 4316 4315
11813 CLIENT: Up:0 ds:301 act:2
11814 [8810] got start action
11815 [8810] I am going to restart
11816 Starting downstairs at port 8810
11817 Make output file at "/tmp/dsc/downstairs-8810.txt"
11818 [8810][0] reports Starting
11819 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1213)
11820 [8810][0] reports Running
11821 ----------------------------------------------------------------
11822 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11823 States: Offline Active Active
11824 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11825 4019 Acked Read 3 Sent Done Done false
11826 4020 Acked Write 2 Sent Done Done false
11827 4021 Acked Flush 0 Sent Done Done false
11828 4022 Acked Read 4 Sent Done Done false
11829 4023 Acked Write 6 Sent Done Done false
11830 4024 Acked Write 2 Sent Done Done false
11831 4025 Acked Write 10 Sent Done Done false
11832 4026 Acked Read 3 Sent Done Done false
11833 4027 Acked Write 4 Sent Done Done false
11834 4028 Acked Write 10 Sent Done Done false
11835 4029 Acked Flush 0 Sent Done Done false
11836 4030 Acked Read 9 Sent Done Done false
11837 4031 Acked Read 5 Sent Done Done false
11838 4032 Acked Flush 0 Sent Done Done false
11839 4033 Acked Read 7 Sent Done Done false
11840 4034 Acked Write 2 Sent Done Done false
11841 4035 Acked Read 5 Sent Done Done false
11842 4036 Acked Write 9 Sent Done Done false
11843 4037 Acked Write 10 Sent Done Done false
11844 4038 Acked Read 4 Sent Done Done false
11845 4039 Acked Flush 0 Sent Done Done false
11846 4040 Acked Flush 0 Sent Done Done false
11847 4041 Acked Flush 0 Sent Done Done false
11848 4042 Acked Write 6 Sent Done Done false
11849 4043 Acked Read 2 Sent Done Done false
11850 4044 Acked Read 2 Sent Done Done false
11851 4045 Acked Write 2 Sent Done Done false
11852 4046 Acked Read 4 Sent Done Done false
11853 4047 Acked Read 8 Sent Done Done false
11854 4048 Acked Read 7 Sent Done Done false
11855 4049 Acked Flush 0 Sent Done Done false
11856 4050 Acked Write 7 Sent Done Done false
11857 4051 Acked Read 1 Sent Done Done false
11858 4052 Acked Read 7 Sent Done Done false
11859 4053 Acked Write 3 Sent Done Done false
11860 4054 Acked Read 6 Sent Done Done false
11861 4055 Acked Read 9 Sent Done Done false
11862 4056 Acked Flush 0 Sent Done Done false
11863 4057 Acked Read 6 Sent Done Done false
11864 4058 Acked Write 5 Sent Done Done false
11865 4059 Acked Read 9 Sent Done Done false
11866 4060 Acked Read 1 Sent Done Done false
11867 4061 Acked Read 10 Sent Done Done false
11868 4062 Acked Write 9 Sent Done Done false
11869 4063 Acked Write 2 Sent Done Done false
11870 4064 Acked Read 5 Sent Done Done false
11871 4065 Acked Write 8 Sent Done Done false
11872 4066 Acked Read 2 Sent Done Done false
11873 4067 Acked Write 2 Sent Done Done false
11874 4068 Acked Read 9 Sent Done Done false
11875 4069 Acked Read 2 Sent Done Done false
11876 4070 Acked Write 10 Sent Done Done false
11877 4071 Acked Read 5 Sent Done Done false
11878 4072 Acked Read 9 Sent Done Done false
11879 4073 Acked Read 10 Sent Done Done false
11880 4074 Acked Read 2 Sent Done Done false
11881 4075 Acked Read 1 Sent Done Done false
11882 4076 Acked Read 9 Sent Done Done false
11883 4077 Acked Read 8 Sent Done Done false
11884 4078 Acked Read 7 Sent Done Done false
11885 4079 Acked Flush 0 Sent Done Done false
11886 4080 Acked Read 7 Sent Done Done false
11887 4081 Acked Read 9 Sent Done Done false
11888 4082 Acked Read 7 Sent Done Done false
11889 4083 Acked Read 3 Sent Done Done false
11890 4084 Acked Read 2 Sent Done Done false
11891 4085 Acked Write 8 Sent Done Done false
11892 4086 Acked Read 8 Sent Done Done false
11893 4087 Acked Write 10 Sent Done Done false
11894 4088 Acked Read 5 Sent Done Done false
11895 4089 Acked Read 4 Sent Done Done false
11896 4090 Acked Read 10 Sent Done Done false
11897 4091 Acked Flush 0 Sent Done Done false
11898 4092 Acked Write 5 Sent Done Done false
11899 4093 Acked Write 6 Sent Done Done false
11900 4094 Acked Write 9 Sent Done Done false
11901 4095 Acked Write 2 Sent Done Done false
11902 4096 Acked Read 7 Sent Done Done false
11903 4097 Acked Read 8 Sent Done Done false
11904 4098 Acked Write 3 Sent Done Done false
11905 4099 Acked Write 6 Sent Done Done false
11906 4100 Acked Write 6 Sent Done Done false
11907 4101 Acked Read 4 Sent Done Done false
11908 4102 Acked Read 9 Sent Done Done false
11909 4103 Acked Flush 0 Sent Done Done false
11910 4104 Acked Flush 0 Sent Done Done false
11911 4105 Acked Write 6 Sent Done Done false
11912 4106 Acked Read 5 Sent Done Done false
11913 4107 Acked Write 3 Sent Done Done false
11914 4108 Acked Write 2 Sent Done Done false
11915 4109 Acked Read 9 Sent Done Done false
11916 4110 Acked Read 3 Sent Done Done false
11917 4111 Acked Read 1 Sent Done Done false
11918 4112 Acked Flush 0 Sent Done Done false
11919 4113 Acked Write 10 Sent Done Done false
11920 4114 Acked Read 2 Sent Done Done false
11921 4115 Acked Flush 0 Sent Done Done false
11922 4116 Acked Read 4 Sent Done Done false
11923 4117 Acked Write 10 Sent Done Done false
11924 4118 Acked Read 6 Sent Done Done false
11925 4119 Acked Flush 0 Sent Done Done false
11926 4120 Acked Write 5 Sent Done Done false
11927 4121 Acked Read 6 Sent Done Done false
11928 4122 Acked Read 5 Sent Done Done false
11929 4123 Acked Read 7 Sent Done Done false
11930 4124 Acked Read 1 Sent Done Done false
11931 4125 Acked Read 4 Sent Done Done false
11932 4126 Acked Write 10 Sent Done Done false
11933 4127 Acked Read 1 Sent Done Done false
11934 4128 Acked Write 10 Sent Done Done false
11935 4129 Acked Flush 0 Sent Done Done false
11936 4130 Acked Read 5 Sent Done Done false
11937 4131 Acked Read 4 Sent Done Done false
11938 4132 Acked Read 8 Sent Done Done false
11939 4133 Acked Read 4 Sent Done Done false
11940 4134 Acked Read 5 Sent Done Done false
11941 4135 Acked Read 4 Sent Done Done false
11942 4136 Acked Read 8 Sent Done Done false
11943 4137 Acked Read 7 Sent Done Done false
11944 4138 Acked Read 1 Sent Done Done false
11945 4139 Acked Read 9 Sent Done Done false
11946 4140 Acked Read 4 Sent Done Done false
11947 4141 Acked Flush 0 Sent Done Done false
11948 4142 Acked Read 7 Sent Done Done false
11949 4143 Acked Flush 0 Sent Done Done false
11950 4144 Acked Read 5 Sent Done Done false
11951 4145 Acked Read 3 Sent Done Done false
11952 4146 Acked Flush 0 Sent Done Done false
11953 4147 Acked Write 9 Sent Done Done false
11954 4148 Acked Write 9 Sent Done Done false
11955 4149 Acked Flush 0 Sent Done Done false
11956 4150 Acked Read 9 Sent Done Done false
11957 4151 Acked Write 3 Sent Done Done false
11958 4152 Acked Write 1 Sent Done Done false
11959 4153 Acked Write 3 Sent Done Done false
11960 4154 Acked Flush 0 Sent Done Done false
11961 4155 Acked Flush 0 Sent Done Done false
11962 4156 Acked Read 10 Sent Done Done false
11963 4157 Acked Flush 0 Sent Done Done false
11964 4158 Acked Read 5 Sent Done Done false
11965 4159 Acked Read 8 Sent Done Done false
11966 4160 Acked Write 9 Sent Done Done false
11967 4161 Acked Read 5 Sent Done Done false
11968 4162 Acked Read 5 Sent Done Done false
11969 4163 Acked Write 10 Sent Done Done false
11970 4164 Acked Read 4 Sent Done Done false
11971 4165 Acked Write 10 Sent Done Done false
11972 4166 Acked Read 1 Sent Done Done false
11973 4167 Acked Read 8 Sent Done Done false
11974 4168 Acked Flush 0 Sent Done Done false
11975 4169 Acked Write 4 Sent Done Done false
11976 4170 Acked Read 7 Sent Done Done false
11977 4171 Acked Flush 0 Sent Done Done false
11978 4172 Acked Read 10 Sent Done Done false
11979 4173 Acked Write 5 Sent Done Done false
11980 4174 Acked Write 1 Sent Done Done false
11981 4175 Acked Write 9 Sent Done Done false
11982 4176 Acked Read 8 Sent Done Done false
11983 4177 Acked Write 3 Sent Done Done false
11984 4178 Acked Read 8 Sent Done Done false
11985 4179 Acked Read 10 Sent Done Done false
11986 4180 Acked Write 3 Sent Done Done false
11987 4181 Acked Read 7 Sent Done Done false
11988 4182 Acked Read 2 Sent Done Done false
11989 4183 Acked Write 7 Sent Done Done false
11990 4184 Acked Read 6 Sent Done Done false
11991 4185 Acked Read 6 Sent Done Done false
11992 4186 Acked Read 9 Sent Done Done false
11993 4187 Acked Read 5 Sent Done Done false
11994 4188 Acked Read 5 Sent Done Done false
11995 4189 Acked Read 10 Sent Done Done false
11996 4190 Acked Write 4 Sent Done Done false
11997 4191 Acked Flush 0 Sent Done Done false
11998 4192 Acked Write 2 Sent Done Done false
11999 4193 Acked Read 5 Sent Done Done false
12000 4194 Acked Read 2 Sent Done Done false
12001 4195 Acked Read 7 Sent Done Done false
12002 4196 Acked Write 2 Sent Done Done false
12003 4197 Acked Read 3 Sent Done Done false
12004 4198 Acked Flush 0 Sent Done Done false
12005 4199 Acked Flush 0 Sent Done Done false
12006 4200 Acked Read 2 Sent Done Done false
12007 4201 Acked Write 6 Sent Done Done false
12008 4202 Acked Read 10 Sent Done Done false
12009 4203 Acked Write 9 Sent Done Done false
12010 4204 Acked Read 4 Sent Done Done false
12011 4205 Acked Write 5 Sent Done Done false
12012 4206 Acked Write 6 Sent Done Done false
12013 4207 Acked Write 2 Sent Done Done false
12014 4208 Acked Read 9 Sent Done Done false
12015 4209 Acked Flush 0 Sent Done Done false
12016 4210 Acked Write 10 Sent Done Done false
12017 4211 Acked Read 5 Sent Done Done false
12018 4212 Acked Read 7 Sent Done Done false
12019 4213 Acked Read 7 Sent Done Done false
12020 4214 Acked Flush 0 Sent Done Done false
12021 4215 Acked Read 10 Sent Done Done false
12022 4216 Acked Flush 0 Sent Done Done false
12023 4217 Acked Read 6 Sent Done Done false
12024 4218 Acked Read 5 Sent Done Done false
12025 4219 Acked Write 8 Sent Done Done false
12026 4220 Acked Write 3 Sent Done Done false
12027 4221 Acked Read 1 Sent Done Done false
12028 4222 Acked Read 4 Sent Done Done false
12029 4223 Acked Flush 0 Sent Done Done false
12030 4224 Acked Write 9 Sent Done Done false
12031 4225 Acked Read 9 Sent Done Done false
12032 4226 Acked Write 8 Sent Done Done false
12033 4227 Acked Read 4 Sent Done Done false
12034 4228 Acked Write 9 Sent Done Done false
12035 4229 Acked Flush 0 Sent Done Done false
12036 4230 Acked Read 5 Sent Done Done false
12037 4231 Acked Flush 0 Sent Done Done false
12038 4232 Acked Read 9 Sent Done Done false
12039 4233 Acked Read 7 Sent Done Done false
12040 4234 Acked Read 3 Sent Done Done false
12041 4235 Acked Read 3 Sent Done Done false
12042 4236 Acked Read 2 Sent Done Done false
12043 4237 Acked Flush 0 Sent Done Done false
12044 4238 Acked Read 9 Sent Done Done false
12045 4239 Acked Write 5 Sent Done Done false
12046 4240 Acked Write 7 Sent Done Done false
12047 4241 Acked Read 6 Sent Done Done false
12048 4242 Acked Flush 0 Sent Done Done false
12049 4243 Acked Read 4 Sent Done Done false
12050 4244 Acked Write 10 Sent Done Done false
12051 4245 Acked Write 1 Sent Done Done false
12052 4246 Acked Read 3 Sent Done Done false
12053 4247 Acked Read 9 Sent Done Done false
12054 4248 Acked Write 7 Sent Done Done false
12055 4249 Acked Write 5 Sent Done Done false
12056 4250 Acked Write 2 Sent Done Done false
12057 4251 Acked Write 1 Sent Done Done false
12058 4252 Acked Read 10 Sent Done Done false
12059 4253 Acked Write 4 Sent Done Done false
12060 4254 Acked Write 5 Sent Done Done false
12061 4255 Acked Write 8 Sent Done Done false
12062 4256 Acked Read 7 Sent Done Done false
12063 4257 Acked Read 1 Sent Done Done false
12064 4258 Acked Read 8 Sent Done Done false
12065 4259 Acked Read 6 Sent Done Done false
12066 4260 Acked Write 2 Sent Done Done false
12067 4261 Acked Read 5 Sent Done Done false
12068 4262 Acked Write 3 Sent Done Done false
12069 4263 Acked Read 5 Sent Done Done false
12070 4264 Acked Write 9 Sent Done Done false
12071 4265 Acked Write 6 Sent Done Done false
12072 4266 Acked Write 5 Sent Done Done false
12073 4267 Acked Read 3 Sent Done Done false
12074 4268 Acked Write 3 Sent Done Done false
12075 4269 Acked Flush 0 Sent Done Done false
12076 4270 Acked Write 2 Sent Done Done false
12077 4271 Acked Write 5 Sent Done Done false
12078 4272 Acked Write 9 Sent Done Done false
12079 4273 Acked Read 10 Sent Done Done false
12080 4274 Acked Read 5 Sent Done Done false
12081 4275 Acked Flush 0 Sent Done Done false
12082 4276 Acked Read 8 Sent Done Done false
12083 4277 Acked Write 6 Sent Done Done false
12084 4278 Acked Write 6 Sent Done Done false
12085 4279 Acked Read 3 Sent Done Done false
12086 4280 Acked Read 10 Sent Done Done false
12087 4281 Acked Read 8 Sent Done Done false
12088 4282 Acked Flush 0 Sent Done Done false
12089 4283 Acked Read 8 Sent Done Done false
12090 4284 Acked Write 8 Sent Done Done false
12091 4285 Acked Read 6 Sent Done Done false
12092 4286 Acked Write 4 Sent Done Done false
12093 4287 Acked Read 10 Sent Done Done false
12094 4288 Acked Write 5 Sent Done Done false
12095 4289 Acked Flush 0 Sent Done Done false
12096 4290 Acked Read 3 Sent Done Done false
12097 4291 Acked Write 6 Sent Done Done false
12098 4292 Acked Flush 0 Sent Done Done false
12099 4293 Acked Read 6 Sent Done Done false
12100 4294 Acked Flush 0 Sent Done Done false
12101 4295 Acked Write 5 Sent Done Done false
12102 4296 Acked Read 8 Sent Done Done false
12103 4297 Acked Flush 0 Sent Done Done false
12104 4298 Acked Write 1 Sent Done Done false
12105 4299 Acked Write 7 Sent Done Done false
12106 4300 Acked Write 4 Sent Done Done false
12107 4301 Acked Read 10 Sent Done Done false
12108 4302 Acked Write 6 Sent Done Done false
12109 4303 Acked Read 6 Sent Done Done false
12110 4304 Acked Write 2 Sent Done Done false
12111 4305 Acked Flush 0 Sent Done Done false
12112 4306 Acked Write 7 Sent Done Done false
12113 4307 Acked Write 3 Sent Done Done false
12114 4308 Acked Flush 0 Sent Done Done false
12115 4309 Acked Read 10 Sent Done Done false
12116 4310 Acked Flush 0 Sent Done Done false
12117 4311 Acked Flush 0 Sent Done Done false
12118 4312 Acked Read 1 Sent Done Done false
12119 4313 Acked Write 3 Sent Done Done false
12120 4314 Acked Read 8 Sent Done Done false
12121 4315 Acked Write 6 Sent Done Done false
12122 4316 Acked Write 7 Sent Done Done false
12123 4317 Acked Read 1 Sent Done Done false
12124 4318 Acked Write 6 Sent Done Done false
12125 4319 Acked Write 5 Sent Done Done false
12126 4320 Acked Flush 0 Sent Done Done false
12127 STATES DS:0 DS:1 DS:2 TOTAL
12128 Sent 302 0 0 302
12129 Done 0 302 302 604
12130 Skipped 0 0 0 0
12131 Error 0 0 0 0
12132 Last Flush: 4018 4320 4320
12133 Downstairs last five completed: 4018 4017 4016 4015 4014
12134 Upstairs last five completed: 4320 4319 4318 4317 4316
12135 CLIENT: Up:0 ds:302 act:2
121362025-01-15T05:21:31.426ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
121372025-01-15T05:21:31.426ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:39168) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
121382025-01-15T05:21:31.427ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
121392025-01-15T05:21:31.428ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
121402025-01-15T05:21:31.428ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
121412025-01-15T05:21:31.428ZINFOcrucible: send last flush ID to this DS: 4018 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
121422025-01-15T05:21:31.428ZINFOcrucible: Replied this last flush ID: 4018 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
121432025-01-15T05:21:31.428ZINFOcrucible: [0] client re-new 302 jobs since flush 4018 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
121442025-01-15T05:21:31.428ZINFOcrucible: [0] Replayed 302 jobs since flush: 4018 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
12145 ----------------------------------------------------------------
12146 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12147 Downstairs last five completed: 4320 4319 4318 4317 4316
12148 Upstairs last five completed: 4320 4319 4318 4317 4316
12149 CLIENT: Up:0 ds:0 act:3
12150 Replay: All jobs finished
12151 Main task has work to do, go find it
12152 got dsc Stop(2)
12153 stop 2
12154 Jan 15 05:21:33.671 INFO request completed, latency_us: 328, response_code: 204, uri: /stop/cid/2, [8830] Got stop action so:false kr:false
12155 method: POST, req_id: d2f93f89-128d-42f0-b494-60ccc39e85ee, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
12156 Jan 15 05:21:33.671 INFO request completed, latency_us:{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:21:33.671916947Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
12157 112, response_code: 200, uri: /state/cid/2, method: GET, req_id: ad2aaea0-eb6c-4974-b8aa-1b8e60b97355, remote_addr:{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:21:33.671977271Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
12158 127.0.0.1:42402, local_addr:{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:21:33.672001229Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
12159 127.0.0.1:9998
121602025-01-15T05:21:33.672ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
12161 [8830] Exited with: ExitStatus(unix_wait_status(9))
12162 [8830][2] reports Exit
12163 Jan 15 05:21:37.674 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/2, method: GET, req_id: e9ee7905-1b1e-4918-b968-b1892276a1f9, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
12164 001/300 Read block 167 len 12288
12165 002/300 Write block 4013 len 20480 data: 2 2 2 4 4
12166 003/300 Write block 4240 len 12288 data: 4 5 4
12167 004/300 Write block 2425 len 28672 data: 2 2 3 4 4 4 4
12168 005/300 Read block 909 len 8192
12169 006/300 Write block 3446 len 12288 data: 3 3 4
12170 007/300 Read block 3607 len 8192
12171 008/300 Write block 3317 len 36864 data: 3 3 3 2 3 3 3 3 3
12172 009/300 Write block 4672 len 24576 data: 4 4 3 3 3 3
12173 010/300 Write block 4241 len 40960 data: 6 5 4 4 5 5 3 4 4 4
12174 011/300 Write block 1665 len 32768 data: 2 3 3 2 2 2 2 2
12175 012/300 Flush
12176 013/300 Read block 2672 len 4096
12177 014/300 Read block 717 len 32768
12178 015/300 Write block 3101 len 4096 data: 7
12179 016/300 Flush
12180 017/300 Write block 1891 len 20480 data: 4 4 5 6 5
12181 018/300 Write block 2368 len 28672 data: 6 5 5 4 4 3 3
12182 019/300 Write block 3790 len 12288 data: 3 3 3
12183 020/300 Read block 3947 len 20480
12184 021/300 Flush
12185 022/300 Flush
12186 023/300 Read block 964 len 32768
12187 024/300 Write block 1595 len 36864 data: 2 3 4 4 2 2 2 2 3
12188 025/300 Write block 1527 len 36864 data: 3 4 4 3 4 4 3 2 2
12189 026/300 Write block 2831 len 24576 data: 6 5 6 5 5 5
12190 027/300 Read block 2071 len 28672
12191 028/300 Read block 2196 len 16384
12192 029/300 Read block 2577 len 16384
12193 030/300 Write block 2394 len 4096 data: 3
12194 031/300 Flush
12195 032/300 Write block 2314 len 32768 data: 2 2 2 3 3 2 2 2
12196 033/300 Write block 2016 len 36864 data: 4 4 4 4 5 5 7 6 6
12197 034/300 Flush
12198 035/300 Flush
12199 036/300 Read block 4755 len 4096
12200 037/300 Write block 3676 len 24576 data: 6 5 5 5 7 7
12201 038/300 Write block 1573 len 40960 data: 7 7 7 5 5 5 5 3 2 5
12202 039/300 Write block 4258 len 24576 data: 5 6 6 6 5 5
12203 040/300 Read block 4134 len 32768
12204 041/300 Read block 2083 len 20480
12205 042/300 Flush
12206 043/300 Read block 651 len 24576
12207 044/300 Read block 562 len 20480
12208 045/300 Read block 1918 len 4096
12209 046/300 Write block 1355 len 8192 data: 3 3
12210 047/300 Write block 4087 len 4096 data: 2
12211 048/300 Write block 3703 len 12288 data: 2 2 2
12212 049/300 Flush
12213 050/300 Write block 639 len 4096 data: 5
12214 051/300 Read block 1108 len 8192
12215 052/300 Write block 2671 len 16384 data: 3 3 3 3
12216 053/300 Read block 1594 len 20480
12217 054/300 Write block 2341 len 4096 data: 3
12218 055/300 Read block 3441 len 20480
12219 056/300 Read block 4569 len 8192
12220 057/300 Read block 2646 len 4096
12221 058/300 Read block 3889 len 20480
12222 059/300 Read block 941 len 12288
12223 060/300 Write block 3319 len 36864 data: 4 3 4 4 4 4 4 3 5
12224 061/300 Flush
12225 062/300 Flush
12226 063/300 Write block 3072 len 28672 data: 4 3 3 3 3 3 4
12227 064/300 Read block 2064 len 16384
12228 065/300 Flush
12229 066/300 Write block 1678 len 8192 data: 4 4
12230 067/300 Read block 3021 len 32768
12231 068/300 Write block 3667 len 28672 data: 3 3 5 5 6 6 5
12232 069/300 Flush
12233 070/300 Write block 1297 len 32768 data: 2 2 2 3 4 3 3 4
12234 071/300 Read block 4150 len 12288
12235 072/300 Read block 2039 len 4096
12236 073/300 Read block 4939 len 36864
12237 074/300 Read block 1103 len 12288
12238 075/300 Read block 1813 len 12288
12239 076/300 Write block 469 len 20480 data: 5 5 5 5 5
12240 077/300 Write block 1265 len 40960 data: 3 3 3 2 2 2 2 2 2 2
12241 078/300 Write block 4656 len 36864 data: 4 4 3 3 2 2 2 3 4
12242 079/300 Read block 754 len 20480
12243 080/300 Write block 2777 len 4096 data: 2
12244 081/300 Read block 1099 len 8192
12245 082/300 Read block 644 len 32768
12246 083/300 Read block 2822 len 4096
12247 084/300 Read block 2508 len 20480
12248 085/300 Read block 3211 len 12288
12249 086/300 Read block 93 len 4096
12250 087/300 Read block 2673 len 36864
12251 088/300 Flush
12252 089/300 Read block 2004 len 36864
12253 090/300 Read block 306 len 36864
12254 091/300 Write block 227 len 20480 data: 3 2 2 2 3
12255 092/300 Read block 3448 len 36864
12256 093/300 Write block 4266 len 36864 data: 3 3 3 2 2 2 2 2 2
12257 094/300 Read block 4356 len 36864
12258 095/300 Read block 1564 len 4096
12259 096/300 Read block 1121 len 20480
12260 097/300 Write block 777 len 16384 data: 3 3 3 2
12261 098/300 Write block 4287 len 16384 data: 3 4 4 4
12262 099/300 Write block 1715 len 24576 data: 4 4 4 4 3 2
12263 100/300 Flush
12264 101/300 Read block 4392 len 20480
12265 102/300 Write block 1070 len 36864 data: 2 2 2 2 2 2 2 3 4
12266 103/300 Write block 1741 len 4096 data: 2
12267 104/300 Read block 4695 len 20480
12268 105/300 Write block 547 len 32768 data: 2 2 2 2 2 3 3 3
12269 106/300 Write block 4625 len 40960 data: 3 3 4 3 3 3 4 4 2 2
12270 107/300 Write block 3359 len 8192 data: 2 2
12271 108/300 Read block 3779 len 40960
12272 109/300 Write block 4676 len 8192 data: 4 4
12273 110/300 Read block 1155 len 8192
12274 111/300 Write block 4550 len 12288 data: 5 5 7
12275 112/300 Read block 3195 len 16384
12276 113/300 Write block 2012 len 32768 data: 3 3 3 3 5 5 5 5
12277 114/300 Read block 4288 len 32768
12278 115/300 Read block 4463 len 24576
12279 116/300 Read block 1006 len 4096
12280 117/300 Read block 829 len 20480
12281 118/300 Write block 189 len 20480 data: 3 3 4 4 3
12282 119/300 Write block 458 len 4096 data: 3
12283 120/300 Flush
12284 121/300 Write block 1734 len 16384 data: 3 2 2 2
12285 122/300 Read block 4291 len 24576
12286 123/300 Read block 2674 len 32768
12287 124/300 Read block 1303 len 40960
12288 125/300 Flush
12289 126/300 Read block 4626 len 40960
12290 127/300 Read block 1520 len 16384
12291 128/300 Write block 2802 len 28672 data: 4 4 4 4 2 3 4
12292 129/300 Write block 730 len 16384 data: 2 2 2 3
12293 130/300 Read block 2133 len 4096
12294 131/300 Read block 3458 len 28672
12295 132/300 Write block 4367 len 32768 data: 4 4 3 2 3 3 4 4
12296 133/300 Write block 1147 len 8192 data: 6 6
12297 134/300 Write block 1727 len 24576 data: 3 2 2 2 2 2
12298 135/300 Write block 2879 len 20480 data: 5 6 6 6 5
12299 136/300 Read block 1028 len 8192
12300 137/300 Read block 3300 len 36864
12301 138/300 Write block 3438 len 28672 data: 3 3 3 3 3 3 2
12302 139/300 Read block 773 len 12288
12303 140/300 Read block 1280 len 20480
12304 141/300 Write block 3395 len 24576 data: 2 3 4 3 3 3
12305 142/300 Read block 765 len 36864
12306 143/300 Read block 3352 len 12288
12307 144/300 Read block 3261 len 32768
12308 145/300 Write block 901 len 16384 data: 2 2 2 2
12309 146/300 Read block 3387 len 20480
12310 147/300 Read block 2854 len 12288
12311 148/300 Write block 254 len 20480 data: 4 5 5 5 6
12312 149/300 Read block 1174 len 24576
12313 150/300 Read block 3477 len 32768
12314 151/300 Write block 3378 len 36864 data: 3 2 3 4 7 6 5 5 5
12315 152/300 Write block 1913 len 20480 data: 5 5 5 5 4
12316 153/300 Write block 1271 len 4096 data: 3
12317 154/300 Read block 1406 len 4096
12318 155/300 Write block 2463 len 40960 data: 5 4 4 4 2 3 3 3 3 3
12319 156/300 Flush
12320 157/300 Read block 2189 len 24576
12321 158/300 Write block 1615 len 40960 data: 3 3 3 3 2 2 2 3 4 4
12322 159/300 Read block 2903 len 20480
12323 160/300 Flush
12324 161/300 Read block 1673 len 8192
12325 162/300 Write block 610 len 12288 data: 3 3 3
12326 163/300 Flush
12327 164/300 Read block 4430 len 8192
12328 165/300 Write block 1500 len 28672 data: 3 4 5 5 4 4 2
12329 166/300 Write block 1828 len 28672 data: 6 6 5 5 4 3 3
12330 167/300 Flush
12331 168/300 Write block 4126 len 28672 data: 5 5 4 4 4 5 6
12332 169/300 Read block 4148 len 16384
12333 170/300 Flush
12334 171/300 Write block 2302 len 8192 data: 4 4
12335 172/300 Read block 2076 len 28672
12336 173/300 Write block 2053 len 20480 data: 4 4 4 4 4
12337 174/300 Write block 4367 len 20480 data: 5 5 4 3 4
12338 175/300 Read block 4049 len 40960
12339 176/300 Flush
12340 177/300 Write block 1147 len 24576 data: 7 7 5 5 3 3
12341 178/300 Write block 869 len 24576 data: 4 5 5 5 4 3
12342 179/300 Read block 4641 len 28672
12343 180/300 Read block 2021 len 4096
12344 181/300 Read block 46 len 4096
12345 182/300 Flush
12346 183/300 Read block 2687 len 28672
12347 184/300 Read block 347 len 16384
12348 185/300 Write block 3511 len 24576 data: 3 2 2 2 3 3
12349 186/300 Write block 4860 len 8192 data: 3 3
12350 187/300 Flush
12351 188/300 Read block 4692 len 32768
12352 189/300 Write block 2865 len 28672 data: 5 5 6 5 5 3 4
12353 190/300 Read block 533 len 24576
12354 191/300 Write block 1882 len 32768 data: 3 3 3 3 3 2 4 4
12355 192/300 Write block 4376 len 8192 data: 4 4
12356 193/300 Write block 44 len 32768 data: 3 2 2 2 2 3 3 3
12357 194/300 Read block 3798 len 32768
12358 195/300 Read block 4531 len 36864
12359 196/300 Flush
12360 197/300 Write block 1259 len 16384 data: 3 3 4 3
12361 198/300 Write block 3157 len 8192 data: 5 5
12362 199/300 Write block 1092 len 24576 data: 4 3 3 3 3 3
12363 200/300 Read block 1137 len 24576
12364 201/300 Read block 2277 len 28672
12365 202/300 Write block 978 len 12288 data: 3 3 3
12366 203/300 Read block 4235 len 12288
12367 204/300 Write block 3839 len 12288 data: 3 3 3
12368 205/300 Read block 2223 len 36864
12369 206/300 Flush
12370 207/300 Write block 3374 len 16384 data: 3 3 3 3
12371 208/300 Write block 1510 len 32768 data: 2 2 3 4 5 5 4 2
12372 209/300 Read block 3361 len 40960
12373 210/300 Write block 4192 len 12288 data: 4 5 5
12374 211/300 Read block 3874 len 20480
12375 212/300 Flush
12376 213/300 Write block 1579 len 36864 data: 6 4 3 6 5 5 6 7 7
12377 214/300 Write block 3684 len 32768 data: 3 3 3 3 3 3 3 4
12378 215/300 Read block 2956 len 28672
12379 216/300 Read block 1331 len 4096
12380 217/300 Write block 4959 len 20480 data: 3 2 2 2 3
12381 218/300 Read block 1360 len 36864
12382 219/300 Write block 2703 len 8192 data: 4 3
12383 220/300 Write block 3241 len 32768 data: 3 3 5 3 4 3 3 3
12384 221/300 Write block 821 len 40960 data: 2 3 3 4 3 3 2 2 2 2
12385 222/300 Flush
12386 223/300 Write block 3473 len 20480 data: 4 4 3 3 3
12387 224/300 Read block 579 len 8192
12388 225/300 Flush
12389 226/300 Write block 3558 len 40960 data: 5 5 4 5 5 4 4 4 3 3
12390 227/300 Flush
12391 228/300 Read block 713 len 28672
12392 229/300 Write block 3189 len 12288 data: 3 3 4
12393 230/300 Write block 1179 len 12288 data: 3 3 2
12394 231/300 Read block 148 len 16384
12395 232/300 Read block 360 len 24576
12396 233/300 Read block 2476 len 32768
12397 234/300 Read block 4009 len 32768
12398 235/300 Read block 3538 len 20480
12399 236/300 Read block 1126 len 28672
12400 237/300 Write block 2946 len 32768 data: 3 3 3 2 2 2 3 3
12401 238/300 Read block 124 len 8192
12402 239/300 Write block 1170 len 16384 data: 6 6 5 4
12403 240/300 Read block 640 len 16384
12404 241/300 Read block 1941 len 32768
12405 242/300 Read block 1653 len 32768
12406 243/300 Read block 3184 len 4096
12407 244/300 Read block 2329 len 32768
12408 245/300 Write block 1599 len 40960 data: 3 3 3 3 4 3 3 4 3 3
12409 246/300 Read block 3056 len 40960
12410 247/300 Read block 66 len 28672
12411 248/300 Read block 3943 len 40960
12412 249/300 Write block 4450 len 16384 data: 6 5 5 5
12413 250/300 Read block 4099 len 20480
12414 251/300 Write block 3917 len 20480 data: 5 6 5 5 4
12415 252/300 Write block 4979 len 12288 data: 6 6 4
12416 253/300 Write block 468 len 40960 data: 4 6 6 6 6 6 4 3 3 3
12417 254/300 Write block 3667 len 36864 data: 4 4 6 6 7 7 6 6 5
12418 255/300 Read block 4698 len 24576
12419 256/300 Write block 4188 len 16384 data: 3 3 3 3
12420 257/300 Flush
12421 258/300 Write block 1260 len 12288 data: 4 5 4
12422 259/300 Write block 1842 len 4096 data: 5
12423 260/300 Read block 1207 len 24576
12424 261/300 Write block 703 len 12288 data: 3 3 2
12425 262/300 Read block 195 len 28672
12426 263/300 Read block 2640 len 16384
12427 264/300 Read block 4336 len 40960
12428 265/300 Write block 3154 len 16384 data: 3 4 5 6
12429 266/300 Read block 3953 len 12288
12430 267/300 Read block 1685 len 40960
12431 268/300 Write block 2978 len 20480 data: 3 3 3 3 5
12432 269/300 Write block 1760 len 24576 data: 4 5 5 5 4 4
12433 270/300 Read block 1902 len 40960
12434 271/300 Read block 259 len 16384
12435 272/300 Read block 799 len 8192
12436 273/300 Read block 26 len 8192
12437 274/300 Read block 3970 len 40960
12438 275/300 Read block 673 len 40960
12439 276/300 Write block 450 len 36864 data: 4 3 3 3 4 5 5 3 4
12440 277/300 Read block 2613 len 36864
12441 278/300 Write block 2852 len 4096 data: 4
12442 279/300 Read block 1705 len 32768
12443 280/300 Flush
12444 281/300 Write block 3392 len 16384 data: 3 2 2 3
12445 282/300 Read block 956 len 32768
12446 283/300 Read block 2433 len 4096
12447 284/300 Flush
12448 285/300 Read block 849 len 20480
12449 286/300 Write block 4648 len 4096 data: 2
12450 287/300 Write block 271 len 16384 data: 4 3 2 2
12451 288/300 Read block 2830 len 40960
12452 289/300 Write block 3546 len 32768 data: 4 3 3 3 3 3 2 3
12453 290/300 Write block 4319 len 12288 data: 3 3 3
12454 291/300 Read block 3796 len 8192
12455 292/300 Read block 2133 len 40960
12456 293/300 Write block 2014 len 24576 data: 4 4 6 6 6 6
12457 294/300 Read block 4444 len 12288
12458 295/300 Write block 104 len 36864 data: 5 4 4 3 3 3 3 3 3
12459 296/300 Write block 729 len 8192 data: 2 3
12460 297/300 Flush
12461 298/300 Write block 4285 len 24576 data: 4 4 4 5 5 5
12462 299/300 Write block 3160 len 4096 data: 6
12463 300/300 Write block 4481 len 32768 data: 3 3 4 5 4 4 4 4
12464 301/300 Read block 1449 len 20480
12465 Main task has work to do, go find it
12466 got dsc Start(2)
12467 start 2
12468 Jan 15 05:21:37.898 INFO request completed, latency_us: 179, response_code: 204, uri: /start/cid/2, method: POST, req_id: b60106d3-658d-4c8d-bb0f-870e067d4b98, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
12469 Replay: started 2, returned:Ok(())
12470 ----------------------------------------------------------------
12471 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12472 States: Active Active Offline
12473 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12474 4321 Acked Read 3 Done Done Sent false
12475 4322 Acked Write 5 Done Done Sent false
12476 4323 Acked Write 3 Done Done Sent false
12477 4324 Acked Write 7 Done Done Sent false
12478 4325 Acked Read 2 Done Done Sent false
12479 4326 Acked Write 3 Done Done Sent false
12480 4327 Acked Read 2 Done Done Sent false
12481 4328 Acked Write 9 Done Done Sent false
12482 4329 Acked Write 6 Done Done Sent false
12483 4330 Acked Write 10 Done Done Sent false
12484 4331 Acked Write 8 Done Done Sent false
12485 4332 Acked Flush 0 Done Done Sent false
12486 4333 Acked Read 1 Done Done Sent false
12487 4334 Acked Read 8 Done Done Sent false
12488 4335 Acked Write 1 Done Done Sent false
12489 4336 Acked Flush 0 Done Done Sent false
12490 4337 Acked Write 5 Done Done Sent false
12491 4338 Acked Write 7 Done Done Sent false
12492 4339 Acked Write 3 Done Done Sent false
12493 4340 Acked Read 5 Done Done Sent false
12494 4341 Acked Flush 0 Done Done Sent false
12495 4342 Acked Flush 0 Done Done Sent false
12496 4343 Acked Read 8 Done Done Sent false
12497 4344 Acked Write 9 Done Done Sent false
12498 4345 Acked Write 9 Done Done Sent false
12499 4346 Acked Write 6 Done Done Sent false
12500 4347 Acked Read 7 Done Done Sent false
12501 4348 Acked Read 4 Done Done Sent false
12502 4349 Acked Read 4 Done Done Sent false
12503 4350 Acked Write 1 Done Done Sent false
12504 4351 Acked Flush 0 Done Done Sent false
12505 4352 Acked Write 8 Done Done Sent false
12506 4353 Acked Write 9 Done Done Sent false
12507 4354 Acked Flush 0 Done Done Sent false
12508 4355 Acked Flush 0 Done Done Sent false
12509 4356 Acked Read 1 Done Done Sent false
12510 4357 Acked Write 6 Done Done Sent false
12511 4358 Acked Write 10 Done Done Sent false
12512 4359 Acked Write 6 Done Done Sent false
12513 4360 Acked Read 8 Done Done Sent false
12514 4361 Acked Read 5 Done Done Sent false
12515 4362 Acked Flush 0 Done Done Sent false
12516 4363 Acked Read 6 Done Done Sent false
12517 4364 Acked Read 5 Done Done Sent false
12518 4365 Acked Read 1 Done Done Sent false
12519 4366 Acked Write 2 Done Done Sent false
12520 4367 Acked Write 1 Done Done Sent false
12521 4368 Acked Write 3 Done Done Sent false
12522 4369 Acked Flush 0 Done Done Sent false
12523 4370 Acked Write 1 Done Done Sent false
12524 4371 Acked Read 2 Done Done Sent false
12525 4372 Acked Write 4 Done Done Sent false
12526 4373 Acked Read 5 Done Done Sent false
12527 4374 Acked Write 1 Done Done Sent false
12528 4375 Acked Read 5 Done Done Sent false
12529 4376 Acked Read 2 Done Done Sent false
12530 4377 Acked Read 1 Done Done Sent false
12531 4378 Acked Read 5 Done Done Sent false
12532 4379 Acked Read 3 Done Done Sent false
12533 4380 Acked Write 9 Done Done Sent false
12534 4381 Acked Flush 0 Done Done Sent false
12535 4382 Acked Flush 0 Done Done Sent false
12536 4383 Acked Write 7 Done Done Sent false
12537 4384 Acked Read 4 Done Done Sent false
12538 4385 Acked Flush 0 Done Done Sent false
12539 4386 Acked Write 2 Done Done Sent false
12540 4387 Acked Read 8 Done Done Sent false
12541 4388 Acked Write 7 Done Done Sent false
12542 4389 Acked Flush 0 Done Done Sent false
12543 4390 Acked Write 8 Done Done Sent false
12544 4391 Acked Read 3 Done Done Sent false
12545 4392 Acked Read 1 Done Done Sent false
12546 4393 Acked Read 9 Done Done Sent false
12547 4394 Acked Read 3 Done Done Sent false
12548 4395 Acked Read 3 Done Done Sent false
12549 4396 Acked Write 5 Done Done Sent false
12550 4397 Acked Write 10 Done Done Sent false
12551 4398 Acked Write 9 Done Done Sent false
12552 4399 Acked Read 5 Done Done Sent false
12553 4400 Acked Write 1 Done Done Sent false
12554 4401 Acked Read 2 Done Done Sent false
12555 4402 Acked Read 8 Done Done Sent false
12556 4403 Acked Read 1 Done Done Sent false
12557 4404 Acked Read 5 Done Done Sent false
12558 4405 Acked Read 3 Done Done Sent false
12559 4406 Acked Read 1 Done Done Sent false
12560 4407 Acked Read 9 Done Done Sent false
12561 4408 Acked Flush 0 Done Done Sent false
12562 4409 Acked Read 9 Done Done Sent false
12563 4410 Acked Read 9 Done Done Sent false
12564 4411 Acked Write 5 Done Done Sent false
12565 4412 Acked Read 9 Done Done Sent false
12566 4413 Acked Write 9 Done Done Sent false
12567 4414 Acked Read 9 Done Done Sent false
12568 4415 Acked Read 1 Done Done Sent false
12569 4416 Acked Read 5 Done Done Sent false
12570 4417 Acked Write 4 Done Done Sent false
12571 4418 Acked Write 4 Done Done Sent false
12572 4419 Acked Write 6 Done Done Sent false
12573 4420 Acked Flush 0 Done Done Sent false
12574 4421 Acked Read 5 Done Done Sent false
12575 4422 Acked Write 9 Done Done Sent false
12576 4423 Acked Write 1 Done Done Sent false
12577 4424 Acked Read 5 Done Done Sent false
12578 4425 Acked Write 8 Done Done Sent false
12579 4426 Acked Write 10 Done Done Sent false
12580 4427 Acked Write 2 Done Done Sent false
12581 4428 Acked Read 10 Done Done Sent false
12582 4429 Acked Write 2 Done Done Sent false
12583 4430 Acked Read 2 Done Done Sent false
12584 4431 Acked Write 3 Done Done Sent false
12585 4432 Acked Read 4 Done Done Sent false
12586 4433 Acked Write 8 Done Done Sent false
12587 4434 Acked Read 8 Done Done Sent false
12588 4435 Acked Read 6 Done Done Sent false
12589 4436 Acked Read 1 Done Done Sent false
12590 4437 Acked Read 5 Done Done Sent false
12591 4438 Acked Write 5 Done Done Sent false
12592 4439 Acked Write 1 Done Done Sent false
12593 4440 Acked Flush 0 Done Done Sent false
12594 4441 Acked Write 4 Done Done Sent false
12595 4442 Acked Read 6 Done Done Sent false
12596 4443 Acked Read 8 Done Done Sent false
12597 4444 Acked Read 10 Done Done Sent false
12598 4445 Acked Flush 0 Done Done Sent false
12599 4446 Acked Read 10 Done Done Sent false
12600 4447 Acked Read 4 Done Done Sent false
12601 4448 Acked Write 7 Done Done Sent false
12602 4449 Acked Write 4 Done Done Sent false
12603 4450 Acked Read 1 Done Done Sent false
12604 4451 Acked Read 7 Done Done Sent false
12605 4452 Acked Write 8 Done Done Sent false
12606 4453 Acked Write 2 Done Done Sent false
12607 4454 Acked Write 6 Done Done Sent false
12608 4455 Acked Write 5 Done Done Sent false
12609 4456 Acked Read 2 Done Done Sent false
12610 4457 Acked Read 9 Done Done Sent false
12611 4458 Acked Write 7 Done Done Sent false
12612 4459 Acked Read 3 Done Done Sent false
12613 4460 Acked Read 5 Done Done Sent false
12614 4461 Acked Write 6 Done Done Sent false
12615 4462 Acked Read 9 Done Done Sent false
12616 4463 Acked Read 3 Done Done Sent false
12617 4464 Acked Read 8 Done Done Sent false
12618 4465 Acked Write 4 Done Done Sent false
12619 4466 Acked Read 5 Done Done Sent false
12620 4467 Acked Read 3 Done Done Sent false
12621 4468 Acked Write 5 Done Done Sent false
12622 4469 Acked Read 6 Done Done Sent false
12623 4470 Acked Read 8 Done Done Sent false
12624 4471 Acked Write 9 Done Done Sent false
12625 4472 Acked Write 5 Done Done Sent false
12626 4473 Acked Write 1 Done Done Sent false
12627 4474 Acked Read 1 Done Done Sent false
12628 4475 Acked Write 10 Done Done Sent false
12629 4476 Acked Flush 0 Done Done Sent false
12630 4477 Acked Read 6 Done Done Sent false
12631 4478 Acked Write 10 Done Done Sent false
12632 4479 Acked Read 5 Done Done Sent false
12633 4480 Acked Flush 0 Done Done Sent false
12634 4481 Acked Read 2 Done Done Sent false
12635 4482 Acked Write 3 Done Done Sent false
12636 4483 Acked Flush 0 Done Done Sent false
12637 4484 Acked Read 2 Done Done Sent false
12638 4485 Acked Write 7 Done Done Sent false
12639 4486 Acked Write 7 Done Done Sent false
12640 4487 Acked Flush 0 Done Done Sent false
12641 4488 Acked Write 7 Done Done Sent false
12642 4489 Acked Read 4 Done Done Sent false
12643 4490 Acked Flush 0 Done Done Sent false
12644 4491 Acked Write 2 Done Done Sent false
12645 4492 Acked Read 7 Done Done Sent false
12646 4493 Acked Write 5 Done Done Sent false
12647 4494 Acked Write 5 Done Done Sent false
12648 4495 Acked Read 10 Done Done Sent false
12649 4496 Acked Flush 0 Done Done Sent false
12650 4497 Acked Write 6 Done Done Sent false
12651 4498 Acked Write 6 Done Done Sent false
12652 4499 Acked Read 7 Done Done Sent false
12653 4500 Acked Read 1 Done Done Sent false
12654 4501 Acked Read 1 Done Done Sent false
12655 4502 Acked Flush 0 Done Done Sent false
12656 4503 Acked Read 7 Done Done Sent false
12657 4504 Acked Read 4 Done Done Sent false
12658 4505 Acked Write 6 Done Done Sent false
12659 4506 Acked Write 2 Done Done Sent false
12660 4507 Acked Flush 0 Done Done Sent false
12661 4508 Acked Read 8 Done Done Sent false
12662 4509 Acked Write 7 Done Done Sent false
12663 4510 Acked Read 6 Done Done Sent false
12664 4511 Acked Write 8 Done Done Sent false
12665 4512 Acked Write 2 Done Done Sent false
12666 4513 Acked Write 8 Done Done Sent false
12667 4514 Acked Read 8 Done Done Sent false
12668 4515 Acked Read 9 Done Done Sent false
12669 4516 Acked Flush 0 Done Done Sent false
12670 4517 Acked Write 4 Done Done Sent false
12671 4518 Acked Write 2 Done Done Sent false
12672 4519 Acked Write 6 Done Done Sent false
12673 4520 Acked Read 6 Done Done Sent false
12674 4521 Acked Read 7 Done Done Sent false
12675 4522 Acked Write 3 Done Done Sent false
12676 4523 Acked Read 3 Done Done Sent false
12677 4524 Acked Write 3 Done Done Sent false
12678 4525 Acked Read 9 Done Done Sent false
12679 4526 Acked Flush 0 Done Done Sent false
12680 4527 Acked Write 4 Done Done Sent false
12681 4528 Acked Write 8 Done Done Sent false
12682 4529 Acked Read 10 Done Done Sent false
12683 4530 Acked Write 3 Done Done Sent false
12684 4531 Acked Read 5 Done Done Sent false
12685 4532 Acked Flush 0 Done Done Sent false
12686 4533 Acked Write 9 Done Done Sent false
12687 4534 Acked Write 8 Done Done Sent false
12688 4535 Acked Read 7 Done Done Sent false
12689 4536 Acked Read 1 Done Done Sent false
12690 4537 Acked Write 5 Done Done Sent false
12691 4538 Acked Read 9 Done Done Sent false
12692 4539 Acked Write 2 Done Done Sent false
12693 4540 Acked Write 8 Done Done Sent false
12694 4541 Acked Write 10 Done Done Sent false
12695 4542 Acked Flush 0 Done Done Sent false
12696 4543 Acked Write 5 Done Done Sent false
12697 4544 Acked Read 2 Done Done Sent false
12698 4545 Acked Flush 0 Done Done Sent false
12699 4546 Acked Write 10 Done Done Sent false
12700 4547 Acked Flush 0 Done Done Sent false
12701 4548 Acked Read 7 Done Done Sent false
12702 4549 Acked Write 3 Done Done Sent false
12703 4550 Acked Write 3 Done Done Sent false
12704 4551 Acked Read 4 Done Done Sent false
12705 4552 Acked Read 6 Done Done Sent false
12706 4553 Acked Read 8 Done Done Sent false
12707 4554 Acked Read 8 Done Done Sent false
12708 4555 Acked Read 5 Done Done Sent false
12709 4556 Acked Read 7 Done Done Sent false
12710 4557 Acked Write 8 Done Done Sent false
12711 4558 Acked Read 2 Done Done Sent false
12712 4559 Acked Write 4 Done Done Sent false
12713 4560 Acked Read 4 Done Done Sent false
12714 4561 Acked Read 8 Done Done Sent false
12715 4562 Acked Read 8 Done Done Sent false
12716 4563 Acked Read 1 Done Done Sent false
12717 4564 Acked Read 8 Done Done Sent false
12718 4565 Acked Write 10 Done Done Sent false
12719 4566 Acked Read 10 Done Done Sent false
12720 4567 Acked Read 7 Done Done Sent false
12721 4568 Acked Read 10 Done Done Sent false
12722 4569 Acked Write 4 Done Done Sent false
12723 4570 Acked Read 5 Done Done Sent false
12724 4571 Acked Write 5 Done Done Sent false
12725 4572 Acked Write 3 Done Done Sent false
12726 4573 Acked Write 10 Done Done Sent false
12727 4574 Acked Write 9 Done Done Sent false
12728 4575 Acked Read 6 Done Done Sent false
12729 4576 Acked Write 4 Done Done Sent false
12730 4577 Acked Flush 0 Done Done Sent false
12731 4578 Acked Write 3 Done Done Sent false
12732 4579 Acked Write 1 Done Done Sent false
12733 4580 Acked Read 6 Done Done Sent false
12734 4581 Acked Write 3 Done Done Sent false
12735 4582 Acked Read 7 Done Done Sent false
12736 4583 Acked Read 4 Done Done Sent false
12737 4584 Acked Read 10 Done Done Sent false
12738 4585 Acked Write 4 Done Done Sent false
12739 4586 Acked Read 3 Done Done Sent false
12740 4587 Acked Read 10 Done Done Sent false
12741 4588 Acked Write 5 Done Done Sent false
12742 4589 Acked Write 6 Done Done Sent false
12743 4590 Acked Read 10 Done Done Sent false
12744 4591 Acked Read 4 Done Done Sent false
12745 4592 Acked Read 2 Done Done Sent false
12746 4593 Acked Read 2 Done Done Sent false
12747 4594 Acked Read 10 Done Done Sent false
12748 4595 Acked Read 10 Done Done Sent false
12749 4596 Acked Write 9 Done Done Sent false
12750 4597 Acked Read 9 Done Done Sent false
12751 4598 Acked Write 1 Done Done Sent false
12752 4599 Acked Read 8 Done Done Sent false
12753 4600 Acked Flush 0 Done Done Sent false
12754 4601 Acked Write 4 Done Done Sent false
12755 4602 Acked Read 8 Done Done Sent false
12756 4603 Acked Read 1 Done Done Sent false
12757 4604 Acked Flush 0 Done Done Sent false
12758 4605 Acked Read 5 Done Done Sent false
12759 4606 Acked Write 1 Done Done Sent false
12760 4607 Acked Write 4 Done Done Sent false
12761 4608 Acked Read 10 Done Done Sent false
12762 4609 Acked Write 8 Done Done Sent false
12763 4610 Acked Write 3 Done Done Sent false
12764 4611 Acked Read 2 Done Done Sent false
12765 4612 Acked Read 10 Done Done Sent false
12766 4613 Acked Write 6 Done Done Sent false
12767 4614 Acked Read 3 Done Done Sent false
12768 4615 Acked Write 9 Done Done Sent false
12769 4616 Acked Write 2 Done Done Sent false
12770 4617 Acked Flush 0 Done Done Sent false
12771 4618 Acked Write 6 Done Done Sent false
12772 4619 Acked Write 1 Done Done Sent false
12773 4620 Acked Write 8 Done Done Sent false
12774 4621 Acked Read 5 Done Done Sent false
12775 STATES DS:0 DS:1 DS:2 TOTAL
12776 Sent 0 0 301 301
12777 Done 301 301 0 602
12778 Skipped 0 0 0 0
12779 Error 0 0 0 0
12780 Last Flush: 4617 4617 4320
12781 Downstairs last five completed: 4320 4319 4318 4317 4316
12782 Upstairs last five completed: 4621 4620 4619 4618 4617
12783 CLIENT: Up:0 ds:301 act:2
12784 ----------------------------------------------------------------
12785 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12786 States: Active Active Offline
12787 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12788 4321 Acked Read 3 Done Done Sent false
12789 4322 Acked Write 5 Done Done Sent false
12790 4323 Acked Write 3 Done Done Sent false
12791 4324 Acked Write 7 Done Done Sent false
12792 4325 Acked Read 2 Done Done Sent false
12793 4326 Acked Write 3 Done Done Sent false
12794 4327 Acked Read 2 Done Done Sent false
12795 4328 Acked Write 9 Done Done Sent false
12796 4329 Acked Write 6 Done Done Sent false
12797 4330 Acked Write 10 Done Done Sent false
12798 4331 Acked Write 8 Done Done Sent false
12799 4332 Acked Flush 0 Done Done Sent false
12800 4333 Acked Read 1 Done Done Sent false
12801 4334 Acked Read 8 Done Done Sent false
12802 4335 Acked Write 1 Done Done Sent false
12803 4336 Acked Flush 0 Done Done Sent false
12804 4337 Acked Write 5 Done Done Sent false
12805 4338 Acked Write 7 Done Done Sent false
12806 4339 Acked Write 3 Done Done Sent false
12807 4340 Acked Read 5 Done Done Sent false
12808 4341 Acked Flush 0 Done Done Sent false
12809 4342 Acked Flush 0 Done Done Sent false
12810 4343 Acked Read 8 Done Done Sent false
12811 4344 Acked Write 9 Done Done Sent false
12812 4345 Acked Write 9 Done Done Sent false
12813 4346 Acked Write 6 Done Done Sent false
12814 4347 Acked Read 7 Done Done Sent false
12815 4348 Acked Read 4 Done Done Sent false
12816 4349 Acked Read 4 Done Done Sent false
12817 4350 Acked Write 1 Done Done Sent false
12818 4351 Acked Flush 0 Done Done Sent false
12819 4352 Acked Write 8 Done Done Sent false
12820 4353 Acked Write 9 Done Done Sent false
12821 4354 Acked Flush 0 Done Done Sent false
12822 4355 Acked Flush 0 Done Done Sent false
12823 4356 Acked Read 1 Done Done Sent false
12824 4357 Acked Write 6 Done Done Sent false
12825 4358 Acked Write 10 Done Done Sent false
12826 4359 Acked Write 6 Done Done Sent false
12827 4360 Acked Read 8 Done Done Sent false
12828 4361 Acked Read 5 Done Done Sent false
12829 4362 Acked Flush 0 Done Done Sent false
12830 4363 Acked Read 6 Done Done Sent false
12831 4364 Acked Read 5 Done Done Sent false
12832 4365 Acked Read 1 Done Done Sent false
12833 4366 Acked Write 2 Done Done Sent false
12834 4367 Acked Write 1 Done Done Sent false
12835 4368 Acked Write 3 Done Done Sent false
12836 4369 Acked Flush 0 Done Done Sent false
12837 4370 Acked Write 1 Done Done Sent false
12838 4371 Acked Read 2 Done Done Sent false
12839 4372 Acked Write 4 Done Done Sent false
12840 4373 Acked Read 5 Done Done Sent false
12841 4374 Acked Write 1 Done Done Sent false
12842 4375 Acked Read 5 Done Done Sent false
12843 4376 Acked Read 2 Done Done Sent false
12844 4377 Acked Read 1 Done Done Sent false
12845 4378 Acked Read 5 Done Done Sent false
12846 4379 Acked Read 3 Done Done Sent false
12847 4380 Acked Write 9 Done Done Sent false
12848 4381 Acked Flush 0 Done Done Sent false
12849 4382 Acked Flush 0 Done Done Sent false
12850 4383 Acked Write 7 Done Done Sent false
12851 4384 Acked Read 4 Done Done Sent false
12852 4385 Acked Flush 0 Done Done Sent false
12853 4386 Acked Write 2 Done Done Sent false
12854 4387 Acked Read 8 Done Done Sent false
12855 4388 Acked Write 7 Done Done Sent false
12856 4389 Acked Flush 0 Done Done Sent false
12857 4390 Acked Write 8 Done Done Sent false
12858 4391 Acked Read 3 Done Done Sent false
12859 4392 Acked Read 1 Done Done Sent false
12860 4393 Acked Read 9 Done Done Sent false
12861 4394 Acked Read 3 Done Done Sent false
12862 4395 Acked Read 3 Done Done Sent false
12863 4396 Acked Write 5 Done Done Sent false
12864 4397 Acked Write 10 Done Done Sent false
12865 4398 Acked Write 9 Done Done Sent false
12866 4399 Acked Read 5 Done Done Sent false
12867 4400 Acked Write 1 Done Done Sent false
12868 4401 Acked Read 2 Done Done Sent false
12869 4402 Acked Read 8 Done Done Sent false
12870 4403 Acked Read 1 Done Done Sent false
12871 4404 Acked Read 5 Done Done Sent false
12872 4405 Acked Read 3 Done Done Sent false
12873 4406 Acked Read 1 Done Done Sent false
12874 4407 Acked Read 9 Done Done Sent false
12875 4408 Acked Flush 0 Done Done Sent false
12876 4409 Acked Read 9 Done Done Sent false
12877 4410 Acked Read 9 Done Done Sent false
12878 4411 Acked Write 5 Done Done Sent false
12879 4412 Acked Read 9 Done Done Sent false
12880 4413 Acked Write 9 Done Done Sent false
12881 4414 Acked Read 9 Done Done Sent false
12882 4415 Acked Read 1 Done Done Sent false
12883 4416 Acked Read 5 Done Done Sent false
12884 4417 Acked Write 4 Done Done Sent false
12885 4418 Acked Write 4 Done Done Sent false
12886 4419 Acked Write 6 Done Done Sent false
12887 4420 Acked Flush 0 Done Done Sent false
12888 4421 Acked Read 5 Done Done Sent false
12889 4422 Acked Write 9 Done Done Sent false
12890 4423 Acked Write 1 Done Done Sent false
12891 4424 Acked Read 5 Done Done Sent false
12892 4425 Acked Write 8 Done Done Sent false
12893 4426 Acked Write 10 Done Done Sent false
12894 4427 Acked Write 2 Done Done Sent false
12895 4428 Acked Read 10 Done Done Sent false
12896 4429 Acked Write 2 Done Done Sent false
12897 4430 Acked Read 2 Done Done Sent false
12898 4431 Acked Write 3 Done Done Sent false
12899 4432 Acked Read 4 Done Done Sent false
12900 4433 Acked Write 8 Done Done Sent false
12901 4434 Acked Read 8 Done Done Sent false
12902 4435 Acked Read 6 Done Done Sent false
12903 4436 Acked Read 1 Done Done Sent false
12904 4437 Acked Read 5 Done Done Sent false
12905 4438 Acked Write 5 Done Done Sent false
12906 4439 Acked Write 1 Done Done Sent false
12907 4440 Acked Flush 0 Done Done Sent false
12908 4441 Acked Write 4 Done Done Sent false
12909 4442 Acked Read 6 Done Done Sent false
12910 4443 Acked Read 8 Done Done Sent false
12911 4444 Acked Read 10 Done Done Sent false
12912 4445 Acked Flush 0 Done Done Sent false
12913 4446 Acked Read 10 Done Done Sent false
12914 4447 Acked Read 4 Done Done Sent false
12915 4448 Acked Write 7 Done Done Sent false
12916 4449 Acked Write 4 Done Done Sent false
12917 4450 Acked Read 1 Done Done Sent false
12918 4451 Acked Read 7 Done Done Sent false
12919 4452 Acked Write 8 Done Done Sent false
12920 4453 Acked Write 2 Done Done Sent false
12921 4454 Acked Write 6 Done Done Sent false
12922 4455 Acked Write 5 Done Done Sent false
12923 4456 Acked Read 2 Done Done Sent false
12924 4457 Acked Read 9 Done Done Sent false
12925 4458 Acked Write 7 Done Done Sent false
12926 4459 Acked Read 3 Done Done Sent false
12927 4460 Acked Read 5 Done Done Sent false
12928 4461 Acked Write 6 Done Done Sent false
12929 4462 Acked Read 9 Done Done Sent false
12930 4463 Acked Read 3 Done Done Sent false
12931 4464 Acked Read 8 Done Done Sent false
12932 4465 Acked Write 4 Done Done Sent false
12933 4466 Acked Read 5 Done Done Sent false
12934 4467 Acked Read 3 Done Done Sent false
12935 4468 Acked Write 5 Done Done Sent false
12936 4469 Acked Read 6 Done Done Sent false
12937 4470 Acked Read 8 Done Done Sent false
12938 4471 Acked Write 9 Done Done Sent false
12939 4472 Acked Write 5 Done Done Sent false
12940 4473 Acked Write 1 Done Done Sent false
12941 4474 Acked Read 1 Done Done Sent false
12942 4475 Acked Write 10 Done Done Sent false
12943 4476 Acked Flush 0 Done Done Sent false
12944 4477 Acked Read 6 Done Done Sent false
12945 4478 Acked Write 10 Done Done Sent false
12946 4479 Acked Read 5 Done Done Sent false
12947 4480 Acked Flush 0 Done Done Sent false
12948 4481 Acked Read 2 Done Done Sent false
12949 4482 Acked Write 3 Done Done Sent false
12950 4483 Acked Flush 0 Done Done Sent false
12951 4484 Acked Read 2 Done Done Sent false
12952 4485 Acked Write 7 Done Done Sent false
12953 4486 Acked Write 7 Done Done Sent false
12954 4487 Acked Flush 0 Done Done Sent false
12955 4488 Acked Write 7 Done Done Sent false
12956 4489 Acked Read 4 Done Done Sent false
12957 4490 Acked Flush 0 Done Done Sent false
12958 4491 Acked Write 2 Done Done Sent false
12959 4492 Acked Read 7 Done Done Sent false
12960 4493 Acked Write 5 Done Done Sent false
12961 4494 Acked Write 5 Done Done Sent false
12962 4495 Acked Read 10 Done Done Sent false
12963 4496 Acked Flush 0 Done Done Sent false
12964 4497 Acked Write 6 Done Done Sent false
12965 4498 Acked Write 6 Done Done Sent false
12966 4499 Acked Read 7 Done Done Sent false
12967 4500 Acked Read 1 Done Done Sent false
12968 4501 Acked Read 1 Done Done Sent false
12969 4502 Acked Flush 0 Done Done Sent false
12970 4503 Acked Read 7 Done Done Sent false
12971 4504 Acked Read 4 Done Done Sent false
12972 4505 Acked Write 6 Done Done Sent false
12973 4506 Acked Write 2 Done Done Sent false
12974 4507 Acked Flush 0 Done Done Sent false
12975 4508 Acked Read 8 Done Done Sent false
12976 4509 Acked Write 7 Done Done Sent false
12977 4510 Acked Read 6 Done Done Sent false
12978 4511 Acked Write 8 Done Done Sent false
12979 4512 Acked Write 2 Done Done Sent false
12980 4513 Acked Write 8 Done Done Sent false
12981 4514 Acked Read 8 Done Done Sent false
12982 4515 Acked Read 9 Done Done Sent false
12983 4516 Acked Flush 0 Done Done Sent false
12984 4517 Acked Write 4 Done Done Sent false
12985 4518 Acked Write 2 Done Done Sent false
12986 4519 Acked Write 6 Done Done Sent false
12987 4520 Acked Read 6 Done Done Sent false
12988 4521 Acked Read 7 Done Done Sent false
12989 4522 Acked Write 3 Done Done Sent false
12990 4523 Acked Read 3 Done Done Sent false
12991 4524 Acked Write 3 Done Done Sent false
12992 4525 Acked Read 9 Done Done Sent false
12993 4526 Acked Flush 0 Done Done Sent false
12994 4527 Acked Write 4 Done Done Sent false
12995 4528 Acked Write 8 Done Done Sent false
12996 4529 Acked Read 10 Done Done Sent false
12997 4530 Acked Write 3 Done Done Sent false
12998 4531 Acked Read 5 Done Done Sent false
12999 4532 Acked Flush 0 Done Done Sent false
13000 4533 Acked Write 9 Done Done Sent false
13001 4534 Acked Write 8 Done Done Sent false
13002 4535 Acked Read 7 Done Done Sent false
13003 4536 Acked Read 1 Done Done Sent false
13004 4537 Acked Write 5 Done Done Sent false
13005 4538 Acked Read 9 Done Done Sent false
13006 4539 Acked Write 2 Done Done Sent false
13007 4540 Acked Write 8 Done Done Sent false
13008 4541 Acked Write 10 Done Done Sent false
13009 4542 Acked Flush 0 Done Done Sent false
13010 4543 Acked Write 5 Done Done Sent false
13011 4544 Acked Read 2 Done Done Sent false
13012 4545 Acked Flush 0 Done Done Sent false
13013 4546 Acked Write 10 Done Done Sent false
13014 4547 Acked Flush 0 Done Done Sent false
13015 4548 Acked Read 7 Done Done Sent false
13016 4549 Acked Write 3 Done Done Sent false
13017 4550 Acked Write 3 Done Done Sent false
13018 4551 Acked Read 4 Done Done Sent false
13019 4552 Acked Read 6 Done Done Sent false
13020 4553 Acked Read 8 Done Done Sent false
13021 4554 Acked Read 8 Done Done Sent false
13022 4555 Acked Read 5 Done Done Sent false
13023 4556 Acked Read 7 Done Done Sent false
13024 4557 Acked Write 8 Done Done Sent false
13025 4558 Acked Read 2 Done Done Sent false
13026 4559 Acked Write 4 Done Done Sent false
13027 4560 Acked Read 4 Done Done Sent false
13028 4561 Acked Read 8 Done Done Sent false
13029 4562 Acked Read 8 Done Done Sent false
13030 4563 Acked Read 1 Done Done Sent false
13031 4564 Acked Read 8 Done Done Sent false
13032 4565 Acked Write 10 Done Done Sent false
13033 4566 Acked Read 10 Done Done Sent false
13034 4567 Acked Read 7 Done Done Sent false
13035 4568 Acked Read 10 Done Done Sent false
13036 4569 Acked Write 4 Done Done Sent false
13037 4570 Acked Read 5 Done Done Sent false
13038 4571 Acked Write 5 Done Done Sent false
13039 4572 Acked Write 3 Done Done Sent false
13040 4573 Acked Write 10 Done Done Sent false
13041 4574 Acked Write 9 Done Done Sent false
13042 4575 Acked Read 6 Done Done Sent false
13043 4576 Acked Write 4 Done Done Sent false
13044 4577 Acked Flush 0 Done Done Sent false
13045 4578 Acked Write 3 Done Done Sent false
13046 4579 Acked Write 1 Done Done Sent false
13047 4580 Acked Read 6 Done Done Sent false
13048 4581 Acked Write 3 Done Done Sent false
13049 4582 Acked Read 7 Done Done Sent false
13050 4583 Acked Read 4 Done Done Sent false
13051 4584 Acked Read 10 Done Done Sent false
13052 4585 Acked Write 4 Done Done Sent false
13053 4586 Acked Read 3 Done Done Sent false
13054 4587 Acked Read 10 Done Done Sent false
13055 4588 Acked Write 5 Done Done Sent false
13056 4589 Acked Write 6 Done Done Sent false
13057 4590 Acked Read 10 Done Done Sent false
13058 4591 Acked Read 4 Done Done Sent false
13059 4592 Acked Read 2 Done Done Sent false
13060 4593 Acked Read 2 Done Done Sent false
13061 4594 Acked Read 10 Done Done Sent false
13062 4595 Acked Read 10 Done Done Sent false
13063 4596 Acked Write 9 Done Done Sent false
13064 4597 Acked Read 9 Done Done Sent false
13065 4598 Acked Write 1 Done Done Sent false
13066 4599 Acked Read 8 Done Done Sent false
13067 4600 Acked Flush 0 Done Done Sent false
13068 4601 Acked Write 4 Done Done Sent false
13069 4602 Acked Read 8 Done Done Sent false
13070 4603 Acked Read 1 Done Done Sent false
13071 4604 Acked Flush 0 Done Done Sent false
13072 4605 Acked Read 5 Done Done Sent false
13073 4606 Acked Write 1 Done Done Sent false
13074 4607 Acked Write 4 Done Done Sent false
13075 4608 Acked Read 10 Done Done Sent false
13076 4609 Acked Write 8 Done Done Sent false
13077 4610 Acked Write 3 Done Done Sent false
13078 4611 Acked Read 2 Done Done Sent false
13079 4612 Acked Read 10 Done Done Sent false
13080 4613 Acked Write 6 Done Done Sent false
13081 4614 Acked Read 3 Done Done Sent false
13082 4615 Acked Write 9 Done Done Sent false
13083 4616 Acked Write 2 Done Done Sent false
13084 4617 Acked Flush 0 Done Done Sent false
13085 4618 Acked Write 6 Done Done Sent false
13086 4619 Acked Write 1 Done Done Sent false
13087 4620 Acked Write 8 Done Done Sent false
13088 4621 Acked Read 5 Done Done Sent false
13089 4622 Acked Flush 0 Done Done Sent false
13090 STATES DS:0 DS:1 DS:2 TOTAL
13091 Sent 0 0 302 302
13092 Done 302 302 0 604
13093 Skipped 0 0 0 0
13094 Error 0 0 0 0
13095 Last Flush: 4622 4622 4320
13096 Downstairs last five completed: 4320 4319 4318 4317 4316
13097 Upstairs last five completed: 4622 4621 4620 4619 4618
13098 CLIENT: Up:0 ds:302 act:2
130992025-01-15T05:21:43.673ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
131002025-01-15T05:21:43.673ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
131012025-01-15T05:21:43.673ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
131022025-01-15T05:21:43.673ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
131032025-01-15T05:21:43.673ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
13104 [8830] got start action
13105 [8830] I am going to restart
13106 Starting downstairs at port 8830
13107 Make output file at "/tmp/dsc/downstairs-8830.txt"
13108 [8830][2] reports Starting
13109 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1214)
13110 [8830][2] reports Running
13111 ----------------------------------------------------------------
13112 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13113 States: Active Active Offline
13114 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13115 4321 Acked Read 3 Done Done Sent false
13116 4322 Acked Write 5 Done Done Sent false
13117 4323 Acked Write 3 Done Done Sent false
13118 4324 Acked Write 7 Done Done Sent false
13119 4325 Acked Read 2 Done Done Sent false
13120 4326 Acked Write 3 Done Done Sent false
13121 4327 Acked Read 2 Done Done Sent false
13122 4328 Acked Write 9 Done Done Sent false
13123 4329 Acked Write 6 Done Done Sent false
13124 4330 Acked Write 10 Done Done Sent false
13125 4331 Acked Write 8 Done Done Sent false
13126 4332 Acked Flush 0 Done Done Sent false
13127 4333 Acked Read 1 Done Done Sent false
13128 4334 Acked Read 8 Done Done Sent false
13129 4335 Acked Write 1 Done Done Sent false
13130 4336 Acked Flush 0 Done Done Sent false
13131 4337 Acked Write 5 Done Done Sent false
13132 4338 Acked Write 7 Done Done Sent false
13133 4339 Acked Write 3 Done Done Sent false
13134 4340 Acked Read 5 Done Done Sent false
13135 4341 Acked Flush 0 Done Done Sent false
13136 4342 Acked Flush 0 Done Done Sent false
13137 4343 Acked Read 8 Done Done Sent false
13138 4344 Acked Write 9 Done Done Sent false
13139 4345 Acked Write 9 Done Done Sent false
13140 4346 Acked Write 6 Done Done Sent false
13141 4347 Acked Read 7 Done Done Sent false
13142 4348 Acked Read 4 Done Done Sent false
13143 4349 Acked Read 4 Done Done Sent false
13144 4350 Acked Write 1 Done Done Sent false
13145 4351 Acked Flush 0 Done Done Sent false
13146 4352 Acked Write 8 Done Done Sent false
13147 4353 Acked Write 9 Done Done Sent false
13148 4354 Acked Flush 0 Done Done Sent false
13149 4355 Acked Flush 0 Done Done Sent false
13150 4356 Acked Read 1 Done Done Sent false
13151 4357 Acked Write 6 Done Done Sent false
13152 4358 Acked Write 10 Done Done Sent false
13153 4359 Acked Write 6 Done Done Sent false
13154 4360 Acked Read 8 Done Done Sent false
13155 4361 Acked Read 5 Done Done Sent false
13156 4362 Acked Flush 0 Done Done Sent false
13157 4363 Acked Read 6 Done Done Sent false
13158 4364 Acked Read 5 Done Done Sent false
13159 4365 Acked Read 1 Done Done Sent false
13160 4366 Acked Write 2 Done Done Sent false
13161 4367 Acked Write 1 Done Done Sent false
13162 4368 Acked Write 3 Done Done Sent false
13163 4369 Acked Flush 0 Done Done Sent false
13164 4370 Acked Write 1 Done Done Sent false
13165 4371 Acked Read 2 Done Done Sent false
13166 4372 Acked Write 4 Done Done Sent false
13167 4373 Acked Read 5 Done Done Sent false
13168 4374 Acked Write 1 Done Done Sent false
13169 4375 Acked Read 5 Done Done Sent false
13170 4376 Acked Read 2 Done Done Sent false
13171 4377 Acked Read 1 Done Done Sent false
13172 4378 Acked Read 5 Done Done Sent false
13173 4379 Acked Read 3 Done Done Sent false
13174 4380 Acked Write 9 Done Done Sent false
13175 4381 Acked Flush 0 Done Done Sent false
13176 4382 Acked Flush 0 Done Done Sent false
13177 4383 Acked Write 7 Done Done Sent false
13178 4384 Acked Read 4 Done Done Sent false
13179 4385 Acked Flush 0 Done Done Sent false
13180 4386 Acked Write 2 Done Done Sent false
13181 4387 Acked Read 8 Done Done Sent false
13182 4388 Acked Write 7 Done Done Sent false
13183 4389 Acked Flush 0 Done Done Sent false
13184 4390 Acked Write 8 Done Done Sent false
13185 4391 Acked Read 3 Done Done Sent false
13186 4392 Acked Read 1 Done Done Sent false
13187 4393 Acked Read 9 Done Done Sent false
13188 4394 Acked Read 3 Done Done Sent false
13189 4395 Acked Read 3 Done Done Sent false
13190 4396 Acked Write 5 Done Done Sent false
13191 4397 Acked Write 10 Done Done Sent false
13192 4398 Acked Write 9 Done Done Sent false
13193 4399 Acked Read 5 Done Done Sent false
13194 4400 Acked Write 1 Done Done Sent false
13195 4401 Acked Read 2 Done Done Sent false
13196 4402 Acked Read 8 Done Done Sent false
13197 4403 Acked Read 1 Done Done Sent false
13198 4404 Acked Read 5 Done Done Sent false
13199 4405 Acked Read 3 Done Done Sent false
13200 4406 Acked Read 1 Done Done Sent false
13201 4407 Acked Read 9 Done Done Sent false
13202 4408 Acked Flush 0 Done Done Sent false
13203 4409 Acked Read 9 Done Done Sent false
13204 4410 Acked Read 9 Done Done Sent false
13205 4411 Acked Write 5 Done Done Sent false
13206 4412 Acked Read 9 Done Done Sent false
13207 4413 Acked Write 9 Done Done Sent false
13208 4414 Acked Read 9 Done Done Sent false
13209 4415 Acked Read 1 Done Done Sent false
13210 4416 Acked Read 5 Done Done Sent false
13211 4417 Acked Write 4 Done Done Sent false
13212 4418 Acked Write 4 Done Done Sent false
13213 4419 Acked Write 6 Done Done Sent false
13214 4420 Acked Flush 0 Done Done Sent false
13215 4421 Acked Read 5 Done Done Sent false
13216 4422 Acked Write 9 Done Done Sent false
13217 4423 Acked Write 1 Done Done Sent false
13218 4424 Acked Read 5 Done Done Sent false
13219 4425 Acked Write 8 Done Done Sent false
13220 4426 Acked Write 10 Done Done Sent false
13221 4427 Acked Write 2 Done Done Sent false
13222 4428 Acked Read 10 Done Done Sent false
13223 4429 Acked Write 2 Done Done Sent false
13224 4430 Acked Read 2 Done Done Sent false
13225 4431 Acked Write 3 Done Done Sent false
13226 4432 Acked Read 4 Done Done Sent false
13227 4433 Acked Write 8 Done Done Sent false
13228 4434 Acked Read 8 Done Done Sent false
13229 4435 Acked Read 6 Done Done Sent false
13230 4436 Acked Read 1 Done Done Sent false
13231 4437 Acked Read 5 Done Done Sent false
13232 4438 Acked Write 5 Done Done Sent false
13233 4439 Acked Write 1 Done Done Sent false
13234 4440 Acked Flush 0 Done Done Sent false
13235 4441 Acked Write 4 Done Done Sent false
13236 4442 Acked Read 6 Done Done Sent false
13237 4443 Acked Read 8 Done Done Sent false
13238 4444 Acked Read 10 Done Done Sent false
13239 4445 Acked Flush 0 Done Done Sent false
13240 4446 Acked Read 10 Done Done Sent false
13241 4447 Acked Read 4 Done Done Sent false
13242 4448 Acked Write 7 Done Done Sent false
13243 4449 Acked Write 4 Done Done Sent false
13244 4450 Acked Read 1 Done Done Sent false
13245 4451 Acked Read 7 Done Done Sent false
13246 4452 Acked Write 8 Done Done Sent false
13247 4453 Acked Write 2 Done Done Sent false
13248 4454 Acked Write 6 Done Done Sent false
13249 4455 Acked Write 5 Done Done Sent false
13250 4456 Acked Read 2 Done Done Sent false
13251 4457 Acked Read 9 Done Done Sent false
13252 4458 Acked Write 7 Done Done Sent false
13253 4459 Acked Read 3 Done Done Sent false
13254 4460 Acked Read 5 Done Done Sent false
13255 4461 Acked Write 6 Done Done Sent false
13256 4462 Acked Read 9 Done Done Sent false
13257 4463 Acked Read 3 Done Done Sent false
13258 4464 Acked Read 8 Done Done Sent false
13259 4465 Acked Write 4 Done Done Sent false
13260 4466 Acked Read 5 Done Done Sent false
13261 4467 Acked Read 3 Done Done Sent false
13262 4468 Acked Write 5 Done Done Sent false
13263 4469 Acked Read 6 Done Done Sent false
13264 4470 Acked Read 8 Done Done Sent false
13265 4471 Acked Write 9 Done Done Sent false
13266 4472 Acked Write 5 Done Done Sent false
13267 4473 Acked Write 1 Done Done Sent false
13268 4474 Acked Read 1 Done Done Sent false
13269 4475 Acked Write 10 Done Done Sent false
13270 4476 Acked Flush 0 Done Done Sent false
13271 4477 Acked Read 6 Done Done Sent false
13272 4478 Acked Write 10 Done Done Sent false
13273 4479 Acked Read 5 Done Done Sent false
13274 4480 Acked Flush 0 Done Done Sent false
13275 4481 Acked Read 2 Done Done Sent false
13276 4482 Acked Write 3 Done Done Sent false
13277 4483 Acked Flush 0 Done Done Sent false
13278 4484 Acked Read 2 Done Done Sent false
13279 4485 Acked Write 7 Done Done Sent false
13280 4486 Acked Write 7 Done Done Sent false
13281 4487 Acked Flush 0 Done Done Sent false
13282 4488 Acked Write 7 Done Done Sent false
13283 4489 Acked Read 4 Done Done Sent false
13284 4490 Acked Flush 0 Done Done Sent false
13285 4491 Acked Write 2 Done Done Sent false
13286 4492 Acked Read 7 Done Done Sent false
13287 4493 Acked Write 5 Done Done Sent false
13288 4494 Acked Write 5 Done Done Sent false
13289 4495 Acked Read 10 Done Done Sent false
13290 4496 Acked Flush 0 Done Done Sent false
13291 4497 Acked Write 6 Done Done Sent false
13292 4498 Acked Write 6 Done Done Sent false
13293 4499 Acked Read 7 Done Done Sent false
13294 4500 Acked Read 1 Done Done Sent false
13295 4501 Acked Read 1 Done Done Sent false
13296 4502 Acked Flush 0 Done Done Sent false
13297 4503 Acked Read 7 Done Done Sent false
13298 4504 Acked Read 4 Done Done Sent false
13299 4505 Acked Write 6 Done Done Sent false
13300 4506 Acked Write 2 Done Done Sent false
13301 4507 Acked Flush 0 Done Done Sent false
13302 4508 Acked Read 8 Done Done Sent false
13303 4509 Acked Write 7 Done Done Sent false
13304 4510 Acked Read 6 Done Done Sent false
13305 4511 Acked Write 8 Done Done Sent false
13306 4512 Acked Write 2 Done Done Sent false
13307 4513 Acked Write 8 Done Done Sent false
13308 4514 Acked Read 8 Done Done Sent false
13309 4515 Acked Read 9 Done Done Sent false
13310 4516 Acked Flush 0 Done Done Sent false
13311 4517 Acked Write 4 Done Done Sent false
13312 4518 Acked Write 2 Done Done Sent false
13313 4519 Acked Write 6 Done Done Sent false
13314 4520 Acked Read 6 Done Done Sent false
13315 4521 Acked Read 7 Done Done Sent false
13316 4522 Acked Write 3 Done Done Sent false
13317 4523 Acked Read 3 Done Done Sent false
13318 4524 Acked Write 3 Done Done Sent false
13319 4525 Acked Read 9 Done Done Sent false
13320 4526 Acked Flush 0 Done Done Sent false
13321 4527 Acked Write 4 Done Done Sent false
13322 4528 Acked Write 8 Done Done Sent false
13323 4529 Acked Read 10 Done Done Sent false
13324 4530 Acked Write 3 Done Done Sent false
13325 4531 Acked Read 5 Done Done Sent false
13326 4532 Acked Flush 0 Done Done Sent false
13327 4533 Acked Write 9 Done Done Sent false
13328 4534 Acked Write 8 Done Done Sent false
13329 4535 Acked Read 7 Done Done Sent false
13330 4536 Acked Read 1 Done Done Sent false
13331 4537 Acked Write 5 Done Done Sent false
13332 4538 Acked Read 9 Done Done Sent false
13333 4539 Acked Write 2 Done Done Sent false
13334 4540 Acked Write 8 Done Done Sent false
13335 4541 Acked Write 10 Done Done Sent false
13336 4542 Acked Flush 0 Done Done Sent false
13337 4543 Acked Write 5 Done Done Sent false
13338 4544 Acked Read 2 Done Done Sent false
13339 4545 Acked Flush 0 Done Done Sent false
13340 4546 Acked Write 10 Done Done Sent false
13341 4547 Acked Flush 0 Done Done Sent false
13342 4548 Acked Read 7 Done Done Sent false
13343 4549 Acked Write 3 Done Done Sent false
13344 4550 Acked Write 3 Done Done Sent false
13345 4551 Acked Read 4 Done Done Sent false
13346 4552 Acked Read 6 Done Done Sent false
13347 4553 Acked Read 8 Done Done Sent false
13348 4554 Acked Read 8 Done Done Sent false
13349 4555 Acked Read 5 Done Done Sent false
13350 4556 Acked Read 7 Done Done Sent false
13351 4557 Acked Write 8 Done Done Sent false
13352 4558 Acked Read 2 Done Done Sent false
13353 4559 Acked Write 4 Done Done Sent false
13354 4560 Acked Read 4 Done Done Sent false
13355 4561 Acked Read 8 Done Done Sent false
13356 4562 Acked Read 8 Done Done Sent false
13357 4563 Acked Read 1 Done Done Sent false
13358 4564 Acked Read 8 Done Done Sent false
13359 4565 Acked Write 10 Done Done Sent false
13360 4566 Acked Read 10 Done Done Sent false
13361 4567 Acked Read 7 Done Done Sent false
13362 4568 Acked Read 10 Done Done Sent false
13363 4569 Acked Write 4 Done Done Sent false
13364 4570 Acked Read 5 Done Done Sent false
13365 4571 Acked Write 5 Done Done Sent false
13366 4572 Acked Write 3 Done Done Sent false
13367 4573 Acked Write 10 Done Done Sent false
13368 4574 Acked Write 9 Done Done Sent false
13369 4575 Acked Read 6 Done Done Sent false
13370 4576 Acked Write 4 Done Done Sent false
13371 4577 Acked Flush 0 Done Done Sent false
13372 4578 Acked Write 3 Done Done Sent false
13373 4579 Acked Write 1 Done Done Sent false
13374 4580 Acked Read 6 Done Done Sent false
13375 4581 Acked Write 3 Done Done Sent false
13376 4582 Acked Read 7 Done Done Sent false
13377 4583 Acked Read 4 Done Done Sent false
13378 4584 Acked Read 10 Done Done Sent false
13379 4585 Acked Write 4 Done Done Sent false
13380 4586 Acked Read 3 Done Done Sent false
13381 4587 Acked Read 10 Done Done Sent false
13382 4588 Acked Write 5 Done Done Sent false
13383 4589 Acked Write 6 Done Done Sent false
13384 4590 Acked Read 10 Done Done Sent false
13385 4591 Acked Read 4 Done Done Sent false
13386 4592 Acked Read 2 Done Done Sent false
13387 4593 Acked Read 2 Done Done Sent false
13388 4594 Acked Read 10 Done Done Sent false
13389 4595 Acked Read 10 Done Done Sent false
13390 4596 Acked Write 9 Done Done Sent false
13391 4597 Acked Read 9 Done Done Sent false
13392 4598 Acked Write 1 Done Done Sent false
13393 4599 Acked Read 8 Done Done Sent false
13394 4600 Acked Flush 0 Done Done Sent false
13395 4601 Acked Write 4 Done Done Sent false
13396 4602 Acked Read 8 Done Done Sent false
13397 4603 Acked Read 1 Done Done Sent false
13398 4604 Acked Flush 0 Done Done Sent false
13399 4605 Acked Read 5 Done Done Sent false
13400 4606 Acked Write 1 Done Done Sent false
13401 4607 Acked Write 4 Done Done Sent false
13402 4608 Acked Read 10 Done Done Sent false
13403 4609 Acked Write 8 Done Done Sent false
13404 4610 Acked Write 3 Done Done Sent false
13405 4611 Acked Read 2 Done Done Sent false
13406 4612 Acked Read 10 Done Done Sent false
13407 4613 Acked Write 6 Done Done Sent false
13408 4614 Acked Read 3 Done Done Sent false
13409 4615 Acked Write 9 Done Done Sent false
13410 4616 Acked Write 2 Done Done Sent false
13411 4617 Acked Flush 0 Done Done Sent false
13412 4618 Acked Write 6 Done Done Sent false
13413 4619 Acked Write 1 Done Done Sent false
13414 4620 Acked Write 8 Done Done Sent false
13415 4621 Acked Read 5 Done Done Sent false
13416 4622 Acked Flush 0 Done Done Sent false
13417 STATES DS:0 DS:1 DS:2 TOTAL
13418 Sent 0 0 302 302
13419 Done 302 302 0 604
13420 Skipped 0 0 0 0
13421 Error 0 0 0 0
13422 Last Flush: 4622 4622 4320
13423 Downstairs last five completed: 4320 4319 4318 4317 4316
13424 Upstairs last five completed: 4622 4621 4620 4619 4618
13425 CLIENT: Up:0 ds:302 act:2
13426 ----------------------------------------------------------------
13427 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13428 States: Active Active Offline
13429 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13430 4321 Acked Read 3 Done Done Sent false
13431 4322 Acked Write 5 Done Done Sent false
13432 4323 Acked Write 3 Done Done Sent false
13433 4324 Acked Write 7 Done Done Sent false
13434 4325 Acked Read 2 Done Done Sent false
13435 4326 Acked Write 3 Done Done Sent false
13436 4327 Acked Read 2 Done Done Sent false
13437 4328 Acked Write 9 Done Done Sent false
13438 4329 Acked Write 6 Done Done Sent false
13439 4330 Acked Write 10 Done Done Sent false
13440 4331 Acked Write 8 Done Done Sent false
13441 4332 Acked Flush 0 Done Done Sent false
13442 4333 Acked Read 1 Done Done Sent false
13443 4334 Acked Read 8 Done Done Sent false
13444 4335 Acked Write 1 Done Done Sent false
13445 4336 Acked Flush 0 Done Done Sent false
13446 4337 Acked Write 5 Done Done Sent false
13447 4338 Acked Write 7 Done Done Sent false
13448 4339 Acked Write 3 Done Done Sent false
13449 4340 Acked Read 5 Done Done Sent false
13450 4341 Acked Flush 0 Done Done Sent false
13451 4342 Acked Flush 0 Done Done Sent false
13452 4343 Acked Read 8 Done Done Sent false
13453 4344 Acked Write 9 Done Done Sent false
13454 4345 Acked Write 9 Done Done Sent false
13455 4346 Acked Write 6 Done Done Sent false
13456 4347 Acked Read 7 Done Done Sent false
13457 4348 Acked Read 4 Done Done Sent false
13458 4349 Acked Read 4 Done Done Sent false
13459 4350 Acked Write 1 Done Done Sent false
13460 4351 Acked Flush 0 Done Done Sent false
13461 4352 Acked Write 8 Done Done Sent false
13462 4353 Acked Write 9 Done Done Sent false
13463 4354 Acked Flush 0 Done Done Sent false
13464 4355 Acked Flush 0 Done Done Sent false
13465 4356 Acked Read 1 Done Done Sent false
13466 4357 Acked Write 6 Done Done Sent false
13467 4358 Acked Write 10 Done Done Sent false
13468 4359 Acked Write 6 Done Done Sent false
13469 4360 Acked Read 8 Done Done Sent false
13470 4361 Acked Read 5 Done Done Sent false
13471 4362 Acked Flush 0 Done Done Sent false
13472 4363 Acked Read 6 Done Done Sent false
13473 4364 Acked Read 5 Done Done Sent false
13474 4365 Acked Read 1 Done Done Sent false
13475 4366 Acked Write 2 Done Done Sent false
13476 4367 Acked Write 1 Done Done Sent false
13477 4368 Acked Write 3 Done Done Sent false
13478 4369 Acked Flush 0 Done Done Sent false
13479 4370 Acked Write 1 Done Done Sent false
13480 4371 Acked Read 2 Done Done Sent false
13481 4372 Acked Write 4 Done Done Sent false
13482 4373 Acked Read 5 Done Done Sent false
13483 4374 Acked Write 1 Done Done Sent false
13484 4375 Acked Read 5 Done Done Sent false
13485 4376 Acked Read 2 Done Done Sent false
13486 4377 Acked Read 1 Done Done Sent false
13487 4378 Acked Read 5 Done Done Sent false
13488 4379 Acked Read 3 Done Done Sent false
13489 4380 Acked Write 9 Done Done Sent false
13490 4381 Acked Flush 0 Done Done Sent false
13491 4382 Acked Flush 0 Done Done Sent false
13492 4383 Acked Write 7 Done Done Sent false
13493 4384 Acked Read 4 Done Done Sent false
13494 4385 Acked Flush 0 Done Done Sent false
13495 4386 Acked Write 2 Done Done Sent false
13496 4387 Acked Read 8 Done Done Sent false
13497 4388 Acked Write 7 Done Done Sent false
13498 4389 Acked Flush 0 Done Done Sent false
13499 4390 Acked Write 8 Done Done Sent false
13500 4391 Acked Read 3 Done Done Sent false
13501 4392 Acked Read 1 Done Done Sent false
13502 4393 Acked Read 9 Done Done Sent false
13503 4394 Acked Read 3 Done Done Sent false
13504 4395 Acked Read 3 Done Done Sent false
13505 4396 Acked Write 5 Done Done Sent false
13506 4397 Acked Write 10 Done Done Sent false
13507 4398 Acked Write 9 Done Done Sent false
13508 4399 Acked Read 5 Done Done Sent false
13509 4400 Acked Write 1 Done Done Sent false
13510 4401 Acked Read 2 Done Done Sent false
13511 4402 Acked Read 8 Done Done Sent false
13512 4403 Acked Read 1 Done Done Sent false
13513 4404 Acked Read 5 Done Done Sent false
13514 4405 Acked Read 3 Done Done Sent false
13515 4406 Acked Read 1 Done Done Sent false
13516 4407 Acked Read 9 Done Done Sent false
13517 4408 Acked Flush 0 Done Done Sent false
13518 4409 Acked Read 9 Done Done Sent false
13519 4410 Acked Read 9 Done Done Sent false
13520 4411 Acked Write 5 Done Done Sent false
13521 4412 Acked Read 9 Done Done Sent false
13522 4413 Acked Write 9 Done Done Sent false
13523 4414 Acked Read 9 Done Done Sent false
13524 4415 Acked Read 1 Done Done Sent false
13525 4416 Acked Read 5 Done Done Sent false
13526 4417 Acked Write 4 Done Done Sent false
13527 4418 Acked Write 4 Done Done Sent false
13528 4419 Acked Write 6 Done Done Sent false
13529 4420 Acked Flush 0 Done Done Sent false
13530 4421 Acked Read 5 Done Done Sent false
13531 4422 Acked Write 9 Done Done Sent false
13532 4423 Acked Write 1 Done Done Sent false
13533 4424 Acked Read 5 Done Done Sent false
13534 4425 Acked Write 8 Done Done Sent false
13535 4426 Acked Write 10 Done Done Sent false
13536 4427 Acked Write 2 Done Done Sent false
13537 4428 Acked Read 10 Done Done Sent false
13538 4429 Acked Write 2 Done Done Sent false
13539 4430 Acked Read 2 Done Done Sent false
13540 4431 Acked Write 3 Done Done Sent false
13541 4432 Acked Read 4 Done Done Sent false
13542 4433 Acked Write 8 Done Done Sent false
13543 4434 Acked Read 8 Done Done Sent false
13544 4435 Acked Read 6 Done Done Sent false
13545 4436 Acked Read 1 Done Done Sent false
13546 4437 Acked Read 5 Done Done Sent false
13547 4438 Acked Write 5 Done Done Sent false
13548 4439 Acked Write 1 Done Done Sent false
13549 4440 Acked Flush 0 Done Done Sent false
13550 4441 Acked Write 4 Done Done Sent false
13551 4442 Acked Read 6 Done Done Sent false
13552 4443 Acked Read 8 Done Done Sent false
13553 4444 Acked Read 10 Done Done Sent false
13554 4445 Acked Flush 0 Done Done Sent false
13555 4446 Acked Read 10 Done Done Sent false
13556 4447 Acked Read 4 Done Done Sent false
13557 4448 Acked Write 7 Done Done Sent false
13558 4449 Acked Write 4 Done Done Sent false
13559 4450 Acked Read 1 Done Done Sent false
13560 4451 Acked Read 7 Done Done Sent false
13561 4452 Acked Write 8 Done Done Sent false
13562 4453 Acked Write 2 Done Done Sent false
13563 4454 Acked Write 6 Done Done Sent false
13564 4455 Acked Write 5 Done Done Sent false
13565 4456 Acked Read 2 Done Done Sent false
13566 4457 Acked Read 9 Done Done Sent false
13567 4458 Acked Write 7 Done Done Sent false
13568 4459 Acked Read 3 Done Done Sent false
13569 4460 Acked Read 5 Done Done Sent false
13570 4461 Acked Write 6 Done Done Sent false
13571 4462 Acked Read 9 Done Done Sent false
13572 4463 Acked Read 3 Done Done Sent false
13573 4464 Acked Read 8 Done Done Sent false
13574 4465 Acked Write 4 Done Done Sent false
13575 4466 Acked Read 5 Done Done Sent false
13576 4467 Acked Read 3 Done Done Sent false
13577 4468 Acked Write 5 Done Done Sent false
13578 4469 Acked Read 6 Done Done Sent false
13579 4470 Acked Read 8 Done Done Sent false
13580 4471 Acked Write 9 Done Done Sent false
13581 4472 Acked Write 5 Done Done Sent false
13582 4473 Acked Write 1 Done Done Sent false
13583 4474 Acked Read 1 Done Done Sent false
13584 4475 Acked Write 10 Done Done Sent false
13585 4476 Acked Flush 0 Done Done Sent false
13586 4477 Acked Read 6 Done Done Sent false
13587 4478 Acked Write 10 Done Done Sent false
13588 4479 Acked Read 5 Done Done Sent false
13589 4480 Acked Flush 0 Done Done Sent false
13590 4481 Acked Read 2 Done Done Sent false
13591 4482 Acked Write 3 Done Done Sent false
13592 4483 Acked Flush 0 Done Done Sent false
13593 4484 Acked Read 2 Done Done Sent false
13594 4485 Acked Write 7 Done Done Sent false
13595 4486 Acked Write 7 Done Done Sent false
13596 4487 Acked Flush 0 Done Done Sent false
13597 4488 Acked Write 7 Done Done Sent false
13598 4489 Acked Read 4 Done Done Sent false
13599 4490 Acked Flush 0 Done Done Sent false
13600 4491 Acked Write 2 Done Done Sent false
13601 4492 Acked Read 7 Done Done Sent false
13602 4493 Acked Write 5 Done Done Sent false
13603 4494 Acked Write 5 Done Done Sent false
13604 4495 Acked Read 10 Done Done Sent false
13605 4496 Acked Flush 0 Done Done Sent false
13606 4497 Acked Write 6 Done Done Sent false
13607 4498 Acked Write 6 Done Done Sent false
13608 4499 Acked Read 7 Done Done Sent false
13609 4500 Acked Read 1 Done Done Sent false
13610 4501 Acked Read 1 Done Done Sent false
13611 4502 Acked Flush 0 Done Done Sent false
13612 4503 Acked Read 7 Done Done Sent false
13613 4504 Acked Read 4 Done Done Sent false
13614 4505 Acked Write 6 Done Done Sent false
13615 4506 Acked Write 2 Done Done Sent false
13616 4507 Acked Flush 0 Done Done Sent false
13617 4508 Acked Read 8 Done Done Sent false
13618 4509 Acked Write 7 Done Done Sent false
13619 4510 Acked Read 6 Done Done Sent false
13620 4511 Acked Write 8 Done Done Sent false
13621 4512 Acked Write 2 Done Done Sent false
13622 4513 Acked Write 8 Done Done Sent false
13623 4514 Acked Read 8 Done Done Sent false
13624 4515 Acked Read 9 Done Done Sent false
13625 4516 Acked Flush 0 Done Done Sent false
13626 4517 Acked Write 4 Done Done Sent false
13627 4518 Acked Write 2 Done Done Sent false
13628 4519 Acked Write 6 Done Done Sent false
13629 4520 Acked Read 6 Done Done Sent false
13630 4521 Acked Read 7 Done Done Sent false
13631 4522 Acked Write 3 Done Done Sent false
13632 4523 Acked Read 3 Done Done Sent false
13633 4524 Acked Write 3 Done Done Sent false
13634 4525 Acked Read 9 Done Done Sent false
13635 4526 Acked Flush 0 Done Done Sent false
13636 4527 Acked Write 4 Done Done Sent false
13637 4528 Acked Write 8 Done Done Sent false
13638 4529 Acked Read 10 Done Done Sent false
13639 4530 Acked Write 3 Done Done Sent false
13640 4531 Acked Read 5 Done Done Sent false
13641 4532 Acked Flush 0 Done Done Sent false
13642 4533 Acked Write 9 Done Done Sent false
13643 4534 Acked Write 8 Done Done Sent false
13644 4535 Acked Read 7 Done Done Sent false
13645 4536 Acked Read 1 Done Done Sent false
13646 4537 Acked Write 5 Done Done Sent false
13647 4538 Acked Read 9 Done Done Sent false
13648 4539 Acked Write 2 Done Done Sent false
13649 4540 Acked Write 8 Done Done Sent false
13650 4541 Acked Write 10 Done Done Sent false
13651 4542 Acked Flush 0 Done Done Sent false
13652 4543 Acked Write 5 Done Done Sent false
13653 4544 Acked Read 2 Done Done Sent false
13654 4545 Acked Flush 0 Done Done Sent false
13655 4546 Acked Write 10 Done Done Sent false
13656 4547 Acked Flush 0 Done Done Sent false
13657 4548 Acked Read 7 Done Done Sent false
13658 4549 Acked Write 3 Done Done Sent false
13659 4550 Acked Write 3 Done Done Sent false
13660 4551 Acked Read 4 Done Done Sent false
13661 4552 Acked Read 6 Done Done Sent false
13662 4553 Acked Read 8 Done Done Sent false
13663 4554 Acked Read 8 Done Done Sent false
13664 4555 Acked Read 5 Done Done Sent false
13665 4556 Acked Read 7 Done Done Sent false
13666 4557 Acked Write 8 Done Done Sent false
13667 4558 Acked Read 2 Done Done Sent false
13668 4559 Acked Write 4 Done Done Sent false
13669 4560 Acked Read 4 Done Done Sent false
13670 4561 Acked Read 8 Done Done Sent false
13671 4562 Acked Read 8 Done Done Sent false
13672 4563 Acked Read 1 Done Done Sent false
13673 4564 Acked Read 8 Done Done Sent false
13674 4565 Acked Write 10 Done Done Sent false
13675 4566 Acked Read 10 Done Done Sent false
13676 4567 Acked Read 7 Done Done Sent false
13677 4568 Acked Read 10 Done Done Sent false
13678 4569 Acked Write 4 Done Done Sent false
13679 4570 Acked Read 5 Done Done Sent false
13680 4571 Acked Write 5 Done Done Sent false
13681 4572 Acked Write 3 Done Done Sent false
13682 4573 Acked Write 10 Done Done Sent false
13683 4574 Acked Write 9 Done Done Sent false
13684 4575 Acked Read 6 Done Done Sent false
13685 4576 Acked Write 4 Done Done Sent false
13686 4577 Acked Flush 0 Done Done Sent false
13687 4578 Acked Write 3 Done Done Sent false
13688 4579 Acked Write 1 Done Done Sent false
13689 4580 Acked Read 6 Done Done Sent false
13690 4581 Acked Write 3 Done Done Sent false
13691 4582 Acked Read 7 Done Done Sent false
13692 4583 Acked Read 4 Done Done Sent false
13693 4584 Acked Read 10 Done Done Sent false
13694 4585 Acked Write 4 Done Done Sent false
13695 4586 Acked Read 3 Done Done Sent false
13696 4587 Acked Read 10 Done Done Sent false
13697 4588 Acked Write 5 Done Done Sent false
13698 4589 Acked Write 6 Done Done Sent false
13699 4590 Acked Read 10 Done Done Sent false
13700 4591 Acked Read 4 Done Done Sent false
13701 4592 Acked Read 2 Done Done Sent false
13702 4593 Acked Read 2 Done Done Sent false
13703 4594 Acked Read 10 Done Done Sent false
13704 4595 Acked Read 10 Done Done Sent false
13705 4596 Acked Write 9 Done Done Sent false
13706 4597 Acked Read 9 Done Done Sent false
13707 4598 Acked Write 1 Done Done Sent false
13708 4599 Acked Read 8 Done Done Sent false
13709 4600 Acked Flush 0 Done Done Sent false
13710 4601 Acked Write 4 Done Done Sent false
13711 4602 Acked Read 8 Done Done Sent false
13712 4603 Acked Read 1 Done Done Sent false
13713 4604 Acked Flush 0 Done Done Sent false
13714 4605 Acked Read 5 Done Done Sent false
13715 4606 Acked Write 1 Done Done Sent false
13716 4607 Acked Write 4 Done Done Sent false
13717 4608 Acked Read 10 Done Done Sent false
13718 4609 Acked Write 8 Done Done Sent false
13719 4610 Acked Write 3 Done Done Sent false
13720 4611 Acked Read 2 Done Done Sent false
13721 4612 Acked Read 10 Done Done Sent false
13722 4613 Acked Write 6 Done Done Sent false
13723 4614 Acked Read 3 Done Done Sent false
13724 4615 Acked Write 9 Done Done Sent false
13725 4616 Acked Write 2 Done Done Sent false
13726 4617 Acked Flush 0 Done Done Sent false
13727 4618 Acked Write 6 Done Done Sent false
13728 4619 Acked Write 1 Done Done Sent false
13729 4620 Acked Write 8 Done Done Sent false
13730 4621 Acked Read 5 Done Done Sent false
13731 4622 Acked Flush 0 Done Done Sent false
13732 STATES DS:0 DS:1 DS:2 TOTAL
13733 Sent 0 0 302 302
13734 Done 302 302 0 604
13735 Skipped 0 0 0 0
13736 Error 0 0 0 0
13737 Last Flush: 4622 4622 4320
13738 Downstairs last five completed: 4320 4319 4318 4317 4316
13739 Upstairs last five completed: 4622 4621 4620 4619 4618
13740 CLIENT: Up:0 ds:302 act:2
137412025-01-15T05:21:53.674ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137422025-01-15T05:21:53.674ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46368) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137432025-01-15T05:21:53.675ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137442025-01-15T05:21:53.676ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137452025-01-15T05:21:53.676ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137462025-01-15T05:21:53.676ZINFOcrucible: send last flush ID to this DS: 4320 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137472025-01-15T05:21:53.676ZINFOcrucible: Replied this last flush ID: 4320 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137482025-01-15T05:21:53.676ZINFOcrucible: [2] client re-new 302 jobs since flush 4320 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137492025-01-15T05:21:53.676ZINFOcrucible: [2] Replayed 302 jobs since flush: 4320 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
13750 ----------------------------------------------------------------
13751 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13752 Downstairs last five completed: 4622 4621 4620 4619 4618
13753 Upstairs last five completed: 4622 4621 4620 4619 4618
13754 CLIENT: Up:0 ds:0 act:3
13755 Replay: All jobs finished
13756 Main task has work to do, go find it
13757 got dsc Stop(2)
13758 stop 2
13759 [8830] Got stop action so:false kr:false
13760 Jan 15 05:21:53.907 INFO request completed, latency_us: 371, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 23ab1dc4-d005-4cfb-a19b-07ab738075e4, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
137612025-01-15T05:21:53.907ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137622025-01-15T05:21:53.907ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
137632025-01-15T05:21:53.907ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
13764 Jan 15 05:21:53.908 INFO request completed, latency_us:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:21:53.90803381Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
13765 150, response_code: 200, uri: /state/cid/2, method: GET, req_id: e5a9ba95-fb43-4124-80db-2c9194e26df3, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
13766 [8830] Exited with: ExitStatus(unix_wait_status(9))
13767 [8830][2] reports Exit
13768 Jan 15 05:21:57.909 INFO request completed, latency_us: 194, response_code: 200, uri: /state/cid/2, method: GET, req_id: 89a0cd8c-e82b-4675-bc5a-ede7a2c3dacc, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
13769 001/300 Write block 2727 len 24576 data: 2 3 2 2 2 2
13770 002/300 Read block 4350 len 40960
13771 003/300 Read block 221 len 4096
13772 004/300 Write block 1844 len 20480 data: 6 6 5 6 5
13773 005/300 Read block 3916 len 16384
13774 006/300 Flush
13775 007/300 Read block 3195 len 36864
13776 008/300 Write block 2286 len 4096 data: 3
13777 009/300 Read block 4344 len 20480
13778 010/300 Read block 2355 len 12288
13779 011/300 Write block 343 len 12288 data: 4 3 3
13780 012/300 Write block 4964 len 20480 data: 3 3 3 3 3
13781 013/300 Read block 1200 len 20480
13782 014/300 Read block 3231 len 20480
13783 015/300 Write block 3644 len 4096 data: 3
13784 016/300 Flush
13785 017/300 Write block 2822 len 20480 data: 7 6 6 5 5
13786 018/300 Flush
13787 019/300 Write block 1388 len 16384 data: 2 2 2 2
13788 020/300 Read block 4759 len 20480
13789 021/300 Read block 1717 len 8192
13790 022/300 Read block 1414 len 32768
13791 023/300 Flush
13792 024/300 Flush
13793 025/300 Read block 3865 len 4096
13794 026/300 Write block 4633 len 20480 data: 3 3 2 2 2
13795 027/300 Read block 21 len 8192
13796 028/300 Read block 3149 len 4096
13797 029/300 Write block 4302 len 4096 data: 2
13798 030/300 Read block 4910 len 16384
13799 031/300 Read block 565 len 8192
13800 032/300 Write block 1707 len 16384 data: 3 3 3 3
13801 033/300 Write block 4529 len 16384 data: 2 2 2 2
13802 034/300 Write block 1454 len 12288 data: 3 3 4
13803 035/300 Write block 3117 len 16384 data: 3 3 3 4
13804 036/300 Write block 40 len 4096 data: 3
13805 037/300 Read block 3931 len 28672
13806 038/300 Flush
13807 039/300 Write block 2284 len 36864 data: 4 3 4 3 2 2 2 3 3
13808 040/300 Read block 3701 len 20480
13809 041/300 Read block 599 len 20480
13810 042/300 Read block 3301 len 4096
13811 043/300 Write block 1623 len 24576 data: 5 5 4 5 4 4
13812 044/300 Read block 2474 len 24576
13813 045/300 Write block 1323 len 32768 data: 2 3 3 4 4 4 3 2
13814 046/300 Read block 4477 len 40960
13815 047/300 Read block 3914 len 16384
13816 048/300 Read block 552 len 4096
13817 049/300 Read block 915 len 12288
13818 050/300 Write block 4676 len 20480 data: 5 5 3 3 2
13819 051/300 Flush
13820 052/300 Write block 2273 len 24576 data: 3 3 3 3 2 3
13821 053/300 Read block 1639 len 24576
13822 054/300 Write block 172 len 40960 data: 2 2 2 2 2 3 2 2 2 3
13823 055/300 Write block 2394 len 12288 data: 4 3 3
13824 056/300 Read block 2616 len 12288
13825 057/300 Read block 736 len 12288
13826 058/300 Write block 4051 len 16384 data: 6 4 4 3
13827 059/300 Flush
13828 060/300 Write block 3924 len 12288 data: 3 3 3
13829 061/300 Read block 428 len 4096
13830 062/300 Write block 1429 len 4096 data: 4
13831 063/300 Read block 88 len 28672
13832 064/300 Read block 3138 len 8192
13833 065/300 Read block 3632 len 36864
13834 066/300 Read block 4467 len 32768
13835 067/300 Read block 4828 len 12288
13836 068/300 Read block 2171 len 24576
13837 069/300 Read block 1 len 28672
13838 070/300 Read block 2894 len 12288
13839 071/300 Read block 2019 len 36864
13840 072/300 Write block 2058 len 36864 data: 4 3 3 2 2 2 2 2 2
13841 073/300 Read block 4275 len 40960
13842 074/300 Flush
13843 075/300 Write block 4262 len 4096 data: 6
13844 076/300 Write block 1564 len 4096 data: 5
13845 077/300 Write block 196 len 8192 data: 4 4
13846 078/300 Flush
13847 079/300 Read block 1067 len 16384
13848 080/300 Write block 2062 len 32768 data: 3 3 3 3 3 3 3 3
13849 081/300 Read block 4901 len 12288
13850 082/300 Write block 1530 len 28672 data: 4 5 5 4 3 3 2
13851 083/300 Read block 1165 len 40960
13852 084/300 Write block 4189 len 40960 data: 4 4 4 5 6 6 4 4 4 3
13853 085/300 Read block 1187 len 8192
13854 086/300 Read block 1970 len 4096
13855 087/300 Read block 243 len 28672
13856 088/300 Read block 4251 len 40960
13857 089/300 Write block 2009 len 12288 data: 3 3 4
13858 090/300 Write block 4581 len 20480 data: 5 4 3 3 3
13859 091/300 Flush
13860 092/300 Write block 4617 len 28672 data: 3 4 4 4 4 4 4
13861 093/300 Read block 4081 len 36864
13862 094/300 Read block 229 len 8192
13863 095/300 Read block 822 len 24576
13864 096/300 Write block 4445 len 12288 data: 4 3 4
13865 097/300 Write block 1638 len 4096 data: 6
13866 098/300 Read block 1419 len 16384
13867 099/300 Read block 3139 len 20480
13868 100/300 Write block 786 len 28672 data: 3 3 3 4 4 4 3
13869 101/300 Flush
13870 102/300 Write block 820 len 40960 data: 3 3 4 4 5 4 4 3 3 3
13871 103/300 Write block 4067 len 4096 data: 3
13872 104/300 Write block 3725 len 40960 data: 3 4 3 5 5 4 4 3 3 2
13873 105/300 Read block 4042 len 32768
13874 106/300 Read block 3670 len 12288
13875 107/300 Flush
13876 108/300 Flush
13877 109/300 Write block 4702 len 12288 data: 3 3 4
13878 110/300 Write block 2605 len 36864 data: 4 3 2 2 3 3 3 3 3
13879 111/300 Flush
13880 112/300 Read block 2912 len 32768
13881 113/300 Read block 4585 len 28672
13882 114/300 Write block 1457 len 20480 data: 3 3 4 4 4
13883 115/300 Write block 3411 len 24576 data: 3 3 4 4 5 5
13884 116/300 Read block 243 len 12288
13885 117/300 Write block 720 len 32768 data: 2 2 2 2 2 2 2 2
13886 118/300 Write block 3345 len 24576 data: 2 2 2 2 3 3
13887 119/300 Read block 3199 len 32768
13888 120/300 Flush
13889 121/300 Read block 3512 len 4096
13890 122/300 Read block 664 len 20480
13891 123/300 Read block 2697 len 24576
13892 124/300 Write block 746 len 28672 data: 4 4 4 3 4 5 6
13893 125/300 Read block 3770 len 40960
13894 126/300 Write block 2277 len 24576 data: 3 4 3 3 4 4
13895 127/300 Write block 3029 len 40960 data: 5 4 5 5 5 5 6 6 6 5
13896 128/300 Write block 2625 len 24576 data: 4 4 4 4 4 2
13897 129/300 Write block 3747 len 36864 data: 3 3 3 3 3 3 3 4 4
13898 130/300 Read block 4402 len 32768
13899 131/300 Read block 722 len 8192
13900 132/300 Read block 1261 len 8192
13901 133/300 Read block 3802 len 16384
13902 134/300 Read block 1036 len 40960
13903 135/300 Read block 2851 len 12288
13904 136/300 Write block 968 len 12288 data: 4 4 4
13905 137/300 Write block 327 len 8192 data: 5 5
13906 138/300 Write block 476 len 28672 data: 4 4 3 2 2 3 3
13907 139/300 Flush
13908 140/300 Write block 951 len 8192 data: 3 2
13909 141/300 Write block 3673 len 8192 data: 7 7
13910 142/300 Write block 2530 len 36864 data: 3 4 4 5 4 4 3 3 3
13911 143/300 Flush
13912 144/300 Read block 2873 len 32768
13913 145/300 Write block 3112 len 4096 data: 3
13914 146/300 Write block 3910 len 36864 data: 3 3 4 3 4 5 5 6 7
13915 147/300 Read block 2052 len 36864
13916 148/300 Read block 2589 len 24576
13917 149/300 Write block 1571 len 36864 data: 5 6 8 8 8 6 6 6 7
13918 150/300 Write block 970 len 12288 data: 5 2 2
13919 151/300 Read block 3334 len 24576
13920 152/300 Read block 2282 len 36864
13921 153/300 Write block 1207 len 24576 data: 5 5 4 2 2 2
13922 154/300 Read block 3712 len 8192
13923 155/300 Write block 4101 len 16384 data: 2 2 2 3
13924 156/300 Read block 145 len 24576
13925 157/300 Read block 1162 len 8192
13926 158/300 Read block 3065 len 4096
13927 159/300 Read block 3186 len 20480
13928 160/300 Write block 3795 len 12288 data: 5 5 4
13929 161/300 Write block 360 len 36864 data: 4 4 3 3 3 3 3 2 2
13930 162/300 Write block 215 len 4096 data: 3
13931 163/300 Read block 3991 len 12288
13932 164/300 Flush
13933 165/300 Write block 176 len 4096 data: 3
13934 166/300 Write block 25 len 40960 data: 2 2 2 2 2 2 2 2 2 3
13935 167/300 Read block 265 len 16384
13936 168/300 Read block 4903 len 12288
13937 169/300 Read block 4269 len 36864
13938 170/300 Read block 4939 len 28672
13939 171/300 Flush
13940 172/300 Read block 1286 len 32768
13941 173/300 Write block 115 len 28672 data: 4 3 3 3 3 4 3
13942 174/300 Read block 2692 len 8192
13943 175/300 Write block 2499 len 28672 data: 4 3 3 3 4 4 4
13944 176/300 Write block 47 len 28672 data: 3 3 4 4 4 3 3
13945 177/300 Read block 199 len 28672
13946 178/300 Read block 407 len 20480
13947 179/300 Read block 4203 len 16384
13948 180/300 Write block 2654 len 36864 data: 3 4 4 5 5 4 3 3 3
13949 181/300 Read block 1176 len 28672
13950 182/300 Read block 1163 len 4096
13951 183/300 Write block 2911 len 32768 data: 4 3 3 3 3 4 5 6
13952 184/300 Flush
13953 185/300 Write block 2915 len 20480 data: 4 5 6 7 7
13954 186/300 Read block 3435 len 40960
13955 187/300 Write block 691 len 8192 data: 6 5
13956 188/300 Read block 2937 len 36864
13957 189/300 Read block 1573 len 32768
13958 190/300 Read block 2704 len 32768
13959 191/300 Read block 2904 len 12288
13960 192/300 Read block 1214 len 32768
13961 193/300 Write block 4718 len 20480 data: 2 2 2 3 2
13962 194/300 Read block 3652 len 12288
13963 195/300 Flush
13964 196/300 Write block 697 len 8192 data: 5 5
13965 197/300 Write block 1898 len 20480 data: 3 4 5 4 4
13966 198/300 Write block 726 len 36864 data: 3 3 2 3 4 3 3 4 3
13967 199/300 Read block 3865 len 8192
13968 200/300 Write block 1969 len 24576 data: 3 3 4 3 3 2
13969 201/300 Write block 2584 len 40960 data: 3 3 5 5 6 6 7 5 5 5
13970 202/300 Write block 1644 len 40960 data: 5 4 3 3 2 3 2 2 2 2
13971 203/300 Read block 4653 len 28672
13972 204/300 Read block 2000 len 12288
13973 205/300 Flush
13974 206/300 Write block 3145 len 4096 data: 3
13975 207/300 Write block 1843 len 32768 data: 6 7 7 6 7 6 5 4
13976 208/300 Flush
13977 209/300 Read block 2373 len 32768
13978 210/300 Write block 2645 len 24576 data: 3 4 4 3 4 5
13979 211/300 Write block 2863 len 12288 data: 4 4 6
13980 212/300 Write block 218 len 12288 data: 3 3 3
13981 213/300 Read block 1120 len 32768
13982 214/300 Write block 1482 len 20480 data: 3 2 2 3 3
13983 215/300 Read block 2985 len 28672
13984 216/300 Read block 4703 len 8192
13985 217/300 Flush
13986 218/300 Read block 1616 len 28672
13987 219/300 Read block 238 len 40960
13988 220/300 Read block 3025 len 8192
13989 221/300 Write block 2195 len 24576 data: 2 2 2 2 3 3
13990 222/300 Write block 4783 len 16384 data: 3 2 2 2
13991 223/300 Write block 3182 len 28672 data: 6 6 5 4 4 4 3
13992 224/300 Write block 2188 len 8192 data: 4 3
13993 225/300 Read block 764 len 16384
13994 226/300 Write block 3103 len 36864 data: 5 5 4 4 3 3 2 2 2
13995 227/300 Write block 1833 len 36864 data: 4 4 3 2 2 3 3 3 4
13996 228/300 Write block 284 len 24576 data: 3 3 3 3 3 4
13997 229/300 Write block 1274 len 12288 data: 3 2 2
13998 230/300 Read block 81 len 16384
13999 231/300 Read block 916 len 32768
14000 232/300 Read block 3322 len 12288
14001 233/300 Read block 2415 len 16384
14002 234/300 Read block 4337 len 20480
14003 235/300 Write block 3442 len 4096 data: 4
14004 236/300 Read block 3816 len 4096
14005 237/300 Write block 2982 len 36864 data: 6 5 6 6 6 4 2 2 2
14006 238/300 Read block 2779 len 12288
14007 239/300 Read block 2438 len 16384
14008 240/300 Flush
14009 241/300 Write block 58 len 40960 data: 2 2 2 2 2 3 4 4 4 4
14010 242/300 Flush
14011 243/300 Read block 2489 len 20480
14012 244/300 Write block 701 len 12288 data: 3 3 4
14013 245/300 Write block 1388 len 28672 data: 3 3 3 3 2 2 2
14014 246/300 Read block 2686 len 4096
14015 247/300 Write block 1574 len 24576 data: 9 9 7 7 7 8
14016 248/300 Write block 4117 len 16384 data: 6 7 7 7
14017 249/300 Write block 3003 len 28672 data: 4 4 3 3 3 4 4
14018 250/300 Read block 2517 len 16384
14019 251/300 Flush
14020 252/300 Read block 3363 len 16384
14021 253/300 Write block 4240 len 32768 data: 5 7 6 5 5 6 6 4
14022 254/300 Write block 519 len 8192 data: 4 4
14023 255/300 Read block 2469 len 40960
14024 256/300 Read block 3356 len 24576
14025 257/300 Write block 4001 len 40960 data: 3 3 3 4 5 4 3 3 2 2
14026 258/300 Write block 4420 len 12288 data: 3 5 5
14027 259/300 Read block 3613 len 12288
14028 260/300 Write block 2188 len 32768 data: 5 4 3 3 2 2 2 3
14029 261/300 Read block 4115 len 36864
14030 262/300 Write block 2731 len 36864 data: 3 3 2 2 2 2 2 2 2
14031 263/300 Read block 1561 len 24576
14032 264/300 Read block 1390 len 12288
14033 265/300 Read block 1738 len 28672
14034 266/300 Read block 159 len 4096
14035 267/300 Write block 251 len 24576 data: 3 3 3 5 6 6
14036 268/300 Write block 2257 len 20480 data: 3 3 4 4 4
14037 269/300 Write block 1539 len 40960 data: 3 3 3 3 4 4 5 4 4 2
14038 270/300 Write block 2341 len 16384 data: 4 4 4 5
14039 271/300 Write block 1654 len 16384 data: 2 2 2 2
14040 272/300 Flush
14041 273/300 Read block 4742 len 16384
14042 274/300 Write block 1425 len 40960 data: 4 5 5 4 5 5 4 4 5 4
14043 275/300 Read block 4962 len 8192
14044 276/300 Read block 982 len 8192
14045 277/300 Write block 2345 len 16384 data: 5 5 6 7
14046 278/300 Write block 533 len 24576 data: 3 2 2 2 2 2
14047 279/300 Read block 1452 len 40960
14048 280/300 Read block 1942 len 40960
14049 281/300 Write block 2505 len 4096 data: 5
14050 282/300 Read block 494 len 36864
14051 283/300 Write block 327 len 32768 data: 6 6 3 3 3 3 3 3
14052 284/300 Read block 3894 len 12288
14053 285/300 Read block 4587 len 36864
14054 286/300 Read block 357 len 32768
14055 287/300 Read block 2219 len 28672
14056 288/300 Flush
14057 289/300 Write block 4962 len 40960 data: 3 4 4 4 4 4 4 2 2 2
14058 290/300 Read block 2600 len 8192
14059 291/300 Read block 2999 len 8192
14060 292/300 Write block 4387 len 16384 data: 4 3 2 3
14061 293/300 Read block 271 len 8192
14062 294/300 Read block 4685 len 20480
14063 295/300 Write block 4821 len 20480 data: 4 4 4 4 2
14064 296/300 Read block 3263 len 36864
14065 297/300 Read block 1692 len 4096
14066 298/300 Write block 3500 len 24576 data: 3 3 3 3 3 4
14067 299/300 Write block 4770 len 8192 data: 3 4
14068 300/300 Write block 2199 len 8192 data: 4 4
14069 301/300 Write block 1782 len 40960 data: 3 6 6 6 8 8 7 6 6 6
14070 Main task has work to do, go find it
14071 got dsc Start(2)
14072 start 2
14073 Jan 15 05:21:58.133 INFO request completed, latency_us: 185, response_code: 204, uri: /start/cid/2, method: POST, req_id: 187c895a-42e7-4336-86af-8faed6017935, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
14074 Replay: started 2, returned:Ok(())
14075 ----------------------------------------------------------------
14076 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14077 States: Active Active Offline
14078 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14079 4623 Acked Write 6 Done Done Sent false
14080 4624 Acked Read 10 Done Done Sent false
14081 4625 Acked Read 1 Done Done Sent false
14082 4626 Acked Write 5 Done Done Sent false
14083 4627 Acked Read 4 Done Done Sent false
14084 4628 Acked Flush 0 Done Done Sent false
14085 4629 Acked Read 9 Done Done Sent false
14086 4630 Acked Write 1 Done Done Sent false
14087 4631 Acked Read 5 Done Done Sent false
14088 4632 Acked Read 3 Done Done Sent false
14089 4633 Acked Write 3 Done Done Sent false
14090 4634 Acked Write 5 Done Done Sent false
14091 4635 Acked Read 5 Done Done Sent false
14092 4636 Acked Read 5 Done Done Sent false
14093 4637 Acked Write 1 Done Done Sent false
14094 4638 Acked Flush 0 Done Done Sent false
14095 4639 Acked Write 5 Done Done Sent false
14096 4640 Acked Flush 0 Done Done Sent false
14097 4641 Acked Write 4 Done Done Sent false
14098 4642 Acked Read 5 Done Done Sent false
14099 4643 Acked Read 2 Done Done Sent false
14100 4644 Acked Read 8 Done Done Sent false
14101 4645 Acked Flush 0 Done Done Sent false
14102 4646 Acked Flush 0 Done Done Sent false
14103 4647 Acked Read 1 Done Done Sent false
14104 4648 Acked Write 5 Done Done Sent false
14105 4649 Acked Read 2 Done Done Sent false
14106 4650 Acked Read 1 Done Done Sent false
14107 4651 Acked Write 1 Done Done Sent false
14108 4652 Acked Read 4 Done Done Sent false
14109 4653 Acked Read 2 Done Done Sent false
14110 4654 Acked Write 4 Done Done Sent false
14111 4655 Acked Write 4 Done Done Sent false
14112 4656 Acked Write 3 Done Done Sent false
14113 4657 Acked Write 4 Done Done Sent false
14114 4658 Acked Write 1 Done Done Sent false
14115 4659 Acked Read 7 Done Done Sent false
14116 4660 Acked Flush 0 Done Done Sent false
14117 4661 Acked Write 9 Done Done Sent false
14118 4662 Acked Read 5 Done Done Sent false
14119 4663 Acked Read 5 Done Done Sent false
14120 4664 Acked Read 1 Done Done Sent false
14121 4665 Acked Write 6 Done Done Sent false
14122 4666 Acked Read 6 Done Done Sent false
14123 4667 Acked Write 8 Done Done Sent false
14124 4668 Acked Read 10 Done Done Sent false
14125 4669 Acked Read 4 Done Done Sent false
14126 4670 Acked Read 1 Done Done Sent false
14127 4671 Acked Read 3 Done Done Sent false
14128 4672 Acked Write 5 Done Done Sent false
14129 4673 Acked Flush 0 Done Done Sent false
14130 4674 Acked Write 6 Done Done Sent false
14131 4675 Acked Read 6 Done Done Sent false
14132 4676 Acked Write 10 Done Done Sent false
14133 4677 Acked Write 3 Done Done Sent false
14134 4678 Acked Read 3 Done Done Sent false
14135 4679 Acked Read 3 Done Done Sent false
14136 4680 Acked Write 4 Done Done Sent false
14137 4681 Acked Flush 0 Done Done Sent false
14138 4682 Acked Write 3 Done Done Sent false
14139 4683 Acked Read 1 Done Done Sent false
14140 4684 Acked Write 1 Done Done Sent false
14141 4685 Acked Read 7 Done Done Sent false
14142 4686 Acked Read 2 Done Done Sent false
14143 4687 Acked Read 9 Done Done Sent false
14144 4688 Acked Read 8 Done Done Sent false
14145 4689 Acked Read 3 Done Done Sent false
14146 4690 Acked Read 6 Done Done Sent false
14147 4691 Acked Read 7 Done Done Sent false
14148 4692 Acked Read 3 Done Done Sent false
14149 4693 Acked Read 9 Done Done Sent false
14150 4694 Acked Write 9 Done Done Sent false
14151 4695 Acked Read 10 Done Done Sent false
14152 4696 Acked Flush 0 Done Done Sent false
14153 4697 Acked Write 1 Done Done Sent false
14154 4698 Acked Write 1 Done Done Sent false
14155 4699 Acked Write 2 Done Done Sent false
14156 4700 Acked Flush 0 Done Done Sent false
14157 4701 Acked Read 4 Done Done Sent false
14158 4702 Acked Write 8 Done Done Sent false
14159 4703 Acked Read 3 Done Done Sent false
14160 4704 Acked Write 7 Done Done Sent false
14161 4705 Acked Read 10 Done Done Sent false
14162 4706 Acked Write 10 Done Done Sent false
14163 4707 Acked Read 2 Done Done Sent false
14164 4708 Acked Read 1 Done Done Sent false
14165 4709 Acked Read 7 Done Done Sent false
14166 4710 Acked Read 10 Done Done Sent false
14167 4711 Acked Write 3 Done Done Sent false
14168 4712 Acked Write 5 Done Done Sent false
14169 4713 Acked Flush 0 Done Done Sent false
14170 4714 Acked Write 7 Done Done Sent false
14171 4715 Acked Read 9 Done Done Sent false
14172 4716 Acked Read 2 Done Done Sent false
14173 4717 Acked Read 6 Done Done Sent false
14174 4718 Acked Write 3 Done Done Sent false
14175 4719 Acked Write 1 Done Done Sent false
14176 4720 Acked Read 4 Done Done Sent false
14177 4721 Acked Read 5 Done Done Sent false
14178 4722 Acked Write 7 Done Done Sent false
14179 4723 Acked Flush 0 Done Done Sent false
14180 4724 Acked Write 10 Done Done Sent false
14181 4725 Acked Write 1 Done Done Sent false
14182 4726 Acked Write 10 Done Done Sent false
14183 4727 Acked Read 8 Done Done Sent false
14184 4728 Acked Read 3 Done Done Sent false
14185 4729 Acked Flush 0 Done Done Sent false
14186 4730 Acked Flush 0 Done Done Sent false
14187 4731 Acked Write 3 Done Done Sent false
14188 4732 Acked Write 9 Done Done Sent false
14189 4733 Acked Flush 0 Done Done Sent false
14190 4734 Acked Read 8 Done Done Sent false
14191 4735 Acked Read 7 Done Done Sent false
14192 4736 Acked Write 5 Done Done Sent false
14193 4737 Acked Write 6 Done Done Sent false
14194 4738 Acked Read 3 Done Done Sent false
14195 4739 Acked Write 8 Done Done Sent false
14196 4740 Acked Write 6 Done Done Sent false
14197 4741 Acked Read 8 Done Done Sent false
14198 4742 Acked Flush 0 Done Done Sent false
14199 4743 Acked Read 1 Done Done Sent false
14200 4744 Acked Read 5 Done Done Sent false
14201 4745 Acked Read 6 Done Done Sent false
14202 4746 Acked Write 7 Done Done Sent false
14203 4747 Acked Read 10 Done Done Sent false
14204 4748 Acked Write 6 Done Done Sent false
14205 4749 Acked Write 10 Done Done Sent false
14206 4750 Acked Write 6 Done Done Sent false
14207 4751 Acked Write 9 Done Done Sent false
14208 4752 Acked Read 8 Done Done Sent false
14209 4753 Acked Read 2 Done Done Sent false
14210 4754 Acked Read 2 Done Done Sent false
14211 4755 Acked Read 4 Done Done Sent false
14212 4756 Acked Read 10 Done Done Sent false
14213 4757 Acked Read 3 Done Done Sent false
14214 4758 Acked Write 3 Done Done Sent false
14215 4759 Acked Write 2 Done Done Sent false
14216 4760 Acked Write 7 Done Done Sent false
14217 4761 Acked Flush 0 Done Done Sent false
14218 4762 Acked Write 2 Done Done Sent false
14219 4763 Acked Write 2 Done Done Sent false
14220 4764 Acked Write 9 Done Done Sent false
14221 4765 Acked Flush 0 Done Done Sent false
14222 4766 Acked Read 8 Done Done Sent false
14223 4767 Acked Write 1 Done Done Sent false
14224 4768 Acked Write 9 Done Done Sent false
14225 4769 Acked Read 9 Done Done Sent false
14226 4770 Acked Read 6 Done Done Sent false
14227 4771 Acked Write 9 Done Done Sent false
14228 4772 Acked Write 3 Done Done Sent false
14229 4773 Acked Read 6 Done Done Sent false
14230 4774 Acked Read 9 Done Done Sent false
14231 4775 Acked Write 6 Done Done Sent false
14232 4776 Acked Read 2 Done Done Sent false
14233 4777 Acked Write 4 Done Done Sent false
14234 4778 Acked Read 6 Done Done Sent false
14235 4779 Acked Read 2 Done Done Sent false
14236 4780 Acked Read 1 Done Done Sent false
14237 4781 Acked Read 5 Done Done Sent false
14238 4782 Acked Write 3 Done Done Sent false
14239 4783 Acked Write 9 Done Done Sent false
14240 4784 Acked Write 1 Done Done Sent false
14241 4785 Acked Read 3 Done Done Sent false
14242 4786 Acked Flush 0 Done Done Sent false
14243 4787 Acked Write 1 Done Done Sent false
14244 4788 Acked Write 10 Done Done Sent false
14245 4789 Acked Read 4 Done Done Sent false
14246 4790 Acked Read 3 Done Done Sent false
14247 4791 Acked Read 9 Done Done Sent false
14248 4792 Acked Read 7 Done Done Sent false
14249 4793 Acked Flush 0 Done Done Sent false
14250 4794 Acked Read 8 Done Done Sent false
14251 4795 Acked Write 7 Done Done Sent false
14252 4796 Acked Read 2 Done Done Sent false
14253 4797 Acked Write 7 Done Done Sent false
14254 4798 Acked Write 7 Done Done Sent false
14255 4799 Acked Read 7 Done Done Sent false
14256 4800 Acked Read 5 Done Done Sent false
14257 4801 Acked Read 4 Done Done Sent false
14258 4802 Acked Write 9 Done Done Sent false
14259 4803 Acked Read 7 Done Done Sent false
14260 4804 Acked Read 1 Done Done Sent false
14261 4805 Acked Write 8 Done Done Sent false
14262 4806 Acked Flush 0 Done Done Sent false
14263 4807 Acked Write 5 Done Done Sent false
14264 4808 Acked Read 10 Done Done Sent false
14265 4809 Acked Write 2 Done Done Sent false
14266 4810 Acked Read 9 Done Done Sent false
14267 4811 Acked Read 8 Done Done Sent false
14268 4812 Acked Read 8 Done Done Sent false
14269 4813 Acked Read 3 Done Done Sent false
14270 4814 Acked Read 8 Done Done Sent false
14271 4815 Acked Write 5 Done Done Sent false
14272 4816 Acked Read 3 Done Done Sent false
14273 4817 Acked Flush 0 Done Done Sent false
14274 4818 Acked Write 2 Done Done Sent false
14275 4819 Acked Write 5 Done Done Sent false
14276 4820 Acked Write 9 Done Done Sent false
14277 4821 Acked Read 2 Done Done Sent false
14278 4822 Acked Write 6 Done Done Sent false
14279 4823 Acked Write 10 Done Done Sent false
14280 4824 Acked Write 10 Done Done Sent false
14281 4825 Acked Read 7 Done Done Sent false
14282 4826 Acked Read 3 Done Done Sent false
14283 4827 Acked Flush 0 Done Done Sent false
14284 4828 Acked Write 1 Done Done Sent false
14285 4829 Acked Write 8 Done Done Sent false
14286 4830 Acked Flush 0 Done Done Sent false
14287 4831 Acked Read 8 Done Done Sent false
14288 4832 Acked Write 6 Done Done Sent false
14289 4833 Acked Write 3 Done Done Sent false
14290 4834 Acked Write 3 Done Done Sent false
14291 4835 Acked Read 8 Done Done Sent false
14292 4836 Acked Write 5 Done Done Sent false
14293 4837 Acked Read 7 Done Done Sent false
14294 4838 Acked Read 2 Done Done Sent false
14295 4839 Acked Flush 0 Done Done Sent false
14296 4840 Acked Read 7 Done Done Sent false
14297 4841 Acked Read 10 Done Done Sent false
14298 4842 Acked Read 2 Done Done Sent false
14299 4843 Acked Write 6 Done Done Sent false
14300 4844 Acked Write 4 Done Done Sent false
14301 4845 Acked Write 7 Done Done Sent false
14302 4846 Acked Write 2 Done Done Sent false
14303 4847 Acked Read 4 Done Done Sent false
14304 4848 Acked Write 9 Done Done Sent false
14305 4849 Acked Write 9 Done Done Sent false
14306 4850 Acked Write 6 Done Done Sent false
14307 4851 Acked Write 3 Done Done Sent false
14308 4852 Acked Read 4 Done Done Sent false
14309 4853 Acked Read 8 Done Done Sent false
14310 4854 Acked Read 3 Done Done Sent false
14311 4855 Acked Read 4 Done Done Sent false
14312 4856 Acked Read 5 Done Done Sent false
14313 4857 Acked Write 1 Done Done Sent false
14314 4858 Acked Read 1 Done Done Sent false
14315 4859 Acked Write 9 Done Done Sent false
14316 4860 Acked Read 3 Done Done Sent false
14317 4861 Acked Read 4 Done Done Sent false
14318 4862 Acked Flush 0 Done Done Sent false
14319 4863 Acked Write 10 Done Done Sent false
14320 4864 Acked Flush 0 Done Done Sent false
14321 4865 Acked Read 5 Done Done Sent false
14322 4866 Acked Write 3 Done Done Sent false
14323 4867 Acked Write 7 Done Done Sent false
14324 4868 Acked Read 1 Done Done Sent false
14325 4869 Acked Write 6 Done Done Sent false
14326 4870 Acked Write 4 Done Done Sent false
14327 4871 Acked Write 7 Done Done Sent false
14328 4872 Acked Read 4 Done Done Sent false
14329 4873 Acked Flush 0 Done Done Sent false
14330 4874 Acked Read 4 Done Done Sent false
14331 4875 Acked Write 8 Done Done Sent false
14332 4876 Acked Write 2 Done Done Sent false
14333 4877 Acked Read 10 Done Done Sent false
14334 4878 Acked Read 6 Done Done Sent false
14335 4879 Acked Write 10 Done Done Sent false
14336 4880 Acked Write 3 Done Done Sent false
14337 4881 Acked Read 3 Done Done Sent false
14338 4882 Acked Write 8 Done Done Sent false
14339 4883 Acked Read 9 Done Done Sent false
14340 4884 Acked Write 9 Done Done Sent false
14341 4885 Acked Read 6 Done Done Sent false
14342 4886 Acked Read 3 Done Done Sent false
14343 4887 Acked Read 7 Done Done Sent false
14344 4888 Acked Read 1 Done Done Sent false
14345 4889 Acked Write 6 Done Done Sent false
14346 4890 Acked Write 5 Done Done Sent false
14347 4891 Acked Write 10 Done Done Sent false
14348 4892 Acked Write 4 Done Done Sent false
14349 4893 Acked Write 4 Done Done Sent false
14350 4894 Acked Flush 0 Done Done Sent false
14351 4895 Acked Read 4 Done Done Sent false
14352 4896 Acked Write 10 Done Done Sent false
14353 4897 Acked Read 2 Done Done Sent false
14354 4898 Acked Read 2 Done Done Sent false
14355 4899 Acked Write 4 Done Done Sent false
14356 4900 Acked Write 6 Done Done Sent false
14357 4901 Acked Read 10 Done Done Sent false
14358 4902 Acked Read 10 Done Done Sent false
14359 4903 Acked Write 1 Done Done Sent false
14360 4904 Acked Read 9 Done Done Sent false
14361 4905 Acked Write 8 Done Done Sent false
14362 4906 Acked Read 3 Done Done Sent false
14363 4907 Acked Read 9 Done Done Sent false
14364 4908 Acked Read 8 Done Done Sent false
14365 4909 Acked Read 7 Done Done Sent false
14366 4910 Acked Flush 0 Done Done Sent false
14367 4911 Acked Write 10 Done Done Sent false
14368 4912 Acked Read 2 Done Done Sent false
14369 4913 Acked Read 2 Done Done Sent false
14370 4914 Acked Write 4 Done Done Sent false
14371 4915 Acked Read 2 Done Done Sent false
14372 4916 Acked Read 5 Done Done Sent false
14373 4917 Acked Write 5 Done Done Sent false
14374 4918 Acked Read 9 Done Done Sent false
14375 4919 Acked Read 1 Done Done Sent false
14376 4920 Acked Write 6 Done Done Sent false
14377 4921 Acked Write 2 Done Done Sent false
14378 4922 Acked Write 2 Done Done Sent false
14379 4923 Acked Write 10 Sent Sent Sent false
14380 STATES DS:0 DS:1 DS:2 TOTAL
14381 Sent 1 1 301 303
14382 Done 300 300 0 600
14383 Skipped 0 0 0 0
14384 Error 0 0 0 0
14385 Last Flush: 4910 4910 4622
14386 Downstairs last five completed: 4622 4621 4620 4619 4618
14387 Upstairs last five completed: 4923 4922 4921 4920 4919
14388 CLIENT: Up:0 ds:301 act:2
14389 ----------------------------------------------------------------
14390 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14391 States: Active Active Offline
14392 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14393 4623 Acked Write 6 Done Done Sent false
14394 4624 Acked Read 10 Done Done Sent false
14395 4625 Acked Read 1 Done Done Sent false
14396 4626 Acked Write 5 Done Done Sent false
14397 4627 Acked Read 4 Done Done Sent false
14398 4628 Acked Flush 0 Done Done Sent false
14399 4629 Acked Read 9 Done Done Sent false
14400 4630 Acked Write 1 Done Done Sent false
14401 4631 Acked Read 5 Done Done Sent false
14402 4632 Acked Read 3 Done Done Sent false
14403 4633 Acked Write 3 Done Done Sent false
14404 4634 Acked Write 5 Done Done Sent false
14405 4635 Acked Read 5 Done Done Sent false
14406 4636 Acked Read 5 Done Done Sent false
14407 4637 Acked Write 1 Done Done Sent false
14408 4638 Acked Flush 0 Done Done Sent false
14409 4639 Acked Write 5 Done Done Sent false
14410 4640 Acked Flush 0 Done Done Sent false
14411 4641 Acked Write 4 Done Done Sent false
14412 4642 Acked Read 5 Done Done Sent false
14413 4643 Acked Read 2 Done Done Sent false
14414 4644 Acked Read 8 Done Done Sent false
14415 4645 Acked Flush 0 Done Done Sent false
14416 4646 Acked Flush 0 Done Done Sent false
14417 4647 Acked Read 1 Done Done Sent false
14418 4648 Acked Write 5 Done Done Sent false
14419 4649 Acked Read 2 Done Done Sent false
14420 4650 Acked Read 1 Done Done Sent false
14421 4651 Acked Write 1 Done Done Sent false
14422 4652 Acked Read 4 Done Done Sent false
14423 4653 Acked Read 2 Done Done Sent false
14424 4654 Acked Write 4 Done Done Sent false
14425 4655 Acked Write 4 Done Done Sent false
14426 4656 Acked Write 3 Done Done Sent false
14427 4657 Acked Write 4 Done Done Sent false
14428 4658 Acked Write 1 Done Done Sent false
14429 4659 Acked Read 7 Done Done Sent false
14430 4660 Acked Flush 0 Done Done Sent false
14431 4661 Acked Write 9 Done Done Sent false
14432 4662 Acked Read 5 Done Done Sent false
14433 4663 Acked Read 5 Done Done Sent false
14434 4664 Acked Read 1 Done Done Sent false
14435 4665 Acked Write 6 Done Done Sent false
14436 4666 Acked Read 6 Done Done Sent false
14437 4667 Acked Write 8 Done Done Sent false
14438 4668 Acked Read 10 Done Done Sent false
14439 4669 Acked Read 4 Done Done Sent false
14440 4670 Acked Read 1 Done Done Sent false
14441 4671 Acked Read 3 Done Done Sent false
14442 4672 Acked Write 5 Done Done Sent false
14443 4673 Acked Flush 0 Done Done Sent false
14444 4674 Acked Write 6 Done Done Sent false
14445 4675 Acked Read 6 Done Done Sent false
14446 4676 Acked Write 10 Done Done Sent false
14447 4677 Acked Write 3 Done Done Sent false
14448 4678 Acked Read 3 Done Done Sent false
14449 4679 Acked Read 3 Done Done Sent false
14450 4680 Acked Write 4 Done Done Sent false
14451 4681 Acked Flush 0 Done Done Sent false
14452 4682 Acked Write 3 Done Done Sent false
14453 4683 Acked Read 1 Done Done Sent false
14454 4684 Acked Write 1 Done Done Sent false
14455 4685 Acked Read 7 Done Done Sent false
14456 4686 Acked Read 2 Done Done Sent false
14457 4687 Acked Read 9 Done Done Sent false
14458 4688 Acked Read 8 Done Done Sent false
14459 4689 Acked Read 3 Done Done Sent false
14460 4690 Acked Read 6 Done Done Sent false
14461 4691 Acked Read 7 Done Done Sent false
14462 4692 Acked Read 3 Done Done Sent false
14463 4693 Acked Read 9 Done Done Sent false
14464 4694 Acked Write 9 Done Done Sent false
14465 4695 Acked Read 10 Done Done Sent false
14466 4696 Acked Flush 0 Done Done Sent false
14467 4697 Acked Write 1 Done Done Sent false
14468 4698 Acked Write 1 Done Done Sent false
14469 4699 Acked Write 2 Done Done Sent false
14470 4700 Acked Flush 0 Done Done Sent false
14471 4701 Acked Read 4 Done Done Sent false
14472 4702 Acked Write 8 Done Done Sent false
14473 4703 Acked Read 3 Done Done Sent false
14474 4704 Acked Write 7 Done Done Sent false
14475 4705 Acked Read 10 Done Done Sent false
14476 4706 Acked Write 10 Done Done Sent false
14477 4707 Acked Read 2 Done Done Sent false
14478 4708 Acked Read 1 Done Done Sent false
14479 4709 Acked Read 7 Done Done Sent false
14480 4710 Acked Read 10 Done Done Sent false
14481 4711 Acked Write 3 Done Done Sent false
14482 4712 Acked Write 5 Done Done Sent false
14483 4713 Acked Flush 0 Done Done Sent false
14484 4714 Acked Write 7 Done Done Sent false
14485 4715 Acked Read 9 Done Done Sent false
14486 4716 Acked Read 2 Done Done Sent false
14487 4717 Acked Read 6 Done Done Sent false
14488 4718 Acked Write 3 Done Done Sent false
14489 4719 Acked Write 1 Done Done Sent false
14490 4720 Acked Read 4 Done Done Sent false
14491 4721 Acked Read 5 Done Done Sent false
14492 4722 Acked Write 7 Done Done Sent false
14493 4723 Acked Flush 0 Done Done Sent false
14494 4724 Acked Write 10 Done Done Sent false
14495 4725 Acked Write 1 Done Done Sent false
14496 4726 Acked Write 10 Done Done Sent false
14497 4727 Acked Read 8 Done Done Sent false
14498 4728 Acked Read 3 Done Done Sent false
14499 4729 Acked Flush 0 Done Done Sent false
14500 4730 Acked Flush 0 Done Done Sent false
14501 4731 Acked Write 3 Done Done Sent false
14502 4732 Acked Write 9 Done Done Sent false
14503 4733 Acked Flush 0 Done Done Sent false
14504 4734 Acked Read 8 Done Done Sent false
14505 4735 Acked Read 7 Done Done Sent false
14506 4736 Acked Write 5 Done Done Sent false
14507 4737 Acked Write 6 Done Done Sent false
14508 4738 Acked Read 3 Done Done Sent false
14509 4739 Acked Write 8 Done Done Sent false
14510 4740 Acked Write 6 Done Done Sent false
14511 4741 Acked Read 8 Done Done Sent false
14512 4742 Acked Flush 0 Done Done Sent false
14513 4743 Acked Read 1 Done Done Sent false
14514 4744 Acked Read 5 Done Done Sent false
14515 4745 Acked Read 6 Done Done Sent false
14516 4746 Acked Write 7 Done Done Sent false
14517 4747 Acked Read 10 Done Done Sent false
14518 4748 Acked Write 6 Done Done Sent false
14519 4749 Acked Write 10 Done Done Sent false
14520 4750 Acked Write 6 Done Done Sent false
14521 4751 Acked Write 9 Done Done Sent false
14522 4752 Acked Read 8 Done Done Sent false
14523 4753 Acked Read 2 Done Done Sent false
14524 4754 Acked Read 2 Done Done Sent false
14525 4755 Acked Read 4 Done Done Sent false
14526 4756 Acked Read 10 Done Done Sent false
14527 4757 Acked Read 3 Done Done Sent false
14528 4758 Acked Write 3 Done Done Sent false
14529 4759 Acked Write 2 Done Done Sent false
14530 4760 Acked Write 7 Done Done Sent false
14531 4761 Acked Flush 0 Done Done Sent false
14532 4762 Acked Write 2 Done Done Sent false
14533 4763 Acked Write 2 Done Done Sent false
14534 4764 Acked Write 9 Done Done Sent false
14535 4765 Acked Flush 0 Done Done Sent false
14536 4766 Acked Read 8 Done Done Sent false
14537 4767 Acked Write 1 Done Done Sent false
14538 4768 Acked Write 9 Done Done Sent false
14539 4769 Acked Read 9 Done Done Sent false
14540 4770 Acked Read 6 Done Done Sent false
14541 4771 Acked Write 9 Done Done Sent false
14542 4772 Acked Write 3 Done Done Sent false
14543 4773 Acked Read 6 Done Done Sent false
14544 4774 Acked Read 9 Done Done Sent false
14545 4775 Acked Write 6 Done Done Sent false
14546 4776 Acked Read 2 Done Done Sent false
14547 4777 Acked Write 4 Done Done Sent false
14548 4778 Acked Read 6 Done Done Sent false
14549 4779 Acked Read 2 Done Done Sent false
14550 4780 Acked Read 1 Done Done Sent false
14551 4781 Acked Read 5 Done Done Sent false
14552 4782 Acked Write 3 Done Done Sent false
14553 4783 Acked Write 9 Done Done Sent false
14554 4784 Acked Write 1 Done Done Sent false
14555 4785 Acked Read 3 Done Done Sent false
14556 4786 Acked Flush 0 Done Done Sent false
14557 4787 Acked Write 1 Done Done Sent false
14558 4788 Acked Write 10 Done Done Sent false
14559 4789 Acked Read 4 Done Done Sent false
14560 4790 Acked Read 3 Done Done Sent false
14561 4791 Acked Read 9 Done Done Sent false
14562 4792 Acked Read 7 Done Done Sent false
14563 4793 Acked Flush 0 Done Done Sent false
14564 4794 Acked Read 8 Done Done Sent false
14565 4795 Acked Write 7 Done Done Sent false
14566 4796 Acked Read 2 Done Done Sent false
14567 4797 Acked Write 7 Done Done Sent false
14568 4798 Acked Write 7 Done Done Sent false
14569 4799 Acked Read 7 Done Done Sent false
14570 4800 Acked Read 5 Done Done Sent false
14571 4801 Acked Read 4 Done Done Sent false
14572 4802 Acked Write 9 Done Done Sent false
14573 4803 Acked Read 7 Done Done Sent false
14574 4804 Acked Read 1 Done Done Sent false
14575 4805 Acked Write 8 Done Done Sent false
14576 4806 Acked Flush 0 Done Done Sent false
14577 4807 Acked Write 5 Done Done Sent false
14578 4808 Acked Read 10 Done Done Sent false
14579 4809 Acked Write 2 Done Done Sent false
14580 4810 Acked Read 9 Done Done Sent false
14581 4811 Acked Read 8 Done Done Sent false
14582 4812 Acked Read 8 Done Done Sent false
14583 4813 Acked Read 3 Done Done Sent false
14584 4814 Acked Read 8 Done Done Sent false
14585 4815 Acked Write 5 Done Done Sent false
14586 4816 Acked Read 3 Done Done Sent false
14587 4817 Acked Flush 0 Done Done Sent false
14588 4818 Acked Write 2 Done Done Sent false
14589 4819 Acked Write 5 Done Done Sent false
14590 4820 Acked Write 9 Done Done Sent false
14591 4821 Acked Read 2 Done Done Sent false
14592 4822 Acked Write 6 Done Done Sent false
14593 4823 Acked Write 10 Done Done Sent false
14594 4824 Acked Write 10 Done Done Sent false
14595 4825 Acked Read 7 Done Done Sent false
14596 4826 Acked Read 3 Done Done Sent false
14597 4827 Acked Flush 0 Done Done Sent false
14598 4828 Acked Write 1 Done Done Sent false
14599 4829 Acked Write 8 Done Done Sent false
14600 4830 Acked Flush 0 Done Done Sent false
14601 4831 Acked Read 8 Done Done Sent false
14602 4832 Acked Write 6 Done Done Sent false
14603 4833 Acked Write 3 Done Done Sent false
14604 4834 Acked Write 3 Done Done Sent false
14605 4835 Acked Read 8 Done Done Sent false
14606 4836 Acked Write 5 Done Done Sent false
14607 4837 Acked Read 7 Done Done Sent false
14608 4838 Acked Read 2 Done Done Sent false
14609 4839 Acked Flush 0 Done Done Sent false
14610 4840 Acked Read 7 Done Done Sent false
14611 4841 Acked Read 10 Done Done Sent false
14612 4842 Acked Read 2 Done Done Sent false
14613 4843 Acked Write 6 Done Done Sent false
14614 4844 Acked Write 4 Done Done Sent false
14615 4845 Acked Write 7 Done Done Sent false
14616 4846 Acked Write 2 Done Done Sent false
14617 4847 Acked Read 4 Done Done Sent false
14618 4848 Acked Write 9 Done Done Sent false
14619 4849 Acked Write 9 Done Done Sent false
14620 4850 Acked Write 6 Done Done Sent false
14621 4851 Acked Write 3 Done Done Sent false
14622 4852 Acked Read 4 Done Done Sent false
14623 4853 Acked Read 8 Done Done Sent false
14624 4854 Acked Read 3 Done Done Sent false
14625 4855 Acked Read 4 Done Done Sent false
14626 4856 Acked Read 5 Done Done Sent false
14627 4857 Acked Write 1 Done Done Sent false
14628 4858 Acked Read 1 Done Done Sent false
14629 4859 Acked Write 9 Done Done Sent false
14630 4860 Acked Read 3 Done Done Sent false
14631 4861 Acked Read 4 Done Done Sent false
14632 4862 Acked Flush 0 Done Done Sent false
14633 4863 Acked Write 10 Done Done Sent false
14634 4864 Acked Flush 0 Done Done Sent false
14635 4865 Acked Read 5 Done Done Sent false
14636 4866 Acked Write 3 Done Done Sent false
14637 4867 Acked Write 7 Done Done Sent false
14638 4868 Acked Read 1 Done Done Sent false
14639 4869 Acked Write 6 Done Done Sent false
14640 4870 Acked Write 4 Done Done Sent false
14641 4871 Acked Write 7 Done Done Sent false
14642 4872 Acked Read 4 Done Done Sent false
14643 4873 Acked Flush 0 Done Done Sent false
14644 4874 Acked Read 4 Done Done Sent false
14645 4875 Acked Write 8 Done Done Sent false
14646 4876 Acked Write 2 Done Done Sent false
14647 4877 Acked Read 10 Done Done Sent false
14648 4878 Acked Read 6 Done Done Sent false
14649 4879 Acked Write 10 Done Done Sent false
14650 4880 Acked Write 3 Done Done Sent false
14651 4881 Acked Read 3 Done Done Sent false
14652 4882 Acked Write 8 Done Done Sent false
14653 4883 Acked Read 9 Done Done Sent false
14654 4884 Acked Write 9 Done Done Sent false
14655 4885 Acked Read 6 Done Done Sent false
14656 4886 Acked Read 3 Done Done Sent false
14657 4887 Acked Read 7 Done Done Sent false
14658 4888 Acked Read 1 Done Done Sent false
14659 4889 Acked Write 6 Done Done Sent false
14660 4890 Acked Write 5 Done Done Sent false
14661 4891 Acked Write 10 Done Done Sent false
14662 4892 Acked Write 4 Done Done Sent false
14663 4893 Acked Write 4 Done Done Sent false
14664 4894 Acked Flush 0 Done Done Sent false
14665 4895 Acked Read 4 Done Done Sent false
14666 4896 Acked Write 10 Done Done Sent false
14667 4897 Acked Read 2 Done Done Sent false
14668 4898 Acked Read 2 Done Done Sent false
14669 4899 Acked Write 4 Done Done Sent false
14670 4900 Acked Write 6 Done Done Sent false
14671 4901 Acked Read 10 Done Done Sent false
14672 4902 Acked Read 10 Done Done Sent false
14673 4903 Acked Write 1 Done Done Sent false
14674 4904 Acked Read 9 Done Done Sent false
14675 4905 Acked Write 8 Done Done Sent false
14676 4906 Acked Read 3 Done Done Sent false
14677 4907 Acked Read 9 Done Done Sent false
14678 4908 Acked Read 8 Done Done Sent false
14679 4909 Acked Read 7 Done Done Sent false
14680 4910 Acked Flush 0 Done Done Sent false
14681 4911 Acked Write 10 Done Done Sent false
14682 4912 Acked Read 2 Done Done Sent false
14683 4913 Acked Read 2 Done Done Sent false
14684 4914 Acked Write 4 Done Done Sent false
14685 4915 Acked Read 2 Done Done Sent false
14686 4916 Acked Read 5 Done Done Sent false
14687 4917 Acked Write 5 Done Done Sent false
14688 4918 Acked Read 9 Done Done Sent false
14689 4919 Acked Read 1 Done Done Sent false
14690 4920 Acked Write 6 Done Done Sent false
14691 4921 Acked Write 2 Done Done Sent false
14692 4922 Acked Write 2 Done Done Sent false
14693 4923 Acked Write 10 Done Done Sent false
14694 4924 Acked Flush 0 Done Done Sent false
14695 STATES DS:0 DS:1 DS:2 TOTAL
14696 Sent 0 0 302 302
14697 Done 302 302 0 604
14698 Skipped 0 0 0 0
14699 Error 0 0 0 0
14700 Last Flush: 4924 4924 4622
14701 Downstairs last five completed: 4622 4621 4620 4619 4618
14702 Upstairs last five completed: 4924 4923 4922 4921 4920
14703 CLIENT: Up:0 ds:302 act:2
147042025-01-15T05:22:03.908ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
147052025-01-15T05:22:03.908ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
147062025-01-15T05:22:03.908ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
147072025-01-15T05:22:03.908ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
147082025-01-15T05:22:03.908ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
14709 ----------------------------------------------------------------
14710 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14711 States: Active Active Offline
14712 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14713 4623 Acked Write 6 Done Done Sent false
14714 4624 Acked Read 10 Done Done Sent false
14715 4625 Acked Read 1 Done Done Sent false
14716 4626 Acked Write 5 Done Done Sent false
14717 4627 Acked Read 4 Done Done Sent false
14718 4628 Acked Flush 0 Done Done Sent false
14719 4629 Acked Read 9 Done Done Sent false
14720 4630 Acked Write 1 Done Done Sent false
14721 4631 Acked Read 5 Done Done Sent false
14722 4632 Acked Read 3 Done Done Sent false
14723 4633 Acked Write 3 Done Done Sent false
14724 4634 Acked Write 5 Done Done Sent false
14725 4635 Acked Read 5 Done Done Sent false
14726 4636 Acked Read 5 Done Done Sent false
14727 4637 Acked Write 1 Done Done Sent false
14728 4638 Acked Flush 0 Done Done Sent false
14729 4639 Acked Write 5 Done Done Sent false
14730 4640 Acked Flush 0 Done Done Sent false
14731 4641 Acked Write 4 Done Done Sent false
14732 4642 Acked Read 5 Done Done Sent false
14733 4643 Acked Read 2 Done Done Sent false
14734 4644 Acked Read 8 Done Done Sent false
14735 4645 Acked Flush 0 Done Done Sent false
14736 4646 Acked Flush 0 Done Done Sent false
14737 4647 Acked Read 1 Done Done Sent false
14738 4648 Acked Write 5 Done Done Sent false
14739 4649 Acked Read 2 Done Done Sent false
14740 4650 Acked Read 1 Done Done Sent false
14741 4651 Acked Write 1 Done Done Sent false
14742 4652 Acked Read 4 Done Done Sent false
14743 4653 Acked Read 2 Done Done Sent false
14744 4654 Acked Write 4 Done Done Sent false
14745 4655 Acked Write 4 Done Done Sent false
14746 4656 Acked Write 3 Done Done Sent false
14747 4657 Acked Write 4 Done Done Sent false
14748 4658 Acked Write 1 Done Done Sent false
14749 4659 Acked Read 7 Done Done Sent false
14750 4660 Acked Flush 0 Done Done Sent false
14751 4661 Acked Write 9 Done Done Sent false
14752 4662 Acked Read 5 Done Done Sent false
14753 4663 Acked Read 5 Done Done Sent false
14754 4664 Acked Read 1 Done Done Sent false
14755 4665 Acked Write 6 Done Done Sent false
14756 4666 Acked Read 6 Done Done Sent false
14757 4667 Acked Write 8 Done Done Sent false
14758 4668 Acked Read 10 Done Done Sent false
14759 4669 Acked Read 4 Done Done Sent false
14760 4670 Acked Read 1 Done Done Sent false
14761 4671 Acked Read 3 Done Done Sent false
14762 4672 Acked Write 5 Done Done Sent false
14763 4673 Acked Flush 0 Done Done Sent false
14764 4674 Acked Write 6 Done Done Sent false
14765 4675 Acked Read 6 Done Done Sent false
14766 4676 Acked Write 10 Done Done Sent false
14767 4677 Acked Write 3 Done Done Sent false
14768 4678 Acked Read 3 Done Done Sent false
14769 4679 Acked Read 3 Done Done Sent false
14770 4680 Acked Write 4 Done Done Sent false
14771 4681 Acked Flush 0 Done Done Sent false
14772 4682 Acked Write 3 Done Done Sent false
14773 4683 Acked Read 1 Done Done Sent false
14774 4684 Acked Write 1 Done Done Sent false
14775 4685 Acked Read 7 Done Done Sent false
14776 4686 Acked Read 2 Done Done Sent false
14777 4687 Acked Read 9 Done Done Sent false
14778 4688 Acked Read 8 Done Done Sent false
14779 4689 Acked Read 3 Done Done Sent false
14780 4690 Acked Read 6 Done Done Sent false
14781 4691 Acked Read 7 Done Done Sent false
14782 4692 Acked Read 3 Done Done Sent false
14783 4693 Acked Read 9 Done Done Sent false
14784 4694 Acked Write 9 Done Done Sent false
14785 4695 Acked Read 10 Done Done Sent false
14786 4696 Acked Flush 0 Done Done Sent false
14787 4697 Acked Write 1 Done Done Sent false
14788 4698 Acked Write 1 Done Done Sent false
14789 4699 Acked Write 2 Done Done Sent false
14790 4700 Acked Flush 0 Done Done Sent false
14791 4701 Acked Read 4 Done Done Sent false
14792 4702 Acked Write 8 Done Done Sent false
14793 4703 Acked Read 3 Done Done Sent false
14794 4704 Acked Write 7 Done Done Sent false
14795 4705 Acked Read 10 Done Done Sent false
14796 4706 Acked Write 10 Done Done Sent false
14797 4707 Acked Read 2 Done Done Sent false
14798 4708 Acked Read 1 Done Done Sent false
14799 4709 Acked Read 7 Done Done Sent false
14800 4710 Acked Read 10 Done Done Sent false
14801 4711 Acked Write 3 Done Done Sent false
14802 4712 Acked Write 5 Done Done Sent false
14803 4713 Acked Flush 0 Done Done Sent false
14804 4714 Acked Write 7 Done Done Sent false
14805 4715 Acked Read 9 Done Done Sent false
14806 4716 Acked Read 2 Done Done Sent false
14807 4717 Acked Read 6 Done Done Sent false
14808 4718 Acked Write 3 Done Done Sent false
14809 4719 Acked Write 1 Done Done Sent false
14810 4720 Acked Read 4 Done Done Sent false
14811 4721 Acked Read 5 Done Done Sent false
14812 4722 Acked Write 7 Done Done Sent false
14813 4723 Acked Flush 0 Done Done Sent false
14814 4724 Acked Write 10 Done Done Sent false
14815 4725 Acked Write 1 Done Done Sent false
14816 4726 Acked Write 10 Done Done Sent false
14817 4727 Acked Read 8 Done Done Sent false
14818 4728 Acked Read 3 Done Done Sent false
14819 4729 Acked Flush 0 Done Done Sent false
14820 4730 Acked Flush 0 Done Done Sent false
14821 4731 Acked Write 3 Done Done Sent false
14822 4732 Acked Write 9 Done Done Sent false
14823 4733 Acked Flush 0 Done Done Sent false
14824 4734 Acked Read 8 Done Done Sent false
14825 4735 Acked Read 7 Done Done Sent false
14826 4736 Acked Write 5 Done Done Sent false
14827 4737 Acked Write 6 Done Done Sent false
14828 4738 Acked Read 3 Done Done Sent false
14829 4739 Acked Write 8 Done Done Sent false
14830 4740 Acked Write 6 Done Done Sent false
14831 4741 Acked Read 8 Done Done Sent false
14832 4742 Acked Flush 0 Done Done Sent false
14833 4743 Acked Read 1 Done Done Sent false
14834 4744 Acked Read 5 Done Done Sent false
14835 4745 Acked Read 6 Done Done Sent false
14836 4746 Acked Write 7 Done Done Sent false
14837 4747 Acked Read 10 Done Done Sent false
14838 4748 Acked Write 6 Done Done Sent false
14839 4749 Acked Write 10 Done Done Sent false
14840 4750 Acked Write 6 Done Done Sent false
14841 4751 Acked Write 9 Done Done Sent false
14842 4752 Acked Read 8 Done Done Sent false
14843 4753 Acked Read 2 Done Done Sent false
14844 4754 Acked Read 2 Done Done Sent false
14845 4755 Acked Read 4 Done Done Sent false
14846 4756 Acked Read 10 Done Done Sent false
14847 4757 Acked Read 3 Done Done Sent false
14848 4758 Acked Write 3 Done Done Sent false
14849 4759 Acked Write 2 Done Done Sent false
14850 4760 Acked Write 7 Done Done Sent false
14851 4761 Acked Flush 0 Done Done Sent false
14852 4762 Acked Write 2 Done Done Sent false
14853 4763 Acked Write 2 Done Done Sent false
14854 4764 Acked Write 9 Done Done Sent false
14855 4765 Acked Flush 0 Done Done Sent false
14856 4766 Acked Read 8 Done Done Sent false
14857 4767 Acked Write 1 Done Done Sent false
14858 4768 Acked Write 9 Done Done Sent false
14859 4769 Acked Read 9 Done Done Sent false
14860 4770 Acked Read 6 Done Done Sent false
14861 4771 Acked Write 9 Done Done Sent false
14862 4772 Acked Write 3 Done Done Sent false
14863 4773 Acked Read 6 Done Done Sent false
14864 4774 Acked Read 9 Done Done Sent false
14865 4775 Acked Write 6 Done Done Sent false
14866 4776 Acked Read 2 Done Done Sent false
14867 4777 Acked Write 4 Done Done Sent false
14868 4778 Acked Read 6 Done Done Sent false
14869 4779 Acked Read 2 Done Done Sent false
14870 4780 Acked Read 1 Done Done Sent false
14871 4781 Acked Read 5 Done Done Sent false
14872 4782 Acked Write 3 Done Done Sent false
14873 4783 Acked Write 9 Done Done Sent false
14874 4784 Acked Write 1 Done Done Sent false
14875 4785 Acked Read 3 Done Done Sent false
14876 4786 Acked Flush 0 Done Done Sent false
14877 4787 Acked Write 1 Done Done Sent false
14878 4788 Acked Write 10 Done Done Sent false
14879 4789 Acked Read 4 Done Done Sent false
14880 4790 Acked Read 3 Done Done Sent false
14881 4791 Acked Read 9 Done Done Sent false
14882 4792 Acked Read 7 Done Done Sent false
14883 4793 Acked Flush 0 Done Done Sent false
14884 4794 Acked Read 8 Done Done Sent false
14885 4795 Acked Write 7 Done Done Sent false
14886 4796 Acked Read 2 Done Done Sent false
14887 4797 Acked Write 7 Done Done Sent false
14888 4798 Acked Write 7 Done Done Sent false
14889 4799 Acked Read 7 Done Done Sent false
14890 4800 Acked Read 5 Done Done Sent false
14891 4801 Acked Read 4 Done Done Sent false
14892 4802 Acked Write 9 Done Done Sent false
14893 4803 Acked Read 7 Done Done Sent false
14894 4804 Acked Read 1 Done Done Sent false
14895 4805 Acked Write 8 Done Done Sent false
14896 4806 Acked Flush 0 Done Done Sent false
14897 4807 Acked Write 5 Done Done Sent false
14898 4808 Acked Read 10 Done Done Sent false
14899 4809 Acked Write 2 Done Done Sent false
14900 4810 Acked Read 9 Done Done Sent false
14901 4811 Acked Read 8 Done Done Sent false
14902 4812 Acked Read 8 Done Done Sent false
14903 4813 Acked Read 3 Done Done Sent false
14904 4814 Acked Read 8 Done Done Sent false
14905 4815 Acked Write 5 Done Done Sent false
14906 4816 Acked Read 3 Done Done Sent false
14907 4817 Acked Flush 0 Done Done Sent false
14908 4818 Acked Write 2 Done Done Sent false
14909 4819 Acked Write 5 Done Done Sent false
14910 4820 Acked Write 9 Done Done Sent false
14911 4821 Acked Read 2 Done Done Sent false
14912 4822 Acked Write 6 Done Done Sent false
14913 4823 Acked Write 10 Done Done Sent false
14914 4824 Acked Write 10 Done Done Sent false
14915 4825 Acked Read 7 Done Done Sent false
14916 4826 Acked Read 3 Done Done Sent false
14917 4827 Acked Flush 0 Done Done Sent false
14918 4828 Acked Write 1 Done Done Sent false
14919 4829 Acked Write 8 Done Done Sent false
14920 4830 Acked Flush 0 Done Done Sent false
14921 4831 Acked Read 8 Done Done Sent false
14922 4832 Acked Write 6 Done Done Sent false
14923 4833 Acked Write 3 Done Done Sent false
14924 4834 Acked Write 3 Done Done Sent false
14925 4835 Acked Read 8 Done Done Sent false
14926 4836 Acked Write 5 Done Done Sent false
14927 4837 Acked Read 7 Done Done Sent false
14928 4838 Acked Read 2 Done Done Sent false
14929 4839 Acked Flush 0 Done Done Sent false
14930 4840 Acked Read 7 Done Done Sent false
14931 4841 Acked Read 10 Done Done Sent false
14932 4842 Acked Read 2 Done Done Sent false
14933 4843 Acked Write 6 Done Done Sent false
14934 4844 Acked Write 4 Done Done Sent false
14935 4845 Acked Write 7 Done Done Sent false
14936 4846 Acked Write 2 Done Done Sent false
14937 4847 Acked Read 4 Done Done Sent false
14938 4848 Acked Write 9 Done Done Sent false
14939 4849 Acked Write 9 Done Done Sent false
14940 4850 Acked Write 6 Done Done Sent false
14941 4851 Acked Write 3 Done Done Sent false
14942 4852 Acked Read 4 Done Done Sent false
14943 4853 Acked Read 8 Done Done Sent false
14944 4854 Acked Read 3 Done Done Sent false
14945 4855 Acked Read 4 Done Done Sent false
14946 4856 Acked Read 5 Done Done Sent false
14947 4857 Acked Write 1 Done Done Sent false
14948 4858 Acked Read 1 Done Done Sent false
14949 4859 Acked Write 9 Done Done Sent false
14950 4860 Acked Read 3 Done Done Sent false
14951 4861 Acked Read 4 Done Done Sent false
14952 4862 Acked Flush 0 Done Done Sent false
14953 4863 Acked Write 10 Done Done Sent false
14954 4864 Acked Flush 0 Done Done Sent false
14955 4865 Acked Read 5 Done Done Sent false
14956 4866 Acked Write 3 Done Done Sent false
14957 4867 Acked Write 7 Done Done Sent false
14958 4868 Acked Read 1 Done Done Sent false
14959 4869 Acked Write 6 Done Done Sent false
14960 4870 Acked Write 4 Done Done Sent false
14961 4871 Acked Write 7 Done Done Sent false
14962 4872 Acked Read 4 Done Done Sent false
14963 4873 Acked Flush 0 Done Done Sent false
14964 4874 Acked Read 4 Done Done Sent false
14965 4875 Acked Write 8 Done Done Sent false
14966 4876 Acked Write 2 Done Done Sent false
14967 4877 Acked Read 10 Done Done Sent false
14968 4878 Acked Read 6 Done Done Sent false
14969 4879 Acked Write 10 Done Done Sent false
14970 4880 Acked Write 3 Done Done Sent false
14971 4881 Acked Read 3 Done Done Sent false
14972 4882 Acked Write 8 Done Done Sent false
14973 4883 Acked Read 9 Done Done Sent false
14974 4884 Acked Write 9 Done Done Sent false
14975 4885 Acked Read 6 Done Done Sent false
14976 4886 Acked Read 3 Done Done Sent false
14977 4887 Acked Read 7 Done Done Sent false
14978 4888 Acked Read 1 Done Done Sent false
14979 4889 Acked Write 6 Done Done Sent false
14980 4890 Acked Write 5 Done Done Sent false
14981 4891 Acked Write 10 Done Done Sent false
14982 4892 Acked Write 4 Done Done Sent false
14983 4893 Acked Write 4 Done Done Sent false
14984 4894 Acked Flush 0 Done Done Sent false
14985 4895 Acked Read 4 Done Done Sent false
14986 4896 Acked Write 10 Done Done Sent false
14987 4897 Acked Read 2 Done Done Sent false
14988 4898 Acked Read 2 Done Done Sent false
14989 4899 Acked Write 4 Done Done Sent false
14990 4900 Acked Write 6 Done Done Sent false
14991 4901 Acked Read 10 Done Done Sent false
14992 4902 Acked Read 10 Done Done Sent false
14993 4903 Acked Write 1 Done Done Sent false
14994 4904 Acked Read 9 Done Done Sent false
14995 4905 Acked Write 8 Done Done Sent false
14996 4906 Acked Read 3 Done Done Sent false
14997 4907 Acked Read 9 Done Done Sent false
14998 4908 Acked Read 8 Done Done Sent false
14999 4909 Acked Read 7 Done Done Sent false
15000 4910 Acked Flush 0 Done Done Sent false
15001 4911 Acked Write 10 Done Done Sent false
15002 4912 Acked Read 2 Done Done Sent false
15003 4913 Acked Read 2 Done Done Sent false
15004 4914 Acked Write 4 Done Done Sent false
15005 4915 Acked Read 2 Done Done Sent false
15006 4916 Acked Read 5 Done Done Sent false
15007 4917 Acked Write 5 Done Done Sent false
15008 4918 Acked Read 9 Done Done Sent false
15009 4919 Acked Read 1 Done Done Sent false
15010 4920 Acked Write 6 Done Done Sent false
15011 4921 Acked Write 2 Done Done Sent false
15012 4922 Acked Write 2 Done Done Sent false
15013 4923 Acked Write 10 Done Done Sent false
15014 4924 Acked Flush 0 Done Done Sent false
15015 STATES DS:0 DS:1 DS:2 TOTAL
15016 Sent 0 0 302 302
15017 Done 302 302 0 604
15018 Skipped 0 0 0 0
15019 Error 0 0 0 0
15020 Last Flush: 4924 4924 4622
15021 Downstairs last five completed: 4622 4621 4620 4619 4618
15022 Upstairs last five completed: 4924 4923 4922 4921 4920
15023 CLIENT: Up:0 ds:302 act:2
15024 [8830] got start action
15025 [8830] I am going to restart
15026 Starting downstairs at port 8830
15027 Make output file at "/tmp/dsc/downstairs-8830.txt"
15028 [8830][2] reports Starting
15029 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1215)
15030 [8830][2] reports Running
15031 ----------------------------------------------------------------
15032 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15033 States: Active Active Offline
15034 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15035 4623 Acked Write 6 Done Done Sent false
15036 4624 Acked Read 10 Done Done Sent false
15037 4625 Acked Read 1 Done Done Sent false
15038 4626 Acked Write 5 Done Done Sent false
15039 4627 Acked Read 4 Done Done Sent false
15040 4628 Acked Flush 0 Done Done Sent false
15041 4629 Acked Read 9 Done Done Sent false
15042 4630 Acked Write 1 Done Done Sent false
15043 4631 Acked Read 5 Done Done Sent false
15044 4632 Acked Read 3 Done Done Sent false
15045 4633 Acked Write 3 Done Done Sent false
15046 4634 Acked Write 5 Done Done Sent false
15047 4635 Acked Read 5 Done Done Sent false
15048 4636 Acked Read 5 Done Done Sent false
15049 4637 Acked Write 1 Done Done Sent false
15050 4638 Acked Flush 0 Done Done Sent false
15051 4639 Acked Write 5 Done Done Sent false
15052 4640 Acked Flush 0 Done Done Sent false
15053 4641 Acked Write 4 Done Done Sent false
15054 4642 Acked Read 5 Done Done Sent false
15055 4643 Acked Read 2 Done Done Sent false
15056 4644 Acked Read 8 Done Done Sent false
15057 4645 Acked Flush 0 Done Done Sent false
15058 4646 Acked Flush 0 Done Done Sent false
15059 4647 Acked Read 1 Done Done Sent false
15060 4648 Acked Write 5 Done Done Sent false
15061 4649 Acked Read 2 Done Done Sent false
15062 4650 Acked Read 1 Done Done Sent false
15063 4651 Acked Write 1 Done Done Sent false
15064 4652 Acked Read 4 Done Done Sent false
15065 4653 Acked Read 2 Done Done Sent false
15066 4654 Acked Write 4 Done Done Sent false
15067 4655 Acked Write 4 Done Done Sent false
15068 4656 Acked Write 3 Done Done Sent false
15069 4657 Acked Write 4 Done Done Sent false
15070 4658 Acked Write 1 Done Done Sent false
15071 4659 Acked Read 7 Done Done Sent false
15072 4660 Acked Flush 0 Done Done Sent false
15073 4661 Acked Write 9 Done Done Sent false
15074 4662 Acked Read 5 Done Done Sent false
15075 4663 Acked Read 5 Done Done Sent false
15076 4664 Acked Read 1 Done Done Sent false
15077 4665 Acked Write 6 Done Done Sent false
15078 4666 Acked Read 6 Done Done Sent false
15079 4667 Acked Write 8 Done Done Sent false
15080 4668 Acked Read 10 Done Done Sent false
15081 4669 Acked Read 4 Done Done Sent false
15082 4670 Acked Read 1 Done Done Sent false
15083 4671 Acked Read 3 Done Done Sent false
15084 4672 Acked Write 5 Done Done Sent false
15085 4673 Acked Flush 0 Done Done Sent false
15086 4674 Acked Write 6 Done Done Sent false
15087 4675 Acked Read 6 Done Done Sent false
15088 4676 Acked Write 10 Done Done Sent false
15089 4677 Acked Write 3 Done Done Sent false
15090 4678 Acked Read 3 Done Done Sent false
15091 4679 Acked Read 3 Done Done Sent false
15092 4680 Acked Write 4 Done Done Sent false
15093 4681 Acked Flush 0 Done Done Sent false
15094 4682 Acked Write 3 Done Done Sent false
15095 4683 Acked Read 1 Done Done Sent false
15096 4684 Acked Write 1 Done Done Sent false
15097 4685 Acked Read 7 Done Done Sent false
15098 4686 Acked Read 2 Done Done Sent false
15099 4687 Acked Read 9 Done Done Sent false
15100 4688 Acked Read 8 Done Done Sent false
15101 4689 Acked Read 3 Done Done Sent false
15102 4690 Acked Read 6 Done Done Sent false
15103 4691 Acked Read 7 Done Done Sent false
15104 4692 Acked Read 3 Done Done Sent false
15105 4693 Acked Read 9 Done Done Sent false
15106 4694 Acked Write 9 Done Done Sent false
15107 4695 Acked Read 10 Done Done Sent false
15108 4696 Acked Flush 0 Done Done Sent false
15109 4697 Acked Write 1 Done Done Sent false
15110 4698 Acked Write 1 Done Done Sent false
15111 4699 Acked Write 2 Done Done Sent false
15112 4700 Acked Flush 0 Done Done Sent false
15113 4701 Acked Read 4 Done Done Sent false
15114 4702 Acked Write 8 Done Done Sent false
15115 4703 Acked Read 3 Done Done Sent false
15116 4704 Acked Write 7 Done Done Sent false
15117 4705 Acked Read 10 Done Done Sent false
15118 4706 Acked Write 10 Done Done Sent false
15119 4707 Acked Read 2 Done Done Sent false
15120 4708 Acked Read 1 Done Done Sent false
15121 4709 Acked Read 7 Done Done Sent false
15122 4710 Acked Read 10 Done Done Sent false
15123 4711 Acked Write 3 Done Done Sent false
15124 4712 Acked Write 5 Done Done Sent false
15125 4713 Acked Flush 0 Done Done Sent false
15126 4714 Acked Write 7 Done Done Sent false
15127 4715 Acked Read 9 Done Done Sent false
15128 4716 Acked Read 2 Done Done Sent false
15129 4717 Acked Read 6 Done Done Sent false
15130 4718 Acked Write 3 Done Done Sent false
15131 4719 Acked Write 1 Done Done Sent false
15132 4720 Acked Read 4 Done Done Sent false
15133 4721 Acked Read 5 Done Done Sent false
15134 4722 Acked Write 7 Done Done Sent false
15135 4723 Acked Flush 0 Done Done Sent false
15136 4724 Acked Write 10 Done Done Sent false
15137 4725 Acked Write 1 Done Done Sent false
15138 4726 Acked Write 10 Done Done Sent false
15139 4727 Acked Read 8 Done Done Sent false
15140 4728 Acked Read 3 Done Done Sent false
15141 4729 Acked Flush 0 Done Done Sent false
15142 4730 Acked Flush 0 Done Done Sent false
15143 4731 Acked Write 3 Done Done Sent false
15144 4732 Acked Write 9 Done Done Sent false
15145 4733 Acked Flush 0 Done Done Sent false
15146 4734 Acked Read 8 Done Done Sent false
15147 4735 Acked Read 7 Done Done Sent false
15148 4736 Acked Write 5 Done Done Sent false
15149 4737 Acked Write 6 Done Done Sent false
15150 4738 Acked Read 3 Done Done Sent false
15151 4739 Acked Write 8 Done Done Sent false
15152 4740 Acked Write 6 Done Done Sent false
15153 4741 Acked Read 8 Done Done Sent false
15154 4742 Acked Flush 0 Done Done Sent false
15155 4743 Acked Read 1 Done Done Sent false
15156 4744 Acked Read 5 Done Done Sent false
15157 4745 Acked Read 6 Done Done Sent false
15158 4746 Acked Write 7 Done Done Sent false
15159 4747 Acked Read 10 Done Done Sent false
15160 4748 Acked Write 6 Done Done Sent false
15161 4749 Acked Write 10 Done Done Sent false
15162 4750 Acked Write 6 Done Done Sent false
15163 4751 Acked Write 9 Done Done Sent false
15164 4752 Acked Read 8 Done Done Sent false
15165 4753 Acked Read 2 Done Done Sent false
15166 4754 Acked Read 2 Done Done Sent false
15167 4755 Acked Read 4 Done Done Sent false
15168 4756 Acked Read 10 Done Done Sent false
15169 4757 Acked Read 3 Done Done Sent false
15170 4758 Acked Write 3 Done Done Sent false
15171 4759 Acked Write 2 Done Done Sent false
15172 4760 Acked Write 7 Done Done Sent false
15173 4761 Acked Flush 0 Done Done Sent false
15174 4762 Acked Write 2 Done Done Sent false
15175 4763 Acked Write 2 Done Done Sent false
15176 4764 Acked Write 9 Done Done Sent false
15177 4765 Acked Flush 0 Done Done Sent false
15178 4766 Acked Read 8 Done Done Sent false
15179 4767 Acked Write 1 Done Done Sent false
15180 4768 Acked Write 9 Done Done Sent false
15181 4769 Acked Read 9 Done Done Sent false
15182 4770 Acked Read 6 Done Done Sent false
15183 4771 Acked Write 9 Done Done Sent false
15184 4772 Acked Write 3 Done Done Sent false
15185 4773 Acked Read 6 Done Done Sent false
15186 4774 Acked Read 9 Done Done Sent false
15187 4775 Acked Write 6 Done Done Sent false
15188 4776 Acked Read 2 Done Done Sent false
15189 4777 Acked Write 4 Done Done Sent false
15190 4778 Acked Read 6 Done Done Sent false
15191 4779 Acked Read 2 Done Done Sent false
15192 4780 Acked Read 1 Done Done Sent false
15193 4781 Acked Read 5 Done Done Sent false
15194 4782 Acked Write 3 Done Done Sent false
15195 4783 Acked Write 9 Done Done Sent false
15196 4784 Acked Write 1 Done Done Sent false
15197 4785 Acked Read 3 Done Done Sent false
15198 4786 Acked Flush 0 Done Done Sent false
15199 4787 Acked Write 1 Done Done Sent false
15200 4788 Acked Write 10 Done Done Sent false
15201 4789 Acked Read 4 Done Done Sent false
15202 4790 Acked Read 3 Done Done Sent false
15203 4791 Acked Read 9 Done Done Sent false
15204 4792 Acked Read 7 Done Done Sent false
15205 4793 Acked Flush 0 Done Done Sent false
15206 4794 Acked Read 8 Done Done Sent false
15207 4795 Acked Write 7 Done Done Sent false
15208 4796 Acked Read 2 Done Done Sent false
15209 4797 Acked Write 7 Done Done Sent false
15210 4798 Acked Write 7 Done Done Sent false
15211 4799 Acked Read 7 Done Done Sent false
15212 4800 Acked Read 5 Done Done Sent false
15213 4801 Acked Read 4 Done Done Sent false
15214 4802 Acked Write 9 Done Done Sent false
15215 4803 Acked Read 7 Done Done Sent false
15216 4804 Acked Read 1 Done Done Sent false
15217 4805 Acked Write 8 Done Done Sent false
15218 4806 Acked Flush 0 Done Done Sent false
15219 4807 Acked Write 5 Done Done Sent false
15220 4808 Acked Read 10 Done Done Sent false
15221 4809 Acked Write 2 Done Done Sent false
15222 4810 Acked Read 9 Done Done Sent false
15223 4811 Acked Read 8 Done Done Sent false
15224 4812 Acked Read 8 Done Done Sent false
15225 4813 Acked Read 3 Done Done Sent false
15226 4814 Acked Read 8 Done Done Sent false
15227 4815 Acked Write 5 Done Done Sent false
15228 4816 Acked Read 3 Done Done Sent false
15229 4817 Acked Flush 0 Done Done Sent false
15230 4818 Acked Write 2 Done Done Sent false
15231 4819 Acked Write 5 Done Done Sent false
15232 4820 Acked Write 9 Done Done Sent false
15233 4821 Acked Read 2 Done Done Sent false
15234 4822 Acked Write 6 Done Done Sent false
15235 4823 Acked Write 10 Done Done Sent false
15236 4824 Acked Write 10 Done Done Sent false
15237 4825 Acked Read 7 Done Done Sent false
15238 4826 Acked Read 3 Done Done Sent false
15239 4827 Acked Flush 0 Done Done Sent false
15240 4828 Acked Write 1 Done Done Sent false
15241 4829 Acked Write 8 Done Done Sent false
15242 4830 Acked Flush 0 Done Done Sent false
15243 4831 Acked Read 8 Done Done Sent false
15244 4832 Acked Write 6 Done Done Sent false
15245 4833 Acked Write 3 Done Done Sent false
15246 4834 Acked Write 3 Done Done Sent false
15247 4835 Acked Read 8 Done Done Sent false
15248 4836 Acked Write 5 Done Done Sent false
15249 4837 Acked Read 7 Done Done Sent false
15250 4838 Acked Read 2 Done Done Sent false
15251 4839 Acked Flush 0 Done Done Sent false
15252 4840 Acked Read 7 Done Done Sent false
15253 4841 Acked Read 10 Done Done Sent false
15254 4842 Acked Read 2 Done Done Sent false
15255 4843 Acked Write 6 Done Done Sent false
15256 4844 Acked Write 4 Done Done Sent false
15257 4845 Acked Write 7 Done Done Sent false
15258 4846 Acked Write 2 Done Done Sent false
15259 4847 Acked Read 4 Done Done Sent false
15260 4848 Acked Write 9 Done Done Sent false
15261 4849 Acked Write 9 Done Done Sent false
15262 4850 Acked Write 6 Done Done Sent false
15263 4851 Acked Write 3 Done Done Sent false
15264 4852 Acked Read 4 Done Done Sent false
15265 4853 Acked Read 8 Done Done Sent false
15266 4854 Acked Read 3 Done Done Sent false
15267 4855 Acked Read 4 Done Done Sent false
15268 4856 Acked Read 5 Done Done Sent false
15269 4857 Acked Write 1 Done Done Sent false
15270 4858 Acked Read 1 Done Done Sent false
15271 4859 Acked Write 9 Done Done Sent false
15272 4860 Acked Read 3 Done Done Sent false
15273 4861 Acked Read 4 Done Done Sent false
15274 4862 Acked Flush 0 Done Done Sent false
15275 4863 Acked Write 10 Done Done Sent false
15276 4864 Acked Flush 0 Done Done Sent false
15277 4865 Acked Read 5 Done Done Sent false
15278 4866 Acked Write 3 Done Done Sent false
15279 4867 Acked Write 7 Done Done Sent false
15280 4868 Acked Read 1 Done Done Sent false
15281 4869 Acked Write 6 Done Done Sent false
15282 4870 Acked Write 4 Done Done Sent false
15283 4871 Acked Write 7 Done Done Sent false
15284 4872 Acked Read 4 Done Done Sent false
15285 4873 Acked Flush 0 Done Done Sent false
15286 4874 Acked Read 4 Done Done Sent false
15287 4875 Acked Write 8 Done Done Sent false
15288 4876 Acked Write 2 Done Done Sent false
15289 4877 Acked Read 10 Done Done Sent false
15290 4878 Acked Read 6 Done Done Sent false
15291 4879 Acked Write 10 Done Done Sent false
15292 4880 Acked Write 3 Done Done Sent false
15293 4881 Acked Read 3 Done Done Sent false
15294 4882 Acked Write 8 Done Done Sent false
15295 4883 Acked Read 9 Done Done Sent false
15296 4884 Acked Write 9 Done Done Sent false
15297 4885 Acked Read 6 Done Done Sent false
15298 4886 Acked Read 3 Done Done Sent false
15299 4887 Acked Read 7 Done Done Sent false
15300 4888 Acked Read 1 Done Done Sent false
15301 4889 Acked Write 6 Done Done Sent false
15302 4890 Acked Write 5 Done Done Sent false
15303 4891 Acked Write 10 Done Done Sent false
15304 4892 Acked Write 4 Done Done Sent false
15305 4893 Acked Write 4 Done Done Sent false
15306 4894 Acked Flush 0 Done Done Sent false
15307 4895 Acked Read 4 Done Done Sent false
15308 4896 Acked Write 10 Done Done Sent false
15309 4897 Acked Read 2 Done Done Sent false
15310 4898 Acked Read 2 Done Done Sent false
15311 4899 Acked Write 4 Done Done Sent false
15312 4900 Acked Write 6 Done Done Sent false
15313 4901 Acked Read 10 Done Done Sent false
15314 4902 Acked Read 10 Done Done Sent false
15315 4903 Acked Write 1 Done Done Sent false
15316 4904 Acked Read 9 Done Done Sent false
15317 4905 Acked Write 8 Done Done Sent false
15318 4906 Acked Read 3 Done Done Sent false
15319 4907 Acked Read 9 Done Done Sent false
15320 4908 Acked Read 8 Done Done Sent false
15321 4909 Acked Read 7 Done Done Sent false
15322 4910 Acked Flush 0 Done Done Sent false
15323 4911 Acked Write 10 Done Done Sent false
15324 4912 Acked Read 2 Done Done Sent false
15325 4913 Acked Read 2 Done Done Sent false
15326 4914 Acked Write 4 Done Done Sent false
15327 4915 Acked Read 2 Done Done Sent false
15328 4916 Acked Read 5 Done Done Sent false
15329 4917 Acked Write 5 Done Done Sent false
15330 4918 Acked Read 9 Done Done Sent false
15331 4919 Acked Read 1 Done Done Sent false
15332 4920 Acked Write 6 Done Done Sent false
15333 4921 Acked Write 2 Done Done Sent false
15334 4922 Acked Write 2 Done Done Sent false
15335 4923 Acked Write 10 Done Done Sent false
15336 4924 Acked Flush 0 Done Done Sent false
15337 STATES DS:0 DS:1 DS:2 TOTAL
15338 Sent 0 0 302 302
15339 Done 302 302 0 604
15340 Skipped 0 0 0 0
15341 Error 0 0 0 0
15342 Last Flush: 4924 4924 4622
15343 Downstairs last five completed: 4622 4621 4620 4619 4618
15344 Upstairs last five completed: 4924 4923 4922 4921 4920
15345 CLIENT: Up:0 ds:302 act:2
153462025-01-15T05:22:13.909ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153472025-01-15T05:22:13.909ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42212) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153482025-01-15T05:22:13.910ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153492025-01-15T05:22:13.911ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153502025-01-15T05:22:13.911ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153512025-01-15T05:22:13.911ZINFOcrucible: send last flush ID to this DS: 4622 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153522025-01-15T05:22:13.911ZINFOcrucible: Replied this last flush ID: 4622 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153532025-01-15T05:22:13.911ZINFOcrucible: [2] client re-new 302 jobs since flush 4622 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153542025-01-15T05:22:13.912ZINFOcrucible: [2] Replayed 302 jobs since flush: 4622 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
15355 ----------------------------------------------------------------
15356 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15357 Downstairs last five completed: 4924 4923 4922 4921 4920
15358 Upstairs last five completed: 4924 4923 4922 4921 4920
15359 CLIENT: Up:0 ds:0 act:3
15360 Replay: All jobs finished
15361 Main task has work to do, go find it
15362 got dsc Stop(1)
15363 stop 1
15364 [8820] Got stop action so:false kr:false
15365 Jan 15 05:22:14.142 INFO request completed, latency_us: 312, response_code: 204, uri: /stop/cid/1, method: POST, req_id: b0247bdb-742f-40a7-94f4-7f5afecb91ca, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
153662025-01-15T05:22:14.143ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153672025-01-15T05:22:14.143ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153682025-01-15T05:22:14.143ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
153692025-01-15T05:22:14.143ZINFOcrucible: client task is exiting = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
15370 Jan 15 05:22:14.143 INFO request completed, latency_us: 140, response_code: 200, uri: /state/cid/1, method: GET, req_id: 8cb61712-aade-422e-8767-46c557953e0f, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
15371 [8820] Exited with: ExitStatus(unix_wait_status(9))
15372 [8820][1] reports Exit
15373 Jan 15 05:22:18.145 INFO request completed, latency_us: 183, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3687e363-8451-4509-84aa-47d0d0314735, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
15374 001/300 Write block 3901 len 20480 data: 4 3 2 3 3
15375 002/300 Read block 3806 len 4096
15376 003/300 Read block 3315 len 16384
15377 004/300 Read block 4545 len 12288
15378 005/300 Read block 4518 len 24576
15379 006/300 Flush
15380 007/300 Flush
15381 008/300 Read block 2032 len 12288
15382 009/300 Write block 1603 len 32768 data: 5 4 4 5 4 4 3 3
15383 010/300 Read block 3674 len 4096
15384 011/300 Write block 558 len 32768 data: 4 3 3 3 2 3 3 3
15385 012/300 Read block 3545 len 4096
15386 013/300 Read block 4196 len 16384
15387 014/300 Write block 4614 len 16384 data: 5 4 4 4
15388 015/300 Flush
15389 016/300 Read block 2760 len 16384
15390 017/300 Read block 1750 len 8192
15391 018/300 Read block 4116 len 20480
15392 019/300 Write block 3090 len 40960 data: 4 4 5 4 4 4 3 3 4 5
15393 020/300 Write block 314 len 24576 data: 2 2 2 2 2 3
15394 021/300 Read block 4614 len 4096
15395 022/300 Write block 3497 len 36864 data: 3 4 3 4 4 4 4 4 5
15396 023/300 Read block 4890 len 20480
15397 024/300 Write block 2794 len 16384 data: 3 3 3 3
15398 025/300 Read block 3205 len 40960
15399 026/300 Write block 25 len 16384 data: 3 3 3 3
15400 027/300 Write block 1119 len 16384 data: 3 3 3 3
15401 028/300 Write block 4120 len 8192 data: 8 6
15402 029/300 Write block 467 len 4096 data: 4
15403 030/300 Write block 1924 len 40960 data: 5 3 3 3 3 3 4 3 4 4
15404 031/300 Flush
15405 032/300 Write block 3801 len 40960 data: 3 3 2 2 2 2 2 2 2 2
15406 033/300 Read block 589 len 12288
15407 034/300 Write block 2681 len 36864 data: 2 2 2 2 2 2 2 2 2
15408 035/300 Read block 1932 len 20480
15409 036/300 Read block 2753 len 20480
15410 037/300 Write block 2841 len 12288 data: 3 3 2
15411 038/300 Write block 3364 len 28672 data: 4 4 4 3 3 3 4
15412 039/300 Read block 4812 len 16384
15413 040/300 Read block 3050 len 32768
15414 041/300 Read block 4130 len 24576
15415 042/300 Write block 1244 len 20480 data: 3 4 3 4 3
15416 043/300 Read block 860 len 4096
15417 044/300 Write block 4381 len 12288 data: 6 6 5
15418 045/300 Read block 1280 len 12288
15419 046/300 Flush
15420 047/300 Read block 538 len 8192
15421 048/300 Flush
15422 049/300 Write block 2280 len 12288 data: 4 5 5
15423 050/300 Write block 4152 len 24576 data: 3 3 3 3 3 3
15424 051/300 Read block 4747 len 12288
15425 052/300 Write block 3615 len 40960 data: 3 3 3 3 4 3 2 2 2 2
15426 053/300 Write block 3936 len 24576 data: 3 4 5 6 4 4
15427 054/300 Read block 971 len 4096
15428 055/300 Flush
15429 056/300 Read block 3641 len 16384
15430 057/300 Write block 4660 len 8192 data: 3 3
15431 058/300 Write block 4154 len 28672 data: 4 4 4 4 3 4 4
15432 059/300 Read block 3028 len 28672
15433 060/300 Flush
15434 061/300 Write block 2817 len 20480 data: 2 2 3 4 4
15435 062/300 Read block 126 len 36864
15436 063/300 Read block 4765 len 12288
15437 064/300 Write block 63 len 28672 data: 4 5 5 5 5 4 4
15438 065/300 Read block 645 len 24576
15439 066/300 Read block 1270 len 8192
15440 067/300 Write block 257 len 4096 data: 6
15441 068/300 Write block 2181 len 24576 data: 3 6 6 5 5 5
15442 069/300 Flush
15443 070/300 Flush
15444 071/300 Write block 2397 len 12288 data: 2 2 2
15445 072/300 Write block 1845 len 24576 data: 8 7 8 7 6 5
15446 073/300 Write block 2007 len 4096 data: 2
15447 074/300 Write block 138 len 16384 data: 4 4 3 2
15448 075/300 Read block 4457 len 4096
15449 076/300 Write block 2890 len 36864 data: 3 2 3 3 3 4 4 4 4
15450 077/300 Read block 3471 len 20480
15451 078/300 Read block 2762 len 32768
15452 079/300 Read block 3652 len 8192
15453 080/300 Read block 3062 len 24576
15454 081/300 Write block 753 len 36864 data: 5 4 3 3 3 4 3 3 3
15455 082/300 Read block 2540 len 32768
15456 083/300 Read block 3423 len 36864
15457 084/300 Write block 263 len 36864 data: 4 4 4 5 4 4 4 4 5
15458 085/300 Read block 244 len 20480
15459 086/300 Write block 3332 len 32768 data: 2 2 4 5 5 5 5 4
15460 087/300 Flush
15461 088/300 Write block 2364 len 28672 data: 5 6 8 6 7 6 6
15462 089/300 Write block 3784 len 40960 data: 4 4 3 3 2 3 4 4 4 3
15463 090/300 Write block 4314 len 40960 data: 2 2 3 3 3 4 4 4 3 3
15464 091/300 Read block 2301 len 40960
15465 092/300 Write block 4696 len 12288 data: 4 3 3
15466 093/300 Write block 3120 len 36864 data: 5 3 3 3 4 3 3 4 5
15467 094/300 Write block 2255 len 20480 data: 4 4 4 4 5
15468 095/300 Write block 520 len 40960 data: 5 3 3 3 3 3 3 3 3 4
15469 096/300 Write block 4329 len 24576 data: 4 4 4 4 3 3
15470 097/300 Write block 2680 len 16384 data: 4 3 3 3
15471 098/300 Read block 4857 len 16384
15472 099/300 Read block 4123 len 20480
15473 100/300 Read block 2295 len 20480
15474 101/300 Write block 4205 len 36864 data: 3 3 3 3 4 4 4 3 3
15475 102/300 Read block 3588 len 8192
15476 103/300 Write block 1378 len 40960 data: 2 2 3 3 2 2 2 3 3 2
15477 104/300 Write block 2974 len 36864 data: 3 3 4 4 4 4 4 4 7
15478 105/300 Read block 3802 len 36864
15479 106/300 Write block 549 len 16384 data: 3 3 3 4
15480 107/300 Read block 2524 len 24576
15481 108/300 Write block 1576 len 20480 data: 8 8 8 9 5
15482 109/300 Read block 4567 len 8192
15483 110/300 Read block 2020 len 20480
15484 111/300 Flush
15485 112/300 Read block 1314 len 40960
15486 113/300 Write block 915 len 4096 data: 2
15487 114/300 Flush
15488 115/300 Write block 1637 len 32768 data: 5 7 6 7 7 5 5 6
15489 116/300 Write block 4815 len 36864 data: 4 4 6 5 5 5 5 5 5
15490 117/300 Read block 4953 len 24576
15491 118/300 Read block 2969 len 20480
15492 119/300 Read block 4187 len 28672
15493 120/300 Write block 3365 len 12288 data: 5 5 4
15494 121/300 Write block 3810 len 32768 data: 3 2 2 2 3 4 5 4
15495 122/300 Write block 2941 len 36864 data: 3 3 3 3 3 4 4 4 3
15496 123/300 Write block 1157 len 40960 data: 4 3 3 3 3 3 4 4 3 3
15497 124/300 Read block 1274 len 4096
15498 125/300 Read block 704 len 20480
15499 126/300 Read block 1275 len 20480
15500 127/300 Read block 4547 len 28672
15501 128/300 Write block 3986 len 16384 data: 2 3 3 3
15502 129/300 Write block 1523 len 28672 data: 2 2 2 3 4 5 5
15503 130/300 Read block 1000 len 40960
15504 131/300 Write block 3656 len 32768 data: 5 6 5 4 4 3 3 3
15505 132/300 Write block 1952 len 28672 data: 4 4 4 4 4 4 3
15506 133/300 Read block 177 len 24576
15507 134/300 Write block 2059 len 28672 data: 4 4 3 4 4 4 4
15508 135/300 Read block 3450 len 36864
15509 136/300 Read block 2003 len 32768
15510 137/300 Read block 3986 len 12288
15511 138/300 Read block 2668 len 40960
15512 139/300 Write block 1536 len 4096 data: 3
15513 140/300 Flush
15514 141/300 Read block 3606 len 40960
15515 142/300 Read block 4639 len 8192
15516 143/300 Read block 1786 len 20480
15517 144/300 Read block 2114 len 24576
15518 145/300 Read block 4229 len 28672
15519 146/300 Read block 483 len 8192
15520 147/300 Flush
15521 148/300 Read block 450 len 12288
15522 149/300 Write block 2287 len 40960 data: 4 3 3 3 4 4 3 3 3 3
15523 150/300 Flush
15524 151/300 Read block 70 len 32768
15525 152/300 Read block 4609 len 28672
15526 153/300 Read block 3452 len 12288
15527 154/300 Read block 262 len 36864
15528 155/300 Flush
15529 156/300 Read block 1028 len 20480
15530 157/300 Write block 2786 len 24576 data: 4 4 4 3 2 2
15531 158/300 Read block 4680 len 32768
15532 159/300 Read block 1388 len 32768
15533 160/300 Write block 4961 len 4096 data: 3
15534 161/300 Flush
15535 162/300 Read block 3680 len 40960
15536 163/300 Write block 4023 len 32768 data: 6 4 2 3 4 4 5 5
15537 164/300 Read block 467 len 16384
15538 165/300 Write block 4937 len 4096 data: 2
15539 166/300 Write block 40 len 16384 data: 4 3 4 4
15540 167/300 Read block 4881 len 8192
15541 168/300 Read block 1716 len 8192
15542 169/300 Read block 502 len 24576
15543 170/300 Write block 290 len 20480 data: 4 4 4 4 4
15544 171/300 Read block 3625 len 12288
15545 172/300 Read block 3502 len 4096
15546 173/300 Write block 3921 len 32768 data: 5 5 2 4 4 4 2 2
15547 174/300 Read block 2727 len 32768
15548 175/300 Write block 4340 len 8192 data: 4 3
15549 176/300 Read block 4052 len 32768
15550 177/300 Write block 2784 len 28672 data: 5 4 5 5 5 4 3
15551 178/300 Flush
15552 179/300 Read block 3668 len 28672
15553 180/300 Write block 170 len 32768 data: 2 2 3 3 3 3 4 4
15554 181/300 Read block 1222 len 4096
15555 182/300 Read block 3832 len 20480
15556 183/300 Read block 2638 len 32768
15557 184/300 Write block 4029 len 4096 data: 6
15558 185/300 Write block 4508 len 40960 data: 3 3 3 2 2 2 2 2 2 3
15559 186/300 Read block 4249 len 32768
15560 187/300 Read block 1684 len 36864
15561 188/300 Write block 4218 len 28672 data: 5 3 2 2 2 3 3
15562 189/300 Write block 4398 len 4096 data: 2
15563 190/300 Read block 4991 len 28672
15564 191/300 Read block 971 len 32768
15565 192/300 Read block 4233 len 36864
15566 193/300 Read block 577 len 4096
15567 194/300 Write block 1541 len 24576 data: 4 4 5 5 6 5
15568 195/300 Read block 1557 len 40960
15569 196/300 Read block 2564 len 24576
15570 197/300 Write block 3470 len 24576 data: 4 3 4 5 5 4
15571 198/300 Write block 906 len 28672 data: 2 2 2 2 3 3 3
15572 199/300 Read block 2012 len 36864
15573 200/300 Write block 3896 len 32768 data: 5 5 5 4 4 5 4 3
15574 201/300 Read block 3976 len 28672
15575 202/300 Flush
15576 203/300 Read block 1898 len 32768
15577 204/300 Read block 4813 len 12288
15578 205/300 Write block 1907 len 12288 data: 4 3 3
15579 206/300 Read block 2776 len 28672
15580 207/300 Read block 4777 len 28672
15581 208/300 Read block 3696 len 40960
15582 209/300 Write block 3087 len 24576 data: 2 3 4 5 5 6
15583 210/300 Write block 2569 len 8192 data: 5 4
15584 211/300 Read block 2323 len 32768
15585 212/300 Write block 2711 len 36864 data: 2 4 4 4 4 4 4 3 3
15586 213/300 Write block 224 len 40960 data: 3 3 3 4 3 3 3 4 3 3
15587 214/300 Write block 2003 len 28672 data: 3 3 3 2 3 2 4
15588 215/300 Write block 4241 len 40960 data: 8 7 6 6 7 7 5 5 5 5
15589 216/300 Write block 3750 len 32768 data: 4 4 4 4 5 5 3 4
15590 217/300 Read block 3569 len 12288
15591 218/300 Write block 122 len 16384 data: 3 2 2 3
15592 219/300 Write block 4805 len 20480 data: 2 3 3 2 2
15593 220/300 Read block 2739 len 36864
15594 221/300 Read block 878 len 28672
15595 222/300 Write block 1287 len 16384 data: 2 2 2 2
15596 223/300 Write block 3128 len 40960 data: 6 4 4 4 4 3 3 3 3 3
15597 224/300 Flush
15598 225/300 Read block 3891 len 28672
15599 226/300 Read block 4017 len 12288
15600 227/300 Write block 2310 len 40960 data: 2 2 2 2 3 3 3 4 4 3
15601 228/300 Read block 4344 len 4096
15602 229/300 Write block 4036 len 8192 data: 4 4
15603 230/300 Read block 4985 len 4096
15604 231/300 Read block 1149 len 20480
15605 232/300 Write block 4053 len 20480 data: 5 4 2 2 3
15606 233/300 Write block 277 len 8192 data: 2 2
15607 234/300 Read block 2741 len 4096
15608 235/300 Write block 2484 len 16384 data: 3 5 6 6
15609 236/300 Read block 3706 len 16384
15610 237/300 Write block 1003 len 8192 data: 2 2
15611 238/300 Read block 2756 len 4096
15612 239/300 Read block 805 len 40960
15613 240/300 Read block 4853 len 40960
15614 241/300 Read block 2136 len 40960
15615 242/300 Write block 294 len 12288 data: 5 6 6
15616 243/300 Read block 2154 len 12288
15617 244/300 Write block 3892 len 36864 data: 5 5 6 5 6 6 6 5 5
15618 245/300 Write block 775 len 16384 data: 3 3 4 4
15619 246/300 Read block 252 len 4096
15620 247/300 Write block 2921 len 4096 data: 8
15621 248/300 Write block 496 len 28672 data: 3 3 2 4 3 3 3
15622 249/300 Read block 2308 len 32768
15623 250/300 Read block 2499 len 20480
15624 251/300 Read block 1179 len 36864
15625 252/300 Read block 2401 len 32768
15626 253/300 Read block 1972 len 8192
15627 254/300 Flush
15628 255/300 Read block 1653 len 32768
15629 256/300 Read block 4843 len 36864
15630 257/300 Flush
15631 258/300 Read block 2883 len 32768
15632 259/300 Write block 4217 len 28672 data: 5 6 4 3 3 3 4
15633 260/300 Read block 1910 len 32768
15634 261/300 Write block 3597 len 32768 data: 2 2 3 4 4 4 4 4
15635 262/300 Read block 4538 len 20480
15636 263/300 Write block 2556 len 16384 data: 2 2 3 3
15637 264/300 Write block 3738 len 12288 data: 5 4 4
15638 265/300 Read block 1004 len 40960
15639 266/300 Read block 2108 len 8192
15640 267/300 Flush
15641 268/300 Read block 1670 len 16384
15642 269/300 Write block 3122 len 4096 data: 4
15643 270/300 Write block 2937 len 20480 data: 3 3 3 3 4
15644 271/300 Write block 1196 len 8192 data: 3 2
15645 272/300 Read block 2629 len 36864
15646 273/300 Read block 4250 len 28672
15647 274/300 Write block 454 len 12288 data: 5 6 6
15648 275/300 Write block 2397 len 28672 data: 3 3 3 2 2 2 2
15649 276/300 Write block 2179 len 40960 data: 3 3 4 7 7 6 6 6 5 6
15650 277/300 Write block 158 len 36864 data: 2 2 2 2 2 2 2 2 3
15651 278/300 Read block 2854 len 20480
15652 279/300 Read block 504 len 24576
15653 280/300 Write block 1585 len 8192 data: 7 8
15654 281/300 Write block 1169 len 8192 data: 4 7
15655 282/300 Read block 2539 len 36864
15656 283/300 Read block 2662 len 4096
15657 284/300 Write block 4182 len 12288 data: 3 2 2
15658 285/300 Write block 2286 len 20480 data: 5 5 4 4 4
15659 286/300 Write block 1174 len 16384 data: 4 3 3 2
15660 287/300 Read block 3652 len 36864
15661 288/300 Read block 514 len 28672
15662 289/300 Read block 3681 len 24576
15663 290/300 Write block 1949 len 16384 data: 3 2 4 5
15664 291/300 Read block 4164 len 4096
15665 292/300 Read block 4638 len 40960
15666 293/300 Read block 3709 len 8192
15667 294/300 Flush
15668 295/300 Write block 4684 len 20480 data: 2 4 4 4 3
15669 296/300 Read block 4154 len 4096
15670 297/300 Read block 4524 len 4096
15671 298/300 Read block 2045 len 20480
15672 299/300 Write block 1674 len 4096 data: 2
15673 300/300 Flush
15674 301/300 Read block 938 len 16384
15675 Main task has work to do, go find it
15676 got dsc Start(1)
15677 start 1
15678 Jan 15 05:22:18.380 INFO request completed, latency_us: 178, response_code: 204, uri: /start/cid/1, method: POST, req_id: af8b5a96-65a4-4d4c-912f-3e8b459506ee, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
15679 Replay: started 1, returned:Ok(())
15680 ----------------------------------------------------------------
15681 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15682 States: Active Offline Active
15683 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15684 4925 Acked Write 5 Done Sent Done false
15685 4926 Acked Read 1 Done Sent Done false
15686 4927 Acked Read 4 Done Sent Done false
15687 4928 Acked Read 3 Done Sent Done false
15688 4929 Acked Read 6 Done Sent Done false
15689 4930 Acked Flush 0 Done Sent Done false
15690 4931 Acked Flush 0 Done Sent Done false
15691 4932 Acked Read 3 Done Sent Done false
15692 4933 Acked Write 8 Done Sent Done false
15693 4934 Acked Read 1 Done Sent Done false
15694 4935 Acked Write 8 Done Sent Done false
15695 4936 Acked Read 1 Done Sent Done false
15696 4937 Acked Read 4 Done Sent Done false
15697 4938 Acked Write 4 Done Sent Done false
15698 4939 Acked Flush 0 Done Sent Done false
15699 4940 Acked Read 4 Done Sent Done false
15700 4941 Acked Read 2 Done Sent Done false
15701 4942 Acked Read 5 Done Sent Done false
15702 4943 Acked Write 10 Done Sent Done false
15703 4944 Acked Write 6 Done Sent Done false
15704 4945 Acked Read 1 Done Sent Done false
15705 4946 Acked Write 9 Done Sent Done false
15706 4947 Acked Read 5 Done Sent Done false
15707 4948 Acked Write 4 Done Sent Done false
15708 4949 Acked Read 10 Done Sent Done false
15709 4950 Acked Write 4 Done Sent Done false
15710 4951 Acked Write 4 Done Sent Done false
15711 4952 Acked Write 2 Done Sent Done false
15712 4953 Acked Write 1 Done Sent Done false
15713 4954 Acked Write 10 Done Sent Done false
15714 4955 Acked Flush 0 Done Sent Done false
15715 4956 Acked Write 10 Done Sent Done false
15716 4957 Acked Read 3 Done Sent Done false
15717 4958 Acked Write 9 Done Sent Done false
15718 4959 Acked Read 5 Done Sent Done false
15719 4960 Acked Read 5 Done Sent Done false
15720 4961 Acked Write 3 Done Sent Done false
15721 4962 Acked Write 7 Done Sent Done false
15722 4963 Acked Read 4 Done Sent Done false
15723 4964 Acked Read 8 Done Sent Done false
15724 4965 Acked Read 6 Done Sent Done false
15725 4966 Acked Write 5 Done Sent Done false
15726 4967 Acked Read 1 Done Sent Done false
15727 4968 Acked Write 3 Done Sent Done false
15728 4969 Acked Read 3 Done Sent Done false
15729 4970 Acked Flush 0 Done Sent Done false
15730 4971 Acked Read 2 Done Sent Done false
15731 4972 Acked Flush 0 Done Sent Done false
15732 4973 Acked Write 3 Done Sent Done false
15733 4974 Acked Write 6 Done Sent Done false
15734 4975 Acked Read 3 Done Sent Done false
15735 4976 Acked Write 10 Done Sent Done false
15736 4977 Acked Write 6 Done Sent Done false
15737 4978 Acked Read 1 Done Sent Done false
15738 4979 Acked Flush 0 Done Sent Done false
15739 4980 Acked Read 4 Done Sent Done false
15740 4981 Acked Write 2 Done Sent Done false
15741 4982 Acked Write 7 Done Sent Done false
15742 4983 Acked Read 7 Done Sent Done false
15743 4984 Acked Flush 0 Done Sent Done false
15744 4985 Acked Write 5 Done Sent Done false
15745 4986 Acked Read 9 Done Sent Done false
15746 4987 Acked Read 3 Done Sent Done false
15747 4988 Acked Write 7 Done Sent Done false
15748 4989 Acked Read 6 Done Sent Done false
15749 4990 Acked Read 2 Done Sent Done false
15750 4991 Acked Write 1 Done Sent Done false
15751 4992 Acked Write 6 Done Sent Done false
15752 4993 Acked Flush 0 Done Sent Done false
15753 4994 Acked Flush 0 Done Sent Done false
15754 4995 Acked Write 3 Done Sent Done false
15755 4996 Acked Write 6 Done Sent Done false
15756 4997 Acked Write 1 Done Sent Done false
15757 4998 Acked Write 4 Done Sent Done false
15758 4999 Acked Read 1 Done Sent Done false
15759 5000 Acked Write 9 Done Sent Done false
15760 5001 Acked Read 5 Done Sent Done false
15761 5002 Acked Read 8 Done Sent Done false
15762 5003 Acked Read 2 Done Sent Done false
15763 5004 Acked Read 6 Done Sent Done false
15764 5005 Acked Write 9 Done Sent Done false
15765 5006 Acked Read 8 Done Sent Done false
15766 5007 Acked Read 9 Done Sent Done false
15767 5008 Acked Write 9 Done Sent Done false
15768 5009 Acked Read 5 Done Sent Done false
15769 5010 Acked Write 8 Done Sent Done false
15770 5011 Acked Flush 0 Done Sent Done false
15771 5012 Acked Write 7 Done Sent Done false
15772 5013 Acked Write 10 Done Sent Done false
15773 5014 Acked Write 10 Done Sent Done false
15774 5015 Acked Read 10 Done Sent Done false
15775 5016 Acked Write 3 Done Sent Done false
15776 5017 Acked Write 9 Done Sent Done false
15777 5018 Acked Write 5 Done Sent Done false
15778 5019 Acked Write 10 Done Sent Done false
15779 5020 Acked Write 6 Done Sent Done false
15780 5021 Acked Write 4 Done Sent Done false
15781 5022 Acked Read 4 Done Sent Done false
15782 5023 Acked Read 5 Done Sent Done false
15783 5024 Acked Read 5 Done Sent Done false
15784 5025 Acked Write 9 Done Sent Done false
15785 5026 Acked Read 2 Done Sent Done false
15786 5027 Acked Write 10 Done Sent Done false
15787 5028 Acked Write 9 Done Sent Done false
15788 5029 Acked Read 9 Done Sent Done false
15789 5030 Acked Write 4 Done Sent Done false
15790 5031 Acked Read 6 Done Sent Done false
15791 5032 Acked Write 5 Done Sent Done false
15792 5033 Acked Read 2 Done Sent Done false
15793 5034 Acked Read 5 Done Sent Done false
15794 5035 Acked Flush 0 Done Sent Done false
15795 5036 Acked Read 10 Done Sent Done false
15796 5037 Acked Write 1 Done Sent Done false
15797 5038 Acked Flush 0 Done Sent Done false
15798 5039 Acked Write 8 Done Sent Done false
15799 5040 Acked Write 9 Done Sent Done false
15800 5041 Acked Read 6 Done Sent Done false
15801 5042 Acked Read 5 Done Sent Done false
15802 5043 Acked Read 7 Done Sent Done false
15803 5044 Acked Write 3 Done Sent Done false
15804 5045 Acked Write 8 Done Sent Done false
15805 5046 Acked Write 9 Done Sent Done false
15806 5047 Acked Write 10 Done Sent Done false
15807 5048 Acked Read 1 Done Sent Done false
15808 5049 Acked Read 5 Done Sent Done false
15809 5050 Acked Read 5 Done Sent Done false
15810 5051 Acked Read 7 Done Sent Done false
15811 5052 Acked Write 4 Done Sent Done false
15812 5053 Acked Write 7 Done Sent Done false
15813 5054 Acked Read 10 Done Sent Done false
15814 5055 Acked Write 8 Done Sent Done false
15815 5056 Acked Write 7 Done Sent Done false
15816 5057 Acked Read 6 Done Sent Done false
15817 5058 Acked Write 7 Done Sent Done false
15818 5059 Acked Read 9 Done Sent Done false
15819 5060 Acked Read 8 Done Sent Done false
15820 5061 Acked Read 3 Done Sent Done false
15821 5062 Acked Read 10 Done Sent Done false
15822 5063 Acked Write 1 Done Sent Done false
15823 5064 Acked Flush 0 Done Sent Done false
15824 5065 Acked Read 10 Done Sent Done false
15825 5066 Acked Read 2 Done Sent Done false
15826 5067 Acked Read 5 Done Sent Done false
15827 5068 Acked Read 6 Done Sent Done false
15828 5069 Acked Read 7 Done Sent Done false
15829 5070 Acked Read 2 Done Sent Done false
15830 5071 Acked Flush 0 Done Sent Done false
15831 5072 Acked Read 3 Done Sent Done false
15832 5073 Acked Write 10 Done Sent Done false
15833 5074 Acked Flush 0 Done Sent Done false
15834 5075 Acked Read 8 Done Sent Done false
15835 5076 Acked Read 7 Done Sent Done false
15836 5077 Acked Read 3 Done Sent Done false
15837 5078 Acked Read 9 Done Sent Done false
15838 5079 Acked Flush 0 Done Sent Done false
15839 5080 Acked Read 5 Done Sent Done false
15840 5081 Acked Write 6 Done Sent Done false
15841 5082 Acked Read 8 Done Sent Done false
15842 5083 Acked Read 8 Done Sent Done false
15843 5084 Acked Write 1 Done Sent Done false
15844 5085 Acked Flush 0 Done Sent Done false
15845 5086 Acked Read 10 Done Sent Done false
15846 5087 Acked Write 8 Done Sent Done false
15847 5088 Acked Read 4 Done Sent Done false
15848 5089 Acked Write 1 Done Sent Done false
15849 5090 Acked Write 4 Done Sent Done false
15850 5091 Acked Read 2 Done Sent Done false
15851 5092 Acked Read 2 Done Sent Done false
15852 5093 Acked Read 6 Done Sent Done false
15853 5094 Acked Write 5 Done Sent Done false
15854 5095 Acked Read 3 Done Sent Done false
15855 5096 Acked Read 1 Done Sent Done false
15856 5097 Acked Write 8 Done Sent Done false
15857 5098 Acked Read 8 Done Sent Done false
15858 5099 Acked Write 2 Done Sent Done false
15859 5100 Acked Read 8 Done Sent Done false
15860 5101 Acked Write 7 Done Sent Done false
15861 5102 Acked Flush 0 Done Sent Done false
15862 5103 Acked Read 7 Done Sent Done false
15863 5104 Acked Write 8 Done Sent Done false
15864 5105 Acked Read 1 Done Sent Done false
15865 5106 Acked Read 5 Done Sent Done false
15866 5107 Acked Read 8 Done Sent Done false
15867 5108 Acked Write 1 Done Sent Done false
15868 5109 Acked Write 10 Done Sent Done false
15869 5110 Acked Read 8 Done Sent Done false
15870 5111 Acked Read 9 Done Sent Done false
15871 5112 Acked Write 7 Done Sent Done false
15872 5113 Acked Write 1 Done Sent Done false
15873 5114 Acked Read 7 Done Sent Done false
15874 5115 Acked Read 8 Done Sent Done false
15875 5116 Acked Read 9 Done Sent Done false
15876 5117 Acked Read 1 Done Sent Done false
15877 5118 Acked Write 6 Done Sent Done false
15878 5119 Acked Read 10 Done Sent Done false
15879 5120 Acked Read 6 Done Sent Done false
15880 5121 Acked Write 6 Done Sent Done false
15881 5122 Acked Write 7 Done Sent Done false
15882 5123 Acked Read 9 Done Sent Done false
15883 5124 Acked Write 8 Done Sent Done false
15884 5125 Acked Read 7 Done Sent Done false
15885 5126 Acked Flush 0 Done Sent Done false
15886 5127 Acked Read 8 Done Sent Done false
15887 5128 Acked Read 3 Done Sent Done false
15888 5129 Acked Write 3 Done Sent Done false
15889 5130 Acked Read 7 Done Sent Done false
15890 5131 Acked Read 7 Done Sent Done false
15891 5132 Acked Read 10 Done Sent Done false
15892 5133 Acked Write 6 Done Sent Done false
15893 5134 Acked Write 2 Done Sent Done false
15894 5135 Acked Read 8 Done Sent Done false
15895 5136 Acked Write 9 Done Sent Done false
15896 5137 Acked Write 10 Done Sent Done false
15897 5138 Acked Write 7 Done Sent Done false
15898 5139 Acked Write 10 Done Sent Done false
15899 5140 Acked Write 8 Done Sent Done false
15900 5141 Acked Read 3 Done Sent Done false
15901 5142 Acked Write 4 Done Sent Done false
15902 5143 Acked Write 5 Done Sent Done false
15903 5144 Acked Read 9 Done Sent Done false
15904 5145 Acked Read 7 Done Sent Done false
15905 5146 Acked Write 4 Done Sent Done false
15906 5147 Acked Write 10 Done Sent Done false
15907 5148 Acked Flush 0 Done Sent Done false
15908 5149 Acked Read 7 Done Sent Done false
15909 5150 Acked Read 3 Done Sent Done false
15910 5151 Acked Write 10 Done Sent Done false
15911 5152 Acked Read 1 Done Sent Done false
15912 5153 Acked Write 2 Done Sent Done false
15913 5154 Acked Read 1 Done Sent Done false
15914 5155 Acked Read 5 Done Sent Done false
15915 5156 Acked Write 5 Done Sent Done false
15916 5157 Acked Write 2 Done Sent Done false
15917 5158 Acked Read 1 Done Sent Done false
15918 5159 Acked Write 4 Done Sent Done false
15919 5160 Acked Read 4 Done Sent Done false
15920 5161 Acked Write 2 Done Sent Done false
15921 5162 Acked Read 1 Done Sent Done false
15922 5163 Acked Read 10 Done Sent Done false
15923 5164 Acked Read 10 Done Sent Done false
15924 5165 Acked Read 10 Done Sent Done false
15925 5166 Acked Write 3 Done Sent Done false
15926 5167 Acked Read 3 Done Sent Done false
15927 5168 Acked Write 9 Done Sent Done false
15928 5169 Acked Write 4 Done Sent Done false
15929 5170 Acked Read 1 Done Sent Done false
15930 5171 Acked Write 1 Done Sent Done false
15931 5172 Acked Write 7 Done Sent Done false
15932 5173 Acked Read 8 Done Sent Done false
15933 5174 Acked Read 5 Done Sent Done false
15934 5175 Acked Read 9 Done Sent Done false
15935 5176 Acked Read 8 Done Sent Done false
15936 5177 Acked Read 2 Done Sent Done false
15937 5178 Acked Flush 0 Done Sent Done false
15938 5179 Acked Read 8 Done Sent Done false
15939 5180 Acked Read 9 Done Sent Done false
15940 5181 Acked Flush 0 Done Sent Done false
15941 5182 Acked Read 8 Done Sent Done false
15942 5183 Acked Write 7 Done Sent Done false
15943 5184 Acked Read 8 Done Sent Done false
15944 5185 Acked Write 8 Done Sent Done false
15945 5186 Acked Read 5 Done Sent Done false
15946 5187 Acked Write 4 Done Sent Done false
15947 5188 Acked Write 3 Done Sent Done false
15948 5189 Acked Read 10 Done Sent Done false
15949 5190 Acked Read 2 Done Sent Done false
15950 5191 Acked Flush 0 Done Sent Done false
15951 5192 Acked Read 4 Done Sent Done false
15952 5193 Acked Write 1 Done Sent Done false
15953 5194 Acked Write 5 Done Sent Done false
15954 5195 Acked Write 2 Done Sent Done false
15955 5196 Acked Read 9 Done Sent Done false
15956 5197 Acked Read 7 Done Sent Done false
15957 5198 Acked Write 3 Done Sent Done false
15958 5199 Acked Write 7 Done Sent Done false
15959 5200 Acked Write 10 Done Sent Done false
15960 5201 Acked Write 9 Done Sent Done false
15961 5202 Acked Read 5 Done Sent Done false
15962 5203 Acked Read 6 Done Sent Done false
15963 5204 Acked Write 2 Done Sent Done false
15964 5205 Acked Write 2 Done Sent Done false
15965 5206 Acked Read 9 Done Sent Done false
15966 5207 Acked Read 1 Done Sent Done false
15967 5208 Acked Write 3 Done Sent Done false
15968 5209 Acked Write 5 Done Sent Done false
15969 5210 Acked Write 4 Done Sent Done false
15970 5211 Acked Read 9 Done Sent Done false
15971 5212 Acked Read 7 Done Sent Done false
15972 5213 Acked Read 6 Done Sent Done false
15973 5214 Acked Write 4 Done Sent Done false
15974 5215 Acked Read 1 Done Sent Done false
15975 5216 Acked Read 10 Done Sent Done false
15976 5217 Acked Read 2 Done Sent Done false
15977 5218 Acked Flush 0 Done Sent Done false
15978 5219 Acked Write 5 Done Sent Done false
15979 5220 Acked Read 1 Done Sent Done false
15980 5221 Acked Read 1 Done Sent Done false
15981 5222 Acked Read 5 Done Sent Done false
15982 5223 Acked Write 1 Done Sent Done false
15983 5224 Acked Flush 0 Done Sent Done false
15984 5225 Acked Read 4 Done Sent Done false
15985 STATES DS:0 DS:1 DS:2 TOTAL
15986 Sent 0 301 0 301
15987 Done 301 0 301 602
15988 Skipped 0 0 0 0
15989 Error 0 0 0 0
15990 Last Flush: 5224 4924 5224
15991 Downstairs last five completed: 4924 4923 4922 4921 4920
15992 Upstairs last five completed: 5225 5224 5223 5222 5221
15993 CLIENT: Up:0 ds:301 act:2
15994 [8820] got start action
15995 [8820] I am going to restart
15996 Starting downstairs at port 8820
15997 Make output file at "/tmp/dsc/downstairs-8820.txt"
15998 [8820][1] reports Starting
15999 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1216)
16000 [8820][1] reports Running
16001 ----------------------------------------------------------------
16002 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16003 States: Active Offline Active
16004 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16005 4925 Acked Write 5 Done Sent Done false
16006 4926 Acked Read 1 Done Sent Done false
16007 4927 Acked Read 4 Done Sent Done false
16008 4928 Acked Read 3 Done Sent Done false
16009 4929 Acked Read 6 Done Sent Done false
16010 4930 Acked Flush 0 Done Sent Done false
16011 4931 Acked Flush 0 Done Sent Done false
16012 4932 Acked Read 3 Done Sent Done false
16013 4933 Acked Write 8 Done Sent Done false
16014 4934 Acked Read 1 Done Sent Done false
16015 4935 Acked Write 8 Done Sent Done false
16016 4936 Acked Read 1 Done Sent Done false
16017 4937 Acked Read 4 Done Sent Done false
16018 4938 Acked Write 4 Done Sent Done false
16019 4939 Acked Flush 0 Done Sent Done false
16020 4940 Acked Read 4 Done Sent Done false
16021 4941 Acked Read 2 Done Sent Done false
16022 4942 Acked Read 5 Done Sent Done false
16023 4943 Acked Write 10 Done Sent Done false
16024 4944 Acked Write 6 Done Sent Done false
16025 4945 Acked Read 1 Done Sent Done false
16026 4946 Acked Write 9 Done Sent Done false
16027 4947 Acked Read 5 Done Sent Done false
16028 4948 Acked Write 4 Done Sent Done false
16029 4949 Acked Read 10 Done Sent Done false
16030 4950 Acked Write 4 Done Sent Done false
16031 4951 Acked Write 4 Done Sent Done false
16032 4952 Acked Write 2 Done Sent Done false
16033 4953 Acked Write 1 Done Sent Done false
16034 4954 Acked Write 10 Done Sent Done false
16035 4955 Acked Flush 0 Done Sent Done false
16036 4956 Acked Write 10 Done Sent Done false
16037 4957 Acked Read 3 Done Sent Done false
16038 4958 Acked Write 9 Done Sent Done false
16039 4959 Acked Read 5 Done Sent Done false
16040 4960 Acked Read 5 Done Sent Done false
16041 4961 Acked Write 3 Done Sent Done false
16042 4962 Acked Write 7 Done Sent Done false
16043 4963 Acked Read 4 Done Sent Done false
16044 4964 Acked Read 8 Done Sent Done false
16045 4965 Acked Read 6 Done Sent Done false
16046 4966 Acked Write 5 Done Sent Done false
16047 4967 Acked Read 1 Done Sent Done false
16048 4968 Acked Write 3 Done Sent Done false
16049 4969 Acked Read 3 Done Sent Done false
16050 4970 Acked Flush 0 Done Sent Done false
16051 4971 Acked Read 2 Done Sent Done false
16052 4972 Acked Flush 0 Done Sent Done false
16053 4973 Acked Write 3 Done Sent Done false
16054 4974 Acked Write 6 Done Sent Done false
16055 4975 Acked Read 3 Done Sent Done false
16056 4976 Acked Write 10 Done Sent Done false
16057 4977 Acked Write 6 Done Sent Done false
16058 4978 Acked Read 1 Done Sent Done false
16059 4979 Acked Flush 0 Done Sent Done false
16060 4980 Acked Read 4 Done Sent Done false
16061 4981 Acked Write 2 Done Sent Done false
16062 4982 Acked Write 7 Done Sent Done false
16063 4983 Acked Read 7 Done Sent Done false
16064 4984 Acked Flush 0 Done Sent Done false
16065 4985 Acked Write 5 Done Sent Done false
16066 4986 Acked Read 9 Done Sent Done false
16067 4987 Acked Read 3 Done Sent Done false
16068 4988 Acked Write 7 Done Sent Done false
16069 4989 Acked Read 6 Done Sent Done false
16070 4990 Acked Read 2 Done Sent Done false
16071 4991 Acked Write 1 Done Sent Done false
16072 4992 Acked Write 6 Done Sent Done false
16073 4993 Acked Flush 0 Done Sent Done false
16074 4994 Acked Flush 0 Done Sent Done false
16075 4995 Acked Write 3 Done Sent Done false
16076 4996 Acked Write 6 Done Sent Done false
16077 4997 Acked Write 1 Done Sent Done false
16078 4998 Acked Write 4 Done Sent Done false
16079 4999 Acked Read 1 Done Sent Done false
16080 5000 Acked Write 9 Done Sent Done false
16081 5001 Acked Read 5 Done Sent Done false
16082 5002 Acked Read 8 Done Sent Done false
16083 5003 Acked Read 2 Done Sent Done false
16084 5004 Acked Read 6 Done Sent Done false
16085 5005 Acked Write 9 Done Sent Done false
16086 5006 Acked Read 8 Done Sent Done false
16087 5007 Acked Read 9 Done Sent Done false
16088 5008 Acked Write 9 Done Sent Done false
16089 5009 Acked Read 5 Done Sent Done false
16090 5010 Acked Write 8 Done Sent Done false
16091 5011 Acked Flush 0 Done Sent Done false
16092 5012 Acked Write 7 Done Sent Done false
16093 5013 Acked Write 10 Done Sent Done false
16094 5014 Acked Write 10 Done Sent Done false
16095 5015 Acked Read 10 Done Sent Done false
16096 5016 Acked Write 3 Done Sent Done false
16097 5017 Acked Write 9 Done Sent Done false
16098 5018 Acked Write 5 Done Sent Done false
16099 5019 Acked Write 10 Done Sent Done false
16100 5020 Acked Write 6 Done Sent Done false
16101 5021 Acked Write 4 Done Sent Done false
16102 5022 Acked Read 4 Done Sent Done false
16103 5023 Acked Read 5 Done Sent Done false
16104 5024 Acked Read 5 Done Sent Done false
16105 5025 Acked Write 9 Done Sent Done false
16106 5026 Acked Read 2 Done Sent Done false
16107 5027 Acked Write 10 Done Sent Done false
16108 5028 Acked Write 9 Done Sent Done false
16109 5029 Acked Read 9 Done Sent Done false
16110 5030 Acked Write 4 Done Sent Done false
16111 5031 Acked Read 6 Done Sent Done false
16112 5032 Acked Write 5 Done Sent Done false
16113 5033 Acked Read 2 Done Sent Done false
16114 5034 Acked Read 5 Done Sent Done false
16115 5035 Acked Flush 0 Done Sent Done false
16116 5036 Acked Read 10 Done Sent Done false
16117 5037 Acked Write 1 Done Sent Done false
16118 5038 Acked Flush 0 Done Sent Done false
16119 5039 Acked Write 8 Done Sent Done false
16120 5040 Acked Write 9 Done Sent Done false
16121 5041 Acked Read 6 Done Sent Done false
16122 5042 Acked Read 5 Done Sent Done false
16123 5043 Acked Read 7 Done Sent Done false
16124 5044 Acked Write 3 Done Sent Done false
16125 5045 Acked Write 8 Done Sent Done false
16126 5046 Acked Write 9 Done Sent Done false
16127 5047 Acked Write 10 Done Sent Done false
16128 5048 Acked Read 1 Done Sent Done false
16129 5049 Acked Read 5 Done Sent Done false
16130 5050 Acked Read 5 Done Sent Done false
16131 5051 Acked Read 7 Done Sent Done false
16132 5052 Acked Write 4 Done Sent Done false
16133 5053 Acked Write 7 Done Sent Done false
16134 5054 Acked Read 10 Done Sent Done false
16135 5055 Acked Write 8 Done Sent Done false
16136 5056 Acked Write 7 Done Sent Done false
16137 5057 Acked Read 6 Done Sent Done false
16138 5058 Acked Write 7 Done Sent Done false
16139 5059 Acked Read 9 Done Sent Done false
16140 5060 Acked Read 8 Done Sent Done false
16141 5061 Acked Read 3 Done Sent Done false
16142 5062 Acked Read 10 Done Sent Done false
16143 5063 Acked Write 1 Done Sent Done false
16144 5064 Acked Flush 0 Done Sent Done false
16145 5065 Acked Read 10 Done Sent Done false
16146 5066 Acked Read 2 Done Sent Done false
16147 5067 Acked Read 5 Done Sent Done false
16148 5068 Acked Read 6 Done Sent Done false
16149 5069 Acked Read 7 Done Sent Done false
16150 5070 Acked Read 2 Done Sent Done false
16151 5071 Acked Flush 0 Done Sent Done false
16152 5072 Acked Read 3 Done Sent Done false
16153 5073 Acked Write 10 Done Sent Done false
16154 5074 Acked Flush 0 Done Sent Done false
16155 5075 Acked Read 8 Done Sent Done false
16156 5076 Acked Read 7 Done Sent Done false
16157 5077 Acked Read 3 Done Sent Done false
16158 5078 Acked Read 9 Done Sent Done false
16159 5079 Acked Flush 0 Done Sent Done false
16160 5080 Acked Read 5 Done Sent Done false
16161 5081 Acked Write 6 Done Sent Done false
16162 5082 Acked Read 8 Done Sent Done false
16163 5083 Acked Read 8 Done Sent Done false
16164 5084 Acked Write 1 Done Sent Done false
16165 5085 Acked Flush 0 Done Sent Done false
16166 5086 Acked Read 10 Done Sent Done false
16167 5087 Acked Write 8 Done Sent Done false
16168 5088 Acked Read 4 Done Sent Done false
16169 5089 Acked Write 1 Done Sent Done false
16170 5090 Acked Write 4 Done Sent Done false
16171 5091 Acked Read 2 Done Sent Done false
16172 5092 Acked Read 2 Done Sent Done false
16173 5093 Acked Read 6 Done Sent Done false
16174 5094 Acked Write 5 Done Sent Done false
16175 5095 Acked Read 3 Done Sent Done false
16176 5096 Acked Read 1 Done Sent Done false
16177 5097 Acked Write 8 Done Sent Done false
16178 5098 Acked Read 8 Done Sent Done false
16179 5099 Acked Write 2 Done Sent Done false
16180 5100 Acked Read 8 Done Sent Done false
16181 5101 Acked Write 7 Done Sent Done false
16182 5102 Acked Flush 0 Done Sent Done false
16183 5103 Acked Read 7 Done Sent Done false
16184 5104 Acked Write 8 Done Sent Done false
16185 5105 Acked Read 1 Done Sent Done false
16186 5106 Acked Read 5 Done Sent Done false
16187 5107 Acked Read 8 Done Sent Done false
16188 5108 Acked Write 1 Done Sent Done false
16189 5109 Acked Write 10 Done Sent Done false
16190 5110 Acked Read 8 Done Sent Done false
16191 5111 Acked Read 9 Done Sent Done false
16192 5112 Acked Write 7 Done Sent Done false
16193 5113 Acked Write 1 Done Sent Done false
16194 5114 Acked Read 7 Done Sent Done false
16195 5115 Acked Read 8 Done Sent Done false
16196 5116 Acked Read 9 Done Sent Done false
16197 5117 Acked Read 1 Done Sent Done false
16198 5118 Acked Write 6 Done Sent Done false
16199 5119 Acked Read 10 Done Sent Done false
16200 5120 Acked Read 6 Done Sent Done false
16201 5121 Acked Write 6 Done Sent Done false
16202 5122 Acked Write 7 Done Sent Done false
16203 5123 Acked Read 9 Done Sent Done false
16204 5124 Acked Write 8 Done Sent Done false
16205 5125 Acked Read 7 Done Sent Done false
16206 5126 Acked Flush 0 Done Sent Done false
16207 5127 Acked Read 8 Done Sent Done false
16208 5128 Acked Read 3 Done Sent Done false
16209 5129 Acked Write 3 Done Sent Done false
16210 5130 Acked Read 7 Done Sent Done false
16211 5131 Acked Read 7 Done Sent Done false
16212 5132 Acked Read 10 Done Sent Done false
16213 5133 Acked Write 6 Done Sent Done false
16214 5134 Acked Write 2 Done Sent Done false
16215 5135 Acked Read 8 Done Sent Done false
16216 5136 Acked Write 9 Done Sent Done false
16217 5137 Acked Write 10 Done Sent Done false
16218 5138 Acked Write 7 Done Sent Done false
16219 5139 Acked Write 10 Done Sent Done false
16220 5140 Acked Write 8 Done Sent Done false
16221 5141 Acked Read 3 Done Sent Done false
16222 5142 Acked Write 4 Done Sent Done false
16223 5143 Acked Write 5 Done Sent Done false
16224 5144 Acked Read 9 Done Sent Done false
16225 5145 Acked Read 7 Done Sent Done false
16226 5146 Acked Write 4 Done Sent Done false
16227 5147 Acked Write 10 Done Sent Done false
16228 5148 Acked Flush 0 Done Sent Done false
16229 5149 Acked Read 7 Done Sent Done false
16230 5150 Acked Read 3 Done Sent Done false
16231 5151 Acked Write 10 Done Sent Done false
16232 5152 Acked Read 1 Done Sent Done false
16233 5153 Acked Write 2 Done Sent Done false
16234 5154 Acked Read 1 Done Sent Done false
16235 5155 Acked Read 5 Done Sent Done false
16236 5156 Acked Write 5 Done Sent Done false
16237 5157 Acked Write 2 Done Sent Done false
16238 5158 Acked Read 1 Done Sent Done false
16239 5159 Acked Write 4 Done Sent Done false
16240 5160 Acked Read 4 Done Sent Done false
16241 5161 Acked Write 2 Done Sent Done false
16242 5162 Acked Read 1 Done Sent Done false
16243 5163 Acked Read 10 Done Sent Done false
16244 5164 Acked Read 10 Done Sent Done false
16245 5165 Acked Read 10 Done Sent Done false
16246 5166 Acked Write 3 Done Sent Done false
16247 5167 Acked Read 3 Done Sent Done false
16248 5168 Acked Write 9 Done Sent Done false
16249 5169 Acked Write 4 Done Sent Done false
16250 5170 Acked Read 1 Done Sent Done false
16251 5171 Acked Write 1 Done Sent Done false
16252 5172 Acked Write 7 Done Sent Done false
16253 5173 Acked Read 8 Done Sent Done false
16254 5174 Acked Read 5 Done Sent Done false
16255 5175 Acked Read 9 Done Sent Done false
16256 5176 Acked Read 8 Done Sent Done false
16257 5177 Acked Read 2 Done Sent Done false
16258 5178 Acked Flush 0 Done Sent Done false
16259 5179 Acked Read 8 Done Sent Done false
16260 5180 Acked Read 9 Done Sent Done false
16261 5181 Acked Flush 0 Done Sent Done false
16262 5182 Acked Read 8 Done Sent Done false
16263 5183 Acked Write 7 Done Sent Done false
16264 5184 Acked Read 8 Done Sent Done false
16265 5185 Acked Write 8 Done Sent Done false
16266 5186 Acked Read 5 Done Sent Done false
16267 5187 Acked Write 4 Done Sent Done false
16268 5188 Acked Write 3 Done Sent Done false
16269 5189 Acked Read 10 Done Sent Done false
16270 5190 Acked Read 2 Done Sent Done false
16271 5191 Acked Flush 0 Done Sent Done false
16272 5192 Acked Read 4 Done Sent Done false
16273 5193 Acked Write 1 Done Sent Done false
16274 5194 Acked Write 5 Done Sent Done false
16275 5195 Acked Write 2 Done Sent Done false
16276 5196 Acked Read 9 Done Sent Done false
16277 5197 Acked Read 7 Done Sent Done false
16278 5198 Acked Write 3 Done Sent Done false
16279 5199 Acked Write 7 Done Sent Done false
16280 5200 Acked Write 10 Done Sent Done false
16281 5201 Acked Write 9 Done Sent Done false
16282 5202 Acked Read 5 Done Sent Done false
16283 5203 Acked Read 6 Done Sent Done false
16284 5204 Acked Write 2 Done Sent Done false
16285 5205 Acked Write 2 Done Sent Done false
16286 5206 Acked Read 9 Done Sent Done false
16287 5207 Acked Read 1 Done Sent Done false
16288 5208 Acked Write 3 Done Sent Done false
16289 5209 Acked Write 5 Done Sent Done false
16290 5210 Acked Write 4 Done Sent Done false
16291 5211 Acked Read 9 Done Sent Done false
16292 5212 Acked Read 7 Done Sent Done false
16293 5213 Acked Read 6 Done Sent Done false
16294 5214 Acked Write 4 Done Sent Done false
16295 5215 Acked Read 1 Done Sent Done false
16296 5216 Acked Read 10 Done Sent Done false
16297 5217 Acked Read 2 Done Sent Done false
16298 5218 Acked Flush 0 Done Sent Done false
16299 5219 Acked Write 5 Done Sent Done false
16300 5220 Acked Read 1 Done Sent Done false
16301 5221 Acked Read 1 Done Sent Done false
16302 5222 Acked Read 5 Done Sent Done false
16303 5223 Acked Write 1 Done Sent Done false
16304 5224 Acked Flush 0 Done Sent Done false
16305 5225 Acked Read 4 Done Sent Done false
16306 5226 Acked Flush 0 Done Sent Done false
16307 STATES DS:0 DS:1 DS:2 TOTAL
16308 Sent 0 302 0 302
16309 Done 302 0 302 604
16310 Skipped 0 0 0 0
16311 Error 0 0 0 0
16312 Last Flush: 5226 4924 5226
16313 Downstairs last five completed: 4924 4923 4922 4921 4920
16314 Upstairs last five completed: 5226 5225 5224 5223 5222
16315 CLIENT: Up:0 ds:302 act:2
163162025-01-15T05:22:24.144ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
163172025-01-15T05:22:24.144ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37179) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
163182025-01-15T05:22:24.145ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
163192025-01-15T05:22:24.146ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
163202025-01-15T05:22:24.146ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
163212025-01-15T05:22:24.146ZINFOcrucible: send last flush ID to this DS: 4924 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
163222025-01-15T05:22:24.146ZINFOcrucible: Replied this last flush ID: 4924 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
163232025-01-15T05:22:24.146ZINFOcrucible: [1] client re-new 302 jobs since flush 4924 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
163242025-01-15T05:22:24.146ZINFOcrucible: [1] Replayed 302 jobs since flush: 4924 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
16325 ----------------------------------------------------------------
16326 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16327 Downstairs last five completed: 5226 5225 5224 5223 5222
16328 Upstairs last five completed: 5226 5225 5224 5223 5222
16329 CLIENT: Up:0 ds:0 act:3
16330 Replay: All jobs finished
16331 Main task has work to do, go find it
16332 got dsc Stop(1)
16333 stop 1
16334 Jan 15 05:22:26.385 INFO [8820] Got stop action so:false kr:false
16335 request completed, latency_us: 328, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 02619c02-277b-4016-bd0d-5fd16631dcfb, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
16336 Jan 15 05:22:26.385 INFO request completed, latency_us: 107, {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:22:26.385791868Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
16337 response_code: 200, uri: /state/cid/1, method: GET, req_id: a012c0a6-9fb9-48cd-a5ce-0e94cf8e498c, remote_addr:{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:22:26.38587857Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
16338 127.0.0.1:{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:22:26.385923876Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
16339 42402, local_addr: 127{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:22:26.385954994Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
16340 .0.0.1:9998
16341 [8820] Exited with: ExitStatus(unix_wait_status(9))
16342 [8820][1] reports Exit
16343 Jan 15 05:22:30.388 INFO request completed, latency_us: 203, response_code: 200, uri: /state/cid/1, method: GET, req_id: 64f07917-2387-4795-8dd1-a550e6aec433, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
16344 001/300 Write block 218 len 36864 data: 4 4 4 3 3 3 4 4 4
16345 002/300 Read block 3212 len 32768
16346 003/300 Write block 3018 len 4096 data: 2
16347 004/300 Write block 1845 len 32768 data: 9 8 9 8 7 6 5 5
16348 005/300 Write block 888 len 32768 data: 3 2 2 3 4 4 5 4
16349 006/300 Read block 1515 len 16384
16350 007/300 Read block 4287 len 12288
16351 008/300 Write block 406 len 16384 data: 2 3 4 4
16352 009/300 Read block 925 len 40960
16353 010/300 Read block 544 len 36864
16354 011/300 Read block 777 len 16384
16355 012/300 Write block 4485 len 4096 data: 5
16356 013/300 Read block 1727 len 36864
16357 014/300 Flush
16358 015/300 Read block 734 len 12288
16359 016/300 Write block 3659 len 12288 data: 5 5 4
16360 017/300 Flush
16361 018/300 Read block 3577 len 20480
16362 019/300 Write block 1320 len 4096 data: 4
16363 020/300 Read block 1501 len 36864
16364 021/300 Write block 2448 len 28672 data: 5 5 5 4 4 4 4
16365 022/300 Read block 3408 len 20480
16366 023/300 Write block 1432 len 32768 data: 5 6 5 4 4 3 3 3
16367 024/300 Read block 4404 len 28672
16368 025/300 Read block 1336 len 8192
16369 026/300 Read block 23 len 24576
16370 027/300 Write block 1796 len 20480 data: 2 2 2 2 3
16371 028/300 Read block 3719 len 8192
16372 029/300 Read block 862 len 12288
16373 030/300 Write block 3884 len 32768 data: 3 3 3 4 3 3 3 3
16374 031/300 Read block 1064 len 4096
16375 032/300 Read block 868 len 16384
16376 033/300 Read block 852 len 16384
16377 034/300 Write block 1881 len 36864 data: 4 4 4 4 4 4 3 5 5
16378 035/300 Read block 4816 len 40960
16379 036/300 Read block 1192 len 8192
16380 037/300 Read block 1012 len 28672
16381 038/300 Read block 1461 len 36864
16382 039/300 Read block 1983 len 32768
16383 040/300 Read block 4956 len 40960
16384 041/300 Write block 1218 len 8192 data: 5 3
16385 042/300 Write block 2607 len 4096 data: 3
16386 043/300 Write block 3269 len 40960 data: 4 3 3 3 4 4 4 4 4 4
16387 044/300 Read block 1675 len 12288
16388 045/300 Read block 537 len 8192
16389 046/300 Flush
16390 047/300 Read block 4442 len 36864
16391 048/300 Read block 4294 len 36864
16392 049/300 Write block 2233 len 24576 data: 5 5 4 3 3 3
16393 050/300 Write block 2215 len 36864 data: 2 2 2 2 2 3 4 4 5
16394 051/300 Flush
16395 052/300 Read block 2272 len 28672
16396 053/300 Write block 1781 len 20480 data: 4 4 7 7 7
16397 054/300 Read block 4208 len 24576
16398 055/300 Read block 1783 len 24576
16399 056/300 Read block 2695 len 16384
16400 057/300 Read block 3625 len 8192
16401 058/300 Write block 4296 len 20480 data: 2 2 2 2 2
16402 059/300 Read block 1107 len 24576
16403 060/300 Read block 3788 len 24576
16404 061/300 Read block 2238 len 4096
16405 062/300 Read block 496 len 36864
16406 063/300 Write block 3035 len 40960 data: 7 7 7 6 6 7 7 7 6 6
16407 064/300 Read block 4687 len 28672
16408 065/300 Write block 3996 len 36864 data: 2 2 2 3 3 4 4 4 5
16409 066/300 Read block 378 len 4096
16410 067/300 Read block 131 len 4096
16411 068/300 Read block 253 len 40960
16412 069/300 Read block 2398 len 16384
16413 070/300 Read block 2062 len 32768
16414 071/300 Write block 2236 len 40960 data: 4 4 4 4 5 5 5 5 6 5
16415 072/300 Write block 3850 len 36864 data: 3 3 3 3 4 5 6 6 6
16416 073/300 Read block 404 len 8192
16417 074/300 Write block 4920 len 4096 data: 3
16418 075/300 Read block 3464 len 12288
16419 076/300 Read block 807 len 24576
16420 077/300 Flush
16421 078/300 Write block 3425 len 4096 data: 3
16422 079/300 Read block 154 len 12288
16423 080/300 Read block 3403 len 4096
16424 081/300 Write block 4715 len 28672 data: 2 2 2 3 3 3 4
16425 082/300 Read block 3476 len 24576
16426 083/300 Write block 918 len 8192 data: 2 2
16427 084/300 Write block 3169 len 32768 data: 3 2 3 2 3 3 2 2
16428 085/300 Write block 436 len 40960 data: 3 3 4 4 5 6 8 8 8 9
16429 086/300 Write block 4256 len 12288 data: 3 4 6
16430 087/300 Read block 330 len 4096
16431 088/300 Write block 1074 len 32768 data: 3 3 3 4 5 3 4 4
16432 089/300 Write block 3702 len 20480 data: 2 3 3 3 2
16433 090/300 Write block 4301 len 12288 data: 2 3 3
16434 091/300 Write block 2277 len 8192 data: 4 5
16435 092/300 Write block 235 len 32768 data: 3 3 3 3 3 4 3 3
16436 093/300 Read block 2820 len 4096
16437 094/300 Read block 2022 len 20480
16438 095/300 Flush
16439 096/300 Write block 4344 len 20480 data: 5 5 5 6 5
16440 097/300 Write block 3303 len 40960 data: 5 5 5 6 5 4 4 4 4 3
16441 098/300 Write block 520 len 28672 data: 6 4 4 4 4 4 4
16442 099/300 Flush
16443 100/300 Read block 1416 len 8192
16444 101/300 Read block 1562 len 20480
16445 102/300 Write block 1634 len 32768 data: 5 5 5 6 8 7 8 8
16446 103/300 Write block 1856 len 8192 data: 3 3
16447 104/300 Read block 3841 len 8192
16448 105/300 Write block 1961 len 24576 data: 2 2 2 2 2 2
16449 106/300 Read block 3285 len 40960
16450 107/300 Read block 1481 len 36864
16451 108/300 Flush
16452 109/300 Write block 3096 len 4096 data: 4
16453 110/300 Read block 4793 len 12288
16454 111/300 Write block 769 len 16384 data: 4 4 3 3
16455 112/300 Read block 3895 len 28672
16456 113/300 Write block 2407 len 36864 data: 4 4 5 4 5 5 5 5 4
16457 114/300 Read block 3661 len 40960
16458 115/300 Read block 925 len 16384
16459 116/300 Write block 2529 len 4096 data: 4
16460 117/300 Write block 3143 len 16384 data: 3 2 4 3
16461 118/300 Write block 4800 len 24576 data: 3 3 3 2 2 3
16462 119/300 Read block 3093 len 40960
16463 120/300 Read block 443 len 36864
16464 121/300 Read block 4503 len 20480
16465 122/300 Read block 4318 len 24576
16466 123/300 Read block 4761 len 16384
16467 124/300 Read block 1288 len 16384
16468 125/300 Write block 2289 len 4096 data: 5
16469 126/300 Write block 4657 len 12288 data: 5 4 4
16470 127/300 Read block 4778 len 16384
16471 128/300 Read block 1792 len 20480
16472 129/300 Read block 3125 len 16384
16473 130/300 Read block 4541 len 28672
16474 131/300 Read block 2681 len 4096
16475 132/300 Read block 2387 len 28672
16476 133/300 Read block 512 len 4096
16477 134/300 Write block 2494 len 40960 data: 3 2 3 4 4 5 4 4 4 5
16478 135/300 Read block 211 len 32768
16479 136/300 Read block 2008 len 4096
16480 137/300 Read block 2708 len 20480
16481 138/300 Read block 4362 len 16384
16482 139/300 Read block 3874 len 28672
16483 140/300 Read block 4104 len 4096
16484 141/300 Write block 1073 len 4096 data: 3
16485 142/300 Write block 51 len 16384 data: 5 4 4 2
16486 143/300 Write block 3297 len 24576 data: 5 3 5 5 5 5
16487 144/300 Write block 17 len 36864 data: 2 2 2 2 2 2 2 2 4
16488 145/300 Write block 3492 len 40960 data: 3 3 3 3 3 4 5 4 5 5
16489 146/300 Write block 2777 len 20480 data: 3 2 2 5 4
16490 147/300 Write block 1545 len 32768 data: 7 6 5 3 2 2 2 2
16491 148/300 Write block 159 len 24576 data: 3 3 3 3 3 3
16492 149/300 Read block 2948 len 4096
16493 150/300 Write block 1536 len 24576 data: 4 3 3 4 4 5
16494 151/300 Read block 2431 len 20480
16495 152/300 Write block 2746 len 12288 data: 4 4 3
16496 153/300 Read block 2130 len 36864
16497 154/300 Write block 2780 len 8192 data: 6 5
16498 155/300 Read block 429 len 20480
16499 156/300 Write block 3747 len 12288 data: 4 4 4
16500 157/300 Read block 4562 len 40960
16501 158/300 Write block 2349 len 36864 data: 6 6 4 4 4 4 3 3 4
16502 159/300 Read block 452 len 28672
16503 160/300 Flush
16504 161/300 Read block 1758 len 24576
16505 162/300 Write block 2950 len 24576 data: 3 3 4 4 3 3
16506 163/300 Read block 4067 len 32768
16507 164/300 Write block 538 len 36864 data: 3 2 2 2 2 2 2 2 2
16508 165/300 Write block 3842 len 32768 data: 3 3 6 5 3 3 2 2
16509 166/300 Write block 1182 len 20480 data: 3 4 3 2 3
16510 167/300 Read block 4365 len 32768
16511 168/300 Flush
16512 169/300 Write block 1162 len 36864 data: 4 5 5 4 4 4 4 5 8
16513 170/300 Write block 2852 len 12288 data: 5 3 3
16514 171/300 Write block 1251 len 32768 data: 2 3 3 3 3 3 3 3
16515 172/300 Write block 1250 len 4096 data: 2
16516 173/300 Write block 3694 len 40960 data: 4 3 4 4 4 3 3 2 3 4
16517 174/300 Read block 4710 len 20480
16518 175/300 Read block 4459 len 12288
16519 176/300 Write block 2205 len 28672 data: 4 4 3 3 4 3 3
16520 177/300 Read block 1501 len 36864
16521 178/300 Flush
16522 179/300 Flush
16523 180/300 Read block 1051 len 20480
16524 181/300 Read block 4427 len 36864
16525 182/300 Write block 1769 len 8192 data: 3 3
16526 183/300 Write block 1342 len 40960 data: 5 5 5 5 6 7 6 5 5 4
16527 184/300 Write block 4409 len 24576 data: 4 5 5 4 4 4
16528 185/300 Write block 2420 len 4096 data: 3
16529 186/300 Write block 3152 len 36864 data: 4 4 4 5 6 7 6 5 7
16530 187/300 Write block 1998 len 20480 data: 4 4 4 4 3
16531 188/300 Write block 1099 len 8192 data: 3 2
16532 189/300 Write block 2267 len 20480 data: 3 3 3 3 3
16533 190/300 Read block 4741 len 28672
16534 191/300 Write block 2179 len 40960 data: 4 4 5 8 8 7 7 7 6 7
16535 192/300 Flush
16536 193/300 Write block 4134 len 8192 data: 4 3
16537 194/300 Write block 3118 len 12288 data: 4 4 6
16538 195/300 Read block 181 len 20480
16539 196/300 Write block 2520 len 8192 data: 3 4
16540 197/300 Read block 76 len 32768
16541 198/300 Read block 1701 len 36864
16542 199/300 Read block 3802 len 36864
16543 200/300 Write block 3368 len 16384 data: 4 4 5 2
16544 201/300 Write block 2143 len 32768 data: 3 3 3 2 3 3 4 4
16545 202/300 Read block 4162 len 20480
16546 203/300 Read block 3170 len 32768
16547 204/300 Flush
16548 205/300 Write block 895 len 12288 data: 5 3 3
16549 206/300 Write block 1979 len 32768 data: 3 3 4 4 4 4 3 2
16550 207/300 Write block 2099 len 36864 data: 2 2 2 2 2 2 3 3 3
16551 208/300 Read block 4242 len 40960
16552 209/300 Write block 2865 len 8192 data: 7 6
16553 210/300 Read block 3900 len 8192
16554 211/300 Write block 3573 len 8192 data: 3 3
16555 212/300 Read block 4746 len 28672
16556 213/300 Flush
16557 214/300 Read block 19 len 4096
16558 215/300 Write block 1042 len 40960 data: 2 3 3 3 3 3 3 3 3 3
16559 216/300 Write block 4743 len 20480 data: 5 6 6 4 3
16560 217/300 Flush
16561 218/300 Write block 1468 len 12288 data: 4 4 4
16562 219/300 Read block 126 len 24576
16563 220/300 Write block 4987 len 16384 data: 2 2 2 2
16564 221/300 Read block 4544 len 16384
16565 222/300 Read block 4055 len 16384
16566 223/300 Flush
16567 224/300 Read block 617 len 4096
16568 225/300 Flush
16569 226/300 Write block 3832 len 28672 data: 2 2 2 3 4 4 3
16570 227/300 Write block 2185 len 4096 data: 8
16571 228/300 Flush
16572 229/300 Read block 1327 len 28672
16573 230/300 Read block 1193 len 8192
16574 231/300 Read block 788 len 24576
16575 232/300 Read block 4860 len 28672
16576 233/300 Read block 736 len 36864
16577 234/300 Write block 866 len 16384 data: 3 3 3 5
16578 235/300 Read block 1322 len 20480
16579 236/300 Flush
16580 237/300 Write block 174 len 12288 data: 4 4 5
16581 238/300 Read block 550 len 20480
16582 239/300 Write block 2945 len 8192 data: 4 5
16583 240/300 Write block 379 len 36864 data: 5 5 5 5 4 4 2 2 2
16584 241/300 Read block 1256 len 28672
16585 242/300 Write block 4117 len 4096 data: 7
16586 243/300 Write block 1326 len 16384 data: 5 5 5 4
16587 244/300 Read block 3246 len 40960
16588 245/300 Read block 2097 len 28672
16589 246/300 Read block 2187 len 36864
16590 247/300 Write block 3833 len 4096 data: 3
16591 248/300 Write block 1676 len 32768 data: 2 3 5 5 4 5 6 4
16592 249/300 Write block 2479 len 24576 data: 3 3 3 3 3 4
16593 250/300 Write block 4155 len 20480 data: 5 5 5 4 5
16594 251/300 Write block 2620 len 28672 data: 2 2 2 3 3 5 5
16595 252/300 Read block 736 len 36864
16596 253/300 Read block 444 len 40960
16597 254/300 Read block 3307 len 12288
16598 255/300 Write block 2729 len 36864 data: 3 3 4 4 3 3 3 3 3
16599 256/300 Read block 173 len 12288
16600 257/300 Read block 3988 len 24576
16601 258/300 Read block 277 len 32768
16602 259/300 Flush
16603 260/300 Write block 1816 len 8192 data: 4 4
16604 261/300 Write block 1292 len 12288 data: 3 4 4
16605 262/300 Flush
16606 263/300 Read block 2238 len 16384
16607 264/300 Read block 4230 len 16384
16608 265/300 Read block 3308 len 16384
16609 266/300 Read block 258 len 28672
16610 267/300 Flush
16611 268/300 Read block 3482 len 32768
16612 269/300 Write block 2206 len 12288 data: 5 4 4
16613 270/300 Read block 598 len 4096
16614 271/300 Read block 1965 len 20480
16615 272/300 Write block 1468 len 40960 data: 5 5 5 4 3 3 3 4 3 3
16616 273/300 Write block 2298 len 24576 data: 3 3 3 4 5 5
16617 274/300 Read block 1927 len 24576
16618 275/300 Read block 4578 len 12288
16619 276/300 Write block 4869 len 28672 data: 5 5 4 5 5 6 5
16620 277/300 Write block 4882 len 28672 data: 3 2 4 4 4 3 4
16621 278/300 Write block 2628 len 28672 data: 5 5 3 2 3 4 4
16622 279/300 Write block 40 len 32768 data: 5 4 5 5 4 3 3 4
16623 280/300 Read block 3605 len 12288
16624 281/300 Write block 3348 len 16384 data: 3 4 4 4
16625 282/300 Read block 958 len 20480
16626 283/300 Write block 808 len 24576 data: 3 3 3 3 2 2
16627 284/300 Flush
16628 285/300 Write block 3898 len 20480 data: 7 6 6 6 5
16629 286/300 Read block 4932 len 4096
16630 287/300 Read block 2386 len 24576
16631 288/300 Flush
16632 289/300 Read block 1577 len 12288
16633 290/300 Write block 3521 len 24576 data: 2 3 4 4 4 4
16634 291/300 Read block 2510 len 20480
16635 292/300 Write block 2917 len 40960 data: 7 8 8 8 9 8 7 6 6 4
16636 293/300 Write block 2683 len 36864 data: 4 3 3 3 3 3 3 2 2
16637 294/300 Flush
16638 295/300 Read block 753 len 16384
16639 296/300 Read block 2376 len 32768
16640 297/300 Read block 2713 len 12288
16641 298/300 Read block 1786 len 40960
16642 299/300 Write block 3143 len 8192 data: 4 3
16643 300/300 Write block 3261 len 24576 data: 3 4 3 4 4 4
16644 301/300 Read block 3679 len 4096
16645 Main task has work to do, go find it
16646 got dsc Start(1)
16647 start 1
16648 Jan 15 05:22:30.617 INFO request completed, latency_us: 185, response_code: 204, uri: /start/cid/1, method: POST, req_id: 65d7daa4-d5a8-4761-8848-edf70ee79a95, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
16649 Replay: started 1, returned:Ok(())
16650 ----------------------------------------------------------------
16651 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16652 States: Active Offline Active
16653 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16654 5227 Acked Write 9 Done Sent Done false
16655 5228 Acked Read 8 Done Sent Done false
16656 5229 Acked Write 1 Done Sent Done false
16657 5230 Acked Write 8 Done Sent Done false
16658 5231 Acked Write 8 Done Sent Done false
16659 5232 Acked Read 4 Done Sent Done false
16660 5233 Acked Read 3 Done Sent Done false
16661 5234 Acked Write 4 Done Sent Done false
16662 5235 Acked Read 10 Done Sent Done false
16663 5236 Acked Read 9 Done Sent Done false
16664 5237 Acked Read 4 Done Sent Done false
16665 5238 Acked Write 1 Done Sent Done false
16666 5239 Acked Read 9 Done Sent Done false
16667 5240 Acked Flush 0 Done Sent Done false
16668 5241 Acked Read 3 Done Sent Done false
16669 5242 Acked Write 3 Done Sent Done false
16670 5243 Acked Flush 0 Done Sent Done false
16671 5244 Acked Read 5 Done Sent Done false
16672 5245 Acked Write 1 Done Sent Done false
16673 5246 Acked Read 9 Done Sent Done false
16674 5247 Acked Write 7 Done Sent Done false
16675 5248 Acked Read 5 Done Sent Done false
16676 5249 Acked Write 8 Done Sent Done false
16677 5250 Acked Read 7 Done Sent Done false
16678 5251 Acked Read 2 Done Sent Done false
16679 5252 Acked Read 6 Done Sent Done false
16680 5253 Acked Write 5 Done Sent Done false
16681 5254 Acked Read 2 Done Sent Done false
16682 5255 Acked Read 3 Done Sent Done false
16683 5256 Acked Write 8 Done Sent Done false
16684 5257 Acked Read 1 Done Sent Done false
16685 5258 Acked Read 4 Done Sent Done false
16686 5259 Acked Read 4 Done Sent Done false
16687 5260 Acked Write 9 Done Sent Done false
16688 5261 Acked Read 10 Done Sent Done false
16689 5262 Acked Read 2 Done Sent Done false
16690 5263 Acked Read 7 Done Sent Done false
16691 5264 Acked Read 9 Done Sent Done false
16692 5265 Acked Read 8 Done Sent Done false
16693 5266 Acked Read 10 Done Sent Done false
16694 5267 Acked Write 2 Done Sent Done false
16695 5268 Acked Write 1 Done Sent Done false
16696 5269 Acked Write 10 Done Sent Done false
16697 5270 Acked Read 3 Done Sent Done false
16698 5271 Acked Read 2 Done Sent Done false
16699 5272 Acked Flush 0 Done Sent Done false
16700 5273 Acked Read 9 Done Sent Done false
16701 5274 Acked Read 9 Done Sent Done false
16702 5275 Acked Write 6 Done Sent Done false
16703 5276 Acked Write 9 Done Sent Done false
16704 5277 Acked Flush 0 Done Sent Done false
16705 5278 Acked Read 7 Done Sent Done false
16706 5279 Acked Write 5 Done Sent Done false
16707 5280 Acked Read 6 Done Sent Done false
16708 5281 Acked Read 6 Done Sent Done false
16709 5282 Acked Read 4 Done Sent Done false
16710 5283 Acked Read 2 Done Sent Done false
16711 5284 Acked Write 5 Done Sent Done false
16712 5285 Acked Read 6 Done Sent Done false
16713 5286 Acked Read 6 Done Sent Done false
16714 5287 Acked Read 1 Done Sent Done false
16715 5288 Acked Read 9 Done Sent Done false
16716 5289 Acked Write 10 Done Sent Done false
16717 5290 Acked Read 7 Done Sent Done false
16718 5291 Acked Write 9 Done Sent Done false
16719 5292 Acked Read 1 Done Sent Done false
16720 5293 Acked Read 1 Done Sent Done false
16721 5294 Acked Read 10 Done Sent Done false
16722 5295 Acked Read 4 Done Sent Done false
16723 5296 Acked Read 8 Done Sent Done false
16724 5297 Acked Write 10 Done Sent Done false
16725 5298 Acked Write 9 Done Sent Done false
16726 5299 Acked Read 2 Done Sent Done false
16727 5300 Acked Write 1 Done Sent Done false
16728 5301 Acked Read 3 Done Sent Done false
16729 5302 Acked Read 6 Done Sent Done false
16730 5303 Acked Flush 0 Done Sent Done false
16731 5304 Acked Write 1 Done Sent Done false
16732 5305 Acked Read 3 Done Sent Done false
16733 5306 Acked Read 1 Done Sent Done false
16734 5307 Acked Write 7 Done Sent Done false
16735 5308 Acked Read 6 Done Sent Done false
16736 5309 Acked Write 2 Done Sent Done false
16737 5310 Acked Write 8 Done Sent Done false
16738 5311 Acked Write 10 Done Sent Done false
16739 5312 Acked Write 3 Done Sent Done false
16740 5313 Acked Read 1 Done Sent Done false
16741 5314 Acked Write 8 Done Sent Done false
16742 5315 Acked Write 5 Done Sent Done false
16743 5316 Acked Write 3 Done Sent Done false
16744 5317 Acked Write 2 Done Sent Done false
16745 5318 Acked Write 8 Done Sent Done false
16746 5319 Acked Read 1 Done Sent Done false
16747 5320 Acked Read 5 Done Sent Done false
16748 5321 Acked Flush 0 Done Sent Done false
16749 5322 Acked Write 5 Done Sent Done false
16750 5323 Acked Write 10 Done Sent Done false
16751 5324 Acked Write 7 Done Sent Done false
16752 5325 Acked Flush 0 Done Sent Done false
16753 5326 Acked Read 2 Done Sent Done false
16754 5327 Acked Read 5 Done Sent Done false
16755 5328 Acked Write 8 Done Sent Done false
16756 5329 Acked Write 2 Done Sent Done false
16757 5330 Acked Read 2 Done Sent Done false
16758 5331 Acked Write 6 Done Sent Done false
16759 5332 Acked Read 10 Done Sent Done false
16760 5333 Acked Read 9 Done Sent Done false
16761 5334 Acked Flush 0 Done Sent Done false
16762 5335 Acked Write 1 Done Sent Done false
16763 5336 Acked Read 3 Done Sent Done false
16764 5337 Acked Write 4 Done Sent Done false
16765 5338 Acked Read 7 Done Sent Done false
16766 5339 Acked Write 9 Done Sent Done false
16767 5340 Acked Read 10 Done Sent Done false
16768 5341 Acked Read 4 Done Sent Done false
16769 5342 Acked Write 1 Done Sent Done false
16770 5343 Acked Write 4 Done Sent Done false
16771 5344 Acked Write 6 Done Sent Done false
16772 5345 Acked Read 10 Done Sent Done false
16773 5346 Acked Read 9 Done Sent Done false
16774 5347 Acked Read 5 Done Sent Done false
16775 5348 Acked Read 6 Done Sent Done false
16776 5349 Acked Read 4 Done Sent Done false
16777 5350 Acked Read 4 Done Sent Done false
16778 5351 Acked Write 1 Done Sent Done false
16779 5352 Acked Write 3 Done Sent Done false
16780 5353 Acked Read 4 Done Sent Done false
16781 5354 Acked Read 5 Done Sent Done false
16782 5355 Acked Read 4 Done Sent Done false
16783 5356 Acked Read 7 Done Sent Done false
16784 5357 Acked Read 1 Done Sent Done false
16785 5358 Acked Read 7 Done Sent Done false
16786 5359 Acked Read 1 Done Sent Done false
16787 5360 Acked Write 10 Done Sent Done false
16788 5361 Acked Read 8 Done Sent Done false
16789 5362 Acked Read 1 Done Sent Done false
16790 5363 Acked Read 5 Done Sent Done false
16791 5364 Acked Read 4 Done Sent Done false
16792 5365 Acked Read 7 Done Sent Done false
16793 5366 Acked Read 1 Done Sent Done false
16794 5367 Acked Write 1 Done Sent Done false
16795 5368 Acked Write 4 Done Sent Done false
16796 5369 Acked Write 6 Done Sent Done false
16797 5370 Acked Write 9 Done Sent Done false
16798 5371 Acked Write 10 Done Sent Done false
16799 5372 Acked Write 5 Done Sent Done false
16800 5373 Acked Write 8 Done Sent Done false
16801 5374 Acked Write 6 Done Sent Done false
16802 5375 Acked Read 1 Done Sent Done false
16803 5376 Acked Write 6 Done Sent Done false
16804 5377 Acked Read 5 Done Sent Done false
16805 5378 Acked Write 3 Done Sent Done false
16806 5379 Acked Read 9 Done Sent Done false
16807 5380 Acked Write 2 Done Sent Done false
16808 5381 Acked Read 5 Done Sent Done false
16809 5382 Acked Write 3 Done Sent Done false
16810 5383 Acked Read 10 Done Sent Done false
16811 5384 Acked Write 9 Done Sent Done false
16812 5385 Acked Read 7 Done Sent Done false
16813 5386 Acked Flush 0 Done Sent Done false
16814 5387 Acked Read 6 Done Sent Done false
16815 5388 Acked Write 6 Done Sent Done false
16816 5389 Acked Read 8 Done Sent Done false
16817 5390 Acked Write 9 Done Sent Done false
16818 5391 Acked Write 8 Done Sent Done false
16819 5392 Acked Write 5 Done Sent Done false
16820 5393 Acked Read 8 Done Sent Done false
16821 5394 Acked Flush 0 Done Sent Done false
16822 5395 Acked Write 9 Done Sent Done false
16823 5396 Acked Write 3 Done Sent Done false
16824 5397 Acked Write 8 Done Sent Done false
16825 5398 Acked Write 1 Done Sent Done false
16826 5399 Acked Write 10 Done Sent Done false
16827 5400 Acked Read 5 Done Sent Done false
16828 5401 Acked Read 3 Done Sent Done false
16829 5402 Acked Write 7 Done Sent Done false
16830 5403 Acked Read 9 Done Sent Done false
16831 5404 Acked Flush 0 Done Sent Done false
16832 5405 Acked Flush 0 Done Sent Done false
16833 5406 Acked Read 5 Done Sent Done false
16834 5407 Acked Read 9 Done Sent Done false
16835 5408 Acked Write 2 Done Sent Done false
16836 5409 Acked Write 10 Done Sent Done false
16837 5410 Acked Write 6 Done Sent Done false
16838 5411 Acked Write 1 Done Sent Done false
16839 5412 Acked Write 9 Done Sent Done false
16840 5413 Acked Write 5 Done Sent Done false
16841 5414 Acked Write 2 Done Sent Done false
16842 5415 Acked Write 5 Done Sent Done false
16843 5416 Acked Read 7 Done Sent Done false
16844 5417 Acked Write 10 Done Sent Done false
16845 5418 Acked Flush 0 Done Sent Done false
16846 5419 Acked Write 2 Done Sent Done false
16847 5420 Acked Write 3 Done Sent Done false
16848 5421 Acked Read 5 Done Sent Done false
16849 5422 Acked Write 2 Done Sent Done false
16850 5423 Acked Read 8 Done Sent Done false
16851 5424 Acked Read 9 Done Sent Done false
16852 5425 Acked Read 9 Done Sent Done false
16853 5426 Acked Write 4 Done Sent Done false
16854 5427 Acked Write 8 Done Sent Done false
16855 5428 Acked Read 5 Done Sent Done false
16856 5429 Acked Read 8 Done Sent Done false
16857 5430 Acked Flush 0 Done Sent Done false
16858 5431 Acked Write 3 Done Sent Done false
16859 5432 Acked Write 8 Done Sent Done false
16860 5433 Acked Write 9 Done Sent Done false
16861 5434 Acked Read 10 Done Sent Done false
16862 5435 Acked Write 2 Done Sent Done false
16863 5436 Acked Read 2 Done Sent Done false
16864 5437 Acked Write 2 Done Sent Done false
16865 5438 Acked Read 7 Done Sent Done false
16866 5439 Acked Flush 0 Done Sent Done false
16867 5440 Acked Read 1 Done Sent Done false
16868 5441 Acked Write 10 Done Sent Done false
16869 5442 Acked Write 5 Done Sent Done false
16870 5443 Acked Flush 0 Done Sent Done false
16871 5444 Acked Write 3 Done Sent Done false
16872 5445 Acked Read 6 Done Sent Done false
16873 5446 Acked Write 4 Done Sent Done false
16874 5447 Acked Read 4 Done Sent Done false
16875 5448 Acked Read 4 Done Sent Done false
16876 5449 Acked Flush 0 Done Sent Done false
16877 5450 Acked Read 1 Done Sent Done false
16878 5451 Acked Flush 0 Done Sent Done false
16879 5452 Acked Write 7 Done Sent Done false
16880 5453 Acked Write 1 Done Sent Done false
16881 5454 Acked Flush 0 Done Sent Done false
16882 5455 Acked Read 7 Done Sent Done false
16883 5456 Acked Read 2 Done Sent Done false
16884 5457 Acked Read 6 Done Sent Done false
16885 5458 Acked Read 7 Done Sent Done false
16886 5459 Acked Read 9 Done Sent Done false
16887 5460 Acked Write 4 Done Sent Done false
16888 5461 Acked Read 5 Done Sent Done false
16889 5462 Acked Flush 0 Done Sent Done false
16890 5463 Acked Write 3 Done Sent Done false
16891 5464 Acked Read 5 Done Sent Done false
16892 5465 Acked Write 2 Done Sent Done false
16893 5466 Acked Write 9 Done Sent Done false
16894 5467 Acked Read 7 Done Sent Done false
16895 5468 Acked Write 1 Done Sent Done false
16896 5469 Acked Write 4 Done Sent Done false
16897 5470 Acked Read 10 Done Sent Done false
16898 5471 Acked Read 7 Done Sent Done false
16899 5472 Acked Read 9 Done Sent Done false
16900 5473 Acked Write 1 Done Sent Done false
16901 5474 Acked Write 8 Done Sent Done false
16902 5475 Acked Write 6 Done Sent Done false
16903 5476 Acked Write 5 Done Sent Done false
16904 5477 Acked Write 7 Done Sent Done false
16905 5478 Acked Read 9 Done Sent Done false
16906 5479 Acked Read 10 Done Sent Done false
16907 5480 Acked Read 3 Done Sent Done false
16908 5481 Acked Write 9 Done Sent Done false
16909 5482 Acked Read 3 Done Sent Done false
16910 5483 Acked Read 6 Done Sent Done false
16911 5484 Acked Read 8 Done Sent Done false
16912 5485 Acked Flush 0 Done Sent Done false
16913 5486 Acked Write 2 Done Sent Done false
16914 5487 Acked Write 3 Done Sent Done false
16915 5488 Acked Flush 0 Done Sent Done false
16916 5489 Acked Read 4 Done Sent Done false
16917 5490 Acked Read 4 Done Sent Done false
16918 5491 Acked Read 4 Done Sent Done false
16919 5492 Acked Read 7 Done Sent Done false
16920 5493 Acked Flush 0 Done Sent Done false
16921 5494 Acked Read 8 Done Sent Done false
16922 5495 Acked Write 3 Done Sent Done false
16923 5496 Acked Read 1 Done Sent Done false
16924 5497 Acked Read 5 Done Sent Done false
16925 5498 Acked Write 10 Done Sent Done false
16926 5499 Acked Write 6 Done Sent Done false
16927 5500 Acked Read 6 Done Sent Done false
16928 5501 Acked Read 3 Done Sent Done false
16929 5502 Acked Write 7 Done Sent Done false
16930 5503 Acked Write 7 Done Sent Done false
16931 5504 Acked Write 7 Done Sent Done false
16932 5505 Acked Write 8 Done Sent Done false
16933 5506 Acked Read 3 Done Sent Done false
16934 5507 Acked Write 4 Done Sent Done false
16935 5508 Acked Read 5 Done Sent Done false
16936 5509 Acked Write 6 Done Sent Done false
16937 5510 Acked Flush 0 Done Sent Done false
16938 5511 Acked Write 5 Done Sent Done false
16939 5512 Acked Read 1 Done Sent Done false
16940 5513 Acked Read 6 Done Sent Done false
16941 5514 Acked Flush 0 Done Sent Done false
16942 5515 Acked Read 3 Done Sent Done false
16943 5516 Acked Write 6 Done Sent Done false
16944 5517 Acked Read 5 Done Sent Done false
16945 5518 Acked Write 10 Done Sent Done false
16946 5519 Acked Write 9 Done Sent Done false
16947 5520 Acked Flush 0 Done Sent Done false
16948 5521 Acked Read 4 Done Sent Done false
16949 5522 Acked Read 8 Done Sent Done false
16950 5523 Acked Read 3 Done Sent Done false
16951 5524 Acked Read 10 Done Sent Done false
16952 5525 Acked Write 2 Done Sent Done false
16953 5526 Acked Write 6 Done Sent Done false
16954 5527 Acked Read 1 Done Sent Done false
16955 STATES DS:0 DS:1 DS:2 TOTAL
16956 Sent 0 301 0 301
16957 Done 301 0 301 602
16958 Skipped 0 0 0 0
16959 Error 0 0 0 0
16960 Last Flush: 5520 5226 5520
16961 Downstairs last five completed: 5226 5225 5224 5223 5222
16962 Upstairs last five completed: 5527 5526 5525 5524 5523
16963 CLIENT: Up:0 ds:301 act:2
16964 [8820] got start action
16965 [8820] I am going to restart
16966 Starting downstairs at port 8820
16967 Make output file at "/tmp/dsc/downstairs-8820.txt"
16968 [8820][1] reports Starting
16969 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1217)
16970 [8820][1] reports Running
16971 ----------------------------------------------------------------
16972 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16973 States: Active Offline Active
16974 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16975 5227 Acked Write 9 Done Sent Done false
16976 5228 Acked Read 8 Done Sent Done false
16977 5229 Acked Write 1 Done Sent Done false
16978 5230 Acked Write 8 Done Sent Done false
16979 5231 Acked Write 8 Done Sent Done false
16980 5232 Acked Read 4 Done Sent Done false
16981 5233 Acked Read 3 Done Sent Done false
16982 5234 Acked Write 4 Done Sent Done false
16983 5235 Acked Read 10 Done Sent Done false
16984 5236 Acked Read 9 Done Sent Done false
16985 5237 Acked Read 4 Done Sent Done false
16986 5238 Acked Write 1 Done Sent Done false
16987 5239 Acked Read 9 Done Sent Done false
16988 5240 Acked Flush 0 Done Sent Done false
16989 5241 Acked Read 3 Done Sent Done false
16990 5242 Acked Write 3 Done Sent Done false
16991 5243 Acked Flush 0 Done Sent Done false
16992 5244 Acked Read 5 Done Sent Done false
16993 5245 Acked Write 1 Done Sent Done false
16994 5246 Acked Read 9 Done Sent Done false
16995 5247 Acked Write 7 Done Sent Done false
16996 5248 Acked Read 5 Done Sent Done false
16997 5249 Acked Write 8 Done Sent Done false
16998 5250 Acked Read 7 Done Sent Done false
16999 5251 Acked Read 2 Done Sent Done false
17000 5252 Acked Read 6 Done Sent Done false
17001 5253 Acked Write 5 Done Sent Done false
17002 5254 Acked Read 2 Done Sent Done false
17003 5255 Acked Read 3 Done Sent Done false
17004 5256 Acked Write 8 Done Sent Done false
17005 5257 Acked Read 1 Done Sent Done false
17006 5258 Acked Read 4 Done Sent Done false
17007 5259 Acked Read 4 Done Sent Done false
17008 5260 Acked Write 9 Done Sent Done false
17009 5261 Acked Read 10 Done Sent Done false
17010 5262 Acked Read 2 Done Sent Done false
17011 5263 Acked Read 7 Done Sent Done false
17012 5264 Acked Read 9 Done Sent Done false
17013 5265 Acked Read 8 Done Sent Done false
17014 5266 Acked Read 10 Done Sent Done false
17015 5267 Acked Write 2 Done Sent Done false
17016 5268 Acked Write 1 Done Sent Done false
17017 5269 Acked Write 10 Done Sent Done false
17018 5270 Acked Read 3 Done Sent Done false
17019 5271 Acked Read 2 Done Sent Done false
17020 5272 Acked Flush 0 Done Sent Done false
17021 5273 Acked Read 9 Done Sent Done false
17022 5274 Acked Read 9 Done Sent Done false
17023 5275 Acked Write 6 Done Sent Done false
17024 5276 Acked Write 9 Done Sent Done false
17025 5277 Acked Flush 0 Done Sent Done false
17026 5278 Acked Read 7 Done Sent Done false
17027 5279 Acked Write 5 Done Sent Done false
17028 5280 Acked Read 6 Done Sent Done false
17029 5281 Acked Read 6 Done Sent Done false
17030 5282 Acked Read 4 Done Sent Done false
17031 5283 Acked Read 2 Done Sent Done false
17032 5284 Acked Write 5 Done Sent Done false
17033 5285 Acked Read 6 Done Sent Done false
17034 5286 Acked Read 6 Done Sent Done false
17035 5287 Acked Read 1 Done Sent Done false
17036 5288 Acked Read 9 Done Sent Done false
17037 5289 Acked Write 10 Done Sent Done false
17038 5290 Acked Read 7 Done Sent Done false
17039 5291 Acked Write 9 Done Sent Done false
17040 5292 Acked Read 1 Done Sent Done false
17041 5293 Acked Read 1 Done Sent Done false
17042 5294 Acked Read 10 Done Sent Done false
17043 5295 Acked Read 4 Done Sent Done false
17044 5296 Acked Read 8 Done Sent Done false
17045 5297 Acked Write 10 Done Sent Done false
17046 5298 Acked Write 9 Done Sent Done false
17047 5299 Acked Read 2 Done Sent Done false
17048 5300 Acked Write 1 Done Sent Done false
17049 5301 Acked Read 3 Done Sent Done false
17050 5302 Acked Read 6 Done Sent Done false
17051 5303 Acked Flush 0 Done Sent Done false
17052 5304 Acked Write 1 Done Sent Done false
17053 5305 Acked Read 3 Done Sent Done false
17054 5306 Acked Read 1 Done Sent Done false
17055 5307 Acked Write 7 Done Sent Done false
17056 5308 Acked Read 6 Done Sent Done false
17057 5309 Acked Write 2 Done Sent Done false
17058 5310 Acked Write 8 Done Sent Done false
17059 5311 Acked Write 10 Done Sent Done false
17060 5312 Acked Write 3 Done Sent Done false
17061 5313 Acked Read 1 Done Sent Done false
17062 5314 Acked Write 8 Done Sent Done false
17063 5315 Acked Write 5 Done Sent Done false
17064 5316 Acked Write 3 Done Sent Done false
17065 5317 Acked Write 2 Done Sent Done false
17066 5318 Acked Write 8 Done Sent Done false
17067 5319 Acked Read 1 Done Sent Done false
17068 5320 Acked Read 5 Done Sent Done false
17069 5321 Acked Flush 0 Done Sent Done false
17070 5322 Acked Write 5 Done Sent Done false
17071 5323 Acked Write 10 Done Sent Done false
17072 5324 Acked Write 7 Done Sent Done false
17073 5325 Acked Flush 0 Done Sent Done false
17074 5326 Acked Read 2 Done Sent Done false
17075 5327 Acked Read 5 Done Sent Done false
17076 5328 Acked Write 8 Done Sent Done false
17077 5329 Acked Write 2 Done Sent Done false
17078 5330 Acked Read 2 Done Sent Done false
17079 5331 Acked Write 6 Done Sent Done false
17080 5332 Acked Read 10 Done Sent Done false
17081 5333 Acked Read 9 Done Sent Done false
17082 5334 Acked Flush 0 Done Sent Done false
17083 5335 Acked Write 1 Done Sent Done false
17084 5336 Acked Read 3 Done Sent Done false
17085 5337 Acked Write 4 Done Sent Done false
17086 5338 Acked Read 7 Done Sent Done false
17087 5339 Acked Write 9 Done Sent Done false
17088 5340 Acked Read 10 Done Sent Done false
17089 5341 Acked Read 4 Done Sent Done false
17090 5342 Acked Write 1 Done Sent Done false
17091 5343 Acked Write 4 Done Sent Done false
17092 5344 Acked Write 6 Done Sent Done false
17093 5345 Acked Read 10 Done Sent Done false
17094 5346 Acked Read 9 Done Sent Done false
17095 5347 Acked Read 5 Done Sent Done false
17096 5348 Acked Read 6 Done Sent Done false
17097 5349 Acked Read 4 Done Sent Done false
17098 5350 Acked Read 4 Done Sent Done false
17099 5351 Acked Write 1 Done Sent Done false
17100 5352 Acked Write 3 Done Sent Done false
17101 5353 Acked Read 4 Done Sent Done false
17102 5354 Acked Read 5 Done Sent Done false
17103 5355 Acked Read 4 Done Sent Done false
17104 5356 Acked Read 7 Done Sent Done false
17105 5357 Acked Read 1 Done Sent Done false
17106 5358 Acked Read 7 Done Sent Done false
17107 5359 Acked Read 1 Done Sent Done false
17108 5360 Acked Write 10 Done Sent Done false
17109 5361 Acked Read 8 Done Sent Done false
17110 5362 Acked Read 1 Done Sent Done false
17111 5363 Acked Read 5 Done Sent Done false
17112 5364 Acked Read 4 Done Sent Done false
17113 5365 Acked Read 7 Done Sent Done false
17114 5366 Acked Read 1 Done Sent Done false
17115 5367 Acked Write 1 Done Sent Done false
17116 5368 Acked Write 4 Done Sent Done false
17117 5369 Acked Write 6 Done Sent Done false
17118 5370 Acked Write 9 Done Sent Done false
17119 5371 Acked Write 10 Done Sent Done false
17120 5372 Acked Write 5 Done Sent Done false
17121 5373 Acked Write 8 Done Sent Done false
17122 5374 Acked Write 6 Done Sent Done false
17123 5375 Acked Read 1 Done Sent Done false
17124 5376 Acked Write 6 Done Sent Done false
17125 5377 Acked Read 5 Done Sent Done false
17126 5378 Acked Write 3 Done Sent Done false
17127 5379 Acked Read 9 Done Sent Done false
17128 5380 Acked Write 2 Done Sent Done false
17129 5381 Acked Read 5 Done Sent Done false
17130 5382 Acked Write 3 Done Sent Done false
17131 5383 Acked Read 10 Done Sent Done false
17132 5384 Acked Write 9 Done Sent Done false
17133 5385 Acked Read 7 Done Sent Done false
17134 5386 Acked Flush 0 Done Sent Done false
17135 5387 Acked Read 6 Done Sent Done false
17136 5388 Acked Write 6 Done Sent Done false
17137 5389 Acked Read 8 Done Sent Done false
17138 5390 Acked Write 9 Done Sent Done false
17139 5391 Acked Write 8 Done Sent Done false
17140 5392 Acked Write 5 Done Sent Done false
17141 5393 Acked Read 8 Done Sent Done false
17142 5394 Acked Flush 0 Done Sent Done false
17143 5395 Acked Write 9 Done Sent Done false
17144 5396 Acked Write 3 Done Sent Done false
17145 5397 Acked Write 8 Done Sent Done false
17146 5398 Acked Write 1 Done Sent Done false
17147 5399 Acked Write 10 Done Sent Done false
17148 5400 Acked Read 5 Done Sent Done false
17149 5401 Acked Read 3 Done Sent Done false
17150 5402 Acked Write 7 Done Sent Done false
17151 5403 Acked Read 9 Done Sent Done false
17152 5404 Acked Flush 0 Done Sent Done false
17153 5405 Acked Flush 0 Done Sent Done false
17154 5406 Acked Read 5 Done Sent Done false
17155 5407 Acked Read 9 Done Sent Done false
17156 5408 Acked Write 2 Done Sent Done false
17157 5409 Acked Write 10 Done Sent Done false
17158 5410 Acked Write 6 Done Sent Done false
17159 5411 Acked Write 1 Done Sent Done false
17160 5412 Acked Write 9 Done Sent Done false
17161 5413 Acked Write 5 Done Sent Done false
17162 5414 Acked Write 2 Done Sent Done false
17163 5415 Acked Write 5 Done Sent Done false
17164 5416 Acked Read 7 Done Sent Done false
17165 5417 Acked Write 10 Done Sent Done false
17166 5418 Acked Flush 0 Done Sent Done false
17167 5419 Acked Write 2 Done Sent Done false
17168 5420 Acked Write 3 Done Sent Done false
17169 5421 Acked Read 5 Done Sent Done false
17170 5422 Acked Write 2 Done Sent Done false
17171 5423 Acked Read 8 Done Sent Done false
17172 5424 Acked Read 9 Done Sent Done false
17173 5425 Acked Read 9 Done Sent Done false
17174 5426 Acked Write 4 Done Sent Done false
17175 5427 Acked Write 8 Done Sent Done false
17176 5428 Acked Read 5 Done Sent Done false
17177 5429 Acked Read 8 Done Sent Done false
17178 5430 Acked Flush 0 Done Sent Done false
17179 5431 Acked Write 3 Done Sent Done false
17180 5432 Acked Write 8 Done Sent Done false
17181 5433 Acked Write 9 Done Sent Done false
17182 5434 Acked Read 10 Done Sent Done false
17183 5435 Acked Write 2 Done Sent Done false
17184 5436 Acked Read 2 Done Sent Done false
17185 5437 Acked Write 2 Done Sent Done false
17186 5438 Acked Read 7 Done Sent Done false
17187 5439 Acked Flush 0 Done Sent Done false
17188 5440 Acked Read 1 Done Sent Done false
17189 5441 Acked Write 10 Done Sent Done false
17190 5442 Acked Write 5 Done Sent Done false
17191 5443 Acked Flush 0 Done Sent Done false
17192 5444 Acked Write 3 Done Sent Done false
17193 5445 Acked Read 6 Done Sent Done false
17194 5446 Acked Write 4 Done Sent Done false
17195 5447 Acked Read 4 Done Sent Done false
17196 5448 Acked Read 4 Done Sent Done false
17197 5449 Acked Flush 0 Done Sent Done false
17198 5450 Acked Read 1 Done Sent Done false
17199 5451 Acked Flush 0 Done Sent Done false
17200 5452 Acked Write 7 Done Sent Done false
17201 5453 Acked Write 1 Done Sent Done false
17202 5454 Acked Flush 0 Done Sent Done false
17203 5455 Acked Read 7 Done Sent Done false
17204 5456 Acked Read 2 Done Sent Done false
17205 5457 Acked Read 6 Done Sent Done false
17206 5458 Acked Read 7 Done Sent Done false
17207 5459 Acked Read 9 Done Sent Done false
17208 5460 Acked Write 4 Done Sent Done false
17209 5461 Acked Read 5 Done Sent Done false
17210 5462 Acked Flush 0 Done Sent Done false
17211 5463 Acked Write 3 Done Sent Done false
17212 5464 Acked Read 5 Done Sent Done false
17213 5465 Acked Write 2 Done Sent Done false
17214 5466 Acked Write 9 Done Sent Done false
17215 5467 Acked Read 7 Done Sent Done false
17216 5468 Acked Write 1 Done Sent Done false
17217 5469 Acked Write 4 Done Sent Done false
17218 5470 Acked Read 10 Done Sent Done false
17219 5471 Acked Read 7 Done Sent Done false
17220 5472 Acked Read 9 Done Sent Done false
17221 5473 Acked Write 1 Done Sent Done false
17222 5474 Acked Write 8 Done Sent Done false
17223 5475 Acked Write 6 Done Sent Done false
17224 5476 Acked Write 5 Done Sent Done false
17225 5477 Acked Write 7 Done Sent Done false
17226 5478 Acked Read 9 Done Sent Done false
17227 5479 Acked Read 10 Done Sent Done false
17228 5480 Acked Read 3 Done Sent Done false
17229 5481 Acked Write 9 Done Sent Done false
17230 5482 Acked Read 3 Done Sent Done false
17231 5483 Acked Read 6 Done Sent Done false
17232 5484 Acked Read 8 Done Sent Done false
17233 5485 Acked Flush 0 Done Sent Done false
17234 5486 Acked Write 2 Done Sent Done false
17235 5487 Acked Write 3 Done Sent Done false
17236 5488 Acked Flush 0 Done Sent Done false
17237 5489 Acked Read 4 Done Sent Done false
17238 5490 Acked Read 4 Done Sent Done false
17239 5491 Acked Read 4 Done Sent Done false
17240 5492 Acked Read 7 Done Sent Done false
17241 5493 Acked Flush 0 Done Sent Done false
17242 5494 Acked Read 8 Done Sent Done false
17243 5495 Acked Write 3 Done Sent Done false
17244 5496 Acked Read 1 Done Sent Done false
17245 5497 Acked Read 5 Done Sent Done false
17246 5498 Acked Write 10 Done Sent Done false
17247 5499 Acked Write 6 Done Sent Done false
17248 5500 Acked Read 6 Done Sent Done false
17249 5501 Acked Read 3 Done Sent Done false
17250 5502 Acked Write 7 Done Sent Done false
17251 5503 Acked Write 7 Done Sent Done false
17252 5504 Acked Write 7 Done Sent Done false
17253 5505 Acked Write 8 Done Sent Done false
17254 5506 Acked Read 3 Done Sent Done false
17255 5507 Acked Write 4 Done Sent Done false
17256 5508 Acked Read 5 Done Sent Done false
17257 5509 Acked Write 6 Done Sent Done false
17258 5510 Acked Flush 0 Done Sent Done false
17259 5511 Acked Write 5 Done Sent Done false
17260 5512 Acked Read 1 Done Sent Done false
17261 5513 Acked Read 6 Done Sent Done false
17262 5514 Acked Flush 0 Done Sent Done false
17263 5515 Acked Read 3 Done Sent Done false
17264 5516 Acked Write 6 Done Sent Done false
17265 5517 Acked Read 5 Done Sent Done false
17266 5518 Acked Write 10 Done Sent Done false
17267 5519 Acked Write 9 Done Sent Done false
17268 5520 Acked Flush 0 Done Sent Done false
17269 5521 Acked Read 4 Done Sent Done false
17270 5522 Acked Read 8 Done Sent Done false
17271 5523 Acked Read 3 Done Sent Done false
17272 5524 Acked Read 10 Done Sent Done false
17273 5525 Acked Write 2 Done Sent Done false
17274 5526 Acked Write 6 Done Sent Done false
17275 5527 Acked Read 1 Done Sent Done false
17276 5528 Acked Flush 0 Done Sent Done false
17277 STATES DS:0 DS:1 DS:2 TOTAL
17278 Sent 0 302 0 302
17279 Done 302 0 302 604
17280 Skipped 0 0 0 0
17281 Error 0 0 0 0
17282 Last Flush: 5528 5226 5528
17283 Downstairs last five completed: 5226 5225 5224 5223 5222
17284 Upstairs last five completed: 5528 5527 5526 5525 5524
17285 CLIENT: Up:0 ds:302 act:2
172862025-01-15T05:22:36.386ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
172872025-01-15T05:22:36.386ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42118) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
172882025-01-15T05:22:36.387ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
172892025-01-15T05:22:36.388ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
172902025-01-15T05:22:36.388ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
172912025-01-15T05:22:36.388ZINFOcrucible: send last flush ID to this DS: 5226 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
172922025-01-15T05:22:36.388ZINFOcrucible: Replied this last flush ID: 5226 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
172932025-01-15T05:22:36.388ZINFOcrucible: [1] client re-new 302 jobs since flush 5226 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
172942025-01-15T05:22:36.388ZINFOcrucible: [1] Replayed 302 jobs since flush: 5226 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
17295 ----------------------------------------------------------------
17296 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17297 Downstairs last five completed: 5528 5527 5526 5525 5524
17298 Upstairs last five completed: 5528 5527 5526 5525 5524
17299 CLIENT: Up:0 ds:0 act:3
17300 Replay: All jobs finished
17301 Main task has work to do, go find it
17302 got dsc Stop(2)
17303 stop 2
17304 Jan 15 05:22:38.623 INFO request completed[8830] Got stop action so:false kr:false
17305 , latency_us: 376, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 7a6e5f82-931b-49cc-9670-e9d0221c3409, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
173062025-01-15T05:22:38.623ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
17307 Jan 15 05:22:38.624 INFO request completed, latency_us: 121, response_code: 200, uri: /state/cid/2, method: GET, req_id: c0d1af99-6271-4433-8b44-cfce23db71c6, remote_addr: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:22:38.623996679Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
17308 127.0.0.1:42402, local_addr: 127.0.0.1:9998
173092025-01-15T05:22:38.624ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
173102025-01-15T05:22:38.624ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
17311 [8830] Exited with: ExitStatus(unix_wait_status(9))
17312 [8830][2] reports Exit
17313 Jan 15 05:22:42.626 INFO request completed, latency_us: 194, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9785f01d-f6a2-4d49-9840-112b9650c80e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
17314 001/300 Read block 1977 len 8192
17315 002/300 Read block 2308 len 36864
17316 003/300 Read block 4471 len 28672
17317 004/300 Write block 1538 len 16384 data: 4 5 5 6
17318 005/300 Read block 2730 len 20480
17319 006/300 Read block 4400 len 40960
17320 007/300 Flush
17321 008/300 Read block 1329 len 4096
17322 009/300 Write block 4495 len 12288 data: 5 5 5
17323 010/300 Flush
17324 011/300 Read block 2872 len 28672
17325 012/300 Flush
17326 013/300 Write block 634 len 32768 data: 5 4 5 5 4 6 5 5
17327 014/300 Read block 1747 len 16384
17328 015/300 Flush
17329 016/300 Read block 3988 len 4096
17330 017/300 Write block 4736 len 36864 data: 4 4 3 4 5 4 4 6 7
17331 018/300 Flush
17332 019/300 Write block 3672 len 16384 data: 8 8 8 6
17333 020/300 Read block 4598 len 8192
17334 021/300 Read block 525 len 24576
17335 022/300 Read block 3898 len 28672
17336 023/300 Write block 738 len 36864 data: 4 4 5 5 4 5 6 4 5
17337 024/300 Read block 1544 len 36864
17338 025/300 Read block 3176 len 32768
17339 026/300 Read block 230 len 28672
17340 027/300 Write block 3379 len 32768 data: 3 4 5 8 7 6 6 6
17341 028/300 Read block 227 len 40960
17342 029/300 Read block 912 len 28672
17343 030/300 Write block 3143 len 8192 data: 5 4
17344 031/300 Write block 1645 len 8192 data: 5 4
17345 032/300 Write block 4780 len 8192 data: 4 4
17346 033/300 Read block 2653 len 8192
17347 034/300 Read block 3979 len 28672
17348 035/300 Read block 848 len 12288
17349 036/300 Write block 1409 len 24576 data: 3 3 3 3 3 3
17350 037/300 Read block 565 len 12288
17351 038/300 Write block 379 len 28672 data: 6 6 6 6 5 5 3
17352 039/300 Read block 4904 len 4096
17353 040/300 Read block 530 len 40960
17354 041/300 Read block 447 len 16384
17355 042/300 Read block 2813 len 36864
17356 043/300 Read block 3858 len 24576
17357 044/300 Read block 3774 len 12288
17358 045/300 Write block 3123 len 32768 data: 4 5 4 4 5 7 5 5
17359 046/300 Write block 3249 len 20480 data: 3 5 5 5 4
17360 047/300 Read block 649 len 8192
17361 048/300 Flush
17362 049/300 Flush
17363 050/300 Write block 2604 len 32768 data: 4 5 4 4 3 4 4 4
17364 051/300 Read block 1083 len 28672
17365 052/300 Write block 2256 len 16384 data: 5 5 5 6
17366 053/300 Read block 3269 len 32768
17367 054/300 Read block 1949 len 24576
17368 055/300 Read block 2606 len 40960
17369 056/300 Read block 1807 len 16384
17370 057/300 Write block 3277 len 24576 data: 5 5 5 4 3 3
17371 058/300 Write block 3497 len 12288 data: 5 6 5
17372 059/300 Write block 1299 len 20480 data: 3 4 5 4 4
17373 060/300 Read block 4337 len 8192
17374 061/300 Read block 2368 len 4096
17375 062/300 Write block 1603 len 36864 data: 6 5 5 6 5 5 4 4 3
17376 063/300 Write block 4618 len 36864 data: 5 5 5 5 5 5 3 4 4
17377 064/300 Write block 2353 len 40960 data: 5 5 4 4 5 3 3 2 2 5
17378 065/300 Read block 2704 len 20480
17379 066/300 Read block 180 len 12288
17380 067/300 Read block 3869 len 32768
17381 068/300 Read block 4783 len 12288
17382 069/300 Write block 4917 len 8192 data: 5 3
17383 070/300 Read block 741 len 4096
17384 071/300 Write block 1434 len 20480 data: 6 5 5 4 4
17385 072/300 Write block 4114 len 40960 data: 5 5 5 8 8 8 9 7 6 3
17386 073/300 Write block 2723 len 8192 data: 2 2
17387 074/300 Read block 2524 len 40960
17388 075/300 Write block 4215 len 4096 data: 4
17389 076/300 Write block 2539 len 8192 data: 3 4
17390 077/300 Read block 2919 len 8192
17391 078/300 Read block 1807 len 4096
17392 079/300 Write block 533 len 20480 data: 4 3 3 3 3
17393 080/300 Write block 2768 len 28672 data: 3 2 2 2 2 2 2
17394 081/300 Read block 2406 len 40960
17395 082/300 Write block 3694 len 28672 data: 5 4 5 5 5 4 4
17396 083/300 Read block 3071 len 8192
17397 084/300 Read block 1382 len 12288
17398 085/300 Read block 568 len 32768
17399 086/300 Write block 589 len 40960 data: 3 3 3 3 4 6 6 5 3 3
17400 087/300 Write block 604 len 32768 data: 2 2 2 2 3 3 4 4
17401 088/300 Read block 4750 len 4096
17402 089/300 Write block 3797 len 8192 data: 5 5
17403 090/300 Write block 2956 len 28672 data: 3 3 2 2 2 3 3
17404 091/300 Read block 2710 len 32768
17405 092/300 Read block 424 len 40960
17406 093/300 Write block 92 len 36864 data: 5 5 4 5 4 4 3 3 4
17407 094/300 Read block 3304 len 36864
17408 095/300 Read block 4178 len 12288
17409 096/300 Read block 1165 len 8192
17410 097/300 Read block 4032 len 8192
17411 098/300 Write block 3820 len 20480 data: 3 3 3 3 2
17412 099/300 Write block 1298 len 16384 data: 3 4 5 6
17413 100/300 Read block 3283 len 12288
17414 101/300 Read block 2136 len 4096
17415 102/300 Write block 742 len 12288 data: 5 6 7
17416 103/300 Read block 504 len 16384
17417 104/300 Write block 3304 len 32768 data: 6 6 7 6 5 5 5 5
17418 105/300 Write block 3881 len 36864 data: 3 3 3 4 4 4 5 4 4
17419 106/300 Read block 3027 len 36864
17420 107/300 Read block 621 len 8192
17421 108/300 Read block 1611 len 28672
17422 109/300 Read block 1558 len 40960
17423 110/300 Write block 317 len 12288 data: 3 3 4
17424 111/300 Write block 2894 len 24576 data: 4 5 5 5 5 5
17425 112/300 Read block 2564 len 24576
17426 113/300 Write block 2540 len 12288 data: 5 4 3
17427 114/300 Read block 2153 len 40960
17428 115/300 Write block 1326 len 16384 data: 6 6 6 5
17429 116/300 Write block 1477 len 40960 data: 4 3 4 4 3 4 3 3 4 4
17430 117/300 Write block 1831 len 40960 data: 6 5 5 5 4 3 3 4 4 4
17431 118/300 Read block 3425 len 16384
17432 119/300 Read block 2476 len 36864
17433 120/300 Read block 1621 len 32768
17434 121/300 Write block 4156 len 8192 data: 6 6
17435 122/300 Write block 546 len 24576 data: 3 3 3 4 4 4
17436 123/300 Read block 3235 len 12288
17437 124/300 Read block 1792 len 32768
17438 125/300 Read block 4049 len 36864
17439 126/300 Write block 1808 len 40960 data: 3 3 3 3 4 4 4 4 5 5
17440 127/300 Read block 3520 len 32768
17441 128/300 Read block 770 len 16384
17442 129/300 Write block 218 len 32768 data: 5 5 5 4 4 4 5 5
17443 130/300 Read block 61 len 28672
17444 131/300 Read block 2400 len 12288
17445 132/300 Read block 4184 len 16384
17446 133/300 Write block 3334 len 4096 data: 5
17447 134/300 Read block 1148 len 32768
17448 135/300 Read block 4058 len 32768
17449 136/300 Read block 50 len 32768
17450 137/300 Read block 4772 len 20480
17451 138/300 Read block 4990 len 8192
17452 139/300 Read block 4941 len 12288
17453 140/300 Write block 736 len 20480 data: 4 4 5 5 6
17454 141/300 Read block 4731 len 16384
17455 142/300 Write block 3093 len 24576 data: 5 5 5 5 4 5
17456 143/300 Read block 133 len 16384
17457 144/300 Read block 3125 len 28672
17458 145/300 Flush
17459 146/300 Write block 3937 len 20480 data: 5 6 7 5 5
17460 147/300 Write block 4246 len 12288 data: 8 6 6
17461 148/300 Write block 4515 len 20480 data: 3 3 4 3 5
17462 149/300 Read block 3685 len 4096
17463 150/300 Write block 4320 len 20480 data: 5 5 4 4 2
17464 151/300 Read block 4045 len 24576
17465 152/300 Read block 4778 len 32768
17466 153/300 Read block 2138 len 20480
17467 154/300 Read block 2893 len 40960
17468 155/300 Flush
17469 156/300 Read block 1498 len 24576
17470 157/300 Write block 4750 len 20480 data: 3 4 4 5 6
17471 158/300 Read block 489 len 8192
17472 159/300 Read block 103 len 16384
17473 160/300 Write block 1752 len 8192 data: 3 3
17474 161/300 Read block 845 len 24576
17475 162/300 Flush
17476 163/300 Flush
17477 164/300 Flush
17478 165/300 Read block 2679 len 36864
17479 166/300 Read block 4525 len 32768
17480 167/300 Read block 1909 len 28672
17481 168/300 Read block 2167 len 24576
17482 169/300 Read block 1525 len 32768
17483 170/300 Write block 684 len 16384 data: 6 7 4 4
17484 171/300 Read block 4752 len 12288
17485 172/300 Write block 4915 len 12288 data: 5 5 6
17486 173/300 Write block 4330 len 12288 data: 5 5 5
17487 174/300 Write block 2185 len 8192 data: 9 8
17488 175/300 Write block 3820 len 8192 data: 4 4
17489 176/300 Write block 3367 len 36864 data: 5 5 5 6 3 2 3 4 4
17490 177/300 Read block 4582 len 36864
17491 178/300 Write block 4991 len 24576 data: 2 2 2 2 3 3
17492 179/300 Read block 2138 len 24576
17493 180/300 Write block 4116 len 8192 data: 6 9
17494 181/300 Read block 2578 len 24576
17495 182/300 Read block 1419 len 12288
17496 183/300 Write block 6 len 36864 data: 3 3 3 3 3 3 3 3 3
17497 184/300 Write block 3158 len 4096 data: 7
17498 185/300 Read block 2467 len 16384
17499 186/300 Write block 3359 len 16384 data: 3 3 2 3
17500 187/300 Write block 3023 len 8192 data: 4 4
17501 188/300 Write block 3303 len 36864 data: 6 7 7 8 7 6 6 6 6
17502 189/300 Write block 2001 len 8192 data: 5 4
17503 190/300 Read block 753 len 4096
17504 191/300 Read block 2480 len 4096
17505 192/300 Read block 3648 len 4096
17506 193/300 Write block 4936 len 28672 data: 2 3 2 2 2 2 2
17507 194/300 Write block 4082 len 16384 data: 3 4 3 3
17508 195/300 Read block 2190 len 36864
17509 196/300 Flush
17510 197/300 Write block 2975 len 28672 data: 4 5 5 5 5 5 5
17511 198/300 Read block 1932 len 28672
17512 199/300 Read block 2357 len 24576
17513 200/300 Write block 2711 len 4096 data: 3
17514 201/300 Read block 2057 len 4096
17515 202/300 Flush
17516 203/300 Read block 610 len 24576
17517 204/300 Read block 1177 len 16384
17518 205/300 Read block 2071 len 40960
17519 206/300 Write block 3 len 36864 data: 2 2 2 4 4 4 4 4 4
17520 207/300 Read block 4217 len 12288
17521 208/300 Read block 3996 len 32768
17522 209/300 Read block 1460 len 28672
17523 210/300 Write block 1561 len 32768 data: 3 3 4 6 4 3 2 4
17524 211/300 Flush
17525 212/300 Read block 4036 len 24576
17526 213/300 Write block 3388 len 36864 data: 4 3 3 3 4 3 3 4 4
17527 214/300 Write block 4206 len 28672 data: 4 4 4 5 5 5 4
17528 215/300 Write block 4331 len 28672 data: 6 6 4 4 3 3 3
17529 216/300 Flush
17530 217/300 Write block 2597 len 24576 data: 5 5 5 5 3 3
17531 218/300 Read block 1341 len 40960
17532 219/300 Write block 1220 len 12288 data: 3 3 3
17533 220/300 Read block 4989 len 40960
17534 221/300 Read block 1134 len 20480
17535 222/300 Read block 4910 len 16384
17536 223/300 Read block 4600 len 8192
17537 224/300 Write block 3690 len 36864 data: 4 5 4 4 6 5 6 6 6
17538 225/300 Read block 1451 len 32768
17539 226/300 Read block 2042 len 32768
17540 227/300 Read block 1806 len 24576
17541 228/300 Read block 29 len 36864
17542 229/300 Read block 2336 len 4096
17543 230/300 Write block 962 len 20480 data: 4 4 4 4 3
17544 231/300 Read block 4756 len 28672
17545 232/300 Read block 3881 len 24576
17546 233/300 Write block 693 len 20480 data: 4 4 4 5 6
17547 234/300 Write block 1453 len 20480 data: 3 4 4 5 4
17548 235/300 Write block 3472 len 24576 data: 5 6 6 5 4 4
17549 236/300 Write block 3912 len 12288 data: 5 4 5
17550 237/300 Write block 287 len 20480 data: 4 4 5 5 5
17551 238/300 Read block 2920 len 36864
17552 239/300 Write block 4843 len 28672 data: 2 4 4 4 4 4 4
17553 240/300 Flush
17554 241/300 Write block 1797 len 32768 data: 3 3 3 4 3 4 4 3
17555 242/300 Read block 3686 len 4096
17556 243/300 Read block 4907 len 20480
17557 244/300 Write block 48 len 40960 data: 4 5 5 6 5 5 3 2 2 2
17558 245/300 Read block 1921 len 28672
17559 246/300 Write block 1267 len 8192 data: 4 3
17560 247/300 Write block 3487 len 12288 data: 2 2 2
17561 248/300 Read block 2054 len 20480
17562 249/300 Write block 1212 len 8192 data: 3 2
17563 250/300 Read block 2877 len 16384
17564 251/300 Write block 1933 len 8192 data: 5 3
17565 252/300 Read block 2950 len 16384
17566 253/300 Read block 2513 len 16384
17567 254/300 Read block 3961 len 40960
17568 255/300 Read block 2474 len 4096
17569 256/300 Write block 4217 len 12288 data: 6 7 5
17570 257/300 Read block 4290 len 16384
17571 258/300 Write block 4072 len 40960 data: 3 3 3 3 3 2 3 3 2 3
17572 259/300 Read block 2410 len 36864
17573 260/300 Write block 4158 len 4096 data: 5
17574 261/300 Write block 559 len 40960 data: 4 4 4 3 4 4 4 4 3 4
17575 262/300 Write block 3839 len 28672 data: 4 4 4 4 4 7 6
17576 263/300 Read block 4479 len 36864
17577 264/300 Write block 1786 len 36864 data: 9 9 8 7 7 7 5 3 3
17578 265/300 Write block 4151 len 36864 data: 3 4 4 5 6 7 7 6 6
17579 266/300 Write block 2975 len 12288 data: 5 6 6
17580 267/300 Read block 4331 len 4096
17581 268/300 Write block 1348 len 8192 data: 7 6
17582 269/300 Write block 275 len 24576 data: 2 2 3 3 2 2
17583 270/300 Read block 1642 len 12288
17584 271/300 Write block 3114 len 4096 data: 3
17585 272/300 Write block 687 len 20480 data: 5 3 4 6 7
17586 273/300 Read block 2138 len 28672
17587 274/300 Write block 358 len 28672 data: 6 5 5 5 4 4 4
17588 275/300 Flush
17589 276/300 Flush
17590 277/300 Write block 4713 len 24576 data: 3 3 3 3 3 4
17591 278/300 Read block 3937 len 40960
17592 279/300 Read block 434 len 32768
17593 280/300 Read block 2323 len 36864
17594 281/300 Write block 739 len 24576 data: 6 7 6 6 7 8
17595 282/300 Read block 4377 len 24576
17596 283/300 Read block 2042 len 4096
17597 284/300 Read block 4565 len 24576
17598 285/300 Write block 3740 len 24576 data: 5 4 4 4 4 4
17599 286/300 Read block 1558 len 8192
17600 287/300 Read block 2152 len 32768
17601 288/300 Read block 1435 len 16384
17602 289/300 Read block 1244 len 40960
17603 290/300 Write block 4287 len 32768 data: 5 6 6 6 4 4 3 3
17604 291/300 Write block 4659 len 24576 data: 5 4 4 3 4 5
17605 292/300 Read block 3199 len 24576
17606 293/300 Write block 3698 len 12288 data: 7 5 5
17607 294/300 Write block 4586 len 28672 data: 3 3 3 3 2 2 2
17608 295/300 Read block 4255 len 16384
17609 296/300 Read block 1714 len 32768
17610 297/300 Write block 899 len 36864 data: 3 2 3 3 3 3 2 3 3
17611 298/300 Read block 2814 len 4096
17612 299/300 Write block 3253 len 8192 data: 5 2
17613 300/300 Write block 171 len 8192 data: 3 4
17614 301/300 Write block 4406 len 12288 data: 3 3 5
17615 Main task has work to do, go find it
17616 got dsc Start(2)
17617 start 2
17618 Jan 15 05:22:42.864 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/2, method: POST, req_id: 1f8bcf7e-03cd-4b57-ae22-39aa19f0e22b, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
17619 Replay: started 2, returned:Ok(())
17620 ----------------------------------------------------------------
17621 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17622 States: Active Active Offline
17623 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17624 5529 Acked Read 2 Done Done Sent false
17625 5530 Acked Read 9 Done Done Sent false
17626 5531 Acked Read 7 Done Done Sent false
17627 5532 Acked Write 4 Done Done Sent false
17628 5533 Acked Read 5 Done Done Sent false
17629 5534 Acked Read 10 Done Done Sent false
17630 5535 Acked Flush 0 Done Done Sent false
17631 5536 Acked Read 1 Done Done Sent false
17632 5537 Acked Write 3 Done Done Sent false
17633 5538 Acked Flush 0 Done Done Sent false
17634 5539 Acked Read 7 Done Done Sent false
17635 5540 Acked Flush 0 Done Done Sent false
17636 5541 Acked Write 8 Done Done Sent false
17637 5542 Acked Read 4 Done Done Sent false
17638 5543 Acked Flush 0 Done Done Sent false
17639 5544 Acked Read 1 Done Done Sent false
17640 5545 Acked Write 9 Done Done Sent false
17641 5546 Acked Flush 0 Done Done Sent false
17642 5547 Acked Write 4 Done Done Sent false
17643 5548 Acked Read 2 Done Done Sent false
17644 5549 Acked Read 6 Done Done Sent false
17645 5550 Acked Read 7 Done Done Sent false
17646 5551 Acked Write 9 Done Done Sent false
17647 5552 Acked Read 9 Done Done Sent false
17648 5553 Acked Read 8 Done Done Sent false
17649 5554 Acked Read 7 Done Done Sent false
17650 5555 Acked Write 8 Done Done Sent false
17651 5556 Acked Read 10 Done Done Sent false
17652 5557 Acked Read 7 Done Done Sent false
17653 5558 Acked Write 2 Done Done Sent false
17654 5559 Acked Write 2 Done Done Sent false
17655 5560 Acked Write 2 Done Done Sent false
17656 5561 Acked Read 2 Done Done Sent false
17657 5562 Acked Read 7 Done Done Sent false
17658 5563 Acked Read 3 Done Done Sent false
17659 5564 Acked Write 6 Done Done Sent false
17660 5565 Acked Read 3 Done Done Sent false
17661 5566 Acked Write 7 Done Done Sent false
17662 5567 Acked Read 1 Done Done Sent false
17663 5568 Acked Read 10 Done Done Sent false
17664 5569 Acked Read 4 Done Done Sent false
17665 5570 Acked Read 9 Done Done Sent false
17666 5571 Acked Read 6 Done Done Sent false
17667 5572 Acked Read 3 Done Done Sent false
17668 5573 Acked Write 8 Done Done Sent false
17669 5574 Acked Write 5 Done Done Sent false
17670 5575 Acked Read 2 Done Done Sent false
17671 5576 Acked Flush 0 Done Done Sent false
17672 5577 Acked Flush 0 Done Done Sent false
17673 5578 Acked Write 8 Done Done Sent false
17674 5579 Acked Read 7 Done Done Sent false
17675 5580 Acked Write 4 Done Done Sent false
17676 5581 Acked Read 8 Done Done Sent false
17677 5582 Acked Read 6 Done Done Sent false
17678 5583 Acked Read 10 Done Done Sent false
17679 5584 Acked Read 4 Done Done Sent false
17680 5585 Acked Write 6 Done Done Sent false
17681 5586 Acked Write 3 Done Done Sent false
17682 5587 Acked Write 5 Done Done Sent false
17683 5588 Acked Read 2 Done Done Sent false
17684 5589 Acked Read 1 Done Done Sent false
17685 5590 Acked Write 9 Done Done Sent false
17686 5591 Acked Write 9 Done Done Sent false
17687 5592 Acked Write 10 Done Done Sent false
17688 5593 Acked Read 5 Done Done Sent false
17689 5594 Acked Read 3 Done Done Sent false
17690 5595 Acked Read 8 Done Done Sent false
17691 5596 Acked Read 3 Done Done Sent false
17692 5597 Acked Write 2 Done Done Sent false
17693 5598 Acked Read 1 Done Done Sent false
17694 5599 Acked Write 5 Done Done Sent false
17695 5600 Acked Write 10 Done Done Sent false
17696 5601 Acked Write 2 Done Done Sent false
17697 5602 Acked Read 10 Done Done Sent false
17698 5603 Acked Write 1 Done Done Sent false
17699 5604 Acked Write 2 Done Done Sent false
17700 5605 Acked Read 2 Done Done Sent false
17701 5606 Acked Read 1 Done Done Sent false
17702 5607 Acked Write 5 Done Done Sent false
17703 5608 Acked Write 7 Done Done Sent false
17704 5609 Acked Read 10 Done Done Sent false
17705 5610 Acked Write 7 Done Done Sent false
17706 5611 Acked Read 2 Done Done Sent false
17707 5612 Acked Read 3 Done Done Sent false
17708 5613 Acked Read 8 Done Done Sent false
17709 5614 Acked Write 10 Done Done Sent false
17710 5615 Acked Write 8 Done Done Sent false
17711 5616 Acked Read 1 Done Done Sent false
17712 5617 Acked Write 2 Done Done Sent false
17713 5618 Acked Write 7 Done Done Sent false
17714 5619 Acked Read 8 Done Done Sent false
17715 5620 Acked Read 10 Done Done Sent false
17716 5621 Acked Write 9 Done Done Sent false
17717 5622 Acked Read 9 Done Done Sent false
17718 5623 Acked Read 3 Done Done Sent false
17719 5624 Acked Read 2 Done Done Sent false
17720 5625 Acked Read 2 Done Done Sent false
17721 5626 Acked Write 5 Done Done Sent false
17722 5627 Acked Write 4 Done Done Sent false
17723 5628 Acked Read 3 Done Done Sent false
17724 5629 Acked Read 1 Done Done Sent false
17725 5630 Acked Write 3 Done Done Sent false
17726 5631 Acked Read 4 Done Done Sent false
17727 5632 Acked Write 8 Done Done Sent false
17728 5633 Acked Write 9 Done Done Sent false
17729 5634 Acked Read 9 Done Done Sent false
17730 5635 Acked Read 2 Done Done Sent false
17731 5636 Acked Read 7 Done Done Sent false
17732 5637 Acked Read 10 Done Done Sent false
17733 5638 Acked Write 3 Done Done Sent false
17734 5639 Acked Write 6 Done Done Sent false
17735 5640 Acked Read 6 Done Done Sent false
17736 5641 Acked Write 3 Done Done Sent false
17737 5642 Acked Read 10 Done Done Sent false
17738 5643 Acked Write 4 Done Done Sent false
17739 5644 Acked Write 10 Done Done Sent false
17740 5645 Acked Write 10 Done Done Sent false
17741 5646 Acked Read 4 Done Done Sent false
17742 5647 Acked Read 9 Done Done Sent false
17743 5648 Acked Read 8 Done Done Sent false
17744 5649 Acked Write 2 Done Done Sent false
17745 5650 Acked Write 6 Done Done Sent false
17746 5651 Acked Read 3 Done Done Sent false
17747 5652 Acked Read 8 Done Done Sent false
17748 5653 Acked Read 9 Done Done Sent false
17749 5654 Acked Write 10 Done Done Sent false
17750 5655 Acked Read 8 Done Done Sent false
17751 5656 Acked Read 4 Done Done Sent false
17752 5657 Acked Write 8 Done Done Sent false
17753 5658 Acked Read 7 Done Done Sent false
17754 5659 Acked Read 3 Done Done Sent false
17755 5660 Acked Read 4 Done Done Sent false
17756 5661 Acked Write 1 Done Done Sent false
17757 5662 Acked Read 8 Done Done Sent false
17758 5663 Acked Read 8 Done Done Sent false
17759 5664 Acked Read 8 Done Done Sent false
17760 5665 Acked Read 5 Done Done Sent false
17761 5666 Acked Read 2 Done Done Sent false
17762 5667 Acked Read 3 Done Done Sent false
17763 5668 Acked Write 5 Done Done Sent false
17764 5669 Acked Read 4 Done Done Sent false
17765 5670 Acked Write 6 Done Done Sent false
17766 5671 Acked Read 4 Done Done Sent false
17767 5672 Acked Read 7 Done Done Sent false
17768 5673 Acked Flush 0 Done Done Sent false
17769 5674 Acked Write 5 Done Done Sent false
17770 5675 Acked Write 3 Done Done Sent false
17771 5676 Acked Write 5 Done Done Sent false
17772 5677 Acked Read 1 Done Done Sent false
17773 5678 Acked Write 5 Done Done Sent false
17774 5679 Acked Read 6 Done Done Sent false
17775 5680 Acked Read 8 Done Done Sent false
17776 5681 Acked Read 5 Done Done Sent false
17777 5682 Acked Read 10 Done Done Sent false
17778 5683 Acked Flush 0 Done Done Sent false
17779 5684 Acked Read 6 Done Done Sent false
17780 5685 Acked Write 5 Done Done Sent false
17781 5686 Acked Read 2 Done Done Sent false
17782 5687 Acked Read 4 Done Done Sent false
17783 5688 Acked Write 2 Done Done Sent false
17784 5689 Acked Read 6 Done Done Sent false
17785 5690 Acked Flush 0 Done Done Sent false
17786 5691 Acked Flush 0 Done Done Sent false
17787 5692 Acked Flush 0 Done Done Sent false
17788 5693 Acked Read 9 Done Done Sent false
17789 5694 Acked Read 8 Done Done Sent false
17790 5695 Acked Read 7 Done Done Sent false
17791 5696 Acked Read 6 Done Done Sent false
17792 5697 Acked Read 8 Done Done Sent false
17793 5698 Acked Write 4 Done Done Sent false
17794 5699 Acked Read 3 Done Done Sent false
17795 5700 Acked Write 3 Done Done Sent false
17796 5701 Acked Write 3 Done Done Sent false
17797 5702 Acked Write 2 Done Done Sent false
17798 5703 Acked Write 2 Done Done Sent false
17799 5704 Acked Write 9 Done Done Sent false
17800 5705 Acked Read 9 Done Done Sent false
17801 5706 Acked Write 6 Done Done Sent false
17802 5707 Acked Read 6 Done Done Sent false
17803 5708 Acked Write 2 Done Done Sent false
17804 5709 Acked Read 6 Done Done Sent false
17805 5710 Acked Read 3 Done Done Sent false
17806 5711 Acked Write 9 Done Done Sent false
17807 5712 Acked Write 1 Done Done Sent false
17808 5713 Acked Read 4 Done Done Sent false
17809 5714 Acked Write 4 Done Done Sent false
17810 5715 Acked Write 2 Done Done Sent false
17811 5716 Acked Write 9 Done Done Sent false
17812 5717 Acked Write 2 Done Done Sent false
17813 5718 Acked Read 1 Done Done Sent false
17814 5719 Acked Read 1 Done Done Sent false
17815 5720 Acked Read 1 Done Done Sent false
17816 5721 Acked Write 7 Done Done Sent false
17817 5722 Acked Write 4 Done Done Sent false
17818 5723 Acked Read 9 Done Done Sent false
17819 5724 Acked Flush 0 Done Done Sent false
17820 5725 Acked Write 7 Done Done Sent false
17821 5726 Acked Read 7 Done Done Sent false
17822 5727 Acked Read 6 Done Done Sent false
17823 5728 Acked Write 1 Done Done Sent false
17824 5729 Acked Read 1 Done Done Sent false
17825 5730 Acked Flush 0 Done Done Sent false
17826 5731 Acked Read 6 Done Done Sent false
17827 5732 Acked Read 4 Done Done Sent false
17828 5733 Acked Read 10 Done Done Sent false
17829 5734 Acked Write 9 Done Done Sent false
17830 5735 Acked Read 3 Done Done Sent false
17831 5736 Acked Read 8 Done Done Sent false
17832 5737 Acked Read 7 Done Done Sent false
17833 5738 Acked Write 8 Done Done Sent false
17834 5739 Acked Flush 0 Done Done Sent false
17835 5740 Acked Read 6 Done Done Sent false
17836 5741 Acked Write 9 Done Done Sent false
17837 5742 Acked Write 7 Done Done Sent false
17838 5743 Acked Write 7 Done Done Sent false
17839 5744 Acked Flush 0 Done Done Sent false
17840 5745 Acked Write 6 Done Done Sent false
17841 5746 Acked Read 10 Done Done Sent false
17842 5747 Acked Write 3 Done Done Sent false
17843 5748 Acked Read 10 Done Done Sent false
17844 5749 Acked Read 5 Done Done Sent false
17845 5750 Acked Read 4 Done Done Sent false
17846 5751 Acked Read 2 Done Done Sent false
17847 5752 Acked Write 9 Done Done Sent false
17848 5753 Acked Read 8 Done Done Sent false
17849 5754 Acked Read 8 Done Done Sent false
17850 5755 Acked Read 6 Done Done Sent false
17851 5756 Acked Read 9 Done Done Sent false
17852 5757 Acked Read 1 Done Done Sent false
17853 5758 Acked Write 5 Done Done Sent false
17854 5759 Acked Read 7 Done Done Sent false
17855 5760 Acked Read 6 Done Done Sent false
17856 5761 Acked Write 5 Done Done Sent false
17857 5762 Acked Write 5 Done Done Sent false
17858 5763 Acked Write 6 Done Done Sent false
17859 5764 Acked Write 3 Done Done Sent false
17860 5765 Acked Write 5 Done Done Sent false
17861 5766 Acked Read 9 Done Done Sent false
17862 5767 Acked Write 7 Done Done Sent false
17863 5768 Acked Flush 0 Done Done Sent false
17864 5769 Acked Write 8 Done Done Sent false
17865 5770 Acked Read 1 Done Done Sent false
17866 5771 Acked Read 5 Done Done Sent false
17867 5772 Acked Write 10 Done Done Sent false
17868 5773 Acked Read 7 Done Done Sent false
17869 5774 Acked Write 2 Done Done Sent false
17870 5775 Acked Write 3 Done Done Sent false
17871 5776 Acked Read 5 Done Done Sent false
17872 5777 Acked Write 2 Done Done Sent false
17873 5778 Acked Read 4 Done Done Sent false
17874 5779 Acked Write 2 Done Done Sent false
17875 5780 Acked Read 4 Done Done Sent false
17876 5781 Acked Read 4 Done Done Sent false
17877 5782 Acked Read 10 Done Done Sent false
17878 5783 Acked Read 1 Done Done Sent false
17879 5784 Acked Write 3 Done Done Sent false
17880 5785 Acked Read 4 Done Done Sent false
17881 5786 Acked Write 10 Done Done Sent false
17882 5787 Acked Read 9 Done Done Sent false
17883 5788 Acked Write 1 Done Done Sent false
17884 5789 Acked Write 10 Done Done Sent false
17885 5790 Acked Write 7 Done Done Sent false
17886 5791 Acked Read 9 Done Done Sent false
17887 5792 Acked Write 9 Done Done Sent false
17888 5793 Acked Write 9 Done Done Sent false
17889 5794 Acked Write 3 Done Done Sent false
17890 5795 Acked Read 1 Done Done Sent false
17891 5796 Acked Write 2 Done Done Sent false
17892 5797 Acked Write 6 Done Done Sent false
17893 5798 Acked Read 3 Done Done Sent false
17894 5799 Acked Write 1 Done Done Sent false
17895 5800 Acked Write 5 Done Done Sent false
17896 5801 Acked Read 7 Done Done Sent false
17897 5802 Acked Write 7 Done Done Sent false
17898 5803 Acked Flush 0 Done Done Sent false
17899 5804 Acked Flush 0 Done Done Sent false
17900 5805 Acked Write 6 Done Done Sent false
17901 5806 Acked Read 10 Done Done Sent false
17902 5807 Acked Read 8 Done Done Sent false
17903 5808 Acked Read 9 Done Done Sent false
17904 5809 Acked Write 6 Done Done Sent false
17905 5810 Acked Read 6 Done Done Sent false
17906 5811 Acked Read 1 Done Done Sent false
17907 5812 Acked Read 6 Done Done Sent false
17908 5813 Acked Write 6 Done Done Sent false
17909 5814 Acked Read 2 Done Done Sent false
17910 5815 Acked Read 8 Done Done Sent false
17911 5816 Acked Read 4 Done Done Sent false
17912 5817 Acked Read 10 Done Done Sent false
17913 5818 Acked Write 8 Done Done Sent false
17914 5819 Acked Write 6 Done Done Sent false
17915 5820 Acked Read 6 Done Done Sent false
17916 5821 Acked Write 3 Done Done Sent false
17917 5822 Acked Write 7 Done Done Sent false
17918 5823 Acked Read 4 Done Done Sent false
17919 5824 Acked Read 8 Done Done Sent false
17920 5825 Acked Write 9 Done Done Sent false
17921 5826 Acked Read 1 Done Done Sent false
17922 5827 Acked Write 2 Done Done Sent false
17923 5828 Acked Write 2 Done Done Sent false
17924 5829 Acked Write 3 Done Done Sent false
17925 STATES DS:0 DS:1 DS:2 TOTAL
17926 Sent 0 0 301 301
17927 Done 301 301 0 602
17928 Skipped 0 0 0 0
17929 Error 0 0 0 0
17930 Last Flush: 5804 5804 5528
17931 Downstairs last five completed: 5528 5527 5526 5525 5524
17932 Upstairs last five completed: 5829 5828 5827 5826 5825
17933 CLIENT: Up:0 ds:301 act:2
17934 ----------------------------------------------------------------
17935 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17936 States: Active Active Offline
17937 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17938 5529 Acked Read 2 Done Done Sent false
17939 5530 Acked Read 9 Done Done Sent false
17940 5531 Acked Read 7 Done Done Sent false
17941 5532 Acked Write 4 Done Done Sent false
17942 5533 Acked Read 5 Done Done Sent false
17943 5534 Acked Read 10 Done Done Sent false
17944 5535 Acked Flush 0 Done Done Sent false
17945 5536 Acked Read 1 Done Done Sent false
17946 5537 Acked Write 3 Done Done Sent false
17947 5538 Acked Flush 0 Done Done Sent false
17948 5539 Acked Read 7 Done Done Sent false
17949 5540 Acked Flush 0 Done Done Sent false
17950 5541 Acked Write 8 Done Done Sent false
17951 5542 Acked Read 4 Done Done Sent false
17952 5543 Acked Flush 0 Done Done Sent false
17953 5544 Acked Read 1 Done Done Sent false
17954 5545 Acked Write 9 Done Done Sent false
17955 5546 Acked Flush 0 Done Done Sent false
17956 5547 Acked Write 4 Done Done Sent false
17957 5548 Acked Read 2 Done Done Sent false
17958 5549 Acked Read 6 Done Done Sent false
17959 5550 Acked Read 7 Done Done Sent false
17960 5551 Acked Write 9 Done Done Sent false
17961 5552 Acked Read 9 Done Done Sent false
17962 5553 Acked Read 8 Done Done Sent false
17963 5554 Acked Read 7 Done Done Sent false
17964 5555 Acked Write 8 Done Done Sent false
17965 5556 Acked Read 10 Done Done Sent false
17966 5557 Acked Read 7 Done Done Sent false
17967 5558 Acked Write 2 Done Done Sent false
17968 5559 Acked Write 2 Done Done Sent false
17969 5560 Acked Write 2 Done Done Sent false
17970 5561 Acked Read 2 Done Done Sent false
17971 5562 Acked Read 7 Done Done Sent false
17972 5563 Acked Read 3 Done Done Sent false
17973 5564 Acked Write 6 Done Done Sent false
17974 5565 Acked Read 3 Done Done Sent false
17975 5566 Acked Write 7 Done Done Sent false
17976 5567 Acked Read 1 Done Done Sent false
17977 5568 Acked Read 10 Done Done Sent false
17978 5569 Acked Read 4 Done Done Sent false
17979 5570 Acked Read 9 Done Done Sent false
17980 5571 Acked Read 6 Done Done Sent false
17981 5572 Acked Read 3 Done Done Sent false
17982 5573 Acked Write 8 Done Done Sent false
17983 5574 Acked Write 5 Done Done Sent false
17984 5575 Acked Read 2 Done Done Sent false
17985 5576 Acked Flush 0 Done Done Sent false
17986 5577 Acked Flush 0 Done Done Sent false
17987 5578 Acked Write 8 Done Done Sent false
17988 5579 Acked Read 7 Done Done Sent false
17989 5580 Acked Write 4 Done Done Sent false
17990 5581 Acked Read 8 Done Done Sent false
17991 5582 Acked Read 6 Done Done Sent false
17992 5583 Acked Read 10 Done Done Sent false
17993 5584 Acked Read 4 Done Done Sent false
17994 5585 Acked Write 6 Done Done Sent false
17995 5586 Acked Write 3 Done Done Sent false
17996 5587 Acked Write 5 Done Done Sent false
17997 5588 Acked Read 2 Done Done Sent false
17998 5589 Acked Read 1 Done Done Sent false
17999 5590 Acked Write 9 Done Done Sent false
18000 5591 Acked Write 9 Done Done Sent false
18001 5592 Acked Write 10 Done Done Sent false
18002 5593 Acked Read 5 Done Done Sent false
18003 5594 Acked Read 3 Done Done Sent false
18004 5595 Acked Read 8 Done Done Sent false
18005 5596 Acked Read 3 Done Done Sent false
18006 5597 Acked Write 2 Done Done Sent false
18007 5598 Acked Read 1 Done Done Sent false
18008 5599 Acked Write 5 Done Done Sent false
18009 5600 Acked Write 10 Done Done Sent false
18010 5601 Acked Write 2 Done Done Sent false
18011 5602 Acked Read 10 Done Done Sent false
18012 5603 Acked Write 1 Done Done Sent false
18013 5604 Acked Write 2 Done Done Sent false
18014 5605 Acked Read 2 Done Done Sent false
18015 5606 Acked Read 1 Done Done Sent false
18016 5607 Acked Write 5 Done Done Sent false
18017 5608 Acked Write 7 Done Done Sent false
18018 5609 Acked Read 10 Done Done Sent false
18019 5610 Acked Write 7 Done Done Sent false
18020 5611 Acked Read 2 Done Done Sent false
18021 5612 Acked Read 3 Done Done Sent false
18022 5613 Acked Read 8 Done Done Sent false
18023 5614 Acked Write 10 Done Done Sent false
18024 5615 Acked Write 8 Done Done Sent false
18025 5616 Acked Read 1 Done Done Sent false
18026 5617 Acked Write 2 Done Done Sent false
18027 5618 Acked Write 7 Done Done Sent false
18028 5619 Acked Read 8 Done Done Sent false
18029 5620 Acked Read 10 Done Done Sent false
18030 5621 Acked Write 9 Done Done Sent false
18031 5622 Acked Read 9 Done Done Sent false
18032 5623 Acked Read 3 Done Done Sent false
18033 5624 Acked Read 2 Done Done Sent false
18034 5625 Acked Read 2 Done Done Sent false
18035 5626 Acked Write 5 Done Done Sent false
18036 5627 Acked Write 4 Done Done Sent false
18037 5628 Acked Read 3 Done Done Sent false
18038 5629 Acked Read 1 Done Done Sent false
18039 5630 Acked Write 3 Done Done Sent false
18040 5631 Acked Read 4 Done Done Sent false
18041 5632 Acked Write 8 Done Done Sent false
18042 5633 Acked Write 9 Done Done Sent false
18043 5634 Acked Read 9 Done Done Sent false
18044 5635 Acked Read 2 Done Done Sent false
18045 5636 Acked Read 7 Done Done Sent false
18046 5637 Acked Read 10 Done Done Sent false
18047 5638 Acked Write 3 Done Done Sent false
18048 5639 Acked Write 6 Done Done Sent false
18049 5640 Acked Read 6 Done Done Sent false
18050 5641 Acked Write 3 Done Done Sent false
18051 5642 Acked Read 10 Done Done Sent false
18052 5643 Acked Write 4 Done Done Sent false
18053 5644 Acked Write 10 Done Done Sent false
18054 5645 Acked Write 10 Done Done Sent false
18055 5646 Acked Read 4 Done Done Sent false
18056 5647 Acked Read 9 Done Done Sent false
18057 5648 Acked Read 8 Done Done Sent false
18058 5649 Acked Write 2 Done Done Sent false
18059 5650 Acked Write 6 Done Done Sent false
18060 5651 Acked Read 3 Done Done Sent false
18061 5652 Acked Read 8 Done Done Sent false
18062 5653 Acked Read 9 Done Done Sent false
18063 5654 Acked Write 10 Done Done Sent false
18064 5655 Acked Read 8 Done Done Sent false
18065 5656 Acked Read 4 Done Done Sent false
18066 5657 Acked Write 8 Done Done Sent false
18067 5658 Acked Read 7 Done Done Sent false
18068 5659 Acked Read 3 Done Done Sent false
18069 5660 Acked Read 4 Done Done Sent false
18070 5661 Acked Write 1 Done Done Sent false
18071 5662 Acked Read 8 Done Done Sent false
18072 5663 Acked Read 8 Done Done Sent false
18073 5664 Acked Read 8 Done Done Sent false
18074 5665 Acked Read 5 Done Done Sent false
18075 5666 Acked Read 2 Done Done Sent false
18076 5667 Acked Read 3 Done Done Sent false
18077 5668 Acked Write 5 Done Done Sent false
18078 5669 Acked Read 4 Done Done Sent false
18079 5670 Acked Write 6 Done Done Sent false
18080 5671 Acked Read 4 Done Done Sent false
18081 5672 Acked Read 7 Done Done Sent false
18082 5673 Acked Flush 0 Done Done Sent false
18083 5674 Acked Write 5 Done Done Sent false
18084 5675 Acked Write 3 Done Done Sent false
18085 5676 Acked Write 5 Done Done Sent false
18086 5677 Acked Read 1 Done Done Sent false
18087 5678 Acked Write 5 Done Done Sent false
18088 5679 Acked Read 6 Done Done Sent false
18089 5680 Acked Read 8 Done Done Sent false
18090 5681 Acked Read 5 Done Done Sent false
18091 5682 Acked Read 10 Done Done Sent false
18092 5683 Acked Flush 0 Done Done Sent false
18093 5684 Acked Read 6 Done Done Sent false
18094 5685 Acked Write 5 Done Done Sent false
18095 5686 Acked Read 2 Done Done Sent false
18096 5687 Acked Read 4 Done Done Sent false
18097 5688 Acked Write 2 Done Done Sent false
18098 5689 Acked Read 6 Done Done Sent false
18099 5690 Acked Flush 0 Done Done Sent false
18100 5691 Acked Flush 0 Done Done Sent false
18101 5692 Acked Flush 0 Done Done Sent false
18102 5693 Acked Read 9 Done Done Sent false
18103 5694 Acked Read 8 Done Done Sent false
18104 5695 Acked Read 7 Done Done Sent false
18105 5696 Acked Read 6 Done Done Sent false
18106 5697 Acked Read 8 Done Done Sent false
18107 5698 Acked Write 4 Done Done Sent false
18108 5699 Acked Read 3 Done Done Sent false
18109 5700 Acked Write 3 Done Done Sent false
18110 5701 Acked Write 3 Done Done Sent false
18111 5702 Acked Write 2 Done Done Sent false
18112 5703 Acked Write 2 Done Done Sent false
18113 5704 Acked Write 9 Done Done Sent false
18114 5705 Acked Read 9 Done Done Sent false
18115 5706 Acked Write 6 Done Done Sent false
18116 5707 Acked Read 6 Done Done Sent false
18117 5708 Acked Write 2 Done Done Sent false
18118 5709 Acked Read 6 Done Done Sent false
18119 5710 Acked Read 3 Done Done Sent false
18120 5711 Acked Write 9 Done Done Sent false
18121 5712 Acked Write 1 Done Done Sent false
18122 5713 Acked Read 4 Done Done Sent false
18123 5714 Acked Write 4 Done Done Sent false
18124 5715 Acked Write 2 Done Done Sent false
18125 5716 Acked Write 9 Done Done Sent false
18126 5717 Acked Write 2 Done Done Sent false
18127 5718 Acked Read 1 Done Done Sent false
18128 5719 Acked Read 1 Done Done Sent false
18129 5720 Acked Read 1 Done Done Sent false
18130 5721 Acked Write 7 Done Done Sent false
18131 5722 Acked Write 4 Done Done Sent false
18132 5723 Acked Read 9 Done Done Sent false
18133 5724 Acked Flush 0 Done Done Sent false
18134 5725 Acked Write 7 Done Done Sent false
18135 5726 Acked Read 7 Done Done Sent false
18136 5727 Acked Read 6 Done Done Sent false
18137 5728 Acked Write 1 Done Done Sent false
18138 5729 Acked Read 1 Done Done Sent false
18139 5730 Acked Flush 0 Done Done Sent false
18140 5731 Acked Read 6 Done Done Sent false
18141 5732 Acked Read 4 Done Done Sent false
18142 5733 Acked Read 10 Done Done Sent false
18143 5734 Acked Write 9 Done Done Sent false
18144 5735 Acked Read 3 Done Done Sent false
18145 5736 Acked Read 8 Done Done Sent false
18146 5737 Acked Read 7 Done Done Sent false
18147 5738 Acked Write 8 Done Done Sent false
18148 5739 Acked Flush 0 Done Done Sent false
18149 5740 Acked Read 6 Done Done Sent false
18150 5741 Acked Write 9 Done Done Sent false
18151 5742 Acked Write 7 Done Done Sent false
18152 5743 Acked Write 7 Done Done Sent false
18153 5744 Acked Flush 0 Done Done Sent false
18154 5745 Acked Write 6 Done Done Sent false
18155 5746 Acked Read 10 Done Done Sent false
18156 5747 Acked Write 3 Done Done Sent false
18157 5748 Acked Read 10 Done Done Sent false
18158 5749 Acked Read 5 Done Done Sent false
18159 5750 Acked Read 4 Done Done Sent false
18160 5751 Acked Read 2 Done Done Sent false
18161 5752 Acked Write 9 Done Done Sent false
18162 5753 Acked Read 8 Done Done Sent false
18163 5754 Acked Read 8 Done Done Sent false
18164 5755 Acked Read 6 Done Done Sent false
18165 5756 Acked Read 9 Done Done Sent false
18166 5757 Acked Read 1 Done Done Sent false
18167 5758 Acked Write 5 Done Done Sent false
18168 5759 Acked Read 7 Done Done Sent false
18169 5760 Acked Read 6 Done Done Sent false
18170 5761 Acked Write 5 Done Done Sent false
18171 5762 Acked Write 5 Done Done Sent false
18172 5763 Acked Write 6 Done Done Sent false
18173 5764 Acked Write 3 Done Done Sent false
18174 5765 Acked Write 5 Done Done Sent false
18175 5766 Acked Read 9 Done Done Sent false
18176 5767 Acked Write 7 Done Done Sent false
18177 5768 Acked Flush 0 Done Done Sent false
18178 5769 Acked Write 8 Done Done Sent false
18179 5770 Acked Read 1 Done Done Sent false
18180 5771 Acked Read 5 Done Done Sent false
18181 5772 Acked Write 10 Done Done Sent false
18182 5773 Acked Read 7 Done Done Sent false
18183 5774 Acked Write 2 Done Done Sent false
18184 5775 Acked Write 3 Done Done Sent false
18185 5776 Acked Read 5 Done Done Sent false
18186 5777 Acked Write 2 Done Done Sent false
18187 5778 Acked Read 4 Done Done Sent false
18188 5779 Acked Write 2 Done Done Sent false
18189 5780 Acked Read 4 Done Done Sent false
18190 5781 Acked Read 4 Done Done Sent false
18191 5782 Acked Read 10 Done Done Sent false
18192 5783 Acked Read 1 Done Done Sent false
18193 5784 Acked Write 3 Done Done Sent false
18194 5785 Acked Read 4 Done Done Sent false
18195 5786 Acked Write 10 Done Done Sent false
18196 5787 Acked Read 9 Done Done Sent false
18197 5788 Acked Write 1 Done Done Sent false
18198 5789 Acked Write 10 Done Done Sent false
18199 5790 Acked Write 7 Done Done Sent false
18200 5791 Acked Read 9 Done Done Sent false
18201 5792 Acked Write 9 Done Done Sent false
18202 5793 Acked Write 9 Done Done Sent false
18203 5794 Acked Write 3 Done Done Sent false
18204 5795 Acked Read 1 Done Done Sent false
18205 5796 Acked Write 2 Done Done Sent false
18206 5797 Acked Write 6 Done Done Sent false
18207 5798 Acked Read 3 Done Done Sent false
18208 5799 Acked Write 1 Done Done Sent false
18209 5800 Acked Write 5 Done Done Sent false
18210 5801 Acked Read 7 Done Done Sent false
18211 5802 Acked Write 7 Done Done Sent false
18212 5803 Acked Flush 0 Done Done Sent false
18213 5804 Acked Flush 0 Done Done Sent false
18214 5805 Acked Write 6 Done Done Sent false
18215 5806 Acked Read 10 Done Done Sent false
18216 5807 Acked Read 8 Done Done Sent false
18217 5808 Acked Read 9 Done Done Sent false
18218 5809 Acked Write 6 Done Done Sent false
18219 5810 Acked Read 6 Done Done Sent false
18220 5811 Acked Read 1 Done Done Sent false
18221 5812 Acked Read 6 Done Done Sent false
18222 5813 Acked Write 6 Done Done Sent false
18223 5814 Acked Read 2 Done Done Sent false
18224 5815 Acked Read 8 Done Done Sent false
18225 5816 Acked Read 4 Done Done Sent false
18226 5817 Acked Read 10 Done Done Sent false
18227 5818 Acked Write 8 Done Done Sent false
18228 5819 Acked Write 6 Done Done Sent false
18229 5820 Acked Read 6 Done Done Sent false
18230 5821 Acked Write 3 Done Done Sent false
18231 5822 Acked Write 7 Done Done Sent false
18232 5823 Acked Read 4 Done Done Sent false
18233 5824 Acked Read 8 Done Done Sent false
18234 5825 Acked Write 9 Done Done Sent false
18235 5826 Acked Read 1 Done Done Sent false
18236 5827 Acked Write 2 Done Done Sent false
18237 5828 Acked Write 2 Done Done Sent false
18238 5829 Acked Write 3 Done Done Sent false
18239 5830 Acked Flush 0 Done Done Sent false
18240 STATES DS:0 DS:1 DS:2 TOTAL
18241 Sent 0 0 302 302
18242 Done 302 302 0 604
18243 Skipped 0 0 0 0
18244 Error 0 0 0 0
18245 Last Flush: 5830 5830 5528
18246 Downstairs last five completed: 5528 5527 5526 5525 5524
18247 Upstairs last five completed: 5830 5829 5828 5827 5826
18248 CLIENT: Up:0 ds:302 act:2
182492025-01-15T05:22:48.625ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
182502025-01-15T05:22:48.625ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
182512025-01-15T05:22:48.625ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
182522025-01-15T05:22:48.625ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
182532025-01-15T05:22:48.626ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
18254 [8830] got start action
18255 [8830] I am going to restart
18256 Starting downstairs at port 8830
18257 Make output file at "/tmp/dsc/downstairs-8830.txt"
18258 [8830][2] reports Starting
18259 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1218)
18260 [8830][2] reports Running
18261 ----------------------------------------------------------------
18262 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18263 States: Active Active Offline
18264 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18265 5529 Acked Read 2 Done Done Sent false
18266 5530 Acked Read 9 Done Done Sent false
18267 5531 Acked Read 7 Done Done Sent false
18268 5532 Acked Write 4 Done Done Sent false
18269 5533 Acked Read 5 Done Done Sent false
18270 5534 Acked Read 10 Done Done Sent false
18271 5535 Acked Flush 0 Done Done Sent false
18272 5536 Acked Read 1 Done Done Sent false
18273 5537 Acked Write 3 Done Done Sent false
18274 5538 Acked Flush 0 Done Done Sent false
18275 5539 Acked Read 7 Done Done Sent false
18276 5540 Acked Flush 0 Done Done Sent false
18277 5541 Acked Write 8 Done Done Sent false
18278 5542 Acked Read 4 Done Done Sent false
18279 5543 Acked Flush 0 Done Done Sent false
18280 5544 Acked Read 1 Done Done Sent false
18281 5545 Acked Write 9 Done Done Sent false
18282 5546 Acked Flush 0 Done Done Sent false
18283 5547 Acked Write 4 Done Done Sent false
18284 5548 Acked Read 2 Done Done Sent false
18285 5549 Acked Read 6 Done Done Sent false
18286 5550 Acked Read 7 Done Done Sent false
18287 5551 Acked Write 9 Done Done Sent false
18288 5552 Acked Read 9 Done Done Sent false
18289 5553 Acked Read 8 Done Done Sent false
18290 5554 Acked Read 7 Done Done Sent false
18291 5555 Acked Write 8 Done Done Sent false
18292 5556 Acked Read 10 Done Done Sent false
18293 5557 Acked Read 7 Done Done Sent false
18294 5558 Acked Write 2 Done Done Sent false
18295 5559 Acked Write 2 Done Done Sent false
18296 5560 Acked Write 2 Done Done Sent false
18297 5561 Acked Read 2 Done Done Sent false
18298 5562 Acked Read 7 Done Done Sent false
18299 5563 Acked Read 3 Done Done Sent false
18300 5564 Acked Write 6 Done Done Sent false
18301 5565 Acked Read 3 Done Done Sent false
18302 5566 Acked Write 7 Done Done Sent false
18303 5567 Acked Read 1 Done Done Sent false
18304 5568 Acked Read 10 Done Done Sent false
18305 5569 Acked Read 4 Done Done Sent false
18306 5570 Acked Read 9 Done Done Sent false
18307 5571 Acked Read 6 Done Done Sent false
18308 5572 Acked Read 3 Done Done Sent false
18309 5573 Acked Write 8 Done Done Sent false
18310 5574 Acked Write 5 Done Done Sent false
18311 5575 Acked Read 2 Done Done Sent false
18312 5576 Acked Flush 0 Done Done Sent false
18313 5577 Acked Flush 0 Done Done Sent false
18314 5578 Acked Write 8 Done Done Sent false
18315 5579 Acked Read 7 Done Done Sent false
18316 5580 Acked Write 4 Done Done Sent false
18317 5581 Acked Read 8 Done Done Sent false
18318 5582 Acked Read 6 Done Done Sent false
18319 5583 Acked Read 10 Done Done Sent false
18320 5584 Acked Read 4 Done Done Sent false
18321 5585 Acked Write 6 Done Done Sent false
18322 5586 Acked Write 3 Done Done Sent false
18323 5587 Acked Write 5 Done Done Sent false
18324 5588 Acked Read 2 Done Done Sent false
18325 5589 Acked Read 1 Done Done Sent false
18326 5590 Acked Write 9 Done Done Sent false
18327 5591 Acked Write 9 Done Done Sent false
18328 5592 Acked Write 10 Done Done Sent false
18329 5593 Acked Read 5 Done Done Sent false
18330 5594 Acked Read 3 Done Done Sent false
18331 5595 Acked Read 8 Done Done Sent false
18332 5596 Acked Read 3 Done Done Sent false
18333 5597 Acked Write 2 Done Done Sent false
18334 5598 Acked Read 1 Done Done Sent false
18335 5599 Acked Write 5 Done Done Sent false
18336 5600 Acked Write 10 Done Done Sent false
18337 5601 Acked Write 2 Done Done Sent false
18338 5602 Acked Read 10 Done Done Sent false
18339 5603 Acked Write 1 Done Done Sent false
18340 5604 Acked Write 2 Done Done Sent false
18341 5605 Acked Read 2 Done Done Sent false
18342 5606 Acked Read 1 Done Done Sent false
18343 5607 Acked Write 5 Done Done Sent false
18344 5608 Acked Write 7 Done Done Sent false
18345 5609 Acked Read 10 Done Done Sent false
18346 5610 Acked Write 7 Done Done Sent false
18347 5611 Acked Read 2 Done Done Sent false
18348 5612 Acked Read 3 Done Done Sent false
18349 5613 Acked Read 8 Done Done Sent false
18350 5614 Acked Write 10 Done Done Sent false
18351 5615 Acked Write 8 Done Done Sent false
18352 5616 Acked Read 1 Done Done Sent false
18353 5617 Acked Write 2 Done Done Sent false
18354 5618 Acked Write 7 Done Done Sent false
18355 5619 Acked Read 8 Done Done Sent false
18356 5620 Acked Read 10 Done Done Sent false
18357 5621 Acked Write 9 Done Done Sent false
18358 5622 Acked Read 9 Done Done Sent false
18359 5623 Acked Read 3 Done Done Sent false
18360 5624 Acked Read 2 Done Done Sent false
18361 5625 Acked Read 2 Done Done Sent false
18362 5626 Acked Write 5 Done Done Sent false
18363 5627 Acked Write 4 Done Done Sent false
18364 5628 Acked Read 3 Done Done Sent false
18365 5629 Acked Read 1 Done Done Sent false
18366 5630 Acked Write 3 Done Done Sent false
18367 5631 Acked Read 4 Done Done Sent false
18368 5632 Acked Write 8 Done Done Sent false
18369 5633 Acked Write 9 Done Done Sent false
18370 5634 Acked Read 9 Done Done Sent false
18371 5635 Acked Read 2 Done Done Sent false
18372 5636 Acked Read 7 Done Done Sent false
18373 5637 Acked Read 10 Done Done Sent false
18374 5638 Acked Write 3 Done Done Sent false
18375 5639 Acked Write 6 Done Done Sent false
18376 5640 Acked Read 6 Done Done Sent false
18377 5641 Acked Write 3 Done Done Sent false
18378 5642 Acked Read 10 Done Done Sent false
18379 5643 Acked Write 4 Done Done Sent false
18380 5644 Acked Write 10 Done Done Sent false
18381 5645 Acked Write 10 Done Done Sent false
18382 5646 Acked Read 4 Done Done Sent false
18383 5647 Acked Read 9 Done Done Sent false
18384 5648 Acked Read 8 Done Done Sent false
18385 5649 Acked Write 2 Done Done Sent false
18386 5650 Acked Write 6 Done Done Sent false
18387 5651 Acked Read 3 Done Done Sent false
18388 5652 Acked Read 8 Done Done Sent false
18389 5653 Acked Read 9 Done Done Sent false
18390 5654 Acked Write 10 Done Done Sent false
18391 5655 Acked Read 8 Done Done Sent false
18392 5656 Acked Read 4 Done Done Sent false
18393 5657 Acked Write 8 Done Done Sent false
18394 5658 Acked Read 7 Done Done Sent false
18395 5659 Acked Read 3 Done Done Sent false
18396 5660 Acked Read 4 Done Done Sent false
18397 5661 Acked Write 1 Done Done Sent false
18398 5662 Acked Read 8 Done Done Sent false
18399 5663 Acked Read 8 Done Done Sent false
18400 5664 Acked Read 8 Done Done Sent false
18401 5665 Acked Read 5 Done Done Sent false
18402 5666 Acked Read 2 Done Done Sent false
18403 5667 Acked Read 3 Done Done Sent false
18404 5668 Acked Write 5 Done Done Sent false
18405 5669 Acked Read 4 Done Done Sent false
18406 5670 Acked Write 6 Done Done Sent false
18407 5671 Acked Read 4 Done Done Sent false
18408 5672 Acked Read 7 Done Done Sent false
18409 5673 Acked Flush 0 Done Done Sent false
18410 5674 Acked Write 5 Done Done Sent false
18411 5675 Acked Write 3 Done Done Sent false
18412 5676 Acked Write 5 Done Done Sent false
18413 5677 Acked Read 1 Done Done Sent false
18414 5678 Acked Write 5 Done Done Sent false
18415 5679 Acked Read 6 Done Done Sent false
18416 5680 Acked Read 8 Done Done Sent false
18417 5681 Acked Read 5 Done Done Sent false
18418 5682 Acked Read 10 Done Done Sent false
18419 5683 Acked Flush 0 Done Done Sent false
18420 5684 Acked Read 6 Done Done Sent false
18421 5685 Acked Write 5 Done Done Sent false
18422 5686 Acked Read 2 Done Done Sent false
18423 5687 Acked Read 4 Done Done Sent false
18424 5688 Acked Write 2 Done Done Sent false
18425 5689 Acked Read 6 Done Done Sent false
18426 5690 Acked Flush 0 Done Done Sent false
18427 5691 Acked Flush 0 Done Done Sent false
18428 5692 Acked Flush 0 Done Done Sent false
18429 5693 Acked Read 9 Done Done Sent false
18430 5694 Acked Read 8 Done Done Sent false
18431 5695 Acked Read 7 Done Done Sent false
18432 5696 Acked Read 6 Done Done Sent false
18433 5697 Acked Read 8 Done Done Sent false
18434 5698 Acked Write 4 Done Done Sent false
18435 5699 Acked Read 3 Done Done Sent false
18436 5700 Acked Write 3 Done Done Sent false
18437 5701 Acked Write 3 Done Done Sent false
18438 5702 Acked Write 2 Done Done Sent false
18439 5703 Acked Write 2 Done Done Sent false
18440 5704 Acked Write 9 Done Done Sent false
18441 5705 Acked Read 9 Done Done Sent false
18442 5706 Acked Write 6 Done Done Sent false
18443 5707 Acked Read 6 Done Done Sent false
18444 5708 Acked Write 2 Done Done Sent false
18445 5709 Acked Read 6 Done Done Sent false
18446 5710 Acked Read 3 Done Done Sent false
18447 5711 Acked Write 9 Done Done Sent false
18448 5712 Acked Write 1 Done Done Sent false
18449 5713 Acked Read 4 Done Done Sent false
18450 5714 Acked Write 4 Done Done Sent false
18451 5715 Acked Write 2 Done Done Sent false
18452 5716 Acked Write 9 Done Done Sent false
18453 5717 Acked Write 2 Done Done Sent false
18454 5718 Acked Read 1 Done Done Sent false
18455 5719 Acked Read 1 Done Done Sent false
18456 5720 Acked Read 1 Done Done Sent false
18457 5721 Acked Write 7 Done Done Sent false
18458 5722 Acked Write 4 Done Done Sent false
18459 5723 Acked Read 9 Done Done Sent false
18460 5724 Acked Flush 0 Done Done Sent false
18461 5725 Acked Write 7 Done Done Sent false
18462 5726 Acked Read 7 Done Done Sent false
18463 5727 Acked Read 6 Done Done Sent false
18464 5728 Acked Write 1 Done Done Sent false
18465 5729 Acked Read 1 Done Done Sent false
18466 5730 Acked Flush 0 Done Done Sent false
18467 5731 Acked Read 6 Done Done Sent false
18468 5732 Acked Read 4 Done Done Sent false
18469 5733 Acked Read 10 Done Done Sent false
18470 5734 Acked Write 9 Done Done Sent false
18471 5735 Acked Read 3 Done Done Sent false
18472 5736 Acked Read 8 Done Done Sent false
18473 5737 Acked Read 7 Done Done Sent false
18474 5738 Acked Write 8 Done Done Sent false
18475 5739 Acked Flush 0 Done Done Sent false
18476 5740 Acked Read 6 Done Done Sent false
18477 5741 Acked Write 9 Done Done Sent false
18478 5742 Acked Write 7 Done Done Sent false
18479 5743 Acked Write 7 Done Done Sent false
18480 5744 Acked Flush 0 Done Done Sent false
18481 5745 Acked Write 6 Done Done Sent false
18482 5746 Acked Read 10 Done Done Sent false
18483 5747 Acked Write 3 Done Done Sent false
18484 5748 Acked Read 10 Done Done Sent false
18485 5749 Acked Read 5 Done Done Sent false
18486 5750 Acked Read 4 Done Done Sent false
18487 5751 Acked Read 2 Done Done Sent false
18488 5752 Acked Write 9 Done Done Sent false
18489 5753 Acked Read 8 Done Done Sent false
18490 5754 Acked Read 8 Done Done Sent false
18491 5755 Acked Read 6 Done Done Sent false
18492 5756 Acked Read 9 Done Done Sent false
18493 5757 Acked Read 1 Done Done Sent false
18494 5758 Acked Write 5 Done Done Sent false
18495 5759 Acked Read 7 Done Done Sent false
18496 5760 Acked Read 6 Done Done Sent false
18497 5761 Acked Write 5 Done Done Sent false
18498 5762 Acked Write 5 Done Done Sent false
18499 5763 Acked Write 6 Done Done Sent false
18500 5764 Acked Write 3 Done Done Sent false
18501 5765 Acked Write 5 Done Done Sent false
18502 5766 Acked Read 9 Done Done Sent false
18503 5767 Acked Write 7 Done Done Sent false
18504 5768 Acked Flush 0 Done Done Sent false
18505 5769 Acked Write 8 Done Done Sent false
18506 5770 Acked Read 1 Done Done Sent false
18507 5771 Acked Read 5 Done Done Sent false
18508 5772 Acked Write 10 Done Done Sent false
18509 5773 Acked Read 7 Done Done Sent false
18510 5774 Acked Write 2 Done Done Sent false
18511 5775 Acked Write 3 Done Done Sent false
18512 5776 Acked Read 5 Done Done Sent false
18513 5777 Acked Write 2 Done Done Sent false
18514 5778 Acked Read 4 Done Done Sent false
18515 5779 Acked Write 2 Done Done Sent false
18516 5780 Acked Read 4 Done Done Sent false
18517 5781 Acked Read 4 Done Done Sent false
18518 5782 Acked Read 10 Done Done Sent false
18519 5783 Acked Read 1 Done Done Sent false
18520 5784 Acked Write 3 Done Done Sent false
18521 5785 Acked Read 4 Done Done Sent false
18522 5786 Acked Write 10 Done Done Sent false
18523 5787 Acked Read 9 Done Done Sent false
18524 5788 Acked Write 1 Done Done Sent false
18525 5789 Acked Write 10 Done Done Sent false
18526 5790 Acked Write 7 Done Done Sent false
18527 5791 Acked Read 9 Done Done Sent false
18528 5792 Acked Write 9 Done Done Sent false
18529 5793 Acked Write 9 Done Done Sent false
18530 5794 Acked Write 3 Done Done Sent false
18531 5795 Acked Read 1 Done Done Sent false
18532 5796 Acked Write 2 Done Done Sent false
18533 5797 Acked Write 6 Done Done Sent false
18534 5798 Acked Read 3 Done Done Sent false
18535 5799 Acked Write 1 Done Done Sent false
18536 5800 Acked Write 5 Done Done Sent false
18537 5801 Acked Read 7 Done Done Sent false
18538 5802 Acked Write 7 Done Done Sent false
18539 5803 Acked Flush 0 Done Done Sent false
18540 5804 Acked Flush 0 Done Done Sent false
18541 5805 Acked Write 6 Done Done Sent false
18542 5806 Acked Read 10 Done Done Sent false
18543 5807 Acked Read 8 Done Done Sent false
18544 5808 Acked Read 9 Done Done Sent false
18545 5809 Acked Write 6 Done Done Sent false
18546 5810 Acked Read 6 Done Done Sent false
18547 5811 Acked Read 1 Done Done Sent false
18548 5812 Acked Read 6 Done Done Sent false
18549 5813 Acked Write 6 Done Done Sent false
18550 5814 Acked Read 2 Done Done Sent false
18551 5815 Acked Read 8 Done Done Sent false
18552 5816 Acked Read 4 Done Done Sent false
18553 5817 Acked Read 10 Done Done Sent false
18554 5818 Acked Write 8 Done Done Sent false
18555 5819 Acked Write 6 Done Done Sent false
18556 5820 Acked Read 6 Done Done Sent false
18557 5821 Acked Write 3 Done Done Sent false
18558 5822 Acked Write 7 Done Done Sent false
18559 5823 Acked Read 4 Done Done Sent false
18560 5824 Acked Read 8 Done Done Sent false
18561 5825 Acked Write 9 Done Done Sent false
18562 5826 Acked Read 1 Done Done Sent false
18563 5827 Acked Write 2 Done Done Sent false
18564 5828 Acked Write 2 Done Done Sent false
18565 5829 Acked Write 3 Done Done Sent false
18566 5830 Acked Flush 0 Done Done Sent false
18567 STATES DS:0 DS:1 DS:2 TOTAL
18568 Sent 0 0 302 302
18569 Done 302 302 0 604
18570 Skipped 0 0 0 0
18571 Error 0 0 0 0
18572 Last Flush: 5830 5830 5528
18573 Downstairs last five completed: 5528 5527 5526 5525 5524
18574 Upstairs last five completed: 5830 5829 5828 5827 5826
18575 CLIENT: Up:0 ds:302 act:2
18576 ----------------------------------------------------------------
18577 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18578 States: Active Active Offline
18579 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18580 5529 Acked Read 2 Done Done Sent false
18581 5530 Acked Read 9 Done Done Sent false
18582 5531 Acked Read 7 Done Done Sent false
18583 5532 Acked Write 4 Done Done Sent false
18584 5533 Acked Read 5 Done Done Sent false
18585 5534 Acked Read 10 Done Done Sent false
18586 5535 Acked Flush 0 Done Done Sent false
18587 5536 Acked Read 1 Done Done Sent false
18588 5537 Acked Write 3 Done Done Sent false
18589 5538 Acked Flush 0 Done Done Sent false
18590 5539 Acked Read 7 Done Done Sent false
18591 5540 Acked Flush 0 Done Done Sent false
18592 5541 Acked Write 8 Done Done Sent false
18593 5542 Acked Read 4 Done Done Sent false
18594 5543 Acked Flush 0 Done Done Sent false
18595 5544 Acked Read 1 Done Done Sent false
18596 5545 Acked Write 9 Done Done Sent false
18597 5546 Acked Flush 0 Done Done Sent false
18598 5547 Acked Write 4 Done Done Sent false
18599 5548 Acked Read 2 Done Done Sent false
18600 5549 Acked Read 6 Done Done Sent false
18601 5550 Acked Read 7 Done Done Sent false
18602 5551 Acked Write 9 Done Done Sent false
18603 5552 Acked Read 9 Done Done Sent false
18604 5553 Acked Read 8 Done Done Sent false
18605 5554 Acked Read 7 Done Done Sent false
18606 5555 Acked Write 8 Done Done Sent false
18607 5556 Acked Read 10 Done Done Sent false
18608 5557 Acked Read 7 Done Done Sent false
18609 5558 Acked Write 2 Done Done Sent false
18610 5559 Acked Write 2 Done Done Sent false
18611 5560 Acked Write 2 Done Done Sent false
18612 5561 Acked Read 2 Done Done Sent false
18613 5562 Acked Read 7 Done Done Sent false
18614 5563 Acked Read 3 Done Done Sent false
18615 5564 Acked Write 6 Done Done Sent false
18616 5565 Acked Read 3 Done Done Sent false
18617 5566 Acked Write 7 Done Done Sent false
18618 5567 Acked Read 1 Done Done Sent false
18619 5568 Acked Read 10 Done Done Sent false
18620 5569 Acked Read 4 Done Done Sent false
18621 5570 Acked Read 9 Done Done Sent false
18622 5571 Acked Read 6 Done Done Sent false
18623 5572 Acked Read 3 Done Done Sent false
18624 5573 Acked Write 8 Done Done Sent false
18625 5574 Acked Write 5 Done Done Sent false
18626 5575 Acked Read 2 Done Done Sent false
18627 5576 Acked Flush 0 Done Done Sent false
18628 5577 Acked Flush 0 Done Done Sent false
18629 5578 Acked Write 8 Done Done Sent false
18630 5579 Acked Read 7 Done Done Sent false
18631 5580 Acked Write 4 Done Done Sent false
18632 5581 Acked Read 8 Done Done Sent false
18633 5582 Acked Read 6 Done Done Sent false
18634 5583 Acked Read 10 Done Done Sent false
18635 5584 Acked Read 4 Done Done Sent false
18636 5585 Acked Write 6 Done Done Sent false
18637 5586 Acked Write 3 Done Done Sent false
18638 5587 Acked Write 5 Done Done Sent false
18639 5588 Acked Read 2 Done Done Sent false
18640 5589 Acked Read 1 Done Done Sent false
18641 5590 Acked Write 9 Done Done Sent false
18642 5591 Acked Write 9 Done Done Sent false
18643 5592 Acked Write 10 Done Done Sent false
18644 5593 Acked Read 5 Done Done Sent false
18645 5594 Acked Read 3 Done Done Sent false
18646 5595 Acked Read 8 Done Done Sent false
18647 5596 Acked Read 3 Done Done Sent false
18648 5597 Acked Write 2 Done Done Sent false
18649 5598 Acked Read 1 Done Done Sent false
18650 5599 Acked Write 5 Done Done Sent false
18651 5600 Acked Write 10 Done Done Sent false
18652 5601 Acked Write 2 Done Done Sent false
18653 5602 Acked Read 10 Done Done Sent false
18654 5603 Acked Write 1 Done Done Sent false
18655 5604 Acked Write 2 Done Done Sent false
18656 5605 Acked Read 2 Done Done Sent false
18657 5606 Acked Read 1 Done Done Sent false
18658 5607 Acked Write 5 Done Done Sent false
18659 5608 Acked Write 7 Done Done Sent false
18660 5609 Acked Read 10 Done Done Sent false
18661 5610 Acked Write 7 Done Done Sent false
18662 5611 Acked Read 2 Done Done Sent false
18663 5612 Acked Read 3 Done Done Sent false
18664 5613 Acked Read 8 Done Done Sent false
18665 5614 Acked Write 10 Done Done Sent false
18666 5615 Acked Write 8 Done Done Sent false
18667 5616 Acked Read 1 Done Done Sent false
18668 5617 Acked Write 2 Done Done Sent false
18669 5618 Acked Write 7 Done Done Sent false
18670 5619 Acked Read 8 Done Done Sent false
18671 5620 Acked Read 10 Done Done Sent false
18672 5621 Acked Write 9 Done Done Sent false
18673 5622 Acked Read 9 Done Done Sent false
18674 5623 Acked Read 3 Done Done Sent false
18675 5624 Acked Read 2 Done Done Sent false
18676 5625 Acked Read 2 Done Done Sent false
18677 5626 Acked Write 5 Done Done Sent false
18678 5627 Acked Write 4 Done Done Sent false
18679 5628 Acked Read 3 Done Done Sent false
18680 5629 Acked Read 1 Done Done Sent false
18681 5630 Acked Write 3 Done Done Sent false
18682 5631 Acked Read 4 Done Done Sent false
18683 5632 Acked Write 8 Done Done Sent false
18684 5633 Acked Write 9 Done Done Sent false
18685 5634 Acked Read 9 Done Done Sent false
18686 5635 Acked Read 2 Done Done Sent false
18687 5636 Acked Read 7 Done Done Sent false
18688 5637 Acked Read 10 Done Done Sent false
18689 5638 Acked Write 3 Done Done Sent false
18690 5639 Acked Write 6 Done Done Sent false
18691 5640 Acked Read 6 Done Done Sent false
18692 5641 Acked Write 3 Done Done Sent false
18693 5642 Acked Read 10 Done Done Sent false
18694 5643 Acked Write 4 Done Done Sent false
18695 5644 Acked Write 10 Done Done Sent false
18696 5645 Acked Write 10 Done Done Sent false
18697 5646 Acked Read 4 Done Done Sent false
18698 5647 Acked Read 9 Done Done Sent false
18699 5648 Acked Read 8 Done Done Sent false
18700 5649 Acked Write 2 Done Done Sent false
18701 5650 Acked Write 6 Done Done Sent false
18702 5651 Acked Read 3 Done Done Sent false
18703 5652 Acked Read 8 Done Done Sent false
18704 5653 Acked Read 9 Done Done Sent false
18705 5654 Acked Write 10 Done Done Sent false
18706 5655 Acked Read 8 Done Done Sent false
18707 5656 Acked Read 4 Done Done Sent false
18708 5657 Acked Write 8 Done Done Sent false
18709 5658 Acked Read 7 Done Done Sent false
18710 5659 Acked Read 3 Done Done Sent false
18711 5660 Acked Read 4 Done Done Sent false
18712 5661 Acked Write 1 Done Done Sent false
18713 5662 Acked Read 8 Done Done Sent false
18714 5663 Acked Read 8 Done Done Sent false
18715 5664 Acked Read 8 Done Done Sent false
18716 5665 Acked Read 5 Done Done Sent false
18717 5666 Acked Read 2 Done Done Sent false
18718 5667 Acked Read 3 Done Done Sent false
18719 5668 Acked Write 5 Done Done Sent false
18720 5669 Acked Read 4 Done Done Sent false
18721 5670 Acked Write 6 Done Done Sent false
18722 5671 Acked Read 4 Done Done Sent false
18723 5672 Acked Read 7 Done Done Sent false
18724 5673 Acked Flush 0 Done Done Sent false
18725 5674 Acked Write 5 Done Done Sent false
18726 5675 Acked Write 3 Done Done Sent false
18727 5676 Acked Write 5 Done Done Sent false
18728 5677 Acked Read 1 Done Done Sent false
18729 5678 Acked Write 5 Done Done Sent false
18730 5679 Acked Read 6 Done Done Sent false
18731 5680 Acked Read 8 Done Done Sent false
18732 5681 Acked Read 5 Done Done Sent false
18733 5682 Acked Read 10 Done Done Sent false
18734 5683 Acked Flush 0 Done Done Sent false
18735 5684 Acked Read 6 Done Done Sent false
18736 5685 Acked Write 5 Done Done Sent false
18737 5686 Acked Read 2 Done Done Sent false
18738 5687 Acked Read 4 Done Done Sent false
18739 5688 Acked Write 2 Done Done Sent false
18740 5689 Acked Read 6 Done Done Sent false
18741 5690 Acked Flush 0 Done Done Sent false
18742 5691 Acked Flush 0 Done Done Sent false
18743 5692 Acked Flush 0 Done Done Sent false
18744 5693 Acked Read 9 Done Done Sent false
18745 5694 Acked Read 8 Done Done Sent false
18746 5695 Acked Read 7 Done Done Sent false
18747 5696 Acked Read 6 Done Done Sent false
18748 5697 Acked Read 8 Done Done Sent false
18749 5698 Acked Write 4 Done Done Sent false
18750 5699 Acked Read 3 Done Done Sent false
18751 5700 Acked Write 3 Done Done Sent false
18752 5701 Acked Write 3 Done Done Sent false
18753 5702 Acked Write 2 Done Done Sent false
18754 5703 Acked Write 2 Done Done Sent false
18755 5704 Acked Write 9 Done Done Sent false
18756 5705 Acked Read 9 Done Done Sent false
18757 5706 Acked Write 6 Done Done Sent false
18758 5707 Acked Read 6 Done Done Sent false
18759 5708 Acked Write 2 Done Done Sent false
18760 5709 Acked Read 6 Done Done Sent false
18761 5710 Acked Read 3 Done Done Sent false
18762 5711 Acked Write 9 Done Done Sent false
18763 5712 Acked Write 1 Done Done Sent false
18764 5713 Acked Read 4 Done Done Sent false
18765 5714 Acked Write 4 Done Done Sent false
18766 5715 Acked Write 2 Done Done Sent false
18767 5716 Acked Write 9 Done Done Sent false
18768 5717 Acked Write 2 Done Done Sent false
18769 5718 Acked Read 1 Done Done Sent false
18770 5719 Acked Read 1 Done Done Sent false
18771 5720 Acked Read 1 Done Done Sent false
18772 5721 Acked Write 7 Done Done Sent false
18773 5722 Acked Write 4 Done Done Sent false
18774 5723 Acked Read 9 Done Done Sent false
18775 5724 Acked Flush 0 Done Done Sent false
18776 5725 Acked Write 7 Done Done Sent false
18777 5726 Acked Read 7 Done Done Sent false
18778 5727 Acked Read 6 Done Done Sent false
18779 5728 Acked Write 1 Done Done Sent false
18780 5729 Acked Read 1 Done Done Sent false
18781 5730 Acked Flush 0 Done Done Sent false
18782 5731 Acked Read 6 Done Done Sent false
18783 5732 Acked Read 4 Done Done Sent false
18784 5733 Acked Read 10 Done Done Sent false
18785 5734 Acked Write 9 Done Done Sent false
18786 5735 Acked Read 3 Done Done Sent false
18787 5736 Acked Read 8 Done Done Sent false
18788 5737 Acked Read 7 Done Done Sent false
18789 5738 Acked Write 8 Done Done Sent false
18790 5739 Acked Flush 0 Done Done Sent false
18791 5740 Acked Read 6 Done Done Sent false
18792 5741 Acked Write 9 Done Done Sent false
18793 5742 Acked Write 7 Done Done Sent false
18794 5743 Acked Write 7 Done Done Sent false
18795 5744 Acked Flush 0 Done Done Sent false
18796 5745 Acked Write 6 Done Done Sent false
18797 5746 Acked Read 10 Done Done Sent false
18798 5747 Acked Write 3 Done Done Sent false
18799 5748 Acked Read 10 Done Done Sent false
18800 5749 Acked Read 5 Done Done Sent false
18801 5750 Acked Read 4 Done Done Sent false
18802 5751 Acked Read 2 Done Done Sent false
18803 5752 Acked Write 9 Done Done Sent false
18804 5753 Acked Read 8 Done Done Sent false
18805 5754 Acked Read 8 Done Done Sent false
18806 5755 Acked Read 6 Done Done Sent false
18807 5756 Acked Read 9 Done Done Sent false
18808 5757 Acked Read 1 Done Done Sent false
18809 5758 Acked Write 5 Done Done Sent false
18810 5759 Acked Read 7 Done Done Sent false
18811 5760 Acked Read 6 Done Done Sent false
18812 5761 Acked Write 5 Done Done Sent false
18813 5762 Acked Write 5 Done Done Sent false
18814 5763 Acked Write 6 Done Done Sent false
18815 5764 Acked Write 3 Done Done Sent false
18816 5765 Acked Write 5 Done Done Sent false
18817 5766 Acked Read 9 Done Done Sent false
18818 5767 Acked Write 7 Done Done Sent false
18819 5768 Acked Flush 0 Done Done Sent false
18820 5769 Acked Write 8 Done Done Sent false
18821 5770 Acked Read 1 Done Done Sent false
18822 5771 Acked Read 5 Done Done Sent false
18823 5772 Acked Write 10 Done Done Sent false
18824 5773 Acked Read 7 Done Done Sent false
18825 5774 Acked Write 2 Done Done Sent false
18826 5775 Acked Write 3 Done Done Sent false
18827 5776 Acked Read 5 Done Done Sent false
18828 5777 Acked Write 2 Done Done Sent false
18829 5778 Acked Read 4 Done Done Sent false
18830 5779 Acked Write 2 Done Done Sent false
18831 5780 Acked Read 4 Done Done Sent false
18832 5781 Acked Read 4 Done Done Sent false
18833 5782 Acked Read 10 Done Done Sent false
18834 5783 Acked Read 1 Done Done Sent false
18835 5784 Acked Write 3 Done Done Sent false
18836 5785 Acked Read 4 Done Done Sent false
18837 5786 Acked Write 10 Done Done Sent false
18838 5787 Acked Read 9 Done Done Sent false
18839 5788 Acked Write 1 Done Done Sent false
18840 5789 Acked Write 10 Done Done Sent false
18841 5790 Acked Write 7 Done Done Sent false
18842 5791 Acked Read 9 Done Done Sent false
18843 5792 Acked Write 9 Done Done Sent false
18844 5793 Acked Write 9 Done Done Sent false
18845 5794 Acked Write 3 Done Done Sent false
18846 5795 Acked Read 1 Done Done Sent false
18847 5796 Acked Write 2 Done Done Sent false
18848 5797 Acked Write 6 Done Done Sent false
18849 5798 Acked Read 3 Done Done Sent false
18850 5799 Acked Write 1 Done Done Sent false
18851 5800 Acked Write 5 Done Done Sent false
18852 5801 Acked Read 7 Done Done Sent false
18853 5802 Acked Write 7 Done Done Sent false
18854 5803 Acked Flush 0 Done Done Sent false
18855 5804 Acked Flush 0 Done Done Sent false
18856 5805 Acked Write 6 Done Done Sent false
18857 5806 Acked Read 10 Done Done Sent false
18858 5807 Acked Read 8 Done Done Sent false
18859 5808 Acked Read 9 Done Done Sent false
18860 5809 Acked Write 6 Done Done Sent false
18861 5810 Acked Read 6 Done Done Sent false
18862 5811 Acked Read 1 Done Done Sent false
18863 5812 Acked Read 6 Done Done Sent false
18864 5813 Acked Write 6 Done Done Sent false
18865 5814 Acked Read 2 Done Done Sent false
18866 5815 Acked Read 8 Done Done Sent false
18867 5816 Acked Read 4 Done Done Sent false
18868 5817 Acked Read 10 Done Done Sent false
18869 5818 Acked Write 8 Done Done Sent false
18870 5819 Acked Write 6 Done Done Sent false
18871 5820 Acked Read 6 Done Done Sent false
18872 5821 Acked Write 3 Done Done Sent false
18873 5822 Acked Write 7 Done Done Sent false
18874 5823 Acked Read 4 Done Done Sent false
18875 5824 Acked Read 8 Done Done Sent false
18876 5825 Acked Write 9 Done Done Sent false
18877 5826 Acked Read 1 Done Done Sent false
18878 5827 Acked Write 2 Done Done Sent false
18879 5828 Acked Write 2 Done Done Sent false
18880 5829 Acked Write 3 Done Done Sent false
18881 5830 Acked Flush 0 Done Done Sent false
18882 STATES DS:0 DS:1 DS:2 TOTAL
18883 Sent 0 0 302 302
18884 Done 302 302 0 604
18885 Skipped 0 0 0 0
18886 Error 0 0 0 0
18887 Last Flush: 5830 5830 5528
18888 Downstairs last five completed: 5528 5527 5526 5525 5524
18889 Upstairs last five completed: 5830 5829 5828 5827 5826
18890 CLIENT: Up:0 ds:302 act:2
188912025-01-15T05:22:58.628ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
188922025-01-15T05:22:58.628ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42477) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
188932025-01-15T05:22:58.629ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
188942025-01-15T05:22:58.629ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
188952025-01-15T05:22:58.629ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
188962025-01-15T05:22:58.629ZINFOcrucible: send last flush ID to this DS: 5528 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
188972025-01-15T05:22:58.630ZINFOcrucible: Replied this last flush ID: 5528 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
188982025-01-15T05:22:58.630ZINFOcrucible: [2] client re-new 302 jobs since flush 5528 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
188992025-01-15T05:22:58.630ZINFOcrucible: [2] Replayed 302 jobs since flush: 5528 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
18900 ----------------------------------------------------------------
18901 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18902 Downstairs last five completed: 5830 5829 5828 5827 5826
18903 Upstairs last five completed: 5830 5829 5828 5827 5826
18904 CLIENT: Up:0 ds:0 act:3
18905 Replay: All jobs finished
18906 Main task has work to do, go find it
18907 got dsc Stop(0)
18908 stop 0
18909 Jan 15 05:22:58.873 [8810] Got stop action so:false kr:false
18910 INFO request completed, latency_us: 303, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 166d6046-2f5d-433d-a607-9764abd4c8ab, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
189112025-01-15T05:22:58.874ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
189122025-01-15T05:22:58.874ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
189132025-01-15T05:22:58.874ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
18914 Jan 15 05:22:58.874 INFO request completed, latency_us: 155, response_code: 200, uri: /state/cid/0, method: GET, req_id: f0064169-cffb-4c86-9162-05469f65e09c, remote_addr: 127.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:22:58.874292127Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
18915 0.0.1:42402, local_addr: 127.0.0.1:9998
18916 [8810] Exited with: ExitStatus(unix_wait_status(9))
18917 [8810][0] reports Exit
18918 Jan 15 05:23:02.876 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/0, method: GET, req_id: 711a4ff5-3ac0-4fd4-8a68-cba5aa35e244, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
18919 001/300 Flush
18920 002/300 Write block 2768 len 40960 data: 4 3 3 3 3 3 3 3 2 4
18921 003/300 Read block 1262 len 24576
18922 004/300 Write block 2036 len 36864 data: 2 2 2 3 3 3 3 4 4
18923 005/300 Write block 2211 len 24576 data: 4 2 2 2 3 3
18924 006/300 Flush
18925 007/300 Flush
18926 008/300 Read block 4847 len 32768
18927 009/300 Flush
18928 010/300 Write block 1897 len 8192 data: 4 4
18929 011/300 Flush
18930 012/300 Write block 2368 len 28672 data: 8 7 7 5 5 4 4
18931 013/300 Write block 3195 len 24576 data: 2 2 2 2 3 3
18932 014/300 Read block 3150 len 20480
18933 015/300 Write block 2786 len 20480 data: 6 6 6 5 4
18934 016/300 Read block 2575 len 12288
18935 017/300 Read block 4666 len 36864
18936 018/300 Write block 4583 len 32768 data: 4 4 4 4 4 4 4 3
18937 019/300 Flush
18938 020/300 Write block 4511 len 20480 data: 3 3 3 3 4
18939 021/300 Write block 2377 len 24576 data: 3 2 2 2 2 2
18940 022/300 Write block 1761 len 24576 data: 6 6 6 5 5 4
18941 023/300 Read block 2251 len 12288
18942 024/300 Write block 4040 len 24576 data: 4 3 3 2 2 3
18943 025/300 Read block 1743 len 24576
18944 026/300 Read block 1651 len 20480
18945 027/300 Write block 2138 len 8192 data: 2 2
18946 028/300 Read block 2944 len 4096
18947 029/300 Read block 2567 len 12288
18948 030/300 Write block 3498 len 8192 data: 7 6
18949 031/300 Write block 4050 len 32768 data: 5 7 5 6 5 3 3 4
18950 032/300 Flush
18951 033/300 Write block 1395 len 16384 data: 2 2 3 3
18952 034/300 Write block 3036 len 28672 data: 8 8 7 7 8 8 8
18953 035/300 Read block 4211 len 12288
18954 036/300 Write block 1164 len 4096 data: 6
18955 037/300 Read block 2049 len 16384
18956 038/300 Write block 3205 len 20480 data: 3 3 3 2 2
18957 039/300 Write block 247 len 32768 data: 2 3 3 3 4 4 4 6
18958 040/300 Flush
18959 041/300 Read block 2827 len 16384
18960 042/300 Read block 894 len 28672
18961 043/300 Flush
18962 044/300 Read block 945 len 24576
18963 045/300 Read block 1638 len 20480
18964 046/300 Read block 2981 len 40960
18965 047/300 Write block 1698 len 36864 data: 3 3 4 4 3 3 4 3 3
18966 048/300 Read block 2511 len 12288
18967 049/300 Write block 2492 len 12288 data: 5 4 4
18968 050/300 Read block 2522 len 24576
18969 051/300 Read block 1800 len 28672
18970 052/300 Flush
18971 053/300 Read block 4482 len 8192
18972 054/300 Write block 213 len 24576 data: 2 3 4 3 3 6
18973 055/300 Flush
18974 056/300 Flush
18975 057/300 Read block 1648 len 16384
18976 058/300 Read block 3898 len 40960
18977 059/300 Write block 4865 len 40960 data: 4 3 3 3 6 6 5 6 6 7
18978 060/300 Read block 4235 len 40960
18979 061/300 Flush
18980 062/300 Write block 547 len 4096 data: 4
18981 063/300 Write block 2194 len 32768 data: 3 4 3 3 3 5 5 3
18982 064/300 Read block 4216 len 36864
18983 065/300 Write block 2524 len 8192 data: 3 4
18984 066/300 Write block 1374 len 4096 data: 4
18985 067/300 Read block 2295 len 12288
18986 068/300 Write block 688 len 36864 data: 4 5 7 8 6 5 5 5 6
18987 069/300 Write block 1229 len 8192 data: 2 3
18988 070/300 Write block 3625 len 28672 data: 2 2 2 2 2 2 2
18989 071/300 Read block 149 len 40960
18990 072/300 Read block 4291 len 12288
18991 073/300 Read block 1441 len 8192
18992 074/300 Write block 1833 len 32768 data: 6 6 5 4 4 5 5 5
18993 075/300 Read block 2719 len 20480
18994 076/300 Write block 2636 len 40960 data: 4 4 3 3 3 3 4 4 4 4
18995 077/300 Write block 4137 len 32768 data: 3 3 2 3 3 4 6 6
18996 078/300 Read block 759 len 8192
18997 079/300 Read block 2800 len 24576
18998 080/300 Read block 2043 len 16384
18999 081/300 Write block 4063 len 12288 data: 2 2 2
19000 082/300 Read block 3900 len 16384
19001 083/300 Flush
19002 084/300 Read block 3110 len 40960
19003 085/300 Read block 1099 len 28672
19004 086/300 Write block 445 len 32768 data: 10 10 8 7 6 5 4 4
19005 087/300 Flush
19006 088/300 Read block 3181 len 20480
19007 089/300 Read block 574 len 40960
19008 090/300 Write block 1753 len 12288 data: 4 3 3
19009 091/300 Write block 2158 len 28672 data: 2 2 2 2 2 3 3
19010 092/300 Write block 2197 len 24576 data: 4 4 6 6 4 4
19011 093/300 Write block 4810 len 24576 data: 2 2 2 2 3 5
19012 094/300 Read block 3991 len 28672
19013 095/300 Read block 2059 len 4096
19014 096/300 Write block 4929 len 36864 data: 4 3 2 3 3 3 3 3 4
19015 097/300 Read block 2268 len 36864
19016 098/300 Flush
19017 099/300 Write block 2304 len 36864 data: 2 2 2 2 2 2 3 3 3
19018 100/300 Flush
19019 101/300 Read block 3130 len 40960
19020 102/300 Read block 1511 len 24576
19021 103/300 Write block 4268 len 32768 data: 4 3 3 3 3 3 3 3
19022 104/300 Write block 1800 len 40960 data: 5 4 5 5 4 3 4 3 4 4
19023 105/300 Write block 2236 len 28672 data: 5 5 5 5 6 6 6
19024 106/300 Read block 2984 len 40960
19025 107/300 Read block 1140 len 8192
19026 108/300 Read block 1601 len 24576
19027 109/300 Read block 2961 len 24576
19028 110/300 Write block 256 len 20480 data: 7 7 7 4 4
19029 111/300 Read block 4435 len 24576
19030 112/300 Write block 2316 len 28672 data: 4 5 5 4 3 3 2
19031 113/300 Read block 2678 len 8192
19032 114/300 Read block 1768 len 8192
19033 115/300 Write block 2850 len 20480 data: 4 3 6 4 4
19034 116/300 Read block 1977 len 32768
19035 117/300 Read block 4931 len 20480
19036 118/300 Read block 4365 len 36864
19037 119/300 Write block 1306 len 8192 data: 5 4
19038 120/300 Read block 3250 len 8192
19039 121/300 Read block 2403 len 36864
19040 122/300 Write block 4940 len 40960 data: 3 3 3 2 2 2 2 2 2 2
19041 123/300 Write block 4415 len 12288 data: 4 4 4
19042 124/300 Write block 3639 len 28672 data: 3 3 3 4 4 4 3
19043 125/300 Read block 4796 len 32768
19044 126/300 Write block 909 len 16384 data: 3 4 4 4
19045 127/300 Read block 2804 len 20480
19046 128/300 Read block 4694 len 12288
19047 129/300 Write block 1773 len 32768 data: 3 4 4 4 4 4 3 4
19048 130/300 Read block 3114 len 20480
19049 131/300 Write block 3974 len 8192 data: 4 5
19050 132/300 Flush
19051 133/300 Write block 4501 len 20480 data: 7 6 5 4 3
19052 134/300 Read block 3175 len 36864
19053 135/300 Flush
19054 136/300 Write block 630 len 40960 data: 3 3 3 3 6 5 6 6 5 7
19055 137/300 Read block 2531 len 8192
19056 138/300 Flush
19057 139/300 Read block 89 len 20480
19058 140/300 Write block 559 len 32768 data: 5 5 5 4 5 5 5 5
19059 141/300 Write block 3091 len 12288 data: 6 7 6
19060 142/300 Read block 2882 len 40960
19061 143/300 Flush
19062 144/300 Read block 480 len 8192
19063 145/300 Write block 1553 len 12288 data: 2 2 3
19064 146/300 Read block 1119 len 4096
19065 147/300 Write block 4274 len 4096 data: 4
19066 148/300 Write block 2714 len 16384 data: 5 5 5 5
19067 149/300 Write block 3783 len 16384 data: 4 5 5 4
19068 150/300 Read block 3615 len 24576
19069 151/300 Write block 2470 len 32768 data: 4 4 4 3 3 3 3 3
19070 152/300 Write block 4727 len 8192 data: 5 6
19071 153/300 Read block 172 len 28672
19072 154/300 Write block 4938 len 16384 data: 3 3 4 4
19073 155/300 Read block 3754 len 12288
19074 156/300 Write block 278 len 28672 data: 4 3 3 2 2 2 4
19075 157/300 Read block 583 len 24576
19076 158/300 Write block 1649 len 20480 data: 4 3 3 3 3
19077 159/300 Read block 960 len 12288
19078 160/300 Write block 4937 len 36864 data: 5 4 4 5 5 4 3 3 3
19079 161/300 Write block 3658 len 16384 data: 6 6 6 5
19080 162/300 Write block 2994 len 4096 data: 3
19081 163/300 Read block 3417 len 8192
19082 164/300 Write block 458 len 20480 data: 5 3 3 3 3
19083 165/300 Read block 3876 len 20480
19084 166/300 Write block 4572 len 40960 data: 2 4 4 4 3 3 3 4 5 6
19085 167/300 Read block 3568 len 36864
19086 168/300 Write block 964 len 12288 data: 5 5 4
19087 169/300 Read block 841 len 24576
19088 170/300 Read block 4693 len 40960
19089 171/300 Write block 161 len 28672 data: 4 4 4 4 3 4 3
19090 172/300 Read block 4988 len 40960
19091 173/300 Read block 2408 len 28672
19092 174/300 Write block 1055 len 24576 data: 2 2 2 2 2 2
19093 175/300 Read block 4590 len 4096
19094 176/300 Write block 3511 len 4096 data: 4
19095 177/300 Read block 3553 len 4096
19096 178/300 Write block 3630 len 32768 data: 3 3 2 3 2 2 2 3
19097 179/300 Read block 229 len 16384
19098 180/300 Read block 4131 len 24576
19099 181/300 Flush
19100 182/300 Read block 3637 len 4096
19101 183/300 Read block 1877 len 36864
19102 184/300 Flush
19103 185/300 Read block 2592 len 16384
19104 186/300 Flush
19105 187/300 Read block 2704 len 16384
19106 188/300 Read block 1733 len 40960
19107 189/300 Read block 1249 len 32768
19108 190/300 Flush
19109 191/300 Read block 3734 len 12288
19110 192/300 Read block 1258 len 32768
19111 193/300 Read block 1855 len 16384
19112 194/300 Read block 134 len 20480
19113 195/300 Read block 232 len 32768
19114 196/300 Read block 3781 len 32768
19115 197/300 Write block 3488 len 12288 data: 3 3 2
19116 198/300 Read block 2996 len 40960
19117 199/300 Flush
19118 200/300 Write block 1646 len 40960 data: 5 4 3 5 4 4 4 4 3 3
19119 201/300 Flush
19120 202/300 Read block 4014 len 32768
19121 203/300 Read block 1148 len 20480
19122 204/300 Write block 2338 len 16384 data: 3 3 3 5
19123 205/300 Read block 4989 len 20480
19124 206/300 Write block 1267 len 24576 data: 5 4 3 3 4 3
19125 207/300 Write block 3834 len 36864 data: 3 4 5 5 4 5 5 5 5
19126 208/300 Write block 2774 len 36864 data: 4 4 3 5 3 3 7 6 5
19127 209/300 Write block 4631 len 12288 data: 5 5 4
19128 210/300 Read block 704 len 28672
19129 211/300 Write block 1134 len 32768 data: 2 2 2 2 2 2 2 3
19130 212/300 Read block 2344 len 20480
19131 213/300 Read block 3156 len 16384
19132 214/300 Read block 2809 len 28672
19133 215/300 Write block 4884 len 36864 data: 5 5 5 4 5 4 4 3 4
19134 216/300 Read block 366 len 12288
19135 217/300 Read block 837 len 4096
19136 218/300 Read block 802 len 24576
19137 219/300 Flush
19138 220/300 Read block 2035 len 16384
19139 221/300 Read block 3036 len 16384
19140 222/300 Read block 93 len 28672
19141 223/300 Read block 1638 len 32768
19142 224/300 Read block 3667 len 36864
19143 225/300 Read block 4794 len 40960
19144 226/300 Write block 1679 len 16384 data: 6 5 6 7
19145 227/300 Write block 264 len 12288 data: 5 5 6
19146 228/300 Write block 1295 len 40960 data: 4 3 3 4 5 6 7 5 5 5
19147 229/300 Write block 4577 len 20480 data: 4 4 5 6 7
19148 230/300 Write block 2137 len 40960 data: 2 3 3 2 2 2 4 4 4 3
19149 231/300 Write block 3680 len 4096 data: 8
19150 232/300 Read block 1013 len 28672
19151 233/300 Read block 378 len 20480
19152 234/300 Write block 1216 len 24576 data: 4 5 6 4 4 4
19153 235/300 Flush
19154 236/300 Write block 3166 len 32768 data: 6 4 3 4 3 4 3 4
19155 237/300 Read block 3546 len 24576
19156 238/300 Read block 3490 len 24576
19157 239/300 Write block 749 len 20480 data: 4 5 6 7 6
19158 240/300 Write block 3455 len 28672 data: 4 4 4 3 3 3 2
19159 241/300 Read block 3960 len 20480
19160 242/300 Write block 3483 len 16384 data: 3 2 2 2
19161 243/300 Read block 2041 len 40960
19162 244/300 Read block 3771 len 12288
19163 245/300 Read block 198 len 20480
19164 246/300 Read block 2349 len 12288
19165 247/300 Read block 2713 len 8192
19166 248/300 Read block 2664 len 12288
19167 249/300 Write block 1907 len 12288 data: 5 4 4
19168 250/300 Write block 1084 len 12288 data: 4 4 3
19169 251/300 Write block 4677 len 8192 data: 6 4
19170 252/300 Read block 3577 len 16384
19171 253/300 Read block 3638 len 4096
19172 254/300 Read block 4635 len 4096
19173 255/300 Read block 3738 len 4096
19174 256/300 Read block 1984 len 4096
19175 257/300 Read block 2006 len 28672
19176 258/300 Read block 4871 len 36864
19177 259/300 Read block 4469 len 8192
19178 260/300 Read block 3182 len 16384
19179 261/300 Write block 4742 len 12288 data: 5 7 8
19180 262/300 Write block 1598 len 24576 data: 5 4 4 4 4 7
19181 263/300 Read block 21 len 24576
19182 264/300 Read block 432 len 16384
19183 265/300 Read block 4268 len 40960
19184 266/300 Read block 4627 len 40960
19185 267/300 Write block 3692 len 20480 data: 5 5 7 6 7
19186 268/300 Read block 3886 len 36864
19187 269/300 Read block 4638 len 12288
19188 270/300 Write block 1673 len 24576 data: 2 3 2 3 4 6
19189 271/300 Flush
19190 272/300 Read block 3855 len 20480
19191 273/300 Flush
19192 274/300 Write block 1635 len 40960 data: 6 6 7 9 8 9 9 6 6 7
19193 275/300 Read block 1789 len 28672
19194 276/300 Read block 853 len 16384
19195 277/300 Read block 2385 len 20480
19196 278/300 Write block 2900 len 20480 data: 5 4 3 3 4
19197 279/300 Write block 660 len 32768 data: 3 3 4 4 3 3 3 3
19198 280/300 Write block 1087 len 24576 data: 4 3 2 2 4 5
19199 281/300 Write block 401 len 40960 data: 3 3 3 3 2 3 4 5 5 4
19200 282/300 Write block 4530 len 16384 data: 3 3 3 2
19201 283/300 Read block 4647 len 28672
19202 284/300 Read block 676 len 16384
19203 285/300 Read block 767 len 32768
19204 286/300 Read block 981 len 40960
19205 287/300 Read block 2542 len 12288
19206 288/300 Flush
19207 289/300 Write block 1173 len 12288 data: 5 5 4
19208 290/300 Read block 4021 len 20480
19209 291/300 Read block 4827 len 12288
19210 292/300 Write block 2488 len 40960 data: 6 5 5 6 6 5 5 3 4 5
19211 293/300 Write block 1209 len 16384 data: 5 3 3 4
19212 294/300 Flush
19213 295/300 Read block 3583 len 12288
19214 296/300 Read block 600 len 32768
19215 297/300 Read block 300 len 12288
19216 298/300 Read block 4335 len 32768
19217 299/300 Read block 1526 len 16384
19218 300/300 Read block 4851 len 32768
19219 301/300 Read block 3660 len 32768
19220 Main task has work to do, go find it
19221 got dsc Start(0)
19222 start 0
19223 Jan 15 05:23:03.118 INFO request completed, latency_us: 276, response_code: 204, uri: /start/cid/0, method: POST, req_id: 82826dda-6893-48b8-9221-17a914be24a5, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
19224 Replay: started 0, returned:Ok(())
19225 ----------------------------------------------------------------
19226 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19227 States: Offline Active Active
19228 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19229 5831 Acked Flush 0 Sent Done Done false
19230 5832 Acked Write 10 Sent Done Done false
19231 5833 Acked Read 6 Sent Done Done false
19232 5834 Acked Write 9 Sent Done Done false
19233 5835 Acked Write 6 Sent Done Done false
19234 5836 Acked Flush 0 Sent Done Done false
19235 5837 Acked Flush 0 Sent Done Done false
19236 5838 Acked Read 8 Sent Done Done false
19237 5839 Acked Flush 0 Sent Done Done false
19238 5840 Acked Write 2 Sent Done Done false
19239 5841 Acked Flush 0 Sent Done Done false
19240 5842 Acked Write 7 Sent Done Done false
19241 5843 Acked Write 6 Sent Done Done false
19242 5844 Acked Read 5 Sent Done Done false
19243 5845 Acked Write 5 Sent Done Done false
19244 5846 Acked Read 3 Sent Done Done false
19245 5847 Acked Read 9 Sent Done Done false
19246 5848 Acked Write 8 Sent Done Done false
19247 5849 Acked Flush 0 Sent Done Done false
19248 5850 Acked Write 5 Sent Done Done false
19249 5851 Acked Write 6 Sent Done Done false
19250 5852 Acked Write 6 Sent Done Done false
19251 5853 Acked Read 3 Sent Done Done false
19252 5854 Acked Write 6 Sent Done Done false
19253 5855 Acked Read 6 Sent Done Done false
19254 5856 Acked Read 5 Sent Done Done false
19255 5857 Acked Write 2 Sent Done Done false
19256 5858 Acked Read 1 Sent Done Done false
19257 5859 Acked Read 3 Sent Done Done false
19258 5860 Acked Write 2 Sent Done Done false
19259 5861 Acked Write 8 Sent Done Done false
19260 5862 Acked Flush 0 Sent Done Done false
19261 5863 Acked Write 4 Sent Done Done false
19262 5864 Acked Write 7 Sent Done Done false
19263 5865 Acked Read 3 Sent Done Done false
19264 5866 Acked Write 1 Sent Done Done false
19265 5867 Acked Read 4 Sent Done Done false
19266 5868 Acked Write 5 Sent Done Done false
19267 5869 Acked Write 8 Sent Done Done false
19268 5870 Acked Flush 0 Sent Done Done false
19269 5871 Acked Read 4 Sent Done Done false
19270 5872 Acked Read 7 Sent Done Done false
19271 5873 Acked Flush 0 Sent Done Done false
19272 5874 Acked Read 6 Sent Done Done false
19273 5875 Acked Read 5 Sent Done Done false
19274 5876 Acked Read 10 Sent Done Done false
19275 5877 Acked Write 9 Sent Done Done false
19276 5878 Acked Read 3 Sent Done Done false
19277 5879 Acked Write 3 Sent Done Done false
19278 5880 Acked Read 6 Sent Done Done false
19279 5881 Acked Read 7 Sent Done Done false
19280 5882 Acked Flush 0 Sent Done Done false
19281 5883 Acked Read 2 Sent Done Done false
19282 5884 Acked Write 6 Sent Done Done false
19283 5885 Acked Flush 0 Sent Done Done false
19284 5886 Acked Flush 0 Sent Done Done false
19285 5887 Acked Read 4 Sent Done Done false
19286 5888 Acked Read 10 Sent Done Done false
19287 5889 Acked Write 10 Sent Done Done false
19288 5890 Acked Read 10 Sent Done Done false
19289 5891 Acked Flush 0 Sent Done Done false
19290 5892 Acked Write 1 Sent Done Done false
19291 5893 Acked Write 8 Sent Done Done false
19292 5894 Acked Read 9 Sent Done Done false
19293 5895 Acked Write 2 Sent Done Done false
19294 5896 Acked Write 1 Sent Done Done false
19295 5897 Acked Read 3 Sent Done Done false
19296 5898 Acked Write 9 Sent Done Done false
19297 5899 Acked Write 2 Sent Done Done false
19298 5900 Acked Write 7 Sent Done Done false
19299 5901 Acked Read 10 Sent Done Done false
19300 5902 Acked Read 3 Sent Done Done false
19301 5903 Acked Read 2 Sent Done Done false
19302 5904 Acked Write 8 Sent Done Done false
19303 5905 Acked Read 5 Sent Done Done false
19304 5906 Acked Write 10 Sent Done Done false
19305 5907 Acked Write 8 Sent Done Done false
19306 5908 Acked Read 2 Sent Done Done false
19307 5909 Acked Read 6 Sent Done Done false
19308 5910 Acked Read 4 Sent Done Done false
19309 5911 Acked Write 3 Sent Done Done false
19310 5912 Acked Read 4 Sent Done Done false
19311 5913 Acked Flush 0 Sent Done Done false
19312 5914 Acked Read 10 Sent Done Done false
19313 5915 Acked Read 7 Sent Done Done false
19314 5916 Acked Write 8 Sent Done Done false
19315 5917 Acked Flush 0 Sent Done Done false
19316 5918 Acked Read 5 Sent Done Done false
19317 5919 Acked Read 10 Sent Done Done false
19318 5920 Acked Write 3 Sent Done Done false
19319 5921 Acked Write 7 Sent Done Done false
19320 5922 Acked Write 6 Sent Done Done false
19321 5923 Acked Write 6 Sent Done Done false
19322 5924 Acked Read 7 Sent Done Done false
19323 5925 Acked Read 1 Sent Done Done false
19324 5926 Acked Write 9 Sent Done Done false
19325 5927 Acked Read 9 Sent Done Done false
19326 5928 Acked Flush 0 Sent Done Done false
19327 5929 Acked Write 9 Sent Done Done false
19328 5930 Acked Flush 0 Sent Done Done false
19329 5931 Acked Read 10 Sent Done Done false
19330 5932 Acked Read 6 Sent Done Done false
19331 5933 Acked Write 8 Sent Done Done false
19332 5934 Acked Write 10 Sent Done Done false
19333 5935 Acked Write 7 Sent Done Done false
19334 5936 Acked Read 10 Sent Done Done false
19335 5937 Acked Read 2 Sent Done Done false
19336 5938 Acked Read 6 Sent Done Done false
19337 5939 Acked Read 6 Sent Done Done false
19338 5940 Acked Write 5 Sent Done Done false
19339 5941 Acked Read 6 Sent Done Done false
19340 5942 Acked Write 7 Sent Done Done false
19341 5943 Acked Read 2 Sent Done Done false
19342 5944 Acked Read 2 Sent Done Done false
19343 5945 Acked Write 5 Sent Done Done false
19344 5946 Acked Read 8 Sent Done Done false
19345 5947 Acked Read 5 Sent Done Done false
19346 5948 Acked Read 9 Sent Done Done false
19347 5949 Acked Write 2 Sent Done Done false
19348 5950 Acked Read 2 Sent Done Done false
19349 5951 Acked Read 9 Sent Done Done false
19350 5952 Acked Write 10 Sent Done Done false
19351 5953 Acked Write 3 Sent Done Done false
19352 5954 Acked Write 7 Sent Done Done false
19353 5955 Acked Read 8 Sent Done Done false
19354 5956 Acked Write 4 Sent Done Done false
19355 5957 Acked Read 5 Sent Done Done false
19356 5958 Acked Read 3 Sent Done Done false
19357 5959 Acked Write 8 Sent Done Done false
19358 5960 Acked Read 5 Sent Done Done false
19359 5961 Acked Write 2 Sent Done Done false
19360 5962 Acked Flush 0 Sent Done Done false
19361 5963 Acked Write 5 Sent Done Done false
19362 5964 Acked Read 9 Sent Done Done false
19363 5965 Acked Flush 0 Sent Done Done false
19364 5966 Acked Write 10 Sent Done Done false
19365 5967 Acked Read 2 Sent Done Done false
19366 5968 Acked Flush 0 Sent Done Done false
19367 5969 Acked Read 5 Sent Done Done false
19368 5970 Acked Write 8 Sent Done Done false
19369 5971 Acked Write 3 Sent Done Done false
19370 5972 Acked Read 10 Sent Done Done false
19371 5973 Acked Flush 0 Sent Done Done false
19372 5974 Acked Read 2 Sent Done Done false
19373 5975 Acked Write 3 Sent Done Done false
19374 5976 Acked Read 1 Sent Done Done false
19375 5977 Acked Write 1 Sent Done Done false
19376 5978 Acked Write 4 Sent Done Done false
19377 5979 Acked Write 4 Sent Done Done false
19378 5980 Acked Read 6 Sent Done Done false
19379 5981 Acked Write 8 Sent Done Done false
19380 5982 Acked Write 2 Sent Done Done false
19381 5983 Acked Read 7 Sent Done Done false
19382 5984 Acked Write 4 Sent Done Done false
19383 5985 Acked Read 3 Sent Done Done false
19384 5986 Acked Write 7 Sent Done Done false
19385 5987 Acked Read 6 Sent Done Done false
19386 5988 Acked Write 5 Sent Done Done false
19387 5989 Acked Read 3 Sent Done Done false
19388 5990 Acked Write 9 Sent Done Done false
19389 5991 Acked Write 4 Sent Done Done false
19390 5992 Acked Write 1 Sent Done Done false
19391 5993 Acked Read 2 Sent Done Done false
19392 5994 Acked Write 5 Sent Done Done false
19393 5995 Acked Read 5 Sent Done Done false
19394 5996 Acked Write 10 Sent Done Done false
19395 5997 Acked Read 9 Sent Done Done false
19396 5998 Acked Write 3 Sent Done Done false
19397 5999 Acked Read 6 Sent Done Done false
19398 6000 Acked Read 10 Sent Done Done false
19399 6001 Acked Write 7 Sent Done Done false
19400 6002 Acked Read 10 Sent Done Done false
19401 6003 Acked Read 7 Sent Done Done false
19402 6004 Acked Write 6 Sent Done Done false
19403 6005 Acked Read 1 Sent Done Done false
19404 6006 Acked Write 1 Sent Done Done false
19405 6007 Acked Read 1 Sent Done Done false
19406 6008 Acked Write 8 Sent Done Done false
19407 6009 Acked Read 4 Sent Done Done false
19408 6010 Acked Read 6 Sent Done Done false
19409 6011 Acked Flush 0 Sent Done Done false
19410 6012 Acked Read 1 Sent Done Done false
19411 6013 Acked Read 9 Sent Done Done false
19412 6014 Acked Flush 0 Sent Done Done false
19413 6015 Acked Read 4 Sent Done Done false
19414 6016 Acked Flush 0 Sent Done Done false
19415 6017 Acked Read 4 Sent Done Done false
19416 6018 Acked Read 10 Sent Done Done false
19417 6019 Acked Read 8 Sent Done Done false
19418 6020 Acked Flush 0 Sent Done Done false
19419 6021 Acked Read 3 Sent Done Done false
19420 6022 Acked Read 8 Sent Done Done false
19421 6023 Acked Read 4 Sent Done Done false
19422 6024 Acked Read 5 Sent Done Done false
19423 6025 Acked Read 8 Sent Done Done false
19424 6026 Acked Read 8 Sent Done Done false
19425 6027 Acked Write 3 Sent Done Done false
19426 6028 Acked Read 10 Sent Done Done false
19427 6029 Acked Flush 0 Sent Done Done false
19428 6030 Acked Write 10 Sent Done Done false
19429 6031 Acked Flush 0 Sent Done Done false
19430 6032 Acked Read 8 Sent Done Done false
19431 6033 Acked Read 5 Sent Done Done false
19432 6034 Acked Write 4 Sent Done Done false
19433 6035 Acked Read 5 Sent Done Done false
19434 6036 Acked Write 6 Sent Done Done false
19435 6037 Acked Write 9 Sent Done Done false
19436 6038 Acked Write 9 Sent Done Done false
19437 6039 Acked Write 3 Sent Done Done false
19438 6040 Acked Read 7 Sent Done Done false
19439 6041 Acked Write 8 Sent Done Done false
19440 6042 Acked Read 5 Sent Done Done false
19441 6043 Acked Read 4 Sent Done Done false
19442 6044 Acked Read 7 Sent Done Done false
19443 6045 Acked Write 9 Sent Done Done false
19444 6046 Acked Read 3 Sent Done Done false
19445 6047 Acked Read 1 Sent Done Done false
19446 6048 Acked Read 6 Sent Done Done false
19447 6049 Acked Flush 0 Sent Done Done false
19448 6050 Acked Read 4 Sent Done Done false
19449 6051 Acked Read 4 Sent Done Done false
19450 6052 Acked Read 7 Sent Done Done false
19451 6053 Acked Read 8 Sent Done Done false
19452 6054 Acked Read 9 Sent Done Done false
19453 6055 Acked Read 10 Sent Done Done false
19454 6056 Acked Write 4 Sent Done Done false
19455 6057 Acked Write 3 Sent Done Done false
19456 6058 Acked Write 10 Sent Done Done false
19457 6059 Acked Write 5 Sent Done Done false
19458 6060 Acked Write 10 Sent Done Done false
19459 6061 Acked Write 1 Sent Done Done false
19460 6062 Acked Read 7 Sent Done Done false
19461 6063 Acked Read 5 Sent Done Done false
19462 6064 Acked Write 6 Sent Done Done false
19463 6065 Acked Flush 0 Sent Done Done false
19464 6066 Acked Write 8 Sent Done Done false
19465 6067 Acked Read 6 Sent Done Done false
19466 6068 Acked Read 6 Sent Done Done false
19467 6069 Acked Write 5 Sent Done Done false
19468 6070 Acked Write 7 Sent Done Done false
19469 6071 Acked Read 5 Sent Done Done false
19470 6072 Acked Write 4 Sent Done Done false
19471 6073 Acked Read 10 Sent Done Done false
19472 6074 Acked Read 3 Sent Done Done false
19473 6075 Acked Read 5 Sent Done Done false
19474 6076 Acked Read 3 Sent Done Done false
19475 6077 Acked Read 2 Sent Done Done false
19476 6078 Acked Read 3 Sent Done Done false
19477 6079 Acked Write 3 Sent Done Done false
19478 6080 Acked Write 3 Sent Done Done false
19479 6081 Acked Write 2 Sent Done Done false
19480 6082 Acked Read 4 Sent Done Done false
19481 6083 Acked Read 1 Sent Done Done false
19482 6084 Acked Read 1 Sent Done Done false
19483 6085 Acked Read 1 Sent Done Done false
19484 6086 Acked Read 1 Sent Done Done false
19485 6087 Acked Read 7 Sent Done Done false
19486 6088 Acked Read 9 Sent Done Done false
19487 6089 Acked Read 2 Sent Done Done false
19488 6090 Acked Read 4 Sent Done Done false
19489 6091 Acked Write 3 Sent Done Done false
19490 6092 Acked Write 6 Sent Done Done false
19491 6093 Acked Read 6 Sent Done Done false
19492 6094 Acked Read 4 Sent Done Done false
19493 6095 Acked Read 10 Sent Done Done false
19494 6096 Acked Read 10 Sent Done Done false
19495 6097 Acked Write 5 Sent Done Done false
19496 6098 Acked Read 9 Sent Done Done false
19497 6099 Acked Read 3 Sent Done Done false
19498 6100 Acked Write 6 Sent Done Done false
19499 6101 Acked Flush 0 Sent Done Done false
19500 6102 Acked Read 5 Sent Done Done false
19501 6103 Acked Flush 0 Sent Done Done false
19502 6104 Acked Write 10 Sent Done Done false
19503 6105 Acked Read 7 Sent Done Done false
19504 6106 Acked Read 4 Sent Done Done false
19505 6107 Acked Read 5 Sent Done Done false
19506 6108 Acked Write 5 Sent Done Done false
19507 6109 Acked Write 8 Sent Done Done false
19508 6110 Acked Write 6 Sent Done Done false
19509 6111 Acked Write 10 Sent Done Done false
19510 6112 Acked Write 4 Sent Done Done false
19511 6113 Acked Read 7 Sent Done Done false
19512 6114 Acked Read 4 Sent Done Done false
19513 6115 Acked Read 8 Sent Done Done false
19514 6116 Acked Read 10 Sent Done Done false
19515 6117 Acked Read 3 Sent Done Done false
19516 6118 Acked Flush 0 Sent Done Done false
19517 6119 Acked Write 3 Sent Done Done false
19518 6120 Acked Read 5 Sent Done Done false
19519 6121 Acked Read 3 Sent Done Done false
19520 6122 Acked Write 10 Sent Done Done false
19521 6123 Acked Write 4 Sent Done Done false
19522 6124 Acked Flush 0 Sent Done Done false
19523 6125 Acked Read 3 Sent Done Done false
19524 6126 Acked Read 8 Sent Done Done false
19525 6127 Acked Read 3 Sent Done Done false
19526 6128 Acked Read 8 Sent Done Done false
19527 6129 Acked Read 4 Sent Done Done false
19528 6130 Acked Read 8 Sent Done Done false
19529 6131 Acked Read 8 Sent Done Done false
19530 STATES DS:0 DS:1 DS:2 TOTAL
19531 Sent 301 0 0 301
19532 Done 0 301 301 602
19533 Skipped 0 0 0 0
19534 Error 0 0 0 0
19535 Last Flush: 5830 6124 6124
19536 Downstairs last five completed: 5830 5829 5828 5827 5826
19537 Upstairs last five completed: 6131 6130 6129 6128 6127
19538 CLIENT: Up:0 ds:301 act:2
19539 [8810] got start action
19540 [8810] I am going to restart
19541 Starting downstairs at port 8810
19542 Make output file at "/tmp/dsc/downstairs-8810.txt"
19543 [8810][0] reports Starting
19544 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
19545 [8810][0] reports Running
19546 ----------------------------------------------------------------
19547 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19548 States: Offline Active Active
19549 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19550 5831 Acked Flush 0 Sent Done Done false
19551 5832 Acked Write 10 Sent Done Done false
19552 5833 Acked Read 6 Sent Done Done false
19553 5834 Acked Write 9 Sent Done Done false
19554 5835 Acked Write 6 Sent Done Done false
19555 5836 Acked Flush 0 Sent Done Done false
19556 5837 Acked Flush 0 Sent Done Done false
19557 5838 Acked Read 8 Sent Done Done false
19558 5839 Acked Flush 0 Sent Done Done false
19559 5840 Acked Write 2 Sent Done Done false
19560 5841 Acked Flush 0 Sent Done Done false
19561 5842 Acked Write 7 Sent Done Done false
19562 5843 Acked Write 6 Sent Done Done false
19563 5844 Acked Read 5 Sent Done Done false
19564 5845 Acked Write 5 Sent Done Done false
19565 5846 Acked Read 3 Sent Done Done false
19566 5847 Acked Read 9 Sent Done Done false
19567 5848 Acked Write 8 Sent Done Done false
19568 5849 Acked Flush 0 Sent Done Done false
19569 5850 Acked Write 5 Sent Done Done false
19570 5851 Acked Write 6 Sent Done Done false
19571 5852 Acked Write 6 Sent Done Done false
19572 5853 Acked Read 3 Sent Done Done false
19573 5854 Acked Write 6 Sent Done Done false
19574 5855 Acked Read 6 Sent Done Done false
19575 5856 Acked Read 5 Sent Done Done false
19576 5857 Acked Write 2 Sent Done Done false
19577 5858 Acked Read 1 Sent Done Done false
19578 5859 Acked Read 3 Sent Done Done false
19579 5860 Acked Write 2 Sent Done Done false
19580 5861 Acked Write 8 Sent Done Done false
19581 5862 Acked Flush 0 Sent Done Done false
19582 5863 Acked Write 4 Sent Done Done false
19583 5864 Acked Write 7 Sent Done Done false
19584 5865 Acked Read 3 Sent Done Done false
19585 5866 Acked Write 1 Sent Done Done false
19586 5867 Acked Read 4 Sent Done Done false
19587 5868 Acked Write 5 Sent Done Done false
19588 5869 Acked Write 8 Sent Done Done false
19589 5870 Acked Flush 0 Sent Done Done false
19590 5871 Acked Read 4 Sent Done Done false
19591 5872 Acked Read 7 Sent Done Done false
19592 5873 Acked Flush 0 Sent Done Done false
19593 5874 Acked Read 6 Sent Done Done false
19594 5875 Acked Read 5 Sent Done Done false
19595 5876 Acked Read 10 Sent Done Done false
19596 5877 Acked Write 9 Sent Done Done false
19597 5878 Acked Read 3 Sent Done Done false
19598 5879 Acked Write 3 Sent Done Done false
19599 5880 Acked Read 6 Sent Done Done false
19600 5881 Acked Read 7 Sent Done Done false
19601 5882 Acked Flush 0 Sent Done Done false
19602 5883 Acked Read 2 Sent Done Done false
19603 5884 Acked Write 6 Sent Done Done false
19604 5885 Acked Flush 0 Sent Done Done false
19605 5886 Acked Flush 0 Sent Done Done false
19606 5887 Acked Read 4 Sent Done Done false
19607 5888 Acked Read 10 Sent Done Done false
19608 5889 Acked Write 10 Sent Done Done false
19609 5890 Acked Read 10 Sent Done Done false
19610 5891 Acked Flush 0 Sent Done Done false
19611 5892 Acked Write 1 Sent Done Done false
19612 5893 Acked Write 8 Sent Done Done false
19613 5894 Acked Read 9 Sent Done Done false
19614 5895 Acked Write 2 Sent Done Done false
19615 5896 Acked Write 1 Sent Done Done false
19616 5897 Acked Read 3 Sent Done Done false
19617 5898 Acked Write 9 Sent Done Done false
19618 5899 Acked Write 2 Sent Done Done false
19619 5900 Acked Write 7 Sent Done Done false
19620 5901 Acked Read 10 Sent Done Done false
19621 5902 Acked Read 3 Sent Done Done false
19622 5903 Acked Read 2 Sent Done Done false
19623 5904 Acked Write 8 Sent Done Done false
19624 5905 Acked Read 5 Sent Done Done false
19625 5906 Acked Write 10 Sent Done Done false
19626 5907 Acked Write 8 Sent Done Done false
19627 5908 Acked Read 2 Sent Done Done false
19628 5909 Acked Read 6 Sent Done Done false
19629 5910 Acked Read 4 Sent Done Done false
19630 5911 Acked Write 3 Sent Done Done false
19631 5912 Acked Read 4 Sent Done Done false
19632 5913 Acked Flush 0 Sent Done Done false
19633 5914 Acked Read 10 Sent Done Done false
19634 5915 Acked Read 7 Sent Done Done false
19635 5916 Acked Write 8 Sent Done Done false
19636 5917 Acked Flush 0 Sent Done Done false
19637 5918 Acked Read 5 Sent Done Done false
19638 5919 Acked Read 10 Sent Done Done false
19639 5920 Acked Write 3 Sent Done Done false
19640 5921 Acked Write 7 Sent Done Done false
19641 5922 Acked Write 6 Sent Done Done false
19642 5923 Acked Write 6 Sent Done Done false
19643 5924 Acked Read 7 Sent Done Done false
19644 5925 Acked Read 1 Sent Done Done false
19645 5926 Acked Write 9 Sent Done Done false
19646 5927 Acked Read 9 Sent Done Done false
19647 5928 Acked Flush 0 Sent Done Done false
19648 5929 Acked Write 9 Sent Done Done false
19649 5930 Acked Flush 0 Sent Done Done false
19650 5931 Acked Read 10 Sent Done Done false
19651 5932 Acked Read 6 Sent Done Done false
19652 5933 Acked Write 8 Sent Done Done false
19653 5934 Acked Write 10 Sent Done Done false
19654 5935 Acked Write 7 Sent Done Done false
19655 5936 Acked Read 10 Sent Done Done false
19656 5937 Acked Read 2 Sent Done Done false
19657 5938 Acked Read 6 Sent Done Done false
19658 5939 Acked Read 6 Sent Done Done false
19659 5940 Acked Write 5 Sent Done Done false
19660 5941 Acked Read 6 Sent Done Done false
19661 5942 Acked Write 7 Sent Done Done false
19662 5943 Acked Read 2 Sent Done Done false
19663 5944 Acked Read 2 Sent Done Done false
19664 5945 Acked Write 5 Sent Done Done false
19665 5946 Acked Read 8 Sent Done Done false
19666 5947 Acked Read 5 Sent Done Done false
19667 5948 Acked Read 9 Sent Done Done false
19668 5949 Acked Write 2 Sent Done Done false
19669 5950 Acked Read 2 Sent Done Done false
19670 5951 Acked Read 9 Sent Done Done false
19671 5952 Acked Write 10 Sent Done Done false
19672 5953 Acked Write 3 Sent Done Done false
19673 5954 Acked Write 7 Sent Done Done false
19674 5955 Acked Read 8 Sent Done Done false
19675 5956 Acked Write 4 Sent Done Done false
19676 5957 Acked Read 5 Sent Done Done false
19677 5958 Acked Read 3 Sent Done Done false
19678 5959 Acked Write 8 Sent Done Done false
19679 5960 Acked Read 5 Sent Done Done false
19680 5961 Acked Write 2 Sent Done Done false
19681 5962 Acked Flush 0 Sent Done Done false
19682 5963 Acked Write 5 Sent Done Done false
19683 5964 Acked Read 9 Sent Done Done false
19684 5965 Acked Flush 0 Sent Done Done false
19685 5966 Acked Write 10 Sent Done Done false
19686 5967 Acked Read 2 Sent Done Done false
19687 5968 Acked Flush 0 Sent Done Done false
19688 5969 Acked Read 5 Sent Done Done false
19689 5970 Acked Write 8 Sent Done Done false
19690 5971 Acked Write 3 Sent Done Done false
19691 5972 Acked Read 10 Sent Done Done false
19692 5973 Acked Flush 0 Sent Done Done false
19693 5974 Acked Read 2 Sent Done Done false
19694 5975 Acked Write 3 Sent Done Done false
19695 5976 Acked Read 1 Sent Done Done false
19696 5977 Acked Write 1 Sent Done Done false
19697 5978 Acked Write 4 Sent Done Done false
19698 5979 Acked Write 4 Sent Done Done false
19699 5980 Acked Read 6 Sent Done Done false
19700 5981 Acked Write 8 Sent Done Done false
19701 5982 Acked Write 2 Sent Done Done false
19702 5983 Acked Read 7 Sent Done Done false
19703 5984 Acked Write 4 Sent Done Done false
19704 5985 Acked Read 3 Sent Done Done false
19705 5986 Acked Write 7 Sent Done Done false
19706 5987 Acked Read 6 Sent Done Done false
19707 5988 Acked Write 5 Sent Done Done false
19708 5989 Acked Read 3 Sent Done Done false
19709 5990 Acked Write 9 Sent Done Done false
19710 5991 Acked Write 4 Sent Done Done false
19711 5992 Acked Write 1 Sent Done Done false
19712 5993 Acked Read 2 Sent Done Done false
19713 5994 Acked Write 5 Sent Done Done false
19714 5995 Acked Read 5 Sent Done Done false
19715 5996 Acked Write 10 Sent Done Done false
19716 5997 Acked Read 9 Sent Done Done false
19717 5998 Acked Write 3 Sent Done Done false
19718 5999 Acked Read 6 Sent Done Done false
19719 6000 Acked Read 10 Sent Done Done false
19720 6001 Acked Write 7 Sent Done Done false
19721 6002 Acked Read 10 Sent Done Done false
19722 6003 Acked Read 7 Sent Done Done false
19723 6004 Acked Write 6 Sent Done Done false
19724 6005 Acked Read 1 Sent Done Done false
19725 6006 Acked Write 1 Sent Done Done false
19726 6007 Acked Read 1 Sent Done Done false
19727 6008 Acked Write 8 Sent Done Done false
19728 6009 Acked Read 4 Sent Done Done false
19729 6010 Acked Read 6 Sent Done Done false
19730 6011 Acked Flush 0 Sent Done Done false
19731 6012 Acked Read 1 Sent Done Done false
19732 6013 Acked Read 9 Sent Done Done false
19733 6014 Acked Flush 0 Sent Done Done false
19734 6015 Acked Read 4 Sent Done Done false
19735 6016 Acked Flush 0 Sent Done Done false
19736 6017 Acked Read 4 Sent Done Done false
19737 6018 Acked Read 10 Sent Done Done false
19738 6019 Acked Read 8 Sent Done Done false
19739 6020 Acked Flush 0 Sent Done Done false
19740 6021 Acked Read 3 Sent Done Done false
19741 6022 Acked Read 8 Sent Done Done false
19742 6023 Acked Read 4 Sent Done Done false
19743 6024 Acked Read 5 Sent Done Done false
19744 6025 Acked Read 8 Sent Done Done false
19745 6026 Acked Read 8 Sent Done Done false
19746 6027 Acked Write 3 Sent Done Done false
19747 6028 Acked Read 10 Sent Done Done false
19748 6029 Acked Flush 0 Sent Done Done false
19749 6030 Acked Write 10 Sent Done Done false
19750 6031 Acked Flush 0 Sent Done Done false
19751 6032 Acked Read 8 Sent Done Done false
19752 6033 Acked Read 5 Sent Done Done false
19753 6034 Acked Write 4 Sent Done Done false
19754 6035 Acked Read 5 Sent Done Done false
19755 6036 Acked Write 6 Sent Done Done false
19756 6037 Acked Write 9 Sent Done Done false
19757 6038 Acked Write 9 Sent Done Done false
19758 6039 Acked Write 3 Sent Done Done false
19759 6040 Acked Read 7 Sent Done Done false
19760 6041 Acked Write 8 Sent Done Done false
19761 6042 Acked Read 5 Sent Done Done false
19762 6043 Acked Read 4 Sent Done Done false
19763 6044 Acked Read 7 Sent Done Done false
19764 6045 Acked Write 9 Sent Done Done false
19765 6046 Acked Read 3 Sent Done Done false
19766 6047 Acked Read 1 Sent Done Done false
19767 6048 Acked Read 6 Sent Done Done false
19768 6049 Acked Flush 0 Sent Done Done false
19769 6050 Acked Read 4 Sent Done Done false
19770 6051 Acked Read 4 Sent Done Done false
19771 6052 Acked Read 7 Sent Done Done false
19772 6053 Acked Read 8 Sent Done Done false
19773 6054 Acked Read 9 Sent Done Done false
19774 6055 Acked Read 10 Sent Done Done false
19775 6056 Acked Write 4 Sent Done Done false
19776 6057 Acked Write 3 Sent Done Done false
19777 6058 Acked Write 10 Sent Done Done false
19778 6059 Acked Write 5 Sent Done Done false
19779 6060 Acked Write 10 Sent Done Done false
19780 6061 Acked Write 1 Sent Done Done false
19781 6062 Acked Read 7 Sent Done Done false
19782 6063 Acked Read 5 Sent Done Done false
19783 6064 Acked Write 6 Sent Done Done false
19784 6065 Acked Flush 0 Sent Done Done false
19785 6066 Acked Write 8 Sent Done Done false
19786 6067 Acked Read 6 Sent Done Done false
19787 6068 Acked Read 6 Sent Done Done false
19788 6069 Acked Write 5 Sent Done Done false
19789 6070 Acked Write 7 Sent Done Done false
19790 6071 Acked Read 5 Sent Done Done false
19791 6072 Acked Write 4 Sent Done Done false
19792 6073 Acked Read 10 Sent Done Done false
19793 6074 Acked Read 3 Sent Done Done false
19794 6075 Acked Read 5 Sent Done Done false
19795 6076 Acked Read 3 Sent Done Done false
19796 6077 Acked Read 2 Sent Done Done false
19797 6078 Acked Read 3 Sent Done Done false
19798 6079 Acked Write 3 Sent Done Done false
19799 6080 Acked Write 3 Sent Done Done false
19800 6081 Acked Write 2 Sent Done Done false
19801 6082 Acked Read 4 Sent Done Done false
19802 6083 Acked Read 1 Sent Done Done false
19803 6084 Acked Read 1 Sent Done Done false
19804 6085 Acked Read 1 Sent Done Done false
19805 6086 Acked Read 1 Sent Done Done false
19806 6087 Acked Read 7 Sent Done Done false
19807 6088 Acked Read 9 Sent Done Done false
19808 6089 Acked Read 2 Sent Done Done false
19809 6090 Acked Read 4 Sent Done Done false
19810 6091 Acked Write 3 Sent Done Done false
19811 6092 Acked Write 6 Sent Done Done false
19812 6093 Acked Read 6 Sent Done Done false
19813 6094 Acked Read 4 Sent Done Done false
19814 6095 Acked Read 10 Sent Done Done false
19815 6096 Acked Read 10 Sent Done Done false
19816 6097 Acked Write 5 Sent Done Done false
19817 6098 Acked Read 9 Sent Done Done false
19818 6099 Acked Read 3 Sent Done Done false
19819 6100 Acked Write 6 Sent Done Done false
19820 6101 Acked Flush 0 Sent Done Done false
19821 6102 Acked Read 5 Sent Done Done false
19822 6103 Acked Flush 0 Sent Done Done false
19823 6104 Acked Write 10 Sent Done Done false
19824 6105 Acked Read 7 Sent Done Done false
19825 6106 Acked Read 4 Sent Done Done false
19826 6107 Acked Read 5 Sent Done Done false
19827 6108 Acked Write 5 Sent Done Done false
19828 6109 Acked Write 8 Sent Done Done false
19829 6110 Acked Write 6 Sent Done Done false
19830 6111 Acked Write 10 Sent Done Done false
19831 6112 Acked Write 4 Sent Done Done false
19832 6113 Acked Read 7 Sent Done Done false
19833 6114 Acked Read 4 Sent Done Done false
19834 6115 Acked Read 8 Sent Done Done false
19835 6116 Acked Read 10 Sent Done Done false
19836 6117 Acked Read 3 Sent Done Done false
19837 6118 Acked Flush 0 Sent Done Done false
19838 6119 Acked Write 3 Sent Done Done false
19839 6120 Acked Read 5 Sent Done Done false
19840 6121 Acked Read 3 Sent Done Done false
19841 6122 Acked Write 10 Sent Done Done false
19842 6123 Acked Write 4 Sent Done Done false
19843 6124 Acked Flush 0 Sent Done Done false
19844 6125 Acked Read 3 Sent Done Done false
19845 6126 Acked Read 8 Sent Done Done false
19846 6127 Acked Read 3 Sent Done Done false
19847 6128 Acked Read 8 Sent Done Done false
19848 6129 Acked Read 4 Sent Done Done false
19849 6130 Acked Read 8 Sent Done Done false
19850 6131 Acked Read 8 Sent Done Done false
19851 6132 Acked Flush 0 Sent Done Done false
19852 STATES DS:0 DS:1 DS:2 TOTAL
19853 Sent 302 0 0 302
19854 Done 0 302 302 604
19855 Skipped 0 0 0 0
19856 Error 0 0 0 0
19857 Last Flush: 5830 6132 6132
19858 Downstairs last five completed: 5830 5829 5828 5827 5826
19859 Upstairs last five completed: 6132 6131 6130 6129 6128
19860 CLIENT: Up:0 ds:302 act:2
198612025-01-15T05:23:08.875ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198622025-01-15T05:23:08.875ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52733) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198632025-01-15T05:23:08.876ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198642025-01-15T05:23:08.877ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198652025-01-15T05:23:08.877ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198662025-01-15T05:23:08.877ZINFOcrucible: send last flush ID to this DS: 5830 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198672025-01-15T05:23:08.877ZINFOcrucible: Replied this last flush ID: 5830 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198682025-01-15T05:23:08.877ZINFOcrucible: [0] client re-new 302 jobs since flush 5830 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
198692025-01-15T05:23:08.877ZINFOcrucible: [0] Replayed 302 jobs since flush: 5830 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
19870 ----------------------------------------------------------------
19871 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19872 Downstairs last five completed: 6132 6131 6130 6129 6128
19873 Upstairs last five completed: 6132 6131 6130 6129 6128
19874 CLIENT: Up:0 ds:0 act:3
19875 Replay: All jobs finished
19876 Main task has work to do, go find it
19877 got dsc Stop(1)
19878 stop 1
19879 Jan 15 05:23:11.122[8820] Got stop action so:false kr:false
19880 INFO request completed, latency_us: 351, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 82afe957-fa74-42ff-aa1d-0b8b2e37b39c, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
198812025-01-15T05:23:11.123ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
19882 Jan 15 05:23:11.123 INFO request completed, latency_us: 107, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:23:11.123704848Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
19883 response_code: 200, uri{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:23:11.123741795Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
19884 : /state/cid/1, method: GET, req_id: 3545f21d-8c80-4e48-b360-02a51b0cc360, remote_addr: 127.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:23:11.123762473Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
19885 0.1:42402, local_addr: 127.0.0.1:9998
19886 [8820] Exited with: ExitStatus(unix_wait_status(9))
19887 [8820][1] reports Exit
19888 Jan 15 05:23:15.124 INFO request completed, latency_us: 183, response_code: 200, uri: /state/cid/1, method: GET, req_id: 838760c7-e4b5-4f25-acf8-f38bc46f4c01, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
19889 001/300 Flush
19890 002/300 Write block 3841 len 16384 data: 6 6 5 8
19891 003/300 Write block 3370 len 36864 data: 7 4 3 4 5 5 4 4 4
19892 004/300 Flush
19893 005/300 Write block 674 len 28672 data: 4 3 3 5 7 6 5
19894 006/300 Read block 3022 len 28672
19895 007/300 Write block 3072 len 12288 data: 5 4 4
19896 008/300 Write block 1763 len 24576 data: 7 6 6 5 4 4
19897 009/300 Write block 256 len 28672 data: 8 8 8 5 5 3 3
19898 010/300 Read block 1293 len 40960
19899 011/300 Read block 4213 len 28672
19900 012/300 Write block 3087 len 16384 data: 3 4 5 6
19901 013/300 Read block 1333 len 32768
19902 014/300 Flush
19903 015/300 Flush
19904 016/300 Read block 641 len 40960
19905 017/300 Read block 3547 len 32768
19906 018/300 Read block 4103 len 36864
19907 019/300 Read block 725 len 12288
19908 020/300 Read block 3133 len 32768
19909 021/300 Read block 1954 len 8192
19910 022/300 Write block 1216 len 8192 data: 5 6
19911 023/300 Write block 904 len 4096 data: 4
19912 024/300 Write block 4977 len 32768 data: 4 5 7 7 5 4 3 3
19913 025/300 Write block 2579 len 20480 data: 2 2 2 2 3
19914 026/300 Write block 3196 len 8192 data: 3 3
19915 027/300 Read block 2847 len 24576
19916 028/300 Read block 1209 len 40960
19917 029/300 Read block 604 len 36864
19918 030/300 Read block 3825 len 16384
19919 031/300 Write block 2989 len 28672 data: 3 3 2 2 3 4 3
19920 032/300 Write block 3796 len 20480 data: 6 6 6 4 4
19921 033/300 Read block 3555 len 24576
19922 034/300 Read block 1978 len 8192
19923 035/300 Read block 4733 len 20480
19924 036/300 Read block 4436 len 24576
19925 037/300 Write block 1917 len 28672 data: 5 4 4 5 5 5 5
19926 038/300 Write block 4330 len 8192 data: 6 7
19927 039/300 Read block 1105 len 12288
19928 040/300 Write block 4549 len 28672 data: 5 6 6 8 7 6 5
19929 041/300 Read block 4057 len 12288
19930 042/300 Read block 3537 len 32768
19931 043/300 Read block 4611 len 8192
19932 044/300 Write block 2695 len 28672 data: 2 2 3 3 2 3 3
19933 045/300 Write block 2355 len 28672 data: 5 5 6 4 4 3 3
19934 046/300 Flush
19935 047/300 Write block 4883 len 12288 data: 3 6 6
19936 048/300 Write block 2635 len 32768 data: 4 5 5 4 4 4 4 5
19937 049/300 Read block 3200 len 16384
19938 050/300 Write block 2010 len 20480 data: 4 5 4 4 5
19939 051/300 Read block 2606 len 4096
19940 052/300 Write block 4741 len 32768 data: 5 6 8 9 7 5 4 2
19941 053/300 Read block 2565 len 40960
19942 054/300 Read block 2802 len 32768
19943 055/300 Read block 1192 len 24576
19944 056/300 Read block 302 len 40960
19945 057/300 Write block 4542 len 16384 data: 4 4 3 4
19946 058/300 Write block 2323 len 20480 data: 2 2 3 3 3
19947 059/300 Read block 4873 len 12288
19948 060/300 Write block 4824 len 12288 data: 5 3 2
19949 061/300 Read block 4356 len 36864
19950 062/300 Read block 4750 len 4096
19951 063/300 Write block 4137 len 40960 data: 4 4 3 4 4 5 7 7 6 6
19952 064/300 Write block 2432 len 12288 data: 5 5 4
19953 065/300 Read block 2020 len 32768
19954 066/300 Write block 854 len 8192 data: 4 4
19955 067/300 Read block 4944 len 4096
19956 068/300 Read block 4726 len 24576
19957 069/300 Write block 950 len 24576 data: 3 4 3 2 2 2
19958 070/300 Write block 4313 len 32768 data: 2 3 3 4 4 4 5 6
19959 071/300 Flush
19960 072/300 Read block 755 len 40960
19961 073/300 Flush
19962 074/300 Read block 4300 len 36864
19963 075/300 Write block 4491 len 28672 data: 4 2 4 4 6 6 6
19964 076/300 Write block 829 len 16384 data: 4 3 2 2
19965 077/300 Write block 1076 len 20480 data: 4 5 6 4 5
19966 078/300 Read block 2109 len 12288
19967 079/300 Flush
19968 080/300 Read block 2156 len 8192
19969 081/300 Flush
19970 082/300 Write block 1892 len 36864 data: 5 6 7 6 4 5 5 5 6
19971 083/300 Read block 3246 len 28672
19972 084/300 Write block 2131 len 16384 data: 2 2 2 2
19973 085/300 Write block 3636 len 8192 data: 3 4
19974 086/300 Write block 3990 len 16384 data: 3 4 4 3
19975 087/300 Read block 1766 len 32768
19976 088/300 Write block 4135 len 12288 data: 4 3 5
19977 089/300 Read block 699 len 20480
19978 090/300 Read block 3856 len 8192
19979 091/300 Read block 381 len 32768
19980 092/300 Write block 1377 len 20480 data: 2 3 3 4 4
19981 093/300 Read block 3564 len 8192
19982 094/300 Read block 888 len 36864
19983 095/300 Write block 1099 len 28672 data: 4 3 2 3 3 3 3
19984 096/300 Write block 4872 len 8192 data: 7 7
19985 097/300 Read block 3531 len 40960
19986 098/300 Write block 1450 len 28672 data: 3 3 3 4 5 5 6
19987 099/300 Flush
19988 100/300 Read block 889 len 32768
19989 101/300 Read block 678 len 4096
19990 102/300 Flush
19991 103/300 Read block 59 len 32768
19992 104/300 Read block 1969 len 36864
19993 105/300 Read block 4095 len 8192
19994 106/300 Write block 1656 len 8192 data: 3 3
19995 107/300 Read block 2936 len 28672
19996 108/300 Flush
19997 109/300 Write block 4912 len 8192 data: 6 5
19998 110/300 Write block 4743 len 24576 data: 9 10 8 6 5 3
19999 111/300 Read block 2741 len 12288
20000 112/300 Read block 3025 len 40960
20001 113/300 Write block 111 len 32768 data: 4 4 3 3 5 4 4 4
20002 114/300 Flush
20003 115/300 Write block 3965 len 20480 data: 3 2 3 3 4
20004 116/300 Write block 4187 len 40960 data: 3 4 5 5 5 6 7 7 5 5
20005 117/300 Read block 4766 len 36864
20006 118/300 Write block 4295 len 36864 data: 3 3 3 3 3 3 3 4 4
20007 119/300 Write block 4862 len 36864 data: 4 4 4 5 4 4 4 7 7
20008 120/300 Read block 3865 len 32768
20009 121/300 Write block 639 len 8192 data: 8 6
20010 122/300 Read block 4829 len 16384
20011 123/300 Read block 3306 len 28672
20012 124/300 Write block 579 len 40960 data: 5 4 4 5 4 4 4 4 4 4
20013 125/300 Read block 4831 len 20480
20014 126/300 Read block 3175 len 4096
20015 127/300 Write block 792 len 16384 data: 4 4 4 4
20016 128/300 Read block 4007 len 32768
20017 129/300 Flush
20018 130/300 Read block 2500 len 4096
20019 131/300 Read block 4541 len 28672
20020 132/300 Read block 2147 len 16384
20021 133/300 Write block 2369 len 20480 data: 8 8 6 6 5
20022 134/300 Write block 3269 len 8192 data: 5 4
20023 135/300 Read block 567 len 12288
20024 136/300 Write block 4857 len 4096 data: 3
20025 137/300 Write block 2006 len 4096 data: 3
20026 138/300 Read block 284 len 36864
20027 139/300 Read block 4314 len 36864
20028 140/300 Read block 2850 len 20480
20029 141/300 Flush
20030 142/300 Write block 4151 len 24576 data: 4 5 5 6 7 8
20031 143/300 Read block 4231 len 32768
20032 144/300 Read block 3986 len 40960
20033 145/300 Read block 3986 len 20480
20034 146/300 Write block 3403 len 36864 data: 2 2 2 2 2 2 2 2 4
20035 147/300 Write block 2392 len 24576 data: 3 3 5 4 4 4
20036 148/300 Write block 3157 len 32768 data: 8 8 6 8 5 4 5 5
20037 149/300 Flush
20038 150/300 Flush
20039 151/300 Flush
20040 152/300 Read block 3402 len 20480
20041 153/300 Read block 2621 len 36864
20042 154/300 Write block 3031 len 36864 data: 6 6 6 6 8 9 9 8 8
20043 155/300 Write block 2226 len 12288 data: 5 5 5
20044 156/300 Read block 4612 len 40960
20045 157/300 Read block 2774 len 8192
20046 158/300 Read block 3627 len 12288
20047 159/300 Read block 1875 len 28672
20048 160/300 Write block 3931 len 4096 data: 3
20049 161/300 Flush
20050 162/300 Read block 1685 len 40960
20051 163/300 Read block 2394 len 4096
20052 164/300 Write block 2431 len 28672 data: 5 6 6 5 2 2 2
20053 165/300 Write block 129 len 24576 data: 4 5 5 4 5 5
20054 166/300 Read block 4431 len 12288
20055 167/300 Read block 4324 len 32768
20056 168/300 Write block 70 len 8192 data: 3 3
20057 169/300 Write block 4829 len 36864 data: 3 2 2 3 3 5 5 5 5
20058 170/300 Read block 1671 len 16384
20059 171/300 Read block 2345 len 4096
20060 172/300 Read block 938 len 40960
20061 173/300 Write block 4012 len 28672 data: 2 3 3 3 5 5 5
20062 174/300 Flush
20063 175/300 Write block 4502 len 12288 data: 7 6 5
20064 176/300 Read block 594 len 8192
20065 177/300 Read block 4186 len 16384
20066 178/300 Flush
20067 179/300 Flush
20068 180/300 Write block 3493 len 32768 data: 4 4 4 4 6 8 7 6
20069 181/300 Read block 2502 len 20480
20070 182/300 Read block 142 len 12288
20071 183/300 Flush
20072 184/300 Read block 3642 len 4096
20073 185/300 Write block 660 len 12288 data: 4 4 5
20074 186/300 Read block 1231 len 20480
20075 187/300 Read block 2499 len 8192
20076 188/300 Read block 2067 len 36864
20077 189/300 Read block 4714 len 36864
20078 190/300 Write block 1531 len 24576 data: 6 6 5 4 4 5
20079 191/300 Write block 3606 len 28672 data: 5 5 4 3 4 4 3
20080 192/300 Read block 1410 len 16384
20081 193/300 Flush
20082 194/300 Write block 1232 len 8192 data: 3 2
20083 195/300 Flush
20084 196/300 Write block 4223 len 4096 data: 5
20085 197/300 Write block 390 len 12288 data: 2 2 3
20086 198/300 Write block 1034 len 40960 data: 3 4 5 4 4 5 5 3 3 4
20087 199/300 Write block 676 len 4096 data: 4
20088 200/300 Read block 4142 len 16384
20089 201/300 Read block 4368 len 20480
20090 202/300 Flush
20091 203/300 Read block 3639 len 40960
20092 204/300 Write block 3157 len 16384 data: 9 9 7 9
20093 205/300 Flush
20094 206/300 Read block 4374 len 24576
20095 207/300 Flush
20096 208/300 Flush
20097 209/300 Read block 1634 len 36864
20098 210/300 Write block 926 len 12288 data: 3 3 3
20099 211/300 Write block 1432 len 32768 data: 6 7 7 6 6 5 5 4
20100 212/300 Write block 262 len 16384 data: 4 5 6 6
20101 213/300 Read block 1786 len 40960
20102 214/300 Write block 4327 len 4096 data: 2
20103 215/300 Read block 208 len 32768
20104 216/300 Write block 3626 len 40960 data: 3 3 3 3 4 4 3 4 3 3
20105 217/300 Write block 795 len 8192 data: 5 4
20106 218/300 Write block 338 len 40960 data: 3 4 4 4 4 5 4 4 2 2
20107 219/300 Write block 4932 len 20480 data: 4 4 4 4 4
20108 220/300 Write block 2268 len 28672 data: 4 4 4 4 3 4 4
20109 221/300 Read block 1589 len 8192
20110 222/300 Read block 602 len 16384
20111 223/300 Read block 1487 len 20480
20112 224/300 Write block 926 len 4096 data: 4
20113 225/300 Read block 514 len 4096
20114 226/300 Write block 4547 len 20480 data: 5 4 6 7 7
20115 227/300 Read block 3298 len 4096
20116 228/300 Write block 2193 len 40960 data: 3 4 5 4 5 5 7 7 5 5
20117 229/300 Flush
20118 230/300 Write block 2137 len 36864 data: 3 4 4 3 3 3 5 5 5
20119 231/300 Read block 989 len 4096
20120 232/300 Read block 653 len 40960
20121 233/300 Flush
20122 234/300 Flush
20123 235/300 Write block 4485 len 8192 data: 6 5
20124 236/300 Flush
20125 237/300 Read block 1572 len 32768
20126 238/300 Read block 4806 len 40960
20127 239/300 Read block 2580 len 20480
20128 240/300 Read block 4314 len 16384
20129 241/300 Write block 4535 len 4096 data: 2
20130 242/300 Read block 48 len 20480
20131 243/300 Read block 2096 len 28672
20132 244/300 Read block 1284 len 24576
20133 245/300 Write block 3126 len 8192 data: 5 6
20134 246/300 Read block 3496 len 24576
20135 247/300 Flush
20136 248/300 Read block 1435 len 16384
20137 249/300 Read block 2949 len 24576
20138 250/300 Read block 4636 len 28672
20139 251/300 Read block 4806 len 16384
20140 252/300 Write block 4019 len 28672 data: 6 7 7 7 7 5 3
20141 253/300 Read block 1674 len 36864
20142 254/300 Read block 3378 len 32768
20143 255/300 Read block 3470 len 8192
20144 256/300 Flush
20145 257/300 Write block 2071 len 12288 data: 4 4 4
20146 258/300 Write block 4089 len 36864 data: 2 2 2 2 2 2 2 2 3
20147 259/300 Flush
20148 260/300 Write block 4981 len 24576 data: 6 5 4 4 3 3
20149 261/300 Write block 2538 len 20480 data: 4 4 6 5 4
20150 262/300 Write block 4091 len 32768 data: 3 3 3 3 3 3 4 3
20151 263/300 Read block 4985 len 20480
20152 264/300 Read block 4943 len 4096
20153 265/300 Write block 3206 len 4096 data: 4
20154 266/300 Read block 1546 len 16384
20155 267/300 Read block 2677 len 36864
20156 268/300 Read block 2227 len 36864
20157 269/300 Write block 2617 len 28672 data: 2 2 2 3 3 3 4
20158 270/300 Read block 4995 len 20480
20159 271/300 Flush
20160 272/300 Read block 4573 len 8192
20161 273/300 Write block 1304 len 28672 data: 6 5 6 5 3 3 3
20162 274/300 Write block 3406 len 24576 data: 3 3 3 3 3 5
20163 275/300 Flush
20164 276/300 Write block 3612 len 20480 data: 4 2 2 4 4
20165 277/300 Write block 4427 len 4096 data: 8
20166 278/300 Write block 2706 len 28672 data: 3 2 2 2 2 4 5
20167 279/300 Write block 4554 len 20480 data: 7 6 5 5 4
20168 280/300 Write block 4891 len 40960 data: 4 5 5 5 5 6 7 8 6 7
20169 281/300 Write block 344 len 32768 data: 5 5 3 3 2 2 2 3
20170 282/300 Read block 1438 len 20480
20171 283/300 Write block 4563 len 12288 data: 4 3 3
20172 284/300 Flush
20173 285/300 Write block 4236 len 36864 data: 3 3 3 4 6 9 8 7 7
20174 286/300 Read block 3496 len 36864
20175 287/300 Write block 173 len 8192 data: 4 5
20176 288/300 Write block 3761 len 36864 data: 5 4 5 4 5 6 5 5 3
20177 289/300 Flush
20178 290/300 Write block 377 len 24576 data: 6 5 7 7 7 7
20179 291/300 Read block 4786 len 28672
20180 292/300 Write block 1609 len 24576 data: 5 5 4 4 3 3
20181 293/300 Write block 4316 len 28672 data: 5 5 5 6 7 6 5
20182 294/300 Read block 1977 len 4096
20183 295/300 Write block 2629 len 12288 data: 6 4 3
20184 296/300 Read block 436 len 20480
20185 297/300 Read block 2067 len 12288
20186 298/300 Write block 2014 len 20480 data: 6 5 7 7 7
20187 299/300 Write block 1073 len 36864 data: 4 4 4 5 6 7 5 6 5
20188 300/300 Write block 4622 len 8192 data: 6 6
20189 301/300 Write block 820 len 32768 data: 4 4 5 5 6 5 5 4
20190 Main task has work to do, go find it
20191 got dsc Start(1)
20192 start 1
20193 Jan 15 05:23:15.351 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/1, method: POST, req_id: fe3a508a-e94a-49f7-b06b-da864e87c436, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
20194 Replay: started 1, returned:Ok(())
20195 ----------------------------------------------------------------
20196 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20197 States: Active Offline Active
20198 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20199 6133 Acked Flush 0 Done Sent Done false
20200 6134 Acked Write 4 Done Sent Done false
20201 6135 Acked Write 9 Done Sent Done false
20202 6136 Acked Flush 0 Done Sent Done false
20203 6137 Acked Write 7 Done Sent Done false
20204 6138 Acked Read 7 Done Sent Done false
20205 6139 Acked Write 3 Done Sent Done false
20206 6140 Acked Write 6 Done Sent Done false
20207 6141 Acked Write 7 Done Sent Done false
20208 6142 Acked Read 10 Done Sent Done false
20209 6143 Acked Read 7 Done Sent Done false
20210 6144 Acked Write 4 Done Sent Done false
20211 6145 Acked Read 8 Done Sent Done false
20212 6146 Acked Flush 0 Done Sent Done false
20213 6147 Acked Flush 0 Done Sent Done false
20214 6148 Acked Read 10 Done Sent Done false
20215 6149 Acked Read 8 Done Sent Done false
20216 6150 Acked Read 9 Done Sent Done false
20217 6151 Acked Read 3 Done Sent Done false
20218 6152 Acked Read 8 Done Sent Done false
20219 6153 Acked Read 2 Done Sent Done false
20220 6154 Acked Write 2 Done Sent Done false
20221 6155 Acked Write 1 Done Sent Done false
20222 6156 Acked Write 8 Done Sent Done false
20223 6157 Acked Write 5 Done Sent Done false
20224 6158 Acked Write 2 Done Sent Done false
20225 6159 Acked Read 6 Done Sent Done false
20226 6160 Acked Read 10 Done Sent Done false
20227 6161 Acked Read 9 Done Sent Done false
20228 6162 Acked Read 4 Done Sent Done false
20229 6163 Acked Write 7 Done Sent Done false
20230 6164 Acked Write 5 Done Sent Done false
20231 6165 Acked Read 6 Done Sent Done false
20232 6166 Acked Read 2 Done Sent Done false
20233 6167 Acked Read 5 Done Sent Done false
20234 6168 Acked Read 6 Done Sent Done false
20235 6169 Acked Write 7 Done Sent Done false
20236 6170 Acked Write 2 Done Sent Done false
20237 6171 Acked Read 3 Done Sent Done false
20238 6172 Acked Write 7 Done Sent Done false
20239 6173 Acked Read 3 Done Sent Done false
20240 6174 Acked Read 8 Done Sent Done false
20241 6175 Acked Read 2 Done Sent Done false
20242 6176 Acked Write 7 Done Sent Done false
20243 6177 Acked Write 7 Done Sent Done false
20244 6178 Acked Flush 0 Done Sent Done false
20245 6179 Acked Write 3 Done Sent Done false
20246 6180 Acked Write 8 Done Sent Done false
20247 6181 Acked Read 4 Done Sent Done false
20248 6182 Acked Write 5 Done Sent Done false
20249 6183 Acked Read 1 Done Sent Done false
20250 6184 Acked Write 8 Done Sent Done false
20251 6185 Acked Read 10 Done Sent Done false
20252 6186 Acked Read 8 Done Sent Done false
20253 6187 Acked Read 6 Done Sent Done false
20254 6188 Acked Read 10 Done Sent Done false
20255 6189 Acked Write 4 Done Sent Done false
20256 6190 Acked Write 5 Done Sent Done false
20257 6191 Acked Read 3 Done Sent Done false
20258 6192 Acked Write 3 Done Sent Done false
20259 6193 Acked Read 9 Done Sent Done false
20260 6194 Acked Read 1 Done Sent Done false
20261 6195 Acked Write 10 Done Sent Done false
20262 6196 Acked Write 3 Done Sent Done false
20263 6197 Acked Read 8 Done Sent Done false
20264 6198 Acked Write 2 Done Sent Done false
20265 6199 Acked Read 1 Done Sent Done false
20266 6200 Acked Read 6 Done Sent Done false
20267 6201 Acked Write 6 Done Sent Done false
20268 6202 Acked Write 8 Done Sent Done false
20269 6203 Acked Flush 0 Done Sent Done false
20270 6204 Acked Read 10 Done Sent Done false
20271 6205 Acked Flush 0 Done Sent Done false
20272 6206 Acked Read 9 Done Sent Done false
20273 6207 Acked Write 7 Done Sent Done false
20274 6208 Acked Write 4 Done Sent Done false
20275 6209 Acked Write 5 Done Sent Done false
20276 6210 Acked Read 3 Done Sent Done false
20277 6211 Acked Flush 0 Done Sent Done false
20278 6212 Acked Read 2 Done Sent Done false
20279 6213 Acked Flush 0 Done Sent Done false
20280 6214 Acked Write 9 Done Sent Done false
20281 6215 Acked Read 7 Done Sent Done false
20282 6216 Acked Write 4 Done Sent Done false
20283 6217 Acked Write 2 Done Sent Done false
20284 6218 Acked Write 4 Done Sent Done false
20285 6219 Acked Read 8 Done Sent Done false
20286 6220 Acked Write 3 Done Sent Done false
20287 6221 Acked Read 5 Done Sent Done false
20288 6222 Acked Read 2 Done Sent Done false
20289 6223 Acked Read 8 Done Sent Done false
20290 6224 Acked Write 5 Done Sent Done false
20291 6225 Acked Read 2 Done Sent Done false
20292 6226 Acked Read 9 Done Sent Done false
20293 6227 Acked Write 7 Done Sent Done false
20294 6228 Acked Write 2 Done Sent Done false
20295 6229 Acked Read 10 Done Sent Done false
20296 6230 Acked Write 7 Done Sent Done false
20297 6231 Acked Flush 0 Done Sent Done false
20298 6232 Acked Read 8 Done Sent Done false
20299 6233 Acked Read 1 Done Sent Done false
20300 6234 Acked Flush 0 Done Sent Done false
20301 6235 Acked Read 8 Done Sent Done false
20302 6236 Acked Read 9 Done Sent Done false
20303 6237 Acked Read 2 Done Sent Done false
20304 6238 Acked Write 2 Done Sent Done false
20305 6239 Acked Read 7 Done Sent Done false
20306 6240 Acked Flush 0 Done Sent Done false
20307 6241 Acked Write 2 Done Sent Done false
20308 6242 Acked Write 6 Done Sent Done false
20309 6243 Acked Read 3 Done Sent Done false
20310 6244 Acked Read 10 Done Sent Done false
20311 6245 Acked Write 8 Done Sent Done false
20312 6246 Acked Flush 0 Done Sent Done false
20313 6247 Acked Write 5 Done Sent Done false
20314 6248 Acked Write 10 Done Sent Done false
20315 6249 Acked Read 9 Done Sent Done false
20316 6250 Acked Write 9 Done Sent Done false
20317 6251 Acked Write 9 Done Sent Done false
20318 6252 Acked Read 8 Done Sent Done false
20319 6253 Acked Write 2 Done Sent Done false
20320 6254 Acked Read 4 Done Sent Done false
20321 6255 Acked Read 7 Done Sent Done false
20322 6256 Acked Write 10 Done Sent Done false
20323 6257 Acked Read 5 Done Sent Done false
20324 6258 Acked Read 1 Done Sent Done false
20325 6259 Acked Write 4 Done Sent Done false
20326 6260 Acked Read 8 Done Sent Done false
20327 6261 Acked Flush 0 Done Sent Done false
20328 6262 Acked Read 1 Done Sent Done false
20329 6263 Acked Read 7 Done Sent Done false
20330 6264 Acked Read 4 Done Sent Done false
20331 6265 Acked Write 5 Done Sent Done false
20332 6266 Acked Write 2 Done Sent Done false
20333 6267 Acked Read 3 Done Sent Done false
20334 6268 Acked Write 1 Done Sent Done false
20335 6269 Acked Write 1 Done Sent Done false
20336 6270 Acked Read 9 Done Sent Done false
20337 6271 Acked Read 9 Done Sent Done false
20338 6272 Acked Read 5 Done Sent Done false
20339 6273 Acked Flush 0 Done Sent Done false
20340 6274 Acked Write 6 Done Sent Done false
20341 6275 Acked Read 8 Done Sent Done false
20342 6276 Acked Read 10 Done Sent Done false
20343 6277 Acked Read 5 Done Sent Done false
20344 6278 Acked Write 9 Done Sent Done false
20345 6279 Acked Write 6 Done Sent Done false
20346 6280 Acked Write 8 Done Sent Done false
20347 6281 Acked Flush 0 Done Sent Done false
20348 6282 Acked Flush 0 Done Sent Done false
20349 6283 Acked Flush 0 Done Sent Done false
20350 6284 Acked Read 5 Done Sent Done false
20351 6285 Acked Read 9 Done Sent Done false
20352 6286 Acked Write 9 Done Sent Done false
20353 6287 Acked Write 3 Done Sent Done false
20354 6288 Acked Read 10 Done Sent Done false
20355 6289 Acked Read 2 Done Sent Done false
20356 6290 Acked Read 3 Done Sent Done false
20357 6291 Acked Read 7 Done Sent Done false
20358 6292 Acked Write 1 Done Sent Done false
20359 6293 Acked Flush 0 Done Sent Done false
20360 6294 Acked Read 10 Done Sent Done false
20361 6295 Acked Read 1 Done Sent Done false
20362 6296 Acked Write 7 Done Sent Done false
20363 6297 Acked Write 6 Done Sent Done false
20364 6298 Acked Read 3 Done Sent Done false
20365 6299 Acked Read 8 Done Sent Done false
20366 6300 Acked Write 2 Done Sent Done false
20367 6301 Acked Write 9 Done Sent Done false
20368 6302 Acked Read 4 Done Sent Done false
20369 6303 Acked Read 1 Done Sent Done false
20370 6304 Acked Read 10 Done Sent Done false
20371 6305 Acked Write 7 Done Sent Done false
20372 6306 Acked Flush 0 Done Sent Done false
20373 6307 Acked Write 3 Done Sent Done false
20374 6308 Acked Read 2 Done Sent Done false
20375 6309 Acked Read 4 Done Sent Done false
20376 6310 Acked Flush 0 Done Sent Done false
20377 6311 Acked Flush 0 Done Sent Done false
20378 6312 Acked Write 8 Done Sent Done false
20379 6313 Acked Read 5 Done Sent Done false
20380 6314 Acked Read 3 Done Sent Done false
20381 6315 Acked Flush 0 Done Sent Done false
20382 6316 Acked Read 1 Done Sent Done false
20383 6317 Acked Write 3 Done Sent Done false
20384 6318 Acked Read 5 Done Sent Done false
20385 6319 Acked Read 2 Done Sent Done false
20386 6320 Acked Read 9 Done Sent Done false
20387 6321 Acked Read 9 Done Sent Done false
20388 6322 Acked Write 6 Done Sent Done false
20389 6323 Acked Write 7 Done Sent Done false
20390 6324 Acked Read 4 Done Sent Done false
20391 6325 Acked Flush 0 Done Sent Done false
20392 6326 Acked Write 2 Done Sent Done false
20393 6327 Acked Flush 0 Done Sent Done false
20394 6328 Acked Write 1 Done Sent Done false
20395 6329 Acked Write 3 Done Sent Done false
20396 6330 Acked Write 10 Done Sent Done false
20397 6331 Acked Write 1 Done Sent Done false
20398 6332 Acked Read 4 Done Sent Done false
20399 6333 Acked Read 5 Done Sent Done false
20400 6334 Acked Flush 0 Done Sent Done false
20401 6335 Acked Read 10 Done Sent Done false
20402 6336 Acked Write 4 Done Sent Done false
20403 6337 Acked Flush 0 Done Sent Done false
20404 6338 Acked Read 6 Done Sent Done false
20405 6339 Acked Flush 0 Done Sent Done false
20406 6340 Acked Flush 0 Done Sent Done false
20407 6341 Acked Read 9 Done Sent Done false
20408 6342 Acked Write 3 Done Sent Done false
20409 6343 Acked Write 8 Done Sent Done false
20410 6344 Acked Write 4 Done Sent Done false
20411 6345 Acked Read 10 Done Sent Done false
20412 6346 Acked Write 1 Done Sent Done false
20413 6347 Acked Read 8 Done Sent Done false
20414 6348 Acked Write 10 Done Sent Done false
20415 6349 Acked Write 2 Done Sent Done false
20416 6350 Acked Write 10 Done Sent Done false
20417 6351 Acked Write 5 Done Sent Done false
20418 6352 Acked Write 7 Done Sent Done false
20419 6353 Acked Read 2 Done Sent Done false
20420 6354 Acked Read 4 Done Sent Done false
20421 6355 Acked Read 5 Done Sent Done false
20422 6356 Acked Write 1 Done Sent Done false
20423 6357 Acked Read 1 Done Sent Done false
20424 6358 Acked Write 5 Done Sent Done false
20425 6359 Acked Read 1 Done Sent Done false
20426 6360 Acked Write 10 Done Sent Done false
20427 6361 Acked Flush 0 Done Sent Done false
20428 6362 Acked Write 9 Done Sent Done false
20429 6363 Acked Read 1 Done Sent Done false
20430 6364 Acked Read 10 Done Sent Done false
20431 6365 Acked Flush 0 Done Sent Done false
20432 6366 Acked Flush 0 Done Sent Done false
20433 6367 Acked Write 2 Done Sent Done false
20434 6368 Acked Flush 0 Done Sent Done false
20435 6369 Acked Read 8 Done Sent Done false
20436 6370 Acked Read 10 Done Sent Done false
20437 6371 Acked Read 5 Done Sent Done false
20438 6372 Acked Read 4 Done Sent Done false
20439 6373 Acked Write 1 Done Sent Done false
20440 6374 Acked Read 5 Done Sent Done false
20441 6375 Acked Read 7 Done Sent Done false
20442 6376 Acked Read 6 Done Sent Done false
20443 6377 Acked Write 2 Done Sent Done false
20444 6378 Acked Read 6 Done Sent Done false
20445 6379 Acked Flush 0 Done Sent Done false
20446 6380 Acked Read 4 Done Sent Done false
20447 6381 Acked Read 6 Done Sent Done false
20448 6382 Acked Read 7 Done Sent Done false
20449 6383 Acked Read 4 Done Sent Done false
20450 6384 Acked Write 7 Done Sent Done false
20451 6385 Acked Read 9 Done Sent Done false
20452 6386 Acked Read 8 Done Sent Done false
20453 6387 Acked Read 2 Done Sent Done false
20454 6388 Acked Flush 0 Done Sent Done false
20455 6389 Acked Write 3 Done Sent Done false
20456 6390 Acked Write 9 Done Sent Done false
20457 6391 Acked Flush 0 Done Sent Done false
20458 6392 Acked Write 6 Done Sent Done false
20459 6393 Acked Write 5 Done Sent Done false
20460 6394 Acked Write 8 Done Sent Done false
20461 6395 Acked Read 5 Done Sent Done false
20462 6396 Acked Read 1 Done Sent Done false
20463 6397 Acked Write 1 Done Sent Done false
20464 6398 Acked Read 4 Done Sent Done false
20465 6399 Acked Read 9 Done Sent Done false
20466 6400 Acked Read 9 Done Sent Done false
20467 6401 Acked Write 7 Done Sent Done false
20468 6402 Acked Read 5 Done Sent Done false
20469 6403 Acked Flush 0 Done Sent Done false
20470 6404 Acked Read 2 Done Sent Done false
20471 6405 Acked Write 7 Done Sent Done false
20472 6406 Acked Write 6 Done Sent Done false
20473 6407 Acked Flush 0 Done Sent Done false
20474 6408 Acked Write 5 Done Sent Done false
20475 6409 Acked Write 1 Done Sent Done false
20476 6410 Acked Write 7 Done Sent Done false
20477 6411 Acked Write 5 Done Sent Done false
20478 6412 Acked Write 10 Done Sent Done false
20479 6413 Acked Write 8 Done Sent Done false
20480 6414 Acked Read 5 Done Sent Done false
20481 6415 Acked Write 3 Done Sent Done false
20482 6416 Acked Flush 0 Done Sent Done false
20483 6417 Acked Write 9 Done Sent Done false
20484 6418 Acked Read 9 Done Sent Done false
20485 6419 Acked Write 2 Done Sent Done false
20486 6420 Acked Write 9 Done Sent Done false
20487 6421 Acked Flush 0 Done Sent Done false
20488 6422 Acked Write 6 Done Sent Done false
20489 6423 Acked Read 7 Done Sent Done false
20490 6424 Acked Write 6 Done Sent Done false
20491 6425 Acked Write 7 Done Sent Done false
20492 6426 Acked Read 1 Done Sent Done false
20493 6427 Acked Write 3 Done Sent Done false
20494 6428 Acked Read 5 Done Sent Done false
20495 6429 Acked Read 3 Done Sent Done false
20496 6430 Acked Write 5 Done Sent Done false
20497 6431 Acked Write 9 Done Sent Done false
20498 6432 Acked Write 2 Done Sent Done false
20499 6433 Acked Write 8 Done Sent Done false
20500 STATES DS:0 DS:1 DS:2 TOTAL
20501 Sent 0 301 0 301
20502 Done 301 0 301 602
20503 Skipped 0 0 0 0
20504 Error 0 0 0 0
20505 Last Flush: 6421 6132 6421
20506 Downstairs last five completed: 6132 6131 6130 6129 6128
20507 Upstairs last five completed: 6433 6432 6431 6430 6429
20508 CLIENT: Up:0 ds:301 act:2
20509 [8820] got start action
20510 [8820] I am going to restart
20511 Starting downstairs at port 8820
20512 Make output file at "/tmp/dsc/downstairs-8820.txt"
20513 [8820][1] reports Starting
20514 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1220)
20515 [8820][1] reports Running
20516 ----------------------------------------------------------------
20517 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20518 States: Active Offline Active
20519 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20520 6133 Acked Flush 0 Done Sent Done false
20521 6134 Acked Write 4 Done Sent Done false
20522 6135 Acked Write 9 Done Sent Done false
20523 6136 Acked Flush 0 Done Sent Done false
20524 6137 Acked Write 7 Done Sent Done false
20525 6138 Acked Read 7 Done Sent Done false
20526 6139 Acked Write 3 Done Sent Done false
20527 6140 Acked Write 6 Done Sent Done false
20528 6141 Acked Write 7 Done Sent Done false
20529 6142 Acked Read 10 Done Sent Done false
20530 6143 Acked Read 7 Done Sent Done false
20531 6144 Acked Write 4 Done Sent Done false
20532 6145 Acked Read 8 Done Sent Done false
20533 6146 Acked Flush 0 Done Sent Done false
20534 6147 Acked Flush 0 Done Sent Done false
20535 6148 Acked Read 10 Done Sent Done false
20536 6149 Acked Read 8 Done Sent Done false
20537 6150 Acked Read 9 Done Sent Done false
20538 6151 Acked Read 3 Done Sent Done false
20539 6152 Acked Read 8 Done Sent Done false
20540 6153 Acked Read 2 Done Sent Done false
20541 6154 Acked Write 2 Done Sent Done false
20542 6155 Acked Write 1 Done Sent Done false
20543 6156 Acked Write 8 Done Sent Done false
20544 6157 Acked Write 5 Done Sent Done false
20545 6158 Acked Write 2 Done Sent Done false
20546 6159 Acked Read 6 Done Sent Done false
20547 6160 Acked Read 10 Done Sent Done false
20548 6161 Acked Read 9 Done Sent Done false
20549 6162 Acked Read 4 Done Sent Done false
20550 6163 Acked Write 7 Done Sent Done false
20551 6164 Acked Write 5 Done Sent Done false
20552 6165 Acked Read 6 Done Sent Done false
20553 6166 Acked Read 2 Done Sent Done false
20554 6167 Acked Read 5 Done Sent Done false
20555 6168 Acked Read 6 Done Sent Done false
20556 6169 Acked Write 7 Done Sent Done false
20557 6170 Acked Write 2 Done Sent Done false
20558 6171 Acked Read 3 Done Sent Done false
20559 6172 Acked Write 7 Done Sent Done false
20560 6173 Acked Read 3 Done Sent Done false
20561 6174 Acked Read 8 Done Sent Done false
20562 6175 Acked Read 2 Done Sent Done false
20563 6176 Acked Write 7 Done Sent Done false
20564 6177 Acked Write 7 Done Sent Done false
20565 6178 Acked Flush 0 Done Sent Done false
20566 6179 Acked Write 3 Done Sent Done false
20567 6180 Acked Write 8 Done Sent Done false
20568 6181 Acked Read 4 Done Sent Done false
20569 6182 Acked Write 5 Done Sent Done false
20570 6183 Acked Read 1 Done Sent Done false
20571 6184 Acked Write 8 Done Sent Done false
20572 6185 Acked Read 10 Done Sent Done false
20573 6186 Acked Read 8 Done Sent Done false
20574 6187 Acked Read 6 Done Sent Done false
20575 6188 Acked Read 10 Done Sent Done false
20576 6189 Acked Write 4 Done Sent Done false
20577 6190 Acked Write 5 Done Sent Done false
20578 6191 Acked Read 3 Done Sent Done false
20579 6192 Acked Write 3 Done Sent Done false
20580 6193 Acked Read 9 Done Sent Done false
20581 6194 Acked Read 1 Done Sent Done false
20582 6195 Acked Write 10 Done Sent Done false
20583 6196 Acked Write 3 Done Sent Done false
20584 6197 Acked Read 8 Done Sent Done false
20585 6198 Acked Write 2 Done Sent Done false
20586 6199 Acked Read 1 Done Sent Done false
20587 6200 Acked Read 6 Done Sent Done false
20588 6201 Acked Write 6 Done Sent Done false
20589 6202 Acked Write 8 Done Sent Done false
20590 6203 Acked Flush 0 Done Sent Done false
20591 6204 Acked Read 10 Done Sent Done false
20592 6205 Acked Flush 0 Done Sent Done false
20593 6206 Acked Read 9 Done Sent Done false
20594 6207 Acked Write 7 Done Sent Done false
20595 6208 Acked Write 4 Done Sent Done false
20596 6209 Acked Write 5 Done Sent Done false
20597 6210 Acked Read 3 Done Sent Done false
20598 6211 Acked Flush 0 Done Sent Done false
20599 6212 Acked Read 2 Done Sent Done false
20600 6213 Acked Flush 0 Done Sent Done false
20601 6214 Acked Write 9 Done Sent Done false
20602 6215 Acked Read 7 Done Sent Done false
20603 6216 Acked Write 4 Done Sent Done false
20604 6217 Acked Write 2 Done Sent Done false
20605 6218 Acked Write 4 Done Sent Done false
20606 6219 Acked Read 8 Done Sent Done false
20607 6220 Acked Write 3 Done Sent Done false
20608 6221 Acked Read 5 Done Sent Done false
20609 6222 Acked Read 2 Done Sent Done false
20610 6223 Acked Read 8 Done Sent Done false
20611 6224 Acked Write 5 Done Sent Done false
20612 6225 Acked Read 2 Done Sent Done false
20613 6226 Acked Read 9 Done Sent Done false
20614 6227 Acked Write 7 Done Sent Done false
20615 6228 Acked Write 2 Done Sent Done false
20616 6229 Acked Read 10 Done Sent Done false
20617 6230 Acked Write 7 Done Sent Done false
20618 6231 Acked Flush 0 Done Sent Done false
20619 6232 Acked Read 8 Done Sent Done false
20620 6233 Acked Read 1 Done Sent Done false
20621 6234 Acked Flush 0 Done Sent Done false
20622 6235 Acked Read 8 Done Sent Done false
20623 6236 Acked Read 9 Done Sent Done false
20624 6237 Acked Read 2 Done Sent Done false
20625 6238 Acked Write 2 Done Sent Done false
20626 6239 Acked Read 7 Done Sent Done false
20627 6240 Acked Flush 0 Done Sent Done false
20628 6241 Acked Write 2 Done Sent Done false
20629 6242 Acked Write 6 Done Sent Done false
20630 6243 Acked Read 3 Done Sent Done false
20631 6244 Acked Read 10 Done Sent Done false
20632 6245 Acked Write 8 Done Sent Done false
20633 6246 Acked Flush 0 Done Sent Done false
20634 6247 Acked Write 5 Done Sent Done false
20635 6248 Acked Write 10 Done Sent Done false
20636 6249 Acked Read 9 Done Sent Done false
20637 6250 Acked Write 9 Done Sent Done false
20638 6251 Acked Write 9 Done Sent Done false
20639 6252 Acked Read 8 Done Sent Done false
20640 6253 Acked Write 2 Done Sent Done false
20641 6254 Acked Read 4 Done Sent Done false
20642 6255 Acked Read 7 Done Sent Done false
20643 6256 Acked Write 10 Done Sent Done false
20644 6257 Acked Read 5 Done Sent Done false
20645 6258 Acked Read 1 Done Sent Done false
20646 6259 Acked Write 4 Done Sent Done false
20647 6260 Acked Read 8 Done Sent Done false
20648 6261 Acked Flush 0 Done Sent Done false
20649 6262 Acked Read 1 Done Sent Done false
20650 6263 Acked Read 7 Done Sent Done false
20651 6264 Acked Read 4 Done Sent Done false
20652 6265 Acked Write 5 Done Sent Done false
20653 6266 Acked Write 2 Done Sent Done false
20654 6267 Acked Read 3 Done Sent Done false
20655 6268 Acked Write 1 Done Sent Done false
20656 6269 Acked Write 1 Done Sent Done false
20657 6270 Acked Read 9 Done Sent Done false
20658 6271 Acked Read 9 Done Sent Done false
20659 6272 Acked Read 5 Done Sent Done false
20660 6273 Acked Flush 0 Done Sent Done false
20661 6274 Acked Write 6 Done Sent Done false
20662 6275 Acked Read 8 Done Sent Done false
20663 6276 Acked Read 10 Done Sent Done false
20664 6277 Acked Read 5 Done Sent Done false
20665 6278 Acked Write 9 Done Sent Done false
20666 6279 Acked Write 6 Done Sent Done false
20667 6280 Acked Write 8 Done Sent Done false
20668 6281 Acked Flush 0 Done Sent Done false
20669 6282 Acked Flush 0 Done Sent Done false
20670 6283 Acked Flush 0 Done Sent Done false
20671 6284 Acked Read 5 Done Sent Done false
20672 6285 Acked Read 9 Done Sent Done false
20673 6286 Acked Write 9 Done Sent Done false
20674 6287 Acked Write 3 Done Sent Done false
20675 6288 Acked Read 10 Done Sent Done false
20676 6289 Acked Read 2 Done Sent Done false
20677 6290 Acked Read 3 Done Sent Done false
20678 6291 Acked Read 7 Done Sent Done false
20679 6292 Acked Write 1 Done Sent Done false
20680 6293 Acked Flush 0 Done Sent Done false
20681 6294 Acked Read 10 Done Sent Done false
20682 6295 Acked Read 1 Done Sent Done false
20683 6296 Acked Write 7 Done Sent Done false
20684 6297 Acked Write 6 Done Sent Done false
20685 6298 Acked Read 3 Done Sent Done false
20686 6299 Acked Read 8 Done Sent Done false
20687 6300 Acked Write 2 Done Sent Done false
20688 6301 Acked Write 9 Done Sent Done false
20689 6302 Acked Read 4 Done Sent Done false
20690 6303 Acked Read 1 Done Sent Done false
20691 6304 Acked Read 10 Done Sent Done false
20692 6305 Acked Write 7 Done Sent Done false
20693 6306 Acked Flush 0 Done Sent Done false
20694 6307 Acked Write 3 Done Sent Done false
20695 6308 Acked Read 2 Done Sent Done false
20696 6309 Acked Read 4 Done Sent Done false
20697 6310 Acked Flush 0 Done Sent Done false
20698 6311 Acked Flush 0 Done Sent Done false
20699 6312 Acked Write 8 Done Sent Done false
20700 6313 Acked Read 5 Done Sent Done false
20701 6314 Acked Read 3 Done Sent Done false
20702 6315 Acked Flush 0 Done Sent Done false
20703 6316 Acked Read 1 Done Sent Done false
20704 6317 Acked Write 3 Done Sent Done false
20705 6318 Acked Read 5 Done Sent Done false
20706 6319 Acked Read 2 Done Sent Done false
20707 6320 Acked Read 9 Done Sent Done false
20708 6321 Acked Read 9 Done Sent Done false
20709 6322 Acked Write 6 Done Sent Done false
20710 6323 Acked Write 7 Done Sent Done false
20711 6324 Acked Read 4 Done Sent Done false
20712 6325 Acked Flush 0 Done Sent Done false
20713 6326 Acked Write 2 Done Sent Done false
20714 6327 Acked Flush 0 Done Sent Done false
20715 6328 Acked Write 1 Done Sent Done false
20716 6329 Acked Write 3 Done Sent Done false
20717 6330 Acked Write 10 Done Sent Done false
20718 6331 Acked Write 1 Done Sent Done false
20719 6332 Acked Read 4 Done Sent Done false
20720 6333 Acked Read 5 Done Sent Done false
20721 6334 Acked Flush 0 Done Sent Done false
20722 6335 Acked Read 10 Done Sent Done false
20723 6336 Acked Write 4 Done Sent Done false
20724 6337 Acked Flush 0 Done Sent Done false
20725 6338 Acked Read 6 Done Sent Done false
20726 6339 Acked Flush 0 Done Sent Done false
20727 6340 Acked Flush 0 Done Sent Done false
20728 6341 Acked Read 9 Done Sent Done false
20729 6342 Acked Write 3 Done Sent Done false
20730 6343 Acked Write 8 Done Sent Done false
20731 6344 Acked Write 4 Done Sent Done false
20732 6345 Acked Read 10 Done Sent Done false
20733 6346 Acked Write 1 Done Sent Done false
20734 6347 Acked Read 8 Done Sent Done false
20735 6348 Acked Write 10 Done Sent Done false
20736 6349 Acked Write 2 Done Sent Done false
20737 6350 Acked Write 10 Done Sent Done false
20738 6351 Acked Write 5 Done Sent Done false
20739 6352 Acked Write 7 Done Sent Done false
20740 6353 Acked Read 2 Done Sent Done false
20741 6354 Acked Read 4 Done Sent Done false
20742 6355 Acked Read 5 Done Sent Done false
20743 6356 Acked Write 1 Done Sent Done false
20744 6357 Acked Read 1 Done Sent Done false
20745 6358 Acked Write 5 Done Sent Done false
20746 6359 Acked Read 1 Done Sent Done false
20747 6360 Acked Write 10 Done Sent Done false
20748 6361 Acked Flush 0 Done Sent Done false
20749 6362 Acked Write 9 Done Sent Done false
20750 6363 Acked Read 1 Done Sent Done false
20751 6364 Acked Read 10 Done Sent Done false
20752 6365 Acked Flush 0 Done Sent Done false
20753 6366 Acked Flush 0 Done Sent Done false
20754 6367 Acked Write 2 Done Sent Done false
20755 6368 Acked Flush 0 Done Sent Done false
20756 6369 Acked Read 8 Done Sent Done false
20757 6370 Acked Read 10 Done Sent Done false
20758 6371 Acked Read 5 Done Sent Done false
20759 6372 Acked Read 4 Done Sent Done false
20760 6373 Acked Write 1 Done Sent Done false
20761 6374 Acked Read 5 Done Sent Done false
20762 6375 Acked Read 7 Done Sent Done false
20763 6376 Acked Read 6 Done Sent Done false
20764 6377 Acked Write 2 Done Sent Done false
20765 6378 Acked Read 6 Done Sent Done false
20766 6379 Acked Flush 0 Done Sent Done false
20767 6380 Acked Read 4 Done Sent Done false
20768 6381 Acked Read 6 Done Sent Done false
20769 6382 Acked Read 7 Done Sent Done false
20770 6383 Acked Read 4 Done Sent Done false
20771 6384 Acked Write 7 Done Sent Done false
20772 6385 Acked Read 9 Done Sent Done false
20773 6386 Acked Read 8 Done Sent Done false
20774 6387 Acked Read 2 Done Sent Done false
20775 6388 Acked Flush 0 Done Sent Done false
20776 6389 Acked Write 3 Done Sent Done false
20777 6390 Acked Write 9 Done Sent Done false
20778 6391 Acked Flush 0 Done Sent Done false
20779 6392 Acked Write 6 Done Sent Done false
20780 6393 Acked Write 5 Done Sent Done false
20781 6394 Acked Write 8 Done Sent Done false
20782 6395 Acked Read 5 Done Sent Done false
20783 6396 Acked Read 1 Done Sent Done false
20784 6397 Acked Write 1 Done Sent Done false
20785 6398 Acked Read 4 Done Sent Done false
20786 6399 Acked Read 9 Done Sent Done false
20787 6400 Acked Read 9 Done Sent Done false
20788 6401 Acked Write 7 Done Sent Done false
20789 6402 Acked Read 5 Done Sent Done false
20790 6403 Acked Flush 0 Done Sent Done false
20791 6404 Acked Read 2 Done Sent Done false
20792 6405 Acked Write 7 Done Sent Done false
20793 6406 Acked Write 6 Done Sent Done false
20794 6407 Acked Flush 0 Done Sent Done false
20795 6408 Acked Write 5 Done Sent Done false
20796 6409 Acked Write 1 Done Sent Done false
20797 6410 Acked Write 7 Done Sent Done false
20798 6411 Acked Write 5 Done Sent Done false
20799 6412 Acked Write 10 Done Sent Done false
20800 6413 Acked Write 8 Done Sent Done false
20801 6414 Acked Read 5 Done Sent Done false
20802 6415 Acked Write 3 Done Sent Done false
20803 6416 Acked Flush 0 Done Sent Done false
20804 6417 Acked Write 9 Done Sent Done false
20805 6418 Acked Read 9 Done Sent Done false
20806 6419 Acked Write 2 Done Sent Done false
20807 6420 Acked Write 9 Done Sent Done false
20808 6421 Acked Flush 0 Done Sent Done false
20809 6422 Acked Write 6 Done Sent Done false
20810 6423 Acked Read 7 Done Sent Done false
20811 6424 Acked Write 6 Done Sent Done false
20812 6425 Acked Write 7 Done Sent Done false
20813 6426 Acked Read 1 Done Sent Done false
20814 6427 Acked Write 3 Done Sent Done false
20815 6428 Acked Read 5 Done Sent Done false
20816 6429 Acked Read 3 Done Sent Done false
20817 6430 Acked Write 5 Done Sent Done false
20818 6431 Acked Write 9 Done Sent Done false
20819 6432 Acked Write 2 Done Sent Done false
20820 6433 Acked Write 8 Done Sent Done false
20821 6434 Acked Flush 0 Done Sent Done false
20822 STATES DS:0 DS:1 DS:2 TOTAL
20823 Sent 0 302 0 302
20824 Done 302 0 302 604
20825 Skipped 0 0 0 0
20826 Error 0 0 0 0
20827 Last Flush: 6434 6132 6434
20828 Downstairs last five completed: 6132 6131 6130 6129 6128
20829 Upstairs last five completed: 6434 6433 6432 6431 6430
20830 CLIENT: Up:0 ds:302 act:2
208312025-01-15T05:23:21.125ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208322025-01-15T05:23:21.125ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62214) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208332025-01-15T05:23:21.126ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208342025-01-15T05:23:21.126ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208352025-01-15T05:23:21.126ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208362025-01-15T05:23:21.126ZINFOcrucible: send last flush ID to this DS: 6132 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208372025-01-15T05:23:21.127ZINFOcrucible: Replied this last flush ID: 6132 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208382025-01-15T05:23:21.127ZINFOcrucible: [1] client re-new 302 jobs since flush 6132 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208392025-01-15T05:23:21.127ZINFOcrucible: [1] Replayed 302 jobs since flush: 6132 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
20840 ----------------------------------------------------------------
20841 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20842 Downstairs last five completed: 6434 6433 6432 6431 6430
20843 Upstairs last five completed: 6434 6433 6432 6431 6430
20844 CLIENT: Up:0 ds:0 act:3
20845 Replay: All jobs finished
20846 Main task has work to do, go find it
20847 got dsc Stop(0)
20848 stop 0
20849 Jan 15 05:23:23.355 INFO request completed, latency_us[8810] Got stop action so:false kr:false
20850 : 374, response_code: 204, uri: /stop/cid/0, method: POST, req_id: d4842759-28c0-4e11-9716-ffff31480851, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
20851 Jan 15 05:23:23.356 INFO request completed, latency_us: 157, response_code: 200, uri: /state/cid/0, method: GET, req_id: 54cfe701-3aec-408f-96b5-f470642246a9, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
208522025-01-15T05:23:23.356ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208532025-01-15T05:23:23.356ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208542025-01-15T05:23:23.356ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
208552025-01-15T05:23:23.356ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
20856 [8810] Exited with: ExitStatus(unix_wait_status(9))
20857 [8810][0] reports Exit
20858 Jan 15 05:23:27.357 INFO request completed, latency_us: 178, response_code: 200, uri: /state/cid/0, method: GET, req_id: 5d792328-6b16-4be1-ae05-68efdd9697e4, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
20859 001/300 Write block 3243 len 16384 data: 6 4 5 4
20860 002/300 Write block 2778 len 24576 data: 4 4 8 7 6 5
20861 003/300 Read block 4317 len 36864
20862 004/300 Read block 4199 len 12288
20863 005/300 Flush
20864 006/300 Write block 1783 len 40960 data: 8 8 8 10 10 9 8 8 8 6
20865 007/300 Read block 694 len 32768
20866 008/300 Read block 1671 len 40960
20867 009/300 Read block 3851 len 32768
20868 010/300 Read block 2628 len 4096
20869 011/300 Read block 1531 len 12288
20870 012/300 Read block 13 len 36864
20871 013/300 Read block 394 len 40960
20872 014/300 Read block 3739 len 40960
20873 015/300 Read block 4665 len 28672
20874 016/300 Write block 2891 len 28672 data: 3 4 4 5 6 6 6
20875 017/300 Flush
20876 018/300 Write block 4319 len 20480 data: 7 8 7 6 5
20877 019/300 Read block 4394 len 20480
20878 020/300 Read block 3648 len 12288
20879 021/300 Read block 969 len 12288
20880 022/300 Read block 3876 len 24576
20881 023/300 Read block 1961 len 20480
20882 024/300 Write block 1771 len 20480 data: 3 3 4 5 5
20883 025/300 Read block 3261 len 20480
20884 026/300 Read block 3707 len 4096
20885 027/300 Write block 4886 len 24576 data: 6 5 6 5 5 5
20886 028/300 Read block 4529 len 4096
20887 029/300 Read block 2184 len 32768
20888 030/300 Write block 1649 len 8192 data: 6 5
20889 031/300 Write block 3583 len 8192 data: 4 4
20890 032/300 Read block 3821 len 36864
20891 033/300 Write block 1608 len 36864 data: 6 6 6 5 5 4 4 4 4
20892 034/300 Read block 4493 len 24576
20893 035/300 Read block 1907 len 4096
20894 036/300 Read block 2268 len 20480
20895 037/300 Write block 3364 len 40960 data: 5 6 6 6 6 6 8 5 4 5
20896 038/300 Read block 51 len 36864
20897 039/300 Read block 3893 len 36864
20898 040/300 Write block 538 len 12288 data: 4 3 3
20899 041/300 Write block 4829 len 16384 data: 4 3 3 4
20900 042/300 Read block 1021 len 12288
20901 043/300 Write block 1926 len 32768 data: 4 4 4 4 5 4 5 6
20902 044/300 Read block 610 len 32768
20903 045/300 Write block 1988 len 40960 data: 2 2 2 2 3 2 2 2 3 4
20904 046/300 Read block 2735 len 28672
20905 047/300 Read block 1622 len 32768
20906 048/300 Read block 3422 len 32768
20907 049/300 Read block 1971 len 16384
20908 050/300 Read block 356 len 32768
20909 051/300 Read block 3869 len 20480
20910 052/300 Read block 4691 len 40960
20911 053/300 Write block 2024 len 8192 data: 7 6
20912 054/300 Write block 1043 len 16384 data: 5 4 4 4
20913 055/300 Write block 494 len 8192 data: 3 3
20914 056/300 Read block 436 len 24576
20915 057/300 Write block 3816 len 36864 data: 6 5 4 4 5 5 4 4 3
20916 058/300 Write block 257 len 32768 data: 9 9 6 6 4 5 6 7
20917 059/300 Write block 3675 len 8192 data: 7 7
20918 060/300 Read block 528 len 8192
20919 061/300 Read block 3323 len 32768
20920 062/300 Read block 730 len 20480
20921 063/300 Flush
20922 064/300 Write block 3182 len 32768 data: 7 7 6 5 5 5 4 4
20923 065/300 Flush
20924 066/300 Write block 3319 len 12288 data: 5 4 5
20925 067/300 Read block 3735 len 8192
20926 068/300 Read block 1935 len 8192
20927 069/300 Write block 3303 len 4096 data: 7
20928 070/300 Write block 1926 len 16384 data: 5 5 5 5
20929 071/300 Write block 3861 len 24576 data: 4 5 4 4 4 3
20930 072/300 Read block 861 len 28672
20931 073/300 Write block 307 len 28672 data: 3 3 3 4 4 3 2
20932 074/300 Write block 328 len 28672 data: 7 4 4 4 4 4 4
20933 075/300 Write block 87 len 8192 data: 3 7
20934 076/300 Read block 2713 len 20480
20935 077/300 Write block 2507 len 12288 data: 4 5 4
20936 078/300 Read block 1587 len 24576
20937 079/300 Write block 4317 len 24576 data: 6 6 8 9 8 7
20938 080/300 Write block 2284 len 20480 data: 5 4 6 6 5
20939 081/300 Read block 163 len 4096
20940 082/300 Write block 2491 len 4096 data: 7
20941 083/300 Write block 1668 len 36864 data: 3 3 3 3 3 3 4 3 4
20942 084/300 Read block 2195 len 12288
20943 085/300 Write block 3533 len 12288 data: 5 4 5
20944 086/300 Write block 4543 len 32768 data: 5 4 5 4 6 5 7 8
20945 087/300 Read block 175 len 28672
20946 088/300 Write block 824 len 8192 data: 7 6
20947 089/300 Write block 666 len 24576 data: 4 4 3 3 3 3
20948 090/300 Write block 1754 len 12288 data: 4 4 3
20949 091/300 Read block 3040 len 36864
20950 092/300 Read block 463 len 28672
20951 093/300 Read block 4943 len 40960
20952 094/300 Read block 2608 len 40960
20953 095/300 Write block 3018 len 36864 data: 3 3 4 4 4 5 5 4 4
20954 096/300 Flush
20955 097/300 Read block 4034 len 32768
20956 098/300 Write block 4597 len 12288 data: 2 2 3
20957 099/300 Read block 561 len 40960
20958 100/300 Write block 1571 len 24576 data: 6 7 9 10 10 9
20959 101/300 Read block 1727 len 4096
20960 102/300 Flush
20961 103/300 Read block 1553 len 16384
20962 104/300 Read block 2298 len 36864
20963 105/300 Read block 2510 len 20480
20964 106/300 Write block 4937 len 12288 data: 6 5 5
20965 107/300 Flush
20966 108/300 Read block 1265 len 24576
20967 109/300 Read block 4840 len 36864
20968 110/300 Read block 1917 len 4096
20969 111/300 Read block 2859 len 28672
20970 112/300 Write block 1411 len 32768 data: 4 4 4 4 3 4 4 4
20971 113/300 Write block 3944 len 8192 data: 7 6
20972 114/300 Write block 2176 len 32768 data: 2 2 3 5 5 6 9 9
20973 115/300 Write block 1358 len 16384 data: 2 2 2 2
20974 116/300 Read block 4735 len 16384
20975 117/300 Flush
20976 118/300 Write block 527 len 32768 data: 4 4 5 4 4 4 5 4
20977 119/300 Flush
20978 120/300 Write block 450 len 40960 data: 6 5 5 4 6 7 7 4 6 4
20979 121/300 Write block 4231 len 16384 data: 5 5 5 4
20980 122/300 Write block 773 len 8192 data: 3 3
20981 123/300 Read block 1528 len 24576
20982 124/300 Read block 1120 len 12288
20983 125/300 Read block 573 len 40960
20984 126/300 Read block 4134 len 24576
20985 127/300 Read block 2777 len 40960
20986 128/300 Read block 3029 len 24576
20987 129/300 Write block 1201 len 28672 data: 4 5 7 7 6 5 6
20988 130/300 Write block 1898 len 28672 data: 6 6 7 5 5 4 4
20989 131/300 Read block 837 len 36864
20990 132/300 Write block 4810 len 36864 data: 3 3 3 3 4 6 5 7 6
20991 133/300 Write block 1602 len 40960 data: 5 8 6 6 7 6 7 7 7 6
20992 134/300 Read block 4658 len 28672
20993 135/300 Write block 4615 len 12288 data: 5 5 5
20994 136/300 Write block 4646 len 24576 data: 2 2 3 2 2 4
20995 137/300 Write block 4210 len 16384 data: 6 6 5 4
20996 138/300 Write block 2747 len 4096 data: 5
20997 139/300 Read block 449 len 36864
20998 140/300 Read block 1446 len 40960
20999 141/300 Flush
21000 142/300 Read block 287 len 28672
21001 143/300 Write block 4 len 32768 data: 3 3 5 5 5 5 5 5
21002 144/300 Write block 3031 len 40960 data: 7 7 7 7 9 10 10 9 9 9
21003 145/300 Read block 4580 len 4096
21004 146/300 Write block 3812 len 24576 data: 3 3 4 5 7 6
21005 147/300 Write block 2877 len 40960 data: 4 5 6 7 7 7 6 6 6 5
21006 148/300 Read block 444 len 40960
21007 149/300 Write block 274 len 24576 data: 3 3 3 4 5 4
21008 150/300 Write block 3712 len 24576 data: 3 4 4 4 6 5
21009 151/300 Write block 1286 len 32768 data: 2 3 3 3 3 3 4 5
21010 152/300 Write block 2965 len 12288 data: 4 2 2
21011 153/300 Write block 4969 len 8192 data: 3 3
21012 154/300 Write block 2077 len 8192 data: 2 2
21013 155/300 Write block 2089 len 24576 data: 3 3 3 3 3 3
21014 156/300 Flush
21015 157/300 Read block 3726 len 4096
21016 158/300 Write block 1365 len 36864 data: 3 3 3 4 6 6 7 6 5
21017 159/300 Read block 645 len 8192
21018 160/300 Flush
21019 161/300 Write block 2563 len 4096 data: 5
21020 162/300 Write block 1586 len 32768 data: 9 8 6 5 4 4 2 2
21021 163/300 Write block 3835 len 40960 data: 5 6 6 5 6 6 7 7 6 9
21022 164/300 Write block 3152 len 16384 data: 5 5 5 6
21023 165/300 Write block 1142 len 4096 data: 3
21024 166/300 Write block 3333 len 36864 data: 3 6 6 6 6 6 5 4 3
21025 167/300 Read block 2412 len 36864
21026 168/300 Read block 1083 len 24576
21027 169/300 Write block 3449 len 40960 data: 4 4 4 5 5 5 5 5 5 4
21028 170/300 Write block 542 len 8192 data: 3 3
21029 171/300 Read block 3761 len 32768
21030 172/300 Read block 4927 len 36864
21031 173/300 Write block 2825 len 28672 data: 6 6 5 4 3 6 7
21032 174/300 Write block 3560 len 28672 data: 5 6 6 5 5 5 4
21033 175/300 Read block 3915 len 32768
21034 176/300 Flush
21035 177/300 Write block 789 len 24576 data: 5 5 5 5 5 5
21036 178/300 Read block 2794 len 24576
21037 179/300 Write block 1153 len 40960 data: 5 5 4 4 5 4 4 4 4 5
21038 180/300 Write block 2293 len 32768 data: 4 4 4 4 3 4 4 4
21039 181/300 Write block 3799 len 8192 data: 5 5
21040 182/300 Write block 154 len 36864 data: 3 2 2 2 3 4 4 5 5
21041 183/300 Read block 196 len 12288
21042 184/300 Read block 3193 len 12288
21043 185/300 Write block 861 len 40960 data: 3 3 3 3 2 4 4 4 6 6
21044 186/300 Write block 4906 len 12288 data: 4 4 4
21045 187/300 Read block 2614 len 32768
21046 188/300 Flush
21047 189/300 Write block 4383 len 36864 data: 6 5 5 4 5 4 3 4 3
21048 190/300 Write block 2476 len 28672 data: 4 4 2 4 4 4 4
21049 191/300 Read block 4072 len 20480
21050 192/300 Write block 512 len 36864 data: 2 2 2 3 3 3 4 5 7
21051 193/300 Read block 63 len 36864
21052 194/300 Read block 4550 len 24576
21053 195/300 Read block 4434 len 20480
21054 196/300 Read block 1408 len 4096
21055 197/300 Write block 2905 len 28672 data: 4 4 3 3 4 4 5
21056 198/300 Write block 3869 len 28672 data: 4 5 5 7 6 5 4
21057 199/300 Write block 3267 len 28672 data: 4 4 6 5 4 4 5
21058 200/300 Write block 4675 len 36864 data: 4 6 7 5 4 3 2 2 2
21059 201/300 Read block 378 len 28672
21060 202/300 Write block 1635 len 20480 data: 7 7 8 10 9
21061 203/300 Write block 3436 len 24576 data: 3 2 4 4 4 4
21062 204/300 Read block 3322 len 32768
21063 205/300 Read block 3032 len 32768
21064 206/300 Write block 554 len 12288 data: 4 3 4
21065 207/300 Write block 3737 len 24576 data: 4 6 5 6 5 5
21066 208/300 Flush
21067 209/300 Read block 334 len 4096
21068 210/300 Read block 378 len 24576
21069 211/300 Flush
21070 212/300 Write block 4429 len 36864 data: 7 5 5 6 5 5 3 3 3
21071 213/300 Write block 2557 len 36864 data: 3 4 4 3 3 5 6 6 5
21072 214/300 Write block 4731 len 8192 data: 5 7
21073 215/300 Read block 1294 len 12288
21074 216/300 Write block 3384 len 12288 data: 7 7 7
21075 217/300 Read block 2644 len 4096
21076 218/300 Read block 1552 len 40960
21077 219/300 Write block 557 len 12288 data: 4 5 6
21078 220/300 Read block 4405 len 12288
21079 221/300 Flush
21080 222/300 Read block 4974 len 20480
21081 223/300 Read block 4913 len 4096
21082 224/300 Read block 1131 len 8192
21083 225/300 Read block 2986 len 28672
21084 226/300 Write block 678 len 36864 data: 8 7 6 5 5 6 7 8 5
21085 227/300 Read block 3170 len 4096
21086 228/300 Read block 2395 len 4096
21087 229/300 Read block 2496 len 40960
21088 230/300 Flush
21089 231/300 Read block 1950 len 4096
21090 232/300 Read block 2559 len 12288
21091 233/300 Read block 2092 len 12288
21092 234/300 Write block 3834 len 36864 data: 4 6 7 7 6 7 7 8 8
21093 235/300 Flush
21094 236/300 Write block 2723 len 28672 data: 3 3 2 2 3 4 4
21095 237/300 Read block 4959 len 8192
21096 238/300 Read block 4092 len 28672
21097 239/300 Write block 3672 len 4096 data: 9
21098 240/300 Write block 4523 len 28672 data: 4 4 5 2 2 2 3
21099 241/300 Read block 4798 len 28672
21100 242/300 Write block 1057 len 4096 data: 3
21101 243/300 Read block 1911 len 12288
21102 244/300 Read block 4603 len 28672
21103 245/300 Read block 1873 len 40960
21104 246/300 Read block 272 len 24576
21105 247/300 Write block 700 len 20480 data: 3 4 4 5 4
21106 248/300 Read block 1698 len 36864
21107 249/300 Write block 3669 len 8192 data: 7 7
21108 250/300 Read block 590 len 12288
21109 251/300 Read block 220 len 16384
21110 252/300 Read block 188 len 8192
21111 253/300 Read block 3823 len 4096
21112 254/300 Write block 196 len 4096 data: 5
21113 255/300 Write block 3781 len 28672 data: 4 4 5 6 6 5 4
21114 256/300 Write block 832 len 24576 data: 3 2 2 2 3 3
21115 257/300 Read block 1941 len 32768
21116 258/300 Read block 921 len 4096
21117 259/300 Read block 1015 len 36864
21118 260/300 Read block 1681 len 8192
21119 261/300 Read block 264 len 12288
21120 262/300 Write block 847 len 8192 data: 2 2
21121 263/300 Read block 2452 len 20480
21122 264/300 Read block 3569 len 12288
21123 265/300 Read block 475 len 36864
21124 266/300 Write block 21 len 40960 data: 3 3 3 3 5 4 4 4 3 3
21125 267/300 Write block 3825 len 8192 data: 2 2
21126 268/300 Read block 4608 len 36864
21127 269/300 Write block 1741 len 36864 data: 3 2 2 2 2 2 2 3 4
21128 270/300 Read block 485 len 20480
21129 271/300 Write block 1684 len 36864 data: 4 4 3 3 2 2 2 2 3
21130 272/300 Write block 2831 len 16384 data: 8 6 7 6
21131 273/300 Write block 4190 len 20480 data: 6 6 7 8 8
21132 274/300 Write block 2950 len 12288 data: 4 4 5
21133 275/300 Write block 371 len 32768 data: 3 3 5 5 6 5 7 6
21134 276/300 Read block 285 len 40960
21135 277/300 Read block 4418 len 24576
21136 278/300 Read block 551 len 32768
21137 279/300 Write block 355 len 24576 data: 5 5 6 7 6 6
21138 280/300 Write block 1922 len 36864 data: 6 6 6 4 6 6 6 6 6
21139 281/300 Read block 4755 len 4096
21140 282/300 Read block 964 len 24576
21141 283/300 Write block 3403 len 20480 data: 3 3 3 4 4
21142 284/300 Read block 1864 len 16384
21143 285/300 Read block 2670 len 28672
21144 286/300 Write block 3946 len 24576 data: 6 4 4 4 6 5
21145 287/300 Flush
21146 288/300 Read block 3275 len 32768
21147 289/300 Read block 4048 len 12288
21148 290/300 Read block 3915 len 20480
21149 291/300 Flush
21150 292/300 Write block 3137 len 28672 data: 4 2 3 3 3 3 6
21151 293/300 Write block 3047 len 8192 data: 3 4
21152 294/300 Write block 79 len 24576 data: 6 6 3 3 2 3
21153 295/300 Flush
21154 296/300 Read block 2038 len 4096
21155 297/300 Read block 4193 len 16384
21156 298/300 Flush
21157 299/300 Write block 2466 len 24576 data: 5 3 4 4 5 5
21158 300/300 Write block 308 len 4096 data: 4
21159 301/300 Read block 1418 len 40960
21160 Main task has work to do, go find it
21161 got dsc Start(0)
21162 start 0
21163 Jan 15 05:23:27.593 INFO request completed, latency_us: 177, response_code: 204, uri: /start/cid/0, method: POST, req_id: 40afa0a1-c220-4341-992d-476d58d45594, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
21164 Replay: started 0, returned:Ok(())
21165 ----------------------------------------------------------------
21166 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21167 States: Offline Active Active
21168 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21169 6435 Acked Write 4 Sent Done Done false
21170 6436 Acked Write 6 Sent Done Done false
21171 6437 Acked Read 9 Sent Done Done false
21172 6438 Acked Read 3 Sent Done Done false
21173 6439 Acked Flush 0 Sent Done Done false
21174 6440 Acked Write 10 Sent Done Done false
21175 6441 Acked Read 8 Sent Done Done false
21176 6442 Acked Read 10 Sent Done Done false
21177 6443 Acked Read 8 Sent Done Done false
21178 6444 Acked Read 1 Sent Done Done false
21179 6445 Acked Read 3 Sent Done Done false
21180 6446 Acked Read 9 Sent Done Done false
21181 6447 Acked Read 10 Sent Done Done false
21182 6448 Acked Read 10 Sent Done Done false
21183 6449 Acked Read 7 Sent Done Done false
21184 6450 Acked Write 7 Sent Done Done false
21185 6451 Acked Flush 0 Sent Done Done false
21186 6452 Acked Write 5 Sent Done Done false
21187 6453 Acked Read 5 Sent Done Done false
21188 6454 Acked Read 3 Sent Done Done false
21189 6455 Acked Read 3 Sent Done Done false
21190 6456 Acked Read 6 Sent Done Done false
21191 6457 Acked Read 5 Sent Done Done false
21192 6458 Acked Write 5 Sent Done Done false
21193 6459 Acked Read 5 Sent Done Done false
21194 6460 Acked Read 1 Sent Done Done false
21195 6461 Acked Write 6 Sent Done Done false
21196 6462 Acked Read 1 Sent Done Done false
21197 6463 Acked Read 8 Sent Done Done false
21198 6464 Acked Write 2 Sent Done Done false
21199 6465 Acked Write 2 Sent Done Done false
21200 6466 Acked Read 9 Sent Done Done false
21201 6467 Acked Write 9 Sent Done Done false
21202 6468 Acked Read 6 Sent Done Done false
21203 6469 Acked Read 1 Sent Done Done false
21204 6470 Acked Read 5 Sent Done Done false
21205 6471 Acked Write 10 Sent Done Done false
21206 6472 Acked Read 9 Sent Done Done false
21207 6473 Acked Read 9 Sent Done Done false
21208 6474 Acked Write 3 Sent Done Done false
21209 6475 Acked Write 4 Sent Done Done false
21210 6476 Acked Read 3 Sent Done Done false
21211 6477 Acked Write 8 Sent Done Done false
21212 6478 Acked Read 8 Sent Done Done false
21213 6479 Acked Write 10 Sent Done Done false
21214 6480 Acked Read 7 Sent Done Done false
21215 6481 Acked Read 8 Sent Done Done false
21216 6482 Acked Read 8 Sent Done Done false
21217 6483 Acked Read 4 Sent Done Done false
21218 6484 Acked Read 8 Sent Done Done false
21219 6485 Acked Read 5 Sent Done Done false
21220 6486 Acked Read 10 Sent Done Done false
21221 6487 Acked Write 2 Sent Done Done false
21222 6488 Acked Write 4 Sent Done Done false
21223 6489 Acked Write 2 Sent Done Done false
21224 6490 Acked Read 6 Sent Done Done false
21225 6491 Acked Write 9 Sent Done Done false
21226 6492 Acked Write 8 Sent Done Done false
21227 6493 Acked Write 2 Sent Done Done false
21228 6494 Acked Read 2 Sent Done Done false
21229 6495 Acked Read 8 Sent Done Done false
21230 6496 Acked Read 5 Sent Done Done false
21231 6497 Acked Flush 0 Sent Done Done false
21232 6498 Acked Write 8 Sent Done Done false
21233 6499 Acked Flush 0 Sent Done Done false
21234 6500 Acked Write 3 Sent Done Done false
21235 6501 Acked Read 2 Sent Done Done false
21236 6502 Acked Read 2 Sent Done Done false
21237 6503 Acked Write 1 Sent Done Done false
21238 6504 Acked Write 4 Sent Done Done false
21239 6505 Acked Write 6 Sent Done Done false
21240 6506 Acked Read 7 Sent Done Done false
21241 6507 Acked Write 7 Sent Done Done false
21242 6508 Acked Write 7 Sent Done Done false
21243 6509 Acked Write 2 Sent Done Done false
21244 6510 Acked Read 5 Sent Done Done false
21245 6511 Acked Write 3 Sent Done Done false
21246 6512 Acked Read 6 Sent Done Done false
21247 6513 Acked Write 6 Sent Done Done false
21248 6514 Acked Write 5 Sent Done Done false
21249 6515 Acked Read 1 Sent Done Done false
21250 6516 Acked Write 1 Sent Done Done false
21251 6517 Acked Write 9 Sent Done Done false
21252 6518 Acked Read 3 Sent Done Done false
21253 6519 Acked Write 3 Sent Done Done false
21254 6520 Acked Write 8 Sent Done Done false
21255 6521 Acked Read 7 Sent Done Done false
21256 6522 Acked Write 2 Sent Done Done false
21257 6523 Acked Write 6 Sent Done Done false
21258 6524 Acked Write 3 Sent Done Done false
21259 6525 Acked Read 9 Sent Done Done false
21260 6526 Acked Read 7 Sent Done Done false
21261 6527 Acked Read 10 Sent Done Done false
21262 6528 Acked Read 10 Sent Done Done false
21263 6529 Acked Write 9 Sent Done Done false
21264 6530 Acked Flush 0 Sent Done Done false
21265 6531 Acked Read 8 Sent Done Done false
21266 6532 Acked Write 3 Sent Done Done false
21267 6533 Acked Read 10 Sent Done Done false
21268 6534 Acked Write 6 Sent Done Done false
21269 6535 Acked Read 1 Sent Done Done false
21270 6536 Acked Flush 0 Sent Done Done false
21271 6537 Acked Read 4 Sent Done Done false
21272 6538 Acked Read 9 Sent Done Done false
21273 6539 Acked Read 5 Sent Done Done false
21274 6540 Acked Write 3 Sent Done Done false
21275 6541 Acked Flush 0 Sent Done Done false
21276 6542 Acked Read 6 Sent Done Done false
21277 6543 Acked Read 9 Sent Done Done false
21278 6544 Acked Read 1 Sent Done Done false
21279 6545 Acked Read 7 Sent Done Done false
21280 6546 Acked Write 8 Sent Done Done false
21281 6547 Acked Write 2 Sent Done Done false
21282 6548 Acked Write 8 Sent Done Done false
21283 6549 Acked Write 4 Sent Done Done false
21284 6550 Acked Read 4 Sent Done Done false
21285 6551 Acked Flush 0 Sent Done Done false
21286 6552 Acked Write 8 Sent Done Done false
21287 6553 Acked Flush 0 Sent Done Done false
21288 6554 Acked Write 10 Sent Done Done false
21289 6555 Acked Write 4 Sent Done Done false
21290 6556 Acked Write 2 Sent Done Done false
21291 6557 Acked Read 6 Sent Done Done false
21292 6558 Acked Read 3 Sent Done Done false
21293 6559 Acked Read 10 Sent Done Done false
21294 6560 Acked Read 6 Sent Done Done false
21295 6561 Acked Read 10 Sent Done Done false
21296 6562 Acked Read 6 Sent Done Done false
21297 6563 Acked Write 7 Sent Done Done false
21298 6564 Acked Write 7 Sent Done Done false
21299 6565 Acked Read 9 Sent Done Done false
21300 6566 Acked Write 9 Sent Done Done false
21301 6567 Acked Write 10 Sent Done Done false
21302 6568 Acked Read 7 Sent Done Done false
21303 6569 Acked Write 3 Sent Done Done false
21304 6570 Acked Write 6 Sent Done Done false
21305 6571 Acked Write 4 Sent Done Done false
21306 6572 Acked Write 1 Sent Done Done false
21307 6573 Acked Read 9 Sent Done Done false
21308 6574 Acked Read 10 Sent Done Done false
21309 6575 Acked Flush 0 Sent Done Done false
21310 6576 Acked Read 7 Sent Done Done false
21311 6577 Acked Write 8 Sent Done Done false
21312 6578 Acked Write 10 Sent Done Done false
21313 6579 Acked Read 1 Sent Done Done false
21314 6580 Acked Write 6 Sent Done Done false
21315 6581 Acked Write 10 Sent Done Done false
21316 6582 Acked Read 10 Sent Done Done false
21317 6583 Acked Write 6 Sent Done Done false
21318 6584 Acked Write 6 Sent Done Done false
21319 6585 Acked Write 8 Sent Done Done false
21320 6586 Acked Write 3 Sent Done Done false
21321 6587 Acked Write 2 Sent Done Done false
21322 6588 Acked Write 2 Sent Done Done false
21323 6589 Acked Write 6 Sent Done Done false
21324 6590 Acked Flush 0 Sent Done Done false
21325 6591 Acked Read 1 Sent Done Done false
21326 6592 Acked Write 9 Sent Done Done false
21327 6593 Acked Read 2 Sent Done Done false
21328 6594 Acked Flush 0 Sent Done Done false
21329 6595 Acked Write 1 Sent Done Done false
21330 6596 Acked Write 8 Sent Done Done false
21331 6597 Acked Write 10 Sent Done Done false
21332 6598 Acked Write 4 Sent Done Done false
21333 6599 Acked Write 1 Sent Done Done false
21334 6600 Acked Write 9 Sent Done Done false
21335 6601 Acked Read 9 Sent Done Done false
21336 6602 Acked Read 6 Sent Done Done false
21337 6603 Acked Write 10 Sent Done Done false
21338 6604 Acked Write 2 Sent Done Done false
21339 6605 Acked Read 8 Sent Done Done false
21340 6606 Acked Read 9 Sent Done Done false
21341 6607 Acked Write 7 Sent Done Done false
21342 6608 Acked Write 7 Sent Done Done false
21343 6609 Acked Read 8 Sent Done Done false
21344 6610 Acked Flush 0 Sent Done Done false
21345 6611 Acked Write 6 Sent Done Done false
21346 6612 Acked Read 6 Sent Done Done false
21347 6613 Acked Write 10 Sent Done Done false
21348 6614 Acked Write 8 Sent Done Done false
21349 6615 Acked Write 2 Sent Done Done false
21350 6616 Acked Write 9 Sent Done Done false
21351 6617 Acked Read 3 Sent Done Done false
21352 6618 Acked Read 3 Sent Done Done false
21353 6619 Acked Write 10 Sent Done Done false
21354 6620 Acked Write 3 Sent Done Done false
21355 6621 Acked Read 8 Sent Done Done false
21356 6622 Acked Flush 0 Sent Done Done false
21357 6623 Acked Write 9 Sent Done Done false
21358 6624 Acked Write 7 Sent Done Done false
21359 6625 Acked Read 5 Sent Done Done false
21360 6626 Acked Write 9 Sent Done Done false
21361 6627 Acked Read 9 Sent Done Done false
21362 6628 Acked Read 6 Sent Done Done false
21363 6629 Acked Read 5 Sent Done Done false
21364 6630 Acked Read 1 Sent Done Done false
21365 6631 Acked Write 7 Sent Done Done false
21366 6632 Acked Write 7 Sent Done Done false
21367 6633 Acked Write 7 Sent Done Done false
21368 6634 Acked Write 9 Sent Done Done false
21369 6635 Acked Read 7 Sent Done Done false
21370 6636 Acked Write 5 Sent Done Done false
21371 6637 Acked Write 6 Sent Done Done false
21372 6638 Acked Read 8 Sent Done Done false
21373 6639 Acked Read 8 Sent Done Done false
21374 6640 Acked Write 3 Sent Done Done false
21375 6641 Acked Write 6 Sent Done Done false
21376 6642 Acked Flush 0 Sent Done Done false
21377 6643 Acked Read 1 Sent Done Done false
21378 6644 Acked Read 6 Sent Done Done false
21379 6645 Acked Flush 0 Sent Done Done false
21380 6646 Acked Write 9 Sent Done Done false
21381 6647 Acked Write 9 Sent Done Done false
21382 6648 Acked Write 2 Sent Done Done false
21383 6649 Acked Read 3 Sent Done Done false
21384 6650 Acked Write 3 Sent Done Done false
21385 6651 Acked Read 1 Sent Done Done false
21386 6652 Acked Read 10 Sent Done Done false
21387 6653 Acked Write 3 Sent Done Done false
21388 6654 Acked Read 3 Sent Done Done false
21389 6655 Acked Flush 0 Sent Done Done false
21390 6656 Acked Read 5 Sent Done Done false
21391 6657 Acked Read 1 Sent Done Done false
21392 6658 Acked Read 2 Sent Done Done false
21393 6659 Acked Read 7 Sent Done Done false
21394 6660 Acked Write 9 Sent Done Done false
21395 6661 Acked Read 1 Sent Done Done false
21396 6662 Acked Read 1 Sent Done Done false
21397 6663 Acked Read 10 Sent Done Done false
21398 6664 Acked Flush 0 Sent Done Done false
21399 6665 Acked Read 1 Sent Done Done false
21400 6666 Acked Read 3 Sent Done Done false
21401 6667 Acked Read 3 Sent Done Done false
21402 6668 Acked Write 9 Sent Done Done false
21403 6669 Acked Flush 0 Sent Done Done false
21404 6670 Acked Write 7 Sent Done Done false
21405 6671 Acked Read 2 Sent Done Done false
21406 6672 Acked Read 7 Sent Done Done false
21407 6673 Acked Write 1 Sent Done Done false
21408 6674 Acked Write 7 Sent Done Done false
21409 6675 Acked Read 7 Sent Done Done false
21410 6676 Acked Write 1 Sent Done Done false
21411 6677 Acked Read 3 Sent Done Done false
21412 6678 Acked Read 7 Sent Done Done false
21413 6679 Acked Read 10 Sent Done Done false
21414 6680 Acked Read 6 Sent Done Done false
21415 6681 Acked Write 5 Sent Done Done false
21416 6682 Acked Read 9 Sent Done Done false
21417 6683 Acked Write 2 Sent Done Done false
21418 6684 Acked Read 3 Sent Done Done false
21419 6685 Acked Read 4 Sent Done Done false
21420 6686 Acked Read 2 Sent Done Done false
21421 6687 Acked Read 1 Sent Done Done false
21422 6688 Acked Write 1 Sent Done Done false
21423 6689 Acked Write 7 Sent Done Done false
21424 6690 Acked Write 6 Sent Done Done false
21425 6691 Acked Read 8 Sent Done Done false
21426 6692 Acked Read 1 Sent Done Done false
21427 6693 Acked Read 9 Sent Done Done false
21428 6694 Acked Read 2 Sent Done Done false
21429 6695 Acked Read 3 Sent Done Done false
21430 6696 Acked Write 2 Sent Done Done false
21431 6697 Acked Read 5 Sent Done Done false
21432 6698 Acked Read 3 Sent Done Done false
21433 6699 Acked Read 9 Sent Done Done false
21434 6700 Acked Write 10 Sent Done Done false
21435 6701 Acked Write 2 Sent Done Done false
21436 6702 Acked Read 9 Sent Done Done false
21437 6703 Acked Write 9 Sent Done Done false
21438 6704 Acked Read 5 Sent Done Done false
21439 6705 Acked Write 9 Sent Done Done false
21440 6706 Acked Write 4 Sent Done Done false
21441 6707 Acked Write 5 Sent Done Done false
21442 6708 Acked Write 3 Sent Done Done false
21443 6709 Acked Write 8 Sent Done Done false
21444 6710 Acked Read 10 Sent Done Done false
21445 6711 Acked Read 6 Sent Done Done false
21446 6712 Acked Read 8 Sent Done Done false
21447 6713 Acked Write 6 Sent Done Done false
21448 6714 Acked Write 9 Sent Done Done false
21449 6715 Acked Read 1 Sent Done Done false
21450 6716 Acked Read 6 Sent Done Done false
21451 6717 Acked Write 5 Sent Done Done false
21452 6718 Acked Read 4 Sent Done Done false
21453 6719 Acked Read 7 Sent Done Done false
21454 6720 Acked Write 6 Sent Done Done false
21455 6721 Acked Flush 0 Sent Done Done false
21456 6722 Acked Read 8 Sent Done Done false
21457 6723 Acked Read 3 Sent Done Done false
21458 6724 Acked Read 5 Sent Done Done false
21459 6725 Acked Flush 0 Sent Done Done false
21460 6726 Acked Write 7 Sent Done Done false
21461 6727 Acked Write 2 Sent Done Done false
21462 6728 Acked Write 6 Sent Done Done false
21463 6729 Acked Flush 0 Sent Done Done false
21464 6730 Acked Read 1 Sent Done Done false
21465 6731 Acked Read 4 Sent Done Done false
21466 6732 Acked Flush 0 Sent Done Done false
21467 6733 Acked Write 6 Sent Done Done false
21468 6734 Acked Write 1 Sent Done Done false
21469 6735 Acked Read 10 Sent Done Done false
21470 STATES DS:0 DS:1 DS:2 TOTAL
21471 Sent 301 0 0 301
21472 Done 0 301 301 602
21473 Skipped 0 0 0 0
21474 Error 0 0 0 0
21475 Last Flush: 6434 6732 6732
21476 Downstairs last five completed: 6434 6433 6432 6431 6430
21477 Upstairs last five completed: 6735 6734 6733 6732 6731
21478 CLIENT: Up:0 ds:301 act:2
21479 [8810] got start action
21480 [8810] I am going to restart
21481 Starting downstairs at port 8810
21482 Make output file at "/tmp/dsc/downstairs-8810.txt"
21483 [8810][0] reports Starting
21484 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1221)
21485 [8810][0] reports Running
21486 ----------------------------------------------------------------
21487 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21488 States: Offline Active Active
21489 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21490 6435 Acked Write 4 Sent Done Done false
21491 6436 Acked Write 6 Sent Done Done false
21492 6437 Acked Read 9 Sent Done Done false
21493 6438 Acked Read 3 Sent Done Done false
21494 6439 Acked Flush 0 Sent Done Done false
21495 6440 Acked Write 10 Sent Done Done false
21496 6441 Acked Read 8 Sent Done Done false
21497 6442 Acked Read 10 Sent Done Done false
21498 6443 Acked Read 8 Sent Done Done false
21499 6444 Acked Read 1 Sent Done Done false
21500 6445 Acked Read 3 Sent Done Done false
21501 6446 Acked Read 9 Sent Done Done false
21502 6447 Acked Read 10 Sent Done Done false
21503 6448 Acked Read 10 Sent Done Done false
21504 6449 Acked Read 7 Sent Done Done false
21505 6450 Acked Write 7 Sent Done Done false
21506 6451 Acked Flush 0 Sent Done Done false
21507 6452 Acked Write 5 Sent Done Done false
21508 6453 Acked Read 5 Sent Done Done false
21509 6454 Acked Read 3 Sent Done Done false
21510 6455 Acked Read 3 Sent Done Done false
21511 6456 Acked Read 6 Sent Done Done false
21512 6457 Acked Read 5 Sent Done Done false
21513 6458 Acked Write 5 Sent Done Done false
21514 6459 Acked Read 5 Sent Done Done false
21515 6460 Acked Read 1 Sent Done Done false
21516 6461 Acked Write 6 Sent Done Done false
21517 6462 Acked Read 1 Sent Done Done false
21518 6463 Acked Read 8 Sent Done Done false
21519 6464 Acked Write 2 Sent Done Done false
21520 6465 Acked Write 2 Sent Done Done false
21521 6466 Acked Read 9 Sent Done Done false
21522 6467 Acked Write 9 Sent Done Done false
21523 6468 Acked Read 6 Sent Done Done false
21524 6469 Acked Read 1 Sent Done Done false
21525 6470 Acked Read 5 Sent Done Done false
21526 6471 Acked Write 10 Sent Done Done false
21527 6472 Acked Read 9 Sent Done Done false
21528 6473 Acked Read 9 Sent Done Done false
21529 6474 Acked Write 3 Sent Done Done false
21530 6475 Acked Write 4 Sent Done Done false
21531 6476 Acked Read 3 Sent Done Done false
21532 6477 Acked Write 8 Sent Done Done false
21533 6478 Acked Read 8 Sent Done Done false
21534 6479 Acked Write 10 Sent Done Done false
21535 6480 Acked Read 7 Sent Done Done false
21536 6481 Acked Read 8 Sent Done Done false
21537 6482 Acked Read 8 Sent Done Done false
21538 6483 Acked Read 4 Sent Done Done false
21539 6484 Acked Read 8 Sent Done Done false
21540 6485 Acked Read 5 Sent Done Done false
21541 6486 Acked Read 10 Sent Done Done false
21542 6487 Acked Write 2 Sent Done Done false
21543 6488 Acked Write 4 Sent Done Done false
21544 6489 Acked Write 2 Sent Done Done false
21545 6490 Acked Read 6 Sent Done Done false
21546 6491 Acked Write 9 Sent Done Done false
21547 6492 Acked Write 8 Sent Done Done false
21548 6493 Acked Write 2 Sent Done Done false
21549 6494 Acked Read 2 Sent Done Done false
21550 6495 Acked Read 8 Sent Done Done false
21551 6496 Acked Read 5 Sent Done Done false
21552 6497 Acked Flush 0 Sent Done Done false
21553 6498 Acked Write 8 Sent Done Done false
21554 6499 Acked Flush 0 Sent Done Done false
21555 6500 Acked Write 3 Sent Done Done false
21556 6501 Acked Read 2 Sent Done Done false
21557 6502 Acked Read 2 Sent Done Done false
21558 6503 Acked Write 1 Sent Done Done false
21559 6504 Acked Write 4 Sent Done Done false
21560 6505 Acked Write 6 Sent Done Done false
21561 6506 Acked Read 7 Sent Done Done false
21562 6507 Acked Write 7 Sent Done Done false
21563 6508 Acked Write 7 Sent Done Done false
21564 6509 Acked Write 2 Sent Done Done false
21565 6510 Acked Read 5 Sent Done Done false
21566 6511 Acked Write 3 Sent Done Done false
21567 6512 Acked Read 6 Sent Done Done false
21568 6513 Acked Write 6 Sent Done Done false
21569 6514 Acked Write 5 Sent Done Done false
21570 6515 Acked Read 1 Sent Done Done false
21571 6516 Acked Write 1 Sent Done Done false
21572 6517 Acked Write 9 Sent Done Done false
21573 6518 Acked Read 3 Sent Done Done false
21574 6519 Acked Write 3 Sent Done Done false
21575 6520 Acked Write 8 Sent Done Done false
21576 6521 Acked Read 7 Sent Done Done false
21577 6522 Acked Write 2 Sent Done Done false
21578 6523 Acked Write 6 Sent Done Done false
21579 6524 Acked Write 3 Sent Done Done false
21580 6525 Acked Read 9 Sent Done Done false
21581 6526 Acked Read 7 Sent Done Done false
21582 6527 Acked Read 10 Sent Done Done false
21583 6528 Acked Read 10 Sent Done Done false
21584 6529 Acked Write 9 Sent Done Done false
21585 6530 Acked Flush 0 Sent Done Done false
21586 6531 Acked Read 8 Sent Done Done false
21587 6532 Acked Write 3 Sent Done Done false
21588 6533 Acked Read 10 Sent Done Done false
21589 6534 Acked Write 6 Sent Done Done false
21590 6535 Acked Read 1 Sent Done Done false
21591 6536 Acked Flush 0 Sent Done Done false
21592 6537 Acked Read 4 Sent Done Done false
21593 6538 Acked Read 9 Sent Done Done false
21594 6539 Acked Read 5 Sent Done Done false
21595 6540 Acked Write 3 Sent Done Done false
21596 6541 Acked Flush 0 Sent Done Done false
21597 6542 Acked Read 6 Sent Done Done false
21598 6543 Acked Read 9 Sent Done Done false
21599 6544 Acked Read 1 Sent Done Done false
21600 6545 Acked Read 7 Sent Done Done false
21601 6546 Acked Write 8 Sent Done Done false
21602 6547 Acked Write 2 Sent Done Done false
21603 6548 Acked Write 8 Sent Done Done false
21604 6549 Acked Write 4 Sent Done Done false
21605 6550 Acked Read 4 Sent Done Done false
21606 6551 Acked Flush 0 Sent Done Done false
21607 6552 Acked Write 8 Sent Done Done false
21608 6553 Acked Flush 0 Sent Done Done false
21609 6554 Acked Write 10 Sent Done Done false
21610 6555 Acked Write 4 Sent Done Done false
21611 6556 Acked Write 2 Sent Done Done false
21612 6557 Acked Read 6 Sent Done Done false
21613 6558 Acked Read 3 Sent Done Done false
21614 6559 Acked Read 10 Sent Done Done false
21615 6560 Acked Read 6 Sent Done Done false
21616 6561 Acked Read 10 Sent Done Done false
21617 6562 Acked Read 6 Sent Done Done false
21618 6563 Acked Write 7 Sent Done Done false
21619 6564 Acked Write 7 Sent Done Done false
21620 6565 Acked Read 9 Sent Done Done false
21621 6566 Acked Write 9 Sent Done Done false
21622 6567 Acked Write 10 Sent Done Done false
21623 6568 Acked Read 7 Sent Done Done false
21624 6569 Acked Write 3 Sent Done Done false
21625 6570 Acked Write 6 Sent Done Done false
21626 6571 Acked Write 4 Sent Done Done false
21627 6572 Acked Write 1 Sent Done Done false
21628 6573 Acked Read 9 Sent Done Done false
21629 6574 Acked Read 10 Sent Done Done false
21630 6575 Acked Flush 0 Sent Done Done false
21631 6576 Acked Read 7 Sent Done Done false
21632 6577 Acked Write 8 Sent Done Done false
21633 6578 Acked Write 10 Sent Done Done false
21634 6579 Acked Read 1 Sent Done Done false
21635 6580 Acked Write 6 Sent Done Done false
21636 6581 Acked Write 10 Sent Done Done false
21637 6582 Acked Read 10 Sent Done Done false
21638 6583 Acked Write 6 Sent Done Done false
21639 6584 Acked Write 6 Sent Done Done false
21640 6585 Acked Write 8 Sent Done Done false
21641 6586 Acked Write 3 Sent Done Done false
21642 6587 Acked Write 2 Sent Done Done false
21643 6588 Acked Write 2 Sent Done Done false
21644 6589 Acked Write 6 Sent Done Done false
21645 6590 Acked Flush 0 Sent Done Done false
21646 6591 Acked Read 1 Sent Done Done false
21647 6592 Acked Write 9 Sent Done Done false
21648 6593 Acked Read 2 Sent Done Done false
21649 6594 Acked Flush 0 Sent Done Done false
21650 6595 Acked Write 1 Sent Done Done false
21651 6596 Acked Write 8 Sent Done Done false
21652 6597 Acked Write 10 Sent Done Done false
21653 6598 Acked Write 4 Sent Done Done false
21654 6599 Acked Write 1 Sent Done Done false
21655 6600 Acked Write 9 Sent Done Done false
21656 6601 Acked Read 9 Sent Done Done false
21657 6602 Acked Read 6 Sent Done Done false
21658 6603 Acked Write 10 Sent Done Done false
21659 6604 Acked Write 2 Sent Done Done false
21660 6605 Acked Read 8 Sent Done Done false
21661 6606 Acked Read 9 Sent Done Done false
21662 6607 Acked Write 7 Sent Done Done false
21663 6608 Acked Write 7 Sent Done Done false
21664 6609 Acked Read 8 Sent Done Done false
21665 6610 Acked Flush 0 Sent Done Done false
21666 6611 Acked Write 6 Sent Done Done false
21667 6612 Acked Read 6 Sent Done Done false
21668 6613 Acked Write 10 Sent Done Done false
21669 6614 Acked Write 8 Sent Done Done false
21670 6615 Acked Write 2 Sent Done Done false
21671 6616 Acked Write 9 Sent Done Done false
21672 6617 Acked Read 3 Sent Done Done false
21673 6618 Acked Read 3 Sent Done Done false
21674 6619 Acked Write 10 Sent Done Done false
21675 6620 Acked Write 3 Sent Done Done false
21676 6621 Acked Read 8 Sent Done Done false
21677 6622 Acked Flush 0 Sent Done Done false
21678 6623 Acked Write 9 Sent Done Done false
21679 6624 Acked Write 7 Sent Done Done false
21680 6625 Acked Read 5 Sent Done Done false
21681 6626 Acked Write 9 Sent Done Done false
21682 6627 Acked Read 9 Sent Done Done false
21683 6628 Acked Read 6 Sent Done Done false
21684 6629 Acked Read 5 Sent Done Done false
21685 6630 Acked Read 1 Sent Done Done false
21686 6631 Acked Write 7 Sent Done Done false
21687 6632 Acked Write 7 Sent Done Done false
21688 6633 Acked Write 7 Sent Done Done false
21689 6634 Acked Write 9 Sent Done Done false
21690 6635 Acked Read 7 Sent Done Done false
21691 6636 Acked Write 5 Sent Done Done false
21692 6637 Acked Write 6 Sent Done Done false
21693 6638 Acked Read 8 Sent Done Done false
21694 6639 Acked Read 8 Sent Done Done false
21695 6640 Acked Write 3 Sent Done Done false
21696 6641 Acked Write 6 Sent Done Done false
21697 6642 Acked Flush 0 Sent Done Done false
21698 6643 Acked Read 1 Sent Done Done false
21699 6644 Acked Read 6 Sent Done Done false
21700 6645 Acked Flush 0 Sent Done Done false
21701 6646 Acked Write 9 Sent Done Done false
21702 6647 Acked Write 9 Sent Done Done false
21703 6648 Acked Write 2 Sent Done Done false
21704 6649 Acked Read 3 Sent Done Done false
21705 6650 Acked Write 3 Sent Done Done false
21706 6651 Acked Read 1 Sent Done Done false
21707 6652 Acked Read 10 Sent Done Done false
21708 6653 Acked Write 3 Sent Done Done false
21709 6654 Acked Read 3 Sent Done Done false
21710 6655 Acked Flush 0 Sent Done Done false
21711 6656 Acked Read 5 Sent Done Done false
21712 6657 Acked Read 1 Sent Done Done false
21713 6658 Acked Read 2 Sent Done Done false
21714 6659 Acked Read 7 Sent Done Done false
21715 6660 Acked Write 9 Sent Done Done false
21716 6661 Acked Read 1 Sent Done Done false
21717 6662 Acked Read 1 Sent Done Done false
21718 6663 Acked Read 10 Sent Done Done false
21719 6664 Acked Flush 0 Sent Done Done false
21720 6665 Acked Read 1 Sent Done Done false
21721 6666 Acked Read 3 Sent Done Done false
21722 6667 Acked Read 3 Sent Done Done false
21723 6668 Acked Write 9 Sent Done Done false
21724 6669 Acked Flush 0 Sent Done Done false
21725 6670 Acked Write 7 Sent Done Done false
21726 6671 Acked Read 2 Sent Done Done false
21727 6672 Acked Read 7 Sent Done Done false
21728 6673 Acked Write 1 Sent Done Done false
21729 6674 Acked Write 7 Sent Done Done false
21730 6675 Acked Read 7 Sent Done Done false
21731 6676 Acked Write 1 Sent Done Done false
21732 6677 Acked Read 3 Sent Done Done false
21733 6678 Acked Read 7 Sent Done Done false
21734 6679 Acked Read 10 Sent Done Done false
21735 6680 Acked Read 6 Sent Done Done false
21736 6681 Acked Write 5 Sent Done Done false
21737 6682 Acked Read 9 Sent Done Done false
21738 6683 Acked Write 2 Sent Done Done false
21739 6684 Acked Read 3 Sent Done Done false
21740 6685 Acked Read 4 Sent Done Done false
21741 6686 Acked Read 2 Sent Done Done false
21742 6687 Acked Read 1 Sent Done Done false
21743 6688 Acked Write 1 Sent Done Done false
21744 6689 Acked Write 7 Sent Done Done false
21745 6690 Acked Write 6 Sent Done Done false
21746 6691 Acked Read 8 Sent Done Done false
21747 6692 Acked Read 1 Sent Done Done false
21748 6693 Acked Read 9 Sent Done Done false
21749 6694 Acked Read 2 Sent Done Done false
21750 6695 Acked Read 3 Sent Done Done false
21751 6696 Acked Write 2 Sent Done Done false
21752 6697 Acked Read 5 Sent Done Done false
21753 6698 Acked Read 3 Sent Done Done false
21754 6699 Acked Read 9 Sent Done Done false
21755 6700 Acked Write 10 Sent Done Done false
21756 6701 Acked Write 2 Sent Done Done false
21757 6702 Acked Read 9 Sent Done Done false
21758 6703 Acked Write 9 Sent Done Done false
21759 6704 Acked Read 5 Sent Done Done false
21760 6705 Acked Write 9 Sent Done Done false
21761 6706 Acked Write 4 Sent Done Done false
21762 6707 Acked Write 5 Sent Done Done false
21763 6708 Acked Write 3 Sent Done Done false
21764 6709 Acked Write 8 Sent Done Done false
21765 6710 Acked Read 10 Sent Done Done false
21766 6711 Acked Read 6 Sent Done Done false
21767 6712 Acked Read 8 Sent Done Done false
21768 6713 Acked Write 6 Sent Done Done false
21769 6714 Acked Write 9 Sent Done Done false
21770 6715 Acked Read 1 Sent Done Done false
21771 6716 Acked Read 6 Sent Done Done false
21772 6717 Acked Write 5 Sent Done Done false
21773 6718 Acked Read 4 Sent Done Done false
21774 6719 Acked Read 7 Sent Done Done false
21775 6720 Acked Write 6 Sent Done Done false
21776 6721 Acked Flush 0 Sent Done Done false
21777 6722 Acked Read 8 Sent Done Done false
21778 6723 Acked Read 3 Sent Done Done false
21779 6724 Acked Read 5 Sent Done Done false
21780 6725 Acked Flush 0 Sent Done Done false
21781 6726 Acked Write 7 Sent Done Done false
21782 6727 Acked Write 2 Sent Done Done false
21783 6728 Acked Write 6 Sent Done Done false
21784 6729 Acked Flush 0 Sent Done Done false
21785 6730 Acked Read 1 Sent Done Done false
21786 6731 Acked Read 4 Sent Done Done false
21787 6732 Acked Flush 0 Sent Done Done false
21788 6733 Acked Write 6 Sent Done Done false
21789 6734 Acked Write 1 Sent Done Done false
21790 6735 Acked Read 10 Sent Done Done false
21791 6736 Acked Flush 0 Sent Done Done false
21792 STATES DS:0 DS:1 DS:2 TOTAL
21793 Sent 302 0 0 302
21794 Done 0 302 302 604
21795 Skipped 0 0 0 0
21796 Error 0 0 0 0
21797 Last Flush: 6434 6736 6736
21798 Downstairs last five completed: 6434 6433 6432 6431 6430
21799 Upstairs last five completed: 6736 6735 6734 6733 6732
21800 CLIENT: Up:0 ds:302 act:2
218012025-01-15T05:23:33.357ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218022025-01-15T05:23:33.357ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59171) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218032025-01-15T05:23:33.358ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218042025-01-15T05:23:33.359ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218052025-01-15T05:23:33.359ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218062025-01-15T05:23:33.359ZINFOcrucible: send last flush ID to this DS: 6434 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218072025-01-15T05:23:33.359ZINFOcrucible: Replied this last flush ID: 6434 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218082025-01-15T05:23:33.359ZINFOcrucible: [0] client re-new 302 jobs since flush 6434 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218092025-01-15T05:23:33.359ZINFOcrucible: [0] Replayed 302 jobs since flush: 6434 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
21810 ----------------------------------------------------------------
21811 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21812 Downstairs last five completed: 6736 6735 6734 6733 6732
21813 Upstairs last five completed: 6736 6735 6734 6733 6732
21814 CLIENT: Up:0 ds:0 act:3
21815 Replay: All jobs finished
21816 Main task has work to do, go find it
21817 got dsc Stop(0)
21818 stop 0
21819 [8810] Got stop action so:false kr:false
21820 Jan 15 05:23:35.598 INFO request completed, latency_us: 293, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 99ff82b9-2268-44d6-b4e6-4614f25e039e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
218212025-01-15T05:23:35.599ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218222025-01-15T05:23:35.599ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
218232025-01-15T05:23:35.599ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
21824 Jan 15 05:23:35.599 {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:23:35.599433589Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
21825 INFO request completed, latency_us: 127, response_code: 200, uri: /state/cid/0, method: GET, req_id: 250df227-8a3a-45ac-9882-2a5e01f30a17, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
21826 [8810] Exited with: ExitStatus(unix_wait_status(9))
21827 [8810][0] reports Exit
21828 Jan 15 05:23:39.601 INFO request completed, latency_us: 235, response_code: 200, uri: /state/cid/0, method: GET, req_id: 5be45af8-3b7d-4281-875d-e8a8b9e11556, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
21829 001/300 Write block 1111 len 20480 data: 4 4 3 2 2
21830 002/300 Write block 4131 len 28672 data: 6 7 4 5 5 4 6
21831 003/300 Read block 4328 len 8192
21832 004/300 Write block 1500 len 24576 data: 4 5 6 6 5 5
21833 005/300 Read block 3477 len 8192
21834 006/300 Write block 2444 len 4096 data: 3
21835 007/300 Write block 4606 len 8192 data: 3 4
21836 008/300 Write block 2455 len 36864 data: 4 5 4 4 5 6 6 6 6
21837 009/300 Write block 382 len 4096 data: 8
21838 010/300 Read block 3762 len 8192
21839 011/300 Flush
21840 012/300 Read block 237 len 16384
21841 013/300 Read block 4664 len 28672
21842 014/300 Read block 3683 len 40960
21843 015/300 Flush
21844 016/300 Write block 1085 len 20480 data: 5 4 5 4 3
21845 017/300 Write block 183 len 24576 data: 4 4 4 4 4 4
21846 018/300 Write block 2644 len 12288 data: 5 5 5
21847 019/300 Write block 71 len 12288 data: 4 4 3
21848 020/300 Write block 4177 len 20480 data: 2 2 3 3 3
21849 021/300 Write block 4178 len 20480 data: 3 4 4 4 4
21850 022/300 Write block 3086 len 36864 data: 2 4 5 6 7 7 8 7 6
21851 023/300 Read block 1196 len 40960
21852 024/300 Write block 2298 len 32768 data: 5 5 5 5 6 6 3 3
21853 025/300 Read block 3407 len 8192
21854 026/300 Flush
21855 027/300 Write block 1960 len 28672 data: 2 3 3 3 3 3 3
21856 028/300 Read block 1226 len 4096
21857 029/300 Write block 2088 len 24576 data: 4 4 4 4 4 4
21858 030/300 Write block 1026 len 28672 data: 2 2 2 2 2 2 2
21859 031/300 Write block 1729 len 32768 data: 3 3 3 3 3 4 3 3
21860 032/300 Write block 2847 len 12288 data: 2 2 2
21861 033/300 Read block 2419 len 8192
21862 034/300 Write block 3774 len 32768 data: 4 3 3 3 3 4 5 5
21863 035/300 Write block 3300 len 24576 data: 6 6 6 8 8 8
21864 036/300 Write block 552 len 16384 data: 5 4 5 4
21865 037/300 Write block 3326 len 16384 data: 4 6 4 3
21866 038/300 Write block 1514 len 28672 data: 6 6 5 3 2 2 2
21867 039/300 Read block 700 len 40960
21868 040/300 Flush
21869 041/300 Write block 2260 len 28672 data: 5 5 5 5 4 4 3
21870 042/300 Read block 2859 len 8192
21871 043/300 Write block 78 len 28672 data: 6 7 7 4 4 3 4
21872 044/300 Write block 4963 len 36864 data: 5 5 5 5 5 5 4 4 3
21873 045/300 Write block 1478 len 20480 data: 4 5 5 4 5
21874 046/300 Write block 987 len 24576 data: 2 2 2 2 2 3
21875 047/300 Write block 601 len 28672 data: 3 3 2 3 3 3 3
21876 048/300 Write block 4240 len 40960 data: 7 10 9 8 8 8 9 7 7 6
21877 049/300 Write block 1958 len 20480 data: 4 3 3 4 4
21878 050/300 Read block 970 len 36864
21879 051/300 Read block 2429 len 12288
21880 052/300 Read block 1046 len 28672
21881 053/300 Read block 497 len 16384
21882 054/300 Read block 2735 len 28672
21883 055/300 Write block 2652 len 36864 data: 3 3 4 5 5 6 6 5 4
21884 056/300 Read block 2576 len 24576
21885 057/300 Read block 1098 len 28672
21886 058/300 Flush
21887 059/300 Read block 2163 len 12288
21888 060/300 Flush
21889 061/300 Read block 494 len 4096
21890 062/300 Read block 3400 len 16384
21891 063/300 Read block 90 len 24576
21892 064/300 Write block 3468 len 36864 data: 4 4 5 4 6 7 7 6 5
21893 065/300 Read block 1499 len 40960
21894 066/300 Read block 525 len 16384
21895 067/300 Read block 2047 len 20480
21896 068/300 Read block 2766 len 36864
21897 069/300 Read block 164 len 20480
21898 070/300 Write block 2019 len 28672 data: 7 6 6 8 7 8 7
21899 071/300 Read block 4010 len 8192
21900 072/300 Write block 4349 len 4096 data: 6
21901 073/300 Read block 1137 len 24576
21902 074/300 Write block 2162 len 32768 data: 3 4 4 2 3 4 4 4
21903 075/300 Write block 600 len 12288 data: 4 4 4
21904 076/300 Read block 4258 len 4096
21905 077/300 Read block 580 len 8192
21906 078/300 Write block 2100 len 32768 data: 3 3 3 3 3 4 4 4
21907 079/300 Read block 790 len 8192
21908 080/300 Write block 423 len 32768 data: 2 2 2 2 3 3 4 3
21909 081/300 Flush
21910 082/300 Write block 272 len 12288 data: 4 3 4
21911 083/300 Write block 309 len 20480 data: 4 5 5 4 3
21912 084/300 Read block 4913 len 8192
21913 085/300 Read block 1914 len 40960
21914 086/300 Read block 4528 len 20480
21915 087/300 Write block 4284 len 36864 data: 4 5 5 6 7 7 7 5 5
21916 088/300 Write block 4742 len 24576 data: 7 10 11 9 7 6
21917 089/300 Read block 468 len 20480
21918 090/300 Read block 1627 len 24576
21919 091/300 Read block 2779 len 28672
21920 092/300 Flush
21921 093/300 Write block 2945 len 32768 data: 5 6 5 5 4 5 5 6
21922 094/300 Read block 4747 len 32768
21923 095/300 Write block 4536 len 4096 data: 2
21924 096/300 Read block 4277 len 8192
21925 097/300 Write block 3707 len 16384 data: 3 3 3 3
21926 098/300 Write block 985 len 28672 data: 3 3 3 3 3 3 3
21927 099/300 Write block 2505 len 36864 data: 6 4 5 6 5 4 4 4 5
21928 100/300 Read block 727 len 20480
21929 101/300 Write block 3827 len 32768 data: 2 2 2 2 2 3 4 5
21930 102/300 Flush
21931 103/300 Write block 3085 len 28672 data: 2 3 5 6 7 8 8
21932 104/300 Flush
21933 105/300 Write block 4286 len 36864 data: 6 7 8 8 8 6 6 4 4
21934 106/300 Write block 2302 len 32768 data: 7 7 4 4 3 3 3 3
21935 107/300 Read block 1523 len 24576
21936 108/300 Read block 441 len 16384
21937 109/300 Read block 2617 len 4096
21938 110/300 Write block 401 len 20480 data: 4 4 4 4 3
21939 111/300 Write block 3213 len 8192 data: 3 3
21940 112/300 Write block 1404 len 4096 data: 4
21941 113/300 Read block 2414 len 40960
21942 114/300 Write block 1920 len 32768 data: 6 6 7 7 7 5 7 7
21943 115/300 Flush
21944 116/300 Flush
21945 117/300 Write block 2224 len 32768 data: 4 4 6 6 6 5 4 3
21946 118/300 Flush
21947 119/300 Read block 1154 len 4096
21948 120/300 Flush
21949 121/300 Read block 650 len 20480
21950 122/300 Read block 3898 len 16384
21951 123/300 Write block 3351 len 24576 data: 5 4 5 3 3 3
21952 124/300 Read block 3998 len 36864
21953 125/300 Write block 717 len 24576 data: 2 2 2 3 3 3
21954 126/300 Write block 2433 len 36864 data: 7 6 3 3 3 4 6 6 5
21955 127/300 Write block 3196 len 32768 data: 4 4 3 4 4 5 5 4
21956 128/300 Write block 2433 len 36864 data: 8 7 4 4 4 5 7 7 6
21957 129/300 Flush
21958 130/300 Read block 3677 len 40960
21959 131/300 Write block 1810 len 32768 data: 4 4 5 5 5 5 6 6
21960 132/300 Read block 3493 len 16384
21961 133/300 Read block 2119 len 40960
21962 134/300 Read block 3206 len 12288
21963 135/300 Flush
21964 136/300 Write block 3929 len 16384 data: 2 2 4 3
21965 137/300 Flush
21966 138/300 Read block 3988 len 36864
21967 139/300 Read block 3603 len 28672
21968 140/300 Read block 1895 len 28672
21969 141/300 Read block 374 len 40960
21970 142/300 Read block 1842 len 4096
21971 143/300 Write block 400 len 32768 data: 3 5 5 5 5 4 4 5
21972 144/300 Flush
21973 145/300 Read block 4107 len 16384
21974 146/300 Read block 666 len 24576
21975 147/300 Write block 2999 len 24576 data: 2 3 3 4 5 5
21976 148/300 Flush
21977 149/300 Read block 3857 len 28672
21978 150/300 Write block 4726 len 36864 data: 5 6 7 6 5 6 8 6 6
21979 151/300 Write block 3043 len 32768 data: 7 7 4 4 4 5 2 2
21980 152/300 Read block 4160 len 24576
21981 153/300 Read block 1450 len 20480
21982 154/300 Write block 68 len 20480 data: 5 5 4 5 5
21983 155/300 Write block 4721 len 8192 data: 5 3
21984 156/300 Write block 2683 len 4096 data: 5
21985 157/300 Read block 2807 len 16384
21986 158/300 Write block 1956 len 32768 data: 5 5 5 4 4 5 5 4
21987 159/300 Flush
21988 160/300 Read block 2508 len 32768
21989 161/300 Write block 208 len 8192 data: 3 4
21990 162/300 Read block 3695 len 4096
21991 163/300 Write block 3327 len 20480 data: 7 5 4 3 3
21992 164/300 Flush
21993 165/300 Read block 980 len 24576
21994 166/300 Read block 3338 len 4096
21995 167/300 Write block 245 len 16384 data: 2 2 3 4
21996 168/300 Write block 491 len 20480 data: 4 3 3 4 4
21997 169/300 Write block 1961 len 40960 data: 6 6 5 4 4 4 2 3 4 4
21998 170/300 Read block 4817 len 28672
21999 171/300 Read block 2345 len 28672
22000 172/300 Read block 3256 len 32768
22001 173/300 Read block 757 len 16384
22002 174/300 Write block 401 len 20480 data: 6 6 6 6 5
22003 175/300 Write block 4712 len 32768 data: 3 4 4 4 4 4 5 4
22004 176/300 Write block 3000 len 28672 data: 4 4 5 6 6 4 4
22005 177/300 Write block 961 len 20480 data: 4 5 5 6 6
22006 178/300 Write block 4678 len 40960 data: 6 5 4 3 3 3 3 5 5 5
22007 179/300 Write block 1200 len 40960 data: 3 5 6 8 8 7 6 7 6 6
22008 180/300 Read block 2416 len 32768
22009 181/300 Write block 4192 len 12288 data: 8 9 9
22010 182/300 Flush
22011 183/300 Write block 2036 len 20480 data: 3 3 3 4 4
22012 184/300 Read block 838 len 36864
22013 185/300 Read block 169 len 12288
22014 186/300 Flush
22015 187/300 Read block 4249 len 8192
22016 188/300 Write block 4147 len 40960 data: 6 5 4 3 5 6 6 7 8 9
22017 189/300 Flush
22018 190/300 Write block 561 len 8192 data: 6 5
22019 191/300 Write block 213 len 4096 data: 3
22020 192/300 Read block 234 len 28672
22021 193/300 Write block 1617 len 40960 data: 4 4 3 3 3 4 6 6 5 6
22022 194/300 Read block 2573 len 8192
22023 195/300 Read block 2808 len 20480
22024 196/300 Write block 176 len 8192 data: 6 5
22025 197/300 Flush
22026 198/300 Flush
22027 199/300 Read block 4044 len 20480
22028 200/300 Read block 3281 len 36864
22029 201/300 Read block 1295 len 8192
22030 202/300 Read block 4167 len 8192
22031 203/300 Write block 62 len 16384 data: 3 5 6 6
22032 204/300 Write block 1154 len 4096 data: 6
22033 205/300 Read block 2857 len 24576
22034 206/300 Write block 867 len 16384 data: 5 5 7 7
22035 207/300 Read block 1030 len 40960
22036 208/300 Read block 633 len 24576
22037 209/300 Read block 2918 len 20480
22038 210/300 Write block 3778 len 36864 data: 4 5 6 6 5 6 7 7 6
22039 211/300 Write block 4366 len 32768 data: 4 6 6 5 4 5 4 5
22040 212/300 Read block 3591 len 40960
22041 213/300 Read block 4493 len 16384
22042 214/300 Read block 622 len 20480
22043 215/300 Flush
22044 216/300 Read block 2182 len 32768
22045 217/300 Read block 1595 len 16384
22046 218/300 Write block 303 len 8192 data: 4 4
22047 219/300 Read block 3024 len 36864
22048 220/300 Read block 3980 len 40960
22049 221/300 Read block 697 len 28672
22050 222/300 Flush
22051 223/300 Read block 4518 len 8192
22052 224/300 Read block 4578 len 20480
22053 225/300 Write block 465 len 16384 data: 3 4 5 5
22054 226/300 Flush
22055 227/300 Flush
22056 228/300 Read block 299 len 40960
22057 229/300 Write block 699 len 16384 data: 3 4 5 5
22058 230/300 Flush
22059 231/300 Read block 65 len 12288
22060 232/300 Write block 4468 len 4096 data: 4
22061 233/300 Write block 1305 len 32768 data: 6 7 6 4 4 4 3 2
22062 234/300 Write block 2895 len 12288 data: 7 7 7
22063 235/300 Write block 1425 len 28672 data: 5 6 6 5 6 6 5
22064 236/300 Write block 4029 len 4096 data: 7
22065 237/300 Write block 3933 len 28672 data: 4 4 4 4 6 7 8
22066 238/300 Read block 1147 len 16384
22067 239/300 Read block 908 len 36864
22068 240/300 Write block 175 len 28672 data: 5 7 6 3 3 3 4
22069 241/300 Read block 827 len 32768
22070 242/300 Read block 4797 len 28672
22071 243/300 Write block 2501 len 12288 data: 5 5 6
22072 244/300 Read block 2864 len 32768
22073 245/300 Read block 3659 len 24576
22074 246/300 Write block 2532 len 28672 data: 5 6 5 5 4 4 5
22075 247/300 Read block 240 len 28672
22076 248/300 Write block 2726 len 40960 data: 3 4 5 5 4 5 5 4 4 4
22077 249/300 Read block 4464 len 12288
22078 250/300 Write block 1455 len 28672 data: 6 7 5 4 5 5 5
22079 251/300 Read block 3623 len 4096
22080 252/300 Read block 563 len 36864
22081 253/300 Write block 3044 len 16384 data: 8 5 5 5
22082 254/300 Read block 2355 len 36864
22083 255/300 Read block 1549 len 4096
22084 256/300 Write block 1316 len 36864 data: 3 4 3 4 5 3 2 3 4
22085 257/300 Read block 2884 len 24576
22086 258/300 Read block 3901 len 36864
22087 259/300 Write block 648 len 12288 data: 5 5 5
22088 260/300 Write block 1667 len 28672 data: 4 4 4 4 4 4 4
22089 261/300 Read block 1564 len 8192
22090 262/300 Write block 515 len 36864 data: 4 4 4 5 6 8 5 5 5
22091 263/300 Write block 1800 len 32768 data: 6 5 6 6 5 4 5 4
22092 264/300 Write block 2026 len 40960 data: 6 5 6 8 7 6 4 5 4 4
22093 265/300 Read block 569 len 4096
22094 266/300 Read block 2834 len 8192
22095 267/300 Write block 1075 len 16384 data: 5 6 7 8
22096 268/300 Write block 942 len 28672 data: 3 3 3 3 3 3 3
22097 269/300 Read block 835 len 36864
22098 270/300 Write block 625 len 12288 data: 2 2 3
22099 271/300 Read block 894 len 16384
22100 272/300 Read block 1447 len 36864
22101 273/300 Read block 901 len 20480
22102 274/300 Write block 1437 len 28672 data: 6 6 5 3 3 3 3
22103 275/300 Write block 4759 len 8192 data: 2 2
22104 276/300 Read block 1771 len 40960
22105 277/300 Read block 2263 len 16384
22106 278/300 Write block 412 len 12288 data: 4 4 3
22107 279/300 Write block 553 len 32768 data: 5 6 5 5 5 6 7 6
22108 280/300 Read block 4115 len 36864
22109 281/300 Read block 4880 len 40960
22110 282/300 Read block 3703 len 8192
22111 283/300 Read block 276 len 24576
22112 284/300 Flush
22113 285/300 Write block 4991 len 32768 data: 3 3 3 3 4 4 2 2
22114 286/300 Flush
22115 287/300 Read block 1706 len 8192
22116 288/300 Read block 3478 len 36864
22117 289/300 Read block 2449 len 32768
22118 290/300 Read block 3628 len 4096
22119 291/300 Write block 3307 len 4096 data: 8
22120 292/300 Write block 4933 len 16384 data: 5 5 5 5
22121 293/300 Read block 2993 len 8192
22122 294/300 Read block 1191 len 16384
22123 295/300 Read block 1542 len 32768
22124 296/300 Read block 1911 len 36864
22125 297/300 Write block 3677 len 32768 data: 6 6 6 9 8 7 5 4
22126 298/300 Read block 4168 len 12288
22127 299/300 Read block 1960 len 36864
22128 300/300 Write block 40 len 28672 data: 6 5 6 6 5 4 4
22129 301/300 Read block 1708 len 24576
22130 Main task has work to do, go find it
22131 got dsc Start(0)
22132 start 0
22133 Jan 15 05:23:39.828 INFO request completed, latency_us: 197, response_code: 204, uri: /start/cid/0, method: POST, req_id: 33e4b0b1-2f32-4f69-a4fd-cc4ffe971a14, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
22134 Replay: started 0, returned:Ok(())
22135 ----------------------------------------------------------------
22136 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22137 States: Offline Active Active
22138 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22139 6737 Acked Write 5 Sent Done Done false
22140 6738 Acked Write 7 Sent Done Done false
22141 6739 Acked Read 2 Sent Done Done false
22142 6740 Acked Write 6 Sent Done Done false
22143 6741 Acked Read 2 Sent Done Done false
22144 6742 Acked Write 1 Sent Done Done false
22145 6743 Acked Write 2 Sent Done Done false
22146 6744 Acked Write 9 Sent Done Done false
22147 6745 Acked Write 1 Sent Done Done false
22148 6746 Acked Read 2 Sent Done Done false
22149 6747 Acked Flush 0 Sent Done Done false
22150 6748 Acked Read 4 Sent Done Done false
22151 6749 Acked Read 7 Sent Done Done false
22152 6750 Acked Read 10 Sent Done Done false
22153 6751 Acked Flush 0 Sent Done Done false
22154 6752 Acked Write 5 Sent Done Done false
22155 6753 Acked Write 6 Sent Done Done false
22156 6754 Acked Write 3 Sent Done Done false
22157 6755 Acked Write 3 Sent Done Done false
22158 6756 Acked Write 5 Sent Done Done false
22159 6757 Acked Write 5 Sent Done Done false
22160 6758 Acked Write 9 Sent Done Done false
22161 6759 Acked Read 10 Sent Done Done false
22162 6760 Acked Write 8 Sent Done Done false
22163 6761 Acked Read 2 Sent Done Done false
22164 6762 Acked Flush 0 Sent Done Done false
22165 6763 Acked Write 7 Sent Done Done false
22166 6764 Acked Read 1 Sent Done Done false
22167 6765 Acked Write 6 Sent Done Done false
22168 6766 Acked Write 7 Sent Done Done false
22169 6767 Acked Write 8 Sent Done Done false
22170 6768 Acked Write 3 Sent Done Done false
22171 6769 Acked Read 2 Sent Done Done false
22172 6770 Acked Write 8 Sent Done Done false
22173 6771 Acked Write 6 Sent Done Done false
22174 6772 Acked Write 4 Sent Done Done false
22175 6773 Acked Write 4 Sent Done Done false
22176 6774 Acked Write 7 Sent Done Done false
22177 6775 Acked Read 10 Sent Done Done false
22178 6776 Acked Flush 0 Sent Done Done false
22179 6777 Acked Write 7 Sent Done Done false
22180 6778 Acked Read 2 Sent Done Done false
22181 6779 Acked Write 7 Sent Done Done false
22182 6780 Acked Write 9 Sent Done Done false
22183 6781 Acked Write 5 Sent Done Done false
22184 6782 Acked Write 6 Sent Done Done false
22185 6783 Acked Write 7 Sent Done Done false
22186 6784 Acked Write 10 Sent Done Done false
22187 6785 Acked Write 5 Sent Done Done false
22188 6786 Acked Read 9 Sent Done Done false
22189 6787 Acked Read 3 Sent Done Done false
22190 6788 Acked Read 7 Sent Done Done false
22191 6789 Acked Read 4 Sent Done Done false
22192 6790 Acked Read 7 Sent Done Done false
22193 6791 Acked Write 9 Sent Done Done false
22194 6792 Acked Read 6 Sent Done Done false
22195 6793 Acked Read 7 Sent Done Done false
22196 6794 Acked Flush 0 Sent Done Done false
22197 6795 Acked Read 3 Sent Done Done false
22198 6796 Acked Flush 0 Sent Done Done false
22199 6797 Acked Read 1 Sent Done Done false
22200 6798 Acked Read 4 Sent Done Done false
22201 6799 Acked Read 6 Sent Done Done false
22202 6800 Acked Write 9 Sent Done Done false
22203 6801 Acked Read 10 Sent Done Done false
22204 6802 Acked Read 4 Sent Done Done false
22205 6803 Acked Read 5 Sent Done Done false
22206 6804 Acked Read 9 Sent Done Done false
22207 6805 Acked Read 5 Sent Done Done false
22208 6806 Acked Write 7 Sent Done Done false
22209 6807 Acked Read 2 Sent Done Done false
22210 6808 Acked Write 1 Sent Done Done false
22211 6809 Acked Read 6 Sent Done Done false
22212 6810 Acked Write 8 Sent Done Done false
22213 6811 Acked Write 3 Sent Done Done false
22214 6812 Acked Read 1 Sent Done Done false
22215 6813 Acked Read 2 Sent Done Done false
22216 6814 Acked Write 8 Sent Done Done false
22217 6815 Acked Read 2 Sent Done Done false
22218 6816 Acked Write 8 Sent Done Done false
22219 6817 Acked Flush 0 Sent Done Done false
22220 6818 Acked Write 3 Sent Done Done false
22221 6819 Acked Write 5 Sent Done Done false
22222 6820 Acked Read 2 Sent Done Done false
22223 6821 Acked Read 10 Sent Done Done false
22224 6822 Acked Read 5 Sent Done Done false
22225 6823 Acked Write 9 Sent Done Done false
22226 6824 Acked Write 6 Sent Done Done false
22227 6825 Acked Read 5 Sent Done Done false
22228 6826 Acked Read 6 Sent Done Done false
22229 6827 Acked Read 7 Sent Done Done false
22230 6828 Acked Flush 0 Sent Done Done false
22231 6829 Acked Write 8 Sent Done Done false
22232 6830 Acked Read 8 Sent Done Done false
22233 6831 Acked Write 1 Sent Done Done false
22234 6832 Acked Read 2 Sent Done Done false
22235 6833 Acked Write 4 Sent Done Done false
22236 6834 Acked Write 7 Sent Done Done false
22237 6835 Acked Write 9 Sent Done Done false
22238 6836 Acked Read 5 Sent Done Done false
22239 6837 Acked Write 8 Sent Done Done false
22240 6838 Acked Flush 0 Sent Done Done false
22241 6839 Acked Write 7 Sent Done Done false
22242 6840 Acked Flush 0 Sent Done Done false
22243 6841 Acked Write 9 Sent Done Done false
22244 6842 Acked Write 8 Sent Done Done false
22245 6843 Acked Read 6 Sent Done Done false
22246 6844 Acked Read 4 Sent Done Done false
22247 6845 Acked Read 1 Sent Done Done false
22248 6846 Acked Write 5 Sent Done Done false
22249 6847 Acked Write 2 Sent Done Done false
22250 6848 Acked Write 1 Sent Done Done false
22251 6849 Acked Read 10 Sent Done Done false
22252 6850 Acked Write 8 Sent Done Done false
22253 6851 Acked Flush 0 Sent Done Done false
22254 6852 Acked Flush 0 Sent Done Done false
22255 6853 Acked Write 8 Sent Done Done false
22256 6854 Acked Flush 0 Sent Done Done false
22257 6855 Acked Read 1 Sent Done Done false
22258 6856 Acked Flush 0 Sent Done Done false
22259 6857 Acked Read 5 Sent Done Done false
22260 6858 Acked Read 4 Sent Done Done false
22261 6859 Acked Write 6 Sent Done Done false
22262 6860 Acked Read 9 Sent Done Done false
22263 6861 Acked Write 6 Sent Done Done false
22264 6862 Acked Write 9 Sent Done Done false
22265 6863 Acked Write 8 Sent Done Done false
22266 6864 Acked Write 9 Sent Done Done false
22267 6865 Acked Flush 0 Sent Done Done false
22268 6866 Acked Read 10 Sent Done Done false
22269 6867 Acked Write 8 Sent Done Done false
22270 6868 Acked Read 4 Sent Done Done false
22271 6869 Acked Read 10 Sent Done Done false
22272 6870 Acked Read 3 Sent Done Done false
22273 6871 Acked Flush 0 Sent Done Done false
22274 6872 Acked Write 4 Sent Done Done false
22275 6873 Acked Flush 0 Sent Done Done false
22276 6874 Acked Read 9 Sent Done Done false
22277 6875 Acked Read 7 Sent Done Done false
22278 6876 Acked Read 7 Sent Done Done false
22279 6877 Acked Read 10 Sent Done Done false
22280 6878 Acked Read 1 Sent Done Done false
22281 6879 Acked Write 8 Sent Done Done false
22282 6880 Acked Flush 0 Sent Done Done false
22283 6881 Acked Read 4 Sent Done Done false
22284 6882 Acked Read 6 Sent Done Done false
22285 6883 Acked Write 6 Sent Done Done false
22286 6884 Acked Flush 0 Sent Done Done false
22287 6885 Acked Read 7 Sent Done Done false
22288 6886 Acked Write 9 Sent Done Done false
22289 6887 Acked Write 8 Sent Done Done false
22290 6888 Acked Read 6 Sent Done Done false
22291 6889 Acked Read 5 Sent Done Done false
22292 6890 Acked Write 5 Sent Done Done false
22293 6891 Acked Write 2 Sent Done Done false
22294 6892 Acked Write 1 Sent Done Done false
22295 6893 Acked Read 4 Sent Done Done false
22296 6894 Acked Write 8 Sent Done Done false
22297 6895 Acked Flush 0 Sent Done Done false
22298 6896 Acked Read 8 Sent Done Done false
22299 6897 Acked Write 2 Sent Done Done false
22300 6898 Acked Read 1 Sent Done Done false
22301 6899 Acked Write 5 Sent Done Done false
22302 6900 Acked Flush 0 Sent Done Done false
22303 6901 Acked Read 6 Sent Done Done false
22304 6902 Acked Read 1 Sent Done Done false
22305 6903 Acked Write 4 Sent Done Done false
22306 6904 Acked Write 5 Sent Done Done false
22307 6905 Acked Write 10 Sent Done Done false
22308 6906 Acked Read 7 Sent Done Done false
22309 6907 Acked Read 7 Sent Done Done false
22310 6908 Acked Read 8 Sent Done Done false
22311 6909 Acked Read 4 Sent Done Done false
22312 6910 Acked Write 5 Sent Done Done false
22313 6911 Acked Write 8 Sent Done Done false
22314 6912 Acked Write 7 Sent Done Done false
22315 6913 Acked Write 5 Sent Done Done false
22316 6914 Acked Write 10 Sent Done Done false
22317 6915 Acked Write 10 Sent Done Done false
22318 6916 Acked Read 8 Sent Done Done false
22319 6917 Acked Write 3 Sent Done Done false
22320 6918 Acked Flush 0 Sent Done Done false
22321 6919 Acked Write 5 Sent Done Done false
22322 6920 Acked Read 9 Sent Done Done false
22323 6921 Acked Read 3 Sent Done Done false
22324 6922 Acked Flush 0 Sent Done Done false
22325 6923 Acked Read 2 Sent Done Done false
22326 6924 Acked Write 10 Sent Done Done false
22327 6925 Acked Flush 0 Sent Done Done false
22328 6926 Acked Write 2 Sent Done Done false
22329 6927 Acked Write 1 Sent Done Done false
22330 6928 Acked Read 7 Sent Done Done false
22331 6929 Acked Write 10 Sent Done Done false
22332 6930 Acked Read 2 Sent Done Done false
22333 6931 Acked Read 5 Sent Done Done false
22334 6932 Acked Write 2 Sent Done Done false
22335 6933 Acked Flush 0 Sent Done Done false
22336 6934 Acked Flush 0 Sent Done Done false
22337 6935 Acked Read 5 Sent Done Done false
22338 6936 Acked Read 9 Sent Done Done false
22339 6937 Acked Read 2 Sent Done Done false
22340 6938 Acked Read 2 Sent Done Done false
22341 6939 Acked Write 4 Sent Done Done false
22342 6940 Acked Write 1 Sent Done Done false
22343 6941 Acked Read 6 Sent Done Done false
22344 6942 Acked Write 4 Sent Done Done false
22345 6943 Acked Read 10 Sent Done Done false
22346 6944 Acked Read 6 Sent Done Done false
22347 6945 Acked Read 5 Sent Done Done false
22348 6946 Acked Write 9 Sent Done Done false
22349 6947 Acked Write 8 Sent Done Done false
22350 6948 Acked Read 10 Sent Done Done false
22351 6949 Acked Read 4 Sent Done Done false
22352 6950 Acked Read 5 Sent Done Done false
22353 6951 Acked Flush 0 Sent Done Done false
22354 6952 Acked Read 8 Sent Done Done false
22355 6953 Acked Read 4 Sent Done Done false
22356 6954 Acked Write 2 Sent Done Done false
22357 6955 Acked Read 9 Sent Done Done false
22358 6956 Acked Read 10 Sent Done Done false
22359 6957 Acked Read 7 Sent Done Done false
22360 6958 Acked Flush 0 Sent Done Done false
22361 6959 Acked Read 2 Sent Done Done false
22362 6960 Acked Read 5 Sent Done Done false
22363 6961 Acked Write 4 Sent Done Done false
22364 6962 Acked Flush 0 Sent Done Done false
22365 6963 Acked Flush 0 Sent Done Done false
22366 6964 Acked Read 10 Sent Done Done false
22367 6965 Acked Write 4 Sent Done Done false
22368 6966 Acked Flush 0 Sent Done Done false
22369 6967 Acked Read 3 Sent Done Done false
22370 6968 Acked Write 1 Sent Done Done false
22371 6969 Acked Write 8 Sent Done Done false
22372 6970 Acked Write 3 Sent Done Done false
22373 6971 Acked Write 7 Sent Done Done false
22374 6972 Acked Write 1 Sent Done Done false
22375 6973 Acked Write 7 Sent Done Done false
22376 6974 Acked Read 4 Sent Done Done false
22377 6975 Acked Read 9 Sent Done Done false
22378 6976 Acked Write 7 Sent Done Done false
22379 6977 Acked Read 8 Sent Done Done false
22380 6978 Acked Read 7 Sent Done Done false
22381 6979 Acked Write 3 Sent Done Done false
22382 6980 Acked Read 8 Sent Done Done false
22383 6981 Acked Read 6 Sent Done Done false
22384 6982 Acked Write 7 Sent Done Done false
22385 6983 Acked Read 7 Sent Done Done false
22386 6984 Acked Write 10 Sent Done Done false
22387 6985 Acked Read 3 Sent Done Done false
22388 6986 Acked Write 7 Sent Done Done false
22389 6987 Acked Read 1 Sent Done Done false
22390 6988 Acked Read 9 Sent Done Done false
22391 6989 Acked Write 4 Sent Done Done false
22392 6990 Acked Read 9 Sent Done Done false
22393 6991 Acked Read 1 Sent Done Done false
22394 6992 Acked Write 9 Sent Done Done false
22395 6993 Acked Read 6 Sent Done Done false
22396 6994 Acked Read 9 Sent Done Done false
22397 6995 Acked Write 3 Sent Done Done false
22398 6996 Acked Write 7 Sent Done Done false
22399 6997 Acked Read 2 Sent Done Done false
22400 6998 Acked Write 9 Sent Done Done false
22401 6999 Acked Write 8 Sent Done Done false
22402 7000 Acked Write 10 Sent Done Done false
22403 7001 Acked Read 1 Sent Done Done false
22404 7002 Acked Read 2 Sent Done Done false
22405 7003 Acked Write 4 Sent Done Done false
22406 7004 Acked Write 7 Sent Done Done false
22407 7005 Acked Read 9 Sent Done Done false
22408 7006 Acked Write 3 Sent Done Done false
22409 7007 Acked Read 4 Sent Done Done false
22410 7008 Acked Read 9 Sent Done Done false
22411 7009 Acked Read 5 Sent Done Done false
22412 7010 Acked Write 7 Sent Done Done false
22413 7011 Acked Write 2 Sent Done Done false
22414 7012 Acked Read 10 Sent Done Done false
22415 7013 Acked Read 4 Sent Done Done false
22416 7014 Acked Write 3 Sent Done Done false
22417 7015 Acked Write 8 Sent Done Done false
22418 7016 Acked Read 9 Sent Done Done false
22419 7017 Acked Read 10 Sent Done Done false
22420 7018 Acked Read 2 Sent Done Done false
22421 7019 Acked Read 6 Sent Done Done false
22422 7020 Acked Flush 0 Sent Done Done false
22423 7021 Acked Write 8 Sent Done Done false
22424 7022 Acked Flush 0 Sent Done Done false
22425 7023 Acked Read 2 Sent Done Done false
22426 7024 Acked Read 9 Sent Done Done false
22427 7025 Acked Read 8 Sent Done Done false
22428 7026 Acked Read 1 Sent Done Done false
22429 7027 Acked Write 1 Sent Done Done false
22430 7028 Acked Write 4 Sent Done Done false
22431 7029 Acked Read 2 Sent Done Done false
22432 7030 Acked Read 4 Sent Done Done false
22433 7031 Acked Read 8 Sent Done Done false
22434 7032 Acked Read 9 Sent Done Done false
22435 7033 Acked Write 8 Sent Done Done false
22436 7034 Acked Read 3 Sent Done Done false
22437 7035 Acked Read 9 Sent Done Done false
22438 7036 Acked Write 7 Sent Done Done false
22439 7037 Acked Read 6 Sent Done Done false
22440 STATES DS:0 DS:1 DS:2 TOTAL
22441 Sent 301 0 0 301
22442 Done 0 301 301 602
22443 Skipped 0 0 0 0
22444 Error 0 0 0 0
22445 Last Flush: 6736 7022 7022
22446 Downstairs last five completed: 6736 6735 6734 6733 6732
22447 Upstairs last five completed: 7037 7036 7035 7034 7033
22448 CLIENT: Up:0 ds:301 act:2
22449 [8810] got start action
22450 [8810] I am going to restart
22451 Starting downstairs at port 8810
22452 Make output file at "/tmp/dsc/downstairs-8810.txt"
22453 [8810][0] reports Starting
22454 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1222)
22455 [8810][0] reports Running
22456 ----------------------------------------------------------------
22457 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22458 States: Offline Active Active
22459 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22460 6737 Acked Write 5 Sent Done Done false
22461 6738 Acked Write 7 Sent Done Done false
22462 6739 Acked Read 2 Sent Done Done false
22463 6740 Acked Write 6 Sent Done Done false
22464 6741 Acked Read 2 Sent Done Done false
22465 6742 Acked Write 1 Sent Done Done false
22466 6743 Acked Write 2 Sent Done Done false
22467 6744 Acked Write 9 Sent Done Done false
22468 6745 Acked Write 1 Sent Done Done false
22469 6746 Acked Read 2 Sent Done Done false
22470 6747 Acked Flush 0 Sent Done Done false
22471 6748 Acked Read 4 Sent Done Done false
22472 6749 Acked Read 7 Sent Done Done false
22473 6750 Acked Read 10 Sent Done Done false
22474 6751 Acked Flush 0 Sent Done Done false
22475 6752 Acked Write 5 Sent Done Done false
22476 6753 Acked Write 6 Sent Done Done false
22477 6754 Acked Write 3 Sent Done Done false
22478 6755 Acked Write 3 Sent Done Done false
22479 6756 Acked Write 5 Sent Done Done false
22480 6757 Acked Write 5 Sent Done Done false
22481 6758 Acked Write 9 Sent Done Done false
22482 6759 Acked Read 10 Sent Done Done false
22483 6760 Acked Write 8 Sent Done Done false
22484 6761 Acked Read 2 Sent Done Done false
22485 6762 Acked Flush 0 Sent Done Done false
22486 6763 Acked Write 7 Sent Done Done false
22487 6764 Acked Read 1 Sent Done Done false
22488 6765 Acked Write 6 Sent Done Done false
22489 6766 Acked Write 7 Sent Done Done false
22490 6767 Acked Write 8 Sent Done Done false
22491 6768 Acked Write 3 Sent Done Done false
22492 6769 Acked Read 2 Sent Done Done false
22493 6770 Acked Write 8 Sent Done Done false
22494 6771 Acked Write 6 Sent Done Done false
22495 6772 Acked Write 4 Sent Done Done false
22496 6773 Acked Write 4 Sent Done Done false
22497 6774 Acked Write 7 Sent Done Done false
22498 6775 Acked Read 10 Sent Done Done false
22499 6776 Acked Flush 0 Sent Done Done false
22500 6777 Acked Write 7 Sent Done Done false
22501 6778 Acked Read 2 Sent Done Done false
22502 6779 Acked Write 7 Sent Done Done false
22503 6780 Acked Write 9 Sent Done Done false
22504 6781 Acked Write 5 Sent Done Done false
22505 6782 Acked Write 6 Sent Done Done false
22506 6783 Acked Write 7 Sent Done Done false
22507 6784 Acked Write 10 Sent Done Done false
22508 6785 Acked Write 5 Sent Done Done false
22509 6786 Acked Read 9 Sent Done Done false
22510 6787 Acked Read 3 Sent Done Done false
22511 6788 Acked Read 7 Sent Done Done false
22512 6789 Acked Read 4 Sent Done Done false
22513 6790 Acked Read 7 Sent Done Done false
22514 6791 Acked Write 9 Sent Done Done false
22515 6792 Acked Read 6 Sent Done Done false
22516 6793 Acked Read 7 Sent Done Done false
22517 6794 Acked Flush 0 Sent Done Done false
22518 6795 Acked Read 3 Sent Done Done false
22519 6796 Acked Flush 0 Sent Done Done false
22520 6797 Acked Read 1 Sent Done Done false
22521 6798 Acked Read 4 Sent Done Done false
22522 6799 Acked Read 6 Sent Done Done false
22523 6800 Acked Write 9 Sent Done Done false
22524 6801 Acked Read 10 Sent Done Done false
22525 6802 Acked Read 4 Sent Done Done false
22526 6803 Acked Read 5 Sent Done Done false
22527 6804 Acked Read 9 Sent Done Done false
22528 6805 Acked Read 5 Sent Done Done false
22529 6806 Acked Write 7 Sent Done Done false
22530 6807 Acked Read 2 Sent Done Done false
22531 6808 Acked Write 1 Sent Done Done false
22532 6809 Acked Read 6 Sent Done Done false
22533 6810 Acked Write 8 Sent Done Done false
22534 6811 Acked Write 3 Sent Done Done false
22535 6812 Acked Read 1 Sent Done Done false
22536 6813 Acked Read 2 Sent Done Done false
22537 6814 Acked Write 8 Sent Done Done false
22538 6815 Acked Read 2 Sent Done Done false
22539 6816 Acked Write 8 Sent Done Done false
22540 6817 Acked Flush 0 Sent Done Done false
22541 6818 Acked Write 3 Sent Done Done false
22542 6819 Acked Write 5 Sent Done Done false
22543 6820 Acked Read 2 Sent Done Done false
22544 6821 Acked Read 10 Sent Done Done false
22545 6822 Acked Read 5 Sent Done Done false
22546 6823 Acked Write 9 Sent Done Done false
22547 6824 Acked Write 6 Sent Done Done false
22548 6825 Acked Read 5 Sent Done Done false
22549 6826 Acked Read 6 Sent Done Done false
22550 6827 Acked Read 7 Sent Done Done false
22551 6828 Acked Flush 0 Sent Done Done false
22552 6829 Acked Write 8 Sent Done Done false
22553 6830 Acked Read 8 Sent Done Done false
22554 6831 Acked Write 1 Sent Done Done false
22555 6832 Acked Read 2 Sent Done Done false
22556 6833 Acked Write 4 Sent Done Done false
22557 6834 Acked Write 7 Sent Done Done false
22558 6835 Acked Write 9 Sent Done Done false
22559 6836 Acked Read 5 Sent Done Done false
22560 6837 Acked Write 8 Sent Done Done false
22561 6838 Acked Flush 0 Sent Done Done false
22562 6839 Acked Write 7 Sent Done Done false
22563 6840 Acked Flush 0 Sent Done Done false
22564 6841 Acked Write 9 Sent Done Done false
22565 6842 Acked Write 8 Sent Done Done false
22566 6843 Acked Read 6 Sent Done Done false
22567 6844 Acked Read 4 Sent Done Done false
22568 6845 Acked Read 1 Sent Done Done false
22569 6846 Acked Write 5 Sent Done Done false
22570 6847 Acked Write 2 Sent Done Done false
22571 6848 Acked Write 1 Sent Done Done false
22572 6849 Acked Read 10 Sent Done Done false
22573 6850 Acked Write 8 Sent Done Done false
22574 6851 Acked Flush 0 Sent Done Done false
22575 6852 Acked Flush 0 Sent Done Done false
22576 6853 Acked Write 8 Sent Done Done false
22577 6854 Acked Flush 0 Sent Done Done false
22578 6855 Acked Read 1 Sent Done Done false
22579 6856 Acked Flush 0 Sent Done Done false
22580 6857 Acked Read 5 Sent Done Done false
22581 6858 Acked Read 4 Sent Done Done false
22582 6859 Acked Write 6 Sent Done Done false
22583 6860 Acked Read 9 Sent Done Done false
22584 6861 Acked Write 6 Sent Done Done false
22585 6862 Acked Write 9 Sent Done Done false
22586 6863 Acked Write 8 Sent Done Done false
22587 6864 Acked Write 9 Sent Done Done false
22588 6865 Acked Flush 0 Sent Done Done false
22589 6866 Acked Read 10 Sent Done Done false
22590 6867 Acked Write 8 Sent Done Done false
22591 6868 Acked Read 4 Sent Done Done false
22592 6869 Acked Read 10 Sent Done Done false
22593 6870 Acked Read 3 Sent Done Done false
22594 6871 Acked Flush 0 Sent Done Done false
22595 6872 Acked Write 4 Sent Done Done false
22596 6873 Acked Flush 0 Sent Done Done false
22597 6874 Acked Read 9 Sent Done Done false
22598 6875 Acked Read 7 Sent Done Done false
22599 6876 Acked Read 7 Sent Done Done false
22600 6877 Acked Read 10 Sent Done Done false
22601 6878 Acked Read 1 Sent Done Done false
22602 6879 Acked Write 8 Sent Done Done false
22603 6880 Acked Flush 0 Sent Done Done false
22604 6881 Acked Read 4 Sent Done Done false
22605 6882 Acked Read 6 Sent Done Done false
22606 6883 Acked Write 6 Sent Done Done false
22607 6884 Acked Flush 0 Sent Done Done false
22608 6885 Acked Read 7 Sent Done Done false
22609 6886 Acked Write 9 Sent Done Done false
22610 6887 Acked Write 8 Sent Done Done false
22611 6888 Acked Read 6 Sent Done Done false
22612 6889 Acked Read 5 Sent Done Done false
22613 6890 Acked Write 5 Sent Done Done false
22614 6891 Acked Write 2 Sent Done Done false
22615 6892 Acked Write 1 Sent Done Done false
22616 6893 Acked Read 4 Sent Done Done false
22617 6894 Acked Write 8 Sent Done Done false
22618 6895 Acked Flush 0 Sent Done Done false
22619 6896 Acked Read 8 Sent Done Done false
22620 6897 Acked Write 2 Sent Done Done false
22621 6898 Acked Read 1 Sent Done Done false
22622 6899 Acked Write 5 Sent Done Done false
22623 6900 Acked Flush 0 Sent Done Done false
22624 6901 Acked Read 6 Sent Done Done false
22625 6902 Acked Read 1 Sent Done Done false
22626 6903 Acked Write 4 Sent Done Done false
22627 6904 Acked Write 5 Sent Done Done false
22628 6905 Acked Write 10 Sent Done Done false
22629 6906 Acked Read 7 Sent Done Done false
22630 6907 Acked Read 7 Sent Done Done false
22631 6908 Acked Read 8 Sent Done Done false
22632 6909 Acked Read 4 Sent Done Done false
22633 6910 Acked Write 5 Sent Done Done false
22634 6911 Acked Write 8 Sent Done Done false
22635 6912 Acked Write 7 Sent Done Done false
22636 6913 Acked Write 5 Sent Done Done false
22637 6914 Acked Write 10 Sent Done Done false
22638 6915 Acked Write 10 Sent Done Done false
22639 6916 Acked Read 8 Sent Done Done false
22640 6917 Acked Write 3 Sent Done Done false
22641 6918 Acked Flush 0 Sent Done Done false
22642 6919 Acked Write 5 Sent Done Done false
22643 6920 Acked Read 9 Sent Done Done false
22644 6921 Acked Read 3 Sent Done Done false
22645 6922 Acked Flush 0 Sent Done Done false
22646 6923 Acked Read 2 Sent Done Done false
22647 6924 Acked Write 10 Sent Done Done false
22648 6925 Acked Flush 0 Sent Done Done false
22649 6926 Acked Write 2 Sent Done Done false
22650 6927 Acked Write 1 Sent Done Done false
22651 6928 Acked Read 7 Sent Done Done false
22652 6929 Acked Write 10 Sent Done Done false
22653 6930 Acked Read 2 Sent Done Done false
22654 6931 Acked Read 5 Sent Done Done false
22655 6932 Acked Write 2 Sent Done Done false
22656 6933 Acked Flush 0 Sent Done Done false
22657 6934 Acked Flush 0 Sent Done Done false
22658 6935 Acked Read 5 Sent Done Done false
22659 6936 Acked Read 9 Sent Done Done false
22660 6937 Acked Read 2 Sent Done Done false
22661 6938 Acked Read 2 Sent Done Done false
22662 6939 Acked Write 4 Sent Done Done false
22663 6940 Acked Write 1 Sent Done Done false
22664 6941 Acked Read 6 Sent Done Done false
22665 6942 Acked Write 4 Sent Done Done false
22666 6943 Acked Read 10 Sent Done Done false
22667 6944 Acked Read 6 Sent Done Done false
22668 6945 Acked Read 5 Sent Done Done false
22669 6946 Acked Write 9 Sent Done Done false
22670 6947 Acked Write 8 Sent Done Done false
22671 6948 Acked Read 10 Sent Done Done false
22672 6949 Acked Read 4 Sent Done Done false
22673 6950 Acked Read 5 Sent Done Done false
22674 6951 Acked Flush 0 Sent Done Done false
22675 6952 Acked Read 8 Sent Done Done false
22676 6953 Acked Read 4 Sent Done Done false
22677 6954 Acked Write 2 Sent Done Done false
22678 6955 Acked Read 9 Sent Done Done false
22679 6956 Acked Read 10 Sent Done Done false
22680 6957 Acked Read 7 Sent Done Done false
22681 6958 Acked Flush 0 Sent Done Done false
22682 6959 Acked Read 2 Sent Done Done false
22683 6960 Acked Read 5 Sent Done Done false
22684 6961 Acked Write 4 Sent Done Done false
22685 6962 Acked Flush 0 Sent Done Done false
22686 6963 Acked Flush 0 Sent Done Done false
22687 6964 Acked Read 10 Sent Done Done false
22688 6965 Acked Write 4 Sent Done Done false
22689 6966 Acked Flush 0 Sent Done Done false
22690 6967 Acked Read 3 Sent Done Done false
22691 6968 Acked Write 1 Sent Done Done false
22692 6969 Acked Write 8 Sent Done Done false
22693 6970 Acked Write 3 Sent Done Done false
22694 6971 Acked Write 7 Sent Done Done false
22695 6972 Acked Write 1 Sent Done Done false
22696 6973 Acked Write 7 Sent Done Done false
22697 6974 Acked Read 4 Sent Done Done false
22698 6975 Acked Read 9 Sent Done Done false
22699 6976 Acked Write 7 Sent Done Done false
22700 6977 Acked Read 8 Sent Done Done false
22701 6978 Acked Read 7 Sent Done Done false
22702 6979 Acked Write 3 Sent Done Done false
22703 6980 Acked Read 8 Sent Done Done false
22704 6981 Acked Read 6 Sent Done Done false
22705 6982 Acked Write 7 Sent Done Done false
22706 6983 Acked Read 7 Sent Done Done false
22707 6984 Acked Write 10 Sent Done Done false
22708 6985 Acked Read 3 Sent Done Done false
22709 6986 Acked Write 7 Sent Done Done false
22710 6987 Acked Read 1 Sent Done Done false
22711 6988 Acked Read 9 Sent Done Done false
22712 6989 Acked Write 4 Sent Done Done false
22713 6990 Acked Read 9 Sent Done Done false
22714 6991 Acked Read 1 Sent Done Done false
22715 6992 Acked Write 9 Sent Done Done false
22716 6993 Acked Read 6 Sent Done Done false
22717 6994 Acked Read 9 Sent Done Done false
22718 6995 Acked Write 3 Sent Done Done false
22719 6996 Acked Write 7 Sent Done Done false
22720 6997 Acked Read 2 Sent Done Done false
22721 6998 Acked Write 9 Sent Done Done false
22722 6999 Acked Write 8 Sent Done Done false
22723 7000 Acked Write 10 Sent Done Done false
22724 7001 Acked Read 1 Sent Done Done false
22725 7002 Acked Read 2 Sent Done Done false
22726 7003 Acked Write 4 Sent Done Done false
22727 7004 Acked Write 7 Sent Done Done false
22728 7005 Acked Read 9 Sent Done Done false
22729 7006 Acked Write 3 Sent Done Done false
22730 7007 Acked Read 4 Sent Done Done false
22731 7008 Acked Read 9 Sent Done Done false
22732 7009 Acked Read 5 Sent Done Done false
22733 7010 Acked Write 7 Sent Done Done false
22734 7011 Acked Write 2 Sent Done Done false
22735 7012 Acked Read 10 Sent Done Done false
22736 7013 Acked Read 4 Sent Done Done false
22737 7014 Acked Write 3 Sent Done Done false
22738 7015 Acked Write 8 Sent Done Done false
22739 7016 Acked Read 9 Sent Done Done false
22740 7017 Acked Read 10 Sent Done Done false
22741 7018 Acked Read 2 Sent Done Done false
22742 7019 Acked Read 6 Sent Done Done false
22743 7020 Acked Flush 0 Sent Done Done false
22744 7021 Acked Write 8 Sent Done Done false
22745 7022 Acked Flush 0 Sent Done Done false
22746 7023 Acked Read 2 Sent Done Done false
22747 7024 Acked Read 9 Sent Done Done false
22748 7025 Acked Read 8 Sent Done Done false
22749 7026 Acked Read 1 Sent Done Done false
22750 7027 Acked Write 1 Sent Done Done false
22751 7028 Acked Write 4 Sent Done Done false
22752 7029 Acked Read 2 Sent Done Done false
22753 7030 Acked Read 4 Sent Done Done false
22754 7031 Acked Read 8 Sent Done Done false
22755 7032 Acked Read 9 Sent Done Done false
22756 7033 Acked Write 8 Sent Done Done false
22757 7034 Acked Read 3 Sent Done Done false
22758 7035 Acked Read 9 Sent Done Done false
22759 7036 Acked Write 7 Sent Done Done false
22760 7037 Acked Read 6 Sent Done Done false
22761 7038 Acked Flush 0 Sent Done Done false
22762 STATES DS:0 DS:1 DS:2 TOTAL
22763 Sent 302 0 0 302
22764 Done 0 302 302 604
22765 Skipped 0 0 0 0
22766 Error 0 0 0 0
22767 Last Flush: 6736 7038 7038
22768 Downstairs last five completed: 6736 6735 6734 6733 6732
22769 Upstairs last five completed: 7038 7037 7036 7035 7034
22770 CLIENT: Up:0 ds:302 act:2
227712025-01-15T05:23:45.600ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
227722025-01-15T05:23:45.600ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:44017) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
227732025-01-15T05:23:45.601ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
227742025-01-15T05:23:45.602ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
227752025-01-15T05:23:45.602ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
227762025-01-15T05:23:45.602ZINFOcrucible: send last flush ID to this DS: 6736 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
227772025-01-15T05:23:45.602ZINFOcrucible: Replied this last flush ID: 6736 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
227782025-01-15T05:23:45.602ZINFOcrucible: [0] client re-new 302 jobs since flush 6736 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
227792025-01-15T05:23:45.602ZINFOcrucible: [0] Replayed 302 jobs since flush: 6736 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
22780 ----------------------------------------------------------------
22781 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22782 Downstairs last five completed: 7038 7037 7036 7035 7034
22783 Upstairs last five completed: 7038 7037 7036 7035 7034
22784 CLIENT: Up:0 ds:0 act:3
22785 Replay: All jobs finished
22786 Main task has work to do, go find it
22787 got dsc Stop(1)
22788 stop 1
22789 [8820] Got stop action so:false kr:false
22790 Jan 15 05:23:47.833 INFO request completed, latency_us: 338, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 30477f7d-c020-41a6-b9f1-b7abe317ce9e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
22791 Jan 15 05:23:47.834 INFO request completed, latency_us: 140, response_code: 200, {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:23:47.834266812Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
22792 uri: /state/cid/1, method: GET, req_id: 6f0a06a9-e870-4d23-9c7c-47634c382131, remote_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:23:47.834343725Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
22793 .0.0.{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:23:47.834387461Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
22794 1:42402, local_addr: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:23:47.834427168Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
22795 127.0.0.1:9998
22796 [8820] Exited with: ExitStatus(unix_wait_status(9))
22797 [8820][1] reports Exit
22798 Jan 15 05:23:51.836 INFO request completed, latency_us: 193, response_code: 200, uri: /state/cid/1, method: GET, req_id: 716f1556-b86a-48ac-9df7-32088dde7227, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
22799 001/300 Read block 3758 len 32768
22800 002/300 Read block 1551 len 12288
22801 003/300 Read block 3977 len 24576
22802 004/300 Flush
22803 005/300 Write block 456 len 36864 data: 8 5 7 5 4 4 4 2 2
22804 006/300 Read block 515 len 40960
22805 007/300 Read block 4878 len 20480
22806 008/300 Write block 714 len 28672 data: 3 3 2 3 3 3 4
22807 009/300 Write block 4319 len 12288 data: 9 10 9
22808 010/300 Read block 4259 len 28672
22809 011/300 Read block 930 len 24576
22810 012/300 Read block 2489 len 16384
22811 013/300 Read block 144 len 28672
22812 014/300 Read block 2477 len 20480
22813 015/300 Read block 4683 len 8192
22814 016/300 Write block 4076 len 16384 data: 4 3 4 4
22815 017/300 Read block 3008 len 24576
22816 018/300 Write block 3094 len 8192 data: 7 6
22817 019/300 Read block 1576 len 8192
22818 020/300 Read block 707 len 28672
22819 021/300 Flush
22820 022/300 Read block 3638 len 24576
22821 023/300 Write block 1402 len 4096 data: 4
22822 024/300 Write block 1890 len 40960 data: 4 5 6 7 8 7 5 6 7 7
22823 025/300 Write block 3815 len 40960 data: 6 8 7 5 5 6 6 5 5 4
22824 026/300 Write block 2179 len 8192 data: 6 6
22825 027/300 Write block 1084 len 28672 data: 5 6 5 6 5 4 3
22826 028/300 Read block 3440 len 16384
22827 029/300 Write block 3740 len 4096 data: 7
22828 030/300 Write block 743 len 12288 data: 8 9 5
22829 031/300 Write block 267 len 24576 data: 5 5 5 5 6 5
22830 032/300 Flush
22831 033/300 Write block 1802 len 28672 data: 7 7 6 5 6 5 5
22832 034/300 Flush
22833 035/300 Write block 2208 len 24576 data: 5 5 4 5 3 3
22834 036/300 Write block 3695 len 24576 data: 7 8 7 8 6 6
22835 037/300 Read block 2669 len 32768
22836 038/300 Read block 4969 len 24576
22837 039/300 Flush
22838 040/300 Read block 879 len 4096
22839 041/300 Read block 1286 len 4096
22840 042/300 Write block 3274 len 32768 data: 5 5 5 6 6 6 5 4
22841 043/300 Read block 397 len 40960
22842 044/300 Read block 218 len 4096
22843 045/300 Write block 1472 len 32768 data: 4 4 4 5 4 5 5 6
22844 046/300 Read block 1577 len 28672
22845 047/300 Write block 3669 len 40960 data: 8 8 8 10 9 9 8 8 7 7
22846 048/300 Write block 4813 len 20480 data: 4 5 7 6 8
22847 049/300 Write block 2432 len 36864 data: 7 9 8 5 5 5 6 8 8
22848 050/300 Read block 4053 len 4096
22849 051/300 Flush
22850 052/300 Read block 4857 len 4096
22851 053/300 Read block 308 len 32768
22852 054/300 Write block 3984 len 16384 data: 2 2 3 4
22853 055/300 Flush
22854 056/300 Write block 3914 len 24576 data: 6 6 6 7 8 6
22855 057/300 Write block 1214 len 12288 data: 4 3 6
22856 058/300 Read block 1573 len 28672
22857 059/300 Read block 3357 len 12288
22858 060/300 Read block 2118 len 36864
22859 061/300 Write block 3132 len 8192 data: 5 4
22860 062/300 Read block 2109 len 40960
22861 063/300 Read block 2373 len 16384
22862 064/300 Write block 4450 len 8192 data: 7 6
22863 065/300 Read block 682 len 40960
22864 066/300 Write block 3217 len 36864 data: 3 3 3 3 3 3 3 3 3
22865 067/300 Flush
22866 068/300 Read block 1511 len 32768
22867 069/300 Read block 1810 len 20480
22868 070/300 Write block 16 len 28672 data: 2 3 3 3 3 4 4
22869 071/300 Write block 4347 len 40960 data: 7 6 7 5 5 5 4 3 3 4
22870 072/300 Write block 2432 len 24576 data: 8 10 9 6 6 6
22871 073/300 Flush
22872 074/300 Write block 1126 len 12288 data: 3 3 3
22873 075/300 Read block 3895 len 16384
22874 076/300 Flush
22875 077/300 Read block 1061 len 20480
22876 078/300 Write block 576 len 20480 data: 5 5 5 6 5
22877 079/300 Read block 332 len 40960
22878 080/300 Flush
22879 081/300 Write block 4191 len 4096 data: 7
22880 082/300 Read block 3472 len 32768
22881 083/300 Read block 4456 len 32768
22882 084/300 Read block 742 len 32768
22883 085/300 Read block 1087 len 4096
22884 086/300 Read block 2165 len 4096
22885 087/300 Read block 2658 len 24576
22886 088/300 Read block 2443 len 40960
22887 089/300 Read block 2664 len 28672
22888 090/300 Read block 3033 len 4096
22889 091/300 Flush
22890 092/300 Write block 1756 len 24576 data: 4 5 4 4 5 7
22891 093/300 Flush
22892 094/300 Read block 550 len 40960
22893 095/300 Write block 229 len 20480 data: 4 4 5 4 4
22894 096/300 Write block 2418 len 8192 data: 3 3
22895 097/300 Write block 1655 len 28672 data: 4 4 4 2 2 2 2
22896 098/300 Read block 2278 len 4096
22897 099/300 Flush
22898 100/300 Read block 2875 len 8192
22899 101/300 Write block 2121 len 28672 data: 3 2 2 3 3 3 3
22900 102/300 Flush
22901 103/300 Read block 2372 len 20480
22902 104/300 Write block 127 len 8192 data: 4 4
22903 105/300 Flush
22904 106/300 Read block 3626 len 16384
22905 107/300 Read block 1103 len 16384
22906 108/300 Read block 4158 len 28672
22907 109/300 Write block 3519 len 8192 data: 3 2
22908 110/300 Write block 3356 len 12288 data: 4 3 2
22909 111/300 Read block 2159 len 32768
22910 112/300 Read block 3513 len 32768
22911 113/300 Read block 3636 len 20480
22912 114/300 Read block 1741 len 28672
22913 115/300 Read block 4837 len 16384
22914 116/300 Write block 1266 len 36864 data: 4 6 5 4 4 5 4 3 4
22915 117/300 Write block 4446 len 40960 data: 4 5 5 5 8 7 6 6 4 5
22916 118/300 Read block 3565 len 32768
22917 119/300 Write block 3567 len 20480 data: 4 2 2 3 3
22918 120/300 Write block 2395 len 24576 data: 5 5 5 4 4 3
22919 121/300 Read block 956 len 12288
22920 122/300 Read block 4275 len 40960
22921 123/300 Read block 2899 len 16384
22922 124/300 Read block 1155 len 20480
22923 125/300 Write block 1908 len 20480 data: 5 5 4 4 5
22924 126/300 Write block 4323 len 8192 data: 6 3
22925 127/300 Read block 2462 len 32768
22926 128/300 Write block 2201 len 32768 data: 6 6 5 4 5 6 5 6
22927 129/300 Read block 3705 len 16384
22928 130/300 Write block 1204 len 28672 data: 9 8 7 8 7 7 4
22929 131/300 Flush
22930 132/300 Write block 2909 len 4096 data: 5
22931 133/300 Write block 3852 len 8192 data: 4 4
22932 134/300 Write block 442 len 36864 data: 9 9 9 11 11 9 8 7 7
22933 135/300 Read block 4993 len 20480
22934 136/300 Write block 4613 len 28672 data: 6 6 6 6 6 6 6
22935 137/300 Write block 18 len 4096 data: 4
22936 138/300 Write block 285 len 40960 data: 4 4 5 5 6 6 6 5 5 6
22937 139/300 Read block 2522 len 28672
22938 140/300 Read block 3940 len 24576
22939 141/300 Read block 2978 len 24576
22940 142/300 Write block 297 len 28672 data: 4 3 3 3 3 3 5
22941 143/300 Write block 3445 len 4096 data: 3
22942 144/300 Write block 1516 len 12288 data: 6 4 3
22943 145/300 Write block 0 len 8192 data: 2 2
22944 146/300 Flush
22945 147/300 Read block 4491 len 32768
22946 148/300 Write block 2155 len 8192 data: 4 4
22947 149/300 Write block 1893 len 16384 data: 8 9 8 6
22948 150/300 Write block 3809 len 24576 data: 3 4 3 4 4 5
22949 151/300 Write block 2708 len 36864 data: 3 3 3 5 6 5 6 6 6
22950 152/300 Write block 3893 len 12288 data: 6 7 6
22951 153/300 Write block 1375 len 4096 data: 4
22952 154/300 Read block 2791 len 24576
22953 155/300 Read block 1579 len 8192
22954 156/300 Read block 3411 len 12288
22955 157/300 Write block 4302 len 20480 data: 5 5 3 3 3
22956 158/300 Read block 341 len 16384
22957 159/300 Flush
22958 160/300 Write block 1522 len 24576 data: 2 3 3 3 4 5
22959 161/300 Read block 3817 len 16384
22960 162/300 Read block 4981 len 28672
22961 163/300 Read block 2343 len 4096
22962 164/300 Read block 4473 len 36864
22963 165/300 Read block 2416 len 4096
22964 166/300 Read block 1194 len 24576
22965 167/300 Read block 2211 len 20480
22966 168/300 Read block 517 len 28672
22967 169/300 Write block 177 len 40960 data: 7 4 4 4 5 4 5 5 5 5
22968 170/300 Write block 1253 len 20480 data: 4 4 4 4 4
22969 171/300 Read block 109 len 12288
22970 172/300 Read block 1966 len 36864
22971 173/300 Write block 1646 len 4096 data: 6
22972 174/300 Write block 893 len 20480 data: 5 6 6 4 4
22973 175/300 Read block 448 len 8192
22974 176/300 Read block 3880 len 24576
22975 177/300 Write block 2997 len 4096 data: 4
22976 178/300 Write block 998 len 12288 data: 3 3 3
22977 179/300 Read block 4319 len 36864
22978 180/300 Write block 2572 len 28672 data: 3 3 3 3 3 3 3
22979 181/300 Read block 2344 len 16384
22980 182/300 Read block 4983 len 20480
22981 183/300 Write block 1843 len 16384 data: 7 8 10 9
22982 184/300 Read block 3956 len 36864
22983 185/300 Read block 2739 len 28672
22984 186/300 Read block 3634 len 8192
22985 187/300 Write block 4130 len 36864 data: 5 7 8 5 6 6 5 7 5
22986 188/300 Read block 1592 len 4096
22987 189/300 Read block 398 len 32768
22988 190/300 Read block 2553 len 36864
22989 191/300 Read block 2613 len 32768
22990 192/300 Read block 2388 len 16384
22991 193/300 Write block 3479 len 12288 data: 4 4 3
22992 194/300 Write block 23 len 16384 data: 4 4 6 5
22993 195/300 Write block 1767 len 24576 data: 5 5 4 4 4 4
22994 196/300 Read block 3576 len 24576
22995 197/300 Write block 4799 len 16384 data: 3 4 4 4
22996 198/300 Write block 3252 len 28672 data: 6 6 3 2 3 3 3
22997 199/300 Write block 3537 len 40960 data: 5 4 4 4 4 4 4 5 4 5
22998 200/300 Read block 908 len 16384
22999 201/300 Read block 2586 len 40960
23000 202/300 Write block 1134 len 20480 data: 3 3 3 3 3
23001 203/300 Read block 4112 len 16384
23002 204/300 Read block 1813 len 40960
23003 205/300 Read block 3672 len 8192
23004 206/300 Flush
23005 207/300 Read block 1454 len 12288
23006 208/300 Flush
23007 209/300 Write block 799 len 8192 data: 2 2
23008 210/300 Read block 1268 len 8192
23009 211/300 Write block 4755 len 12288 data: 7 7 4
23010 212/300 Write block 2333 len 40960 data: 5 5 4 3 3 4 4 4 6 5
23011 213/300 Read block 4949 len 20480
23012 214/300 Read block 331 len 40960
23013 215/300 Write block 4573 len 12288 data: 5 5 5
23014 216/300 Read block 1886 len 28672
23015 217/300 Flush
23016 218/300 Flush
23017 219/300 Flush
23018 220/300 Read block 430 len 20480
23019 221/300 Write block 0 len 16384 data: 3 3 2 3
23020 222/300 Read block 1582 len 32768
23021 223/300 Write block 1616 len 20480 data: 5 5 5 4 4
23022 224/300 Read block 2620 len 28672
23023 225/300 Read block 2156 len 24576
23024 226/300 Write block 857 len 24576 data: 5 3 4 3 4 4
23025 227/300 Read block 2064 len 32768
23026 228/300 Write block 3273 len 32768 data: 6 6 6 6 7 7 7 6
23027 229/300 Write block 3134 len 24576 data: 4 4 4 5 3 4
23028 230/300 Read block 4752 len 28672
23029 231/300 Read block 681 len 36864
23030 232/300 Write block 1363 len 32768 data: 2 2 4 4 4 5 7 7
23031 233/300 Write block 3674 len 20480 data: 10 9 9 8 8
23032 234/300 Read block 194 len 12288
23033 235/300 Write block 2048 len 12288 data: 4 3 2
23034 236/300 Read block 721 len 16384
23035 237/300 Read block 127 len 12288
23036 238/300 Read block 3940 len 20480
23037 239/300 Write block 4183 len 24576 data: 3 3 2 2 4 5
23038 240/300 Write block 2090 len 20480 data: 5 5 5 5 4
23039 241/300 Write block 3508 len 16384 data: 3 3 3 5
23040 242/300 Write block 1421 len 12288 data: 4 4 4
23041 243/300 Read block 2987 len 40960
23042 244/300 Write block 1206 len 4096 data: 8
23043 245/300 Write block 2528 len 32768 data: 4 5 4 5 6 7 6 6
23044 246/300 Read block 1436 len 24576
23045 247/300 Read block 3922 len 36864
23046 248/300 Flush
23047 249/300 Flush
23048 250/300 Write block 2047 len 20480 data: 4 5 4 3 2
23049 251/300 Write block 868 len 16384 data: 6 8 8 6
23050 252/300 Read block 4972 len 32768
23051 253/300 Write block 3501 len 12288 data: 6 5 5
23052 254/300 Write block 3656 len 40960 data: 6 7 7 7 7 6 4 4 3 3
23053 255/300 Write block 2724 len 16384 data: 4 3 4 5
23054 256/300 Write block 2795 len 32768 data: 4 4 4 3 3 3 4 5
23055 257/300 Read block 2903 len 8192
23056 258/300 Read block 3569 len 40960
23057 259/300 Flush
23058 260/300 Flush
23059 261/300 Read block 538 len 20480
23060 262/300 Write block 2057 len 20480 data: 5 5 5 5 4
23061 263/300 Read block 805 len 32768
23062 264/300 Write block 3589 len 12288 data: 3 3 3
23063 265/300 Write block 2099 len 4096 data: 3
23064 266/300 Write block 2337 len 24576 data: 4 5 5 5 7 6
23065 267/300 Read block 4593 len 16384
23066 268/300 Read block 1603 len 16384
23067 269/300 Read block 2593 len 40960
23068 270/300 Flush
23069 271/300 Read block 4425 len 32768
23070 272/300 Write block 671 len 24576 data: 4 3 4 5 4 5
23071 273/300 Read block 3242 len 40960
23072 274/300 Write block 4042 len 8192 data: 4 3
23073 275/300 Write block 10 len 40960 data: 6 6 4 4 4 2 3 4 5 4
23074 276/300 Read block 2926 len 36864
23075 277/300 Read block 2716 len 36864
23076 278/300 Read block 3868 len 16384
23077 279/300 Write block 4630 len 20480 data: 4 6 6 5 4
23078 280/300 Flush
23079 281/300 Write block 1126 len 40960 data: 4 4 4 3 3 3 3 2 4 4
23080 282/300 Read block 3916 len 16384
23081 283/300 Read block 313 len 40960
23082 284/300 Write block 2540 len 28672 data: 7 6 5 4 2 2 2
23083 285/300 Read block 1951 len 8192
23084 286/300 Read block 312 len 24576
23085 287/300 Read block 3225 len 32768
23086 288/300 Write block 2420 len 12288 data: 4 2 2
23087 289/300 Read block 988 len 12288
23088 290/300 Write block 4887 len 16384 data: 6 7 6 6
23089 291/300 Read block 3617 len 12288
23090 292/300 Write block 3019 len 36864 data: 4 5 5 5 6 6 5 5 5
23091 293/300 Read block 258 len 8192
23092 294/300 Write block 3193 len 20480 data: 3 2 3 5 5
23093 295/300 Write block 2280 len 4096 data: 5
23094 296/300 Read block 4408 len 36864
23095 297/300 Read block 1131 len 28672
23096 298/300 Read block 810 len 4096
23097 299/300 Read block 3735 len 20480
23098 300/300 Flush
23099 301/300 Read block 3213 len 16384
23100 Main task has work to do, go find it
23101 got dsc Start(1)
23102 start 1
23103 Jan 15 05:23:52.070 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/1, method: POST, req_id: 5257b5a1-b1c5-4748-aa32-92179fd3d426, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
23104 Replay: started 1, returned:Ok(())
23105 ----------------------------------------------------------------
23106 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23107 States: Active Offline Active
23108 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23109 7039 Acked Read 8 Done Sent Done false
23110 7040 Acked Read 3 Done Sent Done false
23111 7041 Acked Read 6 Done Sent Done false
23112 7042 Acked Flush 0 Done Sent Done false
23113 7043 Acked Write 9 Done Sent Done false
23114 7044 Acked Read 10 Done Sent Done false
23115 7045 Acked Read 5 Done Sent Done false
23116 7046 Acked Write 7 Done Sent Done false
23117 7047 Acked Write 3 Done Sent Done false
23118 7048 Acked Read 7 Done Sent Done false
23119 7049 Acked Read 6 Done Sent Done false
23120 7050 Acked Read 4 Done Sent Done false
23121 7051 Acked Read 7 Done Sent Done false
23122 7052 Acked Read 5 Done Sent Done false
23123 7053 Acked Read 2 Done Sent Done false
23124 7054 Acked Write 4 Done Sent Done false
23125 7055 Acked Read 6 Done Sent Done false
23126 7056 Acked Write 2 Done Sent Done false
23127 7057 Acked Read 2 Done Sent Done false
23128 7058 Acked Read 7 Done Sent Done false
23129 7059 Acked Flush 0 Done Sent Done false
23130 7060 Acked Read 6 Done Sent Done false
23131 7061 Acked Write 1 Done Sent Done false
23132 7062 Acked Write 10 Done Sent Done false
23133 7063 Acked Write 10 Done Sent Done false
23134 7064 Acked Write 2 Done Sent Done false
23135 7065 Acked Write 7 Done Sent Done false
23136 7066 Acked Read 4 Done Sent Done false
23137 7067 Acked Write 1 Done Sent Done false
23138 7068 Acked Write 3 Done Sent Done false
23139 7069 Acked Write 6 Done Sent Done false
23140 7070 Acked Flush 0 Done Sent Done false
23141 7071 Acked Write 7 Done Sent Done false
23142 7072 Acked Flush 0 Done Sent Done false
23143 7073 Acked Write 6 Done Sent Done false
23144 7074 Acked Write 6 Done Sent Done false
23145 7075 Acked Read 8 Done Sent Done false
23146 7076 Acked Read 6 Done Sent Done false
23147 7077 Acked Flush 0 Done Sent Done false
23148 7078 Acked Read 1 Done Sent Done false
23149 7079 Acked Read 1 Done Sent Done false
23150 7080 Acked Write 8 Done Sent Done false
23151 7081 Acked Read 10 Done Sent Done false
23152 7082 Acked Read 1 Done Sent Done false
23153 7083 Acked Write 8 Done Sent Done false
23154 7084 Acked Read 7 Done Sent Done false
23155 7085 Acked Write 10 Done Sent Done false
23156 7086 Acked Write 5 Done Sent Done false
23157 7087 Acked Write 9 Done Sent Done false
23158 7088 Acked Read 1 Done Sent Done false
23159 7089 Acked Flush 0 Done Sent Done false
23160 7090 Acked Read 1 Done Sent Done false
23161 7091 Acked Read 8 Done Sent Done false
23162 7092 Acked Write 4 Done Sent Done false
23163 7093 Acked Flush 0 Done Sent Done false
23164 7094 Acked Write 6 Done Sent Done false
23165 7095 Acked Write 3 Done Sent Done false
23166 7096 Acked Read 7 Done Sent Done false
23167 7097 Acked Read 3 Done Sent Done false
23168 7098 Acked Read 9 Done Sent Done false
23169 7099 Acked Write 2 Done Sent Done false
23170 7100 Acked Read 10 Done Sent Done false
23171 7101 Acked Read 4 Done Sent Done false
23172 7102 Acked Write 2 Done Sent Done false
23173 7103 Acked Read 10 Done Sent Done false
23174 7104 Acked Write 9 Done Sent Done false
23175 7105 Acked Flush 0 Done Sent Done false
23176 7106 Acked Read 8 Done Sent Done false
23177 7107 Acked Read 5 Done Sent Done false
23178 7108 Acked Write 7 Done Sent Done false
23179 7109 Acked Write 10 Done Sent Done false
23180 7110 Acked Write 6 Done Sent Done false
23181 7111 Acked Flush 0 Done Sent Done false
23182 7112 Acked Write 3 Done Sent Done false
23183 7113 Acked Read 4 Done Sent Done false
23184 7114 Acked Flush 0 Done Sent Done false
23185 7115 Acked Read 5 Done Sent Done false
23186 7116 Acked Write 5 Done Sent Done false
23187 7117 Acked Read 10 Done Sent Done false
23188 7118 Acked Flush 0 Done Sent Done false
23189 7119 Acked Write 1 Done Sent Done false
23190 7120 Acked Read 8 Done Sent Done false
23191 7121 Acked Read 8 Done Sent Done false
23192 7122 Acked Read 8 Done Sent Done false
23193 7123 Acked Read 1 Done Sent Done false
23194 7124 Acked Read 1 Done Sent Done false
23195 7125 Acked Read 6 Done Sent Done false
23196 7126 Acked Read 10 Done Sent Done false
23197 7127 Acked Read 7 Done Sent Done false
23198 7128 Acked Read 1 Done Sent Done false
23199 7129 Acked Flush 0 Done Sent Done false
23200 7130 Acked Write 6 Done Sent Done false
23201 7131 Acked Flush 0 Done Sent Done false
23202 7132 Acked Read 10 Done Sent Done false
23203 7133 Acked Write 5 Done Sent Done false
23204 7134 Acked Write 2 Done Sent Done false
23205 7135 Acked Write 7 Done Sent Done false
23206 7136 Acked Read 1 Done Sent Done false
23207 7137 Acked Flush 0 Done Sent Done false
23208 7138 Acked Read 2 Done Sent Done false
23209 7139 Acked Write 7 Done Sent Done false
23210 7140 Acked Flush 0 Done Sent Done false
23211 7141 Acked Read 5 Done Sent Done false
23212 7142 Acked Write 2 Done Sent Done false
23213 7143 Acked Flush 0 Done Sent Done false
23214 7144 Acked Read 4 Done Sent Done false
23215 7145 Acked Read 4 Done Sent Done false
23216 7146 Acked Read 7 Done Sent Done false
23217 7147 Acked Write 2 Done Sent Done false
23218 7148 Acked Write 3 Done Sent Done false
23219 7149 Acked Read 8 Done Sent Done false
23220 7150 Acked Read 8 Done Sent Done false
23221 7151 Acked Read 5 Done Sent Done false
23222 7152 Acked Read 7 Done Sent Done false
23223 7153 Acked Read 4 Done Sent Done false
23224 7154 Acked Write 9 Done Sent Done false
23225 7155 Acked Write 10 Done Sent Done false
23226 7156 Acked Read 8 Done Sent Done false
23227 7157 Acked Write 5 Done Sent Done false
23228 7158 Acked Write 6 Done Sent Done false
23229 7159 Acked Read 3 Done Sent Done false
23230 7160 Acked Read 10 Done Sent Done false
23231 7161 Acked Read 4 Done Sent Done false
23232 7162 Acked Read 5 Done Sent Done false
23233 7163 Acked Write 5 Done Sent Done false
23234 7164 Acked Write 2 Done Sent Done false
23235 7165 Acked Read 8 Done Sent Done false
23236 7166 Acked Write 8 Done Sent Done false
23237 7167 Acked Read 4 Done Sent Done false
23238 7168 Acked Write 7 Done Sent Done false
23239 7169 Acked Flush 0 Done Sent Done false
23240 7170 Acked Write 1 Done Sent Done false
23241 7171 Acked Write 2 Done Sent Done false
23242 7172 Acked Write 9 Done Sent Done false
23243 7173 Acked Read 5 Done Sent Done false
23244 7174 Acked Write 7 Done Sent Done false
23245 7175 Acked Write 1 Done Sent Done false
23246 7176 Acked Write 10 Done Sent Done false
23247 7177 Acked Read 7 Done Sent Done false
23248 7178 Acked Read 6 Done Sent Done false
23249 7179 Acked Read 6 Done Sent Done false
23250 7180 Acked Write 7 Done Sent Done false
23251 7181 Acked Write 1 Done Sent Done false
23252 7182 Acked Write 3 Done Sent Done false
23253 7183 Acked Write 2 Done Sent Done false
23254 7184 Acked Flush 0 Done Sent Done false
23255 7185 Acked Read 8 Done Sent Done false
23256 7186 Acked Write 2 Done Sent Done false
23257 7187 Acked Write 4 Done Sent Done false
23258 7188 Acked Write 6 Done Sent Done false
23259 7189 Acked Write 9 Done Sent Done false
23260 7190 Acked Write 3 Done Sent Done false
23261 7191 Acked Write 1 Done Sent Done false
23262 7192 Acked Read 6 Done Sent Done false
23263 7193 Acked Read 2 Done Sent Done false
23264 7194 Acked Read 3 Done Sent Done false
23265 7195 Acked Write 5 Done Sent Done false
23266 7196 Acked Read 4 Done Sent Done false
23267 7197 Acked Flush 0 Done Sent Done false
23268 7198 Acked Write 6 Done Sent Done false
23269 7199 Acked Read 4 Done Sent Done false
23270 7200 Acked Read 7 Done Sent Done false
23271 7201 Acked Read 1 Done Sent Done false
23272 7202 Acked Read 9 Done Sent Done false
23273 7203 Acked Read 1 Done Sent Done false
23274 7204 Acked Read 6 Done Sent Done false
23275 7205 Acked Read 5 Done Sent Done false
23276 7206 Acked Read 7 Done Sent Done false
23277 7207 Acked Write 10 Done Sent Done false
23278 7208 Acked Write 5 Done Sent Done false
23279 7209 Acked Read 3 Done Sent Done false
23280 7210 Acked Read 9 Done Sent Done false
23281 7211 Acked Write 1 Done Sent Done false
23282 7212 Acked Write 5 Done Sent Done false
23283 7213 Acked Read 2 Done Sent Done false
23284 7214 Acked Read 6 Done Sent Done false
23285 7215 Acked Write 1 Done Sent Done false
23286 7216 Acked Write 3 Done Sent Done false
23287 7217 Acked Read 9 Done Sent Done false
23288 7218 Acked Write 7 Done Sent Done false
23289 7219 Acked Read 4 Done Sent Done false
23290 7220 Acked Read 5 Done Sent Done false
23291 7221 Acked Write 4 Done Sent Done false
23292 7222 Acked Read 9 Done Sent Done false
23293 7223 Acked Read 7 Done Sent Done false
23294 7224 Acked Read 2 Done Sent Done false
23295 7225 Acked Write 9 Done Sent Done false
23296 7226 Acked Read 1 Done Sent Done false
23297 7227 Acked Read 8 Done Sent Done false
23298 7228 Acked Read 9 Done Sent Done false
23299 7229 Acked Read 8 Done Sent Done false
23300 7230 Acked Read 4 Done Sent Done false
23301 7231 Acked Write 3 Done Sent Done false
23302 7232 Acked Write 4 Done Sent Done false
23303 7233 Acked Write 6 Done Sent Done false
23304 7234 Acked Read 6 Done Sent Done false
23305 7235 Acked Write 4 Done Sent Done false
23306 7236 Acked Write 7 Done Sent Done false
23307 7237 Acked Write 10 Done Sent Done false
23308 7238 Acked Read 4 Done Sent Done false
23309 7239 Acked Read 10 Done Sent Done false
23310 7240 Acked Write 5 Done Sent Done false
23311 7241 Acked Read 4 Done Sent Done false
23312 7242 Acked Read 10 Done Sent Done false
23313 7243 Acked Read 2 Done Sent Done false
23314 7244 Acked Flush 0 Done Sent Done false
23315 7245 Acked Read 3 Done Sent Done false
23316 7246 Acked Flush 0 Done Sent Done false
23317 7247 Acked Write 2 Done Sent Done false
23318 7248 Acked Read 2 Done Sent Done false
23319 7249 Acked Write 3 Done Sent Done false
23320 7250 Acked Write 10 Done Sent Done false
23321 7251 Acked Read 5 Done Sent Done false
23322 7252 Acked Read 10 Done Sent Done false
23323 7253 Acked Write 3 Done Sent Done false
23324 7254 Acked Read 7 Done Sent Done false
23325 7255 Acked Flush 0 Done Sent Done false
23326 7256 Acked Flush 0 Done Sent Done false
23327 7257 Acked Flush 0 Done Sent Done false
23328 7258 Acked Read 5 Done Sent Done false
23329 7259 Acked Write 4 Done Sent Done false
23330 7260 Acked Read 8 Done Sent Done false
23331 7261 Acked Write 5 Done Sent Done false
23332 7262 Acked Read 7 Done Sent Done false
23333 7263 Acked Read 6 Done Sent Done false
23334 7264 Acked Write 6 Done Sent Done false
23335 7265 Acked Read 8 Done Sent Done false
23336 7266 Acked Write 8 Done Sent Done false
23337 7267 Acked Write 6 Done Sent Done false
23338 7268 Acked Read 7 Done Sent Done false
23339 7269 Acked Read 9 Done Sent Done false
23340 7270 Acked Write 8 Done Sent Done false
23341 7271 Acked Write 5 Done Sent Done false
23342 7272 Acked Read 3 Done Sent Done false
23343 7273 Acked Write 3 Done Sent Done false
23344 7274 Acked Read 4 Done Sent Done false
23345 7275 Acked Read 3 Done Sent Done false
23346 7276 Acked Read 5 Done Sent Done false
23347 7277 Acked Write 6 Done Sent Done false
23348 7278 Acked Write 5 Done Sent Done false
23349 7279 Acked Write 4 Done Sent Done false
23350 7280 Acked Write 3 Done Sent Done false
23351 7281 Acked Read 10 Done Sent Done false
23352 7282 Acked Write 1 Done Sent Done false
23353 7283 Acked Write 8 Done Sent Done false
23354 7284 Acked Read 6 Done Sent Done false
23355 7285 Acked Read 9 Done Sent Done false
23356 7286 Acked Flush 0 Done Sent Done false
23357 7287 Acked Flush 0 Done Sent Done false
23358 7288 Acked Write 5 Done Sent Done false
23359 7289 Acked Write 4 Done Sent Done false
23360 7290 Acked Read 8 Done Sent Done false
23361 7291 Acked Write 3 Done Sent Done false
23362 7292 Acked Write 10 Done Sent Done false
23363 7293 Acked Write 4 Done Sent Done false
23364 7294 Acked Write 8 Done Sent Done false
23365 7295 Acked Read 2 Done Sent Done false
23366 7296 Acked Read 10 Done Sent Done false
23367 7297 Acked Flush 0 Done Sent Done false
23368 7298 Acked Flush 0 Done Sent Done false
23369 7299 Acked Read 5 Done Sent Done false
23370 7300 Acked Write 5 Done Sent Done false
23371 7301 Acked Read 8 Done Sent Done false
23372 7302 Acked Write 3 Done Sent Done false
23373 7303 Acked Write 1 Done Sent Done false
23374 7304 Acked Write 6 Done Sent Done false
23375 7305 Acked Read 4 Done Sent Done false
23376 7306 Acked Read 4 Done Sent Done false
23377 7307 Acked Read 10 Done Sent Done false
23378 7308 Acked Flush 0 Done Sent Done false
23379 7309 Acked Read 8 Done Sent Done false
23380 7310 Acked Write 6 Done Sent Done false
23381 7311 Acked Read 10 Done Sent Done false
23382 7312 Acked Write 2 Done Sent Done false
23383 7313 Acked Write 10 Done Sent Done false
23384 7314 Acked Read 9 Done Sent Done false
23385 7315 Acked Read 9 Done Sent Done false
23386 7316 Acked Read 4 Done Sent Done false
23387 7317 Acked Write 5 Done Sent Done false
23388 7318 Acked Flush 0 Done Sent Done false
23389 7319 Acked Write 10 Done Sent Done false
23390 7320 Acked Read 4 Done Sent Done false
23391 7321 Acked Read 10 Done Sent Done false
23392 7322 Acked Write 7 Done Sent Done false
23393 7323 Acked Read 2 Done Sent Done false
23394 7324 Acked Read 6 Done Sent Done false
23395 7325 Acked Read 8 Done Sent Done false
23396 7326 Acked Write 3 Done Sent Done false
23397 7327 Acked Read 3 Done Sent Done false
23398 7328 Acked Write 4 Done Sent Done false
23399 7329 Acked Read 3 Done Sent Done false
23400 7330 Acked Write 9 Done Sent Done false
23401 7331 Acked Read 2 Done Sent Done false
23402 7332 Acked Write 5 Done Sent Done false
23403 7333 Acked Write 1 Done Sent Done false
23404 7334 Acked Read 9 Done Sent Done false
23405 7335 Acked Read 7 Done Sent Done false
23406 7336 Acked Read 1 Done Sent Done false
23407 7337 Acked Read 5 Done Sent Done false
23408 7338 Acked Flush 0 Done Sent Done false
23409 7339 Acked Read 4 Done Sent Done false
23410 STATES DS:0 DS:1 DS:2 TOTAL
23411 Sent 0 301 0 301
23412 Done 301 0 301 602
23413 Skipped 0 0 0 0
23414 Error 0 0 0 0
23415 Last Flush: 7338 7038 7338
23416 Downstairs last five completed: 7038 7037 7036 7035 7034
23417 Upstairs last five completed: 7339 7338 7337 7336 7335
23418 CLIENT: Up:0 ds:301 act:2
23419 [8820] got start action
23420 [8820] I am going to restart
23421 Starting downstairs at port 8820
23422 Make output file at "/tmp/dsc/downstairs-8820.txt"
23423 [8820][1] reports Starting
23424 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1223)
23425 [8820][1] reports Running
23426 ----------------------------------------------------------------
23427 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23428 States: Active Offline Active
23429 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23430 7039 Acked Read 8 Done Sent Done false
23431 7040 Acked Read 3 Done Sent Done false
23432 7041 Acked Read 6 Done Sent Done false
23433 7042 Acked Flush 0 Done Sent Done false
23434 7043 Acked Write 9 Done Sent Done false
23435 7044 Acked Read 10 Done Sent Done false
23436 7045 Acked Read 5 Done Sent Done false
23437 7046 Acked Write 7 Done Sent Done false
23438 7047 Acked Write 3 Done Sent Done false
23439 7048 Acked Read 7 Done Sent Done false
23440 7049 Acked Read 6 Done Sent Done false
23441 7050 Acked Read 4 Done Sent Done false
23442 7051 Acked Read 7 Done Sent Done false
23443 7052 Acked Read 5 Done Sent Done false
23444 7053 Acked Read 2 Done Sent Done false
23445 7054 Acked Write 4 Done Sent Done false
23446 7055 Acked Read 6 Done Sent Done false
23447 7056 Acked Write 2 Done Sent Done false
23448 7057 Acked Read 2 Done Sent Done false
23449 7058 Acked Read 7 Done Sent Done false
23450 7059 Acked Flush 0 Done Sent Done false
23451 7060 Acked Read 6 Done Sent Done false
23452 7061 Acked Write 1 Done Sent Done false
23453 7062 Acked Write 10 Done Sent Done false
23454 7063 Acked Write 10 Done Sent Done false
23455 7064 Acked Write 2 Done Sent Done false
23456 7065 Acked Write 7 Done Sent Done false
23457 7066 Acked Read 4 Done Sent Done false
23458 7067 Acked Write 1 Done Sent Done false
23459 7068 Acked Write 3 Done Sent Done false
23460 7069 Acked Write 6 Done Sent Done false
23461 7070 Acked Flush 0 Done Sent Done false
23462 7071 Acked Write 7 Done Sent Done false
23463 7072 Acked Flush 0 Done Sent Done false
23464 7073 Acked Write 6 Done Sent Done false
23465 7074 Acked Write 6 Done Sent Done false
23466 7075 Acked Read 8 Done Sent Done false
23467 7076 Acked Read 6 Done Sent Done false
23468 7077 Acked Flush 0 Done Sent Done false
23469 7078 Acked Read 1 Done Sent Done false
23470 7079 Acked Read 1 Done Sent Done false
23471 7080 Acked Write 8 Done Sent Done false
23472 7081 Acked Read 10 Done Sent Done false
23473 7082 Acked Read 1 Done Sent Done false
23474 7083 Acked Write 8 Done Sent Done false
23475 7084 Acked Read 7 Done Sent Done false
23476 7085 Acked Write 10 Done Sent Done false
23477 7086 Acked Write 5 Done Sent Done false
23478 7087 Acked Write 9 Done Sent Done false
23479 7088 Acked Read 1 Done Sent Done false
23480 7089 Acked Flush 0 Done Sent Done false
23481 7090 Acked Read 1 Done Sent Done false
23482 7091 Acked Read 8 Done Sent Done false
23483 7092 Acked Write 4 Done Sent Done false
23484 7093 Acked Flush 0 Done Sent Done false
23485 7094 Acked Write 6 Done Sent Done false
23486 7095 Acked Write 3 Done Sent Done false
23487 7096 Acked Read 7 Done Sent Done false
23488 7097 Acked Read 3 Done Sent Done false
23489 7098 Acked Read 9 Done Sent Done false
23490 7099 Acked Write 2 Done Sent Done false
23491 7100 Acked Read 10 Done Sent Done false
23492 7101 Acked Read 4 Done Sent Done false
23493 7102 Acked Write 2 Done Sent Done false
23494 7103 Acked Read 10 Done Sent Done false
23495 7104 Acked Write 9 Done Sent Done false
23496 7105 Acked Flush 0 Done Sent Done false
23497 7106 Acked Read 8 Done Sent Done false
23498 7107 Acked Read 5 Done Sent Done false
23499 7108 Acked Write 7 Done Sent Done false
23500 7109 Acked Write 10 Done Sent Done false
23501 7110 Acked Write 6 Done Sent Done false
23502 7111 Acked Flush 0 Done Sent Done false
23503 7112 Acked Write 3 Done Sent Done false
23504 7113 Acked Read 4 Done Sent Done false
23505 7114 Acked Flush 0 Done Sent Done false
23506 7115 Acked Read 5 Done Sent Done false
23507 7116 Acked Write 5 Done Sent Done false
23508 7117 Acked Read 10 Done Sent Done false
23509 7118 Acked Flush 0 Done Sent Done false
23510 7119 Acked Write 1 Done Sent Done false
23511 7120 Acked Read 8 Done Sent Done false
23512 7121 Acked Read 8 Done Sent Done false
23513 7122 Acked Read 8 Done Sent Done false
23514 7123 Acked Read 1 Done Sent Done false
23515 7124 Acked Read 1 Done Sent Done false
23516 7125 Acked Read 6 Done Sent Done false
23517 7126 Acked Read 10 Done Sent Done false
23518 7127 Acked Read 7 Done Sent Done false
23519 7128 Acked Read 1 Done Sent Done false
23520 7129 Acked Flush 0 Done Sent Done false
23521 7130 Acked Write 6 Done Sent Done false
23522 7131 Acked Flush 0 Done Sent Done false
23523 7132 Acked Read 10 Done Sent Done false
23524 7133 Acked Write 5 Done Sent Done false
23525 7134 Acked Write 2 Done Sent Done false
23526 7135 Acked Write 7 Done Sent Done false
23527 7136 Acked Read 1 Done Sent Done false
23528 7137 Acked Flush 0 Done Sent Done false
23529 7138 Acked Read 2 Done Sent Done false
23530 7139 Acked Write 7 Done Sent Done false
23531 7140 Acked Flush 0 Done Sent Done false
23532 7141 Acked Read 5 Done Sent Done false
23533 7142 Acked Write 2 Done Sent Done false
23534 7143 Acked Flush 0 Done Sent Done false
23535 7144 Acked Read 4 Done Sent Done false
23536 7145 Acked Read 4 Done Sent Done false
23537 7146 Acked Read 7 Done Sent Done false
23538 7147 Acked Write 2 Done Sent Done false
23539 7148 Acked Write 3 Done Sent Done false
23540 7149 Acked Read 8 Done Sent Done false
23541 7150 Acked Read 8 Done Sent Done false
23542 7151 Acked Read 5 Done Sent Done false
23543 7152 Acked Read 7 Done Sent Done false
23544 7153 Acked Read 4 Done Sent Done false
23545 7154 Acked Write 9 Done Sent Done false
23546 7155 Acked Write 10 Done Sent Done false
23547 7156 Acked Read 8 Done Sent Done false
23548 7157 Acked Write 5 Done Sent Done false
23549 7158 Acked Write 6 Done Sent Done false
23550 7159 Acked Read 3 Done Sent Done false
23551 7160 Acked Read 10 Done Sent Done false
23552 7161 Acked Read 4 Done Sent Done false
23553 7162 Acked Read 5 Done Sent Done false
23554 7163 Acked Write 5 Done Sent Done false
23555 7164 Acked Write 2 Done Sent Done false
23556 7165 Acked Read 8 Done Sent Done false
23557 7166 Acked Write 8 Done Sent Done false
23558 7167 Acked Read 4 Done Sent Done false
23559 7168 Acked Write 7 Done Sent Done false
23560 7169 Acked Flush 0 Done Sent Done false
23561 7170 Acked Write 1 Done Sent Done false
23562 7171 Acked Write 2 Done Sent Done false
23563 7172 Acked Write 9 Done Sent Done false
23564 7173 Acked Read 5 Done Sent Done false
23565 7174 Acked Write 7 Done Sent Done false
23566 7175 Acked Write 1 Done Sent Done false
23567 7176 Acked Write 10 Done Sent Done false
23568 7177 Acked Read 7 Done Sent Done false
23569 7178 Acked Read 6 Done Sent Done false
23570 7179 Acked Read 6 Done Sent Done false
23571 7180 Acked Write 7 Done Sent Done false
23572 7181 Acked Write 1 Done Sent Done false
23573 7182 Acked Write 3 Done Sent Done false
23574 7183 Acked Write 2 Done Sent Done false
23575 7184 Acked Flush 0 Done Sent Done false
23576 7185 Acked Read 8 Done Sent Done false
23577 7186 Acked Write 2 Done Sent Done false
23578 7187 Acked Write 4 Done Sent Done false
23579 7188 Acked Write 6 Done Sent Done false
23580 7189 Acked Write 9 Done Sent Done false
23581 7190 Acked Write 3 Done Sent Done false
23582 7191 Acked Write 1 Done Sent Done false
23583 7192 Acked Read 6 Done Sent Done false
23584 7193 Acked Read 2 Done Sent Done false
23585 7194 Acked Read 3 Done Sent Done false
23586 7195 Acked Write 5 Done Sent Done false
23587 7196 Acked Read 4 Done Sent Done false
23588 7197 Acked Flush 0 Done Sent Done false
23589 7198 Acked Write 6 Done Sent Done false
23590 7199 Acked Read 4 Done Sent Done false
23591 7200 Acked Read 7 Done Sent Done false
23592 7201 Acked Read 1 Done Sent Done false
23593 7202 Acked Read 9 Done Sent Done false
23594 7203 Acked Read 1 Done Sent Done false
23595 7204 Acked Read 6 Done Sent Done false
23596 7205 Acked Read 5 Done Sent Done false
23597 7206 Acked Read 7 Done Sent Done false
23598 7207 Acked Write 10 Done Sent Done false
23599 7208 Acked Write 5 Done Sent Done false
23600 7209 Acked Read 3 Done Sent Done false
23601 7210 Acked Read 9 Done Sent Done false
23602 7211 Acked Write 1 Done Sent Done false
23603 7212 Acked Write 5 Done Sent Done false
23604 7213 Acked Read 2 Done Sent Done false
23605 7214 Acked Read 6 Done Sent Done false
23606 7215 Acked Write 1 Done Sent Done false
23607 7216 Acked Write 3 Done Sent Done false
23608 7217 Acked Read 9 Done Sent Done false
23609 7218 Acked Write 7 Done Sent Done false
23610 7219 Acked Read 4 Done Sent Done false
23611 7220 Acked Read 5 Done Sent Done false
23612 7221 Acked Write 4 Done Sent Done false
23613 7222 Acked Read 9 Done Sent Done false
23614 7223 Acked Read 7 Done Sent Done false
23615 7224 Acked Read 2 Done Sent Done false
23616 7225 Acked Write 9 Done Sent Done false
23617 7226 Acked Read 1 Done Sent Done false
23618 7227 Acked Read 8 Done Sent Done false
23619 7228 Acked Read 9 Done Sent Done false
23620 7229 Acked Read 8 Done Sent Done false
23621 7230 Acked Read 4 Done Sent Done false
23622 7231 Acked Write 3 Done Sent Done false
23623 7232 Acked Write 4 Done Sent Done false
23624 7233 Acked Write 6 Done Sent Done false
23625 7234 Acked Read 6 Done Sent Done false
23626 7235 Acked Write 4 Done Sent Done false
23627 7236 Acked Write 7 Done Sent Done false
23628 7237 Acked Write 10 Done Sent Done false
23629 7238 Acked Read 4 Done Sent Done false
23630 7239 Acked Read 10 Done Sent Done false
23631 7240 Acked Write 5 Done Sent Done false
23632 7241 Acked Read 4 Done Sent Done false
23633 7242 Acked Read 10 Done Sent Done false
23634 7243 Acked Read 2 Done Sent Done false
23635 7244 Acked Flush 0 Done Sent Done false
23636 7245 Acked Read 3 Done Sent Done false
23637 7246 Acked Flush 0 Done Sent Done false
23638 7247 Acked Write 2 Done Sent Done false
23639 7248 Acked Read 2 Done Sent Done false
23640 7249 Acked Write 3 Done Sent Done false
23641 7250 Acked Write 10 Done Sent Done false
23642 7251 Acked Read 5 Done Sent Done false
23643 7252 Acked Read 10 Done Sent Done false
23644 7253 Acked Write 3 Done Sent Done false
23645 7254 Acked Read 7 Done Sent Done false
23646 7255 Acked Flush 0 Done Sent Done false
23647 7256 Acked Flush 0 Done Sent Done false
23648 7257 Acked Flush 0 Done Sent Done false
23649 7258 Acked Read 5 Done Sent Done false
23650 7259 Acked Write 4 Done Sent Done false
23651 7260 Acked Read 8 Done Sent Done false
23652 7261 Acked Write 5 Done Sent Done false
23653 7262 Acked Read 7 Done Sent Done false
23654 7263 Acked Read 6 Done Sent Done false
23655 7264 Acked Write 6 Done Sent Done false
23656 7265 Acked Read 8 Done Sent Done false
23657 7266 Acked Write 8 Done Sent Done false
23658 7267 Acked Write 6 Done Sent Done false
23659 7268 Acked Read 7 Done Sent Done false
23660 7269 Acked Read 9 Done Sent Done false
23661 7270 Acked Write 8 Done Sent Done false
23662 7271 Acked Write 5 Done Sent Done false
23663 7272 Acked Read 3 Done Sent Done false
23664 7273 Acked Write 3 Done Sent Done false
23665 7274 Acked Read 4 Done Sent Done false
23666 7275 Acked Read 3 Done Sent Done false
23667 7276 Acked Read 5 Done Sent Done false
23668 7277 Acked Write 6 Done Sent Done false
23669 7278 Acked Write 5 Done Sent Done false
23670 7279 Acked Write 4 Done Sent Done false
23671 7280 Acked Write 3 Done Sent Done false
23672 7281 Acked Read 10 Done Sent Done false
23673 7282 Acked Write 1 Done Sent Done false
23674 7283 Acked Write 8 Done Sent Done false
23675 7284 Acked Read 6 Done Sent Done false
23676 7285 Acked Read 9 Done Sent Done false
23677 7286 Acked Flush 0 Done Sent Done false
23678 7287 Acked Flush 0 Done Sent Done false
23679 7288 Acked Write 5 Done Sent Done false
23680 7289 Acked Write 4 Done Sent Done false
23681 7290 Acked Read 8 Done Sent Done false
23682 7291 Acked Write 3 Done Sent Done false
23683 7292 Acked Write 10 Done Sent Done false
23684 7293 Acked Write 4 Done Sent Done false
23685 7294 Acked Write 8 Done Sent Done false
23686 7295 Acked Read 2 Done Sent Done false
23687 7296 Acked Read 10 Done Sent Done false
23688 7297 Acked Flush 0 Done Sent Done false
23689 7298 Acked Flush 0 Done Sent Done false
23690 7299 Acked Read 5 Done Sent Done false
23691 7300 Acked Write 5 Done Sent Done false
23692 7301 Acked Read 8 Done Sent Done false
23693 7302 Acked Write 3 Done Sent Done false
23694 7303 Acked Write 1 Done Sent Done false
23695 7304 Acked Write 6 Done Sent Done false
23696 7305 Acked Read 4 Done Sent Done false
23697 7306 Acked Read 4 Done Sent Done false
23698 7307 Acked Read 10 Done Sent Done false
23699 7308 Acked Flush 0 Done Sent Done false
23700 7309 Acked Read 8 Done Sent Done false
23701 7310 Acked Write 6 Done Sent Done false
23702 7311 Acked Read 10 Done Sent Done false
23703 7312 Acked Write 2 Done Sent Done false
23704 7313 Acked Write 10 Done Sent Done false
23705 7314 Acked Read 9 Done Sent Done false
23706 7315 Acked Read 9 Done Sent Done false
23707 7316 Acked Read 4 Done Sent Done false
23708 7317 Acked Write 5 Done Sent Done false
23709 7318 Acked Flush 0 Done Sent Done false
23710 7319 Acked Write 10 Done Sent Done false
23711 7320 Acked Read 4 Done Sent Done false
23712 7321 Acked Read 10 Done Sent Done false
23713 7322 Acked Write 7 Done Sent Done false
23714 7323 Acked Read 2 Done Sent Done false
23715 7324 Acked Read 6 Done Sent Done false
23716 7325 Acked Read 8 Done Sent Done false
23717 7326 Acked Write 3 Done Sent Done false
23718 7327 Acked Read 3 Done Sent Done false
23719 7328 Acked Write 4 Done Sent Done false
23720 7329 Acked Read 3 Done Sent Done false
23721 7330 Acked Write 9 Done Sent Done false
23722 7331 Acked Read 2 Done Sent Done false
23723 7332 Acked Write 5 Done Sent Done false
23724 7333 Acked Write 1 Done Sent Done false
23725 7334 Acked Read 9 Done Sent Done false
23726 7335 Acked Read 7 Done Sent Done false
23727 7336 Acked Read 1 Done Sent Done false
23728 7337 Acked Read 5 Done Sent Done false
23729 7338 Acked Flush 0 Done Sent Done false
23730 7339 Acked Read 4 Done Sent Done false
23731 7340 Acked Flush 0 Done Sent Done false
23732 STATES DS:0 DS:1 DS:2 TOTAL
23733 Sent 0 302 0 302
23734 Done 302 0 302 604
23735 Skipped 0 0 0 0
23736 Error 0 0 0 0
23737 Last Flush: 7340 7038 7340
23738 Downstairs last five completed: 7038 7037 7036 7035 7034
23739 Upstairs last five completed: 7340 7339 7338 7337 7336
23740 CLIENT: Up:0 ds:302 act:2
237412025-01-15T05:23:57.835ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237422025-01-15T05:23:57.835ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49001) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237432025-01-15T05:23:57.836ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237442025-01-15T05:23:57.836ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237452025-01-15T05:23:57.837ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237462025-01-15T05:23:57.837ZINFOcrucible: send last flush ID to this DS: 7038 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237472025-01-15T05:23:57.837ZINFOcrucible: Replied this last flush ID: 7038 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237482025-01-15T05:23:57.837ZINFOcrucible: [1] client re-new 302 jobs since flush 7038 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237492025-01-15T05:23:57.837ZINFOcrucible: [1] Replayed 302 jobs since flush: 7038 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
23750 ----------------------------------------------------------------
23751 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23752 Downstairs last five completed: 7340 7339 7338 7337 7336
23753 Upstairs last five completed: 7340 7339 7338 7337 7336
23754 CLIENT: Up:0 ds:0 act:3
23755 Replay: All jobs finished
23756 Main task has work to do, go find it
23757 got dsc Stop(1)
23758 stop 1
23759 Jan 15 05:24:00.074 INFO request completed, latency_us: 342, response_code: [8820] Got stop action so:false kr:false
23760 204, uri: /stop/cid/1, method: POST, req_id: 12ce9567-c193-4693-8b67-15162119e752, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
23761 Jan 15 05:24:00.075 INFO request completed, latency_us: 120, response_code: 200, uri: /state/cid/1, method: GET, req_id: 9f8be1cc-375c-41e5-b527-f6df5bf71ac3, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:24:00.075410628Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
23762 .1:9998
237632025-01-15T05:24:00.075ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237642025-01-15T05:24:00.075ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
237652025-01-15T05:24:00.075ZINFOcrucible: client task is exiting = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
23766 [8820] Exited with: ExitStatus(unix_wait_status(9))
23767 [8820][1] reports Exit
23768 Jan 15 05:24:04.076 INFO request completed, latency_us: 192, response_code: 200, uri: /state/cid/1, method: GET, req_id: ee2429f3-abc3-47e8-b221-7c7040ef2380, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
23769 001/300 Read block 4183 len 20480
23770 002/300 Read block 2009 len 40960
23771 003/300 Read block 1788 len 8192
23772 004/300 Read block 1939 len 40960
23773 005/300 Read block 2355 len 24576
23774 006/300 Read block 3599 len 12288
23775 007/300 Write block 1372 len 8192 data: 7 6
23776 008/300 Read block 429 len 24576
23777 009/300 Write block 4020 len 32768 data: 8 8 8 8 6 4 4 5
23778 010/300 Flush
23779 011/300 Read block 4912 len 28672
23780 012/300 Read block 3820 len 20480
23781 013/300 Read block 1220 len 36864
23782 014/300 Write block 2780 len 12288 data: 9 8 7
23783 015/300 Write block 1675 len 16384 data: 4 5 5 7
23784 016/300 Read block 195 len 8192
23785 017/300 Write block 3954 len 40960 data: 4 4 4 3 3 3 3 3 3 2
23786 018/300 Write block 1460 len 28672 data: 6 6 5 5 5 5 5
23787 019/300 Write block 2465 len 8192 data: 5 6
23788 020/300 Flush
23789 021/300 Read block 2858 len 20480
23790 022/300 Write block 3136 len 32768 data: 5 6 4 5 4 4 4 7
23791 023/300 Read block 1546 len 28672
23792 024/300 Read block 921 len 16384
23793 025/300 Read block 2296 len 20480
23794 026/300 Read block 464 len 16384
23795 027/300 Write block 525 len 28672 data: 5 5 5 5 6 5 5
23796 028/300 Read block 4693 len 32768
23797 029/300 Read block 91 len 12288
23798 030/300 Write block 1616 len 20480 data: 6 6 6 5 5
23799 031/300 Write block 179 len 36864 data: 5 5 6 5 6 6 6 6 5
23800 032/300 Read block 180 len 32768
23801 033/300 Write block 808 len 40960 data: 4 4 4 4 3 3 2 2 2 2
23802 034/300 Write block 1491 len 4096 data: 4
23803 035/300 Read block 1541 len 24576
23804 036/300 Read block 4045 len 4096
23805 037/300 Read block 392 len 16384
23806 038/300 Read block 733 len 12288
23807 039/300 Write block 863 len 12288 data: 4 4 3
23808 040/300 Read block 3944 len 4096
23809 041/300 Read block 1339 len 4096
23810 042/300 Read block 3384 len 8192
23811 043/300 Read block 4424 len 40960
23812 044/300 Write block 1866 len 40960 data: 4 3 5 5 6 5 5 5 6 5
23813 045/300 Write block 933 len 4096 data: 4
23814 046/300 Flush
23815 047/300 Write block 2640 len 4096 data: 5
23816 048/300 Read block 2951 len 40960
23817 049/300 Write block 1123 len 36864 data: 3 3 4 5 5 5 4 4 4
23818 050/300 Write block 1364 len 20480 data: 3 5 5 5 6
23819 051/300 Write block 3120 len 12288 data: 7 4 5
23820 052/300 Read block 1053 len 8192
23821 053/300 Flush
23822 054/300 Write block 258 len 16384 data: 10 7 7 5
23823 055/300 Write block 2605 len 4096 data: 6
23824 056/300 Flush
23825 057/300 Read block 4666 len 24576
23826 058/300 Write block 656 len 40960 data: 3 3 3 3 5 5 6 5 4 4
23827 059/300 Write block 891 len 8192 data: 4 5
23828 060/300 Write block 3007 len 28672 data: 4 5 5 3 2 2 3
23829 061/300 Read block 4571 len 32768
23830 062/300 Write block 767 len 28672 data: 3 4 5 5 4 4 4
23831 063/300 Read block 1636 len 8192
23832 064/300 Read block 3416 len 32768
23833 065/300 Read block 4158 len 28672
23834 066/300 Read block 1249 len 16384
23835 067/300 Read block 2948 len 36864
23836 068/300 Write block 4354 len 24576 data: 4 4 5 4 4 6
23837 069/300 Write block 4254 len 24576 data: 3 3 4 5 7 7
23838 070/300 Write block 3611 len 12288 data: 5 5 3
23839 071/300 Write block 554 len 12288 data: 7 6 6
23840 072/300 Write block 477 len 32768 data: 5 4 3 3 4 4 2 3
23841 073/300 Read block 1385 len 20480
23842 074/300 Read block 1383 len 16384
23843 075/300 Flush
23844 076/300 Read block 3044 len 24576
23845 077/300 Write block 3671 len 32768 data: 9 11 10 11 10 10 9 9
23846 078/300 Read block 4149 len 16384
23847 079/300 Flush
23848 080/300 Read block 534 len 16384
23849 081/300 Write block 4723 len 24576 data: 4 4 5 6 7 8
23850 082/300 Write block 27 len 8192 data: 5 5
23851 083/300 Flush
23852 084/300 Flush
23853 085/300 Flush
23854 086/300 Write block 4085 len 16384 data: 4 3 3 2
23855 087/300 Write block 3421 len 20480 data: 3 4 4 3 4
23856 088/300 Read block 928 len 24576
23857 089/300 Read block 4259 len 28672
23858 090/300 Read block 1261 len 4096
23859 091/300 Write block 1777 len 40960 data: 5 5 4 5 5 5 9 9 9 11
23860 092/300 Write block 2383 len 40960 data: 3 4 3 3 3 2 2 2 3 4
23861 093/300 Flush
23862 094/300 Read block 3708 len 36864
23863 095/300 Write block 824 len 8192 data: 8 7
23864 096/300 Read block 4664 len 4096
23865 097/300 Read block 4092 len 20480
23866 098/300 Write block 2707 len 8192 data: 3 4
23867 099/300 Write block 4075 len 40960 data: 4 5 4 5 5 3 4 4 5 4
23868 100/300 Read block 4456 len 32768
23869 101/300 Read block 657 len 32768
23870 102/300 Write block 2771 len 24576 data: 4 4 4 5 5 4
23871 103/300 Read block 2557 len 4096
23872 104/300 Read block 2937 len 24576
23873 105/300 Write block 1192 len 8192 data: 4 4
23874 106/300 Read block 3189 len 8192
23875 107/300 Flush
23876 108/300 Write block 3144 len 40960 data: 5 5 4 3 3 3 3 4 6 6
23877 109/300 Flush
23878 110/300 Read block 532 len 28672
23879 111/300 Read block 2023 len 32768
23880 112/300 Write block 3419 len 4096 data: 4
23881 113/300 Write block 1292 len 8192 data: 5 6
23882 114/300 Write block 2992 len 32768 data: 3 4 5 4 4 5 2 3
23883 115/300 Write block 1723 len 28672 data: 3 3 3 3 4 3 4
23884 116/300 Write block 3149 len 8192 data: 4 4
23885 117/300 Write block 2712 len 32768 data: 7 6 7 7 7 6 4 4
23886 118/300 Read block 3156 len 8192
23887 119/300 Read block 3334 len 12288
23888 120/300 Read block 757 len 12288
23889 121/300 Write block 52 len 12288 data: 6 6 4
23890 122/300 Write block 3242 len 36864 data: 4 7 5 6 5 4 4 4 6
23891 123/300 Write block 4959 len 24576 data: 4 3 4 4 6 6
23892 124/300 Flush
23893 125/300 Write block 4499 len 8192 data: 7 6
23894 126/300 Flush
23895 127/300 Write block 2127 len 28672 data: 4 2 2 2 3 3 3
23896 128/300 Flush
23897 129/300 Read block 4825 len 32768
23898 130/300 Read block 4483 len 12288
23899 131/300 Read block 4173 len 24576
23900 132/300 Write block 564 len 20480 data: 6 6 6 4 5
23901 133/300 Read block 4375 len 12288
23902 134/300 Flush
23903 135/300 Write block 890 len 20480 data: 3 5 6 6 7
23904 136/300 Write block 2273 len 12288 data: 5 5 4
23905 137/300 Read block 3188 len 16384
23906 138/300 Read block 3496 len 32768
23907 139/300 Write block 3934 len 4096 data: 5
23908 140/300 Read block 1974 len 20480
23909 141/300 Write block 2296 len 8192 data: 5 4
23910 142/300 Read block 4761 len 16384
23911 143/300 Read block 3952 len 24576
23912 144/300 Read block 4672 len 16384
23913 145/300 Write block 1493 len 16384 data: 4 3 4 4
23914 146/300 Read block 652 len 24576
23915 147/300 Read block 1025 len 32768
23916 148/300 Flush
23917 149/300 Read block 181 len 20480
23918 150/300 Write block 37 len 12288 data: 4 4 3
23919 151/300 Write block 2696 len 36864 data: 3 4 4 3 4 4 3 5 4
23920 152/300 Write block 3448 len 40960 data: 5 5 5 5 6 6 6 6 6 6
23921 153/300 Write block 3100 len 4096 data: 6
23922 154/300 Flush
23923 155/300 Read block 213 len 8192
23924 156/300 Write block 704 len 4096 data: 5
23925 157/300 Flush
23926 158/300 Read block 2581 len 40960
23927 159/300 Read block 2272 len 32768
23928 160/300 Write block 2847 len 16384 data: 3 3 3 5
23929 161/300 Read block 2193 len 36864
23930 162/300 Write block 4428 len 16384 data: 8 8 6 6
23931 163/300 Flush
23932 164/300 Write block 771 len 16384 data: 5 5 5 4
23933 165/300 Write block 43 len 4096 data: 7
23934 166/300 Read block 3980 len 32768
23935 167/300 Write block 3272 len 16384 data: 5 7 7 7
23936 168/300 Read block 3388 len 24576
23937 169/300 Read block 4442 len 12288
23938 170/300 Write block 3179 len 24576 data: 4 5 6 8 8 7
23939 171/300 Flush
23940 172/300 Read block 4015 len 24576
23941 173/300 Write block 2705 len 8192 data: 3 4
23942 174/300 Flush
23943 175/300 Flush
23944 176/300 Write block 2682 len 16384 data: 4 6 4 4
23945 177/300 Write block 3706 len 4096 data: 3
23946 178/300 Write block 817 len 40960 data: 3 2 3 5 5 6 6 9 8 6
23947 179/300 Read block 4640 len 20480
23948 180/300 Read block 3778 len 32768
23949 181/300 Write block 2474 len 32768 data: 4 4 5 5 3 5 5 5
23950 182/300 Write block 1846 len 4096 data: 10
23951 183/300 Write block 879 len 24576 data: 5 5 4 4 4 4
23952 184/300 Read block 2798 len 32768
23953 185/300 Write block 2420 len 40960 data: 5 3 3 2 2 3 3 4 5 5
23954 186/300 Write block 2590 len 36864 data: 8 6 6 6 6 6 3 6 6
23955 187/300 Read block 647 len 4096
23956 188/300 Read block 1699 len 4096
23957 189/300 Write block 3910 len 4096 data: 4
23958 190/300 Write block 3578 len 4096 data: 4
23959 191/300 Write block 2916 len 4096 data: 6
23960 192/300 Write block 2067 len 32768 data: 4 4 4 3 5 5 5 4
23961 193/300 Read block 3624 len 20480
23962 194/300 Read block 2051 len 28672
23963 195/300 Write block 4635 len 24576 data: 3 3 3 2 2 2
23964 196/300 Write block 3685 len 12288 data: 4 4 4
23965 197/300 Read block 492 len 36864
23966 198/300 Read block 1344 len 20480
23967 199/300 Read block 2412 len 40960
23968 200/300 Write block 1209 len 24576 data: 8 5 4 5 3 5
23969 201/300 Write block 2244 len 4096 data: 7
23970 202/300 Write block 3881 len 16384 data: 4 4 4 5
23971 203/300 Write block 9 len 20480 data: 6 7 7 5 5
23972 204/300 Read block 4855 len 36864
23973 205/300 Read block 2862 len 12288
23974 206/300 Read block 2137 len 32768
23975 207/300 Write block 132 len 4096 data: 5
23976 208/300 Flush
23977 209/300 Write block 1950 len 20480 data: 3 5 6 5 5
23978 210/300 Write block 2369 len 4096 data: 9
23979 211/300 Write block 4098 len 24576 data: 4 2 2 3 3 3
23980 212/300 Write block 2210 len 28672 data: 5 6 4 4 3 4 4
23981 213/300 Read block 1332 len 40960
23982 214/300 Read block 4391 len 28672
23983 215/300 Read block 454 len 20480
23984 216/300 Read block 3320 len 36864
23985 217/300 Read block 2530 len 20480
23986 218/300 Write block 4043 len 36864 data: 4 3 4 3 3 4 5 6 8
23987 219/300 Flush
23988 220/300 Read block 2180 len 12288
23989 221/300 Write block 457 len 16384 data: 6 8 6 5
23990 222/300 Write block 4997 len 12288 data: 3 3 2
23991 223/300 Read block 2045 len 24576
23992 224/300 Read block 872 len 24576
23993 225/300 Read block 4191 len 8192
23994 226/300 Write block 3624 len 16384 data: 3 3 4 4
23995 227/300 Write block 4849 len 4096 data: 5
23996 228/300 Write block 2377 len 36864 data: 4 3 3 3 3 3 4 5 4
23997 229/300 Write block 933 len 32768 data: 5 3 3 3 4 2 3 3
23998 230/300 Read block 3039 len 20480
23999 231/300 Read block 3804 len 24576
24000 232/300 Read block 1728 len 16384
24001 233/300 Write block 2120 len 12288 data: 2 4 3
24002 234/300 Read block 2419 len 40960
24003 235/300 Read block 3165 len 16384
24004 236/300 Read block 2824 len 12288
24005 237/300 Read block 1942 len 28672
24006 238/300 Read block 2348 len 12288
24007 239/300 Write block 2543 len 4096 data: 5
24008 240/300 Flush
24009 241/300 Flush
24010 242/300 Flush
24011 243/300 Read block 221 len 20480
24012 244/300 Read block 487 len 12288
24013 245/300 Read block 2217 len 36864
24014 246/300 Read block 3586 len 32768
24015 247/300 Read block 3421 len 12288
24016 248/300 Write block 4006 len 8192 data: 5 4
24017 249/300 Read block 3413 len 20480
24018 250/300 Read block 1446 len 4096
24019 251/300 Read block 2860 len 40960
24020 252/300 Read block 455 len 8192
24021 253/300 Write block 4414 len 36864 data: 5 5 5 5 4 3 4 6 6
24022 254/300 Read block 957 len 28672
24023 255/300 Flush
24024 256/300 Read block 359 len 4096
24025 257/300 Read block 2396 len 20480
24026 258/300 Read block 1017 len 40960
24027 259/300 Read block 1780 len 4096
24028 260/300 Write block 4897 len 32768 data: 8 9 7 8 6 6 5 5
24029 261/300 Read block 1416 len 4096
24030 262/300 Flush
24031 263/300 Read block 3079 len 4096
24032 264/300 Write block 4349 len 28672 data: 8 6 6 6 5 5 5
24033 265/300 Write block 922 len 8192 data: 2 2
24034 266/300 Read block 2647 len 36864
24035 267/300 Flush
24036 268/300 Write block 1276 len 24576 data: 3 2 2 2 2 2
24037 269/300 Flush
24038 270/300 Read block 276 len 24576
24039 271/300 Write block 3740 len 12288 data: 8 6 6
24040 272/300 Write block 248 len 28672 data: 5 4 4 5 5 5 7
24041 273/300 Write block 2888 len 20480 data: 3 3 4 4 5
24042 274/300 Write block 2674 len 24576 data: 4 3 3 3 4 4
24043 275/300 Read block 4974 len 32768
24044 276/300 Write block 2052 len 12288 data: 4 5 5
24045 277/300 Read block 2626 len 8192
24046 278/300 Read block 2590 len 8192
24047 279/300 Read block 1608 len 32768
24048 280/300 Write block 1904 len 24576 data: 5 4 3 6 6 6
24049 281/300 Read block 3432 len 20480
24050 282/300 Read block 4024 len 28672
24051 283/300 Write block 3391 len 12288 data: 4 5 4
24052 284/300 Write block 4422 len 40960 data: 7 6 5 8 8 9 9 9 7 7
24053 285/300 Read block 2052 len 24576
24054 286/300 Write block 646 len 16384 data: 5 5 6 6
24055 287/300 Read block 2758 len 24576
24056 288/300 Write block 4996 len 12288 data: 5 4 4
24057 289/300 Read block 4219 len 4096
24058 290/300 Write block 3251 len 24576 data: 6 7 7 4 3 4
24059 291/300 Read block 1718 len 24576
24060 292/300 Read block 2823 len 24576
24061 293/300 Read block 3051 len 24576
24062 294/300 Read block 3932 len 12288
24063 295/300 Write block 4056 len 16384 data: 4 5 3 4
24064 296/300 Flush
24065 297/300 Read block 3517 len 36864
24066 298/300 Read block 9 len 32768
24067 299/300 Read block 940 len 4096
24068 300/300 Read block 4038 len 40960
24069 301/300 Write block 4677 len 24576 data: 8 7 6 5 4 4
24070 Main task has work to do, go find it
24071 got dsc Start(1)
24072 start 1
24073 Jan 15 05:24:04.301 INFO request completed, latency_us: 200, response_code: 204, uri: /start/cid/1, method: POST, req_id: c13b9b5d-9e3b-4b26-ae2a-045631fe303f, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
24074 Replay: started 1, returned:Ok(())
24075 ----------------------------------------------------------------
24076 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24077 States: Active Offline Active
24078 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24079 7341 Acked Read 5 Done Sent Done false
24080 7342 Acked Read 10 Done Sent Done false
24081 7343 Acked Read 2 Done Sent Done false
24082 7344 Acked Read 10 Done Sent Done false
24083 7345 Acked Read 6 Done Sent Done false
24084 7346 Acked Read 3 Done Sent Done false
24085 7347 Acked Write 2 Done Sent Done false
24086 7348 Acked Read 6 Done Sent Done false
24087 7349 Acked Write 8 Done Sent Done false
24088 7350 Acked Flush 0 Done Sent Done false
24089 7351 Acked Read 7 Done Sent Done false
24090 7352 Acked Read 5 Done Sent Done false
24091 7353 Acked Read 9 Done Sent Done false
24092 7354 Acked Write 3 Done Sent Done false
24093 7355 Acked Write 4 Done Sent Done false
24094 7356 Acked Read 2 Done Sent Done false
24095 7357 Acked Write 10 Done Sent Done false
24096 7358 Acked Write 7 Done Sent Done false
24097 7359 Acked Write 2 Done Sent Done false
24098 7360 Acked Flush 0 Done Sent Done false
24099 7361 Acked Read 5 Done Sent Done false
24100 7362 Acked Write 8 Done Sent Done false
24101 7363 Acked Read 7 Done Sent Done false
24102 7364 Acked Read 4 Done Sent Done false
24103 7365 Acked Read 5 Done Sent Done false
24104 7366 Acked Read 4 Done Sent Done false
24105 7367 Acked Write 7 Done Sent Done false
24106 7368 Acked Read 8 Done Sent Done false
24107 7369 Acked Read 3 Done Sent Done false
24108 7370 Acked Write 5 Done Sent Done false
24109 7371 Acked Write 9 Done Sent Done false
24110 7372 Acked Read 8 Done Sent Done false
24111 7373 Acked Write 10 Done Sent Done false
24112 7374 Acked Write 1 Done Sent Done false
24113 7375 Acked Read 6 Done Sent Done false
24114 7376 Acked Read 1 Done Sent Done false
24115 7377 Acked Read 4 Done Sent Done false
24116 7378 Acked Read 3 Done Sent Done false
24117 7379 Acked Write 3 Done Sent Done false
24118 7380 Acked Read 1 Done Sent Done false
24119 7381 Acked Read 1 Done Sent Done false
24120 7382 Acked Read 2 Done Sent Done false
24121 7383 Acked Read 10 Done Sent Done false
24122 7384 Acked Write 10 Done Sent Done false
24123 7385 Acked Write 1 Done Sent Done false
24124 7386 Acked Flush 0 Done Sent Done false
24125 7387 Acked Write 1 Done Sent Done false
24126 7388 Acked Read 10 Done Sent Done false
24127 7389 Acked Write 9 Done Sent Done false
24128 7390 Acked Write 5 Done Sent Done false
24129 7391 Acked Write 3 Done Sent Done false
24130 7392 Acked Read 2 Done Sent Done false
24131 7393 Acked Flush 0 Done Sent Done false
24132 7394 Acked Write 4 Done Sent Done false
24133 7395 Acked Write 1 Done Sent Done false
24134 7396 Acked Flush 0 Done Sent Done false
24135 7397 Acked Read 6 Done Sent Done false
24136 7398 Acked Write 10 Done Sent Done false
24137 7399 Acked Write 2 Done Sent Done false
24138 7400 Acked Write 7 Done Sent Done false
24139 7401 Acked Read 8 Done Sent Done false
24140 7402 Acked Write 7 Done Sent Done false
24141 7403 Acked Read 2 Done Sent Done false
24142 7404 Acked Read 8 Done Sent Done false
24143 7405 Acked Read 7 Done Sent Done false
24144 7406 Acked Read 4 Done Sent Done false
24145 7407 Acked Read 9 Done Sent Done false
24146 7408 Acked Write 6 Done Sent Done false
24147 7409 Acked Write 6 Done Sent Done false
24148 7410 Acked Write 3 Done Sent Done false
24149 7411 Acked Write 3 Done Sent Done false
24150 7412 Acked Write 8 Done Sent Done false
24151 7413 Acked Read 5 Done Sent Done false
24152 7414 Acked Read 4 Done Sent Done false
24153 7415 Acked Flush 0 Done Sent Done false
24154 7416 Acked Read 6 Done Sent Done false
24155 7417 Acked Write 8 Done Sent Done false
24156 7418 Acked Read 4 Done Sent Done false
24157 7419 Acked Flush 0 Done Sent Done false
24158 7420 Acked Read 4 Done Sent Done false
24159 7421 Acked Write 6 Done Sent Done false
24160 7422 Acked Write 2 Done Sent Done false
24161 7423 Acked Flush 0 Done Sent Done false
24162 7424 Acked Flush 0 Done Sent Done false
24163 7425 Acked Flush 0 Done Sent Done false
24164 7426 Acked Write 4 Done Sent Done false
24165 7427 Acked Write 5 Done Sent Done false
24166 7428 Acked Read 6 Done Sent Done false
24167 7429 Acked Read 7 Done Sent Done false
24168 7430 Acked Read 1 Done Sent Done false
24169 7431 Acked Write 10 Done Sent Done false
24170 7432 Acked Write 10 Done Sent Done false
24171 7433 Acked Flush 0 Done Sent Done false
24172 7434 Acked Read 9 Done Sent Done false
24173 7435 Acked Write 2 Done Sent Done false
24174 7436 Acked Read 1 Done Sent Done false
24175 7437 Acked Read 5 Done Sent Done false
24176 7438 Acked Write 2 Done Sent Done false
24177 7439 Acked Write 10 Done Sent Done false
24178 7440 Acked Read 8 Done Sent Done false
24179 7441 Acked Read 8 Done Sent Done false
24180 7442 Acked Write 6 Done Sent Done false
24181 7443 Acked Read 1 Done Sent Done false
24182 7444 Acked Read 6 Done Sent Done false
24183 7445 Acked Write 2 Done Sent Done false
24184 7446 Acked Read 2 Done Sent Done false
24185 7447 Acked Flush 0 Done Sent Done false
24186 7448 Acked Write 10 Done Sent Done false
24187 7449 Acked Flush 0 Done Sent Done false
24188 7450 Acked Read 7 Done Sent Done false
24189 7451 Acked Read 8 Done Sent Done false
24190 7452 Acked Write 1 Done Sent Done false
24191 7453 Acked Write 2 Done Sent Done false
24192 7454 Acked Write 8 Done Sent Done false
24193 7455 Acked Write 7 Done Sent Done false
24194 7456 Acked Write 2 Done Sent Done false
24195 7457 Acked Write 8 Done Sent Done false
24196 7458 Acked Read 2 Done Sent Done false
24197 7459 Acked Read 3 Done Sent Done false
24198 7460 Acked Read 3 Done Sent Done false
24199 7461 Acked Write 3 Done Sent Done false
24200 7462 Acked Write 9 Done Sent Done false
24201 7463 Acked Write 6 Done Sent Done false
24202 7464 Acked Flush 0 Done Sent Done false
24203 7465 Acked Write 2 Done Sent Done false
24204 7466 Acked Flush 0 Done Sent Done false
24205 7467 Acked Write 7 Done Sent Done false
24206 7468 Acked Flush 0 Done Sent Done false
24207 7469 Acked Read 8 Done Sent Done false
24208 7470 Acked Read 3 Done Sent Done false
24209 7471 Acked Read 6 Done Sent Done false
24210 7472 Acked Write 5 Done Sent Done false
24211 7473 Acked Read 3 Done Sent Done false
24212 7474 Acked Flush 0 Done Sent Done false
24213 7475 Acked Write 5 Done Sent Done false
24214 7476 Acked Write 3 Done Sent Done false
24215 7477 Acked Read 4 Done Sent Done false
24216 7478 Acked Read 8 Done Sent Done false
24217 7479 Acked Write 1 Done Sent Done false
24218 7480 Acked Read 5 Done Sent Done false
24219 7481 Acked Write 2 Done Sent Done false
24220 7482 Acked Read 4 Done Sent Done false
24221 7483 Acked Read 6 Done Sent Done false
24222 7484 Acked Read 4 Done Sent Done false
24223 7485 Acked Write 4 Done Sent Done false
24224 7486 Acked Read 6 Done Sent Done false
24225 7487 Acked Read 8 Done Sent Done false
24226 7488 Acked Flush 0 Done Sent Done false
24227 7489 Acked Read 5 Done Sent Done false
24228 7490 Acked Write 3 Done Sent Done false
24229 7491 Acked Write 9 Done Sent Done false
24230 7492 Acked Write 10 Done Sent Done false
24231 7493 Acked Write 1 Done Sent Done false
24232 7494 Acked Flush 0 Done Sent Done false
24233 7495 Acked Read 2 Done Sent Done false
24234 7496 Acked Write 1 Done Sent Done false
24235 7497 Acked Flush 0 Done Sent Done false
24236 7498 Acked Read 10 Done Sent Done false
24237 7499 Acked Read 8 Done Sent Done false
24238 7500 Acked Write 4 Done Sent Done false
24239 7501 Acked Read 9 Done Sent Done false
24240 7502 Acked Write 4 Done Sent Done false
24241 7503 Acked Flush 0 Done Sent Done false
24242 7504 Acked Write 4 Done Sent Done false
24243 7505 Acked Write 1 Done Sent Done false
24244 7506 Acked Read 8 Done Sent Done false
24245 7507 Acked Write 4 Done Sent Done false
24246 7508 Acked Read 6 Done Sent Done false
24247 7509 Acked Read 3 Done Sent Done false
24248 7510 Acked Write 6 Done Sent Done false
24249 7511 Acked Flush 0 Done Sent Done false
24250 7512 Acked Read 6 Done Sent Done false
24251 7513 Acked Write 2 Done Sent Done false
24252 7514 Acked Flush 0 Done Sent Done false
24253 7515 Acked Flush 0 Done Sent Done false
24254 7516 Acked Write 4 Done Sent Done false
24255 7517 Acked Write 1 Done Sent Done false
24256 7518 Acked Write 10 Done Sent Done false
24257 7519 Acked Read 5 Done Sent Done false
24258 7520 Acked Read 8 Done Sent Done false
24259 7521 Acked Write 8 Done Sent Done false
24260 7522 Acked Write 1 Done Sent Done false
24261 7523 Acked Write 6 Done Sent Done false
24262 7524 Acked Read 8 Done Sent Done false
24263 7525 Acked Write 10 Done Sent Done false
24264 7526 Acked Write 9 Done Sent Done false
24265 7527 Acked Read 1 Done Sent Done false
24266 7528 Acked Read 1 Done Sent Done false
24267 7529 Acked Write 1 Done Sent Done false
24268 7530 Acked Write 1 Done Sent Done false
24269 7531 Acked Write 1 Done Sent Done false
24270 7532 Acked Write 8 Done Sent Done false
24271 7533 Acked Read 5 Done Sent Done false
24272 7534 Acked Read 7 Done Sent Done false
24273 7535 Acked Write 6 Done Sent Done false
24274 7536 Acked Write 3 Done Sent Done false
24275 7537 Acked Read 9 Done Sent Done false
24276 7538 Acked Read 5 Done Sent Done false
24277 7539 Acked Read 10 Done Sent Done false
24278 7540 Acked Write 6 Done Sent Done false
24279 7541 Acked Write 1 Done Sent Done false
24280 7542 Acked Write 4 Done Sent Done false
24281 7543 Acked Write 5 Done Sent Done false
24282 7544 Acked Read 9 Done Sent Done false
24283 7545 Acked Read 3 Done Sent Done false
24284 7546 Acked Read 8 Done Sent Done false
24285 7547 Acked Write 1 Done Sent Done false
24286 7548 Acked Flush 0 Done Sent Done false
24287 7549 Acked Write 5 Done Sent Done false
24288 7550 Acked Write 1 Done Sent Done false
24289 7551 Acked Write 6 Done Sent Done false
24290 7552 Acked Write 7 Done Sent Done false
24291 7553 Acked Read 10 Done Sent Done false
24292 7554 Acked Read 7 Done Sent Done false
24293 7555 Acked Read 5 Done Sent Done false
24294 7556 Acked Read 9 Done Sent Done false
24295 7557 Acked Read 5 Done Sent Done false
24296 7558 Acked Write 9 Done Sent Done false
24297 7559 Acked Flush 0 Done Sent Done false
24298 7560 Acked Read 3 Done Sent Done false
24299 7561 Acked Write 4 Done Sent Done false
24300 7562 Acked Write 3 Done Sent Done false
24301 7563 Acked Read 6 Done Sent Done false
24302 7564 Acked Read 6 Done Sent Done false
24303 7565 Acked Read 2 Done Sent Done false
24304 7566 Acked Write 4 Done Sent Done false
24305 7567 Acked Write 1 Done Sent Done false
24306 7568 Acked Write 9 Done Sent Done false
24307 7569 Acked Write 8 Done Sent Done false
24308 7570 Acked Read 5 Done Sent Done false
24309 7571 Acked Read 6 Done Sent Done false
24310 7572 Acked Read 4 Done Sent Done false
24311 7573 Acked Write 3 Done Sent Done false
24312 7574 Acked Read 10 Done Sent Done false
24313 7575 Acked Read 4 Done Sent Done false
24314 7576 Acked Read 3 Done Sent Done false
24315 7577 Acked Read 7 Done Sent Done false
24316 7578 Acked Read 3 Done Sent Done false
24317 7579 Acked Write 1 Done Sent Done false
24318 7580 Acked Flush 0 Done Sent Done false
24319 7581 Acked Flush 0 Done Sent Done false
24320 7582 Acked Flush 0 Done Sent Done false
24321 7583 Acked Read 5 Done Sent Done false
24322 7584 Acked Read 3 Done Sent Done false
24323 7585 Acked Read 9 Done Sent Done false
24324 7586 Acked Read 8 Done Sent Done false
24325 7587 Acked Read 3 Done Sent Done false
24326 7588 Acked Write 2 Done Sent Done false
24327 7589 Acked Read 5 Done Sent Done false
24328 7590 Acked Read 1 Done Sent Done false
24329 7591 Acked Read 10 Done Sent Done false
24330 7592 Acked Read 2 Done Sent Done false
24331 7593 Acked Write 9 Done Sent Done false
24332 7594 Acked Read 7 Done Sent Done false
24333 7595 Acked Flush 0 Done Sent Done false
24334 7596 Acked Read 1 Done Sent Done false
24335 7597 Acked Read 5 Done Sent Done false
24336 7598 Acked Read 10 Done Sent Done false
24337 7599 Acked Read 1 Done Sent Done false
24338 7600 Acked Write 8 Done Sent Done false
24339 7601 Acked Read 1 Done Sent Done false
24340 7602 Acked Flush 0 Done Sent Done false
24341 7603 Acked Read 1 Done Sent Done false
24342 7604 Acked Write 7 Done Sent Done false
24343 7605 Acked Write 2 Done Sent Done false
24344 7606 Acked Read 9 Done Sent Done false
24345 7607 Acked Flush 0 Done Sent Done false
24346 7608 Acked Write 6 Done Sent Done false
24347 7609 Acked Flush 0 Done Sent Done false
24348 7610 Acked Read 6 Done Sent Done false
24349 7611 Acked Write 3 Done Sent Done false
24350 7612 Acked Write 7 Done Sent Done false
24351 7613 Acked Write 5 Done Sent Done false
24352 7614 Acked Write 6 Done Sent Done false
24353 7615 Acked Read 8 Done Sent Done false
24354 7616 Acked Write 3 Done Sent Done false
24355 7617 Acked Read 2 Done Sent Done false
24356 7618 Acked Read 2 Done Sent Done false
24357 7619 Acked Read 8 Done Sent Done false
24358 7620 Acked Write 6 Done Sent Done false
24359 7621 Acked Read 5 Done Sent Done false
24360 7622 Acked Read 7 Done Sent Done false
24361 7623 Acked Write 3 Done Sent Done false
24362 7624 Acked Write 10 Done Sent Done false
24363 7625 Acked Read 6 Done Sent Done false
24364 7626 Acked Write 4 Done Sent Done false
24365 7627 Acked Read 6 Done Sent Done false
24366 7628 Acked Write 3 Done Sent Done false
24367 7629 Acked Read 1 Done Sent Done false
24368 7630 Acked Write 6 Done Sent Done false
24369 7631 Acked Read 6 Done Sent Done false
24370 7632 Acked Read 6 Done Sent Done false
24371 7633 Acked Read 6 Done Sent Done false
24372 7634 Acked Read 3 Done Sent Done false
24373 7635 Acked Write 4 Done Sent Done false
24374 7636 Acked Flush 0 Done Sent Done false
24375 7637 Acked Read 9 Done Sent Done false
24376 7638 Acked Read 8 Done Sent Done false
24377 7639 Acked Read 1 Done Sent Done false
24378 7640 Acked Read 10 Done Sent Done false
24379 7641 Acked Write 6 Done Sent Done false
24380 STATES DS:0 DS:1 DS:2 TOTAL
24381 Sent 0 301 0 301
24382 Done 301 0 301 602
24383 Skipped 0 0 0 0
24384 Error 0 0 0 0
24385 Last Flush: 7636 7340 7636
24386 Downstairs last five completed: 7340 7339 7338 7337 7336
24387 Upstairs last five completed: 7641 7640 7639 7638 7637
24388 CLIENT: Up:0 ds:301 act:2
24389 ----------------------------------------------------------------
24390 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24391 States: Active Offline Active
24392 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24393 7341 Acked Read 5 Done Sent Done false
24394 7342 Acked Read 10 Done Sent Done false
24395 7343 Acked Read 2 Done Sent Done false
24396 7344 Acked Read 10 Done Sent Done false
24397 7345 Acked Read 6 Done Sent Done false
24398 7346 Acked Read 3 Done Sent Done false
24399 7347 Acked Write 2 Done Sent Done false
24400 7348 Acked Read 6 Done Sent Done false
24401 7349 Acked Write 8 Done Sent Done false
24402 7350 Acked Flush 0 Done Sent Done false
24403 7351 Acked Read 7 Done Sent Done false
24404 7352 Acked Read 5 Done Sent Done false
24405 7353 Acked Read 9 Done Sent Done false
24406 7354 Acked Write 3 Done Sent Done false
24407 7355 Acked Write 4 Done Sent Done false
24408 7356 Acked Read 2 Done Sent Done false
24409 7357 Acked Write 10 Done Sent Done false
24410 7358 Acked Write 7 Done Sent Done false
24411 7359 Acked Write 2 Done Sent Done false
24412 7360 Acked Flush 0 Done Sent Done false
24413 7361 Acked Read 5 Done Sent Done false
24414 7362 Acked Write 8 Done Sent Done false
24415 7363 Acked Read 7 Done Sent Done false
24416 7364 Acked Read 4 Done Sent Done false
24417 7365 Acked Read 5 Done Sent Done false
24418 7366 Acked Read 4 Done Sent Done false
24419 7367 Acked Write 7 Done Sent Done false
24420 7368 Acked Read 8 Done Sent Done false
24421 7369 Acked Read 3 Done Sent Done false
24422 7370 Acked Write 5 Done Sent Done false
24423 7371 Acked Write 9 Done Sent Done false
24424 7372 Acked Read 8 Done Sent Done false
24425 7373 Acked Write 10 Done Sent Done false
24426 7374 Acked Write 1 Done Sent Done false
24427 7375 Acked Read 6 Done Sent Done false
24428 7376 Acked Read 1 Done Sent Done false
24429 7377 Acked Read 4 Done Sent Done false
24430 7378 Acked Read 3 Done Sent Done false
24431 7379 Acked Write 3 Done Sent Done false
24432 7380 Acked Read 1 Done Sent Done false
24433 7381 Acked Read 1 Done Sent Done false
24434 7382 Acked Read 2 Done Sent Done false
24435 7383 Acked Read 10 Done Sent Done false
24436 7384 Acked Write 10 Done Sent Done false
24437 7385 Acked Write 1 Done Sent Done false
24438 7386 Acked Flush 0 Done Sent Done false
24439 7387 Acked Write 1 Done Sent Done false
24440 7388 Acked Read 10 Done Sent Done false
24441 7389 Acked Write 9 Done Sent Done false
24442 7390 Acked Write 5 Done Sent Done false
24443 7391 Acked Write 3 Done Sent Done false
24444 7392 Acked Read 2 Done Sent Done false
24445 7393 Acked Flush 0 Done Sent Done false
24446 7394 Acked Write 4 Done Sent Done false
24447 7395 Acked Write 1 Done Sent Done false
24448 7396 Acked Flush 0 Done Sent Done false
24449 7397 Acked Read 6 Done Sent Done false
24450 7398 Acked Write 10 Done Sent Done false
24451 7399 Acked Write 2 Done Sent Done false
24452 7400 Acked Write 7 Done Sent Done false
24453 7401 Acked Read 8 Done Sent Done false
24454 7402 Acked Write 7 Done Sent Done false
24455 7403 Acked Read 2 Done Sent Done false
24456 7404 Acked Read 8 Done Sent Done false
24457 7405 Acked Read 7 Done Sent Done false
24458 7406 Acked Read 4 Done Sent Done false
24459 7407 Acked Read 9 Done Sent Done false
24460 7408 Acked Write 6 Done Sent Done false
24461 7409 Acked Write 6 Done Sent Done false
24462 7410 Acked Write 3 Done Sent Done false
24463 7411 Acked Write 3 Done Sent Done false
24464 7412 Acked Write 8 Done Sent Done false
24465 7413 Acked Read 5 Done Sent Done false
24466 7414 Acked Read 4 Done Sent Done false
24467 7415 Acked Flush 0 Done Sent Done false
24468 7416 Acked Read 6 Done Sent Done false
24469 7417 Acked Write 8 Done Sent Done false
24470 7418 Acked Read 4 Done Sent Done false
24471 7419 Acked Flush 0 Done Sent Done false
24472 7420 Acked Read 4 Done Sent Done false
24473 7421 Acked Write 6 Done Sent Done false
24474 7422 Acked Write 2 Done Sent Done false
24475 7423 Acked Flush 0 Done Sent Done false
24476 7424 Acked Flush 0 Done Sent Done false
24477 7425 Acked Flush 0 Done Sent Done false
24478 7426 Acked Write 4 Done Sent Done false
24479 7427 Acked Write 5 Done Sent Done false
24480 7428 Acked Read 6 Done Sent Done false
24481 7429 Acked Read 7 Done Sent Done false
24482 7430 Acked Read 1 Done Sent Done false
24483 7431 Acked Write 10 Done Sent Done false
24484 7432 Acked Write 10 Done Sent Done false
24485 7433 Acked Flush 0 Done Sent Done false
24486 7434 Acked Read 9 Done Sent Done false
24487 7435 Acked Write 2 Done Sent Done false
24488 7436 Acked Read 1 Done Sent Done false
24489 7437 Acked Read 5 Done Sent Done false
24490 7438 Acked Write 2 Done Sent Done false
24491 7439 Acked Write 10 Done Sent Done false
24492 7440 Acked Read 8 Done Sent Done false
24493 7441 Acked Read 8 Done Sent Done false
24494 7442 Acked Write 6 Done Sent Done false
24495 7443 Acked Read 1 Done Sent Done false
24496 7444 Acked Read 6 Done Sent Done false
24497 7445 Acked Write 2 Done Sent Done false
24498 7446 Acked Read 2 Done Sent Done false
24499 7447 Acked Flush 0 Done Sent Done false
24500 7448 Acked Write 10 Done Sent Done false
24501 7449 Acked Flush 0 Done Sent Done false
24502 7450 Acked Read 7 Done Sent Done false
24503 7451 Acked Read 8 Done Sent Done false
24504 7452 Acked Write 1 Done Sent Done false
24505 7453 Acked Write 2 Done Sent Done false
24506 7454 Acked Write 8 Done Sent Done false
24507 7455 Acked Write 7 Done Sent Done false
24508 7456 Acked Write 2 Done Sent Done false
24509 7457 Acked Write 8 Done Sent Done false
24510 7458 Acked Read 2 Done Sent Done false
24511 7459 Acked Read 3 Done Sent Done false
24512 7460 Acked Read 3 Done Sent Done false
24513 7461 Acked Write 3 Done Sent Done false
24514 7462 Acked Write 9 Done Sent Done false
24515 7463 Acked Write 6 Done Sent Done false
24516 7464 Acked Flush 0 Done Sent Done false
24517 7465 Acked Write 2 Done Sent Done false
24518 7466 Acked Flush 0 Done Sent Done false
24519 7467 Acked Write 7 Done Sent Done false
24520 7468 Acked Flush 0 Done Sent Done false
24521 7469 Acked Read 8 Done Sent Done false
24522 7470 Acked Read 3 Done Sent Done false
24523 7471 Acked Read 6 Done Sent Done false
24524 7472 Acked Write 5 Done Sent Done false
24525 7473 Acked Read 3 Done Sent Done false
24526 7474 Acked Flush 0 Done Sent Done false
24527 7475 Acked Write 5 Done Sent Done false
24528 7476 Acked Write 3 Done Sent Done false
24529 7477 Acked Read 4 Done Sent Done false
24530 7478 Acked Read 8 Done Sent Done false
24531 7479 Acked Write 1 Done Sent Done false
24532 7480 Acked Read 5 Done Sent Done false
24533 7481 Acked Write 2 Done Sent Done false
24534 7482 Acked Read 4 Done Sent Done false
24535 7483 Acked Read 6 Done Sent Done false
24536 7484 Acked Read 4 Done Sent Done false
24537 7485 Acked Write 4 Done Sent Done false
24538 7486 Acked Read 6 Done Sent Done false
24539 7487 Acked Read 8 Done Sent Done false
24540 7488 Acked Flush 0 Done Sent Done false
24541 7489 Acked Read 5 Done Sent Done false
24542 7490 Acked Write 3 Done Sent Done false
24543 7491 Acked Write 9 Done Sent Done false
24544 7492 Acked Write 10 Done Sent Done false
24545 7493 Acked Write 1 Done Sent Done false
24546 7494 Acked Flush 0 Done Sent Done false
24547 7495 Acked Read 2 Done Sent Done false
24548 7496 Acked Write 1 Done Sent Done false
24549 7497 Acked Flush 0 Done Sent Done false
24550 7498 Acked Read 10 Done Sent Done false
24551 7499 Acked Read 8 Done Sent Done false
24552 7500 Acked Write 4 Done Sent Done false
24553 7501 Acked Read 9 Done Sent Done false
24554 7502 Acked Write 4 Done Sent Done false
24555 7503 Acked Flush 0 Done Sent Done false
24556 7504 Acked Write 4 Done Sent Done false
24557 7505 Acked Write 1 Done Sent Done false
24558 7506 Acked Read 8 Done Sent Done false
24559 7507 Acked Write 4 Done Sent Done false
24560 7508 Acked Read 6 Done Sent Done false
24561 7509 Acked Read 3 Done Sent Done false
24562 7510 Acked Write 6 Done Sent Done false
24563 7511 Acked Flush 0 Done Sent Done false
24564 7512 Acked Read 6 Done Sent Done false
24565 7513 Acked Write 2 Done Sent Done false
24566 7514 Acked Flush 0 Done Sent Done false
24567 7515 Acked Flush 0 Done Sent Done false
24568 7516 Acked Write 4 Done Sent Done false
24569 7517 Acked Write 1 Done Sent Done false
24570 7518 Acked Write 10 Done Sent Done false
24571 7519 Acked Read 5 Done Sent Done false
24572 7520 Acked Read 8 Done Sent Done false
24573 7521 Acked Write 8 Done Sent Done false
24574 7522 Acked Write 1 Done Sent Done false
24575 7523 Acked Write 6 Done Sent Done false
24576 7524 Acked Read 8 Done Sent Done false
24577 7525 Acked Write 10 Done Sent Done false
24578 7526 Acked Write 9 Done Sent Done false
24579 7527 Acked Read 1 Done Sent Done false
24580 7528 Acked Read 1 Done Sent Done false
24581 7529 Acked Write 1 Done Sent Done false
24582 7530 Acked Write 1 Done Sent Done false
24583 7531 Acked Write 1 Done Sent Done false
24584 7532 Acked Write 8 Done Sent Done false
24585 7533 Acked Read 5 Done Sent Done false
24586 7534 Acked Read 7 Done Sent Done false
24587 7535 Acked Write 6 Done Sent Done false
24588 7536 Acked Write 3 Done Sent Done false
24589 7537 Acked Read 9 Done Sent Done false
24590 7538 Acked Read 5 Done Sent Done false
24591 7539 Acked Read 10 Done Sent Done false
24592 7540 Acked Write 6 Done Sent Done false
24593 7541 Acked Write 1 Done Sent Done false
24594 7542 Acked Write 4 Done Sent Done false
24595 7543 Acked Write 5 Done Sent Done false
24596 7544 Acked Read 9 Done Sent Done false
24597 7545 Acked Read 3 Done Sent Done false
24598 7546 Acked Read 8 Done Sent Done false
24599 7547 Acked Write 1 Done Sent Done false
24600 7548 Acked Flush 0 Done Sent Done false
24601 7549 Acked Write 5 Done Sent Done false
24602 7550 Acked Write 1 Done Sent Done false
24603 7551 Acked Write 6 Done Sent Done false
24604 7552 Acked Write 7 Done Sent Done false
24605 7553 Acked Read 10 Done Sent Done false
24606 7554 Acked Read 7 Done Sent Done false
24607 7555 Acked Read 5 Done Sent Done false
24608 7556 Acked Read 9 Done Sent Done false
24609 7557 Acked Read 5 Done Sent Done false
24610 7558 Acked Write 9 Done Sent Done false
24611 7559 Acked Flush 0 Done Sent Done false
24612 7560 Acked Read 3 Done Sent Done false
24613 7561 Acked Write 4 Done Sent Done false
24614 7562 Acked Write 3 Done Sent Done false
24615 7563 Acked Read 6 Done Sent Done false
24616 7564 Acked Read 6 Done Sent Done false
24617 7565 Acked Read 2 Done Sent Done false
24618 7566 Acked Write 4 Done Sent Done false
24619 7567 Acked Write 1 Done Sent Done false
24620 7568 Acked Write 9 Done Sent Done false
24621 7569 Acked Write 8 Done Sent Done false
24622 7570 Acked Read 5 Done Sent Done false
24623 7571 Acked Read 6 Done Sent Done false
24624 7572 Acked Read 4 Done Sent Done false
24625 7573 Acked Write 3 Done Sent Done false
24626 7574 Acked Read 10 Done Sent Done false
24627 7575 Acked Read 4 Done Sent Done false
24628 7576 Acked Read 3 Done Sent Done false
24629 7577 Acked Read 7 Done Sent Done false
24630 7578 Acked Read 3 Done Sent Done false
24631 7579 Acked Write 1 Done Sent Done false
24632 7580 Acked Flush 0 Done Sent Done false
24633 7581 Acked Flush 0 Done Sent Done false
24634 7582 Acked Flush 0 Done Sent Done false
24635 7583 Acked Read 5 Done Sent Done false
24636 7584 Acked Read 3 Done Sent Done false
24637 7585 Acked Read 9 Done Sent Done false
24638 7586 Acked Read 8 Done Sent Done false
24639 7587 Acked Read 3 Done Sent Done false
24640 7588 Acked Write 2 Done Sent Done false
24641 7589 Acked Read 5 Done Sent Done false
24642 7590 Acked Read 1 Done Sent Done false
24643 7591 Acked Read 10 Done Sent Done false
24644 7592 Acked Read 2 Done Sent Done false
24645 7593 Acked Write 9 Done Sent Done false
24646 7594 Acked Read 7 Done Sent Done false
24647 7595 Acked Flush 0 Done Sent Done false
24648 7596 Acked Read 1 Done Sent Done false
24649 7597 Acked Read 5 Done Sent Done false
24650 7598 Acked Read 10 Done Sent Done false
24651 7599 Acked Read 1 Done Sent Done false
24652 7600 Acked Write 8 Done Sent Done false
24653 7601 Acked Read 1 Done Sent Done false
24654 7602 Acked Flush 0 Done Sent Done false
24655 7603 Acked Read 1 Done Sent Done false
24656 7604 Acked Write 7 Done Sent Done false
24657 7605 Acked Write 2 Done Sent Done false
24658 7606 Acked Read 9 Done Sent Done false
24659 7607 Acked Flush 0 Done Sent Done false
24660 7608 Acked Write 6 Done Sent Done false
24661 7609 Acked Flush 0 Done Sent Done false
24662 7610 Acked Read 6 Done Sent Done false
24663 7611 Acked Write 3 Done Sent Done false
24664 7612 Acked Write 7 Done Sent Done false
24665 7613 Acked Write 5 Done Sent Done false
24666 7614 Acked Write 6 Done Sent Done false
24667 7615 Acked Read 8 Done Sent Done false
24668 7616 Acked Write 3 Done Sent Done false
24669 7617 Acked Read 2 Done Sent Done false
24670 7618 Acked Read 2 Done Sent Done false
24671 7619 Acked Read 8 Done Sent Done false
24672 7620 Acked Write 6 Done Sent Done false
24673 7621 Acked Read 5 Done Sent Done false
24674 7622 Acked Read 7 Done Sent Done false
24675 7623 Acked Write 3 Done Sent Done false
24676 7624 Acked Write 10 Done Sent Done false
24677 7625 Acked Read 6 Done Sent Done false
24678 7626 Acked Write 4 Done Sent Done false
24679 7627 Acked Read 6 Done Sent Done false
24680 7628 Acked Write 3 Done Sent Done false
24681 7629 Acked Read 1 Done Sent Done false
24682 7630 Acked Write 6 Done Sent Done false
24683 7631 Acked Read 6 Done Sent Done false
24684 7632 Acked Read 6 Done Sent Done false
24685 7633 Acked Read 6 Done Sent Done false
24686 7634 Acked Read 3 Done Sent Done false
24687 7635 Acked Write 4 Done Sent Done false
24688 7636 Acked Flush 0 Done Sent Done false
24689 7637 Acked Read 9 Done Sent Done false
24690 7638 Acked Read 8 Done Sent Done false
24691 7639 Acked Read 1 Done Sent Done false
24692 7640 Acked Read 10 Done Sent Done false
24693 7641 Acked Write 6 Done Sent Done false
24694 7642 Acked Flush 0 Done Sent Done false
24695 STATES DS:0 DS:1 DS:2 TOTAL
24696 Sent 0 302 0 302
24697 Done 302 0 302 604
24698 Skipped 0 0 0 0
24699 Error 0 0 0 0
24700 Last Flush: 7642 7340 7642
24701 Downstairs last five completed: 7340 7339 7338 7337 7336
24702 Upstairs last five completed: 7642 7641 7640 7639 7638
24703 CLIENT: Up:0 ds:302 act:2
247042025-01-15T05:24:10.076ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
247052025-01-15T05:24:10.076ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
247062025-01-15T05:24:10.076ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
247072025-01-15T05:24:10.076ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
247082025-01-15T05:24:10.076ZINFOcrucible: client task is exiting = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
24709 [8820] got start action
24710 [8820] I am going to restart
24711 Starting downstairs at port 8820
24712 Make output file at "/tmp/dsc/downstairs-8820.txt"
24713 [8820][1] reports Starting
24714 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1224)
24715 [8820][1] reports Running
24716 ----------------------------------------------------------------
24717 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24718 States: Active Offline Active
24719 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24720 7341 Acked Read 5 Done Sent Done false
24721 7342 Acked Read 10 Done Sent Done false
24722 7343 Acked Read 2 Done Sent Done false
24723 7344 Acked Read 10 Done Sent Done false
24724 7345 Acked Read 6 Done Sent Done false
24725 7346 Acked Read 3 Done Sent Done false
24726 7347 Acked Write 2 Done Sent Done false
24727 7348 Acked Read 6 Done Sent Done false
24728 7349 Acked Write 8 Done Sent Done false
24729 7350 Acked Flush 0 Done Sent Done false
24730 7351 Acked Read 7 Done Sent Done false
24731 7352 Acked Read 5 Done Sent Done false
24732 7353 Acked Read 9 Done Sent Done false
24733 7354 Acked Write 3 Done Sent Done false
24734 7355 Acked Write 4 Done Sent Done false
24735 7356 Acked Read 2 Done Sent Done false
24736 7357 Acked Write 10 Done Sent Done false
24737 7358 Acked Write 7 Done Sent Done false
24738 7359 Acked Write 2 Done Sent Done false
24739 7360 Acked Flush 0 Done Sent Done false
24740 7361 Acked Read 5 Done Sent Done false
24741 7362 Acked Write 8 Done Sent Done false
24742 7363 Acked Read 7 Done Sent Done false
24743 7364 Acked Read 4 Done Sent Done false
24744 7365 Acked Read 5 Done Sent Done false
24745 7366 Acked Read 4 Done Sent Done false
24746 7367 Acked Write 7 Done Sent Done false
24747 7368 Acked Read 8 Done Sent Done false
24748 7369 Acked Read 3 Done Sent Done false
24749 7370 Acked Write 5 Done Sent Done false
24750 7371 Acked Write 9 Done Sent Done false
24751 7372 Acked Read 8 Done Sent Done false
24752 7373 Acked Write 10 Done Sent Done false
24753 7374 Acked Write 1 Done Sent Done false
24754 7375 Acked Read 6 Done Sent Done false
24755 7376 Acked Read 1 Done Sent Done false
24756 7377 Acked Read 4 Done Sent Done false
24757 7378 Acked Read 3 Done Sent Done false
24758 7379 Acked Write 3 Done Sent Done false
24759 7380 Acked Read 1 Done Sent Done false
24760 7381 Acked Read 1 Done Sent Done false
24761 7382 Acked Read 2 Done Sent Done false
24762 7383 Acked Read 10 Done Sent Done false
24763 7384 Acked Write 10 Done Sent Done false
24764 7385 Acked Write 1 Done Sent Done false
24765 7386 Acked Flush 0 Done Sent Done false
24766 7387 Acked Write 1 Done Sent Done false
24767 7388 Acked Read 10 Done Sent Done false
24768 7389 Acked Write 9 Done Sent Done false
24769 7390 Acked Write 5 Done Sent Done false
24770 7391 Acked Write 3 Done Sent Done false
24771 7392 Acked Read 2 Done Sent Done false
24772 7393 Acked Flush 0 Done Sent Done false
24773 7394 Acked Write 4 Done Sent Done false
24774 7395 Acked Write 1 Done Sent Done false
24775 7396 Acked Flush 0 Done Sent Done false
24776 7397 Acked Read 6 Done Sent Done false
24777 7398 Acked Write 10 Done Sent Done false
24778 7399 Acked Write 2 Done Sent Done false
24779 7400 Acked Write 7 Done Sent Done false
24780 7401 Acked Read 8 Done Sent Done false
24781 7402 Acked Write 7 Done Sent Done false
24782 7403 Acked Read 2 Done Sent Done false
24783 7404 Acked Read 8 Done Sent Done false
24784 7405 Acked Read 7 Done Sent Done false
24785 7406 Acked Read 4 Done Sent Done false
24786 7407 Acked Read 9 Done Sent Done false
24787 7408 Acked Write 6 Done Sent Done false
24788 7409 Acked Write 6 Done Sent Done false
24789 7410 Acked Write 3 Done Sent Done false
24790 7411 Acked Write 3 Done Sent Done false
24791 7412 Acked Write 8 Done Sent Done false
24792 7413 Acked Read 5 Done Sent Done false
24793 7414 Acked Read 4 Done Sent Done false
24794 7415 Acked Flush 0 Done Sent Done false
24795 7416 Acked Read 6 Done Sent Done false
24796 7417 Acked Write 8 Done Sent Done false
24797 7418 Acked Read 4 Done Sent Done false
24798 7419 Acked Flush 0 Done Sent Done false
24799 7420 Acked Read 4 Done Sent Done false
24800 7421 Acked Write 6 Done Sent Done false
24801 7422 Acked Write 2 Done Sent Done false
24802 7423 Acked Flush 0 Done Sent Done false
24803 7424 Acked Flush 0 Done Sent Done false
24804 7425 Acked Flush 0 Done Sent Done false
24805 7426 Acked Write 4 Done Sent Done false
24806 7427 Acked Write 5 Done Sent Done false
24807 7428 Acked Read 6 Done Sent Done false
24808 7429 Acked Read 7 Done Sent Done false
24809 7430 Acked Read 1 Done Sent Done false
24810 7431 Acked Write 10 Done Sent Done false
24811 7432 Acked Write 10 Done Sent Done false
24812 7433 Acked Flush 0 Done Sent Done false
24813 7434 Acked Read 9 Done Sent Done false
24814 7435 Acked Write 2 Done Sent Done false
24815 7436 Acked Read 1 Done Sent Done false
24816 7437 Acked Read 5 Done Sent Done false
24817 7438 Acked Write 2 Done Sent Done false
24818 7439 Acked Write 10 Done Sent Done false
24819 7440 Acked Read 8 Done Sent Done false
24820 7441 Acked Read 8 Done Sent Done false
24821 7442 Acked Write 6 Done Sent Done false
24822 7443 Acked Read 1 Done Sent Done false
24823 7444 Acked Read 6 Done Sent Done false
24824 7445 Acked Write 2 Done Sent Done false
24825 7446 Acked Read 2 Done Sent Done false
24826 7447 Acked Flush 0 Done Sent Done false
24827 7448 Acked Write 10 Done Sent Done false
24828 7449 Acked Flush 0 Done Sent Done false
24829 7450 Acked Read 7 Done Sent Done false
24830 7451 Acked Read 8 Done Sent Done false
24831 7452 Acked Write 1 Done Sent Done false
24832 7453 Acked Write 2 Done Sent Done false
24833 7454 Acked Write 8 Done Sent Done false
24834 7455 Acked Write 7 Done Sent Done false
24835 7456 Acked Write 2 Done Sent Done false
24836 7457 Acked Write 8 Done Sent Done false
24837 7458 Acked Read 2 Done Sent Done false
24838 7459 Acked Read 3 Done Sent Done false
24839 7460 Acked Read 3 Done Sent Done false
24840 7461 Acked Write 3 Done Sent Done false
24841 7462 Acked Write 9 Done Sent Done false
24842 7463 Acked Write 6 Done Sent Done false
24843 7464 Acked Flush 0 Done Sent Done false
24844 7465 Acked Write 2 Done Sent Done false
24845 7466 Acked Flush 0 Done Sent Done false
24846 7467 Acked Write 7 Done Sent Done false
24847 7468 Acked Flush 0 Done Sent Done false
24848 7469 Acked Read 8 Done Sent Done false
24849 7470 Acked Read 3 Done Sent Done false
24850 7471 Acked Read 6 Done Sent Done false
24851 7472 Acked Write 5 Done Sent Done false
24852 7473 Acked Read 3 Done Sent Done false
24853 7474 Acked Flush 0 Done Sent Done false
24854 7475 Acked Write 5 Done Sent Done false
24855 7476 Acked Write 3 Done Sent Done false
24856 7477 Acked Read 4 Done Sent Done false
24857 7478 Acked Read 8 Done Sent Done false
24858 7479 Acked Write 1 Done Sent Done false
24859 7480 Acked Read 5 Done Sent Done false
24860 7481 Acked Write 2 Done Sent Done false
24861 7482 Acked Read 4 Done Sent Done false
24862 7483 Acked Read 6 Done Sent Done false
24863 7484 Acked Read 4 Done Sent Done false
24864 7485 Acked Write 4 Done Sent Done false
24865 7486 Acked Read 6 Done Sent Done false
24866 7487 Acked Read 8 Done Sent Done false
24867 7488 Acked Flush 0 Done Sent Done false
24868 7489 Acked Read 5 Done Sent Done false
24869 7490 Acked Write 3 Done Sent Done false
24870 7491 Acked Write 9 Done Sent Done false
24871 7492 Acked Write 10 Done Sent Done false
24872 7493 Acked Write 1 Done Sent Done false
24873 7494 Acked Flush 0 Done Sent Done false
24874 7495 Acked Read 2 Done Sent Done false
24875 7496 Acked Write 1 Done Sent Done false
24876 7497 Acked Flush 0 Done Sent Done false
24877 7498 Acked Read 10 Done Sent Done false
24878 7499 Acked Read 8 Done Sent Done false
24879 7500 Acked Write 4 Done Sent Done false
24880 7501 Acked Read 9 Done Sent Done false
24881 7502 Acked Write 4 Done Sent Done false
24882 7503 Acked Flush 0 Done Sent Done false
24883 7504 Acked Write 4 Done Sent Done false
24884 7505 Acked Write 1 Done Sent Done false
24885 7506 Acked Read 8 Done Sent Done false
24886 7507 Acked Write 4 Done Sent Done false
24887 7508 Acked Read 6 Done Sent Done false
24888 7509 Acked Read 3 Done Sent Done false
24889 7510 Acked Write 6 Done Sent Done false
24890 7511 Acked Flush 0 Done Sent Done false
24891 7512 Acked Read 6 Done Sent Done false
24892 7513 Acked Write 2 Done Sent Done false
24893 7514 Acked Flush 0 Done Sent Done false
24894 7515 Acked Flush 0 Done Sent Done false
24895 7516 Acked Write 4 Done Sent Done false
24896 7517 Acked Write 1 Done Sent Done false
24897 7518 Acked Write 10 Done Sent Done false
24898 7519 Acked Read 5 Done Sent Done false
24899 7520 Acked Read 8 Done Sent Done false
24900 7521 Acked Write 8 Done Sent Done false
24901 7522 Acked Write 1 Done Sent Done false
24902 7523 Acked Write 6 Done Sent Done false
24903 7524 Acked Read 8 Done Sent Done false
24904 7525 Acked Write 10 Done Sent Done false
24905 7526 Acked Write 9 Done Sent Done false
24906 7527 Acked Read 1 Done Sent Done false
24907 7528 Acked Read 1 Done Sent Done false
24908 7529 Acked Write 1 Done Sent Done false
24909 7530 Acked Write 1 Done Sent Done false
24910 7531 Acked Write 1 Done Sent Done false
24911 7532 Acked Write 8 Done Sent Done false
24912 7533 Acked Read 5 Done Sent Done false
24913 7534 Acked Read 7 Done Sent Done false
24914 7535 Acked Write 6 Done Sent Done false
24915 7536 Acked Write 3 Done Sent Done false
24916 7537 Acked Read 9 Done Sent Done false
24917 7538 Acked Read 5 Done Sent Done false
24918 7539 Acked Read 10 Done Sent Done false
24919 7540 Acked Write 6 Done Sent Done false
24920 7541 Acked Write 1 Done Sent Done false
24921 7542 Acked Write 4 Done Sent Done false
24922 7543 Acked Write 5 Done Sent Done false
24923 7544 Acked Read 9 Done Sent Done false
24924 7545 Acked Read 3 Done Sent Done false
24925 7546 Acked Read 8 Done Sent Done false
24926 7547 Acked Write 1 Done Sent Done false
24927 7548 Acked Flush 0 Done Sent Done false
24928 7549 Acked Write 5 Done Sent Done false
24929 7550 Acked Write 1 Done Sent Done false
24930 7551 Acked Write 6 Done Sent Done false
24931 7552 Acked Write 7 Done Sent Done false
24932 7553 Acked Read 10 Done Sent Done false
24933 7554 Acked Read 7 Done Sent Done false
24934 7555 Acked Read 5 Done Sent Done false
24935 7556 Acked Read 9 Done Sent Done false
24936 7557 Acked Read 5 Done Sent Done false
24937 7558 Acked Write 9 Done Sent Done false
24938 7559 Acked Flush 0 Done Sent Done false
24939 7560 Acked Read 3 Done Sent Done false
24940 7561 Acked Write 4 Done Sent Done false
24941 7562 Acked Write 3 Done Sent Done false
24942 7563 Acked Read 6 Done Sent Done false
24943 7564 Acked Read 6 Done Sent Done false
24944 7565 Acked Read 2 Done Sent Done false
24945 7566 Acked Write 4 Done Sent Done false
24946 7567 Acked Write 1 Done Sent Done false
24947 7568 Acked Write 9 Done Sent Done false
24948 7569 Acked Write 8 Done Sent Done false
24949 7570 Acked Read 5 Done Sent Done false
24950 7571 Acked Read 6 Done Sent Done false
24951 7572 Acked Read 4 Done Sent Done false
24952 7573 Acked Write 3 Done Sent Done false
24953 7574 Acked Read 10 Done Sent Done false
24954 7575 Acked Read 4 Done Sent Done false
24955 7576 Acked Read 3 Done Sent Done false
24956 7577 Acked Read 7 Done Sent Done false
24957 7578 Acked Read 3 Done Sent Done false
24958 7579 Acked Write 1 Done Sent Done false
24959 7580 Acked Flush 0 Done Sent Done false
24960 7581 Acked Flush 0 Done Sent Done false
24961 7582 Acked Flush 0 Done Sent Done false
24962 7583 Acked Read 5 Done Sent Done false
24963 7584 Acked Read 3 Done Sent Done false
24964 7585 Acked Read 9 Done Sent Done false
24965 7586 Acked Read 8 Done Sent Done false
24966 7587 Acked Read 3 Done Sent Done false
24967 7588 Acked Write 2 Done Sent Done false
24968 7589 Acked Read 5 Done Sent Done false
24969 7590 Acked Read 1 Done Sent Done false
24970 7591 Acked Read 10 Done Sent Done false
24971 7592 Acked Read 2 Done Sent Done false
24972 7593 Acked Write 9 Done Sent Done false
24973 7594 Acked Read 7 Done Sent Done false
24974 7595 Acked Flush 0 Done Sent Done false
24975 7596 Acked Read 1 Done Sent Done false
24976 7597 Acked Read 5 Done Sent Done false
24977 7598 Acked Read 10 Done Sent Done false
24978 7599 Acked Read 1 Done Sent Done false
24979 7600 Acked Write 8 Done Sent Done false
24980 7601 Acked Read 1 Done Sent Done false
24981 7602 Acked Flush 0 Done Sent Done false
24982 7603 Acked Read 1 Done Sent Done false
24983 7604 Acked Write 7 Done Sent Done false
24984 7605 Acked Write 2 Done Sent Done false
24985 7606 Acked Read 9 Done Sent Done false
24986 7607 Acked Flush 0 Done Sent Done false
24987 7608 Acked Write 6 Done Sent Done false
24988 7609 Acked Flush 0 Done Sent Done false
24989 7610 Acked Read 6 Done Sent Done false
24990 7611 Acked Write 3 Done Sent Done false
24991 7612 Acked Write 7 Done Sent Done false
24992 7613 Acked Write 5 Done Sent Done false
24993 7614 Acked Write 6 Done Sent Done false
24994 7615 Acked Read 8 Done Sent Done false
24995 7616 Acked Write 3 Done Sent Done false
24996 7617 Acked Read 2 Done Sent Done false
24997 7618 Acked Read 2 Done Sent Done false
24998 7619 Acked Read 8 Done Sent Done false
24999 7620 Acked Write 6 Done Sent Done false
25000 7621 Acked Read 5 Done Sent Done false
25001 7622 Acked Read 7 Done Sent Done false
25002 7623 Acked Write 3 Done Sent Done false
25003 7624 Acked Write 10 Done Sent Done false
25004 7625 Acked Read 6 Done Sent Done false
25005 7626 Acked Write 4 Done Sent Done false
25006 7627 Acked Read 6 Done Sent Done false
25007 7628 Acked Write 3 Done Sent Done false
25008 7629 Acked Read 1 Done Sent Done false
25009 7630 Acked Write 6 Done Sent Done false
25010 7631 Acked Read 6 Done Sent Done false
25011 7632 Acked Read 6 Done Sent Done false
25012 7633 Acked Read 6 Done Sent Done false
25013 7634 Acked Read 3 Done Sent Done false
25014 7635 Acked Write 4 Done Sent Done false
25015 7636 Acked Flush 0 Done Sent Done false
25016 7637 Acked Read 9 Done Sent Done false
25017 7638 Acked Read 8 Done Sent Done false
25018 7639 Acked Read 1 Done Sent Done false
25019 7640 Acked Read 10 Done Sent Done false
25020 7641 Acked Write 6 Done Sent Done false
25021 7642 Acked Flush 0 Done Sent Done false
25022 STATES DS:0 DS:1 DS:2 TOTAL
25023 Sent 0 302 0 302
25024 Done 302 0 302 604
25025 Skipped 0 0 0 0
25026 Error 0 0 0 0
25027 Last Flush: 7642 7340 7642
25028 Downstairs last five completed: 7340 7339 7338 7337 7336
25029 Upstairs last five completed: 7642 7641 7640 7639 7638
25030 CLIENT: Up:0 ds:302 act:2
25031 ----------------------------------------------------------------
25032 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25033 States: Active Offline Active
25034 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25035 7341 Acked Read 5 Done Sent Done false
25036 7342 Acked Read 10 Done Sent Done false
25037 7343 Acked Read 2 Done Sent Done false
25038 7344 Acked Read 10 Done Sent Done false
25039 7345 Acked Read 6 Done Sent Done false
25040 7346 Acked Read 3 Done Sent Done false
25041 7347 Acked Write 2 Done Sent Done false
25042 7348 Acked Read 6 Done Sent Done false
25043 7349 Acked Write 8 Done Sent Done false
25044 7350 Acked Flush 0 Done Sent Done false
25045 7351 Acked Read 7 Done Sent Done false
25046 7352 Acked Read 5 Done Sent Done false
25047 7353 Acked Read 9 Done Sent Done false
25048 7354 Acked Write 3 Done Sent Done false
25049 7355 Acked Write 4 Done Sent Done false
25050 7356 Acked Read 2 Done Sent Done false
25051 7357 Acked Write 10 Done Sent Done false
25052 7358 Acked Write 7 Done Sent Done false
25053 7359 Acked Write 2 Done Sent Done false
25054 7360 Acked Flush 0 Done Sent Done false
25055 7361 Acked Read 5 Done Sent Done false
25056 7362 Acked Write 8 Done Sent Done false
25057 7363 Acked Read 7 Done Sent Done false
25058 7364 Acked Read 4 Done Sent Done false
25059 7365 Acked Read 5 Done Sent Done false
25060 7366 Acked Read 4 Done Sent Done false
25061 7367 Acked Write 7 Done Sent Done false
25062 7368 Acked Read 8 Done Sent Done false
25063 7369 Acked Read 3 Done Sent Done false
25064 7370 Acked Write 5 Done Sent Done false
25065 7371 Acked Write 9 Done Sent Done false
25066 7372 Acked Read 8 Done Sent Done false
25067 7373 Acked Write 10 Done Sent Done false
25068 7374 Acked Write 1 Done Sent Done false
25069 7375 Acked Read 6 Done Sent Done false
25070 7376 Acked Read 1 Done Sent Done false
25071 7377 Acked Read 4 Done Sent Done false
25072 7378 Acked Read 3 Done Sent Done false
25073 7379 Acked Write 3 Done Sent Done false
25074 7380 Acked Read 1 Done Sent Done false
25075 7381 Acked Read 1 Done Sent Done false
25076 7382 Acked Read 2 Done Sent Done false
25077 7383 Acked Read 10 Done Sent Done false
25078 7384 Acked Write 10 Done Sent Done false
25079 7385 Acked Write 1 Done Sent Done false
25080 7386 Acked Flush 0 Done Sent Done false
25081 7387 Acked Write 1 Done Sent Done false
25082 7388 Acked Read 10 Done Sent Done false
25083 7389 Acked Write 9 Done Sent Done false
25084 7390 Acked Write 5 Done Sent Done false
25085 7391 Acked Write 3 Done Sent Done false
25086 7392 Acked Read 2 Done Sent Done false
25087 7393 Acked Flush 0 Done Sent Done false
25088 7394 Acked Write 4 Done Sent Done false
25089 7395 Acked Write 1 Done Sent Done false
25090 7396 Acked Flush 0 Done Sent Done false
25091 7397 Acked Read 6 Done Sent Done false
25092 7398 Acked Write 10 Done Sent Done false
25093 7399 Acked Write 2 Done Sent Done false
25094 7400 Acked Write 7 Done Sent Done false
25095 7401 Acked Read 8 Done Sent Done false
25096 7402 Acked Write 7 Done Sent Done false
25097 7403 Acked Read 2 Done Sent Done false
25098 7404 Acked Read 8 Done Sent Done false
25099 7405 Acked Read 7 Done Sent Done false
25100 7406 Acked Read 4 Done Sent Done false
25101 7407 Acked Read 9 Done Sent Done false
25102 7408 Acked Write 6 Done Sent Done false
25103 7409 Acked Write 6 Done Sent Done false
25104 7410 Acked Write 3 Done Sent Done false
25105 7411 Acked Write 3 Done Sent Done false
25106 7412 Acked Write 8 Done Sent Done false
25107 7413 Acked Read 5 Done Sent Done false
25108 7414 Acked Read 4 Done Sent Done false
25109 7415 Acked Flush 0 Done Sent Done false
25110 7416 Acked Read 6 Done Sent Done false
25111 7417 Acked Write 8 Done Sent Done false
25112 7418 Acked Read 4 Done Sent Done false
25113 7419 Acked Flush 0 Done Sent Done false
25114 7420 Acked Read 4 Done Sent Done false
25115 7421 Acked Write 6 Done Sent Done false
25116 7422 Acked Write 2 Done Sent Done false
25117 7423 Acked Flush 0 Done Sent Done false
25118 7424 Acked Flush 0 Done Sent Done false
25119 7425 Acked Flush 0 Done Sent Done false
25120 7426 Acked Write 4 Done Sent Done false
25121 7427 Acked Write 5 Done Sent Done false
25122 7428 Acked Read 6 Done Sent Done false
25123 7429 Acked Read 7 Done Sent Done false
25124 7430 Acked Read 1 Done Sent Done false
25125 7431 Acked Write 10 Done Sent Done false
25126 7432 Acked Write 10 Done Sent Done false
25127 7433 Acked Flush 0 Done Sent Done false
25128 7434 Acked Read 9 Done Sent Done false
25129 7435 Acked Write 2 Done Sent Done false
25130 7436 Acked Read 1 Done Sent Done false
25131 7437 Acked Read 5 Done Sent Done false
25132 7438 Acked Write 2 Done Sent Done false
25133 7439 Acked Write 10 Done Sent Done false
25134 7440 Acked Read 8 Done Sent Done false
25135 7441 Acked Read 8 Done Sent Done false
25136 7442 Acked Write 6 Done Sent Done false
25137 7443 Acked Read 1 Done Sent Done false
25138 7444 Acked Read 6 Done Sent Done false
25139 7445 Acked Write 2 Done Sent Done false
25140 7446 Acked Read 2 Done Sent Done false
25141 7447 Acked Flush 0 Done Sent Done false
25142 7448 Acked Write 10 Done Sent Done false
25143 7449 Acked Flush 0 Done Sent Done false
25144 7450 Acked Read 7 Done Sent Done false
25145 7451 Acked Read 8 Done Sent Done false
25146 7452 Acked Write 1 Done Sent Done false
25147 7453 Acked Write 2 Done Sent Done false
25148 7454 Acked Write 8 Done Sent Done false
25149 7455 Acked Write 7 Done Sent Done false
25150 7456 Acked Write 2 Done Sent Done false
25151 7457 Acked Write 8 Done Sent Done false
25152 7458 Acked Read 2 Done Sent Done false
25153 7459 Acked Read 3 Done Sent Done false
25154 7460 Acked Read 3 Done Sent Done false
25155 7461 Acked Write 3 Done Sent Done false
25156 7462 Acked Write 9 Done Sent Done false
25157 7463 Acked Write 6 Done Sent Done false
25158 7464 Acked Flush 0 Done Sent Done false
25159 7465 Acked Write 2 Done Sent Done false
25160 7466 Acked Flush 0 Done Sent Done false
25161 7467 Acked Write 7 Done Sent Done false
25162 7468 Acked Flush 0 Done Sent Done false
25163 7469 Acked Read 8 Done Sent Done false
25164 7470 Acked Read 3 Done Sent Done false
25165 7471 Acked Read 6 Done Sent Done false
25166 7472 Acked Write 5 Done Sent Done false
25167 7473 Acked Read 3 Done Sent Done false
25168 7474 Acked Flush 0 Done Sent Done false
25169 7475 Acked Write 5 Done Sent Done false
25170 7476 Acked Write 3 Done Sent Done false
25171 7477 Acked Read 4 Done Sent Done false
25172 7478 Acked Read 8 Done Sent Done false
25173 7479 Acked Write 1 Done Sent Done false
25174 7480 Acked Read 5 Done Sent Done false
25175 7481 Acked Write 2 Done Sent Done false
25176 7482 Acked Read 4 Done Sent Done false
25177 7483 Acked Read 6 Done Sent Done false
25178 7484 Acked Read 4 Done Sent Done false
25179 7485 Acked Write 4 Done Sent Done false
25180 7486 Acked Read 6 Done Sent Done false
25181 7487 Acked Read 8 Done Sent Done false
25182 7488 Acked Flush 0 Done Sent Done false
25183 7489 Acked Read 5 Done Sent Done false
25184 7490 Acked Write 3 Done Sent Done false
25185 7491 Acked Write 9 Done Sent Done false
25186 7492 Acked Write 10 Done Sent Done false
25187 7493 Acked Write 1 Done Sent Done false
25188 7494 Acked Flush 0 Done Sent Done false
25189 7495 Acked Read 2 Done Sent Done false
25190 7496 Acked Write 1 Done Sent Done false
25191 7497 Acked Flush 0 Done Sent Done false
25192 7498 Acked Read 10 Done Sent Done false
25193 7499 Acked Read 8 Done Sent Done false
25194 7500 Acked Write 4 Done Sent Done false
25195 7501 Acked Read 9 Done Sent Done false
25196 7502 Acked Write 4 Done Sent Done false
25197 7503 Acked Flush 0 Done Sent Done false
25198 7504 Acked Write 4 Done Sent Done false
25199 7505 Acked Write 1 Done Sent Done false
25200 7506 Acked Read 8 Done Sent Done false
25201 7507 Acked Write 4 Done Sent Done false
25202 7508 Acked Read 6 Done Sent Done false
25203 7509 Acked Read 3 Done Sent Done false
25204 7510 Acked Write 6 Done Sent Done false
25205 7511 Acked Flush 0 Done Sent Done false
25206 7512 Acked Read 6 Done Sent Done false
25207 7513 Acked Write 2 Done Sent Done false
25208 7514 Acked Flush 0 Done Sent Done false
25209 7515 Acked Flush 0 Done Sent Done false
25210 7516 Acked Write 4 Done Sent Done false
25211 7517 Acked Write 1 Done Sent Done false
25212 7518 Acked Write 10 Done Sent Done false
25213 7519 Acked Read 5 Done Sent Done false
25214 7520 Acked Read 8 Done Sent Done false
25215 7521 Acked Write 8 Done Sent Done false
25216 7522 Acked Write 1 Done Sent Done false
25217 7523 Acked Write 6 Done Sent Done false
25218 7524 Acked Read 8 Done Sent Done false
25219 7525 Acked Write 10 Done Sent Done false
25220 7526 Acked Write 9 Done Sent Done false
25221 7527 Acked Read 1 Done Sent Done false
25222 7528 Acked Read 1 Done Sent Done false
25223 7529 Acked Write 1 Done Sent Done false
25224 7530 Acked Write 1 Done Sent Done false
25225 7531 Acked Write 1 Done Sent Done false
25226 7532 Acked Write 8 Done Sent Done false
25227 7533 Acked Read 5 Done Sent Done false
25228 7534 Acked Read 7 Done Sent Done false
25229 7535 Acked Write 6 Done Sent Done false
25230 7536 Acked Write 3 Done Sent Done false
25231 7537 Acked Read 9 Done Sent Done false
25232 7538 Acked Read 5 Done Sent Done false
25233 7539 Acked Read 10 Done Sent Done false
25234 7540 Acked Write 6 Done Sent Done false
25235 7541 Acked Write 1 Done Sent Done false
25236 7542 Acked Write 4 Done Sent Done false
25237 7543 Acked Write 5 Done Sent Done false
25238 7544 Acked Read 9 Done Sent Done false
25239 7545 Acked Read 3 Done Sent Done false
25240 7546 Acked Read 8 Done Sent Done false
25241 7547 Acked Write 1 Done Sent Done false
25242 7548 Acked Flush 0 Done Sent Done false
25243 7549 Acked Write 5 Done Sent Done false
25244 7550 Acked Write 1 Done Sent Done false
25245 7551 Acked Write 6 Done Sent Done false
25246 7552 Acked Write 7 Done Sent Done false
25247 7553 Acked Read 10 Done Sent Done false
25248 7554 Acked Read 7 Done Sent Done false
25249 7555 Acked Read 5 Done Sent Done false
25250 7556 Acked Read 9 Done Sent Done false
25251 7557 Acked Read 5 Done Sent Done false
25252 7558 Acked Write 9 Done Sent Done false
25253 7559 Acked Flush 0 Done Sent Done false
25254 7560 Acked Read 3 Done Sent Done false
25255 7561 Acked Write 4 Done Sent Done false
25256 7562 Acked Write 3 Done Sent Done false
25257 7563 Acked Read 6 Done Sent Done false
25258 7564 Acked Read 6 Done Sent Done false
25259 7565 Acked Read 2 Done Sent Done false
25260 7566 Acked Write 4 Done Sent Done false
25261 7567 Acked Write 1 Done Sent Done false
25262 7568 Acked Write 9 Done Sent Done false
25263 7569 Acked Write 8 Done Sent Done false
25264 7570 Acked Read 5 Done Sent Done false
25265 7571 Acked Read 6 Done Sent Done false
25266 7572 Acked Read 4 Done Sent Done false
25267 7573 Acked Write 3 Done Sent Done false
25268 7574 Acked Read 10 Done Sent Done false
25269 7575 Acked Read 4 Done Sent Done false
25270 7576 Acked Read 3 Done Sent Done false
25271 7577 Acked Read 7 Done Sent Done false
25272 7578 Acked Read 3 Done Sent Done false
25273 7579 Acked Write 1 Done Sent Done false
25274 7580 Acked Flush 0 Done Sent Done false
25275 7581 Acked Flush 0 Done Sent Done false
25276 7582 Acked Flush 0 Done Sent Done false
25277 7583 Acked Read 5 Done Sent Done false
25278 7584 Acked Read 3 Done Sent Done false
25279 7585 Acked Read 9 Done Sent Done false
25280 7586 Acked Read 8 Done Sent Done false
25281 7587 Acked Read 3 Done Sent Done false
25282 7588 Acked Write 2 Done Sent Done false
25283 7589 Acked Read 5 Done Sent Done false
25284 7590 Acked Read 1 Done Sent Done false
25285 7591 Acked Read 10 Done Sent Done false
25286 7592 Acked Read 2 Done Sent Done false
25287 7593 Acked Write 9 Done Sent Done false
25288 7594 Acked Read 7 Done Sent Done false
25289 7595 Acked Flush 0 Done Sent Done false
25290 7596 Acked Read 1 Done Sent Done false
25291 7597 Acked Read 5 Done Sent Done false
25292 7598 Acked Read 10 Done Sent Done false
25293 7599 Acked Read 1 Done Sent Done false
25294 7600 Acked Write 8 Done Sent Done false
25295 7601 Acked Read 1 Done Sent Done false
25296 7602 Acked Flush 0 Done Sent Done false
25297 7603 Acked Read 1 Done Sent Done false
25298 7604 Acked Write 7 Done Sent Done false
25299 7605 Acked Write 2 Done Sent Done false
25300 7606 Acked Read 9 Done Sent Done false
25301 7607 Acked Flush 0 Done Sent Done false
25302 7608 Acked Write 6 Done Sent Done false
25303 7609 Acked Flush 0 Done Sent Done false
25304 7610 Acked Read 6 Done Sent Done false
25305 7611 Acked Write 3 Done Sent Done false
25306 7612 Acked Write 7 Done Sent Done false
25307 7613 Acked Write 5 Done Sent Done false
25308 7614 Acked Write 6 Done Sent Done false
25309 7615 Acked Read 8 Done Sent Done false
25310 7616 Acked Write 3 Done Sent Done false
25311 7617 Acked Read 2 Done Sent Done false
25312 7618 Acked Read 2 Done Sent Done false
25313 7619 Acked Read 8 Done Sent Done false
25314 7620 Acked Write 6 Done Sent Done false
25315 7621 Acked Read 5 Done Sent Done false
25316 7622 Acked Read 7 Done Sent Done false
25317 7623 Acked Write 3 Done Sent Done false
25318 7624 Acked Write 10 Done Sent Done false
25319 7625 Acked Read 6 Done Sent Done false
25320 7626 Acked Write 4 Done Sent Done false
25321 7627 Acked Read 6 Done Sent Done false
25322 7628 Acked Write 3 Done Sent Done false
25323 7629 Acked Read 1 Done Sent Done false
25324 7630 Acked Write 6 Done Sent Done false
25325 7631 Acked Read 6 Done Sent Done false
25326 7632 Acked Read 6 Done Sent Done false
25327 7633 Acked Read 6 Done Sent Done false
25328 7634 Acked Read 3 Done Sent Done false
25329 7635 Acked Write 4 Done Sent Done false
25330 7636 Acked Flush 0 Done Sent Done false
25331 7637 Acked Read 9 Done Sent Done false
25332 7638 Acked Read 8 Done Sent Done false
25333 7639 Acked Read 1 Done Sent Done false
25334 7640 Acked Read 10 Done Sent Done false
25335 7641 Acked Write 6 Done Sent Done false
25336 7642 Acked Flush 0 Done Sent Done false
25337 STATES DS:0 DS:1 DS:2 TOTAL
25338 Sent 0 302 0 302
25339 Done 302 0 302 604
25340 Skipped 0 0 0 0
25341 Error 0 0 0 0
25342 Last Flush: 7642 7340 7642
25343 Downstairs last five completed: 7340 7339 7338 7337 7336
25344 Upstairs last five completed: 7642 7641 7640 7639 7638
25345 CLIENT: Up:0 ds:302 act:2
253462025-01-15T05:24:20.077ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
253472025-01-15T05:24:20.077ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55681) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
253482025-01-15T05:24:20.078ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
253492025-01-15T05:24:20.078ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
253502025-01-15T05:24:20.078ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
253512025-01-15T05:24:20.078ZINFOcrucible: send last flush ID to this DS: 7340 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
253522025-01-15T05:24:20.079ZINFOcrucible: Replied this last flush ID: 7340 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
253532025-01-15T05:24:20.079ZINFOcrucible: [1] client re-new 302 jobs since flush 7340 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
253542025-01-15T05:24:20.079ZINFOcrucible: [1] Replayed 302 jobs since flush: 7340 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
25355 ----------------------------------------------------------------
25356 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25357 Downstairs last five completed: 7642 7641 7640 7639 7638
25358 Upstairs last five completed: 7642 7641 7640 7639 7638
25359 CLIENT: Up:0 ds:0 act:3
25360 Replay: All jobs finished
25361 Main task has work to do, go find it
25362 got dsc Stop(1)
25363 stop 1
25364 Jan 15 05:24:20.310 [8820] Got stop action so:false kr:false
25365 INFO request completed, latency_us: 319, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 50c7fb67-0e47-43a9-8c92-4642efd0571b, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
253662025-01-15T05:24:20.311ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
25367 Jan 15 05:24:20.311 INFO request completed, latency_us: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:24:20.311380035Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
25368 110, response_code: 200, {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:24:20.311426531Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
25369 uri: /state/cid/1, method: GET, req_id: 67347ba6-d434-4905-b0f0-8b5da3d7a711, remote_addr: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:24:20.311452379Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"1","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
25370 127.0.0.1:42402, local_addr: 127.0.0.1:9998
25371 [8820] Exited with: ExitStatus(unix_wait_status(9))
25372 [8820][1] reports Exit
25373 Jan 15 05:24:24.312 INFO request completed, latency_us: 203, response_code: 200, uri: /state/cid/1, method: GET, req_id: d4d9dbd7-04a7-4d4a-b23f-8d5c94a54edc, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
25374 001/300 Flush
25375 002/300 Read block 4600 len 4096
25376 003/300 Flush
25377 004/300 Write block 4140 len 24576 data: 5 5 6 8 8 7
25378 005/300 Read block 204 len 20480
25379 006/300 Read block 3368 len 32768
25380 007/300 Write block 612 len 16384 data: 4 3 3 4
25381 008/300 Flush
25382 009/300 Write block 4749 len 4096 data: 2
25383 010/300 Read block 189 len 4096
25384 011/300 Write block 865 len 12288 data: 4 5 6
25385 012/300 Read block 2930 len 28672
25386 013/300 Read block 812 len 40960
25387 014/300 Write block 4951 len 28672 data: 2 2 2 2 2 2 2
25388 015/300 Read block 3361 len 16384
25389 016/300 Flush
25390 017/300 Read block 4478 len 16384
25391 018/300 Read block 3424 len 24576
25392 019/300 Read block 2723 len 24576
25393 020/300 Write block 969 len 28672 data: 5 6 3 3 2 2 3
25394 021/300 Read block 1405 len 4096
25395 022/300 Write block 4572 len 16384 data: 3 6 6 6
25396 023/300 Read block 1432 len 12288
25397 024/300 Read block 3150 len 8192
25398 025/300 Read block 3502 len 32768
25399 026/300 Write block 2454 len 16384 data: 5 5 6 5
25400 027/300 Read block 15 len 16384
25401 028/300 Read block 4064 len 8192
25402 029/300 Read block 2177 len 36864
25403 030/300 Write block 945 len 24576 data: 4 4 4 4 3 4
25404 031/300 Write block 2635 len 32768 data: 5 6 6 5 5 6 5 6
25405 032/300 Write block 3859 len 32768 data: 5 4 5 6 5 5 5 4
25406 033/300 Read block 556 len 20480
25407 034/300 Write block 4572 len 8192 data: 4 7
25408 035/300 Read block 2322 len 40960
25409 036/300 Flush
25410 037/300 Read block 2407 len 12288
25411 038/300 Read block 1047 len 12288
25412 039/300 Write block 1564 len 4096 data: 7
25413 040/300 Read block 2457 len 8192
25414 041/300 Read block 3517 len 8192
25415 042/300 Write block 4000 len 4096 data: 4
25416 043/300 Write block 3145 len 8192 data: 6 5
25417 044/300 Read block 2142 len 28672
25418 045/300 Read block 2256 len 4096
25419 046/300 Write block 2391 len 8192 data: 4 5
25420 047/300 Write block 1421 len 16384 data: 5 5 5 5
25421 048/300 Write block 2717 len 24576 data: 7 5 5 3 2 2
25422 049/300 Read block 956 len 28672
25423 050/300 Write block 1996 len 32768 data: 4 5 5 5 5 6 5 4
25424 051/300 Write block 3279 len 20480 data: 8 7 5 4 3
25425 052/300 Write block 416 len 20480 data: 2 2 2 3 3
25426 053/300 Write block 2676 len 4096 data: 4
25427 054/300 Write block 589 len 36864 data: 4 4 4 4 5 7 7 6 4
25428 055/300 Read block 2686 len 36864
25429 056/300 Write block 723 len 40960 data: 3 3 3 4 4 3 4 5 4 4
25430 057/300 Read block 540 len 40960
25431 058/300 Read block 3796 len 36864
25432 059/300 Read block 893 len 16384
25433 060/300 Read block 978 len 36864
25434 061/300 Write block 2741 len 8192 data: 4 4
25435 062/300 Write block 3093 len 4096 data: 8
25436 063/300 Read block 4861 len 36864
25437 064/300 Write block 2107 len 28672 data: 5 4 6 5 6 4 6
25438 065/300 Write block 1288 len 4096 data: 4
25439 066/300 Write block 3866 len 32768 data: 5 3 3 5 6 6 8 7
25440 067/300 Read block 4819 len 8192
25441 068/300 Write block 2566 len 32768 data: 7 6 5 6 5 3 4 4
25442 069/300 Write block 2491 len 20480 data: 8 7 6 6 4
25443 070/300 Read block 891 len 24576
25444 071/300 Flush
25445 072/300 Read block 3143 len 40960
25446 073/300 Write block 246 len 20480 data: 3 4 6 5 5
25447 074/300 Read block 2621 len 28672
25448 075/300 Write block 2760 len 4096 data: 8
25449 076/300 Read block 345 len 28672
25450 077/300 Write block 1719 len 24576 data: 4 3 2 3 4 4
25451 078/300 Write block 2546 len 40960 data: 3 2 2 2 3 4 4 3 2 2
25452 079/300 Write block 3781 len 4096 data: 7
25453 080/300 Flush
25454 081/300 Write block 1265 len 16384 data: 4 5 7 6
25455 082/300 Read block 405 len 40960
25456 083/300 Read block 4591 len 4096
25457 084/300 Read block 3722 len 4096
25458 085/300 Flush
25459 086/300 Read block 1375 len 4096
25460 087/300 Write block 3833 len 40960 data: 5 6 7 8 8 7 8 8 9 9
25461 088/300 Read block 3949 len 4096
25462 089/300 Read block 3478 len 24576
25463 090/300 Flush
25464 091/300 Write block 3336 len 28672 data: 7 7 7 6 5 4 3
25465 092/300 Write block 2777 len 32768 data: 6 5 5 10 9 8 6 6
25466 093/300 Write block 4177 len 16384 data: 3 4 5 5
25467 094/300 Read block 2295 len 28672
25468 095/300 Read block 720 len 24576
25469 096/300 Write block 625 len 28672 data: 3 3 4 3 3 4 4
25470 097/300 Read block 1640 len 16384
25471 098/300 Write block 3245 len 8192 data: 7 6
25472 099/300 Write block 4571 len 32768 data: 2 5 8 7 7 4 5 5
25473 100/300 Read block 1427 len 36864
25474 101/300 Write block 3540 len 16384 data: 5 5 5 5
25475 102/300 Write block 1809 len 8192 data: 5 5
25476 103/300 Read block 1433 len 4096
25477 104/300 Read block 935 len 4096
25478 105/300 Write block 2086 len 16384 data: 6 4 5 5
25479 106/300 Write block 1141 len 32768 data: 4 4 2 2 4 5 8 8
25480 107/300 Flush
25481 108/300 Read block 1543 len 40960
25482 109/300 Write block 2254 len 32768 data: 4 5 6 6 6 7 6 6
25483 110/300 Read block 162 len 28672
25484 111/300 Write block 3118 len 28672 data: 5 5 8 5 6 5 6
25485 112/300 Read block 4011 len 40960
25486 113/300 Read block 112 len 20480
25487 114/300 Write block 1523 len 40960 data: 4 4 4 5 6 6 6 5 7 7
25488 115/300 Read block 1416 len 12288
25489 116/300 Flush
25490 117/300 Write block 1937 len 32768 data: 2 2 2 3 3 3 3 3
25491 118/300 Read block 2450 len 12288
25492 119/300 Write block 3940 len 32768 data: 6 6 4 7 8 7 7 5
25493 120/300 Read block 3309 len 8192
25494 121/300 Write block 1762 len 24576 data: 7 8 7 7 6 6
25495 122/300 Write block 326 len 28672 data: 7 7 8 5 5 5 5
25496 123/300 Flush
25497 124/300 Read block 4795 len 40960
25498 125/300 Write block 1836 len 8192 data: 5 5
25499 126/300 Read block 451 len 36864
25500 127/300 Read block 800 len 12288
25501 128/300 Write block 2586 len 12288 data: 6 6 7
25502 129/300 Read block 1643 len 24576
25503 130/300 Read block 1802 len 40960
25504 131/300 Read block 3629 len 40960
25505 132/300 Write block 3833 len 40960 data: 6 7 8 9 9 8 9 9 10 10
25506 133/300 Write block 3876 len 24576 data: 3 3 4 5 3 5
25507 134/300 Read block 1997 len 24576
25508 135/300 Read block 4816 len 4096
25509 136/300 Read block 554 len 16384
25510 137/300 Read block 1120 len 32768
25511 138/300 Read block 2042 len 4096
25512 139/300 Flush
25513 140/300 Read block 2761 len 12288
25514 141/300 Read block 1903 len 4096
25515 142/300 Write block 827 len 36864 data: 5 4 5 4 3 4 3 3 3
25516 143/300 Flush
25517 144/300 Read block 3097 len 4096
25518 145/300 Write block 3454 len 8192 data: 7 7
25519 146/300 Write block 1400 len 28672 data: 4 4 5 4 5 3 3
25520 147/300 Read block 256 len 8192
25521 148/300 Write block 3617 len 24576 data: 4 4 5 4 3 3
25522 149/300 Write block 2299 len 28672 data: 6 6 6 8 8 5 5
25523 150/300 Write block 4393 len 8192 data: 3 3
25524 151/300 Read block 172 len 40960
25525 152/300 Read block 2219 len 20480
25526 153/300 Read block 4528 len 20480
25527 154/300 Read block 4410 len 4096
25528 155/300 Write block 2600 len 24576 data: 6 4 4 3 5 7
25529 156/300 Read block 2774 len 20480
25530 157/300 Write block 1756 len 28672 data: 5 6 5 5 6 8 8
25531 158/300 Read block 1106 len 28672
25532 159/300 Write block 1499 len 24576 data: 3 5 6 7 7 6
25533 160/300 Read block 4397 len 20480
25534 161/300 Read block 3688 len 8192
25535 162/300 Write block 685 len 8192 data: 9 6
25536 163/300 Write block 4642 len 16384 data: 2 2 3 3
25537 164/300 Write block 4780 len 36864 data: 5 5 4 4 3 3 3 2 2
25538 165/300 Read block 1773 len 28672
25539 166/300 Read block 468 len 4096
25540 167/300 Write block 1225 len 32768 data: 3 3 3 2 3 4 3 4
25541 168/300 Read block 353 len 28672
25542 169/300 Write block 1796 len 4096 data: 3
25543 170/300 Write block 3481 len 40960 data: 4 3 4 3 3 3 3 4 4 3
25544 171/300 Write block 3373 len 20480 data: 6 6 6 5 5
25545 172/300 Write block 3596 len 32768 data: 2 3 3 4 5 5 5 5
25546 173/300 Flush
25547 174/300 Read block 2644 len 8192
25548 175/300 Read block 3766 len 16384
25549 176/300 Flush
25550 177/300 Read block 3268 len 40960
25551 178/300 Read block 1807 len 12288
25552 179/300 Write block 1371 len 28672 data: 8 8 7 5 5 3 3
25553 180/300 Flush
25554 181/300 Read block 3982 len 36864
25555 182/300 Write block 2422 len 20480 data: 4 3 3 4 4
25556 183/300 Read block 3352 len 20480
25557 184/300 Write block 4651 len 16384 data: 5 4 4 3
25558 185/300 Read block 4858 len 8192
25559 186/300 Write block 3411 len 4096 data: 6
25560 187/300 Flush
25561 188/300 Read block 2487 len 24576
25562 189/300 Read block 2719 len 4096
25563 190/300 Read block 2967 len 40960
25564 191/300 Write block 1317 len 40960 data: 5 4 5 6 4 3 4 5 4 7
25565 192/300 Write block 3976 len 20480 data: 6 4 3 3 3
25566 193/300 Write block 538 len 20480 data: 5 4 4 3 4
25567 194/300 Read block 4727 len 36864
25568 195/300 Write block 1028 len 12288 data: 3 3 3
25569 196/300 Read block 2484 len 16384
25570 197/300 Write block 1247 len 20480 data: 5 4 3 3 3
25571 198/300 Read block 3275 len 40960
25572 199/300 Read block 1084 len 8192
25573 200/300 Flush
25574 201/300 Read block 4958 len 24576
25575 202/300 Write block 3415 len 12288 data: 6 6 5
25576 203/300 Read block 2292 len 12288
25577 204/300 Write block 1012 len 24576 data: 4 6 5 5 5 4
25578 205/300 Read block 3019 len 16384
25579 206/300 Read block 1049 len 16384
25580 207/300 Read block 523 len 32768
25581 208/300 Read block 705 len 4096
25582 209/300 Read block 4019 len 12288
25583 210/300 Read block 4417 len 20480
25584 211/300 Read block 1738 len 32768
25585 212/300 Flush
25586 213/300 Write block 4614 len 24576 data: 7 7 7 7 7 7
25587 214/300 Read block 3354 len 20480
25588 215/300 Flush
25589 216/300 Read block 4546 len 40960
25590 217/300 Write block 782 len 12288 data: 2 2 2
25591 218/300 Write block 453 len 8192 data: 5 7
25592 219/300 Read block 4765 len 20480
25593 220/300 Write block 2170 len 12288 data: 5 5 5
25594 221/300 Read block 2164 len 32768
25595 222/300 Read block 104 len 36864
25596 223/300 Write block 794 len 40960 data: 6 6 5 3 3 3 3 2 2 2
25597 224/300 Read block 4423 len 4096
25598 225/300 Read block 4090 len 20480
25599 226/300 Read block 3440 len 36864
25600 227/300 Read block 4013 len 12288
25601 228/300 Write block 3359 len 28672 data: 4 4 3 4 4 6 7
25602 229/300 Flush
25603 230/300 Write block 2838 len 32768 data: 5 4 4 4 4 3 2 2
25604 231/300 Write block 2282 len 16384 data: 6 4 6 5
25605 232/300 Flush
25606 233/300 Read block 4356 len 28672
25607 234/300 Write block 4908 len 16384 data: 5 6 5 6
25608 235/300 Read block 1507 len 20480
25609 236/300 Write block 1610 len 4096 data: 8
25610 237/300 Read block 4386 len 12288
25611 238/300 Read block 2880 len 32768
25612 239/300 Write block 3444 len 32768 data: 3 4 4 4 6 6 6 6
25613 240/300 Write block 4128 len 4096 data: 5
25614 241/300 Read block 1367 len 4096
25615 242/300 Read block 4971 len 32768
25616 243/300 Write block 972 len 20480 data: 4 3 3 4 3
25617 244/300 Read block 2828 len 8192
25618 245/300 Read block 1745 len 20480
25619 246/300 Write block 3421 len 28672 data: 4 5 5 4 5 3 3
25620 247/300 Read block 2733 len 28672
25621 248/300 Read block 3721 len 24576
25622 249/300 Write block 4995 len 8192 data: 5 6
25623 250/300 Read block 2276 len 20480
25624 251/300 Read block 553 len 32768
25625 252/300 Read block 3048 len 20480
25626 253/300 Write block 2443 len 12288 data: 3 4 3
25627 254/300 Read block 3315 len 8192
25628 255/300 Write block 1133 len 36864 data: 3 5 5 4 4 4 3 3 5
25629 256/300 Read block 3337 len 32768
25630 257/300 Write block 1595 len 8192 data: 3 4
25631 258/300 Write block 510 len 28672 data: 4 2 3 3 3 5 5
25632 259/300 Write block 2696 len 4096 data: 4
25633 260/300 Read block 328 len 32768
25634 261/300 Write block 4446 len 28672 data: 5 6 6 6 9 8 7
25635 262/300 Write block 74 len 28672 data: 4 6 6 6 7 8 8
25636 263/300 Read block 4532 len 28672
25637 264/300 Write block 3609 len 36864 data: 4 5 6 6 4 3 5 5 5
25638 265/300 Write block 2259 len 40960 data: 8 7 7 6 6 5 5 4 4 5
25639 266/300 Read block 4477 len 4096
25640 267/300 Write block 1644 len 8192 data: 8 6
25641 268/300 Write block 24 len 36864 data: 5 7 6 6 6 4 4 3 3
25642 269/300 Read block 1123 len 40960
25643 270/300 Read block 4238 len 24576
25644 271/300 Flush
25645 272/300 Read block 3434 len 8192
25646 273/300 Write block 3180 len 24576 data: 6 7 9 9 8 6
25647 274/300 Read block 2708 len 36864
25648 275/300 Flush
25649 276/300 Read block 2861 len 16384
25650 277/300 Write block 3079 len 4096 data: 3
25651 278/300 Read block 2452 len 8192
25652 279/300 Write block 3013 len 16384 data: 4 3 2 2
25653 280/300 Write block 4124 len 16384 data: 3 5 6 6
25654 281/300 Read block 4779 len 4096
25655 282/300 Flush
25656 283/300 Read block 2043 len 12288
25657 284/300 Write block 1332 len 28672 data: 2 3 3 4 4 4 4
25658 285/300 Write block 3722 len 4096 data: 3
25659 286/300 Write block 4787 len 12288 data: 3 3 2
25660 287/300 Read block 4808 len 24576
25661 288/300 Read block 124 len 36864
25662 289/300 Read block 71 len 20480
25663 290/300 Flush
25664 291/300 Read block 1674 len 24576
25665 292/300 Write block 4585 len 4096 data: 5
25666 293/300 Write block 1249 len 28672 data: 4 4 4 4 5 5 5
25667 294/300 Read block 4923 len 4096
25668 295/300 Read block 438 len 40960
25669 296/300 Read block 686 len 28672
25670 297/300 Read block 2263 len 36864
25671 298/300 Read block 4922 len 4096
25672 299/300 Read block 1711 len 24576
25673 300/300 Read block 4680 len 32768
25674 301/300 Read block 3488 len 12288
25675 Main task has work to do, go find it
25676 got dsc Start(1)
25677 start 1
25678 Jan 15 05:24:24.538 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/1, method: POST, req_id: 449313d3-5e1c-47b6-8e76-fa315dc4c61b, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
25679 Replay: started 1, returned:Ok(())
25680 ----------------------------------------------------------------
25681 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25682 States: Active Offline Active
25683 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25684 7643 Acked Flush 0 Done Sent Done false
25685 7644 Acked Read 1 Done Sent Done false
25686 7645 Acked Flush 0 Done Sent Done false
25687 7646 Acked Write 6 Done Sent Done false
25688 7647 Acked Read 5 Done Sent Done false
25689 7648 Acked Read 8 Done Sent Done false
25690 7649 Acked Write 4 Done Sent Done false
25691 7650 Acked Flush 0 Done Sent Done false
25692 7651 Acked Write 1 Done Sent Done false
25693 7652 Acked Read 1 Done Sent Done false
25694 7653 Acked Write 3 Done Sent Done false
25695 7654 Acked Read 7 Done Sent Done false
25696 7655 Acked Read 10 Done Sent Done false
25697 7656 Acked Write 7 Done Sent Done false
25698 7657 Acked Read 4 Done Sent Done false
25699 7658 Acked Flush 0 Done Sent Done false
25700 7659 Acked Read 4 Done Sent Done false
25701 7660 Acked Read 6 Done Sent Done false
25702 7661 Acked Read 6 Done Sent Done false
25703 7662 Acked Write 7 Done Sent Done false
25704 7663 Acked Read 1 Done Sent Done false
25705 7664 Acked Write 4 Done Sent Done false
25706 7665 Acked Read 3 Done Sent Done false
25707 7666 Acked Read 2 Done Sent Done false
25708 7667 Acked Read 8 Done Sent Done false
25709 7668 Acked Write 4 Done Sent Done false
25710 7669 Acked Read 4 Done Sent Done false
25711 7670 Acked Read 2 Done Sent Done false
25712 7671 Acked Read 9 Done Sent Done false
25713 7672 Acked Write 6 Done Sent Done false
25714 7673 Acked Write 8 Done Sent Done false
25715 7674 Acked Write 8 Done Sent Done false
25716 7675 Acked Read 5 Done Sent Done false
25717 7676 Acked Write 2 Done Sent Done false
25718 7677 Acked Read 10 Done Sent Done false
25719 7678 Acked Flush 0 Done Sent Done false
25720 7679 Acked Read 3 Done Sent Done false
25721 7680 Acked Read 3 Done Sent Done false
25722 7681 Acked Write 1 Done Sent Done false
25723 7682 Acked Read 2 Done Sent Done false
25724 7683 Acked Read 2 Done Sent Done false
25725 7684 Acked Write 1 Done Sent Done false
25726 7685 Acked Write 2 Done Sent Done false
25727 7686 Acked Read 7 Done Sent Done false
25728 7687 Acked Read 1 Done Sent Done false
25729 7688 Acked Write 2 Done Sent Done false
25730 7689 Acked Write 4 Done Sent Done false
25731 7690 Acked Write 6 Done Sent Done false
25732 7691 Acked Read 7 Done Sent Done false
25733 7692 Acked Write 8 Done Sent Done false
25734 7693 Acked Write 5 Done Sent Done false
25735 7694 Acked Write 5 Done Sent Done false
25736 7695 Acked Write 1 Done Sent Done false
25737 7696 Acked Write 9 Done Sent Done false
25738 7697 Acked Read 9 Done Sent Done false
25739 7698 Acked Write 10 Done Sent Done false
25740 7699 Acked Read 10 Done Sent Done false
25741 7700 Acked Read 9 Done Sent Done false
25742 7701 Acked Read 4 Done Sent Done false
25743 7702 Acked Read 9 Done Sent Done false
25744 7703 Acked Write 2 Done Sent Done false
25745 7704 Acked Write 1 Done Sent Done false
25746 7705 Acked Read 9 Done Sent Done false
25747 7706 Acked Write 7 Done Sent Done false
25748 7707 Acked Write 1 Done Sent Done false
25749 7708 Acked Write 8 Done Sent Done false
25750 7709 Acked Read 2 Done Sent Done false
25751 7710 Acked Write 8 Done Sent Done false
25752 7711 Acked Write 5 Done Sent Done false
25753 7712 Acked Read 6 Done Sent Done false
25754 7713 Acked Flush 0 Done Sent Done false
25755 7714 Acked Read 10 Done Sent Done false
25756 7715 Acked Write 5 Done Sent Done false
25757 7716 Acked Read 7 Done Sent Done false
25758 7717 Acked Write 1 Done Sent Done false
25759 7718 Acked Read 7 Done Sent Done false
25760 7719 Acked Write 6 Done Sent Done false
25761 7720 Acked Write 10 Done Sent Done false
25762 7721 Acked Write 1 Done Sent Done false
25763 7722 Acked Flush 0 Done Sent Done false
25764 7723 Acked Write 4 Done Sent Done false
25765 7724 Acked Read 10 Done Sent Done false
25766 7725 Acked Read 1 Done Sent Done false
25767 7726 Acked Read 1 Done Sent Done false
25768 7727 Acked Flush 0 Done Sent Done false
25769 7728 Acked Read 1 Done Sent Done false
25770 7729 Acked Write 10 Done Sent Done false
25771 7730 Acked Read 1 Done Sent Done false
25772 7731 Acked Read 6 Done Sent Done false
25773 7732 Acked Flush 0 Done Sent Done false
25774 7733 Acked Write 7 Done Sent Done false
25775 7734 Acked Write 8 Done Sent Done false
25776 7735 Acked Write 4 Done Sent Done false
25777 7736 Acked Read 7 Done Sent Done false
25778 7737 Acked Read 6 Done Sent Done false
25779 7738 Acked Write 7 Done Sent Done false
25780 7739 Acked Read 4 Done Sent Done false
25781 7740 Acked Write 2 Done Sent Done false
25782 7741 Acked Write 8 Done Sent Done false
25783 7742 Acked Read 9 Done Sent Done false
25784 7743 Acked Write 4 Done Sent Done false
25785 7744 Acked Write 2 Done Sent Done false
25786 7745 Acked Read 1 Done Sent Done false
25787 7746 Acked Read 1 Done Sent Done false
25788 7747 Acked Write 4 Done Sent Done false
25789 7748 Acked Write 8 Done Sent Done false
25790 7749 Acked Flush 0 Done Sent Done false
25791 7750 Acked Read 10 Done Sent Done false
25792 7751 Acked Write 8 Done Sent Done false
25793 7752 Acked Read 7 Done Sent Done false
25794 7753 Acked Write 7 Done Sent Done false
25795 7754 Acked Read 10 Done Sent Done false
25796 7755 Acked Read 5 Done Sent Done false
25797 7756 Acked Write 10 Done Sent Done false
25798 7757 Acked Read 3 Done Sent Done false
25799 7758 Acked Flush 0 Done Sent Done false
25800 7759 Acked Write 8 Done Sent Done false
25801 7760 Acked Read 3 Done Sent Done false
25802 7761 Acked Write 8 Done Sent Done false
25803 7762 Acked Read 2 Done Sent Done false
25804 7763 Acked Write 6 Done Sent Done false
25805 7764 Acked Write 7 Done Sent Done false
25806 7765 Acked Flush 0 Done Sent Done false
25807 7766 Acked Read 10 Done Sent Done false
25808 7767 Acked Write 2 Done Sent Done false
25809 7768 Acked Read 9 Done Sent Done false
25810 7769 Acked Read 3 Done Sent Done false
25811 7770 Acked Write 3 Done Sent Done false
25812 7771 Acked Read 6 Done Sent Done false
25813 7772 Acked Read 10 Done Sent Done false
25814 7773 Acked Read 10 Done Sent Done false
25815 7774 Acked Write 10 Done Sent Done false
25816 7775 Acked Write 6 Done Sent Done false
25817 7776 Acked Read 6 Done Sent Done false
25818 7777 Acked Read 1 Done Sent Done false
25819 7778 Acked Read 4 Done Sent Done false
25820 7779 Acked Read 8 Done Sent Done false
25821 7780 Acked Read 1 Done Sent Done false
25822 7781 Acked Flush 0 Done Sent Done false
25823 7782 Acked Read 3 Done Sent Done false
25824 7783 Acked Read 1 Done Sent Done false
25825 7784 Acked Write 9 Done Sent Done false
25826 7785 Acked Flush 0 Done Sent Done false
25827 7786 Acked Read 1 Done Sent Done false
25828 7787 Acked Write 2 Done Sent Done false
25829 7788 Acked Write 7 Done Sent Done false
25830 7789 Acked Read 2 Done Sent Done false
25831 7790 Acked Write 6 Done Sent Done false
25832 7791 Acked Write 7 Done Sent Done false
25833 7792 Acked Write 2 Done Sent Done false
25834 7793 Acked Read 10 Done Sent Done false
25835 7794 Acked Read 5 Done Sent Done false
25836 7795 Acked Read 5 Done Sent Done false
25837 7796 Acked Read 1 Done Sent Done false
25838 7797 Acked Write 6 Done Sent Done false
25839 7798 Acked Read 5 Done Sent Done false
25840 7799 Acked Write 7 Done Sent Done false
25841 7800 Acked Read 7 Done Sent Done false
25842 7801 Acked Write 6 Done Sent Done false
25843 7802 Acked Read 5 Done Sent Done false
25844 7803 Acked Read 2 Done Sent Done false
25845 7804 Acked Write 2 Done Sent Done false
25846 7805 Acked Write 4 Done Sent Done false
25847 7806 Acked Write 9 Done Sent Done false
25848 7807 Acked Read 7 Done Sent Done false
25849 7808 Acked Read 1 Done Sent Done false
25850 7809 Acked Write 8 Done Sent Done false
25851 7810 Acked Read 7 Done Sent Done false
25852 7811 Acked Write 1 Done Sent Done false
25853 7812 Acked Write 10 Done Sent Done false
25854 7813 Acked Write 5 Done Sent Done false
25855 7814 Acked Write 8 Done Sent Done false
25856 7815 Acked Flush 0 Done Sent Done false
25857 7816 Acked Read 2 Done Sent Done false
25858 7817 Acked Read 4 Done Sent Done false
25859 7818 Acked Flush 0 Done Sent Done false
25860 7819 Acked Read 10 Done Sent Done false
25861 7820 Acked Read 3 Done Sent Done false
25862 7821 Acked Write 7 Done Sent Done false
25863 7822 Acked Flush 0 Done Sent Done false
25864 7823 Acked Read 9 Done Sent Done false
25865 7824 Acked Write 5 Done Sent Done false
25866 7825 Acked Read 5 Done Sent Done false
25867 7826 Acked Write 4 Done Sent Done false
25868 7827 Acked Read 2 Done Sent Done false
25869 7828 Acked Write 1 Done Sent Done false
25870 7829 Acked Flush 0 Done Sent Done false
25871 7830 Acked Read 6 Done Sent Done false
25872 7831 Acked Read 1 Done Sent Done false
25873 7832 Acked Read 10 Done Sent Done false
25874 7833 Acked Write 10 Done Sent Done false
25875 7834 Acked Write 5 Done Sent Done false
25876 7835 Acked Write 5 Done Sent Done false
25877 7836 Acked Read 9 Done Sent Done false
25878 7837 Acked Write 3 Done Sent Done false
25879 7838 Acked Read 4 Done Sent Done false
25880 7839 Acked Write 5 Done Sent Done false
25881 7840 Acked Read 10 Done Sent Done false
25882 7841 Acked Read 2 Done Sent Done false
25883 7842 Acked Flush 0 Done Sent Done false
25884 7843 Acked Read 6 Done Sent Done false
25885 7844 Acked Write 3 Done Sent Done false
25886 7845 Acked Read 3 Done Sent Done false
25887 7846 Acked Write 6 Done Sent Done false
25888 7847 Acked Read 4 Done Sent Done false
25889 7848 Acked Read 4 Done Sent Done false
25890 7849 Acked Read 8 Done Sent Done false
25891 7850 Acked Read 1 Done Sent Done false
25892 7851 Acked Read 3 Done Sent Done false
25893 7852 Acked Read 5 Done Sent Done false
25894 7853 Acked Read 8 Done Sent Done false
25895 7854 Acked Flush 0 Done Sent Done false
25896 7855 Acked Write 6 Done Sent Done false
25897 7856 Acked Read 5 Done Sent Done false
25898 7857 Acked Flush 0 Done Sent Done false
25899 7858 Acked Read 10 Done Sent Done false
25900 7859 Acked Write 3 Done Sent Done false
25901 7860 Acked Write 2 Done Sent Done false
25902 7861 Acked Read 5 Done Sent Done false
25903 7862 Acked Write 3 Done Sent Done false
25904 7863 Acked Read 8 Done Sent Done false
25905 7864 Acked Read 9 Done Sent Done false
25906 7865 Acked Write 10 Done Sent Done false
25907 7866 Acked Read 1 Done Sent Done false
25908 7867 Acked Read 5 Done Sent Done false
25909 7868 Acked Read 9 Done Sent Done false
25910 7869 Acked Read 3 Done Sent Done false
25911 7870 Acked Write 7 Done Sent Done false
25912 7871 Acked Flush 0 Done Sent Done false
25913 7872 Acked Write 8 Done Sent Done false
25914 7873 Acked Write 4 Done Sent Done false
25915 7874 Acked Flush 0 Done Sent Done false
25916 7875 Acked Read 7 Done Sent Done false
25917 7876 Acked Write 4 Done Sent Done false
25918 7877 Acked Read 5 Done Sent Done false
25919 7878 Acked Write 1 Done Sent Done false
25920 7879 Acked Read 3 Done Sent Done false
25921 7880 Acked Read 8 Done Sent Done false
25922 7881 Acked Write 8 Done Sent Done false
25923 7882 Acked Write 1 Done Sent Done false
25924 7883 Acked Read 1 Done Sent Done false
25925 7884 Acked Read 8 Done Sent Done false
25926 7885 Acked Write 5 Done Sent Done false
25927 7886 Acked Read 2 Done Sent Done false
25928 7887 Acked Read 5 Done Sent Done false
25929 7888 Acked Write 7 Done Sent Done false
25930 7889 Acked Read 7 Done Sent Done false
25931 7890 Acked Read 6 Done Sent Done false
25932 7891 Acked Write 2 Done Sent Done false
25933 7892 Acked Read 5 Done Sent Done false
25934 7893 Acked Read 8 Done Sent Done false
25935 7894 Acked Read 5 Done Sent Done false
25936 7895 Acked Write 3 Done Sent Done false
25937 7896 Acked Read 2 Done Sent Done false
25938 7897 Acked Write 9 Done Sent Done false
25939 7898 Acked Read 8 Done Sent Done false
25940 7899 Acked Write 2 Done Sent Done false
25941 7900 Acked Write 7 Done Sent Done false
25942 7901 Acked Write 1 Done Sent Done false
25943 7902 Acked Read 8 Done Sent Done false
25944 7903 Acked Write 7 Done Sent Done false
25945 7904 Acked Write 7 Done Sent Done false
25946 7905 Acked Read 7 Done Sent Done false
25947 7906 Acked Write 9 Done Sent Done false
25948 7907 Acked Write 10 Done Sent Done false
25949 7908 Acked Read 1 Done Sent Done false
25950 7909 Acked Write 2 Done Sent Done false
25951 7910 Acked Write 9 Done Sent Done false
25952 7911 Acked Read 10 Done Sent Done false
25953 7912 Acked Read 6 Done Sent Done false
25954 7913 Acked Flush 0 Done Sent Done false
25955 7914 Acked Read 2 Done Sent Done false
25956 7915 Acked Write 6 Done Sent Done false
25957 7916 Acked Read 9 Done Sent Done false
25958 7917 Acked Flush 0 Done Sent Done false
25959 7918 Acked Read 4 Done Sent Done false
25960 7919 Acked Write 1 Done Sent Done false
25961 7920 Acked Read 2 Done Sent Done false
25962 7921 Acked Write 4 Done Sent Done false
25963 7922 Acked Write 4 Done Sent Done false
25964 7923 Acked Read 1 Done Sent Done false
25965 7924 Acked Flush 0 Done Sent Done false
25966 7925 Acked Read 3 Done Sent Done false
25967 7926 Acked Write 7 Done Sent Done false
25968 7927 Acked Write 1 Done Sent Done false
25969 7928 Acked Write 3 Done Sent Done false
25970 7929 Acked Read 6 Done Sent Done false
25971 7930 Acked Read 9 Done Sent Done false
25972 7931 Acked Read 5 Done Sent Done false
25973 7932 Acked Flush 0 Done Sent Done false
25974 7933 Acked Read 6 Done Sent Done false
25975 7934 Acked Write 1 Done Sent Done false
25976 7935 Acked Write 7 Done Sent Done false
25977 7936 Acked Read 1 Done Sent Done false
25978 7937 Acked Read 10 Done Sent Done false
25979 7938 Acked Read 7 Done Sent Done false
25980 7939 Acked Read 9 Done Sent Done false
25981 7940 Acked Read 1 Done Sent Done false
25982 7941 Acked Read 6 Done Sent Done false
25983 7942 Acked Read 8 Done Sent Done false
25984 7943 Acked Read 3 Done Sent Done false
25985 STATES DS:0 DS:1 DS:2 TOTAL
25986 Sent 0 301 0 301
25987 Done 301 0 301 602
25988 Skipped 0 0 0 0
25989 Error 0 0 0 0
25990 Last Flush: 7932 7642 7932
25991 Downstairs last five completed: 7642 7641 7640 7639 7638
25992 Upstairs last five completed: 7943 7942 7941 7940 7939
25993 CLIENT: Up:0 ds:301 act:2
25994 [8820] got start action
25995 [8820] I am going to restart
25996 Starting downstairs at port 8820
25997 Make output file at "/tmp/dsc/downstairs-8820.txt"
25998 [8820][1] reports Starting
25999 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1225)
26000 [8820][1] reports Running
26001 ----------------------------------------------------------------
26002 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26003 States: Active Offline Active
26004 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26005 7643 Acked Flush 0 Done Sent Done false
26006 7644 Acked Read 1 Done Sent Done false
26007 7645 Acked Flush 0 Done Sent Done false
26008 7646 Acked Write 6 Done Sent Done false
26009 7647 Acked Read 5 Done Sent Done false
26010 7648 Acked Read 8 Done Sent Done false
26011 7649 Acked Write 4 Done Sent Done false
26012 7650 Acked Flush 0 Done Sent Done false
26013 7651 Acked Write 1 Done Sent Done false
26014 7652 Acked Read 1 Done Sent Done false
26015 7653 Acked Write 3 Done Sent Done false
26016 7654 Acked Read 7 Done Sent Done false
26017 7655 Acked Read 10 Done Sent Done false
26018 7656 Acked Write 7 Done Sent Done false
26019 7657 Acked Read 4 Done Sent Done false
26020 7658 Acked Flush 0 Done Sent Done false
26021 7659 Acked Read 4 Done Sent Done false
26022 7660 Acked Read 6 Done Sent Done false
26023 7661 Acked Read 6 Done Sent Done false
26024 7662 Acked Write 7 Done Sent Done false
26025 7663 Acked Read 1 Done Sent Done false
26026 7664 Acked Write 4 Done Sent Done false
26027 7665 Acked Read 3 Done Sent Done false
26028 7666 Acked Read 2 Done Sent Done false
26029 7667 Acked Read 8 Done Sent Done false
26030 7668 Acked Write 4 Done Sent Done false
26031 7669 Acked Read 4 Done Sent Done false
26032 7670 Acked Read 2 Done Sent Done false
26033 7671 Acked Read 9 Done Sent Done false
26034 7672 Acked Write 6 Done Sent Done false
26035 7673 Acked Write 8 Done Sent Done false
26036 7674 Acked Write 8 Done Sent Done false
26037 7675 Acked Read 5 Done Sent Done false
26038 7676 Acked Write 2 Done Sent Done false
26039 7677 Acked Read 10 Done Sent Done false
26040 7678 Acked Flush 0 Done Sent Done false
26041 7679 Acked Read 3 Done Sent Done false
26042 7680 Acked Read 3 Done Sent Done false
26043 7681 Acked Write 1 Done Sent Done false
26044 7682 Acked Read 2 Done Sent Done false
26045 7683 Acked Read 2 Done Sent Done false
26046 7684 Acked Write 1 Done Sent Done false
26047 7685 Acked Write 2 Done Sent Done false
26048 7686 Acked Read 7 Done Sent Done false
26049 7687 Acked Read 1 Done Sent Done false
26050 7688 Acked Write 2 Done Sent Done false
26051 7689 Acked Write 4 Done Sent Done false
26052 7690 Acked Write 6 Done Sent Done false
26053 7691 Acked Read 7 Done Sent Done false
26054 7692 Acked Write 8 Done Sent Done false
26055 7693 Acked Write 5 Done Sent Done false
26056 7694 Acked Write 5 Done Sent Done false
26057 7695 Acked Write 1 Done Sent Done false
26058 7696 Acked Write 9 Done Sent Done false
26059 7697 Acked Read 9 Done Sent Done false
26060 7698 Acked Write 10 Done Sent Done false
26061 7699 Acked Read 10 Done Sent Done false
26062 7700 Acked Read 9 Done Sent Done false
26063 7701 Acked Read 4 Done Sent Done false
26064 7702 Acked Read 9 Done Sent Done false
26065 7703 Acked Write 2 Done Sent Done false
26066 7704 Acked Write 1 Done Sent Done false
26067 7705 Acked Read 9 Done Sent Done false
26068 7706 Acked Write 7 Done Sent Done false
26069 7707 Acked Write 1 Done Sent Done false
26070 7708 Acked Write 8 Done Sent Done false
26071 7709 Acked Read 2 Done Sent Done false
26072 7710 Acked Write 8 Done Sent Done false
26073 7711 Acked Write 5 Done Sent Done false
26074 7712 Acked Read 6 Done Sent Done false
26075 7713 Acked Flush 0 Done Sent Done false
26076 7714 Acked Read 10 Done Sent Done false
26077 7715 Acked Write 5 Done Sent Done false
26078 7716 Acked Read 7 Done Sent Done false
26079 7717 Acked Write 1 Done Sent Done false
26080 7718 Acked Read 7 Done Sent Done false
26081 7719 Acked Write 6 Done Sent Done false
26082 7720 Acked Write 10 Done Sent Done false
26083 7721 Acked Write 1 Done Sent Done false
26084 7722 Acked Flush 0 Done Sent Done false
26085 7723 Acked Write 4 Done Sent Done false
26086 7724 Acked Read 10 Done Sent Done false
26087 7725 Acked Read 1 Done Sent Done false
26088 7726 Acked Read 1 Done Sent Done false
26089 7727 Acked Flush 0 Done Sent Done false
26090 7728 Acked Read 1 Done Sent Done false
26091 7729 Acked Write 10 Done Sent Done false
26092 7730 Acked Read 1 Done Sent Done false
26093 7731 Acked Read 6 Done Sent Done false
26094 7732 Acked Flush 0 Done Sent Done false
26095 7733 Acked Write 7 Done Sent Done false
26096 7734 Acked Write 8 Done Sent Done false
26097 7735 Acked Write 4 Done Sent Done false
26098 7736 Acked Read 7 Done Sent Done false
26099 7737 Acked Read 6 Done Sent Done false
26100 7738 Acked Write 7 Done Sent Done false
26101 7739 Acked Read 4 Done Sent Done false
26102 7740 Acked Write 2 Done Sent Done false
26103 7741 Acked Write 8 Done Sent Done false
26104 7742 Acked Read 9 Done Sent Done false
26105 7743 Acked Write 4 Done Sent Done false
26106 7744 Acked Write 2 Done Sent Done false
26107 7745 Acked Read 1 Done Sent Done false
26108 7746 Acked Read 1 Done Sent Done false
26109 7747 Acked Write 4 Done Sent Done false
26110 7748 Acked Write 8 Done Sent Done false
26111 7749 Acked Flush 0 Done Sent Done false
26112 7750 Acked Read 10 Done Sent Done false
26113 7751 Acked Write 8 Done Sent Done false
26114 7752 Acked Read 7 Done Sent Done false
26115 7753 Acked Write 7 Done Sent Done false
26116 7754 Acked Read 10 Done Sent Done false
26117 7755 Acked Read 5 Done Sent Done false
26118 7756 Acked Write 10 Done Sent Done false
26119 7757 Acked Read 3 Done Sent Done false
26120 7758 Acked Flush 0 Done Sent Done false
26121 7759 Acked Write 8 Done Sent Done false
26122 7760 Acked Read 3 Done Sent Done false
26123 7761 Acked Write 8 Done Sent Done false
26124 7762 Acked Read 2 Done Sent Done false
26125 7763 Acked Write 6 Done Sent Done false
26126 7764 Acked Write 7 Done Sent Done false
26127 7765 Acked Flush 0 Done Sent Done false
26128 7766 Acked Read 10 Done Sent Done false
26129 7767 Acked Write 2 Done Sent Done false
26130 7768 Acked Read 9 Done Sent Done false
26131 7769 Acked Read 3 Done Sent Done false
26132 7770 Acked Write 3 Done Sent Done false
26133 7771 Acked Read 6 Done Sent Done false
26134 7772 Acked Read 10 Done Sent Done false
26135 7773 Acked Read 10 Done Sent Done false
26136 7774 Acked Write 10 Done Sent Done false
26137 7775 Acked Write 6 Done Sent Done false
26138 7776 Acked Read 6 Done Sent Done false
26139 7777 Acked Read 1 Done Sent Done false
26140 7778 Acked Read 4 Done Sent Done false
26141 7779 Acked Read 8 Done Sent Done false
26142 7780 Acked Read 1 Done Sent Done false
26143 7781 Acked Flush 0 Done Sent Done false
26144 7782 Acked Read 3 Done Sent Done false
26145 7783 Acked Read 1 Done Sent Done false
26146 7784 Acked Write 9 Done Sent Done false
26147 7785 Acked Flush 0 Done Sent Done false
26148 7786 Acked Read 1 Done Sent Done false
26149 7787 Acked Write 2 Done Sent Done false
26150 7788 Acked Write 7 Done Sent Done false
26151 7789 Acked Read 2 Done Sent Done false
26152 7790 Acked Write 6 Done Sent Done false
26153 7791 Acked Write 7 Done Sent Done false
26154 7792 Acked Write 2 Done Sent Done false
26155 7793 Acked Read 10 Done Sent Done false
26156 7794 Acked Read 5 Done Sent Done false
26157 7795 Acked Read 5 Done Sent Done false
26158 7796 Acked Read 1 Done Sent Done false
26159 7797 Acked Write 6 Done Sent Done false
26160 7798 Acked Read 5 Done Sent Done false
26161 7799 Acked Write 7 Done Sent Done false
26162 7800 Acked Read 7 Done Sent Done false
26163 7801 Acked Write 6 Done Sent Done false
26164 7802 Acked Read 5 Done Sent Done false
26165 7803 Acked Read 2 Done Sent Done false
26166 7804 Acked Write 2 Done Sent Done false
26167 7805 Acked Write 4 Done Sent Done false
26168 7806 Acked Write 9 Done Sent Done false
26169 7807 Acked Read 7 Done Sent Done false
26170 7808 Acked Read 1 Done Sent Done false
26171 7809 Acked Write 8 Done Sent Done false
26172 7810 Acked Read 7 Done Sent Done false
26173 7811 Acked Write 1 Done Sent Done false
26174 7812 Acked Write 10 Done Sent Done false
26175 7813 Acked Write 5 Done Sent Done false
26176 7814 Acked Write 8 Done Sent Done false
26177 7815 Acked Flush 0 Done Sent Done false
26178 7816 Acked Read 2 Done Sent Done false
26179 7817 Acked Read 4 Done Sent Done false
26180 7818 Acked Flush 0 Done Sent Done false
26181 7819 Acked Read 10 Done Sent Done false
26182 7820 Acked Read 3 Done Sent Done false
26183 7821 Acked Write 7 Done Sent Done false
26184 7822 Acked Flush 0 Done Sent Done false
26185 7823 Acked Read 9 Done Sent Done false
26186 7824 Acked Write 5 Done Sent Done false
26187 7825 Acked Read 5 Done Sent Done false
26188 7826 Acked Write 4 Done Sent Done false
26189 7827 Acked Read 2 Done Sent Done false
26190 7828 Acked Write 1 Done Sent Done false
26191 7829 Acked Flush 0 Done Sent Done false
26192 7830 Acked Read 6 Done Sent Done false
26193 7831 Acked Read 1 Done Sent Done false
26194 7832 Acked Read 10 Done Sent Done false
26195 7833 Acked Write 10 Done Sent Done false
26196 7834 Acked Write 5 Done Sent Done false
26197 7835 Acked Write 5 Done Sent Done false
26198 7836 Acked Read 9 Done Sent Done false
26199 7837 Acked Write 3 Done Sent Done false
26200 7838 Acked Read 4 Done Sent Done false
26201 7839 Acked Write 5 Done Sent Done false
26202 7840 Acked Read 10 Done Sent Done false
26203 7841 Acked Read 2 Done Sent Done false
26204 7842 Acked Flush 0 Done Sent Done false
26205 7843 Acked Read 6 Done Sent Done false
26206 7844 Acked Write 3 Done Sent Done false
26207 7845 Acked Read 3 Done Sent Done false
26208 7846 Acked Write 6 Done Sent Done false
26209 7847 Acked Read 4 Done Sent Done false
26210 7848 Acked Read 4 Done Sent Done false
26211 7849 Acked Read 8 Done Sent Done false
26212 7850 Acked Read 1 Done Sent Done false
26213 7851 Acked Read 3 Done Sent Done false
26214 7852 Acked Read 5 Done Sent Done false
26215 7853 Acked Read 8 Done Sent Done false
26216 7854 Acked Flush 0 Done Sent Done false
26217 7855 Acked Write 6 Done Sent Done false
26218 7856 Acked Read 5 Done Sent Done false
26219 7857 Acked Flush 0 Done Sent Done false
26220 7858 Acked Read 10 Done Sent Done false
26221 7859 Acked Write 3 Done Sent Done false
26222 7860 Acked Write 2 Done Sent Done false
26223 7861 Acked Read 5 Done Sent Done false
26224 7862 Acked Write 3 Done Sent Done false
26225 7863 Acked Read 8 Done Sent Done false
26226 7864 Acked Read 9 Done Sent Done false
26227 7865 Acked Write 10 Done Sent Done false
26228 7866 Acked Read 1 Done Sent Done false
26229 7867 Acked Read 5 Done Sent Done false
26230 7868 Acked Read 9 Done Sent Done false
26231 7869 Acked Read 3 Done Sent Done false
26232 7870 Acked Write 7 Done Sent Done false
26233 7871 Acked Flush 0 Done Sent Done false
26234 7872 Acked Write 8 Done Sent Done false
26235 7873 Acked Write 4 Done Sent Done false
26236 7874 Acked Flush 0 Done Sent Done false
26237 7875 Acked Read 7 Done Sent Done false
26238 7876 Acked Write 4 Done Sent Done false
26239 7877 Acked Read 5 Done Sent Done false
26240 7878 Acked Write 1 Done Sent Done false
26241 7879 Acked Read 3 Done Sent Done false
26242 7880 Acked Read 8 Done Sent Done false
26243 7881 Acked Write 8 Done Sent Done false
26244 7882 Acked Write 1 Done Sent Done false
26245 7883 Acked Read 1 Done Sent Done false
26246 7884 Acked Read 8 Done Sent Done false
26247 7885 Acked Write 5 Done Sent Done false
26248 7886 Acked Read 2 Done Sent Done false
26249 7887 Acked Read 5 Done Sent Done false
26250 7888 Acked Write 7 Done Sent Done false
26251 7889 Acked Read 7 Done Sent Done false
26252 7890 Acked Read 6 Done Sent Done false
26253 7891 Acked Write 2 Done Sent Done false
26254 7892 Acked Read 5 Done Sent Done false
26255 7893 Acked Read 8 Done Sent Done false
26256 7894 Acked Read 5 Done Sent Done false
26257 7895 Acked Write 3 Done Sent Done false
26258 7896 Acked Read 2 Done Sent Done false
26259 7897 Acked Write 9 Done Sent Done false
26260 7898 Acked Read 8 Done Sent Done false
26261 7899 Acked Write 2 Done Sent Done false
26262 7900 Acked Write 7 Done Sent Done false
26263 7901 Acked Write 1 Done Sent Done false
26264 7902 Acked Read 8 Done Sent Done false
26265 7903 Acked Write 7 Done Sent Done false
26266 7904 Acked Write 7 Done Sent Done false
26267 7905 Acked Read 7 Done Sent Done false
26268 7906 Acked Write 9 Done Sent Done false
26269 7907 Acked Write 10 Done Sent Done false
26270 7908 Acked Read 1 Done Sent Done false
26271 7909 Acked Write 2 Done Sent Done false
26272 7910 Acked Write 9 Done Sent Done false
26273 7911 Acked Read 10 Done Sent Done false
26274 7912 Acked Read 6 Done Sent Done false
26275 7913 Acked Flush 0 Done Sent Done false
26276 7914 Acked Read 2 Done Sent Done false
26277 7915 Acked Write 6 Done Sent Done false
26278 7916 Acked Read 9 Done Sent Done false
26279 7917 Acked Flush 0 Done Sent Done false
26280 7918 Acked Read 4 Done Sent Done false
26281 7919 Acked Write 1 Done Sent Done false
26282 7920 Acked Read 2 Done Sent Done false
26283 7921 Acked Write 4 Done Sent Done false
26284 7922 Acked Write 4 Done Sent Done false
26285 7923 Acked Read 1 Done Sent Done false
26286 7924 Acked Flush 0 Done Sent Done false
26287 7925 Acked Read 3 Done Sent Done false
26288 7926 Acked Write 7 Done Sent Done false
26289 7927 Acked Write 1 Done Sent Done false
26290 7928 Acked Write 3 Done Sent Done false
26291 7929 Acked Read 6 Done Sent Done false
26292 7930 Acked Read 9 Done Sent Done false
26293 7931 Acked Read 5 Done Sent Done false
26294 7932 Acked Flush 0 Done Sent Done false
26295 7933 Acked Read 6 Done Sent Done false
26296 7934 Acked Write 1 Done Sent Done false
26297 7935 Acked Write 7 Done Sent Done false
26298 7936 Acked Read 1 Done Sent Done false
26299 7937 Acked Read 10 Done Sent Done false
26300 7938 Acked Read 7 Done Sent Done false
26301 7939 Acked Read 9 Done Sent Done false
26302 7940 Acked Read 1 Done Sent Done false
26303 7941 Acked Read 6 Done Sent Done false
26304 7942 Acked Read 8 Done Sent Done false
26305 7943 Acked Read 3 Done Sent Done false
26306 7944 Acked Flush 0 Done Sent Done false
26307 STATES DS:0 DS:1 DS:2 TOTAL
26308 Sent 0 302 0 302
26309 Done 302 0 302 604
26310 Skipped 0 0 0 0
26311 Error 0 0 0 0
26312 Last Flush: 7944 7642 7944
26313 Downstairs last five completed: 7642 7641 7640 7639 7638
26314 Upstairs last five completed: 7944 7943 7942 7941 7940
26315 CLIENT: Up:0 ds:302 act:2
263162025-01-15T05:24:30.312ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263172025-01-15T05:24:30.313ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50397) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263182025-01-15T05:24:30.314ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263192025-01-15T05:24:30.314ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263202025-01-15T05:24:30.314ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263212025-01-15T05:24:30.314ZINFOcrucible: send last flush ID to this DS: 7642 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263222025-01-15T05:24:30.315ZINFOcrucible: Replied this last flush ID: 7642 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263232025-01-15T05:24:30.315ZINFOcrucible: [1] client re-new 302 jobs since flush 7642 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263242025-01-15T05:24:30.315ZINFOcrucible: [1] Replayed 302 jobs since flush: 7642 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
26325 ----------------------------------------------------------------
26326 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26327 Downstairs last five completed: 7944 7943 7942 7941 7940
26328 Upstairs last five completed: 7944 7943 7942 7941 7940
26329 CLIENT: Up:0 ds:0 act:3
26330 Replay: All jobs finished
26331 Main task has work to do, go find it
26332 got dsc Stop(2)
26333 stop 2
26334 Jan 15 05:24:32.543 INFO request completed, latency_us: 352, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 910cc15f-c11f-4bb6-b386-6938043b5cb0, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
26335 [8830] Got stop action so:false kr:false
26336 Jan 15 05:24:32.544 INFO request completed, latency_us: 121, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2ae4a91c-ef43-4d2f-be10-4127c79686e8, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
263372025-01-15T05:24:32.544ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263382025-01-15T05:24:32.544ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263392025-01-15T05:24:32.544ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
263402025-01-15T05:24:32.544ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
26341 [8830] Exited with: ExitStatus(unix_wait_status(9))
26342 [8830][2] reports Exit
26343 Jan 15 05:24:36.547 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9171cbd8-e8b7-413b-836c-91d40bb984fe, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
26344 001/300 Read block 2143 len 12288
26345 002/300 Read block 811 len 12288
26346 003/300 Write block 3257 len 8192 data: 4 4
26347 004/300 Write block 4403 len 8192 data: 4 3
26348 005/300 Read block 3783 len 32768
26349 006/300 Write block 2108 len 16384 data: 5 7 6 7
26350 007/300 Read block 2564 len 36864
26351 008/300 Read block 392 len 24576
26352 009/300 Flush
26353 010/300 Flush
26354 011/300 Read block 4990 len 24576
26355 012/300 Write block 3282 len 24576 data: 5 4 3 3 3 3
26356 013/300 Write block 1368 len 4096 data: 7
26357 014/300 Read block 1529 len 12288
26358 015/300 Write block 2410 len 32768 data: 5 6 6 6 6 5 3 3
26359 016/300 Read block 4759 len 8192
26360 017/300 Flush
26361 018/300 Read block 3801 len 4096
26362 019/300 Write block 3221 len 8192 data: 4 4
26363 020/300 Flush
26364 021/300 Write block 589 len 36864 data: 5 5 5 5 6 8 8 7 5
26365 022/300 Read block 4950 len 8192
26366 023/300 Read block 4220 len 16384
26367 024/300 Write block 2958 len 12288 data: 3 3 3
26368 025/300 Read block 4346 len 28672
26369 026/300 Write block 448 len 20480 data: 9 8 8 6 6
26370 027/300 Read block 2265 len 8192
26371 028/300 Read block 4227 len 12288
26372 029/300 Read block 2281 len 4096
26373 030/300 Write block 4907 len 40960 data: 5 6 7 6 7 7 6 5 6 6
26374 031/300 Read block 4685 len 32768
26375 032/300 Write block 408 len 40960 data: 6 6 5 4 5 5 4 3 3 3
26376 033/300 Write block 2843 len 8192 data: 4 3
26377 034/300 Write block 2592 len 8192 data: 7 7
26378 035/300 Read block 947 len 24576
26379 036/300 Read block 2559 len 12288
26380 037/300 Read block 3493 len 16384
26381 038/300 Write block 1612 len 8192 data: 6 5
26382 039/300 Read block 1098 len 36864
26383 040/300 Write block 257 len 20480 data: 10 11 8 8 6
26384 041/300 Read block 1188 len 32768
26385 042/300 Flush
26386 043/300 Flush
26387 044/300 Write block 40 len 40960 data: 7 6 7 8 6 5 5 5 5 6
26388 045/300 Read block 1226 len 8192
26389 046/300 Read block 1799 len 12288
26390 047/300 Write block 166 len 8192 data: 5 4
26391 048/300 Write block 1683 len 4096 data: 5
26392 049/300 Read block 1803 len 4096
26393 050/300 Read block 312 len 32768
26394 051/300 Read block 348 len 24576
26395 052/300 Write block 663 len 12288 data: 6 5 5
26396 053/300 Read block 3751 len 12288
26397 054/300 Read block 1026 len 20480
26398 055/300 Write block 4249 len 4096 data: 7
26399 056/300 Read block 3135 len 20480
26400 057/300 Write block 3831 len 8192 data: 3 4
26401 058/300 Write block 2957 len 36864 data: 4 4 4 4 4 4 3 3 5
26402 059/300 Write block 4627 len 20480 data: 5 4 4 5 7
26403 060/300 Write block 4795 len 32768 data: 2 3 3 4 4 5 5 5
26404 061/300 Write block 3449 len 24576 data: 7 7 7 7 7 8
26405 062/300 Read block 3785 len 24576
26406 063/300 Write block 981 len 40960 data: 2 2 2 3 4 4 4 4 4 4
26407 064/300 Read block 4938 len 16384
26408 065/300 Flush
26409 066/300 Read block 4939 len 20480
26410 067/300 Write block 1006 len 16384 data: 2 2 2 2
26411 068/300 Read block 607 len 12288
26412 069/300 Flush
26413 070/300 Read block 1342 len 24576
26414 071/300 Read block 3967 len 36864
26415 072/300 Read block 751 len 28672
26416 073/300 Flush
26417 074/300 Read block 2447 len 12288
26418 075/300 Write block 195 len 32768 data: 4 6 5 3 5 5 4 4
26419 076/300 Write block 4210 len 32768 data: 7 7 6 5 4 5 5 7
26420 077/300 Flush
26421 078/300 Read block 941 len 4096
26422 079/300 Flush
26423 080/300 Write block 632 len 8192 data: 4 4
26424 081/300 Write block 1056 len 36864 data: 3 4 3 3 3 2 2 2 2
26425 082/300 Write block 2990 len 40960 data: 4 3 4 5 6 5 5 6 3 4
26426 083/300 Read block 2590 len 12288
26427 084/300 Flush
26428 085/300 Write block 262 len 36864 data: 6 7 8 7 7 6 6 6 6
26429 086/300 Read block 304 len 8192
26430 087/300 Read block 1685 len 28672
26431 088/300 Read block 2840 len 20480
26432 089/300 Write block 1035 len 12288 data: 5 6 5
26433 090/300 Write block 4634 len 12288 data: 5 4 4
26434 091/300 Read block 1321 len 32768
26435 092/300 Write block 2090 len 8192 data: 6 6
26436 093/300 Read block 3428 len 32768
26437 094/300 Read block 1995 len 4096
26438 095/300 Flush
26439 096/300 Write block 539 len 24576 data: 5 5 4 5 4 3
26440 097/300 Write block 3442 len 16384 data: 5 4 4 5
26441 098/300 Write block 201 len 24576 data: 5 5 5 5 5 7
26442 099/300 Read block 627 len 12288
26443 100/300 Write block 245 len 40960 data: 3 4 5 7 6 6 6 6 6 8
26444 101/300 Read block 3015 len 32768
26445 102/300 Write block 2981 len 28672 data: 6 8 6 7 7 7 5
26446 103/300 Read block 842 len 28672
26447 104/300 Flush
26448 105/300 Write block 1981 len 8192 data: 5 5
26449 106/300 Read block 1502 len 28672
26450 107/300 Write block 3468 len 12288 data: 5 5 6
26451 108/300 Write block 2384 len 40960 data: 6 5 4 4 3 3 3 5 6 4
26452 109/300 Read block 4254 len 36864
26453 110/300 Read block 1600 len 16384
26454 111/300 Read block 217 len 40960
26455 112/300 Write block 4567 len 36864 data: 3 3 3 3 3 6 9 8 8
26456 113/300 Write block 1445 len 16384 data: 2 2 2 2
26457 114/300 Flush
26458 115/300 Read block 1522 len 24576
26459 116/300 Write block 1150 len 32768 data: 6 4 4 6 7 5 5 6
26460 117/300 Write block 255 len 8192 data: 7 9
26461 118/300 Read block 1670 len 28672
26462 119/300 Read block 1941 len 40960
26463 120/300 Read block 4128 len 24576
26464 121/300 Read block 4873 len 40960
26465 122/300 Read block 1927 len 40960
26466 123/300 Write block 37 len 4096 data: 5
26467 124/300 Write block 3826 len 36864 data: 3 3 3 3 3 4 5 7 8
26468 125/300 Read block 4540 len 36864
26469 126/300 Write block 4868 len 8192 data: 5 8
26470 127/300 Read block 4517 len 24576
26471 128/300 Read block 1113 len 20480
26472 129/300 Read block 2320 len 16384
26473 130/300 Read block 2756 len 36864
26474 131/300 Read block 891 len 28672
26475 132/300 Read block 3773 len 24576
26476 133/300 Read block 1357 len 36864
26477 134/300 Read block 4550 len 28672
26478 135/300 Write block 3037 len 28672 data: 11 10 10 10 9 9 8
26479 136/300 Read block 2510 len 8192
26480 137/300 Read block 4505 len 4096
26481 138/300 Read block 4089 len 20480
26482 139/300 Write block 345 len 8192 data: 6 4
26483 140/300 Read block 1574 len 28672
26484 141/300 Write block 3651 len 4096 data: 8
26485 142/300 Write block 4633 len 28672 data: 6 6 5 5 4 3 3
26486 143/300 Read block 2371 len 28672
26487 144/300 Read block 421 len 36864
26488 145/300 Read block 2703 len 20480
26489 146/300 Write block 577 len 8192 data: 6 6
26490 147/300 Flush
26491 148/300 Flush
26492 149/300 Write block 726 len 32768 data: 5 5 4 5 6 5 5 5
26493 150/300 Read block 2807 len 40960
26494 151/300 Read block 1320 len 20480
26495 152/300 Read block 726 len 8192
26496 153/300 Flush
26497 154/300 Read block 3483 len 40960
26498 155/300 Read block 3103 len 28672
26499 156/300 Flush
26500 157/300 Write block 3600 len 12288 data: 6 6 6
26501 158/300 Read block 2144 len 12288
26502 159/300 Write block 82 len 12288 data: 5 4 5
26503 160/300 Write block 548 len 12288 data: 4 5 5
26504 161/300 Write block 610 len 36864 data: 5 5 5 4 4 5 3 3 3
26505 162/300 Read block 3499 len 32768
26506 163/300 Flush
26507 164/300 Read block 2931 len 32768
26508 165/300 Flush
26509 166/300 Read block 4775 len 8192
26510 167/300 Read block 1833 len 8192
26511 168/300 Read block 173 len 28672
26512 169/300 Write block 4055 len 16384 data: 4 5 6 4
26513 170/300 Read block 4691 len 12288
26514 171/300 Read block 1227 len 8192
26515 172/300 Write block 3198 len 12288 data: 4 5 5
26516 173/300 Write block 3412 len 12288 data: 4 5 5
26517 174/300 Read block 4024 len 24576
26518 175/300 Read block 543 len 16384
26519 176/300 Read block 133 len 40960
26520 177/300 Write block 3483 len 16384 data: 5 4 4 4
26521 178/300 Read block 891 len 16384
26522 179/300 Write block 762 len 12288 data: 3 2 2
26523 180/300 Flush
26524 181/300 Write block 4237 len 20480 data: 4 4 5 8 11
26525 182/300 Read block 3297 len 4096
26526 183/300 Write block 1255 len 20480 data: 6 5 5 4 4
26527 184/300 Write block 1100 len 16384 data: 4 3 4 4
26528 185/300 Read block 2029 len 40960
26529 186/300 Flush
26530 187/300 Read block 1256 len 12288
26531 188/300 Read block 586 len 40960
26532 189/300 Write block 1355 len 8192 data: 4 4
26533 190/300 Read block 1964 len 40960
26534 191/300 Read block 2285 len 28672
26535 192/300 Write block 834 len 24576 data: 4 4 4 4 3 4
26536 193/300 Write block 1443 len 28672 data: 4 2 3 3 3 3 3
26537 194/300 Flush
26538 195/300 Read block 250 len 4096
26539 196/300 Read block 2535 len 16384
26540 197/300 Write block 4763 len 24576 data: 3 3 3 3 3 3
26541 198/300 Read block 2415 len 28672
26542 199/300 Write block 1534 len 28672 data: 5 5 6 4 5 6 6
26543 200/300 Read block 1775 len 40960
26544 201/300 Read block 3795 len 24576
26545 202/300 Read block 1986 len 24576
26546 203/300 Write block 2606 len 20480 data: 5 5 4 5 5
26547 204/300 Write block 3167 len 40960 data: 5 4 5 4 5 4 5 4 3 3
26548 205/300 Read block 805 len 16384
26549 206/300 Read block 3732 len 4096
26550 207/300 Read block 974 len 28672
26551 208/300 Read block 3489 len 36864
26552 209/300 Read block 465 len 16384
26553 210/300 Write block 4489 len 28672 data: 4 4 5 3 5 5 7
26554 211/300 Write block 1854 len 36864 data: 3 3 4 4 2 2 2 4 4
26555 212/300 Read block 4157 len 28672
26556 213/300 Read block 315 len 4096
26557 214/300 Read block 1266 len 12288
26558 215/300 Read block 2563 len 28672
26559 216/300 Write block 3767 len 16384 data: 6 6 4 5
26560 217/300 Flush
26561 218/300 Read block 236 len 12288
26562 219/300 Read block 1066 len 32768
26563 220/300 Read block 4726 len 20480
26564 221/300 Read block 3531 len 16384
26565 222/300 Write block 1696 len 40960 data: 3 3 4 4 5 5 4 4 5 4
26566 223/300 Read block 2117 len 24576
26567 224/300 Write block 3440 len 32768 data: 5 5 6 5 5 6 5 5
26568 225/300 Write block 1302 len 28672 data: 6 6 7 7 8 7 5
26569 226/300 Write block 3018 len 4096 data: 4
26570 227/300 Write block 4884 len 20480 data: 7 7 7 7 8
26571 228/300 Read block 1575 len 12288
26572 229/300 Read block 2772 len 12288
26573 230/300 Flush
26574 231/300 Read block 273 len 24576
26575 232/300 Read block 2571 len 24576
26576 233/300 Read block 665 len 28672
26577 234/300 Flush
26578 235/300 Flush
26579 236/300 Write block 506 len 40960 data: 5 5 5 4 5 3 4 4 4 6
26580 237/300 Write block 4109 len 24576 data: 4 4 4 3 3 6
26581 238/300 Read block 4694 len 36864
26582 239/300 Read block 3240 len 8192
26583 240/300 Write block 4454 len 20480 data: 5 6 5 5 4
26584 241/300 Read block 752 len 40960
26585 242/300 Read block 3989 len 28672
26586 243/300 Read block 1341 len 16384
26587 244/300 Write block 1260 len 24576 data: 5 6 5 3 3 5
26588 245/300 Read block 510 len 32768
26589 246/300 Write block 4308 len 36864 data: 3 3 3 3 2 3 4 4 6
26590 247/300 Read block 4816 len 36864
26591 248/300 Read block 1995 len 36864
26592 249/300 Write block 489 len 28672 data: 4 5 5 4 4 5 5
26593 250/300 Read block 3358 len 12288
26594 251/300 Write block 3804 len 36864 data: 3 3 3 3 3 4 5 4 5
26595 252/300 Write block 716 len 28672 data: 3 4 4 4 5 4 4
26596 253/300 Write block 947 len 36864 data: 5 5 4 5 5 4 3 3 3
26597 254/300 Read block 2352 len 20480
26598 255/300 Read block 3192 len 8192
26599 256/300 Write block 4195 len 40960 data: 6 6 5 4 2 2 2 2 2 2
26600 257/300 Flush
26601 258/300 Write block 133 len 12288 data: 6 6 4
26602 259/300 Read block 4465 len 8192
26603 260/300 Flush
26604 261/300 Read block 2827 len 4096
26605 262/300 Write block 585 len 28672 data: 5 5 5 5 6 6 6
26606 263/300 Write block 3447 len 24576 data: 6 7 8 8 8 8
26607 264/300 Write block 3770 len 40960 data: 6 4 4 5 5 4 4 4 5 6
26608 265/300 Read block 786 len 4096
26609 266/300 Read block 3725 len 40960
26610 267/300 Read block 104 len 24576
26611 268/300 Read block 4592 len 20480
26612 269/300 Flush
26613 270/300 Read block 340 len 32768
26614 271/300 Read block 3814 len 20480
26615 272/300 Read block 2647 len 24576
26616 273/300 Write block 287 len 20480 data: 6 6 7 7 7
26617 274/300 Read block 640 len 4096
26618 275/300 Read block 221 len 16384
26619 276/300 Read block 3226 len 8192
26620 277/300 Write block 2772 len 8192 data: 5 5
26621 278/300 Read block 2167 len 24576
26622 279/300 Read block 3139 len 16384
26623 280/300 Write block 2667 len 12288 data: 2 2 3
26624 281/300 Write block 3089 len 20480 data: 8 9 9 9 9
26625 282/300 Read block 376 len 36864
26626 283/300 Write block 200 len 20480 data: 6 6 6 6 6
26627 284/300 Read block 2095 len 12288
26628 285/300 Write block 4175 len 24576 data: 2 2 4 5 6 6
26629 286/300 Read block 4831 len 24576
26630 287/300 Write block 1813 len 16384 data: 6 6 6 7
26631 288/300 Write block 2301 len 20480 data: 7 9 9 6 6
26632 289/300 Write block 3831 len 16384 data: 5 6 8 9
26633 290/300 Write block 442 len 20480 data: 10 10 10 12 12
26634 291/300 Write block 2161 len 36864 data: 3 4 5 5 3 4 5 5 5
26635 292/300 Write block 3261 len 20480 data: 4 5 4 5 5
26636 293/300 Write block 2279 len 24576 data: 4 6 6 7 5 7
26637 294/300 Read block 1177 len 20480
26638 295/300 Write block 4388 len 20480 data: 5 4 5 4 3
26639 296/300 Read block 634 len 12288
26640 297/300 Write block 935 len 36864 data: 4 4 5 3 4 4 3 4 4
26641 298/300 Write block 4951 len 12288 data: 3 3 3
26642 299/300 Write block 3211 len 40960 data: 2 3 4 4 3 3 4 4 4 4
26643 300/300 Write block 921 len 20480 data: 2 3 3 3 3
26644 301/300 Write block 2674 len 36864 data: 5 4 5 4 5 5 5 4 5
26645 Main task has work to do, go find it
26646 got dsc Start(2)
26647 start 2
26648 Jan 15 05:24:36.778 INFO request completed, latency_us: 230, response_code: 204, uri: /start/cid/2, method: POST, req_id: 9ae21b2b-9523-4bb3-a234-eb00542a07e4, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
26649 Replay: started 2, returned:Ok(())
26650 ----------------------------------------------------------------
26651 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26652 States: Active Active Offline
26653 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26654 7945 Acked Read 3 Done Done Sent false
26655 7946 Acked Read 3 Done Done Sent false
26656 7947 Acked Write 2 Done Done Sent false
26657 7948 Acked Write 2 Done Done Sent false
26658 7949 Acked Read 8 Done Done Sent false
26659 7950 Acked Write 4 Done Done Sent false
26660 7951 Acked Read 9 Done Done Sent false
26661 7952 Acked Read 6 Done Done Sent false
26662 7953 Acked Flush 0 Done Done Sent false
26663 7954 Acked Flush 0 Done Done Sent false
26664 7955 Acked Read 6 Done Done Sent false
26665 7956 Acked Write 6 Done Done Sent false
26666 7957 Acked Write 1 Done Done Sent false
26667 7958 Acked Read 3 Done Done Sent false
26668 7959 Acked Write 8 Done Done Sent false
26669 7960 Acked Read 2 Done Done Sent false
26670 7961 Acked Flush 0 Done Done Sent false
26671 7962 Acked Read 1 Done Done Sent false
26672 7963 Acked Write 2 Done Done Sent false
26673 7964 Acked Flush 0 Done Done Sent false
26674 7965 Acked Write 9 Done Done Sent false
26675 7966 Acked Read 2 Done Done Sent false
26676 7967 Acked Read 4 Done Done Sent false
26677 7968 Acked Write 3 Done Done Sent false
26678 7969 Acked Read 7 Done Done Sent false
26679 7970 Acked Write 5 Done Done Sent false
26680 7971 Acked Read 2 Done Done Sent false
26681 7972 Acked Read 3 Done Done Sent false
26682 7973 Acked Read 1 Done Done Sent false
26683 7974 Acked Write 10 Done Done Sent false
26684 7975 Acked Read 8 Done Done Sent false
26685 7976 Acked Write 10 Done Done Sent false
26686 7977 Acked Write 2 Done Done Sent false
26687 7978 Acked Write 2 Done Done Sent false
26688 7979 Acked Read 6 Done Done Sent false
26689 7980 Acked Read 3 Done Done Sent false
26690 7981 Acked Read 4 Done Done Sent false
26691 7982 Acked Write 2 Done Done Sent false
26692 7983 Acked Read 9 Done Done Sent false
26693 7984 Acked Write 5 Done Done Sent false
26694 7985 Acked Read 8 Done Done Sent false
26695 7986 Acked Flush 0 Done Done Sent false
26696 7987 Acked Flush 0 Done Done Sent false
26697 7988 Acked Write 10 Done Done Sent false
26698 7989 Acked Read 2 Done Done Sent false
26699 7990 Acked Read 3 Done Done Sent false
26700 7991 Acked Write 2 Done Done Sent false
26701 7992 Acked Write 1 Done Done Sent false
26702 7993 Acked Read 1 Done Done Sent false
26703 7994 Acked Read 8 Done Done Sent false
26704 7995 Acked Read 6 Done Done Sent false
26705 7996 Acked Write 3 Done Done Sent false
26706 7997 Acked Read 3 Done Done Sent false
26707 7998 Acked Read 5 Done Done Sent false
26708 7999 Acked Write 1 Done Done Sent false
26709 8000 Acked Read 5 Done Done Sent false
26710 8001 Acked Write 2 Done Done Sent false
26711 8002 Acked Write 9 Done Done Sent false
26712 8003 Acked Write 5 Done Done Sent false
26713 8004 Acked Write 8 Done Done Sent false
26714 8005 Acked Write 6 Done Done Sent false
26715 8006 Acked Read 6 Done Done Sent false
26716 8007 Acked Write 10 Done Done Sent false
26717 8008 Acked Read 4 Done Done Sent false
26718 8009 Acked Flush 0 Done Done Sent false
26719 8010 Acked Read 5 Done Done Sent false
26720 8011 Acked Write 4 Done Done Sent false
26721 8012 Acked Read 3 Done Done Sent false
26722 8013 Acked Flush 0 Done Done Sent false
26723 8014 Acked Read 6 Done Done Sent false
26724 8015 Acked Read 9 Done Done Sent false
26725 8016 Acked Read 7 Done Done Sent false
26726 8017 Acked Flush 0 Done Done Sent false
26727 8018 Acked Read 3 Done Done Sent false
26728 8019 Acked Write 8 Done Done Sent false
26729 8020 Acked Write 8 Done Done Sent false
26730 8021 Acked Flush 0 Done Done Sent false
26731 8022 Acked Read 1 Done Done Sent false
26732 8023 Acked Flush 0 Done Done Sent false
26733 8024 Acked Write 2 Done Done Sent false
26734 8025 Acked Write 9 Done Done Sent false
26735 8026 Acked Write 10 Done Done Sent false
26736 8027 Acked Read 3 Done Done Sent false
26737 8028 Acked Flush 0 Done Done Sent false
26738 8029 Acked Write 9 Done Done Sent false
26739 8030 Acked Read 2 Done Done Sent false
26740 8031 Acked Read 7 Done Done Sent false
26741 8032 Acked Read 5 Done Done Sent false
26742 8033 Acked Write 3 Done Done Sent false
26743 8034 Acked Write 3 Done Done Sent false
26744 8035 Acked Read 8 Done Done Sent false
26745 8036 Acked Write 2 Done Done Sent false
26746 8037 Acked Read 8 Done Done Sent false
26747 8038 Acked Read 1 Done Done Sent false
26748 8039 Acked Flush 0 Done Done Sent false
26749 8040 Acked Write 6 Done Done Sent false
26750 8041 Acked Write 4 Done Done Sent false
26751 8042 Acked Write 6 Done Done Sent false
26752 8043 Acked Read 3 Done Done Sent false
26753 8044 Acked Write 10 Done Done Sent false
26754 8045 Acked Read 8 Done Done Sent false
26755 8046 Acked Write 7 Done Done Sent false
26756 8047 Acked Read 7 Done Done Sent false
26757 8048 Acked Flush 0 Done Done Sent false
26758 8049 Acked Write 2 Done Done Sent false
26759 8050 Acked Read 7 Done Done Sent false
26760 8051 Acked Write 3 Done Done Sent false
26761 8052 Acked Write 10 Done Done Sent false
26762 8053 Acked Read 9 Done Done Sent false
26763 8054 Acked Read 4 Done Done Sent false
26764 8055 Acked Read 10 Done Done Sent false
26765 8056 Acked Write 9 Done Done Sent false
26766 8057 Acked Write 4 Done Done Sent false
26767 8058 Acked Flush 0 Done Done Sent false
26768 8059 Acked Read 6 Done Done Sent false
26769 8060 Acked Write 8 Done Done Sent false
26770 8061 Acked Write 2 Done Done Sent false
26771 8062 Acked Read 7 Done Done Sent false
26772 8063 Acked Read 10 Done Done Sent false
26773 8064 Acked Read 6 Done Done Sent false
26774 8065 Acked Read 10 Done Done Sent false
26775 8066 Acked Read 10 Done Done Sent false
26776 8067 Acked Write 1 Done Done Sent false
26777 8068 Acked Write 9 Done Done Sent false
26778 8069 Acked Read 9 Done Done Sent false
26779 8070 Acked Write 2 Done Done Sent false
26780 8071 Acked Read 6 Done Done Sent false
26781 8072 Acked Read 5 Done Done Sent false
26782 8073 Acked Read 4 Done Done Sent false
26783 8074 Acked Read 9 Done Done Sent false
26784 8075 Acked Read 7 Done Done Sent false
26785 8076 Acked Read 6 Done Done Sent false
26786 8077 Acked Read 9 Done Done Sent false
26787 8078 Acked Read 7 Done Done Sent false
26788 8079 Acked Write 7 Done Done Sent false
26789 8080 Acked Read 2 Done Done Sent false
26790 8081 Acked Read 1 Done Done Sent false
26791 8082 Acked Read 5 Done Done Sent false
26792 8083 Acked Write 2 Done Done Sent false
26793 8084 Acked Read 7 Done Done Sent false
26794 8085 Acked Write 1 Done Done Sent false
26795 8086 Acked Write 7 Done Done Sent false
26796 8087 Acked Read 7 Done Done Sent false
26797 8088 Acked Read 9 Done Done Sent false
26798 8089 Acked Read 5 Done Done Sent false
26799 8090 Acked Write 2 Done Done Sent false
26800 8091 Acked Flush 0 Done Done Sent false
26801 8092 Acked Flush 0 Done Done Sent false
26802 8093 Acked Write 8 Done Done Sent false
26803 8094 Acked Read 10 Done Done Sent false
26804 8095 Acked Read 5 Done Done Sent false
26805 8096 Acked Read 2 Done Done Sent false
26806 8097 Acked Flush 0 Done Done Sent false
26807 8098 Acked Read 10 Done Done Sent false
26808 8099 Acked Read 7 Done Done Sent false
26809 8100 Acked Flush 0 Done Done Sent false
26810 8101 Acked Write 3 Done Done Sent false
26811 8102 Acked Read 3 Done Done Sent false
26812 8103 Acked Write 3 Done Done Sent false
26813 8104 Acked Write 3 Done Done Sent false
26814 8105 Acked Write 9 Done Done Sent false
26815 8106 Acked Read 8 Done Done Sent false
26816 8107 Acked Flush 0 Done Done Sent false
26817 8108 Acked Read 8 Done Done Sent false
26818 8109 Acked Flush 0 Done Done Sent false
26819 8110 Acked Read 2 Done Done Sent false
26820 8111 Acked Read 2 Done Done Sent false
26821 8112 Acked Read 7 Done Done Sent false
26822 8113 Acked Write 4 Done Done Sent false
26823 8114 Acked Read 3 Done Done Sent false
26824 8115 Acked Read 2 Done Done Sent false
26825 8116 Acked Write 3 Done Done Sent false
26826 8117 Acked Write 3 Done Done Sent false
26827 8118 Acked Read 6 Done Done Sent false
26828 8119 Acked Read 4 Done Done Sent false
26829 8120 Acked Read 10 Done Done Sent false
26830 8121 Acked Write 4 Done Done Sent false
26831 8122 Acked Read 4 Done Done Sent false
26832 8123 Acked Write 3 Done Done Sent false
26833 8124 Acked Flush 0 Done Done Sent false
26834 8125 Acked Write 5 Done Done Sent false
26835 8126 Acked Read 1 Done Done Sent false
26836 8127 Acked Write 5 Done Done Sent false
26837 8128 Acked Write 4 Done Done Sent false
26838 8129 Acked Read 10 Done Done Sent false
26839 8130 Acked Flush 0 Done Done Sent false
26840 8131 Acked Read 3 Done Done Sent false
26841 8132 Acked Read 10 Done Done Sent false
26842 8133 Acked Write 2 Done Done Sent false
26843 8134 Acked Read 10 Done Done Sent false
26844 8135 Acked Read 7 Done Done Sent false
26845 8136 Acked Write 6 Done Done Sent false
26846 8137 Acked Write 7 Done Done Sent false
26847 8138 Acked Flush 0 Done Done Sent false
26848 8139 Acked Read 1 Done Done Sent false
26849 8140 Acked Read 4 Done Done Sent false
26850 8141 Acked Write 6 Done Done Sent false
26851 8142 Acked Read 7 Done Done Sent false
26852 8143 Acked Write 7 Done Done Sent false
26853 8144 Acked Read 10 Done Done Sent false
26854 8145 Acked Read 6 Done Done Sent false
26855 8146 Acked Read 6 Done Done Sent false
26856 8147 Acked Write 5 Done Done Sent false
26857 8148 Acked Write 10 Done Done Sent false
26858 8149 Acked Read 4 Done Done Sent false
26859 8150 Acked Read 1 Done Done Sent false
26860 8151 Acked Read 7 Done Done Sent false
26861 8152 Acked Read 9 Done Done Sent false
26862 8153 Acked Read 4 Done Done Sent false
26863 8154 Acked Write 7 Done Done Sent false
26864 8155 Acked Write 9 Done Done Sent false
26865 8156 Acked Read 7 Done Done Sent false
26866 8157 Acked Read 1 Done Done Sent false
26867 8158 Acked Read 3 Done Done Sent false
26868 8159 Acked Read 7 Done Done Sent false
26869 8160 Acked Write 4 Done Done Sent false
26870 8161 Acked Flush 0 Done Done Sent false
26871 8162 Acked Read 3 Done Done Sent false
26872 8163 Acked Read 8 Done Done Sent false
26873 8164 Acked Read 5 Done Done Sent false
26874 8165 Acked Read 4 Done Done Sent false
26875 8166 Acked Write 10 Done Done Sent false
26876 8167 Acked Read 6 Done Done Sent false
26877 8168 Acked Write 8 Done Done Sent false
26878 8169 Acked Write 7 Done Done Sent false
26879 8170 Acked Write 1 Done Done Sent false
26880 8171 Acked Write 5 Done Done Sent false
26881 8172 Acked Read 3 Done Done Sent false
26882 8173 Acked Read 3 Done Done Sent false
26883 8174 Acked Flush 0 Done Done Sent false
26884 8175 Acked Read 6 Done Done Sent false
26885 8176 Acked Read 6 Done Done Sent false
26886 8177 Acked Read 7 Done Done Sent false
26887 8178 Acked Flush 0 Done Done Sent false
26888 8179 Acked Flush 0 Done Done Sent false
26889 8180 Acked Write 10 Done Done Sent false
26890 8181 Acked Write 6 Done Done Sent false
26891 8182 Acked Read 9 Done Done Sent false
26892 8183 Acked Read 2 Done Done Sent false
26893 8184 Acked Write 5 Done Done Sent false
26894 8185 Acked Read 10 Done Done Sent false
26895 8186 Acked Read 7 Done Done Sent false
26896 8187 Acked Read 4 Done Done Sent false
26897 8188 Acked Write 6 Done Done Sent false
26898 8189 Acked Read 8 Done Done Sent false
26899 8190 Acked Write 9 Done Done Sent false
26900 8191 Acked Read 9 Done Done Sent false
26901 8192 Acked Read 9 Done Done Sent false
26902 8193 Acked Write 7 Done Done Sent false
26903 8194 Acked Read 3 Done Done Sent false
26904 8195 Acked Write 9 Done Done Sent false
26905 8196 Acked Write 7 Done Done Sent false
26906 8197 Acked Write 9 Done Done Sent false
26907 8198 Acked Read 5 Done Done Sent false
26908 8199 Acked Read 2 Done Done Sent false
26909 8200 Acked Write 10 Done Done Sent false
26910 8201 Acked Flush 0 Done Done Sent false
26911 8202 Acked Write 3 Done Done Sent false
26912 8203 Acked Read 2 Done Done Sent false
26913 8204 Acked Flush 0 Done Done Sent false
26914 8205 Acked Read 1 Done Done Sent false
26915 8206 Acked Write 7 Done Done Sent false
26916 8207 Acked Write 6 Done Done Sent false
26917 8208 Acked Write 10 Done Done Sent false
26918 8209 Acked Read 1 Done Done Sent false
26919 8210 Acked Read 10 Done Done Sent false
26920 8211 Acked Read 6 Done Done Sent false
26921 8212 Acked Read 5 Done Done Sent false
26922 8213 Acked Flush 0 Done Done Sent false
26923 8214 Acked Read 8 Done Done Sent false
26924 8215 Acked Read 5 Done Done Sent false
26925 8216 Acked Read 6 Done Done Sent false
26926 8217 Acked Write 5 Done Done Sent false
26927 8218 Acked Read 1 Done Done Sent false
26928 8219 Acked Read 4 Done Done Sent false
26929 8220 Acked Read 2 Done Done Sent false
26930 8221 Acked Write 2 Done Done Sent false
26931 8222 Acked Read 6 Done Done Sent false
26932 8223 Acked Read 4 Done Done Sent false
26933 8224 Acked Write 3 Done Done Sent false
26934 8225 Acked Write 5 Done Done Sent false
26935 8226 Acked Read 9 Done Done Sent false
26936 8227 Acked Write 5 Done Done Sent false
26937 8228 Acked Read 3 Done Done Sent false
26938 8229 Acked Write 6 Done Done Sent false
26939 8230 Acked Read 6 Done Done Sent false
26940 8231 Acked Write 4 Done Done Sent false
26941 8232 Acked Write 5 Done Done Sent false
26942 8233 Acked Write 4 Done Done Sent false
26943 8234 Acked Write 5 Done Done Sent false
26944 8235 Acked Write 9 Done Done Sent false
26945 8236 Acked Write 5 Done Done Sent false
26946 8237 Acked Write 6 Done Done Sent false
26947 8238 Acked Read 5 Done Done Sent false
26948 8239 Acked Write 5 Done Done Sent false
26949 8240 Acked Read 3 Done Done Sent false
26950 8241 Acked Write 9 Done Done Sent false
26951 8242 Acked Write 3 Done Done Sent false
26952 8243 Acked Write 10 Done Done Sent false
26953 8244 Acked Write 5 Done Done Sent false
26954 8245 Acked Write 9 Done Done Sent false
26955 STATES DS:0 DS:1 DS:2 TOTAL
26956 Sent 0 0 301 301
26957 Done 301 301 0 602
26958 Skipped 0 0 0 0
26959 Error 0 0 0 0
26960 Last Flush: 8213 8213 7944
26961 Downstairs last five completed: 7944 7943 7942 7941 7940
26962 Upstairs last five completed: 8245 8244 8243 8242 8241
26963 CLIENT: Up:0 ds:301 act:2
26964 ----------------------------------------------------------------
26965 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26966 States: Active Active Offline
26967 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26968 7945 Acked Read 3 Done Done Sent false
26969 7946 Acked Read 3 Done Done Sent false
26970 7947 Acked Write 2 Done Done Sent false
26971 7948 Acked Write 2 Done Done Sent false
26972 7949 Acked Read 8 Done Done Sent false
26973 7950 Acked Write 4 Done Done Sent false
26974 7951 Acked Read 9 Done Done Sent false
26975 7952 Acked Read 6 Done Done Sent false
26976 7953 Acked Flush 0 Done Done Sent false
26977 7954 Acked Flush 0 Done Done Sent false
26978 7955 Acked Read 6 Done Done Sent false
26979 7956 Acked Write 6 Done Done Sent false
26980 7957 Acked Write 1 Done Done Sent false
26981 7958 Acked Read 3 Done Done Sent false
26982 7959 Acked Write 8 Done Done Sent false
26983 7960 Acked Read 2 Done Done Sent false
26984 7961 Acked Flush 0 Done Done Sent false
26985 7962 Acked Read 1 Done Done Sent false
26986 7963 Acked Write 2 Done Done Sent false
26987 7964 Acked Flush 0 Done Done Sent false
26988 7965 Acked Write 9 Done Done Sent false
26989 7966 Acked Read 2 Done Done Sent false
26990 7967 Acked Read 4 Done Done Sent false
26991 7968 Acked Write 3 Done Done Sent false
26992 7969 Acked Read 7 Done Done Sent false
26993 7970 Acked Write 5 Done Done Sent false
26994 7971 Acked Read 2 Done Done Sent false
26995 7972 Acked Read 3 Done Done Sent false
26996 7973 Acked Read 1 Done Done Sent false
26997 7974 Acked Write 10 Done Done Sent false
26998 7975 Acked Read 8 Done Done Sent false
26999 7976 Acked Write 10 Done Done Sent false
27000 7977 Acked Write 2 Done Done Sent false
27001 7978 Acked Write 2 Done Done Sent false
27002 7979 Acked Read 6 Done Done Sent false
27003 7980 Acked Read 3 Done Done Sent false
27004 7981 Acked Read 4 Done Done Sent false
27005 7982 Acked Write 2 Done Done Sent false
27006 7983 Acked Read 9 Done Done Sent false
27007 7984 Acked Write 5 Done Done Sent false
27008 7985 Acked Read 8 Done Done Sent false
27009 7986 Acked Flush 0 Done Done Sent false
27010 7987 Acked Flush 0 Done Done Sent false
27011 7988 Acked Write 10 Done Done Sent false
27012 7989 Acked Read 2 Done Done Sent false
27013 7990 Acked Read 3 Done Done Sent false
27014 7991 Acked Write 2 Done Done Sent false
27015 7992 Acked Write 1 Done Done Sent false
27016 7993 Acked Read 1 Done Done Sent false
27017 7994 Acked Read 8 Done Done Sent false
27018 7995 Acked Read 6 Done Done Sent false
27019 7996 Acked Write 3 Done Done Sent false
27020 7997 Acked Read 3 Done Done Sent false
27021 7998 Acked Read 5 Done Done Sent false
27022 7999 Acked Write 1 Done Done Sent false
27023 8000 Acked Read 5 Done Done Sent false
27024 8001 Acked Write 2 Done Done Sent false
27025 8002 Acked Write 9 Done Done Sent false
27026 8003 Acked Write 5 Done Done Sent false
27027 8004 Acked Write 8 Done Done Sent false
27028 8005 Acked Write 6 Done Done Sent false
27029 8006 Acked Read 6 Done Done Sent false
27030 8007 Acked Write 10 Done Done Sent false
27031 8008 Acked Read 4 Done Done Sent false
27032 8009 Acked Flush 0 Done Done Sent false
27033 8010 Acked Read 5 Done Done Sent false
27034 8011 Acked Write 4 Done Done Sent false
27035 8012 Acked Read 3 Done Done Sent false
27036 8013 Acked Flush 0 Done Done Sent false
27037 8014 Acked Read 6 Done Done Sent false
27038 8015 Acked Read 9 Done Done Sent false
27039 8016 Acked Read 7 Done Done Sent false
27040 8017 Acked Flush 0 Done Done Sent false
27041 8018 Acked Read 3 Done Done Sent false
27042 8019 Acked Write 8 Done Done Sent false
27043 8020 Acked Write 8 Done Done Sent false
27044 8021 Acked Flush 0 Done Done Sent false
27045 8022 Acked Read 1 Done Done Sent false
27046 8023 Acked Flush 0 Done Done Sent false
27047 8024 Acked Write 2 Done Done Sent false
27048 8025 Acked Write 9 Done Done Sent false
27049 8026 Acked Write 10 Done Done Sent false
27050 8027 Acked Read 3 Done Done Sent false
27051 8028 Acked Flush 0 Done Done Sent false
27052 8029 Acked Write 9 Done Done Sent false
27053 8030 Acked Read 2 Done Done Sent false
27054 8031 Acked Read 7 Done Done Sent false
27055 8032 Acked Read 5 Done Done Sent false
27056 8033 Acked Write 3 Done Done Sent false
27057 8034 Acked Write 3 Done Done Sent false
27058 8035 Acked Read 8 Done Done Sent false
27059 8036 Acked Write 2 Done Done Sent false
27060 8037 Acked Read 8 Done Done Sent false
27061 8038 Acked Read 1 Done Done Sent false
27062 8039 Acked Flush 0 Done Done Sent false
27063 8040 Acked Write 6 Done Done Sent false
27064 8041 Acked Write 4 Done Done Sent false
27065 8042 Acked Write 6 Done Done Sent false
27066 8043 Acked Read 3 Done Done Sent false
27067 8044 Acked Write 10 Done Done Sent false
27068 8045 Acked Read 8 Done Done Sent false
27069 8046 Acked Write 7 Done Done Sent false
27070 8047 Acked Read 7 Done Done Sent false
27071 8048 Acked Flush 0 Done Done Sent false
27072 8049 Acked Write 2 Done Done Sent false
27073 8050 Acked Read 7 Done Done Sent false
27074 8051 Acked Write 3 Done Done Sent false
27075 8052 Acked Write 10 Done Done Sent false
27076 8053 Acked Read 9 Done Done Sent false
27077 8054 Acked Read 4 Done Done Sent false
27078 8055 Acked Read 10 Done Done Sent false
27079 8056 Acked Write 9 Done Done Sent false
27080 8057 Acked Write 4 Done Done Sent false
27081 8058 Acked Flush 0 Done Done Sent false
27082 8059 Acked Read 6 Done Done Sent false
27083 8060 Acked Write 8 Done Done Sent false
27084 8061 Acked Write 2 Done Done Sent false
27085 8062 Acked Read 7 Done Done Sent false
27086 8063 Acked Read 10 Done Done Sent false
27087 8064 Acked Read 6 Done Done Sent false
27088 8065 Acked Read 10 Done Done Sent false
27089 8066 Acked Read 10 Done Done Sent false
27090 8067 Acked Write 1 Done Done Sent false
27091 8068 Acked Write 9 Done Done Sent false
27092 8069 Acked Read 9 Done Done Sent false
27093 8070 Acked Write 2 Done Done Sent false
27094 8071 Acked Read 6 Done Done Sent false
27095 8072 Acked Read 5 Done Done Sent false
27096 8073 Acked Read 4 Done Done Sent false
27097 8074 Acked Read 9 Done Done Sent false
27098 8075 Acked Read 7 Done Done Sent false
27099 8076 Acked Read 6 Done Done Sent false
27100 8077 Acked Read 9 Done Done Sent false
27101 8078 Acked Read 7 Done Done Sent false
27102 8079 Acked Write 7 Done Done Sent false
27103 8080 Acked Read 2 Done Done Sent false
27104 8081 Acked Read 1 Done Done Sent false
27105 8082 Acked Read 5 Done Done Sent false
27106 8083 Acked Write 2 Done Done Sent false
27107 8084 Acked Read 7 Done Done Sent false
27108 8085 Acked Write 1 Done Done Sent false
27109 8086 Acked Write 7 Done Done Sent false
27110 8087 Acked Read 7 Done Done Sent false
27111 8088 Acked Read 9 Done Done Sent false
27112 8089 Acked Read 5 Done Done Sent false
27113 8090 Acked Write 2 Done Done Sent false
27114 8091 Acked Flush 0 Done Done Sent false
27115 8092 Acked Flush 0 Done Done Sent false
27116 8093 Acked Write 8 Done Done Sent false
27117 8094 Acked Read 10 Done Done Sent false
27118 8095 Acked Read 5 Done Done Sent false
27119 8096 Acked Read 2 Done Done Sent false
27120 8097 Acked Flush 0 Done Done Sent false
27121 8098 Acked Read 10 Done Done Sent false
27122 8099 Acked Read 7 Done Done Sent false
27123 8100 Acked Flush 0 Done Done Sent false
27124 8101 Acked Write 3 Done Done Sent false
27125 8102 Acked Read 3 Done Done Sent false
27126 8103 Acked Write 3 Done Done Sent false
27127 8104 Acked Write 3 Done Done Sent false
27128 8105 Acked Write 9 Done Done Sent false
27129 8106 Acked Read 8 Done Done Sent false
27130 8107 Acked Flush 0 Done Done Sent false
27131 8108 Acked Read 8 Done Done Sent false
27132 8109 Acked Flush 0 Done Done Sent false
27133 8110 Acked Read 2 Done Done Sent false
27134 8111 Acked Read 2 Done Done Sent false
27135 8112 Acked Read 7 Done Done Sent false
27136 8113 Acked Write 4 Done Done Sent false
27137 8114 Acked Read 3 Done Done Sent false
27138 8115 Acked Read 2 Done Done Sent false
27139 8116 Acked Write 3 Done Done Sent false
27140 8117 Acked Write 3 Done Done Sent false
27141 8118 Acked Read 6 Done Done Sent false
27142 8119 Acked Read 4 Done Done Sent false
27143 8120 Acked Read 10 Done Done Sent false
27144 8121 Acked Write 4 Done Done Sent false
27145 8122 Acked Read 4 Done Done Sent false
27146 8123 Acked Write 3 Done Done Sent false
27147 8124 Acked Flush 0 Done Done Sent false
27148 8125 Acked Write 5 Done Done Sent false
27149 8126 Acked Read 1 Done Done Sent false
27150 8127 Acked Write 5 Done Done Sent false
27151 8128 Acked Write 4 Done Done Sent false
27152 8129 Acked Read 10 Done Done Sent false
27153 8130 Acked Flush 0 Done Done Sent false
27154 8131 Acked Read 3 Done Done Sent false
27155 8132 Acked Read 10 Done Done Sent false
27156 8133 Acked Write 2 Done Done Sent false
27157 8134 Acked Read 10 Done Done Sent false
27158 8135 Acked Read 7 Done Done Sent false
27159 8136 Acked Write 6 Done Done Sent false
27160 8137 Acked Write 7 Done Done Sent false
27161 8138 Acked Flush 0 Done Done Sent false
27162 8139 Acked Read 1 Done Done Sent false
27163 8140 Acked Read 4 Done Done Sent false
27164 8141 Acked Write 6 Done Done Sent false
27165 8142 Acked Read 7 Done Done Sent false
27166 8143 Acked Write 7 Done Done Sent false
27167 8144 Acked Read 10 Done Done Sent false
27168 8145 Acked Read 6 Done Done Sent false
27169 8146 Acked Read 6 Done Done Sent false
27170 8147 Acked Write 5 Done Done Sent false
27171 8148 Acked Write 10 Done Done Sent false
27172 8149 Acked Read 4 Done Done Sent false
27173 8150 Acked Read 1 Done Done Sent false
27174 8151 Acked Read 7 Done Done Sent false
27175 8152 Acked Read 9 Done Done Sent false
27176 8153 Acked Read 4 Done Done Sent false
27177 8154 Acked Write 7 Done Done Sent false
27178 8155 Acked Write 9 Done Done Sent false
27179 8156 Acked Read 7 Done Done Sent false
27180 8157 Acked Read 1 Done Done Sent false
27181 8158 Acked Read 3 Done Done Sent false
27182 8159 Acked Read 7 Done Done Sent false
27183 8160 Acked Write 4 Done Done Sent false
27184 8161 Acked Flush 0 Done Done Sent false
27185 8162 Acked Read 3 Done Done Sent false
27186 8163 Acked Read 8 Done Done Sent false
27187 8164 Acked Read 5 Done Done Sent false
27188 8165 Acked Read 4 Done Done Sent false
27189 8166 Acked Write 10 Done Done Sent false
27190 8167 Acked Read 6 Done Done Sent false
27191 8168 Acked Write 8 Done Done Sent false
27192 8169 Acked Write 7 Done Done Sent false
27193 8170 Acked Write 1 Done Done Sent false
27194 8171 Acked Write 5 Done Done Sent false
27195 8172 Acked Read 3 Done Done Sent false
27196 8173 Acked Read 3 Done Done Sent false
27197 8174 Acked Flush 0 Done Done Sent false
27198 8175 Acked Read 6 Done Done Sent false
27199 8176 Acked Read 6 Done Done Sent false
27200 8177 Acked Read 7 Done Done Sent false
27201 8178 Acked Flush 0 Done Done Sent false
27202 8179 Acked Flush 0 Done Done Sent false
27203 8180 Acked Write 10 Done Done Sent false
27204 8181 Acked Write 6 Done Done Sent false
27205 8182 Acked Read 9 Done Done Sent false
27206 8183 Acked Read 2 Done Done Sent false
27207 8184 Acked Write 5 Done Done Sent false
27208 8185 Acked Read 10 Done Done Sent false
27209 8186 Acked Read 7 Done Done Sent false
27210 8187 Acked Read 4 Done Done Sent false
27211 8188 Acked Write 6 Done Done Sent false
27212 8189 Acked Read 8 Done Done Sent false
27213 8190 Acked Write 9 Done Done Sent false
27214 8191 Acked Read 9 Done Done Sent false
27215 8192 Acked Read 9 Done Done Sent false
27216 8193 Acked Write 7 Done Done Sent false
27217 8194 Acked Read 3 Done Done Sent false
27218 8195 Acked Write 9 Done Done Sent false
27219 8196 Acked Write 7 Done Done Sent false
27220 8197 Acked Write 9 Done Done Sent false
27221 8198 Acked Read 5 Done Done Sent false
27222 8199 Acked Read 2 Done Done Sent false
27223 8200 Acked Write 10 Done Done Sent false
27224 8201 Acked Flush 0 Done Done Sent false
27225 8202 Acked Write 3 Done Done Sent false
27226 8203 Acked Read 2 Done Done Sent false
27227 8204 Acked Flush 0 Done Done Sent false
27228 8205 Acked Read 1 Done Done Sent false
27229 8206 Acked Write 7 Done Done Sent false
27230 8207 Acked Write 6 Done Done Sent false
27231 8208 Acked Write 10 Done Done Sent false
27232 8209 Acked Read 1 Done Done Sent false
27233 8210 Acked Read 10 Done Done Sent false
27234 8211 Acked Read 6 Done Done Sent false
27235 8212 Acked Read 5 Done Done Sent false
27236 8213 Acked Flush 0 Done Done Sent false
27237 8214 Acked Read 8 Done Done Sent false
27238 8215 Acked Read 5 Done Done Sent false
27239 8216 Acked Read 6 Done Done Sent false
27240 8217 Acked Write 5 Done Done Sent false
27241 8218 Acked Read 1 Done Done Sent false
27242 8219 Acked Read 4 Done Done Sent false
27243 8220 Acked Read 2 Done Done Sent false
27244 8221 Acked Write 2 Done Done Sent false
27245 8222 Acked Read 6 Done Done Sent false
27246 8223 Acked Read 4 Done Done Sent false
27247 8224 Acked Write 3 Done Done Sent false
27248 8225 Acked Write 5 Done Done Sent false
27249 8226 Acked Read 9 Done Done Sent false
27250 8227 Acked Write 5 Done Done Sent false
27251 8228 Acked Read 3 Done Done Sent false
27252 8229 Acked Write 6 Done Done Sent false
27253 8230 Acked Read 6 Done Done Sent false
27254 8231 Acked Write 4 Done Done Sent false
27255 8232 Acked Write 5 Done Done Sent false
27256 8233 Acked Write 4 Done Done Sent false
27257 8234 Acked Write 5 Done Done Sent false
27258 8235 Acked Write 9 Done Done Sent false
27259 8236 Acked Write 5 Done Done Sent false
27260 8237 Acked Write 6 Done Done Sent false
27261 8238 Acked Read 5 Done Done Sent false
27262 8239 Acked Write 5 Done Done Sent false
27263 8240 Acked Read 3 Done Done Sent false
27264 8241 Acked Write 9 Done Done Sent false
27265 8242 Acked Write 3 Done Done Sent false
27266 8243 Acked Write 10 Done Done Sent false
27267 8244 Acked Write 5 Done Done Sent false
27268 8245 Acked Write 9 Done Done Sent false
27269 8246 Acked Flush 0 Done Done Sent false
27270 STATES DS:0 DS:1 DS:2 TOTAL
27271 Sent 0 0 302 302
27272 Done 302 302 0 604
27273 Skipped 0 0 0 0
27274 Error 0 0 0 0
27275 Last Flush: 8246 8246 7944
27276 Downstairs last five completed: 7944 7943 7942 7941 7940
27277 Upstairs last five completed: 8246 8245 8244 8243 8242
27278 CLIENT: Up:0 ds:302 act:2
272792025-01-15T05:24:42.545ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
272802025-01-15T05:24:42.545ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
272812025-01-15T05:24:42.546ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
272822025-01-15T05:24:42.546ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
272832025-01-15T05:24:42.546ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27284 [8830] got start action
27285 [8830] I am going to restart
27286 Starting downstairs at port 8830
27287 Make output file at "/tmp/dsc/downstairs-8830.txt"
27288 [8830][2] reports Starting
27289 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1226)
27290 [8830][2] reports Running
27291 ----------------------------------------------------------------
27292 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27293 States: Active Active Offline
27294 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27295 7945 Acked Read 3 Done Done Sent false
27296 7946 Acked Read 3 Done Done Sent false
27297 7947 Acked Write 2 Done Done Sent false
27298 7948 Acked Write 2 Done Done Sent false
27299 7949 Acked Read 8 Done Done Sent false
27300 7950 Acked Write 4 Done Done Sent false
27301 7951 Acked Read 9 Done Done Sent false
27302 7952 Acked Read 6 Done Done Sent false
27303 7953 Acked Flush 0 Done Done Sent false
27304 7954 Acked Flush 0 Done Done Sent false
27305 7955 Acked Read 6 Done Done Sent false
27306 7956 Acked Write 6 Done Done Sent false
27307 7957 Acked Write 1 Done Done Sent false
27308 7958 Acked Read 3 Done Done Sent false
27309 7959 Acked Write 8 Done Done Sent false
27310 7960 Acked Read 2 Done Done Sent false
27311 7961 Acked Flush 0 Done Done Sent false
27312 7962 Acked Read 1 Done Done Sent false
27313 7963 Acked Write 2 Done Done Sent false
27314 7964 Acked Flush 0 Done Done Sent false
27315 7965 Acked Write 9 Done Done Sent false
27316 7966 Acked Read 2 Done Done Sent false
27317 7967 Acked Read 4 Done Done Sent false
27318 7968 Acked Write 3 Done Done Sent false
27319 7969 Acked Read 7 Done Done Sent false
27320 7970 Acked Write 5 Done Done Sent false
27321 7971 Acked Read 2 Done Done Sent false
27322 7972 Acked Read 3 Done Done Sent false
27323 7973 Acked Read 1 Done Done Sent false
27324 7974 Acked Write 10 Done Done Sent false
27325 7975 Acked Read 8 Done Done Sent false
27326 7976 Acked Write 10 Done Done Sent false
27327 7977 Acked Write 2 Done Done Sent false
27328 7978 Acked Write 2 Done Done Sent false
27329 7979 Acked Read 6 Done Done Sent false
27330 7980 Acked Read 3 Done Done Sent false
27331 7981 Acked Read 4 Done Done Sent false
27332 7982 Acked Write 2 Done Done Sent false
27333 7983 Acked Read 9 Done Done Sent false
27334 7984 Acked Write 5 Done Done Sent false
27335 7985 Acked Read 8 Done Done Sent false
27336 7986 Acked Flush 0 Done Done Sent false
27337 7987 Acked Flush 0 Done Done Sent false
27338 7988 Acked Write 10 Done Done Sent false
27339 7989 Acked Read 2 Done Done Sent false
27340 7990 Acked Read 3 Done Done Sent false
27341 7991 Acked Write 2 Done Done Sent false
27342 7992 Acked Write 1 Done Done Sent false
27343 7993 Acked Read 1 Done Done Sent false
27344 7994 Acked Read 8 Done Done Sent false
27345 7995 Acked Read 6 Done Done Sent false
27346 7996 Acked Write 3 Done Done Sent false
27347 7997 Acked Read 3 Done Done Sent false
27348 7998 Acked Read 5 Done Done Sent false
27349 7999 Acked Write 1 Done Done Sent false
27350 8000 Acked Read 5 Done Done Sent false
27351 8001 Acked Write 2 Done Done Sent false
27352 8002 Acked Write 9 Done Done Sent false
27353 8003 Acked Write 5 Done Done Sent false
27354 8004 Acked Write 8 Done Done Sent false
27355 8005 Acked Write 6 Done Done Sent false
27356 8006 Acked Read 6 Done Done Sent false
27357 8007 Acked Write 10 Done Done Sent false
27358 8008 Acked Read 4 Done Done Sent false
27359 8009 Acked Flush 0 Done Done Sent false
27360 8010 Acked Read 5 Done Done Sent false
27361 8011 Acked Write 4 Done Done Sent false
27362 8012 Acked Read 3 Done Done Sent false
27363 8013 Acked Flush 0 Done Done Sent false
27364 8014 Acked Read 6 Done Done Sent false
27365 8015 Acked Read 9 Done Done Sent false
27366 8016 Acked Read 7 Done Done Sent false
27367 8017 Acked Flush 0 Done Done Sent false
27368 8018 Acked Read 3 Done Done Sent false
27369 8019 Acked Write 8 Done Done Sent false
27370 8020 Acked Write 8 Done Done Sent false
27371 8021 Acked Flush 0 Done Done Sent false
27372 8022 Acked Read 1 Done Done Sent false
27373 8023 Acked Flush 0 Done Done Sent false
27374 8024 Acked Write 2 Done Done Sent false
27375 8025 Acked Write 9 Done Done Sent false
27376 8026 Acked Write 10 Done Done Sent false
27377 8027 Acked Read 3 Done Done Sent false
27378 8028 Acked Flush 0 Done Done Sent false
27379 8029 Acked Write 9 Done Done Sent false
27380 8030 Acked Read 2 Done Done Sent false
27381 8031 Acked Read 7 Done Done Sent false
27382 8032 Acked Read 5 Done Done Sent false
27383 8033 Acked Write 3 Done Done Sent false
27384 8034 Acked Write 3 Done Done Sent false
27385 8035 Acked Read 8 Done Done Sent false
27386 8036 Acked Write 2 Done Done Sent false
27387 8037 Acked Read 8 Done Done Sent false
27388 8038 Acked Read 1 Done Done Sent false
27389 8039 Acked Flush 0 Done Done Sent false
27390 8040 Acked Write 6 Done Done Sent false
27391 8041 Acked Write 4 Done Done Sent false
27392 8042 Acked Write 6 Done Done Sent false
27393 8043 Acked Read 3 Done Done Sent false
27394 8044 Acked Write 10 Done Done Sent false
27395 8045 Acked Read 8 Done Done Sent false
27396 8046 Acked Write 7 Done Done Sent false
27397 8047 Acked Read 7 Done Done Sent false
27398 8048 Acked Flush 0 Done Done Sent false
27399 8049 Acked Write 2 Done Done Sent false
27400 8050 Acked Read 7 Done Done Sent false
27401 8051 Acked Write 3 Done Done Sent false
27402 8052 Acked Write 10 Done Done Sent false
27403 8053 Acked Read 9 Done Done Sent false
27404 8054 Acked Read 4 Done Done Sent false
27405 8055 Acked Read 10 Done Done Sent false
27406 8056 Acked Write 9 Done Done Sent false
27407 8057 Acked Write 4 Done Done Sent false
27408 8058 Acked Flush 0 Done Done Sent false
27409 8059 Acked Read 6 Done Done Sent false
27410 8060 Acked Write 8 Done Done Sent false
27411 8061 Acked Write 2 Done Done Sent false
27412 8062 Acked Read 7 Done Done Sent false
27413 8063 Acked Read 10 Done Done Sent false
27414 8064 Acked Read 6 Done Done Sent false
27415 8065 Acked Read 10 Done Done Sent false
27416 8066 Acked Read 10 Done Done Sent false
27417 8067 Acked Write 1 Done Done Sent false
27418 8068 Acked Write 9 Done Done Sent false
27419 8069 Acked Read 9 Done Done Sent false
27420 8070 Acked Write 2 Done Done Sent false
27421 8071 Acked Read 6 Done Done Sent false
27422 8072 Acked Read 5 Done Done Sent false
27423 8073 Acked Read 4 Done Done Sent false
27424 8074 Acked Read 9 Done Done Sent false
27425 8075 Acked Read 7 Done Done Sent false
27426 8076 Acked Read 6 Done Done Sent false
27427 8077 Acked Read 9 Done Done Sent false
27428 8078 Acked Read 7 Done Done Sent false
27429 8079 Acked Write 7 Done Done Sent false
27430 8080 Acked Read 2 Done Done Sent false
27431 8081 Acked Read 1 Done Done Sent false
27432 8082 Acked Read 5 Done Done Sent false
27433 8083 Acked Write 2 Done Done Sent false
27434 8084 Acked Read 7 Done Done Sent false
27435 8085 Acked Write 1 Done Done Sent false
27436 8086 Acked Write 7 Done Done Sent false
27437 8087 Acked Read 7 Done Done Sent false
27438 8088 Acked Read 9 Done Done Sent false
27439 8089 Acked Read 5 Done Done Sent false
27440 8090 Acked Write 2 Done Done Sent false
27441 8091 Acked Flush 0 Done Done Sent false
27442 8092 Acked Flush 0 Done Done Sent false
27443 8093 Acked Write 8 Done Done Sent false
27444 8094 Acked Read 10 Done Done Sent false
27445 8095 Acked Read 5 Done Done Sent false
27446 8096 Acked Read 2 Done Done Sent false
27447 8097 Acked Flush 0 Done Done Sent false
27448 8098 Acked Read 10 Done Done Sent false
27449 8099 Acked Read 7 Done Done Sent false
27450 8100 Acked Flush 0 Done Done Sent false
27451 8101 Acked Write 3 Done Done Sent false
27452 8102 Acked Read 3 Done Done Sent false
27453 8103 Acked Write 3 Done Done Sent false
27454 8104 Acked Write 3 Done Done Sent false
27455 8105 Acked Write 9 Done Done Sent false
27456 8106 Acked Read 8 Done Done Sent false
27457 8107 Acked Flush 0 Done Done Sent false
27458 8108 Acked Read 8 Done Done Sent false
27459 8109 Acked Flush 0 Done Done Sent false
27460 8110 Acked Read 2 Done Done Sent false
27461 8111 Acked Read 2 Done Done Sent false
27462 8112 Acked Read 7 Done Done Sent false
27463 8113 Acked Write 4 Done Done Sent false
27464 8114 Acked Read 3 Done Done Sent false
27465 8115 Acked Read 2 Done Done Sent false
27466 8116 Acked Write 3 Done Done Sent false
27467 8117 Acked Write 3 Done Done Sent false
27468 8118 Acked Read 6 Done Done Sent false
27469 8119 Acked Read 4 Done Done Sent false
27470 8120 Acked Read 10 Done Done Sent false
27471 8121 Acked Write 4 Done Done Sent false
27472 8122 Acked Read 4 Done Done Sent false
27473 8123 Acked Write 3 Done Done Sent false
27474 8124 Acked Flush 0 Done Done Sent false
27475 8125 Acked Write 5 Done Done Sent false
27476 8126 Acked Read 1 Done Done Sent false
27477 8127 Acked Write 5 Done Done Sent false
27478 8128 Acked Write 4 Done Done Sent false
27479 8129 Acked Read 10 Done Done Sent false
27480 8130 Acked Flush 0 Done Done Sent false
27481 8131 Acked Read 3 Done Done Sent false
27482 8132 Acked Read 10 Done Done Sent false
27483 8133 Acked Write 2 Done Done Sent false
27484 8134 Acked Read 10 Done Done Sent false
27485 8135 Acked Read 7 Done Done Sent false
27486 8136 Acked Write 6 Done Done Sent false
27487 8137 Acked Write 7 Done Done Sent false
27488 8138 Acked Flush 0 Done Done Sent false
27489 8139 Acked Read 1 Done Done Sent false
27490 8140 Acked Read 4 Done Done Sent false
27491 8141 Acked Write 6 Done Done Sent false
27492 8142 Acked Read 7 Done Done Sent false
27493 8143 Acked Write 7 Done Done Sent false
27494 8144 Acked Read 10 Done Done Sent false
27495 8145 Acked Read 6 Done Done Sent false
27496 8146 Acked Read 6 Done Done Sent false
27497 8147 Acked Write 5 Done Done Sent false
27498 8148 Acked Write 10 Done Done Sent false
27499 8149 Acked Read 4 Done Done Sent false
27500 8150 Acked Read 1 Done Done Sent false
27501 8151 Acked Read 7 Done Done Sent false
27502 8152 Acked Read 9 Done Done Sent false
27503 8153 Acked Read 4 Done Done Sent false
27504 8154 Acked Write 7 Done Done Sent false
27505 8155 Acked Write 9 Done Done Sent false
27506 8156 Acked Read 7 Done Done Sent false
27507 8157 Acked Read 1 Done Done Sent false
27508 8158 Acked Read 3 Done Done Sent false
27509 8159 Acked Read 7 Done Done Sent false
27510 8160 Acked Write 4 Done Done Sent false
27511 8161 Acked Flush 0 Done Done Sent false
27512 8162 Acked Read 3 Done Done Sent false
27513 8163 Acked Read 8 Done Done Sent false
27514 8164 Acked Read 5 Done Done Sent false
27515 8165 Acked Read 4 Done Done Sent false
27516 8166 Acked Write 10 Done Done Sent false
27517 8167 Acked Read 6 Done Done Sent false
27518 8168 Acked Write 8 Done Done Sent false
27519 8169 Acked Write 7 Done Done Sent false
27520 8170 Acked Write 1 Done Done Sent false
27521 8171 Acked Write 5 Done Done Sent false
27522 8172 Acked Read 3 Done Done Sent false
27523 8173 Acked Read 3 Done Done Sent false
27524 8174 Acked Flush 0 Done Done Sent false
27525 8175 Acked Read 6 Done Done Sent false
27526 8176 Acked Read 6 Done Done Sent false
27527 8177 Acked Read 7 Done Done Sent false
27528 8178 Acked Flush 0 Done Done Sent false
27529 8179 Acked Flush 0 Done Done Sent false
27530 8180 Acked Write 10 Done Done Sent false
27531 8181 Acked Write 6 Done Done Sent false
27532 8182 Acked Read 9 Done Done Sent false
27533 8183 Acked Read 2 Done Done Sent false
27534 8184 Acked Write 5 Done Done Sent false
27535 8185 Acked Read 10 Done Done Sent false
27536 8186 Acked Read 7 Done Done Sent false
27537 8187 Acked Read 4 Done Done Sent false
27538 8188 Acked Write 6 Done Done Sent false
27539 8189 Acked Read 8 Done Done Sent false
27540 8190 Acked Write 9 Done Done Sent false
27541 8191 Acked Read 9 Done Done Sent false
27542 8192 Acked Read 9 Done Done Sent false
27543 8193 Acked Write 7 Done Done Sent false
27544 8194 Acked Read 3 Done Done Sent false
27545 8195 Acked Write 9 Done Done Sent false
27546 8196 Acked Write 7 Done Done Sent false
27547 8197 Acked Write 9 Done Done Sent false
27548 8198 Acked Read 5 Done Done Sent false
27549 8199 Acked Read 2 Done Done Sent false
27550 8200 Acked Write 10 Done Done Sent false
27551 8201 Acked Flush 0 Done Done Sent false
27552 8202 Acked Write 3 Done Done Sent false
27553 8203 Acked Read 2 Done Done Sent false
27554 8204 Acked Flush 0 Done Done Sent false
27555 8205 Acked Read 1 Done Done Sent false
27556 8206 Acked Write 7 Done Done Sent false
27557 8207 Acked Write 6 Done Done Sent false
27558 8208 Acked Write 10 Done Done Sent false
27559 8209 Acked Read 1 Done Done Sent false
27560 8210 Acked Read 10 Done Done Sent false
27561 8211 Acked Read 6 Done Done Sent false
27562 8212 Acked Read 5 Done Done Sent false
27563 8213 Acked Flush 0 Done Done Sent false
27564 8214 Acked Read 8 Done Done Sent false
27565 8215 Acked Read 5 Done Done Sent false
27566 8216 Acked Read 6 Done Done Sent false
27567 8217 Acked Write 5 Done Done Sent false
27568 8218 Acked Read 1 Done Done Sent false
27569 8219 Acked Read 4 Done Done Sent false
27570 8220 Acked Read 2 Done Done Sent false
27571 8221 Acked Write 2 Done Done Sent false
27572 8222 Acked Read 6 Done Done Sent false
27573 8223 Acked Read 4 Done Done Sent false
27574 8224 Acked Write 3 Done Done Sent false
27575 8225 Acked Write 5 Done Done Sent false
27576 8226 Acked Read 9 Done Done Sent false
27577 8227 Acked Write 5 Done Done Sent false
27578 8228 Acked Read 3 Done Done Sent false
27579 8229 Acked Write 6 Done Done Sent false
27580 8230 Acked Read 6 Done Done Sent false
27581 8231 Acked Write 4 Done Done Sent false
27582 8232 Acked Write 5 Done Done Sent false
27583 8233 Acked Write 4 Done Done Sent false
27584 8234 Acked Write 5 Done Done Sent false
27585 8235 Acked Write 9 Done Done Sent false
27586 8236 Acked Write 5 Done Done Sent false
27587 8237 Acked Write 6 Done Done Sent false
27588 8238 Acked Read 5 Done Done Sent false
27589 8239 Acked Write 5 Done Done Sent false
27590 8240 Acked Read 3 Done Done Sent false
27591 8241 Acked Write 9 Done Done Sent false
27592 8242 Acked Write 3 Done Done Sent false
27593 8243 Acked Write 10 Done Done Sent false
27594 8244 Acked Write 5 Done Done Sent false
27595 8245 Acked Write 9 Done Done Sent false
27596 8246 Acked Flush 0 Done Done Sent false
27597 STATES DS:0 DS:1 DS:2 TOTAL
27598 Sent 0 0 302 302
27599 Done 302 302 0 604
27600 Skipped 0 0 0 0
27601 Error 0 0 0 0
27602 Last Flush: 8246 8246 7944
27603 Downstairs last five completed: 7944 7943 7942 7941 7940
27604 Upstairs last five completed: 8246 8245 8244 8243 8242
27605 CLIENT: Up:0 ds:302 act:2
27606 ----------------------------------------------------------------
27607 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27608 States: Active Active Offline
27609 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27610 7945 Acked Read 3 Done Done Sent false
27611 7946 Acked Read 3 Done Done Sent false
27612 7947 Acked Write 2 Done Done Sent false
27613 7948 Acked Write 2 Done Done Sent false
27614 7949 Acked Read 8 Done Done Sent false
27615 7950 Acked Write 4 Done Done Sent false
27616 7951 Acked Read 9 Done Done Sent false
27617 7952 Acked Read 6 Done Done Sent false
27618 7953 Acked Flush 0 Done Done Sent false
27619 7954 Acked Flush 0 Done Done Sent false
27620 7955 Acked Read 6 Done Done Sent false
27621 7956 Acked Write 6 Done Done Sent false
27622 7957 Acked Write 1 Done Done Sent false
27623 7958 Acked Read 3 Done Done Sent false
27624 7959 Acked Write 8 Done Done Sent false
27625 7960 Acked Read 2 Done Done Sent false
27626 7961 Acked Flush 0 Done Done Sent false
27627 7962 Acked Read 1 Done Done Sent false
27628 7963 Acked Write 2 Done Done Sent false
27629 7964 Acked Flush 0 Done Done Sent false
27630 7965 Acked Write 9 Done Done Sent false
27631 7966 Acked Read 2 Done Done Sent false
27632 7967 Acked Read 4 Done Done Sent false
27633 7968 Acked Write 3 Done Done Sent false
27634 7969 Acked Read 7 Done Done Sent false
27635 7970 Acked Write 5 Done Done Sent false
27636 7971 Acked Read 2 Done Done Sent false
27637 7972 Acked Read 3 Done Done Sent false
27638 7973 Acked Read 1 Done Done Sent false
27639 7974 Acked Write 10 Done Done Sent false
27640 7975 Acked Read 8 Done Done Sent false
27641 7976 Acked Write 10 Done Done Sent false
27642 7977 Acked Write 2 Done Done Sent false
27643 7978 Acked Write 2 Done Done Sent false
27644 7979 Acked Read 6 Done Done Sent false
27645 7980 Acked Read 3 Done Done Sent false
27646 7981 Acked Read 4 Done Done Sent false
27647 7982 Acked Write 2 Done Done Sent false
27648 7983 Acked Read 9 Done Done Sent false
27649 7984 Acked Write 5 Done Done Sent false
27650 7985 Acked Read 8 Done Done Sent false
27651 7986 Acked Flush 0 Done Done Sent false
27652 7987 Acked Flush 0 Done Done Sent false
27653 7988 Acked Write 10 Done Done Sent false
27654 7989 Acked Read 2 Done Done Sent false
27655 7990 Acked Read 3 Done Done Sent false
27656 7991 Acked Write 2 Done Done Sent false
27657 7992 Acked Write 1 Done Done Sent false
27658 7993 Acked Read 1 Done Done Sent false
27659 7994 Acked Read 8 Done Done Sent false
27660 7995 Acked Read 6 Done Done Sent false
27661 7996 Acked Write 3 Done Done Sent false
27662 7997 Acked Read 3 Done Done Sent false
27663 7998 Acked Read 5 Done Done Sent false
27664 7999 Acked Write 1 Done Done Sent false
27665 8000 Acked Read 5 Done Done Sent false
27666 8001 Acked Write 2 Done Done Sent false
27667 8002 Acked Write 9 Done Done Sent false
27668 8003 Acked Write 5 Done Done Sent false
27669 8004 Acked Write 8 Done Done Sent false
27670 8005 Acked Write 6 Done Done Sent false
27671 8006 Acked Read 6 Done Done Sent false
27672 8007 Acked Write 10 Done Done Sent false
27673 8008 Acked Read 4 Done Done Sent false
27674 8009 Acked Flush 0 Done Done Sent false
27675 8010 Acked Read 5 Done Done Sent false
27676 8011 Acked Write 4 Done Done Sent false
27677 8012 Acked Read 3 Done Done Sent false
27678 8013 Acked Flush 0 Done Done Sent false
27679 8014 Acked Read 6 Done Done Sent false
27680 8015 Acked Read 9 Done Done Sent false
27681 8016 Acked Read 7 Done Done Sent false
27682 8017 Acked Flush 0 Done Done Sent false
27683 8018 Acked Read 3 Done Done Sent false
27684 8019 Acked Write 8 Done Done Sent false
27685 8020 Acked Write 8 Done Done Sent false
27686 8021 Acked Flush 0 Done Done Sent false
27687 8022 Acked Read 1 Done Done Sent false
27688 8023 Acked Flush 0 Done Done Sent false
27689 8024 Acked Write 2 Done Done Sent false
27690 8025 Acked Write 9 Done Done Sent false
27691 8026 Acked Write 10 Done Done Sent false
27692 8027 Acked Read 3 Done Done Sent false
27693 8028 Acked Flush 0 Done Done Sent false
27694 8029 Acked Write 9 Done Done Sent false
27695 8030 Acked Read 2 Done Done Sent false
27696 8031 Acked Read 7 Done Done Sent false
27697 8032 Acked Read 5 Done Done Sent false
27698 8033 Acked Write 3 Done Done Sent false
27699 8034 Acked Write 3 Done Done Sent false
27700 8035 Acked Read 8 Done Done Sent false
27701 8036 Acked Write 2 Done Done Sent false
27702 8037 Acked Read 8 Done Done Sent false
27703 8038 Acked Read 1 Done Done Sent false
27704 8039 Acked Flush 0 Done Done Sent false
27705 8040 Acked Write 6 Done Done Sent false
27706 8041 Acked Write 4 Done Done Sent false
27707 8042 Acked Write 6 Done Done Sent false
27708 8043 Acked Read 3 Done Done Sent false
27709 8044 Acked Write 10 Done Done Sent false
27710 8045 Acked Read 8 Done Done Sent false
27711 8046 Acked Write 7 Done Done Sent false
27712 8047 Acked Read 7 Done Done Sent false
27713 8048 Acked Flush 0 Done Done Sent false
27714 8049 Acked Write 2 Done Done Sent false
27715 8050 Acked Read 7 Done Done Sent false
27716 8051 Acked Write 3 Done Done Sent false
27717 8052 Acked Write 10 Done Done Sent false
27718 8053 Acked Read 9 Done Done Sent false
27719 8054 Acked Read 4 Done Done Sent false
27720 8055 Acked Read 10 Done Done Sent false
27721 8056 Acked Write 9 Done Done Sent false
27722 8057 Acked Write 4 Done Done Sent false
27723 8058 Acked Flush 0 Done Done Sent false
27724 8059 Acked Read 6 Done Done Sent false
27725 8060 Acked Write 8 Done Done Sent false
27726 8061 Acked Write 2 Done Done Sent false
27727 8062 Acked Read 7 Done Done Sent false
27728 8063 Acked Read 10 Done Done Sent false
27729 8064 Acked Read 6 Done Done Sent false
27730 8065 Acked Read 10 Done Done Sent false
27731 8066 Acked Read 10 Done Done Sent false
27732 8067 Acked Write 1 Done Done Sent false
27733 8068 Acked Write 9 Done Done Sent false
27734 8069 Acked Read 9 Done Done Sent false
27735 8070 Acked Write 2 Done Done Sent false
27736 8071 Acked Read 6 Done Done Sent false
27737 8072 Acked Read 5 Done Done Sent false
27738 8073 Acked Read 4 Done Done Sent false
27739 8074 Acked Read 9 Done Done Sent false
27740 8075 Acked Read 7 Done Done Sent false
27741 8076 Acked Read 6 Done Done Sent false
27742 8077 Acked Read 9 Done Done Sent false
27743 8078 Acked Read 7 Done Done Sent false
27744 8079 Acked Write 7 Done Done Sent false
27745 8080 Acked Read 2 Done Done Sent false
27746 8081 Acked Read 1 Done Done Sent false
27747 8082 Acked Read 5 Done Done Sent false
27748 8083 Acked Write 2 Done Done Sent false
27749 8084 Acked Read 7 Done Done Sent false
27750 8085 Acked Write 1 Done Done Sent false
27751 8086 Acked Write 7 Done Done Sent false
27752 8087 Acked Read 7 Done Done Sent false
27753 8088 Acked Read 9 Done Done Sent false
27754 8089 Acked Read 5 Done Done Sent false
27755 8090 Acked Write 2 Done Done Sent false
27756 8091 Acked Flush 0 Done Done Sent false
27757 8092 Acked Flush 0 Done Done Sent false
27758 8093 Acked Write 8 Done Done Sent false
27759 8094 Acked Read 10 Done Done Sent false
27760 8095 Acked Read 5 Done Done Sent false
27761 8096 Acked Read 2 Done Done Sent false
27762 8097 Acked Flush 0 Done Done Sent false
27763 8098 Acked Read 10 Done Done Sent false
27764 8099 Acked Read 7 Done Done Sent false
27765 8100 Acked Flush 0 Done Done Sent false
27766 8101 Acked Write 3 Done Done Sent false
27767 8102 Acked Read 3 Done Done Sent false
27768 8103 Acked Write 3 Done Done Sent false
27769 8104 Acked Write 3 Done Done Sent false
27770 8105 Acked Write 9 Done Done Sent false
27771 8106 Acked Read 8 Done Done Sent false
27772 8107 Acked Flush 0 Done Done Sent false
27773 8108 Acked Read 8 Done Done Sent false
27774 8109 Acked Flush 0 Done Done Sent false
27775 8110 Acked Read 2 Done Done Sent false
27776 8111 Acked Read 2 Done Done Sent false
27777 8112 Acked Read 7 Done Done Sent false
27778 8113 Acked Write 4 Done Done Sent false
27779 8114 Acked Read 3 Done Done Sent false
27780 8115 Acked Read 2 Done Done Sent false
27781 8116 Acked Write 3 Done Done Sent false
27782 8117 Acked Write 3 Done Done Sent false
27783 8118 Acked Read 6 Done Done Sent false
27784 8119 Acked Read 4 Done Done Sent false
27785 8120 Acked Read 10 Done Done Sent false
27786 8121 Acked Write 4 Done Done Sent false
27787 8122 Acked Read 4 Done Done Sent false
27788 8123 Acked Write 3 Done Done Sent false
27789 8124 Acked Flush 0 Done Done Sent false
27790 8125 Acked Write 5 Done Done Sent false
27791 8126 Acked Read 1 Done Done Sent false
27792 8127 Acked Write 5 Done Done Sent false
27793 8128 Acked Write 4 Done Done Sent false
27794 8129 Acked Read 10 Done Done Sent false
27795 8130 Acked Flush 0 Done Done Sent false
27796 8131 Acked Read 3 Done Done Sent false
27797 8132 Acked Read 10 Done Done Sent false
27798 8133 Acked Write 2 Done Done Sent false
27799 8134 Acked Read 10 Done Done Sent false
27800 8135 Acked Read 7 Done Done Sent false
27801 8136 Acked Write 6 Done Done Sent false
27802 8137 Acked Write 7 Done Done Sent false
27803 8138 Acked Flush 0 Done Done Sent false
27804 8139 Acked Read 1 Done Done Sent false
27805 8140 Acked Read 4 Done Done Sent false
27806 8141 Acked Write 6 Done Done Sent false
27807 8142 Acked Read 7 Done Done Sent false
27808 8143 Acked Write 7 Done Done Sent false
27809 8144 Acked Read 10 Done Done Sent false
27810 8145 Acked Read 6 Done Done Sent false
27811 8146 Acked Read 6 Done Done Sent false
27812 8147 Acked Write 5 Done Done Sent false
27813 8148 Acked Write 10 Done Done Sent false
27814 8149 Acked Read 4 Done Done Sent false
27815 8150 Acked Read 1 Done Done Sent false
27816 8151 Acked Read 7 Done Done Sent false
27817 8152 Acked Read 9 Done Done Sent false
27818 8153 Acked Read 4 Done Done Sent false
27819 8154 Acked Write 7 Done Done Sent false
27820 8155 Acked Write 9 Done Done Sent false
27821 8156 Acked Read 7 Done Done Sent false
27822 8157 Acked Read 1 Done Done Sent false
27823 8158 Acked Read 3 Done Done Sent false
27824 8159 Acked Read 7 Done Done Sent false
27825 8160 Acked Write 4 Done Done Sent false
27826 8161 Acked Flush 0 Done Done Sent false
27827 8162 Acked Read 3 Done Done Sent false
27828 8163 Acked Read 8 Done Done Sent false
27829 8164 Acked Read 5 Done Done Sent false
27830 8165 Acked Read 4 Done Done Sent false
27831 8166 Acked Write 10 Done Done Sent false
27832 8167 Acked Read 6 Done Done Sent false
27833 8168 Acked Write 8 Done Done Sent false
27834 8169 Acked Write 7 Done Done Sent false
27835 8170 Acked Write 1 Done Done Sent false
27836 8171 Acked Write 5 Done Done Sent false
27837 8172 Acked Read 3 Done Done Sent false
27838 8173 Acked Read 3 Done Done Sent false
27839 8174 Acked Flush 0 Done Done Sent false
27840 8175 Acked Read 6 Done Done Sent false
27841 8176 Acked Read 6 Done Done Sent false
27842 8177 Acked Read 7 Done Done Sent false
27843 8178 Acked Flush 0 Done Done Sent false
27844 8179 Acked Flush 0 Done Done Sent false
27845 8180 Acked Write 10 Done Done Sent false
27846 8181 Acked Write 6 Done Done Sent false
27847 8182 Acked Read 9 Done Done Sent false
27848 8183 Acked Read 2 Done Done Sent false
27849 8184 Acked Write 5 Done Done Sent false
27850 8185 Acked Read 10 Done Done Sent false
27851 8186 Acked Read 7 Done Done Sent false
27852 8187 Acked Read 4 Done Done Sent false
27853 8188 Acked Write 6 Done Done Sent false
27854 8189 Acked Read 8 Done Done Sent false
27855 8190 Acked Write 9 Done Done Sent false
27856 8191 Acked Read 9 Done Done Sent false
27857 8192 Acked Read 9 Done Done Sent false
27858 8193 Acked Write 7 Done Done Sent false
27859 8194 Acked Read 3 Done Done Sent false
27860 8195 Acked Write 9 Done Done Sent false
27861 8196 Acked Write 7 Done Done Sent false
27862 8197 Acked Write 9 Done Done Sent false
27863 8198 Acked Read 5 Done Done Sent false
27864 8199 Acked Read 2 Done Done Sent false
27865 8200 Acked Write 10 Done Done Sent false
27866 8201 Acked Flush 0 Done Done Sent false
27867 8202 Acked Write 3 Done Done Sent false
27868 8203 Acked Read 2 Done Done Sent false
27869 8204 Acked Flush 0 Done Done Sent false
27870 8205 Acked Read 1 Done Done Sent false
27871 8206 Acked Write 7 Done Done Sent false
27872 8207 Acked Write 6 Done Done Sent false
27873 8208 Acked Write 10 Done Done Sent false
27874 8209 Acked Read 1 Done Done Sent false
27875 8210 Acked Read 10 Done Done Sent false
27876 8211 Acked Read 6 Done Done Sent false
27877 8212 Acked Read 5 Done Done Sent false
27878 8213 Acked Flush 0 Done Done Sent false
27879 8214 Acked Read 8 Done Done Sent false
27880 8215 Acked Read 5 Done Done Sent false
27881 8216 Acked Read 6 Done Done Sent false
27882 8217 Acked Write 5 Done Done Sent false
27883 8218 Acked Read 1 Done Done Sent false
27884 8219 Acked Read 4 Done Done Sent false
27885 8220 Acked Read 2 Done Done Sent false
27886 8221 Acked Write 2 Done Done Sent false
27887 8222 Acked Read 6 Done Done Sent false
27888 8223 Acked Read 4 Done Done Sent false
27889 8224 Acked Write 3 Done Done Sent false
27890 8225 Acked Write 5 Done Done Sent false
27891 8226 Acked Read 9 Done Done Sent false
27892 8227 Acked Write 5 Done Done Sent false
27893 8228 Acked Read 3 Done Done Sent false
27894 8229 Acked Write 6 Done Done Sent false
27895 8230 Acked Read 6 Done Done Sent false
27896 8231 Acked Write 4 Done Done Sent false
27897 8232 Acked Write 5 Done Done Sent false
27898 8233 Acked Write 4 Done Done Sent false
27899 8234 Acked Write 5 Done Done Sent false
27900 8235 Acked Write 9 Done Done Sent false
27901 8236 Acked Write 5 Done Done Sent false
27902 8237 Acked Write 6 Done Done Sent false
27903 8238 Acked Read 5 Done Done Sent false
27904 8239 Acked Write 5 Done Done Sent false
27905 8240 Acked Read 3 Done Done Sent false
27906 8241 Acked Write 9 Done Done Sent false
27907 8242 Acked Write 3 Done Done Sent false
27908 8243 Acked Write 10 Done Done Sent false
27909 8244 Acked Write 5 Done Done Sent false
27910 8245 Acked Write 9 Done Done Sent false
27911 8246 Acked Flush 0 Done Done Sent false
27912 STATES DS:0 DS:1 DS:2 TOTAL
27913 Sent 0 0 302 302
27914 Done 302 302 0 604
27915 Skipped 0 0 0 0
27916 Error 0 0 0 0
27917 Last Flush: 8246 8246 7944
27918 Downstairs last five completed: 7944 7943 7942 7941 7940
27919 Upstairs last five completed: 8246 8245 8244 8243 8242
27920 CLIENT: Up:0 ds:302 act:2
279212025-01-15T05:24:52.547ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279222025-01-15T05:24:52.548ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60984) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279232025-01-15T05:24:52.548ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279242025-01-15T05:24:52.549ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279252025-01-15T05:24:52.549ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279262025-01-15T05:24:52.549ZINFOcrucible: send last flush ID to this DS: 7944 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279272025-01-15T05:24:52.549ZINFOcrucible: Replied this last flush ID: 7944 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279282025-01-15T05:24:52.549ZINFOcrucible: [2] client re-new 302 jobs since flush 7944 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279292025-01-15T05:24:52.550ZINFOcrucible: [2] Replayed 302 jobs since flush: 7944 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27930 ----------------------------------------------------------------
27931 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27932 Downstairs last five completed: 8246 8245 8244 8243 8242
27933 Upstairs last five completed: 8246 8245 8244 8243 8242
27934 CLIENT: Up:0 ds:0 act:3
27935 Replay: All jobs finished
27936 Main task has work to do, go find it
27937 got dsc Stop(1)
27938 stop 1
27939 Jan 15 05:24:52.787[8820] Got stop action so:false kr:false
27940 INFO request completed, latency_us: 291, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 5440160d-5938-4836-8591-206397946031, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
27941 Jan 15 05:24:52.788 INFO accepted connection, remote_addr: 127.0.0.1:35558, local_addr: 127.0.0.1:9998
279422025-01-15T05:24:52.788ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279432025-01-15T05:24:52.788ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279442025-01-15T05:24:52.788ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
279452025-01-15T05:24:52.788ZINFOcrucible: client task is exiting = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
27946 Jan 15 05:24:52.788 INFO request completed, latency_us: 146, response_code: 200, uri: /state/cid/1, method: GET, req_id: d93fcba8-1735-4759-93a1-fd872c222e22, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
27947 [8820] Exited with: ExitStatus(unix_wait_status(9))
27948 [8820][1] reports Exit
27949 Jan 15 05:24:56.789 INFO request completed, latency_us: 178, response_code: 200, uri: /state/cid/1, method: GET, req_id: c0e4f790-a026-4a96-88d3-a4aa3b8f7989, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
27950 001/300 Read block 1308 len 8192
27951 002/300 Write block 2658 len 24576 data: 7 6 5 4 4 3
27952 003/300 Flush
27953 004/300 Read block 3478 len 36864
27954 005/300 Read block 3546 len 4096
27955 006/300 Write block 4682 len 40960 data: 5 4 4 6 6 6 4 3 3 3
27956 007/300 Read block 954 len 16384
27957 008/300 Read block 3613 len 4096
27958 009/300 Flush
27959 010/300 Write block 4093 len 40960 data: 4 4 4 4 5 5 3 3 4 4
27960 011/300 Write block 974 len 28672 data: 4 5 4 3 4 4 4
27961 012/300 Read block 3978 len 40960
27962 013/300 Write block 2792 len 12288 data: 3 3 4
27963 014/300 Write block 3460 len 24576 data: 4 3 4 4 4 4
27964 015/300 Write block 3528 len 32768 data: 6 5 5 4 5 6 5 6
27965 016/300 Write block 1700 len 24576 data: 6 6 5 5 6 5
27966 017/300 Read block 2127 len 36864
27967 018/300 Write block 4237 len 24576 data: 5 5 6 9 12 10
27968 019/300 Read block 30 len 16384
27969 020/300 Read block 2620 len 16384
27970 021/300 Write block 2953 len 28672 data: 5 4 4 4 5 5 5
27971 022/300 Flush
27972 023/300 Write block 4909 len 24576 data: 8 7 8 8 7 6
27973 024/300 Write block 422 len 36864 data: 2 3 3 3 3 4 4 5 4
27974 025/300 Read block 4997 len 4096
27975 026/300 Read block 4403 len 28672
27976 027/300 Read block 1224 len 12288
27977 028/300 Write block 2591 len 24576 data: 7 8 8 7 7 4
27978 029/300 Write block 246 len 8192 data: 5 6
27979 030/300 Read block 2298 len 32768
27980 031/300 Read block 1998 len 24576
27981 032/300 Write block 1492 len 12288 data: 4 5 4
27982 033/300 Write block 2387 len 24576 data: 5 4 4 4 6 7
27983 034/300 Read block 3838 len 28672
27984 035/300 Read block 2565 len 20480
27985 036/300 Write block 1678 len 4096 data: 8
27986 037/300 Read block 1399 len 24576
27987 038/300 Write block 3430 len 36864 data: 5 5 5 5 4 3 4 3 5
27988 039/300 Read block 3645 len 4096
27989 040/300 Write block 3520 len 24576 data: 3 3 4 5 5 5
27990 041/300 Write block 3582 len 20480 data: 4 5 5 3 3
27991 042/300 Write block 2822 len 36864 data: 8 7 7 7 7 6 5 4 7
27992 043/300 Read block 4454 len 28672
27993 044/300 Write block 3282 len 28672 data: 6 5 4 4 4 4 4
27994 045/300 Write block 572 len 32768 data: 3 3 5 5 6 7 7 7
27995 046/300 Read block 2457 len 36864
27996 047/300 Read block 4459 len 32768
27997 048/300 Write block 3058 len 40960 data: 6 4 5 4 4 4 5 5 3 4
27998 049/300 Read block 1133 len 24576
27999 050/300 Write block 2513 len 28672 data: 6 4 3 3 3 3 3
28000 051/300 Read block 4500 len 4096
28001 052/300 Write block 2661 len 12288 data: 5 5 4
28002 053/300 Read block 4806 len 16384
28003 054/300 Flush
28004 055/300 Read block 4164 len 12288
28005 056/300 Read block 1470 len 4096
28006 057/300 Write block 2432 len 12288 data: 9 11 10
28007 058/300 Write block 4025 len 40960 data: 5 5 6 5 8 6 4 4 4 3
28008 059/300 Write block 2686 len 8192 data: 4 4
28009 060/300 Write block 2208 len 8192 data: 7 6
28010 061/300 Write block 265 len 8192 data: 8 8
28011 062/300 Flush
28012 063/300 Flush
28013 064/300 Write block 652 len 40960 data: 6 5 3 3 4 4 4 4 6 6
28014 065/300 Write block 2332 len 28672 data: 4 6 6 5 4 5 6
28015 066/300 Write block 4378 len 16384 data: 5 5 6 7
28016 067/300 Read block 776 len 40960
28017 068/300 Read block 1133 len 8192
28018 069/300 Write block 4941 len 4096 data: 6
28019 070/300 Read block 3333 len 16384
28020 071/300 Flush
28021 072/300 Write block 3926 len 8192 data: 5 3
28022 073/300 Flush
28023 074/300 Write block 1125 len 32768 data: 5 6 6 6 5 5 5 4
28024 075/300 Read block 470 len 36864
28025 076/300 Read block 1208 len 16384
28026 077/300 Write block 3456 len 20480 data: 7 7 5 4 5
28027 078/300 Read block 2623 len 12288
28028 079/300 Write block 1538 len 4096 data: 6
28029 080/300 Write block 515 len 4096 data: 7
28030 081/300 Write block 2354 len 40960 data: 6 6 6 7 5 5 4 4 6 6
28031 082/300 Flush
28032 083/300 Read block 4021 len 8192
28033 084/300 Read block 4005 len 16384
28034 085/300 Read block 4357 len 16384
28035 086/300 Write block 4455 len 40960 data: 7 6 6 5 4 3 3 2 2 2
28036 087/300 Read block 1112 len 16384
28037 088/300 Write block 3267 len 24576 data: 5 5 7 6 5 6
28038 089/300 Flush
28039 090/300 Read block 539 len 28672
28040 091/300 Read block 2914 len 40960
28041 092/300 Write block 1090 len 24576 data: 4 5 6 4 4 4
28042 093/300 Read block 1756 len 24576
28043 094/300 Write block 1080 len 32768 data: 7 6 4 4 6 7 6 7
28044 095/300 Read block 2594 len 16384
28045 096/300 Read block 3925 len 8192
28046 097/300 Read block 1326 len 40960
28047 098/300 Flush
28048 099/300 Read block 1736 len 32768
28049 100/300 Read block 1074 len 40960
28050 101/300 Read block 3490 len 8192
28051 102/300 Read block 391 len 20480
28052 103/300 Write block 298 len 4096 data: 4
28053 104/300 Read block 1921 len 16384
28054 105/300 Write block 3085 len 40960 data: 3 4 6 7 9 10 10 10 10 8
28055 106/300 Write block 871 len 4096 data: 7
28056 107/300 Read block 2912 len 24576
28057 108/300 Read block 3640 len 32768
28058 109/300 Flush
28059 110/300 Read block 2008 len 24576
28060 111/300 Write block 2948 len 20480 data: 6 5 6 6 7
28061 112/300 Read block 1961 len 20480
28062 113/300 Write block 2510 len 36864 data: 5 5 5 7 5 4 4 4 4
28063 114/300 Write block 3582 len 12288 data: 5 6 6
28064 115/300 Flush
28065 116/300 Read block 2926 len 4096
28066 117/300 Write block 409 len 32768 data: 7 6 5 6 6 5 4 4
28067 118/300 Read block 974 len 36864
28068 119/300 Read block 2532 len 32768
28069 120/300 Read block 4762 len 40960
28070 121/300 Write block 2984 len 12288 data: 8 8 8
28071 122/300 Flush
28072 123/300 Read block 4490 len 12288
28073 124/300 Read block 39 len 16384
28074 125/300 Read block 177 len 36864
28075 126/300 Read block 1613 len 28672
28076 127/300 Read block 2731 len 20480
28077 128/300 Read block 1637 len 36864
28078 129/300 Write block 1747 len 28672 data: 3 4 5 4 4 4 5
28079 130/300 Read block 3004 len 36864
28080 131/300 Read block 3142 len 12288
28081 132/300 Read block 4203 len 12288
28082 133/300 Read block 755 len 36864
28083 134/300 Write block 2919 len 16384 data: 9 9 10 9
28084 135/300 Read block 1868 len 32768
28085 136/300 Write block 3973 len 12288 data: 4 5 6
28086 137/300 Write block 202 len 32768 data: 7 7 7 6 8 5 4 5
28087 138/300 Read block 2766 len 28672
28088 139/300 Write block 1555 len 4096 data: 4
28089 140/300 Read block 3548 len 24576
28090 141/300 Write block 1492 len 8192 data: 5 6
28091 142/300 Write block 960 len 24576 data: 3 5 6 6 7 7
28092 143/300 Read block 643 len 8192
28093 144/300 Write block 2420 len 16384 data: 6 4 5 4
28094 145/300 Read block 2475 len 4096
28095 146/300 Read block 2612 len 12288
28096 147/300 Write block 3891 len 24576 data: 4 6 7 8 7 7
28097 148/300 Write block 3933 len 36864 data: 5 6 5 5 7 8 9 7 7
28098 149/300 Write block 926 len 32768 data: 5 4 4 4 4 4 4 6
28099 150/300 Read block 599 len 12288
28100 151/300 Flush
28101 152/300 Flush
28102 153/300 Write block 4390 len 24576 data: 6 5 4 4 4 2
28103 154/300 Write block 1028 len 36864 data: 4 4 4 3 3 3 4 6 7
28104 155/300 Read block 4486 len 12288
28105 156/300 Write block 2529 len 4096 data: 6
28106 157/300 Read block 2170 len 40960
28107 158/300 Read block 2724 len 28672
28108 159/300 Write block 4330 len 36864 data: 7 8 7 5 5 4 4 4 4
28109 160/300 Write block 3507 len 32768 data: 3 4 4 4 6 3 3 3
28110 161/300 Read block 4908 len 8192
28111 162/300 Read block 3145 len 16384
28112 163/300 Read block 4909 len 28672
28113 164/300 Read block 987 len 24576
28114 165/300 Read block 3920 len 8192
28115 166/300 Write block 1569 len 8192 data: 5 6
28116 167/300 Write block 1288 len 4096 data: 5
28117 168/300 Read block 3908 len 32768
28118 169/300 Read block 281 len 16384
28119 170/300 Read block 3800 len 12288
28120 171/300 Read block 765 len 12288
28121 172/300 Write block 718 len 28672 data: 5 5 6 5 5 4 4
28122 173/300 Read block 4086 len 24576
28123 174/300 Write block 745 len 12288 data: 6 6 5
28124 175/300 Read block 643 len 20480
28125 176/300 Read block 3155 len 16384
28126 177/300 Write block 2714 len 4096 data: 8
28127 178/300 Read block 4511 len 36864
28128 179/300 Write block 3990 len 40960 data: 4 5 5 4 3 3 3 3 3 4
28129 180/300 Read block 3763 len 40960
28130 181/300 Write block 3407 len 20480 data: 5 4 4 4 7
28131 182/300 Write block 4017 len 40960 data: 6 6 7 9 9 9 9 7 6 6
28132 183/300 Read block 3444 len 16384
28133 184/300 Read block 1162 len 4096
28134 185/300 Flush
28135 186/300 Write block 1925 len 32768 data: 6 8 8 7 7 7 5 6
28136 187/300 Read block 2308 len 24576
28137 188/300 Flush
28138 189/300 Read block 3504 len 12288
28139 190/300 Flush
28140 191/300 Write block 4586 len 20480 data: 5 5 5 5 4
28141 192/300 Read block 624 len 12288
28142 193/300 Read block 4340 len 28672
28143 194/300 Read block 147 len 4096
28144 195/300 Read block 2425 len 40960
28145 196/300 Write block 4461 len 36864 data: 4 3 3 3 3 3 3 5 3
28146 197/300 Read block 2603 len 36864
28147 198/300 Write block 3011 len 12288 data: 3 3 5
28148 199/300 Write block 949 len 16384 data: 5 6 6 5
28149 200/300 Read block 4977 len 24576
28150 201/300 Write block 1305 len 8192 data: 8 9
28151 202/300 Read block 4683 len 36864
28152 203/300 Write block 3339 len 20480 data: 7 6 5 4 3
28153 204/300 Write block 2826 len 12288 data: 8 7 6
28154 205/300 Read block 19 len 8192
28155 206/300 Write block 1150 len 32768 data: 7 5 5 7 8 6 6 7
28156 207/300 Read block 2503 len 36864
28157 208/300 Read block 4479 len 24576
28158 209/300 Write block 731 len 20480 data: 6 6 6 4 4
28159 210/300 Write block 2639 len 16384 data: 6 7 6 7
28160 211/300 Write block 2014 len 36864 data: 7 6 8 8 8 8 7 7 9
28161 212/300 Read block 2459 len 24576
28162 213/300 Flush
28163 214/300 Write block 3818 len 36864 data: 6 6 7 7 6 6 5 3 4
28164 215/300 Flush
28165 216/300 Read block 691 len 16384
28166 217/300 Read block 1097 len 20480
28167 218/300 Read block 2434 len 16384
28168 219/300 Read block 2486 len 16384
28169 220/300 Read block 1401 len 20480
28170 221/300 Write block 3532 len 24576 data: 6 7 6 7 5 6
28171 222/300 Read block 3052 len 36864
28172 223/300 Write block 894 len 20480 data: 8 7 5 5 3
28173 224/300 Read block 2362 len 8192
28174 225/300 Read block 2823 len 20480
28175 226/300 Read block 2281 len 32768
28176 227/300 Write block 1217 len 40960 data: 7 7 5 5 5 4 3 3 4 4
28177 228/300 Read block 3839 len 8192
28178 229/300 Write block 1032 len 20480 data: 4 4 5 7 8
28179 230/300 Write block 3789 len 8192 data: 4 5
28180 231/300 Write block 3575 len 20480 data: 3 4 4 5 4
28181 232/300 Write block 894 len 40960 data: 9 8 6 6 4 4 3 4 4 4
28182 233/300 Read block 1641 len 20480
28183 234/300 Read block 1324 len 36864
28184 235/300 Read block 1429 len 16384
28185 236/300 Write block 2778 len 8192 data: 6 6
28186 237/300 Write block 2637 len 12288 data: 7 6 7
28187 238/300 Write block 3936 len 20480 data: 6 8 9 10 8
28188 239/300 Read block 4709 len 28672
28189 240/300 Write block 1902 len 16384 data: 6 5 6 5
28190 241/300 Read block 3181 len 20480
28191 242/300 Write block 4126 len 28672 data: 7 7 6 5 6 8 9
28192 243/300 Read block 2516 len 32768
28193 244/300 Write block 1515 len 24576 data: 7 7 5 4 3 3
28194 245/300 Write block 1157 len 32768 data: 8 5 5 5 5 6 6 7
28195 246/300 Read block 2441 len 28672
28196 247/300 Read block 1533 len 4096
28197 248/300 Write block 3382 len 36864 data: 9 8 8 8 8 4 5 4 4
28198 249/300 Read block 3894 len 8192
28199 250/300 Read block 8 len 32768
28200 251/300 Read block 3793 len 40960
28201 252/300 Read block 3714 len 12288
28202 253/300 Read block 1982 len 4096
28203 254/300 Read block 1652 len 20480
28204 255/300 Write block 2941 len 24576 data: 5 4 4 4 6 7
28205 256/300 Write block 3608 len 20480 data: 5 5 6 7 7
28206 257/300 Read block 1226 len 36864
28207 258/300 Write block 1334 len 8192 data: 4 5
28208 259/300 Write block 3985 len 8192 data: 3 4
28209 260/300 Write block 2318 len 32768 data: 6 5 4 4 3 3 3 4
28210 261/300 Flush
28211 262/300 Read block 4943 len 28672
28212 263/300 Read block 3037 len 12288
28213 264/300 Write block 2580 len 8192 data: 3 3
28214 265/300 Write block 350 len 32768 data: 3 4 4 5 5 6 6 7
28215 266/300 Read block 568 len 12288
28216 267/300 Read block 858 len 16384
28217 268/300 Read block 1131 len 4096
28218 269/300 Read block 2941 len 12288
28219 270/300 Read block 1773 len 8192
28220 271/300 Write block 713 len 12288 data: 3 4 4
28221 272/300 Read block 661 len 28672
28222 273/300 Write block 2181 len 4096 data: 7
28223 274/300 Write block 1111 len 8192 data: 5 5
28224 275/300 Read block 4319 len 24576
28225 276/300 Read block 261 len 8192
28226 277/300 Write block 1380 len 12288 data: 5 5 3
28227 278/300 Write block 4413 len 40960 data: 5 6 6 6 6 5 4 5 7 8
28228 279/300 Read block 1874 len 12288
28229 280/300 Flush
28230 281/300 Read block 1585 len 4096
28231 282/300 Flush
28232 283/300 Write block 2502 len 4096 data: 6
28233 284/300 Read block 4920 len 8192
28234 285/300 Flush
28235 286/300 Write block 2027 len 12288 data: 6 7 9
28236 287/300 Flush
28237 288/300 Read block 2443 len 16384
28238 289/300 Read block 296 len 4096
28239 290/300 Read block 3940 len 8192
28240 291/300 Write block 1511 len 12288 data: 3 4 5
28241 292/300 Write block 2716 len 20480 data: 8 8 6 6 4
28242 293/300 Read block 4988 len 24576
28243 294/300 Read block 233 len 16384
28244 295/300 Write block 1447 len 8192 data: 4 4
28245 296/300 Write block 3405 len 40960 data: 4 5 6 5 5 5 8 5 6 6
28246 297/300 Read block 688 len 32768
28247 298/300 Read block 3322 len 24576
28248 299/300 Flush
28249 300/300 Write block 4591 len 32768 data: 3 3 2 2 2 2 3 3
28250 301/300 Flush
28251 Main task has work to do, go find it
28252 got dsc Start(1)
28253 start 1
28254 Jan 15 05:24:57.019 INFO request completed, latency_us: 182, response_code: 204, uri: /start/cid/1, method: POST, req_id: e1e9cd2f-419a-4d4d-81ae-0c0b08d0f4f4, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
28255 Replay: started 1, returned:Ok(())
28256 ----------------------------------------------------------------
28257 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28258 States: Active Offline Active
28259 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28260 8247 Acked Read 2 Done Sent Done false
28261 8248 Acked Write 6 Done Sent Done false
28262 8249 Acked Flush 0 Done Sent Done false
28263 8250 Acked Read 9 Done Sent Done false
28264 8251 Acked Read 1 Done Sent Done false
28265 8252 Acked Write 10 Done Sent Done false
28266 8253 Acked Read 4 Done Sent Done false
28267 8254 Acked Read 1 Done Sent Done false
28268 8255 Acked Flush 0 Done Sent Done false
28269 8256 Acked Write 10 Done Sent Done false
28270 8257 Acked Write 7 Done Sent Done false
28271 8258 Acked Read 10 Done Sent Done false
28272 8259 Acked Write 3 Done Sent Done false
28273 8260 Acked Write 6 Done Sent Done false
28274 8261 Acked Write 8 Done Sent Done false
28275 8262 Acked Write 6 Done Sent Done false
28276 8263 Acked Read 9 Done Sent Done false
28277 8264 Acked Write 6 Done Sent Done false
28278 8265 Acked Read 4 Done Sent Done false
28279 8266 Acked Read 4 Done Sent Done false
28280 8267 Acked Write 7 Done Sent Done false
28281 8268 Acked Flush 0 Done Sent Done false
28282 8269 Acked Write 6 Done Sent Done false
28283 8270 Acked Write 9 Done Sent Done false
28284 8271 Acked Read 1 Done Sent Done false
28285 8272 Acked Read 7 Done Sent Done false
28286 8273 Acked Read 3 Done Sent Done false
28287 8274 Acked Write 6 Done Sent Done false
28288 8275 Acked Write 2 Done Sent Done false
28289 8276 Acked Read 8 Done Sent Done false
28290 8277 Acked Read 6 Done Sent Done false
28291 8278 Acked Write 3 Done Sent Done false
28292 8279 Acked Write 6 Done Sent Done false
28293 8280 Acked Read 7 Done Sent Done false
28294 8281 Acked Read 5 Done Sent Done false
28295 8282 Acked Write 1 Done Sent Done false
28296 8283 Acked Read 6 Done Sent Done false
28297 8284 Acked Write 9 Done Sent Done false
28298 8285 Acked Read 1 Done Sent Done false
28299 8286 Acked Write 6 Done Sent Done false
28300 8287 Acked Write 5 Done Sent Done false
28301 8288 Acked Write 9 Done Sent Done false
28302 8289 Acked Read 7 Done Sent Done false
28303 8290 Acked Write 7 Done Sent Done false
28304 8291 Acked Write 8 Done Sent Done false
28305 8292 Acked Read 9 Done Sent Done false
28306 8293 Acked Read 8 Done Sent Done false
28307 8294 Acked Write 10 Done Sent Done false
28308 8295 Acked Read 6 Done Sent Done false
28309 8296 Acked Write 7 Done Sent Done false
28310 8297 Acked Read 1 Done Sent Done false
28311 8298 Acked Write 3 Done Sent Done false
28312 8299 Acked Read 4 Done Sent Done false
28313 8300 Acked Flush 0 Done Sent Done false
28314 8301 Acked Read 3 Done Sent Done false
28315 8302 Acked Read 1 Done Sent Done false
28316 8303 Acked Write 3 Done Sent Done false
28317 8304 Acked Write 10 Done Sent Done false
28318 8305 Acked Write 2 Done Sent Done false
28319 8306 Acked Write 2 Done Sent Done false
28320 8307 Acked Write 2 Done Sent Done false
28321 8308 Acked Flush 0 Done Sent Done false
28322 8309 Acked Flush 0 Done Sent Done false
28323 8310 Acked Write 10 Done Sent Done false
28324 8311 Acked Write 7 Done Sent Done false
28325 8312 Acked Write 4 Done Sent Done false
28326 8313 Acked Read 10 Done Sent Done false
28327 8314 Acked Read 2 Done Sent Done false
28328 8315 Acked Write 1 Done Sent Done false
28329 8316 Acked Read 4 Done Sent Done false
28330 8317 Acked Flush 0 Done Sent Done false
28331 8318 Acked Write 2 Done Sent Done false
28332 8319 Acked Flush 0 Done Sent Done false
28333 8320 Acked Write 8 Done Sent Done false
28334 8321 Acked Read 9 Done Sent Done false
28335 8322 Acked Read 4 Done Sent Done false
28336 8323 Acked Write 5 Done Sent Done false
28337 8324 Acked Read 3 Done Sent Done false
28338 8325 Acked Write 1 Done Sent Done false
28339 8326 Acked Write 1 Done Sent Done false
28340 8327 Acked Write 10 Done Sent Done false
28341 8328 Acked Flush 0 Done Sent Done false
28342 8329 Acked Read 2 Done Sent Done false
28343 8330 Acked Read 4 Done Sent Done false
28344 8331 Acked Read 4 Done Sent Done false
28345 8332 Acked Write 10 Done Sent Done false
28346 8333 Acked Read 4 Done Sent Done false
28347 8334 Acked Write 6 Done Sent Done false
28348 8335 Acked Flush 0 Done Sent Done false
28349 8336 Acked Read 7 Done Sent Done false
28350 8337 Acked Read 10 Done Sent Done false
28351 8338 Acked Write 6 Done Sent Done false
28352 8339 Acked Read 6 Done Sent Done false
28353 8340 Acked Write 8 Done Sent Done false
28354 8341 Acked Read 4 Done Sent Done false
28355 8342 Acked Read 2 Done Sent Done false
28356 8343 Acked Read 10 Done Sent Done false
28357 8344 Acked Flush 0 Done Sent Done false
28358 8345 Acked Read 8 Done Sent Done false
28359 8346 Acked Read 10 Done Sent Done false
28360 8347 Acked Read 2 Done Sent Done false
28361 8348 Acked Read 5 Done Sent Done false
28362 8349 Acked Write 1 Done Sent Done false
28363 8350 Acked Read 4 Done Sent Done false
28364 8351 Acked Write 10 Done Sent Done false
28365 8352 Acked Write 1 Done Sent Done false
28366 8353 Acked Read 6 Done Sent Done false
28367 8354 Acked Read 8 Done Sent Done false
28368 8355 Acked Flush 0 Done Sent Done false
28369 8356 Acked Read 6 Done Sent Done false
28370 8357 Acked Write 5 Done Sent Done false
28371 8358 Acked Read 5 Done Sent Done false
28372 8359 Acked Write 9 Done Sent Done false
28373 8360 Acked Write 3 Done Sent Done false
28374 8361 Acked Flush 0 Done Sent Done false
28375 8362 Acked Read 1 Done Sent Done false
28376 8363 Acked Write 8 Done Sent Done false
28377 8364 Acked Read 9 Done Sent Done false
28378 8365 Acked Read 8 Done Sent Done false
28379 8366 Acked Read 10 Done Sent Done false
28380 8367 Acked Write 3 Done Sent Done false
28381 8368 Acked Flush 0 Done Sent Done false
28382 8369 Acked Read 3 Done Sent Done false
28383 8370 Acked Read 4 Done Sent Done false
28384 8371 Acked Read 9 Done Sent Done false
28385 8372 Acked Read 7 Done Sent Done false
28386 8373 Acked Read 5 Done Sent Done false
28387 8374 Acked Read 9 Done Sent Done false
28388 8375 Acked Write 7 Done Sent Done false
28389 8376 Acked Read 9 Done Sent Done false
28390 8377 Acked Read 3 Done Sent Done false
28391 8378 Acked Read 3 Done Sent Done false
28392 8379 Acked Read 9 Done Sent Done false
28393 8380 Acked Write 4 Done Sent Done false
28394 8381 Acked Read 8 Done Sent Done false
28395 8382 Acked Write 3 Done Sent Done false
28396 8383 Acked Write 8 Done Sent Done false
28397 8384 Acked Read 7 Done Sent Done false
28398 8385 Acked Write 1 Done Sent Done false
28399 8386 Acked Read 6 Done Sent Done false
28400 8387 Acked Write 2 Done Sent Done false
28401 8388 Acked Write 6 Done Sent Done false
28402 8389 Acked Read 2 Done Sent Done false
28403 8390 Acked Write 4 Done Sent Done false
28404 8391 Acked Read 1 Done Sent Done false
28405 8392 Acked Read 3 Done Sent Done false
28406 8393 Acked Write 6 Done Sent Done false
28407 8394 Acked Write 9 Done Sent Done false
28408 8395 Acked Write 8 Done Sent Done false
28409 8396 Acked Read 3 Done Sent Done false
28410 8397 Acked Flush 0 Done Sent Done false
28411 8398 Acked Flush 0 Done Sent Done false
28412 8399 Acked Write 6 Done Sent Done false
28413 8400 Acked Write 9 Done Sent Done false
28414 8401 Acked Read 3 Done Sent Done false
28415 8402 Acked Write 1 Done Sent Done false
28416 8403 Acked Read 10 Done Sent Done false
28417 8404 Acked Read 7 Done Sent Done false
28418 8405 Acked Write 9 Done Sent Done false
28419 8406 Acked Write 8 Done Sent Done false
28420 8407 Acked Read 2 Done Sent Done false
28421 8408 Acked Read 4 Done Sent Done false
28422 8409 Acked Read 7 Done Sent Done false
28423 8410 Acked Read 6 Done Sent Done false
28424 8411 Acked Read 2 Done Sent Done false
28425 8412 Acked Write 2 Done Sent Done false
28426 8413 Acked Write 1 Done Sent Done false
28427 8414 Acked Read 8 Done Sent Done false
28428 8415 Acked Read 4 Done Sent Done false
28429 8416 Acked Read 3 Done Sent Done false
28430 8417 Acked Read 3 Done Sent Done false
28431 8418 Acked Write 7 Done Sent Done false
28432 8419 Acked Read 6 Done Sent Done false
28433 8420 Acked Write 3 Done Sent Done false
28434 8421 Acked Read 5 Done Sent Done false
28435 8422 Acked Read 4 Done Sent Done false
28436 8423 Acked Write 1 Done Sent Done false
28437 8424 Acked Read 9 Done Sent Done false
28438 8425 Acked Write 10 Done Sent Done false
28439 8426 Acked Read 10 Done Sent Done false
28440 8427 Acked Write 5 Done Sent Done false
28441 8428 Acked Write 10 Done Sent Done false
28442 8429 Acked Read 4 Done Sent Done false
28443 8430 Acked Read 1 Done Sent Done false
28444 8431 Acked Flush 0 Done Sent Done false
28445 8432 Acked Write 8 Done Sent Done false
28446 8433 Acked Read 6 Done Sent Done false
28447 8434 Acked Flush 0 Done Sent Done false
28448 8435 Acked Read 3 Done Sent Done false
28449 8436 Acked Flush 0 Done Sent Done false
28450 8437 Acked Write 5 Done Sent Done false
28451 8438 Acked Read 3 Done Sent Done false
28452 8439 Acked Read 7 Done Sent Done false
28453 8440 Acked Read 1 Done Sent Done false
28454 8441 Acked Read 10 Done Sent Done false
28455 8442 Acked Write 9 Done Sent Done false
28456 8443 Acked Read 9 Done Sent Done false
28457 8444 Acked Write 3 Done Sent Done false
28458 8445 Acked Write 4 Done Sent Done false
28459 8446 Acked Read 6 Done Sent Done false
28460 8447 Acked Write 2 Done Sent Done false
28461 8448 Acked Read 9 Done Sent Done false
28462 8449 Acked Write 5 Done Sent Done false
28463 8450 Acked Write 3 Done Sent Done false
28464 8451 Acked Read 2 Done Sent Done false
28465 8452 Acked Write 8 Done Sent Done false
28466 8453 Acked Read 9 Done Sent Done false
28467 8454 Acked Read 6 Done Sent Done false
28468 8455 Acked Write 5 Done Sent Done false
28469 8456 Acked Write 4 Done Sent Done false
28470 8457 Acked Write 9 Done Sent Done false
28471 8458 Acked Read 6 Done Sent Done false
28472 8459 Acked Flush 0 Done Sent Done false
28473 8460 Acked Write 9 Done Sent Done false
28474 8461 Acked Flush 0 Done Sent Done false
28475 8462 Acked Read 4 Done Sent Done false
28476 8463 Acked Read 5 Done Sent Done false
28477 8464 Acked Read 4 Done Sent Done false
28478 8465 Acked Read 4 Done Sent Done false
28479 8466 Acked Read 5 Done Sent Done false
28480 8467 Acked Write 6 Done Sent Done false
28481 8468 Acked Read 9 Done Sent Done false
28482 8469 Acked Write 5 Done Sent Done false
28483 8470 Acked Read 2 Done Sent Done false
28484 8471 Acked Read 5 Done Sent Done false
28485 8472 Acked Read 8 Done Sent Done false
28486 8473 Acked Write 10 Done Sent Done false
28487 8474 Acked Read 2 Done Sent Done false
28488 8475 Acked Write 5 Done Sent Done false
28489 8476 Acked Write 2 Done Sent Done false
28490 8477 Acked Write 5 Done Sent Done false
28491 8478 Acked Write 10 Done Sent Done false
28492 8479 Acked Read 5 Done Sent Done false
28493 8480 Acked Read 9 Done Sent Done false
28494 8481 Acked Read 4 Done Sent Done false
28495 8482 Acked Write 2 Done Sent Done false
28496 8483 Acked Write 3 Done Sent Done false
28497 8484 Acked Write 5 Done Sent Done false
28498 8485 Acked Read 7 Done Sent Done false
28499 8486 Acked Write 4 Done Sent Done false
28500 8487 Acked Read 5 Done Sent Done false
28501 8488 Acked Write 7 Done Sent Done false
28502 8489 Acked Read 8 Done Sent Done false
28503 8490 Acked Write 6 Done Sent Done false
28504 8491 Acked Write 8 Done Sent Done false
28505 8492 Acked Read 7 Done Sent Done false
28506 8493 Acked Read 1 Done Sent Done false
28507 8494 Acked Write 9 Done Sent Done false
28508 8495 Acked Read 2 Done Sent Done false
28509 8496 Acked Read 8 Done Sent Done false
28510 8497 Acked Read 10 Done Sent Done false
28511 8498 Acked Read 3 Done Sent Done false
28512 8499 Acked Read 1 Done Sent Done false
28513 8500 Acked Read 5 Done Sent Done false
28514 8501 Acked Write 6 Done Sent Done false
28515 8502 Acked Write 5 Done Sent Done false
28516 8503 Acked Read 9 Done Sent Done false
28517 8504 Acked Write 2 Done Sent Done false
28518 8505 Acked Write 2 Done Sent Done false
28519 8506 Acked Write 8 Done Sent Done false
28520 8507 Acked Flush 0 Done Sent Done false
28521 8508 Acked Read 7 Done Sent Done false
28522 8509 Acked Read 3 Done Sent Done false
28523 8510 Acked Write 2 Done Sent Done false
28524 8511 Acked Write 8 Done Sent Done false
28525 8512 Acked Read 3 Done Sent Done false
28526 8513 Acked Read 4 Done Sent Done false
28527 8514 Acked Read 1 Done Sent Done false
28528 8515 Acked Read 3 Done Sent Done false
28529 8516 Acked Read 2 Done Sent Done false
28530 8517 Acked Write 3 Done Sent Done false
28531 8518 Acked Read 7 Done Sent Done false
28532 8519 Acked Write 1 Done Sent Done false
28533 8520 Acked Write 2 Done Sent Done false
28534 8521 Acked Read 6 Done Sent Done false
28535 8522 Acked Read 2 Done Sent Done false
28536 8523 Acked Write 3 Done Sent Done false
28537 8524 Acked Write 10 Done Sent Done false
28538 8525 Acked Read 3 Done Sent Done false
28539 8526 Acked Flush 0 Done Sent Done false
28540 8527 Acked Read 1 Done Sent Done false
28541 8528 Acked Flush 0 Done Sent Done false
28542 8529 Acked Write 1 Done Sent Done false
28543 8530 Acked Read 2 Done Sent Done false
28544 8531 Acked Flush 0 Done Sent Done false
28545 8532 Acked Write 3 Done Sent Done false
28546 8533 Acked Flush 0 Done Sent Done false
28547 8534 Acked Read 4 Done Sent Done false
28548 8535 Acked Read 1 Done Sent Done false
28549 8536 Acked Read 2 Done Sent Done false
28550 8537 Acked Write 3 Done Sent Done false
28551 8538 Acked Write 5 Done Sent Done false
28552 8539 Acked Read 6 Done Sent Done false
28553 8540 Acked Read 4 Done Sent Done false
28554 8541 Acked Write 2 Done Sent Done false
28555 8542 Acked Write 10 Done Sent Done false
28556 8543 Acked Read 8 Done Sent Done false
28557 8544 Acked Read 6 Done Sent Done false
28558 8545 Acked Flush 0 Done Sent Done false
28559 8546 Acked Write 8 Done Sent Done false
28560 8547 Acked Flush 0 Done Sent Done false
28561 STATES DS:0 DS:1 DS:2 TOTAL
28562 Sent 0 301 0 301
28563 Done 301 0 301 602
28564 Skipped 0 0 0 0
28565 Error 0 0 0 0
28566 Last Flush: 8547 8246 8547
28567 Downstairs last five completed: 8246 8245 8244 8243 8242
28568 Upstairs last five completed: 8547 8546 8545 8544 8543
28569 CLIENT: Up:0 ds:301 act:2
28570 [8820] got start action
28571 [8820] I am going to restart
28572 Starting downstairs at port 8820
28573 Make output file at "/tmp/dsc/downstairs-8820.txt"
28574 [8820][1] reports Starting
28575 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1227)
28576 [8820][1] reports Running
28577 ----------------------------------------------------------------
28578 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28579 States: Active Offline Active
28580 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28581 8247 Acked Read 2 Done Sent Done false
28582 8248 Acked Write 6 Done Sent Done false
28583 8249 Acked Flush 0 Done Sent Done false
28584 8250 Acked Read 9 Done Sent Done false
28585 8251 Acked Read 1 Done Sent Done false
28586 8252 Acked Write 10 Done Sent Done false
28587 8253 Acked Read 4 Done Sent Done false
28588 8254 Acked Read 1 Done Sent Done false
28589 8255 Acked Flush 0 Done Sent Done false
28590 8256 Acked Write 10 Done Sent Done false
28591 8257 Acked Write 7 Done Sent Done false
28592 8258 Acked Read 10 Done Sent Done false
28593 8259 Acked Write 3 Done Sent Done false
28594 8260 Acked Write 6 Done Sent Done false
28595 8261 Acked Write 8 Done Sent Done false
28596 8262 Acked Write 6 Done Sent Done false
28597 8263 Acked Read 9 Done Sent Done false
28598 8264 Acked Write 6 Done Sent Done false
28599 8265 Acked Read 4 Done Sent Done false
28600 8266 Acked Read 4 Done Sent Done false
28601 8267 Acked Write 7 Done Sent Done false
28602 8268 Acked Flush 0 Done Sent Done false
28603 8269 Acked Write 6 Done Sent Done false
28604 8270 Acked Write 9 Done Sent Done false
28605 8271 Acked Read 1 Done Sent Done false
28606 8272 Acked Read 7 Done Sent Done false
28607 8273 Acked Read 3 Done Sent Done false
28608 8274 Acked Write 6 Done Sent Done false
28609 8275 Acked Write 2 Done Sent Done false
28610 8276 Acked Read 8 Done Sent Done false
28611 8277 Acked Read 6 Done Sent Done false
28612 8278 Acked Write 3 Done Sent Done false
28613 8279 Acked Write 6 Done Sent Done false
28614 8280 Acked Read 7 Done Sent Done false
28615 8281 Acked Read 5 Done Sent Done false
28616 8282 Acked Write 1 Done Sent Done false
28617 8283 Acked Read 6 Done Sent Done false
28618 8284 Acked Write 9 Done Sent Done false
28619 8285 Acked Read 1 Done Sent Done false
28620 8286 Acked Write 6 Done Sent Done false
28621 8287 Acked Write 5 Done Sent Done false
28622 8288 Acked Write 9 Done Sent Done false
28623 8289 Acked Read 7 Done Sent Done false
28624 8290 Acked Write 7 Done Sent Done false
28625 8291 Acked Write 8 Done Sent Done false
28626 8292 Acked Read 9 Done Sent Done false
28627 8293 Acked Read 8 Done Sent Done false
28628 8294 Acked Write 10 Done Sent Done false
28629 8295 Acked Read 6 Done Sent Done false
28630 8296 Acked Write 7 Done Sent Done false
28631 8297 Acked Read 1 Done Sent Done false
28632 8298 Acked Write 3 Done Sent Done false
28633 8299 Acked Read 4 Done Sent Done false
28634 8300 Acked Flush 0 Done Sent Done false
28635 8301 Acked Read 3 Done Sent Done false
28636 8302 Acked Read 1 Done Sent Done false
28637 8303 Acked Write 3 Done Sent Done false
28638 8304 Acked Write 10 Done Sent Done false
28639 8305 Acked Write 2 Done Sent Done false
28640 8306 Acked Write 2 Done Sent Done false
28641 8307 Acked Write 2 Done Sent Done false
28642 8308 Acked Flush 0 Done Sent Done false
28643 8309 Acked Flush 0 Done Sent Done false
28644 8310 Acked Write 10 Done Sent Done false
28645 8311 Acked Write 7 Done Sent Done false
28646 8312 Acked Write 4 Done Sent Done false
28647 8313 Acked Read 10 Done Sent Done false
28648 8314 Acked Read 2 Done Sent Done false
28649 8315 Acked Write 1 Done Sent Done false
28650 8316 Acked Read 4 Done Sent Done false
28651 8317 Acked Flush 0 Done Sent Done false
28652 8318 Acked Write 2 Done Sent Done false
28653 8319 Acked Flush 0 Done Sent Done false
28654 8320 Acked Write 8 Done Sent Done false
28655 8321 Acked Read 9 Done Sent Done false
28656 8322 Acked Read 4 Done Sent Done false
28657 8323 Acked Write 5 Done Sent Done false
28658 8324 Acked Read 3 Done Sent Done false
28659 8325 Acked Write 1 Done Sent Done false
28660 8326 Acked Write 1 Done Sent Done false
28661 8327 Acked Write 10 Done Sent Done false
28662 8328 Acked Flush 0 Done Sent Done false
28663 8329 Acked Read 2 Done Sent Done false
28664 8330 Acked Read 4 Done Sent Done false
28665 8331 Acked Read 4 Done Sent Done false
28666 8332 Acked Write 10 Done Sent Done false
28667 8333 Acked Read 4 Done Sent Done false
28668 8334 Acked Write 6 Done Sent Done false
28669 8335 Acked Flush 0 Done Sent Done false
28670 8336 Acked Read 7 Done Sent Done false
28671 8337 Acked Read 10 Done Sent Done false
28672 8338 Acked Write 6 Done Sent Done false
28673 8339 Acked Read 6 Done Sent Done false
28674 8340 Acked Write 8 Done Sent Done false
28675 8341 Acked Read 4 Done Sent Done false
28676 8342 Acked Read 2 Done Sent Done false
28677 8343 Acked Read 10 Done Sent Done false
28678 8344 Acked Flush 0 Done Sent Done false
28679 8345 Acked Read 8 Done Sent Done false
28680 8346 Acked Read 10 Done Sent Done false
28681 8347 Acked Read 2 Done Sent Done false
28682 8348 Acked Read 5 Done Sent Done false
28683 8349 Acked Write 1 Done Sent Done false
28684 8350 Acked Read 4 Done Sent Done false
28685 8351 Acked Write 10 Done Sent Done false
28686 8352 Acked Write 1 Done Sent Done false
28687 8353 Acked Read 6 Done Sent Done false
28688 8354 Acked Read 8 Done Sent Done false
28689 8355 Acked Flush 0 Done Sent Done false
28690 8356 Acked Read 6 Done Sent Done false
28691 8357 Acked Write 5 Done Sent Done false
28692 8358 Acked Read 5 Done Sent Done false
28693 8359 Acked Write 9 Done Sent Done false
28694 8360 Acked Write 3 Done Sent Done false
28695 8361 Acked Flush 0 Done Sent Done false
28696 8362 Acked Read 1 Done Sent Done false
28697 8363 Acked Write 8 Done Sent Done false
28698 8364 Acked Read 9 Done Sent Done false
28699 8365 Acked Read 8 Done Sent Done false
28700 8366 Acked Read 10 Done Sent Done false
28701 8367 Acked Write 3 Done Sent Done false
28702 8368 Acked Flush 0 Done Sent Done false
28703 8369 Acked Read 3 Done Sent Done false
28704 8370 Acked Read 4 Done Sent Done false
28705 8371 Acked Read 9 Done Sent Done false
28706 8372 Acked Read 7 Done Sent Done false
28707 8373 Acked Read 5 Done Sent Done false
28708 8374 Acked Read 9 Done Sent Done false
28709 8375 Acked Write 7 Done Sent Done false
28710 8376 Acked Read 9 Done Sent Done false
28711 8377 Acked Read 3 Done Sent Done false
28712 8378 Acked Read 3 Done Sent Done false
28713 8379 Acked Read 9 Done Sent Done false
28714 8380 Acked Write 4 Done Sent Done false
28715 8381 Acked Read 8 Done Sent Done false
28716 8382 Acked Write 3 Done Sent Done false
28717 8383 Acked Write 8 Done Sent Done false
28718 8384 Acked Read 7 Done Sent Done false
28719 8385 Acked Write 1 Done Sent Done false
28720 8386 Acked Read 6 Done Sent Done false
28721 8387 Acked Write 2 Done Sent Done false
28722 8388 Acked Write 6 Done Sent Done false
28723 8389 Acked Read 2 Done Sent Done false
28724 8390 Acked Write 4 Done Sent Done false
28725 8391 Acked Read 1 Done Sent Done false
28726 8392 Acked Read 3 Done Sent Done false
28727 8393 Acked Write 6 Done Sent Done false
28728 8394 Acked Write 9 Done Sent Done false
28729 8395 Acked Write 8 Done Sent Done false
28730 8396 Acked Read 3 Done Sent Done false
28731 8397 Acked Flush 0 Done Sent Done false
28732 8398 Acked Flush 0 Done Sent Done false
28733 8399 Acked Write 6 Done Sent Done false
28734 8400 Acked Write 9 Done Sent Done false
28735 8401 Acked Read 3 Done Sent Done false
28736 8402 Acked Write 1 Done Sent Done false
28737 8403 Acked Read 10 Done Sent Done false
28738 8404 Acked Read 7 Done Sent Done false
28739 8405 Acked Write 9 Done Sent Done false
28740 8406 Acked Write 8 Done Sent Done false
28741 8407 Acked Read 2 Done Sent Done false
28742 8408 Acked Read 4 Done Sent Done false
28743 8409 Acked Read 7 Done Sent Done false
28744 8410 Acked Read 6 Done Sent Done false
28745 8411 Acked Read 2 Done Sent Done false
28746 8412 Acked Write 2 Done Sent Done false
28747 8413 Acked Write 1 Done Sent Done false
28748 8414 Acked Read 8 Done Sent Done false
28749 8415 Acked Read 4 Done Sent Done false
28750 8416 Acked Read 3 Done Sent Done false
28751 8417 Acked Read 3 Done Sent Done false
28752 8418 Acked Write 7 Done Sent Done false
28753 8419 Acked Read 6 Done Sent Done false
28754 8420 Acked Write 3 Done Sent Done false
28755 8421 Acked Read 5 Done Sent Done false
28756 8422 Acked Read 4 Done Sent Done false
28757 8423 Acked Write 1 Done Sent Done false
28758 8424 Acked Read 9 Done Sent Done false
28759 8425 Acked Write 10 Done Sent Done false
28760 8426 Acked Read 10 Done Sent Done false
28761 8427 Acked Write 5 Done Sent Done false
28762 8428 Acked Write 10 Done Sent Done false
28763 8429 Acked Read 4 Done Sent Done false
28764 8430 Acked Read 1 Done Sent Done false
28765 8431 Acked Flush 0 Done Sent Done false
28766 8432 Acked Write 8 Done Sent Done false
28767 8433 Acked Read 6 Done Sent Done false
28768 8434 Acked Flush 0 Done Sent Done false
28769 8435 Acked Read 3 Done Sent Done false
28770 8436 Acked Flush 0 Done Sent Done false
28771 8437 Acked Write 5 Done Sent Done false
28772 8438 Acked Read 3 Done Sent Done false
28773 8439 Acked Read 7 Done Sent Done false
28774 8440 Acked Read 1 Done Sent Done false
28775 8441 Acked Read 10 Done Sent Done false
28776 8442 Acked Write 9 Done Sent Done false
28777 8443 Acked Read 9 Done Sent Done false
28778 8444 Acked Write 3 Done Sent Done false
28779 8445 Acked Write 4 Done Sent Done false
28780 8446 Acked Read 6 Done Sent Done false
28781 8447 Acked Write 2 Done Sent Done false
28782 8448 Acked Read 9 Done Sent Done false
28783 8449 Acked Write 5 Done Sent Done false
28784 8450 Acked Write 3 Done Sent Done false
28785 8451 Acked Read 2 Done Sent Done false
28786 8452 Acked Write 8 Done Sent Done false
28787 8453 Acked Read 9 Done Sent Done false
28788 8454 Acked Read 6 Done Sent Done false
28789 8455 Acked Write 5 Done Sent Done false
28790 8456 Acked Write 4 Done Sent Done false
28791 8457 Acked Write 9 Done Sent Done false
28792 8458 Acked Read 6 Done Sent Done false
28793 8459 Acked Flush 0 Done Sent Done false
28794 8460 Acked Write 9 Done Sent Done false
28795 8461 Acked Flush 0 Done Sent Done false
28796 8462 Acked Read 4 Done Sent Done false
28797 8463 Acked Read 5 Done Sent Done false
28798 8464 Acked Read 4 Done Sent Done false
28799 8465 Acked Read 4 Done Sent Done false
28800 8466 Acked Read 5 Done Sent Done false
28801 8467 Acked Write 6 Done Sent Done false
28802 8468 Acked Read 9 Done Sent Done false
28803 8469 Acked Write 5 Done Sent Done false
28804 8470 Acked Read 2 Done Sent Done false
28805 8471 Acked Read 5 Done Sent Done false
28806 8472 Acked Read 8 Done Sent Done false
28807 8473 Acked Write 10 Done Sent Done false
28808 8474 Acked Read 2 Done Sent Done false
28809 8475 Acked Write 5 Done Sent Done false
28810 8476 Acked Write 2 Done Sent Done false
28811 8477 Acked Write 5 Done Sent Done false
28812 8478 Acked Write 10 Done Sent Done false
28813 8479 Acked Read 5 Done Sent Done false
28814 8480 Acked Read 9 Done Sent Done false
28815 8481 Acked Read 4 Done Sent Done false
28816 8482 Acked Write 2 Done Sent Done false
28817 8483 Acked Write 3 Done Sent Done false
28818 8484 Acked Write 5 Done Sent Done false
28819 8485 Acked Read 7 Done Sent Done false
28820 8486 Acked Write 4 Done Sent Done false
28821 8487 Acked Read 5 Done Sent Done false
28822 8488 Acked Write 7 Done Sent Done false
28823 8489 Acked Read 8 Done Sent Done false
28824 8490 Acked Write 6 Done Sent Done false
28825 8491 Acked Write 8 Done Sent Done false
28826 8492 Acked Read 7 Done Sent Done false
28827 8493 Acked Read 1 Done Sent Done false
28828 8494 Acked Write 9 Done Sent Done false
28829 8495 Acked Read 2 Done Sent Done false
28830 8496 Acked Read 8 Done Sent Done false
28831 8497 Acked Read 10 Done Sent Done false
28832 8498 Acked Read 3 Done Sent Done false
28833 8499 Acked Read 1 Done Sent Done false
28834 8500 Acked Read 5 Done Sent Done false
28835 8501 Acked Write 6 Done Sent Done false
28836 8502 Acked Write 5 Done Sent Done false
28837 8503 Acked Read 9 Done Sent Done false
28838 8504 Acked Write 2 Done Sent Done false
28839 8505 Acked Write 2 Done Sent Done false
28840 8506 Acked Write 8 Done Sent Done false
28841 8507 Acked Flush 0 Done Sent Done false
28842 8508 Acked Read 7 Done Sent Done false
28843 8509 Acked Read 3 Done Sent Done false
28844 8510 Acked Write 2 Done Sent Done false
28845 8511 Acked Write 8 Done Sent Done false
28846 8512 Acked Read 3 Done Sent Done false
28847 8513 Acked Read 4 Done Sent Done false
28848 8514 Acked Read 1 Done Sent Done false
28849 8515 Acked Read 3 Done Sent Done false
28850 8516 Acked Read 2 Done Sent Done false
28851 8517 Acked Write 3 Done Sent Done false
28852 8518 Acked Read 7 Done Sent Done false
28853 8519 Acked Write 1 Done Sent Done false
28854 8520 Acked Write 2 Done Sent Done false
28855 8521 Acked Read 6 Done Sent Done false
28856 8522 Acked Read 2 Done Sent Done false
28857 8523 Acked Write 3 Done Sent Done false
28858 8524 Acked Write 10 Done Sent Done false
28859 8525 Acked Read 3 Done Sent Done false
28860 8526 Acked Flush 0 Done Sent Done false
28861 8527 Acked Read 1 Done Sent Done false
28862 8528 Acked Flush 0 Done Sent Done false
28863 8529 Acked Write 1 Done Sent Done false
28864 8530 Acked Read 2 Done Sent Done false
28865 8531 Acked Flush 0 Done Sent Done false
28866 8532 Acked Write 3 Done Sent Done false
28867 8533 Acked Flush 0 Done Sent Done false
28868 8534 Acked Read 4 Done Sent Done false
28869 8535 Acked Read 1 Done Sent Done false
28870 8536 Acked Read 2 Done Sent Done false
28871 8537 Acked Write 3 Done Sent Done false
28872 8538 Acked Write 5 Done Sent Done false
28873 8539 Acked Read 6 Done Sent Done false
28874 8540 Acked Read 4 Done Sent Done false
28875 8541 Acked Write 2 Done Sent Done false
28876 8542 Acked Write 10 Done Sent Done false
28877 8543 Acked Read 8 Done Sent Done false
28878 8544 Acked Read 6 Done Sent Done false
28879 8545 Acked Flush 0 Done Sent Done false
28880 8546 Acked Write 8 Done Sent Done false
28881 8547 Acked Flush 0 Done Sent Done false
28882 STATES DS:0 DS:1 DS:2 TOTAL
28883 Sent 0 301 0 301
28884 Done 301 0 301 602
28885 Skipped 0 0 0 0
28886 Error 0 0 0 0
28887 Last Flush: 8547 8246 8547
28888 Downstairs last five completed: 8246 8245 8244 8243 8242
28889 Upstairs last five completed: 8547 8546 8545 8544 8543
28890 CLIENT: Up:0 ds:301 act:2
288912025-01-15T05:25:02.789ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
288922025-01-15T05:25:02.789ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36428) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
288932025-01-15T05:25:02.790ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
288942025-01-15T05:25:02.791ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
288952025-01-15T05:25:02.791ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
288962025-01-15T05:25:02.791ZINFOcrucible: send last flush ID to this DS: 8246 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
288972025-01-15T05:25:02.791ZINFOcrucible: Replied this last flush ID: 8246 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
288982025-01-15T05:25:02.791ZINFOcrucible: [1] client re-new 301 jobs since flush 8246 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
288992025-01-15T05:25:02.791ZINFOcrucible: [1] Replayed 301 jobs since flush: 8246 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
28900 ----------------------------------------------------------------
28901 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28902 Downstairs last five completed: 8547 8546 8545 8544 8543
28903 Upstairs last five completed: 8547 8546 8545 8544 8543
28904 CLIENT: Up:0 ds:0 act:3
28905 Replay: All jobs finished
28906 Main task has work to do, go find it
28907 got dsc Stop(2)
28908 stop 2
28909 [8830] Got stop action so:false kr:false
28910 Jan 15 05:25:05.024 INFO request completed, latency_us: 342, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 204ab774-b824-42a2-b8fd-919f559ee1b7, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
289112025-01-15T05:25:05.025ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
289122025-01-15T05:25:05.025ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
289132025-01-15T05:25:05.025ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
28914 Jan 15 05:25:05.025 INFO request completed, latency_us: 105, response_code: 200, uri: /state/cid/2, method: GET, req_id: a58d1344-8050-4eb7-8eb2-4a8122543f52, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:25:05.025394022Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"2","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
28915 
28916 [8830] Exited with: ExitStatus(unix_wait_status(9))
28917 [8830][2] reports Exit
28918 Jan 15 05:25:09.027 INFO request completed, latency_us: 196, response_code: 200, uri: /state/cid/2, method: GET, req_id: 33aef2a4-0428-4675-95ce-15cbe793b0dc, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
28919 001/300 Read block 2012 len 36864
28920 002/300 Read block 1440 len 24576
28921 003/300 Read block 253 len 12288
28922 004/300 Write block 20 len 28672 data: 4 5 5 5 6 8 7
28923 005/300 Write block 3884 len 8192 data: 6 5
28924 006/300 Read block 4228 len 16384
28925 007/300 Write block 4109 len 20480 data: 5 5 5 4 4
28926 008/300 Write block 4783 len 12288 data: 5 4 4
28927 009/300 Read block 4235 len 40960
28928 010/300 Write block 4210 len 24576 data: 8 8 7 6 5 6
28929 011/300 Read block 425 len 16384
28930 012/300 Write block 1407 len 8192 data: 3 4
28931 013/300 Read block 2791 len 40960
28932 014/300 Read block 3573 len 16384
28933 015/300 Write block 4991 len 28672 data: 4 4 4 4 6 7 5
28934 016/300 Write block 1671 len 24576 data: 5 5 5 5 5 6
28935 017/300 Read block 1471 len 8192
28936 018/300 Write block 604 len 16384 data: 4 4 4 4
28937 019/300 Flush
28938 020/300 Read block 1269 len 36864
28939 021/300 Flush
28940 022/300 Write block 2661 len 28672 data: 6 6 5 2 2 2 3
28941 023/300 Write block 1592 len 24576 data: 3 3 2 4 5 5
28942 024/300 Write block 2819 len 12288 data: 4 5 5
28943 025/300 Write block 2735 len 40960 data: 5 4 4 3 3 2 5 5 5 5
28944 026/300 Write block 239 len 36864 data: 4 5 4 4 3 2 4 6 7
28945 027/300 Read block 860 len 12288
28946 028/300 Read block 3236 len 20480
28947 029/300 Read block 3280 len 28672
28948 030/300 Read block 937 len 20480
28949 031/300 Write block 1285 len 12288 data: 2 3 4
28950 032/300 Write block 688 len 40960 data: 5 6 8 9 7 6 6 6 7 7
28951 033/300 Read block 4156 len 36864
28952 034/300 Read block 881 len 16384
28953 035/300 Read block 168 len 16384
28954 036/300 Write block 4951 len 40960 data: 4 4 4 3 3 3 3 2 5 4
28955 037/300 Flush
28956 038/300 Read block 1672 len 16384
28957 039/300 Read block 4120 len 40960
28958 040/300 Write block 2570 len 12288 data: 6 4 5
28959 041/300 Read block 2199 len 16384
28960 042/300 Read block 1663 len 4096
28961 043/300 Write block 469 len 16384 data: 7 7 7 7
28962 044/300 Read block 3382 len 40960
28963 045/300 Read block 3240 len 4096
28964 046/300 Read block 1174 len 28672
28965 047/300 Read block 2305 len 32768
28966 048/300 Write block 347 len 36864 data: 4 3 3 4 5 5 6 6 7
28967 049/300 Write block 2136 len 8192 data: 2 4
28968 050/300 Read block 2566 len 12288
28969 051/300 Read block 3231 len 4096
28970 052/300 Read block 711 len 8192
28971 053/300 Read block 2892 len 12288
28972 054/300 Read block 256 len 40960
28973 055/300 Read block 560 len 36864
28974 056/300 Write block 218 len 12288 data: 7 6 6
28975 057/300 Write block 2463 len 12288 data: 7 5 6
28976 058/300 Read block 527 len 24576
28977 059/300 Read block 3882 len 12288
28978 060/300 Read block 2973 len 20480
28979 061/300 Read block 2340 len 16384
28980 062/300 Write block 2625 len 32768 data: 6 6 5 6 7 5 4 4
28981 063/300 Read block 1110 len 32768
28982 064/300 Write block 3104 len 28672 data: 6 5 5 4 4 3 3
28983 065/300 Write block 4274 len 36864 data: 5 4 4 4 4 5 4 4 5
28984 066/300 Read block 633 len 20480
28985 067/300 Read block 3099 len 12288
28986 068/300 Write block 2214 len 20480 data: 4 5 5 3 3
28987 069/300 Flush
28988 070/300 Write block 997 len 16384 data: 3 4 4 4
28989 071/300 Write block 3336 len 12288 data: 8 8 8
28990 072/300 Read block 3057 len 12288
28991 073/300 Write block 4633 len 36864 data: 7 7 6 6 5 4 4 3 2
28992 074/300 Read block 3293 len 16384
28993 075/300 Write block 3883 len 36864 data: 5 7 6 5 6 5 5 4 5
28994 076/300 Write block 2722 len 36864 data: 3 4 5 4 5 6 6 6 5
28995 077/300 Write block 2000 len 12288 data: 6 7 6
28996 078/300 Read block 2081 len 12288
28997 079/300 Write block 4421 len 20480 data: 8 9 7 6 9
28998 080/300 Write block 2013 len 28672 data: 5 8 7 9 9 9 9
28999 081/300 Flush
29000 082/300 Write block 376 len 8192 data: 6 8
29001 083/300 Flush
29002 084/300 Write block 4487 len 12288 data: 5 5 5
29003 085/300 Write block 16 len 28672 data: 4 5 6 5 5 6 6
29004 086/300 Read block 1343 len 40960
29005 087/300 Write block 838 len 28672 data: 4 5 3 3 4 4 3
29006 088/300 Read block 2588 len 12288
29007 089/300 Read block 591 len 36864
29008 090/300 Flush
29009 091/300 Write block 4044 len 36864 data: 4 5 4 4 5 6 7 9 6
29010 092/300 Write block 3371 len 28672 data: 6 5 7 7 7 6 6
29011 093/300 Write block 171 len 16384 data: 4 5 5 6
29012 094/300 Read block 1780 len 36864
29013 095/300 Write block 757 len 12288 data: 4 5 4
29014 096/300 Write block 3523 len 8192 data: 6 6
29015 097/300 Read block 1232 len 28672
29016 098/300 Flush
29017 099/300 Read block 1307 len 4096
29018 100/300 Read block 1386 len 8192
29019 101/300 Read block 1295 len 12288
29020 102/300 Write block 4500 len 12288 data: 7 8 8
29021 103/300 Read block 3514 len 12288
29022 104/300 Read block 731 len 28672
29023 105/300 Write block 453 len 16384 data: 6 8 8 9
29024 106/300 Read block 4436 len 16384
29025 107/300 Write block 547 len 16384 data: 5 5 6 6
29026 108/300 Write block 1518 len 28672 data: 5 4 4 2 3 5 5
29027 109/300 Write block 489 len 36864 data: 5 6 6 5 5 6 6 4 4
29028 110/300 Read block 1094 len 4096
29029 111/300 Flush
29030 112/300 Read block 435 len 16384
29031 113/300 Read block 4782 len 40960
29032 114/300 Read block 1028 len 8192
29033 115/300 Read block 2175 len 36864
29034 116/300 Read block 4789 len 8192
29035 117/300 Write block 2860 len 40960 data: 5 6 5 5 5 8 7 7 6 6
29036 118/300 Read block 2434 len 32768
29037 119/300 Flush
29038 120/300 Read block 337 len 16384
29039 121/300 Read block 4713 len 40960
29040 122/300 Read block 3496 len 24576
29041 123/300 Read block 761 len 28672
29042 124/300 Read block 1595 len 12288
29043 125/300 Write block 826 len 24576 data: 7 6 5 6 5 4
29044 126/300 Read block 1435 len 40960
29045 127/300 Read block 404 len 28672
29046 128/300 Write block 2424 len 28672 data: 4 5 5 5 6 6 5
29047 129/300 Read block 3580 len 32768
29048 130/300 Read block 4854 len 32768
29049 131/300 Read block 1373 len 40960
29050 132/300 Write block 863 len 28672 data: 5 5 5 6 7 7 9
29051 133/300 Read block 3644 len 40960
29052 134/300 Write block 1778 len 32768 data: 6 5 6 6 6 10 10 10
29053 135/300 Write block 161 len 28672 data: 6 6 5 5 4 6 5
29054 136/300 Read block 3837 len 28672
29055 137/300 Write block 1712 len 16384 data: 3 4 4 5
29056 138/300 Read block 3853 len 16384
29057 139/300 Write block 4503 len 12288 data: 7 6 4
29058 140/300 Read block 1246 len 4096
29059 141/300 Write block 4822 len 32768 data: 6 6 6 4 3 3 3 5
29060 142/300 Flush
29061 143/300 Flush
29062 144/300 Write block 944 len 36864 data: 4 5 5 6 6 6 7 7 6
29063 145/300 Write block 2447 len 32768 data: 4 6 6 6 5 5 5 6
29064 146/300 Write block 3815 len 20480 data: 7 9 8 7 7
29065 147/300 Write block 22 len 24576 data: 7 6 7 9 8 7
29066 148/300 Write block 3682 len 32768 data: 8 6 5 5 5 5 4 4
29067 149/300 Read block 1074 len 32768
29068 150/300 Write block 1405 len 28672 data: 4 4 4 5 4 4 5
29069 151/300 Write block 1711 len 24576 data: 3 4 5 5 6 5
29070 152/300 Read block 745 len 12288
29071 153/300 Read block 965 len 40960
29072 154/300 Read block 4831 len 36864
29073 155/300 Write block 501 len 8192 data: 4 4
29074 156/300 Write block 4924 len 28672 data: 4 3 3 3 4 5 4
29075 157/300 Read block 4632 len 36864
29076 158/300 Write block 285 len 36864 data: 5 5 7 7 8 8 8 6 6
29077 159/300 Read block 1119 len 36864
29078 160/300 Read block 907 len 16384
29079 161/300 Read block 231 len 36864
29080 162/300 Write block 943 len 16384 data: 5 5 6 6
29081 163/300 Write block 3988 len 36864 data: 4 4 5 6 6 5 4 4 4
29082 164/300 Write block 402 len 16384 data: 7 7 7 6
29083 165/300 Read block 2515 len 12288
29084 166/300 Read block 2854 len 32768
29085 167/300 Read block 3423 len 20480
29086 168/300 Read block 778 len 32768
29087 169/300 Read block 1004 len 28672
29088 170/300 Read block 3192 len 20480
29089 171/300 Write block 828 len 8192 data: 6 7
29090 172/300 Write block 4967 len 16384 data: 6 6 5 5
29091 173/300 Write block 2775 len 16384 data: 6 5 7 7
29092 174/300 Write block 3849 len 32768 data: 3 4 4 5 5 5 6 7
29093 175/300 Write block 4400 len 24576 data: 2 4 4 5 4 3
29094 176/300 Flush
29095 177/300 Write block 2205 len 12288 data: 6 7 6
29096 178/300 Read block 1102 len 24576
29097 179/300 Write block 400 len 32768 data: 4 7 8 8 8 7 5 6
29098 180/300 Write block 4808 len 12288 data: 3 3 4
29099 181/300 Read block 3560 len 12288
29100 182/300 Write block 4924 len 12288 data: 5 4 4
29101 183/300 Write block 4182 len 24576 data: 5 4 4 3 3 5
29102 184/300 Flush
29103 185/300 Write block 802 len 12288 data: 3 3 4
29104 186/300 Read block 457 len 40960
29105 187/300 Write block 3539 len 16384 data: 5 6 6 6
29106 188/300 Read block 3442 len 8192
29107 189/300 Write block 4928 len 8192 data: 5 6
29108 190/300 Read block 2611 len 28672
29109 191/300 Write block 2374 len 28672 data: 5 3 3 5 4 4 4
29110 192/300 Read block 3117 len 8192
29111 193/300 Read block 4456 len 4096
29112 194/300 Write block 1563 len 36864 data: 5 8 5 4 3 5 6 7 7
29113 195/300 Write block 2433 len 4096 data: 12
29114 196/300 Write block 1197 len 16384 data: 3 3 3 4
29115 197/300 Flush
29116 198/300 Write block 212 len 40960 data: 2 4 4 5 4 4 8 7 7 5
29117 199/300 Read block 4949 len 4096
29118 200/300 Write block 4573 len 24576 data: 10 9 9 5 6 6
29119 201/300 Flush
29120 202/300 Read block 1905 len 20480
29121 203/300 Read block 3869 len 4096
29122 204/300 Write block 3919 len 16384 data: 7 6 6 6
29123 205/300 Read block 3989 len 20480
29124 206/300 Read block 2485 len 16384
29125 207/300 Write block 1272 len 32768 data: 5 4 5 3 4 3 3 3
29126 208/300 Write block 4742 len 20480 data: 8 11 12 10 8
29127 209/300 Write block 4936 len 32768 data: 6 7 6 6 6 7 5 4
29128 210/300 Flush
29129 211/300 Read block 2692 len 36864
29130 212/300 Write block 2304 len 36864 data: 7 7 4 4 4 4 4 4 4
29131 213/300 Read block 319 len 28672
29132 214/300 Flush
29133 215/300 Write block 1647 len 4096 data: 5
29134 216/300 Write block 3619 len 24576 data: 6 5 4 4 3 4
29135 217/300 Write block 3070 len 12288 data: 3 4 6
29136 218/300 Write block 3440 len 16384 data: 6 6 7 6
29137 219/300 Flush
29138 220/300 Flush
29139 221/300 Write block 4380 len 40960 data: 7 8 7 7 6 6 5 6 6 5
29140 222/300 Write block 1926 len 20480 data: 9 9 8 8 8
29141 223/300 Read block 2103 len 36864
29142 224/300 Write block 39 len 16384 data: 4 8 7 8
29143 225/300 Read block 4179 len 8192
29144 226/300 Read block 556 len 16384
29145 227/300 Read block 4079 len 32768
29146 228/300 Write block 619 len 36864 data: 3 3 3 3 2 2 4 4 5
29147 229/300 Write block 3287 len 32768 data: 5 5 3 3 3 3 4 4
29148 230/300 Write block 668 len 4096 data: 4
29149 231/300 Flush
29150 232/300 Read block 2497 len 12288
29151 233/300 Flush
29152 234/300 Write block 1071 len 32768 data: 3 3 5 5 6 7 8 9
29153 235/300 Read block 3519 len 16384
29154 236/300 Read block 1349 len 4096
29155 237/300 Read block 1795 len 36864
29156 238/300 Read block 2497 len 40960
29157 239/300 Write block 4223 len 28672 data: 6 4 3 4 4 5 5
29158 240/300 Read block 4139 len 28672
29159 241/300 Read block 1211 len 28672
29160 242/300 Write block 148 len 24576 data: 3 4 4 4 4 3
29161 243/300 Write block 2004 len 28672 data: 4 4 4 4 3 5 5
29162 244/300 Read block 2452 len 4096
29163 245/300 Flush
29164 246/300 Flush
29165 247/300 Read block 1304 len 20480
29166 248/300 Write block 3481 len 40960 data: 5 4 6 5 5 5 4 5 5 4
29167 249/300 Flush
29168 250/300 Read block 3694 len 32768
29169 251/300 Read block 395 len 32768
29170 252/300 Write block 1739 len 28672 data: 2 2 4 3 3 3 3
29171 253/300 Read block 570 len 20480
29172 254/300 Read block 1278 len 36864
29173 255/300 Read block 1969 len 36864
29174 256/300 Write block 163 len 28672 data: 6 6 5 7 6 2 2
29175 257/300 Read block 4990 len 32768
29176 258/300 Write block 4490 len 24576 data: 5 6 4 6 6 8
29177 259/300 Read block 606 len 4096
29178 260/300 Write block 1476 len 8192 data: 5 6
29179 261/300 Read block 4535 len 4096
29180 262/300 Read block 3261 len 8192
29181 263/300 Read block 251 len 28672
29182 264/300 Read block 4868 len 24576
29183 265/300 Read block 1667 len 4096
29184 266/300 Write block 4138 len 8192 data: 6 4
29185 267/300 Read block 1326 len 28672
29186 268/300 Read block 4209 len 16384
29187 269/300 Flush
29188 270/300 Read block 733 len 36864
29189 271/300 Read block 310 len 8192
29190 272/300 Read block 1126 len 36864
29191 273/300 Flush
29192 274/300 Read block 2519 len 36864
29193 275/300 Write block 3459 len 40960 data: 5 6 4 5 5 5 5 4 4 6
29194 276/300 Write block 2523 len 36864 data: 3 4 5 4 4 5 7 5 6
29195 277/300 Write block 1417 len 24576 data: 5 5 5 5 6 6
29196 278/300 Read block 1270 len 24576
29197 279/300 Read block 4446 len 40960
29198 280/300 Write block 228 len 40960 data: 4 5 5 6 5 5 4 4 4 4
29199 281/300 Read block 4802 len 20480
29200 282/300 Read block 2733 len 36864
29201 283/300 Read block 2545 len 16384
29202 284/300 Write block 3131 len 40960 data: 5 6 5 5 5 6 7 5 6 5
29203 285/300 Write block 937 len 28672 data: 6 4 5 5 4 5 6
29204 286/300 Read block 4109 len 24576
29205 287/300 Write block 2427 len 40960 data: 6 7 7 6 6 10 13 11 7 7
29206 288/300 Read block 435 len 16384
29207 289/300 Read block 942 len 20480
29208 290/300 Read block 3414 len 28672
29209 291/300 Read block 2298 len 40960
29210 292/300 Write block 2720 len 28672 data: 5 3 4 5 6 5 6
29211 293/300 Write block 361 len 28672 data: 6 5 5 5 4 4 3
29212 294/300 Flush
29213 295/300 Read block 4077 len 32768
29214 296/300 Write block 4842 len 32768 data: 2 3 5 5 5 5 5 6
29215 297/300 Write block 4592 len 8192 data: 4 3
29216 298/300 Read block 1187 len 32768
29217 299/300 Read block 68 len 32768
29218 300/300 Flush
29219 301/300 Write block 811 len 20480 data: 5 4 4 3 3
29220 Main task has work to do, go find it
29221 got dsc Start(2)
29222 start 2
29223 Jan 15 05:25:09.268 INFO request completed, latency_us: 286, response_code: 204, uri: /start/cid/2, method: POST, req_id: e7d4c9ad-f94d-494e-8b48-ce81d127ad06, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
29224 Replay: started 2, returned:Ok(())
29225 ----------------------------------------------------------------
29226 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29227 States: Active Active Offline
29228 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29229 8548 Acked Read 9 Done Done Sent false
29230 8549 Acked Read 6 Done Done Sent false
29231 8550 Acked Read 3 Done Done Sent false
29232 8551 Acked Write 7 Done Done Sent false
29233 8552 Acked Write 2 Done Done Sent false
29234 8553 Acked Read 4 Done Done Sent false
29235 8554 Acked Write 5 Done Done Sent false
29236 8555 Acked Write 3 Done Done Sent false
29237 8556 Acked Read 10 Done Done Sent false
29238 8557 Acked Write 6 Done Done Sent false
29239 8558 Acked Read 4 Done Done Sent false
29240 8559 Acked Write 2 Done Done Sent false
29241 8560 Acked Read 10 Done Done Sent false
29242 8561 Acked Read 4 Done Done Sent false
29243 8562 Acked Write 7 Done Done Sent false
29244 8563 Acked Write 6 Done Done Sent false
29245 8564 Acked Read 2 Done Done Sent false
29246 8565 Acked Write 4 Done Done Sent false
29247 8566 Acked Flush 0 Done Done Sent false
29248 8567 Acked Read 9 Done Done Sent false
29249 8568 Acked Flush 0 Done Done Sent false
29250 8569 Acked Write 7 Done Done Sent false
29251 8570 Acked Write 6 Done Done Sent false
29252 8571 Acked Write 3 Done Done Sent false
29253 8572 Acked Write 10 Done Done Sent false
29254 8573 Acked Write 9 Done Done Sent false
29255 8574 Acked Read 3 Done Done Sent false
29256 8575 Acked Read 5 Done Done Sent false
29257 8576 Acked Read 7 Done Done Sent false
29258 8577 Acked Read 5 Done Done Sent false
29259 8578 Acked Write 3 Done Done Sent false
29260 8579 Acked Write 10 Done Done Sent false
29261 8580 Acked Read 9 Done Done Sent false
29262 8581 Acked Read 4 Done Done Sent false
29263 8582 Acked Read 4 Done Done Sent false
29264 8583 Acked Write 10 Done Done Sent false
29265 8584 Acked Flush 0 Done Done Sent false
29266 8585 Acked Read 4 Done Done Sent false
29267 8586 Acked Read 10 Done Done Sent false
29268 8587 Acked Write 3 Done Done Sent false
29269 8588 Acked Read 4 Done Done Sent false
29270 8589 Acked Read 1 Done Done Sent false
29271 8590 Acked Write 4 Done Done Sent false
29272 8591 Acked Read 10 Done Done Sent false
29273 8592 Acked Read 1 Done Done Sent false
29274 8593 Acked Read 7 Done Done Sent false
29275 8594 Acked Read 8 Done Done Sent false
29276 8595 Acked Write 9 Done Done Sent false
29277 8596 Acked Write 2 Done Done Sent false
29278 8597 Acked Read 3 Done Done Sent false
29279 8598 Acked Read 1 Done Done Sent false
29280 8599 Acked Read 2 Done Done Sent false
29281 8600 Acked Read 3 Done Done Sent false
29282 8601 Acked Read 10 Done Done Sent false
29283 8602 Acked Read 9 Done Done Sent false
29284 8603 Acked Write 3 Done Done Sent false
29285 8604 Acked Write 3 Done Done Sent false
29286 8605 Acked Read 6 Done Done Sent false
29287 8606 Acked Read 3 Done Done Sent false
29288 8607 Acked Read 5 Done Done Sent false
29289 8608 Acked Read 4 Done Done Sent false
29290 8609 Acked Write 8 Done Done Sent false
29291 8610 Acked Read 8 Done Done Sent false
29292 8611 Acked Write 7 Done Done Sent false
29293 8612 Acked Write 9 Done Done Sent false
29294 8613 Acked Read 5 Done Done Sent false
29295 8614 Acked Read 3 Done Done Sent false
29296 8615 Acked Write 5 Done Done Sent false
29297 8616 Acked Flush 0 Done Done Sent false
29298 8617 Acked Write 4 Done Done Sent false
29299 8618 Acked Write 3 Done Done Sent false
29300 8619 Acked Read 3 Done Done Sent false
29301 8620 Acked Write 9 Done Done Sent false
29302 8621 Acked Read 4 Done Done Sent false
29303 8622 Acked Write 9 Done Done Sent false
29304 8623 Acked Write 9 Done Done Sent false
29305 8624 Acked Write 3 Done Done Sent false
29306 8625 Acked Read 3 Done Done Sent false
29307 8626 Acked Write 5 Done Done Sent false
29308 8627 Acked Write 7 Done Done Sent false
29309 8628 Acked Flush 0 Done Done Sent false
29310 8629 Acked Write 2 Done Done Sent false
29311 8630 Acked Flush 0 Done Done Sent false
29312 8631 Acked Write 3 Done Done Sent false
29313 8632 Acked Write 7 Done Done Sent false
29314 8633 Acked Read 10 Done Done Sent false
29315 8634 Acked Write 7 Done Done Sent false
29316 8635 Acked Read 3 Done Done Sent false
29317 8636 Acked Read 9 Done Done Sent false
29318 8637 Acked Flush 0 Done Done Sent false
29319 8638 Acked Write 9 Done Done Sent false
29320 8639 Acked Write 7 Done Done Sent false
29321 8640 Acked Write 4 Done Done Sent false
29322 8641 Acked Read 9 Done Done Sent false
29323 8642 Acked Write 3 Done Done Sent false
29324 8643 Acked Write 2 Done Done Sent false
29325 8644 Acked Read 7 Done Done Sent false
29326 8645 Acked Flush 0 Done Done Sent false
29327 8646 Acked Read 1 Done Done Sent false
29328 8647 Acked Read 2 Done Done Sent false
29329 8648 Acked Read 3 Done Done Sent false
29330 8649 Acked Write 3 Done Done Sent false
29331 8650 Acked Read 3 Done Done Sent false
29332 8651 Acked Read 7 Done Done Sent false
29333 8652 Acked Write 4 Done Done Sent false
29334 8653 Acked Read 4 Done Done Sent false
29335 8654 Acked Write 4 Done Done Sent false
29336 8655 Acked Write 7 Done Done Sent false
29337 8656 Acked Write 9 Done Done Sent false
29338 8657 Acked Read 1 Done Done Sent false
29339 8658 Acked Flush 0 Done Done Sent false
29340 8659 Acked Read 4 Done Done Sent false
29341 8660 Acked Read 10 Done Done Sent false
29342 8661 Acked Read 2 Done Done Sent false
29343 8662 Acked Read 9 Done Done Sent false
29344 8663 Acked Read 2 Done Done Sent false
29345 8664 Acked Write 10 Done Done Sent false
29346 8665 Acked Read 8 Done Done Sent false
29347 8666 Acked Flush 0 Done Done Sent false
29348 8667 Acked Read 4 Done Done Sent false
29349 8668 Acked Read 10 Done Done Sent false
29350 8669 Acked Read 6 Done Done Sent false
29351 8670 Acked Read 7 Done Done Sent false
29352 8671 Acked Read 3 Done Done Sent false
29353 8672 Acked Write 6 Done Done Sent false
29354 8673 Acked Read 10 Done Done Sent false
29355 8674 Acked Read 7 Done Done Sent false
29356 8675 Acked Write 7 Done Done Sent false
29357 8676 Acked Read 8 Done Done Sent false
29358 8677 Acked Read 8 Done Done Sent false
29359 8678 Acked Read 10 Done Done Sent false
29360 8679 Acked Write 7 Done Done Sent false
29361 8680 Acked Read 10 Done Done Sent false
29362 8681 Acked Write 8 Done Done Sent false
29363 8682 Acked Write 7 Done Done Sent false
29364 8683 Acked Read 7 Done Done Sent false
29365 8684 Acked Write 4 Done Done Sent false
29366 8685 Acked Read 4 Done Done Sent false
29367 8686 Acked Write 3 Done Done Sent false
29368 8687 Acked Read 1 Done Done Sent false
29369 8688 Acked Write 8 Done Done Sent false
29370 8689 Acked Flush 0 Done Done Sent false
29371 8690 Acked Flush 0 Done Done Sent false
29372 8691 Acked Write 9 Done Done Sent false
29373 8692 Acked Write 8 Done Done Sent false
29374 8693 Acked Write 5 Done Done Sent false
29375 8694 Acked Write 6 Done Done Sent false
29376 8695 Acked Write 8 Done Done Sent false
29377 8696 Acked Read 8 Done Done Sent false
29378 8697 Acked Write 7 Done Done Sent false
29379 8698 Acked Write 6 Done Done Sent false
29380 8699 Acked Read 3 Done Done Sent false
29381 8700 Acked Read 10 Done Done Sent false
29382 8701 Acked Read 9 Done Done Sent false
29383 8702 Acked Write 2 Done Done Sent false
29384 8703 Acked Write 7 Done Done Sent false
29385 8704 Acked Read 9 Done Done Sent false
29386 8705 Acked Write 9 Done Done Sent false
29387 8706 Acked Read 9 Done Done Sent false
29388 8707 Acked Read 4 Done Done Sent false
29389 8708 Acked Read 9 Done Done Sent false
29390 8709 Acked Write 4 Done Done Sent false
29391 8710 Acked Write 9 Done Done Sent false
29392 8711 Acked Write 4 Done Done Sent false
29393 8712 Acked Read 3 Done Done Sent false
29394 8713 Acked Read 8 Done Done Sent false
29395 8714 Acked Read 5 Done Done Sent false
29396 8715 Acked Read 8 Done Done Sent false
29397 8716 Acked Read 7 Done Done Sent false
29398 8717 Acked Read 5 Done Done Sent false
29399 8718 Acked Write 2 Done Done Sent false
29400 8719 Acked Write 4 Done Done Sent false
29401 8720 Acked Write 4 Done Done Sent false
29402 8721 Acked Write 8 Done Done Sent false
29403 8722 Acked Write 6 Done Done Sent false
29404 8723 Acked Flush 0 Done Done Sent false
29405 8724 Acked Write 3 Done Done Sent false
29406 8725 Acked Read 6 Done Done Sent false
29407 8726 Acked Write 8 Done Done Sent false
29408 8727 Acked Write 3 Done Done Sent false
29409 8728 Acked Read 3 Done Done Sent false
29410 8729 Acked Write 3 Done Done Sent false
29411 8730 Acked Write 6 Done Done Sent false
29412 8731 Acked Flush 0 Done Done Sent false
29413 8732 Acked Write 3 Done Done Sent false
29414 8733 Acked Read 10 Done Done Sent false
29415 8734 Acked Write 4 Done Done Sent false
29416 8735 Acked Read 2 Done Done Sent false
29417 8736 Acked Write 2 Done Done Sent false
29418 8737 Acked Read 7 Done Done Sent false
29419 8738 Acked Write 7 Done Done Sent false
29420 8739 Acked Read 2 Done Done Sent false
29421 8740 Acked Read 1 Done Done Sent false
29422 8741 Acked Write 9 Done Done Sent false
29423 8742 Acked Write 1 Done Done Sent false
29424 8743 Acked Write 4 Done Done Sent false
29425 8744 Acked Flush 0 Done Done Sent false
29426 8745 Acked Write 10 Done Done Sent false
29427 8746 Acked Read 1 Done Done Sent false
29428 8747 Acked Write 6 Done Done Sent false
29429 8748 Acked Flush 0 Done Done Sent false
29430 8749 Acked Read 5 Done Done Sent false
29431 8750 Acked Read 1 Done Done Sent false
29432 8751 Acked Write 4 Done Done Sent false
29433 8752 Acked Read 5 Done Done Sent false
29434 8753 Acked Read 4 Done Done Sent false
29435 8754 Acked Write 8 Done Done Sent false
29436 8755 Acked Write 5 Done Done Sent false
29437 8756 Acked Write 8 Done Done Sent false
29438 8757 Acked Flush 0 Done Done Sent false
29439 8758 Acked Read 9 Done Done Sent false
29440 8759 Acked Write 9 Done Done Sent false
29441 8760 Acked Read 7 Done Done Sent false
29442 8761 Acked Flush 0 Done Done Sent false
29443 8762 Acked Write 1 Done Done Sent false
29444 8763 Acked Write 6 Done Done Sent false
29445 8764 Acked Write 3 Done Done Sent false
29446 8765 Acked Write 4 Done Done Sent false
29447 8766 Acked Flush 0 Done Done Sent false
29448 8767 Acked Flush 0 Done Done Sent false
29449 8768 Acked Write 10 Done Done Sent false
29450 8769 Acked Write 5 Done Done Sent false
29451 8770 Acked Read 9 Done Done Sent false
29452 8771 Acked Write 4 Done Done Sent false
29453 8772 Acked Read 2 Done Done Sent false
29454 8773 Acked Read 4 Done Done Sent false
29455 8774 Acked Read 8 Done Done Sent false
29456 8775 Acked Write 9 Done Done Sent false
29457 8776 Acked Write 8 Done Done Sent false
29458 8777 Acked Write 1 Done Done Sent false
29459 8778 Acked Flush 0 Done Done Sent false
29460 8779 Acked Read 3 Done Done Sent false
29461 8780 Acked Flush 0 Done Done Sent false
29462 8781 Acked Write 8 Done Done Sent false
29463 8782 Acked Read 4 Done Done Sent false
29464 8783 Acked Read 1 Done Done Sent false
29465 8784 Acked Read 9 Done Done Sent false
29466 8785 Acked Read 10 Done Done Sent false
29467 8786 Acked Write 7 Done Done Sent false
29468 8787 Acked Read 7 Done Done Sent false
29469 8788 Acked Read 7 Done Done Sent false
29470 8789 Acked Write 6 Done Done Sent false
29471 8790 Acked Write 7 Done Done Sent false
29472 8791 Acked Read 1 Done Done Sent false
29473 8792 Acked Flush 0 Done Done Sent false
29474 8793 Acked Flush 0 Done Done Sent false
29475 8794 Acked Read 5 Done Done Sent false
29476 8795 Acked Write 10 Done Done Sent false
29477 8796 Acked Flush 0 Done Done Sent false
29478 8797 Acked Read 8 Done Done Sent false
29479 8798 Acked Read 8 Done Done Sent false
29480 8799 Acked Write 7 Done Done Sent false
29481 8800 Acked Read 5 Done Done Sent false
29482 8801 Acked Read 9 Done Done Sent false
29483 8802 Acked Read 9 Done Done Sent false
29484 8803 Acked Write 7 Done Done Sent false
29485 8804 Acked Read 8 Done Done Sent false
29486 8805 Acked Write 6 Done Done Sent false
29487 8806 Acked Read 1 Done Done Sent false
29488 8807 Acked Write 2 Done Done Sent false
29489 8808 Acked Read 1 Done Done Sent false
29490 8809 Acked Read 2 Done Done Sent false
29491 8810 Acked Read 7 Done Done Sent false
29492 8811 Acked Read 6 Done Done Sent false
29493 8812 Acked Read 1 Done Done Sent false
29494 8813 Acked Write 2 Done Done Sent false
29495 8814 Acked Read 7 Done Done Sent false
29496 8815 Acked Read 4 Done Done Sent false
29497 8816 Acked Flush 0 Done Done Sent false
29498 8817 Acked Read 9 Done Done Sent false
29499 8818 Acked Read 2 Done Done Sent false
29500 8819 Acked Read 9 Done Done Sent false
29501 8820 Acked Flush 0 Done Done Sent false
29502 8821 Acked Read 9 Done Done Sent false
29503 8822 Acked Write 10 Done Done Sent false
29504 8823 Acked Write 9 Done Done Sent false
29505 8824 Acked Write 6 Done Done Sent false
29506 8825 Acked Read 6 Done Done Sent false
29507 8826 Acked Read 10 Done Done Sent false
29508 8827 Acked Write 10 Done Done Sent false
29509 8828 Acked Read 5 Done Done Sent false
29510 8829 Acked Read 9 Done Done Sent false
29511 8830 Acked Read 4 Done Done Sent false
29512 8831 Acked Write 10 Done Done Sent false
29513 8832 Acked Write 7 Done Done Sent false
29514 8833 Acked Read 6 Done Done Sent false
29515 8834 Acked Write 10 Done Done Sent false
29516 8835 Acked Read 4 Done Done Sent false
29517 8836 Acked Read 5 Done Done Sent false
29518 8837 Acked Read 7 Done Done Sent false
29519 8838 Acked Read 10 Done Done Sent false
29520 8839 Acked Write 7 Done Done Sent false
29521 8840 Acked Write 7 Done Done Sent false
29522 8841 Acked Flush 0 Done Done Sent false
29523 8842 Acked Read 8 Done Done Sent false
29524 8843 Acked Write 8 Done Done Sent false
29525 8844 Acked Write 2 Done Done Sent false
29526 8845 Acked Read 8 Done Done Sent false
29527 8846 Acked Read 8 Done Done Sent false
29528 8847 Acked Flush 0 Done Done Sent false
29529 8848 Acked Write 5 Done Done Sent false
29530 STATES DS:0 DS:1 DS:2 TOTAL
29531 Sent 0 0 301 301
29532 Done 301 301 0 602
29533 Skipped 0 0 0 0
29534 Error 0 0 0 0
29535 Last Flush: 8847 8847 8547
29536 Downstairs last five completed: 8547 8546 8545 8544 8543
29537 Upstairs last five completed: 8848 8847 8846 8845 8844
29538 CLIENT: Up:0 ds:301 act:2
29539 [8830] got start action
29540 [8830] I am going to restart
29541 Starting downstairs at port 8830
29542 Make output file at "/tmp/dsc/downstairs-8830.txt"
29543 [8830][2] reports Starting
29544 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1228)
29545 [8830][2] reports Running
29546 ----------------------------------------------------------------
29547 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29548 States: Active Active Offline
29549 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29550 8548 Acked Read 9 Done Done Sent false
29551 8549 Acked Read 6 Done Done Sent false
29552 8550 Acked Read 3 Done Done Sent false
29553 8551 Acked Write 7 Done Done Sent false
29554 8552 Acked Write 2 Done Done Sent false
29555 8553 Acked Read 4 Done Done Sent false
29556 8554 Acked Write 5 Done Done Sent false
29557 8555 Acked Write 3 Done Done Sent false
29558 8556 Acked Read 10 Done Done Sent false
29559 8557 Acked Write 6 Done Done Sent false
29560 8558 Acked Read 4 Done Done Sent false
29561 8559 Acked Write 2 Done Done Sent false
29562 8560 Acked Read 10 Done Done Sent false
29563 8561 Acked Read 4 Done Done Sent false
29564 8562 Acked Write 7 Done Done Sent false
29565 8563 Acked Write 6 Done Done Sent false
29566 8564 Acked Read 2 Done Done Sent false
29567 8565 Acked Write 4 Done Done Sent false
29568 8566 Acked Flush 0 Done Done Sent false
29569 8567 Acked Read 9 Done Done Sent false
29570 8568 Acked Flush 0 Done Done Sent false
29571 8569 Acked Write 7 Done Done Sent false
29572 8570 Acked Write 6 Done Done Sent false
29573 8571 Acked Write 3 Done Done Sent false
29574 8572 Acked Write 10 Done Done Sent false
29575 8573 Acked Write 9 Done Done Sent false
29576 8574 Acked Read 3 Done Done Sent false
29577 8575 Acked Read 5 Done Done Sent false
29578 8576 Acked Read 7 Done Done Sent false
29579 8577 Acked Read 5 Done Done Sent false
29580 8578 Acked Write 3 Done Done Sent false
29581 8579 Acked Write 10 Done Done Sent false
29582 8580 Acked Read 9 Done Done Sent false
29583 8581 Acked Read 4 Done Done Sent false
29584 8582 Acked Read 4 Done Done Sent false
29585 8583 Acked Write 10 Done Done Sent false
29586 8584 Acked Flush 0 Done Done Sent false
29587 8585 Acked Read 4 Done Done Sent false
29588 8586 Acked Read 10 Done Done Sent false
29589 8587 Acked Write 3 Done Done Sent false
29590 8588 Acked Read 4 Done Done Sent false
29591 8589 Acked Read 1 Done Done Sent false
29592 8590 Acked Write 4 Done Done Sent false
29593 8591 Acked Read 10 Done Done Sent false
29594 8592 Acked Read 1 Done Done Sent false
29595 8593 Acked Read 7 Done Done Sent false
29596 8594 Acked Read 8 Done Done Sent false
29597 8595 Acked Write 9 Done Done Sent false
29598 8596 Acked Write 2 Done Done Sent false
29599 8597 Acked Read 3 Done Done Sent false
29600 8598 Acked Read 1 Done Done Sent false
29601 8599 Acked Read 2 Done Done Sent false
29602 8600 Acked Read 3 Done Done Sent false
29603 8601 Acked Read 10 Done Done Sent false
29604 8602 Acked Read 9 Done Done Sent false
29605 8603 Acked Write 3 Done Done Sent false
29606 8604 Acked Write 3 Done Done Sent false
29607 8605 Acked Read 6 Done Done Sent false
29608 8606 Acked Read 3 Done Done Sent false
29609 8607 Acked Read 5 Done Done Sent false
29610 8608 Acked Read 4 Done Done Sent false
29611 8609 Acked Write 8 Done Done Sent false
29612 8610 Acked Read 8 Done Done Sent false
29613 8611 Acked Write 7 Done Done Sent false
29614 8612 Acked Write 9 Done Done Sent false
29615 8613 Acked Read 5 Done Done Sent false
29616 8614 Acked Read 3 Done Done Sent false
29617 8615 Acked Write 5 Done Done Sent false
29618 8616 Acked Flush 0 Done Done Sent false
29619 8617 Acked Write 4 Done Done Sent false
29620 8618 Acked Write 3 Done Done Sent false
29621 8619 Acked Read 3 Done Done Sent false
29622 8620 Acked Write 9 Done Done Sent false
29623 8621 Acked Read 4 Done Done Sent false
29624 8622 Acked Write 9 Done Done Sent false
29625 8623 Acked Write 9 Done Done Sent false
29626 8624 Acked Write 3 Done Done Sent false
29627 8625 Acked Read 3 Done Done Sent false
29628 8626 Acked Write 5 Done Done Sent false
29629 8627 Acked Write 7 Done Done Sent false
29630 8628 Acked Flush 0 Done Done Sent false
29631 8629 Acked Write 2 Done Done Sent false
29632 8630 Acked Flush 0 Done Done Sent false
29633 8631 Acked Write 3 Done Done Sent false
29634 8632 Acked Write 7 Done Done Sent false
29635 8633 Acked Read 10 Done Done Sent false
29636 8634 Acked Write 7 Done Done Sent false
29637 8635 Acked Read 3 Done Done Sent false
29638 8636 Acked Read 9 Done Done Sent false
29639 8637 Acked Flush 0 Done Done Sent false
29640 8638 Acked Write 9 Done Done Sent false
29641 8639 Acked Write 7 Done Done Sent false
29642 8640 Acked Write 4 Done Done Sent false
29643 8641 Acked Read 9 Done Done Sent false
29644 8642 Acked Write 3 Done Done Sent false
29645 8643 Acked Write 2 Done Done Sent false
29646 8644 Acked Read 7 Done Done Sent false
29647 8645 Acked Flush 0 Done Done Sent false
29648 8646 Acked Read 1 Done Done Sent false
29649 8647 Acked Read 2 Done Done Sent false
29650 8648 Acked Read 3 Done Done Sent false
29651 8649 Acked Write 3 Done Done Sent false
29652 8650 Acked Read 3 Done Done Sent false
29653 8651 Acked Read 7 Done Done Sent false
29654 8652 Acked Write 4 Done Done Sent false
29655 8653 Acked Read 4 Done Done Sent false
29656 8654 Acked Write 4 Done Done Sent false
29657 8655 Acked Write 7 Done Done Sent false
29658 8656 Acked Write 9 Done Done Sent false
29659 8657 Acked Read 1 Done Done Sent false
29660 8658 Acked Flush 0 Done Done Sent false
29661 8659 Acked Read 4 Done Done Sent false
29662 8660 Acked Read 10 Done Done Sent false
29663 8661 Acked Read 2 Done Done Sent false
29664 8662 Acked Read 9 Done Done Sent false
29665 8663 Acked Read 2 Done Done Sent false
29666 8664 Acked Write 10 Done Done Sent false
29667 8665 Acked Read 8 Done Done Sent false
29668 8666 Acked Flush 0 Done Done Sent false
29669 8667 Acked Read 4 Done Done Sent false
29670 8668 Acked Read 10 Done Done Sent false
29671 8669 Acked Read 6 Done Done Sent false
29672 8670 Acked Read 7 Done Done Sent false
29673 8671 Acked Read 3 Done Done Sent false
29674 8672 Acked Write 6 Done Done Sent false
29675 8673 Acked Read 10 Done Done Sent false
29676 8674 Acked Read 7 Done Done Sent false
29677 8675 Acked Write 7 Done Done Sent false
29678 8676 Acked Read 8 Done Done Sent false
29679 8677 Acked Read 8 Done Done Sent false
29680 8678 Acked Read 10 Done Done Sent false
29681 8679 Acked Write 7 Done Done Sent false
29682 8680 Acked Read 10 Done Done Sent false
29683 8681 Acked Write 8 Done Done Sent false
29684 8682 Acked Write 7 Done Done Sent false
29685 8683 Acked Read 7 Done Done Sent false
29686 8684 Acked Write 4 Done Done Sent false
29687 8685 Acked Read 4 Done Done Sent false
29688 8686 Acked Write 3 Done Done Sent false
29689 8687 Acked Read 1 Done Done Sent false
29690 8688 Acked Write 8 Done Done Sent false
29691 8689 Acked Flush 0 Done Done Sent false
29692 8690 Acked Flush 0 Done Done Sent false
29693 8691 Acked Write 9 Done Done Sent false
29694 8692 Acked Write 8 Done Done Sent false
29695 8693 Acked Write 5 Done Done Sent false
29696 8694 Acked Write 6 Done Done Sent false
29697 8695 Acked Write 8 Done Done Sent false
29698 8696 Acked Read 8 Done Done Sent false
29699 8697 Acked Write 7 Done Done Sent false
29700 8698 Acked Write 6 Done Done Sent false
29701 8699 Acked Read 3 Done Done Sent false
29702 8700 Acked Read 10 Done Done Sent false
29703 8701 Acked Read 9 Done Done Sent false
29704 8702 Acked Write 2 Done Done Sent false
29705 8703 Acked Write 7 Done Done Sent false
29706 8704 Acked Read 9 Done Done Sent false
29707 8705 Acked Write 9 Done Done Sent false
29708 8706 Acked Read 9 Done Done Sent false
29709 8707 Acked Read 4 Done Done Sent false
29710 8708 Acked Read 9 Done Done Sent false
29711 8709 Acked Write 4 Done Done Sent false
29712 8710 Acked Write 9 Done Done Sent false
29713 8711 Acked Write 4 Done Done Sent false
29714 8712 Acked Read 3 Done Done Sent false
29715 8713 Acked Read 8 Done Done Sent false
29716 8714 Acked Read 5 Done Done Sent false
29717 8715 Acked Read 8 Done Done Sent false
29718 8716 Acked Read 7 Done Done Sent false
29719 8717 Acked Read 5 Done Done Sent false
29720 8718 Acked Write 2 Done Done Sent false
29721 8719 Acked Write 4 Done Done Sent false
29722 8720 Acked Write 4 Done Done Sent false
29723 8721 Acked Write 8 Done Done Sent false
29724 8722 Acked Write 6 Done Done Sent false
29725 8723 Acked Flush 0 Done Done Sent false
29726 8724 Acked Write 3 Done Done Sent false
29727 8725 Acked Read 6 Done Done Sent false
29728 8726 Acked Write 8 Done Done Sent false
29729 8727 Acked Write 3 Done Done Sent false
29730 8728 Acked Read 3 Done Done Sent false
29731 8729 Acked Write 3 Done Done Sent false
29732 8730 Acked Write 6 Done Done Sent false
29733 8731 Acked Flush 0 Done Done Sent false
29734 8732 Acked Write 3 Done Done Sent false
29735 8733 Acked Read 10 Done Done Sent false
29736 8734 Acked Write 4 Done Done Sent false
29737 8735 Acked Read 2 Done Done Sent false
29738 8736 Acked Write 2 Done Done Sent false
29739 8737 Acked Read 7 Done Done Sent false
29740 8738 Acked Write 7 Done Done Sent false
29741 8739 Acked Read 2 Done Done Sent false
29742 8740 Acked Read 1 Done Done Sent false
29743 8741 Acked Write 9 Done Done Sent false
29744 8742 Acked Write 1 Done Done Sent false
29745 8743 Acked Write 4 Done Done Sent false
29746 8744 Acked Flush 0 Done Done Sent false
29747 8745 Acked Write 10 Done Done Sent false
29748 8746 Acked Read 1 Done Done Sent false
29749 8747 Acked Write 6 Done Done Sent false
29750 8748 Acked Flush 0 Done Done Sent false
29751 8749 Acked Read 5 Done Done Sent false
29752 8750 Acked Read 1 Done Done Sent false
29753 8751 Acked Write 4 Done Done Sent false
29754 8752 Acked Read 5 Done Done Sent false
29755 8753 Acked Read 4 Done Done Sent false
29756 8754 Acked Write 8 Done Done Sent false
29757 8755 Acked Write 5 Done Done Sent false
29758 8756 Acked Write 8 Done Done Sent false
29759 8757 Acked Flush 0 Done Done Sent false
29760 8758 Acked Read 9 Done Done Sent false
29761 8759 Acked Write 9 Done Done Sent false
29762 8760 Acked Read 7 Done Done Sent false
29763 8761 Acked Flush 0 Done Done Sent false
29764 8762 Acked Write 1 Done Done Sent false
29765 8763 Acked Write 6 Done Done Sent false
29766 8764 Acked Write 3 Done Done Sent false
29767 8765 Acked Write 4 Done Done Sent false
29768 8766 Acked Flush 0 Done Done Sent false
29769 8767 Acked Flush 0 Done Done Sent false
29770 8768 Acked Write 10 Done Done Sent false
29771 8769 Acked Write 5 Done Done Sent false
29772 8770 Acked Read 9 Done Done Sent false
29773 8771 Acked Write 4 Done Done Sent false
29774 8772 Acked Read 2 Done Done Sent false
29775 8773 Acked Read 4 Done Done Sent false
29776 8774 Acked Read 8 Done Done Sent false
29777 8775 Acked Write 9 Done Done Sent false
29778 8776 Acked Write 8 Done Done Sent false
29779 8777 Acked Write 1 Done Done Sent false
29780 8778 Acked Flush 0 Done Done Sent false
29781 8779 Acked Read 3 Done Done Sent false
29782 8780 Acked Flush 0 Done Done Sent false
29783 8781 Acked Write 8 Done Done Sent false
29784 8782 Acked Read 4 Done Done Sent false
29785 8783 Acked Read 1 Done Done Sent false
29786 8784 Acked Read 9 Done Done Sent false
29787 8785 Acked Read 10 Done Done Sent false
29788 8786 Acked Write 7 Done Done Sent false
29789 8787 Acked Read 7 Done Done Sent false
29790 8788 Acked Read 7 Done Done Sent false
29791 8789 Acked Write 6 Done Done Sent false
29792 8790 Acked Write 7 Done Done Sent false
29793 8791 Acked Read 1 Done Done Sent false
29794 8792 Acked Flush 0 Done Done Sent false
29795 8793 Acked Flush 0 Done Done Sent false
29796 8794 Acked Read 5 Done Done Sent false
29797 8795 Acked Write 10 Done Done Sent false
29798 8796 Acked Flush 0 Done Done Sent false
29799 8797 Acked Read 8 Done Done Sent false
29800 8798 Acked Read 8 Done Done Sent false
29801 8799 Acked Write 7 Done Done Sent false
29802 8800 Acked Read 5 Done Done Sent false
29803 8801 Acked Read 9 Done Done Sent false
29804 8802 Acked Read 9 Done Done Sent false
29805 8803 Acked Write 7 Done Done Sent false
29806 8804 Acked Read 8 Done Done Sent false
29807 8805 Acked Write 6 Done Done Sent false
29808 8806 Acked Read 1 Done Done Sent false
29809 8807 Acked Write 2 Done Done Sent false
29810 8808 Acked Read 1 Done Done Sent false
29811 8809 Acked Read 2 Done Done Sent false
29812 8810 Acked Read 7 Done Done Sent false
29813 8811 Acked Read 6 Done Done Sent false
29814 8812 Acked Read 1 Done Done Sent false
29815 8813 Acked Write 2 Done Done Sent false
29816 8814 Acked Read 7 Done Done Sent false
29817 8815 Acked Read 4 Done Done Sent false
29818 8816 Acked Flush 0 Done Done Sent false
29819 8817 Acked Read 9 Done Done Sent false
29820 8818 Acked Read 2 Done Done Sent false
29821 8819 Acked Read 9 Done Done Sent false
29822 8820 Acked Flush 0 Done Done Sent false
29823 8821 Acked Read 9 Done Done Sent false
29824 8822 Acked Write 10 Done Done Sent false
29825 8823 Acked Write 9 Done Done Sent false
29826 8824 Acked Write 6 Done Done Sent false
29827 8825 Acked Read 6 Done Done Sent false
29828 8826 Acked Read 10 Done Done Sent false
29829 8827 Acked Write 10 Done Done Sent false
29830 8828 Acked Read 5 Done Done Sent false
29831 8829 Acked Read 9 Done Done Sent false
29832 8830 Acked Read 4 Done Done Sent false
29833 8831 Acked Write 10 Done Done Sent false
29834 8832 Acked Write 7 Done Done Sent false
29835 8833 Acked Read 6 Done Done Sent false
29836 8834 Acked Write 10 Done Done Sent false
29837 8835 Acked Read 4 Done Done Sent false
29838 8836 Acked Read 5 Done Done Sent false
29839 8837 Acked Read 7 Done Done Sent false
29840 8838 Acked Read 10 Done Done Sent false
29841 8839 Acked Write 7 Done Done Sent false
29842 8840 Acked Write 7 Done Done Sent false
29843 8841 Acked Flush 0 Done Done Sent false
29844 8842 Acked Read 8 Done Done Sent false
29845 8843 Acked Write 8 Done Done Sent false
29846 8844 Acked Write 2 Done Done Sent false
29847 8845 Acked Read 8 Done Done Sent false
29848 8846 Acked Read 8 Done Done Sent false
29849 8847 Acked Flush 0 Done Done Sent false
29850 8848 Acked Write 5 Done Done Sent false
29851 8849 Acked Flush 0 Done Done Sent false
29852 STATES DS:0 DS:1 DS:2 TOTAL
29853 Sent 0 0 302 302
29854 Done 302 302 0 604
29855 Skipped 0 0 0 0
29856 Error 0 0 0 0
29857 Last Flush: 8849 8849 8547
29858 Downstairs last five completed: 8547 8546 8545 8544 8543
29859 Upstairs last five completed: 8849 8848 8847 8846 8845
29860 CLIENT: Up:0 ds:302 act:2
298612025-01-15T05:25:15.026ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298622025-01-15T05:25:15.026ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42522) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298632025-01-15T05:25:15.027ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298642025-01-15T05:25:15.028ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298652025-01-15T05:25:15.028ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298662025-01-15T05:25:15.028ZINFOcrucible: send last flush ID to this DS: 8547 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298672025-01-15T05:25:15.028ZINFOcrucible: Replied this last flush ID: 8547 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298682025-01-15T05:25:15.028ZINFOcrucible: [2] client re-new 302 jobs since flush 8547 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298692025-01-15T05:25:15.028ZINFOcrucible: [2] Replayed 302 jobs since flush: 8547 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
29870 ----------------------------------------------------------------
29871 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29872 Downstairs last five completed: 8849 8848 8847 8846 8845
29873 Upstairs last five completed: 8849 8848 8847 8846 8845
29874 CLIENT: Up:0 ds:0 act:3
29875 Replay: All jobs finished
29876 Main task has work to do, go find it
29877 got dsc Stop(0)
29878 stop 0
29879 Jan 15 05:25:17.273 INFO request completed, latency_us: 338, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 19cc72a4-4be6-44ef-a698-ec71ef433733, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
29880 [8810] Got stop action so:false kr:false
29881 Jan 15 05:25:17.274 INFO request completed, latency_us: 106, response_code: 200, uri: /state/cid/0, method: GET, req_id: fdb00eb8-1da5-446b-a86c-763bd24a48da, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
298822025-01-15T05:25:17.274ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298832025-01-15T05:25:17.274ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298842025-01-15T05:25:17.274ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
298852025-01-15T05:25:17.274ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
29886 [8810] Exited with: ExitStatus(unix_wait_status(9))
29887 [8810][0] reports Exit
29888 Jan 15 05:25:21.275 INFO request completed, latency_us: 189, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8ea20be7-74fe-4cca-91a7-ac8db237ea2e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
29889 001/300 Read block 3842 len 24576
29890 002/300 Read block 1752 len 16384
29891 003/300 Read block 784 len 24576
29892 004/300 Write block 4685 len 4096 data: 7
29893 005/300 Read block 4869 len 28672
29894 006/300 Read block 501 len 32768
29895 007/300 Write block 1698 len 40960 data: 5 5 7 7 6 6 7 6 4 4
29896 008/300 Write block 3953 len 24576 data: 3 5 5 5 4 4
29897 009/300 Flush
29898 010/300 Read block 337 len 40960
29899 011/300 Read block 1562 len 32768
29900 012/300 Read block 4035 len 32768
29901 013/300 Write block 1819 len 4096 data: 3
29902 014/300 Read block 256 len 8192
29903 015/300 Write block 373 len 16384 data: 6 6 7 7
29904 016/300 Read block 3216 len 24576
29905 017/300 Flush
29906 018/300 Flush
29907 019/300 Read block 4929 len 12288
29908 020/300 Read block 4724 len 40960
29909 021/300 Write block 3680 len 28672 data: 10 9 9 7 6 6 6
29910 022/300 Write block 1129 len 4096 data: 6
29911 023/300 Read block 828 len 4096
29912 024/300 Write block 2422 len 28672 data: 6 5 5 6 6 7 8
29913 025/300 Read block 210 len 36864
29914 026/300 Read block 4014 len 40960
29915 027/300 Write block 4122 len 40960 data: 7 4 4 6 8 8 7 6 7 9
29916 028/300 Read block 4627 len 40960
29917 029/300 Write block 102 len 32768 data: 5 5 6 5 5 4 4 4
29918 030/300 Write block 1636 len 12288 data: 8 9 11
29919 031/300 Write block 2701 len 20480 data: 5 4 6 5 4
29920 032/300 Write block 844 len 20480 data: 4 3 2 3 3
29921 033/300 Write block 1194 len 36864 data: 4 3 4 4 4 4 5 6 7
29922 034/300 Write block 269 len 24576 data: 7 7 7 6 4 5
29923 035/300 Read block 1231 len 32768
29924 036/300 Write block 3334 len 36864 data: 7 7 9 9 9 8 7 6 5
29925 037/300 Write block 897 len 32768 data: 7 5 5 4 5 5 5 5
29926 038/300 Flush
29927 039/300 Write block 4650 len 28672 data: 3 6 5 5 4 3 5
29928 040/300 Read block 2095 len 12288
29929 041/300 Flush
29930 042/300 Write block 4696 len 32768 data: 5 4 4 3 5 4 4 4
29931 043/300 Read block 2043 len 36864
29932 044/300 Read block 4834 len 24576
29933 045/300 Write block 3240 len 20480 data: 3 4 5 8 6
29934 046/300 Write block 3650 len 28672 data: 8 9 6 6 5 5 7
29935 047/300 Read block 1679 len 40960
29936 048/300 Flush
29937 049/300 Read block 4864 len 12288
29938 050/300 Read block 4800 len 8192
29939 051/300 Read block 2368 len 4096
29940 052/300 Write block 1881 len 24576 data: 5 5 5 5 5 5
29941 053/300 Write block 4196 len 16384 data: 7 6 5 3
29942 054/300 Read block 3925 len 4096
29943 055/300 Write block 575 len 20480 data: 6 7 8 8 8
29944 056/300 Flush
29945 057/300 Write block 1985 len 24576 data: 4 3 2 3 3 3
29946 058/300 Write block 599 len 40960 data: 4 5 5 5 3 5 5 5 5 4
29947 059/300 Read block 3511 len 8192
29948 060/300 Read block 2837 len 4096
29949 061/300 Read block 3691 len 8192
29950 062/300 Write block 1651 len 20480 data: 5 5 5 4 5
29951 063/300 Read block 1494 len 16384
29952 064/300 Read block 3542 len 16384
29953 065/300 Write block 538 len 24576 data: 6 6 6 5 6 5
29954 066/300 Read block 404 len 28672
29955 067/300 Write block 1674 len 20480 data: 6 6 7 6 9
29956 068/300 Read block 840 len 8192
29957 069/300 Write block 292 len 40960 data: 7 7 7 7 7 5 5 4 4 4
29958 070/300 Write block 1635 len 36864 data: 8 9 10 12 10 10 10 7 7
29959 071/300 Read block 1636 len 20480
29960 072/300 Read block 4911 len 8192
29961 073/300 Read block 1635 len 16384
29962 074/300 Write block 1451 len 24576 data: 4 4 5 6 7 8
29963 075/300 Write block 692 len 32768 data: 8 7 7 7 8 8 6 4
29964 076/300 Write block 3941 len 16384 data: 8 5 8 9
29965 077/300 Write block 879 len 32768 data: 6 6 5 5 5 5 4 3
29966 078/300 Read block 3908 len 8192
29967 079/300 Read block 681 len 12288
29968 080/300 Read block 736 len 4096
29969 081/300 Write block 2879 len 32768 data: 7 8 8 8 7 7 7 6
29970 082/300 Flush
29971 083/300 Read block 2838 len 32768
29972 084/300 Read block 2906 len 28672
29973 085/300 Read block 3428 len 12288
29974 086/300 Read block 2866 len 12288
29975 087/300 Read block 765 len 28672
29976 088/300 Write block 1375 len 28672 data: 6 4 4 4 4 6 6
29977 089/300 Read block 3761 len 24576
29978 090/300 Write block 2631 len 40960 data: 5 5 5 5 6 7 8 7 8 8
29979 091/300 Write block 4120 len 4096 data: 10
29980 092/300 Read block 3101 len 24576
29981 093/300 Read block 881 len 24576
29982 094/300 Read block 3860 len 32768
29983 095/300 Write block 626 len 8192 data: 5 6
29984 096/300 Read block 4281 len 20480
29985 097/300 Read block 1268 len 16384
29986 098/300 Read block 1499 len 24576
29987 099/300 Write block 935 len 16384 data: 5 5 7 5
29988 100/300 Read block 254 len 28672
29989 101/300 Read block 4566 len 12288
29990 102/300 Write block 4776 len 24576 data: 4 6 6 5 6 6
29991 103/300 Write block 167 len 28672 data: 7 3 3 3 5 6 6
29992 104/300 Write block 471 len 24576 data: 8 8 7 5 4 5
29993 105/300 Read block 3027 len 8192
29994 106/300 Write block 2734 len 28672 data: 5 6 5 5 4 4 3
29995 107/300 Flush
29996 108/300 Write block 3442 len 12288 data: 8 7 6
29997 109/300 Write block 391 len 8192 data: 3 4
29998 110/300 Read block 1811 len 36864
29999 111/300 Write block 4506 len 24576 data: 3 3 4 4 4 4
30000 112/300 Read block 2205 len 20480
30001 113/300 Read block 2744 len 24576
30002 114/300 Write block 1260 len 36864 data: 6 7 6 4 4 6 6 8 7
30003 115/300 Read block 3908 len 20480
30004 116/300 Read block 2843 len 20480
30005 117/300 Read block 4799 len 24576
30006 118/300 Read block 1764 len 20480
30007 119/300 Write block 3370 len 16384 data: 9 7 6 8
30008 120/300 Flush
30009 121/300 Read block 203 len 28672
30010 122/300 Read block 1528 len 32768
30011 123/300 Write block 1254 len 20480 data: 6 7 6 6 5
30012 124/300 Write block 173 len 8192 data: 7 7
30013 125/300 Read block 900 len 12288
30014 126/300 Write block 2983 len 28672 data: 7 9 9 9 6 3 4
30015 127/300 Read block 3503 len 36864
30016 128/300 Read block 4514 len 4096
30017 129/300 Read block 2099 len 24576
30018 130/300 Flush
30019 131/300 Write block 1241 len 12288 data: 2 3 3
30020 132/300 Write block 1391 len 20480 data: 4 3 3 3 3
30021 133/300 Read block 3928 len 40960
30022 134/300 Write block 4086 len 40960 data: 4 4 3 3 3 4 4 5 5 5
30023 135/300 Write block 2336 len 16384 data: 5 6 7 6
30024 136/300 Read block 1967 len 20480
30025 137/300 Read block 2365 len 28672
30026 138/300 Read block 1229 len 24576
30027 139/300 Write block 386 len 24576 data: 3 3 2 2 3 4
30028 140/300 Read block 4598 len 40960
30029 141/300 Write block 3807 len 40960 data: 4 4 5 6 5 6 5 6 8 10
30030 142/300 Write block 1712 len 36864 data: 5 6 6 7 6 5 5 5 4
30031 143/300 Read block 1464 len 8192
30032 144/300 Read block 3116 len 40960
30033 145/300 Read block 37 len 36864
30034 146/300 Flush
30035 147/300 Read block 61 len 36864
30036 148/300 Read block 4031 len 20480
30037 149/300 Write block 1593 len 40960 data: 4 3 5 6 6 6 5 5 5 6
30038 150/300 Read block 1809 len 32768
30039 151/300 Read block 580 len 40960
30040 152/300 Write block 4154 len 32768 data: 8 9 10 8 7 7 5 4
30041 153/300 Write block 2804 len 24576 data: 5 5 3 4 5 3
30042 154/300 Write block 3689 len 8192 data: 5 5
30043 155/300 Read block 4853 len 36864
30044 156/300 Read block 3897 len 36864
30045 157/300 Write block 2067 len 8192 data: 5 5
30046 158/300 Read block 4137 len 40960
30047 159/300 Read block 2001 len 8192
30048 160/300 Write block 909 len 32768 data: 4 5 5 5 3 3 3 2
30049 161/300 Write block 3681 len 20480 data: 10 10 8 7 7
30050 162/300 Read block 3976 len 32768
30051 163/300 Write block 1881 len 16384 data: 6 6 6 6
30052 164/300 Read block 3654 len 8192
30053 165/300 Read block 1540 len 28672
30054 166/300 Read block 3430 len 8192
30055 167/300 Flush
30056 168/300 Write block 2939 len 36864 data: 4 4 6 5 5 5 7 8 6
30057 169/300 Write block 4218 len 16384 data: 8 6 4 4
30058 170/300 Read block 1307 len 24576
30059 171/300 Read block 1253 len 16384
30060 172/300 Flush
30061 173/300 Read block 3426 len 12288
30062 174/300 Read block 2212 len 20480
30063 175/300 Write block 515 len 20480 data: 8 6 5 6 7
30064 176/300 Read block 937 len 4096
30065 177/300 Write block 3058 len 16384 data: 7 5 6 5
30066 178/300 Write block 3835 len 4096 data: 9
30067 179/300 Read block 3496 len 16384
30068 180/300 Read block 2094 len 12288
30069 181/300 Read block 2997 len 16384
30070 182/300 Write block 3680 len 40960 data: 11 11 11 9 8 8 7 6 5 6
30071 183/300 Write block 4343 len 8192 data: 3 6
30072 184/300 Read block 4967 len 28672
30073 185/300 Read block 3853 len 20480
30074 186/300 Read block 2582 len 24576
30075 187/300 Flush
30076 188/300 Write block 720 len 16384 data: 7 6 6 5
30077 189/300 Read block 145 len 28672
30078 190/300 Read block 2583 len 40960
30079 191/300 Read block 1903 len 40960
30080 192/300 Read block 4555 len 12288
30081 193/300 Write block 2414 len 36864 data: 7 6 4 4 4 4 7 5 7
30082 194/300 Read block 1138 len 36864
30083 195/300 Write block 2588 len 20480 data: 8 7 9 8 9
30084 196/300 Flush
30085 197/300 Read block 2877 len 36864
30086 198/300 Flush
30087 199/300 Write block 4866 len 24576 data: 5 5 6 9 8 6
30088 200/300 Read block 2954 len 16384
30089 201/300 Write block 3579 len 40960 data: 5 4 4 6 7 7 4 4 3 3
30090 202/300 Read block 2520 len 24576
30091 203/300 Read block 2428 len 16384
30092 204/300 Read block 4894 len 8192
30093 205/300 Read block 2256 len 24576
30094 206/300 Read block 869 len 8192
30095 207/300 Write block 723 len 36864 data: 6 5 4 6 6 5 6 7 7
30096 208/300 Write block 4780 len 24576 data: 7 7 5 6 5 5
30097 209/300 Write block 3128 len 8192 data: 8 6
30098 210/300 Read block 4993 len 16384
30099 211/300 Read block 1846 len 24576
30100 212/300 Flush
30101 213/300 Read block 3455 len 20480
30102 214/300 Read block 4330 len 8192
30103 215/300 Write block 3287 len 28672 data: 6 6 4 4 4 4 5
30104 216/300 Write block 1698 len 16384 data: 6 6 8 8
30105 217/300 Read block 1560 len 12288
30106 218/300 Read block 4100 len 28672
30107 219/300 Read block 2017 len 4096
30108 220/300 Write block 3132 len 40960 data: 7 6 6 6 7 8 6 7 6 5
30109 221/300 Flush
30110 222/300 Write block 2101 len 28672 data: 4 4 4 4 5 5 6
30111 223/300 Write block 2010 len 20480 data: 6 6 5 6 9
30112 224/300 Write block 109 len 28672 data: 5 4 5 5 4 4 6
30113 225/300 Read block 848 len 8192
30114 226/300 Read block 2617 len 36864
30115 227/300 Flush
30116 228/300 Read block 385 len 40960
30117 229/300 Read block 2548 len 32768
30118 230/300 Read block 4185 len 16384
30119 231/300 Read block 3313 len 36864
30120 232/300 Read block 2443 len 28672
30121 233/300 Read block 3041 len 28672
30122 234/300 Write block 1012 len 36864 data: 5 7 6 6 6 5 4 3 3
30123 235/300 Flush
30124 236/300 Flush
30125 237/300 Flush
30126 238/300 Read block 92 len 12288
30127 239/300 Read block 74 len 36864
30128 240/300 Read block 470 len 20480
30129 241/300 Read block 3463 len 4096
30130 242/300 Read block 1547 len 32768
30131 243/300 Read block 2416 len 4096
30132 244/300 Flush
30133 245/300 Read block 3592 len 36864
30134 246/300 Read block 1748 len 32768
30135 247/300 Read block 3379 len 12288
30136 248/300 Write block 2558 len 12288 data: 5 5 4
30137 249/300 Read block 99 len 36864
30138 250/300 Write block 1901 len 20480 data: 6 7 6 7 6
30139 251/300 Read block 1189 len 4096
30140 252/300 Write block 1708 len 20480 data: 4 4 4 4 6
30141 253/300 Read block 3512 len 40960
30142 254/300 Read block 1250 len 12288
30143 255/300 Read block 3924 len 40960
30144 256/300 Read block 2792 len 4096
30145 257/300 Flush
30146 258/300 Write block 1154 len 8192 data: 9 7
30147 259/300 Read block 4555 len 32768
30148 260/300 Write block 4734 len 24576 data: 7 5 5 5 4 5
30149 261/300 Flush
30150 262/300 Read block 4080 len 40960
30151 263/300 Read block 1799 len 24576
30152 264/300 Flush
30153 265/300 Read block 4510 len 16384
30154 266/300 Write block 1685 len 24576 data: 5 4 4 3 3 3
30155 267/300 Flush
30156 268/300 Write block 4852 len 32768 data: 3 4 3 3 3 4 3 3
30157 269/300 Write block 2362 len 12288 data: 7 7 6
30158 270/300 Read block 1353 len 24576
30159 271/300 Write block 3953 len 4096 data: 4
30160 272/300 Flush
30161 273/300 Read block 1118 len 36864
30162 274/300 Read block 47 len 8192
30163 275/300 Read block 2816 len 20480
30164 276/300 Write block 2364 len 32768 data: 7 7 9 7 9 10 9 7
30165 277/300 Write block 1079 len 8192 data: 6 8
30166 278/300 Read block 548 len 16384
30167 279/300 Write block 72 len 28672 data: 6 4 5 7 7 7 8
30168 280/300 Flush
30169 281/300 Read block 3233 len 32768
30170 282/300 Write block 973 len 40960 data: 4 5 6 5 4 5 5 5 3 3
30171 283/300 Write block 880 len 20480 data: 7 6 6 6 6
30172 284/300 Write block 865 len 20480 data: 6 7 8 8 10
30173 285/300 Write block 134 len 32768 data: 7 5 4 4 5 5 4 3
30174 286/300 Read block 2044 len 24576
30175 287/300 Write block 1935 len 28672 data: 4 3 3 3 3 4 4
30176 288/300 Write block 3200 len 28672 data: 6 6 6 5 4 4 5
30177 289/300 Write block 842 len 24576 data: 5 5 5 4 3 4
30178 290/300 Write block 4578 len 32768 data: 7 6 7 8 5 5 5 6
30179 291/300 Read block 32 len 24576
30180 292/300 Read block 2669 len 12288
30181 293/300 Write block 1599 len 12288 data: 6 6 6
30182 294/300 Write block 3524 len 36864 data: 7 6 5 5 7 6 6 5 7
30183 295/300 Read block 1147 len 24576
30184 296/300 Write block 2164 len 20480 data: 6 4 5 6 6
30185 297/300 Read block 2729 len 20480
30186 298/300 Write block 781 len 36864 data: 2 3 3 3 2 4 4 4 6
30187 299/300 Read block 4385 len 36864
30188 300/300 Read block 2558 len 12288
30189 301/300 Write block 3431 len 16384 data: 6 6 6 5
30190 Main task has work to do, go find it
30191 got dsc Start(0)
30192 start 0
30193 Jan 15 05:25:21.518 INFO request completed, latency_us: 241, response_code: 204, uri: /start/cid/0, method: POST, req_id: 78638b2f-03be-4a46-9a89-b62265ce9871, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
30194 Replay: started 0, returned:Ok(())
30195 ----------------------------------------------------------------
30196 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30197 States: Offline Active Active
30198 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30199 8850 Acked Read 6 Sent Done Done false
30200 8851 Acked Read 4 Sent Done Done false
30201 8852 Acked Read 6 Sent Done Done false
30202 8853 Acked Write 1 Sent Done Done false
30203 8854 Acked Read 7 Sent Done Done false
30204 8855 Acked Read 8 Sent Done Done false
30205 8856 Acked Write 10 Sent Done Done false
30206 8857 Acked Write 6 Sent Done Done false
30207 8858 Acked Flush 0 Sent Done Done false
30208 8859 Acked Read 10 Sent Done Done false
30209 8860 Acked Read 8 Sent Done Done false
30210 8861 Acked Read 8 Sent Done Done false
30211 8862 Acked Write 1 Sent Done Done false
30212 8863 Acked Read 2 Sent Done Done false
30213 8864 Acked Write 4 Sent Done Done false
30214 8865 Acked Read 6 Sent Done Done false
30215 8866 Acked Flush 0 Sent Done Done false
30216 8867 Acked Flush 0 Sent Done Done false
30217 8868 Acked Read 3 Sent Done Done false
30218 8869 Acked Read 10 Sent Done Done false
30219 8870 Acked Write 7 Sent Done Done false
30220 8871 Acked Write 1 Sent Done Done false
30221 8872 Acked Read 1 Sent Done Done false
30222 8873 Acked Write 7 Sent Done Done false
30223 8874 Acked Read 9 Sent Done Done false
30224 8875 Acked Read 10 Sent Done Done false
30225 8876 Acked Write 10 Sent Done Done false
30226 8877 Acked Read 10 Sent Done Done false
30227 8878 Acked Write 8 Sent Done Done false
30228 8879 Acked Write 3 Sent Done Done false
30229 8880 Acked Write 5 Sent Done Done false
30230 8881 Acked Write 5 Sent Done Done false
30231 8882 Acked Write 9 Sent Done Done false
30232 8883 Acked Write 6 Sent Done Done false
30233 8884 Acked Read 8 Sent Done Done false
30234 8885 Acked Write 9 Sent Done Done false
30235 8886 Acked Write 8 Sent Done Done false
30236 8887 Acked Flush 0 Sent Done Done false
30237 8888 Acked Write 7 Sent Done Done false
30238 8889 Acked Read 3 Sent Done Done false
30239 8890 Acked Flush 0 Sent Done Done false
30240 8891 Acked Write 8 Sent Done Done false
30241 8892 Acked Read 9 Sent Done Done false
30242 8893 Acked Read 6 Sent Done Done false
30243 8894 Acked Write 5 Sent Done Done false
30244 8895 Acked Write 7 Sent Done Done false
30245 8896 Acked Read 10 Sent Done Done false
30246 8897 Acked Flush 0 Sent Done Done false
30247 8898 Acked Read 3 Sent Done Done false
30248 8899 Acked Read 2 Sent Done Done false
30249 8900 Acked Read 1 Sent Done Done false
30250 8901 Acked Write 6 Sent Done Done false
30251 8902 Acked Write 4 Sent Done Done false
30252 8903 Acked Read 1 Sent Done Done false
30253 8904 Acked Write 5 Sent Done Done false
30254 8905 Acked Flush 0 Sent Done Done false
30255 8906 Acked Write 6 Sent Done Done false
30256 8907 Acked Write 10 Sent Done Done false
30257 8908 Acked Read 2 Sent Done Done false
30258 8909 Acked Read 1 Sent Done Done false
30259 8910 Acked Read 2 Sent Done Done false
30260 8911 Acked Write 5 Sent Done Done false
30261 8912 Acked Read 4 Sent Done Done false
30262 8913 Acked Read 4 Sent Done Done false
30263 8914 Acked Write 6 Sent Done Done false
30264 8915 Acked Read 7 Sent Done Done false
30265 8916 Acked Write 5 Sent Done Done false
30266 8917 Acked Read 2 Sent Done Done false
30267 8918 Acked Write 10 Sent Done Done false
30268 8919 Acked Write 9 Sent Done Done false
30269 8920 Acked Read 5 Sent Done Done false
30270 8921 Acked Read 2 Sent Done Done false
30271 8922 Acked Read 4 Sent Done Done false
30272 8923 Acked Write 6 Sent Done Done false
30273 8924 Acked Write 8 Sent Done Done false
30274 8925 Acked Write 4 Sent Done Done false
30275 8926 Acked Write 8 Sent Done Done false
30276 8927 Acked Read 2 Sent Done Done false
30277 8928 Acked Read 3 Sent Done Done false
30278 8929 Acked Read 1 Sent Done Done false
30279 8930 Acked Write 8 Sent Done Done false
30280 8931 Acked Flush 0 Sent Done Done false
30281 8932 Acked Read 8 Sent Done Done false
30282 8933 Acked Read 7 Sent Done Done false
30283 8934 Acked Read 3 Sent Done Done false
30284 8935 Acked Read 3 Sent Done Done false
30285 8936 Acked Read 7 Sent Done Done false
30286 8937 Acked Write 7 Sent Done Done false
30287 8938 Acked Read 6 Sent Done Done false
30288 8939 Acked Write 10 Sent Done Done false
30289 8940 Acked Write 1 Sent Done Done false
30290 8941 Acked Read 6 Sent Done Done false
30291 8942 Acked Read 6 Sent Done Done false
30292 8943 Acked Read 8 Sent Done Done false
30293 8944 Acked Write 2 Sent Done Done false
30294 8945 Acked Read 5 Sent Done Done false
30295 8946 Acked Read 4 Sent Done Done false
30296 8947 Acked Read 6 Sent Done Done false
30297 8948 Acked Write 4 Sent Done Done false
30298 8949 Acked Read 7 Sent Done Done false
30299 8950 Acked Read 3 Sent Done Done false
30300 8951 Acked Write 6 Sent Done Done false
30301 8952 Acked Write 7 Sent Done Done false
30302 8953 Acked Write 6 Sent Done Done false
30303 8954 Acked Read 2 Sent Done Done false
30304 8955 Acked Write 7 Sent Done Done false
30305 8956 Acked Flush 0 Sent Done Done false
30306 8957 Acked Write 3 Sent Done Done false
30307 8958 Acked Write 2 Sent Done Done false
30308 8959 Acked Read 9 Sent Done Done false
30309 8960 Acked Write 6 Sent Done Done false
30310 8961 Acked Read 5 Sent Done Done false
30311 8962 Acked Read 6 Sent Done Done false
30312 8963 Acked Write 9 Sent Done Done false
30313 8964 Acked Read 5 Sent Done Done false
30314 8965 Acked Read 5 Sent Done Done false
30315 8966 Acked Read 6 Sent Done Done false
30316 8967 Acked Read 5 Sent Done Done false
30317 8968 Acked Write 4 Sent Done Done false
30318 8969 Acked Flush 0 Sent Done Done false
30319 8970 Acked Read 7 Sent Done Done false
30320 8971 Acked Read 8 Sent Done Done false
30321 8972 Acked Write 5 Sent Done Done false
30322 8973 Acked Write 2 Sent Done Done false
30323 8974 Acked Read 3 Sent Done Done false
30324 8975 Acked Write 7 Sent Done Done false
30325 8976 Acked Read 9 Sent Done Done false
30326 8977 Acked Read 1 Sent Done Done false
30327 8978 Acked Read 6 Sent Done Done false
30328 8979 Acked Flush 0 Sent Done Done false
30329 8980 Acked Write 3 Sent Done Done false
30330 8981 Acked Write 5 Sent Done Done false
30331 8982 Acked Read 10 Sent Done Done false
30332 8983 Acked Write 10 Sent Done Done false
30333 8984 Acked Write 4 Sent Done Done false
30334 8985 Acked Read 5 Sent Done Done false
30335 8986 Acked Read 7 Sent Done Done false
30336 8987 Acked Read 6 Sent Done Done false
30337 8988 Acked Write 6 Sent Done Done false
30338 8989 Acked Read 10 Sent Done Done false
30339 8990 Acked Write 10 Sent Done Done false
30340 8991 Acked Write 9 Sent Done Done false
30341 8992 Acked Read 2 Sent Done Done false
30342 8993 Acked Read 10 Sent Done Done false
30343 8994 Acked Read 9 Sent Done Done false
30344 8995 Acked Flush 0 Sent Done Done false
30345 8996 Acked Read 9 Sent Done Done false
30346 8997 Acked Read 5 Sent Done Done false
30347 8998 Acked Write 10 Sent Done Done false
30348 8999 Acked Read 8 Sent Done Done false
30349 9000 Acked Read 10 Sent Done Done false
30350 9001 Acked Write 8 Sent Done Done false
30351 9002 Acked Write 6 Sent Done Done false
30352 9003 Acked Write 2 Sent Done Done false
30353 9004 Acked Read 9 Sent Done Done false
30354 9005 Acked Read 9 Sent Done Done false
30355 9006 Acked Write 2 Sent Done Done false
30356 9007 Acked Read 10 Sent Done Done false
30357 9008 Acked Read 2 Sent Done Done false
30358 9009 Acked Write 8 Sent Done Done false
30359 9010 Acked Write 5 Sent Done Done false
30360 9011 Acked Read 8 Sent Done Done false
30361 9012 Acked Write 4 Sent Done Done false
30362 9013 Acked Read 2 Sent Done Done false
30363 9014 Acked Read 7 Sent Done Done false
30364 9015 Acked Read 2 Sent Done Done false
30365 9016 Acked Flush 0 Sent Done Done false
30366 9017 Acked Write 9 Sent Done Done false
30367 9018 Acked Write 4 Sent Done Done false
30368 9019 Acked Read 6 Sent Done Done false
30369 9020 Acked Read 4 Sent Done Done false
30370 9021 Acked Flush 0 Sent Done Done false
30371 9022 Acked Read 3 Sent Done Done false
30372 9023 Acked Read 5 Sent Done Done false
30373 9024 Acked Write 5 Sent Done Done false
30374 9025 Acked Read 1 Sent Done Done false
30375 9026 Acked Write 4 Sent Done Done false
30376 9027 Acked Write 1 Sent Done Done false
30377 9028 Acked Read 4 Sent Done Done false
30378 9029 Acked Read 3 Sent Done Done false
30379 9030 Acked Read 4 Sent Done Done false
30380 9031 Acked Write 10 Sent Done Done false
30381 9032 Acked Write 2 Sent Done Done false
30382 9033 Acked Read 7 Sent Done Done false
30383 9034 Acked Read 5 Sent Done Done false
30384 9035 Acked Read 6 Sent Done Done false
30385 9036 Acked Flush 0 Sent Done Done false
30386 9037 Acked Write 4 Sent Done Done false
30387 9038 Acked Read 7 Sent Done Done false
30388 9039 Acked Read 10 Sent Done Done false
30389 9040 Acked Read 10 Sent Done Done false
30390 9041 Acked Read 3 Sent Done Done false
30391 9042 Acked Write 9 Sent Done Done false
30392 9043 Acked Read 9 Sent Done Done false
30393 9044 Acked Write 5 Sent Done Done false
30394 9045 Acked Flush 0 Sent Done Done false
30395 9046 Acked Read 9 Sent Done Done false
30396 9047 Acked Flush 0 Sent Done Done false
30397 9048 Acked Write 6 Sent Done Done false
30398 9049 Acked Read 4 Sent Done Done false
30399 9050 Acked Write 10 Sent Done Done false
30400 9051 Acked Read 6 Sent Done Done false
30401 9052 Acked Read 4 Sent Done Done false
30402 9053 Acked Read 2 Sent Done Done false
30403 9054 Acked Read 6 Sent Done Done false
30404 9055 Acked Read 2 Sent Done Done false
30405 9056 Acked Write 9 Sent Done Done false
30406 9057 Acked Write 6 Sent Done Done false
30407 9058 Acked Write 2 Sent Done Done false
30408 9059 Acked Read 4 Sent Done Done false
30409 9060 Acked Read 6 Sent Done Done false
30410 9061 Acked Flush 0 Sent Done Done false
30411 9062 Acked Read 5 Sent Done Done false
30412 9063 Acked Read 2 Sent Done Done false
30413 9064 Acked Write 7 Sent Done Done false
30414 9065 Acked Write 4 Sent Done Done false
30415 9066 Acked Read 3 Sent Done Done false
30416 9067 Acked Read 7 Sent Done Done false
30417 9068 Acked Read 1 Sent Done Done false
30418 9069 Acked Write 10 Sent Done Done false
30419 9070 Acked Flush 0 Sent Done Done false
30420 9071 Acked Write 7 Sent Done Done false
30421 9072 Acked Write 5 Sent Done Done false
30422 9073 Acked Write 7 Sent Done Done false
30423 9074 Acked Read 2 Sent Done Done false
30424 9075 Acked Read 9 Sent Done Done false
30425 9076 Acked Flush 0 Sent Done Done false
30426 9077 Acked Read 10 Sent Done Done false
30427 9078 Acked Read 8 Sent Done Done false
30428 9079 Acked Read 4 Sent Done Done false
30429 9080 Acked Read 9 Sent Done Done false
30430 9081 Acked Read 7 Sent Done Done false
30431 9082 Acked Read 7 Sent Done Done false
30432 9083 Acked Write 9 Sent Done Done false
30433 9084 Acked Flush 0 Sent Done Done false
30434 9085 Acked Flush 0 Sent Done Done false
30435 9086 Acked Flush 0 Sent Done Done false
30436 9087 Acked Read 3 Sent Done Done false
30437 9088 Acked Read 9 Sent Done Done false
30438 9089 Acked Read 5 Sent Done Done false
30439 9090 Acked Read 1 Sent Done Done false
30440 9091 Acked Read 8 Sent Done Done false
30441 9092 Acked Read 1 Sent Done Done false
30442 9093 Acked Flush 0 Sent Done Done false
30443 9094 Acked Read 9 Sent Done Done false
30444 9095 Acked Read 8 Sent Done Done false
30445 9096 Acked Read 3 Sent Done Done false
30446 9097 Acked Write 3 Sent Done Done false
30447 9098 Acked Read 9 Sent Done Done false
30448 9099 Acked Write 5 Sent Done Done false
30449 9100 Acked Read 1 Sent Done Done false
30450 9101 Acked Write 5 Sent Done Done false
30451 9102 Acked Read 10 Sent Done Done false
30452 9103 Acked Read 3 Sent Done Done false
30453 9104 Acked Read 10 Sent Done Done false
30454 9105 Acked Read 1 Sent Done Done false
30455 9106 Acked Flush 0 Sent Done Done false
30456 9107 Acked Write 2 Sent Done Done false
30457 9108 Acked Read 8 Sent Done Done false
30458 9109 Acked Write 6 Sent Done Done false
30459 9110 Acked Flush 0 Sent Done Done false
30460 9111 Acked Read 10 Sent Done Done false
30461 9112 Acked Read 6 Sent Done Done false
30462 9113 Acked Flush 0 Sent Done Done false
30463 9114 Acked Read 4 Sent Done Done false
30464 9115 Acked Write 6 Sent Done Done false
30465 9116 Acked Flush 0 Sent Done Done false
30466 9117 Acked Write 8 Sent Done Done false
30467 9118 Acked Write 3 Sent Done Done false
30468 9119 Acked Read 6 Sent Done Done false
30469 9120 Acked Write 1 Sent Done Done false
30470 9121 Acked Flush 0 Sent Done Done false
30471 9122 Acked Read 9 Sent Done Done false
30472 9123 Acked Read 2 Sent Done Done false
30473 9124 Acked Read 5 Sent Done Done false
30474 9125 Acked Write 8 Sent Done Done false
30475 9126 Acked Write 2 Sent Done Done false
30476 9127 Acked Read 4 Sent Done Done false
30477 9128 Acked Write 7 Sent Done Done false
30478 9129 Acked Flush 0 Sent Done Done false
30479 9130 Acked Read 8 Sent Done Done false
30480 9131 Acked Write 10 Sent Done Done false
30481 9132 Acked Write 5 Sent Done Done false
30482 9133 Acked Write 5 Sent Done Done false
30483 9134 Acked Write 8 Sent Done Done false
30484 9135 Acked Read 6 Sent Done Done false
30485 9136 Acked Write 7 Sent Done Done false
30486 9137 Acked Write 7 Sent Done Done false
30487 9138 Acked Write 6 Sent Done Done false
30488 9139 Acked Write 8 Sent Done Done false
30489 9140 Acked Read 6 Sent Done Done false
30490 9141 Acked Read 3 Sent Done Done false
30491 9142 Acked Write 3 Sent Done Done false
30492 9143 Acked Write 9 Sent Done Done false
30493 9144 Acked Read 6 Sent Done Done false
30494 9145 Acked Write 5 Sent Done Done false
30495 9146 Acked Read 5 Sent Done Done false
30496 9147 Acked Write 9 Sent Done Done false
30497 9148 Acked Read 9 Sent Done Done false
30498 9149 Acked Read 3 Sent Done Done false
30499 9150 Acked Write 4 Sent Done Done false
30500 STATES DS:0 DS:1 DS:2 TOTAL
30501 Sent 301 0 0 301
30502 Done 0 301 301 602
30503 Skipped 0 0 0 0
30504 Error 0 0 0 0
30505 Last Flush: 8849 9129 9129
30506 Downstairs last five completed: 8849 8848 8847 8846 8845
30507 Upstairs last five completed: 9150 9149 9148 9147 9146
30508 CLIENT: Up:0 ds:301 act:2
30509 [8810] got start action
30510 [8810] I am going to restart
30511 Starting downstairs at port 8810
30512 Make output file at "/tmp/dsc/downstairs-8810.txt"
30513 [8810][0] reports Starting
30514 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1229)
30515 [8810][0] reports Running
30516 ----------------------------------------------------------------
30517 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30518 States: Offline Active Active
30519 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30520 8850 Acked Read 6 Sent Done Done false
30521 8851 Acked Read 4 Sent Done Done false
30522 8852 Acked Read 6 Sent Done Done false
30523 8853 Acked Write 1 Sent Done Done false
30524 8854 Acked Read 7 Sent Done Done false
30525 8855 Acked Read 8 Sent Done Done false
30526 8856 Acked Write 10 Sent Done Done false
30527 8857 Acked Write 6 Sent Done Done false
30528 8858 Acked Flush 0 Sent Done Done false
30529 8859 Acked Read 10 Sent Done Done false
30530 8860 Acked Read 8 Sent Done Done false
30531 8861 Acked Read 8 Sent Done Done false
30532 8862 Acked Write 1 Sent Done Done false
30533 8863 Acked Read 2 Sent Done Done false
30534 8864 Acked Write 4 Sent Done Done false
30535 8865 Acked Read 6 Sent Done Done false
30536 8866 Acked Flush 0 Sent Done Done false
30537 8867 Acked Flush 0 Sent Done Done false
30538 8868 Acked Read 3 Sent Done Done false
30539 8869 Acked Read 10 Sent Done Done false
30540 8870 Acked Write 7 Sent Done Done false
30541 8871 Acked Write 1 Sent Done Done false
30542 8872 Acked Read 1 Sent Done Done false
30543 8873 Acked Write 7 Sent Done Done false
30544 8874 Acked Read 9 Sent Done Done false
30545 8875 Acked Read 10 Sent Done Done false
30546 8876 Acked Write 10 Sent Done Done false
30547 8877 Acked Read 10 Sent Done Done false
30548 8878 Acked Write 8 Sent Done Done false
30549 8879 Acked Write 3 Sent Done Done false
30550 8880 Acked Write 5 Sent Done Done false
30551 8881 Acked Write 5 Sent Done Done false
30552 8882 Acked Write 9 Sent Done Done false
30553 8883 Acked Write 6 Sent Done Done false
30554 8884 Acked Read 8 Sent Done Done false
30555 8885 Acked Write 9 Sent Done Done false
30556 8886 Acked Write 8 Sent Done Done false
30557 8887 Acked Flush 0 Sent Done Done false
30558 8888 Acked Write 7 Sent Done Done false
30559 8889 Acked Read 3 Sent Done Done false
30560 8890 Acked Flush 0 Sent Done Done false
30561 8891 Acked Write 8 Sent Done Done false
30562 8892 Acked Read 9 Sent Done Done false
30563 8893 Acked Read 6 Sent Done Done false
30564 8894 Acked Write 5 Sent Done Done false
30565 8895 Acked Write 7 Sent Done Done false
30566 8896 Acked Read 10 Sent Done Done false
30567 8897 Acked Flush 0 Sent Done Done false
30568 8898 Acked Read 3 Sent Done Done false
30569 8899 Acked Read 2 Sent Done Done false
30570 8900 Acked Read 1 Sent Done Done false
30571 8901 Acked Write 6 Sent Done Done false
30572 8902 Acked Write 4 Sent Done Done false
30573 8903 Acked Read 1 Sent Done Done false
30574 8904 Acked Write 5 Sent Done Done false
30575 8905 Acked Flush 0 Sent Done Done false
30576 8906 Acked Write 6 Sent Done Done false
30577 8907 Acked Write 10 Sent Done Done false
30578 8908 Acked Read 2 Sent Done Done false
30579 8909 Acked Read 1 Sent Done Done false
30580 8910 Acked Read 2 Sent Done Done false
30581 8911 Acked Write 5 Sent Done Done false
30582 8912 Acked Read 4 Sent Done Done false
30583 8913 Acked Read 4 Sent Done Done false
30584 8914 Acked Write 6 Sent Done Done false
30585 8915 Acked Read 7 Sent Done Done false
30586 8916 Acked Write 5 Sent Done Done false
30587 8917 Acked Read 2 Sent Done Done false
30588 8918 Acked Write 10 Sent Done Done false
30589 8919 Acked Write 9 Sent Done Done false
30590 8920 Acked Read 5 Sent Done Done false
30591 8921 Acked Read 2 Sent Done Done false
30592 8922 Acked Read 4 Sent Done Done false
30593 8923 Acked Write 6 Sent Done Done false
30594 8924 Acked Write 8 Sent Done Done false
30595 8925 Acked Write 4 Sent Done Done false
30596 8926 Acked Write 8 Sent Done Done false
30597 8927 Acked Read 2 Sent Done Done false
30598 8928 Acked Read 3 Sent Done Done false
30599 8929 Acked Read 1 Sent Done Done false
30600 8930 Acked Write 8 Sent Done Done false
30601 8931 Acked Flush 0 Sent Done Done false
30602 8932 Acked Read 8 Sent Done Done false
30603 8933 Acked Read 7 Sent Done Done false
30604 8934 Acked Read 3 Sent Done Done false
30605 8935 Acked Read 3 Sent Done Done false
30606 8936 Acked Read 7 Sent Done Done false
30607 8937 Acked Write 7 Sent Done Done false
30608 8938 Acked Read 6 Sent Done Done false
30609 8939 Acked Write 10 Sent Done Done false
30610 8940 Acked Write 1 Sent Done Done false
30611 8941 Acked Read 6 Sent Done Done false
30612 8942 Acked Read 6 Sent Done Done false
30613 8943 Acked Read 8 Sent Done Done false
30614 8944 Acked Write 2 Sent Done Done false
30615 8945 Acked Read 5 Sent Done Done false
30616 8946 Acked Read 4 Sent Done Done false
30617 8947 Acked Read 6 Sent Done Done false
30618 8948 Acked Write 4 Sent Done Done false
30619 8949 Acked Read 7 Sent Done Done false
30620 8950 Acked Read 3 Sent Done Done false
30621 8951 Acked Write 6 Sent Done Done false
30622 8952 Acked Write 7 Sent Done Done false
30623 8953 Acked Write 6 Sent Done Done false
30624 8954 Acked Read 2 Sent Done Done false
30625 8955 Acked Write 7 Sent Done Done false
30626 8956 Acked Flush 0 Sent Done Done false
30627 8957 Acked Write 3 Sent Done Done false
30628 8958 Acked Write 2 Sent Done Done false
30629 8959 Acked Read 9 Sent Done Done false
30630 8960 Acked Write 6 Sent Done Done false
30631 8961 Acked Read 5 Sent Done Done false
30632 8962 Acked Read 6 Sent Done Done false
30633 8963 Acked Write 9 Sent Done Done false
30634 8964 Acked Read 5 Sent Done Done false
30635 8965 Acked Read 5 Sent Done Done false
30636 8966 Acked Read 6 Sent Done Done false
30637 8967 Acked Read 5 Sent Done Done false
30638 8968 Acked Write 4 Sent Done Done false
30639 8969 Acked Flush 0 Sent Done Done false
30640 8970 Acked Read 7 Sent Done Done false
30641 8971 Acked Read 8 Sent Done Done false
30642 8972 Acked Write 5 Sent Done Done false
30643 8973 Acked Write 2 Sent Done Done false
30644 8974 Acked Read 3 Sent Done Done false
30645 8975 Acked Write 7 Sent Done Done false
30646 8976 Acked Read 9 Sent Done Done false
30647 8977 Acked Read 1 Sent Done Done false
30648 8978 Acked Read 6 Sent Done Done false
30649 8979 Acked Flush 0 Sent Done Done false
30650 8980 Acked Write 3 Sent Done Done false
30651 8981 Acked Write 5 Sent Done Done false
30652 8982 Acked Read 10 Sent Done Done false
30653 8983 Acked Write 10 Sent Done Done false
30654 8984 Acked Write 4 Sent Done Done false
30655 8985 Acked Read 5 Sent Done Done false
30656 8986 Acked Read 7 Sent Done Done false
30657 8987 Acked Read 6 Sent Done Done false
30658 8988 Acked Write 6 Sent Done Done false
30659 8989 Acked Read 10 Sent Done Done false
30660 8990 Acked Write 10 Sent Done Done false
30661 8991 Acked Write 9 Sent Done Done false
30662 8992 Acked Read 2 Sent Done Done false
30663 8993 Acked Read 10 Sent Done Done false
30664 8994 Acked Read 9 Sent Done Done false
30665 8995 Acked Flush 0 Sent Done Done false
30666 8996 Acked Read 9 Sent Done Done false
30667 8997 Acked Read 5 Sent Done Done false
30668 8998 Acked Write 10 Sent Done Done false
30669 8999 Acked Read 8 Sent Done Done false
30670 9000 Acked Read 10 Sent Done Done false
30671 9001 Acked Write 8 Sent Done Done false
30672 9002 Acked Write 6 Sent Done Done false
30673 9003 Acked Write 2 Sent Done Done false
30674 9004 Acked Read 9 Sent Done Done false
30675 9005 Acked Read 9 Sent Done Done false
30676 9006 Acked Write 2 Sent Done Done false
30677 9007 Acked Read 10 Sent Done Done false
30678 9008 Acked Read 2 Sent Done Done false
30679 9009 Acked Write 8 Sent Done Done false
30680 9010 Acked Write 5 Sent Done Done false
30681 9011 Acked Read 8 Sent Done Done false
30682 9012 Acked Write 4 Sent Done Done false
30683 9013 Acked Read 2 Sent Done Done false
30684 9014 Acked Read 7 Sent Done Done false
30685 9015 Acked Read 2 Sent Done Done false
30686 9016 Acked Flush 0 Sent Done Done false
30687 9017 Acked Write 9 Sent Done Done false
30688 9018 Acked Write 4 Sent Done Done false
30689 9019 Acked Read 6 Sent Done Done false
30690 9020 Acked Read 4 Sent Done Done false
30691 9021 Acked Flush 0 Sent Done Done false
30692 9022 Acked Read 3 Sent Done Done false
30693 9023 Acked Read 5 Sent Done Done false
30694 9024 Acked Write 5 Sent Done Done false
30695 9025 Acked Read 1 Sent Done Done false
30696 9026 Acked Write 4 Sent Done Done false
30697 9027 Acked Write 1 Sent Done Done false
30698 9028 Acked Read 4 Sent Done Done false
30699 9029 Acked Read 3 Sent Done Done false
30700 9030 Acked Read 4 Sent Done Done false
30701 9031 Acked Write 10 Sent Done Done false
30702 9032 Acked Write 2 Sent Done Done false
30703 9033 Acked Read 7 Sent Done Done false
30704 9034 Acked Read 5 Sent Done Done false
30705 9035 Acked Read 6 Sent Done Done false
30706 9036 Acked Flush 0 Sent Done Done false
30707 9037 Acked Write 4 Sent Done Done false
30708 9038 Acked Read 7 Sent Done Done false
30709 9039 Acked Read 10 Sent Done Done false
30710 9040 Acked Read 10 Sent Done Done false
30711 9041 Acked Read 3 Sent Done Done false
30712 9042 Acked Write 9 Sent Done Done false
30713 9043 Acked Read 9 Sent Done Done false
30714 9044 Acked Write 5 Sent Done Done false
30715 9045 Acked Flush 0 Sent Done Done false
30716 9046 Acked Read 9 Sent Done Done false
30717 9047 Acked Flush 0 Sent Done Done false
30718 9048 Acked Write 6 Sent Done Done false
30719 9049 Acked Read 4 Sent Done Done false
30720 9050 Acked Write 10 Sent Done Done false
30721 9051 Acked Read 6 Sent Done Done false
30722 9052 Acked Read 4 Sent Done Done false
30723 9053 Acked Read 2 Sent Done Done false
30724 9054 Acked Read 6 Sent Done Done false
30725 9055 Acked Read 2 Sent Done Done false
30726 9056 Acked Write 9 Sent Done Done false
30727 9057 Acked Write 6 Sent Done Done false
30728 9058 Acked Write 2 Sent Done Done false
30729 9059 Acked Read 4 Sent Done Done false
30730 9060 Acked Read 6 Sent Done Done false
30731 9061 Acked Flush 0 Sent Done Done false
30732 9062 Acked Read 5 Sent Done Done false
30733 9063 Acked Read 2 Sent Done Done false
30734 9064 Acked Write 7 Sent Done Done false
30735 9065 Acked Write 4 Sent Done Done false
30736 9066 Acked Read 3 Sent Done Done false
30737 9067 Acked Read 7 Sent Done Done false
30738 9068 Acked Read 1 Sent Done Done false
30739 9069 Acked Write 10 Sent Done Done false
30740 9070 Acked Flush 0 Sent Done Done false
30741 9071 Acked Write 7 Sent Done Done false
30742 9072 Acked Write 5 Sent Done Done false
30743 9073 Acked Write 7 Sent Done Done false
30744 9074 Acked Read 2 Sent Done Done false
30745 9075 Acked Read 9 Sent Done Done false
30746 9076 Acked Flush 0 Sent Done Done false
30747 9077 Acked Read 10 Sent Done Done false
30748 9078 Acked Read 8 Sent Done Done false
30749 9079 Acked Read 4 Sent Done Done false
30750 9080 Acked Read 9 Sent Done Done false
30751 9081 Acked Read 7 Sent Done Done false
30752 9082 Acked Read 7 Sent Done Done false
30753 9083 Acked Write 9 Sent Done Done false
30754 9084 Acked Flush 0 Sent Done Done false
30755 9085 Acked Flush 0 Sent Done Done false
30756 9086 Acked Flush 0 Sent Done Done false
30757 9087 Acked Read 3 Sent Done Done false
30758 9088 Acked Read 9 Sent Done Done false
30759 9089 Acked Read 5 Sent Done Done false
30760 9090 Acked Read 1 Sent Done Done false
30761 9091 Acked Read 8 Sent Done Done false
30762 9092 Acked Read 1 Sent Done Done false
30763 9093 Acked Flush 0 Sent Done Done false
30764 9094 Acked Read 9 Sent Done Done false
30765 9095 Acked Read 8 Sent Done Done false
30766 9096 Acked Read 3 Sent Done Done false
30767 9097 Acked Write 3 Sent Done Done false
30768 9098 Acked Read 9 Sent Done Done false
30769 9099 Acked Write 5 Sent Done Done false
30770 9100 Acked Read 1 Sent Done Done false
30771 9101 Acked Write 5 Sent Done Done false
30772 9102 Acked Read 10 Sent Done Done false
30773 9103 Acked Read 3 Sent Done Done false
30774 9104 Acked Read 10 Sent Done Done false
30775 9105 Acked Read 1 Sent Done Done false
30776 9106 Acked Flush 0 Sent Done Done false
30777 9107 Acked Write 2 Sent Done Done false
30778 9108 Acked Read 8 Sent Done Done false
30779 9109 Acked Write 6 Sent Done Done false
30780 9110 Acked Flush 0 Sent Done Done false
30781 9111 Acked Read 10 Sent Done Done false
30782 9112 Acked Read 6 Sent Done Done false
30783 9113 Acked Flush 0 Sent Done Done false
30784 9114 Acked Read 4 Sent Done Done false
30785 9115 Acked Write 6 Sent Done Done false
30786 9116 Acked Flush 0 Sent Done Done false
30787 9117 Acked Write 8 Sent Done Done false
30788 9118 Acked Write 3 Sent Done Done false
30789 9119 Acked Read 6 Sent Done Done false
30790 9120 Acked Write 1 Sent Done Done false
30791 9121 Acked Flush 0 Sent Done Done false
30792 9122 Acked Read 9 Sent Done Done false
30793 9123 Acked Read 2 Sent Done Done false
30794 9124 Acked Read 5 Sent Done Done false
30795 9125 Acked Write 8 Sent Done Done false
30796 9126 Acked Write 2 Sent Done Done false
30797 9127 Acked Read 4 Sent Done Done false
30798 9128 Acked Write 7 Sent Done Done false
30799 9129 Acked Flush 0 Sent Done Done false
30800 9130 Acked Read 8 Sent Done Done false
30801 9131 Acked Write 10 Sent Done Done false
30802 9132 Acked Write 5 Sent Done Done false
30803 9133 Acked Write 5 Sent Done Done false
30804 9134 Acked Write 8 Sent Done Done false
30805 9135 Acked Read 6 Sent Done Done false
30806 9136 Acked Write 7 Sent Done Done false
30807 9137 Acked Write 7 Sent Done Done false
30808 9138 Acked Write 6 Sent Done Done false
30809 9139 Acked Write 8 Sent Done Done false
30810 9140 Acked Read 6 Sent Done Done false
30811 9141 Acked Read 3 Sent Done Done false
30812 9142 Acked Write 3 Sent Done Done false
30813 9143 Acked Write 9 Sent Done Done false
30814 9144 Acked Read 6 Sent Done Done false
30815 9145 Acked Write 5 Sent Done Done false
30816 9146 Acked Read 5 Sent Done Done false
30817 9147 Acked Write 9 Sent Done Done false
30818 9148 Acked Read 9 Sent Done Done false
30819 9149 Acked Read 3 Sent Done Done false
30820 9150 Acked Write 4 Sent Done Done false
30821 9151 Acked Flush 0 Sent Done Done false
30822 STATES DS:0 DS:1 DS:2 TOTAL
30823 Sent 302 0 0 302
30824 Done 0 302 302 604
30825 Skipped 0 0 0 0
30826 Error 0 0 0 0
30827 Last Flush: 8849 9151 9151
30828 Downstairs last five completed: 8849 8848 8847 8846 8845
30829 Upstairs last five completed: 9151 9150 9149 9148 9147
30830 CLIENT: Up:0 ds:302 act:2
308312025-01-15T05:25:27.275ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308322025-01-15T05:25:27.275ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34526) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308332025-01-15T05:25:27.276ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308342025-01-15T05:25:27.277ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308352025-01-15T05:25:27.277ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308362025-01-15T05:25:27.277ZINFOcrucible: send last flush ID to this DS: 8849 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308372025-01-15T05:25:27.277ZINFOcrucible: Replied this last flush ID: 8849 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308382025-01-15T05:25:27.277ZINFOcrucible: [0] client re-new 302 jobs since flush 8849 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308392025-01-15T05:25:27.277ZINFOcrucible: [0] Replayed 302 jobs since flush: 8849 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
30840 ----------------------------------------------------------------
30841 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30842 Downstairs last five completed: 9151 9150 9149 9148 9147
30843 Upstairs last five completed: 9151 9150 9149 9148 9147
30844 CLIENT: Up:0 ds:0 act:3
30845 Replay: All jobs finished
30846 Main task has work to do, go find it
30847 got dsc Stop(0)
30848 stop 0
30849 [8810] Got stop action so:false kr:false
30850 Jan 15 05:25:29.523 INFO request completed, latency_us: 361, response_code: 204, uri: /stop/cid/0, method: POST, req_id: ae728730-4bfb-487b-90df-c9f1c5222a01, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
30851 Jan 15 05:25:29.524 INFO request completed, latency_us: 127, response_code: 200, uri: /state/cid/0, method: GET, req_id: 82ff836b-a3e0-4adf-aebd-537ea20b558b, remote_addr:{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:25:29.524292849Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
30852 127.0.0.1:42402, local_addr: 127.0.0.1:9998
308532025-01-15T05:25:29.524ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308542025-01-15T05:25:29.524ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
308552025-01-15T05:25:29.524ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
30856 [8810] Exited with: ExitStatus(unix_wait_status(9))
30857 [8810][0] reports Exit
30858 Jan 15 05:25:33.526 INFO request completed, latency_us: 180, response_code: 200, uri: /state/cid/0, method: GET, req_id: 5c668b8c-edfc-4a49-a10a-6b1a54d69fff, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
30859 001/300 Read block 909 len 32768
30860 002/300 Write block 2925 len 32768 data: 7 5 2 2 2 2 2 2
30861 003/300 Write block 2253 len 24576 data: 3 5 6 7 7 7
30862 004/300 Write block 2816 len 8192 data: 2 3
30863 005/300 Read block 2349 len 40960
30864 006/300 Read block 2897 len 32768
30865 007/300 Write block 4138 len 32768 data: 7 5 6 6 7 9 9 8
30866 008/300 Read block 386 len 24576
30867 009/300 Read block 2343 len 32768
30868 010/300 Flush
30869 011/300 Flush
30870 012/300 Read block 2629 len 40960
30871 013/300 Read block 2391 len 28672
30872 014/300 Read block 4773 len 8192
30873 015/300 Read block 4450 len 24576
30874 016/300 Flush
30875 017/300 Write block 1406 len 24576 data: 5 5 6 5 5 6
30876 018/300 Write block 4280 len 8192 data: 5 5
30877 019/300 Read block 2193 len 20480
30878 020/300 Read block 2341 len 40960
30879 021/300 Read block 1951 len 16384
30880 022/300 Write block 3484 len 8192 data: 6 6
30881 023/300 Write block 1807 len 20480 data: 6 6 6 6 5
30882 024/300 Write block 1082 len 4096 data: 5
30883 025/300 Write block 4455 len 32768 data: 8 7 7 6 5 4 5 4
30884 026/300 Write block 375 len 12288 data: 8 8 9
30885 027/300 Read block 1738 len 28672
30886 028/300 Write block 2008 len 12288 data: 4 6 7
30887 029/300 Write block 3222 len 20480 data: 5 4 4 4 3
30888 030/300 Flush
30889 031/300 Read block 4312 len 8192
30890 032/300 Read block 1117 len 24576
30891 033/300 Read block 4564 len 4096
30892 034/300 Read block 4954 len 36864
30893 035/300 Read block 3162 len 32768
30894 036/300 Flush
30895 037/300 Write block 4582 len 24576 data: 6 6 6 7 6 6
30896 038/300 Write block 3877 len 32768 data: 4 5 6 4 6 5 6 8
30897 039/300 Write block 3326 len 36864 data: 5 8 6 5 4 4 3 4 8
30898 040/300 Flush
30899 041/300 Write block 3709 len 8192 data: 4 4
30900 042/300 Read block 3593 len 4096
30901 043/300 Write block 996 len 24576 data: 3 4 5 5 5 2
30902 044/300 Flush
30903 045/300 Write block 1585 len 16384 data: 8 10 9 7
30904 046/300 Read block 1917 len 20480
30905 047/300 Read block 1391 len 4096
30906 048/300 Write block 1161 len 40960 data: 6 7 7 8 5 5 5 5 6 9
30907 049/300 Read block 651 len 24576
30908 050/300 Read block 2472 len 8192
30909 051/300 Write block 1230 len 16384 data: 5 4 5 3
30910 052/300 Flush
30911 053/300 Write block 2228 len 8192 data: 7 6
30912 054/300 Write block 2230 len 12288 data: 5 4 4
30913 055/300 Write block 1687 len 28672 data: 5 4 4 4 3 4 3
30914 056/300 Read block 1053 len 20480
30915 057/300 Read block 3192 len 40960
30916 058/300 Write block 3682 len 8192 data: 12 10
30917 059/300 Write block 4760 len 12288 data: 3 2 2
30918 060/300 Read block 2907 len 8192
30919 061/300 Write block 972 len 28672 data: 5 5 6 7 6 5 6
30920 062/300 Write block 1580 len 32768 data: 6 4 7 6 6 9 11 10
30921 063/300 Read block 1904 len 28672
30922 064/300 Write block 4282 len 40960 data: 6 4 5 6 7 8 9 9 9 7
30923 065/300 Read block 3425 len 28672
30924 066/300 Flush
30925 067/300 Write block 3257 len 24576 data: 5 5 3 3 5 6
30926 068/300 Write block 4631 len 24576 data: 8 7 8 8 7 7
30927 069/300 Read block 1866 len 20480
30928 070/300 Flush
30929 071/300 Read block 2329 len 24576
30930 072/300 Read block 2496 len 40960
30931 073/300 Read block 3563 len 36864
30932 074/300 Write block 2487 len 28672 data: 7 7 6 6 9 8 7
30933 075/300 Write block 4031 len 36864 data: 5 5 5 4 4 5 5 4 4
30934 076/300 Flush
30935 077/300 Read block 4025 len 24576
30936 078/300 Read block 3190 len 36864
30937 079/300 Write block 337 len 28672 data: 3 4 5 5 5 5 6
30938 080/300 Read block 1896 len 24576
30939 081/300 Write block 617 len 40960 data: 4 4 4 4 4 4 3 3 5 6
30940 082/300 Write block 4638 len 36864 data: 5 5 4 3 3 3 4 4 3
30941 083/300 Read block 2936 len 24576
30942 084/300 Write block 44 len 8192 data: 7 6
30943 085/300 Read block 3806 len 32768
30944 086/300 Write block 1619 len 36864 data: 6 6 4 5 7 7 6 7 5
30945 087/300 Flush
30946 088/300 Write block 1679 len 4096 data: 7
30947 089/300 Read block 1507 len 32768
30948 090/300 Read block 264 len 36864
30949 091/300 Write block 14 len 12288 data: 5 3 5
30950 092/300 Read block 2640 len 8192
30951 093/300 Read block 3187 len 12288
30952 094/300 Flush
30953 095/300 Read block 1679 len 8192
30954 096/300 Write block 3157 len 40960 data: 10 10 8 10 6 5 6 6 5 7
30955 097/300 Flush
30956 098/300 Read block 2747 len 8192
30957 099/300 Write block 4654 len 32768 data: 5 4 6 6 5 6 5 5
30958 100/300 Read block 4253 len 8192
30959 101/300 Write block 4238 len 16384 data: 6 7 10 13
30960 102/300 Flush
30961 103/300 Read block 1695 len 32768
30962 104/300 Read block 827 len 20480
30963 105/300 Read block 1292 len 32768
30964 106/300 Write block 3046 len 24576 data: 6 6 6 3 3 2
30965 107/300 Write block 1452 len 4096 data: 5
30966 108/300 Write block 4145 len 40960 data: 9 7 7 6 5 4 6 7 7 9
30967 109/300 Read block 1115 len 24576
30968 110/300 Read block 591 len 24576
30969 111/300 Write block 3502 len 40960 data: 6 6 5 6 3 4 5 5 5 7
30970 112/300 Write block 3576 len 24576 data: 5 5 6 6 5 5
30971 113/300 Read block 3220 len 24576
30972 114/300 Read block 4133 len 36864
30973 115/300 Read block 2352 len 36864
30974 116/300 Read block 3560 len 28672
30975 117/300 Read block 1017 len 40960
30976 118/300 Read block 2275 len 24576
30977 119/300 Read block 2210 len 20480
30978 120/300 Read block 546 len 20480
30979 121/300 Flush
30980 122/300 Write block 4159 len 24576 data: 8 6 5 3 2 3
30981 123/300 Read block 4102 len 8192
30982 124/300 Write block 1888 len 12288 data: 6 6 5
30983 125/300 Write block 1449 len 24576 data: 4 4 5 6 6 7
30984 126/300 Write block 2796 len 16384 data: 5 5 4 4
30985 127/300 Read block 2287 len 16384
30986 128/300 Read block 4670 len 36864
30987 129/300 Write block 4676 len 16384 data: 7 9 8 7
30988 130/300 Write block 3546 len 16384 data: 6 4 4 4
30989 131/300 Read block 3358 len 20480
30990 132/300 Write block 2568 len 20480 data: 6 7 7 5 6
30991 133/300 Flush
30992 134/300 Write block 3307 len 8192 data: 9 7
30993 135/300 Write block 4723 len 16384 data: 5 5 6 7
30994 136/300 Write block 2789 len 16384 data: 6 5 3 4
30995 137/300 Read block 2684 len 16384
30996 138/300 Write block 4091 len 16384 data: 5 5 6 6
30997 139/300 Write block 3254 len 36864 data: 5 4 5 6 6 4 4 6 7
30998 140/300 Read block 239 len 12288
30999 141/300 Read block 1500 len 12288
31000 142/300 Flush
31001 143/300 Write block 2238 len 36864 data: 6 6 7 7 7 6 8 6 5
31002 144/300 Write block 4135 len 8192 data: 7 6
31003 145/300 Read block 3779 len 8192
31004 146/300 Write block 191 len 8192 data: 5 5
31005 147/300 Write block 1556 len 28672 data: 3 3 3 3 3 4 4
31006 148/300 Read block 1923 len 16384
31007 149/300 Read block 2837 len 12288
31008 150/300 Flush
31009 151/300 Read block 4080 len 24576
31010 152/300 Write block 1201 len 20480 data: 7 8 9 10 9
31011 153/300 Read block 3571 len 20480
31012 154/300 Write block 3532 len 4096 data: 8
31013 155/300 Write block 3381 len 4096 data: 6
31014 156/300 Read block 4891 len 36864
31015 157/300 Write block 3972 len 16384 data: 4 5 6 7
31016 158/300 Read block 1027 len 28672
31017 159/300 Read block 1338 len 20480
31018 160/300 Read block 4464 len 36864
31019 161/300 Read block 3785 len 40960
31020 162/300 Read block 201 len 12288
31021 163/300 Write block 854 len 16384 data: 5 5 5 6
31022 164/300 Read block 364 len 28672
31023 165/300 Read block 2310 len 32768
31024 166/300 Read block 950 len 28672
31025 167/300 Read block 1646 len 12288
31026 168/300 Read block 1287 len 28672
31027 169/300 Read block 3251 len 20480
31028 170/300 Read block 3084 len 36864
31029 171/300 Read block 812 len 24576
31030 172/300 Read block 3908 len 28672
31031 173/300 Write block 4227 len 8192 data: 5 6
31032 174/300 Read block 1264 len 28672
31033 175/300 Read block 3627 len 12288
31034 176/300 Read block 1112 len 8192
31035 177/300 Flush
31036 178/300 Read block 2587 len 24576
31037 179/300 Read block 42 len 20480
31038 180/300 Flush
31039 181/300 Write block 3506 len 40960 data: 4 5 6 6 6 8 4 4 4 4
31040 182/300 Write block 4379 len 12288 data: 6 8 9
31041 183/300 Read block 762 len 24576
31042 184/300 Read block 1139 len 4096
31043 185/300 Read block 1985 len 20480
31044 186/300 Write block 4967 len 40960 data: 7 7 6 6 4 3 4 4 4 5
31045 187/300 Read block 590 len 4096
31046 188/300 Write block 1936 len 20480 data: 4 4 4 4 5
31047 189/300 Read block 2868 len 36864
31048 190/300 Read block 3255 len 28672
31049 191/300 Read block 2844 len 12288
31050 192/300 Read block 1800 len 32768
31051 193/300 Read block 1733 len 28672
31052 194/300 Read block 3641 len 24576
31053 195/300 Write block 4892 len 16384 data: 6 6 6 6
31054 196/300 Write block 2549 len 4096 data: 3
31055 197/300 Write block 3293 len 28672 data: 6 5 4 5 6 4 6
31056 198/300 Read block 329 len 12288
31057 199/300 Write block 4576 len 40960 data: 6 7 8 7 8 9 7 7 7 8
31058 200/300 Flush
31059 201/300 Read block 4434 len 28672
31060 202/300 Read block 410 len 16384
31061 203/300 Write block 2851 len 12288 data: 4 7 5
31062 204/300 Write block 2290 len 32768 data: 5 5 5 5 5 5 6 5
31063 205/300 Read block 4430 len 32768
31064 206/300 Write block 1810 len 32768 data: 7 6 6 7 7 7 8 7
31065 207/300 Read block 3878 len 40960
31066 208/300 Write block 4977 len 24576 data: 5 6 8 8 7 6
31067 209/300 Write block 4146 len 8192 data: 8 8
31068 210/300 Write block 664 len 20480 data: 6 6 5 5 5
31069 211/300 Read block 3338 len 4096
31070 212/300 Read block 4178 len 4096
31071 213/300 Flush
31072 214/300 Write block 1222 len 24576 data: 5 4 4 5 5 4
31073 215/300 Flush
31074 216/300 Flush
31075 217/300 Read block 1872 len 20480
31076 218/300 Read block 2440 len 40960
31077 219/300 Read block 3927 len 12288
31078 220/300 Read block 915 len 28672
31079 221/300 Write block 276 len 40960 data: 4 5 6 5 4 3 3 3 5 6
31080 222/300 Write block 3621 len 4096 data: 5
31081 223/300 Read block 4768 len 40960
31082 224/300 Read block 1085 len 8192
31083 225/300 Write block 2927 len 20480 data: 3 3 3 3 3
31084 226/300 Read block 4316 len 24576
31085 227/300 Write block 2265 len 36864 data: 6 5 5 6 5 5 5 4 6
31086 228/300 Write block 4173 len 20480 data: 2 2 3 3 5
31087 229/300 Read block 331 len 12288
31088 230/300 Write block 3565 len 16384 data: 6 5 5 3
31089 231/300 Read block 914 len 40960
31090 232/300 Read block 3921 len 20480
31091 233/300 Read block 3431 len 24576
31092 234/300 Write block 4540 len 40960 data: 3 3 5 6 5 6 5 7 6 8
31093 235/300 Read block 314 len 36864
31094 236/300 Write block 225 len 8192 data: 6 5
31095 237/300 Read block 3916 len 8192
31096 238/300 Write block 2577 len 40960 data: 4 4 3 4 4 3 4 4 4 7
31097 239/300 Write block 491 len 16384 data: 7 6 6 7
31098 240/300 Read block 4332 len 8192
31099 241/300 Write block 1831 len 8192 data: 7 6
31100 242/300 Flush
31101 243/300 Write block 4493 len 12288 data: 7 7 9
31102 244/300 Write block 1319 len 12288 data: 6 7 5
31103 245/300 Read block 3650 len 12288
31104 246/300 Write block 3873 len 32768 data: 8 6 5 4 5 6 7 5
31105 247/300 Write block 2044 len 32768 data: 5 4 4 5 6 5 4 3
31106 248/300 Write block 1138 len 8192 data: 5 4
31107 249/300 Write block 3017 len 28672 data: 2 5 5 6 6 6 7
31108 250/300 Read block 3088 len 24576
31109 251/300 Write block 4333 len 36864 data: 6 6 5 5 5 5 5 5 4
31110 252/300 Read block 1191 len 12288
31111 253/300 Read block 791 len 28672
31112 254/300 Flush
31113 255/300 Read block 1251 len 8192
31114 256/300 Flush
31115 257/300 Read block 1715 len 40960
31116 258/300 Write block 1804 len 32768 data: 7 6 7 7 7 7 8 7
31117 259/300 Write block 477 len 8192 data: 6 5
31118 260/300 Read block 1158 len 4096
31119 261/300 Write block 2668 len 36864 data: 3 4 3 4 4 4 6 5 6
31120 262/300 Write block 2496 len 28672 data: 5 6 5 6 5 6 7
31121 263/300 Read block 1020 len 24576
31122 264/300 Write block 1729 len 20480 data: 5 4 4 4 4
31123 265/300 Write block 1023 len 12288 data: 2 2 2
31124 266/300 Read block 427 len 8192
31125 267/300 Read block 818 len 24576
31126 268/300 Write block 1167 len 36864 data: 6 6 7 10 7 6 6 6 5
31127 269/300 Write block 2490 len 40960 data: 7 10 9 8 7 5 6 7 6 7
31128 270/300 Read block 1565 len 8192
31129 271/300 Flush
31130 272/300 Write block 2709 len 4096 data: 4
31131 273/300 Write block 552 len 32768 data: 6 6 8 7 7 6 7 8
31132 274/300 Read block 1361 len 40960
31133 275/300 Flush
31134 276/300 Write block 4478 len 32768 data: 3 3 3 4 4 5 6 7
31135 277/300 Write block 1979 len 20480 data: 4 4 6 6 5
31136 278/300 Read block 662 len 40960
31137 279/300 Read block 3191 len 24576
31138 280/300 Read block 1168 len 12288
31139 281/300 Write block 2975 len 24576 data: 6 7 7 6 6 6
31140 282/300 Write block 1494 len 36864 data: 5 5 5 3 4 4 6 7 8
31141 283/300 Read block 4681 len 20480
31142 284/300 Write block 3800 len 8192 data: 6 4
31143 285/300 Flush
31144 286/300 Write block 3771 len 12288 data: 5 5 6
31145 287/300 Write block 411 len 24576 data: 6 7 7 6 5 5
31146 288/300 Flush
31147 289/300 Write block 3365 len 24576 data: 8 7 7 7 7 10
31148 290/300 Read block 3342 len 32768
31149 291/300 Write block 13 len 40960 data: 6 6 4 6 6 7 6 6 7 8
31150 292/300 Write block 3756 len 32768 data: 4 5 5 5 6 6 5 6
31151 293/300 Write block 2317 len 16384 data: 6 7 6 5
31152 294/300 Read block 4725 len 24576
31153 295/300 Read block 4500 len 40960
31154 296/300 Read block 4394 len 12288
31155 297/300 Flush
31156 298/300 Read block 2221 len 36864
31157 299/300 Read block 1680 len 12288
31158 300/300 Read block 3673 len 16384
31159 301/300 Write block 4961 len 40960 data: 5 5 7 7 6 6 8 8 7 7
31160 Main task has work to do, go find it
31161 got dsc Start(0)
31162 start 0
31163 Jan 15 05:25:33.761 INFO request completed, latency_us: 224, response_code: 204, uri: /start/cid/0, method: POST, req_id: f13bb41c-482c-4d27-8c0c-9a52e3097983, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
31164 Replay: started 0, returned:Ok(())
31165 ----------------------------------------------------------------
31166 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31167 States: Offline Active Active
31168 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31169 9152 Acked Read 8 Sent Done Done false
31170 9153 Acked Write 8 Sent Done Done false
31171 9154 Acked Write 6 Sent Done Done false
31172 9155 Acked Write 2 Sent Done Done false
31173 9156 Acked Read 10 Sent Done Done false
31174 9157 Acked Read 8 Sent Done Done false
31175 9158 Acked Write 8 Sent Done Done false
31176 9159 Acked Read 6 Sent Done Done false
31177 9160 Acked Read 8 Sent Done Done false
31178 9161 Acked Flush 0 Sent Done Done false
31179 9162 Acked Flush 0 Sent Done Done false
31180 9163 Acked Read 10 Sent Done Done false
31181 9164 Acked Read 7 Sent Done Done false
31182 9165 Acked Read 2 Sent Done Done false
31183 9166 Acked Read 6 Sent Done Done false
31184 9167 Acked Flush 0 Sent Done Done false
31185 9168 Acked Write 6 Sent Done Done false
31186 9169 Acked Write 2 Sent Done Done false
31187 9170 Acked Read 5 Sent Done Done false
31188 9171 Acked Read 10 Sent Done Done false
31189 9172 Acked Read 4 Sent Done Done false
31190 9173 Acked Write 2 Sent Done Done false
31191 9174 Acked Write 5 Sent Done Done false
31192 9175 Acked Write 1 Sent Done Done false
31193 9176 Acked Write 8 Sent Done Done false
31194 9177 Acked Write 3 Sent Done Done false
31195 9178 Acked Read 7 Sent Done Done false
31196 9179 Acked Write 3 Sent Done Done false
31197 9180 Acked Write 5 Sent Done Done false
31198 9181 Acked Flush 0 Sent Done Done false
31199 9182 Acked Read 2 Sent Done Done false
31200 9183 Acked Read 6 Sent Done Done false
31201 9184 Acked Read 1 Sent Done Done false
31202 9185 Acked Read 9 Sent Done Done false
31203 9186 Acked Read 8 Sent Done Done false
31204 9187 Acked Flush 0 Sent Done Done false
31205 9188 Acked Write 6 Sent Done Done false
31206 9189 Acked Write 8 Sent Done Done false
31207 9190 Acked Write 9 Sent Done Done false
31208 9191 Acked Flush 0 Sent Done Done false
31209 9192 Acked Write 2 Sent Done Done false
31210 9193 Acked Read 1 Sent Done Done false
31211 9194 Acked Write 6 Sent Done Done false
31212 9195 Acked Flush 0 Sent Done Done false
31213 9196 Acked Write 4 Sent Done Done false
31214 9197 Acked Read 5 Sent Done Done false
31215 9198 Acked Read 1 Sent Done Done false
31216 9199 Acked Write 10 Sent Done Done false
31217 9200 Acked Read 6 Sent Done Done false
31218 9201 Acked Read 2 Sent Done Done false
31219 9202 Acked Write 4 Sent Done Done false
31220 9203 Acked Flush 0 Sent Done Done false
31221 9204 Acked Write 2 Sent Done Done false
31222 9205 Acked Write 3 Sent Done Done false
31223 9206 Acked Write 7 Sent Done Done false
31224 9207 Acked Read 5 Sent Done Done false
31225 9208 Acked Read 10 Sent Done Done false
31226 9209 Acked Write 2 Sent Done Done false
31227 9210 Acked Write 3 Sent Done Done false
31228 9211 Acked Read 2 Sent Done Done false
31229 9212 Acked Write 7 Sent Done Done false
31230 9213 Acked Write 8 Sent Done Done false
31231 9214 Acked Read 7 Sent Done Done false
31232 9215 Acked Write 10 Sent Done Done false
31233 9216 Acked Read 7 Sent Done Done false
31234 9217 Acked Flush 0 Sent Done Done false
31235 9218 Acked Write 6 Sent Done Done false
31236 9219 Acked Write 6 Sent Done Done false
31237 9220 Acked Read 5 Sent Done Done false
31238 9221 Acked Flush 0 Sent Done Done false
31239 9222 Acked Read 6 Sent Done Done false
31240 9223 Acked Read 10 Sent Done Done false
31241 9224 Acked Read 9 Sent Done Done false
31242 9225 Acked Write 7 Sent Done Done false
31243 9226 Acked Write 9 Sent Done Done false
31244 9227 Acked Flush 0 Sent Done Done false
31245 9228 Acked Read 6 Sent Done Done false
31246 9229 Acked Read 9 Sent Done Done false
31247 9230 Acked Write 7 Sent Done Done false
31248 9231 Acked Read 6 Sent Done Done false
31249 9232 Acked Write 10 Sent Done Done false
31250 9233 Acked Write 9 Sent Done Done false
31251 9234 Acked Read 6 Sent Done Done false
31252 9235 Acked Write 2 Sent Done Done false
31253 9236 Acked Read 8 Sent Done Done false
31254 9237 Acked Write 9 Sent Done Done false
31255 9238 Acked Flush 0 Sent Done Done false
31256 9239 Acked Write 1 Sent Done Done false
31257 9240 Acked Read 8 Sent Done Done false
31258 9241 Acked Read 9 Sent Done Done false
31259 9242 Acked Write 3 Sent Done Done false
31260 9243 Acked Read 2 Sent Done Done false
31261 9244 Acked Read 3 Sent Done Done false
31262 9245 Acked Flush 0 Sent Done Done false
31263 9246 Acked Read 2 Sent Done Done false
31264 9247 Acked Write 10 Sent Done Done false
31265 9248 Acked Flush 0 Sent Done Done false
31266 9249 Acked Read 2 Sent Done Done false
31267 9250 Acked Write 8 Sent Done Done false
31268 9251 Acked Read 2 Sent Done Done false
31269 9252 Acked Write 4 Sent Done Done false
31270 9253 Acked Flush 0 Sent Done Done false
31271 9254 Acked Read 8 Sent Done Done false
31272 9255 Acked Read 5 Sent Done Done false
31273 9256 Acked Read 8 Sent Done Done false
31274 9257 Acked Write 6 Sent Done Done false
31275 9258 Acked Write 1 Sent Done Done false
31276 9259 Acked Write 10 Sent Done Done false
31277 9260 Acked Read 6 Sent Done Done false
31278 9261 Acked Read 6 Sent Done Done false
31279 9262 Acked Write 10 Sent Done Done false
31280 9263 Acked Write 6 Sent Done Done false
31281 9264 Acked Read 6 Sent Done Done false
31282 9265 Acked Read 9 Sent Done Done false
31283 9266 Acked Read 9 Sent Done Done false
31284 9267 Acked Read 7 Sent Done Done false
31285 9268 Acked Read 10 Sent Done Done false
31286 9269 Acked Read 6 Sent Done Done false
31287 9270 Acked Read 5 Sent Done Done false
31288 9271 Acked Read 5 Sent Done Done false
31289 9272 Acked Flush 0 Sent Done Done false
31290 9273 Acked Write 6 Sent Done Done false
31291 9274 Acked Read 2 Sent Done Done false
31292 9275 Acked Write 3 Sent Done Done false
31293 9276 Acked Write 6 Sent Done Done false
31294 9277 Acked Write 4 Sent Done Done false
31295 9278 Acked Read 4 Sent Done Done false
31296 9279 Acked Read 9 Sent Done Done false
31297 9280 Acked Write 4 Sent Done Done false
31298 9281 Acked Write 4 Sent Done Done false
31299 9282 Acked Read 5 Sent Done Done false
31300 9283 Acked Write 5 Sent Done Done false
31301 9284 Acked Flush 0 Sent Done Done false
31302 9285 Acked Write 2 Sent Done Done false
31303 9286 Acked Write 4 Sent Done Done false
31304 9287 Acked Write 4 Sent Done Done false
31305 9288 Acked Read 4 Sent Done Done false
31306 9289 Acked Write 4 Sent Done Done false
31307 9290 Acked Write 9 Sent Done Done false
31308 9291 Acked Read 3 Sent Done Done false
31309 9292 Acked Read 3 Sent Done Done false
31310 9293 Acked Flush 0 Sent Done Done false
31311 9294 Acked Write 9 Sent Done Done false
31312 9295 Acked Write 2 Sent Done Done false
31313 9296 Acked Read 2 Sent Done Done false
31314 9297 Acked Write 2 Sent Done Done false
31315 9298 Acked Write 7 Sent Done Done false
31316 9299 Acked Read 4 Sent Done Done false
31317 9300 Acked Read 3 Sent Done Done false
31318 9301 Acked Flush 0 Sent Done Done false
31319 9302 Acked Read 6 Sent Done Done false
31320 9303 Acked Write 5 Sent Done Done false
31321 9304 Acked Read 5 Sent Done Done false
31322 9305 Acked Write 1 Sent Done Done false
31323 9306 Acked Write 1 Sent Done Done false
31324 9307 Acked Read 9 Sent Done Done false
31325 9308 Acked Write 4 Sent Done Done false
31326 9309 Acked Read 7 Sent Done Done false
31327 9310 Acked Read 5 Sent Done Done false
31328 9311 Acked Read 9 Sent Done Done false
31329 9312 Acked Read 10 Sent Done Done false
31330 9313 Acked Read 3 Sent Done Done false
31331 9314 Acked Write 4 Sent Done Done false
31332 9315 Acked Read 7 Sent Done Done false
31333 9316 Acked Read 8 Sent Done Done false
31334 9317 Acked Read 7 Sent Done Done false
31335 9318 Acked Read 3 Sent Done Done false
31336 9319 Acked Read 7 Sent Done Done false
31337 9320 Acked Read 5 Sent Done Done false
31338 9321 Acked Read 9 Sent Done Done false
31339 9322 Acked Read 6 Sent Done Done false
31340 9323 Acked Read 7 Sent Done Done false
31341 9324 Acked Write 2 Sent Done Done false
31342 9325 Acked Read 7 Sent Done Done false
31343 9326 Acked Read 3 Sent Done Done false
31344 9327 Acked Read 2 Sent Done Done false
31345 9328 Acked Flush 0 Sent Done Done false
31346 9329 Acked Read 6 Sent Done Done false
31347 9330 Acked Read 5 Sent Done Done false
31348 9331 Acked Flush 0 Sent Done Done false
31349 9332 Acked Write 10 Sent Done Done false
31350 9333 Acked Write 3 Sent Done Done false
31351 9334 Acked Read 6 Sent Done Done false
31352 9335 Acked Read 1 Sent Done Done false
31353 9336 Acked Read 5 Sent Done Done false
31354 9337 Acked Write 10 Sent Done Done false
31355 9338 Acked Read 1 Sent Done Done false
31356 9339 Acked Write 5 Sent Done Done false
31357 9340 Acked Read 9 Sent Done Done false
31358 9341 Acked Read 7 Sent Done Done false
31359 9342 Acked Read 3 Sent Done Done false
31360 9343 Acked Read 8 Sent Done Done false
31361 9344 Acked Read 7 Sent Done Done false
31362 9345 Acked Read 6 Sent Done Done false
31363 9346 Acked Write 4 Sent Done Done false
31364 9347 Acked Write 1 Sent Done Done false
31365 9348 Acked Write 7 Sent Done Done false
31366 9349 Acked Read 3 Sent Done Done false
31367 9350 Acked Write 10 Sent Done Done false
31368 9351 Acked Flush 0 Sent Done Done false
31369 9352 Acked Read 7 Sent Done Done false
31370 9353 Acked Read 4 Sent Done Done false
31371 9354 Acked Write 3 Sent Done Done false
31372 9355 Acked Write 8 Sent Done Done false
31373 9356 Acked Read 8 Sent Done Done false
31374 9357 Acked Write 8 Sent Done Done false
31375 9358 Acked Read 10 Sent Done Done false
31376 9359 Acked Write 6 Sent Done Done false
31377 9360 Acked Write 2 Sent Done Done false
31378 9361 Acked Write 5 Sent Done Done false
31379 9362 Acked Read 1 Sent Done Done false
31380 9363 Acked Read 1 Sent Done Done false
31381 9364 Acked Flush 0 Sent Done Done false
31382 9365 Acked Write 6 Sent Done Done false
31383 9366 Acked Flush 0 Sent Done Done false
31384 9367 Acked Flush 0 Sent Done Done false
31385 9368 Acked Read 5 Sent Done Done false
31386 9369 Acked Read 10 Sent Done Done false
31387 9370 Acked Read 3 Sent Done Done false
31388 9371 Acked Read 7 Sent Done Done false
31389 9372 Acked Write 10 Sent Done Done false
31390 9373 Acked Write 1 Sent Done Done false
31391 9374 Acked Read 10 Sent Done Done false
31392 9375 Acked Read 2 Sent Done Done false
31393 9376 Acked Write 5 Sent Done Done false
31394 9377 Acked Read 6 Sent Done Done false
31395 9378 Acked Write 9 Sent Done Done false
31396 9379 Acked Write 5 Sent Done Done false
31397 9380 Acked Read 3 Sent Done Done false
31398 9381 Acked Write 4 Sent Done Done false
31399 9382 Acked Read 10 Sent Done Done false
31400 9383 Acked Read 5 Sent Done Done false
31401 9384 Acked Read 6 Sent Done Done false
31402 9385 Acked Write 10 Sent Done Done false
31403 9386 Acked Read 9 Sent Done Done false
31404 9387 Acked Write 2 Sent Done Done false
31405 9388 Acked Read 2 Sent Done Done false
31406 9389 Acked Write 10 Sent Done Done false
31407 9390 Acked Write 4 Sent Done Done false
31408 9391 Acked Read 2 Sent Done Done false
31409 9392 Acked Write 2 Sent Done Done false
31410 9393 Acked Flush 0 Sent Done Done false
31411 9394 Acked Write 3 Sent Done Done false
31412 9395 Acked Write 3 Sent Done Done false
31413 9396 Acked Read 3 Sent Done Done false
31414 9397 Acked Write 8 Sent Done Done false
31415 9398 Acked Write 8 Sent Done Done false
31416 9399 Acked Write 2 Sent Done Done false
31417 9400 Acked Write 7 Sent Done Done false
31418 9401 Acked Read 6 Sent Done Done false
31419 9402 Acked Write 9 Sent Done Done false
31420 9403 Acked Read 3 Sent Done Done false
31421 9404 Acked Read 7 Sent Done Done false
31422 9405 Acked Flush 0 Sent Done Done false
31423 9406 Acked Read 2 Sent Done Done false
31424 9407 Acked Flush 0 Sent Done Done false
31425 9408 Acked Read 10 Sent Done Done false
31426 9409 Acked Write 8 Sent Done Done false
31427 9410 Acked Write 2 Sent Done Done false
31428 9411 Acked Read 1 Sent Done Done false
31429 9412 Acked Write 9 Sent Done Done false
31430 9413 Acked Write 7 Sent Done Done false
31431 9414 Acked Read 6 Sent Done Done false
31432 9415 Acked Write 5 Sent Done Done false
31433 9416 Acked Write 3 Sent Done Done false
31434 9417 Acked Read 2 Sent Done Done false
31435 9418 Acked Read 6 Sent Done Done false
31436 9419 Acked Write 9 Sent Done Done false
31437 9420 Acked Write 10 Sent Done Done false
31438 9421 Acked Read 2 Sent Done Done false
31439 9422 Acked Flush 0 Sent Done Done false
31440 9423 Acked Write 1 Sent Done Done false
31441 9424 Acked Write 8 Sent Done Done false
31442 9425 Acked Read 10 Sent Done Done false
31443 9426 Acked Flush 0 Sent Done Done false
31444 9427 Acked Write 8 Sent Done Done false
31445 9428 Acked Write 5 Sent Done Done false
31446 9429 Acked Read 10 Sent Done Done false
31447 9430 Acked Read 6 Sent Done Done false
31448 9431 Acked Read 3 Sent Done Done false
31449 9432 Acked Write 6 Sent Done Done false
31450 9433 Acked Write 9 Sent Done Done false
31451 9434 Acked Read 5 Sent Done Done false
31452 9435 Acked Write 2 Sent Done Done false
31453 9436 Acked Flush 0 Sent Done Done false
31454 9437 Acked Write 3 Sent Done Done false
31455 9438 Acked Write 6 Sent Done Done false
31456 9439 Acked Flush 0 Sent Done Done false
31457 9440 Acked Write 6 Sent Done Done false
31458 9441 Acked Read 8 Sent Done Done false
31459 9442 Acked Write 10 Sent Done Done false
31460 9443 Acked Write 8 Sent Done Done false
31461 9444 Acked Write 4 Sent Done Done false
31462 9445 Acked Read 6 Sent Done Done false
31463 9446 Acked Read 10 Sent Done Done false
31464 9447 Acked Read 3 Sent Done Done false
31465 9448 Acked Flush 0 Sent Done Done false
31466 9449 Acked Read 9 Sent Done Done false
31467 9450 Acked Read 3 Sent Done Done false
31468 9451 Acked Read 4 Sent Done Done false
31469 9452 Acked Write 10 Sent Done Done false
31470 STATES DS:0 DS:1 DS:2 TOTAL
31471 Sent 301 0 0 301
31472 Done 0 301 301 602
31473 Skipped 0 0 0 0
31474 Error 0 0 0 0
31475 Last Flush: 9151 9448 9448
31476 Downstairs last five completed: 9151 9150 9149 9148 9147
31477 Upstairs last five completed: 9452 9451 9450 9449 9448
31478 CLIENT: Up:0 ds:301 act:2
31479 [8810] got start action
31480 [8810] I am going to restart
31481 Starting downstairs at port 8810
31482 Make output file at "/tmp/dsc/downstairs-8810.txt"
31483 [8810][0] reports Starting
31484 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1230)
31485 [8810][0] reports Running
31486 ----------------------------------------------------------------
31487 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31488 States: Offline Active Active
31489 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31490 9152 Acked Read 8 Sent Done Done false
31491 9153 Acked Write 8 Sent Done Done false
31492 9154 Acked Write 6 Sent Done Done false
31493 9155 Acked Write 2 Sent Done Done false
31494 9156 Acked Read 10 Sent Done Done false
31495 9157 Acked Read 8 Sent Done Done false
31496 9158 Acked Write 8 Sent Done Done false
31497 9159 Acked Read 6 Sent Done Done false
31498 9160 Acked Read 8 Sent Done Done false
31499 9161 Acked Flush 0 Sent Done Done false
31500 9162 Acked Flush 0 Sent Done Done false
31501 9163 Acked Read 10 Sent Done Done false
31502 9164 Acked Read 7 Sent Done Done false
31503 9165 Acked Read 2 Sent Done Done false
31504 9166 Acked Read 6 Sent Done Done false
31505 9167 Acked Flush 0 Sent Done Done false
31506 9168 Acked Write 6 Sent Done Done false
31507 9169 Acked Write 2 Sent Done Done false
31508 9170 Acked Read 5 Sent Done Done false
31509 9171 Acked Read 10 Sent Done Done false
31510 9172 Acked Read 4 Sent Done Done false
31511 9173 Acked Write 2 Sent Done Done false
31512 9174 Acked Write 5 Sent Done Done false
31513 9175 Acked Write 1 Sent Done Done false
31514 9176 Acked Write 8 Sent Done Done false
31515 9177 Acked Write 3 Sent Done Done false
31516 9178 Acked Read 7 Sent Done Done false
31517 9179 Acked Write 3 Sent Done Done false
31518 9180 Acked Write 5 Sent Done Done false
31519 9181 Acked Flush 0 Sent Done Done false
31520 9182 Acked Read 2 Sent Done Done false
31521 9183 Acked Read 6 Sent Done Done false
31522 9184 Acked Read 1 Sent Done Done false
31523 9185 Acked Read 9 Sent Done Done false
31524 9186 Acked Read 8 Sent Done Done false
31525 9187 Acked Flush 0 Sent Done Done false
31526 9188 Acked Write 6 Sent Done Done false
31527 9189 Acked Write 8 Sent Done Done false
31528 9190 Acked Write 9 Sent Done Done false
31529 9191 Acked Flush 0 Sent Done Done false
31530 9192 Acked Write 2 Sent Done Done false
31531 9193 Acked Read 1 Sent Done Done false
31532 9194 Acked Write 6 Sent Done Done false
31533 9195 Acked Flush 0 Sent Done Done false
31534 9196 Acked Write 4 Sent Done Done false
31535 9197 Acked Read 5 Sent Done Done false
31536 9198 Acked Read 1 Sent Done Done false
31537 9199 Acked Write 10 Sent Done Done false
31538 9200 Acked Read 6 Sent Done Done false
31539 9201 Acked Read 2 Sent Done Done false
31540 9202 Acked Write 4 Sent Done Done false
31541 9203 Acked Flush 0 Sent Done Done false
31542 9204 Acked Write 2 Sent Done Done false
31543 9205 Acked Write 3 Sent Done Done false
31544 9206 Acked Write 7 Sent Done Done false
31545 9207 Acked Read 5 Sent Done Done false
31546 9208 Acked Read 10 Sent Done Done false
31547 9209 Acked Write 2 Sent Done Done false
31548 9210 Acked Write 3 Sent Done Done false
31549 9211 Acked Read 2 Sent Done Done false
31550 9212 Acked Write 7 Sent Done Done false
31551 9213 Acked Write 8 Sent Done Done false
31552 9214 Acked Read 7 Sent Done Done false
31553 9215 Acked Write 10 Sent Done Done false
31554 9216 Acked Read 7 Sent Done Done false
31555 9217 Acked Flush 0 Sent Done Done false
31556 9218 Acked Write 6 Sent Done Done false
31557 9219 Acked Write 6 Sent Done Done false
31558 9220 Acked Read 5 Sent Done Done false
31559 9221 Acked Flush 0 Sent Done Done false
31560 9222 Acked Read 6 Sent Done Done false
31561 9223 Acked Read 10 Sent Done Done false
31562 9224 Acked Read 9 Sent Done Done false
31563 9225 Acked Write 7 Sent Done Done false
31564 9226 Acked Write 9 Sent Done Done false
31565 9227 Acked Flush 0 Sent Done Done false
31566 9228 Acked Read 6 Sent Done Done false
31567 9229 Acked Read 9 Sent Done Done false
31568 9230 Acked Write 7 Sent Done Done false
31569 9231 Acked Read 6 Sent Done Done false
31570 9232 Acked Write 10 Sent Done Done false
31571 9233 Acked Write 9 Sent Done Done false
31572 9234 Acked Read 6 Sent Done Done false
31573 9235 Acked Write 2 Sent Done Done false
31574 9236 Acked Read 8 Sent Done Done false
31575 9237 Acked Write 9 Sent Done Done false
31576 9238 Acked Flush 0 Sent Done Done false
31577 9239 Acked Write 1 Sent Done Done false
31578 9240 Acked Read 8 Sent Done Done false
31579 9241 Acked Read 9 Sent Done Done false
31580 9242 Acked Write 3 Sent Done Done false
31581 9243 Acked Read 2 Sent Done Done false
31582 9244 Acked Read 3 Sent Done Done false
31583 9245 Acked Flush 0 Sent Done Done false
31584 9246 Acked Read 2 Sent Done Done false
31585 9247 Acked Write 10 Sent Done Done false
31586 9248 Acked Flush 0 Sent Done Done false
31587 9249 Acked Read 2 Sent Done Done false
31588 9250 Acked Write 8 Sent Done Done false
31589 9251 Acked Read 2 Sent Done Done false
31590 9252 Acked Write 4 Sent Done Done false
31591 9253 Acked Flush 0 Sent Done Done false
31592 9254 Acked Read 8 Sent Done Done false
31593 9255 Acked Read 5 Sent Done Done false
31594 9256 Acked Read 8 Sent Done Done false
31595 9257 Acked Write 6 Sent Done Done false
31596 9258 Acked Write 1 Sent Done Done false
31597 9259 Acked Write 10 Sent Done Done false
31598 9260 Acked Read 6 Sent Done Done false
31599 9261 Acked Read 6 Sent Done Done false
31600 9262 Acked Write 10 Sent Done Done false
31601 9263 Acked Write 6 Sent Done Done false
31602 9264 Acked Read 6 Sent Done Done false
31603 9265 Acked Read 9 Sent Done Done false
31604 9266 Acked Read 9 Sent Done Done false
31605 9267 Acked Read 7 Sent Done Done false
31606 9268 Acked Read 10 Sent Done Done false
31607 9269 Acked Read 6 Sent Done Done false
31608 9270 Acked Read 5 Sent Done Done false
31609 9271 Acked Read 5 Sent Done Done false
31610 9272 Acked Flush 0 Sent Done Done false
31611 9273 Acked Write 6 Sent Done Done false
31612 9274 Acked Read 2 Sent Done Done false
31613 9275 Acked Write 3 Sent Done Done false
31614 9276 Acked Write 6 Sent Done Done false
31615 9277 Acked Write 4 Sent Done Done false
31616 9278 Acked Read 4 Sent Done Done false
31617 9279 Acked Read 9 Sent Done Done false
31618 9280 Acked Write 4 Sent Done Done false
31619 9281 Acked Write 4 Sent Done Done false
31620 9282 Acked Read 5 Sent Done Done false
31621 9283 Acked Write 5 Sent Done Done false
31622 9284 Acked Flush 0 Sent Done Done false
31623 9285 Acked Write 2 Sent Done Done false
31624 9286 Acked Write 4 Sent Done Done false
31625 9287 Acked Write 4 Sent Done Done false
31626 9288 Acked Read 4 Sent Done Done false
31627 9289 Acked Write 4 Sent Done Done false
31628 9290 Acked Write 9 Sent Done Done false
31629 9291 Acked Read 3 Sent Done Done false
31630 9292 Acked Read 3 Sent Done Done false
31631 9293 Acked Flush 0 Sent Done Done false
31632 9294 Acked Write 9 Sent Done Done false
31633 9295 Acked Write 2 Sent Done Done false
31634 9296 Acked Read 2 Sent Done Done false
31635 9297 Acked Write 2 Sent Done Done false
31636 9298 Acked Write 7 Sent Done Done false
31637 9299 Acked Read 4 Sent Done Done false
31638 9300 Acked Read 3 Sent Done Done false
31639 9301 Acked Flush 0 Sent Done Done false
31640 9302 Acked Read 6 Sent Done Done false
31641 9303 Acked Write 5 Sent Done Done false
31642 9304 Acked Read 5 Sent Done Done false
31643 9305 Acked Write 1 Sent Done Done false
31644 9306 Acked Write 1 Sent Done Done false
31645 9307 Acked Read 9 Sent Done Done false
31646 9308 Acked Write 4 Sent Done Done false
31647 9309 Acked Read 7 Sent Done Done false
31648 9310 Acked Read 5 Sent Done Done false
31649 9311 Acked Read 9 Sent Done Done false
31650 9312 Acked Read 10 Sent Done Done false
31651 9313 Acked Read 3 Sent Done Done false
31652 9314 Acked Write 4 Sent Done Done false
31653 9315 Acked Read 7 Sent Done Done false
31654 9316 Acked Read 8 Sent Done Done false
31655 9317 Acked Read 7 Sent Done Done false
31656 9318 Acked Read 3 Sent Done Done false
31657 9319 Acked Read 7 Sent Done Done false
31658 9320 Acked Read 5 Sent Done Done false
31659 9321 Acked Read 9 Sent Done Done false
31660 9322 Acked Read 6 Sent Done Done false
31661 9323 Acked Read 7 Sent Done Done false
31662 9324 Acked Write 2 Sent Done Done false
31663 9325 Acked Read 7 Sent Done Done false
31664 9326 Acked Read 3 Sent Done Done false
31665 9327 Acked Read 2 Sent Done Done false
31666 9328 Acked Flush 0 Sent Done Done false
31667 9329 Acked Read 6 Sent Done Done false
31668 9330 Acked Read 5 Sent Done Done false
31669 9331 Acked Flush 0 Sent Done Done false
31670 9332 Acked Write 10 Sent Done Done false
31671 9333 Acked Write 3 Sent Done Done false
31672 9334 Acked Read 6 Sent Done Done false
31673 9335 Acked Read 1 Sent Done Done false
31674 9336 Acked Read 5 Sent Done Done false
31675 9337 Acked Write 10 Sent Done Done false
31676 9338 Acked Read 1 Sent Done Done false
31677 9339 Acked Write 5 Sent Done Done false
31678 9340 Acked Read 9 Sent Done Done false
31679 9341 Acked Read 7 Sent Done Done false
31680 9342 Acked Read 3 Sent Done Done false
31681 9343 Acked Read 8 Sent Done Done false
31682 9344 Acked Read 7 Sent Done Done false
31683 9345 Acked Read 6 Sent Done Done false
31684 9346 Acked Write 4 Sent Done Done false
31685 9347 Acked Write 1 Sent Done Done false
31686 9348 Acked Write 7 Sent Done Done false
31687 9349 Acked Read 3 Sent Done Done false
31688 9350 Acked Write 10 Sent Done Done false
31689 9351 Acked Flush 0 Sent Done Done false
31690 9352 Acked Read 7 Sent Done Done false
31691 9353 Acked Read 4 Sent Done Done false
31692 9354 Acked Write 3 Sent Done Done false
31693 9355 Acked Write 8 Sent Done Done false
31694 9356 Acked Read 8 Sent Done Done false
31695 9357 Acked Write 8 Sent Done Done false
31696 9358 Acked Read 10 Sent Done Done false
31697 9359 Acked Write 6 Sent Done Done false
31698 9360 Acked Write 2 Sent Done Done false
31699 9361 Acked Write 5 Sent Done Done false
31700 9362 Acked Read 1 Sent Done Done false
31701 9363 Acked Read 1 Sent Done Done false
31702 9364 Acked Flush 0 Sent Done Done false
31703 9365 Acked Write 6 Sent Done Done false
31704 9366 Acked Flush 0 Sent Done Done false
31705 9367 Acked Flush 0 Sent Done Done false
31706 9368 Acked Read 5 Sent Done Done false
31707 9369 Acked Read 10 Sent Done Done false
31708 9370 Acked Read 3 Sent Done Done false
31709 9371 Acked Read 7 Sent Done Done false
31710 9372 Acked Write 10 Sent Done Done false
31711 9373 Acked Write 1 Sent Done Done false
31712 9374 Acked Read 10 Sent Done Done false
31713 9375 Acked Read 2 Sent Done Done false
31714 9376 Acked Write 5 Sent Done Done false
31715 9377 Acked Read 6 Sent Done Done false
31716 9378 Acked Write 9 Sent Done Done false
31717 9379 Acked Write 5 Sent Done Done false
31718 9380 Acked Read 3 Sent Done Done false
31719 9381 Acked Write 4 Sent Done Done false
31720 9382 Acked Read 10 Sent Done Done false
31721 9383 Acked Read 5 Sent Done Done false
31722 9384 Acked Read 6 Sent Done Done false
31723 9385 Acked Write 10 Sent Done Done false
31724 9386 Acked Read 9 Sent Done Done false
31725 9387 Acked Write 2 Sent Done Done false
31726 9388 Acked Read 2 Sent Done Done false
31727 9389 Acked Write 10 Sent Done Done false
31728 9390 Acked Write 4 Sent Done Done false
31729 9391 Acked Read 2 Sent Done Done false
31730 9392 Acked Write 2 Sent Done Done false
31731 9393 Acked Flush 0 Sent Done Done false
31732 9394 Acked Write 3 Sent Done Done false
31733 9395 Acked Write 3 Sent Done Done false
31734 9396 Acked Read 3 Sent Done Done false
31735 9397 Acked Write 8 Sent Done Done false
31736 9398 Acked Write 8 Sent Done Done false
31737 9399 Acked Write 2 Sent Done Done false
31738 9400 Acked Write 7 Sent Done Done false
31739 9401 Acked Read 6 Sent Done Done false
31740 9402 Acked Write 9 Sent Done Done false
31741 9403 Acked Read 3 Sent Done Done false
31742 9404 Acked Read 7 Sent Done Done false
31743 9405 Acked Flush 0 Sent Done Done false
31744 9406 Acked Read 2 Sent Done Done false
31745 9407 Acked Flush 0 Sent Done Done false
31746 9408 Acked Read 10 Sent Done Done false
31747 9409 Acked Write 8 Sent Done Done false
31748 9410 Acked Write 2 Sent Done Done false
31749 9411 Acked Read 1 Sent Done Done false
31750 9412 Acked Write 9 Sent Done Done false
31751 9413 Acked Write 7 Sent Done Done false
31752 9414 Acked Read 6 Sent Done Done false
31753 9415 Acked Write 5 Sent Done Done false
31754 9416 Acked Write 3 Sent Done Done false
31755 9417 Acked Read 2 Sent Done Done false
31756 9418 Acked Read 6 Sent Done Done false
31757 9419 Acked Write 9 Sent Done Done false
31758 9420 Acked Write 10 Sent Done Done false
31759 9421 Acked Read 2 Sent Done Done false
31760 9422 Acked Flush 0 Sent Done Done false
31761 9423 Acked Write 1 Sent Done Done false
31762 9424 Acked Write 8 Sent Done Done false
31763 9425 Acked Read 10 Sent Done Done false
31764 9426 Acked Flush 0 Sent Done Done false
31765 9427 Acked Write 8 Sent Done Done false
31766 9428 Acked Write 5 Sent Done Done false
31767 9429 Acked Read 10 Sent Done Done false
31768 9430 Acked Read 6 Sent Done Done false
31769 9431 Acked Read 3 Sent Done Done false
31770 9432 Acked Write 6 Sent Done Done false
31771 9433 Acked Write 9 Sent Done Done false
31772 9434 Acked Read 5 Sent Done Done false
31773 9435 Acked Write 2 Sent Done Done false
31774 9436 Acked Flush 0 Sent Done Done false
31775 9437 Acked Write 3 Sent Done Done false
31776 9438 Acked Write 6 Sent Done Done false
31777 9439 Acked Flush 0 Sent Done Done false
31778 9440 Acked Write 6 Sent Done Done false
31779 9441 Acked Read 8 Sent Done Done false
31780 9442 Acked Write 10 Sent Done Done false
31781 9443 Acked Write 8 Sent Done Done false
31782 9444 Acked Write 4 Sent Done Done false
31783 9445 Acked Read 6 Sent Done Done false
31784 9446 Acked Read 10 Sent Done Done false
31785 9447 Acked Read 3 Sent Done Done false
31786 9448 Acked Flush 0 Sent Done Done false
31787 9449 Acked Read 9 Sent Done Done false
31788 9450 Acked Read 3 Sent Done Done false
31789 9451 Acked Read 4 Sent Done Done false
31790 9452 Acked Write 10 Sent Done Done false
31791 9453 Acked Flush 0 Sent Done Done false
31792 STATES DS:0 DS:1 DS:2 TOTAL
31793 Sent 302 0 0 302
31794 Done 0 302 302 604
31795 Skipped 0 0 0 0
31796 Error 0 0 0 0
31797 Last Flush: 9151 9453 9453
31798 Downstairs last five completed: 9151 9150 9149 9148 9147
31799 Upstairs last five completed: 9453 9452 9451 9450 9449
31800 CLIENT: Up:0 ds:302 act:2
318012025-01-15T05:25:39.525ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318022025-01-15T05:25:39.525ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52715) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318032025-01-15T05:25:39.526ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318042025-01-15T05:25:39.527ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318052025-01-15T05:25:39.527ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318062025-01-15T05:25:39.527ZINFOcrucible: send last flush ID to this DS: 9151 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318072025-01-15T05:25:39.527ZINFOcrucible: Replied this last flush ID: 9151 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318082025-01-15T05:25:39.527ZINFOcrucible: [0] client re-new 302 jobs since flush 9151 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318092025-01-15T05:25:39.527ZINFOcrucible: [0] Replayed 302 jobs since flush: 9151 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
31810 ----------------------------------------------------------------
31811 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31812 Downstairs last five completed: 9453 9452 9451 9450 9449
31813 Upstairs last five completed: 9453 9452 9451 9450 9449
31814 CLIENT: Up:0 ds:0 act:3
31815 Replay: All jobs finished
31816 Main task has work to do, go find it
31817 got dsc Stop(2)
31818 stop 2
31819 Jan 15 05:25:41.765 INFO request completed, latency_us: 331, response_code: 204, uri: /stop/cid/2, method: POST, req_id: e43aefea-327b-40fa-8cfa-840eed4c0fd9, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
31820 [8830] Got stop action so:false kr:false
31821 Jan 15 05:25:41.766 INFO request completed, latency_us: 177, response_code: 200, uri: /state/cid/2, method: GET, req_id: 088bd360-d34f-4c5b-a15d-b9596958a2c0, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
318222025-01-15T05:25:41.766ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318232025-01-15T05:25:41.766ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318242025-01-15T05:25:41.766ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
318252025-01-15T05:25:41.766ZINFOcrucible: client task is exiting = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
31826 [8830] Exited with: ExitStatus(unix_wait_status(9))
31827 [8830][2] reports Exit
31828 Jan 15 05:25:45.769 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/2, method: GET, req_id: a8d99902-01da-43ca-bebe-edaf73877840, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
31829 001/300 Write block 2219 len 8192 data: 3 4
31830 002/300 Read block 4439 len 40960
31831 003/300 Write block 4054 len 40960 data: 6 5 6 7 5 5 2 2 2 3
31832 004/300 Write block 1908 len 24576 data: 7 7 5 5 6 6
31833 005/300 Read block 648 len 4096
31834 006/300 Read block 3950 len 16384
31835 007/300 Read block 1983 len 40960
31836 008/300 Read block 3170 len 24576
31837 009/300 Flush
31838 010/300 Read block 4778 len 8192
31839 011/300 Write block 1206 len 32768 data: 9 9 8 9 6 5 6 4
31840 012/300 Write block 2216 len 8192 data: 6 4
31841 013/300 Write block 1892 len 28672 data: 7 9 10 9 7 7 8
31842 014/300 Flush
31843 015/300 Write block 3163 len 8192 data: 7 7
31844 016/300 Flush
31845 017/300 Read block 2670 len 12288
31846 018/300 Read block 4719 len 4096
31847 019/300 Read block 4343 len 36864
31848 020/300 Write block 2595 len 20480 data: 8 5 7 7 6
31849 021/300 Read block 2652 len 36864
31850 022/300 Write block 1323 len 36864 data: 5 6 5 8 7 7 6 3 2
31851 023/300 Flush
31852 024/300 Flush
31853 025/300 Flush
31854 026/300 Write block 4305 len 24576 data: 4 4 3 4 4 4
31855 027/300 Flush
31856 028/300 Write block 3537 len 8192 data: 7 5
31857 029/300 Write block 91 len 8192 data: 5 6
31858 030/300 Flush
31859 031/300 Flush
31860 032/300 Read block 767 len 8192
31861 033/300 Write block 893 len 4096 data: 7
31862 034/300 Read block 2781 len 40960
31863 035/300 Write block 2533 len 32768 data: 8 7 7 5 5 6 5 8
31864 036/300 Write block 1774 len 12288 data: 6 6 5
31865 037/300 Read block 1584 len 4096
31866 038/300 Read block 2696 len 36864
31867 039/300 Write block 5 len 40960 data: 4 6 6 6 7 8 8 6 7 7
31868 040/300 Read block 4128 len 24576
31869 041/300 Read block 4241 len 16384
31870 042/300 Read block 1557 len 8192
31871 043/300 Write block 1746 len 4096 data: 3
31872 044/300 Write block 2247 len 40960 data: 6 5 5 4 4 3 4 6 7 8
31873 045/300 Write block 2948 len 12288 data: 7 6 7
31874 046/300 Write block 3279 len 12288 data: 9 8 6
31875 047/300 Read block 3307 len 4096
31876 048/300 Write block 4842 len 8192 data: 3 4
31877 049/300 Read block 3654 len 28672
31878 050/300 Write block 2319 len 28672 data: 7 6 5 4 4 4 5
31879 051/300 Read block 1983 len 32768
31880 052/300 Read block 2574 len 32768
31881 053/300 Read block 2881 len 36864
31882 054/300 Read block 3987 len 4096
31883 055/300 Read block 1673 len 32768
31884 056/300 Read block 1665 len 36864
31885 057/300 Read block 1908 len 4096
31886 058/300 Write block 238 len 40960 data: 4 5 6 5 5 4 3 5 7 8
31887 059/300 Read block 4844 len 24576
31888 060/300 Write block 3769 len 36864 data: 5 7 6 6 7 6 5 5 5
31889 061/300 Write block 2854 len 24576 data: 5 3 4 4 5 5
31890 062/300 Read block 4671 len 4096
31891 063/300 Read block 279 len 32768
31892 064/300 Flush
31893 065/300 Write block 1393 len 4096 data: 4
31894 066/300 Read block 2507 len 40960
31895 067/300 Flush
31896 068/300 Read block 1441 len 8192
31897 069/300 Flush
31898 070/300 Read block 4687 len 28672
31899 071/300 Read block 3417 len 16384
31900 072/300 Write block 2585 len 40960 data: 5 8 7 9 8 10 9 10 9 8
31901 073/300 Read block 2167 len 36864
31902 074/300 Read block 1469 len 16384
31903 075/300 Read block 3897 len 36864
31904 076/300 Write block 2151 len 4096 data: 4
31905 077/300 Write block 1253 len 20480 data: 6 7 8 7 7
31906 078/300 Flush
31907 079/300 Flush
31908 080/300 Read block 793 len 24576
31909 081/300 Write block 2160 len 36864 data: 3 4 5 6 7 5 6 7 7
31910 082/300 Read block 369 len 36864
31911 083/300 Write block 1740 len 4096 data: 3
31912 084/300 Write block 2512 len 32768 data: 6 8 6 5 5 5 5 4
31913 085/300 Read block 3846 len 40960
31914 086/300 Write block 4485 len 4096 data: 8
31915 087/300 Read block 1920 len 4096
31916 088/300 Read block 2320 len 32768
31917 089/300 Read block 110 len 24576
31918 090/300 Write block 3735 len 36864 data: 3 4 5 7 6 9 7 7 5
31919 091/300 Flush
31920 092/300 Read block 1895 len 8192
31921 093/300 Read block 145 len 40960
31922 094/300 Read block 843 len 20480
31923 095/300 Flush
31924 096/300 Read block 3141 len 4096
31925 097/300 Write block 1879 len 40960 data: 6 4 7 7 7 7 6 6 4 7
31926 098/300 Read block 720 len 8192
31927 099/300 Read block 3229 len 20480
31928 100/300 Read block 2298 len 8192
31929 101/300 Flush
31930 102/300 Read block 3286 len 40960
31931 103/300 Read block 472 len 40960
31932 104/300 Write block 1817 len 40960 data: 8 3 4 3 3 3 4 3 4 5
31933 105/300 Flush
31934 106/300 Write block 2236 len 12288 data: 6 6 7
31935 107/300 Read block 90 len 28672
31936 108/300 Write block 2432 len 40960 data: 11 14 12 8 8 7 7 9 9 7
31937 109/300 Write block 1892 len 24576 data: 8 10 11 10 8 8
31938 110/300 Write block 713 len 40960 data: 4 5 5 4 5 6 6 8 7 7
31939 111/300 Write block 4848 len 32768 data: 6 7 4 2 4 5 4 4
31940 112/300 Read block 1432 len 8192
31941 113/300 Read block 1302 len 36864
31942 114/300 Write block 3 len 32768 data: 4 4 5 7 7 7 8 9
31943 115/300 Flush
31944 116/300 Read block 2307 len 12288
31945 117/300 Flush
31946 118/300 Read block 244 len 16384
31947 119/300 Write block 2364 len 8192 data: 8 8
31948 120/300 Write block 4833 len 12288 data: 4 6 6
31949 121/300 Read block 4399 len 40960
31950 122/300 Read block 4586 len 32768
31951 123/300 Read block 228 len 40960
31952 124/300 Write block 2784 len 16384 data: 7 5 7 7
31953 125/300 Write block 4035 len 36864 data: 5 6 6 5 5 5 4 5 5
31954 126/300 Read block 4750 len 32768
31955 127/300 Write block 1434 len 12288 data: 8 7 7
31956 128/300 Write block 3956 len 8192 data: 6 5
31957 129/300 Write block 2674 len 36864 data: 7 6 7 5 6 6 6 5 6
31958 130/300 Flush
31959 131/300 Read block 88 len 24576
31960 132/300 Write block 1264 len 28672 data: 5 7 7 9 8 5 5
31961 133/300 Write block 3437 len 28672 data: 4 6 5 7 7 9 8
31962 134/300 Read block 957 len 40960
31963 135/300 Flush
31964 136/300 Write block 3510 len 40960 data: 7 9 5 5 5 5 4 3 3 4
31965 137/300 Read block 3600 len 36864
31966 138/300 Read block 280 len 28672
31967 139/300 Read block 4598 len 32768
31968 140/300 Read block 71 len 12288
31969 141/300 Write block 2746 len 4096 data: 5
31970 142/300 Read block 2714 len 4096
31971 143/300 Write block 1468 len 16384 data: 6 6 6 5
31972 144/300 Read block 2356 len 40960
31973 145/300 Read block 4639 len 20480
31974 146/300 Read block 2970 len 28672
31975 147/300 Write block 1881 len 8192 data: 8 8
31976 148/300 Read block 4989 len 32768
31977 149/300 Write block 4050 len 4096 data: 8
31978 150/300 Read block 344 len 40960
31979 151/300 Write block 3392 len 24576 data: 6 5 4 5 5 5
31980 152/300 Read block 3259 len 40960
31981 153/300 Write block 4374 len 12288 data: 5 4 5
31982 154/300 Read block 3710 len 24576
31983 155/300 Read block 1744 len 4096
31984 156/300 Write block 377 len 36864 data: 10 7 8 8 8 9 6 6 4
31985 157/300 Write block 3531 len 40960 data: 6 9 8 7 8 6 8 6 6 7
31986 158/300 Write block 4196 len 40960 data: 8 7 6 4 3 3 3 3 3 4
31987 159/300 Read block 2962 len 8192
31988 160/300 Write block 2961 len 36864 data: 5 5 4 4 6 3 3 2 2
31989 161/300 Flush
31990 162/300 Read block 3924 len 16384
31991 163/300 Write block 112 len 24576 data: 6 5 5 7 5 5
31992 164/300 Write block 1245 len 4096 data: 5
31993 165/300 Read block 1085 len 32768
31994 166/300 Flush
31995 167/300 Read block 2312 len 24576
31996 168/300 Read block 4117 len 36864
31997 169/300 Write block 3868 len 24576 data: 4 6 7 7 9 9
31998 170/300 Read block 2016 len 16384
31999 171/300 Write block 1490 len 24576 data: 3 5 6 7 6 6
32000 172/300 Write block 189 len 40960 data: 4 4 6 6 4 4 5 7 6 4
32001 173/300 Read block 895 len 28672
32002 174/300 Write block 2211 len 16384 data: 7 5 5 5
32003 175/300 Write block 3271 len 12288 data: 6 7 8
32004 176/300 Write block 4747 len 28672 data: 7 4 3 4 5 5 6
32005 177/300 Write block 1794 len 32768 data: 4 2 4 4 4 4 7 6
32006 178/300 Read block 3810 len 40960
32007 179/300 Read block 183 len 36864
32008 180/300 Read block 1325 len 32768
32009 181/300 Write block 310 len 8192 data: 6 6
32010 182/300 Flush
32011 183/300 Write block 2556 len 12288 data: 3 4 6
32012 184/300 Write block 2561 len 20480 data: 4 6 7 7 6
32013 185/300 Read block 4 len 24576
32014 186/300 Flush
32015 187/300 Read block 4157 len 40960
32016 188/300 Write block 1705 len 4096 data: 7
32017 189/300 Read block 4130 len 36864
32018 190/300 Read block 1096 len 16384
32019 191/300 Read block 80 len 24576
32020 192/300 Write block 741 len 16384 data: 7 7 9 10
32021 193/300 Flush
32022 194/300 Read block 2851 len 20480
32023 195/300 Flush
32024 196/300 Flush
32025 197/300 Read block 3651 len 32768
32026 198/300 Read block 2830 len 40960
32027 199/300 Write block 1856 len 28672 data: 5 5 3 3 3 5 5
32028 200/300 Write block 4799 len 12288 data: 5 6 6
32029 201/300 Read block 1771 len 28672
32030 202/300 Read block 4626 len 40960
32031 203/300 Read block 4194 len 36864
32032 204/300 Write block 3180 len 16384 data: 7 8 10 10
32033 205/300 Write block 3487 len 20480 data: 5 6 6 5 2
32034 206/300 Read block 1899 len 12288
32035 207/300 Write block 1346 len 28672 data: 7 8 8 7 6 5 4
32036 208/300 Read block 119 len 24576
32037 209/300 Write block 2216 len 16384 data: 7 5 4 4
32038 210/300 Write block 903 len 36864 data: 6 6 3 4 4 3 5 6 6
32039 211/300 Read block 3848 len 12288
32040 212/300 Flush
32041 213/300 Read block 1406 len 28672
32042 214/300 Read block 2242 len 8192
32043 215/300 Write block 927 len 20480 data: 5 5 5 5 5
32044 216/300 Write block 3594 len 32768 data: 3 2 3 4 4 5 7 7
32045 217/300 Read block 4376 len 32768
32046 218/300 Write block 1813 len 28672 data: 8 8 8 9 9 4 5
32047 219/300 Flush
32048 220/300 Write block 4335 len 24576 data: 6 6 6 6 6 6
32049 221/300 Write block 2101 len 8192 data: 5 5
32050 222/300 Read block 4011 len 40960
32051 223/300 Read block 2356 len 40960
32052 224/300 Read block 3032 len 24576
32053 225/300 Write block 3334 len 20480 data: 9 8 10 10 10
32054 226/300 Write block 2431 len 24576 data: 7 12 15 13 9 9
32055 227/300 Flush
32056 228/300 Write block 2913 len 16384 data: 4 4 5 7
32057 229/300 Write block 4067 len 12288 data: 4 3 2
32058 230/300 Read block 2460 len 40960
32059 231/300 Write block 384 len 4096 data: 7
32060 232/300 Write block 778 len 12288 data: 5 4 3
32061 233/300 Write block 4061 len 28672 data: 3 3 4 3 3 2 5
32062 234/300 Read block 3169 len 20480
32063 235/300 Write block 211 len 4096 data: 3
32064 236/300 Read block 211 len 40960
32065 237/300 Read block 1469 len 20480
32066 238/300 Read block 3226 len 20480
32067 239/300 Read block 4109 len 16384
32068 240/300 Read block 4584 len 36864
32069 241/300 Write block 1279 len 32768 data: 4 3 3 2 2 2 3 4
32070 242/300 Read block 1499 len 36864
32071 243/300 Write block 2168 len 4096 data: 8
32072 244/300 Write block 2960 len 24576 data: 5 6 6 5 5 7
32073 245/300 Read block 1599 len 36864
32074 246/300 Flush
32075 247/300 Flush
32076 248/300 Flush
32077 249/300 Read block 3705 len 12288
32078 250/300 Read block 3888 len 12288
32079 251/300 Write block 4028 len 24576 data: 6 9 7 6 6 6
32080 252/300 Write block 4610 len 8192 data: 5 5
32081 253/300 Read block 708 len 16384
32082 254/300 Write block 1556 len 28672 data: 4 4 4 4 4 5 5
32083 255/300 Read block 2029 len 20480
32084 256/300 Write block 306 len 36864 data: 3 4 5 5 7 7 5 4 3
32085 257/300 Read block 4552 len 40960
32086 258/300 Write block 2212 len 32768 data: 6 6 6 6 8 6 5 5
32087 259/300 Read block 3191 len 16384
32088 260/300 Read block 1292 len 4096
32089 261/300 Flush
32090 262/300 Write block 1940 len 4096 data: 6
32091 263/300 Read block 1786 len 12288
32092 264/300 Write block 2770 len 20480 data: 4 5 6 6 6
32093 265/300 Read block 3114 len 4096
32094 266/300 Read block 2308 len 32768
32095 267/300 Flush
32096 268/300 Write block 1928 len 20480 data: 9 9 9 6 7
32097 269/300 Read block 276 len 36864
32098 270/300 Read block 974 len 8192
32099 271/300 Write block 3448 len 20480 data: 8 9 9 9 9
32100 272/300 Read block 1259 len 4096
32101 273/300 Read block 3581 len 4096
32102 274/300 Write block 1684 len 20480 data: 5 6 5 6 5
32103 275/300 Write block 2524 len 4096 data: 5
32104 276/300 Write block 386 len 20480 data: 4 4 3 3 4
32105 277/300 Write block 2898 len 32768 data: 6 6 6 5 4 4 5 5
32106 278/300 Read block 3271 len 12288
32107 279/300 Write block 1232 len 20480 data: 6 4 2 2 2
32108 280/300 Flush
32109 281/300 Flush
32110 282/300 Read block 4991 len 36864
32111 283/300 Write block 3835 len 16384 data: 10 10 10 9
32112 284/300 Write block 713 len 32768 data: 5 6 6 5 6 7 7 9
32113 285/300 Write block 4907 len 12288 data: 6 7 9
32114 286/300 Flush
32115 287/300 Read block 4075 len 8192
32116 288/300 Write block 1511 len 4096 data: 4
32117 289/300 Write block 1144 len 28672 data: 3 5 6 9 9 6 8
32118 290/300 Read block 805 len 8192
32119 291/300 Read block 1209 len 8192
32120 292/300 Read block 3825 len 4096
32121 293/300 Read block 1999 len 40960
32122 294/300 Write block 4358 len 20480 data: 5 7 6 6 5
32123 295/300 Read block 4548 len 12288
32124 296/300 Read block 3467 len 12288
32125 297/300 Read block 2358 len 4096
32126 298/300 Write block 3789 len 20480 data: 5 6 5 5 4
32127 299/300 Write block 1103 len 36864 data: 5 4 4 3 2 3 4 4 6
32128 300/300 Write block 3361 len 8192 data: 4 5
32129 301/300 Read block 2636 len 24576
32130 Main task has work to do, go find it
32131 got dsc Start(2)
32132 start 2
32133 Jan 15 05:25:45.999 INFO request completed, latency_us: 197, response_code: 204, uri: /start/cid/2, method: POST, req_id: 99cb0190-8db0-42a2-b95c-fa2dff05d1b5, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
32134 Replay: started 2, returned:Ok(())
32135 ----------------------------------------------------------------
32136 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
32137 States: Active Active Offline
32138 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32139 9454 Acked Write 2 Done Done Sent false
32140 9455 Acked Read 10 Done Done Sent false
32141 9456 Acked Write 10 Done Done Sent false
32142 9457 Acked Write 6 Done Done Sent false
32143 9458 Acked Read 1 Done Done Sent false
32144 9459 Acked Read 4 Done Done Sent false
32145 9460 Acked Read 10 Done Done Sent false
32146 9461 Acked Read 6 Done Done Sent false
32147 9462 Acked Flush 0 Done Done Sent false
32148 9463 Acked Read 2 Done Done Sent false
32149 9464 Acked Write 8 Done Done Sent false
32150 9465 Acked Write 2 Done Done Sent false
32151 9466 Acked Write 7 Done Done Sent false
32152 9467 Acked Flush 0 Done Done Sent false
32153 9468 Acked Write 2 Done Done Sent false
32154 9469 Acked Flush 0 Done Done Sent false
32155 9470 Acked Read 3 Done Done Sent false
32156 9471 Acked Read 1 Done Done Sent false
32157 9472 Acked Read 9 Done Done Sent false
32158 9473 Acked Write 5 Done Done Sent false
32159 9474 Acked Read 9 Done Done Sent false
32160 9475 Acked Write 9 Done Done Sent false
32161 9476 Acked Flush 0 Done Done Sent false
32162 9477 Acked Flush 0 Done Done Sent false
32163 9478 Acked Flush 0 Done Done Sent false
32164 9479 Acked Write 6 Done Done Sent false
32165 9480 Acked Flush 0 Done Done Sent false
32166 9481 Acked Write 2 Done Done Sent false
32167 9482 Acked Write 2 Done Done Sent false
32168 9483 Acked Flush 0 Done Done Sent false
32169 9484 Acked Flush 0 Done Done Sent false
32170 9485 Acked Read 2 Done Done Sent false
32171 9486 Acked Write 1 Done Done Sent false
32172 9487 Acked Read 10 Done Done Sent false
32173 9488 Acked Write 8 Done Done Sent false
32174 9489 Acked Write 3 Done Done Sent false
32175 9490 Acked Read 1 Done Done Sent false
32176 9491 Acked Read 9 Done Done Sent false
32177 9492 Acked Write 10 Done Done Sent false
32178 9493 Acked Read 6 Done Done Sent false
32179 9494 Acked Read 4 Done Done Sent false
32180 9495 Acked Read 2 Done Done Sent false
32181 9496 Acked Write 1 Done Done Sent false
32182 9497 Acked Write 10 Done Done Sent false
32183 9498 Acked Write 3 Done Done Sent false
32184 9499 Acked Write 3 Done Done Sent false
32185 9500 Acked Read 1 Done Done Sent false
32186 9501 Acked Write 2 Done Done Sent false
32187 9502 Acked Read 7 Done Done Sent false
32188 9503 Acked Write 7 Done Done Sent false
32189 9504 Acked Read 8 Done Done Sent false
32190 9505 Acked Read 8 Done Done Sent false
32191 9506 Acked Read 9 Done Done Sent false
32192 9507 Acked Read 1 Done Done Sent false
32193 9508 Acked Read 8 Done Done Sent false
32194 9509 Acked Read 9 Done Done Sent false
32195 9510 Acked Read 1 Done Done Sent false
32196 9511 Acked Write 10 Done Done Sent false
32197 9512 Acked Read 6 Done Done Sent false
32198 9513 Acked Write 9 Done Done Sent false
32199 9514 Acked Write 6 Done Done Sent false
32200 9515 Acked Read 1 Done Done Sent false
32201 9516 Acked Read 8 Done Done Sent false
32202 9517 Acked Flush 0 Done Done Sent false
32203 9518 Acked Write 1 Done Done Sent false
32204 9519 Acked Read 10 Done Done Sent false
32205 9520 Acked Flush 0 Done Done Sent false
32206 9521 Acked Read 2 Done Done Sent false
32207 9522 Acked Flush 0 Done Done Sent false
32208 9523 Acked Read 7 Done Done Sent false
32209 9524 Acked Read 4 Done Done Sent false
32210 9525 Acked Write 10 Done Done Sent false
32211 9526 Acked Read 9 Done Done Sent false
32212 9527 Acked Read 4 Done Done Sent false
32213 9528 Acked Read 9 Done Done Sent false
32214 9529 Acked Write 1 Done Done Sent false
32215 9530 Acked Write 5 Done Done Sent false
32216 9531 Acked Flush 0 Done Done Sent false
32217 9532 Acked Flush 0 Done Done Sent false
32218 9533 Acked Read 6 Done Done Sent false
32219 9534 Acked Write 9 Done Done Sent false
32220 9535 Acked Read 9 Done Done Sent false
32221 9536 Acked Write 1 Done Done Sent false
32222 9537 Acked Write 8 Done Done Sent false
32223 9538 Acked Read 10 Done Done Sent false
32224 9539 Acked Write 1 Done Done Sent false
32225 9540 Acked Read 1 Done Done Sent false
32226 9541 Acked Read 8 Done Done Sent false
32227 9542 Acked Read 6 Done Done Sent false
32228 9543 Acked Write 9 Done Done Sent false
32229 9544 Acked Flush 0 Done Done Sent false
32230 9545 Acked Read 2 Done Done Sent false
32231 9546 Acked Read 10 Done Done Sent false
32232 9547 Acked Read 5 Done Done Sent false
32233 9548 Acked Flush 0 Done Done Sent false
32234 9549 Acked Read 1 Done Done Sent false
32235 9550 Acked Write 10 Done Done Sent false
32236 9551 Acked Read 2 Done Done Sent false
32237 9552 Acked Read 5 Done Done Sent false
32238 9553 Acked Read 2 Done Done Sent false
32239 9554 Acked Flush 0 Done Done Sent false
32240 9555 Acked Read 10 Done Done Sent false
32241 9556 Acked Read 10 Done Done Sent false
32242 9557 Acked Write 10 Done Done Sent false
32243 9558 Acked Flush 0 Done Done Sent false
32244 9559 Acked Write 3 Done Done Sent false
32245 9560 Acked Read 7 Done Done Sent false
32246 9561 Acked Write 10 Done Done Sent false
32247 9562 Acked Write 6 Done Done Sent false
32248 9563 Acked Write 10 Done Done Sent false
32249 9564 Acked Write 8 Done Done Sent false
32250 9565 Acked Read 2 Done Done Sent false
32251 9566 Acked Read 9 Done Done Sent false
32252 9567 Acked Write 8 Done Done Sent false
32253 9568 Acked Flush 0 Done Done Sent false
32254 9569 Acked Read 3 Done Done Sent false
32255 9570 Acked Flush 0 Done Done Sent false
32256 9571 Acked Read 4 Done Done Sent false
32257 9572 Acked Write 2 Done Done Sent false
32258 9573 Acked Write 3 Done Done Sent false
32259 9574 Acked Read 10 Done Done Sent false
32260 9575 Acked Read 8 Done Done Sent false
32261 9576 Acked Read 10 Done Done Sent false
32262 9577 Acked Write 4 Done Done Sent false
32263 9578 Acked Write 9 Done Done Sent false
32264 9579 Acked Read 8 Done Done Sent false
32265 9580 Acked Write 3 Done Done Sent false
32266 9581 Acked Write 2 Done Done Sent false
32267 9582 Acked Write 9 Done Done Sent false
32268 9583 Acked Flush 0 Done Done Sent false
32269 9584 Acked Read 6 Done Done Sent false
32270 9585 Acked Write 7 Done Done Sent false
32271 9586 Acked Write 7 Done Done Sent false
32272 9587 Acked Read 10 Done Done Sent false
32273 9588 Acked Flush 0 Done Done Sent false
32274 9589 Acked Write 10 Done Done Sent false
32275 9590 Acked Read 9 Done Done Sent false
32276 9591 Acked Read 7 Done Done Sent false
32277 9592 Acked Read 8 Done Done Sent false
32278 9593 Acked Read 3 Done Done Sent false
32279 9594 Acked Write 1 Done Done Sent false
32280 9595 Acked Read 1 Done Done Sent false
32281 9596 Acked Write 4 Done Done Sent false
32282 9597 Acked Read 10 Done Done Sent false
32283 9598 Acked Read 5 Done Done Sent false
32284 9599 Acked Read 7 Done Done Sent false
32285 9600 Acked Write 2 Done Done Sent false
32286 9601 Acked Read 8 Done Done Sent false
32287 9602 Acked Write 1 Done Done Sent false
32288 9603 Acked Read 10 Done Done Sent false
32289 9604 Acked Write 6 Done Done Sent false
32290 9605 Acked Read 10 Done Done Sent false
32291 9606 Acked Write 3 Done Done Sent false
32292 9607 Acked Read 6 Done Done Sent false
32293 9608 Acked Read 1 Done Done Sent false
32294 9609 Acked Write 9 Done Done Sent false
32295 9610 Acked Write 10 Done Done Sent false
32296 9611 Acked Write 10 Done Done Sent false
32297 9612 Acked Read 2 Done Done Sent false
32298 9613 Acked Write 9 Done Done Sent false
32299 9614 Acked Flush 0 Done Done Sent false
32300 9615 Acked Read 4 Done Done Sent false
32301 9616 Acked Write 6 Done Done Sent false
32302 9617 Acked Write 1 Done Done Sent false
32303 9618 Acked Read 8 Done Done Sent false
32304 9619 Acked Flush 0 Done Done Sent false
32305 9620 Acked Read 6 Done Done Sent false
32306 9621 Acked Read 9 Done Done Sent false
32307 9622 Acked Write 6 Done Done Sent false
32308 9623 Acked Read 4 Done Done Sent false
32309 9624 Acked Write 6 Done Done Sent false
32310 9625 Acked Write 10 Done Done Sent false
32311 9626 Acked Read 7 Done Done Sent false
32312 9627 Acked Write 4 Done Done Sent false
32313 9628 Acked Write 3 Done Done Sent false
32314 9629 Acked Write 7 Done Done Sent false
32315 9630 Acked Write 8 Done Done Sent false
32316 9631 Acked Read 10 Done Done Sent false
32317 9632 Acked Read 9 Done Done Sent false
32318 9633 Acked Read 8 Done Done Sent false
32319 9634 Acked Write 2 Done Done Sent false
32320 9635 Acked Flush 0 Done Done Sent false
32321 9636 Acked Write 3 Done Done Sent false
32322 9637 Acked Write 5 Done Done Sent false
32323 9638 Acked Read 6 Done Done Sent false
32324 9639 Acked Flush 0 Done Done Sent false
32325 9640 Acked Read 10 Done Done Sent false
32326 9641 Acked Write 1 Done Done Sent false
32327 9642 Acked Read 9 Done Done Sent false
32328 9643 Acked Read 4 Done Done Sent false
32329 9644 Acked Read 6 Done Done Sent false
32330 9645 Acked Write 4 Done Done Sent false
32331 9646 Acked Flush 0 Done Done Sent false
32332 9647 Acked Read 5 Done Done Sent false
32333 9648 Acked Flush 0 Done Done Sent false
32334 9649 Acked Flush 0 Done Done Sent false
32335 9650 Acked Read 8 Done Done Sent false
32336 9651 Acked Read 10 Done Done Sent false
32337 9652 Acked Write 7 Done Done Sent false
32338 9653 Acked Write 3 Done Done Sent false
32339 9654 Acked Read 7 Done Done Sent false
32340 9655 Acked Read 10 Done Done Sent false
32341 9656 Acked Read 9 Done Done Sent false
32342 9657 Acked Write 4 Done Done Sent false
32343 9658 Acked Write 5 Done Done Sent false
32344 9659 Acked Read 3 Done Done Sent false
32345 9660 Acked Write 7 Done Done Sent false
32346 9661 Acked Read 6 Done Done Sent false
32347 9662 Acked Write 4 Done Done Sent false
32348 9663 Acked Write 9 Done Done Sent false
32349 9664 Acked Read 3 Done Done Sent false
32350 9665 Acked Flush 0 Done Done Sent false
32351 9666 Acked Read 7 Done Done Sent false
32352 9667 Acked Read 2 Done Done Sent false
32353 9668 Acked Write 5 Done Done Sent false
32354 9669 Acked Write 8 Done Done Sent false
32355 9670 Acked Read 8 Done Done Sent false
32356 9671 Acked Write 7 Done Done Sent false
32357 9672 Acked Flush 0 Done Done Sent false
32358 9673 Acked Write 6 Done Done Sent false
32359 9674 Acked Write 2 Done Done Sent false
32360 9675 Acked Read 10 Done Done Sent false
32361 9676 Acked Read 10 Done Done Sent false
32362 9677 Acked Read 6 Done Done Sent false
32363 9678 Acked Write 5 Done Done Sent false
32364 9679 Acked Write 6 Done Done Sent false
32365 9680 Acked Flush 0 Done Done Sent false
32366 9681 Acked Write 4 Done Done Sent false
32367 9682 Acked Write 3 Done Done Sent false
32368 9683 Acked Read 10 Done Done Sent false
32369 9684 Acked Write 1 Done Done Sent false
32370 9685 Acked Write 3 Done Done Sent false
32371 9686 Acked Write 7 Done Done Sent false
32372 9687 Acked Read 5 Done Done Sent false
32373 9688 Acked Write 1 Done Done Sent false
32374 9689 Acked Read 10 Done Done Sent false
32375 9690 Acked Read 5 Done Done Sent false
32376 9691 Acked Read 5 Done Done Sent false
32377 9692 Acked Read 4 Done Done Sent false
32378 9693 Acked Read 9 Done Done Sent false
32379 9694 Acked Write 8 Done Done Sent false
32380 9695 Acked Read 9 Done Done Sent false
32381 9696 Acked Write 1 Done Done Sent false
32382 9697 Acked Write 6 Done Done Sent false
32383 9698 Acked Read 9 Done Done Sent false
32384 9699 Acked Flush 0 Done Done Sent false
32385 9700 Acked Flush 0 Done Done Sent false
32386 9701 Acked Flush 0 Done Done Sent false
32387 9702 Acked Read 3 Done Done Sent false
32388 9703 Acked Read 3 Done Done Sent false
32389 9704 Acked Write 6 Done Done Sent false
32390 9705 Acked Write 2 Done Done Sent false
32391 9706 Acked Read 4 Done Done Sent false
32392 9707 Acked Write 7 Done Done Sent false
32393 9708 Acked Read 5 Done Done Sent false
32394 9709 Acked Write 9 Done Done Sent false
32395 9710 Acked Read 10 Done Done Sent false
32396 9711 Acked Write 8 Done Done Sent false
32397 9712 Acked Read 4 Done Done Sent false
32398 9713 Acked Read 1 Done Done Sent false
32399 9714 Acked Flush 0 Done Done Sent false
32400 9715 Acked Write 1 Done Done Sent false
32401 9716 Acked Read 3 Done Done Sent false
32402 9717 Acked Write 5 Done Done Sent false
32403 9718 Acked Read 1 Done Done Sent false
32404 9719 Acked Read 8 Done Done Sent false
32405 9720 Acked Flush 0 Done Done Sent false
32406 9721 Acked Write 5 Done Done Sent false
32407 9722 Acked Read 9 Done Done Sent false
32408 9723 Acked Read 2 Done Done Sent false
32409 9724 Acked Write 5 Done Done Sent false
32410 9725 Acked Read 1 Done Done Sent false
32411 9726 Acked Read 1 Done Done Sent false
32412 9727 Acked Write 5 Done Done Sent false
32413 9728 Acked Write 1 Done Done Sent false
32414 9729 Acked Write 5 Done Done Sent false
32415 9730 Acked Write 8 Done Done Sent false
32416 9731 Acked Read 3 Done Done Sent false
32417 9732 Acked Write 5 Done Done Sent false
32418 9733 Acked Flush 0 Done Done Sent false
32419 9734 Acked Flush 0 Done Done Sent false
32420 9735 Acked Read 9 Done Done Sent false
32421 9736 Acked Write 4 Done Done Sent false
32422 9737 Acked Write 8 Done Done Sent false
32423 9738 Acked Write 3 Done Done Sent false
32424 9739 Acked Flush 0 Done Done Sent false
32425 9740 Acked Read 2 Done Done Sent false
32426 9741 Acked Write 1 Done Done Sent false
32427 9742 Acked Write 7 Done Done Sent false
32428 9743 Acked Read 2 Done Done Sent false
32429 9744 Acked Read 2 Done Done Sent false
32430 9745 Acked Read 1 Done Done Sent false
32431 9746 Acked Read 10 Done Done Sent false
32432 9747 Acked Write 5 Done Done Sent false
32433 9748 Acked Read 3 Done Done Sent false
32434 9749 Acked Read 3 Done Done Sent false
32435 9750 Acked Read 1 Done Done Sent false
32436 9751 Acked Write 5 Done Done Sent false
32437 9752 Acked Write 9 Done Done Sent false
32438 9753 Acked Write 2 Done Done Sent false
32439 9754 Acked Read 6 Done Done Sent false
32440 STATES DS:0 DS:1 DS:2 TOTAL
32441 Sent 0 0 301 301
32442 Done 301 301 0 602
32443 Skipped 0 0 0 0
32444 Error 0 0 0 0
32445 Last Flush: 9739 9739 9453
32446 Downstairs last five completed: 9453 9452 9451 9450 9449
32447 Upstairs last five completed: 9754 9753 9752 9751 9750
32448 CLIENT: Up:0 ds:301 act:2
32449 [8830] got start action
32450 [8830] I am going to restart
32451 Starting downstairs at port 8830
32452 Make output file at "/tmp/dsc/downstairs-8830.txt"
32453 [8830][2] reports Starting
32454 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1231)
32455 [8830][2] reports Running
32456 ----------------------------------------------------------------
32457 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32458 States: Active Active Offline
32459 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32460 9454 Acked Write 2 Done Done Sent false
32461 9455 Acked Read 10 Done Done Sent false
32462 9456 Acked Write 10 Done Done Sent false
32463 9457 Acked Write 6 Done Done Sent false
32464 9458 Acked Read 1 Done Done Sent false
32465 9459 Acked Read 4 Done Done Sent false
32466 9460 Acked Read 10 Done Done Sent false
32467 9461 Acked Read 6 Done Done Sent false
32468 9462 Acked Flush 0 Done Done Sent false
32469 9463 Acked Read 2 Done Done Sent false
32470 9464 Acked Write 8 Done Done Sent false
32471 9465 Acked Write 2 Done Done Sent false
32472 9466 Acked Write 7 Done Done Sent false
32473 9467 Acked Flush 0 Done Done Sent false
32474 9468 Acked Write 2 Done Done Sent false
32475 9469 Acked Flush 0 Done Done Sent false
32476 9470 Acked Read 3 Done Done Sent false
32477 9471 Acked Read 1 Done Done Sent false
32478 9472 Acked Read 9 Done Done Sent false
32479 9473 Acked Write 5 Done Done Sent false
32480 9474 Acked Read 9 Done Done Sent false
32481 9475 Acked Write 9 Done Done Sent false
32482 9476 Acked Flush 0 Done Done Sent false
32483 9477 Acked Flush 0 Done Done Sent false
32484 9478 Acked Flush 0 Done Done Sent false
32485 9479 Acked Write 6 Done Done Sent false
32486 9480 Acked Flush 0 Done Done Sent false
32487 9481 Acked Write 2 Done Done Sent false
32488 9482 Acked Write 2 Done Done Sent false
32489 9483 Acked Flush 0 Done Done Sent false
32490 9484 Acked Flush 0 Done Done Sent false
32491 9485 Acked Read 2 Done Done Sent false
32492 9486 Acked Write 1 Done Done Sent false
32493 9487 Acked Read 10 Done Done Sent false
32494 9488 Acked Write 8 Done Done Sent false
32495 9489 Acked Write 3 Done Done Sent false
32496 9490 Acked Read 1 Done Done Sent false
32497 9491 Acked Read 9 Done Done Sent false
32498 9492 Acked Write 10 Done Done Sent false
32499 9493 Acked Read 6 Done Done Sent false
32500 9494 Acked Read 4 Done Done Sent false
32501 9495 Acked Read 2 Done Done Sent false
32502 9496 Acked Write 1 Done Done Sent false
32503 9497 Acked Write 10 Done Done Sent false
32504 9498 Acked Write 3 Done Done Sent false
32505 9499 Acked Write 3 Done Done Sent false
32506 9500 Acked Read 1 Done Done Sent false
32507 9501 Acked Write 2 Done Done Sent false
32508 9502 Acked Read 7 Done Done Sent false
32509 9503 Acked Write 7 Done Done Sent false
32510 9504 Acked Read 8 Done Done Sent false
32511 9505 Acked Read 8 Done Done Sent false
32512 9506 Acked Read 9 Done Done Sent false
32513 9507 Acked Read 1 Done Done Sent false
32514 9508 Acked Read 8 Done Done Sent false
32515 9509 Acked Read 9 Done Done Sent false
32516 9510 Acked Read 1 Done Done Sent false
32517 9511 Acked Write 10 Done Done Sent false
32518 9512 Acked Read 6 Done Done Sent false
32519 9513 Acked Write 9 Done Done Sent false
32520 9514 Acked Write 6 Done Done Sent false
32521 9515 Acked Read 1 Done Done Sent false
32522 9516 Acked Read 8 Done Done Sent false
32523 9517 Acked Flush 0 Done Done Sent false
32524 9518 Acked Write 1 Done Done Sent false
32525 9519 Acked Read 10 Done Done Sent false
32526 9520 Acked Flush 0 Done Done Sent false
32527 9521 Acked Read 2 Done Done Sent false
32528 9522 Acked Flush 0 Done Done Sent false
32529 9523 Acked Read 7 Done Done Sent false
32530 9524 Acked Read 4 Done Done Sent false
32531 9525 Acked Write 10 Done Done Sent false
32532 9526 Acked Read 9 Done Done Sent false
32533 9527 Acked Read 4 Done Done Sent false
32534 9528 Acked Read 9 Done Done Sent false
32535 9529 Acked Write 1 Done Done Sent false
32536 9530 Acked Write 5 Done Done Sent false
32537 9531 Acked Flush 0 Done Done Sent false
32538 9532 Acked Flush 0 Done Done Sent false
32539 9533 Acked Read 6 Done Done Sent false
32540 9534 Acked Write 9 Done Done Sent false
32541 9535 Acked Read 9 Done Done Sent false
32542 9536 Acked Write 1 Done Done Sent false
32543 9537 Acked Write 8 Done Done Sent false
32544 9538 Acked Read 10 Done Done Sent false
32545 9539 Acked Write 1 Done Done Sent false
32546 9540 Acked Read 1 Done Done Sent false
32547 9541 Acked Read 8 Done Done Sent false
32548 9542 Acked Read 6 Done Done Sent false
32549 9543 Acked Write 9 Done Done Sent false
32550 9544 Acked Flush 0 Done Done Sent false
32551 9545 Acked Read 2 Done Done Sent false
32552 9546 Acked Read 10 Done Done Sent false
32553 9547 Acked Read 5 Done Done Sent false
32554 9548 Acked Flush 0 Done Done Sent false
32555 9549 Acked Read 1 Done Done Sent false
32556 9550 Acked Write 10 Done Done Sent false
32557 9551 Acked Read 2 Done Done Sent false
32558 9552 Acked Read 5 Done Done Sent false
32559 9553 Acked Read 2 Done Done Sent false
32560 9554 Acked Flush 0 Done Done Sent false
32561 9555 Acked Read 10 Done Done Sent false
32562 9556 Acked Read 10 Done Done Sent false
32563 9557 Acked Write 10 Done Done Sent false
32564 9558 Acked Flush 0 Done Done Sent false
32565 9559 Acked Write 3 Done Done Sent false
32566 9560 Acked Read 7 Done Done Sent false
32567 9561 Acked Write 10 Done Done Sent false
32568 9562 Acked Write 6 Done Done Sent false
32569 9563 Acked Write 10 Done Done Sent false
32570 9564 Acked Write 8 Done Done Sent false
32571 9565 Acked Read 2 Done Done Sent false
32572 9566 Acked Read 9 Done Done Sent false
32573 9567 Acked Write 8 Done Done Sent false
32574 9568 Acked Flush 0 Done Done Sent false
32575 9569 Acked Read 3 Done Done Sent false
32576 9570 Acked Flush 0 Done Done Sent false
32577 9571 Acked Read 4 Done Done Sent false
32578 9572 Acked Write 2 Done Done Sent false
32579 9573 Acked Write 3 Done Done Sent false
32580 9574 Acked Read 10 Done Done Sent false
32581 9575 Acked Read 8 Done Done Sent false
32582 9576 Acked Read 10 Done Done Sent false
32583 9577 Acked Write 4 Done Done Sent false
32584 9578 Acked Write 9 Done Done Sent false
32585 9579 Acked Read 8 Done Done Sent false
32586 9580 Acked Write 3 Done Done Sent false
32587 9581 Acked Write 2 Done Done Sent false
32588 9582 Acked Write 9 Done Done Sent false
32589 9583 Acked Flush 0 Done Done Sent false
32590 9584 Acked Read 6 Done Done Sent false
32591 9585 Acked Write 7 Done Done Sent false
32592 9586 Acked Write 7 Done Done Sent false
32593 9587 Acked Read 10 Done Done Sent false
32594 9588 Acked Flush 0 Done Done Sent false
32595 9589 Acked Write 10 Done Done Sent false
32596 9590 Acked Read 9 Done Done Sent false
32597 9591 Acked Read 7 Done Done Sent false
32598 9592 Acked Read 8 Done Done Sent false
32599 9593 Acked Read 3 Done Done Sent false
32600 9594 Acked Write 1 Done Done Sent false
32601 9595 Acked Read 1 Done Done Sent false
32602 9596 Acked Write 4 Done Done Sent false
32603 9597 Acked Read 10 Done Done Sent false
32604 9598 Acked Read 5 Done Done Sent false
32605 9599 Acked Read 7 Done Done Sent false
32606 9600 Acked Write 2 Done Done Sent false
32607 9601 Acked Read 8 Done Done Sent false
32608 9602 Acked Write 1 Done Done Sent false
32609 9603 Acked Read 10 Done Done Sent false
32610 9604 Acked Write 6 Done Done Sent false
32611 9605 Acked Read 10 Done Done Sent false
32612 9606 Acked Write 3 Done Done Sent false
32613 9607 Acked Read 6 Done Done Sent false
32614 9608 Acked Read 1 Done Done Sent false
32615 9609 Acked Write 9 Done Done Sent false
32616 9610 Acked Write 10 Done Done Sent false
32617 9611 Acked Write 10 Done Done Sent false
32618 9612 Acked Read 2 Done Done Sent false
32619 9613 Acked Write 9 Done Done Sent false
32620 9614 Acked Flush 0 Done Done Sent false
32621 9615 Acked Read 4 Done Done Sent false
32622 9616 Acked Write 6 Done Done Sent false
32623 9617 Acked Write 1 Done Done Sent false
32624 9618 Acked Read 8 Done Done Sent false
32625 9619 Acked Flush 0 Done Done Sent false
32626 9620 Acked Read 6 Done Done Sent false
32627 9621 Acked Read 9 Done Done Sent false
32628 9622 Acked Write 6 Done Done Sent false
32629 9623 Acked Read 4 Done Done Sent false
32630 9624 Acked Write 6 Done Done Sent false
32631 9625 Acked Write 10 Done Done Sent false
32632 9626 Acked Read 7 Done Done Sent false
32633 9627 Acked Write 4 Done Done Sent false
32634 9628 Acked Write 3 Done Done Sent false
32635 9629 Acked Write 7 Done Done Sent false
32636 9630 Acked Write 8 Done Done Sent false
32637 9631 Acked Read 10 Done Done Sent false
32638 9632 Acked Read 9 Done Done Sent false
32639 9633 Acked Read 8 Done Done Sent false
32640 9634 Acked Write 2 Done Done Sent false
32641 9635 Acked Flush 0 Done Done Sent false
32642 9636 Acked Write 3 Done Done Sent false
32643 9637 Acked Write 5 Done Done Sent false
32644 9638 Acked Read 6 Done Done Sent false
32645 9639 Acked Flush 0 Done Done Sent false
32646 9640 Acked Read 10 Done Done Sent false
32647 9641 Acked Write 1 Done Done Sent false
32648 9642 Acked Read 9 Done Done Sent false
32649 9643 Acked Read 4 Done Done Sent false
32650 9644 Acked Read 6 Done Done Sent false
32651 9645 Acked Write 4 Done Done Sent false
32652 9646 Acked Flush 0 Done Done Sent false
32653 9647 Acked Read 5 Done Done Sent false
32654 9648 Acked Flush 0 Done Done Sent false
32655 9649 Acked Flush 0 Done Done Sent false
32656 9650 Acked Read 8 Done Done Sent false
32657 9651 Acked Read 10 Done Done Sent false
32658 9652 Acked Write 7 Done Done Sent false
32659 9653 Acked Write 3 Done Done Sent false
32660 9654 Acked Read 7 Done Done Sent false
32661 9655 Acked Read 10 Done Done Sent false
32662 9656 Acked Read 9 Done Done Sent false
32663 9657 Acked Write 4 Done Done Sent false
32664 9658 Acked Write 5 Done Done Sent false
32665 9659 Acked Read 3 Done Done Sent false
32666 9660 Acked Write 7 Done Done Sent false
32667 9661 Acked Read 6 Done Done Sent false
32668 9662 Acked Write 4 Done Done Sent false
32669 9663 Acked Write 9 Done Done Sent false
32670 9664 Acked Read 3 Done Done Sent false
32671 9665 Acked Flush 0 Done Done Sent false
32672 9666 Acked Read 7 Done Done Sent false
32673 9667 Acked Read 2 Done Done Sent false
32674 9668 Acked Write 5 Done Done Sent false
32675 9669 Acked Write 8 Done Done Sent false
32676 9670 Acked Read 8 Done Done Sent false
32677 9671 Acked Write 7 Done Done Sent false
32678 9672 Acked Flush 0 Done Done Sent false
32679 9673 Acked Write 6 Done Done Sent false
32680 9674 Acked Write 2 Done Done Sent false
32681 9675 Acked Read 10 Done Done Sent false
32682 9676 Acked Read 10 Done Done Sent false
32683 9677 Acked Read 6 Done Done Sent false
32684 9678 Acked Write 5 Done Done Sent false
32685 9679 Acked Write 6 Done Done Sent false
32686 9680 Acked Flush 0 Done Done Sent false
32687 9681 Acked Write 4 Done Done Sent false
32688 9682 Acked Write 3 Done Done Sent false
32689 9683 Acked Read 10 Done Done Sent false
32690 9684 Acked Write 1 Done Done Sent false
32691 9685 Acked Write 3 Done Done Sent false
32692 9686 Acked Write 7 Done Done Sent false
32693 9687 Acked Read 5 Done Done Sent false
32694 9688 Acked Write 1 Done Done Sent false
32695 9689 Acked Read 10 Done Done Sent false
32696 9690 Acked Read 5 Done Done Sent false
32697 9691 Acked Read 5 Done Done Sent false
32698 9692 Acked Read 4 Done Done Sent false
32699 9693 Acked Read 9 Done Done Sent false
32700 9694 Acked Write 8 Done Done Sent false
32701 9695 Acked Read 9 Done Done Sent false
32702 9696 Acked Write 1 Done Done Sent false
32703 9697 Acked Write 6 Done Done Sent false
32704 9698 Acked Read 9 Done Done Sent false
32705 9699 Acked Flush 0 Done Done Sent false
32706 9700 Acked Flush 0 Done Done Sent false
32707 9701 Acked Flush 0 Done Done Sent false
32708 9702 Acked Read 3 Done Done Sent false
32709 9703 Acked Read 3 Done Done Sent false
32710 9704 Acked Write 6 Done Done Sent false
32711 9705 Acked Write 2 Done Done Sent false
32712 9706 Acked Read 4 Done Done Sent false
32713 9707 Acked Write 7 Done Done Sent false
32714 9708 Acked Read 5 Done Done Sent false
32715 9709 Acked Write 9 Done Done Sent false
32716 9710 Acked Read 10 Done Done Sent false
32717 9711 Acked Write 8 Done Done Sent false
32718 9712 Acked Read 4 Done Done Sent false
32719 9713 Acked Read 1 Done Done Sent false
32720 9714 Acked Flush 0 Done Done Sent false
32721 9715 Acked Write 1 Done Done Sent false
32722 9716 Acked Read 3 Done Done Sent false
32723 9717 Acked Write 5 Done Done Sent false
32724 9718 Acked Read 1 Done Done Sent false
32725 9719 Acked Read 8 Done Done Sent false
32726 9720 Acked Flush 0 Done Done Sent false
32727 9721 Acked Write 5 Done Done Sent false
32728 9722 Acked Read 9 Done Done Sent false
32729 9723 Acked Read 2 Done Done Sent false
32730 9724 Acked Write 5 Done Done Sent false
32731 9725 Acked Read 1 Done Done Sent false
32732 9726 Acked Read 1 Done Done Sent false
32733 9727 Acked Write 5 Done Done Sent false
32734 9728 Acked Write 1 Done Done Sent false
32735 9729 Acked Write 5 Done Done Sent false
32736 9730 Acked Write 8 Done Done Sent false
32737 9731 Acked Read 3 Done Done Sent false
32738 9732 Acked Write 5 Done Done Sent false
32739 9733 Acked Flush 0 Done Done Sent false
32740 9734 Acked Flush 0 Done Done Sent false
32741 9735 Acked Read 9 Done Done Sent false
32742 9736 Acked Write 4 Done Done Sent false
32743 9737 Acked Write 8 Done Done Sent false
32744 9738 Acked Write 3 Done Done Sent false
32745 9739 Acked Flush 0 Done Done Sent false
32746 9740 Acked Read 2 Done Done Sent false
32747 9741 Acked Write 1 Done Done Sent false
32748 9742 Acked Write 7 Done Done Sent false
32749 9743 Acked Read 2 Done Done Sent false
32750 9744 Acked Read 2 Done Done Sent false
32751 9745 Acked Read 1 Done Done Sent false
32752 9746 Acked Read 10 Done Done Sent false
32753 9747 Acked Write 5 Done Done Sent false
32754 9748 Acked Read 3 Done Done Sent false
32755 9749 Acked Read 3 Done Done Sent false
32756 9750 Acked Read 1 Done Done Sent false
32757 9751 Acked Write 5 Done Done Sent false
32758 9752 Acked Write 9 Done Done Sent false
32759 9753 Acked Write 2 Done Done Sent false
32760 9754 Acked Read 6 Done Done Sent false
32761 9755 Acked Flush 0 Done Done Sent false
32762 STATES DS:0 DS:1 DS:2 TOTAL
32763 Sent 0 0 302 302
32764 Done 302 302 0 604
32765 Skipped 0 0 0 0
32766 Error 0 0 0 0
32767 Last Flush: 9755 9755 9453
32768 Downstairs last five completed: 9453 9452 9451 9450 9449
32769 Upstairs last five completed: 9755 9754 9753 9752 9751
32770 CLIENT: Up:0 ds:302 act:2
327712025-01-15T05:25:51.768ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327722025-01-15T05:25:51.768ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41498) = io task client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327732025-01-15T05:25:51.769ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327742025-01-15T05:25:51.769ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327752025-01-15T05:25:51.769ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327762025-01-15T05:25:51.769ZINFOcrucible: send last flush ID to this DS: 9453 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327772025-01-15T05:25:51.770ZINFOcrucible: Replied this last flush ID: 9453 = downstairs client = 2 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327782025-01-15T05:25:51.770ZINFOcrucible: [2] client re-new 302 jobs since flush 9453 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327792025-01-15T05:25:51.770ZINFOcrucible: [2] Replayed 302 jobs since flush: 9453 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
32780 ----------------------------------------------------------------
32781 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32782 Downstairs last five completed: 9755 9754 9753 9752 9751
32783 Upstairs last five completed: 9755 9754 9753 9752 9751
32784 CLIENT: Up:0 ds:0 act:3
32785 Replay: All jobs finished
32786 Main task has work to do, go find it
32787 got dsc Stop(1)
32788 stop 1
32789 Jan 15 05:25:54.005 INFO request completed, latency_us: 329, response_code: 204, uri: /stop/cid/1, method: POST, req_id: aae664a7-82b0-4b16-9142-86d8b7ffad1e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
32790 [8820] Got stop action so:false kr:false
32791 Jan 15 05:25:54.006 INFO request completed, latency_us: 117, response_code: 200, uri: /state/cid/1, method: GET, req_id: bbe1001c-18de-41fd-ab28-3fb5981f06e5, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
327922025-01-15T05:25:54.006ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327932025-01-15T05:25:54.006ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327942025-01-15T05:25:54.006ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
327952025-01-15T05:25:54.006ZINFOcrucible: client task is exiting = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
32796 [8820] Exited with: ExitStatus(unix_wait_status(9))
32797 [8820][1] reports Exit
32798 Jan 15 05:25:58.007 INFO request completed, latency_us: 195, response_code: 200, uri: /state/cid/1, method: GET, req_id: 6c8d8403-bdbf-4a08-a6a2-0fb91fe3954e, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
32799 001/300 Write block 3507 len 8192 data: 6 7
32800 002/300 Read block 451 len 4096
32801 003/300 Write block 4916 len 20480 data: 7 7 4 3 4
32802 004/300 Read block 1306 len 8192
32803 005/300 Read block 4280 len 4096
32804 006/300 Read block 3965 len 20480
32805 007/300 Read block 2620 len 28672
32806 008/300 Write block 1353 len 36864 data: 4 3 5 5 2 3 3 3 3
32807 009/300 Write block 1897 len 20480 data: 9 9 8 8 7
32808 010/300 Write block 4799 len 40960 data: 6 7 7 6 3 3 4 4 4 4
32809 011/300 Flush
32810 012/300 Read block 651 len 36864
32811 013/300 Read block 265 len 32768
32812 014/300 Read block 563 len 12288
32813 015/300 Read block 3003 len 40960
32814 016/300 Write block 1435 len 36864 data: 8 8 7 7 6 4 4 4 5
32815 017/300 Read block 2635 len 24576
32816 018/300 Read block 3657 len 32768
32817 019/300 Read block 4874 len 20480
32818 020/300 Read block 3329 len 20480
32819 021/300 Write block 4678 len 16384 data: 9 8 6 5
32820 022/300 Read block 1996 len 16384
32821 023/300 Write block 670 len 24576 data: 4 5 4 5 6 5
32822 024/300 Write block 3126 len 8192 data: 6 7
32823 025/300 Read block 467 len 28672
32824 026/300 Read block 4688 len 32768
32825 027/300 Read block 4923 len 36864
32826 028/300 Write block 4293 len 40960 data: 5 5 4 4 4 4 4 4 4 6
32827 029/300 Write block 1279 len 36864 data: 5 4 4 3 3 3 4 5 5
32828 030/300 Flush
32829 031/300 Write block 1108 len 24576 data: 4 5 5 7 6 4
32830 032/300 Read block 1594 len 4096
32831 033/300 Write block 3074 len 40960 data: 5 4 4 4 5 4 3 2 2 2
32832 034/300 Read block 966 len 16384
32833 035/300 Read block 4087 len 24576
32834 036/300 Write block 4316 len 4096 data: 7
32835 037/300 Write block 3875 len 4096 data: 6
32836 038/300 Write block 4637 len 4096 data: 6
32837 039/300 Read block 217 len 16384
32838 040/300 Flush
32839 041/300 Read block 3506 len 24576
32840 042/300 Read block 4018 len 16384
32841 043/300 Read block 1269 len 24576
32842 044/300 Read block 2522 len 40960
32843 045/300 Read block 1269 len 16384
32844 046/300 Read block 4364 len 8192
32845 047/300 Write block 1094 len 4096 data: 5
32846 048/300 Write block 4547 len 16384 data: 8 7 9 9
32847 049/300 Write block 347 len 20480 data: 5 4 4 5 6
32848 050/300 Write block 4498 len 24576 data: 6 8 8 9 9 8
32849 051/300 Read block 431 len 28672
32850 052/300 Read block 3638 len 16384
32851 053/300 Flush
32852 054/300 Read block 4421 len 4096
32853 055/300 Write block 1386 len 20480 data: 4 3 4 4 4
32854 056/300 Write block 3044 len 12288 data: 9 6 7
32855 057/300 Write block 181 len 40960 data: 7 6 7 7 7 7 6 5 5 5
32856 058/300 Read block 3575 len 12288
32857 059/300 Read block 3390 len 24576
32858 060/300 Read block 3256 len 12288
32859 061/300 Read block 1354 len 28672
32860 062/300 Flush
32861 063/300 Flush
32862 064/300 Flush
32863 065/300 Read block 4812 len 36864
32864 066/300 Read block 2378 len 28672
32865 067/300 Write block 3249 len 8192 data: 5 7
32866 068/300 Write block 3682 len 16384 data: 13 11 9 9
32867 069/300 Flush
32868 070/300 Read block 2776 len 40960
32869 071/300 Write block 851 len 24576 data: 5 4 4 6 6 6
32870 072/300 Write block 4973 len 24576 data: 5 5 5 6 6 7
32871 073/300 Write block 2236 len 8192 data: 7 7
32872 074/300 Flush
32873 075/300 Write block 1965 len 32768 data: 5 5 3 4 5 5 5 4
32874 076/300 Write block 4972 len 20480 data: 4 6 6 6 7
32875 077/300 Read block 2393 len 40960
32876 078/300 Read block 4143 len 20480
32877 079/300 Write block 2937 len 20480 data: 4 4 5 5 7
32878 080/300 Write block 2880 len 4096 data: 9
32879 081/300 Read block 3909 len 40960
32880 082/300 Read block 176 len 36864
32881 083/300 Write block 4825 len 8192 data: 5 4
32882 084/300 Read block 1846 len 12288
32883 085/300 Read block 4919 len 20480
32884 086/300 Read block 3511 len 28672
32885 087/300 Read block 1451 len 36864
32886 088/300 Flush
32887 089/300 Read block 2403 len 24576
32888 090/300 Write block 382 len 40960 data: 10 7 8 5 5 5 4 4 5 5
32889 091/300 Write block 3588 len 24576 data: 4 4 4 4 4 4
32890 092/300 Write block 3445 len 40960 data: 7 6 7 9 10 10 10 10 8 9
32891 093/300 Write block 4271 len 40960 data: 4 4 4 6 5 5 5 5 6 6
32892 094/300 Read block 2773 len 28672
32893 095/300 Read block 2763 len 36864
32894 096/300 Read block 3615 len 32768
32895 097/300 Read block 3497 len 8192
32896 098/300 Read block 4859 len 8192
32897 099/300 Read block 4371 len 28672
32898 100/300 Write block 3270 len 8192 data: 7 7
32899 101/300 Write block 3448 len 12288 data: 10 11 11
32900 102/300 Read block 3713 len 28672
32901 103/300 Write block 2372 len 36864 data: 7 6 6 4 4 6 5 5 5
32902 104/300 Read block 228 len 40960
32903 105/300 Write block 3952 len 4096 data: 3
32904 106/300 Read block 4349 len 32768
32905 107/300 Read block 3492 len 12288
32906 108/300 Read block 762 len 8192
32907 109/300 Read block 2241 len 32768
32908 110/300 Write block 4459 len 8192 data: 6 5
32909 111/300 Write block 4742 len 4096 data: 9
32910 112/300 Read block 8 len 8192
32911 113/300 Read block 402 len 40960
32912 114/300 Flush
32913 115/300 Read block 1140 len 20480
32914 116/300 Write block 4454 len 12288 data: 6 9 8
32915 117/300 Write block 981 len 28672 data: 4 4 3 4 5 5 5
32916 118/300 Write block 3993 len 40960 data: 6 5 5 5 4 4 5 5 5 5
32917 119/300 Read block 4915 len 24576
32918 120/300 Write block 3387 len 36864 data: 5 6 5 5 5 7 6 5 6
32919 121/300 Write block 4626 len 12288 data: 5 6 5
32920 122/300 Read block 1740 len 20480
32921 123/300 Read block 979 len 40960
32922 124/300 Read block 4679 len 16384
32923 125/300 Read block 535 len 16384
32924 126/300 Write block 456 len 40960 data: 10 7 9 7 6 5 5 3 3 4
32925 127/300 Read block 2040 len 24576
32926 128/300 Flush
32927 129/300 Flush
32928 130/300 Read block 248 len 16384
32929 131/300 Write block 3580 len 16384 data: 6 6 7 8
32930 132/300 Read block 886 len 32768
32931 133/300 Flush
32932 134/300 Read block 0 len 4096
32933 135/300 Read block 2341 len 28672
32934 136/300 Flush
32935 137/300 Write block 2400 len 4096 data: 4
32936 138/300 Flush
32937 139/300 Write block 3703 len 4096 data: 5
32938 140/300 Write block 3627 len 24576 data: 5 4 4 5 5 4
32939 141/300 Read block 1380 len 28672
32940 142/300 Write block 403 len 12288 data: 9 9 8
32941 143/300 Write block 257 len 20480 data: 11 12 9 9 7
32942 144/300 Read block 917 len 28672
32943 145/300 Read block 1811 len 36864
32944 146/300 Read block 1522 len 40960
32945 147/300 Write block 4160 len 8192 data: 7 6
32946 148/300 Read block 4813 len 40960
32947 149/300 Write block 3273 len 36864 data: 9 8 8 7 8 8 10 9 7
32948 150/300 Write block 60 len 16384 data: 3 3 4 6
32949 151/300 Read block 666 len 8192
32950 152/300 Read block 3354 len 40960
32951 153/300 Read block 2021 len 12288
32952 154/300 Read block 779 len 36864
32953 155/300 Read block 2417 len 32768
32954 156/300 Write block 3750 len 4096 data: 5
32955 157/300 Flush
32956 158/300 Write block 1419 len 12288 data: 6 6 7
32957 159/300 Read block 2949 len 20480
32958 160/300 Flush
32959 161/300 Write block 1664 len 16384 data: 2 3 4 5
32960 162/300 Flush
32961 163/300 Write block 4977 len 36864 data: 7 8 9 9 8 7 5 5 4
32962 164/300 Read block 3151 len 16384
32963 165/300 Read block 5 len 16384
32964 166/300 Write block 1550 len 20480 data: 3 3 3 3 3
32965 167/300 Flush
32966 168/300 Read block 3015 len 20480
32967 169/300 Read block 3832 len 40960
32968 170/300 Read block 2060 len 24576
32969 171/300 Read block 795 len 16384
32970 172/300 Read block 1435 len 32768
32971 173/300 Read block 559 len 40960
32972 174/300 Flush
32973 175/300 Read block 2781 len 28672
32974 176/300 Write block 1247 len 32768 data: 6 5 5 5 5 5 7 8
32975 177/300 Flush
32976 178/300 Write block 3597 len 20480 data: 5 5 6 8 8
32977 179/300 Write block 2841 len 16384 data: 5 5 5 4
32978 180/300 Write block 4028 len 40960 data: 7 10 8 7 7 7 5 6 7 7
32979 181/300 Write block 4573 len 28672 data: 11 10 10 7 8 9 8
32980 182/300 Read block 3573 len 24576
32981 183/300 Flush
32982 184/300 Read block 1822 len 32768
32983 185/300 Read block 2570 len 16384
32984 186/300 Read block 2935 len 24576
32985 187/300 Write block 3284 len 8192 data: 5 5
32986 188/300 Read block 3615 len 28672
32987 189/300 Read block 1428 len 4096
32988 190/300 Read block 2741 len 36864
32989 191/300 Flush
32990 192/300 Read block 309 len 8192
32991 193/300 Read block 2277 len 36864
32992 194/300 Read block 1657 len 24576
32993 195/300 Read block 2512 len 28672
32994 196/300 Read block 418 len 24576
32995 197/300 Write block 4442 len 24576 data: 2 2 4 5 6 7
32996 198/300 Write block 4303 len 8192 data: 6 4
32997 199/300 Write block 2407 len 28672 data: 5 5 6 6 7 7 7
32998 200/300 Write block 1588 len 24576 data: 8 6 5 5 4 5
32999 201/300 Write block 3108 len 8192 data: 5 4
33000 202/300 Flush
33001 203/300 Read block 4920 len 40960
33002 204/300 Write block 2557 len 12288 data: 5 7 6
33003 205/300 Read block 619 len 4096
33004 206/300 Read block 2902 len 24576
33005 207/300 Read block 782 len 24576
33006 208/300 Write block 651 len 28672 data: 5 7 6 4 4 5 5
33007 209/300 Read block 4839 len 28672
33008 210/300 Write block 2766 len 12288 data: 4 3 5
33009 211/300 Write block 3467 len 20480 data: 5 7 6 7 5
33010 212/300 Write block 3758 len 8192 data: 6 6
33011 213/300 Read block 1175 len 12288
33012 214/300 Write block 2223 len 20480 data: 6 5 5 7 7
33013 215/300 Write block 1495 len 32768 data: 7 6 4 5 5 7 8 9
33014 216/300 Read block 3648 len 36864
33015 217/300 Write block 4065 len 16384 data: 4 3 6 4
33016 218/300 Write block 4144 len 40960 data: 10 10 9 9 7 6 5 7 8 8
33017 219/300 Read block 2769 len 28672
33018 220/300 Write block 2202 len 12288 data: 7 6 5
33019 221/300 Read block 217 len 16384
33020 222/300 Write block 1728 len 36864 data: 4 6 5 5 5 5 5 4 4
33021 223/300 Read block 3068 len 12288
33022 224/300 Read block 3242 len 20480
33023 225/300 Write block 1783 len 16384 data: 11 11 11 12
33024 226/300 Read block 2979 len 40960
33025 227/300 Read block 393 len 20480
33026 228/300 Read block 2174 len 36864
33027 229/300 Write block 4129 len 12288 data: 7 8 10
33028 230/300 Read block 3774 len 16384
33029 231/300 Read block 2484 len 16384
33030 232/300 Write block 4297 len 16384 data: 5 5 5 5
33031 233/300 Write block 2622 len 36864 data: 4 5 4 7 7 6 7 8 6
33032 234/300 Write block 3950 len 32768 data: 7 6 4 5 6 6 7 6
33033 235/300 Flush
33034 236/300 Write block 3905 len 40960 data: 4 3 3 3 3 5 4 6 5 7
33035 237/300 Write block 4954 len 16384 data: 4 4 4 4
33036 238/300 Read block 2684 len 20480
33037 239/300 Write block 4489 len 28672 data: 6 6 7 5 8 8 10
33038 240/300 Write block 4431 len 20480 data: 8 7 6 6 4
33039 241/300 Flush
33040 242/300 Read block 718 len 24576
33041 243/300 Read block 3429 len 16384
33042 244/300 Write block 502 len 20480 data: 5 3 4 5 6
33043 245/300 Read block 120 len 8192
33044 246/300 Read block 1503 len 28672
33045 247/300 Read block 4252 len 32768
33046 248/300 Write block 3150 len 40960 data: 5 5 7 7 6 7 7 11 11 9
33047 249/300 Write block 4452 len 36864 data: 8 7 7 10 9 8 7 7 6
33048 250/300 Read block 1369 len 20480
33049 251/300 Write block 4658 len 28672 data: 6 7 6 6 4 5 6
33050 252/300 Write block 4571 len 32768 data: 4 7 12 11 11 8 9 10
33051 253/300 Write block 1778 len 4096 data: 7
33052 254/300 Read block 3236 len 24576
33053 255/300 Write block 4013 len 32768 data: 4 4 4 6 7 7 8 10
33054 256/300 Read block 2793 len 40960
33055 257/300 Flush
33056 258/300 Read block 1937 len 40960
33057 259/300 Write block 2003 len 40960 data: 5 5 5 5 5 5 7 8 7 6
33058 260/300 Write block 4307 len 4096 data: 4
33059 261/300 Write block 3333 len 20480 data: 5 10 9 11 11
33060 262/300 Read block 12 len 36864
33061 263/300 Write block 3492 len 20480 data: 4 5 5 5 5
33062 264/300 Write block 1656 len 40960 data: 5 5 3 3 3 3 2 2 3 4
33063 265/300 Read block 4682 len 8192
33064 266/300 Read block 3675 len 24576
33065 267/300 Write block 3981 len 4096 data: 2
33066 268/300 Write block 1453 len 28672 data: 7 8 8 9 6 5 6
33067 269/300 Write block 2661 len 40960 data: 7 7 6 3 3 3 4 4 5 4
33068 270/300 Read block 1407 len 8192
33069 271/300 Read block 3420 len 36864
33070 272/300 Read block 1471 len 12288
33071 273/300 Write block 1384 len 20480 data: 3 4 5 4 5
33072 274/300 Write block 3061 len 20480 data: 6 5 5 6 6
33073 275/300 Read block 645 len 24576
33074 276/300 Read block 4707 len 28672
33075 277/300 Read block 2461 len 24576
33076 278/300 Read block 1143 len 28672
33077 279/300 Write block 225 len 8192 data: 7 6
33078 280/300 Flush
33079 281/300 Write block 4103 len 16384 data: 4 4 3 4
33080 282/300 Read block 1852 len 20480
33081 283/300 Read block 3922 len 8192
33082 284/300 Read block 3396 len 36864
33083 285/300 Read block 166 len 4096
33084 286/300 Read block 785 len 28672
33085 287/300 Write block 3 len 20480 data: 5 5 6 8 8
33086 288/300 Write block 4541 len 32768 data: 4 6 7 6 7 6 9 8
33087 289/300 Write block 2812 len 12288 data: 3 3 3
33088 290/300 Write block 3258 len 40960 data: 7 5 5 7 8 5 6 6 5 6
33089 291/300 Read block 1573 len 32768
33090 292/300 Read block 4998 len 4096
33091 293/300 Write block 3319 len 36864 data: 6 5 6 5 5 5 5 6 9
33092 294/300 Write block 3097 len 4096 data: 5
33093 295/300 Read block 3171 len 20480
33094 296/300 Read block 821 len 24576
33095 297/300 Flush
33096 298/300 Write block 1695 len 28672 data: 3 4 4 7 7 9 9
33097 299/300 Read block 3001 len 32768
33098 300/300 Read block 3118 len 24576
33099 301/300 Write block 529 len 40960 data: 7 6 6 5 6 5 4 4 4 7
33100 Main task has work to do, go find it
33101 got dsc Start(1)
33102 start 1
33103 Jan 15 05:25:58.246 INFO request completed, latency_us: 183, response_code: 204, uri: /start/cid/1, method: POST, req_id: e02c3aae-5748-41bc-935e-5d8b1ae5c7ae, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
33104 Replay: started 1, returned:Ok(())
33105 ----------------------------------------------------------------
33106 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33107 States: Active Offline Active
33108 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33109 9756 Acked Write 2 Done Sent Done false
33110 9757 Acked Read 1 Done Sent Done false
33111 9758 Acked Write 5 Done Sent Done false
33112 9759 Acked Read 2 Done Sent Done false
33113 9760 Acked Read 1 Done Sent Done false
33114 9761 Acked Read 5 Done Sent Done false
33115 9762 Acked Read 7 Done Sent Done false
33116 9763 Acked Write 9 Done Sent Done false
33117 9764 Acked Write 5 Done Sent Done false
33118 9765 Acked Write 10 Done Sent Done false
33119 9766 Acked Flush 0 Done Sent Done false
33120 9767 Acked Read 9 Done Sent Done false
33121 9768 Acked Read 8 Done Sent Done false
33122 9769 Acked Read 3 Done Sent Done false
33123 9770 Acked Read 10 Done Sent Done false
33124 9771 Acked Write 9 Done Sent Done false
33125 9772 Acked Read 6 Done Sent Done false
33126 9773 Acked Read 8 Done Sent Done false
33127 9774 Acked Read 5 Done Sent Done false
33128 9775 Acked Read 5 Done Sent Done false
33129 9776 Acked Write 4 Done Sent Done false
33130 9777 Acked Read 4 Done Sent Done false
33131 9778 Acked Write 6 Done Sent Done false
33132 9779 Acked Write 2 Done Sent Done false
33133 9780 Acked Read 7 Done Sent Done false
33134 9781 Acked Read 8 Done Sent Done false
33135 9782 Acked Read 9 Done Sent Done false
33136 9783 Acked Write 10 Done Sent Done false
33137 9784 Acked Write 9 Done Sent Done false
33138 9785 Acked Flush 0 Done Sent Done false
33139 9786 Acked Write 6 Done Sent Done false
33140 9787 Acked Read 1 Done Sent Done false
33141 9788 Acked Write 10 Done Sent Done false
33142 9789 Acked Read 4 Done Sent Done false
33143 9790 Acked Read 6 Done Sent Done false
33144 9791 Acked Write 1 Done Sent Done false
33145 9792 Acked Write 1 Done Sent Done false
33146 9793 Acked Write 1 Done Sent Done false
33147 9794 Acked Read 4 Done Sent Done false
33148 9795 Acked Flush 0 Done Sent Done false
33149 9796 Acked Read 6 Done Sent Done false
33150 9797 Acked Read 4 Done Sent Done false
33151 9798 Acked Read 6 Done Sent Done false
33152 9799 Acked Read 10 Done Sent Done false
33153 9800 Acked Read 4 Done Sent Done false
33154 9801 Acked Read 2 Done Sent Done false
33155 9802 Acked Write 1 Done Sent Done false
33156 9803 Acked Write 4 Done Sent Done false
33157 9804 Acked Write 5 Done Sent Done false
33158 9805 Acked Write 6 Done Sent Done false
33159 9806 Acked Read 7 Done Sent Done false
33160 9807 Acked Read 4 Done Sent Done false
33161 9808 Acked Flush 0 Done Sent Done false
33162 9809 Acked Read 1 Done Sent Done false
33163 9810 Acked Write 5 Done Sent Done false
33164 9811 Acked Write 3 Done Sent Done false
33165 9812 Acked Write 10 Done Sent Done false
33166 9813 Acked Read 3 Done Sent Done false
33167 9814 Acked Read 6 Done Sent Done false
33168 9815 Acked Read 3 Done Sent Done false
33169 9816 Acked Read 7 Done Sent Done false
33170 9817 Acked Flush 0 Done Sent Done false
33171 9818 Acked Flush 0 Done Sent Done false
33172 9819 Acked Flush 0 Done Sent Done false
33173 9820 Acked Read 9 Done Sent Done false
33174 9821 Acked Read 7 Done Sent Done false
33175 9822 Acked Write 2 Done Sent Done false
33176 9823 Acked Write 4 Done Sent Done false
33177 9824 Acked Flush 0 Done Sent Done false
33178 9825 Acked Read 10 Done Sent Done false
33179 9826 Acked Write 6 Done Sent Done false
33180 9827 Acked Write 6 Done Sent Done false
33181 9828 Acked Write 2 Done Sent Done false
33182 9829 Acked Flush 0 Done Sent Done false
33183 9830 Acked Write 8 Done Sent Done false
33184 9831 Acked Write 5 Done Sent Done false
33185 9832 Acked Read 10 Done Sent Done false
33186 9833 Acked Read 5 Done Sent Done false
33187 9834 Acked Write 5 Done Sent Done false
33188 9835 Acked Write 1 Done Sent Done false
33189 9836 Acked Read 10 Done Sent Done false
33190 9837 Acked Read 9 Done Sent Done false
33191 9838 Acked Write 2 Done Sent Done false
33192 9839 Acked Read 3 Done Sent Done false
33193 9840 Acked Read 5 Done Sent Done false
33194 9841 Acked Read 7 Done Sent Done false
33195 9842 Acked Read 9 Done Sent Done false
33196 9843 Acked Flush 0 Done Sent Done false
33197 9844 Acked Read 6 Done Sent Done false
33198 9845 Acked Write 10 Done Sent Done false
33199 9846 Acked Write 6 Done Sent Done false
33200 9847 Acked Write 10 Done Sent Done false
33201 9848 Acked Write 10 Done Sent Done false
33202 9849 Acked Read 7 Done Sent Done false
33203 9850 Acked Read 9 Done Sent Done false
33204 9851 Acked Read 8 Done Sent Done false
33205 9852 Acked Read 2 Done Sent Done false
33206 9853 Acked Read 2 Done Sent Done false
33207 9854 Acked Read 7 Done Sent Done false
33208 9855 Acked Write 2 Done Sent Done false
33209 9856 Acked Write 3 Done Sent Done false
33210 9857 Acked Read 7 Done Sent Done false
33211 9858 Acked Write 9 Done Sent Done false
33212 9859 Acked Read 10 Done Sent Done false
33213 9860 Acked Write 1 Done Sent Done false
33214 9861 Acked Read 8 Done Sent Done false
33215 9862 Acked Read 3 Done Sent Done false
33216 9863 Acked Read 2 Done Sent Done false
33217 9864 Acked Read 8 Done Sent Done false
33218 9865 Acked Write 2 Done Sent Done false
33219 9866 Acked Write 1 Done Sent Done false
33220 9867 Acked Read 2 Done Sent Done false
33221 9868 Acked Read 10 Done Sent Done false
33222 9869 Acked Flush 0 Done Sent Done false
33223 9870 Acked Read 5 Done Sent Done false
33224 9871 Acked Write 3 Done Sent Done false
33225 9872 Acked Write 7 Done Sent Done false
33226 9873 Acked Write 10 Done Sent Done false
33227 9874 Acked Read 6 Done Sent Done false
33228 9875 Acked Write 9 Done Sent Done false
33229 9876 Acked Write 3 Done Sent Done false
33230 9877 Acked Read 5 Done Sent Done false
33231 9878 Acked Read 10 Done Sent Done false
33232 9879 Acked Read 4 Done Sent Done false
33233 9880 Acked Read 4 Done Sent Done false
33234 9881 Acked Write 10 Done Sent Done false
33235 9882 Acked Read 6 Done Sent Done false
33236 9883 Acked Flush 0 Done Sent Done false
33237 9884 Acked Flush 0 Done Sent Done false
33238 9885 Acked Read 4 Done Sent Done false
33239 9886 Acked Write 4 Done Sent Done false
33240 9887 Acked Read 8 Done Sent Done false
33241 9888 Acked Flush 0 Done Sent Done false
33242 9889 Acked Read 1 Done Sent Done false
33243 9890 Acked Read 7 Done Sent Done false
33244 9891 Acked Flush 0 Done Sent Done false
33245 9892 Acked Write 1 Done Sent Done false
33246 9893 Acked Flush 0 Done Sent Done false
33247 9894 Acked Write 1 Done Sent Done false
33248 9895 Acked Write 6 Done Sent Done false
33249 9896 Acked Read 7 Done Sent Done false
33250 9897 Acked Write 3 Done Sent Done false
33251 9898 Acked Write 5 Done Sent Done false
33252 9899 Acked Read 7 Done Sent Done false
33253 9900 Acked Read 9 Done Sent Done false
33254 9901 Acked Read 10 Done Sent Done false
33255 9902 Acked Write 2 Done Sent Done false
33256 9903 Acked Read 10 Done Sent Done false
33257 9904 Acked Write 9 Done Sent Done false
33258 9905 Acked Write 4 Done Sent Done false
33259 9906 Acked Read 2 Done Sent Done false
33260 9907 Acked Read 10 Done Sent Done false
33261 9908 Acked Read 3 Done Sent Done false
33262 9909 Acked Read 9 Done Sent Done false
33263 9910 Acked Read 8 Done Sent Done false
33264 9911 Acked Write 1 Done Sent Done false
33265 9912 Acked Flush 0 Done Sent Done false
33266 9913 Acked Write 3 Done Sent Done false
33267 9914 Acked Read 5 Done Sent Done false
33268 9915 Acked Flush 0 Done Sent Done false
33269 9916 Acked Write 4 Done Sent Done false
33270 9917 Acked Flush 0 Done Sent Done false
33271 9918 Acked Write 9 Done Sent Done false
33272 9919 Acked Read 4 Done Sent Done false
33273 9920 Acked Read 4 Done Sent Done false
33274 9921 Acked Write 5 Done Sent Done false
33275 9922 Acked Flush 0 Done Sent Done false
33276 9923 Acked Read 5 Done Sent Done false
33277 9924 Acked Read 10 Done Sent Done false
33278 9925 Acked Read 6 Done Sent Done false
33279 9926 Acked Read 4 Done Sent Done false
33280 9927 Acked Read 8 Done Sent Done false
33281 9928 Acked Read 10 Done Sent Done false
33282 9929 Acked Flush 0 Done Sent Done false
33283 9930 Acked Read 7 Done Sent Done false
33284 9931 Acked Write 8 Done Sent Done false
33285 9932 Acked Flush 0 Done Sent Done false
33286 9933 Acked Write 5 Done Sent Done false
33287 9934 Acked Write 4 Done Sent Done false
33288 9935 Acked Write 10 Done Sent Done false
33289 9936 Acked Write 7 Done Sent Done false
33290 9937 Acked Read 6 Done Sent Done false
33291 9938 Acked Flush 0 Done Sent Done false
33292 9939 Acked Read 8 Done Sent Done false
33293 9940 Acked Read 4 Done Sent Done false
33294 9941 Acked Read 6 Done Sent Done false
33295 9942 Acked Write 2 Done Sent Done false
33296 9943 Acked Read 7 Done Sent Done false
33297 9944 Acked Read 1 Done Sent Done false
33298 9945 Acked Read 9 Done Sent Done false
33299 9946 Acked Flush 0 Done Sent Done false
33300 9947 Acked Read 2 Done Sent Done false
33301 9948 Acked Read 9 Done Sent Done false
33302 9949 Acked Read 6 Done Sent Done false
33303 9950 Acked Read 7 Done Sent Done false
33304 9951 Acked Read 6 Done Sent Done false
33305 9952 Acked Write 6 Done Sent Done false
33306 9953 Acked Write 2 Done Sent Done false
33307 9954 Acked Write 7 Done Sent Done false
33308 9955 Acked Write 6 Done Sent Done false
33309 9956 Acked Write 2 Done Sent Done false
33310 9957 Acked Flush 0 Done Sent Done false
33311 9958 Acked Read 10 Done Sent Done false
33312 9959 Acked Write 3 Done Sent Done false
33313 9960 Acked Read 1 Done Sent Done false
33314 9961 Acked Read 6 Done Sent Done false
33315 9962 Acked Read 6 Done Sent Done false
33316 9963 Acked Write 7 Done Sent Done false
33317 9964 Acked Read 7 Done Sent Done false
33318 9965 Acked Write 3 Done Sent Done false
33319 9966 Acked Write 5 Done Sent Done false
33320 9967 Acked Write 2 Done Sent Done false
33321 9968 Acked Read 3 Done Sent Done false
33322 9969 Acked Write 5 Done Sent Done false
33323 9970 Acked Write 8 Done Sent Done false
33324 9971 Acked Read 9 Done Sent Done false
33325 9972 Acked Write 4 Done Sent Done false
33326 9973 Acked Write 10 Done Sent Done false
33327 9974 Acked Read 7 Done Sent Done false
33328 9975 Acked Write 3 Done Sent Done false
33329 9976 Acked Read 4 Done Sent Done false
33330 9977 Acked Write 9 Done Sent Done false
33331 9978 Acked Read 3 Done Sent Done false
33332 9979 Acked Read 5 Done Sent Done false
33333 9980 Acked Write 4 Done Sent Done false
33334 9981 Acked Read 10 Done Sent Done false
33335 9982 Acked Read 5 Done Sent Done false
33336 9983 Acked Read 9 Done Sent Done false
33337 9984 Acked Write 3 Done Sent Done false
33338 9985 Acked Read 4 Done Sent Done false
33339 9986 Acked Read 4 Done Sent Done false
33340 9987 Acked Write 4 Done Sent Done false
33341 9988 Acked Write 9 Done Sent Done false
33342 9989 Acked Write 8 Done Sent Done false
33343 9990 Acked Flush 0 Done Sent Done false
33344 9991 Acked Write 10 Done Sent Done false
33345 9992 Acked Write 4 Done Sent Done false
33346 9993 Acked Read 5 Done Sent Done false
33347 9994 Acked Write 7 Done Sent Done false
33348 9995 Acked Write 5 Done Sent Done false
33349 9996 Acked Flush 0 Done Sent Done false
33350 9997 Acked Read 6 Done Sent Done false
33351 9998 Acked Read 4 Done Sent Done false
33352 9999 Acked Write 5 Done Sent Done false
33353 10000 Acked Read 2 Done Sent Done false
33354 10001 Acked Read 7 Done Sent Done false
33355 10002 Acked Read 8 Done Sent Done false
33356 10003 Acked Write 10 Done Sent Done false
33357 10004 Acked Write 9 Done Sent Done false
33358 10005 Acked Read 5 Done Sent Done false
33359 10006 Acked Write 7 Done Sent Done false
33360 10007 Acked Write 8 Done Sent Done false
33361 10008 Acked Write 1 Done Sent Done false
33362 10009 Acked Read 6 Done Sent Done false
33363 10010 Acked Write 8 Done Sent Done false
33364 10011 Acked Read 10 Done Sent Done false
33365 10012 Acked Flush 0 Done Sent Done false
33366 10013 Acked Read 10 Done Sent Done false
33367 10014 Acked Write 10 Done Sent Done false
33368 10015 Acked Write 1 Done Sent Done false
33369 10016 Acked Write 5 Done Sent Done false
33370 10017 Acked Read 9 Done Sent Done false
33371 10018 Acked Write 5 Done Sent Done false
33372 10019 Acked Write 10 Done Sent Done false
33373 10020 Acked Read 2 Done Sent Done false
33374 10021 Acked Read 6 Done Sent Done false
33375 10022 Acked Write 1 Done Sent Done false
33376 10023 Acked Write 7 Done Sent Done false
33377 10024 Acked Write 10 Done Sent Done false
33378 10025 Acked Read 2 Done Sent Done false
33379 10026 Acked Read 9 Done Sent Done false
33380 10027 Acked Read 3 Done Sent Done false
33381 10028 Acked Write 5 Done Sent Done false
33382 10029 Acked Write 5 Done Sent Done false
33383 10030 Acked Read 6 Done Sent Done false
33384 10031 Acked Read 7 Done Sent Done false
33385 10032 Acked Read 6 Done Sent Done false
33386 10033 Acked Read 7 Done Sent Done false
33387 10034 Acked Write 2 Done Sent Done false
33388 10035 Acked Flush 0 Done Sent Done false
33389 10036 Acked Write 4 Done Sent Done false
33390 10037 Acked Read 5 Done Sent Done false
33391 10038 Acked Read 2 Done Sent Done false
33392 10039 Acked Read 9 Done Sent Done false
33393 10040 Acked Read 1 Done Sent Done false
33394 10041 Acked Read 7 Done Sent Done false
33395 10042 Acked Write 5 Done Sent Done false
33396 10043 Acked Write 8 Done Sent Done false
33397 10044 Acked Write 3 Done Sent Done false
33398 10045 Acked Write 10 Done Sent Done false
33399 10046 Acked Read 8 Done Sent Done false
33400 10047 Acked Read 1 Done Sent Done false
33401 10048 Acked Write 9 Done Sent Done false
33402 10049 Acked Write 1 Done Sent Done false
33403 10050 Acked Read 5 Done Sent Done false
33404 10051 Acked Read 6 Done Sent Done false
33405 10052 Acked Flush 0 Done Sent Done false
33406 10053 Acked Write 7 Done Sent Done false
33407 10054 Acked Read 8 Done Sent Done false
33408 10055 Acked Read 6 Done Sent Done false
33409 10056 Acked Write 10 Done Sent Done false
33410 STATES DS:0 DS:1 DS:2 TOTAL
33411 Sent 0 301 0 301
33412 Done 301 0 301 602
33413 Skipped 0 0 0 0
33414 Error 0 0 0 0
33415 Last Flush: 10052 9755 10052
33416 Downstairs last five completed: 9755 9754 9753 9752 9751
33417 Upstairs last five completed: 10056 10055 10054 10053 10052
33418 CLIENT: Up:0 ds:301 act:2
33419 [8820] got start action
33420 [8820] I am going to restart
33421 Starting downstairs at port 8820
33422 Make output file at "/tmp/dsc/downstairs-8820.txt"
33423 [8820][1] reports Starting
33424 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1232)
33425 [8820][1] reports Running
33426 ----------------------------------------------------------------
33427 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33428 States: Active Offline Active
33429 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33430 9756 Acked Write 2 Done Sent Done false
33431 9757 Acked Read 1 Done Sent Done false
33432 9758 Acked Write 5 Done Sent Done false
33433 9759 Acked Read 2 Done Sent Done false
33434 9760 Acked Read 1 Done Sent Done false
33435 9761 Acked Read 5 Done Sent Done false
33436 9762 Acked Read 7 Done Sent Done false
33437 9763 Acked Write 9 Done Sent Done false
33438 9764 Acked Write 5 Done Sent Done false
33439 9765 Acked Write 10 Done Sent Done false
33440 9766 Acked Flush 0 Done Sent Done false
33441 9767 Acked Read 9 Done Sent Done false
33442 9768 Acked Read 8 Done Sent Done false
33443 9769 Acked Read 3 Done Sent Done false
33444 9770 Acked Read 10 Done Sent Done false
33445 9771 Acked Write 9 Done Sent Done false
33446 9772 Acked Read 6 Done Sent Done false
33447 9773 Acked Read 8 Done Sent Done false
33448 9774 Acked Read 5 Done Sent Done false
33449 9775 Acked Read 5 Done Sent Done false
33450 9776 Acked Write 4 Done Sent Done false
33451 9777 Acked Read 4 Done Sent Done false
33452 9778 Acked Write 6 Done Sent Done false
33453 9779 Acked Write 2 Done Sent Done false
33454 9780 Acked Read 7 Done Sent Done false
33455 9781 Acked Read 8 Done Sent Done false
33456 9782 Acked Read 9 Done Sent Done false
33457 9783 Acked Write 10 Done Sent Done false
33458 9784 Acked Write 9 Done Sent Done false
33459 9785 Acked Flush 0 Done Sent Done false
33460 9786 Acked Write 6 Done Sent Done false
33461 9787 Acked Read 1 Done Sent Done false
33462 9788 Acked Write 10 Done Sent Done false
33463 9789 Acked Read 4 Done Sent Done false
33464 9790 Acked Read 6 Done Sent Done false
33465 9791 Acked Write 1 Done Sent Done false
33466 9792 Acked Write 1 Done Sent Done false
33467 9793 Acked Write 1 Done Sent Done false
33468 9794 Acked Read 4 Done Sent Done false
33469 9795 Acked Flush 0 Done Sent Done false
33470 9796 Acked Read 6 Done Sent Done false
33471 9797 Acked Read 4 Done Sent Done false
33472 9798 Acked Read 6 Done Sent Done false
33473 9799 Acked Read 10 Done Sent Done false
33474 9800 Acked Read 4 Done Sent Done false
33475 9801 Acked Read 2 Done Sent Done false
33476 9802 Acked Write 1 Done Sent Done false
33477 9803 Acked Write 4 Done Sent Done false
33478 9804 Acked Write 5 Done Sent Done false
33479 9805 Acked Write 6 Done Sent Done false
33480 9806 Acked Read 7 Done Sent Done false
33481 9807 Acked Read 4 Done Sent Done false
33482 9808 Acked Flush 0 Done Sent Done false
33483 9809 Acked Read 1 Done Sent Done false
33484 9810 Acked Write 5 Done Sent Done false
33485 9811 Acked Write 3 Done Sent Done false
33486 9812 Acked Write 10 Done Sent Done false
33487 9813 Acked Read 3 Done Sent Done false
33488 9814 Acked Read 6 Done Sent Done false
33489 9815 Acked Read 3 Done Sent Done false
33490 9816 Acked Read 7 Done Sent Done false
33491 9817 Acked Flush 0 Done Sent Done false
33492 9818 Acked Flush 0 Done Sent Done false
33493 9819 Acked Flush 0 Done Sent Done false
33494 9820 Acked Read 9 Done Sent Done false
33495 9821 Acked Read 7 Done Sent Done false
33496 9822 Acked Write 2 Done Sent Done false
33497 9823 Acked Write 4 Done Sent Done false
33498 9824 Acked Flush 0 Done Sent Done false
33499 9825 Acked Read 10 Done Sent Done false
33500 9826 Acked Write 6 Done Sent Done false
33501 9827 Acked Write 6 Done Sent Done false
33502 9828 Acked Write 2 Done Sent Done false
33503 9829 Acked Flush 0 Done Sent Done false
33504 9830 Acked Write 8 Done Sent Done false
33505 9831 Acked Write 5 Done Sent Done false
33506 9832 Acked Read 10 Done Sent Done false
33507 9833 Acked Read 5 Done Sent Done false
33508 9834 Acked Write 5 Done Sent Done false
33509 9835 Acked Write 1 Done Sent Done false
33510 9836 Acked Read 10 Done Sent Done false
33511 9837 Acked Read 9 Done Sent Done false
33512 9838 Acked Write 2 Done Sent Done false
33513 9839 Acked Read 3 Done Sent Done false
33514 9840 Acked Read 5 Done Sent Done false
33515 9841 Acked Read 7 Done Sent Done false
33516 9842 Acked Read 9 Done Sent Done false
33517 9843 Acked Flush 0 Done Sent Done false
33518 9844 Acked Read 6 Done Sent Done false
33519 9845 Acked Write 10 Done Sent Done false
33520 9846 Acked Write 6 Done Sent Done false
33521 9847 Acked Write 10 Done Sent Done false
33522 9848 Acked Write 10 Done Sent Done false
33523 9849 Acked Read 7 Done Sent Done false
33524 9850 Acked Read 9 Done Sent Done false
33525 9851 Acked Read 8 Done Sent Done false
33526 9852 Acked Read 2 Done Sent Done false
33527 9853 Acked Read 2 Done Sent Done false
33528 9854 Acked Read 7 Done Sent Done false
33529 9855 Acked Write 2 Done Sent Done false
33530 9856 Acked Write 3 Done Sent Done false
33531 9857 Acked Read 7 Done Sent Done false
33532 9858 Acked Write 9 Done Sent Done false
33533 9859 Acked Read 10 Done Sent Done false
33534 9860 Acked Write 1 Done Sent Done false
33535 9861 Acked Read 8 Done Sent Done false
33536 9862 Acked Read 3 Done Sent Done false
33537 9863 Acked Read 2 Done Sent Done false
33538 9864 Acked Read 8 Done Sent Done false
33539 9865 Acked Write 2 Done Sent Done false
33540 9866 Acked Write 1 Done Sent Done false
33541 9867 Acked Read 2 Done Sent Done false
33542 9868 Acked Read 10 Done Sent Done false
33543 9869 Acked Flush 0 Done Sent Done false
33544 9870 Acked Read 5 Done Sent Done false
33545 9871 Acked Write 3 Done Sent Done false
33546 9872 Acked Write 7 Done Sent Done false
33547 9873 Acked Write 10 Done Sent Done false
33548 9874 Acked Read 6 Done Sent Done false
33549 9875 Acked Write 9 Done Sent Done false
33550 9876 Acked Write 3 Done Sent Done false
33551 9877 Acked Read 5 Done Sent Done false
33552 9878 Acked Read 10 Done Sent Done false
33553 9879 Acked Read 4 Done Sent Done false
33554 9880 Acked Read 4 Done Sent Done false
33555 9881 Acked Write 10 Done Sent Done false
33556 9882 Acked Read 6 Done Sent Done false
33557 9883 Acked Flush 0 Done Sent Done false
33558 9884 Acked Flush 0 Done Sent Done false
33559 9885 Acked Read 4 Done Sent Done false
33560 9886 Acked Write 4 Done Sent Done false
33561 9887 Acked Read 8 Done Sent Done false
33562 9888 Acked Flush 0 Done Sent Done false
33563 9889 Acked Read 1 Done Sent Done false
33564 9890 Acked Read 7 Done Sent Done false
33565 9891 Acked Flush 0 Done Sent Done false
33566 9892 Acked Write 1 Done Sent Done false
33567 9893 Acked Flush 0 Done Sent Done false
33568 9894 Acked Write 1 Done Sent Done false
33569 9895 Acked Write 6 Done Sent Done false
33570 9896 Acked Read 7 Done Sent Done false
33571 9897 Acked Write 3 Done Sent Done false
33572 9898 Acked Write 5 Done Sent Done false
33573 9899 Acked Read 7 Done Sent Done false
33574 9900 Acked Read 9 Done Sent Done false
33575 9901 Acked Read 10 Done Sent Done false
33576 9902 Acked Write 2 Done Sent Done false
33577 9903 Acked Read 10 Done Sent Done false
33578 9904 Acked Write 9 Done Sent Done false
33579 9905 Acked Write 4 Done Sent Done false
33580 9906 Acked Read 2 Done Sent Done false
33581 9907 Acked Read 10 Done Sent Done false
33582 9908 Acked Read 3 Done Sent Done false
33583 9909 Acked Read 9 Done Sent Done false
33584 9910 Acked Read 8 Done Sent Done false
33585 9911 Acked Write 1 Done Sent Done false
33586 9912 Acked Flush 0 Done Sent Done false
33587 9913 Acked Write 3 Done Sent Done false
33588 9914 Acked Read 5 Done Sent Done false
33589 9915 Acked Flush 0 Done Sent Done false
33590 9916 Acked Write 4 Done Sent Done false
33591 9917 Acked Flush 0 Done Sent Done false
33592 9918 Acked Write 9 Done Sent Done false
33593 9919 Acked Read 4 Done Sent Done false
33594 9920 Acked Read 4 Done Sent Done false
33595 9921 Acked Write 5 Done Sent Done false
33596 9922 Acked Flush 0 Done Sent Done false
33597 9923 Acked Read 5 Done Sent Done false
33598 9924 Acked Read 10 Done Sent Done false
33599 9925 Acked Read 6 Done Sent Done false
33600 9926 Acked Read 4 Done Sent Done false
33601 9927 Acked Read 8 Done Sent Done false
33602 9928 Acked Read 10 Done Sent Done false
33603 9929 Acked Flush 0 Done Sent Done false
33604 9930 Acked Read 7 Done Sent Done false
33605 9931 Acked Write 8 Done Sent Done false
33606 9932 Acked Flush 0 Done Sent Done false
33607 9933 Acked Write 5 Done Sent Done false
33608 9934 Acked Write 4 Done Sent Done false
33609 9935 Acked Write 10 Done Sent Done false
33610 9936 Acked Write 7 Done Sent Done false
33611 9937 Acked Read 6 Done Sent Done false
33612 9938 Acked Flush 0 Done Sent Done false
33613 9939 Acked Read 8 Done Sent Done false
33614 9940 Acked Read 4 Done Sent Done false
33615 9941 Acked Read 6 Done Sent Done false
33616 9942 Acked Write 2 Done Sent Done false
33617 9943 Acked Read 7 Done Sent Done false
33618 9944 Acked Read 1 Done Sent Done false
33619 9945 Acked Read 9 Done Sent Done false
33620 9946 Acked Flush 0 Done Sent Done false
33621 9947 Acked Read 2 Done Sent Done false
33622 9948 Acked Read 9 Done Sent Done false
33623 9949 Acked Read 6 Done Sent Done false
33624 9950 Acked Read 7 Done Sent Done false
33625 9951 Acked Read 6 Done Sent Done false
33626 9952 Acked Write 6 Done Sent Done false
33627 9953 Acked Write 2 Done Sent Done false
33628 9954 Acked Write 7 Done Sent Done false
33629 9955 Acked Write 6 Done Sent Done false
33630 9956 Acked Write 2 Done Sent Done false
33631 9957 Acked Flush 0 Done Sent Done false
33632 9958 Acked Read 10 Done Sent Done false
33633 9959 Acked Write 3 Done Sent Done false
33634 9960 Acked Read 1 Done Sent Done false
33635 9961 Acked Read 6 Done Sent Done false
33636 9962 Acked Read 6 Done Sent Done false
33637 9963 Acked Write 7 Done Sent Done false
33638 9964 Acked Read 7 Done Sent Done false
33639 9965 Acked Write 3 Done Sent Done false
33640 9966 Acked Write 5 Done Sent Done false
33641 9967 Acked Write 2 Done Sent Done false
33642 9968 Acked Read 3 Done Sent Done false
33643 9969 Acked Write 5 Done Sent Done false
33644 9970 Acked Write 8 Done Sent Done false
33645 9971 Acked Read 9 Done Sent Done false
33646 9972 Acked Write 4 Done Sent Done false
33647 9973 Acked Write 10 Done Sent Done false
33648 9974 Acked Read 7 Done Sent Done false
33649 9975 Acked Write 3 Done Sent Done false
33650 9976 Acked Read 4 Done Sent Done false
33651 9977 Acked Write 9 Done Sent Done false
33652 9978 Acked Read 3 Done Sent Done false
33653 9979 Acked Read 5 Done Sent Done false
33654 9980 Acked Write 4 Done Sent Done false
33655 9981 Acked Read 10 Done Sent Done false
33656 9982 Acked Read 5 Done Sent Done false
33657 9983 Acked Read 9 Done Sent Done false
33658 9984 Acked Write 3 Done Sent Done false
33659 9985 Acked Read 4 Done Sent Done false
33660 9986 Acked Read 4 Done Sent Done false
33661 9987 Acked Write 4 Done Sent Done false
33662 9988 Acked Write 9 Done Sent Done false
33663 9989 Acked Write 8 Done Sent Done false
33664 9990 Acked Flush 0 Done Sent Done false
33665 9991 Acked Write 10 Done Sent Done false
33666 9992 Acked Write 4 Done Sent Done false
33667 9993 Acked Read 5 Done Sent Done false
33668 9994 Acked Write 7 Done Sent Done false
33669 9995 Acked Write 5 Done Sent Done false
33670 9996 Acked Flush 0 Done Sent Done false
33671 9997 Acked Read 6 Done Sent Done false
33672 9998 Acked Read 4 Done Sent Done false
33673 9999 Acked Write 5 Done Sent Done false
33674 10000 Acked Read 2 Done Sent Done false
33675 10001 Acked Read 7 Done Sent Done false
33676 10002 Acked Read 8 Done Sent Done false
33677 10003 Acked Write 10 Done Sent Done false
33678 10004 Acked Write 9 Done Sent Done false
33679 10005 Acked Read 5 Done Sent Done false
33680 10006 Acked Write 7 Done Sent Done false
33681 10007 Acked Write 8 Done Sent Done false
33682 10008 Acked Write 1 Done Sent Done false
33683 10009 Acked Read 6 Done Sent Done false
33684 10010 Acked Write 8 Done Sent Done false
33685 10011 Acked Read 10 Done Sent Done false
33686 10012 Acked Flush 0 Done Sent Done false
33687 10013 Acked Read 10 Done Sent Done false
33688 10014 Acked Write 10 Done Sent Done false
33689 10015 Acked Write 1 Done Sent Done false
33690 10016 Acked Write 5 Done Sent Done false
33691 10017 Acked Read 9 Done Sent Done false
33692 10018 Acked Write 5 Done Sent Done false
33693 10019 Acked Write 10 Done Sent Done false
33694 10020 Acked Read 2 Done Sent Done false
33695 10021 Acked Read 6 Done Sent Done false
33696 10022 Acked Write 1 Done Sent Done false
33697 10023 Acked Write 7 Done Sent Done false
33698 10024 Acked Write 10 Done Sent Done false
33699 10025 Acked Read 2 Done Sent Done false
33700 10026 Acked Read 9 Done Sent Done false
33701 10027 Acked Read 3 Done Sent Done false
33702 10028 Acked Write 5 Done Sent Done false
33703 10029 Acked Write 5 Done Sent Done false
33704 10030 Acked Read 6 Done Sent Done false
33705 10031 Acked Read 7 Done Sent Done false
33706 10032 Acked Read 6 Done Sent Done false
33707 10033 Acked Read 7 Done Sent Done false
33708 10034 Acked Write 2 Done Sent Done false
33709 10035 Acked Flush 0 Done Sent Done false
33710 10036 Acked Write 4 Done Sent Done false
33711 10037 Acked Read 5 Done Sent Done false
33712 10038 Acked Read 2 Done Sent Done false
33713 10039 Acked Read 9 Done Sent Done false
33714 10040 Acked Read 1 Done Sent Done false
33715 10041 Acked Read 7 Done Sent Done false
33716 10042 Acked Write 5 Done Sent Done false
33717 10043 Acked Write 8 Done Sent Done false
33718 10044 Acked Write 3 Done Sent Done false
33719 10045 Acked Write 10 Done Sent Done false
33720 10046 Acked Read 8 Done Sent Done false
33721 10047 Acked Read 1 Done Sent Done false
33722 10048 Acked Write 9 Done Sent Done false
33723 10049 Acked Write 1 Done Sent Done false
33724 10050 Acked Read 5 Done Sent Done false
33725 10051 Acked Read 6 Done Sent Done false
33726 10052 Acked Flush 0 Done Sent Done false
33727 10053 Acked Write 7 Done Sent Done false
33728 10054 Acked Read 8 Done Sent Done false
33729 10055 Acked Read 6 Done Sent Done false
33730 10056 Acked Write 10 Done Sent Done false
33731 10057 Acked Flush 0 Done Sent Done false
33732 STATES DS:0 DS:1 DS:2 TOTAL
33733 Sent 0 302 0 302
33734 Done 302 0 302 604
33735 Skipped 0 0 0 0
33736 Error 0 0 0 0
33737 Last Flush: 10057 9755 10057
33738 Downstairs last five completed: 9755 9754 9753 9752 9751
33739 Upstairs last five completed: 10057 10056 10055 10054 10053
33740 CLIENT: Up:0 ds:302 act:2
337412025-01-15T05:26:04.007ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
337422025-01-15T05:26:04.007ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38207) = io task client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
337432025-01-15T05:26:04.008ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
337442025-01-15T05:26:04.008ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
337452025-01-15T05:26:04.009ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
337462025-01-15T05:26:04.009ZINFOcrucible: send last flush ID to this DS: 9755 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
337472025-01-15T05:26:04.009ZINFOcrucible: Replied this last flush ID: 9755 = downstairs client = 1 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
337482025-01-15T05:26:04.009ZINFOcrucible: [1] client re-new 302 jobs since flush 9755 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
337492025-01-15T05:26:04.009ZINFOcrucible: [1] Replayed 302 jobs since flush: 9755 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
33750 ----------------------------------------------------------------
33751 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33752 Downstairs last five completed: 10057 10056 10055 10054 10053
33753 Upstairs last five completed: 10057 10056 10055 10054 10053
33754 CLIENT: Up:0 ds:0 act:3
33755 Replay: All jobs finished
33756 Main task has work to do, go find it
33757 got dsc Stop(0)
33758 stop 0
33759 [8810] Got stop action so:false kr:false
33760 Jan 15 05:26:06.251 INFO request completed, latency_us: 357, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 7b9fcf6f-ec52-48ee-b0c1-9f9d951edd15, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
337612025-01-15T05:26:06.252ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
33762 Jan 15 05:26:06.252{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:26:06.252524232Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
33763 INFO request completed, latency_us: 118, response_code: 200{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-15T05:26:06.252577758Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
33764 , uri: /state/cid/0, method: GET, req_id: 7b303023-d89d-47c2-94e3-e9f8f8bb332e, remote_addr:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:26:06.252630133Z","hostname":"w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1202,"":"io task","client":"0","":"downstairs","session_id":"91709b1d-df49-4651-b81a-5daddf421e6f"}
33765 127.0.0.1:42402, local_addr: 127.0.0.1:9998
33766 [8810] Exited with: ExitStatus(unix_wait_status(9))
33767 [8810][0] reports Exit
33768 Jan 15 05:26:10.254 INFO request completed, latency_us: 194, response_code: 200, uri: /state/cid/0, method: GET, req_id: 9e913761-ac4b-458d-857e-adde4901ed77, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
33769 001/300 Write block 4118 len 16384 data: 9 9 11 8
33770 002/300 Read block 226 len 36864
33771 003/300 Flush
33772 004/300 Flush
33773 005/300 Flush
33774 006/300 Read block 3378 len 16384
33775 007/300 Read block 111 len 4096
33776 008/300 Flush
33777 009/300 Read block 2455 len 12288
33778 010/300 Read block 105 len 32768
33779 011/300 Read block 2250 len 28672
33780 012/300 Read block 2208 len 12288
33781 013/300 Write block 1642 len 24576 data: 8 8 9 7 7 6
33782 014/300 Write block 1182 len 28672 data: 4 5 4 3 4 3 4
33783 015/300 Write block 1841 len 20480 data: 5 6 8 9 11
33784 016/300 Write block 4347 len 28672 data: 8 7 9 7 7 7 6
33785 017/300 Write block 4768 len 16384 data: 4 3 4 5
33786 018/300 Write block 4384 len 20480 data: 7 7 6 7 7
33787 019/300 Write block 1003 len 36864 data: 3 3 2 3 3 3 3 4 4
33788 020/300 Flush
33789 021/300 Read block 3954 len 28672
33790 022/300 Write block 1705 len 12288 data: 8 5 5
33791 023/300 Write block 1291 len 40960 data: 4 6 7 5 5 4 4 5 6 7
33792 024/300 Flush
33793 025/300 Write block 3149 len 40960 data: 5 6 6 8 8 7 8 8 12 12
33794 026/300 Write block 4876 len 32768 data: 5 5 5 4 4 3 4 4
33795 027/300 Write block 2056 len 12288 data: 5 6 6
33796 028/300 Read block 2053 len 8192
33797 029/300 Flush
33798 030/300 Flush
33799 031/300 Write block 800 len 24576 data: 4 3 4 4 5 4
33800 032/300 Read block 4440 len 32768
33801 033/300 Read block 4555 len 40960
33802 034/300 Write block 2295 len 36864 data: 6 7 6 6 7 7 8 10 10
33803 035/300 Write block 396 len 36864 data: 4 4 4 3 5 8 9 10 10
33804 036/300 Read block 4474 len 12288
33805 037/300 Write block 3649 len 24576 data: 5 9 10 7 7 6
33806 038/300 Read block 4650 len 16384
33807 039/300 Write block 3575 len 40960 data: 4 6 6 7 7 7 7 8 9 8
33808 040/300 Flush
33809 041/300 Write block 1705 len 12288 data: 9 6 6
33810 042/300 Read block 3075 len 28672
33811 043/300 Write block 1053 len 28672 data: 2 2 3 4 5 4 4
33812 044/300 Flush
33813 045/300 Read block 4392 len 20480
33814 046/300 Read block 4068 len 20480
33815 047/300 Read block 3921 len 28672
33816 048/300 Write block 1199 len 40960 data: 5 6 8 9 10 11 10 10 10 9
33817 049/300 Read block 4279 len 4096
33818 050/300 Write block 1244 len 28672 data: 4 6 4 7 6 6 6
33819 051/300 Read block 467 len 32768
33820 052/300 Flush
33821 053/300 Read block 727 len 24576
33822 054/300 Read block 4792 len 20480
33823 055/300 Read block 1764 len 32768
33824 056/300 Write block 2600 len 8192 data: 7 5
33825 057/300 Write block 3963 len 8192 data: 3 3
33826 058/300 Read block 1730 len 16384
33827 059/300 Write block 754 len 12288 data: 5 4 4
33828 060/300 Read block 3813 len 12288
33829 061/300 Write block 191 len 40960 data: 7 7 5 5 6 8 7 5 6 7
33830 062/300 Read block 3713 len 36864
33831 063/300 Write block 2126 len 12288 data: 4 5 3
33832 064/300 Write block 240 len 8192 data: 7 6
33833 065/300 Write block 1450 len 12288 data: 5 6 7
33834 066/300 Write block 580 len 16384 data: 6 5 6 5
33835 067/300 Read block 3464 len 12288
33836 068/300 Read block 2186 len 32768
33837 069/300 Flush
33838 070/300 Write block 199 len 24576 data: 7 8 7 8 8 8
33839 071/300 Read block 1905 len 36864
33840 072/300 Read block 3930 len 20480
33841 073/300 Read block 478 len 16384
33842 074/300 Read block 2166 len 12288
33843 075/300 Read block 126 len 40960
33844 076/300 Write block 1290 len 20480 data: 4 5 7 8 6
33845 077/300 Read block 4303 len 8192
33846 078/300 Flush
33847 079/300 Read block 419 len 4096
33848 080/300 Read block 919 len 36864
33849 081/300 Write block 2296 len 12288 data: 8 7 7
33850 082/300 Write block 4416 len 4096 data: 7
33851 083/300 Read block 55 len 28672
33852 084/300 Write block 1336 len 40960 data: 5 5 5 4 4 4 6 6 6 6
33853 085/300 Write block 3225 len 28672 data: 5 4 2 2 2 2 2
33854 086/300 Write block 4789 len 24576 data: 3 2 2 2 2 2
33855 087/300 Read block 4575 len 32768
33856 088/300 Read block 4335 len 28672
33857 089/300 Read block 1010 len 8192
33858 090/300 Write block 4846 len 4096 data: 6
33859 091/300 Write block 1600 len 32768 data: 7 7 7 9 7 7 8 7
33860 092/300 Flush
33861 093/300 Write block 135 len 16384 data: 6 5 5 6
33862 094/300 Write block 827 len 4096 data: 7
33863 095/300 Read block 801 len 28672
33864 096/300 Read block 2130 len 16384
33865 097/300 Write block 995 len 28672 data: 3 4 5 6 6 6 3
33866 098/300 Read block 3114 len 40960
33867 099/300 Write block 4293 len 32768 data: 6 6 5 5 6 6 6 6
33868 100/300 Read block 4454 len 8192
33869 101/300 Read block 2568 len 40960
33870 102/300 Write block 844 len 12288 data: 6 5 4
33871 103/300 Read block 531 len 40960
33872 104/300 Write block 947 len 8192 data: 7 7
33873 105/300 Write block 4205 len 12288 data: 5 5 5
33874 106/300 Read block 3957 len 40960
33875 107/300 Flush
33876 108/300 Write block 1235 len 36864 data: 3 3 2 2 2 2 3 4 4
33877 109/300 Read block 1586 len 20480
33878 110/300 Read block 4102 len 28672
33879 111/300 Write block 4339 len 36864 data: 7 7 5 3 4 7 6 6 9
33880 112/300 Write block 2621 len 20480 data: 4 5 6 5 8
33881 113/300 Read block 3369 len 40960
33882 114/300 Write block 4432 len 8192 data: 8 7
33883 115/300 Read block 4894 len 12288
33884 116/300 Read block 2528 len 28672
33885 117/300 Read block 934 len 4096
33886 118/300 Flush
33887 119/300 Write block 4437 len 28672 data: 4 3 4 5 4 3 3
33888 120/300 Read block 4823 len 4096
33889 121/300 Read block 4176 len 16384
33890 122/300 Write block 3043 len 4096 data: 9
33891 123/300 Read block 4792 len 40960
33892 124/300 Write block 436 len 16384 data: 4 4 5 5
33893 125/300 Read block 1963 len 32768
33894 126/300 Write block 2995 len 16384 data: 6 6 7 4
33895 127/300 Write block 1534 len 12288 data: 6 6 7
33896 128/300 Write block 968 len 32768 data: 5 6 7 4 6 6 7 8
33897 129/300 Write block 184 len 12288 data: 8 8 8
33898 130/300 Read block 1930 len 8192
33899 131/300 Write block 1050 len 20480 data: 4 4 3 3 3
33900 132/300 Read block 982 len 32768
33901 133/300 Read block 4409 len 4096
33902 134/300 Write block 3566 len 28672 data: 6 6 4 3 4 4 3
33903 135/300 Write block 4665 len 4096 data: 4
33904 136/300 Read block 2085 len 20480
33905 137/300 Read block 1381 len 16384
33906 138/300 Read block 2185 len 28672
33907 139/300 Read block 1634 len 28672
33908 140/300 Flush
33909 141/300 Read block 3564 len 36864
33910 142/300 Write block 134 len 12288 data: 8 7 6
33911 143/300 Read block 2564 len 32768
33912 144/300 Read block 3124 len 32768
33913 145/300 Read block 4824 len 20480
33914 146/300 Write block 1354 len 28672 data: 4 6 6 3 4 4 4
33915 147/300 Write block 3850 len 40960 data: 5 5 6 6 6 7 8 7 7 6
33916 148/300 Write block 2254 len 24576 data: 7 8 9 8 8 9
33917 149/300 Write block 2062 len 8192 data: 5 5
33918 150/300 Flush
33919 151/300 Flush
33920 152/300 Read block 948 len 12288
33921 153/300 Write block 4211 len 32768 data: 9 8 7 6 7 6 8 9
33922 154/300 Read block 2715 len 32768
33923 155/300 Flush
33924 156/300 Write block 3296 len 4096 data: 6
33925 157/300 Write block 2100 len 16384 data: 4 6 6 5
33926 158/300 Read block 2653 len 12288
33927 159/300 Read block 1228 len 28672
33928 160/300 Write block 2502 len 40960 data: 8 7 5 7 5 6 7 6 6 6
33929 161/300 Write block 4931 len 32768 data: 3 5 6 6 6 7 8 7
33930 162/300 Read block 4591 len 16384
33931 163/300 Write block 342 len 32768 data: 6 7 6 7 5 6 5 5
33932 164/300 Write block 2382 len 32768 data: 4 5 7 6 5 6 5 5
33933 165/300 Read block 4764 len 12288
33934 166/300 Write block 4473 len 32768 data: 2 2 2 3 3 4 4 4
33935 167/300 Read block 4954 len 40960
33936 168/300 Read block 4723 len 8192
33937 169/300 Write block 584 len 28672 data: 5 6 6 6 6 7 7
33938 170/300 Flush
33939 171/300 Write block 174 len 12288 data: 8 6 8
33940 172/300 Read block 2550 len 32768
33941 173/300 Read block 3983 len 40960
33942 174/300 Write block 1477 len 24576 data: 7 6 7 6 5 6
33943 175/300 Read block 1006 len 28672
33944 176/300 Read block 2402 len 8192
33945 177/300 Write block 4642 len 16384 data: 4 4 5 5
33946 178/300 Read block 1048 len 40960
33947 179/300 Read block 166 len 28672
33948 180/300 Read block 859 len 20480
33949 181/300 Read block 3 len 20480
33950 182/300 Read block 2902 len 12288
33951 183/300 Write block 1222 len 4096 data: 6
33952 184/300 Read block 2045 len 40960
33953 185/300 Flush
33954 186/300 Read block 313 len 8192
33955 187/300 Write block 3519 len 40960 data: 5 4 4 5 7 8 7 6 6 8
33956 188/300 Read block 3220 len 32768
33957 189/300 Write block 3052 len 16384 data: 2 2 4 4
33958 190/300 Read block 4135 len 16384
33959 191/300 Read block 3266 len 32768
33960 192/300 Write block 1462 len 16384 data: 6 6 6 6
33961 193/300 Write block 2666 len 20480 data: 4 5 5 6 5
33962 194/300 Flush
33963 195/300 Write block 4955 len 20480 data: 5 5 5 3 6
33964 196/300 Read block 1032 len 28672
33965 197/300 Write block 1977 len 24576 data: 4 4 5 5 7 7
33966 198/300 Read block 3650 len 16384
33967 199/300 Read block 3249 len 16384
33968 200/300 Write block 4833 len 4096 data: 5
33969 201/300 Read block 3895 len 40960
33970 202/300 Write block 2938 len 20480 data: 5 6 6 8 6
33971 203/300 Read block 3071 len 12288
33972 204/300 Write block 1266 len 24576 data: 8 10 9 6 6 6
33973 205/300 Flush
33974 206/300 Write block 3458 len 20480 data: 6 6 7 5 6
33975 207/300 Write block 3898 len 12288 data: 8 7 7
33976 208/300 Read block 4523 len 4096
33977 209/300 Write block 952 len 20480 data: 7 4 4 4 2
33978 210/300 Read block 4106 len 40960
33979 211/300 Read block 2185 len 12288
33980 212/300 Write block 3782 len 28672 data: 6 7 8 8 7 5 3
33981 213/300 Flush
33982 214/300 Flush
33983 215/300 Read block 967 len 4096
33984 216/300 Write block 3093 len 28672 data: 11 9 7 6 6 6 6
33985 217/300 Read block 2724 len 4096
33986 218/300 Read block 4728 len 4096
33987 219/300 Write block 777 len 16384 data: 5 6 5 4
33988 220/300 Read block 1319 len 28672
33989 221/300 Write block 4345 len 24576 data: 7 7 10 8 10 8
33990 222/300 Write block 3248 len 32768 data: 5 6 8 7 8 8 6 5
33991 223/300 Read block 3316 len 8192
33992 224/300 Write block 1354 len 40960 data: 5 7 7 4 5 5 5 4 2 3
33993 225/300 Write block 37 len 24576 data: 6 5 5 9 8 9
33994 226/300 Read block 3561 len 12288
33995 227/300 Read block 663 len 40960
33996 228/300 Write block 2029 len 20480 data: 10 8 7 5 6
33997 229/300 Read block 1232 len 32768
33998 230/300 Read block 3268 len 36864
33999 231/300 Write block 1170 len 12288 data: 11 8 7
34000 232/300 Read block 2410 len 16384
34001 233/300 Write block 872 len 8192 data: 6 5
34002 234/300 Read block 4197 len 16384
34003 235/300 Read block 4923 len 20480
34004 236/300 Read block 3028 len 16384
34005 237/300 Read block 2183 len 32768
34006 238/300 Read block 1209 len 16384
34007 239/300 Read block 3894 len 24576
34008 240/300 Write block 178 len 40960 data: 5 6 6 8 7 8 9 9 9 7
34009 241/300 Flush
34010 242/300 Flush
34011 243/300 Write block 2235 len 40960 data: 5 8 8 8 7 8 8 8 7 9
34012 244/300 Write block 1273 len 20480 data: 5 6 4 5 4
34013 245/300 Write block 2607 len 4096 data: 6
34014 246/300 Write block 508 len 28672 data: 6 5 6 4 5 5 5
34015 247/300 Write block 354 len 24576 data: 7 8 7 8 8 7
34016 248/300 Read block 351 len 24576
34017 249/300 Read block 3940 len 28672
34018 250/300 Write block 4333 len 24576 data: 7 7 7 7 7 7
34019 251/300 Flush
34020 252/300 Read block 1484 len 24576
34021 253/300 Write block 4400 len 8192 data: 3 5
34022 254/300 Write block 2675 len 16384 data: 7 8 6 7
34023 255/300 Read block 961 len 16384
34024 256/300 Read block 2374 len 16384
34025 257/300 Read block 3486 len 36864
34026 258/300 Read block 1579 len 40960
34027 259/300 Read block 1678 len 40960
34028 260/300 Read block 2375 len 28672
34029 261/300 Read block 3874 len 40960
34030 262/300 Read block 3345 len 40960
34031 263/300 Write block 3975 len 20480 data: 8 7 5 4 4
34032 264/300 Read block 1210 len 40960
34033 265/300 Read block 3450 len 20480
34034 266/300 Read block 2611 len 20480
34035 267/300 Write block 2761 len 4096 data: 8
34036 268/300 Read block 4683 len 4096
34037 269/300 Write block 2303 len 20480 data: 11 8 8 5 5
34038 270/300 Read block 3754 len 36864
34039 271/300 Read block 437 len 4096
34040 272/300 Read block 2894 len 40960
34041 273/300 Read block 4543 len 20480
34042 274/300 Read block 1777 len 36864
34043 275/300 Read block 4415 len 40960
34044 276/300 Read block 2950 len 28672
34045 277/300 Write block 3923 len 32768 data: 3 5 5 6 4 3 3 3
34046 278/300 Write block 867 len 12288 data: 9 9 11
34047 279/300 Flush
34048 280/300 Read block 1718 len 8192
34049 281/300 Read block 3902 len 4096
34050 282/300 Write block 1456 len 4096 data: 10
34051 283/300 Write block 1837 len 32768 data: 6 6 6 6 6 7 9 10
34052 284/300 Read block 479 len 16384
34053 285/300 Write block 3267 len 32768 data: 7 6 8 8 8 8 10 9
34054 286/300 Read block 3606 len 8192
34055 287/300 Read block 523 len 32768
34056 288/300 Write block 2901 len 20480 data: 6 5 5 6 6
34057 289/300 Read block 3964 len 16384
34058 290/300 Read block 4838 len 28672
34059 291/300 Read block 2430 len 28672
34060 292/300 Write block 1946 len 20480 data: 3 4 4 4 4
34061 293/300 Read block 1452 len 4096
34062 294/300 Flush
34063 295/300 Read block 3537 len 4096
34064 296/300 Flush
34065 297/300 Read block 3292 len 40960
34066 298/300 Read block 2137 len 32768
34067 299/300 Read block 4842 len 12288
34068 300/300 Write block 2817 len 20480 data: 4 3 5 6 6
34069 301/300 Read block 492 len 36864
34070 Main task has work to do, go find it
34071 got dsc Start(0)
34072 start 0
34073 Jan 15 05:26:10.492 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/0, method: POST, req_id: bbadf1c3-bad1-457d-8211-f869aaf8eebc, remote_addr: 127.0.0.1:42402, local_addr: 127.0.0.1:9998
34074 Replay: started 0, returned:Ok(())
34075 ----------------------------------------------------------------
34076 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34077 States: Offline Active Active
34078 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34079 10058 Acked Write 4 Sent Done Done false
34080 10059 Acked Read 9 Sent Done Done false
34081 10060 Acked Flush 0 Sent Done Done false
34082 10061 Acked Flush 0 Sent Done Done false
34083 10062 Acked Flush 0 Sent Done Done false
34084 10063 Acked Read 4 Sent Done Done false
34085 10064 Acked Read 1 Sent Done Done false
34086 10065 Acked Flush 0 Sent Done Done false
34087 10066 Acked Read 3 Sent Done Done false
34088 10067 Acked Read 8 Sent Done Done false
34089 10068 Acked Read 7 Sent Done Done false
34090 10069 Acked Read 3 Sent Done Done false
34091 10070 Acked Write 6 Sent Done Done false
34092 10071 Acked Write 7 Sent Done Done false
34093 10072 Acked Write 5 Sent Done Done false
34094 10073 Acked Write 7 Sent Done Done false
34095 10074 Acked Write 4 Sent Done Done false
34096 10075 Acked Write 5 Sent Done Done false
34097 10076 Acked Write 9 Sent Done Done false
34098 10077 Acked Flush 0 Sent Done Done false
34099 10078 Acked Read 7 Sent Done Done false
34100 10079 Acked Write 3 Sent Done Done false
34101 10080 Acked Write 10 Sent Done Done false
34102 10081 Acked Flush 0 Sent Done Done false
34103 10082 Acked Write 10 Sent Done Done false
34104 10083 Acked Write 8 Sent Done Done false
34105 10084 Acked Write 3 Sent Done Done false
34106 10085 Acked Read 2 Sent Done Done false
34107 10086 Acked Flush 0 Sent Done Done false
34108 10087 Acked Flush 0 Sent Done Done false
34109 10088 Acked Write 6 Sent Done Done false
34110 10089 Acked Read 8 Sent Done Done false
34111 10090 Acked Read 10 Sent Done Done false
34112 10091 Acked Write 9 Sent Done Done false
34113 10092 Acked Write 9 Sent Done Done false
34114 10093 Acked Read 3 Sent Done Done false
34115 10094 Acked Write 6 Sent Done Done false
34116 10095 Acked Read 4 Sent Done Done false
34117 10096 Acked Write 10 Sent Done Done false
34118 10097 Acked Flush 0 Sent Done Done false
34119 10098 Acked Write 3 Sent Done Done false
34120 10099 Acked Read 7 Sent Done Done false
34121 10100 Acked Write 7 Sent Done Done false
34122 10101 Acked Flush 0 Sent Done Done false
34123 10102 Acked Read 5 Sent Done Done false
34124 10103 Acked Read 5 Sent Done Done false
34125 10104 Acked Read 7 Sent Done Done false
34126 10105 Acked Write 10 Sent Done Done false
34127 10106 Acked Read 1 Sent Done Done false
34128 10107 Acked Write 7 Sent Done Done false
34129 10108 Acked Read 8 Sent Done Done false
34130 10109 Acked Flush 0 Sent Done Done false
34131 10110 Acked Read 6 Sent Done Done false
34132 10111 Acked Read 5 Sent Done Done false
34133 10112 Acked Read 8 Sent Done Done false
34134 10113 Acked Write 2 Sent Done Done false
34135 10114 Acked Write 2 Sent Done Done false
34136 10115 Acked Read 4 Sent Done Done false
34137 10116 Acked Write 3 Sent Done Done false
34138 10117 Acked Read 3 Sent Done Done false
34139 10118 Acked Write 10 Sent Done Done false
34140 10119 Acked Read 9 Sent Done Done false
34141 10120 Acked Write 3 Sent Done Done false
34142 10121 Acked Write 2 Sent Done Done false
34143 10122 Acked Write 3 Sent Done Done false
34144 10123 Acked Write 4 Sent Done Done false
34145 10124 Acked Read 3 Sent Done Done false
34146 10125 Acked Read 8 Sent Done Done false
34147 10126 Acked Flush 0 Sent Done Done false
34148 10127 Acked Write 6 Sent Done Done false
34149 10128 Acked Read 9 Sent Done Done false
34150 10129 Acked Read 5 Sent Done Done false
34151 10130 Acked Read 4 Sent Done Done false
34152 10131 Acked Read 3 Sent Done Done false
34153 10132 Acked Read 10 Sent Done Done false
34154 10133 Acked Write 5 Sent Done Done false
34155 10134 Acked Read 2 Sent Done Done false
34156 10135 Acked Flush 0 Sent Done Done false
34157 10136 Acked Read 1 Sent Done Done false
34158 10137 Acked Read 9 Sent Done Done false
34159 10138 Acked Write 3 Sent Done Done false
34160 10139 Acked Write 1 Sent Done Done false
34161 10140 Acked Read 7 Sent Done Done false
34162 10141 Acked Write 10 Sent Done Done false
34163 10142 Acked Write 7 Sent Done Done false
34164 10143 Acked Write 6 Sent Done Done false
34165 10144 Acked Read 8 Sent Done Done false
34166 10145 Acked Read 7 Sent Done Done false
34167 10146 Acked Read 2 Sent Done Done false
34168 10147 Acked Write 1 Sent Done Done false
34169 10148 Acked Write 8 Sent Done Done false
34170 10149 Acked Flush 0 Sent Done Done false
34171 10150 Acked Write 4 Sent Done Done false
34172 10151 Acked Write 1 Sent Done Done false
34173 10152 Acked Read 7 Sent Done Done false
34174 10153 Acked Read 4 Sent Done Done false
34175 10154 Acked Write 7 Sent Done Done false
34176 10155 Acked Read 10 Sent Done Done false
34177 10156 Acked Write 8 Sent Done Done false
34178 10157 Acked Read 2 Sent Done Done false
34179 10158 Acked Read 10 Sent Done Done false
34180 10159 Acked Write 3 Sent Done Done false
34181 10160 Acked Read 10 Sent Done Done false
34182 10161 Acked Write 2 Sent Done Done false
34183 10162 Acked Write 3 Sent Done Done false
34184 10163 Acked Read 10 Sent Done Done false
34185 10164 Acked Flush 0 Sent Done Done false
34186 10165 Acked Write 9 Sent Done Done false
34187 10166 Acked Read 5 Sent Done Done false
34188 10167 Acked Read 7 Sent Done Done false
34189 10168 Acked Write 9 Sent Done Done false
34190 10169 Acked Write 5 Sent Done Done false
34191 10170 Acked Read 10 Sent Done Done false
34192 10171 Acked Write 2 Sent Done Done false
34193 10172 Acked Read 3 Sent Done Done false
34194 10173 Acked Read 7 Sent Done Done false
34195 10174 Acked Read 1 Sent Done Done false
34196 10175 Acked Flush 0 Sent Done Done false
34197 10176 Acked Write 7 Sent Done Done false
34198 10177 Acked Read 1 Sent Done Done false
34199 10178 Acked Read 4 Sent Done Done false
34200 10179 Acked Write 1 Sent Done Done false
34201 10180 Acked Read 10 Sent Done Done false
34202 10181 Acked Write 4 Sent Done Done false
34203 10182 Acked Read 8 Sent Done Done false
34204 10183 Acked Write 4 Sent Done Done false
34205 10184 Acked Write 3 Sent Done Done false
34206 10185 Acked Write 8 Sent Done Done false
34207 10186 Acked Write 3 Sent Done Done false
34208 10187 Acked Read 2 Sent Done Done false
34209 10188 Acked Write 5 Sent Done Done false
34210 10189 Acked Read 8 Sent Done Done false
34211 10190 Acked Read 1 Sent Done Done false
34212 10191 Acked Write 7 Sent Done Done false
34213 10192 Acked Write 1 Sent Done Done false
34214 10193 Acked Read 5 Sent Done Done false
34215 10194 Acked Read 4 Sent Done Done false
34216 10195 Acked Read 7 Sent Done Done false
34217 10196 Acked Read 7 Sent Done Done false
34218 10197 Acked Flush 0 Sent Done Done false
34219 10198 Acked Read 9 Sent Done Done false
34220 10199 Acked Write 3 Sent Done Done false
34221 10200 Acked Read 8 Sent Done Done false
34222 10201 Acked Read 8 Sent Done Done false
34223 10202 Acked Read 5 Sent Done Done false
34224 10203 Acked Write 7 Sent Done Done false
34225 10204 Acked Write 10 Sent Done Done false
34226 10205 Acked Write 6 Sent Done Done false
34227 10206 Acked Write 2 Sent Done Done false
34228 10207 Acked Flush 0 Sent Done Done false
34229 10208 Acked Flush 0 Sent Done Done false
34230 10209 Acked Read 3 Sent Done Done false
34231 10210 Acked Write 8 Sent Done Done false
34232 10211 Acked Read 8 Sent Done Done false
34233 10212 Acked Flush 0 Sent Done Done false
34234 10213 Acked Write 1 Sent Done Done false
34235 10214 Acked Write 4 Sent Done Done false
34236 10215 Acked Read 3 Sent Done Done false
34237 10216 Acked Read 7 Sent Done Done false
34238 10217 Acked Write 10 Sent Done Done false
34239 10218 Acked Write 8 Sent Done Done false
34240 10219 Acked Read 4 Sent Done Done false
34241 10220 Acked Write 8 Sent Done Done false
34242 10221 Acked Write 8 Sent Done Done false
34243 10222 Acked Read 3 Sent Done Done false
34244 10223 Acked Write 8 Sent Done Done false
34245 10224 Acked Read 10 Sent Done Done false
34246 10225 Acked Read 2 Sent Done Done false
34247 10226 Acked Write 7 Sent Done Done false
34248 10227 Acked Flush 0 Sent Done Done false
34249 10228 Acked Write 3 Sent Done Done false
34250 10229 Acked Read 8 Sent Done Done false
34251 10230 Acked Read 10 Sent Done Done false
34252 10231 Acked Write 6 Sent Done Done false
34253 10232 Acked Read 7 Sent Done Done false
34254 10233 Acked Read 2 Sent Done Done false
34255 10234 Acked Write 4 Sent Done Done false
34256 10235 Acked Read 10 Sent Done Done false
34257 10236 Acked Read 7 Sent Done Done false
34258 10237 Acked Read 5 Sent Done Done false
34259 10238 Acked Read 5 Sent Done Done false
34260 10239 Acked Read 3 Sent Done Done false
34261 10240 Acked Write 1 Sent Done Done false
34262 10241 Acked Read 10 Sent Done Done false
34263 10242 Acked Flush 0 Sent Done Done false
34264 10243 Acked Read 2 Sent Done Done false
34265 10244 Acked Write 10 Sent Done Done false
34266 10245 Acked Read 8 Sent Done Done false
34267 10246 Acked Write 4 Sent Done Done false
34268 10247 Acked Read 4 Sent Done Done false
34269 10248 Acked Read 8 Sent Done Done false
34270 10249 Acked Write 4 Sent Done Done false
34271 10250 Acked Write 5 Sent Done Done false
34272 10251 Acked Flush 0 Sent Done Done false
34273 10252 Acked Write 5 Sent Done Done false
34274 10253 Acked Read 7 Sent Done Done false
34275 10254 Acked Write 6 Sent Done Done false
34276 10255 Acked Read 4 Sent Done Done false
34277 10256 Acked Read 4 Sent Done Done false
34278 10257 Acked Write 1 Sent Done Done false
34279 10258 Acked Read 10 Sent Done Done false
34280 10259 Acked Write 5 Sent Done Done false
34281 10260 Acked Read 3 Sent Done Done false
34282 10261 Acked Write 6 Sent Done Done false
34283 10262 Acked Flush 0 Sent Done Done false
34284 10263 Acked Write 5 Sent Done Done false
34285 10264 Acked Write 3 Sent Done Done false
34286 10265 Acked Read 1 Sent Done Done false
34287 10266 Acked Write 5 Sent Done Done false
34288 10267 Acked Read 10 Sent Done Done false
34289 10268 Acked Read 3 Sent Done Done false
34290 10269 Acked Write 7 Sent Done Done false
34291 10270 Acked Flush 0 Sent Done Done false
34292 10271 Acked Flush 0 Sent Done Done false
34293 10272 Acked Read 1 Sent Done Done false
34294 10273 Acked Write 7 Sent Done Done false
34295 10274 Acked Read 1 Sent Done Done false
34296 10275 Acked Read 1 Sent Done Done false
34297 10276 Acked Write 4 Sent Done Done false
34298 10277 Acked Read 7 Sent Done Done false
34299 10278 Acked Write 6 Sent Done Done false
34300 10279 Acked Write 8 Sent Done Done false
34301 10280 Acked Read 2 Sent Done Done false
34302 10281 Acked Write 10 Sent Done Done false
34303 10282 Acked Write 6 Sent Done Done false
34304 10283 Acked Read 3 Sent Done Done false
34305 10284 Acked Read 10 Sent Done Done false
34306 10285 Acked Write 5 Sent Done Done false
34307 10286 Acked Read 8 Sent Done Done false
34308 10287 Acked Read 9 Sent Done Done false
34309 10288 Acked Write 3 Sent Done Done false
34310 10289 Acked Read 4 Sent Done Done false
34311 10290 Acked Write 2 Sent Done Done false
34312 10291 Acked Read 4 Sent Done Done false
34313 10292 Acked Read 5 Sent Done Done false
34314 10293 Acked Read 4 Sent Done Done false
34315 10294 Acked Read 8 Sent Done Done false
34316 10295 Acked Read 4 Sent Done Done false
34317 10296 Acked Read 6 Sent Done Done false
34318 10297 Acked Write 10 Sent Done Done false
34319 10298 Acked Flush 0 Sent Done Done false
34320 10299 Acked Flush 0 Sent Done Done false
34321 10300 Acked Write 10 Sent Done Done false
34322 10301 Acked Write 5 Sent Done Done false
34323 10302 Acked Write 1 Sent Done Done false
34324 10303 Acked Write 7 Sent Done Done false
34325 10304 Acked Write 6 Sent Done Done false
34326 10305 Acked Read 6 Sent Done Done false
34327 10306 Acked Read 7 Sent Done Done false
34328 10307 Acked Write 6 Sent Done Done false
34329 10308 Acked Flush 0 Sent Done Done false
34330 10309 Acked Read 6 Sent Done Done false
34331 10310 Acked Write 2 Sent Done Done false
34332 10311 Acked Write 4 Sent Done Done false
34333 10312 Acked Read 4 Sent Done Done false
34334 10313 Acked Read 4 Sent Done Done false
34335 10314 Acked Read 9 Sent Done Done false
34336 10315 Acked Read 10 Sent Done Done false
34337 10316 Acked Read 10 Sent Done Done false
34338 10317 Acked Read 7 Sent Done Done false
34339 10318 Acked Read 10 Sent Done Done false
34340 10319 Acked Read 10 Sent Done Done false
34341 10320 Acked Write 5 Sent Done Done false
34342 10321 Acked Read 10 Sent Done Done false
34343 10322 Acked Read 5 Sent Done Done false
34344 10323 Acked Read 5 Sent Done Done false
34345 10324 Acked Write 1 Sent Done Done false
34346 10325 Acked Read 1 Sent Done Done false
34347 10326 Acked Write 5 Sent Done Done false
34348 10327 Acked Read 9 Sent Done Done false
34349 10328 Acked Read 1 Sent Done Done false
34350 10329 Acked Read 10 Sent Done Done false
34351 10330 Acked Read 5 Sent Done Done false
34352 10331 Acked Read 9 Sent Done Done false
34353 10332 Acked Read 10 Sent Done Done false
34354 10333 Acked Read 7 Sent Done Done false
34355 10334 Acked Write 8 Sent Done Done false
34356 10335 Acked Write 3 Sent Done Done false
34357 10336 Acked Flush 0 Sent Done Done false
34358 10337 Acked Read 2 Sent Done Done false
34359 10338 Acked Read 1 Sent Done Done false
34360 10339 Acked Write 1 Sent Done Done false
34361 10340 Acked Write 8 Sent Done Done false
34362 10341 Acked Read 4 Sent Done Done false
34363 10342 Acked Write 8 Sent Done Done false
34364 10343 Acked Read 2 Sent Done Done false
34365 10344 Acked Read 8 Sent Done Done false
34366 10345 Acked Write 5 Sent Done Done false
34367 10346 Acked Read 4 Sent Done Done false
34368 10347 Acked Read 7 Sent Done Done false
34369 10348 Acked Read 7 Sent Done Done false
34370 10349 Acked Write 5 Sent Done Done false
34371 10350 Acked Read 1 Sent Done Done false
34372 10351 Acked Flush 0 Sent Done Done false
34373 10352 Acked Read 1 Sent Done Done false
34374 10353 Acked Flush 0 Sent Done Done false
34375 10354 Acked Read 10 Sent Done Done false
34376 10355 Acked Read 8 Sent Done Done false
34377 10356 Acked Read 3 Sent Done Done false
34378 10357 Acked Write 5 Sent Done Done false
34379 10358 Acked Read 9 Sent Done Done false
34380 STATES DS:0 DS:1 DS:2 TOTAL
34381 Sent 301 0 0 301
34382 Done 0 301 301 602
34383 Skipped 0 0 0 0
34384 Error 0 0 0 0
34385 Last Flush: 10057 10353 10353
34386 Downstairs last five completed: 10057 10056 10055 10054 10053
34387 Upstairs last five completed: 10358 10357 10356 10355 10354
34388 CLIENT: Up:0 ds:301 act:2
34389 ----------------------------------------------------------------
34390 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34391 States: Offline Active Active
34392 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34393 10058 Acked Write 4 Sent Done Done false
34394 10059 Acked Read 9 Sent Done Done false
34395 10060 Acked Flush 0 Sent Done Done false
34396 10061 Acked Flush 0 Sent Done Done false
34397 10062 Acked Flush 0 Sent Done Done false
34398 10063 Acked Read 4 Sent Done Done false
34399 10064 Acked Read 1 Sent Done Done false
34400 10065 Acked Flush 0 Sent Done Done false
34401 10066 Acked Read 3 Sent Done Done false
34402 10067 Acked Read 8 Sent Done Done false
34403 10068 Acked Read 7 Sent Done Done false
34404 10069 Acked Read 3 Sent Done Done false
34405 10070 Acked Write 6 Sent Done Done false
34406 10071 Acked Write 7 Sent Done Done false
34407 10072 Acked Write 5 Sent Done Done false
34408 10073 Acked Write 7 Sent Done Done false
34409 10074 Acked Write 4 Sent Done Done false
34410 10075 Acked Write 5 Sent Done Done false
34411 10076 Acked Write 9 Sent Done Done false
34412 10077 Acked Flush 0 Sent Done Done false
34413 10078 Acked Read 7 Sent Done Done false
34414 10079 Acked Write 3 Sent Done Done false
34415 10080 Acked Write 10 Sent Done Done false
34416 10081 Acked Flush 0 Sent Done Done false
34417 10082 Acked Write 10 Sent Done Done false
34418 10083 Acked Write 8 Sent Done Done false
34419 10084 Acked Write 3 Sent Done Done false
34420 10085 Acked Read 2 Sent Done Done false
34421 10086 Acked Flush 0 Sent Done Done false
34422 10087 Acked Flush 0 Sent Done Done false
34423 10088 Acked Write 6 Sent Done Done false
34424 10089 Acked Read 8 Sent Done Done false
34425 10090 Acked Read 10 Sent Done Done false
34426 10091 Acked Write 9 Sent Done Done false
34427 10092 Acked Write 9 Sent Done Done false
34428 10093 Acked Read 3 Sent Done Done false
34429 10094 Acked Write 6 Sent Done Done false
34430 10095 Acked Read 4 Sent Done Done false
34431 10096 Acked Write 10 Sent Done Done false
34432 10097 Acked Flush 0 Sent Done Done false
34433 10098 Acked Write 3 Sent Done Done false
34434 10099 Acked Read 7 Sent Done Done false
34435 10100 Acked Write 7 Sent Done Done false
34436 10101 Acked Flush 0 Sent Done Done false
34437 10102 Acked Read 5 Sent Done Done false
34438 10103 Acked Read 5 Sent Done Done false
34439 10104 Acked Read 7 Sent Done Done false
34440 10105 Acked Write 10 Sent Done Done false
34441 10106 Acked Read 1 Sent Done Done false
34442 10107 Acked Write 7 Sent Done Done false
34443 10108 Acked Read 8 Sent Done Done false
34444 10109 Acked Flush 0 Sent Done Done false
34445 10110 Acked Read 6 Sent Done Done false
34446 10111 Acked Read 5 Sent Done Done false
34447 10112 Acked Read 8 Sent Done Done false
34448 10113 Acked Write 2 Sent Done Done false
34449 10114 Acked Write 2 Sent Done Done false
34450 10115 Acked Read 4 Sent Done Done false
34451 10116 Acked Write 3 Sent Done Done false
34452 10117 Acked Read 3 Sent Done Done false
34453 10118 Acked Write 10 Sent Done Done false
34454 10119 Acked Read 9 Sent Done Done false
34455 10120 Acked Write 3 Sent Done Done false
34456 10121 Acked Write 2 Sent Done Done false
34457 10122 Acked Write 3 Sent Done Done false
34458 10123 Acked Write 4 Sent Done Done false
34459 10124 Acked Read 3 Sent Done Done false
34460 10125 Acked Read 8 Sent Done Done false
34461 10126 Acked Flush 0 Sent Done Done false
34462 10127 Acked Write 6 Sent Done Done false
34463 10128 Acked Read 9 Sent Done Done false
34464 10129 Acked Read 5 Sent Done Done false
34465 10130 Acked Read 4 Sent Done Done false
34466 10131 Acked Read 3 Sent Done Done false
34467 10132 Acked Read 10 Sent Done Done false
34468 10133 Acked Write 5 Sent Done Done false
34469 10134 Acked Read 2 Sent Done Done false
34470 10135 Acked Flush 0 Sent Done Done false
34471 10136 Acked Read 1 Sent Done Done false
34472 10137 Acked Read 9 Sent Done Done false
34473 10138 Acked Write 3 Sent Done Done false
34474 10139 Acked Write 1 Sent Done Done false
34475 10140 Acked Read 7 Sent Done Done false
34476 10141 Acked Write 10 Sent Done Done false
34477 10142 Acked Write 7 Sent Done Done false
34478 10143 Acked Write 6 Sent Done Done false
34479 10144 Acked Read 8 Sent Done Done false
34480 10145 Acked Read 7 Sent Done Done false
34481 10146 Acked Read 2 Sent Done Done false
34482 10147 Acked Write 1 Sent Done Done false
34483 10148 Acked Write 8 Sent Done Done false
34484 10149 Acked Flush 0 Sent Done Done false
34485 10150 Acked Write 4 Sent Done Done false
34486 10151 Acked Write 1 Sent Done Done false
34487 10152 Acked Read 7 Sent Done Done false
34488 10153 Acked Read 4 Sent Done Done false
34489 10154 Acked Write 7 Sent Done Done false
34490 10155 Acked Read 10 Sent Done Done false
34491 10156 Acked Write 8 Sent Done Done false
34492 10157 Acked Read 2 Sent Done Done false
34493 10158 Acked Read 10 Sent Done Done false
34494 10159 Acked Write 3 Sent Done Done false
34495 10160 Acked Read 10 Sent Done Done false
34496 10161 Acked Write 2 Sent Done Done false
34497 10162 Acked Write 3 Sent Done Done false
34498 10163 Acked Read 10 Sent Done Done false
34499 10164 Acked Flush 0 Sent Done Done false
34500 10165 Acked Write 9 Sent Done Done false
34501 10166 Acked Read 5 Sent Done Done false
34502 10167 Acked Read 7 Sent Done Done false
34503 10168 Acked Write 9 Sent Done Done false
34504 10169 Acked Write 5 Sent Done Done false
34505 10170 Acked Read 10 Sent Done Done false
34506 10171 Acked Write 2 Sent Done Done false
34507 10172 Acked Read 3 Sent Done Done false
34508 10173 Acked Read 7 Sent Done Done false
34509 10174 Acked Read 1 Sent Done Done false
34510 10175 Acked Flush 0 Sent Done Done false
34511 10176 Acked Write 7 Sent Done Done false
34512 10177 Acked Read 1 Sent Done Done false
34513 10178 Acked Read 4 Sent Done Done false
34514 10179 Acked Write 1 Sent Done Done false
34515 10180 Acked Read 10 Sent Done Done false
34516 10181 Acked Write 4 Sent Done Done false
34517 10182 Acked Read 8 Sent Done Done false
34518 10183 Acked Write 4 Sent Done Done false
34519 10184 Acked Write 3 Sent Done Done false
34520 10185 Acked Write 8 Sent Done Done false
34521 10186 Acked Write 3 Sent Done Done false
34522 10187 Acked Read 2 Sent Done Done false
34523 10188 Acked Write 5 Sent Done Done false
34524 10189 Acked Read 8 Sent Done Done false
34525 10190 Acked Read 1 Sent Done Done false
34526 10191 Acked Write 7 Sent Done Done false
34527 10192 Acked Write 1 Sent Done Done false
34528 10193 Acked Read 5 Sent Done Done false
34529 10194 Acked Read 4 Sent Done Done false
34530 10195 Acked Read 7 Sent Done Done false
34531 10196 Acked Read 7 Sent Done Done false
34532 10197 Acked Flush 0 Sent Done Done false
34533 10198 Acked Read 9 Sent Done Done false
34534 10199 Acked Write 3 Sent Done Done false
34535 10200 Acked Read 8 Sent Done Done false
34536 10201 Acked Read 8 Sent Done Done false
34537 10202 Acked Read 5 Sent Done Done false
34538 10203 Acked Write 7 Sent Done Done false
34539 10204 Acked Write 10 Sent Done Done false
34540 10205 Acked Write 6 Sent Done Done false
34541 10206 Acked Write 2 Sent Done Done false
34542 10207 Acked Flush 0 Sent Done Done false
34543 10208 Acked Flush 0 Sent Done Done false
34544 10209 Acked Read 3 Sent Done Done false
34545 10210 Acked Write 8 Sent Done Done false
34546 10211 Acked Read 8 Sent Done Done false
34547 10212 Acked Flush 0 Sent Done Done false
34548 10213 Acked Write 1 Sent Done Done false
34549 10214 Acked Write 4 Sent Done Done false
34550 10215 Acked Read 3 Sent Done Done false
34551 10216 Acked Read 7 Sent Done Done false
34552 10217 Acked Write 10 Sent Done Done false
34553 10218 Acked Write 8 Sent Done Done false
34554 10219 Acked Read 4 Sent Done Done false
34555 10220 Acked Write 8 Sent Done Done false
34556 10221 Acked Write 8 Sent Done Done false
34557 10222 Acked Read 3 Sent Done Done false
34558 10223 Acked Write 8 Sent Done Done false
34559 10224 Acked Read 10 Sent Done Done false
34560 10225 Acked Read 2 Sent Done Done false
34561 10226 Acked Write 7 Sent Done Done false
34562 10227 Acked Flush 0 Sent Done Done false
34563 10228 Acked Write 3 Sent Done Done false
34564 10229 Acked Read 8 Sent Done Done false
34565 10230 Acked Read 10 Sent Done Done false
34566 10231 Acked Write 6 Sent Done Done false
34567 10232 Acked Read 7 Sent Done Done false
34568 10233 Acked Read 2 Sent Done Done false
34569 10234 Acked Write 4 Sent Done Done false
34570 10235 Acked Read 10 Sent Done Done false
34571 10236 Acked Read 7 Sent Done Done false
34572 10237 Acked Read 5 Sent Done Done false
34573 10238 Acked Read 5 Sent Done Done false
34574 10239 Acked Read 3 Sent Done Done false
34575 10240 Acked Write 1 Sent Done Done false
34576 10241 Acked Read 10 Sent Done Done false
34577 10242 Acked Flush 0 Sent Done Done false
34578 10243 Acked Read 2 Sent Done Done false
34579 10244 Acked Write 10 Sent Done Done false
34580 10245 Acked Read 8 Sent Done Done false
34581 10246 Acked Write 4 Sent Done Done false
34582 10247 Acked Read 4 Sent Done Done false
34583 10248 Acked Read 8 Sent Done Done false
34584 10249 Acked Write 4 Sent Done Done false
34585 10250 Acked Write 5 Sent Done Done false
34586 10251 Acked Flush 0 Sent Done Done false
34587 10252 Acked Write 5 Sent Done Done false
34588 10253 Acked Read 7 Sent Done Done false
34589 10254 Acked Write 6 Sent Done Done false
34590 10255 Acked Read 4 Sent Done Done false
34591 10256 Acked Read 4 Sent Done Done false
34592 10257 Acked Write 1 Sent Done Done false
34593 10258 Acked Read 10 Sent Done Done false
34594 10259 Acked Write 5 Sent Done Done false
34595 10260 Acked Read 3 Sent Done Done false
34596 10261 Acked Write 6 Sent Done Done false
34597 10262 Acked Flush 0 Sent Done Done false
34598 10263 Acked Write 5 Sent Done Done false
34599 10264 Acked Write 3 Sent Done Done false
34600 10265 Acked Read 1 Sent Done Done false
34601 10266 Acked Write 5 Sent Done Done false
34602 10267 Acked Read 10 Sent Done Done false
34603 10268 Acked Read 3 Sent Done Done false
34604 10269 Acked Write 7 Sent Done Done false
34605 10270 Acked Flush 0 Sent Done Done false
34606 10271 Acked Flush 0 Sent Done Done false
34607 10272 Acked Read 1 Sent Done Done false
34608 10273 Acked Write 7 Sent Done Done false
34609 10274 Acked Read 1 Sent Done Done false
34610 10275 Acked Read 1 Sent Done Done false
34611 10276 Acked Write 4 Sent Done Done false
34612 10277 Acked Read 7 Sent Done Done false
34613 10278 Acked Write 6 Sent Done Done false
34614 10279 Acked Write 8 Sent Done Done false
34615 10280 Acked Read 2 Sent Done Done false
34616 10281 Acked Write 10 Sent Done Done false
34617 10282 Acked Write 6 Sent Done Done false
34618 10283 Acked Read 3 Sent Done Done false
34619 10284 Acked Read 10 Sent Done Done false
34620 10285 Acked Write 5 Sent Done Done false
34621 10286 Acked Read 8 Sent Done Done false
34622 10287 Acked Read 9 Sent Done Done false
34623 10288 Acked Write 3 Sent Done Done false
34624 10289 Acked Read 4 Sent Done Done false
34625 10290 Acked Write 2 Sent Done Done false
34626 10291 Acked Read 4 Sent Done Done false
34627 10292 Acked Read 5 Sent Done Done false
34628 10293 Acked Read 4 Sent Done Done false
34629 10294 Acked Read 8 Sent Done Done false
34630 10295 Acked Read 4 Sent Done Done false
34631 10296 Acked Read 6 Sent Done Done false
34632 10297 Acked Write 10 Sent Done Done false
34633 10298 Acked Flush 0 Sent Done Done false
34634 10299 Acked Flush 0 Sent Done Done false
34635 10300 Acked Write 10 Sent Done Done false
34636 10301 Acked Write 5 Sent Done Done false
34637 10302 Acked Write 1 Sent Done Done false
34638 10303 Acked Write 7 Sent Done Done false
34639 10304 Acked Write 6 Sent Done Done false
34640 10305 Acked Read 6 Sent Done Done false
34641 10306 Acked Read 7 Sent Done Done false
34642 10307 Acked Write 6 Sent Done Done false
34643 10308 Acked Flush 0 Sent Done Done false
34644 10309 Acked Read 6 Sent Done Done false
34645 10310 Acked Write 2 Sent Done Done false
34646 10311 Acked Write 4 Sent Done Done false
34647 10312 Acked Read 4 Sent Done Done false
34648 10313 Acked Read 4 Sent Done Done false
34649 10314 Acked Read 9 Sent Done Done false
34650 10315 Acked Read 10 Sent Done Done false
34651 10316 Acked Read 10 Sent Done Done false
34652 10317 Acked Read 7 Sent Done Done false
34653 10318 Acked Read 10 Sent Done Done false
34654 10319 Acked Read 10 Sent Done Done false
34655 10320 Acked Write 5 Sent Done Done false
34656 10321 Acked Read 10 Sent Done Done false
34657 10322 Acked Read 5 Sent Done Done false
34658 10323 Acked Read 5 Sent Done Done false
34659 10324 Acked Write 1 Sent Done Done false
34660 10325 Acked Read 1 Sent Done Done false
34661 10326 Acked Write 5 Sent Done Done false
34662 10327 Acked Read 9 Sent Done Done false
34663 10328 Acked Read 1 Sent Done Done false
34664 10329 Acked Read 10 Sent Done Done false
34665 10330 Acked Read 5 Sent Done Done false
34666 10331 Acked Read 9 Sent Done Done false
34667 10332 Acked Read 10 Sent Done Done false
34668 10333 Acked Read 7 Sent Done Done false
34669 10334 Acked Write 8 Sent Done Done false
34670 10335 Acked Write 3 Sent Done Done false
34671 10336 Acked Flush 0 Sent Done Done false
34672 10337 Acked Read 2 Sent Done Done false
34673 10338 Acked Read 1 Sent Done Done false
34674 10339 Acked Write 1 Sent Done Done false
34675 10340 Acked Write 8 Sent Done Done false
34676 10341 Acked Read 4 Sent Done Done false
34677 10342 Acked Write 8 Sent Done Done false
34678 10343 Acked Read 2 Sent Done Done false
34679 10344 Acked Read 8 Sent Done Done false
34680 10345 Acked Write 5 Sent Done Done false
34681 10346 Acked Read 4 Sent Done Done false
34682 10347 Acked Read 7 Sent Done Done false
34683 10348 Acked Read 7 Sent Done Done false
34684 10349 Acked Write 5 Sent Done Done false
34685 10350 Acked Read 1 Sent Done Done false
34686 10351 Acked Flush 0 Sent Done Done false
34687 10352 Acked Read 1 Sent Done Done false
34688 10353 Acked Flush 0 Sent Done Done false
34689 10354 Acked Read 10 Sent Done Done false
34690 10355 Acked Read 8 Sent Done Done false
34691 10356 Acked Read 3 Sent Done Done false
34692 10357 Acked Write 5 Sent Done Done false
34693 10358 Acked Read 9 Sent Done Done false
34694 10359 Acked Flush 0 Sent Done Done false
34695 STATES DS:0 DS:1 DS:2 TOTAL
34696 Sent 302 0 0 302
34697 Done 0 302 302 604
34698 Skipped 0 0 0 0
34699 Error 0 0 0 0
34700 Last Flush: 10057 10359 10359
34701 Downstairs last five completed: 10057 10056 10055 10054 10053
34702 Upstairs last five completed: 10359 10358 10357 10356 10355
34703 CLIENT: Up:0 ds:302 act:2
347042025-01-15T05:26:16.253ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
347052025-01-15T05:26:16.253ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
347062025-01-15T05:26:16.253ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
347072025-01-15T05:26:16.253ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
347082025-01-15T05:26:16.253ZINFOcrucible: client task is exiting = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
34709 [8810] got start action
34710 [8810] I am going to restart
34711 Starting downstairs at port 8810
34712 Make output file at "/tmp/dsc/downstairs-8810.txt"
34713 [8810][0] reports Starting
34714 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1233)
34715 [8810][0] reports Running
34716 ----------------------------------------------------------------
34717 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34718 States: Offline Active Active
34719 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34720 10058 Acked Write 4 Sent Done Done false
34721 10059 Acked Read 9 Sent Done Done false
34722 10060 Acked Flush 0 Sent Done Done false
34723 10061 Acked Flush 0 Sent Done Done false
34724 10062 Acked Flush 0 Sent Done Done false
34725 10063 Acked Read 4 Sent Done Done false
34726 10064 Acked Read 1 Sent Done Done false
34727 10065 Acked Flush 0 Sent Done Done false
34728 10066 Acked Read 3 Sent Done Done false
34729 10067 Acked Read 8 Sent Done Done false
34730 10068 Acked Read 7 Sent Done Done false
34731 10069 Acked Read 3 Sent Done Done false
34732 10070 Acked Write 6 Sent Done Done false
34733 10071 Acked Write 7 Sent Done Done false
34734 10072 Acked Write 5 Sent Done Done false
34735 10073 Acked Write 7 Sent Done Done false
34736 10074 Acked Write 4 Sent Done Done false
34737 10075 Acked Write 5 Sent Done Done false
34738 10076 Acked Write 9 Sent Done Done false
34739 10077 Acked Flush 0 Sent Done Done false
34740 10078 Acked Read 7 Sent Done Done false
34741 10079 Acked Write 3 Sent Done Done false
34742 10080 Acked Write 10 Sent Done Done false
34743 10081 Acked Flush 0 Sent Done Done false
34744 10082 Acked Write 10 Sent Done Done false
34745 10083 Acked Write 8 Sent Done Done false
34746 10084 Acked Write 3 Sent Done Done false
34747 10085 Acked Read 2 Sent Done Done false
34748 10086 Acked Flush 0 Sent Done Done false
34749 10087 Acked Flush 0 Sent Done Done false
34750 10088 Acked Write 6 Sent Done Done false
34751 10089 Acked Read 8 Sent Done Done false
34752 10090 Acked Read 10 Sent Done Done false
34753 10091 Acked Write 9 Sent Done Done false
34754 10092 Acked Write 9 Sent Done Done false
34755 10093 Acked Read 3 Sent Done Done false
34756 10094 Acked Write 6 Sent Done Done false
34757 10095 Acked Read 4 Sent Done Done false
34758 10096 Acked Write 10 Sent Done Done false
34759 10097 Acked Flush 0 Sent Done Done false
34760 10098 Acked Write 3 Sent Done Done false
34761 10099 Acked Read 7 Sent Done Done false
34762 10100 Acked Write 7 Sent Done Done false
34763 10101 Acked Flush 0 Sent Done Done false
34764 10102 Acked Read 5 Sent Done Done false
34765 10103 Acked Read 5 Sent Done Done false
34766 10104 Acked Read 7 Sent Done Done false
34767 10105 Acked Write 10 Sent Done Done false
34768 10106 Acked Read 1 Sent Done Done false
34769 10107 Acked Write 7 Sent Done Done false
34770 10108 Acked Read 8 Sent Done Done false
34771 10109 Acked Flush 0 Sent Done Done false
34772 10110 Acked Read 6 Sent Done Done false
34773 10111 Acked Read 5 Sent Done Done false
34774 10112 Acked Read 8 Sent Done Done false
34775 10113 Acked Write 2 Sent Done Done false
34776 10114 Acked Write 2 Sent Done Done false
34777 10115 Acked Read 4 Sent Done Done false
34778 10116 Acked Write 3 Sent Done Done false
34779 10117 Acked Read 3 Sent Done Done false
34780 10118 Acked Write 10 Sent Done Done false
34781 10119 Acked Read 9 Sent Done Done false
34782 10120 Acked Write 3 Sent Done Done false
34783 10121 Acked Write 2 Sent Done Done false
34784 10122 Acked Write 3 Sent Done Done false
34785 10123 Acked Write 4 Sent Done Done false
34786 10124 Acked Read 3 Sent Done Done false
34787 10125 Acked Read 8 Sent Done Done false
34788 10126 Acked Flush 0 Sent Done Done false
34789 10127 Acked Write 6 Sent Done Done false
34790 10128 Acked Read 9 Sent Done Done false
34791 10129 Acked Read 5 Sent Done Done false
34792 10130 Acked Read 4 Sent Done Done false
34793 10131 Acked Read 3 Sent Done Done false
34794 10132 Acked Read 10 Sent Done Done false
34795 10133 Acked Write 5 Sent Done Done false
34796 10134 Acked Read 2 Sent Done Done false
34797 10135 Acked Flush 0 Sent Done Done false
34798 10136 Acked Read 1 Sent Done Done false
34799 10137 Acked Read 9 Sent Done Done false
34800 10138 Acked Write 3 Sent Done Done false
34801 10139 Acked Write 1 Sent Done Done false
34802 10140 Acked Read 7 Sent Done Done false
34803 10141 Acked Write 10 Sent Done Done false
34804 10142 Acked Write 7 Sent Done Done false
34805 10143 Acked Write 6 Sent Done Done false
34806 10144 Acked Read 8 Sent Done Done false
34807 10145 Acked Read 7 Sent Done Done false
34808 10146 Acked Read 2 Sent Done Done false
34809 10147 Acked Write 1 Sent Done Done false
34810 10148 Acked Write 8 Sent Done Done false
34811 10149 Acked Flush 0 Sent Done Done false
34812 10150 Acked Write 4 Sent Done Done false
34813 10151 Acked Write 1 Sent Done Done false
34814 10152 Acked Read 7 Sent Done Done false
34815 10153 Acked Read 4 Sent Done Done false
34816 10154 Acked Write 7 Sent Done Done false
34817 10155 Acked Read 10 Sent Done Done false
34818 10156 Acked Write 8 Sent Done Done false
34819 10157 Acked Read 2 Sent Done Done false
34820 10158 Acked Read 10 Sent Done Done false
34821 10159 Acked Write 3 Sent Done Done false
34822 10160 Acked Read 10 Sent Done Done false
34823 10161 Acked Write 2 Sent Done Done false
34824 10162 Acked Write 3 Sent Done Done false
34825 10163 Acked Read 10 Sent Done Done false
34826 10164 Acked Flush 0 Sent Done Done false
34827 10165 Acked Write 9 Sent Done Done false
34828 10166 Acked Read 5 Sent Done Done false
34829 10167 Acked Read 7 Sent Done Done false
34830 10168 Acked Write 9 Sent Done Done false
34831 10169 Acked Write 5 Sent Done Done false
34832 10170 Acked Read 10 Sent Done Done false
34833 10171 Acked Write 2 Sent Done Done false
34834 10172 Acked Read 3 Sent Done Done false
34835 10173 Acked Read 7 Sent Done Done false
34836 10174 Acked Read 1 Sent Done Done false
34837 10175 Acked Flush 0 Sent Done Done false
34838 10176 Acked Write 7 Sent Done Done false
34839 10177 Acked Read 1 Sent Done Done false
34840 10178 Acked Read 4 Sent Done Done false
34841 10179 Acked Write 1 Sent Done Done false
34842 10180 Acked Read 10 Sent Done Done false
34843 10181 Acked Write 4 Sent Done Done false
34844 10182 Acked Read 8 Sent Done Done false
34845 10183 Acked Write 4 Sent Done Done false
34846 10184 Acked Write 3 Sent Done Done false
34847 10185 Acked Write 8 Sent Done Done false
34848 10186 Acked Write 3 Sent Done Done false
34849 10187 Acked Read 2 Sent Done Done false
34850 10188 Acked Write 5 Sent Done Done false
34851 10189 Acked Read 8 Sent Done Done false
34852 10190 Acked Read 1 Sent Done Done false
34853 10191 Acked Write 7 Sent Done Done false
34854 10192 Acked Write 1 Sent Done Done false
34855 10193 Acked Read 5 Sent Done Done false
34856 10194 Acked Read 4 Sent Done Done false
34857 10195 Acked Read 7 Sent Done Done false
34858 10196 Acked Read 7 Sent Done Done false
34859 10197 Acked Flush 0 Sent Done Done false
34860 10198 Acked Read 9 Sent Done Done false
34861 10199 Acked Write 3 Sent Done Done false
34862 10200 Acked Read 8 Sent Done Done false
34863 10201 Acked Read 8 Sent Done Done false
34864 10202 Acked Read 5 Sent Done Done false
34865 10203 Acked Write 7 Sent Done Done false
34866 10204 Acked Write 10 Sent Done Done false
34867 10205 Acked Write 6 Sent Done Done false
34868 10206 Acked Write 2 Sent Done Done false
34869 10207 Acked Flush 0 Sent Done Done false
34870 10208 Acked Flush 0 Sent Done Done false
34871 10209 Acked Read 3 Sent Done Done false
34872 10210 Acked Write 8 Sent Done Done false
34873 10211 Acked Read 8 Sent Done Done false
34874 10212 Acked Flush 0 Sent Done Done false
34875 10213 Acked Write 1 Sent Done Done false
34876 10214 Acked Write 4 Sent Done Done false
34877 10215 Acked Read 3 Sent Done Done false
34878 10216 Acked Read 7 Sent Done Done false
34879 10217 Acked Write 10 Sent Done Done false
34880 10218 Acked Write 8 Sent Done Done false
34881 10219 Acked Read 4 Sent Done Done false
34882 10220 Acked Write 8 Sent Done Done false
34883 10221 Acked Write 8 Sent Done Done false
34884 10222 Acked Read 3 Sent Done Done false
34885 10223 Acked Write 8 Sent Done Done false
34886 10224 Acked Read 10 Sent Done Done false
34887 10225 Acked Read 2 Sent Done Done false
34888 10226 Acked Write 7 Sent Done Done false
34889 10227 Acked Flush 0 Sent Done Done false
34890 10228 Acked Write 3 Sent Done Done false
34891 10229 Acked Read 8 Sent Done Done false
34892 10230 Acked Read 10 Sent Done Done false
34893 10231 Acked Write 6 Sent Done Done false
34894 10232 Acked Read 7 Sent Done Done false
34895 10233 Acked Read 2 Sent Done Done false
34896 10234 Acked Write 4 Sent Done Done false
34897 10235 Acked Read 10 Sent Done Done false
34898 10236 Acked Read 7 Sent Done Done false
34899 10237 Acked Read 5 Sent Done Done false
34900 10238 Acked Read 5 Sent Done Done false
34901 10239 Acked Read 3 Sent Done Done false
34902 10240 Acked Write 1 Sent Done Done false
34903 10241 Acked Read 10 Sent Done Done false
34904 10242 Acked Flush 0 Sent Done Done false
34905 10243 Acked Read 2 Sent Done Done false
34906 10244 Acked Write 10 Sent Done Done false
34907 10245 Acked Read 8 Sent Done Done false
34908 10246 Acked Write 4 Sent Done Done false
34909 10247 Acked Read 4 Sent Done Done false
34910 10248 Acked Read 8 Sent Done Done false
34911 10249 Acked Write 4 Sent Done Done false
34912 10250 Acked Write 5 Sent Done Done false
34913 10251 Acked Flush 0 Sent Done Done false
34914 10252 Acked Write 5 Sent Done Done false
34915 10253 Acked Read 7 Sent Done Done false
34916 10254 Acked Write 6 Sent Done Done false
34917 10255 Acked Read 4 Sent Done Done false
34918 10256 Acked Read 4 Sent Done Done false
34919 10257 Acked Write 1 Sent Done Done false
34920 10258 Acked Read 10 Sent Done Done false
34921 10259 Acked Write 5 Sent Done Done false
34922 10260 Acked Read 3 Sent Done Done false
34923 10261 Acked Write 6 Sent Done Done false
34924 10262 Acked Flush 0 Sent Done Done false
34925 10263 Acked Write 5 Sent Done Done false
34926 10264 Acked Write 3 Sent Done Done false
34927 10265 Acked Read 1 Sent Done Done false
34928 10266 Acked Write 5 Sent Done Done false
34929 10267 Acked Read 10 Sent Done Done false
34930 10268 Acked Read 3 Sent Done Done false
34931 10269 Acked Write 7 Sent Done Done false
34932 10270 Acked Flush 0 Sent Done Done false
34933 10271 Acked Flush 0 Sent Done Done false
34934 10272 Acked Read 1 Sent Done Done false
34935 10273 Acked Write 7 Sent Done Done false
34936 10274 Acked Read 1 Sent Done Done false
34937 10275 Acked Read 1 Sent Done Done false
34938 10276 Acked Write 4 Sent Done Done false
34939 10277 Acked Read 7 Sent Done Done false
34940 10278 Acked Write 6 Sent Done Done false
34941 10279 Acked Write 8 Sent Done Done false
34942 10280 Acked Read 2 Sent Done Done false
34943 10281 Acked Write 10 Sent Done Done false
34944 10282 Acked Write 6 Sent Done Done false
34945 10283 Acked Read 3 Sent Done Done false
34946 10284 Acked Read 10 Sent Done Done false
34947 10285 Acked Write 5 Sent Done Done false
34948 10286 Acked Read 8 Sent Done Done false
34949 10287 Acked Read 9 Sent Done Done false
34950 10288 Acked Write 3 Sent Done Done false
34951 10289 Acked Read 4 Sent Done Done false
34952 10290 Acked Write 2 Sent Done Done false
34953 10291 Acked Read 4 Sent Done Done false
34954 10292 Acked Read 5 Sent Done Done false
34955 10293 Acked Read 4 Sent Done Done false
34956 10294 Acked Read 8 Sent Done Done false
34957 10295 Acked Read 4 Sent Done Done false
34958 10296 Acked Read 6 Sent Done Done false
34959 10297 Acked Write 10 Sent Done Done false
34960 10298 Acked Flush 0 Sent Done Done false
34961 10299 Acked Flush 0 Sent Done Done false
34962 10300 Acked Write 10 Sent Done Done false
34963 10301 Acked Write 5 Sent Done Done false
34964 10302 Acked Write 1 Sent Done Done false
34965 10303 Acked Write 7 Sent Done Done false
34966 10304 Acked Write 6 Sent Done Done false
34967 10305 Acked Read 6 Sent Done Done false
34968 10306 Acked Read 7 Sent Done Done false
34969 10307 Acked Write 6 Sent Done Done false
34970 10308 Acked Flush 0 Sent Done Done false
34971 10309 Acked Read 6 Sent Done Done false
34972 10310 Acked Write 2 Sent Done Done false
34973 10311 Acked Write 4 Sent Done Done false
34974 10312 Acked Read 4 Sent Done Done false
34975 10313 Acked Read 4 Sent Done Done false
34976 10314 Acked Read 9 Sent Done Done false
34977 10315 Acked Read 10 Sent Done Done false
34978 10316 Acked Read 10 Sent Done Done false
34979 10317 Acked Read 7 Sent Done Done false
34980 10318 Acked Read 10 Sent Done Done false
34981 10319 Acked Read 10 Sent Done Done false
34982 10320 Acked Write 5 Sent Done Done false
34983 10321 Acked Read 10 Sent Done Done false
34984 10322 Acked Read 5 Sent Done Done false
34985 10323 Acked Read 5 Sent Done Done false
34986 10324 Acked Write 1 Sent Done Done false
34987 10325 Acked Read 1 Sent Done Done false
34988 10326 Acked Write 5 Sent Done Done false
34989 10327 Acked Read 9 Sent Done Done false
34990 10328 Acked Read 1 Sent Done Done false
34991 10329 Acked Read 10 Sent Done Done false
34992 10330 Acked Read 5 Sent Done Done false
34993 10331 Acked Read 9 Sent Done Done false
34994 10332 Acked Read 10 Sent Done Done false
34995 10333 Acked Read 7 Sent Done Done false
34996 10334 Acked Write 8 Sent Done Done false
34997 10335 Acked Write 3 Sent Done Done false
34998 10336 Acked Flush 0 Sent Done Done false
34999 10337 Acked Read 2 Sent Done Done false
35000 10338 Acked Read 1 Sent Done Done false
35001 10339 Acked Write 1 Sent Done Done false
35002 10340 Acked Write 8 Sent Done Done false
35003 10341 Acked Read 4 Sent Done Done false
35004 10342 Acked Write 8 Sent Done Done false
35005 10343 Acked Read 2 Sent Done Done false
35006 10344 Acked Read 8 Sent Done Done false
35007 10345 Acked Write 5 Sent Done Done false
35008 10346 Acked Read 4 Sent Done Done false
35009 10347 Acked Read 7 Sent Done Done false
35010 10348 Acked Read 7 Sent Done Done false
35011 10349 Acked Write 5 Sent Done Done false
35012 10350 Acked Read 1 Sent Done Done false
35013 10351 Acked Flush 0 Sent Done Done false
35014 10352 Acked Read 1 Sent Done Done false
35015 10353 Acked Flush 0 Sent Done Done false
35016 10354 Acked Read 10 Sent Done Done false
35017 10355 Acked Read 8 Sent Done Done false
35018 10356 Acked Read 3 Sent Done Done false
35019 10357 Acked Write 5 Sent Done Done false
35020 10358 Acked Read 9 Sent Done Done false
35021 10359 Acked Flush 0 Sent Done Done false
35022 STATES DS:0 DS:1 DS:2 TOTAL
35023 Sent 302 0 0 302
35024 Done 0 302 302 604
35025 Skipped 0 0 0 0
35026 Error 0 0 0 0
35027 Last Flush: 10057 10359 10359
35028 Downstairs last five completed: 10057 10056 10055 10054 10053
35029 Upstairs last five completed: 10359 10358 10357 10356 10355
35030 CLIENT: Up:0 ds:302 act:2
35031 ----------------------------------------------------------------
35032 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35033 States: Offline Active Active
35034 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35035 10058 Acked Write 4 Sent Done Done false
35036 10059 Acked Read 9 Sent Done Done false
35037 10060 Acked Flush 0 Sent Done Done false
35038 10061 Acked Flush 0 Sent Done Done false
35039 10062 Acked Flush 0 Sent Done Done false
35040 10063 Acked Read 4 Sent Done Done false
35041 10064 Acked Read 1 Sent Done Done false
35042 10065 Acked Flush 0 Sent Done Done false
35043 10066 Acked Read 3 Sent Done Done false
35044 10067 Acked Read 8 Sent Done Done false
35045 10068 Acked Read 7 Sent Done Done false
35046 10069 Acked Read 3 Sent Done Done false
35047 10070 Acked Write 6 Sent Done Done false
35048 10071 Acked Write 7 Sent Done Done false
35049 10072 Acked Write 5 Sent Done Done false
35050 10073 Acked Write 7 Sent Done Done false
35051 10074 Acked Write 4 Sent Done Done false
35052 10075 Acked Write 5 Sent Done Done false
35053 10076 Acked Write 9 Sent Done Done false
35054 10077 Acked Flush 0 Sent Done Done false
35055 10078 Acked Read 7 Sent Done Done false
35056 10079 Acked Write 3 Sent Done Done false
35057 10080 Acked Write 10 Sent Done Done false
35058 10081 Acked Flush 0 Sent Done Done false
35059 10082 Acked Write 10 Sent Done Done false
35060 10083 Acked Write 8 Sent Done Done false
35061 10084 Acked Write 3 Sent Done Done false
35062 10085 Acked Read 2 Sent Done Done false
35063 10086 Acked Flush 0 Sent Done Done false
35064 10087 Acked Flush 0 Sent Done Done false
35065 10088 Acked Write 6 Sent Done Done false
35066 10089 Acked Read 8 Sent Done Done false
35067 10090 Acked Read 10 Sent Done Done false
35068 10091 Acked Write 9 Sent Done Done false
35069 10092 Acked Write 9 Sent Done Done false
35070 10093 Acked Read 3 Sent Done Done false
35071 10094 Acked Write 6 Sent Done Done false
35072 10095 Acked Read 4 Sent Done Done false
35073 10096 Acked Write 10 Sent Done Done false
35074 10097 Acked Flush 0 Sent Done Done false
35075 10098 Acked Write 3 Sent Done Done false
35076 10099 Acked Read 7 Sent Done Done false
35077 10100 Acked Write 7 Sent Done Done false
35078 10101 Acked Flush 0 Sent Done Done false
35079 10102 Acked Read 5 Sent Done Done false
35080 10103 Acked Read 5 Sent Done Done false
35081 10104 Acked Read 7 Sent Done Done false
35082 10105 Acked Write 10 Sent Done Done false
35083 10106 Acked Read 1 Sent Done Done false
35084 10107 Acked Write 7 Sent Done Done false
35085 10108 Acked Read 8 Sent Done Done false
35086 10109 Acked Flush 0 Sent Done Done false
35087 10110 Acked Read 6 Sent Done Done false
35088 10111 Acked Read 5 Sent Done Done false
35089 10112 Acked Read 8 Sent Done Done false
35090 10113 Acked Write 2 Sent Done Done false
35091 10114 Acked Write 2 Sent Done Done false
35092 10115 Acked Read 4 Sent Done Done false
35093 10116 Acked Write 3 Sent Done Done false
35094 10117 Acked Read 3 Sent Done Done false
35095 10118 Acked Write 10 Sent Done Done false
35096 10119 Acked Read 9 Sent Done Done false
35097 10120 Acked Write 3 Sent Done Done false
35098 10121 Acked Write 2 Sent Done Done false
35099 10122 Acked Write 3 Sent Done Done false
35100 10123 Acked Write 4 Sent Done Done false
35101 10124 Acked Read 3 Sent Done Done false
35102 10125 Acked Read 8 Sent Done Done false
35103 10126 Acked Flush 0 Sent Done Done false
35104 10127 Acked Write 6 Sent Done Done false
35105 10128 Acked Read 9 Sent Done Done false
35106 10129 Acked Read 5 Sent Done Done false
35107 10130 Acked Read 4 Sent Done Done false
35108 10131 Acked Read 3 Sent Done Done false
35109 10132 Acked Read 10 Sent Done Done false
35110 10133 Acked Write 5 Sent Done Done false
35111 10134 Acked Read 2 Sent Done Done false
35112 10135 Acked Flush 0 Sent Done Done false
35113 10136 Acked Read 1 Sent Done Done false
35114 10137 Acked Read 9 Sent Done Done false
35115 10138 Acked Write 3 Sent Done Done false
35116 10139 Acked Write 1 Sent Done Done false
35117 10140 Acked Read 7 Sent Done Done false
35118 10141 Acked Write 10 Sent Done Done false
35119 10142 Acked Write 7 Sent Done Done false
35120 10143 Acked Write 6 Sent Done Done false
35121 10144 Acked Read 8 Sent Done Done false
35122 10145 Acked Read 7 Sent Done Done false
35123 10146 Acked Read 2 Sent Done Done false
35124 10147 Acked Write 1 Sent Done Done false
35125 10148 Acked Write 8 Sent Done Done false
35126 10149 Acked Flush 0 Sent Done Done false
35127 10150 Acked Write 4 Sent Done Done false
35128 10151 Acked Write 1 Sent Done Done false
35129 10152 Acked Read 7 Sent Done Done false
35130 10153 Acked Read 4 Sent Done Done false
35131 10154 Acked Write 7 Sent Done Done false
35132 10155 Acked Read 10 Sent Done Done false
35133 10156 Acked Write 8 Sent Done Done false
35134 10157 Acked Read 2 Sent Done Done false
35135 10158 Acked Read 10 Sent Done Done false
35136 10159 Acked Write 3 Sent Done Done false
35137 10160 Acked Read 10 Sent Done Done false
35138 10161 Acked Write 2 Sent Done Done false
35139 10162 Acked Write 3 Sent Done Done false
35140 10163 Acked Read 10 Sent Done Done false
35141 10164 Acked Flush 0 Sent Done Done false
35142 10165 Acked Write 9 Sent Done Done false
35143 10166 Acked Read 5 Sent Done Done false
35144 10167 Acked Read 7 Sent Done Done false
35145 10168 Acked Write 9 Sent Done Done false
35146 10169 Acked Write 5 Sent Done Done false
35147 10170 Acked Read 10 Sent Done Done false
35148 10171 Acked Write 2 Sent Done Done false
35149 10172 Acked Read 3 Sent Done Done false
35150 10173 Acked Read 7 Sent Done Done false
35151 10174 Acked Read 1 Sent Done Done false
35152 10175 Acked Flush 0 Sent Done Done false
35153 10176 Acked Write 7 Sent Done Done false
35154 10177 Acked Read 1 Sent Done Done false
35155 10178 Acked Read 4 Sent Done Done false
35156 10179 Acked Write 1 Sent Done Done false
35157 10180 Acked Read 10 Sent Done Done false
35158 10181 Acked Write 4 Sent Done Done false
35159 10182 Acked Read 8 Sent Done Done false
35160 10183 Acked Write 4 Sent Done Done false
35161 10184 Acked Write 3 Sent Done Done false
35162 10185 Acked Write 8 Sent Done Done false
35163 10186 Acked Write 3 Sent Done Done false
35164 10187 Acked Read 2 Sent Done Done false
35165 10188 Acked Write 5 Sent Done Done false
35166 10189 Acked Read 8 Sent Done Done false
35167 10190 Acked Read 1 Sent Done Done false
35168 10191 Acked Write 7 Sent Done Done false
35169 10192 Acked Write 1 Sent Done Done false
35170 10193 Acked Read 5 Sent Done Done false
35171 10194 Acked Read 4 Sent Done Done false
35172 10195 Acked Read 7 Sent Done Done false
35173 10196 Acked Read 7 Sent Done Done false
35174 10197 Acked Flush 0 Sent Done Done false
35175 10198 Acked Read 9 Sent Done Done false
35176 10199 Acked Write 3 Sent Done Done false
35177 10200 Acked Read 8 Sent Done Done false
35178 10201 Acked Read 8 Sent Done Done false
35179 10202 Acked Read 5 Sent Done Done false
35180 10203 Acked Write 7 Sent Done Done false
35181 10204 Acked Write 10 Sent Done Done false
35182 10205 Acked Write 6 Sent Done Done false
35183 10206 Acked Write 2 Sent Done Done false
35184 10207 Acked Flush 0 Sent Done Done false
35185 10208 Acked Flush 0 Sent Done Done false
35186 10209 Acked Read 3 Sent Done Done false
35187 10210 Acked Write 8 Sent Done Done false
35188 10211 Acked Read 8 Sent Done Done false
35189 10212 Acked Flush 0 Sent Done Done false
35190 10213 Acked Write 1 Sent Done Done false
35191 10214 Acked Write 4 Sent Done Done false
35192 10215 Acked Read 3 Sent Done Done false
35193 10216 Acked Read 7 Sent Done Done false
35194 10217 Acked Write 10 Sent Done Done false
35195 10218 Acked Write 8 Sent Done Done false
35196 10219 Acked Read 4 Sent Done Done false
35197 10220 Acked Write 8 Sent Done Done false
35198 10221 Acked Write 8 Sent Done Done false
35199 10222 Acked Read 3 Sent Done Done false
35200 10223 Acked Write 8 Sent Done Done false
35201 10224 Acked Read 10 Sent Done Done false
35202 10225 Acked Read 2 Sent Done Done false
35203 10226 Acked Write 7 Sent Done Done false
35204 10227 Acked Flush 0 Sent Done Done false
35205 10228 Acked Write 3 Sent Done Done false
35206 10229 Acked Read 8 Sent Done Done false
35207 10230 Acked Read 10 Sent Done Done false
35208 10231 Acked Write 6 Sent Done Done false
35209 10232 Acked Read 7 Sent Done Done false
35210 10233 Acked Read 2 Sent Done Done false
35211 10234 Acked Write 4 Sent Done Done false
35212 10235 Acked Read 10 Sent Done Done false
35213 10236 Acked Read 7 Sent Done Done false
35214 10237 Acked Read 5 Sent Done Done false
35215 10238 Acked Read 5 Sent Done Done false
35216 10239 Acked Read 3 Sent Done Done false
35217 10240 Acked Write 1 Sent Done Done false
35218 10241 Acked Read 10 Sent Done Done false
35219 10242 Acked Flush 0 Sent Done Done false
35220 10243 Acked Read 2 Sent Done Done false
35221 10244 Acked Write 10 Sent Done Done false
35222 10245 Acked Read 8 Sent Done Done false
35223 10246 Acked Write 4 Sent Done Done false
35224 10247 Acked Read 4 Sent Done Done false
35225 10248 Acked Read 8 Sent Done Done false
35226 10249 Acked Write 4 Sent Done Done false
35227 10250 Acked Write 5 Sent Done Done false
35228 10251 Acked Flush 0 Sent Done Done false
35229 10252 Acked Write 5 Sent Done Done false
35230 10253 Acked Read 7 Sent Done Done false
35231 10254 Acked Write 6 Sent Done Done false
35232 10255 Acked Read 4 Sent Done Done false
35233 10256 Acked Read 4 Sent Done Done false
35234 10257 Acked Write 1 Sent Done Done false
35235 10258 Acked Read 10 Sent Done Done false
35236 10259 Acked Write 5 Sent Done Done false
35237 10260 Acked Read 3 Sent Done Done false
35238 10261 Acked Write 6 Sent Done Done false
35239 10262 Acked Flush 0 Sent Done Done false
35240 10263 Acked Write 5 Sent Done Done false
35241 10264 Acked Write 3 Sent Done Done false
35242 10265 Acked Read 1 Sent Done Done false
35243 10266 Acked Write 5 Sent Done Done false
35244 10267 Acked Read 10 Sent Done Done false
35245 10268 Acked Read 3 Sent Done Done false
35246 10269 Acked Write 7 Sent Done Done false
35247 10270 Acked Flush 0 Sent Done Done false
35248 10271 Acked Flush 0 Sent Done Done false
35249 10272 Acked Read 1 Sent Done Done false
35250 10273 Acked Write 7 Sent Done Done false
35251 10274 Acked Read 1 Sent Done Done false
35252 10275 Acked Read 1 Sent Done Done false
35253 10276 Acked Write 4 Sent Done Done false
35254 10277 Acked Read 7 Sent Done Done false
35255 10278 Acked Write 6 Sent Done Done false
35256 10279 Acked Write 8 Sent Done Done false
35257 10280 Acked Read 2 Sent Done Done false
35258 10281 Acked Write 10 Sent Done Done false
35259 10282 Acked Write 6 Sent Done Done false
35260 10283 Acked Read 3 Sent Done Done false
35261 10284 Acked Read 10 Sent Done Done false
35262 10285 Acked Write 5 Sent Done Done false
35263 10286 Acked Read 8 Sent Done Done false
35264 10287 Acked Read 9 Sent Done Done false
35265 10288 Acked Write 3 Sent Done Done false
35266 10289 Acked Read 4 Sent Done Done false
35267 10290 Acked Write 2 Sent Done Done false
35268 10291 Acked Read 4 Sent Done Done false
35269 10292 Acked Read 5 Sent Done Done false
35270 10293 Acked Read 4 Sent Done Done false
35271 10294 Acked Read 8 Sent Done Done false
35272 10295 Acked Read 4 Sent Done Done false
35273 10296 Acked Read 6 Sent Done Done false
35274 10297 Acked Write 10 Sent Done Done false
35275 10298 Acked Flush 0 Sent Done Done false
35276 10299 Acked Flush 0 Sent Done Done false
35277 10300 Acked Write 10 Sent Done Done false
35278 10301 Acked Write 5 Sent Done Done false
35279 10302 Acked Write 1 Sent Done Done false
35280 10303 Acked Write 7 Sent Done Done false
35281 10304 Acked Write 6 Sent Done Done false
35282 10305 Acked Read 6 Sent Done Done false
35283 10306 Acked Read 7 Sent Done Done false
35284 10307 Acked Write 6 Sent Done Done false
35285 10308 Acked Flush 0 Sent Done Done false
35286 10309 Acked Read 6 Sent Done Done false
35287 10310 Acked Write 2 Sent Done Done false
35288 10311 Acked Write 4 Sent Done Done false
35289 10312 Acked Read 4 Sent Done Done false
35290 10313 Acked Read 4 Sent Done Done false
35291 10314 Acked Read 9 Sent Done Done false
35292 10315 Acked Read 10 Sent Done Done false
35293 10316 Acked Read 10 Sent Done Done false
35294 10317 Acked Read 7 Sent Done Done false
35295 10318 Acked Read 10 Sent Done Done false
35296 10319 Acked Read 10 Sent Done Done false
35297 10320 Acked Write 5 Sent Done Done false
35298 10321 Acked Read 10 Sent Done Done false
35299 10322 Acked Read 5 Sent Done Done false
35300 10323 Acked Read 5 Sent Done Done false
35301 10324 Acked Write 1 Sent Done Done false
35302 10325 Acked Read 1 Sent Done Done false
35303 10326 Acked Write 5 Sent Done Done false
35304 10327 Acked Read 9 Sent Done Done false
35305 10328 Acked Read 1 Sent Done Done false
35306 10329 Acked Read 10 Sent Done Done false
35307 10330 Acked Read 5 Sent Done Done false
35308 10331 Acked Read 9 Sent Done Done false
35309 10332 Acked Read 10 Sent Done Done false
35310 10333 Acked Read 7 Sent Done Done false
35311 10334 Acked Write 8 Sent Done Done false
35312 10335 Acked Write 3 Sent Done Done false
35313 10336 Acked Flush 0 Sent Done Done false
35314 10337 Acked Read 2 Sent Done Done false
35315 10338 Acked Read 1 Sent Done Done false
35316 10339 Acked Write 1 Sent Done Done false
35317 10340 Acked Write 8 Sent Done Done false
35318 10341 Acked Read 4 Sent Done Done false
35319 10342 Acked Write 8 Sent Done Done false
35320 10343 Acked Read 2 Sent Done Done false
35321 10344 Acked Read 8 Sent Done Done false
35322 10345 Acked Write 5 Sent Done Done false
35323 10346 Acked Read 4 Sent Done Done false
35324 10347 Acked Read 7 Sent Done Done false
35325 10348 Acked Read 7 Sent Done Done false
35326 10349 Acked Write 5 Sent Done Done false
35327 10350 Acked Read 1 Sent Done Done false
35328 10351 Acked Flush 0 Sent Done Done false
35329 10352 Acked Read 1 Sent Done Done false
35330 10353 Acked Flush 0 Sent Done Done false
35331 10354 Acked Read 10 Sent Done Done false
35332 10355 Acked Read 8 Sent Done Done false
35333 10356 Acked Read 3 Sent Done Done false
35334 10357 Acked Write 5 Sent Done Done false
35335 10358 Acked Read 9 Sent Done Done false
35336 10359 Acked Flush 0 Sent Done Done false
35337 STATES DS:0 DS:1 DS:2 TOTAL
35338 Sent 302 0 0 302
35339 Done 0 302 302 604
35340 Skipped 0 0 0 0
35341 Error 0 0 0 0
35342 Last Flush: 10057 10359 10359
35343 Downstairs last five completed: 10057 10056 10055 10054 10053
35344 Upstairs last five completed: 10359 10358 10357 10356 10355
35345 CLIENT: Up:0 ds:302 act:2
353462025-01-15T05:26:26.254ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
353472025-01-15T05:26:26.254ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63180) = io task client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
353482025-01-15T05:26:26.255ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
353492025-01-15T05:26:26.256ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
353502025-01-15T05:26:26.256ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
353512025-01-15T05:26:26.256ZINFOcrucible: send last flush ID to this DS: 10057 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
353522025-01-15T05:26:26.256ZINFOcrucible: Replied this last flush ID: 10057 = downstairs client = 0 session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
353532025-01-15T05:26:26.256ZINFOcrucible: [0] client re-new 302 jobs since flush 10057 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
353542025-01-15T05:26:26.256ZINFOcrucible: [0] Replayed 302 jobs since flush: 10057 = downstairs session_id = 91709b1d-df49-4651-b81a-5daddf421e6f
35355 ----------------------------------------------------------------
35356 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35357 Downstairs last five completed: 10359 10358 10357 10356 10355
35358 Upstairs last five completed: 10359 10358 10357 10356 10355
35359 CLIENT: Up:0 ds:0 act:3
35360 Replay: All jobs finished
35361 Test replay has completed
35362 Wrote out file "/tmp/test_replay_verify.log"
35363 CLIENT: Tests done. All submitted work has been ACK'd
35364 ----------------------------------------------------------------
35365 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35366 Downstairs last five completed: 10359 10358 10357 10356 10355
35367 Upstairs last five completed: 10359 10358 10357 10356 10355
35368 CLIENT: Up:0 ds:0 act:3
35369 CLIENT: All jobs finished, all DS active.
353702025-01-15T05:26:26.504ZWARNcrucible: Guest handle has been dropped
35371 Replays:30 time: 7:24
35372 Do final verify
35373 Jan 15 05:26:26.573 INFO accepted connection, remote_addr: 127.0.0.1:40427, local_addr: 127.0.0.1:9998
35374 Jan 15 05:26:26.573 INFO request completed, latency_us: 190, response_code: 200, uri: /regioninfo, method: GET, req_id: c5e9ec1f-327d-4fb9-bcfb-5d01c8be44d8, remote_addr: 127.0.0.1:40427, local_addr: 127.0.0.1:9998
353752025-01-15T05:26:26.574ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
35376 Jan 15 05:26:26.574 INFO request completed, latency_us: 84, response_code: 200, uri: /regioncount, method: GET, req_id: 23fb2173-b416-49a3-b430-af0c831ca541, remote_addr: 127.0.0.1:40427, local_addr: 127.0.0.1:9998
353772025-01-15T05:26:26.575ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
35378 Jan 15 05:26:26.575 INFO request completed, latency_us: 103, response_code: 200, uri: /port/cid/0, method: GET, req_id: b19cb9c3-2f10-47a8-a305-7f910d039cec, remote_addr: 127.0.0.1:40427, local_addr: 127.0.0.1:9998
35379 Jan 15 05:26:26.576 INFO request completed, latency_us: 90, response_code: 200, uri: /port/cid/1, method: GET, req_id: 23e3ebb0-72d0-49fa-9bed-cfc4dae3427e, remote_addr: 127.0.0.1:40427, local_addr: 127.0.0.1:9998
35380 Jan 15 05:26:26.576 INFO request completed, latency_us: 86, response_code: 200, uri: /port/cid/2, method: GET, req_id: 557ef1ac-ace4-4438-b589-512b180b1ac9, remote_addr: 127.0.0.1:40427, local_addr: 127.0.0.1:9998
353812025-01-15T05:26:26.576ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
353822025-01-15T05:26:26.576ZINFOcrucible: Upstairs starts
353832025-01-15T05:26:26.576ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "008b9b7890522309c5657fa001c5655325f06c1f", git_commit_timestamp: "2025-01-15T05:03:03.000000000Z", git_branch: "renovate/http-1.x", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
353842025-01-15T05:26:26.577ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
353852025-01-15T05:26:26.577ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
353862025-01-15T05:26:26.577ZINFOcrucible: Crucible 994fd0e2-5de1-4f66-83b0-966ba3916429 has session id: 322b25dc-9a82-4693-ad83-1f8038efc957 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353872025-01-15T05:26:26.577ZINFOcrucible: Upstairs opts: Upstairs UUID: 994fd0e2-5de1-4f66-83b0-966ba3916429, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353882025-01-15T05:26:26.577ZINFOcrucible: Crucible stats registered with UUID: 994fd0e2-5de1-4f66-83b0-966ba3916429 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353892025-01-15T05:26:26.577ZWARNcrucible: could not find Downstairs address for Nexus session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353902025-01-15T05:26:26.578ZINFOcrucible: Downstairs in volume = 3 task = crutest
353912025-01-15T05:26:26.578ZINFOcrucible: The guest has requested activation with gen:3
353922025-01-15T05:26:26.578ZINFOcrucible: 994fd0e2-5de1-4f66-83b0-966ba3916429 active request set session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353932025-01-15T05:26:26.578ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353942025-01-15T05:26:26.578ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353952025-01-15T05:26:26.578ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353962025-01-15T05:26:26.578ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353972025-01-15T05:26:26.578ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353982025-01-15T05:26:26.578ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
353992025-01-15T05:26:26.578ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354002025-01-15T05:26:26.578ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354012025-01-15T05:26:26.578ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354022025-01-15T05:26:26.578ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54205) = io task client = 2 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354032025-01-15T05:26:26.578ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50998) = io task client = 1 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354042025-01-15T05:26:26.578ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40790) = io task client = 0 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354052025-01-15T05:26:26.579ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354062025-01-15T05:26:26.579ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354072025-01-15T05:26:26.579ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354082025-01-15T05:26:26.580ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354092025-01-15T05:26:26.580ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354102025-01-15T05:26:26.580ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354112025-01-15T05:26:26.580ZINFOcrucible: Waiting for more clients to be ready session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354122025-01-15T05:26:26.580ZINFOcrucible: Waiting for more clients to be ready session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354132025-01-15T05:26:26.580ZINFOcrucible: [0]R flush_numbers[0..12]: [960, 960, 946, 962, 950, 962, 928, 960, 963, 958, 950, 960] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354142025-01-15T05:26:26.580ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354152025-01-15T05:26:26.580ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354162025-01-15T05:26:26.580ZINFOcrucible: [1]R flush_numbers[0..12]: [960, 960, 946, 962, 950, 962, 928, 960, 963, 958, 950, 960] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354172025-01-15T05:26:26.580ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354182025-01-15T05:26:26.580ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354192025-01-15T05:26:26.580ZINFOcrucible: [2]R flush_numbers[0..12]: [960, 960, 946, 962, 950, 962, 928, 960, 963, 958, 950, 960] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354202025-01-15T05:26:26.580ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354212025-01-15T05:26:26.580ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354222025-01-15T05:26:26.580ZINFOcrucible: Max found gen is 3 = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354232025-01-15T05:26:26.581ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354242025-01-15T05:26:26.581ZINFOcrucible: Next flush: 967 = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354252025-01-15T05:26:26.581ZINFOcrucible: All extents match = downstairs session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
35426 {"msg":"No downstairs reconciliation required","v":0,"name":"crucible","level":30,"time":"2025-01-15T05:26:26.581050428Z","hostname":"Wait for a query_work_queue command to finish before sending IO
35427 w-01JHM6ATJ34A0T6R7Q4VPY6N83","pid":1238,"session_id":"322b25dc-9a82-4693-ad83-1f8038efc957"}
354282025-01-15T05:26:26.581ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354292025-01-15T05:26:26.581ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354302025-01-15T05:26:26.581ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354312025-01-15T05:26:26.581ZINFOcrucible: All required reconciliation work is completed session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354322025-01-15T05:26:26.581ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354332025-01-15T05:26:26.581ZINFOcrucible: 994fd0e2-5de1-4f66-83b0-966ba3916429 is now active with session: 322b25dc-9a82-4693-ad83-1f8038efc957 session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354342025-01-15T05:26:26.581ZINFOcrucible: Set Active after no reconciliation session_id = 322b25dc-9a82-4693-ad83-1f8038efc957
354352025-01-15T05:26:26.581ZINFOcrucible: The guest has finished waiting for activation with:3
35436 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
35437 Loading write count information from file "/tmp/test_replay_verify.log"
35438 Read and Verify all blocks (0..5000 range:false)
35439 Wrote out file "/tmp/test_replay_verify.log"
35440 Verify test completed
35441 Wrote out file "/tmp/test_replay_verify.log"
35442 CLIENT: Tests done. All submitted work has been ACK'd
35443 ----------------------------------------------------------------
35444 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
35445 States: Active Active Active
35446 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35447 1000 Acked Read 256 Done Done Done false
35448 1001 Acked Read 256 Done Done Done false
35449 1002 Acked Read 256 Done Done Done false
35450 1003 Acked Read 256 Done Done Done false
35451 1004 Acked Read 256 Done Done Done false
35452 1005 Acked Read 256 Done Done Done false
35453 1006 Acked Read 256 Done Done Done false
35454 1007 Acked Read 256 Done Done Done false
35455 1008 Acked Read 256 Done Done Done false
35456 1009 Acked Read 256 Done Done Done false
35457 1010 Acked Read 256 Done Done Done false
35458 1011 Acked Read 256 Done Done Done false
35459 1012 Acked Read 256 Done Done Done false
35460 1013 Acked Read 256 Done Done Done false
35461 1014 Acked Read 256 Done Done Done false
35462 1015 Acked Read 256 Done Done Done false
35463 1016 Acked Read 256 Done Done Done false
35464 1017 Acked Read 256 Done Done Done false
35465 1018 Acked Read 256 Done Done Done false
35466 1019 Acked Read 136 Done Done Done false
35467 STATES DS:0 DS:1 DS:2 TOTAL
35468 Sent 0 0 0 0
35469 Done 20 20 20 60
35470 Skipped 0 0 0 0
35471 Error 0 0 0 0
35472 Last Flush: 0 0 0
35473 Downstairs last five completed:
35474 Upstairs last five completed: 1019 1018 1017 1016 1015
35475 CLIENT: Up:0 ds:20 act:3
35476 ----------------------------------------------------------------
35477 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
35478 Downstairs last five completed: 1020 1019 1018 1017 1016
35479 Upstairs last five completed: 1020 1019 1018 1017 1016
35480 CLIENT: Up:0 ds:0 act:3
35481 CLIENT: All crucible jobs finished, exiting program
354822025-01-15T05:26:30.987ZWARNcrucible: Guest handle has been dropped
35483 Jan 15 05:26:31.049 INFO accepted connection, remote_addr: 127.0.0.1:33077, local_addr: 127.0.0.1:9998
35484 Main task has work to do, go find it
35485 got dsc Shutdown
35486 Shutdown
35487 Jan 15 05:26:31.050 INFO request completed, latency_us: 158, response_code: 204, uri: /shutdown, method: POST, req_id: 249fb2ea-67d7-488e-951d-050e6d0b5fb1, remote_addr: [8810] Disable keep_running
35488 127Shut it down
35489 .0.0.1:33077[8810] Got stop action so:false kr:false
35490 , local_addr: 127.0.0.1:9998
35491 [8820] Disable keep_running
35492 [8830] Disable keep_running
35493 [8820] Got stop action so:false kr:false
35494 [8830] Got stop action so:false kr:false
35495 [8830] Exited with: ExitStatus(unix_wait_status(9))
35496 [8830][2] reports Exit
35497 [8820] Exited with: ExitStatus(unix_wait_status(9))
35498 [8820][1] reports Exit
35499 [8810] Exited with: ExitStatus(unix_wait_status(9))
35500 [8810][0] reports Exit
35501 January 15, 2025 at 05:26:36 AM UTC Test ends with 0