Max level shown:
1 starting September 22, 2023 at 11:23:36 PM UTC
2 Creating downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.28918764
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.09586245
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.1998776
8 Region set with 3 regions was created
9 Starting downstairs
10 Using existing output directory "/tmp/dsc"
11 start ds: 8810
12 start ds: 8820
13 start ds: 8830
14 start access at:127.0.0.1:9998
15 Starting downstairs at port 8810
16 Make output file at "/tmp/dsc/downstairs-8810.txt"
17 Starting downstairs at port 8830
18 Make output file at "/tmp/dsc/downstairs-8830.txt"
19 [8810][0] reports Starting
20 [8830][2] reports Starting
21 Control access at:127.0.0.1:9998
22 Starting downstairs at port 8820
23 Make output file at "/tmp/dsc/downstairs-8820.txt"
24 [8820][1] reports Starting
25 Sep 22 23:23:37.190 INFO listening, local_addr: 127.0.0.1:9998
26 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1163)
27 [8830][2] reports Running
28 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1164)
29 [8820][1] reports Running
30 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1165)
31 [8810][0] reports Running
32 Running initial fill
332023-09-22T23:23:42.365ZINFOcrucible: Upstairs starts
342023-09-22T23:23:42.366ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "ed48f294784d46ea7d4bb99336918b74358eca46", git_commit_timestamp: "2023-09-22T22:51:18.000000000Z", git_branch: "main", rustc_semver: "1.70.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "90c541806f23a127002de5b4038be731ba1458ca", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
352023-09-22T23:23:42.366ZINFOcrucible: Upstairs <-> Downstairs Message Version: 4
362023-09-22T23:23:42.366ZINFOcrucible: Crucible stats registered with UUID: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0
372023-09-22T23:23:42.366ZINFOcrucible: Crucible 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 has session id: e06fdc4f-5a11-41f6-a2ba-6ac1fd64bee2
38 Crucible runtime is spawned
39 {"The guest has requested activation with gen:1
40 msg":"up_listen starts","v":0,"name":"crucible","level":30,"time":"2023-09-22T23:23:42.367576204Z","hostname":"ip-10-150-1-70.us-west-2.compute.internal","pid":1169,"task":"up_listen"}
412023-09-22T23:23:42.367ZINFOcrucible: Wait for all three downstairs to come online
422023-09-22T23:23:42.367ZINFOcrucible: Flush timeout: 0.5
432023-09-22T23:23:42.368ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 active request set
442023-09-22T23:23:42.368ZINFOcrucible: Set desired generation to :1
452023-09-22T23:23:42.368ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
462023-09-22T23:23:42.368ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
472023-09-22T23:23:42.368ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
482023-09-22T23:23:42.368ZINFOcrucible: [1] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 looper connected looper = 1
492023-09-22T23:23:42.368ZINFOcrucible: [0] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 looper connected looper = 0
502023-09-22T23:23:42.369ZINFOcrucible: [2] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 looper connected looper = 2
512023-09-22T23:23:42.369ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state New
522023-09-22T23:23:42.369ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state New
532023-09-22T23:23:42.370ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state New
542023-09-22T23:23:42.372ZINFOcrucible: [0] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 (232f5bd6-4d0f-4674-a3ba-a01cb050cefb) New New New ds_transition to WaitActive
552023-09-22T23:23:42.372ZINFOcrucible: [0] Transition from New to WaitActive
562023-09-22T23:23:42.372ZINFOcrucible: [2] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 (232f5bd6-4d0f-4674-a3ba-a01cb050cefb) WaitActive New New ds_transition to WaitActive
572023-09-22T23:23:42.372ZINFOcrucible: [2] Transition from New to WaitActive
582023-09-22T23:23:42.372ZINFOcrucible: [0] client is_active_req TRUE, promote! session 232f5bd6-4d0f-4674-a3ba-a01cb050cefb
592023-09-22T23:23:42.372ZINFOcrucible: [2] client is_active_req TRUE, promote! session 232f5bd6-4d0f-4674-a3ba-a01cb050cefb
602023-09-22T23:23:42.372ZINFOcrucible: [1] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 (232f5bd6-4d0f-4674-a3ba-a01cb050cefb) WaitActive New WaitActive ds_transition to WaitActive
612023-09-22T23:23:42.372ZINFOcrucible: [1] Transition from New to WaitActive
622023-09-22T23:23:42.372ZINFOcrucible: [1] client is_active_req TRUE, promote! session 232f5bd6-4d0f-4674-a3ba-a01cb050cefb
632023-09-22T23:23:42.375ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
642023-09-22T23:23:42.375ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
652023-09-22T23:23:42.375ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
662023-09-22T23:23:42.375ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
672023-09-22T23:23:42.375ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
682023-09-22T23:23:42.375ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
692023-09-22T23:23:42.375ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 WaitActive WaitActive WaitActive
702023-09-22T23:23:42.375ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 WaitActive WaitActive WaitActive
712023-09-22T23:23:42.375ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 WaitActive WaitActive WaitActive
722023-09-22T23:23:42.380ZINFOcrucible: [2] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 (232f5bd6-4d0f-4674-a3ba-a01cb050cefb) WaitActive WaitActive WaitActive ds_transition to WaitQuorum
732023-09-22T23:23:42.380ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum
742023-09-22T23:23:42.380ZWARNcrucible: [2] new RM replaced this: None
752023-09-22T23:23:42.380ZINFOcrucible: [2] Starts reconcile loop
762023-09-22T23:23:42.380ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
772023-09-22T23:23:42.380ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 WaitActive WaitActive WaitQuorum
782023-09-22T23:23:42.380ZINFOcrucible: Waiting for 2 more clients to be ready
792023-09-22T23:23:42.380ZINFOcrucible: Set check for repair
802023-09-22T23:23:42.381ZINFOcrucible: [0] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 (232f5bd6-4d0f-4674-a3ba-a01cb050cefb) WaitActive WaitActive WaitQuorum ds_transition to WaitQuorum
812023-09-22T23:23:42.381ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum
822023-09-22T23:23:42.381ZWARNcrucible: [0] new RM replaced this: None
832023-09-22T23:23:42.381ZINFOcrucible: [0] Starts reconcile loop
842023-09-22T23:23:42.381ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
852023-09-22T23:23:42.381ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 WaitQuorum WaitActive WaitQuorum
862023-09-22T23:23:42.381ZINFOcrucible: Waiting for 1 more clients to be ready
872023-09-22T23:23:42.381ZINFOcrucible: Set check for repair
882023-09-22T23:23:42.381ZINFOcrucible: [1] 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 (232f5bd6-4d0f-4674-a3ba-a01cb050cefb) WaitQuorum WaitActive WaitQuorum ds_transition to WaitQuorum
892023-09-22T23:23:42.381ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum
902023-09-22T23:23:42.381ZWARNcrucible: [1] new RM replaced this: None
912023-09-22T23:23:42.381ZINFOcrucible: [1] Starts reconcile loop
922023-09-22T23:23:42.381ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
932023-09-22T23:23:42.381ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 WaitQuorum WaitQuorum WaitQuorum
942023-09-22T23:23:42.381ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
952023-09-22T23:23:42.381ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
962023-09-22T23:23:42.381ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
972023-09-22T23:23:42.381ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
982023-09-22T23:23:42.381ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
992023-09-22T23:23:42.381ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
1002023-09-22T23:23:42.381ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
1012023-09-22T23:23:42.381ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
1022023-09-22T23:23:42.381ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
1032023-09-22T23:23:42.381ZINFOcrucible: Max found gen is 1
1042023-09-22T23:23:42.381ZINFOcrucible: Generation requested: 1 >= found:1
1052023-09-22T23:23:42.381ZINFOcrucible: Next flush: 1
1062023-09-22T23:23:42.382ZINFOcrucible: All extents match
1072023-09-22T23:23:42.382ZINFOcrucible: No downstairs repair required
1082023-09-22T23:23:42.382ZINFOcrucible: No initial repair work was required
1092023-09-22T23:23:42.382ZINFOcrucible: Set Downstairs and Upstairs active
1102023-09-22T23:23:42.382ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 is now active with session: 232f5bd6-4d0f-4674-a3ba-a01cb050cefb
1112023-09-22T23:23:42.382ZINFOcrucible: 62a244e4-eb91-44fd-b327-f9e04ec3ffe0 Set Active after no repair
1122023-09-22T23:23:42.382ZINFOcrucible: Notify all downstairs, region set compare is done.
1132023-09-22T23:23:42.382ZINFOcrucible: Set check for repair
114 {"msg":"[2] received reconcile messageThe guest has finished waiting for activation with:1
115 ","v":0,"Activate successful
116 nameWait for a query_work_queue command to finish before sending IO
117 ":"crucible","level":30,"time":"2023-09-22T23:23:42.382257953Z","hostname":"ip-10-150-1-70.us-west-2.compute.internal","pid":1169}
1182023-09-22T23:23:42.382ZINFOcrucible: [2] All repairs completed, exit
1192023-09-22T23:23:42.382ZINFOcrucible: [2] Starts cmd_loop
1202023-09-22T23:23:42.382ZINFOcrucible: [0] received reconcile message
1212023-09-22T23:23:42.382ZINFOcrucible: [0] All repairs completed, exit
1222023-09-22T23:23:42.382ZINFOcrucible: [0] Starts cmd_loop
1232023-09-22T23:23:42.382ZINFOcrucible: [1] received reconcile message
1242023-09-22T23:23:42.382ZINFOcrucible: [1] All repairs completed, exit
1252023-09-22T23:23:42.382ZINFOcrucible: [1] Starts cmd_loop
126 Region: es:100 ec:50 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
127 Fill test
1282023-09-22T23:23:43.383ZINFOcrucible: Checking if live repair is needed
1292023-09-22T23:23:43.383ZINFOcrucible: No Live Repair required at this time
130 Read and Verify all blocks (0..5000 range:false)
131 Wrote out file "alan"
132 CLIENT: Tests done. All submitted work has been ACK'd
133 ----------------------------------------------------------------
134 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:7
135 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
136 100 Acked 1099 Read 100 Done Done Done false
137 101 Acked 1100 Read 100 Done Done Done false
138 102 Acked 1101 Read 100 Done Done Done false
139 103 Acked 1102 Read 100 Done Done Done false
140 104 Acked 1103 Read 100 Done Done Done false
141 105 Acked 1104 Read 100 Done Done Done false
142 106 Acked 1105 Read 100 Done Done Done false
143 STATES DS:0 DS:1 DS:2 TOTAL
144 New 0 0 0 0
145 Sent 0 0 0 0
146 Done 7 7 7 21
147 Skipped 0 0 0 0
148 Error 0 0 0 0
149 Last Flush: 1098 1098 1098
150 Downstairs last five completed: 1098 1097 1096 1095 1094 1093
151 Upstairs last five completed: 106 105 104 103 102 101
152 CLIENT: Up:0 ds:7 act:3
153 ----------------------------------------------------------------
154 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
155 Downstairs last five completed: 1106 1105 1104 1103 1102 1101
156 Upstairs last five completed: 107 106 105 104 103 102
157 CLIENT: Up:0 ds:0 act:3
158 CLIENT: All crucible jobs finished, exiting program
159 Replay loop starts now September 22, 2023 at 11:23:49 PM UTC
1602023-09-22T23:23:49.735ZINFOcrucible: Upstairs starts
1612023-09-22T23:23:49.735ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "ed48f294784d46ea7d4bb99336918b74358eca46", git_commit_timestamp: "2023-09-22T22:51:18.000000000Z", git_branch: "main", rustc_semver: "1.70.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "90c541806f23a127002de5b4038be731ba1458ca", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1622023-09-22T23:23:49.735ZINFOcrucible: Upstairs <-> Downstairs Message Version: 4
1632023-09-22T23:23:49.735ZINFOcrucible: Crucible stats registered with UUID: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697
1642023-09-22T23:23:49.735ZINFOcrucible: Crucible ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 has session id: 061f65dc-2363-4883-b4ee-c8652002da03
165 Crucible runtime is spawned
166 The guest has requested activation with gen:2
1672023-09-22T23:23:49.736ZINFOcrucible: up_listen starts task = up_listen
1682023-09-22T23:23:49.736ZINFOcrucible: Wait for all three downstairs to come online
1692023-09-22T23:23:49.736ZINFOcrucible: Flush timeout: 0.5
1702023-09-22T23:23:49.736ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
1712023-09-22T23:23:49.736ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
1722023-09-22T23:23:49.736ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
1732023-09-22T23:23:49.736ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
1742023-09-22T23:23:49.737ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
1752023-09-22T23:23:49.737ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
1762023-09-22T23:23:49.737ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 active request set
1772023-09-22T23:23:49.737ZINFOcrucible: Set desired generation to :2
1782023-09-22T23:23:49.737ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state New
1792023-09-22T23:23:49.737ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state New
1802023-09-22T23:23:49.738ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state New
1812023-09-22T23:23:49.738ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) New New New ds_transition to WaitActive
1822023-09-22T23:23:49.738ZINFOcrucible: [1] Transition from New to WaitActive
1832023-09-22T23:23:49.738ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) New WaitActive New ds_transition to WaitActive
1842023-09-22T23:23:49.738ZINFOcrucible: [2] Transition from New to WaitActive
1852023-09-22T23:23:49.738ZINFOcrucible: [1] client is_active_req TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
1862023-09-22T23:23:49.738ZINFOcrucible: [2] client is_active_req TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
1872023-09-22T23:23:49.738ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) New WaitActive WaitActive ds_transition to WaitActive
1882023-09-22T23:23:49.738ZINFOcrucible: [0] Transition from New to WaitActive
1892023-09-22T23:23:49.738ZINFOcrucible: [0] client is_active_req TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
1902023-09-22T23:23:49.739ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
1912023-09-22T23:23:49.739ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
1922023-09-22T23:23:49.739ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
1932023-09-22T23:23:49.739ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
1942023-09-22T23:23:49.739ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 WaitActive WaitActive WaitActive
1952023-09-22T23:23:49.739ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 WaitActive WaitActive WaitActive
1962023-09-22T23:23:49.739ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
1972023-09-22T23:23:49.739ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
1982023-09-22T23:23:49.739ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 WaitActive WaitActive WaitActive
1992023-09-22T23:23:49.741ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) WaitActive WaitActive WaitActive ds_transition to WaitQuorum
2002023-09-22T23:23:49.742ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum
2012023-09-22T23:23:49.742ZWARNcrucible: [1] new RM replaced this: None
2022023-09-22T23:23:49.742ZINFOcrucible: [1] Starts reconcile loop
2032023-09-22T23:23:49.742ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
2042023-09-22T23:23:49.742ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 WaitActive WaitQuorum WaitActive
2052023-09-22T23:23:49.742ZINFOcrucible: Waiting for 2 more clients to be ready
2062023-09-22T23:23:49.742ZINFOcrucible: Set check for repair
2072023-09-22T23:23:49.742ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) WaitActive WaitQuorum WaitActive ds_transition to WaitQuorum
2082023-09-22T23:23:49.742ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum
2092023-09-22T23:23:49.742ZWARNcrucible: [2] new RM replaced this: None
2102023-09-22T23:23:49.742ZINFOcrucible: [2] Starts reconcile loop
2112023-09-22T23:23:49.742ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
2122023-09-22T23:23:49.743ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 WaitActive WaitQuorum WaitQuorum
2132023-09-22T23:23:49.743ZINFOcrucible: Waiting for 1 more clients to be ready
2142023-09-22T23:23:49.743ZINFOcrucible: Set check for repair
2152023-09-22T23:23:49.743ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) WaitActive WaitQuorum WaitQuorum ds_transition to WaitQuorum
2162023-09-22T23:23:49.743ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum
2172023-09-22T23:23:49.743ZWARNcrucible: [0] new RM replaced this: None
2182023-09-22T23:23:49.743ZINFOcrucible: [0] Starts reconcile loop
2192023-09-22T23:23:49.743ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
2202023-09-22T23:23:49.743ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 WaitQuorum WaitQuorum WaitQuorum
2212023-09-22T23:23:49.743ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
2222023-09-22T23:23:49.743ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
2232023-09-22T23:23:49.743ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
2242023-09-22T23:23:49.743ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
2252023-09-22T23:23:49.743ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
2262023-09-22T23:23:49.743ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
2272023-09-22T23:23:49.743ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
228 {"The guest has finished waiting for activation with:2
229 msg":"[2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]","v":Activate successful
230 0Wait for a query_work_queue command to finish before sending IO
231 ,"name":"crucible","level":30,"time":"2023-09-22T23:23:49.743318973Z","hostname":"ip-10-150-1-70.us-west-2.compute.internal","pid":1173}
2322023-09-22T23:23:49.743ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
2332023-09-22T23:23:49.743ZINFOcrucible: Max found gen is 2
2342023-09-22T23:23:49.743ZINFOcrucible: Generation requested: 2 >= found:2
2352023-09-22T23:23:49.743ZINFOcrucible: Next flush: 3
2362023-09-22T23:23:49.743ZINFOcrucible: All extents match
2372023-09-22T23:23:49.743ZINFOcrucible: No downstairs repair required
2382023-09-22T23:23:49.743ZINFOcrucible: No initial repair work was required
2392023-09-22T23:23:49.743ZINFOcrucible: Set Downstairs and Upstairs active
2402023-09-22T23:23:49.743ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 is now active with session: 09e58a14-b4d7-4418-a55e-529861aa483b
2412023-09-22T23:23:49.743ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Set Active after no repair
2422023-09-22T23:23:49.743ZINFOcrucible: Notify all downstairs, region set compare is done.
2432023-09-22T23:23:49.743ZINFOcrucible: Set check for repair
2442023-09-22T23:23:49.743ZINFOcrucible: [0] received reconcile message
2452023-09-22T23:23:49.743ZINFOcrucible: [0] All repairs completed, exit
2462023-09-22T23:23:49.743ZINFOcrucible: [2] received reconcile message
2472023-09-22T23:23:49.743ZINFOcrucible: [2] All repairs completed, exit
2482023-09-22T23:23:49.743ZINFOcrucible: [2] Starts cmd_loop
2492023-09-22T23:23:49.743ZINFOcrucible: [0] Starts cmd_loop
2502023-09-22T23:23:49.743ZINFOcrucible: [1] received reconcile message
2512023-09-22T23:23:49.743ZINFOcrucible: [1] All repairs completed, exit
2522023-09-22T23:23:49.743ZINFOcrucible: [1] Starts cmd_loop
253 Region: es:100 ec:50 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
254 Loading write count information from file "alan"
255 Sep 22 23:23:49.835 INFO accepted connection, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
256 Main task has work to do, go find it
257 got dsc Stop(1)
258 stop 1
259 [8820] Got stop action so:false kr:false
260 Sep 22 23:23:49.840 INFO request completed, latency_us: 1207, response_code: 204, uri: /stop/cid/1, method: POST, req_id: acfff1a8-a5bc-4a2f-b383-ce94477d93f1, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
2612023-09-22T23:23:49.841ZWARNcrucible: [1] downstairs disconnected
2622023-09-22T23:23:49.841ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
2632023-09-22T23:23:49.841ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
2642023-09-22T23:23:49.842ZWARNcrucible: [1] pm_task rx.recv() is None
2652023-09-22T23:23:49.842ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
2662023-09-22T23:23:49.842ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
2672023-09-22T23:23:49.842ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
268 Sep 22 23:23:49.843 INFO request completed, latency_us: 1243, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4a0f1c2b-49bc-4963-b6b4-faac5b3b64c8, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
269 [8820] Exited with: ExitStatus(unix_wait_status(9))
270 [8820][1] reports Exit
2712023-09-22T23:23:50.744ZINFOcrucible: Checking if live repair is needed
2722023-09-22T23:23:50.744ZINFOcrucible: No Live Repair required at this time
273 Sep 22 23:23:53.846 INFO request completed, latency_us: 259, response_code: 200, uri: /state/cid/1, method: GET, req_id: f0e17c8e-7ebd-436f-bf36-52d8ee0b7f01, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
274 001/300 Write block 2476 len 4096 data: 2
275 002/300 Read block 3266 len 12288
276 003/300 Read block 459 len 28672
277 004/300 Write block 112 len 16384 data: 2 2 2 2
278 005/300 Write block 4303 len 24576 data: 2 2 2 2 2 2
279 006/300 Write block 4329 len 36864 data: 2 2 2 2 2 2 2 2 2
280 007/300 Write block 258 len 36864 data: 2 2 2 2 2 2 2 2 2
281 008/300 Flush
282 009/300 Read block 2819 len 36864
283 010/300 Read block 3992 len 36864
284 011/300 Read block 115 len 40960
285 012/300 Read block 3178 len 4096
286 013/300 Write block 1118 len 16384 data: 2 2 2 2
287 014/300 Read block 230 len 24576
288 015/300 Read block 632 len 32768
289 016/300 Read block 871 len 20480
290 017/300 Write block 1855 len 16384 data: 2 2 2 2
291 018/300 Read block 1171 len 12288
292 019/300 Read block 3285 len 16384
293 020/300 Write block 1565 len 32768 data: 2 2 2 2 2 2 2 2
294 021/300 Flush
295 022/300 Read block 176 len 4096
296 023/300 Read block 1222 len 12288
297 024/300 Read block 3615 len 36864
298 025/300 Read block 3622 len 12288
299 026/300 Read block 4984 len 8192
300 027/300 Write block 1182 len 36864 data: 2 2 2 2 2 2 2 2 2
301 028/300 Write block 1337 len 40960 data: 2 2 2 2 2 2 2 2 2 2
302 029/300 Read block 4370 len 8192
303 030/300 Write block 1165 len 8192 data: 2 2
304 031/300 Read block 4560 len 36864
305 032/300 Read block 1357 len 40960
306 033/300 Read block 4987 len 8192
307 034/300 Read block 3917 len 16384
308 035/300 Write block 4256 len 32768 data: 2 2 2 2 2 2 2 2
309 036/300 Flush
310 037/300 Flush
311 038/300 Write block 423 len 24576 data: 2 2 2 2 2 2
312 039/300 Write block 1457 len 40960 data: 2 2 2 2 2 2 2 2 2 2
313 040/300 Read block 2429 len 32768
314 041/300 Write block 3622 len 12288 data: 2 2 2
315 042/300 Write block 4428 len 4096 data: 2
316 043/300 Write block 2182 len 36864 data: 2 2 2 2 2 2 2 2 2
317 044/300 Read block 2631 len 36864
318 045/300 Write block 728 len 16384 data: 2 2 2 2
319 046/300 Flush
320 047/300 Read block 2636 len 32768
321 048/300 Read block 2292 len 24576
322 049/300 Read block 743 len 4096
323 050/300 Flush
324 051/300 Read block 2042 len 4096
325 052/300 Write block 46 len 16384 data: 2 2 2 2
326 053/300 Write block 3039 len 36864 data: 2 2 2 2 2 2 2 2 2
327 054/300 Write block 3861 len 4096 data: 2
328 055/300 Read block 1113 len 28672
329 056/300 Read block 1747 len 20480
330 057/300 Write block 3367 len 16384 data: 2 2 2 2
331 058/300 Write block 4417 len 8192 data: 2 2
332 059/300 Write block 57 len 36864 data: 2 2 2 2 2 2 2 2 2
333 060/300 Write block 770 len 24576 data: 2 2 2 2 2 2
334 061/300 Write block 3831 len 20480 data: 2 2 2 2 2
335 062/300 Read block 726 len 4096
336 063/300 Flush
337 064/300 Read block 1522 len 36864
338 065/300 Read block 344 len 28672
339 066/300 Read block 1881 len 12288
340 067/300 Read block 4336 len 28672
341 068/300 Read block 4989 len 8192
342 069/300 Read block 2708 len 8192
343 070/300 Write block 2838 len 36864 data: 2 2 2 2 2 2 2 2 2
344 071/300 Write block 699 len 16384 data: 2 2 2 2
345 072/300 Write block 595 len 40960 data: 2 2 2 2 2 2 2 2 2 2
346 073/300 Read block 326 len 36864
347 074/300 Flush
348 075/300 Read block 1229 len 12288
349 076/300 Read block 1898 len 8192
350 077/300 Write block 4889 len 12288 data: 2 2 2
351 078/300 Write block 1331 len 40960 data: 2 2 2 2 2 2 3 3 3 3
352 079/300 Read block 1270 len 32768
353 080/300 Write block 2577 len 20480 data: 2 2 2 2 2
354 081/300 Write block 3097 len 32768 data: 2 2 2 2 2 2 2 2
355 082/300 Flush
356 083/300 Read block 872 len 32768
357 084/300 Read block 3867 len 24576
358 085/300 Read block 3877 len 40960
359 086/300 Read block 311 len 8192
360 087/300 Write block 186 len 20480 data: 2 2 2 2 2
361 088/300 Read block 2750 len 36864
362 089/300 Read block 93 len 36864
363 090/300 Read block 4965 len 36864
364 091/300 Read block 4496 len 12288
365 092/300 Read block 2402 len 16384
366 093/300 Read block 194 len 4096
367 094/300 Write block 4307 len 16384 data: 3 3 2 2
368 095/300 Write block 2829 len 8192 data: 2 2
369 096/300 Read block 238 len 40960
370 097/300 Read block 3813 len 36864
371 098/300 Read block 4933 len 8192
372 099/300 Read block 4157 len 24576
373 100/300 Read block 4712 len 20480
374 101/300 Read block 2723 len 28672
375 102/300 Write block 4662 len 4096 data: 2
376 103/300 Read block 2253 len 36864
377 104/300 Write block 1861 len 4096 data: 2
378 105/300 Write block 4645 len 20480 data: 2 2 2 2 2
379 106/300 Write block 4243 len 28672 data: 2 2 2 2 2 2 2
380 107/300 Read block 2340 len 20480
381 108/300 Read block 4322 len 16384
382 109/300 Write block 3143 len 20480 data: 2 2 2 2 2
383 110/300 Read block 1590 len 28672
384 111/300 Write block 958 len 36864 data: 2 2 2 2 2 2 2 2 2
385 112/300 Read block 1271 len 40960
386 113/300 Read block 1277 len 16384
387 114/300 Write block 2704 len 16384 data: 2 2 2 2
388 115/300 Read block 4460 len 24576
389 116/300 Write block 252 len 8192 data: 2 2
390 117/300 Read block 3855 len 12288
391 118/300 Write block 3633 len 4096 data: 2
392 119/300 Write block 1984 len 12288 data: 2 2 2
393 120/300 Write block 3254 len 12288 data: 2 2 2
394 121/300 Read block 4191 len 12288
395 122/300 Write block 763 len 36864 data: 2 2 2 2 2 2 2 3 3
396 123/300 Read block 2983 len 24576
397 124/300 Write block 3739 len 20480 data: 2 2 2 2 2
398 125/300 Write block 3155 len 8192 data: 2 2
399 126/300 Write block 172 len 40960 data: 2 2 2 2 2 2 2 2 2 2
400 127/300 Read block 3496 len 36864
401 128/300 Write block 889 len 36864 data: 2 2 2 2 2 2 2 2 2
402 129/300 Read block 1575 len 12288
403 130/300 Write block 807 len 36864 data: 2 2 2 2 2 2 2 2 2
404 131/300 Read block 1570 len 24576
405 132/300 Write block 929 len 20480 data: 2 2 2 2 2
406 133/300 Read block 1360 len 8192
407 134/300 Read block 3613 len 40960
408 135/300 Read block 2494 len 16384
409 136/300 Read block 1233 len 28672
410 137/300 Read block 2887 len 12288
411 138/300 Read block 3646 len 16384
412 139/300 Write block 4488 len 24576 data: 2 2 2 2 2 2
413 140/300 Write block 4870 len 24576 data: 2 2 2 2 2 2
414 141/300 Read block 1939 len 32768
415 142/300 Read block 3119 len 20480
416 143/300 Read block 840 len 16384
417 144/300 Read block 3989 len 40960
418 145/300 Write block 3790 len 16384 data: 2 2 2 2
419 146/300 Read block 1773 len 12288
420 147/300 Read block 2429 len 32768
421 148/300 Write block 4279 len 28672 data: 2 2 2 2 2 2 2
422 149/300 Read block 2999 len 28672
423 150/300 Read block 2201 len 12288
424 151/300 Write block 270 len 28672 data: 2 2 2 2 2 2 2
425 152/300 Flush
426 153/300 Write block 2838 len 32768 data: 3 3 3 3 3 3 3 3
427 154/300 Read block 1290 len 4096
428 155/300 Read block 4010 len 8192
429 156/300 Read block 3366 len 8192
430 157/300 Read block 2000 len 36864
431 158/300 Write block 3386 len 36864 data: 2 2 2 2 2 2 2 2 2
432 159/300 Read block 4013 len 36864
433 160/300 Write block 3456 len 16384 data: 2 2 2 2
434 161/300 Write block 2238 len 16384 data: 2 2 2 2
435 162/300 Read block 3211 len 20480
436 163/300 Write block 22 len 4096 data: 2
437 164/300 Read block 3543 len 16384
438 165/300 Read block 4193 len 32768
439 166/300 Write block 2249 len 16384 data: 2 2 2 2
440 167/300 Write block 1789 len 36864 data: 2 2 2 2 2 2 2 2 2
441 168/300 Flush
442 169/300 Read block 1217 len 12288
443 170/300 Write block 2368 len 40960 data: 2 2 2 2 2 2 2 2 2 2
444 171/300 Write block 1542 len 8192 data: 2 2
445 172/300 Read block 1636 len 12288
446 173/300 Read block 1699 len 8192
447 174/300 Flush
448 175/300 Read block 923 len 40960
449 176/300 Read block 1697 len 36864
450 177/300 Read block 2695 len 8192
451 178/300 Read block 4073 len 20480
452 179/300 Write block 4192 len 8192 data: 2 2
453 180/300 Read block 2787 len 36864
454 181/300 Read block 4141 len 4096
455 182/300 Read block 1247 len 20480
456 183/300 Read block 3861 len 16384
457 184/300 Read block 501 len 12288
458 185/300 Read block 2357 len 20480
459 186/300 Read block 3362 len 36864
460 187/300 Read block 4195 len 16384
461 188/300 Write block 2983 len 12288 data: 2 2 2
462 189/300 Read block 12 len 32768
463 190/300 Write block 3997 len 32768 data: 2 2 2 2 2 2 2 2
464 191/300 Read block 1018 len 20480
465 192/300 Read block 54 len 20480
466 193/300 Read block 4020 len 12288
467 194/300 Read block 2403 len 8192
468 195/300 Write block 3439 len 32768 data: 2 2 2 2 2 2 2 2
469 196/300 Read block 2258 len 4096
470 197/300 Write block 1921 len 16384 data: 2 2 2 2
471 198/300 Flush
472 199/300 Write block 932 len 24576 data: 3 3 2 2 2 2
473 200/300 Read block 3671 len 40960
474 201/300 Write block 2268 len 40960 data: 2 2 2 2 2 2 2 2 2 2
475 202/300 Read block 1703 len 8192
476 203/300 Write block 3667 len 20480 data: 2 2 2 2 2
477 204/300 Flush
478 205/300 Write block 4627 len 28672 data: 2 2 2 2 2 2 2
479 206/300 Read block 740 len 16384
480 207/300 Read block 3473 len 24576
481 208/300 Write block 976 len 32768 data: 2 2 2 2 2 2 2 2
482 209/300 Read block 1254 len 40960
483 210/300 Write block 1375 len 4096 data: 2
484 211/300 Read block 453 len 12288
485 212/300 Read block 3517 len 28672
486 213/300 Write block 1049 len 8192 data: 2 2
487 214/300 Read block 4960 len 32768
488 215/300 Read block 4957 len 32768
489 216/300 Read block 3302 len 8192
490 217/300 Read block 1702 len 4096
491 218/300 Write block 4926 len 16384 data: 2 2 2 2
492 219/300 Write block 1781 len 4096 data: 2
493 220/300 Write block 3985 len 40960 data: 2 2 2 2 2 2 2 2 2 2
494 221/300 Write block 1961 len 36864 data: 2 2 2 2 2 2 2 2 2
495 222/300 Write block 1540 len 28672 data: 2 2 3 3 2 2 2
496 223/300 Read block 3337 len 36864
497 224/300 Write block 1591 len 28672 data: 2 2 2 2 2 2 2
498 225/300 Read block 3663 len 8192
499 226/300 Read block 3436 len 4096
500 227/300 Read block 1132 len 24576
501 228/300 Read block 3878 len 20480
502 229/300 Read block 1234 len 32768
503 230/300 Read block 3300 len 8192
504 231/300 Read block 3970 len 20480
505 232/300 Read block 4904 len 20480
506 233/300 Write block 282 len 24576 data: 2 2 2 2 2 2
507 234/300 Flush
508 235/300 Read block 1132 len 32768
509 236/300 Read block 1747 len 40960
510 237/300 Write block 2726 len 20480 data: 2 2 2 2 2
511 238/300 Read block 3882 len 12288
512 239/300 Read block 2374 len 20480
513 240/300 Read block 1766 len 40960
514 241/300 Read block 1040 len 8192
515 242/300 Flush
516 243/300 Read block 1680 len 8192
517 244/300 Read block 3242 len 32768
518 245/300 Read block 2981 len 24576
519 246/300 Write block 3590 len 8192 data: 2 2
520 247/300 Write block 2086 len 8192 data: 2 2
521 248/300 Write block 2718 len 24576 data: 2 2 2 2 2 2
522 249/300 Read block 368 len 16384
523 250/300 Read block 4230 len 40960
524 251/300 Read block 3068 len 20480
525 252/300 Read block 2867 len 8192
526 253/300 Write block 2573 len 24576 data: 2 2 2 2 3 3
527 254/300 Flush
528 255/300 Read block 4004 len 32768
529 256/300 Write block 4530 len 16384 data: 2 2 2 2
530 257/300 Read block 4533 len 36864
531 258/300 Write block 4539 len 16384 data: 2 2 2 2
532 259/300 Write block 3337 len 32768 data: 2 2 2 2 2 2 2 2
533 260/300 Write block 2952 len 4096 data: 2
534 261/300 Write block 1859 len 8192 data: 2 2
535 262/300 Write block 1735 len 40960 data: 2 2 2 2 2 2 2 2 2 2
536 263/300 Read block 212 len 36864
537 264/300 Write block 1750 len 20480 data: 2 2 2 2 2
538 265/300 Read block 1529 len 28672
539 266/300 Read block 4661 len 20480
540 267/300 Write block 1528 len 8192 data: 2 2
541 268/300 Write block 2739 len 20480 data: 2 2 2 2 2
542 269/300 Write block 924 len 8192 data: 2 2
543 270/300 Read block 375 len 24576
544 271/300 Write block 1078 len 20480 data: 2 2 2 2 2
545 272/300 Write block 1708 len 12288 data: 2 2 2
546 273/300 Write block 1332 len 8192 data: 3 3
547 274/300 Read block 140 len 16384
548 275/300 Read block 640 len 40960
549 276/300 Read block 1158 len 12288
550 277/300 Read block 1172 len 16384
551 278/300 Read block 2975 len 28672
552 279/300 Read block 619 len 16384
553 280/300 Write block 3300 len 12288 data: 2 2 2
554 281/300 Read block 3262 len 40960
555 282/300 Read block 2755 len 20480
556 283/300 Write block 3715 len 40960 data: 2 2 2 2 2 2 2 2 2 2
557 284/300 Write block 327 len 4096 data: 2
558 285/300 Read block 3004 len 40960
559 286/300 Read block 3413 len 20480
560 287/300 Flush
561 288/300 Read block 4675 len 20480
562 289/300 Write block 757 len 28672 data: 2 2 2 2 2 2 3
563 290/300 Read block 4089 len 12288
564 291/300 Write block 1980 len 8192 data: 2 2
565 292/300 Write block 3251 len 40960 data: 2 2 2 3 3 3 2 2 2 2
566 293/300 Write block 4437 len 20480 data: 2 2 2 2 2
567 294/300 Write block 3907 len 12288 data: 2 2 2
568 295/300 Read block 945 len 32768
569 296/300 Read block 318 len 12288
570 297/300 Write block 4396 len 28672 data: 2 2 2 2 2 2 2
571 298/300 Read block 3857 len 20480
572 299/300 Read block 1966 len 40960
573 300/300 Read block 3829 len 4096
574 Main task has work to do, go find it
575 got dsc Start(1)
576 start 1
577 Sep 22 23:23:54.791 INFO request completed, latency_us: 315, response_code: 204, uri: /start/cid/1, method: POST, req_id: b4c10170-88df-4531-ac38-56bca7f5835c, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
578 Replay: started 1, returned:Ok(())
579 ----------------------------------------------------------------
580 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
581 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
582 1 Acked 1000 Write 1 Done New Done false
583 2 Acked 1001 Read 3 Done New Done false
584 3 Acked 1002 Read 7 Done New Done false
585 4 Acked 1003 Write 4 Done New Done false
586 5 Acked 1004 Write 6 Done New Done false
587 6 Acked 1005 Write 9 Done New Done false
588 7 Acked 1006 Write 9 Done New Done false
589 8 Acked 1007 Flush 0 Done New Done false
590 9 Acked 1008 Read 9 Done New Done false
591 10 Acked 1009 Read 9 Done New Done false
592 11 Acked 1010 Read 10 Done New Done false
593 12 Acked 1011 Read 1 Done New Done false
594 13 Acked 1012 Write 4 Done New Done false
595 14 Acked 1013 Read 6 Done New Done false
596 15 Acked 1014 Read 8 Done New Done false
597 16 Acked 1015 Read 5 Done New Done false
598 17 Acked 1016 Write 4 Done New Done false
599 18 Acked 1017 Read 3 Done New Done false
600 19 Acked 1018 Read 4 Done New Done false
601 20 Acked 1019 Write 8 Done New Done false
602 21 Acked 1020 Flush 0 Done New Done false
603 22 Acked 1021 Read 1 Done New Done false
604 23 Acked 1022 Read 3 Done New Done false
605 24 Acked 1023 Read 9 Done New Done false
606 25 Acked 1024 Read 3 Done New Done false
607 26 Acked 1025 Read 2 Done New Done false
608 27 Acked 1026 Write 9 Done New Done false
609 28 Acked 1027 Write 10 Done New Done false
610 29 Acked 1028 Read 2 Done New Done false
611 30 Acked 1029 Write 2 Done New Done false
612 31 Acked 1030 Read 9 Done New Done false
613 32 Acked 1031 Read 10 Done New Done false
614 33 Acked 1032 Read 2 Done New Done false
615 34 Acked 1033 Read 4 Done New Done false
616 35 Acked 1034 Write 8 Done New Done false
617 36 Acked 1035 Flush 0 Done New Done false
618 37 Acked 1036 Flush 0 Done New Done false
619 38 Acked 1037 Write 6 Done New Done false
620 39 Acked 1038 Write 10 Done New Done false
621 40 Acked 1039 Read 8 Done New Done false
622 41 Acked 1040 Write 3 Done New Done false
623 42 Acked 1041 Write 1 Done New Done false
624 43 Acked 1042 Write 9 Done New Done false
625 44 Acked 1043 Read 9 Done New Done false
626 45 Acked 1044 Write 4 Done New Done false
627 46 Acked 1045 Flush 0 Done New Done false
628 47 Acked 1046 Read 8 Done New Done false
629 48 Acked 1047 Read 6 Done New Done false
630 49 Acked 1048 Read 1 Done New Done false
631 50 Acked 1049 Flush 0 Done New Done false
632 51 Acked 1050 Read 1 Done New Done false
633 52 Acked 1051 Write 4 Done New Done false
634 53 Acked 1052 Write 9 Done New Done false
635 54 Acked 1053 Write 1 Done New Done false
636 55 Acked 1054 Read 7 Done New Done false
637 56 Acked 1055 Read 5 Done New Done false
638 57 Acked 1056 Write 4 Done New Done false
639 58 Acked 1057 Write 2 Done New Done false
640 59 Acked 1058 Write 9 Done New Done false
641 60 Acked 1059 Write 6 Done New Done false
642 61 Acked 1060 Write 5 Done New Done false
643 62 Acked 1061 Read 1 Done New Done false
644 63 Acked 1062 Flush 0 Done New Done false
645 64 Acked 1063 Read 9 Done New Done false
646 65 Acked 1064 Read 7 Done New Done false
647 66 Acked 1065 Read 3 Done New Done false
648 67 Acked 1066 Read 7 Done New Done false
649 68 Acked 1067 Read 2 Done New Done false
650 69 Acked 1068 Read 2 Done New Done false
651 70 Acked 1069 Write 9 Done New Done false
652 71 Acked 1070 Write 4 Done New Done false
653 72 Acked 1071 Write 10 Done New Done false
654 73 Acked 1072 Read 9 Done New Done false
655 74 Acked 1073 Flush 0 Done New Done false
656 75 Acked 1074 Read 3 Done New Done false
657 76 Acked 1075 Read 2 Done New Done false
658 77 Acked 1076 Write 3 Done New Done false
659 78 Acked 1077 Write 10 Done New Done false
660 79 Acked 1078 Read 8 Done New Done false
661 80 Acked 1079 Write 5 Done New Done false
662 81 Acked 1080 Write 8 Done New Done false
663 82 Acked 1081 Flush 0 Done New Done false
664 83 Acked 1082 Read 8 Done New Done false
665 84 Acked 1083 Read 6 Done New Done false
666 85 Acked 1084 Read 10 Done New Done false
667 86 Acked 1085 Read 2 Done New Done false
668 87 Acked 1086 Write 5 Done New Done false
669 88 Acked 1087 Read 9 Done New Done false
670 89 Acked 1088 Read 9 Done New Done false
671 90 Acked 1089 Read 9 Done New Done false
672 91 Acked 1090 Read 3 Done New Done false
673 92 Acked 1091 Read 4 Done New Done false
674 93 Acked 1092 Read 1 Done New Done false
675 94 Acked 1093 Write 4 Done New Done false
676 95 Acked 1094 Write 2 Done New Done false
677 96 Acked 1095 Read 10 Done New Done false
678 97 Acked 1096 Read 9 Done New Done false
679 98 Acked 1097 Read 2 Done New Done false
680 99 Acked 1098 Read 6 Done New Done false
681 100 Acked 1099 Read 5 Done New Done false
682 101 Acked 1100 Read 7 Done New Done false
683 102 Acked 1101 Write 1 Done New Done false
684 103 Acked 1102 Read 9 Done New Done false
685 104 Acked 1103 Write 1 Done New Done false
686 105 Acked 1104 Write 5 Done New Done false
687 106 Acked 1105 Write 7 Done New Done false
688 107 Acked 1106 Read 5 Done New Done false
689 108 Acked 1107 Read 4 Done New Done false
690 109 Acked 1108 Write 5 Done New Done false
691 110 Acked 1109 Read 7 Done New Done false
692 111 Acked 1110 Write 9 Done New Done false
693 112 Acked 1111 Read 10 Done New Done false
694 113 Acked 1112 Read 4 Done New Done false
695 114 Acked 1113 Write 4 Done New Done false
696 115 Acked 1114 Read 6 Done New Done false
697 116 Acked 1115 Write 2 Done New Done false
698 117 Acked 1116 Read 3 Done New Done false
699 118 Acked 1117 Write 1 Done New Done false
700 119 Acked 1118 Write 3 Done New Done false
701 120 Acked 1119 Write 3 Done New Done false
702 121 Acked 1120 Read 3 Done New Done false
703 122 Acked 1121 Write 9 Done New Done false
704 123 Acked 1122 Read 6 Done New Done false
705 124 Acked 1123 Write 5 Done New Done false
706 125 Acked 1124 Write 2 Done New Done false
707 126 Acked 1125 Write 10 Done New Done false
708 127 Acked 1126 Read 9 Done New Done false
709 128 Acked 1127 Write 9 Done New Done false
710 129 Acked 1128 Read 3 Done New Done false
711 130 Acked 1129 Write 9 Done New Done false
712 131 Acked 1130 Read 6 Done New Done false
713 132 Acked 1131 Flush 0 Done New Done false
714 133 Acked 1132 Write 5 Done New Done false
715 134 Acked 1133 Read 2 Done New Done false
716 135 Acked 1134 Read 10 Done New Done false
717 136 Acked 1135 Read 4 Done New Done false
718 137 Acked 1136 Read 7 Done New Done false
719 138 Acked 1137 Read 3 Done New Done false
720 139 Acked 1138 Read 4 Done New Done false
721 140 Acked 1139 Write 6 Done New Done false
722 141 Acked 1140 Write 6 Done New Done false
723 142 Acked 1141 Read 8 Done New Done false
724 143 Acked 1142 Read 5 Done New Done false
725 144 Acked 1143 Read 4 Done New Done false
726 145 Acked 1144 Read 10 Done New Done false
727 146 Acked 1145 Write 4 Done New Done false
728 147 Acked 1146 Read 3 Done New Done false
729 148 Acked 1147 Read 8 Done New Done false
730 149 Acked 1148 Write 7 Done New Done false
731 150 Acked 1149 Read 7 Done New Done false
732 151 Acked 1150 Read 3 Done New Done false
733 152 Acked 1151 Write 7 Done New Done false
734 153 Acked 1152 Flush 0 Done New Done false
735 154 Acked 1153 Write 8 Done New Done false
736 155 Acked 1154 Read 1 Done New Done false
737 156 Acked 1155 Read 2 Done New Done false
738 157 Acked 1156 Read 2 Done New Done false
739 158 Acked 1157 Read 9 Done New Done false
740 159 Acked 1158 Write 9 Done New Done false
741 160 Acked 1159 Read 9 Done New Done false
742 161 Acked 1160 Write 4 Done New Done false
743 162 Acked 1161 Write 4 Done New Done false
744 163 Acked 1162 Read 5 Done New Done false
745 164 Acked 1163 Write 1 Done New Done false
746 165 Acked 1164 Read 4 Done New Done false
747 166 Acked 1165 Read 8 Done New Done false
748 167 Acked 1166 Write 4 Done New Done false
749 168 Acked 1167 Write 9 Done New Done false
750 169 Acked 1168 Flush 0 Done New Done false
751 170 Acked 1169 Read 3 Done New Done false
752 171 Acked 1170 Write 10 Done New Done false
753 172 Acked 1171 Write 2 Done New Done false
754 173 Acked 1172 Read 3 Done New Done false
755 174 Acked 1173 Read 2 Done New Done false
756 175 Acked 1174 Flush 0 Done New Done false
757 176 Acked 1175 Read 10 Done New Done false
758 177 Acked 1176 Read 9 Done New Done false
759 178 Acked 1177 Read 2 Done New Done false
760 179 Acked 1178 Read 5 Done New Done false
761 180 Acked 1179 Write 2 Done New Done false
762 181 Acked 1180 Read 9 Done New Done false
763 182 Acked 1181 Read 1 Done New Done false
764 183 Acked 1182 Read 5 Done New Done false
765 184 Acked 1183 Read 4 Done New Done false
766 185 Acked 1184 Read 3 Done New Done false
767 186 Acked 1185 Read 5 Done New Done false
768 187 Acked 1186 Read 9 Done New Done false
769 188 Acked 1187 Read 4 Done New Done false
770 189 Acked 1188 Write 3 Done New Done false
771 190 Acked 1189 Read 8 Done New Done false
772 191 Acked 1190 Write 8 Done New Done false
773 192 Acked 1191 Read 5 Done New Done false
774 193 Acked 1192 Read 5 Done New Done false
775 194 Acked 1193 Read 3 Done New Done false
776 195 Acked 1194 Read 2 Done New Done false
777 196 Acked 1195 Write 8 Done New Done false
778 197 Acked 1196 Read 1 Done New Done false
779 198 Acked 1197 Write 4 Done New Done false
780 199 Acked 1198 Flush 0 Done New Done false
781 200 Acked 1199 Write 6 Done New Done false
782 201 Acked 1200 Read 10 Done New Done false
783 202 Acked 1201 Write 10 Done New Done false
784 203 Acked 1202 Read 2 Done New Done false
785 204 Acked 1203 Write 5 Done New Done false
786 205 Acked 1204 Flush 0 Done New Done false
787 206 Acked 1205 Write 7 Done New Done false
788 207 Acked 1206 Read 4 Done New Done false
789 208 Acked 1207 Read 6 Done New Done false
790 209 Acked 1208 Write 8 Done New Done false
791 210 Acked 1209 Read 10 Done New Done false
792 211 Acked 1210 Write 1 Done New Done false
793 212 Acked 1211 Read 3 Done New Done false
794 213 Acked 1212 Read 7 Done New Done false
795 214 Acked 1213 Write 2 Done New Done false
796 215 Acked 1214 Read 8 Done New Done false
797 216 Acked 1215 Read 8 Done New Done false
798 217 Acked 1216 Read 2 Done New Done false
799 218 Acked 1217 Read 1 Done New Done false
800 219 Acked 1218 Write 4 Done New Done false
801 220 Acked 1219 Write 1 Done New Done false
802 221 Acked 1220 Write 10 Done New Done false
803 222 Acked 1221 Write 9 Done New Done false
804 223 Acked 1222 Write 7 Done New Done false
805 224 Acked 1223 Read 9 Done New Done false
806 225 Acked 1224 Write 7 Done New Done false
807 226 Acked 1225 Read 2 Done New Done false
808 227 Acked 1226 Read 1 Done New Done false
809 228 Acked 1227 Read 6 Done New Done false
810 229 Acked 1228 Read 5 Done New Done false
811 230 Acked 1229 Read 8 Done New Done false
812 231 Acked 1230 Read 2 Done New Done false
813 232 Acked 1231 Read 5 Done New Done false
814 233 Acked 1232 Read 5 Done New Done false
815 234 Acked 1233 Write 6 Done New Done false
816 235 Acked 1234 Flush 0 Done New Done false
817 236 Acked 1235 Read 8 Done New Done false
818 237 Acked 1236 Read 10 Done New Done false
819 238 Acked 1237 Write 5 Done New Done false
820 239 Acked 1238 Read 3 Done New Done false
821 240 Acked 1239 Read 5 Done New Done false
822 241 Acked 1240 Read 10 Done New Done false
823 242 Acked 1241 Read 2 Done New Done false
824 243 Acked 1242 Flush 0 Done New Done false
825 244 Acked 1243 Read 2 Done New Done false
826 245 Acked 1244 Read 8 Done New Done false
827 246 Acked 1245 Read 6 Done New Done false
828 247 Acked 1246 Write 2 Done New Done false
829 248 Acked 1247 Write 2 Done New Done false
830 249 Acked 1248 Write 6 Done New Done false
831 250 Acked 1249 Read 4 Done New Done false
832 251 Acked 1250 Read 10 Done New Done false
833 252 Acked 1251 Read 5 Done New Done false
834 253 Acked 1252 Read 2 Done New Done false
835 254 Acked 1253 Write 6 Done New Done false
836 255 Acked 1254 Flush 0 Done New Done false
837 256 Acked 1255 Read 8 Done New Done false
838 257 Acked 1256 Write 4 Done New Done false
839 258 Acked 1257 Read 9 Done New Done false
840 259 Acked 1258 Write 4 Done New Done false
841 260 Acked 1259 Write 8 Done New Done false
842 261 Acked 1260 Write 1 Done New Done false
843 262 Acked 1261 Write 2 Done New Done false
844 263 Acked 1262 Write 10 Done New Done false
845 264 Acked 1263 Read 9 Done New Done false
846 265 Acked 1264 Write 5 Done New Done false
847 266 Acked 1265 Read 7 Done New Done false
848 267 Acked 1266 Read 5 Done New Done false
849 268 Acked 1267 Write 2 Done New Done false
850 269 Acked 1268 Write 5 Done New Done false
851 270 Acked 1269 Write 2 Done New Done false
852 271 Acked 1270 Read 6 Done New Done false
853 272 Acked 1271 Write 5 Done New Done false
854 273 Acked 1272 Write 3 Done New Done false
855 274 Acked 1273 Write 2 Done New Done false
856 275 Acked 1274 Read 4 Done New Done false
857 276 Acked 1275 Read 10 Done New Done false
858 277 Acked 1276 Read 3 Done New Done false
859 278 Acked 1277 Read 4 Done New Done false
860 279 Acked 1278 Read 7 Done New Done false
861 280 Acked 1279 Read 4 Done New Done false
862 281 Acked 1280 Write 3 Done New Done false
863 282 Acked 1281 Read 10 Done New Done false
864 283 Acked 1282 Read 5 Done New Done false
865 284 Acked 1283 Write 10 Done New Done false
866 285 Acked 1284 Write 1 Done New Done false
867 286 Acked 1285 Read 10 Done New Done false
868 287 Acked 1286 Read 5 Done New Done false
869 288 Acked 1287 Flush 0 Done New Done false
870 289 Acked 1288 Read 5 Done New Done false
871 290 Acked 1289 Write 7 Done New Done false
872 291 Acked 1290 Read 3 Done New Done false
873 292 Acked 1291 Write 2 Done New Done false
874 293 Acked 1292 Write 10 Done New Done false
875 294 Acked 1293 Write 5 Done New Done false
876 295 Acked 1294 Write 3 Done New Done false
877 296 Acked 1295 Read 8 Done New Done false
878 297 Acked 1296 Read 3 Done New Done false
879 298 Acked 1297 Write 7 Done New Done false
880 299 Acked 1298 Read 5 Done New Done false
881 300 Acked 1299 Read 10 Done New Done false
882 301 Acked 1300 Read 1 Done New Done false
883 STATES DS:0 DS:1 DS:2 TOTAL
884 New 0 301 0 301
885 Sent 0 0 0 0
886 Done 301 0 301 602
887 Skipped 0 0 0 0
888 Error 0 0 0 0
889 Last Flush: 1287 0 1287
890 Downstairs last five completed:
891 Upstairs last five completed: 301 300 299 298 297 296
892 CLIENT: Up:0 ds:301 act:2
893 [8820] got start action
894 [8820] I am going to restart
895 Starting downstairs at port 8820
896 Make output file at "/tmp/dsc/downstairs-8820.txt"
897 [8820][1] reports Starting
898 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1174)
899 [8820][1] reports Running
9002023-09-22T23:23:56.850ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
9012023-09-22T23:23:56.850ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
9022023-09-22T23:23:56.850ZINFOcrucible: [1] client re-new 302 jobs since flush 0 = downstairs
9032023-09-22T23:23:56.850ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
9042023-09-22T23:23:56.851ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
9052023-09-22T23:23:56.851ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
9062023-09-22T23:23:56.851ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
9072023-09-22T23:23:56.851ZINFOcrucible: [1] send last flush ID to this DS: 0
9082023-09-22T23:23:56.851ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
9092023-09-22T23:23:56.852ZINFOcrucible: [1] Replied this last flush ID: 0
9102023-09-22T23:23:56.852ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
9112023-09-22T23:23:56.852ZINFOcrucible: [1] Transition from Offline to Replay
9122023-09-22T23:23:56.852ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
9132023-09-22T23:23:56.852ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
9142023-09-22T23:23:56.852ZINFOcrucible: [1] Transition from Replay to Active
9152023-09-22T23:23:56.852ZINFOcrucible: [1] Starts cmd_loop
9162023-09-22T23:23:56.852ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
9172023-09-22T23:23:56.852ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
9182023-09-22T23:23:56.852ZINFOcrucible: Set check for repair
9192023-09-22T23:23:57.853ZWARNcrucible: [1] flow control end
9202023-09-22T23:23:57.854ZINFOcrucible: Checking if live repair is needed
9212023-09-22T23:23:57.854ZINFOcrucible: No Live Repair required at this time
922 ----------------------------------------------------------------
923 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
924 Downstairs last five completed: 1301 1300 1299 1298 1297 1296
925 Upstairs last five completed: 302 301 300 299 298 297
926 CLIENT: Up:0 ds:0 act:3
927 Replay: All jobs finished, all DS active.
928 Main task has work to do, go find it
929 got dsc Stop(0)
930 stop 0
931 Sep 22 23:23:58.795 INFO request completed, latency_us: 494, response_code: 204, uri: /stop/cid/0, method: POST, req_id: fb76450a-555b-46f0-91ce-c9580a921e68[8810] Got stop action so:false kr:false
932 , remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
933 Sep 22 23:23:58.797 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/0, method: GET, req_id: d3b104a6-6baa-4975-8306-88ddcec285af, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
9342023-09-22T23:23:58.797ZWARNcrucible: [0] downstairs disconnected
9352023-09-22T23:23:58.797ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
9362023-09-22T23:23:58.797ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
9372023-09-22T23:23:58.797ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
9382023-09-22T23:23:58.797ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
9392023-09-22T23:23:58.797ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
9402023-09-22T23:23:58.797ZWARNcrucible: [0] pm_task rx.recv() is None
941 [8810] Exited with: ExitStatus(unix_wait_status(9))
942 [8810][0] reports Exit
943 Sep 22 23:24:02.800 INFO request completed, latency_us: 266, response_code: 200, uri: /state/cid/0, method: GET, req_id: 7fb02ea9-2dd6-404d-afdc-e22fda312134, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
944 001/300 Write block 4266 len 12288 data: 2 2 2
945 002/300 Write block 3286 len 12288 data: 2 2 2
946 003/300 Flush
947 004/300 Read block 561 len 36864
948 005/300 Read block 596 len 8192
949 006/300 Read block 3289 len 20480
950 007/300 Read block 2251 len 32768
951 008/300 Write block 1086 len 24576 data: 2 2 2 2 2 2
952 009/300 Write block 3398 len 40960 data: 2 2 2 2 2 2 2 2 2 2
953 010/300 Write block 4129 len 20480 data: 2 2 2 2 2
954 011/300 Read block 3350 len 32768
955 012/300 Read block 976 len 32768
956 013/300 Write block 3982 len 28672 data: 2 2 2 3 3 3 3
957 014/300 Write block 3305 len 12288 data: 2 2 2
958 015/300 Read block 1841 len 12288
959 016/300 Write block 1430 len 32768 data: 2 2 2 2 2 2 2 2
960 017/300 Read block 2224 len 40960
961 018/300 Read block 1759 len 20480
962 019/300 Flush
963 020/300 Write block 1495 len 28672 data: 2 2 2 2 2 2 2
964 021/300 Read block 4921 len 16384
965 022/300 Write block 4996 len 16384 data: 2 2 2 2
966 023/300 Write block 726 len 24576 data: 2 2 3 3 3 3
967 024/300 Write block 790 len 20480 data: 2 2 2 2 2
968 025/300 Flush
969 026/300 Write block 109 len 40960 data: 2 2 2 3 3 3 3 2 2 2
970 027/300 Read block 3451 len 16384
971 028/300 Read block 768 len 4096
972 029/300 Write block 2343 len 40960 data: 2 2 2 2 2 2 2 2 2 2
973 030/300 Flush
974 031/300 Read block 536 len 16384
975 032/300 Write block 463 len 12288 data: 2 2 2
976 033/300 Flush
977 034/300 Write block 3898 len 24576 data: 2 2 2 2 2 2
978 035/300 Write block 4911 len 36864 data: 2 2 2 2 2 2 2 2 2
979 036/300 Flush
980 037/300 Write block 1078 len 4096 data: 3
981 038/300 Read block 3764 len 28672
982 039/300 Read block 1316 len 28672
983 040/300 Read block 845 len 4096
984 041/300 Write block 2381 len 24576 data: 2 2 2 2 2 2
985 042/300 Read block 2775 len 36864
986 043/300 Flush
987 044/300 Write block 4602 len 32768 data: 2 2 2 2 2 2 2 2
988 045/300 Write block 2110 len 24576 data: 2 2 2 2 2 2
989 046/300 Read block 2537 len 24576
990 047/300 Write block 4411 len 40960 data: 2 2 2 2 2 2 3 3 2 2
991 048/300 Read block 1360 len 16384
992 049/300 Read block 4099 len 24576
993 050/300 Read block 4510 len 4096
994 051/300 Write block 4419 len 36864 data: 3 3 2 2 2 2 2 2 2
995 052/300 Read block 4885 len 16384
996 053/300 Read block 1948 len 20480
997 054/300 Write block 2312 len 36864 data: 2 2 2 2 2 2 2 2 2
998 055/300 Write block 3743 len 12288 data: 3 2 2
999 056/300 Write block 380 len 20480 data: 2 2 2 2 2
1000 057/300 Read block 1403 len 24576
1001 058/300 Write block 2022 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1002 059/300 Write block 2321 len 8192 data: 2 2
1003 060/300 Read block 4322 len 24576
1004 061/300 Write block 2842 len 16384 data: 4 4 4 4
1005 062/300 Read block 373 len 12288
1006 063/300 Read block 4726 len 24576
1007 064/300 Write block 4294 len 8192 data: 2 2
1008 065/300 Write block 4983 len 28672 data: 2 2 2 2 2 2 2
1009 066/300 Read block 4125 len 4096
1010 067/300 Write block 2596 len 16384 data: 2 2 2 2
1011 068/300 Read block 1427 len 8192
1012 069/300 Write block 1903 len 36864 data: 2 2 2 2 2 2 2 2 2
1013 070/300 Read block 4702 len 12288
1014 071/300 Read block 4256 len 24576
1015 072/300 Write block 2215 len 36864 data: 2 2 2 2 2 2 2 2 2
1016 073/300 Read block 3722 len 16384
1017 074/300 Write block 723 len 28672 data: 2 2 2 3 3 4 4
1018 075/300 Read block 2424 len 16384
1019 076/300 Write block 19 len 24576 data: 2 2 2 3 2 2
1020 077/300 Flush
1021 078/300 Read block 3506 len 12288
1022 079/300 Read block 4660 len 16384
1023 080/300 Write block 2076 len 4096 data: 2
1024 081/300 Read block 2539 len 12288
1025 082/300 Write block 833 len 32768 data: 2 2 2 2 2 2 2 2
1026 083/300 Write block 50 len 28672 data: 2 2 2 2 2 2 2
1027 084/300 Read block 3371 len 8192
1028 085/300 Read block 1097 len 28672
1029 086/300 Write block 1887 len 20480 data: 2 2 2 2 2
1030 087/300 Write block 34 len 12288 data: 2 2 2
1031 088/300 Read block 3521 len 36864
1032 089/300 Read block 3893 len 40960
1033 090/300 Write block 4954 len 12288 data: 2 2 2
1034 091/300 Read block 3351 len 20480
1035 092/300 Read block 4132 len 16384
1036 093/300 Flush
1037 094/300 Read block 4231 len 12288
1038 095/300 Write block 4747 len 32768 data: 2 2 2 2 2 2 2 2
1039 096/300 Read block 1031 len 24576
1040 097/300 Read block 4857 len 20480
1041 098/300 Read block 1920 len 28672
1042 099/300 Read block 1667 len 8192
1043 100/300 Read block 4036 len 20480
1044 101/300 Write block 2917 len 4096 data: 2
1045 102/300 Write block 4976 len 36864 data: 2 2 2 2 2 2 2 3 3
1046 103/300 Write block 1549 len 36864 data: 2 2 2 2 2 2 2 2 2
1047 104/300 Read block 1376 len 20480
1048 105/300 Read block 1822 len 36864
1049 106/300 Write block 2696 len 16384 data: 2 2 2 2
1050 107/300 Write block 1044 len 8192 data: 2 2
1051 108/300 Read block 4254 len 12288
1052 109/300 Read block 596 len 24576
1053 110/300 Read block 1652 len 16384
1054 111/300 Read block 4628 len 20480
1055 112/300 Read block 2210 len 32768
1056 113/300 Flush
1057 114/300 Write block 3366 len 8192 data: 2 3
1058 115/300 Write block 4903 len 24576 data: 2 2 2 2 2 2
1059 116/300 Read block 2427 len 4096
1060 117/300 Read block 3150 len 36864
1061 118/300 Write block 2390 len 28672 data: 2 2 2 2 2 2 2
1062 119/300 Write block 48 len 36864 data: 3 3 3 3 3 3 3 3 3
1063 120/300 Read block 1659 len 8192
1064 121/300 Read block 3725 len 40960
1065 122/300 Read block 4720 len 16384
1066 123/300 Read block 4040 len 24576
1067 124/300 Read block 4411 len 12288
1068 125/300 Read block 1566 len 20480
1069 126/300 Read block 2400 len 24576
1070 127/300 Write block 4686 len 4096 data: 2
1071 128/300 Read block 3846 len 32768
1072 129/300 Read block 2326 len 20480
1073 130/300 Flush
1074 131/300 Write block 3043 len 24576 data: 3 3 3 3 3 2
1075 132/300 Write block 2549 len 16384 data: 2 2 2 2
1076 133/300 Flush
1077 134/300 Flush
1078 135/300 Read block 960 len 20480
1079 136/300 Read block 4731 len 28672
1080 137/300 Read block 2933 len 4096
1081 138/300 Write block 4626 len 16384 data: 2 3 3 3
1082 139/300 Write block 3549 len 20480 data: 2 2 2 2 2
1083 140/300 Write block 756 len 40960 data: 2 3 3 3 3 3 3 4 3 3
1084 141/300 Read block 1584 len 40960
1085 142/300 Read block 3101 len 8192
1086 143/300 Flush
1087 144/300 Read block 3005 len 12288
1088 145/300 Read block 2220 len 28672
1089 146/300 Read block 2660 len 32768
1090 147/300 Flush
1091 148/300 Flush
1092 149/300 Read block 1915 len 28672
1093 150/300 Write block 575 len 12288 data: 2 2 2
1094 151/300 Read block 2073 len 40960
1095 152/300 Flush
1096 153/300 Write block 2573 len 36864 data: 3 3 3 3 4 4 3 3 3
1097 154/300 Write block 1406 len 28672 data: 2 2 2 2 2 2 2
1098 155/300 Write block 2583 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1099 156/300 Flush
1100 157/300 Write block 1092 len 16384 data: 2 2 2 2
1101 158/300 Read block 4597 len 4096
1102 159/300 Read block 4069 len 24576
1103 160/300 Read block 3059 len 36864
1104 161/300 Write block 4638 len 16384 data: 2 2 2 2
1105 162/300 Read block 2169 len 20480
1106 163/300 Write block 999 len 20480 data: 2 2 2 2 2
1107 164/300 Read block 675 len 12288
1108 165/300 Write block 3606 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1109 166/300 Write block 808 len 40960 data: 3 3 3 3 3 3 3 3 2 2
1110 167/300 Flush
1111 168/300 Write block 1955 len 12288 data: 2 2 2
1112 169/300 Read block 3381 len 28672
1113 170/300 Read block 1688 len 24576
1114 171/300 Read block 4679 len 32768
1115 172/300 Write block 4144 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1116 173/300 Read block 2331 len 20480
1117 174/300 Write block 4285 len 36864 data: 3 2 2 2 2 2 2 2 2
1118 175/300 Read block 3854 len 36864
1119 176/300 Read block 398 len 32768
1120 177/300 Read block 4854 len 12288
1121 178/300 Flush
1122 179/300 Write block 4927 len 40960 data: 3 3 3 2 2 2 2 2 2 2
1123 180/300 Flush
1124 181/300 Write block 4940 len 12288 data: 2 2 2
1125 182/300 Read block 2920 len 12288
1126 183/300 Read block 259 len 32768
1127 184/300 Write block 4213 len 20480 data: 2 2 2 2 2
1128 185/300 Read block 4862 len 28672
1129 186/300 Write block 354 len 24576 data: 2 2 2 2 2 2
1130 187/300 Write block 2693 len 20480 data: 2 2 2 3 3
1131 188/300 Write block 4090 len 36864 data: 2 2 2 2 2 2 2 2 2
1132 189/300 Read block 4083 len 36864
1133 190/300 Flush
1134 191/300 Read block 2518 len 20480
1135 192/300 Read block 3928 len 32768
1136 193/300 Read block 973 len 20480
1137 194/300 Write block 820 len 8192 data: 2 2
1138 195/300 Read block 1783 len 24576
1139 196/300 Write block 4741 len 28672 data: 2 2 2 2 2 2 3
1140 197/300 Read block 1782 len 40960
1141 198/300 Write block 4433 len 20480 data: 2 2 2 2 3
1142 199/300 Read block 2172 len 8192
1143 200/300 Write block 143 len 36864 data: 2 2 2 2 2 2 2 2 2
1144 201/300 Read block 374 len 36864
1145 202/300 Read block 1181 len 40960
1146 203/300 Read block 888 len 32768
1147 204/300 Read block 2546 len 8192
1148 205/300 Read block 3895 len 12288
1149 206/300 Read block 1952 len 24576
1150 207/300 Write block 1824 len 12288 data: 2 2 2
1151 208/300 Write block 1474 len 24576 data: 2 2 2 2 2 2
1152 209/300 Read block 2037 len 4096
1153 210/300 Read block 4412 len 40960
1154 211/300 Write block 4402 len 36864 data: 3 2 2 2 2 2 2 2 2
1155 212/300 Read block 1107 len 40960
1156 213/300 Write block 2022 len 4096 data: 3
1157 214/300 Read block 592 len 32768
1158 215/300 Read block 914 len 4096
1159 216/300 Read block 3974 len 28672
1160 217/300 Write block 2323 len 24576 data: 2 2 2 2 2 2
1161 218/300 Read block 922 len 28672
1162 219/300 Write block 4876 len 24576 data: 2 2 2 2 2 2
1163 220/300 Write block 4402 len 40960 data: 4 3 3 3 3 3 3 3 3 3
1164 221/300 Write block 4365 len 28672 data: 2 2 2 2 2 2 2
1165 222/300 Read block 3502 len 24576
1166 223/300 Write block 1998 len 4096 data: 2
1167 224/300 Read block 3717 len 36864
1168 225/300 Read block 2652 len 32768
1169 226/300 Write block 229 len 8192 data: 2 2
1170 227/300 Write block 951 len 20480 data: 2 2 2 2 2
1171 228/300 Write block 988 len 36864 data: 2 2 2 2 2 2 2 2 2
1172 229/300 Write block 3751 len 8192 data: 2 2
1173 230/300 Read block 3390 len 32768
1174 231/300 Write block 3995 len 40960 data: 2 2 3 3 3 3 3 3 3 3
1175 232/300 Read block 2264 len 40960
1176 233/300 Flush
1177 234/300 Flush
1178 235/300 Read block 4682 len 12288
1179 236/300 Write block 1683 len 8192 data: 2 2
1180 237/300 Read block 4812 len 12288
1181 238/300 Read block 1061 len 12288
1182 239/300 Write block 4345 len 4096 data: 2
1183 240/300 Write block 4599 len 36864 data: 2 2 2 3 3 3 3 3 3
1184 241/300 Read block 1844 len 16384
1185 242/300 Read block 2076 len 20480
1186 243/300 Flush
1187 244/300 Write block 4060 len 32768 data: 2 2 2 2 2 2 2 2
1188 245/300 Read block 4836 len 28672
1189 246/300 Write block 1514 len 28672 data: 2 2 2 2 2 2 2
1190 247/300 Flush
1191 248/300 Read block 1757 len 36864
1192 249/300 Write block 4097 len 28672 data: 3 3 2 2 2 2 2
1193 250/300 Read block 924 len 28672
1194 251/300 Write block 1322 len 36864 data: 2 2 2 2 2 2 2 2 2
1195 252/300 Read block 1067 len 12288
1196 253/300 Read block 1901 len 40960
1197 254/300 Read block 2270 len 8192
1198 255/300 Write block 4739 len 24576 data: 2 2 3 3 3 3
1199 256/300 Read block 2903 len 8192
1200 257/300 Flush
1201 258/300 Write block 2182 len 12288 data: 3 3 3
1202 259/300 Read block 1954 len 28672
1203 260/300 Read block 2495 len 28672
1204 261/300 Read block 412 len 20480
1205 262/300 Read block 4428 len 16384
1206 263/300 Flush
1207 264/300 Flush
1208 265/300 Read block 4498 len 36864
1209 266/300 Read block 2084 len 20480
1210 267/300 Write block 51 len 28672 data: 4 4 4 4 4 4 3
1211 268/300 Write block 4201 len 24576 data: 2 2 2 2 2 2
1212 269/300 Write block 2049 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1213 270/300 Flush
1214 271/300 Read block 1011 len 40960
1215 272/300 Write block 2828 len 24576 data: 2 3 3 2 2 2
1216 273/300 Write block 4574 len 16384 data: 2 2 2 2
1217 274/300 Read block 4900 len 8192
1218 275/300 Read block 2696 len 8192
1219 276/300 Write block 80 len 16384 data: 2 2 2 2
1220 277/300 Write block 569 len 36864 data: 2 2 2 2 2 2 3 3 3
1221 278/300 Read block 141 len 32768
1222 279/300 Read block 3181 len 12288
1223 280/300 Write block 851 len 28672 data: 2 2 2 2 2 2 2
1224 281/300 Write block 2424 len 24576 data: 2 2 2 2 2 2
1225 282/300 Read block 1269 len 8192
1226 283/300 Write block 1752 len 40960 data: 3 3 3 2 2 2 2 2 2 2
1227 284/300 Write block 777 len 12288 data: 2 2 2
1228 285/300 Write block 2522 len 28672 data: 2 2 2 2 2 2 2
1229 286/300 Write block 4359 len 24576 data: 2 2 2 2 2 2
1230 287/300 Read block 4919 len 28672
1231 288/300 Read block 4327 len 32768
1232 289/300 Read block 1665 len 12288
1233 290/300 Read block 4381 len 32768
1234 291/300 Write block 4897 len 28672 data: 2 2 2 2 2 2 3
1235 292/300 Write block 3140 len 16384 data: 2 2 2 3
1236 293/300 Write block 3205 len 24576 data: 2 2 2 2 2 2
1237 294/300 Read block 3542 len 20480
1238 295/300 Write block 2453 len 4096 data: 2
1239 296/300 Flush
1240 297/300 Flush
1241 298/300 Write block 4487 len 32768 data: 2 3 3 3 3 3 3 2
1242 299/300 Write block 1655 len 4096 data: 2
1243 300/300 Write block 4819 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1244 Main task has work to do, go find it
1245 got dsc Start(0)
1246 start 0
1247 Sep 22 23:24:03.757 INFO request completed, latency_us: 307, response_code: 204, uri: /start/cid/0, method: POST, req_id: 0fba962f-82ea-4041-bca7-ee38483ee607, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
1248 Replay: started 0, returned:Ok(())
1249 ----------------------------------------------------------------
1250 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1251 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
1252 303 Acked 1302 Write 3 New Done Done false
1253 304 Acked 1303 Write 3 New Done Done false
1254 305 Acked 1304 Flush 0 New Done Done false
1255 306 Acked 1305 Read 9 New Done Done false
1256 307 Acked 1306 Read 2 New Done Done false
1257 308 Acked 1307 Read 5 New Done Done false
1258 309 Acked 1308 Read 8 New Done Done false
1259 310 Acked 1309 Write 6 New Done Done false
1260 311 Acked 1310 Write 10 New Done Done false
1261 312 Acked 1311 Write 5 New Done Done false
1262 313 Acked 1312 Read 8 New Done Done false
1263 314 Acked 1313 Read 8 New Done Done false
1264 315 Acked 1314 Write 7 New Done Done false
1265 316 Acked 1315 Write 3 New Done Done false
1266 317 Acked 1316 Read 3 New Done Done false
1267 318 Acked 1317 Write 8 New Done Done false
1268 319 Acked 1318 Read 10 New Done Done false
1269 320 Acked 1319 Read 5 New Done Done false
1270 321 Acked 1320 Flush 0 New Done Done false
1271 322 Acked 1321 Write 7 New Done Done false
1272 323 Acked 1322 Read 4 New Done Done false
1273 324 Acked 1323 Write 4 New Done Done false
1274 325 Acked 1324 Write 6 New Done Done false
1275 326 Acked 1325 Write 5 New Done Done false
1276 327 Acked 1326 Flush 0 New Done Done false
1277 328 Acked 1327 Write 10 New Done Done false
1278 329 Acked 1328 Read 4 New Done Done false
1279 330 Acked 1329 Read 1 New Done Done false
1280 331 Acked 1330 Write 10 New Done Done false
1281 332 Acked 1331 Flush 0 New Done Done false
1282 333 Acked 1332 Read 4 New Done Done false
1283 334 Acked 1333 Write 3 New Done Done false
1284 335 Acked 1334 Flush 0 New Done Done false
1285 336 Acked 1335 Write 6 New Done Done false
1286 337 Acked 1336 Write 9 New Done Done false
1287 338 Acked 1337 Flush 0 New Done Done false
1288 339 Acked 1338 Write 1 New Done Done false
1289 340 Acked 1339 Read 7 New Done Done false
1290 341 Acked 1340 Read 7 New Done Done false
1291 342 Acked 1341 Read 1 New Done Done false
1292 343 Acked 1342 Write 6 New Done Done false
1293 344 Acked 1343 Read 9 New Done Done false
1294 345 Acked 1344 Flush 0 New Done Done false
1295 346 Acked 1345 Write 8 New Done Done false
1296 347 Acked 1346 Write 6 New Done Done false
1297 348 Acked 1347 Read 6 New Done Done false
1298 349 Acked 1348 Write 10 New Done Done false
1299 350 Acked 1349 Read 4 New Done Done false
1300 351 Acked 1350 Read 6 New Done Done false
1301 352 Acked 1351 Read 1 New Done Done false
1302 353 Acked 1352 Write 9 New Done Done false
1303 354 Acked 1353 Read 4 New Done Done false
1304 355 Acked 1354 Read 5 New Done Done false
1305 356 Acked 1355 Write 9 New Done Done false
1306 357 Acked 1356 Write 3 New Done Done false
1307 358 Acked 1357 Write 5 New Done Done false
1308 359 Acked 1358 Read 6 New Done Done false
1309 360 Acked 1359 Write 10 New Done Done false
1310 361 Acked 1360 Write 2 New Done Done false
1311 362 Acked 1361 Read 6 New Done Done false
1312 363 Acked 1362 Write 4 New Done Done false
1313 364 Acked 1363 Read 3 New Done Done false
1314 365 Acked 1364 Read 6 New Done Done false
1315 366 Acked 1365 Write 2 New Done Done false
1316 367 Acked 1366 Write 7 New Done Done false
1317 368 Acked 1367 Read 1 New Done Done false
1318 369 Acked 1368 Write 4 New Done Done false
1319 370 Acked 1369 Read 2 New Done Done false
1320 371 Acked 1370 Write 9 New Done Done false
1321 372 Acked 1371 Read 3 New Done Done false
1322 373 Acked 1372 Read 6 New Done Done false
1323 374 Acked 1373 Write 9 New Done Done false
1324 375 Acked 1374 Read 4 New Done Done false
1325 376 Acked 1375 Write 7 New Done Done false
1326 377 Acked 1376 Read 4 New Done Done false
1327 378 Acked 1377 Write 6 New Done Done false
1328 379 Acked 1378 Flush 0 New Done Done false
1329 380 Acked 1379 Read 3 New Done Done false
1330 381 Acked 1380 Read 4 New Done Done false
1331 382 Acked 1381 Write 1 New Done Done false
1332 383 Acked 1382 Read 3 New Done Done false
1333 384 Acked 1383 Write 8 New Done Done false
1334 385 Acked 1384 Write 7 New Done Done false
1335 386 Acked 1385 Read 2 New Done Done false
1336 387 Acked 1386 Read 7 New Done Done false
1337 388 Acked 1387 Write 5 New Done Done false
1338 389 Acked 1388 Write 3 New Done Done false
1339 390 Acked 1389 Read 9 New Done Done false
1340 391 Acked 1390 Read 10 New Done Done false
1341 392 Acked 1391 Write 3 New Done Done false
1342 393 Acked 1392 Read 5 New Done Done false
1343 394 Acked 1393 Read 4 New Done Done false
1344 395 Acked 1394 Flush 0 New Done Done false
1345 396 Acked 1395 Read 3 New Done Done false
1346 397 Acked 1396 Write 8 New Done Done false
1347 398 Acked 1397 Read 6 New Done Done false
1348 399 Acked 1398 Read 5 New Done Done false
1349 400 Acked 1399 Read 7 New Done Done false
1350 401 Acked 1400 Read 2 New Done Done false
1351 402 Acked 1401 Read 5 New Done Done false
1352 403 Acked 1402 Write 1 New Done Done false
1353 404 Acked 1403 Write 9 New Done Done false
1354 405 Acked 1404 Write 9 New Done Done false
1355 406 Acked 1405 Read 5 New Done Done false
1356 407 Acked 1406 Read 9 New Done Done false
1357 408 Acked 1407 Write 4 New Done Done false
1358 409 Acked 1408 Write 2 New Done Done false
1359 410 Acked 1409 Read 3 New Done Done false
1360 411 Acked 1410 Read 6 New Done Done false
1361 412 Acked 1411 Read 4 New Done Done false
1362 413 Acked 1412 Read 5 New Done Done false
1363 414 Acked 1413 Read 8 New Done Done false
1364 415 Acked 1414 Flush 0 New Done Done false
1365 416 Acked 1415 Write 2 New Done Done false
1366 417 Acked 1416 Write 6 New Done Done false
1367 418 Acked 1417 Read 1 New Done Done false
1368 419 Acked 1418 Read 9 New Done Done false
1369 420 Acked 1419 Write 7 New Done Done false
1370 421 Acked 1420 Write 9 New Done Done false
1371 422 Acked 1421 Read 2 New Done Done false
1372 423 Acked 1422 Read 10 New Done Done false
1373 424 Acked 1423 Read 4 New Done Done false
1374 425 Acked 1424 Read 6 New Done Done false
1375 426 Acked 1425 Read 3 New Done Done false
1376 427 Acked 1426 Read 5 New Done Done false
1377 428 Acked 1427 Read 6 New Done Done false
1378 429 Acked 1428 Write 1 New Done Done false
1379 430 Acked 1429 Read 8 New Done Done false
1380 431 Acked 1430 Read 5 New Done Done false
1381 432 Acked 1431 Flush 0 New Done Done false
1382 433 Acked 1432 Write 6 New Done Done false
1383 434 Acked 1433 Write 4 New Done Done false
1384 435 Acked 1434 Flush 0 New Done Done false
1385 436 Acked 1435 Flush 0 New Done Done false
1386 437 Acked 1436 Read 5 New Done Done false
1387 438 Acked 1437 Read 7 New Done Done false
1388 439 Acked 1438 Read 1 New Done Done false
1389 440 Acked 1439 Write 4 New Done Done false
1390 441 Acked 1440 Write 5 New Done Done false
1391 442 Acked 1441 Write 10 New Done Done false
1392 443 Acked 1442 Read 10 New Done Done false
1393 444 Acked 1443 Read 2 New Done Done false
1394 445 Acked 1444 Flush 0 New Done Done false
1395 446 Acked 1445 Read 3 New Done Done false
1396 447 Acked 1446 Read 7 New Done Done false
1397 448 Acked 1447 Read 8 New Done Done false
1398 449 Acked 1448 Flush 0 New Done Done false
1399 450 Acked 1449 Flush 0 New Done Done false
1400 451 Acked 1450 Read 7 New Done Done false
1401 452 Acked 1451 Flush 0 New Done Done false
1402 453 Acked 1452 Write 3 New Done Done false
1403 454 Acked 1453 Read 10 New Done Done false
1404 455 Acked 1454 Flush 0 New Done Done false
1405 456 Acked 1455 Write 9 New Done Done false
1406 457 Acked 1456 Write 7 New Done Done false
1407 458 Acked 1457 Write 10 New Done Done false
1408 459 Acked 1458 Flush 0 New Done Done false
1409 460 Acked 1459 Write 4 New Done Done false
1410 461 Acked 1460 Read 1 New Done Done false
1411 462 Acked 1461 Read 6 New Done Done false
1412 463 Acked 1462 Read 9 New Done Done false
1413 464 Acked 1463 Write 4 New Done Done false
1414 465 Acked 1464 Read 5 New Done Done false
1415 466 Acked 1465 Write 5 New Done Done false
1416 467 Acked 1466 Read 3 New Done Done false
1417 468 Acked 1467 Write 10 New Done Done false
1418 469 Acked 1468 Write 10 New Done Done false
1419 470 Acked 1469 Flush 0 New Done Done false
1420 471 Acked 1470 Write 3 New Done Done false
1421 472 Acked 1471 Read 7 New Done Done false
1422 473 Acked 1472 Read 6 New Done Done false
1423 474 Acked 1473 Read 8 New Done Done false
1424 475 Acked 1474 Write 10 New Done Done false
1425 476 Acked 1475 Read 5 New Done Done false
1426 477 Acked 1476 Write 9 New Done Done false
1427 478 Acked 1477 Read 9 New Done Done false
1428 479 Acked 1478 Read 8 New Done Done false
1429 480 Acked 1479 Read 3 New Done Done false
1430 481 Acked 1480 Flush 0 New Done Done false
1431 482 Acked 1481 Write 10 New Done Done false
1432 483 Acked 1482 Flush 0 New Done Done false
1433 484 Acked 1483 Write 3 New Done Done false
1434 485 Acked 1484 Read 3 New Done Done false
1435 486 Acked 1485 Read 8 New Done Done false
1436 487 Acked 1486 Write 5 New Done Done false
1437 488 Acked 1487 Read 7 New Done Done false
1438 489 Acked 1488 Write 6 New Done Done false
1439 490 Acked 1489 Write 5 New Done Done false
1440 491 Acked 1490 Write 9 New Done Done false
1441 492 Acked 1491 Read 9 New Done Done false
1442 493 Acked 1492 Flush 0 New Done Done false
1443 494 Acked 1493 Read 5 New Done Done false
1444 495 Acked 1494 Read 8 New Done Done false
1445 496 Acked 1495 Read 5 New Done Done false
1446 497 Acked 1496 Write 2 New Done Done false
1447 498 Acked 1497 Read 6 New Done Done false
1448 499 Acked 1498 Write 7 New Done Done false
1449 500 Acked 1499 Read 10 New Done Done false
1450 501 Acked 1500 Write 5 New Done Done false
1451 502 Acked 1501 Read 2 New Done Done false
1452 503 Acked 1502 Write 9 New Done Done false
1453 504 Acked 1503 Read 9 New Done Done false
1454 505 Acked 1504 Read 10 New Done Done false
1455 506 Acked 1505 Read 8 New Done Done false
1456 507 Acked 1506 Read 2 New Done Done false
1457 508 Acked 1507 Read 3 New Done Done false
1458 509 Acked 1508 Read 6 New Done Done false
1459 510 Acked 1509 Write 3 New Done Done false
1460 511 Acked 1510 Write 6 New Done Done false
1461 512 Acked 1511 Read 1 New Done Done false
1462 513 Acked 1512 Read 10 New Done Done false
1463 514 Acked 1513 Write 9 New Done Done false
1464 515 Acked 1514 Read 10 New Done Done false
1465 516 Acked 1515 Write 1 New Done Done false
1466 517 Acked 1516 Read 8 New Done Done false
1467 518 Acked 1517 Read 1 New Done Done false
1468 519 Acked 1518 Read 7 New Done Done false
1469 520 Acked 1519 Write 6 New Done Done false
1470 521 Acked 1520 Read 7 New Done Done false
1471 522 Acked 1521 Write 6 New Done Done false
1472 523 Acked 1522 Write 10 New Done Done false
1473 524 Acked 1523 Write 7 New Done Done false
1474 525 Acked 1524 Read 6 New Done Done false
1475 526 Acked 1525 Write 1 New Done Done false
1476 527 Acked 1526 Read 9 New Done Done false
1477 528 Acked 1527 Read 8 New Done Done false
1478 529 Acked 1528 Write 2 New Done Done false
1479 530 Acked 1529 Write 5 New Done Done false
1480 531 Acked 1530 Write 9 New Done Done false
1481 532 Acked 1531 Write 2 New Done Done false
1482 533 Acked 1532 Read 8 New Done Done false
1483 534 Acked 1533 Write 10 New Done Done false
1484 535 Acked 1534 Read 10 New Done Done false
1485 536 Acked 1535 Flush 0 New Done Done false
1486 537 Acked 1536 Flush 0 New Done Done false
1487 538 Acked 1537 Read 3 New Done Done false
1488 539 Acked 1538 Write 2 New Done Done false
1489 540 Acked 1539 Read 3 New Done Done false
1490 541 Acked 1540 Read 3 New Done Done false
1491 542 Acked 1541 Write 1 New Done Done false
1492 543 Acked 1542 Write 9 New Done Done false
1493 544 Acked 1543 Read 4 New Done Done false
1494 545 Acked 1544 Read 5 New Done Done false
1495 546 Acked 1545 Flush 0 New Done Done false
1496 547 Acked 1546 Write 8 New Done Done false
1497 548 Acked 1547 Read 7 New Done Done false
1498 549 Acked 1548 Write 7 New Done Done false
1499 550 Acked 1549 Flush 0 New Done Done false
1500 551 Acked 1550 Read 9 New Done Done false
1501 552 Acked 1551 Write 7 New Done Done false
1502 553 Acked 1552 Read 7 New Done Done false
1503 554 Acked 1553 Write 9 New Done Done false
1504 555 Acked 1554 Read 3 New Done Done false
1505 556 Acked 1555 Read 10 New Done Done false
1506 557 Acked 1556 Read 2 New Done Done false
1507 558 Acked 1557 Write 6 New Done Done false
1508 559 Acked 1558 Read 2 New Done Done false
1509 560 Acked 1559 Flush 0 New Done Done false
1510 561 Acked 1560 Write 3 New Done Done false
1511 562 Acked 1561 Read 7 New Done Done false
1512 563 Acked 1562 Read 7 New Done Done false
1513 564 Acked 1563 Read 5 New Done Done false
1514 565 Acked 1564 Read 4 New Done Done false
1515 566 Acked 1565 Flush 0 New Done Done false
1516 567 Acked 1566 Flush 0 New Done Done false
1517 568 Acked 1567 Read 9 New Done Done false
1518 569 Acked 1568 Read 5 New Done Done false
1519 570 Acked 1569 Write 7 New Done Done false
1520 571 Acked 1570 Write 6 New Done Done false
1521 572 Acked 1571 Write 10 New Done Done false
1522 573 Acked 1572 Flush 0 New Done Done false
1523 574 Acked 1573 Read 10 New Done Done false
1524 575 Acked 1574 Write 6 New Done Done false
1525 576 Acked 1575 Write 4 New Done Done false
1526 577 Acked 1576 Read 2 New Done Done false
1527 578 Acked 1577 Read 2 New Done Done false
1528 579 Acked 1578 Write 4 New Done Done false
1529 580 Acked 1579 Write 9 New Done Done false
1530 581 Acked 1580 Read 8 New Done Done false
1531 582 Acked 1581 Read 3 New Done Done false
1532 583 Acked 1582 Write 7 New Done Done false
1533 584 Acked 1583 Write 6 New Done Done false
1534 585 Acked 1584 Read 2 New Done Done false
1535 586 Acked 1585 Write 10 New Done Done false
1536 587 Acked 1586 Write 3 New Done Done false
1537 588 Acked 1587 Write 7 New Done Done false
1538 589 Acked 1588 Write 6 New Done Done false
1539 590 Acked 1589 Read 7 New Done Done false
1540 591 Acked 1590 Read 8 New Done Done false
1541 592 Acked 1591 Read 3 New Done Done false
1542 593 Acked 1592 Read 8 New Done Done false
1543 594 Acked 1593 Write 7 New Done Done false
1544 595 Acked 1594 Write 4 New Done Done false
1545 596 Acked 1595 Write 6 New Done Done false
1546 597 Acked 1596 Read 5 New Done Done false
1547 598 Acked 1597 Write 1 New Done Done false
1548 599 Acked 1598 Flush 0 New Done Done false
1549 600 Acked 1599 Flush 0 New Done Done false
1550 601 Acked 1600 Write 8 New Done Done false
1551 602 Acked 1601 Write 1 New Sent Sent false
1552 603 Acked 1602 Write 10 New Sent Sent false
1553 STATES DS:0 DS:1 DS:2 TOTAL
1554 New 301 0 0 301
1555 Sent 0 2 2 4
1556 Done 0 299 299 598
1557 Skipped 0 0 0 0
1558 Error 0 0 0 0
1559 Last Flush: 1301 1599 1599
1560 Downstairs last five completed: 1301 1300 1299 1298 1297 1296
1561 Upstairs last five completed: 603 602 601 600 599 598
1562 CLIENT: Up:0 ds:301 act:2
1563 [8810] got start action
1564 [8810] I am going to restart
1565 Starting downstairs at port 8810
1566 Make output file at "/tmp/dsc/downstairs-8810.txt"
1567 [8810][0] reports Starting
1568 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1175)
1569 [8810][0] reports Running
15702023-09-22T23:24:05.805ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
15712023-09-22T23:24:05.805ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
15722023-09-22T23:24:05.805ZINFOcrucible: [0] client re-new 302 jobs since flush 1301 = downstairs
15732023-09-22T23:24:05.806ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
15742023-09-22T23:24:05.806ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
15752023-09-22T23:24:05.806ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
15762023-09-22T23:24:05.806ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
15772023-09-22T23:24:05.806ZINFOcrucible: [0] send last flush ID to this DS: 1301
15782023-09-22T23:24:05.806ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
15792023-09-22T23:24:05.807ZINFOcrucible: [0] Replied this last flush ID: 1301
15802023-09-22T23:24:05.807ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
15812023-09-22T23:24:05.807ZINFOcrucible: [0] Transition from Offline to Replay
15822023-09-22T23:24:05.807ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
15832023-09-22T23:24:05.807ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
15842023-09-22T23:24:05.807ZINFOcrucible: [0] Transition from Replay to Active
15852023-09-22T23:24:05.807ZINFOcrucible: [0] Starts cmd_loop
15862023-09-22T23:24:05.807ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
15872023-09-22T23:24:05.807ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
15882023-09-22T23:24:05.807ZINFOcrucible: Set check for repair
15892023-09-22T23:24:06.807ZWARNcrucible: [0] flow control end
15902023-09-22T23:24:06.808ZINFOcrucible: Checking if live repair is needed
15912023-09-22T23:24:06.808ZINFOcrucible: No Live Repair required at this time
1592 ----------------------------------------------------------------
1593 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1594 Downstairs last five completed: 1603 1602 1601 1600 1599 1598
1595 Upstairs last five completed: 604 603 602 601 600 599
1596 CLIENT: Up:0 ds:0 act:3
1597 Replay: All jobs finished, all DS active.
1598 Main task has work to do, go find it
1599 got dsc Stop(1)
1600 stop 1
1601 Sep 22 23:24:07.762 INFO request completed, latency_us: 496, response_code: 204, uri: /stop/cid/1, method: POST, req_id: ba9b9004-bc4b-4ab4-a4b2-0cf4a0251f14, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
1602 [8820] Got stop action so:false kr:false
1603 Sep 22 23:24:07.763 INFO request completed, latency_us: 289, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4eec3edc-ba42-401e-8a9f-1a477082f5fa, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
16042023-09-22T23:24:07.763ZWARNcrucible: [1] downstairs disconnected
16052023-09-22T23:24:07.764ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
16062023-09-22T23:24:07.764ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
16072023-09-22T23:24:07.764ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
16082023-09-22T23:24:07.764ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
16092023-09-22T23:24:07.764ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
16102023-09-22T23:24:07.764ZWARNcrucible: [1] pm_task rx.recv() is None
1611 [8820] Exited with: ExitStatus(unix_wait_status(9))
1612 [8820][1] reports Exit
16132023-09-22T23:24:10.766ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
1614 Sep 22 23:24:11.765 INFO request completed, latency_us: 280, response_code: 200, uri: /state/cid/1, method: GET, req_id: 2c152a20-815f-40eb-8806-8632ce2ab74f, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
1615 001/300 Flush
1616 002/300 Write block 4084 len 8192 data: 2 2
1617 003/300 Read block 1039 len 32768
1618 004/300 Read block 894 len 12288
1619 005/300 Read block 1491 len 16384
1620 006/300 Write block 356 len 20480 data: 3 3 3 3 2
1621 007/300 Flush
1622 008/300 Write block 1334 len 20480 data: 3 3 3 4 4
1623 009/300 Read block 3532 len 32768
1624 010/300 Write block 3277 len 8192 data: 2 2
1625 011/300 Flush
1626 012/300 Read block 3647 len 28672
1627 013/300 Write block 111 len 36864 data: 3 4 4 4 4 3 3 3 2
1628 014/300 Write block 4397 len 32768 data: 3 3 3 3 3 5 4 4
1629 015/300 Read block 735 len 12288
1630 016/300 Read block 3460 len 24576
1631 017/300 Read block 4198 len 28672
1632 018/300 Write block 719 len 36864 data: 2 2 2 2 3 3 3 4 4
1633 019/300 Read block 3838 len 40960
1634 020/300 Write block 1803 len 12288 data: 2 2 2
1635 021/300 Read block 4805 len 8192
1636 022/300 Read block 3730 len 4096
1637 023/300 Write block 3994 len 36864 data: 3 3 3 4 4 4 4 4 4
1638 024/300 Write block 4680 len 12288 data: 2 2 2
1639 025/300 Read block 2932 len 12288
1640 026/300 Write block 2871 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1641 027/300 Write block 2257 len 8192 data: 2 2
1642 028/300 Read block 415 len 8192
1643 029/300 Write block 3151 len 40960 data: 2 2 2 2 3 3 2 2 2 2
1644 030/300 Write block 2782 len 12288 data: 2 2 2
1645 031/300 Read block 4124 len 32768
1646 032/300 Write block 3700 len 16384 data: 2 2 2 2
1647 033/300 Write block 4916 len 16384 data: 3 3 3 3
1648 034/300 Read block 2253 len 4096
1649 035/300 Read block 3465 len 16384
1650 036/300 Read block 897 len 16384
1651 037/300 Write block 3841 len 16384 data: 2 2 2 2
1652 038/300 Read block 96 len 40960
1653 039/300 Read block 2843 len 4096
1654 040/300 Read block 2781 len 16384
1655 041/300 Flush
1656 042/300 Read block 4399 len 28672
1657 043/300 Write block 3900 len 24576 data: 3 3 3 3 2 2
1658 044/300 Read block 530 len 8192
1659 045/300 Read block 845 len 36864
1660 046/300 Read block 3942 len 24576
1661 047/300 Read block 4443 len 4096
1662 048/300 Read block 4343 len 12288
1663 049/300 Read block 3803 len 28672
1664 050/300 Write block 1592 len 20480 data: 3 3 3 3 3
1665 051/300 Flush
1666 052/300 Write block 3963 len 36864 data: 2 2 2 2 2 2 2 2 2
1667 053/300 Read block 727 len 20480
1668 054/300 Write block 4394 len 8192 data: 2 2
1669 055/300 Flush
1670 056/300 Flush
1671 057/300 Read block 3473 len 28672
1672 058/300 Read block 3999 len 24576
1673 059/300 Write block 4273 len 20480 data: 2 2 2 2 2
1674 060/300 Write block 4325 len 8192 data: 2 2
1675 061/300 Write block 2205 len 20480 data: 2 2 2 2 2
1676 062/300 Read block 3809 len 12288
1677 063/300 Read block 3687 len 36864
1678 064/300 Read block 200 len 24576
1679 065/300 Read block 3565 len 24576
1680 066/300 Write block 1566 len 40960 data: 3 3 3 3 3 3 3 2 2 2
1681 067/300 Write block 1223 len 32768 data: 2 2 2 2 2 2 2 2
1682 068/300 Write block 4357 len 4096 data: 2
1683 069/300 Flush
1684 070/300 Read block 4334 len 36864
1685 071/300 Write block 4124 len 24576 data: 2 2 2 2 2 3
1686 072/300 Write block 4365 len 36864 data: 3 3 3 3 3 3 3 2 2
1687 073/300 Read block 3185 len 20480
1688 074/300 Read block 2409 len 20480
1689 075/300 Read block 2415 len 12288
1690 076/300 Write block 1363 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1691 077/300 Read block 960 len 28672
1692 078/300 Read block 1832 len 40960
1693 079/300 Read block 3383 len 20480
1694 080/300 Write block 3718 len 8192 data: 3 3
1695 081/300 Flush
1696 082/300 Read block 4485 len 28672
1697 083/300 Read block 2999 len 32768
1698 084/300 Write block 835 len 40960 data: 3 3 3 3 3 3 2 2 2 2
1699 085/300 Write block 4509 len 4096 data: 2
1700 086/300 Write block 3462 len 8192 data: 2 2
1701 087/300 Write block 384 len 16384 data: 3 2 2 2
1702 088/300 Write block 1787 len 24576 data: 2 2 3 3 3 3
1703 089/300 Write block 2901 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1704 090/300 Write block 3848 len 24576 data: 2 2 2 2 2 2
1705 091/300 Read block 4943 len 16384
1706 092/300 Read block 2210 len 32768
1707 093/300 Read block 4524 len 8192
1708 094/300 Read block 1247 len 24576
1709 095/300 Write block 3260 len 36864 data: 3 2 2 2 2 2 2 2 2
1710 096/300 Read block 399 len 20480
1711 097/300 Write block 1006 len 12288 data: 2 2 2
1712 098/300 Read block 110 len 32768
1713 099/300 Write block 3507 len 32768 data: 2 2 2 2 2 2 2 2
1714 100/300 Read block 4392 len 12288
1715 101/300 Write block 3071 len 12288 data: 2 2 2
1716 102/300 Write block 1678 len 40960 data: 2 2 2 2 2 3 3 2 2 2
1717 103/300 Flush
1718 104/300 Write block 470 len 16384 data: 2 2 2 2
1719 105/300 Write block 2942 len 20480 data: 2 2 2 2 2
1720 106/300 Write block 398 len 12288 data: 2 2 2
1721 107/300 Read block 3173 len 36864
1722 108/300 Write block 3589 len 28672 data: 2 3 3 2 2 2 2
1723 109/300 Read block 4745 len 12288
1724 110/300 Write block 238 len 20480 data: 2 2 2 2 2
1725 111/300 Flush
1726 112/300 Write block 37 len 40960 data: 2 2 2 2 2 2 2 2 2 3
1727 113/300 Read block 4624 len 28672
1728 114/300 Flush
1729 115/300 Read block 4675 len 32768
1730 116/300 Write block 2449 len 16384 data: 2 2 2 2
1731 117/300 Read block 437 len 16384
1732 118/300 Read block 3102 len 32768
1733 119/300 Write block 3906 len 24576 data: 2 3 3 3 2 2
1734 120/300 Flush
1735 121/300 Write block 4077 len 28672 data: 2 2 2 2 2 2 2
1736 122/300 Read block 945 len 12288
1737 123/300 Read block 537 len 40960
1738 124/300 Read block 2520 len 28672
1739 125/300 Flush
1740 126/300 Flush
1741 127/300 Write block 1413 len 36864 data: 2 2 2 2 2 2 2 2 2
1742 128/300 Read block 2303 len 28672
1743 129/300 Read block 4415 len 28672
1744 130/300 Read block 1934 len 28672
1745 131/300 Flush
1746 132/300 Write block 444 len 12288 data: 2 2 2
1747 133/300 Read block 3595 len 40960
1748 134/300 Write block 2832 len 24576 data: 3 3 2 2 2 2
1749 135/300 Read block 2748 len 36864
1750 136/300 Flush
1751 137/300 Read block 338 len 12288
1752 138/300 Write block 4351 len 32768 data: 2 2 2 2 2 2 3 2
1753 139/300 Read block 3524 len 40960
1754 140/300 Read block 4293 len 16384
1755 141/300 Flush
1756 142/300 Write block 3837 len 16384 data: 2 2 2 2
1757 143/300 Write block 2300 len 12288 data: 2 2 2
1758 144/300 Read block 1336 len 40960
1759 145/300 Read block 1112 len 28672
1760 146/300 Flush
1761 147/300 Read block 1592 len 12288
1762 148/300 Flush
1763 149/300 Read block 2462 len 36864
1764 150/300 Read block 3047 len 4096
1765 151/300 Write block 1154 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1766 152/300 Write block 3081 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1767 153/300 Write block 2947 len 32768 data: 2 2 2 2 2 3 2 2
1768 154/300 Write block 436 len 8192 data: 2 2
1769 155/300 Read block 592 len 36864
1770 156/300 Read block 632 len 36864
1771 157/300 Write block 4106 len 20480 data: 2 2 2 2 2
1772 158/300 Flush
1773 159/300 Flush
1774 160/300 Read block 4494 len 32768
1775 161/300 Read block 260 len 24576
1776 162/300 Read block 619 len 40960
1777 163/300 Write block 4819 len 8192 data: 3 3
1778 164/300 Write block 665 len 8192 data: 2 2
1779 165/300 Write block 2844 len 32768 data: 5 5 3 2 2 2 2 2
1780 166/300 Write block 1171 len 12288 data: 2 2 2
1781 167/300 Read block 3472 len 16384
1782 168/300 Write block 3091 len 16384 data: 2 2 2 2
1783 169/300 Read block 2484 len 36864
1784 170/300 Write block 4319 len 4096 data: 2
1785 171/300 Flush
1786 172/300 Write block 2424 len 32768 data: 3 3 3 3 3 3 2 2
1787 173/300 Read block 1676 len 4096
1788 174/300 Read block 3462 len 16384
1789 175/300 Write block 3215 len 16384 data: 2 2 2 2
1790 176/300 Read block 4923 len 24576
1791 177/300 Write block 3903 len 12288 data: 4 3 3
1792 178/300 Write block 333 len 32768 data: 2 2 2 2 2 2 2 2
1793 179/300 Write block 245 len 40960 data: 2 2 2 2 2 2 2 3 3 2
1794 180/300 Write block 3210 len 24576 data: 3 2 2 2 2 3
1795 181/300 Read block 4390 len 16384
1796 182/300 Read block 3720 len 12288
1797 183/300 Read block 2964 len 28672
1798 184/300 Write block 1062 len 12288 data: 2 2 2
1799 185/300 Read block 3242 len 24576
1800 186/300 Read block 3063 len 20480
1801 187/300 Read block 4710 len 32768
1802 188/300 Read block 45 len 12288
1803 189/300 Read block 1641 len 16384
1804 190/300 Write block 3340 len 32768 data: 3 3 3 3 3 2 2 2
1805 191/300 Read block 3743 len 36864
1806 192/300 Read block 1968 len 8192
1807 193/300 Write block 4647 len 4096 data: 3
1808 194/300 Read block 3014 len 40960
1809 195/300 Read block 4350 len 20480
1810 196/300 Read block 4218 len 16384
1811 197/300 Write block 2692 len 20480 data: 2 3 3 3 4
1812 198/300 Write block 4505 len 16384 data: 2 2 2 2
1813 199/300 Write block 2353 len 8192 data: 2 2
1814 200/300 Read block 2187 len 20480
1815 201/300 Read block 1394 len 20480
1816 202/300 Read block 4272 len 4096
1817 203/300 Read block 2709 len 12288
1818 204/300 Read block 4327 len 8192
1819 205/300 Read block 2196 len 8192
1820 206/300 Write block 1002 len 8192 data: 3 3
1821 207/300 Read block 1876 len 28672
1822 208/300 Read block 3650 len 8192
1823 209/300 Read block 359 len 40960
1824 210/300 Write block 3744 len 20480 data: 3 3 2 2 2
1825 211/300 Read block 4949 len 12288
1826 212/300 Read block 834 len 40960
1827 213/300 Write block 4034 len 28672 data: 2 2 2 2 2 2 2
1828 214/300 Read block 4079 len 8192
1829 215/300 Flush
1830 216/300 Write block 3095 len 32768 data: 2 2 3 3 3 3 3 3
1831 217/300 Write block 2977 len 4096 data: 2
1832 218/300 Write block 2657 len 8192 data: 2 2
1833 219/300 Read block 2288 len 20480
1834 220/300 Read block 4399 len 20480
1835 221/300 Write block 4612 len 20480 data: 2 2 2 2 2
1836 222/300 Write block 3709 len 12288 data: 2 2 2
1837 223/300 Flush
1838 224/300 Read block 3235 len 16384
1839 225/300 Write block 1203 len 24576 data: 2 2 2 2 2 2
1840 226/300 Read block 2770 len 24576
1841 227/300 Read block 2912 len 8192
1842 228/300 Read block 2164 len 20480
1843 229/300 Read block 3972 len 8192
1844 230/300 Flush
1845 231/300 Read block 1720 len 32768
1846 232/300 Flush
1847 233/300 Read block 970 len 36864
1848 234/300 Write block 2335 len 28672 data: 2 2 2 2 2 2 2
1849 235/300 Read block 3333 len 16384
1850 236/300 Flush
1851 237/300 Write block 17 len 4096 data: 2
1852 238/300 Flush
1853 239/300 Write block 1269 len 36864 data: 2 2 2 2 2 2 2 2 2
1854 240/300 Write block 4987 len 4096 data: 3
1855 241/300 Read block 2226 len 16384
1856 242/300 Read block 4155 len 8192
1857 243/300 Read block 358 len 8192
1858 244/300 Read block 1607 len 8192
1859 245/300 Read block 4182 len 4096
1860 246/300 Read block 2555 len 16384
1861 247/300 Write block 1949 len 16384 data: 2 2 2 2
1862 248/300 Write block 2853 len 36864 data: 2 2 2 2 2 2 2 2 2
1863 249/300 Read block 2608 len 36864
1864 250/300 Flush
1865 251/300 Read block 1861 len 12288
1866 252/300 Read block 2506 len 20480
1867 253/300 Write block 3659 len 24576 data: 2 2 2 2 2 2
1868 254/300 Read block 4880 len 28672
1869 255/300 Read block 4536 len 28672
1870 256/300 Write block 1172 len 8192 data: 3 3
1871 257/300 Read block 1211 len 8192
1872 258/300 Read block 2516 len 12288
1873 259/300 Read block 2847 len 28672
1874 260/300 Read block 36 len 32768
1875 261/300 Write block 3122 len 20480 data: 2 2 2 2 2
1876 262/300 Write block 812 len 36864 data: 4 4 4 4 3 3 2 2 3
1877 263/300 Read block 758 len 12288
1878 264/300 Flush
1879 265/300 Read block 3511 len 4096
1880 266/300 Write block 4491 len 32768 data: 4 4 4 3 2 2 2 2
1881 267/300 Write block 4419 len 32768 data: 4 4 3 3 3 3 3 3
1882 268/300 Read block 111 len 32768
1883 269/300 Read block 2500 len 20480
1884 270/300 Read block 535 len 40960
1885 271/300 Read block 1270 len 24576
1886 272/300 Read block 2562 len 20480
1887 273/300 Flush
1888 274/300 Read block 123 len 8192
1889 275/300 Flush
1890 276/300 Write block 2832 len 16384 data: 4 4 3 3
1891 277/300 Read block 1294 len 8192
1892 278/300 Write block 2511 len 32768 data: 2 2 2 2 2 2 2 2
1893 279/300 Read block 142 len 28672
1894 280/300 Read block 3746 len 20480
1895 281/300 Read block 3657 len 20480
1896 282/300 Write block 168 len 16384 data: 2 2 2 2
1897 283/300 Read block 2340 len 28672
1898 284/300 Read block 4185 len 24576
1899 285/300 Read block 4986 len 16384
1900 286/300 Read block 4295 len 36864
1901 287/300 Write block 4674 len 32768 data: 2 2 2 2 2 2 3 3
1902 288/300 Flush
1903 289/300 Write block 1018 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1904 290/300 Read block 4361 len 8192
1905 291/300 Read block 2365 len 36864
1906 292/300 Read block 977 len 16384
1907 293/300 Read block 4208 len 40960
1908 294/300 Read block 4850 len 12288
1909 295/300 Flush
1910 296/300 Read block 84 len 20480
1911 297/300 Read block 4618 len 20480
1912 298/300 Read block 2880 len 20480
1913 299/300 Write block 8 len 20480 data: 2 2 2 2 2
1914 300/300 Write block 3467 len 16384 data: 2 2 2 2
1915 Main task has work to do, go find it
1916 got dsc Start(1)
1917 start 1
1918 Sep 22 23:24:12.764 INFO request completed, latency_us: 303, response_code: 204, uri: /start/cid/1, method: POST, req_id: 337abab1-18c2-4812-9c48-182cde2589af, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
1919 Replay: started 1, returned:Ok(())
1920 ----------------------------------------------------------------
1921 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1922 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
1923 605 Acked 1604 Flush 0 Done New Done false
1924 606 Acked 1605 Write 2 Done New Done false
1925 607 Acked 1606 Read 8 Done New Done false
1926 608 Acked 1607 Read 3 Done New Done false
1927 609 Acked 1608 Read 4 Done New Done false
1928 610 Acked 1609 Write 5 Done New Done false
1929 611 Acked 1610 Flush 0 Done New Done false
1930 612 Acked 1611 Write 5 Done New Done false
1931 613 Acked 1612 Read 8 Done New Done false
1932 614 Acked 1613 Write 2 Done New Done false
1933 615 Acked 1614 Flush 0 Done New Done false
1934 616 Acked 1615 Read 7 Done New Done false
1935 617 Acked 1616 Write 9 Done New Done false
1936 618 Acked 1617 Write 8 Done New Done false
1937 619 Acked 1618 Read 3 Done New Done false
1938 620 Acked 1619 Read 6 Done New Done false
1939 621 Acked 1620 Read 7 Done New Done false
1940 622 Acked 1621 Write 9 Done New Done false
1941 623 Acked 1622 Read 10 Done New Done false
1942 624 Acked 1623 Write 3 Done New Done false
1943 625 Acked 1624 Read 2 Done New Done false
1944 626 Acked 1625 Read 1 Done New Done false
1945 627 Acked 1626 Write 9 Done New Done false
1946 628 Acked 1627 Write 3 Done New Done false
1947 629 Acked 1628 Read 3 Done New Done false
1948 630 Acked 1629 Write 10 Done New Done false
1949 631 Acked 1630 Write 2 Done New Done false
1950 632 Acked 1631 Read 2 Done New Done false
1951 633 Acked 1632 Write 10 Done New Done false
1952 634 Acked 1633 Write 3 Done New Done false
1953 635 Acked 1634 Read 8 Done New Done false
1954 636 Acked 1635 Write 4 Done New Done false
1955 637 Acked 1636 Write 4 Done New Done false
1956 638 Acked 1637 Read 1 Done New Done false
1957 639 Acked 1638 Read 4 Done New Done false
1958 640 Acked 1639 Read 4 Done New Done false
1959 641 Acked 1640 Write 4 Done New Done false
1960 642 Acked 1641 Read 10 Done New Done false
1961 643 Acked 1642 Read 1 Done New Done false
1962 644 Acked 1643 Read 4 Done New Done false
1963 645 Acked 1644 Flush 0 Done New Done false
1964 646 Acked 1645 Read 7 Done New Done false
1965 647 Acked 1646 Write 6 Done New Done false
1966 648 Acked 1647 Read 2 Done New Done false
1967 649 Acked 1648 Read 9 Done New Done false
1968 650 Acked 1649 Read 6 Done New Done false
1969 651 Acked 1650 Read 1 Done New Done false
1970 652 Acked 1651 Read 3 Done New Done false
1971 653 Acked 1652 Read 7 Done New Done false
1972 654 Acked 1653 Write 5 Done New Done false
1973 655 Acked 1654 Flush 0 Done New Done false
1974 656 Acked 1655 Write 9 Done New Done false
1975 657 Acked 1656 Read 5 Done New Done false
1976 658 Acked 1657 Write 2 Done New Done false
1977 659 Acked 1658 Flush 0 Done New Done false
1978 660 Acked 1659 Flush 0 Done New Done false
1979 661 Acked 1660 Read 7 Done New Done false
1980 662 Acked 1661 Read 6 Done New Done false
1981 663 Acked 1662 Write 5 Done New Done false
1982 664 Acked 1663 Write 2 Done New Done false
1983 665 Acked 1664 Write 5 Done New Done false
1984 666 Acked 1665 Read 3 Done New Done false
1985 667 Acked 1666 Read 9 Done New Done false
1986 668 Acked 1667 Read 6 Done New Done false
1987 669 Acked 1668 Read 6 Done New Done false
1988 670 Acked 1669 Write 10 Done New Done false
1989 671 Acked 1670 Write 8 Done New Done false
1990 672 Acked 1671 Write 1 Done New Done false
1991 673 Acked 1672 Flush 0 Done New Done false
1992 674 Acked 1673 Read 9 Done New Done false
1993 675 Acked 1674 Write 6 Done New Done false
1994 676 Acked 1675 Write 9 Done New Done false
1995 677 Acked 1676 Read 5 Done New Done false
1996 678 Acked 1677 Read 5 Done New Done false
1997 679 Acked 1678 Read 3 Done New Done false
1998 680 Acked 1679 Write 10 Done New Done false
1999 681 Acked 1680 Read 7 Done New Done false
2000 682 Acked 1681 Read 10 Done New Done false
2001 683 Acked 1682 Read 5 Done New Done false
2002 684 Acked 1683 Write 2 Done New Done false
2003 685 Acked 1684 Flush 0 Done New Done false
2004 686 Acked 1685 Read 7 Done New Done false
2005 687 Acked 1686 Read 8 Done New Done false
2006 688 Acked 1687 Write 10 Done New Done false
2007 689 Acked 1688 Write 1 Done New Done false
2008 690 Acked 1689 Write 2 Done New Done false
2009 691 Acked 1690 Write 4 Done New Done false
2010 692 Acked 1691 Write 6 Done New Done false
2011 693 Acked 1692 Write 10 Done New Done false
2012 694 Acked 1693 Write 6 Done New Done false
2013 695 Acked 1694 Read 4 Done New Done false
2014 696 Acked 1695 Read 8 Done New Done false
2015 697 Acked 1696 Read 2 Done New Done false
2016 698 Acked 1697 Read 6 Done New Done false
2017 699 Acked 1698 Write 9 Done New Done false
2018 700 Acked 1699 Read 5 Done New Done false
2019 701 Acked 1700 Write 3 Done New Done false
2020 702 Acked 1701 Read 8 Done New Done false
2021 703 Acked 1702 Write 8 Done New Done false
2022 704 Acked 1703 Read 3 Done New Done false
2023 705 Acked 1704 Write 3 Done New Done false
2024 706 Acked 1705 Write 10 Done New Done false
2025 707 Acked 1706 Flush 0 Done New Done false
2026 708 Acked 1707 Write 4 Done New Done false
2027 709 Acked 1708 Write 5 Done New Done false
2028 710 Acked 1709 Write 3 Done New Done false
2029 711 Acked 1710 Read 9 Done New Done false
2030 712 Acked 1711 Write 7 Done New Done false
2031 713 Acked 1712 Read 3 Done New Done false
2032 714 Acked 1713 Write 5 Done New Done false
2033 715 Acked 1714 Flush 0 Done New Done false
2034 716 Acked 1715 Write 10 Done New Done false
2035 717 Acked 1716 Read 7 Done New Done false
2036 718 Acked 1717 Flush 0 Done New Done false
2037 719 Acked 1718 Read 8 Done New Done false
2038 720 Acked 1719 Write 4 Done New Done false
2039 721 Acked 1720 Read 4 Done New Done false
2040 722 Acked 1721 Read 8 Done New Done false
2041 723 Acked 1722 Write 6 Done New Done false
2042 724 Acked 1723 Flush 0 Done New Done false
2043 725 Acked 1724 Write 7 Done New Done false
2044 726 Acked 1725 Read 3 Done New Done false
2045 727 Acked 1726 Read 10 Done New Done false
2046 728 Acked 1727 Read 7 Done New Done false
2047 729 Acked 1728 Flush 0 Done New Done false
2048 730 Acked 1729 Flush 0 Done New Done false
2049 731 Acked 1730 Write 9 Done New Done false
2050 732 Acked 1731 Read 7 Done New Done false
2051 733 Acked 1732 Read 7 Done New Done false
2052 734 Acked 1733 Read 7 Done New Done false
2053 735 Acked 1734 Flush 0 Done New Done false
2054 736 Acked 1735 Write 3 Done New Done false
2055 737 Acked 1736 Read 10 Done New Done false
2056 738 Acked 1737 Write 6 Done New Done false
2057 739 Acked 1738 Read 9 Done New Done false
2058 740 Acked 1739 Flush 0 Done New Done false
2059 741 Acked 1740 Read 3 Done New Done false
2060 742 Acked 1741 Write 8 Done New Done false
2061 743 Acked 1742 Read 10 Done New Done false
2062 744 Acked 1743 Flush 0 Done New Done false
2063 745 Acked 1744 Read 4 Done New Done false
2064 746 Acked 1745 Flush 0 Done New Done false
2065 747 Acked 1746 Write 4 Done New Done false
2066 748 Acked 1747 Write 3 Done New Done false
2067 749 Acked 1748 Read 10 Done New Done false
2068 750 Acked 1749 Read 7 Done New Done false
2069 751 Acked 1750 Flush 0 Done New Done false
2070 752 Acked 1751 Read 3 Done New Done false
2071 753 Acked 1752 Flush 0 Done New Done false
2072 754 Acked 1753 Read 9 Done New Done false
2073 755 Acked 1754 Read 1 Done New Done false
2074 756 Acked 1755 Write 10 Done New Done false
2075 757 Acked 1756 Write 10 Done New Done false
2076 758 Acked 1757 Write 8 Done New Done false
2077 759 Acked 1758 Write 2 Done New Done false
2078 760 Acked 1759 Read 9 Done New Done false
2079 761 Acked 1760 Read 9 Done New Done false
2080 762 Acked 1761 Write 5 Done New Done false
2081 763 Acked 1762 Flush 0 Done New Done false
2082 764 Acked 1763 Flush 0 Done New Done false
2083 765 Acked 1764 Read 8 Done New Done false
2084 766 Acked 1765 Read 6 Done New Done false
2085 767 Acked 1766 Read 10 Done New Done false
2086 768 Acked 1767 Write 2 Done New Done false
2087 769 Acked 1768 Write 2 Done New Done false
2088 770 Acked 1769 Write 8 Done New Done false
2089 771 Acked 1770 Write 3 Done New Done false
2090 772 Acked 1771 Read 4 Done New Done false
2091 773 Acked 1772 Write 4 Done New Done false
2092 774 Acked 1773 Read 9 Done New Done false
2093 775 Acked 1774 Write 1 Done New Done false
2094 776 Acked 1775 Flush 0 Done New Done false
2095 777 Acked 1776 Write 8 Done New Done false
2096 778 Acked 1777 Read 1 Done New Done false
2097 779 Acked 1778 Read 4 Done New Done false
2098 780 Acked 1779 Write 4 Done New Done false
2099 781 Acked 1780 Read 6 Done New Done false
2100 782 Acked 1781 Write 3 Done New Done false
2101 783 Acked 1782 Write 8 Done New Done false
2102 784 Acked 1783 Write 10 Done New Done false
2103 785 Acked 1784 Write 6 Done New Done false
2104 786 Acked 1785 Read 4 Done New Done false
2105 787 Acked 1786 Read 3 Done New Done false
2106 788 Acked 1787 Read 7 Done New Done false
2107 789 Acked 1788 Write 3 Done New Done false
2108 790 Acked 1789 Read 6 Done New Done false
2109 791 Acked 1790 Read 5 Done New Done false
2110 792 Acked 1791 Read 8 Done New Done false
2111 793 Acked 1792 Read 3 Done New Done false
2112 794 Acked 1793 Read 4 Done New Done false
2113 795 Acked 1794 Write 8 Done New Done false
2114 796 Acked 1795 Read 9 Done New Done false
2115 797 Acked 1796 Read 2 Done New Done false
2116 798 Acked 1797 Write 1 Done New Done false
2117 799 Acked 1798 Read 10 Done New Done false
2118 800 Acked 1799 Read 5 Done New Done false
2119 801 Acked 1800 Read 4 Done New Done false
2120 802 Acked 1801 Write 5 Done New Done false
2121 803 Acked 1802 Write 4 Done New Done false
2122 804 Acked 1803 Write 2 Done New Done false
2123 805 Acked 1804 Read 5 Done New Done false
2124 806 Acked 1805 Read 5 Done New Done false
2125 807 Acked 1806 Read 1 Done New Done false
2126 808 Acked 1807 Read 3 Done New Done false
2127 809 Acked 1808 Read 2 Done New Done false
2128 810 Acked 1809 Read 2 Done New Done false
2129 811 Acked 1810 Write 2 Done New Done false
2130 812 Acked 1811 Read 7 Done New Done false
2131 813 Acked 1812 Read 2 Done New Done false
2132 814 Acked 1813 Read 10 Done New Done false
2133 815 Acked 1814 Write 5 Done New Done false
2134 816 Acked 1815 Read 3 Done New Done false
2135 817 Acked 1816 Read 10 Done New Done false
2136 818 Acked 1817 Write 7 Done New Done false
2137 819 Acked 1818 Read 2 Done New Done false
2138 820 Acked 1819 Flush 0 Done New Done false
2139 821 Acked 1820 Write 8 Done New Done false
2140 822 Acked 1821 Write 1 Done New Done false
2141 823 Acked 1822 Write 2 Done New Done false
2142 824 Acked 1823 Read 5 Done New Done false
2143 825 Acked 1824 Read 5 Done New Done false
2144 826 Acked 1825 Write 5 Done New Done false
2145 827 Acked 1826 Write 3 Done New Done false
2146 828 Acked 1827 Flush 0 Done New Done false
2147 829 Acked 1828 Read 4 Done New Done false
2148 830 Acked 1829 Write 6 Done New Done false
2149 831 Acked 1830 Read 6 Done New Done false
2150 832 Acked 1831 Read 2 Done New Done false
2151 833 Acked 1832 Read 5 Done New Done false
2152 834 Acked 1833 Read 2 Done New Done false
2153 835 Acked 1834 Flush 0 Done New Done false
2154 836 Acked 1835 Read 8 Done New Done false
2155 837 Acked 1836 Flush 0 Done New Done false
2156 838 Acked 1837 Read 9 Done New Done false
2157 839 Acked 1838 Write 7 Done New Done false
2158 840 Acked 1839 Read 4 Done New Done false
2159 841 Acked 1840 Flush 0 Done New Done false
2160 842 Acked 1841 Write 1 Done New Done false
2161 843 Acked 1842 Flush 0 Done New Done false
2162 844 Acked 1843 Write 9 Done New Done false
2163 845 Acked 1844 Write 1 Done New Done false
2164 846 Acked 1845 Read 4 Done New Done false
2165 847 Acked 1846 Read 2 Done New Done false
2166 848 Acked 1847 Read 2 Done New Done false
2167 849 Acked 1848 Read 2 Done New Done false
2168 850 Acked 1849 Read 1 Done New Done false
2169 851 Acked 1850 Read 4 Done New Done false
2170 852 Acked 1851 Write 4 Done New Done false
2171 853 Acked 1852 Write 9 Done New Done false
2172 854 Acked 1853 Read 9 Done New Done false
2173 855 Acked 1854 Flush 0 Done New Done false
2174 856 Acked 1855 Read 3 Done New Done false
2175 857 Acked 1856 Read 5 Done New Done false
2176 858 Acked 1857 Write 6 Done New Done false
2177 859 Acked 1858 Read 7 Done New Done false
2178 860 Acked 1859 Read 7 Done New Done false
2179 861 Acked 1860 Write 2 Done New Done false
2180 862 Acked 1861 Read 2 Done New Done false
2181 863 Acked 1862 Read 3 Done New Done false
2182 864 Acked 1863 Read 7 Done New Done false
2183 865 Acked 1864 Read 8 Done New Done false
2184 866 Acked 1865 Write 5 Done New Done false
2185 867 Acked 1866 Write 9 Done New Done false
2186 868 Acked 1867 Read 3 Done New Done false
2187 869 Acked 1868 Flush 0 Done New Done false
2188 870 Acked 1869 Read 1 Done New Done false
2189 871 Acked 1870 Write 8 Done New Done false
2190 872 Acked 1871 Write 8 Done New Done false
2191 873 Acked 1872 Read 8 Done New Done false
2192 874 Acked 1873 Read 5 Done New Done false
2193 875 Acked 1874 Read 10 Done New Done false
2194 876 Acked 1875 Read 6 Done New Done false
2195 877 Acked 1876 Read 5 Done New Done false
2196 878 Acked 1877 Flush 0 Done New Done false
2197 879 Acked 1878 Read 2 Done New Done false
2198 880 Acked 1879 Flush 0 Done New Done false
2199 881 Acked 1880 Write 4 Done New Done false
2200 882 Acked 1881 Read 2 Done New Done false
2201 883 Acked 1882 Write 8 Done New Done false
2202 884 Acked 1883 Read 7 Done New Done false
2203 885 Acked 1884 Read 5 Done New Done false
2204 886 Acked 1885 Read 5 Done New Done false
2205 887 Acked 1886 Write 4 Done New Done false
2206 888 Acked 1887 Read 7 Done New Done false
2207 889 Acked 1888 Read 6 Done New Done false
2208 890 Acked 1889 Read 4 Done New Done false
2209 891 Acked 1890 Read 9 Done New Done false
2210 892 Acked 1891 Write 8 Done New Done false
2211 893 Acked 1892 Flush 0 Done New Done false
2212 894 Acked 1893 Write 10 Done New Done false
2213 895 Acked 1894 Read 2 Done New Done false
2214 896 Acked 1895 Read 9 Done New Done false
2215 897 Acked 1896 Read 4 Done New Done false
2216 898 Acked 1897 Read 10 Done New Done false
2217 899 Acked 1898 Read 3 Done New Done false
2218 900 Acked 1899 Flush 0 Done New Done false
2219 901 Acked 1900 Read 5 Done New Done false
2220 902 Acked 1901 Read 5 Done New Done false
2221 903 Acked 1902 Read 5 Done New Done false
2222 904 Acked 1903 Write 5 Done New Done false
2223 905 Acked 1904 Write 4 Sent New Sent false
2224 STATES DS:0 DS:1 DS:2 TOTAL
2225 New 0 301 0 301
2226 Sent 1 0 1 2
2227 Done 300 0 300 600
2228 Skipped 0 0 0 0
2229 Error 0 0 0 0
2230 Last Flush: 1899 1603 1899
2231 Downstairs last five completed: 1603 1602 1601 1600 1599 1598
2232 Upstairs last five completed: 905 904 903 902 901 900
2233 CLIENT: Up:0 ds:301 act:2
2234 ----------------------------------------------------------------
2235 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2236 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
2237 605 Acked 1604 Flush 0 Done New Done false
2238 606 Acked 1605 Write 2 Done New Done false
2239 607 Acked 1606 Read 8 Done New Done false
2240 608 Acked 1607 Read 3 Done New Done false
2241 609 Acked 1608 Read 4 Done New Done false
2242 610 Acked 1609 Write 5 Done New Done false
2243 611 Acked 1610 Flush 0 Done New Done false
2244 612 Acked 1611 Write 5 Done New Done false
2245 613 Acked 1612 Read 8 Done New Done false
2246 614 Acked 1613 Write 2 Done New Done false
2247 615 Acked 1614 Flush 0 Done New Done false
2248 616 Acked 1615 Read 7 Done New Done false
2249 617 Acked 1616 Write 9 Done New Done false
2250 618 Acked 1617 Write 8 Done New Done false
2251 619 Acked 1618 Read 3 Done New Done false
2252 620 Acked 1619 Read 6 Done New Done false
2253 621 Acked 1620 Read 7 Done New Done false
2254 622 Acked 1621 Write 9 Done New Done false
2255 623 Acked 1622 Read 10 Done New Done false
2256 624 Acked 1623 Write 3 Done New Done false
2257 625 Acked 1624 Read 2 Done New Done false
2258 626 Acked 1625 Read 1 Done New Done false
2259 627 Acked 1626 Write 9 Done New Done false
2260 628 Acked 1627 Write 3 Done New Done false
2261 629 Acked 1628 Read 3 Done New Done false
2262 630 Acked 1629 Write 10 Done New Done false
2263 631 Acked 1630 Write 2 Done New Done false
2264 632 Acked 1631 Read 2 Done New Done false
2265 633 Acked 1632 Write 10 Done New Done false
2266 634 Acked 1633 Write 3 Done New Done false
2267 635 Acked 1634 Read 8 Done New Done false
2268 636 Acked 1635 Write 4 Done New Done false
2269 637 Acked 1636 Write 4 Done New Done false
2270 638 Acked 1637 Read 1 Done New Done false
2271 639 Acked 1638 Read 4 Done New Done false
2272 640 Acked 1639 Read 4 Done New Done false
2273 641 Acked 1640 Write 4 Done New Done false
2274 642 Acked 1641 Read 10 Done New Done false
2275 643 Acked 1642 Read 1 Done New Done false
2276 644 Acked 1643 Read 4 Done New Done false
2277 645 Acked 1644 Flush 0 Done New Done false
2278 646 Acked 1645 Read 7 Done New Done false
2279 647 Acked 1646 Write 6 Done New Done false
2280 648 Acked 1647 Read 2 Done New Done false
2281 649 Acked 1648 Read 9 Done New Done false
2282 650 Acked 1649 Read 6 Done New Done false
2283 651 Acked 1650 Read 1 Done New Done false
2284 652 Acked 1651 Read 3 Done New Done false
2285 653 Acked 1652 Read 7 Done New Done false
2286 654 Acked 1653 Write 5 Done New Done false
2287 655 Acked 1654 Flush 0 Done New Done false
2288 656 Acked 1655 Write 9 Done New Done false
2289 657 Acked 1656 Read 5 Done New Done false
2290 658 Acked 1657 Write 2 Done New Done false
2291 659 Acked 1658 Flush 0 Done New Done false
2292 660 Acked 1659 Flush 0 Done New Done false
2293 661 Acked 1660 Read 7 Done New Done false
2294 662 Acked 1661 Read 6 Done New Done false
2295 663 Acked 1662 Write 5 Done New Done false
2296 664 Acked 1663 Write 2 Done New Done false
2297 665 Acked 1664 Write 5 Done New Done false
2298 666 Acked 1665 Read 3 Done New Done false
2299 667 Acked 1666 Read 9 Done New Done false
2300 668 Acked 1667 Read 6 Done New Done false
2301 669 Acked 1668 Read 6 Done New Done false
2302 670 Acked 1669 Write 10 Done New Done false
2303 671 Acked 1670 Write 8 Done New Done false
2304 672 Acked 1671 Write 1 Done New Done false
2305 673 Acked 1672 Flush 0 Done New Done false
2306 674 Acked 1673 Read 9 Done New Done false
2307 675 Acked 1674 Write 6 Done New Done false
2308 676 Acked 1675 Write 9 Done New Done false
2309 677 Acked 1676 Read 5 Done New Done false
2310 678 Acked 1677 Read 5 Done New Done false
2311 679 Acked 1678 Read 3 Done New Done false
2312 680 Acked 1679 Write 10 Done New Done false
2313 681 Acked 1680 Read 7 Done New Done false
2314 682 Acked 1681 Read 10 Done New Done false
2315 683 Acked 1682 Read 5 Done New Done false
2316 684 Acked 1683 Write 2 Done New Done false
2317 685 Acked 1684 Flush 0 Done New Done false
2318 686 Acked 1685 Read 7 Done New Done false
2319 687 Acked 1686 Read 8 Done New Done false
2320 688 Acked 1687 Write 10 Done New Done false
2321 689 Acked 1688 Write 1 Done New Done false
2322 690 Acked 1689 Write 2 Done New Done false
2323 691 Acked 1690 Write 4 Done New Done false
2324 692 Acked 1691 Write 6 Done New Done false
2325 693 Acked 1692 Write 10 Done New Done false
2326 694 Acked 1693 Write 6 Done New Done false
2327 695 Acked 1694 Read 4 Done New Done false
2328 696 Acked 1695 Read 8 Done New Done false
2329 697 Acked 1696 Read 2 Done New Done false
2330 698 Acked 1697 Read 6 Done New Done false
2331 699 Acked 1698 Write 9 Done New Done false
2332 700 Acked 1699 Read 5 Done New Done false
2333 701 Acked 1700 Write 3 Done New Done false
2334 702 Acked 1701 Read 8 Done New Done false
2335 703 Acked 1702 Write 8 Done New Done false
2336 704 Acked 1703 Read 3 Done New Done false
2337 705 Acked 1704 Write 3 Done New Done false
2338 706 Acked 1705 Write 10 Done New Done false
2339 707 Acked 1706 Flush 0 Done New Done false
2340 708 Acked 1707 Write 4 Done New Done false
2341 709 Acked 1708 Write 5 Done New Done false
2342 710 Acked 1709 Write 3 Done New Done false
2343 711 Acked 1710 Read 9 Done New Done false
2344 712 Acked 1711 Write 7 Done New Done false
2345 713 Acked 1712 Read 3 Done New Done false
2346 714 Acked 1713 Write 5 Done New Done false
2347 715 Acked 1714 Flush 0 Done New Done false
2348 716 Acked 1715 Write 10 Done New Done false
2349 717 Acked 1716 Read 7 Done New Done false
2350 718 Acked 1717 Flush 0 Done New Done false
2351 719 Acked 1718 Read 8 Done New Done false
2352 720 Acked 1719 Write 4 Done New Done false
2353 721 Acked 1720 Read 4 Done New Done false
2354 722 Acked 1721 Read 8 Done New Done false
2355 723 Acked 1722 Write 6 Done New Done false
2356 724 Acked 1723 Flush 0 Done New Done false
2357 725 Acked 1724 Write 7 Done New Done false
2358 726 Acked 1725 Read 3 Done New Done false
2359 727 Acked 1726 Read 10 Done New Done false
2360 728 Acked 1727 Read 7 Done New Done false
2361 729 Acked 1728 Flush 0 Done New Done false
2362 730 Acked 1729 Flush 0 Done New Done false
2363 731 Acked 1730 Write 9 Done New Done false
2364 732 Acked 1731 Read 7 Done New Done false
2365 733 Acked 1732 Read 7 Done New Done false
2366 734 Acked 1733 Read 7 Done New Done false
2367 735 Acked 1734 Flush 0 Done New Done false
2368 736 Acked 1735 Write 3 Done New Done false
2369 737 Acked 1736 Read 10 Done New Done false
2370 738 Acked 1737 Write 6 Done New Done false
2371 739 Acked 1738 Read 9 Done New Done false
2372 740 Acked 1739 Flush 0 Done New Done false
2373 741 Acked 1740 Read 3 Done New Done false
2374 742 Acked 1741 Write 8 Done New Done false
2375 743 Acked 1742 Read 10 Done New Done false
2376 744 Acked 1743 Flush 0 Done New Done false
2377 745 Acked 1744 Read 4 Done New Done false
2378 746 Acked 1745 Flush 0 Done New Done false
2379 747 Acked 1746 Write 4 Done New Done false
2380 748 Acked 1747 Write 3 Done New Done false
2381 749 Acked 1748 Read 10 Done New Done false
2382 750 Acked 1749 Read 7 Done New Done false
2383 751 Acked 1750 Flush 0 Done New Done false
2384 752 Acked 1751 Read 3 Done New Done false
2385 753 Acked 1752 Flush 0 Done New Done false
2386 754 Acked 1753 Read 9 Done New Done false
2387 755 Acked 1754 Read 1 Done New Done false
2388 756 Acked 1755 Write 10 Done New Done false
2389 757 Acked 1756 Write 10 Done New Done false
2390 758 Acked 1757 Write 8 Done New Done false
2391 759 Acked 1758 Write 2 Done New Done false
2392 760 Acked 1759 Read 9 Done New Done false
2393 761 Acked 1760 Read 9 Done New Done false
2394 762 Acked 1761 Write 5 Done New Done false
2395 763 Acked 1762 Flush 0 Done New Done false
2396 764 Acked 1763 Flush 0 Done New Done false
2397 765 Acked 1764 Read 8 Done New Done false
2398 766 Acked 1765 Read 6 Done New Done false
2399 767 Acked 1766 Read 10 Done New Done false
2400 768 Acked 1767 Write 2 Done New Done false
2401 769 Acked 1768 Write 2 Done New Done false
2402 770 Acked 1769 Write 8 Done New Done false
2403 771 Acked 1770 Write 3 Done New Done false
2404 772 Acked 1771 Read 4 Done New Done false
2405 773 Acked 1772 Write 4 Done New Done false
2406 774 Acked 1773 Read 9 Done New Done false
2407 775 Acked 1774 Write 1 Done New Done false
2408 776 Acked 1775 Flush 0 Done New Done false
2409 777 Acked 1776 Write 8 Done New Done false
2410 778 Acked 1777 Read 1 Done New Done false
2411 779 Acked 1778 Read 4 Done New Done false
2412 780 Acked 1779 Write 4 Done New Done false
2413 781 Acked 1780 Read 6 Done New Done false
2414 782 Acked 1781 Write 3 Done New Done false
2415 783 Acked 1782 Write 8 Done New Done false
2416 784 Acked 1783 Write 10 Done New Done false
2417 785 Acked 1784 Write 6 Done New Done false
2418 786 Acked 1785 Read 4 Done New Done false
2419 787 Acked 1786 Read 3 Done New Done false
2420 788 Acked 1787 Read 7 Done New Done false
2421 789 Acked 1788 Write 3 Done New Done false
2422 790 Acked 1789 Read 6 Done New Done false
2423 791 Acked 1790 Read 5 Done New Done false
2424 792 Acked 1791 Read 8 Done New Done false
2425 793 Acked 1792 Read 3 Done New Done false
2426 794 Acked 1793 Read 4 Done New Done false
2427 795 Acked 1794 Write 8 Done New Done false
2428 796 Acked 1795 Read 9 Done New Done false
2429 797 Acked 1796 Read 2 Done New Done false
2430 798 Acked 1797 Write 1 Done New Done false
2431 799 Acked 1798 Read 10 Done New Done false
2432 800 Acked 1799 Read 5 Done New Done false
2433 801 Acked 1800 Read 4 Done New Done false
2434 802 Acked 1801 Write 5 Done New Done false
2435 803 Acked 1802 Write 4 Done New Done false
2436 804 Acked 1803 Write 2 Done New Done false
2437 805 Acked 1804 Read 5 Done New Done false
2438 806 Acked 1805 Read 5 Done New Done false
2439 807 Acked 1806 Read 1 Done New Done false
2440 808 Acked 1807 Read 3 Done New Done false
2441 809 Acked 1808 Read 2 Done New Done false
2442 810 Acked 1809 Read 2 Done New Done false
2443 811 Acked 1810 Write 2 Done New Done false
2444 812 Acked 1811 Read 7 Done New Done false
2445 813 Acked 1812 Read 2 Done New Done false
2446 814 Acked 1813 Read 10 Done New Done false
2447 815 Acked 1814 Write 5 Done New Done false
2448 816 Acked 1815 Read 3 Done New Done false
2449 817 Acked 1816 Read 10 Done New Done false
2450 818 Acked 1817 Write 7 Done New Done false
2451 819 Acked 1818 Read 2 Done New Done false
2452 820 Acked 1819 Flush 0 Done New Done false
2453 821 Acked 1820 Write 8 Done New Done false
2454 822 Acked 1821 Write 1 Done New Done false
2455 823 Acked 1822 Write 2 Done New Done false
2456 824 Acked 1823 Read 5 Done New Done false
2457 825 Acked 1824 Read 5 Done New Done false
2458 826 Acked 1825 Write 5 Done New Done false
2459 827 Acked 1826 Write 3 Done New Done false
2460 828 Acked 1827 Flush 0 Done New Done false
2461 829 Acked 1828 Read 4 Done New Done false
2462 830 Acked 1829 Write 6 Done New Done false
2463 831 Acked 1830 Read 6 Done New Done false
2464 832 Acked 1831 Read 2 Done New Done false
2465 833 Acked 1832 Read 5 Done New Done false
2466 834 Acked 1833 Read 2 Done New Done false
2467 835 Acked 1834 Flush 0 Done New Done false
2468 836 Acked 1835 Read 8 Done New Done false
2469 837 Acked 1836 Flush 0 Done New Done false
2470 838 Acked 1837 Read 9 Done New Done false
2471 839 Acked 1838 Write 7 Done New Done false
2472 840 Acked 1839 Read 4 Done New Done false
2473 841 Acked 1840 Flush 0 Done New Done false
2474 842 Acked 1841 Write 1 Done New Done false
2475 843 Acked 1842 Flush 0 Done New Done false
2476 844 Acked 1843 Write 9 Done New Done false
2477 845 Acked 1844 Write 1 Done New Done false
2478 846 Acked 1845 Read 4 Done New Done false
2479 847 Acked 1846 Read 2 Done New Done false
2480 848 Acked 1847 Read 2 Done New Done false
2481 849 Acked 1848 Read 2 Done New Done false
2482 850 Acked 1849 Read 1 Done New Done false
2483 851 Acked 1850 Read 4 Done New Done false
2484 852 Acked 1851 Write 4 Done New Done false
2485 853 Acked 1852 Write 9 Done New Done false
2486 854 Acked 1853 Read 9 Done New Done false
2487 855 Acked 1854 Flush 0 Done New Done false
2488 856 Acked 1855 Read 3 Done New Done false
2489 857 Acked 1856 Read 5 Done New Done false
2490 858 Acked 1857 Write 6 Done New Done false
2491 859 Acked 1858 Read 7 Done New Done false
2492 860 Acked 1859 Read 7 Done New Done false
2493 861 Acked 1860 Write 2 Done New Done false
2494 862 Acked 1861 Read 2 Done New Done false
2495 863 Acked 1862 Read 3 Done New Done false
2496 864 Acked 1863 Read 7 Done New Done false
2497 865 Acked 1864 Read 8 Done New Done false
2498 866 Acked 1865 Write 5 Done New Done false
2499 867 Acked 1866 Write 9 Done New Done false
2500 868 Acked 1867 Read 3 Done New Done false
2501 869 Acked 1868 Flush 0 Done New Done false
2502 870 Acked 1869 Read 1 Done New Done false
2503 871 Acked 1870 Write 8 Done New Done false
2504 872 Acked 1871 Write 8 Done New Done false
2505 873 Acked 1872 Read 8 Done New Done false
2506 874 Acked 1873 Read 5 Done New Done false
2507 875 Acked 1874 Read 10 Done New Done false
2508 876 Acked 1875 Read 6 Done New Done false
2509 877 Acked 1876 Read 5 Done New Done false
2510 878 Acked 1877 Flush 0 Done New Done false
2511 879 Acked 1878 Read 2 Done New Done false
2512 880 Acked 1879 Flush 0 Done New Done false
2513 881 Acked 1880 Write 4 Done New Done false
2514 882 Acked 1881 Read 2 Done New Done false
2515 883 Acked 1882 Write 8 Done New Done false
2516 884 Acked 1883 Read 7 Done New Done false
2517 885 Acked 1884 Read 5 Done New Done false
2518 886 Acked 1885 Read 5 Done New Done false
2519 887 Acked 1886 Write 4 Done New Done false
2520 888 Acked 1887 Read 7 Done New Done false
2521 889 Acked 1888 Read 6 Done New Done false
2522 890 Acked 1889 Read 4 Done New Done false
2523 891 Acked 1890 Read 9 Done New Done false
2524 892 Acked 1891 Write 8 Done New Done false
2525 893 Acked 1892 Flush 0 Done New Done false
2526 894 Acked 1893 Write 10 Done New Done false
2527 895 Acked 1894 Read 2 Done New Done false
2528 896 Acked 1895 Read 9 Done New Done false
2529 897 Acked 1896 Read 4 Done New Done false
2530 898 Acked 1897 Read 10 Done New Done false
2531 899 Acked 1898 Read 3 Done New Done false
2532 900 Acked 1899 Flush 0 Done New Done false
2533 901 Acked 1900 Read 5 Done New Done false
2534 902 Acked 1901 Read 5 Done New Done false
2535 903 Acked 1902 Read 5 Done New Done false
2536 904 Acked 1903 Write 5 Done New Done false
2537 905 Acked 1904 Write 4 Done New Done false
2538 906 Acked 1905 Flush 0 Done New Done false
2539 STATES DS:0 DS:1 DS:2 TOTAL
2540 New 0 302 0 302
2541 Sent 0 0 0 0
2542 Done 302 0 302 604
2543 Skipped 0 0 0 0
2544 Error 0 0 0 0
2545 Last Flush: 1905 1603 1905
2546 Downstairs last five completed: 1603 1602 1601 1600 1599 1598
2547 Upstairs last five completed: 906 905 904 903 902 901
2548 CLIENT: Up:0 ds:302 act:2
2549 [8820] got start action
2550 [8820] I am going to restart
2551 Starting downstairs at port 8820
2552 Make output file at "/tmp/dsc/downstairs-8820.txt"
2553 [8820][1] reports Starting
2554 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1176)
2555 [8820][1] reports Running
25562023-09-22T23:24:18.777ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
25572023-09-22T23:24:18.777ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
25582023-09-22T23:24:18.777ZINFOcrucible: [1] client re-new 302 jobs since flush 1603 = downstairs
25592023-09-22T23:24:18.778ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
25602023-09-22T23:24:18.778ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
25612023-09-22T23:24:18.778ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
25622023-09-22T23:24:18.778ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
25632023-09-22T23:24:18.778ZINFOcrucible: [1] send last flush ID to this DS: 1603
25642023-09-22T23:24:18.778ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
25652023-09-22T23:24:18.779ZINFOcrucible: [1] Replied this last flush ID: 1603
25662023-09-22T23:24:18.779ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
25672023-09-22T23:24:18.779ZINFOcrucible: [1] Transition from Offline to Replay
25682023-09-22T23:24:18.779ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
25692023-09-22T23:24:18.779ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
25702023-09-22T23:24:18.779ZINFOcrucible: [1] Transition from Replay to Active
25712023-09-22T23:24:18.779ZINFOcrucible: [1] Starts cmd_loop
25722023-09-22T23:24:18.779ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
25732023-09-22T23:24:18.779ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
25742023-09-22T23:24:18.779ZINFOcrucible: Set check for repair
25752023-09-22T23:24:19.779ZWARNcrucible: [1] flow control end
25762023-09-22T23:24:19.780ZINFOcrucible: Checking if live repair is needed
25772023-09-22T23:24:19.780ZINFOcrucible: No Live Repair required at this time
2578 ----------------------------------------------------------------
2579 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2580 Downstairs last five completed: 1905 1904 1903 1902 1901 1900
2581 Upstairs last five completed: 906 905 904 903 902 901
2582 CLIENT: Up:0 ds:0 act:3
2583 Replay: All jobs finished, all DS active.
2584 Main task has work to do, go find it
2585 got dsc Stop(2)
2586 stop 2
2587 Sep 22 23:24:20.772 INFO request completed, latency_us: 468, response_code: 204, uri: /stop/cid/2, method: POST, req_id: ca1aa7a3-e8cf-4a8b-88ad-6a7aa3206fe3, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0[8830] Got stop action so:false kr:false
2588 .1:9998
2589 Sep 22 23:24:20.773 INFO request completed, latency_us: 170, response_code: 200, uri: /state/cid/2, method: GET, req_id: a4439d49-b8b2-4989-a96e-d617f81f37ac, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
25902023-09-22T23:24:20.773ZWARNcrucible: [2] downstairs disconnected
25912023-09-22T23:24:20.773ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
25922023-09-22T23:24:20.773ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
25932023-09-22T23:24:20.773ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
25942023-09-22T23:24:20.773ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
25952023-09-22T23:24:20.773ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
25962023-09-22T23:24:20.773ZWARNcrucible: [2] pm_task rx.recv() is None
2597 [8830] Exited with: ExitStatus(unix_wait_status(9))
2598 [8830][2] reports Exit
2599 Sep 22 23:24:24.775 INFO request completed, latency_us: 259, response_code: 200, uri: /state/cid/2, method: GET, req_id: ad2483e6-5887-429d-a012-a562195f0532, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
2600 001/300 Read block 1554 len 28672
2601 002/300 Write block 1474 len 28672 data: 3 3 3 3 3 3 2
2602 003/300 Write block 4082 len 24576 data: 3 3 3 3 2 2
2603 004/300 Write block 4074 len 40960 data: 2 2 2 3 3 3 3 3 4 4
2604 005/300 Read block 4385 len 12288
2605 006/300 Read block 4994 len 8192
2606 007/300 Write block 4556 len 32768 data: 2 2 2 2 2 2 2 2
2607 008/300 Read block 953 len 20480
2608 009/300 Write block 4031 len 32768 data: 2 2 2 3 3 3 3 3
2609 010/300 Write block 298 len 28672 data: 2 2 2 2 2 2 2
2610 011/300 Read block 1873 len 32768
2611 012/300 Write block 2345 len 36864 data: 3 3 3 3 3 3 3 3 3
2612 013/300 Read block 1062 len 40960
2613 014/300 Flush
2614 015/300 Flush
2615 016/300 Read block 2440 len 8192
2616 017/300 Write block 4309 len 28672 data: 3 3 2 2 2 2 2
2617 018/300 Read block 366 len 28672
2618 019/300 Write block 1977 len 4096 data: 2
2619 020/300 Write block 445 len 28672 data: 3 3 2 2 2 2 2
2620 021/300 Read block 3358 len 20480
2621 022/300 Read block 199 len 4096
2622 023/300 Read block 51 len 32768
2623 024/300 Write block 2576 len 40960 data: 4 5 5 4 4 4 2 3 3 3
2624 025/300 Read block 4099 len 20480
2625 026/300 Read block 2274 len 20480
2626 027/300 Read block 2415 len 8192
2627 028/300 Write block 2359 len 32768 data: 2 2 2 2 2 2 2 2
2628 029/300 Read block 627 len 20480
2629 030/300 Write block 2611 len 24576 data: 2 2 2 2 2 2
2630 031/300 Read block 4864 len 24576
2631 032/300 Write block 295 len 28672 data: 2 2 2 3 3 3 3
2632 033/300 Write block 337 len 4096 data: 3
2633 034/300 Read block 2821 len 4096
2634 035/300 Write block 1770 len 28672 data: 2 2 2 2 2 2 2
2635 036/300 Read block 270 len 32768
2636 037/300 Read block 1275 len 40960
2637 038/300 Write block 1347 len 20480 data: 2 2 2 2 2
2638 039/300 Read block 1325 len 12288
2639 040/300 Read block 1598 len 4096
2640 041/300 Write block 1480 len 8192 data: 3 2
2641 042/300 Read block 4142 len 32768
2642 043/300 Read block 3357 len 20480
2643 044/300 Read block 2072 len 28672
2644 045/300 Read block 666 len 8192
2645 046/300 Read block 874 len 4096
2646 047/300 Write block 1167 len 32768 data: 2 2 2 2 3 4 4 2
2647 048/300 Read block 393 len 8192
2648 049/300 Read block 4574 len 12288
2649 050/300 Read block 1899 len 32768
2650 051/300 Flush
2651 052/300 Write block 4320 len 28672 data: 2 2 2 2 2 3 3
2652 053/300 Write block 4245 len 28672 data: 3 3 3 3 3 2 2
2653 054/300 Write block 4258 len 32768 data: 3 3 3 3 3 3 2 2
2654 055/300 Read block 1829 len 28672
2655 056/300 Write block 3385 len 20480 data: 2 3 3 3 3
2656 057/300 Read block 2261 len 20480
2657 058/300 Write block 3912 len 8192 data: 2 2
2658 059/300 Write block 1204 len 4096 data: 3
2659 060/300 Read block 259 len 40960
2660 061/300 Read block 2101 len 40960
2661 062/300 Write block 4853 len 28672 data: 2 2 2 2 2 2 2
2662 063/300 Write block 4348 len 32768 data: 2 2 2 3 3 3 3 3
2663 064/300 Write block 3133 len 16384 data: 2 2 2 2
2664 065/300 Read block 3019 len 24576
2665 066/300 Write block 215 len 8192 data: 2 2
2666 067/300 Write block 1807 len 8192 data: 2 2
2667 068/300 Write block 3022 len 36864 data: 2 2 2 2 2 2 2 2 2
2668 069/300 Read block 2417 len 20480
2669 070/300 Write block 1947 len 12288 data: 2 2 3
2670 071/300 Read block 1749 len 12288
2671 072/300 Write block 3473 len 32768 data: 2 2 2 2 2 2 2 2
2672 073/300 Write block 3181 len 28672 data: 2 2 2 2 2 2 2
2673 074/300 Write block 1617 len 16384 data: 2 2 2 2
2674 075/300 Flush
2675 076/300 Read block 2356 len 32768
2676 077/300 Read block 3100 len 28672
2677 078/300 Read block 551 len 12288
2678 079/300 Read block 4418 len 12288
2679 080/300 Write block 58 len 20480 data: 3 3 3 3 3
2680 081/300 Read block 3756 len 28672
2681 082/300 Write block 3954 len 8192 data: 2 2
2682 083/300 Read block 2961 len 32768
2683 084/300 Write block 3931 len 16384 data: 2 2 2 2
2684 085/300 Write block 4523 len 32768 data: 2 2 2 2 2 2 2 3
2685 086/300 Write block 2278 len 8192 data: 2 2
2686 087/300 Write block 4128 len 28672 data: 3 4 3 3 3 3 2
2687 088/300 Read block 1822 len 16384
2688 089/300 Write block 2905 len 20480 data: 3 3 3 3 3
2689 090/300 Write block 3005 len 20480 data: 2 2 2 2 2
2690 091/300 Write block 3791 len 20480 data: 3 3 3 2 2
2691 092/300 Read block 3094 len 8192
2692 093/300 Read block 4492 len 36864
2693 094/300 Read block 4801 len 16384
2694 095/300 Read block 1237 len 12288
2695 096/300 Flush
2696 097/300 Read block 4017 len 24576
2697 098/300 Write block 1141 len 12288 data: 2 2 2
2698 099/300 Write block 3916 len 32768 data: 2 2 2 2 2 2 2 2
2699 100/300 Read block 2774 len 4096
2700 101/300 Read block 4807 len 16384
2701 102/300 Write block 3153 len 24576 data: 3 3 4 4 3 3
2702 103/300 Read block 4201 len 4096
2703 104/300 Read block 1379 len 8192
2704 105/300 Read block 4718 len 4096
2705 106/300 Read block 4961 len 40960
2706 107/300 Write block 2294 len 40960 data: 2 2 2 2 2 2 3 3 3 2
2707 108/300 Write block 2809 len 36864 data: 2 2 2 2 2 2 2 2 2
2708 109/300 Read block 18 len 16384
2709 110/300 Read block 4285 len 20480
2710 111/300 Write block 2844 len 8192 data: 6 6
2711 112/300 Read block 3187 len 8192
2712 113/300 Read block 4353 len 20480
2713 114/300 Write block 4735 len 12288 data: 2 2 2
2714 115/300 Read block 514 len 32768
2715 116/300 Read block 2287 len 40960
2716 117/300 Read block 1495 len 8192
2717 118/300 Write block 4046 len 20480 data: 2 2 2 2 2
2718 119/300 Read block 1844 len 36864
2719 120/300 Read block 3088 len 32768
2720 121/300 Read block 189 len 16384
2721 122/300 Read block 2336 len 20480
2722 123/300 Flush
2723 124/300 Read block 2363 len 8192
2724 125/300 Write block 2143 len 32768 data: 2 2 2 2 2 2 2 2
2725 126/300 Read block 174 len 8192
2726 127/300 Write block 571 len 20480 data: 3 3 3 3 4
2727 128/300 Write block 199 len 20480 data: 2 2 2 2 2
2728 129/300 Write block 1231 len 4096 data: 2
2729 130/300 Read block 474 len 32768
2730 131/300 Read block 2509 len 12288
2731 132/300 Read block 3624 len 24576
2732 133/300 Read block 1191 len 4096
2733 134/300 Read block 196 len 40960
2734 135/300 Write block 4657 len 32768 data: 2 2 2 2 2 3 2 2
2735 136/300 Flush
2736 137/300 Write block 2794 len 36864 data: 2 2 2 2 2 2 2 2 2
2737 138/300 Read block 2431 len 32768
2738 139/300 Write block 2179 len 4096 data: 2
2739 140/300 Write block 339 len 16384 data: 3 3 2 2
2740 141/300 Flush
2741 142/300 Read block 416 len 20480
2742 143/300 Write block 1629 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2743 144/300 Flush
2744 145/300 Write block 50 len 12288 data: 4 5 5
2745 146/300 Read block 3742 len 32768
2746 147/300 Write block 4227 len 16384 data: 2 2 2 2
2747 148/300 Write block 3335 len 24576 data: 2 2 3 3 3 4
2748 149/300 Read block 4539 len 36864
2749 150/300 Read block 1150 len 20480
2750 151/300 Write block 181 len 36864 data: 3 2 2 2 2 3 3 3 3
2751 152/300 Read block 1622 len 12288
2752 153/300 Read block 2366 len 24576
2753 154/300 Read block 4105 len 4096
2754 155/300 Read block 4873 len 16384
2755 156/300 Write block 222 len 40960 data: 2 2 2 2 2 2 2 3 3 2
2756 157/300 Read block 1108 len 4096
2757 158/300 Write block 3403 len 20480 data: 3 3 3 3 3
2758 159/300 Write block 1568 len 4096 data: 4
2759 160/300 Write block 4950 len 36864 data: 2 2 2 2 3 3 3 2 2
2760 161/300 Read block 4689 len 12288
2761 162/300 Read block 3310 len 8192
2762 163/300 Write block 3146 len 20480 data: 3 3 2 2 2
2763 164/300 Write block 1939 len 20480 data: 2 2 2 2 2
2764 165/300 Read block 1198 len 40960
2765 166/300 Read block 19 len 24576
2766 167/300 Write block 4049 len 20480 data: 3 3 2 2 2
2767 168/300 Read block 4335 len 12288
2768 169/300 Read block 2438 len 28672
2769 170/300 Write block 3867 len 28672 data: 2 2 2 2 2 2 2
2770 171/300 Write block 358 len 16384 data: 4 4 3 2
2771 172/300 Write block 2058 len 40960 data: 3 2 2 2 2 2 2 2 2 2
2772 173/300 Read block 3414 len 4096
2773 174/300 Write block 374 len 20480 data: 2 2 2 2 2
2774 175/300 Write block 2885 len 8192 data: 2 2
2775 176/300 Read block 4374 len 40960
2776 177/300 Flush
2777 178/300 Read block 2106 len 4096
2778 179/300 Write block 3123 len 24576 data: 3 3 3 3 2 2
2779 180/300 Read block 2145 len 16384
2780 181/300 Read block 1211 len 16384
2781 182/300 Read block 1860 len 16384
2782 183/300 Write block 578 len 8192 data: 2 2
2783 184/300 Write block 4086 len 16384 data: 3 3 2 2
2784 185/300 Read block 3957 len 32768
2785 186/300 Read block 4045 len 20480
2786 187/300 Write block 3929 len 16384 data: 2 2 3 3
2787 188/300 Read block 502 len 32768
2788 189/300 Read block 3689 len 20480
2789 190/300 Read block 4234 len 16384
2790 191/300 Read block 1085 len 24576
2791 192/300 Flush
2792 193/300 Read block 2244 len 36864
2793 194/300 Read block 1636 len 4096
2794 195/300 Flush
2795 196/300 Read block 4654 len 12288
2796 197/300 Read block 948 len 36864
2797 198/300 Read block 706 len 8192
2798 199/300 Read block 2430 len 40960
2799 200/300 Flush
2800 201/300 Read block 4743 len 24576
2801 202/300 Flush
2802 203/300 Flush
2803 204/300 Read block 505 len 20480
2804 205/300 Read block 3609 len 36864
2805 206/300 Read block 3132 len 28672
2806 207/300 Write block 2783 len 32768 data: 3 3 2 2 2 2 2 2
2807 208/300 Flush
2808 209/300 Read block 4611 len 20480
2809 210/300 Read block 4197 len 16384
2810 211/300 Read block 4089 len 28672
2811 212/300 Write block 1194 len 16384 data: 2 2 2 2
2812 213/300 Flush
2813 214/300 Write block 3310 len 20480 data: 2 2 2 2 2
2814 215/300 Read block 151 len 12288
2815 216/300 Write block 1624 len 36864 data: 2 2 2 2 2 3 3 3 3
2816 217/300 Read block 4807 len 12288
2817 218/300 Read block 3418 len 20480
2818 219/300 Flush
2819 220/300 Write block 4056 len 28672 data: 2 2 2 2 3 3 3
2820 221/300 Flush
2821 222/300 Write block 1345 len 36864 data: 3 3 3 3 3 3 3 2 2
2822 223/300 Read block 1731 len 36864
2823 224/300 Read block 2556 len 8192
2824 225/300 Read block 3966 len 8192
2825 226/300 Read block 1432 len 40960
2826 227/300 Read block 4626 len 4096
2827 228/300 Write block 4321 len 20480 data: 3 3 3 3 4
2828 229/300 Read block 3948 len 8192
2829 230/300 Write block 2437 len 32768 data: 2 2 2 2 2 2 2 2
2830 231/300 Write block 2412 len 16384 data: 2 2 2 2
2831 232/300 Write block 106 len 20480 data: 2 2 2 3 3
2832 233/300 Read block 4664 len 8192
2833 234/300 Read block 998 len 28672
2834 235/300 Read block 381 len 16384
2835 236/300 Write block 1871 len 20480 data: 2 2 2 2 2
2836 237/300 Read block 1530 len 20480
2837 238/300 Read block 639 len 24576
2838 239/300 Read block 3895 len 28672
2839 240/300 Read block 4687 len 40960
2840 241/300 Write block 3769 len 28672 data: 2 2 2 2 2 2 2
2841 242/300 Flush
2842 243/300 Write block 986 len 4096 data: 2
2843 244/300 Write block 4712 len 20480 data: 2 2 2 2 2
2844 245/300 Write block 4556 len 36864 data: 3 3 3 3 3 3 3 3 2
2845 246/300 Write block 4210 len 28672 data: 2 2 2 3 3 3 3
2846 247/300 Read block 880 len 28672
2847 248/300 Write block 4621 len 12288 data: 2 2 2
2848 249/300 Read block 1661 len 20480
2849 250/300 Flush
2850 251/300 Flush
2851 252/300 Read block 2999 len 28672
2852 253/300 Read block 227 len 28672
2853 254/300 Write block 3685 len 32768 data: 2 2 2 2 2 2 2 2
2854 255/300 Write block 4912 len 16384 data: 3 3 3 3
2855 256/300 Write block 2758 len 4096 data: 2
2856 257/300 Write block 4656 len 20480 data: 2 3 3 3 3
2857 258/300 Write block 4816 len 36864 data: 2 2 2 4 4 3 3 3 3
2858 259/300 Read block 3796 len 8192
2859 260/300 Read block 3354 len 16384
2860 261/300 Write block 2064 len 36864 data: 3 3 3 3 2 2 2 2 2
2861 262/300 Read block 3395 len 12288
2862 263/300 Write block 24 len 40960 data: 3 2 2 2 2 2 2 2 2 2
2863 264/300 Write block 1447 len 20480 data: 2 2 2 2 2
2864 265/300 Write block 3398 len 28672 data: 3 3 3 3 3 4 4
2865 266/300 Read block 3957 len 36864
2866 267/300 Read block 4963 len 36864
2867 268/300 Write block 1554 len 32768 data: 3 3 3 3 2 2 2 2
2868 269/300 Write block 4247 len 16384 data: 4 4 4 3
2869 270/300 Read block 2741 len 36864
2870 271/300 Write block 1818 len 8192 data: 2 2
2871 272/300 Read block 1559 len 12288
2872 273/300 Read block 4686 len 28672
2873 274/300 Read block 2091 len 40960
2874 275/300 Write block 1600 len 24576 data: 2 2 2 2 2 2
2875 276/300 Read block 1946 len 40960
2876 277/300 Write block 3850 len 24576 data: 3 3 3 3 2 2
2877 278/300 Read block 1930 len 28672
2878 279/300 Write block 726 len 12288 data: 5 5 5
2879 280/300 Read block 3753 len 36864
2880 281/300 Write block 4046 len 28672 data: 3 3 3 4 4 3 3
2881 282/300 Write block 2763 len 24576 data: 2 2 2 2 2 2
2882 283/300 Write block 2785 len 32768 data: 3 3 3 3 3 3 2 2
2883 284/300 Write block 2820 len 24576 data: 2 2 2 2 2 2
2884 285/300 Write block 3292 len 24576 data: 2 2 2 2 2 2
2885 286/300 Read block 1025 len 16384
2886 287/300 Write block 116 len 8192 data: 4 4
2887 288/300 Flush
2888 289/300 Read block 532 len 32768
2889 290/300 Read block 4904 len 28672
2890 291/300 Write block 1389 len 8192 data: 2 2
2891 292/300 Write block 831 len 40960 data: 2 2 3 3 4 4 4 4 4 4
2892 293/300 Read block 4285 len 20480
2893 294/300 Write block 3290 len 4096 data: 2
2894 295/300 Read block 3043 len 24576
2895 296/300 Read block 2343 len 40960
2896 297/300 Read block 4801 len 36864
2897 298/300 Write block 4291 len 16384 data: 3 3 3 3
2898 299/300 Read block 2612 len 8192
2899 300/300 Write block 1668 len 16384 data: 2 2 2 2
2900 Main task has work to do, go find it
2901 got dsc Start(2)
2902 start 2
2903 Sep 22 23:24:25.710 INFO request completed, latency_us: 274, response_code: 204, uri: /start/cid/2, method: POST, req_id: d1609e80-21a0-4307-94a4-51074a1851bd, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
2904 Replay: started 2, returned:Ok(())
2905 ----------------------------------------------------------------
2906 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2907 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
2908 907 Acked 1906 Read 7 Done Done New false
2909 908 Acked 1907 Write 7 Done Done New false
2910 909 Acked 1908 Write 6 Done Done New false
2911 910 Acked 1909 Write 10 Done Done New false
2912 911 Acked 1910 Read 3 Done Done New false
2913 912 Acked 1911 Read 2 Done Done New false
2914 913 Acked 1912 Write 8 Done Done New false
2915 914 Acked 1913 Read 5 Done Done New false
2916 915 Acked 1914 Write 8 Done Done New false
2917 916 Acked 1915 Write 7 Done Done New false
2918 917 Acked 1916 Read 8 Done Done New false
2919 918 Acked 1917 Flush 0 Done Done New false
2920 919 Acked 1918 Write 9 Done Done New false
2921 920 Acked 1919 Read 10 Done Done New false
2922 921 Acked 1920 Flush 0 Done Done New false
2923 922 Acked 1921 Flush 0 Done Done New false
2924 923 Acked 1922 Read 2 Done Done New false
2925 924 Acked 1923 Write 7 Done Done New false
2926 925 Acked 1924 Read 7 Done Done New false
2927 926 Acked 1925 Write 1 Done Done New false
2928 927 Acked 1926 Write 7 Done Done New false
2929 928 Acked 1927 Read 5 Done Done New false
2930 929 Acked 1928 Read 1 Done Done New false
2931 930 Acked 1929 Read 8 Done Done New false
2932 931 Acked 1930 Write 10 Done Done New false
2933 932 Acked 1931 Read 5 Done Done New false
2934 933 Acked 1932 Read 5 Done Done New false
2935 934 Acked 1933 Read 2 Done Done New false
2936 935 Acked 1934 Write 8 Done Done New false
2937 936 Acked 1935 Read 5 Done Done New false
2938 937 Acked 1936 Write 6 Done Done New false
2939 938 Acked 1937 Read 6 Done Done New false
2940 939 Acked 1938 Write 7 Done Done New false
2941 940 Acked 1939 Write 1 Done Done New false
2942 941 Acked 1940 Read 1 Done Done New false
2943 942 Acked 1941 Write 7 Done Done New false
2944 943 Acked 1942 Read 8 Done Done New false
2945 944 Acked 1943 Read 10 Done Done New false
2946 945 Acked 1944 Write 5 Done Done New false
2947 946 Acked 1945 Read 3 Done Done New false
2948 947 Acked 1946 Read 1 Done Done New false
2949 948 Acked 1947 Write 2 Done Done New false
2950 949 Acked 1948 Read 8 Done Done New false
2951 950 Acked 1949 Read 5 Done Done New false
2952 951 Acked 1950 Read 7 Done Done New false
2953 952 Acked 1951 Read 2 Done Done New false
2954 953 Acked 1952 Read 1 Done Done New false
2955 954 Acked 1953 Write 8 Done Done New false
2956 955 Acked 1954 Read 2 Done Done New false
2957 956 Acked 1955 Read 3 Done Done New false
2958 957 Acked 1956 Read 8 Done Done New false
2959 958 Acked 1957 Flush 0 Done Done New false
2960 959 Acked 1958 Write 7 Done Done New false
2961 960 Acked 1959 Write 7 Done Done New false
2962 961 Acked 1960 Write 8 Done Done New false
2963 962 Acked 1961 Read 7 Done Done New false
2964 963 Acked 1962 Write 5 Done Done New false
2965 964 Acked 1963 Read 5 Done Done New false
2966 965 Acked 1964 Write 2 Done Done New false
2967 966 Acked 1965 Write 1 Done Done New false
2968 967 Acked 1966 Read 10 Done Done New false
2969 968 Acked 1967 Read 10 Done Done New false
2970 969 Acked 1968 Write 7 Done Done New false
2971 970 Acked 1969 Write 8 Done Done New false
2972 971 Acked 1970 Write 4 Done Done New false
2973 972 Acked 1971 Read 6 Done Done New false
2974 973 Acked 1972 Write 2 Done Done New false
2975 974 Acked 1973 Write 2 Done Done New false
2976 975 Acked 1974 Write 9 Done Done New false
2977 976 Acked 1975 Read 5 Done Done New false
2978 977 Acked 1976 Write 3 Done Done New false
2979 978 Acked 1977 Read 3 Done Done New false
2980 979 Acked 1978 Write 8 Done Done New false
2981 980 Acked 1979 Write 7 Done Done New false
2982 981 Acked 1980 Write 4 Done Done New false
2983 982 Acked 1981 Flush 0 Done Done New false
2984 983 Acked 1982 Read 8 Done Done New false
2985 984 Acked 1983 Read 7 Done Done New false
2986 985 Acked 1984 Read 3 Done Done New false
2987 986 Acked 1985 Read 3 Done Done New false
2988 987 Acked 1986 Write 5 Done Done New false
2989 988 Acked 1987 Read 7 Done Done New false
2990 989 Acked 1988 Write 2 Done Done New false
2991 990 Acked 1989 Read 8 Done Done New false
2992 991 Acked 1990 Write 4 Done Done New false
2993 992 Acked 1991 Write 8 Done Done New false
2994 993 Acked 1992 Write 2 Done Done New false
2995 994 Acked 1993 Write 7 Done Done New false
2996 995 Acked 1994 Read 4 Done Done New false
2997 996 Acked 1995 Write 5 Done Done New false
2998 997 Acked 1996 Write 5 Done Done New false
2999 998 Acked 1997 Write 5 Done Done New false
3000 999 Acked 1998 Read 2 Done Done New false
3001 1000 Acked 1999 Read 9 Done Done New false
3002 1001 Acked 2000 Read 4 Done Done New false
3003 1002 Acked 2001 Read 3 Done Done New false
3004 1003 Acked 2002 Flush 0 Done Done New false
3005 1004 Acked 2003 Read 6 Done Done New false
3006 1005 Acked 2004 Write 3 Done Done New false
3007 1006 Acked 2005 Write 8 Done Done New false
3008 1007 Acked 2006 Read 1 Done Done New false
3009 1008 Acked 2007 Read 4 Done Done New false
3010 1009 Acked 2008 Write 6 Done Done New false
3011 1010 Acked 2009 Read 1 Done Done New false
3012 1011 Acked 2010 Read 2 Done Done New false
3013 1012 Acked 2011 Read 1 Done Done New false
3014 1013 Acked 2012 Read 10 Done Done New false
3015 1014 Acked 2013 Write 10 Done Done New false
3016 1015 Acked 2014 Write 9 Done Done New false
3017 1016 Acked 2015 Read 4 Done Done New false
3018 1017 Acked 2016 Read 5 Done Done New false
3019 1018 Acked 2017 Write 2 Done Done New false
3020 1019 Acked 2018 Read 2 Done Done New false
3021 1020 Acked 2019 Read 5 Done Done New false
3022 1021 Acked 2020 Write 3 Done Done New false
3023 1022 Acked 2021 Read 8 Done Done New false
3024 1023 Acked 2022 Read 10 Done Done New false
3025 1024 Acked 2023 Read 2 Done Done New false
3026 1025 Acked 2024 Write 5 Done Done New false
3027 1026 Acked 2025 Read 9 Done Done New false
3028 1027 Acked 2026 Read 8 Done Done New false
3029 1028 Acked 2027 Read 4 Done Done New false
3030 1029 Acked 2028 Read 5 Done Done New false
3031 1030 Acked 2029 Flush 0 Done Done New false
3032 1031 Acked 2030 Read 2 Done Done New false
3033 1032 Acked 2031 Write 8 Done Done New false
3034 1033 Acked 2032 Read 2 Done Done New false
3035 1034 Acked 2033 Write 5 Done Done New false
3036 1035 Acked 2034 Write 5 Done Done New false
3037 1036 Acked 2035 Write 1 Done Done New false
3038 1037 Acked 2036 Read 8 Done Done New false
3039 1038 Acked 2037 Read 3 Done Done New false
3040 1039 Acked 2038 Read 6 Done Done New false
3041 1040 Acked 2039 Read 1 Done Done New false
3042 1041 Acked 2040 Read 10 Done Done New false
3043 1042 Acked 2041 Write 8 Done Done New false
3044 1043 Acked 2042 Flush 0 Done Done New false
3045 1044 Acked 2043 Write 9 Done Done New false
3046 1045 Acked 2044 Read 8 Done Done New false
3047 1046 Acked 2045 Write 1 Done Done New false
3048 1047 Acked 2046 Write 4 Done Done New false
3049 1048 Acked 2047 Flush 0 Done Done New false
3050 1049 Acked 2048 Read 5 Done Done New false
3051 1050 Acked 2049 Write 10 Done Done New false
3052 1051 Acked 2050 Flush 0 Done Done New false
3053 1052 Acked 2051 Write 3 Done Done New false
3054 1053 Acked 2052 Read 8 Done Done New false
3055 1054 Acked 2053 Write 4 Done Done New false
3056 1055 Acked 2054 Write 6 Done Done New false
3057 1056 Acked 2055 Read 9 Done Done New false
3058 1057 Acked 2056 Read 5 Done Done New false
3059 1058 Acked 2057 Write 9 Done Done New false
3060 1059 Acked 2058 Read 3 Done Done New false
3061 1060 Acked 2059 Read 6 Done Done New false
3062 1061 Acked 2060 Read 1 Done Done New false
3063 1062 Acked 2061 Read 4 Done Done New false
3064 1063 Acked 2062 Write 10 Done Done New false
3065 1064 Acked 2063 Read 1 Done Done New false
3066 1065 Acked 2064 Write 5 Done Done New false
3067 1066 Acked 2065 Write 1 Done Done New false
3068 1067 Acked 2066 Write 9 Done Done New false
3069 1068 Acked 2067 Read 3 Done Done New false
3070 1069 Acked 2068 Read 2 Done Done New false
3071 1070 Acked 2069 Write 5 Done Done New false
3072 1071 Acked 2070 Write 5 Done Done New false
3073 1072 Acked 2071 Read 10 Done Done New false
3074 1073 Acked 2072 Read 6 Done Done New false
3075 1074 Acked 2073 Write 5 Done Done New false
3076 1075 Acked 2074 Read 3 Done Done New false
3077 1076 Acked 2075 Read 7 Done Done New false
3078 1077 Acked 2076 Write 7 Done Done New false
3079 1078 Acked 2077 Write 4 Done Done New false
3080 1079 Acked 2078 Write 10 Done Done New false
3081 1080 Acked 2079 Read 1 Done Done New false
3082 1081 Acked 2080 Write 5 Done Done New false
3083 1082 Acked 2081 Write 2 Done Done New false
3084 1083 Acked 2082 Read 10 Done Done New false
3085 1084 Acked 2083 Flush 0 Done Done New false
3086 1085 Acked 2084 Flush 0 Done Done New false
3087 1086 Acked 2085 Read 1 Done Done New false
3088 1087 Acked 2086 Write 6 Done Done New false
3089 1088 Acked 2087 Read 4 Done Done New false
3090 1089 Acked 2088 Read 4 Done Done New false
3091 1090 Acked 2089 Read 4 Done Done New false
3092 1091 Acked 2090 Write 2 Done Done New false
3093 1092 Acked 2091 Write 4 Done Done New false
3094 1093 Acked 2092 Read 8 Done Done New false
3095 1094 Acked 2093 Read 5 Done Done New false
3096 1095 Acked 2094 Write 4 Done Done New false
3097 1096 Acked 2095 Read 8 Done Done New false
3098 1097 Acked 2096 Read 5 Done Done New false
3099 1098 Acked 2097 Read 4 Done Done New false
3100 1099 Acked 2098 Read 6 Done Done New false
3101 1100 Acked 2099 Flush 0 Done Done New false
3102 1101 Acked 2100 Read 9 Done Done New false
3103 1102 Acked 2101 Read 1 Done Done New false
3104 1103 Acked 2102 Flush 0 Done Done New false
3105 1104 Acked 2103 Read 3 Done Done New false
3106 1105 Acked 2104 Read 9 Done Done New false
3107 1106 Acked 2105 Read 2 Done Done New false
3108 1107 Acked 2106 Read 10 Done Done New false
3109 1108 Acked 2107 Flush 0 Done Done New false
3110 1109 Acked 2108 Read 6 Done Done New false
3111 1110 Acked 2109 Flush 0 Done Done New false
3112 1111 Acked 2110 Flush 0 Done Done New false
3113 1112 Acked 2111 Read 5 Done Done New false
3114 1113 Acked 2112 Read 9 Done Done New false
3115 1114 Acked 2113 Read 7 Done Done New false
3116 1115 Acked 2114 Write 8 Done Done New false
3117 1116 Acked 2115 Flush 0 Done Done New false
3118 1117 Acked 2116 Read 5 Done Done New false
3119 1118 Acked 2117 Read 4 Done Done New false
3120 1119 Acked 2118 Read 7 Done Done New false
3121 1120 Acked 2119 Write 4 Done Done New false
3122 1121 Acked 2120 Flush 0 Done Done New false
3123 1122 Acked 2121 Write 5 Done Done New false
3124 1123 Acked 2122 Read 3 Done Done New false
3125 1124 Acked 2123 Write 9 Done Done New false
3126 1125 Acked 2124 Read 3 Done Done New false
3127 1126 Acked 2125 Read 5 Done Done New false
3128 1127 Acked 2126 Flush 0 Done Done New false
3129 1128 Acked 2127 Write 7 Done Done New false
3130 1129 Acked 2128 Flush 0 Done Done New false
3131 1130 Acked 2129 Write 9 Done Done New false
3132 1131 Acked 2130 Read 9 Done Done New false
3133 1132 Acked 2131 Read 2 Done Done New false
3134 1133 Acked 2132 Read 2 Done Done New false
3135 1134 Acked 2133 Read 10 Done Done New false
3136 1135 Acked 2134 Read 1 Done Done New false
3137 1136 Acked 2135 Write 5 Done Done New false
3138 1137 Acked 2136 Read 2 Done Done New false
3139 1138 Acked 2137 Write 8 Done Done New false
3140 1139 Acked 2138 Write 4 Done Done New false
3141 1140 Acked 2139 Write 5 Done Done New false
3142 1141 Acked 2140 Read 2 Done Done New false
3143 1142 Acked 2141 Read 7 Done Done New false
3144 1143 Acked 2142 Read 4 Done Done New false
3145 1144 Acked 2143 Write 5 Done Done New false
3146 1145 Acked 2144 Read 5 Done Done New false
3147 1146 Acked 2145 Read 6 Done Done New false
3148 1147 Acked 2146 Read 7 Done Done New false
3149 1148 Acked 2147 Read 10 Done Done New false
3150 1149 Acked 2148 Write 7 Done Done New false
3151 1150 Acked 2149 Flush 0 Done Done New false
3152 1151 Acked 2150 Write 1 Done Done New false
3153 1152 Acked 2151 Write 5 Done Done New false
3154 1153 Acked 2152 Write 9 Done Done New false
3155 1154 Acked 2153 Write 7 Done Done New false
3156 1155 Acked 2154 Read 7 Done Done New false
3157 1156 Acked 2155 Write 3 Done Done New false
3158 1157 Acked 2156 Read 5 Done Done New false
3159 1158 Acked 2157 Flush 0 Done Done New false
3160 1159 Acked 2158 Flush 0 Done Done New false
3161 1160 Acked 2159 Read 7 Done Done New false
3162 1161 Acked 2160 Read 7 Done Done New false
3163 1162 Acked 2161 Write 8 Done Done New false
3164 1163 Acked 2162 Write 4 Done Done New false
3165 1164 Acked 2163 Write 1 Done Done New false
3166 1165 Acked 2164 Write 5 Done Done New false
3167 1166 Acked 2165 Write 9 Done Done New false
3168 1167 Acked 2166 Read 2 Done Done New false
3169 1168 Acked 2167 Read 4 Done Done New false
3170 1169 Acked 2168 Write 9 Done Done New false
3171 1170 Acked 2169 Read 3 Done Done New false
3172 1171 Acked 2170 Write 10 Done Done New false
3173 1172 Acked 2171 Write 5 Done Done New false
3174 1173 Acked 2172 Write 7 Done Done New false
3175 1174 Acked 2173 Read 9 Done Done New false
3176 1175 Acked 2174 Read 9 Done Done New false
3177 1176 Acked 2175 Write 8 Done Done New false
3178 1177 Acked 2176 Write 4 Done Done New false
3179 1178 Acked 2177 Read 9 Done Done New false
3180 1179 Acked 2178 Write 2 Done Done New false
3181 1180 Acked 2179 Read 3 Done Done New false
3182 1181 Acked 2180 Read 7 Done Done New false
3183 1182 Acked 2181 Read 10 Done Done New false
3184 1183 Acked 2182 Write 6 Done Done New false
3185 1184 Acked 2183 Read 10 Done Done New false
3186 1185 Acked 2184 Write 6 Done Done New false
3187 1186 Acked 2185 Read 7 Done Done New false
3188 1187 Acked 2186 Write 3 Done Done New false
3189 1188 Acked 2187 Read 9 Done Done New false
3190 1189 Acked 2188 Write 7 Done Done New false
3191 1190 Acked 2189 Write 6 Done Done New false
3192 1191 Acked 2190 Write 8 Done Done New false
3193 1192 Acked 2191 Write 6 Done Done New false
3194 1193 Acked 2192 Write 6 Done Done New false
3195 1194 Acked 2193 Read 4 Done Done New false
3196 1195 Acked 2194 Write 2 Done Done New false
3197 1196 Acked 2195 Flush 0 Done Done New false
3198 1197 Acked 2196 Read 8 Done Done New false
3199 1198 Acked 2197 Read 7 Done Done New false
3200 1199 Acked 2198 Write 2 Done Done New false
3201 1200 Acked 2199 Write 10 Done Done New false
3202 1201 Acked 2200 Read 5 Done Done New false
3203 1202 Acked 2201 Write 1 Done Done New false
3204 1203 Acked 2202 Read 6 Done Done New false
3205 1204 Acked 2203 Read 10 Done Done New false
3206 1205 Acked 2204 Read 9 Done Done New false
3207 1206 Acked 2205 Write 4 Done Done New false
3208 1207 Acked 2206 Read 2 Done Done New false
3209 1208 Acked 2207 Write 4 Sent Sent New false
3210 STATES DS:0 DS:1 DS:2 TOTAL
3211 New 0 0 302 302
3212 Sent 1 1 0 2
3213 Done 301 301 0 602
3214 Skipped 0 0 0 0
3215 Error 0 0 0 0
3216 Last Flush: 2195 2195 1905
3217 Downstairs last five completed: 1905 1904 1903 1902 1901 1900
3218 Upstairs last five completed: 1208 1207 1206 1205 1204 1203
3219 CLIENT: Up:0 ds:302 act:2
3220 [8830] got start action
3221 [8830] I am going to restart
3222 Starting downstairs at port 8830
3223 Make output file at "/tmp/dsc/downstairs-8830.txt"
3224 [8830][2] reports Starting
3225 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1177)
3226 [8830][2] reports Running
32272023-09-22T23:24:27.783ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
32282023-09-22T23:24:27.783ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
32292023-09-22T23:24:27.783ZINFOcrucible: [2] client re-new 303 jobs since flush 1905 = downstairs
32302023-09-22T23:24:27.784ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
32312023-09-22T23:24:27.784ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
32322023-09-22T23:24:27.784ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
32332023-09-22T23:24:27.784ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
32342023-09-22T23:24:27.784ZINFOcrucible: [2] send last flush ID to this DS: 1905
32352023-09-22T23:24:27.784ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
32362023-09-22T23:24:27.785ZINFOcrucible: [2] Replied this last flush ID: 1905
32372023-09-22T23:24:27.785ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
32382023-09-22T23:24:27.785ZINFOcrucible: [2] Transition from Offline to Replay
32392023-09-22T23:24:27.785ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
32402023-09-22T23:24:27.785ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
32412023-09-22T23:24:27.785ZINFOcrucible: [2] Transition from Replay to Active
32422023-09-22T23:24:27.785ZINFOcrucible: [2] Starts cmd_loop
32432023-09-22T23:24:27.785ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
32442023-09-22T23:24:27.785ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
32452023-09-22T23:24:27.785ZINFOcrucible: Set check for repair
32462023-09-22T23:24:28.785ZWARNcrucible: [2] flow control end
32472023-09-22T23:24:28.786ZINFOcrucible: Checking if live repair is needed
32482023-09-22T23:24:28.786ZINFOcrucible: No Live Repair required at this time
3249 ----------------------------------------------------------------
3250 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3251 Downstairs last five completed: 2208 2207 2206 2205 2204 2203
3252 Upstairs last five completed: 1209 1208 1207 1206 1205 1204
3253 CLIENT: Up:0 ds:0 act:3
3254 Replay: All jobs finished, all DS active.
3255 Main task has work to do, go find it
3256 got dsc Stop(1)
3257 stop 1
3258 Sep 22 23:24:29.715 INFO request completed, latency_us: 511, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 9ca9cc18-6520-4f3d-99e9-a7f56842c62f, remote_addr: 127.0.0.1:52471[8820] Got stop action so:false kr:false
3259 , local_addr: 127.0.0.1:9998
3260 Sep 22 23:24:29.716 INFO request completed, latency_us: 284, response_code: 200, uri: /state/cid/1, method: GET, req_id: 71ab1d59-3acd-4209-a908-bb7594fd4fc1, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
32612023-09-22T23:24:29.716ZWARNcrucible: [1] downstairs disconnected
32622023-09-22T23:24:29.716ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
32632023-09-22T23:24:29.716ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
32642023-09-22T23:24:29.716ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
32652023-09-22T23:24:29.716ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
32662023-09-22T23:24:29.716ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
32672023-09-22T23:24:29.716ZWARNcrucible: [1] pm_task rx.recv() is None
3268 [8820] Exited with: ExitStatus(unix_wait_status(9))
3269 [8820][1] reports Exit
32702023-09-22T23:24:31.718ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
3271 Sep 22 23:24:33.719 INFO request completed, latency_us: 297, response_code: 200, uri: /state/cid/1, method: GET, req_id: c0e4d5ea-25da-4b29-ba67-f71036aac231, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
3272 001/300 Write block 3228 len 36864 data: 2 2 2 2 2 2 2 2 2
3273 002/300 Read block 2441 len 24576
3274 003/300 Read block 1779 len 16384
3275 004/300 Read block 453 len 40960
3276 005/300 Read block 2250 len 32768
3277 006/300 Write block 671 len 8192 data: 2 2
3278 007/300 Read block 2189 len 24576
3279 008/300 Read block 2452 len 4096
3280 009/300 Write block 1468 len 32768 data: 2 2 2 2 2 2 4 4
3281 010/300 Write block 1764 len 36864 data: 2 2 2 2 2 2 3 3 3
3282 011/300 Read block 823 len 32768
3283 012/300 Write block 4228 len 36864 data: 3 3 3 2 2 2 2 2 2
3284 013/300 Write block 655 len 36864 data: 2 2 2 2 2 2 2 2 2
3285 014/300 Read block 3416 len 4096
3286 015/300 Read block 2937 len 40960
3287 016/300 Write block 1683 len 32768 data: 4 4 3 3 3 2 2 2
3288 017/300 Write block 4820 len 40960 data: 5 4 4 4 4 3 3 3 3 2
3289 018/300 Read block 3468 len 36864
3290 019/300 Read block 953 len 32768
3291 020/300 Flush
3292 021/300 Read block 3202 len 4096
3293 022/300 Flush
3294 023/300 Read block 481 len 12288
3295 024/300 Write block 2713 len 32768 data: 2 2 2 2 2 3 3 3
3296 025/300 Read block 4019 len 32768
3297 026/300 Write block 2310 len 12288 data: 2 2 3
3298 027/300 Read block 538 len 40960
3299 028/300 Write block 3960 len 24576 data: 2 2 2 3 3 3
3300 029/300 Write block 2347 len 28672 data: 4 4 4 4 4 4 4
3301 030/300 Read block 2003 len 28672
3302 031/300 Write block 2200 len 20480 data: 2 2 2 2 2
3303 032/300 Flush
3304 033/300 Write block 4294 len 36864 data: 4 3 2 2 2 2 2 2 2
3305 034/300 Write block 4245 len 32768 data: 4 4 5 5 5 4 3 2
3306 035/300 Read block 3716 len 20480
3307 036/300 Write block 591 len 36864 data: 2 2 2 2 3 3 3 3 3
3308 037/300 Read block 1291 len 20480
3309 038/300 Write block 2664 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3310 039/300 Read block 647 len 28672
3311 040/300 Read block 1242 len 4096
3312 041/300 Read block 4773 len 32768
3313 042/300 Read block 3931 len 32768
3314 043/300 Write block 2251 len 16384 data: 3 3 2 2
3315 044/300 Write block 2651 len 12288 data: 2 2 2
3316 045/300 Write block 2596 len 24576 data: 3 3 3 3 2 2
3317 046/300 Read block 2634 len 36864
3318 047/300 Write block 843 len 40960 data: 3 3 2 2 2 2 2 2 3 3
3319 048/300 Read block 4581 len 16384
3320 049/300 Write block 1238 len 32768 data: 2 2 2 2 2 2 2 2
3321 050/300 Write block 3249 len 16384 data: 2 2 3 3
3322 051/300 Read block 4848 len 12288
3323 052/300 Read block 966 len 32768
3324 053/300 Write block 3710 len 36864 data: 3 3 2 2 2 3 3 3 4
3325 054/300 Write block 3772 len 12288 data: 3 3 3
3326 055/300 Read block 889 len 36864
3327 056/300 Read block 1359 len 32768
3328 057/300 Write block 2847 len 36864 data: 3 3 3 3 3 2 3 3 3
3329 058/300 Read block 3567 len 40960
3330 059/300 Write block 68 len 36864 data: 2 2 2 2 2 2 2 2 2
3331 060/300 Read block 3965 len 24576
3332 061/300 Read block 1554 len 24576
3333 062/300 Flush
3334 063/300 Write block 609 len 4096 data: 2
3335 064/300 Write block 792 len 40960 data: 3 3 3 2 2 2 2 2 2 2
3336 065/300 Read block 242 len 32768
3337 066/300 Read block 4297 len 8192
3338 067/300 Read block 715 len 24576
3339 068/300 Read block 1710 len 16384
3340 069/300 Read block 2418 len 32768
3341 070/300 Write block 3478 len 20480 data: 3 3 3 2 2
3342 071/300 Write block 2684 len 32768 data: 2 2 2 2 2 2 2 2
3343 072/300 Read block 1800 len 32768
3344 073/300 Write block 4405 len 20480 data: 4 4 4 4 4
3345 074/300 Read block 3680 len 4096
3346 075/300 Write block 2283 len 16384 data: 2 2 2 2
3347 076/300 Read block 1144 len 24576
3348 077/300 Flush
3349 078/300 Read block 2034 len 32768
3350 079/300 Write block 4837 len 28672 data: 2 2 2 2 2 2 2
3351 080/300 Read block 4001 len 20480
3352 081/300 Write block 675 len 8192 data: 2 2
3353 082/300 Read block 2593 len 24576
3354 083/300 Read block 3397 len 4096
3355 084/300 Write block 3888 len 24576 data: 2 2 2 2 2 2
3356 085/300 Read block 4048 len 32768
3357 086/300 Write block 2725 len 24576 data: 2 3 3 3 3 3
3358 087/300 Flush
3359 088/300 Write block 4818 len 16384 data: 3 5 6 5
3360 089/300 Flush
3361 090/300 Write block 59 len 24576 data: 4 4 4 4 3 3
3362 091/300 Write block 1984 len 12288 data: 3 3 3
3363 092/300 Read block 830 len 20480
3364 093/300 Read block 423 len 8192
3365 094/300 Flush
3366 095/300 Write block 2551 len 32768 data: 3 3 2 2 2 2 2 2
3367 096/300 Read block 4794 len 4096
3368 097/300 Flush
3369 098/300 Read block 4668 len 28672
3370 099/300 Write block 3988 len 40960 data: 4 3 3 3 3 3 4 4 4 5
3371 100/300 Read block 169 len 28672
3372 101/300 Write block 934 len 28672 data: 3 3 3 3 2 2 2
3373 102/300 Read block 2073 len 4096
3374 103/300 Write block 3497 len 32768 data: 2 2 2 2 2 2 2 2
3375 104/300 Read block 4954 len 16384
3376 105/300 Write block 925 len 24576 data: 3 2 2 2 3 3
3377 106/300 Flush
3378 107/300 Read block 1022 len 4096
3379 108/300 Flush
3380 109/300 Read block 575 len 4096
3381 110/300 Write block 1951 len 28672 data: 3 3 2 2 3 3 3
3382 111/300 Read block 4022 len 32768
3383 112/300 Read block 4035 len 12288
3384 113/300 Read block 3514 len 24576
3385 114/300 Flush
3386 115/300 Read block 3107 len 20480
3387 116/300 Read block 2832 len 8192
3388 117/300 Read block 55 len 28672
3389 118/300 Write block 3227 len 8192 data: 2 3
3390 119/300 Read block 2685 len 40960
3391 120/300 Read block 1386 len 32768
3392 121/300 Read block 33 len 28672
3393 122/300 Flush
3394 123/300 Read block 2907 len 28672
3395 124/300 Read block 964 len 32768
3396 125/300 Read block 814 len 12288
3397 126/300 Flush
3398 127/300 Write block 1205 len 40960 data: 3 3 3 3 2 2 2 2 2 2
3399 128/300 Read block 3910 len 16384
3400 129/300 Read block 2186 len 28672
3401 130/300 Flush
3402 131/300 Read block 4108 len 32768
3403 132/300 Write block 2622 len 20480 data: 2 2 2 2 2
3404 133/300 Write block 1696 len 28672 data: 2 2 2 2 2 2 2
3405 134/300 Write block 2271 len 36864 data: 3 3 3 3 3 3 3 3 3
3406 135/300 Read block 1773 len 32768
3407 136/300 Write block 1579 len 8192 data: 2 2
3408 137/300 Read block 1044 len 12288
3409 138/300 Read block 745 len 36864
3410 139/300 Read block 2834 len 36864
3411 140/300 Read block 2631 len 28672
3412 141/300 Write block 4757 len 32768 data: 2 2 2 2 2 2 2 2
3413 142/300 Write block 3144 len 16384 data: 3 3 4 4
3414 143/300 Write block 1159 len 16384 data: 3 3 3 3
3415 144/300 Write block 2294 len 28672 data: 3 3 3 3 3 3 4
3416 145/300 Write block 353 len 8192 data: 2 3
3417 146/300 Read block 4442 len 28672
3418 147/300 Read block 783 len 24576
3419 148/300 Write block 4180 len 24576 data: 2 2 2 2 2 2
3420 149/300 Read block 1929 len 32768
3421 150/300 Write block 4454 len 32768 data: 2 2 2 2 2 2 2 2
3422 151/300 Read block 2405 len 40960
3423 152/300 Write block 713 len 24576 data: 2 2 2 2 2 2
3424 153/300 Read block 3757 len 20480
3425 154/300 Read block 3491 len 24576
3426 155/300 Read block 4739 len 32768
3427 156/300 Read block 2859 len 4096
3428 157/300 Write block 4602 len 12288 data: 4 4 4
3429 158/300 Read block 419 len 16384
3430 159/300 Read block 1325 len 36864
3431 160/300 Flush
3432 161/300 Read block 2636 len 32768
3433 162/300 Write block 385 len 36864 data: 3 3 3 2 2 2 2 2 2
3434 163/300 Write block 3664 len 8192 data: 3 2
3435 164/300 Write block 3634 len 8192 data: 2 2
3436 165/300 Read block 4018 len 24576
3437 166/300 Flush
3438 167/300 Write block 4004 len 28672 data: 4 2 2 2 2 2 2
3439 168/300 Flush
3440 169/300 Read block 2726 len 20480
3441 170/300 Read block 4172 len 40960
3442 171/300 Read block 2289 len 12288
3443 172/300 Read block 24 len 20480
3444 173/300 Write block 2621 len 20480 data: 2 3 3 3 3
3445 174/300 Write block 1859 len 36864 data: 3 3 3 2 2 2 2 2 2
3446 175/300 Write block 1312 len 24576 data: 2 2 2 2 2 2
3447 176/300 Read block 463 len 28672
3448 177/300 Write block 2373 len 40960 data: 3 3 3 3 3 2 2 2 3 3
3449 178/300 Write block 441 len 20480 data: 2 2 2 3 4
3450 179/300 Read block 330 len 4096
3451 180/300 Write block 3508 len 36864 data: 3 3 3 3 3 3 3 2 2
3452 181/300 Read block 3389 len 32768
3453 182/300 Read block 1916 len 24576
3454 183/300 Write block 597 len 8192 data: 4 4
3455 184/300 Read block 1887 len 36864
3456 185/300 Write block 4891 len 4096 data: 3
3457 186/300 Write block 2444 len 36864 data: 3 2 2 2 2 3 3 3 3
3458 187/300 Write block 4053 len 28672 data: 3 2 2 3 3 3 3
3459 188/300 Write block 3869 len 28672 data: 3 3 3 3 3 2 2
3460 189/300 Read block 664 len 16384
3461 190/300 Write block 468 len 32768 data: 2 2 3 3 3 3 2 2
3462 191/300 Read block 46 len 12288
3463 192/300 Read block 301 len 8192
3464 193/300 Write block 2968 len 36864 data: 2 2 2 2 2 2 2 2 2
3465 194/300 Write block 4604 len 8192 data: 5 4
3466 195/300 Write block 1117 len 8192 data: 2 3
3467 196/300 Read block 685 len 16384
3468 197/300 Flush
3469 198/300 Write block 2405 len 20480 data: 2 2 2 2 2
3470 199/300 Write block 2924 len 8192 data: 2 2
3471 200/300 Write block 3583 len 12288 data: 2 2 2
3472 201/300 Write block 418 len 28672 data: 2 2 2 2 2 3 3
3473 202/300 Flush
3474 203/300 Write block 246 len 4096 data: 3
3475 204/300 Write block 4209 len 20480 data: 2 3 3 3 4
3476 205/300 Read block 1083 len 36864
3477 206/300 Read block 627 len 36864
3478 207/300 Write block 4779 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3479 208/300 Write block 1632 len 4096 data: 4
3480 209/300 Write block 4328 len 24576 data: 2 3 3 3 3 3
3481 210/300 Write block 3141 len 20480 data: 3 3 4 4 4
3482 211/300 Read block 4817 len 20480
3483 212/300 Read block 1704 len 24576
3484 213/300 Write block 4549 len 8192 data: 2 2
3485 214/300 Write block 3964 len 20480 data: 4 4 3 3 3
3486 215/300 Flush
3487 216/300 Read block 485 len 32768
3488 217/300 Read block 3703 len 8192
3489 218/300 Write block 2776 len 16384 data: 2 2 2 2
3490 219/300 Read block 3149 len 8192
3491 220/300 Read block 3860 len 24576
3492 221/300 Read block 3367 len 4096
3493 222/300 Write block 884 len 36864 data: 2 2 2 2 2 3 3 3 3
3494 223/300 Write block 2028 len 20480 data: 3 3 3 3 2
3495 224/300 Read block 700 len 32768
3496 225/300 Read block 1534 len 32768
3497 226/300 Read block 814 len 28672
3498 227/300 Read block 2260 len 24576
3499 228/300 Read block 3898 len 12288
3500 229/300 Write block 3804 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3501 230/300 Write block 4964 len 8192 data: 2 2
3502 231/300 Write block 4311 len 20480 data: 3 3 3 3 3
3503 232/300 Write block 73 len 36864 data: 3 3 3 3 2 2 2 3 3
3504 233/300 Read block 1589 len 12288
3505 234/300 Read block 1241 len 28672
3506 235/300 Write block 1267 len 32768 data: 2 2 3 3 3 3 3 3
3507 236/300 Write block 3648 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3508 237/300 Read block 2998 len 24576
3509 238/300 Read block 4881 len 12288
3510 239/300 Write block 2472 len 8192 data: 2 2
3511 240/300 Flush
3512 241/300 Write block 2212 len 4096 data: 2
3513 242/300 Flush
3514 243/300 Read block 2719 len 28672
3515 244/300 Read block 1466 len 40960
3516 245/300 Read block 4782 len 24576
3517 246/300 Read block 3858 len 24576
3518 247/300 Flush
3519 248/300 Write block 4956 len 16384 data: 4 3 3 2
3520 249/300 Read block 863 len 4096
3521 250/300 Read block 3250 len 20480
3522 251/300 Write block 1382 len 16384 data: 2 2 2 2
3523 252/300 Read block 329 len 24576
3524 253/300 Read block 4685 len 8192
3525 254/300 Flush
3526 255/300 Read block 574 len 28672
3527 256/300 Write block 4517 len 20480 data: 2 2 2 2 2
3528 257/300 Write block 1918 len 12288 data: 2 2 2
3529 258/300 Flush
3530 259/300 Read block 4201 len 40960
3531 260/300 Write block 1877 len 16384 data: 2 2 2 2
3532 261/300 Write block 1327 len 24576 data: 3 3 3 3 3 4
3533 262/300 Write block 2877 len 16384 data: 3 3 3 3
3534 263/300 Read block 4134 len 28672
3535 264/300 Read block 3224 len 36864
3536 265/300 Read block 866 len 20480
3537 266/300 Read block 2248 len 4096
3538 267/300 Read block 3973 len 40960
3539 268/300 Write block 3078 len 36864 data: 2 2 2 3 3 3 3 3 3
3540 269/300 Read block 1180 len 28672
3541 270/300 Write block 4744 len 20480 data: 4 3 3 4 3
3542 271/300 Read block 373 len 32768
3543 272/300 Write block 2511 len 32768 data: 3 3 3 3 3 3 3 3
3544 273/300 Write block 1475 len 16384 data: 5 4 4 4
3545 274/300 Write block 43 len 12288 data: 3 3 3
3546 275/300 Read block 1760 len 4096
3547 276/300 Write block 3589 len 32768 data: 3 4 4 3 3 3 3 2
3548 277/300 Write block 1405 len 4096 data: 2
3549 278/300 Write block 3325 len 20480 data: 2 2 2 2 2
3550 279/300 Read block 3611 len 40960
3551 280/300 Write block 1263 len 24576 data: 2 2 2 2 3 3
3552 281/300 Write block 2806 len 40960 data: 2 2 2 3 3 3 3 3 3 3
3553 282/300 Write block 561 len 8192 data: 2 2
3554 283/300 Read block 2634 len 12288
3555 284/300 Write block 3314 len 4096 data: 3
3556 285/300 Flush
3557 286/300 Read block 3375 len 40960
3558 287/300 Read block 4034 len 32768
3559 288/300 Read block 2735 len 40960
3560 289/300 Write block 934 len 32768 data: 4 4 4 4 3 3 3 2
3561 290/300 Write block 3897 len 12288 data: 2 3 3
3562 291/300 Write block 1910 len 40960 data: 3 3 2 2 2 2 2 2 3 3
3563 292/300 Flush
3564 293/300 Read block 1699 len 32768
3565 294/300 Write block 2371 len 20480 data: 3 3 4 4 4
3566 295/300 Read block 1772 len 8192
3567 296/300 Read block 3096 len 24576
3568 297/300 Write block 3611 len 24576 data: 3 3 3 3 3 2
3569 298/300 Read block 2903 len 32768
3570 299/300 Read block 2369 len 16384
3571 300/300 Read block 1437 len 24576
3572 Main task has work to do, go find it
3573 got dsc Start(1)
3574 start 1
3575 Sep 22 23:24:34.689 INFO request completed, latency_us: 306, response_code: 204, uri: /start/cid/1, method: POST, req_id: 8ac84982-7d8f-43c7-b206-65c3ed213e4e, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
3576 Replay: started 1, returned:Ok(())
3577 ----------------------------------------------------------------
3578 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3579 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
3580 1210 Acked 2209 Write 9 Done New Done false
3581 1211 Acked 2210 Read 6 Done New Done false
3582 1212 Acked 2211 Read 4 Done New Done false
3583 1213 Acked 2212 Read 10 Done New Done false
3584 1214 Acked 2213 Read 8 Done New Done false
3585 1215 Acked 2214 Write 2 Done New Done false
3586 1216 Acked 2215 Read 6 Done New Done false
3587 1217 Acked 2216 Read 1 Done New Done false
3588 1218 Acked 2217 Write 8 Done New Done false
3589 1219 Acked 2218 Write 9 Done New Done false
3590 1220 Acked 2219 Read 8 Done New Done false
3591 1221 Acked 2220 Write 9 Done New Done false
3592 1222 Acked 2221 Write 9 Done New Done false
3593 1223 Acked 2222 Read 1 Done New Done false
3594 1224 Acked 2223 Read 10 Done New Done false
3595 1225 Acked 2224 Write 8 Done New Done false
3596 1226 Acked 2225 Write 10 Done New Done false
3597 1227 Acked 2226 Read 9 Done New Done false
3598 1228 Acked 2227 Read 8 Done New Done false
3599 1229 Acked 2228 Flush 0 Done New Done false
3600 1230 Acked 2229 Read 1 Done New Done false
3601 1231 Acked 2230 Flush 0 Done New Done false
3602 1232 Acked 2231 Read 3 Done New Done false
3603 1233 Acked 2232 Write 8 Done New Done false
3604 1234 Acked 2233 Read 8 Done New Done false
3605 1235 Acked 2234 Write 3 Done New Done false
3606 1236 Acked 2235 Read 10 Done New Done false
3607 1237 Acked 2236 Write 6 Done New Done false
3608 1238 Acked 2237 Write 7 Done New Done false
3609 1239 Acked 2238 Read 7 Done New Done false
3610 1240 Acked 2239 Write 5 Done New Done false
3611 1241 Acked 2240 Flush 0 Done New Done false
3612 1242 Acked 2241 Write 9 Done New Done false
3613 1243 Acked 2242 Write 8 Done New Done false
3614 1244 Acked 2243 Read 5 Done New Done false
3615 1245 Acked 2244 Flush 0 Done New Done false
3616 1246 Acked 2245 Write 9 Done New Done false
3617 1247 Acked 2246 Read 5 Done New Done false
3618 1248 Acked 2247 Write 10 Done New Done false
3619 1249 Acked 2248 Read 7 Done New Done false
3620 1250 Acked 2249 Read 1 Done New Done false
3621 1251 Acked 2250 Read 8 Done New Done false
3622 1252 Acked 2251 Read 8 Done New Done false
3623 1253 Acked 2252 Write 4 Done New Done false
3624 1254 Acked 2253 Write 3 Done New Done false
3625 1255 Acked 2254 Write 6 Done New Done false
3626 1256 Acked 2255 Read 9 Done New Done false
3627 1257 Acked 2256 Write 10 Done New Done false
3628 1258 Acked 2257 Read 4 Done New Done false
3629 1259 Acked 2258 Write 8 Done New Done false
3630 1260 Acked 2259 Write 4 Done New Done false
3631 1261 Acked 2260 Read 3 Done New Done false
3632 1262 Acked 2261 Read 8 Done New Done false
3633 1263 Acked 2262 Write 9 Done New Done false
3634 1264 Acked 2263 Write 3 Done New Done false
3635 1265 Acked 2264 Read 9 Done New Done false
3636 1266 Acked 2265 Read 8 Done New Done false
3637 1267 Acked 2266 Write 9 Done New Done false
3638 1268 Acked 2267 Read 10 Done New Done false
3639 1269 Acked 2268 Write 9 Done New Done false
3640 1270 Acked 2269 Read 6 Done New Done false
3641 1271 Acked 2270 Read 6 Done New Done false
3642 1272 Acked 2271 Flush 0 Done New Done false
3643 1273 Acked 2272 Write 1 Done New Done false
3644 1274 Acked 2273 Write 10 Done New Done false
3645 1275 Acked 2274 Read 8 Done New Done false
3646 1276 Acked 2275 Read 2 Done New Done false
3647 1277 Acked 2276 Read 6 Done New Done false
3648 1278 Acked 2277 Read 4 Done New Done false
3649 1279 Acked 2278 Read 8 Done New Done false
3650 1280 Acked 2279 Write 5 Done New Done false
3651 1281 Acked 2280 Write 8 Done New Done false
3652 1282 Acked 2281 Read 8 Done New Done false
3653 1283 Acked 2282 Write 5 Done New Done false
3654 1284 Acked 2283 Read 1 Done New Done false
3655 1285 Acked 2284 Write 4 Done New Done false
3656 1286 Acked 2285 Read 6 Done New Done false
3657 1287 Acked 2286 Flush 0 Done New Done false
3658 1288 Acked 2287 Read 8 Done New Done false
3659 1289 Acked 2288 Write 7 Done New Done false
3660 1290 Acked 2289 Read 5 Done New Done false
3661 1291 Acked 2290 Write 2 Done New Done false
3662 1292 Acked 2291 Read 6 Done New Done false
3663 1293 Acked 2292 Read 1 Done New Done false
3664 1294 Acked 2293 Write 6 Done New Done false
3665 1295 Acked 2294 Read 8 Done New Done false
3666 1296 Acked 2295 Write 6 Done New Done false
3667 1297 Acked 2296 Flush 0 Done New Done false
3668 1298 Acked 2297 Write 4 Done New Done false
3669 1299 Acked 2298 Flush 0 Done New Done false
3670 1300 Acked 2299 Write 6 Done New Done false
3671 1301 Acked 2300 Write 3 Done New Done false
3672 1302 Acked 2301 Read 5 Done New Done false
3673 1303 Acked 2302 Read 2 Done New Done false
3674 1304 Acked 2303 Flush 0 Done New Done false
3675 1305 Acked 2304 Write 8 Done New Done false
3676 1306 Acked 2305 Read 1 Done New Done false
3677 1307 Acked 2306 Flush 0 Done New Done false
3678 1308 Acked 2307 Read 7 Done New Done false
3679 1309 Acked 2308 Write 10 Done New Done false
3680 1310 Acked 2309 Read 7 Done New Done false
3681 1311 Acked 2310 Write 7 Done New Done false
3682 1312 Acked 2311 Read 1 Done New Done false
3683 1313 Acked 2312 Write 8 Done New Done false
3684 1314 Acked 2313 Read 4 Done New Done false
3685 1315 Acked 2314 Write 6 Done New Done false
3686 1316 Acked 2315 Flush 0 Done New Done false
3687 1317 Acked 2316 Read 1 Done New Done false
3688 1318 Acked 2317 Flush 0 Done New Done false
3689 1319 Acked 2318 Read 1 Done New Done false
3690 1320 Acked 2319 Write 7 Done New Done false
3691 1321 Acked 2320 Read 8 Done New Done false
3692 1322 Acked 2321 Read 3 Done New Done false
3693 1323 Acked 2322 Read 6 Done New Done false
3694 1324 Acked 2323 Flush 0 Done New Done false
3695 1325 Acked 2324 Read 5 Done New Done false
3696 1326 Acked 2325 Read 2 Done New Done false
3697 1327 Acked 2326 Read 7 Done New Done false
3698 1328 Acked 2327 Write 2 Done New Done false
3699 1329 Acked 2328 Read 10 Done New Done false
3700 1330 Acked 2329 Read 8 Done New Done false
3701 1331 Acked 2330 Read 7 Done New Done false
3702 1332 Acked 2331 Flush 0 Done New Done false
3703 1333 Acked 2332 Read 7 Done New Done false
3704 1334 Acked 2333 Read 8 Done New Done false
3705 1335 Acked 2334 Read 3 Done New Done false
3706 1336 Acked 2335 Flush 0 Done New Done false
3707 1337 Acked 2336 Write 10 Done New Done false
3708 1338 Acked 2337 Read 4 Done New Done false
3709 1339 Acked 2338 Read 7 Done New Done false
3710 1340 Acked 2339 Flush 0 Done New Done false
3711 1341 Acked 2340 Read 8 Done New Done false
3712 1342 Acked 2341 Write 5 Done New Done false
3713 1343 Acked 2342 Write 7 Done New Done false
3714 1344 Acked 2343 Write 9 Done New Done false
3715 1345 Acked 2344 Read 8 Done New Done false
3716 1346 Acked 2345 Write 2 Done New Done false
3717 1347 Acked 2346 Read 3 Done New Done false
3718 1348 Acked 2347 Read 9 Done New Done false
3719 1349 Acked 2348 Read 9 Done New Done false
3720 1350 Acked 2349 Read 7 Done New Done false
3721 1351 Acked 2350 Write 8 Done New Done false
3722 1352 Acked 2351 Write 4 Done New Done false
3723 1353 Acked 2352 Write 4 Done New Done false
3724 1354 Acked 2353 Write 7 Done New Done false
3725 1355 Acked 2354 Write 2 Done New Done false
3726 1356 Acked 2355 Read 7 Done New Done false
3727 1357 Acked 2356 Read 6 Done New Done false
3728 1358 Acked 2357 Write 6 Done New Done false
3729 1359 Acked 2358 Read 8 Done New Done false
3730 1360 Acked 2359 Write 8 Done New Done false
3731 1361 Acked 2360 Read 10 Done New Done false
3732 1362 Acked 2361 Write 6 Done New Done false
3733 1363 Acked 2362 Read 5 Done New Done false
3734 1364 Acked 2363 Read 6 Done New Done false
3735 1365 Acked 2364 Read 8 Done New Done false
3736 1366 Acked 2365 Read 1 Done New Done false
3737 1367 Acked 2366 Write 3 Done New Done false
3738 1368 Acked 2367 Read 4 Done New Done false
3739 1369 Acked 2368 Read 9 Done New Done false
3740 1370 Acked 2369 Flush 0 Done New Done false
3741 1371 Acked 2370 Read 8 Done New Done false
3742 1372 Acked 2371 Write 9 Done New Done false
3743 1373 Acked 2372 Write 2 Done New Done false
3744 1374 Acked 2373 Write 2 Done New Done false
3745 1375 Acked 2374 Read 6 Done New Done false
3746 1376 Acked 2375 Flush 0 Done New Done false
3747 1377 Acked 2376 Write 7 Done New Done false
3748 1378 Acked 2377 Flush 0 Done New Done false
3749 1379 Acked 2378 Read 5 Done New Done false
3750 1380 Acked 2379 Read 10 Done New Done false
3751 1381 Acked 2380 Read 3 Done New Done false
3752 1382 Acked 2381 Read 5 Done New Done false
3753 1383 Acked 2382 Write 5 Done New Done false
3754 1384 Acked 2383 Write 9 Done New Done false
3755 1385 Acked 2384 Write 6 Done New Done false
3756 1386 Acked 2385 Read 7 Done New Done false
3757 1387 Acked 2386 Write 10 Done New Done false
3758 1388 Acked 2387 Write 5 Done New Done false
3759 1389 Acked 2388 Read 1 Done New Done false
3760 1390 Acked 2389 Write 9 Done New Done false
3761 1391 Acked 2390 Read 8 Done New Done false
3762 1392 Acked 2391 Read 6 Done New Done false
3763 1393 Acked 2392 Write 2 Done New Done false
3764 1394 Acked 2393 Read 9 Done New Done false
3765 1395 Acked 2394 Write 1 Done New Done false
3766 1396 Acked 2395 Write 9 Done New Done false
3767 1397 Acked 2396 Write 7 Done New Done false
3768 1398 Acked 2397 Write 7 Done New Done false
3769 1399 Acked 2398 Read 4 Done New Done false
3770 1400 Acked 2399 Flush 0 Done New Done false
3771 1401 Acked 2400 Write 8 Done New Done false
3772 1402 Acked 2401 Read 3 Done New Done false
3773 1403 Acked 2402 Read 2 Done New Done false
3774 1404 Acked 2403 Write 9 Done New Done false
3775 1405 Acked 2404 Write 2 Done New Done false
3776 1406 Acked 2405 Write 2 Done New Done false
3777 1407 Acked 2406 Read 4 Done New Done false
3778 1408 Acked 2407 Flush 0 Done New Done false
3779 1409 Acked 2408 Write 5 Done New Done false
3780 1410 Acked 2409 Write 2 Done New Done false
3781 1411 Acked 2410 Write 3 Done New Done false
3782 1412 Acked 2411 Write 7 Done New Done false
3783 1413 Acked 2412 Flush 0 Done New Done false
3784 1414 Acked 2413 Write 1 Done New Done false
3785 1415 Acked 2414 Write 5 Done New Done false
3786 1416 Acked 2415 Read 9 Done New Done false
3787 1417 Acked 2416 Read 9 Done New Done false
3788 1418 Acked 2417 Write 10 Done New Done false
3789 1419 Acked 2418 Write 1 Done New Done false
3790 1420 Acked 2419 Write 6 Done New Done false
3791 1421 Acked 2420 Write 5 Done New Done false
3792 1422 Acked 2421 Read 5 Done New Done false
3793 1423 Acked 2422 Read 6 Done New Done false
3794 1424 Acked 2423 Write 2 Done New Done false
3795 1425 Acked 2424 Write 5 Done New Done false
3796 1426 Acked 2425 Flush 0 Done New Done false
3797 1427 Acked 2426 Read 8 Done New Done false
3798 1428 Acked 2427 Read 2 Done New Done false
3799 1429 Acked 2428 Write 4 Done New Done false
3800 1430 Acked 2429 Read 2 Done New Done false
3801 1431 Acked 2430 Read 6 Done New Done false
3802 1432 Acked 2431 Read 1 Done New Done false
3803 1433 Acked 2432 Write 9 Done New Done false
3804 1434 Acked 2433 Write 5 Done New Done false
3805 1435 Acked 2434 Read 8 Done New Done false
3806 1436 Acked 2435 Read 8 Done New Done false
3807 1437 Acked 2436 Read 7 Done New Done false
3808 1438 Acked 2437 Read 6 Done New Done false
3809 1439 Acked 2438 Read 3 Done New Done false
3810 1440 Acked 2439 Write 10 Done New Done false
3811 1441 Acked 2440 Write 2 Done New Done false
3812 1442 Acked 2441 Write 5 Done New Done false
3813 1443 Acked 2442 Write 9 Done New Done false
3814 1444 Acked 2443 Read 3 Done New Done false
3815 1445 Acked 2444 Read 7 Done New Done false
3816 1446 Acked 2445 Write 8 Done New Done false
3817 1447 Acked 2446 Write 10 Done New Done false
3818 1448 Acked 2447 Read 6 Done New Done false
3819 1449 Acked 2448 Read 3 Done New Done false
3820 1450 Acked 2449 Write 2 Done New Done false
3821 1451 Acked 2450 Flush 0 Done New Done false
3822 1452 Acked 2451 Write 1 Done New Done false
3823 1453 Acked 2452 Flush 0 Done New Done false
3824 1454 Acked 2453 Read 7 Done New Done false
3825 1455 Acked 2454 Read 10 Done New Done false
3826 1456 Acked 2455 Read 6 Done New Done false
3827 1457 Acked 2456 Read 6 Done New Done false
3828 1458 Acked 2457 Flush 0 Done New Done false
3829 1459 Acked 2458 Write 4 Done New Done false
3830 1460 Acked 2459 Read 1 Done New Done false
3831 1461 Acked 2460 Read 5 Done New Done false
3832 1462 Acked 2461 Write 4 Done New Done false
3833 1463 Acked 2462 Read 6 Done New Done false
3834 1464 Acked 2463 Read 2 Done New Done false
3835 1465 Acked 2464 Flush 0 Done New Done false
3836 1466 Acked 2465 Read 7 Done New Done false
3837 1467 Acked 2466 Write 5 Done New Done false
3838 1468 Acked 2467 Write 3 Done New Done false
3839 1469 Acked 2468 Flush 0 Done New Done false
3840 1470 Acked 2469 Read 10 Done New Done false
3841 1471 Acked 2470 Write 4 Done New Done false
3842 1472 Acked 2471 Write 6 Done New Done false
3843 1473 Acked 2472 Write 4 Done New Done false
3844 1474 Acked 2473 Read 7 Done New Done false
3845 1475 Acked 2474 Read 9 Done New Done false
3846 1476 Acked 2475 Read 5 Done New Done false
3847 1477 Acked 2476 Read 1 Done New Done false
3848 1478 Acked 2477 Read 10 Done New Done false
3849 1479 Acked 2478 Write 9 Done New Done false
3850 1480 Acked 2479 Read 7 Done New Done false
3851 1481 Acked 2480 Write 5 Done New Done false
3852 1482 Acked 2481 Read 8 Done New Done false
3853 1483 Acked 2482 Write 8 Done New Done false
3854 1484 Acked 2483 Write 4 Done New Done false
3855 1485 Acked 2484 Write 3 Done New Done false
3856 1486 Acked 2485 Read 1 Done New Done false
3857 1487 Acked 2486 Write 8 Done New Done false
3858 1488 Acked 2487 Write 1 Done New Done false
3859 1489 Acked 2488 Write 5 Done New Done false
3860 1490 Acked 2489 Read 10 Done New Done false
3861 1491 Acked 2490 Write 6 Done New Done false
3862 1492 Acked 2491 Write 10 Done New Done false
3863 1493 Acked 2492 Write 2 Done New Done false
3864 1494 Acked 2493 Read 3 Done New Done false
3865 1495 Acked 2494 Write 1 Done New Done false
3866 1496 Acked 2495 Flush 0 Done New Done false
3867 1497 Acked 2496 Read 10 Done New Done false
3868 1498 Acked 2497 Read 8 Done New Done false
3869 1499 Acked 2498 Read 10 Done New Done false
3870 1500 Acked 2499 Write 8 Done New Done false
3871 1501 Acked 2500 Write 3 Done New Done false
3872 1502 Acked 2501 Write 10 Done New Done false
3873 1503 Acked 2502 Flush 0 Done New Done false
3874 1504 Acked 2503 Read 8 Done New Done false
3875 1505 Acked 2504 Write 5 Done New Done false
3876 1506 Acked 2505 Read 2 Done New Done false
3877 1507 Acked 2506 Read 6 Done New Done false
3878 1508 Acked 2507 Write 6 Done New Done false
3879 1509 Acked 2508 Read 8 Done New Done false
3880 1510 Acked 2509 Read 4 Done New Done false
3881 1511 Acked 2510 Read 6 Done New Done false
3882 STATES DS:0 DS:1 DS:2 TOTAL
3883 New 0 302 0 302
3884 Sent 0 0 0 0
3885 Done 302 0 302 604
3886 Skipped 0 0 0 0
3887 Error 0 0 0 0
3888 Last Flush: 2502 2208 2502
3889 Downstairs last five completed: 2208 2207 2206 2205 2204 2203
3890 Upstairs last five completed: 1511 1510 1509 1508 1507 1506
3891 CLIENT: Up:0 ds:302 act:2
3892 [8820] got start action
3893 [8820] I am going to restart
3894 Starting downstairs at port 8820
3895 Make output file at "/tmp/dsc/downstairs-8820.txt"
3896 [8820][1] reports Starting
3897 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1178)
3898 [8820][1] reports Running
3899 ----------------------------------------------------------------
3900 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
3901 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
3902 1210 Acked 2209 Write 9 Done New Done false
3903 1211 Acked 2210 Read 6 Done New Done false
3904 1212 Acked 2211 Read 4 Done New Done false
3905 1213 Acked 2212 Read 10 Done New Done false
3906 1214 Acked 2213 Read 8 Done New Done false
3907 1215 Acked 2214 Write 2 Done New Done false
3908 1216 Acked 2215 Read 6 Done New Done false
3909 1217 Acked 2216 Read 1 Done New Done false
3910 1218 Acked 2217 Write 8 Done New Done false
3911 1219 Acked 2218 Write 9 Done New Done false
3912 1220 Acked 2219 Read 8 Done New Done false
3913 1221 Acked 2220 Write 9 Done New Done false
3914 1222 Acked 2221 Write 9 Done New Done false
3915 1223 Acked 2222 Read 1 Done New Done false
3916 1224 Acked 2223 Read 10 Done New Done false
3917 1225 Acked 2224 Write 8 Done New Done false
3918 1226 Acked 2225 Write 10 Done New Done false
3919 1227 Acked 2226 Read 9 Done New Done false
3920 1228 Acked 2227 Read 8 Done New Done false
3921 1229 Acked 2228 Flush 0 Done New Done false
3922 1230 Acked 2229 Read 1 Done New Done false
3923 1231 Acked 2230 Flush 0 Done New Done false
3924 1232 Acked 2231 Read 3 Done New Done false
3925 1233 Acked 2232 Write 8 Done New Done false
3926 1234 Acked 2233 Read 8 Done New Done false
3927 1235 Acked 2234 Write 3 Done New Done false
3928 1236 Acked 2235 Read 10 Done New Done false
3929 1237 Acked 2236 Write 6 Done New Done false
3930 1238 Acked 2237 Write 7 Done New Done false
3931 1239 Acked 2238 Read 7 Done New Done false
3932 1240 Acked 2239 Write 5 Done New Done false
3933 1241 Acked 2240 Flush 0 Done New Done false
3934 1242 Acked 2241 Write 9 Done New Done false
3935 1243 Acked 2242 Write 8 Done New Done false
3936 1244 Acked 2243 Read 5 Done New Done false
3937 1245 Acked 2244 Flush 0 Done New Done false
3938 1246 Acked 2245 Write 9 Done New Done false
3939 1247 Acked 2246 Read 5 Done New Done false
3940 1248 Acked 2247 Write 10 Done New Done false
3941 1249 Acked 2248 Read 7 Done New Done false
3942 1250 Acked 2249 Read 1 Done New Done false
3943 1251 Acked 2250 Read 8 Done New Done false
3944 1252 Acked 2251 Read 8 Done New Done false
3945 1253 Acked 2252 Write 4 Done New Done false
3946 1254 Acked 2253 Write 3 Done New Done false
3947 1255 Acked 2254 Write 6 Done New Done false
3948 1256 Acked 2255 Read 9 Done New Done false
3949 1257 Acked 2256 Write 10 Done New Done false
3950 1258 Acked 2257 Read 4 Done New Done false
3951 1259 Acked 2258 Write 8 Done New Done false
3952 1260 Acked 2259 Write 4 Done New Done false
3953 1261 Acked 2260 Read 3 Done New Done false
3954 1262 Acked 2261 Read 8 Done New Done false
3955 1263 Acked 2262 Write 9 Done New Done false
3956 1264 Acked 2263 Write 3 Done New Done false
3957 1265 Acked 2264 Read 9 Done New Done false
3958 1266 Acked 2265 Read 8 Done New Done false
3959 1267 Acked 2266 Write 9 Done New Done false
3960 1268 Acked 2267 Read 10 Done New Done false
3961 1269 Acked 2268 Write 9 Done New Done false
3962 1270 Acked 2269 Read 6 Done New Done false
3963 1271 Acked 2270 Read 6 Done New Done false
3964 1272 Acked 2271 Flush 0 Done New Done false
3965 1273 Acked 2272 Write 1 Done New Done false
3966 1274 Acked 2273 Write 10 Done New Done false
3967 1275 Acked 2274 Read 8 Done New Done false
3968 1276 Acked 2275 Read 2 Done New Done false
3969 1277 Acked 2276 Read 6 Done New Done false
3970 1278 Acked 2277 Read 4 Done New Done false
3971 1279 Acked 2278 Read 8 Done New Done false
3972 1280 Acked 2279 Write 5 Done New Done false
3973 1281 Acked 2280 Write 8 Done New Done false
3974 1282 Acked 2281 Read 8 Done New Done false
3975 1283 Acked 2282 Write 5 Done New Done false
3976 1284 Acked 2283 Read 1 Done New Done false
3977 1285 Acked 2284 Write 4 Done New Done false
3978 1286 Acked 2285 Read 6 Done New Done false
3979 1287 Acked 2286 Flush 0 Done New Done false
3980 1288 Acked 2287 Read 8 Done New Done false
3981 1289 Acked 2288 Write 7 Done New Done false
3982 1290 Acked 2289 Read 5 Done New Done false
3983 1291 Acked 2290 Write 2 Done New Done false
3984 1292 Acked 2291 Read 6 Done New Done false
3985 1293 Acked 2292 Read 1 Done New Done false
3986 1294 Acked 2293 Write 6 Done New Done false
3987 1295 Acked 2294 Read 8 Done New Done false
3988 1296 Acked 2295 Write 6 Done New Done false
3989 1297 Acked 2296 Flush 0 Done New Done false
3990 1298 Acked 2297 Write 4 Done New Done false
3991 1299 Acked 2298 Flush 0 Done New Done false
3992 1300 Acked 2299 Write 6 Done New Done false
3993 1301 Acked 2300 Write 3 Done New Done false
3994 1302 Acked 2301 Read 5 Done New Done false
3995 1303 Acked 2302 Read 2 Done New Done false
3996 1304 Acked 2303 Flush 0 Done New Done false
3997 1305 Acked 2304 Write 8 Done New Done false
3998 1306 Acked 2305 Read 1 Done New Done false
3999 1307 Acked 2306 Flush 0 Done New Done false
4000 1308 Acked 2307 Read 7 Done New Done false
4001 1309 Acked 2308 Write 10 Done New Done false
4002 1310 Acked 2309 Read 7 Done New Done false
4003 1311 Acked 2310 Write 7 Done New Done false
4004 1312 Acked 2311 Read 1 Done New Done false
4005 1313 Acked 2312 Write 8 Done New Done false
4006 1314 Acked 2313 Read 4 Done New Done false
4007 1315 Acked 2314 Write 6 Done New Done false
4008 1316 Acked 2315 Flush 0 Done New Done false
4009 1317 Acked 2316 Read 1 Done New Done false
4010 1318 Acked 2317 Flush 0 Done New Done false
4011 1319 Acked 2318 Read 1 Done New Done false
4012 1320 Acked 2319 Write 7 Done New Done false
4013 1321 Acked 2320 Read 8 Done New Done false
4014 1322 Acked 2321 Read 3 Done New Done false
4015 1323 Acked 2322 Read 6 Done New Done false
4016 1324 Acked 2323 Flush 0 Done New Done false
4017 1325 Acked 2324 Read 5 Done New Done false
4018 1326 Acked 2325 Read 2 Done New Done false
4019 1327 Acked 2326 Read 7 Done New Done false
4020 1328 Acked 2327 Write 2 Done New Done false
4021 1329 Acked 2328 Read 10 Done New Done false
4022 1330 Acked 2329 Read 8 Done New Done false
4023 1331 Acked 2330 Read 7 Done New Done false
4024 1332 Acked 2331 Flush 0 Done New Done false
4025 1333 Acked 2332 Read 7 Done New Done false
4026 1334 Acked 2333 Read 8 Done New Done false
4027 1335 Acked 2334 Read 3 Done New Done false
4028 1336 Acked 2335 Flush 0 Done New Done false
4029 1337 Acked 2336 Write 10 Done New Done false
4030 1338 Acked 2337 Read 4 Done New Done false
4031 1339 Acked 2338 Read 7 Done New Done false
4032 1340 Acked 2339 Flush 0 Done New Done false
4033 1341 Acked 2340 Read 8 Done New Done false
4034 1342 Acked 2341 Write 5 Done New Done false
4035 1343 Acked 2342 Write 7 Done New Done false
4036 1344 Acked 2343 Write 9 Done New Done false
4037 1345 Acked 2344 Read 8 Done New Done false
4038 1346 Acked 2345 Write 2 Done New Done false
4039 1347 Acked 2346 Read 3 Done New Done false
4040 1348 Acked 2347 Read 9 Done New Done false
4041 1349 Acked 2348 Read 9 Done New Done false
4042 1350 Acked 2349 Read 7 Done New Done false
4043 1351 Acked 2350 Write 8 Done New Done false
4044 1352 Acked 2351 Write 4 Done New Done false
4045 1353 Acked 2352 Write 4 Done New Done false
4046 1354 Acked 2353 Write 7 Done New Done false
4047 1355 Acked 2354 Write 2 Done New Done false
4048 1356 Acked 2355 Read 7 Done New Done false
4049 1357 Acked 2356 Read 6 Done New Done false
4050 1358 Acked 2357 Write 6 Done New Done false
4051 1359 Acked 2358 Read 8 Done New Done false
4052 1360 Acked 2359 Write 8 Done New Done false
4053 1361 Acked 2360 Read 10 Done New Done false
4054 1362 Acked 2361 Write 6 Done New Done false
4055 1363 Acked 2362 Read 5 Done New Done false
4056 1364 Acked 2363 Read 6 Done New Done false
4057 1365 Acked 2364 Read 8 Done New Done false
4058 1366 Acked 2365 Read 1 Done New Done false
4059 1367 Acked 2366 Write 3 Done New Done false
4060 1368 Acked 2367 Read 4 Done New Done false
4061 1369 Acked 2368 Read 9 Done New Done false
4062 1370 Acked 2369 Flush 0 Done New Done false
4063 1371 Acked 2370 Read 8 Done New Done false
4064 1372 Acked 2371 Write 9 Done New Done false
4065 1373 Acked 2372 Write 2 Done New Done false
4066 1374 Acked 2373 Write 2 Done New Done false
4067 1375 Acked 2374 Read 6 Done New Done false
4068 1376 Acked 2375 Flush 0 Done New Done false
4069 1377 Acked 2376 Write 7 Done New Done false
4070 1378 Acked 2377 Flush 0 Done New Done false
4071 1379 Acked 2378 Read 5 Done New Done false
4072 1380 Acked 2379 Read 10 Done New Done false
4073 1381 Acked 2380 Read 3 Done New Done false
4074 1382 Acked 2381 Read 5 Done New Done false
4075 1383 Acked 2382 Write 5 Done New Done false
4076 1384 Acked 2383 Write 9 Done New Done false
4077 1385 Acked 2384 Write 6 Done New Done false
4078 1386 Acked 2385 Read 7 Done New Done false
4079 1387 Acked 2386 Write 10 Done New Done false
4080 1388 Acked 2387 Write 5 Done New Done false
4081 1389 Acked 2388 Read 1 Done New Done false
4082 1390 Acked 2389 Write 9 Done New Done false
4083 1391 Acked 2390 Read 8 Done New Done false
4084 1392 Acked 2391 Read 6 Done New Done false
4085 1393 Acked 2392 Write 2 Done New Done false
4086 1394 Acked 2393 Read 9 Done New Done false
4087 1395 Acked 2394 Write 1 Done New Done false
4088 1396 Acked 2395 Write 9 Done New Done false
4089 1397 Acked 2396 Write 7 Done New Done false
4090 1398 Acked 2397 Write 7 Done New Done false
4091 1399 Acked 2398 Read 4 Done New Done false
4092 1400 Acked 2399 Flush 0 Done New Done false
4093 1401 Acked 2400 Write 8 Done New Done false
4094 1402 Acked 2401 Read 3 Done New Done false
4095 1403 Acked 2402 Read 2 Done New Done false
4096 1404 Acked 2403 Write 9 Done New Done false
4097 1405 Acked 2404 Write 2 Done New Done false
4098 1406 Acked 2405 Write 2 Done New Done false
4099 1407 Acked 2406 Read 4 Done New Done false
4100 1408 Acked 2407 Flush 0 Done New Done false
4101 1409 Acked 2408 Write 5 Done New Done false
4102 1410 Acked 2409 Write 2 Done New Done false
4103 1411 Acked 2410 Write 3 Done New Done false
4104 1412 Acked 2411 Write 7 Done New Done false
4105 1413 Acked 2412 Flush 0 Done New Done false
4106 1414 Acked 2413 Write 1 Done New Done false
4107 1415 Acked 2414 Write 5 Done New Done false
4108 1416 Acked 2415 Read 9 Done New Done false
4109 1417 Acked 2416 Read 9 Done New Done false
4110 1418 Acked 2417 Write 10 Done New Done false
4111 1419 Acked 2418 Write 1 Done New Done false
4112 1420 Acked 2419 Write 6 Done New Done false
4113 1421 Acked 2420 Write 5 Done New Done false
4114 1422 Acked 2421 Read 5 Done New Done false
4115 1423 Acked 2422 Read 6 Done New Done false
4116 1424 Acked 2423 Write 2 Done New Done false
4117 1425 Acked 2424 Write 5 Done New Done false
4118 1426 Acked 2425 Flush 0 Done New Done false
4119 1427 Acked 2426 Read 8 Done New Done false
4120 1428 Acked 2427 Read 2 Done New Done false
4121 1429 Acked 2428 Write 4 Done New Done false
4122 1430 Acked 2429 Read 2 Done New Done false
4123 1431 Acked 2430 Read 6 Done New Done false
4124 1432 Acked 2431 Read 1 Done New Done false
4125 1433 Acked 2432 Write 9 Done New Done false
4126 1434 Acked 2433 Write 5 Done New Done false
4127 1435 Acked 2434 Read 8 Done New Done false
4128 1436 Acked 2435 Read 8 Done New Done false
4129 1437 Acked 2436 Read 7 Done New Done false
4130 1438 Acked 2437 Read 6 Done New Done false
4131 1439 Acked 2438 Read 3 Done New Done false
4132 1440 Acked 2439 Write 10 Done New Done false
4133 1441 Acked 2440 Write 2 Done New Done false
4134 1442 Acked 2441 Write 5 Done New Done false
4135 1443 Acked 2442 Write 9 Done New Done false
4136 1444 Acked 2443 Read 3 Done New Done false
4137 1445 Acked 2444 Read 7 Done New Done false
4138 1446 Acked 2445 Write 8 Done New Done false
4139 1447 Acked 2446 Write 10 Done New Done false
4140 1448 Acked 2447 Read 6 Done New Done false
4141 1449 Acked 2448 Read 3 Done New Done false
4142 1450 Acked 2449 Write 2 Done New Done false
4143 1451 Acked 2450 Flush 0 Done New Done false
4144 1452 Acked 2451 Write 1 Done New Done false
4145 1453 Acked 2452 Flush 0 Done New Done false
4146 1454 Acked 2453 Read 7 Done New Done false
4147 1455 Acked 2454 Read 10 Done New Done false
4148 1456 Acked 2455 Read 6 Done New Done false
4149 1457 Acked 2456 Read 6 Done New Done false
4150 1458 Acked 2457 Flush 0 Done New Done false
4151 1459 Acked 2458 Write 4 Done New Done false
4152 1460 Acked 2459 Read 1 Done New Done false
4153 1461 Acked 2460 Read 5 Done New Done false
4154 1462 Acked 2461 Write 4 Done New Done false
4155 1463 Acked 2462 Read 6 Done New Done false
4156 1464 Acked 2463 Read 2 Done New Done false
4157 1465 Acked 2464 Flush 0 Done New Done false
4158 1466 Acked 2465 Read 7 Done New Done false
4159 1467 Acked 2466 Write 5 Done New Done false
4160 1468 Acked 2467 Write 3 Done New Done false
4161 1469 Acked 2468 Flush 0 Done New Done false
4162 1470 Acked 2469 Read 10 Done New Done false
4163 1471 Acked 2470 Write 4 Done New Done false
4164 1472 Acked 2471 Write 6 Done New Done false
4165 1473 Acked 2472 Write 4 Done New Done false
4166 1474 Acked 2473 Read 7 Done New Done false
4167 1475 Acked 2474 Read 9 Done New Done false
4168 1476 Acked 2475 Read 5 Done New Done false
4169 1477 Acked 2476 Read 1 Done New Done false
4170 1478 Acked 2477 Read 10 Done New Done false
4171 1479 Acked 2478 Write 9 Done New Done false
4172 1480 Acked 2479 Read 7 Done New Done false
4173 1481 Acked 2480 Write 5 Done New Done false
4174 1482 Acked 2481 Read 8 Done New Done false
4175 1483 Acked 2482 Write 8 Done New Done false
4176 1484 Acked 2483 Write 4 Done New Done false
4177 1485 Acked 2484 Write 3 Done New Done false
4178 1486 Acked 2485 Read 1 Done New Done false
4179 1487 Acked 2486 Write 8 Done New Done false
4180 1488 Acked 2487 Write 1 Done New Done false
4181 1489 Acked 2488 Write 5 Done New Done false
4182 1490 Acked 2489 Read 10 Done New Done false
4183 1491 Acked 2490 Write 6 Done New Done false
4184 1492 Acked 2491 Write 10 Done New Done false
4185 1493 Acked 2492 Write 2 Done New Done false
4186 1494 Acked 2493 Read 3 Done New Done false
4187 1495 Acked 2494 Write 1 Done New Done false
4188 1496 Acked 2495 Flush 0 Done New Done false
4189 1497 Acked 2496 Read 10 Done New Done false
4190 1498 Acked 2497 Read 8 Done New Done false
4191 1499 Acked 2498 Read 10 Done New Done false
4192 1500 Acked 2499 Write 8 Done New Done false
4193 1501 Acked 2500 Write 3 Done New Done false
4194 1502 Acked 2501 Write 10 Done New Done false
4195 1503 Acked 2502 Flush 0 Done New Done false
4196 1504 Acked 2503 Read 8 Done New Done false
4197 1505 Acked 2504 Write 5 Done New Done false
4198 1506 Acked 2505 Read 2 Done New Done false
4199 1507 Acked 2506 Read 6 Done New Done false
4200 1508 Acked 2507 Write 6 Done New Done false
4201 1509 Acked 2508 Read 8 Done New Done false
4202 1510 Acked 2509 Read 4 Done New Done false
4203 1511 Acked 2510 Read 6 Done New Done false
4204 1512 Acked 2511 Flush 0 Done New Done false
4205 STATES DS:0 DS:1 DS:2 TOTAL
4206 New 0 303 0 303
4207 Sent 0 0 0 0
4208 Done 303 0 303 606
4209 Skipped 0 0 0 0
4210 Error 0 0 0 0
4211 Last Flush: 2511 2208 2511
4212 Downstairs last five completed: 2208 2207 2206 2205 2204 2203
4213 Upstairs last five completed: 1512 1511 1510 1509 1508 1507
4214 CLIENT: Up:0 ds:303 act:2
42152023-09-22T23:24:38.728ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
42162023-09-22T23:24:38.728ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
42172023-09-22T23:24:38.728ZINFOcrucible: [1] client re-new 303 jobs since flush 2208 = downstairs
42182023-09-22T23:24:38.729ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
42192023-09-22T23:24:38.730ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
42202023-09-22T23:24:38.730ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
42212023-09-22T23:24:38.730ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
42222023-09-22T23:24:38.730ZINFOcrucible: [1] send last flush ID to this DS: 2208
42232023-09-22T23:24:38.730ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
42242023-09-22T23:24:38.730ZINFOcrucible: [1] Replied this last flush ID: 2208
42252023-09-22T23:24:38.730ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
42262023-09-22T23:24:38.730ZINFOcrucible: [1] Transition from Offline to Replay
42272023-09-22T23:24:38.730ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
42282023-09-22T23:24:38.730ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
42292023-09-22T23:24:38.730ZINFOcrucible: [1] Transition from Replay to Active
42302023-09-22T23:24:38.730ZINFOcrucible: [1] Starts cmd_loop
42312023-09-22T23:24:38.730ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
42322023-09-22T23:24:38.730ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
42332023-09-22T23:24:38.730ZINFOcrucible: Set check for repair
42342023-09-22T23:24:39.732ZINFOcrucible: Checking if live repair is needed
42352023-09-22T23:24:39.732ZINFOcrucible: No Live Repair required at this time
42362023-09-22T23:24:39.732ZWARNcrucible: [1] flow control end
4237 ----------------------------------------------------------------
4238 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4239 Downstairs last five completed: 2511 2510 2509 2508 2507 2506
4240 Upstairs last five completed: 1512 1511 1510 1509 1508 1507
4241 CLIENT: Up:0 ds:0 act:3
4242 Replay: All jobs finished, all DS active.
4243 Main task has work to do, go find it
4244 got dsc Stop(2)
4245 stop 2
4246 Sep 22 23:24:42.698 INFO request completed, latency_us: 524, response_code: 204, uri: /stop/cid/2, [8830] Got stop action so:false kr:false
4247 method: POST, req_id: 1a10399a-9b0d-4409-a305-a11ab4f18dfe, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
4248 Sep 22 23:24:42.699 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/2, method: GET, req_id: b41efd74-dc7a-4036-9a2f-748b9e11e15e, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
42492023-09-22T23:24:42.699ZWARNcrucible: [2] downstairs disconnected
42502023-09-22T23:24:42.699ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
42512023-09-22T23:24:42.699ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
42522023-09-22T23:24:42.699ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
42532023-09-22T23:24:42.699ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
42542023-09-22T23:24:42.700ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
42552023-09-22T23:24:42.700ZWARNcrucible: [2] pm_task rx.recv() is None
4256 [8830] Exited with: ExitStatus(unix_wait_status(9))
4257 [8830][2] reports Exit
42582023-09-22T23:24:45.702ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
4259 Sep 22 23:24:46.701 INFO request completed, latency_us: 260, response_code: 200, uri: /state/cid/2, method: GET, req_id: 82174b4c-54d1-4376-b45f-e656b073b901, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
4260 001/300 Read block 4941 len 4096
4261 002/300 Read block 825 len 20480
4262 003/300 Write block 3255 len 32768 data: 4 4 3 3 3 4 3 3
4263 004/300 Write block 2136 len 20480 data: 2 2 2 2 2
4264 005/300 Read block 3166 len 32768
4265 006/300 Write block 1499 len 20480 data: 3 3 3 2 2
4266 007/300 Write block 4728 len 8192 data: 2 2
4267 008/300 Read block 1698 len 16384
4268 009/300 Flush
4269 010/300 Read block 1668 len 36864
4270 011/300 Read block 3540 len 40960
4271 012/300 Write block 67 len 12288 data: 2 3 3
4272 013/300 Read block 4774 len 32768
4273 014/300 Read block 4363 len 24576
4274 015/300 Write block 748 len 4096 data: 2
4275 016/300 Read block 1993 len 12288
4276 017/300 Read block 3373 len 20480
4277 018/300 Read block 4016 len 16384
4278 019/300 Write block 1069 len 8192 data: 2 2
4279 020/300 Read block 3825 len 16384
4280 021/300 Write block 2629 len 12288 data: 2 2 2
4281 022/300 Write block 2148 len 28672 data: 3 3 3 2 2 2 2
4282 023/300 Read block 807 len 28672
4283 024/300 Write block 3983 len 36864 data: 3 3 4 4 4 5 4 4 4
4284 025/300 Read block 1275 len 24576
4285 026/300 Read block 2293 len 16384
4286 027/300 Read block 1726 len 36864
4287 028/300 Flush
4288 029/300 Read block 2061 len 20480
4289 030/300 Read block 453 len 40960
4290 031/300 Flush
4291 032/300 Write block 174 len 28672 data: 3 3 3 3 3 3 3
4292 033/300 Write block 1316 len 4096 data: 3
4293 034/300 Write block 1701 len 4096 data: 3
4294 035/300 Read block 166 len 28672
4295 036/300 Read block 4032 len 36864
4296 037/300 Write block 3158 len 24576 data: 4 3 3 2 2 2
4297 038/300 Read block 4858 len 12288
4298 039/300 Read block 1956 len 4096
4299 040/300 Read block 4837 len 20480
4300 041/300 Read block 4552 len 4096
4301 042/300 Write block 418 len 40960 data: 3 3 3 3 3 4 4 3 3 3
4302 043/300 Write block 1049 len 12288 data: 3 3 2
4303 044/300 Write block 1290 len 36864 data: 2 2 2 2 2 2 2 2 2
4304 045/300 Write block 2620 len 36864 data: 2 3 4 4 4 4 3 2 2
4305 046/300 Read block 2599 len 20480
4306 047/300 Read block 602 len 36864
4307 048/300 Write block 1293 len 24576 data: 3 3 3 3 3 3
4308 049/300 Write block 399 len 28672 data: 3 3 2 2 2 2 2
4309 050/300 Read block 2571 len 28672
4310 051/300 Write block 4276 len 16384 data: 3 3 2 3
4311 052/300 Write block 523 len 36864 data: 2 2 2 2 2 2 2 2 2
4312 053/300 Write block 2605 len 12288 data: 2 2 2
4313 054/300 Write block 3719 len 20480 data: 4 3 3 3 3
4314 055/300 Write block 2020 len 8192 data: 2 2
4315 056/300 Read block 4578 len 36864
4316 057/300 Read block 4851 len 8192
4317 058/300 Read block 524 len 16384
4318 059/300 Write block 3968 len 36864 data: 4 3 3 3 2 2 2 2 2
4319 060/300 Read block 2859 len 4096
4320 061/300 Flush
4321 062/300 Read block 2708 len 4096
4322 063/300 Flush
4323 064/300 Read block 1897 len 16384
4324 065/300 Read block 2477 len 4096
4325 066/300 Write block 2508 len 28672 data: 2 2 2 4 4 4 4
4326 067/300 Flush
4327 068/300 Write block 2636 len 8192 data: 2 2
4328 069/300 Flush
4329 070/300 Write block 501 len 12288 data: 2 2 2
4330 071/300 Write block 142 len 36864 data: 2 3 3 3 3 3 3 3 3
4331 072/300 Read block 4832 len 16384
4332 073/300 Write block 146 len 40960 data: 4 4 4 4 4 3 2 2 2 2
4333 074/300 Write block 349 len 8192 data: 2 2
4334 075/300 Read block 1575 len 20480
4335 076/300 Read block 4798 len 4096
4336 077/300 Read block 265 len 20480
4337 078/300 Read block 4661 len 8192
4338 079/300 Write block 4522 len 8192 data: 2 3
4339 080/300 Read block 1023 len 12288
4340 081/300 Flush
4341 082/300 Write block 1937 len 40960 data: 2 2 3 3 3 3 3 2 2 2
4342 083/300 Read block 4377 len 16384
4343 084/300 Write block 3894 len 32768 data: 2 2 2 3 4 4 4 4
4344 085/300 Read block 3276 len 12288
4345 086/300 Write block 4204 len 24576 data: 3 3 3 2 2 3
4346 087/300 Write block 3047 len 40960 data: 4 3 2 2 2 2 2 2 2 2
4347 088/300 Read block 3413 len 20480
4348 089/300 Read block 3479 len 12288
4349 090/300 Write block 1200 len 40960 data: 2 2 2 3 4 4 4 4 4 3
4350 091/300 Read block 596 len 4096
4351 092/300 Read block 994 len 8192
4352 093/300 Flush
4353 094/300 Read block 4956 len 36864
4354 095/300 Write block 4448 len 40960 data: 2 2 2 2 2 2 3 3 3 3
4355 096/300 Read block 662 len 12288
4356 097/300 Read block 1566 len 4096
4357 098/300 Read block 3796 len 16384
4358 099/300 Read block 2845 len 32768
4359 100/300 Write block 4714 len 16384 data: 3 3 3 2
4360 101/300 Read block 115 len 16384
4361 102/300 Read block 2188 len 4096
4362 103/300 Flush
4363 104/300 Read block 3161 len 4096
4364 105/300 Write block 3528 len 12288 data: 2 2 2
4365 106/300 Read block 4142 len 4096
4366 107/300 Write block 560 len 24576 data: 2 3 3 2 2 2
4367 108/300 Read block 381 len 36864
4368 109/300 Flush
4369 110/300 Write block 318 len 28672 data: 2 2 2 2 2 2 2
4370 111/300 Read block 974 len 32768
4371 112/300 Read block 4919 len 36864
4372 113/300 Read block 4257 len 16384
4373 114/300 Write block 948 len 8192 data: 2 2
4374 115/300 Write block 1651 len 32768 data: 2 2 2 2 3 2 2 2
4375 116/300 Read block 2624 len 20480
4376 117/300 Write block 4988 len 24576 data: 3 3 2 2 2 2
4377 118/300 Write block 166 len 36864 data: 2 2 3 3 3 3 3 3 4
4378 119/300 Write block 3423 len 20480 data: 2 2 2 2 2
4379 120/300 Write block 819 len 8192 data: 3 4
4380 121/300 Read block 3884 len 32768
4381 122/300 Read block 722 len 32768
4382 123/300 Read block 354 len 16384
4383 124/300 Read block 1262 len 28672
4384 125/300 Read block 3137 len 16384
4385 126/300 Flush
4386 127/300 Read block 3476 len 16384
4387 128/300 Read block 160 len 40960
4388 129/300 Read block 134 len 24576
4389 130/300 Read block 4078 len 36864
4390 131/300 Flush
4391 132/300 Flush
4392 133/300 Write block 1290 len 4096 data: 3
4393 134/300 Read block 985 len 4096
4394 135/300 Read block 4618 len 24576
4395 136/300 Read block 3344 len 40960
4396 137/300 Read block 662 len 4096
4397 138/300 Write block 1791 len 32768 data: 4 4 3 3 3 3 3 2
4398 139/300 Read block 3157 len 28672
4399 140/300 Read block 1030 len 40960
4400 141/300 Read block 2927 len 36864
4401 142/300 Read block 1695 len 12288
4402 143/300 Read block 2155 len 8192
4403 144/300 Write block 3551 len 32768 data: 3 3 3 2 2 2 2 2
4404 145/300 Write block 2160 len 28672 data: 2 2 2 2 2 2 2
4405 146/300 Read block 3511 len 12288
4406 147/300 Read block 446 len 8192
4407 148/300 Read block 2852 len 28672
4408 149/300 Read block 1951 len 4096
4409 150/300 Write block 4910 len 32768 data: 2 3 4 4 4 4 4 4
4410 151/300 Read block 3367 len 40960
4411 152/300 Write block 897 len 32768 data: 3 2 2 2 2 2 2 2
4412 153/300 Flush
4413 154/300 Read block 3639 len 40960
4414 155/300 Read block 1731 len 32768
4415 156/300 Write block 404 len 32768 data: 3 3 2 2 2 2 2 2
4416 157/300 Write block 604 len 20480 data: 3 2 2 2 2
4417 158/300 Flush
4418 159/300 Write block 3427 len 8192 data: 3 2
4419 160/300 Read block 1850 len 12288
4420 161/300 Write block 3654 len 16384 data: 3 3 3 3
4421 162/300 Read block 1246 len 4096
4422 163/300 Read block 4091 len 24576
4423 164/300 Write block 1893 len 28672 data: 2 2 2 2 2 2 2
4424 165/300 Write block 487 len 36864 data: 2 2 2 2 2 2 2 2 2
4425 166/300 Write block 4385 len 4096 data: 2
4426 167/300 Read block 1903 len 40960
4427 168/300 Read block 2331 len 16384
4428 169/300 Write block 4559 len 32768 data: 4 4 4 4 4 3 2 2
4429 170/300 Write block 631 len 24576 data: 2 2 2 2 2 2
4430 171/300 Read block 2456 len 16384
4431 172/300 Write block 1186 len 24576 data: 3 3 3 3 3 2
4432 173/300 Read block 1590 len 32768
4433 174/300 Write block 3776 len 36864 data: 2 2 2 2 2 2 2 2 2
4434 175/300 Read block 3332 len 16384
4435 176/300 Write block 3081 len 24576 data: 4 4 4 4 4 4
4436 177/300 Flush
4437 178/300 Write block 2368 len 4096 data: 3
4438 179/300 Write block 2832 len 20480 data: 5 5 4 4 3
4439 180/300 Write block 3528 len 8192 data: 3 3
4440 181/300 Flush
4441 182/300 Read block 634 len 8192
4442 183/300 Read block 2332 len 4096
4443 184/300 Write block 2138 len 8192 data: 3 3
4444 185/300 Read block 1792 len 40960
4445 186/300 Write block 1620 len 12288 data: 3 2 2
4446 187/300 Write block 4645 len 24576 data: 3 3 4 3 3 2
4447 188/300 Read block 1825 len 36864
4448 189/300 Read block 1660 len 8192
4449 190/300 Write block 1629 len 4096 data: 4
4450 191/300 Write block 356 len 36864 data: 4 4 5 5 4 3 2 2 2
4451 192/300 Read block 4511 len 24576
4452 193/300 Flush
4453 194/300 Read block 805 len 16384
4454 195/300 Read block 4354 len 24576
4455 196/300 Read block 4851 len 16384
4456 197/300 Flush
4457 198/300 Write block 1403 len 24576 data: 2 2 3 3 3 3
4458 199/300 Write block 1729 len 28672 data: 2 2 2 2 2 2 3
4459 200/300 Write block 4107 len 4096 data: 3
4460 201/300 Read block 2605 len 24576
4461 202/300 Flush
4462 203/300 Write block 3465 len 28672 data: 2 2 3 3 3 3 2
4463 204/300 Write block 3498 len 12288 data: 3 3 3
4464 205/300 Write block 3806 len 8192 data: 3 3
4465 206/300 Read block 1663 len 8192
4466 207/300 Read block 3127 len 36864
4467 208/300 Flush
4468 209/300 Read block 1588 len 20480
4469 210/300 Read block 3194 len 4096
4470 211/300 Read block 1478 len 8192
4471 212/300 Write block 759 len 16384 data: 4 4 4 4
4472 213/300 Read block 63 len 16384
4473 214/300 Write block 186 len 32768 data: 4 4 4 4 3 2 2 2
4474 215/300 Read block 4359 len 16384
4475 216/300 Flush
4476 217/300 Flush
4477 218/300 Read block 1275 len 24576
4478 219/300 Read block 2023 len 32768
4479 220/300 Write block 738 len 24576 data: 2 2 2 2 2 2
4480 221/300 Read block 4940 len 16384
4481 222/300 Write block 4956 len 16384 data: 5 4 4 3
4482 223/300 Read block 2409 len 20480
4483 224/300 Read block 4346 len 32768
4484 225/300 Write block 194 len 12288 data: 2 2 2
4485 226/300 Write block 740 len 12288 data: 3 3 3
4486 227/300 Read block 1970 len 32768
4487 228/300 Read block 3812 len 8192
4488 229/300 Read block 2472 len 40960
4489 230/300 Read block 2822 len 32768
4490 231/300 Write block 2904 len 20480 data: 3 4 4 4 4
4491 232/300 Write block 296 len 8192 data: 3 3
4492 233/300 Write block 592 len 4096 data: 3
4493 234/300 Flush
4494 235/300 Read block 2987 len 20480
4495 236/300 Read block 4100 len 8192
4496 237/300 Write block 2515 len 16384 data: 4 4 4 4
4497 238/300 Write block 1427 len 36864 data: 2 2 2 3 3 3 3 3 3
4498 239/300 Read block 3110 len 28672
4499 240/300 Read block 1786 len 4096
4500 241/300 Read block 1436 len 8192
4501 242/300 Write block 318 len 32768 data: 3 3 3 3 3 3 3 2
4502 243/300 Write block 4117 len 8192 data: 2 2
4503 244/300 Read block 680 len 28672
4504 245/300 Write block 4190 len 16384 data: 2 2 3 3
4505 246/300 Read block 4613 len 32768
4506 247/300 Read block 4359 len 8192
4507 248/300 Read block 3713 len 40960
4508 249/300 Read block 4276 len 40960
4509 250/300 Read block 2707 len 24576
4510 251/300 Read block 4927 len 16384
4511 252/300 Read block 1480 len 8192
4512 253/300 Write block 1668 len 20480 data: 3 3 3 3 2
4513 254/300 Read block 2145 len 12288
4514 255/300 Write block 2841 len 8192 data: 4 5
4515 256/300 Write block 258 len 4096 data: 3
4516 257/300 Write block 3166 len 28672 data: 2 2 2 2 2 2 2
4517 258/300 Read block 1208 len 20480
4518 259/300 Write block 2196 len 40960 data: 2 2 2 2 3 3 3 3 3 3
4519 260/300 Read block 4271 len 12288
4520 261/300 Read block 1687 len 24576
4521 262/300 Write block 4464 len 28672 data: 2 2 2 2 2 2 2
4522 263/300 Read block 1258 len 28672
4523 264/300 Write block 1794 len 28672 data: 4 4 4 4 3 2 2
4524 265/300 Write block 40 len 32768 data: 3 3 3 4 4 4 4 3
4525 266/300 Read block 1685 len 32768
4526 267/300 Write block 4581 len 24576 data: 2 2 2 2 2 2
4527 268/300 Read block 3142 len 20480
4528 269/300 Read block 3328 len 4096
4529 270/300 Read block 3842 len 20480
4530 271/300 Read block 2353 len 36864
4531 272/300 Read block 2251 len 28672
4532 273/300 Write block 3206 len 12288 data: 3 3 3
4533 274/300 Read block 2419 len 40960
4534 275/300 Write block 3617 len 40960 data: 2 2 2 2 2 3 3 3 2 2
4535 276/300 Flush
4536 277/300 Write block 3514 len 40960 data: 4 3 3 2 2 2 2 2 2 2
4537 278/300 Read block 567 len 24576
4538 279/300 Write block 4373 len 36864 data: 3 2 2 2 2 2 2 2 2
4539 280/300 Write block 3581 len 20480 data: 2 2 3 3 3
4540 281/300 Write block 287 len 20480 data: 3 2 2 2 2
4541 282/300 Read block 2740 len 36864
4542 283/300 Flush
4543 284/300 Read block 3387 len 28672
4544 285/300 Write block 266 len 4096 data: 3
4545 286/300 Read block 4861 len 32768
4546 287/300 Read block 4479 len 24576
4547 288/300 Write block 1057 len 20480 data: 2 2 2 2 2
4548 289/300 Read block 4609 len 32768
4549 290/300 Read block 2009 len 8192
4550 291/300 Write block 2309 len 4096 data: 2
4551 292/300 Read block 2661 len 24576
4552 293/300 Write block 4826 len 8192 data: 4 4
4553 294/300 Read block 2456 len 40960
4554 295/300 Read block 289 len 40960
4555 296/300 Read block 2027 len 12288
4556 297/300 Write block 2723 len 24576 data: 3 2 3 4 4 4
4557 298/300 Read block 4285 len 40960
4558 299/300 Read block 3400 len 32768
4559 300/300 Read block 2165 len 32768
4560 Main task has work to do, go find it
4561 got dsc Start(2)
4562 start 2
4563 Sep 22 23:24:47.617 INFO request completed, latency_us: 301, response_code: 204, uri: /start/cid/2, method: POST, req_id: 474a2ffd-58f8-433a-b735-875b75068d31, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
4564 Replay: started 2, returned:Ok(())
4565 ----------------------------------------------------------------
4566 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4567 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
4568 1513 Acked 2512 Read 1 Done Done New false
4569 1514 Acked 2513 Read 5 Done Done New false
4570 1515 Acked 2514 Write 8 Done Done New false
4571 1516 Acked 2515 Write 5 Done Done New false
4572 1517 Acked 2516 Read 8 Done Done New false
4573 1518 Acked 2517 Write 5 Done Done New false
4574 1519 Acked 2518 Write 2 Done Done New false
4575 1520 Acked 2519 Read 4 Done Done New false
4576 1521 Acked 2520 Flush 0 Done Done New false
4577 1522 Acked 2521 Read 9 Done Done New false
4578 1523 Acked 2522 Read 10 Done Done New false
4579 1524 Acked 2523 Write 3 Done Done New false
4580 1525 Acked 2524 Read 8 Done Done New false
4581 1526 Acked 2525 Read 6 Done Done New false
4582 1527 Acked 2526 Write 1 Done Done New false
4583 1528 Acked 2527 Read 3 Done Done New false
4584 1529 Acked 2528 Read 5 Done Done New false
4585 1530 Acked 2529 Read 4 Done Done New false
4586 1531 Acked 2530 Write 2 Done Done New false
4587 1532 Acked 2531 Read 4 Done Done New false
4588 1533 Acked 2532 Write 3 Done Done New false
4589 1534 Acked 2533 Write 7 Done Done New false
4590 1535 Acked 2534 Read 7 Done Done New false
4591 1536 Acked 2535 Write 9 Done Done New false
4592 1537 Acked 2536 Read 6 Done Done New false
4593 1538 Acked 2537 Read 4 Done Done New false
4594 1539 Acked 2538 Read 9 Done Done New false
4595 1540 Acked 2539 Flush 0 Done Done New false
4596 1541 Acked 2540 Read 5 Done Done New false
4597 1542 Acked 2541 Read 10 Done Done New false
4598 1543 Acked 2542 Flush 0 Done Done New false
4599 1544 Acked 2543 Write 7 Done Done New false
4600 1545 Acked 2544 Write 1 Done Done New false
4601 1546 Acked 2545 Write 1 Done Done New false
4602 1547 Acked 2546 Read 7 Done Done New false
4603 1548 Acked 2547 Read 9 Done Done New false
4604 1549 Acked 2548 Write 6 Done Done New false
4605 1550 Acked 2549 Read 3 Done Done New false
4606 1551 Acked 2550 Read 1 Done Done New false
4607 1552 Acked 2551 Read 5 Done Done New false
4608 1553 Acked 2552 Read 1 Done Done New false
4609 1554 Acked 2553 Write 10 Done Done New false
4610 1555 Acked 2554 Write 3 Done Done New false
4611 1556 Acked 2555 Write 9 Done Done New false
4612 1557 Acked 2556 Write 9 Done Done New false
4613 1558 Acked 2557 Read 5 Done Done New false
4614 1559 Acked 2558 Read 9 Done Done New false
4615 1560 Acked 2559 Write 6 Done Done New false
4616 1561 Acked 2560 Write 7 Done Done New false
4617 1562 Acked 2561 Read 7 Done Done New false
4618 1563 Acked 2562 Write 4 Done Done New false
4619 1564 Acked 2563 Write 9 Done Done New false
4620 1565 Acked 2564 Write 3 Done Done New false
4621 1566 Acked 2565 Write 5 Done Done New false
4622 1567 Acked 2566 Write 2 Done Done New false
4623 1568 Acked 2567 Read 9 Done Done New false
4624 1569 Acked 2568 Flush 0 Done Done New false
4625 1570 Acked 2569 Read 2 Done Done New false
4626 1571 Acked 2570 Read 4 Done Done New false
4627 1572 Acked 2571 Write 9 Done Done New false
4628 1573 Acked 2572 Read 1 Done Done New false
4629 1574 Acked 2573 Flush 0 Done Done New false
4630 1575 Acked 2574 Read 1 Done Done New false
4631 1576 Acked 2575 Flush 0 Done Done New false
4632 1577 Acked 2576 Read 4 Done Done New false
4633 1578 Acked 2577 Read 1 Done Done New false
4634 1579 Acked 2578 Write 7 Done Done New false
4635 1580 Acked 2579 Flush 0 Done Done New false
4636 1581 Acked 2580 Write 2 Done Done New false
4637 1582 Acked 2581 Flush 0 Done Done New false
4638 1583 Acked 2582 Write 3 Done Done New false
4639 1584 Acked 2583 Write 9 Done Done New false
4640 1585 Acked 2584 Read 4 Done Done New false
4641 1586 Acked 2585 Write 10 Done Done New false
4642 1587 Acked 2586 Write 2 Done Done New false
4643 1588 Acked 2587 Read 5 Done Done New false
4644 1589 Acked 2588 Read 1 Done Done New false
4645 1590 Acked 2589 Read 5 Done Done New false
4646 1591 Acked 2590 Read 2 Done Done New false
4647 1592 Acked 2591 Write 2 Done Done New false
4648 1593 Acked 2592 Read 3 Done Done New false
4649 1594 Acked 2593 Flush 0 Done Done New false
4650 1595 Acked 2594 Write 10 Done Done New false
4651 1596 Acked 2595 Read 4 Done Done New false
4652 1597 Acked 2596 Write 8 Done Done New false
4653 1598 Acked 2597 Read 3 Done Done New false
4654 1599 Acked 2598 Write 6 Done Done New false
4655 1600 Acked 2599 Write 10 Done Done New false
4656 1601 Acked 2600 Read 5 Done Done New false
4657 1602 Acked 2601 Read 3 Done Done New false
4658 1603 Acked 2602 Write 10 Done Done New false
4659 1604 Acked 2603 Read 1 Done Done New false
4660 1605 Acked 2604 Read 2 Done Done New false
4661 1606 Acked 2605 Flush 0 Done Done New false
4662 1607 Acked 2606 Read 9 Done Done New false
4663 1608 Acked 2607 Write 10 Done Done New false
4664 1609 Acked 2608 Read 3 Done Done New false
4665 1610 Acked 2609 Read 1 Done Done New false
4666 1611 Acked 2610 Read 4 Done Done New false
4667 1612 Acked 2611 Read 8 Done Done New false
4668 1613 Acked 2612 Write 4 Done Done New false
4669 1614 Acked 2613 Read 4 Done Done New false
4670 1615 Acked 2614 Read 1 Done Done New false
4671 1616 Acked 2615 Flush 0 Done Done New false
4672 1617 Acked 2616 Read 1 Done Done New false
4673 1618 Acked 2617 Write 3 Done Done New false
4674 1619 Acked 2618 Read 1 Done Done New false
4675 1620 Acked 2619 Write 6 Done Done New false
4676 1621 Acked 2620 Read 9 Done Done New false
4677 1622 Acked 2621 Flush 0 Done Done New false
4678 1623 Acked 2622 Write 7 Done Done New false
4679 1624 Acked 2623 Read 8 Done Done New false
4680 1625 Acked 2624 Read 9 Done Done New false
4681 1626 Acked 2625 Read 4 Done Done New false
4682 1627 Acked 2626 Write 2 Done Done New false
4683 1628 Acked 2627 Write 8 Done Done New false
4684 1629 Acked 2628 Read 5 Done Done New false
4685 1630 Acked 2629 Write 6 Done Done New false
4686 1631 Acked 2630 Write 9 Done Done New false
4687 1632 Acked 2631 Write 5 Done Done New false
4688 1633 Acked 2632 Write 2 Done Done New false
4689 1634 Acked 2633 Read 8 Done Done New false
4690 1635 Acked 2634 Read 8 Done Done New false
4691 1636 Acked 2635 Read 4 Done Done New false
4692 1637 Acked 2636 Read 7 Done Done New false
4693 1638 Acked 2637 Read 4 Done Done New false
4694 1639 Acked 2638 Flush 0 Done Done New false
4695 1640 Acked 2639 Read 4 Done Done New false
4696 1641 Acked 2640 Read 10 Done Done New false
4697 1642 Acked 2641 Read 6 Done Done New false
4698 1643 Acked 2642 Read 9 Done Done New false
4699 1644 Acked 2643 Flush 0 Done Done New false
4700 1645 Acked 2644 Flush 0 Done Done New false
4701 1646 Acked 2645 Write 1 Done Done New false
4702 1647 Acked 2646 Read 1 Done Done New false
4703 1648 Acked 2647 Read 6 Done Done New false
4704 1649 Acked 2648 Read 10 Done Done New false
4705 1650 Acked 2649 Read 1 Done Done New false
4706 1651 Acked 2650 Write 8 Done Done New false
4707 1652 Acked 2651 Read 7 Done Done New false
4708 1653 Acked 2652 Read 10 Done Done New false
4709 1654 Acked 2653 Read 9 Done Done New false
4710 1655 Acked 2654 Read 3 Done Done New false
4711 1656 Acked 2655 Read 2 Done Done New false
4712 1657 Acked 2656 Write 8 Done Done New false
4713 1658 Acked 2657 Write 7 Done Done New false
4714 1659 Acked 2658 Read 3 Done Done New false
4715 1660 Acked 2659 Read 2 Done Done New false
4716 1661 Acked 2660 Read 7 Done Done New false
4717 1662 Acked 2661 Read 1 Done Done New false
4718 1663 Acked 2662 Write 8 Done Done New false
4719 1664 Acked 2663 Read 10 Done Done New false
4720 1665 Acked 2664 Write 8 Done Done New false
4721 1666 Acked 2665 Flush 0 Done Done New false
4722 1667 Acked 2666 Read 10 Done Done New false
4723 1668 Acked 2667 Read 8 Done Done New false
4724 1669 Acked 2668 Write 8 Done Done New false
4725 1670 Acked 2669 Write 5 Done Done New false
4726 1671 Acked 2670 Flush 0 Done Done New false
4727 1672 Acked 2671 Write 2 Done Done New false
4728 1673 Acked 2672 Read 3 Done Done New false
4729 1674 Acked 2673 Write 4 Done Done New false
4730 1675 Acked 2674 Read 1 Done Done New false
4731 1676 Acked 2675 Read 6 Done Done New false
4732 1677 Acked 2676 Write 7 Done Done New false
4733 1678 Acked 2677 Write 9 Done Done New false
4734 1679 Acked 2678 Write 1 Done Done New false
4735 1680 Acked 2679 Read 10 Done Done New false
4736 1681 Acked 2680 Read 4 Done Done New false
4737 1682 Acked 2681 Write 8 Done Done New false
4738 1683 Acked 2682 Write 6 Done Done New false
4739 1684 Acked 2683 Read 4 Done Done New false
4740 1685 Acked 2684 Write 6 Done Done New false
4741 1686 Acked 2685 Read 8 Done Done New false
4742 1687 Acked 2686 Write 9 Done Done New false
4743 1688 Acked 2687 Read 4 Done Done New false
4744 1689 Acked 2688 Write 6 Done Done New false
4745 1690 Acked 2689 Flush 0 Done Done New false
4746 1691 Acked 2690 Write 1 Done Done New false
4747 1692 Acked 2691 Write 5 Done Done New false
4748 1693 Acked 2692 Write 2 Done Done New false
4749 1694 Acked 2693 Flush 0 Done Done New false
4750 1695 Acked 2694 Read 2 Done Done New false
4751 1696 Acked 2695 Read 1 Done Done New false
4752 1697 Acked 2696 Write 2 Done Done New false
4753 1698 Acked 2697 Read 10 Done Done New false
4754 1699 Acked 2698 Write 3 Done Done New false
4755 1700 Acked 2699 Write 6 Done Done New false
4756 1701 Acked 2700 Read 9 Done Done New false
4757 1702 Acked 2701 Read 2 Done Done New false
4758 1703 Acked 2702 Write 1 Done Done New false
4759 1704 Acked 2703 Write 9 Done Done New false
4760 1705 Acked 2704 Read 6 Done Done New false
4761 1706 Acked 2705 Flush 0 Done Done New false
4762 1707 Acked 2706 Read 4 Done Done New false
4763 1708 Acked 2707 Read 6 Done Done New false
4764 1709 Acked 2708 Read 4 Done Done New false
4765 1710 Acked 2709 Flush 0 Done Done New false
4766 1711 Acked 2710 Write 6 Done Done New false
4767 1712 Acked 2711 Write 7 Done Done New false
4768 1713 Acked 2712 Write 1 Done Done New false
4769 1714 Acked 2713 Read 6 Done Done New false
4770 1715 Acked 2714 Flush 0 Done Done New false
4771 1716 Acked 2715 Write 7 Done Done New false
4772 1717 Acked 2716 Write 3 Done Done New false
4773 1718 Acked 2717 Write 2 Done Done New false
4774 1719 Acked 2718 Read 2 Done Done New false
4775 1720 Acked 2719 Read 9 Done Done New false
4776 1721 Acked 2720 Flush 0 Done Done New false
4777 1722 Acked 2721 Read 5 Done Done New false
4778 1723 Acked 2722 Read 1 Done Done New false
4779 1724 Acked 2723 Read 2 Done Done New false
4780 1725 Acked 2724 Write 4 Done Done New false
4781 1726 Acked 2725 Read 4 Done Done New false
4782 1727 Acked 2726 Write 8 Done Done New false
4783 1728 Acked 2727 Read 4 Done Done New false
4784 1729 Acked 2728 Flush 0 Done Done New false
4785 1730 Acked 2729 Flush 0 Done Done New false
4786 1731 Acked 2730 Read 6 Done Done New false
4787 1732 Acked 2731 Read 8 Done Done New false
4788 1733 Acked 2732 Write 6 Done Done New false
4789 1734 Acked 2733 Read 4 Done Done New false
4790 1735 Acked 2734 Write 4 Done Done New false
4791 1736 Acked 2735 Read 5 Done Done New false
4792 1737 Acked 2736 Read 8 Done Done New false
4793 1738 Acked 2737 Flush 0 Done Done New false
4794 1739 Acked 2738 Write 3 Done Done New false
4795 1740 Acked 2739 Write 3 Done Done New false
4796 1741 Acked 2740 Read 8 Done Done New false
4797 1742 Acked 2741 Read 2 Done Done New false
4798 1743 Acked 2742 Read 10 Done Done New false
4799 1744 Acked 2743 Read 8 Done Done New false
4800 1745 Acked 2744 Write 5 Done Done New false
4801 1746 Acked 2745 Write 2 Done Done New false
4802 1747 Acked 2746 Write 1 Done Done New false
4803 1748 Acked 2747 Flush 0 Done Done New false
4804 1749 Acked 2748 Read 5 Done Done New false
4805 1750 Acked 2749 Read 2 Done Done New false
4806 1751 Acked 2750 Write 4 Done Done New false
4807 1752 Acked 2751 Write 9 Done Done New false
4808 1753 Acked 2752 Read 7 Done Done New false
4809 1754 Acked 2753 Read 1 Done Done New false
4810 1755 Acked 2754 Read 2 Done Done New false
4811 1756 Acked 2755 Write 8 Done Done New false
4812 1757 Acked 2756 Write 2 Done Done New false
4813 1758 Acked 2757 Read 7 Done Done New false
4814 1759 Acked 2758 Write 4 Done Done New false
4815 1760 Acked 2759 Read 8 Done Done New false
4816 1761 Acked 2760 Read 2 Done Done New false
4817 1762 Acked 2761 Read 10 Done Done New false
4818 1763 Acked 2762 Read 10 Done Done New false
4819 1764 Acked 2763 Read 6 Done Done New false
4820 1765 Acked 2764 Read 4 Done Done New false
4821 1766 Acked 2765 Read 2 Done Done New false
4822 1767 Acked 2766 Write 5 Done Done New false
4823 1768 Acked 2767 Read 3 Done Done New false
4824 1769 Acked 2768 Write 2 Done Done New false
4825 1770 Acked 2769 Write 1 Done Done New false
4826 1771 Acked 2770 Write 7 Done Done New false
4827 1772 Acked 2771 Read 5 Done Done New false
4828 1773 Acked 2772 Write 10 Done Done New false
4829 1774 Acked 2773 Read 3 Done Done New false
4830 1775 Acked 2774 Read 6 Done Done New false
4831 1776 Acked 2775 Write 7 Done Done New false
4832 1777 Acked 2776 Read 7 Done Done New false
4833 1778 Acked 2777 Write 7 Done Done New false
4834 1779 Acked 2778 Write 8 Done Done New false
4835 1780 Acked 2779 Read 8 Done Done New false
4836 1781 Acked 2780 Write 6 Done Done New false
4837 1782 Acked 2781 Read 5 Done Done New false
4838 1783 Acked 2782 Read 1 Done Done New false
4839 1784 Acked 2783 Read 5 Done Done New false
4840 1785 Acked 2784 Read 9 Done Done New false
4841 1786 Acked 2785 Read 7 Done Done New false
4842 1787 Acked 2786 Write 3 Done Done New false
4843 1788 Acked 2787 Read 10 Done Done New false
4844 1789 Acked 2788 Write 10 Done Done New false
4845 1790 Acked 2789 Flush 0 Done Done New false
4846 1791 Acked 2790 Write 10 Done Done New false
4847 1792 Acked 2791 Read 6 Done Done New false
4848 1793 Acked 2792 Write 9 Done Done New false
4849 1794 Acked 2793 Write 5 Done Done New false
4850 1795 Acked 2794 Write 5 Done Done New false
4851 1796 Acked 2795 Read 9 Done Done New false
4852 1797 Acked 2796 Flush 0 Done Done New false
4853 1798 Acked 2797 Read 7 Done Done New false
4854 1799 Acked 2798 Write 1 Done Done New false
4855 1800 Acked 2799 Read 8 Done Done New false
4856 1801 Acked 2800 Read 6 Done Done New false
4857 1802 Acked 2801 Write 5 Done Done New false
4858 1803 Acked 2802 Read 8 Done Done New false
4859 1804 Acked 2803 Read 2 Done Done New false
4860 1805 Acked 2804 Write 1 Done Done New false
4861 1806 Acked 2805 Read 6 Done Done New false
4862 1807 Acked 2806 Write 2 Done Done New false
4863 1808 Acked 2807 Read 10 Done Done New false
4864 1809 Acked 2808 Read 10 Done Done New false
4865 1810 Acked 2809 Read 3 Done Done New false
4866 1811 Acked 2810 Write 6 Done Done New false
4867 1812 Acked 2811 Read 10 Done Done New false
4868 1813 Acked 2812 Read 8 Done Done New false
4869 1814 Acked 2813 Read 8 Done Done New false
4870 STATES DS:0 DS:1 DS:2 TOTAL
4871 New 0 0 302 302
4872 Sent 0 0 0 0
4873 Done 302 302 0 604
4874 Skipped 0 0 0 0
4875 Error 0 0 0 0
4876 Last Flush: 2796 2796 2511
4877 Downstairs last five completed: 2511 2510 2509 2508 2507 2506
4878 Upstairs last five completed: 1814 1813 1812 1811 1810 1809
4879 CLIENT: Up:0 ds:302 act:2
4880 [8830] got start action
4881 [8830] I am going to restart
4882 Starting downstairs at port 8830
4883 Make output file at "/tmp/dsc/downstairs-8830.txt"
4884 [8830][2] reports Starting
4885 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1179)
4886 [8830][2] reports Running
48872023-09-22T23:24:49.708ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
48882023-09-22T23:24:49.708ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
48892023-09-22T23:24:49.708ZINFOcrucible: [2] client re-new 303 jobs since flush 2511 = downstairs
48902023-09-22T23:24:49.709ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
48912023-09-22T23:24:49.709ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
48922023-09-22T23:24:49.709ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
48932023-09-22T23:24:49.709ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
48942023-09-22T23:24:49.709ZINFOcrucible: [2] send last flush ID to this DS: 2511
48952023-09-22T23:24:49.710ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
48962023-09-22T23:24:49.710ZINFOcrucible: [2] Replied this last flush ID: 2511
48972023-09-22T23:24:49.710ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
48982023-09-22T23:24:49.710ZINFOcrucible: [2] Transition from Offline to Replay
48992023-09-22T23:24:49.710ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
49002023-09-22T23:24:49.710ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
49012023-09-22T23:24:49.710ZINFOcrucible: [2] Transition from Replay to Active
49022023-09-22T23:24:49.710ZINFOcrucible: [2] Starts cmd_loop
49032023-09-22T23:24:49.710ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
49042023-09-22T23:24:49.710ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
49052023-09-22T23:24:49.710ZINFOcrucible: Set check for repair
49062023-09-22T23:24:50.710ZWARNcrucible: [2] flow control end
49072023-09-22T23:24:50.711ZINFOcrucible: Checking if live repair is needed
49082023-09-22T23:24:50.711ZINFOcrucible: No Live Repair required at this time
4909 ----------------------------------------------------------------
4910 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4911 Downstairs last five completed: 2814 2813 2812 2811 2810 2809
4912 Upstairs last five completed: 1815 1814 1813 1812 1811 1810
4913 CLIENT: Up:0 ds:0 act:3
4914 Replay: All jobs finished, all DS active.
4915 Main task has work to do, go find it
4916 got dsc Stop(2)
4917 stop 2
4918 Sep 22 23:24:51.623 INFO request completed, latency_us: 541, response_code: 204, uri: [8830] Got stop action so:false kr:false
4919 /stop/cid/2, method: POST, req_id: 1c6ce9b8-56a1-4c04-8293-73cea7ad087f, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
4920 Sep 22 23:24:51.624 INFO request completed, latency_us: 332, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9d96274d-43ee-44f0-8269-65b9ab47f4db, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
49212023-09-22T23:24:51.624ZWARNcrucible: [2] downstairs disconnected
49222023-09-22T23:24:51.624ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
49232023-09-22T23:24:51.624ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
49242023-09-22T23:24:51.624ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
49252023-09-22T23:24:51.624ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
49262023-09-22T23:24:51.624ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
49272023-09-22T23:24:51.624ZWARNcrucible: [2] pm_task rx.recv() is None
4928 [8830] Exited with: ExitStatus(unix_wait_status(9))
4929 [8830][2] reports Exit
4930 Sep 22 23:24:55.626 INFO request completed, latency_us: 262, response_code: 200, uri: /state/cid/2, method: GET, req_id: d6d78e90-cf10-4f36-8176-4c3d0084b588, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
4931 001/300 Write block 2452 len 24576 data: 4 3 2 2 2 2
4932 002/300 Write block 501 len 32768 data: 3 3 3 2 2 2 2 2
4933 003/300 Write block 619 len 20480 data: 2 2 2 2 2
4934 004/300 Write block 4898 len 36864 data: 3 3 3 3 3 4 3 3 3
4935 005/300 Read block 420 len 40960
4936 006/300 Read block 4319 len 12288
4937 007/300 Read block 4935 len 20480
4938 008/300 Read block 4365 len 8192
4939 009/300 Read block 4660 len 8192
4940 010/300 Write block 4928 len 20480 data: 4 4 3 3 3
4941 011/300 Write block 3833 len 8192 data: 3 3
4942 012/300 Flush
4943 013/300 Read block 4656 len 32768
4944 014/300 Write block 1476 len 40960 data: 5 5 5 4 4 3 2 2 2 2
4945 015/300 Write block 988 len 12288 data: 3 3 3
4946 016/300 Write block 2470 len 32768 data: 2 2 3 3 2 2 3 2
4947 017/300 Read block 2053 len 40960
4948 018/300 Read block 591 len 4096
4949 019/300 Write block 3957 len 36864 data: 2 2 2 3 3 3 4 5 5
4950 020/300 Write block 4444 len 28672 data: 2 2 2 2 3 3 3
4951 021/300 Read block 4344 len 4096
4952 022/300 Write block 1360 len 8192 data: 2 2
4953 023/300 Flush
4954 024/300 Write block 369 len 20480 data: 2 2 2 2 2
4955 025/300 Write block 2425 len 28672 data: 4 4 4 4 4 3 3
4956 026/300 Write block 577 len 16384 data: 4 3 3 2
4957 027/300 Read block 1081 len 24576
4958 028/300 Read block 1102 len 20480
4959 029/300 Read block 3111 len 40960
4960 030/300 Read block 4858 len 40960
4961 031/300 Read block 530 len 20480
4962 032/300 Read block 2413 len 28672
4963 033/300 Read block 4710 len 28672
4964 034/300 Read block 4174 len 36864
4965 035/300 Write block 662 len 32768 data: 3 3 2 3 3 2 2 2
4966 036/300 Read block 1733 len 20480
4967 037/300 Write block 3987 len 4096 data: 5
4968 038/300 Write block 2675 len 40960 data: 2 2 2 2 2 2 2 2 2 3
4969 039/300 Flush
4970 040/300 Write block 2769 len 36864 data: 2 2 2 2 2 2 2 3 3
4971 041/300 Write block 2927 len 12288 data: 2 2 2
4972 042/300 Write block 4575 len 28672 data: 3 3 3 2 2 2 3
4973 043/300 Read block 3797 len 24576
4974 044/300 Flush
4975 045/300 Read block 426 len 36864
4976 046/300 Write block 3111 len 4096 data: 2
4977 047/300 Read block 4051 len 36864
4978 048/300 Write block 4767 len 16384 data: 2 2 2 2
4979 049/300 Write block 4592 len 4096 data: 2
4980 050/300 Read block 751 len 12288
4981 051/300 Write block 2234 len 8192 data: 2 2
4982 052/300 Read block 2902 len 4096
4983 053/300 Write block 4910 len 32768 data: 3 4 5 5 5 5 5 5
4984 054/300 Write block 3174 len 32768 data: 2 2 2 2 2 2 2 3
4985 055/300 Read block 3175 len 24576
4986 056/300 Read block 4084 len 8192
4987 057/300 Read block 1528 len 8192
4988 058/300 Write block 4813 len 4096 data: 2
4989 059/300 Read block 843 len 40960
4990 060/300 Write block 2190 len 36864 data: 3 2 2 2 2 2 3 3 3
4991 061/300 Read block 1157 len 24576
4992 062/300 Read block 3210 len 12288
4993 063/300 Write block 2755 len 8192 data: 2 2
4994 064/300 Write block 3446 len 16384 data: 3 2 2 2
4995 065/300 Write block 1456 len 40960 data: 2 3 3 3 3 3 3 3 3 3
4996 066/300 Read block 1770 len 32768
4997 067/300 Write block 2763 len 28672 data: 3 3 3 3 3 3 3
4998 068/300 Read block 2966 len 8192
4999 069/300 Flush
5000 070/300 Write block 3055 len 4096 data: 3
5001 071/300 Read block 4553 len 32768
5002 072/300 Read block 1186 len 12288
5003 073/300 Flush
5004 074/300 Write block 2110 len 24576 data: 3 3 3 3 3 3
5005 075/300 Read block 4949 len 16384
5006 076/300 Write block 1765 len 12288 data: 3 3 3
5007 077/300 Write block 4426 len 4096 data: 4
5008 078/300 Write block 2449 len 4096 data: 4
5009 079/300 Write block 1931 len 12288 data: 2 2 2
5010 080/300 Read block 1004 len 8192
5011 081/300 Read block 2538 len 16384
5012 082/300 Write block 592 len 40960 data: 4 3 3 4 4 5 5 4 3 3
5013 083/300 Read block 841 len 4096
5014 084/300 Write block 873 len 36864 data: 2 2 2 2 2 2 2 2 2
5015 085/300 Write block 666 len 32768 data: 4 3 3 3 2 3 3 2
5016 086/300 Read block 4235 len 4096
5017 087/300 Read block 1204 len 16384
5018 088/300 Read block 3917 len 12288
5019 089/300 Read block 328 len 28672
5020 090/300 Write block 2617 len 36864 data: 2 2 2 3 4 5 5 5 5
5021 091/300 Write block 724 len 12288 data: 4 4 6
5022 092/300 Write block 2860 len 12288 data: 3 3 2
5023 093/300 Read block 2738 len 28672
5024 094/300 Read block 3745 len 24576
5025 095/300 Read block 2789 len 28672
5026 096/300 Write block 385 len 4096 data: 4
5027 097/300 Flush
5028 098/300 Flush
5029 099/300 Read block 40 len 24576
5030 100/300 Read block 3777 len 4096
5031 101/300 Write block 750 len 4096 data: 2
5032 102/300 Read block 3433 len 24576
5033 103/300 Write block 165 len 4096 data: 2
5034 104/300 Write block 4645 len 16384 data: 4 4 5 4
5035 105/300 Read block 3498 len 24576
5036 106/300 Read block 2700 len 28672
5037 107/300 Write block 760 len 28672 data: 5 5 5 5 4 4 3
5038 108/300 Flush
5039 109/300 Write block 246 len 28672 data: 4 3 3 3 3 3 4
5040 110/300 Write block 1521 len 4096 data: 2
5041 111/300 Write block 1316 len 4096 data: 4
5042 112/300 Write block 2512 len 24576 data: 5 5 5 5 5 5
5043 113/300 Read block 2353 len 24576
5044 114/300 Read block 2528 len 16384
5045 115/300 Read block 1711 len 16384
5046 116/300 Read block 597 len 20480
5047 117/300 Read block 2232 len 32768
5048 118/300 Flush
5049 119/300 Write block 160 len 16384 data: 2 2 2 2
5050 120/300 Read block 3947 len 4096
5051 121/300 Write block 3682 len 16384 data: 2 2 2 3
5052 122/300 Flush
5053 123/300 Read block 1085 len 32768
5054 124/300 Write block 1698 len 12288 data: 3 3 3
5055 125/300 Write block 257 len 16384 data: 2 4 3 3
5056 126/300 Flush
5057 127/300 Write block 4847 len 28672 data: 2 2 2 2 2 2 3
5058 128/300 Write block 4385 len 24576 data: 3 2 2 2 2 2
5059 129/300 Read block 2082 len 16384
5060 130/300 Write block 1435 len 20480 data: 4 3 3 2 2
5061 131/300 Read block 3892 len 8192
5062 132/300 Flush
5063 133/300 Flush
5064 134/300 Read block 367 len 36864
5065 135/300 Read block 4789 len 12288
5066 136/300 Read block 2204 len 24576
5067 137/300 Read block 4657 len 40960
5068 138/300 Read block 2641 len 8192
5069 139/300 Read block 2757 len 32768
5070 140/300 Flush
5071 141/300 Read block 2118 len 24576
5072 142/300 Write block 4799 len 24576 data: 2 2 2 2 2 2
5073 143/300 Flush
5074 144/300 Read block 333 len 32768
5075 145/300 Read block 1369 len 8192
5076 146/300 Write block 27 len 20480 data: 3 3 3 3 3
5077 147/300 Write block 1795 len 20480 data: 5 5 5 4 3
5078 148/300 Write block 28 len 28672 data: 4 4 4 4 3 3 3
5079 149/300 Read block 3385 len 16384
5080 150/300 Write block 909 len 36864 data: 2 2 2 2 2 2 2 2 2
5081 151/300 Write block 868 len 36864 data: 2 2 2 2 2 3 3 3 3
5082 152/300 Write block 4435 len 32768 data: 3 3 4 3 3 3 3 2
5083 153/300 Read block 792 len 28672
5084 154/300 Write block 3118 len 32768 data: 2 2 2 2 3 4 4 4
5085 155/300 Read block 903 len 12288
5086 156/300 Flush
5087 157/300 Write block 3665 len 20480 data: 3 2 3 3 3
5088 158/300 Write block 4726 len 40960 data: 2 2 3 3 2 2 2 2 2 3
5089 159/300 Read block 4872 len 20480
5090 160/300 Read block 2341 len 24576
5091 161/300 Read block 679 len 16384
5092 162/300 Read block 3661 len 4096
5093 163/300 Read block 4357 len 32768
5094 164/300 Read block 2419 len 24576
5095 165/300 Read block 189 len 16384
5096 166/300 Write block 2621 len 28672 data: 5 6 6 6 6 4 3
5097 167/300 Write block 3285 len 36864 data: 2 3 3 3 2 3 2 3 3
5098 168/300 Write block 2928 len 16384 data: 3 3 2 2
5099 169/300 Write block 4470 len 16384 data: 3 2 2 2
5100 170/300 Read block 3130 len 36864
5101 171/300 Write block 3865 len 20480 data: 2 2 3 3 4
5102 172/300 Write block 4510 len 16384 data: 2 2 2 2
5103 173/300 Read block 3995 len 40960
5104 174/300 Read block 515 len 4096
5105 175/300 Write block 880 len 36864 data: 3 3 2 2 3 3 3 3 3
5106 176/300 Flush
5107 177/300 Write block 1361 len 4096 data: 3
5108 178/300 Read block 3846 len 8192
5109 179/300 Write block 969 len 36864 data: 2 2 2 2 2 2 2 3 3
5110 180/300 Write block 4805 len 40960 data: 2 2 2 2 2 2 2 2 3 2
5111 181/300 Write block 4448 len 4096 data: 4
5112 182/300 Read block 556 len 24576
5113 183/300 Read block 78 len 40960
5114 184/300 Read block 3241 len 28672
5115 185/300 Read block 2346 len 40960
5116 186/300 Read block 2408 len 28672
5117 187/300 Write block 3675 len 16384 data: 2 2 2 2
5118 188/300 Write block 327 len 16384 data: 3 2 2 2
5119 189/300 Read block 3509 len 28672
5120 190/300 Read block 2027 len 8192
5121 191/300 Read block 3133 len 28672
5122 192/300 Write block 3734 len 4096 data: 2
5123 193/300 Flush
5124 194/300 Write block 2307 len 12288 data: 2 2 3
5125 195/300 Read block 3953 len 4096
5126 196/300 Write block 883 len 16384 data: 3 4 4 4
5127 197/300 Write block 994 len 8192 data: 3 3
5128 198/300 Read block 4356 len 16384
5129 199/300 Read block 2683 len 20480
5130 200/300 Write block 4322 len 16384 data: 4 4 4 5
5131 201/300 Read block 4981 len 32768
5132 202/300 Write block 3718 len 4096 data: 5
5133 203/300 Flush
5134 204/300 Write block 506 len 4096 data: 3
5135 205/300 Read block 656 len 4096
5136 206/300 Write block 2092 len 36864 data: 2 2 2 2 2 2 2 2 2
5137 207/300 Write block 1234 len 40960 data: 2 2 2 2 3 3 3 3 3 3
5138 208/300 Read block 678 len 32768
5139 209/300 Write block 4400 len 16384 data: 4 4 6 5
5140 210/300 Write block 379 len 8192 data: 2 3
5141 211/300 Read block 3092 len 12288
5142 212/300 Read block 4340 len 32768
5143 213/300 Write block 704 len 32768 data: 2 2 2 2 2 2 2 2
5144 214/300 Read block 3914 len 20480
5145 215/300 Write block 4733 len 4096 data: 3
5146 216/300 Write block 4167 len 4096 data: 2
5147 217/300 Write block 4156 len 20480 data: 2 2 2 2 2
5148 218/300 Write block 4762 len 20480 data: 3 3 3 2 2
5149 219/300 Write block 2956 len 12288 data: 2 2 2
5150 220/300 Read block 396 len 12288
5151 221/300 Write block 626 len 16384 data: 2 2 2 2
5152 222/300 Write block 3746 len 4096 data: 3
5153 223/300 Write block 4233 len 12288 data: 3 3 3
5154 224/300 Read block 4634 len 4096
5155 225/300 Write block 420 len 12288 data: 4 4 4
5156 226/300 Read block 1020 len 32768
5157 227/300 Write block 4879 len 8192 data: 3 3
5158 228/300 Read block 2434 len 16384
5159 229/300 Write block 1371 len 32768 data: 3 3 2 2 3 2 2 2
5160 230/300 Flush
5161 231/300 Read block 3991 len 8192
5162 232/300 Flush
5163 233/300 Read block 3077 len 16384
5164 234/300 Flush
5165 235/300 Read block 527 len 20480
5166 236/300 Read block 2188 len 20480
5167 237/300 Flush
5168 238/300 Read block 4814 len 4096
5169 239/300 Write block 250 len 12288 data: 4 4 5
5170 240/300 Read block 3761 len 28672
5171 241/300 Write block 4191 len 40960 data: 3 4 4 2 2 2 2 2 2 2
5172 242/300 Read block 1565 len 8192
5173 243/300 Flush
5174 244/300 Write block 4614 len 40960 data: 3 3 3 2 2 2 2 3 3 3
5175 245/300 Read block 4097 len 12288
5176 246/300 Read block 4016 len 12288
5177 247/300 Write block 4041 len 28672 data: 2 2 2 2 2 4 4
5178 248/300 Read block 4122 len 12288
5179 249/300 Write block 4142 len 8192 data: 2 2
5180 250/300 Read block 3687 len 16384
5181 251/300 Read block 3356 len 20480
5182 252/300 Read block 2670 len 4096
5183 253/300 Flush
5184 254/300 Write block 3301 len 36864 data: 3 3 2 2 3 3 3 2 2
5185 255/300 Read block 3921 len 20480
5186 256/300 Read block 2575 len 16384
5187 257/300 Read block 953 len 28672
5188 258/300 Flush
5189 259/300 Flush
5190 260/300 Read block 4415 len 40960
5191 261/300 Read block 2838 len 8192
5192 262/300 Read block 1532 len 40960
5193 263/300 Read block 531 len 20480
5194 264/300 Write block 4682 len 20480 data: 3 2 2 2 3
5195 265/300 Write block 3696 len 32768 data: 2 2 2 2 3 3 3 3
5196 266/300 Read block 4828 len 24576
5197 267/300 Read block 3526 len 16384
5198 268/300 Read block 549 len 4096
5199 269/300 Write block 1063 len 36864 data: 3 3 2 2 2 2 3 3 2
5200 270/300 Write block 3258 len 24576 data: 4 4 5 4 4 3
5201 271/300 Read block 2554 len 4096
5202 272/300 Write block 4883 len 36864 data: 2 2 2 2 2 2 3 3 4
5203 273/300 Read block 1770 len 16384
5204 274/300 Read block 1297 len 4096
5205 275/300 Read block 4226 len 12288
5206 276/300 Read block 3887 len 36864
5207 277/300 Flush
5208 278/300 Read block 3086 len 32768
5209 279/300 Write block 4127 len 16384 data: 3 4 5 4
5210 280/300 Write block 1647 len 12288 data: 2 2 2
5211 281/300 Write block 3302 len 12288 data: 4 3 3
5212 282/300 Write block 3418 len 24576 data: 2 2 2 2 2 3
5213 283/300 Write block 4523 len 4096 data: 4
5214 284/300 Read block 1199 len 40960
5215 285/300 Write block 4375 len 28672 data: 3 3 3 3 3 3 3
5216 286/300 Write block 538 len 4096 data: 2
5217 287/300 Read block 4447 len 4096
5218 288/300 Read block 438 len 28672
5219 289/300 Write block 2732 len 24576 data: 2 2 2 2 2 2
5220 290/300 Read block 4142 len 32768
5221 291/300 Read block 2821 len 40960
5222 292/300 Write block 1830 len 20480 data: 2 2 2 2 2
5223 293/300 Read block 3749 len 32768
5224 294/300 Flush
5225 295/300 Read block 2144 len 32768
5226 296/300 Write block 2507 len 24576 data: 2 3 3 3 5 6
5227 297/300 Read block 3253 len 28672
5228 298/300 Read block 4617 len 28672
5229 299/300 Write block 3148 len 4096 data: 3
5230 300/300 Read block 4988 len 16384
5231 Main task has work to do, go find it
5232 got dsc Start(2)
5233 start 2
5234 Sep 22 23:24:56.518 INFO request completed, latency_us: 301, response_code: 204, uri: /start/cid/2, method: POST, req_id: e03ef497-b489-4956-bdad-46253cdc4092, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
5235 Replay: started 2, returned:Ok(())
5236 ----------------------------------------------------------------
5237 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5238 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
5239 1816 Acked 2815 Write 6 Done Done New false
5240 1817 Acked 2816 Write 8 Done Done New false
5241 1818 Acked 2817 Write 5 Done Done New false
5242 1819 Acked 2818 Write 9 Done Done New false
5243 1820 Acked 2819 Read 10 Done Done New false
5244 1821 Acked 2820 Read 3 Done Done New false
5245 1822 Acked 2821 Read 5 Done Done New false
5246 1823 Acked 2822 Read 2 Done Done New false
5247 1824 Acked 2823 Read 2 Done Done New false
5248 1825 Acked 2824 Write 5 Done Done New false
5249 1826 Acked 2825 Write 2 Done Done New false
5250 1827 Acked 2826 Flush 0 Done Done New false
5251 1828 Acked 2827 Read 8 Done Done New false
5252 1829 Acked 2828 Write 10 Done Done New false
5253 1830 Acked 2829 Write 3 Done Done New false
5254 1831 Acked 2830 Write 8 Done Done New false
5255 1832 Acked 2831 Read 10 Done Done New false
5256 1833 Acked 2832 Read 1 Done Done New false
5257 1834 Acked 2833 Write 9 Done Done New false
5258 1835 Acked 2834 Write 7 Done Done New false
5259 1836 Acked 2835 Read 1 Done Done New false
5260 1837 Acked 2836 Write 2 Done Done New false
5261 1838 Acked 2837 Flush 0 Done Done New false
5262 1839 Acked 2838 Write 5 Done Done New false
5263 1840 Acked 2839 Write 7 Done Done New false
5264 1841 Acked 2840 Write 4 Done Done New false
5265 1842 Acked 2841 Read 6 Done Done New false
5266 1843 Acked 2842 Read 5 Done Done New false
5267 1844 Acked 2843 Read 10 Done Done New false
5268 1845 Acked 2844 Read 10 Done Done New false
5269 1846 Acked 2845 Read 5 Done Done New false
5270 1847 Acked 2846 Read 7 Done Done New false
5271 1848 Acked 2847 Read 7 Done Done New false
5272 1849 Acked 2848 Read 9 Done Done New false
5273 1850 Acked 2849 Write 8 Done Done New false
5274 1851 Acked 2850 Read 5 Done Done New false
5275 1852 Acked 2851 Write 1 Done Done New false
5276 1853 Acked 2852 Write 10 Done Done New false
5277 1854 Acked 2853 Flush 0 Done Done New false
5278 1855 Acked 2854 Write 9 Done Done New false
5279 1856 Acked 2855 Write 3 Done Done New false
5280 1857 Acked 2856 Write 7 Done Done New false
5281 1858 Acked 2857 Read 6 Done Done New false
5282 1859 Acked 2858 Flush 0 Done Done New false
5283 1860 Acked 2859 Read 9 Done Done New false
5284 1861 Acked 2860 Write 1 Done Done New false
5285 1862 Acked 2861 Read 9 Done Done New false
5286 1863 Acked 2862 Write 4 Done Done New false
5287 1864 Acked 2863 Write 1 Done Done New false
5288 1865 Acked 2864 Read 3 Done Done New false
5289 1866 Acked 2865 Write 2 Done Done New false
5290 1867 Acked 2866 Read 1 Done Done New false
5291 1868 Acked 2867 Write 8 Done Done New false
5292 1869 Acked 2868 Write 8 Done Done New false
5293 1870 Acked 2869 Read 6 Done Done New false
5294 1871 Acked 2870 Read 2 Done Done New false
5295 1872 Acked 2871 Read 2 Done Done New false
5296 1873 Acked 2872 Write 1 Done Done New false
5297 1874 Acked 2873 Read 10 Done Done New false
5298 1875 Acked 2874 Write 9 Done Done New false
5299 1876 Acked 2875 Read 6 Done Done New false
5300 1877 Acked 2876 Read 3 Done Done New false
5301 1878 Acked 2877 Write 2 Done Done New false
5302 1879 Acked 2878 Write 4 Done Done New false
5303 1880 Acked 2879 Write 10 Done Done New false
5304 1881 Acked 2880 Read 8 Done Done New false
5305 1882 Acked 2881 Write 7 Done Done New false
5306 1883 Acked 2882 Read 2 Done Done New false
5307 1884 Acked 2883 Flush 0 Done Done New false
5308 1885 Acked 2884 Write 1 Done Done New false
5309 1886 Acked 2885 Read 8 Done Done New false
5310 1887 Acked 2886 Read 3 Done Done New false
5311 1888 Acked 2887 Flush 0 Done Done New false
5312 1889 Acked 2888 Write 6 Done Done New false
5313 1890 Acked 2889 Read 4 Done Done New false
5314 1891 Acked 2890 Write 3 Done Done New false
5315 1892 Acked 2891 Write 1 Done Done New false
5316 1893 Acked 2892 Write 1 Done Done New false
5317 1894 Acked 2893 Write 3 Done Done New false
5318 1895 Acked 2894 Read 2 Done Done New false
5319 1896 Acked 2895 Read 4 Done Done New false
5320 1897 Acked 2896 Write 10 Done Done New false
5321 1898 Acked 2897 Read 1 Done Done New false
5322 1899 Acked 2898 Flush 0 Done Done New false
5323 1900 Acked 2899 Write 9 Done Done New false
5324 1901 Acked 2900 Write 8 Done Done New false
5325 1902 Acked 2901 Read 1 Done Done New false
5326 1903 Acked 2902 Read 4 Done Done New false
5327 1904 Acked 2903 Read 3 Done Done New false
5328 1905 Acked 2904 Read 7 Done Done New false
5329 1906 Acked 2905 Write 9 Done Done New false
5330 1907 Acked 2906 Write 3 Done Done New false
5331 1908 Acked 2907 Write 3 Done Done New false
5332 1909 Acked 2908 Read 7 Done Done New false
5333 1910 Acked 2909 Read 6 Done Done New false
5334 1911 Acked 2910 Read 7 Done Done New false
5335 1912 Acked 2911 Write 1 Done Done New false
5336 1913 Acked 2912 Flush 0 Done Done New false
5337 1914 Acked 2913 Flush 0 Done Done New false
5338 1915 Acked 2914 Read 6 Done Done New false
5339 1916 Acked 2915 Read 1 Done Done New false
5340 1917 Acked 2916 Write 1 Done Done New false
5341 1918 Acked 2917 Read 6 Done Done New false
5342 1919 Acked 2918 Write 1 Done Done New false
5343 1920 Acked 2919 Write 4 Done Done New false
5344 1921 Acked 2920 Read 6 Done Done New false
5345 1922 Acked 2921 Read 7 Done Done New false
5346 1923 Acked 2922 Write 7 Done Done New false
5347 1924 Acked 2923 Flush 0 Done Done New false
5348 1925 Acked 2924 Write 7 Done Done New false
5349 1926 Acked 2925 Write 1 Done Done New false
5350 1927 Acked 2926 Write 1 Done Done New false
5351 1928 Acked 2927 Write 6 Done Done New false
5352 1929 Acked 2928 Read 6 Done Done New false
5353 1930 Acked 2929 Read 4 Done Done New false
5354 1931 Acked 2930 Read 4 Done Done New false
5355 1932 Acked 2931 Read 5 Done Done New false
5356 1933 Acked 2932 Read 8 Done Done New false
5357 1934 Acked 2933 Flush 0 Done Done New false
5358 1935 Acked 2934 Write 4 Done Done New false
5359 1936 Acked 2935 Read 1 Done Done New false
5360 1937 Acked 2936 Write 4 Done Done New false
5361 1938 Acked 2937 Flush 0 Done Done New false
5362 1939 Acked 2938 Read 8 Done Done New false
5363 1940 Acked 2939 Write 3 Done Done New false
5364 1941 Acked 2940 Write 4 Done Done New false
5365 1942 Acked 2941 Flush 0 Done Done New false
5366 1943 Acked 2942 Write 7 Done Done New false
5367 1944 Acked 2943 Write 6 Done Done New false
5368 1945 Acked 2944 Read 4 Done Done New false
5369 1946 Acked 2945 Write 5 Done Done New false
5370 1947 Acked 2946 Read 2 Done Done New false
5371 1948 Acked 2947 Flush 0 Done Done New false
5372 1949 Acked 2948 Flush 0 Done Done New false
5373 1950 Acked 2949 Read 9 Done Done New false
5374 1951 Acked 2950 Read 3 Done Done New false
5375 1952 Acked 2951 Read 6 Done Done New false
5376 1953 Acked 2952 Read 10 Done Done New false
5377 1954 Acked 2953 Read 2 Done Done New false
5378 1955 Acked 2954 Read 8 Done Done New false
5379 1956 Acked 2955 Flush 0 Done Done New false
5380 1957 Acked 2956 Read 6 Done Done New false
5381 1958 Acked 2957 Write 6 Done Done New false
5382 1959 Acked 2958 Flush 0 Done Done New false
5383 1960 Acked 2959 Read 8 Done Done New false
5384 1961 Acked 2960 Read 2 Done Done New false
5385 1962 Acked 2961 Write 5 Done Done New false
5386 1963 Acked 2962 Write 5 Done Done New false
5387 1964 Acked 2963 Write 7 Done Done New false
5388 1965 Acked 2964 Read 4 Done Done New false
5389 1966 Acked 2965 Write 9 Done Done New false
5390 1967 Acked 2966 Write 9 Done Done New false
5391 1968 Acked 2967 Write 8 Done Done New false
5392 1969 Acked 2968 Read 7 Done Done New false
5393 1970 Acked 2969 Write 8 Done Done New false
5394 1971 Acked 2970 Read 3 Done Done New false
5395 1972 Acked 2971 Flush 0 Done Done New false
5396 1973 Acked 2972 Write 5 Done Done New false
5397 1974 Acked 2973 Write 10 Done Done New false
5398 1975 Acked 2974 Read 5 Done Done New false
5399 1976 Acked 2975 Read 6 Done Done New false
5400 1977 Acked 2976 Read 4 Done Done New false
5401 1978 Acked 2977 Read 1 Done Done New false
5402 1979 Acked 2978 Read 8 Done Done New false
5403 1980 Acked 2979 Read 6 Done Done New false
5404 1981 Acked 2980 Read 4 Done Done New false
5405 1982 Acked 2981 Write 7 Done Done New false
5406 1983 Acked 2982 Write 9 Done Done New false
5407 1984 Acked 2983 Write 4 Done Done New false
5408 1985 Acked 2984 Write 4 Done Done New false
5409 1986 Acked 2985 Read 9 Done Done New false
5410 1987 Acked 2986 Write 5 Done Done New false
5411 1988 Acked 2987 Write 4 Done Done New false
5412 1989 Acked 2988 Read 10 Done Done New false
5413 1990 Acked 2989 Read 1 Done Done New false
5414 1991 Acked 2990 Write 9 Done Done New false
5415 1992 Acked 2991 Flush 0 Done Done New false
5416 1993 Acked 2992 Write 1 Done Done New false
5417 1994 Acked 2993 Read 2 Done Done New false
5418 1995 Acked 2994 Write 9 Done Done New false
5419 1996 Acked 2995 Write 10 Done Done New false
5420 1997 Acked 2996 Write 1 Done Done New false
5421 1998 Acked 2997 Read 6 Done Done New false
5422 1999 Acked 2998 Read 10 Done Done New false
5423 2000 Acked 2999 Read 7 Done Done New false
5424 2001 Acked 3000 Read 10 Done Done New false
5425 2002 Acked 3001 Read 7 Done Done New false
5426 2003 Acked 3002 Write 4 Done Done New false
5427 2004 Acked 3003 Write 4 Done Done New false
5428 2005 Acked 3004 Read 7 Done Done New false
5429 2006 Acked 3005 Read 2 Done Done New false
5430 2007 Acked 3006 Read 7 Done Done New false
5431 2008 Acked 3007 Write 1 Done Done New false
5432 2009 Acked 3008 Flush 0 Done Done New false
5433 2010 Acked 3009 Write 3 Done Done New false
5434 2011 Acked 3010 Read 1 Done Done New false
5435 2012 Acked 3011 Write 4 Done Done New false
5436 2013 Acked 3012 Write 2 Done Done New false
5437 2014 Acked 3013 Read 4 Done Done New false
5438 2015 Acked 3014 Read 5 Done Done New false
5439 2016 Acked 3015 Write 4 Done Done New false
5440 2017 Acked 3016 Read 8 Done Done New false
5441 2018 Acked 3017 Write 1 Done Done New false
5442 2019 Acked 3018 Flush 0 Done Done New false
5443 2020 Acked 3019 Write 1 Done Done New false
5444 2021 Acked 3020 Read 1 Done Done New false
5445 2022 Acked 3021 Write 9 Done Done New false
5446 2023 Acked 3022 Write 10 Done Done New false
5447 2024 Acked 3023 Read 8 Done Done New false
5448 2025 Acked 3024 Write 4 Done Done New false
5449 2026 Acked 3025 Write 2 Done Done New false
5450 2027 Acked 3026 Read 3 Done Done New false
5451 2028 Acked 3027 Read 8 Done Done New false
5452 2029 Acked 3028 Write 8 Done Done New false
5453 2030 Acked 3029 Read 5 Done Done New false
5454 2031 Acked 3030 Write 1 Done Done New false
5455 2032 Acked 3031 Write 1 Done Done New false
5456 2033 Acked 3032 Write 5 Done Done New false
5457 2034 Acked 3033 Write 5 Done Done New false
5458 2035 Acked 3034 Write 3 Done Done New false
5459 2036 Acked 3035 Read 3 Done Done New false
5460 2037 Acked 3036 Write 4 Done Done New false
5461 2038 Acked 3037 Write 1 Done Done New false
5462 2039 Acked 3038 Write 3 Done Done New false
5463 2040 Acked 3039 Read 1 Done Done New false
5464 2041 Acked 3040 Write 3 Done Done New false
5465 2042 Acked 3041 Read 8 Done Done New false
5466 2043 Acked 3042 Write 2 Done Done New false
5467 2044 Acked 3043 Read 4 Done Done New false
5468 2045 Acked 3044 Write 8 Done Done New false
5469 2046 Acked 3045 Flush 0 Done Done New false
5470 2047 Acked 3046 Read 2 Done Done New false
5471 2048 Acked 3047 Flush 0 Done Done New false
5472 2049 Acked 3048 Read 4 Done Done New false
5473 2050 Acked 3049 Flush 0 Done Done New false
5474 2051 Acked 3050 Read 5 Done Done New false
5475 2052 Acked 3051 Read 5 Done Done New false
5476 2053 Acked 3052 Flush 0 Done Done New false
5477 2054 Acked 3053 Read 1 Done Done New false
5478 2055 Acked 3054 Write 3 Done Done New false
5479 2056 Acked 3055 Read 7 Done Done New false
5480 2057 Acked 3056 Write 10 Done Done New false
5481 2058 Acked 3057 Read 2 Done Done New false
5482 2059 Acked 3058 Flush 0 Done Done New false
5483 2060 Acked 3059 Write 10 Done Done New false
5484 2061 Acked 3060 Read 3 Done Done New false
5485 2062 Acked 3061 Read 3 Done Done New false
5486 2063 Acked 3062 Write 7 Done Done New false
5487 2064 Acked 3063 Read 3 Done Done New false
5488 2065 Acked 3064 Write 2 Done Done New false
5489 2066 Acked 3065 Read 4 Done Done New false
5490 2067 Acked 3066 Read 5 Done Done New false
5491 2068 Acked 3067 Read 1 Done Done New false
5492 2069 Acked 3068 Flush 0 Done Done New false
5493 2070 Acked 3069 Write 9 Done Done New false
5494 2071 Acked 3070 Read 5 Done Done New false
5495 2072 Acked 3071 Read 4 Done Done New false
5496 2073 Acked 3072 Read 7 Done Done New false
5497 2074 Acked 3073 Flush 0 Done Done New false
5498 2075 Acked 3074 Flush 0 Done Done New false
5499 2076 Acked 3075 Read 10 Done Done New false
5500 2077 Acked 3076 Read 2 Done Done New false
5501 2078 Acked 3077 Read 10 Done Done New false
5502 2079 Acked 3078 Read 5 Done Done New false
5503 2080 Acked 3079 Write 5 Done Done New false
5504 2081 Acked 3080 Write 8 Done Done New false
5505 2082 Acked 3081 Read 6 Done Done New false
5506 2083 Acked 3082 Read 4 Done Done New false
5507 2084 Acked 3083 Read 1 Done Done New false
5508 2085 Acked 3084 Write 9 Done Done New false
5509 2086 Acked 3085 Write 6 Done Done New false
5510 2087 Acked 3086 Read 1 Done Done New false
5511 2088 Acked 3087 Write 9 Done Done New false
5512 2089 Acked 3088 Read 4 Done Done New false
5513 2090 Acked 3089 Read 1 Done Done New false
5514 2091 Acked 3090 Read 3 Done Done New false
5515 2092 Acked 3091 Read 9 Done Done New false
5516 2093 Acked 3092 Flush 0 Done Done New false
5517 2094 Acked 3093 Read 8 Done Done New false
5518 2095 Acked 3094 Write 4 Done Done New false
5519 2096 Acked 3095 Write 3 Done Done New false
5520 2097 Acked 3096 Write 3 Done Done New false
5521 2098 Acked 3097 Write 6 Done Done New false
5522 2099 Acked 3098 Write 1 Done Done New false
5523 2100 Acked 3099 Read 10 Done Done New false
5524 2101 Acked 3100 Write 7 Done Done New false
5525 2102 Acked 3101 Write 1 Done Done New false
5526 2103 Acked 3102 Read 1 Done Done New false
5527 2104 Acked 3103 Read 7 Done Done New false
5528 2105 Acked 3104 Write 6 Done Done New false
5529 2106 Acked 3105 Read 8 Done Done New false
5530 2107 Acked 3106 Read 10 Done Done New false
5531 2108 Acked 3107 Write 5 Done Done New false
5532 2109 Acked 3108 Read 8 Done Done New false
5533 2110 Acked 3109 Flush 0 Done Done New false
5534 2111 Acked 3110 Read 8 Done Done New false
5535 2112 Acked 3111 Write 6 Done Done New false
5536 2113 Acked 3112 Read 7 Done Done New false
5537 2114 Acked 3113 Read 7 Done Done New false
5538 2115 Acked 3114 Write 1 Done Done New false
5539 2116 Acked 3115 Read 4 Done Done New false
5540 STATES DS:0 DS:1 DS:2 TOTAL
5541 New 0 0 301 301
5542 Sent 0 0 0 0
5543 Done 301 301 0 602
5544 Skipped 0 0 0 0
5545 Error 0 0 0 0
5546 Last Flush: 3109 3109 2814
5547 Downstairs last five completed: 2814 2813 2812 2811 2810 2809
5548 Upstairs last five completed: 2116 2115 2114 2113 2112 2111
5549 CLIENT: Up:0 ds:301 act:2
55502023-09-22T23:24:57.630ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
5551 [8830] got start action
5552 [8830] I am going to restart
5553 Starting downstairs at port 8830
5554 Make output file at "/tmp/dsc/downstairs-8830.txt"
5555 [8830][2] reports Starting
5556 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1180)
5557 [8830][2] reports Running
55582023-09-22T23:24:58.631ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
55592023-09-22T23:24:58.631ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
55602023-09-22T23:24:58.631ZINFOcrucible: [2] client re-new 302 jobs since flush 2814 = downstairs
55612023-09-22T23:24:58.632ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
55622023-09-22T23:24:58.633ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
55632023-09-22T23:24:58.633ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
55642023-09-22T23:24:58.633ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
55652023-09-22T23:24:58.633ZINFOcrucible: [2] send last flush ID to this DS: 2814
55662023-09-22T23:24:58.633ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
55672023-09-22T23:24:58.633ZINFOcrucible: [2] Replied this last flush ID: 2814
55682023-09-22T23:24:58.633ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
55692023-09-22T23:24:58.633ZINFOcrucible: [2] Transition from Offline to Replay
55702023-09-22T23:24:58.633ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
55712023-09-22T23:24:58.633ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
55722023-09-22T23:24:58.633ZINFOcrucible: [2] Transition from Replay to Active
55732023-09-22T23:24:58.633ZINFOcrucible: [2] Starts cmd_loop
55742023-09-22T23:24:58.633ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
55752023-09-22T23:24:58.634ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
55762023-09-22T23:24:58.634ZINFOcrucible: Set check for repair
55772023-09-22T23:24:59.634ZINFOcrucible: Checking if live repair is needed
55782023-09-22T23:24:59.634ZINFOcrucible: No Live Repair required at this time
55792023-09-22T23:24:59.634ZWARNcrucible: [2] flow control end
5580 ----------------------------------------------------------------
5581 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5582 Downstairs last five completed: 3116 3115 3114 3113 3112 3111
5583 Upstairs last five completed: 2117 2116 2115 2114 2113 2112
5584 CLIENT: Up:0 ds:0 act:3
5585 Replay: All jobs finished, all DS active.
5586 Main task has work to do, go find it
5587 got dsc Stop(1)
5588 stop 1
5589 Sep 22 23:25:00.522 INFO request completed, latency_us: 523, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 7cdf934d-d7a2-4389-a4df-b1d605dc869c, remote_addr[8820] Got stop action so:false kr:false
5590 : 127.0.0.1:52471, local_addr: 127.0.0.1:9998
5591 Sep 22 23:25:00.523 INFO request completed, latency_us: 349, response_code: 200, uri: /state/cid/1, method: GET, req_id: 6e02245c-f88b-4583-b17d-2853332d7215, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
55922023-09-22T23:25:00.524ZWARNcrucible: [1] downstairs disconnected
55932023-09-22T23:25:00.524ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
55942023-09-22T23:25:00.524ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
55952023-09-22T23:25:00.524ZWARNcrucible: [1] pm_task rx.recv() is None
55962023-09-22T23:25:00.524ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
55972023-09-22T23:25:00.524ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
55982023-09-22T23:25:00.524ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
5599 [8820] Exited with: ExitStatus(unix_wait_status(9))
5600 [8820][1] reports Exit
56012023-09-22T23:25:03.527ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
5602 Sep 22 23:25:04.525 INFO request completed, latency_us: 295, response_code: 200, uri: /state/cid/1, method: GET, req_id: 2a24916f-0532-4857-b8ea-4c3ffa42fcb9, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
5603 001/300 Flush
5604 002/300 Write block 3290 len 40960 data: 4 3 4 4 3 3 3 3 2 2
5605 003/300 Write block 4850 len 12288 data: 3 3 3
5606 004/300 Read block 3921 len 12288
5607 005/300 Read block 1930 len 24576
5608 006/300 Write block 3828 len 16384 data: 2 2 2 3
5609 007/300 Read block 1371 len 8192
5610 008/300 Write block 2106 len 8192 data: 2 2
5611 009/300 Read block 4950 len 32768
5612 010/300 Read block 4409 len 28672
5613 011/300 Read block 3122 len 4096
5614 012/300 Write block 1277 len 28672 data: 3 2 2 2 2 2 2
5615 013/300 Read block 165 len 24576
5616 014/300 Write block 2582 len 32768 data: 3 4 4 4 3 3 3 3
5617 015/300 Flush
5618 016/300 Write block 2056 len 12288 data: 3 3 4
5619 017/300 Read block 2677 len 8192
5620 018/300 Read block 4973 len 28672
5621 019/300 Read block 3835 len 40960
5622 020/300 Read block 704 len 16384
5623 021/300 Write block 310 len 12288 data: 2 2 2
5624 022/300 Read block 2428 len 28672
5625 023/300 Write block 1930 len 40960 data: 2 3 3 3 2 2 2 3 3 4
5626 024/300 Read block 3073 len 24576
5627 025/300 Write block 3191 len 28672 data: 2 2 2 2 2 2 2
5628 026/300 Read block 426 len 12288
5629 027/300 Read block 3547 len 8192
5630 028/300 Read block 1802 len 20480
5631 029/300 Read block 2970 len 28672
5632 030/300 Write block 2894 len 40960 data: 2 2 2 2 2 2 2 3 3 3
5633 031/300 Write block 62 len 20480 data: 5 4 4 3 2
5634 032/300 Write block 3427 len 20480 data: 4 3 2 2 2
5635 033/300 Read block 3734 len 24576
5636 034/300 Read block 3353 len 12288
5637 035/300 Read block 132 len 40960
5638 036/300 Read block 180 len 16384
5639 037/300 Read block 1750 len 20480
5640 038/300 Read block 4067 len 4096
5641 039/300 Read block 943 len 36864
5642 040/300 Flush
5643 041/300 Read block 4236 len 8192
5644 042/300 Read block 4285 len 4096
5645 043/300 Write block 3265 len 28672 data: 3 3 3 3 2 2 2
5646 044/300 Read block 3093 len 16384
5647 045/300 Read block 4326 len 36864
5648 046/300 Write block 3857 len 24576 data: 2 2 2 2 3 2
5649 047/300 Write block 3594 len 12288 data: 4 4 3
5650 048/300 Write block 3367 len 24576 data: 4 3 3 3 2 2
5651 049/300 Read block 3171 len 16384
5652 050/300 Read block 2845 len 8192
5653 051/300 Write block 4756 len 16384 data: 2 3 3 3
5654 052/300 Read block 3909 len 4096
5655 053/300 Write block 1605 len 32768 data: 3 2 2 2 2 2 2 2
5656 054/300 Read block 4710 len 20480
5657 055/300 Read block 2607 len 24576
5658 056/300 Read block 3487 len 8192
5659 057/300 Read block 4594 len 16384
5660 058/300 Flush
5661 059/300 Write block 4820 len 40960 data: 7 6 5 5 5 4 5 5 4 3
5662 060/300 Write block 1397 len 36864 data: 2 2 2 2 2 2 3 3 4
5663 061/300 Flush
5664 062/300 Read block 3309 len 16384
5665 063/300 Write block 483 len 24576 data: 2 2 2 2 3 3
5666 064/300 Read block 3177 len 8192
5667 065/300 Write block 1640 len 8192 data: 2 2
5668 066/300 Write block 3255 len 4096 data: 5
5669 067/300 Write block 4122 len 4096 data: 2
5670 068/300 Read block 3819 len 24576
5671 069/300 Write block 4846 len 36864 data: 2 3 3 3 4 4 4 4 3
5672 070/300 Write block 1621 len 20480 data: 3 3 2 3 3
5673 071/300 Read block 456 len 16384
5674 072/300 Read block 575 len 20480
5675 073/300 Read block 369 len 28672
5676 074/300 Read block 3789 len 16384
5677 075/300 Read block 2045 len 28672
5678 076/300 Write block 2924 len 4096 data: 3
5679 077/300 Write block 3352 len 8192 data: 2 2
5680 078/300 Read block 3116 len 4096
5681 079/300 Read block 2556 len 24576
5682 080/300 Read block 3369 len 12288
5683 081/300 Read block 3859 len 32768
5684 082/300 Write block 4863 len 12288 data: 2 2 2
5685 083/300 Write block 3647 len 40960 data: 2 3 3 3 3 3 3 4 4 4
5686 084/300 Read block 2648 len 40960
5687 085/300 Read block 1303 len 16384
5688 086/300 Read block 1842 len 8192
5689 087/300 Read block 2426 len 20480
5690 088/300 Read block 3690 len 4096
5691 089/300 Read block 4634 len 40960
5692 090/300 Read block 539 len 8192
5693 091/300 Write block 126 len 24576 data: 2 2 2 2 2 2
5694 092/300 Flush
5695 093/300 Read block 1736 len 32768
5696 094/300 Read block 276 len 32768
5697 095/300 Read block 118 len 4096
5698 096/300 Read block 2239 len 8192
5699 097/300 Read block 4141 len 12288
5700 098/300 Write block 3840 len 20480 data: 3 3 3 3 3
5701 099/300 Read block 2887 len 24576
5702 100/300 Write block 3909 len 8192 data: 4 3
5703 101/300 Read block 569 len 24576
5704 102/300 Read block 911 len 4096
5705 103/300 Read block 1485 len 28672
5706 104/300 Read block 1480 len 20480
5707 105/300 Write block 1476 len 20480 data: 6 6 6 5 5
5708 106/300 Read block 3705 len 40960
5709 107/300 Write block 4854 len 28672 data: 4 3 3 3 3 3 2
5710 108/300 Read block 68 len 4096
5711 109/300 Read block 3591 len 8192
5712 110/300 Write block 666 len 8192 data: 5 4
5713 111/300 Read block 1532 len 12288
5714 112/300 Read block 2791 len 28672
5715 113/300 Read block 4597 len 12288
5716 114/300 Read block 2134 len 28672
5717 115/300 Write block 1980 len 40960 data: 3 3 2 2 4 4 4 2 2 2
5718 116/300 Write block 3091 len 12288 data: 3 3 3
5719 117/300 Write block 283 len 32768 data: 3 3 3 3 4 3 3 3
5720 118/300 Write block 4957 len 4096 data: 5
5721 119/300 Read block 599 len 36864
5722 120/300 Write block 4969 len 28672 data: 2 2 2 2 2 2 2
5723 121/300 Read block 3565 len 36864
5724 122/300 Read block 1694 len 8192
5725 123/300 Write block 4095 len 16384 data: 3 3 4 4
5726 124/300 Flush
5727 125/300 Read block 3118 len 8192
5728 126/300 Read block 529 len 16384
5729 127/300 Flush
5730 128/300 Read block 1162 len 4096
5731 129/300 Read block 600 len 12288
5732 130/300 Read block 1660 len 24576
5733 131/300 Flush
5734 132/300 Read block 346 len 8192
5735 133/300 Write block 2481 len 16384 data: 2 2 2 2
5736 134/300 Read block 2508 len 20480
5737 135/300 Write block 4523 len 32768 data: 5 3 3 3 3 3 3 4
5738 136/300 Write block 2550 len 4096 data: 3
5739 137/300 Write block 785 len 8192 data: 2 2
5740 138/300 Write block 383 len 8192 data: 3 4
5741 139/300 Read block 1153 len 12288
5742 140/300 Read block 3443 len 32768
5743 141/300 Write block 4227 len 8192 data: 3 4
5744 142/300 Read block 402 len 28672
5745 143/300 Read block 2779 len 40960
5746 144/300 Read block 197 len 16384
5747 145/300 Write block 3547 len 36864 data: 2 2 3 3 4 4 4 3 3
5748 146/300 Flush
5749 147/300 Read block 4694 len 8192
5750 148/300 Read block 2369 len 4096
5751 149/300 Read block 273 len 8192
5752 150/300 Flush
5753 151/300 Write block 3858 len 32768 data: 3 3 3 4 3 2 2 3
5754 152/300 Read block 733 len 40960
5755 153/300 Flush
5756 154/300 Read block 1320 len 8192
5757 155/300 Read block 4005 len 36864
5758 156/300 Read block 3400 len 12288
5759 157/300 Read block 655 len 4096
5760 158/300 Read block 138 len 24576
5761 159/300 Write block 3783 len 32768 data: 3 3 2 2 2 2 2 3
5762 160/300 Write block 3285 len 28672 data: 3 4 4 4 3 5 4
5763 161/300 Write block 252 len 4096 data: 6
5764 162/300 Read block 3294 len 32768
5765 163/300 Write block 1382 len 12288 data: 3 3 3
5766 164/300 Write block 1426 len 36864 data: 2 3 3 3 4 4 4 4 4
5767 165/300 Read block 4583 len 8192
5768 166/300 Read block 1901 len 20480
5769 167/300 Read block 1817 len 32768
5770 168/300 Flush
5771 169/300 Read block 2924 len 40960
5772 170/300 Write block 837 len 40960 data: 5 5 5 5 3 3 4 4 3 3
5773 171/300 Read block 2198 len 28672
5774 172/300 Write block 3409 len 4096 data: 2
5775 173/300 Write block 4723 len 4096 data: 2
5776 174/300 Read block 4186 len 36864
5777 175/300 Write block 4147 len 32768 data: 3 3 3 3 3 3 3 2
5778 176/300 Write block 1824 len 40960 data: 3 3 3 2 2 2 3 3 3 3
5779 177/300 Read block 4212 len 28672
5780 178/300 Write block 1428 len 40960 data: 4 4 5 5 5 5 5 5 4 4
5781 179/300 Write block 4755 len 16384 data: 2 3 4 4
5782 180/300 Read block 293 len 16384
5783 181/300 Read block 934 len 16384
5784 182/300 Read block 1490 len 4096
5785 183/300 Read block 3506 len 16384
5786 184/300 Read block 4662 len 36864
5787 185/300 Write block 3578 len 12288 data: 2 2 2
5788 186/300 Write block 4332 len 28672 data: 4 4 3 3 3 3 2
5789 187/300 Read block 4687 len 40960
5790 188/300 Write block 4185 len 4096 data: 3
5791 189/300 Read block 4169 len 4096
5792 190/300 Read block 227 len 36864
5793 191/300 Flush
5794 192/300 Read block 2633 len 28672
5795 193/300 Write block 2091 len 32768 data: 2 3 3 3 3 3 3 3
5796 194/300 Read block 3684 len 20480
5797 195/300 Write block 1459 len 36864 data: 4 4 4 4 4 4 4 3 2
5798 196/300 Write block 2383 len 36864 data: 3 3 3 3 2 2 2 3 3
5799 197/300 Write block 485 len 12288 data: 3 3 4
5800 198/300 Write block 497 len 36864 data: 2 2 2 2 4 4 4 3 3
5801 199/300 Write block 4456 len 16384 data: 4 4 3 3
5802 200/300 Read block 3412 len 4096
5803 201/300 Read block 255 len 16384
5804 202/300 Write block 1208 len 16384 data: 5 4 3 3
5805 203/300 Write block 3072 len 12288 data: 3 3 2
5806 204/300 Read block 1420 len 40960
5807 205/300 Flush
5808 206/300 Write block 851 len 40960 data: 4 4 3 3 3 3 3 2 2 2
5809 207/300 Read block 4383 len 4096
5810 208/300 Read block 3221 len 24576
5811 209/300 Read block 1482 len 16384
5812 210/300 Read block 4711 len 32768
5813 211/300 Read block 2517 len 8192
5814 212/300 Read block 2989 len 16384
5815 213/300 Write block 3521 len 16384 data: 3 3 3 2
5816 214/300 Write block 962 len 28672 data: 3 3 3 3 3 2 2
5817 215/300 Read block 884 len 28672
5818 216/300 Read block 4956 len 32768
5819 217/300 Read block 4053 len 24576
5820 218/300 Write block 1101 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5821 219/300 Write block 1039 len 12288 data: 2 2 2
5822 220/300 Read block 3290 len 24576
5823 221/300 Write block 2305 len 40960 data: 2 2 3 3 4 3 3 4 3 3
5824 222/300 Read block 2532 len 32768
5825 223/300 Read block 4865 len 12288
5826 224/300 Read block 2623 len 28672
5827 225/300 Read block 4904 len 4096
5828 226/300 Read block 1031 len 20480
5829 227/300 Write block 258 len 12288 data: 5 4 4
5830 228/300 Read block 2793 len 40960
5831 229/300 Write block 220 len 36864 data: 2 2 3 3 3 3 3 3 3
5832 230/300 Read block 4291 len 36864
5833 231/300 Read block 396 len 32768
5834 232/300 Read block 276 len 28672
5835 233/300 Read block 2363 len 40960
5836 234/300 Read block 1035 len 20480
5837 235/300 Read block 4001 len 8192
5838 236/300 Write block 4614 len 36864 data: 4 4 4 3 3 3 3 4 4
5839 237/300 Write block 2002 len 36864 data: 2 2 2 2 2 2 2 2 2
5840 238/300 Read block 275 len 40960
5841 239/300 Read block 4606 len 4096
5842 240/300 Read block 4208 len 12288
5843 241/300 Read block 4612 len 40960
5844 242/300 Read block 2409 len 20480
5845 243/300 Read block 4478 len 8192
5846 244/300 Read block 3630 len 24576
5847 245/300 Write block 933 len 4096 data: 4
5848 246/300 Read block 1348 len 28672
5849 247/300 Read block 303 len 24576
5850 248/300 Write block 153 len 4096 data: 3
5851 249/300 Write block 1487 len 20480 data: 2 2 2 2 2
5852 250/300 Read block 238 len 28672
5853 251/300 Read block 4335 len 8192
5854 252/300 Read block 3299 len 40960
5855 253/300 Read block 571 len 20480
5856 254/300 Write block 1418 len 32768 data: 3 3 3 3 2 2 2 2
5857 255/300 Flush
5858 256/300 Read block 4511 len 12288
5859 257/300 Write block 153 len 20480 data: 4 3 3 2 2
5860 258/300 Flush
5861 259/300 Read block 2958 len 32768
5862 260/300 Flush
5863 261/300 Read block 1334 len 24576
5864 262/300 Read block 1065 len 32768
5865 263/300 Read block 2705 len 16384
5866 264/300 Write block 291 len 4096 data: 3
5867 265/300 Read block 28 len 16384
5868 266/300 Write block 3746 len 32768 data: 4 3 3 2 2 3 3 2
5869 267/300 Read block 3380 len 16384
5870 268/300 Flush
5871 269/300 Read block 150 len 12288
5872 270/300 Read block 1566 len 32768
5873 271/300 Flush
5874 272/300 Read block 673 len 20480
5875 273/300 Read block 1510 len 28672
5876 274/300 Read block 732 len 12288
5877 275/300 Flush
5878 276/300 Write block 304 len 32768 data: 3 2 2 2 2 2 3 3
5879 277/300 Read block 4563 len 24576
5880 278/300 Read block 845 len 12288
5881 279/300 Read block 3468 len 32768
5882 280/300 Flush
5883 281/300 Read block 1990 len 20480
5884 282/300 Flush
5885 283/300 Read block 413 len 12288
5886 284/300 Write block 4696 len 4096 data: 2
5887 285/300 Write block 1474 len 36864 data: 5 6 7 7 7 6 6 4 3
5888 286/300 Write block 1410 len 40960 data: 3 3 3 3 3 3 3 3 4 4
5889 287/300 Write block 2877 len 12288 data: 4 4 4
5890 288/300 Read block 3074 len 4096
5891 289/300 Read block 1257 len 24576
5892 290/300 Read block 4949 len 36864
5893 291/300 Flush
5894 292/300 Read block 4238 len 24576
5895 293/300 Write block 3053 len 24576 data: 3 3 4 3 2 2
5896 294/300 Read block 1263 len 4096
5897 295/300 Read block 2960 len 32768
5898 296/300 Write block 2160 len 12288 data: 3 3 3
5899 297/300 Write block 2569 len 28672 data: 2 2 2 2 4 4 4
5900 298/300 Write block 1536 len 32768 data: 2 2 2 2 3 3 4 4
5901 299/300 Write block 485 len 36864 data: 4 4 5 4 3 3 3 3 3
5902 300/300 Write block 1465 len 28672 data: 5 4 3 3 3 3 3
5903 Main task has work to do, go find it
5904 got dsc Start(1)
5905 start 1
5906 Sep 22 23:25:05.422 INFO request completed, latency_us: 301, response_code: 204, uri: /start/cid/1, method: POST, req_id: 7b361321-96e4-4d87-a990-5629968b070e, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
5907 Replay: started 1, returned:Ok(())
5908 ----------------------------------------------------------------
5909 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5910 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
5911 2118 Acked 3117 Flush 0 Done New Done false
5912 2119 Acked 3118 Write 10 Done New Done false
5913 2120 Acked 3119 Write 3 Done New Done false
5914 2121 Acked 3120 Read 3 Done New Done false
5915 2122 Acked 3121 Read 6 Done New Done false
5916 2123 Acked 3122 Write 4 Done New Done false
5917 2124 Acked 3123 Read 2 Done New Done false
5918 2125 Acked 3124 Write 2 Done New Done false
5919 2126 Acked 3125 Read 8 Done New Done false
5920 2127 Acked 3126 Read 7 Done New Done false
5921 2128 Acked 3127 Read 1 Done New Done false
5922 2129 Acked 3128 Write 7 Done New Done false
5923 2130 Acked 3129 Read 6 Done New Done false
5924 2131 Acked 3130 Write 8 Done New Done false
5925 2132 Acked 3131 Flush 0 Done New Done false
5926 2133 Acked 3132 Write 3 Done New Done false
5927 2134 Acked 3133 Read 2 Done New Done false
5928 2135 Acked 3134 Read 7 Done New Done false
5929 2136 Acked 3135 Read 10 Done New Done false
5930 2137 Acked 3136 Read 4 Done New Done false
5931 2138 Acked 3137 Write 3 Done New Done false
5932 2139 Acked 3138 Read 7 Done New Done false
5933 2140 Acked 3139 Write 10 Done New Done false
5934 2141 Acked 3140 Read 6 Done New Done false
5935 2142 Acked 3141 Write 7 Done New Done false
5936 2143 Acked 3142 Read 3 Done New Done false
5937 2144 Acked 3143 Read 2 Done New Done false
5938 2145 Acked 3144 Read 5 Done New Done false
5939 2146 Acked 3145 Read 7 Done New Done false
5940 2147 Acked 3146 Write 10 Done New Done false
5941 2148 Acked 3147 Write 5 Done New Done false
5942 2149 Acked 3148 Write 5 Done New Done false
5943 2150 Acked 3149 Read 6 Done New Done false
5944 2151 Acked 3150 Read 3 Done New Done false
5945 2152 Acked 3151 Read 10 Done New Done false
5946 2153 Acked 3152 Read 4 Done New Done false
5947 2154 Acked 3153 Read 5 Done New Done false
5948 2155 Acked 3154 Read 1 Done New Done false
5949 2156 Acked 3155 Read 9 Done New Done false
5950 2157 Acked 3156 Flush 0 Done New Done false
5951 2158 Acked 3157 Read 2 Done New Done false
5952 2159 Acked 3158 Read 1 Done New Done false
5953 2160 Acked 3159 Write 7 Done New Done false
5954 2161 Acked 3160 Read 4 Done New Done false
5955 2162 Acked 3161 Read 9 Done New Done false
5956 2163 Acked 3162 Write 6 Done New Done false
5957 2164 Acked 3163 Write 3 Done New Done false
5958 2165 Acked 3164 Write 6 Done New Done false
5959 2166 Acked 3165 Read 4 Done New Done false
5960 2167 Acked 3166 Read 2 Done New Done false
5961 2168 Acked 3167 Write 4 Done New Done false
5962 2169 Acked 3168 Read 1 Done New Done false
5963 2170 Acked 3169 Write 8 Done New Done false
5964 2171 Acked 3170 Read 5 Done New Done false
5965 2172 Acked 3171 Read 6 Done New Done false
5966 2173 Acked 3172 Read 2 Done New Done false
5967 2174 Acked 3173 Read 4 Done New Done false
5968 2175 Acked 3174 Flush 0 Done New Done false
5969 2176 Acked 3175 Write 10 Done New Done false
5970 2177 Acked 3176 Write 9 Done New Done false
5971 2178 Acked 3177 Flush 0 Done New Done false
5972 2179 Acked 3178 Read 4 Done New Done false
5973 2180 Acked 3179 Write 6 Done New Done false
5974 2181 Acked 3180 Read 2 Done New Done false
5975 2182 Acked 3181 Write 2 Done New Done false
5976 2183 Acked 3182 Write 1 Done New Done false
5977 2184 Acked 3183 Write 1 Done New Done false
5978 2185 Acked 3184 Read 6 Done New Done false
5979 2186 Acked 3185 Write 9 Done New Done false
5980 2187 Acked 3186 Write 5 Done New Done false
5981 2188 Acked 3187 Read 4 Done New Done false
5982 2189 Acked 3188 Read 5 Done New Done false
5983 2190 Acked 3189 Read 7 Done New Done false
5984 2191 Acked 3190 Read 4 Done New Done false
5985 2192 Acked 3191 Read 7 Done New Done false
5986 2193 Acked 3192 Write 1 Done New Done false
5987 2194 Acked 3193 Write 2 Done New Done false
5988 2195 Acked 3194 Read 1 Done New Done false
5989 2196 Acked 3195 Read 6 Done New Done false
5990 2197 Acked 3196 Read 3 Done New Done false
5991 2198 Acked 3197 Read 8 Done New Done false
5992 2199 Acked 3198 Write 3 Done New Done false
5993 2200 Acked 3199 Write 10 Done New Done false
5994 2201 Acked 3200 Read 10 Done New Done false
5995 2202 Acked 3201 Read 4 Done New Done false
5996 2203 Acked 3202 Read 2 Done New Done false
5997 2204 Acked 3203 Read 5 Done New Done false
5998 2205 Acked 3204 Read 1 Done New Done false
5999 2206 Acked 3205 Read 10 Done New Done false
6000 2207 Acked 3206 Read 2 Done New Done false
6001 2208 Acked 3207 Write 6 Done New Done false
6002 2209 Acked 3208 Flush 0 Done New Done false
6003 2210 Acked 3209 Read 8 Done New Done false
6004 2211 Acked 3210 Read 8 Done New Done false
6005 2212 Acked 3211 Read 1 Done New Done false
6006 2213 Acked 3212 Read 2 Done New Done false
6007 2214 Acked 3213 Read 3 Done New Done false
6008 2215 Acked 3214 Write 5 Done New Done false
6009 2216 Acked 3215 Read 6 Done New Done false
6010 2217 Acked 3216 Write 2 Done New Done false
6011 2218 Acked 3217 Read 6 Done New Done false
6012 2219 Acked 3218 Read 1 Done New Done false
6013 2220 Acked 3219 Read 7 Done New Done false
6014 2221 Acked 3220 Read 5 Done New Done false
6015 2222 Acked 3221 Write 5 Done New Done false
6016 2223 Acked 3222 Read 10 Done New Done false
6017 2224 Acked 3223 Write 7 Done New Done false
6018 2225 Acked 3224 Read 1 Done New Done false
6019 2226 Acked 3225 Read 2 Done New Done false
6020 2227 Acked 3226 Write 2 Done New Done false
6021 2228 Acked 3227 Read 3 Done New Done false
6022 2229 Acked 3228 Read 7 Done New Done false
6023 2230 Acked 3229 Read 3 Done New Done false
6024 2231 Acked 3230 Read 7 Done New Done false
6025 2232 Acked 3231 Write 10 Done New Done false
6026 2233 Acked 3232 Write 3 Done New Done false
6027 2234 Acked 3233 Write 8 Done New Done false
6028 2235 Acked 3234 Write 1 Done New Done false
6029 2236 Acked 3235 Read 9 Done New Done false
6030 2237 Acked 3236 Write 7 Done New Done false
6031 2238 Acked 3237 Read 9 Done New Done false
6032 2239 Acked 3238 Read 2 Done New Done false
6033 2240 Acked 3239 Write 4 Done New Done false
6034 2241 Acked 3240 Flush 0 Done New Done false
6035 2242 Acked 3241 Read 2 Done New Done false
6036 2243 Acked 3242 Read 4 Done New Done false
6037 2244 Acked 3243 Flush 0 Done New Done false
6038 2245 Acked 3244 Flush 0 Done New Done false
6039 2246 Acked 3245 Read 1 Done New Done false
6040 2247 Acked 3246 Read 3 Done New Done false
6041 2248 Acked 3247 Read 6 Done New Done false
6042 2249 Acked 3248 Flush 0 Done New Done false
6043 2250 Acked 3249 Read 2 Done New Done false
6044 2251 Acked 3250 Write 4 Done New Done false
6045 2252 Acked 3251 Read 5 Done New Done false
6046 2253 Acked 3252 Write 8 Done New Done false
6047 2254 Acked 3253 Write 1 Done New Done false
6048 2255 Acked 3254 Write 2 Done New Done false
6049 2256 Acked 3255 Write 2 Done New Done false
6050 2257 Acked 3256 Read 3 Done New Done false
6051 2258 Acked 3257 Read 8 Done New Done false
6052 2259 Acked 3258 Write 2 Done New Done false
6053 2260 Acked 3259 Read 7 Done New Done false
6054 2261 Acked 3260 Read 10 Done New Done false
6055 2262 Acked 3261 Read 4 Done New Done false
6056 2263 Acked 3262 Write 9 Done New Done false
6057 2264 Acked 3263 Flush 0 Done New Done false
6058 2265 Acked 3264 Read 2 Done New Done false
6059 2266 Acked 3265 Read 1 Done New Done false
6060 2267 Acked 3266 Read 2 Done New Done false
6061 2268 Acked 3267 Flush 0 Done New Done false
6062 2269 Acked 3268 Write 8 Done New Done false
6063 2270 Acked 3269 Read 10 Done New Done false
6064 2271 Acked 3270 Flush 0 Done New Done false
6065 2272 Acked 3271 Read 2 Done New Done false
6066 2273 Acked 3272 Read 9 Done New Done false
6067 2274 Acked 3273 Read 3 Done New Done false
6068 2275 Acked 3274 Read 1 Done New Done false
6069 2276 Acked 3275 Read 6 Done New Done false
6070 2277 Acked 3276 Write 8 Done New Done false
6071 2278 Acked 3277 Write 7 Done New Done false
6072 2279 Acked 3278 Write 1 Done New Done false
6073 2280 Acked 3279 Read 8 Done New Done false
6074 2281 Acked 3280 Write 3 Done New Done false
6075 2282 Acked 3281 Write 9 Done New Done false
6076 2283 Acked 3282 Read 2 Done New Done false
6077 2284 Acked 3283 Read 5 Done New Done false
6078 2285 Acked 3284 Read 8 Done New Done false
6079 2286 Acked 3285 Flush 0 Done New Done false
6080 2287 Acked 3286 Read 10 Done New Done false
6081 2288 Acked 3287 Write 10 Done New Done false
6082 2289 Acked 3288 Read 7 Done New Done false
6083 2290 Acked 3289 Write 1 Done New Done false
6084 2291 Acked 3290 Write 1 Done New Done false
6085 2292 Acked 3291 Read 9 Done New Done false
6086 2293 Acked 3292 Write 8 Done New Done false
6087 2294 Acked 3293 Write 10 Done New Done false
6088 2295 Acked 3294 Read 7 Done New Done false
6089 2296 Acked 3295 Write 10 Done New Done false
6090 2297 Acked 3296 Write 4 Done New Done false
6091 2298 Acked 3297 Read 4 Done New Done false
6092 2299 Acked 3298 Read 4 Done New Done false
6093 2300 Acked 3299 Read 1 Done New Done false
6094 2301 Acked 3300 Read 4 Done New Done false
6095 2302 Acked 3301 Read 9 Done New Done false
6096 2303 Acked 3302 Write 3 Done New Done false
6097 2304 Acked 3303 Write 7 Done New Done false
6098 2305 Acked 3304 Read 10 Done New Done false
6099 2306 Acked 3305 Write 1 Done New Done false
6100 2307 Acked 3306 Read 1 Done New Done false
6101 2308 Acked 3307 Read 9 Done New Done false
6102 2309 Acked 3308 Flush 0 Done New Done false
6103 2310 Acked 3309 Read 7 Done New Done false
6104 2311 Acked 3310 Write 8 Done New Done false
6105 2312 Acked 3311 Read 5 Done New Done false
6106 2313 Acked 3312 Write 9 Done New Done false
6107 2314 Acked 3313 Write 9 Done New Done false
6108 2315 Acked 3314 Write 3 Done New Done false
6109 2316 Acked 3315 Write 9 Done New Done false
6110 2317 Acked 3316 Write 4 Done New Done false
6111 2318 Acked 3317 Read 1 Done New Done false
6112 2319 Acked 3318 Read 4 Done New Done false
6113 2320 Acked 3319 Write 4 Done New Done false
6114 2321 Acked 3320 Write 3 Done New Done false
6115 2322 Acked 3321 Read 10 Done New Done false
6116 2323 Acked 3322 Flush 0 Done New Done false
6117 2324 Acked 3323 Write 10 Done New Done false
6118 2325 Acked 3324 Read 1 Done New Done false
6119 2326 Acked 3325 Read 6 Done New Done false
6120 2327 Acked 3326 Read 4 Done New Done false
6121 2328 Acked 3327 Read 8 Done New Done false
6122 2329 Acked 3328 Read 2 Done New Done false
6123 2330 Acked 3329 Read 4 Done New Done false
6124 2331 Acked 3330 Write 4 Done New Done false
6125 2332 Acked 3331 Write 7 Done New Done false
6126 2333 Acked 3332 Read 7 Done New Done false
6127 2334 Acked 3333 Read 8 Done New Done false
6128 2335 Acked 3334 Read 6 Done New Done false
6129 2336 Acked 3335 Write 10 Done New Done false
6130 2337 Acked 3336 Write 3 Done New Done false
6131 2338 Acked 3337 Read 6 Done New Done false
6132 2339 Acked 3338 Write 10 Done New Done false
6133 2340 Acked 3339 Read 8 Done New Done false
6134 2341 Acked 3340 Read 3 Done New Done false
6135 2342 Acked 3341 Read 7 Done New Done false
6136 2343 Acked 3342 Read 1 Done New Done false
6137 2344 Acked 3343 Read 5 Done New Done false
6138 2345 Acked 3344 Write 3 Done New Done false
6139 2346 Acked 3345 Read 10 Done New Done false
6140 2347 Acked 3346 Write 9 Done New Done false
6141 2348 Acked 3347 Read 9 Done New Done false
6142 2349 Acked 3348 Read 8 Done New Done false
6143 2350 Acked 3349 Read 7 Done New Done false
6144 2351 Acked 3350 Read 10 Done New Done false
6145 2352 Acked 3351 Read 5 Done New Done false
6146 2353 Acked 3352 Read 2 Done New Done false
6147 2354 Acked 3353 Write 9 Done New Done false
6148 2355 Acked 3354 Write 9 Done New Done false
6149 2356 Acked 3355 Read 10 Done New Done false
6150 2357 Acked 3356 Read 1 Done New Done false
6151 2358 Acked 3357 Read 3 Done New Done false
6152 2359 Acked 3358 Read 10 Done New Done false
6153 2360 Acked 3359 Read 5 Done New Done false
6154 2361 Acked 3360 Read 2 Done New Done false
6155 2362 Acked 3361 Read 6 Done New Done false
6156 2363 Acked 3362 Write 1 Done New Done false
6157 2364 Acked 3363 Read 7 Done New Done false
6158 2365 Acked 3364 Read 6 Done New Done false
6159 2366 Acked 3365 Write 1 Done New Done false
6160 2367 Acked 3366 Write 5 Done New Done false
6161 2368 Acked 3367 Read 7 Done New Done false
6162 2369 Acked 3368 Read 2 Done New Done false
6163 2370 Acked 3369 Read 10 Done New Done false
6164 2371 Acked 3370 Read 5 Done New Done false
6165 2372 Acked 3371 Write 8 Done New Done false
6166 2373 Acked 3372 Flush 0 Done New Done false
6167 2374 Acked 3373 Read 3 Done New Done false
6168 2375 Acked 3374 Write 5 Done New Done false
6169 2376 Acked 3375 Flush 0 Done New Done false
6170 2377 Acked 3376 Read 8 Done New Done false
6171 2378 Acked 3377 Flush 0 Done New Done false
6172 2379 Acked 3378 Read 6 Done New Done false
6173 2380 Acked 3379 Read 8 Done New Done false
6174 2381 Acked 3380 Read 4 Done New Done false
6175 2382 Acked 3381 Write 1 Done New Done false
6176 2383 Acked 3382 Read 4 Done New Done false
6177 2384 Acked 3383 Write 8 Done New Done false
6178 2385 Acked 3384 Read 4 Done New Done false
6179 2386 Acked 3385 Flush 0 Done New Done false
6180 2387 Acked 3386 Read 3 Done New Done false
6181 2388 Acked 3387 Read 8 Done New Done false
6182 2389 Acked 3388 Flush 0 Done New Done false
6183 2390 Acked 3389 Read 5 Done New Done false
6184 2391 Acked 3390 Read 7 Done New Done false
6185 2392 Acked 3391 Read 3 Done New Done false
6186 2393 Acked 3392 Flush 0 Done New Done false
6187 2394 Acked 3393 Write 8 Done New Done false
6188 2395 Acked 3394 Read 6 Done New Done false
6189 2396 Acked 3395 Read 3 Done New Done false
6190 2397 Acked 3396 Read 8 Done New Done false
6191 2398 Acked 3397 Flush 0 Done New Done false
6192 2399 Acked 3398 Read 5 Done New Done false
6193 2400 Acked 3399 Flush 0 Done New Done false
6194 2401 Acked 3400 Read 3 Done New Done false
6195 2402 Acked 3401 Write 1 Done New Done false
6196 2403 Acked 3402 Write 9 Done New Done false
6197 2404 Acked 3403 Write 10 Done New Done false
6198 2405 Acked 3404 Write 3 Done New Done false
6199 2406 Acked 3405 Read 1 Done New Done false
6200 2407 Acked 3406 Read 6 Done New Done false
6201 2408 Acked 3407 Read 9 Done New Done false
6202 2409 Acked 3408 Flush 0 Done New Done false
6203 2410 Acked 3409 Read 6 Done New Done false
6204 2411 Acked 3410 Write 6 Done New Done false
6205 2412 Acked 3411 Read 1 Done New Done false
6206 2413 Acked 3412 Read 8 Done New Done false
6207 2414 Acked 3413 Write 3 Done New Done false
6208 2415 Acked 3414 Write 7 Done New Done false
6209 2416 Acked 3415 Write 8 Sent New Done false
6210 2417 Acked 3416 Write 9 Sent New Sent false
6211 2418 Acked 3417 Write 7 Sent New Sent false
6212 STATES DS:0 DS:1 DS:2 TOTAL
6213 New 0 301 0 301
6214 Sent 3 0 2 5
6215 Done 298 0 299 597
6216 Skipped 0 0 0 0
6217 Error 0 0 0 0
6218 Last Flush: 3408 3116 3408
6219 Downstairs last five completed: 3116 3115 3114 3113 3112 3111
6220 Upstairs last five completed: 2418 2417 2416 2415 2414 2413
6221 CLIENT: Up:0 ds:301 act:2
6222 [8820] got start action
6223 [8820] I am going to restart
6224 Starting downstairs at port 8820
6225 Make output file at "/tmp/dsc/downstairs-8820.txt"
6226 [8820][1] reports Starting
6227 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1181)
6228 [8820][1] reports Running
62292023-09-22T23:25:07.531ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
62302023-09-22T23:25:07.531ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
62312023-09-22T23:25:07.531ZINFOcrucible: [1] client re-new 302 jobs since flush 3116 = downstairs
62322023-09-22T23:25:07.532ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
62332023-09-22T23:25:07.533ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
62342023-09-22T23:25:07.533ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
62352023-09-22T23:25:07.533ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
62362023-09-22T23:25:07.533ZINFOcrucible: [1] send last flush ID to this DS: 3116
62372023-09-22T23:25:07.533ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
62382023-09-22T23:25:07.533ZINFOcrucible: [1] Replied this last flush ID: 3116
62392023-09-22T23:25:07.533ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
62402023-09-22T23:25:07.533ZINFOcrucible: [1] Transition from Offline to Replay
62412023-09-22T23:25:07.533ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
62422023-09-22T23:25:07.533ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
62432023-09-22T23:25:07.533ZINFOcrucible: [1] Transition from Replay to Active
62442023-09-22T23:25:07.533ZINFOcrucible: [1] Starts cmd_loop
62452023-09-22T23:25:07.534ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
62462023-09-22T23:25:07.534ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
62472023-09-22T23:25:07.534ZINFOcrucible: Set check for repair
62482023-09-22T23:25:08.535ZINFOcrucible: Checking if live repair is needed
62492023-09-22T23:25:08.535ZINFOcrucible: No Live Repair required at this time
62502023-09-22T23:25:08.535ZWARNcrucible: [1] flow control end
6251 ----------------------------------------------------------------
6252 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6253 Downstairs last five completed: 3418 3417 3416 3415 3414 3413
6254 Upstairs last five completed: 2419 2418 2417 2416 2415 2414
6255 CLIENT: Up:0 ds:0 act:3
6256 Replay: All jobs finished, all DS active.
6257 Main task has work to do, go find it
6258 got dsc Stop(0)
6259 stop 0
6260 Sep 22 23:25:09.428 INFO request completed, latency_us: 521, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 73e7abe1-534c-42af-a3b2-6f5ac6c6fbad[8810] Got stop action so:false kr:false
6261 , remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
6262 Sep 22 23:25:09.429 INFO request completed, latency_us: 180, response_code: 200, uri: /state/cid/0, method: GET, req_id: 525af027-4f27-4df4-a745-20eeb164d622, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
62632023-09-22T23:25:09.429ZWARNcrucible: [0] downstairs disconnected
62642023-09-22T23:25:09.430ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
62652023-09-22T23:25:09.430ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
62662023-09-22T23:25:09.430ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
62672023-09-22T23:25:09.430ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
62682023-09-22T23:25:09.430ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
62692023-09-22T23:25:09.430ZWARNcrucible: [0] pm_task rx.recv() is None
6270 [8810] Exited with: ExitStatus(unix_wait_status(9))
6271 [8810][0] reports Exit
62722023-09-22T23:25:12.432ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
6273 Sep 22 23:25:13.431 INFO request completed, latency_us: 288, response_code: 200, uri: /state/cid/0, method: GET, req_id: 9138b198-97b4-4620-980c-81a106a61d75, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
6274 001/300 Write block 1729 len 36864 data: 3 3 3 3 3 3 4 3 3
6275 002/300 Write block 3103 len 32768 data: 3 3 2 2 2 2 2 2
6276 003/300 Write block 3153 len 12288 data: 4 4 5
6277 004/300 Write block 982 len 40960 data: 3 3 2 2 3 2 4 4 4 3
6278 005/300 Read block 1709 len 32768
6279 006/300 Read block 580 len 16384
6280 007/300 Read block 4105 len 40960
6281 008/300 Write block 4908 len 20480 data: 3 2 4 5 6
6282 009/300 Read block 4736 len 20480
6283 010/300 Write block 3618 len 4096 data: 3
6284 011/300 Read block 2143 len 36864
6285 012/300 Write block 244 len 4096 data: 2
6286 013/300 Read block 1099 len 12288
6287 014/300 Write block 2101 len 8192 data: 2 2
6288 015/300 Write block 1862 len 36864 data: 3 3 3 3 3 3 2 2 2
6289 016/300 Write block 2362 len 12288 data: 3 3 3
6290 017/300 Write block 4276 len 4096 data: 4
6291 018/300 Write block 3684 len 16384 data: 3 4 3 3
6292 019/300 Read block 3374 len 8192
6293 020/300 Read block 3765 len 32768
6294 021/300 Write block 1233 len 20480 data: 2 3 3 3 3
6295 022/300 Read block 375 len 36864
6296 023/300 Read block 941 len 20480
6297 024/300 Flush
6298 025/300 Read block 3039 len 4096
6299 026/300 Read block 111 len 12288
6300 027/300 Read block 1139 len 28672
6301 028/300 Write block 2796 len 36864 data: 3 3 3 3 3 3 3 2 2
6302 029/300 Read block 615 len 16384
6303 030/300 Read block 1170 len 36864
6304 031/300 Read block 1678 len 24576
6305 032/300 Write block 2125 len 24576 data: 2 2 2 2 2 2
6306 033/300 Write block 797 len 20480 data: 3 3 3 3 3
6307 034/300 Read block 1845 len 36864
6308 035/300 Write block 4143 len 40960 data: 3 3 3 3 4 4 4 4 4 4
6309 036/300 Read block 1169 len 8192
6310 037/300 Write block 4762 len 24576 data: 4 4 4 3 3 3
6311 038/300 Write block 1728 len 28672 data: 2 4 4 4 4 4 4
6312 039/300 Read block 505 len 20480
6313 040/300 Read block 2511 len 32768
6314 041/300 Read block 2207 len 28672
6315 042/300 Read block 3504 len 24576
6316 043/300 Flush
6317 044/300 Read block 517 len 28672
6318 045/300 Write block 3458 len 8192 data: 3 3
6319 046/300 Read block 3324 len 8192
6320 047/300 Write block 3918 len 12288 data: 3 3 3
6321 048/300 Read block 3974 len 36864
6322 049/300 Read block 1198 len 32768
6323 050/300 Read block 2266 len 32768
6324 051/300 Write block 865 len 4096 data: 2
6325 052/300 Read block 3080 len 32768
6326 053/300 Write block 3773 len 28672 data: 4 4 3 3 3 3 3
6327 054/300 Write block 2223 len 8192 data: 3 2
6328 055/300 Read block 4343 len 16384
6329 056/300 Read block 998 len 40960
6330 057/300 Write block 1361 len 20480 data: 4 2 3 3 3
6331 058/300 Flush
6332 059/300 Read block 2759 len 12288
6333 060/300 Write block 2922 len 12288 data: 2 2 4
6334 061/300 Read block 3928 len 24576
6335 062/300 Read block 3352 len 20480
6336 063/300 Read block 3470 len 32768
6337 064/300 Read block 1372 len 32768
6338 065/300 Read block 2638 len 40960
6339 066/300 Write block 1779 len 40960 data: 2 2 3 2 2 2 2 2 3 3
6340 067/300 Read block 3991 len 20480
6341 068/300 Read block 421 len 16384
6342 069/300 Write block 485 len 24576 data: 5 5 6 5 4 4
6343 070/300 Flush
6344 071/300 Read block 2348 len 36864
6345 072/300 Write block 4613 len 4096 data: 3
6346 073/300 Write block 263 len 16384 data: 3 3 3 4
6347 074/300 Write block 2110 len 32768 data: 4 4 4 4 4 4 2 2
6348 075/300 Write block 2572 len 24576 data: 3 5 5 5 5 6
6349 076/300 Read block 1752 len 12288
6350 077/300 Read block 3137 len 32768
6351 078/300 Write block 1794 len 32768 data: 5 6 6 6 5 4 3 2
6352 079/300 Read block 216 len 28672
6353 080/300 Read block 1191 len 20480
6354 081/300 Flush
6355 082/300 Write block 4292 len 16384 data: 4 4 5 4
6356 083/300 Write block 4084 len 20480 data: 4 4 4 4 3
6357 084/300 Read block 4015 len 28672
6358 085/300 Read block 4098 len 4096
6359 086/300 Write block 4094 len 32768 data: 3 4 4 5 5 3 3 3
6360 087/300 Write block 2289 len 20480 data: 2 2 2 2 2
6361 088/300 Read block 3342 len 8192
6362 089/300 Read block 446 len 36864
6363 090/300 Write block 3443 len 8192 data: 3 3
6364 091/300 Write block 157 len 28672 data: 3 2 2 3 3 3 3
6365 092/300 Write block 263 len 4096 data: 4
6366 093/300 Write block 3436 len 4096 data: 2
6367 094/300 Write block 4422 len 40960 data: 4 4 4 4 5 3 3 2 2 2
6368 095/300 Write block 1708 len 36864 data: 3 3 3 2 2 2 2 2 2
6369 096/300 Read block 1404 len 28672
6370 097/300 Write block 2699 len 28672 data: 3 2 2 2 2 3 3
6371 098/300 Write block 1973 len 36864 data: 2 2 2 2 3 2 2 4 4
6372 099/300 Write block 1955 len 36864 data: 4 4 4 2 2 2 3 3 3
6373 100/300 Write block 3823 len 28672 data: 2 2 2 2 2 3 3
6374 101/300 Flush
6375 102/300 Read block 4322 len 24576
6376 103/300 Write block 1503 len 12288 data: 3 2 2
6377 104/300 Write block 1000 len 32768 data: 3 3 4 4 2 2 3 3
6378 105/300 Flush
6379 106/300 Read block 4077 len 24576
6380 107/300 Flush
6381 108/300 Read block 4462 len 16384
6382 109/300 Read block 3883 len 24576
6383 110/300 Write block 300 len 32768 data: 4 4 3 3 4 3 3 3
6384 111/300 Read block 4253 len 8192
6385 112/300 Read block 350 len 12288
6386 113/300 Read block 782 len 28672
6387 114/300 Read block 1595 len 36864
6388 115/300 Write block 3127 len 8192 data: 3 3
6389 116/300 Read block 4432 len 20480
6390 117/300 Read block 610 len 12288
6391 118/300 Read block 159 len 24576
6392 119/300 Write block 2404 len 16384 data: 2 3 3 3
6393 120/300 Read block 3510 len 20480
6394 121/300 Write block 3926 len 4096 data: 2
6395 122/300 Write block 345 len 36864 data: 2 2 2 2 3 3 2 2 3
6396 123/300 Read block 753 len 4096
6397 124/300 Read block 2558 len 4096
6398 125/300 Write block 701 len 16384 data: 3 3 2 3
6399 126/300 Write block 2889 len 40960 data: 2 2 2 2 2 3 3 3 3 3
6400 127/300 Write block 4347 len 8192 data: 2 3
6401 128/300 Read block 570 len 8192
6402 129/300 Read block 47 len 32768
6403 130/300 Read block 3431 len 36864
6404 131/300 Read block 3623 len 28672
6405 132/300 Read block 2120 len 24576
6406 133/300 Read block 4964 len 32768
6407 134/300 Write block 4604 len 4096 data: 6
6408 135/300 Flush
6409 136/300 Write block 3419 len 16384 data: 3 3 3 3
6410 137/300 Read block 2301 len 32768
6411 138/300 Write block 1079 len 28672 data: 3 3 3 3 2 2 2
6412 139/300 Read block 2280 len 20480
6413 140/300 Read block 2040 len 32768
6414 141/300 Read block 3884 len 28672
6415 142/300 Read block 1799 len 4096
6416 143/300 Write block 57 len 4096 data: 4
6417 144/300 Write block 1861 len 28672 data: 4 4 4 4 4 4 4
6418 145/300 Write block 1943 len 12288 data: 4 3 3
6419 146/300 Read block 4546 len 24576
6420 147/300 Write block 725 len 4096 data: 5
6421 148/300 Write block 3969 len 36864 data: 4 4 4 3 3 3 3 3 2
6422 149/300 Read block 691 len 36864
6423 150/300 Write block 3427 len 32768 data: 5 4 3 3 3 2 2 2
6424 151/300 Read block 298 len 8192
6425 152/300 Write block 543 len 28672 data: 2 2 2 2 2 2 2
6426 153/300 Read block 665 len 36864
6427 154/300 Write block 813 len 36864 data: 5 5 5 4 4 3 4 5 3
6428 155/300 Write block 4937 len 16384 data: 2 2 2 3
6429 156/300 Read block 1709 len 24576
6430 157/300 Read block 2604 len 36864
6431 158/300 Flush
6432 159/300 Write block 793 len 24576 data: 4 4 3 3 4 4
6433 160/300 Write block 4308 len 32768 data: 4 4 4 4 4 4 4 4
6434 161/300 Read block 3007 len 24576
6435 162/300 Write block 2035 len 4096 data: 2
6436 163/300 Write block 501 len 12288 data: 5 5 5
6437 164/300 Write block 3216 len 40960 data: 3 3 3 2 2 2 2 2 2 2
6438 165/300 Write block 4515 len 4096 data: 2
6439 166/300 Write block 4479 len 16384 data: 2 2 2 2
6440 167/300 Write block 2003 len 20480 data: 3 3 3 3 3
6441 168/300 Flush
6442 169/300 Read block 2057 len 20480
6443 170/300 Read block 3251 len 40960
6444 171/300 Read block 2620 len 36864
6445 172/300 Flush
6446 173/300 Write block 1613 len 20480 data: 2 2 2 2 3
6447 174/300 Read block 1535 len 12288
6448 175/300 Read block 4741 len 4096
6449 176/300 Write block 98 len 4096 data: 2
6450 177/300 Flush
6451 178/300 Read block 562 len 8192
6452 179/300 Write block 160 len 32768 data: 4 4 4 4 2 3 3 3
6453 180/300 Flush
6454 181/300 Write block 1582 len 16384 data: 2 2 2 2
6455 182/300 Read block 2423 len 8192
6456 183/300 Read block 1538 len 20480
6457 184/300 Write block 3761 len 12288 data: 2 2 2
6458 185/300 Read block 4577 len 12288
6459 186/300 Flush
6460 187/300 Write block 4802 len 36864 data: 3 3 3 3 3 3 3 3 3
6461 188/300 Read block 1994 len 12288
6462 189/300 Write block 4780 len 40960 data: 3 3 3 3 3 3 3 3 3 2
6463 190/300 Write block 4299 len 40960 data: 3 3 3 3 3 3 3 3 4 5
6464 191/300 Read block 1146 len 36864
6465 192/300 Read block 1024 len 36864
6466 193/300 Write block 4586 len 16384 data: 3 2 2 2
6467 194/300 Read block 3152 len 8192
6468 195/300 Read block 125 len 40960
6469 196/300 Read block 1222 len 40960
6470 197/300 Read block 825 len 40960
6471 198/300 Read block 1702 len 8192
6472 199/300 Read block 574 len 8192
6473 200/300 Flush
6474 201/300 Write block 1817 len 20480 data: 2 3 3 2 2
6475 202/300 Flush
6476 203/300 Flush
6477 204/300 Read block 4426 len 4096
6478 205/300 Read block 3232 len 16384
6479 206/300 Write block 2877 len 36864 data: 5 5 5 4 2 2 2 2 3
6480 207/300 Read block 2336 len 20480
6481 208/300 Read block 3714 len 4096
6482 209/300 Write block 2934 len 8192 data: 2 2
6483 210/300 Flush
6484 211/300 Flush
6485 212/300 Write block 3975 len 24576 data: 4 4 3 2 2 2
6486 213/300 Write block 3392 len 16384 data: 3 3 3 2
6487 214/300 Write block 1204 len 36864 data: 5 5 5 5 6 5 4 4 3
6488 215/300 Read block 1518 len 28672
6489 216/300 Read block 1445 len 8192
6490 217/300 Write block 2631 len 36864 data: 3 2 2 2 2 3 3 2 2
6491 218/300 Read block 574 len 16384
6492 219/300 Write block 4729 len 36864 data: 4 3 3 3 4 3 4 3 3
6493 220/300 Write block 1802 len 4096 data: 2
6494 221/300 Write block 3249 len 8192 data: 3 3
6495 222/300 Read block 2476 len 4096
6496 223/300 Read block 3727 len 8192
6497 224/300 Write block 3482 len 12288 data: 3 2 2
6498 225/300 Write block 4677 len 16384 data: 3 3 3 4
6499 226/300 Write block 2885 len 40960 data: 4 3 2 2 3 3 3 3 3 4
6500 227/300 Write block 3717 len 16384 data: 4 6 5 4
6501 228/300 Read block 1367 len 16384
6502 229/300 Write block 4474 len 20480 data: 2 2 2 2 2
6503 230/300 Read block 3902 len 4096
6504 231/300 Write block 2956 len 40960 data: 3 3 3 2 2 2 2 2 2 2
6505 232/300 Write block 1766 len 40960 data: 4 4 3 3 4 4 4 3 3 3
6506 233/300 Flush
6507 234/300 Read block 3764 len 16384
6508 235/300 Read block 606 len 28672
6509 236/300 Read block 4240 len 16384
6510 237/300 Write block 3851 len 24576 data: 4 4 4 3 3 2
6511 238/300 Write block 2525 len 32768 data: 3 3 3 3 2 2 2 2
6512 239/300 Write block 162 len 4096 data: 5
6513 240/300 Write block 3322 len 40960 data: 2 2 2 3 3 3 3 3 2 2
6514 241/300 Read block 2694 len 12288
6515 242/300 Read block 38 len 32768
6516 243/300 Read block 4350 len 40960
6517 244/300 Write block 2647 len 20480 data: 2 2 2 2 3
6518 245/300 Read block 858 len 32768
6519 246/300 Write block 11 len 12288 data: 3 3 2
6520 247/300 Read block 2362 len 40960
6521 248/300 Read block 268 len 32768
6522 249/300 Write block 4976 len 36864 data: 3 3 3 3 3 3 3 4 4
6523 250/300 Write block 1899 len 36864 data: 3 2 2 2 3 3 3 3 3
6524 251/300 Read block 309 len 28672
6525 252/300 Write block 871 len 28672 data: 3 3 4 4 4 4 3
6526 253/300 Write block 1302 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6527 254/300 Read block 3199 len 40960
6528 255/300 Read block 3006 len 20480
6529 256/300 Read block 2397 len 16384
6530 257/300 Read block 3307 len 12288
6531 258/300 Write block 1995 len 8192 data: 2 2
6532 259/300 Write block 1711 len 12288 data: 3 3 3
6533 260/300 Read block 4061 len 24576
6534 261/300 Read block 3595 len 16384
6535 262/300 Read block 1101 len 24576
6536 263/300 Write block 1951 len 8192 data: 4 4
6537 264/300 Read block 1194 len 40960
6538 265/300 Read block 292 len 16384
6539 266/300 Write block 2252 len 20480 data: 4 3 3 2 2
6540 267/300 Write block 1662 len 8192 data: 2 2
6541 268/300 Write block 3376 len 24576 data: 2 2 2 2 2 2
6542 269/300 Write block 2455 len 4096 data: 3
6543 270/300 Read block 3167 len 8192
6544 271/300 Read block 1735 len 40960
6545 272/300 Write block 1726 len 16384 data: 2 2 3 5
6546 273/300 Write block 3457 len 20480 data: 3 4 4 2 2
6547 274/300 Read block 2625 len 36864
6548 275/300 Read block 4743 len 4096
6549 276/300 Read block 549 len 16384
6550 277/300 Read block 3126 len 12288
6551 278/300 Write block 3966 len 4096 data: 4
6552 279/300 Flush
6553 280/300 Read block 4054 len 4096
6554 281/300 Read block 748 len 28672
6555 282/300 Flush
6556 283/300 Read block 2678 len 24576
6557 284/300 Read block 4387 len 28672
6558 285/300 Read block 2662 len 40960
6559 286/300 Write block 3183 len 8192 data: 3 3
6560 287/300 Flush
6561 288/300 Read block 283 len 8192
6562 289/300 Read block 2777 len 24576
6563 290/300 Read block 1772 len 24576
6564 291/300 Read block 4019 len 4096
6565 292/300 Read block 2735 len 40960
6566 293/300 Read block 227 len 28672
6567 294/300 Read block 4557 len 24576
6568 295/300 Flush
6569 296/300 Read block 4784 len 8192
6570 297/300 Write block 1979 len 4096 data: 3
6571 298/300 Read block 4784 len 40960
6572 299/300 Write block 821 len 24576 data: 4 2 2 2 2 2
6573 300/300 Write block 1313 len 16384 data: 3 3 3 5
6574 Main task has work to do, go find it
6575 got dsc Start(0)
6576 start 0
6577 Sep 22 23:25:14.356 INFO request completed, latency_us: 306, response_code: 204, uri: /start/cid/0, method: POST, req_id: 780eff67-882e-4375-8f30-1a6413b229c3, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
6578 Replay: started 0, returned:Ok(())
6579 ----------------------------------------------------------------
6580 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:300
6581 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
6582 2420 Acked 3419 Write 9 New Done Done false
6583 2421 Acked 3420 Write 8 New Done Done false
6584 2422 Acked 3421 Write 3 New Done Done false
6585 2423 Acked 3422 Write 10 New Done Done false
6586 2424 Acked 3423 Read 8 New Done Done false
6587 2425 Acked 3424 Read 4 New Done Done false
6588 2426 Acked 3425 Read 10 New Done Done false
6589 2427 Acked 3426 Write 5 New Done Done false
6590 2428 Acked 3427 Read 5 New Done Done false
6591 2429 Acked 3428 Write 1 New Done Done false
6592 2430 Acked 3429 Read 9 New Done Done false
6593 2431 Acked 3430 Write 1 New Done Done false
6594 2432 Acked 3431 Read 3 New Done Done false
6595 2433 Acked 3432 Write 2 New Done Done false
6596 2434 Acked 3433 Write 9 New Done Done false
6597 2435 Acked 3434 Write 3 New Done Done false
6598 2436 Acked 3435 Write 1 New Done Done false
6599 2437 Acked 3436 Write 4 New Done Done false
6600 2438 Acked 3437 Read 2 New Done Done false
6601 2439 Acked 3438 Read 8 New Done Done false
6602 2440 Acked 3439 Write 5 New Done Done false
6603 2441 Acked 3440 Read 9 New Done Done false
6604 2442 Acked 3441 Read 5 New Done Done false
6605 2443 Acked 3442 Flush 0 New Done Done false
6606 2444 Acked 3443 Read 1 New Done Done false
6607 2445 Acked 3444 Read 3 New Done Done false
6608 2446 Acked 3445 Read 7 New Done Done false
6609 2447 Acked 3446 Write 9 New Done Done false
6610 2448 Acked 3447 Read 4 New Done Done false
6611 2449 Acked 3448 Read 9 New Done Done false
6612 2450 Acked 3449 Read 6 New Done Done false
6613 2451 Acked 3450 Write 6 New Done Done false
6614 2452 Acked 3451 Write 5 New Done Done false
6615 2453 Acked 3452 Read 9 New Done Done false
6616 2454 Acked 3453 Write 10 New Done Done false
6617 2455 Acked 3454 Read 2 New Done Done false
6618 2456 Acked 3455 Write 6 New Done Done false
6619 2457 Acked 3456 Write 7 New Done Done false
6620 2458 Acked 3457 Read 5 New Done Done false
6621 2459 Acked 3458 Read 8 New Done Done false
6622 2460 Acked 3459 Read 7 New Done Done false
6623 2461 Acked 3460 Read 6 New Done Done false
6624 2462 Acked 3461 Flush 0 New Done Done false
6625 2463 Acked 3462 Read 7 New Done Done false
6626 2464 Acked 3463 Write 2 New Done Done false
6627 2465 Acked 3464 Read 2 New Done Done false
6628 2466 Acked 3465 Write 3 New Done Done false
6629 2467 Acked 3466 Read 9 New Done Done false
6630 2468 Acked 3467 Read 8 New Done Done false
6631 2469 Acked 3468 Read 8 New Done Done false
6632 2470 Acked 3469 Write 1 New Done Done false
6633 2471 Acked 3470 Read 8 New Done Done false
6634 2472 Acked 3471 Write 7 New Done Done false
6635 2473 Acked 3472 Write 2 New Done Done false
6636 2474 Acked 3473 Read 4 New Done Done false
6637 2475 Acked 3474 Read 10 New Done Done false
6638 2476 Acked 3475 Write 5 New Done Done false
6639 2477 Acked 3476 Flush 0 New Done Done false
6640 2478 Acked 3477 Read 3 New Done Done false
6641 2479 Acked 3478 Write 3 New Done Done false
6642 2480 Acked 3479 Read 6 New Done Done false
6643 2481 Acked 3480 Read 5 New Done Done false
6644 2482 Acked 3481 Read 8 New Done Done false
6645 2483 Acked 3482 Read 8 New Done Done false
6646 2484 Acked 3483 Read 10 New Done Done false
6647 2485 Acked 3484 Write 10 New Done Done false
6648 2486 Acked 3485 Read 5 New Done Done false
6649 2487 Acked 3486 Read 4 New Done Done false
6650 2488 Acked 3487 Write 6 New Done Done false
6651 2489 Acked 3488 Flush 0 New Done Done false
6652 2490 Acked 3489 Read 9 New Done Done false
6653 2491 Acked 3490 Write 1 New Done Done false
6654 2492 Acked 3491 Write 4 New Done Done false
6655 2493 Acked 3492 Write 8 New Done Done false
6656 2494 Acked 3493 Write 6 New Done Done false
6657 2495 Acked 3494 Read 3 New Done Done false
6658 2496 Acked 3495 Read 8 New Done Done false
6659 2497 Acked 3496 Write 8 New Done Done false
6660 2498 Acked 3497 Read 7 New Done Done false
6661 2499 Acked 3498 Read 5 New Done Done false
6662 2500 Acked 3499 Flush 0 New Done Done false
6663 2501 Acked 3500 Write 4 New Done Done false
6664 2502 Acked 3501 Write 5 New Done Done false
6665 2503 Acked 3502 Read 7 New Done Done false
6666 2504 Acked 3503 Read 1 New Done Done false
6667 2505 Acked 3504 Write 8 New Done Done false
6668 2506 Acked 3505 Write 5 New Done Done false
6669 2507 Acked 3506 Read 2 New Done Done false
6670 2508 Acked 3507 Read 9 New Done Done false
6671 2509 Acked 3508 Write 2 New Done Done false
6672 2510 Acked 3509 Write 7 New Done Done false
6673 2511 Acked 3510 Write 1 New Done Done false
6674 2512 Acked 3511 Write 1 New Done Done false
6675 2513 Acked 3512 Write 10 New Done Done false
6676 2514 Acked 3513 Write 9 New Done Done false
6677 2515 Acked 3514 Read 7 New Done Done false
6678 2516 Acked 3515 Write 7 New Done Done false
6679 2517 Acked 3516 Write 9 New Done Done false
6680 2518 Acked 3517 Write 9 New Done Done false
6681 2519 Acked 3518 Write 7 New Done Done false
6682 2520 Acked 3519 Flush 0 New Done Done false
6683 2521 Acked 3520 Read 6 New Done Done false
6684 2522 Acked 3521 Write 3 New Done Done false
6685 2523 Acked 3522 Write 8 New Done Done false
6686 2524 Acked 3523 Flush 0 New Done Done false
6687 2525 Acked 3524 Read 6 New Done Done false
6688 2526 Acked 3525 Flush 0 New Done Done false
6689 2527 Acked 3526 Read 4 New Done Done false
6690 2528 Acked 3527 Read 6 New Done Done false
6691 2529 Acked 3528 Write 8 New Done Done false
6692 2530 Acked 3529 Read 2 New Done Done false
6693 2531 Acked 3530 Read 3 New Done Done false
6694 2532 Acked 3531 Read 7 New Done Done false
6695 2533 Acked 3532 Read 9 New Done Done false
6696 2534 Acked 3533 Write 2 New Done Done false
6697 2535 Acked 3534 Read 5 New Done Done false
6698 2536 Acked 3535 Read 3 New Done Done false
6699 2537 Acked 3536 Read 6 New Done Done false
6700 2538 Acked 3537 Write 4 New Done Done false
6701 2539 Acked 3538 Read 5 New Done Done false
6702 2540 Acked 3539 Write 1 New Done Done false
6703 2541 Acked 3540 Write 9 New Done Done false
6704 2542 Acked 3541 Read 1 New Done Done false
6705 2543 Acked 3542 Read 1 New Done Done false
6706 2544 Acked 3543 Write 4 New Done Done false
6707 2545 Acked 3544 Write 10 New Done Done false
6708 2546 Acked 3545 Write 2 New Done Done false
6709 2547 Acked 3546 Read 2 New Done Done false
6710 2548 Acked 3547 Read 8 New Done Done false
6711 2549 Acked 3548 Read 9 New Done Done false
6712 2550 Acked 3549 Read 7 New Done Done false
6713 2551 Acked 3550 Read 6 New Done Done false
6714 2552 Acked 3551 Read 8 New Done Done false
6715 2553 Acked 3552 Write 1 New Done Done false
6716 2554 Acked 3553 Flush 0 New Done Done false
6717 2555 Acked 3554 Write 4 New Done Done false
6718 2556 Acked 3555 Read 8 New Done Done false
6719 2557 Acked 3556 Write 7 New Done Done false
6720 2558 Acked 3557 Read 5 New Done Done false
6721 2559 Acked 3558 Read 8 New Done Done false
6722 2560 Acked 3559 Read 7 New Done Done false
6723 2561 Acked 3560 Read 1 New Done Done false
6724 2562 Acked 3561 Write 1 New Done Done false
6725 2563 Acked 3562 Write 7 New Done Done false
6726 2564 Acked 3563 Write 3 New Done Done false
6727 2565 Acked 3564 Read 6 New Done Done false
6728 2566 Acked 3565 Write 1 New Done Done false
6729 2567 Acked 3566 Write 9 New Done Done false
6730 2568 Acked 3567 Read 9 New Done Done false
6731 2569 Acked 3568 Write 8 New Done Done false
6732 2570 Acked 3569 Read 2 New Done Done false
6733 2571 Acked 3570 Write 7 New Done Done false
6734 2572 Acked 3571 Read 9 New Done Done false
6735 2573 Acked 3572 Write 9 New Done Done false
6736 2574 Acked 3573 Write 4 New Done Done false
6737 2575 Acked 3574 Read 6 New Done Done false
6738 2576 Acked 3575 Read 9 New Done Done false
6739 2577 Acked 3576 Flush 0 New Done Done false
6740 2578 Acked 3577 Write 6 New Done Done false
6741 2579 Acked 3578 Write 8 New Done Done false
6742 2580 Acked 3579 Read 6 New Done Done false
6743 2581 Acked 3580 Write 1 New Done Done false
6744 2582 Acked 3581 Write 3 New Done Done false
6745 2583 Acked 3582 Write 10 New Done Done false
6746 2584 Acked 3583 Write 1 New Done Done false
6747 2585 Acked 3584 Write 4 New Done Done false
6748 2586 Acked 3585 Write 5 New Done Done false
6749 2587 Acked 3586 Flush 0 New Done Done false
6750 2588 Acked 3587 Read 5 New Done Done false
6751 2589 Acked 3588 Read 10 New Done Done false
6752 2590 Acked 3589 Read 9 New Done Done false
6753 2591 Acked 3590 Flush 0 New Done Done false
6754 2592 Acked 3591 Write 5 New Done Done false
6755 2593 Acked 3592 Read 3 New Done Done false
6756 2594 Acked 3593 Read 1 New Done Done false
6757 2595 Acked 3594 Write 1 New Done Done false
6758 2596 Acked 3595 Flush 0 New Done Done false
6759 2597 Acked 3596 Read 2 New Done Done false
6760 2598 Acked 3597 Write 8 New Done Done false
6761 2599 Acked 3598 Flush 0 New Done Done false
6762 2600 Acked 3599 Write 4 New Done Done false
6763 2601 Acked 3600 Read 2 New Done Done false
6764 2602 Acked 3601 Read 5 New Done Done false
6765 2603 Acked 3602 Write 3 New Done Done false
6766 2604 Acked 3603 Read 3 New Done Done false
6767 2605 Acked 3604 Flush 0 New Done Done false
6768 2606 Acked 3605 Write 9 New Done Done false
6769 2607 Acked 3606 Read 3 New Done Done false
6770 2608 Acked 3607 Write 10 New Done Done false
6771 2609 Acked 3608 Write 10 New Done Done false
6772 2610 Acked 3609 Read 9 New Done Done false
6773 2611 Acked 3610 Read 9 New Done Done false
6774 2612 Acked 3611 Write 4 New Done Done false
6775 2613 Acked 3612 Read 2 New Done Done false
6776 2614 Acked 3613 Read 10 New Done Done false
6777 2615 Acked 3614 Read 10 New Done Done false
6778 2616 Acked 3615 Read 10 New Done Done false
6779 2617 Acked 3616 Read 2 New Done Done false
6780 2618 Acked 3617 Read 2 New Done Done false
6781 2619 Acked 3618 Flush 0 New Done Done false
6782 2620 Acked 3619 Write 5 New Done Done false
6783 2621 Acked 3620 Flush 0 New Done Done false
6784 2622 Acked 3621 Flush 0 New Done Done false
6785 2623 Acked 3622 Read 1 New Done Done false
6786 2624 Acked 3623 Read 4 New Done Done false
6787 2625 Acked 3624 Write 9 New Done Done false
6788 2626 Acked 3625 Read 5 New Done Done false
6789 2627 Acked 3626 Read 1 New Done Done false
6790 2628 Acked 3627 Write 2 New Done Done false
6791 2629 Acked 3628 Flush 0 New Done Done false
6792 2630 Acked 3629 Flush 0 New Done Done false
6793 2631 Acked 3630 Write 6 New Done Done false
6794 2632 Acked 3631 Write 4 New Done Done false
6795 2633 Acked 3632 Write 9 New Done Done false
6796 2634 Acked 3633 Read 7 New Done Done false
6797 2635 Acked 3634 Read 2 New Done Done false
6798 2636 Acked 3635 Write 9 New Done Done false
6799 2637 Acked 3636 Read 4 New Done Done false
6800 2638 Acked 3637 Write 9 New Done Done false
6801 2639 Acked 3638 Write 1 New Done Done false
6802 2640 Acked 3639 Write 2 New Done Done false
6803 2641 Acked 3640 Read 1 New Done Done false
6804 2642 Acked 3641 Read 2 New Done Done false
6805 2643 Acked 3642 Write 3 New Done Done false
6806 2644 Acked 3643 Write 4 New Done Done false
6807 2645 Acked 3644 Write 10 New Done Done false
6808 2646 Acked 3645 Write 4 New Done Done false
6809 2647 Acked 3646 Read 4 New Done Done false
6810 2648 Acked 3647 Write 5 New Done Done false
6811 2649 Acked 3648 Read 1 New Done Done false
6812 2650 Acked 3649 Write 10 New Done Done false
6813 2651 Acked 3650 Write 10 New Done Done false
6814 2652 Acked 3651 Flush 0 New Done Done false
6815 2653 Acked 3652 Read 4 New Done Done false
6816 2654 Acked 3653 Read 7 New Done Done false
6817 2655 Acked 3654 Read 4 New Done Done false
6818 2656 Acked 3655 Write 6 New Done Done false
6819 2657 Acked 3656 Write 8 New Done Done false
6820 2658 Acked 3657 Write 1 New Done Done false
6821 2659 Acked 3658 Write 10 New Done Done false
6822 2660 Acked 3659 Read 3 New Done Done false
6823 2661 Acked 3660 Read 8 New Done Done false
6824 2662 Acked 3661 Read 10 New Done Done false
6825 2663 Acked 3662 Write 5 New Done Done false
6826 2664 Acked 3663 Read 8 New Done Done false
6827 2665 Acked 3664 Write 3 New Done Done false
6828 2666 Acked 3665 Read 10 New Done Done false
6829 2667 Acked 3666 Read 8 New Done Done false
6830 2668 Acked 3667 Write 9 New Done Done false
6831 2669 Acked 3668 Write 9 New Done Done false
6832 2670 Acked 3669 Read 7 New Done Done false
6833 2671 Acked 3670 Write 7 New Done Done false
6834 2672 Acked 3671 Write 10 New Done Done false
6835 2673 Acked 3672 Read 10 New Done Done false
6836 2674 Acked 3673 Read 5 New Done Done false
6837 2675 Acked 3674 Read 4 New Done Done false
6838 2676 Acked 3675 Read 3 New Done Done false
6839 2677 Acked 3676 Write 2 New Done Done false
6840 2678 Acked 3677 Write 3 New Done Done false
6841 2679 Acked 3678 Read 6 New Done Done false
6842 2680 Acked 3679 Read 4 New Done Done false
6843 2681 Acked 3680 Read 6 New Done Done false
6844 2682 Acked 3681 Write 2 New Done Done false
6845 2683 Acked 3682 Read 10 New Done Done false
6846 2684 Acked 3683 Read 4 New Done Done false
6847 2685 Acked 3684 Write 5 New Done Done false
6848 2686 Acked 3685 Write 2 New Done Done false
6849 2687 Acked 3686 Write 6 New Done Done false
6850 2688 Acked 3687 Write 1 New Done Done false
6851 2689 Acked 3688 Read 2 New Done Done false
6852 2690 Acked 3689 Read 10 New Done Done false
6853 2691 Acked 3690 Write 4 New Done Done false
6854 2692 Acked 3691 Write 5 New Done Done false
6855 2693 Acked 3692 Read 9 New Done Done false
6856 2694 Acked 3693 Read 1 New Done Done false
6857 2695 Acked 3694 Read 4 New Done Done false
6858 2696 Acked 3695 Read 3 New Done Done false
6859 2697 Acked 3696 Write 1 New Done Done false
6860 2698 Acked 3697 Flush 0 New Done Done false
6861 2699 Acked 3698 Read 1 New Done Done false
6862 2700 Acked 3699 Read 7 New Done Done false
6863 2701 Acked 3700 Flush 0 New Done Done false
6864 2702 Acked 3701 Read 6 New Done Done false
6865 2703 Acked 3702 Read 7 New Done Done false
6866 2704 Acked 3703 Read 10 New Done Done false
6867 2705 Acked 3704 Write 2 New Done Done false
6868 2706 Acked 3705 Flush 0 New Done Done false
6869 2707 Acked 3706 Read 2 New Done Done false
6870 2708 Acked 3707 Read 6 New Done Done false
6871 2709 Acked 3708 Read 6 New Done Done false
6872 2710 Acked 3709 Read 1 New Done Done false
6873 2711 Acked 3710 Read 10 New Done Done false
6874 2712 Acked 3711 Read 7 New Done Done false
6875 2713 Acked 3712 Read 6 New Done Done false
6876 2714 Acked 3713 Flush 0 New Done Done false
6877 2715 Acked 3714 Read 2 New Done Done false
6878 2716 Acked 3715 Write 1 New Done Done false
6879 2717 Acked 3716 Read 10 New Done Done false
6880 2718 Acked 3717 Write 6 New Done Sent false
6881 2719 Acked 3718 Write 4 New Sent Sent false
6882 STATES DS:0 DS:1 DS:2 TOTAL
6883 New 300 0 0 300
6884 Sent 0 1 2 3
6885 Done 0 299 298 597
6886 Skipped 0 0 0 0
6887 Error 0 0 0 0
6888 Last Flush: 3418 3713 3713
6889 Downstairs last five completed: 3418 3417 3416 3415 3414 3413
6890 Upstairs last five completed: 2719 2718 2717 2716 2715 2714
6891 CLIENT: Up:0 ds:300 act:2
6892 [8810] got start action
6893 [8810] I am going to restart
6894 Starting downstairs at port 8810
6895 Make output file at "/tmp/dsc/downstairs-8810.txt"
6896 [8810][0] reports Starting
6897 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1182)
6898 [8810][0] reports Running
68992023-09-22T23:25:16.439ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
69002023-09-22T23:25:16.439ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
69012023-09-22T23:25:16.439ZINFOcrucible: [0] client re-new 301 jobs since flush 3418 = downstairs
69022023-09-22T23:25:16.440ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
69032023-09-22T23:25:16.441ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
69042023-09-22T23:25:16.441ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
69052023-09-22T23:25:16.441ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
69062023-09-22T23:25:16.441ZINFOcrucible: [0] send last flush ID to this DS: 3418
69072023-09-22T23:25:16.441ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
69082023-09-22T23:25:16.441ZINFOcrucible: [0] Replied this last flush ID: 3418
69092023-09-22T23:25:16.441ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
69102023-09-22T23:25:16.441ZINFOcrucible: [0] Transition from Offline to Replay
69112023-09-22T23:25:16.441ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
69122023-09-22T23:25:16.441ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
69132023-09-22T23:25:16.441ZINFOcrucible: [0] Transition from Replay to Active
69142023-09-22T23:25:16.441ZINFOcrucible: [0] Starts cmd_loop
69152023-09-22T23:25:16.441ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
69162023-09-22T23:25:16.441ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
69172023-09-22T23:25:16.441ZINFOcrucible: Set check for repair
69182023-09-22T23:25:17.442ZINFOcrucible: Checking if live repair is needed
69192023-09-22T23:25:17.442ZINFOcrucible: No Live Repair required at this time
69202023-09-22T23:25:17.442ZWARNcrucible: [0] flow control end
6921 ----------------------------------------------------------------
6922 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6923 Downstairs last five completed: 3719 3718 3717 3716 3715 3714
6924 Upstairs last five completed: 2720 2719 2718 2717 2716 2715
6925 CLIENT: Up:0 ds:0 act:3
6926 Replay: All jobs finished, all DS active.
6927 Main task has work to do, go find it
6928 got dsc Stop(2)
6929 stop 2
6930 Sep 22 23:25:18.362 INFO request completed, latency_us: 535, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 43e3657d-17b2-42b7-b629-fa41139b0890, remote_addr: 127.0.0.1:52471[8830] Got stop action so:false kr:false
6931 , local_addr: 127.0.0.1:9998
6932 Sep 22 23:25:18.363 INFO request completed, latency_us: 181, response_code: 200, uri: /state/cid/2, method: GET, req_id: 08f79aa8-a30f-4c02-801e-bfd3f36b254b, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
69332023-09-22T23:25:18.363ZWARNcrucible: [2] downstairs disconnected
69342023-09-22T23:25:18.363ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
69352023-09-22T23:25:18.363ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
69362023-09-22T23:25:18.363ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
69372023-09-22T23:25:18.363ZWARNcrucible: [2] pm_task rx.recv() is None
69382023-09-22T23:25:18.363ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
69392023-09-22T23:25:18.363ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
6940 [8830] Exited with: ExitStatus(unix_wait_status(9))
6941 [8830][2] reports Exit
6942 Sep 22 23:25:22.365 INFO request completed, latency_us: 264, response_code: 200, uri: /state/cid/2, method: GET, req_id: 96470f53-8e5e-4f07-a955-8ebc15c201c3, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
6943 001/300 Read block 2120 len 40960
6944 002/300 Read block 224 len 28672
6945 003/300 Read block 3680 len 12288
6946 004/300 Flush
6947 005/300 Flush
6948 006/300 Read block 3301 len 40960
6949 007/300 Read block 3885 len 36864
6950 008/300 Read block 3186 len 36864
6951 009/300 Write block 4873 len 12288 data: 3 3 3
6952 010/300 Write block 4876 len 32768 data: 3 3 3 4 4 3 2 3
6953 011/300 Flush
6954 012/300 Read block 69 len 28672
6955 013/300 Write block 2483 len 8192 data: 3 3
6956 014/300 Flush
6957 015/300 Read block 3810 len 8192
6958 016/300 Flush
6959 017/300 Read block 426 len 36864
6960 018/300 Flush
6961 019/300 Write block 4398 len 28672 data: 4 4 5 5 7 6 5
6962 020/300 Read block 2913 len 40960
6963 021/300 Read block 1905 len 16384
6964 022/300 Read block 4953 len 12288
6965 023/300 Write block 190 len 20480 data: 4 3 3 3 3
6966 024/300 Write block 3190 len 32768 data: 2 3 3 3 3 3 3 3
6967 025/300 Read block 1871 len 24576
6968 026/300 Write block 3110 len 16384 data: 3 3 2 2
6969 027/300 Read block 3728 len 32768
6970 028/300 Read block 3170 len 36864
6971 029/300 Read block 3041 len 16384
6972 030/300 Write block 881 len 8192 data: 4 3
6973 031/300 Read block 1802 len 36864
6974 032/300 Read block 2903 len 16384
6975 033/300 Write block 693 len 36864 data: 2 2 2 2 2 2 3 3 4
6976 034/300 Write block 987 len 12288 data: 3 5 5
6977 035/300 Read block 4164 len 36864
6978 036/300 Read block 4536 len 20480
6979 037/300 Write block 1850 len 24576 data: 2 2 2 2 2 3
6980 038/300 Read block 291 len 8192
6981 039/300 Write block 2799 len 36864 data: 4 4 4 4 3 3 2 3 3
6982 040/300 Write block 4808 len 16384 data: 4 4 4 3
6983 041/300 Write block 247 len 8192 data: 4 4
6984 042/300 Read block 867 len 12288
6985 043/300 Read block 2342 len 8192
6986 044/300 Write block 3486 len 20480 data: 2 2 2 2 2
6987 045/300 Read block 50 len 24576
6988 046/300 Write block 2633 len 8192 data: 3 3
6989 047/300 Read block 1175 len 36864
6990 048/300 Read block 4936 len 24576
6991 049/300 Write block 4603 len 4096 data: 5
6992 050/300 Write block 174 len 32768 data: 5 4 4 4 4 4 4 4
6993 051/300 Write block 1709 len 24576 data: 4 4 4 4 4 3
6994 052/300 Read block 2125 len 40960
6995 053/300 Read block 722 len 24576
6996 054/300 Read block 541 len 4096
6997 055/300 Write block 2172 len 4096 data: 2
6998 056/300 Write block 3415 len 8192 data: 2 2
6999 057/300 Read block 2061 len 32768
7000 058/300 Write block 1564 len 36864 data: 2 3 4 4 5 4 4 4 4
7001 059/300 Write block 1494 len 32768 data: 2 3 3 3 3 4 4 4
7002 060/300 Read block 3964 len 36864
7003 061/300 Read block 621 len 32768
7004 062/300 Write block 3717 len 20480 data: 5 7 6 5 4
7005 063/300 Write block 1269 len 12288 data: 4 4 4
7006 064/300 Read block 3182 len 12288
7007 065/300 Read block 1751 len 32768
7008 066/300 Read block 4521 len 28672
7009 067/300 Write block 4934 len 40960 data: 3 3 3 3 3 3 4 3 3 2
7010 068/300 Write block 862 len 36864 data: 2 2 2 3 2 2 3 3 3
7011 069/300 Read block 1056 len 12288
7012 070/300 Read block 3806 len 16384
7013 071/300 Read block 4935 len 8192
7014 072/300 Flush
7015 073/300 Read block 839 len 36864
7016 074/300 Read block 3015 len 8192
7017 075/300 Write block 3262 len 4096 data: 5
7018 076/300 Write block 4949 len 20480 data: 2 3 3 3 3
7019 077/300 Read block 49 len 12288
7020 078/300 Write block 799 len 32768 data: 4 4 4 2 2 2 2 2
7021 079/300 Read block 723 len 8192
7022 080/300 Write block 2303 len 16384 data: 3 2 3 3
7023 081/300 Write block 1139 len 20480 data: 2 2 3 3 3
7024 082/300 Write block 2248 len 8192 data: 2 3
7025 083/300 Flush
7026 084/300 Write block 3702 len 24576 data: 4 4 2 2 2 2
7027 085/300 Read block 2116 len 40960
7028 086/300 Read block 2875 len 4096
7029 087/300 Write block 2621 len 8192 data: 6 7
7030 088/300 Write block 679 len 4096 data: 2
7031 089/300 Read block 612 len 32768
7032 090/300 Read block 4175 len 32768
7033 091/300 Read block 4098 len 24576
7034 092/300 Read block 3705 len 28672
7035 093/300 Write block 2875 len 8192 data: 3 3
7036 094/300 Write block 2894 len 36864 data: 5 4 4 4 4 3 3 4 4
7037 095/300 Read block 2875 len 12288
7038 096/300 Read block 4607 len 20480
7039 097/300 Read block 3165 len 24576
7040 098/300 Read block 878 len 8192
7041 099/300 Write block 3012 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7042 100/300 Write block 1789 len 40960 data: 4 4 5 5 4 6 7 7 7 6
7043 101/300 Write block 2575 len 12288 data: 6 6 7
7044 102/300 Read block 2009 len 12288
7045 103/300 Write block 4761 len 8192 data: 3 5
7046 104/300 Write block 388 len 20480 data: 3 3 3 3 3
7047 105/300 Read block 4210 len 36864
7048 106/300 Read block 2403 len 16384
7049 107/300 Read block 1797 len 4096
7050 108/300 Write block 3778 len 32768 data: 4 4 3 3 3 4 4 3
7051 109/300 Flush
7052 110/300 Read block 2902 len 12288
7053 111/300 Write block 1834 len 28672 data: 3 2 2 2 2 2 2
7054 112/300 Read block 581 len 32768
7055 113/300 Write block 4548 len 24576 data: 2 3 3 2 2 2
7056 114/300 Read block 4196 len 4096
7057 115/300 Read block 2941 len 28672
7058 116/300 Read block 3690 len 28672
7059 117/300 Read block 3562 len 32768
7060 118/300 Write block 2485 len 4096 data: 2
7061 119/300 Write block 4767 len 4096 data: 4
7062 120/300 Read block 1894 len 40960
7063 121/300 Write block 2715 len 8192 data: 3 3
7064 122/300 Read block 4404 len 32768
7065 123/300 Read block 1852 len 32768
7066 124/300 Read block 3533 len 12288
7067 125/300 Write block 4192 len 4096 data: 5
7068 126/300 Read block 3803 len 4096
7069 127/300 Write block 1671 len 16384 data: 4 3 2 2
7070 128/300 Write block 4927 len 32768 data: 4 5 5 4 4 4 3 4
7071 129/300 Write block 4271 len 28672 data: 2 2 3 3 3 5 4
7072 130/300 Flush
7073 131/300 Read block 4876 len 28672
7074 132/300 Read block 4593 len 24576
7075 133/300 Write block 402 len 40960 data: 3 3 4 4 3 3 3 3 3 3
7076 134/300 Write block 3063 len 8192 data: 2 2
7077 135/300 Read block 1433 len 40960
7078 136/300 Write block 4189 len 20480 data: 2 3 4 6 5
7079 137/300 Read block 1767 len 8192
7080 138/300 Write block 2774 len 28672 data: 3 3 4 4 3 3 2
7081 139/300 Write block 3513 len 16384 data: 4 5 4 4
7082 140/300 Read block 3263 len 24576
7083 141/300 Read block 3450 len 40960
7084 142/300 Write block 1942 len 8192 data: 4 5
7085 143/300 Read block 373 len 24576
7086 144/300 Write block 2552 len 24576 data: 4 3 3 3 3 3
7087 145/300 Write block 4684 len 28672 data: 3 3 4 2 2 2 2
7088 146/300 Write block 3553 len 16384 data: 5 4 4 3
7089 147/300 Write block 801 len 40960 data: 5 3 3 3 3 3 3 4 4 4
7090 148/300 Write block 3929 len 28672 data: 3 3 4 4 3 3 2
7091 149/300 Write block 688 len 4096 data: 2
7092 150/300 Read block 1277 len 20480
7093 151/300 Read block 1708 len 4096
7094 152/300 Read block 3685 len 8192
7095 153/300 Write block 587 len 12288 data: 2 2 2
7096 154/300 Read block 4472 len 40960
7097 155/300 Flush
7098 156/300 Read block 2521 len 36864
7099 157/300 Read block 1798 len 32768
7100 158/300 Read block 1286 len 32768
7101 159/300 Read block 4671 len 16384
7102 160/300 Write block 651 len 36864 data: 2 2 2 2 3 3 3 3 3
7103 161/300 Write block 2526 len 32768 data: 4 4 4 3 3 3 3 2
7104 162/300 Write block 3563 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7105 163/300 Write block 755 len 32768 data: 2 3 4 4 5 6 6 6
7106 164/300 Read block 1072 len 20480
7107 165/300 Write block 1810 len 12288 data: 2 2 2
7108 166/300 Read block 1174 len 16384
7109 167/300 Read block 516 len 16384
7110 168/300 Write block 684 len 4096 data: 2
7111 169/300 Write block 680 len 28672 data: 2 2 2 2 3 2 2
7112 170/300 Flush
7113 171/300 Read block 3206 len 28672
7114 172/300 Read block 4260 len 40960
7115 173/300 Read block 3379 len 8192
7116 174/300 Write block 4172 len 16384 data: 2 2 2 2
7117 175/300 Write block 3969 len 16384 data: 5 5 5 4
7118 176/300 Write block 4504 len 16384 data: 2 3 3 3
7119 177/300 Write block 3526 len 24576 data: 2 2 4 4 3 2
7120 178/300 Read block 3190 len 28672
7121 179/300 Read block 2871 len 20480
7122 180/300 Read block 1627 len 4096
7123 181/300 Read block 1695 len 24576
7124 182/300 Read block 749 len 28672
7125 183/300 Read block 2005 len 28672
7126 184/300 Read block 1801 len 8192
7127 185/300 Read block 1850 len 24576
7128 186/300 Flush
7129 187/300 Read block 1986 len 24576
7130 188/300 Write block 137 len 28672 data: 2 2 2 2 2 3 4
7131 189/300 Write block 430 len 12288 data: 2 2 2
7132 190/300 Write block 2359 len 8192 data: 3 3
7133 191/300 Write block 1949 len 40960 data: 4 3 5 5 3 3 5 5 5 3
7134 192/300 Read block 3285 len 24576
7135 193/300 Read block 4598 len 16384
7136 194/300 Write block 3915 len 12288 data: 2 3 3
7137 195/300 Read block 1561 len 36864
7138 196/300 Read block 1572 len 40960
7139 197/300 Write block 2349 len 40960 data: 5 5 5 5 5 3 2 2 2 2
7140 198/300 Read block 3437 len 36864
7141 199/300 Flush
7142 200/300 Write block 578 len 8192 data: 4 4
7143 201/300 Read block 3865 len 36864
7144 202/300 Write block 4941 len 16384 data: 4 4 3 2
7145 203/300 Read block 3492 len 28672
7146 204/300 Write block 2279 len 12288 data: 4 2 2
7147 205/300 Flush
7148 206/300 Write block 3835 len 24576 data: 3 2 3 3 3 4
7149 207/300 Read block 1526 len 12288
7150 208/300 Read block 1474 len 36864
7151 209/300 Write block 1893 len 16384 data: 3 3 3 3
7152 210/300 Read block 1442 len 32768
7153 211/300 Read block 2913 len 40960
7154 212/300 Read block 2667 len 4096
7155 213/300 Read block 2050 len 36864
7156 214/300 Write block 2081 len 16384 data: 2 2 2 2
7157 215/300 Write block 1556 len 16384 data: 4 4 3 3
7158 216/300 Read block 488 len 20480
7159 217/300 Read block 514 len 20480
7160 218/300 Write block 2032 len 20480 data: 3 2 2 3 2
7161 219/300 Read block 4257 len 28672
7162 220/300 Write block 4335 len 20480 data: 4 4 4 3 2
7163 221/300 Write block 1071 len 8192 data: 3 2
7164 222/300 Read block 3783 len 36864
7165 223/300 Write block 1185 len 4096 data: 3
7166 224/300 Write block 1631 len 32768 data: 4 5 3 3 3 3 3 3
7167 225/300 Write block 1357 len 20480 data: 2 2 2 3 5
7168 226/300 Read block 4655 len 8192
7169 227/300 Flush
7170 228/300 Write block 4602 len 24576 data: 5 6 7 5 4 4
7171 229/300 Read block 17 len 28672
7172 230/300 Write block 4579 len 12288 data: 3 3 4
7173 231/300 Read block 1474 len 8192
7174 232/300 Flush
7175 233/300 Read block 3597 len 8192
7176 234/300 Read block 2128 len 4096
7177 235/300 Write block 1500 len 32768 data: 5 5 3 4 3 3 2 2
7178 236/300 Write block 2213 len 36864 data: 2 2 3 3 3 3 3 3 3
7179 237/300 Write block 2921 len 24576 data: 2 3 3 5 3 2
7180 238/300 Write block 3933 len 28672 data: 4 4 3 2 2 2 2
7181 239/300 Read block 1709 len 28672
7182 240/300 Read block 1496 len 16384
7183 241/300 Write block 662 len 40960 data: 4 4 3 4 6 5 4 4 3 4
7184 242/300 Write block 2465 len 8192 data: 2 2
7185 243/300 Write block 2482 len 8192 data: 3 4
7186 244/300 Flush
7187 245/300 Read block 782 len 28672
7188 246/300 Flush
7189 247/300 Write block 1668 len 24576 data: 4 4 4 5 4 3
7190 248/300 Write block 2815 len 24576 data: 4 3 3 2 2 3
7191 249/300 Read block 3595 len 36864
7192 250/300 Write block 32 len 40960 data: 4 4 4 3 3 3 3 3 4 4
7193 251/300 Write block 2106 len 12288 data: 3 3 2
7194 252/300 Read block 3235 len 40960
7195 253/300 Read block 1318 len 12288
7196 254/300 Write block 3328 len 8192 data: 4 4
7197 255/300 Flush
7198 256/300 Write block 1573 len 12288 data: 3 3 3
7199 257/300 Read block 4684 len 40960
7200 258/300 Read block 4631 len 24576
7201 259/300 Write block 4994 len 8192 data: 2 2
7202 260/300 Read block 1723 len 4096
7203 261/300 Write block 3433 len 20480 data: 3 3 2 3 2
7204 262/300 Read block 1287 len 24576
7205 263/300 Read block 2688 len 20480
7206 264/300 Write block 4952 len 4096 data: 4
7207 265/300 Write block 4007 len 12288 data: 3 3 3
7208 266/300 Flush
7209 267/300 Read block 817 len 20480
7210 268/300 Write block 4077 len 40960 data: 4 4 4 4 4 5 5 5 5 5
7211 269/300 Write block 261 len 4096 data: 3
7212 270/300 Read block 3809 len 28672
7213 271/300 Read block 1841 len 8192
7214 272/300 Write block 2665 len 16384 data: 3 3 3 3
7215 273/300 Flush
7216 274/300 Write block 4279 len 12288 data: 4 3 3
7217 275/300 Read block 2080 len 36864
7218 276/300 Write block 4511 len 16384 data: 3 3 3 2
7219 277/300 Write block 4753 len 16384 data: 3 3 3 4
7220 278/300 Flush
7221 279/300 Read block 4673 len 20480
7222 280/300 Read block 1520 len 36864
7223 281/300 Write block 1429 len 20480 data: 5 6 6 6 6
7224 282/300 Read block 849 len 32768
7225 283/300 Write block 3160 len 24576 data: 4 3 3 3 2 2
7226 284/300 Write block 1016 len 8192 data: 2 2
7227 285/300 Read block 388 len 32768
7228 286/300 Write block 4058 len 28672 data: 4 4 4 4 4 3 3
7229 287/300 Write block 203 len 28672 data: 3 2 2 2 2 2 2
7230 288/300 Read block 2179 len 32768
7231 289/300 Write block 1408 len 40960 data: 4 3 4 4 4 4 4 4 4 4
7232 290/300 Read block 558 len 40960
7233 291/300 Write block 2270 len 20480 data: 3 4 4 4 4
7234 292/300 Write block 4227 len 16384 data: 4 5 4 4
7235 293/300 Write block 4035 len 16384 data: 4 4 4 4
7236 294/300 Read block 936 len 16384
7237 295/300 Read block 3007 len 40960
7238 296/300 Read block 1524 len 8192
7239 297/300 Flush
7240 298/300 Write block 2508 len 40960 data: 4 4 4 6 7 6 6 6 6 6
7241 299/300 Write block 2911 len 28672 data: 2 2 2 2 2 2 3
7242 300/300 Write block 1963 len 28672 data: 4 3 3 3 3 3 3
7243 Main task has work to do, go find it
7244 got dsc Start(2)
7245 start 2
7246 Sep 22 23:25:23.331 INFO request completed, latency_us: 389, response_code: 204, uri: /start/cid/2, method: POST, req_id: 2009700e-3ca3-4cc6-a5ce-efd6ed3d304c, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
7247 Replay: started 2, returned:Ok(())
7248 ----------------------------------------------------------------
7249 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7250 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
7251 2721 Acked 3720 Read 10 Done Done New false
7252 2722 Acked 3721 Read 7 Done Done New false
7253 2723 Acked 3722 Read 3 Done Done New false
7254 2724 Acked 3723 Flush 0 Done Done New false
7255 2725 Acked 3724 Flush 0 Done Done New false
7256 2726 Acked 3725 Read 10 Done Done New false
7257 2727 Acked 3726 Read 9 Done Done New false
7258 2728 Acked 3727 Read 9 Done Done New false
7259 2729 Acked 3728 Write 3 Done Done New false
7260 2730 Acked 3729 Write 8 Done Done New false
7261 2731 Acked 3730 Flush 0 Done Done New false
7262 2732 Acked 3731 Read 7 Done Done New false
7263 2733 Acked 3732 Write 2 Done Done New false
7264 2734 Acked 3733 Flush 0 Done Done New false
7265 2735 Acked 3734 Read 2 Done Done New false
7266 2736 Acked 3735 Flush 0 Done Done New false
7267 2737 Acked 3736 Read 9 Done Done New false
7268 2738 Acked 3737 Flush 0 Done Done New false
7269 2739 Acked 3738 Write 7 Done Done New false
7270 2740 Acked 3739 Read 10 Done Done New false
7271 2741 Acked 3740 Read 4 Done Done New false
7272 2742 Acked 3741 Read 3 Done Done New false
7273 2743 Acked 3742 Write 5 Done Done New false
7274 2744 Acked 3743 Write 8 Done Done New false
7275 2745 Acked 3744 Read 6 Done Done New false
7276 2746 Acked 3745 Flush 0 Done Done New false
7277 2747 Acked 3746 Write 4 Done Done New false
7278 2748 Acked 3747 Read 8 Done Done New false
7279 2749 Acked 3748 Read 9 Done Done New false
7280 2750 Acked 3749 Read 4 Done Done New false
7281 2751 Acked 3750 Write 2 Done Done New false
7282 2752 Acked 3751 Read 9 Done Done New false
7283 2753 Acked 3752 Read 4 Done Done New false
7284 2754 Acked 3753 Write 9 Done Done New false
7285 2755 Acked 3754 Write 3 Done Done New false
7286 2756 Acked 3755 Read 9 Done Done New false
7287 2757 Acked 3756 Read 5 Done Done New false
7288 2758 Acked 3757 Write 6 Done Done New false
7289 2759 Acked 3758 Read 2 Done Done New false
7290 2760 Acked 3759 Write 9 Done Done New false
7291 2761 Acked 3760 Write 4 Done Done New false
7292 2762 Acked 3761 Write 2 Done Done New false
7293 2763 Acked 3762 Read 3 Done Done New false
7294 2764 Acked 3763 Read 2 Done Done New false
7295 2765 Acked 3764 Write 5 Done Done New false
7296 2766 Acked 3765 Read 6 Done Done New false
7297 2767 Acked 3766 Write 2 Done Done New false
7298 2768 Acked 3767 Read 9 Done Done New false
7299 2769 Acked 3768 Read 6 Done Done New false
7300 2770 Acked 3769 Write 1 Done Done New false
7301 2771 Acked 3770 Write 8 Done Done New false
7302 2772 Acked 3771 Write 6 Done Done New false
7303 2773 Acked 3772 Read 10 Done Done New false
7304 2774 Acked 3773 Read 6 Done Done New false
7305 2775 Acked 3774 Read 1 Done Done New false
7306 2776 Acked 3775 Write 1 Done Done New false
7307 2777 Acked 3776 Write 2 Done Done New false
7308 2778 Acked 3777 Read 8 Done Done New false
7309 2779 Acked 3778 Write 9 Done Done New false
7310 2780 Acked 3779 Write 8 Done Done New false
7311 2781 Acked 3780 Read 9 Done Done New false
7312 2782 Acked 3781 Read 8 Done Done New false
7313 2783 Acked 3782 Write 5 Done Done New false
7314 2784 Acked 3783 Write 3 Done Done New false
7315 2785 Acked 3784 Read 3 Done Done New false
7316 2786 Acked 3785 Read 8 Done Done New false
7317 2787 Acked 3786 Read 7 Done Done New false
7318 2788 Acked 3787 Write 10 Done Done New false
7319 2789 Acked 3788 Write 9 Done Done New false
7320 2790 Acked 3789 Read 3 Done Done New false
7321 2791 Acked 3790 Read 4 Done Done New false
7322 2792 Acked 3791 Read 2 Done Done New false
7323 2793 Acked 3792 Flush 0 Done Done New false
7324 2794 Acked 3793 Read 9 Done Done New false
7325 2795 Acked 3794 Read 2 Done Done New false
7326 2796 Acked 3795 Write 1 Done Done New false
7327 2797 Acked 3796 Write 5 Done Done New false
7328 2798 Acked 3797 Read 3 Done Done New false
7329 2799 Acked 3798 Write 8 Done Done New false
7330 2800 Acked 3799 Read 2 Done Done New false
7331 2801 Acked 3800 Write 4 Done Done New false
7332 2802 Acked 3801 Write 5 Done Done New false
7333 2803 Acked 3802 Write 2 Done Done New false
7334 2804 Acked 3803 Flush 0 Done Done New false
7335 2805 Acked 3804 Write 6 Done Done New false
7336 2806 Acked 3805 Read 10 Done Done New false
7337 2807 Acked 3806 Read 1 Done Done New false
7338 2808 Acked 3807 Write 2 Done Done New false
7339 2809 Acked 3808 Write 1 Done Done New false
7340 2810 Acked 3809 Read 8 Done Done New false
7341 2811 Acked 3810 Read 8 Done Done New false
7342 2812 Acked 3811 Read 6 Done Done New false
7343 2813 Acked 3812 Read 7 Done Done New false
7344 2814 Acked 3813 Write 2 Done Done New false
7345 2815 Acked 3814 Write 9 Done Done New false
7346 2816 Acked 3815 Read 3 Done Done New false
7347 2817 Acked 3816 Read 5 Done Done New false
7348 2818 Acked 3817 Read 6 Done Done New false
7349 2819 Acked 3818 Read 2 Done Done New false
7350 2820 Acked 3819 Write 10 Done Done New false
7351 2821 Acked 3820 Write 10 Done Done New false
7352 2822 Acked 3821 Write 3 Done Done New false
7353 2823 Acked 3822 Read 3 Done Done New false
7354 2824 Acked 3823 Write 2 Done Done New false
7355 2825 Acked 3824 Write 5 Done Done New false
7356 2826 Acked 3825 Read 9 Done Done New false
7357 2827 Acked 3826 Read 4 Done Done New false
7358 2828 Acked 3827 Read 1 Done Done New false
7359 2829 Acked 3828 Write 8 Done Done New false
7360 2830 Acked 3829 Flush 0 Done Done New false
7361 2831 Acked 3830 Read 3 Done Done New false
7362 2832 Acked 3831 Write 7 Done Done New false
7363 2833 Acked 3832 Read 8 Done Done New false
7364 2834 Acked 3833 Write 6 Done Done New false
7365 2835 Acked 3834 Read 1 Done Done New false
7366 2836 Acked 3835 Read 7 Done Done New false
7367 2837 Acked 3836 Read 7 Done Done New false
7368 2838 Acked 3837 Read 8 Done Done New false
7369 2839 Acked 3838 Write 1 Done Done New false
7370 2840 Acked 3839 Write 1 Done Done New false
7371 2841 Acked 3840 Read 10 Done Done New false
7372 2842 Acked 3841 Write 2 Done Done New false
7373 2843 Acked 3842 Read 8 Done Done New false
7374 2844 Acked 3843 Read 8 Done Done New false
7375 2845 Acked 3844 Read 3 Done Done New false
7376 2846 Acked 3845 Write 1 Done Done New false
7377 2847 Acked 3846 Read 1 Done Done New false
7378 2848 Acked 3847 Write 4 Done Done New false
7379 2849 Acked 3848 Write 8 Done Done New false
7380 2850 Acked 3849 Write 7 Done Done New false
7381 2851 Acked 3850 Flush 0 Done Done New false
7382 2852 Acked 3851 Read 7 Done Done New false
7383 2853 Acked 3852 Read 6 Done Done New false
7384 2854 Acked 3853 Write 10 Done Done New false
7385 2855 Acked 3854 Write 2 Done Done New false
7386 2856 Acked 3855 Read 10 Done Done New false
7387 2857 Acked 3856 Write 5 Done Done New false
7388 2858 Acked 3857 Read 2 Done Done New false
7389 2859 Acked 3858 Write 7 Done Done New false
7390 2860 Acked 3859 Write 4 Done Done New false
7391 2861 Acked 3860 Read 6 Done Done New false
7392 2862 Acked 3861 Read 10 Done Done New false
7393 2863 Acked 3862 Write 2 Done Done New false
7394 2864 Acked 3863 Read 6 Done Done New false
7395 2865 Acked 3864 Write 6 Done Done New false
7396 2866 Acked 3865 Write 7 Done Done New false
7397 2867 Acked 3866 Write 4 Done Done New false
7398 2868 Acked 3867 Write 10 Done Done New false
7399 2869 Acked 3868 Write 7 Done Done New false
7400 2870 Acked 3869 Write 1 Done Done New false
7401 2871 Acked 3870 Read 5 Done Done New false
7402 2872 Acked 3871 Read 1 Done Done New false
7403 2873 Acked 3872 Read 2 Done Done New false
7404 2874 Acked 3873 Write 3 Done Done New false
7405 2875 Acked 3874 Read 10 Done Done New false
7406 2876 Acked 3875 Flush 0 Done Done New false
7407 2877 Acked 3876 Read 9 Done Done New false
7408 2878 Acked 3877 Read 8 Done Done New false
7409 2879 Acked 3878 Read 8 Done Done New false
7410 2880 Acked 3879 Read 4 Done Done New false
7411 2881 Acked 3880 Write 9 Done Done New false
7412 2882 Acked 3881 Write 8 Done Done New false
7413 2883 Acked 3882 Write 10 Done Done New false
7414 2884 Acked 3883 Write 8 Done Done New false
7415 2885 Acked 3884 Read 5 Done Done New false
7416 2886 Acked 3885 Write 3 Done Done New false
7417 2887 Acked 3886 Read 4 Done Done New false
7418 2888 Acked 3887 Read 4 Done Done New false
7419 2889 Acked 3888 Write 1 Done Done New false
7420 2890 Acked 3889 Write 7 Done Done New false
7421 2891 Acked 3890 Flush 0 Done Done New false
7422 2892 Acked 3891 Read 7 Done Done New false
7423 2893 Acked 3892 Read 10 Done Done New false
7424 2894 Acked 3893 Read 2 Done Done New false
7425 2895 Acked 3894 Write 4 Done Done New false
7426 2896 Acked 3895 Write 4 Done Done New false
7427 2897 Acked 3896 Write 4 Done Done New false
7428 2898 Acked 3897 Write 6 Done Done New false
7429 2899 Acked 3898 Read 7 Done Done New false
7430 2900 Acked 3899 Flush 0 Done Done New false
7431 2901 Acked 3900 Read 5 Done Done New false
7432 2902 Acked 3901 Read 1 Done Done New false
7433 2903 Acked 3902 Read 6 Done Done New false
7434 2904 Acked 3903 Read 7 Done Done New false
7435 2905 Acked 3904 Read 7 Done Done New false
7436 2906 Acked 3905 Read 2 Done Done New false
7437 2907 Acked 3906 Read 6 Done Done New false
7438 2908 Acked 3907 Flush 0 Done Done New false
7439 2909 Acked 3908 Read 6 Done Done New false
7440 2910 Acked 3909 Write 7 Done Done New false
7441 2911 Acked 3910 Write 3 Done Done New false
7442 2912 Acked 3911 Write 2 Done Done New false
7443 2913 Acked 3912 Write 10 Done Done New false
7444 2914 Acked 3913 Read 6 Done Done New false
7445 2915 Acked 3914 Read 4 Done Done New false
7446 2916 Acked 3915 Write 3 Done Done New false
7447 2917 Acked 3916 Read 9 Done Done New false
7448 2918 Acked 3917 Read 10 Done Done New false
7449 2919 Acked 3918 Write 10 Done Done New false
7450 2920 Acked 3919 Read 9 Done Done New false
7451 2921 Acked 3920 Flush 0 Done Done New false
7452 2922 Acked 3921 Write 2 Done Done New false
7453 2923 Acked 3922 Read 9 Done Done New false
7454 2924 Acked 3923 Write 4 Done Done New false
7455 2925 Acked 3924 Read 7 Done Done New false
7456 2926 Acked 3925 Write 3 Done Done New false
7457 2927 Acked 3926 Flush 0 Done Done New false
7458 2928 Acked 3927 Write 6 Done Done New false
7459 2929 Acked 3928 Read 3 Done Done New false
7460 2930 Acked 3929 Read 9 Done Done New false
7461 2931 Acked 3930 Write 4 Done Done New false
7462 2932 Acked 3931 Read 8 Done Done New false
7463 2933 Acked 3932 Read 10 Done Done New false
7464 2934 Acked 3933 Read 1 Done Done New false
7465 2935 Acked 3934 Read 9 Done Done New false
7466 2936 Acked 3935 Write 4 Done Done New false
7467 2937 Acked 3936 Write 4 Done Done New false
7468 2938 Acked 3937 Read 5 Done Done New false
7469 2939 Acked 3938 Read 5 Done Done New false
7470 2940 Acked 3939 Write 5 Done Done New false
7471 2941 Acked 3940 Read 7 Done Done New false
7472 2942 Acked 3941 Write 5 Done Done New false
7473 2943 Acked 3942 Write 2 Done Done New false
7474 2944 Acked 3943 Read 9 Done Done New false
7475 2945 Acked 3944 Write 1 Done Done New false
7476 2946 Acked 3945 Write 8 Done Done New false
7477 2947 Acked 3946 Write 5 Done Done New false
7478 2948 Acked 3947 Read 2 Done Done New false
7479 2949 Acked 3948 Flush 0 Done Done New false
7480 2950 Acked 3949 Write 6 Done Done New false
7481 2951 Acked 3950 Read 7 Done Done New false
7482 2952 Acked 3951 Write 3 Done Done New false
7483 2953 Acked 3952 Read 2 Done Done New false
7484 2954 Acked 3953 Flush 0 Done Done New false
7485 2955 Acked 3954 Read 2 Done Done New false
7486 2956 Acked 3955 Read 1 Done Done New false
7487 2957 Acked 3956 Write 8 Done Done New false
7488 2958 Acked 3957 Write 9 Done Done New false
7489 2959 Acked 3958 Write 6 Done Done New false
7490 2960 Acked 3959 Write 7 Done Done New false
7491 2961 Acked 3960 Read 7 Done Done New false
7492 2962 Acked 3961 Read 4 Done Done New false
7493 2963 Acked 3962 Write 10 Done Done New false
7494 2964 Acked 3963 Write 2 Done Done New false
7495 2965 Acked 3964 Write 2 Done Done New false
7496 2966 Acked 3965 Flush 0 Done Done New false
7497 2967 Acked 3966 Read 7 Done Done New false
7498 2968 Acked 3967 Flush 0 Done Done New false
7499 2969 Acked 3968 Write 6 Done Done New false
7500 2970 Acked 3969 Write 6 Done Done New false
7501 2971 Acked 3970 Read 9 Done Done New false
7502 2972 Acked 3971 Write 10 Done Done New false
7503 2973 Acked 3972 Write 3 Done Done New false
7504 2974 Acked 3973 Read 10 Done Done New false
7505 2975 Acked 3974 Read 3 Done Done New false
7506 2976 Acked 3975 Write 2 Done Done New false
7507 2977 Acked 3976 Flush 0 Done Done New false
7508 2978 Acked 3977 Write 3 Done Done New false
7509 2979 Acked 3978 Read 10 Done Done New false
7510 2980 Acked 3979 Read 6 Done Done New false
7511 2981 Acked 3980 Write 2 Done Done New false
7512 2982 Acked 3981 Read 1 Done Done New false
7513 2983 Acked 3982 Write 5 Done Done New false
7514 2984 Acked 3983 Read 6 Done Done New false
7515 2985 Acked 3984 Read 5 Done Done New false
7516 2986 Acked 3985 Write 1 Done Done New false
7517 2987 Acked 3986 Write 3 Done Done New false
7518 2988 Acked 3987 Flush 0 Done Done New false
7519 2989 Acked 3988 Read 5 Done Done New false
7520 2990 Acked 3989 Write 10 Done Done New false
7521 2991 Acked 3990 Write 1 Done Done New false
7522 2992 Acked 3991 Read 7 Done Done New false
7523 2993 Acked 3992 Read 2 Done Done New false
7524 2994 Acked 3993 Write 4 Done Done New false
7525 2995 Acked 3994 Flush 0 Done Done New false
7526 2996 Acked 3995 Write 3 Done Done New false
7527 2997 Acked 3996 Read 9 Done Done New false
7528 2998 Acked 3997 Write 4 Done Done New false
7529 2999 Acked 3998 Write 4 Done Done New false
7530 3000 Acked 3999 Flush 0 Done Done New false
7531 3001 Acked 4000 Read 5 Done Done New false
7532 3002 Acked 4001 Read 9 Done Done New false
7533 3003 Acked 4002 Write 5 Done Done New false
7534 3004 Acked 4003 Read 8 Done Done New false
7535 3005 Acked 4004 Write 6 Done Done New false
7536 3006 Acked 4005 Write 2 Done Done New false
7537 3007 Acked 4006 Read 8 Done Done New false
7538 3008 Acked 4007 Write 7 Done Done New false
7539 3009 Acked 4008 Write 7 Done Done New false
7540 3010 Acked 4009 Read 8 Done Done New false
7541 3011 Acked 4010 Write 10 Done Done New false
7542 3012 Acked 4011 Read 10 Done Done New false
7543 3013 Acked 4012 Write 5 Done Done New false
7544 3014 Acked 4013 Write 4 Done Done New false
7545 3015 Acked 4014 Write 4 Done Done New false
7546 3016 Acked 4015 Read 4 Done Done New false
7547 3017 Acked 4016 Read 10 Done Done New false
7548 3018 Acked 4017 Read 2 Done Done New false
7549 3019 Acked 4018 Flush 0 Done Done New false
7550 3020 Acked 4019 Write 10 Done Done New false
7551 3021 Acked 4020 Write 7 Sent Sent New false
7552 3022 Acked 4021 Write 7 Sent Sent New false
7553 STATES DS:0 DS:1 DS:2 TOTAL
7554 New 0 0 302 302
7555 Sent 2 2 0 4
7556 Done 300 300 0 600
7557 Skipped 0 0 0 0
7558 Error 0 0 0 0
7559 Last Flush: 4018 4018 3719
7560 Downstairs last five completed: 3719 3718 3717 3716 3715 3714
7561 Upstairs last five completed: 3022 3021 3020 3019 3018 3017
7562 CLIENT: Up:0 ds:302 act:2
7563 [8830] got start action
7564 [8830] I am going to restart
7565 Starting downstairs at port 8830
7566 Make output file at "/tmp/dsc/downstairs-8830.txt"
7567 [8830][2] reports Starting
7568 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1183)
7569 [8830][2] reports Running
75702023-09-22T23:25:25.370ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
75712023-09-22T23:25:25.370ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
75722023-09-22T23:25:25.370ZINFOcrucible: [2] client re-new 303 jobs since flush 3719 = downstairs
75732023-09-22T23:25:25.371ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
75742023-09-22T23:25:25.372ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
75752023-09-22T23:25:25.372ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
75762023-09-22T23:25:25.372ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
75772023-09-22T23:25:25.372ZINFOcrucible: [2] send last flush ID to this DS: 3719
75782023-09-22T23:25:25.372ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
75792023-09-22T23:25:25.372ZINFOcrucible: [2] Replied this last flush ID: 3719
75802023-09-22T23:25:25.372ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
75812023-09-22T23:25:25.372ZINFOcrucible: [2] Transition from Offline to Replay
75822023-09-22T23:25:25.372ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
75832023-09-22T23:25:25.372ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
75842023-09-22T23:25:25.372ZINFOcrucible: [2] Transition from Replay to Active
75852023-09-22T23:25:25.372ZINFOcrucible: [2] Starts cmd_loop
75862023-09-22T23:25:25.373ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
75872023-09-22T23:25:25.373ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
75882023-09-22T23:25:25.373ZINFOcrucible: Set check for repair
75892023-09-22T23:25:26.374ZINFOcrucible: Checking if live repair is needed
75902023-09-22T23:25:26.374ZINFOcrucible: No Live Repair required at this time
75912023-09-22T23:25:26.374ZWARNcrucible: [2] flow control end
7592 ----------------------------------------------------------------
7593 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7594 Downstairs last five completed: 4022 4021 4020 4019 4018 4017
7595 Upstairs last five completed: 3023 3022 3021 3020 3019 3018
7596 CLIENT: Up:0 ds:0 act:3
7597 Replay: All jobs finished, all DS active.
7598 Main task has work to do, go find it
7599 got dsc Stop(2)
7600 stop 2
7601 Sep 22 23:25:27.337 INFO request completed, latency_us: 547, response_code: 204, uri: /stop/cid/2, method[8830] Got stop action so:false kr:false
7602 : POST, req_id: b8fcd564-c8c7-4f81-95d8-5df0f99e3f42, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
7603 Sep 22 23:25:27.338 INFO request completed, latency_us: 245, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9f8c9c0b-b9ea-4140-9c32-70335f7a13ba, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
76042023-09-22T23:25:27.338ZWARNcrucible: [2] downstairs disconnected
76052023-09-22T23:25:27.338ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
76062023-09-22T23:25:27.338ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
76072023-09-22T23:25:27.338ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
76082023-09-22T23:25:27.338ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
76092023-09-22T23:25:27.339ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
76102023-09-22T23:25:27.339ZWARNcrucible: [2] pm_task rx.recv() is None
7611 [8830] Exited with: ExitStatus(unix_wait_status(9))
7612 [8830][2] reports Exit
76132023-09-22T23:25:29.340ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
7614 Sep 22 23:25:31.557 INFO request completed, latency_us: 264, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9d49ef19-791d-4f77-b58a-3e7d011b480a, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
7615 001/300 Write block 1401 len 12288 data: 3 3 4
7616 002/300 Read block 3902 len 12288
7617 003/300 Read block 4644 len 24576
7618 004/300 Read block 3086 len 4096
7619 005/300 Read block 2678 len 32768
7620 006/300 Read block 2199 len 24576
7621 007/300 Read block 2866 len 36864
7622 008/300 Read block 2888 len 16384
7623 009/300 Write block 4185 len 32768 data: 4 2 2 2 3 4 5 7
7624 010/300 Write block 2816 len 16384 data: 4 4 3 3
7625 011/300 Flush
7626 012/300 Read block 3763 len 32768
7627 013/300 Write block 2464 len 12288 data: 2 3 3
7628 014/300 Write block 4011 len 4096 data: 2
7629 015/300 Read block 3010 len 12288
7630 016/300 Flush
7631 017/300 Write block 2270 len 36864 data: 4 5 5 5 5 4 4 4 4
7632 018/300 Write block 3846 len 32768 data: 2 2 3 3 4 5 5 5
7633 019/300 Read block 1048 len 16384
7634 020/300 Write block 726 len 32768 data: 7 6 6 5 4 4 2 2
7635 021/300 Read block 3649 len 32768
7636 022/300 Write block 4984 len 36864 data: 5 3 3 4 4 4 3 3 3
7637 023/300 Write block 3305 len 8192 data: 4 4
7638 024/300 Read block 4025 len 8192
7639 025/300 Write block 2870 len 40960 data: 2 3 3 3 3 4 4 6 6 6
7640 026/300 Read block 4492 len 20480
7641 027/300 Write block 3892 len 8192 data: 3 3
7642 028/300 Write block 1977 len 8192 data: 4 3
7643 029/300 Write block 2401 len 16384 data: 2 2 2 3
7644 030/300 Flush
7645 031/300 Write block 4958 len 32768 data: 5 4 2 2 2 2 3 3
7646 032/300 Write block 4598 len 20480 data: 2 3 3 3 6
7647 033/300 Read block 4024 len 16384
7648 034/300 Write block 2180 len 12288 data: 2 2 4
7649 035/300 Read block 408 len 32768
7650 036/300 Write block 2529 len 32768 data: 4 4 4 4 3 2 2 2
7651 037/300 Read block 1171 len 4096
7652 038/300 Read block 2432 len 12288
7653 039/300 Read block 777 len 28672
7654 040/300 Read block 3922 len 4096
7655 041/300 Read block 4248 len 40960
7656 042/300 Read block 2238 len 28672
7657 043/300 Flush
7658 044/300 Read block 3403 len 20480
7659 045/300 Read block 4968 len 12288
7660 046/300 Read block 714 len 4096
7661 047/300 Read block 2540 len 28672
7662 048/300 Read block 4116 len 24576
7663 049/300 Read block 1951 len 24576
7664 050/300 Flush
7665 051/300 Write block 331 len 40960 data: 2 2 3 3 3 3 4 3 4 4
7666 052/300 Read block 877 len 8192
7667 053/300 Read block 3729 len 24576
7668 054/300 Write block 2430 len 24576 data: 4 4 2 2 2 2
7669 055/300 Read block 2895 len 36864
7670 056/300 Read block 276 len 20480
7671 057/300 Read block 1152 len 16384
7672 058/300 Write block 2658 len 20480 data: 3 2 2 2 2
7673 059/300 Read block 2294 len 36864
7674 060/300 Read block 3326 len 4096
7675 061/300 Read block 3331 len 24576
7676 062/300 Read block 3546 len 16384
7677 063/300 Write block 4984 len 20480 data: 6 4 4 5 5
7678 064/300 Read block 18 len 4096
7679 065/300 Read block 1190 len 28672
7680 066/300 Read block 3003 len 24576
7681 067/300 Read block 4355 len 4096
7682 068/300 Read block 1570 len 4096
7683 069/300 Write block 3023 len 24576 data: 3 3 3 3 3 3
7684 070/300 Write block 2471 len 12288 data: 3 4 4
7685 071/300 Write block 3323 len 4096 data: 3
7686 072/300 Read block 3214 len 20480
7687 073/300 Read block 1006 len 24576
7688 074/300 Write block 375 len 16384 data: 3 3 3 3
7689 075/300 Flush
7690 076/300 Read block 3798 len 12288
7691 077/300 Write block 3895 len 36864 data: 3 3 4 5 5 5 5 4 5
7692 078/300 Write block 3168 len 8192 data: 3 3
7693 079/300 Read block 154 len 12288
7694 080/300 Read block 2048 len 36864
7695 081/300 Write block 3567 len 20480 data: 3 3 3 3 3
7696 082/300 Read block 109 len 36864
7697 083/300 Write block 1944 len 28672 data: 4 4 3 3 3 5 4
7698 084/300 Read block 3862 len 24576
7699 085/300 Read block 1785 len 8192
7700 086/300 Flush
7701 087/300 Write block 2534 len 28672 data: 3 3 3 2 2 2 2
7702 088/300 Write block 57 len 28672 data: 5 4 5 5 5 6 5
7703 089/300 Read block 2649 len 20480
7704 090/300 Read block 1108 len 24576
7705 091/300 Read block 2565 len 32768
7706 092/300 Write block 2070 len 4096 data: 3
7707 093/300 Read block 621 len 8192
7708 094/300 Read block 3170 len 28672
7709 095/300 Read block 991 len 36864
7710 096/300 Read block 1535 len 28672
7711 097/300 Read block 1946 len 32768
7712 098/300 Write block 108 len 32768 data: 3 4 4 4 5 5 5 5
7713 099/300 Write block 3733 len 36864 data: 2 3 2 2 2 2 3 3 3
7714 100/300 Write block 4853 len 36864 data: 5 5 4 4 4 4 4 3 2
7715 101/300 Flush
7716 102/300 Write block 523 len 16384 data: 3 3 3 3
7717 103/300 Flush
7718 104/300 Read block 1340 len 24576
7719 105/300 Write block 1035 len 16384 data: 2 2 2 2
7720 106/300 Write block 3714 len 16384 data: 3 4 4 6
7721 107/300 Read block 3360 len 24576
7722 108/300 Flush
7723 109/300 Read block 3307 len 28672
7724 110/300 Write block 1027 len 40960 data: 3 2 2 2 2 2 2 2 3 3
7725 111/300 Read block 2122 len 20480
7726 112/300 Flush
7727 113/300 Flush
7728 114/300 Read block 4401 len 36864
7729 115/300 Read block 1283 len 36864
7730 116/300 Write block 3559 len 24576 data: 2 2 2 2 3 3
7731 117/300 Write block 1942 len 16384 data: 5 6 5 5
7732 118/300 Write block 3577 len 8192 data: 2 3
7733 119/300 Write block 4206 len 4096 data: 4
7734 120/300 Flush
7735 121/300 Read block 2533 len 24576
7736 122/300 Read block 253 len 8192
7737 123/300 Write block 4716 len 28672 data: 4 3 2 2 2 2 2
7738 124/300 Write block 4771 len 4096 data: 2
7739 125/300 Read block 4493 len 20480
7740 126/300 Read block 1536 len 28672
7741 127/300 Read block 3909 len 36864
7742 128/300 Write block 1441 len 32768 data: 2 2 2 2 2 2 3 3
7743 129/300 Flush
7744 130/300 Write block 4159 len 12288 data: 3 3 2
7745 131/300 Read block 4372 len 8192
7746 132/300 Write block 1382 len 24576 data: 4 4 4 3 2 2
7747 133/300 Read block 201 len 16384
7748 134/300 Read block 4733 len 16384
7749 135/300 Write block 217 len 16384 data: 2 2 2 3
7750 136/300 Read block 1427 len 16384
7751 137/300 Write block 2452 len 4096 data: 5
7752 138/300 Write block 1859 len 4096 data: 4
7753 139/300 Write block 4656 len 36864 data: 3 4 4 4 4 3 4 3 3
7754 140/300 Flush
7755 141/300 Write block 16 len 28672 data: 2 3 2 3 3 3 4
7756 142/300 Read block 580 len 32768
7757 143/300 Read block 1098 len 36864
7758 144/300 Flush
7759 145/300 Read block 2783 len 28672
7760 146/300 Read block 2441 len 24576
7761 147/300 Write block 1431 len 12288 data: 7 7 7
7762 148/300 Write block 687 len 16384 data: 2 3 2 2
7763 149/300 Write block 1926 len 16384 data: 2 2 2 2
7764 150/300 Write block 1623 len 40960 data: 3 4 4 3 3 3 5 4 5 6
7765 151/300 Read block 3426 len 36864
7766 152/300 Read block 3373 len 32768
7767 153/300 Write block 2463 len 40960 data: 2 3 4 4 2 2 2 3 4 5
7768 154/300 Write block 2662 len 40960 data: 3 2 3 4 4 4 4 3 3 3
7769 155/300 Read block 3837 len 24576
7770 156/300 Flush
7771 157/300 Flush
7772 158/300 Write block 2999 len 24576 data: 2 2 2 2 2 2
7773 159/300 Read block 1771 len 32768
7774 160/300 Write block 1339 len 16384 data: 4 4 3 3
7775 161/300 Read block 2422 len 24576
7776 162/300 Read block 3088 len 24576
7777 163/300 Read block 2428 len 20480
7778 164/300 Read block 3076 len 16384
7779 165/300 Write block 2509 len 32768 data: 5 5 7 8 7 7 7 7
7780 166/300 Write block 4489 len 28672 data: 4 4 5 5 5 4 3
7781 167/300 Write block 3610 len 36864 data: 3 4 4 4 4 4 3 3 4
7782 168/300 Read block 4331 len 16384
7783 169/300 Read block 348 len 36864
7784 170/300 Flush
7785 171/300 Read block 3875 len 28672
7786 172/300 Read block 4686 len 4096
7787 173/300 Write block 2926 len 4096 data: 3
7788 174/300 Flush
7789 175/300 Read block 855 len 36864
7790 176/300 Read block 4460 len 36864
7791 177/300 Write block 325 len 16384 data: 3 2 4 3
7792 178/300 Read block 3516 len 40960
7793 179/300 Read block 2165 len 4096
7794 180/300 Write block 2112 len 4096 data: 5
7795 181/300 Read block 2051 len 8192
7796 182/300 Write block 3022 len 4096 data: 3
7797 183/300 Flush
7798 184/300 Read block 1381 len 28672
7799 185/300 Read block 2452 len 36864
7800 186/300 Write block 4039 len 8192 data: 3 3
7801 187/300 Read block 2707 len 28672
7802 188/300 Flush
7803 189/300 Read block 2937 len 20480
7804 190/300 Write block 4756 len 32768 data: 5 5 5 4 3 4 6 5
7805 191/300 Write block 913 len 12288 data: 3 3 3
7806 192/300 Write block 2328 len 36864 data: 3 2 2 2 2 2 2 3 3
7807 193/300 Write block 1632 len 28672 data: 7 4 4 4 4 4 4
7808 194/300 Read block 1238 len 36864
7809 195/300 Write block 3100 len 40960 data: 4 4 4 4 4 3 3 3 3 3
7810 196/300 Read block 1910 len 12288
7811 197/300 Read block 4355 len 36864
7812 198/300 Read block 4277 len 40960
7813 199/300 Read block 406 len 28672
7814 200/300 Read block 3649 len 32768
7815 201/300 Flush
7816 202/300 Read block 4555 len 8192
7817 203/300 Read block 2940 len 28672
7818 204/300 Flush
7819 205/300 Read block 2033 len 16384
7820 206/300 Read block 3602 len 36864
7821 207/300 Write block 1401 len 28672 data: 4 4 5 4 5 4 4
7822 208/300 Read block 1423 len 24576
7823 209/300 Read block 105 len 20480
7824 210/300 Read block 4455 len 12288
7825 211/300 Read block 337 len 4096
7826 212/300 Flush
7827 213/300 Write block 2051 len 28672 data: 3 3 3 3 3 4 4
7828 214/300 Write block 768 len 36864 data: 3 3 4 4 3 3 3 3 2
7829 215/300 Flush
7830 216/300 Read block 3525 len 40960
7831 217/300 Write block 3794 len 40960 data: 3 3 2 2 2 2 2 2 2 2
7832 218/300 Read block 2864 len 32768
7833 219/300 Write block 599 len 8192 data: 5 4
7834 220/300 Read block 1604 len 36864
7835 221/300 Read block 632 len 4096
7836 222/300 Write block 4324 len 32768 data: 5 6 4 2 3 4 4 4
7837 223/300 Write block 2299 len 8192 data: 4 5
7838 224/300 Read block 3539 len 28672
7839 225/300 Read block 3745 len 12288
7840 226/300 Write block 4032 len 24576 data: 3 3 4 5 5 5
7841 227/300 Read block 1479 len 4096
7842 228/300 Write block 4489 len 12288 data: 5 5 6
7843 229/300 Write block 2069 len 8192 data: 3 4
7844 230/300 Write block 2087 len 40960 data: 3 2 2 2 3 4 4 4 4 4
7845 231/300 Read block 2843 len 32768
7846 232/300 Write block 2274 len 36864 data: 6 5 5 5 5 5 3 3 2
7847 233/300 Read block 3289 len 12288
7848 234/300 Read block 724 len 20480
7849 235/300 Write block 1791 len 28672 data: 6 6 5 7 8 8 8
7850 236/300 Read block 4215 len 36864
7851 237/300 Write block 3312 len 32768 data: 3 3 4 2 2 2 2 2
7852 238/300 Read block 1841 len 20480
7853 239/300 Read block 1803 len 16384
7854 240/300 Flush
7855 241/300 Write block 2236 len 36864 data: 2 2 3 3 3 3 2 2 2
7856 242/300 Read block 2811 len 36864
7857 243/300 Read block 612 len 4096
7858 244/300 Write block 4948 len 4096 data: 2
7859 245/300 Write block 731 len 40960 data: 5 3 3 2 2 2 2 3 3 4
7860 246/300 Write block 1205 len 20480 data: 6 6 6 7 6
7861 247/300 Write block 3488 len 36864 data: 3 3 3 2 2 2 2 2 2
7862 248/300 Read block 2048 len 8192
7863 249/300 Write block 2894 len 24576 data: 6 5 5 5 5 4
7864 250/300 Write block 4099 len 32768 data: 4 4 4 3 3 2 2 3
7865 251/300 Read block 3557 len 16384
7866 252/300 Read block 579 len 40960
7867 253/300 Write block 1522 len 12288 data: 2 2 2
7868 254/300 Write block 4211 len 28672 data: 4 4 5 4 4 4 3
7869 255/300 Write block 1139 len 4096 data: 3
7870 256/300 Read block 4327 len 32768
7871 257/300 Read block 970 len 16384
7872 258/300 Flush
7873 259/300 Write block 546 len 8192 data: 3 3
7874 260/300 Read block 3134 len 4096
7875 261/300 Read block 4359 len 16384
7876 262/300 Write block 1296 len 40960 data: 4 4 4 2 2 2 3 3 3 3
7877 263/300 Read block 1799 len 24576
7878 264/300 Write block 2390 len 36864 data: 4 4 3 3 3 3 3 2 2
7879 265/300 Read block 4645 len 40960
7880 266/300 Write block 4543 len 28672 data: 2 2 2 2 2 3 4
7881 267/300 Read block 429 len 20480
7882 268/300 Read block 2200 len 4096
7883 269/300 Read block 2419 len 40960
7884 270/300 Write block 4737 len 36864 data: 4 2 3 3 4 4 4 5 4
7885 271/300 Write block 806 len 36864 data: 4 4 5 5 5 4 5 6 6
7886 272/300 Read block 4637 len 8192
7887 273/300 Flush
7888 274/300 Write block 2643 len 8192 data: 2 2
7889 275/300 Read block 3196 len 36864
7890 276/300 Write block 1880 len 20480 data: 3 2 2 2 2
7891 277/300 Write block 4225 len 16384 data: 2 2 5 6
7892 278/300 Read block 239 len 4096
7893 279/300 Write block 3701 len 28672 data: 4 5 5 3 3 3 3
7894 280/300 Write block 1591 len 36864 data: 3 4 4 4 4 4 3 2 2
7895 281/300 Write block 666 len 20480 data: 7 6 5 5 4
7896 282/300 Read block 3798 len 16384
7897 283/300 Flush
7898 284/300 Write block 4168 len 28672 data: 2 2 2 2 3 3 3
7899 285/300 Write block 2219 len 12288 data: 4 4 4
7900 286/300 Read block 4975 len 32768
7901 287/300 Read block 4454 len 8192
7902 288/300 Read block 3692 len 28672
7903 289/300 Read block 3773 len 32768
7904 290/300 Read block 1574 len 16384
7905 291/300 Read block 1462 len 32768
7906 292/300 Read block 3202 len 40960
7907 293/300 Read block 3476 len 24576
7908 294/300 Read block 2604 len 32768
7909 295/300 Write block 3296 len 28672 data: 4 4 3 3 3 4 5
7910 296/300 Read block 2668 len 4096
7911 297/300 Write block 4812 len 12288 data: 3 4 3
7912 298/300 Read block 1182 len 40960
7913 299/300 Read block 662 len 8192
7914 300/300 Read block 3759 len 4096
7915 Main task has work to do, go find it
7916 got dsc Start(2)
7917 start 2
7918 Sep 22 23:25:32.606 INFO request completed, latency_us: 303, response_code: 204, uri: /start/cid/2, method: POST, req_id: a58235ea-83dc-4cf1-ab31-9d3de7400fac, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
7919 Replay: started 2, returned:Ok(())
7920 ----------------------------------------------------------------
7921 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7922 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
7923 3024 Acked 4023 Write 3 Done Done New false
7924 3025 Acked 4024 Read 3 Done Done New false
7925 3026 Acked 4025 Read 6 Done Done New false
7926 3027 Acked 4026 Read 1 Done Done New false
7927 3028 Acked 4027 Read 8 Done Done New false
7928 3029 Acked 4028 Read 6 Done Done New false
7929 3030 Acked 4029 Read 9 Done Done New false
7930 3031 Acked 4030 Read 4 Done Done New false
7931 3032 Acked 4031 Write 8 Done Done New false
7932 3033 Acked 4032 Write 4 Done Done New false
7933 3034 Acked 4033 Flush 0 Done Done New false
7934 3035 Acked 4034 Read 8 Done Done New false
7935 3036 Acked 4035 Write 3 Done Done New false
7936 3037 Acked 4036 Write 1 Done Done New false
7937 3038 Acked 4037 Read 3 Done Done New false
7938 3039 Acked 4038 Flush 0 Done Done New false
7939 3040 Acked 4039 Write 9 Done Done New false
7940 3041 Acked 4040 Write 8 Done Done New false
7941 3042 Acked 4041 Read 4 Done Done New false
7942 3043 Acked 4042 Write 8 Done Done New false
7943 3044 Acked 4043 Read 8 Done Done New false
7944 3045 Acked 4044 Write 9 Done Done New false
7945 3046 Acked 4045 Write 2 Done Done New false
7946 3047 Acked 4046 Read 2 Done Done New false
7947 3048 Acked 4047 Write 10 Done Done New false
7948 3049 Acked 4048 Read 5 Done Done New false
7949 3050 Acked 4049 Write 2 Done Done New false
7950 3051 Acked 4050 Write 2 Done Done New false
7951 3052 Acked 4051 Write 4 Done Done New false
7952 3053 Acked 4052 Flush 0 Done Done New false
7953 3054 Acked 4053 Write 8 Done Done New false
7954 3055 Acked 4054 Write 5 Done Done New false
7955 3056 Acked 4055 Read 4 Done Done New false
7956 3057 Acked 4056 Write 3 Done Done New false
7957 3058 Acked 4057 Read 8 Done Done New false
7958 3059 Acked 4058 Write 8 Done Done New false
7959 3060 Acked 4059 Read 1 Done Done New false
7960 3061 Acked 4060 Read 3 Done Done New false
7961 3062 Acked 4061 Read 7 Done Done New false
7962 3063 Acked 4062 Read 1 Done Done New false
7963 3064 Acked 4063 Read 10 Done Done New false
7964 3065 Acked 4064 Read 7 Done Done New false
7965 3066 Acked 4065 Flush 0 Done Done New false
7966 3067 Acked 4066 Flush 0 Done Done New false
7967 3068 Acked 4067 Read 5 Done Done New false
7968 3069 Acked 4068 Read 3 Done Done New false
7969 3070 Acked 4069 Read 1 Done Done New false
7970 3071 Acked 4070 Read 7 Done Done New false
7971 3072 Acked 4071 Read 6 Done Done New false
7972 3073 Acked 4072 Read 6 Done Done New false
7973 3074 Acked 4073 Flush 0 Done Done New false
7974 3075 Acked 4074 Write 10 Done Done New false
7975 3076 Acked 4075 Read 2 Done Done New false
7976 3077 Acked 4076 Read 6 Done Done New false
7977 3078 Acked 4077 Write 6 Done Done New false
7978 3079 Acked 4078 Read 9 Done Done New false
7979 3080 Acked 4079 Read 5 Done Done New false
7980 3081 Acked 4080 Read 4 Done Done New false
7981 3082 Acked 4081 Write 5 Done Done New false
7982 3083 Acked 4082 Read 9 Done Done New false
7983 3084 Acked 4083 Read 1 Done Done New false
7984 3085 Acked 4084 Read 6 Done Done New false
7985 3086 Acked 4085 Read 4 Done Done New false
7986 3087 Acked 4086 Write 5 Done Done New false
7987 3088 Acked 4087 Read 1 Done Done New false
7988 3089 Acked 4088 Read 7 Done Done New false
7989 3090 Acked 4089 Read 6 Done Done New false
7990 3091 Acked 4090 Read 1 Done Done New false
7991 3092 Acked 4091 Read 1 Done Done New false
7992 3093 Acked 4092 Write 6 Done Done New false
7993 3094 Acked 4093 Write 3 Done Done New false
7994 3095 Acked 4094 Write 1 Done Done New false
7995 3096 Acked 4095 Read 5 Done Done New false
7996 3097 Acked 4096 Read 6 Done Done New false
7997 3098 Acked 4097 Write 4 Done Done New false
7998 3099 Acked 4098 Flush 0 Done Done New false
7999 3100 Acked 4099 Read 3 Done Done New false
8000 3101 Acked 4100 Write 9 Done Done New false
8001 3102 Acked 4101 Write 2 Done Done New false
8002 3103 Acked 4102 Read 3 Done Done New false
8003 3104 Acked 4103 Read 9 Done Done New false
8004 3105 Acked 4104 Write 5 Done Done New false
8005 3106 Acked 4105 Read 9 Done Done New false
8006 3107 Acked 4106 Write 7 Done Done New false
8007 3108 Acked 4107 Read 6 Done Done New false
8008 3109 Acked 4108 Read 2 Done Done New false
8009 3110 Acked 4109 Flush 0 Done Done New false
8010 3111 Acked 4110 Write 7 Done Done New false
8011 3112 Acked 4111 Write 7 Done Done New false
8012 3113 Acked 4112 Read 5 Done Done New false
8013 3114 Acked 4113 Read 6 Done Done New false
8014 3115 Acked 4114 Read 8 Done Done New false
8015 3116 Acked 4115 Write 1 Done Done New false
8016 3117 Acked 4116 Read 2 Done Done New false
8017 3118 Acked 4117 Read 7 Done Done New false
8018 3119 Acked 4118 Read 9 Done Done New false
8019 3120 Acked 4119 Read 7 Done Done New false
8020 3121 Acked 4120 Read 8 Done Done New false
8021 3122 Acked 4121 Write 8 Done Done New false
8022 3123 Acked 4122 Write 9 Done Done New false
8023 3124 Acked 4123 Write 9 Done Done New false
8024 3125 Acked 4124 Flush 0 Done Done New false
8025 3126 Acked 4125 Write 4 Done Done New false
8026 3127 Acked 4126 Flush 0 Done Done New false
8027 3128 Acked 4127 Read 6 Done Done New false
8028 3129 Acked 4128 Write 4 Done Done New false
8029 3130 Acked 4129 Write 4 Done Done New false
8030 3131 Acked 4130 Read 6 Done Done New false
8031 3132 Acked 4131 Flush 0 Done Done New false
8032 3133 Acked 4132 Read 7 Done Done New false
8033 3134 Acked 4133 Write 10 Done Done New false
8034 3135 Acked 4134 Read 5 Done Done New false
8035 3136 Acked 4135 Flush 0 Done Done New false
8036 3137 Acked 4136 Flush 0 Done Done New false
8037 3138 Acked 4137 Read 9 Done Done New false
8038 3139 Acked 4138 Read 9 Done Done New false
8039 3140 Acked 4139 Write 6 Done Done New false
8040 3141 Acked 4140 Write 4 Done Done New false
8041 3142 Acked 4141 Write 2 Done Done New false
8042 3143 Acked 4142 Write 1 Done Done New false
8043 3144 Acked 4143 Flush 0 Done Done New false
8044 3145 Acked 4144 Read 6 Done Done New false
8045 3146 Acked 4145 Read 2 Done Done New false
8046 3147 Acked 4146 Write 7 Done Done New false
8047 3148 Acked 4147 Write 1 Done Done New false
8048 3149 Acked 4148 Read 5 Done Done New false
8049 3150 Acked 4149 Read 7 Done Done New false
8050 3151 Acked 4150 Read 9 Done Done New false
8051 3152 Acked 4151 Write 8 Done Done New false
8052 3153 Acked 4152 Flush 0 Done Done New false
8053 3154 Acked 4153 Write 3 Done Done New false
8054 3155 Acked 4154 Read 2 Done Done New false
8055 3156 Acked 4155 Write 6 Done Done New false
8056 3157 Acked 4156 Read 4 Done Done New false
8057 3158 Acked 4157 Read 4 Done Done New false
8058 3159 Acked 4158 Write 4 Done Done New false
8059 3160 Acked 4159 Read 4 Done Done New false
8060 3161 Acked 4160 Write 1 Done Done New false
8061 3162 Acked 4161 Write 1 Done Done New false
8062 3163 Acked 4162 Write 9 Done Done New false
8063 3164 Acked 4163 Flush 0 Done Done New false
8064 3165 Acked 4164 Write 7 Done Done New false
8065 3166 Acked 4165 Read 8 Done Done New false
8066 3167 Acked 4166 Read 9 Done Done New false
8067 3168 Acked 4167 Flush 0 Done Done New false
8068 3169 Acked 4168 Read 7 Done Done New false
8069 3170 Acked 4169 Read 6 Done Done New false
8070 3171 Acked 4170 Write 3 Done Done New false
8071 3172 Acked 4171 Write 4 Done Done New false
8072 3173 Acked 4172 Write 4 Done Done New false
8073 3174 Acked 4173 Write 10 Done Done New false
8074 3175 Acked 4174 Read 9 Done Done New false
8075 3176 Acked 4175 Read 8 Done Done New false
8076 3177 Acked 4176 Write 10 Done Done New false
8077 3178 Acked 4177 Write 10 Done Done New false
8078 3179 Acked 4178 Read 6 Done Done New false
8079 3180 Acked 4179 Flush 0 Done Done New false
8080 3181 Acked 4180 Flush 0 Done Done New false
8081 3182 Acked 4181 Write 6 Done Done New false
8082 3183 Acked 4182 Read 8 Done Done New false
8083 3184 Acked 4183 Write 4 Done Done New false
8084 3185 Acked 4184 Read 6 Done Done New false
8085 3186 Acked 4185 Read 6 Done Done New false
8086 3187 Acked 4186 Read 5 Done Done New false
8087 3188 Acked 4187 Read 4 Done Done New false
8088 3189 Acked 4188 Write 8 Done Done New false
8089 3190 Acked 4189 Write 7 Done Done New false
8090 3191 Acked 4190 Write 9 Done Done New false
8091 3192 Acked 4191 Read 4 Done Done New false
8092 3193 Acked 4192 Read 9 Done Done New false
8093 3194 Acked 4193 Flush 0 Done Done New false
8094 3195 Acked 4194 Read 7 Done Done New false
8095 3196 Acked 4195 Flush 0 Done Done New false
8096 3197 Acked 4196 Read 1 Done Done New false
8097 3198 Acked 4197 Write 1 Done Done New false
8098 3199 Acked 4198 Flush 0 Done Done New false
8099 3200 Acked 4199 Read 9 Done Done New false
8100 3201 Acked 4200 Read 9 Done Done New false
8101 3202 Acked 4201 Write 4 Done Done New false
8102 3203 Acked 4202 Read 10 Done Done New false
8103 3204 Acked 4203 Read 1 Done Done New false
8104 3205 Acked 4204 Write 1 Done Done New false
8105 3206 Acked 4205 Read 2 Done Done New false
8106 3207 Acked 4206 Write 1 Done Done New false
8107 3208 Acked 4207 Flush 0 Done Done New false
8108 3209 Acked 4208 Read 7 Done Done New false
8109 3210 Acked 4209 Read 9 Done Done New false
8110 3211 Acked 4210 Write 2 Done Done New false
8111 3212 Acked 4211 Read 7 Done Done New false
8112 3213 Acked 4212 Flush 0 Done Done New false
8113 3214 Acked 4213 Read 5 Done Done New false
8114 3215 Acked 4214 Write 8 Done Done New false
8115 3216 Acked 4215 Write 3 Done Done New false
8116 3217 Acked 4216 Write 9 Done Done New false
8117 3218 Acked 4217 Write 7 Done Done New false
8118 3219 Acked 4218 Read 9 Done Done New false
8119 3220 Acked 4219 Write 10 Done Done New false
8120 3221 Acked 4220 Read 3 Done Done New false
8121 3222 Acked 4221 Read 9 Done Done New false
8122 3223 Acked 4222 Read 10 Done Done New false
8123 3224 Acked 4223 Read 7 Done Done New false
8124 3225 Acked 4224 Read 8 Done Done New false
8125 3226 Acked 4225 Flush 0 Done Done New false
8126 3227 Acked 4226 Read 2 Done Done New false
8127 3228 Acked 4227 Read 7 Done Done New false
8128 3229 Acked 4228 Flush 0 Done Done New false
8129 3230 Acked 4229 Read 4 Done Done New false
8130 3231 Acked 4230 Read 9 Done Done New false
8131 3232 Acked 4231 Write 7 Done Done New false
8132 3233 Acked 4232 Read 6 Done Done New false
8133 3234 Acked 4233 Read 5 Done Done New false
8134 3235 Acked 4234 Read 3 Done Done New false
8135 3236 Acked 4235 Read 1 Done Done New false
8136 3237 Acked 4236 Flush 0 Done Done New false
8137 3238 Acked 4237 Write 7 Done Done New false
8138 3239 Acked 4238 Write 9 Done Done New false
8139 3240 Acked 4239 Flush 0 Done Done New false
8140 3241 Acked 4240 Read 10 Done Done New false
8141 3242 Acked 4241 Write 10 Done Done New false
8142 3243 Acked 4242 Read 8 Done Done New false
8143 3244 Acked 4243 Write 2 Done Done New false
8144 3245 Acked 4244 Read 9 Done Done New false
8145 3246 Acked 4245 Read 1 Done Done New false
8146 3247 Acked 4246 Write 8 Done Done New false
8147 3248 Acked 4247 Write 2 Done Done New false
8148 3249 Acked 4248 Read 7 Done Done New false
8149 3250 Acked 4249 Read 3 Done Done New false
8150 3251 Acked 4250 Write 6 Done Done New false
8151 3252 Acked 4251 Read 1 Done Done New false
8152 3253 Acked 4252 Write 3 Done Done New false
8153 3254 Acked 4253 Write 2 Done Done New false
8154 3255 Acked 4254 Write 10 Done Done New false
8155 3256 Acked 4255 Read 8 Done Done New false
8156 3257 Acked 4256 Write 9 Done Done New false
8157 3258 Acked 4257 Read 3 Done Done New false
8158 3259 Acked 4258 Read 5 Done Done New false
8159 3260 Acked 4259 Write 7 Done Done New false
8160 3261 Acked 4260 Read 9 Done Done New false
8161 3262 Acked 4261 Write 8 Done Done New false
8162 3263 Acked 4262 Read 5 Done Done New false
8163 3264 Acked 4263 Read 4 Done Done New false
8164 3265 Acked 4264 Flush 0 Done Done New false
8165 3266 Acked 4265 Write 9 Done Done New false
8166 3267 Acked 4266 Read 9 Done Done New false
8167 3268 Acked 4267 Read 1 Done Done New false
8168 3269 Acked 4268 Write 1 Done Done New false
8169 3270 Acked 4269 Write 10 Done Done New false
8170 3271 Acked 4270 Write 5 Done Done New false
8171 3272 Acked 4271 Write 9 Done Done New false
8172 3273 Acked 4272 Read 2 Done Done New false
8173 3274 Acked 4273 Write 6 Done Done New false
8174 3275 Acked 4274 Write 8 Done Done New false
8175 3276 Acked 4275 Read 4 Done Done New false
8176 3277 Acked 4276 Read 10 Done Done New false
8177 3278 Acked 4277 Write 3 Done Done New false
8178 3279 Acked 4278 Write 7 Done Done New false
8179 3280 Acked 4279 Write 1 Done Done New false
8180 3281 Acked 4280 Read 8 Done Done New false
8181 3282 Acked 4281 Read 4 Done Done New false
8182 3283 Acked 4282 Flush 0 Done Done New false
8183 3284 Acked 4283 Write 2 Done Done New false
8184 3285 Acked 4284 Read 1 Done Done New false
8185 3286 Acked 4285 Read 4 Done Done New false
8186 3287 Acked 4286 Write 10 Done Done New false
8187 3288 Acked 4287 Read 6 Done Done New false
8188 3289 Acked 4288 Write 9 Done Done New false
8189 3290 Acked 4289 Read 10 Done Done New false
8190 3291 Acked 4290 Write 7 Done Done New false
8191 3292 Acked 4291 Read 5 Done Done New false
8192 3293 Acked 4292 Read 1 Done Done New false
8193 3294 Acked 4293 Read 10 Done Done New false
8194 3295 Acked 4294 Write 9 Done Done New false
8195 3296 Acked 4295 Write 9 Done Done New false
8196 3297 Acked 4296 Read 2 Done Done New false
8197 3298 Acked 4297 Flush 0 Done Done New false
8198 3299 Acked 4298 Write 2 Done Done New false
8199 3300 Acked 4299 Read 9 Done Done New false
8200 3301 Acked 4300 Write 5 Done Done New false
8201 3302 Acked 4301 Write 4 Done Done New false
8202 3303 Acked 4302 Read 1 Done Done New false
8203 3304 Acked 4303 Write 7 Done Done New false
8204 3305 Acked 4304 Write 9 Done Done New false
8205 3306 Acked 4305 Write 5 Done Done New false
8206 3307 Acked 4306 Read 4 Done Done New false
8207 3308 Acked 4307 Flush 0 Done Done New false
8208 3309 Acked 4308 Write 7 Done Done New false
8209 3310 Acked 4309 Write 3 Done Done New false
8210 3311 Acked 4310 Read 8 Done Done New false
8211 3312 Acked 4311 Read 2 Done Done New false
8212 3313 Acked 4312 Read 7 Done Done New false
8213 3314 Acked 4313 Read 8 Done Done New false
8214 3315 Acked 4314 Read 4 Done Done New false
8215 3316 Acked 4315 Read 8 Done Done New false
8216 3317 Acked 4316 Read 10 Done Done New false
8217 3318 Acked 4317 Read 6 Done Done New false
8218 3319 Acked 4318 Read 8 Done Done New false
8219 3320 Acked 4319 Write 7 Done Done New false
8220 3321 Acked 4320 Read 1 Done Done New false
8221 3322 Acked 4321 Write 3 Done Done New false
8222 3323 Acked 4322 Read 10 Done Done New false
8223 3324 Acked 4323 Read 2 Done Done New false
8224 3325 Acked 4324 Read 1 Done Done New false
8225 STATES DS:0 DS:1 DS:2 TOTAL
8226 New 0 0 302 302
8227 Sent 0 0 0 0
8228 Done 302 302 0 604
8229 Skipped 0 0 0 0
8230 Error 0 0 0 0
8231 Last Flush: 4307 4307 4022
8232 Downstairs last five completed: 4022 4021 4020 4019 4018 4017
8233 Upstairs last five completed: 3325 3324 3323 3322 3321 3320
8234 CLIENT: Up:0 ds:302 act:2
8235 ----------------------------------------------------------------
8236 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
8237 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
8238 3024 Acked 4023 Write 3 Done Done New false
8239 3025 Acked 4024 Read 3 Done Done New false
8240 3026 Acked 4025 Read 6 Done Done New false
8241 3027 Acked 4026 Read 1 Done Done New false
8242 3028 Acked 4027 Read 8 Done Done New false
8243 3029 Acked 4028 Read 6 Done Done New false
8244 3030 Acked 4029 Read 9 Done Done New false
8245 3031 Acked 4030 Read 4 Done Done New false
8246 3032 Acked 4031 Write 8 Done Done New false
8247 3033 Acked 4032 Write 4 Done Done New false
8248 3034 Acked 4033 Flush 0 Done Done New false
8249 3035 Acked 4034 Read 8 Done Done New false
8250 3036 Acked 4035 Write 3 Done Done New false
8251 3037 Acked 4036 Write 1 Done Done New false
8252 3038 Acked 4037 Read 3 Done Done New false
8253 3039 Acked 4038 Flush 0 Done Done New false
8254 3040 Acked 4039 Write 9 Done Done New false
8255 3041 Acked 4040 Write 8 Done Done New false
8256 3042 Acked 4041 Read 4 Done Done New false
8257 3043 Acked 4042 Write 8 Done Done New false
8258 3044 Acked 4043 Read 8 Done Done New false
8259 3045 Acked 4044 Write 9 Done Done New false
8260 3046 Acked 4045 Write 2 Done Done New false
8261 3047 Acked 4046 Read 2 Done Done New false
8262 3048 Acked 4047 Write 10 Done Done New false
8263 3049 Acked 4048 Read 5 Done Done New false
8264 3050 Acked 4049 Write 2 Done Done New false
8265 3051 Acked 4050 Write 2 Done Done New false
8266 3052 Acked 4051 Write 4 Done Done New false
8267 3053 Acked 4052 Flush 0 Done Done New false
8268 3054 Acked 4053 Write 8 Done Done New false
8269 3055 Acked 4054 Write 5 Done Done New false
8270 3056 Acked 4055 Read 4 Done Done New false
8271 3057 Acked 4056 Write 3 Done Done New false
8272 3058 Acked 4057 Read 8 Done Done New false
8273 3059 Acked 4058 Write 8 Done Done New false
8274 3060 Acked 4059 Read 1 Done Done New false
8275 3061 Acked 4060 Read 3 Done Done New false
8276 3062 Acked 4061 Read 7 Done Done New false
8277 3063 Acked 4062 Read 1 Done Done New false
8278 3064 Acked 4063 Read 10 Done Done New false
8279 3065 Acked 4064 Read 7 Done Done New false
8280 3066 Acked 4065 Flush 0 Done Done New false
8281 3067 Acked 4066 Flush 0 Done Done New false
8282 3068 Acked 4067 Read 5 Done Done New false
8283 3069 Acked 4068 Read 3 Done Done New false
8284 3070 Acked 4069 Read 1 Done Done New false
8285 3071 Acked 4070 Read 7 Done Done New false
8286 3072 Acked 4071 Read 6 Done Done New false
8287 3073 Acked 4072 Read 6 Done Done New false
8288 3074 Acked 4073 Flush 0 Done Done New false
8289 3075 Acked 4074 Write 10 Done Done New false
8290 3076 Acked 4075 Read 2 Done Done New false
8291 3077 Acked 4076 Read 6 Done Done New false
8292 3078 Acked 4077 Write 6 Done Done New false
8293 3079 Acked 4078 Read 9 Done Done New false
8294 3080 Acked 4079 Read 5 Done Done New false
8295 3081 Acked 4080 Read 4 Done Done New false
8296 3082 Acked 4081 Write 5 Done Done New false
8297 3083 Acked 4082 Read 9 Done Done New false
8298 3084 Acked 4083 Read 1 Done Done New false
8299 3085 Acked 4084 Read 6 Done Done New false
8300 3086 Acked 4085 Read 4 Done Done New false
8301 3087 Acked 4086 Write 5 Done Done New false
8302 3088 Acked 4087 Read 1 Done Done New false
8303 3089 Acked 4088 Read 7 Done Done New false
8304 3090 Acked 4089 Read 6 Done Done New false
8305 3091 Acked 4090 Read 1 Done Done New false
8306 3092 Acked 4091 Read 1 Done Done New false
8307 3093 Acked 4092 Write 6 Done Done New false
8308 3094 Acked 4093 Write 3 Done Done New false
8309 3095 Acked 4094 Write 1 Done Done New false
8310 3096 Acked 4095 Read 5 Done Done New false
8311 3097 Acked 4096 Read 6 Done Done New false
8312 3098 Acked 4097 Write 4 Done Done New false
8313 3099 Acked 4098 Flush 0 Done Done New false
8314 3100 Acked 4099 Read 3 Done Done New false
8315 3101 Acked 4100 Write 9 Done Done New false
8316 3102 Acked 4101 Write 2 Done Done New false
8317 3103 Acked 4102 Read 3 Done Done New false
8318 3104 Acked 4103 Read 9 Done Done New false
8319 3105 Acked 4104 Write 5 Done Done New false
8320 3106 Acked 4105 Read 9 Done Done New false
8321 3107 Acked 4106 Write 7 Done Done New false
8322 3108 Acked 4107 Read 6 Done Done New false
8323 3109 Acked 4108 Read 2 Done Done New false
8324 3110 Acked 4109 Flush 0 Done Done New false
8325 3111 Acked 4110 Write 7 Done Done New false
8326 3112 Acked 4111 Write 7 Done Done New false
8327 3113 Acked 4112 Read 5 Done Done New false
8328 3114 Acked 4113 Read 6 Done Done New false
8329 3115 Acked 4114 Read 8 Done Done New false
8330 3116 Acked 4115 Write 1 Done Done New false
8331 3117 Acked 4116 Read 2 Done Done New false
8332 3118 Acked 4117 Read 7 Done Done New false
8333 3119 Acked 4118 Read 9 Done Done New false
8334 3120 Acked 4119 Read 7 Done Done New false
8335 3121 Acked 4120 Read 8 Done Done New false
8336 3122 Acked 4121 Write 8 Done Done New false
8337 3123 Acked 4122 Write 9 Done Done New false
8338 3124 Acked 4123 Write 9 Done Done New false
8339 3125 Acked 4124 Flush 0 Done Done New false
8340 3126 Acked 4125 Write 4 Done Done New false
8341 3127 Acked 4126 Flush 0 Done Done New false
8342 3128 Acked 4127 Read 6 Done Done New false
8343 3129 Acked 4128 Write 4 Done Done New false
8344 3130 Acked 4129 Write 4 Done Done New false
8345 3131 Acked 4130 Read 6 Done Done New false
8346 3132 Acked 4131 Flush 0 Done Done New false
8347 3133 Acked 4132 Read 7 Done Done New false
8348 3134 Acked 4133 Write 10 Done Done New false
8349 3135 Acked 4134 Read 5 Done Done New false
8350 3136 Acked 4135 Flush 0 Done Done New false
8351 3137 Acked 4136 Flush 0 Done Done New false
8352 3138 Acked 4137 Read 9 Done Done New false
8353 3139 Acked 4138 Read 9 Done Done New false
8354 3140 Acked 4139 Write 6 Done Done New false
8355 3141 Acked 4140 Write 4 Done Done New false
8356 3142 Acked 4141 Write 2 Done Done New false
8357 3143 Acked 4142 Write 1 Done Done New false
8358 3144 Acked 4143 Flush 0 Done Done New false
8359 3145 Acked 4144 Read 6 Done Done New false
8360 3146 Acked 4145 Read 2 Done Done New false
8361 3147 Acked 4146 Write 7 Done Done New false
8362 3148 Acked 4147 Write 1 Done Done New false
8363 3149 Acked 4148 Read 5 Done Done New false
8364 3150 Acked 4149 Read 7 Done Done New false
8365 3151 Acked 4150 Read 9 Done Done New false
8366 3152 Acked 4151 Write 8 Done Done New false
8367 3153 Acked 4152 Flush 0 Done Done New false
8368 3154 Acked 4153 Write 3 Done Done New false
8369 3155 Acked 4154 Read 2 Done Done New false
8370 3156 Acked 4155 Write 6 Done Done New false
8371 3157 Acked 4156 Read 4 Done Done New false
8372 3158 Acked 4157 Read 4 Done Done New false
8373 3159 Acked 4158 Write 4 Done Done New false
8374 3160 Acked 4159 Read 4 Done Done New false
8375 3161 Acked 4160 Write 1 Done Done New false
8376 3162 Acked 4161 Write 1 Done Done New false
8377 3163 Acked 4162 Write 9 Done Done New false
8378 3164 Acked 4163 Flush 0 Done Done New false
8379 3165 Acked 4164 Write 7 Done Done New false
8380 3166 Acked 4165 Read 8 Done Done New false
8381 3167 Acked 4166 Read 9 Done Done New false
8382 3168 Acked 4167 Flush 0 Done Done New false
8383 3169 Acked 4168 Read 7 Done Done New false
8384 3170 Acked 4169 Read 6 Done Done New false
8385 3171 Acked 4170 Write 3 Done Done New false
8386 3172 Acked 4171 Write 4 Done Done New false
8387 3173 Acked 4172 Write 4 Done Done New false
8388 3174 Acked 4173 Write 10 Done Done New false
8389 3175 Acked 4174 Read 9 Done Done New false
8390 3176 Acked 4175 Read 8 Done Done New false
8391 3177 Acked 4176 Write 10 Done Done New false
8392 3178 Acked 4177 Write 10 Done Done New false
8393 3179 Acked 4178 Read 6 Done Done New false
8394 3180 Acked 4179 Flush 0 Done Done New false
8395 3181 Acked 4180 Flush 0 Done Done New false
8396 3182 Acked 4181 Write 6 Done Done New false
8397 3183 Acked 4182 Read 8 Done Done New false
8398 3184 Acked 4183 Write 4 Done Done New false
8399 3185 Acked 4184 Read 6 Done Done New false
8400 3186 Acked 4185 Read 6 Done Done New false
8401 3187 Acked 4186 Read 5 Done Done New false
8402 3188 Acked 4187 Read 4 Done Done New false
8403 3189 Acked 4188 Write 8 Done Done New false
8404 3190 Acked 4189 Write 7 Done Done New false
8405 3191 Acked 4190 Write 9 Done Done New false
8406 3192 Acked 4191 Read 4 Done Done New false
8407 3193 Acked 4192 Read 9 Done Done New false
8408 3194 Acked 4193 Flush 0 Done Done New false
8409 3195 Acked 4194 Read 7 Done Done New false
8410 3196 Acked 4195 Flush 0 Done Done New false
8411 3197 Acked 4196 Read 1 Done Done New false
8412 3198 Acked 4197 Write 1 Done Done New false
8413 3199 Acked 4198 Flush 0 Done Done New false
8414 3200 Acked 4199 Read 9 Done Done New false
8415 3201 Acked 4200 Read 9 Done Done New false
8416 3202 Acked 4201 Write 4 Done Done New false
8417 3203 Acked 4202 Read 10 Done Done New false
8418 3204 Acked 4203 Read 1 Done Done New false
8419 3205 Acked 4204 Write 1 Done Done New false
8420 3206 Acked 4205 Read 2 Done Done New false
8421 3207 Acked 4206 Write 1 Done Done New false
8422 3208 Acked 4207 Flush 0 Done Done New false
8423 3209 Acked 4208 Read 7 Done Done New false
8424 3210 Acked 4209 Read 9 Done Done New false
8425 3211 Acked 4210 Write 2 Done Done New false
8426 3212 Acked 4211 Read 7 Done Done New false
8427 3213 Acked 4212 Flush 0 Done Done New false
8428 3214 Acked 4213 Read 5 Done Done New false
8429 3215 Acked 4214 Write 8 Done Done New false
8430 3216 Acked 4215 Write 3 Done Done New false
8431 3217 Acked 4216 Write 9 Done Done New false
8432 3218 Acked 4217 Write 7 Done Done New false
8433 3219 Acked 4218 Read 9 Done Done New false
8434 3220 Acked 4219 Write 10 Done Done New false
8435 3221 Acked 4220 Read 3 Done Done New false
8436 3222 Acked 4221 Read 9 Done Done New false
8437 3223 Acked 4222 Read 10 Done Done New false
8438 3224 Acked 4223 Read 7 Done Done New false
8439 3225 Acked 4224 Read 8 Done Done New false
8440 3226 Acked 4225 Flush 0 Done Done New false
8441 3227 Acked 4226 Read 2 Done Done New false
8442 3228 Acked 4227 Read 7 Done Done New false
8443 3229 Acked 4228 Flush 0 Done Done New false
8444 3230 Acked 4229 Read 4 Done Done New false
8445 3231 Acked 4230 Read 9 Done Done New false
8446 3232 Acked 4231 Write 7 Done Done New false
8447 3233 Acked 4232 Read 6 Done Done New false
8448 3234 Acked 4233 Read 5 Done Done New false
8449 3235 Acked 4234 Read 3 Done Done New false
8450 3236 Acked 4235 Read 1 Done Done New false
8451 3237 Acked 4236 Flush 0 Done Done New false
8452 3238 Acked 4237 Write 7 Done Done New false
8453 3239 Acked 4238 Write 9 Done Done New false
8454 3240 Acked 4239 Flush 0 Done Done New false
8455 3241 Acked 4240 Read 10 Done Done New false
8456 3242 Acked 4241 Write 10 Done Done New false
8457 3243 Acked 4242 Read 8 Done Done New false
8458 3244 Acked 4243 Write 2 Done Done New false
8459 3245 Acked 4244 Read 9 Done Done New false
8460 3246 Acked 4245 Read 1 Done Done New false
8461 3247 Acked 4246 Write 8 Done Done New false
8462 3248 Acked 4247 Write 2 Done Done New false
8463 3249 Acked 4248 Read 7 Done Done New false
8464 3250 Acked 4249 Read 3 Done Done New false
8465 3251 Acked 4250 Write 6 Done Done New false
8466 3252 Acked 4251 Read 1 Done Done New false
8467 3253 Acked 4252 Write 3 Done Done New false
8468 3254 Acked 4253 Write 2 Done Done New false
8469 3255 Acked 4254 Write 10 Done Done New false
8470 3256 Acked 4255 Read 8 Done Done New false
8471 3257 Acked 4256 Write 9 Done Done New false
8472 3258 Acked 4257 Read 3 Done Done New false
8473 3259 Acked 4258 Read 5 Done Done New false
8474 3260 Acked 4259 Write 7 Done Done New false
8475 3261 Acked 4260 Read 9 Done Done New false
8476 3262 Acked 4261 Write 8 Done Done New false
8477 3263 Acked 4262 Read 5 Done Done New false
8478 3264 Acked 4263 Read 4 Done Done New false
8479 3265 Acked 4264 Flush 0 Done Done New false
8480 3266 Acked 4265 Write 9 Done Done New false
8481 3267 Acked 4266 Read 9 Done Done New false
8482 3268 Acked 4267 Read 1 Done Done New false
8483 3269 Acked 4268 Write 1 Done Done New false
8484 3270 Acked 4269 Write 10 Done Done New false
8485 3271 Acked 4270 Write 5 Done Done New false
8486 3272 Acked 4271 Write 9 Done Done New false
8487 3273 Acked 4272 Read 2 Done Done New false
8488 3274 Acked 4273 Write 6 Done Done New false
8489 3275 Acked 4274 Write 8 Done Done New false
8490 3276 Acked 4275 Read 4 Done Done New false
8491 3277 Acked 4276 Read 10 Done Done New false
8492 3278 Acked 4277 Write 3 Done Done New false
8493 3279 Acked 4278 Write 7 Done Done New false
8494 3280 Acked 4279 Write 1 Done Done New false
8495 3281 Acked 4280 Read 8 Done Done New false
8496 3282 Acked 4281 Read 4 Done Done New false
8497 3283 Acked 4282 Flush 0 Done Done New false
8498 3284 Acked 4283 Write 2 Done Done New false
8499 3285 Acked 4284 Read 1 Done Done New false
8500 3286 Acked 4285 Read 4 Done Done New false
8501 3287 Acked 4286 Write 10 Done Done New false
8502 3288 Acked 4287 Read 6 Done Done New false
8503 3289 Acked 4288 Write 9 Done Done New false
8504 3290 Acked 4289 Read 10 Done Done New false
8505 3291 Acked 4290 Write 7 Done Done New false
8506 3292 Acked 4291 Read 5 Done Done New false
8507 3293 Acked 4292 Read 1 Done Done New false
8508 3294 Acked 4293 Read 10 Done Done New false
8509 3295 Acked 4294 Write 9 Done Done New false
8510 3296 Acked 4295 Write 9 Done Done New false
8511 3297 Acked 4296 Read 2 Done Done New false
8512 3298 Acked 4297 Flush 0 Done Done New false
8513 3299 Acked 4298 Write 2 Done Done New false
8514 3300 Acked 4299 Read 9 Done Done New false
8515 3301 Acked 4300 Write 5 Done Done New false
8516 3302 Acked 4301 Write 4 Done Done New false
8517 3303 Acked 4302 Read 1 Done Done New false
8518 3304 Acked 4303 Write 7 Done Done New false
8519 3305 Acked 4304 Write 9 Done Done New false
8520 3306 Acked 4305 Write 5 Done Done New false
8521 3307 Acked 4306 Read 4 Done Done New false
8522 3308 Acked 4307 Flush 0 Done Done New false
8523 3309 Acked 4308 Write 7 Done Done New false
8524 3310 Acked 4309 Write 3 Done Done New false
8525 3311 Acked 4310 Read 8 Done Done New false
8526 3312 Acked 4311 Read 2 Done Done New false
8527 3313 Acked 4312 Read 7 Done Done New false
8528 3314 Acked 4313 Read 8 Done Done New false
8529 3315 Acked 4314 Read 4 Done Done New false
8530 3316 Acked 4315 Read 8 Done Done New false
8531 3317 Acked 4316 Read 10 Done Done New false
8532 3318 Acked 4317 Read 6 Done Done New false
8533 3319 Acked 4318 Read 8 Done Done New false
8534 3320 Acked 4319 Write 7 Done Done New false
8535 3321 Acked 4320 Read 1 Done Done New false
8536 3322 Acked 4321 Write 3 Done Done New false
8537 3323 Acked 4322 Read 10 Done Done New false
8538 3324 Acked 4323 Read 2 Done Done New false
8539 3325 Acked 4324 Read 1 Done Done New false
8540 3326 Acked 4325 Flush 0 Done Done New false
8541 STATES DS:0 DS:1 DS:2 TOTAL
8542 New 0 0 303 303
8543 Sent 0 0 0 0
8544 Done 303 303 0 606
8545 Skipped 0 0 0 0
8546 Error 0 0 0 0
8547 Last Flush: 4325 4325 4022
8548 Downstairs last five completed: 4022 4021 4020 4019 4018 4017
8549 Upstairs last five completed: 3326 3325 3324 3323 3322 3321
8550 CLIENT: Up:0 ds:303 act:2
85512023-09-22T23:25:39.571ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
8552 ----------------------------------------------------------------
8553 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
8554 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
8555 3024 Acked 4023 Write 3 Done Done New false
8556 3025 Acked 4024 Read 3 Done Done New false
8557 3026 Acked 4025 Read 6 Done Done New false
8558 3027 Acked 4026 Read 1 Done Done New false
8559 3028 Acked 4027 Read 8 Done Done New false
8560 3029 Acked 4028 Read 6 Done Done New false
8561 3030 Acked 4029 Read 9 Done Done New false
8562 3031 Acked 4030 Read 4 Done Done New false
8563 3032 Acked 4031 Write 8 Done Done New false
8564 3033 Acked 4032 Write 4 Done Done New false
8565 3034 Acked 4033 Flush 0 Done Done New false
8566 3035 Acked 4034 Read 8 Done Done New false
8567 3036 Acked 4035 Write 3 Done Done New false
8568 3037 Acked 4036 Write 1 Done Done New false
8569 3038 Acked 4037 Read 3 Done Done New false
8570 3039 Acked 4038 Flush 0 Done Done New false
8571 3040 Acked 4039 Write 9 Done Done New false
8572 3041 Acked 4040 Write 8 Done Done New false
8573 3042 Acked 4041 Read 4 Done Done New false
8574 3043 Acked 4042 Write 8 Done Done New false
8575 3044 Acked 4043 Read 8 Done Done New false
8576 3045 Acked 4044 Write 9 Done Done New false
8577 3046 Acked 4045 Write 2 Done Done New false
8578 3047 Acked 4046 Read 2 Done Done New false
8579 3048 Acked 4047 Write 10 Done Done New false
8580 3049 Acked 4048 Read 5 Done Done New false
8581 3050 Acked 4049 Write 2 Done Done New false
8582 3051 Acked 4050 Write 2 Done Done New false
8583 3052 Acked 4051 Write 4 Done Done New false
8584 3053 Acked 4052 Flush 0 Done Done New false
8585 3054 Acked 4053 Write 8 Done Done New false
8586 3055 Acked 4054 Write 5 Done Done New false
8587 3056 Acked 4055 Read 4 Done Done New false
8588 3057 Acked 4056 Write 3 Done Done New false
8589 3058 Acked 4057 Read 8 Done Done New false
8590 3059 Acked 4058 Write 8 Done Done New false
8591 3060 Acked 4059 Read 1 Done Done New false
8592 3061 Acked 4060 Read 3 Done Done New false
8593 3062 Acked 4061 Read 7 Done Done New false
8594 3063 Acked 4062 Read 1 Done Done New false
8595 3064 Acked 4063 Read 10 Done Done New false
8596 3065 Acked 4064 Read 7 Done Done New false
8597 3066 Acked 4065 Flush 0 Done Done New false
8598 3067 Acked 4066 Flush 0 Done Done New false
8599 3068 Acked 4067 Read 5 Done Done New false
8600 3069 Acked 4068 Read 3 Done Done New false
8601 3070 Acked 4069 Read 1 Done Done New false
8602 3071 Acked 4070 Read 7 Done Done New false
8603 3072 Acked 4071 Read 6 Done Done New false
8604 3073 Acked 4072 Read 6 Done Done New false
8605 3074 Acked 4073 Flush 0 Done Done New false
8606 3075 Acked 4074 Write 10 Done Done New false
8607 3076 Acked 4075 Read 2 Done Done New false
8608 3077 Acked 4076 Read 6 Done Done New false
8609 3078 Acked 4077 Write 6 Done Done New false
8610 3079 Acked 4078 Read 9 Done Done New false
8611 3080 Acked 4079 Read 5 Done Done New false
8612 3081 Acked 4080 Read 4 Done Done New false
8613 3082 Acked 4081 Write 5 Done Done New false
8614 3083 Acked 4082 Read 9 Done Done New false
8615 3084 Acked 4083 Read 1 Done Done New false
8616 3085 Acked 4084 Read 6 Done Done New false
8617 3086 Acked 4085 Read 4 Done Done New false
8618 3087 Acked 4086 Write 5 Done Done New false
8619 3088 Acked 4087 Read 1 Done Done New false
8620 3089 Acked 4088 Read 7 Done Done New false
8621 3090 Acked 4089 Read 6 Done Done New false
8622 3091 Acked 4090 Read 1 Done Done New false
8623 3092 Acked 4091 Read 1 Done Done New false
8624 3093 Acked 4092 Write 6 Done Done New false
8625 3094 Acked 4093 Write 3 Done Done New false
8626 3095 Acked 4094 Write 1 Done Done New false
8627 3096 Acked 4095 Read 5 Done Done New false
8628 3097 Acked 4096 Read 6 Done Done New false
8629 3098 Acked 4097 Write 4 Done Done New false
8630 3099 Acked 4098 Flush 0 Done Done New false
8631 3100 Acked 4099 Read 3 Done Done New false
8632 3101 Acked 4100 Write 9 Done Done New false
8633 3102 Acked 4101 Write 2 Done Done New false
8634 3103 Acked 4102 Read 3 Done Done New false
8635 3104 Acked 4103 Read 9 Done Done New false
8636 3105 Acked 4104 Write 5 Done Done New false
8637 3106 Acked 4105 Read 9 Done Done New false
8638 3107 Acked 4106 Write 7 Done Done New false
8639 3108 Acked 4107 Read 6 Done Done New false
8640 3109 Acked 4108 Read 2 Done Done New false
8641 3110 Acked 4109 Flush 0 Done Done New false
8642 3111 Acked 4110 Write 7 Done Done New false
8643 3112 Acked 4111 Write 7 Done Done New false
8644 3113 Acked 4112 Read 5 Done Done New false
8645 3114 Acked 4113 Read 6 Done Done New false
8646 3115 Acked 4114 Read 8 Done Done New false
8647 3116 Acked 4115 Write 1 Done Done New false
8648 3117 Acked 4116 Read 2 Done Done New false
8649 3118 Acked 4117 Read 7 Done Done New false
8650 3119 Acked 4118 Read 9 Done Done New false
8651 3120 Acked 4119 Read 7 Done Done New false
8652 3121 Acked 4120 Read 8 Done Done New false
8653 3122 Acked 4121 Write 8 Done Done New false
8654 3123 Acked 4122 Write 9 Done Done New false
8655 3124 Acked 4123 Write 9 Done Done New false
8656 3125 Acked 4124 Flush 0 Done Done New false
8657 3126 Acked 4125 Write 4 Done Done New false
8658 3127 Acked 4126 Flush 0 Done Done New false
8659 3128 Acked 4127 Read 6 Done Done New false
8660 3129 Acked 4128 Write 4 Done Done New false
8661 3130 Acked 4129 Write 4 Done Done New false
8662 3131 Acked 4130 Read 6 Done Done New false
8663 3132 Acked 4131 Flush 0 Done Done New false
8664 3133 Acked 4132 Read 7 Done Done New false
8665 3134 Acked 4133 Write 10 Done Done New false
8666 3135 Acked 4134 Read 5 Done Done New false
8667 3136 Acked 4135 Flush 0 Done Done New false
8668 3137 Acked 4136 Flush 0 Done Done New false
8669 3138 Acked 4137 Read 9 Done Done New false
8670 3139 Acked 4138 Read 9 Done Done New false
8671 3140 Acked 4139 Write 6 Done Done New false
8672 3141 Acked 4140 Write 4 Done Done New false
8673 3142 Acked 4141 Write 2 Done Done New false
8674 3143 Acked 4142 Write 1 Done Done New false
8675 3144 Acked 4143 Flush 0 Done Done New false
8676 3145 Acked 4144 Read 6 Done Done New false
8677 3146 Acked 4145 Read 2 Done Done New false
8678 3147 Acked 4146 Write 7 Done Done New false
8679 3148 Acked 4147 Write 1 Done Done New false
8680 3149 Acked 4148 Read 5 Done Done New false
8681 3150 Acked 4149 Read 7 Done Done New false
8682 3151 Acked 4150 Read 9 Done Done New false
8683 3152 Acked 4151 Write 8 Done Done New false
8684 3153 Acked 4152 Flush 0 Done Done New false
8685 3154 Acked 4153 Write 3 Done Done New false
8686 3155 Acked 4154 Read 2 Done Done New false
8687 3156 Acked 4155 Write 6 Done Done New false
8688 3157 Acked 4156 Read 4 Done Done New false
8689 3158 Acked 4157 Read 4 Done Done New false
8690 3159 Acked 4158 Write 4 Done Done New false
8691 3160 Acked 4159 Read 4 Done Done New false
8692 3161 Acked 4160 Write 1 Done Done New false
8693 3162 Acked 4161 Write 1 Done Done New false
8694 3163 Acked 4162 Write 9 Done Done New false
8695 3164 Acked 4163 Flush 0 Done Done New false
8696 3165 Acked 4164 Write 7 Done Done New false
8697 3166 Acked 4165 Read 8 Done Done New false
8698 3167 Acked 4166 Read 9 Done Done New false
8699 3168 Acked 4167 Flush 0 Done Done New false
8700 3169 Acked 4168 Read 7 Done Done New false
8701 3170 Acked 4169 Read 6 Done Done New false
8702 3171 Acked 4170 Write 3 Done Done New false
8703 3172 Acked 4171 Write 4 Done Done New false
8704 3173 Acked 4172 Write 4 Done Done New false
8705 3174 Acked 4173 Write 10 Done Done New false
8706 3175 Acked 4174 Read 9 Done Done New false
8707 3176 Acked 4175 Read 8 Done Done New false
8708 3177 Acked 4176 Write 10 Done Done New false
8709 3178 Acked 4177 Write 10 Done Done New false
8710 3179 Acked 4178 Read 6 Done Done New false
8711 3180 Acked 4179 Flush 0 Done Done New false
8712 3181 Acked 4180 Flush 0 Done Done New false
8713 3182 Acked 4181 Write 6 Done Done New false
8714 3183 Acked 4182 Read 8 Done Done New false
8715 3184 Acked 4183 Write 4 Done Done New false
8716 3185 Acked 4184 Read 6 Done Done New false
8717 3186 Acked 4185 Read 6 Done Done New false
8718 3187 Acked 4186 Read 5 Done Done New false
8719 3188 Acked 4187 Read 4 Done Done New false
8720 3189 Acked 4188 Write 8 Done Done New false
8721 3190 Acked 4189 Write 7 Done Done New false
8722 3191 Acked 4190 Write 9 Done Done New false
8723 3192 Acked 4191 Read 4 Done Done New false
8724 3193 Acked 4192 Read 9 Done Done New false
8725 3194 Acked 4193 Flush 0 Done Done New false
8726 3195 Acked 4194 Read 7 Done Done New false
8727 3196 Acked 4195 Flush 0 Done Done New false
8728 3197 Acked 4196 Read 1 Done Done New false
8729 3198 Acked 4197 Write 1 Done Done New false
8730 3199 Acked 4198 Flush 0 Done Done New false
8731 3200 Acked 4199 Read 9 Done Done New false
8732 3201 Acked 4200 Read 9 Done Done New false
8733 3202 Acked 4201 Write 4 Done Done New false
8734 3203 Acked 4202 Read 10 Done Done New false
8735 3204 Acked 4203 Read 1 Done Done New false
8736 3205 Acked 4204 Write 1 Done Done New false
8737 3206 Acked 4205 Read 2 Done Done New false
8738 3207 Acked 4206 Write 1 Done Done New false
8739 3208 Acked 4207 Flush 0 Done Done New false
8740 3209 Acked 4208 Read 7 Done Done New false
8741 3210 Acked 4209 Read 9 Done Done New false
8742 3211 Acked 4210 Write 2 Done Done New false
8743 3212 Acked 4211 Read 7 Done Done New false
8744 3213 Acked 4212 Flush 0 Done Done New false
8745 3214 Acked 4213 Read 5 Done Done New false
8746 3215 Acked 4214 Write 8 Done Done New false
8747 3216 Acked 4215 Write 3 Done Done New false
8748 3217 Acked 4216 Write 9 Done Done New false
8749 3218 Acked 4217 Write 7 Done Done New false
8750 3219 Acked 4218 Read 9 Done Done New false
8751 3220 Acked 4219 Write 10 Done Done New false
8752 3221 Acked 4220 Read 3 Done Done New false
8753 3222 Acked 4221 Read 9 Done Done New false
8754 3223 Acked 4222 Read 10 Done Done New false
8755 3224 Acked 4223 Read 7 Done Done New false
8756 3225 Acked 4224 Read 8 Done Done New false
8757 3226 Acked 4225 Flush 0 Done Done New false
8758 3227 Acked 4226 Read 2 Done Done New false
8759 3228 Acked 4227 Read 7 Done Done New false
8760 3229 Acked 4228 Flush 0 Done Done New false
8761 3230 Acked 4229 Read 4 Done Done New false
8762 3231 Acked 4230 Read 9 Done Done New false
8763 3232 Acked 4231 Write 7 Done Done New false
8764 3233 Acked 4232 Read 6 Done Done New false
8765 3234 Acked 4233 Read 5 Done Done New false
8766 3235 Acked 4234 Read 3 Done Done New false
8767 3236 Acked 4235 Read 1 Done Done New false
8768 3237 Acked 4236 Flush 0 Done Done New false
8769 3238 Acked 4237 Write 7 Done Done New false
8770 3239 Acked 4238 Write 9 Done Done New false
8771 3240 Acked 4239 Flush 0 Done Done New false
8772 3241 Acked 4240 Read 10 Done Done New false
8773 3242 Acked 4241 Write 10 Done Done New false
8774 3243 Acked 4242 Read 8 Done Done New false
8775 3244 Acked 4243 Write 2 Done Done New false
8776 3245 Acked 4244 Read 9 Done Done New false
8777 3246 Acked 4245 Read 1 Done Done New false
8778 3247 Acked 4246 Write 8 Done Done New false
8779 3248 Acked 4247 Write 2 Done Done New false
8780 3249 Acked 4248 Read 7 Done Done New false
8781 3250 Acked 4249 Read 3 Done Done New false
8782 3251 Acked 4250 Write 6 Done Done New false
8783 3252 Acked 4251 Read 1 Done Done New false
8784 3253 Acked 4252 Write 3 Done Done New false
8785 3254 Acked 4253 Write 2 Done Done New false
8786 3255 Acked 4254 Write 10 Done Done New false
8787 3256 Acked 4255 Read 8 Done Done New false
8788 3257 Acked 4256 Write 9 Done Done New false
8789 3258 Acked 4257 Read 3 Done Done New false
8790 3259 Acked 4258 Read 5 Done Done New false
8791 3260 Acked 4259 Write 7 Done Done New false
8792 3261 Acked 4260 Read 9 Done Done New false
8793 3262 Acked 4261 Write 8 Done Done New false
8794 3263 Acked 4262 Read 5 Done Done New false
8795 3264 Acked 4263 Read 4 Done Done New false
8796 3265 Acked 4264 Flush 0 Done Done New false
8797 3266 Acked 4265 Write 9 Done Done New false
8798 3267 Acked 4266 Read 9 Done Done New false
8799 3268 Acked 4267 Read 1 Done Done New false
8800 3269 Acked 4268 Write 1 Done Done New false
8801 3270 Acked 4269 Write 10 Done Done New false
8802 3271 Acked 4270 Write 5 Done Done New false
8803 3272 Acked 4271 Write 9 Done Done New false
8804 3273 Acked 4272 Read 2 Done Done New false
8805 3274 Acked 4273 Write 6 Done Done New false
8806 3275 Acked 4274 Write 8 Done Done New false
8807 3276 Acked 4275 Read 4 Done Done New false
8808 3277 Acked 4276 Read 10 Done Done New false
8809 3278 Acked 4277 Write 3 Done Done New false
8810 3279 Acked 4278 Write 7 Done Done New false
8811 3280 Acked 4279 Write 1 Done Done New false
8812 3281 Acked 4280 Read 8 Done Done New false
8813 3282 Acked 4281 Read 4 Done Done New false
8814 3283 Acked 4282 Flush 0 Done Done New false
8815 3284 Acked 4283 Write 2 Done Done New false
8816 3285 Acked 4284 Read 1 Done Done New false
8817 3286 Acked 4285 Read 4 Done Done New false
8818 3287 Acked 4286 Write 10 Done Done New false
8819 3288 Acked 4287 Read 6 Done Done New false
8820 3289 Acked 4288 Write 9 Done Done New false
8821 3290 Acked 4289 Read 10 Done Done New false
8822 3291 Acked 4290 Write 7 Done Done New false
8823 3292 Acked 4291 Read 5 Done Done New false
8824 3293 Acked 4292 Read 1 Done Done New false
8825 3294 Acked 4293 Read 10 Done Done New false
8826 3295 Acked 4294 Write 9 Done Done New false
8827 3296 Acked 4295 Write 9 Done Done New false
8828 3297 Acked 4296 Read 2 Done Done New false
8829 3298 Acked 4297 Flush 0 Done Done New false
8830 3299 Acked 4298 Write 2 Done Done New false
8831 3300 Acked 4299 Read 9 Done Done New false
8832 3301 Acked 4300 Write 5 Done Done New false
8833 3302 Acked 4301 Write 4 Done Done New false
8834 3303 Acked 4302 Read 1 Done Done New false
8835 3304 Acked 4303 Write 7 Done Done New false
8836 3305 Acked 4304 Write 9 Done Done New false
8837 3306 Acked 4305 Write 5 Done Done New false
8838 3307 Acked 4306 Read 4 Done Done New false
8839 3308 Acked 4307 Flush 0 Done Done New false
8840 3309 Acked 4308 Write 7 Done Done New false
8841 3310 Acked 4309 Write 3 Done Done New false
8842 3311 Acked 4310 Read 8 Done Done New false
8843 3312 Acked 4311 Read 2 Done Done New false
8844 3313 Acked 4312 Read 7 Done Done New false
8845 3314 Acked 4313 Read 8 Done Done New false
8846 3315 Acked 4314 Read 4 Done Done New false
8847 3316 Acked 4315 Read 8 Done Done New false
8848 3317 Acked 4316 Read 10 Done Done New false
8849 3318 Acked 4317 Read 6 Done Done New false
8850 3319 Acked 4318 Read 8 Done Done New false
8851 3320 Acked 4319 Write 7 Done Done New false
8852 3321 Acked 4320 Read 1 Done Done New false
8853 3322 Acked 4321 Write 3 Done Done New false
8854 3323 Acked 4322 Read 10 Done Done New false
8855 3324 Acked 4323 Read 2 Done Done New false
8856 3325 Acked 4324 Read 1 Done Done New false
8857 3326 Acked 4325 Flush 0 Done Done New false
8858 STATES DS:0 DS:1 DS:2 TOTAL
8859 New 0 0 303 303
8860 Sent 0 0 0 0
8861 Done 303 303 0 606
8862 Skipped 0 0 0 0
8863 Error 0 0 0 0
8864 Last Flush: 4325 4325 4022
8865 Downstairs last five completed: 4022 4021 4020 4019 4018 4017
8866 Upstairs last five completed: 3326 3325 3324 3323 3322 3321
8867 CLIENT: Up:0 ds:303 act:2
8868 ----------------------------------------------------------------
8869 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
8870 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
8871 3024 Acked 4023 Write 3 Done Done New false
8872 3025 Acked 4024 Read 3 Done Done New false
8873 3026 Acked 4025 Read 6 Done Done New false
8874 3027 Acked 4026 Read 1 Done Done New false
8875 3028 Acked 4027 Read 8 Done Done New false
8876 3029 Acked 4028 Read 6 Done Done New false
8877 3030 Acked 4029 Read 9 Done Done New false
8878 3031 Acked 4030 Read 4 Done Done New false
8879 3032 Acked 4031 Write 8 Done Done New false
8880 3033 Acked 4032 Write 4 Done Done New false
8881 3034 Acked 4033 Flush 0 Done Done New false
8882 3035 Acked 4034 Read 8 Done Done New false
8883 3036 Acked 4035 Write 3 Done Done New false
8884 3037 Acked 4036 Write 1 Done Done New false
8885 3038 Acked 4037 Read 3 Done Done New false
8886 3039 Acked 4038 Flush 0 Done Done New false
8887 3040 Acked 4039 Write 9 Done Done New false
8888 3041 Acked 4040 Write 8 Done Done New false
8889 3042 Acked 4041 Read 4 Done Done New false
8890 3043 Acked 4042 Write 8 Done Done New false
8891 3044 Acked 4043 Read 8 Done Done New false
8892 3045 Acked 4044 Write 9 Done Done New false
8893 3046 Acked 4045 Write 2 Done Done New false
8894 3047 Acked 4046 Read 2 Done Done New false
8895 3048 Acked 4047 Write 10 Done Done New false
8896 3049 Acked 4048 Read 5 Done Done New false
8897 3050 Acked 4049 Write 2 Done Done New false
8898 3051 Acked 4050 Write 2 Done Done New false
8899 3052 Acked 4051 Write 4 Done Done New false
8900 3053 Acked 4052 Flush 0 Done Done New false
8901 3054 Acked 4053 Write 8 Done Done New false
8902 3055 Acked 4054 Write 5 Done Done New false
8903 3056 Acked 4055 Read 4 Done Done New false
8904 3057 Acked 4056 Write 3 Done Done New false
8905 3058 Acked 4057 Read 8 Done Done New false
8906 3059 Acked 4058 Write 8 Done Done New false
8907 3060 Acked 4059 Read 1 Done Done New false
8908 3061 Acked 4060 Read 3 Done Done New false
8909 3062 Acked 4061 Read 7 Done Done New false
8910 3063 Acked 4062 Read 1 Done Done New false
8911 3064 Acked 4063 Read 10 Done Done New false
8912 3065 Acked 4064 Read 7 Done Done New false
8913 3066 Acked 4065 Flush 0 Done Done New false
8914 3067 Acked 4066 Flush 0 Done Done New false
8915 3068 Acked 4067 Read 5 Done Done New false
8916 3069 Acked 4068 Read 3 Done Done New false
8917 3070 Acked 4069 Read 1 Done Done New false
8918 3071 Acked 4070 Read 7 Done Done New false
8919 3072 Acked 4071 Read 6 Done Done New false
8920 3073 Acked 4072 Read 6 Done Done New false
8921 3074 Acked 4073 Flush 0 Done Done New false
8922 3075 Acked 4074 Write 10 Done Done New false
8923 3076 Acked 4075 Read 2 Done Done New false
8924 3077 Acked 4076 Read 6 Done Done New false
8925 3078 Acked 4077 Write 6 Done Done New false
8926 3079 Acked 4078 Read 9 Done Done New false
8927 3080 Acked 4079 Read 5 Done Done New false
8928 3081 Acked 4080 Read 4 Done Done New false
8929 3082 Acked 4081 Write 5 Done Done New false
8930 3083 Acked 4082 Read 9 Done Done New false
8931 3084 Acked 4083 Read 1 Done Done New false
8932 3085 Acked 4084 Read 6 Done Done New false
8933 3086 Acked 4085 Read 4 Done Done New false
8934 3087 Acked 4086 Write 5 Done Done New false
8935 3088 Acked 4087 Read 1 Done Done New false
8936 3089 Acked 4088 Read 7 Done Done New false
8937 3090 Acked 4089 Read 6 Done Done New false
8938 3091 Acked 4090 Read 1 Done Done New false
8939 3092 Acked 4091 Read 1 Done Done New false
8940 3093 Acked 4092 Write 6 Done Done New false
8941 3094 Acked 4093 Write 3 Done Done New false
8942 3095 Acked 4094 Write 1 Done Done New false
8943 3096 Acked 4095 Read 5 Done Done New false
8944 3097 Acked 4096 Read 6 Done Done New false
8945 3098 Acked 4097 Write 4 Done Done New false
8946 3099 Acked 4098 Flush 0 Done Done New false
8947 3100 Acked 4099 Read 3 Done Done New false
8948 3101 Acked 4100 Write 9 Done Done New false
8949 3102 Acked 4101 Write 2 Done Done New false
8950 3103 Acked 4102 Read 3 Done Done New false
8951 3104 Acked 4103 Read 9 Done Done New false
8952 3105 Acked 4104 Write 5 Done Done New false
8953 3106 Acked 4105 Read 9 Done Done New false
8954 3107 Acked 4106 Write 7 Done Done New false
8955 3108 Acked 4107 Read 6 Done Done New false
8956 3109 Acked 4108 Read 2 Done Done New false
8957 3110 Acked 4109 Flush 0 Done Done New false
8958 3111 Acked 4110 Write 7 Done Done New false
8959 3112 Acked 4111 Write 7 Done Done New false
8960 3113 Acked 4112 Read 5 Done Done New false
8961 3114 Acked 4113 Read 6 Done Done New false
8962 3115 Acked 4114 Read 8 Done Done New false
8963 3116 Acked 4115 Write 1 Done Done New false
8964 3117 Acked 4116 Read 2 Done Done New false
8965 3118 Acked 4117 Read 7 Done Done New false
8966 3119 Acked 4118 Read 9 Done Done New false
8967 3120 Acked 4119 Read 7 Done Done New false
8968 3121 Acked 4120 Read 8 Done Done New false
8969 3122 Acked 4121 Write 8 Done Done New false
8970 3123 Acked 4122 Write 9 Done Done New false
8971 3124 Acked 4123 Write 9 Done Done New false
8972 3125 Acked 4124 Flush 0 Done Done New false
8973 3126 Acked 4125 Write 4 Done Done New false
8974 3127 Acked 4126 Flush 0 Done Done New false
8975 3128 Acked 4127 Read 6 Done Done New false
8976 3129 Acked 4128 Write 4 Done Done New false
8977 3130 Acked 4129 Write 4 Done Done New false
8978 3131 Acked 4130 Read 6 Done Done New false
8979 3132 Acked 4131 Flush 0 Done Done New false
8980 3133 Acked 4132 Read 7 Done Done New false
8981 3134 Acked 4133 Write 10 Done Done New false
8982 3135 Acked 4134 Read 5 Done Done New false
8983 3136 Acked 4135 Flush 0 Done Done New false
8984 3137 Acked 4136 Flush 0 Done Done New false
8985 3138 Acked 4137 Read 9 Done Done New false
8986 3139 Acked 4138 Read 9 Done Done New false
8987 3140 Acked 4139 Write 6 Done Done New false
8988 3141 Acked 4140 Write 4 Done Done New false
8989 3142 Acked 4141 Write 2 Done Done New false
8990 3143 Acked 4142 Write 1 Done Done New false
8991 3144 Acked 4143 Flush 0 Done Done New false
8992 3145 Acked 4144 Read 6 Done Done New false
8993 3146 Acked 4145 Read 2 Done Done New false
8994 3147 Acked 4146 Write 7 Done Done New false
8995 3148 Acked 4147 Write 1 Done Done New false
8996 3149 Acked 4148 Read 5 Done Done New false
8997 3150 Acked 4149 Read 7 Done Done New false
8998 3151 Acked 4150 Read 9 Done Done New false
8999 3152 Acked 4151 Write 8 Done Done New false
9000 3153 Acked 4152 Flush 0 Done Done New false
9001 3154 Acked 4153 Write 3 Done Done New false
9002 3155 Acked 4154 Read 2 Done Done New false
9003 3156 Acked 4155 Write 6 Done Done New false
9004 3157 Acked 4156 Read 4 Done Done New false
9005 3158 Acked 4157 Read 4 Done Done New false
9006 3159 Acked 4158 Write 4 Done Done New false
9007 3160 Acked 4159 Read 4 Done Done New false
9008 3161 Acked 4160 Write 1 Done Done New false
9009 3162 Acked 4161 Write 1 Done Done New false
9010 3163 Acked 4162 Write 9 Done Done New false
9011 3164 Acked 4163 Flush 0 Done Done New false
9012 3165 Acked 4164 Write 7 Done Done New false
9013 3166 Acked 4165 Read 8 Done Done New false
9014 3167 Acked 4166 Read 9 Done Done New false
9015 3168 Acked 4167 Flush 0 Done Done New false
9016 3169 Acked 4168 Read 7 Done Done New false
9017 3170 Acked 4169 Read 6 Done Done New false
9018 3171 Acked 4170 Write 3 Done Done New false
9019 3172 Acked 4171 Write 4 Done Done New false
9020 3173 Acked 4172 Write 4 Done Done New false
9021 3174 Acked 4173 Write 10 Done Done New false
9022 3175 Acked 4174 Read 9 Done Done New false
9023 3176 Acked 4175 Read 8 Done Done New false
9024 3177 Acked 4176 Write 10 Done Done New false
9025 3178 Acked 4177 Write 10 Done Done New false
9026 3179 Acked 4178 Read 6 Done Done New false
9027 3180 Acked 4179 Flush 0 Done Done New false
9028 3181 Acked 4180 Flush 0 Done Done New false
9029 3182 Acked 4181 Write 6 Done Done New false
9030 3183 Acked 4182 Read 8 Done Done New false
9031 3184 Acked 4183 Write 4 Done Done New false
9032 3185 Acked 4184 Read 6 Done Done New false
9033 3186 Acked 4185 Read 6 Done Done New false
9034 3187 Acked 4186 Read 5 Done Done New false
9035 3188 Acked 4187 Read 4 Done Done New false
9036 3189 Acked 4188 Write 8 Done Done New false
9037 3190 Acked 4189 Write 7 Done Done New false
9038 3191 Acked 4190 Write 9 Done Done New false
9039 3192 Acked 4191 Read 4 Done Done New false
9040 3193 Acked 4192 Read 9 Done Done New false
9041 3194 Acked 4193 Flush 0 Done Done New false
9042 3195 Acked 4194 Read 7 Done Done New false
9043 3196 Acked 4195 Flush 0 Done Done New false
9044 3197 Acked 4196 Read 1 Done Done New false
9045 3198 Acked 4197 Write 1 Done Done New false
9046 3199 Acked 4198 Flush 0 Done Done New false
9047 3200 Acked 4199 Read 9 Done Done New false
9048 3201 Acked 4200 Read 9 Done Done New false
9049 3202 Acked 4201 Write 4 Done Done New false
9050 3203 Acked 4202 Read 10 Done Done New false
9051 3204 Acked 4203 Read 1 Done Done New false
9052 3205 Acked 4204 Write 1 Done Done New false
9053 3206 Acked 4205 Read 2 Done Done New false
9054 3207 Acked 4206 Write 1 Done Done New false
9055 3208 Acked 4207 Flush 0 Done Done New false
9056 3209 Acked 4208 Read 7 Done Done New false
9057 3210 Acked 4209 Read 9 Done Done New false
9058 3211 Acked 4210 Write 2 Done Done New false
9059 3212 Acked 4211 Read 7 Done Done New false
9060 3213 Acked 4212 Flush 0 Done Done New false
9061 3214 Acked 4213 Read 5 Done Done New false
9062 3215 Acked 4214 Write 8 Done Done New false
9063 3216 Acked 4215 Write 3 Done Done New false
9064 3217 Acked 4216 Write 9 Done Done New false
9065 3218 Acked 4217 Write 7 Done Done New false
9066 3219 Acked 4218 Read 9 Done Done New false
9067 3220 Acked 4219 Write 10 Done Done New false
9068 3221 Acked 4220 Read 3 Done Done New false
9069 3222 Acked 4221 Read 9 Done Done New false
9070 3223 Acked 4222 Read 10 Done Done New false
9071 3224 Acked 4223 Read 7 Done Done New false
9072 3225 Acked 4224 Read 8 Done Done New false
9073 3226 Acked 4225 Flush 0 Done Done New false
9074 3227 Acked 4226 Read 2 Done Done New false
9075 3228 Acked 4227 Read 7 Done Done New false
9076 3229 Acked 4228 Flush 0 Done Done New false
9077 3230 Acked 4229 Read 4 Done Done New false
9078 3231 Acked 4230 Read 9 Done Done New false
9079 3232 Acked 4231 Write 7 Done Done New false
9080 3233 Acked 4232 Read 6 Done Done New false
9081 3234 Acked 4233 Read 5 Done Done New false
9082 3235 Acked 4234 Read 3 Done Done New false
9083 3236 Acked 4235 Read 1 Done Done New false
9084 3237 Acked 4236 Flush 0 Done Done New false
9085 3238 Acked 4237 Write 7 Done Done New false
9086 3239 Acked 4238 Write 9 Done Done New false
9087 3240 Acked 4239 Flush 0 Done Done New false
9088 3241 Acked 4240 Read 10 Done Done New false
9089 3242 Acked 4241 Write 10 Done Done New false
9090 3243 Acked 4242 Read 8 Done Done New false
9091 3244 Acked 4243 Write 2 Done Done New false
9092 3245 Acked 4244 Read 9 Done Done New false
9093 3246 Acked 4245 Read 1 Done Done New false
9094 3247 Acked 4246 Write 8 Done Done New false
9095 3248 Acked 4247 Write 2 Done Done New false
9096 3249 Acked 4248 Read 7 Done Done New false
9097 3250 Acked 4249 Read 3 Done Done New false
9098 3251 Acked 4250 Write 6 Done Done New false
9099 3252 Acked 4251 Read 1 Done Done New false
9100 3253 Acked 4252 Write 3 Done Done New false
9101 3254 Acked 4253 Write 2 Done Done New false
9102 3255 Acked 4254 Write 10 Done Done New false
9103 3256 Acked 4255 Read 8 Done Done New false
9104 3257 Acked 4256 Write 9 Done Done New false
9105 3258 Acked 4257 Read 3 Done Done New false
9106 3259 Acked 4258 Read 5 Done Done New false
9107 3260 Acked 4259 Write 7 Done Done New false
9108 3261 Acked 4260 Read 9 Done Done New false
9109 3262 Acked 4261 Write 8 Done Done New false
9110 3263 Acked 4262 Read 5 Done Done New false
9111 3264 Acked 4263 Read 4 Done Done New false
9112 3265 Acked 4264 Flush 0 Done Done New false
9113 3266 Acked 4265 Write 9 Done Done New false
9114 3267 Acked 4266 Read 9 Done Done New false
9115 3268 Acked 4267 Read 1 Done Done New false
9116 3269 Acked 4268 Write 1 Done Done New false
9117 3270 Acked 4269 Write 10 Done Done New false
9118 3271 Acked 4270 Write 5 Done Done New false
9119 3272 Acked 4271 Write 9 Done Done New false
9120 3273 Acked 4272 Read 2 Done Done New false
9121 3274 Acked 4273 Write 6 Done Done New false
9122 3275 Acked 4274 Write 8 Done Done New false
9123 3276 Acked 4275 Read 4 Done Done New false
9124 3277 Acked 4276 Read 10 Done Done New false
9125 3278 Acked 4277 Write 3 Done Done New false
9126 3279 Acked 4278 Write 7 Done Done New false
9127 3280 Acked 4279 Write 1 Done Done New false
9128 3281 Acked 4280 Read 8 Done Done New false
9129 3282 Acked 4281 Read 4 Done Done New false
9130 3283 Acked 4282 Flush 0 Done Done New false
9131 3284 Acked 4283 Write 2 Done Done New false
9132 3285 Acked 4284 Read 1 Done Done New false
9133 3286 Acked 4285 Read 4 Done Done New false
9134 3287 Acked 4286 Write 10 Done Done New false
9135 3288 Acked 4287 Read 6 Done Done New false
9136 3289 Acked 4288 Write 9 Done Done New false
9137 3290 Acked 4289 Read 10 Done Done New false
9138 3291 Acked 4290 Write 7 Done Done New false
9139 3292 Acked 4291 Read 5 Done Done New false
9140 3293 Acked 4292 Read 1 Done Done New false
9141 3294 Acked 4293 Read 10 Done Done New false
9142 3295 Acked 4294 Write 9 Done Done New false
9143 3296 Acked 4295 Write 9 Done Done New false
9144 3297 Acked 4296 Read 2 Done Done New false
9145 3298 Acked 4297 Flush 0 Done Done New false
9146 3299 Acked 4298 Write 2 Done Done New false
9147 3300 Acked 4299 Read 9 Done Done New false
9148 3301 Acked 4300 Write 5 Done Done New false
9149 3302 Acked 4301 Write 4 Done Done New false
9150 3303 Acked 4302 Read 1 Done Done New false
9151 3304 Acked 4303 Write 7 Done Done New false
9152 3305 Acked 4304 Write 9 Done Done New false
9153 3306 Acked 4305 Write 5 Done Done New false
9154 3307 Acked 4306 Read 4 Done Done New false
9155 3308 Acked 4307 Flush 0 Done Done New false
9156 3309 Acked 4308 Write 7 Done Done New false
9157 3310 Acked 4309 Write 3 Done Done New false
9158 3311 Acked 4310 Read 8 Done Done New false
9159 3312 Acked 4311 Read 2 Done Done New false
9160 3313 Acked 4312 Read 7 Done Done New false
9161 3314 Acked 4313 Read 8 Done Done New false
9162 3315 Acked 4314 Read 4 Done Done New false
9163 3316 Acked 4315 Read 8 Done Done New false
9164 3317 Acked 4316 Read 10 Done Done New false
9165 3318 Acked 4317 Read 6 Done Done New false
9166 3319 Acked 4318 Read 8 Done Done New false
9167 3320 Acked 4319 Write 7 Done Done New false
9168 3321 Acked 4320 Read 1 Done Done New false
9169 3322 Acked 4321 Write 3 Done Done New false
9170 3323 Acked 4322 Read 10 Done Done New false
9171 3324 Acked 4323 Read 2 Done Done New false
9172 3325 Acked 4324 Read 1 Done Done New false
9173 3326 Acked 4325 Flush 0 Done Done New false
9174 STATES DS:0 DS:1 DS:2 TOTAL
9175 New 0 0 303 303
9176 Sent 0 0 0 0
9177 Done 303 303 0 606
9178 Skipped 0 0 0 0
9179 Error 0 0 0 0
9180 Last Flush: 4325 4325 4022
9181 Downstairs last five completed: 4022 4021 4020 4019 4018 4017
9182 Upstairs last five completed: 3326 3325 3324 3323 3322 3321
9183 CLIENT: Up:0 ds:303 act:2
9184 ----------------------------------------------------------------
9185 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
9186 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
9187 3024 Acked 4023 Write 3 Done Done New false
9188 3025 Acked 4024 Read 3 Done Done New false
9189 3026 Acked 4025 Read 6 Done Done New false
9190 3027 Acked 4026 Read 1 Done Done New false
9191 3028 Acked 4027 Read 8 Done Done New false
9192 3029 Acked 4028 Read 6 Done Done New false
9193 3030 Acked 4029 Read 9 Done Done New false
9194 3031 Acked 4030 Read 4 Done Done New false
9195 3032 Acked 4031 Write 8 Done Done New false
9196 3033 Acked 4032 Write 4 Done Done New false
9197 3034 Acked 4033 Flush 0 Done Done New false
9198 3035 Acked 4034 Read 8 Done Done New false
9199 3036 Acked 4035 Write 3 Done Done New false
9200 3037 Acked 4036 Write 1 Done Done New false
9201 3038 Acked 4037 Read 3 Done Done New false
9202 3039 Acked 4038 Flush 0 Done Done New false
9203 3040 Acked 4039 Write 9 Done Done New false
9204 3041 Acked 4040 Write 8 Done Done New false
9205 3042 Acked 4041 Read 4 Done Done New false
9206 3043 Acked 4042 Write 8 Done Done New false
9207 3044 Acked 4043 Read 8 Done Done New false
9208 3045 Acked 4044 Write 9 Done Done New false
9209 3046 Acked 4045 Write 2 Done Done New false
9210 3047 Acked 4046 Read 2 Done Done New false
9211 3048 Acked 4047 Write 10 Done Done New false
9212 3049 Acked 4048 Read 5 Done Done New false
9213 3050 Acked 4049 Write 2 Done Done New false
9214 3051 Acked 4050 Write 2 Done Done New false
9215 3052 Acked 4051 Write 4 Done Done New false
9216 3053 Acked 4052 Flush 0 Done Done New false
9217 3054 Acked 4053 Write 8 Done Done New false
9218 3055 Acked 4054 Write 5 Done Done New false
9219 3056 Acked 4055 Read 4 Done Done New false
9220 3057 Acked 4056 Write 3 Done Done New false
9221 3058 Acked 4057 Read 8 Done Done New false
9222 3059 Acked 4058 Write 8 Done Done New false
9223 3060 Acked 4059 Read 1 Done Done New false
9224 3061 Acked 4060 Read 3 Done Done New false
9225 3062 Acked 4061 Read 7 Done Done New false
9226 3063 Acked 4062 Read 1 Done Done New false
9227 3064 Acked 4063 Read 10 Done Done New false
9228 3065 Acked 4064 Read 7 Done Done New false
9229 3066 Acked 4065 Flush 0 Done Done New false
9230 3067 Acked 4066 Flush 0 Done Done New false
9231 3068 Acked 4067 Read 5 Done Done New false
9232 3069 Acked 4068 Read 3 Done Done New false
9233 3070 Acked 4069 Read 1 Done Done New false
9234 3071 Acked 4070 Read 7 Done Done New false
9235 3072 Acked 4071 Read 6 Done Done New false
9236 3073 Acked 4072 Read 6 Done Done New false
9237 3074 Acked 4073 Flush 0 Done Done New false
9238 3075 Acked 4074 Write 10 Done Done New false
9239 3076 Acked 4075 Read 2 Done Done New false
9240 3077 Acked 4076 Read 6 Done Done New false
9241 3078 Acked 4077 Write 6 Done Done New false
9242 3079 Acked 4078 Read 9 Done Done New false
9243 3080 Acked 4079 Read 5 Done Done New false
9244 3081 Acked 4080 Read 4 Done Done New false
9245 3082 Acked 4081 Write 5 Done Done New false
9246 3083 Acked 4082 Read 9 Done Done New false
9247 3084 Acked 4083 Read 1 Done Done New false
9248 3085 Acked 4084 Read 6 Done Done New false
9249 3086 Acked 4085 Read 4 Done Done New false
9250 3087 Acked 4086 Write 5 Done Done New false
9251 3088 Acked 4087 Read 1 Done Done New false
9252 3089 Acked 4088 Read 7 Done Done New false
9253 3090 Acked 4089 Read 6 Done Done New false
9254 3091 Acked 4090 Read 1 Done Done New false
9255 3092 Acked 4091 Read 1 Done Done New false
9256 3093 Acked 4092 Write 6 Done Done New false
9257 3094 Acked 4093 Write 3 Done Done New false
9258 3095 Acked 4094 Write 1 Done Done New false
9259 3096 Acked 4095 Read 5 Done Done New false
9260 3097 Acked 4096 Read 6 Done Done New false
9261 3098 Acked 4097 Write 4 Done Done New false
9262 3099 Acked 4098 Flush 0 Done Done New false
9263 3100 Acked 4099 Read 3 Done Done New false
9264 3101 Acked 4100 Write 9 Done Done New false
9265 3102 Acked 4101 Write 2 Done Done New false
9266 3103 Acked 4102 Read 3 Done Done New false
9267 3104 Acked 4103 Read 9 Done Done New false
9268 3105 Acked 4104 Write 5 Done Done New false
9269 3106 Acked 4105 Read 9 Done Done New false
9270 3107 Acked 4106 Write 7 Done Done New false
9271 3108 Acked 4107 Read 6 Done Done New false
9272 3109 Acked 4108 Read 2 Done Done New false
9273 3110 Acked 4109 Flush 0 Done Done New false
9274 3111 Acked 4110 Write 7 Done Done New false
9275 3112 Acked 4111 Write 7 Done Done New false
9276 3113 Acked 4112 Read 5 Done Done New false
9277 3114 Acked 4113 Read 6 Done Done New false
9278 3115 Acked 4114 Read 8 Done Done New false
9279 3116 Acked 4115 Write 1 Done Done New false
9280 3117 Acked 4116 Read 2 Done Done New false
9281 3118 Acked 4117 Read 7 Done Done New false
9282 3119 Acked 4118 Read 9 Done Done New false
9283 3120 Acked 4119 Read 7 Done Done New false
9284 3121 Acked 4120 Read 8 Done Done New false
9285 3122 Acked 4121 Write 8 Done Done New false
9286 3123 Acked 4122 Write 9 Done Done New false
9287 3124 Acked 4123 Write 9 Done Done New false
9288 3125 Acked 4124 Flush 0 Done Done New false
9289 3126 Acked 4125 Write 4 Done Done New false
9290 3127 Acked 4126 Flush 0 Done Done New false
9291 3128 Acked 4127 Read 6 Done Done New false
9292 3129 Acked 4128 Write 4 Done Done New false
9293 3130 Acked 4129 Write 4 Done Done New false
9294 3131 Acked 4130 Read 6 Done Done New false
9295 3132 Acked 4131 Flush 0 Done Done New false
9296 3133 Acked 4132 Read 7 Done Done New false
9297 3134 Acked 4133 Write 10 Done Done New false
9298 3135 Acked 4134 Read 5 Done Done New false
9299 3136 Acked 4135 Flush 0 Done Done New false
9300 3137 Acked 4136 Flush 0 Done Done New false
9301 3138 Acked 4137 Read 9 Done Done New false
9302 3139 Acked 4138 Read 9 Done Done New false
9303 3140 Acked 4139 Write 6 Done Done New false
9304 3141 Acked 4140 Write 4 Done Done New false
9305 3142 Acked 4141 Write 2 Done Done New false
9306 3143 Acked 4142 Write 1 Done Done New false
9307 3144 Acked 4143 Flush 0 Done Done New false
9308 3145 Acked 4144 Read 6 Done Done New false
9309 3146 Acked 4145 Read 2 Done Done New false
9310 3147 Acked 4146 Write 7 Done Done New false
9311 3148 Acked 4147 Write 1 Done Done New false
9312 3149 Acked 4148 Read 5 Done Done New false
9313 3150 Acked 4149 Read 7 Done Done New false
9314 3151 Acked 4150 Read 9 Done Done New false
9315 3152 Acked 4151 Write 8 Done Done New false
9316 3153 Acked 4152 Flush 0 Done Done New false
9317 3154 Acked 4153 Write 3 Done Done New false
9318 3155 Acked 4154 Read 2 Done Done New false
9319 3156 Acked 4155 Write 6 Done Done New false
9320 3157 Acked 4156 Read 4 Done Done New false
9321 3158 Acked 4157 Read 4 Done Done New false
9322 3159 Acked 4158 Write 4 Done Done New false
9323 3160 Acked 4159 Read 4 Done Done New false
9324 3161 Acked 4160 Write 1 Done Done New false
9325 3162 Acked 4161 Write 1 Done Done New false
9326 3163 Acked 4162 Write 9 Done Done New false
9327 3164 Acked 4163 Flush 0 Done Done New false
9328 3165 Acked 4164 Write 7 Done Done New false
9329 3166 Acked 4165 Read 8 Done Done New false
9330 3167 Acked 4166 Read 9 Done Done New false
9331 3168 Acked 4167 Flush 0 Done Done New false
9332 3169 Acked 4168 Read 7 Done Done New false
9333 3170 Acked 4169 Read 6 Done Done New false
9334 3171 Acked 4170 Write 3 Done Done New false
9335 3172 Acked 4171 Write 4 Done Done New false
9336 3173 Acked 4172 Write 4 Done Done New false
9337 3174 Acked 4173 Write 10 Done Done New false
9338 3175 Acked 4174 Read 9 Done Done New false
9339 3176 Acked 4175 Read 8 Done Done New false
9340 3177 Acked 4176 Write 10 Done Done New false
9341 3178 Acked 4177 Write 10 Done Done New false
9342 3179 Acked 4178 Read 6 Done Done New false
9343 3180 Acked 4179 Flush 0 Done Done New false
9344 3181 Acked 4180 Flush 0 Done Done New false
9345 3182 Acked 4181 Write 6 Done Done New false
9346 3183 Acked 4182 Read 8 Done Done New false
9347 3184 Acked 4183 Write 4 Done Done New false
9348 3185 Acked 4184 Read 6 Done Done New false
9349 3186 Acked 4185 Read 6 Done Done New false
9350 3187 Acked 4186 Read 5 Done Done New false
9351 3188 Acked 4187 Read 4 Done Done New false
9352 3189 Acked 4188 Write 8 Done Done New false
9353 3190 Acked 4189 Write 7 Done Done New false
9354 3191 Acked 4190 Write 9 Done Done New false
9355 3192 Acked 4191 Read 4 Done Done New false
9356 3193 Acked 4192 Read 9 Done Done New false
9357 3194 Acked 4193 Flush 0 Done Done New false
9358 3195 Acked 4194 Read 7 Done Done New false
9359 3196 Acked 4195 Flush 0 Done Done New false
9360 3197 Acked 4196 Read 1 Done Done New false
9361 3198 Acked 4197 Write 1 Done Done New false
9362 3199 Acked 4198 Flush 0 Done Done New false
9363 3200 Acked 4199 Read 9 Done Done New false
9364 3201 Acked 4200 Read 9 Done Done New false
9365 3202 Acked 4201 Write 4 Done Done New false
9366 3203 Acked 4202 Read 10 Done Done New false
9367 3204 Acked 4203 Read 1 Done Done New false
9368 3205 Acked 4204 Write 1 Done Done New false
9369 3206 Acked 4205 Read 2 Done Done New false
9370 3207 Acked 4206 Write 1 Done Done New false
9371 3208 Acked 4207 Flush 0 Done Done New false
9372 3209 Acked 4208 Read 7 Done Done New false
9373 3210 Acked 4209 Read 9 Done Done New false
9374 3211 Acked 4210 Write 2 Done Done New false
9375 3212 Acked 4211 Read 7 Done Done New false
9376 3213 Acked 4212 Flush 0 Done Done New false
9377 3214 Acked 4213 Read 5 Done Done New false
9378 3215 Acked 4214 Write 8 Done Done New false
9379 3216 Acked 4215 Write 3 Done Done New false
9380 3217 Acked 4216 Write 9 Done Done New false
9381 3218 Acked 4217 Write 7 Done Done New false
9382 3219 Acked 4218 Read 9 Done Done New false
9383 3220 Acked 4219 Write 10 Done Done New false
9384 3221 Acked 4220 Read 3 Done Done New false
9385 3222 Acked 4221 Read 9 Done Done New false
9386 3223 Acked 4222 Read 10 Done Done New false
9387 3224 Acked 4223 Read 7 Done Done New false
9388 3225 Acked 4224 Read 8 Done Done New false
9389 3226 Acked 4225 Flush 0 Done Done New false
9390 3227 Acked 4226 Read 2 Done Done New false
9391 3228 Acked 4227 Read 7 Done Done New false
9392 3229 Acked 4228 Flush 0 Done Done New false
9393 3230 Acked 4229 Read 4 Done Done New false
9394 3231 Acked 4230 Read 9 Done Done New false
9395 3232 Acked 4231 Write 7 Done Done New false
9396 3233 Acked 4232 Read 6 Done Done New false
9397 3234 Acked 4233 Read 5 Done Done New false
9398 3235 Acked 4234 Read 3 Done Done New false
9399 3236 Acked 4235 Read 1 Done Done New false
9400 3237 Acked 4236 Flush 0 Done Done New false
9401 3238 Acked 4237 Write 7 Done Done New false
9402 3239 Acked 4238 Write 9 Done Done New false
9403 3240 Acked 4239 Flush 0 Done Done New false
9404 3241 Acked 4240 Read 10 Done Done New false
9405 3242 Acked 4241 Write 10 Done Done New false
9406 3243 Acked 4242 Read 8 Done Done New false
9407 3244 Acked 4243 Write 2 Done Done New false
9408 3245 Acked 4244 Read 9 Done Done New false
9409 3246 Acked 4245 Read 1 Done Done New false
9410 3247 Acked 4246 Write 8 Done Done New false
9411 3248 Acked 4247 Write 2 Done Done New false
9412 3249 Acked 4248 Read 7 Done Done New false
9413 3250 Acked 4249 Read 3 Done Done New false
9414 3251 Acked 4250 Write 6 Done Done New false
9415 3252 Acked 4251 Read 1 Done Done New false
9416 3253 Acked 4252 Write 3 Done Done New false
9417 3254 Acked 4253 Write 2 Done Done New false
9418 3255 Acked 4254 Write 10 Done Done New false
9419 3256 Acked 4255 Read 8 Done Done New false
9420 3257 Acked 4256 Write 9 Done Done New false
9421 3258 Acked 4257 Read 3 Done Done New false
9422 3259 Acked 4258 Read 5 Done Done New false
9423 3260 Acked 4259 Write 7 Done Done New false
9424 3261 Acked 4260 Read 9 Done Done New false
9425 3262 Acked 4261 Write 8 Done Done New false
9426 3263 Acked 4262 Read 5 Done Done New false
9427 3264 Acked 4263 Read 4 Done Done New false
9428 3265 Acked 4264 Flush 0 Done Done New false
9429 3266 Acked 4265 Write 9 Done Done New false
9430 3267 Acked 4266 Read 9 Done Done New false
9431 3268 Acked 4267 Read 1 Done Done New false
9432 3269 Acked 4268 Write 1 Done Done New false
9433 3270 Acked 4269 Write 10 Done Done New false
9434 3271 Acked 4270 Write 5 Done Done New false
9435 3272 Acked 4271 Write 9 Done Done New false
9436 3273 Acked 4272 Read 2 Done Done New false
9437 3274 Acked 4273 Write 6 Done Done New false
9438 3275 Acked 4274 Write 8 Done Done New false
9439 3276 Acked 4275 Read 4 Done Done New false
9440 3277 Acked 4276 Read 10 Done Done New false
9441 3278 Acked 4277 Write 3 Done Done New false
9442 3279 Acked 4278 Write 7 Done Done New false
9443 3280 Acked 4279 Write 1 Done Done New false
9444 3281 Acked 4280 Read 8 Done Done New false
9445 3282 Acked 4281 Read 4 Done Done New false
9446 3283 Acked 4282 Flush 0 Done Done New false
9447 3284 Acked 4283 Write 2 Done Done New false
9448 3285 Acked 4284 Read 1 Done Done New false
9449 3286 Acked 4285 Read 4 Done Done New false
9450 3287 Acked 4286 Write 10 Done Done New false
9451 3288 Acked 4287 Read 6 Done Done New false
9452 3289 Acked 4288 Write 9 Done Done New false
9453 3290 Acked 4289 Read 10 Done Done New false
9454 3291 Acked 4290 Write 7 Done Done New false
9455 3292 Acked 4291 Read 5 Done Done New false
9456 3293 Acked 4292 Read 1 Done Done New false
9457 3294 Acked 4293 Read 10 Done Done New false
9458 3295 Acked 4294 Write 9 Done Done New false
9459 3296 Acked 4295 Write 9 Done Done New false
9460 3297 Acked 4296 Read 2 Done Done New false
9461 3298 Acked 4297 Flush 0 Done Done New false
9462 3299 Acked 4298 Write 2 Done Done New false
9463 3300 Acked 4299 Read 9 Done Done New false
9464 3301 Acked 4300 Write 5 Done Done New false
9465 3302 Acked 4301 Write 4 Done Done New false
9466 3303 Acked 4302 Read 1 Done Done New false
9467 3304 Acked 4303 Write 7 Done Done New false
9468 3305 Acked 4304 Write 9 Done Done New false
9469 3306 Acked 4305 Write 5 Done Done New false
9470 3307 Acked 4306 Read 4 Done Done New false
9471 3308 Acked 4307 Flush 0 Done Done New false
9472 3309 Acked 4308 Write 7 Done Done New false
9473 3310 Acked 4309 Write 3 Done Done New false
9474 3311 Acked 4310 Read 8 Done Done New false
9475 3312 Acked 4311 Read 2 Done Done New false
9476 3313 Acked 4312 Read 7 Done Done New false
9477 3314 Acked 4313 Read 8 Done Done New false
9478 3315 Acked 4314 Read 4 Done Done New false
9479 3316 Acked 4315 Read 8 Done Done New false
9480 3317 Acked 4316 Read 10 Done Done New false
9481 3318 Acked 4317 Read 6 Done Done New false
9482 3319 Acked 4318 Read 8 Done Done New false
9483 3320 Acked 4319 Write 7 Done Done New false
9484 3321 Acked 4320 Read 1 Done Done New false
9485 3322 Acked 4321 Write 3 Done Done New false
9486 3323 Acked 4322 Read 10 Done Done New false
9487 3324 Acked 4323 Read 2 Done Done New false
9488 3325 Acked 4324 Read 1 Done Done New false
9489 3326 Acked 4325 Flush 0 Done Done New false
9490 STATES DS:0 DS:1 DS:2 TOTAL
9491 New 0 0 303 303
9492 Sent 0 0 0 0
9493 Done 303 303 0 606
9494 Skipped 0 0 0 0
9495 Error 0 0 0 0
9496 Last Flush: 4325 4325 4022
9497 Downstairs last five completed: 4022 4021 4020 4019 4018 4017
9498 Upstairs last five completed: 3326 3325 3324 3323 3322 3321
9499 CLIENT: Up:0 ds:303 act:2
9500 [8830] got start action
9501 [8830] I am going to restart
9502 Starting downstairs at port 8830
9503 Make output file at "/tmp/dsc/downstairs-8830.txt"
9504 [8830][2] reports Starting
9505 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1184)
9506 [8830][2] reports Running
95072023-09-22T23:25:49.583ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
95082023-09-22T23:25:50.585ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
95092023-09-22T23:25:50.585ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
95102023-09-22T23:25:50.585ZINFOcrucible: [2] client re-new 303 jobs since flush 4022 = downstairs
95112023-09-22T23:25:50.586ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
95122023-09-22T23:25:50.587ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
95132023-09-22T23:25:50.587ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
95142023-09-22T23:25:50.587ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
95152023-09-22T23:25:50.587ZINFOcrucible: [2] send last flush ID to this DS: 4022
95162023-09-22T23:25:50.587ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
95172023-09-22T23:25:50.587ZINFOcrucible: [2] Replied this last flush ID: 4022
95182023-09-22T23:25:50.587ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
95192023-09-22T23:25:50.587ZINFOcrucible: [2] Transition from Offline to Replay
95202023-09-22T23:25:50.587ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
95212023-09-22T23:25:50.587ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
95222023-09-22T23:25:50.587ZINFOcrucible: [2] Transition from Replay to Active
95232023-09-22T23:25:50.587ZINFOcrucible: [2] Starts cmd_loop
95242023-09-22T23:25:50.587ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
95252023-09-22T23:25:50.587ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
95262023-09-22T23:25:50.587ZINFOcrucible: Set check for repair
95272023-09-22T23:25:51.589ZINFOcrucible: Checking if live repair is needed
95282023-09-22T23:25:51.589ZINFOcrucible: No Live Repair required at this time
95292023-09-22T23:25:51.589ZWARNcrucible: [2] flow control end
9530 ----------------------------------------------------------------
9531 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9532 Downstairs last five completed: 4325 4324 4323 4322 4321 4320
9533 Upstairs last five completed: 3326 3325 3324 3323 3322 3321
9534 CLIENT: Up:0 ds:0 act:3
9535 Replay: All jobs finished, all DS active.
9536 Main task has work to do, go find it
9537 got dsc Stop(1)
9538 stop 1
9539 Sep 22 23:25:52.624 INFO request completed, latency_us: 515, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 55f60a70-124c-4501-96e9-e558497213d4, remote_addr: 127.[8820] Got stop action so:false kr:false
9540 0.0.1:52471, local_addr: 127.0.0.1:9998
9541 Sep 22 23:25:52.625 INFO request completed, latency_us: 183, response_code: 200, uri: /state/cid/1, method: GET, req_id: efce2ae9-3514-40fa-974d-84dcff8252d9, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
95422023-09-22T23:25:52.625ZWARNcrucible: [1] downstairs disconnected
95432023-09-22T23:25:52.626ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
95442023-09-22T23:25:52.626ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
95452023-09-22T23:25:52.626ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
95462023-09-22T23:25:52.626ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
95472023-09-22T23:25:52.626ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
95482023-09-22T23:25:52.626ZWARNcrucible: [1] pm_task rx.recv() is None
9549 [8820] Exited with: ExitStatus(unix_wait_status(9))
9550 [8820][1] reports Exit
9551 Sep 22 23:25:56.628 INFO request completed, latency_us: 262, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1037a04f-53c4-44ec-945a-1e622e4eaddf, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
9552 001/300 Read block 4137 len 20480
9553 002/300 Write block 4652 len 20480 data: 2 2 2 2 4
9554 003/300 Write block 95 len 16384 data: 2 2 2 3
9555 004/300 Read block 196 len 16384
9556 005/300 Write block 1207 len 4096 data: 7
9557 006/300 Write block 1297 len 24576 data: 5 5 3 3 3 4
9558 007/300 Read block 3895 len 24576
9559 008/300 Write block 1187 len 12288 data: 4 4 4
9560 009/300 Read block 1590 len 8192
9561 010/300 Read block 4405 len 20480
9562 011/300 Read block 4318 len 40960
9563 012/300 Write block 3133 len 16384 data: 3 3 3 3
9564 013/300 Read block 1369 len 24576
9565 014/300 Read block 423 len 4096
9566 015/300 Read block 3752 len 36864
9567 016/300 Read block 1082 len 40960
9568 017/300 Read block 3414 len 8192
9569 018/300 Read block 1347 len 12288
9570 019/300 Write block 2537 len 20480 data: 3 3 3 3 2
9571 020/300 Flush
9572 021/300 Read block 925 len 16384
9573 022/300 Write block 4598 len 8192 data: 3 4
9574 023/300 Write block 4000 len 16384 data: 5 5 5 4
9575 024/300 Read block 3719 len 12288
9576 025/300 Write block 3727 len 32768 data: 2 2 2 2 2 2 3 4
9577 026/300 Write block 3065 len 12288 data: 2 2 2
9578 027/300 Write block 3817 len 28672 data: 2 2 2 2 2 2 3
9579 028/300 Flush
9580 029/300 Read block 2850 len 36864
9581 030/300 Flush
9582 031/300 Read block 132 len 12288
9583 032/300 Write block 2798 len 20480 data: 4 5 5 5 5
9584 033/300 Write block 32 len 40960 data: 5 5 5 4 4 4 4 4 5 5
9585 034/300 Write block 2381 len 8192 data: 4 4
9586 035/300 Read block 3723 len 16384
9587 036/300 Read block 4232 len 16384
9588 037/300 Read block 645 len 4096
9589 038/300 Write block 484 len 4096 data: 3
9590 039/300 Read block 1164 len 28672
9591 040/300 Read block 334 len 20480
9592 041/300 Write block 1438 len 8192 data: 3 3
9593 042/300 Read block 1842 len 20480
9594 043/300 Read block 470 len 36864
9595 044/300 Write block 2861 len 36864 data: 4 3 2 2 2 2 2 2 2
9596 045/300 Read block 4741 len 4096
9597 046/300 Write block 33 len 32768 data: 6 6 5 5 5 5 5 6
9598 047/300 Flush
9599 048/300 Read block 4135 len 16384
9600 049/300 Read block 3102 len 16384
9601 050/300 Read block 2434 len 36864
9602 051/300 Write block 4958 len 40960 data: 6 5 3 3 3 3 4 4 2 2
9603 052/300 Read block 3726 len 20480
9604 053/300 Read block 1651 len 8192
9605 054/300 Write block 1543 len 20480 data: 5 3 3 3 2
9606 055/300 Flush
9607 056/300 Write block 132 len 20480 data: 2 2 2 2 2
9608 057/300 Write block 4787 len 32768 data: 4 4 3 2 2 2 2 2
9609 058/300 Read block 3015 len 12288
9610 059/300 Read block 4375 len 12288
9611 060/300 Read block 529 len 12288
9612 061/300 Flush
9613 062/300 Read block 3211 len 16384
9614 063/300 Read block 643 len 24576
9615 064/300 Write block 2018 len 32768 data: 2 2 3 3 4 3 3 3
9616 065/300 Read block 760 len 24576
9617 066/300 Read block 1175 len 4096
9618 067/300 Write block 251 len 20480 data: 5 7 4 3 2
9619 068/300 Write block 4476 len 24576 data: 3 3 3 3 3 3
9620 069/300 Read block 898 len 12288
9621 070/300 Write block 1478 len 8192 data: 8 7
9622 071/300 Write block 2226 len 24576 data: 2 2 2 2 2 2
9623 072/300 Write block 728 len 4096 data: 7
9624 073/300 Read block 4004 len 4096
9625 074/300 Read block 2244 len 32768
9626 075/300 Write block 4272 len 36864 data: 3 4 4 4 6 5 3 5 4
9627 076/300 Read block 3027 len 36864
9628 077/300 Write block 3235 len 16384 data: 3 3 2 2
9629 078/300 Read block 3560 len 20480
9630 079/300 Write block 1982 len 12288 data: 3 3 5
9631 080/300 Write block 3665 len 36864 data: 4 3 4 4 4 3 3 2 2
9632 081/300 Flush
9633 082/300 Write block 2167 len 24576 data: 2 2 2 2 2 3
9634 083/300 Write block 3808 len 20480 data: 3 3 3 3 3
9635 084/300 Write block 452 len 8192 data: 2 2
9636 085/300 Read block 4705 len 20480
9637 086/300 Write block 4103 len 8192 data: 4 3
9638 087/300 Read block 4467 len 28672
9639 088/300 Read block 171 len 28672
9640 089/300 Read block 197 len 4096
9641 090/300 Read block 890 len 20480
9642 091/300 Write block 3366 len 28672 data: 3 5 4 4 4 3 3
9643 092/300 Read block 4282 len 36864
9644 093/300 Write block 4845 len 4096 data: 2
9645 094/300 Flush
9646 095/300 Write block 1049 len 20480 data: 4 4 3 2 2
9647 096/300 Read block 4233 len 20480
9648 097/300 Flush
9649 098/300 Write block 3199 len 12288 data: 2 2 2
9650 099/300 Read block 4425 len 24576
9651 100/300 Write block 1151 len 28672 data: 2 2 2 3 3 3 3
9652 101/300 Write block 1815 len 12288 data: 2 2 3
9653 102/300 Flush
9654 103/300 Read block 4892 len 32768
9655 104/300 Write block 3400 len 32768 data: 4 4 4 5 5 4 4 4
9656 105/300 Write block 4842 len 8192 data: 3 3
9657 106/300 Write block 4173 len 32768 data: 4 4 3 2 2 2 2 3
9658 107/300 Write block 2399 len 4096 data: 2
9659 108/300 Read block 1146 len 4096
9660 109/300 Write block 283 len 4096 data: 4
9661 110/300 Write block 4657 len 28672 data: 5 5 5 5 4 5 4
9662 111/300 Read block 2622 len 8192
9663 112/300 Read block 3813 len 8192
9664 113/300 Flush
9665 114/300 Read block 4303 len 28672
9666 115/300 Write block 1497 len 40960 data: 4 4 5 6 6 4 5 4 4 3
9667 116/300 Flush
9668 117/300 Read block 842 len 8192
9669 118/300 Write block 3178 len 40960 data: 3 3 3 4 3 4 4 3 3 3
9670 119/300 Flush
9671 120/300 Flush
9672 121/300 Write block 1763 len 36864 data: 2 3 4 5 5 4 4 5 5
9673 122/300 Read block 2259 len 32768
9674 123/300 Flush
9675 124/300 Read block 4387 len 32768
9676 125/300 Write block 2361 len 28672 data: 3 4 4 4 3 3 2
9677 126/300 Write block 4475 len 36864 data: 3 4 4 4 4 4 4 3 2
9678 127/300 Read block 3383 len 24576
9679 128/300 Read block 4111 len 24576
9680 129/300 Read block 1634 len 12288
9681 130/300 Read block 448 len 4096
9682 131/300 Read block 4905 len 12288
9683 132/300 Read block 1258 len 16384
9684 133/300 Read block 346 len 20480
9685 134/300 Read block 3741 len 12288
9686 135/300 Read block 2428 len 20480
9687 136/300 Flush
9688 137/300 Read block 2459 len 20480
9689 138/300 Write block 2074 len 8192 data: 2 2
9690 139/300 Read block 4505 len 36864
9691 140/300 Flush
9692 141/300 Read block 4540 len 20480
9693 142/300 Read block 454 len 4096
9694 143/300 Write block 3975 len 20480 data: 5 5 4 3 3
9695 144/300 Write block 324 len 32768 data: 4 4 3 5 4 3 3 3
9696 145/300 Write block 2447 len 16384 data: 3 3 5 4
9697 146/300 Read block 4108 len 28672
9698 147/300 Read block 2004 len 32768
9699 148/300 Flush
9700 149/300 Flush
9701 150/300 Write block 3904 len 36864 data: 4 4 3 4 4 5 4 3 3
9702 151/300 Write block 4913 len 36864 data: 6 6 6 6 6 4 4 2 2
9703 152/300 Write block 4147 len 20480 data: 5 5 5 5 5
9704 153/300 Write block 3797 len 4096 data: 3
9705 154/300 Write block 2421 len 12288 data: 2 2 2
9706 155/300 Write block 2680 len 8192 data: 3 3
9707 156/300 Read block 3910 len 40960
9708 157/300 Read block 4849 len 40960
9709 158/300 Read block 926 len 32768
9710 159/300 Write block 1326 len 16384 data: 3 4 4 4
9711 160/300 Read block 705 len 36864
9712 161/300 Write block 2138 len 20480 data: 4 4 3 2 2
9713 162/300 Write block 3889 len 4096 data: 3
9714 163/300 Write block 4907 len 12288 data: 3 4 3
9715 164/300 Flush
9716 165/300 Write block 1316 len 36864 data: 6 3 2 2 2 2 3 3 3
9717 166/300 Write block 552 len 28672 data: 2 2 2 2 2 2 2
9718 167/300 Read block 3792 len 32768
9719 168/300 Read block 2537 len 8192
9720 169/300 Write block 809 len 20480 data: 6 6 5 6 7
9721 170/300 Read block 4744 len 40960
9722 171/300 Write block 3699 len 12288 data: 3 4 5
9723 172/300 Flush
9724 173/300 Write block 1185 len 20480 data: 4 4 5 5 5
9725 174/300 Write block 2198 len 8192 data: 4 3
9726 175/300 Read block 3460 len 8192
9727 176/300 Read block 1706 len 20480
9728 177/300 Read block 2575 len 32768
9729 178/300 Read block 4082 len 24576
9730 179/300 Write block 2433 len 4096 data: 3
9731 180/300 Read block 559 len 8192
9732 181/300 Read block 4972 len 24576
9733 182/300 Read block 281 len 20480
9734 183/300 Read block 4686 len 8192
9735 184/300 Read block 1254 len 32768
9736 185/300 Read block 212 len 32768
9737 186/300 Read block 2753 len 4096
9738 187/300 Write block 3019 len 32768 data: 3 3 3 4 4 4 4 4
9739 188/300 Read block 3184 len 16384
9740 189/300 Write block 2549 len 28672 data: 3 4 4 5 4 4 4
9741 190/300 Read block 1600 len 16384
9742 191/300 Write block 4258 len 8192 data: 4 4
9743 192/300 Write block 2376 len 24576 data: 4 4 3 3 3 5
9744 193/300 Write block 3360 len 4096 data: 2
9745 194/300 Write block 2806 len 28672 data: 4 4 3 4 4 4 4
9746 195/300 Write block 2105 len 4096 data: 2
9747 196/300 Read block 434 len 16384
9748 197/300 Write block 2261 len 4096 data: 2
9749 198/300 Write block 960 len 16384 data: 3 3 4 4
9750 199/300 Read block 1075 len 20480
9751 200/300 Write block 1140 len 4096 data: 3
9752 201/300 Read block 993 len 8192
9753 202/300 Read block 1056 len 20480
9754 203/300 Read block 4442 len 32768
9755 204/300 Write block 1341 len 20480 data: 4 4 3 3 4
9756 205/300 Write block 1333 len 4096 data: 4
9757 206/300 Read block 331 len 40960
9758 207/300 Flush
9759 208/300 Read block 4718 len 8192
9760 209/300 Write block 958 len 28672 data: 3 3 4 4 5 5 4
9761 210/300 Write block 1361 len 24576 data: 6 3 4 4 4 3
9762 211/300 Read block 2481 len 8192
9763 212/300 Read block 4737 len 24576
9764 213/300 Read block 4257 len 4096
9765 214/300 Flush
9766 215/300 Write block 212 len 28672 data: 2 2 2 3 3 3 3
9767 216/300 Read block 2939 len 36864
9768 217/300 Read block 2129 len 8192
9769 218/300 Write block 1309 len 20480 data: 3 3 3 3 4
9770 219/300 Read block 4047 len 16384
9771 220/300 Read block 2972 len 20480
9772 221/300 Read block 2049 len 20480
9773 222/300 Flush
9774 223/300 Read block 1472 len 8192
9775 224/300 Write block 4642 len 8192 data: 2 2
9776 225/300 Read block 2731 len 40960
9777 226/300 Read block 1309 len 40960
9778 227/300 Write block 3006 len 36864 data: 3 3 3 3 2 2 3 3 3
9779 228/300 Read block 1343 len 32768
9780 229/300 Write block 4421 len 20480 data: 4 5 5 5 5
9781 230/300 Flush
9782 231/300 Read block 4896 len 24576
9783 232/300 Read block 1548 len 24576
9784 233/300 Write block 4487 len 8192 data: 3 4
9785 234/300 Read block 3446 len 4096
9786 235/300 Write block 4451 len 8192 data: 3 3
9787 236/300 Write block 2337 len 24576 data: 3 3 3 3 3 2
9788 237/300 Write block 2066 len 28672 data: 4 4 3 4 5 3 3
9789 238/300 Write block 4844 len 28672 data: 2 3 3 4 4 4 5
9790 239/300 Write block 4723 len 40960 data: 3 2 2 3 3 4 5 4 4 4
9791 240/300 Write block 2000 len 20480 data: 2 2 3 4 4
9792 241/300 Write block 4472 len 12288 data: 3 3 3
9793 242/300 Write block 1973 len 4096 data: 3
9794 243/300 Read block 2561 len 20480
9795 244/300 Read block 4147 len 32768
9796 245/300 Write block 2108 len 40960 data: 3 2 5 5 6 5 5 5 3 3
9797 246/300 Read block 1699 len 20480
9798 247/300 Read block 1640 len 16384
9799 248/300 Flush
9800 249/300 Write block 1757 len 24576 data: 3 3 3 3 3 2
9801 250/300 Read block 3694 len 8192
9802 251/300 Flush
9803 252/300 Write block 3755 len 28672 data: 2 2 2 2 2 2 3
9804 253/300 Write block 991 len 40960 data: 4 3 3 4 4 3 2 2 3 4
9805 254/300 Write block 4167 len 32768 data: 3 3 3 3 3 4 5 5
9806 255/300 Read block 86 len 36864
9807 256/300 Flush
9808 257/300 Read block 645 len 36864
9809 258/300 Read block 4268 len 24576
9810 259/300 Write block 392 len 36864 data: 4 3 2 2 2 2 3 4 4
9811 260/300 Read block 2814 len 8192
9812 261/300 Read block 4510 len 4096
9813 262/300 Read block 3735 len 4096
9814 263/300 Read block 2473 len 32768
9815 264/300 Read block 300 len 8192
9816 265/300 Read block 1544 len 16384
9817 266/300 Read block 3292 len 8192
9818 267/300 Read block 2139 len 40960
9819 268/300 Write block 3024 len 36864 data: 5 5 5 4 4 3 3 2 2
9820 269/300 Read block 3341 len 12288
9821 270/300 Read block 2675 len 28672
9822 271/300 Read block 2816 len 16384
9823 272/300 Write block 2253 len 32768 data: 4 4 3 3 3 3 2 2
9824 273/300 Write block 915 len 40960 data: 4 3 3 2 2 2 2 2 2 3
9825 274/300 Read block 1232 len 12288
9826 275/300 Write block 3340 len 12288 data: 5 4 4
9827 276/300 Read block 3636 len 16384
9828 277/300 Write block 3433 len 32768 data: 4 4 3 4 3 2 3 3
9829 278/300 Write block 403 len 40960 data: 4 5 5 4 4 4 4 4 4 2
9830 279/300 Read block 3239 len 20480
9831 280/300 Write block 1856 len 16384 data: 3 3 3 5
9832 281/300 Write block 3870 len 32768 data: 4 4 4 4 3 3 2 2
9833 282/300 Read block 524 len 36864
9834 283/300 Write block 412 len 36864 data: 3 2 2 2 2 2 4 4 5
9835 284/300 Read block 3745 len 12288
9836 285/300 Read block 2625 len 8192
9837 286/300 Read block 601 len 32768
9838 287/300 Read block 931 len 4096
9839 288/300 Read block 197 len 28672
9840 289/300 Read block 2048 len 12288
9841 290/300 Flush
9842 291/300 Write block 3704 len 32768 data: 4 4 4 4 2 3 4 4
9843 292/300 Write block 4212 len 40960 data: 5 6 5 5 5 4 2 2 2 2
9844 293/300 Read block 372 len 40960
9845 294/300 Write block 876 len 16384 data: 5 4 3 3
9846 295/300 Write block 2226 len 28672 data: 3 3 3 3 3 3 2
9847 296/300 Read block 3248 len 16384
9848 297/300 Write block 4848 len 24576 data: 5 5 6 5 5 6
9849 298/300 Write block 2105 len 24576 data: 3 4 4 4 3 6
9850 299/300 Read block 2543 len 16384
9851 300/300 Write block 1737 len 16384 data: 4 3 3 3
9852 Main task has work to do, go find it
9853 got dsc Start(1)
9854 start 1
9855 Sep 22 23:25:57.631 INFO request completed, latency_us: 316, response_code: 204, uri: /start/cid/1, method: POST, req_id: d83f199f-1222-42ee-bebd-ba828f5555d7, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
9856 Replay: started 1, returned:Ok(())
9857 ----------------------------------------------------------------
9858 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9859 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
9860 3327 Acked 4326 Read 5 Done New Done false
9861 3328 Acked 4327 Write 5 Done New Done false
9862 3329 Acked 4328 Write 4 Done New Done false
9863 3330 Acked 4329 Read 4 Done New Done false
9864 3331 Acked 4330 Write 1 Done New Done false
9865 3332 Acked 4331 Write 6 Done New Done false
9866 3333 Acked 4332 Read 6 Done New Done false
9867 3334 Acked 4333 Write 3 Done New Done false
9868 3335 Acked 4334 Read 2 Done New Done false
9869 3336 Acked 4335 Read 5 Done New Done false
9870 3337 Acked 4336 Read 10 Done New Done false
9871 3338 Acked 4337 Write 4 Done New Done false
9872 3339 Acked 4338 Read 6 Done New Done false
9873 3340 Acked 4339 Read 1 Done New Done false
9874 3341 Acked 4340 Read 9 Done New Done false
9875 3342 Acked 4341 Read 10 Done New Done false
9876 3343 Acked 4342 Read 2 Done New Done false
9877 3344 Acked 4343 Read 3 Done New Done false
9878 3345 Acked 4344 Write 5 Done New Done false
9879 3346 Acked 4345 Flush 0 Done New Done false
9880 3347 Acked 4346 Read 4 Done New Done false
9881 3348 Acked 4347 Write 2 Done New Done false
9882 3349 Acked 4348 Write 4 Done New Done false
9883 3350 Acked 4349 Read 3 Done New Done false
9884 3351 Acked 4350 Write 8 Done New Done false
9885 3352 Acked 4351 Write 3 Done New Done false
9886 3353 Acked 4352 Write 7 Done New Done false
9887 3354 Acked 4353 Flush 0 Done New Done false
9888 3355 Acked 4354 Read 9 Done New Done false
9889 3356 Acked 4355 Flush 0 Done New Done false
9890 3357 Acked 4356 Read 3 Done New Done false
9891 3358 Acked 4357 Write 5 Done New Done false
9892 3359 Acked 4358 Write 10 Done New Done false
9893 3360 Acked 4359 Write 2 Done New Done false
9894 3361 Acked 4360 Read 4 Done New Done false
9895 3362 Acked 4361 Read 4 Done New Done false
9896 3363 Acked 4362 Read 1 Done New Done false
9897 3364 Acked 4363 Write 1 Done New Done false
9898 3365 Acked 4364 Read 7 Done New Done false
9899 3366 Acked 4365 Read 5 Done New Done false
9900 3367 Acked 4366 Flush 0 Done New Done false
9901 3368 Acked 4367 Write 2 Done New Done false
9902 3369 Acked 4368 Read 5 Done New Done false
9903 3370 Acked 4369 Read 9 Done New Done false
9904 3371 Acked 4370 Write 9 Done New Done false
9905 3372 Acked 4371 Read 1 Done New Done false
9906 3373 Acked 4372 Write 8 Done New Done false
9907 3374 Acked 4373 Flush 0 Done New Done false
9908 3375 Acked 4374 Read 4 Done New Done false
9909 3376 Acked 4375 Read 4 Done New Done false
9910 3377 Acked 4376 Read 9 Done New Done false
9911 3378 Acked 4377 Write 10 Done New Done false
9912 3379 Acked 4378 Read 5 Done New Done false
9913 3380 Acked 4379 Read 2 Done New Done false
9914 3381 Acked 4380 Write 5 Done New Done false
9915 3382 Acked 4381 Flush 0 Done New Done false
9916 3383 Acked 4382 Write 5 Done New Done false
9917 3384 Acked 4383 Write 8 Done New Done false
9918 3385 Acked 4384 Read 3 Done New Done false
9919 3386 Acked 4385 Read 3 Done New Done false
9920 3387 Acked 4386 Read 3 Done New Done false
9921 3388 Acked 4387 Flush 0 Done New Done false
9922 3389 Acked 4388 Read 4 Done New Done false
9923 3390 Acked 4389 Read 6 Done New Done false
9924 3391 Acked 4390 Write 8 Done New Done false
9925 3392 Acked 4391 Read 6 Done New Done false
9926 3393 Acked 4392 Read 1 Done New Done false
9927 3394 Acked 4393 Write 5 Done New Done false
9928 3395 Acked 4394 Write 6 Done New Done false
9929 3396 Acked 4395 Read 3 Done New Done false
9930 3397 Acked 4396 Write 2 Done New Done false
9931 3398 Acked 4397 Write 6 Done New Done false
9932 3399 Acked 4398 Write 1 Done New Done false
9933 3400 Acked 4399 Read 1 Done New Done false
9934 3401 Acked 4400 Read 8 Done New Done false
9935 3402 Acked 4401 Write 9 Done New Done false
9936 3403 Acked 4402 Read 9 Done New Done false
9937 3404 Acked 4403 Write 4 Done New Done false
9938 3405 Acked 4404 Read 5 Done New Done false
9939 3406 Acked 4405 Write 3 Done New Done false
9940 3407 Acked 4406 Write 9 Done New Done false
9941 3408 Acked 4407 Flush 0 Done New Done false
9942 3409 Acked 4408 Write 6 Done New Done false
9943 3410 Acked 4409 Write 5 Done New Done false
9944 3411 Acked 4410 Write 2 Done New Done false
9945 3412 Acked 4411 Read 5 Done New Done false
9946 3413 Acked 4412 Write 2 Done New Done false
9947 3414 Acked 4413 Read 7 Done New Done false
9948 3415 Acked 4414 Read 7 Done New Done false
9949 3416 Acked 4415 Read 1 Done New Done false
9950 3417 Acked 4416 Read 5 Done New Done false
9951 3418 Acked 4417 Write 7 Done New Done false
9952 3419 Acked 4418 Read 9 Done New Done false
9953 3420 Acked 4419 Write 1 Done New Done false
9954 3421 Acked 4420 Flush 0 Done New Done false
9955 3422 Acked 4421 Write 5 Done New Done false
9956 3423 Acked 4422 Read 5 Done New Done false
9957 3424 Acked 4423 Flush 0 Done New Done false
9958 3425 Acked 4424 Write 3 Done New Done false
9959 3426 Acked 4425 Read 6 Done New Done false
9960 3427 Acked 4426 Write 7 Done New Done false
9961 3428 Acked 4427 Write 3 Done New Done false
9962 3429 Acked 4428 Flush 0 Done New Done false
9963 3430 Acked 4429 Read 8 Done New Done false
9964 3431 Acked 4430 Write 8 Done New Done false
9965 3432 Acked 4431 Write 2 Done New Done false
9966 3433 Acked 4432 Write 8 Done New Done false
9967 3434 Acked 4433 Write 1 Done New Done false
9968 3435 Acked 4434 Read 1 Done New Done false
9969 3436 Acked 4435 Write 1 Done New Done false
9970 3437 Acked 4436 Write 7 Done New Done false
9971 3438 Acked 4437 Read 2 Done New Done false
9972 3439 Acked 4438 Read 2 Done New Done false
9973 3440 Acked 4439 Flush 0 Done New Done false
9974 3441 Acked 4440 Read 7 Done New Done false
9975 3442 Acked 4441 Write 10 Done New Done false
9976 3443 Acked 4442 Flush 0 Done New Done false
9977 3444 Acked 4443 Read 2 Done New Done false
9978 3445 Acked 4444 Write 10 Done New Done false
9979 3446 Acked 4445 Flush 0 Done New Done false
9980 3447 Acked 4446 Flush 0 Done New Done false
9981 3448 Acked 4447 Write 9 Done New Done false
9982 3449 Acked 4448 Read 8 Done New Done false
9983 3450 Acked 4449 Flush 0 Done New Done false
9984 3451 Acked 4450 Read 8 Done New Done false
9985 3452 Acked 4451 Write 7 Done New Done false
9986 3453 Acked 4452 Write 9 Done New Done false
9987 3454 Acked 4453 Read 6 Done New Done false
9988 3455 Acked 4454 Read 6 Done New Done false
9989 3456 Acked 4455 Read 3 Done New Done false
9990 3457 Acked 4456 Read 1 Done New Done false
9991 3458 Acked 4457 Read 3 Done New Done false
9992 3459 Acked 4458 Read 4 Done New Done false
9993 3460 Acked 4459 Read 5 Done New Done false
9994 3461 Acked 4460 Read 3 Done New Done false
9995 3462 Acked 4461 Read 5 Done New Done false
9996 3463 Acked 4462 Flush 0 Done New Done false
9997 3464 Acked 4463 Read 5 Done New Done false
9998 3465 Acked 4464 Write 2 Done New Done false
9999 3466 Acked 4465 Read 9 Done New Done false
10000 3467 Acked 4466 Flush 0 Done New Done false
10001 3468 Acked 4467 Read 5 Done New Done false
10002 3469 Acked 4468 Read 1 Done New Done false
10003 3470 Acked 4469 Write 5 Done New Done false
10004 3471 Acked 4470 Write 8 Done New Done false
10005 3472 Acked 4471 Write 4 Done New Done false
10006 3473 Acked 4472 Read 7 Done New Done false
10007 3474 Acked 4473 Read 8 Done New Done false
10008 3475 Acked 4474 Flush 0 Done New Done false
10009 3476 Acked 4475 Flush 0 Done New Done false
10010 3477 Acked 4476 Write 9 Done New Done false
10011 3478 Acked 4477 Write 9 Done New Done false
10012 3479 Acked 4478 Write 5 Done New Done false
10013 3480 Acked 4479 Write 1 Done New Done false
10014 3481 Acked 4480 Write 3 Done New Done false
10015 3482 Acked 4481 Write 2 Done New Done false
10016 3483 Acked 4482 Read 10 Done New Done false
10017 3484 Acked 4483 Read 10 Done New Done false
10018 3485 Acked 4484 Read 8 Done New Done false
10019 3486 Acked 4485 Write 4 Done New Done false
10020 3487 Acked 4486 Read 9 Done New Done false
10021 3488 Acked 4487 Write 5 Done New Done false
10022 3489 Acked 4488 Write 1 Done New Done false
10023 3490 Acked 4489 Write 3 Done New Done false
10024 3491 Acked 4490 Flush 0 Done New Done false
10025 3492 Acked 4491 Write 9 Done New Done false
10026 3493 Acked 4492 Write 7 Done New Done false
10027 3494 Acked 4493 Read 8 Done New Done false
10028 3495 Acked 4494 Read 2 Done New Done false
10029 3496 Acked 4495 Write 5 Done New Done false
10030 3497 Acked 4496 Read 10 Done New Done false
10031 3498 Acked 4497 Write 3 Done New Done false
10032 3499 Acked 4498 Flush 0 Done New Done false
10033 3500 Acked 4499 Write 5 Done New Done false
10034 3501 Acked 4500 Write 2 Done New Done false
10035 3502 Acked 4501 Read 2 Done New Done false
10036 3503 Acked 4502 Read 5 Done New Done false
10037 3504 Acked 4503 Read 8 Done New Done false
10038 3505 Acked 4504 Read 6 Done New Done false
10039 3506 Acked 4505 Write 1 Done New Done false
10040 3507 Acked 4506 Read 2 Done New Done false
10041 3508 Acked 4507 Read 6 Done New Done false
10042 3509 Acked 4508 Read 5 Done New Done false
10043 3510 Acked 4509 Read 2 Done New Done false
10044 3511 Acked 4510 Read 8 Done New Done false
10045 3512 Acked 4511 Read 8 Done New Done false
10046 3513 Acked 4512 Read 1 Done New Done false
10047 3514 Acked 4513 Write 8 Done New Done false
10048 3515 Acked 4514 Read 4 Done New Done false
10049 3516 Acked 4515 Write 7 Done New Done false
10050 3517 Acked 4516 Read 4 Done New Done false
10051 3518 Acked 4517 Write 2 Done New Done false
10052 3519 Acked 4518 Write 6 Done New Done false
10053 3520 Acked 4519 Write 1 Done New Done false
10054 3521 Acked 4520 Write 7 Done New Done false
10055 3522 Acked 4521 Write 1 Done New Done false
10056 3523 Acked 4522 Read 4 Done New Done false
10057 3524 Acked 4523 Write 1 Done New Done false
10058 3525 Acked 4524 Write 4 Done New Done false
10059 3526 Acked 4525 Read 5 Done New Done false
10060 3527 Acked 4526 Write 1 Done New Done false
10061 3528 Acked 4527 Read 2 Done New Done false
10062 3529 Acked 4528 Read 5 Done New Done false
10063 3530 Acked 4529 Read 8 Done New Done false
10064 3531 Acked 4530 Write 5 Done New Done false
10065 3532 Acked 4531 Write 1 Done New Done false
10066 3533 Acked 4532 Read 10 Done New Done false
10067 3534 Acked 4533 Flush 0 Done New Done false
10068 3535 Acked 4534 Flush 0 Done New Done false
10069 3536 Acked 4535 Read 2 Done New Done false
10070 3537 Acked 4536 Write 7 Done New Done false
10071 3538 Acked 4537 Write 6 Done New Done false
10072 3539 Acked 4538 Read 2 Done New Done false
10073 3540 Acked 4539 Read 6 Done New Done false
10074 3541 Acked 4540 Read 1 Done New Done false
10075 3542 Acked 4541 Flush 0 Done New Done false
10076 3543 Acked 4542 Write 7 Done New Done false
10077 3544 Acked 4543 Read 9 Done New Done false
10078 3545 Acked 4544 Read 2 Done New Done false
10079 3546 Acked 4545 Write 5 Done New Done false
10080 3547 Acked 4546 Read 4 Done New Done false
10081 3548 Acked 4547 Read 5 Done New Done false
10082 3549 Acked 4548 Read 5 Done New Done false
10083 3550 Acked 4549 Flush 0 Done New Done false
10084 3551 Acked 4550 Read 2 Done New Done false
10085 3552 Acked 4551 Write 2 Done New Done false
10086 3553 Acked 4552 Read 10 Done New Done false
10087 3554 Acked 4553 Read 10 Done New Done false
10088 3555 Acked 4554 Write 9 Done New Done false
10089 3556 Acked 4555 Read 8 Done New Done false
10090 3557 Acked 4556 Write 5 Done New Done false
10091 3558 Acked 4557 Flush 0 Done New Done false
10092 3559 Acked 4558 Read 6 Done New Done false
10093 3560 Acked 4559 Read 6 Done New Done false
10094 3561 Acked 4560 Write 2 Done New Done false
10095 3562 Acked 4561 Read 1 Done New Done false
10096 3563 Acked 4562 Write 2 Done New Done false
10097 3564 Acked 4563 Write 6 Done New Done false
10098 3565 Acked 4564 Write 7 Done New Done false
10099 3566 Acked 4565 Write 7 Done New Done false
10100 3567 Acked 4566 Write 10 Done New Done false
10101 3568 Acked 4567 Write 5 Done New Done false
10102 3569 Acked 4568 Write 3 Done New Done false
10103 3570 Acked 4569 Write 1 Done New Done false
10104 3571 Acked 4570 Read 5 Done New Done false
10105 3572 Acked 4571 Read 8 Done New Done false
10106 3573 Acked 4572 Write 10 Done New Done false
10107 3574 Acked 4573 Read 5 Done New Done false
10108 3575 Acked 4574 Read 4 Done New Done false
10109 3576 Acked 4575 Flush 0 Done New Done false
10110 3577 Acked 4576 Write 6 Done New Done false
10111 3578 Acked 4577 Read 2 Done New Done false
10112 3579 Acked 4578 Flush 0 Done New Done false
10113 3580 Acked 4579 Write 7 Done New Done false
10114 3581 Acked 4580 Write 10 Done New Done false
10115 3582 Acked 4581 Write 8 Done New Done false
10116 3583 Acked 4582 Read 9 Done New Done false
10117 3584 Acked 4583 Flush 0 Done New Done false
10118 3585 Acked 4584 Read 9 Done New Done false
10119 3586 Acked 4585 Read 6 Done New Done false
10120 3587 Acked 4586 Write 9 Done New Done false
10121 3588 Acked 4587 Read 2 Done New Done false
10122 3589 Acked 4588 Read 1 Done New Done false
10123 3590 Acked 4589 Read 1 Done New Done false
10124 3591 Acked 4590 Read 8 Done New Done false
10125 3592 Acked 4591 Read 2 Done New Done false
10126 3593 Acked 4592 Read 4 Done New Done false
10127 3594 Acked 4593 Read 2 Done New Done false
10128 3595 Acked 4594 Read 10 Done New Done false
10129 3596 Acked 4595 Write 9 Done New Done false
10130 3597 Acked 4596 Read 3 Done New Done false
10131 3598 Acked 4597 Read 7 Done New Done false
10132 3599 Acked 4598 Read 4 Done New Done false
10133 3600 Acked 4599 Write 8 Done New Done false
10134 3601 Acked 4600 Write 10 Done New Done false
10135 3602 Acked 4601 Read 3 Done New Done false
10136 3603 Acked 4602 Write 3 Done New Done false
10137 3604 Acked 4603 Read 4 Done New Done false
10138 3605 Acked 4604 Write 8 Done New Done false
10139 3606 Acked 4605 Write 10 Done New Done false
10140 3607 Acked 4606 Read 5 Done New Done false
10141 3608 Acked 4607 Write 4 Done New Done false
10142 3609 Acked 4608 Write 8 Done New Done false
10143 3610 Acked 4609 Read 9 Done New Done false
10144 3611 Acked 4610 Write 9 Done New Done false
10145 3612 Acked 4611 Read 3 Done New Done false
10146 3613 Acked 4612 Read 2 Done New Done false
10147 3614 Acked 4613 Read 8 Done New Done false
10148 3615 Acked 4614 Read 1 Done New Done false
10149 3616 Acked 4615 Read 7 Done New Done false
10150 3617 Acked 4616 Read 3 Done New Done false
10151 3618 Acked 4617 Flush 0 Done New Done false
10152 3619 Acked 4618 Write 8 Done New Done false
10153 3620 Acked 4619 Write 10 Done New Done false
10154 3621 Acked 4620 Read 10 Done New Done false
10155 3622 Acked 4621 Write 4 Done New Done false
10156 3623 Acked 4622 Write 7 Done New Done false
10157 3624 Acked 4623 Read 4 Done New Done false
10158 3625 Acked 4624 Write 6 Done New Done false
10159 3626 Acked 4625 Write 6 Done New Done false
10160 3627 Acked 4626 Read 4 Done New Done false
10161 3628 Acked 4627 Write 4 Sent New Sent false
10162 STATES DS:0 DS:1 DS:2 TOTAL
10163 New 0 302 0 302
10164 Sent 1 0 1 2
10165 Done 301 0 301 602
10166 Skipped 0 0 0 0
10167 Error 0 0 0 0
10168 Last Flush: 4617 4325 4617
10169 Downstairs last five completed: 4325 4324 4323 4322 4321 4320
10170 Upstairs last five completed: 3628 3627 3626 3625 3624 3623
10171 CLIENT: Up:0 ds:302 act:2
101722023-09-22T23:25:58.632ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
10173 [8820] got start action
10174 [8820] I am going to restart
10175 Starting downstairs at port 8820
10176 Make output file at "/tmp/dsc/downstairs-8820.txt"
10177 [8820][1] reports Starting
10178 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1185)
10179 [8820][1] reports Running
101802023-09-22T23:25:59.633ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
101812023-09-22T23:25:59.633ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
101822023-09-22T23:25:59.633ZINFOcrucible: [1] client re-new 303 jobs since flush 4325 = downstairs
101832023-09-22T23:25:59.634ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
101842023-09-22T23:25:59.635ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
101852023-09-22T23:25:59.635ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
101862023-09-22T23:25:59.635ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
101872023-09-22T23:25:59.635ZINFOcrucible: [1] send last flush ID to this DS: 4325
101882023-09-22T23:25:59.635ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
101892023-09-22T23:25:59.635ZINFOcrucible: [1] Replied this last flush ID: 4325
101902023-09-22T23:25:59.635ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
101912023-09-22T23:25:59.635ZINFOcrucible: [1] Transition from Offline to Replay
101922023-09-22T23:25:59.635ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
101932023-09-22T23:25:59.635ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
101942023-09-22T23:25:59.636ZINFOcrucible: [1] Transition from Replay to Active
101952023-09-22T23:25:59.636ZINFOcrucible: [1] Starts cmd_loop
101962023-09-22T23:25:59.636ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
101972023-09-22T23:25:59.636ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
101982023-09-22T23:25:59.636ZINFOcrucible: Set check for repair
101992023-09-22T23:26:00.636ZINFOcrucible: Checking if live repair is needed
102002023-09-22T23:26:00.636ZINFOcrucible: No Live Repair required at this time
102012023-09-22T23:26:00.636ZWARNcrucible: [1] flow control end
10202 ----------------------------------------------------------------
10203 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10204 Downstairs last five completed: 4628 4627 4626 4625 4624 4623
10205 Upstairs last five completed: 3629 3628 3627 3626 3625 3624
10206 CLIENT: Up:0 ds:0 act:3
10207 Replay: All jobs finished, all DS active.
10208 Main task has work to do, go find it
10209 got dsc Stop(2)
10210 stop 2
10211 Sep 22 23:26:01.635 INFO request completed, latency_us: 517, response_code: 204, uri: /stop/cid/2, method[8830] Got stop action so:false kr:false
10212 : POST, req_id: 734a43e3-a795-4706-b11f-632e8c2d58b7, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
10213 Sep 22 23:26:01.637 INFO request completed, latency_us: 344, response_code: 200, uri: /state/cid/2, method: GET, req_id: d9becb77-a9d3-4925-b2e2-33851d602cf1, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
102142023-09-22T23:26:01.637ZWARNcrucible: [2] downstairs disconnected
102152023-09-22T23:26:01.637ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
102162023-09-22T23:26:01.637ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
102172023-09-22T23:26:01.637ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
102182023-09-22T23:26:01.637ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
102192023-09-22T23:26:01.638ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
102202023-09-22T23:26:01.638ZWARNcrucible: [2] pm_task rx.recv() is None
10221 [8830] Exited with: ExitStatus(unix_wait_status(9))
10222 [8830][2] reports Exit
10223 Sep 22 23:26:05.639 INFO request completed, latency_us: 267, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8ddaec4e-77b4-45be-9312-173a7a9886ba, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
10224 001/300 Write block 546 len 36864 data: 4 4 3 3 2 2 3 3 3
10225 002/300 Write block 4185 len 4096 data: 5
10226 003/300 Read block 4068 len 36864
10227 004/300 Read block 4232 len 24576
10228 005/300 Write block 1649 len 36864 data: 3 2 3 3 3 3 4 3 3
10229 006/300 Read block 2570 len 24576
10230 007/300 Read block 2959 len 24576
10231 008/300 Write block 3860 len 4096 data: 4
10232 009/300 Write block 1889 len 16384 data: 3 3 3 2
10233 010/300 Write block 375 len 28672 data: 4 4 4 4 3 4 3
10234 011/300 Read block 1444 len 20480
10235 012/300 Write block 2638 len 12288 data: 3 3 2
10236 013/300 Read block 43 len 32768
10237 014/300 Write block 4134 len 40960 data: 3 2 2 2 2 2 2 2 3 4
10238 015/300 Write block 4326 len 36864 data: 5 3 4 5 5 5 5 5 4
10239 016/300 Read block 1790 len 24576
10240 017/300 Write block 4159 len 24576 data: 4 4 3 2 2 2
10241 018/300 Read block 2284 len 20480
10242 019/300 Write block 4412 len 24576 data: 3 3 3 3 3 4
10243 020/300 Write block 1652 len 28672 data: 4 4 4 5 4 4 3
10244 021/300 Read block 128 len 28672
10245 022/300 Write block 93 len 28672 data: 2 2 3 3 3 4 2
10246 023/300 Write block 4418 len 8192 data: 4 5
10247 024/300 Read block 2834 len 12288
10248 025/300 Write block 2542 len 32768 data: 2 2 2 2 2 2 2 4
10249 026/300 Read block 3117 len 28672
10250 027/300 Read block 1852 len 24576
10251 028/300 Write block 1145 len 32768 data: 2 2 2 2 2 2 3 3
10252 029/300 Write block 1051 len 12288 data: 4 3 3
10253 030/300 Read block 591 len 12288
10254 031/300 Write block 2749 len 16384 data: 2 2 2 2
10255 032/300 Write block 3835 len 16384 data: 4 3 4 4
10256 033/300 Read block 3613 len 36864
10257 034/300 Write block 4363 len 28672 data: 3 3 4 4 4 4 4
10258 035/300 Write block 2040 len 16384 data: 2 2 2 2
10259 036/300 Flush
10260 037/300 Read block 4527 len 4096
10261 038/300 Read block 675 len 8192
10262 039/300 Write block 4196 len 28672 data: 3 3 3 3 3 3 3
10263 040/300 Write block 641 len 24576 data: 2 2 2 2 2 2
10264 041/300 Read block 2664 len 40960
10265 042/300 Write block 4284 len 28672 data: 3 4 3 3 3 3 3
10266 043/300 Write block 3307 len 40960 data: 4 3 3 3 3 4 4 5 3 3
10267 044/300 Write block 15 len 24576 data: 2 3 4 3 4 4
10268 045/300 Read block 1935 len 28672
10269 046/300 Write block 1898 len 20480 data: 3 4 3 3 3
10270 047/300 Read block 1517 len 12288
10271 048/300 Read block 1108 len 24576
10272 049/300 Read block 1473 len 4096
10273 050/300 Read block 633 len 36864
10274 051/300 Read block 4092 len 28672
10275 052/300 Read block 4676 len 8192
10276 053/300 Write block 2008 len 4096 data: 3
10277 054/300 Read block 4231 len 28672
10278 055/300 Read block 91 len 40960
10279 056/300 Flush
10280 057/300 Read block 1726 len 24576
10281 058/300 Read block 4241 len 16384
10282 059/300 Write block 994 len 28672 data: 5 5 4 3 3 4 5
10283 060/300 Write block 1268 len 28672 data: 4 5 5 5 4 4 4
10284 061/300 Write block 2782 len 20480 data: 3 4 4 4 4
10285 062/300 Read block 2522 len 4096
10286 063/300 Read block 4754 len 4096
10287 064/300 Write block 1934 len 8192 data: 3 3
10288 065/300 Write block 233 len 32768 data: 2 2 2 2 2 3 3 3
10289 066/300 Write block 4549 len 20480 data: 5 4 3 3 3
10290 067/300 Read block 4666 len 40960
10291 068/300 Read block 4443 len 16384
10292 069/300 Write block 3176 len 40960 data: 3 3 4 4 4 5 4 5 5 4
10293 070/300 Read block 4145 len 32768
10294 071/300 Flush
10295 072/300 Read block 2870 len 20480
10296 073/300 Read block 1012 len 32768
10297 074/300 Read block 1947 len 32768
10298 075/300 Read block 830 len 32768
10299 076/300 Read block 940 len 24576
10300 077/300 Flush
10301 078/300 Read block 224 len 24576
10302 079/300 Read block 799 len 16384
10303 080/300 Read block 4013 len 32768
10304 081/300 Write block 1174 len 8192 data: 3 2
10305 082/300 Read block 2097 len 24576
10306 083/300 Read block 1642 len 4096
10307 084/300 Read block 4766 len 8192
10308 085/300 Read block 3016 len 40960
10309 086/300 Write block 2370 len 12288 data: 3 4 4
10310 087/300 Write block 840 len 24576 data: 6 4 4 5 5 4
10311 088/300 Flush
10312 089/300 Write block 4026 len 16384 data: 2 2 2 2
10313 090/300 Read block 823 len 4096
10314 091/300 Write block 3026 len 12288 data: 6 5 5
10315 092/300 Read block 425 len 12288
10316 093/300 Read block 1972 len 12288
10317 094/300 Read block 4762 len 4096
10318 095/300 Read block 3528 len 20480
10319 096/300 Write block 3403 len 36864 data: 6 6 5 5 5 2 3 2 2
10320 097/300 Read block 4861 len 20480
10321 098/300 Write block 325 len 24576 data: 5 4 6 5 4 4
10322 099/300 Flush
10323 100/300 Write block 1496 len 8192 data: 4 5
10324 101/300 Flush
10325 102/300 Read block 4371 len 32768
10326 103/300 Flush
10327 104/300 Write block 3457 len 12288 data: 4 5 5
10328 105/300 Write block 570 len 24576 data: 3 4 4 4 4 5
10329 106/300 Write block 2584 len 32768 data: 5 5 4 4 4 4 3 3
10330 107/300 Write block 1373 len 40960 data: 3 3 4 3 3 3 2 2 2 5
10331 108/300 Write block 158 len 32768 data: 3 3 5 5 6 5 3 4
10332 109/300 Read block 3451 len 36864
10333 110/300 Write block 1206 len 40960 data: 7 8 8 7 5 5 4 3 3 2
10334 111/300 Write block 3795 len 32768 data: 4 3 4 3 3 3 3 3
10335 112/300 Write block 1262 len 28672 data: 2 3 3 3 3 4 5
10336 113/300 Write block 3613 len 12288 data: 5 5 5
10337 114/300 Write block 3096 len 32768 data: 3 4 4 4 5 5 5 5
10338 115/300 Write block 2717 len 28672 data: 3 4 4 4 3 3 4
10339 116/300 Read block 4708 len 40960
10340 117/300 Read block 1660 len 4096
10341 118/300 Write block 4017 len 32768 data: 2 2 2 2 2 2 2 2
10342 119/300 Write block 1809 len 32768 data: 2 3 3 3 2 2 3 3
10343 120/300 Read block 1125 len 40960
10344 121/300 Read block 2420 len 24576
10345 122/300 Read block 4173 len 32768
10346 123/300 Read block 3029 len 36864
10347 124/300 Write block 2789 len 28672 data: 4 4 3 3 2 3 3
10348 125/300 Read block 1557 len 8192
10349 126/300 Write block 3743 len 40960 data: 4 4 4 5 4 4 3 3 4 4
10350 127/300 Read block 1824 len 20480
10351 128/300 Write block 4035 len 36864 data: 6 6 6 5 4 4 3 3 3
10352 129/300 Write block 2341 len 28672 data: 4 3 3 3 4 4 5
10353 130/300 Write block 3240 len 8192 data: 2 2
10354 131/300 Write block 673 len 40960 data: 3 2 3 3 2 2 3 3 3 3
10355 132/300 Write block 856 len 8192 data: 4 4
10356 133/300 Read block 4392 len 8192
10357 134/300 Read block 3079 len 24576
10358 135/300 Flush
10359 136/300 Read block 3822 len 12288
10360 137/300 Read block 4894 len 32768
10361 138/300 Write block 3039 len 8192 data: 3 3
10362 139/300 Flush
10363 140/300 Flush
10364 141/300 Write block 1250 len 24576 data: 2 2 2 2 2 2
10365 142/300 Read block 1579 len 16384
10366 143/300 Read block 1183 len 16384
10367 144/300 Read block 4487 len 28672
10368 145/300 Read block 4648 len 32768
10369 146/300 Read block 2390 len 8192
10370 147/300 Read block 2420 len 36864
10371 148/300 Write block 144 len 28672 data: 4 4 5 5 5 5 5
10372 149/300 Write block 26 len 16384 data: 3 4 5 5
10373 150/300 Read block 5 len 32768
10374 151/300 Flush
10375 152/300 Read block 1163 len 8192
10376 153/300 Flush
10377 154/300 Flush
10378 155/300 Read block 1745 len 4096
10379 156/300 Read block 4908 len 28672
10380 157/300 Write block 1656 len 16384 data: 5 5 4 2
10381 158/300 Write block 4068 len 8192 data: 2 2
10382 159/300 Write block 2617 len 28672 data: 3 3 3 4 7 8 7
10383 160/300 Write block 2299 len 24576 data: 5 6 4 4 4 3
10384 161/300 Write block 1640 len 24576 data: 3 3 2 2 2 2
10385 162/300 Write block 4870 len 20480 data: 3 3 3 4 4
10386 163/300 Read block 655 len 4096
10387 164/300 Read block 3472 len 32768
10388 165/300 Write block 2247 len 24576 data: 2 3 4 3 4 5
10389 166/300 Write block 1110 len 24576 data: 3 2 2 2 2 2
10390 167/300 Read block 2866 len 8192
10391 168/300 Flush
10392 169/300 Write block 4303 len 16384 data: 4 4 4 4
10393 170/300 Read block 772 len 32768
10394 171/300 Write block 10 len 16384 data: 3 4 4 3
10395 172/300 Write block 3891 len 32768 data: 3 4 4 3 4 4 5 6
10396 173/300 Read block 197 len 32768
10397 174/300 Write block 4284 len 16384 data: 4 5 4 4
10398 175/300 Flush
10399 176/300 Flush
10400 177/300 Write block 2450 len 32768 data: 5 4 6 4 3 4 3 3
10401 178/300 Read block 3600 len 36864
10402 179/300 Read block 2470 len 8192
10403 180/300 Read block 561 len 20480
10404 181/300 Write block 2031 len 24576 data: 4 4 3 3 4 3
10405 182/300 Read block 2175 len 32768
10406 183/300 Write block 3908 len 12288 data: 5 6 5
10407 184/300 Write block 2282 len 40960 data: 3 3 3 3 3 2 2 3 3 3
10408 185/300 Flush
10409 186/300 Write block 1160 len 12288 data: 4 4 4
10410 187/300 Write block 3283 len 12288 data: 2 2 4
10411 188/300 Read block 1514 len 28672
10412 189/300 Read block 2900 len 16384
10413 190/300 Read block 2726 len 36864
10414 191/300 Read block 451 len 32768
10415 192/300 Write block 3904 len 16384 data: 5 5 4 5
10416 193/300 Read block 832 len 24576
10417 194/300 Read block 4624 len 24576
10418 195/300 Write block 947 len 16384 data: 2 3 3 2
10419 196/300 Write block 1304 len 28672 data: 4 4 3 3 3 4 4
10420 197/300 Write block 2585 len 16384 data: 6 5 5 5
10421 198/300 Read block 3888 len 40960
10422 199/300 Read block 182 len 40960
10423 200/300 Write block 498 len 24576 data: 3 3 3 6 6 6
10424 201/300 Read block 2531 len 40960
10425 202/300 Read block 4109 len 40960
10426 203/300 Write block 4988 len 28672 data: 6 5 4 4 4 3 3
10427 204/300 Write block 4061 len 8192 data: 5 5
10428 205/300 Read block 4630 len 24576
10429 206/300 Write block 4769 len 40960 data: 3 3 3 2 2 2 2 2 2 2
10430 207/300 Write block 65 len 24576 data: 4 3 3 4 4 3
10431 208/300 Read block 108 len 24576
10432 209/300 Flush
10433 210/300 Read block 261 len 8192
10434 211/300 Read block 4289 len 36864
10435 212/300 Write block 1760 len 20480 data: 4 4 3 3 4
10436 213/300 Read block 3783 len 8192
10437 214/300 Write block 4696 len 36864 data: 3 2 2 2 2 2 2 2 2
10438 215/300 Read block 1038 len 12288
10439 216/300 Read block 3557 len 24576
10440 217/300 Read block 4152 len 36864
10441 218/300 Flush
10442 219/300 Read block 574 len 4096
10443 220/300 Read block 92 len 8192
10444 221/300 Write block 4281 len 16384 data: 4 3 3 5
10445 222/300 Flush
10446 223/300 Flush
10447 224/300 Read block 1738 len 40960
10448 225/300 Flush
10449 226/300 Read block 1213 len 8192
10450 227/300 Read block 35 len 40960
10451 228/300 Write block 3026 len 4096 data: 7
10452 229/300 Read block 1673 len 12288
10453 230/300 Read block 1632 len 40960
10454 231/300 Flush
10455 232/300 Read block 3663 len 20480
10456 233/300 Write block 1616 len 12288 data: 3 4 3
10457 234/300 Read block 121 len 40960
10458 235/300 Read block 2563 len 20480
10459 236/300 Write block 381 len 16384 data: 4 3 4 5
10460 237/300 Write block 123 len 40960 data: 2 2 2 3 3 3 3 3 3 3
10461 238/300 Read block 4008 len 36864
10462 239/300 Read block 3355 len 20480
10463 240/300 Read block 3143 len 28672
10464 241/300 Read block 2013 len 20480
10465 242/300 Write block 1652 len 20480 data: 5 5 5 6 6
10466 243/300 Write block 1521 len 32768 data: 3 3 3 3 2 2 2 3
10467 244/300 Read block 3090 len 16384
10468 245/300 Flush
10469 246/300 Read block 3107 len 24576
10470 247/300 Read block 667 len 16384
10471 248/300 Write block 1115 len 8192 data: 3 2
10472 249/300 Read block 1415 len 20480
10473 250/300 Read block 306 len 40960
10474 251/300 Write block 4146 len 16384 data: 4 6 6 6
10475 252/300 Read block 4969 len 20480
10476 253/300 Read block 4743 len 32768
10477 254/300 Write block 2909 len 8192 data: 4 3
10478 255/300 Write block 4905 len 24576 data: 4 4 4 5 4 5
10479 256/300 Read block 3410 len 40960
10480 257/300 Flush
10481 258/300 Read block 820 len 32768
10482 259/300 Write block 4616 len 32768 data: 5 4 4 4 4 5 5 4
10483 260/300 Write block 823 len 40960 data: 3 3 3 3 2 2 2 2 3 3
10484 261/300 Flush
10485 262/300 Read block 4308 len 12288
10486 263/300 Read block 2997 len 8192
10487 264/300 Read block 1407 len 40960
10488 265/300 Read block 1833 len 4096
10489 266/300 Read block 520 len 4096
10490 267/300 Write block 1767 len 28672 data: 6 5 5 6 6 5 4
10491 268/300 Write block 331 len 36864 data: 4 3 4 4 4 4 5 4 5
10492 269/300 Write block 2295 len 8192 data: 4 4
10493 270/300 Flush
10494 271/300 Read block 2177 len 32768
10495 272/300 Read block 1151 len 40960
10496 273/300 Read block 4977 len 16384
10497 274/300 Write block 2162 len 12288 data: 4 3 3
10498 275/300 Read block 1334 len 4096
10499 276/300 Write block 515 len 4096 data: 2
10500 277/300 Write block 415 len 12288 data: 3 3 3
10501 278/300 Write block 4097 len 40960 data: 6 6 5 5 5 4 5 4 3 4
10502 279/300 Read block 3988 len 16384
10503 280/300 Write block 4094 len 32768 data: 4 5 5 7 7 6 6 6
10504 281/300 Read block 2696 len 4096
10505 282/300 Write block 4346 len 40960 data: 2 3 4 3 3 4 4 4 4 4
10506 283/300 Read block 3715 len 36864
10507 284/300 Write block 880 len 32768 data: 4 5 4 4 5 5 5 4
10508 285/300 Write block 299 len 40960 data: 4 5 5 4 4 5 4 4 4 3
10509 286/300 Write block 146 len 24576 data: 6 6 6 6 6 4
10510 287/300 Write block 1505 len 20480 data: 5 4 3 2 2
10511 288/300 Write block 4568 len 24576 data: 2 2 2 2 2 2
10512 289/300 Read block 1548 len 40960
10513 290/300 Write block 2176 len 8192 data: 2 2
10514 291/300 Write block 4910 len 8192 data: 6 6
10515 292/300 Read block 1700 len 8192
10516 293/300 Write block 3511 len 16384 data: 4 4 5 6
10517 294/300 Read block 4812 len 40960
10518 295/300 Write block 2944 len 36864 data: 3 3 3 3 3 3 3 3 4
10519 296/300 Write block 3948 len 36864 data: 2 2 2 2 2 2 3 3 2
10520 297/300 Read block 2043 len 4096
10521 298/300 Read block 1087 len 24576
10522 299/300 Read block 2955 len 28672
10523 300/300 Read block 121 len 20480
10524 Main task has work to do, go find it
10525 got dsc Start(2)
10526 start 2
10527 Sep 22 23:26:06.576 INFO request completed, latency_us: 316, response_code: 204, uri: /start/cid/2, method: POST, req_id: 56bfa645-fe27-444d-94d3-44fd21f64c66, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
10528 Replay: started 2, returned:Ok(())
10529 ----------------------------------------------------------------
10530 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10531 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
10532 3630 Acked 4629 Write 9 Done Done New false
10533 3631 Acked 4630 Write 1 Done Done New false
10534 3632 Acked 4631 Read 9 Done Done New false
10535 3633 Acked 4632 Read 6 Done Done New false
10536 3634 Acked 4633 Write 9 Done Done New false
10537 3635 Acked 4634 Read 6 Done Done New false
10538 3636 Acked 4635 Read 6 Done Done New false
10539 3637 Acked 4636 Write 1 Done Done New false
10540 3638 Acked 4637 Write 4 Done Done New false
10541 3639 Acked 4638 Write 7 Done Done New false
10542 3640 Acked 4639 Read 5 Done Done New false
10543 3641 Acked 4640 Write 3 Done Done New false
10544 3642 Acked 4641 Read 8 Done Done New false
10545 3643 Acked 4642 Write 10 Done Done New false
10546 3644 Acked 4643 Write 9 Done Done New false
10547 3645 Acked 4644 Read 6 Done Done New false
10548 3646 Acked 4645 Write 6 Done Done New false
10549 3647 Acked 4646 Read 5 Done Done New false
10550 3648 Acked 4647 Write 6 Done Done New false
10551 3649 Acked 4648 Write 7 Done Done New false
10552 3650 Acked 4649 Read 7 Done Done New false
10553 3651 Acked 4650 Write 7 Done Done New false
10554 3652 Acked 4651 Write 2 Done Done New false
10555 3653 Acked 4652 Read 3 Done Done New false
10556 3654 Acked 4653 Write 8 Done Done New false
10557 3655 Acked 4654 Read 7 Done Done New false
10558 3656 Acked 4655 Read 6 Done Done New false
10559 3657 Acked 4656 Write 8 Done Done New false
10560 3658 Acked 4657 Write 3 Done Done New false
10561 3659 Acked 4658 Read 3 Done Done New false
10562 3660 Acked 4659 Write 4 Done Done New false
10563 3661 Acked 4660 Write 4 Done Done New false
10564 3662 Acked 4661 Read 9 Done Done New false
10565 3663 Acked 4662 Write 7 Done Done New false
10566 3664 Acked 4663 Write 4 Done Done New false
10567 3665 Acked 4664 Flush 0 Done Done New false
10568 3666 Acked 4665 Read 1 Done Done New false
10569 3667 Acked 4666 Read 2 Done Done New false
10570 3668 Acked 4667 Write 7 Done Done New false
10571 3669 Acked 4668 Write 6 Done Done New false
10572 3670 Acked 4669 Flush 0 Done Done New false
10573 3671 Acked 4670 Read 10 Done Done New false
10574 3672 Acked 4671 Write 7 Done Done New false
10575 3673 Acked 4672 Write 10 Done Done New false
10576 3674 Acked 4673 Write 6 Done Done New false
10577 3675 Acked 4674 Read 7 Done Done New false
10578 3676 Acked 4675 Write 5 Done Done New false
10579 3677 Acked 4676 Read 3 Done Done New false
10580 3678 Acked 4677 Read 6 Done Done New false
10581 3679 Acked 4678 Read 1 Done Done New false
10582 3680 Acked 4679 Read 9 Done Done New false
10583 3681 Acked 4680 Read 7 Done Done New false
10584 3682 Acked 4681 Read 2 Done Done New false
10585 3683 Acked 4682 Write 1 Done Done New false
10586 3684 Acked 4683 Read 7 Done Done New false
10587 3685 Acked 4684 Read 10 Done Done New false
10588 3686 Acked 4685 Flush 0 Done Done New false
10589 3687 Acked 4686 Read 6 Done Done New false
10590 3688 Acked 4687 Read 4 Done Done New false
10591 3689 Acked 4688 Write 7 Done Done New false
10592 3690 Acked 4689 Write 7 Done Done New false
10593 3691 Acked 4690 Write 5 Done Done New false
10594 3692 Acked 4691 Read 1 Done Done New false
10595 3693 Acked 4692 Read 1 Done Done New false
10596 3694 Acked 4693 Write 2 Done Done New false
10597 3695 Acked 4694 Write 8 Done Done New false
10598 3696 Acked 4695 Write 5 Done Done New false
10599 3697 Acked 4696 Read 10 Done Done New false
10600 3698 Acked 4697 Read 4 Done Done New false
10601 3699 Acked 4698 Write 10 Done Done New false
10602 3700 Acked 4699 Read 8 Done Done New false
10603 3701 Acked 4700 Flush 0 Done Done New false
10604 3702 Acked 4701 Read 5 Done Done New false
10605 3703 Acked 4702 Read 8 Done Done New false
10606 3704 Acked 4703 Read 8 Done Done New false
10607 3705 Acked 4704 Read 8 Done Done New false
10608 3706 Acked 4705 Read 6 Done Done New false
10609 3707 Acked 4706 Flush 0 Done Done New false
10610 3708 Acked 4707 Read 6 Done Done New false
10611 3709 Acked 4708 Read 4 Done Done New false
10612 3710 Acked 4709 Read 8 Done Done New false
10613 3711 Acked 4710 Write 2 Done Done New false
10614 3712 Acked 4711 Read 6 Done Done New false
10615 3713 Acked 4712 Read 1 Done Done New false
10616 3714 Acked 4713 Read 2 Done Done New false
10617 3715 Acked 4714 Read 10 Done Done New false
10618 3716 Acked 4715 Write 3 Done Done New false
10619 3717 Acked 4716 Write 6 Done Done New false
10620 3718 Acked 4717 Flush 0 Done Done New false
10621 3719 Acked 4718 Write 4 Done Done New false
10622 3720 Acked 4719 Read 1 Done Done New false
10623 3721 Acked 4720 Write 3 Done Done New false
10624 3722 Acked 4721 Read 3 Done Done New false
10625 3723 Acked 4722 Read 3 Done Done New false
10626 3724 Acked 4723 Read 1 Done Done New false
10627 3725 Acked 4724 Read 5 Done Done New false
10628 3726 Acked 4725 Write 9 Done Done New false
10629 3727 Acked 4726 Read 5 Done Done New false
10630 3728 Acked 4727 Write 6 Done Done New false
10631 3729 Acked 4728 Flush 0 Done Done New false
10632 3730 Acked 4729 Write 2 Done Done New false
10633 3731 Acked 4730 Flush 0 Done Done New false
10634 3732 Acked 4731 Read 8 Done Done New false
10635 3733 Acked 4732 Flush 0 Done Done New false
10636 3734 Acked 4733 Write 3 Done Done New false
10637 3735 Acked 4734 Write 6 Done Done New false
10638 3736 Acked 4735 Write 8 Done Done New false
10639 3737 Acked 4736 Write 10 Done Done New false
10640 3738 Acked 4737 Write 8 Done Done New false
10641 3739 Acked 4738 Read 9 Done Done New false
10642 3740 Acked 4739 Write 10 Done Done New false
10643 3741 Acked 4740 Write 8 Done Done New false
10644 3742 Acked 4741 Write 7 Done Done New false
10645 3743 Acked 4742 Write 3 Done Done New false
10646 3744 Acked 4743 Write 8 Done Done New false
10647 3745 Acked 4744 Write 7 Done Done New false
10648 3746 Acked 4745 Read 10 Done Done New false
10649 3747 Acked 4746 Read 1 Done Done New false
10650 3748 Acked 4747 Write 8 Done Done New false
10651 3749 Acked 4748 Write 8 Done Done New false
10652 3750 Acked 4749 Read 10 Done Done New false
10653 3751 Acked 4750 Read 6 Done Done New false
10654 3752 Acked 4751 Read 8 Done Done New false
10655 3753 Acked 4752 Read 9 Done Done New false
10656 3754 Acked 4753 Write 7 Done Done New false
10657 3755 Acked 4754 Read 2 Done Done New false
10658 3756 Acked 4755 Write 10 Done Done New false
10659 3757 Acked 4756 Read 5 Done Done New false
10660 3758 Acked 4757 Write 9 Done Done New false
10661 3759 Acked 4758 Write 7 Done Done New false
10662 3760 Acked 4759 Write 2 Done Done New false
10663 3761 Acked 4760 Write 10 Done Done New false
10664 3762 Acked 4761 Write 2 Done Done New false
10665 3763 Acked 4762 Read 2 Done Done New false
10666 3764 Acked 4763 Read 6 Done Done New false
10667 3765 Acked 4764 Flush 0 Done Done New false
10668 3766 Acked 4765 Read 3 Done Done New false
10669 3767 Acked 4766 Read 8 Done Done New false
10670 3768 Acked 4767 Write 2 Done Done New false
10671 3769 Acked 4768 Flush 0 Done Done New false
10672 3770 Acked 4769 Flush 0 Done Done New false
10673 3771 Acked 4770 Write 6 Done Done New false
10674 3772 Acked 4771 Read 4 Done Done New false
10675 3773 Acked 4772 Read 4 Done Done New false
10676 3774 Acked 4773 Read 7 Done Done New false
10677 3775 Acked 4774 Read 8 Done Done New false
10678 3776 Acked 4775 Read 2 Done Done New false
10679 3777 Acked 4776 Read 9 Done Done New false
10680 3778 Acked 4777 Write 7 Done Done New false
10681 3779 Acked 4778 Write 4 Done Done New false
10682 3780 Acked 4779 Read 8 Done Done New false
10683 3781 Acked 4780 Flush 0 Done Done New false
10684 3782 Acked 4781 Read 2 Done Done New false
10685 3783 Acked 4782 Flush 0 Done Done New false
10686 3784 Acked 4783 Flush 0 Done Done New false
10687 3785 Acked 4784 Read 1 Done Done New false
10688 3786 Acked 4785 Read 7 Done Done New false
10689 3787 Acked 4786 Write 4 Done Done New false
10690 3788 Acked 4787 Write 2 Done Done New false
10691 3789 Acked 4788 Write 7 Done Done New false
10692 3790 Acked 4789 Write 6 Done Done New false
10693 3791 Acked 4790 Write 6 Done Done New false
10694 3792 Acked 4791 Write 5 Done Done New false
10695 3793 Acked 4792 Read 1 Done Done New false
10696 3794 Acked 4793 Read 8 Done Done New false
10697 3795 Acked 4794 Write 6 Done Done New false
10698 3796 Acked 4795 Write 6 Done Done New false
10699 3797 Acked 4796 Read 2 Done Done New false
10700 3798 Acked 4797 Flush 0 Done Done New false
10701 3799 Acked 4798 Write 4 Done Done New false
10702 3800 Acked 4799 Read 8 Done Done New false
10703 3801 Acked 4800 Write 4 Done Done New false
10704 3802 Acked 4801 Write 8 Done Done New false
10705 3803 Acked 4802 Read 8 Done Done New false
10706 3804 Acked 4803 Write 4 Done Done New false
10707 3805 Acked 4804 Flush 0 Done Done New false
10708 3806 Acked 4805 Flush 0 Done Done New false
10709 3807 Acked 4806 Write 8 Done Done New false
10710 3808 Acked 4807 Read 9 Done Done New false
10711 3809 Acked 4808 Read 2 Done Done New false
10712 3810 Acked 4809 Read 5 Done Done New false
10713 3811 Acked 4810 Write 6 Done Done New false
10714 3812 Acked 4811 Read 8 Done Done New false
10715 3813 Acked 4812 Write 3 Done Done New false
10716 3814 Acked 4813 Write 10 Done Done New false
10717 3815 Acked 4814 Flush 0 Done Done New false
10718 3816 Acked 4815 Write 3 Done Done New false
10719 3817 Acked 4816 Write 3 Done Done New false
10720 3818 Acked 4817 Read 7 Done Done New false
10721 3819 Acked 4818 Read 4 Done Done New false
10722 3820 Acked 4819 Read 9 Done Done New false
10723 3821 Acked 4820 Read 8 Done Done New false
10724 3822 Acked 4821 Write 4 Done Done New false
10725 3823 Acked 4822 Read 6 Done Done New false
10726 3824 Acked 4823 Read 6 Done Done New false
10727 3825 Acked 4824 Write 4 Done Done New false
10728 3826 Acked 4825 Write 7 Done Done New false
10729 3827 Acked 4826 Write 4 Done Done New false
10730 3828 Acked 4827 Read 10 Done Done New false
10731 3829 Acked 4828 Read 10 Done Done New false
10732 3830 Acked 4829 Write 6 Done Done New false
10733 3831 Acked 4830 Read 10 Done Done New false
10734 3832 Acked 4831 Flush 0 Done Done New false
10735 3833 Acked 4832 Read 10 Done Done New false
10736 3834 Acked 4833 Write 7 Done Done New false
10737 3835 Acked 4834 Write 2 Done Done New false
10738 3836 Acked 4835 Read 6 Done Done New false
10739 3837 Acked 4836 Write 10 Done Done New false
10740 3838 Acked 4837 Write 6 Done Done New false
10741 3839 Acked 4838 Read 6 Done Done New false
10742 3840 Acked 4839 Flush 0 Done Done New false
10743 3841 Acked 4840 Read 2 Done Done New false
10744 3842 Acked 4841 Read 9 Done Done New false
10745 3843 Acked 4842 Write 5 Done Done New false
10746 3844 Acked 4843 Read 2 Done Done New false
10747 3845 Acked 4844 Write 9 Done Done New false
10748 3846 Acked 4845 Read 3 Done Done New false
10749 3847 Acked 4846 Read 6 Done Done New false
10750 3848 Acked 4847 Read 9 Done Done New false
10751 3849 Acked 4848 Flush 0 Done Done New false
10752 3850 Acked 4849 Read 1 Done Done New false
10753 3851 Acked 4850 Read 2 Done Done New false
10754 3852 Acked 4851 Write 4 Done Done New false
10755 3853 Acked 4852 Flush 0 Done Done New false
10756 3854 Acked 4853 Flush 0 Done Done New false
10757 3855 Acked 4854 Read 10 Done Done New false
10758 3856 Acked 4855 Flush 0 Done Done New false
10759 3857 Acked 4856 Read 2 Done Done New false
10760 3858 Acked 4857 Read 10 Done Done New false
10761 3859 Acked 4858 Write 1 Done Done New false
10762 3860 Acked 4859 Read 3 Done Done New false
10763 3861 Acked 4860 Read 10 Done Done New false
10764 3862 Acked 4861 Flush 0 Done Done New false
10765 3863 Acked 4862 Read 5 Done Done New false
10766 3864 Acked 4863 Write 3 Done Done New false
10767 3865 Acked 4864 Read 10 Done Done New false
10768 3866 Acked 4865 Read 5 Done Done New false
10769 3867 Acked 4866 Write 4 Done Done New false
10770 3868 Acked 4867 Write 10 Done Done New false
10771 3869 Acked 4868 Read 9 Done Done New false
10772 3870 Acked 4869 Read 5 Done Done New false
10773 3871 Acked 4870 Read 7 Done Done New false
10774 3872 Acked 4871 Read 5 Done Done New false
10775 3873 Acked 4872 Write 5 Done Done New false
10776 3874 Acked 4873 Write 8 Done Done New false
10777 3875 Acked 4874 Read 4 Done Done New false
10778 3876 Acked 4875 Flush 0 Done Done New false
10779 3877 Acked 4876 Read 6 Done Done New false
10780 3878 Acked 4877 Read 4 Done Done New false
10781 3879 Acked 4878 Write 2 Done Done New false
10782 3880 Acked 4879 Read 5 Done Done New false
10783 3881 Acked 4880 Read 10 Done Done New false
10784 3882 Acked 4881 Write 4 Done Done New false
10785 3883 Acked 4882 Read 5 Done Done New false
10786 3884 Acked 4883 Read 8 Done Done New false
10787 3885 Acked 4884 Write 2 Done Done New false
10788 3886 Acked 4885 Write 6 Done Done New false
10789 3887 Acked 4886 Read 10 Done Done New false
10790 3888 Acked 4887 Flush 0 Done Done New false
10791 3889 Acked 4888 Read 8 Done Done New false
10792 3890 Acked 4889 Write 8 Done Done New false
10793 3891 Acked 4890 Write 10 Done Done New false
10794 3892 Acked 4891 Flush 0 Done Done New false
10795 3893 Acked 4892 Read 3 Done Done New false
10796 3894 Acked 4893 Read 2 Done Done New false
10797 3895 Acked 4894 Read 10 Done Done New false
10798 3896 Acked 4895 Read 1 Done Done New false
10799 3897 Acked 4896 Read 1 Done Done New false
10800 3898 Acked 4897 Write 7 Done Done New false
10801 3899 Acked 4898 Write 9 Done Done New false
10802 3900 Acked 4899 Write 2 Done Done New false
10803 3901 Acked 4900 Flush 0 Done Done New false
10804 3902 Acked 4901 Read 8 Done Done New false
10805 3903 Acked 4902 Read 10 Done Done New false
10806 3904 Acked 4903 Read 4 Done Done New false
10807 3905 Acked 4904 Write 3 Done Done New false
10808 3906 Acked 4905 Read 1 Done Done New false
10809 3907 Acked 4906 Write 1 Done Done New false
10810 3908 Acked 4907 Write 3 Done Done New false
10811 3909 Acked 4908 Write 10 Done Done New false
10812 3910 Acked 4909 Read 4 Done Done New false
10813 3911 Acked 4910 Write 8 Done Done New false
10814 3912 Acked 4911 Read 1 Done Done New false
10815 3913 Acked 4912 Write 10 Done Done New false
10816 3914 Acked 4913 Read 9 Done Done New false
10817 3915 Acked 4914 Write 8 Done Done New false
10818 3916 Acked 4915 Write 10 Done Done New false
10819 3917 Acked 4916 Write 6 Done Done New false
10820 3918 Acked 4917 Write 5 Done Done New false
10821 3919 Acked 4918 Write 6 Done Done New false
10822 3920 Acked 4919 Read 10 Done Done New false
10823 3921 Acked 4920 Write 2 Done Done New false
10824 3922 Acked 4921 Write 2 Done Done New false
10825 3923 Acked 4922 Read 2 Done Done New false
10826 3924 Acked 4923 Write 4 Done Done New false
10827 3925 Acked 4924 Read 10 Done Done New false
10828 3926 Acked 4925 Write 9 Done Done New false
10829 3927 Acked 4926 Write 9 Done Done New false
10830 3928 Acked 4927 Read 1 Done Done New false
10831 3929 Acked 4928 Read 6 Done Done New false
10832 3930 Acked 4929 Read 7 Done Done New false
10833 3931 Acked 4930 Read 5 Done Done New false
10834 STATES DS:0 DS:1 DS:2 TOTAL
10835 New 0 0 302 302
10836 Sent 0 0 0 0
10837 Done 302 302 0 604
10838 Skipped 0 0 0 0
10839 Error 0 0 0 0
10840 Last Flush: 4900 4900 4628
10841 Downstairs last five completed: 4628 4627 4626 4625 4624 4623
10842 Upstairs last five completed: 3931 3930 3929 3928 3927 3926
10843 CLIENT: Up:0 ds:302 act:2
10844 [8830] got start action
10845 [8830] I am going to restart
10846 Starting downstairs at port 8830
10847 Make output file at "/tmp/dsc/downstairs-8830.txt"
10848 [8830][2] reports Starting
10849 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1186)
10850 [8830][2] reports Running
10851 ----------------------------------------------------------------
10852 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
10853 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
10854 3630 Acked 4629 Write 9 Done Done New false
10855 3631 Acked 4630 Write 1 Done Done New false
10856 3632 Acked 4631 Read 9 Done Done New false
10857 3633 Acked 4632 Read 6 Done Done New false
10858 3634 Acked 4633 Write 9 Done Done New false
10859 3635 Acked 4634 Read 6 Done Done New false
10860 3636 Acked 4635 Read 6 Done Done New false
10861 3637 Acked 4636 Write 1 Done Done New false
10862 3638 Acked 4637 Write 4 Done Done New false
10863 3639 Acked 4638 Write 7 Done Done New false
10864 3640 Acked 4639 Read 5 Done Done New false
10865 3641 Acked 4640 Write 3 Done Done New false
10866 3642 Acked 4641 Read 8 Done Done New false
10867 3643 Acked 4642 Write 10 Done Done New false
10868 3644 Acked 4643 Write 9 Done Done New false
10869 3645 Acked 4644 Read 6 Done Done New false
10870 3646 Acked 4645 Write 6 Done Done New false
10871 3647 Acked 4646 Read 5 Done Done New false
10872 3648 Acked 4647 Write 6 Done Done New false
10873 3649 Acked 4648 Write 7 Done Done New false
10874 3650 Acked 4649 Read 7 Done Done New false
10875 3651 Acked 4650 Write 7 Done Done New false
10876 3652 Acked 4651 Write 2 Done Done New false
10877 3653 Acked 4652 Read 3 Done Done New false
10878 3654 Acked 4653 Write 8 Done Done New false
10879 3655 Acked 4654 Read 7 Done Done New false
10880 3656 Acked 4655 Read 6 Done Done New false
10881 3657 Acked 4656 Write 8 Done Done New false
10882 3658 Acked 4657 Write 3 Done Done New false
10883 3659 Acked 4658 Read 3 Done Done New false
10884 3660 Acked 4659 Write 4 Done Done New false
10885 3661 Acked 4660 Write 4 Done Done New false
10886 3662 Acked 4661 Read 9 Done Done New false
10887 3663 Acked 4662 Write 7 Done Done New false
10888 3664 Acked 4663 Write 4 Done Done New false
10889 3665 Acked 4664 Flush 0 Done Done New false
10890 3666 Acked 4665 Read 1 Done Done New false
10891 3667 Acked 4666 Read 2 Done Done New false
10892 3668 Acked 4667 Write 7 Done Done New false
10893 3669 Acked 4668 Write 6 Done Done New false
10894 3670 Acked 4669 Flush 0 Done Done New false
10895 3671 Acked 4670 Read 10 Done Done New false
10896 3672 Acked 4671 Write 7 Done Done New false
10897 3673 Acked 4672 Write 10 Done Done New false
10898 3674 Acked 4673 Write 6 Done Done New false
10899 3675 Acked 4674 Read 7 Done Done New false
10900 3676 Acked 4675 Write 5 Done Done New false
10901 3677 Acked 4676 Read 3 Done Done New false
10902 3678 Acked 4677 Read 6 Done Done New false
10903 3679 Acked 4678 Read 1 Done Done New false
10904 3680 Acked 4679 Read 9 Done Done New false
10905 3681 Acked 4680 Read 7 Done Done New false
10906 3682 Acked 4681 Read 2 Done Done New false
10907 3683 Acked 4682 Write 1 Done Done New false
10908 3684 Acked 4683 Read 7 Done Done New false
10909 3685 Acked 4684 Read 10 Done Done New false
10910 3686 Acked 4685 Flush 0 Done Done New false
10911 3687 Acked 4686 Read 6 Done Done New false
10912 3688 Acked 4687 Read 4 Done Done New false
10913 3689 Acked 4688 Write 7 Done Done New false
10914 3690 Acked 4689 Write 7 Done Done New false
10915 3691 Acked 4690 Write 5 Done Done New false
10916 3692 Acked 4691 Read 1 Done Done New false
10917 3693 Acked 4692 Read 1 Done Done New false
10918 3694 Acked 4693 Write 2 Done Done New false
10919 3695 Acked 4694 Write 8 Done Done New false
10920 3696 Acked 4695 Write 5 Done Done New false
10921 3697 Acked 4696 Read 10 Done Done New false
10922 3698 Acked 4697 Read 4 Done Done New false
10923 3699 Acked 4698 Write 10 Done Done New false
10924 3700 Acked 4699 Read 8 Done Done New false
10925 3701 Acked 4700 Flush 0 Done Done New false
10926 3702 Acked 4701 Read 5 Done Done New false
10927 3703 Acked 4702 Read 8 Done Done New false
10928 3704 Acked 4703 Read 8 Done Done New false
10929 3705 Acked 4704 Read 8 Done Done New false
10930 3706 Acked 4705 Read 6 Done Done New false
10931 3707 Acked 4706 Flush 0 Done Done New false
10932 3708 Acked 4707 Read 6 Done Done New false
10933 3709 Acked 4708 Read 4 Done Done New false
10934 3710 Acked 4709 Read 8 Done Done New false
10935 3711 Acked 4710 Write 2 Done Done New false
10936 3712 Acked 4711 Read 6 Done Done New false
10937 3713 Acked 4712 Read 1 Done Done New false
10938 3714 Acked 4713 Read 2 Done Done New false
10939 3715 Acked 4714 Read 10 Done Done New false
10940 3716 Acked 4715 Write 3 Done Done New false
10941 3717 Acked 4716 Write 6 Done Done New false
10942 3718 Acked 4717 Flush 0 Done Done New false
10943 3719 Acked 4718 Write 4 Done Done New false
10944 3720 Acked 4719 Read 1 Done Done New false
10945 3721 Acked 4720 Write 3 Done Done New false
10946 3722 Acked 4721 Read 3 Done Done New false
10947 3723 Acked 4722 Read 3 Done Done New false
10948 3724 Acked 4723 Read 1 Done Done New false
10949 3725 Acked 4724 Read 5 Done Done New false
10950 3726 Acked 4725 Write 9 Done Done New false
10951 3727 Acked 4726 Read 5 Done Done New false
10952 3728 Acked 4727 Write 6 Done Done New false
10953 3729 Acked 4728 Flush 0 Done Done New false
10954 3730 Acked 4729 Write 2 Done Done New false
10955 3731 Acked 4730 Flush 0 Done Done New false
10956 3732 Acked 4731 Read 8 Done Done New false
10957 3733 Acked 4732 Flush 0 Done Done New false
10958 3734 Acked 4733 Write 3 Done Done New false
10959 3735 Acked 4734 Write 6 Done Done New false
10960 3736 Acked 4735 Write 8 Done Done New false
10961 3737 Acked 4736 Write 10 Done Done New false
10962 3738 Acked 4737 Write 8 Done Done New false
10963 3739 Acked 4738 Read 9 Done Done New false
10964 3740 Acked 4739 Write 10 Done Done New false
10965 3741 Acked 4740 Write 8 Done Done New false
10966 3742 Acked 4741 Write 7 Done Done New false
10967 3743 Acked 4742 Write 3 Done Done New false
10968 3744 Acked 4743 Write 8 Done Done New false
10969 3745 Acked 4744 Write 7 Done Done New false
10970 3746 Acked 4745 Read 10 Done Done New false
10971 3747 Acked 4746 Read 1 Done Done New false
10972 3748 Acked 4747 Write 8 Done Done New false
10973 3749 Acked 4748 Write 8 Done Done New false
10974 3750 Acked 4749 Read 10 Done Done New false
10975 3751 Acked 4750 Read 6 Done Done New false
10976 3752 Acked 4751 Read 8 Done Done New false
10977 3753 Acked 4752 Read 9 Done Done New false
10978 3754 Acked 4753 Write 7 Done Done New false
10979 3755 Acked 4754 Read 2 Done Done New false
10980 3756 Acked 4755 Write 10 Done Done New false
10981 3757 Acked 4756 Read 5 Done Done New false
10982 3758 Acked 4757 Write 9 Done Done New false
10983 3759 Acked 4758 Write 7 Done Done New false
10984 3760 Acked 4759 Write 2 Done Done New false
10985 3761 Acked 4760 Write 10 Done Done New false
10986 3762 Acked 4761 Write 2 Done Done New false
10987 3763 Acked 4762 Read 2 Done Done New false
10988 3764 Acked 4763 Read 6 Done Done New false
10989 3765 Acked 4764 Flush 0 Done Done New false
10990 3766 Acked 4765 Read 3 Done Done New false
10991 3767 Acked 4766 Read 8 Done Done New false
10992 3768 Acked 4767 Write 2 Done Done New false
10993 3769 Acked 4768 Flush 0 Done Done New false
10994 3770 Acked 4769 Flush 0 Done Done New false
10995 3771 Acked 4770 Write 6 Done Done New false
10996 3772 Acked 4771 Read 4 Done Done New false
10997 3773 Acked 4772 Read 4 Done Done New false
10998 3774 Acked 4773 Read 7 Done Done New false
10999 3775 Acked 4774 Read 8 Done Done New false
11000 3776 Acked 4775 Read 2 Done Done New false
11001 3777 Acked 4776 Read 9 Done Done New false
11002 3778 Acked 4777 Write 7 Done Done New false
11003 3779 Acked 4778 Write 4 Done Done New false
11004 3780 Acked 4779 Read 8 Done Done New false
11005 3781 Acked 4780 Flush 0 Done Done New false
11006 3782 Acked 4781 Read 2 Done Done New false
11007 3783 Acked 4782 Flush 0 Done Done New false
11008 3784 Acked 4783 Flush 0 Done Done New false
11009 3785 Acked 4784 Read 1 Done Done New false
11010 3786 Acked 4785 Read 7 Done Done New false
11011 3787 Acked 4786 Write 4 Done Done New false
11012 3788 Acked 4787 Write 2 Done Done New false
11013 3789 Acked 4788 Write 7 Done Done New false
11014 3790 Acked 4789 Write 6 Done Done New false
11015 3791 Acked 4790 Write 6 Done Done New false
11016 3792 Acked 4791 Write 5 Done Done New false
11017 3793 Acked 4792 Read 1 Done Done New false
11018 3794 Acked 4793 Read 8 Done Done New false
11019 3795 Acked 4794 Write 6 Done Done New false
11020 3796 Acked 4795 Write 6 Done Done New false
11021 3797 Acked 4796 Read 2 Done Done New false
11022 3798 Acked 4797 Flush 0 Done Done New false
11023 3799 Acked 4798 Write 4 Done Done New false
11024 3800 Acked 4799 Read 8 Done Done New false
11025 3801 Acked 4800 Write 4 Done Done New false
11026 3802 Acked 4801 Write 8 Done Done New false
11027 3803 Acked 4802 Read 8 Done Done New false
11028 3804 Acked 4803 Write 4 Done Done New false
11029 3805 Acked 4804 Flush 0 Done Done New false
11030 3806 Acked 4805 Flush 0 Done Done New false
11031 3807 Acked 4806 Write 8 Done Done New false
11032 3808 Acked 4807 Read 9 Done Done New false
11033 3809 Acked 4808 Read 2 Done Done New false
11034 3810 Acked 4809 Read 5 Done Done New false
11035 3811 Acked 4810 Write 6 Done Done New false
11036 3812 Acked 4811 Read 8 Done Done New false
11037 3813 Acked 4812 Write 3 Done Done New false
11038 3814 Acked 4813 Write 10 Done Done New false
11039 3815 Acked 4814 Flush 0 Done Done New false
11040 3816 Acked 4815 Write 3 Done Done New false
11041 3817 Acked 4816 Write 3 Done Done New false
11042 3818 Acked 4817 Read 7 Done Done New false
11043 3819 Acked 4818 Read 4 Done Done New false
11044 3820 Acked 4819 Read 9 Done Done New false
11045 3821 Acked 4820 Read 8 Done Done New false
11046 3822 Acked 4821 Write 4 Done Done New false
11047 3823 Acked 4822 Read 6 Done Done New false
11048 3824 Acked 4823 Read 6 Done Done New false
11049 3825 Acked 4824 Write 4 Done Done New false
11050 3826 Acked 4825 Write 7 Done Done New false
11051 3827 Acked 4826 Write 4 Done Done New false
11052 3828 Acked 4827 Read 10 Done Done New false
11053 3829 Acked 4828 Read 10 Done Done New false
11054 3830 Acked 4829 Write 6 Done Done New false
11055 3831 Acked 4830 Read 10 Done Done New false
11056 3832 Acked 4831 Flush 0 Done Done New false
11057 3833 Acked 4832 Read 10 Done Done New false
11058 3834 Acked 4833 Write 7 Done Done New false
11059 3835 Acked 4834 Write 2 Done Done New false
11060 3836 Acked 4835 Read 6 Done Done New false
11061 3837 Acked 4836 Write 10 Done Done New false
11062 3838 Acked 4837 Write 6 Done Done New false
11063 3839 Acked 4838 Read 6 Done Done New false
11064 3840 Acked 4839 Flush 0 Done Done New false
11065 3841 Acked 4840 Read 2 Done Done New false
11066 3842 Acked 4841 Read 9 Done Done New false
11067 3843 Acked 4842 Write 5 Done Done New false
11068 3844 Acked 4843 Read 2 Done Done New false
11069 3845 Acked 4844 Write 9 Done Done New false
11070 3846 Acked 4845 Read 3 Done Done New false
11071 3847 Acked 4846 Read 6 Done Done New false
11072 3848 Acked 4847 Read 9 Done Done New false
11073 3849 Acked 4848 Flush 0 Done Done New false
11074 3850 Acked 4849 Read 1 Done Done New false
11075 3851 Acked 4850 Read 2 Done Done New false
11076 3852 Acked 4851 Write 4 Done Done New false
11077 3853 Acked 4852 Flush 0 Done Done New false
11078 3854 Acked 4853 Flush 0 Done Done New false
11079 3855 Acked 4854 Read 10 Done Done New false
11080 3856 Acked 4855 Flush 0 Done Done New false
11081 3857 Acked 4856 Read 2 Done Done New false
11082 3858 Acked 4857 Read 10 Done Done New false
11083 3859 Acked 4858 Write 1 Done Done New false
11084 3860 Acked 4859 Read 3 Done Done New false
11085 3861 Acked 4860 Read 10 Done Done New false
11086 3862 Acked 4861 Flush 0 Done Done New false
11087 3863 Acked 4862 Read 5 Done Done New false
11088 3864 Acked 4863 Write 3 Done Done New false
11089 3865 Acked 4864 Read 10 Done Done New false
11090 3866 Acked 4865 Read 5 Done Done New false
11091 3867 Acked 4866 Write 4 Done Done New false
11092 3868 Acked 4867 Write 10 Done Done New false
11093 3869 Acked 4868 Read 9 Done Done New false
11094 3870 Acked 4869 Read 5 Done Done New false
11095 3871 Acked 4870 Read 7 Done Done New false
11096 3872 Acked 4871 Read 5 Done Done New false
11097 3873 Acked 4872 Write 5 Done Done New false
11098 3874 Acked 4873 Write 8 Done Done New false
11099 3875 Acked 4874 Read 4 Done Done New false
11100 3876 Acked 4875 Flush 0 Done Done New false
11101 3877 Acked 4876 Read 6 Done Done New false
11102 3878 Acked 4877 Read 4 Done Done New false
11103 3879 Acked 4878 Write 2 Done Done New false
11104 3880 Acked 4879 Read 5 Done Done New false
11105 3881 Acked 4880 Read 10 Done Done New false
11106 3882 Acked 4881 Write 4 Done Done New false
11107 3883 Acked 4882 Read 5 Done Done New false
11108 3884 Acked 4883 Read 8 Done Done New false
11109 3885 Acked 4884 Write 2 Done Done New false
11110 3886 Acked 4885 Write 6 Done Done New false
11111 3887 Acked 4886 Read 10 Done Done New false
11112 3888 Acked 4887 Flush 0 Done Done New false
11113 3889 Acked 4888 Read 8 Done Done New false
11114 3890 Acked 4889 Write 8 Done Done New false
11115 3891 Acked 4890 Write 10 Done Done New false
11116 3892 Acked 4891 Flush 0 Done Done New false
11117 3893 Acked 4892 Read 3 Done Done New false
11118 3894 Acked 4893 Read 2 Done Done New false
11119 3895 Acked 4894 Read 10 Done Done New false
11120 3896 Acked 4895 Read 1 Done Done New false
11121 3897 Acked 4896 Read 1 Done Done New false
11122 3898 Acked 4897 Write 7 Done Done New false
11123 3899 Acked 4898 Write 9 Done Done New false
11124 3900 Acked 4899 Write 2 Done Done New false
11125 3901 Acked 4900 Flush 0 Done Done New false
11126 3902 Acked 4901 Read 8 Done Done New false
11127 3903 Acked 4902 Read 10 Done Done New false
11128 3904 Acked 4903 Read 4 Done Done New false
11129 3905 Acked 4904 Write 3 Done Done New false
11130 3906 Acked 4905 Read 1 Done Done New false
11131 3907 Acked 4906 Write 1 Done Done New false
11132 3908 Acked 4907 Write 3 Done Done New false
11133 3909 Acked 4908 Write 10 Done Done New false
11134 3910 Acked 4909 Read 4 Done Done New false
11135 3911 Acked 4910 Write 8 Done Done New false
11136 3912 Acked 4911 Read 1 Done Done New false
11137 3913 Acked 4912 Write 10 Done Done New false
11138 3914 Acked 4913 Read 9 Done Done New false
11139 3915 Acked 4914 Write 8 Done Done New false
11140 3916 Acked 4915 Write 10 Done Done New false
11141 3917 Acked 4916 Write 6 Done Done New false
11142 3918 Acked 4917 Write 5 Done Done New false
11143 3919 Acked 4918 Write 6 Done Done New false
11144 3920 Acked 4919 Read 10 Done Done New false
11145 3921 Acked 4920 Write 2 Done Done New false
11146 3922 Acked 4921 Write 2 Done Done New false
11147 3923 Acked 4922 Read 2 Done Done New false
11148 3924 Acked 4923 Write 4 Done Done New false
11149 3925 Acked 4924 Read 10 Done Done New false
11150 3926 Acked 4925 Write 9 Done Done New false
11151 3927 Acked 4926 Write 9 Done Done New false
11152 3928 Acked 4927 Read 1 Done Done New false
11153 3929 Acked 4928 Read 6 Done Done New false
11154 3930 Acked 4929 Read 7 Done Done New false
11155 3931 Acked 4930 Read 5 Done Done New false
11156 3932 Acked 4931 Flush 0 Done Done New false
11157 STATES DS:0 DS:1 DS:2 TOTAL
11158 New 0 0 303 303
11159 Sent 0 0 0 0
11160 Done 303 303 0 606
11161 Skipped 0 0 0 0
11162 Error 0 0 0 0
11163 Last Flush: 4931 4931 4628
11164 Downstairs last five completed: 4628 4627 4626 4625 4624 4623
11165 Upstairs last five completed: 3932 3931 3930 3929 3928 3927
11166 CLIENT: Up:0 ds:303 act:2
111672023-09-22T23:26:10.646ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
111682023-09-22T23:26:10.646ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
111692023-09-22T23:26:10.646ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
111702023-09-22T23:26:10.646ZINFOcrucible: [2] client re-new 303 jobs since flush 4628 = downstairs
111712023-09-22T23:26:10.647ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
111722023-09-22T23:26:10.648ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
111732023-09-22T23:26:10.648ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
111742023-09-22T23:26:10.648ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
111752023-09-22T23:26:10.648ZINFOcrucible: [2] send last flush ID to this DS: 4628
111762023-09-22T23:26:10.648ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
111772023-09-22T23:26:10.648ZINFOcrucible: [2] Replied this last flush ID: 4628
111782023-09-22T23:26:10.648ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
111792023-09-22T23:26:10.648ZINFOcrucible: [2] Transition from Offline to Replay
111802023-09-22T23:26:10.648ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
111812023-09-22T23:26:10.648ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
111822023-09-22T23:26:10.648ZINFOcrucible: [2] Transition from Replay to Active
111832023-09-22T23:26:10.649ZINFOcrucible: [2] Starts cmd_loop
111842023-09-22T23:26:10.649ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
111852023-09-22T23:26:10.649ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
111862023-09-22T23:26:10.649ZINFOcrucible: Set check for repair
111872023-09-22T23:26:11.650ZINFOcrucible: Checking if live repair is needed
111882023-09-22T23:26:11.650ZINFOcrucible: No Live Repair required at this time
111892023-09-22T23:26:11.650ZWARNcrucible: [2] flow control end
11190 ----------------------------------------------------------------
11191 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11192 Downstairs last five completed: 4931 4930 4929 4928 4927 4926
11193 Upstairs last five completed: 3932 3931 3930 3929 3928 3927
11194 CLIENT: Up:0 ds:0 act:3
11195 Replay: All jobs finished, all DS active.
11196 Main task has work to do, go find it
11197 got dsc Stop(0)
11198 stop 0
11199 Sep 22 23:26:14.583 INFO request completed, latency_us: 523, response_code: 204, uri: /stop/cid/0, method: POST, req_id: fa12774d-380b-441d-bfa7-0c1cda0d970b[8810] Got stop action so:false kr:false
11200 , remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
11201 Sep 22 23:26:14.584 INFO request completed, latency_us: 331, response_code: 200, uri: /state/cid/0, method: GET, req_id: 883be27a-a3bb-4cbe-ae08-a68bb62d9de9, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
112022023-09-22T23:26:14.585ZWARNcrucible: [0] downstairs disconnected
112032023-09-22T23:26:14.585ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
112042023-09-22T23:26:14.585ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
112052023-09-22T23:26:14.585ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
112062023-09-22T23:26:14.585ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
112072023-09-22T23:26:14.585ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
112082023-09-22T23:26:14.585ZWARNcrucible: [0] pm_task rx.recv() is None
11209 [8810] Exited with: ExitStatus(unix_wait_status(9))
11210 [8810][0] reports Exit
11211 Sep 22 23:26:18.586 INFO request completed, latency_us: 284, response_code: 200, uri: /state/cid/0, method: GET, req_id: 733b606e-86a4-4937-9276-93dcb4016d75, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
11212 001/300 Write block 4365 len 8192 data: 5 5
11213 002/300 Write block 3271 len 24576 data: 3 2 2 2 2 2
11214 003/300 Write block 3997 len 28672 data: 6 5 5 6 6 6 5
11215 004/300 Read block 2027 len 16384
11216 005/300 Write block 85 len 20480 data: 2 2 2 2 2
11217 006/300 Read block 2381 len 20480
11218 007/300 Read block 4562 len 24576
11219 008/300 Read block 4246 len 24576
11220 009/300 Read block 2842 len 36864
11221 010/300 Read block 2630 len 12288
11222 011/300 Read block 3934 len 16384
11223 012/300 Read block 1463 len 20480
11224 013/300 Write block 1209 len 4096 data: 8
11225 014/300 Read block 473 len 24576
11226 015/300 Read block 4642 len 12288
11227 016/300 Read block 631 len 40960
11228 017/300 Write block 3532 len 24576 data: 2 2 2 2 2 2
11229 018/300 Write block 482 len 20480 data: 2 3 4 6 6
11230 019/300 Read block 4652 len 12288
11231 020/300 Flush
11232 021/300 Read block 4212 len 4096
11233 022/300 Write block 2706 len 4096 data: 3
11234 023/300 Write block 4376 len 20480 data: 4 4 4 4 4
11235 024/300 Write block 4629 len 8192 data: 4 3
11236 025/300 Read block 3293 len 24576
11237 026/300 Write block 2945 len 4096 data: 4
11238 027/300 Write block 4942 len 40960 data: 5 4 3 2 2 2 3 3 4 4
11239 028/300 Read block 2352 len 20480
11240 029/300 Write block 4687 len 16384 data: 3 3 3 3
11241 030/300 Write block 1421 len 12288 data: 4 3 3
11242 031/300 Read block 3651 len 8192
11243 032/300 Write block 3345 len 12288 data: 3 3 3
11244 033/300 Read block 1013 len 12288
11245 034/300 Write block 4281 len 8192 data: 5 4
11246 035/300 Read block 2003 len 24576
11247 036/300 Read block 1253 len 8192
11248 037/300 Write block 742 len 28672 data: 4 3 2 2 2 2 3
11249 038/300 Write block 3713 len 20480 data: 3 4 5 5 7
11250 039/300 Write block 342 len 20480 data: 3 2 2 3 3
11251 040/300 Read block 4051 len 36864
11252 041/300 Read block 388 len 36864
11253 042/300 Write block 3201 len 12288 data: 3 2 2
11254 043/300 Read block 3853 len 8192
11255 044/300 Read block 980 len 40960
11256 045/300 Read block 785 len 4096
11257 046/300 Write block 2897 len 20480 data: 6 6 5 4 5
11258 047/300 Read block 1212 len 8192
11259 048/300 Write block 1214 len 40960 data: 4 3 2 2 2 2 2 2 2 3
11260 049/300 Read block 1687 len 16384
11261 050/300 Read block 3226 len 12288
11262 051/300 Read block 3092 len 24576
11263 052/300 Read block 1847 len 4096
11264 053/300 Read block 2668 len 32768
11265 054/300 Write block 2507 len 24576 data: 3 5 6 6 8 9
11266 055/300 Read block 1288 len 12288
11267 056/300 Flush
11268 057/300 Write block 4385 len 28672 data: 4 3 3 3 3 3 2
11269 058/300 Write block 1727 len 40960 data: 3 4 6 5 5 5 5 5 5 4
11270 059/300 Read block 3776 len 20480
11271 060/300 Write block 1346 len 20480 data: 4 4 4 4 4
11272 061/300 Read block 4473 len 24576
11273 062/300 Write block 1443 len 36864 data: 3 3 3 3 4 4 3 3 3
11274 063/300 Read block 2545 len 24576
11275 064/300 Read block 4198 len 40960
11276 065/300 Read block 1655 len 12288
11277 066/300 Read block 3272 len 28672
11278 067/300 Write block 2611 len 8192 data: 3 3
11279 068/300 Write block 3508 len 40960 data: 4 4 4 5 5 6 7 5 5 3
11280 069/300 Read block 526 len 28672
11281 070/300 Read block 972 len 16384
11282 071/300 Read block 1335 len 28672
11283 072/300 Read block 1151 len 16384
11284 073/300 Read block 4545 len 32768
11285 074/300 Read block 3747 len 40960
11286 075/300 Write block 1186 len 28672 data: 5 6 6 6 4 3 2
11287 076/300 Read block 4961 len 8192
11288 077/300 Flush
11289 078/300 Read block 2650 len 8192
11290 079/300 Flush
11291 080/300 Write block 2922 len 20480 data: 4 4 6 4 4
11292 081/300 Read block 1018 len 4096
11293 082/300 Write block 2829 len 20480 data: 4 4 3 6 6
11294 083/300 Write block 477 len 36864 data: 2 2 2 2 2 3 4 5 7
11295 084/300 Read block 1152 len 20480
11296 085/300 Write block 272 len 20480 data: 3 3 3 3 3
11297 086/300 Flush
11298 087/300 Write block 4285 len 40960 data: 6 5 5 4 4 4 4 5 5 6
11299 088/300 Read block 3252 len 32768
11300 089/300 Read block 1612 len 8192
11301 090/300 Write block 3047 len 16384 data: 5 4 3 3
11302 091/300 Read block 4115 len 20480
11303 092/300 Flush
11304 093/300 Write block 965 len 24576 data: 4 4 3 3 3 3
11305 094/300 Write block 985 len 28672 data: 3 4 4 6 6 5 5
11306 095/300 Read block 3501 len 4096
11307 096/300 Flush
11308 097/300 Flush
11309 098/300 Write block 3151 len 16384 data: 3 3 5 5
11310 099/300 Read block 1925 len 24576
11311 100/300 Write block 3571 len 16384 data: 4 3 2 2
11312 101/300 Read block 4817 len 8192
11313 102/300 Write block 622 len 12288 data: 3 3 2
11314 103/300 Write block 1179 len 12288 data: 2 2 2
11315 104/300 Write block 1954 len 8192 data: 4 6
11316 105/300 Read block 1833 len 40960
11317 106/300 Write block 3365 len 16384 data: 2 4 6 5
11318 107/300 Read block 2983 len 24576
11319 108/300 Read block 4911 len 8192
11320 109/300 Read block 37 len 16384
11321 110/300 Flush
11322 111/300 Read block 4398 len 20480
11323 112/300 Read block 3188 len 36864
11324 113/300 Read block 1008 len 12288
11325 114/300 Flush
11326 115/300 Write block 2853 len 8192 data: 4 4
11327 116/300 Read block 121 len 24576
11328 117/300 Read block 2945 len 8192
11329 118/300 Read block 2118 len 36864
11330 119/300 Read block 1455 len 16384
11331 120/300 Write block 1999 len 12288 data: 2 3 3
11332 121/300 Write block 4497 len 16384 data: 3 3 2 2
11333 122/300 Read block 3476 len 4096
11334 123/300 Read block 876 len 12288
11335 124/300 Write block 4071 len 8192 data: 2 2
11336 125/300 Read block 36 len 20480
11337 126/300 Read block 2141 len 28672
11338 127/300 Read block 4195 len 36864
11339 128/300 Write block 358 len 16384 data: 6 6 5 4
11340 129/300 Read block 654 len 40960
11341 130/300 Write block 3052 len 8192 data: 3 4
11342 131/300 Read block 1511 len 24576
11343 132/300 Read block 561 len 20480
11344 133/300 Read block 3400 len 24576
11345 134/300 Write block 2192 len 32768 data: 3 3 3 3 4 4 5 4
11346 135/300 Read block 3475 len 24576
11347 136/300 Flush
11348 137/300 Read block 3824 len 28672
11349 138/300 Read block 3796 len 20480
11350 139/300 Write block 2479 len 32768 data: 2 2 3 4 5 4 3 2
11351 140/300 Read block 2284 len 12288
11352 141/300 Read block 1600 len 4096
11353 142/300 Read block 4702 len 8192
11354 143/300 Write block 108 len 24576 data: 4 5 5 5 6 6
11355 144/300 Flush
11356 145/300 Flush
11357 146/300 Read block 1180 len 32768
11358 147/300 Read block 4768 len 24576
11359 148/300 Read block 2391 len 8192
11360 149/300 Read block 1547 len 4096
11361 150/300 Read block 1193 len 24576
11362 151/300 Write block 3720 len 40960 data: 6 5 4 4 3 2 2 3 3 3
11363 152/300 Read block 2432 len 32768
11364 153/300 Write block 2475 len 20480 data: 3 4 3 2 3
11365 154/300 Write block 3118 len 36864 data: 3 3 3 3 4 5 5 5 4
11366 155/300 Read block 842 len 16384
11367 156/300 Read block 2914 len 16384
11368 157/300 Read block 3473 len 20480
11369 158/300 Write block 695 len 20480 data: 3 3 3 3 4
11370 159/300 Read block 4934 len 4096
11371 160/300 Write block 4275 len 32768 data: 5 7 6 4 6 5 6 5
11372 161/300 Write block 1623 len 36864 data: 4 5 5 4 4 4 6 5 6
11373 162/300 Read block 3113 len 28672
11374 163/300 Read block 3171 len 12288
11375 164/300 Write block 4837 len 8192 data: 3 3
11376 165/300 Read block 1013 len 12288
11377 166/300 Read block 1696 len 40960
11378 167/300 Read block 1601 len 20480
11379 168/300 Read block 1882 len 8192
11380 169/300 Read block 575 len 20480
11381 170/300 Write block 3094 len 24576 data: 3 3 4 5 5 5
11382 171/300 Flush
11383 172/300 Read block 3831 len 28672
11384 173/300 Read block 3662 len 4096
11385 174/300 Read block 3919 len 16384
11386 175/300 Write block 2302 len 8192 data: 5 5
11387 176/300 Write block 470 len 8192 data: 4 4
11388 177/300 Flush
11389 178/300 Read block 1011 len 20480
11390 179/300 Read block 2387 len 28672
11391 180/300 Write block 1732 len 4096 data: 6
11392 181/300 Read block 3923 len 36864
11393 182/300 Read block 3005 len 36864
11394 183/300 Read block 2824 len 20480
11395 184/300 Read block 1643 len 8192
11396 185/300 Read block 4376 len 16384
11397 186/300 Read block 4121 len 40960
11398 187/300 Write block 2975 len 24576 data: 3 3 3 2 2 2
11399 188/300 Read block 788 len 28672
11400 189/300 Read block 3169 len 36864
11401 190/300 Write block 460 len 16384 data: 2 2 2 3
11402 191/300 Write block 3629 len 8192 data: 2 2
11403 192/300 Read block 565 len 36864
11404 193/300 Read block 705 len 20480
11405 194/300 Write block 3126 len 16384 data: 5 4 4 2
11406 195/300 Read block 976 len 32768
11407 196/300 Read block 3621 len 28672
11408 197/300 Read block 3313 len 36864
11409 198/300 Write block 2993 len 32768 data: 2 2 2 2 2 2 3 3
11410 199/300 Write block 262 len 4096 data: 3
11411 200/300 Read block 2038 len 16384
11412 201/300 Write block 1829 len 4096 data: 3
11413 202/300 Read block 4803 len 32768
11414 203/300 Write block 933 len 12288 data: 5 5 5
11415 204/300 Write block 4762 len 16384 data: 7 6 5 4
11416 205/300 Write block 1677 len 32768 data: 2 3 3 3 3 3 5 5
11417 206/300 Read block 4260 len 8192
11418 207/300 Write block 2269 len 8192 data: 3 5
11419 208/300 Read block 4576 len 4096
11420 209/300 Read block 1587 len 8192
11421 210/300 Read block 2570 len 28672
11422 211/300 Read block 2834 len 16384
11423 212/300 Read block 3265 len 12288
11424 213/300 Write block 1704 len 40960 data: 2 2 2 2 4 5 5 5 5 5
11425 214/300 Read block 1816 len 28672
11426 215/300 Read block 1680 len 20480
11427 216/300 Read block 4676 len 20480
11428 217/300 Read block 3027 len 28672
11429 218/300 Read block 1455 len 24576
11430 219/300 Write block 891 len 24576 data: 4 4 3 3 3 3
11431 220/300 Write block 3928 len 28672 data: 2 4 4 5 5 5 5
11432 221/300 Write block 3252 len 32768 data: 4 3 4 6 5 4 5 5
11433 222/300 Write block 1400 len 36864 data: 3 5 5 6 5 6 5 5 5
11434 223/300 Read block 1427 len 28672
11435 224/300 Write block 499 len 20480 data: 4 4 7 7 7
11436 225/300 Read block 304 len 8192
11437 226/300 Write block 4609 len 4096 data: 3
11438 227/300 Read block 1190 len 32768
11439 228/300 Write block 4589 len 20480 data: 3 2 2 3 2
11440 229/300 Read block 319 len 32768
11441 230/300 Write block 801 len 28672 data: 6 4 4 4 4 5 5
11442 231/300 Flush
11443 232/300 Write block 2574 len 28672 data: 6 7 7 8 6 5 5
11444 233/300 Write block 3708 len 28672 data: 3 4 5 5 3 4 5
11445 234/300 Write block 3680 len 28672 data: 2 2 3 3 4 5 4
11446 235/300 Write block 1131 len 12288 data: 2 2 2
11447 236/300 Write block 3586 len 28672 data: 2 2 2 4 5 5 4
11448 237/300 Read block 211 len 28672
11449 238/300 Read block 8 len 16384
11450 239/300 Read block 207 len 4096
11451 240/300 Write block 2934 len 4096 data: 3
11452 241/300 Write block 3699 len 36864 data: 4 5 6 6 6 5 5 5 5
11453 242/300 Write block 1862 len 16384 data: 5 5 5 5
11454 243/300 Read block 3219 len 20480
11455 244/300 Write block 4387 len 40960 data: 4 4 4 4 3 2 2 3 3 3
11456 245/300 Read block 2147 len 36864
11457 246/300 Read block 787 len 20480
11458 247/300 Read block 1746 len 28672
11459 248/300 Flush
11460 249/300 Read block 1382 len 36864
11461 250/300 Write block 2633 len 12288 data: 4 4 3
11462 251/300 Write block 1851 len 20480 data: 3 3 3 3 4
11463 252/300 Write block 4680 len 20480 data: 5 4 4 3 4
11464 253/300 Read block 4271 len 28672
11465 254/300 Write block 3442 len 20480 data: 3 4 4 3 4
11466 255/300 Flush
11467 256/300 Read block 4497 len 8192
11468 257/300 Read block 3829 len 36864
11469 258/300 Read block 3932 len 16384
11470 259/300 Write block 1032 len 8192 data: 3 3
11471 260/300 Read block 3145 len 24576
11472 261/300 Write block 1365 len 16384 data: 5 4 3 3
11473 262/300 Write block 1527 len 36864 data: 3 4 3 2 2 2 2 2 2
11474 263/300 Write block 4111 len 12288 data: 2 2 2
11475 264/300 Write block 2911 len 20480 data: 3 3 3 3 3
11476 265/300 Read block 3502 len 8192
11477 266/300 Write block 4968 len 24576 data: 2 3 3 3 3 3
11478 267/300 Flush
11479 268/300 Write block 3849 len 24576 data: 4 5 6 6 6 4
11480 269/300 Read block 592 len 12288
11481 270/300 Write block 322 len 24576 data: 4 4 5 6 5 7
11482 271/300 Read block 2510 len 12288
11483 272/300 Write block 3286 len 32768 data: 5 5 5 4 6 5 5 5
11484 273/300 Read block 2404 len 32768
11485 274/300 Read block 665 len 8192
11486 275/300 Write block 2152 len 16384 data: 3 3 3 2
11487 276/300 Write block 379 len 16384 data: 4 5 5 4
11488 277/300 Read block 4886 len 24576
11489 278/300 Read block 1775 len 12288
11490 279/300 Read block 4771 len 36864
11491 280/300 Write block 4245 len 36864 data: 5 5 6 6 6 5 4 3 2
11492 281/300 Read block 2658 len 12288
11493 282/300 Read block 386 len 8192
11494 283/300 Read block 2073 len 28672
11495 284/300 Read block 3825 len 20480
11496 285/300 Write block 862 len 40960 data: 3 3 3 4 3 3 4 4 4 4
11497 286/300 Flush
11498 287/300 Write block 1291 len 40960 data: 3 3 4 4 4 5 6 6 4 4
11499 288/300 Write block 2394 len 28672 data: 4 4 4 3 3 3 2
11500 289/300 Read block 4982 len 4096
11501 290/300 Read block 3796 len 24576
11502 291/300 Flush
11503 292/300 Write block 3463 len 20480 data: 3 2 3 3 4
11504 293/300 Write block 2155 len 32768 data: 3 2 2 2 2 4 4 5
11505 294/300 Read block 1694 len 28672
11506 295/300 Write block 3034 len 40960 data: 2 2 2 2 2 4 4 3 3 4
11507 296/300 Read block 395 len 36864
11508 297/300 Flush
11509 298/300 Read block 1712 len 4096
11510 299/300 Write block 4254 len 20480 data: 2 2 3 3 5
11511 300/300 Write block 1001 len 12288 data: 4 5 5
11512 Main task has work to do, go find it
11513 got dsc Start(0)
11514 start 0
11515 Sep 22 23:26:19.511 INFO request completed, latency_us: 338, response_code: 204, uri: /start/cid/0, method: POST, req_id: 1e8db8d9-539f-4dfb-bf78-7c3bfce847af, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
11516 Replay: started 0, returned:Ok(())
11517 ----------------------------------------------------------------
11518 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11519 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
11520 3933 Acked 4932 Write 2 New Done Done false
11521 3934 Acked 4933 Write 6 New Done Done false
11522 3935 Acked 4934 Write 7 New Done Done false
11523 3936 Acked 4935 Read 4 New Done Done false
11524 3937 Acked 4936 Write 5 New Done Done false
11525 3938 Acked 4937 Read 5 New Done Done false
11526 3939 Acked 4938 Read 6 New Done Done false
11527 3940 Acked 4939 Read 6 New Done Done false
11528 3941 Acked 4940 Read 9 New Done Done false
11529 3942 Acked 4941 Read 3 New Done Done false
11530 3943 Acked 4942 Read 4 New Done Done false
11531 3944 Acked 4943 Read 5 New Done Done false
11532 3945 Acked 4944 Write 1 New Done Done false
11533 3946 Acked 4945 Read 6 New Done Done false
11534 3947 Acked 4946 Read 3 New Done Done false
11535 3948 Acked 4947 Read 10 New Done Done false
11536 3949 Acked 4948 Write 6 New Done Done false
11537 3950 Acked 4949 Write 5 New Done Done false
11538 3951 Acked 4950 Read 3 New Done Done false
11539 3952 Acked 4951 Flush 0 New Done Done false
11540 3953 Acked 4952 Read 1 New Done Done false
11541 3954 Acked 4953 Write 1 New Done Done false
11542 3955 Acked 4954 Write 5 New Done Done false
11543 3956 Acked 4955 Write 2 New Done Done false
11544 3957 Acked 4956 Read 6 New Done Done false
11545 3958 Acked 4957 Write 1 New Done Done false
11546 3959 Acked 4958 Write 10 New Done Done false
11547 3960 Acked 4959 Read 5 New Done Done false
11548 3961 Acked 4960 Write 4 New Done Done false
11549 3962 Acked 4961 Write 3 New Done Done false
11550 3963 Acked 4962 Read 2 New Done Done false
11551 3964 Acked 4963 Write 3 New Done Done false
11552 3965 Acked 4964 Read 3 New Done Done false
11553 3966 Acked 4965 Write 2 New Done Done false
11554 3967 Acked 4966 Read 6 New Done Done false
11555 3968 Acked 4967 Read 2 New Done Done false
11556 3969 Acked 4968 Write 7 New Done Done false
11557 3970 Acked 4969 Write 5 New Done Done false
11558 3971 Acked 4970 Write 5 New Done Done false
11559 3972 Acked 4971 Read 9 New Done Done false
11560 3973 Acked 4972 Read 9 New Done Done false
11561 3974 Acked 4973 Write 3 New Done Done false
11562 3975 Acked 4974 Read 2 New Done Done false
11563 3976 Acked 4975 Read 10 New Done Done false
11564 3977 Acked 4976 Read 1 New Done Done false
11565 3978 Acked 4977 Write 5 New Done Done false
11566 3979 Acked 4978 Read 2 New Done Done false
11567 3980 Acked 4979 Write 10 New Done Done false
11568 3981 Acked 4980 Read 4 New Done Done false
11569 3982 Acked 4981 Read 3 New Done Done false
11570 3983 Acked 4982 Read 6 New Done Done false
11571 3984 Acked 4983 Read 1 New Done Done false
11572 3985 Acked 4984 Read 8 New Done Done false
11573 3986 Acked 4985 Write 6 New Done Done false
11574 3987 Acked 4986 Read 3 New Done Done false
11575 3988 Acked 4987 Flush 0 New Done Done false
11576 3989 Acked 4988 Write 7 New Done Done false
11577 3990 Acked 4989 Write 10 New Done Done false
11578 3991 Acked 4990 Read 5 New Done Done false
11579 3992 Acked 4991 Write 5 New Done Done false
11580 3993 Acked 4992 Read 6 New Done Done false
11581 3994 Acked 4993 Write 9 New Done Done false
11582 3995 Acked 4994 Read 6 New Done Done false
11583 3996 Acked 4995 Read 10 New Done Done false
11584 3997 Acked 4996 Read 3 New Done Done false
11585 3998 Acked 4997 Read 7 New Done Done false
11586 3999 Acked 4998 Write 2 New Done Done false
11587 4000 Acked 4999 Write 10 New Done Done false
11588 4001 Acked 5000 Read 7 New Done Done false
11589 4002 Acked 5001 Flush 0 New Done Done false
11590 4003 Acked 5002 Read 4 New Done Done false
11591 4004 Acked 5003 Read 7 New Done Done false
11592 4005 Acked 5004 Read 4 New Done Done false
11593 4006 Acked 5005 Read 8 New Done Done false
11594 4007 Acked 5006 Read 10 New Done Done false
11595 4008 Acked 5007 Write 7 New Done Done false
11596 4009 Acked 5008 Read 2 New Done Done false
11597 4010 Acked 5009 Flush 0 New Done Done false
11598 4011 Acked 5010 Read 2 New Done Done false
11599 4012 Acked 5011 Flush 0 New Done Done false
11600 4013 Acked 5012 Write 5 New Done Done false
11601 4014 Acked 5013 Read 1 New Done Done false
11602 4015 Acked 5014 Write 5 New Done Done false
11603 4016 Acked 5015 Write 9 New Done Done false
11604 4017 Acked 5016 Read 5 New Done Done false
11605 4018 Acked 5017 Write 5 New Done Done false
11606 4019 Acked 5018 Flush 0 New Done Done false
11607 4020 Acked 5019 Write 10 New Done Done false
11608 4021 Acked 5020 Read 8 New Done Done false
11609 4022 Acked 5021 Read 2 New Done Done false
11610 4023 Acked 5022 Write 4 New Done Done false
11611 4024 Acked 5023 Read 5 New Done Done false
11612 4025 Acked 5024 Flush 0 New Done Done false
11613 4026 Acked 5025 Write 6 New Done Done false
11614 4027 Acked 5026 Write 7 New Done Done false
11615 4028 Acked 5027 Read 1 New Done Done false
11616 4029 Acked 5028 Flush 0 New Done Done false
11617 4030 Acked 5029 Flush 0 New Done Done false
11618 4031 Acked 5030 Write 4 New Done Done false
11619 4032 Acked 5031 Read 6 New Done Done false
11620 4033 Acked 5032 Write 4 New Done Done false
11621 4034 Acked 5033 Read 2 New Done Done false
11622 4035 Acked 5034 Write 3 New Done Done false
11623 4036 Acked 5035 Write 3 New Done Done false
11624 4037 Acked 5036 Write 2 New Done Done false
11625 4038 Acked 5037 Read 10 New Done Done false
11626 4039 Acked 5038 Write 4 New Done Done false
11627 4040 Acked 5039 Read 6 New Done Done false
11628 4041 Acked 5040 Read 2 New Done Done false
11629 4042 Acked 5041 Read 4 New Done Done false
11630 4043 Acked 5042 Flush 0 New Done Done false
11631 4044 Acked 5043 Read 5 New Done Done false
11632 4045 Acked 5044 Read 9 New Done Done false
11633 4046 Acked 5045 Read 3 New Done Done false
11634 4047 Acked 5046 Flush 0 New Done Done false
11635 4048 Acked 5047 Write 2 New Done Done false
11636 4049 Acked 5048 Read 6 New Done Done false
11637 4050 Acked 5049 Read 2 New Done Done false
11638 4051 Acked 5050 Read 9 New Done Done false
11639 4052 Acked 5051 Read 4 New Done Done false
11640 4053 Acked 5052 Write 3 New Done Done false
11641 4054 Acked 5053 Write 4 New Done Done false
11642 4055 Acked 5054 Read 1 New Done Done false
11643 4056 Acked 5055 Read 3 New Done Done false
11644 4057 Acked 5056 Write 2 New Done Done false
11645 4058 Acked 5057 Read 5 New Done Done false
11646 4059 Acked 5058 Read 7 New Done Done false
11647 4060 Acked 5059 Read 9 New Done Done false
11648 4061 Acked 5060 Write 4 New Done Done false
11649 4062 Acked 5061 Read 10 New Done Done false
11650 4063 Acked 5062 Write 2 New Done Done false
11651 4064 Acked 5063 Read 6 New Done Done false
11652 4065 Acked 5064 Read 5 New Done Done false
11653 4066 Acked 5065 Read 6 New Done Done false
11654 4067 Acked 5066 Write 8 New Done Done false
11655 4068 Acked 5067 Read 6 New Done Done false
11656 4069 Acked 5068 Flush 0 New Done Done false
11657 4070 Acked 5069 Read 7 New Done Done false
11658 4071 Acked 5070 Read 5 New Done Done false
11659 4072 Acked 5071 Write 8 New Done Done false
11660 4073 Acked 5072 Read 3 New Done Done false
11661 4074 Acked 5073 Read 1 New Done Done false
11662 4075 Acked 5074 Read 2 New Done Done false
11663 4076 Acked 5075 Write 6 New Done Done false
11664 4077 Acked 5076 Flush 0 New Done Done false
11665 4078 Acked 5077 Flush 0 New Done Done false
11666 4079 Acked 5078 Read 8 New Done Done false
11667 4080 Acked 5079 Read 6 New Done Done false
11668 4081 Acked 5080 Read 2 New Done Done false
11669 4082 Acked 5081 Read 1 New Done Done false
11670 4083 Acked 5082 Read 6 New Done Done false
11671 4084 Acked 5083 Write 10 New Done Done false
11672 4085 Acked 5084 Read 8 New Done Done false
11673 4086 Acked 5085 Write 5 New Done Done false
11674 4087 Acked 5086 Write 9 New Done Done false
11675 4088 Acked 5087 Read 4 New Done Done false
11676 4089 Acked 5088 Read 4 New Done Done false
11677 4090 Acked 5089 Read 5 New Done Done false
11678 4091 Acked 5090 Write 5 New Done Done false
11679 4092 Acked 5091 Read 1 New Done Done false
11680 4093 Acked 5092 Write 8 New Done Done false
11681 4094 Acked 5093 Write 9 New Done Done false
11682 4095 Acked 5094 Read 7 New Done Done false
11683 4096 Acked 5095 Read 3 New Done Done false
11684 4097 Acked 5096 Write 2 New Done Done false
11685 4098 Acked 5097 Read 3 New Done Done false
11686 4099 Acked 5098 Read 10 New Done Done false
11687 4100 Acked 5099 Read 5 New Done Done false
11688 4101 Acked 5100 Read 2 New Done Done false
11689 4102 Acked 5101 Read 5 New Done Done false
11690 4103 Acked 5102 Write 6 New Done Done false
11691 4104 Acked 5103 Flush 0 New Done Done false
11692 4105 Acked 5104 Read 7 New Done Done false
11693 4106 Acked 5105 Read 1 New Done Done false
11694 4107 Acked 5106 Read 4 New Done Done false
11695 4108 Acked 5107 Write 2 New Done Done false
11696 4109 Acked 5108 Write 2 New Done Done false
11697 4110 Acked 5109 Flush 0 New Done Done false
11698 4111 Acked 5110 Read 5 New Done Done false
11699 4112 Acked 5111 Read 7 New Done Done false
11700 4113 Acked 5112 Write 1 New Done Done false
11701 4114 Acked 5113 Read 9 New Done Done false
11702 4115 Acked 5114 Read 9 New Done Done false
11703 4116 Acked 5115 Read 5 New Done Done false
11704 4117 Acked 5116 Read 2 New Done Done false
11705 4118 Acked 5117 Read 4 New Done Done false
11706 4119 Acked 5118 Read 10 New Done Done false
11707 4120 Acked 5119 Write 6 New Done Done false
11708 4121 Acked 5120 Read 7 New Done Done false
11709 4122 Acked 5121 Read 9 New Done Done false
11710 4123 Acked 5122 Write 4 New Done Done false
11711 4124 Acked 5123 Write 2 New Done Done false
11712 4125 Acked 5124 Read 9 New Done Done false
11713 4126 Acked 5125 Read 5 New Done Done false
11714 4127 Acked 5126 Write 4 New Done Done false
11715 4128 Acked 5127 Read 8 New Done Done false
11716 4129 Acked 5128 Read 7 New Done Done false
11717 4130 Acked 5129 Read 9 New Done Done false
11718 4131 Acked 5130 Write 8 New Done Done false
11719 4132 Acked 5131 Write 1 New Done Done false
11720 4133 Acked 5132 Read 4 New Done Done false
11721 4134 Acked 5133 Write 1 New Done Done false
11722 4135 Acked 5134 Read 8 New Done Done false
11723 4136 Acked 5135 Write 3 New Done Done false
11724 4137 Acked 5136 Write 4 New Done Done false
11725 4138 Acked 5137 Write 8 New Done Done false
11726 4139 Acked 5138 Read 2 New Done Done false
11727 4140 Acked 5139 Write 2 New Done Done false
11728 4141 Acked 5140 Read 1 New Done Done false
11729 4142 Acked 5141 Read 2 New Done Done false
11730 4143 Acked 5142 Read 7 New Done Done false
11731 4144 Acked 5143 Read 4 New Done Done false
11732 4145 Acked 5144 Read 3 New Done Done false
11733 4146 Acked 5145 Write 10 New Done Done false
11734 4147 Acked 5146 Read 7 New Done Done false
11735 4148 Acked 5147 Read 5 New Done Done false
11736 4149 Acked 5148 Read 5 New Done Done false
11737 4150 Acked 5149 Read 7 New Done Done false
11738 4151 Acked 5150 Read 6 New Done Done false
11739 4152 Acked 5151 Write 6 New Done Done false
11740 4153 Acked 5152 Write 7 New Done Done false
11741 4154 Acked 5153 Write 8 New Done Done false
11742 4155 Acked 5154 Write 9 New Done Done false
11743 4156 Acked 5155 Read 7 New Done Done false
11744 4157 Acked 5156 Write 5 New Done Done false
11745 4158 Acked 5157 Read 2 New Done Done false
11746 4159 Acked 5158 Write 1 New Done Done false
11747 4160 Acked 5159 Read 8 New Done Done false
11748 4161 Acked 5160 Write 5 New Done Done false
11749 4162 Acked 5161 Read 8 New Done Done false
11750 4163 Acked 5162 Write 7 New Done Done false
11751 4164 Acked 5163 Flush 0 New Done Done false
11752 4165 Acked 5164 Write 7 New Done Done false
11753 4166 Acked 5165 Write 7 New Done Done false
11754 4167 Acked 5166 Flush 0 New Done Done false
11755 4168 Acked 5167 Write 7 New Done Done false
11756 4169 Acked 5168 Write 3 New Done Done false
11757 4170 Acked 5169 Write 7 New Done Done false
11758 4171 Acked 5170 Read 7 New Done Done false
11759 4172 Acked 5171 Read 4 New Done Done false
11760 4173 Acked 5172 Read 1 New Done Done false
11761 4174 Acked 5173 Write 1 New Done Done false
11762 4175 Acked 5174 Write 9 New Done Done false
11763 4176 Acked 5175 Write 4 New Done Done false
11764 4177 Acked 5176 Read 5 New Done Done false
11765 4178 Acked 5177 Write 10 New Done Done false
11766 4179 Acked 5178 Read 9 New Done Done false
11767 4180 Acked 5179 Read 5 New Done Done false
11768 4181 Acked 5180 Read 7 New Done Done false
11769 4182 Acked 5181 Flush 0 New Done Done false
11770 4183 Acked 5182 Read 9 New Done Done false
11771 4184 Acked 5183 Write 3 New Done Done false
11772 4185 Acked 5184 Write 5 New Done Done false
11773 4186 Acked 5185 Write 5 New Done Done false
11774 4187 Acked 5186 Read 7 New Done Done false
11775 4188 Acked 5187 Write 5 New Done Done false
11776 4189 Acked 5188 Flush 0 New Done Done false
11777 4190 Acked 5189 Read 2 New Done Done false
11778 4191 Acked 5190 Read 9 New Done Done false
11779 4192 Acked 5191 Read 4 New Done Done false
11780 4193 Acked 5192 Write 2 New Done Done false
11781 4194 Acked 5193 Read 6 New Done Done false
11782 4195 Acked 5194 Write 4 New Done Done false
11783 4196 Acked 5195 Write 9 New Done Done false
11784 4197 Acked 5196 Write 3 New Done Done false
11785 4198 Acked 5197 Write 5 New Done Done false
11786 4199 Acked 5198 Read 2 New Done Done false
11787 4200 Acked 5199 Write 6 New Done Done false
11788 4201 Acked 5200 Flush 0 New Done Done false
11789 4202 Acked 5201 Write 6 New Done Done false
11790 4203 Acked 5202 Read 3 New Done Done false
11791 4204 Acked 5203 Write 6 New Done Done false
11792 4205 Acked 5204 Read 3 New Done Done false
11793 4206 Acked 5205 Write 8 New Done Done false
11794 4207 Acked 5206 Read 8 New Done Done false
11795 4208 Acked 5207 Read 2 New Done Done false
11796 4209 Acked 5208 Write 4 New Done Done false
11797 4210 Acked 5209 Write 4 New Done Done false
11798 4211 Acked 5210 Read 6 New Done Done false
11799 4212 Acked 5211 Read 3 New Done Done false
11800 4213 Acked 5212 Read 9 New Done Done false
11801 4214 Acked 5213 Write 9 New Done Done false
11802 4215 Acked 5214 Read 3 New Done Done false
11803 4216 Acked 5215 Read 2 New Done Done false
11804 4217 Acked 5216 Read 7 New Done Done false
11805 4218 Acked 5217 Read 5 New Done Done false
11806 4219 Acked 5218 Write 10 New Done Done false
11807 4220 Acked 5219 Flush 0 New Done Done false
11808 4221 Acked 5220 Write 10 New Done Done false
11809 4222 Acked 5221 Write 7 New Done Done false
11810 4223 Acked 5222 Read 1 New Done Done false
11811 4224 Acked 5223 Read 6 New Done Done false
11812 4225 Acked 5224 Flush 0 New Done Done false
11813 4226 Acked 5225 Write 5 New Done Done false
11814 4227 Acked 5226 Write 8 New Done Done false
11815 4228 Acked 5227 Read 7 New Done Done false
11816 4229 Acked 5228 Write 10 New Done Done false
11817 4230 Acked 5229 Read 9 New Done Done false
11818 4231 Acked 5230 Flush 0 New Done Done false
11819 4232 Acked 5231 Read 1 New Done Done false
11820 4233 Acked 5232 Write 5 New Sent Done false
11821 4234 Acked 5233 Write 3 New Sent Sent false
11822 STATES DS:0 DS:1 DS:2 TOTAL
11823 New 302 0 0 302
11824 Sent 0 2 1 3
11825 Done 0 300 301 601
11826 Skipped 0 0 0 0
11827 Error 0 0 0 0
11828 Last Flush: 4931 5230 5230
11829 Downstairs last five completed: 4931 4930 4929 4928 4927 4926
11830 Upstairs last five completed: 4234 4233 4232 4231 4230 4229
11831 CLIENT: Up:0 ds:302 act:2
118322023-09-22T23:26:20.592ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
11833 [8810] got start action
11834 [8810] I am going to restart
11835 Starting downstairs at port 8810
11836 Make output file at "/tmp/dsc/downstairs-8810.txt"
11837 [8810][0] reports Starting
11838 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1187)
11839 [8810][0] reports Running
118402023-09-22T23:26:21.594ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
118412023-09-22T23:26:21.594ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
118422023-09-22T23:26:21.594ZINFOcrucible: [0] client re-new 303 jobs since flush 4931 = downstairs
118432023-09-22T23:26:21.595ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
118442023-09-22T23:26:21.595ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
118452023-09-22T23:26:21.595ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
118462023-09-22T23:26:21.595ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
118472023-09-22T23:26:21.595ZINFOcrucible: [0] send last flush ID to this DS: 4931
118482023-09-22T23:26:21.595ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
118492023-09-22T23:26:21.596ZINFOcrucible: [0] Replied this last flush ID: 4931
118502023-09-22T23:26:21.596ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
118512023-09-22T23:26:21.596ZINFOcrucible: [0] Transition from Offline to Replay
118522023-09-22T23:26:21.596ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
118532023-09-22T23:26:21.596ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
118542023-09-22T23:26:21.596ZINFOcrucible: [0] Transition from Replay to Active
118552023-09-22T23:26:21.596ZINFOcrucible: [0] Starts cmd_loop
118562023-09-22T23:26:21.596ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
118572023-09-22T23:26:21.596ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
118582023-09-22T23:26:21.596ZINFOcrucible: Set check for repair
118592023-09-22T23:26:22.597ZINFOcrucible: Checking if live repair is needed
118602023-09-22T23:26:22.597ZINFOcrucible: No Live Repair required at this time
118612023-09-22T23:26:22.597ZWARNcrucible: [0] flow control end
11862 ----------------------------------------------------------------
11863 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11864 Downstairs last five completed: 5234 5233 5232 5231 5230 5229
11865 Upstairs last five completed: 4235 4234 4233 4232 4231 4230
11866 CLIENT: Up:0 ds:0 act:3
11867 Replay: All jobs finished, all DS active.
11868 Main task has work to do, go find it
11869 got dsc Stop(0)
11870 stop 0
11871 Sep 22 23:26:23.515 INFO request completed, latency_us: 506[8810] Got stop action so:false kr:false
11872 , response_code: 204, uri: /stop/cid/0, method: POST, req_id: 6e3764ee-2cc4-4cea-9250-dc1f5cb438c0, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
11873 Sep 22 23:26:23.517 INFO request completed, latency_us: 351, response_code: 200, uri: /state/cid/0, method: GET, req_id: e382a7b5-4370-4afa-8ec3-413ec00ffd84, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
118742023-09-22T23:26:23.517ZWARNcrucible: [0] downstairs disconnected
118752023-09-22T23:26:23.517ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
118762023-09-22T23:26:23.517ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
118772023-09-22T23:26:23.517ZWARNcrucible: [0] pm_task rx.recv() is None
118782023-09-22T23:26:23.517ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
118792023-09-22T23:26:23.517ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
118802023-09-22T23:26:23.517ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
11881 [8810] Exited with: ExitStatus(unix_wait_status(9))
11882 [8810][0] reports Exit
11883 Sep 22 23:26:27.520 INFO request completed, latency_us: 292, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2f696771-2e37-4c27-bb55-c6c5c7cbd9af, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
11884 001/300 Write block 3096 len 36864 data: 5 6 6 6 6 6 6 6 5
11885 002/300 Flush
11886 003/300 Write block 1774 len 40960 data: 4 4 3 2 2 3 3 4 3 3
11887 004/300 Read block 2933 len 24576
11888 005/300 Read block 4893 len 20480
11889 006/300 Read block 3234 len 20480
11890 007/300 Write block 3697 len 16384 data: 3 3 5 6
11891 008/300 Write block 176 len 16384 data: 5 5 5 5
11892 009/300 Read block 3909 len 28672
11893 010/300 Write block 3244 len 16384 data: 2 2 2 2
11894 011/300 Read block 2078 len 36864
11895 012/300 Read block 328 len 32768
11896 013/300 Read block 2966 len 40960
11897 014/300 Flush
11898 015/300 Read block 3696 len 40960
11899 016/300 Write block 2849 len 16384 data: 4 4 4 3
11900 017/300 Read block 3595 len 32768
11901 018/300 Write block 3762 len 12288 data: 3 3 2
11902 019/300 Read block 1250 len 16384
11903 020/300 Write block 842 len 32768 data: 5 6 6 5 4 3 3 3
11904 021/300 Write block 4391 len 40960 data: 4 3 3 4 4 4 4 5 5 6
11905 022/300 Read block 4342 len 32768
11906 023/300 Read block 1651 len 20480
11907 024/300 Write block 270 len 28672 data: 3 3 4 4 4 4 4
11908 025/300 Write block 2161 len 16384 data: 5 6 4 4
11909 026/300 Read block 2140 len 24576
11910 027/300 Read block 1938 len 20480
11911 028/300 Write block 2280 len 40960 data: 4 4 4 4 4 4 4 3 3 4
11912 029/300 Write block 931 len 36864 data: 3 4 6 6 6 5 5 4 4
11913 030/300 Write block 707 len 16384 data: 3 3 3 3
11914 031/300 Read block 2115 len 32768
11915 032/300 Write block 769 len 16384 data: 4 5 5 4
11916 033/300 Write block 2338 len 12288 data: 4 4 4
11917 034/300 Write block 1893 len 8192 data: 4 4
11918 035/300 Write block 4645 len 24576 data: 5 5 6 5 4 3
11919 036/300 Write block 1555 len 36864 data: 4 5 5 4 4 3 3 2 2
11920 037/300 Flush
11921 038/300 Read block 1116 len 4096
11922 039/300 Read block 3313 len 8192
11923 040/300 Read block 4560 len 24576
11924 041/300 Read block 3859 len 12288
11925 042/300 Write block 1918 len 36864 data: 4 4 3 3 3 3 3 2 3
11926 043/300 Read block 1916 len 40960
11927 044/300 Write block 2742 len 36864 data: 3 3 2 2 2 2 2 3 3
11928 045/300 Read block 4188 len 36864
11929 046/300 Read block 3807 len 20480
11930 047/300 Flush
11931 048/300 Read block 539 len 16384
11932 049/300 Write block 3114 len 32768 data: 2 2 2 2 4 4 4 4
11933 050/300 Write block 2219 len 24576 data: 5 5 5 3 4 3
11934 051/300 Read block 3230 len 16384
11935 052/300 Write block 2198 len 36864 data: 6 5 4 4 4 4 4 4 3
11936 053/300 Flush
11937 054/300 Read block 2118 len 4096
11938 055/300 Write block 1127 len 24576 data: 2 2 2 2 3 3
11939 056/300 Read block 2467 len 28672
11940 057/300 Write block 2045 len 20480 data: 2 2 2 2 3
11941 058/300 Write block 620 len 36864 data: 3 3 4 4 3 2 3 3 3
11942 059/300 Read block 3072 len 36864
11943 060/300 Read block 736 len 36864
11944 061/300 Write block 3450 len 28672 data: 2 2 2 2 2 2 3
11945 062/300 Read block 1544 len 8192
11946 063/300 Write block 3389 len 8192 data: 4 3
11947 064/300 Read block 412 len 16384
11948 065/300 Write block 3617 len 8192 data: 4 5
11949 066/300 Read block 2481 len 24576
11950 067/300 Write block 3932 len 20480 data: 6 6 6 4 3
11951 068/300 Read block 3134 len 4096
11952 069/300 Read block 3296 len 32768
11953 070/300 Write block 3998 len 12288 data: 6 6 7
11954 071/300 Write block 804 len 28672 data: 5 5 6 6 6 7 7
11955 072/300 Read block 1961 len 32768
11956 073/300 Read block 509 len 28672
11957 074/300 Flush
11958 075/300 Read block 65 len 16384
11959 076/300 Read block 915 len 28672
11960 077/300 Read block 786 len 20480
11961 078/300 Read block 1655 len 16384
11962 079/300 Read block 4778 len 32768
11963 080/300 Read block 3078 len 36864
11964 081/300 Write block 3267 len 4096 data: 4
11965 082/300 Read block 4888 len 4096
11966 083/300 Read block 4435 len 28672
11967 084/300 Read block 1705 len 24576
11968 085/300 Write block 777 len 8192 data: 3 3
11969 086/300 Write block 2894 len 28672 data: 7 6 6 7 7 6 5
11970 087/300 Write block 3049 len 32768 data: 4 4 3 4 5 4 5 4
11971 088/300 Read block 4032 len 12288
11972 089/300 Read block 2189 len 8192
11973 090/300 Write block 2790 len 36864 data: 5 4 4 3 4 4 4 4 5
11974 091/300 Flush
11975 092/300 Write block 2280 len 16384 data: 5 5 5 5
11976 093/300 Read block 1344 len 4096
11977 094/300 Read block 1014 len 32768
11978 095/300 Flush
11979 096/300 Flush
11980 097/300 Read block 2297 len 12288
11981 098/300 Read block 2 len 12288
11982 099/300 Read block 2007 len 32768
11983 100/300 Read block 4524 len 12288
11984 101/300 Read block 2600 len 36864
11985 102/300 Write block 1212 len 32768 data: 5 4 5 4 3 3 3 3
11986 103/300 Read block 87 len 24576
11987 104/300 Write block 3802 len 40960 data: 4 3 3 3 4 4 4 4 4 4
11988 105/300 Read block 2744 len 36864
11989 106/300 Read block 4506 len 32768
11990 107/300 Flush
11991 108/300 Read block 4180 len 28672
11992 109/300 Write block 459 len 16384 data: 2 3 3 3
11993 110/300 Write block 3322 len 16384 data: 3 4 3 4
11994 111/300 Read block 1469 len 32768
11995 112/300 Read block 3304 len 8192
11996 113/300 Read block 4499 len 20480
11997 114/300 Read block 1767 len 20480
11998 115/300 Read block 1150 len 12288
11999 116/300 Flush
12000 117/300 Read block 3982 len 32768
12001 118/300 Read block 4824 len 12288
12002 119/300 Read block 4077 len 32768
12003 120/300 Write block 1630 len 4096 data: 6
12004 121/300 Write block 3477 len 28672 data: 3 4 4 4 3 4 3
12005 122/300 Write block 3006 len 16384 data: 4 4 4 4
12006 123/300 Write block 1960 len 36864 data: 3 4 4 5 4 4 4 4 4
12007 124/300 Read block 1507 len 4096
12008 125/300 Write block 2476 len 24576 data: 5 4 3 4 3 4
12009 126/300 Flush
12010 127/300 Read block 2734 len 28672
12011 128/300 Read block 34 len 8192
12012 129/300 Write block 870 len 32768 data: 5 5 4 5 5 5 6 5
12013 130/300 Read block 4883 len 12288
12014 131/300 Write block 1292 len 28672 data: 4 5 5 5 6 7 7
12015 132/300 Write block 3929 len 16384 data: 5 5 6 7
12016 133/300 Read block 3799 len 28672
12017 134/300 Read block 3667 len 12288
12018 135/300 Read block 2896 len 40960
12019 136/300 Write block 3936 len 28672 data: 4 3 3 3 2 2 2
12020 137/300 Read block 4638 len 36864
12021 138/300 Read block 1203 len 36864
12022 139/300 Write block 4162 len 24576 data: 3 3 3 2 2 4
12023 140/300 Read block 2723 len 32768
12024 141/300 Write block 984 len 12288 data: 3 4 5
12025 142/300 Read block 2957 len 36864
12026 143/300 Read block 2745 len 32768
12027 144/300 Read block 258 len 8192
12028 145/300 Write block 4055 len 32768 data: 3 4 4 5 5 5 6 6
12029 146/300 Write block 4831 len 8192 data: 2 2
12030 147/300 Read block 4392 len 20480
12031 148/300 Write block 2177 len 40960 data: 3 2 3 3 3 5 4 4 3 3
12032 149/300 Write block 2598 len 16384 data: 4 4 3 3
12033 150/300 Read block 4678 len 36864
12034 151/300 Read block 1362 len 12288
12035 152/300 Write block 2876 len 28672 data: 5 7 7 7 5 3 3
12036 153/300 Write block 3283 len 28672 data: 3 3 5 6 6 6 5
12037 154/300 Write block 1313 len 20480 data: 5 4 4 7 4
12038 155/300 Read block 2138 len 8192
12039 156/300 Read block 4822 len 4096
12040 157/300 Read block 3504 len 4096
12041 158/300 Read block 4478 len 32768
12042 159/300 Write block 2312 len 28672 data: 5 4 4 3 3 3 3
12043 160/300 Write block 1448 len 4096 data: 5
12044 161/300 Read block 1443 len 36864
12045 162/300 Read block 4553 len 32768
12046 163/300 Read block 1874 len 16384
12047 164/300 Read block 1609 len 28672
12048 165/300 Read block 3908 len 36864
12049 166/300 Read block 3084 len 36864
12050 167/300 Read block 2178 len 28672
12051 168/300 Read block 3772 len 28672
12052 169/300 Read block 3384 len 40960
12053 170/300 Read block 4786 len 16384
12054 171/300 Write block 413 len 28672 data: 3 3 4 4 4 5 5
12055 172/300 Read block 2778 len 24576
12056 173/300 Read block 1147 len 8192
12057 174/300 Read block 4106 len 8192
12058 175/300 Read block 370 len 32768
12059 176/300 Flush
12060 177/300 Read block 4275 len 12288
12061 178/300 Write block 3501 len 20480 data: 3 3 3 3 2
12062 179/300 Write block 4940 len 28672 data: 5 5 6 5 4 3 3
12063 180/300 Read block 2731 len 40960
12064 181/300 Read block 3975 len 24576
12065 182/300 Read block 4903 len 32768
12066 183/300 Flush
12067 184/300 Read block 4699 len 20480
12068 185/300 Read block 4265 len 24576
12069 186/300 Read block 4377 len 4096
12070 187/300 Write block 1268 len 36864 data: 6 6 6 6 5 5 5 3 3
12071 188/300 Read block 908 len 4096
12072 189/300 Read block 3110 len 12288
12073 190/300 Write block 1966 len 8192 data: 5 5
12074 191/300 Flush
12075 192/300 Flush
12076 193/300 Read block 3990 len 28672
12077 194/300 Read block 2331 len 40960
12078 195/300 Read block 717 len 16384
12079 196/300 Read block 4831 len 36864
12080 197/300 Read block 3980 len 40960
12081 198/300 Flush
12082 199/300 Read block 541 len 16384
12083 200/300 Write block 2244 len 20480 data: 3 2 2 3 4
12084 201/300 Read block 4529 len 24576
12085 202/300 Write block 1873 len 20480 data: 3 3 3 2 3
12086 203/300 Write block 3834 len 16384 data: 4 5 4 5
12087 204/300 Write block 755 len 20480 data: 3 4 5 5 6
12088 205/300 Read block 2510 len 20480
12089 206/300 Write block 2587 len 4096 data: 6
12090 207/300 Write block 2935 len 32768 data: 3 2 2 2 2 2 2 3
12091 208/300 Read block 3916 len 20480
12092 209/300 Read block 4923 len 4096
12093 210/300 Write block 2131 len 20480 data: 2 2 2 2 2
12094 211/300 Read block 3979 len 12288
12095 212/300 Read block 3037 len 16384
12096 213/300 Read block 941 len 8192
12097 214/300 Read block 4080 len 40960
12098 215/300 Read block 1877 len 40960
12099 216/300 Write block 4985 len 32768 data: 5 5 6 7 6 5 5 5
12100 217/300 Read block 980 len 28672
12101 218/300 Read block 4830 len 12288
12102 219/300 Read block 1750 len 28672
12103 220/300 Write block 3892 len 28672 data: 5 5 4 5 5 6 7
12104 221/300 Read block 3616 len 40960
12105 222/300 Read block 2916 len 20480
12106 223/300 Write block 596 len 24576 data: 5 6 6 6 5 4
12107 224/300 Read block 3982 len 16384
12108 225/300 Write block 1323 len 24576 data: 4 4 3 4 5 5
12109 226/300 Read block 2906 len 40960
12110 227/300 Write block 1735 len 24576 data: 6 5 5 4 4 4
12111 228/300 Flush
12112 229/300 Read block 1220 len 4096
12113 230/300 Write block 2541 len 28672 data: 3 3 3 3 3 3 3
12114 231/300 Write block 2103 len 36864 data: 2 2 4 5 5 5 4 7 6
12115 232/300 Read block 2286 len 40960
12116 233/300 Read block 4162 len 32768
12117 234/300 Write block 1771 len 36864 data: 7 6 5 5 5 4 3 3 4
12118 235/300 Read block 3008 len 40960
12119 236/300 Read block 2848 len 4096
12120 237/300 Read block 2847 len 20480
12121 238/300 Read block 3188 len 12288
12122 239/300 Write block 3653 len 16384 data: 4 5 5 5
12123 240/300 Read block 3462 len 28672
12124 241/300 Read block 4628 len 28672
12125 242/300 Read block 1831 len 24576
12126 243/300 Write block 4732 len 20480 data: 5 5 4 5 4
12127 244/300 Read block 1010 len 12288
12128 245/300 Read block 1818 len 16384
12129 246/300 Write block 4451 len 32768 data: 4 4 3 4 4 5 5 4
12130 247/300 Write block 4019 len 24576 data: 3 3 3 3 3 3
12131 248/300 Write block 4771 len 28672 data: 4 3 3 3 3 3 3
12132 249/300 Read block 1684 len 4096
12133 250/300 Write block 4333 len 8192 data: 6 5
12134 251/300 Flush
12135 252/300 Write block 482 len 12288 data: 4 5 6
12136 253/300 Flush
12137 254/300 Read block 176 len 24576
12138 255/300 Write block 1806 len 32768 data: 2 3 3 3 4 4 4 3
12139 256/300 Write block 3341 len 24576 data: 5 5 4 4 4 4
12140 257/300 Write block 388 len 32768 data: 4 4 4 4 5 4 3 3
12141 258/300 Read block 4708 len 36864
12142 259/300 Read block 3290 len 24576
12143 260/300 Read block 4611 len 36864
12144 261/300 Read block 97 len 12288
12145 262/300 Write block 1249 len 16384 data: 2 3 3 3
12146 263/300 Read block 4052 len 28672
12147 264/300 Write block 3725 len 24576 data: 3 3 4 4 4 3
12148 265/300 Read block 2660 len 32768
12149 266/300 Read block 2756 len 28672
12150 267/300 Read block 2715 len 32768
12151 268/300 Read block 3376 len 20480
12152 269/300 Write block 65 len 12288 data: 5 4 4
12153 270/300 Read block 1732 len 40960
12154 271/300 Read block 2559 len 4096
12155 272/300 Read block 523 len 24576
12156 273/300 Read block 3026 len 20480
12157 274/300 Read block 3123 len 4096
12158 275/300 Read block 731 len 28672
12159 276/300 Write block 3480 len 8192 data: 5 4
12160 277/300 Read block 3096 len 28672
12161 278/300 Read block 3050 len 36864
12162 279/300 Write block 4825 len 36864 data: 5 6 6 5 4 2 3 3 2
12163 280/300 Write block 1484 len 40960 data: 3 3 2 3 3 3 3 3 2 2
12164 281/300 Write block 2163 len 4096 data: 5
12165 282/300 Write block 625 len 24576 data: 3 4 4 4 3 2
12166 283/300 Write block 1051 len 36864 data: 5 4 4 2 2 2 3 3 3
12167 284/300 Write block 129 len 36864 data: 4 4 4 4 3 3 3 3 3
12168 285/300 Read block 3324 len 24576
12169 286/300 Read block 1163 len 36864
12170 287/300 Read block 3442 len 12288
12171 288/300 Read block 3268 len 12288
12172 289/300 Write block 1561 len 4096 data: 4
12173 290/300 Read block 2708 len 40960
12174 291/300 Read block 3980 len 16384
12175 292/300 Write block 3310 len 8192 data: 4 4
12176 293/300 Flush
12177 294/300 Write block 1336 len 4096 data: 4
12178 295/300 Read block 3915 len 16384
12179 296/300 Write block 3462 len 16384 data: 3 4 3 4
12180 297/300 Write block 4165 len 32768 data: 3 3 5 4 4 4 4 5
12181 298/300 Read block 1613 len 28672
12182 299/300 Write block 2719 len 28672 data: 5 5 4 4 5 3 4
12183 300/300 Read block 3510 len 12288
12184 Main task has work to do, go find it
12185 got dsc Start(0)
12186 start 0
12187 Sep 22 23:26:28.527 INFO request completed, latency_us: 304, response_code: 204, uri: /start/cid/0, method: POST, req_id: ba0f03e7-56b5-4535-8a33-ea093f159b5b, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
12188 Replay: started 0, returned:Ok(())
12189 ----------------------------------------------------------------
12190 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12191 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
12192 4236 Acked 5235 Write 9 New Done Done false
12193 4237 Acked 5236 Flush 0 New Done Done false
12194 4238 Acked 5237 Write 10 New Done Done false
12195 4239 Acked 5238 Read 6 New Done Done false
12196 4240 Acked 5239 Read 5 New Done Done false
12197 4241 Acked 5240 Read 5 New Done Done false
12198 4242 Acked 5241 Write 4 New Done Done false
12199 4243 Acked 5242 Write 4 New Done Done false
12200 4244 Acked 5243 Read 7 New Done Done false
12201 4245 Acked 5244 Write 4 New Done Done false
12202 4246 Acked 5245 Read 9 New Done Done false
12203 4247 Acked 5246 Read 8 New Done Done false
12204 4248 Acked 5247 Read 10 New Done Done false
12205 4249 Acked 5248 Flush 0 New Done Done false
12206 4250 Acked 5249 Read 10 New Done Done false
12207 4251 Acked 5250 Write 4 New Done Done false
12208 4252 Acked 5251 Read 8 New Done Done false
12209 4253 Acked 5252 Write 3 New Done Done false
12210 4254 Acked 5253 Read 4 New Done Done false
12211 4255 Acked 5254 Write 8 New Done Done false
12212 4256 Acked 5255 Write 10 New Done Done false
12213 4257 Acked 5256 Read 8 New Done Done false
12214 4258 Acked 5257 Read 5 New Done Done false
12215 4259 Acked 5258 Write 7 New Done Done false
12216 4260 Acked 5259 Write 4 New Done Done false
12217 4261 Acked 5260 Read 6 New Done Done false
12218 4262 Acked 5261 Read 5 New Done Done false
12219 4263 Acked 5262 Write 10 New Done Done false
12220 4264 Acked 5263 Write 9 New Done Done false
12221 4265 Acked 5264 Write 4 New Done Done false
12222 4266 Acked 5265 Read 8 New Done Done false
12223 4267 Acked 5266 Write 4 New Done Done false
12224 4268 Acked 5267 Write 3 New Done Done false
12225 4269 Acked 5268 Write 2 New Done Done false
12226 4270 Acked 5269 Write 6 New Done Done false
12227 4271 Acked 5270 Write 9 New Done Done false
12228 4272 Acked 5271 Flush 0 New Done Done false
12229 4273 Acked 5272 Read 1 New Done Done false
12230 4274 Acked 5273 Read 2 New Done Done false
12231 4275 Acked 5274 Read 6 New Done Done false
12232 4276 Acked 5275 Read 3 New Done Done false
12233 4277 Acked 5276 Write 9 New Done Done false
12234 4278 Acked 5277 Read 10 New Done Done false
12235 4279 Acked 5278 Write 9 New Done Done false
12236 4280 Acked 5279 Read 9 New Done Done false
12237 4281 Acked 5280 Read 5 New Done Done false
12238 4282 Acked 5281 Flush 0 New Done Done false
12239 4283 Acked 5282 Read 4 New Done Done false
12240 4284 Acked 5283 Write 8 New Done Done false
12241 4285 Acked 5284 Write 6 New Done Done false
12242 4286 Acked 5285 Read 4 New Done Done false
12243 4287 Acked 5286 Write 9 New Done Done false
12244 4288 Acked 5287 Flush 0 New Done Done false
12245 4289 Acked 5288 Read 1 New Done Done false
12246 4290 Acked 5289 Write 6 New Done Done false
12247 4291 Acked 5290 Read 7 New Done Done false
12248 4292 Acked 5291 Write 5 New Done Done false
12249 4293 Acked 5292 Write 9 New Done Done false
12250 4294 Acked 5293 Read 9 New Done Done false
12251 4295 Acked 5294 Read 9 New Done Done false
12252 4296 Acked 5295 Write 7 New Done Done false
12253 4297 Acked 5296 Read 2 New Done Done false
12254 4298 Acked 5297 Write 2 New Done Done false
12255 4299 Acked 5298 Read 4 New Done Done false
12256 4300 Acked 5299 Write 2 New Done Done false
12257 4301 Acked 5300 Read 6 New Done Done false
12258 4302 Acked 5301 Write 5 New Done Done false
12259 4303 Acked 5302 Read 1 New Done Done false
12260 4304 Acked 5303 Read 8 New Done Done false
12261 4305 Acked 5304 Write 3 New Done Done false
12262 4306 Acked 5305 Write 7 New Done Done false
12263 4307 Acked 5306 Read 8 New Done Done false
12264 4308 Acked 5307 Read 7 New Done Done false
12265 4309 Acked 5308 Flush 0 New Done Done false
12266 4310 Acked 5309 Read 4 New Done Done false
12267 4311 Acked 5310 Read 7 New Done Done false
12268 4312 Acked 5311 Read 5 New Done Done false
12269 4313 Acked 5312 Read 4 New Done Done false
12270 4314 Acked 5313 Read 8 New Done Done false
12271 4315 Acked 5314 Read 9 New Done Done false
12272 4316 Acked 5315 Write 1 New Done Done false
12273 4317 Acked 5316 Read 1 New Done Done false
12274 4318 Acked 5317 Read 7 New Done Done false
12275 4319 Acked 5318 Read 6 New Done Done false
12276 4320 Acked 5319 Write 2 New Done Done false
12277 4321 Acked 5320 Write 7 New Done Done false
12278 4322 Acked 5321 Write 8 New Done Done false
12279 4323 Acked 5322 Read 3 New Done Done false
12280 4324 Acked 5323 Read 2 New Done Done false
12281 4325 Acked 5324 Flush 0 New Done Done false
12282 4326 Acked 5325 Write 9 New Done Done false
12283 4327 Acked 5326 Flush 0 New Done Done false
12284 4328 Acked 5327 Write 4 New Done Done false
12285 4329 Acked 5328 Read 1 New Done Done false
12286 4330 Acked 5329 Read 8 New Done Done false
12287 4331 Acked 5330 Flush 0 New Done Done false
12288 4332 Acked 5331 Flush 0 New Done Done false
12289 4333 Acked 5332 Read 3 New Done Done false
12290 4334 Acked 5333 Read 3 New Done Done false
12291 4335 Acked 5334 Read 8 New Done Done false
12292 4336 Acked 5335 Read 3 New Done Done false
12293 4337 Acked 5336 Read 9 New Done Done false
12294 4338 Acked 5337 Write 8 New Done Done false
12295 4339 Acked 5338 Read 6 New Done Done false
12296 4340 Acked 5339 Write 10 New Done Done false
12297 4341 Acked 5340 Read 9 New Done Done false
12298 4342 Acked 5341 Read 8 New Done Done false
12299 4343 Acked 5342 Flush 0 New Done Done false
12300 4344 Acked 5343 Read 7 New Done Done false
12301 4345 Acked 5344 Write 4 New Done Done false
12302 4346 Acked 5345 Write 4 New Done Done false
12303 4347 Acked 5346 Read 8 New Done Done false
12304 4348 Acked 5347 Read 2 New Done Done false
12305 4349 Acked 5348 Read 5 New Done Done false
12306 4350 Acked 5349 Read 5 New Done Done false
12307 4351 Acked 5350 Read 3 New Done Done false
12308 4352 Acked 5351 Flush 0 New Done Done false
12309 4353 Acked 5352 Read 8 New Done Done false
12310 4354 Acked 5353 Read 3 New Done Done false
12311 4355 Acked 5354 Read 8 New Done Done false
12312 4356 Acked 5355 Write 1 New Done Done false
12313 4357 Acked 5356 Write 7 New Done Done false
12314 4358 Acked 5357 Write 4 New Done Done false
12315 4359 Acked 5358 Write 9 New Done Done false
12316 4360 Acked 5359 Read 1 New Done Done false
12317 4361 Acked 5360 Write 6 New Done Done false
12318 4362 Acked 5361 Flush 0 New Done Done false
12319 4363 Acked 5362 Read 7 New Done Done false
12320 4364 Acked 5363 Read 2 New Done Done false
12321 4365 Acked 5364 Write 8 New Done Done false
12322 4366 Acked 5365 Read 3 New Done Done false
12323 4367 Acked 5366 Write 7 New Done Done false
12324 4368 Acked 5367 Write 4 New Done Done false
12325 4369 Acked 5368 Read 7 New Done Done false
12326 4370 Acked 5369 Read 3 New Done Done false
12327 4371 Acked 5370 Read 10 New Done Done false
12328 4372 Acked 5371 Write 7 New Done Done false
12329 4373 Acked 5372 Read 9 New Done Done false
12330 4374 Acked 5373 Read 9 New Done Done false
12331 4375 Acked 5374 Write 6 New Done Done false
12332 4376 Acked 5375 Read 8 New Done Done false
12333 4377 Acked 5376 Write 3 New Done Done false
12334 4378 Acked 5377 Read 9 New Done Done false
12335 4379 Acked 5378 Read 8 New Done Done false
12336 4380 Acked 5379 Read 2 New Done Done false
12337 4381 Acked 5380 Write 8 New Done Done false
12338 4382 Acked 5381 Write 2 New Done Done false
12339 4383 Acked 5382 Read 5 New Done Done false
12340 4384 Acked 5383 Write 10 New Done Done false
12341 4385 Acked 5384 Write 4 New Done Done false
12342 4386 Acked 5385 Read 9 New Done Done false
12343 4387 Acked 5386 Read 3 New Done Done false
12344 4388 Acked 5387 Write 7 New Done Done false
12345 4389 Acked 5388 Write 7 New Done Done false
12346 4390 Acked 5389 Write 5 New Done Done false
12347 4391 Acked 5390 Read 2 New Done Done false
12348 4392 Acked 5391 Read 1 New Done Done false
12349 4393 Acked 5392 Read 1 New Done Done false
12350 4394 Acked 5393 Read 8 New Done Done false
12351 4395 Acked 5394 Write 7 New Done Done false
12352 4396 Acked 5395 Write 1 New Done Done false
12353 4397 Acked 5396 Read 9 New Done Done false
12354 4398 Acked 5397 Read 8 New Done Done false
12355 4399 Acked 5398 Read 4 New Done Done false
12356 4400 Acked 5399 Read 7 New Done Done false
12357 4401 Acked 5400 Read 9 New Done Done false
12358 4402 Acked 5401 Read 9 New Done Done false
12359 4403 Acked 5402 Read 7 New Done Done false
12360 4404 Acked 5403 Read 7 New Done Done false
12361 4405 Acked 5404 Read 10 New Done Done false
12362 4406 Acked 5405 Read 4 New Done Done false
12363 4407 Acked 5406 Write 7 New Done Done false
12364 4408 Acked 5407 Read 6 New Done Done false
12365 4409 Acked 5408 Read 2 New Done Done false
12366 4410 Acked 5409 Read 2 New Done Done false
12367 4411 Acked 5410 Read 8 New Done Done false
12368 4412 Acked 5411 Flush 0 New Done Done false
12369 4413 Acked 5412 Read 3 New Done Done false
12370 4414 Acked 5413 Write 5 New Done Done false
12371 4415 Acked 5414 Write 7 New Done Done false
12372 4416 Acked 5415 Read 10 New Done Done false
12373 4417 Acked 5416 Read 6 New Done Done false
12374 4418 Acked 5417 Read 8 New Done Done false
12375 4419 Acked 5418 Flush 0 New Done Done false
12376 4420 Acked 5419 Read 5 New Done Done false
12377 4421 Acked 5420 Read 6 New Done Done false
12378 4422 Acked 5421 Read 1 New Done Done false
12379 4423 Acked 5422 Write 9 New Done Done false
12380 4424 Acked 5423 Read 1 New Done Done false
12381 4425 Acked 5424 Read 3 New Done Done false
12382 4426 Acked 5425 Write 2 New Done Done false
12383 4427 Acked 5426 Flush 0 New Done Done false
12384 4428 Acked 5427 Flush 0 New Done Done false
12385 4429 Acked 5428 Read 7 New Done Done false
12386 4430 Acked 5429 Read 10 New Done Done false
12387 4431 Acked 5430 Read 4 New Done Done false
12388 4432 Acked 5431 Read 9 New Done Done false
12389 4433 Acked 5432 Read 10 New Done Done false
12390 4434 Acked 5433 Flush 0 New Done Done false
12391 4435 Acked 5434 Read 4 New Done Done false
12392 4436 Acked 5435 Write 5 New Done Done false
12393 4437 Acked 5436 Read 6 New Done Done false
12394 4438 Acked 5437 Write 5 New Done Done false
12395 4439 Acked 5438 Write 4 New Done Done false
12396 4440 Acked 5439 Write 5 New Done Done false
12397 4441 Acked 5440 Read 5 New Done Done false
12398 4442 Acked 5441 Write 1 New Done Done false
12399 4443 Acked 5442 Write 8 New Done Done false
12400 4444 Acked 5443 Read 5 New Done Done false
12401 4445 Acked 5444 Read 1 New Done Done false
12402 4446 Acked 5445 Write 5 New Done Done false
12403 4447 Acked 5446 Read 3 New Done Done false
12404 4448 Acked 5447 Read 4 New Done Done false
12405 4449 Acked 5448 Read 2 New Done Done false
12406 4450 Acked 5449 Read 10 New Done Done false
12407 4451 Acked 5450 Read 10 New Done Done false
12408 4452 Acked 5451 Write 8 New Done Done false
12409 4453 Acked 5452 Read 7 New Done Done false
12410 4454 Acked 5453 Read 3 New Done Done false
12411 4455 Acked 5454 Read 7 New Done Done false
12412 4456 Acked 5455 Write 7 New Done Done false
12413 4457 Acked 5456 Read 10 New Done Done false
12414 4458 Acked 5457 Read 5 New Done Done false
12415 4459 Acked 5458 Write 6 New Done Done false
12416 4460 Acked 5459 Read 4 New Done Done false
12417 4461 Acked 5460 Write 6 New Done Done false
12418 4462 Acked 5461 Read 10 New Done Done false
12419 4463 Acked 5462 Write 6 New Done Done false
12420 4464 Acked 5463 Flush 0 New Done Done false
12421 4465 Acked 5464 Read 1 New Done Done false
12422 4466 Acked 5465 Write 7 New Done Done false
12423 4467 Acked 5466 Write 9 New Done Done false
12424 4468 Acked 5467 Read 10 New Done Done false
12425 4469 Acked 5468 Read 8 New Done Done false
12426 4470 Acked 5469 Write 9 New Done Done false
12427 4471 Acked 5470 Read 10 New Done Done false
12428 4472 Acked 5471 Read 1 New Done Done false
12429 4473 Acked 5472 Read 5 New Done Done false
12430 4474 Acked 5473 Read 3 New Done Done false
12431 4475 Acked 5474 Write 4 New Done Done false
12432 4476 Acked 5475 Read 7 New Done Done false
12433 4477 Acked 5476 Read 7 New Done Done false
12434 4478 Acked 5477 Read 6 New Done Done false
12435 4479 Acked 5478 Write 5 New Done Done false
12436 4480 Acked 5479 Read 3 New Done Done false
12437 4481 Acked 5480 Read 4 New Done Done false
12438 4482 Acked 5481 Flush 0 New Done Done false
12439 4483 Acked 5482 Write 8 New Done Done false
12440 4484 Acked 5483 Write 6 New Done Done false
12441 4485 Acked 5484 Write 7 New Done Done false
12442 4486 Acked 5485 Read 1 New Done Done false
12443 4487 Acked 5486 Write 2 New Done Done false
12444 4488 Acked 5487 Flush 0 New Done Done false
12445 4489 Acked 5488 Write 3 New Done Done false
12446 4490 Acked 5489 Flush 0 New Done Done false
12447 4491 Acked 5490 Read 6 New Done Done false
12448 4492 Acked 5491 Write 8 New Done Done false
12449 4493 Acked 5492 Write 6 New Done Done false
12450 4494 Acked 5493 Write 8 New Done Done false
12451 4495 Acked 5494 Read 9 New Done Done false
12452 4496 Acked 5495 Read 6 New Done Done false
12453 4497 Acked 5496 Read 9 New Done Done false
12454 4498 Acked 5497 Read 3 New Done Done false
12455 4499 Acked 5498 Write 4 New Done Done false
12456 4500 Acked 5499 Read 7 New Done Done false
12457 4501 Acked 5500 Write 6 New Done Done false
12458 4502 Acked 5501 Read 8 New Done Done false
12459 4503 Acked 5502 Read 7 New Done Done false
12460 4504 Acked 5503 Read 8 New Done Done false
12461 4505 Acked 5504 Read 5 New Done Done false
12462 4506 Acked 5505 Write 3 New Done Done false
12463 4507 Acked 5506 Read 10 New Done Done false
12464 4508 Acked 5507 Read 1 New Done Done false
12465 4509 Acked 5508 Read 6 New Done Done false
12466 4510 Acked 5509 Read 5 New Done Done false
12467 4511 Acked 5510 Read 1 New Done Done false
12468 4512 Acked 5511 Read 7 New Done Done false
12469 4513 Acked 5512 Write 2 New Done Done false
12470 4514 Acked 5513 Read 7 New Done Done false
12471 4515 Acked 5514 Read 9 New Done Done false
12472 4516 Acked 5515 Write 9 New Done Done false
12473 4517 Acked 5516 Write 10 New Done Done false
12474 4518 Acked 5517 Write 1 New Done Done false
12475 4519 Acked 5518 Write 6 New Done Done false
12476 4520 Acked 5519 Write 9 New Done Done false
12477 4521 Acked 5520 Write 9 New Done Done false
12478 4522 Acked 5521 Read 6 New Done Done false
12479 4523 Acked 5522 Read 9 New Done Done false
12480 4524 Acked 5523 Read 3 New Done Done false
12481 4525 Acked 5524 Read 3 New Done Done false
12482 4526 Acked 5525 Write 1 New Done Done false
12483 4527 Acked 5526 Read 10 New Done Done false
12484 4528 Acked 5527 Read 4 New Done Done false
12485 4529 Acked 5528 Write 2 New Done Done false
12486 4530 Acked 5529 Flush 0 New Done Done false
12487 4531 Acked 5530 Write 1 New Done Done false
12488 4532 Acked 5531 Read 4 New Done Done false
12489 4533 Acked 5532 Write 4 New Done Done false
12490 4534 Acked 5533 Write 8 New Done Done false
12491 4535 Acked 5534 Read 7 New Done Done false
12492 4536 Acked 5535 Write 7 New Done Done false
12493 4537 Acked 5536 Read 3 New Done Done false
12494 STATES DS:0 DS:1 DS:2 TOTAL
12495 New 302 0 0 302
12496 Sent 0 0 0 0
12497 Done 0 302 302 604
12498 Skipped 0 0 0 0
12499 Error 0 0 0 0
12500 Last Flush: 5234 5529 5529
12501 Downstairs last five completed: 5234 5233 5232 5231 5230 5229
12502 Upstairs last five completed: 4537 4536 4535 4534 4533 4532
12503 CLIENT: Up:0 ds:302 act:2
12504 [8810] got start action
12505 [8810] I am going to restart
12506 Starting downstairs at port 8810
12507 Make output file at "/tmp/dsc/downstairs-8810.txt"
12508 [8810][0] reports Starting
12509 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1188)
12510 [8810][0] reports Running
125112023-09-22T23:26:30.528ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
125122023-09-22T23:26:30.528ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
125132023-09-22T23:26:30.528ZINFOcrucible: [0] client re-new 303 jobs since flush 5234 = downstairs
125142023-09-22T23:26:30.529ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
125152023-09-22T23:26:30.529ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
125162023-09-22T23:26:30.529ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
125172023-09-22T23:26:30.529ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
125182023-09-22T23:26:30.529ZINFOcrucible: [0] send last flush ID to this DS: 5234
125192023-09-22T23:26:30.530ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
125202023-09-22T23:26:30.530ZINFOcrucible: [0] Replied this last flush ID: 5234
125212023-09-22T23:26:30.530ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
125222023-09-22T23:26:30.530ZINFOcrucible: [0] Transition from Offline to Replay
125232023-09-22T23:26:30.530ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
125242023-09-22T23:26:30.530ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
125252023-09-22T23:26:30.530ZINFOcrucible: [0] Transition from Replay to Active
125262023-09-22T23:26:30.530ZINFOcrucible: [0] Starts cmd_loop
125272023-09-22T23:26:30.530ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
125282023-09-22T23:26:30.530ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
125292023-09-22T23:26:30.530ZINFOcrucible: Set check for repair
125302023-09-22T23:26:31.530ZWARNcrucible: [0] flow control end
125312023-09-22T23:26:31.532ZINFOcrucible: Checking if live repair is needed
125322023-09-22T23:26:31.532ZINFOcrucible: No Live Repair required at this time
12533 ----------------------------------------------------------------
12534 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12535 Downstairs last five completed: 5537 5536 5535 5534 5533 5532
12536 Upstairs last five completed: 4538 4537 4536 4535 4534 4533
12537 CLIENT: Up:0 ds:0 act:3
12538 Replay: All jobs finished, all DS active.
12539 Main task has work to do, go find it
12540 got dsc Stop(1)
12541 stop 1
12542 Sep 22 23:26:32.531 INFO request completed, latency_us: 516, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 607a406e-aa1d-4ed1-a6e2-2f8bbe12b54a, remote_addr: 127.0[8820] Got stop action so:false kr:false
12543 .0.1:52471, local_addr: 127.0.0.1:9998
12544 Sep 22 23:26:32.532 INFO request completed, latency_us: 272, response_code: 200, uri: /state/cid/1, method: GET, req_id: 084c446b-64f5-4d7e-bb54-cb947a254440, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
125452023-09-22T23:26:32.533ZWARNcrucible: [1] downstairs disconnected
125462023-09-22T23:26:32.533ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
125472023-09-22T23:26:32.533ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
125482023-09-22T23:26:32.533ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
125492023-09-22T23:26:32.533ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
125502023-09-22T23:26:32.533ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
125512023-09-22T23:26:32.533ZWARNcrucible: [1] pm_task rx.recv() is None
12552 [8820] Exited with: ExitStatus(unix_wait_status(9))
12553 [8820][1] reports Exit
12554 Sep 22 23:26:36.534 INFO request completed, latency_us: 264, response_code: 200, uri: /state/cid/1, method: GET, req_id: 981da9d9-b587-4c6c-b962-51de15b0403b, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
12555 001/300 Read block 824 len 40960
12556 002/300 Read block 1005 len 32768
12557 003/300 Write block 4822 len 36864 data: 6 6 6 6 7 7 6 5 3
12558 004/300 Write block 4444 len 4096 data: 3
12559 005/300 Read block 925 len 40960
12560 006/300 Read block 4677 len 4096
12561 007/300 Write block 408 len 40960 data: 5 5 5 5 4 4 4 5 5 5
12562 008/300 Write block 2555 len 16384 data: 5 4 4 3
12563 009/300 Flush
12564 010/300 Read block 3146 len 40960
12565 011/300 Read block 175 len 4096
12566 012/300 Read block 4834 len 4096
12567 013/300 Flush
12568 014/300 Read block 638 len 40960
12569 015/300 Write block 4532 len 32768 data: 3 3 2 2 2 2 2 3
12570 016/300 Read block 1872 len 4096
12571 017/300 Read block 2173 len 40960
12572 018/300 Read block 2174 len 16384
12573 019/300 Read block 3708 len 16384
12574 020/300 Read block 4816 len 20480
12575 021/300 Read block 2583 len 36864
12576 022/300 Read block 1742 len 8192
12577 023/300 Write block 3494 len 28672 data: 3 3 3 3 4 4 4
12578 024/300 Read block 94 len 16384
12579 025/300 Read block 3652 len 4096
12580 026/300 Write block 4417 len 32768 data: 5 5 6 5 5 6 6 6
12581 027/300 Read block 2095 len 40960
12582 028/300 Write block 4946 len 36864 data: 4 3 4 4 5 5 5 4 4
12583 029/300 Read block 4856 len 28672
12584 030/300 Write block 348 len 16384 data: 3 4 4 3
12585 031/300 Write block 585 len 16384 data: 2 2 3 3
12586 032/300 Read block 2007 len 8192
12587 033/300 Write block 4967 len 12288 data: 3 3 4
12588 034/300 Read block 4827 len 28672
12589 035/300 Flush
12590 036/300 Write block 605 len 12288 data: 3 3 3
12591 037/300 Read block 4645 len 36864
12592 038/300 Write block 4062 len 32768 data: 7 4 4 3 3 3 3 3
12593 039/300 Read block 3078 len 8192
12594 040/300 Write block 3875 len 20480 data: 4 3 3 2 2
12595 041/300 Write block 3708 len 28672 data: 4 5 6 6 4 5 6
12596 042/300 Write block 3511 len 24576 data: 6 6 7 8 6 6
12597 043/300 Read block 172 len 20480
12598 044/300 Write block 349 len 28672 data: 5 5 4 3 4 4 3
12599 045/300 Read block 2914 len 12288
12600 046/300 Write block 663 len 4096 data: 5
12601 047/300 Write block 2118 len 8192 data: 2 2
12602 048/300 Write block 4985 len 36864 data: 6 6 7 8 7 6 6 6 4
12603 049/300 Read block 3010 len 20480
12604 050/300 Write block 0 len 20480 data: 2 2 2 2 2
12605 051/300 Flush
12606 052/300 Write block 845 len 8192 data: 6 5
12607 053/300 Read block 1217 len 28672
12608 054/300 Read block 4423 len 8192
12609 055/300 Write block 4799 len 32768 data: 3 3 3 4 4 4 4 4
12610 056/300 Write block 693 len 40960 data: 3 3 4 4 4 4 5 4 5 4
12611 057/300 Read block 3550 len 8192
12612 058/300 Read block 3160 len 4096
12613 059/300 Read block 4738 len 12288
12614 060/300 Write block 731 len 8192 data: 6 4
12615 061/300 Read block 4903 len 36864
12616 062/300 Write block 2683 len 20480 data: 3 4 3 3 3
12617 063/300 Write block 4412 len 20480 data: 4 4 4 4 4
12618 064/300 Write block 3704 len 36864 data: 6 6 6 6 5 6 7 7 5
12619 065/300 Write block 2689 len 24576 data: 3 3 3 3 4 4
12620 066/300 Write block 3339 len 32768 data: 4 6 6 6 5 5 5 5
12621 067/300 Flush
12622 068/300 Read block 3894 len 8192
12623 069/300 Read block 738 len 40960
12624 070/300 Write block 2618 len 32768 data: 4 4 5 8 9 8 7 7
12625 071/300 Write block 2418 len 20480 data: 2 2 2 3 3
12626 072/300 Read block 2156 len 28672
12627 073/300 Read block 521 len 4096
12628 074/300 Read block 3604 len 20480
12629 075/300 Write block 485 len 40960 data: 8 7 7 6 5 5 4 4 4 3
12630 076/300 Write block 1223 len 16384 data: 4 3 3 3
12631 077/300 Write block 227 len 20480 data: 4 4 4 4 3
12632 078/300 Read block 2087 len 36864
12633 079/300 Write block 1429 len 8192 data: 6 7
12634 080/300 Write block 740 len 12288 data: 5 4 5
12635 081/300 Write block 4125 len 12288 data: 3 3 4
12636 082/300 Read block 4304 len 20480
12637 083/300 Read block 3598 len 4096
12638 084/300 Read block 4342 len 12288
12639 085/300 Read block 3363 len 12288
12640 086/300 Write block 4416 len 24576 data: 5 6 6 7 6 6
12641 087/300 Write block 708 len 28672 data: 4 4 4 3 2 3 3
12642 088/300 Read block 3868 len 32768
12643 089/300 Read block 3418 len 36864
12644 090/300 Flush
12645 091/300 Read block 3774 len 28672
12646 092/300 Read block 4512 len 32768
12647 093/300 Read block 4109 len 36864
12648 094/300 Read block 2752 len 8192
12649 095/300 Write block 3818 len 24576 data: 3 3 3 3 3 4
12650 096/300 Flush
12651 097/300 Write block 3392 len 20480 data: 4 4 4 3 2
12652 098/300 Flush
12653 099/300 Read block 2865 len 8192
12654 100/300 Flush
12655 101/300 Read block 3292 len 36864
12656 102/300 Read block 4739 len 8192
12657 103/300 Read block 1939 len 28672
12658 104/300 Read block 4587 len 16384
12659 105/300 Read block 802 len 12288
12660 106/300 Flush
12661 107/300 Read block 4050 len 12288
12662 108/300 Write block 4048 len 8192 data: 4 5
12663 109/300 Read block 2619 len 28672
12664 110/300 Read block 791 len 32768
12665 111/300 Read block 1234 len 12288
12666 112/300 Write block 1066 len 20480 data: 3 3 3 4 4
12667 113/300 Read block 2127 len 24576
12668 114/300 Write block 783 len 20480 data: 2 2 3 3 2
12669 115/300 Write block 2578 len 8192 data: 7 6
12670 116/300 Write block 1796 len 8192 data: 9 9
12671 117/300 Read block 626 len 40960
12672 118/300 Write block 928 len 4096 data: 3
12673 119/300 Write block 4314 len 40960 data: 5 5 2 2 2 3 3 4 5 5
12674 120/300 Write block 254 len 16384 data: 4 3 2 3
12675 121/300 Read block 1947 len 24576
12676 122/300 Read block 3537 len 20480
12677 123/300 Write block 3466 len 4096 data: 4
12678 124/300 Write block 4726 len 16384 data: 4 4 5 6
12679 125/300 Read block 3988 len 16384
12680 126/300 Read block 428 len 28672
12681 127/300 Read block 1648 len 32768
12682 128/300 Read block 3157 len 28672
12683 129/300 Read block 4564 len 36864
12684 130/300 Read block 2783 len 12288
12685 131/300 Write block 3025 len 16384 data: 6 8 6 6
12686 132/300 Read block 2125 len 20480
12687 133/300 Read block 3622 len 36864
12688 134/300 Read block 3366 len 28672
12689 135/300 Write block 16 len 40960 data: 4 5 4 5 5 4 5 3 4 3
12690 136/300 Write block 81 len 24576 data: 4 3 3 2 3 3
12691 137/300 Read block 4445 len 40960
12692 138/300 Write block 2621 len 20480 data: 9 10 9 8 8
12693 139/300 Write block 4102 len 8192 data: 5 6
12694 140/300 Write block 3356 len 4096 data: 2
12695 141/300 Write block 2587 len 4096 data: 7
12696 142/300 Write block 1795 len 28672 data: 9 10 10 7 5 4 3
12697 143/300 Write block 2700 len 28672 data: 3 3 3 3 4 4 4
12698 144/300 Read block 2581 len 16384
12699 145/300 Flush
12700 146/300 Read block 908 len 20480
12701 147/300 Write block 3012 len 4096 data: 4
12702 148/300 Read block 266 len 28672
12703 149/300 Read block 4105 len 36864
12704 150/300 Flush
12705 151/300 Write block 2090 len 20480 data: 3 4 5 5 5
12706 152/300 Read block 2923 len 24576
12707 153/300 Write block 2669 len 40960 data: 4 4 4 3 3 2 3 3 3 3
12708 154/300 Write block 3647 len 16384 data: 3 4 4 4
12709 155/300 Read block 4182 len 12288
12710 156/300 Read block 4158 len 16384
12711 157/300 Write block 1872 len 12288 data: 3 4 4
12712 158/300 Write block 1036 len 40960 data: 4 3 3 3 3 3 2 2 3 3
12713 159/300 Write block 2843 len 16384 data: 5 7 7 4
12714 160/300 Read block 2757 len 4096
12715 161/300 Read block 4620 len 16384
12716 162/300 Write block 1448 len 28672 data: 6 4 4 4 2 2 2
12717 163/300 Flush
12718 164/300 Write block 4538 len 24576 data: 3 4 3 3 3 3
12719 165/300 Write block 3416 len 32768 data: 3 2 3 4 4 4 4 4
12720 166/300 Write block 4612 len 12288 data: 3 4 5
12721 167/300 Read block 951 len 24576
12722 168/300 Read block 631 len 40960
12723 169/300 Read block 2102 len 28672
12724 170/300 Read block 2818 len 4096
12725 171/300 Flush
12726 172/300 Read block 2750 len 36864
12727 173/300 Read block 640 len 20480
12728 174/300 Flush
12729 175/300 Write block 2155 len 8192 data: 4 3
12730 176/300 Read block 3722 len 36864
12731 177/300 Write block 4217 len 32768 data: 5 3 3 3 3 2 2 2
12732 178/300 Read block 4601 len 4096
12733 179/300 Read block 3950 len 40960
12734 180/300 Read block 4165 len 28672
12735 181/300 Read block 929 len 36864
12736 182/300 Write block 1354 len 24576 data: 2 2 2 3 3 3
12737 183/300 Read block 570 len 24576
12738 184/300 Read block 340 len 16384
12739 185/300 Read block 1051 len 4096
12740 186/300 Write block 1097 len 24576 data: 2 2 2 2 3 3
12741 187/300 Read block 775 len 32768
12742 188/300 Write block 2525 len 24576 data: 4 5 5 5 5 5
12743 189/300 Write block 2259 len 16384 data: 3 3 3 2
12744 190/300 Read block 2884 len 12288
12745 191/300 Write block 1172 len 8192 data: 5 5
12746 192/300 Read block 4846 len 20480
12747 193/300 Write block 2668 len 20480 data: 5 5 5 5 4
12748 194/300 Write block 1065 len 32768 data: 3 4 4 4 5 5 4 3
12749 195/300 Write block 2380 len 12288 data: 4 6 5
12750 196/300 Read block 845 len 16384
12751 197/300 Read block 244 len 32768
12752 198/300 Read block 1381 len 12288
12753 199/300 Write block 1273 len 8192 data: 6 6
12754 200/300 Flush
12755 201/300 Write block 2822 len 28672 data: 3 3 3 3 2 2 3
12756 202/300 Write block 2574 len 16384 data: 7 8 8 9
12757 203/300 Flush
12758 204/300 Read block 1871 len 16384
12759 205/300 Read block 1031 len 20480
12760 206/300 Write block 148 len 20480 data: 7 7 7 5 3
12761 207/300 Read block 2074 len 4096
12762 208/300 Read block 230 len 12288
12763 209/300 Read block 3888 len 24576
12764 210/300 Write block 3124 len 12288 data: 6 6 6
12765 211/300 Write block 2585 len 40960 data: 7 6 8 6 5 4 4 3 2 2
12766 212/300 Write block 608 len 8192 data: 3 3
12767 213/300 Write block 1310 len 20480 data: 5 4 4 6 5
12768 214/300 Read block 2961 len 8192
12769 215/300 Read block 1775 len 40960
12770 216/300 Read block 1313 len 40960
12771 217/300 Read block 3887 len 12288
12772 218/300 Flush
12773 219/300 Write block 4720 len 12288 data: 3 3 3
12774 220/300 Read block 1667 len 4096
12775 221/300 Read block 3769 len 28672
12776 222/300 Read block 3026 len 28672
12777 223/300 Read block 2938 len 40960
12778 224/300 Write block 4574 len 24576 data: 3 4 4 4 3 4
12779 225/300 Read block 2229 len 40960
12780 226/300 Write block 3803 len 8192 data: 4 4
12781 227/300 Write block 4464 len 24576 data: 3 3 3 3 3 3
12782 228/300 Read block 4519 len 36864
12783 229/300 Read block 2517 len 40960
12784 230/300 Read block 4922 len 20480
12785 231/300 Read block 2353 len 32768
12786 232/300 Read block 1412 len 40960
12787 233/300 Read block 3141 len 8192
12788 234/300 Read block 1098 len 20480
12789 235/300 Write block 1015 len 4096 data: 2
12790 236/300 Flush
12791 237/300 Read block 4530 len 8192
12792 238/300 Read block 2770 len 12288
12793 239/300 Read block 4285 len 16384
12794 240/300 Read block 4531 len 4096
12795 241/300 Write block 273 len 4096 data: 5
12796 242/300 Read block 138 len 40960
12797 243/300 Write block 992 len 36864 data: 4 4 6 6 5 4 4 5 6
12798 244/300 Write block 1196 len 32768 data: 3 3 2 2 3 3 3 4
12799 245/300 Read block 731 len 24576
12800 246/300 Write block 2919 len 8192 data: 2 2
12801 247/300 Read block 2097 len 32768
12802 248/300 Read block 903 len 24576
12803 249/300 Read block 3312 len 36864
12804 250/300 Read block 3804 len 36864
12805 251/300 Write block 4132 len 28672 data: 4 4 4 3 3 3 3
12806 252/300 Read block 1210 len 16384
12807 253/300 Read block 4152 len 36864
12808 254/300 Flush
12809 255/300 Write block 1400 len 20480 data: 4 6 6 7 6
12810 256/300 Read block 1706 len 16384
12811 257/300 Read block 1156 len 20480
12812 258/300 Read block 4348 len 28672
12813 259/300 Read block 4809 len 32768
12814 260/300 Read block 2222 len 8192
12815 261/300 Flush
12816 262/300 Write block 492 len 28672 data: 5 5 4 3 2 3 4
12817 263/300 Write block 480 len 8192 data: 3 3
12818 264/300 Flush
12819 265/300 Read block 1011 len 40960
12820 266/300 Read block 1607 len 20480
12821 267/300 Write block 3670 len 24576 data: 4 4 3 3 2 3
12822 268/300 Read block 975 len 20480
12823 269/300 Read block 4416 len 28672
12824 270/300 Read block 2727 len 32768
12825 271/300 Read block 3458 len 8192
12826 272/300 Flush
12827 273/300 Flush
12828 274/300 Write block 1658 len 36864 data: 5 3 2 2 3 3 2 2 2
12829 275/300 Write block 3085 len 8192 data: 5 5
12830 276/300 Write block 4000 len 4096 data: 8
12831 277/300 Read block 1553 len 28672
12832 278/300 Read block 4778 len 16384
12833 279/300 Read block 4591 len 20480
12834 280/300 Flush
12835 281/300 Write block 768 len 28672 data: 4 5 6 6 5 4 4
12836 282/300 Read block 4341 len 16384
12837 283/300 Write block 3110 len 40960 data: 4 4 3 3 3 3 3 3 5 5
12838 284/300 Write block 2529 len 16384 data: 6 6 5 5
12839 285/300 Write block 4527 len 8192 data: 4 4
12840 286/300 Flush
12841 287/300 Read block 355 len 16384
12842 288/300 Read block 896 len 8192
12843 289/300 Write block 1478 len 20480 data: 9 8 7 5 4
12844 290/300 Read block 747 len 32768
12845 291/300 Read block 2729 len 28672
12846 292/300 Read block 4900 len 32768
12847 293/300 Write block 254 len 32768 data: 5 4 3 4 6 5 5 4
12848 294/300 Read block 2470 len 12288
12849 295/300 Write block 351 len 24576 data: 5 4 5 5 4 5
12850 296/300 Flush
12851 297/300 Write block 3381 len 12288 data: 3 2 2
12852 298/300 Write block 979 len 24576 data: 3 3 3 4 4 4
12853 299/300 Flush
12854 300/300 Read block 1551 len 40960
12855 Main task has work to do, go find it
12856 got dsc Start(1)
12857 start 1
12858 Sep 22 23:26:37.471 INFO request completed, latency_us: 302, response_code: 204, uri: /start/cid/1, method: POST, req_id: 26d53282-131b-4949-ab48-309c8507a14e, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
12859 Replay: started 1, returned:Ok(())
12860 ----------------------------------------------------------------
12861 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12862 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
12863 4539 Acked 5538 Read 10 Done New Done false
12864 4540 Acked 5539 Read 8 Done New Done false
12865 4541 Acked 5540 Write 9 Done New Done false
12866 4542 Acked 5541 Write 1 Done New Done false
12867 4543 Acked 5542 Read 10 Done New Done false
12868 4544 Acked 5543 Read 1 Done New Done false
12869 4545 Acked 5544 Write 10 Done New Done false
12870 4546 Acked 5545 Write 4 Done New Done false
12871 4547 Acked 5546 Flush 0 Done New Done false
12872 4548 Acked 5547 Read 10 Done New Done false
12873 4549 Acked 5548 Read 1 Done New Done false
12874 4550 Acked 5549 Read 1 Done New Done false
12875 4551 Acked 5550 Flush 0 Done New Done false
12876 4552 Acked 5551 Read 10 Done New Done false
12877 4553 Acked 5552 Write 8 Done New Done false
12878 4554 Acked 5553 Read 1 Done New Done false
12879 4555 Acked 5554 Read 10 Done New Done false
12880 4556 Acked 5555 Read 4 Done New Done false
12881 4557 Acked 5556 Read 4 Done New Done false
12882 4558 Acked 5557 Read 5 Done New Done false
12883 4559 Acked 5558 Read 9 Done New Done false
12884 4560 Acked 5559 Read 2 Done New Done false
12885 4561 Acked 5560 Write 7 Done New Done false
12886 4562 Acked 5561 Read 4 Done New Done false
12887 4563 Acked 5562 Read 1 Done New Done false
12888 4564 Acked 5563 Write 8 Done New Done false
12889 4565 Acked 5564 Read 10 Done New Done false
12890 4566 Acked 5565 Write 9 Done New Done false
12891 4567 Acked 5566 Read 7 Done New Done false
12892 4568 Acked 5567 Write 4 Done New Done false
12893 4569 Acked 5568 Write 4 Done New Done false
12894 4570 Acked 5569 Read 2 Done New Done false
12895 4571 Acked 5570 Write 3 Done New Done false
12896 4572 Acked 5571 Read 7 Done New Done false
12897 4573 Acked 5572 Flush 0 Done New Done false
12898 4574 Acked 5573 Write 3 Done New Done false
12899 4575 Acked 5574 Read 9 Done New Done false
12900 4576 Acked 5575 Write 8 Done New Done false
12901 4577 Acked 5576 Read 2 Done New Done false
12902 4578 Acked 5577 Write 5 Done New Done false
12903 4579 Acked 5578 Write 7 Done New Done false
12904 4580 Acked 5579 Write 6 Done New Done false
12905 4581 Acked 5580 Read 5 Done New Done false
12906 4582 Acked 5581 Write 7 Done New Done false
12907 4583 Acked 5582 Read 3 Done New Done false
12908 4584 Acked 5583 Write 1 Done New Done false
12909 4585 Acked 5584 Write 2 Done New Done false
12910 4586 Acked 5585 Write 9 Done New Done false
12911 4587 Acked 5586 Read 5 Done New Done false
12912 4588 Acked 5587 Write 5 Done New Done false
12913 4589 Acked 5588 Flush 0 Done New Done false
12914 4590 Acked 5589 Write 2 Done New Done false
12915 4591 Acked 5590 Read 7 Done New Done false
12916 4592 Acked 5591 Read 2 Done New Done false
12917 4593 Acked 5592 Write 8 Done New Done false
12918 4594 Acked 5593 Write 10 Done New Done false
12919 4595 Acked 5594 Read 2 Done New Done false
12920 4596 Acked 5595 Read 1 Done New Done false
12921 4597 Acked 5596 Read 3 Done New Done false
12922 4598 Acked 5597 Write 2 Done New Done false
12923 4599 Acked 5598 Read 9 Done New Done false
12924 4600 Acked 5599 Write 5 Done New Done false
12925 4601 Acked 5600 Write 5 Done New Done false
12926 4602 Acked 5601 Write 9 Done New Done false
12927 4603 Acked 5602 Write 6 Done New Done false
12928 4604 Acked 5603 Write 8 Done New Done false
12929 4605 Acked 5604 Flush 0 Done New Done false
12930 4606 Acked 5605 Read 2 Done New Done false
12931 4607 Acked 5606 Read 10 Done New Done false
12932 4608 Acked 5607 Write 8 Done New Done false
12933 4609 Acked 5608 Write 5 Done New Done false
12934 4610 Acked 5609 Read 7 Done New Done false
12935 4611 Acked 5610 Read 1 Done New Done false
12936 4612 Acked 5611 Read 5 Done New Done false
12937 4613 Acked 5612 Write 10 Done New Done false
12938 4614 Acked 5613 Write 4 Done New Done false
12939 4615 Acked 5614 Write 5 Done New Done false
12940 4616 Acked 5615 Read 9 Done New Done false
12941 4617 Acked 5616 Write 2 Done New Done false
12942 4618 Acked 5617 Write 3 Done New Done false
12943 4619 Acked 5618 Write 3 Done New Done false
12944 4620 Acked 5619 Read 5 Done New Done false
12945 4621 Acked 5620 Read 1 Done New Done false
12946 4622 Acked 5621 Read 3 Done New Done false
12947 4623 Acked 5622 Read 3 Done New Done false
12948 4624 Acked 5623 Write 6 Done New Done false
12949 4625 Acked 5624 Write 7 Done New Done false
12950 4626 Acked 5625 Read 8 Done New Done false
12951 4627 Acked 5626 Read 9 Done New Done false
12952 4628 Acked 5627 Flush 0 Done New Done false
12953 4629 Acked 5628 Read 7 Done New Done false
12954 4630 Acked 5629 Read 8 Done New Done false
12955 4631 Acked 5630 Read 9 Done New Done false
12956 4632 Acked 5631 Read 2 Done New Done false
12957 4633 Acked 5632 Write 6 Done New Done false
12958 4634 Acked 5633 Flush 0 Done New Done false
12959 4635 Acked 5634 Write 5 Done New Done false
12960 4636 Acked 5635 Flush 0 Done New Done false
12961 4637 Acked 5636 Flush 0 Done New Done false
12962 4638 Acked 5637 Read 2 Done New Done false
12963 4639 Acked 5638 Flush 0 Done New Done false
12964 4640 Acked 5639 Read 9 Done New Done false
12965 4641 Acked 5640 Read 2 Done New Done false
12966 4642 Acked 5641 Read 7 Done New Done false
12967 4643 Acked 5642 Read 4 Done New Done false
12968 4644 Acked 5643 Read 3 Done New Done false
12969 4645 Acked 5644 Flush 0 Done New Done false
12970 4646 Acked 5645 Read 3 Done New Done false
12971 4647 Acked 5646 Write 2 Done New Done false
12972 4648 Acked 5647 Read 7 Done New Done false
12973 4649 Acked 5648 Read 8 Done New Done false
12974 4650 Acked 5649 Read 3 Done New Done false
12975 4651 Acked 5650 Write 5 Done New Done false
12976 4652 Acked 5651 Read 6 Done New Done false
12977 4653 Acked 5652 Write 5 Done New Done false
12978 4654 Acked 5653 Write 2 Done New Done false
12979 4655 Acked 5654 Write 2 Done New Done false
12980 4656 Acked 5655 Read 10 Done New Done false
12981 4657 Acked 5656 Write 1 Done New Done false
12982 4658 Acked 5657 Write 10 Done New Done false
12983 4659 Acked 5658 Write 4 Done New Done false
12984 4660 Acked 5659 Read 6 Done New Done false
12985 4661 Acked 5660 Read 5 Done New Done false
12986 4662 Acked 5661 Write 1 Done New Done false
12987 4663 Acked 5662 Write 4 Done New Done false
12988 4664 Acked 5663 Read 4 Done New Done false
12989 4665 Acked 5664 Read 7 Done New Done false
12990 4666 Acked 5665 Read 8 Done New Done false
12991 4667 Acked 5666 Read 7 Done New Done false
12992 4668 Acked 5667 Read 9 Done New Done false
12993 4669 Acked 5668 Read 3 Done New Done false
12994 4670 Acked 5669 Write 4 Done New Done false
12995 4671 Acked 5670 Read 5 Done New Done false
12996 4672 Acked 5671 Read 9 Done New Done false
12997 4673 Acked 5672 Read 7 Done New Done false
12998 4674 Acked 5673 Write 10 Done New Done false
12999 4675 Acked 5674 Write 6 Done New Done false
13000 4676 Acked 5675 Read 10 Done New Done false
13001 4677 Acked 5676 Write 5 Done New Done false
13002 4678 Acked 5677 Write 2 Done New Done false
13003 4679 Acked 5678 Write 1 Done New Done false
13004 4680 Acked 5679 Write 1 Done New Done false
13005 4681 Acked 5680 Write 7 Done New Done false
13006 4682 Acked 5681 Write 7 Done New Done false
13007 4683 Acked 5682 Read 4 Done New Done false
13008 4684 Acked 5683 Flush 0 Done New Done false
13009 4685 Acked 5684 Read 5 Done New Done false
13010 4686 Acked 5685 Write 1 Done New Done false
13011 4687 Acked 5686 Read 7 Done New Done false
13012 4688 Acked 5687 Read 9 Done New Done false
13013 4689 Acked 5688 Flush 0 Done New Done false
13014 4690 Acked 5689 Write 5 Done New Done false
13015 4691 Acked 5690 Read 6 Done New Done false
13016 4692 Acked 5691 Write 10 Done New Done false
13017 4693 Acked 5692 Write 4 Done New Done false
13018 4694 Acked 5693 Read 3 Done New Done false
13019 4695 Acked 5694 Read 4 Done New Done false
13020 4696 Acked 5695 Write 3 Done New Done false
13021 4697 Acked 5696 Write 10 Done New Done false
13022 4698 Acked 5697 Write 4 Done New Done false
13023 4699 Acked 5698 Read 1 Done New Done false
13024 4700 Acked 5699 Read 4 Done New Done false
13025 4701 Acked 5700 Write 7 Done New Done false
13026 4702 Acked 5701 Flush 0 Done New Done false
13027 4703 Acked 5702 Write 6 Done New Done false
13028 4704 Acked 5703 Write 8 Done New Done false
13029 4705 Acked 5704 Write 3 Done New Done false
13030 4706 Acked 5705 Read 6 Done New Done false
13031 4707 Acked 5706 Read 10 Done New Done false
13032 4708 Acked 5707 Read 7 Done New Done false
13033 4709 Acked 5708 Read 1 Done New Done false
13034 4710 Acked 5709 Flush 0 Done New Done false
13035 4711 Acked 5710 Read 9 Done New Done false
13036 4712 Acked 5711 Read 5 Done New Done false
13037 4713 Acked 5712 Flush 0 Done New Done false
13038 4714 Acked 5713 Write 2 Done New Done false
13039 4715 Acked 5714 Read 9 Done New Done false
13040 4716 Acked 5715 Write 8 Done New Done false
13041 4717 Acked 5716 Read 1 Done New Done false
13042 4718 Acked 5717 Read 10 Done New Done false
13043 4719 Acked 5718 Read 7 Done New Done false
13044 4720 Acked 5719 Read 9 Done New Done false
13045 4721 Acked 5720 Write 6 Done New Done false
13046 4722 Acked 5721 Read 6 Done New Done false
13047 4723 Acked 5722 Read 4 Done New Done false
13048 4724 Acked 5723 Read 1 Done New Done false
13049 4725 Acked 5724 Write 6 Done New Done false
13050 4726 Acked 5725 Read 8 Done New Done false
13051 4727 Acked 5726 Write 6 Done New Done false
13052 4728 Acked 5727 Write 4 Done New Done false
13053 4729 Acked 5728 Read 3 Done New Done false
13054 4730 Acked 5729 Write 2 Done New Done false
13055 4731 Acked 5730 Read 5 Done New Done false
13056 4732 Acked 5731 Write 5 Done New Done false
13057 4733 Acked 5732 Write 8 Done New Done false
13058 4734 Acked 5733 Write 3 Done New Done false
13059 4735 Acked 5734 Read 4 Done New Done false
13060 4736 Acked 5735 Read 8 Done New Done false
13061 4737 Acked 5736 Read 3 Done New Done false
13062 4738 Acked 5737 Write 2 Done New Done false
13063 4739 Acked 5738 Flush 0 Done New Done false
13064 4740 Acked 5739 Write 7 Done New Done false
13065 4741 Acked 5740 Write 4 Done New Done false
13066 4742 Acked 5741 Flush 0 Done New Done false
13067 4743 Acked 5742 Read 4 Done New Done false
13068 4744 Acked 5743 Read 5 Done New Done false
13069 4745 Acked 5744 Write 5 Done New Done false
13070 4746 Acked 5745 Read 1 Done New Done false
13071 4747 Acked 5746 Read 3 Done New Done false
13072 4748 Acked 5747 Read 6 Done New Done false
13073 4749 Acked 5748 Write 3 Done New Done false
13074 4750 Acked 5749 Write 10 Done New Done false
13075 4751 Acked 5750 Write 2 Done New Done false
13076 4752 Acked 5751 Write 5 Done New Done false
13077 4753 Acked 5752 Read 2 Done New Done false
13078 4754 Acked 5753 Read 10 Done New Done false
13079 4755 Acked 5754 Read 10 Done New Done false
13080 4756 Acked 5755 Read 3 Done New Done false
13081 4757 Acked 5756 Flush 0 Done New Done false
13082 4758 Acked 5757 Write 3 Done New Done false
13083 4759 Acked 5758 Read 1 Done New Done false
13084 4760 Acked 5759 Read 7 Done New Done false
13085 4761 Acked 5760 Read 7 Done New Done false
13086 4762 Acked 5761 Read 10 Done New Done false
13087 4763 Acked 5762 Write 6 Done New Done false
13088 4764 Acked 5763 Read 10 Done New Done false
13089 4765 Acked 5764 Write 2 Done New Done false
13090 4766 Acked 5765 Write 6 Done New Done false
13091 4767 Acked 5766 Read 9 Done New Done false
13092 4768 Acked 5767 Read 10 Done New Done false
13093 4769 Acked 5768 Read 5 Done New Done false
13094 4770 Acked 5769 Read 8 Done New Done false
13095 4771 Acked 5770 Read 10 Done New Done false
13096 4772 Acked 5771 Read 2 Done New Done false
13097 4773 Acked 5772 Read 5 Done New Done false
13098 4774 Acked 5773 Write 1 Done New Done false
13099 4775 Acked 5774 Flush 0 Done New Done false
13100 4776 Acked 5775 Read 2 Done New Done false
13101 4777 Acked 5776 Read 3 Done New Done false
13102 4778 Acked 5777 Read 4 Done New Done false
13103 4779 Acked 5778 Read 1 Done New Done false
13104 4780 Acked 5779 Write 1 Done New Done false
13105 4781 Acked 5780 Read 10 Done New Done false
13106 4782 Acked 5781 Write 9 Done New Done false
13107 4783 Acked 5782 Write 8 Done New Done false
13108 4784 Acked 5783 Read 6 Done New Done false
13109 4785 Acked 5784 Write 2 Done New Done false
13110 4786 Acked 5785 Read 8 Done New Done false
13111 4787 Acked 5786 Read 6 Done New Done false
13112 4788 Acked 5787 Read 9 Done New Done false
13113 4789 Acked 5788 Read 9 Done New Done false
13114 4790 Acked 5789 Write 7 Done New Done false
13115 4791 Acked 5790 Read 4 Done New Done false
13116 4792 Acked 5791 Read 9 Done New Done false
13117 4793 Acked 5792 Flush 0 Done New Done false
13118 4794 Acked 5793 Write 5 Done New Done false
13119 4795 Acked 5794 Read 4 Done New Done false
13120 4796 Acked 5795 Read 5 Done New Done false
13121 4797 Acked 5796 Flush 0 Done New Done false
13122 4798 Acked 5797 Read 7 Done New Done false
13123 4799 Acked 5798 Read 8 Done New Done false
13124 4800 Acked 5799 Read 2 Done New Done false
13125 4801 Acked 5800 Flush 0 Done New Done false
13126 4802 Acked 5801 Write 7 Done New Done false
13127 4803 Acked 5802 Write 2 Done New Done false
13128 4804 Acked 5803 Flush 0 Done New Done false
13129 4805 Acked 5804 Read 10 Done New Done false
13130 4806 Acked 5805 Read 5 Done New Done false
13131 4807 Acked 5806 Write 6 Done New Done false
13132 4808 Acked 5807 Read 5 Done New Done false
13133 4809 Acked 5808 Read 7 Done New Done false
13134 4810 Acked 5809 Read 8 Done New Done false
13135 4811 Acked 5810 Read 2 Done New Done false
13136 4812 Acked 5811 Flush 0 Done New Done false
13137 4813 Acked 5812 Flush 0 Done New Done false
13138 4814 Acked 5813 Write 9 Done New Done false
13139 4815 Acked 5814 Write 2 Done New Done false
13140 4816 Acked 5815 Write 1 Done New Done false
13141 4817 Acked 5816 Read 7 Done New Done false
13142 4818 Acked 5817 Read 4 Done New Done false
13143 4819 Acked 5818 Read 5 Done New Done false
13144 4820 Acked 5819 Flush 0 Done New Done false
13145 4821 Acked 5820 Write 7 Done New Done false
13146 4822 Acked 5821 Read 4 Done New Done false
13147 4823 Acked 5822 Write 10 Done New Done false
13148 4824 Acked 5823 Write 4 Done New Done false
13149 4825 Acked 5824 Write 2 Done New Done false
13150 4826 Acked 5825 Flush 0 Done New Done false
13151 4827 Acked 5826 Read 4 Done New Done false
13152 4828 Acked 5827 Read 2 Done New Done false
13153 4829 Acked 5828 Write 5 Done New Done false
13154 4830 Acked 5829 Read 8 Done New Done false
13155 4831 Acked 5830 Read 7 Done New Done false
13156 4832 Acked 5831 Read 8 Done New Done false
13157 4833 Acked 5832 Write 8 Done New Done false
13158 4834 Acked 5833 Read 3 Done New Done false
13159 4835 Acked 5834 Write 6 Done New Done false
13160 4836 Acked 5835 Flush 0 Done New Done false
13161 4837 Acked 5836 Write 3 Done New Done false
13162 4838 Acked 5837 Write 6 Done New Done false
13163 4839 Acked 5838 Flush 0 Done New Done false
13164 4840 Acked 5839 Read 10 Done New Done false
13165 STATES DS:0 DS:1 DS:2 TOTAL
13166 New 0 302 0 302
13167 Sent 0 0 0 0
13168 Done 302 0 302 604
13169 Skipped 0 0 0 0
13170 Error 0 0 0 0
13171 Last Flush: 5838 5537 5838
13172 Downstairs last five completed: 5537 5536 5535 5534 5533 5532
13173 Upstairs last five completed: 4840 4839 4838 4837 4836 4835
13174 CLIENT: Up:0 ds:302 act:2
13175 ----------------------------------------------------------------
13176 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
13177 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
13178 4539 Acked 5538 Read 10 Done New Done false
13179 4540 Acked 5539 Read 8 Done New Done false
13180 4541 Acked 5540 Write 9 Done New Done false
13181 4542 Acked 5541 Write 1 Done New Done false
13182 4543 Acked 5542 Read 10 Done New Done false
13183 4544 Acked 5543 Read 1 Done New Done false
13184 4545 Acked 5544 Write 10 Done New Done false
13185 4546 Acked 5545 Write 4 Done New Done false
13186 4547 Acked 5546 Flush 0 Done New Done false
13187 4548 Acked 5547 Read 10 Done New Done false
13188 4549 Acked 5548 Read 1 Done New Done false
13189 4550 Acked 5549 Read 1 Done New Done false
13190 4551 Acked 5550 Flush 0 Done New Done false
13191 4552 Acked 5551 Read 10 Done New Done false
13192 4553 Acked 5552 Write 8 Done New Done false
13193 4554 Acked 5553 Read 1 Done New Done false
13194 4555 Acked 5554 Read 10 Done New Done false
13195 4556 Acked 5555 Read 4 Done New Done false
13196 4557 Acked 5556 Read 4 Done New Done false
13197 4558 Acked 5557 Read 5 Done New Done false
13198 4559 Acked 5558 Read 9 Done New Done false
13199 4560 Acked 5559 Read 2 Done New Done false
13200 4561 Acked 5560 Write 7 Done New Done false
13201 4562 Acked 5561 Read 4 Done New Done false
13202 4563 Acked 5562 Read 1 Done New Done false
13203 4564 Acked 5563 Write 8 Done New Done false
13204 4565 Acked 5564 Read 10 Done New Done false
13205 4566 Acked 5565 Write 9 Done New Done false
13206 4567 Acked 5566 Read 7 Done New Done false
13207 4568 Acked 5567 Write 4 Done New Done false
13208 4569 Acked 5568 Write 4 Done New Done false
13209 4570 Acked 5569 Read 2 Done New Done false
13210 4571 Acked 5570 Write 3 Done New Done false
13211 4572 Acked 5571 Read 7 Done New Done false
13212 4573 Acked 5572 Flush 0 Done New Done false
13213 4574 Acked 5573 Write 3 Done New Done false
13214 4575 Acked 5574 Read 9 Done New Done false
13215 4576 Acked 5575 Write 8 Done New Done false
13216 4577 Acked 5576 Read 2 Done New Done false
13217 4578 Acked 5577 Write 5 Done New Done false
13218 4579 Acked 5578 Write 7 Done New Done false
13219 4580 Acked 5579 Write 6 Done New Done false
13220 4581 Acked 5580 Read 5 Done New Done false
13221 4582 Acked 5581 Write 7 Done New Done false
13222 4583 Acked 5582 Read 3 Done New Done false
13223 4584 Acked 5583 Write 1 Done New Done false
13224 4585 Acked 5584 Write 2 Done New Done false
13225 4586 Acked 5585 Write 9 Done New Done false
13226 4587 Acked 5586 Read 5 Done New Done false
13227 4588 Acked 5587 Write 5 Done New Done false
13228 4589 Acked 5588 Flush 0 Done New Done false
13229 4590 Acked 5589 Write 2 Done New Done false
13230 4591 Acked 5590 Read 7 Done New Done false
13231 4592 Acked 5591 Read 2 Done New Done false
13232 4593 Acked 5592 Write 8 Done New Done false
13233 4594 Acked 5593 Write 10 Done New Done false
13234 4595 Acked 5594 Read 2 Done New Done false
13235 4596 Acked 5595 Read 1 Done New Done false
13236 4597 Acked 5596 Read 3 Done New Done false
13237 4598 Acked 5597 Write 2 Done New Done false
13238 4599 Acked 5598 Read 9 Done New Done false
13239 4600 Acked 5599 Write 5 Done New Done false
13240 4601 Acked 5600 Write 5 Done New Done false
13241 4602 Acked 5601 Write 9 Done New Done false
13242 4603 Acked 5602 Write 6 Done New Done false
13243 4604 Acked 5603 Write 8 Done New Done false
13244 4605 Acked 5604 Flush 0 Done New Done false
13245 4606 Acked 5605 Read 2 Done New Done false
13246 4607 Acked 5606 Read 10 Done New Done false
13247 4608 Acked 5607 Write 8 Done New Done false
13248 4609 Acked 5608 Write 5 Done New Done false
13249 4610 Acked 5609 Read 7 Done New Done false
13250 4611 Acked 5610 Read 1 Done New Done false
13251 4612 Acked 5611 Read 5 Done New Done false
13252 4613 Acked 5612 Write 10 Done New Done false
13253 4614 Acked 5613 Write 4 Done New Done false
13254 4615 Acked 5614 Write 5 Done New Done false
13255 4616 Acked 5615 Read 9 Done New Done false
13256 4617 Acked 5616 Write 2 Done New Done false
13257 4618 Acked 5617 Write 3 Done New Done false
13258 4619 Acked 5618 Write 3 Done New Done false
13259 4620 Acked 5619 Read 5 Done New Done false
13260 4621 Acked 5620 Read 1 Done New Done false
13261 4622 Acked 5621 Read 3 Done New Done false
13262 4623 Acked 5622 Read 3 Done New Done false
13263 4624 Acked 5623 Write 6 Done New Done false
13264 4625 Acked 5624 Write 7 Done New Done false
13265 4626 Acked 5625 Read 8 Done New Done false
13266 4627 Acked 5626 Read 9 Done New Done false
13267 4628 Acked 5627 Flush 0 Done New Done false
13268 4629 Acked 5628 Read 7 Done New Done false
13269 4630 Acked 5629 Read 8 Done New Done false
13270 4631 Acked 5630 Read 9 Done New Done false
13271 4632 Acked 5631 Read 2 Done New Done false
13272 4633 Acked 5632 Write 6 Done New Done false
13273 4634 Acked 5633 Flush 0 Done New Done false
13274 4635 Acked 5634 Write 5 Done New Done false
13275 4636 Acked 5635 Flush 0 Done New Done false
13276 4637 Acked 5636 Flush 0 Done New Done false
13277 4638 Acked 5637 Read 2 Done New Done false
13278 4639 Acked 5638 Flush 0 Done New Done false
13279 4640 Acked 5639 Read 9 Done New Done false
13280 4641 Acked 5640 Read 2 Done New Done false
13281 4642 Acked 5641 Read 7 Done New Done false
13282 4643 Acked 5642 Read 4 Done New Done false
13283 4644 Acked 5643 Read 3 Done New Done false
13284 4645 Acked 5644 Flush 0 Done New Done false
13285 4646 Acked 5645 Read 3 Done New Done false
13286 4647 Acked 5646 Write 2 Done New Done false
13287 4648 Acked 5647 Read 7 Done New Done false
13288 4649 Acked 5648 Read 8 Done New Done false
13289 4650 Acked 5649 Read 3 Done New Done false
13290 4651 Acked 5650 Write 5 Done New Done false
13291 4652 Acked 5651 Read 6 Done New Done false
13292 4653 Acked 5652 Write 5 Done New Done false
13293 4654 Acked 5653 Write 2 Done New Done false
13294 4655 Acked 5654 Write 2 Done New Done false
13295 4656 Acked 5655 Read 10 Done New Done false
13296 4657 Acked 5656 Write 1 Done New Done false
13297 4658 Acked 5657 Write 10 Done New Done false
13298 4659 Acked 5658 Write 4 Done New Done false
13299 4660 Acked 5659 Read 6 Done New Done false
13300 4661 Acked 5660 Read 5 Done New Done false
13301 4662 Acked 5661 Write 1 Done New Done false
13302 4663 Acked 5662 Write 4 Done New Done false
13303 4664 Acked 5663 Read 4 Done New Done false
13304 4665 Acked 5664 Read 7 Done New Done false
13305 4666 Acked 5665 Read 8 Done New Done false
13306 4667 Acked 5666 Read 7 Done New Done false
13307 4668 Acked 5667 Read 9 Done New Done false
13308 4669 Acked 5668 Read 3 Done New Done false
13309 4670 Acked 5669 Write 4 Done New Done false
13310 4671 Acked 5670 Read 5 Done New Done false
13311 4672 Acked 5671 Read 9 Done New Done false
13312 4673 Acked 5672 Read 7 Done New Done false
13313 4674 Acked 5673 Write 10 Done New Done false
13314 4675 Acked 5674 Write 6 Done New Done false
13315 4676 Acked 5675 Read 10 Done New Done false
13316 4677 Acked 5676 Write 5 Done New Done false
13317 4678 Acked 5677 Write 2 Done New Done false
13318 4679 Acked 5678 Write 1 Done New Done false
13319 4680 Acked 5679 Write 1 Done New Done false
13320 4681 Acked 5680 Write 7 Done New Done false
13321 4682 Acked 5681 Write 7 Done New Done false
13322 4683 Acked 5682 Read 4 Done New Done false
13323 4684 Acked 5683 Flush 0 Done New Done false
13324 4685 Acked 5684 Read 5 Done New Done false
13325 4686 Acked 5685 Write 1 Done New Done false
13326 4687 Acked 5686 Read 7 Done New Done false
13327 4688 Acked 5687 Read 9 Done New Done false
13328 4689 Acked 5688 Flush 0 Done New Done false
13329 4690 Acked 5689 Write 5 Done New Done false
13330 4691 Acked 5690 Read 6 Done New Done false
13331 4692 Acked 5691 Write 10 Done New Done false
13332 4693 Acked 5692 Write 4 Done New Done false
13333 4694 Acked 5693 Read 3 Done New Done false
13334 4695 Acked 5694 Read 4 Done New Done false
13335 4696 Acked 5695 Write 3 Done New Done false
13336 4697 Acked 5696 Write 10 Done New Done false
13337 4698 Acked 5697 Write 4 Done New Done false
13338 4699 Acked 5698 Read 1 Done New Done false
13339 4700 Acked 5699 Read 4 Done New Done false
13340 4701 Acked 5700 Write 7 Done New Done false
13341 4702 Acked 5701 Flush 0 Done New Done false
13342 4703 Acked 5702 Write 6 Done New Done false
13343 4704 Acked 5703 Write 8 Done New Done false
13344 4705 Acked 5704 Write 3 Done New Done false
13345 4706 Acked 5705 Read 6 Done New Done false
13346 4707 Acked 5706 Read 10 Done New Done false
13347 4708 Acked 5707 Read 7 Done New Done false
13348 4709 Acked 5708 Read 1 Done New Done false
13349 4710 Acked 5709 Flush 0 Done New Done false
13350 4711 Acked 5710 Read 9 Done New Done false
13351 4712 Acked 5711 Read 5 Done New Done false
13352 4713 Acked 5712 Flush 0 Done New Done false
13353 4714 Acked 5713 Write 2 Done New Done false
13354 4715 Acked 5714 Read 9 Done New Done false
13355 4716 Acked 5715 Write 8 Done New Done false
13356 4717 Acked 5716 Read 1 Done New Done false
13357 4718 Acked 5717 Read 10 Done New Done false
13358 4719 Acked 5718 Read 7 Done New Done false
13359 4720 Acked 5719 Read 9 Done New Done false
13360 4721 Acked 5720 Write 6 Done New Done false
13361 4722 Acked 5721 Read 6 Done New Done false
13362 4723 Acked 5722 Read 4 Done New Done false
13363 4724 Acked 5723 Read 1 Done New Done false
13364 4725 Acked 5724 Write 6 Done New Done false
13365 4726 Acked 5725 Read 8 Done New Done false
13366 4727 Acked 5726 Write 6 Done New Done false
13367 4728 Acked 5727 Write 4 Done New Done false
13368 4729 Acked 5728 Read 3 Done New Done false
13369 4730 Acked 5729 Write 2 Done New Done false
13370 4731 Acked 5730 Read 5 Done New Done false
13371 4732 Acked 5731 Write 5 Done New Done false
13372 4733 Acked 5732 Write 8 Done New Done false
13373 4734 Acked 5733 Write 3 Done New Done false
13374 4735 Acked 5734 Read 4 Done New Done false
13375 4736 Acked 5735 Read 8 Done New Done false
13376 4737 Acked 5736 Read 3 Done New Done false
13377 4738 Acked 5737 Write 2 Done New Done false
13378 4739 Acked 5738 Flush 0 Done New Done false
13379 4740 Acked 5739 Write 7 Done New Done false
13380 4741 Acked 5740 Write 4 Done New Done false
13381 4742 Acked 5741 Flush 0 Done New Done false
13382 4743 Acked 5742 Read 4 Done New Done false
13383 4744 Acked 5743 Read 5 Done New Done false
13384 4745 Acked 5744 Write 5 Done New Done false
13385 4746 Acked 5745 Read 1 Done New Done false
13386 4747 Acked 5746 Read 3 Done New Done false
13387 4748 Acked 5747 Read 6 Done New Done false
13388 4749 Acked 5748 Write 3 Done New Done false
13389 4750 Acked 5749 Write 10 Done New Done false
13390 4751 Acked 5750 Write 2 Done New Done false
13391 4752 Acked 5751 Write 5 Done New Done false
13392 4753 Acked 5752 Read 2 Done New Done false
13393 4754 Acked 5753 Read 10 Done New Done false
13394 4755 Acked 5754 Read 10 Done New Done false
13395 4756 Acked 5755 Read 3 Done New Done false
13396 4757 Acked 5756 Flush 0 Done New Done false
13397 4758 Acked 5757 Write 3 Done New Done false
13398 4759 Acked 5758 Read 1 Done New Done false
13399 4760 Acked 5759 Read 7 Done New Done false
13400 4761 Acked 5760 Read 7 Done New Done false
13401 4762 Acked 5761 Read 10 Done New Done false
13402 4763 Acked 5762 Write 6 Done New Done false
13403 4764 Acked 5763 Read 10 Done New Done false
13404 4765 Acked 5764 Write 2 Done New Done false
13405 4766 Acked 5765 Write 6 Done New Done false
13406 4767 Acked 5766 Read 9 Done New Done false
13407 4768 Acked 5767 Read 10 Done New Done false
13408 4769 Acked 5768 Read 5 Done New Done false
13409 4770 Acked 5769 Read 8 Done New Done false
13410 4771 Acked 5770 Read 10 Done New Done false
13411 4772 Acked 5771 Read 2 Done New Done false
13412 4773 Acked 5772 Read 5 Done New Done false
13413 4774 Acked 5773 Write 1 Done New Done false
13414 4775 Acked 5774 Flush 0 Done New Done false
13415 4776 Acked 5775 Read 2 Done New Done false
13416 4777 Acked 5776 Read 3 Done New Done false
13417 4778 Acked 5777 Read 4 Done New Done false
13418 4779 Acked 5778 Read 1 Done New Done false
13419 4780 Acked 5779 Write 1 Done New Done false
13420 4781 Acked 5780 Read 10 Done New Done false
13421 4782 Acked 5781 Write 9 Done New Done false
13422 4783 Acked 5782 Write 8 Done New Done false
13423 4784 Acked 5783 Read 6 Done New Done false
13424 4785 Acked 5784 Write 2 Done New Done false
13425 4786 Acked 5785 Read 8 Done New Done false
13426 4787 Acked 5786 Read 6 Done New Done false
13427 4788 Acked 5787 Read 9 Done New Done false
13428 4789 Acked 5788 Read 9 Done New Done false
13429 4790 Acked 5789 Write 7 Done New Done false
13430 4791 Acked 5790 Read 4 Done New Done false
13431 4792 Acked 5791 Read 9 Done New Done false
13432 4793 Acked 5792 Flush 0 Done New Done false
13433 4794 Acked 5793 Write 5 Done New Done false
13434 4795 Acked 5794 Read 4 Done New Done false
13435 4796 Acked 5795 Read 5 Done New Done false
13436 4797 Acked 5796 Flush 0 Done New Done false
13437 4798 Acked 5797 Read 7 Done New Done false
13438 4799 Acked 5798 Read 8 Done New Done false
13439 4800 Acked 5799 Read 2 Done New Done false
13440 4801 Acked 5800 Flush 0 Done New Done false
13441 4802 Acked 5801 Write 7 Done New Done false
13442 4803 Acked 5802 Write 2 Done New Done false
13443 4804 Acked 5803 Flush 0 Done New Done false
13444 4805 Acked 5804 Read 10 Done New Done false
13445 4806 Acked 5805 Read 5 Done New Done false
13446 4807 Acked 5806 Write 6 Done New Done false
13447 4808 Acked 5807 Read 5 Done New Done false
13448 4809 Acked 5808 Read 7 Done New Done false
13449 4810 Acked 5809 Read 8 Done New Done false
13450 4811 Acked 5810 Read 2 Done New Done false
13451 4812 Acked 5811 Flush 0 Done New Done false
13452 4813 Acked 5812 Flush 0 Done New Done false
13453 4814 Acked 5813 Write 9 Done New Done false
13454 4815 Acked 5814 Write 2 Done New Done false
13455 4816 Acked 5815 Write 1 Done New Done false
13456 4817 Acked 5816 Read 7 Done New Done false
13457 4818 Acked 5817 Read 4 Done New Done false
13458 4819 Acked 5818 Read 5 Done New Done false
13459 4820 Acked 5819 Flush 0 Done New Done false
13460 4821 Acked 5820 Write 7 Done New Done false
13461 4822 Acked 5821 Read 4 Done New Done false
13462 4823 Acked 5822 Write 10 Done New Done false
13463 4824 Acked 5823 Write 4 Done New Done false
13464 4825 Acked 5824 Write 2 Done New Done false
13465 4826 Acked 5825 Flush 0 Done New Done false
13466 4827 Acked 5826 Read 4 Done New Done false
13467 4828 Acked 5827 Read 2 Done New Done false
13468 4829 Acked 5828 Write 5 Done New Done false
13469 4830 Acked 5829 Read 8 Done New Done false
13470 4831 Acked 5830 Read 7 Done New Done false
13471 4832 Acked 5831 Read 8 Done New Done false
13472 4833 Acked 5832 Write 8 Done New Done false
13473 4834 Acked 5833 Read 3 Done New Done false
13474 4835 Acked 5834 Write 6 Done New Done false
13475 4836 Acked 5835 Flush 0 Done New Done false
13476 4837 Acked 5836 Write 3 Done New Done false
13477 4838 Acked 5837 Write 6 Done New Done false
13478 4839 Acked 5838 Flush 0 Done New Done false
13479 4840 Acked 5839 Read 10 Done New Done false
13480 4841 Acked 5840 Flush 0 Done New Done false
13481 STATES DS:0 DS:1 DS:2 TOTAL
13482 New 0 303 0 303
13483 Sent 0 0 0 0
13484 Done 303 0 303 606
13485 Skipped 0 0 0 0
13486 Error 0 0 0 0
13487 Last Flush: 5840 5537 5840
13488 Downstairs last five completed: 5537 5536 5535 5534 5533 5532
13489 Upstairs last five completed: 4841 4840 4839 4838 4837 4836
13490 CLIENT: Up:0 ds:303 act:2
134912023-09-22T23:26:41.544ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
13492 ----------------------------------------------------------------
13493 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
13494 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
13495 4539 Acked 5538 Read 10 Done New Done false
13496 4540 Acked 5539 Read 8 Done New Done false
13497 4541 Acked 5540 Write 9 Done New Done false
13498 4542 Acked 5541 Write 1 Done New Done false
13499 4543 Acked 5542 Read 10 Done New Done false
13500 4544 Acked 5543 Read 1 Done New Done false
13501 4545 Acked 5544 Write 10 Done New Done false
13502 4546 Acked 5545 Write 4 Done New Done false
13503 4547 Acked 5546 Flush 0 Done New Done false
13504 4548 Acked 5547 Read 10 Done New Done false
13505 4549 Acked 5548 Read 1 Done New Done false
13506 4550 Acked 5549 Read 1 Done New Done false
13507 4551 Acked 5550 Flush 0 Done New Done false
13508 4552 Acked 5551 Read 10 Done New Done false
13509 4553 Acked 5552 Write 8 Done New Done false
13510 4554 Acked 5553 Read 1 Done New Done false
13511 4555 Acked 5554 Read 10 Done New Done false
13512 4556 Acked 5555 Read 4 Done New Done false
13513 4557 Acked 5556 Read 4 Done New Done false
13514 4558 Acked 5557 Read 5 Done New Done false
13515 4559 Acked 5558 Read 9 Done New Done false
13516 4560 Acked 5559 Read 2 Done New Done false
13517 4561 Acked 5560 Write 7 Done New Done false
13518 4562 Acked 5561 Read 4 Done New Done false
13519 4563 Acked 5562 Read 1 Done New Done false
13520 4564 Acked 5563 Write 8 Done New Done false
13521 4565 Acked 5564 Read 10 Done New Done false
13522 4566 Acked 5565 Write 9 Done New Done false
13523 4567 Acked 5566 Read 7 Done New Done false
13524 4568 Acked 5567 Write 4 Done New Done false
13525 4569 Acked 5568 Write 4 Done New Done false
13526 4570 Acked 5569 Read 2 Done New Done false
13527 4571 Acked 5570 Write 3 Done New Done false
13528 4572 Acked 5571 Read 7 Done New Done false
13529 4573 Acked 5572 Flush 0 Done New Done false
13530 4574 Acked 5573 Write 3 Done New Done false
13531 4575 Acked 5574 Read 9 Done New Done false
13532 4576 Acked 5575 Write 8 Done New Done false
13533 4577 Acked 5576 Read 2 Done New Done false
13534 4578 Acked 5577 Write 5 Done New Done false
13535 4579 Acked 5578 Write 7 Done New Done false
13536 4580 Acked 5579 Write 6 Done New Done false
13537 4581 Acked 5580 Read 5 Done New Done false
13538 4582 Acked 5581 Write 7 Done New Done false
13539 4583 Acked 5582 Read 3 Done New Done false
13540 4584 Acked 5583 Write 1 Done New Done false
13541 4585 Acked 5584 Write 2 Done New Done false
13542 4586 Acked 5585 Write 9 Done New Done false
13543 4587 Acked 5586 Read 5 Done New Done false
13544 4588 Acked 5587 Write 5 Done New Done false
13545 4589 Acked 5588 Flush 0 Done New Done false
13546 4590 Acked 5589 Write 2 Done New Done false
13547 4591 Acked 5590 Read 7 Done New Done false
13548 4592 Acked 5591 Read 2 Done New Done false
13549 4593 Acked 5592 Write 8 Done New Done false
13550 4594 Acked 5593 Write 10 Done New Done false
13551 4595 Acked 5594 Read 2 Done New Done false
13552 4596 Acked 5595 Read 1 Done New Done false
13553 4597 Acked 5596 Read 3 Done New Done false
13554 4598 Acked 5597 Write 2 Done New Done false
13555 4599 Acked 5598 Read 9 Done New Done false
13556 4600 Acked 5599 Write 5 Done New Done false
13557 4601 Acked 5600 Write 5 Done New Done false
13558 4602 Acked 5601 Write 9 Done New Done false
13559 4603 Acked 5602 Write 6 Done New Done false
13560 4604 Acked 5603 Write 8 Done New Done false
13561 4605 Acked 5604 Flush 0 Done New Done false
13562 4606 Acked 5605 Read 2 Done New Done false
13563 4607 Acked 5606 Read 10 Done New Done false
13564 4608 Acked 5607 Write 8 Done New Done false
13565 4609 Acked 5608 Write 5 Done New Done false
13566 4610 Acked 5609 Read 7 Done New Done false
13567 4611 Acked 5610 Read 1 Done New Done false
13568 4612 Acked 5611 Read 5 Done New Done false
13569 4613 Acked 5612 Write 10 Done New Done false
13570 4614 Acked 5613 Write 4 Done New Done false
13571 4615 Acked 5614 Write 5 Done New Done false
13572 4616 Acked 5615 Read 9 Done New Done false
13573 4617 Acked 5616 Write 2 Done New Done false
13574 4618 Acked 5617 Write 3 Done New Done false
13575 4619 Acked 5618 Write 3 Done New Done false
13576 4620 Acked 5619 Read 5 Done New Done false
13577 4621 Acked 5620 Read 1 Done New Done false
13578 4622 Acked 5621 Read 3 Done New Done false
13579 4623 Acked 5622 Read 3 Done New Done false
13580 4624 Acked 5623 Write 6 Done New Done false
13581 4625 Acked 5624 Write 7 Done New Done false
13582 4626 Acked 5625 Read 8 Done New Done false
13583 4627 Acked 5626 Read 9 Done New Done false
13584 4628 Acked 5627 Flush 0 Done New Done false
13585 4629 Acked 5628 Read 7 Done New Done false
13586 4630 Acked 5629 Read 8 Done New Done false
13587 4631 Acked 5630 Read 9 Done New Done false
13588 4632 Acked 5631 Read 2 Done New Done false
13589 4633 Acked 5632 Write 6 Done New Done false
13590 4634 Acked 5633 Flush 0 Done New Done false
13591 4635 Acked 5634 Write 5 Done New Done false
13592 4636 Acked 5635 Flush 0 Done New Done false
13593 4637 Acked 5636 Flush 0 Done New Done false
13594 4638 Acked 5637 Read 2 Done New Done false
13595 4639 Acked 5638 Flush 0 Done New Done false
13596 4640 Acked 5639 Read 9 Done New Done false
13597 4641 Acked 5640 Read 2 Done New Done false
13598 4642 Acked 5641 Read 7 Done New Done false
13599 4643 Acked 5642 Read 4 Done New Done false
13600 4644 Acked 5643 Read 3 Done New Done false
13601 4645 Acked 5644 Flush 0 Done New Done false
13602 4646 Acked 5645 Read 3 Done New Done false
13603 4647 Acked 5646 Write 2 Done New Done false
13604 4648 Acked 5647 Read 7 Done New Done false
13605 4649 Acked 5648 Read 8 Done New Done false
13606 4650 Acked 5649 Read 3 Done New Done false
13607 4651 Acked 5650 Write 5 Done New Done false
13608 4652 Acked 5651 Read 6 Done New Done false
13609 4653 Acked 5652 Write 5 Done New Done false
13610 4654 Acked 5653 Write 2 Done New Done false
13611 4655 Acked 5654 Write 2 Done New Done false
13612 4656 Acked 5655 Read 10 Done New Done false
13613 4657 Acked 5656 Write 1 Done New Done false
13614 4658 Acked 5657 Write 10 Done New Done false
13615 4659 Acked 5658 Write 4 Done New Done false
13616 4660 Acked 5659 Read 6 Done New Done false
13617 4661 Acked 5660 Read 5 Done New Done false
13618 4662 Acked 5661 Write 1 Done New Done false
13619 4663 Acked 5662 Write 4 Done New Done false
13620 4664 Acked 5663 Read 4 Done New Done false
13621 4665 Acked 5664 Read 7 Done New Done false
13622 4666 Acked 5665 Read 8 Done New Done false
13623 4667 Acked 5666 Read 7 Done New Done false
13624 4668 Acked 5667 Read 9 Done New Done false
13625 4669 Acked 5668 Read 3 Done New Done false
13626 4670 Acked 5669 Write 4 Done New Done false
13627 4671 Acked 5670 Read 5 Done New Done false
13628 4672 Acked 5671 Read 9 Done New Done false
13629 4673 Acked 5672 Read 7 Done New Done false
13630 4674 Acked 5673 Write 10 Done New Done false
13631 4675 Acked 5674 Write 6 Done New Done false
13632 4676 Acked 5675 Read 10 Done New Done false
13633 4677 Acked 5676 Write 5 Done New Done false
13634 4678 Acked 5677 Write 2 Done New Done false
13635 4679 Acked 5678 Write 1 Done New Done false
13636 4680 Acked 5679 Write 1 Done New Done false
13637 4681 Acked 5680 Write 7 Done New Done false
13638 4682 Acked 5681 Write 7 Done New Done false
13639 4683 Acked 5682 Read 4 Done New Done false
13640 4684 Acked 5683 Flush 0 Done New Done false
13641 4685 Acked 5684 Read 5 Done New Done false
13642 4686 Acked 5685 Write 1 Done New Done false
13643 4687 Acked 5686 Read 7 Done New Done false
13644 4688 Acked 5687 Read 9 Done New Done false
13645 4689 Acked 5688 Flush 0 Done New Done false
13646 4690 Acked 5689 Write 5 Done New Done false
13647 4691 Acked 5690 Read 6 Done New Done false
13648 4692 Acked 5691 Write 10 Done New Done false
13649 4693 Acked 5692 Write 4 Done New Done false
13650 4694 Acked 5693 Read 3 Done New Done false
13651 4695 Acked 5694 Read 4 Done New Done false
13652 4696 Acked 5695 Write 3 Done New Done false
13653 4697 Acked 5696 Write 10 Done New Done false
13654 4698 Acked 5697 Write 4 Done New Done false
13655 4699 Acked 5698 Read 1 Done New Done false
13656 4700 Acked 5699 Read 4 Done New Done false
13657 4701 Acked 5700 Write 7 Done New Done false
13658 4702 Acked 5701 Flush 0 Done New Done false
13659 4703 Acked 5702 Write 6 Done New Done false
13660 4704 Acked 5703 Write 8 Done New Done false
13661 4705 Acked 5704 Write 3 Done New Done false
13662 4706 Acked 5705 Read 6 Done New Done false
13663 4707 Acked 5706 Read 10 Done New Done false
13664 4708 Acked 5707 Read 7 Done New Done false
13665 4709 Acked 5708 Read 1 Done New Done false
13666 4710 Acked 5709 Flush 0 Done New Done false
13667 4711 Acked 5710 Read 9 Done New Done false
13668 4712 Acked 5711 Read 5 Done New Done false
13669 4713 Acked 5712 Flush 0 Done New Done false
13670 4714 Acked 5713 Write 2 Done New Done false
13671 4715 Acked 5714 Read 9 Done New Done false
13672 4716 Acked 5715 Write 8 Done New Done false
13673 4717 Acked 5716 Read 1 Done New Done false
13674 4718 Acked 5717 Read 10 Done New Done false
13675 4719 Acked 5718 Read 7 Done New Done false
13676 4720 Acked 5719 Read 9 Done New Done false
13677 4721 Acked 5720 Write 6 Done New Done false
13678 4722 Acked 5721 Read 6 Done New Done false
13679 4723 Acked 5722 Read 4 Done New Done false
13680 4724 Acked 5723 Read 1 Done New Done false
13681 4725 Acked 5724 Write 6 Done New Done false
13682 4726 Acked 5725 Read 8 Done New Done false
13683 4727 Acked 5726 Write 6 Done New Done false
13684 4728 Acked 5727 Write 4 Done New Done false
13685 4729 Acked 5728 Read 3 Done New Done false
13686 4730 Acked 5729 Write 2 Done New Done false
13687 4731 Acked 5730 Read 5 Done New Done false
13688 4732 Acked 5731 Write 5 Done New Done false
13689 4733 Acked 5732 Write 8 Done New Done false
13690 4734 Acked 5733 Write 3 Done New Done false
13691 4735 Acked 5734 Read 4 Done New Done false
13692 4736 Acked 5735 Read 8 Done New Done false
13693 4737 Acked 5736 Read 3 Done New Done false
13694 4738 Acked 5737 Write 2 Done New Done false
13695 4739 Acked 5738 Flush 0 Done New Done false
13696 4740 Acked 5739 Write 7 Done New Done false
13697 4741 Acked 5740 Write 4 Done New Done false
13698 4742 Acked 5741 Flush 0 Done New Done false
13699 4743 Acked 5742 Read 4 Done New Done false
13700 4744 Acked 5743 Read 5 Done New Done false
13701 4745 Acked 5744 Write 5 Done New Done false
13702 4746 Acked 5745 Read 1 Done New Done false
13703 4747 Acked 5746 Read 3 Done New Done false
13704 4748 Acked 5747 Read 6 Done New Done false
13705 4749 Acked 5748 Write 3 Done New Done false
13706 4750 Acked 5749 Write 10 Done New Done false
13707 4751 Acked 5750 Write 2 Done New Done false
13708 4752 Acked 5751 Write 5 Done New Done false
13709 4753 Acked 5752 Read 2 Done New Done false
13710 4754 Acked 5753 Read 10 Done New Done false
13711 4755 Acked 5754 Read 10 Done New Done false
13712 4756 Acked 5755 Read 3 Done New Done false
13713 4757 Acked 5756 Flush 0 Done New Done false
13714 4758 Acked 5757 Write 3 Done New Done false
13715 4759 Acked 5758 Read 1 Done New Done false
13716 4760 Acked 5759 Read 7 Done New Done false
13717 4761 Acked 5760 Read 7 Done New Done false
13718 4762 Acked 5761 Read 10 Done New Done false
13719 4763 Acked 5762 Write 6 Done New Done false
13720 4764 Acked 5763 Read 10 Done New Done false
13721 4765 Acked 5764 Write 2 Done New Done false
13722 4766 Acked 5765 Write 6 Done New Done false
13723 4767 Acked 5766 Read 9 Done New Done false
13724 4768 Acked 5767 Read 10 Done New Done false
13725 4769 Acked 5768 Read 5 Done New Done false
13726 4770 Acked 5769 Read 8 Done New Done false
13727 4771 Acked 5770 Read 10 Done New Done false
13728 4772 Acked 5771 Read 2 Done New Done false
13729 4773 Acked 5772 Read 5 Done New Done false
13730 4774 Acked 5773 Write 1 Done New Done false
13731 4775 Acked 5774 Flush 0 Done New Done false
13732 4776 Acked 5775 Read 2 Done New Done false
13733 4777 Acked 5776 Read 3 Done New Done false
13734 4778 Acked 5777 Read 4 Done New Done false
13735 4779 Acked 5778 Read 1 Done New Done false
13736 4780 Acked 5779 Write 1 Done New Done false
13737 4781 Acked 5780 Read 10 Done New Done false
13738 4782 Acked 5781 Write 9 Done New Done false
13739 4783 Acked 5782 Write 8 Done New Done false
13740 4784 Acked 5783 Read 6 Done New Done false
13741 4785 Acked 5784 Write 2 Done New Done false
13742 4786 Acked 5785 Read 8 Done New Done false
13743 4787 Acked 5786 Read 6 Done New Done false
13744 4788 Acked 5787 Read 9 Done New Done false
13745 4789 Acked 5788 Read 9 Done New Done false
13746 4790 Acked 5789 Write 7 Done New Done false
13747 4791 Acked 5790 Read 4 Done New Done false
13748 4792 Acked 5791 Read 9 Done New Done false
13749 4793 Acked 5792 Flush 0 Done New Done false
13750 4794 Acked 5793 Write 5 Done New Done false
13751 4795 Acked 5794 Read 4 Done New Done false
13752 4796 Acked 5795 Read 5 Done New Done false
13753 4797 Acked 5796 Flush 0 Done New Done false
13754 4798 Acked 5797 Read 7 Done New Done false
13755 4799 Acked 5798 Read 8 Done New Done false
13756 4800 Acked 5799 Read 2 Done New Done false
13757 4801 Acked 5800 Flush 0 Done New Done false
13758 4802 Acked 5801 Write 7 Done New Done false
13759 4803 Acked 5802 Write 2 Done New Done false
13760 4804 Acked 5803 Flush 0 Done New Done false
13761 4805 Acked 5804 Read 10 Done New Done false
13762 4806 Acked 5805 Read 5 Done New Done false
13763 4807 Acked 5806 Write 6 Done New Done false
13764 4808 Acked 5807 Read 5 Done New Done false
13765 4809 Acked 5808 Read 7 Done New Done false
13766 4810 Acked 5809 Read 8 Done New Done false
13767 4811 Acked 5810 Read 2 Done New Done false
13768 4812 Acked 5811 Flush 0 Done New Done false
13769 4813 Acked 5812 Flush 0 Done New Done false
13770 4814 Acked 5813 Write 9 Done New Done false
13771 4815 Acked 5814 Write 2 Done New Done false
13772 4816 Acked 5815 Write 1 Done New Done false
13773 4817 Acked 5816 Read 7 Done New Done false
13774 4818 Acked 5817 Read 4 Done New Done false
13775 4819 Acked 5818 Read 5 Done New Done false
13776 4820 Acked 5819 Flush 0 Done New Done false
13777 4821 Acked 5820 Write 7 Done New Done false
13778 4822 Acked 5821 Read 4 Done New Done false
13779 4823 Acked 5822 Write 10 Done New Done false
13780 4824 Acked 5823 Write 4 Done New Done false
13781 4825 Acked 5824 Write 2 Done New Done false
13782 4826 Acked 5825 Flush 0 Done New Done false
13783 4827 Acked 5826 Read 4 Done New Done false
13784 4828 Acked 5827 Read 2 Done New Done false
13785 4829 Acked 5828 Write 5 Done New Done false
13786 4830 Acked 5829 Read 8 Done New Done false
13787 4831 Acked 5830 Read 7 Done New Done false
13788 4832 Acked 5831 Read 8 Done New Done false
13789 4833 Acked 5832 Write 8 Done New Done false
13790 4834 Acked 5833 Read 3 Done New Done false
13791 4835 Acked 5834 Write 6 Done New Done false
13792 4836 Acked 5835 Flush 0 Done New Done false
13793 4837 Acked 5836 Write 3 Done New Done false
13794 4838 Acked 5837 Write 6 Done New Done false
13795 4839 Acked 5838 Flush 0 Done New Done false
13796 4840 Acked 5839 Read 10 Done New Done false
13797 4841 Acked 5840 Flush 0 Done New Done false
13798 STATES DS:0 DS:1 DS:2 TOTAL
13799 New 0 303 0 303
13800 Sent 0 0 0 0
13801 Done 303 0 303 606
13802 Skipped 0 0 0 0
13803 Error 0 0 0 0
13804 Last Flush: 5840 5537 5840
13805 Downstairs last five completed: 5537 5536 5535 5534 5533 5532
13806 Upstairs last five completed: 4841 4840 4839 4838 4837 4836
13807 CLIENT: Up:0 ds:303 act:2
13808 ----------------------------------------------------------------
13809 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
13810 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
13811 4539 Acked 5538 Read 10 Done New Done false
13812 4540 Acked 5539 Read 8 Done New Done false
13813 4541 Acked 5540 Write 9 Done New Done false
13814 4542 Acked 5541 Write 1 Done New Done false
13815 4543 Acked 5542 Read 10 Done New Done false
13816 4544 Acked 5543 Read 1 Done New Done false
13817 4545 Acked 5544 Write 10 Done New Done false
13818 4546 Acked 5545 Write 4 Done New Done false
13819 4547 Acked 5546 Flush 0 Done New Done false
13820 4548 Acked 5547 Read 10 Done New Done false
13821 4549 Acked 5548 Read 1 Done New Done false
13822 4550 Acked 5549 Read 1 Done New Done false
13823 4551 Acked 5550 Flush 0 Done New Done false
13824 4552 Acked 5551 Read 10 Done New Done false
13825 4553 Acked 5552 Write 8 Done New Done false
13826 4554 Acked 5553 Read 1 Done New Done false
13827 4555 Acked 5554 Read 10 Done New Done false
13828 4556 Acked 5555 Read 4 Done New Done false
13829 4557 Acked 5556 Read 4 Done New Done false
13830 4558 Acked 5557 Read 5 Done New Done false
13831 4559 Acked 5558 Read 9 Done New Done false
13832 4560 Acked 5559 Read 2 Done New Done false
13833 4561 Acked 5560 Write 7 Done New Done false
13834 4562 Acked 5561 Read 4 Done New Done false
13835 4563 Acked 5562 Read 1 Done New Done false
13836 4564 Acked 5563 Write 8 Done New Done false
13837 4565 Acked 5564 Read 10 Done New Done false
13838 4566 Acked 5565 Write 9 Done New Done false
13839 4567 Acked 5566 Read 7 Done New Done false
13840 4568 Acked 5567 Write 4 Done New Done false
13841 4569 Acked 5568 Write 4 Done New Done false
13842 4570 Acked 5569 Read 2 Done New Done false
13843 4571 Acked 5570 Write 3 Done New Done false
13844 4572 Acked 5571 Read 7 Done New Done false
13845 4573 Acked 5572 Flush 0 Done New Done false
13846 4574 Acked 5573 Write 3 Done New Done false
13847 4575 Acked 5574 Read 9 Done New Done false
13848 4576 Acked 5575 Write 8 Done New Done false
13849 4577 Acked 5576 Read 2 Done New Done false
13850 4578 Acked 5577 Write 5 Done New Done false
13851 4579 Acked 5578 Write 7 Done New Done false
13852 4580 Acked 5579 Write 6 Done New Done false
13853 4581 Acked 5580 Read 5 Done New Done false
13854 4582 Acked 5581 Write 7 Done New Done false
13855 4583 Acked 5582 Read 3 Done New Done false
13856 4584 Acked 5583 Write 1 Done New Done false
13857 4585 Acked 5584 Write 2 Done New Done false
13858 4586 Acked 5585 Write 9 Done New Done false
13859 4587 Acked 5586 Read 5 Done New Done false
13860 4588 Acked 5587 Write 5 Done New Done false
13861 4589 Acked 5588 Flush 0 Done New Done false
13862 4590 Acked 5589 Write 2 Done New Done false
13863 4591 Acked 5590 Read 7 Done New Done false
13864 4592 Acked 5591 Read 2 Done New Done false
13865 4593 Acked 5592 Write 8 Done New Done false
13866 4594 Acked 5593 Write 10 Done New Done false
13867 4595 Acked 5594 Read 2 Done New Done false
13868 4596 Acked 5595 Read 1 Done New Done false
13869 4597 Acked 5596 Read 3 Done New Done false
13870 4598 Acked 5597 Write 2 Done New Done false
13871 4599 Acked 5598 Read 9 Done New Done false
13872 4600 Acked 5599 Write 5 Done New Done false
13873 4601 Acked 5600 Write 5 Done New Done false
13874 4602 Acked 5601 Write 9 Done New Done false
13875 4603 Acked 5602 Write 6 Done New Done false
13876 4604 Acked 5603 Write 8 Done New Done false
13877 4605 Acked 5604 Flush 0 Done New Done false
13878 4606 Acked 5605 Read 2 Done New Done false
13879 4607 Acked 5606 Read 10 Done New Done false
13880 4608 Acked 5607 Write 8 Done New Done false
13881 4609 Acked 5608 Write 5 Done New Done false
13882 4610 Acked 5609 Read 7 Done New Done false
13883 4611 Acked 5610 Read 1 Done New Done false
13884 4612 Acked 5611 Read 5 Done New Done false
13885 4613 Acked 5612 Write 10 Done New Done false
13886 4614 Acked 5613 Write 4 Done New Done false
13887 4615 Acked 5614 Write 5 Done New Done false
13888 4616 Acked 5615 Read 9 Done New Done false
13889 4617 Acked 5616 Write 2 Done New Done false
13890 4618 Acked 5617 Write 3 Done New Done false
13891 4619 Acked 5618 Write 3 Done New Done false
13892 4620 Acked 5619 Read 5 Done New Done false
13893 4621 Acked 5620 Read 1 Done New Done false
13894 4622 Acked 5621 Read 3 Done New Done false
13895 4623 Acked 5622 Read 3 Done New Done false
13896 4624 Acked 5623 Write 6 Done New Done false
13897 4625 Acked 5624 Write 7 Done New Done false
13898 4626 Acked 5625 Read 8 Done New Done false
13899 4627 Acked 5626 Read 9 Done New Done false
13900 4628 Acked 5627 Flush 0 Done New Done false
13901 4629 Acked 5628 Read 7 Done New Done false
13902 4630 Acked 5629 Read 8 Done New Done false
13903 4631 Acked 5630 Read 9 Done New Done false
13904 4632 Acked 5631 Read 2 Done New Done false
13905 4633 Acked 5632 Write 6 Done New Done false
13906 4634 Acked 5633 Flush 0 Done New Done false
13907 4635 Acked 5634 Write 5 Done New Done false
13908 4636 Acked 5635 Flush 0 Done New Done false
13909 4637 Acked 5636 Flush 0 Done New Done false
13910 4638 Acked 5637 Read 2 Done New Done false
13911 4639 Acked 5638 Flush 0 Done New Done false
13912 4640 Acked 5639 Read 9 Done New Done false
13913 4641 Acked 5640 Read 2 Done New Done false
13914 4642 Acked 5641 Read 7 Done New Done false
13915 4643 Acked 5642 Read 4 Done New Done false
13916 4644 Acked 5643 Read 3 Done New Done false
13917 4645 Acked 5644 Flush 0 Done New Done false
13918 4646 Acked 5645 Read 3 Done New Done false
13919 4647 Acked 5646 Write 2 Done New Done false
13920 4648 Acked 5647 Read 7 Done New Done false
13921 4649 Acked 5648 Read 8 Done New Done false
13922 4650 Acked 5649 Read 3 Done New Done false
13923 4651 Acked 5650 Write 5 Done New Done false
13924 4652 Acked 5651 Read 6 Done New Done false
13925 4653 Acked 5652 Write 5 Done New Done false
13926 4654 Acked 5653 Write 2 Done New Done false
13927 4655 Acked 5654 Write 2 Done New Done false
13928 4656 Acked 5655 Read 10 Done New Done false
13929 4657 Acked 5656 Write 1 Done New Done false
13930 4658 Acked 5657 Write 10 Done New Done false
13931 4659 Acked 5658 Write 4 Done New Done false
13932 4660 Acked 5659 Read 6 Done New Done false
13933 4661 Acked 5660 Read 5 Done New Done false
13934 4662 Acked 5661 Write 1 Done New Done false
13935 4663 Acked 5662 Write 4 Done New Done false
13936 4664 Acked 5663 Read 4 Done New Done false
13937 4665 Acked 5664 Read 7 Done New Done false
13938 4666 Acked 5665 Read 8 Done New Done false
13939 4667 Acked 5666 Read 7 Done New Done false
13940 4668 Acked 5667 Read 9 Done New Done false
13941 4669 Acked 5668 Read 3 Done New Done false
13942 4670 Acked 5669 Write 4 Done New Done false
13943 4671 Acked 5670 Read 5 Done New Done false
13944 4672 Acked 5671 Read 9 Done New Done false
13945 4673 Acked 5672 Read 7 Done New Done false
13946 4674 Acked 5673 Write 10 Done New Done false
13947 4675 Acked 5674 Write 6 Done New Done false
13948 4676 Acked 5675 Read 10 Done New Done false
13949 4677 Acked 5676 Write 5 Done New Done false
13950 4678 Acked 5677 Write 2 Done New Done false
13951 4679 Acked 5678 Write 1 Done New Done false
13952 4680 Acked 5679 Write 1 Done New Done false
13953 4681 Acked 5680 Write 7 Done New Done false
13954 4682 Acked 5681 Write 7 Done New Done false
13955 4683 Acked 5682 Read 4 Done New Done false
13956 4684 Acked 5683 Flush 0 Done New Done false
13957 4685 Acked 5684 Read 5 Done New Done false
13958 4686 Acked 5685 Write 1 Done New Done false
13959 4687 Acked 5686 Read 7 Done New Done false
13960 4688 Acked 5687 Read 9 Done New Done false
13961 4689 Acked 5688 Flush 0 Done New Done false
13962 4690 Acked 5689 Write 5 Done New Done false
13963 4691 Acked 5690 Read 6 Done New Done false
13964 4692 Acked 5691 Write 10 Done New Done false
13965 4693 Acked 5692 Write 4 Done New Done false
13966 4694 Acked 5693 Read 3 Done New Done false
13967 4695 Acked 5694 Read 4 Done New Done false
13968 4696 Acked 5695 Write 3 Done New Done false
13969 4697 Acked 5696 Write 10 Done New Done false
13970 4698 Acked 5697 Write 4 Done New Done false
13971 4699 Acked 5698 Read 1 Done New Done false
13972 4700 Acked 5699 Read 4 Done New Done false
13973 4701 Acked 5700 Write 7 Done New Done false
13974 4702 Acked 5701 Flush 0 Done New Done false
13975 4703 Acked 5702 Write 6 Done New Done false
13976 4704 Acked 5703 Write 8 Done New Done false
13977 4705 Acked 5704 Write 3 Done New Done false
13978 4706 Acked 5705 Read 6 Done New Done false
13979 4707 Acked 5706 Read 10 Done New Done false
13980 4708 Acked 5707 Read 7 Done New Done false
13981 4709 Acked 5708 Read 1 Done New Done false
13982 4710 Acked 5709 Flush 0 Done New Done false
13983 4711 Acked 5710 Read 9 Done New Done false
13984 4712 Acked 5711 Read 5 Done New Done false
13985 4713 Acked 5712 Flush 0 Done New Done false
13986 4714 Acked 5713 Write 2 Done New Done false
13987 4715 Acked 5714 Read 9 Done New Done false
13988 4716 Acked 5715 Write 8 Done New Done false
13989 4717 Acked 5716 Read 1 Done New Done false
13990 4718 Acked 5717 Read 10 Done New Done false
13991 4719 Acked 5718 Read 7 Done New Done false
13992 4720 Acked 5719 Read 9 Done New Done false
13993 4721 Acked 5720 Write 6 Done New Done false
13994 4722 Acked 5721 Read 6 Done New Done false
13995 4723 Acked 5722 Read 4 Done New Done false
13996 4724 Acked 5723 Read 1 Done New Done false
13997 4725 Acked 5724 Write 6 Done New Done false
13998 4726 Acked 5725 Read 8 Done New Done false
13999 4727 Acked 5726 Write 6 Done New Done false
14000 4728 Acked 5727 Write 4 Done New Done false
14001 4729 Acked 5728 Read 3 Done New Done false
14002 4730 Acked 5729 Write 2 Done New Done false
14003 4731 Acked 5730 Read 5 Done New Done false
14004 4732 Acked 5731 Write 5 Done New Done false
14005 4733 Acked 5732 Write 8 Done New Done false
14006 4734 Acked 5733 Write 3 Done New Done false
14007 4735 Acked 5734 Read 4 Done New Done false
14008 4736 Acked 5735 Read 8 Done New Done false
14009 4737 Acked 5736 Read 3 Done New Done false
14010 4738 Acked 5737 Write 2 Done New Done false
14011 4739 Acked 5738 Flush 0 Done New Done false
14012 4740 Acked 5739 Write 7 Done New Done false
14013 4741 Acked 5740 Write 4 Done New Done false
14014 4742 Acked 5741 Flush 0 Done New Done false
14015 4743 Acked 5742 Read 4 Done New Done false
14016 4744 Acked 5743 Read 5 Done New Done false
14017 4745 Acked 5744 Write 5 Done New Done false
14018 4746 Acked 5745 Read 1 Done New Done false
14019 4747 Acked 5746 Read 3 Done New Done false
14020 4748 Acked 5747 Read 6 Done New Done false
14021 4749 Acked 5748 Write 3 Done New Done false
14022 4750 Acked 5749 Write 10 Done New Done false
14023 4751 Acked 5750 Write 2 Done New Done false
14024 4752 Acked 5751 Write 5 Done New Done false
14025 4753 Acked 5752 Read 2 Done New Done false
14026 4754 Acked 5753 Read 10 Done New Done false
14027 4755 Acked 5754 Read 10 Done New Done false
14028 4756 Acked 5755 Read 3 Done New Done false
14029 4757 Acked 5756 Flush 0 Done New Done false
14030 4758 Acked 5757 Write 3 Done New Done false
14031 4759 Acked 5758 Read 1 Done New Done false
14032 4760 Acked 5759 Read 7 Done New Done false
14033 4761 Acked 5760 Read 7 Done New Done false
14034 4762 Acked 5761 Read 10 Done New Done false
14035 4763 Acked 5762 Write 6 Done New Done false
14036 4764 Acked 5763 Read 10 Done New Done false
14037 4765 Acked 5764 Write 2 Done New Done false
14038 4766 Acked 5765 Write 6 Done New Done false
14039 4767 Acked 5766 Read 9 Done New Done false
14040 4768 Acked 5767 Read 10 Done New Done false
14041 4769 Acked 5768 Read 5 Done New Done false
14042 4770 Acked 5769 Read 8 Done New Done false
14043 4771 Acked 5770 Read 10 Done New Done false
14044 4772 Acked 5771 Read 2 Done New Done false
14045 4773 Acked 5772 Read 5 Done New Done false
14046 4774 Acked 5773 Write 1 Done New Done false
14047 4775 Acked 5774 Flush 0 Done New Done false
14048 4776 Acked 5775 Read 2 Done New Done false
14049 4777 Acked 5776 Read 3 Done New Done false
14050 4778 Acked 5777 Read 4 Done New Done false
14051 4779 Acked 5778 Read 1 Done New Done false
14052 4780 Acked 5779 Write 1 Done New Done false
14053 4781 Acked 5780 Read 10 Done New Done false
14054 4782 Acked 5781 Write 9 Done New Done false
14055 4783 Acked 5782 Write 8 Done New Done false
14056 4784 Acked 5783 Read 6 Done New Done false
14057 4785 Acked 5784 Write 2 Done New Done false
14058 4786 Acked 5785 Read 8 Done New Done false
14059 4787 Acked 5786 Read 6 Done New Done false
14060 4788 Acked 5787 Read 9 Done New Done false
14061 4789 Acked 5788 Read 9 Done New Done false
14062 4790 Acked 5789 Write 7 Done New Done false
14063 4791 Acked 5790 Read 4 Done New Done false
14064 4792 Acked 5791 Read 9 Done New Done false
14065 4793 Acked 5792 Flush 0 Done New Done false
14066 4794 Acked 5793 Write 5 Done New Done false
14067 4795 Acked 5794 Read 4 Done New Done false
14068 4796 Acked 5795 Read 5 Done New Done false
14069 4797 Acked 5796 Flush 0 Done New Done false
14070 4798 Acked 5797 Read 7 Done New Done false
14071 4799 Acked 5798 Read 8 Done New Done false
14072 4800 Acked 5799 Read 2 Done New Done false
14073 4801 Acked 5800 Flush 0 Done New Done false
14074 4802 Acked 5801 Write 7 Done New Done false
14075 4803 Acked 5802 Write 2 Done New Done false
14076 4804 Acked 5803 Flush 0 Done New Done false
14077 4805 Acked 5804 Read 10 Done New Done false
14078 4806 Acked 5805 Read 5 Done New Done false
14079 4807 Acked 5806 Write 6 Done New Done false
14080 4808 Acked 5807 Read 5 Done New Done false
14081 4809 Acked 5808 Read 7 Done New Done false
14082 4810 Acked 5809 Read 8 Done New Done false
14083 4811 Acked 5810 Read 2 Done New Done false
14084 4812 Acked 5811 Flush 0 Done New Done false
14085 4813 Acked 5812 Flush 0 Done New Done false
14086 4814 Acked 5813 Write 9 Done New Done false
14087 4815 Acked 5814 Write 2 Done New Done false
14088 4816 Acked 5815 Write 1 Done New Done false
14089 4817 Acked 5816 Read 7 Done New Done false
14090 4818 Acked 5817 Read 4 Done New Done false
14091 4819 Acked 5818 Read 5 Done New Done false
14092 4820 Acked 5819 Flush 0 Done New Done false
14093 4821 Acked 5820 Write 7 Done New Done false
14094 4822 Acked 5821 Read 4 Done New Done false
14095 4823 Acked 5822 Write 10 Done New Done false
14096 4824 Acked 5823 Write 4 Done New Done false
14097 4825 Acked 5824 Write 2 Done New Done false
14098 4826 Acked 5825 Flush 0 Done New Done false
14099 4827 Acked 5826 Read 4 Done New Done false
14100 4828 Acked 5827 Read 2 Done New Done false
14101 4829 Acked 5828 Write 5 Done New Done false
14102 4830 Acked 5829 Read 8 Done New Done false
14103 4831 Acked 5830 Read 7 Done New Done false
14104 4832 Acked 5831 Read 8 Done New Done false
14105 4833 Acked 5832 Write 8 Done New Done false
14106 4834 Acked 5833 Read 3 Done New Done false
14107 4835 Acked 5834 Write 6 Done New Done false
14108 4836 Acked 5835 Flush 0 Done New Done false
14109 4837 Acked 5836 Write 3 Done New Done false
14110 4838 Acked 5837 Write 6 Done New Done false
14111 4839 Acked 5838 Flush 0 Done New Done false
14112 4840 Acked 5839 Read 10 Done New Done false
14113 4841 Acked 5840 Flush 0 Done New Done false
14114 STATES DS:0 DS:1 DS:2 TOTAL
14115 New 0 303 0 303
14116 Sent 0 0 0 0
14117 Done 303 0 303 606
14118 Skipped 0 0 0 0
14119 Error 0 0 0 0
14120 Last Flush: 5840 5537 5840
14121 Downstairs last five completed: 5537 5536 5535 5534 5533 5532
14122 Upstairs last five completed: 4841 4840 4839 4838 4837 4836
14123 CLIENT: Up:0 ds:303 act:2
141242023-09-22T23:26:51.557ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
14125 [8820] got start action
14126 [8820] I am going to restart
14127 Starting downstairs at port 8820
14128 Make output file at "/tmp/dsc/downstairs-8820.txt"
14129 [8820][1] reports Starting
14130 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1189)
14131 [8820][1] reports Running
14132 ----------------------------------------------------------------
14133 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
14134 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
14135 4539 Acked 5538 Read 10 Done New Done false
14136 4540 Acked 5539 Read 8 Done New Done false
14137 4541 Acked 5540 Write 9 Done New Done false
14138 4542 Acked 5541 Write 1 Done New Done false
14139 4543 Acked 5542 Read 10 Done New Done false
14140 4544 Acked 5543 Read 1 Done New Done false
14141 4545 Acked 5544 Write 10 Done New Done false
14142 4546 Acked 5545 Write 4 Done New Done false
14143 4547 Acked 5546 Flush 0 Done New Done false
14144 4548 Acked 5547 Read 10 Done New Done false
14145 4549 Acked 5548 Read 1 Done New Done false
14146 4550 Acked 5549 Read 1 Done New Done false
14147 4551 Acked 5550 Flush 0 Done New Done false
14148 4552 Acked 5551 Read 10 Done New Done false
14149 4553 Acked 5552 Write 8 Done New Done false
14150 4554 Acked 5553 Read 1 Done New Done false
14151 4555 Acked 5554 Read 10 Done New Done false
14152 4556 Acked 5555 Read 4 Done New Done false
14153 4557 Acked 5556 Read 4 Done New Done false
14154 4558 Acked 5557 Read 5 Done New Done false
14155 4559 Acked 5558 Read 9 Done New Done false
14156 4560 Acked 5559 Read 2 Done New Done false
14157 4561 Acked 5560 Write 7 Done New Done false
14158 4562 Acked 5561 Read 4 Done New Done false
14159 4563 Acked 5562 Read 1 Done New Done false
14160 4564 Acked 5563 Write 8 Done New Done false
14161 4565 Acked 5564 Read 10 Done New Done false
14162 4566 Acked 5565 Write 9 Done New Done false
14163 4567 Acked 5566 Read 7 Done New Done false
14164 4568 Acked 5567 Write 4 Done New Done false
14165 4569 Acked 5568 Write 4 Done New Done false
14166 4570 Acked 5569 Read 2 Done New Done false
14167 4571 Acked 5570 Write 3 Done New Done false
14168 4572 Acked 5571 Read 7 Done New Done false
14169 4573 Acked 5572 Flush 0 Done New Done false
14170 4574 Acked 5573 Write 3 Done New Done false
14171 4575 Acked 5574 Read 9 Done New Done false
14172 4576 Acked 5575 Write 8 Done New Done false
14173 4577 Acked 5576 Read 2 Done New Done false
14174 4578 Acked 5577 Write 5 Done New Done false
14175 4579 Acked 5578 Write 7 Done New Done false
14176 4580 Acked 5579 Write 6 Done New Done false
14177 4581 Acked 5580 Read 5 Done New Done false
14178 4582 Acked 5581 Write 7 Done New Done false
14179 4583 Acked 5582 Read 3 Done New Done false
14180 4584 Acked 5583 Write 1 Done New Done false
14181 4585 Acked 5584 Write 2 Done New Done false
14182 4586 Acked 5585 Write 9 Done New Done false
14183 4587 Acked 5586 Read 5 Done New Done false
14184 4588 Acked 5587 Write 5 Done New Done false
14185 4589 Acked 5588 Flush 0 Done New Done false
14186 4590 Acked 5589 Write 2 Done New Done false
14187 4591 Acked 5590 Read 7 Done New Done false
14188 4592 Acked 5591 Read 2 Done New Done false
14189 4593 Acked 5592 Write 8 Done New Done false
14190 4594 Acked 5593 Write 10 Done New Done false
14191 4595 Acked 5594 Read 2 Done New Done false
14192 4596 Acked 5595 Read 1 Done New Done false
14193 4597 Acked 5596 Read 3 Done New Done false
14194 4598 Acked 5597 Write 2 Done New Done false
14195 4599 Acked 5598 Read 9 Done New Done false
14196 4600 Acked 5599 Write 5 Done New Done false
14197 4601 Acked 5600 Write 5 Done New Done false
14198 4602 Acked 5601 Write 9 Done New Done false
14199 4603 Acked 5602 Write 6 Done New Done false
14200 4604 Acked 5603 Write 8 Done New Done false
14201 4605 Acked 5604 Flush 0 Done New Done false
14202 4606 Acked 5605 Read 2 Done New Done false
14203 4607 Acked 5606 Read 10 Done New Done false
14204 4608 Acked 5607 Write 8 Done New Done false
14205 4609 Acked 5608 Write 5 Done New Done false
14206 4610 Acked 5609 Read 7 Done New Done false
14207 4611 Acked 5610 Read 1 Done New Done false
14208 4612 Acked 5611 Read 5 Done New Done false
14209 4613 Acked 5612 Write 10 Done New Done false
14210 4614 Acked 5613 Write 4 Done New Done false
14211 4615 Acked 5614 Write 5 Done New Done false
14212 4616 Acked 5615 Read 9 Done New Done false
14213 4617 Acked 5616 Write 2 Done New Done false
14214 4618 Acked 5617 Write 3 Done New Done false
14215 4619 Acked 5618 Write 3 Done New Done false
14216 4620 Acked 5619 Read 5 Done New Done false
14217 4621 Acked 5620 Read 1 Done New Done false
14218 4622 Acked 5621 Read 3 Done New Done false
14219 4623 Acked 5622 Read 3 Done New Done false
14220 4624 Acked 5623 Write 6 Done New Done false
14221 4625 Acked 5624 Write 7 Done New Done false
14222 4626 Acked 5625 Read 8 Done New Done false
14223 4627 Acked 5626 Read 9 Done New Done false
14224 4628 Acked 5627 Flush 0 Done New Done false
14225 4629 Acked 5628 Read 7 Done New Done false
14226 4630 Acked 5629 Read 8 Done New Done false
14227 4631 Acked 5630 Read 9 Done New Done false
14228 4632 Acked 5631 Read 2 Done New Done false
14229 4633 Acked 5632 Write 6 Done New Done false
14230 4634 Acked 5633 Flush 0 Done New Done false
14231 4635 Acked 5634 Write 5 Done New Done false
14232 4636 Acked 5635 Flush 0 Done New Done false
14233 4637 Acked 5636 Flush 0 Done New Done false
14234 4638 Acked 5637 Read 2 Done New Done false
14235 4639 Acked 5638 Flush 0 Done New Done false
14236 4640 Acked 5639 Read 9 Done New Done false
14237 4641 Acked 5640 Read 2 Done New Done false
14238 4642 Acked 5641 Read 7 Done New Done false
14239 4643 Acked 5642 Read 4 Done New Done false
14240 4644 Acked 5643 Read 3 Done New Done false
14241 4645 Acked 5644 Flush 0 Done New Done false
14242 4646 Acked 5645 Read 3 Done New Done false
14243 4647 Acked 5646 Write 2 Done New Done false
14244 4648 Acked 5647 Read 7 Done New Done false
14245 4649 Acked 5648 Read 8 Done New Done false
14246 4650 Acked 5649 Read 3 Done New Done false
14247 4651 Acked 5650 Write 5 Done New Done false
14248 4652 Acked 5651 Read 6 Done New Done false
14249 4653 Acked 5652 Write 5 Done New Done false
14250 4654 Acked 5653 Write 2 Done New Done false
14251 4655 Acked 5654 Write 2 Done New Done false
14252 4656 Acked 5655 Read 10 Done New Done false
14253 4657 Acked 5656 Write 1 Done New Done false
14254 4658 Acked 5657 Write 10 Done New Done false
14255 4659 Acked 5658 Write 4 Done New Done false
14256 4660 Acked 5659 Read 6 Done New Done false
14257 4661 Acked 5660 Read 5 Done New Done false
14258 4662 Acked 5661 Write 1 Done New Done false
14259 4663 Acked 5662 Write 4 Done New Done false
14260 4664 Acked 5663 Read 4 Done New Done false
14261 4665 Acked 5664 Read 7 Done New Done false
14262 4666 Acked 5665 Read 8 Done New Done false
14263 4667 Acked 5666 Read 7 Done New Done false
14264 4668 Acked 5667 Read 9 Done New Done false
14265 4669 Acked 5668 Read 3 Done New Done false
14266 4670 Acked 5669 Write 4 Done New Done false
14267 4671 Acked 5670 Read 5 Done New Done false
14268 4672 Acked 5671 Read 9 Done New Done false
14269 4673 Acked 5672 Read 7 Done New Done false
14270 4674 Acked 5673 Write 10 Done New Done false
14271 4675 Acked 5674 Write 6 Done New Done false
14272 4676 Acked 5675 Read 10 Done New Done false
14273 4677 Acked 5676 Write 5 Done New Done false
14274 4678 Acked 5677 Write 2 Done New Done false
14275 4679 Acked 5678 Write 1 Done New Done false
14276 4680 Acked 5679 Write 1 Done New Done false
14277 4681 Acked 5680 Write 7 Done New Done false
14278 4682 Acked 5681 Write 7 Done New Done false
14279 4683 Acked 5682 Read 4 Done New Done false
14280 4684 Acked 5683 Flush 0 Done New Done false
14281 4685 Acked 5684 Read 5 Done New Done false
14282 4686 Acked 5685 Write 1 Done New Done false
14283 4687 Acked 5686 Read 7 Done New Done false
14284 4688 Acked 5687 Read 9 Done New Done false
14285 4689 Acked 5688 Flush 0 Done New Done false
14286 4690 Acked 5689 Write 5 Done New Done false
14287 4691 Acked 5690 Read 6 Done New Done false
14288 4692 Acked 5691 Write 10 Done New Done false
14289 4693 Acked 5692 Write 4 Done New Done false
14290 4694 Acked 5693 Read 3 Done New Done false
14291 4695 Acked 5694 Read 4 Done New Done false
14292 4696 Acked 5695 Write 3 Done New Done false
14293 4697 Acked 5696 Write 10 Done New Done false
14294 4698 Acked 5697 Write 4 Done New Done false
14295 4699 Acked 5698 Read 1 Done New Done false
14296 4700 Acked 5699 Read 4 Done New Done false
14297 4701 Acked 5700 Write 7 Done New Done false
14298 4702 Acked 5701 Flush 0 Done New Done false
14299 4703 Acked 5702 Write 6 Done New Done false
14300 4704 Acked 5703 Write 8 Done New Done false
14301 4705 Acked 5704 Write 3 Done New Done false
14302 4706 Acked 5705 Read 6 Done New Done false
14303 4707 Acked 5706 Read 10 Done New Done false
14304 4708 Acked 5707 Read 7 Done New Done false
14305 4709 Acked 5708 Read 1 Done New Done false
14306 4710 Acked 5709 Flush 0 Done New Done false
14307 4711 Acked 5710 Read 9 Done New Done false
14308 4712 Acked 5711 Read 5 Done New Done false
14309 4713 Acked 5712 Flush 0 Done New Done false
14310 4714 Acked 5713 Write 2 Done New Done false
14311 4715 Acked 5714 Read 9 Done New Done false
14312 4716 Acked 5715 Write 8 Done New Done false
14313 4717 Acked 5716 Read 1 Done New Done false
14314 4718 Acked 5717 Read 10 Done New Done false
14315 4719 Acked 5718 Read 7 Done New Done false
14316 4720 Acked 5719 Read 9 Done New Done false
14317 4721 Acked 5720 Write 6 Done New Done false
14318 4722 Acked 5721 Read 6 Done New Done false
14319 4723 Acked 5722 Read 4 Done New Done false
14320 4724 Acked 5723 Read 1 Done New Done false
14321 4725 Acked 5724 Write 6 Done New Done false
14322 4726 Acked 5725 Read 8 Done New Done false
14323 4727 Acked 5726 Write 6 Done New Done false
14324 4728 Acked 5727 Write 4 Done New Done false
14325 4729 Acked 5728 Read 3 Done New Done false
14326 4730 Acked 5729 Write 2 Done New Done false
14327 4731 Acked 5730 Read 5 Done New Done false
14328 4732 Acked 5731 Write 5 Done New Done false
14329 4733 Acked 5732 Write 8 Done New Done false
14330 4734 Acked 5733 Write 3 Done New Done false
14331 4735 Acked 5734 Read 4 Done New Done false
14332 4736 Acked 5735 Read 8 Done New Done false
14333 4737 Acked 5736 Read 3 Done New Done false
14334 4738 Acked 5737 Write 2 Done New Done false
14335 4739 Acked 5738 Flush 0 Done New Done false
14336 4740 Acked 5739 Write 7 Done New Done false
14337 4741 Acked 5740 Write 4 Done New Done false
14338 4742 Acked 5741 Flush 0 Done New Done false
14339 4743 Acked 5742 Read 4 Done New Done false
14340 4744 Acked 5743 Read 5 Done New Done false
14341 4745 Acked 5744 Write 5 Done New Done false
14342 4746 Acked 5745 Read 1 Done New Done false
14343 4747 Acked 5746 Read 3 Done New Done false
14344 4748 Acked 5747 Read 6 Done New Done false
14345 4749 Acked 5748 Write 3 Done New Done false
14346 4750 Acked 5749 Write 10 Done New Done false
14347 4751 Acked 5750 Write 2 Done New Done false
14348 4752 Acked 5751 Write 5 Done New Done false
14349 4753 Acked 5752 Read 2 Done New Done false
14350 4754 Acked 5753 Read 10 Done New Done false
14351 4755 Acked 5754 Read 10 Done New Done false
14352 4756 Acked 5755 Read 3 Done New Done false
14353 4757 Acked 5756 Flush 0 Done New Done false
14354 4758 Acked 5757 Write 3 Done New Done false
14355 4759 Acked 5758 Read 1 Done New Done false
14356 4760 Acked 5759 Read 7 Done New Done false
14357 4761 Acked 5760 Read 7 Done New Done false
14358 4762 Acked 5761 Read 10 Done New Done false
14359 4763 Acked 5762 Write 6 Done New Done false
14360 4764 Acked 5763 Read 10 Done New Done false
14361 4765 Acked 5764 Write 2 Done New Done false
14362 4766 Acked 5765 Write 6 Done New Done false
14363 4767 Acked 5766 Read 9 Done New Done false
14364 4768 Acked 5767 Read 10 Done New Done false
14365 4769 Acked 5768 Read 5 Done New Done false
14366 4770 Acked 5769 Read 8 Done New Done false
14367 4771 Acked 5770 Read 10 Done New Done false
14368 4772 Acked 5771 Read 2 Done New Done false
14369 4773 Acked 5772 Read 5 Done New Done false
14370 4774 Acked 5773 Write 1 Done New Done false
14371 4775 Acked 5774 Flush 0 Done New Done false
14372 4776 Acked 5775 Read 2 Done New Done false
14373 4777 Acked 5776 Read 3 Done New Done false
14374 4778 Acked 5777 Read 4 Done New Done false
14375 4779 Acked 5778 Read 1 Done New Done false
14376 4780 Acked 5779 Write 1 Done New Done false
14377 4781 Acked 5780 Read 10 Done New Done false
14378 4782 Acked 5781 Write 9 Done New Done false
14379 4783 Acked 5782 Write 8 Done New Done false
14380 4784 Acked 5783 Read 6 Done New Done false
14381 4785 Acked 5784 Write 2 Done New Done false
14382 4786 Acked 5785 Read 8 Done New Done false
14383 4787 Acked 5786 Read 6 Done New Done false
14384 4788 Acked 5787 Read 9 Done New Done false
14385 4789 Acked 5788 Read 9 Done New Done false
14386 4790 Acked 5789 Write 7 Done New Done false
14387 4791 Acked 5790 Read 4 Done New Done false
14388 4792 Acked 5791 Read 9 Done New Done false
14389 4793 Acked 5792 Flush 0 Done New Done false
14390 4794 Acked 5793 Write 5 Done New Done false
14391 4795 Acked 5794 Read 4 Done New Done false
14392 4796 Acked 5795 Read 5 Done New Done false
14393 4797 Acked 5796 Flush 0 Done New Done false
14394 4798 Acked 5797 Read 7 Done New Done false
14395 4799 Acked 5798 Read 8 Done New Done false
14396 4800 Acked 5799 Read 2 Done New Done false
14397 4801 Acked 5800 Flush 0 Done New Done false
14398 4802 Acked 5801 Write 7 Done New Done false
14399 4803 Acked 5802 Write 2 Done New Done false
14400 4804 Acked 5803 Flush 0 Done New Done false
14401 4805 Acked 5804 Read 10 Done New Done false
14402 4806 Acked 5805 Read 5 Done New Done false
14403 4807 Acked 5806 Write 6 Done New Done false
14404 4808 Acked 5807 Read 5 Done New Done false
14405 4809 Acked 5808 Read 7 Done New Done false
14406 4810 Acked 5809 Read 8 Done New Done false
14407 4811 Acked 5810 Read 2 Done New Done false
14408 4812 Acked 5811 Flush 0 Done New Done false
14409 4813 Acked 5812 Flush 0 Done New Done false
14410 4814 Acked 5813 Write 9 Done New Done false
14411 4815 Acked 5814 Write 2 Done New Done false
14412 4816 Acked 5815 Write 1 Done New Done false
14413 4817 Acked 5816 Read 7 Done New Done false
14414 4818 Acked 5817 Read 4 Done New Done false
14415 4819 Acked 5818 Read 5 Done New Done false
14416 4820 Acked 5819 Flush 0 Done New Done false
14417 4821 Acked 5820 Write 7 Done New Done false
14418 4822 Acked 5821 Read 4 Done New Done false
14419 4823 Acked 5822 Write 10 Done New Done false
14420 4824 Acked 5823 Write 4 Done New Done false
14421 4825 Acked 5824 Write 2 Done New Done false
14422 4826 Acked 5825 Flush 0 Done New Done false
14423 4827 Acked 5826 Read 4 Done New Done false
14424 4828 Acked 5827 Read 2 Done New Done false
14425 4829 Acked 5828 Write 5 Done New Done false
14426 4830 Acked 5829 Read 8 Done New Done false
14427 4831 Acked 5830 Read 7 Done New Done false
14428 4832 Acked 5831 Read 8 Done New Done false
14429 4833 Acked 5832 Write 8 Done New Done false
14430 4834 Acked 5833 Read 3 Done New Done false
14431 4835 Acked 5834 Write 6 Done New Done false
14432 4836 Acked 5835 Flush 0 Done New Done false
14433 4837 Acked 5836 Write 3 Done New Done false
14434 4838 Acked 5837 Write 6 Done New Done false
14435 4839 Acked 5838 Flush 0 Done New Done false
14436 4840 Acked 5839 Read 10 Done New Done false
14437 4841 Acked 5840 Flush 0 Done New Done false
14438 STATES DS:0 DS:1 DS:2 TOTAL
14439 New 0 303 0 303
14440 Sent 0 0 0 0
14441 Done 303 0 303 606
14442 Skipped 0 0 0 0
14443 Error 0 0 0 0
14444 Last Flush: 5840 5537 5840
14445 Downstairs last five completed: 5537 5536 5535 5534 5533 5532
14446 Upstairs last five completed: 4841 4840 4839 4838 4837 4836
14447 CLIENT: Up:0 ds:303 act:2
144482023-09-22T23:26:53.560ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
144492023-09-22T23:26:53.560ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
144502023-09-22T23:26:53.560ZINFOcrucible: [1] client re-new 303 jobs since flush 5537 = downstairs
144512023-09-22T23:26:53.561ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
144522023-09-22T23:26:53.562ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
144532023-09-22T23:26:53.562ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
144542023-09-22T23:26:53.562ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
144552023-09-22T23:26:53.562ZINFOcrucible: [1] send last flush ID to this DS: 5537
144562023-09-22T23:26:53.562ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
144572023-09-22T23:26:53.562ZINFOcrucible: [1] Replied this last flush ID: 5537
144582023-09-22T23:26:53.562ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
144592023-09-22T23:26:53.562ZINFOcrucible: [1] Transition from Offline to Replay
144602023-09-22T23:26:53.562ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
144612023-09-22T23:26:53.562ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
144622023-09-22T23:26:53.562ZINFOcrucible: [1] Transition from Replay to Active
144632023-09-22T23:26:53.562ZINFOcrucible: [1] Starts cmd_loop
144642023-09-22T23:26:53.562ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
144652023-09-22T23:26:53.562ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
144662023-09-22T23:26:53.563ZINFOcrucible: Set check for repair
144672023-09-22T23:26:54.563ZINFOcrucible: Checking if live repair is needed
144682023-09-22T23:26:54.563ZWARNcrucible: [1] flow control end
144692023-09-22T23:26:54.563ZINFOcrucible: No Live Repair required at this time
14470 ----------------------------------------------------------------
14471 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14472 Downstairs last five completed: 5840 5839 5838 5837 5836 5835
14473 Upstairs last five completed: 4841 4840 4839 4838 4837 4836
14474 CLIENT: Up:0 ds:0 act:3
14475 Replay: All jobs finished, all DS active.
14476 Main task has work to do, go find it
14477 got dsc Stop(0)
14478 stop 0
14479 Sep 22 23:26:57.488 INFO request completed, latency_us: 668, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 09e5fb39-ed11-418c-8190-727d87e5a6a6, remote_addr: 127.0.0.1:52471, local_addr: 127.0.[8810] Got stop action so:false kr:false
14480 0.1:9998
14481 Sep 22 23:26:57.489 INFO request completed, latency_us: 289, response_code: 200, uri: /state/cid/0, method: GET, req_id: d3150982-d39a-4970-8395-cded5ff73cfc, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
144822023-09-22T23:26:57.489ZWARNcrucible: [0] downstairs disconnected
144832023-09-22T23:26:57.489ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
144842023-09-22T23:26:57.489ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
144852023-09-22T23:26:57.490ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
144862023-09-22T23:26:57.490ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
144872023-09-22T23:26:57.490ZWARNcrucible: [0] pm_task rx.recv() is None
144882023-09-22T23:26:57.490ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
14489 [8810] Exited with: ExitStatus(unix_wait_status(9))
14490 [8810][0] reports Exit
144912023-09-22T23:26:59.492ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
14492 Sep 22 23:27:01.492 INFO request completed, latency_us: 283, response_code: 200, uri: /state/cid/0, method: GET, req_id: 80afd329-de81-432d-8604-f693e7a19472, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
14493 001/300 Read block 3896 len 32768
14494 002/300 Flush
14495 003/300 Write block 2518 len 36864 data: 5 2 2 2 3 3 3 5 6
14496 004/300 Flush
14497 005/300 Write block 3984 len 20480 data: 4 5 5 6 6
14498 006/300 Write block 3505 len 4096 data: 3
14499 007/300 Read block 976 len 28672
14500 008/300 Write block 1878 len 24576 data: 3 3 4 3 3 3
14501 009/300 Write block 68 len 4096 data: 5
14502 010/300 Read block 3513 len 12288
14503 011/300 Read block 1151 len 4096
14504 012/300 Read block 3450 len 40960
14505 013/300 Read block 779 len 32768
14506 014/300 Write block 1871 len 24576 data: 3 4 5 5 4 3
14507 015/300 Read block 4734 len 4096
14508 016/300 Read block 679 len 32768
14509 017/300 Write block 2305 len 12288 data: 4 4 4
14510 018/300 Read block 97 len 36864
14511 019/300 Write block 790 len 20480 data: 3 3 4 5 5
14512 020/300 Write block 4137 len 4096 data: 4
14513 021/300 Write block 970 len 36864 data: 4 3 3 3 3 3 4 4 3
14514 022/300 Flush
14515 023/300 Write block 2968 len 36864 data: 3 3 3 3 3 3 3 4 4
14516 024/300 Write block 2261 len 24576 data: 4 3 2 2 2 2
14517 025/300 Write block 170 len 24576 data: 4 4 4 4 6 5
14518 026/300 Read block 4809 len 40960
14519 027/300 Read block 3913 len 16384
14520 028/300 Read block 4995 len 8192
14521 029/300 Write block 3648 len 24576 data: 5 5 5 4 4 5
14522 030/300 Write block 1435 len 8192 data: 6 5
14523 031/300 Read block 756 len 28672
14524 032/300 Write block 2350 len 24576 data: 6 6 6 6 4 3
14525 033/300 Read block 4751 len 4096
14526 034/300 Read block 4278 len 8192
14527 035/300 Read block 578 len 4096
14528 036/300 Write block 4905 len 24576 data: 5 5 5 6 5 7
14529 037/300 Read block 4334 len 16384
14530 038/300 Read block 423 len 40960
14531 039/300 Write block 4887 len 28672 data: 3 3 4 4 5 2 2
14532 040/300 Flush
14533 041/300 Read block 2592 len 20480
14534 042/300 Write block 1574 len 12288 data: 4 4 2
14535 043/300 Flush
14536 044/300 Read block 949 len 28672
14537 045/300 Read block 2052 len 24576
14538 046/300 Read block 3058 len 12288
14539 047/300 Read block 800 len 12288
14540 048/300 Write block 4518 len 16384 data: 3 3 3 3
14541 049/300 Write block 3728 len 4096 data: 5
14542 050/300 Write block 3755 len 20480 data: 3 3 3 3 3
14543 051/300 Flush
14544 052/300 Read block 506 len 24576
14545 053/300 Read block 262 len 36864
14546 054/300 Read block 657 len 36864
14547 055/300 Read block 4637 len 16384
14548 056/300 Flush
14549 057/300 Write block 261 len 12288 data: 5 4 5
14550 058/300 Read block 699 len 28672
14551 059/300 Write block 1753 len 28672 data: 4 4 3 3 4 4 4
14552 060/300 Write block 1771 len 40960 data: 8 7 6 6 6 5 4 4 5 4
14553 061/300 Read block 1802 len 24576
14554 062/300 Read block 2391 len 4096
14555 063/300 Read block 3112 len 24576
14556 064/300 Read block 3718 len 40960
14557 065/300 Read block 982 len 12288
14558 066/300 Read block 3917 len 16384
14559 067/300 Flush
14560 068/300 Write block 732 len 32768 data: 5 4 3 3 3 3 4 4
14561 069/300 Read block 1972 len 40960
14562 070/300 Write block 4442 len 24576 data: 3 2 4 3 3 3
14563 071/300 Read block 462 len 28672
14564 072/300 Read block 3549 len 24576
14565 073/300 Read block 428 len 24576
14566 074/300 Read block 2096 len 12288
14567 075/300 Write block 989 len 12288 data: 7 6 6
14568 076/300 Read block 674 len 20480
14569 077/300 Read block 2856 len 16384
14570 078/300 Write block 4529 len 16384 data: 4 5 3 4
14571 079/300 Flush
14572 080/300 Read block 2502 len 36864
14573 081/300 Read block 4407 len 36864
14574 082/300 Write block 1289 len 32768 data: 2 4 4 5 6 6 6 7
14575 083/300 Read block 1178 len 8192
14576 084/300 Read block 161 len 28672
14577 085/300 Write block 1534 len 24576 data: 3 3 3 3 3 3
14578 086/300 Read block 298 len 24576
14579 087/300 Read block 3744 len 20480
14580 088/300 Flush
14581 089/300 Read block 3768 len 36864
14582 090/300 Write block 3632 len 20480 data: 2 3 3 3 2
14583 091/300 Write block 382 len 16384 data: 5 5 6 5
14584 092/300 Write block 3644 len 40960 data: 2 2 2 4 6 6 6 5 5 6
14585 093/300 Read block 3006 len 36864
14586 094/300 Write block 366 len 16384 data: 2 2 2 3
14587 095/300 Write block 766 len 16384 data: 4 3 5 6
14588 096/300 Write block 4280 len 4096 data: 6
14589 097/300 Write block 1335 len 40960 data: 4 5 5 5 5 5 5 5 4 4
14590 098/300 Write block 1682 len 36864 data: 4 6 6 4 4 4 3 3 3
14591 099/300 Read block 3929 len 16384
14592 100/300 Write block 4984 len 32768 data: 7 7 7 8 9 8 7 7
14593 101/300 Write block 3702 len 24576 data: 7 7 7 7 7 7
14594 102/300 Read block 4427 len 4096
14595 103/300 Read block 396 len 40960
14596 104/300 Read block 1327 len 20480
14597 105/300 Write block 2405 len 8192 data: 4 4
14598 106/300 Read block 1172 len 28672
14599 107/300 Flush
14600 108/300 Write block 4014 len 32768 data: 2 2 2 3 3 4 4 4
14601 109/300 Read block 3975 len 12288
14602 110/300 Flush
14603 111/300 Write block 3610 len 32768 data: 4 5 5 6 6 6 4 5
14604 112/300 Read block 4253 len 16384
14605 113/300 Write block 3165 len 40960 data: 3 3 3 4 4 3 3 3 2 3
14606 114/300 Read block 4581 len 24576
14607 115/300 Write block 2489 len 16384 data: 2 2 2 2
14608 116/300 Read block 2688 len 36864
14609 117/300 Read block 862 len 24576
14610 118/300 Write block 2474 len 20480 data: 3 4 6 5 4
14611 119/300 Read block 2104 len 16384
14612 120/300 Read block 4211 len 28672
14613 121/300 Write block 1700 len 40960 data: 4 4 3 2 3 3 3 3 5 6
14614 122/300 Read block 224 len 4096
14615 123/300 Write block 4745 len 32768 data: 5 4 5 4 3 3 3 3
14616 124/300 Read block 1891 len 28672
14617 125/300 Write block 405 len 28672 data: 6 5 5 6 6 6 6
14618 126/300 Read block 3592 len 4096
14619 127/300 Write block 847 len 36864 data: 4 4 4 3 5 5 4 4 4
14620 128/300 Read block 4334 len 40960
14621 129/300 Read block 55 len 4096
14622 130/300 Write block 2857 len 40960 data: 3 3 3 4 5 4 3 3 3 3
14623 131/300 Flush
14624 132/300 Read block 2904 len 28672
14625 133/300 Read block 1492 len 12288
14626 134/300 Flush
14627 135/300 Write block 1410 len 36864 data: 5 5 5 5 5 5 5 5 5
14628 136/300 Read block 1911 len 40960
14629 137/300 Write block 4240 len 32768 data: 2 2 2 3 3 6 6 7
14630 138/300 Read block 2952 len 32768
14631 139/300 Write block 2772 len 16384 data: 3 3 4 4
14632 140/300 Write block 2426 len 16384 data: 5 5 5 5
14633 141/300 Read block 2701 len 12288
14634 142/300 Read block 4417 len 36864
14635 143/300 Flush
14636 144/300 Write block 4246 len 32768 data: 7 8 7 7 6 5 4 3
14637 145/300 Write block 2176 len 12288 data: 3 4 3
14638 146/300 Read block 2935 len 20480
14639 147/300 Read block 3779 len 8192
14640 148/300 Read block 4589 len 8192
14641 149/300 Write block 4845 len 36864 data: 4 4 5 6 6 7 6 6 7
14642 150/300 Flush
14643 151/300 Write block 4969 len 16384 data: 5 4 4 4
14644 152/300 Read block 4721 len 4096
14645 153/300 Write block 2697 len 28672 data: 4 3 4 4 4 4 4
14646 154/300 Read block 3818 len 28672
14647 155/300 Read block 1729 len 24576
14648 156/300 Write block 2630 len 12288 data: 3 4 3
14649 157/300 Read block 4522 len 16384
14650 158/300 Read block 3689 len 40960
14651 159/300 Write block 2783 len 20480 data: 5 5 5 5 4
14652 160/300 Read block 1003 len 16384
14653 161/300 Read block 3055 len 12288
14654 162/300 Write block 1328 len 4096 data: 6
14655 163/300 Read block 3483 len 8192
14656 164/300 Read block 3969 len 4096
14657 165/300 Read block 4607 len 20480
14658 166/300 Flush
14659 167/300 Write block 1002 len 36864 data: 6 6 3 3 4 4 3 2 2
14660 168/300 Write block 4198 len 16384 data: 4 4 4 4
14661 169/300 Read block 1898 len 40960
14662 170/300 Write block 1485 len 28672 data: 4 3 4 4 4 4 4
14663 171/300 Read block 397 len 12288
14664 172/300 Read block 4495 len 20480
14665 173/300 Read block 4075 len 20480
14666 174/300 Write block 4806 len 36864 data: 5 4 5 5 5 4 4 5 4
14667 175/300 Write block 3207 len 28672 data: 4 4 3 4 3 3 3
14668 176/300 Flush
14669 177/300 Write block 3973 len 12288 data: 4 4 6
14670 178/300 Read block 3798 len 4096
14671 179/300 Read block 794 len 20480
14672 180/300 Read block 3213 len 12288
14673 181/300 Write block 3040 len 32768 data: 5 4 4 5 4 4 4 6
14674 182/300 Write block 3262 len 8192 data: 6 4
14675 183/300 Write block 4348 len 36864 data: 5 4 4 5 5 5 5 5 3
14676 184/300 Read block 1074 len 32768
14677 185/300 Read block 740 len 12288
14678 186/300 Write block 802 len 20480 data: 5 5 6 6 7
14679 187/300 Write block 1901 len 32768 data: 4 4 4 4 4 4 4 3
14680 188/300 Read block 4224 len 16384
14681 189/300 Flush
14682 190/300 Read block 4983 len 28672
14683 191/300 Write block 1414 len 36864 data: 6 6 6 6 6 5 4 5 4
14684 192/300 Read block 2141 len 12288
14685 193/300 Read block 1640 len 36864
14686 194/300 Read block 2027 len 20480
14687 195/300 Read block 2778 len 40960
14688 196/300 Write block 1439 len 4096 data: 4
14689 197/300 Write block 2644 len 28672 data: 3 2 2 3 3 3 3
14690 198/300 Write block 1023 len 24576 data: 3 3 3 3 4 3
14691 199/300 Flush
14692 200/300 Write block 1556 len 12288 data: 6 6 5
14693 201/300 Write block 2489 len 40960 data: 3 3 3 3 2 2 2 2 2 2
14694 202/300 Read block 4113 len 32768
14695 203/300 Read block 853 len 40960
14696 204/300 Read block 4195 len 36864
14697 205/300 Write block 4787 len 12288 data: 5 5 4
14698 206/300 Flush
14699 207/300 Read block 4650 len 28672
14700 208/300 Flush
14701 209/300 Flush
14702 210/300 Write block 4909 len 40960 data: 6 8 7 7 7 7 7 7 7 5
14703 211/300 Write block 1864 len 12288 data: 6 6 5
14704 212/300 Write block 4122 len 24576 data: 3 2 3 4 4 5
14705 213/300 Flush
14706 214/300 Read block 4965 len 4096
14707 215/300 Write block 1388 len 24576 data: 2 3 3 2 2 2
14708 216/300 Read block 1513 len 20480
14709 217/300 Write block 2838 len 8192 data: 4 4
14710 218/300 Write block 2135 len 32768 data: 3 3 3 5 5 4 3 3
14711 219/300 Write block 3125 len 32768 data: 7 7 5 5 3 2 2 2
14712 220/300 Flush
14713 221/300 Read block 2011 len 40960
14714 222/300 Flush
14715 223/300 Read block 2624 len 8192
14716 224/300 Read block 4843 len 40960
14717 225/300 Write block 1526 len 16384 data: 3 4 5 4
14718 226/300 Write block 3896 len 40960 data: 6 7 8 6 6 6 5 6 6 6
14719 227/300 Write block 4583 len 16384 data: 3 3 3 4
14720 228/300 Flush
14721 229/300 Read block 2052 len 4096
14722 230/300 Write block 1136 len 20480 data: 2 2 2 4 4
14723 231/300 Read block 871 len 8192
14724 232/300 Write block 2779 len 40960 data: 4 3 2 4 6 6 6 6 5 4
14725 233/300 Read block 2103 len 20480
14726 234/300 Write block 2658 len 20480 data: 4 3 3 3 4
14727 235/300 Write block 3678 len 8192 data: 3 2
14728 236/300 Read block 4934 len 4096
14729 237/300 Read block 1027 len 4096
14730 238/300 Write block 1204 len 28672 data: 6 7 8 9 9 9 6
14731 239/300 Read block 2256 len 32768
14732 240/300 Write block 3802 len 36864 data: 5 5 5 4 5 5 5 5 5
14733 241/300 Read block 4274 len 40960
14734 242/300 Write block 1090 len 8192 data: 3 3
14735 243/300 Flush
14736 244/300 Read block 4379 len 20480
14737 245/300 Read block 4379 len 40960
14738 246/300 Write block 1878 len 20480 data: 4 4 5 4 4
14739 247/300 Read block 2824 len 32768
14740 248/300 Read block 2668 len 12288
14741 249/300 Read block 4965 len 36864
14742 250/300 Write block 1493 len 32768 data: 3 3 4 5 6 5 6 7
14743 251/300 Read block 4291 len 12288
14744 252/300 Read block 590 len 12288
14745 253/300 Write block 1861 len 8192 data: 5 6
14746 254/300 Read block 2250 len 36864
14747 255/300 Write block 1533 len 4096 data: 3
14748 256/300 Write block 103 len 16384 data: 2 2 2 3
14749 257/300 Read block 4432 len 8192
14750 258/300 Flush
14751 259/300 Flush
14752 260/300 Read block 2970 len 8192
14753 261/300 Write block 3895 len 4096 data: 6
14754 262/300 Write block 1290 len 40960 data: 5 5 6 7 7 7 8 8 8 5
14755 263/300 Read block 1662 len 24576
14756 264/300 Write block 2514 len 8192 data: 8 8
14757 265/300 Write block 239 len 36864 data: 4 4 3 3 2 3 3 5 5
14758 266/300 Read block 2505 len 24576
14759 267/300 Read block 1642 len 20480
14760 268/300 Read block 491 len 4096
14761 269/300 Write block 2088 len 4096 data: 3
14762 270/300 Read block 748 len 24576
14763 271/300 Read block 371 len 4096
14764 272/300 Read block 1499 len 20480
14765 273/300 Write block 56 len 32768 data: 5 6 5 6 6 6 7 6
14766 274/300 Read block 3782 len 12288
14767 275/300 Write block 3167 len 40960 data: 4 5 5 4 4 4 3 4 3 4
14768 276/300 Read block 3299 len 24576
14769 277/300 Read block 462 len 12288
14770 278/300 Read block 2281 len 16384
14771 279/300 Flush
14772 280/300 Write block 3034 len 20480 data: 3 3 3 3 3
14773 281/300 Write block 945 len 20480 data: 2 2 3 4 4
14774 282/300 Flush
14775 283/300 Write block 1760 len 4096 data: 5
14776 284/300 Write block 4761 len 16384 data: 5 8 7 6
14777 285/300 Write block 1533 len 8192 data: 4 4
14778 286/300 Read block 2983 len 36864
14779 287/300 Write block 3185 len 8192 data: 5 4
14780 288/300 Read block 2847 len 20480
14781 289/300 Write block 526 len 40960 data: 4 3 3 3 3 3 2 2 2 2
14782 290/300 Read block 2442 len 16384
14783 291/300 Flush
14784 292/300 Read block 2732 len 8192
14785 293/300 Read block 1543 len 32768
14786 294/300 Flush
14787 295/300 Write block 396 len 20480 data: 3 3 4 5 5
14788 296/300 Read block 4452 len 28672
14789 297/300 Write block 1084 len 4096 data: 3
14790 298/300 Read block 4731 len 24576
14791 299/300 Write block 2209 len 12288 data: 3 2 2
14792 300/300 Read block 386 len 4096
14793 Main task has work to do, go find it
14794 got dsc Start(0)
14795 start 0
14796 Sep 22 23:27:02.414 INFO request completed, latency_us: 307, response_code: 204, uri: /start/cid/0, method: POST, req_id: 4943296e-f164-4484-b642-e32db8496572, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
14797 Replay: started 0, returned:Ok(())
14798 ----------------------------------------------------------------
14799 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14800 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
14801 4842 Acked 5841 Read 8 New Done Done false
14802 4843 Acked 5842 Flush 0 New Done Done false
14803 4844 Acked 5843 Write 9 New Done Done false
14804 4845 Acked 5844 Flush 0 New Done Done false
14805 4846 Acked 5845 Write 5 New Done Done false
14806 4847 Acked 5846 Write 1 New Done Done false
14807 4848 Acked 5847 Read 7 New Done Done false
14808 4849 Acked 5848 Write 6 New Done Done false
14809 4850 Acked 5849 Write 1 New Done Done false
14810 4851 Acked 5850 Read 3 New Done Done false
14811 4852 Acked 5851 Read 1 New Done Done false
14812 4853 Acked 5852 Read 10 New Done Done false
14813 4854 Acked 5853 Read 8 New Done Done false
14814 4855 Acked 5854 Write 6 New Done Done false
14815 4856 Acked 5855 Read 1 New Done Done false
14816 4857 Acked 5856 Read 8 New Done Done false
14817 4858 Acked 5857 Write 3 New Done Done false
14818 4859 Acked 5858 Read 9 New Done Done false
14819 4860 Acked 5859 Write 5 New Done Done false
14820 4861 Acked 5860 Write 1 New Done Done false
14821 4862 Acked 5861 Write 9 New Done Done false
14822 4863 Acked 5862 Flush 0 New Done Done false
14823 4864 Acked 5863 Write 9 New Done Done false
14824 4865 Acked 5864 Write 6 New Done Done false
14825 4866 Acked 5865 Write 6 New Done Done false
14826 4867 Acked 5866 Read 10 New Done Done false
14827 4868 Acked 5867 Read 4 New Done Done false
14828 4869 Acked 5868 Read 2 New Done Done false
14829 4870 Acked 5869 Write 6 New Done Done false
14830 4871 Acked 5870 Write 2 New Done Done false
14831 4872 Acked 5871 Read 7 New Done Done false
14832 4873 Acked 5872 Write 6 New Done Done false
14833 4874 Acked 5873 Read 1 New Done Done false
14834 4875 Acked 5874 Read 2 New Done Done false
14835 4876 Acked 5875 Read 1 New Done Done false
14836 4877 Acked 5876 Write 6 New Done Done false
14837 4878 Acked 5877 Read 4 New Done Done false
14838 4879 Acked 5878 Read 10 New Done Done false
14839 4880 Acked 5879 Write 7 New Done Done false
14840 4881 Acked 5880 Flush 0 New Done Done false
14841 4882 Acked 5881 Read 5 New Done Done false
14842 4883 Acked 5882 Write 3 New Done Done false
14843 4884 Acked 5883 Flush 0 New Done Done false
14844 4885 Acked 5884 Read 7 New Done Done false
14845 4886 Acked 5885 Read 6 New Done Done false
14846 4887 Acked 5886 Read 3 New Done Done false
14847 4888 Acked 5887 Read 3 New Done Done false
14848 4889 Acked 5888 Write 4 New Done Done false
14849 4890 Acked 5889 Write 1 New Done Done false
14850 4891 Acked 5890 Write 5 New Done Done false
14851 4892 Acked 5891 Flush 0 New Done Done false
14852 4893 Acked 5892 Read 6 New Done Done false
14853 4894 Acked 5893 Read 9 New Done Done false
14854 4895 Acked 5894 Read 9 New Done Done false
14855 4896 Acked 5895 Read 4 New Done Done false
14856 4897 Acked 5896 Flush 0 New Done Done false
14857 4898 Acked 5897 Write 3 New Done Done false
14858 4899 Acked 5898 Read 7 New Done Done false
14859 4900 Acked 5899 Write 7 New Done Done false
14860 4901 Acked 5900 Write 10 New Done Done false
14861 4902 Acked 5901 Read 6 New Done Done false
14862 4903 Acked 5902 Read 1 New Done Done false
14863 4904 Acked 5903 Read 6 New Done Done false
14864 4905 Acked 5904 Read 10 New Done Done false
14865 4906 Acked 5905 Read 3 New Done Done false
14866 4907 Acked 5906 Read 4 New Done Done false
14867 4908 Acked 5907 Flush 0 New Done Done false
14868 4909 Acked 5908 Write 8 New Done Done false
14869 4910 Acked 5909 Read 10 New Done Done false
14870 4911 Acked 5910 Write 6 New Done Done false
14871 4912 Acked 5911 Read 7 New Done Done false
14872 4913 Acked 5912 Read 6 New Done Done false
14873 4914 Acked 5913 Read 6 New Done Done false
14874 4915 Acked 5914 Read 3 New Done Done false
14875 4916 Acked 5915 Write 3 New Done Done false
14876 4917 Acked 5916 Read 5 New Done Done false
14877 4918 Acked 5917 Read 4 New Done Done false
14878 4919 Acked 5918 Write 4 New Done Done false
14879 4920 Acked 5919 Flush 0 New Done Done false
14880 4921 Acked 5920 Read 9 New Done Done false
14881 4922 Acked 5921 Read 9 New Done Done false
14882 4923 Acked 5922 Write 8 New Done Done false
14883 4924 Acked 5923 Read 2 New Done Done false
14884 4925 Acked 5924 Read 7 New Done Done false
14885 4926 Acked 5925 Write 6 New Done Done false
14886 4927 Acked 5926 Read 6 New Done Done false
14887 4928 Acked 5927 Read 5 New Done Done false
14888 4929 Acked 5928 Flush 0 New Done Done false
14889 4930 Acked 5929 Read 9 New Done Done false
14890 4931 Acked 5930 Write 5 New Done Done false
14891 4932 Acked 5931 Write 4 New Done Done false
14892 4933 Acked 5932 Write 10 New Done Done false
14893 4934 Acked 5933 Read 9 New Done Done false
14894 4935 Acked 5934 Write 4 New Done Done false
14895 4936 Acked 5935 Write 4 New Done Done false
14896 4937 Acked 5936 Write 1 New Done Done false
14897 4938 Acked 5937 Write 10 New Done Done false
14898 4939 Acked 5938 Write 9 New Done Done false
14899 4940 Acked 5939 Read 4 New Done Done false
14900 4941 Acked 5940 Write 8 New Done Done false
14901 4942 Acked 5941 Write 6 New Done Done false
14902 4943 Acked 5942 Read 1 New Done Done false
14903 4944 Acked 5943 Read 10 New Done Done false
14904 4945 Acked 5944 Read 5 New Done Done false
14905 4946 Acked 5945 Write 2 New Done Done false
14906 4947 Acked 5946 Read 7 New Done Done false
14907 4948 Acked 5947 Flush 0 New Done Done false
14908 4949 Acked 5948 Write 8 New Done Done false
14909 4950 Acked 5949 Read 3 New Done Done false
14910 4951 Acked 5950 Flush 0 New Done Done false
14911 4952 Acked 5951 Write 8 New Done Done false
14912 4953 Acked 5952 Read 4 New Done Done false
14913 4954 Acked 5953 Write 10 New Done Done false
14914 4955 Acked 5954 Read 6 New Done Done false
14915 4956 Acked 5955 Write 4 New Done Done false
14916 4957 Acked 5956 Read 9 New Done Done false
14917 4958 Acked 5957 Read 6 New Done Done false
14918 4959 Acked 5958 Write 5 New Done Done false
14919 4960 Acked 5959 Read 4 New Done Done false
14920 4961 Acked 5960 Read 7 New Done Done false
14921 4962 Acked 5961 Write 10 New Done Done false
14922 4963 Acked 5962 Read 1 New Done Done false
14923 4964 Acked 5963 Write 8 New Done Done false
14924 4965 Acked 5964 Read 7 New Done Done false
14925 4966 Acked 5965 Write 7 New Done Done false
14926 4967 Acked 5966 Read 1 New Done Done false
14927 4968 Acked 5967 Write 9 New Done Done false
14928 4969 Acked 5968 Read 10 New Done Done false
14929 4970 Acked 5969 Flush 0 New Done Done false
14930 4971 Acked 5970 Read 1 New Done Done false
14931 4972 Acked 5971 Write 10 New Done Done false
14932 4973 Acked 5972 Flush 0 New Done Done false
14933 4974 Acked 5973 Read 7 New Done Done false
14934 4975 Acked 5974 Read 3 New Done Done false
14935 4976 Acked 5975 Flush 0 New Done Done false
14936 4977 Acked 5976 Write 9 New Done Done false
14937 4978 Acked 5977 Read 10 New Done Done false
14938 4979 Acked 5978 Write 8 New Done Done false
14939 4980 Acked 5979 Read 8 New Done Done false
14940 4981 Acked 5980 Write 4 New Done Done false
14941 4982 Acked 5981 Write 4 New Done Done false
14942 4983 Acked 5982 Read 3 New Done Done false
14943 4984 Acked 5983 Read 9 New Done Done false
14944 4985 Acked 5984 Flush 0 New Done Done false
14945 4986 Acked 5985 Write 8 New Done Done false
14946 4987 Acked 5986 Write 3 New Done Done false
14947 4988 Acked 5987 Read 5 New Done Done false
14948 4989 Acked 5988 Read 2 New Done Done false
14949 4990 Acked 5989 Read 2 New Done Done false
14950 4991 Acked 5990 Write 9 New Done Done false
14951 4992 Acked 5991 Flush 0 New Done Done false
14952 4993 Acked 5992 Write 4 New Done Done false
14953 4994 Acked 5993 Read 1 New Done Done false
14954 4995 Acked 5994 Write 7 New Done Done false
14955 4996 Acked 5995 Read 7 New Done Done false
14956 4997 Acked 5996 Read 6 New Done Done false
14957 4998 Acked 5997 Write 3 New Done Done false
14958 4999 Acked 5998 Read 4 New Done Done false
14959 5000 Acked 5999 Read 10 New Done Done false
14960 5001 Acked 6000 Write 5 New Done Done false
14961 5002 Acked 6001 Read 4 New Done Done false
14962 5003 Acked 6002 Read 3 New Done Done false
14963 5004 Acked 6003 Write 1 New Done Done false
14964 5005 Acked 6004 Read 2 New Done Done false
14965 5006 Acked 6005 Read 1 New Done Done false
14966 5007 Acked 6006 Read 5 New Done Done false
14967 5008 Acked 6007 Flush 0 New Done Done false
14968 5009 Acked 6008 Write 9 New Done Done false
14969 5010 Acked 6009 Write 4 New Done Done false
14970 5011 Acked 6010 Read 10 New Done Done false
14971 5012 Acked 6011 Write 7 New Done Done false
14972 5013 Acked 6012 Read 3 New Done Done false
14973 5014 Acked 6013 Read 5 New Done Done false
14974 5015 Acked 6014 Read 5 New Done Done false
14975 5016 Acked 6015 Write 9 New Done Done false
14976 5017 Acked 6016 Write 7 New Done Done false
14977 5018 Acked 6017 Flush 0 New Done Done false
14978 5019 Acked 6018 Write 3 New Done Done false
14979 5020 Acked 6019 Read 1 New Done Done false
14980 5021 Acked 6020 Read 5 New Done Done false
14981 5022 Acked 6021 Read 3 New Done Done false
14982 5023 Acked 6022 Write 8 New Done Done false
14983 5024 Acked 6023 Write 2 New Done Done false
14984 5025 Acked 6024 Write 9 New Done Done false
14985 5026 Acked 6025 Read 8 New Done Done false
14986 5027 Acked 6026 Read 3 New Done Done false
14987 5028 Acked 6027 Write 5 New Done Done false
14988 5029 Acked 6028 Write 8 New Done Done false
14989 5030 Acked 6029 Read 4 New Done Done false
14990 5031 Acked 6030 Flush 0 New Done Done false
14991 5032 Acked 6031 Read 7 New Done Done false
14992 5033 Acked 6032 Write 9 New Done Done false
14993 5034 Acked 6033 Read 3 New Done Done false
14994 5035 Acked 6034 Read 9 New Done Done false
14995 5036 Acked 6035 Read 5 New Done Done false
14996 5037 Acked 6036 Read 10 New Done Done false
14997 5038 Acked 6037 Write 1 New Done Done false
14998 5039 Acked 6038 Write 7 New Done Done false
14999 5040 Acked 6039 Write 6 New Done Done false
15000 5041 Acked 6040 Flush 0 New Done Done false
15001 5042 Acked 6041 Write 3 New Done Done false
15002 5043 Acked 6042 Write 10 New Done Done false
15003 5044 Acked 6043 Read 8 New Done Done false
15004 5045 Acked 6044 Read 10 New Done Done false
15005 5046 Acked 6045 Read 9 New Done Done false
15006 5047 Acked 6046 Write 3 New Done Done false
15007 5048 Acked 6047 Flush 0 New Done Done false
15008 5049 Acked 6048 Read 7 New Done Done false
15009 5050 Acked 6049 Flush 0 New Done Done false
15010 5051 Acked 6050 Flush 0 New Done Done false
15011 5052 Acked 6051 Write 10 New Done Done false
15012 5053 Acked 6052 Write 3 New Done Done false
15013 5054 Acked 6053 Write 6 New Done Done false
15014 5055 Acked 6054 Flush 0 New Done Done false
15015 5056 Acked 6055 Read 1 New Done Done false
15016 5057 Acked 6056 Write 6 New Done Done false
15017 5058 Acked 6057 Read 5 New Done Done false
15018 5059 Acked 6058 Write 2 New Done Done false
15019 5060 Acked 6059 Write 8 New Done Done false
15020 5061 Acked 6060 Write 8 New Done Done false
15021 5062 Acked 6061 Flush 0 New Done Done false
15022 5063 Acked 6062 Read 10 New Done Done false
15023 5064 Acked 6063 Flush 0 New Done Done false
15024 5065 Acked 6064 Read 2 New Done Done false
15025 5066 Acked 6065 Read 10 New Done Done false
15026 5067 Acked 6066 Write 4 New Done Done false
15027 5068 Acked 6067 Write 10 New Done Done false
15028 5069 Acked 6068 Write 4 New Done Done false
15029 5070 Acked 6069 Flush 0 New Done Done false
15030 5071 Acked 6070 Read 1 New Done Done false
15031 5072 Acked 6071 Write 5 New Done Done false
15032 5073 Acked 6072 Read 2 New Done Done false
15033 5074 Acked 6073 Write 10 New Done Done false
15034 5075 Acked 6074 Read 5 New Done Done false
15035 5076 Acked 6075 Write 5 New Done Done false
15036 5077 Acked 6076 Write 2 New Done Done false
15037 5078 Acked 6077 Read 1 New Done Done false
15038 5079 Acked 6078 Read 1 New Done Done false
15039 5080 Acked 6079 Write 7 New Done Done false
15040 5081 Acked 6080 Read 8 New Done Done false
15041 5082 Acked 6081 Write 9 New Done Done false
15042 5083 Acked 6082 Read 10 New Done Done false
15043 5084 Acked 6083 Write 2 New Done Done false
15044 5085 Acked 6084 Flush 0 New Done Done false
15045 5086 Acked 6085 Read 5 New Done Done false
15046 5087 Acked 6086 Read 10 New Done Done false
15047 5088 Acked 6087 Write 5 New Done Done false
15048 5089 Acked 6088 Read 8 New Done Done false
15049 5090 Acked 6089 Read 3 New Done Done false
15050 5091 Acked 6090 Read 9 New Done Done false
15051 5092 Acked 6091 Write 8 New Done Done false
15052 5093 Acked 6092 Read 3 New Done Done false
15053 5094 Acked 6093 Read 3 New Done Done false
15054 5095 Acked 6094 Write 2 New Done Done false
15055 5096 Acked 6095 Read 9 New Done Done false
15056 5097 Acked 6096 Write 1 New Done Done false
15057 5098 Acked 6097 Write 4 New Done Done false
15058 5099 Acked 6098 Read 2 New Done Done false
15059 5100 Acked 6099 Flush 0 New Done Done false
15060 5101 Acked 6100 Flush 0 New Done Done false
15061 5102 Acked 6101 Read 2 New Done Done false
15062 5103 Acked 6102 Write 1 New Done Done false
15063 5104 Acked 6103 Write 10 New Done Done false
15064 5105 Acked 6104 Read 6 New Done Done false
15065 5106 Acked 6105 Write 2 New Done Done false
15066 5107 Acked 6106 Write 9 New Done Done false
15067 5108 Acked 6107 Read 6 New Done Done false
15068 5109 Acked 6108 Read 5 New Done Done false
15069 5110 Acked 6109 Read 1 New Done Done false
15070 5111 Acked 6110 Write 1 New Done Done false
15071 5112 Acked 6111 Read 6 New Done Done false
15072 5113 Acked 6112 Read 1 New Done Done false
15073 5114 Acked 6113 Read 5 New Done Done false
15074 5115 Acked 6114 Write 8 New Done Done false
15075 5116 Acked 6115 Read 3 New Done Done false
15076 5117 Acked 6116 Write 10 New Done Done false
15077 5118 Acked 6117 Read 6 New Done Done false
15078 5119 Acked 6118 Read 3 New Done Done false
15079 5120 Acked 6119 Read 4 New Done Done false
15080 5121 Acked 6120 Flush 0 New Done Done false
15081 5122 Acked 6121 Write 5 New Done Done false
15082 5123 Acked 6122 Write 5 New Done Done false
15083 5124 Acked 6123 Flush 0 New Done Done false
15084 5125 Acked 6124 Write 1 New Done Done false
15085 5126 Acked 6125 Write 4 New Done Done false
15086 5127 Acked 6126 Write 2 New Done Done false
15087 5128 Acked 6127 Read 9 New Done Done false
15088 5129 Acked 6128 Write 2 New Done Done false
15089 5130 Acked 6129 Read 5 New Done Done false
15090 5131 Acked 6130 Write 10 New Done Done false
15091 5132 Acked 6131 Read 4 New Done Done false
15092 5133 Acked 6132 Flush 0 New Done Done false
15093 5134 Acked 6133 Read 2 New Done Done false
15094 5135 Acked 6134 Read 8 New Done Done false
15095 5136 Acked 6135 Flush 0 New Done Done false
15096 5137 Acked 6136 Write 5 New Done Done false
15097 5138 Acked 6137 Read 7 New Done Done false
15098 5139 Acked 6138 Write 1 New Done Done false
15099 5140 Acked 6139 Read 6 New Done Done false
15100 5141 Acked 6140 Write 3 New Done Done false
15101 5142 Acked 6141 Read 1 New Done Done false
15102 STATES DS:0 DS:1 DS:2 TOTAL
15103 New 301 0 0 301
15104 Sent 0 0 0 0
15105 Done 0 301 301 602
15106 Skipped 0 0 0 0
15107 Error 0 0 0 0
15108 Last Flush: 5840 6135 6135
15109 Downstairs last five completed: 5840 5839 5838 5837 5836 5835
15110 Upstairs last five completed: 5142 5141 5140 5139 5138 5137
15111 CLIENT: Up:0 ds:301 act:2
15112 ----------------------------------------------------------------
15113 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15114 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
15115 4842 Acked 5841 Read 8 New Done Done false
15116 4843 Acked 5842 Flush 0 New Done Done false
15117 4844 Acked 5843 Write 9 New Done Done false
15118 4845 Acked 5844 Flush 0 New Done Done false
15119 4846 Acked 5845 Write 5 New Done Done false
15120 4847 Acked 5846 Write 1 New Done Done false
15121 4848 Acked 5847 Read 7 New Done Done false
15122 4849 Acked 5848 Write 6 New Done Done false
15123 4850 Acked 5849 Write 1 New Done Done false
15124 4851 Acked 5850 Read 3 New Done Done false
15125 4852 Acked 5851 Read 1 New Done Done false
15126 4853 Acked 5852 Read 10 New Done Done false
15127 4854 Acked 5853 Read 8 New Done Done false
15128 4855 Acked 5854 Write 6 New Done Done false
15129 4856 Acked 5855 Read 1 New Done Done false
15130 4857 Acked 5856 Read 8 New Done Done false
15131 4858 Acked 5857 Write 3 New Done Done false
15132 4859 Acked 5858 Read 9 New Done Done false
15133 4860 Acked 5859 Write 5 New Done Done false
15134 4861 Acked 5860 Write 1 New Done Done false
15135 4862 Acked 5861 Write 9 New Done Done false
15136 4863 Acked 5862 Flush 0 New Done Done false
15137 4864 Acked 5863 Write 9 New Done Done false
15138 4865 Acked 5864 Write 6 New Done Done false
15139 4866 Acked 5865 Write 6 New Done Done false
15140 4867 Acked 5866 Read 10 New Done Done false
15141 4868 Acked 5867 Read 4 New Done Done false
15142 4869 Acked 5868 Read 2 New Done Done false
15143 4870 Acked 5869 Write 6 New Done Done false
15144 4871 Acked 5870 Write 2 New Done Done false
15145 4872 Acked 5871 Read 7 New Done Done false
15146 4873 Acked 5872 Write 6 New Done Done false
15147 4874 Acked 5873 Read 1 New Done Done false
15148 4875 Acked 5874 Read 2 New Done Done false
15149 4876 Acked 5875 Read 1 New Done Done false
15150 4877 Acked 5876 Write 6 New Done Done false
15151 4878 Acked 5877 Read 4 New Done Done false
15152 4879 Acked 5878 Read 10 New Done Done false
15153 4880 Acked 5879 Write 7 New Done Done false
15154 4881 Acked 5880 Flush 0 New Done Done false
15155 4882 Acked 5881 Read 5 New Done Done false
15156 4883 Acked 5882 Write 3 New Done Done false
15157 4884 Acked 5883 Flush 0 New Done Done false
15158 4885 Acked 5884 Read 7 New Done Done false
15159 4886 Acked 5885 Read 6 New Done Done false
15160 4887 Acked 5886 Read 3 New Done Done false
15161 4888 Acked 5887 Read 3 New Done Done false
15162 4889 Acked 5888 Write 4 New Done Done false
15163 4890 Acked 5889 Write 1 New Done Done false
15164 4891 Acked 5890 Write 5 New Done Done false
15165 4892 Acked 5891 Flush 0 New Done Done false
15166 4893 Acked 5892 Read 6 New Done Done false
15167 4894 Acked 5893 Read 9 New Done Done false
15168 4895 Acked 5894 Read 9 New Done Done false
15169 4896 Acked 5895 Read 4 New Done Done false
15170 4897 Acked 5896 Flush 0 New Done Done false
15171 4898 Acked 5897 Write 3 New Done Done false
15172 4899 Acked 5898 Read 7 New Done Done false
15173 4900 Acked 5899 Write 7 New Done Done false
15174 4901 Acked 5900 Write 10 New Done Done false
15175 4902 Acked 5901 Read 6 New Done Done false
15176 4903 Acked 5902 Read 1 New Done Done false
15177 4904 Acked 5903 Read 6 New Done Done false
15178 4905 Acked 5904 Read 10 New Done Done false
15179 4906 Acked 5905 Read 3 New Done Done false
15180 4907 Acked 5906 Read 4 New Done Done false
15181 4908 Acked 5907 Flush 0 New Done Done false
15182 4909 Acked 5908 Write 8 New Done Done false
15183 4910 Acked 5909 Read 10 New Done Done false
15184 4911 Acked 5910 Write 6 New Done Done false
15185 4912 Acked 5911 Read 7 New Done Done false
15186 4913 Acked 5912 Read 6 New Done Done false
15187 4914 Acked 5913 Read 6 New Done Done false
15188 4915 Acked 5914 Read 3 New Done Done false
15189 4916 Acked 5915 Write 3 New Done Done false
15190 4917 Acked 5916 Read 5 New Done Done false
15191 4918 Acked 5917 Read 4 New Done Done false
15192 4919 Acked 5918 Write 4 New Done Done false
15193 4920 Acked 5919 Flush 0 New Done Done false
15194 4921 Acked 5920 Read 9 New Done Done false
15195 4922 Acked 5921 Read 9 New Done Done false
15196 4923 Acked 5922 Write 8 New Done Done false
15197 4924 Acked 5923 Read 2 New Done Done false
15198 4925 Acked 5924 Read 7 New Done Done false
15199 4926 Acked 5925 Write 6 New Done Done false
15200 4927 Acked 5926 Read 6 New Done Done false
15201 4928 Acked 5927 Read 5 New Done Done false
15202 4929 Acked 5928 Flush 0 New Done Done false
15203 4930 Acked 5929 Read 9 New Done Done false
15204 4931 Acked 5930 Write 5 New Done Done false
15205 4932 Acked 5931 Write 4 New Done Done false
15206 4933 Acked 5932 Write 10 New Done Done false
15207 4934 Acked 5933 Read 9 New Done Done false
15208 4935 Acked 5934 Write 4 New Done Done false
15209 4936 Acked 5935 Write 4 New Done Done false
15210 4937 Acked 5936 Write 1 New Done Done false
15211 4938 Acked 5937 Write 10 New Done Done false
15212 4939 Acked 5938 Write 9 New Done Done false
15213 4940 Acked 5939 Read 4 New Done Done false
15214 4941 Acked 5940 Write 8 New Done Done false
15215 4942 Acked 5941 Write 6 New Done Done false
15216 4943 Acked 5942 Read 1 New Done Done false
15217 4944 Acked 5943 Read 10 New Done Done false
15218 4945 Acked 5944 Read 5 New Done Done false
15219 4946 Acked 5945 Write 2 New Done Done false
15220 4947 Acked 5946 Read 7 New Done Done false
15221 4948 Acked 5947 Flush 0 New Done Done false
15222 4949 Acked 5948 Write 8 New Done Done false
15223 4950 Acked 5949 Read 3 New Done Done false
15224 4951 Acked 5950 Flush 0 New Done Done false
15225 4952 Acked 5951 Write 8 New Done Done false
15226 4953 Acked 5952 Read 4 New Done Done false
15227 4954 Acked 5953 Write 10 New Done Done false
15228 4955 Acked 5954 Read 6 New Done Done false
15229 4956 Acked 5955 Write 4 New Done Done false
15230 4957 Acked 5956 Read 9 New Done Done false
15231 4958 Acked 5957 Read 6 New Done Done false
15232 4959 Acked 5958 Write 5 New Done Done false
15233 4960 Acked 5959 Read 4 New Done Done false
15234 4961 Acked 5960 Read 7 New Done Done false
15235 4962 Acked 5961 Write 10 New Done Done false
15236 4963 Acked 5962 Read 1 New Done Done false
15237 4964 Acked 5963 Write 8 New Done Done false
15238 4965 Acked 5964 Read 7 New Done Done false
15239 4966 Acked 5965 Write 7 New Done Done false
15240 4967 Acked 5966 Read 1 New Done Done false
15241 4968 Acked 5967 Write 9 New Done Done false
15242 4969 Acked 5968 Read 10 New Done Done false
15243 4970 Acked 5969 Flush 0 New Done Done false
15244 4971 Acked 5970 Read 1 New Done Done false
15245 4972 Acked 5971 Write 10 New Done Done false
15246 4973 Acked 5972 Flush 0 New Done Done false
15247 4974 Acked 5973 Read 7 New Done Done false
15248 4975 Acked 5974 Read 3 New Done Done false
15249 4976 Acked 5975 Flush 0 New Done Done false
15250 4977 Acked 5976 Write 9 New Done Done false
15251 4978 Acked 5977 Read 10 New Done Done false
15252 4979 Acked 5978 Write 8 New Done Done false
15253 4980 Acked 5979 Read 8 New Done Done false
15254 4981 Acked 5980 Write 4 New Done Done false
15255 4982 Acked 5981 Write 4 New Done Done false
15256 4983 Acked 5982 Read 3 New Done Done false
15257 4984 Acked 5983 Read 9 New Done Done false
15258 4985 Acked 5984 Flush 0 New Done Done false
15259 4986 Acked 5985 Write 8 New Done Done false
15260 4987 Acked 5986 Write 3 New Done Done false
15261 4988 Acked 5987 Read 5 New Done Done false
15262 4989 Acked 5988 Read 2 New Done Done false
15263 4990 Acked 5989 Read 2 New Done Done false
15264 4991 Acked 5990 Write 9 New Done Done false
15265 4992 Acked 5991 Flush 0 New Done Done false
15266 4993 Acked 5992 Write 4 New Done Done false
15267 4994 Acked 5993 Read 1 New Done Done false
15268 4995 Acked 5994 Write 7 New Done Done false
15269 4996 Acked 5995 Read 7 New Done Done false
15270 4997 Acked 5996 Read 6 New Done Done false
15271 4998 Acked 5997 Write 3 New Done Done false
15272 4999 Acked 5998 Read 4 New Done Done false
15273 5000 Acked 5999 Read 10 New Done Done false
15274 5001 Acked 6000 Write 5 New Done Done false
15275 5002 Acked 6001 Read 4 New Done Done false
15276 5003 Acked 6002 Read 3 New Done Done false
15277 5004 Acked 6003 Write 1 New Done Done false
15278 5005 Acked 6004 Read 2 New Done Done false
15279 5006 Acked 6005 Read 1 New Done Done false
15280 5007 Acked 6006 Read 5 New Done Done false
15281 5008 Acked 6007 Flush 0 New Done Done false
15282 5009 Acked 6008 Write 9 New Done Done false
15283 5010 Acked 6009 Write 4 New Done Done false
15284 5011 Acked 6010 Read 10 New Done Done false
15285 5012 Acked 6011 Write 7 New Done Done false
15286 5013 Acked 6012 Read 3 New Done Done false
15287 5014 Acked 6013 Read 5 New Done Done false
15288 5015 Acked 6014 Read 5 New Done Done false
15289 5016 Acked 6015 Write 9 New Done Done false
15290 5017 Acked 6016 Write 7 New Done Done false
15291 5018 Acked 6017 Flush 0 New Done Done false
15292 5019 Acked 6018 Write 3 New Done Done false
15293 5020 Acked 6019 Read 1 New Done Done false
15294 5021 Acked 6020 Read 5 New Done Done false
15295 5022 Acked 6021 Read 3 New Done Done false
15296 5023 Acked 6022 Write 8 New Done Done false
15297 5024 Acked 6023 Write 2 New Done Done false
15298 5025 Acked 6024 Write 9 New Done Done false
15299 5026 Acked 6025 Read 8 New Done Done false
15300 5027 Acked 6026 Read 3 New Done Done false
15301 5028 Acked 6027 Write 5 New Done Done false
15302 5029 Acked 6028 Write 8 New Done Done false
15303 5030 Acked 6029 Read 4 New Done Done false
15304 5031 Acked 6030 Flush 0 New Done Done false
15305 5032 Acked 6031 Read 7 New Done Done false
15306 5033 Acked 6032 Write 9 New Done Done false
15307 5034 Acked 6033 Read 3 New Done Done false
15308 5035 Acked 6034 Read 9 New Done Done false
15309 5036 Acked 6035 Read 5 New Done Done false
15310 5037 Acked 6036 Read 10 New Done Done false
15311 5038 Acked 6037 Write 1 New Done Done false
15312 5039 Acked 6038 Write 7 New Done Done false
15313 5040 Acked 6039 Write 6 New Done Done false
15314 5041 Acked 6040 Flush 0 New Done Done false
15315 5042 Acked 6041 Write 3 New Done Done false
15316 5043 Acked 6042 Write 10 New Done Done false
15317 5044 Acked 6043 Read 8 New Done Done false
15318 5045 Acked 6044 Read 10 New Done Done false
15319 5046 Acked 6045 Read 9 New Done Done false
15320 5047 Acked 6046 Write 3 New Done Done false
15321 5048 Acked 6047 Flush 0 New Done Done false
15322 5049 Acked 6048 Read 7 New Done Done false
15323 5050 Acked 6049 Flush 0 New Done Done false
15324 5051 Acked 6050 Flush 0 New Done Done false
15325 5052 Acked 6051 Write 10 New Done Done false
15326 5053 Acked 6052 Write 3 New Done Done false
15327 5054 Acked 6053 Write 6 New Done Done false
15328 5055 Acked 6054 Flush 0 New Done Done false
15329 5056 Acked 6055 Read 1 New Done Done false
15330 5057 Acked 6056 Write 6 New Done Done false
15331 5058 Acked 6057 Read 5 New Done Done false
15332 5059 Acked 6058 Write 2 New Done Done false
15333 5060 Acked 6059 Write 8 New Done Done false
15334 5061 Acked 6060 Write 8 New Done Done false
15335 5062 Acked 6061 Flush 0 New Done Done false
15336 5063 Acked 6062 Read 10 New Done Done false
15337 5064 Acked 6063 Flush 0 New Done Done false
15338 5065 Acked 6064 Read 2 New Done Done false
15339 5066 Acked 6065 Read 10 New Done Done false
15340 5067 Acked 6066 Write 4 New Done Done false
15341 5068 Acked 6067 Write 10 New Done Done false
15342 5069 Acked 6068 Write 4 New Done Done false
15343 5070 Acked 6069 Flush 0 New Done Done false
15344 5071 Acked 6070 Read 1 New Done Done false
15345 5072 Acked 6071 Write 5 New Done Done false
15346 5073 Acked 6072 Read 2 New Done Done false
15347 5074 Acked 6073 Write 10 New Done Done false
15348 5075 Acked 6074 Read 5 New Done Done false
15349 5076 Acked 6075 Write 5 New Done Done false
15350 5077 Acked 6076 Write 2 New Done Done false
15351 5078 Acked 6077 Read 1 New Done Done false
15352 5079 Acked 6078 Read 1 New Done Done false
15353 5080 Acked 6079 Write 7 New Done Done false
15354 5081 Acked 6080 Read 8 New Done Done false
15355 5082 Acked 6081 Write 9 New Done Done false
15356 5083 Acked 6082 Read 10 New Done Done false
15357 5084 Acked 6083 Write 2 New Done Done false
15358 5085 Acked 6084 Flush 0 New Done Done false
15359 5086 Acked 6085 Read 5 New Done Done false
15360 5087 Acked 6086 Read 10 New Done Done false
15361 5088 Acked 6087 Write 5 New Done Done false
15362 5089 Acked 6088 Read 8 New Done Done false
15363 5090 Acked 6089 Read 3 New Done Done false
15364 5091 Acked 6090 Read 9 New Done Done false
15365 5092 Acked 6091 Write 8 New Done Done false
15366 5093 Acked 6092 Read 3 New Done Done false
15367 5094 Acked 6093 Read 3 New Done Done false
15368 5095 Acked 6094 Write 2 New Done Done false
15369 5096 Acked 6095 Read 9 New Done Done false
15370 5097 Acked 6096 Write 1 New Done Done false
15371 5098 Acked 6097 Write 4 New Done Done false
15372 5099 Acked 6098 Read 2 New Done Done false
15373 5100 Acked 6099 Flush 0 New Done Done false
15374 5101 Acked 6100 Flush 0 New Done Done false
15375 5102 Acked 6101 Read 2 New Done Done false
15376 5103 Acked 6102 Write 1 New Done Done false
15377 5104 Acked 6103 Write 10 New Done Done false
15378 5105 Acked 6104 Read 6 New Done Done false
15379 5106 Acked 6105 Write 2 New Done Done false
15380 5107 Acked 6106 Write 9 New Done Done false
15381 5108 Acked 6107 Read 6 New Done Done false
15382 5109 Acked 6108 Read 5 New Done Done false
15383 5110 Acked 6109 Read 1 New Done Done false
15384 5111 Acked 6110 Write 1 New Done Done false
15385 5112 Acked 6111 Read 6 New Done Done false
15386 5113 Acked 6112 Read 1 New Done Done false
15387 5114 Acked 6113 Read 5 New Done Done false
15388 5115 Acked 6114 Write 8 New Done Done false
15389 5116 Acked 6115 Read 3 New Done Done false
15390 5117 Acked 6116 Write 10 New Done Done false
15391 5118 Acked 6117 Read 6 New Done Done false
15392 5119 Acked 6118 Read 3 New Done Done false
15393 5120 Acked 6119 Read 4 New Done Done false
15394 5121 Acked 6120 Flush 0 New Done Done false
15395 5122 Acked 6121 Write 5 New Done Done false
15396 5123 Acked 6122 Write 5 New Done Done false
15397 5124 Acked 6123 Flush 0 New Done Done false
15398 5125 Acked 6124 Write 1 New Done Done false
15399 5126 Acked 6125 Write 4 New Done Done false
15400 5127 Acked 6126 Write 2 New Done Done false
15401 5128 Acked 6127 Read 9 New Done Done false
15402 5129 Acked 6128 Write 2 New Done Done false
15403 5130 Acked 6129 Read 5 New Done Done false
15404 5131 Acked 6130 Write 10 New Done Done false
15405 5132 Acked 6131 Read 4 New Done Done false
15406 5133 Acked 6132 Flush 0 New Done Done false
15407 5134 Acked 6133 Read 2 New Done Done false
15408 5135 Acked 6134 Read 8 New Done Done false
15409 5136 Acked 6135 Flush 0 New Done Done false
15410 5137 Acked 6136 Write 5 New Done Done false
15411 5138 Acked 6137 Read 7 New Done Done false
15412 5139 Acked 6138 Write 1 New Done Done false
15413 5140 Acked 6139 Read 6 New Done Done false
15414 5141 Acked 6140 Write 3 New Done Done false
15415 5142 Acked 6141 Read 1 New Done Done false
15416 5143 Acked 6142 Flush 0 New Done Done false
15417 STATES DS:0 DS:1 DS:2 TOTAL
15418 New 302 0 0 302
15419 Sent 0 0 0 0
15420 Done 0 302 302 604
15421 Skipped 0 0 0 0
15422 Error 0 0 0 0
15423 Last Flush: 5840 6142 6142
15424 Downstairs last five completed: 5840 5839 5838 5837 5836 5835
15425 Upstairs last five completed: 5143 5142 5141 5140 5139 5138
15426 CLIENT: Up:0 ds:302 act:2
15427 [8810] got start action
15428 [8810] I am going to restart
15429 Starting downstairs at port 8810
15430 Make output file at "/tmp/dsc/downstairs-8810.txt"
15431 [8810][0] reports Starting
154322023-09-22T23:27:09.508ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
15433 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1190)
15434 [8810][0] reports Running
15435 ----------------------------------------------------------------
15436 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15437 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
15438 4842 Acked 5841 Read 8 New Done Done false
15439 4843 Acked 5842 Flush 0 New Done Done false
15440 4844 Acked 5843 Write 9 New Done Done false
15441 4845 Acked 5844 Flush 0 New Done Done false
15442 4846 Acked 5845 Write 5 New Done Done false
15443 4847 Acked 5846 Write 1 New Done Done false
15444 4848 Acked 5847 Read 7 New Done Done false
15445 4849 Acked 5848 Write 6 New Done Done false
15446 4850 Acked 5849 Write 1 New Done Done false
15447 4851 Acked 5850 Read 3 New Done Done false
15448 4852 Acked 5851 Read 1 New Done Done false
15449 4853 Acked 5852 Read 10 New Done Done false
15450 4854 Acked 5853 Read 8 New Done Done false
15451 4855 Acked 5854 Write 6 New Done Done false
15452 4856 Acked 5855 Read 1 New Done Done false
15453 4857 Acked 5856 Read 8 New Done Done false
15454 4858 Acked 5857 Write 3 New Done Done false
15455 4859 Acked 5858 Read 9 New Done Done false
15456 4860 Acked 5859 Write 5 New Done Done false
15457 4861 Acked 5860 Write 1 New Done Done false
15458 4862 Acked 5861 Write 9 New Done Done false
15459 4863 Acked 5862 Flush 0 New Done Done false
15460 4864 Acked 5863 Write 9 New Done Done false
15461 4865 Acked 5864 Write 6 New Done Done false
15462 4866 Acked 5865 Write 6 New Done Done false
15463 4867 Acked 5866 Read 10 New Done Done false
15464 4868 Acked 5867 Read 4 New Done Done false
15465 4869 Acked 5868 Read 2 New Done Done false
15466 4870 Acked 5869 Write 6 New Done Done false
15467 4871 Acked 5870 Write 2 New Done Done false
15468 4872 Acked 5871 Read 7 New Done Done false
15469 4873 Acked 5872 Write 6 New Done Done false
15470 4874 Acked 5873 Read 1 New Done Done false
15471 4875 Acked 5874 Read 2 New Done Done false
15472 4876 Acked 5875 Read 1 New Done Done false
15473 4877 Acked 5876 Write 6 New Done Done false
15474 4878 Acked 5877 Read 4 New Done Done false
15475 4879 Acked 5878 Read 10 New Done Done false
15476 4880 Acked 5879 Write 7 New Done Done false
15477 4881 Acked 5880 Flush 0 New Done Done false
15478 4882 Acked 5881 Read 5 New Done Done false
15479 4883 Acked 5882 Write 3 New Done Done false
15480 4884 Acked 5883 Flush 0 New Done Done false
15481 4885 Acked 5884 Read 7 New Done Done false
15482 4886 Acked 5885 Read 6 New Done Done false
15483 4887 Acked 5886 Read 3 New Done Done false
15484 4888 Acked 5887 Read 3 New Done Done false
15485 4889 Acked 5888 Write 4 New Done Done false
15486 4890 Acked 5889 Write 1 New Done Done false
15487 4891 Acked 5890 Write 5 New Done Done false
15488 4892 Acked 5891 Flush 0 New Done Done false
15489 4893 Acked 5892 Read 6 New Done Done false
15490 4894 Acked 5893 Read 9 New Done Done false
15491 4895 Acked 5894 Read 9 New Done Done false
15492 4896 Acked 5895 Read 4 New Done Done false
15493 4897 Acked 5896 Flush 0 New Done Done false
15494 4898 Acked 5897 Write 3 New Done Done false
15495 4899 Acked 5898 Read 7 New Done Done false
15496 4900 Acked 5899 Write 7 New Done Done false
15497 4901 Acked 5900 Write 10 New Done Done false
15498 4902 Acked 5901 Read 6 New Done Done false
15499 4903 Acked 5902 Read 1 New Done Done false
15500 4904 Acked 5903 Read 6 New Done Done false
15501 4905 Acked 5904 Read 10 New Done Done false
15502 4906 Acked 5905 Read 3 New Done Done false
15503 4907 Acked 5906 Read 4 New Done Done false
15504 4908 Acked 5907 Flush 0 New Done Done false
15505 4909 Acked 5908 Write 8 New Done Done false
15506 4910 Acked 5909 Read 10 New Done Done false
15507 4911 Acked 5910 Write 6 New Done Done false
15508 4912 Acked 5911 Read 7 New Done Done false
15509 4913 Acked 5912 Read 6 New Done Done false
15510 4914 Acked 5913 Read 6 New Done Done false
15511 4915 Acked 5914 Read 3 New Done Done false
15512 4916 Acked 5915 Write 3 New Done Done false
15513 4917 Acked 5916 Read 5 New Done Done false
15514 4918 Acked 5917 Read 4 New Done Done false
15515 4919 Acked 5918 Write 4 New Done Done false
15516 4920 Acked 5919 Flush 0 New Done Done false
15517 4921 Acked 5920 Read 9 New Done Done false
15518 4922 Acked 5921 Read 9 New Done Done false
15519 4923 Acked 5922 Write 8 New Done Done false
15520 4924 Acked 5923 Read 2 New Done Done false
15521 4925 Acked 5924 Read 7 New Done Done false
15522 4926 Acked 5925 Write 6 New Done Done false
15523 4927 Acked 5926 Read 6 New Done Done false
15524 4928 Acked 5927 Read 5 New Done Done false
15525 4929 Acked 5928 Flush 0 New Done Done false
15526 4930 Acked 5929 Read 9 New Done Done false
15527 4931 Acked 5930 Write 5 New Done Done false
15528 4932 Acked 5931 Write 4 New Done Done false
15529 4933 Acked 5932 Write 10 New Done Done false
15530 4934 Acked 5933 Read 9 New Done Done false
15531 4935 Acked 5934 Write 4 New Done Done false
15532 4936 Acked 5935 Write 4 New Done Done false
15533 4937 Acked 5936 Write 1 New Done Done false
15534 4938 Acked 5937 Write 10 New Done Done false
15535 4939 Acked 5938 Write 9 New Done Done false
15536 4940 Acked 5939 Read 4 New Done Done false
15537 4941 Acked 5940 Write 8 New Done Done false
15538 4942 Acked 5941 Write 6 New Done Done false
15539 4943 Acked 5942 Read 1 New Done Done false
15540 4944 Acked 5943 Read 10 New Done Done false
15541 4945 Acked 5944 Read 5 New Done Done false
15542 4946 Acked 5945 Write 2 New Done Done false
15543 4947 Acked 5946 Read 7 New Done Done false
15544 4948 Acked 5947 Flush 0 New Done Done false
15545 4949 Acked 5948 Write 8 New Done Done false
15546 4950 Acked 5949 Read 3 New Done Done false
15547 4951 Acked 5950 Flush 0 New Done Done false
15548 4952 Acked 5951 Write 8 New Done Done false
15549 4953 Acked 5952 Read 4 New Done Done false
15550 4954 Acked 5953 Write 10 New Done Done false
15551 4955 Acked 5954 Read 6 New Done Done false
15552 4956 Acked 5955 Write 4 New Done Done false
15553 4957 Acked 5956 Read 9 New Done Done false
15554 4958 Acked 5957 Read 6 New Done Done false
15555 4959 Acked 5958 Write 5 New Done Done false
15556 4960 Acked 5959 Read 4 New Done Done false
15557 4961 Acked 5960 Read 7 New Done Done false
15558 4962 Acked 5961 Write 10 New Done Done false
15559 4963 Acked 5962 Read 1 New Done Done false
15560 4964 Acked 5963 Write 8 New Done Done false
15561 4965 Acked 5964 Read 7 New Done Done false
15562 4966 Acked 5965 Write 7 New Done Done false
15563 4967 Acked 5966 Read 1 New Done Done false
15564 4968 Acked 5967 Write 9 New Done Done false
15565 4969 Acked 5968 Read 10 New Done Done false
15566 4970 Acked 5969 Flush 0 New Done Done false
15567 4971 Acked 5970 Read 1 New Done Done false
15568 4972 Acked 5971 Write 10 New Done Done false
15569 4973 Acked 5972 Flush 0 New Done Done false
15570 4974 Acked 5973 Read 7 New Done Done false
15571 4975 Acked 5974 Read 3 New Done Done false
15572 4976 Acked 5975 Flush 0 New Done Done false
15573 4977 Acked 5976 Write 9 New Done Done false
15574 4978 Acked 5977 Read 10 New Done Done false
15575 4979 Acked 5978 Write 8 New Done Done false
15576 4980 Acked 5979 Read 8 New Done Done false
15577 4981 Acked 5980 Write 4 New Done Done false
15578 4982 Acked 5981 Write 4 New Done Done false
15579 4983 Acked 5982 Read 3 New Done Done false
15580 4984 Acked 5983 Read 9 New Done Done false
15581 4985 Acked 5984 Flush 0 New Done Done false
15582 4986 Acked 5985 Write 8 New Done Done false
15583 4987 Acked 5986 Write 3 New Done Done false
15584 4988 Acked 5987 Read 5 New Done Done false
15585 4989 Acked 5988 Read 2 New Done Done false
15586 4990 Acked 5989 Read 2 New Done Done false
15587 4991 Acked 5990 Write 9 New Done Done false
15588 4992 Acked 5991 Flush 0 New Done Done false
15589 4993 Acked 5992 Write 4 New Done Done false
15590 4994 Acked 5993 Read 1 New Done Done false
15591 4995 Acked 5994 Write 7 New Done Done false
15592 4996 Acked 5995 Read 7 New Done Done false
15593 4997 Acked 5996 Read 6 New Done Done false
15594 4998 Acked 5997 Write 3 New Done Done false
15595 4999 Acked 5998 Read 4 New Done Done false
15596 5000 Acked 5999 Read 10 New Done Done false
15597 5001 Acked 6000 Write 5 New Done Done false
15598 5002 Acked 6001 Read 4 New Done Done false
15599 5003 Acked 6002 Read 3 New Done Done false
15600 5004 Acked 6003 Write 1 New Done Done false
15601 5005 Acked 6004 Read 2 New Done Done false
15602 5006 Acked 6005 Read 1 New Done Done false
15603 5007 Acked 6006 Read 5 New Done Done false
15604 5008 Acked 6007 Flush 0 New Done Done false
15605 5009 Acked 6008 Write 9 New Done Done false
15606 5010 Acked 6009 Write 4 New Done Done false
15607 5011 Acked 6010 Read 10 New Done Done false
15608 5012 Acked 6011 Write 7 New Done Done false
15609 5013 Acked 6012 Read 3 New Done Done false
15610 5014 Acked 6013 Read 5 New Done Done false
15611 5015 Acked 6014 Read 5 New Done Done false
15612 5016 Acked 6015 Write 9 New Done Done false
15613 5017 Acked 6016 Write 7 New Done Done false
15614 5018 Acked 6017 Flush 0 New Done Done false
15615 5019 Acked 6018 Write 3 New Done Done false
15616 5020 Acked 6019 Read 1 New Done Done false
15617 5021 Acked 6020 Read 5 New Done Done false
15618 5022 Acked 6021 Read 3 New Done Done false
15619 5023 Acked 6022 Write 8 New Done Done false
15620 5024 Acked 6023 Write 2 New Done Done false
15621 5025 Acked 6024 Write 9 New Done Done false
15622 5026 Acked 6025 Read 8 New Done Done false
15623 5027 Acked 6026 Read 3 New Done Done false
15624 5028 Acked 6027 Write 5 New Done Done false
15625 5029 Acked 6028 Write 8 New Done Done false
15626 5030 Acked 6029 Read 4 New Done Done false
15627 5031 Acked 6030 Flush 0 New Done Done false
15628 5032 Acked 6031 Read 7 New Done Done false
15629 5033 Acked 6032 Write 9 New Done Done false
15630 5034 Acked 6033 Read 3 New Done Done false
15631 5035 Acked 6034 Read 9 New Done Done false
15632 5036 Acked 6035 Read 5 New Done Done false
15633 5037 Acked 6036 Read 10 New Done Done false
15634 5038 Acked 6037 Write 1 New Done Done false
15635 5039 Acked 6038 Write 7 New Done Done false
15636 5040 Acked 6039 Write 6 New Done Done false
15637 5041 Acked 6040 Flush 0 New Done Done false
15638 5042 Acked 6041 Write 3 New Done Done false
15639 5043 Acked 6042 Write 10 New Done Done false
15640 5044 Acked 6043 Read 8 New Done Done false
15641 5045 Acked 6044 Read 10 New Done Done false
15642 5046 Acked 6045 Read 9 New Done Done false
15643 5047 Acked 6046 Write 3 New Done Done false
15644 5048 Acked 6047 Flush 0 New Done Done false
15645 5049 Acked 6048 Read 7 New Done Done false
15646 5050 Acked 6049 Flush 0 New Done Done false
15647 5051 Acked 6050 Flush 0 New Done Done false
15648 5052 Acked 6051 Write 10 New Done Done false
15649 5053 Acked 6052 Write 3 New Done Done false
15650 5054 Acked 6053 Write 6 New Done Done false
15651 5055 Acked 6054 Flush 0 New Done Done false
15652 5056 Acked 6055 Read 1 New Done Done false
15653 5057 Acked 6056 Write 6 New Done Done false
15654 5058 Acked 6057 Read 5 New Done Done false
15655 5059 Acked 6058 Write 2 New Done Done false
15656 5060 Acked 6059 Write 8 New Done Done false
15657 5061 Acked 6060 Write 8 New Done Done false
15658 5062 Acked 6061 Flush 0 New Done Done false
15659 5063 Acked 6062 Read 10 New Done Done false
15660 5064 Acked 6063 Flush 0 New Done Done false
15661 5065 Acked 6064 Read 2 New Done Done false
15662 5066 Acked 6065 Read 10 New Done Done false
15663 5067 Acked 6066 Write 4 New Done Done false
15664 5068 Acked 6067 Write 10 New Done Done false
15665 5069 Acked 6068 Write 4 New Done Done false
15666 5070 Acked 6069 Flush 0 New Done Done false
15667 5071 Acked 6070 Read 1 New Done Done false
15668 5072 Acked 6071 Write 5 New Done Done false
15669 5073 Acked 6072 Read 2 New Done Done false
15670 5074 Acked 6073 Write 10 New Done Done false
15671 5075 Acked 6074 Read 5 New Done Done false
15672 5076 Acked 6075 Write 5 New Done Done false
15673 5077 Acked 6076 Write 2 New Done Done false
15674 5078 Acked 6077 Read 1 New Done Done false
15675 5079 Acked 6078 Read 1 New Done Done false
15676 5080 Acked 6079 Write 7 New Done Done false
15677 5081 Acked 6080 Read 8 New Done Done false
15678 5082 Acked 6081 Write 9 New Done Done false
15679 5083 Acked 6082 Read 10 New Done Done false
15680 5084 Acked 6083 Write 2 New Done Done false
15681 5085 Acked 6084 Flush 0 New Done Done false
15682 5086 Acked 6085 Read 5 New Done Done false
15683 5087 Acked 6086 Read 10 New Done Done false
15684 5088 Acked 6087 Write 5 New Done Done false
15685 5089 Acked 6088 Read 8 New Done Done false
15686 5090 Acked 6089 Read 3 New Done Done false
15687 5091 Acked 6090 Read 9 New Done Done false
15688 5092 Acked 6091 Write 8 New Done Done false
15689 5093 Acked 6092 Read 3 New Done Done false
15690 5094 Acked 6093 Read 3 New Done Done false
15691 5095 Acked 6094 Write 2 New Done Done false
15692 5096 Acked 6095 Read 9 New Done Done false
15693 5097 Acked 6096 Write 1 New Done Done false
15694 5098 Acked 6097 Write 4 New Done Done false
15695 5099 Acked 6098 Read 2 New Done Done false
15696 5100 Acked 6099 Flush 0 New Done Done false
15697 5101 Acked 6100 Flush 0 New Done Done false
15698 5102 Acked 6101 Read 2 New Done Done false
15699 5103 Acked 6102 Write 1 New Done Done false
15700 5104 Acked 6103 Write 10 New Done Done false
15701 5105 Acked 6104 Read 6 New Done Done false
15702 5106 Acked 6105 Write 2 New Done Done false
15703 5107 Acked 6106 Write 9 New Done Done false
15704 5108 Acked 6107 Read 6 New Done Done false
15705 5109 Acked 6108 Read 5 New Done Done false
15706 5110 Acked 6109 Read 1 New Done Done false
15707 5111 Acked 6110 Write 1 New Done Done false
15708 5112 Acked 6111 Read 6 New Done Done false
15709 5113 Acked 6112 Read 1 New Done Done false
15710 5114 Acked 6113 Read 5 New Done Done false
15711 5115 Acked 6114 Write 8 New Done Done false
15712 5116 Acked 6115 Read 3 New Done Done false
15713 5117 Acked 6116 Write 10 New Done Done false
15714 5118 Acked 6117 Read 6 New Done Done false
15715 5119 Acked 6118 Read 3 New Done Done false
15716 5120 Acked 6119 Read 4 New Done Done false
15717 5121 Acked 6120 Flush 0 New Done Done false
15718 5122 Acked 6121 Write 5 New Done Done false
15719 5123 Acked 6122 Write 5 New Done Done false
15720 5124 Acked 6123 Flush 0 New Done Done false
15721 5125 Acked 6124 Write 1 New Done Done false
15722 5126 Acked 6125 Write 4 New Done Done false
15723 5127 Acked 6126 Write 2 New Done Done false
15724 5128 Acked 6127 Read 9 New Done Done false
15725 5129 Acked 6128 Write 2 New Done Done false
15726 5130 Acked 6129 Read 5 New Done Done false
15727 5131 Acked 6130 Write 10 New Done Done false
15728 5132 Acked 6131 Read 4 New Done Done false
15729 5133 Acked 6132 Flush 0 New Done Done false
15730 5134 Acked 6133 Read 2 New Done Done false
15731 5135 Acked 6134 Read 8 New Done Done false
15732 5136 Acked 6135 Flush 0 New Done Done false
15733 5137 Acked 6136 Write 5 New Done Done false
15734 5138 Acked 6137 Read 7 New Done Done false
15735 5139 Acked 6138 Write 1 New Done Done false
15736 5140 Acked 6139 Read 6 New Done Done false
15737 5141 Acked 6140 Write 3 New Done Done false
15738 5142 Acked 6141 Read 1 New Done Done false
15739 5143 Acked 6142 Flush 0 New Done Done false
15740 STATES DS:0 DS:1 DS:2 TOTAL
15741 New 302 0 0 302
15742 Sent 0 0 0 0
15743 Done 0 302 302 604
15744 Skipped 0 0 0 0
15745 Error 0 0 0 0
15746 Last Flush: 5840 6142 6142
15747 Downstairs last five completed: 5840 5839 5838 5837 5836 5835
15748 Upstairs last five completed: 5143 5142 5141 5140 5139 5138
15749 CLIENT: Up:0 ds:302 act:2
157502023-09-22T23:27:10.509ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
157512023-09-22T23:27:10.509ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
157522023-09-22T23:27:10.509ZINFOcrucible: [0] client re-new 302 jobs since flush 5840 = downstairs
157532023-09-22T23:27:10.510ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
157542023-09-22T23:27:10.511ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
157552023-09-22T23:27:10.511ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
157562023-09-22T23:27:10.511ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
157572023-09-22T23:27:10.511ZINFOcrucible: [0] send last flush ID to this DS: 5840
157582023-09-22T23:27:10.511ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
157592023-09-22T23:27:10.511ZINFOcrucible: [0] Replied this last flush ID: 5840
157602023-09-22T23:27:10.511ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
157612023-09-22T23:27:10.511ZINFOcrucible: [0] Transition from Offline to Replay
157622023-09-22T23:27:10.511ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
157632023-09-22T23:27:10.511ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
157642023-09-22T23:27:10.511ZINFOcrucible: [0] Transition from Replay to Active
157652023-09-22T23:27:10.511ZINFOcrucible: [0] Starts cmd_loop
157662023-09-22T23:27:10.511ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
157672023-09-22T23:27:10.511ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
157682023-09-22T23:27:10.511ZINFOcrucible: Set check for repair
157692023-09-22T23:27:11.512ZINFOcrucible: Checking if live repair is needed
157702023-09-22T23:27:11.512ZINFOcrucible: No Live Repair required at this time
157712023-09-22T23:27:11.512ZWARNcrucible: [0] flow control end
15772 ----------------------------------------------------------------
15773 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15774 Downstairs last five completed: 6142 6141 6140 6139 6138 6137
15775 Upstairs last five completed: 5143 5142 5141 5140 5139 5138
15776 CLIENT: Up:0 ds:0 act:3
15777 Replay: All jobs finished, all DS active.
15778 Main task has work to do, go find it
15779 got dsc Stop(1)
15780 stop 1
15781 Sep 22 23:27:14.426 INFO request completed, latency_us: 508, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 598cb59a-774c-4865-80d9-09855a47f666, remote_addr: 127[8820] Got stop action so:false kr:false
15782 .0.0.1:52471, local_addr: 127.0.0.1:9998
15783 Sep 22 23:27:14.427 INFO request completed, latency_us: 309, response_code: 200, uri: /state/cid/1, method: GET, req_id: 508db294-8fa3-4233-a8d0-bdee29eb3f30, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
157842023-09-22T23:27:14.427ZWARNcrucible: [1] downstairs disconnected
157852023-09-22T23:27:14.428ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
157862023-09-22T23:27:14.428ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
157872023-09-22T23:27:14.428ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
157882023-09-22T23:27:14.428ZWARNcrucible: [1] pm_task rx.recv() is None
157892023-09-22T23:27:14.428ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
157902023-09-22T23:27:14.428ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
15791 [8820] Exited with: ExitStatus(unix_wait_status(9))
15792 [8820][1] reports Exit
15793 Sep 22 23:27:18.430 INFO request completed, latency_us: 293, response_code: 200, uri: /state/cid/1, method: GET, req_id: 90086a0b-0c66-4458-aaca-514be18c75e6, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
15794 001/300 Write block 819 len 36864 data: 5 6 5 3 4 4 4 4 3
15795 002/300 Write block 1621 len 24576 data: 4 4 5 6 6 5
15796 003/300 Write block 4013 len 28672 data: 2 3 3 3 4 4 5
15797 004/300 Read block 2202 len 40960
15798 005/300 Read block 3071 len 32768
15799 006/300 Write block 813 len 28672 data: 8 7 6 5 5 4 6
15800 007/300 Read block 4011 len 8192
15801 008/300 Flush
15802 009/300 Write block 1041 len 28672 data: 4 3 3 4 4 2 2
15803 010/300 Write block 4544 len 8192 data: 3 3
15804 011/300 Write block 1404 len 8192 data: 7 7
15805 012/300 Write block 1119 len 24576 data: 3 3 3 2 2 2
15806 013/300 Write block 1643 len 32768 data: 3 3 3 2 3 3 4 3
15807 014/300 Read block 4513 len 32768
15808 015/300 Flush
15809 016/300 Write block 3854 len 40960 data: 5 4 3 3 4 4 5 5 4 3
15810 017/300 Read block 1656 len 32768
15811 018/300 Read block 1583 len 36864
15812 019/300 Write block 4408 len 36864 data: 5 5 4 4 5 5 5 5 6
15813 020/300 Read block 2009 len 16384
15814 021/300 Write block 4872 len 8192 data: 4 5
15815 022/300 Flush
15816 023/300 Read block 4430 len 32768
15817 024/300 Write block 1745 len 36864 data: 2 2 2 2 2 3 3 4 5
15818 025/300 Read block 3060 len 8192
15819 026/300 Flush
15820 027/300 Write block 1897 len 32768 data: 3 4 5 4 5 5 5 5
15821 028/300 Read block 290 len 12288
15822 029/300 Write block 553 len 36864 data: 4 4 3 3 3 3 2 3 4
15823 030/300 Write block 3768 len 16384 data: 2 3 3 3
15824 031/300 Write block 1087 len 16384 data: 3 3 3 4
15825 032/300 Read block 3374 len 24576
15826 033/300 Read block 1725 len 28672
15827 034/300 Read block 4831 len 32768
15828 035/300 Write block 4185 len 20480 data: 6 3 3 3 4
15829 036/300 Flush
15830 037/300 Read block 1086 len 32768
15831 038/300 Read block 1586 len 8192
15832 039/300 Write block 1588 len 12288 data: 2 2 2
15833 040/300 Flush
15834 041/300 Read block 3892 len 12288
15835 042/300 Write block 4159 len 8192 data: 5 5
15836 043/300 Write block 603 len 32768 data: 3 4 4 4 4 4 4 2
15837 044/300 Read block 2329 len 36864
15838 045/300 Flush
15839 046/300 Write block 3478 len 16384 data: 5 5 6 5
15840 047/300 Read block 2942 len 16384
15841 048/300 Read block 1327 len 12288
15842 049/300 Write block 2825 len 28672 data: 4 3 3 4 5 5 4
15843 050/300 Read block 3021 len 16384
15844 051/300 Read block 7 len 28672
15845 052/300 Read block 3889 len 20480
15846 053/300 Read block 1359 len 20480
15847 054/300 Write block 683 len 24576 data: 3 4 3 3 3 4
15848 055/300 Write block 1274 len 32768 data: 7 4 4 4 3 3 3 3
15849 056/300 Read block 3049 len 20480
15850 057/300 Read block 430 len 32768
15851 058/300 Write block 3495 len 8192 data: 4 4
15852 059/300 Read block 1968 len 28672
15853 060/300 Flush
15854 061/300 Read block 264 len 32768
15855 062/300 Write block 3769 len 32768 data: 4 4 4 4 5 5 4 4
15856 063/300 Read block 482 len 12288
15857 064/300 Read block 2142 len 4096
15858 065/300 Read block 3298 len 32768
15859 066/300 Read block 3473 len 36864
15860 067/300 Read block 3887 len 8192
15861 068/300 Read block 1091 len 12288
15862 069/300 Flush
15863 070/300 Read block 1161 len 24576
15864 071/300 Flush
15865 072/300 Read block 4096 len 16384
15866 073/300 Read block 2725 len 24576
15867 074/300 Write block 3696 len 4096 data: 3
15868 075/300 Read block 4867 len 20480
15869 076/300 Read block 1913 len 28672
15870 077/300 Read block 4909 len 40960
15871 078/300 Write block 1869 len 20480 data: 3 3 4 5 6
15872 079/300 Write block 18 len 12288 data: 5 6 6
15873 080/300 Read block 1183 len 36864
15874 081/300 Write block 1997 len 16384 data: 2 3 3 4
15875 082/300 Read block 2837 len 32768
15876 083/300 Flush
15877 084/300 Read block 4708 len 20480
15878 085/300 Read block 3143 len 20480
15879 086/300 Write block 3024 len 28672 data: 6 7 9 7 7 4 4
15880 087/300 Read block 3206 len 16384
15881 088/300 Read block 2794 len 32768
15882 089/300 Write block 4901 len 24576 data: 4 4 5 4 6 6
15883 090/300 Write block 237 len 12288 data: 3 4 5
15884 091/300 Write block 3536 len 36864 data: 3 3 2 2 2 2 2 2 2
15885 092/300 Read block 318 len 16384
15886 093/300 Write block 2920 len 20480 data: 3 3 5 5 7
15887 094/300 Write block 2859 len 16384 data: 4 5 6 5
15888 095/300 Read block 4502 len 40960
15889 096/300 Read block 691 len 12288
15890 097/300 Read block 4 len 20480
15891 098/300 Flush
15892 099/300 Write block 1025 len 16384 data: 4 4 5 4
15893 100/300 Read block 884 len 4096
15894 101/300 Write block 4084 len 40960 data: 6 6 6 5 4 3 3 3 3 3
15895 102/300 Write block 868 len 24576 data: 5 5 6 6 5 6
15896 103/300 Read block 812 len 20480
15897 104/300 Read block 1555 len 8192
15898 105/300 Read block 1077 len 16384
15899 106/300 Read block 2041 len 40960
15900 107/300 Read block 1972 len 40960
15901 108/300 Write block 4491 len 8192 data: 7 6
15902 109/300 Write block 2855 len 8192 data: 4 3
15903 110/300 Write block 2833 len 40960 data: 7 5 5 4 3 5 5 4 5 6
15904 111/300 Read block 3466 len 36864
15905 112/300 Flush
15906 113/300 Flush
15907 114/300 Write block 4997 len 4096 data: 3
15908 115/300 Read block 935 len 40960
15909 116/300 Read block 654 len 24576
15910 117/300 Write block 2538 len 36864 data: 4 4 4 4 4 4 4 4 4
15911 118/300 Flush
15912 119/300 Write block 1899 len 32768 data: 6 5 6 6 6 6 5 5
15913 120/300 Write block 1072 len 28672 data: 4 2 2 2 2 2 4
15914 121/300 Flush
15915 122/300 Read block 2862 len 32768
15916 123/300 Write block 3460 len 32768 data: 3 3 4 5 4 5 5 5
15917 124/300 Write block 3168 len 16384 data: 6 6 5 5
15918 125/300 Write block 2949 len 40960 data: 4 4 4 5 3 3 2 4 4 4
15919 126/300 Flush
15920 127/300 Write block 3515 len 40960 data: 7 7 4 3 3 3 4 4 4 3
15921 128/300 Read block 4006 len 4096
15922 129/300 Read block 1119 len 20480
15923 130/300 Read block 3692 len 24576
15924 131/300 Flush
15925 132/300 Write block 460 len 40960 data: 4 4 4 4 3 3 2 2 3 3
15926 133/300 Read block 1516 len 36864
15927 134/300 Write block 4568 len 12288 data: 3 3 3
15928 135/300 Write block 2880 len 12288 data: 6 4 4
15929 136/300 Read block 2360 len 28672
15930 137/300 Write block 4574 len 24576 data: 4 5 5 5 4 5
15931 138/300 Flush
15932 139/300 Read block 3708 len 20480
15933 140/300 Read block 459 len 40960
15934 141/300 Write block 1867 len 12288 data: 5 3 4
15935 142/300 Read block 3203 len 36864
15936 143/300 Read block 3377 len 36864
15937 144/300 Write block 4594 len 40960 data: 2 2 2 2 4 5 4 4 7 7
15938 145/300 Write block 630 len 40960 data: 3 3 3 3 3 3 3 2 2 2
15939 146/300 Read block 2198 len 12288
15940 147/300 Read block 4060 len 36864
15941 148/300 Read block 3924 len 28672
15942 149/300 Flush
15943 150/300 Write block 578 len 20480 data: 5 5 3 2 2
15944 151/300 Flush
15945 152/300 Read block 1341 len 16384
15946 153/300 Read block 2114 len 32768
15947 154/300 Read block 1122 len 36864
15948 155/300 Read block 728 len 8192
15949 156/300 Write block 3449 len 20480 data: 3 3 3 3 3
15950 157/300 Read block 4320 len 16384
15951 158/300 Write block 3852 len 40960 data: 7 7 6 5 4 4 5 5 6 6
15952 159/300 Flush
15953 160/300 Read block 2933 len 40960
15954 161/300 Read block 146 len 32768
15955 162/300 Read block 4383 len 20480
15956 163/300 Read block 2093 len 16384
15957 164/300 Read block 4890 len 16384
15958 165/300 Write block 1995 len 24576 data: 3 3 3 4 4 5
15959 166/300 Read block 4382 len 24576
15960 167/300 Write block 2879 len 24576 data: 8 7 5 5 3 3
15961 168/300 Read block 3284 len 32768
15962 169/300 Write block 4069 len 36864 data: 4 2 3 3 2 3 3 3 5
15963 170/300 Read block 2691 len 28672
15964 171/300 Flush
15965 172/300 Read block 2443 len 16384
15966 173/300 Read block 296 len 20480
15967 174/300 Read block 2208 len 28672
15968 175/300 Write block 57 len 24576 data: 7 6 7 7 7 8
15969 176/300 Flush
15970 177/300 Write block 409 len 40960 data: 7 7 7 5 5 5 6 6 6 6
15971 178/300 Read block 1019 len 36864
15972 179/300 Read block 194 len 28672
15973 180/300 Flush
15974 181/300 Read block 2257 len 4096
15975 182/300 Write block 1363 len 28672 data: 5 5 6 5 4 4 3
15976 183/300 Read block 1346 len 36864
15977 184/300 Write block 4069 len 8192 data: 5 3
15978 185/300 Read block 4800 len 4096
15979 186/300 Write block 584 len 36864 data: 2 3 3 4 4 3 2 3 5
15980 187/300 Write block 838 len 20480 data: 6 6 7 5 6
15981 188/300 Write block 3443 len 36864 data: 5 5 4 5 3 3 4 4 4
15982 189/300 Write block 2005 len 16384 data: 4 4 4 4
15983 190/300 Write block 1390 len 24576 data: 4 3 3 3 2 2
15984 191/300 Read block 2070 len 28672
15985 192/300 Read block 1393 len 12288
15986 193/300 Flush
15987 194/300 Read block 1056 len 20480
15988 195/300 Write block 2022 len 36864 data: 5 4 4 4 3 3 4 4 4
15989 196/300 Read block 4931 len 20480
15990 197/300 Read block 2090 len 36864
15991 198/300 Read block 773 len 8192
15992 199/300 Flush
15993 200/300 Write block 117 len 12288 data: 5 4 3
15994 201/300 Read block 3742 len 16384
15995 202/300 Read block 1627 len 40960
15996 203/300 Read block 3372 len 4096
15997 204/300 Read block 1782 len 8192
15998 205/300 Write block 2075 len 20480 data: 3 3 2 2 2
15999 206/300 Read block 3104 len 8192
16000 207/300 Write block 1865 len 4096 data: 7
16001 208/300 Read block 3208 len 16384
16002 209/300 Read block 2127 len 40960
16003 210/300 Read block 310 len 16384
16004 211/300 Read block 4803 len 32768
16005 212/300 Read block 3020 len 28672
16006 213/300 Write block 1806 len 36864 data: 3 4 4 4 5 5 5 4 3
16007 214/300 Read block 1738 len 12288
16008 215/300 Read block 1307 len 40960
16009 216/300 Write block 626 len 36864 data: 5 5 5 4 4 4 4 4 4
16010 217/300 Write block 1593 len 4096 data: 5
16011 218/300 Write block 291 len 32768 data: 4 2 2 2 3 4 4 4
16012 219/300 Write block 1680 len 4096 data: 4
16013 220/300 Write block 625 len 12288 data: 4 6 6
16014 221/300 Write block 2418 len 20480 data: 3 3 3 4 4
16015 222/300 Read block 237 len 4096
16016 223/300 Write block 507 len 16384 data: 3 3 2 2
16017 224/300 Read block 422 len 40960
16018 225/300 Write block 1410 len 12288 data: 6 6 6
16019 226/300 Read block 4691 len 20480
16020 227/300 Read block 1825 len 12288
16021 228/300 Write block 4455 len 28672 data: 5 6 6 5 4 3 3
16022 229/300 Read block 96 len 8192
16023 230/300 Write block 3405 len 20480 data: 6 6 6 3 4
16024 231/300 Read block 3960 len 24576
16025 232/300 Read block 1722 len 36864
16026 233/300 Flush
16027 234/300 Write block 267 len 4096 data: 2
16028 235/300 Read block 3875 len 12288
16029 236/300 Write block 2220 len 4096 data: 6
16030 237/300 Write block 1738 len 16384 data: 5 5 5 3
16031 238/300 Write block 285 len 4096 data: 4
16032 239/300 Read block 4345 len 16384
16033 240/300 Read block 3828 len 28672
16034 241/300 Read block 313 len 4096
16035 242/300 Write block 4306 len 16384 data: 5 5 6 5
16036 243/300 Read block 3355 len 8192
16037 244/300 Read block 3369 len 28672
16038 245/300 Flush
16039 246/300 Write block 2258 len 8192 data: 4 4
16040 247/300 Write block 2328 len 32768 data: 4 3 3 3 3 3 3 4
16041 248/300 Write block 3836 len 20480 data: 5 6 5 4 5
16042 249/300 Write block 3664 len 12288 data: 4 5 4
16043 250/300 Read block 2450 len 32768
16044 251/300 Flush
16045 252/300 Read block 3977 len 16384
16046 253/300 Write block 1812 len 24576 data: 6 5 4 4 4 4
16047 254/300 Read block 4686 len 28672
16048 255/300 Read block 4564 len 8192
16049 256/300 Write block 2133 len 12288 data: 3 3 4
16050 257/300 Read block 1504 len 12288
16051 258/300 Read block 3630 len 28672
16052 259/300 Read block 259 len 28672
16053 260/300 Write block 3456 len 4096 data: 4
16054 261/300 Flush
16055 262/300 Write block 4800 len 8192 data: 4 4
16056 263/300 Read block 773 len 12288
16057 264/300 Read block 2213 len 4096
16058 265/300 Read block 276 len 12288
16059 266/300 Write block 1948 len 40960 data: 4 6 5 6 6 4 5 7 6 6
16060 267/300 Flush
16061 268/300 Write block 4136 len 32768 data: 4 5 4 3 3 3 4 5
16062 269/300 Write block 1951 len 28672 data: 7 7 5 6 8 7 7
16063 270/300 Flush
16064 271/300 Flush
16065 272/300 Read block 1528 len 32768
16066 273/300 Read block 599 len 40960
16067 274/300 Write block 1997 len 20480 data: 4 5 5 6 4
16068 275/300 Read block 2134 len 8192
16069 276/300 Flush
16070 277/300 Write block 1799 len 20480 data: 6 5 4 3 3
16071 278/300 Read block 1375 len 28672
16072 279/300 Read block 3539 len 4096
16073 280/300 Read block 1001 len 28672
16074 281/300 Flush
16075 282/300 Write block 1592 len 40960 data: 5 6 5 5 5 4 3 3 3 3
16076 283/300 Read block 393 len 36864
16077 284/300 Write block 1878 len 4096 data: 5
16078 285/300 Read block 3690 len 24576
16079 286/300 Read block 4173 len 28672
16080 287/300 Write block 2321 len 32768 data: 3 3 3 3 3 3 3 5
16081 288/300 Write block 512 len 32768 data: 2 2 2 3 2 2 2 2
16082 289/300 Flush
16083 290/300 Read block 2102 len 24576
16084 291/300 Write block 3929 len 20480 data: 6 6 7 8 7
16085 292/300 Write block 126 len 40960 data: 4 4 4 5 5 5 5 4 4 4
16086 293/300 Write block 873 len 12288 data: 7 6 6
16087 294/300 Read block 1906 len 36864
16088 295/300 Write block 3146 len 12288 data: 5 5 4
16089 296/300 Read block 18 len 28672
16090 297/300 Write block 3862 len 4096 data: 5
16091 298/300 Read block 600 len 36864
16092 299/300 Write block 2467 len 36864 data: 3 3 3 4 5 6 5 4 5
16093 300/300 Read block 837 len 24576
16094 Main task has work to do, go find it
16095 got dsc Start(1)
16096 start 1
16097 Sep 22 23:27:19.388 INFO request completed, latency_us: 308, response_code: 204, uri: /start/cid/1, method: POST, req_id: 5579b292-f72c-4644-acc5-ee0b0b7c57c3, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
16098 Replay: started 1, returned:Ok(())
16099 ----------------------------------------------------------------
16100 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16101 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
16102 5144 Acked 6143 Write 9 Done New Done false
16103 5145 Acked 6144 Write 6 Done New Done false
16104 5146 Acked 6145 Write 7 Done New Done false
16105 5147 Acked 6146 Read 10 Done New Done false
16106 5148 Acked 6147 Read 8 Done New Done false
16107 5149 Acked 6148 Write 7 Done New Done false
16108 5150 Acked 6149 Read 2 Done New Done false
16109 5151 Acked 6150 Flush 0 Done New Done false
16110 5152 Acked 6151 Write 7 Done New Done false
16111 5153 Acked 6152 Write 2 Done New Done false
16112 5154 Acked 6153 Write 2 Done New Done false
16113 5155 Acked 6154 Write 6 Done New Done false
16114 5156 Acked 6155 Write 8 Done New Done false
16115 5157 Acked 6156 Read 8 Done New Done false
16116 5158 Acked 6157 Flush 0 Done New Done false
16117 5159 Acked 6158 Write 10 Done New Done false
16118 5160 Acked 6159 Read 8 Done New Done false
16119 5161 Acked 6160 Read 9 Done New Done false
16120 5162 Acked 6161 Write 9 Done New Done false
16121 5163 Acked 6162 Read 4 Done New Done false
16122 5164 Acked 6163 Write 2 Done New Done false
16123 5165 Acked 6164 Flush 0 Done New Done false
16124 5166 Acked 6165 Read 8 Done New Done false
16125 5167 Acked 6166 Write 9 Done New Done false
16126 5168 Acked 6167 Read 2 Done New Done false
16127 5169 Acked 6168 Flush 0 Done New Done false
16128 5170 Acked 6169 Write 8 Done New Done false
16129 5171 Acked 6170 Read 3 Done New Done false
16130 5172 Acked 6171 Write 9 Done New Done false
16131 5173 Acked 6172 Write 4 Done New Done false
16132 5174 Acked 6173 Write 4 Done New Done false
16133 5175 Acked 6174 Read 6 Done New Done false
16134 5176 Acked 6175 Read 7 Done New Done false
16135 5177 Acked 6176 Read 8 Done New Done false
16136 5178 Acked 6177 Write 5 Done New Done false
16137 5179 Acked 6178 Flush 0 Done New Done false
16138 5180 Acked 6179 Read 8 Done New Done false
16139 5181 Acked 6180 Read 2 Done New Done false
16140 5182 Acked 6181 Write 3 Done New Done false
16141 5183 Acked 6182 Flush 0 Done New Done false
16142 5184 Acked 6183 Read 3 Done New Done false
16143 5185 Acked 6184 Write 2 Done New Done false
16144 5186 Acked 6185 Write 8 Done New Done false
16145 5187 Acked 6186 Read 9 Done New Done false
16146 5188 Acked 6187 Flush 0 Done New Done false
16147 5189 Acked 6188 Write 4 Done New Done false
16148 5190 Acked 6189 Read 4 Done New Done false
16149 5191 Acked 6190 Read 3 Done New Done false
16150 5192 Acked 6191 Write 7 Done New Done false
16151 5193 Acked 6192 Read 4 Done New Done false
16152 5194 Acked 6193 Read 7 Done New Done false
16153 5195 Acked 6194 Read 5 Done New Done false
16154 5196 Acked 6195 Read 5 Done New Done false
16155 5197 Acked 6196 Write 6 Done New Done false
16156 5198 Acked 6197 Write 8 Done New Done false
16157 5199 Acked 6198 Read 5 Done New Done false
16158 5200 Acked 6199 Read 8 Done New Done false
16159 5201 Acked 6200 Write 2 Done New Done false
16160 5202 Acked 6201 Read 7 Done New Done false
16161 5203 Acked 6202 Flush 0 Done New Done false
16162 5204 Acked 6203 Read 8 Done New Done false
16163 5205 Acked 6204 Write 8 Done New Done false
16164 5206 Acked 6205 Read 3 Done New Done false
16165 5207 Acked 6206 Read 1 Done New Done false
16166 5208 Acked 6207 Read 8 Done New Done false
16167 5209 Acked 6208 Read 9 Done New Done false
16168 5210 Acked 6209 Read 2 Done New Done false
16169 5211 Acked 6210 Read 3 Done New Done false
16170 5212 Acked 6211 Flush 0 Done New Done false
16171 5213 Acked 6212 Read 6 Done New Done false
16172 5214 Acked 6213 Flush 0 Done New Done false
16173 5215 Acked 6214 Read 4 Done New Done false
16174 5216 Acked 6215 Read 6 Done New Done false
16175 5217 Acked 6216 Write 1 Done New Done false
16176 5218 Acked 6217 Read 5 Done New Done false
16177 5219 Acked 6218 Read 7 Done New Done false
16178 5220 Acked 6219 Read 10 Done New Done false
16179 5221 Acked 6220 Write 5 Done New Done false
16180 5222 Acked 6221 Write 3 Done New Done false
16181 5223 Acked 6222 Read 9 Done New Done false
16182 5224 Acked 6223 Write 4 Done New Done false
16183 5225 Acked 6224 Read 8 Done New Done false
16184 5226 Acked 6225 Flush 0 Done New Done false
16185 5227 Acked 6226 Read 5 Done New Done false
16186 5228 Acked 6227 Read 5 Done New Done false
16187 5229 Acked 6228 Write 7 Done New Done false
16188 5230 Acked 6229 Read 4 Done New Done false
16189 5231 Acked 6230 Read 8 Done New Done false
16190 5232 Acked 6231 Write 6 Done New Done false
16191 5233 Acked 6232 Write 3 Done New Done false
16192 5234 Acked 6233 Write 9 Done New Done false
16193 5235 Acked 6234 Read 4 Done New Done false
16194 5236 Acked 6235 Write 5 Done New Done false
16195 5237 Acked 6236 Write 4 Done New Done false
16196 5238 Acked 6237 Read 10 Done New Done false
16197 5239 Acked 6238 Read 3 Done New Done false
16198 5240 Acked 6239 Read 5 Done New Done false
16199 5241 Acked 6240 Flush 0 Done New Done false
16200 5242 Acked 6241 Write 4 Done New Done false
16201 5243 Acked 6242 Read 1 Done New Done false
16202 5244 Acked 6243 Write 10 Done New Done false
16203 5245 Acked 6244 Write 6 Done New Done false
16204 5246 Acked 6245 Read 5 Done New Done false
16205 5247 Acked 6246 Read 2 Done New Done false
16206 5248 Acked 6247 Read 4 Done New Done false
16207 5249 Acked 6248 Read 10 Done New Done false
16208 5250 Acked 6249 Read 10 Done New Done false
16209 5251 Acked 6250 Write 2 Done New Done false
16210 5252 Acked 6251 Write 2 Done New Done false
16211 5253 Acked 6252 Write 10 Done New Done false
16212 5254 Acked 6253 Read 9 Done New Done false
16213 5255 Acked 6254 Flush 0 Done New Done false
16214 5256 Acked 6255 Flush 0 Done New Done false
16215 5257 Acked 6256 Write 1 Done New Done false
16216 5258 Acked 6257 Read 10 Done New Done false
16217 5259 Acked 6258 Read 6 Done New Done false
16218 5260 Acked 6259 Write 9 Done New Done false
16219 5261 Acked 6260 Flush 0 Done New Done false
16220 5262 Acked 6261 Write 8 Done New Done false
16221 5263 Acked 6262 Write 7 Done New Done false
16222 5264 Acked 6263 Flush 0 Done New Done false
16223 5265 Acked 6264 Read 8 Done New Done false
16224 5266 Acked 6265 Write 8 Done New Done false
16225 5267 Acked 6266 Write 4 Done New Done false
16226 5268 Acked 6267 Write 10 Done New Done false
16227 5269 Acked 6268 Flush 0 Done New Done false
16228 5270 Acked 6269 Write 10 Done New Done false
16229 5271 Acked 6270 Read 1 Done New Done false
16230 5272 Acked 6271 Read 5 Done New Done false
16231 5273 Acked 6272 Read 6 Done New Done false
16232 5274 Acked 6273 Flush 0 Done New Done false
16233 5275 Acked 6274 Write 10 Done New Done false
16234 5276 Acked 6275 Read 9 Done New Done false
16235 5277 Acked 6276 Write 3 Done New Done false
16236 5278 Acked 6277 Write 3 Done New Done false
16237 5279 Acked 6278 Read 7 Done New Done false
16238 5280 Acked 6279 Write 6 Done New Done false
16239 5281 Acked 6280 Flush 0 Done New Done false
16240 5282 Acked 6281 Read 5 Done New Done false
16241 5283 Acked 6282 Read 10 Done New Done false
16242 5284 Acked 6283 Write 3 Done New Done false
16243 5285 Acked 6284 Read 9 Done New Done false
16244 5286 Acked 6285 Read 9 Done New Done false
16245 5287 Acked 6286 Write 10 Done New Done false
16246 5288 Acked 6287 Write 10 Done New Done false
16247 5289 Acked 6288 Read 3 Done New Done false
16248 5290 Acked 6289 Read 9 Done New Done false
16249 5291 Acked 6290 Read 7 Done New Done false
16250 5292 Acked 6291 Flush 0 Done New Done false
16251 5293 Acked 6292 Write 5 Done New Done false
16252 5294 Acked 6293 Flush 0 Done New Done false
16253 5295 Acked 6294 Read 4 Done New Done false
16254 5296 Acked 6295 Read 8 Done New Done false
16255 5297 Acked 6296 Flush 0 Done New Done false
16256 5298 Acked 6297 Read 9 Done New Done false
16257 5299 Acked 6298 Read 2 Done New Done false
16258 5300 Acked 6299 Write 5 Done New Done false
16259 5301 Acked 6300 Read 4 Done New Done false
16260 5302 Acked 6301 Write 10 Done New Done false
16261 5303 Acked 6302 Flush 0 Done New Done false
16262 5304 Acked 6303 Read 10 Done New Done false
16263 5305 Acked 6304 Read 8 Done New Done false
16264 5306 Acked 6305 Read 5 Done New Done false
16265 5307 Acked 6306 Read 4 Done New Done false
16266 5308 Acked 6307 Read 4 Done New Done false
16267 5309 Acked 6308 Write 6 Done New Done false
16268 5310 Acked 6309 Read 6 Done New Done false
16269 5311 Acked 6310 Write 6 Done New Done false
16270 5312 Acked 6311 Read 8 Done New Done false
16271 5313 Acked 6312 Write 9 Done New Done false
16272 5314 Acked 6313 Read 7 Done New Done false
16273 5315 Acked 6314 Flush 0 Done New Done false
16274 5316 Acked 6315 Read 4 Done New Done false
16275 5317 Acked 6316 Read 5 Done New Done false
16276 5318 Acked 6317 Read 7 Done New Done false
16277 5319 Acked 6318 Write 6 Done New Done false
16278 5320 Acked 6319 Flush 0 Done New Done false
16279 5321 Acked 6320 Write 10 Done New Done false
16280 5322 Acked 6321 Read 9 Done New Done false
16281 5323 Acked 6322 Read 7 Done New Done false
16282 5324 Acked 6323 Flush 0 Done New Done false
16283 5325 Acked 6324 Read 1 Done New Done false
16284 5326 Acked 6325 Write 7 Done New Done false
16285 5327 Acked 6326 Read 9 Done New Done false
16286 5328 Acked 6327 Write 2 Done New Done false
16287 5329 Acked 6328 Read 1 Done New Done false
16288 5330 Acked 6329 Write 9 Done New Done false
16289 5331 Acked 6330 Write 5 Done New Done false
16290 5332 Acked 6331 Write 9 Done New Done false
16291 5333 Acked 6332 Write 4 Done New Done false
16292 5334 Acked 6333 Write 6 Done New Done false
16293 5335 Acked 6334 Read 7 Done New Done false
16294 5336 Acked 6335 Read 3 Done New Done false
16295 5337 Acked 6336 Flush 0 Done New Done false
16296 5338 Acked 6337 Read 5 Done New Done false
16297 5339 Acked 6338 Write 9 Done New Done false
16298 5340 Acked 6339 Read 5 Done New Done false
16299 5341 Acked 6340 Read 9 Done New Done false
16300 5342 Acked 6341 Read 2 Done New Done false
16301 5343 Acked 6342 Flush 0 Done New Done false
16302 5344 Acked 6343 Write 3 Done New Done false
16303 5345 Acked 6344 Read 4 Done New Done false
16304 5346 Acked 6345 Read 10 Done New Done false
16305 5347 Acked 6346 Read 1 Done New Done false
16306 5348 Acked 6347 Read 2 Done New Done false
16307 5349 Acked 6348 Write 5 Done New Done false
16308 5350 Acked 6349 Read 2 Done New Done false
16309 5351 Acked 6350 Write 1 Done New Done false
16310 5352 Acked 6351 Read 4 Done New Done false
16311 5353 Acked 6352 Read 10 Done New Done false
16312 5354 Acked 6353 Read 4 Done New Done false
16313 5355 Acked 6354 Read 8 Done New Done false
16314 5356 Acked 6355 Read 7 Done New Done false
16315 5357 Acked 6356 Write 9 Done New Done false
16316 5358 Acked 6357 Read 3 Done New Done false
16317 5359 Acked 6358 Read 10 Done New Done false
16318 5360 Acked 6359 Write 9 Done New Done false
16319 5361 Acked 6360 Write 1 Done New Done false
16320 5362 Acked 6361 Write 8 Done New Done false
16321 5363 Acked 6362 Write 1 Done New Done false
16322 5364 Acked 6363 Write 3 Done New Done false
16323 5365 Acked 6364 Write 5 Done New Done false
16324 5366 Acked 6365 Read 1 Done New Done false
16325 5367 Acked 6366 Write 4 Done New Done false
16326 5368 Acked 6367 Read 10 Done New Done false
16327 5369 Acked 6368 Write 3 Done New Done false
16328 5370 Acked 6369 Read 5 Done New Done false
16329 5371 Acked 6370 Read 3 Done New Done false
16330 5372 Acked 6371 Write 7 Done New Done false
16331 5373 Acked 6372 Read 2 Done New Done false
16332 5374 Acked 6373 Write 5 Done New Done false
16333 5375 Acked 6374 Read 6 Done New Done false
16334 5376 Acked 6375 Read 9 Done New Done false
16335 5377 Acked 6376 Flush 0 Done New Done false
16336 5378 Acked 6377 Write 1 Done New Done false
16337 5379 Acked 6378 Read 3 Done New Done false
16338 5380 Acked 6379 Write 1 Done New Done false
16339 5381 Acked 6380 Write 4 Done New Done false
16340 5382 Acked 6381 Write 1 Done New Done false
16341 5383 Acked 6382 Read 4 Done New Done false
16342 5384 Acked 6383 Read 7 Done New Done false
16343 5385 Acked 6384 Read 1 Done New Done false
16344 5386 Acked 6385 Write 4 Done New Done false
16345 5387 Acked 6386 Read 2 Done New Done false
16346 5388 Acked 6387 Read 7 Done New Done false
16347 5389 Acked 6388 Flush 0 Done New Done false
16348 5390 Acked 6389 Write 2 Done New Done false
16349 5391 Acked 6390 Write 8 Done New Done false
16350 5392 Acked 6391 Write 5 Done New Done false
16351 5393 Acked 6392 Write 3 Done New Done false
16352 5394 Acked 6393 Read 8 Done New Done false
16353 5395 Acked 6394 Flush 0 Done New Done false
16354 5396 Acked 6395 Read 4 Done New Done false
16355 5397 Acked 6396 Write 6 Done New Done false
16356 5398 Acked 6397 Read 7 Done New Done false
16357 5399 Acked 6398 Read 2 Done New Done false
16358 5400 Acked 6399 Write 3 Done New Done false
16359 5401 Acked 6400 Read 3 Done New Done false
16360 5402 Acked 6401 Read 7 Done New Done false
16361 5403 Acked 6402 Read 7 Done New Done false
16362 5404 Acked 6403 Write 1 Done New Done false
16363 5405 Acked 6404 Flush 0 Done New Done false
16364 5406 Acked 6405 Write 2 Done New Done false
16365 5407 Acked 6406 Read 3 Done New Done false
16366 5408 Acked 6407 Read 1 Done New Done false
16367 5409 Acked 6408 Read 3 Done New Done false
16368 5410 Acked 6409 Write 10 Done New Done false
16369 5411 Acked 6410 Flush 0 Done New Done false
16370 5412 Acked 6411 Write 8 Done New Done false
16371 5413 Acked 6412 Write 7 Done New Done false
16372 5414 Acked 6413 Flush 0 Done New Done false
16373 5415 Acked 6414 Flush 0 Done New Done false
16374 5416 Acked 6415 Read 8 Done New Done false
16375 5417 Acked 6416 Read 10 Done New Done false
16376 5418 Acked 6417 Write 5 Done New Done false
16377 5419 Acked 6418 Read 2 Done New Done false
16378 5420 Acked 6419 Flush 0 Done New Done false
16379 5421 Acked 6420 Write 5 Done New Done false
16380 5422 Acked 6421 Read 7 Done New Done false
16381 5423 Acked 6422 Read 1 Done New Done false
16382 5424 Acked 6423 Read 7 Done New Done false
16383 5425 Acked 6424 Flush 0 Done New Done false
16384 5426 Acked 6425 Write 10 Done New Done false
16385 5427 Acked 6426 Read 9 Done New Done false
16386 5428 Acked 6427 Write 1 Done New Done false
16387 5429 Acked 6428 Read 6 Done New Done false
16388 5430 Acked 6429 Read 7 Done New Done false
16389 5431 Acked 6430 Write 8 Done New Done false
16390 5432 Acked 6431 Write 8 Done New Done false
16391 5433 Acked 6432 Flush 0 Done New Done false
16392 5434 Acked 6433 Read 6 Done New Done false
16393 5435 Acked 6434 Write 5 Done New Done false
16394 5436 Acked 6435 Write 10 Done New Done false
16395 5437 Acked 6436 Write 3 Done New Done false
16396 5438 Acked 6437 Read 9 Done New Done false
16397 5439 Acked 6438 Write 3 Done New Done false
16398 5440 Acked 6439 Read 7 Done New Done false
16399 5441 Acked 6440 Write 1 Done New Done false
16400 5442 Acked 6441 Read 9 Done New Done false
16401 5443 Acked 6442 Write 9 Done New Done false
16402 5444 Acked 6443 Read 6 Done New Done false
16403 STATES DS:0 DS:1 DS:2 TOTAL
16404 New 0 301 0 301
16405 Sent 0 0 0 0
16406 Done 301 0 301 602
16407 Skipped 0 0 0 0
16408 Error 0 0 0 0
16409 Last Flush: 6432 6142 6432
16410 Downstairs last five completed: 6142 6141 6140 6139 6138 6137
16411 Upstairs last five completed: 5444 5443 5442 5441 5440 5439
16412 CLIENT: Up:0 ds:301 act:2
16413 [8820] got start action
16414 [8820] I am going to restart
16415 Starting downstairs at port 8820
16416 Make output file at "/tmp/dsc/downstairs-8820.txt"
16417 [8820][1] reports Starting
16418 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1191)
16419 [8820][1] reports Running
164202023-09-22T23:27:21.436ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
164212023-09-22T23:27:21.436ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
164222023-09-22T23:27:21.436ZINFOcrucible: [1] client re-new 302 jobs since flush 6142 = downstairs
164232023-09-22T23:27:21.437ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
164242023-09-22T23:27:21.438ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
164252023-09-22T23:27:21.438ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
164262023-09-22T23:27:21.438ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
164272023-09-22T23:27:21.438ZINFOcrucible: [1] send last flush ID to this DS: 6142
164282023-09-22T23:27:21.438ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
164292023-09-22T23:27:21.438ZINFOcrucible: [1] Replied this last flush ID: 6142
164302023-09-22T23:27:21.438ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
164312023-09-22T23:27:21.438ZINFOcrucible: [1] Transition from Offline to Replay
164322023-09-22T23:27:21.438ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
164332023-09-22T23:27:21.438ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
164342023-09-22T23:27:21.438ZINFOcrucible: [1] Transition from Replay to Active
164352023-09-22T23:27:21.438ZINFOcrucible: [1] Starts cmd_loop
164362023-09-22T23:27:21.438ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
164372023-09-22T23:27:21.438ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
164382023-09-22T23:27:21.438ZINFOcrucible: Set check for repair
164392023-09-22T23:27:22.440ZINFOcrucible: Checking if live repair is needed
164402023-09-22T23:27:22.440ZINFOcrucible: No Live Repair required at this time
164412023-09-22T23:27:22.440ZWARNcrucible: [1] flow control end
16442 ----------------------------------------------------------------
16443 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16444 Downstairs last five completed: 6444 6443 6442 6441 6440 6439
16445 Upstairs last five completed: 5445 5444 5443 5442 5441 5440
16446 CLIENT: Up:0 ds:0 act:3
16447 Replay: All jobs finished, all DS active.
16448 Main task has work to do, go find it
16449 got dsc Stop(2)
16450 stop 2
16451 Sep 22 23:27:23.393 INFO request completed, latency_us: 528, response_code: 204[8830] Got stop action so:false kr:false
16452 , uri: /stop/cid/2, method: POST, req_id: 98884155-b3c5-4bcd-8e41-4a9c9b954b9c, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
16453 Sep 22 23:27:23.395 INFO request completed, latency_us: 193, response_code: 200, uri: /state/cid/2, method: GET, req_id: fe6804a9-fc0b-461e-833e-06db6176f349, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
164542023-09-22T23:27:23.395ZWARNcrucible: [2] downstairs disconnected
164552023-09-22T23:27:23.395ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
164562023-09-22T23:27:23.395ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
164572023-09-22T23:27:23.395ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
164582023-09-22T23:27:23.395ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
164592023-09-22T23:27:23.395ZWARNcrucible: [2] pm_task rx.recv() is None
164602023-09-22T23:27:23.395ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
16461 [8830] Exited with: ExitStatus(unix_wait_status(9))
16462 [8830][2] reports Exit
16463 Sep 22 23:27:27.398 INFO request completed, latency_us: 260, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7f0b40f8-83a4-4649-aaa5-12e53be3b5a9, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
16464 001/300 Read block 2282 len 12288
16465 002/300 Flush
16466 003/300 Read block 2175 len 36864
16467 004/300 Write block 1977 len 16384 data: 5 4 4 5
16468 005/300 Write block 2742 len 12288 data: 4 4 3
16469 006/300 Write block 4041 len 24576 data: 4 4 4 3 3 5
16470 007/300 Read block 1348 len 16384
16471 008/300 Read block 2945 len 20480
16472 009/300 Write block 2435 len 36864 data: 3 2 3 3 3 3 3 3 3
16473 010/300 Flush
16474 011/300 Write block 2437 len 24576 data: 4 4 4 4 4 4
16475 012/300 Write block 1882 len 36864 data: 5 4 3 2 2 3 3 4 4
16476 013/300 Write block 4426 len 32768 data: 6 4 4 3 3 3 2 3
16477 014/300 Read block 3740 len 28672
16478 015/300 Flush
16479 016/300 Write block 4161 len 40960 data: 4 4 4 4 4 4 6 5 5 5
16480 017/300 Flush
16481 018/300 Write block 3880 len 32768 data: 2 2 2 2 2 2 2 2
16482 019/300 Read block 4249 len 16384
16483 020/300 Read block 1877 len 40960
16484 021/300 Read block 943 len 20480
16485 022/300 Read block 1414 len 40960
16486 023/300 Write block 3171 len 12288 data: 6 5 4
16487 024/300 Read block 821 len 4096
16488 025/300 Read block 3240 len 4096
16489 026/300 Write block 2093 len 32768 data: 6 6 5 5 4 4 3 3
16490 027/300 Write block 348 len 4096 data: 4
16491 028/300 Write block 925 len 24576 data: 4 3 3 4 4 4
16492 029/300 Write block 2023 len 32768 data: 5 5 5 4 4 5 5 5
16493 030/300 Read block 1949 len 12288
16494 031/300 Write block 2261 len 12288 data: 5 4 3
16495 032/300 Read block 2135 len 4096
16496 033/300 Flush
16497 034/300 Read block 16 len 12288
16498 035/300 Read block 2692 len 32768
16499 036/300 Write block 429 len 20480 data: 2 3 3 3 2
16500 037/300 Read block 2324 len 12288
16501 038/300 Write block 1374 len 4096 data: 4
16502 039/300 Write block 4452 len 20480 data: 5 4 5 6 7
16503 040/300 Read block 1127 len 36864
16504 041/300 Read block 945 len 12288
16505 042/300 Read block 1409 len 36864
16506 043/300 Read block 4553 len 28672
16507 044/300 Flush
16508 045/300 Read block 3811 len 24576
16509 046/300 Read block 2935 len 4096
16510 047/300 Flush
16511 048/300 Read block 2306 len 8192
16512 049/300 Write block 2823 len 28672 data: 4 4 5 4 4 5 6
16513 050/300 Write block 2736 len 4096 data: 3
16514 051/300 Write block 2567 len 36864 data: 2 2 3 3 3 4 6 8 9
16515 052/300 Flush
16516 053/300 Flush
16517 054/300 Write block 2034 len 20480 data: 4 5 4 2 2
16518 055/300 Read block 1208 len 24576
16519 056/300 Read block 1028 len 36864
16520 057/300 Write block 2139 len 16384 data: 6 5 4 4
16521 058/300 Read block 1875 len 40960
16522 059/300 Read block 162 len 36864
16523 060/300 Write block 4980 len 12288 data: 4 4 4
16524 061/300 Read block 2151 len 32768
16525 062/300 Read block 818 len 40960
16526 063/300 Read block 3899 len 8192
16527 064/300 Write block 4453 len 20480 data: 5 6 7 8 7
16528 065/300 Write block 1286 len 20480 data: 2 2 2 3 6
16529 066/300 Read block 2246 len 8192
16530 067/300 Write block 3610 len 4096 data: 5
16531 068/300 Write block 4002 len 32768 data: 7 6 5 3 3 4 4 4
16532 069/300 Read block 2451 len 8192
16533 070/300 Read block 3670 len 36864
16534 071/300 Read block 148 len 8192
16535 072/300 Read block 4733 len 36864
16536 073/300 Read block 3026 len 12288
16537 074/300 Read block 1888 len 32768
16538 075/300 Read block 4977 len 16384
16539 076/300 Read block 2882 len 36864
16540 077/300 Read block 1886 len 8192
16541 078/300 Read block 987 len 8192
16542 079/300 Write block 3263 len 8192 data: 5 3
16543 080/300 Write block 223 len 8192 data: 4 4
16544 081/300 Write block 204 len 20480 data: 3 3 3 3 3
16545 082/300 Read block 4709 len 24576
16546 083/300 Read block 1707 len 28672
16547 084/300 Read block 2824 len 20480
16548 085/300 Write block 3319 len 8192 data: 3 2
16549 086/300 Read block 1709 len 12288
16550 087/300 Read block 3375 len 4096
16551 088/300 Write block 1192 len 20480 data: 3 2 3 3 4
16552 089/300 Write block 1188 len 8192 data: 7 7
16553 090/300 Read block 600 len 28672
16554 091/300 Read block 3857 len 36864
16555 092/300 Read block 809 len 24576
16556 093/300 Write block 4058 len 20480 data: 6 6 6 7 8
16557 094/300 Read block 4963 len 4096
16558 095/300 Read block 2163 len 28672
16559 096/300 Write block 2973 len 4096 data: 4
16560 097/300 Read block 4053 len 40960
16561 098/300 Write block 3289 len 8192 data: 6 7
16562 099/300 Write block 4113 len 4096 data: 3
16563 100/300 Read block 4163 len 20480
16564 101/300 Write block 3265 len 12288 data: 4 4 5
16565 102/300 Write block 3639 len 20480 data: 2 2 2 2 2
16566 103/300 Read block 2571 len 40960
16567 104/300 Read block 2358 len 32768
16568 105/300 Read block 3288 len 32768
16569 106/300 Write block 2496 len 16384 data: 3 3 3 2
16570 107/300 Write block 1028 len 4096 data: 5
16571 108/300 Read block 1303 len 16384
16572 109/300 Read block 3169 len 36864
16573 110/300 Flush
16574 111/300 Read block 2282 len 8192
16575 112/300 Write block 2604 len 32768 data: 2 3 3 3 2 2 2 4
16576 113/300 Write block 699 len 40960 data: 6 5 6 5 3 4 3 3 4 5
16577 114/300 Read block 3567 len 16384
16578 115/300 Read block 602 len 40960
16579 116/300 Read block 2569 len 16384
16580 117/300 Read block 3773 len 24576
16581 118/300 Read block 2114 len 20480
16582 119/300 Write block 3182 len 20480 data: 5 6 6 6 5
16583 120/300 Read block 4254 len 24576
16584 121/300 Write block 4457 len 12288 data: 8 6 5
16585 122/300 Write block 2067 len 28672 data: 5 4 5 6 4 4 2
16586 123/300 Read block 4809 len 16384
16587 124/300 Write block 2859 len 4096 data: 5
16588 125/300 Read block 3061 len 16384
16589 126/300 Read block 3032 len 40960
16590 127/300 Write block 3205 len 28672 data: 3 4 5 5 4 5 4
16591 128/300 Write block 204 len 4096 data: 4
16592 129/300 Read block 1250 len 12288
16593 130/300 Read block 3602 len 24576
16594 131/300 Read block 4472 len 20480
16595 132/300 Flush
16596 133/300 Read block 359 len 40960
16597 134/300 Read block 4950 len 32768
16598 135/300 Read block 3248 len 40960
16599 136/300 Read block 3125 len 16384
16600 137/300 Write block 1396 len 32768 data: 2 3 3 3 5 7 7 8
16601 138/300 Read block 1190 len 16384
16602 139/300 Read block 1253 len 28672
16603 140/300 Write block 868 len 36864 data: 6 6 7 7 6 8 7 7 7
16604 141/300 Read block 4377 len 16384
16605 142/300 Flush
16606 143/300 Write block 524 len 36864 data: 4 4 5 4 4 4 4 4 3
16607 144/300 Write block 339 len 20480 data: 6 5 3 4 3
16608 145/300 Read block 1977 len 8192
16609 146/300 Read block 688 len 12288
16610 147/300 Read block 532 len 4096
16611 148/300 Read block 3154 len 28672
16612 149/300 Read block 1720 len 16384
16613 150/300 Write block 4772 len 20480 data: 4 4 4 4 4
16614 151/300 Read block 2668 len 40960
16615 152/300 Read block 459 len 20480
16616 153/300 Write block 1754 len 4096 data: 5
16617 154/300 Flush
16618 155/300 Write block 2679 len 16384 data: 3 4 4 3
16619 156/300 Read block 3992 len 32768
16620 157/300 Flush
16621 158/300 Read block 3313 len 16384
16622 159/300 Read block 622 len 12288
16623 160/300 Read block 2886 len 16384
16624 161/300 Read block 4762 len 20480
16625 162/300 Write block 304 len 28672 data: 6 5 5 5 4 3 4
16626 163/300 Read block 4509 len 24576
16627 164/300 Read block 1354 len 40960
16628 165/300 Write block 1609 len 8192 data: 3 3
16629 166/300 Write block 2507 len 8192 data: 4 6
16630 167/300 Read block 384 len 16384
16631 168/300 Write block 2123 len 24576 data: 2 2 3 3 3 3
16632 169/300 Write block 4645 len 36864 data: 6 6 7 6 5 4 2 3 3
16633 170/300 Write block 761 len 28672 data: 7 7 6 5 5 5 4
16634 171/300 Read block 3017 len 24576
16635 172/300 Write block 491 len 28672 data: 5 6 6 5 4 3 4
16636 173/300 Write block 4124 len 4096 data: 4
16637 174/300 Flush
16638 175/300 Write block 4637 len 16384 data: 2 3 3 3
16639 176/300 Write block 4581 len 4096 data: 5
16640 177/300 Read block 1447 len 16384
16641 178/300 Write block 74 len 8192 data: 4 4
16642 179/300 Write block 1492 len 28672 data: 3 4 4 5 6 7 6
16643 180/300 Read block 1251 len 36864
16644 181/300 Read block 1752 len 28672
16645 182/300 Read block 4726 len 8192
16646 183/300 Read block 4946 len 4096
16647 184/300 Read block 4496 len 24576
16648 185/300 Read block 598 len 8192
16649 186/300 Write block 4802 len 32768 data: 5 5 5 5 6 5 6 6
16650 187/300 Write block 4979 len 24576 data: 4 5 5 5 5 8
16651 188/300 Read block 2674 len 12288
16652 189/300 Read block 3849 len 36864
16653 190/300 Read block 4940 len 20480
16654 191/300 Write block 4431 len 20480 data: 4 3 4 3 4
16655 192/300 Flush
16656 193/300 Read block 3860 len 4096
16657 194/300 Flush
16658 195/300 Flush
16659 196/300 Write block 2475 len 36864 data: 6 7 6 5 5 4 5 5 6
16660 197/300 Read block 2983 len 8192
16661 198/300 Read block 2934 len 32768
16662 199/300 Write block 4322 len 24576 data: 6 6 6 7 6 4
16663 200/300 Read block 192 len 36864
16664 201/300 Read block 3630 len 36864
16665 202/300 Write block 460 len 20480 data: 5 5 5 5 4
16666 203/300 Write block 2537 len 4096 data: 4
16667 204/300 Read block 1657 len 40960
16668 205/300 Read block 906 len 4096
16669 206/300 Flush
16670 207/300 Flush
16671 208/300 Write block 2827 len 28672 data: 5 6 7 6 5 7 8
16672 209/300 Read block 246 len 28672
16673 210/300 Read block 715 len 20480
16674 211/300 Write block 3723 len 36864 data: 5 4 4 4 5 6 5 4 3
16675 212/300 Read block 3872 len 20480
16676 213/300 Write block 4954 len 12288 data: 5 4 6
16677 214/300 Read block 4425 len 12288
16678 215/300 Flush
16679 216/300 Flush
16680 217/300 Read block 3442 len 8192
16681 218/300 Read block 3938 len 32768
16682 219/300 Read block 3538 len 32768
16683 220/300 Write block 1212 len 40960 data: 6 5 6 5 4 4 4 4 3 3
16684 221/300 Read block 4499 len 40960
16685 222/300 Write block 1282 len 4096 data: 3
16686 223/300 Read block 1700 len 8192
16687 224/300 Read block 345 len 20480
16688 225/300 Read block 4728 len 4096
16689 226/300 Read block 3233 len 8192
16690 227/300 Read block 2524 len 12288
16691 228/300 Read block 748 len 16384
16692 229/300 Read block 2082 len 8192
16693 230/300 Read block 1798 len 28672
16694 231/300 Read block 2352 len 40960
16695 232/300 Read block 4306 len 36864
16696 233/300 Read block 705 len 16384
16697 234/300 Write block 2721 len 4096 data: 5
16698 235/300 Write block 2279 len 40960 data: 6 6 6 6 6 5 5 5 4 4
16699 236/300 Write block 2315 len 32768 data: 4 4 4 4 3 3 4 4
16700 237/300 Read block 169 len 28672
16701 238/300 Read block 1994 len 12288
16702 239/300 Read block 3221 len 28672
16703 240/300 Flush
16704 241/300 Read block 4226 len 16384
16705 242/300 Read block 489 len 8192
16706 243/300 Read block 2975 len 24576
16707 244/300 Read block 1621 len 24576
16708 245/300 Read block 184 len 24576
16709 246/300 Write block 1138 len 20480 data: 3 5 5 4 4
16710 247/300 Flush
16711 248/300 Write block 2374 len 36864 data: 5 5 5 5 4 4 5 7 6
16712 249/300 Flush
16713 250/300 Read block 3030 len 40960
16714 251/300 Read block 4664 len 36864
16715 252/300 Read block 1995 len 16384
16716 253/300 Flush
16717 254/300 Read block 2577 len 4096
16718 255/300 Flush
16719 256/300 Read block 3185 len 8192
16720 257/300 Write block 4473 len 16384 data: 4 4 4 5
16721 258/300 Read block 372 len 16384
16722 259/300 Read block 1040 len 4096
16723 260/300 Write block 776 len 36864 data: 3 4 4 3 2 2 2 3 3
16724 261/300 Read block 4709 len 40960
16725 262/300 Read block 939 len 40960
16726 263/300 Read block 2121 len 20480
16727 264/300 Read block 4651 len 12288
16728 265/300 Read block 478 len 20480
16729 266/300 Read block 2328 len 20480
16730 267/300 Flush
16731 268/300 Write block 1764 len 36864 data: 5 5 6 7 6 6 7 9 8
16732 269/300 Read block 892 len 12288
16733 270/300 Read block 2997 len 40960
16734 271/300 Read block 3941 len 12288
16735 272/300 Read block 225 len 32768
16736 273/300 Read block 716 len 16384
16737 274/300 Read block 1192 len 28672
16738 275/300 Write block 3763 len 24576 data: 4 3 2 2 2 3
16739 276/300 Read block 1493 len 40960
16740 277/300 Write block 4653 len 28672 data: 4 3 3 5 6 6 6
16741 278/300 Read block 1141 len 8192
16742 279/300 Write block 4828 len 20480 data: 7 6 4 4 4
16743 280/300 Read block 3957 len 8192
16744 281/300 Write block 848 len 12288 data: 5 5 4
16745 282/300 Read block 519 len 24576
16746 283/300 Flush
16747 284/300 Write block 98 len 8192 data: 5 3
16748 285/300 Read block 4413 len 40960
16749 286/300 Flush
16750 287/300 Flush
16751 288/300 Flush
16752 289/300 Write block 3894 len 4096 data: 5
16753 290/300 Write block 4576 len 12288 data: 6 6 5
16754 291/300 Read block 596 len 24576
16755 292/300 Write block 2105 len 16384 data: 5 6 6 6
16756 293/300 Read block 1215 len 8192
16757 294/300 Read block 337 len 12288
16758 295/300 Write block 4166 len 12288 data: 5 7 6
16759 296/300 Write block 1875 len 28672 data: 5 4 4 6 5 6 5
16760 297/300 Read block 4621 len 24576
16761 298/300 Read block 2676 len 16384
16762 299/300 Write block 2608 len 20480 data: 3 3 3 5 4
16763 300/300 Read block 661 len 20480
16764 Main task has work to do, go find it
16765 got dsc Start(2)
16766 start 2
16767 Sep 22 23:27:28.309 INFO request completed, latency_us: 305, response_code: 204, uri: /start/cid/2, method: POST, req_id: cfcd6057-5997-4b1d-8b25-6b6537248f30, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
16768 Replay: started 2, returned:Ok(())
16769 ----------------------------------------------------------------
16770 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16771 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
16772 5446 Acked 6445 Read 3 Done Done New false
16773 5447 Acked 6446 Flush 0 Done Done New false
16774 5448 Acked 6447 Read 9 Done Done New false
16775 5449 Acked 6448 Write 4 Done Done New false
16776 5450 Acked 6449 Write 3 Done Done New false
16777 5451 Acked 6450 Write 6 Done Done New false
16778 5452 Acked 6451 Read 4 Done Done New false
16779 5453 Acked 6452 Read 5 Done Done New false
16780 5454 Acked 6453 Write 9 Done Done New false
16781 5455 Acked 6454 Flush 0 Done Done New false
16782 5456 Acked 6455 Write 6 Done Done New false
16783 5457 Acked 6456 Write 9 Done Done New false
16784 5458 Acked 6457 Write 8 Done Done New false
16785 5459 Acked 6458 Read 7 Done Done New false
16786 5460 Acked 6459 Flush 0 Done Done New false
16787 5461 Acked 6460 Write 10 Done Done New false
16788 5462 Acked 6461 Flush 0 Done Done New false
16789 5463 Acked 6462 Write 8 Done Done New false
16790 5464 Acked 6463 Read 4 Done Done New false
16791 5465 Acked 6464 Read 10 Done Done New false
16792 5466 Acked 6465 Read 5 Done Done New false
16793 5467 Acked 6466 Read 10 Done Done New false
16794 5468 Acked 6467 Write 3 Done Done New false
16795 5469 Acked 6468 Read 1 Done Done New false
16796 5470 Acked 6469 Read 1 Done Done New false
16797 5471 Acked 6470 Write 8 Done Done New false
16798 5472 Acked 6471 Write 1 Done Done New false
16799 5473 Acked 6472 Write 6 Done Done New false
16800 5474 Acked 6473 Write 8 Done Done New false
16801 5475 Acked 6474 Read 3 Done Done New false
16802 5476 Acked 6475 Write 3 Done Done New false
16803 5477 Acked 6476 Read 1 Done Done New false
16804 5478 Acked 6477 Flush 0 Done Done New false
16805 5479 Acked 6478 Read 3 Done Done New false
16806 5480 Acked 6479 Read 8 Done Done New false
16807 5481 Acked 6480 Write 5 Done Done New false
16808 5482 Acked 6481 Read 3 Done Done New false
16809 5483 Acked 6482 Write 1 Done Done New false
16810 5484 Acked 6483 Write 5 Done Done New false
16811 5485 Acked 6484 Read 9 Done Done New false
16812 5486 Acked 6485 Read 3 Done Done New false
16813 5487 Acked 6486 Read 9 Done Done New false
16814 5488 Acked 6487 Read 7 Done Done New false
16815 5489 Acked 6488 Flush 0 Done Done New false
16816 5490 Acked 6489 Read 6 Done Done New false
16817 5491 Acked 6490 Read 1 Done Done New false
16818 5492 Acked 6491 Flush 0 Done Done New false
16819 5493 Acked 6492 Read 2 Done Done New false
16820 5494 Acked 6493 Write 7 Done Done New false
16821 5495 Acked 6494 Write 1 Done Done New false
16822 5496 Acked 6495 Write 9 Done Done New false
16823 5497 Acked 6496 Flush 0 Done Done New false
16824 5498 Acked 6497 Flush 0 Done Done New false
16825 5499 Acked 6498 Write 5 Done Done New false
16826 5500 Acked 6499 Read 6 Done Done New false
16827 5501 Acked 6500 Read 9 Done Done New false
16828 5502 Acked 6501 Write 4 Done Done New false
16829 5503 Acked 6502 Read 10 Done Done New false
16830 5504 Acked 6503 Read 9 Done Done New false
16831 5505 Acked 6504 Write 3 Done Done New false
16832 5506 Acked 6505 Read 8 Done Done New false
16833 5507 Acked 6506 Read 10 Done Done New false
16834 5508 Acked 6507 Read 2 Done Done New false
16835 5509 Acked 6508 Write 5 Done Done New false
16836 5510 Acked 6509 Write 5 Done Done New false
16837 5511 Acked 6510 Read 2 Done Done New false
16838 5512 Acked 6511 Write 1 Done Done New false
16839 5513 Acked 6512 Write 8 Done Done New false
16840 5514 Acked 6513 Read 2 Done Done New false
16841 5515 Acked 6514 Read 9 Done Done New false
16842 5516 Acked 6515 Read 2 Done Done New false
16843 5517 Acked 6516 Read 9 Done Done New false
16844 5518 Acked 6517 Read 3 Done Done New false
16845 5519 Acked 6518 Read 8 Done Done New false
16846 5520 Acked 6519 Read 4 Done Done New false
16847 5521 Acked 6520 Read 9 Done Done New false
16848 5522 Acked 6521 Read 2 Done Done New false
16849 5523 Acked 6522 Read 2 Done Done New false
16850 5524 Acked 6523 Write 2 Done Done New false
16851 5525 Acked 6524 Write 2 Done Done New false
16852 5526 Acked 6525 Write 5 Done Done New false
16853 5527 Acked 6526 Read 6 Done Done New false
16854 5528 Acked 6527 Read 7 Done Done New false
16855 5529 Acked 6528 Read 5 Done Done New false
16856 5530 Acked 6529 Write 2 Done Done New false
16857 5531 Acked 6530 Read 3 Done Done New false
16858 5532 Acked 6531 Read 1 Done Done New false
16859 5533 Acked 6532 Write 5 Done Done New false
16860 5534 Acked 6533 Write 2 Done Done New false
16861 5535 Acked 6534 Read 7 Done Done New false
16862 5536 Acked 6535 Read 9 Done Done New false
16863 5537 Acked 6536 Read 6 Done Done New false
16864 5538 Acked 6537 Write 5 Done Done New false
16865 5539 Acked 6538 Read 1 Done Done New false
16866 5540 Acked 6539 Read 7 Done Done New false
16867 5541 Acked 6540 Write 1 Done Done New false
16868 5542 Acked 6541 Read 10 Done Done New false
16869 5543 Acked 6542 Write 2 Done Done New false
16870 5544 Acked 6543 Write 1 Done Done New false
16871 5545 Acked 6544 Read 5 Done Done New false
16872 5546 Acked 6545 Write 3 Done Done New false
16873 5547 Acked 6546 Write 5 Done Done New false
16874 5548 Acked 6547 Read 10 Done Done New false
16875 5549 Acked 6548 Read 8 Done Done New false
16876 5550 Acked 6549 Read 8 Done Done New false
16877 5551 Acked 6550 Write 4 Done Done New false
16878 5552 Acked 6551 Write 1 Done Done New false
16879 5553 Acked 6552 Read 4 Done Done New false
16880 5554 Acked 6553 Read 9 Done Done New false
16881 5555 Acked 6554 Flush 0 Done Done New false
16882 5556 Acked 6555 Read 2 Done Done New false
16883 5557 Acked 6556 Write 8 Done Done New false
16884 5558 Acked 6557 Write 10 Done Done New false
16885 5559 Acked 6558 Read 4 Done Done New false
16886 5560 Acked 6559 Read 10 Done Done New false
16887 5561 Acked 6560 Read 4 Done Done New false
16888 5562 Acked 6561 Read 6 Done Done New false
16889 5563 Acked 6562 Read 5 Done Done New false
16890 5564 Acked 6563 Write 5 Done Done New false
16891 5565 Acked 6564 Read 6 Done Done New false
16892 5566 Acked 6565 Write 3 Done Done New false
16893 5567 Acked 6566 Write 7 Done Done New false
16894 5568 Acked 6567 Read 4 Done Done New false
16895 5569 Acked 6568 Write 1 Done Done New false
16896 5570 Acked 6569 Read 4 Done Done New false
16897 5571 Acked 6570 Read 10 Done Done New false
16898 5572 Acked 6571 Write 7 Done Done New false
16899 5573 Acked 6572 Write 1 Done Done New false
16900 5574 Acked 6573 Read 3 Done Done New false
16901 5575 Acked 6574 Read 6 Done Done New false
16902 5576 Acked 6575 Read 5 Done Done New false
16903 5577 Acked 6576 Flush 0 Done Done New false
16904 5578 Acked 6577 Read 10 Done Done New false
16905 5579 Acked 6578 Read 8 Done Done New false
16906 5580 Acked 6579 Read 10 Done Done New false
16907 5581 Acked 6580 Read 4 Done Done New false
16908 5582 Acked 6581 Write 8 Done Done New false
16909 5583 Acked 6582 Read 4 Done Done New false
16910 5584 Acked 6583 Read 7 Done Done New false
16911 5585 Acked 6584 Write 9 Done Done New false
16912 5586 Acked 6585 Read 4 Done Done New false
16913 5587 Acked 6586 Flush 0 Done Done New false
16914 5588 Acked 6587 Write 9 Done Done New false
16915 5589 Acked 6588 Write 5 Done Done New false
16916 5590 Acked 6589 Read 2 Done Done New false
16917 5591 Acked 6590 Read 3 Done Done New false
16918 5592 Acked 6591 Read 1 Done Done New false
16919 5593 Acked 6592 Read 7 Done Done New false
16920 5594 Acked 6593 Read 4 Done Done New false
16921 5595 Acked 6594 Write 5 Done Done New false
16922 5596 Acked 6595 Read 10 Done Done New false
16923 5597 Acked 6596 Read 5 Done Done New false
16924 5598 Acked 6597 Write 1 Done Done New false
16925 5599 Acked 6598 Flush 0 Done Done New false
16926 5600 Acked 6599 Write 4 Done Done New false
16927 5601 Acked 6600 Read 8 Done Done New false
16928 5602 Acked 6601 Flush 0 Done Done New false
16929 5603 Acked 6602 Read 4 Done Done New false
16930 5604 Acked 6603 Read 3 Done Done New false
16931 5605 Acked 6604 Read 4 Done Done New false
16932 5606 Acked 6605 Read 5 Done Done New false
16933 5607 Acked 6606 Write 7 Done Done New false
16934 5608 Acked 6607 Read 6 Done Done New false
16935 5609 Acked 6608 Read 10 Done Done New false
16936 5610 Acked 6609 Write 2 Done Done New false
16937 5611 Acked 6610 Write 2 Done Done New false
16938 5612 Acked 6611 Read 4 Done Done New false
16939 5613 Acked 6612 Write 6 Done Done New false
16940 5614 Acked 6613 Write 9 Done Done New false
16941 5615 Acked 6614 Write 7 Done Done New false
16942 5616 Acked 6615 Read 6 Done Done New false
16943 5617 Acked 6616 Write 7 Done Done New false
16944 5618 Acked 6617 Write 1 Done Done New false
16945 5619 Acked 6618 Flush 0 Done Done New false
16946 5620 Acked 6619 Write 4 Done Done New false
16947 5621 Acked 6620 Write 1 Done Done New false
16948 5622 Acked 6621 Read 4 Done Done New false
16949 5623 Acked 6622 Flush 0 Done Done New false
16950 5624 Acked 6623 Write 2 Done Done New false
16951 5625 Acked 6624 Write 7 Done Done New false
16952 5626 Acked 6625 Read 9 Done Done New false
16953 5627 Acked 6626 Read 7 Done Done New false
16954 5628 Acked 6627 Read 2 Done Done New false
16955 5629 Acked 6628 Read 1 Done Done New false
16956 5630 Acked 6629 Read 6 Done Done New false
16957 5631 Acked 6630 Read 2 Done Done New false
16958 5632 Acked 6631 Write 8 Done Done New false
16959 5633 Acked 6632 Write 6 Done Done New false
16960 5634 Acked 6633 Read 3 Done Done New false
16961 5635 Acked 6634 Read 9 Done Done New false
16962 5636 Acked 6635 Read 5 Done Done New false
16963 5637 Acked 6636 Write 5 Done Done New false
16964 5638 Acked 6637 Flush 0 Done Done New false
16965 5639 Acked 6638 Read 1 Done Done New false
16966 5640 Acked 6639 Flush 0 Done Done New false
16967 5641 Acked 6640 Flush 0 Done Done New false
16968 5642 Acked 6641 Write 9 Done Done New false
16969 5643 Acked 6642 Read 2 Done Done New false
16970 5644 Acked 6643 Read 8 Done Done New false
16971 5645 Acked 6644 Write 6 Done Done New false
16972 5646 Acked 6645 Read 9 Done Done New false
16973 5647 Acked 6646 Read 9 Done Done New false
16974 5648 Acked 6647 Write 5 Done Done New false
16975 5649 Acked 6648 Write 1 Done Done New false
16976 5650 Acked 6649 Read 10 Done Done New false
16977 5651 Acked 6650 Read 1 Done Done New false
16978 5652 Acked 6651 Flush 0 Done Done New false
16979 5653 Acked 6652 Flush 0 Done Done New false
16980 5654 Acked 6653 Write 7 Done Done New false
16981 5655 Acked 6654 Read 7 Done Done New false
16982 5656 Acked 6655 Read 5 Done Done New false
16983 5657 Acked 6656 Write 9 Done Done New false
16984 5658 Acked 6657 Read 5 Done Done New false
16985 5659 Acked 6658 Write 3 Done Done New false
16986 5660 Acked 6659 Read 3 Done Done New false
16987 5661 Acked 6660 Flush 0 Done Done New false
16988 5662 Acked 6661 Flush 0 Done Done New false
16989 5663 Acked 6662 Read 2 Done Done New false
16990 5664 Acked 6663 Read 8 Done Done New false
16991 5665 Acked 6664 Read 8 Done Done New false
16992 5666 Acked 6665 Write 10 Done Done New false
16993 5667 Acked 6666 Read 10 Done Done New false
16994 5668 Acked 6667 Write 1 Done Done New false
16995 5669 Acked 6668 Read 2 Done Done New false
16996 5670 Acked 6669 Read 5 Done Done New false
16997 5671 Acked 6670 Read 1 Done Done New false
16998 5672 Acked 6671 Read 2 Done Done New false
16999 5673 Acked 6672 Read 3 Done Done New false
17000 5674 Acked 6673 Read 4 Done Done New false
17001 5675 Acked 6674 Read 2 Done Done New false
17002 5676 Acked 6675 Read 7 Done Done New false
17003 5677 Acked 6676 Read 10 Done Done New false
17004 5678 Acked 6677 Read 9 Done Done New false
17005 5679 Acked 6678 Read 4 Done Done New false
17006 5680 Acked 6679 Write 1 Done Done New false
17007 5681 Acked 6680 Write 10 Done Done New false
17008 5682 Acked 6681 Write 8 Done Done New false
17009 5683 Acked 6682 Read 7 Done Done New false
17010 5684 Acked 6683 Read 3 Done Done New false
17011 5685 Acked 6684 Read 7 Done Done New false
17012 5686 Acked 6685 Flush 0 Done Done New false
17013 5687 Acked 6686 Read 4 Done Done New false
17014 5688 Acked 6687 Read 2 Done Done New false
17015 5689 Acked 6688 Read 6 Done Done New false
17016 5690 Acked 6689 Read 6 Done Done New false
17017 5691 Acked 6690 Read 6 Done Done New false
17018 5692 Acked 6691 Write 5 Done Done New false
17019 5693 Acked 6692 Flush 0 Done Done New false
17020 5694 Acked 6693 Write 9 Done Done New false
17021 5695 Acked 6694 Flush 0 Done Done New false
17022 5696 Acked 6695 Read 10 Done Done New false
17023 5697 Acked 6696 Read 9 Done Done New false
17024 5698 Acked 6697 Read 4 Done Done New false
17025 5699 Acked 6698 Flush 0 Done Done New false
17026 5700 Acked 6699 Read 1 Done Done New false
17027 5701 Acked 6700 Flush 0 Done Done New false
17028 5702 Acked 6701 Read 2 Done Done New false
17029 5703 Acked 6702 Write 4 Done Done New false
17030 5704 Acked 6703 Read 4 Done Done New false
17031 5705 Acked 6704 Read 1 Done Done New false
17032 5706 Acked 6705 Write 9 Done Done New false
17033 5707 Acked 6706 Read 10 Done Done New false
17034 5708 Acked 6707 Read 10 Done Done New false
17035 5709 Acked 6708 Read 5 Done Done New false
17036 5710 Acked 6709 Read 3 Done Done New false
17037 5711 Acked 6710 Read 5 Done Done New false
17038 5712 Acked 6711 Read 5 Done Done New false
17039 5713 Acked 6712 Flush 0 Done Done New false
17040 5714 Acked 6713 Write 9 Done Done New false
17041 5715 Acked 6714 Read 3 Done Done New false
17042 5716 Acked 6715 Read 10 Done Done New false
17043 5717 Acked 6716 Read 3 Done Done New false
17044 5718 Acked 6717 Read 8 Done Done New false
17045 5719 Acked 6718 Read 4 Done Done New false
17046 5720 Acked 6719 Read 7 Done Done New false
17047 5721 Acked 6720 Write 6 Done Done New false
17048 5722 Acked 6721 Read 10 Done Done New false
17049 5723 Acked 6722 Write 7 Done Done New false
17050 5724 Acked 6723 Read 2 Done Done New false
17051 5725 Acked 6724 Write 5 Done Done New false
17052 5726 Acked 6725 Read 2 Done Done New false
17053 5727 Acked 6726 Write 3 Done Done New false
17054 5728 Acked 6727 Read 6 Done Done New false
17055 5729 Acked 6728 Flush 0 Done Done New false
17056 5730 Acked 6729 Write 2 Done Done New false
17057 5731 Acked 6730 Read 10 Done Done New false
17058 5732 Acked 6731 Flush 0 Done Done New false
17059 5733 Acked 6732 Flush 0 Done Done New false
17060 5734 Acked 6733 Flush 0 Done Done New false
17061 5735 Acked 6734 Write 1 Done Done New false
17062 5736 Acked 6735 Write 3 Done Done New false
17063 5737 Acked 6736 Read 6 Done Done New false
17064 5738 Acked 6737 Write 4 Done Done New false
17065 5739 Acked 6738 Read 2 Done Done New false
17066 5740 Acked 6739 Read 3 Done Done New false
17067 5741 Acked 6740 Write 3 Done Done New false
17068 5742 Acked 6741 Write 7 Done Done New false
17069 5743 Acked 6742 Read 6 Done Done New false
17070 5744 Acked 6743 Read 4 Done Done New false
17071 5745 Acked 6744 Write 5 Done Done New false
17072 5746 Acked 6745 Read 5 Done Done New false
17073 STATES DS:0 DS:1 DS:2 TOTAL
17074 New 0 0 301 301
17075 Sent 0 0 0 0
17076 Done 301 301 0 602
17077 Skipped 0 0 0 0
17078 Error 0 0 0 0
17079 Last Flush: 6733 6733 6444
17080 Downstairs last five completed: 6444 6443 6442 6441 6440 6439
17081 Upstairs last five completed: 5746 5745 5744 5743 5742 5741
17082 CLIENT: Up:0 ds:301 act:2
17083 [8830] got start action
17084 [8830] I am going to restart
17085 Starting downstairs at port 8830
17086 Make output file at "/tmp/dsc/downstairs-8830.txt"
17087 [8830][2] reports Starting
17088 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1192)
17089 [8830][2] reports Running
170902023-09-22T23:27:30.404ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
170912023-09-22T23:27:30.404ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
170922023-09-22T23:27:30.404ZINFOcrucible: [2] client re-new 302 jobs since flush 6444 = downstairs
170932023-09-22T23:27:30.405ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
170942023-09-22T23:27:30.405ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
170952023-09-22T23:27:30.405ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
170962023-09-22T23:27:30.406ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
170972023-09-22T23:27:30.406ZINFOcrucible: [2] send last flush ID to this DS: 6444
170982023-09-22T23:27:30.406ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
170992023-09-22T23:27:30.406ZINFOcrucible: [2] Replied this last flush ID: 6444
171002023-09-22T23:27:30.406ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
171012023-09-22T23:27:30.406ZINFOcrucible: [2] Transition from Offline to Replay
171022023-09-22T23:27:30.406ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
171032023-09-22T23:27:30.406ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
171042023-09-22T23:27:30.406ZINFOcrucible: [2] Transition from Replay to Active
171052023-09-22T23:27:30.406ZINFOcrucible: [2] Starts cmd_loop
171062023-09-22T23:27:30.406ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
171072023-09-22T23:27:30.406ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
171082023-09-22T23:27:30.406ZINFOcrucible: Set check for repair
171092023-09-22T23:27:31.406ZWARNcrucible: [2] flow control end
171102023-09-22T23:27:31.407ZINFOcrucible: Checking if live repair is needed
171112023-09-22T23:27:31.407ZINFOcrucible: No Live Repair required at this time
17112 ----------------------------------------------------------------
17113 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17114 Downstairs last five completed: 6746 6745 6744 6743 6742 6741
17115 Upstairs last five completed: 5747 5746 5745 5744 5743 5742
17116 CLIENT: Up:0 ds:0 act:3
17117 Replay: All jobs finished, all DS active.
17118 Main task has work to do, go find it
17119 got dsc Stop(2)
17120 stop 2
17121 Sep 22 23:27:32.313 INFO request completed, latency_us: 526, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 314f5ccf-da38-4ad9-b3bb-c4ea287baadf, remote_addr: 127.0.0.[8830] Got stop action so:false kr:false
17122 1:52471, local_addr: 127.0.0.1:9998
17123 Sep 22 23:27:32.314 INFO request completed, latency_us: 308, response_code: 200, uri: /state/cid/2, method: GET, req_id: d0cabc4e-a169-42c3-8111-891fa9fa878c, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
171242023-09-22T23:27:32.314ZWARNcrucible: [2] downstairs disconnected
171252023-09-22T23:27:32.315ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
171262023-09-22T23:27:32.315ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
171272023-09-22T23:27:32.315ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
171282023-09-22T23:27:32.315ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
171292023-09-22T23:27:32.315ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
171302023-09-22T23:27:32.315ZWARNcrucible: [2] pm_task rx.recv() is None
17131 [8830] Exited with: ExitStatus(unix_wait_status(9))
17132 [8830][2] reports Exit
171332023-09-22T23:27:35.318ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
17134 Sep 22 23:27:36.316 INFO request completed, latency_us: 259, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9b2ac551-9e90-4a09-ae38-f9288a320cef, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
17135 001/300 Write block 3352 len 24576 data: 3 3 2 2 3 2
17136 002/300 Read block 2872 len 16384
17137 003/300 Write block 2351 len 32768 data: 7 7 7 5 4 3 3 3
17138 004/300 Write block 3808 len 16384 data: 6 6 6 5
17139 005/300 Read block 907 len 40960
17140 006/300 Write block 1359 len 8192 data: 4 4
17141 007/300 Read block 4676 len 40960
17142 008/300 Read block 3188 len 24576
17143 009/300 Read block 1127 len 24576
17144 010/300 Write block 1170 len 12288 data: 3 4 6
17145 011/300 Write block 382 len 32768 data: 6 6 7 6 4 4 5 5
17146 012/300 Read block 4600 len 4096
17147 013/300 Read block 3521 len 36864
17148 014/300 Read block 2826 len 36864
17149 015/300 Write block 1840 len 32768 data: 3 2 2 2 2 2 2 2
17150 016/300 Read block 4984 len 36864
17151 017/300 Write block 2357 len 4096 data: 4
17152 018/300 Read block 3305 len 28672
17153 019/300 Write block 1376 len 40960 data: 4 4 4 3 3 3 6 5 5 4
17154 020/300 Read block 2040 len 36864
17155 021/300 Flush
17156 022/300 Flush
17157 023/300 Write block 3001 len 20480 data: 3 3 3 3 3
17158 024/300 Read block 4270 len 16384
17159 025/300 Read block 2700 len 8192
17160 026/300 Read block 4136 len 20480
17161 027/300 Read block 231 len 8192
17162 028/300 Flush
17163 029/300 Flush
17164 030/300 Write block 3089 len 8192 data: 3 3
17165 031/300 Read block 816 len 8192
17166 032/300 Write block 183 len 20480 data: 3 3 3 5 5
17167 033/300 Read block 266 len 20480
17168 034/300 Write block 3263 len 16384 data: 6 4 5 5
17169 035/300 Read block 4724 len 12288
17170 036/300 Read block 3380 len 32768
17171 037/300 Read block 4148 len 24576
17172 038/300 Flush
17173 039/300 Write block 1903 len 36864 data: 7 7 6 6 5 4 3 4 4
17174 040/300 Flush
17175 041/300 Write block 4198 len 32768 data: 5 5 5 5 4 3 4 4
17176 042/300 Read block 2965 len 20480
17177 043/300 Read block 1891 len 28672
17178 044/300 Read block 2604 len 40960
17179 045/300 Read block 3804 len 28672
17180 046/300 Write block 2328 len 8192 data: 6 4
17181 047/300 Read block 1105 len 32768
17182 048/300 Read block 2334 len 36864
17183 049/300 Read block 2984 len 24576
17184 050/300 Read block 2837 len 12288
17185 051/300 Read block 3311 len 12288
17186 052/300 Flush
17187 053/300 Read block 343 len 8192
17188 054/300 Write block 2045 len 16384 data: 3 3 3 3
17189 055/300 Read block 4930 len 8192
17190 056/300 Read block 654 len 28672
17191 057/300 Read block 4299 len 8192
17192 058/300 Read block 2691 len 12288
17193 059/300 Write block 408 len 28672 data: 7 8 8 8 6 6 6
17194 060/300 Write block 4353 len 32768 data: 6 6 6 4 4 3 3 3
17195 061/300 Read block 1131 len 16384
17196 062/300 Write block 3869 len 32768 data: 5 5 5 5 5 4 5 4
17197 063/300 Flush
17198 064/300 Read block 66 len 4096
17199 065/300 Write block 2364 len 24576 data: 5 4 4 3 4 3
17200 066/300 Flush
17201 067/300 Write block 3590 len 40960 data: 6 6 5 4 5 5 4 2 2 2
17202 068/300 Read block 1302 len 20480
17203 069/300 Flush
17204 070/300 Write block 949 len 20480 data: 5 3 3 3 3
17205 071/300 Write block 4604 len 36864 data: 8 6 5 5 3 4 2 2 4
17206 072/300 Write block 1489 len 12288 data: 5 5 5
17207 073/300 Read block 3076 len 20480
17208 074/300 Flush
17209 075/300 Read block 2108 len 8192
17210 076/300 Read block 4826 len 32768
17211 077/300 Read block 2433 len 8192
17212 078/300 Read block 1135 len 24576
17213 079/300 Read block 2145 len 8192
17214 080/300 Write block 3502 len 28672 data: 4 4 4 4 2 3 5
17215 081/300 Read block 4311 len 12288
17216 082/300 Flush
17217 083/300 Write block 4728 len 32768 data: 6 7 5 5 6 6 5 6
17218 084/300 Write block 2076 len 36864 data: 4 3 3 3 2 3 3 3 3
17219 085/300 Write block 624 len 32768 data: 4 5 7 7 6 5 5 5
17220 086/300 Write block 3121 len 20480 data: 5 5 6 7 8
17221 087/300 Read block 1093 len 24576
17222 088/300 Write block 4859 len 32768 data: 5 4 3 2 3 3 3 2
17223 089/300 Read block 4894 len 36864
17224 090/300 Write block 1078 len 20480 data: 5 4 4 4 4
17225 091/300 Write block 4012 len 16384 data: 2 3 4 4
17226 092/300 Read block 139 len 20480
17227 093/300 Read block 2122 len 16384
17228 094/300 Read block 1570 len 32768
17229 095/300 Write block 3079 len 36864 data: 3 3 5 5 5 5 6 6 3
17230 096/300 Write block 2311 len 20480 data: 4 6 5 5 5
17231 097/300 Read block 350 len 4096
17232 098/300 Write block 2830 len 40960 data: 7 6 8 9 6 6 5 4 6 6
17233 099/300 Read block 1729 len 36864
17234 100/300 Write block 2793 len 32768 data: 4 5 5 5 5 6 6 6
17235 101/300 Write block 4887 len 12288 data: 4 4 5
17236 102/300 Write block 4992 len 4096 data: 7
17237 103/300 Read block 3905 len 12288
17238 104/300 Read block 2386 len 4096
17239 105/300 Read block 967 len 28672
17240 106/300 Read block 4581 len 20480
17241 107/300 Flush
17242 108/300 Read block 9 len 32768
17243 109/300 Write block 1369 len 40960 data: 4 3 4 4 4 5 5 5 5 5
17244 110/300 Read block 2634 len 32768
17245 111/300 Write block 3487 len 20480 data: 3 4 4 4 3
17246 112/300 Read block 1262 len 20480
17247 113/300 Flush
17248 114/300 Flush
17249 115/300 Flush
17250 116/300 Read block 4041 len 40960
17251 117/300 Write block 1750 len 40960 data: 4 4 5 6 6 4 4 5 5 5
17252 118/300 Flush
17253 119/300 Write block 3462 len 28672 data: 5 6 5 6 6 6 4
17254 120/300 Read block 807 len 4096
17255 121/300 Flush
17256 122/300 Write block 2508 len 4096 data: 7
17257 123/300 Read block 3282 len 12288
17258 124/300 Write block 1523 len 12288 data: 4 4 3
17259 125/300 Write block 4654 len 12288 data: 4 4 6
17260 126/300 Read block 568 len 12288
17261 127/300 Read block 952 len 20480
17262 128/300 Write block 4572 len 28672 data: 3 3 5 6 7 7 6
17263 129/300 Read block 2322 len 20480
17264 130/300 Read block 263 len 4096
17265 131/300 Write block 2688 len 24576 data: 3 4 4 4 4 5
17266 132/300 Flush
17267 133/300 Write block 2214 len 8192 data: 3 4
17268 134/300 Write block 1428 len 8192 data: 5 7
17269 135/300 Read block 1883 len 36864
17270 136/300 Write block 2132 len 12288 data: 3 4 4
17271 137/300 Read block 130 len 12288
17272 138/300 Read block 1519 len 16384
17273 139/300 Read block 2511 len 4096
17274 140/300 Write block 3499 len 24576 data: 5 5 4 5 5 5
17275 141/300 Write block 4590 len 4096 data: 3
17276 142/300 Read block 2069 len 4096
17277 143/300 Flush
17278 144/300 Write block 3008 len 12288 data: 5 5 3
17279 145/300 Read block 2222 len 40960
17280 146/300 Read block 706 len 16384
17281 147/300 Read block 4432 len 8192
17282 148/300 Write block 3543 len 4096 data: 3
17283 149/300 Read block 3991 len 40960
17284 150/300 Write block 2405 len 20480 data: 5 5 4 3 3
17285 151/300 Write block 1377 len 20480 data: 6 6 4 4 4
17286 152/300 Read block 2103 len 28672
17287 153/300 Read block 2351 len 16384
17288 154/300 Read block 19 len 4096
17289 155/300 Read block 692 len 36864
17290 156/300 Read block 685 len 12288
17291 157/300 Flush
17292 158/300 Flush
17293 159/300 Read block 85 len 40960
17294 160/300 Read block 2882 len 24576
17295 161/300 Read block 4363 len 32768
17296 162/300 Read block 1388 len 12288
17297 163/300 Write block 1423 len 8192 data: 4 3
17298 164/300 Read block 2863 len 20480
17299 165/300 Write block 2888 len 8192 data: 3 4
17300 166/300 Read block 3095 len 40960
17301 167/300 Read block 703 len 12288
17302 168/300 Write block 4916 len 40960 data: 8 8 6 5 3 3 2 2 2 2
17303 169/300 Write block 3538 len 40960 data: 3 3 3 3 3 4 3 2 2 3
17304 170/300 Flush
17305 171/300 Read block 3390 len 32768
17306 172/300 Flush
17307 173/300 Read block 1550 len 20480
17308 174/300 Flush
17309 175/300 Flush
17310 176/300 Write block 3607 len 24576 data: 3 3 3 6 6 6
17311 177/300 Read block 3360 len 4096
17312 178/300 Read block 2082 len 16384
17313 179/300 Read block 648 len 16384
17314 180/300 Write block 4270 len 8192 data: 2 3
17315 181/300 Read block 3465 len 16384
17316 182/300 Read block 931 len 40960
17317 183/300 Read block 1027 len 40960
17318 184/300 Write block 3963 len 36864 data: 5 6 6 5 4 5 6 6 6
17319 185/300 Write block 2711 len 32768 data: 2 2 3 3 4 4 4 5
17320 186/300 Read block 130 len 12288
17321 187/300 Read block 1028 len 16384
17322 188/300 Read block 1376 len 16384
17323 189/300 Read block 3429 len 24576
17324 190/300 Read block 4460 len 20480
17325 191/300 Read block 4735 len 40960
17326 192/300 Write block 1767 len 12288 data: 8 7 7
17327 193/300 Read block 3992 len 8192
17328 194/300 Read block 1885 len 4096
17329 195/300 Read block 1500 len 28672
17330 196/300 Read block 1419 len 20480
17331 197/300 Read block 2371 len 20480
17332 198/300 Write block 2603 len 24576 data: 2 3 4 4 4 4
17333 199/300 Read block 154 len 20480
17334 200/300 Read block 2013 len 28672
17335 201/300 Write block 342 len 8192 data: 5 4
17336 202/300 Read block 4644 len 36864
17337 203/300 Write block 2195 len 32768 data: 4 5 5 7 6 5 5 5
17338 204/300 Write block 1673 len 12288 data: 4 3 2
17339 205/300 Write block 2688 len 20480 data: 4 5 5 5 5
17340 206/300 Flush
17341 207/300 Write block 3387 len 24576 data: 4 4 5 4 3 5
17342 208/300 Write block 2732 len 20480 data: 3 3 3 3 4
17343 209/300 Flush
17344 210/300 Read block 15 len 16384
17345 211/300 Flush
17346 212/300 Read block 4783 len 8192
17347 213/300 Read block 2547 len 12288
17348 214/300 Read block 3378 len 12288
17349 215/300 Read block 3796 len 32768
17350 216/300 Read block 3332 len 12288
17351 217/300 Write block 4049 len 40960 data: 6 5 4 4 4 3 4 5 5 7
17352 218/300 Flush
17353 219/300 Read block 2788 len 20480
17354 220/300 Read block 27 len 16384
17355 221/300 Flush
17356 222/300 Read block 4315 len 16384
17357 223/300 Write block 3826 len 4096 data: 3
17358 224/300 Read block 1202 len 36864
17359 225/300 Read block 3159 len 40960
17360 226/300 Write block 3199 len 8192 data: 3 3
17361 227/300 Write block 892 len 24576 data: 5 4 4 4 4 4
17362 228/300 Read block 2042 len 20480
17363 229/300 Write block 4443 len 4096 data: 3
17364 230/300 Read block 3333 len 8192
17365 231/300 Read block 4581 len 36864
17366 232/300 Write block 3126 len 32768 data: 8 6 6 4 3 3 3 4
17367 233/300 Flush
17368 234/300 Read block 4522 len 32768
17369 235/300 Write block 1602 len 32768 data: 3 3 3 4 3 3 3 4
17370 236/300 Read block 4310 len 16384
17371 237/300 Write block 2491 len 28672 data: 4 4 3 3 3 4 4
17372 238/300 Read block 4833 len 20480
17373 239/300 Read block 1854 len 24576
17374 240/300 Write block 2290 len 28672 data: 4 4 3 3 4 5 5
17375 241/300 Write block 3511 len 28672 data: 7 7 8 9 8 8 5
17376 242/300 Read block 839 len 20480
17377 243/300 Write block 3009 len 32768 data: 6 4 3 5 4 4 3 3
17378 244/300 Read block 4180 len 16384
17379 245/300 Read block 1397 len 4096
17380 246/300 Read block 166 len 20480
17381 247/300 Write block 670 len 28672 data: 5 5 4 4 3 4 4
17382 248/300 Read block 2292 len 32768
17383 249/300 Read block 2466 len 16384
17384 250/300 Flush
17385 251/300 Write block 4306 len 4096 data: 6
17386 252/300 Write block 4791 len 8192 data: 3 3
17387 253/300 Read block 1418 len 8192
17388 254/300 Read block 3580 len 20480
17389 255/300 Read block 6 len 12288
17390 256/300 Write block 3364 len 36864 data: 2 3 5 7 6 5 5 4 4
17391 257/300 Read block 558 len 28672
17392 258/300 Write block 1702 len 32768 data: 4 3 4 4 4 4 6 7
17393 259/300 Flush
17394 260/300 Read block 2801 len 32768
17395 261/300 Read block 3267 len 20480
17396 262/300 Read block 481 len 16384
17397 263/300 Write block 3163 len 20480 data: 4 3 4 4 5
17398 264/300 Read block 3637 len 28672
17399 265/300 Read block 2945 len 28672
17400 266/300 Read block 2193 len 12288
17401 267/300 Read block 1610 len 4096
17402 268/300 Write block 415 len 20480 data: 7 7 7 7 6
17403 269/300 Write block 40 len 12288 data: 7 6 4
17404 270/300 Read block 3160 len 20480
17405 271/300 Write block 1682 len 40960 data: 5 7 7 5 5 5 4 4 4 2
17406 272/300 Read block 1726 len 20480
17407 273/300 Write block 2250 len 36864 data: 4 5 6 5 5 4 4 4 5
17408 274/300 Read block 4120 len 24576
17409 275/300 Read block 4239 len 32768
17410 276/300 Read block 1108 len 40960
17411 277/300 Write block 687 len 32768 data: 4 5 3 3 2 2 4 4
17412 278/300 Read block 3993 len 12288
17413 279/300 Write block 3755 len 28672 data: 4 4 4 4 4 3 4
17414 280/300 Read block 3951 len 32768
17415 281/300 Read block 3318 len 40960
17416 282/300 Flush
17417 283/300 Read block 810 len 20480
17418 284/300 Write block 1120 len 32768 data: 4 4 3 3 3 2 2 3
17419 285/300 Read block 4746 len 24576
17420 286/300 Read block 3364 len 4096
17421 287/300 Read block 2585 len 24576
17422 288/300 Write block 4312 len 8192 data: 5 5
17423 289/300 Read block 1077 len 28672
17424 290/300 Write block 2908 len 40960 data: 5 5 4 4 4 4 4 4 3 4
17425 291/300 Write block 1198 len 4096 data: 3
17426 292/300 Read block 2836 len 32768
17427 293/300 Read block 881 len 40960
17428 294/300 Write block 2898 len 32768 data: 8 7 6 6 5 4 4 5
17429 295/300 Write block 1477 len 24576 data: 8 10 9 8 6 5
17430 296/300 Read block 176 len 28672
17431 297/300 Write block 815 len 32768 data: 7 6 6 5 7 7 6 4
17432 298/300 Write block 3113 len 28672 data: 4 4 4 4 4 6 6
17433 299/300 Read block 4083 len 8192
17434 300/300 Write block 194 len 36864 data: 4 3 3 2 2 3 3 3 3
17435 Main task has work to do, go find it
17436 got dsc Start(2)
17437 start 2
17438 Sep 22 23:27:37.235 INFO request completed, latency_us: 316, response_code: 204, uri: /start/cid/2, method: POST, req_id: f587a771-e8ea-4123-aa88-148cb0e4b1f3, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
17439 Replay: started 2, returned:Ok(())
17440 ----------------------------------------------------------------
17441 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17442 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
17443 5748 Acked 6747 Write 6 Done Done New false
17444 5749 Acked 6748 Read 4 Done Done New false
17445 5750 Acked 6749 Write 8 Done Done New false
17446 5751 Acked 6750 Write 4 Done Done New false
17447 5752 Acked 6751 Read 10 Done Done New false
17448 5753 Acked 6752 Write 2 Done Done New false
17449 5754 Acked 6753 Read 10 Done Done New false
17450 5755 Acked 6754 Read 6 Done Done New false
17451 5756 Acked 6755 Read 6 Done Done New false
17452 5757 Acked 6756 Write 3 Done Done New false
17453 5758 Acked 6757 Write 8 Done Done New false
17454 5759 Acked 6758 Read 1 Done Done New false
17455 5760 Acked 6759 Read 9 Done Done New false
17456 5761 Acked 6760 Read 9 Done Done New false
17457 5762 Acked 6761 Write 8 Done Done New false
17458 5763 Acked 6762 Read 9 Done Done New false
17459 5764 Acked 6763 Write 1 Done Done New false
17460 5765 Acked 6764 Read 7 Done Done New false
17461 5766 Acked 6765 Write 10 Done Done New false
17462 5767 Acked 6766 Read 9 Done Done New false
17463 5768 Acked 6767 Flush 0 Done Done New false
17464 5769 Acked 6768 Flush 0 Done Done New false
17465 5770 Acked 6769 Write 5 Done Done New false
17466 5771 Acked 6770 Read 4 Done Done New false
17467 5772 Acked 6771 Read 2 Done Done New false
17468 5773 Acked 6772 Read 5 Done Done New false
17469 5774 Acked 6773 Read 2 Done Done New false
17470 5775 Acked 6774 Flush 0 Done Done New false
17471 5776 Acked 6775 Flush 0 Done Done New false
17472 5777 Acked 6776 Write 2 Done Done New false
17473 5778 Acked 6777 Read 2 Done Done New false
17474 5779 Acked 6778 Write 5 Done Done New false
17475 5780 Acked 6779 Read 5 Done Done New false
17476 5781 Acked 6780 Write 4 Done Done New false
17477 5782 Acked 6781 Read 3 Done Done New false
17478 5783 Acked 6782 Read 8 Done Done New false
17479 5784 Acked 6783 Read 6 Done Done New false
17480 5785 Acked 6784 Flush 0 Done Done New false
17481 5786 Acked 6785 Write 9 Done Done New false
17482 5787 Acked 6786 Flush 0 Done Done New false
17483 5788 Acked 6787 Write 8 Done Done New false
17484 5789 Acked 6788 Read 5 Done Done New false
17485 5790 Acked 6789 Read 7 Done Done New false
17486 5791 Acked 6790 Read 10 Done Done New false
17487 5792 Acked 6791 Read 7 Done Done New false
17488 5793 Acked 6792 Write 2 Done Done New false
17489 5794 Acked 6793 Flush 0 Done Done New false
17490 5795 Acked 6794 Read 8 Done Done New false
17491 5796 Acked 6795 Read 9 Done Done New false
17492 5797 Acked 6796 Read 6 Done Done New false
17493 5798 Acked 6797 Read 3 Done Done New false
17494 5799 Acked 6798 Read 3 Done Done New false
17495 5800 Acked 6799 Flush 0 Done Done New false
17496 5801 Acked 6800 Read 2 Done Done New false
17497 5802 Acked 6801 Write 4 Done Done New false
17498 5803 Acked 6802 Read 2 Done Done New false
17499 5804 Acked 6803 Read 7 Done Done New false
17500 5805 Acked 6804 Read 2 Done Done New false
17501 5806 Acked 6805 Read 3 Done Done New false
17502 5807 Acked 6806 Write 7 Done Done New false
17503 5808 Acked 6807 Write 8 Done Done New false
17504 5809 Acked 6808 Read 4 Done Done New false
17505 5810 Acked 6809 Write 8 Done Done New false
17506 5811 Acked 6810 Flush 0 Done Done New false
17507 5812 Acked 6811 Read 1 Done Done New false
17508 5813 Acked 6812 Write 6 Done Done New false
17509 5814 Acked 6813 Flush 0 Done Done New false
17510 5815 Acked 6814 Write 10 Done Done New false
17511 5816 Acked 6815 Read 5 Done Done New false
17512 5817 Acked 6816 Flush 0 Done Done New false
17513 5818 Acked 6817 Write 5 Done Done New false
17514 5819 Acked 6818 Write 9 Done Done New false
17515 5820 Acked 6819 Write 3 Done Done New false
17516 5821 Acked 6820 Read 5 Done Done New false
17517 5822 Acked 6821 Flush 0 Done Done New false
17518 5823 Acked 6822 Read 2 Done Done New false
17519 5824 Acked 6823 Read 8 Done Done New false
17520 5825 Acked 6824 Read 2 Done Done New false
17521 5826 Acked 6825 Read 6 Done Done New false
17522 5827 Acked 6826 Read 2 Done Done New false
17523 5828 Acked 6827 Write 7 Done Done New false
17524 5829 Acked 6828 Read 3 Done Done New false
17525 5830 Acked 6829 Flush 0 Done Done New false
17526 5831 Acked 6830 Write 8 Done Done New false
17527 5832 Acked 6831 Write 9 Done Done New false
17528 5833 Acked 6832 Write 8 Done Done New false
17529 5834 Acked 6833 Write 5 Done Done New false
17530 5835 Acked 6834 Read 6 Done Done New false
17531 5836 Acked 6835 Write 8 Done Done New false
17532 5837 Acked 6836 Read 9 Done Done New false
17533 5838 Acked 6837 Write 5 Done Done New false
17534 5839 Acked 6838 Write 4 Done Done New false
17535 5840 Acked 6839 Read 5 Done Done New false
17536 5841 Acked 6840 Read 4 Done Done New false
17537 5842 Acked 6841 Read 8 Done Done New false
17538 5843 Acked 6842 Write 9 Done Done New false
17539 5844 Acked 6843 Write 5 Done Done New false
17540 5845 Acked 6844 Read 1 Done Done New false
17541 5846 Acked 6845 Write 10 Done Done New false
17542 5847 Acked 6846 Read 9 Done Done New false
17543 5848 Acked 6847 Write 8 Done Done New false
17544 5849 Acked 6848 Write 3 Done Done New false
17545 5850 Acked 6849 Write 1 Done Done New false
17546 5851 Acked 6850 Read 3 Done Done New false
17547 5852 Acked 6851 Read 1 Done Done New false
17548 5853 Acked 6852 Read 7 Done Done New false
17549 5854 Acked 6853 Read 5 Done Done New false
17550 5855 Acked 6854 Flush 0 Done Done New false
17551 5856 Acked 6855 Read 8 Done Done New false
17552 5857 Acked 6856 Write 10 Done Done New false
17553 5858 Acked 6857 Read 8 Done Done New false
17554 5859 Acked 6858 Write 5 Done Done New false
17555 5860 Acked 6859 Read 5 Done Done New false
17556 5861 Acked 6860 Flush 0 Done Done New false
17557 5862 Acked 6861 Flush 0 Done Done New false
17558 5863 Acked 6862 Flush 0 Done Done New false
17559 5864 Acked 6863 Read 10 Done Done New false
17560 5865 Acked 6864 Write 10 Done Done New false
17561 5866 Acked 6865 Flush 0 Done Done New false
17562 5867 Acked 6866 Write 7 Done Done New false
17563 5868 Acked 6867 Read 1 Done Done New false
17564 5869 Acked 6868 Flush 0 Done Done New false
17565 5870 Acked 6869 Write 1 Done Done New false
17566 5871 Acked 6870 Read 3 Done Done New false
17567 5872 Acked 6871 Write 3 Done Done New false
17568 5873 Acked 6872 Write 3 Done Done New false
17569 5874 Acked 6873 Read 3 Done Done New false
17570 5875 Acked 6874 Read 5 Done Done New false
17571 5876 Acked 6875 Write 7 Done Done New false
17572 5877 Acked 6876 Read 5 Done Done New false
17573 5878 Acked 6877 Read 1 Done Done New false
17574 5879 Acked 6878 Write 6 Done Done New false
17575 5880 Acked 6879 Flush 0 Done Done New false
17576 5881 Acked 6880 Write 2 Done Done New false
17577 5882 Acked 6881 Write 2 Done Done New false
17578 5883 Acked 6882 Read 9 Done Done New false
17579 5884 Acked 6883 Write 3 Done Done New false
17580 5885 Acked 6884 Read 3 Done Done New false
17581 5886 Acked 6885 Read 4 Done Done New false
17582 5887 Acked 6886 Read 1 Done Done New false
17583 5888 Acked 6887 Write 6 Done Done New false
17584 5889 Acked 6888 Write 1 Done Done New false
17585 5890 Acked 6889 Read 1 Done Done New false
17586 5891 Acked 6890 Flush 0 Done Done New false
17587 5892 Acked 6891 Write 3 Done Done New false
17588 5893 Acked 6892 Read 10 Done Done New false
17589 5894 Acked 6893 Read 4 Done Done New false
17590 5895 Acked 6894 Read 2 Done Done New false
17591 5896 Acked 6895 Write 1 Done Done New false
17592 5897 Acked 6896 Read 10 Done Done New false
17593 5898 Acked 6897 Write 5 Done Done New false
17594 5899 Acked 6898 Write 5 Done Done New false
17595 5900 Acked 6899 Read 7 Done Done New false
17596 5901 Acked 6900 Read 4 Done Done New false
17597 5902 Acked 6901 Read 1 Done Done New false
17598 5903 Acked 6902 Read 9 Done Done New false
17599 5904 Acked 6903 Read 3 Done Done New false
17600 5905 Acked 6904 Flush 0 Done Done New false
17601 5906 Acked 6905 Flush 0 Done Done New false
17602 5907 Acked 6906 Read 10 Done Done New false
17603 5908 Acked 6907 Read 6 Done Done New false
17604 5909 Acked 6908 Read 8 Done Done New false
17605 5910 Acked 6909 Read 3 Done Done New false
17606 5911 Acked 6910 Write 2 Done Done New false
17607 5912 Acked 6911 Read 5 Done Done New false
17608 5913 Acked 6912 Write 2 Done Done New false
17609 5914 Acked 6913 Read 10 Done Done New false
17610 5915 Acked 6914 Read 3 Done Done New false
17611 5916 Acked 6915 Write 10 Done Done New false
17612 5917 Acked 6916 Write 10 Done Done New false
17613 5918 Acked 6917 Flush 0 Done Done New false
17614 5919 Acked 6918 Read 8 Done Done New false
17615 5920 Acked 6919 Flush 0 Done Done New false
17616 5921 Acked 6920 Read 5 Done Done New false
17617 5922 Acked 6921 Flush 0 Done Done New false
17618 5923 Acked 6922 Flush 0 Done Done New false
17619 5924 Acked 6923 Write 6 Done Done New false
17620 5925 Acked 6924 Read 1 Done Done New false
17621 5926 Acked 6925 Read 4 Done Done New false
17622 5927 Acked 6926 Read 4 Done Done New false
17623 5928 Acked 6927 Write 2 Done Done New false
17624 5929 Acked 6928 Read 4 Done Done New false
17625 5930 Acked 6929 Read 10 Done Done New false
17626 5931 Acked 6930 Read 10 Done Done New false
17627 5932 Acked 6931 Write 9 Done Done New false
17628 5933 Acked 6932 Write 8 Done Done New false
17629 5934 Acked 6933 Read 3 Done Done New false
17630 5935 Acked 6934 Read 4 Done Done New false
17631 5936 Acked 6935 Read 4 Done Done New false
17632 5937 Acked 6936 Read 6 Done Done New false
17633 5938 Acked 6937 Read 5 Done Done New false
17634 5939 Acked 6938 Read 10 Done Done New false
17635 5940 Acked 6939 Write 3 Done Done New false
17636 5941 Acked 6940 Read 2 Done Done New false
17637 5942 Acked 6941 Read 1 Done Done New false
17638 5943 Acked 6942 Read 7 Done Done New false
17639 5944 Acked 6943 Read 5 Done Done New false
17640 5945 Acked 6944 Read 5 Done Done New false
17641 5946 Acked 6945 Write 6 Done Done New false
17642 5947 Acked 6946 Read 5 Done Done New false
17643 5948 Acked 6947 Read 7 Done Done New false
17644 5949 Acked 6948 Write 2 Done Done New false
17645 5950 Acked 6949 Read 9 Done Done New false
17646 5951 Acked 6950 Write 8 Done Done New false
17647 5952 Acked 6951 Write 3 Done Done New false
17648 5953 Acked 6952 Write 5 Done Done New false
17649 5954 Acked 6953 Flush 0 Done Done New false
17650 5955 Acked 6954 Write 6 Done Done New false
17651 5956 Acked 6955 Write 5 Done Done New false
17652 5957 Acked 6956 Flush 0 Done Done New false
17653 5958 Acked 6957 Read 4 Done Done New false
17654 5959 Acked 6958 Flush 0 Done Done New false
17655 5960 Acked 6959 Read 2 Done Done New false
17656 5961 Acked 6960 Read 3 Done Done New false
17657 5962 Acked 6961 Read 3 Done Done New false
17658 5963 Acked 6962 Read 8 Done Done New false
17659 5964 Acked 6963 Read 3 Done Done New false
17660 5965 Acked 6964 Flush 0 Done Done New false
17661 5966 Acked 6965 Write 10 Done Done New false
17662 5967 Acked 6966 Flush 0 Done Done New false
17663 5968 Acked 6967 Read 5 Done Done New false
17664 5969 Acked 6968 Read 4 Done Done New false
17665 5970 Acked 6969 Flush 0 Done Done New false
17666 5971 Acked 6970 Read 4 Done Done New false
17667 5972 Acked 6971 Write 1 Done Done New false
17668 5973 Acked 6972 Read 9 Done Done New false
17669 5974 Acked 6973 Read 10 Done Done New false
17670 5975 Acked 6974 Write 2 Done Done New false
17671 5976 Acked 6975 Write 6 Done Done New false
17672 5977 Acked 6976 Read 5 Done Done New false
17673 5978 Acked 6977 Write 1 Done Done New false
17674 5979 Acked 6978 Read 2 Done Done New false
17675 5980 Acked 6979 Read 9 Done Done New false
17676 5981 Acked 6980 Write 8 Done Done New false
17677 5982 Acked 6981 Flush 0 Done Done New false
17678 5983 Acked 6982 Read 8 Done Done New false
17679 5984 Acked 6983 Write 8 Done Done New false
17680 5985 Acked 6984 Read 4 Done Done New false
17681 5986 Acked 6985 Write 7 Done Done New false
17682 5987 Acked 6986 Read 5 Done Done New false
17683 5988 Acked 6987 Read 6 Done Done New false
17684 5989 Acked 6988 Write 7 Done Done New false
17685 5990 Acked 6989 Write 7 Done Done New false
17686 5991 Acked 6990 Read 5 Done Done New false
17687 5992 Acked 6991 Write 8 Done Done New false
17688 5993 Acked 6992 Read 4 Done Done New false
17689 5994 Acked 6993 Read 1 Done Done New false
17690 5995 Acked 6994 Read 5 Done Done New false
17691 5996 Acked 6995 Write 7 Done Done New false
17692 5997 Acked 6996 Read 8 Done Done New false
17693 5998 Acked 6997 Read 4 Done Done New false
17694 5999 Acked 6998 Flush 0 Done Done New false
17695 6000 Acked 6999 Write 1 Done Done New false
17696 6001 Acked 7000 Write 2 Done Done New false
17697 6002 Acked 7001 Read 2 Done Done New false
17698 6003 Acked 7002 Read 5 Done Done New false
17699 6004 Acked 7003 Read 3 Done Done New false
17700 6005 Acked 7004 Write 9 Done Done New false
17701 6006 Acked 7005 Read 7 Done Done New false
17702 6007 Acked 7006 Write 8 Done Done New false
17703 6008 Acked 7007 Flush 0 Done Done New false
17704 6009 Acked 7008 Read 8 Done Done New false
17705 6010 Acked 7009 Read 5 Done Done New false
17706 6011 Acked 7010 Read 4 Done Done New false
17707 6012 Acked 7011 Write 5 Done Done New false
17708 6013 Acked 7012 Read 7 Done Done New false
17709 6014 Acked 7013 Read 7 Done Done New false
17710 6015 Acked 7014 Read 3 Done Done New false
17711 6016 Acked 7015 Read 1 Done Done New false
17712 6017 Acked 7016 Write 5 Done Done New false
17713 6018 Acked 7017 Write 3 Done Done New false
17714 6019 Acked 7018 Read 5 Done Done New false
17715 6020 Acked 7019 Write 10 Done Done New false
17716 6021 Acked 7020 Read 5 Done Done New false
17717 6022 Acked 7021 Write 9 Done Done New false
17718 6023 Acked 7022 Read 6 Done Done New false
17719 6024 Acked 7023 Read 8 Done Done New false
17720 6025 Acked 7024 Read 10 Done Done New false
17721 6026 Acked 7025 Write 8 Done Done New false
17722 6027 Acked 7026 Read 3 Done Done New false
17723 6028 Acked 7027 Write 7 Done Done New false
17724 6029 Acked 7028 Read 8 Done Done New false
17725 6030 Acked 7029 Read 10 Done Done New false
17726 6031 Acked 7030 Flush 0 Done Done New false
17727 6032 Acked 7031 Read 5 Done Done New false
17728 6033 Acked 7032 Write 8 Done Done New false
17729 6034 Acked 7033 Read 6 Done Done New false
17730 6035 Acked 7034 Read 1 Done Done New false
17731 6036 Acked 7035 Read 6 Done Done New false
17732 6037 Acked 7036 Write 2 Done Done New false
17733 6038 Acked 7037 Read 7 Done Done New false
17734 6039 Acked 7038 Write 10 Done Done New false
17735 6040 Acked 7039 Write 1 Done Done New false
17736 6041 Acked 7040 Read 8 Done Done New false
17737 6042 Acked 7041 Read 10 Done Done New false
17738 6043 Acked 7042 Write 8 Done Done New false
17739 6044 Acked 7043 Write 6 Done Done New false
17740 6045 Acked 7044 Read 7 Done Done New false
17741 6046 Acked 7045 Write 8 Done Done New false
17742 6047 Acked 7046 Write 7 Done Done New false
17743 6048 Acked 7047 Read 2 Done Done New false
17744 6049 Acked 7048 Write 9 Sent Sent New false
17745 STATES DS:0 DS:1 DS:2 TOTAL
17746 New 0 0 302 302
17747 Sent 1 1 0 2
17748 Done 301 301 0 602
17749 Skipped 0 0 0 0
17750 Error 0 0 0 0
17751 Last Flush: 7030 7030 6746
17752 Downstairs last five completed: 6746 6745 6744 6743 6742 6741
17753 Upstairs last five completed: 6049 6048 6047 6046 6045 6044
17754 CLIENT: Up:0 ds:302 act:2
17755 ----------------------------------------------------------------
17756 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
17757 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
17758 5748 Acked 6747 Write 6 Done Done New false
17759 5749 Acked 6748 Read 4 Done Done New false
17760 5750 Acked 6749 Write 8 Done Done New false
17761 5751 Acked 6750 Write 4 Done Done New false
17762 5752 Acked 6751 Read 10 Done Done New false
17763 5753 Acked 6752 Write 2 Done Done New false
17764 5754 Acked 6753 Read 10 Done Done New false
17765 5755 Acked 6754 Read 6 Done Done New false
17766 5756 Acked 6755 Read 6 Done Done New false
17767 5757 Acked 6756 Write 3 Done Done New false
17768 5758 Acked 6757 Write 8 Done Done New false
17769 5759 Acked 6758 Read 1 Done Done New false
17770 5760 Acked 6759 Read 9 Done Done New false
17771 5761 Acked 6760 Read 9 Done Done New false
17772 5762 Acked 6761 Write 8 Done Done New false
17773 5763 Acked 6762 Read 9 Done Done New false
17774 5764 Acked 6763 Write 1 Done Done New false
17775 5765 Acked 6764 Read 7 Done Done New false
17776 5766 Acked 6765 Write 10 Done Done New false
17777 5767 Acked 6766 Read 9 Done Done New false
17778 5768 Acked 6767 Flush 0 Done Done New false
17779 5769 Acked 6768 Flush 0 Done Done New false
17780 5770 Acked 6769 Write 5 Done Done New false
17781 5771 Acked 6770 Read 4 Done Done New false
17782 5772 Acked 6771 Read 2 Done Done New false
17783 5773 Acked 6772 Read 5 Done Done New false
17784 5774 Acked 6773 Read 2 Done Done New false
17785 5775 Acked 6774 Flush 0 Done Done New false
17786 5776 Acked 6775 Flush 0 Done Done New false
17787 5777 Acked 6776 Write 2 Done Done New false
17788 5778 Acked 6777 Read 2 Done Done New false
17789 5779 Acked 6778 Write 5 Done Done New false
17790 5780 Acked 6779 Read 5 Done Done New false
17791 5781 Acked 6780 Write 4 Done Done New false
17792 5782 Acked 6781 Read 3 Done Done New false
17793 5783 Acked 6782 Read 8 Done Done New false
17794 5784 Acked 6783 Read 6 Done Done New false
17795 5785 Acked 6784 Flush 0 Done Done New false
17796 5786 Acked 6785 Write 9 Done Done New false
17797 5787 Acked 6786 Flush 0 Done Done New false
17798 5788 Acked 6787 Write 8 Done Done New false
17799 5789 Acked 6788 Read 5 Done Done New false
17800 5790 Acked 6789 Read 7 Done Done New false
17801 5791 Acked 6790 Read 10 Done Done New false
17802 5792 Acked 6791 Read 7 Done Done New false
17803 5793 Acked 6792 Write 2 Done Done New false
17804 5794 Acked 6793 Flush 0 Done Done New false
17805 5795 Acked 6794 Read 8 Done Done New false
17806 5796 Acked 6795 Read 9 Done Done New false
17807 5797 Acked 6796 Read 6 Done Done New false
17808 5798 Acked 6797 Read 3 Done Done New false
17809 5799 Acked 6798 Read 3 Done Done New false
17810 5800 Acked 6799 Flush 0 Done Done New false
17811 5801 Acked 6800 Read 2 Done Done New false
17812 5802 Acked 6801 Write 4 Done Done New false
17813 5803 Acked 6802 Read 2 Done Done New false
17814 5804 Acked 6803 Read 7 Done Done New false
17815 5805 Acked 6804 Read 2 Done Done New false
17816 5806 Acked 6805 Read 3 Done Done New false
17817 5807 Acked 6806 Write 7 Done Done New false
17818 5808 Acked 6807 Write 8 Done Done New false
17819 5809 Acked 6808 Read 4 Done Done New false
17820 5810 Acked 6809 Write 8 Done Done New false
17821 5811 Acked 6810 Flush 0 Done Done New false
17822 5812 Acked 6811 Read 1 Done Done New false
17823 5813 Acked 6812 Write 6 Done Done New false
17824 5814 Acked 6813 Flush 0 Done Done New false
17825 5815 Acked 6814 Write 10 Done Done New false
17826 5816 Acked 6815 Read 5 Done Done New false
17827 5817 Acked 6816 Flush 0 Done Done New false
17828 5818 Acked 6817 Write 5 Done Done New false
17829 5819 Acked 6818 Write 9 Done Done New false
17830 5820 Acked 6819 Write 3 Done Done New false
17831 5821 Acked 6820 Read 5 Done Done New false
17832 5822 Acked 6821 Flush 0 Done Done New false
17833 5823 Acked 6822 Read 2 Done Done New false
17834 5824 Acked 6823 Read 8 Done Done New false
17835 5825 Acked 6824 Read 2 Done Done New false
17836 5826 Acked 6825 Read 6 Done Done New false
17837 5827 Acked 6826 Read 2 Done Done New false
17838 5828 Acked 6827 Write 7 Done Done New false
17839 5829 Acked 6828 Read 3 Done Done New false
17840 5830 Acked 6829 Flush 0 Done Done New false
17841 5831 Acked 6830 Write 8 Done Done New false
17842 5832 Acked 6831 Write 9 Done Done New false
17843 5833 Acked 6832 Write 8 Done Done New false
17844 5834 Acked 6833 Write 5 Done Done New false
17845 5835 Acked 6834 Read 6 Done Done New false
17846 5836 Acked 6835 Write 8 Done Done New false
17847 5837 Acked 6836 Read 9 Done Done New false
17848 5838 Acked 6837 Write 5 Done Done New false
17849 5839 Acked 6838 Write 4 Done Done New false
17850 5840 Acked 6839 Read 5 Done Done New false
17851 5841 Acked 6840 Read 4 Done Done New false
17852 5842 Acked 6841 Read 8 Done Done New false
17853 5843 Acked 6842 Write 9 Done Done New false
17854 5844 Acked 6843 Write 5 Done Done New false
17855 5845 Acked 6844 Read 1 Done Done New false
17856 5846 Acked 6845 Write 10 Done Done New false
17857 5847 Acked 6846 Read 9 Done Done New false
17858 5848 Acked 6847 Write 8 Done Done New false
17859 5849 Acked 6848 Write 3 Done Done New false
17860 5850 Acked 6849 Write 1 Done Done New false
17861 5851 Acked 6850 Read 3 Done Done New false
17862 5852 Acked 6851 Read 1 Done Done New false
17863 5853 Acked 6852 Read 7 Done Done New false
17864 5854 Acked 6853 Read 5 Done Done New false
17865 5855 Acked 6854 Flush 0 Done Done New false
17866 5856 Acked 6855 Read 8 Done Done New false
17867 5857 Acked 6856 Write 10 Done Done New false
17868 5858 Acked 6857 Read 8 Done Done New false
17869 5859 Acked 6858 Write 5 Done Done New false
17870 5860 Acked 6859 Read 5 Done Done New false
17871 5861 Acked 6860 Flush 0 Done Done New false
17872 5862 Acked 6861 Flush 0 Done Done New false
17873 5863 Acked 6862 Flush 0 Done Done New false
17874 5864 Acked 6863 Read 10 Done Done New false
17875 5865 Acked 6864 Write 10 Done Done New false
17876 5866 Acked 6865 Flush 0 Done Done New false
17877 5867 Acked 6866 Write 7 Done Done New false
17878 5868 Acked 6867 Read 1 Done Done New false
17879 5869 Acked 6868 Flush 0 Done Done New false
17880 5870 Acked 6869 Write 1 Done Done New false
17881 5871 Acked 6870 Read 3 Done Done New false
17882 5872 Acked 6871 Write 3 Done Done New false
17883 5873 Acked 6872 Write 3 Done Done New false
17884 5874 Acked 6873 Read 3 Done Done New false
17885 5875 Acked 6874 Read 5 Done Done New false
17886 5876 Acked 6875 Write 7 Done Done New false
17887 5877 Acked 6876 Read 5 Done Done New false
17888 5878 Acked 6877 Read 1 Done Done New false
17889 5879 Acked 6878 Write 6 Done Done New false
17890 5880 Acked 6879 Flush 0 Done Done New false
17891 5881 Acked 6880 Write 2 Done Done New false
17892 5882 Acked 6881 Write 2 Done Done New false
17893 5883 Acked 6882 Read 9 Done Done New false
17894 5884 Acked 6883 Write 3 Done Done New false
17895 5885 Acked 6884 Read 3 Done Done New false
17896 5886 Acked 6885 Read 4 Done Done New false
17897 5887 Acked 6886 Read 1 Done Done New false
17898 5888 Acked 6887 Write 6 Done Done New false
17899 5889 Acked 6888 Write 1 Done Done New false
17900 5890 Acked 6889 Read 1 Done Done New false
17901 5891 Acked 6890 Flush 0 Done Done New false
17902 5892 Acked 6891 Write 3 Done Done New false
17903 5893 Acked 6892 Read 10 Done Done New false
17904 5894 Acked 6893 Read 4 Done Done New false
17905 5895 Acked 6894 Read 2 Done Done New false
17906 5896 Acked 6895 Write 1 Done Done New false
17907 5897 Acked 6896 Read 10 Done Done New false
17908 5898 Acked 6897 Write 5 Done Done New false
17909 5899 Acked 6898 Write 5 Done Done New false
17910 5900 Acked 6899 Read 7 Done Done New false
17911 5901 Acked 6900 Read 4 Done Done New false
17912 5902 Acked 6901 Read 1 Done Done New false
17913 5903 Acked 6902 Read 9 Done Done New false
17914 5904 Acked 6903 Read 3 Done Done New false
17915 5905 Acked 6904 Flush 0 Done Done New false
17916 5906 Acked 6905 Flush 0 Done Done New false
17917 5907 Acked 6906 Read 10 Done Done New false
17918 5908 Acked 6907 Read 6 Done Done New false
17919 5909 Acked 6908 Read 8 Done Done New false
17920 5910 Acked 6909 Read 3 Done Done New false
17921 5911 Acked 6910 Write 2 Done Done New false
17922 5912 Acked 6911 Read 5 Done Done New false
17923 5913 Acked 6912 Write 2 Done Done New false
17924 5914 Acked 6913 Read 10 Done Done New false
17925 5915 Acked 6914 Read 3 Done Done New false
17926 5916 Acked 6915 Write 10 Done Done New false
17927 5917 Acked 6916 Write 10 Done Done New false
17928 5918 Acked 6917 Flush 0 Done Done New false
17929 5919 Acked 6918 Read 8 Done Done New false
17930 5920 Acked 6919 Flush 0 Done Done New false
17931 5921 Acked 6920 Read 5 Done Done New false
17932 5922 Acked 6921 Flush 0 Done Done New false
17933 5923 Acked 6922 Flush 0 Done Done New false
17934 5924 Acked 6923 Write 6 Done Done New false
17935 5925 Acked 6924 Read 1 Done Done New false
17936 5926 Acked 6925 Read 4 Done Done New false
17937 5927 Acked 6926 Read 4 Done Done New false
17938 5928 Acked 6927 Write 2 Done Done New false
17939 5929 Acked 6928 Read 4 Done Done New false
17940 5930 Acked 6929 Read 10 Done Done New false
17941 5931 Acked 6930 Read 10 Done Done New false
17942 5932 Acked 6931 Write 9 Done Done New false
17943 5933 Acked 6932 Write 8 Done Done New false
17944 5934 Acked 6933 Read 3 Done Done New false
17945 5935 Acked 6934 Read 4 Done Done New false
17946 5936 Acked 6935 Read 4 Done Done New false
17947 5937 Acked 6936 Read 6 Done Done New false
17948 5938 Acked 6937 Read 5 Done Done New false
17949 5939 Acked 6938 Read 10 Done Done New false
17950 5940 Acked 6939 Write 3 Done Done New false
17951 5941 Acked 6940 Read 2 Done Done New false
17952 5942 Acked 6941 Read 1 Done Done New false
17953 5943 Acked 6942 Read 7 Done Done New false
17954 5944 Acked 6943 Read 5 Done Done New false
17955 5945 Acked 6944 Read 5 Done Done New false
17956 5946 Acked 6945 Write 6 Done Done New false
17957 5947 Acked 6946 Read 5 Done Done New false
17958 5948 Acked 6947 Read 7 Done Done New false
17959 5949 Acked 6948 Write 2 Done Done New false
17960 5950 Acked 6949 Read 9 Done Done New false
17961 5951 Acked 6950 Write 8 Done Done New false
17962 5952 Acked 6951 Write 3 Done Done New false
17963 5953 Acked 6952 Write 5 Done Done New false
17964 5954 Acked 6953 Flush 0 Done Done New false
17965 5955 Acked 6954 Write 6 Done Done New false
17966 5956 Acked 6955 Write 5 Done Done New false
17967 5957 Acked 6956 Flush 0 Done Done New false
17968 5958 Acked 6957 Read 4 Done Done New false
17969 5959 Acked 6958 Flush 0 Done Done New false
17970 5960 Acked 6959 Read 2 Done Done New false
17971 5961 Acked 6960 Read 3 Done Done New false
17972 5962 Acked 6961 Read 3 Done Done New false
17973 5963 Acked 6962 Read 8 Done Done New false
17974 5964 Acked 6963 Read 3 Done Done New false
17975 5965 Acked 6964 Flush 0 Done Done New false
17976 5966 Acked 6965 Write 10 Done Done New false
17977 5967 Acked 6966 Flush 0 Done Done New false
17978 5968 Acked 6967 Read 5 Done Done New false
17979 5969 Acked 6968 Read 4 Done Done New false
17980 5970 Acked 6969 Flush 0 Done Done New false
17981 5971 Acked 6970 Read 4 Done Done New false
17982 5972 Acked 6971 Write 1 Done Done New false
17983 5973 Acked 6972 Read 9 Done Done New false
17984 5974 Acked 6973 Read 10 Done Done New false
17985 5975 Acked 6974 Write 2 Done Done New false
17986 5976 Acked 6975 Write 6 Done Done New false
17987 5977 Acked 6976 Read 5 Done Done New false
17988 5978 Acked 6977 Write 1 Done Done New false
17989 5979 Acked 6978 Read 2 Done Done New false
17990 5980 Acked 6979 Read 9 Done Done New false
17991 5981 Acked 6980 Write 8 Done Done New false
17992 5982 Acked 6981 Flush 0 Done Done New false
17993 5983 Acked 6982 Read 8 Done Done New false
17994 5984 Acked 6983 Write 8 Done Done New false
17995 5985 Acked 6984 Read 4 Done Done New false
17996 5986 Acked 6985 Write 7 Done Done New false
17997 5987 Acked 6986 Read 5 Done Done New false
17998 5988 Acked 6987 Read 6 Done Done New false
17999 5989 Acked 6988 Write 7 Done Done New false
18000 5990 Acked 6989 Write 7 Done Done New false
18001 5991 Acked 6990 Read 5 Done Done New false
18002 5992 Acked 6991 Write 8 Done Done New false
18003 5993 Acked 6992 Read 4 Done Done New false
18004 5994 Acked 6993 Read 1 Done Done New false
18005 5995 Acked 6994 Read 5 Done Done New false
18006 5996 Acked 6995 Write 7 Done Done New false
18007 5997 Acked 6996 Read 8 Done Done New false
18008 5998 Acked 6997 Read 4 Done Done New false
18009 5999 Acked 6998 Flush 0 Done Done New false
18010 6000 Acked 6999 Write 1 Done Done New false
18011 6001 Acked 7000 Write 2 Done Done New false
18012 6002 Acked 7001 Read 2 Done Done New false
18013 6003 Acked 7002 Read 5 Done Done New false
18014 6004 Acked 7003 Read 3 Done Done New false
18015 6005 Acked 7004 Write 9 Done Done New false
18016 6006 Acked 7005 Read 7 Done Done New false
18017 6007 Acked 7006 Write 8 Done Done New false
18018 6008 Acked 7007 Flush 0 Done Done New false
18019 6009 Acked 7008 Read 8 Done Done New false
18020 6010 Acked 7009 Read 5 Done Done New false
18021 6011 Acked 7010 Read 4 Done Done New false
18022 6012 Acked 7011 Write 5 Done Done New false
18023 6013 Acked 7012 Read 7 Done Done New false
18024 6014 Acked 7013 Read 7 Done Done New false
18025 6015 Acked 7014 Read 3 Done Done New false
18026 6016 Acked 7015 Read 1 Done Done New false
18027 6017 Acked 7016 Write 5 Done Done New false
18028 6018 Acked 7017 Write 3 Done Done New false
18029 6019 Acked 7018 Read 5 Done Done New false
18030 6020 Acked 7019 Write 10 Done Done New false
18031 6021 Acked 7020 Read 5 Done Done New false
18032 6022 Acked 7021 Write 9 Done Done New false
18033 6023 Acked 7022 Read 6 Done Done New false
18034 6024 Acked 7023 Read 8 Done Done New false
18035 6025 Acked 7024 Read 10 Done Done New false
18036 6026 Acked 7025 Write 8 Done Done New false
18037 6027 Acked 7026 Read 3 Done Done New false
18038 6028 Acked 7027 Write 7 Done Done New false
18039 6029 Acked 7028 Read 8 Done Done New false
18040 6030 Acked 7029 Read 10 Done Done New false
18041 6031 Acked 7030 Flush 0 Done Done New false
18042 6032 Acked 7031 Read 5 Done Done New false
18043 6033 Acked 7032 Write 8 Done Done New false
18044 6034 Acked 7033 Read 6 Done Done New false
18045 6035 Acked 7034 Read 1 Done Done New false
18046 6036 Acked 7035 Read 6 Done Done New false
18047 6037 Acked 7036 Write 2 Done Done New false
18048 6038 Acked 7037 Read 7 Done Done New false
18049 6039 Acked 7038 Write 10 Done Done New false
18050 6040 Acked 7039 Write 1 Done Done New false
18051 6041 Acked 7040 Read 8 Done Done New false
18052 6042 Acked 7041 Read 10 Done Done New false
18053 6043 Acked 7042 Write 8 Done Done New false
18054 6044 Acked 7043 Write 6 Done Done New false
18055 6045 Acked 7044 Read 7 Done Done New false
18056 6046 Acked 7045 Write 8 Done Done New false
18057 6047 Acked 7046 Write 7 Done Done New false
18058 6048 Acked 7047 Read 2 Done Done New false
18059 6049 Acked 7048 Write 9 Done Done New false
18060 6050 Acked 7049 Flush 0 Done Done New false
18061 STATES DS:0 DS:1 DS:2 TOTAL
18062 New 0 0 303 303
18063 Sent 0 0 0 0
18064 Done 303 303 0 606
18065 Skipped 0 0 0 0
18066 Error 0 0 0 0
18067 Last Flush: 7049 7049 6746
18068 Downstairs last five completed: 6746 6745 6744 6743 6742 6741
18069 Upstairs last five completed: 6050 6049 6048 6047 6046 6045
18070 CLIENT: Up:0 ds:303 act:2
18071 [8830] got start action
18072 [8830] I am going to restart
18073 Starting downstairs at port 8830
18074 Make output file at "/tmp/dsc/downstairs-8830.txt"
18075 [8830][2] reports Starting
18076 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1193)
18077 [8830][2] reports Running
180782023-09-22T23:27:43.326ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
180792023-09-22T23:27:43.326ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
180802023-09-22T23:27:43.326ZINFOcrucible: [2] client re-new 303 jobs since flush 6746 = downstairs
180812023-09-22T23:27:43.327ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
180822023-09-22T23:27:43.328ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
180832023-09-22T23:27:43.328ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
180842023-09-22T23:27:43.328ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
180852023-09-22T23:27:43.328ZINFOcrucible: [2] send last flush ID to this DS: 6746
180862023-09-22T23:27:43.328ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
180872023-09-22T23:27:43.328ZINFOcrucible: [2] Replied this last flush ID: 6746
180882023-09-22T23:27:43.328ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
180892023-09-22T23:27:43.328ZINFOcrucible: [2] Transition from Offline to Replay
180902023-09-22T23:27:43.328ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
180912023-09-22T23:27:43.328ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
180922023-09-22T23:27:43.328ZINFOcrucible: [2] Transition from Replay to Active
180932023-09-22T23:27:43.328ZINFOcrucible: [2] Starts cmd_loop
180942023-09-22T23:27:43.329ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
180952023-09-22T23:27:43.329ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
180962023-09-22T23:27:43.329ZINFOcrucible: Set check for repair
180972023-09-22T23:27:44.329ZINFOcrucible: Checking if live repair is needed
180982023-09-22T23:27:44.329ZINFOcrucible: No Live Repair required at this time
180992023-09-22T23:27:44.329ZWARNcrucible: [2] flow control end
18100 ----------------------------------------------------------------
18101 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18102 Downstairs last five completed: 7049 7048 7047 7046 7045 7044
18103 Upstairs last five completed: 6050 6049 6048 6047 6046 6045
18104 CLIENT: Up:0 ds:0 act:3
18105 Replay: All jobs finished, all DS active.
18106 Main task has work to do, go find it
18107 got dsc Stop(2)
18108 stop 2
18109 Sep 22 23:27:45.243 INFO request completed, latency_us: 506, response_code: 204, uri: /stop/cid/2[8830] Got stop action so:false kr:false
18110 , method: POST, req_id: e4df0bd1-4264-40cd-bbd6-621a7b0abcb2, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
18111 Sep 22 23:27:45.244 INFO request completed, latency_us: 243, response_code: 200, uri: /state/cid/2, method: GET, req_id: a7e38c98-66ac-42ba-b563-5b078acf793a, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
181122023-09-22T23:27:45.244ZWARNcrucible: [2] downstairs disconnected
181132023-09-22T23:27:45.244ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
181142023-09-22T23:27:45.244ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
181152023-09-22T23:27:45.244ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
181162023-09-22T23:27:45.244ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
181172023-09-22T23:27:45.244ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
181182023-09-22T23:27:45.244ZWARNcrucible: [2] pm_task rx.recv() is None
18119 [8830] Exited with: ExitStatus(unix_wait_status(9))
18120 [8830][2] reports Exit
181212023-09-22T23:27:47.247ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
18122 Sep 22 23:27:49.247 INFO request completed, latency_us: 260, response_code: 200, uri: /state/cid/2, method: GET, req_id: dbd40cf0-7fd2-4463-b04b-60749d3c07e4, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
18123 001/300 Write block 2829 len 16384 data: 8 8 7 9
18124 002/300 Flush
18125 003/300 Write block 3566 len 28672 data: 3 4 4 4 4 5 4
18126 004/300 Write block 3931 len 8192 data: 8 9
18127 005/300 Read block 2619 len 16384
18128 006/300 Write block 1833 len 28672 data: 4 4 3 3 3 3 3
18129 007/300 Read block 2832 len 16384
18130 008/300 Read block 4527 len 36864
18131 009/300 Write block 4456 len 32768 data: 9 9 7 6 4 4 2 2
18132 010/300 Write block 4384 len 12288 data: 2 5 4
18133 011/300 Read block 4946 len 8192
18134 012/300 Write block 3510 len 20480 data: 5 8 8 9 10
18135 013/300 Read block 1751 len 8192
18136 014/300 Read block 4384 len 28672
18137 015/300 Write block 2362 len 16384 data: 5 5 6 5
18138 016/300 Read block 321 len 4096
18139 017/300 Read block 282 len 40960
18140 018/300 Write block 992 len 36864 data: 5 5 7 7 6 5 5 6 7
18141 019/300 Read block 4983 len 8192
18142 020/300 Read block 1569 len 40960
18143 021/300 Read block 507 len 28672
18144 022/300 Write block 4814 len 8192 data: 5 2
18145 023/300 Read block 3354 len 4096
18146 024/300 Write block 1759 len 20480 data: 6 6 5 4 4
18147 025/300 Read block 1969 len 40960
18148 026/300 Read block 1526 len 32768
18149 027/300 Write block 3221 len 40960 data: 3 3 3 3 3 2 3 4 3 3
18150 028/300 Read block 4162 len 12288
18151 029/300 Write block 2530 len 8192 data: 7 6
18152 030/300 Read block 4349 len 32768
18153 031/300 Read block 3299 len 20480
18154 032/300 Read block 3487 len 4096
18155 033/300 Read block 2749 len 28672
18156 034/300 Write block 4781 len 40960 data: 4 4 4 4 4 4 6 6 5 3
18157 035/300 Write block 108 len 20480 data: 5 6 6 6 7
18158 036/300 Read block 1231 len 4096
18159 037/300 Flush
18160 038/300 Read block 105 len 24576
18161 039/300 Read block 3919 len 20480
18162 040/300 Write block 3255 len 24576 data: 7 6 5 6 6 6
18163 041/300 Read block 3048 len 12288
18164 042/300 Read block 1560 len 32768
18165 043/300 Read block 1653 len 28672
18166 044/300 Write block 2567 len 20480 data: 3 3 4 4 4
18167 045/300 Read block 1835 len 20480
18168 046/300 Write block 4828 len 4096 data: 8
18169 047/300 Write block 2299 len 32768 data: 6 7 5 6 6 4 5 5
18170 048/300 Write block 729 len 8192 data: 6 5
18171 049/300 Write block 307 len 24576 data: 6 5 4 5 4 3
18172 050/300 Read block 2172 len 4096
18173 051/300 Write block 4688 len 8192 data: 4 4
18174 052/300 Write block 851 len 28672 data: 6 6 5 5 5 5 5
18175 053/300 Read block 1262 len 20480
18176 054/300 Read block 4274 len 24576
18177 055/300 Read block 3148 len 40960
18178 056/300 Flush
18179 057/300 Write block 4712 len 8192 data: 3 3
18180 058/300 Write block 1143 len 16384 data: 4 2 3 3
18181 059/300 Write block 1656 len 16384 data: 7 6 6 4
18182 060/300 Read block 4173 len 36864
18183 061/300 Read block 54 len 16384
18184 062/300 Read block 3713 len 36864
18185 063/300 Read block 3727 len 8192
18186 064/300 Write block 4232 len 32768 data: 3 4 4 4 3 2 2 2
18187 065/300 Write block 3982 len 36864 data: 3 4 5 6 6 7 7 5 5
18188 066/300 Write block 4315 len 32768 data: 6 3 3 3 4 4 5 7
18189 067/300 Write block 463 len 40960 data: 6 5 4 3 3 4 4 5 5 4
18190 068/300 Read block 811 len 28672
18191 069/300 Read block 240 len 40960
18192 070/300 Read block 2806 len 16384
18193 071/300 Write block 57 len 8192 data: 8 7
18194 072/300 Read block 3385 len 36864
18195 073/300 Read block 1536 len 24576
18196 074/300 Write block 165 len 8192 data: 5 4
18197 075/300 Write block 1479 len 32768 data: 10 9 7 6 3 4 5 4
18198 076/300 Write block 2061 len 24576 data: 3 3 3 4 4 5
18199 077/300 Write block 2195 len 16384 data: 5 6 6 8
18200 078/300 Flush
18201 079/300 Read block 3212 len 40960
18202 080/300 Flush
18203 081/300 Write block 1231 len 28672 data: 3 2 3 4 4 4 4
18204 082/300 Read block 521 len 16384
18205 083/300 Read block 1253 len 8192
18206 084/300 Read block 1011 len 16384
18207 085/300 Write block 4 len 40960 data: 3 2 2 2 3 3 4 5 5 4
18208 086/300 Read block 923 len 8192
18209 087/300 Read block 2027 len 40960
18210 088/300 Read block 966 len 16384
18211 089/300 Read block 525 len 16384
18212 090/300 Write block 10 len 20480 data: 5 6 6 5 2
18213 091/300 Read block 3795 len 16384
18214 092/300 Write block 697 len 16384 data: 5 5 7 6
18215 093/300 Write block 668 len 4096 data: 6
18216 094/300 Read block 870 len 40960
18217 095/300 Write block 3972 len 4096 data: 5
18218 096/300 Read block 4154 len 8192
18219 097/300 Write block 460 len 24576 data: 6 6 6 7 6 5
18220 098/300 Read block 2390 len 8192
18221 099/300 Write block 3018 len 8192 data: 3 4
18222 100/300 Write block 4369 len 32768 data: 5 4 4 3 4 3 4 5
18223 101/300 Write block 1800 len 32768 data: 6 5 4 4 3 3 4 5
18224 102/300 Write block 1208 len 36864 data: 10 10 7 6 7 6 7 6 5
18225 103/300 Write block 1334 len 16384 data: 4 5 6 6
18226 104/300 Read block 106 len 4096
18227 105/300 Read block 180 len 24576
18228 106/300 Write block 4161 len 16384 data: 5 5 5 5
18229 107/300 Write block 4730 len 12288 data: 6 6 7
18230 108/300 Read block 4839 len 12288
18231 109/300 Read block 274 len 36864
18232 110/300 Read block 1751 len 8192
18233 111/300 Write block 1396 len 20480 data: 3 4 4 4 6
18234 112/300 Write block 3251 len 32768 data: 4 5 4 5 8 7 6 7
18235 113/300 Read block 4803 len 16384
18236 114/300 Write block 4941 len 28672 data: 6 7 6 5 4 5 4
18237 115/300 Read block 1540 len 28672
18238 116/300 Flush
18239 117/300 Write block 4426 len 24576 data: 7 5 5 4 4 5
18240 118/300 Write block 362 len 40960 data: 3 3 3 2 3 3 3 4 3 3
18241 119/300 Read block 4481 len 24576
18242 120/300 Read block 1177 len 16384
18243 121/300 Read block 1304 len 20480
18244 122/300 Read block 1439 len 20480
18245 123/300 Read block 3846 len 28672
18246 124/300 Read block 3016 len 4096
18247 125/300 Write block 290 len 32768 data: 4 5 3 3 3 4 5 5
18248 126/300 Write block 1273 len 32768 data: 7 8 5 5 5 4 4 4
18249 127/300 Write block 2687 len 36864 data: 4 5 6 6 6 6 6 5 4
18250 128/300 Read block 3208 len 24576
18251 129/300 Write block 357 len 12288 data: 5 7 7
18252 130/300 Write block 665 len 12288 data: 5 8 7
18253 131/300 Flush
18254 132/300 Read block 1362 len 12288
18255 133/300 Write block 2448 len 28672 data: 4 6 6 5 7 5 4
18256 134/300 Flush
18257 135/300 Read block 1917 len 28672
18258 136/300 Read block 2890 len 32768
18259 137/300 Read block 79 len 40960
18260 138/300 Read block 2884 len 40960
18261 139/300 Read block 699 len 36864
18262 140/300 Read block 3388 len 16384
18263 141/300 Read block 1579 len 4096
18264 142/300 Read block 3446 len 20480
18265 143/300 Write block 3456 len 24576 data: 5 5 6 6 4 4
18266 144/300 Write block 1894 len 8192 data: 5 4
18267 145/300 Flush
18268 146/300 Write block 3245 len 4096 data: 3
18269 147/300 Write block 617 len 20480 data: 2 2 3 4 4
18270 148/300 Write block 826 len 16384 data: 5 4 3 3
18271 149/300 Write block 3259 len 4096 data: 7
18272 150/300 Read block 136 len 4096
18273 151/300 Write block 2805 len 20480 data: 3 5 5 4 5
18274 152/300 Read block 30 len 8192
18275 153/300 Read block 596 len 16384
18276 154/300 Read block 3743 len 40960
18277 155/300 Flush
18278 156/300 Read block 3383 len 32768
18279 157/300 Write block 184 len 32768 data: 4 4 6 6 5 5 5 4
18280 158/300 Read block 3224 len 36864
18281 159/300 Flush
18282 160/300 Write block 4497 len 32768 data: 4 4 3 3 2 2 2 3
18283 161/300 Write block 3496 len 12288 data: 5 4 5
18284 162/300 Write block 4562 len 20480 data: 5 5 4 3 3
18285 163/300 Read block 3418 len 40960
18286 164/300 Read block 3097 len 40960
18287 165/300 Write block 4420 len 8192 data: 7 7
18288 166/300 Write block 3425 len 24576 data: 3 3 6 5 4 4
18289 167/300 Write block 457 len 20480 data: 2 2 3 7 7
18290 168/300 Write block 3271 len 28672 data: 4 3 3 3 3 3 3
18291 169/300 Write block 3952 len 36864 data: 3 3 4 4 3 3 3 3 4
18292 170/300 Read block 242 len 40960
18293 171/300 Read block 3224 len 4096
18294 172/300 Read block 2455 len 8192
18295 173/300 Write block 1352 len 36864 data: 3 3 3 3 3 4 4 5 5
18296 174/300 Write block 321 len 4096 data: 4
18297 175/300 Write block 2110 len 16384 data: 8 7 7 6
18298 176/300 Flush
18299 177/300 Read block 2716 len 20480
18300 178/300 Flush
18301 179/300 Write block 1647 len 40960 data: 4 4 5 4 4 6 6 6 7 8
18302 180/300 Write block 176 len 32768 data: 6 6 6 6 5 5 3 4
18303 181/300 Read block 1504 len 8192
18304 182/300 Write block 3893 len 36864 data: 6 6 7 7 8 9 7 7 7
18305 183/300 Read block 4865 len 16384
18306 184/300 Read block 954 len 36864
18307 185/300 Write block 4046 len 28672 data: 6 5 5 7 6 5 5
18308 186/300 Write block 4526 len 40960 data: 4 5 5 5 6 4 5 4 3 3
18309 187/300 Read block 142 len 28672
18310 188/300 Read block 3692 len 12288
18311 189/300 Flush
18312 190/300 Write block 1110 len 4096 data: 4
18313 191/300 Write block 1938 len 8192 data: 4 5
18314 192/300 Read block 1604 len 24576
18315 193/300 Write block 1223 len 8192 data: 5 4
18316 194/300 Read block 3592 len 16384
18317 195/300 Write block 4912 len 36864 data: 8 8 8 8 9 9 7 6 4
18318 196/300 Write block 3158 len 16384 data: 5 4 5 4
18319 197/300 Flush
18320 198/300 Write block 200 len 16384 data: 4 4 4 4
18321 199/300 Read block 4053 len 4096
18322 200/300 Read block 3150 len 16384
18323 201/300 Read block 4233 len 16384
18324 202/300 Read block 4375 len 20480
18325 203/300 Flush
18326 204/300 Read block 2124 len 40960
18327 205/300 Read block 443 len 8192
18328 206/300 Write block 305 len 8192 data: 6 6
18329 207/300 Read block 1016 len 20480
18330 208/300 Read block 3625 len 16384
18331 209/300 Write block 4854 len 16384 data: 6 5 5 5
18332 210/300 Write block 3297 len 24576 data: 5 4 4 4 5 6
18333 211/300 Read block 4892 len 20480
18334 212/300 Write block 4695 len 28672 data: 2 4 3 3 3 3 3
18335 213/300 Read block 3857 len 12288
18336 214/300 Read block 2486 len 40960
18337 215/300 Write block 4074 len 16384 data: 4 4 4 6
18338 216/300 Flush
18339 217/300 Write block 4046 len 20480 data: 7 6 6 8 7
18340 218/300 Write block 602 len 12288 data: 3 4 5
18341 219/300 Read block 4666 len 36864
18342 220/300 Write block 874 len 36864 data: 8 8 8 6 4 4 5 6 5
18343 221/300 Write block 1888 len 8192 data: 4 5
18344 222/300 Read block 4388 len 36864
18345 223/300 Write block 4947 len 24576 data: 5 5 5 6 6 6
18346 224/300 Read block 1172 len 28672
18347 225/300 Write block 3332 len 40960 data: 2 2 2 3 3 4 4 5 7 7
18348 226/300 Write block 591 len 24576 data: 4 6 4 4 5 6
18349 227/300 Read block 1768 len 40960
18350 228/300 Write block 2654 len 16384 data: 2 2 2 3
18351 229/300 Write block 369 len 8192 data: 5 4
18352 230/300 Read block 1241 len 32768
18353 231/300 Read block 1201 len 12288
18354 232/300 Write block 4150 len 16384 data: 6 6 5 4
18355 233/300 Flush
18356 234/300 Read block 4574 len 4096
18357 235/300 Read block 2643 len 28672
18358 236/300 Read block 2043 len 28672
18359 237/300 Write block 1816 len 40960 data: 5 5 4 4 3 3 2 2 4 4
18360 238/300 Flush
18361 239/300 Write block 4561 len 20480 data: 5 6 6 5 4
18362 240/300 Write block 840 len 28672 data: 8 6 7 7 7 7 6
18363 241/300 Read block 2153 len 40960
18364 242/300 Read block 3442 len 12288
18365 243/300 Write block 1864 len 36864 data: 7 8 6 6 4 5 4 5 6
18366 244/300 Read block 2661 len 4096
18367 245/300 Read block 3442 len 16384
18368 246/300 Write block 2552 len 32768 data: 6 5 5 6 5 5 4 2
18369 247/300 Write block 3185 len 36864 data: 7 6 4 2 2 3 4 4 4
18370 248/300 Write block 2975 len 36864 data: 5 5 4 3 3 3 2 2 3
18371 249/300 Read block 4927 len 4096
18372 250/300 Read block 2260 len 24576
18373 251/300 Read block 4665 len 8192
18374 252/300 Write block 2181 len 4096 data: 4
18375 253/300 Read block 4614 len 8192
18376 254/300 Read block 946 len 16384
18377 255/300 Flush
18378 256/300 Read block 2958 len 40960
18379 257/300 Write block 4797 len 8192 data: 2 2
18380 258/300 Write block 4182 len 8192 data: 3 3
18381 259/300 Read block 3329 len 36864
18382 260/300 Write block 2014 len 36864 data: 2 2 2 2 3 3 4 4 6
18383 261/300 Write block 4764 len 40960 data: 7 5 4 5 3 4 4 5 5 5
18384 262/300 Flush
18385 263/300 Write block 2856 len 24576 data: 4 4 4 6 6 7
18386 264/300 Write block 3005 len 20480 data: 4 5 5 6 7
18387 265/300 Read block 3947 len 32768
18388 266/300 Read block 4630 len 16384
18389 267/300 Read block 345 len 32768
18390 268/300 Read block 2378 len 8192
18391 269/300 Read block 1999 len 32768
18392 270/300 Read block 3896 len 40960
18393 271/300 Read block 2490 len 36864
18394 272/300 Read block 1290 len 20480
18395 273/300 Read block 1049 len 4096
18396 274/300 Read block 953 len 12288
18397 275/300 Read block 3738 len 32768
18398 276/300 Read block 950 len 16384
18399 277/300 Write block 2851 len 28672 data: 5 4 5 5 5 5 5
18400 278/300 Read block 1781 len 4096
18401 279/300 Read block 2292 len 24576
18402 280/300 Write block 1701 len 16384 data: 5 5 4 5
18403 281/300 Write block 781 len 20480 data: 3 3 4 4 4
18404 282/300 Read block 2807 len 8192
18405 283/300 Read block 2255 len 8192
18406 284/300 Write block 248 len 24576 data: 5 4 5 6 8 5
18407 285/300 Read block 3034 len 40960
18408 286/300 Flush
18409 287/300 Read block 4646 len 8192
18410 288/300 Read block 1627 len 36864
18411 289/300 Write block 367 len 20480 data: 4 4 6 5 4
18412 290/300 Write block 1017 len 28672 data: 3 3 3 3 3 3 4
18413 291/300 Read block 658 len 8192
18414 292/300 Flush
18415 293/300 Read block 2653 len 32768
18416 294/300 Write block 4567 len 16384 data: 2 4 4 4
18417 295/300 Read block 3539 len 16384
18418 296/300 Write block 3450 len 12288 data: 5 5 4
18419 297/300 Read block 4600 len 24576
18420 298/300 Write block 2794 len 8192 data: 6 6
18421 299/300 Write block 3475 len 36864 data: 3 3 4 6 6 7 6 5 4
18422 300/300 Write block 2553 len 20480 data: 6 6 7 6 6
18423 Main task has work to do, go find it
18424 got dsc Start(2)
18425 start 2
18426 Sep 22 23:27:50.161 INFO request completed, latency_us: 302, response_code: 204, uri: /start/cid/2, method: POST, req_id: 04e69f96-53c4-4988-919f-0937da30b185, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
18427 Replay: started 2, returned:Ok(())
18428 ----------------------------------------------------------------
18429 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18430 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
18431 6051 Acked 7050 Write 4 Done Done New false
18432 6052 Acked 7051 Flush 0 Done Done New false
18433 6053 Acked 7052 Write 7 Done Done New false
18434 6054 Acked 7053 Write 2 Done Done New false
18435 6055 Acked 7054 Read 4 Done Done New false
18436 6056 Acked 7055 Write 7 Done Done New false
18437 6057 Acked 7056 Read 4 Done Done New false
18438 6058 Acked 7057 Read 9 Done Done New false
18439 6059 Acked 7058 Write 8 Done Done New false
18440 6060 Acked 7059 Write 3 Done Done New false
18441 6061 Acked 7060 Read 2 Done Done New false
18442 6062 Acked 7061 Write 5 Done Done New false
18443 6063 Acked 7062 Read 2 Done Done New false
18444 6064 Acked 7063 Read 7 Done Done New false
18445 6065 Acked 7064 Write 4 Done Done New false
18446 6066 Acked 7065 Read 1 Done Done New false
18447 6067 Acked 7066 Read 10 Done Done New false
18448 6068 Acked 7067 Write 9 Done Done New false
18449 6069 Acked 7068 Read 2 Done Done New false
18450 6070 Acked 7069 Read 10 Done Done New false
18451 6071 Acked 7070 Read 7 Done Done New false
18452 6072 Acked 7071 Write 2 Done Done New false
18453 6073 Acked 7072 Read 1 Done Done New false
18454 6074 Acked 7073 Write 5 Done Done New false
18455 6075 Acked 7074 Read 10 Done Done New false
18456 6076 Acked 7075 Read 8 Done Done New false
18457 6077 Acked 7076 Write 10 Done Done New false
18458 6078 Acked 7077 Read 3 Done Done New false
18459 6079 Acked 7078 Write 2 Done Done New false
18460 6080 Acked 7079 Read 8 Done Done New false
18461 6081 Acked 7080 Read 5 Done Done New false
18462 6082 Acked 7081 Read 1 Done Done New false
18463 6083 Acked 7082 Read 7 Done Done New false
18464 6084 Acked 7083 Write 10 Done Done New false
18465 6085 Acked 7084 Write 5 Done Done New false
18466 6086 Acked 7085 Read 1 Done Done New false
18467 6087 Acked 7086 Flush 0 Done Done New false
18468 6088 Acked 7087 Read 6 Done Done New false
18469 6089 Acked 7088 Read 5 Done Done New false
18470 6090 Acked 7089 Write 6 Done Done New false
18471 6091 Acked 7090 Read 3 Done Done New false
18472 6092 Acked 7091 Read 8 Done Done New false
18473 6093 Acked 7092 Read 7 Done Done New false
18474 6094 Acked 7093 Write 5 Done Done New false
18475 6095 Acked 7094 Read 5 Done Done New false
18476 6096 Acked 7095 Write 1 Done Done New false
18477 6097 Acked 7096 Write 8 Done Done New false
18478 6098 Acked 7097 Write 2 Done Done New false
18479 6099 Acked 7098 Write 6 Done Done New false
18480 6100 Acked 7099 Read 1 Done Done New false
18481 6101 Acked 7100 Write 2 Done Done New false
18482 6102 Acked 7101 Write 7 Done Done New false
18483 6103 Acked 7102 Read 5 Done Done New false
18484 6104 Acked 7103 Read 6 Done Done New false
18485 6105 Acked 7104 Read 10 Done Done New false
18486 6106 Acked 7105 Flush 0 Done Done New false
18487 6107 Acked 7106 Write 2 Done Done New false
18488 6108 Acked 7107 Write 4 Done Done New false
18489 6109 Acked 7108 Write 4 Done Done New false
18490 6110 Acked 7109 Read 9 Done Done New false
18491 6111 Acked 7110 Read 4 Done Done New false
18492 6112 Acked 7111 Read 9 Done Done New false
18493 6113 Acked 7112 Read 2 Done Done New false
18494 6114 Acked 7113 Write 8 Done Done New false
18495 6115 Acked 7114 Write 9 Done Done New false
18496 6116 Acked 7115 Write 8 Done Done New false
18497 6117 Acked 7116 Write 10 Done Done New false
18498 6118 Acked 7117 Read 7 Done Done New false
18499 6119 Acked 7118 Read 10 Done Done New false
18500 6120 Acked 7119 Read 4 Done Done New false
18501 6121 Acked 7120 Write 2 Done Done New false
18502 6122 Acked 7121 Read 9 Done Done New false
18503 6123 Acked 7122 Read 6 Done Done New false
18504 6124 Acked 7123 Write 2 Done Done New false
18505 6125 Acked 7124 Write 8 Done Done New false
18506 6126 Acked 7125 Write 6 Done Done New false
18507 6127 Acked 7126 Write 4 Done Done New false
18508 6128 Acked 7127 Flush 0 Done Done New false
18509 6129 Acked 7128 Read 10 Done Done New false
18510 6130 Acked 7129 Flush 0 Done Done New false
18511 6131 Acked 7130 Write 7 Done Done New false
18512 6132 Acked 7131 Read 4 Done Done New false
18513 6133 Acked 7132 Read 2 Done Done New false
18514 6134 Acked 7133 Read 4 Done Done New false
18515 6135 Acked 7134 Write 10 Done Done New false
18516 6136 Acked 7135 Read 2 Done Done New false
18517 6137 Acked 7136 Read 10 Done Done New false
18518 6138 Acked 7137 Read 4 Done Done New false
18519 6139 Acked 7138 Read 4 Done Done New false
18520 6140 Acked 7139 Write 5 Done Done New false
18521 6141 Acked 7140 Read 4 Done Done New false
18522 6142 Acked 7141 Write 4 Done Done New false
18523 6143 Acked 7142 Write 1 Done Done New false
18524 6144 Acked 7143 Read 10 Done Done New false
18525 6145 Acked 7144 Write 1 Done Done New false
18526 6146 Acked 7145 Read 2 Done Done New false
18527 6147 Acked 7146 Write 6 Done Done New false
18528 6148 Acked 7147 Read 2 Done Done New false
18529 6149 Acked 7148 Write 2 Done Done New false
18530 6150 Acked 7149 Write 8 Done Done New false
18531 6151 Acked 7150 Write 8 Done Done New false
18532 6152 Acked 7151 Write 9 Done Done New false
18533 6153 Acked 7152 Write 4 Done Done New false
18534 6154 Acked 7153 Read 1 Done Done New false
18535 6155 Acked 7154 Read 6 Done Done New false
18536 6156 Acked 7155 Write 4 Done Done New false
18537 6157 Acked 7156 Write 3 Done Done New false
18538 6158 Acked 7157 Read 3 Done Done New false
18539 6159 Acked 7158 Read 9 Done Done New false
18540 6160 Acked 7159 Read 2 Done Done New false
18541 6161 Acked 7160 Write 5 Done Done New false
18542 6162 Acked 7161 Write 8 Done Done New false
18543 6163 Acked 7162 Read 4 Done Done New false
18544 6164 Acked 7163 Write 7 Done Done New false
18545 6165 Acked 7164 Read 7 Done Done New false
18546 6166 Acked 7165 Flush 0 Done Done New false
18547 6167 Acked 7166 Write 6 Done Done New false
18548 6168 Acked 7167 Write 10 Done Done New false
18549 6169 Acked 7168 Read 6 Done Done New false
18550 6170 Acked 7169 Read 4 Done Done New false
18551 6171 Acked 7170 Read 5 Done Done New false
18552 6172 Acked 7171 Read 5 Done Done New false
18553 6173 Acked 7172 Read 7 Done Done New false
18554 6174 Acked 7173 Read 1 Done Done New false
18555 6175 Acked 7174 Write 8 Done Done New false
18556 6176 Acked 7175 Write 8 Done Done New false
18557 6177 Acked 7176 Write 9 Done Done New false
18558 6178 Acked 7177 Read 6 Done Done New false
18559 6179 Acked 7178 Write 3 Done Done New false
18560 6180 Acked 7179 Write 3 Done Done New false
18561 6181 Acked 7180 Flush 0 Done Done New false
18562 6182 Acked 7181 Read 3 Done Done New false
18563 6183 Acked 7182 Write 7 Done Done New false
18564 6184 Acked 7183 Flush 0 Done Done New false
18565 6185 Acked 7184 Read 7 Done Done New false
18566 6186 Acked 7185 Read 8 Done Done New false
18567 6187 Acked 7186 Read 10 Done Done New false
18568 6188 Acked 7187 Read 10 Done Done New false
18569 6189 Acked 7188 Read 9 Done Done New false
18570 6190 Acked 7189 Read 4 Done Done New false
18571 6191 Acked 7190 Read 1 Done Done New false
18572 6192 Acked 7191 Read 5 Done Done New false
18573 6193 Acked 7192 Write 6 Done Done New false
18574 6194 Acked 7193 Write 2 Done Done New false
18575 6195 Acked 7194 Flush 0 Done Done New false
18576 6196 Acked 7195 Write 1 Done Done New false
18577 6197 Acked 7196 Write 5 Done Done New false
18578 6198 Acked 7197 Write 4 Done Done New false
18579 6199 Acked 7198 Write 1 Done Done New false
18580 6200 Acked 7199 Read 1 Done Done New false
18581 6201 Acked 7200 Write 5 Done Done New false
18582 6202 Acked 7201 Read 2 Done Done New false
18583 6203 Acked 7202 Read 4 Done Done New false
18584 6204 Acked 7203 Read 10 Done Done New false
18585 6205 Acked 7204 Flush 0 Done Done New false
18586 6206 Acked 7205 Read 8 Done Done New false
18587 6207 Acked 7206 Write 8 Done Done New false
18588 6208 Acked 7207 Read 9 Done Done New false
18589 6209 Acked 7208 Flush 0 Done Done New false
18590 6210 Acked 7209 Write 8 Done Done New false
18591 6211 Acked 7210 Write 3 Done Done New false
18592 6212 Acked 7211 Write 5 Done Done New false
18593 6213 Acked 7212 Read 10 Done Done New false
18594 6214 Acked 7213 Read 10 Done Done New false
18595 6215 Acked 7214 Write 2 Done Done New false
18596 6216 Acked 7215 Write 6 Done Done New false
18597 6217 Acked 7216 Write 5 Done Done New false
18598 6218 Acked 7217 Write 7 Done Done New false
18599 6219 Acked 7218 Write 9 Done Done New false
18600 6220 Acked 7219 Read 10 Done Done New false
18601 6221 Acked 7220 Read 1 Done Done New false
18602 6222 Acked 7221 Read 2 Done Done New false
18603 6223 Acked 7222 Write 9 Done Done New false
18604 6224 Acked 7223 Write 1 Done Done New false
18605 6225 Acked 7224 Write 4 Done Done New false
18606 6226 Acked 7225 Flush 0 Done Done New false
18607 6227 Acked 7226 Read 5 Done Done New false
18608 6228 Acked 7227 Flush 0 Done Done New false
18609 6229 Acked 7228 Write 10 Done Done New false
18610 6230 Acked 7229 Write 8 Done Done New false
18611 6231 Acked 7230 Read 2 Done Done New false
18612 6232 Acked 7231 Write 9 Done Done New false
18613 6233 Acked 7232 Read 4 Done Done New false
18614 6234 Acked 7233 Read 9 Done Done New false
18615 6235 Acked 7234 Write 7 Done Done New false
18616 6236 Acked 7235 Write 10 Done Done New false
18617 6237 Acked 7236 Read 7 Done Done New false
18618 6238 Acked 7237 Read 3 Done Done New false
18619 6239 Acked 7238 Flush 0 Done Done New false
18620 6240 Acked 7239 Write 1 Done Done New false
18621 6241 Acked 7240 Write 2 Done Done New false
18622 6242 Acked 7241 Read 6 Done Done New false
18623 6243 Acked 7242 Write 2 Done Done New false
18624 6244 Acked 7243 Read 4 Done Done New false
18625 6245 Acked 7244 Write 9 Done Done New false
18626 6246 Acked 7245 Write 4 Done Done New false
18627 6247 Acked 7246 Flush 0 Done Done New false
18628 6248 Acked 7247 Write 4 Done Done New false
18629 6249 Acked 7248 Read 1 Done Done New false
18630 6250 Acked 7249 Read 4 Done Done New false
18631 6251 Acked 7250 Read 4 Done Done New false
18632 6252 Acked 7251 Read 5 Done Done New false
18633 6253 Acked 7252 Flush 0 Done Done New false
18634 6254 Acked 7253 Read 10 Done Done New false
18635 6255 Acked 7254 Read 2 Done Done New false
18636 6256 Acked 7255 Write 2 Done Done New false
18637 6257 Acked 7256 Read 5 Done Done New false
18638 6258 Acked 7257 Read 4 Done Done New false
18639 6259 Acked 7258 Write 4 Done Done New false
18640 6260 Acked 7259 Write 6 Done Done New false
18641 6261 Acked 7260 Read 5 Done Done New false
18642 6262 Acked 7261 Write 7 Done Done New false
18643 6263 Acked 7262 Read 3 Done Done New false
18644 6264 Acked 7263 Read 10 Done Done New false
18645 6265 Acked 7264 Write 4 Done Done New false
18646 6266 Acked 7265 Flush 0 Done Done New false
18647 6267 Acked 7266 Write 5 Done Done New false
18648 6268 Acked 7267 Write 3 Done Done New false
18649 6269 Acked 7268 Read 9 Done Done New false
18650 6270 Acked 7269 Write 9 Done Done New false
18651 6271 Acked 7270 Write 2 Done Done New false
18652 6272 Acked 7271 Read 9 Done Done New false
18653 6273 Acked 7272 Write 6 Done Done New false
18654 6274 Acked 7273 Read 7 Done Done New false
18655 6275 Acked 7274 Write 10 Done Done New false
18656 6276 Acked 7275 Write 6 Done Done New false
18657 6277 Acked 7276 Read 10 Done Done New false
18658 6278 Acked 7277 Write 4 Done Done New false
18659 6279 Acked 7278 Write 2 Done Done New false
18660 6280 Acked 7279 Read 8 Done Done New false
18661 6281 Acked 7280 Read 3 Done Done New false
18662 6282 Acked 7281 Write 4 Done Done New false
18663 6283 Acked 7282 Flush 0 Done Done New false
18664 6284 Acked 7283 Read 1 Done Done New false
18665 6285 Acked 7284 Read 7 Done Done New false
18666 6286 Acked 7285 Read 7 Done Done New false
18667 6287 Acked 7286 Write 10 Done Done New false
18668 6288 Acked 7287 Flush 0 Done Done New false
18669 6289 Acked 7288 Write 5 Done Done New false
18670 6290 Acked 7289 Write 7 Done Done New false
18671 6291 Acked 7290 Read 10 Done Done New false
18672 6292 Acked 7291 Flush 0 Done Done New false
18673 6293 Acked 7292 Read 3 Done Done New false
18674 6294 Acked 7293 Write 9 Done Done New false
18675 6295 Acked 7294 Read 1 Done Done New false
18676 6296 Acked 7295 Read 4 Done Done New false
18677 6297 Acked 7296 Write 8 Done Done New false
18678 6298 Acked 7297 Write 9 Done Done New false
18679 6299 Acked 7298 Write 9 Done Done New false
18680 6300 Acked 7299 Read 1 Done Done New false
18681 6301 Acked 7300 Read 6 Done Done New false
18682 6302 Acked 7301 Read 2 Done Done New false
18683 6303 Acked 7302 Write 1 Done Done New false
18684 6304 Acked 7303 Read 2 Done Done New false
18685 6305 Acked 7304 Read 4 Done Done New false
18686 6306 Acked 7305 Flush 0 Done Done New false
18687 6307 Acked 7306 Read 10 Done Done New false
18688 6308 Acked 7307 Write 2 Done Done New false
18689 6309 Acked 7308 Write 2 Done Done New false
18690 6310 Acked 7309 Read 9 Done Done New false
18691 6311 Acked 7310 Write 9 Done Done New false
18692 6312 Acked 7311 Write 10 Done Done New false
18693 6313 Acked 7312 Flush 0 Done Done New false
18694 6314 Acked 7313 Write 6 Done Done New false
18695 6315 Acked 7314 Write 5 Done Done New false
18696 6316 Acked 7315 Read 8 Done Done New false
18697 6317 Acked 7316 Read 4 Done Done New false
18698 6318 Acked 7317 Read 8 Done Done New false
18699 6319 Acked 7318 Read 2 Done Done New false
18700 6320 Acked 7319 Read 8 Done Done New false
18701 6321 Acked 7320 Read 10 Done Done New false
18702 6322 Acked 7321 Read 9 Done Done New false
18703 6323 Acked 7322 Read 5 Done Done New false
18704 6324 Acked 7323 Read 1 Done Done New false
18705 6325 Acked 7324 Read 3 Done Done New false
18706 6326 Acked 7325 Read 8 Done Done New false
18707 6327 Acked 7326 Read 4 Done Done New false
18708 6328 Acked 7327 Write 7 Done Done New false
18709 6329 Acked 7328 Read 1 Done Done New false
18710 6330 Acked 7329 Read 6 Done Done New false
18711 6331 Acked 7330 Write 4 Done Done New false
18712 6332 Acked 7331 Write 5 Done Done New false
18713 6333 Acked 7332 Read 2 Done Done New false
18714 6334 Acked 7333 Read 2 Done Done New false
18715 6335 Acked 7334 Write 6 Done Done New false
18716 6336 Acked 7335 Read 10 Done Done New false
18717 6337 Acked 7336 Flush 0 Done Done New false
18718 6338 Acked 7337 Read 2 Done Done New false
18719 6339 Acked 7338 Read 9 Done Done New false
18720 6340 Acked 7339 Write 5 Done Done New false
18721 6341 Acked 7340 Write 7 Done Done New false
18722 6342 Acked 7341 Read 2 Done Done New false
18723 6343 Acked 7342 Flush 0 Done Done New false
18724 6344 Acked 7343 Read 8 Done Done New false
18725 6345 Acked 7344 Write 4 Done Done New false
18726 6346 Acked 7345 Read 4 Done Done New false
18727 6347 Acked 7346 Write 3 Done Done New false
18728 6348 Acked 7347 Read 6 Done Done New false
18729 6349 Acked 7348 Write 2 Done Done New false
18730 6350 Acked 7349 Write 9 Done Done New false
18731 6351 Acked 7350 Write 5 Sent Sent New false
18732 STATES DS:0 DS:1 DS:2 TOTAL
18733 New 0 0 301 301
18734 Sent 1 1 0 2
18735 Done 300 300 0 600
18736 Skipped 0 0 0 0
18737 Error 0 0 0 0
18738 Last Flush: 7342 7342 7049
18739 Downstairs last five completed: 7049 7048 7047 7046 7045 7044
18740 Upstairs last five completed: 6351 6350 6349 6348 6347 6346
18741 CLIENT: Up:0 ds:301 act:2
18742 ----------------------------------------------------------------
18743 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18744 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
18745 6051 Acked 7050 Write 4 Done Done New false
18746 6052 Acked 7051 Flush 0 Done Done New false
18747 6053 Acked 7052 Write 7 Done Done New false
18748 6054 Acked 7053 Write 2 Done Done New false
18749 6055 Acked 7054 Read 4 Done Done New false
18750 6056 Acked 7055 Write 7 Done Done New false
18751 6057 Acked 7056 Read 4 Done Done New false
18752 6058 Acked 7057 Read 9 Done Done New false
18753 6059 Acked 7058 Write 8 Done Done New false
18754 6060 Acked 7059 Write 3 Done Done New false
18755 6061 Acked 7060 Read 2 Done Done New false
18756 6062 Acked 7061 Write 5 Done Done New false
18757 6063 Acked 7062 Read 2 Done Done New false
18758 6064 Acked 7063 Read 7 Done Done New false
18759 6065 Acked 7064 Write 4 Done Done New false
18760 6066 Acked 7065 Read 1 Done Done New false
18761 6067 Acked 7066 Read 10 Done Done New false
18762 6068 Acked 7067 Write 9 Done Done New false
18763 6069 Acked 7068 Read 2 Done Done New false
18764 6070 Acked 7069 Read 10 Done Done New false
18765 6071 Acked 7070 Read 7 Done Done New false
18766 6072 Acked 7071 Write 2 Done Done New false
18767 6073 Acked 7072 Read 1 Done Done New false
18768 6074 Acked 7073 Write 5 Done Done New false
18769 6075 Acked 7074 Read 10 Done Done New false
18770 6076 Acked 7075 Read 8 Done Done New false
18771 6077 Acked 7076 Write 10 Done Done New false
18772 6078 Acked 7077 Read 3 Done Done New false
18773 6079 Acked 7078 Write 2 Done Done New false
18774 6080 Acked 7079 Read 8 Done Done New false
18775 6081 Acked 7080 Read 5 Done Done New false
18776 6082 Acked 7081 Read 1 Done Done New false
18777 6083 Acked 7082 Read 7 Done Done New false
18778 6084 Acked 7083 Write 10 Done Done New false
18779 6085 Acked 7084 Write 5 Done Done New false
18780 6086 Acked 7085 Read 1 Done Done New false
18781 6087 Acked 7086 Flush 0 Done Done New false
18782 6088 Acked 7087 Read 6 Done Done New false
18783 6089 Acked 7088 Read 5 Done Done New false
18784 6090 Acked 7089 Write 6 Done Done New false
18785 6091 Acked 7090 Read 3 Done Done New false
18786 6092 Acked 7091 Read 8 Done Done New false
18787 6093 Acked 7092 Read 7 Done Done New false
18788 6094 Acked 7093 Write 5 Done Done New false
18789 6095 Acked 7094 Read 5 Done Done New false
18790 6096 Acked 7095 Write 1 Done Done New false
18791 6097 Acked 7096 Write 8 Done Done New false
18792 6098 Acked 7097 Write 2 Done Done New false
18793 6099 Acked 7098 Write 6 Done Done New false
18794 6100 Acked 7099 Read 1 Done Done New false
18795 6101 Acked 7100 Write 2 Done Done New false
18796 6102 Acked 7101 Write 7 Done Done New false
18797 6103 Acked 7102 Read 5 Done Done New false
18798 6104 Acked 7103 Read 6 Done Done New false
18799 6105 Acked 7104 Read 10 Done Done New false
18800 6106 Acked 7105 Flush 0 Done Done New false
18801 6107 Acked 7106 Write 2 Done Done New false
18802 6108 Acked 7107 Write 4 Done Done New false
18803 6109 Acked 7108 Write 4 Done Done New false
18804 6110 Acked 7109 Read 9 Done Done New false
18805 6111 Acked 7110 Read 4 Done Done New false
18806 6112 Acked 7111 Read 9 Done Done New false
18807 6113 Acked 7112 Read 2 Done Done New false
18808 6114 Acked 7113 Write 8 Done Done New false
18809 6115 Acked 7114 Write 9 Done Done New false
18810 6116 Acked 7115 Write 8 Done Done New false
18811 6117 Acked 7116 Write 10 Done Done New false
18812 6118 Acked 7117 Read 7 Done Done New false
18813 6119 Acked 7118 Read 10 Done Done New false
18814 6120 Acked 7119 Read 4 Done Done New false
18815 6121 Acked 7120 Write 2 Done Done New false
18816 6122 Acked 7121 Read 9 Done Done New false
18817 6123 Acked 7122 Read 6 Done Done New false
18818 6124 Acked 7123 Write 2 Done Done New false
18819 6125 Acked 7124 Write 8 Done Done New false
18820 6126 Acked 7125 Write 6 Done Done New false
18821 6127 Acked 7126 Write 4 Done Done New false
18822 6128 Acked 7127 Flush 0 Done Done New false
18823 6129 Acked 7128 Read 10 Done Done New false
18824 6130 Acked 7129 Flush 0 Done Done New false
18825 6131 Acked 7130 Write 7 Done Done New false
18826 6132 Acked 7131 Read 4 Done Done New false
18827 6133 Acked 7132 Read 2 Done Done New false
18828 6134 Acked 7133 Read 4 Done Done New false
18829 6135 Acked 7134 Write 10 Done Done New false
18830 6136 Acked 7135 Read 2 Done Done New false
18831 6137 Acked 7136 Read 10 Done Done New false
18832 6138 Acked 7137 Read 4 Done Done New false
18833 6139 Acked 7138 Read 4 Done Done New false
18834 6140 Acked 7139 Write 5 Done Done New false
18835 6141 Acked 7140 Read 4 Done Done New false
18836 6142 Acked 7141 Write 4 Done Done New false
18837 6143 Acked 7142 Write 1 Done Done New false
18838 6144 Acked 7143 Read 10 Done Done New false
18839 6145 Acked 7144 Write 1 Done Done New false
18840 6146 Acked 7145 Read 2 Done Done New false
18841 6147 Acked 7146 Write 6 Done Done New false
18842 6148 Acked 7147 Read 2 Done Done New false
18843 6149 Acked 7148 Write 2 Done Done New false
18844 6150 Acked 7149 Write 8 Done Done New false
18845 6151 Acked 7150 Write 8 Done Done New false
18846 6152 Acked 7151 Write 9 Done Done New false
18847 6153 Acked 7152 Write 4 Done Done New false
18848 6154 Acked 7153 Read 1 Done Done New false
18849 6155 Acked 7154 Read 6 Done Done New false
18850 6156 Acked 7155 Write 4 Done Done New false
18851 6157 Acked 7156 Write 3 Done Done New false
18852 6158 Acked 7157 Read 3 Done Done New false
18853 6159 Acked 7158 Read 9 Done Done New false
18854 6160 Acked 7159 Read 2 Done Done New false
18855 6161 Acked 7160 Write 5 Done Done New false
18856 6162 Acked 7161 Write 8 Done Done New false
18857 6163 Acked 7162 Read 4 Done Done New false
18858 6164 Acked 7163 Write 7 Done Done New false
18859 6165 Acked 7164 Read 7 Done Done New false
18860 6166 Acked 7165 Flush 0 Done Done New false
18861 6167 Acked 7166 Write 6 Done Done New false
18862 6168 Acked 7167 Write 10 Done Done New false
18863 6169 Acked 7168 Read 6 Done Done New false
18864 6170 Acked 7169 Read 4 Done Done New false
18865 6171 Acked 7170 Read 5 Done Done New false
18866 6172 Acked 7171 Read 5 Done Done New false
18867 6173 Acked 7172 Read 7 Done Done New false
18868 6174 Acked 7173 Read 1 Done Done New false
18869 6175 Acked 7174 Write 8 Done Done New false
18870 6176 Acked 7175 Write 8 Done Done New false
18871 6177 Acked 7176 Write 9 Done Done New false
18872 6178 Acked 7177 Read 6 Done Done New false
18873 6179 Acked 7178 Write 3 Done Done New false
18874 6180 Acked 7179 Write 3 Done Done New false
18875 6181 Acked 7180 Flush 0 Done Done New false
18876 6182 Acked 7181 Read 3 Done Done New false
18877 6183 Acked 7182 Write 7 Done Done New false
18878 6184 Acked 7183 Flush 0 Done Done New false
18879 6185 Acked 7184 Read 7 Done Done New false
18880 6186 Acked 7185 Read 8 Done Done New false
18881 6187 Acked 7186 Read 10 Done Done New false
18882 6188 Acked 7187 Read 10 Done Done New false
18883 6189 Acked 7188 Read 9 Done Done New false
18884 6190 Acked 7189 Read 4 Done Done New false
18885 6191 Acked 7190 Read 1 Done Done New false
18886 6192 Acked 7191 Read 5 Done Done New false
18887 6193 Acked 7192 Write 6 Done Done New false
18888 6194 Acked 7193 Write 2 Done Done New false
18889 6195 Acked 7194 Flush 0 Done Done New false
18890 6196 Acked 7195 Write 1 Done Done New false
18891 6197 Acked 7196 Write 5 Done Done New false
18892 6198 Acked 7197 Write 4 Done Done New false
18893 6199 Acked 7198 Write 1 Done Done New false
18894 6200 Acked 7199 Read 1 Done Done New false
18895 6201 Acked 7200 Write 5 Done Done New false
18896 6202 Acked 7201 Read 2 Done Done New false
18897 6203 Acked 7202 Read 4 Done Done New false
18898 6204 Acked 7203 Read 10 Done Done New false
18899 6205 Acked 7204 Flush 0 Done Done New false
18900 6206 Acked 7205 Read 8 Done Done New false
18901 6207 Acked 7206 Write 8 Done Done New false
18902 6208 Acked 7207 Read 9 Done Done New false
18903 6209 Acked 7208 Flush 0 Done Done New false
18904 6210 Acked 7209 Write 8 Done Done New false
18905 6211 Acked 7210 Write 3 Done Done New false
18906 6212 Acked 7211 Write 5 Done Done New false
18907 6213 Acked 7212 Read 10 Done Done New false
18908 6214 Acked 7213 Read 10 Done Done New false
18909 6215 Acked 7214 Write 2 Done Done New false
18910 6216 Acked 7215 Write 6 Done Done New false
18911 6217 Acked 7216 Write 5 Done Done New false
18912 6218 Acked 7217 Write 7 Done Done New false
18913 6219 Acked 7218 Write 9 Done Done New false
18914 6220 Acked 7219 Read 10 Done Done New false
18915 6221 Acked 7220 Read 1 Done Done New false
18916 6222 Acked 7221 Read 2 Done Done New false
18917 6223 Acked 7222 Write 9 Done Done New false
18918 6224 Acked 7223 Write 1 Done Done New false
18919 6225 Acked 7224 Write 4 Done Done New false
18920 6226 Acked 7225 Flush 0 Done Done New false
18921 6227 Acked 7226 Read 5 Done Done New false
18922 6228 Acked 7227 Flush 0 Done Done New false
18923 6229 Acked 7228 Write 10 Done Done New false
18924 6230 Acked 7229 Write 8 Done Done New false
18925 6231 Acked 7230 Read 2 Done Done New false
18926 6232 Acked 7231 Write 9 Done Done New false
18927 6233 Acked 7232 Read 4 Done Done New false
18928 6234 Acked 7233 Read 9 Done Done New false
18929 6235 Acked 7234 Write 7 Done Done New false
18930 6236 Acked 7235 Write 10 Done Done New false
18931 6237 Acked 7236 Read 7 Done Done New false
18932 6238 Acked 7237 Read 3 Done Done New false
18933 6239 Acked 7238 Flush 0 Done Done New false
18934 6240 Acked 7239 Write 1 Done Done New false
18935 6241 Acked 7240 Write 2 Done Done New false
18936 6242 Acked 7241 Read 6 Done Done New false
18937 6243 Acked 7242 Write 2 Done Done New false
18938 6244 Acked 7243 Read 4 Done Done New false
18939 6245 Acked 7244 Write 9 Done Done New false
18940 6246 Acked 7245 Write 4 Done Done New false
18941 6247 Acked 7246 Flush 0 Done Done New false
18942 6248 Acked 7247 Write 4 Done Done New false
18943 6249 Acked 7248 Read 1 Done Done New false
18944 6250 Acked 7249 Read 4 Done Done New false
18945 6251 Acked 7250 Read 4 Done Done New false
18946 6252 Acked 7251 Read 5 Done Done New false
18947 6253 Acked 7252 Flush 0 Done Done New false
18948 6254 Acked 7253 Read 10 Done Done New false
18949 6255 Acked 7254 Read 2 Done Done New false
18950 6256 Acked 7255 Write 2 Done Done New false
18951 6257 Acked 7256 Read 5 Done Done New false
18952 6258 Acked 7257 Read 4 Done Done New false
18953 6259 Acked 7258 Write 4 Done Done New false
18954 6260 Acked 7259 Write 6 Done Done New false
18955 6261 Acked 7260 Read 5 Done Done New false
18956 6262 Acked 7261 Write 7 Done Done New false
18957 6263 Acked 7262 Read 3 Done Done New false
18958 6264 Acked 7263 Read 10 Done Done New false
18959 6265 Acked 7264 Write 4 Done Done New false
18960 6266 Acked 7265 Flush 0 Done Done New false
18961 6267 Acked 7266 Write 5 Done Done New false
18962 6268 Acked 7267 Write 3 Done Done New false
18963 6269 Acked 7268 Read 9 Done Done New false
18964 6270 Acked 7269 Write 9 Done Done New false
18965 6271 Acked 7270 Write 2 Done Done New false
18966 6272 Acked 7271 Read 9 Done Done New false
18967 6273 Acked 7272 Write 6 Done Done New false
18968 6274 Acked 7273 Read 7 Done Done New false
18969 6275 Acked 7274 Write 10 Done Done New false
18970 6276 Acked 7275 Write 6 Done Done New false
18971 6277 Acked 7276 Read 10 Done Done New false
18972 6278 Acked 7277 Write 4 Done Done New false
18973 6279 Acked 7278 Write 2 Done Done New false
18974 6280 Acked 7279 Read 8 Done Done New false
18975 6281 Acked 7280 Read 3 Done Done New false
18976 6282 Acked 7281 Write 4 Done Done New false
18977 6283 Acked 7282 Flush 0 Done Done New false
18978 6284 Acked 7283 Read 1 Done Done New false
18979 6285 Acked 7284 Read 7 Done Done New false
18980 6286 Acked 7285 Read 7 Done Done New false
18981 6287 Acked 7286 Write 10 Done Done New false
18982 6288 Acked 7287 Flush 0 Done Done New false
18983 6289 Acked 7288 Write 5 Done Done New false
18984 6290 Acked 7289 Write 7 Done Done New false
18985 6291 Acked 7290 Read 10 Done Done New false
18986 6292 Acked 7291 Flush 0 Done Done New false
18987 6293 Acked 7292 Read 3 Done Done New false
18988 6294 Acked 7293 Write 9 Done Done New false
18989 6295 Acked 7294 Read 1 Done Done New false
18990 6296 Acked 7295 Read 4 Done Done New false
18991 6297 Acked 7296 Write 8 Done Done New false
18992 6298 Acked 7297 Write 9 Done Done New false
18993 6299 Acked 7298 Write 9 Done Done New false
18994 6300 Acked 7299 Read 1 Done Done New false
18995 6301 Acked 7300 Read 6 Done Done New false
18996 6302 Acked 7301 Read 2 Done Done New false
18997 6303 Acked 7302 Write 1 Done Done New false
18998 6304 Acked 7303 Read 2 Done Done New false
18999 6305 Acked 7304 Read 4 Done Done New false
19000 6306 Acked 7305 Flush 0 Done Done New false
19001 6307 Acked 7306 Read 10 Done Done New false
19002 6308 Acked 7307 Write 2 Done Done New false
19003 6309 Acked 7308 Write 2 Done Done New false
19004 6310 Acked 7309 Read 9 Done Done New false
19005 6311 Acked 7310 Write 9 Done Done New false
19006 6312 Acked 7311 Write 10 Done Done New false
19007 6313 Acked 7312 Flush 0 Done Done New false
19008 6314 Acked 7313 Write 6 Done Done New false
19009 6315 Acked 7314 Write 5 Done Done New false
19010 6316 Acked 7315 Read 8 Done Done New false
19011 6317 Acked 7316 Read 4 Done Done New false
19012 6318 Acked 7317 Read 8 Done Done New false
19013 6319 Acked 7318 Read 2 Done Done New false
19014 6320 Acked 7319 Read 8 Done Done New false
19015 6321 Acked 7320 Read 10 Done Done New false
19016 6322 Acked 7321 Read 9 Done Done New false
19017 6323 Acked 7322 Read 5 Done Done New false
19018 6324 Acked 7323 Read 1 Done Done New false
19019 6325 Acked 7324 Read 3 Done Done New false
19020 6326 Acked 7325 Read 8 Done Done New false
19021 6327 Acked 7326 Read 4 Done Done New false
19022 6328 Acked 7327 Write 7 Done Done New false
19023 6329 Acked 7328 Read 1 Done Done New false
19024 6330 Acked 7329 Read 6 Done Done New false
19025 6331 Acked 7330 Write 4 Done Done New false
19026 6332 Acked 7331 Write 5 Done Done New false
19027 6333 Acked 7332 Read 2 Done Done New false
19028 6334 Acked 7333 Read 2 Done Done New false
19029 6335 Acked 7334 Write 6 Done Done New false
19030 6336 Acked 7335 Read 10 Done Done New false
19031 6337 Acked 7336 Flush 0 Done Done New false
19032 6338 Acked 7337 Read 2 Done Done New false
19033 6339 Acked 7338 Read 9 Done Done New false
19034 6340 Acked 7339 Write 5 Done Done New false
19035 6341 Acked 7340 Write 7 Done Done New false
19036 6342 Acked 7341 Read 2 Done Done New false
19037 6343 Acked 7342 Flush 0 Done Done New false
19038 6344 Acked 7343 Read 8 Done Done New false
19039 6345 Acked 7344 Write 4 Done Done New false
19040 6346 Acked 7345 Read 4 Done Done New false
19041 6347 Acked 7346 Write 3 Done Done New false
19042 6348 Acked 7347 Read 6 Done Done New false
19043 6349 Acked 7348 Write 2 Done Done New false
19044 6350 Acked 7349 Write 9 Done Done New false
19045 6351 Acked 7350 Write 5 Done Done New false
19046 6352 Acked 7351 Flush 0 Done Done New false
19047 STATES DS:0 DS:1 DS:2 TOTAL
19048 New 0 0 302 302
19049 Sent 0 0 0 0
19050 Done 302 302 0 604
19051 Skipped 0 0 0 0
19052 Error 0 0 0 0
19053 Last Flush: 7351 7351 7049
19054 Downstairs last five completed: 7049 7048 7047 7046 7045 7044
19055 Upstairs last five completed: 6352 6351 6350 6349 6348 6347
19056 CLIENT: Up:0 ds:302 act:2
190572023-09-22T23:27:57.258ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
19058 [8830] got start action
19059 [8830] I am going to restart
19060 Starting downstairs at port 8830
19061 Make output file at "/tmp/dsc/downstairs-8830.txt"
19062 [8830][2] reports Starting
19063 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1194)
19064 [8830][2] reports Running
19065 ----------------------------------------------------------------
19066 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19067 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
19068 6051 Acked 7050 Write 4 Done Done New false
19069 6052 Acked 7051 Flush 0 Done Done New false
19070 6053 Acked 7052 Write 7 Done Done New false
19071 6054 Acked 7053 Write 2 Done Done New false
19072 6055 Acked 7054 Read 4 Done Done New false
19073 6056 Acked 7055 Write 7 Done Done New false
19074 6057 Acked 7056 Read 4 Done Done New false
19075 6058 Acked 7057 Read 9 Done Done New false
19076 6059 Acked 7058 Write 8 Done Done New false
19077 6060 Acked 7059 Write 3 Done Done New false
19078 6061 Acked 7060 Read 2 Done Done New false
19079 6062 Acked 7061 Write 5 Done Done New false
19080 6063 Acked 7062 Read 2 Done Done New false
19081 6064 Acked 7063 Read 7 Done Done New false
19082 6065 Acked 7064 Write 4 Done Done New false
19083 6066 Acked 7065 Read 1 Done Done New false
19084 6067 Acked 7066 Read 10 Done Done New false
19085 6068 Acked 7067 Write 9 Done Done New false
19086 6069 Acked 7068 Read 2 Done Done New false
19087 6070 Acked 7069 Read 10 Done Done New false
19088 6071 Acked 7070 Read 7 Done Done New false
19089 6072 Acked 7071 Write 2 Done Done New false
19090 6073 Acked 7072 Read 1 Done Done New false
19091 6074 Acked 7073 Write 5 Done Done New false
19092 6075 Acked 7074 Read 10 Done Done New false
19093 6076 Acked 7075 Read 8 Done Done New false
19094 6077 Acked 7076 Write 10 Done Done New false
19095 6078 Acked 7077 Read 3 Done Done New false
19096 6079 Acked 7078 Write 2 Done Done New false
19097 6080 Acked 7079 Read 8 Done Done New false
19098 6081 Acked 7080 Read 5 Done Done New false
19099 6082 Acked 7081 Read 1 Done Done New false
19100 6083 Acked 7082 Read 7 Done Done New false
19101 6084 Acked 7083 Write 10 Done Done New false
19102 6085 Acked 7084 Write 5 Done Done New false
19103 6086 Acked 7085 Read 1 Done Done New false
19104 6087 Acked 7086 Flush 0 Done Done New false
19105 6088 Acked 7087 Read 6 Done Done New false
19106 6089 Acked 7088 Read 5 Done Done New false
19107 6090 Acked 7089 Write 6 Done Done New false
19108 6091 Acked 7090 Read 3 Done Done New false
19109 6092 Acked 7091 Read 8 Done Done New false
19110 6093 Acked 7092 Read 7 Done Done New false
19111 6094 Acked 7093 Write 5 Done Done New false
19112 6095 Acked 7094 Read 5 Done Done New false
19113 6096 Acked 7095 Write 1 Done Done New false
19114 6097 Acked 7096 Write 8 Done Done New false
19115 6098 Acked 7097 Write 2 Done Done New false
19116 6099 Acked 7098 Write 6 Done Done New false
19117 6100 Acked 7099 Read 1 Done Done New false
19118 6101 Acked 7100 Write 2 Done Done New false
19119 6102 Acked 7101 Write 7 Done Done New false
19120 6103 Acked 7102 Read 5 Done Done New false
19121 6104 Acked 7103 Read 6 Done Done New false
19122 6105 Acked 7104 Read 10 Done Done New false
19123 6106 Acked 7105 Flush 0 Done Done New false
19124 6107 Acked 7106 Write 2 Done Done New false
19125 6108 Acked 7107 Write 4 Done Done New false
19126 6109 Acked 7108 Write 4 Done Done New false
19127 6110 Acked 7109 Read 9 Done Done New false
19128 6111 Acked 7110 Read 4 Done Done New false
19129 6112 Acked 7111 Read 9 Done Done New false
19130 6113 Acked 7112 Read 2 Done Done New false
19131 6114 Acked 7113 Write 8 Done Done New false
19132 6115 Acked 7114 Write 9 Done Done New false
19133 6116 Acked 7115 Write 8 Done Done New false
19134 6117 Acked 7116 Write 10 Done Done New false
19135 6118 Acked 7117 Read 7 Done Done New false
19136 6119 Acked 7118 Read 10 Done Done New false
19137 6120 Acked 7119 Read 4 Done Done New false
19138 6121 Acked 7120 Write 2 Done Done New false
19139 6122 Acked 7121 Read 9 Done Done New false
19140 6123 Acked 7122 Read 6 Done Done New false
19141 6124 Acked 7123 Write 2 Done Done New false
19142 6125 Acked 7124 Write 8 Done Done New false
19143 6126 Acked 7125 Write 6 Done Done New false
19144 6127 Acked 7126 Write 4 Done Done New false
19145 6128 Acked 7127 Flush 0 Done Done New false
19146 6129 Acked 7128 Read 10 Done Done New false
19147 6130 Acked 7129 Flush 0 Done Done New false
19148 6131 Acked 7130 Write 7 Done Done New false
19149 6132 Acked 7131 Read 4 Done Done New false
19150 6133 Acked 7132 Read 2 Done Done New false
19151 6134 Acked 7133 Read 4 Done Done New false
19152 6135 Acked 7134 Write 10 Done Done New false
19153 6136 Acked 7135 Read 2 Done Done New false
19154 6137 Acked 7136 Read 10 Done Done New false
19155 6138 Acked 7137 Read 4 Done Done New false
19156 6139 Acked 7138 Read 4 Done Done New false
19157 6140 Acked 7139 Write 5 Done Done New false
19158 6141 Acked 7140 Read 4 Done Done New false
19159 6142 Acked 7141 Write 4 Done Done New false
19160 6143 Acked 7142 Write 1 Done Done New false
19161 6144 Acked 7143 Read 10 Done Done New false
19162 6145 Acked 7144 Write 1 Done Done New false
19163 6146 Acked 7145 Read 2 Done Done New false
19164 6147 Acked 7146 Write 6 Done Done New false
19165 6148 Acked 7147 Read 2 Done Done New false
19166 6149 Acked 7148 Write 2 Done Done New false
19167 6150 Acked 7149 Write 8 Done Done New false
19168 6151 Acked 7150 Write 8 Done Done New false
19169 6152 Acked 7151 Write 9 Done Done New false
19170 6153 Acked 7152 Write 4 Done Done New false
19171 6154 Acked 7153 Read 1 Done Done New false
19172 6155 Acked 7154 Read 6 Done Done New false
19173 6156 Acked 7155 Write 4 Done Done New false
19174 6157 Acked 7156 Write 3 Done Done New false
19175 6158 Acked 7157 Read 3 Done Done New false
19176 6159 Acked 7158 Read 9 Done Done New false
19177 6160 Acked 7159 Read 2 Done Done New false
19178 6161 Acked 7160 Write 5 Done Done New false
19179 6162 Acked 7161 Write 8 Done Done New false
19180 6163 Acked 7162 Read 4 Done Done New false
19181 6164 Acked 7163 Write 7 Done Done New false
19182 6165 Acked 7164 Read 7 Done Done New false
19183 6166 Acked 7165 Flush 0 Done Done New false
19184 6167 Acked 7166 Write 6 Done Done New false
19185 6168 Acked 7167 Write 10 Done Done New false
19186 6169 Acked 7168 Read 6 Done Done New false
19187 6170 Acked 7169 Read 4 Done Done New false
19188 6171 Acked 7170 Read 5 Done Done New false
19189 6172 Acked 7171 Read 5 Done Done New false
19190 6173 Acked 7172 Read 7 Done Done New false
19191 6174 Acked 7173 Read 1 Done Done New false
19192 6175 Acked 7174 Write 8 Done Done New false
19193 6176 Acked 7175 Write 8 Done Done New false
19194 6177 Acked 7176 Write 9 Done Done New false
19195 6178 Acked 7177 Read 6 Done Done New false
19196 6179 Acked 7178 Write 3 Done Done New false
19197 6180 Acked 7179 Write 3 Done Done New false
19198 6181 Acked 7180 Flush 0 Done Done New false
19199 6182 Acked 7181 Read 3 Done Done New false
19200 6183 Acked 7182 Write 7 Done Done New false
19201 6184 Acked 7183 Flush 0 Done Done New false
19202 6185 Acked 7184 Read 7 Done Done New false
19203 6186 Acked 7185 Read 8 Done Done New false
19204 6187 Acked 7186 Read 10 Done Done New false
19205 6188 Acked 7187 Read 10 Done Done New false
19206 6189 Acked 7188 Read 9 Done Done New false
19207 6190 Acked 7189 Read 4 Done Done New false
19208 6191 Acked 7190 Read 1 Done Done New false
19209 6192 Acked 7191 Read 5 Done Done New false
19210 6193 Acked 7192 Write 6 Done Done New false
19211 6194 Acked 7193 Write 2 Done Done New false
19212 6195 Acked 7194 Flush 0 Done Done New false
19213 6196 Acked 7195 Write 1 Done Done New false
19214 6197 Acked 7196 Write 5 Done Done New false
19215 6198 Acked 7197 Write 4 Done Done New false
19216 6199 Acked 7198 Write 1 Done Done New false
19217 6200 Acked 7199 Read 1 Done Done New false
19218 6201 Acked 7200 Write 5 Done Done New false
19219 6202 Acked 7201 Read 2 Done Done New false
19220 6203 Acked 7202 Read 4 Done Done New false
19221 6204 Acked 7203 Read 10 Done Done New false
19222 6205 Acked 7204 Flush 0 Done Done New false
19223 6206 Acked 7205 Read 8 Done Done New false
19224 6207 Acked 7206 Write 8 Done Done New false
19225 6208 Acked 7207 Read 9 Done Done New false
19226 6209 Acked 7208 Flush 0 Done Done New false
19227 6210 Acked 7209 Write 8 Done Done New false
19228 6211 Acked 7210 Write 3 Done Done New false
19229 6212 Acked 7211 Write 5 Done Done New false
19230 6213 Acked 7212 Read 10 Done Done New false
19231 6214 Acked 7213 Read 10 Done Done New false
19232 6215 Acked 7214 Write 2 Done Done New false
19233 6216 Acked 7215 Write 6 Done Done New false
19234 6217 Acked 7216 Write 5 Done Done New false
19235 6218 Acked 7217 Write 7 Done Done New false
19236 6219 Acked 7218 Write 9 Done Done New false
19237 6220 Acked 7219 Read 10 Done Done New false
19238 6221 Acked 7220 Read 1 Done Done New false
19239 6222 Acked 7221 Read 2 Done Done New false
19240 6223 Acked 7222 Write 9 Done Done New false
19241 6224 Acked 7223 Write 1 Done Done New false
19242 6225 Acked 7224 Write 4 Done Done New false
19243 6226 Acked 7225 Flush 0 Done Done New false
19244 6227 Acked 7226 Read 5 Done Done New false
19245 6228 Acked 7227 Flush 0 Done Done New false
19246 6229 Acked 7228 Write 10 Done Done New false
19247 6230 Acked 7229 Write 8 Done Done New false
19248 6231 Acked 7230 Read 2 Done Done New false
19249 6232 Acked 7231 Write 9 Done Done New false
19250 6233 Acked 7232 Read 4 Done Done New false
19251 6234 Acked 7233 Read 9 Done Done New false
19252 6235 Acked 7234 Write 7 Done Done New false
19253 6236 Acked 7235 Write 10 Done Done New false
19254 6237 Acked 7236 Read 7 Done Done New false
19255 6238 Acked 7237 Read 3 Done Done New false
19256 6239 Acked 7238 Flush 0 Done Done New false
19257 6240 Acked 7239 Write 1 Done Done New false
19258 6241 Acked 7240 Write 2 Done Done New false
19259 6242 Acked 7241 Read 6 Done Done New false
19260 6243 Acked 7242 Write 2 Done Done New false
19261 6244 Acked 7243 Read 4 Done Done New false
19262 6245 Acked 7244 Write 9 Done Done New false
19263 6246 Acked 7245 Write 4 Done Done New false
19264 6247 Acked 7246 Flush 0 Done Done New false
19265 6248 Acked 7247 Write 4 Done Done New false
19266 6249 Acked 7248 Read 1 Done Done New false
19267 6250 Acked 7249 Read 4 Done Done New false
19268 6251 Acked 7250 Read 4 Done Done New false
19269 6252 Acked 7251 Read 5 Done Done New false
19270 6253 Acked 7252 Flush 0 Done Done New false
19271 6254 Acked 7253 Read 10 Done Done New false
19272 6255 Acked 7254 Read 2 Done Done New false
19273 6256 Acked 7255 Write 2 Done Done New false
19274 6257 Acked 7256 Read 5 Done Done New false
19275 6258 Acked 7257 Read 4 Done Done New false
19276 6259 Acked 7258 Write 4 Done Done New false
19277 6260 Acked 7259 Write 6 Done Done New false
19278 6261 Acked 7260 Read 5 Done Done New false
19279 6262 Acked 7261 Write 7 Done Done New false
19280 6263 Acked 7262 Read 3 Done Done New false
19281 6264 Acked 7263 Read 10 Done Done New false
19282 6265 Acked 7264 Write 4 Done Done New false
19283 6266 Acked 7265 Flush 0 Done Done New false
19284 6267 Acked 7266 Write 5 Done Done New false
19285 6268 Acked 7267 Write 3 Done Done New false
19286 6269 Acked 7268 Read 9 Done Done New false
19287 6270 Acked 7269 Write 9 Done Done New false
19288 6271 Acked 7270 Write 2 Done Done New false
19289 6272 Acked 7271 Read 9 Done Done New false
19290 6273 Acked 7272 Write 6 Done Done New false
19291 6274 Acked 7273 Read 7 Done Done New false
19292 6275 Acked 7274 Write 10 Done Done New false
19293 6276 Acked 7275 Write 6 Done Done New false
19294 6277 Acked 7276 Read 10 Done Done New false
19295 6278 Acked 7277 Write 4 Done Done New false
19296 6279 Acked 7278 Write 2 Done Done New false
19297 6280 Acked 7279 Read 8 Done Done New false
19298 6281 Acked 7280 Read 3 Done Done New false
19299 6282 Acked 7281 Write 4 Done Done New false
19300 6283 Acked 7282 Flush 0 Done Done New false
19301 6284 Acked 7283 Read 1 Done Done New false
19302 6285 Acked 7284 Read 7 Done Done New false
19303 6286 Acked 7285 Read 7 Done Done New false
19304 6287 Acked 7286 Write 10 Done Done New false
19305 6288 Acked 7287 Flush 0 Done Done New false
19306 6289 Acked 7288 Write 5 Done Done New false
19307 6290 Acked 7289 Write 7 Done Done New false
19308 6291 Acked 7290 Read 10 Done Done New false
19309 6292 Acked 7291 Flush 0 Done Done New false
19310 6293 Acked 7292 Read 3 Done Done New false
19311 6294 Acked 7293 Write 9 Done Done New false
19312 6295 Acked 7294 Read 1 Done Done New false
19313 6296 Acked 7295 Read 4 Done Done New false
19314 6297 Acked 7296 Write 8 Done Done New false
19315 6298 Acked 7297 Write 9 Done Done New false
19316 6299 Acked 7298 Write 9 Done Done New false
19317 6300 Acked 7299 Read 1 Done Done New false
19318 6301 Acked 7300 Read 6 Done Done New false
19319 6302 Acked 7301 Read 2 Done Done New false
19320 6303 Acked 7302 Write 1 Done Done New false
19321 6304 Acked 7303 Read 2 Done Done New false
19322 6305 Acked 7304 Read 4 Done Done New false
19323 6306 Acked 7305 Flush 0 Done Done New false
19324 6307 Acked 7306 Read 10 Done Done New false
19325 6308 Acked 7307 Write 2 Done Done New false
19326 6309 Acked 7308 Write 2 Done Done New false
19327 6310 Acked 7309 Read 9 Done Done New false
19328 6311 Acked 7310 Write 9 Done Done New false
19329 6312 Acked 7311 Write 10 Done Done New false
19330 6313 Acked 7312 Flush 0 Done Done New false
19331 6314 Acked 7313 Write 6 Done Done New false
19332 6315 Acked 7314 Write 5 Done Done New false
19333 6316 Acked 7315 Read 8 Done Done New false
19334 6317 Acked 7316 Read 4 Done Done New false
19335 6318 Acked 7317 Read 8 Done Done New false
19336 6319 Acked 7318 Read 2 Done Done New false
19337 6320 Acked 7319 Read 8 Done Done New false
19338 6321 Acked 7320 Read 10 Done Done New false
19339 6322 Acked 7321 Read 9 Done Done New false
19340 6323 Acked 7322 Read 5 Done Done New false
19341 6324 Acked 7323 Read 1 Done Done New false
19342 6325 Acked 7324 Read 3 Done Done New false
19343 6326 Acked 7325 Read 8 Done Done New false
19344 6327 Acked 7326 Read 4 Done Done New false
19345 6328 Acked 7327 Write 7 Done Done New false
19346 6329 Acked 7328 Read 1 Done Done New false
19347 6330 Acked 7329 Read 6 Done Done New false
19348 6331 Acked 7330 Write 4 Done Done New false
19349 6332 Acked 7331 Write 5 Done Done New false
19350 6333 Acked 7332 Read 2 Done Done New false
19351 6334 Acked 7333 Read 2 Done Done New false
19352 6335 Acked 7334 Write 6 Done Done New false
19353 6336 Acked 7335 Read 10 Done Done New false
19354 6337 Acked 7336 Flush 0 Done Done New false
19355 6338 Acked 7337 Read 2 Done Done New false
19356 6339 Acked 7338 Read 9 Done Done New false
19357 6340 Acked 7339 Write 5 Done Done New false
19358 6341 Acked 7340 Write 7 Done Done New false
19359 6342 Acked 7341 Read 2 Done Done New false
19360 6343 Acked 7342 Flush 0 Done Done New false
19361 6344 Acked 7343 Read 8 Done Done New false
19362 6345 Acked 7344 Write 4 Done Done New false
19363 6346 Acked 7345 Read 4 Done Done New false
19364 6347 Acked 7346 Write 3 Done Done New false
19365 6348 Acked 7347 Read 6 Done Done New false
19366 6349 Acked 7348 Write 2 Done Done New false
19367 6350 Acked 7349 Write 9 Done Done New false
19368 6351 Acked 7350 Write 5 Done Done New false
19369 6352 Acked 7351 Flush 0 Done Done New false
19370 STATES DS:0 DS:1 DS:2 TOTAL
19371 New 0 0 302 302
19372 Sent 0 0 0 0
19373 Done 302 302 0 604
19374 Skipped 0 0 0 0
19375 Error 0 0 0 0
19376 Last Flush: 7351 7351 7049
19377 Downstairs last five completed: 7049 7048 7047 7046 7045 7044
19378 Upstairs last five completed: 6352 6351 6350 6349 6348 6347
19379 CLIENT: Up:0 ds:302 act:2
193802023-09-22T23:27:58.259ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
193812023-09-22T23:27:58.259ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
193822023-09-22T23:27:58.259ZINFOcrucible: [2] client re-new 302 jobs since flush 7049 = downstairs
193832023-09-22T23:27:58.260ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
193842023-09-22T23:27:58.261ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
193852023-09-22T23:27:58.261ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
193862023-09-22T23:27:58.261ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
193872023-09-22T23:27:58.261ZINFOcrucible: [2] send last flush ID to this DS: 7049
193882023-09-22T23:27:58.261ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
193892023-09-22T23:27:58.261ZINFOcrucible: [2] Replied this last flush ID: 7049
193902023-09-22T23:27:58.261ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
193912023-09-22T23:27:58.261ZINFOcrucible: [2] Transition from Offline to Replay
193922023-09-22T23:27:58.261ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
193932023-09-22T23:27:58.261ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
193942023-09-22T23:27:58.261ZINFOcrucible: [2] Transition from Replay to Active
193952023-09-22T23:27:58.261ZINFOcrucible: [2] Starts cmd_loop
193962023-09-22T23:27:58.261ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
193972023-09-22T23:27:58.261ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
193982023-09-22T23:27:58.261ZINFOcrucible: Set check for repair
193992023-09-22T23:27:59.263ZINFOcrucible: Checking if live repair is needed
194002023-09-22T23:27:59.263ZINFOcrucible: No Live Repair required at this time
194012023-09-22T23:27:59.263ZWARNcrucible: [2] flow control end
19402 ----------------------------------------------------------------
19403 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19404 Downstairs last five completed: 7351 7350 7349 7348 7347 7346
19405 Upstairs last five completed: 6352 6351 6350 6349 6348 6347
19406 CLIENT: Up:0 ds:0 act:3
19407 Replay: All jobs finished, all DS active.
19408 Main task has work to do, go find it
19409 got dsc Stop(1)
19410 stop 1
19411 Sep 22 23:28:02.173 INFO request completed, latency_us: 543, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 98b8b51e-f223-43f8-b3a0-e3cea1bb5455, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
19412 [8820] Got stop action so:false kr:false
19413 Sep 22 23:28:02.174 INFO request completed, latency_us: 300, response_code: 200, uri: /state/cid/1, method: GET, req_id: e327b0f5-b435-43df-8d13-e4b354c56247, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
194142023-09-22T23:28:02.174ZWARNcrucible: [1] downstairs disconnected
194152023-09-22T23:28:02.175ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
194162023-09-22T23:28:02.175ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
194172023-09-22T23:28:02.175ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
194182023-09-22T23:28:02.175ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
194192023-09-22T23:28:02.175ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
194202023-09-22T23:28:02.175ZWARNcrucible: [1] pm_task rx.recv() is None
19421 [8820] Exited with: ExitStatus(unix_wait_status(9))
19422 [8820][1] reports Exit
194232023-09-22T23:28:03.176ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
19424 Sep 22 23:28:06.176 INFO request completed, latency_us: 267, response_code: 200, uri: /state/cid/1, method: GET, req_id: 85688f3d-5e87-4b92-a656-7919977c337b, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
19425 001/300 Write block 3249 len 12288 data: 4 4 5
19426 002/300 Read block 1652 len 40960
19427 003/300 Read block 4446 len 24576
19428 004/300 Read block 3736 len 20480
19429 005/300 Write block 1294 len 4096 data: 8
19430 006/300 Read block 3228 len 32768
19431 007/300 Flush
19432 008/300 Write block 3055 len 8192 data: 6 5
19433 009/300 Read block 378 len 12288
19434 010/300 Read block 390 len 12288
19435 011/300 Read block 2584 len 20480
19436 012/300 Flush
19437 013/300 Read block 2913 len 32768
19438 014/300 Read block 889 len 28672
19439 015/300 Read block 2039 len 40960
19440 016/300 Flush
19441 017/300 Write block 1802 len 20480 data: 5 5 4 4 5
19442 018/300 Read block 4273 len 4096
19443 019/300 Write block 1007 len 24576 data: 5 4 3 3 2 2
19444 020/300 Read block 2582 len 40960
19445 021/300 Read block 3600 len 8192
19446 022/300 Write block 144 len 36864 data: 5 5 7 7 8 8 8 6 4
19447 023/300 Read block 3422 len 24576
19448 024/300 Flush
19449 025/300 Flush
19450 026/300 Read block 4057 len 24576
19451 027/300 Read block 3539 len 4096
19452 028/300 Read block 1494 len 8192
19453 029/300 Write block 1692 len 28672 data: 2 2 2 2 3 3 4
19454 030/300 Read block 3240 len 24576
19455 031/300 Read block 1770 len 20480
19456 032/300 Write block 545 len 36864 data: 3 5 5 4 4 3 3 4 5
19457 033/300 Write block 4103 len 20480 data: 7 5 4 5 4
19458 034/300 Read block 2238 len 12288
19459 035/300 Flush
19460 036/300 Write block 4281 len 4096 data: 7
19461 037/300 Read block 3280 len 8192
19462 038/300 Read block 4804 len 16384
19463 039/300 Read block 2145 len 4096
19464 040/300 Read block 3985 len 16384
19465 041/300 Read block 1938 len 24576
19466 042/300 Write block 4959 len 20480 data: 6 4 4 4 4
19467 043/300 Flush
19468 044/300 Write block 2417 len 40960 data: 2 4 4 4 5 5 3 4 5 6
19469 045/300 Write block 4378 len 24576 data: 5 5 5 4 2 2
19470 046/300 Write block 208 len 40960 data: 4 3 2 2 3 3 3 4 4 4
19471 047/300 Flush
19472 048/300 Read block 4629 len 16384
19473 049/300 Write block 3683 len 24576 data: 4 5 6 5 4 3
19474 050/300 Write block 3185 len 36864 data: 8 7 5 3 3 4 5 5 5
19475 051/300 Read block 1506 len 12288
19476 052/300 Read block 4521 len 8192
19477 053/300 Read block 603 len 40960
19478 054/300 Read block 222 len 28672
19479 055/300 Flush
19480 056/300 Read block 2380 len 24576
19481 057/300 Write block 3962 len 8192 data: 4 6
19482 058/300 Write block 757 len 40960 data: 6 6 7 7 8 8 7 6 6 6
19483 059/300 Write block 3409 len 32768 data: 5 3 3 2 2 2 3 4
19484 060/300 Read block 2250 len 40960
19485 061/300 Read block 1777 len 28672
19486 062/300 Write block 3875 len 32768 data: 6 5 4 3 3 3 3 3
19487 063/300 Read block 860 len 32768
19488 064/300 Read block 4673 len 20480
19489 065/300 Read block 3962 len 36864
19490 066/300 Write block 1338 len 40960 data: 6 6 6 6 6 5 5 5 5 5
19491 067/300 Flush
19492 068/300 Read block 4069 len 16384
19493 069/300 Read block 617 len 12288
19494 070/300 Write block 4782 len 28672 data: 5 5 5 5 5 7 7
19495 071/300 Read block 2385 len 28672
19496 072/300 Write block 3783 len 28672 data: 5 5 4 3 3 3 3
19497 073/300 Flush
19498 074/300 Write block 1711 len 12288 data: 6 6 6
19499 075/300 Read block 4449 len 16384
19500 076/300 Read block 3886 len 16384
19501 077/300 Write block 4097 len 40960 data: 8 8 7 7 7 6 8 6 5 6
19502 078/300 Write block 3162 len 8192 data: 4 5
19503 079/300 Write block 567 len 32768 data: 2 2 3 4 5 5 5 5
19504 080/300 Flush
19505 081/300 Flush
19506 082/300 Read block 1464 len 4096
19507 083/300 Read block 2213 len 16384
19508 084/300 Write block 765 len 32768 data: 7 7 5 6 7 7 7 6
19509 085/300 Read block 2977 len 28672
19510 086/300 Write block 2259 len 4096 data: 5
19511 087/300 Write block 2270 len 12288 data: 6 6 6
19512 088/300 Read block 1834 len 20480
19513 089/300 Read block 666 len 40960
19514 090/300 Write block 4930 len 32768 data: 5 5 5 4 5 4 4 4
19515 091/300 Read block 1073 len 8192
19516 092/300 Flush
19517 093/300 Read block 598 len 32768
19518 094/300 Read block 110 len 4096
19519 095/300 Read block 875 len 16384
19520 096/300 Flush
19521 097/300 Write block 2853 len 40960 data: 6 6 6 6 6 5 7 7 8 6
19522 098/300 Write block 3538 len 16384 data: 4 4 4 4
19523 099/300 Write block 4017 len 24576 data: 5 5 6 5 5 4
19524 100/300 Flush
19525 101/300 Write block 3518 len 32768 data: 4 4 4 5 5 5 4 2
19526 102/300 Write block 4133 len 4096 data: 5
19527 103/300 Write block 1432 len 12288 data: 8 8 6
19528 104/300 Read block 786 len 20480
19529 105/300 Flush
19530 106/300 Read block 4421 len 28672
19531 107/300 Read block 4598 len 24576
19532 108/300 Flush
19533 109/300 Read block 1173 len 16384
19534 110/300 Flush
19535 111/300 Write block 3660 len 16384 data: 3 3 3 3
19536 112/300 Read block 1828 len 8192
19537 113/300 Read block 1420 len 28672
19538 114/300 Write block 1332 len 4096 data: 5
19539 115/300 Read block 2908 len 36864
19540 116/300 Write block 3587 len 8192 data: 3 3
19541 117/300 Write block 1434 len 40960 data: 7 7 6 5 4 5 2 3 3 4
19542 118/300 Write block 4694 len 28672 data: 2 3 5 4 4 4 4
19543 119/300 Write block 3533 len 16384 data: 3 3 3 4
19544 120/300 Write block 2288 len 12288 data: 5 5 5
19545 121/300 Read block 2094 len 16384
19546 122/300 Read block 3889 len 4096
19547 123/300 Read block 1485 len 8192
19548 124/300 Read block 1756 len 20480
19549 125/300 Read block 2065 len 28672
19550 126/300 Write block 1130 len 28672 data: 3 4 4 3 2 2 3
19551 127/300 Read block 1997 len 20480
19552 128/300 Write block 2650 len 4096 data: 4
19553 129/300 Read block 632 len 8192
19554 130/300 Read block 2811 len 28672
19555 131/300 Write block 595 len 24576 data: 6 7 7 7 7 6
19556 132/300 Write block 4288 len 36864 data: 5 5 5 5 6 6 7 5 3
19557 133/300 Read block 277 len 4096
19558 134/300 Read block 1071 len 36864
19559 135/300 Read block 3056 len 4096
19560 136/300 Read block 2752 len 16384
19561 137/300 Write block 1254 len 36864 data: 3 3 2 2 2 2 2 2 3
19562 138/300 Read block 4669 len 24576
19563 139/300 Write block 2670 len 36864 data: 6 6 5 4 3 4 4 4 4
19564 140/300 Read block 2028 len 40960
19565 141/300 Write block 2026 len 32768 data: 5 5 6 6 6 5 5 4
19566 142/300 Read block 2250 len 16384
19567 143/300 Read block 727 len 4096
19568 144/300 Read block 4833 len 16384
19569 145/300 Read block 1041 len 8192
19570 146/300 Read block 1031 len 20480
19571 147/300 Read block 2387 len 32768
19572 148/300 Write block 1656 len 24576 data: 9 7 7 5 3 3
19573 149/300 Read block 2669 len 24576
19574 150/300 Write block 4845 len 36864 data: 5 5 6 7 7 8 7 7 8
19575 151/300 Read block 1300 len 4096
19576 152/300 Write block 2997 len 32768 data: 3 3 4 4 4 4 4 4
19577 153/300 Write block 1684 len 24576 data: 8 6 6 6 5 5
19578 154/300 Read block 3091 len 32768
19579 155/300 Read block 333 len 32768
19580 156/300 Write block 741 len 4096 data: 5
19581 157/300 Read block 2079 len 28672
19582 158/300 Read block 316 len 36864
19583 159/300 Write block 1104 len 20480 data: 3 3 3 3 3
19584 160/300 Read block 3181 len 24576
19585 161/300 Read block 4048 len 24576
19586 162/300 Write block 1542 len 12288 data: 5 6 4
19587 163/300 Write block 4170 len 12288 data: 6 5 6
19588 164/300 Read block 1654 len 16384
19589 165/300 Read block 3797 len 36864
19590 166/300 Write block 2162 len 32768 data: 7 6 5 3 3 3 3 3
19591 167/300 Write block 771 len 4096 data: 8
19592 168/300 Read block 3993 len 12288
19593 169/300 Read block 2546 len 36864
19594 170/300 Read block 4745 len 12288
19595 171/300 Read block 3691 len 24576
19596 172/300 Read block 2808 len 28672
19597 173/300 Flush
19598 174/300 Write block 2283 len 16384 data: 7 6 6 6
19599 175/300 Read block 3568 len 28672
19600 176/300 Write block 1478 len 32768 data: 11 11 10 8 7 4 5 6
19601 177/300 Write block 4590 len 28672 data: 4 3 4 3 3 3 3
19602 178/300 Read block 614 len 12288
19603 179/300 Flush
19604 180/300 Write block 485 len 4096 data: 9
19605 181/300 Read block 55 len 28672
19606 182/300 Read block 2559 len 36864
19607 183/300 Write block 4220 len 8192 data: 4 4
19608 184/300 Read block 3001 len 24576
19609 185/300 Read block 2775 len 16384
19610 186/300 Write block 4385 len 32768 data: 6 5 5 5 5 5 5 4
19611 187/300 Read block 4164 len 40960
19612 188/300 Write block 3531 len 32768 data: 3 3 4 4 4 5 4 5
19613 189/300 Write block 1460 len 8192 data: 5 5
19614 190/300 Read block 2367 len 8192
19615 191/300 Write block 4505 len 12288 data: 4 4 4
19616 192/300 Write block 3777 len 16384 data: 4 5 5 4
19617 193/300 Read block 1787 len 8192
19618 194/300 Write block 569 len 8192 data: 4 5
19619 195/300 Read block 4666 len 12288
19620 196/300 Read block 1759 len 4096
19621 197/300 Write block 1461 len 12288 data: 6 5 5
19622 198/300 Flush
19623 199/300 Read block 2408 len 36864
19624 200/300 Write block 3196 len 12288 data: 4 4 2
19625 201/300 Read block 3014 len 28672
19626 202/300 Read block 622 len 20480
19627 203/300 Read block 1682 len 12288
19628 204/300 Read block 3121 len 36864
19629 205/300 Read block 2427 len 8192
19630 206/300 Read block 4611 len 40960
19631 207/300 Read block 3473 len 40960
19632 208/300 Read block 41 len 8192
19633 209/300 Read block 2722 len 20480
19634 210/300 Write block 800 len 4096 data: 5
19635 211/300 Write block 4093 len 20480 data: 4 5 6 6 9
19636 212/300 Read block 3488 len 36864
19637 213/300 Read block 1032 len 20480
19638 214/300 Write block 2606 len 20480 data: 5 5 5 4 4
19639 215/300 Read block 4597 len 4096
19640 216/300 Flush
19641 217/300 Write block 4842 len 16384 data: 4 4 3 6
19642 218/300 Read block 4574 len 40960
19643 219/300 Flush
19644 220/300 Read block 879 len 12288
19645 221/300 Read block 3584 len 16384
19646 222/300 Flush
19647 223/300 Read block 3326 len 24576
19648 224/300 Write block 58 len 16384 data: 8 8 8 8
19649 225/300 Write block 675 len 40960 data: 5 5 3 3 4 4 4 4 4 5
19650 226/300 Read block 4950 len 20480
19651 227/300 Write block 4173 len 36864 data: 6 6 4 3 3 3 3 4 3
19652 228/300 Read block 929 len 16384
19653 229/300 Read block 1048 len 12288
19654 230/300 Read block 3866 len 4096
19655 231/300 Flush
19656 232/300 Write block 4426 len 24576 data: 8 6 6 5 5 6
19657 233/300 Write block 2874 len 8192 data: 4 5
19658 234/300 Write block 4186 len 8192 data: 4 4
19659 235/300 Read block 574 len 20480
19660 236/300 Read block 112 len 36864
19661 237/300 Read block 546 len 36864
19662 238/300 Write block 4531 len 32768 data: 5 6 5 4 4 3 3 4
19663 239/300 Write block 700 len 8192 data: 7 7
19664 240/300 Write block 4632 len 4096 data: 3
19665 241/300 Read block 4710 len 36864
19666 242/300 Read block 2112 len 32768
19667 243/300 Write block 1497 len 24576 data: 8 7 7 8 7 5
19668 244/300 Write block 609 len 32768 data: 5 3 2 2 2 2 2 2
19669 245/300 Write block 4471 len 36864 data: 3 4 5 5 5 6 5 5 5
19670 246/300 Flush
19671 247/300 Flush
19672 248/300 Read block 1684 len 12288
19673 249/300 Write block 886 len 20480 data: 6 5 4 4 4
19674 250/300 Write block 449 len 32768 data: 3 3 3 3 3 2 2 2
19675 251/300 Write block 376 len 16384 data: 5 5 5 5
19676 252/300 Read block 1363 len 20480
19677 253/300 Write block 4330 len 12288 data: 6 6 6
19678 254/300 Read block 4263 len 40960
19679 255/300 Read block 2122 len 4096
19680 256/300 Read block 2470 len 24576
19681 257/300 Read block 1235 len 24576
19682 258/300 Write block 1609 len 24576 data: 5 4 3 3 3 3
19683 259/300 Write block 3682 len 8192 data: 4 5
19684 260/300 Read block 1060 len 8192
19685 261/300 Read block 4022 len 40960
19686 262/300 Read block 3953 len 20480
19687 263/300 Read block 1449 len 24576
19688 264/300 Write block 4337 len 4096 data: 5
19689 265/300 Read block 2638 len 36864
19690 266/300 Read block 2952 len 40960
19691 267/300 Read block 355 len 8192
19692 268/300 Read block 281 len 36864
19693 269/300 Write block 226 len 28672 data: 4 5 5 5 5 4 2
19694 270/300 Read block 2630 len 4096
19695 271/300 Write block 3645 len 4096 data: 3
19696 272/300 Write block 3988 len 28672 data: 8 6 6 5 4 4 5
19697 273/300 Read block 3355 len 20480
19698 274/300 Read block 3800 len 12288
19699 275/300 Flush
19700 276/300 Read block 845 len 40960
19701 277/300 Read block 1251 len 28672
19702 278/300 Write block 3721 len 4096 data: 6
19703 279/300 Read block 4005 len 16384
19704 280/300 Read block 1696 len 8192
19705 281/300 Flush
19706 282/300 Write block 12 len 36864 data: 7 6 3 3 5 6 6 7 7
19707 283/300 Write block 3815 len 24576 data: 2 2 3 4 4 4
19708 284/300 Flush
19709 285/300 Write block 1289 len 40960 data: 4 7 6 7 8 9 8 9 9 9
19710 286/300 Write block 4803 len 36864 data: 6 6 6 7 6 7 7 6 5
19711 287/300 Read block 1316 len 40960
19712 288/300 Read block 1986 len 4096
19713 289/300 Read block 67 len 20480
19714 290/300 Read block 518 len 36864
19715 291/300 Read block 1168 len 36864
19716 292/300 Write block 3457 len 12288 data: 6 7 7
19717 293/300 Read block 1644 len 24576
19718 294/300 Read block 1116 len 12288
19719 295/300 Write block 4750 len 32768 data: 4 4 4 4 4 4 6 6
19720 296/300 Read block 1305 len 28672
19721 297/300 Flush
19722 298/300 Write block 3496 len 28672 data: 6 5 6 6 6 5 6
19723 299/300 Write block 109 len 20480 data: 7 7 7 8 7
19724 300/300 Flush
19725 Main task has work to do, go find it
19726 got dsc Start(1)
19727 start 1
19728 Sep 22 23:28:07.103 INFO request completed, latency_us: 309, response_code: 204, uri: /start/cid/1, method: POST, req_id: 5fe51250-6057-45de-9170-1644f7f71597, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
19729 Replay: started 1, returned:Ok(())
19730 ----------------------------------------------------------------
19731 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19732 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
19733 6353 Acked 7352 Write 3 Done New Done false
19734 6354 Acked 7353 Read 10 Done New Done false
19735 6355 Acked 7354 Read 6 Done New Done false
19736 6356 Acked 7355 Read 5 Done New Done false
19737 6357 Acked 7356 Write 1 Done New Done false
19738 6358 Acked 7357 Read 8 Done New Done false
19739 6359 Acked 7358 Flush 0 Done New Done false
19740 6360 Acked 7359 Write 2 Done New Done false
19741 6361 Acked 7360 Read 3 Done New Done false
19742 6362 Acked 7361 Read 3 Done New Done false
19743 6363 Acked 7362 Read 5 Done New Done false
19744 6364 Acked 7363 Flush 0 Done New Done false
19745 6365 Acked 7364 Read 8 Done New Done false
19746 6366 Acked 7365 Read 7 Done New Done false
19747 6367 Acked 7366 Read 10 Done New Done false
19748 6368 Acked 7367 Flush 0 Done New Done false
19749 6369 Acked 7368 Write 5 Done New Done false
19750 6370 Acked 7369 Read 1 Done New Done false
19751 6371 Acked 7370 Write 6 Done New Done false
19752 6372 Acked 7371 Read 10 Done New Done false
19753 6373 Acked 7372 Read 2 Done New Done false
19754 6374 Acked 7373 Write 9 Done New Done false
19755 6375 Acked 7374 Read 6 Done New Done false
19756 6376 Acked 7375 Flush 0 Done New Done false
19757 6377 Acked 7376 Flush 0 Done New Done false
19758 6378 Acked 7377 Read 6 Done New Done false
19759 6379 Acked 7378 Read 1 Done New Done false
19760 6380 Acked 7379 Read 2 Done New Done false
19761 6381 Acked 7380 Write 7 Done New Done false
19762 6382 Acked 7381 Read 6 Done New Done false
19763 6383 Acked 7382 Read 5 Done New Done false
19764 6384 Acked 7383 Write 9 Done New Done false
19765 6385 Acked 7384 Write 5 Done New Done false
19766 6386 Acked 7385 Read 3 Done New Done false
19767 6387 Acked 7386 Flush 0 Done New Done false
19768 6388 Acked 7387 Write 1 Done New Done false
19769 6389 Acked 7388 Read 2 Done New Done false
19770 6390 Acked 7389 Read 4 Done New Done false
19771 6391 Acked 7390 Read 1 Done New Done false
19772 6392 Acked 7391 Read 4 Done New Done false
19773 6393 Acked 7392 Read 6 Done New Done false
19774 6394 Acked 7393 Write 5 Done New Done false
19775 6395 Acked 7394 Flush 0 Done New Done false
19776 6396 Acked 7395 Write 10 Done New Done false
19777 6397 Acked 7396 Write 6 Done New Done false
19778 6398 Acked 7397 Write 10 Done New Done false
19779 6399 Acked 7398 Flush 0 Done New Done false
19780 6400 Acked 7399 Read 4 Done New Done false
19781 6401 Acked 7400 Write 6 Done New Done false
19782 6402 Acked 7401 Write 9 Done New Done false
19783 6403 Acked 7402 Read 3 Done New Done false
19784 6404 Acked 7403 Read 2 Done New Done false
19785 6405 Acked 7404 Read 10 Done New Done false
19786 6406 Acked 7405 Read 7 Done New Done false
19787 6407 Acked 7406 Flush 0 Done New Done false
19788 6408 Acked 7407 Read 6 Done New Done false
19789 6409 Acked 7408 Write 2 Done New Done false
19790 6410 Acked 7409 Write 10 Done New Done false
19791 6411 Acked 7410 Write 8 Done New Done false
19792 6412 Acked 7411 Read 10 Done New Done false
19793 6413 Acked 7412 Read 7 Done New Done false
19794 6414 Acked 7413 Write 8 Done New Done false
19795 6415 Acked 7414 Read 8 Done New Done false
19796 6416 Acked 7415 Read 5 Done New Done false
19797 6417 Acked 7416 Read 9 Done New Done false
19798 6418 Acked 7417 Write 10 Done New Done false
19799 6419 Acked 7418 Flush 0 Done New Done false
19800 6420 Acked 7419 Read 4 Done New Done false
19801 6421 Acked 7420 Read 3 Done New Done false
19802 6422 Acked 7421 Write 7 Done New Done false
19803 6423 Acked 7422 Read 7 Done New Done false
19804 6424 Acked 7423 Write 7 Done New Done false
19805 6425 Acked 7424 Flush 0 Done New Done false
19806 6426 Acked 7425 Write 3 Done New Done false
19807 6427 Acked 7426 Read 4 Done New Done false
19808 6428 Acked 7427 Read 4 Done New Done false
19809 6429 Acked 7428 Write 10 Done New Done false
19810 6430 Acked 7429 Write 2 Done New Done false
19811 6431 Acked 7430 Write 8 Done New Done false
19812 6432 Acked 7431 Flush 0 Done New Done false
19813 6433 Acked 7432 Flush 0 Done New Done false
19814 6434 Acked 7433 Read 1 Done New Done false
19815 6435 Acked 7434 Read 4 Done New Done false
19816 6436 Acked 7435 Write 8 Done New Done false
19817 6437 Acked 7436 Read 7 Done New Done false
19818 6438 Acked 7437 Write 1 Done New Done false
19819 6439 Acked 7438 Write 3 Done New Done false
19820 6440 Acked 7439 Read 5 Done New Done false
19821 6441 Acked 7440 Read 10 Done New Done false
19822 6442 Acked 7441 Write 8 Done New Done false
19823 6443 Acked 7442 Read 2 Done New Done false
19824 6444 Acked 7443 Flush 0 Done New Done false
19825 6445 Acked 7444 Read 8 Done New Done false
19826 6446 Acked 7445 Read 1 Done New Done false
19827 6447 Acked 7446 Read 4 Done New Done false
19828 6448 Acked 7447 Flush 0 Done New Done false
19829 6449 Acked 7448 Write 10 Done New Done false
19830 6450 Acked 7449 Write 4 Done New Done false
19831 6451 Acked 7450 Write 6 Done New Done false
19832 6452 Acked 7451 Flush 0 Done New Done false
19833 6453 Acked 7452 Write 8 Done New Done false
19834 6454 Acked 7453 Write 1 Done New Done false
19835 6455 Acked 7454 Write 3 Done New Done false
19836 6456 Acked 7455 Read 5 Done New Done false
19837 6457 Acked 7456 Flush 0 Done New Done false
19838 6458 Acked 7457 Read 7 Done New Done false
19839 6459 Acked 7458 Read 6 Done New Done false
19840 6460 Acked 7459 Flush 0 Done New Done false
19841 6461 Acked 7460 Read 4 Done New Done false
19842 6462 Acked 7461 Flush 0 Done New Done false
19843 6463 Acked 7462 Write 4 Done New Done false
19844 6464 Acked 7463 Read 2 Done New Done false
19845 6465 Acked 7464 Read 7 Done New Done false
19846 6466 Acked 7465 Write 1 Done New Done false
19847 6467 Acked 7466 Read 9 Done New Done false
19848 6468 Acked 7467 Write 2 Done New Done false
19849 6469 Acked 7468 Write 10 Done New Done false
19850 6470 Acked 7469 Flush 0 Done New Done false
19851 6471 Acked 7470 Write 7 Done New Done false
19852 6472 Acked 7471 Write 4 Done New Done false
19853 6473 Acked 7472 Write 3 Done New Done false
19854 6474 Acked 7473 Read 4 Done New Done false
19855 6475 Acked 7474 Read 1 Done New Done false
19856 6476 Acked 7475 Read 2 Done New Done false
19857 6477 Acked 7476 Read 5 Done New Done false
19858 6478 Acked 7477 Read 7 Done New Done false
19859 6479 Acked 7478 Write 7 Done New Done false
19860 6480 Acked 7479 Read 5 Done New Done false
19861 6481 Acked 7480 Write 1 Done New Done false
19862 6482 Acked 7481 Read 2 Done New Done false
19863 6483 Acked 7482 Read 7 Done New Done false
19864 6484 Acked 7483 Write 6 Done New Done false
19865 6485 Acked 7484 Write 9 Done New Done false
19866 6486 Acked 7485 Read 1 Done New Done false
19867 6487 Acked 7486 Read 9 Done New Done false
19868 6488 Acked 7487 Read 1 Done New Done false
19869 6489 Acked 7488 Read 4 Done New Done false
19870 6490 Acked 7489 Write 9 Done New Done false
19871 6491 Acked 7490 Read 6 Done New Done false
19872 6492 Acked 7491 Write 9 Done New Done false
19873 6493 Acked 7492 Read 10 Done New Done false
19874 6494 Acked 7493 Write 8 Done New Done false
19875 6495 Acked 7494 Read 4 Done New Done false
19876 6496 Acked 7495 Read 1 Done New Done false
19877 6497 Acked 7496 Read 4 Done New Done false
19878 6498 Acked 7497 Read 2 Done New Done false
19879 6499 Acked 7498 Read 5 Done New Done false
19880 6500 Acked 7499 Read 8 Done New Done false
19881 6501 Acked 7500 Write 6 Done New Done false
19882 6502 Acked 7501 Read 6 Done New Done false
19883 6503 Acked 7502 Write 9 Done New Done false
19884 6504 Acked 7503 Read 1 Done New Done false
19885 6505 Acked 7504 Write 8 Done New Done false
19886 6506 Acked 7505 Write 6 Done New Done false
19887 6507 Acked 7506 Read 8 Done New Done false
19888 6508 Acked 7507 Read 8 Done New Done false
19889 6509 Acked 7508 Write 1 Done New Done false
19890 6510 Acked 7509 Read 7 Done New Done false
19891 6511 Acked 7510 Read 9 Done New Done false
19892 6512 Acked 7511 Write 5 Done New Done false
19893 6513 Acked 7512 Read 6 Done New Done false
19894 6514 Acked 7513 Read 6 Done New Done false
19895 6515 Acked 7514 Write 3 Done New Done false
19896 6516 Acked 7515 Write 3 Done New Done false
19897 6517 Acked 7516 Read 4 Done New Done false
19898 6518 Acked 7517 Read 9 Done New Done false
19899 6519 Acked 7518 Write 8 Done New Done false
19900 6520 Acked 7519 Write 1 Done New Done false
19901 6521 Acked 7520 Read 3 Done New Done false
19902 6522 Acked 7521 Read 9 Done New Done false
19903 6523 Acked 7522 Read 3 Done New Done false
19904 6524 Acked 7523 Read 6 Done New Done false
19905 6525 Acked 7524 Read 7 Done New Done false
19906 6526 Acked 7525 Flush 0 Done New Done false
19907 6527 Acked 7526 Write 4 Done New Done false
19908 6528 Acked 7527 Read 7 Done New Done false
19909 6529 Acked 7528 Write 8 Done New Done false
19910 6530 Acked 7529 Write 7 Done New Done false
19911 6531 Acked 7530 Read 3 Done New Done false
19912 6532 Acked 7531 Flush 0 Done New Done false
19913 6533 Acked 7532 Write 1 Done New Done false
19914 6534 Acked 7533 Read 7 Done New Done false
19915 6535 Acked 7534 Read 9 Done New Done false
19916 6536 Acked 7535 Write 2 Done New Done false
19917 6537 Acked 7536 Read 6 Done New Done false
19918 6538 Acked 7537 Read 4 Done New Done false
19919 6539 Acked 7538 Write 8 Done New Done false
19920 6540 Acked 7539 Read 10 Done New Done false
19921 6541 Acked 7540 Write 8 Done New Done false
19922 6542 Acked 7541 Write 2 Done New Done false
19923 6543 Acked 7542 Read 2 Done New Done false
19924 6544 Acked 7543 Write 3 Done New Done false
19925 6545 Acked 7544 Write 4 Done New Done false
19926 6546 Acked 7545 Read 2 Done New Done false
19927 6547 Acked 7546 Write 2 Done New Done false
19928 6548 Acked 7547 Read 3 Done New Done false
19929 6549 Acked 7548 Read 1 Done New Done false
19930 6550 Acked 7549 Write 3 Done New Done false
19931 6551 Acked 7550 Flush 0 Done New Done false
19932 6552 Acked 7551 Read 9 Done New Done false
19933 6553 Acked 7552 Write 3 Done New Done false
19934 6554 Acked 7553 Read 7 Done New Done false
19935 6555 Acked 7554 Read 5 Done New Done false
19936 6556 Acked 7555 Read 3 Done New Done false
19937 6557 Acked 7556 Read 9 Done New Done false
19938 6558 Acked 7557 Read 2 Done New Done false
19939 6559 Acked 7558 Read 10 Done New Done false
19940 6560 Acked 7559 Read 10 Done New Done false
19941 6561 Acked 7560 Read 2 Done New Done false
19942 6562 Acked 7561 Read 5 Done New Done false
19943 6563 Acked 7562 Write 1 Done New Done false
19944 6564 Acked 7563 Write 5 Done New Done false
19945 6565 Acked 7564 Read 9 Done New Done false
19946 6566 Acked 7565 Read 5 Done New Done false
19947 6567 Acked 7566 Write 5 Done New Done false
19948 6568 Acked 7567 Read 1 Done New Done false
19949 6569 Acked 7568 Flush 0 Done New Done false
19950 6570 Acked 7569 Write 4 Done New Done false
19951 6571 Acked 7570 Read 10 Done New Done false
19952 6572 Acked 7571 Flush 0 Done New Done false
19953 6573 Acked 7572 Read 3 Done New Done false
19954 6574 Acked 7573 Read 4 Done New Done false
19955 6575 Acked 7574 Flush 0 Done New Done false
19956 6576 Acked 7575 Read 6 Done New Done false
19957 6577 Acked 7576 Write 4 Done New Done false
19958 6578 Acked 7577 Write 10 Done New Done false
19959 6579 Acked 7578 Read 5 Done New Done false
19960 6580 Acked 7579 Write 9 Done New Done false
19961 6581 Acked 7580 Read 4 Done New Done false
19962 6582 Acked 7581 Read 3 Done New Done false
19963 6583 Acked 7582 Read 1 Done New Done false
19964 6584 Acked 7583 Flush 0 Done New Done false
19965 6585 Acked 7584 Write 6 Done New Done false
19966 6586 Acked 7585 Write 2 Done New Done false
19967 6587 Acked 7586 Write 2 Done New Done false
19968 6588 Acked 7587 Read 5 Done New Done false
19969 6589 Acked 7588 Read 9 Done New Done false
19970 6590 Acked 7589 Read 9 Done New Done false
19971 6591 Acked 7590 Write 8 Done New Done false
19972 6592 Acked 7591 Write 2 Done New Done false
19973 6593 Acked 7592 Write 1 Done New Done false
19974 6594 Acked 7593 Read 9 Done New Done false
19975 6595 Acked 7594 Read 8 Done New Done false
19976 6596 Acked 7595 Write 6 Done New Done false
19977 6597 Acked 7596 Write 8 Done New Done false
19978 6598 Acked 7597 Write 9 Done New Done false
19979 6599 Acked 7598 Flush 0 Done New Done false
19980 6600 Acked 7599 Flush 0 Done New Done false
19981 6601 Acked 7600 Read 3 Done New Done false
19982 6602 Acked 7601 Write 5 Done New Done false
19983 6603 Acked 7602 Write 8 Done New Done false
19984 6604 Acked 7603 Write 4 Done New Done false
19985 6605 Acked 7604 Read 5 Done New Done false
19986 6606 Acked 7605 Write 3 Done New Done false
19987 6607 Acked 7606 Read 10 Done New Done false
19988 6608 Acked 7607 Read 1 Done New Done false
19989 6609 Acked 7608 Read 6 Done New Done false
19990 6610 Acked 7609 Read 6 Done New Done false
19991 6611 Acked 7610 Write 6 Done New Done false
19992 6612 Acked 7611 Write 2 Done New Done false
19993 6613 Acked 7612 Read 2 Done New Done false
19994 6614 Acked 7613 Read 10 Done New Done false
19995 6615 Acked 7614 Read 5 Done New Done false
19996 6616 Acked 7615 Read 6 Done New Done false
19997 6617 Acked 7616 Write 1 Done New Done false
19998 6618 Acked 7617 Read 9 Done New Done false
19999 6619 Acked 7618 Read 10 Done New Done false
20000 6620 Acked 7619 Read 2 Done New Done false
20001 6621 Acked 7620 Read 9 Done New Done false
20002 6622 Acked 7621 Write 7 Done New Done false
20003 6623 Acked 7622 Read 1 Done New Done false
20004 6624 Acked 7623 Write 1 Done New Done false
20005 6625 Acked 7624 Write 7 Done New Done false
20006 6626 Acked 7625 Read 5 Done New Done false
20007 6627 Acked 7626 Read 3 Done New Done false
20008 6628 Acked 7627 Flush 0 Done New Done false
20009 6629 Acked 7628 Read 10 Done New Done false
20010 6630 Acked 7629 Read 7 Done New Done false
20011 6631 Acked 7630 Flush 0 Done New Done false
20012 6632 Acked 7631 Write 1 Done New Done false
20013 6633 Acked 7632 Read 4 Done New Done false
20014 6634 Acked 7633 Read 2 Done New Done false
20015 6635 Acked 7634 Flush 0 Done New Done false
20016 6636 Acked 7635 Write 9 Done New Done false
20017 6637 Acked 7636 Write 6 Done New Done false
20018 6638 Acked 7637 Flush 0 Done New Done false
20019 6639 Acked 7638 Write 10 Done New Done false
20020 6640 Acked 7639 Write 9 Done New Done false
20021 6641 Acked 7640 Read 10 Done New Done false
20022 6642 Acked 7641 Read 1 Done New Done false
20023 6643 Acked 7642 Read 5 Done New Done false
20024 6644 Acked 7643 Read 9 Done New Done false
20025 6645 Acked 7644 Read 9 Done New Done false
20026 6646 Acked 7645 Write 3 Done New Done false
20027 6647 Acked 7646 Read 6 Done New Done false
20028 6648 Acked 7647 Read 3 Done New Done false
20029 6649 Acked 7648 Write 8 Done New Done false
20030 6650 Acked 7649 Read 7 Done New Done false
20031 6651 Acked 7650 Flush 0 Done New Done false
20032 6652 Acked 7651 Write 7 Done New Done false
20033 6653 Acked 7652 Write 5 Done New Done false
20034 6654 Acked 7653 Flush 0 Done New Done false
20035 STATES DS:0 DS:1 DS:2 TOTAL
20036 New 0 302 0 302
20037 Sent 0 0 0 0
20038 Done 302 0 302 604
20039 Skipped 0 0 0 0
20040 Error 0 0 0 0
20041 Last Flush: 7653 7351 7653
20042 Downstairs last five completed: 7351 7350 7349 7348 7347 7346
20043 Upstairs last five completed: 6654 6653 6652 6651 6650 6649
20044 CLIENT: Up:0 ds:302 act:2
20045 [8820] got start action
20046 [8820] I am going to restart
20047 Starting downstairs at port 8820
20048 Make output file at "/tmp/dsc/downstairs-8820.txt"
20049 [8820][1] reports Starting
20050 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1195)
20051 [8820][1] reports Running
20052 ----------------------------------------------------------------
20053 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20054 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
20055 6353 Acked 7352 Write 3 Done New Done false
20056 6354 Acked 7353 Read 10 Done New Done false
20057 6355 Acked 7354 Read 6 Done New Done false
20058 6356 Acked 7355 Read 5 Done New Done false
20059 6357 Acked 7356 Write 1 Done New Done false
20060 6358 Acked 7357 Read 8 Done New Done false
20061 6359 Acked 7358 Flush 0 Done New Done false
20062 6360 Acked 7359 Write 2 Done New Done false
20063 6361 Acked 7360 Read 3 Done New Done false
20064 6362 Acked 7361 Read 3 Done New Done false
20065 6363 Acked 7362 Read 5 Done New Done false
20066 6364 Acked 7363 Flush 0 Done New Done false
20067 6365 Acked 7364 Read 8 Done New Done false
20068 6366 Acked 7365 Read 7 Done New Done false
20069 6367 Acked 7366 Read 10 Done New Done false
20070 6368 Acked 7367 Flush 0 Done New Done false
20071 6369 Acked 7368 Write 5 Done New Done false
20072 6370 Acked 7369 Read 1 Done New Done false
20073 6371 Acked 7370 Write 6 Done New Done false
20074 6372 Acked 7371 Read 10 Done New Done false
20075 6373 Acked 7372 Read 2 Done New Done false
20076 6374 Acked 7373 Write 9 Done New Done false
20077 6375 Acked 7374 Read 6 Done New Done false
20078 6376 Acked 7375 Flush 0 Done New Done false
20079 6377 Acked 7376 Flush 0 Done New Done false
20080 6378 Acked 7377 Read 6 Done New Done false
20081 6379 Acked 7378 Read 1 Done New Done false
20082 6380 Acked 7379 Read 2 Done New Done false
20083 6381 Acked 7380 Write 7 Done New Done false
20084 6382 Acked 7381 Read 6 Done New Done false
20085 6383 Acked 7382 Read 5 Done New Done false
20086 6384 Acked 7383 Write 9 Done New Done false
20087 6385 Acked 7384 Write 5 Done New Done false
20088 6386 Acked 7385 Read 3 Done New Done false
20089 6387 Acked 7386 Flush 0 Done New Done false
20090 6388 Acked 7387 Write 1 Done New Done false
20091 6389 Acked 7388 Read 2 Done New Done false
20092 6390 Acked 7389 Read 4 Done New Done false
20093 6391 Acked 7390 Read 1 Done New Done false
20094 6392 Acked 7391 Read 4 Done New Done false
20095 6393 Acked 7392 Read 6 Done New Done false
20096 6394 Acked 7393 Write 5 Done New Done false
20097 6395 Acked 7394 Flush 0 Done New Done false
20098 6396 Acked 7395 Write 10 Done New Done false
20099 6397 Acked 7396 Write 6 Done New Done false
20100 6398 Acked 7397 Write 10 Done New Done false
20101 6399 Acked 7398 Flush 0 Done New Done false
20102 6400 Acked 7399 Read 4 Done New Done false
20103 6401 Acked 7400 Write 6 Done New Done false
20104 6402 Acked 7401 Write 9 Done New Done false
20105 6403 Acked 7402 Read 3 Done New Done false
20106 6404 Acked 7403 Read 2 Done New Done false
20107 6405 Acked 7404 Read 10 Done New Done false
20108 6406 Acked 7405 Read 7 Done New Done false
20109 6407 Acked 7406 Flush 0 Done New Done false
20110 6408 Acked 7407 Read 6 Done New Done false
20111 6409 Acked 7408 Write 2 Done New Done false
20112 6410 Acked 7409 Write 10 Done New Done false
20113 6411 Acked 7410 Write 8 Done New Done false
20114 6412 Acked 7411 Read 10 Done New Done false
20115 6413 Acked 7412 Read 7 Done New Done false
20116 6414 Acked 7413 Write 8 Done New Done false
20117 6415 Acked 7414 Read 8 Done New Done false
20118 6416 Acked 7415 Read 5 Done New Done false
20119 6417 Acked 7416 Read 9 Done New Done false
20120 6418 Acked 7417 Write 10 Done New Done false
20121 6419 Acked 7418 Flush 0 Done New Done false
20122 6420 Acked 7419 Read 4 Done New Done false
20123 6421 Acked 7420 Read 3 Done New Done false
20124 6422 Acked 7421 Write 7 Done New Done false
20125 6423 Acked 7422 Read 7 Done New Done false
20126 6424 Acked 7423 Write 7 Done New Done false
20127 6425 Acked 7424 Flush 0 Done New Done false
20128 6426 Acked 7425 Write 3 Done New Done false
20129 6427 Acked 7426 Read 4 Done New Done false
20130 6428 Acked 7427 Read 4 Done New Done false
20131 6429 Acked 7428 Write 10 Done New Done false
20132 6430 Acked 7429 Write 2 Done New Done false
20133 6431 Acked 7430 Write 8 Done New Done false
20134 6432 Acked 7431 Flush 0 Done New Done false
20135 6433 Acked 7432 Flush 0 Done New Done false
20136 6434 Acked 7433 Read 1 Done New Done false
20137 6435 Acked 7434 Read 4 Done New Done false
20138 6436 Acked 7435 Write 8 Done New Done false
20139 6437 Acked 7436 Read 7 Done New Done false
20140 6438 Acked 7437 Write 1 Done New Done false
20141 6439 Acked 7438 Write 3 Done New Done false
20142 6440 Acked 7439 Read 5 Done New Done false
20143 6441 Acked 7440 Read 10 Done New Done false
20144 6442 Acked 7441 Write 8 Done New Done false
20145 6443 Acked 7442 Read 2 Done New Done false
20146 6444 Acked 7443 Flush 0 Done New Done false
20147 6445 Acked 7444 Read 8 Done New Done false
20148 6446 Acked 7445 Read 1 Done New Done false
20149 6447 Acked 7446 Read 4 Done New Done false
20150 6448 Acked 7447 Flush 0 Done New Done false
20151 6449 Acked 7448 Write 10 Done New Done false
20152 6450 Acked 7449 Write 4 Done New Done false
20153 6451 Acked 7450 Write 6 Done New Done false
20154 6452 Acked 7451 Flush 0 Done New Done false
20155 6453 Acked 7452 Write 8 Done New Done false
20156 6454 Acked 7453 Write 1 Done New Done false
20157 6455 Acked 7454 Write 3 Done New Done false
20158 6456 Acked 7455 Read 5 Done New Done false
20159 6457 Acked 7456 Flush 0 Done New Done false
20160 6458 Acked 7457 Read 7 Done New Done false
20161 6459 Acked 7458 Read 6 Done New Done false
20162 6460 Acked 7459 Flush 0 Done New Done false
20163 6461 Acked 7460 Read 4 Done New Done false
20164 6462 Acked 7461 Flush 0 Done New Done false
20165 6463 Acked 7462 Write 4 Done New Done false
20166 6464 Acked 7463 Read 2 Done New Done false
20167 6465 Acked 7464 Read 7 Done New Done false
20168 6466 Acked 7465 Write 1 Done New Done false
20169 6467 Acked 7466 Read 9 Done New Done false
20170 6468 Acked 7467 Write 2 Done New Done false
20171 6469 Acked 7468 Write 10 Done New Done false
20172 6470 Acked 7469 Flush 0 Done New Done false
20173 6471 Acked 7470 Write 7 Done New Done false
20174 6472 Acked 7471 Write 4 Done New Done false
20175 6473 Acked 7472 Write 3 Done New Done false
20176 6474 Acked 7473 Read 4 Done New Done false
20177 6475 Acked 7474 Read 1 Done New Done false
20178 6476 Acked 7475 Read 2 Done New Done false
20179 6477 Acked 7476 Read 5 Done New Done false
20180 6478 Acked 7477 Read 7 Done New Done false
20181 6479 Acked 7478 Write 7 Done New Done false
20182 6480 Acked 7479 Read 5 Done New Done false
20183 6481 Acked 7480 Write 1 Done New Done false
20184 6482 Acked 7481 Read 2 Done New Done false
20185 6483 Acked 7482 Read 7 Done New Done false
20186 6484 Acked 7483 Write 6 Done New Done false
20187 6485 Acked 7484 Write 9 Done New Done false
20188 6486 Acked 7485 Read 1 Done New Done false
20189 6487 Acked 7486 Read 9 Done New Done false
20190 6488 Acked 7487 Read 1 Done New Done false
20191 6489 Acked 7488 Read 4 Done New Done false
20192 6490 Acked 7489 Write 9 Done New Done false
20193 6491 Acked 7490 Read 6 Done New Done false
20194 6492 Acked 7491 Write 9 Done New Done false
20195 6493 Acked 7492 Read 10 Done New Done false
20196 6494 Acked 7493 Write 8 Done New Done false
20197 6495 Acked 7494 Read 4 Done New Done false
20198 6496 Acked 7495 Read 1 Done New Done false
20199 6497 Acked 7496 Read 4 Done New Done false
20200 6498 Acked 7497 Read 2 Done New Done false
20201 6499 Acked 7498 Read 5 Done New Done false
20202 6500 Acked 7499 Read 8 Done New Done false
20203 6501 Acked 7500 Write 6 Done New Done false
20204 6502 Acked 7501 Read 6 Done New Done false
20205 6503 Acked 7502 Write 9 Done New Done false
20206 6504 Acked 7503 Read 1 Done New Done false
20207 6505 Acked 7504 Write 8 Done New Done false
20208 6506 Acked 7505 Write 6 Done New Done false
20209 6507 Acked 7506 Read 8 Done New Done false
20210 6508 Acked 7507 Read 8 Done New Done false
20211 6509 Acked 7508 Write 1 Done New Done false
20212 6510 Acked 7509 Read 7 Done New Done false
20213 6511 Acked 7510 Read 9 Done New Done false
20214 6512 Acked 7511 Write 5 Done New Done false
20215 6513 Acked 7512 Read 6 Done New Done false
20216 6514 Acked 7513 Read 6 Done New Done false
20217 6515 Acked 7514 Write 3 Done New Done false
20218 6516 Acked 7515 Write 3 Done New Done false
20219 6517 Acked 7516 Read 4 Done New Done false
20220 6518 Acked 7517 Read 9 Done New Done false
20221 6519 Acked 7518 Write 8 Done New Done false
20222 6520 Acked 7519 Write 1 Done New Done false
20223 6521 Acked 7520 Read 3 Done New Done false
20224 6522 Acked 7521 Read 9 Done New Done false
20225 6523 Acked 7522 Read 3 Done New Done false
20226 6524 Acked 7523 Read 6 Done New Done false
20227 6525 Acked 7524 Read 7 Done New Done false
20228 6526 Acked 7525 Flush 0 Done New Done false
20229 6527 Acked 7526 Write 4 Done New Done false
20230 6528 Acked 7527 Read 7 Done New Done false
20231 6529 Acked 7528 Write 8 Done New Done false
20232 6530 Acked 7529 Write 7 Done New Done false
20233 6531 Acked 7530 Read 3 Done New Done false
20234 6532 Acked 7531 Flush 0 Done New Done false
20235 6533 Acked 7532 Write 1 Done New Done false
20236 6534 Acked 7533 Read 7 Done New Done false
20237 6535 Acked 7534 Read 9 Done New Done false
20238 6536 Acked 7535 Write 2 Done New Done false
20239 6537 Acked 7536 Read 6 Done New Done false
20240 6538 Acked 7537 Read 4 Done New Done false
20241 6539 Acked 7538 Write 8 Done New Done false
20242 6540 Acked 7539 Read 10 Done New Done false
20243 6541 Acked 7540 Write 8 Done New Done false
20244 6542 Acked 7541 Write 2 Done New Done false
20245 6543 Acked 7542 Read 2 Done New Done false
20246 6544 Acked 7543 Write 3 Done New Done false
20247 6545 Acked 7544 Write 4 Done New Done false
20248 6546 Acked 7545 Read 2 Done New Done false
20249 6547 Acked 7546 Write 2 Done New Done false
20250 6548 Acked 7547 Read 3 Done New Done false
20251 6549 Acked 7548 Read 1 Done New Done false
20252 6550 Acked 7549 Write 3 Done New Done false
20253 6551 Acked 7550 Flush 0 Done New Done false
20254 6552 Acked 7551 Read 9 Done New Done false
20255 6553 Acked 7552 Write 3 Done New Done false
20256 6554 Acked 7553 Read 7 Done New Done false
20257 6555 Acked 7554 Read 5 Done New Done false
20258 6556 Acked 7555 Read 3 Done New Done false
20259 6557 Acked 7556 Read 9 Done New Done false
20260 6558 Acked 7557 Read 2 Done New Done false
20261 6559 Acked 7558 Read 10 Done New Done false
20262 6560 Acked 7559 Read 10 Done New Done false
20263 6561 Acked 7560 Read 2 Done New Done false
20264 6562 Acked 7561 Read 5 Done New Done false
20265 6563 Acked 7562 Write 1 Done New Done false
20266 6564 Acked 7563 Write 5 Done New Done false
20267 6565 Acked 7564 Read 9 Done New Done false
20268 6566 Acked 7565 Read 5 Done New Done false
20269 6567 Acked 7566 Write 5 Done New Done false
20270 6568 Acked 7567 Read 1 Done New Done false
20271 6569 Acked 7568 Flush 0 Done New Done false
20272 6570 Acked 7569 Write 4 Done New Done false
20273 6571 Acked 7570 Read 10 Done New Done false
20274 6572 Acked 7571 Flush 0 Done New Done false
20275 6573 Acked 7572 Read 3 Done New Done false
20276 6574 Acked 7573 Read 4 Done New Done false
20277 6575 Acked 7574 Flush 0 Done New Done false
20278 6576 Acked 7575 Read 6 Done New Done false
20279 6577 Acked 7576 Write 4 Done New Done false
20280 6578 Acked 7577 Write 10 Done New Done false
20281 6579 Acked 7578 Read 5 Done New Done false
20282 6580 Acked 7579 Write 9 Done New Done false
20283 6581 Acked 7580 Read 4 Done New Done false
20284 6582 Acked 7581 Read 3 Done New Done false
20285 6583 Acked 7582 Read 1 Done New Done false
20286 6584 Acked 7583 Flush 0 Done New Done false
20287 6585 Acked 7584 Write 6 Done New Done false
20288 6586 Acked 7585 Write 2 Done New Done false
20289 6587 Acked 7586 Write 2 Done New Done false
20290 6588 Acked 7587 Read 5 Done New Done false
20291 6589 Acked 7588 Read 9 Done New Done false
20292 6590 Acked 7589 Read 9 Done New Done false
20293 6591 Acked 7590 Write 8 Done New Done false
20294 6592 Acked 7591 Write 2 Done New Done false
20295 6593 Acked 7592 Write 1 Done New Done false
20296 6594 Acked 7593 Read 9 Done New Done false
20297 6595 Acked 7594 Read 8 Done New Done false
20298 6596 Acked 7595 Write 6 Done New Done false
20299 6597 Acked 7596 Write 8 Done New Done false
20300 6598 Acked 7597 Write 9 Done New Done false
20301 6599 Acked 7598 Flush 0 Done New Done false
20302 6600 Acked 7599 Flush 0 Done New Done false
20303 6601 Acked 7600 Read 3 Done New Done false
20304 6602 Acked 7601 Write 5 Done New Done false
20305 6603 Acked 7602 Write 8 Done New Done false
20306 6604 Acked 7603 Write 4 Done New Done false
20307 6605 Acked 7604 Read 5 Done New Done false
20308 6606 Acked 7605 Write 3 Done New Done false
20309 6607 Acked 7606 Read 10 Done New Done false
20310 6608 Acked 7607 Read 1 Done New Done false
20311 6609 Acked 7608 Read 6 Done New Done false
20312 6610 Acked 7609 Read 6 Done New Done false
20313 6611 Acked 7610 Write 6 Done New Done false
20314 6612 Acked 7611 Write 2 Done New Done false
20315 6613 Acked 7612 Read 2 Done New Done false
20316 6614 Acked 7613 Read 10 Done New Done false
20317 6615 Acked 7614 Read 5 Done New Done false
20318 6616 Acked 7615 Read 6 Done New Done false
20319 6617 Acked 7616 Write 1 Done New Done false
20320 6618 Acked 7617 Read 9 Done New Done false
20321 6619 Acked 7618 Read 10 Done New Done false
20322 6620 Acked 7619 Read 2 Done New Done false
20323 6621 Acked 7620 Read 9 Done New Done false
20324 6622 Acked 7621 Write 7 Done New Done false
20325 6623 Acked 7622 Read 1 Done New Done false
20326 6624 Acked 7623 Write 1 Done New Done false
20327 6625 Acked 7624 Write 7 Done New Done false
20328 6626 Acked 7625 Read 5 Done New Done false
20329 6627 Acked 7626 Read 3 Done New Done false
20330 6628 Acked 7627 Flush 0 Done New Done false
20331 6629 Acked 7628 Read 10 Done New Done false
20332 6630 Acked 7629 Read 7 Done New Done false
20333 6631 Acked 7630 Flush 0 Done New Done false
20334 6632 Acked 7631 Write 1 Done New Done false
20335 6633 Acked 7632 Read 4 Done New Done false
20336 6634 Acked 7633 Read 2 Done New Done false
20337 6635 Acked 7634 Flush 0 Done New Done false
20338 6636 Acked 7635 Write 9 Done New Done false
20339 6637 Acked 7636 Write 6 Done New Done false
20340 6638 Acked 7637 Flush 0 Done New Done false
20341 6639 Acked 7638 Write 10 Done New Done false
20342 6640 Acked 7639 Write 9 Done New Done false
20343 6641 Acked 7640 Read 10 Done New Done false
20344 6642 Acked 7641 Read 1 Done New Done false
20345 6643 Acked 7642 Read 5 Done New Done false
20346 6644 Acked 7643 Read 9 Done New Done false
20347 6645 Acked 7644 Read 9 Done New Done false
20348 6646 Acked 7645 Write 3 Done New Done false
20349 6647 Acked 7646 Read 6 Done New Done false
20350 6648 Acked 7647 Read 3 Done New Done false
20351 6649 Acked 7648 Write 8 Done New Done false
20352 6650 Acked 7649 Read 7 Done New Done false
20353 6651 Acked 7650 Flush 0 Done New Done false
20354 6652 Acked 7651 Write 7 Done New Done false
20355 6653 Acked 7652 Write 5 Done New Done false
20356 6654 Acked 7653 Flush 0 Done New Done false
20357 STATES DS:0 DS:1 DS:2 TOTAL
20358 New 0 302 0 302
20359 Sent 0 0 0 0
20360 Done 302 0 302 604
20361 Skipped 0 0 0 0
20362 Error 0 0 0 0
20363 Last Flush: 7653 7351 7653
20364 Downstairs last five completed: 7351 7350 7349 7348 7347 7346
20365 Upstairs last five completed: 6654 6653 6652 6651 6650 6649
20366 CLIENT: Up:0 ds:302 act:2
203672023-09-22T23:28:11.185ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
203682023-09-22T23:28:11.185ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
203692023-09-22T23:28:11.185ZINFOcrucible: [1] client re-new 302 jobs since flush 7351 = downstairs
203702023-09-22T23:28:11.186ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
203712023-09-22T23:28:11.187ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
203722023-09-22T23:28:11.187ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
203732023-09-22T23:28:11.187ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
203742023-09-22T23:28:11.187ZINFOcrucible: [1] send last flush ID to this DS: 7351
203752023-09-22T23:28:11.187ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
203762023-09-22T23:28:11.187ZINFOcrucible: [1] Replied this last flush ID: 7351
203772023-09-22T23:28:11.187ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
203782023-09-22T23:28:11.187ZINFOcrucible: [1] Transition from Offline to Replay
203792023-09-22T23:28:11.187ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
203802023-09-22T23:28:11.187ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
203812023-09-22T23:28:11.187ZINFOcrucible: [1] Transition from Replay to Active
203822023-09-22T23:28:11.187ZINFOcrucible: [1] Starts cmd_loop
203832023-09-22T23:28:11.187ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
203842023-09-22T23:28:11.187ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
203852023-09-22T23:28:11.187ZINFOcrucible: Set check for repair
203862023-09-22T23:28:12.189ZINFOcrucible: Checking if live repair is needed
203872023-09-22T23:28:12.189ZINFOcrucible: No Live Repair required at this time
203882023-09-22T23:28:12.189ZWARNcrucible: [1] flow control end
20389 ----------------------------------------------------------------
20390 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20391 Downstairs last five completed: 7653 7652 7651 7650 7649 7648
20392 Upstairs last five completed: 6654 6653 6652 6651 6650 6649
20393 CLIENT: Up:0 ds:0 act:3
20394 Replay: All jobs finished, all DS active.
20395 Main task has work to do, go find it
20396 got dsc Stop(0)
20397 stop 0
20398 Sep 22 23:28:15.110 INFO request completed, latency_us: 568, response_code: 204, uri: /stop/cid/0, method: POST, req_id: cf48623a-443a-45ae-8174-c5f3ab0c944e, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
20399 [8810] Got stop action so:false kr:false
20400 Sep 22 23:28:15.111 INFO request completed, latency_us: 236, response_code: 200, uri: /state/cid/0, method: GET, req_id: eb7b2a52-8cad-44ff-8b06-8138834d181b, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
204012023-09-22T23:28:15.112ZWARNcrucible: [0] downstairs disconnected
204022023-09-22T23:28:15.112ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
204032023-09-22T23:28:15.112ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
204042023-09-22T23:28:15.112ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
204052023-09-22T23:28:15.112ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
204062023-09-22T23:28:15.112ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
204072023-09-22T23:28:15.112ZWARNcrucible: [0] pm_task rx.recv() is None
20408 [8810] Exited with: ExitStatus(unix_wait_status(9))
20409 [8810][0] reports Exit
20410 Sep 22 23:28:19.114 INFO request completed, latency_us: 262, response_code: 200, uri: /state/cid/0, method: GET, req_id: 46c00dbe-62ea-4964-ab6e-b5ae19724e97, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
20411 001/300 Read block 4992 len 16384
20412 002/300 Read block 2495 len 28672
20413 003/300 Write block 4420 len 12288 data: 8 8 7
20414 004/300 Write block 1247 len 32768 data: 2 2 3 4 4 4 3 4
20415 005/300 Flush
20416 006/300 Write block 1757 len 36864 data: 6 6 7 7 6 5 5 6 6
20417 007/300 Read block 4074 len 24576
20418 008/300 Write block 145 len 32768 data: 6 8 8 9 9 9 7 5
20419 009/300 Read block 2092 len 40960
20420 010/300 Read block 2532 len 24576
20421 011/300 Write block 4825 len 4096 data: 7
20422 012/300 Read block 2319 len 4096
20423 013/300 Read block 487 len 32768
20424 014/300 Read block 359 len 8192
20425 015/300 Read block 3537 len 28672
20426 016/300 Write block 51 len 12288 data: 6 6 5
20427 017/300 Flush
20428 018/300 Read block 3526 len 16384
20429 019/300 Read block 311 len 16384
20430 020/300 Write block 4944 len 16384 data: 6 5 6 6
20431 021/300 Read block 2587 len 4096
20432 022/300 Flush
20433 023/300 Write block 4959 len 24576 data: 7 5 5 5 5 5
20434 024/300 Write block 1333 len 32768 data: 5 5 6 7 7 7 7 7
20435 025/300 Read block 1183 len 24576
20436 026/300 Read block 1613 len 20480
20437 027/300 Read block 3345 len 36864
20438 028/300 Read block 1204 len 28672
20439 029/300 Read block 2128 len 12288
20440 030/300 Read block 739 len 36864
20441 031/300 Write block 2677 len 32768 data: 5 5 4 5 5 4 4 5
20442 032/300 Read block 1944 len 4096
20443 033/300 Read block 3620 len 12288
20444 034/300 Write block 959 len 28672 data: 4 5 5 6 6 5 5
20445 035/300 Read block 1120 len 12288
20446 036/300 Write block 494 len 16384 data: 6 5 4 5
20447 037/300 Read block 4895 len 4096
20448 038/300 Write block 4440 len 16384 data: 4 4 4 4
20449 039/300 Read block 2364 len 32768
20450 040/300 Write block 4177 len 16384 data: 4 4 4 5
20451 041/300 Read block 1905 len 16384
20452 042/300 Write block 4171 len 28672 data: 6 7 7 7 5 4 5
20453 043/300 Write block 3541 len 12288 data: 5 4 5
20454 044/300 Read block 2551 len 36864
20455 045/300 Write block 876 len 24576 data: 9 7 5 5 6 7
20456 046/300 Read block 654 len 8192
20457 047/300 Write block 919 len 28672 data: 3 3 3 3 3 4 5
20458 048/300 Write block 3489 len 16384 data: 5 5 4 3
20459 049/300 Read block 4257 len 40960
20460 050/300 Write block 4701 len 4096 data: 4
20461 051/300 Read block 3627 len 36864
20462 052/300 Read block 3430 len 20480
20463 053/300 Write block 2927 len 28672 data: 3 4 4 3 3 2 2
20464 054/300 Write block 4236 len 12288 data: 4 3 3
20465 055/300 Write block 1368 len 28672 data: 5 5 4 5 5 5 6
20466 056/300 Read block 2499 len 36864
20467 057/300 Read block 2450 len 28672
20468 058/300 Read block 4444 len 16384
20469 059/300 Read block 998 len 20480
20470 060/300 Write block 4765 len 16384 data: 6 5 6 4
20471 061/300 Write block 3269 len 32768 data: 3 3 5 4 4 4 4 4
20472 062/300 Write block 4319 len 16384 data: 5 5 6 8
20473 063/300 Flush
20474 064/300 Read block 2806 len 4096
20475 065/300 Read block 1430 len 32768
20476 066/300 Read block 4233 len 20480
20477 067/300 Flush
20478 068/300 Read block 2218 len 16384
20479 069/300 Write block 977 len 16384 data: 5 4 4 4
20480 070/300 Write block 3342 len 28672 data: 7 6 6 6 6 4 2
20481 071/300 Flush
20482 072/300 Read block 2484 len 36864
20483 073/300 Write block 3484 len 12288 data: 3 2 3
20484 074/300 Read block 2350 len 16384
20485 075/300 Read block 319 len 36864
20486 076/300 Read block 124 len 20480
20487 077/300 Read block 4007 len 16384
20488 078/300 Write block 2037 len 12288 data: 3 3 2
20489 079/300 Read block 1691 len 24576
20490 080/300 Write block 2228 len 16384 data: 4 4 4 4
20491 081/300 Write block 809 len 36864 data: 8 8 6 7 9 8 8 7 7
20492 082/300 Read block 3383 len 28672
20493 083/300 Read block 1302 len 36864
20494 084/300 Read block 540 len 28672
20495 085/300 Read block 3209 len 20480
20496 086/300 Write block 1001 len 36864 data: 5 7 7 4 4 5 6 5 4
20497 087/300 Write block 4245 len 16384 data: 7 8 9 8
20498 088/300 Read block 159 len 24576
20499 089/300 Read block 157 len 4096
20500 090/300 Write block 1017 len 40960 data: 4 4 4 4 4 4 5 4 5 5
20501 091/300 Read block 4185 len 4096
20502 092/300 Read block 4974 len 28672
20503 093/300 Read block 2897 len 12288
20504 094/300 Write block 4655 len 40960 data: 5 7 7 7 7 6 5 6 5 4
20505 095/300 Flush
20506 096/300 Flush
20507 097/300 Read block 3272 len 32768
20508 098/300 Flush
20509 099/300 Write block 3329 len 36864 data: 5 3 3 3 3 3 4 4 5
20510 100/300 Write block 503 len 36864 data: 8 4 4 4 4 4 3 3 2
20511 101/300 Flush
20512 102/300 Read block 3010 len 12288
20513 103/300 Write block 3296 len 16384 data: 5 6 5 5
20514 104/300 Write block 4719 len 8192 data: 3 4
20515 105/300 Write block 1289 len 20480 data: 5 8 7 8 9
20516 106/300 Read block 3401 len 28672
20517 107/300 Write block 465 len 32768 data: 6 4 4 5 5 6 6 5
20518 108/300 Read block 2241 len 12288
20519 109/300 Read block 4644 len 8192
20520 110/300 Write block 4838 len 20480 data: 4 3 3 3 5
20521 111/300 Write block 4721 len 8192 data: 4 4
20522 112/300 Read block 52 len 12288
20523 113/300 Write block 1293 len 20480 data: 10 10 9 10 10
20524 114/300 Read block 2251 len 4096
20525 115/300 Read block 1055 len 28672
20526 116/300 Flush
20527 117/300 Read block 4754 len 20480
20528 118/300 Read block 2500 len 32768
20529 119/300 Flush
20530 120/300 Read block 3887 len 8192
20531 121/300 Read block 4624 len 32768
20532 122/300 Write block 182 len 24576 data: 4 5 5 5 7 7
20533 123/300 Write block 2608 len 12288 data: 6 5 5
20534 124/300 Write block 2959 len 20480 data: 3 3 3 3 3
20535 125/300 Read block 4046 len 24576
20536 126/300 Write block 2787 len 16384 data: 6 5 5 6
20537 127/300 Read block 2602 len 16384
20538 128/300 Write block 1096 len 20480 data: 2 3 3 3 3
20539 129/300 Read block 4215 len 8192
20540 130/300 Write block 2013 len 16384 data: 2 3 3 3
20541 131/300 Write block 1031 len 24576 data: 3 4 4 3 4 5
20542 132/300 Write block 1286 len 20480 data: 3 3 3 6 9
20543 133/300 Read block 733 len 28672
20544 134/300 Read block 1756 len 16384
20545 135/300 Write block 3143 len 24576 data: 5 5 5 6 6 5
20546 136/300 Read block 1456 len 20480
20547 137/300 Flush
20548 138/300 Read block 2827 len 32768
20549 139/300 Write block 2848 len 8192 data: 4 5
20550 140/300 Write block 2973 len 28672 data: 5 4 6 6 5 4 4
20551 141/300 Flush
20552 142/300 Flush
20553 143/300 Flush
20554 144/300 Write block 628 len 28672 data: 7 6 6 6 5 5 5
20555 145/300 Write block 369 len 28672 data: 7 6 5 3 3 3 5
20556 146/300 Read block 4493 len 12288
20557 147/300 Write block 2731 len 32768 data: 2 4 4 4 4 5 3 2
20558 148/300 Write block 2400 len 8192 data: 3 3
20559 149/300 Flush
20560 150/300 Flush
20561 151/300 Read block 2300 len 20480
20562 152/300 Read block 2688 len 4096
20563 153/300 Read block 1081 len 8192
20564 154/300 Write block 3546 len 28672 data: 3 4 3 4 4 5 5
20565 155/300 Read block 3658 len 28672
20566 156/300 Flush
20567 157/300 Read block 2384 len 32768
20568 158/300 Flush
20569 159/300 Write block 2199 len 40960 data: 7 6 6 6 5 5 5 4 3 3
20570 160/300 Write block 4853 len 32768 data: 9 7 6 6 6 5 6 5
20571 161/300 Read block 3428 len 20480
20572 162/300 Write block 1899 len 12288 data: 7 6 7
20573 163/300 Flush
20574 164/300 Write block 2796 len 20480 data: 6 6 7 7 7
20575 165/300 Read block 4689 len 8192
20576 166/300 Read block 4782 len 16384
20577 167/300 Write block 4790 len 12288 data: 4 4 4
20578 168/300 Read block 964 len 24576
20579 169/300 Read block 3012 len 32768
20580 170/300 Flush
20581 171/300 Read block 2898 len 12288
20582 172/300 Write block 2988 len 40960 data: 2 2 2 2 2 3 3 3 3 4
20583 173/300 Write block 4390 len 24576 data: 6 6 5 4 5 5
20584 174/300 Write block 3703 len 24576 data: 8 8 8 8 8 6
20585 175/300 Write block 2270 len 12288 data: 7 7 7
20586 176/300 Read block 3367 len 32768
20587 177/300 Read block 2038 len 16384
20588 178/300 Read block 2493 len 24576
20589 179/300 Write block 2269 len 8192 data: 4 8
20590 180/300 Write block 273 len 28672 data: 6 5 5 5 2 2 2
20591 181/300 Write block 907 len 24576 data: 2 2 3 3 3 3
20592 182/300 Read block 440 len 40960
20593 183/300 Flush
20594 184/300 Write block 3221 len 8192 data: 4 4
20595 185/300 Read block 1168 len 16384
20596 186/300 Write block 1076 len 8192 data: 3 3
20597 187/300 Read block 4975 len 12288
20598 188/300 Read block 2046 len 8192
20599 189/300 Read block 4118 len 20480
20600 190/300 Write block 2142 len 4096 data: 5
20601 191/300 Write block 1149 len 8192 data: 3 3
20602 192/300 Read block 1840 len 16384
20603 193/300 Write block 778 len 40960 data: 5 4 3 4 4 5 5 5 4 3
20604 194/300 Read block 2674 len 28672
20605 195/300 Write block 4270 len 16384 data: 3 4 4 5
20606 196/300 Read block 1275 len 36864
20607 197/300 Write block 4986 len 32768 data: 8 9 10 9 8 8 8 5
20608 198/300 Read block 1415 len 24576
20609 199/300 Write block 4547 len 28672 data: 3 4 6 5 4 4 4
20610 200/300 Read block 1259 len 28672
20611 201/300 Write block 4441 len 28672 data: 5 5 5 5 4 4 4
20612 202/300 Read block 2541 len 32768
20613 203/300 Write block 123 len 16384 data: 3 3 3 5
20614 204/300 Flush
20615 205/300 Read block 2899 len 12288
20616 206/300 Write block 4346 len 20480 data: 3 4 6 5 5
20617 207/300 Flush
20618 208/300 Write block 3049 len 36864 data: 5 5 4 5 6 5 7 6 3
20619 209/300 Write block 2697 len 20480 data: 5 4 5 5 5
20620 210/300 Read block 3110 len 12288
20621 211/300 Write block 3089 len 4096 data: 4
20622 212/300 Write block 3060 len 24576 data: 2 2 2 3 3 3
20623 213/300 Read block 3231 len 20480
20624 214/300 Read block 1905 len 40960
20625 215/300 Read block 3610 len 16384
20626 216/300 Read block 3684 len 16384
20627 217/300 Write block 2194 len 32768 data: 4 6 7 7 9 8 7 7
20628 218/300 Write block 561 len 24576 data: 5 4 3 3 3 2
20629 219/300 Write block 2694 len 8192 data: 6 5
20630 220/300 Write block 1875 len 16384 data: 6 5 5 7
20631 221/300 Write block 2280 len 8192 data: 7 7
20632 222/300 Read block 2411 len 36864
20633 223/300 Write block 498 len 4096 data: 5
20634 224/300 Write block 1815 len 28672 data: 5 6 6 5 5 4 4
20635 225/300 Write block 1197 len 40960 data: 4 4 3 4 4 4 5 7 8 9
20636 226/300 Read block 224 len 16384
20637 227/300 Write block 1356 len 32768 data: 4 5 5 6 6 7 4 6
20638 228/300 Write block 4992 len 12288 data: 9 6 4
20639 229/300 Write block 412 len 40960 data: 7 7 7 8 8 8 8 7 6 5
20640 230/300 Read block 1128 len 20480
20641 231/300 Read block 3944 len 40960
20642 232/300 Read block 3248 len 8192
20643 233/300 Read block 1013 len 24576
20644 234/300 Write block 999 len 28672 data: 7 8 6 8 8 5 5
20645 235/300 Read block 4012 len 32768
20646 236/300 Flush
20647 237/300 Write block 1836 len 20480 data: 4 4 4 4 4
20648 238/300 Flush
20649 239/300 Flush
20650 240/300 Read block 2797 len 32768
20651 241/300 Read block 4185 len 12288
20652 242/300 Read block 4957 len 8192
20653 243/300 Flush
20654 244/300 Write block 1940 len 40960 data: 4 4 6 7 6 6 4 4 5 7
20655 245/300 Write block 829 len 40960 data: 4 3 4 4 4 4 5 5 6 7
20656 246/300 Read block 2197 len 16384
20657 247/300 Read block 310 len 8192
20658 248/300 Flush
20659 249/300 Write block 2661 len 24576 data: 4 5 3 4 5 5
20660 250/300 Read block 2427 len 36864
20661 251/300 Write block 3291 len 8192 data: 6 6
20662 252/300 Write block 3842 len 20480 data: 4 4 4 2 3
20663 253/300 Write block 3455 len 4096 data: 3
20664 254/300 Flush
20665 255/300 Write block 1575 len 12288 data: 5 3 2
20666 256/300 Read block 146 len 20480
20667 257/300 Write block 4236 len 12288 data: 5 4 4
20668 258/300 Write block 4169 len 24576 data: 6 7 7 8 8 8
20669 259/300 Read block 3036 len 24576
20670 260/300 Read block 1865 len 36864
20671 261/300 Write block 650 len 32768 data: 2 3 3 3 3 4 4 4
20672 262/300 Read block 2642 len 28672
20673 263/300 Read block 3683 len 40960
20674 264/300 Read block 2600 len 8192
20675 265/300 Read block 722 len 28672
20676 266/300 Read block 2306 len 36864
20677 267/300 Flush
20678 268/300 Write block 4459 len 40960 data: 7 5 5 3 3 4 4 4 4 4
20679 269/300 Write block 1144 len 16384 data: 3 4 4 3
20680 270/300 Write block 1719 len 8192 data: 2 2
20681 271/300 Read block 4702 len 12288
20682 272/300 Read block 2566 len 24576
20683 273/300 Write block 4113 len 40960 data: 4 2 2 2 3 3 2 2 2 4
20684 274/300 Read block 3627 len 16384
20685 275/300 Write block 2032 len 20480 data: 6 5 5 6 5
20686 276/300 Read block 3038 len 36864
20687 277/300 Read block 2027 len 16384
20688 278/300 Write block 1950 len 32768 data: 6 8 8 6 7 9 8 8
20689 279/300 Read block 4778 len 12288
20690 280/300 Read block 4263 len 36864
20691 281/300 Write block 192 len 12288 data: 4 4 5
20692 282/300 Write block 2657 len 20480 data: 4 5 4 4 5
20693 283/300 Read block 2088 len 40960
20694 284/300 Write block 2083 len 16384 data: 4 4 2 3
20695 285/300 Write block 4541 len 40960 data: 4 4 4 4 4 3 4 5 7 6
20696 286/300 Read block 981 len 20480
20697 287/300 Read block 4495 len 40960
20698 288/300 Read block 3118 len 20480
20699 289/300 Read block 1908 len 24576
20700 290/300 Read block 3281 len 36864
20701 291/300 Read block 570 len 4096
20702 292/300 Read block 2165 len 36864
20703 293/300 Read block 4622 len 32768
20704 294/300 Read block 2529 len 24576
20705 295/300 Write block 3804 len 4096 data: 6
20706 296/300 Write block 429 len 16384 data: 3 4 4 4
20707 297/300 Write block 823 len 24576 data: 5 5 5 6 5 4
20708 298/300 Read block 2247 len 4096
20709 299/300 Flush
20710 300/300 Write block 500 len 40960 data: 5 8 8 9 5 5 5 5 5 4
20711 Main task has work to do, go find it
20712 got dsc Start(0)
20713 start 0
20714 Sep 22 23:28:20.039 INFO request completed, latency_us: 323, response_code: 204, uri: /start/cid/0, method: POST, req_id: 3f575722-0582-4c01-8cfd-6e2e15c26be0, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
20715 Replay: started 0, returned:Ok(())
20716 ----------------------------------------------------------------
20717 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20718 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
20719 6655 Acked 7654 Read 4 New Done Done false
20720 6656 Acked 7655 Read 7 New Done Done false
20721 6657 Acked 7656 Write 3 New Done Done false
20722 6658 Acked 7657 Write 8 New Done Done false
20723 6659 Acked 7658 Flush 0 New Done Done false
20724 6660 Acked 7659 Write 9 New Done Done false
20725 6661 Acked 7660 Read 6 New Done Done false
20726 6662 Acked 7661 Write 8 New Done Done false
20727 6663 Acked 7662 Read 10 New Done Done false
20728 6664 Acked 7663 Read 6 New Done Done false
20729 6665 Acked 7664 Write 1 New Done Done false
20730 6666 Acked 7665 Read 1 New Done Done false
20731 6667 Acked 7666 Read 8 New Done Done false
20732 6668 Acked 7667 Read 2 New Done Done false
20733 6669 Acked 7668 Read 7 New Done Done false
20734 6670 Acked 7669 Write 3 New Done Done false
20735 6671 Acked 7670 Flush 0 New Done Done false
20736 6672 Acked 7671 Read 4 New Done Done false
20737 6673 Acked 7672 Read 4 New Done Done false
20738 6674 Acked 7673 Write 4 New Done Done false
20739 6675 Acked 7674 Read 1 New Done Done false
20740 6676 Acked 7675 Flush 0 New Done Done false
20741 6677 Acked 7676 Write 6 New Done Done false
20742 6678 Acked 7677 Write 8 New Done Done false
20743 6679 Acked 7678 Read 6 New Done Done false
20744 6680 Acked 7679 Read 5 New Done Done false
20745 6681 Acked 7680 Read 9 New Done Done false
20746 6682 Acked 7681 Read 7 New Done Done false
20747 6683 Acked 7682 Read 3 New Done Done false
20748 6684 Acked 7683 Read 9 New Done Done false
20749 6685 Acked 7684 Write 8 New Done Done false
20750 6686 Acked 7685 Read 1 New Done Done false
20751 6687 Acked 7686 Read 3 New Done Done false
20752 6688 Acked 7687 Write 7 New Done Done false
20753 6689 Acked 7688 Read 3 New Done Done false
20754 6690 Acked 7689 Write 4 New Done Done false
20755 6691 Acked 7690 Read 1 New Done Done false
20756 6692 Acked 7691 Write 4 New Done Done false
20757 6693 Acked 7692 Read 8 New Done Done false
20758 6694 Acked 7693 Write 4 New Done Done false
20759 6695 Acked 7694 Read 4 New Done Done false
20760 6696 Acked 7695 Write 7 New Done Done false
20761 6697 Acked 7696 Write 3 New Done Done false
20762 6698 Acked 7697 Read 9 New Done Done false
20763 6699 Acked 7698 Write 6 New Done Done false
20764 6700 Acked 7699 Read 2 New Done Done false
20765 6701 Acked 7700 Write 7 New Done Done false
20766 6702 Acked 7701 Write 4 New Done Done false
20767 6703 Acked 7702 Read 10 New Done Done false
20768 6704 Acked 7703 Write 1 New Done Done false
20769 6705 Acked 7704 Read 9 New Done Done false
20770 6706 Acked 7705 Read 5 New Done Done false
20771 6707 Acked 7706 Write 7 New Done Done false
20772 6708 Acked 7707 Write 3 New Done Done false
20773 6709 Acked 7708 Write 7 New Done Done false
20774 6710 Acked 7709 Read 9 New Done Done false
20775 6711 Acked 7710 Read 7 New Done Done false
20776 6712 Acked 7711 Read 4 New Done Done false
20777 6713 Acked 7712 Read 5 New Done Done false
20778 6714 Acked 7713 Write 4 New Done Done false
20779 6715 Acked 7714 Write 8 New Done Done false
20780 6716 Acked 7715 Write 4 New Done Done false
20781 6717 Acked 7716 Flush 0 New Done Done false
20782 6718 Acked 7717 Read 1 New Done Done false
20783 6719 Acked 7718 Read 8 New Done Done false
20784 6720 Acked 7719 Read 5 New Done Done false
20785 6721 Acked 7720 Flush 0 New Done Done false
20786 6722 Acked 7721 Read 4 New Done Done false
20787 6723 Acked 7722 Write 4 New Done Done false
20788 6724 Acked 7723 Write 7 New Done Done false
20789 6725 Acked 7724 Flush 0 New Done Done false
20790 6726 Acked 7725 Read 9 New Done Done false
20791 6727 Acked 7726 Write 3 New Done Done false
20792 6728 Acked 7727 Read 4 New Done Done false
20793 6729 Acked 7728 Read 9 New Done Done false
20794 6730 Acked 7729 Read 5 New Done Done false
20795 6731 Acked 7730 Read 4 New Done Done false
20796 6732 Acked 7731 Write 3 New Done Done false
20797 6733 Acked 7732 Read 6 New Done Done false
20798 6734 Acked 7733 Write 4 New Done Done false
20799 6735 Acked 7734 Write 9 New Done Done false
20800 6736 Acked 7735 Read 7 New Done Done false
20801 6737 Acked 7736 Read 9 New Done Done false
20802 6738 Acked 7737 Read 7 New Done Done false
20803 6739 Acked 7738 Read 5 New Done Done false
20804 6740 Acked 7739 Write 9 New Done Done false
20805 6741 Acked 7740 Write 4 New Done Done false
20806 6742 Acked 7741 Read 6 New Done Done false
20807 6743 Acked 7742 Read 1 New Done Done false
20808 6744 Acked 7743 Write 10 New Done Done false
20809 6745 Acked 7744 Read 1 New Done Done false
20810 6746 Acked 7745 Read 7 New Done Done false
20811 6747 Acked 7746 Read 3 New Done Done false
20812 6748 Acked 7747 Write 10 New Done Done false
20813 6749 Acked 7748 Flush 0 New Done Done false
20814 6750 Acked 7749 Flush 0 New Done Done false
20815 6751 Acked 7750 Read 8 New Done Done false
20816 6752 Acked 7751 Flush 0 New Done Done false
20817 6753 Acked 7752 Write 9 New Done Done false
20818 6754 Acked 7753 Write 9 New Done Done false
20819 6755 Acked 7754 Flush 0 New Done Done false
20820 6756 Acked 7755 Read 3 New Done Done false
20821 6757 Acked 7756 Write 4 New Done Done false
20822 6758 Acked 7757 Write 2 New Done Done false
20823 6759 Acked 7758 Write 5 New Done Done false
20824 6760 Acked 7759 Read 7 New Done Done false
20825 6761 Acked 7760 Write 8 New Done Done false
20826 6762 Acked 7761 Read 3 New Done Done false
20827 6763 Acked 7762 Read 2 New Done Done false
20828 6764 Acked 7763 Write 5 New Done Done false
20829 6765 Acked 7764 Write 2 New Done Done false
20830 6766 Acked 7765 Read 3 New Done Done false
20831 6767 Acked 7766 Write 5 New Done Done false
20832 6768 Acked 7767 Read 1 New Done Done false
20833 6769 Acked 7768 Read 7 New Done Done false
20834 6770 Acked 7769 Flush 0 New Done Done false
20835 6771 Acked 7770 Read 5 New Done Done false
20836 6772 Acked 7771 Read 8 New Done Done false
20837 6773 Acked 7772 Flush 0 New Done Done false
20838 6774 Acked 7773 Read 2 New Done Done false
20839 6775 Acked 7774 Read 8 New Done Done false
20840 6776 Acked 7775 Write 6 New Done Done false
20841 6777 Acked 7776 Write 3 New Done Done false
20842 6778 Acked 7777 Write 5 New Done Done false
20843 6779 Acked 7778 Read 6 New Done Done false
20844 6780 Acked 7779 Write 4 New Done Done false
20845 6781 Acked 7780 Read 4 New Done Done false
20846 6782 Acked 7781 Write 5 New Done Done false
20847 6783 Acked 7782 Read 2 New Done Done false
20848 6784 Acked 7783 Write 4 New Done Done false
20849 6785 Acked 7784 Write 6 New Done Done false
20850 6786 Acked 7785 Write 5 New Done Done false
20851 6787 Acked 7786 Read 7 New Done Done false
20852 6788 Acked 7787 Read 4 New Done Done false
20853 6789 Acked 7788 Write 6 New Done Done false
20854 6790 Acked 7789 Read 5 New Done Done false
20855 6791 Acked 7790 Flush 0 New Done Done false
20856 6792 Acked 7791 Read 8 New Done Done false
20857 6793 Acked 7792 Write 2 New Done Done false
20858 6794 Acked 7793 Write 7 New Done Done false
20859 6795 Acked 7794 Flush 0 New Done Done false
20860 6796 Acked 7795 Flush 0 New Done Done false
20861 6797 Acked 7796 Flush 0 New Done Done false
20862 6798 Acked 7797 Write 7 New Done Done false
20863 6799 Acked 7798 Write 7 New Done Done false
20864 6800 Acked 7799 Read 3 New Done Done false
20865 6801 Acked 7800 Write 8 New Done Done false
20866 6802 Acked 7801 Flush 0 New Done Done false
20867 6803 Acked 7802 Write 2 New Done Done false
20868 6804 Acked 7803 Flush 0 New Done Done false
20869 6805 Acked 7804 Flush 0 New Done Done false
20870 6806 Acked 7805 Read 5 New Done Done false
20871 6807 Acked 7806 Read 1 New Done Done false
20872 6808 Acked 7807 Read 2 New Done Done false
20873 6809 Acked 7808 Write 7 New Done Done false
20874 6810 Acked 7809 Read 7 New Done Done false
20875 6811 Acked 7810 Flush 0 New Done Done false
20876 6812 Acked 7811 Read 8 New Done Done false
20877 6813 Acked 7812 Flush 0 New Done Done false
20878 6814 Acked 7813 Write 10 New Done Done false
20879 6815 Acked 7814 Write 8 New Done Done false
20880 6816 Acked 7815 Read 5 New Done Done false
20881 6817 Acked 7816 Write 3 New Done Done false
20882 6818 Acked 7817 Flush 0 New Done Done false
20883 6819 Acked 7818 Write 5 New Done Done false
20884 6820 Acked 7819 Read 2 New Done Done false
20885 6821 Acked 7820 Read 4 New Done Done false
20886 6822 Acked 7821 Write 3 New Done Done false
20887 6823 Acked 7822 Read 6 New Done Done false
20888 6824 Acked 7823 Read 8 New Done Done false
20889 6825 Acked 7824 Flush 0 New Done Done false
20890 6826 Acked 7825 Read 3 New Done Done false
20891 6827 Acked 7826 Write 10 New Done Done false
20892 6828 Acked 7827 Write 6 New Done Done false
20893 6829 Acked 7828 Write 6 New Done Done false
20894 6830 Acked 7829 Write 3 New Done Done false
20895 6831 Acked 7830 Read 8 New Done Done false
20896 6832 Acked 7831 Read 4 New Done Done false
20897 6833 Acked 7832 Read 6 New Done Done false
20898 6834 Acked 7833 Write 2 New Done Done false
20899 6835 Acked 7834 Write 7 New Done Done false
20900 6836 Acked 7835 Write 6 New Done Done false
20901 6837 Acked 7836 Read 10 New Done Done false
20902 6838 Acked 7837 Flush 0 New Done Done false
20903 6839 Acked 7838 Write 2 New Done Done false
20904 6840 Acked 7839 Read 4 New Done Done false
20905 6841 Acked 7840 Write 2 New Done Done false
20906 6842 Acked 7841 Read 3 New Done Done false
20907 6843 Acked 7842 Read 2 New Done Done false
20908 6844 Acked 7843 Read 5 New Done Done false
20909 6845 Acked 7844 Write 1 New Done Done false
20910 6846 Acked 7845 Write 2 New Done Done false
20911 6847 Acked 7846 Read 4 New Done Done false
20912 6848 Acked 7847 Write 10 New Done Done false
20913 6849 Acked 7848 Read 7 New Done Done false
20914 6850 Acked 7849 Write 4 New Done Done false
20915 6851 Acked 7850 Read 9 New Done Done false
20916 6852 Acked 7851 Write 8 New Done Done false
20917 6853 Acked 7852 Read 6 New Done Done false
20918 6854 Acked 7853 Write 7 New Done Done false
20919 6855 Acked 7854 Read 7 New Done Done false
20920 6856 Acked 7855 Write 7 New Done Done false
20921 6857 Acked 7856 Read 8 New Done Done false
20922 6858 Acked 7857 Write 4 New Done Done false
20923 6859 Acked 7858 Flush 0 New Done Done false
20924 6860 Acked 7859 Read 3 New Done Done false
20925 6861 Acked 7860 Write 5 New Done Done false
20926 6862 Acked 7861 Flush 0 New Done Done false
20927 6863 Acked 7862 Write 9 New Done Done false
20928 6864 Acked 7863 Write 5 New Done Done false
20929 6865 Acked 7864 Read 3 New Done Done false
20930 6866 Acked 7865 Write 1 New Done Done false
20931 6867 Acked 7866 Write 6 New Done Done false
20932 6868 Acked 7867 Read 5 New Done Done false
20933 6869 Acked 7868 Read 10 New Done Done false
20934 6870 Acked 7869 Read 4 New Done Done false
20935 6871 Acked 7870 Read 4 New Done Done false
20936 6872 Acked 7871 Write 8 New Done Done false
20937 6873 Acked 7872 Write 6 New Done Done false
20938 6874 Acked 7873 Write 2 New Done Done false
20939 6875 Acked 7874 Write 4 New Done Done false
20940 6876 Acked 7875 Write 2 New Done Done false
20941 6877 Acked 7876 Read 9 New Done Done false
20942 6878 Acked 7877 Write 1 New Done Done false
20943 6879 Acked 7878 Write 7 New Done Done false
20944 6880 Acked 7879 Write 10 New Done Done false
20945 6881 Acked 7880 Read 4 New Done Done false
20946 6882 Acked 7881 Write 8 New Done Done false
20947 6883 Acked 7882 Write 3 New Done Done false
20948 6884 Acked 7883 Write 10 New Done Done false
20949 6885 Acked 7884 Read 5 New Done Done false
20950 6886 Acked 7885 Read 10 New Done Done false
20951 6887 Acked 7886 Read 2 New Done Done false
20952 6888 Acked 7887 Read 6 New Done Done false
20953 6889 Acked 7888 Write 7 New Done Done false
20954 6890 Acked 7889 Read 8 New Done Done false
20955 6891 Acked 7890 Flush 0 New Done Done false
20956 6892 Acked 7891 Write 5 New Done Done false
20957 6893 Acked 7892 Flush 0 New Done Done false
20958 6894 Acked 7893 Flush 0 New Done Done false
20959 6895 Acked 7894 Read 8 New Done Done false
20960 6896 Acked 7895 Read 3 New Done Done false
20961 6897 Acked 7896 Read 2 New Done Done false
20962 6898 Acked 7897 Flush 0 New Done Done false
20963 6899 Acked 7898 Write 10 New Done Done false
20964 6900 Acked 7899 Write 10 New Done Done false
20965 6901 Acked 7900 Read 4 New Done Done false
20966 6902 Acked 7901 Read 2 New Done Done false
20967 6903 Acked 7902 Flush 0 New Done Done false
20968 6904 Acked 7903 Write 6 New Done Done false
20969 6905 Acked 7904 Read 9 New Done Done false
20970 6906 Acked 7905 Write 2 New Done Done false
20971 6907 Acked 7906 Write 5 New Done Done false
20972 6908 Acked 7907 Write 1 New Done Done false
20973 6909 Acked 7908 Flush 0 New Done Done false
20974 6910 Acked 7909 Write 3 New Done Done false
20975 6911 Acked 7910 Read 5 New Done Done false
20976 6912 Acked 7911 Write 3 New Done Done false
20977 6913 Acked 7912 Write 6 New Done Done false
20978 6914 Acked 7913 Read 6 New Done Done false
20979 6915 Acked 7914 Read 9 New Done Done false
20980 6916 Acked 7915 Write 8 New Done Done false
20981 6917 Acked 7916 Read 7 New Done Done false
20982 6918 Acked 7917 Read 10 New Done Done false
20983 6919 Acked 7918 Read 2 New Done Done false
20984 6920 Acked 7919 Read 7 New Done Done false
20985 6921 Acked 7920 Read 9 New Done Done false
20986 6922 Acked 7921 Flush 0 New Done Done false
20987 6923 Acked 7922 Write 10 New Done Done false
20988 6924 Acked 7923 Write 4 New Done Done false
20989 6925 Acked 7924 Write 2 New Done Done false
20990 6926 Acked 7925 Read 3 New Done Done false
20991 6927 Acked 7926 Read 6 New Done Done false
20992 6928 Acked 7927 Write 10 New Done Done false
20993 6929 Acked 7928 Read 4 New Done Done false
20994 6930 Acked 7929 Write 5 New Done Done false
20995 6931 Acked 7930 Read 9 New Done Done false
20996 6932 Acked 7931 Read 4 New Done Done false
20997 6933 Acked 7932 Write 8 New Done Done false
20998 6934 Acked 7933 Read 3 New Done Done false
20999 6935 Acked 7934 Read 9 New Done Done false
21000 6936 Acked 7935 Write 3 New Done Done false
21001 6937 Acked 7936 Write 5 New Done Done false
21002 6938 Acked 7937 Read 10 New Done Done false
21003 6939 Acked 7938 Write 4 New Done Done false
21004 6940 Acked 7939 Write 10 New Done Done false
21005 6941 Acked 7940 Read 5 New Done Done false
21006 6942 Acked 7941 Read 10 New Done Done false
21007 6943 Acked 7942 Read 5 New Done Done false
21008 6944 Acked 7943 Read 6 New Done Done false
21009 6945 Acked 7944 Read 9 New Done Done false
21010 6946 Acked 7945 Read 1 New Done Done false
21011 6947 Acked 7946 Read 9 New Done Done false
21012 6948 Acked 7947 Read 8 New Done Done false
21013 6949 Acked 7948 Read 6 New Done Done false
21014 6950 Acked 7949 Write 1 New Done Done false
21015 6951 Acked 7950 Write 4 New Done Done false
21016 6952 Acked 7951 Write 6 New Done Done false
21017 6953 Acked 7952 Read 1 New Done Done false
21018 6954 Acked 7953 Flush 0 New Done Done false
21019 6955 Acked 7954 Write 10 New Sent Sent false
21020 STATES DS:0 DS:1 DS:2 TOTAL
21021 New 301 0 0 301
21022 Sent 0 1 1 2
21023 Done 0 300 300 600
21024 Skipped 0 0 0 0
21025 Error 0 0 0 0
21026 Last Flush: 7653 7953 7953
21027 Downstairs last five completed: 7653 7652 7651 7650 7649 7648
21028 Upstairs last five completed: 6955 6954 6953 6952 6951 6950
21029 CLIENT: Up:0 ds:301 act:2
21030 [8810] got start action
21031 [8810] I am going to restart
21032 Starting downstairs at port 8810
21033 Make output file at "/tmp/dsc/downstairs-8810.txt"
21034 [8810][0] reports Starting
21035 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1196)
21036 [8810][0] reports Running
210372023-09-22T23:28:22.119ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
210382023-09-22T23:28:22.119ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
210392023-09-22T23:28:22.119ZINFOcrucible: [0] client re-new 302 jobs since flush 7653 = downstairs
210402023-09-22T23:28:22.120ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
210412023-09-22T23:28:22.120ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
210422023-09-22T23:28:22.120ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
210432023-09-22T23:28:22.120ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
210442023-09-22T23:28:22.120ZINFOcrucible: [0] send last flush ID to this DS: 7653
210452023-09-22T23:28:22.120ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
210462023-09-22T23:28:22.121ZINFOcrucible: [0] Replied this last flush ID: 7653
210472023-09-22T23:28:22.121ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
210482023-09-22T23:28:22.121ZINFOcrucible: [0] Transition from Offline to Replay
210492023-09-22T23:28:22.121ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
210502023-09-22T23:28:22.121ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
210512023-09-22T23:28:22.121ZINFOcrucible: [0] Transition from Replay to Active
210522023-09-22T23:28:22.121ZINFOcrucible: [0] Starts cmd_loop
210532023-09-22T23:28:22.121ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
210542023-09-22T23:28:22.121ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
210552023-09-22T23:28:22.121ZINFOcrucible: Set check for repair
210562023-09-22T23:28:23.122ZINFOcrucible: Checking if live repair is needed
210572023-09-22T23:28:23.122ZINFOcrucible: No Live Repair required at this time
210582023-09-22T23:28:23.122ZWARNcrucible: [0] flow control end
21059 ----------------------------------------------------------------
21060 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21061 Downstairs last five completed: 7955 7954 7953 7952 7951 7950
21062 Upstairs last five completed: 6956 6955 6954 6953 6952 6951
21063 CLIENT: Up:0 ds:0 act:3
21064 Replay: All jobs finished, all DS active.
21065 Main task has work to do, go find it
21066 got dsc Stop(1)
21067 stop 1
21068 Sep 22 23:28:24.043 INFO request completed, latency_us: 537, response_code: 204, uri: /stop/cid/1, method: [8820] Got stop action so:false kr:false
21069 POST, req_id: b961fdf2-dce3-4464-a679-4887d512664c, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
21070 Sep 22 23:28:24.044 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/1, method: GET, req_id: 1bdfea67-5472-498d-97f0-428a4e722fe1, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
210712023-09-22T23:28:24.045ZWARNcrucible: [1] downstairs disconnected
210722023-09-22T23:28:24.045ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
210732023-09-22T23:28:24.045ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
210742023-09-22T23:28:24.045ZWARNcrucible: [1] pm_task rx.recv() is None
210752023-09-22T23:28:24.045ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
210762023-09-22T23:28:24.045ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
210772023-09-22T23:28:24.045ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
21078 [8820] Exited with: ExitStatus(unix_wait_status(9))
21079 [8820][1] reports Exit
210802023-09-22T23:28:26.046ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
21081 Sep 22 23:28:28.046 INFO request completed, latency_us: 272, response_code: 200, uri: /state/cid/1, method: GET, req_id: 743bf2e5-0916-4feb-b505-a2ee42b3b070, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
21082 001/300 Read block 4053 len 4096
21083 002/300 Read block 4950 len 8192
21084 003/300 Write block 2361 len 24576 data: 4 6 6 7 6 5
21085 004/300 Write block 1771 len 16384 data: 10 9 7 7
21086 005/300 Read block 318 len 40960
21087 006/300 Read block 724 len 32768
21088 007/300 Read block 3273 len 20480
21089 008/300 Read block 657 len 28672
21090 009/300 Read block 1387 len 4096
21091 010/300 Write block 1171 len 40960 data: 5 7 6 4 3 2 2 2 3 3
21092 011/300 Write block 282 len 36864 data: 3 5 4 5 4 5 4 4 5
21093 012/300 Write block 720 len 32768 data: 3 3 3 4 5 6 8 7
21094 013/300 Write block 1147 len 20480 data: 4 3 4 4 4
21095 014/300 Write block 477 len 24576 data: 3 3 3 4 4 5
21096 015/300 Write block 1666 len 16384 data: 3 2 5 5
21097 016/300 Write block 3866 len 12288 data: 3 4 4
21098 017/300 Read block 874 len 32768
21099 018/300 Write block 4697 len 28672 data: 5 5 5 5 5 3 3
21100 019/300 Read block 3454 len 32768
21101 020/300 Read block 4309 len 32768
21102 021/300 Flush
21103 022/300 Write block 1084 len 4096 data: 4
21104 023/300 Read block 3650 len 16384
21105 024/300 Read block 702 len 28672
21106 025/300 Write block 1778 len 32768 data: 5 6 5 5 4 4 3 3
21107 026/300 Read block 2135 len 40960
21108 027/300 Read block 4742 len 20480
21109 028/300 Read block 2346 len 28672
21110 029/300 Read block 4671 len 4096
21111 030/300 Write block 4992 len 8192 data: 10 7
21112 031/300 Read block 218 len 16384
21113 032/300 Read block 3765 len 16384
21114 033/300 Write block 1574 len 28672 data: 5 6 4 3 2 3 3
21115 034/300 Write block 1172 len 16384 data: 8 7 5 4
21116 035/300 Flush
21117 036/300 Write block 370 len 20480 data: 7 6 4 4 4
21118 037/300 Write block 3678 len 40960 data: 4 3 3 3 5 6 6 7 6 5
21119 038/300 Write block 2129 len 36864 data: 3 3 3 4 5 5 5 4 4
21120 039/300 Write block 4274 len 8192 data: 5 6
21121 040/300 Write block 852 len 24576 data: 7 6 6 6 6 6
21122 041/300 Read block 3953 len 20480
21123 042/300 Write block 4012 len 40960 data: 3 4 5 5 4 6 6 7 6 6
21124 043/300 Read block 1204 len 20480
21125 044/300 Read block 4149 len 16384
21126 045/300 Read block 473 len 40960
21127 046/300 Write block 4559 len 4096 data: 5
21128 047/300 Read block 4724 len 40960
21129 048/300 Write block 2182 len 32768 data: 6 5 5 4 4 3 3 3
21130 049/300 Write block 2954 len 20480 data: 4 3 5 5 5
21131 050/300 Read block 1379 len 24576
21132 051/300 Read block 4857 len 4096
21133 052/300 Flush
21134 053/300 Read block 2385 len 20480
21135 054/300 Write block 215 len 4096 data: 5
21136 055/300 Read block 1115 len 12288
21137 056/300 Read block 522 len 24576
21138 057/300 Read block 4712 len 36864
21139 058/300 Flush
21140 059/300 Read block 1267 len 36864
21141 060/300 Read block 3019 len 32768
21142 061/300 Write block 1613 len 4096 data: 4
21143 062/300 Write block 3306 len 20480 data: 5 5 4 4 5
21144 063/300 Read block 4455 len 32768
21145 064/300 Read block 4944 len 36864
21146 065/300 Write block 2757 len 4096 data: 2
21147 066/300 Read block 1865 len 16384
21148 067/300 Read block 1548 len 8192
21149 068/300 Write block 4002 len 20480 data: 8 7 6 4 4
21150 069/300 Write block 4333 len 16384 data: 7 6 5 5
21151 070/300 Read block 606 len 20480
21152 071/300 Write block 1369 len 4096 data: 6
21153 072/300 Read block 2914 len 4096
21154 073/300 Read block 131 len 16384
21155 074/300 Read block 3985 len 16384
21156 075/300 Write block 4471 len 40960 data: 4 5 6 6 6 7 6 6 6 5
21157 076/300 Write block 4645 len 20480 data: 7 7 8 7 6
21158 077/300 Read block 4891 len 40960
21159 078/300 Write block 1135 len 4096 data: 3
21160 079/300 Read block 948 len 4096
21161 080/300 Write block 2352 len 40960 data: 8 8 6 5 4 5 4 4 4 5
21162 081/300 Write block 2136 len 16384 data: 5 5 6 7
21163 082/300 Read block 1069 len 40960
21164 083/300 Write block 202 len 28672 data: 5 5 5 4 4 4 5
21165 084/300 Flush
21166 085/300 Write block 2475 len 40960 data: 7 8 7 6 6 5 6 6 7 5
21167 086/300 Read block 4281 len 20480
21168 087/300 Read block 877 len 20480
21169 088/300 Write block 3512 len 28672 data: 9 10 11 9 9 6 5
21170 089/300 Write block 3016 len 8192 data: 4 3
21171 090/300 Read block 2222 len 4096
21172 091/300 Read block 4608 len 4096
21173 092/300 Read block 3414 len 24576
21174 093/300 Read block 3421 len 28672
21175 094/300 Read block 3060 len 36864
21176 095/300 Write block 3516 len 36864 data: 10 7 6 5 5 6 6 6 5
21177 096/300 Flush
21178 097/300 Flush
21179 098/300 Write block 2467 len 40960 data: 4 4 4 5 6 7 6 5 8 9
21180 099/300 Read block 1985 len 8192
21181 100/300 Read block 2107 len 8192
21182 101/300 Read block 4281 len 20480
21183 102/300 Read block 1803 len 12288
21184 103/300 Write block 3860 len 28672 data: 7 7 6 4 3 4 4
21185 104/300 Read block 4174 len 28672
21186 105/300 Write block 1422 len 28672 data: 5 5 4 3 3 4 6
21187 106/300 Write block 3396 len 12288 data: 3 2 4
21188 107/300 Read block 28 len 24576
21189 108/300 Read block 1516 len 28672
21190 109/300 Write block 2438 len 16384 data: 5 5 5 5
21191 110/300 Flush
21192 111/300 Read block 1020 len 12288
21193 112/300 Read block 245 len 40960
21194 113/300 Write block 2784 len 32768 data: 7 7 7 7 6 6 7 5
21195 114/300 Flush
21196 115/300 Write block 2679 len 40960 data: 5 6 6 5 5 6 4 4 5 6
21197 116/300 Read block 3487 len 32768
21198 117/300 Read block 2208 len 40960
21199 118/300 Read block 4366 len 24576
21200 119/300 Read block 130 len 20480
21201 120/300 Write block 2648 len 28672 data: 4 4 5 4 3 3 3
21202 121/300 Read block 1349 len 8192
21203 122/300 Write block 1197 len 16384 data: 5 5 4 5
21204 123/300 Flush
21205 124/300 Read block 1582 len 40960
21206 125/300 Write block 1928 len 8192 data: 3 3
21207 126/300 Read block 895 len 40960
21208 127/300 Write block 786 len 16384 data: 5 4 2 2
21209 128/300 Write block 2492 len 8192 data: 5 4
21210 129/300 Write block 4234 len 40960 data: 5 5 6 5 5 3 3 3 3 4
21211 130/300 Write block 1998 len 12288 data: 6 6 7
21212 131/300 Read block 3194 len 20480
21213 132/300 Flush
21214 133/300 Read block 4116 len 20480
21215 134/300 Read block 1142 len 8192
21216 135/300 Flush
21217 136/300 Write block 1548 len 24576 data: 2 3 3 3 3 3
21218 137/300 Read block 3426 len 32768
21219 138/300 Write block 4515 len 12288 data: 3 2 3
21220 139/300 Read block 3204 len 8192
21221 140/300 Read block 512 len 36864
21222 141/300 Write block 1215 len 8192 data: 7 6
21223 142/300 Write block 3633 len 36864 data: 4 4 4 3 2 2 3 3 3
21224 143/300 Read block 3862 len 40960
21225 144/300 Read block 3889 len 24576
21226 145/300 Flush
21227 146/300 Write block 2466 len 12288 data: 5 5 5
21228 147/300 Read block 3738 len 8192
21229 148/300 Read block 4789 len 4096
21230 149/300 Flush
21231 150/300 Read block 4632 len 16384
21232 151/300 Read block 3061 len 4096
21233 152/300 Flush
21234 153/300 Write block 4529 len 16384 data: 6 7 6 7
21235 154/300 Write block 1585 len 4096 data: 3
21236 155/300 Read block 3894 len 12288
21237 156/300 Read block 2843 len 36864
21238 157/300 Write block 4026 len 12288 data: 3 3 3
21239 158/300 Flush
21240 159/300 Read block 4245 len 32768
21241 160/300 Write block 296 len 24576 data: 6 6 5 5 6 6
21242 161/300 Write block 41 len 40960 data: 7 5 5 5 5 5 4 4 4 5
21243 162/300 Write block 3433 len 32768 data: 5 5 4 5 4 3 4 4
21244 163/300 Read block 2514 len 40960
21245 164/300 Read block 3731 len 40960
21246 165/300 Write block 906 len 4096 data: 2
21247 166/300 Read block 2027 len 32768
21248 167/300 Write block 3188 len 32768 data: 4 4 5 6 6 6 4 4
21249 168/300 Read block 3033 len 24576
21250 169/300 Write block 1905 len 20480 data: 7 7 6 5 4
21251 170/300 Read block 4967 len 20480
21252 171/300 Write block 3242 len 32768 data: 2 2 3 4 3 3 2 5
21253 172/300 Read block 1346 len 40960
21254 173/300 Read block 2563 len 28672
21255 174/300 Read block 81 len 8192
21256 175/300 Read block 4834 len 24576
21257 176/300 Read block 844 len 28672
21258 177/300 Write block 1857 len 40960 data: 4 4 6 4 6 7 6 8 9 7
21259 178/300 Read block 4929 len 28672
21260 179/300 Read block 3903 len 36864
21261 180/300 Write block 3384 len 24576 data: 2 3 4 5 5 6
21262 181/300 Write block 4656 len 4096 data: 8
21263 182/300 Write block 155 len 16384 data: 4 3 4 4
21264 183/300 Read block 2982 len 16384
21265 184/300 Write block 54 len 32768 data: 5 5 6 9 9 9 9 9
21266 185/300 Write block 1239 len 16384 data: 4 4 4 4
21267 186/300 Write block 2430 len 40960 data: 5 5 3 4 3 4 3 5 6 6
21268 187/300 Write block 1292 len 8192 data: 9 11
21269 188/300 Write block 2514 len 24576 data: 9 9 8 7 6 3
21270 189/300 Flush
21271 190/300 Read block 2157 len 40960
21272 191/300 Read block 4378 len 32768
21273 192/300 Flush
21274 193/300 Write block 4232 len 8192 data: 4 5
21275 194/300 Read block 1402 len 20480
21276 195/300 Write block 4043 len 12288 data: 5 4 4
21277 196/300 Read block 4301 len 12288
21278 197/300 Read block 3011 len 8192
21279 198/300 Read block 184 len 4096
21280 199/300 Write block 3410 len 4096 data: 4
21281 200/300 Read block 3672 len 32768
21282 201/300 Write block 2726 len 36864 data: 5 5 5 4 4 3 5 5 5
21283 202/300 Write block 701 len 16384 data: 8 6 4 5
21284 203/300 Read block 675 len 16384
21285 204/300 Read block 2087 len 36864
21286 205/300 Write block 2651 len 20480 data: 5 4 4 4 3
21287 206/300 Flush
21288 207/300 Flush
21289 208/300 Flush
21290 209/300 Write block 779 len 32768 data: 5 4 5 5 6 6 6 6
21291 210/300 Read block 2630 len 20480
21292 211/300 Read block 3434 len 16384
21293 212/300 Read block 3897 len 12288
21294 213/300 Write block 1429 len 32768 data: 8 8 8 9 9 8 8 7
21295 214/300 Write block 2208 len 12288 data: 4 4 3
21296 215/300 Read block 347 len 32768
21297 216/300 Read block 2434 len 32768
21298 217/300 Read block 119 len 8192
21299 218/300 Read block 905 len 28672
21300 219/300 Write block 1805 len 4096 data: 5
21301 220/300 Write block 2895 len 36864 data: 7 7 8 9 8 7 7 6 5
21302 221/300 Write block 4426 len 40960 data: 9 7 7 6 6 7 4 5 4 5
21303 222/300 Flush
21304 223/300 Read block 3828 len 36864
21305 224/300 Read block 953 len 4096
21306 225/300 Write block 2179 len 32768 data: 4 4 5 7 6 6 5 5
21307 226/300 Read block 4650 len 32768
21308 227/300 Write block 3940 len 16384 data: 3 3 3 2
21309 228/300 Write block 299 len 8192 data: 6 7
21310 229/300 Write block 734 len 16384 data: 4 4 4 4
21311 230/300 Read block 1408 len 20480
21312 231/300 Write block 1231 len 36864 data: 4 3 4 5 5 5 5 4 5
21313 232/300 Write block 1369 len 20480 data: 7 5 6 6 6
21314 233/300 Read block 1191 len 24576
21315 234/300 Read block 660 len 32768
21316 235/300 Flush
21317 236/300 Write block 9 len 8192 data: 4 6
21318 237/300 Flush
21319 238/300 Read block 3981 len 8192
21320 239/300 Flush
21321 240/300 Write block 752 len 40960 data: 2 2 2 4 5 7 7 8 8 9
21322 241/300 Write block 4046 len 4096 data: 8
21323 242/300 Read block 2796 len 16384
21324 243/300 Write block 4066 len 12288 data: 4 4 4
21325 244/300 Read block 2928 len 12288
21326 245/300 Read block 1708 len 28672
21327 246/300 Read block 411 len 4096
21328 247/300 Write block 1266 len 8192 data: 4 5
21329 248/300 Read block 2048 len 8192
21330 249/300 Write block 1785 len 24576 data: 4 3 4 4 5 5
21331 250/300 Flush
21332 251/300 Read block 1993 len 36864
21333 252/300 Read block 2473 len 12288
21334 253/300 Read block 3557 len 4096
21335 254/300 Read block 320 len 24576
21336 255/300 Read block 3099 len 40960
21337 256/300 Read block 29 len 12288
21338 257/300 Read block 2314 len 12288
21339 258/300 Read block 711 len 24576
21340 259/300 Read block 2375 len 16384
21341 260/300 Flush
21342 261/300 Write block 1203 len 32768 data: 6 8 9 10 10 11 11 8
21343 262/300 Flush
21344 263/300 Read block 1040 len 20480
21345 264/300 Write block 1916 len 20480 data: 3 3 5 5 4
21346 265/300 Read block 2423 len 16384
21347 266/300 Read block 4933 len 16384
21348 267/300 Read block 3612 len 16384
21349 268/300 Flush
21350 269/300 Flush
21351 270/300 Write block 2826 len 40960 data: 5 6 7 9 9 8 10 10 7 7
21352 271/300 Flush
21353 272/300 Write block 1089 len 36864 data: 4 5 4 3 3 3 3 3 4
21354 273/300 Read block 250 len 28672
21355 274/300 Write block 2282 len 28672 data: 7 8 7 7 7 5 6
21356 275/300 Write block 4554 len 28672 data: 2 2 4 4 4 6 5
21357 276/300 Write block 3373 len 12288 data: 2 2 2
21358 277/300 Read block 4608 len 4096
21359 278/300 Read block 4559 len 28672
21360 279/300 Flush
21361 280/300 Write block 1005 len 16384 data: 6 6 7 6
21362 281/300 Write block 345 len 36864 data: 4 4 3 5 6 6 6 5 6
21363 282/300 Write block 4321 len 12288 data: 7 9 7
21364 283/300 Write block 231 len 4096 data: 5
21365 284/300 Write block 1049 len 24576 data: 5 5 6 5 5 3
21366 285/300 Write block 3815 len 36864 data: 3 3 4 5 5 5 4 4 5
21367 286/300 Read block 1142 len 4096
21368 287/300 Write block 4357 len 32768 data: 5 4 4 4 3 3 4 4
21369 288/300 Read block 1195 len 36864
21370 289/300 Write block 4853 len 8192 data: 10 8
21371 290/300 Write block 2345 len 36864 data: 5 5 6 5 6 7 8 9 9
21372 291/300 Read block 4189 len 32768
21373 292/300 Write block 4132 len 40960 data: 5 6 5 4 5 6 5 4 4 4
21374 293/300 Write block 1086 len 4096 data: 3
21375 294/300 Write block 1889 len 40960 data: 6 5 4 3 5 6 5 4 4 5
21376 295/300 Read block 3896 len 32768
21377 296/300 Write block 3076 len 20480 data: 2 2 3 4 4
21378 297/300 Write block 2199 len 36864 data: 9 8 8 7 6 6 6 5 4
21379 298/300 Write block 1541 len 20480 data: 4 6 7 5 4
21380 299/300 Read block 307 len 32768
21381 300/300 Read block 3094 len 28672
21382 Main task has work to do, go find it
21383 got dsc Start(1)
21384 start 1
21385 Sep 22 23:28:28.980 INFO request completed, latency_us: 304, response_code: 204, uri: /start/cid/1, method: POST, req_id: a99f0912-3776-4d04-8976-466ecfdcef52, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
21386 Replay: started 1, returned:Ok(())
21387 ----------------------------------------------------------------
21388 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21389 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
21390 6957 Acked 7956 Read 1 Done New Done false
21391 6958 Acked 7957 Read 2 Done New Done false
21392 6959 Acked 7958 Write 6 Done New Done false
21393 6960 Acked 7959 Write 4 Done New Done false
21394 6961 Acked 7960 Read 10 Done New Done false
21395 6962 Acked 7961 Read 8 Done New Done false
21396 6963 Acked 7962 Read 5 Done New Done false
21397 6964 Acked 7963 Read 7 Done New Done false
21398 6965 Acked 7964 Read 1 Done New Done false
21399 6966 Acked 7965 Write 10 Done New Done false
21400 6967 Acked 7966 Write 9 Done New Done false
21401 6968 Acked 7967 Write 8 Done New Done false
21402 6969 Acked 7968 Flush 0 Done New Done false
21403 6970 Acked 7969 Write 5 Done New Done false
21404 6971 Acked 7970 Write 6 Done New Done false
21405 6972 Acked 7971 Write 4 Done New Done false
21406 6973 Acked 7972 Write 3 Done New Done false
21407 6974 Acked 7973 Read 8 Done New Done false
21408 6975 Acked 7974 Write 7 Done New Done false
21409 6976 Acked 7975 Read 8 Done New Done false
21410 6977 Acked 7976 Read 8 Done New Done false
21411 6978 Acked 7977 Flush 0 Done New Done false
21412 6979 Acked 7978 Write 1 Done New Done false
21413 6980 Acked 7979 Read 4 Done New Done false
21414 6981 Acked 7980 Read 7 Done New Done false
21415 6982 Acked 7981 Write 8 Done New Done false
21416 6983 Acked 7982 Read 10 Done New Done false
21417 6984 Acked 7983 Read 5 Done New Done false
21418 6985 Acked 7984 Read 7 Done New Done false
21419 6986 Acked 7985 Read 1 Done New Done false
21420 6987 Acked 7986 Write 2 Done New Done false
21421 6988 Acked 7987 Read 4 Done New Done false
21422 6989 Acked 7988 Read 4 Done New Done false
21423 6990 Acked 7989 Write 7 Done New Done false
21424 6991 Acked 7990 Write 4 Done New Done false
21425 6992 Acked 7991 Flush 0 Done New Done false
21426 6993 Acked 7992 Write 5 Done New Done false
21427 6994 Acked 7993 Write 10 Done New Done false
21428 6995 Acked 7994 Write 9 Done New Done false
21429 6996 Acked 7995 Write 2 Done New Done false
21430 6997 Acked 7996 Write 6 Done New Done false
21431 6998 Acked 7997 Read 5 Done New Done false
21432 6999 Acked 7998 Write 10 Done New Done false
21433 7000 Acked 7999 Read 5 Done New Done false
21434 7001 Acked 8000 Read 4 Done New Done false
21435 7002 Acked 8001 Read 10 Done New Done false
21436 7003 Acked 8002 Write 1 Done New Done false
21437 7004 Acked 8003 Read 10 Done New Done false
21438 7005 Acked 8004 Write 8 Done New Done false
21439 7006 Acked 8005 Write 5 Done New Done false
21440 7007 Acked 8006 Read 6 Done New Done false
21441 7008 Acked 8007 Read 1 Done New Done false
21442 7009 Acked 8008 Flush 0 Done New Done false
21443 7010 Acked 8009 Read 5 Done New Done false
21444 7011 Acked 8010 Write 1 Done New Done false
21445 7012 Acked 8011 Read 3 Done New Done false
21446 7013 Acked 8012 Read 6 Done New Done false
21447 7014 Acked 8013 Read 9 Done New Done false
21448 7015 Acked 8014 Flush 0 Done New Done false
21449 7016 Acked 8015 Read 9 Done New Done false
21450 7017 Acked 8016 Read 8 Done New Done false
21451 7018 Acked 8017 Write 1 Done New Done false
21452 7019 Acked 8018 Write 5 Done New Done false
21453 7020 Acked 8019 Read 8 Done New Done false
21454 7021 Acked 8020 Read 9 Done New Done false
21455 7022 Acked 8021 Write 1 Done New Done false
21456 7023 Acked 8022 Read 4 Done New Done false
21457 7024 Acked 8023 Read 2 Done New Done false
21458 7025 Acked 8024 Write 5 Done New Done false
21459 7026 Acked 8025 Write 4 Done New Done false
21460 7027 Acked 8026 Read 5 Done New Done false
21461 7028 Acked 8027 Write 1 Done New Done false
21462 7029 Acked 8028 Read 1 Done New Done false
21463 7030 Acked 8029 Read 4 Done New Done false
21464 7031 Acked 8030 Read 4 Done New Done false
21465 7032 Acked 8031 Write 10 Done New Done false
21466 7033 Acked 8032 Write 5 Done New Done false
21467 7034 Acked 8033 Read 10 Done New Done false
21468 7035 Acked 8034 Write 1 Done New Done false
21469 7036 Acked 8035 Read 1 Done New Done false
21470 7037 Acked 8036 Write 10 Done New Done false
21471 7038 Acked 8037 Write 4 Done New Done false
21472 7039 Acked 8038 Read 10 Done New Done false
21473 7040 Acked 8039 Write 7 Done New Done false
21474 7041 Acked 8040 Flush 0 Done New Done false
21475 7042 Acked 8041 Write 10 Done New Done false
21476 7043 Acked 8042 Read 5 Done New Done false
21477 7044 Acked 8043 Read 5 Done New Done false
21478 7045 Acked 8044 Write 7 Done New Done false
21479 7046 Acked 8045 Write 2 Done New Done false
21480 7047 Acked 8046 Read 1 Done New Done false
21481 7048 Acked 8047 Read 1 Done New Done false
21482 7049 Acked 8048 Read 6 Done New Done false
21483 7050 Acked 8049 Read 7 Done New Done false
21484 7051 Acked 8050 Read 9 Done New Done false
21485 7052 Acked 8051 Write 9 Done New Done false
21486 7053 Acked 8052 Flush 0 Done New Done false
21487 7054 Acked 8053 Flush 0 Done New Done false
21488 7055 Acked 8054 Write 10 Done New Done false
21489 7056 Acked 8055 Read 2 Done New Done false
21490 7057 Acked 8056 Read 2 Done New Done false
21491 7058 Acked 8057 Read 5 Done New Done false
21492 7059 Acked 8058 Read 3 Done New Done false
21493 7060 Acked 8059 Write 7 Done New Done false
21494 7061 Acked 8060 Read 7 Done New Done false
21495 7062 Acked 8061 Write 7 Done New Done false
21496 7063 Acked 8062 Write 3 Done New Done false
21497 7064 Acked 8063 Read 6 Done New Done false
21498 7065 Acked 8064 Read 7 Done New Done false
21499 7066 Acked 8065 Write 4 Done New Done false
21500 7067 Acked 8066 Flush 0 Done New Done false
21501 7068 Acked 8067 Read 3 Done New Done false
21502 7069 Acked 8068 Read 10 Done New Done false
21503 7070 Acked 8069 Write 8 Done New Done false
21504 7071 Acked 8070 Flush 0 Done New Done false
21505 7072 Acked 8071 Write 10 Done New Done false
21506 7073 Acked 8072 Read 8 Done New Done false
21507 7074 Acked 8073 Read 10 Done New Done false
21508 7075 Acked 8074 Read 6 Done New Done false
21509 7076 Acked 8075 Read 5 Done New Done false
21510 7077 Acked 8076 Write 7 Done New Done false
21511 7078 Acked 8077 Read 2 Done New Done false
21512 7079 Acked 8078 Write 4 Done New Done false
21513 7080 Acked 8079 Flush 0 Done New Done false
21514 7081 Acked 8080 Read 10 Done New Done false
21515 7082 Acked 8081 Write 2 Done New Done false
21516 7083 Acked 8082 Read 10 Done New Done false
21517 7084 Acked 8083 Write 4 Done New Done false
21518 7085 Acked 8084 Write 2 Done New Done false
21519 7086 Acked 8085 Write 10 Done New Done false
21520 7087 Acked 8086 Write 3 Done New Done false
21521 7088 Acked 8087 Read 5 Done New Done false
21522 7089 Acked 8088 Flush 0 Done New Done false
21523 7090 Acked 8089 Read 5 Done New Done false
21524 7091 Acked 8090 Read 2 Done New Done false
21525 7092 Acked 8091 Flush 0 Done New Done false
21526 7093 Acked 8092 Write 6 Done New Done false
21527 7094 Acked 8093 Read 8 Done New Done false
21528 7095 Acked 8094 Write 3 Done New Done false
21529 7096 Acked 8095 Read 2 Done New Done false
21530 7097 Acked 8096 Read 9 Done New Done false
21531 7098 Acked 8097 Write 2 Done New Done false
21532 7099 Acked 8098 Write 9 Done New Done false
21533 7100 Acked 8099 Read 10 Done New Done false
21534 7101 Acked 8100 Read 6 Done New Done false
21535 7102 Acked 8101 Flush 0 Done New Done false
21536 7103 Acked 8102 Write 3 Done New Done false
21537 7104 Acked 8103 Read 2 Done New Done false
21538 7105 Acked 8104 Read 1 Done New Done false
21539 7106 Acked 8105 Flush 0 Done New Done false
21540 7107 Acked 8106 Read 4 Done New Done false
21541 7108 Acked 8107 Read 1 Done New Done false
21542 7109 Acked 8108 Flush 0 Done New Done false
21543 7110 Acked 8109 Write 4 Done New Done false
21544 7111 Acked 8110 Write 1 Done New Done false
21545 7112 Acked 8111 Read 3 Done New Done false
21546 7113 Acked 8112 Read 9 Done New Done false
21547 7114 Acked 8113 Write 3 Done New Done false
21548 7115 Acked 8114 Flush 0 Done New Done false
21549 7116 Acked 8115 Read 8 Done New Done false
21550 7117 Acked 8116 Write 6 Done New Done false
21551 7118 Acked 8117 Write 10 Done New Done false
21552 7119 Acked 8118 Write 8 Done New Done false
21553 7120 Acked 8119 Read 10 Done New Done false
21554 7121 Acked 8120 Read 10 Done New Done false
21555 7122 Acked 8121 Write 1 Done New Done false
21556 7123 Acked 8122 Read 8 Done New Done false
21557 7124 Acked 8123 Write 8 Done New Done false
21558 7125 Acked 8124 Read 6 Done New Done false
21559 7126 Acked 8125 Flush 0 Done New Done false
21560 7127 Acked 8126 Write 5 Done New Done false
21561 7128 Acked 8127 Read 5 Done New Done false
21562 7129 Acked 8128 Write 8 Done New Done false
21563 7130 Acked 8129 Read 10 Done New Done false
21564 7131 Acked 8130 Read 7 Done New Done false
21565 7132 Acked 8131 Read 2 Done New Done false
21566 7133 Acked 8132 Read 6 Done New Done false
21567 7134 Acked 8133 Read 7 Done New Done false
21568 7135 Acked 8134 Write 10 Done New Done false
21569 7136 Acked 8135 Read 7 Done New Done false
21570 7137 Acked 8136 Read 9 Done New Done false
21571 7138 Acked 8137 Write 6 Done New Done false
21572 7139 Acked 8138 Write 1 Done New Done false
21573 7140 Acked 8139 Write 4 Done New Done false
21574 7141 Acked 8140 Read 4 Done New Done false
21575 7142 Acked 8141 Write 8 Done New Done false
21576 7143 Acked 8142 Write 4 Done New Done false
21577 7144 Acked 8143 Write 10 Done New Done false
21578 7145 Acked 8144 Write 2 Done New Done false
21579 7146 Acked 8145 Write 6 Done New Done false
21580 7147 Acked 8146 Flush 0 Done New Done false
21581 7148 Acked 8147 Read 10 Done New Done false
21582 7149 Acked 8148 Read 8 Done New Done false
21583 7150 Acked 8149 Flush 0 Done New Done false
21584 7151 Acked 8150 Write 2 Done New Done false
21585 7152 Acked 8151 Read 5 Done New Done false
21586 7153 Acked 8152 Write 3 Done New Done false
21587 7154 Acked 8153 Read 3 Done New Done false
21588 7155 Acked 8154 Read 2 Done New Done false
21589 7156 Acked 8155 Read 1 Done New Done false
21590 7157 Acked 8156 Write 1 Done New Done false
21591 7158 Acked 8157 Read 8 Done New Done false
21592 7159 Acked 8158 Write 9 Done New Done false
21593 7160 Acked 8159 Write 4 Done New Done false
21594 7161 Acked 8160 Read 4 Done New Done false
21595 7162 Acked 8161 Read 9 Done New Done false
21596 7163 Acked 8162 Write 5 Done New Done false
21597 7164 Acked 8163 Flush 0 Done New Done false
21598 7165 Acked 8164 Flush 0 Done New Done false
21599 7166 Acked 8165 Flush 0 Done New Done false
21600 7167 Acked 8166 Write 8 Done New Done false
21601 7168 Acked 8167 Read 5 Done New Done false
21602 7169 Acked 8168 Read 4 Done New Done false
21603 7170 Acked 8169 Read 3 Done New Done false
21604 7171 Acked 8170 Write 8 Done New Done false
21605 7172 Acked 8171 Write 3 Done New Done false
21606 7173 Acked 8172 Read 8 Done New Done false
21607 7174 Acked 8173 Read 8 Done New Done false
21608 7175 Acked 8174 Read 2 Done New Done false
21609 7176 Acked 8175 Read 7 Done New Done false
21610 7177 Acked 8176 Write 1 Done New Done false
21611 7178 Acked 8177 Write 9 Done New Done false
21612 7179 Acked 8178 Write 10 Done New Done false
21613 7180 Acked 8179 Flush 0 Done New Done false
21614 7181 Acked 8180 Read 9 Done New Done false
21615 7182 Acked 8181 Read 1 Done New Done false
21616 7183 Acked 8182 Write 8 Done New Done false
21617 7184 Acked 8183 Read 8 Done New Done false
21618 7185 Acked 8184 Write 4 Done New Done false
21619 7186 Acked 8185 Write 2 Done New Done false
21620 7187 Acked 8186 Write 4 Done New Done false
21621 7188 Acked 8187 Read 5 Done New Done false
21622 7189 Acked 8188 Write 9 Done New Done false
21623 7190 Acked 8189 Write 5 Done New Done false
21624 7191 Acked 8190 Read 6 Done New Done false
21625 7192 Acked 8191 Read 8 Done New Done false
21626 7193 Acked 8192 Flush 0 Done New Done false
21627 7194 Acked 8193 Write 2 Done New Done false
21628 7195 Acked 8194 Flush 0 Done New Done false
21629 7196 Acked 8195 Read 2 Done New Done false
21630 7197 Acked 8196 Flush 0 Done New Done false
21631 7198 Acked 8197 Write 10 Done New Done false
21632 7199 Acked 8198 Write 1 Done New Done false
21633 7200 Acked 8199 Read 4 Done New Done false
21634 7201 Acked 8200 Write 3 Done New Done false
21635 7202 Acked 8201 Read 3 Done New Done false
21636 7203 Acked 8202 Read 7 Done New Done false
21637 7204 Acked 8203 Read 1 Done New Done false
21638 7205 Acked 8204 Write 2 Done New Done false
21639 7206 Acked 8205 Read 2 Done New Done false
21640 7207 Acked 8206 Write 6 Done New Done false
21641 7208 Acked 8207 Flush 0 Done New Done false
21642 7209 Acked 8208 Read 9 Done New Done false
21643 7210 Acked 8209 Read 3 Done New Done false
21644 7211 Acked 8210 Read 1 Done New Done false
21645 7212 Acked 8211 Read 6 Done New Done false
21646 7213 Acked 8212 Read 10 Done New Done false
21647 7214 Acked 8213 Read 3 Done New Done false
21648 7215 Acked 8214 Read 3 Done New Done false
21649 7216 Acked 8215 Read 6 Done New Done false
21650 7217 Acked 8216 Read 4 Done New Done false
21651 7218 Acked 8217 Flush 0 Done New Done false
21652 7219 Acked 8218 Write 8 Done New Done false
21653 7220 Acked 8219 Flush 0 Done New Done false
21654 7221 Acked 8220 Read 5 Done New Done false
21655 7222 Acked 8221 Write 5 Done New Done false
21656 7223 Acked 8222 Read 4 Done New Done false
21657 7224 Acked 8223 Read 4 Done New Done false
21658 7225 Acked 8224 Read 4 Done New Done false
21659 7226 Acked 8225 Flush 0 Done New Done false
21660 7227 Acked 8226 Flush 0 Done New Done false
21661 7228 Acked 8227 Write 10 Done New Done false
21662 7229 Acked 8228 Flush 0 Done New Done false
21663 7230 Acked 8229 Write 9 Done New Done false
21664 7231 Acked 8230 Read 7 Done New Done false
21665 7232 Acked 8231 Write 7 Done New Done false
21666 7233 Acked 8232 Write 7 Done New Done false
21667 7234 Acked 8233 Write 3 Done New Done false
21668 7235 Acked 8234 Read 1 Done New Done false
21669 7236 Acked 8235 Read 7 Done New Done false
21670 7237 Acked 8236 Flush 0 Done New Done false
21671 7238 Acked 8237 Write 4 Done New Done false
21672 7239 Acked 8238 Write 9 Done New Done false
21673 7240 Acked 8239 Write 3 Done New Done false
21674 7241 Acked 8240 Write 1 Done New Done false
21675 7242 Acked 8241 Write 6 Done New Done false
21676 7243 Acked 8242 Write 9 Done New Done false
21677 7244 Acked 8243 Read 1 Done New Done false
21678 7245 Acked 8244 Write 8 Done New Done false
21679 7246 Acked 8245 Read 9 Done New Done false
21680 7247 Acked 8246 Write 2 Done New Done false
21681 7248 Acked 8247 Write 9 Done New Done false
21682 7249 Acked 8248 Read 8 Done New Done false
21683 7250 Acked 8249 Write 10 Done New Done false
21684 7251 Acked 8250 Write 1 Done New Done false
21685 7252 Acked 8251 Write 10 Done New Done false
21686 7253 Acked 8252 Read 8 Done New Done false
21687 7254 Acked 8253 Write 5 Done New Done false
21688 7255 Acked 8254 Write 9 Done New Done false
21689 7256 Acked 8255 Write 5 Done New Done false
21690 7257 Acked 8256 Read 8 Done New Done false
21691 7258 Acked 8257 Read 7 Done New Done false
21692 STATES DS:0 DS:1 DS:2 TOTAL
21693 New 0 302 0 302
21694 Sent 0 0 0 0
21695 Done 302 0 302 604
21696 Skipped 0 0 0 0
21697 Error 0 0 0 0
21698 Last Flush: 8236 7955 8236
21699 Downstairs last five completed: 7955 7954 7953 7952 7951 7950
21700 Upstairs last five completed: 7258 7257 7256 7255 7254 7253
21701 CLIENT: Up:0 ds:302 act:2
21702 [8820] got start action
21703 [8820] I am going to restart
21704 Starting downstairs at port 8820
21705 Make output file at "/tmp/dsc/downstairs-8820.txt"
21706 [8820][1] reports Starting
21707 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1197)
21708 [8820][1] reports Running
21709 ----------------------------------------------------------------
21710 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
21711 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
21712 6957 Acked 7956 Read 1 Done New Done false
21713 6958 Acked 7957 Read 2 Done New Done false
21714 6959 Acked 7958 Write 6 Done New Done false
21715 6960 Acked 7959 Write 4 Done New Done false
21716 6961 Acked 7960 Read 10 Done New Done false
21717 6962 Acked 7961 Read 8 Done New Done false
21718 6963 Acked 7962 Read 5 Done New Done false
21719 6964 Acked 7963 Read 7 Done New Done false
21720 6965 Acked 7964 Read 1 Done New Done false
21721 6966 Acked 7965 Write 10 Done New Done false
21722 6967 Acked 7966 Write 9 Done New Done false
21723 6968 Acked 7967 Write 8 Done New Done false
21724 6969 Acked 7968 Flush 0 Done New Done false
21725 6970 Acked 7969 Write 5 Done New Done false
21726 6971 Acked 7970 Write 6 Done New Done false
21727 6972 Acked 7971 Write 4 Done New Done false
21728 6973 Acked 7972 Write 3 Done New Done false
21729 6974 Acked 7973 Read 8 Done New Done false
21730 6975 Acked 7974 Write 7 Done New Done false
21731 6976 Acked 7975 Read 8 Done New Done false
21732 6977 Acked 7976 Read 8 Done New Done false
21733 6978 Acked 7977 Flush 0 Done New Done false
21734 6979 Acked 7978 Write 1 Done New Done false
21735 6980 Acked 7979 Read 4 Done New Done false
21736 6981 Acked 7980 Read 7 Done New Done false
21737 6982 Acked 7981 Write 8 Done New Done false
21738 6983 Acked 7982 Read 10 Done New Done false
21739 6984 Acked 7983 Read 5 Done New Done false
21740 6985 Acked 7984 Read 7 Done New Done false
21741 6986 Acked 7985 Read 1 Done New Done false
21742 6987 Acked 7986 Write 2 Done New Done false
21743 6988 Acked 7987 Read 4 Done New Done false
21744 6989 Acked 7988 Read 4 Done New Done false
21745 6990 Acked 7989 Write 7 Done New Done false
21746 6991 Acked 7990 Write 4 Done New Done false
21747 6992 Acked 7991 Flush 0 Done New Done false
21748 6993 Acked 7992 Write 5 Done New Done false
21749 6994 Acked 7993 Write 10 Done New Done false
21750 6995 Acked 7994 Write 9 Done New Done false
21751 6996 Acked 7995 Write 2 Done New Done false
21752 6997 Acked 7996 Write 6 Done New Done false
21753 6998 Acked 7997 Read 5 Done New Done false
21754 6999 Acked 7998 Write 10 Done New Done false
21755 7000 Acked 7999 Read 5 Done New Done false
21756 7001 Acked 8000 Read 4 Done New Done false
21757 7002 Acked 8001 Read 10 Done New Done false
21758 7003 Acked 8002 Write 1 Done New Done false
21759 7004 Acked 8003 Read 10 Done New Done false
21760 7005 Acked 8004 Write 8 Done New Done false
21761 7006 Acked 8005 Write 5 Done New Done false
21762 7007 Acked 8006 Read 6 Done New Done false
21763 7008 Acked 8007 Read 1 Done New Done false
21764 7009 Acked 8008 Flush 0 Done New Done false
21765 7010 Acked 8009 Read 5 Done New Done false
21766 7011 Acked 8010 Write 1 Done New Done false
21767 7012 Acked 8011 Read 3 Done New Done false
21768 7013 Acked 8012 Read 6 Done New Done false
21769 7014 Acked 8013 Read 9 Done New Done false
21770 7015 Acked 8014 Flush 0 Done New Done false
21771 7016 Acked 8015 Read 9 Done New Done false
21772 7017 Acked 8016 Read 8 Done New Done false
21773 7018 Acked 8017 Write 1 Done New Done false
21774 7019 Acked 8018 Write 5 Done New Done false
21775 7020 Acked 8019 Read 8 Done New Done false
21776 7021 Acked 8020 Read 9 Done New Done false
21777 7022 Acked 8021 Write 1 Done New Done false
21778 7023 Acked 8022 Read 4 Done New Done false
21779 7024 Acked 8023 Read 2 Done New Done false
21780 7025 Acked 8024 Write 5 Done New Done false
21781 7026 Acked 8025 Write 4 Done New Done false
21782 7027 Acked 8026 Read 5 Done New Done false
21783 7028 Acked 8027 Write 1 Done New Done false
21784 7029 Acked 8028 Read 1 Done New Done false
21785 7030 Acked 8029 Read 4 Done New Done false
21786 7031 Acked 8030 Read 4 Done New Done false
21787 7032 Acked 8031 Write 10 Done New Done false
21788 7033 Acked 8032 Write 5 Done New Done false
21789 7034 Acked 8033 Read 10 Done New Done false
21790 7035 Acked 8034 Write 1 Done New Done false
21791 7036 Acked 8035 Read 1 Done New Done false
21792 7037 Acked 8036 Write 10 Done New Done false
21793 7038 Acked 8037 Write 4 Done New Done false
21794 7039 Acked 8038 Read 10 Done New Done false
21795 7040 Acked 8039 Write 7 Done New Done false
21796 7041 Acked 8040 Flush 0 Done New Done false
21797 7042 Acked 8041 Write 10 Done New Done false
21798 7043 Acked 8042 Read 5 Done New Done false
21799 7044 Acked 8043 Read 5 Done New Done false
21800 7045 Acked 8044 Write 7 Done New Done false
21801 7046 Acked 8045 Write 2 Done New Done false
21802 7047 Acked 8046 Read 1 Done New Done false
21803 7048 Acked 8047 Read 1 Done New Done false
21804 7049 Acked 8048 Read 6 Done New Done false
21805 7050 Acked 8049 Read 7 Done New Done false
21806 7051 Acked 8050 Read 9 Done New Done false
21807 7052 Acked 8051 Write 9 Done New Done false
21808 7053 Acked 8052 Flush 0 Done New Done false
21809 7054 Acked 8053 Flush 0 Done New Done false
21810 7055 Acked 8054 Write 10 Done New Done false
21811 7056 Acked 8055 Read 2 Done New Done false
21812 7057 Acked 8056 Read 2 Done New Done false
21813 7058 Acked 8057 Read 5 Done New Done false
21814 7059 Acked 8058 Read 3 Done New Done false
21815 7060 Acked 8059 Write 7 Done New Done false
21816 7061 Acked 8060 Read 7 Done New Done false
21817 7062 Acked 8061 Write 7 Done New Done false
21818 7063 Acked 8062 Write 3 Done New Done false
21819 7064 Acked 8063 Read 6 Done New Done false
21820 7065 Acked 8064 Read 7 Done New Done false
21821 7066 Acked 8065 Write 4 Done New Done false
21822 7067 Acked 8066 Flush 0 Done New Done false
21823 7068 Acked 8067 Read 3 Done New Done false
21824 7069 Acked 8068 Read 10 Done New Done false
21825 7070 Acked 8069 Write 8 Done New Done false
21826 7071 Acked 8070 Flush 0 Done New Done false
21827 7072 Acked 8071 Write 10 Done New Done false
21828 7073 Acked 8072 Read 8 Done New Done false
21829 7074 Acked 8073 Read 10 Done New Done false
21830 7075 Acked 8074 Read 6 Done New Done false
21831 7076 Acked 8075 Read 5 Done New Done false
21832 7077 Acked 8076 Write 7 Done New Done false
21833 7078 Acked 8077 Read 2 Done New Done false
21834 7079 Acked 8078 Write 4 Done New Done false
21835 7080 Acked 8079 Flush 0 Done New Done false
21836 7081 Acked 8080 Read 10 Done New Done false
21837 7082 Acked 8081 Write 2 Done New Done false
21838 7083 Acked 8082 Read 10 Done New Done false
21839 7084 Acked 8083 Write 4 Done New Done false
21840 7085 Acked 8084 Write 2 Done New Done false
21841 7086 Acked 8085 Write 10 Done New Done false
21842 7087 Acked 8086 Write 3 Done New Done false
21843 7088 Acked 8087 Read 5 Done New Done false
21844 7089 Acked 8088 Flush 0 Done New Done false
21845 7090 Acked 8089 Read 5 Done New Done false
21846 7091 Acked 8090 Read 2 Done New Done false
21847 7092 Acked 8091 Flush 0 Done New Done false
21848 7093 Acked 8092 Write 6 Done New Done false
21849 7094 Acked 8093 Read 8 Done New Done false
21850 7095 Acked 8094 Write 3 Done New Done false
21851 7096 Acked 8095 Read 2 Done New Done false
21852 7097 Acked 8096 Read 9 Done New Done false
21853 7098 Acked 8097 Write 2 Done New Done false
21854 7099 Acked 8098 Write 9 Done New Done false
21855 7100 Acked 8099 Read 10 Done New Done false
21856 7101 Acked 8100 Read 6 Done New Done false
21857 7102 Acked 8101 Flush 0 Done New Done false
21858 7103 Acked 8102 Write 3 Done New Done false
21859 7104 Acked 8103 Read 2 Done New Done false
21860 7105 Acked 8104 Read 1 Done New Done false
21861 7106 Acked 8105 Flush 0 Done New Done false
21862 7107 Acked 8106 Read 4 Done New Done false
21863 7108 Acked 8107 Read 1 Done New Done false
21864 7109 Acked 8108 Flush 0 Done New Done false
21865 7110 Acked 8109 Write 4 Done New Done false
21866 7111 Acked 8110 Write 1 Done New Done false
21867 7112 Acked 8111 Read 3 Done New Done false
21868 7113 Acked 8112 Read 9 Done New Done false
21869 7114 Acked 8113 Write 3 Done New Done false
21870 7115 Acked 8114 Flush 0 Done New Done false
21871 7116 Acked 8115 Read 8 Done New Done false
21872 7117 Acked 8116 Write 6 Done New Done false
21873 7118 Acked 8117 Write 10 Done New Done false
21874 7119 Acked 8118 Write 8 Done New Done false
21875 7120 Acked 8119 Read 10 Done New Done false
21876 7121 Acked 8120 Read 10 Done New Done false
21877 7122 Acked 8121 Write 1 Done New Done false
21878 7123 Acked 8122 Read 8 Done New Done false
21879 7124 Acked 8123 Write 8 Done New Done false
21880 7125 Acked 8124 Read 6 Done New Done false
21881 7126 Acked 8125 Flush 0 Done New Done false
21882 7127 Acked 8126 Write 5 Done New Done false
21883 7128 Acked 8127 Read 5 Done New Done false
21884 7129 Acked 8128 Write 8 Done New Done false
21885 7130 Acked 8129 Read 10 Done New Done false
21886 7131 Acked 8130 Read 7 Done New Done false
21887 7132 Acked 8131 Read 2 Done New Done false
21888 7133 Acked 8132 Read 6 Done New Done false
21889 7134 Acked 8133 Read 7 Done New Done false
21890 7135 Acked 8134 Write 10 Done New Done false
21891 7136 Acked 8135 Read 7 Done New Done false
21892 7137 Acked 8136 Read 9 Done New Done false
21893 7138 Acked 8137 Write 6 Done New Done false
21894 7139 Acked 8138 Write 1 Done New Done false
21895 7140 Acked 8139 Write 4 Done New Done false
21896 7141 Acked 8140 Read 4 Done New Done false
21897 7142 Acked 8141 Write 8 Done New Done false
21898 7143 Acked 8142 Write 4 Done New Done false
21899 7144 Acked 8143 Write 10 Done New Done false
21900 7145 Acked 8144 Write 2 Done New Done false
21901 7146 Acked 8145 Write 6 Done New Done false
21902 7147 Acked 8146 Flush 0 Done New Done false
21903 7148 Acked 8147 Read 10 Done New Done false
21904 7149 Acked 8148 Read 8 Done New Done false
21905 7150 Acked 8149 Flush 0 Done New Done false
21906 7151 Acked 8150 Write 2 Done New Done false
21907 7152 Acked 8151 Read 5 Done New Done false
21908 7153 Acked 8152 Write 3 Done New Done false
21909 7154 Acked 8153 Read 3 Done New Done false
21910 7155 Acked 8154 Read 2 Done New Done false
21911 7156 Acked 8155 Read 1 Done New Done false
21912 7157 Acked 8156 Write 1 Done New Done false
21913 7158 Acked 8157 Read 8 Done New Done false
21914 7159 Acked 8158 Write 9 Done New Done false
21915 7160 Acked 8159 Write 4 Done New Done false
21916 7161 Acked 8160 Read 4 Done New Done false
21917 7162 Acked 8161 Read 9 Done New Done false
21918 7163 Acked 8162 Write 5 Done New Done false
21919 7164 Acked 8163 Flush 0 Done New Done false
21920 7165 Acked 8164 Flush 0 Done New Done false
21921 7166 Acked 8165 Flush 0 Done New Done false
21922 7167 Acked 8166 Write 8 Done New Done false
21923 7168 Acked 8167 Read 5 Done New Done false
21924 7169 Acked 8168 Read 4 Done New Done false
21925 7170 Acked 8169 Read 3 Done New Done false
21926 7171 Acked 8170 Write 8 Done New Done false
21927 7172 Acked 8171 Write 3 Done New Done false
21928 7173 Acked 8172 Read 8 Done New Done false
21929 7174 Acked 8173 Read 8 Done New Done false
21930 7175 Acked 8174 Read 2 Done New Done false
21931 7176 Acked 8175 Read 7 Done New Done false
21932 7177 Acked 8176 Write 1 Done New Done false
21933 7178 Acked 8177 Write 9 Done New Done false
21934 7179 Acked 8178 Write 10 Done New Done false
21935 7180 Acked 8179 Flush 0 Done New Done false
21936 7181 Acked 8180 Read 9 Done New Done false
21937 7182 Acked 8181 Read 1 Done New Done false
21938 7183 Acked 8182 Write 8 Done New Done false
21939 7184 Acked 8183 Read 8 Done New Done false
21940 7185 Acked 8184 Write 4 Done New Done false
21941 7186 Acked 8185 Write 2 Done New Done false
21942 7187 Acked 8186 Write 4 Done New Done false
21943 7188 Acked 8187 Read 5 Done New Done false
21944 7189 Acked 8188 Write 9 Done New Done false
21945 7190 Acked 8189 Write 5 Done New Done false
21946 7191 Acked 8190 Read 6 Done New Done false
21947 7192 Acked 8191 Read 8 Done New Done false
21948 7193 Acked 8192 Flush 0 Done New Done false
21949 7194 Acked 8193 Write 2 Done New Done false
21950 7195 Acked 8194 Flush 0 Done New Done false
21951 7196 Acked 8195 Read 2 Done New Done false
21952 7197 Acked 8196 Flush 0 Done New Done false
21953 7198 Acked 8197 Write 10 Done New Done false
21954 7199 Acked 8198 Write 1 Done New Done false
21955 7200 Acked 8199 Read 4 Done New Done false
21956 7201 Acked 8200 Write 3 Done New Done false
21957 7202 Acked 8201 Read 3 Done New Done false
21958 7203 Acked 8202 Read 7 Done New Done false
21959 7204 Acked 8203 Read 1 Done New Done false
21960 7205 Acked 8204 Write 2 Done New Done false
21961 7206 Acked 8205 Read 2 Done New Done false
21962 7207 Acked 8206 Write 6 Done New Done false
21963 7208 Acked 8207 Flush 0 Done New Done false
21964 7209 Acked 8208 Read 9 Done New Done false
21965 7210 Acked 8209 Read 3 Done New Done false
21966 7211 Acked 8210 Read 1 Done New Done false
21967 7212 Acked 8211 Read 6 Done New Done false
21968 7213 Acked 8212 Read 10 Done New Done false
21969 7214 Acked 8213 Read 3 Done New Done false
21970 7215 Acked 8214 Read 3 Done New Done false
21971 7216 Acked 8215 Read 6 Done New Done false
21972 7217 Acked 8216 Read 4 Done New Done false
21973 7218 Acked 8217 Flush 0 Done New Done false
21974 7219 Acked 8218 Write 8 Done New Done false
21975 7220 Acked 8219 Flush 0 Done New Done false
21976 7221 Acked 8220 Read 5 Done New Done false
21977 7222 Acked 8221 Write 5 Done New Done false
21978 7223 Acked 8222 Read 4 Done New Done false
21979 7224 Acked 8223 Read 4 Done New Done false
21980 7225 Acked 8224 Read 4 Done New Done false
21981 7226 Acked 8225 Flush 0 Done New Done false
21982 7227 Acked 8226 Flush 0 Done New Done false
21983 7228 Acked 8227 Write 10 Done New Done false
21984 7229 Acked 8228 Flush 0 Done New Done false
21985 7230 Acked 8229 Write 9 Done New Done false
21986 7231 Acked 8230 Read 7 Done New Done false
21987 7232 Acked 8231 Write 7 Done New Done false
21988 7233 Acked 8232 Write 7 Done New Done false
21989 7234 Acked 8233 Write 3 Done New Done false
21990 7235 Acked 8234 Read 1 Done New Done false
21991 7236 Acked 8235 Read 7 Done New Done false
21992 7237 Acked 8236 Flush 0 Done New Done false
21993 7238 Acked 8237 Write 4 Done New Done false
21994 7239 Acked 8238 Write 9 Done New Done false
21995 7240 Acked 8239 Write 3 Done New Done false
21996 7241 Acked 8240 Write 1 Done New Done false
21997 7242 Acked 8241 Write 6 Done New Done false
21998 7243 Acked 8242 Write 9 Done New Done false
21999 7244 Acked 8243 Read 1 Done New Done false
22000 7245 Acked 8244 Write 8 Done New Done false
22001 7246 Acked 8245 Read 9 Done New Done false
22002 7247 Acked 8246 Write 2 Done New Done false
22003 7248 Acked 8247 Write 9 Done New Done false
22004 7249 Acked 8248 Read 8 Done New Done false
22005 7250 Acked 8249 Write 10 Done New Done false
22006 7251 Acked 8250 Write 1 Done New Done false
22007 7252 Acked 8251 Write 10 Done New Done false
22008 7253 Acked 8252 Read 8 Done New Done false
22009 7254 Acked 8253 Write 5 Done New Done false
22010 7255 Acked 8254 Write 9 Done New Done false
22011 7256 Acked 8255 Write 5 Done New Done false
22012 7257 Acked 8256 Read 8 Done New Done false
22013 7258 Acked 8257 Read 7 Done New Done false
22014 7259 Acked 8258 Flush 0 Done New Done false
22015 STATES DS:0 DS:1 DS:2 TOTAL
22016 New 0 303 0 303
22017 Sent 0 0 0 0
22018 Done 303 0 303 606
22019 Skipped 0 0 0 0
22020 Error 0 0 0 0
22021 Last Flush: 8258 7955 8258
22022 Downstairs last five completed: 7955 7954 7953 7952 7951 7950
22023 Upstairs last five completed: 7259 7258 7257 7256 7255 7254
22024 CLIENT: Up:0 ds:303 act:2
220252023-09-22T23:28:33.056ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
220262023-09-22T23:28:33.056ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
220272023-09-22T23:28:33.056ZINFOcrucible: [1] client re-new 303 jobs since flush 7955 = downstairs
220282023-09-22T23:28:33.057ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
220292023-09-22T23:28:33.057ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
220302023-09-22T23:28:33.057ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
220312023-09-22T23:28:33.057ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
220322023-09-22T23:28:33.057ZINFOcrucible: [1] send last flush ID to this DS: 7955
220332023-09-22T23:28:33.057ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
220342023-09-22T23:28:33.058ZINFOcrucible: [1] Replied this last flush ID: 7955
220352023-09-22T23:28:33.058ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
220362023-09-22T23:28:33.058ZINFOcrucible: [1] Transition from Offline to Replay
220372023-09-22T23:28:33.058ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
220382023-09-22T23:28:33.058ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
220392023-09-22T23:28:33.058ZINFOcrucible: [1] Transition from Replay to Active
220402023-09-22T23:28:33.058ZINFOcrucible: [1] Starts cmd_loop
220412023-09-22T23:28:33.058ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
220422023-09-22T23:28:33.058ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
220432023-09-22T23:28:33.058ZINFOcrucible: Set check for repair
220442023-09-22T23:28:34.059ZWARNcrucible: [1] flow control end
220452023-09-22T23:28:34.059ZINFOcrucible: Checking if live repair is needed
220462023-09-22T23:28:34.059ZINFOcrucible: No Live Repair required at this time
22047 ----------------------------------------------------------------
22048 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22049 Downstairs last five completed: 8258 8257 8256 8255 8254 8253
22050 Upstairs last five completed: 7259 7258 7257 7256 7255 7254
22051 CLIENT: Up:0 ds:0 act:3
22052 Replay: All jobs finished, all DS active.
22053 Main task has work to do, go find it
22054 got dsc Stop(0)
22055 stop 0
22056 Sep 22 23:28:36.987 INFO request completed, latency_us: 515, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 368ab386-bf67-4e30-bf2a-785e96ca77fc, remote_addr: 127.0[8810] Got stop action so:false kr:false
22057 .0.1:52471, local_addr: 127.0.0.1:9998
22058 Sep 22 23:28:36.989 INFO request completed, latency_us: 275, response_code: 200, uri: /state/cid/0, method: GET, req_id: f678c5b3-c272-449f-ae31-7287b8951448, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
220592023-09-22T23:28:36.989ZWARNcrucible: [0] downstairs disconnected
220602023-09-22T23:28:36.990ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
220612023-09-22T23:28:36.990ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
220622023-09-22T23:28:36.990ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
220632023-09-22T23:28:36.990ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
220642023-09-22T23:28:36.990ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
220652023-09-22T23:28:36.990ZWARNcrucible: [0] pm_task rx.recv() is None
22066 [8810] Exited with: ExitStatus(unix_wait_status(9))
22067 [8810][0] reports Exit
220682023-09-22T23:28:38.991ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
22069 Sep 22 23:28:40.991 INFO request completed, latency_us: 260, response_code: 200, uri: /state/cid/0, method: GET, req_id: bf7517eb-af0b-4546-88de-fcf03d2cbef2, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
22070 001/300 Read block 2357 len 16384
22071 002/300 Write block 1931 len 12288 data: 4 4 4
22072 003/300 Write block 904 len 4096 data: 3
22073 004/300 Read block 1390 len 12288
22074 005/300 Read block 2386 len 28672
22075 006/300 Read block 1898 len 32768
22076 007/300 Read block 2686 len 20480
22077 008/300 Flush
22078 009/300 Read block 4261 len 20480
22079 010/300 Read block 4785 len 4096
22080 011/300 Write block 3521 len 4096 data: 7
22081 012/300 Flush
22082 013/300 Read block 2106 len 20480
22083 014/300 Read block 3248 len 36864
22084 015/300 Read block 2633 len 36864
22085 016/300 Write block 4245 len 28672 data: 8 9 10 9 8 7 6
22086 017/300 Write block 716 len 28672 data: 3 3 3 3 4 4 4
22087 018/300 Read block 303 len 20480
22088 019/300 Read block 4090 len 32768
22089 020/300 Read block 2841 len 12288
22090 021/300 Write block 2099 len 4096 data: 4
22091 022/300 Read block 4991 len 8192
22092 023/300 Read block 3470 len 8192
22093 024/300 Write block 856 len 36864 data: 7 7 3 3 3 2 4 4 4
22094 025/300 Write block 2058 len 4096 data: 5
22095 026/300 Read block 2934 len 28672
22096 027/300 Write block 15 len 16384 data: 4 6 7 7
22097 028/300 Write block 1717 len 4096 data: 2
22098 029/300 Read block 1212 len 24576
22099 030/300 Read block 2955 len 28672
22100 031/300 Flush
22101 032/300 Read block 1417 len 20480
22102 033/300 Write block 1443 len 24576 data: 5 4 4 4 5 7
22103 034/300 Flush
22104 035/300 Write block 1879 len 8192 data: 6 7
22105 036/300 Read block 3569 len 28672
22106 037/300 Write block 785 len 12288 data: 7 7 5
22107 038/300 Flush
22108 039/300 Read block 1991 len 32768
22109 040/300 Read block 1500 len 8192
22110 041/300 Read block 4986 len 4096
22111 042/300 Read block 3648 len 16384
22112 043/300 Read block 357 len 4096
22113 044/300 Write block 4592 len 16384 data: 5 4 4 4
22114 045/300 Write block 4791 len 8192 data: 5 5
22115 046/300 Read block 1515 len 24576
22116 047/300 Read block 4448 len 40960
22117 048/300 Write block 4663 len 8192 data: 6 5
22118 049/300 Write block 1080 len 12288 data: 5 5 5
22119 050/300 Read block 3486 len 36864
22120 051/300 Read block 2147 len 36864
22121 052/300 Read block 4465 len 28672
22122 053/300 Read block 3423 len 32768
22123 054/300 Read block 1466 len 12288
22124 055/300 Read block 3105 len 4096
22125 056/300 Read block 2490 len 16384
22126 057/300 Write block 2167 len 24576 data: 4 4 4 3 3 4
22127 058/300 Write block 4261 len 4096 data: 4
22128 059/300 Read block 2828 len 40960
22129 060/300 Flush
22130 061/300 Write block 4337 len 28672 data: 6 4 3 2 2 2 2
22131 062/300 Write block 1239 len 4096 data: 6
22132 063/300 Read block 4195 len 16384
22133 064/300 Read block 4620 len 20480
22134 065/300 Write block 2239 len 40960 data: 4 4 4 3 3 4 3 3 4 5
22135 066/300 Write block 3197 len 32768 data: 5 3 4 4 4 3 3 2
22136 067/300 Read block 189 len 40960
22137 068/300 Read block 2772 len 28672
22138 069/300 Read block 4432 len 40960
22139 070/300 Write block 3732 len 8192 data: 3 4
22140 071/300 Read block 2301 len 16384
22141 072/300 Read block 4955 len 8192
22142 073/300 Read block 843 len 12288
22143 074/300 Read block 578 len 16384
22144 075/300 Read block 2348 len 40960
22145 076/300 Write block 4401 len 24576 data: 6 8 7 6 5 5
22146 077/300 Read block 943 len 24576
22147 078/300 Write block 3239 len 36864 data: 2 3 3 3 3 4 5 4 4
22148 079/300 Flush
22149 080/300 Read block 4275 len 20480
22150 081/300 Read block 3374 len 16384
22151 082/300 Write block 2440 len 16384 data: 6 6 5 4
22152 083/300 Write block 4443 len 4096 data: 6
22153 084/300 Read block 1314 len 4096
22154 085/300 Write block 158 len 32768 data: 5 4 6 6 7 6 4 6
22155 086/300 Read block 1123 len 20480
22156 087/300 Read block 680 len 8192
22157 088/300 Write block 2484 len 32768 data: 6 4 3 2 2 4 4 5
22158 089/300 Write block 2031 len 32768 data: 6 7 6 6 7 6 4 4
22159 090/300 Read block 3313 len 40960
22160 091/300 Read block 284 len 32768
22161 092/300 Read block 2917 len 32768
22162 093/300 Write block 3990 len 40960 data: 7 6 5 5 6 5 5 7 7 7
22163 094/300 Write block 1558 len 12288 data: 6 5 4
22164 095/300 Read block 4005 len 36864
22165 096/300 Read block 3625 len 32768
22166 097/300 Flush
22167 098/300 Write block 3466 len 8192 data: 7 7
22168 099/300 Write block 3248 len 12288 data: 3 6 5
22169 100/300 Write block 652 len 12288 data: 4 4 4
22170 101/300 Flush
22171 102/300 Write block 508 len 8192 data: 6 5
22172 103/300 Read block 683 len 16384
22173 104/300 Write block 420 len 32768 data: 7 6 5 5 5 4 4 4
22174 105/300 Write block 2597 len 24576 data: 4 5 5 4 4 2
22175 106/300 Write block 981 len 16384 data: 4 5 5 5
22176 107/300 Flush
22177 108/300 Write block 446 len 4096 data: 4
22178 109/300 Read block 4527 len 4096
22179 110/300 Read block 2563 len 12288
22180 111/300 Read block 960 len 28672
22181 112/300 Write block 4995 len 16384 data: 3 3 4 3
22182 113/300 Write block 1456 len 4096 data: 3
22183 114/300 Read block 3247 len 8192
22184 115/300 Write block 1113 len 28672 data: 3 3 4 3 3 4 4
22185 116/300 Read block 207 len 28672
22186 117/300 Read block 1791 len 24576
22187 118/300 Read block 118 len 4096
22188 119/300 Read block 4128 len 20480
22189 120/300 Write block 1727 len 20480 data: 4 5 7 6 6
22190 121/300 Read block 1979 len 28672
22191 122/300 Read block 3535 len 8192
22192 123/300 Read block 971 len 8192
22193 124/300 Write block 2830 len 20480 data: 10 9 11 11 8
22194 125/300 Read block 4978 len 8192
22195 126/300 Flush
22196 127/300 Write block 1442 len 36864 data: 4 6 5 5 5 6 8 5 5
22197 128/300 Write block 1889 len 28672 data: 7 6 5 4 6 7 6
22198 129/300 Read block 2849 len 12288
22199 130/300 Read block 4126 len 4096
22200 131/300 Write block 1492 len 36864 data: 4 5 5 6 7 9 8 8 9
22201 132/300 Read block 1322 len 4096
22202 133/300 Read block 3213 len 20480
22203 134/300 Write block 4992 len 20480 data: 11 8 5 4 4
22204 135/300 Read block 2892 len 24576
22205 136/300 Read block 4720 len 28672
22206 137/300 Read block 1725 len 12288
22207 138/300 Read block 400 len 40960
22208 139/300 Write block 1794 len 8192 data: 8 10
22209 140/300 Write block 2748 len 24576 data: 3 4 4 3 3 2
22210 141/300 Write block 2051 len 20480 data: 4 4 4 4 4
22211 142/300 Read block 3791 len 4096
22212 143/300 Write block 3946 len 20480 data: 2 2 3 3 3
22213 144/300 Write block 2698 len 12288 data: 5 6 6
22214 145/300 Read block 2275 len 24576
22215 146/300 Read block 3260 len 20480
22216 147/300 Write block 1520 len 12288 data: 3 4 4
22217 148/300 Read block 532 len 36864
22218 149/300 Read block 1858 len 16384
22219 150/300 Read block 857 len 12288
22220 151/300 Read block 4393 len 28672
22221 152/300 Read block 3429 len 32768
22222 153/300 Read block 659 len 40960
22223 154/300 Read block 3726 len 32768
22224 155/300 Write block 2636 len 12288 data: 4 4 4
22225 156/300 Write block 777 len 32768 data: 5 6 6 5 6 6 7 7
22226 157/300 Write block 3470 len 4096 data: 4
22227 158/300 Read block 1969 len 8192
22228 159/300 Write block 3017 len 40960 data: 4 4 5 4 4 5 5 7 8 10
22229 160/300 Read block 1736 len 16384
22230 161/300 Write block 360 len 16384 data: 6 5 4 4
22231 162/300 Read block 212 len 16384
22232 163/300 Read block 2285 len 20480
22233 164/300 Read block 3824 len 20480
22234 165/300 Write block 4141 len 40960 data: 5 5 6 4 4 5 7 7 7 7
22235 166/300 Write block 3435 len 24576 data: 5 6 5 4 5 5
22236 167/300 Write block 1722 len 16384 data: 2 2 2 2
22237 168/300 Write block 4457 len 20480 data: 10 8 8 6 6
22238 169/300 Read block 2297 len 4096
22239 170/300 Write block 1957 len 32768 data: 9 4 3 4 5 5 6 5
22240 171/300 Read block 1393 len 36864
22241 172/300 Write block 94 len 28672 data: 3 4 4 4 6 4 2
22242 173/300 Write block 2211 len 32768 data: 3 3 3 4 5 4 4 4
22243 174/300 Read block 4324 len 16384
22244 175/300 Write block 4410 len 28672 data: 5 5 6 6 6 6 7
22245 176/300 Write block 4630 len 40960 data: 4 3 4 3 2 2 2 3 4 4
22246 177/300 Write block 1454 len 28672 data: 3 2 4 4 4 5 6
22247 178/300 Read block 4260 len 16384
22248 179/300 Read block 3117 len 20480
22249 180/300 Write block 2440 len 8192 data: 7 7
22250 181/300 Write block 2180 len 16384 data: 5 6 8 7
22251 182/300 Read block 2872 len 24576
22252 183/300 Read block 4442 len 24576
22253 184/300 Read block 3150 len 20480
22254 185/300 Read block 1847 len 16384
22255 186/300 Read block 1833 len 28672
22256 187/300 Flush
22257 188/300 Flush
22258 189/300 Read block 1544 len 36864
22259 190/300 Write block 2921 len 16384 data: 4 6 6 8
22260 191/300 Read block 1095 len 16384
22261 192/300 Read block 37 len 40960
22262 193/300 Read block 140 len 8192
22263 194/300 Read block 1488 len 8192
22264 195/300 Flush
22265 196/300 Write block 873 len 40960 data: 9 9 9 10 8 6 6 7 8 6
22266 197/300 Write block 150 len 8192 data: 10 8
22267 198/300 Flush
22268 199/300 Read block 2198 len 28672
22269 200/300 Read block 1378 len 16384
22270 201/300 Flush
22271 202/300 Read block 684 len 12288
22272 203/300 Read block 2117 len 4096
22273 204/300 Read block 484 len 12288
22274 205/300 Flush
22275 206/300 Read block 1863 len 16384
22276 207/300 Flush
22277 208/300 Write block 2702 len 4096 data: 5
22278 209/300 Write block 3484 len 8192 data: 4 3
22279 210/300 Write block 2279 len 4096 data: 7
22280 211/300 Read block 3443 len 20480
22281 212/300 Read block 1697 len 40960
22282 213/300 Flush
22283 214/300 Read block 1688 len 8192
22284 215/300 Write block 1661 len 20480 data: 4 4 4 3 3
22285 216/300 Flush
22286 217/300 Write block 1268 len 20480 data: 7 7 7 7 6
22287 218/300 Read block 1103 len 24576
22288 219/300 Read block 816 len 16384
22289 220/300 Write block 3209 len 16384 data: 5 6 5 4
22290 221/300 Read block 1287 len 20480
22291 222/300 Write block 4833 len 24576 data: 3 2 2 2 4 5
22292 223/300 Read block 3044 len 4096
22293 224/300 Read block 1902 len 4096
22294 225/300 Write block 378 len 32768 data: 6 6 6 6 7 7 8 7
22295 226/300 Read block 105 len 24576
22296 227/300 Write block 3849 len 40960 data: 5 6 7 8 8 7 6 5 5 6
22297 228/300 Read block 2057 len 12288
22298 229/300 Flush
22299 230/300 Flush
22300 231/300 Write block 1599 len 12288 data: 4 4 4
22301 232/300 Write block 2383 len 40960 data: 4 4 4 4 3 3 3 5 5 4
22302 233/300 Flush
22303 234/300 Flush
22304 235/300 Read block 3006 len 4096
22305 236/300 Write block 2993 len 36864 data: 4 4 4 4 5 4 5 5 5
22306 237/300 Write block 4739 len 4096 data: 4
22307 238/300 Read block 1090 len 40960
22308 239/300 Read block 1490 len 20480
22309 240/300 Read block 3489 len 20480
22310 241/300 Flush
22311 242/300 Read block 4319 len 8192
22312 243/300 Read block 3554 len 16384
22313 244/300 Write block 3138 len 12288 data: 2 2 3
22314 245/300 Read block 133 len 32768
22315 246/300 Write block 4192 len 4096 data: 8
22316 247/300 Read block 4600 len 40960
22317 248/300 Write block 955 len 24576 data: 3 2 2 4 5 6
22318 249/300 Write block 35 len 20480 data: 6 6 6 6 6
22319 250/300 Flush
22320 251/300 Read block 4780 len 20480
22321 252/300 Read block 1590 len 20480
22322 253/300 Flush
22323 254/300 Write block 3697 len 4096 data: 4
22324 255/300 Read block 1202 len 8192
22325 256/300 Read block 1141 len 32768
22326 257/300 Write block 298 len 12288 data: 6 7 8
22327 258/300 Read block 4065 len 8192
22328 259/300 Read block 1618 len 36864
22329 260/300 Read block 790 len 16384
22330 261/300 Read block 3533 len 32768
22331 262/300 Read block 2174 len 28672
22332 263/300 Write block 2064 len 28672 data: 5 5 6 6 5 6 7
22333 264/300 Flush
22334 265/300 Write block 3437 len 20480 data: 6 5 6 6 3
22335 266/300 Read block 118 len 12288
22336 267/300 Read block 2136 len 32768
22337 268/300 Write block 4905 len 4096 data: 7
22338 269/300 Read block 4743 len 8192
22339 270/300 Read block 1345 len 4096
22340 271/300 Write block 2556 len 28672 data: 7 7 5 3 2 2 2
22341 272/300 Write block 143 len 40960 data: 5 6 7 9 9 10 10 11 9 6
22342 273/300 Write block 4180 len 4096 data: 6
22343 274/300 Write block 3126 len 40960 data: 9 7 7 5 4 4 4 5 4 4
22344 275/300 Read block 2556 len 36864
22345 276/300 Write block 2791 len 8192 data: 6 5
22346 277/300 Flush
22347 278/300 Flush
22348 279/300 Read block 1489 len 12288
22349 280/300 Read block 1629 len 4096
22350 281/300 Write block 804 len 12288 data: 7 7 8
22351 282/300 Read block 1294 len 32768
22352 283/300 Write block 711 len 40960 data: 4 3 4 4 3 4 4 4 4 5
22353 284/300 Write block 4418 len 32768 data: 7 8 9 9 8 7 7 6
22354 285/300 Write block 3873 len 36864 data: 6 5 7 6 5 4 4 4 4
22355 286/300 Write block 2724 len 40960 data: 4 5 6 6 6 5 5 4 6 6
22356 287/300 Write block 2891 len 12288 data: 4 4 4
22357 288/300 Read block 4092 len 24576
22358 289/300 Write block 4085 len 24576 data: 7 7 6 5 4 4
22359 290/300 Write block 207 len 12288 data: 5 6 4
22360 291/300 Write block 4188 len 8192 data: 4 5
22361 292/300 Write block 743 len 28672 data: 4 3 3 3 3 4 2
22362 293/300 Write block 1971 len 40960 data: 2 2 4 3 3 3 6 5 5 6
22363 294/300 Read block 92 len 8192
22364 295/300 Read block 1402 len 16384
22365 296/300 Flush
22366 297/300 Read block 1317 len 4096
22367 298/300 Flush
22368 299/300 Read block 1034 len 12288
22369 300/300 Write block 299 len 32768 data: 8 9 7 5 5 7 7 7
22370 Main task has work to do, go find it
22371 got dsc Start(0)
22372 start 0
22373 Sep 22 23:28:41.878 INFO request completed, latency_us: 303, response_code: 204, uri: /start/cid/0, method: POST, req_id: 665c86fb-9025-434b-af89-8037d78d7f6f, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
22374 Replay: started 0, returned:Ok(())
22375 ----------------------------------------------------------------
22376 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22377 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
22378 7260 Acked 8259 Read 4 New Done Done false
22379 7261 Acked 8260 Write 3 New Done Done false
22380 7262 Acked 8261 Write 1 New Done Done false
22381 7263 Acked 8262 Read 3 New Done Done false
22382 7264 Acked 8263 Read 7 New Done Done false
22383 7265 Acked 8264 Read 8 New Done Done false
22384 7266 Acked 8265 Read 5 New Done Done false
22385 7267 Acked 8266 Flush 0 New Done Done false
22386 7268 Acked 8267 Read 5 New Done Done false
22387 7269 Acked 8268 Read 1 New Done Done false
22388 7270 Acked 8269 Write 1 New Done Done false
22389 7271 Acked 8270 Flush 0 New Done Done false
22390 7272 Acked 8271 Read 5 New Done Done false
22391 7273 Acked 8272 Read 9 New Done Done false
22392 7274 Acked 8273 Read 9 New Done Done false
22393 7275 Acked 8274 Write 7 New Done Done false
22394 7276 Acked 8275 Write 7 New Done Done false
22395 7277 Acked 8276 Read 5 New Done Done false
22396 7278 Acked 8277 Read 8 New Done Done false
22397 7279 Acked 8278 Read 3 New Done Done false
22398 7280 Acked 8279 Write 1 New Done Done false
22399 7281 Acked 8280 Read 2 New Done Done false
22400 7282 Acked 8281 Read 2 New Done Done false
22401 7283 Acked 8282 Write 9 New Done Done false
22402 7284 Acked 8283 Write 1 New Done Done false
22403 7285 Acked 8284 Read 7 New Done Done false
22404 7286 Acked 8285 Write 4 New Done Done false
22405 7287 Acked 8286 Write 1 New Done Done false
22406 7288 Acked 8287 Read 6 New Done Done false
22407 7289 Acked 8288 Read 7 New Done Done false
22408 7290 Acked 8289 Flush 0 New Done Done false
22409 7291 Acked 8290 Read 5 New Done Done false
22410 7292 Acked 8291 Write 6 New Done Done false
22411 7293 Acked 8292 Flush 0 New Done Done false
22412 7294 Acked 8293 Write 2 New Done Done false
22413 7295 Acked 8294 Read 7 New Done Done false
22414 7296 Acked 8295 Write 3 New Done Done false
22415 7297 Acked 8296 Flush 0 New Done Done false
22416 7298 Acked 8297 Read 8 New Done Done false
22417 7299 Acked 8298 Flush 0 New Done Done false
22418 7300 Acked 8299 Read 2 New Done Done false
22419 7301 Acked 8300 Read 1 New Done Done false
22420 7302 Acked 8301 Read 4 New Done Done false
22421 7303 Acked 8302 Read 1 New Done Done false
22422 7304 Acked 8303 Write 4 New Done Done false
22423 7305 Acked 8304 Write 2 New Done Done false
22424 7306 Acked 8305 Read 6 New Done Done false
22425 7307 Acked 8306 Read 10 New Done Done false
22426 7308 Acked 8307 Write 2 New Done Done false
22427 7309 Acked 8308 Write 3 New Done Done false
22428 7310 Acked 8309 Read 9 New Done Done false
22429 7311 Acked 8310 Read 9 New Done Done false
22430 7312 Acked 8311 Read 7 New Done Done false
22431 7313 Acked 8312 Read 8 New Done Done false
22432 7314 Acked 8313 Read 3 New Done Done false
22433 7315 Acked 8314 Read 1 New Done Done false
22434 7316 Acked 8315 Read 4 New Done Done false
22435 7317 Acked 8316 Write 6 New Done Done false
22436 7318 Acked 8317 Write 1 New Done Done false
22437 7319 Acked 8318 Read 10 New Done Done false
22438 7320 Acked 8319 Flush 0 New Done Done false
22439 7321 Acked 8320 Write 7 New Done Done false
22440 7322 Acked 8321 Write 1 New Done Done false
22441 7323 Acked 8322 Read 4 New Done Done false
22442 7324 Acked 8323 Read 5 New Done Done false
22443 7325 Acked 8324 Write 10 New Done Done false
22444 7326 Acked 8325 Write 8 New Done Done false
22445 7327 Acked 8326 Read 10 New Done Done false
22446 7328 Acked 8327 Read 7 New Done Done false
22447 7329 Acked 8328 Read 10 New Done Done false
22448 7330 Acked 8329 Write 2 New Done Done false
22449 7331 Acked 8330 Read 4 New Done Done false
22450 7332 Acked 8331 Read 2 New Done Done false
22451 7333 Acked 8332 Read 3 New Done Done false
22452 7334 Acked 8333 Read 4 New Done Done false
22453 7335 Acked 8334 Read 10 New Done Done false
22454 7336 Acked 8335 Write 6 New Done Done false
22455 7337 Acked 8336 Read 6 New Done Done false
22456 7338 Acked 8337 Write 9 New Done Done false
22457 7339 Acked 8338 Flush 0 New Done Done false
22458 7340 Acked 8339 Read 5 New Done Done false
22459 7341 Acked 8340 Read 4 New Done Done false
22460 7342 Acked 8341 Write 4 New Done Done false
22461 7343 Acked 8342 Write 1 New Done Done false
22462 7344 Acked 8343 Read 1 New Done Done false
22463 7345 Acked 8344 Write 8 New Done Done false
22464 7346 Acked 8345 Read 5 New Done Done false
22465 7347 Acked 8346 Read 2 New Done Done false
22466 7348 Acked 8347 Write 8 New Done Done false
22467 7349 Acked 8348 Write 8 New Done Done false
22468 7350 Acked 8349 Read 10 New Done Done false
22469 7351 Acked 8350 Read 8 New Done Done false
22470 7352 Acked 8351 Read 8 New Done Done false
22471 7353 Acked 8352 Write 10 New Done Done false
22472 7354 Acked 8353 Write 3 New Done Done false
22473 7355 Acked 8354 Read 9 New Done Done false
22474 7356 Acked 8355 Read 8 New Done Done false
22475 7357 Acked 8356 Flush 0 New Done Done false
22476 7358 Acked 8357 Write 2 New Done Done false
22477 7359 Acked 8358 Write 3 New Done Done false
22478 7360 Acked 8359 Write 3 New Done Done false
22479 7361 Acked 8360 Flush 0 New Done Done false
22480 7362 Acked 8361 Write 2 New Done Done false
22481 7363 Acked 8362 Read 4 New Done Done false
22482 7364 Acked 8363 Write 8 New Done Done false
22483 7365 Acked 8364 Write 6 New Done Done false
22484 7366 Acked 8365 Write 4 New Done Done false
22485 7367 Acked 8366 Flush 0 New Done Done false
22486 7368 Acked 8367 Write 1 New Done Done false
22487 7369 Acked 8368 Read 1 New Done Done false
22488 7370 Acked 8369 Read 3 New Done Done false
22489 7371 Acked 8370 Read 7 New Done Done false
22490 7372 Acked 8371 Write 4 New Done Done false
22491 7373 Acked 8372 Write 1 New Done Done false
22492 7374 Acked 8373 Read 2 New Done Done false
22493 7375 Acked 8374 Write 7 New Done Done false
22494 7376 Acked 8375 Read 7 New Done Done false
22495 7377 Acked 8376 Read 6 New Done Done false
22496 7378 Acked 8377 Read 1 New Done Done false
22497 7379 Acked 8378 Read 5 New Done Done false
22498 7380 Acked 8379 Write 5 New Done Done false
22499 7381 Acked 8380 Read 7 New Done Done false
22500 7382 Acked 8381 Read 2 New Done Done false
22501 7383 Acked 8382 Read 2 New Done Done false
22502 7384 Acked 8383 Write 5 New Done Done false
22503 7385 Acked 8384 Read 2 New Done Done false
22504 7386 Acked 8385 Flush 0 New Done Done false
22505 7387 Acked 8386 Write 9 New Done Done false
22506 7388 Acked 8387 Write 7 New Done Done false
22507 7389 Acked 8388 Read 3 New Done Done false
22508 7390 Acked 8389 Read 1 New Done Done false
22509 7391 Acked 8390 Write 9 New Done Done false
22510 7392 Acked 8391 Read 1 New Done Done false
22511 7393 Acked 8392 Read 5 New Done Done false
22512 7394 Acked 8393 Write 5 New Done Done false
22513 7395 Acked 8394 Read 6 New Done Done false
22514 7396 Acked 8395 Read 7 New Done Done false
22515 7397 Acked 8396 Read 3 New Done Done false
22516 7398 Acked 8397 Read 10 New Done Done false
22517 7399 Acked 8398 Write 2 New Done Done false
22518 7400 Acked 8399 Write 6 New Done Done false
22519 7401 Acked 8400 Write 5 New Done Done false
22520 7402 Acked 8401 Read 1 New Done Done false
22521 7403 Acked 8402 Write 5 New Done Done false
22522 7404 Acked 8403 Write 3 New Done Done false
22523 7405 Acked 8404 Read 6 New Done Done false
22524 7406 Acked 8405 Read 5 New Done Done false
22525 7407 Acked 8406 Write 3 New Done Done false
22526 7408 Acked 8407 Read 9 New Done Done false
22527 7409 Acked 8408 Read 4 New Done Done false
22528 7410 Acked 8409 Read 3 New Done Done false
22529 7411 Acked 8410 Read 7 New Done Done false
22530 7412 Acked 8411 Read 8 New Done Done false
22531 7413 Acked 8412 Read 10 New Done Done false
22532 7414 Acked 8413 Read 8 New Done Done false
22533 7415 Acked 8414 Write 3 New Done Done false
22534 7416 Acked 8415 Write 8 New Done Done false
22535 7417 Acked 8416 Write 1 New Done Done false
22536 7418 Acked 8417 Read 2 New Done Done false
22537 7419 Acked 8418 Write 10 New Done Done false
22538 7420 Acked 8419 Read 4 New Done Done false
22539 7421 Acked 8420 Write 4 New Done Done false
22540 7422 Acked 8421 Read 4 New Done Done false
22541 7423 Acked 8422 Read 5 New Done Done false
22542 7424 Acked 8423 Read 5 New Done Done false
22543 7425 Acked 8424 Write 10 New Done Done false
22544 7426 Acked 8425 Write 6 New Done Done false
22545 7427 Acked 8426 Write 4 New Done Done false
22546 7428 Acked 8427 Write 5 New Done Done false
22547 7429 Acked 8428 Read 1 New Done Done false
22548 7430 Acked 8429 Write 8 New Done Done false
22549 7431 Acked 8430 Read 9 New Done Done false
22550 7432 Acked 8431 Write 7 New Done Done false
22551 7433 Acked 8432 Write 8 New Done Done false
22552 7434 Acked 8433 Read 4 New Done Done false
22553 7435 Acked 8434 Write 7 New Done Done false
22554 7436 Acked 8435 Write 10 New Done Done false
22555 7437 Acked 8436 Write 7 New Done Done false
22556 7438 Acked 8437 Read 4 New Done Done false
22557 7439 Acked 8438 Read 5 New Done Done false
22558 7440 Acked 8439 Write 2 New Done Done false
22559 7441 Acked 8440 Write 4 New Done Done false
22560 7442 Acked 8441 Read 6 New Done Done false
22561 7443 Acked 8442 Read 6 New Done Done false
22562 7444 Acked 8443 Read 5 New Done Done false
22563 7445 Acked 8444 Read 4 New Done Done false
22564 7446 Acked 8445 Read 7 New Done Done false
22565 7447 Acked 8446 Flush 0 New Done Done false
22566 7448 Acked 8447 Flush 0 New Done Done false
22567 7449 Acked 8448 Read 9 New Done Done false
22568 7450 Acked 8449 Write 4 New Done Done false
22569 7451 Acked 8450 Read 4 New Done Done false
22570 7452 Acked 8451 Read 10 New Done Done false
22571 7453 Acked 8452 Read 2 New Done Done false
22572 7454 Acked 8453 Read 2 New Done Done false
22573 7455 Acked 8454 Flush 0 New Done Done false
22574 7456 Acked 8455 Write 10 New Done Done false
22575 7457 Acked 8456 Write 2 New Done Done false
22576 7458 Acked 8457 Flush 0 New Done Done false
22577 7459 Acked 8458 Read 7 New Done Done false
22578 7460 Acked 8459 Read 4 New Done Done false
22579 7461 Acked 8460 Flush 0 New Done Done false
22580 7462 Acked 8461 Read 3 New Done Done false
22581 7463 Acked 8462 Flush 0 New Done Done false
22582 7464 Acked 8463 Read 1 New Done Done false
22583 7465 Acked 8464 Read 3 New Done Done false
22584 7466 Acked 8465 Flush 0 New Done Done false
22585 7467 Acked 8466 Read 4 New Done Done false
22586 7468 Acked 8467 Flush 0 New Done Done false
22587 7469 Acked 8468 Write 1 New Done Done false
22588 7470 Acked 8469 Write 2 New Done Done false
22589 7471 Acked 8470 Write 1 New Done Done false
22590 7472 Acked 8471 Read 5 New Done Done false
22591 7473 Acked 8472 Read 10 New Done Done false
22592 7474 Acked 8473 Flush 0 New Done Done false
22593 7475 Acked 8474 Read 2 New Done Done false
22594 7476 Acked 8475 Write 5 New Done Done false
22595 7477 Acked 8476 Flush 0 New Done Done false
22596 7478 Acked 8477 Write 5 New Done Done false
22597 7479 Acked 8478 Read 6 New Done Done false
22598 7480 Acked 8479 Read 4 New Done Done false
22599 7481 Acked 8480 Write 4 New Done Done false
22600 7482 Acked 8481 Read 5 New Done Done false
22601 7483 Acked 8482 Write 6 New Done Done false
22602 7484 Acked 8483 Read 1 New Done Done false
22603 7485 Acked 8484 Read 1 New Done Done false
22604 7486 Acked 8485 Write 8 New Done Done false
22605 7487 Acked 8486 Read 6 New Done Done false
22606 7488 Acked 8487 Write 10 New Done Done false
22607 7489 Acked 8488 Read 3 New Done Done false
22608 7490 Acked 8489 Flush 0 New Done Done false
22609 7491 Acked 8490 Flush 0 New Done Done false
22610 7492 Acked 8491 Write 3 New Done Done false
22611 7493 Acked 8492 Write 10 New Done Done false
22612 7494 Acked 8493 Flush 0 New Done Done false
22613 7495 Acked 8494 Flush 0 New Done Done false
22614 7496 Acked 8495 Read 1 New Done Done false
22615 7497 Acked 8496 Write 9 New Done Done false
22616 7498 Acked 8497 Write 1 New Done Done false
22617 7499 Acked 8498 Read 10 New Done Done false
22618 7500 Acked 8499 Read 5 New Done Done false
22619 7501 Acked 8500 Read 5 New Done Done false
22620 7502 Acked 8501 Flush 0 New Done Done false
22621 7503 Acked 8502 Read 2 New Done Done false
22622 7504 Acked 8503 Read 4 New Done Done false
22623 7505 Acked 8504 Write 3 New Done Done false
22624 7506 Acked 8505 Read 8 New Done Done false
22625 7507 Acked 8506 Write 1 New Done Done false
22626 7508 Acked 8507 Read 10 New Done Done false
22627 7509 Acked 8508 Write 6 New Done Done false
22628 7510 Acked 8509 Write 5 New Done Done false
22629 7511 Acked 8510 Flush 0 New Done Done false
22630 7512 Acked 8511 Read 5 New Done Done false
22631 7513 Acked 8512 Read 5 New Done Done false
22632 7514 Acked 8513 Flush 0 New Done Done false
22633 7515 Acked 8514 Write 1 New Done Done false
22634 7516 Acked 8515 Read 2 New Done Done false
22635 7517 Acked 8516 Read 8 New Done Done false
22636 7518 Acked 8517 Write 3 New Done Done false
22637 7519 Acked 8518 Read 2 New Done Done false
22638 7520 Acked 8519 Read 9 New Done Done false
22639 7521 Acked 8520 Read 4 New Done Done false
22640 7522 Acked 8521 Read 8 New Done Done false
22641 7523 Acked 8522 Read 7 New Done Done false
22642 7524 Acked 8523 Write 7 New Done Done false
22643 7525 Acked 8524 Flush 0 New Done Done false
22644 7526 Acked 8525 Write 5 New Done Done false
22645 7527 Acked 8526 Read 3 New Done Done false
22646 7528 Acked 8527 Read 8 New Done Done false
22647 7529 Acked 8528 Write 1 New Done Done false
22648 7530 Acked 8529 Read 2 New Done Done false
22649 7531 Acked 8530 Read 1 New Done Done false
22650 7532 Acked 8531 Write 7 New Done Done false
22651 7533 Acked 8532 Write 10 New Done Done false
22652 7534 Acked 8533 Write 1 New Done Done false
22653 7535 Acked 8534 Write 10 New Done Done false
22654 7536 Acked 8535 Read 9 New Done Done false
22655 7537 Acked 8536 Write 2 New Done Done false
22656 7538 Acked 8537 Flush 0 New Done Done false
22657 7539 Acked 8538 Flush 0 New Done Done false
22658 7540 Acked 8539 Read 3 New Done Done false
22659 7541 Acked 8540 Read 1 New Done Done false
22660 7542 Acked 8541 Write 3 New Done Done false
22661 7543 Acked 8542 Read 8 New Done Done false
22662 7544 Acked 8543 Write 10 New Done Done false
22663 7545 Acked 8544 Write 8 New Done Done false
22664 7546 Acked 8545 Write 9 New Done Done false
22665 7547 Acked 8546 Write 10 New Done Done false
22666 7548 Acked 8547 Write 3 New Done Done false
22667 7549 Acked 8548 Read 6 New Done Done false
22668 7550 Acked 8549 Write 6 New Done Done false
22669 7551 Acked 8550 Write 3 New Done Done false
22670 7552 Acked 8551 Write 2 New Done Done false
22671 7553 Acked 8552 Write 7 New Done Done false
22672 7554 Acked 8553 Write 10 New Done Done false
22673 7555 Acked 8554 Read 2 New Done Done false
22674 7556 Acked 8555 Read 4 New Done Done false
22675 7557 Acked 8556 Flush 0 New Done Done false
22676 7558 Acked 8557 Read 1 New Done Done false
22677 7559 Acked 8558 Flush 0 New Done Done false
22678 7560 Acked 8559 Read 3 New Done Done false
22679 7561 Acked 8560 Write 8 New Sent Sent false
22680 STATES DS:0 DS:1 DS:2 TOTAL
22681 New 302 0 0 302
22682 Sent 0 1 1 2
22683 Done 0 301 301 602
22684 Skipped 0 0 0 0
22685 Error 0 0 0 0
22686 Last Flush: 8258 8558 8558
22687 Downstairs last five completed: 8258 8257 8256 8255 8254 8253
22688 Upstairs last five completed: 7561 7560 7559 7558 7557 7556
22689 CLIENT: Up:0 ds:302 act:2
22690 [8810] got start action
22691 [8810] I am going to restart
22692 Starting downstairs at port 8810
22693 Make output file at "/tmp/dsc/downstairs-8810.txt"
22694 [8810][0] reports Starting
22695 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1198)
22696 [8810][0] reports Running
22697 ----------------------------------------------------------------
22698 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
22699 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
22700 7260 Acked 8259 Read 4 New Done Done false
22701 7261 Acked 8260 Write 3 New Done Done false
22702 7262 Acked 8261 Write 1 New Done Done false
22703 7263 Acked 8262 Read 3 New Done Done false
22704 7264 Acked 8263 Read 7 New Done Done false
22705 7265 Acked 8264 Read 8 New Done Done false
22706 7266 Acked 8265 Read 5 New Done Done false
22707 7267 Acked 8266 Flush 0 New Done Done false
22708 7268 Acked 8267 Read 5 New Done Done false
22709 7269 Acked 8268 Read 1 New Done Done false
22710 7270 Acked 8269 Write 1 New Done Done false
22711 7271 Acked 8270 Flush 0 New Done Done false
22712 7272 Acked 8271 Read 5 New Done Done false
22713 7273 Acked 8272 Read 9 New Done Done false
22714 7274 Acked 8273 Read 9 New Done Done false
22715 7275 Acked 8274 Write 7 New Done Done false
22716 7276 Acked 8275 Write 7 New Done Done false
22717 7277 Acked 8276 Read 5 New Done Done false
22718 7278 Acked 8277 Read 8 New Done Done false
22719 7279 Acked 8278 Read 3 New Done Done false
22720 7280 Acked 8279 Write 1 New Done Done false
22721 7281 Acked 8280 Read 2 New Done Done false
22722 7282 Acked 8281 Read 2 New Done Done false
22723 7283 Acked 8282 Write 9 New Done Done false
22724 7284 Acked 8283 Write 1 New Done Done false
22725 7285 Acked 8284 Read 7 New Done Done false
22726 7286 Acked 8285 Write 4 New Done Done false
22727 7287 Acked 8286 Write 1 New Done Done false
22728 7288 Acked 8287 Read 6 New Done Done false
22729 7289 Acked 8288 Read 7 New Done Done false
22730 7290 Acked 8289 Flush 0 New Done Done false
22731 7291 Acked 8290 Read 5 New Done Done false
22732 7292 Acked 8291 Write 6 New Done Done false
22733 7293 Acked 8292 Flush 0 New Done Done false
22734 7294 Acked 8293 Write 2 New Done Done false
22735 7295 Acked 8294 Read 7 New Done Done false
22736 7296 Acked 8295 Write 3 New Done Done false
22737 7297 Acked 8296 Flush 0 New Done Done false
22738 7298 Acked 8297 Read 8 New Done Done false
22739 7299 Acked 8298 Flush 0 New Done Done false
22740 7300 Acked 8299 Read 2 New Done Done false
22741 7301 Acked 8300 Read 1 New Done Done false
22742 7302 Acked 8301 Read 4 New Done Done false
22743 7303 Acked 8302 Read 1 New Done Done false
22744 7304 Acked 8303 Write 4 New Done Done false
22745 7305 Acked 8304 Write 2 New Done Done false
22746 7306 Acked 8305 Read 6 New Done Done false
22747 7307 Acked 8306 Read 10 New Done Done false
22748 7308 Acked 8307 Write 2 New Done Done false
22749 7309 Acked 8308 Write 3 New Done Done false
22750 7310 Acked 8309 Read 9 New Done Done false
22751 7311 Acked 8310 Read 9 New Done Done false
22752 7312 Acked 8311 Read 7 New Done Done false
22753 7313 Acked 8312 Read 8 New Done Done false
22754 7314 Acked 8313 Read 3 New Done Done false
22755 7315 Acked 8314 Read 1 New Done Done false
22756 7316 Acked 8315 Read 4 New Done Done false
22757 7317 Acked 8316 Write 6 New Done Done false
22758 7318 Acked 8317 Write 1 New Done Done false
22759 7319 Acked 8318 Read 10 New Done Done false
22760 7320 Acked 8319 Flush 0 New Done Done false
22761 7321 Acked 8320 Write 7 New Done Done false
22762 7322 Acked 8321 Write 1 New Done Done false
22763 7323 Acked 8322 Read 4 New Done Done false
22764 7324 Acked 8323 Read 5 New Done Done false
22765 7325 Acked 8324 Write 10 New Done Done false
22766 7326 Acked 8325 Write 8 New Done Done false
22767 7327 Acked 8326 Read 10 New Done Done false
22768 7328 Acked 8327 Read 7 New Done Done false
22769 7329 Acked 8328 Read 10 New Done Done false
22770 7330 Acked 8329 Write 2 New Done Done false
22771 7331 Acked 8330 Read 4 New Done Done false
22772 7332 Acked 8331 Read 2 New Done Done false
22773 7333 Acked 8332 Read 3 New Done Done false
22774 7334 Acked 8333 Read 4 New Done Done false
22775 7335 Acked 8334 Read 10 New Done Done false
22776 7336 Acked 8335 Write 6 New Done Done false
22777 7337 Acked 8336 Read 6 New Done Done false
22778 7338 Acked 8337 Write 9 New Done Done false
22779 7339 Acked 8338 Flush 0 New Done Done false
22780 7340 Acked 8339 Read 5 New Done Done false
22781 7341 Acked 8340 Read 4 New Done Done false
22782 7342 Acked 8341 Write 4 New Done Done false
22783 7343 Acked 8342 Write 1 New Done Done false
22784 7344 Acked 8343 Read 1 New Done Done false
22785 7345 Acked 8344 Write 8 New Done Done false
22786 7346 Acked 8345 Read 5 New Done Done false
22787 7347 Acked 8346 Read 2 New Done Done false
22788 7348 Acked 8347 Write 8 New Done Done false
22789 7349 Acked 8348 Write 8 New Done Done false
22790 7350 Acked 8349 Read 10 New Done Done false
22791 7351 Acked 8350 Read 8 New Done Done false
22792 7352 Acked 8351 Read 8 New Done Done false
22793 7353 Acked 8352 Write 10 New Done Done false
22794 7354 Acked 8353 Write 3 New Done Done false
22795 7355 Acked 8354 Read 9 New Done Done false
22796 7356 Acked 8355 Read 8 New Done Done false
22797 7357 Acked 8356 Flush 0 New Done Done false
22798 7358 Acked 8357 Write 2 New Done Done false
22799 7359 Acked 8358 Write 3 New Done Done false
22800 7360 Acked 8359 Write 3 New Done Done false
22801 7361 Acked 8360 Flush 0 New Done Done false
22802 7362 Acked 8361 Write 2 New Done Done false
22803 7363 Acked 8362 Read 4 New Done Done false
22804 7364 Acked 8363 Write 8 New Done Done false
22805 7365 Acked 8364 Write 6 New Done Done false
22806 7366 Acked 8365 Write 4 New Done Done false
22807 7367 Acked 8366 Flush 0 New Done Done false
22808 7368 Acked 8367 Write 1 New Done Done false
22809 7369 Acked 8368 Read 1 New Done Done false
22810 7370 Acked 8369 Read 3 New Done Done false
22811 7371 Acked 8370 Read 7 New Done Done false
22812 7372 Acked 8371 Write 4 New Done Done false
22813 7373 Acked 8372 Write 1 New Done Done false
22814 7374 Acked 8373 Read 2 New Done Done false
22815 7375 Acked 8374 Write 7 New Done Done false
22816 7376 Acked 8375 Read 7 New Done Done false
22817 7377 Acked 8376 Read 6 New Done Done false
22818 7378 Acked 8377 Read 1 New Done Done false
22819 7379 Acked 8378 Read 5 New Done Done false
22820 7380 Acked 8379 Write 5 New Done Done false
22821 7381 Acked 8380 Read 7 New Done Done false
22822 7382 Acked 8381 Read 2 New Done Done false
22823 7383 Acked 8382 Read 2 New Done Done false
22824 7384 Acked 8383 Write 5 New Done Done false
22825 7385 Acked 8384 Read 2 New Done Done false
22826 7386 Acked 8385 Flush 0 New Done Done false
22827 7387 Acked 8386 Write 9 New Done Done false
22828 7388 Acked 8387 Write 7 New Done Done false
22829 7389 Acked 8388 Read 3 New Done Done false
22830 7390 Acked 8389 Read 1 New Done Done false
22831 7391 Acked 8390 Write 9 New Done Done false
22832 7392 Acked 8391 Read 1 New Done Done false
22833 7393 Acked 8392 Read 5 New Done Done false
22834 7394 Acked 8393 Write 5 New Done Done false
22835 7395 Acked 8394 Read 6 New Done Done false
22836 7396 Acked 8395 Read 7 New Done Done false
22837 7397 Acked 8396 Read 3 New Done Done false
22838 7398 Acked 8397 Read 10 New Done Done false
22839 7399 Acked 8398 Write 2 New Done Done false
22840 7400 Acked 8399 Write 6 New Done Done false
22841 7401 Acked 8400 Write 5 New Done Done false
22842 7402 Acked 8401 Read 1 New Done Done false
22843 7403 Acked 8402 Write 5 New Done Done false
22844 7404 Acked 8403 Write 3 New Done Done false
22845 7405 Acked 8404 Read 6 New Done Done false
22846 7406 Acked 8405 Read 5 New Done Done false
22847 7407 Acked 8406 Write 3 New Done Done false
22848 7408 Acked 8407 Read 9 New Done Done false
22849 7409 Acked 8408 Read 4 New Done Done false
22850 7410 Acked 8409 Read 3 New Done Done false
22851 7411 Acked 8410 Read 7 New Done Done false
22852 7412 Acked 8411 Read 8 New Done Done false
22853 7413 Acked 8412 Read 10 New Done Done false
22854 7414 Acked 8413 Read 8 New Done Done false
22855 7415 Acked 8414 Write 3 New Done Done false
22856 7416 Acked 8415 Write 8 New Done Done false
22857 7417 Acked 8416 Write 1 New Done Done false
22858 7418 Acked 8417 Read 2 New Done Done false
22859 7419 Acked 8418 Write 10 New Done Done false
22860 7420 Acked 8419 Read 4 New Done Done false
22861 7421 Acked 8420 Write 4 New Done Done false
22862 7422 Acked 8421 Read 4 New Done Done false
22863 7423 Acked 8422 Read 5 New Done Done false
22864 7424 Acked 8423 Read 5 New Done Done false
22865 7425 Acked 8424 Write 10 New Done Done false
22866 7426 Acked 8425 Write 6 New Done Done false
22867 7427 Acked 8426 Write 4 New Done Done false
22868 7428 Acked 8427 Write 5 New Done Done false
22869 7429 Acked 8428 Read 1 New Done Done false
22870 7430 Acked 8429 Write 8 New Done Done false
22871 7431 Acked 8430 Read 9 New Done Done false
22872 7432 Acked 8431 Write 7 New Done Done false
22873 7433 Acked 8432 Write 8 New Done Done false
22874 7434 Acked 8433 Read 4 New Done Done false
22875 7435 Acked 8434 Write 7 New Done Done false
22876 7436 Acked 8435 Write 10 New Done Done false
22877 7437 Acked 8436 Write 7 New Done Done false
22878 7438 Acked 8437 Read 4 New Done Done false
22879 7439 Acked 8438 Read 5 New Done Done false
22880 7440 Acked 8439 Write 2 New Done Done false
22881 7441 Acked 8440 Write 4 New Done Done false
22882 7442 Acked 8441 Read 6 New Done Done false
22883 7443 Acked 8442 Read 6 New Done Done false
22884 7444 Acked 8443 Read 5 New Done Done false
22885 7445 Acked 8444 Read 4 New Done Done false
22886 7446 Acked 8445 Read 7 New Done Done false
22887 7447 Acked 8446 Flush 0 New Done Done false
22888 7448 Acked 8447 Flush 0 New Done Done false
22889 7449 Acked 8448 Read 9 New Done Done false
22890 7450 Acked 8449 Write 4 New Done Done false
22891 7451 Acked 8450 Read 4 New Done Done false
22892 7452 Acked 8451 Read 10 New Done Done false
22893 7453 Acked 8452 Read 2 New Done Done false
22894 7454 Acked 8453 Read 2 New Done Done false
22895 7455 Acked 8454 Flush 0 New Done Done false
22896 7456 Acked 8455 Write 10 New Done Done false
22897 7457 Acked 8456 Write 2 New Done Done false
22898 7458 Acked 8457 Flush 0 New Done Done false
22899 7459 Acked 8458 Read 7 New Done Done false
22900 7460 Acked 8459 Read 4 New Done Done false
22901 7461 Acked 8460 Flush 0 New Done Done false
22902 7462 Acked 8461 Read 3 New Done Done false
22903 7463 Acked 8462 Flush 0 New Done Done false
22904 7464 Acked 8463 Read 1 New Done Done false
22905 7465 Acked 8464 Read 3 New Done Done false
22906 7466 Acked 8465 Flush 0 New Done Done false
22907 7467 Acked 8466 Read 4 New Done Done false
22908 7468 Acked 8467 Flush 0 New Done Done false
22909 7469 Acked 8468 Write 1 New Done Done false
22910 7470 Acked 8469 Write 2 New Done Done false
22911 7471 Acked 8470 Write 1 New Done Done false
22912 7472 Acked 8471 Read 5 New Done Done false
22913 7473 Acked 8472 Read 10 New Done Done false
22914 7474 Acked 8473 Flush 0 New Done Done false
22915 7475 Acked 8474 Read 2 New Done Done false
22916 7476 Acked 8475 Write 5 New Done Done false
22917 7477 Acked 8476 Flush 0 New Done Done false
22918 7478 Acked 8477 Write 5 New Done Done false
22919 7479 Acked 8478 Read 6 New Done Done false
22920 7480 Acked 8479 Read 4 New Done Done false
22921 7481 Acked 8480 Write 4 New Done Done false
22922 7482 Acked 8481 Read 5 New Done Done false
22923 7483 Acked 8482 Write 6 New Done Done false
22924 7484 Acked 8483 Read 1 New Done Done false
22925 7485 Acked 8484 Read 1 New Done Done false
22926 7486 Acked 8485 Write 8 New Done Done false
22927 7487 Acked 8486 Read 6 New Done Done false
22928 7488 Acked 8487 Write 10 New Done Done false
22929 7489 Acked 8488 Read 3 New Done Done false
22930 7490 Acked 8489 Flush 0 New Done Done false
22931 7491 Acked 8490 Flush 0 New Done Done false
22932 7492 Acked 8491 Write 3 New Done Done false
22933 7493 Acked 8492 Write 10 New Done Done false
22934 7494 Acked 8493 Flush 0 New Done Done false
22935 7495 Acked 8494 Flush 0 New Done Done false
22936 7496 Acked 8495 Read 1 New Done Done false
22937 7497 Acked 8496 Write 9 New Done Done false
22938 7498 Acked 8497 Write 1 New Done Done false
22939 7499 Acked 8498 Read 10 New Done Done false
22940 7500 Acked 8499 Read 5 New Done Done false
22941 7501 Acked 8500 Read 5 New Done Done false
22942 7502 Acked 8501 Flush 0 New Done Done false
22943 7503 Acked 8502 Read 2 New Done Done false
22944 7504 Acked 8503 Read 4 New Done Done false
22945 7505 Acked 8504 Write 3 New Done Done false
22946 7506 Acked 8505 Read 8 New Done Done false
22947 7507 Acked 8506 Write 1 New Done Done false
22948 7508 Acked 8507 Read 10 New Done Done false
22949 7509 Acked 8508 Write 6 New Done Done false
22950 7510 Acked 8509 Write 5 New Done Done false
22951 7511 Acked 8510 Flush 0 New Done Done false
22952 7512 Acked 8511 Read 5 New Done Done false
22953 7513 Acked 8512 Read 5 New Done Done false
22954 7514 Acked 8513 Flush 0 New Done Done false
22955 7515 Acked 8514 Write 1 New Done Done false
22956 7516 Acked 8515 Read 2 New Done Done false
22957 7517 Acked 8516 Read 8 New Done Done false
22958 7518 Acked 8517 Write 3 New Done Done false
22959 7519 Acked 8518 Read 2 New Done Done false
22960 7520 Acked 8519 Read 9 New Done Done false
22961 7521 Acked 8520 Read 4 New Done Done false
22962 7522 Acked 8521 Read 8 New Done Done false
22963 7523 Acked 8522 Read 7 New Done Done false
22964 7524 Acked 8523 Write 7 New Done Done false
22965 7525 Acked 8524 Flush 0 New Done Done false
22966 7526 Acked 8525 Write 5 New Done Done false
22967 7527 Acked 8526 Read 3 New Done Done false
22968 7528 Acked 8527 Read 8 New Done Done false
22969 7529 Acked 8528 Write 1 New Done Done false
22970 7530 Acked 8529 Read 2 New Done Done false
22971 7531 Acked 8530 Read 1 New Done Done false
22972 7532 Acked 8531 Write 7 New Done Done false
22973 7533 Acked 8532 Write 10 New Done Done false
22974 7534 Acked 8533 Write 1 New Done Done false
22975 7535 Acked 8534 Write 10 New Done Done false
22976 7536 Acked 8535 Read 9 New Done Done false
22977 7537 Acked 8536 Write 2 New Done Done false
22978 7538 Acked 8537 Flush 0 New Done Done false
22979 7539 Acked 8538 Flush 0 New Done Done false
22980 7540 Acked 8539 Read 3 New Done Done false
22981 7541 Acked 8540 Read 1 New Done Done false
22982 7542 Acked 8541 Write 3 New Done Done false
22983 7543 Acked 8542 Read 8 New Done Done false
22984 7544 Acked 8543 Write 10 New Done Done false
22985 7545 Acked 8544 Write 8 New Done Done false
22986 7546 Acked 8545 Write 9 New Done Done false
22987 7547 Acked 8546 Write 10 New Done Done false
22988 7548 Acked 8547 Write 3 New Done Done false
22989 7549 Acked 8548 Read 6 New Done Done false
22990 7550 Acked 8549 Write 6 New Done Done false
22991 7551 Acked 8550 Write 3 New Done Done false
22992 7552 Acked 8551 Write 2 New Done Done false
22993 7553 Acked 8552 Write 7 New Done Done false
22994 7554 Acked 8553 Write 10 New Done Done false
22995 7555 Acked 8554 Read 2 New Done Done false
22996 7556 Acked 8555 Read 4 New Done Done false
22997 7557 Acked 8556 Flush 0 New Done Done false
22998 7558 Acked 8557 Read 1 New Done Done false
22999 7559 Acked 8558 Flush 0 New Done Done false
23000 7560 Acked 8559 Read 3 New Done Done false
23001 7561 Acked 8560 Write 8 New Done Done false
23002 7562 Acked 8561 Flush 0 New Done Done false
23003 STATES DS:0 DS:1 DS:2 TOTAL
23004 New 303 0 0 303
23005 Sent 0 0 0 0
23006 Done 0 303 303 606
23007 Skipped 0 0 0 0
23008 Error 0 0 0 0
23009 Last Flush: 8258 8561 8561
23010 Downstairs last five completed: 8258 8257 8256 8255 8254 8253
23011 Upstairs last five completed: 7562 7561 7560 7559 7558 7557
23012 CLIENT: Up:0 ds:303 act:2
230132023-09-22T23:28:45.999ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
230142023-09-22T23:28:45.999ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
230152023-09-22T23:28:45.999ZINFOcrucible: [0] client re-new 303 jobs since flush 8258 = downstairs
230162023-09-22T23:28:45.999ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
230172023-09-22T23:28:46.000ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
230182023-09-22T23:28:46.000ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
230192023-09-22T23:28:46.000ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
230202023-09-22T23:28:46.000ZINFOcrucible: [0] send last flush ID to this DS: 8258
230212023-09-22T23:28:46.000ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
230222023-09-22T23:28:46.000ZINFOcrucible: [0] Replied this last flush ID: 8258
230232023-09-22T23:28:46.000ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
230242023-09-22T23:28:46.000ZINFOcrucible: [0] Transition from Offline to Replay
230252023-09-22T23:28:46.000ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
230262023-09-22T23:28:46.000ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
230272023-09-22T23:28:46.000ZINFOcrucible: [0] Transition from Replay to Active
230282023-09-22T23:28:46.001ZINFOcrucible: [0] Starts cmd_loop
230292023-09-22T23:28:46.001ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
230302023-09-22T23:28:46.001ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
230312023-09-22T23:28:46.001ZINFOcrucible: Set check for repair
230322023-09-22T23:28:47.002ZINFOcrucible: Checking if live repair is needed
230332023-09-22T23:28:47.002ZINFOcrucible: No Live Repair required at this time
230342023-09-22T23:28:47.002ZWARNcrucible: [0] flow control end
23035 ----------------------------------------------------------------
23036 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23037 Downstairs last five completed: 8561 8560 8559 8558 8557 8556
23038 Upstairs last five completed: 7562 7561 7560 7559 7558 7557
23039 CLIENT: Up:0 ds:0 act:3
23040 Replay: All jobs finished, all DS active.
23041 Main task has work to do, go find it
23042 got dsc Stop(0)
23043 stop 0
23044 Sep 22 23:28:49.886 INFO request completed, latency_us: 562, response_code: 204, uri: [8810] Got stop action so:false kr:false
23045 /stop/cid/0, method: POST, req_id: f1ad5c42-d8e5-4275-9d83-34e58ca16014, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
23046 Sep 22 23:28:49.887 INFO request completed, latency_us: 293, response_code: 200, uri: /state/cid/0, method: GET, req_id: e0dc804f-8402-494a-9735-ba3d0beab257, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
230472023-09-22T23:28:49.887ZWARNcrucible: [0] downstairs disconnected
230482023-09-22T23:28:49.887ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
230492023-09-22T23:28:49.888ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
230502023-09-22T23:28:49.888ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
230512023-09-22T23:28:49.888ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
230522023-09-22T23:28:49.888ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
230532023-09-22T23:28:49.888ZWARNcrucible: [0] pm_task rx.recv() is None
23054 [8810] Exited with: ExitStatus(unix_wait_status(9))
23055 [8810][0] reports Exit
230562023-09-22T23:28:52.891ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
23057 Sep 22 23:28:53.889 INFO request completed, latency_us: 262, response_code: 200, uri: /state/cid/0, method: GET, req_id: c262c410-5804-485c-ae74-9ad59d131958, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
23058 001/300 Write block 471 len 16384 data: 7 6 4 3
23059 002/300 Read block 759 len 24576
23060 003/300 Write block 2085 len 28672 data: 3 4 4 4 3 4 5
23061 004/300 Flush
23062 005/300 Read block 242 len 28672
23063 006/300 Read block 3516 len 36864
23064 007/300 Read block 1500 len 36864
23065 008/300 Write block 3370 len 40960 data: 6 5 5 3 3 3 3 3 3 3
23066 009/300 Read block 2967 len 8192
23067 010/300 Write block 2724 len 32768 data: 5 6 7 7 7 6 6 5
23068 011/300 Write block 4946 len 20480 data: 7 7 6 6 7
23069 012/300 Write block 1469 len 28672 data: 4 4 4 3 3 6 7
23070 013/300 Write block 2619 len 20480 data: 5 6 10 11 10
23071 014/300 Read block 2388 len 40960
23072 015/300 Write block 662 len 16384 data: 5 6 4 6
23073 016/300 Read block 2631 len 28672
23074 017/300 Read block 3879 len 32768
23075 018/300 Read block 1207 len 8192
23076 019/300 Write block 4378 len 12288 data: 6 6 6
23077 020/300 Write block 2497 len 8192 data: 5 4
23078 021/300 Flush
23079 022/300 Flush
23080 023/300 Read block 2610 len 40960
23081 024/300 Write block 4496 len 24576 data: 3 5 5 4 4 3
23082 025/300 Write block 2653 len 24576 data: 5 5 4 3 5 6
23083 026/300 Read block 3745 len 28672
23084 027/300 Write block 2592 len 28672 data: 4 3 3 2 4 5 6
23085 028/300 Read block 2379 len 28672
23086 029/300 Read block 3034 len 4096
23087 030/300 Write block 4035 len 16384 data: 7 7 7 6
23088 031/300 Read block 472 len 24576
23089 032/300 Write block 2138 len 20480 data: 7 8 6 5 6
23090 033/300 Flush
23091 034/300 Write block 4135 len 40960 data: 5 6 7 6 5 5 6 6 7 5
23092 035/300 Write block 273 len 8192 data: 7 6
23093 036/300 Read block 3921 len 36864
23094 037/300 Write block 3093 len 24576 data: 4 4 4 6 7 7
23095 038/300 Read block 4790 len 20480
23096 039/300 Flush
23097 040/300 Read block 3522 len 40960
23098 041/300 Read block 2178 len 24576
23099 042/300 Write block 1299 len 40960 data: 6 5 4 5 4 5 5 4 4 4
23100 043/300 Read block 814 len 32768
23101 044/300 Read block 2989 len 28672
23102 045/300 Read block 2404 len 8192
23103 046/300 Read block 3967 len 4096
23104 047/300 Read block 4501 len 12288
23105 048/300 Flush
23106 049/300 Write block 1256 len 20480 data: 3 3 3 3 3
23107 050/300 Read block 4260 len 4096
23108 051/300 Write block 224 len 32768 data: 5 4 5 6 6 6 6 6
23109 052/300 Read block 3248 len 32768
23110 053/300 Write block 744 len 4096 data: 4
23111 054/300 Write block 782 len 32768 data: 7 8 8 8 8 6 3 3
23112 055/300 Read block 4482 len 12288
23113 056/300 Write block 655 len 28672 data: 5 5 5 4 4 3 3
23114 057/300 Read block 4608 len 28672
23115 058/300 Read block 3666 len 36864
23116 059/300 Read block 4336 len 20480
23117 060/300 Write block 3626 len 16384 data: 3 2 2 3
23118 061/300 Write block 4175 len 8192 data: 6 5
23119 062/300 Read block 1212 len 4096
23120 063/300 Read block 4420 len 12288
23121 064/300 Write block 4872 len 12288 data: 5 6 5
23122 065/300 Flush
23123 066/300 Read block 3253 len 8192
23124 067/300 Read block 85 len 4096
23125 068/300 Read block 335 len 28672
23126 069/300 Write block 3883 len 20480 data: 3 3 3 3 3
23127 070/300 Read block 408 len 16384
23128 071/300 Read block 1244 len 8192
23129 072/300 Write block 2383 len 32768 data: 5 5 5 5 4 4 4 6
23130 073/300 Read block 3808 len 36864
23131 074/300 Write block 2638 len 24576 data: 5 4 3 2 2 3
23132 075/300 Read block 2269 len 20480
23133 076/300 Read block 3633 len 28672
23134 077/300 Read block 2891 len 4096
23135 078/300 Write block 4739 len 24576 data: 5 4 5 5 5 6
23136 079/300 Write block 276 len 24576 data: 6 3 3 3 2 2
23137 080/300 Read block 2089 len 8192
23138 081/300 Read block 1102 len 20480
23139 082/300 Write block 3318 len 36864 data: 3 4 3 2 4 5 4 5 4
23140 083/300 Read block 2937 len 12288
23141 084/300 Read block 579 len 24576
23142 085/300 Read block 3072 len 8192
23143 086/300 Read block 4546 len 28672
23144 087/300 Read block 2581 len 40960
23145 088/300 Write block 38 len 36864 data: 7 7 8 8 6 6 6 6 6
23146 089/300 Write block 377 len 32768 data: 6 7 7 7 7 8 8 9
23147 090/300 Write block 746 len 24576 data: 4 4 5 3 3 2
23148 091/300 Read block 217 len 8192
23149 092/300 Write block 4456 len 16384 data: 10 11 9 9
23150 093/300 Write block 117 len 24576 data: 6 5 4 2 2 2
23151 094/300 Write block 32 len 12288 data: 6 7 7
23152 095/300 Read block 4171 len 40960
23153 096/300 Write block 1624 len 20480 data: 7 7 6 5 5
23154 097/300 Flush
23155 098/300 Read block 557 len 20480
23156 099/300 Read block 1823 len 4096
23157 100/300 Write block 3186 len 36864 data: 8 6 5 5 6 7 7 7 5
23158 101/300 Read block 1666 len 40960
23159 102/300 Write block 3892 len 20480 data: 6 7 7 8 8
23160 103/300 Read block 2646 len 40960
23161 104/300 Write block 336 len 16384 data: 5 6 5 7
23162 105/300 Read block 4523 len 12288
23163 106/300 Write block 2963 len 36864 data: 4 3 3 2 2 4 4 4 4
23164 107/300 Flush
23165 108/300 Flush
23166 109/300 Write block 2760 len 8192 data: 2 2
23167 110/300 Write block 3027 len 20480 data: 8 8 5 5 3
23168 111/300 Write block 389 len 8192 data: 6 5
23169 112/300 Read block 3586 len 16384
23170 113/300 Write block 4271 len 40960 data: 5 5 6 6 7 8 7 5 7 7
23171 114/300 Write block 346 len 12288 data: 5 4 6
23172 115/300 Write block 2930 len 28672 data: 4 4 3 3 4 4 3
23173 116/300 Read block 1837 len 12288
23174 117/300 Read block 473 len 4096
23175 118/300 Write block 4478 len 40960 data: 7 7 6 5 4 3 2 2 2 4
23176 119/300 Write block 1605 len 32768 data: 5 4 4 4 6 5 4 4
23177 120/300 Write block 2917 len 40960 data: 5 2 3 4 5 7 7 9 5 5
23178 121/300 Flush
23179 122/300 Read block 3255 len 32768
23180 123/300 Read block 1203 len 20480
23181 124/300 Write block 2119 len 28672 data: 3 2 2 2 3 3 4
23182 125/300 Write block 105 len 16384 data: 3 4 3 6
23183 126/300 Read block 339 len 4096
23184 127/300 Write block 2113 len 28672 data: 7 6 6 4 4 3 4
23185 128/300 Read block 1470 len 20480
23186 129/300 Read block 4764 len 40960
23187 130/300 Write block 1207 len 8192 data: 11 12
23188 131/300 Write block 4919 len 20480 data: 7 5 4 3 3
23189 132/300 Write block 163 len 12288 data: 7 5 7
23190 133/300 Read block 3790 len 24576
23191 134/300 Read block 2299 len 28672
23192 135/300 Read block 2612 len 28672
23193 136/300 Read block 1195 len 20480
23194 137/300 Write block 1332 len 28672 data: 6 6 6 7 8 8 8
23195 138/300 Read block 1360 len 24576
23196 139/300 Read block 3217 len 4096
23197 140/300 Write block 3415 len 40960 data: 4 5 3 4 5 5 5 5 5 3
23198 141/300 Write block 1738 len 4096 data: 6
23199 142/300 Write block 3746 len 16384 data: 6 5 5 4
23200 143/300 Read block 1113 len 4096
23201 144/300 Read block 2528 len 36864
23202 145/300 Read block 1429 len 20480
23203 146/300 Write block 3728 len 8192 data: 7 6
23204 147/300 Read block 266 len 40960
23205 148/300 Write block 1348 len 16384 data: 5 5 5 4
23206 149/300 Flush
23207 150/300 Write block 2235 len 36864 data: 3 3 3 4 5 5 5 4 4
23208 151/300 Read block 3816 len 40960
23209 152/300 Read block 1631 len 12288
23210 153/300 Read block 1747 len 12288
23211 154/300 Read block 4912 len 8192
23212 155/300 Write block 415 len 28672 data: 9 9 9 9 8 8 7
23213 156/300 Write block 4258 len 40960 data: 6 5 4 5 4 4 3 3 3 3
23214 157/300 Read block 1679 len 32768
23215 158/300 Read block 3647 len 28672
23216 159/300 Read block 1259 len 16384
23217 160/300 Read block 2461 len 20480
23218 161/300 Read block 3813 len 12288
23219 162/300 Read block 3045 len 20480
23220 163/300 Write block 3455 len 8192 data: 4 6
23221 164/300 Flush
23222 165/300 Flush
23223 166/300 Read block 2729 len 16384
23224 167/300 Write block 1633 len 8192 data: 5 5
23225 168/300 Read block 465 len 28672
23226 169/300 Read block 2185 len 20480
23227 170/300 Write block 3282 len 20480 data: 2 4 4 6 7
23228 171/300 Read block 4577 len 32768
23229 172/300 Flush
23230 173/300 Flush
23231 174/300 Flush
23232 175/300 Write block 2118 len 8192 data: 4 5
23233 176/300 Flush
23234 177/300 Read block 860 len 16384
23235 178/300 Write block 1522 len 28672 data: 5 5 5 4 4 5 6
23236 179/300 Write block 2318 len 24576 data: 5 4 4 5 5 4
23237 180/300 Read block 2551 len 12288
23238 181/300 Write block 4370 len 16384 data: 5 5 4 5
23239 182/300 Write block 3225 len 4096 data: 4
23240 183/300 Write block 3498 len 20480 data: 7 7 7 6 7
23241 184/300 Flush
23242 185/300 Read block 1182 len 32768
23243 186/300 Write block 1211 len 4096 data: 7
23244 187/300 Read block 443 len 40960
23245 188/300 Read block 2470 len 16384
23246 189/300 Write block 57 len 20480 data: 10 10 10 10 10
23247 190/300 Write block 3929 len 36864 data: 7 7 9 10 8 7 5 5 4
23248 191/300 Write block 1675 len 32768 data: 3 2 3 4 4 5 4 6
23249 192/300 Read block 2286 len 4096
23250 193/300 Read block 2862 len 12288
23251 194/300 Write block 4393 len 20480 data: 5 6 6 5 5
23252 195/300 Flush
23253 196/300 Write block 3305 len 28672 data: 5 6 6 5 5 6 5
23254 197/300 Write block 4048 len 24576 data: 7 9 8 6 6 5
23255 198/300 Flush
23256 199/300 Flush
23257 200/300 Write block 868 len 24576 data: 7 7 8 8 7 10
23258 201/300 Write block 1842 len 40960 data: 3 3 3 3 3 3 2 2 3 4
23259 202/300 Write block 4394 len 36864 data: 7 7 6 6 6 6 7 7 9
23260 203/300 Read block 4557 len 40960
23261 204/300 Flush
23262 205/300 Read block 4278 len 12288
23263 206/300 Read block 2078 len 28672
23264 207/300 Read block 2272 len 36864
23265 208/300 Write block 4769 len 24576 data: 5 5 6 6 6 5
23266 209/300 Flush
23267 210/300 Write block 2412 len 32768 data: 3 3 3 3 2 3 5 5
23268 211/300 Read block 955 len 4096
23269 212/300 Write block 1981 len 20480 data: 5 4 4 6 5
23270 213/300 Read block 117 len 12288
23271 214/300 Write block 4648 len 4096 data: 8
23272 215/300 Read block 118 len 36864
23273 216/300 Read block 4276 len 4096
23274 217/300 Write block 547 len 4096 data: 6
23275 218/300 Write block 3846 len 20480 data: 4 3 4 6 7
23276 219/300 Read block 2508 len 32768
23277 220/300 Read block 4287 len 36864
23278 221/300 Read block 3340 len 24576
23279 222/300 Write block 2806 len 40960 data: 6 6 5 6 5 5 5 4 4 5
23280 223/300 Read block 564 len 12288
23281 224/300 Read block 3513 len 8192
23282 225/300 Write block 3746 len 20480 data: 7 6 6 5 4
23283 226/300 Read block 3259 len 4096
23284 227/300 Flush
23285 228/300 Write block 1980 len 20480 data: 7 6 5 5 7
23286 229/300 Read block 269 len 4096
23287 230/300 Write block 2184 len 36864 data: 7 6 6 4 4 4 4 3 4
23288 231/300 Read block 863 len 4096
23289 232/300 Read block 71 len 28672
23290 233/300 Read block 203 len 4096
23291 234/300 Read block 4083 len 4096
23292 235/300 Write block 3809 len 12288 data: 7 7 6
23293 236/300 Read block 3392 len 28672
23294 237/300 Write block 4108 len 20480 data: 3 3 3 3 3
23295 238/300 Read block 3512 len 24576
23296 239/300 Read block 1895 len 8192
23297 240/300 Write block 3932 len 28672 data: 11 9 8 6 6 5 4
23298 241/300 Write block 2880 len 32768 data: 8 6 6 4 4 5 4 3
23299 242/300 Write block 3268 len 16384 data: 4 4 4 6
23300 243/300 Read block 925 len 16384
23301 244/300 Read block 4268 len 36864
23302 245/300 Write block 4171 len 12288 data: 8 9 9
23303 246/300 Write block 3490 len 20480 data: 6 5 4 3 4
23304 247/300 Flush
23305 248/300 Write block 4759 len 16384 data: 5 4 6 9
23306 249/300 Write block 3482 len 28672 data: 6 5 5 4 4 4 5
23307 250/300 Read block 1419 len 40960
23308 251/300 Read block 3435 len 28672
23309 252/300 Write block 299 len 24576 data: 9 10 8 6 6 8
23310 253/300 Write block 543 len 28672 data: 3 3 4 6 7 5 5
23311 254/300 Read block 2496 len 4096
23312 255/300 Write block 4067 len 20480 data: 5 5 6 4 4
23313 256/300 Write block 1663 len 12288 data: 5 4 4
23314 257/300 Write block 463 len 16384 data: 8 7 7 5
23315 258/300 Read block 2098 len 28672
23316 259/300 Write block 2477 len 28672 data: 8 7 7 6 7 7 8
23317 260/300 Read block 1416 len 12288
23318 261/300 Flush
23319 262/300 Read block 1465 len 36864
23320 263/300 Read block 981 len 36864
23321 264/300 Write block 521 len 40960 data: 2 2 4 5 5 6 5 5 5 5
23322 265/300 Flush
23323 266/300 Write block 3550 len 28672 data: 5 6 6 6 5 5 4
23324 267/300 Write block 3819 len 12288 data: 6 6 5
23325 268/300 Read block 963 len 36864
23326 269/300 Write block 282 len 16384 data: 4 6 5 6
23327 270/300 Flush
23328 271/300 Read block 2888 len 12288
23329 272/300 Read block 1629 len 40960
23330 273/300 Flush
23331 274/300 Flush
23332 275/300 Read block 4672 len 36864
23333 276/300 Read block 3319 len 4096
23334 277/300 Read block 4290 len 8192
23335 278/300 Read block 984 len 4096
23336 279/300 Write block 973 len 8192 data: 4 4
23337 280/300 Read block 3520 len 16384
23338 281/300 Write block 957 len 24576 data: 3 5 6 7 6 7
23339 282/300 Flush
23340 283/300 Write block 4259 len 20480 data: 6 5 6 5 5
23341 284/300 Read block 2218 len 16384
23342 285/300 Read block 448 len 32768
23343 286/300 Read block 3665 len 8192
23344 287/300 Write block 4827 len 40960 data: 8 9 7 5 5 5 4 3 3 3
23345 288/300 Write block 4647 len 8192 data: 9 9
23346 289/300 Write block 1149 len 36864 data: 5 5 5 4 3 4 4 4 4
23347 290/300 Read block 2766 len 28672
23348 291/300 Write block 4079 len 36864 data: 5 5 5 6 6 7 8 8 7
23349 292/300 Read block 2327 len 40960
23350 293/300 Read block 1419 len 32768
23351 294/300 Read block 3126 len 36864
23352 295/300 Write block 2431 len 24576 data: 6 4 5 4 5 4
23353 296/300 Read block 1756 len 28672
23354 297/300 Flush
23355 298/300 Write block 1540 len 36864 data: 4 5 7 8 6 5 4 3 3
23356 299/300 Write block 850 len 4096 data: 5
23357 300/300 Write block 3748 len 16384 data: 7 6 5 5
23358 Main task has work to do, go find it
23359 got dsc Start(0)
23360 start 0
23361 Sep 22 23:28:54.830 INFO request completed, latency_us: 327, response_code: 204, uri: /start/cid/0, method: POST, req_id: c2867df8-bbde-499b-b3b8-9dce3e15dbf4, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
23362 Replay: started 0, returned:Ok(())
23363 ----------------------------------------------------------------
23364 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23365 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
23366 7563 Acked 8562 Write 4 New Done Done false
23367 7564 Acked 8563 Read 6 New Done Done false
23368 7565 Acked 8564 Write 7 New Done Done false
23369 7566 Acked 8565 Flush 0 New Done Done false
23370 7567 Acked 8566 Read 7 New Done Done false
23371 7568 Acked 8567 Read 9 New Done Done false
23372 7569 Acked 8568 Read 9 New Done Done false
23373 7570 Acked 8569 Write 10 New Done Done false
23374 7571 Acked 8570 Read 2 New Done Done false
23375 7572 Acked 8571 Write 8 New Done Done false
23376 7573 Acked 8572 Write 5 New Done Done false
23377 7574 Acked 8573 Write 7 New Done Done false
23378 7575 Acked 8574 Write 5 New Done Done false
23379 7576 Acked 8575 Read 10 New Done Done false
23380 7577 Acked 8576 Write 4 New Done Done false
23381 7578 Acked 8577 Read 7 New Done Done false
23382 7579 Acked 8578 Read 8 New Done Done false
23383 7580 Acked 8579 Read 2 New Done Done false
23384 7581 Acked 8580 Write 3 New Done Done false
23385 7582 Acked 8581 Write 2 New Done Done false
23386 7583 Acked 8582 Flush 0 New Done Done false
23387 7584 Acked 8583 Flush 0 New Done Done false
23388 7585 Acked 8584 Read 10 New Done Done false
23389 7586 Acked 8585 Write 6 New Done Done false
23390 7587 Acked 8586 Write 6 New Done Done false
23391 7588 Acked 8587 Read 7 New Done Done false
23392 7589 Acked 8588 Write 7 New Done Done false
23393 7590 Acked 8589 Read 7 New Done Done false
23394 7591 Acked 8590 Read 1 New Done Done false
23395 7592 Acked 8591 Write 4 New Done Done false
23396 7593 Acked 8592 Read 6 New Done Done false
23397 7594 Acked 8593 Write 5 New Done Done false
23398 7595 Acked 8594 Flush 0 New Done Done false
23399 7596 Acked 8595 Write 10 New Done Done false
23400 7597 Acked 8596 Write 2 New Done Done false
23401 7598 Acked 8597 Read 9 New Done Done false
23402 7599 Acked 8598 Write 6 New Done Done false
23403 7600 Acked 8599 Read 5 New Done Done false
23404 7601 Acked 8600 Flush 0 New Done Done false
23405 7602 Acked 8601 Read 10 New Done Done false
23406 7603 Acked 8602 Read 6 New Done Done false
23407 7604 Acked 8603 Write 10 New Done Done false
23408 7605 Acked 8604 Read 8 New Done Done false
23409 7606 Acked 8605 Read 7 New Done Done false
23410 7607 Acked 8606 Read 2 New Done Done false
23411 7608 Acked 8607 Read 1 New Done Done false
23412 7609 Acked 8608 Read 3 New Done Done false
23413 7610 Acked 8609 Flush 0 New Done Done false
23414 7611 Acked 8610 Write 5 New Done Done false
23415 7612 Acked 8611 Read 1 New Done Done false
23416 7613 Acked 8612 Write 8 New Done Done false
23417 7614 Acked 8613 Read 8 New Done Done false
23418 7615 Acked 8614 Write 1 New Done Done false
23419 7616 Acked 8615 Write 8 New Done Done false
23420 7617 Acked 8616 Read 3 New Done Done false
23421 7618 Acked 8617 Write 7 New Done Done false
23422 7619 Acked 8618 Read 7 New Done Done false
23423 7620 Acked 8619 Read 9 New Done Done false
23424 7621 Acked 8620 Read 5 New Done Done false
23425 7622 Acked 8621 Write 4 New Done Done false
23426 7623 Acked 8622 Write 2 New Done Done false
23427 7624 Acked 8623 Read 1 New Done Done false
23428 7625 Acked 8624 Read 3 New Done Done false
23429 7626 Acked 8625 Write 3 New Done Done false
23430 7627 Acked 8626 Flush 0 New Done Done false
23431 7628 Acked 8627 Read 2 New Done Done false
23432 7629 Acked 8628 Read 1 New Done Done false
23433 7630 Acked 8629 Read 7 New Done Done false
23434 7631 Acked 8630 Write 5 New Done Done false
23435 7632 Acked 8631 Read 4 New Done Done false
23436 7633 Acked 8632 Read 2 New Done Done false
23437 7634 Acked 8633 Write 8 New Done Done false
23438 7635 Acked 8634 Read 9 New Done Done false
23439 7636 Acked 8635 Write 6 New Done Done false
23440 7637 Acked 8636 Read 5 New Done Done false
23441 7638 Acked 8637 Read 7 New Done Done false
23442 7639 Acked 8638 Read 1 New Done Done false
23443 7640 Acked 8639 Write 6 New Done Done false
23444 7641 Acked 8640 Write 6 New Done Done false
23445 7642 Acked 8641 Read 2 New Done Done false
23446 7643 Acked 8642 Flush 0 New Done Done false
23447 7644 Acked 8643 Read 5 New Done Done false
23448 7645 Acked 8644 Write 9 New Done Done false
23449 7646 Acked 8645 Read 3 New Done Done false
23450 7647 Acked 8646 Read 6 New Done Done false
23451 7648 Acked 8647 Read 2 New Done Done false
23452 7649 Acked 8648 Read 7 New Done Done false
23453 7650 Acked 8649 Read 10 New Done Done false
23454 7651 Acked 8650 Write 9 New Done Done false
23455 7652 Acked 8651 Write 8 New Done Done false
23456 7653 Acked 8652 Write 6 New Done Done false
23457 7654 Acked 8653 Read 2 New Done Done false
23458 7655 Acked 8654 Write 4 New Done Done false
23459 7656 Acked 8655 Write 6 New Done Done false
23460 7657 Acked 8656 Write 3 New Done Done false
23461 7658 Acked 8657 Read 10 New Done Done false
23462 7659 Acked 8658 Write 5 New Done Done false
23463 7660 Acked 8659 Flush 0 New Done Done false
23464 7661 Acked 8660 Read 5 New Done Done false
23465 7662 Acked 8661 Read 1 New Done Done false
23466 7663 Acked 8662 Write 9 New Done Done false
23467 7664 Acked 8663 Read 10 New Done Done false
23468 7665 Acked 8664 Write 5 New Done Done false
23469 7666 Acked 8665 Read 10 New Done Done false
23470 7667 Acked 8666 Write 4 New Done Done false
23471 7668 Acked 8667 Read 3 New Done Done false
23472 7669 Acked 8668 Write 9 New Done Done false
23473 7670 Acked 8669 Flush 0 New Done Done false
23474 7671 Acked 8670 Flush 0 New Done Done false
23475 7672 Acked 8671 Write 2 New Done Done false
23476 7673 Acked 8672 Write 5 New Done Done false
23477 7674 Acked 8673 Write 2 New Done Done false
23478 7675 Acked 8674 Read 4 New Done Done false
23479 7676 Acked 8675 Write 10 New Done Done false
23480 7677 Acked 8676 Write 3 New Done Done false
23481 7678 Acked 8677 Write 7 New Done Done false
23482 7679 Acked 8678 Read 3 New Done Done false
23483 7680 Acked 8679 Read 1 New Done Done false
23484 7681 Acked 8680 Write 10 New Done Done false
23485 7682 Acked 8681 Write 8 New Done Done false
23486 7683 Acked 8682 Write 10 New Done Done false
23487 7684 Acked 8683 Flush 0 New Done Done false
23488 7685 Acked 8684 Read 8 New Done Done false
23489 7686 Acked 8685 Read 5 New Done Done false
23490 7687 Acked 8686 Write 7 New Done Done false
23491 7688 Acked 8687 Write 4 New Done Done false
23492 7689 Acked 8688 Read 1 New Done Done false
23493 7690 Acked 8689 Write 7 New Done Done false
23494 7691 Acked 8690 Read 5 New Done Done false
23495 7692 Acked 8691 Read 10 New Done Done false
23496 7693 Acked 8692 Write 2 New Done Done false
23497 7694 Acked 8693 Write 5 New Done Done false
23498 7695 Acked 8694 Write 3 New Done Done false
23499 7696 Acked 8695 Read 6 New Done Done false
23500 7697 Acked 8696 Read 7 New Done Done false
23501 7698 Acked 8697 Read 7 New Done Done false
23502 7699 Acked 8698 Read 5 New Done Done false
23503 7700 Acked 8699 Write 7 New Done Done false
23504 7701 Acked 8700 Read 6 New Done Done false
23505 7702 Acked 8701 Read 1 New Done Done false
23506 7703 Acked 8702 Write 10 New Done Done false
23507 7704 Acked 8703 Write 1 New Done Done false
23508 7705 Acked 8704 Write 4 New Done Done false
23509 7706 Acked 8705 Read 1 New Done Done false
23510 7707 Acked 8706 Read 9 New Done Done false
23511 7708 Acked 8707 Read 5 New Done Done false
23512 7709 Acked 8708 Write 2 New Done Done false
23513 7710 Acked 8709 Read 10 New Done Done false
23514 7711 Acked 8710 Write 4 New Done Done false
23515 7712 Acked 8711 Flush 0 New Done Done false
23516 7713 Acked 8712 Write 9 New Done Done false
23517 7714 Acked 8713 Read 10 New Done Done false
23518 7715 Acked 8714 Read 3 New Done Done false
23519 7716 Acked 8715 Read 3 New Done Done false
23520 7717 Acked 8716 Read 2 New Done Done false
23521 7718 Acked 8717 Write 7 New Done Done false
23522 7719 Acked 8718 Write 10 New Done Done false
23523 7720 Acked 8719 Read 8 New Done Done false
23524 7721 Acked 8720 Read 7 New Done Done false
23525 7722 Acked 8721 Read 4 New Done Done false
23526 7723 Acked 8722 Read 5 New Done Done false
23527 7724 Acked 8723 Read 3 New Done Done false
23528 7725 Acked 8724 Read 5 New Done Done false
23529 7726 Acked 8725 Write 2 New Done Done false
23530 7727 Acked 8726 Flush 0 New Done Done false
23531 7728 Acked 8727 Flush 0 New Done Done false
23532 7729 Acked 8728 Read 4 New Done Done false
23533 7730 Acked 8729 Write 2 New Done Done false
23534 7731 Acked 8730 Read 7 New Done Done false
23535 7732 Acked 8731 Read 5 New Done Done false
23536 7733 Acked 8732 Write 5 New Done Done false
23537 7734 Acked 8733 Read 8 New Done Done false
23538 7735 Acked 8734 Flush 0 New Done Done false
23539 7736 Acked 8735 Flush 0 New Done Done false
23540 7737 Acked 8736 Flush 0 New Done Done false
23541 7738 Acked 8737 Write 2 New Done Done false
23542 7739 Acked 8738 Flush 0 New Done Done false
23543 7740 Acked 8739 Read 4 New Done Done false
23544 7741 Acked 8740 Write 7 New Done Done false
23545 7742 Acked 8741 Write 6 New Done Done false
23546 7743 Acked 8742 Read 3 New Done Done false
23547 7744 Acked 8743 Write 4 New Done Done false
23548 7745 Acked 8744 Write 1 New Done Done false
23549 7746 Acked 8745 Write 5 New Done Done false
23550 7747 Acked 8746 Flush 0 New Done Done false
23551 7748 Acked 8747 Read 8 New Done Done false
23552 7749 Acked 8748 Write 1 New Done Done false
23553 7750 Acked 8749 Read 10 New Done Done false
23554 7751 Acked 8750 Read 4 New Done Done false
23555 7752 Acked 8751 Write 5 New Done Done false
23556 7753 Acked 8752 Write 9 New Done Done false
23557 7754 Acked 8753 Write 8 New Done Done false
23558 7755 Acked 8754 Read 1 New Done Done false
23559 7756 Acked 8755 Read 3 New Done Done false
23560 7757 Acked 8756 Write 5 New Done Done false
23561 7758 Acked 8757 Flush 0 New Done Done false
23562 7759 Acked 8758 Write 7 New Done Done false
23563 7760 Acked 8759 Write 6 New Done Done false
23564 7761 Acked 8760 Flush 0 New Done Done false
23565 7762 Acked 8761 Flush 0 New Done Done false
23566 7763 Acked 8762 Write 6 New Done Done false
23567 7764 Acked 8763 Write 10 New Done Done false
23568 7765 Acked 8764 Write 9 New Done Done false
23569 7766 Acked 8765 Read 10 New Done Done false
23570 7767 Acked 8766 Flush 0 New Done Done false
23571 7768 Acked 8767 Read 3 New Done Done false
23572 7769 Acked 8768 Read 7 New Done Done false
23573 7770 Acked 8769 Read 9 New Done Done false
23574 7771 Acked 8770 Write 6 New Done Done false
23575 7772 Acked 8771 Flush 0 New Done Done false
23576 7773 Acked 8772 Write 8 New Done Done false
23577 7774 Acked 8773 Read 1 New Done Done false
23578 7775 Acked 8774 Write 5 New Done Done false
23579 7776 Acked 8775 Read 3 New Done Done false
23580 7777 Acked 8776 Write 1 New Done Done false
23581 7778 Acked 8777 Read 9 New Done Done false
23582 7779 Acked 8778 Read 1 New Done Done false
23583 7780 Acked 8779 Write 1 New Done Done false
23584 7781 Acked 8780 Write 5 New Done Done false
23585 7782 Acked 8781 Read 8 New Done Done false
23586 7783 Acked 8782 Read 9 New Done Done false
23587 7784 Acked 8783 Read 6 New Done Done false
23588 7785 Acked 8784 Write 10 New Done Done false
23589 7786 Acked 8785 Read 3 New Done Done false
23590 7787 Acked 8786 Read 2 New Done Done false
23591 7788 Acked 8787 Write 5 New Done Done false
23592 7789 Acked 8788 Read 1 New Done Done false
23593 7790 Acked 8789 Flush 0 New Done Done false
23594 7791 Acked 8790 Write 5 New Done Done false
23595 7792 Acked 8791 Read 1 New Done Done false
23596 7793 Acked 8792 Write 9 New Done Done false
23597 7794 Acked 8793 Read 1 New Done Done false
23598 7795 Acked 8794 Read 7 New Done Done false
23599 7796 Acked 8795 Read 1 New Done Done false
23600 7797 Acked 8796 Read 1 New Done Done false
23601 7798 Acked 8797 Write 3 New Done Done false
23602 7799 Acked 8798 Read 7 New Done Done false
23603 7800 Acked 8799 Write 5 New Done Done false
23604 7801 Acked 8800 Read 6 New Done Done false
23605 7802 Acked 8801 Read 2 New Done Done false
23606 7803 Acked 8802 Write 7 New Done Done false
23607 7804 Acked 8803 Flush 0 New Done Done false
23608 7805 Acked 8804 Write 8 New Done Done false
23609 7806 Acked 8805 Write 4 New Done Done false
23610 7807 Acked 8806 Read 4 New Done Done false
23611 7808 Acked 8807 Read 9 New Done Done false
23612 7809 Acked 8808 Write 3 New Done Done false
23613 7810 Acked 8809 Write 5 New Done Done false
23614 7811 Acked 8810 Flush 0 New Done Done false
23615 7812 Acked 8811 Write 4 New Done Done false
23616 7813 Acked 8812 Write 7 New Done Done false
23617 7814 Acked 8813 Read 10 New Done Done false
23618 7815 Acked 8814 Read 7 New Done Done false
23619 7816 Acked 8815 Write 6 New Done Done false
23620 7817 Acked 8816 Write 7 New Done Done false
23621 7818 Acked 8817 Read 1 New Done Done false
23622 7819 Acked 8818 Write 5 New Done Done false
23623 7820 Acked 8819 Write 3 New Done Done false
23624 7821 Acked 8820 Write 4 New Done Done false
23625 7822 Acked 8821 Read 7 New Done Done false
23626 7823 Acked 8822 Write 7 New Done Done false
23627 7824 Acked 8823 Read 3 New Done Done false
23628 7825 Acked 8824 Flush 0 New Done Done false
23629 7826 Acked 8825 Read 9 New Done Done false
23630 7827 Acked 8826 Read 9 New Done Done false
23631 7828 Acked 8827 Write 10 New Done Done false
23632 7829 Acked 8828 Flush 0 New Done Done false
23633 7830 Acked 8829 Write 7 New Done Done false
23634 7831 Acked 8830 Write 3 New Done Done false
23635 7832 Acked 8831 Read 9 New Done Done false
23636 7833 Acked 8832 Write 4 New Done Done false
23637 7834 Acked 8833 Flush 0 New Done Done false
23638 7835 Acked 8834 Read 3 New Done Done false
23639 7836 Acked 8835 Read 10 New Done Done false
23640 7837 Acked 8836 Flush 0 New Done Done false
23641 7838 Acked 8837 Flush 0 New Done Done false
23642 7839 Acked 8838 Read 9 New Done Done false
23643 7840 Acked 8839 Read 1 New Done Done false
23644 7841 Acked 8840 Read 2 New Done Done false
23645 7842 Acked 8841 Read 1 New Done Done false
23646 7843 Acked 8842 Write 2 New Done Done false
23647 7844 Acked 8843 Read 4 New Done Done false
23648 7845 Acked 8844 Write 6 New Done Done false
23649 7846 Acked 8845 Flush 0 New Done Done false
23650 7847 Acked 8846 Write 5 New Done Done false
23651 7848 Acked 8847 Read 4 New Done Done false
23652 7849 Acked 8848 Read 8 New Done Done false
23653 7850 Acked 8849 Read 2 New Done Done false
23654 7851 Acked 8850 Write 10 New Done Done false
23655 7852 Acked 8851 Write 2 New Done Done false
23656 7853 Acked 8852 Write 9 New Done Done false
23657 7854 Acked 8853 Read 7 New Done Done false
23658 7855 Acked 8854 Write 9 New Done Done false
23659 7856 Acked 8855 Read 10 New Done Done false
23660 7857 Acked 8856 Read 8 New Done Done false
23661 7858 Acked 8857 Read 9 New Done Done false
23662 7859 Acked 8858 Write 6 New Done Done false
23663 7860 Acked 8859 Read 7 New Done Done false
23664 7861 Acked 8860 Flush 0 New Done Done false
23665 7862 Acked 8861 Write 9 New Done Done false
23666 7863 Acked 8862 Write 1 New Sent Sent false
23667 7864 Acked 8863 Write 4 New Sent Sent false
23668 STATES DS:0 DS:1 DS:2 TOTAL
23669 New 302 0 0 302
23670 Sent 0 2 2 4
23671 Done 0 300 300 600
23672 Skipped 0 0 0 0
23673 Error 0 0 0 0
23674 Last Flush: 8561 8860 8860
23675 Downstairs last five completed: 8561 8560 8559 8558 8557 8556
23676 Upstairs last five completed: 7864 7863 7862 7861 7860 7859
23677 CLIENT: Up:0 ds:302 act:2
23678 [8810] got start action
23679 [8810] I am going to restart
23680 Starting downstairs at port 8810
23681 Make output file at "/tmp/dsc/downstairs-8810.txt"
23682 [8810][0] reports Starting
23683 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1199)
23684 [8810][0] reports Running
236852023-09-22T23:28:56.895ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
236862023-09-22T23:28:56.895ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
236872023-09-22T23:28:56.895ZINFOcrucible: [0] client re-new 303 jobs since flush 8561 = downstairs
236882023-09-22T23:28:56.896ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
236892023-09-22T23:28:56.896ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
236902023-09-22T23:28:56.896ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
236912023-09-22T23:28:56.896ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
236922023-09-22T23:28:56.896ZINFOcrucible: [0] send last flush ID to this DS: 8561
236932023-09-22T23:28:56.896ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
236942023-09-22T23:28:56.897ZINFOcrucible: [0] Replied this last flush ID: 8561
236952023-09-22T23:28:56.897ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
236962023-09-22T23:28:56.897ZINFOcrucible: [0] Transition from Offline to Replay
236972023-09-22T23:28:56.897ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
236982023-09-22T23:28:56.897ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
236992023-09-22T23:28:56.897ZINFOcrucible: [0] Transition from Replay to Active
237002023-09-22T23:28:56.897ZINFOcrucible: [0] Starts cmd_loop
237012023-09-22T23:28:56.897ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
237022023-09-22T23:28:56.897ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
237032023-09-22T23:28:56.897ZINFOcrucible: Set check for repair
237042023-09-22T23:28:57.898ZINFOcrucible: Checking if live repair is needed
237052023-09-22T23:28:57.898ZINFOcrucible: No Live Repair required at this time
237062023-09-22T23:28:57.898ZWARNcrucible: [0] flow control end
23707 ----------------------------------------------------------------
23708 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23709 Downstairs last five completed: 8864 8863 8862 8861 8860 8859
23710 Upstairs last five completed: 7865 7864 7863 7862 7861 7860
23711 CLIENT: Up:0 ds:0 act:3
23712 Replay: All jobs finished, all DS active.
23713 Main task has work to do, go find it
23714 got dsc Stop(0)
23715 stop 0
23716 Sep 22 23:28:58.836 INFO request completed, latency_us: 510, response_code: 204, uri: /stop/cid/0, [8810] Got stop action so:false kr:false
23717 method: POST, req_id: 5dd852d7-c983-491a-87a2-af7f0923ff02, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
23718 Sep 22 23:28:58.837 INFO request completed, latency_us: 292, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8b504380-596d-48d2-b0c1-21c983690bf6, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
237192023-09-22T23:28:58.837ZWARNcrucible: [0] downstairs disconnected
237202023-09-22T23:28:58.838ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
237212023-09-22T23:28:58.838ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
237222023-09-22T23:28:58.838ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
237232023-09-22T23:28:58.838ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
237242023-09-22T23:28:58.838ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
237252023-09-22T23:28:58.838ZWARNcrucible: [0] pm_task rx.recv() is None
23726 [8810] Exited with: ExitStatus(unix_wait_status(9))
23727 [8810][0] reports Exit
23728 Sep 22 23:29:02.839 INFO request completed, latency_us: 262, response_code: 200, uri: /state/cid/0, method: GET, req_id: a9f26bb4-6d38-4600-a402-5f6a96873814, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
23729 001/300 Read block 2848 len 12288
23730 002/300 Write block 1242 len 32768 data: 5 4 3 3 2 3 3 4
23731 003/300 Write block 2881 len 16384 data: 7 7 5 5
23732 004/300 Write block 4829 len 36864 data: 8 6 6 6 5 4 4 4 5
23733 005/300 Read block 1802 len 36864
23734 006/300 Write block 4446 len 20480 data: 5 5 5 4 4
23735 007/300 Read block 3382 len 4096
23736 008/300 Read block 4418 len 40960
23737 009/300 Flush
23738 010/300 Read block 4261 len 16384
23739 011/300 Read block 1842 len 28672
23740 012/300 Read block 4017 len 8192
23741 013/300 Read block 4187 len 40960
23742 014/300 Flush
23743 015/300 Read block 693 len 36864
23744 016/300 Write block 3049 len 20480 data: 6 6 5 6 7
23745 017/300 Write block 893 len 40960 data: 5 5 5 5 5 3 3 3 3 3
23746 018/300 Read block 1261 len 12288
23747 019/300 Flush
23748 020/300 Write block 982 len 8192 data: 6 6
23749 021/300 Read block 2922 len 16384
23750 022/300 Write block 4791 len 36864 data: 6 6 3 3 2 2 3 3 4
23751 023/300 Flush
23752 024/300 Flush
23753 025/300 Read block 1140 len 20480
23754 026/300 Read block 529 len 8192
23755 027/300 Read block 4740 len 40960
23756 028/300 Read block 3317 len 36864
23757 029/300 Write block 3499 len 28672 data: 8 8 7 8 6 6 5
23758 030/300 Read block 1092 len 40960
23759 031/300 Read block 3295 len 20480
23760 032/300 Write block 9 len 28672 data: 5 7 7 8 7 4 5
23761 033/300 Read block 4014 len 8192
23762 034/300 Read block 1581 len 4096
23763 035/300 Read block 1723 len 36864
23764 036/300 Read block 1885 len 40960
23765 037/300 Write block 2078 len 32768 data: 4 4 3 4 4 5 5 4
23766 038/300 Write block 2143 len 24576 data: 3 3 3 3 3 4
23767 039/300 Write block 1422 len 28672 data: 6 6 5 4 4 5 7
23768 040/300 Write block 815 len 16384 data: 9 8 8 6
23769 041/300 Read block 873 len 20480
23770 042/300 Read block 4726 len 36864
23771 043/300 Read block 4297 len 4096
23772 044/300 Read block 1805 len 12288
23773 045/300 Read block 2190 len 36864
23774 046/300 Read block 1390 len 4096
23775 047/300 Read block 733 len 24576
23776 048/300 Read block 744 len 4096
23777 049/300 Read block 439 len 12288
23778 050/300 Write block 4476 len 28672 data: 8 7 8 8 7 6 5
23779 051/300 Read block 604 len 20480
23780 052/300 Write block 2776 len 40960 data: 5 5 4 5 4 3 5 7 8 8
23781 053/300 Read block 2727 len 20480
23782 054/300 Read block 3302 len 40960
23783 055/300 Read block 1177 len 8192
23784 056/300 Flush
23785 057/300 Read block 1291 len 40960
23786 058/300 Flush
23787 059/300 Read block 3 len 36864
23788 060/300 Read block 446 len 28672
23789 061/300 Flush
23790 062/300 Write block 2730 len 28672 data: 7 6 7 7 6 5 6
23791 063/300 Read block 2380 len 8192
23792 064/300 Write block 69 len 4096 data: 5
23793 065/300 Write block 3850 len 20480 data: 8 8 9 9 8
23794 066/300 Read block 3708 len 20480
23795 067/300 Read block 2410 len 4096
23796 068/300 Write block 1605 len 4096 data: 6
23797 069/300 Read block 619 len 28672
23798 070/300 Write block 2920 len 28672 data: 5 6 8 8 10 6 6
23799 071/300 Read block 2546 len 12288
23800 072/300 Read block 2494 len 32768
23801 073/300 Write block 541 len 32768 data: 2 2 4 4 5 7 8 6
23802 074/300 Write block 1025 len 8192 data: 6 6
23803 075/300 Read block 2264 len 36864
23804 076/300 Read block 4271 len 36864
23805 077/300 Read block 1652 len 40960
23806 078/300 Flush
23807 079/300 Flush
23808 080/300 Read block 4825 len 4096
23809 081/300 Read block 2064 len 16384
23810 082/300 Write block 2055 len 12288 data: 5 5 5
23811 083/300 Write block 3157 len 12288 data: 4 6 5
23812 084/300 Flush
23813 085/300 Write block 750 len 20480 data: 4 3 3 3 3
23814 086/300 Read block 4167 len 32768
23815 087/300 Write block 1066 len 4096 data: 5
23816 088/300 Write block 2052 len 16384 data: 5 5 5 6
23817 089/300 Write block 3479 len 8192 data: 7 8
23818 090/300 Read block 1489 len 16384
23819 091/300 Read block 1257 len 8192
23820 092/300 Read block 17 len 8192
23821 093/300 Read block 2200 len 8192
23822 094/300 Flush
23823 095/300 Write block 587 len 24576 data: 5 5 4 3 5 7
23824 096/300 Read block 822 len 36864
23825 097/300 Read block 4061 len 8192
23826 098/300 Read block 2170 len 40960
23827 099/300 Read block 3268 len 4096
23828 100/300 Read block 3957 len 32768
23829 101/300 Write block 2424 len 20480 data: 5 6 7 6 6
23830 102/300 Read block 1624 len 28672
23831 103/300 Read block 267 len 24576
23832 104/300 Write block 1373 len 16384 data: 7 7 6 6
23833 105/300 Read block 3365 len 40960
23834 106/300 Read block 3581 len 40960
23835 107/300 Write block 1848 len 32768 data: 3 3 4 5 4 4 4 5
23836 108/300 Write block 1657 len 8192 data: 8 8
23837 109/300 Read block 2770 len 8192
23838 110/300 Read block 2262 len 4096
23839 111/300 Write block 4267 len 36864 data: 4 3 2 4 6 6 7 7 8
23840 112/300 Read block 2413 len 40960
23841 113/300 Flush
23842 114/300 Read block 3785 len 24576
23843 115/300 Write block 2157 len 4096 data: 3
23844 116/300 Read block 4426 len 40960
23845 117/300 Flush
23846 118/300 Read block 2894 len 20480
23847 119/300 Read block 2908 len 32768
23848 120/300 Write block 2747 len 36864 data: 3 4 5 5 4 4 3 2 3
23849 121/300 Read block 3390 len 40960
23850 122/300 Write block 3290 len 16384 data: 8 7 7 6
23851 123/300 Write block 2423 len 8192 data: 4 6
23852 124/300 Read block 4444 len 8192
23853 125/300 Read block 2276 len 24576
23854 126/300 Flush
23855 127/300 Write block 532 len 4096 data: 4
23856 128/300 Write block 1967 len 32768 data: 6 5 4 2 3 3 5 4
23857 129/300 Read block 45 len 36864
23858 130/300 Write block 1605 len 20480 data: 7 5 5 5 7
23859 131/300 Write block 192 len 12288 data: 5 5 6
23860 132/300 Read block 4130 len 24576
23861 133/300 Read block 2388 len 8192
23862 134/300 Read block 1512 len 36864
23863 135/300 Write block 3659 len 40960 data: 3 4 4 4 4 5 6 5 5 5
23864 136/300 Read block 2583 len 12288
23865 137/300 Read block 3744 len 36864
23866 138/300 Write block 1268 len 16384 data: 8 8 8 8
23867 139/300 Read block 1840 len 32768
23868 140/300 Write block 3481 len 16384 data: 7 7 6 6
23869 141/300 Read block 3574 len 28672
23870 142/300 Read block 745 len 24576
23871 143/300 Write block 2028 len 32768 data: 7 7 7 7 8 7 7 8
23872 144/300 Read block 828 len 16384
23873 145/300 Write block 3205 len 16384 data: 4 5 6 6
23874 146/300 Flush
23875 147/300 Read block 3954 len 20480
23876 148/300 Write block 1831 len 40960 data: 4 4 5 5 4 5 5 5 5 5
23877 149/300 Read block 3904 len 36864
23878 150/300 Read block 2050 len 4096
23879 151/300 Read block 1892 len 4096
23880 152/300 Write block 1312 len 20480 data: 5 7 6 5 8
23881 153/300 Write block 3252 len 12288 data: 6 5 6
23882 154/300 Read block 2989 len 20480
23883 155/300 Write block 4948 len 20480 data: 7 7 8 7 7
23884 156/300 Read block 1896 len 36864
23885 157/300 Write block 1318 len 32768 data: 3 3 3 3 4 5 5 4
23886 158/300 Read block 946 len 20480
23887 159/300 Read block 3119 len 36864
23888 160/300 Flush
23889 161/300 Flush
23890 162/300 Read block 3373 len 8192
23891 163/300 Write block 1094 len 12288 data: 4 4 4
23892 164/300 Read block 4208 len 4096
23893 165/300 Write block 3250 len 32768 data: 6 6 7 6 7 9 8 7
23894 166/300 Read block 1159 len 16384
23895 167/300 Read block 3178 len 12288
23896 168/300 Read block 3624 len 12288
23897 169/300 Write block 1969 len 24576 data: 5 3 4 4 6 5
23898 170/300 Write block 4505 len 8192 data: 5 5
23899 171/300 Write block 1147 len 24576 data: 5 4 6 6 6 5
23900 172/300 Read block 2167 len 20480
23901 173/300 Write block 1587 len 16384 data: 2 3 3 3
23902 174/300 Read block 1455 len 4096
23903 175/300 Read block 2001 len 20480
23904 176/300 Write block 3659 len 16384 data: 4 5 5 5
23905 177/300 Write block 4867 len 16384 data: 2 2 2 4
23906 178/300 Write block 321 len 24576 data: 5 5 5 6 7 6
23907 179/300 Flush
23908 180/300 Write block 1156 len 8192 data: 5 5
23909 181/300 Read block 1844 len 24576
23910 182/300 Flush
23911 183/300 Write block 726 len 40960 data: 9 8 8 7 6 7 6 5 5 5
23912 184/300 Write block 2760 len 16384 data: 3 3 2 4
23913 185/300 Read block 2631 len 20480
23914 186/300 Write block 1427 len 32768 data: 6 8 9 9 9 10 10 9
23915 187/300 Read block 3305 len 12288
23916 188/300 Write block 1335 len 36864 data: 8 9 9 9 8 8 7 7 6
23917 189/300 Write block 2731 len 24576 data: 7 8 8 7 6 7
23918 190/300 Write block 3328 len 4096 data: 5
23919 191/300 Write block 4267 len 20480 data: 5 4 3 5 7
23920 192/300 Flush
23921 193/300 Read block 4680 len 8192
23922 194/300 Write block 4222 len 12288 data: 3 3 3
23923 195/300 Read block 4524 len 40960
23924 196/300 Read block 3619 len 32768
23925 197/300 Read block 3233 len 8192
23926 198/300 Write block 844 len 20480 data: 8 8 7 5 6
23927 199/300 Read block 219 len 4096
23928 200/300 Read block 4957 len 32768
23929 201/300 Write block 525 len 20480 data: 6 7 6 6 6
23930 202/300 Write block 591 len 32768 data: 6 8 5 5 7 8 8 8
23931 203/300 Write block 1771 len 4096 data: 11
23932 204/300 Write block 4511 len 16384 data: 4 4 4 3
23933 205/300 Read block 4482 len 28672
23934 206/300 Read block 3145 len 4096
23935 207/300 Read block 3709 len 8192
23936 208/300 Read block 768 len 24576
23937 209/300 Read block 4016 len 24576
23938 210/300 Read block 2281 len 28672
23939 211/300 Read block 1726 len 8192
23940 212/300 Read block 3898 len 36864
23941 213/300 Read block 925 len 28672
23942 214/300 Flush
23943 215/300 Read block 228 len 24576
23944 216/300 Read block 4397 len 40960
23945 217/300 Write block 3835 len 40960 data: 6 6 7 6 5 6 4 5 5 5
23946 218/300 Read block 1023 len 36864
23947 219/300 Flush
23948 220/300 Read block 2250 len 20480
23949 221/300 Flush
23950 222/300 Write block 4349 len 8192 data: 6 6
23951 223/300 Read block 3414 len 4096
23952 224/300 Read block 2086 len 32768
23953 225/300 Read block 1957 len 8192
23954 226/300 Read block 1696 len 4096
23955 227/300 Read block 3483 len 28672
23956 228/300 Read block 3228 len 20480
23957 229/300 Read block 1551 len 36864
23958 230/300 Write block 1177 len 4096 data: 3
23959 231/300 Flush
23960 232/300 Write block 3457 len 20480 data: 7 8 8 5 5
23961 233/300 Read block 3614 len 40960
23962 234/300 Write block 1906 len 4096 data: 8
23963 235/300 Read block 556 len 8192
23964 236/300 Read block 2809 len 20480
23965 237/300 Write block 3091 len 28672 data: 4 4 5 5 5 7 8
23966 238/300 Read block 1670 len 32768
23967 239/300 Write block 490 len 32768 data: 6 6 7 7 7 6 5 6
23968 240/300 Read block 4392 len 8192
23969 241/300 Write block 3396 len 8192 data: 4 3
23970 242/300 Flush
23971 243/300 Read block 4329 len 32768
23972 244/300 Read block 1996 len 12288
23973 245/300 Read block 2738 len 32768
23974 246/300 Read block 2204 len 36864
23975 247/300 Write block 2113 len 32768 data: 8 7 7 5 5 5 6 3
23976 248/300 Read block 2598 len 24576
23977 249/300 Write block 2999 len 16384 data: 6 6 6 5
23978 250/300 Read block 1313 len 24576
23979 251/300 Read block 3010 len 8192
23980 252/300 Flush
23981 253/300 Flush
23982 254/300 Read block 4689 len 16384
23983 255/300 Write block 3941 len 20480 data: 4 4 3 2 2
23984 256/300 Read block 4981 len 20480
23985 257/300 Read block 3347 len 8192
23986 258/300 Read block 1782 len 28672
23987 259/300 Write block 4037 len 24576 data: 8 7 5 5 5 5
23988 260/300 Write block 677 len 36864 data: 4 4 5 5 5 5 5 6 4
23989 261/300 Write block 3051 len 24576 data: 6 7 8 6 8 7
23990 262/300 Read block 3000 len 16384
23991 263/300 Write block 3564 len 12288 data: 4 3 4
23992 264/300 Read block 125 len 32768
23993 265/300 Read block 1731 len 40960
23994 266/300 Read block 3466 len 4096
23995 267/300 Read block 3043 len 8192
23996 268/300 Read block 1360 len 16384
23997 269/300 Read block 3268 len 4096
23998 270/300 Write block 4938 len 4096 data: 4
23999 271/300 Read block 4554 len 12288
24000 272/300 Flush
24001 273/300 Read block 4386 len 32768
24002 274/300 Read block 2798 len 4096
24003 275/300 Write block 3774 len 32768 data: 6 5 5 5 6 6 5 4
24004 276/300 Read block 3726 len 36864
24005 277/300 Write block 2745 len 32768 data: 3 3 4 5 6 6 5 5
24006 278/300 Write block 38 len 28672 data: 8 8 9 9 7 7 7
24007 279/300 Write block 1320 len 28672 data: 4 4 5 6 6 5 5
24008 280/300 Read block 2180 len 12288
24009 281/300 Read block 4561 len 28672
24010 282/300 Write block 2049 len 32768 data: 4 3 5 6 6 6 7 6
24011 283/300 Read block 4954 len 8192
24012 284/300 Read block 4217 len 4096
24013 285/300 Write block 71 len 32768 data: 3 3 4 5 5 4 3 3
24014 286/300 Read block 2065 len 24576
24015 287/300 Write block 1793 len 20480 data: 6 9 11 11 11
24016 288/300 Write block 1567 len 20480 data: 5 6 5 5 5
24017 289/300 Read block 1786 len 36864
24018 290/300 Read block 3209 len 16384
24019 291/300 Write block 589 len 20480 data: 5 4 7 9 6
24020 292/300 Write block 2190 len 24576 data: 5 4 5 4 5 7
24021 293/300 Read block 123 len 32768
24022 294/300 Write block 2810 len 16384 data: 6 6 6 5
24023 295/300 Read block 1189 len 24576
24024 296/300 Flush
24025 297/300 Read block 837 len 32768
24026 298/300 Read block 2326 len 16384
24027 299/300 Write block 1783 len 8192 data: 5 4
24028 300/300 Write block 2018 len 12288 data: 4 4 5
24029 Main task has work to do, go find it
24030 got dsc Start(0)
24031 start 0
24032 Sep 22 23:29:03.792 INFO request completed, latency_us: 342, response_code: 204, uri: /start/cid/0, method: POST, req_id: 966b4f1e-fc58-4c61-9a61-e50046abd92d, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
24033 Replay: started 0, returned:Ok(())
24034 ----------------------------------------------------------------
24035 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24036 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
24037 7866 Acked 8865 Read 3 New Done Done false
24038 7867 Acked 8866 Write 8 New Done Done false
24039 7868 Acked 8867 Write 4 New Done Done false
24040 7869 Acked 8868 Write 9 New Done Done false
24041 7870 Acked 8869 Read 9 New Done Done false
24042 7871 Acked 8870 Write 5 New Done Done false
24043 7872 Acked 8871 Read 1 New Done Done false
24044 7873 Acked 8872 Read 10 New Done Done false
24045 7874 Acked 8873 Flush 0 New Done Done false
24046 7875 Acked 8874 Read 4 New Done Done false
24047 7876 Acked 8875 Read 7 New Done Done false
24048 7877 Acked 8876 Read 2 New Done Done false
24049 7878 Acked 8877 Read 10 New Done Done false
24050 7879 Acked 8878 Flush 0 New Done Done false
24051 7880 Acked 8879 Read 9 New Done Done false
24052 7881 Acked 8880 Write 5 New Done Done false
24053 7882 Acked 8881 Write 10 New Done Done false
24054 7883 Acked 8882 Read 3 New Done Done false
24055 7884 Acked 8883 Flush 0 New Done Done false
24056 7885 Acked 8884 Write 2 New Done Done false
24057 7886 Acked 8885 Read 4 New Done Done false
24058 7887 Acked 8886 Write 9 New Done Done false
24059 7888 Acked 8887 Flush 0 New Done Done false
24060 7889 Acked 8888 Flush 0 New Done Done false
24061 7890 Acked 8889 Read 5 New Done Done false
24062 7891 Acked 8890 Read 2 New Done Done false
24063 7892 Acked 8891 Read 10 New Done Done false
24064 7893 Acked 8892 Read 9 New Done Done false
24065 7894 Acked 8893 Write 7 New Done Done false
24066 7895 Acked 8894 Read 10 New Done Done false
24067 7896 Acked 8895 Read 5 New Done Done false
24068 7897 Acked 8896 Write 7 New Done Done false
24069 7898 Acked 8897 Read 2 New Done Done false
24070 7899 Acked 8898 Read 1 New Done Done false
24071 7900 Acked 8899 Read 9 New Done Done false
24072 7901 Acked 8900 Read 10 New Done Done false
24073 7902 Acked 8901 Write 8 New Done Done false
24074 7903 Acked 8902 Write 6 New Done Done false
24075 7904 Acked 8903 Write 7 New Done Done false
24076 7905 Acked 8904 Write 4 New Done Done false
24077 7906 Acked 8905 Read 5 New Done Done false
24078 7907 Acked 8906 Read 9 New Done Done false
24079 7908 Acked 8907 Read 1 New Done Done false
24080 7909 Acked 8908 Read 3 New Done Done false
24081 7910 Acked 8909 Read 9 New Done Done false
24082 7911 Acked 8910 Read 1 New Done Done false
24083 7912 Acked 8911 Read 6 New Done Done false
24084 7913 Acked 8912 Read 1 New Done Done false
24085 7914 Acked 8913 Read 3 New Done Done false
24086 7915 Acked 8914 Write 7 New Done Done false
24087 7916 Acked 8915 Read 5 New Done Done false
24088 7917 Acked 8916 Write 10 New Done Done false
24089 7918 Acked 8917 Read 5 New Done Done false
24090 7919 Acked 8918 Read 10 New Done Done false
24091 7920 Acked 8919 Read 2 New Done Done false
24092 7921 Acked 8920 Flush 0 New Done Done false
24093 7922 Acked 8921 Read 10 New Done Done false
24094 7923 Acked 8922 Flush 0 New Done Done false
24095 7924 Acked 8923 Read 9 New Done Done false
24096 7925 Acked 8924 Read 7 New Done Done false
24097 7926 Acked 8925 Flush 0 New Done Done false
24098 7927 Acked 8926 Write 7 New Done Done false
24099 7928 Acked 8927 Read 2 New Done Done false
24100 7929 Acked 8928 Write 1 New Done Done false
24101 7930 Acked 8929 Write 5 New Done Done false
24102 7931 Acked 8930 Read 5 New Done Done false
24103 7932 Acked 8931 Read 1 New Done Done false
24104 7933 Acked 8932 Write 1 New Done Done false
24105 7934 Acked 8933 Read 7 New Done Done false
24106 7935 Acked 8934 Write 7 New Done Done false
24107 7936 Acked 8935 Read 3 New Done Done false
24108 7937 Acked 8936 Read 8 New Done Done false
24109 7938 Acked 8937 Write 8 New Done Done false
24110 7939 Acked 8938 Write 2 New Done Done false
24111 7940 Acked 8939 Read 9 New Done Done false
24112 7941 Acked 8940 Read 9 New Done Done false
24113 7942 Acked 8941 Read 10 New Done Done false
24114 7943 Acked 8942 Flush 0 New Done Done false
24115 7944 Acked 8943 Flush 0 New Done Done false
24116 7945 Acked 8944 Read 1 New Done Done false
24117 7946 Acked 8945 Read 4 New Done Done false
24118 7947 Acked 8946 Write 3 New Done Done false
24119 7948 Acked 8947 Write 3 New Done Done false
24120 7949 Acked 8948 Flush 0 New Done Done false
24121 7950 Acked 8949 Write 5 New Done Done false
24122 7951 Acked 8950 Read 8 New Done Done false
24123 7952 Acked 8951 Write 1 New Done Done false
24124 7953 Acked 8952 Write 4 New Done Done false
24125 7954 Acked 8953 Write 2 New Done Done false
24126 7955 Acked 8954 Read 4 New Done Done false
24127 7956 Acked 8955 Read 2 New Done Done false
24128 7957 Acked 8956 Read 2 New Done Done false
24129 7958 Acked 8957 Read 2 New Done Done false
24130 7959 Acked 8958 Flush 0 New Done Done false
24131 7960 Acked 8959 Write 6 New Done Done false
24132 7961 Acked 8960 Read 9 New Done Done false
24133 7962 Acked 8961 Read 2 New Done Done false
24134 7963 Acked 8962 Read 10 New Done Done false
24135 7964 Acked 8963 Read 1 New Done Done false
24136 7965 Acked 8964 Read 8 New Done Done false
24137 7966 Acked 8965 Write 5 New Done Done false
24138 7967 Acked 8966 Read 7 New Done Done false
24139 7968 Acked 8967 Read 6 New Done Done false
24140 7969 Acked 8968 Write 4 New Done Done false
24141 7970 Acked 8969 Read 10 New Done Done false
24142 7971 Acked 8970 Flush 0 New Done Done false
24143 7972 Acked 8971 Read 10 New Done Done false
24144 7973 Acked 8972 Write 8 New Done Done false
24145 7974 Acked 8973 Write 2 New Done Done false
24146 7975 Acked 8974 Read 2 New Done Done false
24147 7976 Acked 8975 Read 1 New Done Done false
24148 7977 Acked 8976 Write 9 New Done Done false
24149 7978 Acked 8977 Read 10 New Done Done false
24150 7979 Acked 8978 Flush 0 New Done Done false
24151 7980 Acked 8979 Read 6 New Done Done false
24152 7981 Acked 8980 Write 1 New Done Done false
24153 7982 Acked 8981 Read 10 New Done Done false
24154 7983 Acked 8982 Flush 0 New Done Done false
24155 7984 Acked 8983 Read 5 New Done Done false
24156 7985 Acked 8984 Read 8 New Done Done false
24157 7986 Acked 8985 Write 9 New Done Done false
24158 7987 Acked 8986 Read 10 New Done Done false
24159 7988 Acked 8987 Write 4 New Done Done false
24160 7989 Acked 8988 Write 2 New Done Done false
24161 7990 Acked 8989 Read 2 New Done Done false
24162 7991 Acked 8990 Read 6 New Done Done false
24163 7992 Acked 8991 Flush 0 New Done Done false
24164 7993 Acked 8992 Write 1 New Done Done false
24165 7994 Acked 8993 Write 8 New Done Done false
24166 7995 Acked 8994 Read 9 New Done Done false
24167 7996 Acked 8995 Write 5 New Done Done false
24168 7997 Acked 8996 Write 3 New Done Done false
24169 7998 Acked 8997 Read 6 New Done Done false
24170 7999 Acked 8998 Read 2 New Done Done false
24171 8000 Acked 8999 Read 9 New Done Done false
24172 8001 Acked 9000 Write 10 New Done Done false
24173 8002 Acked 9001 Read 3 New Done Done false
24174 8003 Acked 9002 Read 9 New Done Done false
24175 8004 Acked 9003 Write 4 New Done Done false
24176 8005 Acked 9004 Read 8 New Done Done false
24177 8006 Acked 9005 Write 4 New Done Done false
24178 8007 Acked 9006 Read 7 New Done Done false
24179 8008 Acked 9007 Read 6 New Done Done false
24180 8009 Acked 9008 Write 8 New Done Done false
24181 8010 Acked 9009 Read 4 New Done Done false
24182 8011 Acked 9010 Write 4 New Done Done false
24183 8012 Acked 9011 Flush 0 New Done Done false
24184 8013 Acked 9012 Read 5 New Done Done false
24185 8014 Acked 9013 Write 10 New Done Done false
24186 8015 Acked 9014 Read 9 New Done Done false
24187 8016 Acked 9015 Read 1 New Done Done false
24188 8017 Acked 9016 Read 1 New Done Done false
24189 8018 Acked 9017 Write 5 New Done Done false
24190 8019 Acked 9018 Write 3 New Done Done false
24191 8020 Acked 9019 Read 5 New Done Done false
24192 8021 Acked 9020 Write 5 New Done Done false
24193 8022 Acked 9021 Read 9 New Done Done false
24194 8023 Acked 9022 Write 8 New Done Done false
24195 8024 Acked 9023 Read 5 New Done Done false
24196 8025 Acked 9024 Read 9 New Done Done false
24197 8026 Acked 9025 Flush 0 New Done Done false
24198 8027 Acked 9026 Flush 0 New Done Done false
24199 8028 Acked 9027 Read 2 New Done Done false
24200 8029 Acked 9028 Write 3 New Done Done false
24201 8030 Acked 9029 Read 1 New Done Done false
24202 8031 Acked 9030 Write 8 New Done Done false
24203 8032 Acked 9031 Read 4 New Done Done false
24204 8033 Acked 9032 Read 3 New Done Done false
24205 8034 Acked 9033 Read 3 New Done Done false
24206 8035 Acked 9034 Write 6 New Done Done false
24207 8036 Acked 9035 Write 2 New Done Done false
24208 8037 Acked 9036 Write 6 New Done Done false
24209 8038 Acked 9037 Read 5 New Done Done false
24210 8039 Acked 9038 Write 4 New Done Done false
24211 8040 Acked 9039 Read 1 New Done Done false
24212 8041 Acked 9040 Read 5 New Done Done false
24213 8042 Acked 9041 Write 4 New Done Done false
24214 8043 Acked 9042 Write 4 New Done Done false
24215 8044 Acked 9043 Write 6 New Done Done false
24216 8045 Acked 9044 Flush 0 New Done Done false
24217 8046 Acked 9045 Write 2 New Done Done false
24218 8047 Acked 9046 Read 6 New Done Done false
24219 8048 Acked 9047 Flush 0 New Done Done false
24220 8049 Acked 9048 Write 10 New Done Done false
24221 8050 Acked 9049 Write 4 New Done Done false
24222 8051 Acked 9050 Read 5 New Done Done false
24223 8052 Acked 9051 Write 8 New Done Done false
24224 8053 Acked 9052 Read 3 New Done Done false
24225 8054 Acked 9053 Write 9 New Done Done false
24226 8055 Acked 9054 Write 6 New Done Done false
24227 8056 Acked 9055 Write 1 New Done Done false
24228 8057 Acked 9056 Write 5 New Done Done false
24229 8058 Acked 9057 Flush 0 New Done Done false
24230 8059 Acked 9058 Read 2 New Done Done false
24231 8060 Acked 9059 Write 3 New Done Done false
24232 8061 Acked 9060 Read 10 New Done Done false
24233 8062 Acked 9061 Read 8 New Done Done false
24234 8063 Acked 9062 Read 2 New Done Done false
24235 8064 Acked 9063 Write 5 New Done Done false
24236 8065 Acked 9064 Read 1 New Done Done false
24237 8066 Acked 9065 Read 8 New Done Done false
24238 8067 Acked 9066 Write 5 New Done Done false
24239 8068 Acked 9067 Write 8 New Done Done false
24240 8069 Acked 9068 Write 1 New Done Done false
24241 8070 Acked 9069 Write 4 New Done Done false
24242 8071 Acked 9070 Read 7 New Done Done false
24243 8072 Acked 9071 Read 1 New Done Done false
24244 8073 Acked 9072 Read 2 New Done Done false
24245 8074 Acked 9073 Read 6 New Done Done false
24246 8075 Acked 9074 Read 6 New Done Done false
24247 8076 Acked 9075 Read 7 New Done Done false
24248 8077 Acked 9076 Read 2 New Done Done false
24249 8078 Acked 9077 Read 9 New Done Done false
24250 8079 Acked 9078 Read 7 New Done Done false
24251 8080 Acked 9079 Flush 0 New Done Done false
24252 8081 Acked 9080 Read 6 New Done Done false
24253 8082 Acked 9081 Read 10 New Done Done false
24254 8083 Acked 9082 Write 10 New Done Done false
24255 8084 Acked 9083 Read 9 New Done Done false
24256 8085 Acked 9084 Flush 0 New Done Done false
24257 8086 Acked 9085 Read 5 New Done Done false
24258 8087 Acked 9086 Flush 0 New Done Done false
24259 8088 Acked 9087 Write 2 New Done Done false
24260 8089 Acked 9088 Read 1 New Done Done false
24261 8090 Acked 9089 Read 8 New Done Done false
24262 8091 Acked 9090 Read 2 New Done Done false
24263 8092 Acked 9091 Read 1 New Done Done false
24264 8093 Acked 9092 Read 7 New Done Done false
24265 8094 Acked 9093 Read 5 New Done Done false
24266 8095 Acked 9094 Read 9 New Done Done false
24267 8096 Acked 9095 Write 1 New Done Done false
24268 8097 Acked 9096 Flush 0 New Done Done false
24269 8098 Acked 9097 Write 5 New Done Done false
24270 8099 Acked 9098 Read 10 New Done Done false
24271 8100 Acked 9099 Write 1 New Done Done false
24272 8101 Acked 9100 Read 2 New Done Done false
24273 8102 Acked 9101 Read 5 New Done Done false
24274 8103 Acked 9102 Write 7 New Done Done false
24275 8104 Acked 9103 Read 8 New Done Done false
24276 8105 Acked 9104 Write 8 New Done Done false
24277 8106 Acked 9105 Read 2 New Done Done false
24278 8107 Acked 9106 Write 2 New Done Done false
24279 8108 Acked 9107 Flush 0 New Done Done false
24280 8109 Acked 9108 Read 8 New Done Done false
24281 8110 Acked 9109 Read 3 New Done Done false
24282 8111 Acked 9110 Read 8 New Done Done false
24283 8112 Acked 9111 Read 9 New Done Done false
24284 8113 Acked 9112 Write 8 New Done Done false
24285 8114 Acked 9113 Read 6 New Done Done false
24286 8115 Acked 9114 Write 4 New Done Done false
24287 8116 Acked 9115 Read 6 New Done Done false
24288 8117 Acked 9116 Read 2 New Done Done false
24289 8118 Acked 9117 Flush 0 New Done Done false
24290 8119 Acked 9118 Flush 0 New Done Done false
24291 8120 Acked 9119 Read 4 New Done Done false
24292 8121 Acked 9120 Write 5 New Done Done false
24293 8122 Acked 9121 Read 5 New Done Done false
24294 8123 Acked 9122 Read 2 New Done Done false
24295 8124 Acked 9123 Flush 0 New Done Done false
24296 8125 Acked 9124 Read 7 New Done Done false
24297 8126 Acked 9125 Write 6 New Done Done false
24298 8127 Acked 9126 Write 9 New Done Done false
24299 8128 Acked 9127 Write 6 New Done Done false
24300 8129 Acked 9128 Read 4 New Done Done false
24301 8130 Acked 9129 Write 3 New Done Done false
24302 8131 Acked 9130 Read 8 New Done Done false
24303 8132 Acked 9131 Read 10 New Done Done false
24304 8133 Acked 9132 Read 1 New Done Done false
24305 8134 Acked 9133 Read 2 New Done Done false
24306 8135 Acked 9134 Read 4 New Done Done false
24307 8136 Acked 9135 Read 1 New Done Done false
24308 8137 Acked 9136 Write 1 New Done Done false
24309 8138 Acked 9137 Read 3 New Done Done false
24310 8139 Acked 9138 Flush 0 New Done Done false
24311 8140 Acked 9139 Read 8 New Done Done false
24312 8141 Acked 9140 Read 1 New Done Done false
24313 8142 Acked 9141 Write 8 New Done Done false
24314 8143 Acked 9142 Read 9 New Done Done false
24315 8144 Acked 9143 Write 8 New Done Done false
24316 8145 Acked 9144 Write 7 New Done Done false
24317 8146 Acked 9145 Write 7 New Done Done false
24318 8147 Acked 9146 Read 3 New Done Done false
24319 8148 Acked 9147 Read 7 New Done Done false
24320 8149 Acked 9148 Write 8 New Done Done false
24321 8150 Acked 9149 Read 2 New Done Done false
24322 8151 Acked 9150 Read 1 New Done Done false
24323 8152 Acked 9151 Write 8 New Done Done false
24324 8153 Acked 9152 Read 6 New Done Done false
24325 8154 Acked 9153 Write 5 New Done Done false
24326 8155 Acked 9154 Write 5 New Done Done false
24327 8156 Acked 9155 Read 9 New Done Done false
24328 8157 Acked 9156 Read 4 New Done Done false
24329 8158 Acked 9157 Write 5 New Done Done false
24330 8159 Acked 9158 Write 6 New Done Done false
24331 8160 Acked 9159 Read 8 New Done Done false
24332 8161 Acked 9160 Write 4 New Done Done false
24333 8162 Acked 9161 Read 6 New Done Done false
24334 8163 Acked 9162 Flush 0 New Done Done false
24335 8164 Acked 9163 Read 8 New Done Done false
24336 8165 Acked 9164 Read 4 New Done Done false
24337 8166 Acked 9165 Write 2 New Done Done false
24338 8167 Acked 9166 Write 3 New Sent Sent false
24339 STATES DS:0 DS:1 DS:2 TOTAL
24340 New 302 0 0 302
24341 Sent 0 1 1 2
24342 Done 0 301 301 602
24343 Skipped 0 0 0 0
24344 Error 0 0 0 0
24345 Last Flush: 8864 9162 9162
24346 Downstairs last five completed: 8864 8863 8862 8861 8860 8859
24347 Upstairs last five completed: 8167 8166 8165 8164 8163 8162
24348 CLIENT: Up:0 ds:302 act:2
243492023-09-22T23:29:04.846ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
24350 ----------------------------------------------------------------
24351 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
24352 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
24353 7866 Acked 8865 Read 3 New Done Done false
24354 7867 Acked 8866 Write 8 New Done Done false
24355 7868 Acked 8867 Write 4 New Done Done false
24356 7869 Acked 8868 Write 9 New Done Done false
24357 7870 Acked 8869 Read 9 New Done Done false
24358 7871 Acked 8870 Write 5 New Done Done false
24359 7872 Acked 8871 Read 1 New Done Done false
24360 7873 Acked 8872 Read 10 New Done Done false
24361 7874 Acked 8873 Flush 0 New Done Done false
24362 7875 Acked 8874 Read 4 New Done Done false
24363 7876 Acked 8875 Read 7 New Done Done false
24364 7877 Acked 8876 Read 2 New Done Done false
24365 7878 Acked 8877 Read 10 New Done Done false
24366 7879 Acked 8878 Flush 0 New Done Done false
24367 7880 Acked 8879 Read 9 New Done Done false
24368 7881 Acked 8880 Write 5 New Done Done false
24369 7882 Acked 8881 Write 10 New Done Done false
24370 7883 Acked 8882 Read 3 New Done Done false
24371 7884 Acked 8883 Flush 0 New Done Done false
24372 7885 Acked 8884 Write 2 New Done Done false
24373 7886 Acked 8885 Read 4 New Done Done false
24374 7887 Acked 8886 Write 9 New Done Done false
24375 7888 Acked 8887 Flush 0 New Done Done false
24376 7889 Acked 8888 Flush 0 New Done Done false
24377 7890 Acked 8889 Read 5 New Done Done false
24378 7891 Acked 8890 Read 2 New Done Done false
24379 7892 Acked 8891 Read 10 New Done Done false
24380 7893 Acked 8892 Read 9 New Done Done false
24381 7894 Acked 8893 Write 7 New Done Done false
24382 7895 Acked 8894 Read 10 New Done Done false
24383 7896 Acked 8895 Read 5 New Done Done false
24384 7897 Acked 8896 Write 7 New Done Done false
24385 7898 Acked 8897 Read 2 New Done Done false
24386 7899 Acked 8898 Read 1 New Done Done false
24387 7900 Acked 8899 Read 9 New Done Done false
24388 7901 Acked 8900 Read 10 New Done Done false
24389 7902 Acked 8901 Write 8 New Done Done false
24390 7903 Acked 8902 Write 6 New Done Done false
24391 7904 Acked 8903 Write 7 New Done Done false
24392 7905 Acked 8904 Write 4 New Done Done false
24393 7906 Acked 8905 Read 5 New Done Done false
24394 7907 Acked 8906 Read 9 New Done Done false
24395 7908 Acked 8907 Read 1 New Done Done false
24396 7909 Acked 8908 Read 3 New Done Done false
24397 7910 Acked 8909 Read 9 New Done Done false
24398 7911 Acked 8910 Read 1 New Done Done false
24399 7912 Acked 8911 Read 6 New Done Done false
24400 7913 Acked 8912 Read 1 New Done Done false
24401 7914 Acked 8913 Read 3 New Done Done false
24402 7915 Acked 8914 Write 7 New Done Done false
24403 7916 Acked 8915 Read 5 New Done Done false
24404 7917 Acked 8916 Write 10 New Done Done false
24405 7918 Acked 8917 Read 5 New Done Done false
24406 7919 Acked 8918 Read 10 New Done Done false
24407 7920 Acked 8919 Read 2 New Done Done false
24408 7921 Acked 8920 Flush 0 New Done Done false
24409 7922 Acked 8921 Read 10 New Done Done false
24410 7923 Acked 8922 Flush 0 New Done Done false
24411 7924 Acked 8923 Read 9 New Done Done false
24412 7925 Acked 8924 Read 7 New Done Done false
24413 7926 Acked 8925 Flush 0 New Done Done false
24414 7927 Acked 8926 Write 7 New Done Done false
24415 7928 Acked 8927 Read 2 New Done Done false
24416 7929 Acked 8928 Write 1 New Done Done false
24417 7930 Acked 8929 Write 5 New Done Done false
24418 7931 Acked 8930 Read 5 New Done Done false
24419 7932 Acked 8931 Read 1 New Done Done false
24420 7933 Acked 8932 Write 1 New Done Done false
24421 7934 Acked 8933 Read 7 New Done Done false
24422 7935 Acked 8934 Write 7 New Done Done false
24423 7936 Acked 8935 Read 3 New Done Done false
24424 7937 Acked 8936 Read 8 New Done Done false
24425 7938 Acked 8937 Write 8 New Done Done false
24426 7939 Acked 8938 Write 2 New Done Done false
24427 7940 Acked 8939 Read 9 New Done Done false
24428 7941 Acked 8940 Read 9 New Done Done false
24429 7942 Acked 8941 Read 10 New Done Done false
24430 7943 Acked 8942 Flush 0 New Done Done false
24431 7944 Acked 8943 Flush 0 New Done Done false
24432 7945 Acked 8944 Read 1 New Done Done false
24433 7946 Acked 8945 Read 4 New Done Done false
24434 7947 Acked 8946 Write 3 New Done Done false
24435 7948 Acked 8947 Write 3 New Done Done false
24436 7949 Acked 8948 Flush 0 New Done Done false
24437 7950 Acked 8949 Write 5 New Done Done false
24438 7951 Acked 8950 Read 8 New Done Done false
24439 7952 Acked 8951 Write 1 New Done Done false
24440 7953 Acked 8952 Write 4 New Done Done false
24441 7954 Acked 8953 Write 2 New Done Done false
24442 7955 Acked 8954 Read 4 New Done Done false
24443 7956 Acked 8955 Read 2 New Done Done false
24444 7957 Acked 8956 Read 2 New Done Done false
24445 7958 Acked 8957 Read 2 New Done Done false
24446 7959 Acked 8958 Flush 0 New Done Done false
24447 7960 Acked 8959 Write 6 New Done Done false
24448 7961 Acked 8960 Read 9 New Done Done false
24449 7962 Acked 8961 Read 2 New Done Done false
24450 7963 Acked 8962 Read 10 New Done Done false
24451 7964 Acked 8963 Read 1 New Done Done false
24452 7965 Acked 8964 Read 8 New Done Done false
24453 7966 Acked 8965 Write 5 New Done Done false
24454 7967 Acked 8966 Read 7 New Done Done false
24455 7968 Acked 8967 Read 6 New Done Done false
24456 7969 Acked 8968 Write 4 New Done Done false
24457 7970 Acked 8969 Read 10 New Done Done false
24458 7971 Acked 8970 Flush 0 New Done Done false
24459 7972 Acked 8971 Read 10 New Done Done false
24460 7973 Acked 8972 Write 8 New Done Done false
24461 7974 Acked 8973 Write 2 New Done Done false
24462 7975 Acked 8974 Read 2 New Done Done false
24463 7976 Acked 8975 Read 1 New Done Done false
24464 7977 Acked 8976 Write 9 New Done Done false
24465 7978 Acked 8977 Read 10 New Done Done false
24466 7979 Acked 8978 Flush 0 New Done Done false
24467 7980 Acked 8979 Read 6 New Done Done false
24468 7981 Acked 8980 Write 1 New Done Done false
24469 7982 Acked 8981 Read 10 New Done Done false
24470 7983 Acked 8982 Flush 0 New Done Done false
24471 7984 Acked 8983 Read 5 New Done Done false
24472 7985 Acked 8984 Read 8 New Done Done false
24473 7986 Acked 8985 Write 9 New Done Done false
24474 7987 Acked 8986 Read 10 New Done Done false
24475 7988 Acked 8987 Write 4 New Done Done false
24476 7989 Acked 8988 Write 2 New Done Done false
24477 7990 Acked 8989 Read 2 New Done Done false
24478 7991 Acked 8990 Read 6 New Done Done false
24479 7992 Acked 8991 Flush 0 New Done Done false
24480 7993 Acked 8992 Write 1 New Done Done false
24481 7994 Acked 8993 Write 8 New Done Done false
24482 7995 Acked 8994 Read 9 New Done Done false
24483 7996 Acked 8995 Write 5 New Done Done false
24484 7997 Acked 8996 Write 3 New Done Done false
24485 7998 Acked 8997 Read 6 New Done Done false
24486 7999 Acked 8998 Read 2 New Done Done false
24487 8000 Acked 8999 Read 9 New Done Done false
24488 8001 Acked 9000 Write 10 New Done Done false
24489 8002 Acked 9001 Read 3 New Done Done false
24490 8003 Acked 9002 Read 9 New Done Done false
24491 8004 Acked 9003 Write 4 New Done Done false
24492 8005 Acked 9004 Read 8 New Done Done false
24493 8006 Acked 9005 Write 4 New Done Done false
24494 8007 Acked 9006 Read 7 New Done Done false
24495 8008 Acked 9007 Read 6 New Done Done false
24496 8009 Acked 9008 Write 8 New Done Done false
24497 8010 Acked 9009 Read 4 New Done Done false
24498 8011 Acked 9010 Write 4 New Done Done false
24499 8012 Acked 9011 Flush 0 New Done Done false
24500 8013 Acked 9012 Read 5 New Done Done false
24501 8014 Acked 9013 Write 10 New Done Done false
24502 8015 Acked 9014 Read 9 New Done Done false
24503 8016 Acked 9015 Read 1 New Done Done false
24504 8017 Acked 9016 Read 1 New Done Done false
24505 8018 Acked 9017 Write 5 New Done Done false
24506 8019 Acked 9018 Write 3 New Done Done false
24507 8020 Acked 9019 Read 5 New Done Done false
24508 8021 Acked 9020 Write 5 New Done Done false
24509 8022 Acked 9021 Read 9 New Done Done false
24510 8023 Acked 9022 Write 8 New Done Done false
24511 8024 Acked 9023 Read 5 New Done Done false
24512 8025 Acked 9024 Read 9 New Done Done false
24513 8026 Acked 9025 Flush 0 New Done Done false
24514 8027 Acked 9026 Flush 0 New Done Done false
24515 8028 Acked 9027 Read 2 New Done Done false
24516 8029 Acked 9028 Write 3 New Done Done false
24517 8030 Acked 9029 Read 1 New Done Done false
24518 8031 Acked 9030 Write 8 New Done Done false
24519 8032 Acked 9031 Read 4 New Done Done false
24520 8033 Acked 9032 Read 3 New Done Done false
24521 8034 Acked 9033 Read 3 New Done Done false
24522 8035 Acked 9034 Write 6 New Done Done false
24523 8036 Acked 9035 Write 2 New Done Done false
24524 8037 Acked 9036 Write 6 New Done Done false
24525 8038 Acked 9037 Read 5 New Done Done false
24526 8039 Acked 9038 Write 4 New Done Done false
24527 8040 Acked 9039 Read 1 New Done Done false
24528 8041 Acked 9040 Read 5 New Done Done false
24529 8042 Acked 9041 Write 4 New Done Done false
24530 8043 Acked 9042 Write 4 New Done Done false
24531 8044 Acked 9043 Write 6 New Done Done false
24532 8045 Acked 9044 Flush 0 New Done Done false
24533 8046 Acked 9045 Write 2 New Done Done false
24534 8047 Acked 9046 Read 6 New Done Done false
24535 8048 Acked 9047 Flush 0 New Done Done false
24536 8049 Acked 9048 Write 10 New Done Done false
24537 8050 Acked 9049 Write 4 New Done Done false
24538 8051 Acked 9050 Read 5 New Done Done false
24539 8052 Acked 9051 Write 8 New Done Done false
24540 8053 Acked 9052 Read 3 New Done Done false
24541 8054 Acked 9053 Write 9 New Done Done false
24542 8055 Acked 9054 Write 6 New Done Done false
24543 8056 Acked 9055 Write 1 New Done Done false
24544 8057 Acked 9056 Write 5 New Done Done false
24545 8058 Acked 9057 Flush 0 New Done Done false
24546 8059 Acked 9058 Read 2 New Done Done false
24547 8060 Acked 9059 Write 3 New Done Done false
24548 8061 Acked 9060 Read 10 New Done Done false
24549 8062 Acked 9061 Read 8 New Done Done false
24550 8063 Acked 9062 Read 2 New Done Done false
24551 8064 Acked 9063 Write 5 New Done Done false
24552 8065 Acked 9064 Read 1 New Done Done false
24553 8066 Acked 9065 Read 8 New Done Done false
24554 8067 Acked 9066 Write 5 New Done Done false
24555 8068 Acked 9067 Write 8 New Done Done false
24556 8069 Acked 9068 Write 1 New Done Done false
24557 8070 Acked 9069 Write 4 New Done Done false
24558 8071 Acked 9070 Read 7 New Done Done false
24559 8072 Acked 9071 Read 1 New Done Done false
24560 8073 Acked 9072 Read 2 New Done Done false
24561 8074 Acked 9073 Read 6 New Done Done false
24562 8075 Acked 9074 Read 6 New Done Done false
24563 8076 Acked 9075 Read 7 New Done Done false
24564 8077 Acked 9076 Read 2 New Done Done false
24565 8078 Acked 9077 Read 9 New Done Done false
24566 8079 Acked 9078 Read 7 New Done Done false
24567 8080 Acked 9079 Flush 0 New Done Done false
24568 8081 Acked 9080 Read 6 New Done Done false
24569 8082 Acked 9081 Read 10 New Done Done false
24570 8083 Acked 9082 Write 10 New Done Done false
24571 8084 Acked 9083 Read 9 New Done Done false
24572 8085 Acked 9084 Flush 0 New Done Done false
24573 8086 Acked 9085 Read 5 New Done Done false
24574 8087 Acked 9086 Flush 0 New Done Done false
24575 8088 Acked 9087 Write 2 New Done Done false
24576 8089 Acked 9088 Read 1 New Done Done false
24577 8090 Acked 9089 Read 8 New Done Done false
24578 8091 Acked 9090 Read 2 New Done Done false
24579 8092 Acked 9091 Read 1 New Done Done false
24580 8093 Acked 9092 Read 7 New Done Done false
24581 8094 Acked 9093 Read 5 New Done Done false
24582 8095 Acked 9094 Read 9 New Done Done false
24583 8096 Acked 9095 Write 1 New Done Done false
24584 8097 Acked 9096 Flush 0 New Done Done false
24585 8098 Acked 9097 Write 5 New Done Done false
24586 8099 Acked 9098 Read 10 New Done Done false
24587 8100 Acked 9099 Write 1 New Done Done false
24588 8101 Acked 9100 Read 2 New Done Done false
24589 8102 Acked 9101 Read 5 New Done Done false
24590 8103 Acked 9102 Write 7 New Done Done false
24591 8104 Acked 9103 Read 8 New Done Done false
24592 8105 Acked 9104 Write 8 New Done Done false
24593 8106 Acked 9105 Read 2 New Done Done false
24594 8107 Acked 9106 Write 2 New Done Done false
24595 8108 Acked 9107 Flush 0 New Done Done false
24596 8109 Acked 9108 Read 8 New Done Done false
24597 8110 Acked 9109 Read 3 New Done Done false
24598 8111 Acked 9110 Read 8 New Done Done false
24599 8112 Acked 9111 Read 9 New Done Done false
24600 8113 Acked 9112 Write 8 New Done Done false
24601 8114 Acked 9113 Read 6 New Done Done false
24602 8115 Acked 9114 Write 4 New Done Done false
24603 8116 Acked 9115 Read 6 New Done Done false
24604 8117 Acked 9116 Read 2 New Done Done false
24605 8118 Acked 9117 Flush 0 New Done Done false
24606 8119 Acked 9118 Flush 0 New Done Done false
24607 8120 Acked 9119 Read 4 New Done Done false
24608 8121 Acked 9120 Write 5 New Done Done false
24609 8122 Acked 9121 Read 5 New Done Done false
24610 8123 Acked 9122 Read 2 New Done Done false
24611 8124 Acked 9123 Flush 0 New Done Done false
24612 8125 Acked 9124 Read 7 New Done Done false
24613 8126 Acked 9125 Write 6 New Done Done false
24614 8127 Acked 9126 Write 9 New Done Done false
24615 8128 Acked 9127 Write 6 New Done Done false
24616 8129 Acked 9128 Read 4 New Done Done false
24617 8130 Acked 9129 Write 3 New Done Done false
24618 8131 Acked 9130 Read 8 New Done Done false
24619 8132 Acked 9131 Read 10 New Done Done false
24620 8133 Acked 9132 Read 1 New Done Done false
24621 8134 Acked 9133 Read 2 New Done Done false
24622 8135 Acked 9134 Read 4 New Done Done false
24623 8136 Acked 9135 Read 1 New Done Done false
24624 8137 Acked 9136 Write 1 New Done Done false
24625 8138 Acked 9137 Read 3 New Done Done false
24626 8139 Acked 9138 Flush 0 New Done Done false
24627 8140 Acked 9139 Read 8 New Done Done false
24628 8141 Acked 9140 Read 1 New Done Done false
24629 8142 Acked 9141 Write 8 New Done Done false
24630 8143 Acked 9142 Read 9 New Done Done false
24631 8144 Acked 9143 Write 8 New Done Done false
24632 8145 Acked 9144 Write 7 New Done Done false
24633 8146 Acked 9145 Write 7 New Done Done false
24634 8147 Acked 9146 Read 3 New Done Done false
24635 8148 Acked 9147 Read 7 New Done Done false
24636 8149 Acked 9148 Write 8 New Done Done false
24637 8150 Acked 9149 Read 2 New Done Done false
24638 8151 Acked 9150 Read 1 New Done Done false
24639 8152 Acked 9151 Write 8 New Done Done false
24640 8153 Acked 9152 Read 6 New Done Done false
24641 8154 Acked 9153 Write 5 New Done Done false
24642 8155 Acked 9154 Write 5 New Done Done false
24643 8156 Acked 9155 Read 9 New Done Done false
24644 8157 Acked 9156 Read 4 New Done Done false
24645 8158 Acked 9157 Write 5 New Done Done false
24646 8159 Acked 9158 Write 6 New Done Done false
24647 8160 Acked 9159 Read 8 New Done Done false
24648 8161 Acked 9160 Write 4 New Done Done false
24649 8162 Acked 9161 Read 6 New Done Done false
24650 8163 Acked 9162 Flush 0 New Done Done false
24651 8164 Acked 9163 Read 8 New Done Done false
24652 8165 Acked 9164 Read 4 New Done Done false
24653 8166 Acked 9165 Write 2 New Done Done false
24654 8167 Acked 9166 Write 3 New Done Done false
24655 8168 Acked 9167 Flush 0 New Done Done false
24656 STATES DS:0 DS:1 DS:2 TOTAL
24657 New 303 0 0 303
24658 Sent 0 0 0 0
24659 Done 0 303 303 606
24660 Skipped 0 0 0 0
24661 Error 0 0 0 0
24662 Last Flush: 8864 9167 9167
24663 Downstairs last five completed: 8864 8863 8862 8861 8860 8859
24664 Upstairs last five completed: 8168 8167 8166 8165 8164 8163
24665 CLIENT: Up:0 ds:303 act:2
24666 ----------------------------------------------------------------
24667 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:303
24668 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
24669 7866 Acked 8865 Read 3 New Done Done false
24670 7867 Acked 8866 Write 8 New Done Done false
24671 7868 Acked 8867 Write 4 New Done Done false
24672 7869 Acked 8868 Write 9 New Done Done false
24673 7870 Acked 8869 Read 9 New Done Done false
24674 7871 Acked 8870 Write 5 New Done Done false
24675 7872 Acked 8871 Read 1 New Done Done false
24676 7873 Acked 8872 Read 10 New Done Done false
24677 7874 Acked 8873 Flush 0 New Done Done false
24678 7875 Acked 8874 Read 4 New Done Done false
24679 7876 Acked 8875 Read 7 New Done Done false
24680 7877 Acked 8876 Read 2 New Done Done false
24681 7878 Acked 8877 Read 10 New Done Done false
24682 7879 Acked 8878 Flush 0 New Done Done false
24683 7880 Acked 8879 Read 9 New Done Done false
24684 7881 Acked 8880 Write 5 New Done Done false
24685 7882 Acked 8881 Write 10 New Done Done false
24686 7883 Acked 8882 Read 3 New Done Done false
24687 7884 Acked 8883 Flush 0 New Done Done false
24688 7885 Acked 8884 Write 2 New Done Done false
24689 7886 Acked 8885 Read 4 New Done Done false
24690 7887 Acked 8886 Write 9 New Done Done false
24691 7888 Acked 8887 Flush 0 New Done Done false
24692 7889 Acked 8888 Flush 0 New Done Done false
24693 7890 Acked 8889 Read 5 New Done Done false
24694 7891 Acked 8890 Read 2 New Done Done false
24695 7892 Acked 8891 Read 10 New Done Done false
24696 7893 Acked 8892 Read 9 New Done Done false
24697 7894 Acked 8893 Write 7 New Done Done false
24698 7895 Acked 8894 Read 10 New Done Done false
24699 7896 Acked 8895 Read 5 New Done Done false
24700 7897 Acked 8896 Write 7 New Done Done false
24701 7898 Acked 8897 Read 2 New Done Done false
24702 7899 Acked 8898 Read 1 New Done Done false
24703 7900 Acked 8899 Read 9 New Done Done false
24704 7901 Acked 8900 Read 10 New Done Done false
24705 7902 Acked 8901 Write 8 New Done Done false
24706 7903 Acked 8902 Write 6 New Done Done false
24707 7904 Acked 8903 Write 7 New Done Done false
24708 7905 Acked 8904 Write 4 New Done Done false
24709 7906 Acked 8905 Read 5 New Done Done false
24710 7907 Acked 8906 Read 9 New Done Done false
24711 7908 Acked 8907 Read 1 New Done Done false
24712 7909 Acked 8908 Read 3 New Done Done false
24713 7910 Acked 8909 Read 9 New Done Done false
24714 7911 Acked 8910 Read 1 New Done Done false
24715 7912 Acked 8911 Read 6 New Done Done false
24716 7913 Acked 8912 Read 1 New Done Done false
24717 7914 Acked 8913 Read 3 New Done Done false
24718 7915 Acked 8914 Write 7 New Done Done false
24719 7916 Acked 8915 Read 5 New Done Done false
24720 7917 Acked 8916 Write 10 New Done Done false
24721 7918 Acked 8917 Read 5 New Done Done false
24722 7919 Acked 8918 Read 10 New Done Done false
24723 7920 Acked 8919 Read 2 New Done Done false
24724 7921 Acked 8920 Flush 0 New Done Done false
24725 7922 Acked 8921 Read 10 New Done Done false
24726 7923 Acked 8922 Flush 0 New Done Done false
24727 7924 Acked 8923 Read 9 New Done Done false
24728 7925 Acked 8924 Read 7 New Done Done false
24729 7926 Acked 8925 Flush 0 New Done Done false
24730 7927 Acked 8926 Write 7 New Done Done false
24731 7928 Acked 8927 Read 2 New Done Done false
24732 7929 Acked 8928 Write 1 New Done Done false
24733 7930 Acked 8929 Write 5 New Done Done false
24734 7931 Acked 8930 Read 5 New Done Done false
24735 7932 Acked 8931 Read 1 New Done Done false
24736 7933 Acked 8932 Write 1 New Done Done false
24737 7934 Acked 8933 Read 7 New Done Done false
24738 7935 Acked 8934 Write 7 New Done Done false
24739 7936 Acked 8935 Read 3 New Done Done false
24740 7937 Acked 8936 Read 8 New Done Done false
24741 7938 Acked 8937 Write 8 New Done Done false
24742 7939 Acked 8938 Write 2 New Done Done false
24743 7940 Acked 8939 Read 9 New Done Done false
24744 7941 Acked 8940 Read 9 New Done Done false
24745 7942 Acked 8941 Read 10 New Done Done false
24746 7943 Acked 8942 Flush 0 New Done Done false
24747 7944 Acked 8943 Flush 0 New Done Done false
24748 7945 Acked 8944 Read 1 New Done Done false
24749 7946 Acked 8945 Read 4 New Done Done false
24750 7947 Acked 8946 Write 3 New Done Done false
24751 7948 Acked 8947 Write 3 New Done Done false
24752 7949 Acked 8948 Flush 0 New Done Done false
24753 7950 Acked 8949 Write 5 New Done Done false
24754 7951 Acked 8950 Read 8 New Done Done false
24755 7952 Acked 8951 Write 1 New Done Done false
24756 7953 Acked 8952 Write 4 New Done Done false
24757 7954 Acked 8953 Write 2 New Done Done false
24758 7955 Acked 8954 Read 4 New Done Done false
24759 7956 Acked 8955 Read 2 New Done Done false
24760 7957 Acked 8956 Read 2 New Done Done false
24761 7958 Acked 8957 Read 2 New Done Done false
24762 7959 Acked 8958 Flush 0 New Done Done false
24763 7960 Acked 8959 Write 6 New Done Done false
24764 7961 Acked 8960 Read 9 New Done Done false
24765 7962 Acked 8961 Read 2 New Done Done false
24766 7963 Acked 8962 Read 10 New Done Done false
24767 7964 Acked 8963 Read 1 New Done Done false
24768 7965 Acked 8964 Read 8 New Done Done false
24769 7966 Acked 8965 Write 5 New Done Done false
24770 7967 Acked 8966 Read 7 New Done Done false
24771 7968 Acked 8967 Read 6 New Done Done false
24772 7969 Acked 8968 Write 4 New Done Done false
24773 7970 Acked 8969 Read 10 New Done Done false
24774 7971 Acked 8970 Flush 0 New Done Done false
24775 7972 Acked 8971 Read 10 New Done Done false
24776 7973 Acked 8972 Write 8 New Done Done false
24777 7974 Acked 8973 Write 2 New Done Done false
24778 7975 Acked 8974 Read 2 New Done Done false
24779 7976 Acked 8975 Read 1 New Done Done false
24780 7977 Acked 8976 Write 9 New Done Done false
24781 7978 Acked 8977 Read 10 New Done Done false
24782 7979 Acked 8978 Flush 0 New Done Done false
24783 7980 Acked 8979 Read 6 New Done Done false
24784 7981 Acked 8980 Write 1 New Done Done false
24785 7982 Acked 8981 Read 10 New Done Done false
24786 7983 Acked 8982 Flush 0 New Done Done false
24787 7984 Acked 8983 Read 5 New Done Done false
24788 7985 Acked 8984 Read 8 New Done Done false
24789 7986 Acked 8985 Write 9 New Done Done false
24790 7987 Acked 8986 Read 10 New Done Done false
24791 7988 Acked 8987 Write 4 New Done Done false
24792 7989 Acked 8988 Write 2 New Done Done false
24793 7990 Acked 8989 Read 2 New Done Done false
24794 7991 Acked 8990 Read 6 New Done Done false
24795 7992 Acked 8991 Flush 0 New Done Done false
24796 7993 Acked 8992 Write 1 New Done Done false
24797 7994 Acked 8993 Write 8 New Done Done false
24798 7995 Acked 8994 Read 9 New Done Done false
24799 7996 Acked 8995 Write 5 New Done Done false
24800 7997 Acked 8996 Write 3 New Done Done false
24801 7998 Acked 8997 Read 6 New Done Done false
24802 7999 Acked 8998 Read 2 New Done Done false
24803 8000 Acked 8999 Read 9 New Done Done false
24804 8001 Acked 9000 Write 10 New Done Done false
24805 8002 Acked 9001 Read 3 New Done Done false
24806 8003 Acked 9002 Read 9 New Done Done false
24807 8004 Acked 9003 Write 4 New Done Done false
24808 8005 Acked 9004 Read 8 New Done Done false
24809 8006 Acked 9005 Write 4 New Done Done false
24810 8007 Acked 9006 Read 7 New Done Done false
24811 8008 Acked 9007 Read 6 New Done Done false
24812 8009 Acked 9008 Write 8 New Done Done false
24813 8010 Acked 9009 Read 4 New Done Done false
24814 8011 Acked 9010 Write 4 New Done Done false
24815 8012 Acked 9011 Flush 0 New Done Done false
24816 8013 Acked 9012 Read 5 New Done Done false
24817 8014 Acked 9013 Write 10 New Done Done false
24818 8015 Acked 9014 Read 9 New Done Done false
24819 8016 Acked 9015 Read 1 New Done Done false
24820 8017 Acked 9016 Read 1 New Done Done false
24821 8018 Acked 9017 Write 5 New Done Done false
24822 8019 Acked 9018 Write 3 New Done Done false
24823 8020 Acked 9019 Read 5 New Done Done false
24824 8021 Acked 9020 Write 5 New Done Done false
24825 8022 Acked 9021 Read 9 New Done Done false
24826 8023 Acked 9022 Write 8 New Done Done false
24827 8024 Acked 9023 Read 5 New Done Done false
24828 8025 Acked 9024 Read 9 New Done Done false
24829 8026 Acked 9025 Flush 0 New Done Done false
24830 8027 Acked 9026 Flush 0 New Done Done false
24831 8028 Acked 9027 Read 2 New Done Done false
24832 8029 Acked 9028 Write 3 New Done Done false
24833 8030 Acked 9029 Read 1 New Done Done false
24834 8031 Acked 9030 Write 8 New Done Done false
24835 8032 Acked 9031 Read 4 New Done Done false
24836 8033 Acked 9032 Read 3 New Done Done false
24837 8034 Acked 9033 Read 3 New Done Done false
24838 8035 Acked 9034 Write 6 New Done Done false
24839 8036 Acked 9035 Write 2 New Done Done false
24840 8037 Acked 9036 Write 6 New Done Done false
24841 8038 Acked 9037 Read 5 New Done Done false
24842 8039 Acked 9038 Write 4 New Done Done false
24843 8040 Acked 9039 Read 1 New Done Done false
24844 8041 Acked 9040 Read 5 New Done Done false
24845 8042 Acked 9041 Write 4 New Done Done false
24846 8043 Acked 9042 Write 4 New Done Done false
24847 8044 Acked 9043 Write 6 New Done Done false
24848 8045 Acked 9044 Flush 0 New Done Done false
24849 8046 Acked 9045 Write 2 New Done Done false
24850 8047 Acked 9046 Read 6 New Done Done false
24851 8048 Acked 9047 Flush 0 New Done Done false
24852 8049 Acked 9048 Write 10 New Done Done false
24853 8050 Acked 9049 Write 4 New Done Done false
24854 8051 Acked 9050 Read 5 New Done Done false
24855 8052 Acked 9051 Write 8 New Done Done false
24856 8053 Acked 9052 Read 3 New Done Done false
24857 8054 Acked 9053 Write 9 New Done Done false
24858 8055 Acked 9054 Write 6 New Done Done false
24859 8056 Acked 9055 Write 1 New Done Done false
24860 8057 Acked 9056 Write 5 New Done Done false
24861 8058 Acked 9057 Flush 0 New Done Done false
24862 8059 Acked 9058 Read 2 New Done Done false
24863 8060 Acked 9059 Write 3 New Done Done false
24864 8061 Acked 9060 Read 10 New Done Done false
24865 8062 Acked 9061 Read 8 New Done Done false
24866 8063 Acked 9062 Read 2 New Done Done false
24867 8064 Acked 9063 Write 5 New Done Done false
24868 8065 Acked 9064 Read 1 New Done Done false
24869 8066 Acked 9065 Read 8 New Done Done false
24870 8067 Acked 9066 Write 5 New Done Done false
24871 8068 Acked 9067 Write 8 New Done Done false
24872 8069 Acked 9068 Write 1 New Done Done false
24873 8070 Acked 9069 Write 4 New Done Done false
24874 8071 Acked 9070 Read 7 New Done Done false
24875 8072 Acked 9071 Read 1 New Done Done false
24876 8073 Acked 9072 Read 2 New Done Done false
24877 8074 Acked 9073 Read 6 New Done Done false
24878 8075 Acked 9074 Read 6 New Done Done false
24879 8076 Acked 9075 Read 7 New Done Done false
24880 8077 Acked 9076 Read 2 New Done Done false
24881 8078 Acked 9077 Read 9 New Done Done false
24882 8079 Acked 9078 Read 7 New Done Done false
24883 8080 Acked 9079 Flush 0 New Done Done false
24884 8081 Acked 9080 Read 6 New Done Done false
24885 8082 Acked 9081 Read 10 New Done Done false
24886 8083 Acked 9082 Write 10 New Done Done false
24887 8084 Acked 9083 Read 9 New Done Done false
24888 8085 Acked 9084 Flush 0 New Done Done false
24889 8086 Acked 9085 Read 5 New Done Done false
24890 8087 Acked 9086 Flush 0 New Done Done false
24891 8088 Acked 9087 Write 2 New Done Done false
24892 8089 Acked 9088 Read 1 New Done Done false
24893 8090 Acked 9089 Read 8 New Done Done false
24894 8091 Acked 9090 Read 2 New Done Done false
24895 8092 Acked 9091 Read 1 New Done Done false
24896 8093 Acked 9092 Read 7 New Done Done false
24897 8094 Acked 9093 Read 5 New Done Done false
24898 8095 Acked 9094 Read 9 New Done Done false
24899 8096 Acked 9095 Write 1 New Done Done false
24900 8097 Acked 9096 Flush 0 New Done Done false
24901 8098 Acked 9097 Write 5 New Done Done false
24902 8099 Acked 9098 Read 10 New Done Done false
24903 8100 Acked 9099 Write 1 New Done Done false
24904 8101 Acked 9100 Read 2 New Done Done false
24905 8102 Acked 9101 Read 5 New Done Done false
24906 8103 Acked 9102 Write 7 New Done Done false
24907 8104 Acked 9103 Read 8 New Done Done false
24908 8105 Acked 9104 Write 8 New Done Done false
24909 8106 Acked 9105 Read 2 New Done Done false
24910 8107 Acked 9106 Write 2 New Done Done false
24911 8108 Acked 9107 Flush 0 New Done Done false
24912 8109 Acked 9108 Read 8 New Done Done false
24913 8110 Acked 9109 Read 3 New Done Done false
24914 8111 Acked 9110 Read 8 New Done Done false
24915 8112 Acked 9111 Read 9 New Done Done false
24916 8113 Acked 9112 Write 8 New Done Done false
24917 8114 Acked 9113 Read 6 New Done Done false
24918 8115 Acked 9114 Write 4 New Done Done false
24919 8116 Acked 9115 Read 6 New Done Done false
24920 8117 Acked 9116 Read 2 New Done Done false
24921 8118 Acked 9117 Flush 0 New Done Done false
24922 8119 Acked 9118 Flush 0 New Done Done false
24923 8120 Acked 9119 Read 4 New Done Done false
24924 8121 Acked 9120 Write 5 New Done Done false
24925 8122 Acked 9121 Read 5 New Done Done false
24926 8123 Acked 9122 Read 2 New Done Done false
24927 8124 Acked 9123 Flush 0 New Done Done false
24928 8125 Acked 9124 Read 7 New Done Done false
24929 8126 Acked 9125 Write 6 New Done Done false
24930 8127 Acked 9126 Write 9 New Done Done false
24931 8128 Acked 9127 Write 6 New Done Done false
24932 8129 Acked 9128 Read 4 New Done Done false
24933 8130 Acked 9129 Write 3 New Done Done false
24934 8131 Acked 9130 Read 8 New Done Done false
24935 8132 Acked 9131 Read 10 New Done Done false
24936 8133 Acked 9132 Read 1 New Done Done false
24937 8134 Acked 9133 Read 2 New Done Done false
24938 8135 Acked 9134 Read 4 New Done Done false
24939 8136 Acked 9135 Read 1 New Done Done false
24940 8137 Acked 9136 Write 1 New Done Done false
24941 8138 Acked 9137 Read 3 New Done Done false
24942 8139 Acked 9138 Flush 0 New Done Done false
24943 8140 Acked 9139 Read 8 New Done Done false
24944 8141 Acked 9140 Read 1 New Done Done false
24945 8142 Acked 9141 Write 8 New Done Done false
24946 8143 Acked 9142 Read 9 New Done Done false
24947 8144 Acked 9143 Write 8 New Done Done false
24948 8145 Acked 9144 Write 7 New Done Done false
24949 8146 Acked 9145 Write 7 New Done Done false
24950 8147 Acked 9146 Read 3 New Done Done false
24951 8148 Acked 9147 Read 7 New Done Done false
24952 8149 Acked 9148 Write 8 New Done Done false
24953 8150 Acked 9149 Read 2 New Done Done false
24954 8151 Acked 9150 Read 1 New Done Done false
24955 8152 Acked 9151 Write 8 New Done Done false
24956 8153 Acked 9152 Read 6 New Done Done false
24957 8154 Acked 9153 Write 5 New Done Done false
24958 8155 Acked 9154 Write 5 New Done Done false
24959 8156 Acked 9155 Read 9 New Done Done false
24960 8157 Acked 9156 Read 4 New Done Done false
24961 8158 Acked 9157 Write 5 New Done Done false
24962 8159 Acked 9158 Write 6 New Done Done false
24963 8160 Acked 9159 Read 8 New Done Done false
24964 8161 Acked 9160 Write 4 New Done Done false
24965 8162 Acked 9161 Read 6 New Done Done false
24966 8163 Acked 9162 Flush 0 New Done Done false
24967 8164 Acked 9163 Read 8 New Done Done false
24968 8165 Acked 9164 Read 4 New Done Done false
24969 8166 Acked 9165 Write 2 New Done Done false
24970 8167 Acked 9166 Write 3 New Done Done false
24971 8168 Acked 9167 Flush 0 New Done Done false
24972 STATES DS:0 DS:1 DS:2 TOTAL
24973 New 303 0 0 303
24974 Sent 0 0 0 0
24975 Done 0 303 303 606
24976 Skipped 0 0 0 0
24977 Error 0 0 0 0
24978 Last Flush: 8864 9167 9167
24979 Downstairs last five completed: 8864 8863 8862 8861 8860 8859
24980 Upstairs last five completed: 8168 8167 8166 8165 8164 8163
24981 CLIENT: Up:0 ds:303 act:2
24982 [8810] got start action
24983 [8810] I am going to restart
24984 Starting downstairs at port 8810
24985 Make output file at "/tmp/dsc/downstairs-8810.txt"
24986 [8810][0] reports Starting
24987 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1200)
24988 [8810][0] reports Running
249892023-09-22T23:29:13.855ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
249902023-09-22T23:29:13.855ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
249912023-09-22T23:29:13.855ZINFOcrucible: [0] client re-new 303 jobs since flush 8864 = downstairs
249922023-09-22T23:29:13.856ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
249932023-09-22T23:29:13.857ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
249942023-09-22T23:29:13.857ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
249952023-09-22T23:29:13.857ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
249962023-09-22T23:29:13.857ZINFOcrucible: [0] send last flush ID to this DS: 8864
249972023-09-22T23:29:13.857ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
249982023-09-22T23:29:13.857ZINFOcrucible: [0] Replied this last flush ID: 8864
249992023-09-22T23:29:13.857ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
250002023-09-22T23:29:13.857ZINFOcrucible: [0] Transition from Offline to Replay
250012023-09-22T23:29:13.857ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
250022023-09-22T23:29:13.858ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
250032023-09-22T23:29:13.858ZINFOcrucible: [0] Transition from Replay to Active
250042023-09-22T23:29:13.858ZINFOcrucible: [0] Starts cmd_loop
250052023-09-22T23:29:13.858ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
250062023-09-22T23:29:13.858ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
250072023-09-22T23:29:13.858ZINFOcrucible: Set check for repair
250082023-09-22T23:29:14.859ZINFOcrucible: Checking if live repair is needed
250092023-09-22T23:29:14.859ZINFOcrucible: No Live Repair required at this time
250102023-09-22T23:29:14.859ZWARNcrucible: [0] flow control end
25011 ----------------------------------------------------------------
25012 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25013 Downstairs last five completed: 9167 9166 9165 9164 9163 9162
25014 Upstairs last five completed: 8168 8167 8166 8165 8164 8163
25015 CLIENT: Up:0 ds:0 act:3
25016 Replay: All jobs finished, all DS active.
25017 Main task has work to do, go find it
25018 got dsc Stop(2)
25019 stop 2
25020 Sep 22 23:29:15.803 INFO request completed, latency_us: 515, response_code: 204, uri: /stop/cid/2, method[8830] Got stop action so:false kr:false
25021 : POST, req_id: d0224700-5fd8-47f8-839a-f6cc5372a8d7, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
25022 Sep 22 23:29:15.804 INFO accepted connection, remote_addr: 127.0.0.1:50413, local_addr: 127.0.0.1:9998
25023 Sep 22 23:29:15.804 INFO request completed, latency_us: 182, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2f86703f-3915-4931-8c7f-4f7c7548a19f, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
250242023-09-22T23:29:15.804ZWARNcrucible: [2] downstairs disconnected
250252023-09-22T23:29:15.804ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
250262023-09-22T23:29:15.804ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8830 closed looper = 2
250272023-09-22T23:29:15.804ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:false
250282023-09-22T23:29:15.805ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
250292023-09-22T23:29:15.805ZINFOcrucible: [2] 127.0.0.1:8830 task reports offline
250302023-09-22T23:29:15.805ZWARNcrucible: [2] pm_task rx.recv() is None
25031 [8830] Exited with: ExitStatus(unix_wait_status(9))
25032 [8830][2] reports Exit
25033 Sep 22 23:29:19.806 INFO request completed, latency_us: 260, response_code: 200, uri: /state/cid/2, method: GET, req_id: 56c0f6d0-6197-4b65-ab53-19159be91d8a, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
25034 001/300 Read block 2060 len 20480
25035 002/300 Read block 2085 len 12288
25036 003/300 Write block 4723 len 4096 data: 4
25037 004/300 Read block 2593 len 36864
25038 005/300 Write block 2956 len 8192 data: 6 6
25039 006/300 Read block 2854 len 24576
25040 007/300 Write block 4446 len 28672 data: 6 6 6 5 5 5 6
25041 008/300 Read block 966 len 24576
25042 009/300 Write block 3461 len 28672 data: 6 6 7 6 7 8 8
25043 010/300 Read block 2244 len 4096
25044 011/300 Read block 2875 len 28672
25045 012/300 Read block 2247 len 20480
25046 013/300 Read block 2481 len 12288
25047 014/300 Read block 1187 len 16384
25048 015/300 Read block 4541 len 4096
25049 016/300 Write block 2923 len 16384 data: 9 11 7 7
25050 017/300 Read block 4448 len 8192
25051 018/300 Write block 3940 len 16384 data: 4 5 5 4
25052 019/300 Read block 3981 len 20480
25053 020/300 Write block 765 len 16384 data: 8 8 6 7
25054 021/300 Write block 4295 len 24576 data: 6 4 3 3 4 4
25055 022/300 Read block 2875 len 16384
25056 023/300 Flush
25057 024/300 Read block 3767 len 36864
25058 025/300 Read block 1666 len 40960
25059 026/300 Read block 2438 len 24576
25060 027/300 Write block 3656 len 36864 data: 6 4 2 5 6 6 6 5 6
25061 028/300 Read block 3666 len 32768
25062 029/300 Write block 2562 len 24576 data: 3 2 2 2 2 4
25063 030/300 Read block 4827 len 16384
25064 031/300 Write block 1142 len 16384 data: 5 5 4 5
25065 032/300 Read block 2337 len 16384
25066 033/300 Write block 308 len 36864 data: 6 5 6 5 4 2 2 2 2
25067 034/300 Write block 3443 len 24576 data: 6 6 5 6 4 4
25068 035/300 Read block 1961 len 32768
25069 036/300 Read block 2195 len 24576
25070 037/300 Read block 4342 len 24576
25071 038/300 Write block 2375 len 12288 data: 6 6 6
25072 039/300 Write block 1252 len 20480 data: 5 4 5 4 4
25073 040/300 Write block 1504 len 16384 data: 5 6 5 4
25074 041/300 Read block 2498 len 4096
25075 042/300 Write block 2977 len 16384 data: 6 5 5 4
25076 043/300 Read block 4485 len 16384
25077 044/300 Write block 667 len 24576 data: 8 7 6 6 6 5
25078 045/300 Flush
25079 046/300 Read block 3743 len 40960
25080 047/300 Write block 2536 len 16384 data: 4 5 5 5
25081 048/300 Read block 141 len 20480
25082 049/300 Read block 4276 len 16384
25083 050/300 Write block 1028 len 36864 data: 6 3 3 4 5 5 4 5 6
25084 051/300 Write block 2843 len 32768 data: 6 8 8 5 4 5 6 5
25085 052/300 Write block 4849 len 4096 data: 8
25086 053/300 Write block 3250 len 12288 data: 7 7 8
25087 054/300 Write block 2316 len 36864 data: 5 5 6 5 5 6 6 5 4
25088 055/300 Read block 4739 len 36864
25089 056/300 Write block 1617 len 12288 data: 5 4 3
25090 057/300 Read block 3223 len 4096
25091 058/300 Read block 1619 len 16384
25092 059/300 Read block 4985 len 12288
25093 060/300 Write block 3707 len 28672 data: 9 7 7 8 8 6 6
25094 061/300 Read block 4766 len 36864
25095 062/300 Read block 1569 len 4096
25096 063/300 Write block 876 len 12288 data: 11 9 7
25097 064/300 Read block 280 len 20480
25098 065/300 Write block 3609 len 8192 data: 4 7
25099 066/300 Read block 610 len 20480
25100 067/300 Write block 419 len 20480 data: 9 9 8 6 6
25101 068/300 Read block 979 len 16384
25102 069/300 Write block 565 len 24576 data: 4 3 3 3 5 6
25103 070/300 Write block 1580 len 28672 data: 4 2 3 3 3 4 2
25104 071/300 Read block 2613 len 32768
25105 072/300 Write block 3493 len 28672 data: 4 5 5 7 6 8 9
25106 073/300 Write block 3643 len 16384 data: 3 3 4 3
25107 074/300 Read block 612 len 4096
25108 075/300 Read block 871 len 12288
25109 076/300 Write block 2729 len 16384 data: 7 8 8 9
25110 077/300 Read block 3949 len 8192
25111 078/300 Read block 4729 len 20480
25112 079/300 Read block 2722 len 40960
25113 080/300 Write block 1489 len 32768 data: 6 6 6 5 6 6 7 8
25114 081/300 Read block 4479 len 8192
25115 082/300 Write block 4781 len 24576 data: 5 6 6 6 6 6
25116 083/300 Write block 4567 len 32768 data: 3 5 5 5 3 4 4 6
25117 084/300 Read block 4105 len 8192
25118 085/300 Write block 4609 len 36864 data: 5 3 3 5 5 6 5 6 5
25119 086/300 Write block 639 len 28672 data: 3 2 3 3 3 3 3
25120 087/300 Read block 4929 len 40960
25121 088/300 Write block 219 len 24576 data: 3 4 3 4 5 6
25122 089/300 Read block 2784 len 8192
25123 090/300 Write block 597 len 12288 data: 9 9 8
25124 091/300 Read block 706 len 24576
25125 092/300 Read block 420 len 24576
25126 093/300 Write block 4472 len 12288 data: 6 7 7
25127 094/300 Write block 2441 len 12288 data: 8 6 5
25128 095/300 Read block 792 len 28672
25129 096/300 Write block 2877 len 28672 data: 8 8 9 9 8 8 6
25130 097/300 Read block 2319 len 32768
25131 098/300 Flush
25132 099/300 Read block 4394 len 20480
25133 100/300 Flush
25134 101/300 Read block 1347 len 40960
25135 102/300 Read block 4106 len 40960
25136 103/300 Read block 4023 len 8192
25137 104/300 Read block 284 len 4096
25138 105/300 Write block 489 len 16384 data: 6 7 7 8
25139 106/300 Read block 2890 len 12288
25140 107/300 Read block 4616 len 28672
25141 108/300 Read block 67 len 4096
25142 109/300 Read block 2196 len 8192
25143 110/300 Read block 2705 len 20480
25144 111/300 Read block 1209 len 8192
25145 112/300 Read block 3863 len 20480
25146 113/300 Write block 1864 len 32768 data: 9 10 8 7 5 6 5 6
25147 114/300 Read block 4894 len 40960
25148 115/300 Flush
25149 116/300 Read block 206 len 8192
25150 117/300 Write block 3756 len 12288 data: 5 5 5
25151 118/300 Flush
25152 119/300 Write block 1353 len 4096 data: 4
25153 120/300 Read block 1398 len 40960
25154 121/300 Read block 3349 len 12288
25155 122/300 Read block 2281 len 28672
25156 123/300 Read block 3068 len 32768
25157 124/300 Read block 2591 len 40960
25158 125/300 Flush
25159 126/300 Read block 495 len 32768
25160 127/300 Write block 3763 len 36864 data: 5 4 3 3 3 4 5 5 5
25161 128/300 Read block 4799 len 28672
25162 129/300 Write block 145 len 8192 data: 8 10
25163 130/300 Read block 2146 len 28672
25164 131/300 Flush
25165 132/300 Read block 4770 len 4096
25166 133/300 Write block 4218 len 8192 data: 4 4
25167 134/300 Read block 995 len 28672
25168 135/300 Read block 2571 len 4096
25169 136/300 Write block 578 len 8192 data: 6 6
25170 137/300 Write block 2265 len 40960 data: 3 3 2 3 5 9 8 8 6 7
25171 138/300 Flush
25172 139/300 Read block 4103 len 32768
25173 140/300 Read block 522 len 24576
25174 141/300 Write block 3838 len 28672 data: 7 6 7 5 6 6 6
25175 142/300 Write block 4254 len 8192 data: 3 3
25176 143/300 Read block 2367 len 32768
25177 144/300 Read block 2351 len 8192
25178 145/300 Read block 1493 len 16384
25179 146/300 Read block 4974 len 8192
25180 147/300 Read block 1863 len 4096
25181 148/300 Write block 1649 len 16384 data: 6 5 5 7
25182 149/300 Write block 87 len 16384 data: 3 3 3 2
25183 150/300 Write block 2948 len 4096 data: 4
25184 151/300 Write block 4932 len 8192 data: 6 5
25185 152/300 Write block 2422 len 12288 data: 6 5 7
25186 153/300 Read block 1036 len 24576
25187 154/300 Flush
25188 155/300 Write block 363 len 24576 data: 5 4 3 4 5 5
25189 156/300 Write block 1247 len 4096 data: 4
25190 157/300 Read block 835 len 24576
25191 158/300 Read block 763 len 16384
25192 159/300 Read block 4459 len 16384
25193 160/300 Write block 3560 len 24576 data: 3 3 3 4 5 4
25194 161/300 Write block 2417 len 8192 data: 4 6
25195 162/300 Read block 3648 len 24576
25196 163/300 Write block 1243 len 4096 data: 5
25197 164/300 Read block 3318 len 8192
25198 165/300 Read block 2693 len 4096
25199 166/300 Flush
25200 167/300 Write block 4677 len 32768 data: 4 4 4 6 5 5 4 5
25201 168/300 Read block 4226 len 40960
25202 169/300 Write block 2391 len 16384 data: 6 5 4 5
25203 170/300 Read block 4795 len 24576
25204 171/300 Flush
25205 172/300 Read block 589 len 16384
25206 173/300 Read block 31 len 16384
25207 174/300 Write block 3351 len 24576 data: 2 4 4 3 3 4
25208 175/300 Write block 1430 len 20480 data: 10 10 11 11 10
25209 176/300 Read block 1684 len 4096
25210 177/300 Read block 2237 len 24576
25211 178/300 Write block 4032 len 16384 data: 4 4 5 8
25212 179/300 Write block 3543 len 4096 data: 6
25213 180/300 Read block 4738 len 20480
25214 181/300 Write block 1120 len 40960 data: 5 5 4 4 4 3 3 4 3 3
25215 182/300 Flush
25216 183/300 Read block 598 len 20480
25217 184/300 Write block 2372 len 36864 data: 5 5 6 7 7 7 5 5 6
25218 185/300 Read block 4894 len 28672
25219 186/300 Flush
25220 187/300 Read block 958 len 28672
25221 188/300 Read block 3048 len 36864
25222 189/300 Read block 3313 len 20480
25223 190/300 Read block 1699 len 32768
25224 191/300 Read block 3564 len 28672
25225 192/300 Read block 1455 len 32768
25226 193/300 Write block 621 len 16384 data: 5 5 5 5
25227 194/300 Read block 4666 len 8192
25228 195/300 Read block 1058 len 36864
25229 196/300 Read block 1876 len 12288
25230 197/300 Read block 3269 len 20480
25231 198/300 Read block 1696 len 24576
25232 199/300 Write block 4369 len 12288 data: 6 6 6
25233 200/300 Write block 4211 len 32768 data: 5 6 7 6 6 6 6 5
25234 201/300 Read block 3123 len 32768
25235 202/300 Read block 3668 len 36864
25236 203/300 Read block 4780 len 12288
25237 204/300 Read block 4101 len 16384
25238 205/300 Write block 1834 len 4096 data: 6
25239 206/300 Read block 1151 len 36864
25240 207/300 Write block 1656 len 40960 data: 10 9 9 6 4 5 5 6 5 5
25241 208/300 Read block 2635 len 20480
25242 209/300 Read block 2173 len 32768
25243 210/300 Read block 4462 len 8192
25244 211/300 Read block 345 len 12288
25245 212/300 Flush
25246 213/300 Write block 3433 len 32768 data: 6 6 6 7 7 6 7 7
25247 214/300 Write block 824 len 28672 data: 6 6 7 6 5 5 4
25248 215/300 Read block 299 len 8192
25249 216/300 Write block 3058 len 12288 data: 3 2 3
25250 217/300 Read block 3462 len 4096
25251 218/300 Read block 1834 len 16384
25252 219/300 Read block 4593 len 12288
25253 220/300 Flush
25254 221/300 Write block 2252 len 4096 data: 7
25255 222/300 Read block 3983 len 24576
25256 223/300 Flush
25257 224/300 Write block 4918 len 4096 data: 8
25258 225/300 Write block 164 len 8192 data: 6 8
25259 226/300 Read block 2499 len 12288
25260 227/300 Flush
25261 228/300 Write block 4598 len 40960 data: 5 6 5 5 8 8 9 7 6 6
25262 229/300 Read block 2299 len 4096
25263 230/300 Read block 3507 len 12288
25264 231/300 Write block 2970 len 32768 data: 5 5 4 6 5 7 7 7
25265 232/300 Write block 4928 len 4096 data: 6
25266 233/300 Write block 4666 len 12288 data: 2 2 2
25267 234/300 Read block 3180 len 16384
25268 235/300 Write block 4510 len 4096 data: 3
25269 236/300 Write block 2794 len 28672 data: 7 7 7 7 8 8 8
25270 237/300 Write block 1574 len 4096 data: 6
25271 238/300 Read block 1740 len 20480
25272 239/300 Write block 2161 len 32768 data: 6 8 7 6 4 4 5 5
25273 240/300 Write block 4852 len 40960 data: 8 11 9 7 7 7 6 7 6 4
25274 241/300 Write block 4194 len 16384 data: 3 3 4 4
25275 242/300 Write block 3710 len 16384 data: 9 9 7 7
25276 243/300 Read block 673 len 36864
25277 244/300 Read block 3783 len 32768
25278 245/300 Read block 3149 len 8192
25279 246/300 Write block 3643 len 40960 data: 4 4 5 4 5 7 7 7 6 6
25280 247/300 Read block 4340 len 24576
25281 248/300 Flush
25282 249/300 Write block 3157 len 12288 data: 5 7 6
25283 250/300 Read block 8 len 40960
25284 251/300 Read block 823 len 32768
25285 252/300 Read block 2375 len 4096
25286 253/300 Read block 3081 len 20480
25287 254/300 Write block 3125 len 36864 data: 9 10 8 8 6 5 5 5 6
25288 255/300 Read block 3088 len 32768
25289 256/300 Read block 786 len 40960
25290 257/300 Read block 4667 len 24576
25291 258/300 Write block 2058 len 24576 data: 6 3 3 4 4 4
25292 259/300 Write block 2561 len 8192 data: 3 4
25293 260/300 Write block 385 len 24576 data: 8 5 5 6 7 6
25294 261/300 Write block 3810 len 16384 data: 8 7 4 3
25295 262/300 Read block 4733 len 20480
25296 263/300 Write block 4293 len 8192 data: 7 8
25297 264/300 Write block 3825 len 36864 data: 3 4 3 4 4 3 4 3 4
25298 265/300 Flush
25299 266/300 Write block 3427 len 8192 data: 7 6
25300 267/300 Write block 997 len 12288 data: 6 6 8
25301 268/300 Write block 1816 len 28672 data: 7 7 6 6 5 5 3
25302 269/300 Read block 4081 len 4096
25303 270/300 Read block 185 len 8192
25304 271/300 Read block 4379 len 40960
25305 272/300 Read block 4345 len 4096
25306 273/300 Write block 4664 len 20480 data: 6 2 3 3 3
25307 274/300 Read block 535 len 28672
25308 275/300 Read block 3875 len 36864
25309 276/300 Read block 3058 len 20480
25310 277/300 Read block 4197 len 36864
25311 278/300 Read block 2404 len 16384
25312 279/300 Write block 1324 len 36864 data: 7 6 6 6 7 5 4 4 7
25313 280/300 Write block 2467 len 36864 data: 6 6 5 6 7 8 7 6 9
25314 281/300 Read block 2267 len 36864
25315 282/300 Read block 4977 len 24576
25316 283/300 Read block 2921 len 28672
25317 284/300 Read block 2351 len 12288
25318 285/300 Write block 2241 len 20480 data: 6 5 5 5 4
25319 286/300 Write block 652 len 8192 data: 5 5
25320 287/300 Write block 4924 len 28672 data: 3 3 3 5 7 6 6
25321 288/300 Write block 1575 len 16384 data: 7 5 4 3
25322 289/300 Read block 2792 len 12288
25323 290/300 Flush
25324 291/300 Read block 895 len 36864
25325 292/300 Read block 4465 len 16384
25326 293/300 Read block 1416 len 20480
25327 294/300 Read block 990 len 32768
25328 295/300 Read block 2354 len 32768
25329 296/300 Read block 4533 len 8192
25330 297/300 Write block 70 len 8192 data: 4 4
25331 298/300 Write block 2714 len 4096 data: 4
25332 299/300 Write block 2108 len 40960 data: 7 5 9 8 8 9 8 8 6 6
25333 300/300 Write block 4866 len 36864 data: 3 3 3 3 5 4 6 7 6
25334 Main task has work to do, go find it
25335 got dsc Start(2)
25336 start 2
25337 Sep 22 23:29:20.719 INFO request completed, latency_us: 364, response_code: 204, uri: /start/cid/2, method: POST, req_id: 65367506-19c4-4a41-9f8e-773b0b5fe1ab, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
25338 Replay: started 2, returned:Ok(())
25339 ----------------------------------------------------------------
25340 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25341 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
25342 8169 Acked 9168 Read 5 Done Done New false
25343 8170 Acked 9169 Read 3 Done Done New false
25344 8171 Acked 9170 Write 1 Done Done New false
25345 8172 Acked 9171 Read 9 Done Done New false
25346 8173 Acked 9172 Write 2 Done Done New false
25347 8174 Acked 9173 Read 6 Done Done New false
25348 8175 Acked 9174 Write 7 Done Done New false
25349 8176 Acked 9175 Read 6 Done Done New false
25350 8177 Acked 9176 Write 7 Done Done New false
25351 8178 Acked 9177 Read 1 Done Done New false
25352 8179 Acked 9178 Read 7 Done Done New false
25353 8180 Acked 9179 Read 5 Done Done New false
25354 8181 Acked 9180 Read 3 Done Done New false
25355 8182 Acked 9181 Read 4 Done Done New false
25356 8183 Acked 9182 Read 1 Done Done New false
25357 8184 Acked 9183 Write 4 Done Done New false
25358 8185 Acked 9184 Read 2 Done Done New false
25359 8186 Acked 9185 Write 4 Done Done New false
25360 8187 Acked 9186 Read 5 Done Done New false
25361 8188 Acked 9187 Write 4 Done Done New false
25362 8189 Acked 9188 Write 6 Done Done New false
25363 8190 Acked 9189 Read 4 Done Done New false
25364 8191 Acked 9190 Flush 0 Done Done New false
25365 8192 Acked 9191 Read 9 Done Done New false
25366 8193 Acked 9192 Read 10 Done Done New false
25367 8194 Acked 9193 Read 6 Done Done New false
25368 8195 Acked 9194 Write 9 Done Done New false
25369 8196 Acked 9195 Read 8 Done Done New false
25370 8197 Acked 9196 Write 6 Done Done New false
25371 8198 Acked 9197 Read 4 Done Done New false
25372 8199 Acked 9198 Write 4 Done Done New false
25373 8200 Acked 9199 Read 4 Done Done New false
25374 8201 Acked 9200 Write 9 Done Done New false
25375 8202 Acked 9201 Write 6 Done Done New false
25376 8203 Acked 9202 Read 8 Done Done New false
25377 8204 Acked 9203 Read 6 Done Done New false
25378 8205 Acked 9204 Read 6 Done Done New false
25379 8206 Acked 9205 Write 3 Done Done New false
25380 8207 Acked 9206 Write 5 Done Done New false
25381 8208 Acked 9207 Write 4 Done Done New false
25382 8209 Acked 9208 Read 1 Done Done New false
25383 8210 Acked 9209 Write 4 Done Done New false
25384 8211 Acked 9210 Read 4 Done Done New false
25385 8212 Acked 9211 Write 6 Done Done New false
25386 8213 Acked 9212 Flush 0 Done Done New false
25387 8214 Acked 9213 Read 10 Done Done New false
25388 8215 Acked 9214 Write 4 Done Done New false
25389 8216 Acked 9215 Read 5 Done Done New false
25390 8217 Acked 9216 Read 4 Done Done New false
25391 8218 Acked 9217 Write 9 Done Done New false
25392 8219 Acked 9218 Write 8 Done Done New false
25393 8220 Acked 9219 Write 1 Done Done New false
25394 8221 Acked 9220 Write 3 Done Done New false
25395 8222 Acked 9221 Write 9 Done Done New false
25396 8223 Acked 9222 Read 9 Done Done New false
25397 8224 Acked 9223 Write 3 Done Done New false
25398 8225 Acked 9224 Read 1 Done Done New false
25399 8226 Acked 9225 Read 4 Done Done New false
25400 8227 Acked 9226 Read 3 Done Done New false
25401 8228 Acked 9227 Write 7 Done Done New false
25402 8229 Acked 9228 Read 9 Done Done New false
25403 8230 Acked 9229 Read 1 Done Done New false
25404 8231 Acked 9230 Write 3 Done Done New false
25405 8232 Acked 9231 Read 5 Done Done New false
25406 8233 Acked 9232 Write 2 Done Done New false
25407 8234 Acked 9233 Read 5 Done Done New false
25408 8235 Acked 9234 Write 5 Done Done New false
25409 8236 Acked 9235 Read 4 Done Done New false
25410 8237 Acked 9236 Write 6 Done Done New false
25411 8238 Acked 9237 Write 7 Done Done New false
25412 8239 Acked 9238 Read 8 Done Done New false
25413 8240 Acked 9239 Write 7 Done Done New false
25414 8241 Acked 9240 Write 4 Done Done New false
25415 8242 Acked 9241 Read 1 Done Done New false
25416 8243 Acked 9242 Read 3 Done Done New false
25417 8244 Acked 9243 Write 4 Done Done New false
25418 8245 Acked 9244 Read 2 Done Done New false
25419 8246 Acked 9245 Read 5 Done Done New false
25420 8247 Acked 9246 Read 10 Done Done New false
25421 8248 Acked 9247 Write 8 Done Done New false
25422 8249 Acked 9248 Read 2 Done Done New false
25423 8250 Acked 9249 Write 6 Done Done New false
25424 8251 Acked 9250 Write 8 Done Done New false
25425 8252 Acked 9251 Read 2 Done Done New false
25426 8253 Acked 9252 Write 9 Done Done New false
25427 8254 Acked 9253 Write 7 Done Done New false
25428 8255 Acked 9254 Read 10 Done Done New false
25429 8256 Acked 9255 Write 6 Done Done New false
25430 8257 Acked 9256 Read 2 Done Done New false
25431 8258 Acked 9257 Write 3 Done Done New false
25432 8259 Acked 9258 Read 6 Done Done New false
25433 8260 Acked 9259 Read 6 Done Done New false
25434 8261 Acked 9260 Write 3 Done Done New false
25435 8262 Acked 9261 Write 3 Done Done New false
25436 8263 Acked 9262 Read 7 Done Done New false
25437 8264 Acked 9263 Write 7 Done Done New false
25438 8265 Acked 9264 Read 8 Done Done New false
25439 8266 Acked 9265 Flush 0 Done Done New false
25440 8267 Acked 9266 Read 5 Done Done New false
25441 8268 Acked 9267 Flush 0 Done Done New false
25442 8269 Acked 9268 Read 10 Done Done New false
25443 8270 Acked 9269 Read 10 Done Done New false
25444 8271 Acked 9270 Read 2 Done Done New false
25445 8272 Acked 9271 Read 1 Done Done New false
25446 8273 Acked 9272 Write 4 Done Done New false
25447 8274 Acked 9273 Read 3 Done Done New false
25448 8275 Acked 9274 Read 7 Done Done New false
25449 8276 Acked 9275 Read 1 Done Done New false
25450 8277 Acked 9276 Read 2 Done Done New false
25451 8278 Acked 9277 Read 5 Done Done New false
25452 8279 Acked 9278 Read 2 Done Done New false
25453 8280 Acked 9279 Read 5 Done Done New false
25454 8281 Acked 9280 Write 8 Done Done New false
25455 8282 Acked 9281 Read 10 Done Done New false
25456 8283 Acked 9282 Flush 0 Done Done New false
25457 8284 Acked 9283 Read 2 Done Done New false
25458 8285 Acked 9284 Write 3 Done Done New false
25459 8286 Acked 9285 Flush 0 Done Done New false
25460 8287 Acked 9286 Write 1 Done Done New false
25461 8288 Acked 9287 Read 10 Done Done New false
25462 8289 Acked 9288 Read 3 Done Done New false
25463 8290 Acked 9289 Read 7 Done Done New false
25464 8291 Acked 9290 Read 8 Done Done New false
25465 8292 Acked 9291 Read 10 Done Done New false
25466 8293 Acked 9292 Flush 0 Done Done New false
25467 8294 Acked 9293 Read 8 Done Done New false
25468 8295 Acked 9294 Write 9 Done Done New false
25469 8296 Acked 9295 Read 7 Done Done New false
25470 8297 Acked 9296 Write 2 Done Done New false
25471 8298 Acked 9297 Read 7 Done Done New false
25472 8299 Acked 9298 Flush 0 Done Done New false
25473 8300 Acked 9299 Read 1 Done Done New false
25474 8301 Acked 9300 Write 2 Done Done New false
25475 8302 Acked 9301 Read 7 Done Done New false
25476 8303 Acked 9302 Read 1 Done Done New false
25477 8304 Acked 9303 Write 2 Done Done New false
25478 8305 Acked 9304 Write 10 Done Done New false
25479 8306 Acked 9305 Flush 0 Done Done New false
25480 8307 Acked 9306 Read 8 Done Done New false
25481 8308 Acked 9307 Read 6 Done Done New false
25482 8309 Acked 9308 Write 7 Done Done New false
25483 8310 Acked 9309 Write 2 Done Done New false
25484 8311 Acked 9310 Read 8 Done Done New false
25485 8312 Acked 9311 Read 2 Done Done New false
25486 8313 Acked 9312 Read 4 Done Done New false
25487 8314 Acked 9313 Read 2 Done Done New false
25488 8315 Acked 9314 Read 1 Done Done New false
25489 8316 Acked 9315 Write 4 Done Done New false
25490 8317 Acked 9316 Write 4 Done Done New false
25491 8318 Acked 9317 Write 1 Done Done New false
25492 8319 Acked 9318 Write 2 Done Done New false
25493 8320 Acked 9319 Write 3 Done Done New false
25494 8321 Acked 9320 Read 6 Done Done New false
25495 8322 Acked 9321 Flush 0 Done Done New false
25496 8323 Acked 9322 Write 6 Done Done New false
25497 8324 Acked 9323 Write 1 Done Done New false
25498 8325 Acked 9324 Read 6 Done Done New false
25499 8326 Acked 9325 Read 4 Done Done New false
25500 8327 Acked 9326 Read 4 Done Done New false
25501 8328 Acked 9327 Write 6 Done Done New false
25502 8329 Acked 9328 Write 2 Done Done New false
25503 8330 Acked 9329 Read 6 Done Done New false
25504 8331 Acked 9330 Write 1 Done Done New false
25505 8332 Acked 9331 Read 2 Done Done New false
25506 8333 Acked 9332 Read 1 Done Done New false
25507 8334 Acked 9333 Flush 0 Done Done New false
25508 8335 Acked 9334 Write 8 Done Done New false
25509 8336 Acked 9335 Read 10 Done Done New false
25510 8337 Acked 9336 Write 4 Done Done New false
25511 8338 Acked 9337 Read 6 Done Done New false
25512 8339 Acked 9338 Flush 0 Done Done New false
25513 8340 Acked 9339 Read 4 Done Done New false
25514 8341 Acked 9340 Read 4 Done Done New false
25515 8342 Acked 9341 Write 6 Done Done New false
25516 8343 Acked 9342 Write 5 Done Done New false
25517 8344 Acked 9343 Read 1 Done Done New false
25518 8345 Acked 9344 Read 6 Done Done New false
25519 8346 Acked 9345 Write 4 Done Done New false
25520 8347 Acked 9346 Write 1 Done Done New false
25521 8348 Acked 9347 Read 5 Done Done New false
25522 8349 Acked 9348 Write 10 Done Done New false
25523 8350 Acked 9349 Flush 0 Done Done New false
25524 8351 Acked 9350 Read 5 Done Done New false
25525 8352 Acked 9351 Write 9 Done Done New false
25526 8353 Acked 9352 Read 7 Done Done New false
25527 8354 Acked 9353 Flush 0 Done Done New false
25528 8355 Acked 9354 Read 7 Done Done New false
25529 8356 Acked 9355 Read 9 Done Done New false
25530 8357 Acked 9356 Read 5 Done Done New false
25531 8358 Acked 9357 Read 8 Done Done New false
25532 8359 Acked 9358 Read 7 Done Done New false
25533 8360 Acked 9359 Read 8 Done Done New false
25534 8361 Acked 9360 Write 4 Done Done New false
25535 8362 Acked 9361 Read 2 Done Done New false
25536 8363 Acked 9362 Read 9 Done Done New false
25537 8364 Acked 9363 Read 3 Done Done New false
25538 8365 Acked 9364 Read 5 Done Done New false
25539 8366 Acked 9365 Read 6 Done Done New false
25540 8367 Acked 9366 Write 3 Done Done New false
25541 8368 Acked 9367 Write 8 Done Done New false
25542 8369 Acked 9368 Read 8 Done Done New false
25543 8370 Acked 9369 Read 9 Done Done New false
25544 8371 Acked 9370 Read 3 Done Done New false
25545 8372 Acked 9371 Read 4 Done Done New false
25546 8373 Acked 9372 Write 1 Done Done New false
25547 8374 Acked 9373 Read 9 Done Done New false
25548 8375 Acked 9374 Write 10 Done Done New false
25549 8376 Acked 9375 Read 5 Done Done New false
25550 8377 Acked 9376 Read 8 Done Done New false
25551 8378 Acked 9377 Read 2 Done Done New false
25552 8379 Acked 9378 Read 3 Done Done New false
25553 8380 Acked 9379 Flush 0 Done Done New false
25554 8381 Acked 9380 Write 8 Done Done New false
25555 8382 Acked 9381 Write 7 Done Done New false
25556 8383 Acked 9382 Read 2 Done Done New false
25557 8384 Acked 9383 Write 3 Done Done New false
25558 8385 Acked 9384 Read 1 Done Done New false
25559 8386 Acked 9385 Read 4 Done Done New false
25560 8387 Acked 9386 Read 3 Done Done New false
25561 8388 Acked 9387 Flush 0 Done Done New false
25562 8389 Acked 9388 Write 1 Done Done New false
25563 8390 Acked 9389 Read 6 Done Done New false
25564 8391 Acked 9390 Flush 0 Done Done New false
25565 8392 Acked 9391 Write 1 Done Done New false
25566 8393 Acked 9392 Write 2 Done Done New false
25567 8394 Acked 9393 Read 3 Done Done New false
25568 8395 Acked 9394 Flush 0 Done Done New false
25569 8396 Acked 9395 Write 10 Done Done New false
25570 8397 Acked 9396 Read 1 Done Done New false
25571 8398 Acked 9397 Read 3 Done Done New false
25572 8399 Acked 9398 Write 8 Done Done New false
25573 8400 Acked 9399 Write 1 Done Done New false
25574 8401 Acked 9400 Write 3 Done Done New false
25575 8402 Acked 9401 Read 4 Done Done New false
25576 8403 Acked 9402 Write 1 Done Done New false
25577 8404 Acked 9403 Write 7 Done Done New false
25578 8405 Acked 9404 Write 1 Done Done New false
25579 8406 Acked 9405 Read 5 Done Done New false
25580 8407 Acked 9406 Write 8 Done Done New false
25581 8408 Acked 9407 Write 10 Done Done New false
25582 8409 Acked 9408 Write 4 Done Done New false
25583 8410 Acked 9409 Write 4 Done Done New false
25584 8411 Acked 9410 Read 9 Done Done New false
25585 8412 Acked 9411 Read 8 Done Done New false
25586 8413 Acked 9412 Read 2 Done Done New false
25587 8414 Acked 9413 Write 10 Done Done New false
25588 8415 Acked 9414 Read 6 Done Done New false
25589 8416 Acked 9415 Flush 0 Done Done New false
25590 8417 Acked 9416 Write 3 Done Done New false
25591 8418 Acked 9417 Read 10 Done Done New false
25592 8419 Acked 9418 Read 8 Done Done New false
25593 8420 Acked 9419 Read 1 Done Done New false
25594 8421 Acked 9420 Read 5 Done Done New false
25595 8422 Acked 9421 Write 9 Done Done New false
25596 8423 Acked 9422 Read 8 Done Done New false
25597 8424 Acked 9423 Read 10 Done Done New false
25598 8425 Acked 9424 Read 6 Done Done New false
25599 8426 Acked 9425 Write 6 Done Done New false
25600 8427 Acked 9426 Write 2 Done Done New false
25601 8428 Acked 9427 Write 6 Done Done New false
25602 8429 Acked 9428 Write 4 Done Done New false
25603 8430 Acked 9429 Read 5 Done Done New false
25604 8431 Acked 9430 Write 2 Done Done New false
25605 8432 Acked 9431 Write 9 Done Done New false
25606 8433 Acked 9432 Flush 0 Done Done New false
25607 8434 Acked 9433 Write 2 Done Done New false
25608 8435 Acked 9434 Write 3 Done Done New false
25609 8436 Acked 9435 Write 7 Done Done New false
25610 8437 Acked 9436 Read 1 Done Done New false
25611 8438 Acked 9437 Read 2 Done Done New false
25612 8439 Acked 9438 Read 10 Done Done New false
25613 8440 Acked 9439 Read 1 Done Done New false
25614 8441 Acked 9440 Write 5 Done Done New false
25615 8442 Acked 9441 Read 7 Done Done New false
25616 8443 Acked 9442 Read 9 Done Done New false
25617 8444 Acked 9443 Read 5 Done Done New false
25618 8445 Acked 9444 Read 9 Done Done New false
25619 8446 Acked 9445 Read 4 Done Done New false
25620 8447 Acked 9446 Write 9 Done Done New false
25621 8448 Acked 9447 Write 9 Done Done New false
25622 8449 Acked 9448 Read 9 Done Done New false
25623 8450 Acked 9449 Read 6 Done Done New false
25624 8451 Acked 9450 Read 7 Done Done New false
25625 8452 Acked 9451 Read 3 Done Done New false
25626 8453 Acked 9452 Write 5 Done Done New false
25627 8454 Acked 9453 Write 2 Done Done New false
25628 8455 Acked 9454 Write 7 Done Done New false
25629 8456 Acked 9455 Write 4 Done Done New false
25630 8457 Acked 9456 Read 3 Done Done New false
25631 8458 Acked 9457 Flush 0 Done Done New false
25632 8459 Acked 9458 Read 9 Done Done New false
25633 8460 Acked 9459 Flush 0 Done Done New false
25634 8461 Acked 9460 Read 4 Done Done New false
25635 8462 Acked 9461 Read 5 Done Done New false
25636 8463 Acked 9462 Read 8 Done Done New false
25637 8464 Acked 9463 Read 8 Done Done New false
25638 8465 Acked 9464 Read 2 Done Done New false
25639 8466 Acked 9465 Write 2 Done Done New false
25640 8467 Acked 9466 Write 1 Done Done New false
25641 8468 Acked 9467 Write 10 Sent Sent New false
25642 8469 Acked 9468 Write 9 Sent Sent New false
25643 STATES DS:0 DS:1 DS:2 TOTAL
25644 New 0 0 301 301
25645 Sent 2 2 0 4
25646 Done 299 299 0 598
25647 Skipped 0 0 0 0
25648 Error 0 0 0 0
25649 Last Flush: 9459 9459 9167
25650 Downstairs last five completed: 9167 9166 9165 9164 9163 9162
25651 Upstairs last five completed: 8469 8468 8467 8466 8465 8464
25652 CLIENT: Up:0 ds:301 act:2
25653 ----------------------------------------------------------------
25654 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25655 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
25656 8169 Acked 9168 Read 5 Done Done New false
25657 8170 Acked 9169 Read 3 Done Done New false
25658 8171 Acked 9170 Write 1 Done Done New false
25659 8172 Acked 9171 Read 9 Done Done New false
25660 8173 Acked 9172 Write 2 Done Done New false
25661 8174 Acked 9173 Read 6 Done Done New false
25662 8175 Acked 9174 Write 7 Done Done New false
25663 8176 Acked 9175 Read 6 Done Done New false
25664 8177 Acked 9176 Write 7 Done Done New false
25665 8178 Acked 9177 Read 1 Done Done New false
25666 8179 Acked 9178 Read 7 Done Done New false
25667 8180 Acked 9179 Read 5 Done Done New false
25668 8181 Acked 9180 Read 3 Done Done New false
25669 8182 Acked 9181 Read 4 Done Done New false
25670 8183 Acked 9182 Read 1 Done Done New false
25671 8184 Acked 9183 Write 4 Done Done New false
25672 8185 Acked 9184 Read 2 Done Done New false
25673 8186 Acked 9185 Write 4 Done Done New false
25674 8187 Acked 9186 Read 5 Done Done New false
25675 8188 Acked 9187 Write 4 Done Done New false
25676 8189 Acked 9188 Write 6 Done Done New false
25677 8190 Acked 9189 Read 4 Done Done New false
25678 8191 Acked 9190 Flush 0 Done Done New false
25679 8192 Acked 9191 Read 9 Done Done New false
25680 8193 Acked 9192 Read 10 Done Done New false
25681 8194 Acked 9193 Read 6 Done Done New false
25682 8195 Acked 9194 Write 9 Done Done New false
25683 8196 Acked 9195 Read 8 Done Done New false
25684 8197 Acked 9196 Write 6 Done Done New false
25685 8198 Acked 9197 Read 4 Done Done New false
25686 8199 Acked 9198 Write 4 Done Done New false
25687 8200 Acked 9199 Read 4 Done Done New false
25688 8201 Acked 9200 Write 9 Done Done New false
25689 8202 Acked 9201 Write 6 Done Done New false
25690 8203 Acked 9202 Read 8 Done Done New false
25691 8204 Acked 9203 Read 6 Done Done New false
25692 8205 Acked 9204 Read 6 Done Done New false
25693 8206 Acked 9205 Write 3 Done Done New false
25694 8207 Acked 9206 Write 5 Done Done New false
25695 8208 Acked 9207 Write 4 Done Done New false
25696 8209 Acked 9208 Read 1 Done Done New false
25697 8210 Acked 9209 Write 4 Done Done New false
25698 8211 Acked 9210 Read 4 Done Done New false
25699 8212 Acked 9211 Write 6 Done Done New false
25700 8213 Acked 9212 Flush 0 Done Done New false
25701 8214 Acked 9213 Read 10 Done Done New false
25702 8215 Acked 9214 Write 4 Done Done New false
25703 8216 Acked 9215 Read 5 Done Done New false
25704 8217 Acked 9216 Read 4 Done Done New false
25705 8218 Acked 9217 Write 9 Done Done New false
25706 8219 Acked 9218 Write 8 Done Done New false
25707 8220 Acked 9219 Write 1 Done Done New false
25708 8221 Acked 9220 Write 3 Done Done New false
25709 8222 Acked 9221 Write 9 Done Done New false
25710 8223 Acked 9222 Read 9 Done Done New false
25711 8224 Acked 9223 Write 3 Done Done New false
25712 8225 Acked 9224 Read 1 Done Done New false
25713 8226 Acked 9225 Read 4 Done Done New false
25714 8227 Acked 9226 Read 3 Done Done New false
25715 8228 Acked 9227 Write 7 Done Done New false
25716 8229 Acked 9228 Read 9 Done Done New false
25717 8230 Acked 9229 Read 1 Done Done New false
25718 8231 Acked 9230 Write 3 Done Done New false
25719 8232 Acked 9231 Read 5 Done Done New false
25720 8233 Acked 9232 Write 2 Done Done New false
25721 8234 Acked 9233 Read 5 Done Done New false
25722 8235 Acked 9234 Write 5 Done Done New false
25723 8236 Acked 9235 Read 4 Done Done New false
25724 8237 Acked 9236 Write 6 Done Done New false
25725 8238 Acked 9237 Write 7 Done Done New false
25726 8239 Acked 9238 Read 8 Done Done New false
25727 8240 Acked 9239 Write 7 Done Done New false
25728 8241 Acked 9240 Write 4 Done Done New false
25729 8242 Acked 9241 Read 1 Done Done New false
25730 8243 Acked 9242 Read 3 Done Done New false
25731 8244 Acked 9243 Write 4 Done Done New false
25732 8245 Acked 9244 Read 2 Done Done New false
25733 8246 Acked 9245 Read 5 Done Done New false
25734 8247 Acked 9246 Read 10 Done Done New false
25735 8248 Acked 9247 Write 8 Done Done New false
25736 8249 Acked 9248 Read 2 Done Done New false
25737 8250 Acked 9249 Write 6 Done Done New false
25738 8251 Acked 9250 Write 8 Done Done New false
25739 8252 Acked 9251 Read 2 Done Done New false
25740 8253 Acked 9252 Write 9 Done Done New false
25741 8254 Acked 9253 Write 7 Done Done New false
25742 8255 Acked 9254 Read 10 Done Done New false
25743 8256 Acked 9255 Write 6 Done Done New false
25744 8257 Acked 9256 Read 2 Done Done New false
25745 8258 Acked 9257 Write 3 Done Done New false
25746 8259 Acked 9258 Read 6 Done Done New false
25747 8260 Acked 9259 Read 6 Done Done New false
25748 8261 Acked 9260 Write 3 Done Done New false
25749 8262 Acked 9261 Write 3 Done Done New false
25750 8263 Acked 9262 Read 7 Done Done New false
25751 8264 Acked 9263 Write 7 Done Done New false
25752 8265 Acked 9264 Read 8 Done Done New false
25753 8266 Acked 9265 Flush 0 Done Done New false
25754 8267 Acked 9266 Read 5 Done Done New false
25755 8268 Acked 9267 Flush 0 Done Done New false
25756 8269 Acked 9268 Read 10 Done Done New false
25757 8270 Acked 9269 Read 10 Done Done New false
25758 8271 Acked 9270 Read 2 Done Done New false
25759 8272 Acked 9271 Read 1 Done Done New false
25760 8273 Acked 9272 Write 4 Done Done New false
25761 8274 Acked 9273 Read 3 Done Done New false
25762 8275 Acked 9274 Read 7 Done Done New false
25763 8276 Acked 9275 Read 1 Done Done New false
25764 8277 Acked 9276 Read 2 Done Done New false
25765 8278 Acked 9277 Read 5 Done Done New false
25766 8279 Acked 9278 Read 2 Done Done New false
25767 8280 Acked 9279 Read 5 Done Done New false
25768 8281 Acked 9280 Write 8 Done Done New false
25769 8282 Acked 9281 Read 10 Done Done New false
25770 8283 Acked 9282 Flush 0 Done Done New false
25771 8284 Acked 9283 Read 2 Done Done New false
25772 8285 Acked 9284 Write 3 Done Done New false
25773 8286 Acked 9285 Flush 0 Done Done New false
25774 8287 Acked 9286 Write 1 Done Done New false
25775 8288 Acked 9287 Read 10 Done Done New false
25776 8289 Acked 9288 Read 3 Done Done New false
25777 8290 Acked 9289 Read 7 Done Done New false
25778 8291 Acked 9290 Read 8 Done Done New false
25779 8292 Acked 9291 Read 10 Done Done New false
25780 8293 Acked 9292 Flush 0 Done Done New false
25781 8294 Acked 9293 Read 8 Done Done New false
25782 8295 Acked 9294 Write 9 Done Done New false
25783 8296 Acked 9295 Read 7 Done Done New false
25784 8297 Acked 9296 Write 2 Done Done New false
25785 8298 Acked 9297 Read 7 Done Done New false
25786 8299 Acked 9298 Flush 0 Done Done New false
25787 8300 Acked 9299 Read 1 Done Done New false
25788 8301 Acked 9300 Write 2 Done Done New false
25789 8302 Acked 9301 Read 7 Done Done New false
25790 8303 Acked 9302 Read 1 Done Done New false
25791 8304 Acked 9303 Write 2 Done Done New false
25792 8305 Acked 9304 Write 10 Done Done New false
25793 8306 Acked 9305 Flush 0 Done Done New false
25794 8307 Acked 9306 Read 8 Done Done New false
25795 8308 Acked 9307 Read 6 Done Done New false
25796 8309 Acked 9308 Write 7 Done Done New false
25797 8310 Acked 9309 Write 2 Done Done New false
25798 8311 Acked 9310 Read 8 Done Done New false
25799 8312 Acked 9311 Read 2 Done Done New false
25800 8313 Acked 9312 Read 4 Done Done New false
25801 8314 Acked 9313 Read 2 Done Done New false
25802 8315 Acked 9314 Read 1 Done Done New false
25803 8316 Acked 9315 Write 4 Done Done New false
25804 8317 Acked 9316 Write 4 Done Done New false
25805 8318 Acked 9317 Write 1 Done Done New false
25806 8319 Acked 9318 Write 2 Done Done New false
25807 8320 Acked 9319 Write 3 Done Done New false
25808 8321 Acked 9320 Read 6 Done Done New false
25809 8322 Acked 9321 Flush 0 Done Done New false
25810 8323 Acked 9322 Write 6 Done Done New false
25811 8324 Acked 9323 Write 1 Done Done New false
25812 8325 Acked 9324 Read 6 Done Done New false
25813 8326 Acked 9325 Read 4 Done Done New false
25814 8327 Acked 9326 Read 4 Done Done New false
25815 8328 Acked 9327 Write 6 Done Done New false
25816 8329 Acked 9328 Write 2 Done Done New false
25817 8330 Acked 9329 Read 6 Done Done New false
25818 8331 Acked 9330 Write 1 Done Done New false
25819 8332 Acked 9331 Read 2 Done Done New false
25820 8333 Acked 9332 Read 1 Done Done New false
25821 8334 Acked 9333 Flush 0 Done Done New false
25822 8335 Acked 9334 Write 8 Done Done New false
25823 8336 Acked 9335 Read 10 Done Done New false
25824 8337 Acked 9336 Write 4 Done Done New false
25825 8338 Acked 9337 Read 6 Done Done New false
25826 8339 Acked 9338 Flush 0 Done Done New false
25827 8340 Acked 9339 Read 4 Done Done New false
25828 8341 Acked 9340 Read 4 Done Done New false
25829 8342 Acked 9341 Write 6 Done Done New false
25830 8343 Acked 9342 Write 5 Done Done New false
25831 8344 Acked 9343 Read 1 Done Done New false
25832 8345 Acked 9344 Read 6 Done Done New false
25833 8346 Acked 9345 Write 4 Done Done New false
25834 8347 Acked 9346 Write 1 Done Done New false
25835 8348 Acked 9347 Read 5 Done Done New false
25836 8349 Acked 9348 Write 10 Done Done New false
25837 8350 Acked 9349 Flush 0 Done Done New false
25838 8351 Acked 9350 Read 5 Done Done New false
25839 8352 Acked 9351 Write 9 Done Done New false
25840 8353 Acked 9352 Read 7 Done Done New false
25841 8354 Acked 9353 Flush 0 Done Done New false
25842 8355 Acked 9354 Read 7 Done Done New false
25843 8356 Acked 9355 Read 9 Done Done New false
25844 8357 Acked 9356 Read 5 Done Done New false
25845 8358 Acked 9357 Read 8 Done Done New false
25846 8359 Acked 9358 Read 7 Done Done New false
25847 8360 Acked 9359 Read 8 Done Done New false
25848 8361 Acked 9360 Write 4 Done Done New false
25849 8362 Acked 9361 Read 2 Done Done New false
25850 8363 Acked 9362 Read 9 Done Done New false
25851 8364 Acked 9363 Read 3 Done Done New false
25852 8365 Acked 9364 Read 5 Done Done New false
25853 8366 Acked 9365 Read 6 Done Done New false
25854 8367 Acked 9366 Write 3 Done Done New false
25855 8368 Acked 9367 Write 8 Done Done New false
25856 8369 Acked 9368 Read 8 Done Done New false
25857 8370 Acked 9369 Read 9 Done Done New false
25858 8371 Acked 9370 Read 3 Done Done New false
25859 8372 Acked 9371 Read 4 Done Done New false
25860 8373 Acked 9372 Write 1 Done Done New false
25861 8374 Acked 9373 Read 9 Done Done New false
25862 8375 Acked 9374 Write 10 Done Done New false
25863 8376 Acked 9375 Read 5 Done Done New false
25864 8377 Acked 9376 Read 8 Done Done New false
25865 8378 Acked 9377 Read 2 Done Done New false
25866 8379 Acked 9378 Read 3 Done Done New false
25867 8380 Acked 9379 Flush 0 Done Done New false
25868 8381 Acked 9380 Write 8 Done Done New false
25869 8382 Acked 9381 Write 7 Done Done New false
25870 8383 Acked 9382 Read 2 Done Done New false
25871 8384 Acked 9383 Write 3 Done Done New false
25872 8385 Acked 9384 Read 1 Done Done New false
25873 8386 Acked 9385 Read 4 Done Done New false
25874 8387 Acked 9386 Read 3 Done Done New false
25875 8388 Acked 9387 Flush 0 Done Done New false
25876 8389 Acked 9388 Write 1 Done Done New false
25877 8390 Acked 9389 Read 6 Done Done New false
25878 8391 Acked 9390 Flush 0 Done Done New false
25879 8392 Acked 9391 Write 1 Done Done New false
25880 8393 Acked 9392 Write 2 Done Done New false
25881 8394 Acked 9393 Read 3 Done Done New false
25882 8395 Acked 9394 Flush 0 Done Done New false
25883 8396 Acked 9395 Write 10 Done Done New false
25884 8397 Acked 9396 Read 1 Done Done New false
25885 8398 Acked 9397 Read 3 Done Done New false
25886 8399 Acked 9398 Write 8 Done Done New false
25887 8400 Acked 9399 Write 1 Done Done New false
25888 8401 Acked 9400 Write 3 Done Done New false
25889 8402 Acked 9401 Read 4 Done Done New false
25890 8403 Acked 9402 Write 1 Done Done New false
25891 8404 Acked 9403 Write 7 Done Done New false
25892 8405 Acked 9404 Write 1 Done Done New false
25893 8406 Acked 9405 Read 5 Done Done New false
25894 8407 Acked 9406 Write 8 Done Done New false
25895 8408 Acked 9407 Write 10 Done Done New false
25896 8409 Acked 9408 Write 4 Done Done New false
25897 8410 Acked 9409 Write 4 Done Done New false
25898 8411 Acked 9410 Read 9 Done Done New false
25899 8412 Acked 9411 Read 8 Done Done New false
25900 8413 Acked 9412 Read 2 Done Done New false
25901 8414 Acked 9413 Write 10 Done Done New false
25902 8415 Acked 9414 Read 6 Done Done New false
25903 8416 Acked 9415 Flush 0 Done Done New false
25904 8417 Acked 9416 Write 3 Done Done New false
25905 8418 Acked 9417 Read 10 Done Done New false
25906 8419 Acked 9418 Read 8 Done Done New false
25907 8420 Acked 9419 Read 1 Done Done New false
25908 8421 Acked 9420 Read 5 Done Done New false
25909 8422 Acked 9421 Write 9 Done Done New false
25910 8423 Acked 9422 Read 8 Done Done New false
25911 8424 Acked 9423 Read 10 Done Done New false
25912 8425 Acked 9424 Read 6 Done Done New false
25913 8426 Acked 9425 Write 6 Done Done New false
25914 8427 Acked 9426 Write 2 Done Done New false
25915 8428 Acked 9427 Write 6 Done Done New false
25916 8429 Acked 9428 Write 4 Done Done New false
25917 8430 Acked 9429 Read 5 Done Done New false
25918 8431 Acked 9430 Write 2 Done Done New false
25919 8432 Acked 9431 Write 9 Done Done New false
25920 8433 Acked 9432 Flush 0 Done Done New false
25921 8434 Acked 9433 Write 2 Done Done New false
25922 8435 Acked 9434 Write 3 Done Done New false
25923 8436 Acked 9435 Write 7 Done Done New false
25924 8437 Acked 9436 Read 1 Done Done New false
25925 8438 Acked 9437 Read 2 Done Done New false
25926 8439 Acked 9438 Read 10 Done Done New false
25927 8440 Acked 9439 Read 1 Done Done New false
25928 8441 Acked 9440 Write 5 Done Done New false
25929 8442 Acked 9441 Read 7 Done Done New false
25930 8443 Acked 9442 Read 9 Done Done New false
25931 8444 Acked 9443 Read 5 Done Done New false
25932 8445 Acked 9444 Read 9 Done Done New false
25933 8446 Acked 9445 Read 4 Done Done New false
25934 8447 Acked 9446 Write 9 Done Done New false
25935 8448 Acked 9447 Write 9 Done Done New false
25936 8449 Acked 9448 Read 9 Done Done New false
25937 8450 Acked 9449 Read 6 Done Done New false
25938 8451 Acked 9450 Read 7 Done Done New false
25939 8452 Acked 9451 Read 3 Done Done New false
25940 8453 Acked 9452 Write 5 Done Done New false
25941 8454 Acked 9453 Write 2 Done Done New false
25942 8455 Acked 9454 Write 7 Done Done New false
25943 8456 Acked 9455 Write 4 Done Done New false
25944 8457 Acked 9456 Read 3 Done Done New false
25945 8458 Acked 9457 Flush 0 Done Done New false
25946 8459 Acked 9458 Read 9 Done Done New false
25947 8460 Acked 9459 Flush 0 Done Done New false
25948 8461 Acked 9460 Read 4 Done Done New false
25949 8462 Acked 9461 Read 5 Done Done New false
25950 8463 Acked 9462 Read 8 Done Done New false
25951 8464 Acked 9463 Read 8 Done Done New false
25952 8465 Acked 9464 Read 2 Done Done New false
25953 8466 Acked 9465 Write 2 Done Done New false
25954 8467 Acked 9466 Write 1 Done Done New false
25955 8468 Acked 9467 Write 10 Done Done New false
25956 8469 Acked 9468 Write 9 Done Done New false
25957 8470 Acked 9469 Flush 0 Done Done New false
25958 STATES DS:0 DS:1 DS:2 TOTAL
25959 New 0 0 302 302
25960 Sent 0 0 0 0
25961 Done 302 302 0 604
25962 Skipped 0 0 0 0
25963 Error 0 0 0 0
25964 Last Flush: 9469 9469 9167
25965 Downstairs last five completed: 9167 9166 9165 9164 9163 9162
25966 Upstairs last five completed: 8470 8469 8468 8467 8466 8465
25967 CLIENT: Up:0 ds:302 act:2
259682023-09-22T23:29:24.814ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
25969 [8830] got start action
25970 [8830] I am going to restart
25971 Starting downstairs at port 8830
25972 Make output file at "/tmp/dsc/downstairs-8830.txt"
25973 [8830][2] reports Starting
25974 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1201)
25975 [8830][2] reports Running
259762023-09-22T23:29:26.816ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 2
259772023-09-22T23:29:26.816ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state Offline
259782023-09-22T23:29:26.816ZINFOcrucible: [2] client re-new 302 jobs since flush 9167 = downstairs
259792023-09-22T23:29:26.817ZINFOcrucible: [2] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
259802023-09-22T23:29:26.818ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
259812023-09-22T23:29:26.818ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
259822023-09-22T23:29:26.818ZINFOcrucible: Returning client:2 UUID:12345678-0000-0000-0000-000000008830 matches
259832023-09-22T23:29:26.818ZINFOcrucible: [2] send last flush ID to this DS: 9167
259842023-09-22T23:29:26.818ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Offline
259852023-09-22T23:29:26.818ZINFOcrucible: [2] Replied this last flush ID: 9167
259862023-09-22T23:29:26.818ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Offline ds_transition to Replay
259872023-09-22T23:29:26.818ZINFOcrucible: [2] Transition from Offline to Replay
259882023-09-22T23:29:26.818ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
259892023-09-22T23:29:26.818ZINFOcrucible: [2] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Active Replay ds_transition to Active
259902023-09-22T23:29:26.818ZINFOcrucible: [2] Transition from Replay to Active
259912023-09-22T23:29:26.818ZINFOcrucible: [2] Starts cmd_loop
259922023-09-22T23:29:26.818ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
259932023-09-22T23:29:26.818ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
259942023-09-22T23:29:26.818ZINFOcrucible: Set check for repair
259952023-09-22T23:29:27.819ZINFOcrucible: Checking if live repair is needed
259962023-09-22T23:29:27.819ZINFOcrucible: No Live Repair required at this time
259972023-09-22T23:29:27.819ZWARNcrucible: [2] flow control end
25998 ----------------------------------------------------------------
25999 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26000 Downstairs last five completed: 9469 9468 9467 9466 9465 9464
26001 Upstairs last five completed: 8470 8469 8468 8467 8466 8465
26002 CLIENT: Up:0 ds:0 act:3
26003 Replay: All jobs finished, all DS active.
26004 Main task has work to do, go find it
26005 got dsc Stop(0)
26006 stop 0
26007 Sep 22 23:29:28.727 INFO request completed, latency_us: 546, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 74fad105-a2e7-4c21-a6f1-8bb65ff86b75, remote_addr: 127.0.0[8810] Got stop action so:false kr:false
26008 .1:52471, local_addr: 127.0.0.1:9998
26009 Sep 22 23:29:28.729 INFO request completed, latency_us: 188, response_code: 200, uri: /state/cid/0, method: GET, req_id: f0af0b2b-1336-4a1c-98ea-c90a7f98a871, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
260102023-09-22T23:29:28.729ZWARNcrucible: [0] downstairs disconnected
260112023-09-22T23:29:28.729ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
260122023-09-22T23:29:28.729ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8810 closed looper = 0
260132023-09-22T23:29:28.729ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:false
260142023-09-22T23:29:28.729ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
260152023-09-22T23:29:28.729ZINFOcrucible: [0] 127.0.0.1:8810 task reports offline
260162023-09-22T23:29:28.729ZWARNcrucible: [0] pm_task rx.recv() is None
26017 [8810] Exited with: ExitStatus(unix_wait_status(9))
26018 [8810][0] reports Exit
260192023-09-22T23:29:29.730ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
26020 Sep 22 23:29:32.731 INFO request completed, latency_us: 254, response_code: 200, uri: /state/cid/0, method: GET, req_id: ca62ca33-44cc-42d6-b8ae-599d446e1d3d, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
26021 001/300 Read block 4816 len 16384
26022 002/300 Flush
26023 003/300 Read block 1920 len 16384
26024 004/300 Write block 3138 len 24576 data: 3 3 4 4 4 6
26025 005/300 Read block 3878 len 4096
26026 006/300 Write block 1492 len 12288 data: 6 7 7
26027 007/300 Write block 495 len 28672 data: 7 6 7 6 5 6 9
26028 008/300 Write block 3173 len 32768 data: 5 5 4 5 4 5 5 5
26029 009/300 Read block 1044 len 28672
26030 010/300 Read block 86 len 4096
26031 011/300 Write block 4487 len 32768 data: 5 5 6 6 8 7 6 5
26032 012/300 Write block 3573 len 4096 data: 3
26033 013/300 Read block 612 len 16384
26034 014/300 Write block 780 len 4096 data: 6
26035 015/300 Write block 90 len 28672 data: 3 2 2 3 4 5 5
26036 016/300 Read block 3924 len 32768
26037 017/300 Write block 2336 len 8192 data: 4 4
26038 018/300 Read block 750 len 24576
26039 019/300 Write block 2050 len 32768 data: 4 6 7 7 7 8 7 6
26040 020/300 Write block 1534 len 4096 data: 5
26041 021/300 Read block 4087 len 24576
26042 022/300 Read block 1077 len 8192
26043 023/300 Flush
26044 024/300 Read block 4651 len 4096
26045 025/300 Flush
26046 026/300 Flush
26047 027/300 Flush
26048 028/300 Write block 796 len 32768 data: 4 5 5 5 6 7 6 6
26049 029/300 Write block 1058 len 24576 data: 4 4 3 3 3 4
26050 030/300 Write block 1687 len 4096 data: 7
26051 031/300 Read block 3592 len 8192
26052 032/300 Read block 2718 len 4096
26053 033/300 Read block 2530 len 8192
26054 034/300 Read block 3831 len 32768
26055 035/300 Write block 1258 len 32768 data: 4 4 4 3 4 4 4 4
26056 036/300 Read block 2604 len 12288
26057 037/300 Read block 3878 len 12288
26058 038/300 Write block 912 len 16384 data: 4 4 4 5
26059 039/300 Read block 452 len 8192
26060 040/300 Read block 2458 len 12288
26061 041/300 Read block 3045 len 40960
26062 042/300 Flush
26063 043/300 Flush
26064 044/300 Read block 610 len 8192
26065 045/300 Flush
26066 046/300 Flush
26067 047/300 Read block 3148 len 36864
26068 048/300 Read block 2640 len 32768
26069 049/300 Read block 2853 len 4096
26070 050/300 Read block 4247 len 40960
26071 051/300 Write block 37 len 20480 data: 7 9 9 10 10
26072 052/300 Read block 2635 len 36864
26073 053/300 Write block 146 len 32768 data: 11 10 11 11 12 10 7 5
26074 054/300 Write block 1393 len 12288 data: 4 3 3
26075 055/300 Flush
26076 056/300 Write block 192 len 4096 data: 6
26077 057/300 Write block 3090 len 24576 data: 4 5 5 6 6 6
26078 058/300 Read block 919 len 40960
26079 059/300 Read block 3562 len 28672
26080 060/300 Write block 2147 len 8192 data: 4 5
26081 061/300 Read block 4811 len 28672
26082 062/300 Read block 715 len 16384
26083 063/300 Write block 4397 len 40960 data: 7 7 7 8 8 10 8 7 6 6
26084 064/300 Write block 2891 len 12288 data: 5 5 5
26085 065/300 Read block 2441 len 24576
26086 066/300 Write block 4370 len 40960 data: 7 7 5 6 4 5 6 5 7 7
26087 067/300 Write block 1351 len 12288 data: 5 4 5
26088 068/300 Write block 1801 len 32768 data: 6 6 6 5 6 6 6 5
26089 069/300 Flush
26090 070/300 Flush
26091 071/300 Flush
26092 072/300 Write block 1445 len 40960 data: 6 6 7 9 6 6 5 3 3 4
26093 073/300 Read block 4961 len 4096
26094 074/300 Write block 4737 len 36864 data: 5 3 6 5 6 6 6 7 6
26095 075/300 Write block 118 len 16384 data: 6 5 3 3
26096 076/300 Read block 4261 len 20480
26097 077/300 Read block 152 len 28672
26098 078/300 Read block 4182 len 4096
26099 079/300 Read block 3210 len 4096
26100 080/300 Read block 2665 len 20480
26101 081/300 Read block 3208 len 36864
26102 082/300 Flush
26103 083/300 Flush
26104 084/300 Read block 1735 len 40960
26105 085/300 Write block 3511 len 24576 data: 9 10 11 12 10 11
26106 086/300 Read block 1747 len 40960
26107 087/300 Write block 1932 len 20480 data: 5 5 4 4 3
26108 088/300 Flush
26109 089/300 Read block 4647 len 20480
26110 090/300 Write block 741 len 4096 data: 6
26111 091/300 Write block 3930 len 12288 data: 8 10 12
26112 092/300 Read block 53 len 12288
26113 093/300 Read block 3144 len 32768
26114 094/300 Read block 4464 len 40960
26115 095/300 Write block 2177 len 32768 data: 5 4 5 6 7 9 8 8
26116 096/300 Read block 2219 len 36864
26117 097/300 Read block 4912 len 16384
26118 098/300 Read block 4935 len 24576
26119 099/300 Read block 355 len 16384
26120 100/300 Write block 4418 len 4096 data: 8
26121 101/300 Read block 4164 len 20480
26122 102/300 Read block 4915 len 36864
26123 103/300 Flush
26124 104/300 Read block 1064 len 28672
26125 105/300 Flush
26126 106/300 Read block 1854 len 4096
26127 107/300 Read block 3820 len 36864
26128 108/300 Read block 4347 len 24576
26129 109/300 Read block 2330 len 32768
26130 110/300 Write block 2604 len 24576 data: 4 5 6 6 7 6
26131 111/300 Write block 3051 len 8192 data: 7 8
26132 112/300 Write block 3113 len 12288 data: 5 5 5
26133 113/300 Read block 4150 len 4096
26134 114/300 Read block 1637 len 28672
26135 115/300 Read block 4172 len 32768
26136 116/300 Read block 1688 len 40960
26137 117/300 Read block 4195 len 8192
26138 118/300 Flush
26139 119/300 Read block 1552 len 4096
26140 120/300 Write block 332 len 16384 data: 4 5 5 5
26141 121/300 Write block 2081 len 16384 data: 5 5 6 6
26142 122/300 Read block 3034 len 12288
26143 123/300 Read block 2014 len 40960
26144 124/300 Flush
26145 125/300 Read block 2382 len 20480
26146 126/300 Read block 2462 len 32768
26147 127/300 Read block 4893 len 40960
26148 128/300 Read block 1398 len 40960
26149 129/300 Read block 1053 len 12288
26150 130/300 Read block 3988 len 4096
26151 131/300 Write block 942 len 12288 data: 2 2 2
26152 132/300 Flush
26153 133/300 Flush
26154 134/300 Write block 1806 len 28672 data: 7 7 6 5 6 6 7
26155 135/300 Write block 4362 len 20480 data: 4 5 5 6 6
26156 136/300 Read block 4724 len 20480
26157 137/300 Read block 3904 len 16384
26158 138/300 Read block 862 len 36864
26159 139/300 Write block 3431 len 36864 data: 4 3 7 7 7 8 8 7 8
26160 140/300 Read block 670 len 8192
26161 141/300 Read block 3190 len 20480
26162 142/300 Write block 4700 len 32768 data: 6 6 4 4 3 2 2 2
26163 143/300 Read block 4510 len 40960
26164 144/300 Write block 2925 len 36864 data: 8 8 4 5 5 5 5 4 4
26165 145/300 Read block 4164 len 16384
26166 146/300 Read block 1651 len 16384
26167 147/300 Read block 769 len 12288
26168 148/300 Flush
26169 149/300 Write block 4296 len 28672 data: 5 4 4 5 5 4 4
26170 150/300 Read block 3233 len 24576
26171 151/300 Read block 205 len 32768
26172 152/300 Write block 3797 len 4096 data: 5
26173 153/300 Read block 345 len 12288
26174 154/300 Read block 4627 len 12288
26175 155/300 Read block 3996 len 4096
26176 156/300 Read block 154 len 8192
26177 157/300 Write block 97 len 16384 data: 5 7 5 3
26178 158/300 Read block 3094 len 20480
26179 159/300 Write block 3046 len 24576 data: 5 7 5 7 7 8
26180 160/300 Read block 2346 len 36864
26181 161/300 Read block 882 len 12288
26182 162/300 Write block 2224 len 40960 data: 4 2 4 4 5 5 5 5 3 2
26183 163/300 Read block 2814 len 16384
26184 164/300 Read block 3924 len 32768
26185 165/300 Flush
26186 166/300 Read block 3258 len 24576
26187 167/300 Flush
26188 168/300 Read block 2765 len 4096
26189 169/300 Flush
26190 170/300 Read block 2899 len 32768
26191 171/300 Write block 1249 len 8192 data: 5 5
26192 172/300 Write block 3378 len 40960 data: 4 4 3 4 3 3 3 4 5 6
26193 173/300 Read block 2372 len 12288
26194 174/300 Write block 1200 len 40960 data: 6 5 5 7 9 10 11 12 13 12
26195 175/300 Read block 2596 len 20480
26196 176/300 Read block 3833 len 12288
26197 177/300 Write block 4115 len 24576 data: 3 3 4 4 3 3
26198 178/300 Write block 1560 len 20480 data: 5 5 3 3 3
26199 179/300 Read block 485 len 20480
26200 180/300 Read block 3971 len 36864
26201 181/300 Read block 2069 len 40960
26202 182/300 Write block 1686 len 12288 data: 7 8 6
26203 183/300 Write block 4285 len 4096 data: 7
26204 184/300 Write block 1926 len 4096 data: 4
26205 185/300 Write block 321 len 4096 data: 6
26206 186/300 Read block 2300 len 32768
26207 187/300 Write block 1816 len 8192 data: 8 8
26208 188/300 Write block 3875 len 40960 data: 8 7 6 5 5 5 5 4 4 4
26209 189/300 Flush
26210 190/300 Read block 1468 len 8192
26211 191/300 Read block 4306 len 4096
26212 192/300 Write block 3476 len 36864 data: 4 5 7 8 9 8 8 7 7
26213 193/300 Write block 705 len 24576 data: 4 4 5 6 5 5
26214 194/300 Read block 2016 len 32768
26215 195/300 Read block 4034 len 20480
26216 196/300 Read block 4708 len 32768
26217 197/300 Write block 4347 len 40960 data: 5 7 7 7 6 6 7 7 7 5
26218 198/300 Write block 3398 len 12288 data: 5 4 5
26219 199/300 Write block 335 len 20480 data: 6 6 7 6 8
26220 200/300 Write block 261 len 16384 data: 6 5 6 4
26221 201/300 Write block 1750 len 20480 data: 5 5 6 7 7
26222 202/300 Read block 59 len 8192
26223 203/300 Flush
26224 204/300 Read block 3471 len 4096
26225 205/300 Read block 1734 len 28672
26226 206/300 Read block 311 len 4096
26227 207/300 Read block 1096 len 32768
26228 208/300 Read block 4480 len 4096
26229 209/300 Flush
26230 210/300 Read block 523 len 24576
26231 211/300 Flush
26232 212/300 Write block 2984 len 20480 data: 3 3 2 2 3
26233 213/300 Write block 3739 len 40960 data: 4 4 4 3 5 5 5 8 7 8
26234 214/300 Write block 4423 len 36864 data: 8 8 7 10 8 8 7 7 8
26235 215/300 Write block 2114 len 16384 data: 9 9 7 7
26236 216/300 Read block 2642 len 28672
26237 217/300 Write block 1056 len 12288 data: 3 4 5
26238 218/300 Write block 1814 len 4096 data: 5
26239 219/300 Read block 3068 len 4096
26240 220/300 Write block 4741 len 16384 data: 7 7 7 8
26241 221/300 Read block 1867 len 36864
26242 222/300 Write block 4598 len 16384 data: 6 7 6 6
26243 223/300 Write block 2130 len 36864 data: 4 4 5 6 6 6 6 6 8
26244 224/300 Write block 3550 len 40960 data: 6 7 7 7 6 6 5 3 3 3
26245 225/300 Write block 4015 len 36864 data: 6 5 7 7 8 7 7 5 4
26246 226/300 Read block 3700 len 32768
26247 227/300 Flush
26248 228/300 Flush
26249 229/300 Read block 2979 len 8192
26250 230/300 Write block 4323 len 12288 data: 8 7 8
26251 231/300 Write block 2660 len 16384 data: 5 6 6 4
26252 232/300 Write block 524 len 28672 data: 6 7 8 7 7 7 6
26253 233/300 Write block 1395 len 32768 data: 4 4 5 5 5 7 8 8
26254 234/300 Write block 1302 len 28672 data: 6 5 6 6 5 5 5
26255 235/300 Write block 3633 len 40960 data: 5 5 5 4 3 3 4 4 4 3
26256 236/300 Write block 4718 len 4096 data: 3
26257 237/300 Read block 2543 len 16384
26258 238/300 Write block 3048 len 16384 data: 6 8 8 9
26259 239/300 Write block 4031 len 12288 data: 3 5 5
26260 240/300 Read block 2081 len 8192
26261 241/300 Flush
26262 242/300 Read block 2962 len 4096
26263 243/300 Write block 3918 len 28672 data: 4 4 4 3 3 3 2
26264 244/300 Read block 2832 len 32768
26265 245/300 Read block 3922 len 12288
26266 246/300 Write block 1743 len 36864 data: 3 3 3 3 3 3 3 6 6
26267 247/300 Read block 3843 len 8192
26268 248/300 Write block 2315 len 28672 data: 6 6 6 7 6 6 7
26269 249/300 Write block 3724 len 20480 data: 5 5 5 6 8
26270 250/300 Write block 4114 len 36864 data: 3 4 4 5 5 4 4 3 5
26271 251/300 Read block 1589 len 8192
26272 252/300 Read block 2894 len 36864
26273 253/300 Write block 2996 len 40960 data: 5 6 5 7 7 7 6 5 5 5
26274 254/300 Write block 4491 len 40960 data: 9 8 7 6 4 4 6 6 5 5
26275 255/300 Write block 1559 len 12288 data: 6 6 6
26276 256/300 Read block 3057 len 16384
26277 257/300 Read block 4148 len 16384
26278 258/300 Read block 2751 len 20480
26279 259/300 Read block 519 len 36864
26280 260/300 Write block 923 len 12288 data: 4 5 6
26281 261/300 Flush
26282 262/300 Read block 4458 len 8192
26283 263/300 Read block 4067 len 36864
26284 264/300 Read block 2632 len 16384
26285 265/300 Write block 3312 len 12288 data: 5 5 6
26286 266/300 Read block 2638 len 40960
26287 267/300 Write block 1241 len 24576 data: 5 6 6 4 4 3
26288 268/300 Read block 3111 len 12288
26289 269/300 Read block 2267 len 4096
26290 270/300 Read block 805 len 20480
26291 271/300 Flush
26292 272/300 Write block 4988 len 8192 data: 11 10
26293 273/300 Write block 2943 len 16384 data: 3 4 5 4
26294 274/300 Write block 2039 len 36864 data: 3 3 3 3 3 2 4 4 4
26295 275/300 Read block 2899 len 4096
26296 276/300 Write block 4703 len 32768 data: 5 4 3 3 3 2 2 2
26297 277/300 Read block 442 len 16384
26298 278/300 Flush
26299 279/300 Read block 2914 len 8192
26300 280/300 Read block 4896 len 4096
26301 281/300 Flush
26302 282/300 Read block 1206 len 24576
26303 283/300 Write block 4964 len 12288 data: 6 5 3
26304 284/300 Read block 256 len 4096
26305 285/300 Read block 3527 len 40960
26306 286/300 Flush
26307 287/300 Read block 4817 len 32768
26308 288/300 Write block 349 len 16384 data: 7 7 7 6
26309 289/300 Write block 914 len 28672 data: 5 6 4 4 3 4 4
26310 290/300 Read block 1149 len 40960
26311 291/300 Read block 1324 len 32768
26312 292/300 Write block 2019 len 40960 data: 5 6 5 7 6 6 6 6 6 8
26313 293/300 Read block 3979 len 4096
26314 294/300 Read block 4425 len 40960
26315 295/300 Read block 3241 len 32768
26316 296/300 Read block 2306 len 12288
26317 297/300 Read block 549 len 36864
26318 298/300 Flush
26319 299/300 Write block 2431 len 20480 data: 7 5 6 5 6
26320 300/300 Write block 3550 len 8192 data: 7 8
26321 Main task has work to do, go find it
26322 got dsc Start(0)
26323 start 0
26324 Sep 22 23:29:33.658 INFO request completed, latency_us: 442, response_code: 204, uri: /start/cid/0, method: POST, req_id: 1c8e591a-7a1b-44f2-8b33-0861accdd9d5, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
26325 Replay: started 0, returned:Ok(())
26326 ----------------------------------------------------------------
26327 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:300
26328 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
26329 8471 Acked 9470 Read 4 New Done Done false
26330 8472 Acked 9471 Flush 0 New Done Done false
26331 8473 Acked 9472 Read 4 New Done Done false
26332 8474 Acked 9473 Write 6 New Done Done false
26333 8475 Acked 9474 Read 1 New Done Done false
26334 8476 Acked 9475 Write 3 New Done Done false
26335 8477 Acked 9476 Write 7 New Done Done false
26336 8478 Acked 9477 Write 8 New Done Done false
26337 8479 Acked 9478 Read 7 New Done Done false
26338 8480 Acked 9479 Read 1 New Done Done false
26339 8481 Acked 9480 Write 8 New Done Done false
26340 8482 Acked 9481 Write 1 New Done Done false
26341 8483 Acked 9482 Read 4 New Done Done false
26342 8484 Acked 9483 Write 1 New Done Done false
26343 8485 Acked 9484 Write 7 New Done Done false
26344 8486 Acked 9485 Read 8 New Done Done false
26345 8487 Acked 9486 Write 2 New Done Done false
26346 8488 Acked 9487 Read 6 New Done Done false
26347 8489 Acked 9488 Write 8 New Done Done false
26348 8490 Acked 9489 Write 1 New Done Done false
26349 8491 Acked 9490 Read 6 New Done Done false
26350 8492 Acked 9491 Read 2 New Done Done false
26351 8493 Acked 9492 Flush 0 New Done Done false
26352 8494 Acked 9493 Read 1 New Done Done false
26353 8495 Acked 9494 Flush 0 New Done Done false
26354 8496 Acked 9495 Flush 0 New Done Done false
26355 8497 Acked 9496 Flush 0 New Done Done false
26356 8498 Acked 9497 Write 8 New Done Done false
26357 8499 Acked 9498 Write 6 New Done Done false
26358 8500 Acked 9499 Write 1 New Done Done false
26359 8501 Acked 9500 Read 2 New Done Done false
26360 8502 Acked 9501 Read 1 New Done Done false
26361 8503 Acked 9502 Read 2 New Done Done false
26362 8504 Acked 9503 Read 8 New Done Done false
26363 8505 Acked 9504 Write 8 New Done Done false
26364 8506 Acked 9505 Read 3 New Done Done false
26365 8507 Acked 9506 Read 3 New Done Done false
26366 8508 Acked 9507 Write 4 New Done Done false
26367 8509 Acked 9508 Read 2 New Done Done false
26368 8510 Acked 9509 Read 3 New Done Done false
26369 8511 Acked 9510 Read 10 New Done Done false
26370 8512 Acked 9511 Flush 0 New Done Done false
26371 8513 Acked 9512 Flush 0 New Done Done false
26372 8514 Acked 9513 Read 2 New Done Done false
26373 8515 Acked 9514 Flush 0 New Done Done false
26374 8516 Acked 9515 Flush 0 New Done Done false
26375 8517 Acked 9516 Read 9 New Done Done false
26376 8518 Acked 9517 Read 8 New Done Done false
26377 8519 Acked 9518 Read 1 New Done Done false
26378 8520 Acked 9519 Read 10 New Done Done false
26379 8521 Acked 9520 Write 5 New Done Done false
26380 8522 Acked 9521 Read 9 New Done Done false
26381 8523 Acked 9522 Write 8 New Done Done false
26382 8524 Acked 9523 Write 3 New Done Done false
26383 8525 Acked 9524 Flush 0 New Done Done false
26384 8526 Acked 9525 Write 1 New Done Done false
26385 8527 Acked 9526 Write 6 New Done Done false
26386 8528 Acked 9527 Read 10 New Done Done false
26387 8529 Acked 9528 Read 7 New Done Done false
26388 8530 Acked 9529 Write 2 New Done Done false
26389 8531 Acked 9530 Read 7 New Done Done false
26390 8532 Acked 9531 Read 4 New Done Done false
26391 8533 Acked 9532 Write 10 New Done Done false
26392 8534 Acked 9533 Write 3 New Done Done false
26393 8535 Acked 9534 Read 6 New Done Done false
26394 8536 Acked 9535 Write 10 New Done Done false
26395 8537 Acked 9536 Write 3 New Done Done false
26396 8538 Acked 9537 Write 8 New Done Done false
26397 8539 Acked 9538 Flush 0 New Done Done false
26398 8540 Acked 9539 Flush 0 New Done Done false
26399 8541 Acked 9540 Flush 0 New Done Done false
26400 8542 Acked 9541 Write 10 New Done Done false
26401 8543 Acked 9542 Read 1 New Done Done false
26402 8544 Acked 9543 Write 9 New Done Done false
26403 8545 Acked 9544 Write 4 New Done Done false
26404 8546 Acked 9545 Read 5 New Done Done false
26405 8547 Acked 9546 Read 7 New Done Done false
26406 8548 Acked 9547 Read 1 New Done Done false
26407 8549 Acked 9548 Read 1 New Done Done false
26408 8550 Acked 9549 Read 5 New Done Done false
26409 8551 Acked 9550 Read 9 New Done Done false
26410 8552 Acked 9551 Flush 0 New Done Done false
26411 8553 Acked 9552 Flush 0 New Done Done false
26412 8554 Acked 9553 Read 10 New Done Done false
26413 8555 Acked 9554 Write 6 New Done Done false
26414 8556 Acked 9555 Read 10 New Done Done false
26415 8557 Acked 9556 Write 5 New Done Done false
26416 8558 Acked 9557 Flush 0 New Done Done false
26417 8559 Acked 9558 Read 5 New Done Done false
26418 8560 Acked 9559 Write 1 New Done Done false
26419 8561 Acked 9560 Write 3 New Done Done false
26420 8562 Acked 9561 Read 3 New Done Done false
26421 8563 Acked 9562 Read 8 New Done Done false
26422 8564 Acked 9563 Read 10 New Done Done false
26423 8565 Acked 9564 Write 8 New Done Done false
26424 8566 Acked 9565 Read 9 New Done Done false
26425 8567 Acked 9566 Read 4 New Done Done false
26426 8568 Acked 9567 Read 6 New Done Done false
26427 8569 Acked 9568 Read 4 New Done Done false
26428 8570 Acked 9569 Write 1 New Done Done false
26429 8571 Acked 9570 Read 5 New Done Done false
26430 8572 Acked 9571 Read 9 New Done Done false
26431 8573 Acked 9572 Flush 0 New Done Done false
26432 8574 Acked 9573 Read 7 New Done Done false
26433 8575 Acked 9574 Flush 0 New Done Done false
26434 8576 Acked 9575 Read 1 New Done Done false
26435 8577 Acked 9576 Read 9 New Done Done false
26436 8578 Acked 9577 Read 6 New Done Done false
26437 8579 Acked 9578 Read 8 New Done Done false
26438 8580 Acked 9579 Write 6 New Done Done false
26439 8581 Acked 9580 Write 2 New Done Done false
26440 8582 Acked 9581 Write 3 New Done Done false
26441 8583 Acked 9582 Read 1 New Done Done false
26442 8584 Acked 9583 Read 7 New Done Done false
26443 8585 Acked 9584 Read 8 New Done Done false
26444 8586 Acked 9585 Read 10 New Done Done false
26445 8587 Acked 9586 Read 2 New Done Done false
26446 8588 Acked 9587 Flush 0 New Done Done false
26447 8589 Acked 9588 Read 1 New Done Done false
26448 8590 Acked 9589 Write 4 New Done Done false
26449 8591 Acked 9590 Write 4 New Done Done false
26450 8592 Acked 9591 Read 3 New Done Done false
26451 8593 Acked 9592 Read 10 New Done Done false
26452 8594 Acked 9593 Flush 0 New Done Done false
26453 8595 Acked 9594 Read 5 New Done Done false
26454 8596 Acked 9595 Read 8 New Done Done false
26455 8597 Acked 9596 Read 10 New Done Done false
26456 8598 Acked 9597 Read 10 New Done Done false
26457 8599 Acked 9598 Read 3 New Done Done false
26458 8600 Acked 9599 Read 1 New Done Done false
26459 8601 Acked 9600 Write 3 New Done Done false
26460 8602 Acked 9601 Flush 0 New Done Done false
26461 8603 Acked 9602 Flush 0 New Done Done false
26462 8604 Acked 9603 Write 7 New Done Done false
26463 8605 Acked 9604 Write 5 New Done Done false
26464 8606 Acked 9605 Read 5 New Done Done false
26465 8607 Acked 9606 Read 4 New Done Done false
26466 8608 Acked 9607 Read 9 New Done Done false
26467 8609 Acked 9608 Write 9 New Done Done false
26468 8610 Acked 9609 Read 2 New Done Done false
26469 8611 Acked 9610 Read 5 New Done Done false
26470 8612 Acked 9611 Write 8 New Done Done false
26471 8613 Acked 9612 Read 10 New Done Done false
26472 8614 Acked 9613 Write 9 New Done Done false
26473 8615 Acked 9614 Read 4 New Done Done false
26474 8616 Acked 9615 Read 4 New Done Done false
26475 8617 Acked 9616 Read 3 New Done Done false
26476 8618 Acked 9617 Flush 0 New Done Done false
26477 8619 Acked 9618 Write 7 New Done Done false
26478 8620 Acked 9619 Read 6 New Done Done false
26479 8621 Acked 9620 Read 8 New Done Done false
26480 8622 Acked 9621 Write 1 New Done Done false
26481 8623 Acked 9622 Read 3 New Done Done false
26482 8624 Acked 9623 Read 3 New Done Done false
26483 8625 Acked 9624 Read 1 New Done Done false
26484 8626 Acked 9625 Read 2 New Done Done false
26485 8627 Acked 9626 Write 4 New Done Done false
26486 8628 Acked 9627 Read 5 New Done Done false
26487 8629 Acked 9628 Write 6 New Done Done false
26488 8630 Acked 9629 Read 9 New Done Done false
26489 8631 Acked 9630 Read 3 New Done Done false
26490 8632 Acked 9631 Write 10 New Done Done false
26491 8633 Acked 9632 Read 4 New Done Done false
26492 8634 Acked 9633 Read 8 New Done Done false
26493 8635 Acked 9634 Flush 0 New Done Done false
26494 8636 Acked 9635 Read 6 New Done Done false
26495 8637 Acked 9636 Flush 0 New Done Done false
26496 8638 Acked 9637 Read 1 New Done Done false
26497 8639 Acked 9638 Flush 0 New Done Done false
26498 8640 Acked 9639 Read 8 New Done Done false
26499 8641 Acked 9640 Write 2 New Done Done false
26500 8642 Acked 9641 Write 10 New Done Done false
26501 8643 Acked 9642 Read 3 New Done Done false
26502 8644 Acked 9643 Write 10 New Done Done false
26503 8645 Acked 9644 Read 5 New Done Done false
26504 8646 Acked 9645 Read 3 New Done Done false
26505 8647 Acked 9646 Write 6 New Done Done false
26506 8648 Acked 9647 Write 5 New Done Done false
26507 8649 Acked 9648 Read 5 New Done Done false
26508 8650 Acked 9649 Read 9 New Done Done false
26509 8651 Acked 9650 Read 10 New Done Done false
26510 8652 Acked 9651 Write 3 New Done Done false
26511 8653 Acked 9652 Write 1 New Done Done false
26512 8654 Acked 9653 Write 1 New Done Done false
26513 8655 Acked 9654 Write 1 New Done Done false
26514 8656 Acked 9655 Read 8 New Done Done false
26515 8657 Acked 9656 Write 2 New Done Done false
26516 8658 Acked 9657 Write 10 New Done Done false
26517 8659 Acked 9658 Flush 0 New Done Done false
26518 8660 Acked 9659 Read 2 New Done Done false
26519 8661 Acked 9660 Read 1 New Done Done false
26520 8662 Acked 9661 Write 9 New Done Done false
26521 8663 Acked 9662 Write 6 New Done Done false
26522 8664 Acked 9663 Read 8 New Done Done false
26523 8665 Acked 9664 Read 5 New Done Done false
26524 8666 Acked 9665 Read 8 New Done Done false
26525 8667 Acked 9666 Write 10 New Done Done false
26526 8668 Acked 9667 Write 3 New Done Done false
26527 8669 Acked 9668 Write 5 New Done Done false
26528 8670 Acked 9669 Write 4 New Done Done false
26529 8671 Acked 9670 Write 5 New Done Done false
26530 8672 Acked 9671 Read 2 New Done Done false
26531 8673 Acked 9672 Flush 0 New Done Done false
26532 8674 Acked 9673 Read 1 New Done Done false
26533 8675 Acked 9674 Read 7 New Done Done false
26534 8676 Acked 9675 Read 1 New Done Done false
26535 8677 Acked 9676 Read 8 New Done Done false
26536 8678 Acked 9677 Read 1 New Done Done false
26537 8679 Acked 9678 Flush 0 New Done Done false
26538 8680 Acked 9679 Read 6 New Done Done false
26539 8681 Acked 9680 Flush 0 New Done Done false
26540 8682 Acked 9681 Write 5 New Done Done false
26541 8683 Acked 9682 Write 10 New Done Done false
26542 8684 Acked 9683 Write 9 New Done Done false
26543 8685 Acked 9684 Write 4 New Done Done false
26544 8686 Acked 9685 Read 7 New Done Done false
26545 8687 Acked 9686 Write 3 New Done Done false
26546 8688 Acked 9687 Write 1 New Done Done false
26547 8689 Acked 9688 Read 1 New Done Done false
26548 8690 Acked 9689 Write 4 New Done Done false
26549 8691 Acked 9690 Read 9 New Done Done false
26550 8692 Acked 9691 Write 4 New Done Done false
26551 8693 Acked 9692 Write 9 New Done Done false
26552 8694 Acked 9693 Write 10 New Done Done false
26553 8695 Acked 9694 Write 9 New Done Done false
26554 8696 Acked 9695 Read 8 New Done Done false
26555 8697 Acked 9696 Flush 0 New Done Done false
26556 8698 Acked 9697 Flush 0 New Done Done false
26557 8699 Acked 9698 Read 2 New Done Done false
26558 8700 Acked 9699 Write 3 New Done Done false
26559 8701 Acked 9700 Write 4 New Done Done false
26560 8702 Acked 9701 Write 7 New Done Done false
26561 8703 Acked 9702 Write 8 New Done Done false
26562 8704 Acked 9703 Write 7 New Done Done false
26563 8705 Acked 9704 Write 10 New Done Done false
26564 8706 Acked 9705 Write 1 New Done Done false
26565 8707 Acked 9706 Read 4 New Done Done false
26566 8708 Acked 9707 Write 4 New Done Done false
26567 8709 Acked 9708 Write 3 New Done Done false
26568 8710 Acked 9709 Read 2 New Done Done false
26569 8711 Acked 9710 Flush 0 New Done Done false
26570 8712 Acked 9711 Read 1 New Done Done false
26571 8713 Acked 9712 Write 7 New Done Done false
26572 8714 Acked 9713 Read 8 New Done Done false
26573 8715 Acked 9714 Read 3 New Done Done false
26574 8716 Acked 9715 Write 9 New Done Done false
26575 8717 Acked 9716 Read 2 New Done Done false
26576 8718 Acked 9717 Write 7 New Done Done false
26577 8719 Acked 9718 Write 5 New Done Done false
26578 8720 Acked 9719 Write 9 New Done Done false
26579 8721 Acked 9720 Read 2 New Done Done false
26580 8722 Acked 9721 Read 9 New Done Done false
26581 8723 Acked 9722 Write 10 New Done Done false
26582 8724 Acked 9723 Write 10 New Done Done false
26583 8725 Acked 9724 Write 3 New Done Done false
26584 8726 Acked 9725 Read 4 New Done Done false
26585 8727 Acked 9726 Read 4 New Done Done false
26586 8728 Acked 9727 Read 5 New Done Done false
26587 8729 Acked 9728 Read 9 New Done Done false
26588 8730 Acked 9729 Write 3 New Done Done false
26589 8731 Acked 9730 Flush 0 New Done Done false
26590 8732 Acked 9731 Read 2 New Done Done false
26591 8733 Acked 9732 Read 9 New Done Done false
26592 8734 Acked 9733 Read 4 New Done Done false
26593 8735 Acked 9734 Write 3 New Done Done false
26594 8736 Acked 9735 Read 10 New Done Done false
26595 8737 Acked 9736 Write 6 New Done Done false
26596 8738 Acked 9737 Read 3 New Done Done false
26597 8739 Acked 9738 Read 1 New Done Done false
26598 8740 Acked 9739 Read 5 New Done Done false
26599 8741 Acked 9740 Flush 0 New Done Done false
26600 8742 Acked 9741 Write 2 New Done Done false
26601 8743 Acked 9742 Write 4 New Done Done false
26602 8744 Acked 9743 Write 9 New Done Done false
26603 8745 Acked 9744 Read 1 New Done Done false
26604 8746 Acked 9745 Write 8 New Done Done false
26605 8747 Acked 9746 Read 4 New Done Done false
26606 8748 Acked 9747 Flush 0 New Done Done false
26607 8749 Acked 9748 Read 2 New Done Done false
26608 8750 Acked 9749 Read 1 New Done Done false
26609 8751 Acked 9750 Flush 0 New Done Done false
26610 8752 Acked 9751 Read 6 New Done Done false
26611 8753 Acked 9752 Write 3 New Done Done false
26612 8754 Acked 9753 Read 1 New Done Done false
26613 8755 Acked 9754 Read 10 New Done Done false
26614 8756 Acked 9755 Flush 0 New Done Done false
26615 8757 Acked 9756 Read 8 New Done Done false
26616 8758 Acked 9757 Write 4 New Done Done false
26617 8759 Acked 9758 Write 7 New Done Done false
26618 8760 Acked 9759 Read 10 New Done Done false
26619 8761 Acked 9760 Read 8 New Done Done false
26620 8762 Acked 9761 Write 10 New Done Done false
26621 8763 Acked 9762 Read 1 New Done Done false
26622 8764 Acked 9763 Read 10 New Done Done false
26623 8765 Acked 9764 Read 8 New Done Done false
26624 8766 Acked 9765 Read 3 New Done Done false
26625 8767 Acked 9766 Read 9 New Done Done false
26626 8768 Acked 9767 Flush 0 New Done Done false
26627 8769 Acked 9768 Write 5 New Done Done false
26628 8770 Acked 9769 Write 2 New Sent Sent false
26629 STATES DS:0 DS:1 DS:2 TOTAL
26630 New 300 0 0 300
26631 Sent 0 1 1 2
26632 Done 0 299 299 598
26633 Skipped 0 0 0 0
26634 Error 0 0 0 0
26635 Last Flush: 9469 9767 9767
26636 Downstairs last five completed: 9469 9468 9467 9466 9465 9464
26637 Upstairs last five completed: 8770 8769 8768 8767 8766 8765
26638 CLIENT: Up:0 ds:300 act:2
26639 [8810] got start action
26640 [8810] I am going to restart
26641 Starting downstairs at port 8810
26642 Make output file at "/tmp/dsc/downstairs-8810.txt"
26643 [8810][0] reports Starting
26644 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1202)
26645 [8810][0] reports Running
266462023-09-22T23:29:35.739ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 0
266472023-09-22T23:29:35.739ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state Offline
266482023-09-22T23:29:35.739ZINFOcrucible: [0] client re-new 301 jobs since flush 9469 = downstairs
266492023-09-22T23:29:35.740ZINFOcrucible: [0] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
266502023-09-22T23:29:35.741ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
266512023-09-22T23:29:35.741ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
266522023-09-22T23:29:35.741ZINFOcrucible: Returning client:0 UUID:12345678-0000-0000-0000-000000008810 matches
266532023-09-22T23:29:35.741ZINFOcrucible: [0] send last flush ID to this DS: 9469
266542023-09-22T23:29:35.741ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Offline Active Active
266552023-09-22T23:29:35.741ZINFOcrucible: [0] Replied this last flush ID: 9469
266562023-09-22T23:29:35.741ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Offline Active Active ds_transition to Replay
266572023-09-22T23:29:35.741ZINFOcrucible: [0] Transition from Offline to Replay
266582023-09-22T23:29:35.741ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
266592023-09-22T23:29:35.741ZINFOcrucible: [0] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Replay Active Active ds_transition to Active
266602023-09-22T23:29:35.741ZINFOcrucible: [0] Transition from Replay to Active
266612023-09-22T23:29:35.741ZINFOcrucible: [0] Starts cmd_loop
266622023-09-22T23:29:35.741ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
266632023-09-22T23:29:35.741ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
266642023-09-22T23:29:35.741ZINFOcrucible: Set check for repair
266652023-09-22T23:29:36.743ZINFOcrucible: Checking if live repair is needed
266662023-09-22T23:29:36.743ZINFOcrucible: No Live Repair required at this time
266672023-09-22T23:29:36.743ZWARNcrucible: [0] flow control end
26668 ----------------------------------------------------------------
26669 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26670 Downstairs last five completed: 9770 9769 9768 9767 9766 9765
26671 Upstairs last five completed: 8771 8770 8769 8768 8767 8766
26672 CLIENT: Up:0 ds:0 act:3
26673 Replay: All jobs finished, all DS active.
26674 Main task has work to do, go find it
26675 got dsc Stop(1)
26676 stop 1
26677 Sep 22 23:29:37.663 INFO request completed, latency_us: 502, response_code: 204, uri: /stop/cid/1, method: POST, req_id: ac7f8064-082c-4182-af3c-76764b048929, remote_addr: 127.0.[8820] Got stop action so:false kr:false
26678 0.1:52471, local_addr: 127.0.0.1:9998
26679 Sep 22 23:29:37.664 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/1, method: GET, req_id: 148327db-0d7c-4a8b-a266-8a45d0045268, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
266802023-09-22T23:29:37.665ZWARNcrucible: [1] downstairs disconnected
266812023-09-22T23:29:37.665ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Gone missing, transition from Active to Offline
266822023-09-22T23:29:37.665ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 connection to 127.0.0.1:8820 closed looper = 1
266832023-09-22T23:29:37.665ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:false
266842023-09-22T23:29:37.665ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
266852023-09-22T23:29:37.665ZINFOcrucible: [1] 127.0.0.1:8820 task reports offline
266862023-09-22T23:29:37.665ZWARNcrucible: [1] pm_task rx.recv() is None
26687 [8820] Exited with: ExitStatus(unix_wait_status(9))
26688 [8820][1] reports Exit
266892023-09-22T23:29:40.667ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
26690 Sep 22 23:29:41.667 INFO request completed, latency_us: 282, response_code: 200, uri: /state/cid/1, method: GET, req_id: 14d47947-3637-4b02-9efe-3fe5f2092efa, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
26691 001/300 Flush
26692 002/300 Read block 3805 len 4096
26693 003/300 Write block 1609 len 32768 data: 8 6 5 5 5 4 3 4
26694 004/300 Write block 914 len 8192 data: 6 7
26695 005/300 Read block 4411 len 28672
26696 006/300 Read block 838 len 28672
26697 007/300 Write block 1271 len 8192 data: 9 7
26698 008/300 Flush
26699 009/300 Write block 4378 len 28672 data: 8 8 7 5 3 3 3
26700 010/300 Write block 4898 len 16384 data: 4 4 4 5
26701 011/300 Write block 564 len 4096 data: 4
26702 012/300 Read block 1146 len 12288
26703 013/300 Read block 2327 len 32768
26704 014/300 Read block 987 len 24576
26705 015/300 Read block 1246 len 20480
26706 016/300 Write block 3635 len 8192 data: 6 5
26707 017/300 Write block 415 len 16384 data: 10 10 10 10
26708 018/300 Flush
26709 019/300 Write block 1235 len 8192 data: 6 6
26710 020/300 Write block 1016 len 28672 data: 3 5 5 5 5 5 5
26711 021/300 Read block 2348 len 20480
26712 022/300 Read block 961 len 12288
26713 023/300 Write block 190 len 20480 data: 6 5 7 6 7
26714 024/300 Write block 4293 len 4096 data: 8
26715 025/300 Flush
26716 026/300 Write block 2365 len 28672 data: 7 6 4 5 4 4 5
26717 027/300 Read block 2402 len 4096
26718 028/300 Write block 2148 len 28672 data: 6 4 4 3 4 4 4
26719 029/300 Read block 3920 len 36864
26720 030/300 Write block 1566 len 20480 data: 5 6 7 6 6
26721 031/300 Write block 1336 len 40960 data: 10 10 10 9 9 8 8 7 6 6
26722 032/300 Read block 4542 len 24576
26723 033/300 Read block 2757 len 36864
26724 034/300 Write block 2550 len 32768 data: 5 5 7 7 7 8 8 8
26725 035/300 Write block 3119 len 12288 data: 7 5 6
26726 036/300 Write block 1209 len 8192 data: 13 9
26727 037/300 Write block 2812 len 12288 data: 7 6 5
26728 038/300 Read block 2018 len 16384
26729 039/300 Write block 4438 len 12288 data: 4 4 5
26730 040/300 Write block 313 len 28672 data: 3 3 3 3 2 4 4
26731 041/300 Read block 629 len 36864
26732 042/300 Read block 795 len 20480
26733 043/300 Write block 2781 len 4096 data: 4
26734 044/300 Read block 1504 len 24576
26735 045/300 Read block 3863 len 20480
26736 046/300 Flush
26737 047/300 Read block 1287 len 32768
26738 048/300 Read block 2681 len 32768
26739 049/300 Flush
26740 050/300 Write block 2772 len 8192 data: 4 4
26741 051/300 Flush
26742 052/300 Read block 1352 len 36864
26743 053/300 Flush
26744 054/300 Read block 3271 len 36864
26745 055/300 Read block 4198 len 8192
26746 056/300 Read block 2497 len 28672
26747 057/300 Read block 3821 len 8192
26748 058/300 Read block 4226 len 4096
26749 059/300 Read block 3914 len 36864
26750 060/300 Write block 1603 len 28672 data: 4 4 8 6 6 6 9
26751 061/300 Write block 4252 len 12288 data: 5 4 4
26752 062/300 Flush
26753 063/300 Flush
26754 064/300 Read block 4415 len 24576
26755 065/300 Write block 1209 len 12288 data: 14 10 8
26756 066/300 Flush
26757 067/300 Write block 3734 len 8192 data: 5 3
26758 068/300 Flush
26759 069/300 Write block 885 len 36864 data: 6 7 6 5 5 5 5 6 6
26760 070/300 Read block 3478 len 40960
26761 071/300 Read block 4669 len 4096
26762 072/300 Write block 1217 len 20480 data: 5 5 5 4 4
26763 073/300 Read block 1344 len 8192
26764 074/300 Read block 3874 len 8192
26765 075/300 Write block 1572 len 40960 data: 5 4 7 8 6 5 4 4 5 3
26766 076/300 Read block 3880 len 40960
26767 077/300 Write block 3849 len 20480 data: 7 9 9 10 10
26768 078/300 Read block 1654 len 32768
26769 079/300 Write block 4171 len 36864 data: 9 10 10 9 7 6 6 5 5
26770 080/300 Flush
26771 081/300 Write block 907 len 40960 data: 3 3 4 4 4 5 5 7 8 5
26772 082/300 Flush
26773 083/300 Flush
26774 084/300 Read block 4830 len 4096
26775 085/300 Read block 1306 len 20480
26776 086/300 Write block 2022 len 20480 data: 8 7 7 7 7
26777 087/300 Write block 3870 len 28672 data: 6 6 6 7 6 9 8
26778 088/300 Read block 873 len 28672
26779 089/300 Flush
26780 090/300 Write block 2149 len 32768 data: 5 5 4 5 5 5 5 4
26781 091/300 Write block 2780 len 12288 data: 5 5 6
26782 092/300 Read block 4245 len 20480
26783 093/300 Read block 1599 len 40960
26784 094/300 Read block 682 len 28672
26785 095/300 Write block 3241 len 32768 data: 4 4 4 5 6 5 5 4
26786 096/300 Read block 2620 len 32768
26787 097/300 Read block 948 len 24576
26788 098/300 Write block 411 len 36864 data: 9 8 8 8 11 11 11 11 10
26789 099/300 Read block 3393 len 20480
26790 100/300 Write block 3938 len 12288 data: 5 4 5
26791 101/300 Read block 2075 len 28672
26792 102/300 Write block 1616 len 8192 data: 5 6
26793 103/300 Read block 1598 len 36864
26794 104/300 Write block 3290 len 28672 data: 9 8 8 7 4 4 6
26795 105/300 Write block 4362 len 8192 data: 5 6
26796 106/300 Read block 2564 len 8192
26797 107/300 Read block 3914 len 28672
26798 108/300 Write block 2237 len 20480 data: 4 5 6 6 7
26799 109/300 Read block 3765 len 40960
26800 110/300 Write block 4658 len 12288 data: 8 8 7
26801 111/300 Write block 2098 len 28672 data: 5 5 4 3 3 3 3
26802 112/300 Read block 1650 len 8192
26803 113/300 Read block 4390 len 16384
26804 114/300 Read block 2143 len 4096
26805 115/300 Flush
26806 116/300 Read block 4689 len 16384
26807 117/300 Flush
26808 118/300 Write block 2407 len 16384 data: 5 4 4 2
26809 119/300 Flush
26810 120/300 Read block 471 len 36864
26811 121/300 Write block 2593 len 20480 data: 4 4 3 5 6
26812 122/300 Write block 2245 len 8192 data: 5 4
26813 123/300 Write block 1741 len 36864 data: 4 3 4 4 4 4 4 4 4
26814 124/300 Read block 2845 len 32768
26815 125/300 Write block 4450 len 36864 data: 6 6 7 6 7 8 11 12 10
26816 126/300 Read block 2080 len 36864
26817 127/300 Read block 4936 len 12288
26818 128/300 Read block 4920 len 36864
26819 129/300 Read block 785 len 20480
26820 130/300 Read block 3215 len 16384
26821 131/300 Write block 3113 len 12288 data: 6 6 6
26822 132/300 Write block 3003 len 4096 data: 6
26823 133/300 Read block 3930 len 8192
26824 134/300 Flush
26825 135/300 Read block 3708 len 8192
26826 136/300 Write block 3509 len 12288 data: 5 6 10
26827 137/300 Write block 4369 len 16384 data: 7 8 8 6
26828 138/300 Read block 3694 len 12288
26829 139/300 Write block 4095 len 20480 data: 7 7 10 9 8
26830 140/300 Read block 3319 len 12288
26831 141/300 Read block 2243 len 24576
26832 142/300 Read block 4338 len 16384
26833 143/300 Read block 3072 len 16384
26834 144/300 Read block 284 len 16384
26835 145/300 Write block 4558 len 40960 data: 5 7 6 6 7 7 6 5 4 4
26836 146/300 Read block 317 len 32768
26837 147/300 Read block 301 len 32768
26838 148/300 Read block 2493 len 16384
26839 149/300 Write block 4320 len 28672 data: 6 8 10 9 8 9 7
26840 150/300 Read block 4267 len 4096
26841 151/300 Read block 1813 len 4096
26842 152/300 Read block 3314 len 24576
26843 153/300 Read block 3239 len 16384
26844 154/300 Read block 1410 len 20480
26845 155/300 Read block 1122 len 24576
26846 156/300 Write block 3961 len 40960 data: 4 5 7 7 7 6 5 6 7 7
26847 157/300 Write block 1889 len 36864 data: 8 7 6 5 7 8 7 5 5
26848 158/300 Write block 1770 len 12288 data: 8 12 10
26849 159/300 Read block 152 len 16384
26850 160/300 Write block 2869 len 32768 data: 3 3 4 4 4 5 6 6
26851 161/300 Read block 1549 len 12288
26852 162/300 Flush
26853 163/300 Read block 4707 len 28672
26854 164/300 Read block 516 len 40960
26855 165/300 Flush
26856 166/300 Flush
26857 167/300 Read block 894 len 36864
26858 168/300 Write block 3615 len 40960 data: 7 5 6 6 3 3 3 4 4 4
26859 169/300 Read block 1015 len 8192
26860 170/300 Read block 333 len 8192
26861 171/300 Read block 1964 len 20480
26862 172/300 Write block 1744 len 40960 data: 5 5 5 5 5 5 7 7 7 8
26863 173/300 Write block 1528 len 24576 data: 7 5 3 3 3 5
26864 174/300 Flush
26865 175/300 Read block 4685 len 40960
26866 176/300 Read block 3522 len 12288
26867 177/300 Flush
26868 178/300 Write block 3401 len 8192 data: 5 5
26869 179/300 Write block 2574 len 36864 data: 9 10 9 10 8 7 6 5 4
26870 180/300 Read block 3205 len 24576
26871 181/300 Read block 3922 len 8192
26872 182/300 Read block 3559 len 24576
26873 183/300 Read block 3756 len 12288
26874 184/300 Write block 277 len 20480 data: 4 4 4 3 3
26875 185/300 Flush
26876 186/300 Write block 1436 len 24576 data: 8 6 5 6 3 4
26877 187/300 Read block 972 len 8192
26878 188/300 Read block 1278 len 28672
26879 189/300 Write block 2285 len 20480 data: 8 8 6 7 6
26880 190/300 Write block 4868 len 16384 data: 4 4 6 5
26881 191/300 Write block 4008 len 20480 data: 5 5 3 3 4
26882 192/300 Read block 3973 len 24576
26883 193/300 Read block 2596 len 24576
26884 194/300 Flush
26885 195/300 Write block 3654 len 36864 data: 6 6 7 5 3 6 7 7 7
26886 196/300 Read block 4328 len 36864
26887 197/300 Read block 1516 len 36864
26888 198/300 Read block 4985 len 12288
26889 199/300 Read block 851 len 24576
26890 200/300 Write block 442 len 8192 data: 3 3
26891 201/300 Flush
26892 202/300 Read block 4489 len 36864
26893 203/300 Flush
26894 204/300 Read block 3785 len 36864
26895 205/300 Write block 4138 len 4096 data: 7
26896 206/300 Read block 2578 len 12288
26897 207/300 Flush
26898 208/300 Write block 608 len 28672 data: 5 6 4 3 3 3 3
26899 209/300 Read block 2237 len 16384
26900 210/300 Write block 3563 len 32768 data: 5 6 5 5 5 5 5 5
26901 211/300 Write block 2804 len 20480 data: 4 4 7 7 6
26902 212/300 Read block 3843 len 24576
26903 213/300 Read block 4827 len 36864
26904 214/300 Read block 3580 len 28672
26905 215/300 Write block 879 len 16384 data: 7 8 9 7
26906 216/300 Read block 2485 len 16384
26907 217/300 Flush
26908 218/300 Read block 3276 len 40960
26909 219/300 Write block 4211 len 32768 data: 6 7 8 7 7 7 7 6
26910 220/300 Read block 2741 len 16384
26911 221/300 Write block 4291 len 16384 data: 6 7 9 9
26912 222/300 Write block 542 len 28672 data: 3 5 5 6 8 9 7
26913 223/300 Read block 826 len 32768
26914 224/300 Read block 697 len 16384
26915 225/300 Read block 3713 len 16384
26916 226/300 Read block 2792 len 40960
26917 227/300 Read block 1266 len 28672
26918 228/300 Flush
26919 229/300 Write block 4985 len 8192 data: 8 9
26920 230/300 Read block 4720 len 8192
26921 231/300 Write block 2362 len 16384 data: 7 7 8 8
26922 232/300 Read block 4875 len 24576
26923 233/300 Flush
26924 234/300 Write block 41 len 24576 data: 11 8 8 8 7 7
26925 235/300 Read block 706 len 12288
26926 236/300 Write block 2567 len 12288 data: 5 4 5
26927 237/300 Write block 440 len 16384 data: 2 3 4 4
26928 238/300 Write block 2714 len 20480 data: 5 5 5 5 6
26929 239/300 Write block 49 len 32768 data: 5 6 7 7 6 6 6 7
26930 240/300 Flush
26931 241/300 Read block 4440 len 8192
26932 242/300 Read block 4173 len 20480
26933 243/300 Read block 2421 len 20480
26934 244/300 Read block 839 len 40960
26935 245/300 Write block 1103 len 40960 data: 3 4 4 4 4 4 3 5 3 3
26936 246/300 Read block 1170 len 8192
26937 247/300 Read block 4584 len 28672
26938 248/300 Write block 967 len 28672 data: 4 4 4 5 4 4 5
26939 249/300 Write block 3415 len 28672 data: 5 6 4 5 6 6 6
26940 250/300 Read block 1577 len 4096
26941 251/300 Read block 3007 len 28672
26942 252/300 Write block 79 len 12288 data: 3 4 5
26943 253/300 Read block 4207 len 40960
26944 254/300 Write block 547 len 40960 data: 10 8 6 4 4 5 6 5 4 4
26945 255/300 Read block 4719 len 32768
26946 256/300 Write block 3359 len 36864 data: 2 3 2 2 2 3 4 6 8
26947 257/300 Write block 3271 len 12288 data: 7 5 5
26948 258/300 Read block 4605 len 20480
26949 259/300 Read block 4270 len 36864
26950 260/300 Read block 2029 len 32768
26951 261/300 Flush
26952 262/300 Write block 883 len 40960 data: 5 6 7 8 7 6 6 6 6 7
26953 263/300 Read block 662 len 16384
26954 264/300 Write block 3233 len 8192 data: 3 3
26955 265/300 Read block 1499 len 40960
26956 266/300 Flush
26957 267/300 Read block 2363 len 40960
26958 268/300 Write block 547 len 40960 data: 11 9 7 5 5 6 7 6 5 5
26959 269/300 Flush
26960 270/300 Flush
26961 271/300 Read block 2344 len 36864
26962 272/300 Write block 3347 len 28672 data: 5 3 2 2 3 5 5
26963 273/300 Read block 3156 len 8192
26964 274/300 Read block 3567 len 36864
26965 275/300 Read block 1630 len 40960
26966 276/300 Write block 4931 len 28672 data: 6 7 6 6 5 5 5
26967 277/300 Flush
26968 278/300 Write block 2292 len 32768 data: 4 4 5 6 6 4 4 7
26969 279/300 Write block 313 len 16384 data: 4 4 4 4
26970 280/300 Read block 3939 len 8192
26971 281/300 Write block 4619 len 20480 data: 5 5 6 6 5
26972 282/300 Flush
26973 283/300 Read block 357 len 4096
26974 284/300 Read block 3889 len 32768
26975 285/300 Read block 3682 len 12288
26976 286/300 Read block 4535 len 20480
26977 287/300 Read block 4602 len 40960
26978 288/300 Read block 2384 len 28672
26979 289/300 Write block 2171 len 20480 data: 4 5 2 2 2
26980 290/300 Read block 4004 len 24576
26981 291/300 Read block 3011 len 40960
26982 292/300 Write block 2470 len 36864 data: 7 8 9 8 7 10 10 9 8
26983 293/300 Write block 2595 len 4096 data: 4
26984 294/300 Write block 894 len 32768 data: 6 6 6 6 4 4 4 4
26985 295/300 Read block 4927 len 40960
26986 296/300 Flush
26987 297/300 Write block 3088 len 20480 data: 3 5 5 6 6
26988 298/300 Write block 4065 len 8192 data: 4 5
26989 299/300 Read block 4553 len 16384
26990 300/300 Read block 2178 len 32768
26991 Main task has work to do, go find it
26992 got dsc Start(1)
26993 start 1
26994 Sep 22 23:29:42.621 INFO request completed, latency_us: 312, response_code: 204, uri: /start/cid/1, method: POST, req_id: 2426f44b-d77e-423e-a241-ab8c6a82e088, remote_addr: 127.0.0.1:52471, local_addr: 127.0.0.1:9998
26995 Replay: started 1, returned:Ok(())
26996 ----------------------------------------------------------------
26997 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26998 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
26999 8772 Acked 9771 Flush 0 Done New Done false
27000 8773 Acked 9772 Read 1 Done New Done false
27001 8774 Acked 9773 Write 8 Done New Done false
27002 8775 Acked 9774 Write 2 Done New Done false
27003 8776 Acked 9775 Read 7 Done New Done false
27004 8777 Acked 9776 Read 7 Done New Done false
27005 8778 Acked 9777 Write 2 Done New Done false
27006 8779 Acked 9778 Flush 0 Done New Done false
27007 8780 Acked 9779 Write 7 Done New Done false
27008 8781 Acked 9780 Write 4 Done New Done false
27009 8782 Acked 9781 Write 1 Done New Done false
27010 8783 Acked 9782 Read 3 Done New Done false
27011 8784 Acked 9783 Read 8 Done New Done false
27012 8785 Acked 9784 Read 6 Done New Done false
27013 8786 Acked 9785 Read 5 Done New Done false
27014 8787 Acked 9786 Write 2 Done New Done false
27015 8788 Acked 9787 Write 4 Done New Done false
27016 8789 Acked 9788 Flush 0 Done New Done false
27017 8790 Acked 9789 Write 2 Done New Done false
27018 8791 Acked 9790 Write 7 Done New Done false
27019 8792 Acked 9791 Read 5 Done New Done false
27020 8793 Acked 9792 Read 3 Done New Done false
27021 8794 Acked 9793 Write 5 Done New Done false
27022 8795 Acked 9794 Write 1 Done New Done false
27023 8796 Acked 9795 Flush 0 Done New Done false
27024 8797 Acked 9796 Write 7 Done New Done false
27025 8798 Acked 9797 Read 1 Done New Done false
27026 8799 Acked 9798 Write 7 Done New Done false
27027 8800 Acked 9799 Read 9 Done New Done false
27028 8801 Acked 9800 Write 5 Done New Done false
27029 8802 Acked 9801 Write 10 Done New Done false
27030 8803 Acked 9802 Read 6 Done New Done false
27031 8804 Acked 9803 Read 9 Done New Done false
27032 8805 Acked 9804 Write 8 Done New Done false
27033 8806 Acked 9805 Write 3 Done New Done false
27034 8807 Acked 9806 Write 2 Done New Done false
27035 8808 Acked 9807 Write 3 Done New Done false
27036 8809 Acked 9808 Read 4 Done New Done false
27037 8810 Acked 9809 Write 3 Done New Done false
27038 8811 Acked 9810 Write 7 Done New Done false
27039 8812 Acked 9811 Read 9 Done New Done false
27040 8813 Acked 9812 Read 5 Done New Done false
27041 8814 Acked 9813 Write 1 Done New Done false
27042 8815 Acked 9814 Read 6 Done New Done false
27043 8816 Acked 9815 Read 5 Done New Done false
27044 8817 Acked 9816 Flush 0 Done New Done false
27045 8818 Acked 9817 Read 8 Done New Done false
27046 8819 Acked 9818 Read 8 Done New Done false
27047 8820 Acked 9819 Flush 0 Done New Done false
27048 8821 Acked 9820 Write 2 Done New Done false
27049 8822 Acked 9821 Flush 0 Done New Done false
27050 8823 Acked 9822 Read 9 Done New Done false
27051 8824 Acked 9823 Flush 0 Done New Done false
27052 8825 Acked 9824 Read 9 Done New Done false
27053 8826 Acked 9825 Read 2 Done New Done false
27054 8827 Acked 9826 Read 7 Done New Done false
27055 8828 Acked 9827 Read 2 Done New Done false
27056 8829 Acked 9828 Read 1 Done New Done false
27057 8830 Acked 9829 Read 9 Done New Done false
27058 8831 Acked 9830 Write 7 Done New Done false
27059 8832 Acked 9831 Write 3 Done New Done false
27060 8833 Acked 9832 Flush 0 Done New Done false
27061 8834 Acked 9833 Flush 0 Done New Done false
27062 8835 Acked 9834 Read 6 Done New Done false
27063 8836 Acked 9835 Write 3 Done New Done false
27064 8837 Acked 9836 Flush 0 Done New Done false
27065 8838 Acked 9837 Write 2 Done New Done false
27066 8839 Acked 9838 Flush 0 Done New Done false
27067 8840 Acked 9839 Write 9 Done New Done false
27068 8841 Acked 9840 Read 10 Done New Done false
27069 8842 Acked 9841 Read 1 Done New Done false
27070 8843 Acked 9842 Write 5 Done New Done false
27071 8844 Acked 9843 Read 2 Done New Done false
27072 8845 Acked 9844 Read 2 Done New Done false
27073 8846 Acked 9845 Write 10 Done New Done false
27074 8847 Acked 9846 Read 10 Done New Done false
27075 8848 Acked 9847 Write 5 Done New Done false
27076 8849 Acked 9848 Read 8 Done New Done false
27077 8850 Acked 9849 Write 9 Done New Done false
27078 8851 Acked 9850 Flush 0 Done New Done false
27079 8852 Acked 9851 Write 10 Done New Done false
27080 8853 Acked 9852 Flush 0 Done New Done false
27081 8854 Acked 9853 Flush 0 Done New Done false
27082 8855 Acked 9854 Read 1 Done New Done false
27083 8856 Acked 9855 Read 5 Done New Done false
27084 8857 Acked 9856 Write 5 Done New Done false
27085 8858 Acked 9857 Write 7 Done New Done false
27086 8859 Acked 9858 Read 7 Done New Done false
27087 8860 Acked 9859 Flush 0 Done New Done false
27088 8861 Acked 9860 Write 8 Done New Done false
27089 8862 Acked 9861 Write 3 Done New Done false
27090 8863 Acked 9862 Read 5 Done New Done false
27091 8864 Acked 9863 Read 10 Done New Done false
27092 8865 Acked 9864 Read 7 Done New Done false
27093 8866 Acked 9865 Write 8 Done New Done false
27094 8867 Acked 9866 Read 8 Done New Done false
27095 8868 Acked 9867 Read 6 Done New Done false
27096 8869 Acked 9868 Write 9 Done New Done false
27097 8870 Acked 9869 Read 5 Done New Done false
27098 8871 Acked 9870 Write 3 Done New Done false
27099 8872 Acked 9871 Read 7 Done New Done false
27100 8873 Acked 9872 Write 2 Done New Done false
27101 8874 Acked 9873 Read 9 Done New Done false
27102 8875 Acked 9874 Write 7 Done New Done false
27103 8876 Acked 9875 Write 2 Done New Done false
27104 8877 Acked 9876 Read 2 Done New Done false
27105 8878 Acked 9877 Read 7 Done New Done false
27106 8879 Acked 9878 Write 5 Done New Done false
27107 8880 Acked 9879 Read 10 Done New Done false
27108 8881 Acked 9880 Write 3 Done New Done false
27109 8882 Acked 9881 Write 7 Done New Done false
27110 8883 Acked 9882 Read 2 Done New Done false
27111 8884 Acked 9883 Read 4 Done New Done false
27112 8885 Acked 9884 Read 1 Done New Done false
27113 8886 Acked 9885 Flush 0 Done New Done false
27114 8887 Acked 9886 Read 4 Done New Done false
27115 8888 Acked 9887 Flush 0 Done New Done false
27116 8889 Acked 9888 Write 4 Done New Done false
27117 8890 Acked 9889 Flush 0 Done New Done false
27118 8891 Acked 9890 Read 9 Done New Done false
27119 8892 Acked 9891 Write 5 Done New Done false
27120 8893 Acked 9892 Write 2 Done New Done false
27121 8894 Acked 9893 Write 9 Done New Done false
27122 8895 Acked 9894 Read 8 Done New Done false
27123 8896 Acked 9895 Write 9 Done New Done false
27124 8897 Acked 9896 Read 9 Done New Done false
27125 8898 Acked 9897 Read 3 Done New Done false
27126 8899 Acked 9898 Read 9 Done New Done false
27127 8900 Acked 9899 Read 5 Done New Done false
27128 8901 Acked 9900 Read 4 Done New Done false
27129 8902 Acked 9901 Write 3 Done New Done false
27130 8903 Acked 9902 Write 1 Done New Done false
27131 8904 Acked 9903 Read 2 Done New Done false
27132 8905 Acked 9904 Flush 0 Done New Done false
27133 8906 Acked 9905 Read 2 Done New Done false
27134 8907 Acked 9906 Write 3 Done New Done false
27135 8908 Acked 9907 Write 4 Done New Done false
27136 8909 Acked 9908 Read 3 Done New Done false
27137 8910 Acked 9909 Write 5 Done New Done false
27138 8911 Acked 9910 Read 3 Done New Done false
27139 8912 Acked 9911 Read 6 Done New Done false
27140 8913 Acked 9912 Read 4 Done New Done false
27141 8914 Acked 9913 Read 4 Done New Done false
27142 8915 Acked 9914 Read 4 Done New Done false
27143 8916 Acked 9915 Write 10 Done New Done false
27144 8917 Acked 9916 Read 8 Done New Done false
27145 8918 Acked 9917 Read 8 Done New Done false
27146 8919 Acked 9918 Read 4 Done New Done false
27147 8920 Acked 9919 Write 7 Done New Done false
27148 8921 Acked 9920 Read 1 Done New Done false
27149 8922 Acked 9921 Read 1 Done New Done false
27150 8923 Acked 9922 Read 6 Done New Done false
27151 8924 Acked 9923 Read 4 Done New Done false
27152 8925 Acked 9924 Read 5 Done New Done false
27153 8926 Acked 9925 Read 6 Done New Done false
27154 8927 Acked 9926 Write 10 Done New Done false
27155 8928 Acked 9927 Write 9 Done New Done false
27156 8929 Acked 9928 Write 3 Done New Done false
27157 8930 Acked 9929 Read 4 Done New Done false
27158 8931 Acked 9930 Write 8 Done New Done false
27159 8932 Acked 9931 Read 3 Done New Done false
27160 8933 Acked 9932 Flush 0 Done New Done false
27161 8934 Acked 9933 Read 7 Done New Done false
27162 8935 Acked 9934 Read 10 Done New Done false
27163 8936 Acked 9935 Flush 0 Done New Done false
27164 8937 Acked 9936 Flush 0 Done New Done false
27165 8938 Acked 9937 Read 9 Done New Done false
27166 8939 Acked 9938 Write 10 Done New Done false
27167 8940 Acked 9939 Read 2 Done New Done false
27168 8941 Acked 9940 Read 2 Done New Done false
27169 8942 Acked 9941 Read 5 Done New Done false
27170 8943 Acked 9942 Write 10 Done New Done false
27171 8944 Acked 9943 Write 6 Done New Done false
27172 8945 Acked 9944 Flush 0 Done New Done false
27173 8946 Acked 9945 Read 10 Done New Done false
27174 8947 Acked 9946 Read 3 Done New Done false
27175 8948 Acked 9947 Flush 0 Done New Done false
27176 8949 Acked 9948 Write 2 Done New Done false
27177 8950 Acked 9949 Write 9 Done New Done false
27178 8951 Acked 9950 Read 6 Done New Done false
27179 8952 Acked 9951 Read 2 Done New Done false
27180 8953 Acked 9952 Read 6 Done New Done false
27181 8954 Acked 9953 Read 3 Done New Done false
27182 8955 Acked 9954 Write 5 Done New Done false
27183 8956 Acked 9955 Flush 0 Done New Done false
27184 8957 Acked 9956 Write 6 Done New Done false
27185 8958 Acked 9957 Read 2 Done New Done false
27186 8959 Acked 9958 Read 7 Done New Done false
27187 8960 Acked 9959 Flush 0 Done New Done false
27188 8961 Acked 9960 Write 5 Done New Done false
27189 8962 Acked 9961 Write 4 Done New Done false
27190 8963 Acked 9962 Write 5 Done New Done false
27191 8964 Acked 9963 Read 6 Done New Done false
27192 8965 Acked 9964 Read 6 Done New Done false
27193 8966 Acked 9965 Flush 0 Done New Done false
27194 8967 Acked 9966 Write 9 Done New Done false
27195 8968 Acked 9967 Read 9 Done New Done false
27196 8969 Acked 9968 Read 9 Done New Done false
27197 8970 Acked 9969 Read 3 Done New Done false
27198 8971 Acked 9970 Read 6 Done New Done false
27199 8972 Acked 9971 Write 2 Done New Done false
27200 8973 Acked 9972 Flush 0 Done New Done false
27201 8974 Acked 9973 Read 9 Done New Done false
27202 8975 Acked 9974 Flush 0 Done New Done false
27203 8976 Acked 9975 Read 9 Done New Done false
27204 8977 Acked 9976 Write 1 Done New Done false
27205 8978 Acked 9977 Read 3 Done New Done false
27206 8979 Acked 9978 Flush 0 Done New Done false
27207 8980 Acked 9979 Write 7 Done New Done false
27208 8981 Acked 9980 Read 4 Done New Done false
27209 8982 Acked 9981 Write 8 Done New Done false
27210 8983 Acked 9982 Write 5 Done New Done false
27211 8984 Acked 9983 Read 6 Done New Done false
27212 8985 Acked 9984 Read 9 Done New Done false
27213 8986 Acked 9985 Read 7 Done New Done false
27214 8987 Acked 9986 Write 4 Done New Done false
27215 8988 Acked 9987 Read 4 Done New Done false
27216 8989 Acked 9988 Flush 0 Done New Done false
27217 8990 Acked 9989 Read 10 Done New Done false
27218 8991 Acked 9990 Write 8 Done New Done false
27219 8992 Acked 9991 Read 4 Done New Done false
27220 8993 Acked 9992 Write 4 Done New Done false
27221 8994 Acked 9993 Write 7 Done New Done false
27222 8995 Acked 9994 Read 8 Done New Done false
27223 8996 Acked 9995 Read 4 Done New Done false
27224 8997 Acked 9996 Read 4 Done New Done false
27225 8998 Acked 9997 Read 10 Done New Done false
27226 8999 Acked 9998 Read 7 Done New Done false
27227 9000 Acked 9999 Flush 0 Done New Done false
27228 9001 Acked 10000 Write 2 Done New Done false
27229 9002 Acked 10001 Read 2 Done New Done false
27230 9003 Acked 10002 Write 4 Done New Done false
27231 9004 Acked 10003 Read 6 Done New Done false
27232 9005 Acked 10004 Flush 0 Done New Done false
27233 9006 Acked 10005 Write 6 Done New Done false
27234 9007 Acked 10006 Read 3 Done New Done false
27235 9008 Acked 10007 Write 3 Done New Done false
27236 9009 Acked 10008 Write 4 Done New Done false
27237 9010 Acked 10009 Write 5 Done New Done false
27238 9011 Acked 10010 Write 8 Done New Done false
27239 9012 Acked 10011 Flush 0 Done New Done false
27240 9013 Acked 10012 Read 2 Done New Done false
27241 9014 Acked 10013 Read 5 Done New Done false
27242 9015 Acked 10014 Read 5 Done New Done false
27243 9016 Acked 10015 Read 10 Done New Done false
27244 9017 Acked 10016 Write 10 Done New Done false
27245 9018 Acked 10017 Read 2 Done New Done false
27246 9019 Acked 10018 Read 7 Done New Done false
27247 9020 Acked 10019 Write 7 Done New Done false
27248 9021 Acked 10020 Write 7 Done New Done false
27249 9022 Acked 10021 Read 1 Done New Done false
27250 9023 Acked 10022 Read 7 Done New Done false
27251 9024 Acked 10023 Write 3 Done New Done false
27252 9025 Acked 10024 Read 10 Done New Done false
27253 9026 Acked 10025 Write 10 Done New Done false
27254 9027 Acked 10026 Read 8 Done New Done false
27255 9028 Acked 10027 Write 9 Done New Done false
27256 9029 Acked 10028 Write 3 Done New Done false
27257 9030 Acked 10029 Read 5 Done New Done false
27258 9031 Acked 10030 Read 9 Done New Done false
27259 9032 Acked 10031 Read 8 Done New Done false
27260 9033 Acked 10032 Flush 0 Done New Done false
27261 9034 Acked 10033 Write 10 Done New Done false
27262 9035 Acked 10034 Read 4 Done New Done false
27263 9036 Acked 10035 Write 2 Done New Done false
27264 9037 Acked 10036 Read 10 Done New Done false
27265 9038 Acked 10037 Flush 0 Done New Done false
27266 9039 Acked 10038 Read 10 Done New Done false
27267 9040 Acked 10039 Write 10 Done New Done false
27268 9041 Acked 10040 Flush 0 Done New Done false
27269 9042 Acked 10041 Flush 0 Done New Done false
27270 9043 Acked 10042 Read 9 Done New Done false
27271 9044 Acked 10043 Write 7 Done New Done false
27272 9045 Acked 10044 Read 2 Done New Done false
27273 9046 Acked 10045 Read 9 Done New Done false
27274 9047 Acked 10046 Read 10 Done New Done false
27275 9048 Acked 10047 Write 7 Done New Done false
27276 9049 Acked 10048 Flush 0 Done New Done false
27277 9050 Acked 10049 Write 8 Done New Done false
27278 9051 Acked 10050 Write 4 Done New Done false
27279 9052 Acked 10051 Read 2 Done New Done false
27280 9053 Acked 10052 Write 5 Done New Done false
27281 9054 Acked 10053 Flush 0 Done New Done false
27282 9055 Acked 10054 Read 1 Done New Done false
27283 9056 Acked 10055 Read 8 Done New Done false
27284 9057 Acked 10056 Read 3 Done New Done false
27285 9058 Acked 10057 Read 5 Done New Done false
27286 9059 Acked 10058 Read 10 Done New Done false
27287 9060 Acked 10059 Read 7 Done New Done false
27288 9061 Acked 10060 Write 5 Done New Done false
27289 9062 Acked 10061 Read 6 Done New Done false
27290 9063 Acked 10062 Read 10 Done New Done false
27291 9064 Acked 10063 Write 9 Done New Done false
27292 9065 Acked 10064 Write 1 Done New Done false
27293 9066 Acked 10065 Write 8 Done New Done false
27294 9067 Acked 10066 Read 10 Done New Done false
27295 9068 Acked 10067 Flush 0 Done New Done false
27296 9069 Acked 10068 Write 5 Done New Done false
27297 9070 Acked 10069 Write 2 Done New Done false
27298 9071 Acked 10070 Read 4 Done New Done false
27299 9072 Acked 10071 Read 8 Done New Done false
27300 STATES DS:0 DS:1 DS:2 TOTAL
27301 New 0 301 0 301
27302 Sent 0 0 0 0
27303 Done 301 0 301 602
27304 Skipped 0 0 0 0
27305 Error 0 0 0 0
27306 Last Flush: 10067 9770 10067
27307 Downstairs last five completed: 9770 9769 9768 9767 9766 9765
27308 Upstairs last five completed: 9072 9071 9070 9069 9068 9067
27309 CLIENT: Up:0 ds:301 act:2
27310 [8820] got start action
27311 [8820] I am going to restart
27312 Starting downstairs at port 8820
27313 Make output file at "/tmp/dsc/downstairs-8820.txt"
27314 [8820][1] reports Starting
27315 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1203)
27316 [8820][1] reports Running
273172023-09-22T23:29:44.672ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 looper connected looper = 1
273182023-09-22T23:29:44.672ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state Offline
273192023-09-22T23:29:44.672ZINFOcrucible: [1] client re-new 302 jobs since flush 9770 = downstairs
273202023-09-22T23:29:44.673ZINFOcrucible: [1] upstairs guest_io_ready=TRUE, promote! session 09e58a14-b4d7-4418-a55e-529861aa483b
273212023-09-22T23:29:44.674ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
273222023-09-22T23:29:44.674ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
273232023-09-22T23:29:44.674ZINFOcrucible: Returning client:1 UUID:12345678-0000-0000-0000-000000008820 matches
273242023-09-22T23:29:44.674ZINFOcrucible: [1] send last flush ID to this DS: 9770
273252023-09-22T23:29:44.674ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Offline Active
273262023-09-22T23:29:44.674ZINFOcrucible: [1] Replied this last flush ID: 9770
273272023-09-22T23:29:44.674ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Offline Active ds_transition to Replay
273282023-09-22T23:29:44.674ZINFOcrucible: [1] Transition from Offline to Replay
273292023-09-22T23:29:44.674ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Transition from Replay to Active
273302023-09-22T23:29:44.674ZINFOcrucible: [1] ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 (09e58a14-b4d7-4418-a55e-529861aa483b) Active Replay Active ds_transition to Active
273312023-09-22T23:29:44.674ZINFOcrucible: [1] Transition from Replay to Active
273322023-09-22T23:29:44.674ZINFOcrucible: [1] Starts cmd_loop
273332023-09-22T23:29:44.674ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
273342023-09-22T23:29:44.674ZINFOcrucible: ae4f1b6e-d048-4998-b3fd-1c2ac78b4697 Active Active Active
273352023-09-22T23:29:44.674ZINFOcrucible: Set check for repair
273362023-09-22T23:29:45.675ZWARNcrucible: [1] flow control end
273372023-09-22T23:29:45.676ZINFOcrucible: Checking if live repair is needed
273382023-09-22T23:29:45.676ZINFOcrucible: No Live Repair required at this time
27339 ----------------------------------------------------------------
27340 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27341 Downstairs last five completed: 10072 10071 10070 10069 10068 10067
27342 Upstairs last five completed: 9073 9072 9071 9070 9069 9068
27343 CLIENT: Up:0 ds:0 act:3
27344 Replay: All jobs finished, all DS active.
27345 Test replay has completed
27346 Wrote out file "alan"
27347 CLIENT: Tests done. All submitted work has been ACK'd
27348 ----------------------------------------------------------------
27349 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27350 Downstairs last five completed: 10072 10071 10070 10069 10068 10067
27351 Upstairs last five completed: 9073 9072 9071 9070 9069 9068
27352 CLIENT: Up:0 ds:0 act:3
27353 CLIENT: All jobs finished, all DS active.
27354 Replays:30 time: 5:57
27355 Do final verify
273562023-09-22T23:29:46.671ZINFOcrucible: Upstairs starts
273572023-09-22T23:29:46.671ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "ed48f294784d46ea7d4bb99336918b74358eca46", git_commit_timestamp: "2023-09-22T22:51:18.000000000Z", git_branch: "main", rustc_semver: "1.70.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "90c541806f23a127002de5b4038be731ba1458ca", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
273582023-09-22T23:29:46.671ZINFOcrucible: Upstairs <-> Downstairs Message Version: 4
273592023-09-22T23:29:46.671ZINFOcrucible: Crucible stats registered with UUID: 7880d5bb-7a5d-488f-9de5-7debc45231eb
273602023-09-22T23:29:46.671ZINFOcrucible: Crucible 7880d5bb-7a5d-488f-9de5-7debc45231eb has session id: cf74bff2-5192-4d01-ba38-6da5e3d514d9
27361 Crucible runtime is spawned
27362 The guest has requested activation with gen:3
273632023-09-22T23:29:46.672ZINFOcrucible: up_listen starts task = up_listen
273642023-09-22T23:29:46.672ZINFOcrucible: Wait for all three downstairs to come online
273652023-09-22T23:29:46.672ZINFOcrucible: Flush timeout: 0.5
273662023-09-22T23:29:46.672ZINFOcrucible: [2] connecting to 127.0.0.1:8830 looper = 2
273672023-09-22T23:29:46.672ZINFOcrucible: [1] connecting to 127.0.0.1:8820 looper = 1
273682023-09-22T23:29:46.672ZINFOcrucible: [2] 7880d5bb-7a5d-488f-9de5-7debc45231eb looper connected looper = 2
273692023-09-22T23:29:46.672ZINFOcrucible: [0] connecting to 127.0.0.1:8810 looper = 0
273702023-09-22T23:29:46.672ZINFOcrucible: [1] 7880d5bb-7a5d-488f-9de5-7debc45231eb looper connected looper = 1
273712023-09-22T23:29:46.672ZINFOcrucible: [0] 7880d5bb-7a5d-488f-9de5-7debc45231eb looper connected looper = 0
273722023-09-22T23:29:46.673ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb active request set
273732023-09-22T23:29:46.673ZINFOcrucible: Set desired generation to :3
273742023-09-22T23:29:46.673ZINFOcrucible: [2] Proc runs for 127.0.0.1:8830 in state New
273752023-09-22T23:29:46.673ZINFOcrucible: [1] Proc runs for 127.0.0.1:8820 in state New
273762023-09-22T23:29:46.674ZINFOcrucible: [0] Proc runs for 127.0.0.1:8810 in state New
273772023-09-22T23:29:46.674ZINFOcrucible: [1] 7880d5bb-7a5d-488f-9de5-7debc45231eb (bf2320ae-21d2-4dc0-9653-5b47dcb99ddf) New New New ds_transition to WaitActive
273782023-09-22T23:29:46.674ZINFOcrucible: [1] Transition from New to WaitActive
273792023-09-22T23:29:46.674ZINFOcrucible: [2] 7880d5bb-7a5d-488f-9de5-7debc45231eb (bf2320ae-21d2-4dc0-9653-5b47dcb99ddf) New WaitActive New ds_transition to WaitActive
273802023-09-22T23:29:46.674ZINFOcrucible: [2] Transition from New to WaitActive
273812023-09-22T23:29:46.674ZINFOcrucible: [1] client is_active_req TRUE, promote! session bf2320ae-21d2-4dc0-9653-5b47dcb99ddf
273822023-09-22T23:29:46.674ZINFOcrucible: [2] client is_active_req TRUE, promote! session bf2320ae-21d2-4dc0-9653-5b47dcb99ddf
273832023-09-22T23:29:46.674ZINFOcrucible: [0] 7880d5bb-7a5d-488f-9de5-7debc45231eb (bf2320ae-21d2-4dc0-9653-5b47dcb99ddf) New WaitActive WaitActive ds_transition to WaitActive
273842023-09-22T23:29:46.674ZINFOcrucible: [0] Transition from New to WaitActive
273852023-09-22T23:29:46.674ZINFOcrucible: [0] client is_active_req TRUE, promote! session bf2320ae-21d2-4dc0-9653-5b47dcb99ddf
273862023-09-22T23:29:46.675ZINFOcrucible: [1] downstairs client at 127.0.0.1:8820 has UUID 12345678-0000-0000-0000-000000008820
273872023-09-22T23:29:46.675ZINFOcrucible: [2] downstairs client at 127.0.0.1:8830 has UUID 12345678-0000-0000-0000-000000008830
273882023-09-22T23:29:46.675ZINFOcrucible: [1] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008820, encrypted: false, database_read_version: 1, database_write_version: 1 }
273892023-09-22T23:29:46.675ZINFOcrucible: [2] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008830, encrypted: false, database_read_version: 1, database_write_version: 1 }
273902023-09-22T23:29:46.675ZINFOcrucible: [0] downstairs client at 127.0.0.1:8810 has UUID 12345678-0000-0000-0000-000000008810
273912023-09-22T23:29:46.675ZINFOcrucible: [0] Got region def RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 12345678-0000-0000-0000-000000008810, encrypted: false, database_read_version: 1, database_write_version: 1 }
273922023-09-22T23:29:46.675ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb WaitActive WaitActive WaitActive
273932023-09-22T23:29:46.675ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb WaitActive WaitActive WaitActive
273942023-09-22T23:29:46.675ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb WaitActive WaitActive WaitActive
273952023-09-22T23:29:46.679ZINFOcrucible: [1] 7880d5bb-7a5d-488f-9de5-7debc45231eb (bf2320ae-21d2-4dc0-9653-5b47dcb99ddf) WaitActive WaitActive WaitActive ds_transition to WaitQuorum
273962023-09-22T23:29:46.679ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum
273972023-09-22T23:29:46.679ZWARNcrucible: [1] new RM replaced this: None
273982023-09-22T23:29:46.679ZINFOcrucible: [2] 7880d5bb-7a5d-488f-9de5-7debc45231eb (bf2320ae-21d2-4dc0-9653-5b47dcb99ddf) WaitActive WaitQuorum WaitActive ds_transition to WaitQuorum
273992023-09-22T23:29:46.679ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum
274002023-09-22T23:29:46.679ZWARNcrucible: [2] new RM replaced this: None
274012023-09-22T23:29:46.679ZINFOcrucible: [1] 127.0.0.1:8820 task reports connection:true
274022023-09-22T23:29:46.679ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb WaitActive WaitQuorum WaitQuorum
274032023-09-22T23:29:46.679ZINFOcrucible: [0] 7880d5bb-7a5d-488f-9de5-7debc45231eb (bf2320ae-21d2-4dc0-9653-5b47dcb99ddf) WaitActive WaitQuorum WaitQuorum ds_transition to WaitQuorum
274042023-09-22T23:29:46.679ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum
274052023-09-22T23:29:46.679ZWARNcrucible: [0] new RM replaced this: None
274062023-09-22T23:29:46.679ZINFOcrucible: [1] Starts reconcile loop
274072023-09-22T23:29:46.679ZINFOcrucible: [2] Starts reconcile loop
274082023-09-22T23:29:46.679ZINFOcrucible: [0]R flush_numbers[0..12]: [947, 915, 937, 952, 946, 949, 943, 899, 953, 953, 915, 947]
274092023-09-22T23:29:46.679ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
274102023-09-22T23:29:46.679ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
274112023-09-22T23:29:46.679ZINFOcrucible: [1]R flush_numbers[0..12]: [947, 915, 937, 952, 946, 949, 943, 899, 953, 953, 915, 947]
274122023-09-22T23:29:46.679ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
274132023-09-22T23:29:46.679ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
274142023-09-22T23:29:46.679ZINFOcrucible: [2]R flush_numbers[0..12]: [947, 915, 937, 952, 946, 949, 943, 899, 953, 953, 915, 947]
274152023-09-22T23:29:46.679ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2]
274162023-09-22T23:29:46.680ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]
274172023-09-22T23:29:46.680ZINFOcrucible: Max found gen is 3
274182023-09-22T23:29:46.680ZINFOcrucible: Generation requested: 3 >= found:3
27419 The guest has finished waiting for activation with:3
274202023-09-22T23:29:46.680ZINFOcrucible: Next flush: 955
274212023-09-22T23:29:46.680ZINFOcrucible: All extents match
27422 {"msg":"No downstairs repair required","v":0,"name":"crucible","level":30Wait for a query_work_queue command to finish before sending IO
27423 ,"time":"2023-09-22T23:29:46.680217988Z","hostname":"ip-10-150-1-70.us-west-2.compute.internal","pid":1208}
274242023-09-22T23:29:46.680ZINFOcrucible: [0] Starts reconcile loop
274252023-09-22T23:29:46.680ZINFOcrucible: No initial repair work was required
274262023-09-22T23:29:46.680ZINFOcrucible: Set Downstairs and Upstairs active
274272023-09-22T23:29:46.680ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb is now active with session: bf2320ae-21d2-4dc0-9653-5b47dcb99ddf
274282023-09-22T23:29:46.680ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb Set Active after no repair
274292023-09-22T23:29:46.680ZINFOcrucible: Notify all downstairs, region set compare is done.
274302023-09-22T23:29:46.680ZINFOcrucible: Set check for repair
274312023-09-22T23:29:46.680ZINFOcrucible: [2] 127.0.0.1:8830 task reports connection:true
274322023-09-22T23:29:46.680ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb Active Active Active
274332023-09-22T23:29:46.680ZINFOcrucible: Set check for repair
274342023-09-22T23:29:46.680ZINFOcrucible: [1] received reconcile message
274352023-09-22T23:29:46.680ZINFOcrucible: [1] All repairs completed, exit
274362023-09-22T23:29:46.680ZINFOcrucible: [0] 127.0.0.1:8810 task reports connection:true
274372023-09-22T23:29:46.680ZINFOcrucible: [1] Starts cmd_loop
274382023-09-22T23:29:46.680ZINFOcrucible: [0] received reconcile message
274392023-09-22T23:29:46.680ZINFOcrucible: 7880d5bb-7a5d-488f-9de5-7debc45231eb Active Active Active
274402023-09-22T23:29:46.680ZINFOcrucible: Set check for repair
274412023-09-22T23:29:46.680ZINFOcrucible: [0] All repairs completed, exit
274422023-09-22T23:29:46.680ZINFOcrucible: [0] Starts cmd_loop
274432023-09-22T23:29:46.680ZINFOcrucible: [2] received reconcile message
274442023-09-22T23:29:46.680ZINFOcrucible: [2] All repairs completed, exit
274452023-09-22T23:29:46.681ZINFOcrucible: [2] Starts cmd_loop
27446 Region: es:100 ec:50 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
27447 Loading write count information from file "alan"
27448 Read and Verify all blocks (0..5000 range:false)
274492023-09-22T23:29:47.681ZINFOcrucible: Checking if live repair is needed
274502023-09-22T23:29:47.681ZINFOcrucible: No Live Repair required at this time
27451 Wrote out file "alan"
27452 Verify test completed
27453 Wrote out file "alan"
27454 CLIENT: Tests done. All submitted work has been ACK'd
27455 ----------------------------------------------------------------
27456 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:2
27457 GW_ID ACK DSID TYPE BLOCKS DS:0 DS:1 DS:2 REPLAY
27458 53 Acked 1052 Read 100 Done Done Done false
27459 54 Acked 1053 Read 100 Done Done Done false
27460 STATES DS:0 DS:1 DS:2 TOTAL
27461 New 0 0 0 0
27462 Sent 0 0 0 0
27463 Done 2 2 2 6
27464 Skipped 0 0 0 0
27465 Error 0 0 0 0
27466 Last Flush: 1051 1051 1051
27467 Downstairs last five completed: 1051 1050 1049 1048 1047 1046
27468 Upstairs last five completed: 54 53 52 51 50 49
27469 CLIENT: Up:0 ds:2 act:3
27470 ----------------------------------------------------------------
27471 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
27472 Downstairs last five completed: 1054 1053 1052 1051 1050 1049
27473 Upstairs last five completed: 55 54 53 52 51 50
27474 CLIENT: Up:0 ds:0 act:3
27475 CLIENT: All crucible jobs finished, exiting program
27476 Sep 22 23:29:52.898 INFO accepted connection, remote_addr: 127.0.0.1:45594, local_addr: 127.0.0.1:9998
27477 Main task has work to do, go find it
27478 got dsc Shutdown
27479 Shutdown
27480 Shut it down
27481 Sep 22 23:29:52.900 INFO[8810] Disable keep_running
27482 [8830] Disable keep_running
27483 request completed, latency_us: 263, response_code: 204, uri: /shutdown, method: POST, req_id: 7b343f87-8e21-47e8-a86a-4839e50c2c63, [8830] Got stop action so:false kr:false
27484 remote_addr: 127.0.0.1:45594, local_addr: 127.0.0[8810] Got stop action so:false kr:false
27485 .[8820] Disable keep_running
27486 1[8820] Got stop action so:false kr:false
27487 :9998
27488 [8810] Exited with: ExitStatus(unix_wait_status(9))
27489 [8820] Exited with: ExitStatus(unix_wait_status(9))
27490 [8810][0] reports Exit
27491 [8820][1] reports Exit
27492 [8830] Exited with: ExitStatus(unix_wait_status(9))
27493 [8830][2] reports Exit
27494 September 22, 2023 at 11:29:57 PM UTC Test ends with 0