Max level shown:
1 starting January 3, 2025 at 08:30:47 AM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.35500565
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.021844555
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.021324571
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start ds: 8830
15 start access at:127.0.0.1:9998
16 Starting downstairs at port 8820
17 Starting downstairs at port 8810
18 Make output file at "/tmp/dsc/downstairs-8810.txt"
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Starting downstairs at port 8830
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8820][1] initial start wait reports Starting
23 [8810][0] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1195)
26 [8810][0] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1196)
28 [8820][1] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1197)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Jan 03 08:30:47.678 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Jan 03 08:30:52.745 INFO accepted connection, remote_addr: 127.0.0.1:65087, local_addr: 127.0.0.1:9998
36 Jan 03 08:30:52.747 INFO request completed, latency_us: 523, response_code: 200, uri: /regioninfo, method: GET, req_id: a23c8616-0deb-4b27-bf9d-911db5bf380b, remote_addr: 127.0.0.1:65087, local_addr: 127.0.0.1:9998
37 Jan 03 08:30:52.749 INFO request completed, latency_us: 102, response_code: 200, uri: /regioncount, method: GET, req_id: 626c3113-160e-40b2-89b1-3d09e39576e4, remote_addr: 127.0.0.1:65087, local_addr: 127.0.0.1:9998
382025-01-03T08:30:52.748ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392025-01-03T08:30:52.749ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Jan 03 08:30:52.750 INFO request completed, latency_us: 181, response_code: 200, uri: /port/cid/0, method: GET, req_id: 1a671085-a8a6-4e37-ba1f-5c8ca346cc10, remote_addr: 127.0.0.1:65087, local_addr: 127.0.0.1:9998
41 Jan 03 08:30:52.750 INFO request completed, latency_us: 87, response_code: 200, uri: /port/cid/1, method: GET, req_id: 30a3fb5b-d9e6-4560-b9b7-c38a79a36ac7, remote_addr: 127.0.0.1:65087, local_addr: 127.0.0.1:9998
42 Jan 03 08:30:52.751 INFO request completed, latency_us: 87, response_code: 200, uri: /port/cid/2, method: GET, req_id: 19c980a3-e9a9-4803-8d1f-db6f867d3d04, remote_addr: 127.0.0.1:65087, local_addr: 127.0.0.1:9998
432025-01-03T08:30:52.751ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442025-01-03T08:30:52.751ZINFOcrucible: Upstairs starts
452025-01-03T08:30:52.751ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "94736ad28ed33108b1c88fccc3caea693da35514", git_commit_timestamp: "2025-01-03T08:14:58.000000000Z", git_branch: "renovate/tempfile-3.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462025-01-03T08:30:52.751ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472025-01-03T08:30:52.752ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482025-01-03T08:30:52.752ZINFOcrucible: Crucible d27c351f-bb07-40b6-939b-ff5d97869688 has session id: 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
492025-01-03T08:30:52.752ZINFOcrucible: Upstairs opts: Upstairs UUID: d27c351f-bb07-40b6-939b-ff5d97869688, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
502025-01-03T08:30:52.752ZINFOcrucible: Crucible stats registered with UUID: d27c351f-bb07-40b6-939b-ff5d97869688 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
512025-01-03T08:30:52.787ZINFOcrucible: Downstairs in volume = 3 task = crutest
522025-01-03T08:30:52.787ZINFOcrucible: The guest has requested activation with gen:1
532025-01-03T08:30:52.787ZINFOcrucible: d27c351f-bb07-40b6-939b-ff5d97869688 active request set session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
542025-01-03T08:30:52.787ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
552025-01-03T08:30:52.787ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
562025-01-03T08:30:52.787ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
572025-01-03T08:30:52.787ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
582025-01-03T08:30:52.787ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
592025-01-03T08:30:52.787ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
602025-01-03T08:30:52.787ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
612025-01-03T08:30:52.787ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
622025-01-03T08:30:52.787ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
632025-01-03T08:30:52.787ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52510) = io task client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
642025-01-03T08:30:52.788ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59690) = io task client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
652025-01-03T08:30:52.788ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33715) = io task client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
662025-01-03T08:30:52.790ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
672025-01-03T08:30:52.790ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
682025-01-03T08:30:52.790ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
692025-01-03T08:30:52.790ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
702025-01-03T08:30:52.790ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
712025-01-03T08:30:52.790ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
722025-01-03T08:30:52.791ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
732025-01-03T08:30:52.791ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
742025-01-03T08:30:52.791ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
752025-01-03T08:30:52.791ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
762025-01-03T08:30:52.792ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
772025-01-03T08:30:52.792ZINFOcrucible: Waiting for more clients to be ready session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
782025-01-03T08:30:52.792ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
792025-01-03T08:30:52.792ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
802025-01-03T08:30:52.792ZINFOcrucible: Waiting for more clients to be ready session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
812025-01-03T08:30:52.792ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
822025-01-03T08:30:52.792ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
832025-01-03T08:30:52.792ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
842025-01-03T08:30:52.792ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
852025-01-03T08:30:52.792ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
862025-01-03T08:30:52.792ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
872025-01-03T08:30:52.792ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
882025-01-03T08:30:52.792ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
892025-01-03T08:30:52.792ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
902025-01-03T08:30:52.792ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
912025-01-03T08:30:52.792ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
922025-01-03T08:30:52.792ZINFOcrucible: Max found gen is 1 = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
932025-01-03T08:30:52.792ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
942025-01-03T08:30:52.792ZINFOcrucible: Next flush: 1 = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
952025-01-03T08:30:52.792ZINFOcrucible: All extents match = downstairs session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
962025-01-03T08:30:52.792ZINFOcrucible: No downstairs reconciliation required session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
972025-01-03T08:30:52.792ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
982025-01-03T08:30:52.792ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
992025-01-03T08:30:52.792ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
1002025-01-03T08:30:52.792ZINFOcrucible: All required reconciliation work is completed session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
1012025-01-03T08:30:52.792ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
1022025-01-03T08:30:52.792ZINFOcrucible: d27c351f-bb07-40b6-939b-ff5d97869688 is now active with session: 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999 session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
1032025-01-03T08:30:52.792ZINFOcrucible: Set Active after no reconciliation session_id = 1ff0b1cd-44a6-48d7-b33f-f4c84ee12999
1042025-01-03T08:30:52.792ZINFOcrucible: The guest has finished waiting for activation with:1
105 Activate successful
106 Wait for a query_work_queue command to finish before sending IO
107 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
108 Fill test
109 Read and Verify all blocks (0..5000 range:false)
110 Wrote out file "/tmp/test_replay_verify.log"
111 CLIENT: Tests done. All submitted work has been ACK'd
112 ----------------------------------------------------------------
113 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
114 States: Active Active Active
115 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
116 1021 Acked Read 256 Done Done Done false
117 1022 Acked Read 256 Done Done Done false
118 1023 Acked Read 256 Done Done Done false
119 1024 Acked Read 256 Done Done Done false
120 1025 Acked Read 256 Done Done Done false
121 1026 Acked Read 256 Done Done Done false
122 1027 Acked Read 256 Done Done Done false
123 1028 Acked Read 256 Done Done Done false
124 1029 Acked Read 256 Done Done Done false
125 1030 Acked Read 256 Done Done Done false
126 1031 Acked Read 256 Done Done Done false
127 1032 Acked Read 256 Done Done Done false
128 1033 Acked Read 256 Done Done Done false
129 1034 Acked Read 256 Done Done Done false
130 1035 Acked Read 256 Done Done Done false
131 1036 Acked Read 256 Done Done Done false
132 1037 Acked Read 256 Done Done Done false
133 1038 Acked Read 136 Done Done Done false
134 1039 Acked Read 256 Done Done Done false
135 1040 Acked Read 256 Done Done Done false
136 STATES DS:0 DS:1 DS:2 TOTAL
137 Sent 0 0 0 0
138 Done 20 20 20 60
139 Skipped 0 0 0 0
140 Error 0 0 0 0
141 Last Flush: 1020 1020 1020
142 Downstairs last five completed: 1020 1019 1018 1017 1016
143 Upstairs last five completed: 1040 1039 1038 1037 1036
144 CLIENT: Up:0 ds:20 act:3
145 ----------------------------------------------------------------
146 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
147 Downstairs last five completed: 1041 1040 1039 1038 1037
148 Upstairs last five completed: 1041 1040 1039 1038 1037
149 CLIENT: Up:0 ds:0 act:3
150 CLIENT: All crucible jobs finished, exiting program
1512025-01-03T08:30:57.379ZWARNcrucible: Guest handle has been dropped
152 Replay loop starts now January 3, 2025 at 08:30:57 AM UTC
153 Jan 03 08:30:57.452 INFO accepted connection, remote_addr: 127.0.0.1:44885, local_addr: 127.0.0.1:9998
154 Jan 03 08:30:57.453 INFO request completed, latency_us: 161, response_code: 200, uri: /regioninfo, method: GET, req_id: 1832f604-98af-441a-94a1-132c9ef17fea, remote_addr: 127.0.0.1:44885, local_addr: 127.0.0.1:9998
1552025-01-03T08:30:57.454ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
156 Jan 03 08:30:57.454 INFO request completed, latency_us: 99, response_code: 200, uri: /regioncount, method: GET, req_id: 825a0af0-e194-4373-9c0a-4bed45ccaf2b, remote_addr: 127.0.0.1:44885, local_addr: 127.0.0.1:9998
1572025-01-03T08:30:57.454ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
158 Jan 03 08:30:57.455 INFO request completed, latency_us: 118, response_code: 200, uri: /port/cid/0, method: GET, req_id: 50ce1db8-69a1-4be2-a7c4-c4c421fe6879, remote_addr: 127.0.0.1:44885, local_addr: 127.0.0.1:9998
159 Jan 03 08:30:57.455 INFO request completed, latency_us: 97, response_code: 200, uri: /port/cid/1, method: GET, req_id: f22900eb-0571-4881-8d22-9d7bcdd81dec, remote_addr: 127.0.0.1:44885, local_addr: 127.0.0.1:9998
160 Jan 03 08:30:57.456 INFO request completed, latency_us: 90, response_code: 200, uri: /port/cid/2, method: GET, req_id: 10884acd-e062-4248-9ce7-fff1a9867c6a, remote_addr: 127.0.0.1:44885, local_addr: 127.0.0.1:9998
1612025-01-03T08:30:57.456ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1622025-01-03T08:30:57.456ZINFOcrucible: Upstairs starts
1632025-01-03T08:30:57.456ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "94736ad28ed33108b1c88fccc3caea693da35514", git_commit_timestamp: "2025-01-03T08:14:58.000000000Z", git_branch: "renovate/tempfile-3.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1642025-01-03T08:30:57.456ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1652025-01-03T08:30:57.456ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1662025-01-03T08:30:57.457ZINFOcrucible: Crucible 6be50dd6-b272-49d9-8bce-d0d394fa9cd1 has session id: 4d73cd98-4f0b-450f-bfa8-8a9ca6991800 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1672025-01-03T08:30:57.457ZINFOcrucible: Upstairs opts: Upstairs UUID: 6be50dd6-b272-49d9-8bce-d0d394fa9cd1, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1682025-01-03T08:30:57.457ZINFOcrucible: Crucible stats registered with UUID: 6be50dd6-b272-49d9-8bce-d0d394fa9cd1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1692025-01-03T08:30:57.490ZINFOcrucible: Downstairs in volume = 3 task = crutest
1702025-01-03T08:30:57.490ZINFOcrucible: The guest has requested activation with gen:2
1712025-01-03T08:30:57.490ZINFOcrucible: 6be50dd6-b272-49d9-8bce-d0d394fa9cd1 active request set session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1722025-01-03T08:30:57.490ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1732025-01-03T08:30:57.490ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1742025-01-03T08:30:57.490ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1752025-01-03T08:30:57.491ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1762025-01-03T08:30:57.491ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1772025-01-03T08:30:57.491ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1782025-01-03T08:30:57.491ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1792025-01-03T08:30:57.491ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1802025-01-03T08:30:57.491ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1812025-01-03T08:30:57.491ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40072) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1822025-01-03T08:30:57.491ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43909) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1832025-01-03T08:30:57.491ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37156) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1842025-01-03T08:30:57.492ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1852025-01-03T08:30:57.492ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1862025-01-03T08:30:57.492ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1872025-01-03T08:30:57.492ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1882025-01-03T08:30:57.492ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1892025-01-03T08:30:57.492ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1902025-01-03T08:30:57.492ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1912025-01-03T08:30:57.492ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1922025-01-03T08:30:57.492ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1932025-01-03T08:30:57.493ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1942025-01-03T08:30:57.493ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1952025-01-03T08:30:57.493ZINFOcrucible: Waiting for more clients to be ready session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1962025-01-03T08:30:57.493ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1972025-01-03T08:30:57.493ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1982025-01-03T08:30:57.493ZINFOcrucible: Waiting for more clients to be ready session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1992025-01-03T08:30:57.493ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2002025-01-03T08:30:57.493ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2012025-01-03T08:30:57.493ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2022025-01-03T08:30:57.493ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2032025-01-03T08:30:57.493ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2042025-01-03T08:30:57.493ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2052025-01-03T08:30:57.493ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2062025-01-03T08:30:57.493ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2072025-01-03T08:30:57.493ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2082025-01-03T08:30:57.493ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
209 {"msg":"[2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false]","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:30:57.493847682Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205Activate successful
210 ,Wait for a query_work_queue command to finish before sending IO
211 "":"downstairs","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
2122025-01-03T08:30:57.493ZINFOcrucible: Max found gen is 2 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2132025-01-03T08:30:57.493ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2142025-01-03T08:30:57.493ZINFOcrucible: Next flush: 2 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2152025-01-03T08:30:57.493ZINFOcrucible: All extents match = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2162025-01-03T08:30:57.493ZINFOcrucible: No downstairs reconciliation required session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2172025-01-03T08:30:57.494ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2182025-01-03T08:30:57.494ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2192025-01-03T08:30:57.494ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
220 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
2212025-01-03T08:30:57.494ZINFOcrucible: All required reconciliation work is completed session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2222025-01-03T08:30:57.494ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2232025-01-03T08:30:57.494ZINFOcrucible: 6be50dd6-b272-49d9-8bce-d0d394fa9cd1 is now active with session: 4d73cd98-4f0b-450f-bfa8-8a9ca6991800 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2242025-01-03T08:30:57.494ZINFOcrucible: Set Active after no reconciliation session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2252025-01-03T08:30:57.494ZINFOcrucible: The guest has finished waiting for activation with:2
226 Loading write count information from file "/tmp/test_replay_verify.log"
227 Jan 03 08:30:57.527 INFO accepted connection, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
228 Main task has work to do, go find it
229 got dsc Stop(2)
230 stop 2
231 Jan 03 08:30:57.528 INFO request completed, latency_us: 209, response_code: 204[8830] Got stop action so:false kr:false
232 , uri: /stop/cid/2, method: POST, req_id: fe1d0551-7eca-45b0-8074-5088cec4c169, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
2332025-01-03T08:30:57.528ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2342025-01-03T08:30:57.528ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2352025-01-03T08:30:57.528ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
236 Jan 03 08:30:57.528 INFO request completed, latency_us{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:30:57.52895615Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
237 : 183, response_code: 200, uri: /state/cid/2, method: GET, req_id: f6419e6c-3874-4f47-9d94-e72e4b7e9de3, remote_addr: 127.0.{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:30:57.529039668Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
238 0.1:49669, local_addr: 127.0.0.1{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:30:57.529094687Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
239 :9998
240 [8830] Exited with: ExitStatus(unix_wait_status(9))
241 [8830][2] reports Exit
242 Jan 03 08:31:01.531 INFO request completed, latency_us: 193, response_code: 200, uri: /state/cid/2, method: GET, req_id: a64e88bb-ac47-4f1e-a531-2b6483f59224, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
243 001/300 Write block 3789 len 40960 data: 2 2 2 2 2 2 2 2 2 2
244 002/300 Read block 436 len 32768
245 003/300 Write block 4021 len 24576 data: 2 2 2 2 2 2
246 004/300 Write block 1973 len 4096 data: 2
247 005/300 Read block 370 len 12288
248 006/300 Read block 2255 len 8192
249 007/300 Write block 2383 len 12288 data: 2 2 2
250 008/300 Flush
251 009/300 Write block 2743 len 28672 data: 2 2 2 2 2 2 2
252 010/300 Read block 3783 len 8192
253 011/300 Read block 1809 len 28672
254 012/300 Read block 2569 len 40960
255 013/300 Read block 1737 len 40960
256 014/300 Write block 999 len 12288 data: 2 2 2
257 015/300 Read block 2244 len 36864
258 016/300 Read block 4805 len 24576
259 017/300 Write block 2317 len 4096 data: 2
260 018/300 Read block 1266 len 20480
261 019/300 Read block 2777 len 32768
262 020/300 Read block 930 len 12288
263 021/300 Read block 4530 len 36864
264 022/300 Write block 1603 len 4096 data: 2
265 023/300 Read block 2701 len 24576
266 024/300 Write block 2488 len 4096 data: 2
267 025/300 Flush
268 026/300 Write block 4204 len 12288 data: 2 2 2
269 027/300 Write block 3145 len 40960 data: 2 2 2 2 2 2 2 2 2 2
270 028/300 Write block 500 len 24576 data: 2 2 2 2 2 2
271 029/300 Read block 481 len 28672
272 030/300 Read block 1806 len 28672
273 031/300 Write block 3006 len 4096 data: 2
274 032/300 Read block 4613 len 28672
275 033/300 Read block 4355 len 16384
276 034/300 Read block 677 len 20480
277 035/300 Write block 4494 len 32768 data: 2 2 2 2 2 2 2 2
278 036/300 Read block 3115 len 32768
279 037/300 Write block 3089 len 12288 data: 2 2 2
280 038/300 Flush
281 039/300 Write block 3961 len 4096 data: 2
282 040/300 Write block 3543 len 36864 data: 2 2 2 2 2 2 2 2 2
283 041/300 Write block 3653 len 24576 data: 2 2 2 2 2 2
284 042/300 Read block 3801 len 20480
285 043/300 Flush
286 044/300 Read block 4882 len 4096
287 045/300 Read block 997 len 36864
288 046/300 Write block 3421 len 40960 data: 2 2 2 2 2 2 2 2 2 2
289 047/300 Read block 4051 len 24576
290 048/300 Write block 1248 len 32768 data: 2 2 2 2 2 2 2 2
291 049/300 Read block 3498 len 12288
292 050/300 Read block 3729 len 4096
293 051/300 Write block 1451 len 20480 data: 2 2 2 2 2
294 052/300 Write block 3589 len 20480 data: 2 2 2 2 2
295 053/300 Write block 1752 len 24576 data: 2 2 2 2 2 2
296 054/300 Write block 3825 len 16384 data: 2 2 2 2
297 055/300 Write block 4727 len 12288 data: 2 2 2
298 056/300 Write block 3407 len 8192 data: 2 2
299 057/300 Read block 3201 len 12288
300 058/300 Flush
301 059/300 Write block 1352 len 28672 data: 2 2 2 2 2 2 2
302 060/300 Write block 2732 len 40960 data: 2 2 2 2 2 2 2 2 2 2
303 061/300 Write block 3398 len 8192 data: 2 2
304 062/300 Read block 4742 len 40960
305 063/300 Flush
306 064/300 Read block 1843 len 24576
307 065/300 Read block 343 len 4096
308 066/300 Read block 280 len 20480
309 067/300 Write block 2825 len 24576 data: 2 2 2 2 2 2
310 068/300 Write block 2847 len 12288 data: 2 2 2
311 069/300 Write block 476 len 36864 data: 2 2 2 2 2 2 2 2 2
312 070/300 Read block 2557 len 12288
313 071/300 Read block 1538 len 32768
314 072/300 Read block 3749 len 32768
315 073/300 Flush
316 074/300 Read block 3006 len 36864
317 075/300 Flush
318 076/300 Write block 4723 len 32768 data: 2 2 2 2 3 3 3 2
319 077/300 Read block 1175 len 20480
320 078/300 Write block 4777 len 16384 data: 2 2 2 2
321 079/300 Flush
322 080/300 Write block 37 len 12288 data: 2 2 2
323 081/300 Read block 1811 len 24576
324 082/300 Read block 1583 len 36864
325 083/300 Write block 55 len 24576 data: 2 2 2 2 2 2
326 084/300 Read block 870 len 32768
327 085/300 Read block 4871 len 24576
328 086/300 Read block 500 len 12288
329 087/300 Write block 4042 len 12288 data: 2 2 2
330 088/300 Read block 271 len 16384
331 089/300 Read block 1663 len 8192
332 090/300 Write block 3575 len 28672 data: 2 2 2 2 2 2 2
333 091/300 Write block 1577 len 4096 data: 2
334 092/300 Write block 2970 len 24576 data: 2 2 2 2 2 2
335 093/300 Write block 4197 len 24576 data: 2 2 2 2 2 2
336 094/300 Write block 3687 len 24576 data: 2 2 2 2 2 2
337 095/300 Write block 3126 len 24576 data: 2 2 2 2 2 2
338 096/300 Flush
339 097/300 Read block 4172 len 4096
340 098/300 Write block 1643 len 36864 data: 2 2 2 2 2 2 2 2 2
341 099/300 Read block 2391 len 32768
342 100/300 Read block 4554 len 24576
343 101/300 Write block 1523 len 32768 data: 2 2 2 2 2 2 2 2
344 102/300 Read block 1320 len 40960
345 103/300 Read block 4166 len 16384
346 104/300 Read block 2078 len 28672
347 105/300 Write block 3103 len 4096 data: 2
348 106/300 Write block 4419 len 28672 data: 2 2 2 2 2 2 2
349 107/300 Read block 1911 len 32768
350 108/300 Write block 4704 len 16384 data: 2 2 2 2
351 109/300 Read block 2818 len 28672
352 110/300 Read block 4422 len 40960
353 111/300 Flush
354 112/300 Read block 1795 len 28672
355 113/300 Flush
356 114/300 Read block 3468 len 8192
357 115/300 Write block 3900 len 40960 data: 2 2 2 2 2 2 2 2 2 2
358 116/300 Write block 194 len 20480 data: 2 2 2 2 2
359 117/300 Write block 4802 len 24576 data: 2 2 2 2 2 2
360 118/300 Flush
361 119/300 Read block 46 len 12288
362 120/300 Write block 3946 len 24576 data: 2 2 2 2 2 2
363 121/300 Write block 2047 len 28672 data: 2 2 2 2 2 2 2
364 122/300 Read block 1197 len 8192
365 123/300 Write block 4748 len 16384 data: 2 2 2 2
366 124/300 Read block 513 len 4096
367 125/300 Write block 1345 len 16384 data: 2 2 2 2
368 126/300 Write block 2628 len 8192 data: 2 2
369 127/300 Read block 1257 len 12288
370 128/300 Write block 1886 len 28672 data: 2 2 2 2 2 2 2
371 129/300 Write block 1635 len 8192 data: 2 2
372 130/300 Flush
373 131/300 Read block 1319 len 4096
374 132/300 Read block 1100 len 28672
375 133/300 Read block 3812 len 28672
376 134/300 Read block 1608 len 16384
377 135/300 Read block 4374 len 4096
378 136/300 Write block 4121 len 36864 data: 2 2 2 2 2 2 2 2 2
379 137/300 Read block 3670 len 20480
380 138/300 Flush
381 139/300 Write block 4498 len 36864 data: 3 3 3 3 2 2 2 2 2
382 140/300 Write block 1736 len 12288 data: 2 2 2
383 141/300 Read block 4990 len 32768
384 142/300 Write block 1176 len 28672 data: 2 2 2 2 2 2 2
385 143/300 Read block 953 len 12288
386 144/300 Flush
387 145/300 Write block 3060 len 24576 data: 2 2 2 2 2 2
388 146/300 Read block 822 len 24576
389 147/300 Flush
390 148/300 Read block 1381 len 32768
391 149/300 Write block 3435 len 36864 data: 2 2 2 2 2 2 2 2 2
392 150/300 Write block 3780 len 4096 data: 2
393 151/300 Write block 3232 len 16384 data: 2 2 2 2
394 152/300 Flush
395 153/300 Write block 4439 len 40960 data: 2 2 2 2 2 2 2 2 2 2
396 154/300 Read block 754 len 24576
397 155/300 Write block 64 len 40960 data: 2 2 2 2 2 2 2 2 2 2
398 156/300 Write block 2275 len 8192 data: 2 2
399 157/300 Read block 1910 len 36864
400 158/300 Read block 3016 len 4096
401 159/300 Read block 2306 len 32768
402 160/300 Write block 804 len 4096 data: 2
403 161/300 Read block 2487 len 8192
404 162/300 Read block 1517 len 12288
405 163/300 Read block 3895 len 40960
406 164/300 Read block 777 len 28672
407 165/300 Read block 3211 len 24576
408 166/300 Read block 3138 len 20480
409 167/300 Read block 4176 len 40960
410 168/300 Read block 1053 len 4096
411 169/300 Write block 1593 len 8192 data: 2 2
412 170/300 Read block 1986 len 20480
413 171/300 Write block 3853 len 12288 data: 2 2 2
414 172/300 Read block 2021 len 24576
415 173/300 Write block 3068 len 12288 data: 2 2 2
416 174/300 Read block 2956 len 24576
417 175/300 Read block 4696 len 8192
418 176/300 Write block 405 len 24576 data: 2 2 2 2 2 2
419 177/300 Write block 3872 len 12288 data: 2 2 2
420 178/300 Write block 3193 len 36864 data: 2 2 2 2 2 2 2 2 2
421 179/300 Read block 2685 len 16384
422 180/300 Flush
423 181/300 Read block 4656 len 8192
424 182/300 Read block 240 len 16384
425 183/300 Read block 540 len 12288
426 184/300 Read block 2866 len 12288
427 185/300 Write block 1454 len 24576 data: 3 3 2 2 2 2
428 186/300 Read block 677 len 40960
429 187/300 Read block 2654 len 8192
430 188/300 Read block 4827 len 40960
431 189/300 Read block 574 len 16384
432 190/300 Read block 3225 len 16384
433 191/300 Read block 1080 len 12288
434 192/300 Write block 3307 len 40960 data: 2 2 2 2 2 2 2 2 2 2
435 193/300 Read block 1261 len 28672
436 194/300 Write block 804 len 40960 data: 3 2 2 2 2 2 2 2 2 2
437 195/300 Write block 2687 len 28672 data: 2 2 2 2 2 2 2
438 196/300 Read block 1517 len 36864
439 197/300 Read block 3745 len 36864
440 198/300 Write block 1488 len 24576 data: 2 2 2 2 2 2
441 199/300 Write block 693 len 24576 data: 2 2 2 2 2 2
442 200/300 Read block 141 len 12288
443 201/300 Write block 4040 len 36864 data: 2 2 3 3 3 2 2 2 2
444 202/300 Write block 3117 len 8192 data: 2 2
445 203/300 Write block 3054 len 4096 data: 2
446 204/300 Flush
447 205/300 Read block 1228 len 20480
448 206/300 Write block 4599 len 8192 data: 2 2
449 207/300 Read block 2014 len 24576
450 208/300 Flush
451 209/300 Flush
452 210/300 Read block 3140 len 4096
453 211/300 Write block 1274 len 12288 data: 2 2 2
454 212/300 Read block 3658 len 20480
455 213/300 Read block 4118 len 24576
456 214/300 Flush
457 215/300 Read block 2121 len 8192
458 216/300 Write block 4729 len 12288 data: 4 3 2
459 217/300 Write block 341 len 36864 data: 2 2 2 2 2 2 2 2 2
460 218/300 Write block 2708 len 36864 data: 2 2 2 2 2 2 2 2 2
461 219/300 Read block 291 len 28672
462 220/300 Write block 170 len 28672 data: 2 2 2 2 2 2 2
463 221/300 Write block 172 len 36864 data: 3 3 3 3 3 2 2 2 2
464 222/300 Read block 4632 len 20480
465 223/300 Write block 2184 len 24576 data: 2 2 2 2 2 2
466 224/300 Write block 1632 len 4096 data: 2
467 225/300 Flush
468 226/300 Read block 1061 len 12288
469 227/300 Write block 3422 len 36864 data: 3 3 3 3 3 3 3 3 3
470 228/300 Write block 3218 len 32768 data: 2 2 2 2 2 2 2 2
471 229/300 Read block 2898 len 36864
472 230/300 Read block 3861 len 16384
473 231/300 Write block 3852 len 8192 data: 2 3
474 232/300 Read block 3776 len 32768
475 233/300 Write block 2756 len 32768 data: 2 2 2 2 2 2 2 2
476 234/300 Flush
477 235/300 Read block 637 len 28672
478 236/300 Read block 1873 len 24576
479 237/300 Read block 3011 len 28672
480 238/300 Read block 498 len 4096
481 239/300 Write block 1478 len 4096 data: 2
482 240/300 Flush
483 241/300 Flush
484 242/300 Write block 914 len 40960 data: 2 2 2 2 2 2 2 2 2 2
485 243/300 Write block 3750 len 4096 data: 2
486 244/300 Write block 1089 len 40960 data: 2 2 2 2 2 2 2 2 2 2
487 245/300 Write block 206 len 4096 data: 2
488 246/300 Write block 4138 len 28672 data: 2 2 2 2 2 2 2
489 247/300 Flush
490 248/300 Write block 4633 len 40960 data: 2 2 2 2 2 2 2 2 2 2
491 249/300 Write block 427 len 20480 data: 2 2 2 2 2
492 250/300 Read block 1805 len 16384
493 251/300 Read block 1267 len 20480
494 252/300 Read block 4914 len 16384
495 253/300 Write block 2453 len 28672 data: 2 2 2 2 2 2 2
496 254/300 Read block 2213 len 40960
497 255/300 Write block 3395 len 16384 data: 2 2 2 3
498 256/300 Read block 2897 len 12288
499 257/300 Read block 1904 len 16384
500 258/300 Read block 1667 len 36864
501 259/300 Write block 1919 len 36864 data: 2 2 2 2 2 2 2 2 2
502 260/300 Read block 3169 len 40960
503 261/300 Write block 1093 len 36864 data: 3 3 3 3 3 3 2 2 2
504 262/300 Write block 3999 len 16384 data: 2 2 2 2
505 263/300 Read block 1163 len 40960
506 264/300 Write block 664 len 12288 data: 2 2 2
507 265/300 Write block 1419 len 32768 data: 2 2 2 2 2 2 2 2
508 266/300 Write block 2300 len 12288 data: 2 2 2
509 267/300 Read block 124 len 28672
510 268/300 Read block 4428 len 36864
511 269/300 Read block 3198 len 36864
512 270/300 Write block 218 len 20480 data: 2 2 2 2 2
513 271/300 Write block 1781 len 16384 data: 2 2 2 2
514 272/300 Read block 2804 len 24576
515 273/300 Write block 2124 len 36864 data: 2 2 2 2 2 2 2 2 2
516 274/300 Read block 220 len 40960
517 275/300 Read block 4833 len 4096
518 276/300 Write block 3542 len 28672 data: 2 3 3 3 3 3 3
519 277/300 Read block 515 len 32768
520 278/300 Write block 529 len 12288 data: 2 2 2
521 279/300 Read block 109 len 28672
522 280/300 Write block 895 len 36864 data: 2 2 2 2 2 2 2 2 2
523 281/300 Write block 4789 len 12288 data: 2 2 2
524 282/300 Flush
525 283/300 Write block 2660 len 4096 data: 2
526 284/300 Write block 3347 len 20480 data: 2 2 2 2 2
527 285/300 Write block 1628 len 20480 data: 2 2 2 2 3
528 286/300 Write block 4097 len 4096 data: 2
529 287/300 Read block 344 len 12288
530 288/300 Read block 1716 len 16384
531 289/300 Write block 538 len 32768 data: 2 2 2 2 2 2 2 2
532 290/300 Write block 1992 len 40960 data: 2 2 2 2 2 2 2 2 2 2
533 291/300 Write block 2708 len 28672 data: 3 3 3 3 3 3 3
534 292/300 Write block 4831 len 32768 data: 2 2 2 2 2 2 2 2
535 293/300 Read block 3296 len 20480
536 294/300 Read block 871 len 12288
537 295/300 Write block 1757 len 4096 data: 3
538 296/300 Write block 4252 len 40960 data: 2 2 2 2 2 2 2 2 2 2
539 297/300 Write block 4942 len 28672 data: 2 2 2 2 2 2 2
540 298/300 Write block 1904 len 40960 data: 2 2 2 2 2 2 2 2 2 2
541 299/300 Write block 2259 len 4096 data: 2
542 300/300 Write block 2826 len 4096 data: 3
543 301/300 Write block 708 len 40960 data: 2 2 2 2 2 2 2 2 2 2
544 Main task has work to do, go find it
545 got dsc Start(2)
546 start 2
547 Jan 03 08:31:01.770 INFO request completed, latency_us: 289, response_code: 204, uri: /start/cid/2, method: POST, req_id: bb7c607a-cfd9-456f-951b-2d53e4b65e80, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
548 Replay: started 2, returned:Ok(())
549 ----------------------------------------------------------------
550 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
551 States: Active Active Offline
552 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
553 1000 Acked Write 10 Done Done Sent false
554 1001 Acked Read 8 Done Done Sent false
555 1002 Acked Write 6 Done Done Sent false
556 1003 Acked Write 1 Done Done Sent false
557 1004 Acked Read 3 Done Done Sent false
558 1005 Acked Read 2 Done Done Sent false
559 1006 Acked Write 3 Done Done Sent false
560 1007 Acked Flush 0 Done Done Sent false
561 1008 Acked Write 7 Done Done Sent false
562 1009 Acked Read 2 Done Done Sent false
563 1010 Acked Read 7 Done Done Sent false
564 1011 Acked Read 10 Done Done Sent false
565 1012 Acked Read 10 Done Done Sent false
566 1013 Acked Write 3 Done Done Sent false
567 1014 Acked Read 9 Done Done Sent false
568 1015 Acked Read 6 Done Done Sent false
569 1016 Acked Write 1 Done Done Sent false
570 1017 Acked Read 5 Done Done Sent false
571 1018 Acked Read 8 Done Done Sent false
572 1019 Acked Read 3 Done Done Sent false
573 1020 Acked Read 9 Done Done Sent false
574 1021 Acked Write 1 Done Done Sent false
575 1022 Acked Read 6 Done Done Sent false
576 1023 Acked Write 1 Done Done Sent false
577 1024 Acked Flush 0 Done Done Sent false
578 1025 Acked Write 3 Done Done Sent false
579 1026 Acked Write 10 Done Done Sent false
580 1027 Acked Write 6 Done Done Sent false
581 1028 Acked Read 7 Done Done Sent false
582 1029 Acked Read 7 Done Done Sent false
583 1030 Acked Write 1 Done Done Sent false
584 1031 Acked Read 7 Done Done Sent false
585 1032 Acked Read 4 Done Done Sent false
586 1033 Acked Read 5 Done Done Sent false
587 1034 Acked Write 8 Done Done Sent false
588 1035 Acked Read 8 Done Done Sent false
589 1036 Acked Write 3 Done Done Sent false
590 1037 Acked Flush 0 Done Done Sent false
591 1038 Acked Write 1 Done Done Sent false
592 1039 Acked Write 9 Done Done Sent false
593 1040 Acked Write 6 Done Done Sent false
594 1041 Acked Read 5 Done Done Sent false
595 1042 Acked Flush 0 Done Done Sent false
596 1043 Acked Read 1 Done Done Sent false
597 1044 Acked Read 9 Done Done Sent false
598 1045 Acked Write 10 Done Done Sent false
599 1046 Acked Read 6 Done Done Sent false
600 1047 Acked Write 8 Done Done Sent false
601 1048 Acked Read 3 Done Done Sent false
602 1049 Acked Read 1 Done Done Sent false
603 1050 Acked Write 5 Done Done Sent false
604 1051 Acked Write 5 Done Done Sent false
605 1052 Acked Write 6 Done Done Sent false
606 1053 Acked Write 4 Done Done Sent false
607 1054 Acked Write 3 Done Done Sent false
608 1055 Acked Write 2 Done Done Sent false
609 1056 Acked Read 3 Done Done Sent false
610 1057 Acked Flush 0 Done Done Sent false
611 1058 Acked Write 7 Done Done Sent false
612 1059 Acked Write 10 Done Done Sent false
613 1060 Acked Write 2 Done Done Sent false
614 1061 Acked Read 10 Done Done Sent false
615 1062 Acked Flush 0 Done Done Sent false
616 1063 Acked Read 6 Done Done Sent false
617 1064 Acked Read 1 Done Done Sent false
618 1065 Acked Read 5 Done Done Sent false
619 1066 Acked Write 6 Done Done Sent false
620 1067 Acked Write 3 Done Done Sent false
621 1068 Acked Write 9 Done Done Sent false
622 1069 Acked Read 3 Done Done Sent false
623 1070 Acked Read 8 Done Done Sent false
624 1071 Acked Read 8 Done Done Sent false
625 1072 Acked Flush 0 Done Done Sent false
626 1073 Acked Read 9 Done Done Sent false
627 1074 Acked Flush 0 Done Done Sent false
628 1075 Acked Write 8 Done Done Sent false
629 1076 Acked Read 5 Done Done Sent false
630 1077 Acked Write 4 Done Done Sent false
631 1078 Acked Flush 0 Done Done Sent false
632 1079 Acked Write 3 Done Done Sent false
633 1080 Acked Read 6 Done Done Sent false
634 1081 Acked Read 9 Done Done Sent false
635 1082 Acked Write 6 Done Done Sent false
636 1083 Acked Read 8 Done Done Sent false
637 1084 Acked Read 6 Done Done Sent false
638 1085 Acked Read 3 Done Done Sent false
639 1086 Acked Write 3 Done Done Sent false
640 1087 Acked Read 4 Done Done Sent false
641 1088 Acked Read 2 Done Done Sent false
642 1089 Acked Write 7 Done Done Sent false
643 1090 Acked Write 1 Done Done Sent false
644 1091 Acked Write 6 Done Done Sent false
645 1092 Acked Write 6 Done Done Sent false
646 1093 Acked Write 6 Done Done Sent false
647 1094 Acked Write 6 Done Done Sent false
648 1095 Acked Flush 0 Done Done Sent false
649 1096 Acked Read 1 Done Done Sent false
650 1097 Acked Write 9 Done Done Sent false
651 1098 Acked Read 8 Done Done Sent false
652 1099 Acked Read 6 Done Done Sent false
653 1100 Acked Write 8 Done Done Sent false
654 1101 Acked Read 10 Done Done Sent false
655 1102 Acked Read 4 Done Done Sent false
656 1103 Acked Read 7 Done Done Sent false
657 1104 Acked Write 1 Done Done Sent false
658 1105 Acked Write 7 Done Done Sent false
659 1106 Acked Read 8 Done Done Sent false
660 1107 Acked Write 4 Done Done Sent false
661 1108 Acked Read 7 Done Done Sent false
662 1109 Acked Read 10 Done Done Sent false
663 1110 Acked Flush 0 Done Done Sent false
664 1111 Acked Read 7 Done Done Sent false
665 1112 Acked Flush 0 Done Done Sent false
666 1113 Acked Read 2 Done Done Sent false
667 1114 Acked Write 10 Done Done Sent false
668 1115 Acked Write 5 Done Done Sent false
669 1116 Acked Write 6 Done Done Sent false
670 1117 Acked Flush 0 Done Done Sent false
671 1118 Acked Read 3 Done Done Sent false
672 1119 Acked Write 6 Done Done Sent false
673 1120 Acked Write 7 Done Done Sent false
674 1121 Acked Read 2 Done Done Sent false
675 1122 Acked Write 4 Done Done Sent false
676 1123 Acked Read 1 Done Done Sent false
677 1124 Acked Write 4 Done Done Sent false
678 1125 Acked Write 2 Done Done Sent false
679 1126 Acked Read 3 Done Done Sent false
680 1127 Acked Write 7 Done Done Sent false
681 1128 Acked Write 2 Done Done Sent false
682 1129 Acked Flush 0 Done Done Sent false
683 1130 Acked Read 1 Done Done Sent false
684 1131 Acked Read 7 Done Done Sent false
685 1132 Acked Read 7 Done Done Sent false
686 1133 Acked Read 4 Done Done Sent false
687 1134 Acked Read 1 Done Done Sent false
688 1135 Acked Write 9 Done Done Sent false
689 1136 Acked Read 5 Done Done Sent false
690 1137 Acked Flush 0 Done Done Sent false
691 1138 Acked Write 9 Done Done Sent false
692 1139 Acked Write 3 Done Done Sent false
693 1140 Acked Read 8 Done Done Sent false
694 1141 Acked Write 7 Done Done Sent false
695 1142 Acked Read 3 Done Done Sent false
696 1143 Acked Flush 0 Done Done Sent false
697 1144 Acked Write 6 Done Done Sent false
698 1145 Acked Read 6 Done Done Sent false
699 1146 Acked Flush 0 Done Done Sent false
700 1147 Acked Read 8 Done Done Sent false
701 1148 Acked Write 9 Done Done Sent false
702 1149 Acked Write 1 Done Done Sent false
703 1150 Acked Write 4 Done Done Sent false
704 1151 Acked Flush 0 Done Done Sent false
705 1152 Acked Write 10 Done Done Sent false
706 1153 Acked Read 6 Done Done Sent false
707 1154 Acked Write 10 Done Done Sent false
708 1155 Acked Write 2 Done Done Sent false
709 1156 Acked Read 9 Done Done Sent false
710 1157 Acked Read 1 Done Done Sent false
711 1158 Acked Read 8 Done Done Sent false
712 1159 Acked Write 1 Done Done Sent false
713 1160 Acked Read 2 Done Done Sent false
714 1161 Acked Read 3 Done Done Sent false
715 1162 Acked Read 10 Done Done Sent false
716 1163 Acked Read 7 Done Done Sent false
717 1164 Acked Read 6 Done Done Sent false
718 1165 Acked Read 5 Done Done Sent false
719 1166 Acked Read 10 Done Done Sent false
720 1167 Acked Read 1 Done Done Sent false
721 1168 Acked Write 2 Done Done Sent false
722 1169 Acked Read 5 Done Done Sent false
723 1170 Acked Write 3 Done Done Sent false
724 1171 Acked Read 6 Done Done Sent false
725 1172 Acked Write 3 Done Done Sent false
726 1173 Acked Read 6 Done Done Sent false
727 1174 Acked Read 2 Done Done Sent false
728 1175 Acked Write 6 Done Done Sent false
729 1176 Acked Write 3 Done Done Sent false
730 1177 Acked Write 9 Done Done Sent false
731 1178 Acked Read 4 Done Done Sent false
732 1179 Acked Flush 0 Done Done Sent false
733 1180 Acked Read 2 Done Done Sent false
734 1181 Acked Read 4 Done Done Sent false
735 1182 Acked Read 3 Done Done Sent false
736 1183 Acked Read 3 Done Done Sent false
737 1184 Acked Write 6 Done Done Sent false
738 1185 Acked Read 10 Done Done Sent false
739 1186 Acked Read 2 Done Done Sent false
740 1187 Acked Read 10 Done Done Sent false
741 1188 Acked Read 4 Done Done Sent false
742 1189 Acked Read 4 Done Done Sent false
743 1190 Acked Read 3 Done Done Sent false
744 1191 Acked Write 10 Done Done Sent false
745 1192 Acked Read 7 Done Done Sent false
746 1193 Acked Write 10 Done Done Sent false
747 1194 Acked Write 7 Done Done Sent false
748 1195 Acked Read 9 Done Done Sent false
749 1196 Acked Read 9 Done Done Sent false
750 1197 Acked Write 6 Done Done Sent false
751 1198 Acked Write 6 Done Done Sent false
752 1199 Acked Read 3 Done Done Sent false
753 1200 Acked Write 9 Done Done Sent false
754 1201 Acked Write 2 Done Done Sent false
755 1202 Acked Write 1 Done Done Sent false
756 1203 Acked Flush 0 Done Done Sent false
757 1204 Acked Read 5 Done Done Sent false
758 1205 Acked Write 2 Done Done Sent false
759 1206 Acked Read 6 Done Done Sent false
760 1207 Acked Flush 0 Done Done Sent false
761 1208 Acked Flush 0 Done Done Sent false
762 1209 Acked Read 1 Done Done Sent false
763 1210 Acked Write 3 Done Done Sent false
764 1211 Acked Read 5 Done Done Sent false
765 1212 Acked Read 6 Done Done Sent false
766 1213 Acked Flush 0 Done Done Sent false
767 1214 Acked Read 2 Done Done Sent false
768 1215 Acked Write 3 Done Done Sent false
769 1216 Acked Write 9 Done Done Sent false
770 1217 Acked Write 9 Done Done Sent false
771 1218 Acked Read 7 Done Done Sent false
772 1219 Acked Write 7 Done Done Sent false
773 1220 Acked Write 9 Done Done Sent false
774 1221 Acked Read 5 Done Done Sent false
775 1222 Acked Write 6 Done Done Sent false
776 1223 Acked Write 1 Done Done Sent false
777 1224 Acked Flush 0 Done Done Sent false
778 1225 Acked Read 3 Done Done Sent false
779 1226 Acked Write 9 Done Done Sent false
780 1227 Acked Write 8 Done Done Sent false
781 1228 Acked Read 9 Done Done Sent false
782 1229 Acked Read 4 Done Done Sent false
783 1230 Acked Write 2 Done Done Sent false
784 1231 Acked Read 8 Done Done Sent false
785 1232 Acked Write 8 Done Done Sent false
786 1233 Acked Flush 0 Done Done Sent false
787 1234 Acked Read 7 Done Done Sent false
788 1235 Acked Read 6 Done Done Sent false
789 1236 Acked Read 7 Done Done Sent false
790 1237 Acked Read 1 Done Done Sent false
791 1238 Acked Write 1 Done Done Sent false
792 1239 Acked Flush 0 Done Done Sent false
793 1240 Acked Flush 0 Done Done Sent false
794 1241 Acked Write 10 Done Done Sent false
795 1242 Acked Write 1 Done Done Sent false
796 1243 Acked Write 10 Done Done Sent false
797 1244 Acked Write 1 Done Done Sent false
798 1245 Acked Write 7 Done Done Sent false
799 1246 Acked Flush 0 Done Done Sent false
800 1247 Acked Write 10 Done Done Sent false
801 1248 Acked Write 5 Done Done Sent false
802 1249 Acked Read 4 Done Done Sent false
803 1250 Acked Read 5 Done Done Sent false
804 1251 Acked Read 4 Done Done Sent false
805 1252 Acked Write 7 Done Done Sent false
806 1253 Acked Read 10 Done Done Sent false
807 1254 Acked Write 4 Done Done Sent false
808 1255 Acked Read 3 Done Done Sent false
809 1256 Acked Read 4 Done Done Sent false
810 1257 Acked Read 9 Done Done Sent false
811 1258 Acked Write 9 Done Done Sent false
812 1259 Acked Read 10 Done Done Sent false
813 1260 Acked Write 9 Done Done Sent false
814 1261 Acked Write 4 Done Done Sent false
815 1262 Acked Read 10 Done Done Sent false
816 1263 Acked Write 3 Done Done Sent false
817 1264 Acked Write 8 Done Done Sent false
818 1265 Acked Write 3 Done Done Sent false
819 1266 Acked Read 7 Done Done Sent false
820 1267 Acked Read 9 Done Done Sent false
821 1268 Acked Read 9 Done Done Sent false
822 1269 Acked Write 5 Done Done Sent false
823 1270 Acked Write 4 Done Done Sent false
824 1271 Acked Read 6 Done Done Sent false
825 1272 Acked Write 9 Done Done Sent false
826 1273 Acked Read 10 Done Done Sent false
827 1274 Acked Read 1 Done Done Sent false
828 1275 Acked Write 7 Done Done Sent false
829 1276 Acked Read 8 Done Done Sent false
830 1277 Acked Write 3 Done Done Sent false
831 1278 Acked Read 7 Done Done Sent false
832 1279 Acked Write 9 Done Done Sent false
833 1280 Acked Write 3 Done Done Sent false
834 1281 Acked Flush 0 Done Done Sent false
835 1282 Acked Write 1 Done Done Sent false
836 1283 Acked Write 5 Done Done Sent false
837 1284 Acked Write 5 Done Done Sent false
838 1285 Acked Write 1 Done Done Sent false
839 1286 Acked Read 3 Done Done Sent false
840 1287 Acked Read 4 Done Done Sent false
841 1288 Acked Write 8 Done Done Sent false
842 1289 Acked Write 10 Done Done Sent false
843 1290 Acked Write 7 Done Done Sent false
844 1291 Acked Write 8 Done Done Sent false
845 1292 Acked Read 5 Done Done Sent false
846 1293 Acked Read 3 Done Done Sent false
847 1294 Acked Write 1 Done Done Sent false
848 1295 Acked Write 10 Done Done Sent false
849 1296 Acked Write 7 Done Done Sent false
850 1297 Acked Write 10 Done Done Sent false
851 1298 Acked Write 1 Done Done Sent false
852 1299 Acked Write 1 Done Done Sent false
853 1300 Acked Write 10 Done Done Sent false
854 STATES DS:0 DS:1 DS:2 TOTAL
855 Sent 0 0 301 301
856 Done 301 301 0 602
857 Skipped 0 0 0 0
858 Error 0 0 0 0
859 Last Flush: 1281 1281 0
860 Downstairs last five completed:
861 Upstairs last five completed: 1300 1299 1298 1297 1296
862 CLIENT: Up:0 ds:301 act:2
863 [8830] got start action
864 [8830] I am going to restart
865 Starting downstairs at port 8830
866 Make output file at "/tmp/dsc/downstairs-8830.txt"
867 [8830][2] reports Starting
868 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1206)
869 [8830][2] reports Running
870 ----------------------------------------------------------------
871 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
872 States: Active Active Offline
873 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
874 1000 Acked Write 10 Done Done Sent false
875 1001 Acked Read 8 Done Done Sent false
876 1002 Acked Write 6 Done Done Sent false
877 1003 Acked Write 1 Done Done Sent false
878 1004 Acked Read 3 Done Done Sent false
879 1005 Acked Read 2 Done Done Sent false
880 1006 Acked Write 3 Done Done Sent false
881 1007 Acked Flush 0 Done Done Sent false
882 1008 Acked Write 7 Done Done Sent false
883 1009 Acked Read 2 Done Done Sent false
884 1010 Acked Read 7 Done Done Sent false
885 1011 Acked Read 10 Done Done Sent false
886 1012 Acked Read 10 Done Done Sent false
887 1013 Acked Write 3 Done Done Sent false
888 1014 Acked Read 9 Done Done Sent false
889 1015 Acked Read 6 Done Done Sent false
890 1016 Acked Write 1 Done Done Sent false
891 1017 Acked Read 5 Done Done Sent false
892 1018 Acked Read 8 Done Done Sent false
893 1019 Acked Read 3 Done Done Sent false
894 1020 Acked Read 9 Done Done Sent false
895 1021 Acked Write 1 Done Done Sent false
896 1022 Acked Read 6 Done Done Sent false
897 1023 Acked Write 1 Done Done Sent false
898 1024 Acked Flush 0 Done Done Sent false
899 1025 Acked Write 3 Done Done Sent false
900 1026 Acked Write 10 Done Done Sent false
901 1027 Acked Write 6 Done Done Sent false
902 1028 Acked Read 7 Done Done Sent false
903 1029 Acked Read 7 Done Done Sent false
904 1030 Acked Write 1 Done Done Sent false
905 1031 Acked Read 7 Done Done Sent false
906 1032 Acked Read 4 Done Done Sent false
907 1033 Acked Read 5 Done Done Sent false
908 1034 Acked Write 8 Done Done Sent false
909 1035 Acked Read 8 Done Done Sent false
910 1036 Acked Write 3 Done Done Sent false
911 1037 Acked Flush 0 Done Done Sent false
912 1038 Acked Write 1 Done Done Sent false
913 1039 Acked Write 9 Done Done Sent false
914 1040 Acked Write 6 Done Done Sent false
915 1041 Acked Read 5 Done Done Sent false
916 1042 Acked Flush 0 Done Done Sent false
917 1043 Acked Read 1 Done Done Sent false
918 1044 Acked Read 9 Done Done Sent false
919 1045 Acked Write 10 Done Done Sent false
920 1046 Acked Read 6 Done Done Sent false
921 1047 Acked Write 8 Done Done Sent false
922 1048 Acked Read 3 Done Done Sent false
923 1049 Acked Read 1 Done Done Sent false
924 1050 Acked Write 5 Done Done Sent false
925 1051 Acked Write 5 Done Done Sent false
926 1052 Acked Write 6 Done Done Sent false
927 1053 Acked Write 4 Done Done Sent false
928 1054 Acked Write 3 Done Done Sent false
929 1055 Acked Write 2 Done Done Sent false
930 1056 Acked Read 3 Done Done Sent false
931 1057 Acked Flush 0 Done Done Sent false
932 1058 Acked Write 7 Done Done Sent false
933 1059 Acked Write 10 Done Done Sent false
934 1060 Acked Write 2 Done Done Sent false
935 1061 Acked Read 10 Done Done Sent false
936 1062 Acked Flush 0 Done Done Sent false
937 1063 Acked Read 6 Done Done Sent false
938 1064 Acked Read 1 Done Done Sent false
939 1065 Acked Read 5 Done Done Sent false
940 1066 Acked Write 6 Done Done Sent false
941 1067 Acked Write 3 Done Done Sent false
942 1068 Acked Write 9 Done Done Sent false
943 1069 Acked Read 3 Done Done Sent false
944 1070 Acked Read 8 Done Done Sent false
945 1071 Acked Read 8 Done Done Sent false
946 1072 Acked Flush 0 Done Done Sent false
947 1073 Acked Read 9 Done Done Sent false
948 1074 Acked Flush 0 Done Done Sent false
949 1075 Acked Write 8 Done Done Sent false
950 1076 Acked Read 5 Done Done Sent false
951 1077 Acked Write 4 Done Done Sent false
952 1078 Acked Flush 0 Done Done Sent false
953 1079 Acked Write 3 Done Done Sent false
954 1080 Acked Read 6 Done Done Sent false
955 1081 Acked Read 9 Done Done Sent false
956 1082 Acked Write 6 Done Done Sent false
957 1083 Acked Read 8 Done Done Sent false
958 1084 Acked Read 6 Done Done Sent false
959 1085 Acked Read 3 Done Done Sent false
960 1086 Acked Write 3 Done Done Sent false
961 1087 Acked Read 4 Done Done Sent false
962 1088 Acked Read 2 Done Done Sent false
963 1089 Acked Write 7 Done Done Sent false
964 1090 Acked Write 1 Done Done Sent false
965 1091 Acked Write 6 Done Done Sent false
966 1092 Acked Write 6 Done Done Sent false
967 1093 Acked Write 6 Done Done Sent false
968 1094 Acked Write 6 Done Done Sent false
969 1095 Acked Flush 0 Done Done Sent false
970 1096 Acked Read 1 Done Done Sent false
971 1097 Acked Write 9 Done Done Sent false
972 1098 Acked Read 8 Done Done Sent false
973 1099 Acked Read 6 Done Done Sent false
974 1100 Acked Write 8 Done Done Sent false
975 1101 Acked Read 10 Done Done Sent false
976 1102 Acked Read 4 Done Done Sent false
977 1103 Acked Read 7 Done Done Sent false
978 1104 Acked Write 1 Done Done Sent false
979 1105 Acked Write 7 Done Done Sent false
980 1106 Acked Read 8 Done Done Sent false
981 1107 Acked Write 4 Done Done Sent false
982 1108 Acked Read 7 Done Done Sent false
983 1109 Acked Read 10 Done Done Sent false
984 1110 Acked Flush 0 Done Done Sent false
985 1111 Acked Read 7 Done Done Sent false
986 1112 Acked Flush 0 Done Done Sent false
987 1113 Acked Read 2 Done Done Sent false
988 1114 Acked Write 10 Done Done Sent false
989 1115 Acked Write 5 Done Done Sent false
990 1116 Acked Write 6 Done Done Sent false
991 1117 Acked Flush 0 Done Done Sent false
992 1118 Acked Read 3 Done Done Sent false
993 1119 Acked Write 6 Done Done Sent false
994 1120 Acked Write 7 Done Done Sent false
995 1121 Acked Read 2 Done Done Sent false
996 1122 Acked Write 4 Done Done Sent false
997 1123 Acked Read 1 Done Done Sent false
998 1124 Acked Write 4 Done Done Sent false
999 1125 Acked Write 2 Done Done Sent false
1000 1126 Acked Read 3 Done Done Sent false
1001 1127 Acked Write 7 Done Done Sent false
1002 1128 Acked Write 2 Done Done Sent false
1003 1129 Acked Flush 0 Done Done Sent false
1004 1130 Acked Read 1 Done Done Sent false
1005 1131 Acked Read 7 Done Done Sent false
1006 1132 Acked Read 7 Done Done Sent false
1007 1133 Acked Read 4 Done Done Sent false
1008 1134 Acked Read 1 Done Done Sent false
1009 1135 Acked Write 9 Done Done Sent false
1010 1136 Acked Read 5 Done Done Sent false
1011 1137 Acked Flush 0 Done Done Sent false
1012 1138 Acked Write 9 Done Done Sent false
1013 1139 Acked Write 3 Done Done Sent false
1014 1140 Acked Read 8 Done Done Sent false
1015 1141 Acked Write 7 Done Done Sent false
1016 1142 Acked Read 3 Done Done Sent false
1017 1143 Acked Flush 0 Done Done Sent false
1018 1144 Acked Write 6 Done Done Sent false
1019 1145 Acked Read 6 Done Done Sent false
1020 1146 Acked Flush 0 Done Done Sent false
1021 1147 Acked Read 8 Done Done Sent false
1022 1148 Acked Write 9 Done Done Sent false
1023 1149 Acked Write 1 Done Done Sent false
1024 1150 Acked Write 4 Done Done Sent false
1025 1151 Acked Flush 0 Done Done Sent false
1026 1152 Acked Write 10 Done Done Sent false
1027 1153 Acked Read 6 Done Done Sent false
1028 1154 Acked Write 10 Done Done Sent false
1029 1155 Acked Write 2 Done Done Sent false
1030 1156 Acked Read 9 Done Done Sent false
1031 1157 Acked Read 1 Done Done Sent false
1032 1158 Acked Read 8 Done Done Sent false
1033 1159 Acked Write 1 Done Done Sent false
1034 1160 Acked Read 2 Done Done Sent false
1035 1161 Acked Read 3 Done Done Sent false
1036 1162 Acked Read 10 Done Done Sent false
1037 1163 Acked Read 7 Done Done Sent false
1038 1164 Acked Read 6 Done Done Sent false
1039 1165 Acked Read 5 Done Done Sent false
1040 1166 Acked Read 10 Done Done Sent false
1041 1167 Acked Read 1 Done Done Sent false
1042 1168 Acked Write 2 Done Done Sent false
1043 1169 Acked Read 5 Done Done Sent false
1044 1170 Acked Write 3 Done Done Sent false
1045 1171 Acked Read 6 Done Done Sent false
1046 1172 Acked Write 3 Done Done Sent false
1047 1173 Acked Read 6 Done Done Sent false
1048 1174 Acked Read 2 Done Done Sent false
1049 1175 Acked Write 6 Done Done Sent false
1050 1176 Acked Write 3 Done Done Sent false
1051 1177 Acked Write 9 Done Done Sent false
1052 1178 Acked Read 4 Done Done Sent false
1053 1179 Acked Flush 0 Done Done Sent false
1054 1180 Acked Read 2 Done Done Sent false
1055 1181 Acked Read 4 Done Done Sent false
1056 1182 Acked Read 3 Done Done Sent false
1057 1183 Acked Read 3 Done Done Sent false
1058 1184 Acked Write 6 Done Done Sent false
1059 1185 Acked Read 10 Done Done Sent false
1060 1186 Acked Read 2 Done Done Sent false
1061 1187 Acked Read 10 Done Done Sent false
1062 1188 Acked Read 4 Done Done Sent false
1063 1189 Acked Read 4 Done Done Sent false
1064 1190 Acked Read 3 Done Done Sent false
1065 1191 Acked Write 10 Done Done Sent false
1066 1192 Acked Read 7 Done Done Sent false
1067 1193 Acked Write 10 Done Done Sent false
1068 1194 Acked Write 7 Done Done Sent false
1069 1195 Acked Read 9 Done Done Sent false
1070 1196 Acked Read 9 Done Done Sent false
1071 1197 Acked Write 6 Done Done Sent false
1072 1198 Acked Write 6 Done Done Sent false
1073 1199 Acked Read 3 Done Done Sent false
1074 1200 Acked Write 9 Done Done Sent false
1075 1201 Acked Write 2 Done Done Sent false
1076 1202 Acked Write 1 Done Done Sent false
1077 1203 Acked Flush 0 Done Done Sent false
1078 1204 Acked Read 5 Done Done Sent false
1079 1205 Acked Write 2 Done Done Sent false
1080 1206 Acked Read 6 Done Done Sent false
1081 1207 Acked Flush 0 Done Done Sent false
1082 1208 Acked Flush 0 Done Done Sent false
1083 1209 Acked Read 1 Done Done Sent false
1084 1210 Acked Write 3 Done Done Sent false
1085 1211 Acked Read 5 Done Done Sent false
1086 1212 Acked Read 6 Done Done Sent false
1087 1213 Acked Flush 0 Done Done Sent false
1088 1214 Acked Read 2 Done Done Sent false
1089 1215 Acked Write 3 Done Done Sent false
1090 1216 Acked Write 9 Done Done Sent false
1091 1217 Acked Write 9 Done Done Sent false
1092 1218 Acked Read 7 Done Done Sent false
1093 1219 Acked Write 7 Done Done Sent false
1094 1220 Acked Write 9 Done Done Sent false
1095 1221 Acked Read 5 Done Done Sent false
1096 1222 Acked Write 6 Done Done Sent false
1097 1223 Acked Write 1 Done Done Sent false
1098 1224 Acked Flush 0 Done Done Sent false
1099 1225 Acked Read 3 Done Done Sent false
1100 1226 Acked Write 9 Done Done Sent false
1101 1227 Acked Write 8 Done Done Sent false
1102 1228 Acked Read 9 Done Done Sent false
1103 1229 Acked Read 4 Done Done Sent false
1104 1230 Acked Write 2 Done Done Sent false
1105 1231 Acked Read 8 Done Done Sent false
1106 1232 Acked Write 8 Done Done Sent false
1107 1233 Acked Flush 0 Done Done Sent false
1108 1234 Acked Read 7 Done Done Sent false
1109 1235 Acked Read 6 Done Done Sent false
1110 1236 Acked Read 7 Done Done Sent false
1111 1237 Acked Read 1 Done Done Sent false
1112 1238 Acked Write 1 Done Done Sent false
1113 1239 Acked Flush 0 Done Done Sent false
1114 1240 Acked Flush 0 Done Done Sent false
1115 1241 Acked Write 10 Done Done Sent false
1116 1242 Acked Write 1 Done Done Sent false
1117 1243 Acked Write 10 Done Done Sent false
1118 1244 Acked Write 1 Done Done Sent false
1119 1245 Acked Write 7 Done Done Sent false
1120 1246 Acked Flush 0 Done Done Sent false
1121 1247 Acked Write 10 Done Done Sent false
1122 1248 Acked Write 5 Done Done Sent false
1123 1249 Acked Read 4 Done Done Sent false
1124 1250 Acked Read 5 Done Done Sent false
1125 1251 Acked Read 4 Done Done Sent false
1126 1252 Acked Write 7 Done Done Sent false
1127 1253 Acked Read 10 Done Done Sent false
1128 1254 Acked Write 4 Done Done Sent false
1129 1255 Acked Read 3 Done Done Sent false
1130 1256 Acked Read 4 Done Done Sent false
1131 1257 Acked Read 9 Done Done Sent false
1132 1258 Acked Write 9 Done Done Sent false
1133 1259 Acked Read 10 Done Done Sent false
1134 1260 Acked Write 9 Done Done Sent false
1135 1261 Acked Write 4 Done Done Sent false
1136 1262 Acked Read 10 Done Done Sent false
1137 1263 Acked Write 3 Done Done Sent false
1138 1264 Acked Write 8 Done Done Sent false
1139 1265 Acked Write 3 Done Done Sent false
1140 1266 Acked Read 7 Done Done Sent false
1141 1267 Acked Read 9 Done Done Sent false
1142 1268 Acked Read 9 Done Done Sent false
1143 1269 Acked Write 5 Done Done Sent false
1144 1270 Acked Write 4 Done Done Sent false
1145 1271 Acked Read 6 Done Done Sent false
1146 1272 Acked Write 9 Done Done Sent false
1147 1273 Acked Read 10 Done Done Sent false
1148 1274 Acked Read 1 Done Done Sent false
1149 1275 Acked Write 7 Done Done Sent false
1150 1276 Acked Read 8 Done Done Sent false
1151 1277 Acked Write 3 Done Done Sent false
1152 1278 Acked Read 7 Done Done Sent false
1153 1279 Acked Write 9 Done Done Sent false
1154 1280 Acked Write 3 Done Done Sent false
1155 1281 Acked Flush 0 Done Done Sent false
1156 1282 Acked Write 1 Done Done Sent false
1157 1283 Acked Write 5 Done Done Sent false
1158 1284 Acked Write 5 Done Done Sent false
1159 1285 Acked Write 1 Done Done Sent false
1160 1286 Acked Read 3 Done Done Sent false
1161 1287 Acked Read 4 Done Done Sent false
1162 1288 Acked Write 8 Done Done Sent false
1163 1289 Acked Write 10 Done Done Sent false
1164 1290 Acked Write 7 Done Done Sent false
1165 1291 Acked Write 8 Done Done Sent false
1166 1292 Acked Read 5 Done Done Sent false
1167 1293 Acked Read 3 Done Done Sent false
1168 1294 Acked Write 1 Done Done Sent false
1169 1295 Acked Write 10 Done Done Sent false
1170 1296 Acked Write 7 Done Done Sent false
1171 1297 Acked Write 10 Done Done Sent false
1172 1298 Acked Write 1 Done Done Sent false
1173 1299 Acked Write 1 Done Done Sent false
1174 1300 Acked Write 10 Done Done Sent false
1175 1301 Acked Flush 0 Done Done Sent false
1176 STATES DS:0 DS:1 DS:2 TOTAL
1177 Sent 0 0 302 302
1178 Done 302 302 0 604
1179 Skipped 0 0 0 0
1180 Error 0 0 0 0
1181 Last Flush: 1301 1301 0
1182 Downstairs last five completed:
1183 Upstairs last five completed: 1301 1300 1299 1298 1297
1184 CLIENT: Up:0 ds:302 act:2
11852025-01-03T08:31:07.530ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11862025-01-03T08:31:07.530ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51187) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11872025-01-03T08:31:07.531ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11882025-01-03T08:31:07.532ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11892025-01-03T08:31:07.532ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11902025-01-03T08:31:07.532ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11912025-01-03T08:31:07.532ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11922025-01-03T08:31:07.532ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11932025-01-03T08:31:07.532ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11942025-01-03T08:31:07.532ZINFOcrucible: [2] client re-new 302 jobs since flush 0 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11952025-01-03T08:31:07.532ZINFOcrucible: [2] Replayed 302 jobs since flush: 0 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1196 ----------------------------------------------------------------
1197 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1198 Downstairs last five completed: 1301 1300 1299 1298 1297
1199 Upstairs last five completed: 1301 1300 1299 1298 1297
1200 CLIENT: Up:0 ds:0 act:3
1201 Replay: All jobs finished
1202 Main task has work to do, go find it
1203 got dsc Stop(0)
1204 stop 0
1205 Jan 03 08:31:09.775 INFO[8810] Got stop action so:false kr:false
1206 request completed, latency_us: 369, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 7ed99784-aa44-4fe7-923a-ef96356d8690, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
12072025-01-03T08:31:09.776ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
12082025-01-03T08:31:09.776ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
12092025-01-03T08:31:09.776ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
12102025-01-03T08:31:09.776ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
12112025-01-03T08:31:09.776ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
1212 Jan 03 08:31:09.776 INFO request completed, latency_us: 122, response_code: 200, uri: /state/cid/0, method: GET, req_id: 7f7ab4d7-2571-478a-b094-d9930744ab54, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:31:09.776668955Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
1213 
1214 [8810] Exited with: ExitStatus(unix_wait_status(9))
1215 [8810][0] reports Exit
1216 Jan 03 08:31:13.777 INFO request completed, latency_us: 190, response_code: 200, uri: /state/cid/0, method: GET, req_id: d7b4592d-7bed-4ff8-bcab-065862bf0e55, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
1217 001/300 Read block 829 len 24576
1218 002/300 Write block 3761 len 16384 data: 2 2 2 2
1219 003/300 Write block 1882 len 16384 data: 2 2 2 2
1220 004/300 Write block 3660 len 12288 data: 2 2 2
1221 005/300 Read block 2125 len 28672
1222 006/300 Write block 3267 len 20480 data: 2 2 2 2 2
1223 007/300 Write block 419 len 16384 data: 2 2 2 2
1224 008/300 Read block 2473 len 4096
1225 009/300 Write block 3662 len 12288 data: 3 2 2
1226 010/300 Flush
1227 011/300 Read block 3632 len 12288
1228 012/300 Read block 2178 len 8192
1229 013/300 Flush
1230 014/300 Flush
1231 015/300 Read block 1354 len 40960
1232 016/300 Write block 4199 len 36864 data: 3 3 3 3 2 3 3 3 2
1233 017/300 Write block 876 len 32768 data: 2 2 2 2 2 2 2 2
1234 018/300 Write block 2743 len 40960 data: 3 3 3 3 3 3 3 2 2 2
1235 019/300 Read block 802 len 24576
1236 020/300 Read block 2557 len 20480
1237 021/300 Write block 2443 len 24576 data: 2 2 2 2 2 2
1238 022/300 Read block 2297 len 40960
1239 023/300 Write block 4441 len 40960 data: 3 3 3 3 3 3 3 3 2 2
1240 024/300 Read block 4022 len 8192
1241 025/300 Read block 2475 len 36864
1242 026/300 Write block 4690 len 16384 data: 2 2 2 2
1243 027/300 Read block 1572 len 32768
1244 028/300 Write block 986 len 28672 data: 2 2 2 2 2 2 2
1245 029/300 Write block 2246 len 24576 data: 2 2 2 2 2 2
1246 030/300 Read block 2470 len 8192
1247 031/300 Write block 2695 len 28672 data: 2 2 2 2 2 2 2
1248 032/300 Flush
1249 033/300 Read block 2751 len 36864
1250 034/300 Write block 853 len 36864 data: 2 2 2 2 2 2 2 2 2
1251 035/300 Read block 1740 len 20480
1252 036/300 Read block 1530 len 32768
1253 037/300 Flush
1254 038/300 Write block 709 len 24576 data: 3 3 3 3 3 3
1255 039/300 Read block 2150 len 8192
1256 040/300 Read block 4563 len 28672
1257 041/300 Read block 952 len 12288
1258 042/300 Read block 2787 len 36864
1259 043/300 Read block 3641 len 20480
1260 044/300 Read block 3892 len 32768
1261 045/300 Write block 2625 len 32768 data: 2 2 2 3 3 2 2 2
1262 046/300 Write block 3389 len 16384 data: 2 2 2 2
1263 047/300 Read block 326 len 40960
1264 048/300 Read block 2336 len 16384
1265 049/300 Read block 3900 len 8192
1266 050/300 Write block 249 len 28672 data: 2 2 2 2 2 2 2
1267 051/300 Read block 643 len 32768
1268 052/300 Flush
1269 053/300 Read block 772 len 20480
1270 054/300 Read block 982 len 20480
1271 055/300 Write block 2327 len 24576 data: 2 2 2 2 2 2
1272 056/300 Read block 892 len 28672
1273 057/300 Write block 1517 len 16384 data: 2 2 2 2
1274 058/300 Read block 88 len 4096
1275 059/300 Write block 2937 len 24576 data: 2 2 2 2 2 2
1276 060/300 Write block 3985 len 20480 data: 2 2 2 2 2
1277 061/300 Write block 759 len 8192 data: 2 2
1278 062/300 Write block 137 len 36864 data: 2 2 2 2 2 2 2 2 2
1279 063/300 Read block 4180 len 4096
1280 064/300 Read block 1096 len 24576
1281 065/300 Write block 30 len 8192 data: 2 2
1282 066/300 Flush
1283 067/300 Read block 1239 len 20480
1284 068/300 Read block 4284 len 12288
1285 069/300 Write block 4707 len 20480 data: 3 2 2 2 2
1286 070/300 Read block 4777 len 20480
1287 071/300 Read block 1540 len 28672
1288 072/300 Read block 1016 len 4096
1289 073/300 Write block 824 len 32768 data: 2 2 2 2 2 2 2 2
1290 074/300 Read block 3791 len 20480
1291 075/300 Read block 4644 len 12288
1292 076/300 Flush
1293 077/300 Write block 4466 len 4096 data: 2
1294 078/300 Write block 3171 len 36864 data: 2 2 2 2 2 2 2 2 2
1295 079/300 Read block 350 len 40960
1296 080/300 Write block 1537 len 32768 data: 2 2 2 2 2 2 2 2
1297 081/300 Read block 420 len 20480
1298 082/300 Read block 1959 len 16384
1299 083/300 Read block 1867 len 28672
1300 084/300 Read block 1895 len 12288
1301 085/300 Write block 3666 len 16384 data: 2 2 2 2
1302 086/300 Write block 4907 len 12288 data: 2 2 2
1303 087/300 Read block 3435 len 16384
1304 088/300 Write block 4166 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1305 089/300 Write block 1326 len 16384 data: 2 2 2 2
1306 090/300 Write block 799 len 4096 data: 2
1307 091/300 Write block 4509 len 28672 data: 2 2 2 2 2 2 2
1308 092/300 Read block 3712 len 32768
1309 093/300 Write block 2757 len 32768 data: 3 3 3 3 3 3 3 2
1310 094/300 Write block 4365 len 12288 data: 2 2 2
1311 095/300 Read block 2641 len 20480
1312 096/300 Read block 1520 len 8192
1313 097/300 Write block 4211 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1314 098/300 Write block 4741 len 4096 data: 2
1315 099/300 Flush
1316 100/300 Read block 4485 len 32768
1317 101/300 Read block 800 len 12288
1318 102/300 Read block 4229 len 16384
1319 103/300 Read block 236 len 24576
1320 104/300 Read block 2452 len 16384
1321 105/300 Write block 3814 len 16384 data: 2 2 2 2
1322 106/300 Write block 3850 len 24576 data: 2 2 3 4 3 3
1323 107/300 Read block 4126 len 8192
1324 108/300 Flush
1325 109/300 Write block 2161 len 16384 data: 2 2 2 2
1326 110/300 Read block 791 len 20480
1327 111/300 Write block 1946 len 8192 data: 2 2
1328 112/300 Read block 280 len 40960
1329 113/300 Read block 1494 len 28672
1330 114/300 Read block 3158 len 8192
1331 115/300 Read block 2041 len 16384
1332 116/300 Read block 4840 len 16384
1333 117/300 Write block 4003 len 16384 data: 2 2 2 2
1334 118/300 Write block 2218 len 36864 data: 2 2 2 2 2 2 2 2 2
1335 119/300 Read block 300 len 20480
1336 120/300 Write block 301 len 36864 data: 2 2 2 2 2 2 2 2 2
1337 121/300 Write block 1387 len 8192 data: 2 2
1338 122/300 Read block 4363 len 20480
1339 123/300 Write block 3136 len 16384 data: 2 2 2 2
1340 124/300 Read block 4780 len 24576
1341 125/300 Read block 2710 len 12288
1342 126/300 Read block 383 len 20480
1343 127/300 Write block 72 len 40960 data: 3 3 2 2 2 2 2 2 2 2
1344 128/300 Read block 1341 len 24576
1345 129/300 Write block 3951 len 32768 data: 3 2 2 2 2 2 2 2
1346 130/300 Read block 1240 len 16384
1347 131/300 Read block 4483 len 16384
1348 132/300 Flush
1349 133/300 Read block 1483 len 16384
1350 134/300 Read block 2258 len 40960
1351 135/300 Read block 4984 len 40960
1352 136/300 Read block 863 len 24576
1353 137/300 Write block 4164 len 28672 data: 2 2 3 3 3 3 3
1354 138/300 Write block 2969 len 40960 data: 2 3 3 3 3 3 3 2 2 2
1355 139/300 Read block 1672 len 20480
1356 140/300 Write block 3359 len 8192 data: 2 2
1357 141/300 Write block 843 len 36864 data: 2 2 2 2 2 2 2 2 2
1358 142/300 Read block 2498 len 32768
1359 143/300 Write block 4983 len 8192 data: 2 2
1360 144/300 Write block 654 len 24576 data: 2 2 2 2 2 2
1361 145/300 Read block 2706 len 36864
1362 146/300 Flush
1363 147/300 Write block 3605 len 4096 data: 2
1364 148/300 Read block 803 len 4096
1365 149/300 Write block 4918 len 16384 data: 2 2 2 2
1366 150/300 Read block 3403 len 20480
1367 151/300 Read block 3643 len 36864
1368 152/300 Write block 672 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1369 153/300 Write block 3843 len 8192 data: 2 2
1370 154/300 Read block 3592 len 12288
1371 155/300 Write block 574 len 32768 data: 2 2 2 2 2 2 2 2
1372 156/300 Write block 3085 len 32768 data: 2 2 2 2 3 3 3 2
1373 157/300 Read block 1611 len 20480
1374 158/300 Read block 2225 len 32768
1375 159/300 Write block 2017 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1376 160/300 Write block 3258 len 32768 data: 2 2 2 2 2 2 2 2
1377 161/300 Write block 1414 len 20480 data: 2 2 2 2 2
1378 162/300 Read block 1336 len 8192
1379 163/300 Write block 3356 len 12288 data: 2 2 2
1380 164/300 Read block 4627 len 12288
1381 165/300 Write block 4141 len 20480 data: 3 3 3 3 2
1382 166/300 Write block 709 len 24576 data: 4 4 4 4 4 4
1383 167/300 Flush
1384 168/300 Read block 3578 len 16384
1385 169/300 Write block 1463 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1386 170/300 Read block 1155 len 12288
1387 171/300 Write block 1841 len 20480 data: 2 2 2 2 2
1388 172/300 Read block 2475 len 20480
1389 173/300 Read block 1163 len 16384
1390 174/300 Read block 3757 len 20480
1391 175/300 Read block 856 len 32768
1392 176/300 Write block 3280 len 4096 data: 2
1393 177/300 Flush
1394 178/300 Write block 4702 len 40960 data: 2 2 3 3 3 4 3 3 3 3
1395 179/300 Write block 2046 len 20480 data: 2 3 3 3 3
1396 180/300 Read block 1076 len 12288
1397 181/300 Read block 2386 len 16384
1398 182/300 Read block 10 len 24576
1399 183/300 Flush
1400 184/300 Write block 2609 len 16384 data: 2 2 2 2
1401 185/300 Read block 4973 len 12288
1402 186/300 Read block 1734 len 32768
1403 187/300 Read block 2153 len 20480
1404 188/300 Write block 556 len 36864 data: 2 2 2 2 2 2 2 2 2
1405 189/300 Read block 1559 len 8192
1406 190/300 Read block 760 len 20480
1407 191/300 Read block 50 len 24576
1408 192/300 Write block 4775 len 36864 data: 2 2 3 3 3 3 2 2 2
1409 193/300 Read block 818 len 12288
1410 194/300 Read block 3640 len 36864
1411 195/300 Write block 1278 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1412 196/300 Write block 2619 len 36864 data: 2 2 2 2 2 2 3 3 3
1413 197/300 Write block 2857 len 12288 data: 2 2 2
1414 198/300 Read block 3869 len 8192
1415 199/300 Read block 184 len 8192
1416 200/300 Read block 1603 len 16384
1417 201/300 Flush
1418 202/300 Read block 3811 len 4096
1419 203/300 Read block 2321 len 40960
1420 204/300 Write block 3132 len 16384 data: 2 2 2 2
1421 205/300 Read block 2743 len 40960
1422 206/300 Write block 1955 len 20480 data: 2 2 2 2 2
1423 207/300 Read block 3543 len 12288
1424 208/300 Write block 4972 len 12288 data: 2 2 2
1425 209/300 Read block 3309 len 16384
1426 210/300 Read block 2081 len 4096
1427 211/300 Write block 4111 len 36864 data: 2 2 2 2 2 2 2 2 2
1428 212/300 Read block 3679 len 20480
1429 213/300 Read block 187 len 20480
1430 214/300 Read block 4400 len 12288
1431 215/300 Read block 2808 len 36864
1432 216/300 Write block 3564 len 8192 data: 2 2
1433 217/300 Read block 2316 len 12288
1434 218/300 Write block 885 len 4096 data: 2
1435 219/300 Write block 3015 len 36864 data: 2 2 2 2 2 2 2 2 2
1436 220/300 Flush
1437 221/300 Read block 4273 len 40960
1438 222/300 Write block 3799 len 24576 data: 2 2 2 2 2 2
1439 223/300 Read block 2678 len 36864
1440 224/300 Read block 541 len 4096
1441 225/300 Write block 3716 len 32768 data: 2 2 2 2 2 2 2 2
1442 226/300 Write block 2153 len 24576 data: 2 2 2 2 2 2
1443 227/300 Write block 2833 len 12288 data: 2 2 2
1444 228/300 Read block 99 len 40960
1445 229/300 Read block 1206 len 8192
1446 230/300 Read block 4435 len 28672
1447 231/300 Read block 3269 len 24576
1448 232/300 Write block 3431 len 20480 data: 2 2 2 2 3
1449 233/300 Read block 3805 len 12288
1450 234/300 Read block 1486 len 36864
1451 235/300 Read block 2368 len 4096
1452 236/300 Read block 3899 len 28672
1453 237/300 Flush
1454 238/300 Write block 3585 len 20480 data: 2 2 2 2 3
1455 239/300 Write block 4137 len 4096 data: 2
1456 240/300 Write block 1049 len 36864 data: 2 2 2 2 2 2 2 2 2
1457 241/300 Read block 975 len 16384
1458 242/300 Write block 4622 len 24576 data: 2 2 2 2 2 2
1459 243/300 Write block 3824 len 40960 data: 2 3 3 3 3 2 2 2 2 2
1460 244/300 Read block 877 len 32768
1461 245/300 Write block 378 len 4096 data: 2
1462 246/300 Write block 514 len 32768 data: 2 2 2 2 2 2 2 2
1463 247/300 Read block 1629 len 24576
1464 248/300 Flush
1465 249/300 Read block 1894 len 20480
1466 250/300 Read block 2531 len 40960
1467 251/300 Write block 2568 len 4096 data: 2
1468 252/300 Write block 3183 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1469 253/300 Read block 2654 len 12288
1470 254/300 Read block 3661 len 40960
1471 255/300 Read block 2701 len 12288
1472 256/300 Write block 2805 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1473 257/300 Write block 3973 len 32768 data: 2 2 2 2 2 2 2 2
1474 258/300 Read block 4780 len 40960
1475 259/300 Flush
1476 260/300 Write block 1572 len 28672 data: 2 2 2 2 2 3 2
1477 261/300 Read block 3912 len 40960
1478 262/300 Read block 4138 len 36864
1479 263/300 Read block 1379 len 12288
1480 264/300 Write block 1987 len 24576 data: 2 2 2 2 2 3
1481 265/300 Read block 2325 len 40960
1482 266/300 Read block 4874 len 32768
1483 267/300 Write block 517 len 32768 data: 3 3 3 3 3 2 2 2
1484 268/300 Flush
1485 269/300 Write block 489 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1486 270/300 Write block 2825 len 12288 data: 3 4 3
1487 271/300 Read block 4219 len 36864
1488 272/300 Read block 811 len 24576
1489 273/300 Read block 3573 len 12288
1490 274/300 Read block 662 len 32768
1491 275/300 Write block 160 len 16384 data: 2 2 2 2
1492 276/300 Write block 1874 len 20480 data: 2 2 2 2 2
1493 277/300 Read block 3033 len 4096
1494 278/300 Write block 1089 len 28672 data: 3 3 3 3 4 4 4
1495 279/300 Flush
1496 280/300 Read block 3106 len 36864
1497 281/300 Write block 2472 len 20480 data: 2 2 2 2 2
1498 282/300 Write block 326 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1499 283/300 Read block 2775 len 32768
1500 284/300 Write block 2111 len 28672 data: 2 2 2 2 2 2 2
1501 285/300 Write block 3909 len 24576 data: 3 2 2 2 2 2
1502 286/300 Write block 826 len 24576 data: 3 3 3 3 3 3
1503 287/300 Read block 2277 len 32768
1504 288/300 Read block 849 len 20480
1505 289/300 Write block 4821 len 8192 data: 2 2
1506 290/300 Flush
1507 291/300 Read block 4693 len 32768
1508 292/300 Write block 365 len 32768 data: 2 2 2 2 2 2 2 2
1509 293/300 Read block 1224 len 28672
1510 294/300 Write block 3821 len 16384 data: 2 2 2 3
1511 295/300 Write block 1237 len 4096 data: 2
1512 296/300 Read block 859 len 28672
1513 297/300 Read block 1151 len 24576
1514 298/300 Flush
1515 299/300 Write block 172 len 16384 data: 4 4 4 4
1516 300/300 Write block 4344 len 28672 data: 2 2 2 2 2 2 2
1517 301/300 Write block 4765 len 28672 data: 2 2 2 2 2 2 2
1518 Main task has work to do, go find it
1519 got dsc Start(0)
1520 start 0
1521 Jan 03 08:31:14.021 INFO request completed, latency_us: 240, response_code: 204, uri: /start/cid/0, method: POST, req_id: 9ae36f87-c303-4b25-b745-6f3fb4e29b6a, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
1522 Replay: started 0, returned:Ok(())
1523 ----------------------------------------------------------------
1524 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1525 States: Offline Active Active
1526 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1527 1302 Acked Read 6 Sent Done Done false
1528 1303 Acked Write 4 Sent Done Done false
1529 1304 Acked Write 4 Sent Done Done false
1530 1305 Acked Write 3 Sent Done Done false
1531 1306 Acked Read 7 Sent Done Done false
1532 1307 Acked Write 5 Sent Done Done false
1533 1308 Acked Write 4 Sent Done Done false
1534 1309 Acked Read 1 Sent Done Done false
1535 1310 Acked Write 3 Sent Done Done false
1536 1311 Acked Flush 0 Sent Done Done false
1537 1312 Acked Read 3 Sent Done Done false
1538 1313 Acked Read 2 Sent Done Done false
1539 1314 Acked Flush 0 Sent Done Done false
1540 1315 Acked Flush 0 Sent Done Done false
1541 1316 Acked Read 10 Sent Done Done false
1542 1317 Acked Write 9 Sent Done Done false
1543 1318 Acked Write 8 Sent Done Done false
1544 1319 Acked Write 10 Sent Done Done false
1545 1320 Acked Read 6 Sent Done Done false
1546 1321 Acked Read 5 Sent Done Done false
1547 1322 Acked Write 6 Sent Done Done false
1548 1323 Acked Read 10 Sent Done Done false
1549 1324 Acked Write 10 Sent Done Done false
1550 1325 Acked Read 2 Sent Done Done false
1551 1326 Acked Read 9 Sent Done Done false
1552 1327 Acked Write 4 Sent Done Done false
1553 1328 Acked Read 8 Sent Done Done false
1554 1329 Acked Write 7 Sent Done Done false
1555 1330 Acked Write 6 Sent Done Done false
1556 1331 Acked Read 2 Sent Done Done false
1557 1332 Acked Write 7 Sent Done Done false
1558 1333 Acked Flush 0 Sent Done Done false
1559 1334 Acked Read 9 Sent Done Done false
1560 1335 Acked Write 9 Sent Done Done false
1561 1336 Acked Read 5 Sent Done Done false
1562 1337 Acked Read 8 Sent Done Done false
1563 1338 Acked Flush 0 Sent Done Done false
1564 1339 Acked Write 6 Sent Done Done false
1565 1340 Acked Read 2 Sent Done Done false
1566 1341 Acked Read 7 Sent Done Done false
1567 1342 Acked Read 3 Sent Done Done false
1568 1343 Acked Read 9 Sent Done Done false
1569 1344 Acked Read 5 Sent Done Done false
1570 1345 Acked Read 8 Sent Done Done false
1571 1346 Acked Write 8 Sent Done Done false
1572 1347 Acked Write 4 Sent Done Done false
1573 1348 Acked Read 10 Sent Done Done false
1574 1349 Acked Read 4 Sent Done Done false
1575 1350 Acked Read 2 Sent Done Done false
1576 1351 Acked Write 7 Sent Done Done false
1577 1352 Acked Read 8 Sent Done Done false
1578 1353 Acked Flush 0 Sent Done Done false
1579 1354 Acked Read 5 Sent Done Done false
1580 1355 Acked Read 5 Sent Done Done false
1581 1356 Acked Write 6 Sent Done Done false
1582 1357 Acked Read 7 Sent Done Done false
1583 1358 Acked Write 4 Sent Done Done false
1584 1359 Acked Read 1 Sent Done Done false
1585 1360 Acked Write 6 Sent Done Done false
1586 1361 Acked Write 5 Sent Done Done false
1587 1362 Acked Write 2 Sent Done Done false
1588 1363 Acked Write 9 Sent Done Done false
1589 1364 Acked Read 1 Sent Done Done false
1590 1365 Acked Read 6 Sent Done Done false
1591 1366 Acked Write 2 Sent Done Done false
1592 1367 Acked Flush 0 Sent Done Done false
1593 1368 Acked Read 5 Sent Done Done false
1594 1369 Acked Read 3 Sent Done Done false
1595 1370 Acked Write 5 Sent Done Done false
1596 1371 Acked Read 5 Sent Done Done false
1597 1372 Acked Read 7 Sent Done Done false
1598 1373 Acked Read 1 Sent Done Done false
1599 1374 Acked Write 8 Sent Done Done false
1600 1375 Acked Read 5 Sent Done Done false
1601 1376 Acked Read 3 Sent Done Done false
1602 1377 Acked Flush 0 Sent Done Done false
1603 1378 Acked Write 1 Sent Done Done false
1604 1379 Acked Write 9 Sent Done Done false
1605 1380 Acked Read 10 Sent Done Done false
1606 1381 Acked Write 8 Sent Done Done false
1607 1382 Acked Read 5 Sent Done Done false
1608 1383 Acked Read 4 Sent Done Done false
1609 1384 Acked Read 7 Sent Done Done false
1610 1385 Acked Read 3 Sent Done Done false
1611 1386 Acked Write 4 Sent Done Done false
1612 1387 Acked Write 3 Sent Done Done false
1613 1388 Acked Read 4 Sent Done Done false
1614 1389 Acked Write 10 Sent Done Done false
1615 1390 Acked Write 4 Sent Done Done false
1616 1391 Acked Write 1 Sent Done Done false
1617 1392 Acked Write 7 Sent Done Done false
1618 1393 Acked Read 8 Sent Done Done false
1619 1394 Acked Write 8 Sent Done Done false
1620 1395 Acked Write 3 Sent Done Done false
1621 1396 Acked Read 5 Sent Done Done false
1622 1397 Acked Read 2 Sent Done Done false
1623 1398 Acked Write 10 Sent Done Done false
1624 1399 Acked Write 1 Sent Done Done false
1625 1400 Acked Flush 0 Sent Done Done false
1626 1401 Acked Read 8 Sent Done Done false
1627 1402 Acked Read 3 Sent Done Done false
1628 1403 Acked Read 4 Sent Done Done false
1629 1404 Acked Read 6 Sent Done Done false
1630 1405 Acked Read 4 Sent Done Done false
1631 1406 Acked Write 4 Sent Done Done false
1632 1407 Acked Write 6 Sent Done Done false
1633 1408 Acked Read 2 Sent Done Done false
1634 1409 Acked Flush 0 Sent Done Done false
1635 1410 Acked Write 4 Sent Done Done false
1636 1411 Acked Read 5 Sent Done Done false
1637 1412 Acked Write 2 Sent Done Done false
1638 1413 Acked Read 10 Sent Done Done false
1639 1414 Acked Read 7 Sent Done Done false
1640 1415 Acked Read 2 Sent Done Done false
1641 1416 Acked Read 4 Sent Done Done false
1642 1417 Acked Read 4 Sent Done Done false
1643 1418 Acked Write 4 Sent Done Done false
1644 1419 Acked Write 9 Sent Done Done false
1645 1420 Acked Read 5 Sent Done Done false
1646 1421 Acked Write 9 Sent Done Done false
1647 1422 Acked Write 2 Sent Done Done false
1648 1423 Acked Read 5 Sent Done Done false
1649 1424 Acked Write 4 Sent Done Done false
1650 1425 Acked Read 6 Sent Done Done false
1651 1426 Acked Read 3 Sent Done Done false
1652 1427 Acked Read 5 Sent Done Done false
1653 1428 Acked Write 10 Sent Done Done false
1654 1429 Acked Read 6 Sent Done Done false
1655 1430 Acked Write 8 Sent Done Done false
1656 1431 Acked Read 4 Sent Done Done false
1657 1432 Acked Read 4 Sent Done Done false
1658 1433 Acked Flush 0 Sent Done Done false
1659 1434 Acked Read 4 Sent Done Done false
1660 1435 Acked Read 10 Sent Done Done false
1661 1436 Acked Read 10 Sent Done Done false
1662 1437 Acked Read 6 Sent Done Done false
1663 1438 Acked Write 7 Sent Done Done false
1664 1439 Acked Write 10 Sent Done Done false
1665 1440 Acked Read 5 Sent Done Done false
1666 1441 Acked Write 2 Sent Done Done false
1667 1442 Acked Write 9 Sent Done Done false
1668 1443 Acked Read 8 Sent Done Done false
1669 1444 Acked Write 2 Sent Done Done false
1670 1445 Acked Write 6 Sent Done Done false
1671 1446 Acked Read 9 Sent Done Done false
1672 1447 Acked Flush 0 Sent Done Done false
1673 1448 Acked Write 1 Sent Done Done false
1674 1449 Acked Read 1 Sent Done Done false
1675 1450 Acked Write 4 Sent Done Done false
1676 1451 Acked Read 5 Sent Done Done false
1677 1452 Acked Read 9 Sent Done Done false
1678 1453 Acked Write 10 Sent Done Done false
1679 1454 Acked Write 2 Sent Done Done false
1680 1455 Acked Read 3 Sent Done Done false
1681 1456 Acked Write 8 Sent Done Done false
1682 1457 Acked Write 8 Sent Done Done false
1683 1458 Acked Read 5 Sent Done Done false
1684 1459 Acked Read 8 Sent Done Done false
1685 1460 Acked Write 10 Sent Done Done false
1686 1461 Acked Write 8 Sent Done Done false
1687 1462 Acked Write 5 Sent Done Done false
1688 1463 Acked Read 2 Sent Done Done false
1689 1464 Acked Write 3 Sent Done Done false
1690 1465 Acked Read 3 Sent Done Done false
1691 1466 Acked Write 5 Sent Done Done false
1692 1467 Acked Write 6 Sent Done Done false
1693 1468 Acked Flush 0 Sent Done Done false
1694 1469 Acked Read 4 Sent Done Done false
1695 1470 Acked Write 10 Sent Done Done false
1696 1471 Acked Read 3 Sent Done Done false
1697 1472 Acked Write 5 Sent Done Done false
1698 1473 Acked Read 5 Sent Done Done false
1699 1474 Acked Read 4 Sent Done Done false
1700 1475 Acked Read 5 Sent Done Done false
1701 1476 Acked Read 8 Sent Done Done false
1702 1477 Acked Write 1 Sent Done Done false
1703 1478 Acked Flush 0 Sent Done Done false
1704 1479 Acked Write 10 Sent Done Done false
1705 1480 Acked Write 5 Sent Done Done false
1706 1481 Acked Read 3 Sent Done Done false
1707 1482 Acked Read 4 Sent Done Done false
1708 1483 Acked Read 6 Sent Done Done false
1709 1484 Acked Flush 0 Sent Done Done false
1710 1485 Acked Write 4 Sent Done Done false
1711 1486 Acked Read 3 Sent Done Done false
1712 1487 Acked Read 8 Sent Done Done false
1713 1488 Acked Read 5 Sent Done Done false
1714 1489 Acked Write 9 Sent Done Done false
1715 1490 Acked Read 2 Sent Done Done false
1716 1491 Acked Read 5 Sent Done Done false
1717 1492 Acked Read 6 Sent Done Done false
1718 1493 Acked Write 9 Sent Done Done false
1719 1494 Acked Read 3 Sent Done Done false
1720 1495 Acked Read 9 Sent Done Done false
1721 1496 Acked Write 10 Sent Done Done false
1722 1497 Acked Write 9 Sent Done Done false
1723 1498 Acked Write 3 Sent Done Done false
1724 1499 Acked Read 2 Sent Done Done false
1725 1500 Acked Read 2 Sent Done Done false
1726 1501 Acked Read 4 Sent Done Done false
1727 1502 Acked Flush 0 Sent Done Done false
1728 1503 Acked Read 1 Sent Done Done false
1729 1504 Acked Read 10 Sent Done Done false
1730 1505 Acked Write 4 Sent Done Done false
1731 1506 Acked Read 10 Sent Done Done false
1732 1507 Acked Write 5 Sent Done Done false
1733 1508 Acked Read 3 Sent Done Done false
1734 1509 Acked Write 3 Sent Done Done false
1735 1510 Acked Read 4 Sent Done Done false
1736 1511 Acked Read 1 Sent Done Done false
1737 1512 Acked Write 9 Sent Done Done false
1738 1513 Acked Read 5 Sent Done Done false
1739 1514 Acked Read 5 Sent Done Done false
1740 1515 Acked Read 3 Sent Done Done false
1741 1516 Acked Read 9 Sent Done Done false
1742 1517 Acked Write 2 Sent Done Done false
1743 1518 Acked Read 3 Sent Done Done false
1744 1519 Acked Write 1 Sent Done Done false
1745 1520 Acked Write 9 Sent Done Done false
1746 1521 Acked Flush 0 Sent Done Done false
1747 1522 Acked Read 10 Sent Done Done false
1748 1523 Acked Write 6 Sent Done Done false
1749 1524 Acked Read 9 Sent Done Done false
1750 1525 Acked Read 1 Sent Done Done false
1751 1526 Acked Write 8 Sent Done Done false
1752 1527 Acked Write 6 Sent Done Done false
1753 1528 Acked Write 3 Sent Done Done false
1754 1529 Acked Read 10 Sent Done Done false
1755 1530 Acked Read 2 Sent Done Done false
1756 1531 Acked Read 7 Sent Done Done false
1757 1532 Acked Read 6 Sent Done Done false
1758 1533 Acked Write 5 Sent Done Done false
1759 1534 Acked Read 3 Sent Done Done false
1760 1535 Acked Read 9 Sent Done Done false
1761 1536 Acked Read 1 Sent Done Done false
1762 1537 Acked Read 7 Sent Done Done false
1763 1538 Acked Flush 0 Sent Done Done false
1764 1539 Acked Write 5 Sent Done Done false
1765 1540 Acked Write 1 Sent Done Done false
1766 1541 Acked Write 9 Sent Done Done false
1767 1542 Acked Read 4 Sent Done Done false
1768 1543 Acked Write 6 Sent Done Done false
1769 1544 Acked Write 10 Sent Done Done false
1770 1545 Acked Read 8 Sent Done Done false
1771 1546 Acked Write 1 Sent Done Done false
1772 1547 Acked Write 8 Sent Done Done false
1773 1548 Acked Read 6 Sent Done Done false
1774 1549 Acked Flush 0 Sent Done Done false
1775 1550 Acked Read 5 Sent Done Done false
1776 1551 Acked Read 10 Sent Done Done false
1777 1552 Acked Write 1 Sent Done Done false
1778 1553 Acked Write 10 Sent Done Done false
1779 1554 Acked Read 3 Sent Done Done false
1780 1555 Acked Read 10 Sent Done Done false
1781 1556 Acked Read 3 Sent Done Done false
1782 1557 Acked Write 10 Sent Done Done false
1783 1558 Acked Write 8 Sent Done Done false
1784 1559 Acked Read 10 Sent Done Done false
1785 1560 Acked Flush 0 Sent Done Done false
1786 1561 Acked Write 7 Sent Done Done false
1787 1562 Acked Read 10 Sent Done Done false
1788 1563 Acked Read 9 Sent Done Done false
1789 1564 Acked Read 3 Sent Done Done false
1790 1565 Acked Write 6 Sent Done Done false
1791 1566 Acked Read 10 Sent Done Done false
1792 1567 Acked Read 8 Sent Done Done false
1793 1568 Acked Write 8 Sent Done Done false
1794 1569 Acked Flush 0 Sent Done Done false
1795 1570 Acked Write 10 Sent Done Done false
1796 1571 Acked Write 3 Sent Done Done false
1797 1572 Acked Read 9 Sent Done Done false
1798 1573 Acked Read 6 Sent Done Done false
1799 1574 Acked Read 3 Sent Done Done false
1800 1575 Acked Read 8 Sent Done Done false
1801 1576 Acked Write 4 Sent Done Done false
1802 1577 Acked Write 5 Sent Done Done false
1803 1578 Acked Read 1 Sent Done Done false
1804 1579 Acked Write 7 Sent Done Done false
1805 1580 Acked Flush 0 Sent Done Done false
1806 1581 Acked Read 9 Sent Done Done false
1807 1582 Acked Write 5 Sent Done Done false
1808 1583 Acked Write 10 Sent Done Done false
1809 1584 Acked Read 8 Sent Done Done false
1810 1585 Acked Write 7 Sent Done Done false
1811 1586 Acked Write 6 Sent Done Done false
1812 1587 Acked Write 6 Sent Done Done false
1813 1588 Acked Read 8 Sent Done Done false
1814 1589 Acked Read 5 Sent Done Done false
1815 1590 Acked Write 2 Sent Done Done false
1816 1591 Acked Flush 0 Sent Done Done false
1817 1592 Acked Read 8 Sent Done Done false
1818 1593 Acked Write 8 Sent Done Done false
1819 1594 Acked Read 7 Sent Done Done false
1820 1595 Acked Write 4 Sent Done Done false
1821 1596 Acked Write 1 Sent Done Done false
1822 1597 Acked Read 7 Sent Done Done false
1823 1598 Acked Read 6 Sent Done Done false
1824 1599 Acked Flush 0 Sent Done Done false
1825 1600 Acked Write 4 Sent Done Done false
1826 1601 Acked Write 7 Sent Done Done false
1827 1602 Acked Write 7 Sent Done Sent false
1828 STATES DS:0 DS:1 DS:2 TOTAL
1829 Sent 301 0 1 302
1830 Done 0 301 300 601
1831 Skipped 0 0 0 0
1832 Error 0 0 0 0
1833 Last Flush: 1301 1599 1599
1834 Downstairs last five completed: 1301 1300 1299 1298 1297
1835 Upstairs last five completed: 1602 1601 1600 1599 1598
1836 CLIENT: Up:0 ds:301 act:2
1837 ----------------------------------------------------------------
1838 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1839 States: Offline Active Active
1840 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1841 1302 Acked Read 6 Sent Done Done false
1842 1303 Acked Write 4 Sent Done Done false
1843 1304 Acked Write 4 Sent Done Done false
1844 1305 Acked Write 3 Sent Done Done false
1845 1306 Acked Read 7 Sent Done Done false
1846 1307 Acked Write 5 Sent Done Done false
1847 1308 Acked Write 4 Sent Done Done false
1848 1309 Acked Read 1 Sent Done Done false
1849 1310 Acked Write 3 Sent Done Done false
1850 1311 Acked Flush 0 Sent Done Done false
1851 1312 Acked Read 3 Sent Done Done false
1852 1313 Acked Read 2 Sent Done Done false
1853 1314 Acked Flush 0 Sent Done Done false
1854 1315 Acked Flush 0 Sent Done Done false
1855 1316 Acked Read 10 Sent Done Done false
1856 1317 Acked Write 9 Sent Done Done false
1857 1318 Acked Write 8 Sent Done Done false
1858 1319 Acked Write 10 Sent Done Done false
1859 1320 Acked Read 6 Sent Done Done false
1860 1321 Acked Read 5 Sent Done Done false
1861 1322 Acked Write 6 Sent Done Done false
1862 1323 Acked Read 10 Sent Done Done false
1863 1324 Acked Write 10 Sent Done Done false
1864 1325 Acked Read 2 Sent Done Done false
1865 1326 Acked Read 9 Sent Done Done false
1866 1327 Acked Write 4 Sent Done Done false
1867 1328 Acked Read 8 Sent Done Done false
1868 1329 Acked Write 7 Sent Done Done false
1869 1330 Acked Write 6 Sent Done Done false
1870 1331 Acked Read 2 Sent Done Done false
1871 1332 Acked Write 7 Sent Done Done false
1872 1333 Acked Flush 0 Sent Done Done false
1873 1334 Acked Read 9 Sent Done Done false
1874 1335 Acked Write 9 Sent Done Done false
1875 1336 Acked Read 5 Sent Done Done false
1876 1337 Acked Read 8 Sent Done Done false
1877 1338 Acked Flush 0 Sent Done Done false
1878 1339 Acked Write 6 Sent Done Done false
1879 1340 Acked Read 2 Sent Done Done false
1880 1341 Acked Read 7 Sent Done Done false
1881 1342 Acked Read 3 Sent Done Done false
1882 1343 Acked Read 9 Sent Done Done false
1883 1344 Acked Read 5 Sent Done Done false
1884 1345 Acked Read 8 Sent Done Done false
1885 1346 Acked Write 8 Sent Done Done false
1886 1347 Acked Write 4 Sent Done Done false
1887 1348 Acked Read 10 Sent Done Done false
1888 1349 Acked Read 4 Sent Done Done false
1889 1350 Acked Read 2 Sent Done Done false
1890 1351 Acked Write 7 Sent Done Done false
1891 1352 Acked Read 8 Sent Done Done false
1892 1353 Acked Flush 0 Sent Done Done false
1893 1354 Acked Read 5 Sent Done Done false
1894 1355 Acked Read 5 Sent Done Done false
1895 1356 Acked Write 6 Sent Done Done false
1896 1357 Acked Read 7 Sent Done Done false
1897 1358 Acked Write 4 Sent Done Done false
1898 1359 Acked Read 1 Sent Done Done false
1899 1360 Acked Write 6 Sent Done Done false
1900 1361 Acked Write 5 Sent Done Done false
1901 1362 Acked Write 2 Sent Done Done false
1902 1363 Acked Write 9 Sent Done Done false
1903 1364 Acked Read 1 Sent Done Done false
1904 1365 Acked Read 6 Sent Done Done false
1905 1366 Acked Write 2 Sent Done Done false
1906 1367 Acked Flush 0 Sent Done Done false
1907 1368 Acked Read 5 Sent Done Done false
1908 1369 Acked Read 3 Sent Done Done false
1909 1370 Acked Write 5 Sent Done Done false
1910 1371 Acked Read 5 Sent Done Done false
1911 1372 Acked Read 7 Sent Done Done false
1912 1373 Acked Read 1 Sent Done Done false
1913 1374 Acked Write 8 Sent Done Done false
1914 1375 Acked Read 5 Sent Done Done false
1915 1376 Acked Read 3 Sent Done Done false
1916 1377 Acked Flush 0 Sent Done Done false
1917 1378 Acked Write 1 Sent Done Done false
1918 1379 Acked Write 9 Sent Done Done false
1919 1380 Acked Read 10 Sent Done Done false
1920 1381 Acked Write 8 Sent Done Done false
1921 1382 Acked Read 5 Sent Done Done false
1922 1383 Acked Read 4 Sent Done Done false
1923 1384 Acked Read 7 Sent Done Done false
1924 1385 Acked Read 3 Sent Done Done false
1925 1386 Acked Write 4 Sent Done Done false
1926 1387 Acked Write 3 Sent Done Done false
1927 1388 Acked Read 4 Sent Done Done false
1928 1389 Acked Write 10 Sent Done Done false
1929 1390 Acked Write 4 Sent Done Done false
1930 1391 Acked Write 1 Sent Done Done false
1931 1392 Acked Write 7 Sent Done Done false
1932 1393 Acked Read 8 Sent Done Done false
1933 1394 Acked Write 8 Sent Done Done false
1934 1395 Acked Write 3 Sent Done Done false
1935 1396 Acked Read 5 Sent Done Done false
1936 1397 Acked Read 2 Sent Done Done false
1937 1398 Acked Write 10 Sent Done Done false
1938 1399 Acked Write 1 Sent Done Done false
1939 1400 Acked Flush 0 Sent Done Done false
1940 1401 Acked Read 8 Sent Done Done false
1941 1402 Acked Read 3 Sent Done Done false
1942 1403 Acked Read 4 Sent Done Done false
1943 1404 Acked Read 6 Sent Done Done false
1944 1405 Acked Read 4 Sent Done Done false
1945 1406 Acked Write 4 Sent Done Done false
1946 1407 Acked Write 6 Sent Done Done false
1947 1408 Acked Read 2 Sent Done Done false
1948 1409 Acked Flush 0 Sent Done Done false
1949 1410 Acked Write 4 Sent Done Done false
1950 1411 Acked Read 5 Sent Done Done false
1951 1412 Acked Write 2 Sent Done Done false
1952 1413 Acked Read 10 Sent Done Done false
1953 1414 Acked Read 7 Sent Done Done false
1954 1415 Acked Read 2 Sent Done Done false
1955 1416 Acked Read 4 Sent Done Done false
1956 1417 Acked Read 4 Sent Done Done false
1957 1418 Acked Write 4 Sent Done Done false
1958 1419 Acked Write 9 Sent Done Done false
1959 1420 Acked Read 5 Sent Done Done false
1960 1421 Acked Write 9 Sent Done Done false
1961 1422 Acked Write 2 Sent Done Done false
1962 1423 Acked Read 5 Sent Done Done false
1963 1424 Acked Write 4 Sent Done Done false
1964 1425 Acked Read 6 Sent Done Done false
1965 1426 Acked Read 3 Sent Done Done false
1966 1427 Acked Read 5 Sent Done Done false
1967 1428 Acked Write 10 Sent Done Done false
1968 1429 Acked Read 6 Sent Done Done false
1969 1430 Acked Write 8 Sent Done Done false
1970 1431 Acked Read 4 Sent Done Done false
1971 1432 Acked Read 4 Sent Done Done false
1972 1433 Acked Flush 0 Sent Done Done false
1973 1434 Acked Read 4 Sent Done Done false
1974 1435 Acked Read 10 Sent Done Done false
1975 1436 Acked Read 10 Sent Done Done false
1976 1437 Acked Read 6 Sent Done Done false
1977 1438 Acked Write 7 Sent Done Done false
1978 1439 Acked Write 10 Sent Done Done false
1979 1440 Acked Read 5 Sent Done Done false
1980 1441 Acked Write 2 Sent Done Done false
1981 1442 Acked Write 9 Sent Done Done false
1982 1443 Acked Read 8 Sent Done Done false
1983 1444 Acked Write 2 Sent Done Done false
1984 1445 Acked Write 6 Sent Done Done false
1985 1446 Acked Read 9 Sent Done Done false
1986 1447 Acked Flush 0 Sent Done Done false
1987 1448 Acked Write 1 Sent Done Done false
1988 1449 Acked Read 1 Sent Done Done false
1989 1450 Acked Write 4 Sent Done Done false
1990 1451 Acked Read 5 Sent Done Done false
1991 1452 Acked Read 9 Sent Done Done false
1992 1453 Acked Write 10 Sent Done Done false
1993 1454 Acked Write 2 Sent Done Done false
1994 1455 Acked Read 3 Sent Done Done false
1995 1456 Acked Write 8 Sent Done Done false
1996 1457 Acked Write 8 Sent Done Done false
1997 1458 Acked Read 5 Sent Done Done false
1998 1459 Acked Read 8 Sent Done Done false
1999 1460 Acked Write 10 Sent Done Done false
2000 1461 Acked Write 8 Sent Done Done false
2001 1462 Acked Write 5 Sent Done Done false
2002 1463 Acked Read 2 Sent Done Done false
2003 1464 Acked Write 3 Sent Done Done false
2004 1465 Acked Read 3 Sent Done Done false
2005 1466 Acked Write 5 Sent Done Done false
2006 1467 Acked Write 6 Sent Done Done false
2007 1468 Acked Flush 0 Sent Done Done false
2008 1469 Acked Read 4 Sent Done Done false
2009 1470 Acked Write 10 Sent Done Done false
2010 1471 Acked Read 3 Sent Done Done false
2011 1472 Acked Write 5 Sent Done Done false
2012 1473 Acked Read 5 Sent Done Done false
2013 1474 Acked Read 4 Sent Done Done false
2014 1475 Acked Read 5 Sent Done Done false
2015 1476 Acked Read 8 Sent Done Done false
2016 1477 Acked Write 1 Sent Done Done false
2017 1478 Acked Flush 0 Sent Done Done false
2018 1479 Acked Write 10 Sent Done Done false
2019 1480 Acked Write 5 Sent Done Done false
2020 1481 Acked Read 3 Sent Done Done false
2021 1482 Acked Read 4 Sent Done Done false
2022 1483 Acked Read 6 Sent Done Done false
2023 1484 Acked Flush 0 Sent Done Done false
2024 1485 Acked Write 4 Sent Done Done false
2025 1486 Acked Read 3 Sent Done Done false
2026 1487 Acked Read 8 Sent Done Done false
2027 1488 Acked Read 5 Sent Done Done false
2028 1489 Acked Write 9 Sent Done Done false
2029 1490 Acked Read 2 Sent Done Done false
2030 1491 Acked Read 5 Sent Done Done false
2031 1492 Acked Read 6 Sent Done Done false
2032 1493 Acked Write 9 Sent Done Done false
2033 1494 Acked Read 3 Sent Done Done false
2034 1495 Acked Read 9 Sent Done Done false
2035 1496 Acked Write 10 Sent Done Done false
2036 1497 Acked Write 9 Sent Done Done false
2037 1498 Acked Write 3 Sent Done Done false
2038 1499 Acked Read 2 Sent Done Done false
2039 1500 Acked Read 2 Sent Done Done false
2040 1501 Acked Read 4 Sent Done Done false
2041 1502 Acked Flush 0 Sent Done Done false
2042 1503 Acked Read 1 Sent Done Done false
2043 1504 Acked Read 10 Sent Done Done false
2044 1505 Acked Write 4 Sent Done Done false
2045 1506 Acked Read 10 Sent Done Done false
2046 1507 Acked Write 5 Sent Done Done false
2047 1508 Acked Read 3 Sent Done Done false
2048 1509 Acked Write 3 Sent Done Done false
2049 1510 Acked Read 4 Sent Done Done false
2050 1511 Acked Read 1 Sent Done Done false
2051 1512 Acked Write 9 Sent Done Done false
2052 1513 Acked Read 5 Sent Done Done false
2053 1514 Acked Read 5 Sent Done Done false
2054 1515 Acked Read 3 Sent Done Done false
2055 1516 Acked Read 9 Sent Done Done false
2056 1517 Acked Write 2 Sent Done Done false
2057 1518 Acked Read 3 Sent Done Done false
2058 1519 Acked Write 1 Sent Done Done false
2059 1520 Acked Write 9 Sent Done Done false
2060 1521 Acked Flush 0 Sent Done Done false
2061 1522 Acked Read 10 Sent Done Done false
2062 1523 Acked Write 6 Sent Done Done false
2063 1524 Acked Read 9 Sent Done Done false
2064 1525 Acked Read 1 Sent Done Done false
2065 1526 Acked Write 8 Sent Done Done false
2066 1527 Acked Write 6 Sent Done Done false
2067 1528 Acked Write 3 Sent Done Done false
2068 1529 Acked Read 10 Sent Done Done false
2069 1530 Acked Read 2 Sent Done Done false
2070 1531 Acked Read 7 Sent Done Done false
2071 1532 Acked Read 6 Sent Done Done false
2072 1533 Acked Write 5 Sent Done Done false
2073 1534 Acked Read 3 Sent Done Done false
2074 1535 Acked Read 9 Sent Done Done false
2075 1536 Acked Read 1 Sent Done Done false
2076 1537 Acked Read 7 Sent Done Done false
2077 1538 Acked Flush 0 Sent Done Done false
2078 1539 Acked Write 5 Sent Done Done false
2079 1540 Acked Write 1 Sent Done Done false
2080 1541 Acked Write 9 Sent Done Done false
2081 1542 Acked Read 4 Sent Done Done false
2082 1543 Acked Write 6 Sent Done Done false
2083 1544 Acked Write 10 Sent Done Done false
2084 1545 Acked Read 8 Sent Done Done false
2085 1546 Acked Write 1 Sent Done Done false
2086 1547 Acked Write 8 Sent Done Done false
2087 1548 Acked Read 6 Sent Done Done false
2088 1549 Acked Flush 0 Sent Done Done false
2089 1550 Acked Read 5 Sent Done Done false
2090 1551 Acked Read 10 Sent Done Done false
2091 1552 Acked Write 1 Sent Done Done false
2092 1553 Acked Write 10 Sent Done Done false
2093 1554 Acked Read 3 Sent Done Done false
2094 1555 Acked Read 10 Sent Done Done false
2095 1556 Acked Read 3 Sent Done Done false
2096 1557 Acked Write 10 Sent Done Done false
2097 1558 Acked Write 8 Sent Done Done false
2098 1559 Acked Read 10 Sent Done Done false
2099 1560 Acked Flush 0 Sent Done Done false
2100 1561 Acked Write 7 Sent Done Done false
2101 1562 Acked Read 10 Sent Done Done false
2102 1563 Acked Read 9 Sent Done Done false
2103 1564 Acked Read 3 Sent Done Done false
2104 1565 Acked Write 6 Sent Done Done false
2105 1566 Acked Read 10 Sent Done Done false
2106 1567 Acked Read 8 Sent Done Done false
2107 1568 Acked Write 8 Sent Done Done false
2108 1569 Acked Flush 0 Sent Done Done false
2109 1570 Acked Write 10 Sent Done Done false
2110 1571 Acked Write 3 Sent Done Done false
2111 1572 Acked Read 9 Sent Done Done false
2112 1573 Acked Read 6 Sent Done Done false
2113 1574 Acked Read 3 Sent Done Done false
2114 1575 Acked Read 8 Sent Done Done false
2115 1576 Acked Write 4 Sent Done Done false
2116 1577 Acked Write 5 Sent Done Done false
2117 1578 Acked Read 1 Sent Done Done false
2118 1579 Acked Write 7 Sent Done Done false
2119 1580 Acked Flush 0 Sent Done Done false
2120 1581 Acked Read 9 Sent Done Done false
2121 1582 Acked Write 5 Sent Done Done false
2122 1583 Acked Write 10 Sent Done Done false
2123 1584 Acked Read 8 Sent Done Done false
2124 1585 Acked Write 7 Sent Done Done false
2125 1586 Acked Write 6 Sent Done Done false
2126 1587 Acked Write 6 Sent Done Done false
2127 1588 Acked Read 8 Sent Done Done false
2128 1589 Acked Read 5 Sent Done Done false
2129 1590 Acked Write 2 Sent Done Done false
2130 1591 Acked Flush 0 Sent Done Done false
2131 1592 Acked Read 8 Sent Done Done false
2132 1593 Acked Write 8 Sent Done Done false
2133 1594 Acked Read 7 Sent Done Done false
2134 1595 Acked Write 4 Sent Done Done false
2135 1596 Acked Write 1 Sent Done Done false
2136 1597 Acked Read 7 Sent Done Done false
2137 1598 Acked Read 6 Sent Done Done false
2138 1599 Acked Flush 0 Sent Done Done false
2139 1600 Acked Write 4 Sent Done Done false
2140 1601 Acked Write 7 Sent Done Done false
2141 1602 Acked Write 7 Sent Done Done false
2142 1603 Acked Flush 0 Sent Done Done false
2143 STATES DS:0 DS:1 DS:2 TOTAL
2144 Sent 302 0 0 302
2145 Done 0 302 302 604
2146 Skipped 0 0 0 0
2147 Error 0 0 0 0
2148 Last Flush: 1301 1603 1603
2149 Downstairs last five completed: 1301 1300 1299 1298 1297
2150 Upstairs last five completed: 1603 1602 1601 1600 1599
2151 CLIENT: Up:0 ds:302 act:2
21522025-01-03T08:31:19.777ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
21532025-01-03T08:31:19.777ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
21542025-01-03T08:31:19.777ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
21552025-01-03T08:31:19.777ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
21562025-01-03T08:31:19.777ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2157 [8810] got start action
2158 [8810] I am going to restart
2159 Starting downstairs at port 8810
2160 Make output file at "/tmp/dsc/downstairs-8810.txt"
2161 [8810][0] reports Starting
2162 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1207)
2163 [8810][0] reports Running
2164 ----------------------------------------------------------------
2165 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2166 States: Offline Active Active
2167 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2168 1302 Acked Read 6 Sent Done Done false
2169 1303 Acked Write 4 Sent Done Done false
2170 1304 Acked Write 4 Sent Done Done false
2171 1305 Acked Write 3 Sent Done Done false
2172 1306 Acked Read 7 Sent Done Done false
2173 1307 Acked Write 5 Sent Done Done false
2174 1308 Acked Write 4 Sent Done Done false
2175 1309 Acked Read 1 Sent Done Done false
2176 1310 Acked Write 3 Sent Done Done false
2177 1311 Acked Flush 0 Sent Done Done false
2178 1312 Acked Read 3 Sent Done Done false
2179 1313 Acked Read 2 Sent Done Done false
2180 1314 Acked Flush 0 Sent Done Done false
2181 1315 Acked Flush 0 Sent Done Done false
2182 1316 Acked Read 10 Sent Done Done false
2183 1317 Acked Write 9 Sent Done Done false
2184 1318 Acked Write 8 Sent Done Done false
2185 1319 Acked Write 10 Sent Done Done false
2186 1320 Acked Read 6 Sent Done Done false
2187 1321 Acked Read 5 Sent Done Done false
2188 1322 Acked Write 6 Sent Done Done false
2189 1323 Acked Read 10 Sent Done Done false
2190 1324 Acked Write 10 Sent Done Done false
2191 1325 Acked Read 2 Sent Done Done false
2192 1326 Acked Read 9 Sent Done Done false
2193 1327 Acked Write 4 Sent Done Done false
2194 1328 Acked Read 8 Sent Done Done false
2195 1329 Acked Write 7 Sent Done Done false
2196 1330 Acked Write 6 Sent Done Done false
2197 1331 Acked Read 2 Sent Done Done false
2198 1332 Acked Write 7 Sent Done Done false
2199 1333 Acked Flush 0 Sent Done Done false
2200 1334 Acked Read 9 Sent Done Done false
2201 1335 Acked Write 9 Sent Done Done false
2202 1336 Acked Read 5 Sent Done Done false
2203 1337 Acked Read 8 Sent Done Done false
2204 1338 Acked Flush 0 Sent Done Done false
2205 1339 Acked Write 6 Sent Done Done false
2206 1340 Acked Read 2 Sent Done Done false
2207 1341 Acked Read 7 Sent Done Done false
2208 1342 Acked Read 3 Sent Done Done false
2209 1343 Acked Read 9 Sent Done Done false
2210 1344 Acked Read 5 Sent Done Done false
2211 1345 Acked Read 8 Sent Done Done false
2212 1346 Acked Write 8 Sent Done Done false
2213 1347 Acked Write 4 Sent Done Done false
2214 1348 Acked Read 10 Sent Done Done false
2215 1349 Acked Read 4 Sent Done Done false
2216 1350 Acked Read 2 Sent Done Done false
2217 1351 Acked Write 7 Sent Done Done false
2218 1352 Acked Read 8 Sent Done Done false
2219 1353 Acked Flush 0 Sent Done Done false
2220 1354 Acked Read 5 Sent Done Done false
2221 1355 Acked Read 5 Sent Done Done false
2222 1356 Acked Write 6 Sent Done Done false
2223 1357 Acked Read 7 Sent Done Done false
2224 1358 Acked Write 4 Sent Done Done false
2225 1359 Acked Read 1 Sent Done Done false
2226 1360 Acked Write 6 Sent Done Done false
2227 1361 Acked Write 5 Sent Done Done false
2228 1362 Acked Write 2 Sent Done Done false
2229 1363 Acked Write 9 Sent Done Done false
2230 1364 Acked Read 1 Sent Done Done false
2231 1365 Acked Read 6 Sent Done Done false
2232 1366 Acked Write 2 Sent Done Done false
2233 1367 Acked Flush 0 Sent Done Done false
2234 1368 Acked Read 5 Sent Done Done false
2235 1369 Acked Read 3 Sent Done Done false
2236 1370 Acked Write 5 Sent Done Done false
2237 1371 Acked Read 5 Sent Done Done false
2238 1372 Acked Read 7 Sent Done Done false
2239 1373 Acked Read 1 Sent Done Done false
2240 1374 Acked Write 8 Sent Done Done false
2241 1375 Acked Read 5 Sent Done Done false
2242 1376 Acked Read 3 Sent Done Done false
2243 1377 Acked Flush 0 Sent Done Done false
2244 1378 Acked Write 1 Sent Done Done false
2245 1379 Acked Write 9 Sent Done Done false
2246 1380 Acked Read 10 Sent Done Done false
2247 1381 Acked Write 8 Sent Done Done false
2248 1382 Acked Read 5 Sent Done Done false
2249 1383 Acked Read 4 Sent Done Done false
2250 1384 Acked Read 7 Sent Done Done false
2251 1385 Acked Read 3 Sent Done Done false
2252 1386 Acked Write 4 Sent Done Done false
2253 1387 Acked Write 3 Sent Done Done false
2254 1388 Acked Read 4 Sent Done Done false
2255 1389 Acked Write 10 Sent Done Done false
2256 1390 Acked Write 4 Sent Done Done false
2257 1391 Acked Write 1 Sent Done Done false
2258 1392 Acked Write 7 Sent Done Done false
2259 1393 Acked Read 8 Sent Done Done false
2260 1394 Acked Write 8 Sent Done Done false
2261 1395 Acked Write 3 Sent Done Done false
2262 1396 Acked Read 5 Sent Done Done false
2263 1397 Acked Read 2 Sent Done Done false
2264 1398 Acked Write 10 Sent Done Done false
2265 1399 Acked Write 1 Sent Done Done false
2266 1400 Acked Flush 0 Sent Done Done false
2267 1401 Acked Read 8 Sent Done Done false
2268 1402 Acked Read 3 Sent Done Done false
2269 1403 Acked Read 4 Sent Done Done false
2270 1404 Acked Read 6 Sent Done Done false
2271 1405 Acked Read 4 Sent Done Done false
2272 1406 Acked Write 4 Sent Done Done false
2273 1407 Acked Write 6 Sent Done Done false
2274 1408 Acked Read 2 Sent Done Done false
2275 1409 Acked Flush 0 Sent Done Done false
2276 1410 Acked Write 4 Sent Done Done false
2277 1411 Acked Read 5 Sent Done Done false
2278 1412 Acked Write 2 Sent Done Done false
2279 1413 Acked Read 10 Sent Done Done false
2280 1414 Acked Read 7 Sent Done Done false
2281 1415 Acked Read 2 Sent Done Done false
2282 1416 Acked Read 4 Sent Done Done false
2283 1417 Acked Read 4 Sent Done Done false
2284 1418 Acked Write 4 Sent Done Done false
2285 1419 Acked Write 9 Sent Done Done false
2286 1420 Acked Read 5 Sent Done Done false
2287 1421 Acked Write 9 Sent Done Done false
2288 1422 Acked Write 2 Sent Done Done false
2289 1423 Acked Read 5 Sent Done Done false
2290 1424 Acked Write 4 Sent Done Done false
2291 1425 Acked Read 6 Sent Done Done false
2292 1426 Acked Read 3 Sent Done Done false
2293 1427 Acked Read 5 Sent Done Done false
2294 1428 Acked Write 10 Sent Done Done false
2295 1429 Acked Read 6 Sent Done Done false
2296 1430 Acked Write 8 Sent Done Done false
2297 1431 Acked Read 4 Sent Done Done false
2298 1432 Acked Read 4 Sent Done Done false
2299 1433 Acked Flush 0 Sent Done Done false
2300 1434 Acked Read 4 Sent Done Done false
2301 1435 Acked Read 10 Sent Done Done false
2302 1436 Acked Read 10 Sent Done Done false
2303 1437 Acked Read 6 Sent Done Done false
2304 1438 Acked Write 7 Sent Done Done false
2305 1439 Acked Write 10 Sent Done Done false
2306 1440 Acked Read 5 Sent Done Done false
2307 1441 Acked Write 2 Sent Done Done false
2308 1442 Acked Write 9 Sent Done Done false
2309 1443 Acked Read 8 Sent Done Done false
2310 1444 Acked Write 2 Sent Done Done false
2311 1445 Acked Write 6 Sent Done Done false
2312 1446 Acked Read 9 Sent Done Done false
2313 1447 Acked Flush 0 Sent Done Done false
2314 1448 Acked Write 1 Sent Done Done false
2315 1449 Acked Read 1 Sent Done Done false
2316 1450 Acked Write 4 Sent Done Done false
2317 1451 Acked Read 5 Sent Done Done false
2318 1452 Acked Read 9 Sent Done Done false
2319 1453 Acked Write 10 Sent Done Done false
2320 1454 Acked Write 2 Sent Done Done false
2321 1455 Acked Read 3 Sent Done Done false
2322 1456 Acked Write 8 Sent Done Done false
2323 1457 Acked Write 8 Sent Done Done false
2324 1458 Acked Read 5 Sent Done Done false
2325 1459 Acked Read 8 Sent Done Done false
2326 1460 Acked Write 10 Sent Done Done false
2327 1461 Acked Write 8 Sent Done Done false
2328 1462 Acked Write 5 Sent Done Done false
2329 1463 Acked Read 2 Sent Done Done false
2330 1464 Acked Write 3 Sent Done Done false
2331 1465 Acked Read 3 Sent Done Done false
2332 1466 Acked Write 5 Sent Done Done false
2333 1467 Acked Write 6 Sent Done Done false
2334 1468 Acked Flush 0 Sent Done Done false
2335 1469 Acked Read 4 Sent Done Done false
2336 1470 Acked Write 10 Sent Done Done false
2337 1471 Acked Read 3 Sent Done Done false
2338 1472 Acked Write 5 Sent Done Done false
2339 1473 Acked Read 5 Sent Done Done false
2340 1474 Acked Read 4 Sent Done Done false
2341 1475 Acked Read 5 Sent Done Done false
2342 1476 Acked Read 8 Sent Done Done false
2343 1477 Acked Write 1 Sent Done Done false
2344 1478 Acked Flush 0 Sent Done Done false
2345 1479 Acked Write 10 Sent Done Done false
2346 1480 Acked Write 5 Sent Done Done false
2347 1481 Acked Read 3 Sent Done Done false
2348 1482 Acked Read 4 Sent Done Done false
2349 1483 Acked Read 6 Sent Done Done false
2350 1484 Acked Flush 0 Sent Done Done false
2351 1485 Acked Write 4 Sent Done Done false
2352 1486 Acked Read 3 Sent Done Done false
2353 1487 Acked Read 8 Sent Done Done false
2354 1488 Acked Read 5 Sent Done Done false
2355 1489 Acked Write 9 Sent Done Done false
2356 1490 Acked Read 2 Sent Done Done false
2357 1491 Acked Read 5 Sent Done Done false
2358 1492 Acked Read 6 Sent Done Done false
2359 1493 Acked Write 9 Sent Done Done false
2360 1494 Acked Read 3 Sent Done Done false
2361 1495 Acked Read 9 Sent Done Done false
2362 1496 Acked Write 10 Sent Done Done false
2363 1497 Acked Write 9 Sent Done Done false
2364 1498 Acked Write 3 Sent Done Done false
2365 1499 Acked Read 2 Sent Done Done false
2366 1500 Acked Read 2 Sent Done Done false
2367 1501 Acked Read 4 Sent Done Done false
2368 1502 Acked Flush 0 Sent Done Done false
2369 1503 Acked Read 1 Sent Done Done false
2370 1504 Acked Read 10 Sent Done Done false
2371 1505 Acked Write 4 Sent Done Done false
2372 1506 Acked Read 10 Sent Done Done false
2373 1507 Acked Write 5 Sent Done Done false
2374 1508 Acked Read 3 Sent Done Done false
2375 1509 Acked Write 3 Sent Done Done false
2376 1510 Acked Read 4 Sent Done Done false
2377 1511 Acked Read 1 Sent Done Done false
2378 1512 Acked Write 9 Sent Done Done false
2379 1513 Acked Read 5 Sent Done Done false
2380 1514 Acked Read 5 Sent Done Done false
2381 1515 Acked Read 3 Sent Done Done false
2382 1516 Acked Read 9 Sent Done Done false
2383 1517 Acked Write 2 Sent Done Done false
2384 1518 Acked Read 3 Sent Done Done false
2385 1519 Acked Write 1 Sent Done Done false
2386 1520 Acked Write 9 Sent Done Done false
2387 1521 Acked Flush 0 Sent Done Done false
2388 1522 Acked Read 10 Sent Done Done false
2389 1523 Acked Write 6 Sent Done Done false
2390 1524 Acked Read 9 Sent Done Done false
2391 1525 Acked Read 1 Sent Done Done false
2392 1526 Acked Write 8 Sent Done Done false
2393 1527 Acked Write 6 Sent Done Done false
2394 1528 Acked Write 3 Sent Done Done false
2395 1529 Acked Read 10 Sent Done Done false
2396 1530 Acked Read 2 Sent Done Done false
2397 1531 Acked Read 7 Sent Done Done false
2398 1532 Acked Read 6 Sent Done Done false
2399 1533 Acked Write 5 Sent Done Done false
2400 1534 Acked Read 3 Sent Done Done false
2401 1535 Acked Read 9 Sent Done Done false
2402 1536 Acked Read 1 Sent Done Done false
2403 1537 Acked Read 7 Sent Done Done false
2404 1538 Acked Flush 0 Sent Done Done false
2405 1539 Acked Write 5 Sent Done Done false
2406 1540 Acked Write 1 Sent Done Done false
2407 1541 Acked Write 9 Sent Done Done false
2408 1542 Acked Read 4 Sent Done Done false
2409 1543 Acked Write 6 Sent Done Done false
2410 1544 Acked Write 10 Sent Done Done false
2411 1545 Acked Read 8 Sent Done Done false
2412 1546 Acked Write 1 Sent Done Done false
2413 1547 Acked Write 8 Sent Done Done false
2414 1548 Acked Read 6 Sent Done Done false
2415 1549 Acked Flush 0 Sent Done Done false
2416 1550 Acked Read 5 Sent Done Done false
2417 1551 Acked Read 10 Sent Done Done false
2418 1552 Acked Write 1 Sent Done Done false
2419 1553 Acked Write 10 Sent Done Done false
2420 1554 Acked Read 3 Sent Done Done false
2421 1555 Acked Read 10 Sent Done Done false
2422 1556 Acked Read 3 Sent Done Done false
2423 1557 Acked Write 10 Sent Done Done false
2424 1558 Acked Write 8 Sent Done Done false
2425 1559 Acked Read 10 Sent Done Done false
2426 1560 Acked Flush 0 Sent Done Done false
2427 1561 Acked Write 7 Sent Done Done false
2428 1562 Acked Read 10 Sent Done Done false
2429 1563 Acked Read 9 Sent Done Done false
2430 1564 Acked Read 3 Sent Done Done false
2431 1565 Acked Write 6 Sent Done Done false
2432 1566 Acked Read 10 Sent Done Done false
2433 1567 Acked Read 8 Sent Done Done false
2434 1568 Acked Write 8 Sent Done Done false
2435 1569 Acked Flush 0 Sent Done Done false
2436 1570 Acked Write 10 Sent Done Done false
2437 1571 Acked Write 3 Sent Done Done false
2438 1572 Acked Read 9 Sent Done Done false
2439 1573 Acked Read 6 Sent Done Done false
2440 1574 Acked Read 3 Sent Done Done false
2441 1575 Acked Read 8 Sent Done Done false
2442 1576 Acked Write 4 Sent Done Done false
2443 1577 Acked Write 5 Sent Done Done false
2444 1578 Acked Read 1 Sent Done Done false
2445 1579 Acked Write 7 Sent Done Done false
2446 1580 Acked Flush 0 Sent Done Done false
2447 1581 Acked Read 9 Sent Done Done false
2448 1582 Acked Write 5 Sent Done Done false
2449 1583 Acked Write 10 Sent Done Done false
2450 1584 Acked Read 8 Sent Done Done false
2451 1585 Acked Write 7 Sent Done Done false
2452 1586 Acked Write 6 Sent Done Done false
2453 1587 Acked Write 6 Sent Done Done false
2454 1588 Acked Read 8 Sent Done Done false
2455 1589 Acked Read 5 Sent Done Done false
2456 1590 Acked Write 2 Sent Done Done false
2457 1591 Acked Flush 0 Sent Done Done false
2458 1592 Acked Read 8 Sent Done Done false
2459 1593 Acked Write 8 Sent Done Done false
2460 1594 Acked Read 7 Sent Done Done false
2461 1595 Acked Write 4 Sent Done Done false
2462 1596 Acked Write 1 Sent Done Done false
2463 1597 Acked Read 7 Sent Done Done false
2464 1598 Acked Read 6 Sent Done Done false
2465 1599 Acked Flush 0 Sent Done Done false
2466 1600 Acked Write 4 Sent Done Done false
2467 1601 Acked Write 7 Sent Done Done false
2468 1602 Acked Write 7 Sent Done Done false
2469 1603 Acked Flush 0 Sent Done Done false
2470 STATES DS:0 DS:1 DS:2 TOTAL
2471 Sent 302 0 0 302
2472 Done 0 302 302 604
2473 Skipped 0 0 0 0
2474 Error 0 0 0 0
2475 Last Flush: 1301 1603 1603
2476 Downstairs last five completed: 1301 1300 1299 1298 1297
2477 Upstairs last five completed: 1603 1602 1601 1600 1599
2478 CLIENT: Up:0 ds:302 act:2
2479 ----------------------------------------------------------------
2480 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2481 States: Offline Active Active
2482 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2483 1302 Acked Read 6 Sent Done Done false
2484 1303 Acked Write 4 Sent Done Done false
2485 1304 Acked Write 4 Sent Done Done false
2486 1305 Acked Write 3 Sent Done Done false
2487 1306 Acked Read 7 Sent Done Done false
2488 1307 Acked Write 5 Sent Done Done false
2489 1308 Acked Write 4 Sent Done Done false
2490 1309 Acked Read 1 Sent Done Done false
2491 1310 Acked Write 3 Sent Done Done false
2492 1311 Acked Flush 0 Sent Done Done false
2493 1312 Acked Read 3 Sent Done Done false
2494 1313 Acked Read 2 Sent Done Done false
2495 1314 Acked Flush 0 Sent Done Done false
2496 1315 Acked Flush 0 Sent Done Done false
2497 1316 Acked Read 10 Sent Done Done false
2498 1317 Acked Write 9 Sent Done Done false
2499 1318 Acked Write 8 Sent Done Done false
2500 1319 Acked Write 10 Sent Done Done false
2501 1320 Acked Read 6 Sent Done Done false
2502 1321 Acked Read 5 Sent Done Done false
2503 1322 Acked Write 6 Sent Done Done false
2504 1323 Acked Read 10 Sent Done Done false
2505 1324 Acked Write 10 Sent Done Done false
2506 1325 Acked Read 2 Sent Done Done false
2507 1326 Acked Read 9 Sent Done Done false
2508 1327 Acked Write 4 Sent Done Done false
2509 1328 Acked Read 8 Sent Done Done false
2510 1329 Acked Write 7 Sent Done Done false
2511 1330 Acked Write 6 Sent Done Done false
2512 1331 Acked Read 2 Sent Done Done false
2513 1332 Acked Write 7 Sent Done Done false
2514 1333 Acked Flush 0 Sent Done Done false
2515 1334 Acked Read 9 Sent Done Done false
2516 1335 Acked Write 9 Sent Done Done false
2517 1336 Acked Read 5 Sent Done Done false
2518 1337 Acked Read 8 Sent Done Done false
2519 1338 Acked Flush 0 Sent Done Done false
2520 1339 Acked Write 6 Sent Done Done false
2521 1340 Acked Read 2 Sent Done Done false
2522 1341 Acked Read 7 Sent Done Done false
2523 1342 Acked Read 3 Sent Done Done false
2524 1343 Acked Read 9 Sent Done Done false
2525 1344 Acked Read 5 Sent Done Done false
2526 1345 Acked Read 8 Sent Done Done false
2527 1346 Acked Write 8 Sent Done Done false
2528 1347 Acked Write 4 Sent Done Done false
2529 1348 Acked Read 10 Sent Done Done false
2530 1349 Acked Read 4 Sent Done Done false
2531 1350 Acked Read 2 Sent Done Done false
2532 1351 Acked Write 7 Sent Done Done false
2533 1352 Acked Read 8 Sent Done Done false
2534 1353 Acked Flush 0 Sent Done Done false
2535 1354 Acked Read 5 Sent Done Done false
2536 1355 Acked Read 5 Sent Done Done false
2537 1356 Acked Write 6 Sent Done Done false
2538 1357 Acked Read 7 Sent Done Done false
2539 1358 Acked Write 4 Sent Done Done false
2540 1359 Acked Read 1 Sent Done Done false
2541 1360 Acked Write 6 Sent Done Done false
2542 1361 Acked Write 5 Sent Done Done false
2543 1362 Acked Write 2 Sent Done Done false
2544 1363 Acked Write 9 Sent Done Done false
2545 1364 Acked Read 1 Sent Done Done false
2546 1365 Acked Read 6 Sent Done Done false
2547 1366 Acked Write 2 Sent Done Done false
2548 1367 Acked Flush 0 Sent Done Done false
2549 1368 Acked Read 5 Sent Done Done false
2550 1369 Acked Read 3 Sent Done Done false
2551 1370 Acked Write 5 Sent Done Done false
2552 1371 Acked Read 5 Sent Done Done false
2553 1372 Acked Read 7 Sent Done Done false
2554 1373 Acked Read 1 Sent Done Done false
2555 1374 Acked Write 8 Sent Done Done false
2556 1375 Acked Read 5 Sent Done Done false
2557 1376 Acked Read 3 Sent Done Done false
2558 1377 Acked Flush 0 Sent Done Done false
2559 1378 Acked Write 1 Sent Done Done false
2560 1379 Acked Write 9 Sent Done Done false
2561 1380 Acked Read 10 Sent Done Done false
2562 1381 Acked Write 8 Sent Done Done false
2563 1382 Acked Read 5 Sent Done Done false
2564 1383 Acked Read 4 Sent Done Done false
2565 1384 Acked Read 7 Sent Done Done false
2566 1385 Acked Read 3 Sent Done Done false
2567 1386 Acked Write 4 Sent Done Done false
2568 1387 Acked Write 3 Sent Done Done false
2569 1388 Acked Read 4 Sent Done Done false
2570 1389 Acked Write 10 Sent Done Done false
2571 1390 Acked Write 4 Sent Done Done false
2572 1391 Acked Write 1 Sent Done Done false
2573 1392 Acked Write 7 Sent Done Done false
2574 1393 Acked Read 8 Sent Done Done false
2575 1394 Acked Write 8 Sent Done Done false
2576 1395 Acked Write 3 Sent Done Done false
2577 1396 Acked Read 5 Sent Done Done false
2578 1397 Acked Read 2 Sent Done Done false
2579 1398 Acked Write 10 Sent Done Done false
2580 1399 Acked Write 1 Sent Done Done false
2581 1400 Acked Flush 0 Sent Done Done false
2582 1401 Acked Read 8 Sent Done Done false
2583 1402 Acked Read 3 Sent Done Done false
2584 1403 Acked Read 4 Sent Done Done false
2585 1404 Acked Read 6 Sent Done Done false
2586 1405 Acked Read 4 Sent Done Done false
2587 1406 Acked Write 4 Sent Done Done false
2588 1407 Acked Write 6 Sent Done Done false
2589 1408 Acked Read 2 Sent Done Done false
2590 1409 Acked Flush 0 Sent Done Done false
2591 1410 Acked Write 4 Sent Done Done false
2592 1411 Acked Read 5 Sent Done Done false
2593 1412 Acked Write 2 Sent Done Done false
2594 1413 Acked Read 10 Sent Done Done false
2595 1414 Acked Read 7 Sent Done Done false
2596 1415 Acked Read 2 Sent Done Done false
2597 1416 Acked Read 4 Sent Done Done false
2598 1417 Acked Read 4 Sent Done Done false
2599 1418 Acked Write 4 Sent Done Done false
2600 1419 Acked Write 9 Sent Done Done false
2601 1420 Acked Read 5 Sent Done Done false
2602 1421 Acked Write 9 Sent Done Done false
2603 1422 Acked Write 2 Sent Done Done false
2604 1423 Acked Read 5 Sent Done Done false
2605 1424 Acked Write 4 Sent Done Done false
2606 1425 Acked Read 6 Sent Done Done false
2607 1426 Acked Read 3 Sent Done Done false
2608 1427 Acked Read 5 Sent Done Done false
2609 1428 Acked Write 10 Sent Done Done false
2610 1429 Acked Read 6 Sent Done Done false
2611 1430 Acked Write 8 Sent Done Done false
2612 1431 Acked Read 4 Sent Done Done false
2613 1432 Acked Read 4 Sent Done Done false
2614 1433 Acked Flush 0 Sent Done Done false
2615 1434 Acked Read 4 Sent Done Done false
2616 1435 Acked Read 10 Sent Done Done false
2617 1436 Acked Read 10 Sent Done Done false
2618 1437 Acked Read 6 Sent Done Done false
2619 1438 Acked Write 7 Sent Done Done false
2620 1439 Acked Write 10 Sent Done Done false
2621 1440 Acked Read 5 Sent Done Done false
2622 1441 Acked Write 2 Sent Done Done false
2623 1442 Acked Write 9 Sent Done Done false
2624 1443 Acked Read 8 Sent Done Done false
2625 1444 Acked Write 2 Sent Done Done false
2626 1445 Acked Write 6 Sent Done Done false
2627 1446 Acked Read 9 Sent Done Done false
2628 1447 Acked Flush 0 Sent Done Done false
2629 1448 Acked Write 1 Sent Done Done false
2630 1449 Acked Read 1 Sent Done Done false
2631 1450 Acked Write 4 Sent Done Done false
2632 1451 Acked Read 5 Sent Done Done false
2633 1452 Acked Read 9 Sent Done Done false
2634 1453 Acked Write 10 Sent Done Done false
2635 1454 Acked Write 2 Sent Done Done false
2636 1455 Acked Read 3 Sent Done Done false
2637 1456 Acked Write 8 Sent Done Done false
2638 1457 Acked Write 8 Sent Done Done false
2639 1458 Acked Read 5 Sent Done Done false
2640 1459 Acked Read 8 Sent Done Done false
2641 1460 Acked Write 10 Sent Done Done false
2642 1461 Acked Write 8 Sent Done Done false
2643 1462 Acked Write 5 Sent Done Done false
2644 1463 Acked Read 2 Sent Done Done false
2645 1464 Acked Write 3 Sent Done Done false
2646 1465 Acked Read 3 Sent Done Done false
2647 1466 Acked Write 5 Sent Done Done false
2648 1467 Acked Write 6 Sent Done Done false
2649 1468 Acked Flush 0 Sent Done Done false
2650 1469 Acked Read 4 Sent Done Done false
2651 1470 Acked Write 10 Sent Done Done false
2652 1471 Acked Read 3 Sent Done Done false
2653 1472 Acked Write 5 Sent Done Done false
2654 1473 Acked Read 5 Sent Done Done false
2655 1474 Acked Read 4 Sent Done Done false
2656 1475 Acked Read 5 Sent Done Done false
2657 1476 Acked Read 8 Sent Done Done false
2658 1477 Acked Write 1 Sent Done Done false
2659 1478 Acked Flush 0 Sent Done Done false
2660 1479 Acked Write 10 Sent Done Done false
2661 1480 Acked Write 5 Sent Done Done false
2662 1481 Acked Read 3 Sent Done Done false
2663 1482 Acked Read 4 Sent Done Done false
2664 1483 Acked Read 6 Sent Done Done false
2665 1484 Acked Flush 0 Sent Done Done false
2666 1485 Acked Write 4 Sent Done Done false
2667 1486 Acked Read 3 Sent Done Done false
2668 1487 Acked Read 8 Sent Done Done false
2669 1488 Acked Read 5 Sent Done Done false
2670 1489 Acked Write 9 Sent Done Done false
2671 1490 Acked Read 2 Sent Done Done false
2672 1491 Acked Read 5 Sent Done Done false
2673 1492 Acked Read 6 Sent Done Done false
2674 1493 Acked Write 9 Sent Done Done false
2675 1494 Acked Read 3 Sent Done Done false
2676 1495 Acked Read 9 Sent Done Done false
2677 1496 Acked Write 10 Sent Done Done false
2678 1497 Acked Write 9 Sent Done Done false
2679 1498 Acked Write 3 Sent Done Done false
2680 1499 Acked Read 2 Sent Done Done false
2681 1500 Acked Read 2 Sent Done Done false
2682 1501 Acked Read 4 Sent Done Done false
2683 1502 Acked Flush 0 Sent Done Done false
2684 1503 Acked Read 1 Sent Done Done false
2685 1504 Acked Read 10 Sent Done Done false
2686 1505 Acked Write 4 Sent Done Done false
2687 1506 Acked Read 10 Sent Done Done false
2688 1507 Acked Write 5 Sent Done Done false
2689 1508 Acked Read 3 Sent Done Done false
2690 1509 Acked Write 3 Sent Done Done false
2691 1510 Acked Read 4 Sent Done Done false
2692 1511 Acked Read 1 Sent Done Done false
2693 1512 Acked Write 9 Sent Done Done false
2694 1513 Acked Read 5 Sent Done Done false
2695 1514 Acked Read 5 Sent Done Done false
2696 1515 Acked Read 3 Sent Done Done false
2697 1516 Acked Read 9 Sent Done Done false
2698 1517 Acked Write 2 Sent Done Done false
2699 1518 Acked Read 3 Sent Done Done false
2700 1519 Acked Write 1 Sent Done Done false
2701 1520 Acked Write 9 Sent Done Done false
2702 1521 Acked Flush 0 Sent Done Done false
2703 1522 Acked Read 10 Sent Done Done false
2704 1523 Acked Write 6 Sent Done Done false
2705 1524 Acked Read 9 Sent Done Done false
2706 1525 Acked Read 1 Sent Done Done false
2707 1526 Acked Write 8 Sent Done Done false
2708 1527 Acked Write 6 Sent Done Done false
2709 1528 Acked Write 3 Sent Done Done false
2710 1529 Acked Read 10 Sent Done Done false
2711 1530 Acked Read 2 Sent Done Done false
2712 1531 Acked Read 7 Sent Done Done false
2713 1532 Acked Read 6 Sent Done Done false
2714 1533 Acked Write 5 Sent Done Done false
2715 1534 Acked Read 3 Sent Done Done false
2716 1535 Acked Read 9 Sent Done Done false
2717 1536 Acked Read 1 Sent Done Done false
2718 1537 Acked Read 7 Sent Done Done false
2719 1538 Acked Flush 0 Sent Done Done false
2720 1539 Acked Write 5 Sent Done Done false
2721 1540 Acked Write 1 Sent Done Done false
2722 1541 Acked Write 9 Sent Done Done false
2723 1542 Acked Read 4 Sent Done Done false
2724 1543 Acked Write 6 Sent Done Done false
2725 1544 Acked Write 10 Sent Done Done false
2726 1545 Acked Read 8 Sent Done Done false
2727 1546 Acked Write 1 Sent Done Done false
2728 1547 Acked Write 8 Sent Done Done false
2729 1548 Acked Read 6 Sent Done Done false
2730 1549 Acked Flush 0 Sent Done Done false
2731 1550 Acked Read 5 Sent Done Done false
2732 1551 Acked Read 10 Sent Done Done false
2733 1552 Acked Write 1 Sent Done Done false
2734 1553 Acked Write 10 Sent Done Done false
2735 1554 Acked Read 3 Sent Done Done false
2736 1555 Acked Read 10 Sent Done Done false
2737 1556 Acked Read 3 Sent Done Done false
2738 1557 Acked Write 10 Sent Done Done false
2739 1558 Acked Write 8 Sent Done Done false
2740 1559 Acked Read 10 Sent Done Done false
2741 1560 Acked Flush 0 Sent Done Done false
2742 1561 Acked Write 7 Sent Done Done false
2743 1562 Acked Read 10 Sent Done Done false
2744 1563 Acked Read 9 Sent Done Done false
2745 1564 Acked Read 3 Sent Done Done false
2746 1565 Acked Write 6 Sent Done Done false
2747 1566 Acked Read 10 Sent Done Done false
2748 1567 Acked Read 8 Sent Done Done false
2749 1568 Acked Write 8 Sent Done Done false
2750 1569 Acked Flush 0 Sent Done Done false
2751 1570 Acked Write 10 Sent Done Done false
2752 1571 Acked Write 3 Sent Done Done false
2753 1572 Acked Read 9 Sent Done Done false
2754 1573 Acked Read 6 Sent Done Done false
2755 1574 Acked Read 3 Sent Done Done false
2756 1575 Acked Read 8 Sent Done Done false
2757 1576 Acked Write 4 Sent Done Done false
2758 1577 Acked Write 5 Sent Done Done false
2759 1578 Acked Read 1 Sent Done Done false
2760 1579 Acked Write 7 Sent Done Done false
2761 1580 Acked Flush 0 Sent Done Done false
2762 1581 Acked Read 9 Sent Done Done false
2763 1582 Acked Write 5 Sent Done Done false
2764 1583 Acked Write 10 Sent Done Done false
2765 1584 Acked Read 8 Sent Done Done false
2766 1585 Acked Write 7 Sent Done Done false
2767 1586 Acked Write 6 Sent Done Done false
2768 1587 Acked Write 6 Sent Done Done false
2769 1588 Acked Read 8 Sent Done Done false
2770 1589 Acked Read 5 Sent Done Done false
2771 1590 Acked Write 2 Sent Done Done false
2772 1591 Acked Flush 0 Sent Done Done false
2773 1592 Acked Read 8 Sent Done Done false
2774 1593 Acked Write 8 Sent Done Done false
2775 1594 Acked Read 7 Sent Done Done false
2776 1595 Acked Write 4 Sent Done Done false
2777 1596 Acked Write 1 Sent Done Done false
2778 1597 Acked Read 7 Sent Done Done false
2779 1598 Acked Read 6 Sent Done Done false
2780 1599 Acked Flush 0 Sent Done Done false
2781 1600 Acked Write 4 Sent Done Done false
2782 1601 Acked Write 7 Sent Done Done false
2783 1602 Acked Write 7 Sent Done Done false
2784 1603 Acked Flush 0 Sent Done Done false
2785 STATES DS:0 DS:1 DS:2 TOTAL
2786 Sent 302 0 0 302
2787 Done 0 302 302 604
2788 Skipped 0 0 0 0
2789 Error 0 0 0 0
2790 Last Flush: 1301 1603 1603
2791 Downstairs last five completed: 1301 1300 1299 1298 1297
2792 Upstairs last five completed: 1603 1602 1601 1600 1599
2793 CLIENT: Up:0 ds:302 act:2
27942025-01-03T08:31:29.779ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
27952025-01-03T08:31:29.779ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60646) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
27962025-01-03T08:31:29.780ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
27972025-01-03T08:31:29.780ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
27982025-01-03T08:31:29.780ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
27992025-01-03T08:31:29.780ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28002025-01-03T08:31:29.781ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28012025-01-03T08:31:29.781ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28022025-01-03T08:31:29.781ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28032025-01-03T08:31:29.781ZINFOcrucible: [0] client re-new 302 jobs since flush 1301 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28042025-01-03T08:31:29.781ZINFOcrucible: [0] Replayed 302 jobs since flush: 1301 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2805 ----------------------------------------------------------------
2806 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2807 Downstairs last five completed: 1603 1602 1601 1600 1599
2808 Upstairs last five completed: 1603 1602 1601 1600 1599
2809 CLIENT: Up:0 ds:0 act:3
2810 Replay: All jobs finished
2811 Main task has work to do, go find it
2812 got dsc Stop(1)
2813 stop 1
2814 Jan 03 08:31:30.031 INFO request completed, latency_us: 305, response_code: 204, uri[8820] Got stop action so:false kr:false
2815 : /stop/cid/1, method: POST, req_id: 5b3d5a43-eb20-4d7a-afa9-a79416a2aad8, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
28162025-01-03T08:31:30.031ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28172025-01-03T08:31:30.031ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28182025-01-03T08:31:30.031ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28192025-01-03T08:31:30.031ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28202025-01-03T08:31:30.031ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28212025-01-03T08:31:30.032ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
2822 Jan 03 08:31:30.032 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/1, method: GET, req_id: 82aea408-e5af-4284-80d7-96c80baaf81e, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
2823 [8820] Exited with: ExitStatus(unix_wait_status(9))
2824 [8820][1] reports Exit
2825 Jan 03 08:31:34.033 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/1, method: GET, req_id: 908fec38-35bf-41f3-ba66-70876dda475e, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
2826 001/300 Write block 3397 len 28672 data: 3 4 3 2 2 2 2
2827 002/300 Read block 4790 len 40960
2828 003/300 Read block 3938 len 32768
2829 004/300 Read block 296 len 8192
2830 005/300 Write block 2581 len 24576 data: 2 2 2 2 2 2
2831 006/300 Write block 4251 len 24576 data: 2 3 3 3 3 3
2832 007/300 Write block 1434 len 16384 data: 2 2 2 2
2833 008/300 Read block 107 len 8192
2834 009/300 Write block 3290 len 12288 data: 2 2 2
2835 010/300 Read block 208 len 4096
2836 011/300 Read block 4443 len 40960
2837 012/300 Read block 1286 len 24576
2838 013/300 Read block 1431 len 40960
2839 014/300 Read block 1171 len 32768
2840 015/300 Read block 3165 len 24576
2841 016/300 Write block 1522 len 20480 data: 2 3 3 3 3
2842 017/300 Read block 459 len 28672
2843 018/300 Read block 1444 len 40960
2844 019/300 Read block 2386 len 24576
2845 020/300 Flush
2846 021/300 Read block 2211 len 40960
2847 022/300 Write block 3073 len 8192 data: 2 2
2848 023/300 Read block 1814 len 8192
2849 024/300 Write block 1876 len 16384 data: 3 3 3 2
2850 025/300 Read block 1173 len 8192
2851 026/300 Read block 2357 len 4096
2852 027/300 Write block 638 len 20480 data: 2 2 2 2 2
2853 028/300 Write block 4490 len 32768 data: 2 2 2 2 3 3 3 3
2854 029/300 Read block 4173 len 20480
2855 030/300 Flush
2856 031/300 Read block 3387 len 24576
2857 032/300 Read block 4803 len 20480
2858 033/300 Read block 950 len 8192
2859 034/300 Write block 3042 len 32768 data: 2 2 2 2 2 2 2 2
2860 035/300 Write block 4308 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2861 036/300 Read block 4857 len 36864
2862 037/300 Write block 4358 len 32768 data: 2 2 2 2 2 2 2 3
2863 038/300 Write block 627 len 8192 data: 2 2
2864 039/300 Read block 3167 len 32768
2865 040/300 Write block 4744 len 24576 data: 2 2 2 2 3 3
2866 041/300 Flush
2867 042/300 Write block 1274 len 32768 data: 3 3 3 2 3 3 3 3
2868 043/300 Read block 4650 len 40960
2869 044/300 Read block 1588 len 8192
2870 045/300 Write block 935 len 16384 data: 2 2 2 2
2871 046/300 Write block 4428 len 24576 data: 2 2 2 2 2 2
2872 047/300 Read block 3411 len 20480
2873 048/300 Read block 4877 len 20480
2874 049/300 Write block 70 len 4096 data: 3
2875 050/300 Write block 3465 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2876 051/300 Read block 1636 len 32768
2877 052/300 Write block 1891 len 24576 data: 3 3 2 2 2 2
2878 053/300 Read block 1632 len 24576
2879 054/300 Write block 3612 len 36864 data: 2 2 2 2 2 2 2 2 2
2880 055/300 Write block 2385 len 4096 data: 3
2881 056/300 Write block 4905 len 4096 data: 2
2882 057/300 Write block 350 len 20480 data: 2 2 2 2 2
2883 058/300 Read block 913 len 12288
2884 059/300 Read block 3592 len 8192
2885 060/300 Write block 4878 len 8192 data: 2 2
2886 061/300 Read block 691 len 20480
2887 062/300 Flush
2888 063/300 Read block 4195 len 32768
2889 064/300 Read block 3701 len 36864
2890 065/300 Write block 4478 len 16384 data: 2 2 2 2
2891 066/300 Write block 2103 len 20480 data: 2 2 2 2 2
2892 067/300 Read block 4441 len 40960
2893 068/300 Flush
2894 069/300 Write block 4358 len 8192 data: 3 3
2895 070/300 Write block 1842 len 4096 data: 3
2896 071/300 Read block 493 len 40960
2897 072/300 Read block 597 len 4096
2898 073/300 Read block 4522 len 20480
2899 074/300 Write block 463 len 32768 data: 2 2 2 2 2 2 2 2
2900 075/300 Read block 405 len 24576
2901 076/300 Read block 3629 len 20480
2902 077/300 Read block 1345 len 20480
2903 078/300 Write block 2934 len 20480 data: 2 2 2 3 3
2904 079/300 Read block 4873 len 8192
2905 080/300 Read block 71 len 8192
2906 081/300 Write block 1671 len 36864 data: 2 2 2 2 2 2 2 2 2
2907 082/300 Write block 2597 len 28672 data: 2 2 2 2 2 2 2
2908 083/300 Flush
2909 084/300 Read block 1802 len 28672
2910 085/300 Read block 4299 len 36864
2911 086/300 Write block 4662 len 36864 data: 2 2 2 2 2 2 2 2 2
2912 087/300 Write block 452 len 12288 data: 2 2 2
2913 088/300 Write block 3724 len 4096 data: 2
2914 089/300 Read block 2359 len 8192
2915 090/300 Read block 2543 len 32768
2916 091/300 Read block 1040 len 28672
2917 092/300 Read block 900 len 8192
2918 093/300 Write block 2637 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2919 094/300 Read block 970 len 32768
2920 095/300 Flush
2921 096/300 Read block 3539 len 32768
2922 097/300 Write block 1464 len 12288 data: 3 3 3
2923 098/300 Read block 524 len 20480
2924 099/300 Write block 3931 len 4096 data: 2
2925 100/300 Read block 3606 len 16384
2926 101/300 Read block 3987 len 36864
2927 102/300 Read block 4006 len 40960
2928 103/300 Write block 3173 len 32768 data: 3 3 3 3 3 3 3 2
2929 104/300 Read block 1219 len 24576
2930 105/300 Write block 1213 len 36864 data: 2 2 2 2 2 2 2 2 2
2931 106/300 Write block 895 len 36864 data: 3 3 3 3 3 3 3 3 3
2932 107/300 Write block 2299 len 8192 data: 2 3
2933 108/300 Write block 912 len 20480 data: 2 2 3 3 3
2934 109/300 Write block 1657 len 4096 data: 2
2935 110/300 Write block 2909 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2936 111/300 Write block 1448 len 4096 data: 2
2937 112/300 Write block 83 len 32768 data: 2 2 2 2 2 2 2 2
2938 113/300 Write block 19 len 36864 data: 2 2 2 2 2 2 2 2 2
2939 114/300 Read block 1082 len 28672
2940 115/300 Write block 3946 len 36864 data: 3 3 3 3 3 4 3 3 3
2941 116/300 Flush
2942 117/300 Read block 4274 len 12288
2943 118/300 Write block 4694 len 8192 data: 2 2
2944 119/300 Read block 969 len 4096
2945 120/300 Read block 2288 len 20480
2946 121/300 Read block 575 len 4096
2947 122/300 Write block 587 len 24576 data: 2 2 2 2 2 2
2948 123/300 Read block 3448 len 8192
2949 124/300 Write block 4559 len 28672 data: 2 2 2 2 2 2 2
2950 125/300 Read block 1662 len 32768
2951 126/300 Read block 1575 len 4096
2952 127/300 Write block 1936 len 28672 data: 2 2 2 2 2 2 2
2953 128/300 Read block 2486 len 28672
2954 129/300 Write block 3071 len 20480 data: 2 2 3 3 2
2955 130/300 Read block 4274 len 36864
2956 131/300 Read block 4955 len 24576
2957 132/300 Write block 103 len 36864 data: 2 2 2 2 2 2 2 2 2
2958 133/300 Read block 1291 len 28672
2959 134/300 Read block 1093 len 24576
2960 135/300 Read block 4942 len 32768
2961 136/300 Write block 3929 len 24576 data: 2 2 3 2 2 2
2962 137/300 Flush
2963 138/300 Write block 2757 len 40960 data: 4 4 4 4 4 4 4 3 2 2
2964 139/300 Read block 2108 len 28672
2965 140/300 Write block 447 len 4096 data: 2
2966 141/300 Read block 314 len 12288
2967 142/300 Read block 3739 len 40960
2968 143/300 Read block 2347 len 36864
2969 144/300 Write block 4821 len 32768 data: 3 3 2 2 2 2 2 2
2970 145/300 Write block 745 len 36864 data: 2 2 2 2 2 2 2 2 2
2971 146/300 Write block 2595 len 32768 data: 2 2 3 3 3 3 3 3
2972 147/300 Write block 4765 len 4096 data: 3
2973 148/300 Read block 1449 len 28672
2974 149/300 Read block 1195 len 40960
2975 150/300 Read block 607 len 24576
2976 151/300 Write block 4642 len 28672 data: 3 2 2 2 2 2 2
2977 152/300 Write block 2161 len 4096 data: 3
2978 153/300 Write block 155 len 24576 data: 2 2 2 2 2 3
2979 154/300 Flush
2980 155/300 Write block 974 len 24576 data: 2 2 2 2 2 2
2981 156/300 Flush
2982 157/300 Write block 729 len 4096 data: 2
2983 158/300 Flush
2984 159/300 Read block 3939 len 28672
2985 160/300 Write block 2946 len 16384 data: 2 2 2 2
2986 161/300 Flush
2987 162/300 Read block 1667 len 16384
2988 163/300 Write block 1456 len 12288 data: 3 3 3
2989 164/300 Write block 3976 len 20480 data: 3 3 3 3 3
2990 165/300 Read block 3806 len 24576
2991 166/300 Read block 861 len 40960
2992 167/300 Flush
2993 168/300 Write block 2319 len 36864 data: 2 2 2 2 2 2 2 2 3
2994 169/300 Write block 4361 len 12288 data: 3 3 3
2995 170/300 Write block 3719 len 32768 data: 3 3 3 3 3 3 2 2
2996 171/300 Flush
2997 172/300 Write block 1145 len 12288 data: 2 2 2
2998 173/300 Read block 3970 len 4096
2999 174/300 Read block 3749 len 4096
3000 175/300 Write block 1363 len 32768 data: 2 2 2 2 2 2 2 2
3001 176/300 Read block 4148 len 28672
3002 177/300 Write block 4274 len 20480 data: 2 2 2 2 2
3003 178/300 Write block 1117 len 32768 data: 2 2 2 2 2 2 2 2
3004 179/300 Read block 1116 len 28672
3005 180/300 Read block 204 len 8192
3006 181/300 Flush
3007 182/300 Read block 2933 len 20480
3008 183/300 Read block 4349 len 8192
3009 184/300 Read block 1269 len 24576
3010 185/300 Write block 1535 len 12288 data: 2 2 3
3011 186/300 Read block 67 len 16384
3012 187/300 Write block 629 len 20480 data: 2 2 2 2 2
3013 188/300 Read block 1424 len 24576
3014 189/300 Write block 81 len 36864 data: 3 2 3 3 3 3 3 3 3
3015 190/300 Write block 1271 len 32768 data: 2 2 2 4 4 4 3 4
3016 191/300 Flush
3017 192/300 Write block 662 len 32768 data: 2 2 3 3 3 2 2 2
3018 193/300 Read block 2317 len 16384
3019 194/300 Read block 1386 len 16384
3020 195/300 Write block 2795 len 32768 data: 2 2 2 2 2 2 2 2
3021 196/300 Write block 468 len 24576 data: 3 3 3 2 2 2
3022 197/300 Read block 1158 len 8192
3023 198/300 Write block 1404 len 36864 data: 2 2 2 2 2 2 2 2 2
3024 199/300 Read block 540 len 24576
3025 200/300 Flush
3026 201/300 Read block 2022 len 28672
3027 202/300 Flush
3028 203/300 Read block 3666 len 20480
3029 204/300 Flush
3030 205/300 Write block 4644 len 12288 data: 3 3 3
3031 206/300 Write block 4523 len 12288 data: 2 2 2
3032 207/300 Read block 1746 len 36864
3033 208/300 Read block 3369 len 12288
3034 209/300 Read block 4369 len 16384
3035 210/300 Write block 3804 len 8192 data: 3 2
3036 211/300 Write block 1440 len 40960 data: 2 2 2 2 2 2 2 2 3 2
3037 212/300 Read block 325 len 12288
3038 213/300 Read block 2238 len 36864
3039 214/300 Write block 1751 len 20480 data: 2 3 3 3 3
3040 215/300 Read block 2296 len 12288
3041 216/300 Read block 152 len 36864
3042 217/300 Read block 3116 len 28672
3043 218/300 Read block 4629 len 20480
3044 219/300 Read block 1801 len 8192
3045 220/300 Flush
3046 221/300 Flush
3047 222/300 Write block 3808 len 12288 data: 2 2 2
3048 223/300 Write block 1176 len 16384 data: 3 3 3 3
3049 224/300 Write block 4334 len 36864 data: 2 2 2 2 2 2 2 2 2
3050 225/300 Flush
3051 226/300 Read block 2537 len 20480
3052 227/300 Write block 1839 len 28672 data: 2 2 3 4 3 3 3
3053 228/300 Read block 3662 len 20480
3054 229/300 Write block 2314 len 20480 data: 2 2 2 3 2
3055 230/300 Read block 3301 len 24576
3056 231/300 Write block 4909 len 16384 data: 3 2 2 2
3057 232/300 Read block 742 len 40960
3058 233/300 Flush
3059 234/300 Read block 1891 len 28672
3060 235/300 Read block 2504 len 36864
3061 236/300 Read block 1607 len 4096
3062 237/300 Read block 1415 len 32768
3063 238/300 Read block 3180 len 12288
3064 239/300 Write block 3881 len 32768 data: 2 2 2 2 2 2 2 2
3065 240/300 Write block 314 len 16384 data: 2 2 2 2
3066 241/300 Read block 1879 len 40960
3067 242/300 Read block 2800 len 4096
3068 243/300 Read block 1734 len 40960
3069 244/300 Read block 4624 len 8192
3070 245/300 Read block 3892 len 24576
3071 246/300 Write block 4926 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3072 247/300 Write block 195 len 20480 data: 3 3 3 3 2
3073 248/300 Flush
3074 249/300 Read block 435 len 28672
3075 250/300 Read block 2481 len 16384
3076 251/300 Read block 3134 len 12288
3077 252/300 Read block 4311 len 16384
3078 253/300 Read block 4885 len 4096
3079 254/300 Read block 4314 len 24576
3080 255/300 Write block 990 len 8192 data: 3 3
3081 256/300 Read block 850 len 28672
3082 257/300 Read block 4512 len 16384
3083 258/300 Write block 3866 len 28672 data: 2 2 2 2 2 2 3
3084 259/300 Write block 2860 len 20480 data: 2 2 2 2 2
3085 260/300 Read block 4580 len 24576
3086 261/300 Read block 3402 len 40960
3087 262/300 Read block 3106 len 12288
3088 263/300 Read block 2490 len 4096
3089 264/300 Write block 2736 len 8192 data: 3 3
3090 265/300 Read block 661 len 8192
3091 266/300 Read block 3987 len 12288
3092 267/300 Flush
3093 268/300 Read block 2474 len 36864
3094 269/300 Read block 4052 len 8192
3095 270/300 Read block 4371 len 24576
3096 271/300 Write block 1132 len 8192 data: 2 2
3097 272/300 Read block 1046 len 36864
3098 273/300 Flush
3099 274/300 Write block 3200 len 40960 data: 3 3 2 2 2 2 2 2 2 2
3100 275/300 Write block 1133 len 32768 data: 3 2 2 2 2 2 2 2
3101 276/300 Read block 4557 len 4096
3102 277/300 Read block 4183 len 4096
3103 278/300 Read block 3973 len 32768
3104 279/300 Read block 548 len 32768
3105 280/300 Read block 1704 len 32768
3106 281/300 Read block 2788 len 20480
3107 282/300 Read block 2295 len 20480
3108 283/300 Write block 3083 len 4096 data: 2
3109 284/300 Flush
3110 285/300 Read block 982 len 32768
3111 286/300 Read block 2218 len 40960
3112 287/300 Write block 4903 len 36864 data: 2 2 3 2 3 3 4 3 3
3113 288/300 Write block 2897 len 12288 data: 2 2 2
3114 289/300 Read block 3539 len 40960
3115 290/300 Read block 1819 len 40960
3116 291/300 Write block 3632 len 12288 data: 2 2 2
3117 292/300 Write block 2971 len 32768 data: 4 4 4 4 4 3 3 3
3118 293/300 Read block 120 len 36864
3119 294/300 Write block 1160 len 8192 data: 2 2
3120 295/300 Flush
3121 296/300 Read block 1627 len 36864
3122 297/300 Write block 3349 len 16384 data: 3 3 3 2
3123 298/300 Read block 444 len 32768
3124 299/300 Read block 3800 len 28672
3125 300/300 Write block 1944 len 16384 data: 2 2 3 3
3126 301/300 Read block 1451 len 28672
3127 Main task has work to do, go find it
3128 got dsc Start(1)
3129 start 1
3130 Jan 03 08:31:34.280 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/1, method: POST, req_id: fb99ffb4-5be5-45fa-9680-dc9131b99002, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
3131 Replay: started 1, returned:Ok(())
3132 ----------------------------------------------------------------
3133 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3134 States: Active Offline Active
3135 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3136 1604 Acked Write 7 Done Sent Done false
3137 1605 Acked Read 10 Done Sent Done false
3138 1606 Acked Read 8 Done Sent Done false
3139 1607 Acked Read 2 Done Sent Done false
3140 1608 Acked Write 6 Done Sent Done false
3141 1609 Acked Write 6 Done Sent Done false
3142 1610 Acked Write 4 Done Sent Done false
3143 1611 Acked Read 2 Done Sent Done false
3144 1612 Acked Write 3 Done Sent Done false
3145 1613 Acked Read 1 Done Sent Done false
3146 1614 Acked Read 10 Done Sent Done false
3147 1615 Acked Read 6 Done Sent Done false
3148 1616 Acked Read 10 Done Sent Done false
3149 1617 Acked Read 8 Done Sent Done false
3150 1618 Acked Read 6 Done Sent Done false
3151 1619 Acked Write 5 Done Sent Done false
3152 1620 Acked Read 7 Done Sent Done false
3153 1621 Acked Read 10 Done Sent Done false
3154 1622 Acked Read 6 Done Sent Done false
3155 1623 Acked Flush 0 Done Sent Done false
3156 1624 Acked Read 10 Done Sent Done false
3157 1625 Acked Write 2 Done Sent Done false
3158 1626 Acked Read 2 Done Sent Done false
3159 1627 Acked Write 4 Done Sent Done false
3160 1628 Acked Read 2 Done Sent Done false
3161 1629 Acked Read 1 Done Sent Done false
3162 1630 Acked Write 5 Done Sent Done false
3163 1631 Acked Write 8 Done Sent Done false
3164 1632 Acked Read 5 Done Sent Done false
3165 1633 Acked Flush 0 Done Sent Done false
3166 1634 Acked Read 6 Done Sent Done false
3167 1635 Acked Read 5 Done Sent Done false
3168 1636 Acked Read 2 Done Sent Done false
3169 1637 Acked Write 8 Done Sent Done false
3170 1638 Acked Write 10 Done Sent Done false
3171 1639 Acked Read 9 Done Sent Done false
3172 1640 Acked Write 8 Done Sent Done false
3173 1641 Acked Write 2 Done Sent Done false
3174 1642 Acked Read 8 Done Sent Done false
3175 1643 Acked Write 6 Done Sent Done false
3176 1644 Acked Flush 0 Done Sent Done false
3177 1645 Acked Write 8 Done Sent Done false
3178 1646 Acked Read 10 Done Sent Done false
3179 1647 Acked Read 2 Done Sent Done false
3180 1648 Acked Write 4 Done Sent Done false
3181 1649 Acked Write 6 Done Sent Done false
3182 1650 Acked Read 5 Done Sent Done false
3183 1651 Acked Read 5 Done Sent Done false
3184 1652 Acked Write 1 Done Sent Done false
3185 1653 Acked Write 10 Done Sent Done false
3186 1654 Acked Read 8 Done Sent Done false
3187 1655 Acked Write 6 Done Sent Done false
3188 1656 Acked Read 6 Done Sent Done false
3189 1657 Acked Write 9 Done Sent Done false
3190 1658 Acked Write 1 Done Sent Done false
3191 1659 Acked Write 1 Done Sent Done false
3192 1660 Acked Write 5 Done Sent Done false
3193 1661 Acked Read 3 Done Sent Done false
3194 1662 Acked Read 2 Done Sent Done false
3195 1663 Acked Write 2 Done Sent Done false
3196 1664 Acked Read 5 Done Sent Done false
3197 1665 Acked Flush 0 Done Sent Done false
3198 1666 Acked Read 8 Done Sent Done false
3199 1667 Acked Read 9 Done Sent Done false
3200 1668 Acked Write 4 Done Sent Done false
3201 1669 Acked Write 5 Done Sent Done false
3202 1670 Acked Read 10 Done Sent Done false
3203 1671 Acked Flush 0 Done Sent Done false
3204 1672 Acked Write 2 Done Sent Done false
3205 1673 Acked Write 1 Done Sent Done false
3206 1674 Acked Read 10 Done Sent Done false
3207 1675 Acked Read 1 Done Sent Done false
3208 1676 Acked Read 5 Done Sent Done false
3209 1677 Acked Write 8 Done Sent Done false
3210 1678 Acked Read 6 Done Sent Done false
3211 1679 Acked Read 5 Done Sent Done false
3212 1680 Acked Read 5 Done Sent Done false
3213 1681 Acked Write 5 Done Sent Done false
3214 1682 Acked Read 2 Done Sent Done false
3215 1683 Acked Read 2 Done Sent Done false
3216 1684 Acked Write 9 Done Sent Done false
3217 1685 Acked Write 7 Done Sent Done false
3218 1686 Acked Flush 0 Done Sent Done false
3219 1687 Acked Read 7 Done Sent Done false
3220 1688 Acked Read 9 Done Sent Done false
3221 1689 Acked Write 9 Done Sent Done false
3222 1690 Acked Write 3 Done Sent Done false
3223 1691 Acked Write 1 Done Sent Done false
3224 1692 Acked Read 2 Done Sent Done false
3225 1693 Acked Read 8 Done Sent Done false
3226 1694 Acked Read 7 Done Sent Done false
3227 1695 Acked Read 2 Done Sent Done false
3228 1696 Acked Write 10 Done Sent Done false
3229 1697 Acked Read 8 Done Sent Done false
3230 1698 Acked Flush 0 Done Sent Done false
3231 1699 Acked Read 8 Done Sent Done false
3232 1700 Acked Write 3 Done Sent Done false
3233 1701 Acked Read 5 Done Sent Done false
3234 1702 Acked Write 1 Done Sent Done false
3235 1703 Acked Read 4 Done Sent Done false
3236 1704 Acked Read 9 Done Sent Done false
3237 1705 Acked Read 10 Done Sent Done false
3238 1706 Acked Write 8 Done Sent Done false
3239 1707 Acked Read 6 Done Sent Done false
3240 1708 Acked Write 9 Done Sent Done false
3241 1709 Acked Write 9 Done Sent Done false
3242 1710 Acked Write 2 Done Sent Done false
3243 1711 Acked Write 5 Done Sent Done false
3244 1712 Acked Write 1 Done Sent Done false
3245 1713 Acked Write 10 Done Sent Done false
3246 1714 Acked Write 1 Done Sent Done false
3247 1715 Acked Write 8 Done Sent Done false
3248 1716 Acked Write 9 Done Sent Done false
3249 1717 Acked Read 7 Done Sent Done false
3250 1718 Acked Write 9 Done Sent Done false
3251 1719 Acked Flush 0 Done Sent Done false
3252 1720 Acked Read 3 Done Sent Done false
3253 1721 Acked Write 2 Done Sent Done false
3254 1722 Acked Read 1 Done Sent Done false
3255 1723 Acked Read 5 Done Sent Done false
3256 1724 Acked Read 1 Done Sent Done false
3257 1725 Acked Write 6 Done Sent Done false
3258 1726 Acked Read 2 Done Sent Done false
3259 1727 Acked Write 7 Done Sent Done false
3260 1728 Acked Read 8 Done Sent Done false
3261 1729 Acked Read 1 Done Sent Done false
3262 1730 Acked Write 7 Done Sent Done false
3263 1731 Acked Read 7 Done Sent Done false
3264 1732 Acked Write 5 Done Sent Done false
3265 1733 Acked Read 9 Done Sent Done false
3266 1734 Acked Read 6 Done Sent Done false
3267 1735 Acked Write 9 Done Sent Done false
3268 1736 Acked Read 7 Done Sent Done false
3269 1737 Acked Read 6 Done Sent Done false
3270 1738 Acked Read 8 Done Sent Done false
3271 1739 Acked Write 6 Done Sent Done false
3272 1740 Acked Flush 0 Done Sent Done false
3273 1741 Acked Write 10 Done Sent Done false
3274 1742 Acked Read 7 Done Sent Done false
3275 1743 Acked Write 1 Done Sent Done false
3276 1744 Acked Read 3 Done Sent Done false
3277 1745 Acked Read 10 Done Sent Done false
3278 1746 Acked Read 9 Done Sent Done false
3279 1747 Acked Write 8 Done Sent Done false
3280 1748 Acked Write 9 Done Sent Done false
3281 1749 Acked Write 8 Done Sent Done false
3282 1750 Acked Write 1 Done Sent Done false
3283 1751 Acked Read 7 Done Sent Done false
3284 1752 Acked Read 10 Done Sent Done false
3285 1753 Acked Read 6 Done Sent Done false
3286 1754 Acked Write 7 Done Sent Done false
3287 1755 Acked Write 1 Done Sent Done false
3288 1756 Acked Write 6 Done Sent Done false
3289 1757 Acked Flush 0 Done Sent Done false
3290 1758 Acked Write 6 Done Sent Done false
3291 1759 Acked Flush 0 Done Sent Done false
3292 1760 Acked Write 1 Done Sent Done false
3293 1761 Acked Flush 0 Done Sent Done false
3294 1762 Acked Read 7 Done Sent Done false
3295 1763 Acked Write 4 Done Sent Done false
3296 1764 Acked Flush 0 Done Sent Done false
3297 1765 Acked Read 4 Done Sent Done false
3298 1766 Acked Write 3 Done Sent Done false
3299 1767 Acked Write 5 Done Sent Done false
3300 1768 Acked Read 6 Done Sent Done false
3301 1769 Acked Read 10 Done Sent Done false
3302 1770 Acked Flush 0 Done Sent Done false
3303 1771 Acked Write 9 Done Sent Done false
3304 1772 Acked Write 3 Done Sent Done false
3305 1773 Acked Write 8 Done Sent Done false
3306 1774 Acked Flush 0 Done Sent Done false
3307 1775 Acked Write 3 Done Sent Done false
3308 1776 Acked Read 1 Done Sent Done false
3309 1777 Acked Read 1 Done Sent Done false
3310 1778 Acked Write 8 Done Sent Done false
3311 1779 Acked Read 7 Done Sent Done false
3312 1780 Acked Write 5 Done Sent Done false
3313 1781 Acked Write 8 Done Sent Done false
3314 1782 Acked Read 7 Done Sent Done false
3315 1783 Acked Read 2 Done Sent Done false
3316 1784 Acked Flush 0 Done Sent Done false
3317 1785 Acked Read 5 Done Sent Done false
3318 1786 Acked Read 2 Done Sent Done false
3319 1787 Acked Read 6 Done Sent Done false
3320 1788 Acked Write 3 Done Sent Done false
3321 1789 Acked Read 4 Done Sent Done false
3322 1790 Acked Write 5 Done Sent Done false
3323 1791 Acked Read 6 Done Sent Done false
3324 1792 Acked Write 9 Done Sent Done false
3325 1793 Acked Write 8 Done Sent Done false
3326 1794 Acked Flush 0 Done Sent Done false
3327 1795 Acked Write 8 Done Sent Done false
3328 1796 Acked Read 4 Done Sent Done false
3329 1797 Acked Read 4 Done Sent Done false
3330 1798 Acked Write 8 Done Sent Done false
3331 1799 Acked Write 6 Done Sent Done false
3332 1800 Acked Read 2 Done Sent Done false
3333 1801 Acked Write 9 Done Sent Done false
3334 1802 Acked Read 6 Done Sent Done false
3335 1803 Acked Flush 0 Done Sent Done false
3336 1804 Acked Read 7 Done Sent Done false
3337 1805 Acked Flush 0 Done Sent Done false
3338 1806 Acked Read 5 Done Sent Done false
3339 1807 Acked Flush 0 Done Sent Done false
3340 1808 Acked Write 3 Done Sent Done false
3341 1809 Acked Write 3 Done Sent Done false
3342 1810 Acked Read 9 Done Sent Done false
3343 1811 Acked Read 3 Done Sent Done false
3344 1812 Acked Read 4 Done Sent Done false
3345 1813 Acked Write 2 Done Sent Done false
3346 1814 Acked Write 10 Done Sent Done false
3347 1815 Acked Read 3 Done Sent Done false
3348 1816 Acked Read 9 Done Sent Done false
3349 1817 Acked Write 5 Done Sent Done false
3350 1818 Acked Read 3 Done Sent Done false
3351 1819 Acked Read 9 Done Sent Done false
3352 1820 Acked Read 7 Done Sent Done false
3353 1821 Acked Read 5 Done Sent Done false
3354 1822 Acked Read 2 Done Sent Done false
3355 1823 Acked Flush 0 Done Sent Done false
3356 1824 Acked Flush 0 Done Sent Done false
3357 1825 Acked Write 3 Done Sent Done false
3358 1826 Acked Write 4 Done Sent Done false
3359 1827 Acked Write 9 Done Sent Done false
3360 1828 Acked Flush 0 Done Sent Done false
3361 1829 Acked Read 5 Done Sent Done false
3362 1830 Acked Write 7 Done Sent Done false
3363 1831 Acked Read 5 Done Sent Done false
3364 1832 Acked Write 5 Done Sent Done false
3365 1833 Acked Read 6 Done Sent Done false
3366 1834 Acked Write 4 Done Sent Done false
3367 1835 Acked Read 10 Done Sent Done false
3368 1836 Acked Flush 0 Done Sent Done false
3369 1837 Acked Read 7 Done Sent Done false
3370 1838 Acked Read 9 Done Sent Done false
3371 1839 Acked Read 1 Done Sent Done false
3372 1840 Acked Read 8 Done Sent Done false
3373 1841 Acked Read 3 Done Sent Done false
3374 1842 Acked Write 8 Done Sent Done false
3375 1843 Acked Write 4 Done Sent Done false
3376 1844 Acked Read 10 Done Sent Done false
3377 1845 Acked Read 1 Done Sent Done false
3378 1846 Acked Read 10 Done Sent Done false
3379 1847 Acked Read 2 Done Sent Done false
3380 1848 Acked Read 6 Done Sent Done false
3381 1849 Acked Write 10 Done Sent Done false
3382 1850 Acked Write 5 Done Sent Done false
3383 1851 Acked Flush 0 Done Sent Done false
3384 1852 Acked Read 7 Done Sent Done false
3385 1853 Acked Read 4 Done Sent Done false
3386 1854 Acked Read 3 Done Sent Done false
3387 1855 Acked Read 4 Done Sent Done false
3388 1856 Acked Read 1 Done Sent Done false
3389 1857 Acked Read 6 Done Sent Done false
3390 1858 Acked Write 2 Done Sent Done false
3391 1859 Acked Read 7 Done Sent Done false
3392 1860 Acked Read 4 Done Sent Done false
3393 1861 Acked Write 7 Done Sent Done false
3394 1862 Acked Write 5 Done Sent Done false
3395 1863 Acked Read 6 Done Sent Done false
3396 1864 Acked Read 10 Done Sent Done false
3397 1865 Acked Read 3 Done Sent Done false
3398 1866 Acked Read 1 Done Sent Done false
3399 1867 Acked Write 2 Done Sent Done false
3400 1868 Acked Read 2 Done Sent Done false
3401 1869 Acked Read 3 Done Sent Done false
3402 1870 Acked Flush 0 Done Sent Done false
3403 1871 Acked Read 9 Done Sent Done false
3404 1872 Acked Read 2 Done Sent Done false
3405 1873 Acked Read 6 Done Sent Done false
3406 1874 Acked Write 2 Done Sent Done false
3407 1875 Acked Read 9 Done Sent Done false
3408 1876 Acked Flush 0 Done Sent Done false
3409 1877 Acked Write 10 Done Sent Done false
3410 1878 Acked Write 8 Done Sent Done false
3411 1879 Acked Read 1 Done Sent Done false
3412 1880 Acked Read 1 Done Sent Done false
3413 1881 Acked Read 8 Done Sent Done false
3414 1882 Acked Read 8 Done Sent Done false
3415 1883 Acked Read 8 Done Sent Done false
3416 1884 Acked Read 5 Done Sent Done false
3417 1885 Acked Read 5 Done Sent Done false
3418 1886 Acked Write 1 Done Sent Done false
3419 1887 Acked Flush 0 Done Sent Done false
3420 1888 Acked Read 8 Done Sent Done false
3421 1889 Acked Read 10 Done Sent Done false
3422 1890 Acked Write 9 Done Sent Done false
3423 1891 Acked Write 3 Done Sent Done false
3424 1892 Acked Read 10 Done Sent Done false
3425 1893 Acked Read 10 Done Sent Done false
3426 1894 Acked Write 3 Done Sent Done false
3427 1895 Acked Write 8 Done Sent Done false
3428 1896 Acked Read 9 Done Sent Done false
3429 1897 Acked Write 2 Done Sent Done false
3430 1898 Acked Flush 0 Done Sent Done false
3431 1899 Acked Read 9 Done Sent Done false
3432 1900 Acked Write 4 Done Sent Done false
3433 1901 Acked Read 8 Done Sent Done false
3434 1902 Acked Read 7 Done Sent Done false
3435 1903 Acked Write 4 Done Sent Done false
3436 1904 Acked Read 7 Done Sent Done false
3437 STATES DS:0 DS:1 DS:2 TOTAL
3438 Sent 0 301 0 301
3439 Done 301 0 301 602
3440 Skipped 0 0 0 0
3441 Error 0 0 0 0
3442 Last Flush: 1898 1603 1898
3443 Downstairs last five completed: 1603 1602 1601 1600 1599
3444 Upstairs last five completed: 1904 1903 1902 1901 1900
3445 CLIENT: Up:0 ds:301 act:2
3446 [8820] got start action
3447 [8820] I am going to restart
3448 Starting downstairs at port 8820
3449 Make output file at "/tmp/dsc/downstairs-8820.txt"
3450 [8820][1] reports Starting
3451 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1208)
3452 [8820][1] reports Running
3453 ----------------------------------------------------------------
3454 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3455 States: Active Offline Active
3456 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3457 1604 Acked Write 7 Done Sent Done false
3458 1605 Acked Read 10 Done Sent Done false
3459 1606 Acked Read 8 Done Sent Done false
3460 1607 Acked Read 2 Done Sent Done false
3461 1608 Acked Write 6 Done Sent Done false
3462 1609 Acked Write 6 Done Sent Done false
3463 1610 Acked Write 4 Done Sent Done false
3464 1611 Acked Read 2 Done Sent Done false
3465 1612 Acked Write 3 Done Sent Done false
3466 1613 Acked Read 1 Done Sent Done false
3467 1614 Acked Read 10 Done Sent Done false
3468 1615 Acked Read 6 Done Sent Done false
3469 1616 Acked Read 10 Done Sent Done false
3470 1617 Acked Read 8 Done Sent Done false
3471 1618 Acked Read 6 Done Sent Done false
3472 1619 Acked Write 5 Done Sent Done false
3473 1620 Acked Read 7 Done Sent Done false
3474 1621 Acked Read 10 Done Sent Done false
3475 1622 Acked Read 6 Done Sent Done false
3476 1623 Acked Flush 0 Done Sent Done false
3477 1624 Acked Read 10 Done Sent Done false
3478 1625 Acked Write 2 Done Sent Done false
3479 1626 Acked Read 2 Done Sent Done false
3480 1627 Acked Write 4 Done Sent Done false
3481 1628 Acked Read 2 Done Sent Done false
3482 1629 Acked Read 1 Done Sent Done false
3483 1630 Acked Write 5 Done Sent Done false
3484 1631 Acked Write 8 Done Sent Done false
3485 1632 Acked Read 5 Done Sent Done false
3486 1633 Acked Flush 0 Done Sent Done false
3487 1634 Acked Read 6 Done Sent Done false
3488 1635 Acked Read 5 Done Sent Done false
3489 1636 Acked Read 2 Done Sent Done false
3490 1637 Acked Write 8 Done Sent Done false
3491 1638 Acked Write 10 Done Sent Done false
3492 1639 Acked Read 9 Done Sent Done false
3493 1640 Acked Write 8 Done Sent Done false
3494 1641 Acked Write 2 Done Sent Done false
3495 1642 Acked Read 8 Done Sent Done false
3496 1643 Acked Write 6 Done Sent Done false
3497 1644 Acked Flush 0 Done Sent Done false
3498 1645 Acked Write 8 Done Sent Done false
3499 1646 Acked Read 10 Done Sent Done false
3500 1647 Acked Read 2 Done Sent Done false
3501 1648 Acked Write 4 Done Sent Done false
3502 1649 Acked Write 6 Done Sent Done false
3503 1650 Acked Read 5 Done Sent Done false
3504 1651 Acked Read 5 Done Sent Done false
3505 1652 Acked Write 1 Done Sent Done false
3506 1653 Acked Write 10 Done Sent Done false
3507 1654 Acked Read 8 Done Sent Done false
3508 1655 Acked Write 6 Done Sent Done false
3509 1656 Acked Read 6 Done Sent Done false
3510 1657 Acked Write 9 Done Sent Done false
3511 1658 Acked Write 1 Done Sent Done false
3512 1659 Acked Write 1 Done Sent Done false
3513 1660 Acked Write 5 Done Sent Done false
3514 1661 Acked Read 3 Done Sent Done false
3515 1662 Acked Read 2 Done Sent Done false
3516 1663 Acked Write 2 Done Sent Done false
3517 1664 Acked Read 5 Done Sent Done false
3518 1665 Acked Flush 0 Done Sent Done false
3519 1666 Acked Read 8 Done Sent Done false
3520 1667 Acked Read 9 Done Sent Done false
3521 1668 Acked Write 4 Done Sent Done false
3522 1669 Acked Write 5 Done Sent Done false
3523 1670 Acked Read 10 Done Sent Done false
3524 1671 Acked Flush 0 Done Sent Done false
3525 1672 Acked Write 2 Done Sent Done false
3526 1673 Acked Write 1 Done Sent Done false
3527 1674 Acked Read 10 Done Sent Done false
3528 1675 Acked Read 1 Done Sent Done false
3529 1676 Acked Read 5 Done Sent Done false
3530 1677 Acked Write 8 Done Sent Done false
3531 1678 Acked Read 6 Done Sent Done false
3532 1679 Acked Read 5 Done Sent Done false
3533 1680 Acked Read 5 Done Sent Done false
3534 1681 Acked Write 5 Done Sent Done false
3535 1682 Acked Read 2 Done Sent Done false
3536 1683 Acked Read 2 Done Sent Done false
3537 1684 Acked Write 9 Done Sent Done false
3538 1685 Acked Write 7 Done Sent Done false
3539 1686 Acked Flush 0 Done Sent Done false
3540 1687 Acked Read 7 Done Sent Done false
3541 1688 Acked Read 9 Done Sent Done false
3542 1689 Acked Write 9 Done Sent Done false
3543 1690 Acked Write 3 Done Sent Done false
3544 1691 Acked Write 1 Done Sent Done false
3545 1692 Acked Read 2 Done Sent Done false
3546 1693 Acked Read 8 Done Sent Done false
3547 1694 Acked Read 7 Done Sent Done false
3548 1695 Acked Read 2 Done Sent Done false
3549 1696 Acked Write 10 Done Sent Done false
3550 1697 Acked Read 8 Done Sent Done false
3551 1698 Acked Flush 0 Done Sent Done false
3552 1699 Acked Read 8 Done Sent Done false
3553 1700 Acked Write 3 Done Sent Done false
3554 1701 Acked Read 5 Done Sent Done false
3555 1702 Acked Write 1 Done Sent Done false
3556 1703 Acked Read 4 Done Sent Done false
3557 1704 Acked Read 9 Done Sent Done false
3558 1705 Acked Read 10 Done Sent Done false
3559 1706 Acked Write 8 Done Sent Done false
3560 1707 Acked Read 6 Done Sent Done false
3561 1708 Acked Write 9 Done Sent Done false
3562 1709 Acked Write 9 Done Sent Done false
3563 1710 Acked Write 2 Done Sent Done false
3564 1711 Acked Write 5 Done Sent Done false
3565 1712 Acked Write 1 Done Sent Done false
3566 1713 Acked Write 10 Done Sent Done false
3567 1714 Acked Write 1 Done Sent Done false
3568 1715 Acked Write 8 Done Sent Done false
3569 1716 Acked Write 9 Done Sent Done false
3570 1717 Acked Read 7 Done Sent Done false
3571 1718 Acked Write 9 Done Sent Done false
3572 1719 Acked Flush 0 Done Sent Done false
3573 1720 Acked Read 3 Done Sent Done false
3574 1721 Acked Write 2 Done Sent Done false
3575 1722 Acked Read 1 Done Sent Done false
3576 1723 Acked Read 5 Done Sent Done false
3577 1724 Acked Read 1 Done Sent Done false
3578 1725 Acked Write 6 Done Sent Done false
3579 1726 Acked Read 2 Done Sent Done false
3580 1727 Acked Write 7 Done Sent Done false
3581 1728 Acked Read 8 Done Sent Done false
3582 1729 Acked Read 1 Done Sent Done false
3583 1730 Acked Write 7 Done Sent Done false
3584 1731 Acked Read 7 Done Sent Done false
3585 1732 Acked Write 5 Done Sent Done false
3586 1733 Acked Read 9 Done Sent Done false
3587 1734 Acked Read 6 Done Sent Done false
3588 1735 Acked Write 9 Done Sent Done false
3589 1736 Acked Read 7 Done Sent Done false
3590 1737 Acked Read 6 Done Sent Done false
3591 1738 Acked Read 8 Done Sent Done false
3592 1739 Acked Write 6 Done Sent Done false
3593 1740 Acked Flush 0 Done Sent Done false
3594 1741 Acked Write 10 Done Sent Done false
3595 1742 Acked Read 7 Done Sent Done false
3596 1743 Acked Write 1 Done Sent Done false
3597 1744 Acked Read 3 Done Sent Done false
3598 1745 Acked Read 10 Done Sent Done false
3599 1746 Acked Read 9 Done Sent Done false
3600 1747 Acked Write 8 Done Sent Done false
3601 1748 Acked Write 9 Done Sent Done false
3602 1749 Acked Write 8 Done Sent Done false
3603 1750 Acked Write 1 Done Sent Done false
3604 1751 Acked Read 7 Done Sent Done false
3605 1752 Acked Read 10 Done Sent Done false
3606 1753 Acked Read 6 Done Sent Done false
3607 1754 Acked Write 7 Done Sent Done false
3608 1755 Acked Write 1 Done Sent Done false
3609 1756 Acked Write 6 Done Sent Done false
3610 1757 Acked Flush 0 Done Sent Done false
3611 1758 Acked Write 6 Done Sent Done false
3612 1759 Acked Flush 0 Done Sent Done false
3613 1760 Acked Write 1 Done Sent Done false
3614 1761 Acked Flush 0 Done Sent Done false
3615 1762 Acked Read 7 Done Sent Done false
3616 1763 Acked Write 4 Done Sent Done false
3617 1764 Acked Flush 0 Done Sent Done false
3618 1765 Acked Read 4 Done Sent Done false
3619 1766 Acked Write 3 Done Sent Done false
3620 1767 Acked Write 5 Done Sent Done false
3621 1768 Acked Read 6 Done Sent Done false
3622 1769 Acked Read 10 Done Sent Done false
3623 1770 Acked Flush 0 Done Sent Done false
3624 1771 Acked Write 9 Done Sent Done false
3625 1772 Acked Write 3 Done Sent Done false
3626 1773 Acked Write 8 Done Sent Done false
3627 1774 Acked Flush 0 Done Sent Done false
3628 1775 Acked Write 3 Done Sent Done false
3629 1776 Acked Read 1 Done Sent Done false
3630 1777 Acked Read 1 Done Sent Done false
3631 1778 Acked Write 8 Done Sent Done false
3632 1779 Acked Read 7 Done Sent Done false
3633 1780 Acked Write 5 Done Sent Done false
3634 1781 Acked Write 8 Done Sent Done false
3635 1782 Acked Read 7 Done Sent Done false
3636 1783 Acked Read 2 Done Sent Done false
3637 1784 Acked Flush 0 Done Sent Done false
3638 1785 Acked Read 5 Done Sent Done false
3639 1786 Acked Read 2 Done Sent Done false
3640 1787 Acked Read 6 Done Sent Done false
3641 1788 Acked Write 3 Done Sent Done false
3642 1789 Acked Read 4 Done Sent Done false
3643 1790 Acked Write 5 Done Sent Done false
3644 1791 Acked Read 6 Done Sent Done false
3645 1792 Acked Write 9 Done Sent Done false
3646 1793 Acked Write 8 Done Sent Done false
3647 1794 Acked Flush 0 Done Sent Done false
3648 1795 Acked Write 8 Done Sent Done false
3649 1796 Acked Read 4 Done Sent Done false
3650 1797 Acked Read 4 Done Sent Done false
3651 1798 Acked Write 8 Done Sent Done false
3652 1799 Acked Write 6 Done Sent Done false
3653 1800 Acked Read 2 Done Sent Done false
3654 1801 Acked Write 9 Done Sent Done false
3655 1802 Acked Read 6 Done Sent Done false
3656 1803 Acked Flush 0 Done Sent Done false
3657 1804 Acked Read 7 Done Sent Done false
3658 1805 Acked Flush 0 Done Sent Done false
3659 1806 Acked Read 5 Done Sent Done false
3660 1807 Acked Flush 0 Done Sent Done false
3661 1808 Acked Write 3 Done Sent Done false
3662 1809 Acked Write 3 Done Sent Done false
3663 1810 Acked Read 9 Done Sent Done false
3664 1811 Acked Read 3 Done Sent Done false
3665 1812 Acked Read 4 Done Sent Done false
3666 1813 Acked Write 2 Done Sent Done false
3667 1814 Acked Write 10 Done Sent Done false
3668 1815 Acked Read 3 Done Sent Done false
3669 1816 Acked Read 9 Done Sent Done false
3670 1817 Acked Write 5 Done Sent Done false
3671 1818 Acked Read 3 Done Sent Done false
3672 1819 Acked Read 9 Done Sent Done false
3673 1820 Acked Read 7 Done Sent Done false
3674 1821 Acked Read 5 Done Sent Done false
3675 1822 Acked Read 2 Done Sent Done false
3676 1823 Acked Flush 0 Done Sent Done false
3677 1824 Acked Flush 0 Done Sent Done false
3678 1825 Acked Write 3 Done Sent Done false
3679 1826 Acked Write 4 Done Sent Done false
3680 1827 Acked Write 9 Done Sent Done false
3681 1828 Acked Flush 0 Done Sent Done false
3682 1829 Acked Read 5 Done Sent Done false
3683 1830 Acked Write 7 Done Sent Done false
3684 1831 Acked Read 5 Done Sent Done false
3685 1832 Acked Write 5 Done Sent Done false
3686 1833 Acked Read 6 Done Sent Done false
3687 1834 Acked Write 4 Done Sent Done false
3688 1835 Acked Read 10 Done Sent Done false
3689 1836 Acked Flush 0 Done Sent Done false
3690 1837 Acked Read 7 Done Sent Done false
3691 1838 Acked Read 9 Done Sent Done false
3692 1839 Acked Read 1 Done Sent Done false
3693 1840 Acked Read 8 Done Sent Done false
3694 1841 Acked Read 3 Done Sent Done false
3695 1842 Acked Write 8 Done Sent Done false
3696 1843 Acked Write 4 Done Sent Done false
3697 1844 Acked Read 10 Done Sent Done false
3698 1845 Acked Read 1 Done Sent Done false
3699 1846 Acked Read 10 Done Sent Done false
3700 1847 Acked Read 2 Done Sent Done false
3701 1848 Acked Read 6 Done Sent Done false
3702 1849 Acked Write 10 Done Sent Done false
3703 1850 Acked Write 5 Done Sent Done false
3704 1851 Acked Flush 0 Done Sent Done false
3705 1852 Acked Read 7 Done Sent Done false
3706 1853 Acked Read 4 Done Sent Done false
3707 1854 Acked Read 3 Done Sent Done false
3708 1855 Acked Read 4 Done Sent Done false
3709 1856 Acked Read 1 Done Sent Done false
3710 1857 Acked Read 6 Done Sent Done false
3711 1858 Acked Write 2 Done Sent Done false
3712 1859 Acked Read 7 Done Sent Done false
3713 1860 Acked Read 4 Done Sent Done false
3714 1861 Acked Write 7 Done Sent Done false
3715 1862 Acked Write 5 Done Sent Done false
3716 1863 Acked Read 6 Done Sent Done false
3717 1864 Acked Read 10 Done Sent Done false
3718 1865 Acked Read 3 Done Sent Done false
3719 1866 Acked Read 1 Done Sent Done false
3720 1867 Acked Write 2 Done Sent Done false
3721 1868 Acked Read 2 Done Sent Done false
3722 1869 Acked Read 3 Done Sent Done false
3723 1870 Acked Flush 0 Done Sent Done false
3724 1871 Acked Read 9 Done Sent Done false
3725 1872 Acked Read 2 Done Sent Done false
3726 1873 Acked Read 6 Done Sent Done false
3727 1874 Acked Write 2 Done Sent Done false
3728 1875 Acked Read 9 Done Sent Done false
3729 1876 Acked Flush 0 Done Sent Done false
3730 1877 Acked Write 10 Done Sent Done false
3731 1878 Acked Write 8 Done Sent Done false
3732 1879 Acked Read 1 Done Sent Done false
3733 1880 Acked Read 1 Done Sent Done false
3734 1881 Acked Read 8 Done Sent Done false
3735 1882 Acked Read 8 Done Sent Done false
3736 1883 Acked Read 8 Done Sent Done false
3737 1884 Acked Read 5 Done Sent Done false
3738 1885 Acked Read 5 Done Sent Done false
3739 1886 Acked Write 1 Done Sent Done false
3740 1887 Acked Flush 0 Done Sent Done false
3741 1888 Acked Read 8 Done Sent Done false
3742 1889 Acked Read 10 Done Sent Done false
3743 1890 Acked Write 9 Done Sent Done false
3744 1891 Acked Write 3 Done Sent Done false
3745 1892 Acked Read 10 Done Sent Done false
3746 1893 Acked Read 10 Done Sent Done false
3747 1894 Acked Write 3 Done Sent Done false
3748 1895 Acked Write 8 Done Sent Done false
3749 1896 Acked Read 9 Done Sent Done false
3750 1897 Acked Write 2 Done Sent Done false
3751 1898 Acked Flush 0 Done Sent Done false
3752 1899 Acked Read 9 Done Sent Done false
3753 1900 Acked Write 4 Done Sent Done false
3754 1901 Acked Read 8 Done Sent Done false
3755 1902 Acked Read 7 Done Sent Done false
3756 1903 Acked Write 4 Done Sent Done false
3757 1904 Acked Read 7 Done Sent Done false
3758 1905 Acked Flush 0 Done Sent Done false
3759 STATES DS:0 DS:1 DS:2 TOTAL
3760 Sent 0 302 0 302
3761 Done 302 0 302 604
3762 Skipped 0 0 0 0
3763 Error 0 0 0 0
3764 Last Flush: 1905 1603 1905
3765 Downstairs last five completed: 1603 1602 1601 1600 1599
3766 Upstairs last five completed: 1905 1904 1903 1902 1901
3767 CLIENT: Up:0 ds:302 act:2
37682025-01-03T08:31:40.033ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37692025-01-03T08:31:40.033ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50171) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37702025-01-03T08:31:40.034ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37712025-01-03T08:31:40.035ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37722025-01-03T08:31:40.035ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37732025-01-03T08:31:40.035ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37742025-01-03T08:31:40.035ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37752025-01-03T08:31:40.035ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37762025-01-03T08:31:40.035ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37772025-01-03T08:31:40.035ZINFOcrucible: [1] client re-new 302 jobs since flush 1603 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37782025-01-03T08:31:40.036ZINFOcrucible: [1] Replayed 302 jobs since flush: 1603 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
3779 ----------------------------------------------------------------
3780 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3781 Downstairs last five completed: 1905 1904 1903 1902 1901
3782 Upstairs last five completed: 1905 1904 1903 1902 1901
3783 CLIENT: Up:0 ds:0 act:3
3784 Replay: All jobs finished
3785 Main task has work to do, go find it
3786 got dsc Stop(2)
3787 stop 2
3788 Jan 03 08:31:42.285 INFO request completed, latency_us: 307, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 9c55d721-c01e-408e-b485-f3664a18c1d2, [8830] Got stop action so:false kr:false
3789 remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
37902025-01-03T08:31:42.286ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
37912025-01-03T08:31:42.286ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
3792 Jan 03 08:31:42.286 INFO request completed, latency_us: 133, {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:31:42.286814255Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
3793 response_code: 200, uri: /state/cid/2, method: GET, req_id: f0dda4be-adcc-456d-83ea-c8e1fcc2c000, remote_addr: 127{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:31:42.286853254Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
3794 .0.0.1:49669{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:31:42.286923902Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
3795 , local_addr: 127.0.0.1:9998
37962025-01-03T08:31:42.286ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
3797 [8830] Exited with: ExitStatus(unix_wait_status(9))
3798 [8830][2] reports Exit
3799 Jan 03 08:31:46.289 INFO request completed, latency_us: 199, response_code: 200, uri: /state/cid/2, method: GET, req_id: 1b0a5a0a-d10b-4c92-a271-06b51f7626db, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
3800 001/300 Read block 2838 len 28672
3801 002/300 Read block 2897 len 4096
3802 003/300 Write block 4981 len 4096 data: 2
3803 004/300 Read block 4347 len 16384
3804 005/300 Flush
3805 006/300 Write block 745 len 32768 data: 3 3 3 3 3 3 3 3
3806 007/300 Flush
3807 008/300 Write block 4615 len 36864 data: 2 2 2 2 2 2 2 3 3
3808 009/300 Flush
3809 010/300 Write block 4323 len 8192 data: 2 2
3810 011/300 Read block 599 len 24576
3811 012/300 Write block 4167 len 28672 data: 4 4 4 4 3 3 3
3812 013/300 Read block 1334 len 4096
3813 014/300 Write block 956 len 28672 data: 2 2 2 2 2 2 2
3814 015/300 Write block 4312 len 4096 data: 3
3815 016/300 Flush
3816 017/300 Write block 3277 len 20480 data: 2 2 2 3 2
3817 018/300 Read block 4530 len 8192
3818 019/300 Read block 1570 len 12288
3819 020/300 Write block 2295 len 4096 data: 2
3820 021/300 Read block 3908 len 24576
3821 022/300 Write block 1599 len 36864 data: 2 2 2 2 3 2 2 2 2
3822 023/300 Read block 515 len 28672
3823 024/300 Write block 2467 len 16384 data: 2 2 2 2
3824 025/300 Write block 4457 len 8192 data: 2 2
3825 026/300 Read block 27 len 32768
3826 027/300 Read block 4671 len 20480
3827 028/300 Flush
3828 029/300 Read block 3870 len 24576
3829 030/300 Flush
3830 031/300 Write block 814 len 16384 data: 2 2 2 2
3831 032/300 Read block 1763 len 28672
3832 033/300 Write block 2024 len 36864 data: 3 3 3 2 2 2 2 2 2
3833 034/300 Read block 554 len 32768
3834 035/300 Read block 1654 len 36864
3835 036/300 Read block 1510 len 12288
3836 037/300 Write block 1166 len 24576 data: 2 2 2 2 2 2
3837 038/300 Read block 4603 len 28672
3838 039/300 Write block 1736 len 24576 data: 3 3 3 2 2 2
3839 040/300 Read block 538 len 16384
3840 041/300 Read block 4021 len 8192
3841 042/300 Write block 1249 len 16384 data: 3 3 3 3
3842 043/300 Read block 758 len 4096
3843 044/300 Read block 1987 len 4096
3844 045/300 Write block 783 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3845 046/300 Read block 3631 len 32768
3846 047/300 Read block 4768 len 20480
3847 048/300 Read block 4918 len 8192
3848 049/300 Write block 2485 len 12288 data: 2 2 2
3849 050/300 Write block 2806 len 36864 data: 3 3 3 3 3 3 3 3 3
3850 051/300 Write block 2779 len 8192 data: 2 2
3851 052/300 Write block 2371 len 16384 data: 2 2 2 2
3852 053/300 Write block 793 len 40960 data: 2 2 2 2 2 2 3 2 2 2
3853 054/300 Write block 108 len 12288 data: 3 3 3
3854 055/300 Read block 279 len 8192
3855 056/300 Write block 4662 len 40960 data: 3 3 3 3 3 3 3 3 3 2
3856 057/300 Flush
3857 058/300 Write block 1341 len 4096 data: 2
3858 059/300 Read block 959 len 12288
3859 060/300 Read block 457 len 28672
3860 061/300 Write block 4756 len 24576 data: 2 2 2 2 2 2
3861 062/300 Read block 4623 len 12288
3862 063/300 Write block 865 len 12288 data: 2 2 2
3863 064/300 Write block 1818 len 20480 data: 2 2 2 2 2
3864 065/300 Flush
3865 066/300 Read block 4243 len 28672
3866 067/300 Read block 2950 len 24576
3867 068/300 Write block 1405 len 20480 data: 3 3 3 3 3
3868 069/300 Write block 4880 len 12288 data: 2 2 2
3869 070/300 Read block 2992 len 32768
3870 071/300 Read block 3057 len 8192
3871 072/300 Read block 1810 len 8192
3872 073/300 Write block 1441 len 28672 data: 3 3 3 3 3 3 3
3873 074/300 Read block 975 len 12288
3874 075/300 Write block 3758 len 24576 data: 2 2 2 3 3 3
3875 076/300 Write block 4578 len 32768 data: 2 2 2 2 2 2 2 2
3876 077/300 Read block 2112 len 36864
3877 078/300 Read block 4522 len 36864
3878 079/300 Read block 3715 len 16384
3879 080/300 Write block 4001 len 4096 data: 3
3880 081/300 Write block 1503 len 24576 data: 2 2 2 2 2 2
3881 082/300 Read block 595 len 12288
3882 083/300 Read block 294 len 40960
3883 084/300 Flush
3884 085/300 Write block 4881 len 36864 data: 3 3 2 2 2 2 2 2 2
3885 086/300 Read block 2278 len 40960
3886 087/300 Write block 925 len 8192 data: 2 2
3887 088/300 Read block 3764 len 8192
3888 089/300 Read block 3798 len 24576
3889 090/300 Write block 4641 len 36864 data: 3 4 3 4 4 4 3 3 2
3890 091/300 Write block 2229 len 4096 data: 2
3891 092/300 Read block 2643 len 40960
3892 093/300 Flush
3893 094/300 Write block 2168 len 4096 data: 2
3894 095/300 Read block 68 len 24576
3895 096/300 Write block 4471 len 36864 data: 2 2 2 2 2 2 2 3 3
3896 097/300 Flush
3897 098/300 Write block 4943 len 8192 data: 3 3
3898 099/300 Read block 3618 len 40960
3899 100/300 Read block 4248 len 36864
3900 101/300 Read block 1106 len 24576
3901 102/300 Read block 4363 len 32768
3902 103/300 Write block 1484 len 32768 data: 2 2 2 2 3 3 3 3
3903 104/300 Write block 383 len 16384 data: 2 2 2 2
3904 105/300 Flush
3905 106/300 Write block 4919 len 20480 data: 3 3 3 2 2
3906 107/300 Read block 1323 len 24576
3907 108/300 Flush
3908 109/300 Flush
3909 110/300 Write block 3644 len 4096 data: 2
3910 111/300 Read block 4120 len 36864
3911 112/300 Read block 4844 len 16384
3912 113/300 Read block 1433 len 16384
3913 114/300 Read block 1693 len 8192
3914 115/300 Write block 3644 len 40960 data: 3 2 2 2 2 2 2 2 2 3
3915 116/300 Read block 1900 len 16384
3916 117/300 Flush
3917 118/300 Write block 4838 len 32768 data: 3 2 2 2 2 2 2 2
3918 119/300 Write block 3786 len 40960 data: 2 2 2 3 3 3 3 3 3 3
3919 120/300 Write block 4769 len 4096 data: 3
3920 121/300 Flush
3921 122/300 Flush
3922 123/300 Read block 3064 len 20480
3923 124/300 Read block 2600 len 4096
3924 125/300 Read block 4743 len 24576
3925 126/300 Write block 2631 len 8192 data: 3 3
3926 127/300 Write block 4724 len 12288 data: 3 3 3
3927 128/300 Write block 3743 len 4096 data: 2
3928 129/300 Read block 1656 len 40960
3929 130/300 Read block 1992 len 12288
3930 131/300 Read block 4153 len 32768
3931 132/300 Write block 197 len 16384 data: 4 4 3 2
3932 133/300 Read block 2323 len 28672
3933 134/300 Read block 1468 len 24576
3934 135/300 Write block 3306 len 28672 data: 2 3 3 3 3 3 3
3935 136/300 Write block 4135 len 16384 data: 2 2 3 3
3936 137/300 Read block 2126 len 28672
3937 138/300 Read block 1424 len 16384
3938 139/300 Read block 2945 len 12288
3939 140/300 Flush
3940 141/300 Read block 2592 len 32768
3941 142/300 Read block 868 len 8192
3942 143/300 Read block 4702 len 28672
3943 144/300 Read block 4982 len 8192
3944 145/300 Write block 3191 len 40960 data: 3 3 3 3 3 3 3 3 3 4
3945 146/300 Read block 4499 len 16384
3946 147/300 Flush
3947 148/300 Write block 2620 len 28672 data: 3 3 3 3 3 4 4
3948 149/300 Read block 3617 len 4096
3949 150/300 Read block 2549 len 20480
3950 151/300 Read block 2226 len 24576
3951 152/300 Read block 2443 len 8192
3952 153/300 Read block 2459 len 36864
3953 154/300 Write block 3624 len 40960 data: 2 2 2 2 2 2 2 2 3 3
3954 155/300 Read block 3535 len 24576
3955 156/300 Read block 783 len 20480
3956 157/300 Write block 1443 len 24576 data: 4 4 4 4 4 4
3957 158/300 Write block 4802 len 24576 data: 3 3 3 3 3 3
3958 159/300 Read block 370 len 28672
3959 160/300 Read block 1316 len 36864
3960 161/300 Write block 1291 len 32768 data: 2 2 2 2 2 2 2 2
3961 162/300 Write block 4938 len 24576 data: 2 2 2 2 3 4
3962 163/300 Write block 3960 len 36864 data: 2 3 2 2 2 2 2 2 2
3963 164/300 Read block 2653 len 28672
3964 165/300 Read block 3820 len 36864
3965 166/300 Write block 3816 len 32768 data: 3 3 2 2 2 3 3 3
3966 167/300 Read block 3189 len 24576
3967 168/300 Read block 2477 len 8192
3968 169/300 Read block 2701 len 12288
3969 170/300 Write block 1786 len 32768 data: 2 2 2 2 2 2 2 2
3970 171/300 Read block 3103 len 32768
3971 172/300 Write block 786 len 4096 data: 3
3972 173/300 Write block 3779 len 32768 data: 2 3 2 2 2 2 2 3
3973 174/300 Read block 2681 len 8192
3974 175/300 Read block 3363 len 28672
3975 176/300 Write block 53 len 36864 data: 2 2 3 3 3 3 3 3 2
3976 177/300 Write block 654 len 20480 data: 3 3 3 3 3
3977 178/300 Flush
3978 179/300 Read block 120 len 24576
3979 180/300 Write block 3288 len 32768 data: 2 2 3 3 3 2 2 2
3980 181/300 Write block 4898 len 12288 data: 2 2 2
3981 182/300 Read block 3931 len 40960
3982 183/300 Read block 4660 len 16384
3983 184/300 Flush
3984 185/300 Read block 2844 len 12288
3985 186/300 Write block 471 len 16384 data: 3 3 3 2
3986 187/300 Write block 4263 len 36864 data: 2 2 2 2 2 2 2 2 2
3987 188/300 Write block 4557 len 16384 data: 2 2 3 3
3988 189/300 Read block 2168 len 12288
3989 190/300 Read block 3959 len 32768
3990 191/300 Read block 685 len 12288
3991 192/300 Read block 2640 len 8192
3992 193/300 Read block 3784 len 20480
3993 194/300 Write block 1735 len 28672 data: 2 4 4 4 3 3 3
3994 195/300 Write block 4355 len 12288 data: 2 2 2
3995 196/300 Write block 3493 len 4096 data: 2
3996 197/300 Write block 4853 len 12288 data: 2 2 2
3997 198/300 Read block 1827 len 28672
3998 199/300 Read block 2816 len 4096
3999 200/300 Read block 2397 len 36864
4000 201/300 Read block 2020 len 12288
4001 202/300 Write block 1310 len 32768 data: 2 2 2 2 2 2 2 2
4002 203/300 Read block 2625 len 32768
4003 204/300 Read block 1914 len 24576
4004 205/300 Read block 4165 len 4096
4005 206/300 Write block 1043 len 20480 data: 2 2 2 2 2
4006 207/300 Write block 4043 len 4096 data: 4
4007 208/300 Write block 2239 len 8192 data: 2 2
4008 209/300 Read block 444 len 12288
4009 210/300 Read block 4373 len 36864
4010 211/300 Read block 3554 len 40960
4011 212/300 Write block 3832 len 8192 data: 3 3
4012 213/300 Write block 1355 len 36864 data: 3 3 3 3 2 2 2 2 3
4013 214/300 Read block 1512 len 24576
4014 215/300 Write block 521 len 16384 data: 4 3 3 3
4015 216/300 Read block 3886 len 40960
4016 217/300 Write block 309 len 8192 data: 3 2
4017 218/300 Read block 3585 len 20480
4018 219/300 Flush
4019 220/300 Write block 2453 len 28672 data: 3 3 3 3 3 3 3
4020 221/300 Flush
4021 222/300 Write block 1263 len 32768 data: 2 2 2 2 2 2 2 2
4022 223/300 Read block 4781 len 20480
4023 224/300 Read block 4793 len 20480
4024 225/300 Read block 4447 len 16384
4025 226/300 Write block 1174 len 32768 data: 2 2 4 4 4 4 3 3
4026 227/300 Write block 633 len 36864 data: 3 2 2 2 2 3 3 3 3
4027 228/300 Flush
4028 229/300 Flush
4029 230/300 Read block 682 len 32768
4030 231/300 Read block 251 len 20480
4031 232/300 Read block 2482 len 12288
4032 233/300 Flush
4033 234/300 Write block 11 len 28672 data: 2 2 2 2 2 2 2
4034 235/300 Write block 1988 len 4096 data: 3
4035 236/300 Write block 4203 len 12288 data: 3 4 4
4036 237/300 Read block 4718 len 4096
4037 238/300 Write block 2029 len 12288 data: 3 3 3
4038 239/300 Read block 519 len 4096
4039 240/300 Read block 3638 len 20480
4040 241/300 Read block 4743 len 28672
4041 242/300 Read block 4333 len 28672
4042 243/300 Read block 3876 len 24576
4043 244/300 Read block 425 len 4096
4044 245/300 Write block 2066 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4045 246/300 Write block 820 len 12288 data: 2 2 2
4046 247/300 Write block 1651 len 20480 data: 3 2 2 2 2
4047 248/300 Read block 1235 len 24576
4048 249/300 Read block 2989 len 16384
4049 250/300 Read block 184 len 40960
4050 251/300 Read block 1256 len 12288
4051 252/300 Flush
4052 253/300 Read block 1441 len 32768
4053 254/300 Read block 2381 len 8192
4054 255/300 Write block 1664 len 4096 data: 2
4055 256/300 Read block 2702 len 4096
4056 257/300 Read block 4111 len 24576
4057 258/300 Read block 4414 len 32768
4058 259/300 Write block 284 len 32768 data: 2 2 2 2 2 2 2 2
4059 260/300 Write block 1142 len 32768 data: 2 2 2 3 3 3 2 2
4060 261/300 Flush
4061 262/300 Read block 4824 len 24576
4062 263/300 Write block 2121 len 20480 data: 2 2 2 3 3
4063 264/300 Read block 3321 len 32768
4064 265/300 Flush
4065 266/300 Read block 3517 len 20480
4066 267/300 Read block 3101 len 40960
4067 268/300 Write block 1018 len 20480 data: 2 2 2 2 2
4068 269/300 Write block 2361 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4069 270/300 Read block 3435 len 32768
4070 271/300 Read block 3323 len 12288
4071 272/300 Write block 534 len 12288 data: 2 2 2
4072 273/300 Read block 15 len 20480
4073 274/300 Write block 3463 len 36864 data: 2 2 3 3 3 3 3 3 3
4074 275/300 Write block 3599 len 28672 data: 2 2 2 2 2 2 3
4075 276/300 Read block 2815 len 24576
4076 277/300 Read block 2363 len 16384
4077 278/300 Write block 1004 len 28672 data: 2 2 2 2 2 2 2
4078 279/300 Read block 1113 len 20480
4079 280/300 Write block 4978 len 4096 data: 2
4080 281/300 Write block 4887 len 4096 data: 3
4081 282/300 Read block 3547 len 4096
4082 283/300 Read block 695 len 4096
4083 284/300 Flush
4084 285/300 Write block 2629 len 4096 data: 4
4085 286/300 Read block 4775 len 40960
4086 287/300 Read block 4225 len 32768
4087 288/300 Read block 3437 len 8192
4088 289/300 Write block 766 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4089 290/300 Read block 124 len 32768
4090 291/300 Read block 4092 len 28672
4091 292/300 Write block 1907 len 16384 data: 3 3 3 3
4092 293/300 Read block 3037 len 16384
4093 294/300 Read block 4369 len 24576
4094 295/300 Flush
4095 296/300 Write block 3671 len 28672 data: 2 2 2 2 2 2 2
4096 297/300 Read block 990 len 32768
4097 298/300 Read block 408 len 4096
4098 299/300 Read block 3466 len 8192
4099 300/300 Read block 2967 len 8192
4100 301/300 Write block 3142 len 4096 data: 2
4101 Main task has work to do, go find it
4102 got dsc Start(2)
4103 start 2
4104 Jan 03 08:31:46.529 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/2, method: POST, req_id: 1f493560-ff27-4265-a6f3-e4240bb406c7, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
4105 Replay: started 2, returned:Ok(())
4106 ----------------------------------------------------------------
4107 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4108 States: Active Active Offline
4109 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4110 1906 Acked Read 7 Done Done Sent false
4111 1907 Acked Read 1 Done Done Sent false
4112 1908 Acked Write 1 Done Done Sent false
4113 1909 Acked Read 4 Done Done Sent false
4114 1910 Acked Flush 0 Done Done Sent false
4115 1911 Acked Write 8 Done Done Sent false
4116 1912 Acked Flush 0 Done Done Sent false
4117 1913 Acked Write 9 Done Done Sent false
4118 1914 Acked Flush 0 Done Done Sent false
4119 1915 Acked Write 2 Done Done Sent false
4120 1916 Acked Read 6 Done Done Sent false
4121 1917 Acked Write 7 Done Done Sent false
4122 1918 Acked Read 1 Done Done Sent false
4123 1919 Acked Write 7 Done Done Sent false
4124 1920 Acked Write 1 Done Done Sent false
4125 1921 Acked Flush 0 Done Done Sent false
4126 1922 Acked Write 5 Done Done Sent false
4127 1923 Acked Read 2 Done Done Sent false
4128 1924 Acked Read 3 Done Done Sent false
4129 1925 Acked Write 1 Done Done Sent false
4130 1926 Acked Read 6 Done Done Sent false
4131 1927 Acked Write 9 Done Done Sent false
4132 1928 Acked Read 7 Done Done Sent false
4133 1929 Acked Write 4 Done Done Sent false
4134 1930 Acked Write 2 Done Done Sent false
4135 1931 Acked Read 8 Done Done Sent false
4136 1932 Acked Read 5 Done Done Sent false
4137 1933 Acked Flush 0 Done Done Sent false
4138 1934 Acked Read 6 Done Done Sent false
4139 1935 Acked Flush 0 Done Done Sent false
4140 1936 Acked Write 4 Done Done Sent false
4141 1937 Acked Read 7 Done Done Sent false
4142 1938 Acked Write 9 Done Done Sent false
4143 1939 Acked Read 8 Done Done Sent false
4144 1940 Acked Read 9 Done Done Sent false
4145 1941 Acked Read 3 Done Done Sent false
4146 1942 Acked Write 6 Done Done Sent false
4147 1943 Acked Read 7 Done Done Sent false
4148 1944 Acked Write 6 Done Done Sent false
4149 1945 Acked Read 4 Done Done Sent false
4150 1946 Acked Read 2 Done Done Sent false
4151 1947 Acked Write 4 Done Done Sent false
4152 1948 Acked Read 1 Done Done Sent false
4153 1949 Acked Read 1 Done Done Sent false
4154 1950 Acked Write 10 Done Done Sent false
4155 1951 Acked Read 8 Done Done Sent false
4156 1952 Acked Read 5 Done Done Sent false
4157 1953 Acked Read 2 Done Done Sent false
4158 1954 Acked Write 3 Done Done Sent false
4159 1955 Acked Write 9 Done Done Sent false
4160 1956 Acked Write 2 Done Done Sent false
4161 1957 Acked Write 4 Done Done Sent false
4162 1958 Acked Write 10 Done Done Sent false
4163 1959 Acked Write 3 Done Done Sent false
4164 1960 Acked Read 2 Done Done Sent false
4165 1961 Acked Write 10 Done Done Sent false
4166 1962 Acked Flush 0 Done Done Sent false
4167 1963 Acked Write 1 Done Done Sent false
4168 1964 Acked Read 3 Done Done Sent false
4169 1965 Acked Read 7 Done Done Sent false
4170 1966 Acked Write 6 Done Done Sent false
4171 1967 Acked Read 3 Done Done Sent false
4172 1968 Acked Write 3 Done Done Sent false
4173 1969 Acked Write 5 Done Done Sent false
4174 1970 Acked Flush 0 Done Done Sent false
4175 1971 Acked Read 7 Done Done Sent false
4176 1972 Acked Read 6 Done Done Sent false
4177 1973 Acked Write 5 Done Done Sent false
4178 1974 Acked Write 3 Done Done Sent false
4179 1975 Acked Read 8 Done Done Sent false
4180 1976 Acked Read 2 Done Done Sent false
4181 1977 Acked Read 2 Done Done Sent false
4182 1978 Acked Write 7 Done Done Sent false
4183 1979 Acked Read 3 Done Done Sent false
4184 1980 Acked Write 6 Done Done Sent false
4185 1981 Acked Write 8 Done Done Sent false
4186 1982 Acked Read 9 Done Done Sent false
4187 1983 Acked Read 9 Done Done Sent false
4188 1984 Acked Read 4 Done Done Sent false
4189 1985 Acked Write 1 Done Done Sent false
4190 1986 Acked Write 6 Done Done Sent false
4191 1987 Acked Read 3 Done Done Sent false
4192 1988 Acked Read 10 Done Done Sent false
4193 1989 Acked Flush 0 Done Done Sent false
4194 1990 Acked Write 9 Done Done Sent false
4195 1991 Acked Read 10 Done Done Sent false
4196 1992 Acked Write 2 Done Done Sent false
4197 1993 Acked Read 2 Done Done Sent false
4198 1994 Acked Read 6 Done Done Sent false
4199 1995 Acked Write 9 Done Done Sent false
4200 1996 Acked Write 1 Done Done Sent false
4201 1997 Acked Read 10 Done Done Sent false
4202 1998 Acked Flush 0 Done Done Sent false
4203 1999 Acked Write 1 Done Done Sent false
4204 2000 Acked Read 6 Done Done Sent false
4205 2001 Acked Write 9 Done Done Sent false
4206 2002 Acked Flush 0 Done Done Sent false
4207 2003 Acked Write 2 Done Done Sent false
4208 2004 Acked Read 10 Done Done Sent false
4209 2005 Acked Read 9 Done Done Sent false
4210 2006 Acked Read 6 Done Done Sent false
4211 2007 Acked Read 8 Done Done Sent false
4212 2008 Acked Write 8 Done Done Sent false
4213 2009 Acked Write 4 Done Done Sent false
4214 2010 Acked Flush 0 Done Done Sent false
4215 2011 Acked Write 5 Done Done Sent false
4216 2012 Acked Read 6 Done Done Sent false
4217 2013 Acked Flush 0 Done Done Sent false
4218 2014 Acked Flush 0 Done Done Sent false
4219 2015 Acked Write 1 Done Done Sent false
4220 2016 Acked Read 9 Done Done Sent false
4221 2017 Acked Read 4 Done Done Sent false
4222 2018 Acked Read 4 Done Done Sent false
4223 2019 Acked Read 2 Done Done Sent false
4224 2020 Acked Write 10 Done Done Sent false
4225 2021 Acked Read 4 Done Done Sent false
4226 2022 Acked Flush 0 Done Done Sent false
4227 2023 Acked Write 8 Done Done Sent false
4228 2024 Acked Write 10 Done Done Sent false
4229 2025 Acked Write 1 Done Done Sent false
4230 2026 Acked Flush 0 Done Done Sent false
4231 2027 Acked Flush 0 Done Done Sent false
4232 2028 Acked Read 5 Done Done Sent false
4233 2029 Acked Read 1 Done Done Sent false
4234 2030 Acked Read 6 Done Done Sent false
4235 2031 Acked Write 2 Done Done Sent false
4236 2032 Acked Write 3 Done Done Sent false
4237 2033 Acked Write 1 Done Done Sent false
4238 2034 Acked Read 10 Done Done Sent false
4239 2035 Acked Read 3 Done Done Sent false
4240 2036 Acked Read 8 Done Done Sent false
4241 2037 Acked Write 4 Done Done Sent false
4242 2038 Acked Read 7 Done Done Sent false
4243 2039 Acked Read 6 Done Done Sent false
4244 2040 Acked Write 7 Done Done Sent false
4245 2041 Acked Write 4 Done Done Sent false
4246 2042 Acked Read 7 Done Done Sent false
4247 2043 Acked Read 4 Done Done Sent false
4248 2044 Acked Read 3 Done Done Sent false
4249 2045 Acked Flush 0 Done Done Sent false
4250 2046 Acked Read 8 Done Done Sent false
4251 2047 Acked Read 2 Done Done Sent false
4252 2048 Acked Read 7 Done Done Sent false
4253 2049 Acked Read 2 Done Done Sent false
4254 2050 Acked Write 10 Done Done Sent false
4255 2051 Acked Read 4 Done Done Sent false
4256 2052 Acked Flush 0 Done Done Sent false
4257 2053 Acked Write 7 Done Done Sent false
4258 2054 Acked Read 1 Done Done Sent false
4259 2055 Acked Read 5 Done Done Sent false
4260 2056 Acked Read 6 Done Done Sent false
4261 2057 Acked Read 2 Done Done Sent false
4262 2058 Acked Read 9 Done Done Sent false
4263 2059 Acked Write 10 Done Done Sent false
4264 2060 Acked Read 6 Done Done Sent false
4265 2061 Acked Read 5 Done Done Sent false
4266 2062 Acked Write 6 Done Done Sent false
4267 2063 Acked Write 6 Done Done Sent false
4268 2064 Acked Read 7 Done Done Sent false
4269 2065 Acked Read 9 Done Done Sent false
4270 2066 Acked Write 8 Done Done Sent false
4271 2067 Acked Write 6 Done Done Sent false
4272 2068 Acked Write 9 Done Done Sent false
4273 2069 Acked Read 7 Done Done Sent false
4274 2070 Acked Read 9 Done Done Sent false
4275 2071 Acked Write 8 Done Done Sent false
4276 2072 Acked Read 6 Done Done Sent false
4277 2073 Acked Read 2 Done Done Sent false
4278 2074 Acked Read 3 Done Done Sent false
4279 2075 Acked Write 8 Done Done Sent false
4280 2076 Acked Read 8 Done Done Sent false
4281 2077 Acked Write 1 Done Done Sent false
4282 2078 Acked Write 8 Done Done Sent false
4283 2079 Acked Read 2 Done Done Sent false
4284 2080 Acked Read 7 Done Done Sent false
4285 2081 Acked Write 9 Done Done Sent false
4286 2082 Acked Write 5 Done Done Sent false
4287 2083 Acked Flush 0 Done Done Sent false
4288 2084 Acked Read 6 Done Done Sent false
4289 2085 Acked Write 8 Done Done Sent false
4290 2086 Acked Write 3 Done Done Sent false
4291 2087 Acked Read 10 Done Done Sent false
4292 2088 Acked Read 4 Done Done Sent false
4293 2089 Acked Flush 0 Done Done Sent false
4294 2090 Acked Read 3 Done Done Sent false
4295 2091 Acked Write 4 Done Done Sent false
4296 2092 Acked Write 9 Done Done Sent false
4297 2093 Acked Write 4 Done Done Sent false
4298 2094 Acked Read 3 Done Done Sent false
4299 2095 Acked Read 8 Done Done Sent false
4300 2096 Acked Read 3 Done Done Sent false
4301 2097 Acked Read 2 Done Done Sent false
4302 2098 Acked Read 5 Done Done Sent false
4303 2099 Acked Write 7 Done Done Sent false
4304 2100 Acked Write 3 Done Done Sent false
4305 2101 Acked Write 1 Done Done Sent false
4306 2102 Acked Write 3 Done Done Sent false
4307 2103 Acked Read 7 Done Done Sent false
4308 2104 Acked Read 1 Done Done Sent false
4309 2105 Acked Read 9 Done Done Sent false
4310 2106 Acked Read 3 Done Done Sent false
4311 2107 Acked Write 8 Done Done Sent false
4312 2108 Acked Read 8 Done Done Sent false
4313 2109 Acked Read 6 Done Done Sent false
4314 2110 Acked Read 1 Done Done Sent false
4315 2111 Acked Write 5 Done Done Sent false
4316 2112 Acked Write 1 Done Done Sent false
4317 2113 Acked Write 2 Done Done Sent false
4318 2114 Acked Read 3 Done Done Sent false
4319 2115 Acked Read 9 Done Done Sent false
4320 2116 Acked Read 10 Done Done Sent false
4321 2117 Acked Write 2 Done Done Sent false
4322 2118 Acked Write 9 Done Done Sent false
4323 2119 Acked Read 6 Done Done Sent false
4324 2120 Acked Write 4 Done Done Sent false
4325 2121 Acked Read 10 Done Done Sent false
4326 2122 Acked Write 2 Done Done Sent false
4327 2123 Acked Read 5 Done Done Sent false
4328 2124 Acked Flush 0 Done Done Sent false
4329 2125 Acked Write 7 Done Done Sent false
4330 2126 Acked Flush 0 Done Done Sent false
4331 2127 Acked Write 8 Done Done Sent false
4332 2128 Acked Read 5 Done Done Sent false
4333 2129 Acked Read 5 Done Done Sent false
4334 2130 Acked Read 4 Done Done Sent false
4335 2131 Acked Write 8 Done Done Sent false
4336 2132 Acked Write 9 Done Done Sent false
4337 2133 Acked Flush 0 Done Done Sent false
4338 2134 Acked Flush 0 Done Done Sent false
4339 2135 Acked Read 8 Done Done Sent false
4340 2136 Acked Read 5 Done Done Sent false
4341 2137 Acked Read 3 Done Done Sent false
4342 2138 Acked Flush 0 Done Done Sent false
4343 2139 Acked Write 7 Done Done Sent false
4344 2140 Acked Write 1 Done Done Sent false
4345 2141 Acked Write 3 Done Done Sent false
4346 2142 Acked Read 1 Done Done Sent false
4347 2143 Acked Write 3 Done Done Sent false
4348 2144 Acked Read 1 Done Done Sent false
4349 2145 Acked Read 5 Done Done Sent false
4350 2146 Acked Read 7 Done Done Sent false
4351 2147 Acked Read 7 Done Done Sent false
4352 2148 Acked Read 6 Done Done Sent false
4353 2149 Acked Read 1 Done Done Sent false
4354 2150 Acked Write 10 Done Done Sent false
4355 2151 Acked Write 3 Done Done Sent false
4356 2152 Acked Write 5 Done Done Sent false
4357 2153 Acked Read 6 Done Done Sent false
4358 2154 Acked Read 4 Done Done Sent false
4359 2155 Acked Read 10 Done Done Sent false
4360 2156 Acked Read 3 Done Done Sent false
4361 2157 Acked Flush 0 Done Done Sent false
4362 2158 Acked Read 8 Done Done Sent false
4363 2159 Acked Read 2 Done Done Sent false
4364 2160 Acked Write 1 Done Done Sent false
4365 2161 Acked Read 1 Done Done Sent false
4366 2162 Acked Read 6 Done Done Sent false
4367 2163 Acked Read 8 Done Done Sent false
4368 2164 Acked Write 8 Done Done Sent false
4369 2165 Acked Write 8 Done Done Sent false
4370 2166 Acked Flush 0 Done Done Sent false
4371 2167 Acked Read 6 Done Done Sent false
4372 2168 Acked Write 5 Done Done Sent false
4373 2169 Acked Read 8 Done Done Sent false
4374 2170 Acked Flush 0 Done Done Sent false
4375 2171 Acked Read 5 Done Done Sent false
4376 2172 Acked Read 10 Done Done Sent false
4377 2173 Acked Write 5 Done Done Sent false
4378 2174 Acked Write 10 Done Done Sent false
4379 2175 Acked Read 8 Done Done Sent false
4380 2176 Acked Read 3 Done Done Sent false
4381 2177 Acked Write 3 Done Done Sent false
4382 2178 Acked Read 5 Done Done Sent false
4383 2179 Acked Write 9 Done Done Sent false
4384 2180 Acked Write 7 Done Done Sent false
4385 2181 Acked Read 6 Done Done Sent false
4386 2182 Acked Read 4 Done Done Sent false
4387 2183 Acked Write 7 Done Done Sent false
4388 2184 Acked Read 5 Done Done Sent false
4389 2185 Acked Write 1 Done Done Sent false
4390 2186 Acked Write 1 Done Done Sent false
4391 2187 Acked Read 1 Done Done Sent false
4392 2188 Acked Read 1 Done Done Sent false
4393 2189 Acked Flush 0 Done Done Sent false
4394 2190 Acked Write 1 Done Done Sent false
4395 2191 Acked Read 10 Done Done Sent false
4396 2192 Acked Read 8 Done Done Sent false
4397 2193 Acked Read 2 Done Done Sent false
4398 2194 Acked Write 10 Done Done Sent false
4399 2195 Acked Read 8 Done Done Sent false
4400 2196 Acked Read 7 Done Done Sent false
4401 2197 Acked Write 4 Done Done Sent false
4402 2198 Acked Read 4 Done Done Sent false
4403 2199 Acked Read 6 Done Done Sent false
4404 2200 Acked Flush 0 Done Done Sent false
4405 2201 Acked Write 7 Done Done Sent false
4406 2202 Acked Read 8 Done Done Sent false
4407 2203 Acked Read 1 Done Done Sent false
4408 2204 Acked Read 2 Done Done Sent false
4409 2205 Acked Read 2 Done Done Sent false
4410 2206 Acked Write 1 Done Done Sent false
4411 STATES DS:0 DS:1 DS:2 TOTAL
4412 Sent 0 0 301 301
4413 Done 301 301 0 602
4414 Skipped 0 0 0 0
4415 Error 0 0 0 0
4416 Last Flush: 2200 2200 1905
4417 Downstairs last five completed: 1905 1904 1903 1902 1901
4418 Upstairs last five completed: 2206 2205 2204 2203 2202
4419 CLIENT: Up:0 ds:301 act:2
4420 [8830] got start action
4421 [8830] I am going to restart
4422 Starting downstairs at port 8830
4423 Make output file at "/tmp/dsc/downstairs-8830.txt"
4424 [8830][2] reports Starting
4425 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1209)
4426 [8830][2] reports Running
4427 ----------------------------------------------------------------
4428 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4429 States: Active Active Offline
4430 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4431 1906 Acked Read 7 Done Done Sent false
4432 1907 Acked Read 1 Done Done Sent false
4433 1908 Acked Write 1 Done Done Sent false
4434 1909 Acked Read 4 Done Done Sent false
4435 1910 Acked Flush 0 Done Done Sent false
4436 1911 Acked Write 8 Done Done Sent false
4437 1912 Acked Flush 0 Done Done Sent false
4438 1913 Acked Write 9 Done Done Sent false
4439 1914 Acked Flush 0 Done Done Sent false
4440 1915 Acked Write 2 Done Done Sent false
4441 1916 Acked Read 6 Done Done Sent false
4442 1917 Acked Write 7 Done Done Sent false
4443 1918 Acked Read 1 Done Done Sent false
4444 1919 Acked Write 7 Done Done Sent false
4445 1920 Acked Write 1 Done Done Sent false
4446 1921 Acked Flush 0 Done Done Sent false
4447 1922 Acked Write 5 Done Done Sent false
4448 1923 Acked Read 2 Done Done Sent false
4449 1924 Acked Read 3 Done Done Sent false
4450 1925 Acked Write 1 Done Done Sent false
4451 1926 Acked Read 6 Done Done Sent false
4452 1927 Acked Write 9 Done Done Sent false
4453 1928 Acked Read 7 Done Done Sent false
4454 1929 Acked Write 4 Done Done Sent false
4455 1930 Acked Write 2 Done Done Sent false
4456 1931 Acked Read 8 Done Done Sent false
4457 1932 Acked Read 5 Done Done Sent false
4458 1933 Acked Flush 0 Done Done Sent false
4459 1934 Acked Read 6 Done Done Sent false
4460 1935 Acked Flush 0 Done Done Sent false
4461 1936 Acked Write 4 Done Done Sent false
4462 1937 Acked Read 7 Done Done Sent false
4463 1938 Acked Write 9 Done Done Sent false
4464 1939 Acked Read 8 Done Done Sent false
4465 1940 Acked Read 9 Done Done Sent false
4466 1941 Acked Read 3 Done Done Sent false
4467 1942 Acked Write 6 Done Done Sent false
4468 1943 Acked Read 7 Done Done Sent false
4469 1944 Acked Write 6 Done Done Sent false
4470 1945 Acked Read 4 Done Done Sent false
4471 1946 Acked Read 2 Done Done Sent false
4472 1947 Acked Write 4 Done Done Sent false
4473 1948 Acked Read 1 Done Done Sent false
4474 1949 Acked Read 1 Done Done Sent false
4475 1950 Acked Write 10 Done Done Sent false
4476 1951 Acked Read 8 Done Done Sent false
4477 1952 Acked Read 5 Done Done Sent false
4478 1953 Acked Read 2 Done Done Sent false
4479 1954 Acked Write 3 Done Done Sent false
4480 1955 Acked Write 9 Done Done Sent false
4481 1956 Acked Write 2 Done Done Sent false
4482 1957 Acked Write 4 Done Done Sent false
4483 1958 Acked Write 10 Done Done Sent false
4484 1959 Acked Write 3 Done Done Sent false
4485 1960 Acked Read 2 Done Done Sent false
4486 1961 Acked Write 10 Done Done Sent false
4487 1962 Acked Flush 0 Done Done Sent false
4488 1963 Acked Write 1 Done Done Sent false
4489 1964 Acked Read 3 Done Done Sent false
4490 1965 Acked Read 7 Done Done Sent false
4491 1966 Acked Write 6 Done Done Sent false
4492 1967 Acked Read 3 Done Done Sent false
4493 1968 Acked Write 3 Done Done Sent false
4494 1969 Acked Write 5 Done Done Sent false
4495 1970 Acked Flush 0 Done Done Sent false
4496 1971 Acked Read 7 Done Done Sent false
4497 1972 Acked Read 6 Done Done Sent false
4498 1973 Acked Write 5 Done Done Sent false
4499 1974 Acked Write 3 Done Done Sent false
4500 1975 Acked Read 8 Done Done Sent false
4501 1976 Acked Read 2 Done Done Sent false
4502 1977 Acked Read 2 Done Done Sent false
4503 1978 Acked Write 7 Done Done Sent false
4504 1979 Acked Read 3 Done Done Sent false
4505 1980 Acked Write 6 Done Done Sent false
4506 1981 Acked Write 8 Done Done Sent false
4507 1982 Acked Read 9 Done Done Sent false
4508 1983 Acked Read 9 Done Done Sent false
4509 1984 Acked Read 4 Done Done Sent false
4510 1985 Acked Write 1 Done Done Sent false
4511 1986 Acked Write 6 Done Done Sent false
4512 1987 Acked Read 3 Done Done Sent false
4513 1988 Acked Read 10 Done Done Sent false
4514 1989 Acked Flush 0 Done Done Sent false
4515 1990 Acked Write 9 Done Done Sent false
4516 1991 Acked Read 10 Done Done Sent false
4517 1992 Acked Write 2 Done Done Sent false
4518 1993 Acked Read 2 Done Done Sent false
4519 1994 Acked Read 6 Done Done Sent false
4520 1995 Acked Write 9 Done Done Sent false
4521 1996 Acked Write 1 Done Done Sent false
4522 1997 Acked Read 10 Done Done Sent false
4523 1998 Acked Flush 0 Done Done Sent false
4524 1999 Acked Write 1 Done Done Sent false
4525 2000 Acked Read 6 Done Done Sent false
4526 2001 Acked Write 9 Done Done Sent false
4527 2002 Acked Flush 0 Done Done Sent false
4528 2003 Acked Write 2 Done Done Sent false
4529 2004 Acked Read 10 Done Done Sent false
4530 2005 Acked Read 9 Done Done Sent false
4531 2006 Acked Read 6 Done Done Sent false
4532 2007 Acked Read 8 Done Done Sent false
4533 2008 Acked Write 8 Done Done Sent false
4534 2009 Acked Write 4 Done Done Sent false
4535 2010 Acked Flush 0 Done Done Sent false
4536 2011 Acked Write 5 Done Done Sent false
4537 2012 Acked Read 6 Done Done Sent false
4538 2013 Acked Flush 0 Done Done Sent false
4539 2014 Acked Flush 0 Done Done Sent false
4540 2015 Acked Write 1 Done Done Sent false
4541 2016 Acked Read 9 Done Done Sent false
4542 2017 Acked Read 4 Done Done Sent false
4543 2018 Acked Read 4 Done Done Sent false
4544 2019 Acked Read 2 Done Done Sent false
4545 2020 Acked Write 10 Done Done Sent false
4546 2021 Acked Read 4 Done Done Sent false
4547 2022 Acked Flush 0 Done Done Sent false
4548 2023 Acked Write 8 Done Done Sent false
4549 2024 Acked Write 10 Done Done Sent false
4550 2025 Acked Write 1 Done Done Sent false
4551 2026 Acked Flush 0 Done Done Sent false
4552 2027 Acked Flush 0 Done Done Sent false
4553 2028 Acked Read 5 Done Done Sent false
4554 2029 Acked Read 1 Done Done Sent false
4555 2030 Acked Read 6 Done Done Sent false
4556 2031 Acked Write 2 Done Done Sent false
4557 2032 Acked Write 3 Done Done Sent false
4558 2033 Acked Write 1 Done Done Sent false
4559 2034 Acked Read 10 Done Done Sent false
4560 2035 Acked Read 3 Done Done Sent false
4561 2036 Acked Read 8 Done Done Sent false
4562 2037 Acked Write 4 Done Done Sent false
4563 2038 Acked Read 7 Done Done Sent false
4564 2039 Acked Read 6 Done Done Sent false
4565 2040 Acked Write 7 Done Done Sent false
4566 2041 Acked Write 4 Done Done Sent false
4567 2042 Acked Read 7 Done Done Sent false
4568 2043 Acked Read 4 Done Done Sent false
4569 2044 Acked Read 3 Done Done Sent false
4570 2045 Acked Flush 0 Done Done Sent false
4571 2046 Acked Read 8 Done Done Sent false
4572 2047 Acked Read 2 Done Done Sent false
4573 2048 Acked Read 7 Done Done Sent false
4574 2049 Acked Read 2 Done Done Sent false
4575 2050 Acked Write 10 Done Done Sent false
4576 2051 Acked Read 4 Done Done Sent false
4577 2052 Acked Flush 0 Done Done Sent false
4578 2053 Acked Write 7 Done Done Sent false
4579 2054 Acked Read 1 Done Done Sent false
4580 2055 Acked Read 5 Done Done Sent false
4581 2056 Acked Read 6 Done Done Sent false
4582 2057 Acked Read 2 Done Done Sent false
4583 2058 Acked Read 9 Done Done Sent false
4584 2059 Acked Write 10 Done Done Sent false
4585 2060 Acked Read 6 Done Done Sent false
4586 2061 Acked Read 5 Done Done Sent false
4587 2062 Acked Write 6 Done Done Sent false
4588 2063 Acked Write 6 Done Done Sent false
4589 2064 Acked Read 7 Done Done Sent false
4590 2065 Acked Read 9 Done Done Sent false
4591 2066 Acked Write 8 Done Done Sent false
4592 2067 Acked Write 6 Done Done Sent false
4593 2068 Acked Write 9 Done Done Sent false
4594 2069 Acked Read 7 Done Done Sent false
4595 2070 Acked Read 9 Done Done Sent false
4596 2071 Acked Write 8 Done Done Sent false
4597 2072 Acked Read 6 Done Done Sent false
4598 2073 Acked Read 2 Done Done Sent false
4599 2074 Acked Read 3 Done Done Sent false
4600 2075 Acked Write 8 Done Done Sent false
4601 2076 Acked Read 8 Done Done Sent false
4602 2077 Acked Write 1 Done Done Sent false
4603 2078 Acked Write 8 Done Done Sent false
4604 2079 Acked Read 2 Done Done Sent false
4605 2080 Acked Read 7 Done Done Sent false
4606 2081 Acked Write 9 Done Done Sent false
4607 2082 Acked Write 5 Done Done Sent false
4608 2083 Acked Flush 0 Done Done Sent false
4609 2084 Acked Read 6 Done Done Sent false
4610 2085 Acked Write 8 Done Done Sent false
4611 2086 Acked Write 3 Done Done Sent false
4612 2087 Acked Read 10 Done Done Sent false
4613 2088 Acked Read 4 Done Done Sent false
4614 2089 Acked Flush 0 Done Done Sent false
4615 2090 Acked Read 3 Done Done Sent false
4616 2091 Acked Write 4 Done Done Sent false
4617 2092 Acked Write 9 Done Done Sent false
4618 2093 Acked Write 4 Done Done Sent false
4619 2094 Acked Read 3 Done Done Sent false
4620 2095 Acked Read 8 Done Done Sent false
4621 2096 Acked Read 3 Done Done Sent false
4622 2097 Acked Read 2 Done Done Sent false
4623 2098 Acked Read 5 Done Done Sent false
4624 2099 Acked Write 7 Done Done Sent false
4625 2100 Acked Write 3 Done Done Sent false
4626 2101 Acked Write 1 Done Done Sent false
4627 2102 Acked Write 3 Done Done Sent false
4628 2103 Acked Read 7 Done Done Sent false
4629 2104 Acked Read 1 Done Done Sent false
4630 2105 Acked Read 9 Done Done Sent false
4631 2106 Acked Read 3 Done Done Sent false
4632 2107 Acked Write 8 Done Done Sent false
4633 2108 Acked Read 8 Done Done Sent false
4634 2109 Acked Read 6 Done Done Sent false
4635 2110 Acked Read 1 Done Done Sent false
4636 2111 Acked Write 5 Done Done Sent false
4637 2112 Acked Write 1 Done Done Sent false
4638 2113 Acked Write 2 Done Done Sent false
4639 2114 Acked Read 3 Done Done Sent false
4640 2115 Acked Read 9 Done Done Sent false
4641 2116 Acked Read 10 Done Done Sent false
4642 2117 Acked Write 2 Done Done Sent false
4643 2118 Acked Write 9 Done Done Sent false
4644 2119 Acked Read 6 Done Done Sent false
4645 2120 Acked Write 4 Done Done Sent false
4646 2121 Acked Read 10 Done Done Sent false
4647 2122 Acked Write 2 Done Done Sent false
4648 2123 Acked Read 5 Done Done Sent false
4649 2124 Acked Flush 0 Done Done Sent false
4650 2125 Acked Write 7 Done Done Sent false
4651 2126 Acked Flush 0 Done Done Sent false
4652 2127 Acked Write 8 Done Done Sent false
4653 2128 Acked Read 5 Done Done Sent false
4654 2129 Acked Read 5 Done Done Sent false
4655 2130 Acked Read 4 Done Done Sent false
4656 2131 Acked Write 8 Done Done Sent false
4657 2132 Acked Write 9 Done Done Sent false
4658 2133 Acked Flush 0 Done Done Sent false
4659 2134 Acked Flush 0 Done Done Sent false
4660 2135 Acked Read 8 Done Done Sent false
4661 2136 Acked Read 5 Done Done Sent false
4662 2137 Acked Read 3 Done Done Sent false
4663 2138 Acked Flush 0 Done Done Sent false
4664 2139 Acked Write 7 Done Done Sent false
4665 2140 Acked Write 1 Done Done Sent false
4666 2141 Acked Write 3 Done Done Sent false
4667 2142 Acked Read 1 Done Done Sent false
4668 2143 Acked Write 3 Done Done Sent false
4669 2144 Acked Read 1 Done Done Sent false
4670 2145 Acked Read 5 Done Done Sent false
4671 2146 Acked Read 7 Done Done Sent false
4672 2147 Acked Read 7 Done Done Sent false
4673 2148 Acked Read 6 Done Done Sent false
4674 2149 Acked Read 1 Done Done Sent false
4675 2150 Acked Write 10 Done Done Sent false
4676 2151 Acked Write 3 Done Done Sent false
4677 2152 Acked Write 5 Done Done Sent false
4678 2153 Acked Read 6 Done Done Sent false
4679 2154 Acked Read 4 Done Done Sent false
4680 2155 Acked Read 10 Done Done Sent false
4681 2156 Acked Read 3 Done Done Sent false
4682 2157 Acked Flush 0 Done Done Sent false
4683 2158 Acked Read 8 Done Done Sent false
4684 2159 Acked Read 2 Done Done Sent false
4685 2160 Acked Write 1 Done Done Sent false
4686 2161 Acked Read 1 Done Done Sent false
4687 2162 Acked Read 6 Done Done Sent false
4688 2163 Acked Read 8 Done Done Sent false
4689 2164 Acked Write 8 Done Done Sent false
4690 2165 Acked Write 8 Done Done Sent false
4691 2166 Acked Flush 0 Done Done Sent false
4692 2167 Acked Read 6 Done Done Sent false
4693 2168 Acked Write 5 Done Done Sent false
4694 2169 Acked Read 8 Done Done Sent false
4695 2170 Acked Flush 0 Done Done Sent false
4696 2171 Acked Read 5 Done Done Sent false
4697 2172 Acked Read 10 Done Done Sent false
4698 2173 Acked Write 5 Done Done Sent false
4699 2174 Acked Write 10 Done Done Sent false
4700 2175 Acked Read 8 Done Done Sent false
4701 2176 Acked Read 3 Done Done Sent false
4702 2177 Acked Write 3 Done Done Sent false
4703 2178 Acked Read 5 Done Done Sent false
4704 2179 Acked Write 9 Done Done Sent false
4705 2180 Acked Write 7 Done Done Sent false
4706 2181 Acked Read 6 Done Done Sent false
4707 2182 Acked Read 4 Done Done Sent false
4708 2183 Acked Write 7 Done Done Sent false
4709 2184 Acked Read 5 Done Done Sent false
4710 2185 Acked Write 1 Done Done Sent false
4711 2186 Acked Write 1 Done Done Sent false
4712 2187 Acked Read 1 Done Done Sent false
4713 2188 Acked Read 1 Done Done Sent false
4714 2189 Acked Flush 0 Done Done Sent false
4715 2190 Acked Write 1 Done Done Sent false
4716 2191 Acked Read 10 Done Done Sent false
4717 2192 Acked Read 8 Done Done Sent false
4718 2193 Acked Read 2 Done Done Sent false
4719 2194 Acked Write 10 Done Done Sent false
4720 2195 Acked Read 8 Done Done Sent false
4721 2196 Acked Read 7 Done Done Sent false
4722 2197 Acked Write 4 Done Done Sent false
4723 2198 Acked Read 4 Done Done Sent false
4724 2199 Acked Read 6 Done Done Sent false
4725 2200 Acked Flush 0 Done Done Sent false
4726 2201 Acked Write 7 Done Done Sent false
4727 2202 Acked Read 8 Done Done Sent false
4728 2203 Acked Read 1 Done Done Sent false
4729 2204 Acked Read 2 Done Done Sent false
4730 2205 Acked Read 2 Done Done Sent false
4731 2206 Acked Write 1 Done Done Sent false
4732 2207 Acked Flush 0 Done Done Sent false
4733 STATES DS:0 DS:1 DS:2 TOTAL
4734 Sent 0 0 302 302
4735 Done 302 302 0 604
4736 Skipped 0 0 0 0
4737 Error 0 0 0 0
4738 Last Flush: 2207 2207 1905
4739 Downstairs last five completed: 1905 1904 1903 1902 1901
4740 Upstairs last five completed: 2207 2206 2205 2204 2203
4741 CLIENT: Up:0 ds:302 act:2
47422025-01-03T08:31:52.287ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47432025-01-03T08:31:52.287ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42740) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47442025-01-03T08:31:52.288ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47452025-01-03T08:31:52.289ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47462025-01-03T08:31:52.289ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47472025-01-03T08:31:52.289ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47482025-01-03T08:31:52.289ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47492025-01-03T08:31:52.289ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47502025-01-03T08:31:52.289ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47512025-01-03T08:31:52.289ZINFOcrucible: [2] client re-new 302 jobs since flush 1905 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47522025-01-03T08:31:52.290ZINFOcrucible: [2] Replayed 302 jobs since flush: 1905 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
4753 ----------------------------------------------------------------
4754 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4755 Downstairs last five completed: 2207 2206 2205 2204 2203
4756 Upstairs last five completed: 2207 2206 2205 2204 2203
4757 CLIENT: Up:0 ds:0 act:3
4758 Replay: All jobs finished
4759 Main task has work to do, go find it
4760 got dsc Stop(0)
4761 stop 0
4762 Jan 03 08:31:54.534 INFO request completed, latency_us: 345, response_code: 204, uri: /stop/cid/0, method: POST, req_id: c5328040-f78c-4395-b0dc-4075cd166b6c, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
4763 [8810] Got stop action so:false kr:false
47642025-01-03T08:31:54.535ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47652025-01-03T08:31:54.535ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
47662025-01-03T08:31:54.535ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
4767 Jan 03 08:31:54.535 INFO request completed, latency_us: 138, response_code: 200, uri: /state/cid/0, method: GET, req_id: 046848e3-bcfe-468c-90f3-793e35363a7b, remote_addr: 127{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:31:54.535502053Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"0","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
4768 .0.0.1:49669, local_addr: 127.0.{"msg":"[0] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:31:54.535522183Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"0","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
4769 0.1:9998
47702025-01-03T08:31:54.535ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
4771 [8810] Exited with: ExitStatus(unix_wait_status(9))
4772 [8810][0] reports Exit
4773 Jan 03 08:31:58.537 INFO request completed, latency_us: 229, response_code: 200, uri: /state/cid/0, method: GET, req_id: 0ac48785-993a-40dc-8915-237edd67ab50, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
4774 001/300 Read block 3463 len 40960
4775 002/300 Read block 4657 len 8192
4776 003/300 Read block 2594 len 8192
4777 004/300 Read block 38 len 28672
4778 005/300 Flush
4779 006/300 Read block 3299 len 36864
4780 007/300 Write block 3160 len 24576 data: 2 2 2 2 2 2
4781 008/300 Write block 759 len 36864 data: 3 3 2 2 2 2 2 3 3
4782 009/300 Read block 1182 len 28672
4783 010/300 Write block 3860 len 16384 data: 2 2 2 2
4784 011/300 Read block 3822 len 28672
4785 012/300 Read block 4360 len 32768
4786 013/300 Read block 2976 len 20480
4787 014/300 Read block 2699 len 12288
4788 015/300 Write block 3540 len 12288 data: 2 2 3
4789 016/300 Write block 3788 len 4096 data: 3
4790 017/300 Write block 1686 len 24576 data: 2 2 2 2 2 2
4791 018/300 Read block 961 len 4096
4792 019/300 Write block 4590 len 12288 data: 2 2 2
4793 020/300 Write block 1327 len 32768 data: 3 3 3 2 2 2 2 2
4794 021/300 Read block 2828 len 40960
4795 022/300 Read block 659 len 36864
4796 023/300 Read block 154 len 16384
4797 024/300 Write block 413 len 36864 data: 2 2 2 2 2 2 3 3 3
4798 025/300 Read block 1222 len 20480
4799 026/300 Write block 817 len 32768 data: 3 2 2 3 3 3 2 3
4800 027/300 Flush
4801 028/300 Read block 1706 len 40960
4802 029/300 Write block 4143 len 32768 data: 4 4 3 2 2 2 2 2
4803 030/300 Read block 428 len 12288
4804 031/300 Write block 1332 len 16384 data: 3 3 3 2
4805 032/300 Write block 2084 len 28672 data: 2 2 2 2 2 2 2
4806 033/300 Write block 535 len 36864 data: 3 3 2 3 3 3 3 3 3
4807 034/300 Write block 4532 len 16384 data: 2 2 2 2
4808 035/300 Read block 2754 len 20480
4809 036/300 Read block 3770 len 32768
4810 037/300 Read block 371 len 24576
4811 038/300 Write block 1881 len 40960 data: 2 3 3 3 3 3 3 3 3 3
4812 039/300 Write block 2454 len 12288 data: 4 4 4
4813 040/300 Read block 1817 len 24576
4814 041/300 Write block 3876 len 8192 data: 2 2
4815 042/300 Write block 3651 len 32768 data: 3 3 4 3 3 3 3 3
4816 043/300 Write block 3353 len 36864 data: 2 2 2 3 3 3 3 3 2
4817 044/300 Read block 3748 len 28672
4818 045/300 Write block 1164 len 24576 data: 2 2 3 3 3 3
4819 046/300 Read block 1666 len 20480
4820 047/300 Read block 577 len 20480
4821 048/300 Flush
4822 049/300 Write block 1896 len 16384 data: 3 2 2 2
4823 050/300 Read block 3223 len 36864
4824 051/300 Read block 1347 len 36864
4825 052/300 Read block 4795 len 12288
4826 053/300 Flush
4827 054/300 Write block 2579 len 4096 data: 2
4828 055/300 Read block 4561 len 8192
4829 056/300 Read block 3765 len 12288
4830 057/300 Read block 3516 len 8192
4831 058/300 Flush
4832 059/300 Write block 328 len 20480 data: 3 3 3 3 3
4833 060/300 Flush
4834 061/300 Write block 2696 len 12288 data: 3 3 3
4835 062/300 Write block 1584 len 32768 data: 2 2 2 2 2 2 2 2
4836 063/300 Read block 1963 len 16384
4837 064/300 Read block 1657 len 24576
4838 065/300 Write block 3476 len 20480 data: 2 2 2 2 2
4839 066/300 Write block 4199 len 32768 data: 4 4 4 4 4 5 5 4
4840 067/300 Read block 3667 len 8192
4841 068/300 Write block 1323 len 16384 data: 2 2 2 3
4842 069/300 Write block 1339 len 20480 data: 2 2 3 2 2
4843 070/300 Write block 3422 len 32768 data: 4 4 4 4 4 4 4 4
4844 071/300 Write block 4057 len 28672 data: 2 2 2 2 2 2 2
4845 072/300 Read block 1605 len 20480
4846 073/300 Read block 4529 len 28672
4847 074/300 Read block 4441 len 36864
4848 075/300 Read block 3856 len 12288
4849 076/300 Write block 236 len 24576 data: 2 2 2 2 2 2
4850 077/300 Read block 1790 len 40960
4851 078/300 Read block 1821 len 16384
4852 079/300 Write block 458 len 40960 data: 2 2 2 2 2 3 3 3 3 3
4853 080/300 Read block 2506 len 4096
4854 081/300 Write block 4574 len 36864 data: 2 2 2 2 3 3 3 3 3
4855 082/300 Read block 171 len 4096
4856 083/300 Read block 1367 len 40960
4857 084/300 Read block 3784 len 20480
4858 085/300 Read block 2819 len 32768
4859 086/300 Write block 2816 len 32768 data: 2 2 2 2 2 2 2 2
4860 087/300 Read block 3142 len 12288
4861 088/300 Write block 4720 len 20480 data: 2 2 2 3 4
4862 089/300 Read block 3787 len 20480
4863 090/300 Read block 1914 len 36864
4864 091/300 Read block 2082 len 28672
4865 092/300 Read block 4470 len 24576
4866 093/300 Read block 1674 len 4096
4867 094/300 Read block 1601 len 4096
4868 095/300 Write block 662 len 32768 data: 3 3 4 4 4 3 3 3
4869 096/300 Read block 2589 len 28672
4870 097/300 Write block 4410 len 16384 data: 2 2 2 2
4871 098/300 Read block 1481 len 28672
4872 099/300 Write block 1117 len 24576 data: 3 3 3 3 3 3
4873 100/300 Flush
4874 101/300 Read block 3260 len 20480
4875 102/300 Read block 752 len 20480
4876 103/300 Write block 4769 len 12288 data: 4 3 3
4877 104/300 Write block 920 len 16384 data: 3 3 3 3
4878 105/300 Read block 222 len 36864
4879 106/300 Write block 1922 len 20480 data: 3 3 3 3 3
4880 107/300 Write block 3456 len 20480 data: 2 2 2 2 2
4881 108/300 Read block 3774 len 8192
4882 109/300 Flush
4883 110/300 Read block 2659 len 20480
4884 111/300 Flush
4885 112/300 Write block 3711 len 8192 data: 2 2
4886 113/300 Read block 2749 len 20480
4887 114/300 Flush
4888 115/300 Write block 2483 len 36864 data: 2 2 3 3 3 3 2 2 2
4889 116/300 Write block 4783 len 12288 data: 3 2 2
4890 117/300 Read block 2123 len 28672
4891 118/300 Read block 3274 len 8192
4892 119/300 Read block 1342 len 40960
4893 120/300 Flush
4894 121/300 Write block 4857 len 24576 data: 2 2 2 2 2 2
4895 122/300 Read block 237 len 16384
4896 123/300 Read block 2118 len 24576
4897 124/300 Read block 3417 len 8192
4898 125/300 Read block 1058 len 20480
4899 126/300 Read block 1531 len 8192
4900 127/300 Read block 2878 len 28672
4901 128/300 Write block 1943 len 36864 data: 2 3 3 4 4 2 2 2 2
4902 129/300 Write block 3063 len 16384 data: 3 3 3 2
4903 130/300 Read block 4077 len 8192
4904 131/300 Read block 2306 len 4096
4905 132/300 Read block 1507 len 12288
4906 133/300 Write block 3255 len 32768 data: 2 2 2 3 3 3 3 3
4907 134/300 Write block 4462 len 8192 data: 2 2
4908 135/300 Read block 186 len 32768
4909 136/300 Read block 280 len 20480
4910 137/300 Read block 1972 len 36864
4911 138/300 Read block 729 len 40960
4912 139/300 Read block 3810 len 36864
4913 140/300 Write block 729 len 32768 data: 3 2 2 2 2 2 2 2
4914 141/300 Read block 2020 len 40960
4915 142/300 Flush
4916 143/300 Write block 3284 len 12288 data: 2 2 2
4917 144/300 Read block 4617 len 16384
4918 145/300 Read block 4392 len 32768
4919 146/300 Flush
4920 147/300 Read block 4599 len 4096
4921 148/300 Read block 2777 len 8192
4922 149/300 Read block 2776 len 32768
4923 150/300 Write block 2660 len 20480 data: 3 2 2 2 2
4924 151/300 Read block 1502 len 24576
4925 152/300 Read block 1626 len 40960
4926 153/300 Read block 3240 len 8192
4927 154/300 Write block 3432 len 28672 data: 3 3 3 4 3 3 3
4928 155/300 Write block 2975 len 12288 data: 5 4 4
4929 156/300 Read block 4543 len 20480
4930 157/300 Write block 3533 len 12288 data: 2 2 2
4931 158/300 Read block 813 len 24576
4932 159/300 Read block 3203 len 16384
4933 160/300 Read block 4118 len 40960
4934 161/300 Read block 896 len 4096
4935 162/300 Read block 381 len 8192
4936 163/300 Read block 3653 len 24576
4937 164/300 Write block 1483 len 20480 data: 2 3 3 3 3
4938 165/300 Write block 2563 len 8192 data: 2 2
4939 166/300 Read block 3646 len 16384
4940 167/300 Read block 4839 len 12288
4941 168/300 Read block 752 len 20480
4942 169/300 Write block 1181 len 40960 data: 4 3 2 2 2 2 2 2 2 2
4943 170/300 Read block 4425 len 16384
4944 171/300 Read block 2670 len 36864
4945 172/300 Write block 3492 len 32768 data: 2 3 2 2 2 2 2 2
4946 173/300 Write block 2385 len 4096 data: 4
4947 174/300 Read block 4522 len 16384
4948 175/300 Read block 1095 len 12288
4949 176/300 Write block 669 len 4096 data: 4
4950 177/300 Read block 1293 len 24576
4951 178/300 Write block 603 len 28672 data: 2 2 2 2 2 2 2
4952 179/300 Write block 3081 len 20480 data: 2 2 3 2 3
4953 180/300 Read block 3386 len 28672
4954 181/300 Read block 3977 len 12288
4955 182/300 Read block 120 len 4096
4956 183/300 Write block 4337 len 16384 data: 3 3 3 3
4957 184/300 Read block 1119 len 32768
4958 185/300 Read block 3500 len 16384
4959 186/300 Read block 881 len 12288
4960 187/300 Write block 2479 len 16384 data: 2 2 2 2
4961 188/300 Read block 4109 len 40960
4962 189/300 Write block 2939 len 36864 data: 3 3 3 3 2 2 2 3 3
4963 190/300 Write block 4782 len 32768 data: 3 4 3 3 2 2 2 3
4964 191/300 Read block 3544 len 40960
4965 192/300 Read block 3704 len 24576
4966 193/300 Read block 4329 len 40960
4967 194/300 Write block 4422 len 28672 data: 3 3 3 3 2 2 3
4968 195/300 Read block 1505 len 4096
4969 196/300 Write block 4464 len 28672 data: 2 2 3 2 2 2 2
4970 197/300 Read block 4325 len 36864
4971 198/300 Write block 4255 len 20480 data: 4 4 3 3 3
4972 199/300 Read block 2048 len 32768
4973 200/300 Read block 1123 len 20480
4974 201/300 Read block 1905 len 24576
4975 202/300 Read block 4240 len 20480
4976 203/300 Write block 843 len 36864 data: 3 3 3 3 3 3 3 3 3
4977 204/300 Flush
4978 205/300 Write block 4812 len 8192 data: 2 2
4979 206/300 Write block 395 len 20480 data: 2 2 2 2 2
4980 207/300 Write block 611 len 36864 data: 2 2 2 2 2 2 2 2 2
4981 208/300 Read block 2021 len 32768
4982 209/300 Write block 2974 len 12288 data: 5 6 5
4983 210/300 Read block 3042 len 20480
4984 211/300 Write block 1523 len 20480 data: 4 4 4 4 3
4985 212/300 Read block 4817 len 36864
4986 213/300 Flush
4987 214/300 Read block 1223 len 32768
4988 215/300 Write block 2710 len 24576 data: 4 4 4 4 4 3
4989 216/300 Read block 526 len 40960
4990 217/300 Write block 4385 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4991 218/300 Read block 3827 len 4096
4992 219/300 Read block 412 len 40960
4993 220/300 Write block 2289 len 24576 data: 2 2 2 2 2 2
4994 221/300 Read block 1048 len 16384
4995 222/300 Read block 3195 len 40960
4996 223/300 Read block 3820 len 36864
4997 224/300 Read block 2927 len 32768
4998 225/300 Read block 1878 len 20480
4999 226/300 Read block 4202 len 32768
5000 227/300 Write block 1106 len 8192 data: 2 2
5001 228/300 Read block 191 len 12288
5002 229/300 Read block 3028 len 32768
5003 230/300 Read block 4881 len 32768
5004 231/300 Read block 666 len 28672
5005 232/300 Read block 2061 len 28672
5006 233/300 Write block 2870 len 16384 data: 2 2 2 2
5007 234/300 Read block 146 len 24576
5008 235/300 Write block 527 len 20480 data: 2 2 3 3 3
5009 236/300 Write block 1121 len 12288 data: 4 4 3
5010 237/300 Write block 2328 len 32768 data: 3 3 3 3 3 2 2 2
5011 238/300 Write block 1593 len 28672 data: 3 3 2 2 2 2 3
5012 239/300 Read block 3757 len 20480
5013 240/300 Read block 2331 len 16384
5014 241/300 Write block 836 len 36864 data: 2 2 2 2 2 2 2 4 4
5015 242/300 Flush
5016 243/300 Read block 2227 len 24576
5017 244/300 Write block 4690 len 36864 data: 3 3 3 3 3 3 2 2 2
5018 245/300 Read block 4300 len 28672
5019 246/300 Write block 4628 len 32768 data: 2 2 2 2 2 3 3 3
5020 247/300 Read block 1852 len 36864
5021 248/300 Write block 305 len 4096 data: 3
5022 249/300 Read block 2629 len 24576
5023 250/300 Read block 2671 len 20480
5024 251/300 Read block 238 len 16384
5025 252/300 Flush
5026 253/300 Read block 1280 len 12288
5027 254/300 Flush
5028 255/300 Write block 1213 len 20480 data: 3 3 3 3 3
5029 256/300 Write block 2301 len 40960 data: 3 3 2 2 2 2 2 2 2 2
5030 257/300 Write block 721 len 8192 data: 2 2
5031 258/300 Read block 3648 len 8192
5032 259/300 Write block 616 len 8192 data: 3 3
5033 260/300 Write block 3605 len 16384 data: 4 2 2 2
5034 261/300 Write block 1304 len 8192 data: 2 2
5035 262/300 Read block 2358 len 12288
5036 263/300 Write block 723 len 36864 data: 2 2 2 2 2 2 4 3 3
5037 264/300 Write block 4869 len 16384 data: 2 2 2 2
5038 265/300 Read block 4955 len 4096
5039 266/300 Write block 1914 len 12288 data: 2 2 2
5040 267/300 Read block 2792 len 16384
5041 268/300 Flush
5042 269/300 Flush
5043 270/300 Read block 3913 len 28672
5044 271/300 Read block 1719 len 24576
5045 272/300 Flush
5046 273/300 Write block 4714 len 16384 data: 2 2 2 2
5047 274/300 Read block 818 len 4096
5048 275/300 Read block 1513 len 40960
5049 276/300 Write block 364 len 24576 data: 2 3 3 3 3 3
5050 277/300 Read block 2334 len 32768
5051 278/300 Read block 3664 len 8192
5052 279/300 Read block 386 len 24576
5053 280/300 Flush
5054 281/300 Read block 2032 len 8192
5055 282/300 Read block 2945 len 8192
5056 283/300 Read block 3454 len 8192
5057 284/300 Write block 4636 len 4096 data: 3
5058 285/300 Write block 310 len 16384 data: 3 2 2 2
5059 286/300 Write block 3848 len 32768 data: 2 2 3 3 4 5 4 4
5060 287/300 Read block 4177 len 32768
5061 288/300 Write block 4386 len 28672 data: 3 3 3 3 3 3 3
5062 289/300 Read block 270 len 4096
5063 290/300 Read block 2241 len 28672
5064 291/300 Read block 1853 len 8192
5065 292/300 Read block 2832 len 8192
5066 293/300 Write block 1224 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5067 294/300 Write block 922 len 28672 data: 4 4 2 3 3 2 2
5068 295/300 Read block 1639 len 36864
5069 296/300 Write block 1144 len 32768 data: 3 4 4 4 3 3 2 2
5070 297/300 Write block 3431 len 16384 data: 3 4 4 4
5071 298/300 Write block 63 len 28672 data: 2 3 3 3 3 3 3
5072 299/300 Write block 4888 len 8192 data: 3 3
5073 300/300 Write block 1680 len 12288 data: 2 2 2
5074 301/300 Flush
5075 Main task has work to do, go find it
5076 got dsc Start(0)
5077 start 0
5078 Jan 03 08:31:58.790 INFO request completed, latency_us: 221, response_code: 204, uri: /start/cid/0, method: POST, req_id: 92edfa4e-e324-4392-af74-768997a4019b, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
5079 Replay: started 0, returned:Ok(())
5080 ----------------------------------------------------------------
5081 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5082 States: Offline Active Active
5083 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5084 2208 Acked Read 10 Sent Done Done false
5085 2209 Acked Read 2 Sent Done Done false
5086 2210 Acked Read 2 Sent Done Done false
5087 2211 Acked Read 7 Sent Done Done false
5088 2212 Acked Flush 0 Sent Done Done false
5089 2213 Acked Read 9 Sent Done Done false
5090 2214 Acked Write 6 Sent Done Done false
5091 2215 Acked Write 9 Sent Done Done false
5092 2216 Acked Read 7 Sent Done Done false
5093 2217 Acked Write 4 Sent Done Done false
5094 2218 Acked Read 7 Sent Done Done false
5095 2219 Acked Read 8 Sent Done Done false
5096 2220 Acked Read 5 Sent Done Done false
5097 2221 Acked Read 3 Sent Done Done false
5098 2222 Acked Write 3 Sent Done Done false
5099 2223 Acked Write 1 Sent Done Done false
5100 2224 Acked Write 6 Sent Done Done false
5101 2225 Acked Read 1 Sent Done Done false
5102 2226 Acked Write 3 Sent Done Done false
5103 2227 Acked Write 8 Sent Done Done false
5104 2228 Acked Read 10 Sent Done Done false
5105 2229 Acked Read 9 Sent Done Done false
5106 2230 Acked Read 4 Sent Done Done false
5107 2231 Acked Write 9 Sent Done Done false
5108 2232 Acked Read 5 Sent Done Done false
5109 2233 Acked Write 8 Sent Done Done false
5110 2234 Acked Flush 0 Sent Done Done false
5111 2235 Acked Read 10 Sent Done Done false
5112 2236 Acked Write 8 Sent Done Done false
5113 2237 Acked Read 3 Sent Done Done false
5114 2238 Acked Write 4 Sent Done Done false
5115 2239 Acked Write 7 Sent Done Done false
5116 2240 Acked Write 9 Sent Done Done false
5117 2241 Acked Write 4 Sent Done Done false
5118 2242 Acked Read 5 Sent Done Done false
5119 2243 Acked Read 8 Sent Done Done false
5120 2244 Acked Read 6 Sent Done Done false
5121 2245 Acked Write 10 Sent Done Done false
5122 2246 Acked Write 3 Sent Done Done false
5123 2247 Acked Read 6 Sent Done Done false
5124 2248 Acked Write 2 Sent Done Done false
5125 2249 Acked Write 8 Sent Done Done false
5126 2250 Acked Write 9 Sent Done Done false
5127 2251 Acked Read 7 Sent Done Done false
5128 2252 Acked Write 6 Sent Done Done false
5129 2253 Acked Read 5 Sent Done Done false
5130 2254 Acked Read 5 Sent Done Done false
5131 2255 Acked Flush 0 Sent Done Done false
5132 2256 Acked Write 4 Sent Done Done false
5133 2257 Acked Read 9 Sent Done Done false
5134 2258 Acked Read 9 Sent Done Done false
5135 2259 Acked Read 3 Sent Done Done false
5136 2260 Acked Flush 0 Sent Done Done false
5137 2261 Acked Write 1 Sent Done Done false
5138 2262 Acked Read 2 Sent Done Done false
5139 2263 Acked Read 3 Sent Done Done false
5140 2264 Acked Read 2 Sent Done Done false
5141 2265 Acked Flush 0 Sent Done Done false
5142 2266 Acked Write 5 Sent Done Done false
5143 2267 Acked Flush 0 Sent Done Done false
5144 2268 Acked Write 3 Sent Done Done false
5145 2269 Acked Write 8 Sent Done Done false
5146 2270 Acked Read 4 Sent Done Done false
5147 2271 Acked Read 6 Sent Done Done false
5148 2272 Acked Write 5 Sent Done Done false
5149 2273 Acked Write 8 Sent Done Done false
5150 2274 Acked Read 2 Sent Done Done false
5151 2275 Acked Write 4 Sent Done Done false
5152 2276 Acked Write 5 Sent Done Done false
5153 2277 Acked Write 8 Sent Done Done false
5154 2278 Acked Write 7 Sent Done Done false
5155 2279 Acked Read 5 Sent Done Done false
5156 2280 Acked Read 7 Sent Done Done false
5157 2281 Acked Read 9 Sent Done Done false
5158 2282 Acked Read 3 Sent Done Done false
5159 2283 Acked Write 6 Sent Done Done false
5160 2284 Acked Read 10 Sent Done Done false
5161 2285 Acked Read 4 Sent Done Done false
5162 2286 Acked Write 10 Sent Done Done false
5163 2287 Acked Read 1 Sent Done Done false
5164 2288 Acked Write 9 Sent Done Done false
5165 2289 Acked Read 1 Sent Done Done false
5166 2290 Acked Read 10 Sent Done Done false
5167 2291 Acked Read 5 Sent Done Done false
5168 2292 Acked Read 8 Sent Done Done false
5169 2293 Acked Write 8 Sent Done Done false
5170 2294 Acked Read 3 Sent Done Done false
5171 2295 Acked Write 5 Sent Done Done false
5172 2296 Acked Read 5 Sent Done Done false
5173 2297 Acked Read 9 Sent Done Done false
5174 2298 Acked Read 7 Sent Done Done false
5175 2299 Acked Read 6 Sent Done Done false
5176 2300 Acked Read 1 Sent Done Done false
5177 2301 Acked Read 1 Sent Done Done false
5178 2302 Acked Write 8 Sent Done Done false
5179 2303 Acked Read 7 Sent Done Done false
5180 2304 Acked Write 4 Sent Done Done false
5181 2305 Acked Read 7 Sent Done Done false
5182 2306 Acked Write 6 Sent Done Done false
5183 2307 Acked Flush 0 Sent Done Done false
5184 2308 Acked Read 5 Sent Done Done false
5185 2309 Acked Read 5 Sent Done Done false
5186 2310 Acked Write 3 Sent Done Done false
5187 2311 Acked Write 4 Sent Done Done false
5188 2312 Acked Read 9 Sent Done Done false
5189 2313 Acked Write 5 Sent Done Done false
5190 2314 Acked Write 5 Sent Done Done false
5191 2315 Acked Read 2 Sent Done Done false
5192 2316 Acked Flush 0 Sent Done Done false
5193 2317 Acked Read 5 Sent Done Done false
5194 2318 Acked Flush 0 Sent Done Done false
5195 2319 Acked Write 2 Sent Done Done false
5196 2320 Acked Read 5 Sent Done Done false
5197 2321 Acked Flush 0 Sent Done Done false
5198 2322 Acked Write 9 Sent Done Done false
5199 2323 Acked Write 3 Sent Done Done false
5200 2324 Acked Read 7 Sent Done Done false
5201 2325 Acked Read 2 Sent Done Done false
5202 2326 Acked Read 10 Sent Done Done false
5203 2327 Acked Flush 0 Sent Done Done false
5204 2328 Acked Write 6 Sent Done Done false
5205 2329 Acked Read 4 Sent Done Done false
5206 2330 Acked Read 6 Sent Done Done false
5207 2331 Acked Read 2 Sent Done Done false
5208 2332 Acked Read 5 Sent Done Done false
5209 2333 Acked Read 2 Sent Done Done false
5210 2334 Acked Read 7 Sent Done Done false
5211 2335 Acked Write 9 Sent Done Done false
5212 2336 Acked Write 4 Sent Done Done false
5213 2337 Acked Read 2 Sent Done Done false
5214 2338 Acked Read 1 Sent Done Done false
5215 2339 Acked Read 3 Sent Done Done false
5216 2340 Acked Write 8 Sent Done Done false
5217 2341 Acked Write 2 Sent Done Done false
5218 2342 Acked Read 8 Sent Done Done false
5219 2343 Acked Read 5 Sent Done Done false
5220 2344 Acked Read 9 Sent Done Done false
5221 2345 Acked Read 10 Sent Done Done false
5222 2346 Acked Read 9 Sent Done Done false
5223 2347 Acked Write 8 Sent Done Done false
5224 2348 Acked Read 10 Sent Done Done false
5225 2349 Acked Flush 0 Sent Done Done false
5226 2350 Acked Write 3 Sent Done Done false
5227 2351 Acked Read 4 Sent Done Done false
5228 2352 Acked Read 8 Sent Done Done false
5229 2353 Acked Flush 0 Sent Done Done false
5230 2354 Acked Read 1 Sent Done Done false
5231 2355 Acked Read 2 Sent Done Done false
5232 2356 Acked Read 8 Sent Done Done false
5233 2357 Acked Write 5 Sent Done Done false
5234 2358 Acked Read 6 Sent Done Done false
5235 2359 Acked Read 10 Sent Done Done false
5236 2360 Acked Read 2 Sent Done Done false
5237 2361 Acked Write 7 Sent Done Done false
5238 2362 Acked Write 3 Sent Done Done false
5239 2363 Acked Read 5 Sent Done Done false
5240 2364 Acked Write 3 Sent Done Done false
5241 2365 Acked Read 6 Sent Done Done false
5242 2366 Acked Read 4 Sent Done Done false
5243 2367 Acked Read 10 Sent Done Done false
5244 2368 Acked Read 1 Sent Done Done false
5245 2369 Acked Read 2 Sent Done Done false
5246 2370 Acked Read 6 Sent Done Done false
5247 2371 Acked Write 5 Sent Done Done false
5248 2372 Acked Write 2 Sent Done Done false
5249 2373 Acked Read 4 Sent Done Done false
5250 2374 Acked Read 3 Sent Done Done false
5251 2375 Acked Read 5 Sent Done Done false
5252 2376 Acked Write 10 Sent Done Done false
5253 2377 Acked Read 4 Sent Done Done false
5254 2378 Acked Read 9 Sent Done Done false
5255 2379 Acked Write 8 Sent Done Done false
5256 2380 Acked Write 1 Sent Done Done false
5257 2381 Acked Read 4 Sent Done Done false
5258 2382 Acked Read 3 Sent Done Done false
5259 2383 Acked Write 1 Sent Done Done false
5260 2384 Acked Read 6 Sent Done Done false
5261 2385 Acked Write 7 Sent Done Done false
5262 2386 Acked Write 5 Sent Done Done false
5263 2387 Acked Read 7 Sent Done Done false
5264 2388 Acked Read 3 Sent Done Done false
5265 2389 Acked Read 1 Sent Done Done false
5266 2390 Acked Write 4 Sent Done Done false
5267 2391 Acked Read 8 Sent Done Done false
5268 2392 Acked Read 4 Sent Done Done false
5269 2393 Acked Read 3 Sent Done Done false
5270 2394 Acked Write 4 Sent Done Done false
5271 2395 Acked Read 10 Sent Done Done false
5272 2396 Acked Write 9 Sent Done Done false
5273 2397 Acked Write 8 Sent Done Done false
5274 2398 Acked Read 10 Sent Done Done false
5275 2399 Acked Read 6 Sent Done Done false
5276 2400 Acked Read 10 Sent Done Done false
5277 2401 Acked Write 7 Sent Done Done false
5278 2402 Acked Read 1 Sent Done Done false
5279 2403 Acked Write 7 Sent Done Done false
5280 2404 Acked Read 9 Sent Done Done false
5281 2405 Acked Write 5 Sent Done Done false
5282 2406 Acked Read 8 Sent Done Done false
5283 2407 Acked Read 5 Sent Done Done false
5284 2408 Acked Read 6 Sent Done Done false
5285 2409 Acked Read 5 Sent Done Done false
5286 2410 Acked Write 9 Sent Done Done false
5287 2411 Acked Flush 0 Sent Done Done false
5288 2412 Acked Write 2 Sent Done Done false
5289 2413 Acked Write 5 Sent Done Done false
5290 2414 Acked Write 9 Sent Done Done false
5291 2415 Acked Read 8 Sent Done Done false
5292 2416 Acked Write 3 Sent Done Done false
5293 2417 Acked Read 5 Sent Done Done false
5294 2418 Acked Write 5 Sent Done Done false
5295 2419 Acked Read 9 Sent Done Done false
5296 2420 Acked Flush 0 Sent Done Done false
5297 2421 Acked Read 8 Sent Done Done false
5298 2422 Acked Write 6 Sent Done Done false
5299 2423 Acked Read 10 Sent Done Done false
5300 2424 Acked Write 10 Sent Done Done false
5301 2425 Acked Read 1 Sent Done Done false
5302 2426 Acked Read 10 Sent Done Done false
5303 2427 Acked Write 6 Sent Done Done false
5304 2428 Acked Read 4 Sent Done Done false
5305 2429 Acked Read 10 Sent Done Done false
5306 2430 Acked Read 9 Sent Done Done false
5307 2431 Acked Read 8 Sent Done Done false
5308 2432 Acked Read 5 Sent Done Done false
5309 2433 Acked Read 8 Sent Done Done false
5310 2434 Acked Write 2 Sent Done Done false
5311 2435 Acked Read 3 Sent Done Done false
5312 2436 Acked Read 8 Sent Done Done false
5313 2437 Acked Read 8 Sent Done Done false
5314 2438 Acked Read 7 Sent Done Done false
5315 2439 Acked Read 7 Sent Done Done false
5316 2440 Acked Write 4 Sent Done Done false
5317 2441 Acked Read 6 Sent Done Done false
5318 2442 Acked Write 5 Sent Done Done false
5319 2443 Acked Write 3 Sent Done Done false
5320 2444 Acked Write 8 Sent Done Done false
5321 2445 Acked Write 7 Sent Done Done false
5322 2446 Acked Read 5 Sent Done Done false
5323 2447 Acked Read 4 Sent Done Done false
5324 2448 Acked Write 9 Sent Done Done false
5325 2449 Acked Flush 0 Sent Done Done false
5326 2450 Acked Read 6 Sent Done Done false
5327 2451 Acked Write 9 Sent Done Done false
5328 2452 Acked Read 7 Sent Done Done false
5329 2453 Acked Write 8 Sent Done Done false
5330 2454 Acked Read 9 Sent Done Done false
5331 2455 Acked Write 1 Sent Done Done false
5332 2456 Acked Read 6 Sent Done Done false
5333 2457 Acked Read 5 Sent Done Done false
5334 2458 Acked Read 4 Sent Done Done false
5335 2459 Acked Flush 0 Sent Done Done false
5336 2460 Acked Read 3 Sent Done Done false
5337 2461 Acked Flush 0 Sent Done Done false
5338 2462 Acked Write 5 Sent Done Done false
5339 2463 Acked Write 10 Sent Done Done false
5340 2464 Acked Write 2 Sent Done Done false
5341 2465 Acked Read 2 Sent Done Done false
5342 2466 Acked Write 2 Sent Done Done false
5343 2467 Acked Write 4 Sent Done Done false
5344 2468 Acked Write 2 Sent Done Done false
5345 2469 Acked Read 3 Sent Done Done false
5346 2470 Acked Write 9 Sent Done Done false
5347 2471 Acked Write 4 Sent Done Done false
5348 2472 Acked Read 1 Sent Done Done false
5349 2473 Acked Write 3 Sent Done Done false
5350 2474 Acked Read 4 Sent Done Done false
5351 2475 Acked Flush 0 Sent Done Done false
5352 2476 Acked Flush 0 Sent Done Done false
5353 2477 Acked Read 7 Sent Done Done false
5354 2478 Acked Read 6 Sent Done Done false
5355 2479 Acked Flush 0 Sent Done Done false
5356 2480 Acked Write 4 Sent Done Done false
5357 2481 Acked Read 1 Sent Done Done false
5358 2482 Acked Read 10 Sent Done Done false
5359 2483 Acked Write 6 Sent Done Done false
5360 2484 Acked Read 8 Sent Done Done false
5361 2485 Acked Read 2 Sent Done Done false
5362 2486 Acked Read 6 Sent Done Done false
5363 2487 Acked Flush 0 Sent Done Done false
5364 2488 Acked Read 2 Sent Done Done false
5365 2489 Acked Read 2 Sent Done Done false
5366 2490 Acked Read 2 Sent Done Done false
5367 2491 Acked Write 1 Sent Done Done false
5368 2492 Acked Write 4 Sent Done Done false
5369 2493 Acked Write 8 Sent Done Done false
5370 2494 Acked Read 8 Sent Done Done false
5371 2495 Acked Write 7 Sent Done Done false
5372 2496 Acked Read 1 Sent Done Done false
5373 2497 Acked Read 7 Sent Done Done false
5374 2498 Acked Read 2 Sent Done Done false
5375 2499 Acked Read 2 Sent Done Done false
5376 2500 Acked Write 10 Sent Done Done false
5377 2501 Acked Write 7 Sent Done Done false
5378 2502 Acked Read 9 Sent Done Done false
5379 2503 Acked Write 8 Sent Done Done false
5380 2504 Acked Write 4 Sent Done Done false
5381 2505 Acked Write 7 Sent Done Done false
5382 2506 Acked Write 2 Sent Done Done false
5383 2507 Acked Write 3 Sent Done Done false
5384 2508 Acked Flush 0 Sent Done Done false
5385 STATES DS:0 DS:1 DS:2 TOTAL
5386 Sent 301 0 0 301
5387 Done 0 301 301 602
5388 Skipped 0 0 0 0
5389 Error 0 0 0 0
5390 Last Flush: 2207 2508 2508
5391 Downstairs last five completed: 2207 2206 2205 2204 2203
5392 Upstairs last five completed: 2508 2507 2506 2505 2504
5393 CLIENT: Up:0 ds:301 act:2
5394 ----------------------------------------------------------------
5395 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5396 States: Offline Active Active
5397 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5398 2208 Acked Read 10 Sent Done Done false
5399 2209 Acked Read 2 Sent Done Done false
5400 2210 Acked Read 2 Sent Done Done false
5401 2211 Acked Read 7 Sent Done Done false
5402 2212 Acked Flush 0 Sent Done Done false
5403 2213 Acked Read 9 Sent Done Done false
5404 2214 Acked Write 6 Sent Done Done false
5405 2215 Acked Write 9 Sent Done Done false
5406 2216 Acked Read 7 Sent Done Done false
5407 2217 Acked Write 4 Sent Done Done false
5408 2218 Acked Read 7 Sent Done Done false
5409 2219 Acked Read 8 Sent Done Done false
5410 2220 Acked Read 5 Sent Done Done false
5411 2221 Acked Read 3 Sent Done Done false
5412 2222 Acked Write 3 Sent Done Done false
5413 2223 Acked Write 1 Sent Done Done false
5414 2224 Acked Write 6 Sent Done Done false
5415 2225 Acked Read 1 Sent Done Done false
5416 2226 Acked Write 3 Sent Done Done false
5417 2227 Acked Write 8 Sent Done Done false
5418 2228 Acked Read 10 Sent Done Done false
5419 2229 Acked Read 9 Sent Done Done false
5420 2230 Acked Read 4 Sent Done Done false
5421 2231 Acked Write 9 Sent Done Done false
5422 2232 Acked Read 5 Sent Done Done false
5423 2233 Acked Write 8 Sent Done Done false
5424 2234 Acked Flush 0 Sent Done Done false
5425 2235 Acked Read 10 Sent Done Done false
5426 2236 Acked Write 8 Sent Done Done false
5427 2237 Acked Read 3 Sent Done Done false
5428 2238 Acked Write 4 Sent Done Done false
5429 2239 Acked Write 7 Sent Done Done false
5430 2240 Acked Write 9 Sent Done Done false
5431 2241 Acked Write 4 Sent Done Done false
5432 2242 Acked Read 5 Sent Done Done false
5433 2243 Acked Read 8 Sent Done Done false
5434 2244 Acked Read 6 Sent Done Done false
5435 2245 Acked Write 10 Sent Done Done false
5436 2246 Acked Write 3 Sent Done Done false
5437 2247 Acked Read 6 Sent Done Done false
5438 2248 Acked Write 2 Sent Done Done false
5439 2249 Acked Write 8 Sent Done Done false
5440 2250 Acked Write 9 Sent Done Done false
5441 2251 Acked Read 7 Sent Done Done false
5442 2252 Acked Write 6 Sent Done Done false
5443 2253 Acked Read 5 Sent Done Done false
5444 2254 Acked Read 5 Sent Done Done false
5445 2255 Acked Flush 0 Sent Done Done false
5446 2256 Acked Write 4 Sent Done Done false
5447 2257 Acked Read 9 Sent Done Done false
5448 2258 Acked Read 9 Sent Done Done false
5449 2259 Acked Read 3 Sent Done Done false
5450 2260 Acked Flush 0 Sent Done Done false
5451 2261 Acked Write 1 Sent Done Done false
5452 2262 Acked Read 2 Sent Done Done false
5453 2263 Acked Read 3 Sent Done Done false
5454 2264 Acked Read 2 Sent Done Done false
5455 2265 Acked Flush 0 Sent Done Done false
5456 2266 Acked Write 5 Sent Done Done false
5457 2267 Acked Flush 0 Sent Done Done false
5458 2268 Acked Write 3 Sent Done Done false
5459 2269 Acked Write 8 Sent Done Done false
5460 2270 Acked Read 4 Sent Done Done false
5461 2271 Acked Read 6 Sent Done Done false
5462 2272 Acked Write 5 Sent Done Done false
5463 2273 Acked Write 8 Sent Done Done false
5464 2274 Acked Read 2 Sent Done Done false
5465 2275 Acked Write 4 Sent Done Done false
5466 2276 Acked Write 5 Sent Done Done false
5467 2277 Acked Write 8 Sent Done Done false
5468 2278 Acked Write 7 Sent Done Done false
5469 2279 Acked Read 5 Sent Done Done false
5470 2280 Acked Read 7 Sent Done Done false
5471 2281 Acked Read 9 Sent Done Done false
5472 2282 Acked Read 3 Sent Done Done false
5473 2283 Acked Write 6 Sent Done Done false
5474 2284 Acked Read 10 Sent Done Done false
5475 2285 Acked Read 4 Sent Done Done false
5476 2286 Acked Write 10 Sent Done Done false
5477 2287 Acked Read 1 Sent Done Done false
5478 2288 Acked Write 9 Sent Done Done false
5479 2289 Acked Read 1 Sent Done Done false
5480 2290 Acked Read 10 Sent Done Done false
5481 2291 Acked Read 5 Sent Done Done false
5482 2292 Acked Read 8 Sent Done Done false
5483 2293 Acked Write 8 Sent Done Done false
5484 2294 Acked Read 3 Sent Done Done false
5485 2295 Acked Write 5 Sent Done Done false
5486 2296 Acked Read 5 Sent Done Done false
5487 2297 Acked Read 9 Sent Done Done false
5488 2298 Acked Read 7 Sent Done Done false
5489 2299 Acked Read 6 Sent Done Done false
5490 2300 Acked Read 1 Sent Done Done false
5491 2301 Acked Read 1 Sent Done Done false
5492 2302 Acked Write 8 Sent Done Done false
5493 2303 Acked Read 7 Sent Done Done false
5494 2304 Acked Write 4 Sent Done Done false
5495 2305 Acked Read 7 Sent Done Done false
5496 2306 Acked Write 6 Sent Done Done false
5497 2307 Acked Flush 0 Sent Done Done false
5498 2308 Acked Read 5 Sent Done Done false
5499 2309 Acked Read 5 Sent Done Done false
5500 2310 Acked Write 3 Sent Done Done false
5501 2311 Acked Write 4 Sent Done Done false
5502 2312 Acked Read 9 Sent Done Done false
5503 2313 Acked Write 5 Sent Done Done false
5504 2314 Acked Write 5 Sent Done Done false
5505 2315 Acked Read 2 Sent Done Done false
5506 2316 Acked Flush 0 Sent Done Done false
5507 2317 Acked Read 5 Sent Done Done false
5508 2318 Acked Flush 0 Sent Done Done false
5509 2319 Acked Write 2 Sent Done Done false
5510 2320 Acked Read 5 Sent Done Done false
5511 2321 Acked Flush 0 Sent Done Done false
5512 2322 Acked Write 9 Sent Done Done false
5513 2323 Acked Write 3 Sent Done Done false
5514 2324 Acked Read 7 Sent Done Done false
5515 2325 Acked Read 2 Sent Done Done false
5516 2326 Acked Read 10 Sent Done Done false
5517 2327 Acked Flush 0 Sent Done Done false
5518 2328 Acked Write 6 Sent Done Done false
5519 2329 Acked Read 4 Sent Done Done false
5520 2330 Acked Read 6 Sent Done Done false
5521 2331 Acked Read 2 Sent Done Done false
5522 2332 Acked Read 5 Sent Done Done false
5523 2333 Acked Read 2 Sent Done Done false
5524 2334 Acked Read 7 Sent Done Done false
5525 2335 Acked Write 9 Sent Done Done false
5526 2336 Acked Write 4 Sent Done Done false
5527 2337 Acked Read 2 Sent Done Done false
5528 2338 Acked Read 1 Sent Done Done false
5529 2339 Acked Read 3 Sent Done Done false
5530 2340 Acked Write 8 Sent Done Done false
5531 2341 Acked Write 2 Sent Done Done false
5532 2342 Acked Read 8 Sent Done Done false
5533 2343 Acked Read 5 Sent Done Done false
5534 2344 Acked Read 9 Sent Done Done false
5535 2345 Acked Read 10 Sent Done Done false
5536 2346 Acked Read 9 Sent Done Done false
5537 2347 Acked Write 8 Sent Done Done false
5538 2348 Acked Read 10 Sent Done Done false
5539 2349 Acked Flush 0 Sent Done Done false
5540 2350 Acked Write 3 Sent Done Done false
5541 2351 Acked Read 4 Sent Done Done false
5542 2352 Acked Read 8 Sent Done Done false
5543 2353 Acked Flush 0 Sent Done Done false
5544 2354 Acked Read 1 Sent Done Done false
5545 2355 Acked Read 2 Sent Done Done false
5546 2356 Acked Read 8 Sent Done Done false
5547 2357 Acked Write 5 Sent Done Done false
5548 2358 Acked Read 6 Sent Done Done false
5549 2359 Acked Read 10 Sent Done Done false
5550 2360 Acked Read 2 Sent Done Done false
5551 2361 Acked Write 7 Sent Done Done false
5552 2362 Acked Write 3 Sent Done Done false
5553 2363 Acked Read 5 Sent Done Done false
5554 2364 Acked Write 3 Sent Done Done false
5555 2365 Acked Read 6 Sent Done Done false
5556 2366 Acked Read 4 Sent Done Done false
5557 2367 Acked Read 10 Sent Done Done false
5558 2368 Acked Read 1 Sent Done Done false
5559 2369 Acked Read 2 Sent Done Done false
5560 2370 Acked Read 6 Sent Done Done false
5561 2371 Acked Write 5 Sent Done Done false
5562 2372 Acked Write 2 Sent Done Done false
5563 2373 Acked Read 4 Sent Done Done false
5564 2374 Acked Read 3 Sent Done Done false
5565 2375 Acked Read 5 Sent Done Done false
5566 2376 Acked Write 10 Sent Done Done false
5567 2377 Acked Read 4 Sent Done Done false
5568 2378 Acked Read 9 Sent Done Done false
5569 2379 Acked Write 8 Sent Done Done false
5570 2380 Acked Write 1 Sent Done Done false
5571 2381 Acked Read 4 Sent Done Done false
5572 2382 Acked Read 3 Sent Done Done false
5573 2383 Acked Write 1 Sent Done Done false
5574 2384 Acked Read 6 Sent Done Done false
5575 2385 Acked Write 7 Sent Done Done false
5576 2386 Acked Write 5 Sent Done Done false
5577 2387 Acked Read 7 Sent Done Done false
5578 2388 Acked Read 3 Sent Done Done false
5579 2389 Acked Read 1 Sent Done Done false
5580 2390 Acked Write 4 Sent Done Done false
5581 2391 Acked Read 8 Sent Done Done false
5582 2392 Acked Read 4 Sent Done Done false
5583 2393 Acked Read 3 Sent Done Done false
5584 2394 Acked Write 4 Sent Done Done false
5585 2395 Acked Read 10 Sent Done Done false
5586 2396 Acked Write 9 Sent Done Done false
5587 2397 Acked Write 8 Sent Done Done false
5588 2398 Acked Read 10 Sent Done Done false
5589 2399 Acked Read 6 Sent Done Done false
5590 2400 Acked Read 10 Sent Done Done false
5591 2401 Acked Write 7 Sent Done Done false
5592 2402 Acked Read 1 Sent Done Done false
5593 2403 Acked Write 7 Sent Done Done false
5594 2404 Acked Read 9 Sent Done Done false
5595 2405 Acked Write 5 Sent Done Done false
5596 2406 Acked Read 8 Sent Done Done false
5597 2407 Acked Read 5 Sent Done Done false
5598 2408 Acked Read 6 Sent Done Done false
5599 2409 Acked Read 5 Sent Done Done false
5600 2410 Acked Write 9 Sent Done Done false
5601 2411 Acked Flush 0 Sent Done Done false
5602 2412 Acked Write 2 Sent Done Done false
5603 2413 Acked Write 5 Sent Done Done false
5604 2414 Acked Write 9 Sent Done Done false
5605 2415 Acked Read 8 Sent Done Done false
5606 2416 Acked Write 3 Sent Done Done false
5607 2417 Acked Read 5 Sent Done Done false
5608 2418 Acked Write 5 Sent Done Done false
5609 2419 Acked Read 9 Sent Done Done false
5610 2420 Acked Flush 0 Sent Done Done false
5611 2421 Acked Read 8 Sent Done Done false
5612 2422 Acked Write 6 Sent Done Done false
5613 2423 Acked Read 10 Sent Done Done false
5614 2424 Acked Write 10 Sent Done Done false
5615 2425 Acked Read 1 Sent Done Done false
5616 2426 Acked Read 10 Sent Done Done false
5617 2427 Acked Write 6 Sent Done Done false
5618 2428 Acked Read 4 Sent Done Done false
5619 2429 Acked Read 10 Sent Done Done false
5620 2430 Acked Read 9 Sent Done Done false
5621 2431 Acked Read 8 Sent Done Done false
5622 2432 Acked Read 5 Sent Done Done false
5623 2433 Acked Read 8 Sent Done Done false
5624 2434 Acked Write 2 Sent Done Done false
5625 2435 Acked Read 3 Sent Done Done false
5626 2436 Acked Read 8 Sent Done Done false
5627 2437 Acked Read 8 Sent Done Done false
5628 2438 Acked Read 7 Sent Done Done false
5629 2439 Acked Read 7 Sent Done Done false
5630 2440 Acked Write 4 Sent Done Done false
5631 2441 Acked Read 6 Sent Done Done false
5632 2442 Acked Write 5 Sent Done Done false
5633 2443 Acked Write 3 Sent Done Done false
5634 2444 Acked Write 8 Sent Done Done false
5635 2445 Acked Write 7 Sent Done Done false
5636 2446 Acked Read 5 Sent Done Done false
5637 2447 Acked Read 4 Sent Done Done false
5638 2448 Acked Write 9 Sent Done Done false
5639 2449 Acked Flush 0 Sent Done Done false
5640 2450 Acked Read 6 Sent Done Done false
5641 2451 Acked Write 9 Sent Done Done false
5642 2452 Acked Read 7 Sent Done Done false
5643 2453 Acked Write 8 Sent Done Done false
5644 2454 Acked Read 9 Sent Done Done false
5645 2455 Acked Write 1 Sent Done Done false
5646 2456 Acked Read 6 Sent Done Done false
5647 2457 Acked Read 5 Sent Done Done false
5648 2458 Acked Read 4 Sent Done Done false
5649 2459 Acked Flush 0 Sent Done Done false
5650 2460 Acked Read 3 Sent Done Done false
5651 2461 Acked Flush 0 Sent Done Done false
5652 2462 Acked Write 5 Sent Done Done false
5653 2463 Acked Write 10 Sent Done Done false
5654 2464 Acked Write 2 Sent Done Done false
5655 2465 Acked Read 2 Sent Done Done false
5656 2466 Acked Write 2 Sent Done Done false
5657 2467 Acked Write 4 Sent Done Done false
5658 2468 Acked Write 2 Sent Done Done false
5659 2469 Acked Read 3 Sent Done Done false
5660 2470 Acked Write 9 Sent Done Done false
5661 2471 Acked Write 4 Sent Done Done false
5662 2472 Acked Read 1 Sent Done Done false
5663 2473 Acked Write 3 Sent Done Done false
5664 2474 Acked Read 4 Sent Done Done false
5665 2475 Acked Flush 0 Sent Done Done false
5666 2476 Acked Flush 0 Sent Done Done false
5667 2477 Acked Read 7 Sent Done Done false
5668 2478 Acked Read 6 Sent Done Done false
5669 2479 Acked Flush 0 Sent Done Done false
5670 2480 Acked Write 4 Sent Done Done false
5671 2481 Acked Read 1 Sent Done Done false
5672 2482 Acked Read 10 Sent Done Done false
5673 2483 Acked Write 6 Sent Done Done false
5674 2484 Acked Read 8 Sent Done Done false
5675 2485 Acked Read 2 Sent Done Done false
5676 2486 Acked Read 6 Sent Done Done false
5677 2487 Acked Flush 0 Sent Done Done false
5678 2488 Acked Read 2 Sent Done Done false
5679 2489 Acked Read 2 Sent Done Done false
5680 2490 Acked Read 2 Sent Done Done false
5681 2491 Acked Write 1 Sent Done Done false
5682 2492 Acked Write 4 Sent Done Done false
5683 2493 Acked Write 8 Sent Done Done false
5684 2494 Acked Read 8 Sent Done Done false
5685 2495 Acked Write 7 Sent Done Done false
5686 2496 Acked Read 1 Sent Done Done false
5687 2497 Acked Read 7 Sent Done Done false
5688 2498 Acked Read 2 Sent Done Done false
5689 2499 Acked Read 2 Sent Done Done false
5690 2500 Acked Write 10 Sent Done Done false
5691 2501 Acked Write 7 Sent Done Done false
5692 2502 Acked Read 9 Sent Done Done false
5693 2503 Acked Write 8 Sent Done Done false
5694 2504 Acked Write 4 Sent Done Done false
5695 2505 Acked Write 7 Sent Done Done false
5696 2506 Acked Write 2 Sent Done Done false
5697 2507 Acked Write 3 Sent Done Done false
5698 2508 Acked Flush 0 Sent Done Done false
5699 STATES DS:0 DS:1 DS:2 TOTAL
5700 Sent 301 0 0 301
5701 Done 0 301 301 602
5702 Skipped 0 0 0 0
5703 Error 0 0 0 0
5704 Last Flush: 2207 2508 2508
5705 Downstairs last five completed: 2207 2206 2205 2204 2203
5706 Upstairs last five completed: 2508 2507 2506 2505 2504
5707 CLIENT: Up:0 ds:301 act:2
57082025-01-03T08:32:04.536ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
57092025-01-03T08:32:04.536ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
57102025-01-03T08:32:04.536ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
57112025-01-03T08:32:04.536ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
57122025-01-03T08:32:04.536ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
5713 [8810] got start action
5714 [8810] I am going to restart
5715 Starting downstairs at port 8810
5716 Make output file at "/tmp/dsc/downstairs-8810.txt"
5717 [8810][0] reports Starting
5718 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1210)
5719 [8810][0] reports Running
5720 ----------------------------------------------------------------
5721 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5722 States: Offline Active Active
5723 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5724 2208 Acked Read 10 Sent Done Done false
5725 2209 Acked Read 2 Sent Done Done false
5726 2210 Acked Read 2 Sent Done Done false
5727 2211 Acked Read 7 Sent Done Done false
5728 2212 Acked Flush 0 Sent Done Done false
5729 2213 Acked Read 9 Sent Done Done false
5730 2214 Acked Write 6 Sent Done Done false
5731 2215 Acked Write 9 Sent Done Done false
5732 2216 Acked Read 7 Sent Done Done false
5733 2217 Acked Write 4 Sent Done Done false
5734 2218 Acked Read 7 Sent Done Done false
5735 2219 Acked Read 8 Sent Done Done false
5736 2220 Acked Read 5 Sent Done Done false
5737 2221 Acked Read 3 Sent Done Done false
5738 2222 Acked Write 3 Sent Done Done false
5739 2223 Acked Write 1 Sent Done Done false
5740 2224 Acked Write 6 Sent Done Done false
5741 2225 Acked Read 1 Sent Done Done false
5742 2226 Acked Write 3 Sent Done Done false
5743 2227 Acked Write 8 Sent Done Done false
5744 2228 Acked Read 10 Sent Done Done false
5745 2229 Acked Read 9 Sent Done Done false
5746 2230 Acked Read 4 Sent Done Done false
5747 2231 Acked Write 9 Sent Done Done false
5748 2232 Acked Read 5 Sent Done Done false
5749 2233 Acked Write 8 Sent Done Done false
5750 2234 Acked Flush 0 Sent Done Done false
5751 2235 Acked Read 10 Sent Done Done false
5752 2236 Acked Write 8 Sent Done Done false
5753 2237 Acked Read 3 Sent Done Done false
5754 2238 Acked Write 4 Sent Done Done false
5755 2239 Acked Write 7 Sent Done Done false
5756 2240 Acked Write 9 Sent Done Done false
5757 2241 Acked Write 4 Sent Done Done false
5758 2242 Acked Read 5 Sent Done Done false
5759 2243 Acked Read 8 Sent Done Done false
5760 2244 Acked Read 6 Sent Done Done false
5761 2245 Acked Write 10 Sent Done Done false
5762 2246 Acked Write 3 Sent Done Done false
5763 2247 Acked Read 6 Sent Done Done false
5764 2248 Acked Write 2 Sent Done Done false
5765 2249 Acked Write 8 Sent Done Done false
5766 2250 Acked Write 9 Sent Done Done false
5767 2251 Acked Read 7 Sent Done Done false
5768 2252 Acked Write 6 Sent Done Done false
5769 2253 Acked Read 5 Sent Done Done false
5770 2254 Acked Read 5 Sent Done Done false
5771 2255 Acked Flush 0 Sent Done Done false
5772 2256 Acked Write 4 Sent Done Done false
5773 2257 Acked Read 9 Sent Done Done false
5774 2258 Acked Read 9 Sent Done Done false
5775 2259 Acked Read 3 Sent Done Done false
5776 2260 Acked Flush 0 Sent Done Done false
5777 2261 Acked Write 1 Sent Done Done false
5778 2262 Acked Read 2 Sent Done Done false
5779 2263 Acked Read 3 Sent Done Done false
5780 2264 Acked Read 2 Sent Done Done false
5781 2265 Acked Flush 0 Sent Done Done false
5782 2266 Acked Write 5 Sent Done Done false
5783 2267 Acked Flush 0 Sent Done Done false
5784 2268 Acked Write 3 Sent Done Done false
5785 2269 Acked Write 8 Sent Done Done false
5786 2270 Acked Read 4 Sent Done Done false
5787 2271 Acked Read 6 Sent Done Done false
5788 2272 Acked Write 5 Sent Done Done false
5789 2273 Acked Write 8 Sent Done Done false
5790 2274 Acked Read 2 Sent Done Done false
5791 2275 Acked Write 4 Sent Done Done false
5792 2276 Acked Write 5 Sent Done Done false
5793 2277 Acked Write 8 Sent Done Done false
5794 2278 Acked Write 7 Sent Done Done false
5795 2279 Acked Read 5 Sent Done Done false
5796 2280 Acked Read 7 Sent Done Done false
5797 2281 Acked Read 9 Sent Done Done false
5798 2282 Acked Read 3 Sent Done Done false
5799 2283 Acked Write 6 Sent Done Done false
5800 2284 Acked Read 10 Sent Done Done false
5801 2285 Acked Read 4 Sent Done Done false
5802 2286 Acked Write 10 Sent Done Done false
5803 2287 Acked Read 1 Sent Done Done false
5804 2288 Acked Write 9 Sent Done Done false
5805 2289 Acked Read 1 Sent Done Done false
5806 2290 Acked Read 10 Sent Done Done false
5807 2291 Acked Read 5 Sent Done Done false
5808 2292 Acked Read 8 Sent Done Done false
5809 2293 Acked Write 8 Sent Done Done false
5810 2294 Acked Read 3 Sent Done Done false
5811 2295 Acked Write 5 Sent Done Done false
5812 2296 Acked Read 5 Sent Done Done false
5813 2297 Acked Read 9 Sent Done Done false
5814 2298 Acked Read 7 Sent Done Done false
5815 2299 Acked Read 6 Sent Done Done false
5816 2300 Acked Read 1 Sent Done Done false
5817 2301 Acked Read 1 Sent Done Done false
5818 2302 Acked Write 8 Sent Done Done false
5819 2303 Acked Read 7 Sent Done Done false
5820 2304 Acked Write 4 Sent Done Done false
5821 2305 Acked Read 7 Sent Done Done false
5822 2306 Acked Write 6 Sent Done Done false
5823 2307 Acked Flush 0 Sent Done Done false
5824 2308 Acked Read 5 Sent Done Done false
5825 2309 Acked Read 5 Sent Done Done false
5826 2310 Acked Write 3 Sent Done Done false
5827 2311 Acked Write 4 Sent Done Done false
5828 2312 Acked Read 9 Sent Done Done false
5829 2313 Acked Write 5 Sent Done Done false
5830 2314 Acked Write 5 Sent Done Done false
5831 2315 Acked Read 2 Sent Done Done false
5832 2316 Acked Flush 0 Sent Done Done false
5833 2317 Acked Read 5 Sent Done Done false
5834 2318 Acked Flush 0 Sent Done Done false
5835 2319 Acked Write 2 Sent Done Done false
5836 2320 Acked Read 5 Sent Done Done false
5837 2321 Acked Flush 0 Sent Done Done false
5838 2322 Acked Write 9 Sent Done Done false
5839 2323 Acked Write 3 Sent Done Done false
5840 2324 Acked Read 7 Sent Done Done false
5841 2325 Acked Read 2 Sent Done Done false
5842 2326 Acked Read 10 Sent Done Done false
5843 2327 Acked Flush 0 Sent Done Done false
5844 2328 Acked Write 6 Sent Done Done false
5845 2329 Acked Read 4 Sent Done Done false
5846 2330 Acked Read 6 Sent Done Done false
5847 2331 Acked Read 2 Sent Done Done false
5848 2332 Acked Read 5 Sent Done Done false
5849 2333 Acked Read 2 Sent Done Done false
5850 2334 Acked Read 7 Sent Done Done false
5851 2335 Acked Write 9 Sent Done Done false
5852 2336 Acked Write 4 Sent Done Done false
5853 2337 Acked Read 2 Sent Done Done false
5854 2338 Acked Read 1 Sent Done Done false
5855 2339 Acked Read 3 Sent Done Done false
5856 2340 Acked Write 8 Sent Done Done false
5857 2341 Acked Write 2 Sent Done Done false
5858 2342 Acked Read 8 Sent Done Done false
5859 2343 Acked Read 5 Sent Done Done false
5860 2344 Acked Read 9 Sent Done Done false
5861 2345 Acked Read 10 Sent Done Done false
5862 2346 Acked Read 9 Sent Done Done false
5863 2347 Acked Write 8 Sent Done Done false
5864 2348 Acked Read 10 Sent Done Done false
5865 2349 Acked Flush 0 Sent Done Done false
5866 2350 Acked Write 3 Sent Done Done false
5867 2351 Acked Read 4 Sent Done Done false
5868 2352 Acked Read 8 Sent Done Done false
5869 2353 Acked Flush 0 Sent Done Done false
5870 2354 Acked Read 1 Sent Done Done false
5871 2355 Acked Read 2 Sent Done Done false
5872 2356 Acked Read 8 Sent Done Done false
5873 2357 Acked Write 5 Sent Done Done false
5874 2358 Acked Read 6 Sent Done Done false
5875 2359 Acked Read 10 Sent Done Done false
5876 2360 Acked Read 2 Sent Done Done false
5877 2361 Acked Write 7 Sent Done Done false
5878 2362 Acked Write 3 Sent Done Done false
5879 2363 Acked Read 5 Sent Done Done false
5880 2364 Acked Write 3 Sent Done Done false
5881 2365 Acked Read 6 Sent Done Done false
5882 2366 Acked Read 4 Sent Done Done false
5883 2367 Acked Read 10 Sent Done Done false
5884 2368 Acked Read 1 Sent Done Done false
5885 2369 Acked Read 2 Sent Done Done false
5886 2370 Acked Read 6 Sent Done Done false
5887 2371 Acked Write 5 Sent Done Done false
5888 2372 Acked Write 2 Sent Done Done false
5889 2373 Acked Read 4 Sent Done Done false
5890 2374 Acked Read 3 Sent Done Done false
5891 2375 Acked Read 5 Sent Done Done false
5892 2376 Acked Write 10 Sent Done Done false
5893 2377 Acked Read 4 Sent Done Done false
5894 2378 Acked Read 9 Sent Done Done false
5895 2379 Acked Write 8 Sent Done Done false
5896 2380 Acked Write 1 Sent Done Done false
5897 2381 Acked Read 4 Sent Done Done false
5898 2382 Acked Read 3 Sent Done Done false
5899 2383 Acked Write 1 Sent Done Done false
5900 2384 Acked Read 6 Sent Done Done false
5901 2385 Acked Write 7 Sent Done Done false
5902 2386 Acked Write 5 Sent Done Done false
5903 2387 Acked Read 7 Sent Done Done false
5904 2388 Acked Read 3 Sent Done Done false
5905 2389 Acked Read 1 Sent Done Done false
5906 2390 Acked Write 4 Sent Done Done false
5907 2391 Acked Read 8 Sent Done Done false
5908 2392 Acked Read 4 Sent Done Done false
5909 2393 Acked Read 3 Sent Done Done false
5910 2394 Acked Write 4 Sent Done Done false
5911 2395 Acked Read 10 Sent Done Done false
5912 2396 Acked Write 9 Sent Done Done false
5913 2397 Acked Write 8 Sent Done Done false
5914 2398 Acked Read 10 Sent Done Done false
5915 2399 Acked Read 6 Sent Done Done false
5916 2400 Acked Read 10 Sent Done Done false
5917 2401 Acked Write 7 Sent Done Done false
5918 2402 Acked Read 1 Sent Done Done false
5919 2403 Acked Write 7 Sent Done Done false
5920 2404 Acked Read 9 Sent Done Done false
5921 2405 Acked Write 5 Sent Done Done false
5922 2406 Acked Read 8 Sent Done Done false
5923 2407 Acked Read 5 Sent Done Done false
5924 2408 Acked Read 6 Sent Done Done false
5925 2409 Acked Read 5 Sent Done Done false
5926 2410 Acked Write 9 Sent Done Done false
5927 2411 Acked Flush 0 Sent Done Done false
5928 2412 Acked Write 2 Sent Done Done false
5929 2413 Acked Write 5 Sent Done Done false
5930 2414 Acked Write 9 Sent Done Done false
5931 2415 Acked Read 8 Sent Done Done false
5932 2416 Acked Write 3 Sent Done Done false
5933 2417 Acked Read 5 Sent Done Done false
5934 2418 Acked Write 5 Sent Done Done false
5935 2419 Acked Read 9 Sent Done Done false
5936 2420 Acked Flush 0 Sent Done Done false
5937 2421 Acked Read 8 Sent Done Done false
5938 2422 Acked Write 6 Sent Done Done false
5939 2423 Acked Read 10 Sent Done Done false
5940 2424 Acked Write 10 Sent Done Done false
5941 2425 Acked Read 1 Sent Done Done false
5942 2426 Acked Read 10 Sent Done Done false
5943 2427 Acked Write 6 Sent Done Done false
5944 2428 Acked Read 4 Sent Done Done false
5945 2429 Acked Read 10 Sent Done Done false
5946 2430 Acked Read 9 Sent Done Done false
5947 2431 Acked Read 8 Sent Done Done false
5948 2432 Acked Read 5 Sent Done Done false
5949 2433 Acked Read 8 Sent Done Done false
5950 2434 Acked Write 2 Sent Done Done false
5951 2435 Acked Read 3 Sent Done Done false
5952 2436 Acked Read 8 Sent Done Done false
5953 2437 Acked Read 8 Sent Done Done false
5954 2438 Acked Read 7 Sent Done Done false
5955 2439 Acked Read 7 Sent Done Done false
5956 2440 Acked Write 4 Sent Done Done false
5957 2441 Acked Read 6 Sent Done Done false
5958 2442 Acked Write 5 Sent Done Done false
5959 2443 Acked Write 3 Sent Done Done false
5960 2444 Acked Write 8 Sent Done Done false
5961 2445 Acked Write 7 Sent Done Done false
5962 2446 Acked Read 5 Sent Done Done false
5963 2447 Acked Read 4 Sent Done Done false
5964 2448 Acked Write 9 Sent Done Done false
5965 2449 Acked Flush 0 Sent Done Done false
5966 2450 Acked Read 6 Sent Done Done false
5967 2451 Acked Write 9 Sent Done Done false
5968 2452 Acked Read 7 Sent Done Done false
5969 2453 Acked Write 8 Sent Done Done false
5970 2454 Acked Read 9 Sent Done Done false
5971 2455 Acked Write 1 Sent Done Done false
5972 2456 Acked Read 6 Sent Done Done false
5973 2457 Acked Read 5 Sent Done Done false
5974 2458 Acked Read 4 Sent Done Done false
5975 2459 Acked Flush 0 Sent Done Done false
5976 2460 Acked Read 3 Sent Done Done false
5977 2461 Acked Flush 0 Sent Done Done false
5978 2462 Acked Write 5 Sent Done Done false
5979 2463 Acked Write 10 Sent Done Done false
5980 2464 Acked Write 2 Sent Done Done false
5981 2465 Acked Read 2 Sent Done Done false
5982 2466 Acked Write 2 Sent Done Done false
5983 2467 Acked Write 4 Sent Done Done false
5984 2468 Acked Write 2 Sent Done Done false
5985 2469 Acked Read 3 Sent Done Done false
5986 2470 Acked Write 9 Sent Done Done false
5987 2471 Acked Write 4 Sent Done Done false
5988 2472 Acked Read 1 Sent Done Done false
5989 2473 Acked Write 3 Sent Done Done false
5990 2474 Acked Read 4 Sent Done Done false
5991 2475 Acked Flush 0 Sent Done Done false
5992 2476 Acked Flush 0 Sent Done Done false
5993 2477 Acked Read 7 Sent Done Done false
5994 2478 Acked Read 6 Sent Done Done false
5995 2479 Acked Flush 0 Sent Done Done false
5996 2480 Acked Write 4 Sent Done Done false
5997 2481 Acked Read 1 Sent Done Done false
5998 2482 Acked Read 10 Sent Done Done false
5999 2483 Acked Write 6 Sent Done Done false
6000 2484 Acked Read 8 Sent Done Done false
6001 2485 Acked Read 2 Sent Done Done false
6002 2486 Acked Read 6 Sent Done Done false
6003 2487 Acked Flush 0 Sent Done Done false
6004 2488 Acked Read 2 Sent Done Done false
6005 2489 Acked Read 2 Sent Done Done false
6006 2490 Acked Read 2 Sent Done Done false
6007 2491 Acked Write 1 Sent Done Done false
6008 2492 Acked Write 4 Sent Done Done false
6009 2493 Acked Write 8 Sent Done Done false
6010 2494 Acked Read 8 Sent Done Done false
6011 2495 Acked Write 7 Sent Done Done false
6012 2496 Acked Read 1 Sent Done Done false
6013 2497 Acked Read 7 Sent Done Done false
6014 2498 Acked Read 2 Sent Done Done false
6015 2499 Acked Read 2 Sent Done Done false
6016 2500 Acked Write 10 Sent Done Done false
6017 2501 Acked Write 7 Sent Done Done false
6018 2502 Acked Read 9 Sent Done Done false
6019 2503 Acked Write 8 Sent Done Done false
6020 2504 Acked Write 4 Sent Done Done false
6021 2505 Acked Write 7 Sent Done Done false
6022 2506 Acked Write 2 Sent Done Done false
6023 2507 Acked Write 3 Sent Done Done false
6024 2508 Acked Flush 0 Sent Done Done false
6025 STATES DS:0 DS:1 DS:2 TOTAL
6026 Sent 301 0 0 301
6027 Done 0 301 301 602
6028 Skipped 0 0 0 0
6029 Error 0 0 0 0
6030 Last Flush: 2207 2508 2508
6031 Downstairs last five completed: 2207 2206 2205 2204 2203
6032 Upstairs last five completed: 2508 2507 2506 2505 2504
6033 CLIENT: Up:0 ds:301 act:2
6034 ----------------------------------------------------------------
6035 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6036 States: Offline Active Active
6037 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6038 2208 Acked Read 10 Sent Done Done false
6039 2209 Acked Read 2 Sent Done Done false
6040 2210 Acked Read 2 Sent Done Done false
6041 2211 Acked Read 7 Sent Done Done false
6042 2212 Acked Flush 0 Sent Done Done false
6043 2213 Acked Read 9 Sent Done Done false
6044 2214 Acked Write 6 Sent Done Done false
6045 2215 Acked Write 9 Sent Done Done false
6046 2216 Acked Read 7 Sent Done Done false
6047 2217 Acked Write 4 Sent Done Done false
6048 2218 Acked Read 7 Sent Done Done false
6049 2219 Acked Read 8 Sent Done Done false
6050 2220 Acked Read 5 Sent Done Done false
6051 2221 Acked Read 3 Sent Done Done false
6052 2222 Acked Write 3 Sent Done Done false
6053 2223 Acked Write 1 Sent Done Done false
6054 2224 Acked Write 6 Sent Done Done false
6055 2225 Acked Read 1 Sent Done Done false
6056 2226 Acked Write 3 Sent Done Done false
6057 2227 Acked Write 8 Sent Done Done false
6058 2228 Acked Read 10 Sent Done Done false
6059 2229 Acked Read 9 Sent Done Done false
6060 2230 Acked Read 4 Sent Done Done false
6061 2231 Acked Write 9 Sent Done Done false
6062 2232 Acked Read 5 Sent Done Done false
6063 2233 Acked Write 8 Sent Done Done false
6064 2234 Acked Flush 0 Sent Done Done false
6065 2235 Acked Read 10 Sent Done Done false
6066 2236 Acked Write 8 Sent Done Done false
6067 2237 Acked Read 3 Sent Done Done false
6068 2238 Acked Write 4 Sent Done Done false
6069 2239 Acked Write 7 Sent Done Done false
6070 2240 Acked Write 9 Sent Done Done false
6071 2241 Acked Write 4 Sent Done Done false
6072 2242 Acked Read 5 Sent Done Done false
6073 2243 Acked Read 8 Sent Done Done false
6074 2244 Acked Read 6 Sent Done Done false
6075 2245 Acked Write 10 Sent Done Done false
6076 2246 Acked Write 3 Sent Done Done false
6077 2247 Acked Read 6 Sent Done Done false
6078 2248 Acked Write 2 Sent Done Done false
6079 2249 Acked Write 8 Sent Done Done false
6080 2250 Acked Write 9 Sent Done Done false
6081 2251 Acked Read 7 Sent Done Done false
6082 2252 Acked Write 6 Sent Done Done false
6083 2253 Acked Read 5 Sent Done Done false
6084 2254 Acked Read 5 Sent Done Done false
6085 2255 Acked Flush 0 Sent Done Done false
6086 2256 Acked Write 4 Sent Done Done false
6087 2257 Acked Read 9 Sent Done Done false
6088 2258 Acked Read 9 Sent Done Done false
6089 2259 Acked Read 3 Sent Done Done false
6090 2260 Acked Flush 0 Sent Done Done false
6091 2261 Acked Write 1 Sent Done Done false
6092 2262 Acked Read 2 Sent Done Done false
6093 2263 Acked Read 3 Sent Done Done false
6094 2264 Acked Read 2 Sent Done Done false
6095 2265 Acked Flush 0 Sent Done Done false
6096 2266 Acked Write 5 Sent Done Done false
6097 2267 Acked Flush 0 Sent Done Done false
6098 2268 Acked Write 3 Sent Done Done false
6099 2269 Acked Write 8 Sent Done Done false
6100 2270 Acked Read 4 Sent Done Done false
6101 2271 Acked Read 6 Sent Done Done false
6102 2272 Acked Write 5 Sent Done Done false
6103 2273 Acked Write 8 Sent Done Done false
6104 2274 Acked Read 2 Sent Done Done false
6105 2275 Acked Write 4 Sent Done Done false
6106 2276 Acked Write 5 Sent Done Done false
6107 2277 Acked Write 8 Sent Done Done false
6108 2278 Acked Write 7 Sent Done Done false
6109 2279 Acked Read 5 Sent Done Done false
6110 2280 Acked Read 7 Sent Done Done false
6111 2281 Acked Read 9 Sent Done Done false
6112 2282 Acked Read 3 Sent Done Done false
6113 2283 Acked Write 6 Sent Done Done false
6114 2284 Acked Read 10 Sent Done Done false
6115 2285 Acked Read 4 Sent Done Done false
6116 2286 Acked Write 10 Sent Done Done false
6117 2287 Acked Read 1 Sent Done Done false
6118 2288 Acked Write 9 Sent Done Done false
6119 2289 Acked Read 1 Sent Done Done false
6120 2290 Acked Read 10 Sent Done Done false
6121 2291 Acked Read 5 Sent Done Done false
6122 2292 Acked Read 8 Sent Done Done false
6123 2293 Acked Write 8 Sent Done Done false
6124 2294 Acked Read 3 Sent Done Done false
6125 2295 Acked Write 5 Sent Done Done false
6126 2296 Acked Read 5 Sent Done Done false
6127 2297 Acked Read 9 Sent Done Done false
6128 2298 Acked Read 7 Sent Done Done false
6129 2299 Acked Read 6 Sent Done Done false
6130 2300 Acked Read 1 Sent Done Done false
6131 2301 Acked Read 1 Sent Done Done false
6132 2302 Acked Write 8 Sent Done Done false
6133 2303 Acked Read 7 Sent Done Done false
6134 2304 Acked Write 4 Sent Done Done false
6135 2305 Acked Read 7 Sent Done Done false
6136 2306 Acked Write 6 Sent Done Done false
6137 2307 Acked Flush 0 Sent Done Done false
6138 2308 Acked Read 5 Sent Done Done false
6139 2309 Acked Read 5 Sent Done Done false
6140 2310 Acked Write 3 Sent Done Done false
6141 2311 Acked Write 4 Sent Done Done false
6142 2312 Acked Read 9 Sent Done Done false
6143 2313 Acked Write 5 Sent Done Done false
6144 2314 Acked Write 5 Sent Done Done false
6145 2315 Acked Read 2 Sent Done Done false
6146 2316 Acked Flush 0 Sent Done Done false
6147 2317 Acked Read 5 Sent Done Done false
6148 2318 Acked Flush 0 Sent Done Done false
6149 2319 Acked Write 2 Sent Done Done false
6150 2320 Acked Read 5 Sent Done Done false
6151 2321 Acked Flush 0 Sent Done Done false
6152 2322 Acked Write 9 Sent Done Done false
6153 2323 Acked Write 3 Sent Done Done false
6154 2324 Acked Read 7 Sent Done Done false
6155 2325 Acked Read 2 Sent Done Done false
6156 2326 Acked Read 10 Sent Done Done false
6157 2327 Acked Flush 0 Sent Done Done false
6158 2328 Acked Write 6 Sent Done Done false
6159 2329 Acked Read 4 Sent Done Done false
6160 2330 Acked Read 6 Sent Done Done false
6161 2331 Acked Read 2 Sent Done Done false
6162 2332 Acked Read 5 Sent Done Done false
6163 2333 Acked Read 2 Sent Done Done false
6164 2334 Acked Read 7 Sent Done Done false
6165 2335 Acked Write 9 Sent Done Done false
6166 2336 Acked Write 4 Sent Done Done false
6167 2337 Acked Read 2 Sent Done Done false
6168 2338 Acked Read 1 Sent Done Done false
6169 2339 Acked Read 3 Sent Done Done false
6170 2340 Acked Write 8 Sent Done Done false
6171 2341 Acked Write 2 Sent Done Done false
6172 2342 Acked Read 8 Sent Done Done false
6173 2343 Acked Read 5 Sent Done Done false
6174 2344 Acked Read 9 Sent Done Done false
6175 2345 Acked Read 10 Sent Done Done false
6176 2346 Acked Read 9 Sent Done Done false
6177 2347 Acked Write 8 Sent Done Done false
6178 2348 Acked Read 10 Sent Done Done false
6179 2349 Acked Flush 0 Sent Done Done false
6180 2350 Acked Write 3 Sent Done Done false
6181 2351 Acked Read 4 Sent Done Done false
6182 2352 Acked Read 8 Sent Done Done false
6183 2353 Acked Flush 0 Sent Done Done false
6184 2354 Acked Read 1 Sent Done Done false
6185 2355 Acked Read 2 Sent Done Done false
6186 2356 Acked Read 8 Sent Done Done false
6187 2357 Acked Write 5 Sent Done Done false
6188 2358 Acked Read 6 Sent Done Done false
6189 2359 Acked Read 10 Sent Done Done false
6190 2360 Acked Read 2 Sent Done Done false
6191 2361 Acked Write 7 Sent Done Done false
6192 2362 Acked Write 3 Sent Done Done false
6193 2363 Acked Read 5 Sent Done Done false
6194 2364 Acked Write 3 Sent Done Done false
6195 2365 Acked Read 6 Sent Done Done false
6196 2366 Acked Read 4 Sent Done Done false
6197 2367 Acked Read 10 Sent Done Done false
6198 2368 Acked Read 1 Sent Done Done false
6199 2369 Acked Read 2 Sent Done Done false
6200 2370 Acked Read 6 Sent Done Done false
6201 2371 Acked Write 5 Sent Done Done false
6202 2372 Acked Write 2 Sent Done Done false
6203 2373 Acked Read 4 Sent Done Done false
6204 2374 Acked Read 3 Sent Done Done false
6205 2375 Acked Read 5 Sent Done Done false
6206 2376 Acked Write 10 Sent Done Done false
6207 2377 Acked Read 4 Sent Done Done false
6208 2378 Acked Read 9 Sent Done Done false
6209 2379 Acked Write 8 Sent Done Done false
6210 2380 Acked Write 1 Sent Done Done false
6211 2381 Acked Read 4 Sent Done Done false
6212 2382 Acked Read 3 Sent Done Done false
6213 2383 Acked Write 1 Sent Done Done false
6214 2384 Acked Read 6 Sent Done Done false
6215 2385 Acked Write 7 Sent Done Done false
6216 2386 Acked Write 5 Sent Done Done false
6217 2387 Acked Read 7 Sent Done Done false
6218 2388 Acked Read 3 Sent Done Done false
6219 2389 Acked Read 1 Sent Done Done false
6220 2390 Acked Write 4 Sent Done Done false
6221 2391 Acked Read 8 Sent Done Done false
6222 2392 Acked Read 4 Sent Done Done false
6223 2393 Acked Read 3 Sent Done Done false
6224 2394 Acked Write 4 Sent Done Done false
6225 2395 Acked Read 10 Sent Done Done false
6226 2396 Acked Write 9 Sent Done Done false
6227 2397 Acked Write 8 Sent Done Done false
6228 2398 Acked Read 10 Sent Done Done false
6229 2399 Acked Read 6 Sent Done Done false
6230 2400 Acked Read 10 Sent Done Done false
6231 2401 Acked Write 7 Sent Done Done false
6232 2402 Acked Read 1 Sent Done Done false
6233 2403 Acked Write 7 Sent Done Done false
6234 2404 Acked Read 9 Sent Done Done false
6235 2405 Acked Write 5 Sent Done Done false
6236 2406 Acked Read 8 Sent Done Done false
6237 2407 Acked Read 5 Sent Done Done false
6238 2408 Acked Read 6 Sent Done Done false
6239 2409 Acked Read 5 Sent Done Done false
6240 2410 Acked Write 9 Sent Done Done false
6241 2411 Acked Flush 0 Sent Done Done false
6242 2412 Acked Write 2 Sent Done Done false
6243 2413 Acked Write 5 Sent Done Done false
6244 2414 Acked Write 9 Sent Done Done false
6245 2415 Acked Read 8 Sent Done Done false
6246 2416 Acked Write 3 Sent Done Done false
6247 2417 Acked Read 5 Sent Done Done false
6248 2418 Acked Write 5 Sent Done Done false
6249 2419 Acked Read 9 Sent Done Done false
6250 2420 Acked Flush 0 Sent Done Done false
6251 2421 Acked Read 8 Sent Done Done false
6252 2422 Acked Write 6 Sent Done Done false
6253 2423 Acked Read 10 Sent Done Done false
6254 2424 Acked Write 10 Sent Done Done false
6255 2425 Acked Read 1 Sent Done Done false
6256 2426 Acked Read 10 Sent Done Done false
6257 2427 Acked Write 6 Sent Done Done false
6258 2428 Acked Read 4 Sent Done Done false
6259 2429 Acked Read 10 Sent Done Done false
6260 2430 Acked Read 9 Sent Done Done false
6261 2431 Acked Read 8 Sent Done Done false
6262 2432 Acked Read 5 Sent Done Done false
6263 2433 Acked Read 8 Sent Done Done false
6264 2434 Acked Write 2 Sent Done Done false
6265 2435 Acked Read 3 Sent Done Done false
6266 2436 Acked Read 8 Sent Done Done false
6267 2437 Acked Read 8 Sent Done Done false
6268 2438 Acked Read 7 Sent Done Done false
6269 2439 Acked Read 7 Sent Done Done false
6270 2440 Acked Write 4 Sent Done Done false
6271 2441 Acked Read 6 Sent Done Done false
6272 2442 Acked Write 5 Sent Done Done false
6273 2443 Acked Write 3 Sent Done Done false
6274 2444 Acked Write 8 Sent Done Done false
6275 2445 Acked Write 7 Sent Done Done false
6276 2446 Acked Read 5 Sent Done Done false
6277 2447 Acked Read 4 Sent Done Done false
6278 2448 Acked Write 9 Sent Done Done false
6279 2449 Acked Flush 0 Sent Done Done false
6280 2450 Acked Read 6 Sent Done Done false
6281 2451 Acked Write 9 Sent Done Done false
6282 2452 Acked Read 7 Sent Done Done false
6283 2453 Acked Write 8 Sent Done Done false
6284 2454 Acked Read 9 Sent Done Done false
6285 2455 Acked Write 1 Sent Done Done false
6286 2456 Acked Read 6 Sent Done Done false
6287 2457 Acked Read 5 Sent Done Done false
6288 2458 Acked Read 4 Sent Done Done false
6289 2459 Acked Flush 0 Sent Done Done false
6290 2460 Acked Read 3 Sent Done Done false
6291 2461 Acked Flush 0 Sent Done Done false
6292 2462 Acked Write 5 Sent Done Done false
6293 2463 Acked Write 10 Sent Done Done false
6294 2464 Acked Write 2 Sent Done Done false
6295 2465 Acked Read 2 Sent Done Done false
6296 2466 Acked Write 2 Sent Done Done false
6297 2467 Acked Write 4 Sent Done Done false
6298 2468 Acked Write 2 Sent Done Done false
6299 2469 Acked Read 3 Sent Done Done false
6300 2470 Acked Write 9 Sent Done Done false
6301 2471 Acked Write 4 Sent Done Done false
6302 2472 Acked Read 1 Sent Done Done false
6303 2473 Acked Write 3 Sent Done Done false
6304 2474 Acked Read 4 Sent Done Done false
6305 2475 Acked Flush 0 Sent Done Done false
6306 2476 Acked Flush 0 Sent Done Done false
6307 2477 Acked Read 7 Sent Done Done false
6308 2478 Acked Read 6 Sent Done Done false
6309 2479 Acked Flush 0 Sent Done Done false
6310 2480 Acked Write 4 Sent Done Done false
6311 2481 Acked Read 1 Sent Done Done false
6312 2482 Acked Read 10 Sent Done Done false
6313 2483 Acked Write 6 Sent Done Done false
6314 2484 Acked Read 8 Sent Done Done false
6315 2485 Acked Read 2 Sent Done Done false
6316 2486 Acked Read 6 Sent Done Done false
6317 2487 Acked Flush 0 Sent Done Done false
6318 2488 Acked Read 2 Sent Done Done false
6319 2489 Acked Read 2 Sent Done Done false
6320 2490 Acked Read 2 Sent Done Done false
6321 2491 Acked Write 1 Sent Done Done false
6322 2492 Acked Write 4 Sent Done Done false
6323 2493 Acked Write 8 Sent Done Done false
6324 2494 Acked Read 8 Sent Done Done false
6325 2495 Acked Write 7 Sent Done Done false
6326 2496 Acked Read 1 Sent Done Done false
6327 2497 Acked Read 7 Sent Done Done false
6328 2498 Acked Read 2 Sent Done Done false
6329 2499 Acked Read 2 Sent Done Done false
6330 2500 Acked Write 10 Sent Done Done false
6331 2501 Acked Write 7 Sent Done Done false
6332 2502 Acked Read 9 Sent Done Done false
6333 2503 Acked Write 8 Sent Done Done false
6334 2504 Acked Write 4 Sent Done Done false
6335 2505 Acked Write 7 Sent Done Done false
6336 2506 Acked Write 2 Sent Done Done false
6337 2507 Acked Write 3 Sent Done Done false
6338 2508 Acked Flush 0 Sent Done Done false
6339 STATES DS:0 DS:1 DS:2 TOTAL
6340 Sent 301 0 0 301
6341 Done 0 301 301 602
6342 Skipped 0 0 0 0
6343 Error 0 0 0 0
6344 Last Flush: 2207 2508 2508
6345 Downstairs last five completed: 2207 2206 2205 2204 2203
6346 Upstairs last five completed: 2508 2507 2506 2505 2504
6347 CLIENT: Up:0 ds:301 act:2
63482025-01-03T08:32:14.538ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63492025-01-03T08:32:14.538ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47847) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63502025-01-03T08:32:14.539ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63512025-01-03T08:32:14.539ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63522025-01-03T08:32:14.539ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63532025-01-03T08:32:14.539ZINFOcrucible: send last flush ID to this DS: 2207 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63542025-01-03T08:32:14.540ZINFOcrucible: Replied this last flush ID: 2207 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63552025-01-03T08:32:14.540ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63562025-01-03T08:32:14.540ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63572025-01-03T08:32:14.540ZINFOcrucible: [0] client re-new 301 jobs since flush 2207 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63582025-01-03T08:32:14.540ZINFOcrucible: [0] Replayed 301 jobs since flush: 2207 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
6359 ----------------------------------------------------------------
6360 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6361 Downstairs last five completed: 2508 2507 2506 2505 2504
6362 Upstairs last five completed: 2508 2507 2506 2505 2504
6363 CLIENT: Up:0 ds:0 act:3
6364 Replay: All jobs finished
6365 Main task has work to do, go find it
6366 got dsc Stop(1)
6367 stop 1
6368 Jan 03 08:32:14.800 INFO request completed, latency_us: 362, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 562e9685-23bf-46ef-a64b-5b03207f5508, remote_addr: 127.0.0.1:49669, local_addr: 127.0[8820] Got stop action so:false kr:false
6369 .0.1:9998
63702025-01-03T08:32:14.800ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63712025-01-03T08:32:14.801ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63722025-01-03T08:32:14.801ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63732025-01-03T08:32:14.801ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
63742025-01-03T08:32:14.801ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
6375 Jan 03 08:32:14.801 INFO request completed, latency_us: 241, response_code: 200, uri: /state/cid/1, method: GET, req_id: cd9c384d-c37c-4aab-b055-03e0a6ae469a, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
63762025-01-03T08:32:14.801ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
6377 [8820] Exited with: ExitStatus(unix_wait_status(9))
6378 [8820][1] reports Exit
6379 Jan 03 08:32:18.803 INFO request completed, latency_us: 207, response_code: 200, uri: /state/cid/1, method: GET, req_id: cd1c0aad-4244-476f-b621-35dc961cedc3, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
6380 001/300 Read block 1447 len 16384
6381 002/300 Read block 3445 len 16384
6382 003/300 Read block 34 len 20480
6383 004/300 Read block 3789 len 40960
6384 005/300 Read block 2506 len 16384
6385 006/300 Read block 4127 len 20480
6386 007/300 Write block 1487 len 16384 data: 4 4 4 4
6387 008/300 Write block 1838 len 28672 data: 2 3 3 4 5 4 4
6388 009/300 Write block 3476 len 16384 data: 3 3 3 3
6389 010/300 Flush
6390 011/300 Read block 4946 len 20480
6391 012/300 Write block 4213 len 8192 data: 3 3
6392 013/300 Write block 704 len 4096 data: 2
6393 014/300 Read block 4485 len 32768
6394 015/300 Write block 2559 len 28672 data: 2 2 2 2 3 3 2
6395 016/300 Write block 3715 len 20480 data: 2 3 3 3 4
6396 017/300 Write block 4073 len 20480 data: 2 2 2 2 2
6397 018/300 Read block 3280 len 4096
6398 019/300 Write block 2367 len 4096 data: 3
6399 020/300 Flush
6400 021/300 Read block 4897 len 32768
6401 022/300 Read block 606 len 20480
6402 023/300 Write block 4342 len 8192 data: 3 2
6403 024/300 Read block 2228 len 20480
6404 025/300 Read block 1541 len 16384
6405 026/300 Write block 3871 len 20480 data: 3 4 3 3 2
6406 027/300 Write block 3343 len 20480 data: 2 2 2 2 3
6407 028/300 Read block 3762 len 28672
6408 029/300 Read block 1150 len 8192
6409 030/300 Read block 1282 len 4096
6410 031/300 Read block 3660 len 12288
6411 032/300 Write block 4412 len 20480 data: 3 3 2 2 2
6412 033/300 Read block 439 len 28672
6413 034/300 Write block 1433 len 4096 data: 2
6414 035/300 Read block 4755 len 24576
6415 036/300 Write block 1948 len 12288 data: 3 3 3
6416 037/300 Write block 1221 len 28672 data: 3 2 2 3 3 3 3
6417 038/300 Write block 1535 len 16384 data: 3 3 4 3
6418 039/300 Flush
6419 040/300 Write block 345 len 32768 data: 3 3 3 3 3 3 3 3
6420 041/300 Read block 2571 len 24576
6421 042/300 Read block 992 len 40960
6422 043/300 Read block 2267 len 8192
6423 044/300 Write block 21 len 16384 data: 3 3 3 3
6424 045/300 Read block 2843 len 12288
6425 046/300 Read block 4990 len 24576
6426 047/300 Write block 406 len 32768 data: 3 3 3 3 3 2 2 3
6427 048/300 Read block 530 len 24576
6428 049/300 Write block 2360 len 28672 data: 2 3 3 3 3 3 3
6429 050/300 Write block 2807 len 36864 data: 4 4 4 4 4 4 4 4 2
6430 051/300 Write block 1648 len 24576 data: 3 3 3 4 3 3
6431 052/300 Read block 544 len 8192
6432 053/300 Write block 215 len 4096 data: 2
6433 054/300 Read block 4000 len 24576
6434 055/300 Read block 689 len 40960
6435 056/300 Read block 3810 len 28672
6436 057/300 Flush
6437 058/300 Write block 3400 len 32768 data: 3 3 3 3 2 2 2 3
6438 059/300 Read block 3865 len 28672
6439 060/300 Write block 2568 len 40960 data: 3 2 2 2 2 2 2 2 2 2
6440 061/300 Flush
6441 062/300 Write block 2130 len 20480 data: 3 3 3 2 2
6442 063/300 Read block 1570 len 12288
6443 064/300 Read block 511 len 4096
6444 065/300 Write block 619 len 36864 data: 3 2 2 2 2 2 2 2 3
6445 066/300 Flush
6446 067/300 Write block 3772 len 16384 data: 2 2 2 2
6447 068/300 Write block 4392 len 16384 data: 4 3 3 2
6448 069/300 Read block 1375 len 40960
6449 070/300 Read block 2756 len 20480
6450 071/300 Read block 3355 len 32768
6451 072/300 Read block 4666 len 28672
6452 073/300 Read block 4678 len 28672
6453 074/300 Write block 4952 len 20480 data: 2 2 2 2 2
6454 075/300 Write block 53 len 8192 data: 3 3
6455 076/300 Read block 154 len 36864
6456 077/300 Read block 448 len 32768
6457 078/300 Flush
6458 079/300 Write block 4355 len 12288 data: 3 3 3
6459 080/300 Read block 1359 len 8192
6460 081/300 Flush
6461 082/300 Write block 4102 len 12288 data: 2 2 2
6462 083/300 Write block 995 len 16384 data: 2 2 2 2
6463 084/300 Flush
6464 085/300 Read block 3266 len 8192
6465 086/300 Flush
6466 087/300 Read block 2418 len 28672
6467 088/300 Read block 3269 len 20480
6468 089/300 Write block 66 len 8192 data: 4 4
6469 090/300 Write block 2568 len 20480 data: 4 3 3 3 3
6470 091/300 Write block 4869 len 20480 data: 3 3 3 3 2
6471 092/300 Read block 521 len 32768
6472 093/300 Flush
6473 094/300 Write block 1675 len 36864 data: 3 3 3 3 3 3 3 3 2
6474 095/300 Read block 4333 len 20480
6475 096/300 Write block 2809 len 40960 data: 5 5 5 5 5 5 3 3 3 3
6476 097/300 Read block 1749 len 32768
6477 098/300 Write block 4361 len 32768 data: 4 4 4 3 4 3 3 2
6478 099/300 Read block 2880 len 4096
6479 100/300 Read block 4692 len 32768
6480 101/300 Read block 4431 len 20480
6481 102/300 Flush
6482 103/300 Write block 1618 len 20480 data: 2 2 2 2 2
6483 104/300 Flush
6484 105/300 Write block 435 len 20480 data: 2 2 2 2 2
6485 106/300 Read block 4890 len 36864
6486 107/300 Flush
6487 108/300 Read block 4634 len 8192
6488 109/300 Write block 4680 len 24576 data: 2 2 2 2 2 2
6489 110/300 Read block 4369 len 40960
6490 111/300 Read block 3427 len 12288
6491 112/300 Read block 2165 len 32768
6492 113/300 Read block 4685 len 24576
6493 114/300 Write block 147 len 16384 data: 2 2 2 2
6494 115/300 Write block 3379 len 20480 data: 2 2 2 2 2
6495 116/300 Write block 3994 len 8192 data: 2 2
6496 117/300 Read block 2246 len 36864
6497 118/300 Write block 1091 len 28672 data: 4 4 5 5 5 4 4
6498 119/300 Read block 2798 len 12288
6499 120/300 Read block 1114 len 32768
6500 121/300 Read block 1314 len 8192
6501 122/300 Write block 2370 len 40960 data: 3 3 3 3 3 2 2 2 2 2
6502 123/300 Write block 3635 len 20480 data: 2 2 2 2 2
6503 124/300 Read block 2838 len 4096
6504 125/300 Read block 114 len 16384
6505 126/300 Read block 1789 len 36864
6506 127/300 Read block 3336 len 24576
6507 128/300 Write block 1365 len 4096 data: 3
6508 129/300 Read block 4111 len 28672
6509 130/300 Write block 886 len 12288 data: 2 2 2
6510 131/300 Read block 3072 len 12288
6511 132/300 Read block 775 len 36864
6512 133/300 Write block 1191 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6513 134/300 Read block 1192 len 16384
6514 135/300 Write block 2250 len 36864 data: 3 3 2 2 2 2 2 2 2
6515 136/300 Read block 4772 len 36864
6516 137/300 Write block 3447 len 4096 data: 2
6517 138/300 Read block 617 len 16384
6518 139/300 Write block 3263 len 12288 data: 3 3 3
6519 140/300 Read block 3570 len 24576
6520 141/300 Flush
6521 142/300 Write block 3743 len 28672 data: 3 2 2 2 2 2 2
6522 143/300 Read block 1002 len 36864
6523 144/300 Read block 2377 len 20480
6524 145/300 Flush
6525 146/300 Read block 3270 len 4096
6526 147/300 Read block 4399 len 4096
6527 148/300 Read block 3809 len 24576
6528 149/300 Read block 659 len 36864
6529 150/300 Read block 4063 len 24576
6530 151/300 Flush
6531 152/300 Read block 1002 len 24576
6532 153/300 Read block 1863 len 8192
6533 154/300 Read block 2391 len 24576
6534 155/300 Read block 1027 len 24576
6535 156/300 Read block 4049 len 16384
6536 157/300 Read block 2313 len 24576
6537 158/300 Write block 3544 len 12288 data: 4 4 4
6538 159/300 Write block 4824 len 4096 data: 3
6539 160/300 Read block 4029 len 12288
6540 161/300 Read block 4994 len 24576
6541 162/300 Read block 1380 len 4096
6542 163/300 Write block 1190 len 24576 data: 3 3 3 3 3 3
6543 164/300 Read block 4901 len 28672
6544 165/300 Read block 255 len 32768
6545 166/300 Write block 4929 len 8192 data: 3 3
6546 167/300 Read block 3524 len 40960
6547 168/300 Write block 4705 len 36864 data: 4 4 5 4 4 4 4 2 2
6548 169/300 Read block 4228 len 24576
6549 170/300 Write block 4096 len 4096 data: 2
6550 171/300 Flush
6551 172/300 Read block 2487 len 40960
6552 173/300 Read block 2670 len 40960
6553 174/300 Write block 3202 len 8192 data: 3 3
6554 175/300 Write block 708 len 16384 data: 3 5 5 5
6555 176/300 Write block 4534 len 8192 data: 3 3
6556 177/300 Write block 2517 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6557 178/300 Read block 1157 len 24576
6558 179/300 Read block 3907 len 40960
6559 180/300 Read block 2125 len 40960
6560 181/300 Read block 3876 len 12288
6561 182/300 Read block 3860 len 4096
6562 183/300 Write block 4222 len 32768 data: 2 2 2 2 2 2 2 2
6563 184/300 Flush
6564 185/300 Write block 137 len 12288 data: 3 3 3
6565 186/300 Write block 4965 len 32768 data: 2 2 2 2 2 2 2 3
6566 187/300 Write block 217 len 8192 data: 2 3
6567 188/300 Read block 2500 len 24576
6568 189/300 Write block 501 len 4096 data: 3
6569 190/300 Write block 2840 len 24576 data: 2 2 2 2 2 2
6570 191/300 Write block 550 len 24576 data: 2 2 2 2 2 2
6571 192/300 Write block 3886 len 28672 data: 3 3 3 2 2 2 2
6572 193/300 Read block 4309 len 24576
6573 194/300 Write block 2161 len 20480 data: 4 3 3 3 2
6574 195/300 Read block 4354 len 8192
6575 196/300 Read block 776 len 4096
6576 197/300 Write block 1919 len 4096 data: 3
6577 198/300 Write block 4874 len 20480 data: 2 2 2 2 3
6578 199/300 Write block 4708 len 20480 data: 5 5 5 5 3
6579 200/300 Write block 1753 len 8192 data: 4 4
6580 201/300 Flush
6581 202/300 Read block 297 len 32768
6582 203/300 Write block 3703 len 32768 data: 2 2 2 2 2 2 2 2
6583 204/300 Read block 3282 len 16384
6584 205/300 Write block 3664 len 20480 data: 3 2 3 3 3
6585 206/300 Write block 4992 len 24576 data: 2 2 2 2 2 2
6586 207/300 Read block 2306 len 28672
6587 208/300 Write block 1427 len 4096 data: 2
6588 209/300 Read block 1499 len 40960
6589 210/300 Write block 3641 len 36864 data: 2 2 2 4 3 3 3 3 3
6590 211/300 Write block 4311 len 16384 data: 3 4 3 3
6591 212/300 Read block 2300 len 36864
6592 213/300 Read block 3279 len 20480
6593 214/300 Read block 3369 len 12288
6594 215/300 Flush
6595 216/300 Write block 3053 len 8192 data: 2 3
6596 217/300 Write block 4495 len 20480 data: 4 4 4 4 4
6597 218/300 Read block 517 len 12288
6598 219/300 Read block 4331 len 12288
6599 220/300 Write block 1982 len 4096 data: 2
6600 221/300 Write block 4000 len 20480 data: 3 4 3 3 3
6601 222/300 Write block 936 len 12288 data: 3 3 3
6602 223/300 Read block 4673 len 4096
6603 224/300 Read block 975 len 28672
6604 225/300 Read block 3695 len 36864
6605 226/300 Write block 4878 len 24576 data: 4 3 3 4 4 3
6606 227/300 Write block 3245 len 24576 data: 2 2 2 2 2 2
6607 228/300 Write block 2782 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6608 229/300 Read block 4200 len 16384
6609 230/300 Write block 2370 len 8192 data: 4 4
6610 231/300 Write block 3559 len 16384 data: 2 2 2 2
6611 232/300 Read block 3824 len 16384
6612 233/300 Read block 713 len 28672
6613 234/300 Read block 1340 len 40960
6614 235/300 Flush
6615 236/300 Read block 2786 len 28672
6616 237/300 Flush
6617 238/300 Read block 2773 len 8192
6618 239/300 Read block 2754 len 40960
6619 240/300 Read block 4213 len 36864
6620 241/300 Read block 3610 len 40960
6621 242/300 Read block 4284 len 36864
6622 243/300 Flush
6623 244/300 Read block 2932 len 8192
6624 245/300 Read block 1290 len 4096
6625 246/300 Read block 3424 len 20480
6626 247/300 Write block 3412 len 24576 data: 2 2 2 2 2 2
6627 248/300 Read block 4501 len 16384
6628 249/300 Flush
6629 250/300 Read block 980 len 28672
6630 251/300 Read block 854 len 36864
6631 252/300 Write block 3270 len 32768 data: 3 3 2 2 2 2 2 3
6632 253/300 Read block 577 len 40960
6633 254/300 Read block 4410 len 4096
6634 255/300 Write block 4116 len 36864 data: 3 3 3 3 2 3 3 3 3
6635 256/300 Read block 2118 len 20480
6636 257/300 Read block 1267 len 12288
6637 258/300 Write block 2687 len 36864 data: 3 3 3 3 3 3 3 2 3
6638 259/300 Write block 4495 len 16384 data: 5 5 5 5
6639 260/300 Read block 2351 len 24576
6640 261/300 Read block 4166 len 32768
6641 262/300 Write block 3819 len 8192 data: 3 3
6642 263/300 Write block 3039 len 40960 data: 2 2 2 3 3 3 3 3 3 3
6643 264/300 Read block 360 len 32768
6644 265/300 Read block 4568 len 32768
6645 266/300 Read block 331 len 28672
6646 267/300 Read block 4669 len 36864
6647 268/300 Write block 4814 len 36864 data: 2 2 2 2 2 2 2 4 4
6648 269/300 Flush
6649 270/300 Read block 1509 len 40960
6650 271/300 Read block 948 len 4096
6651 272/300 Write block 1824 len 28672 data: 2 2 2 2 2 2 2
6652 273/300 Read block 1807 len 32768
6653 274/300 Read block 2167 len 24576
6654 275/300 Read block 3598 len 28672
6655 276/300 Write block 4246 len 40960 data: 2 2 2 2 2 3 4 4 4 5
6656 277/300 Flush
6657 278/300 Write block 3140 len 8192 data: 2 2
6658 279/300 Write block 2927 len 8192 data: 2 2
6659 280/300 Read block 1235 len 16384
6660 281/300 Write block 1130 len 4096 data: 2
6661 282/300 Read block 1637 len 20480
6662 283/300 Read block 4399 len 12288
6663 284/300 Write block 3882 len 8192 data: 3 3
6664 285/300 Read block 3733 len 20480
6665 286/300 Write block 4588 len 12288 data: 2 2 3
6666 287/300 Write block 1054 len 16384 data: 3 3 3 3
6667 288/300 Write block 2766 len 8192 data: 3 2
6668 289/300 Write block 2482 len 28672 data: 3 3 3 4 4 4 4
6669 290/300 Read block 1378 len 20480
6670 291/300 Write block 4319 len 32768 data: 2 2 2 2 3 3 2 2
6671 292/300 Read block 2012 len 36864
6672 293/300 Write block 2831 len 16384 data: 2 2 3 3
6673 294/300 Write block 4497 len 12288 data: 6 6 5
6674 295/300 Write block 457 len 28672 data: 2 3 3 3 3 3 4
6675 296/300 Write block 2839 len 12288 data: 2 3 3
6676 297/300 Read block 3493 len 28672
6677 298/300 Read block 564 len 16384
6678 299/300 Flush
6679 300/300 Read block 3624 len 12288
6680 301/300 Read block 1095 len 8192
6681 Main task has work to do, go find it
6682 got dsc Start(1)
6683 start 1
6684 Jan 03 08:32:19.045 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/1, method: POST, req_id: 4a99a40c-9783-46ea-945e-76a9dd485b53, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
6685 Replay: started 1, returned:Ok(())
6686 ----------------------------------------------------------------
6687 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6688 States: Active Offline Active
6689 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6690 2509 Acked Read 4 Done Sent Done false
6691 2510 Acked Read 4 Done Sent Done false
6692 2511 Acked Read 5 Done Sent Done false
6693 2512 Acked Read 10 Done Sent Done false
6694 2513 Acked Read 4 Done Sent Done false
6695 2514 Acked Read 5 Done Sent Done false
6696 2515 Acked Write 4 Done Sent Done false
6697 2516 Acked Write 7 Done Sent Done false
6698 2517 Acked Write 4 Done Sent Done false
6699 2518 Acked Flush 0 Done Sent Done false
6700 2519 Acked Read 5 Done Sent Done false
6701 2520 Acked Write 2 Done Sent Done false
6702 2521 Acked Write 1 Done Sent Done false
6703 2522 Acked Read 8 Done Sent Done false
6704 2523 Acked Write 7 Done Sent Done false
6705 2524 Acked Write 5 Done Sent Done false
6706 2525 Acked Write 5 Done Sent Done false
6707 2526 Acked Read 1 Done Sent Done false
6708 2527 Acked Write 1 Done Sent Done false
6709 2528 Acked Flush 0 Done Sent Done false
6710 2529 Acked Read 8 Done Sent Done false
6711 2530 Acked Read 5 Done Sent Done false
6712 2531 Acked Write 2 Done Sent Done false
6713 2532 Acked Read 5 Done Sent Done false
6714 2533 Acked Read 4 Done Sent Done false
6715 2534 Acked Write 5 Done Sent Done false
6716 2535 Acked Write 5 Done Sent Done false
6717 2536 Acked Read 7 Done Sent Done false
6718 2537 Acked Read 2 Done Sent Done false
6719 2538 Acked Read 1 Done Sent Done false
6720 2539 Acked Read 3 Done Sent Done false
6721 2540 Acked Write 5 Done Sent Done false
6722 2541 Acked Read 7 Done Sent Done false
6723 2542 Acked Write 1 Done Sent Done false
6724 2543 Acked Read 6 Done Sent Done false
6725 2544 Acked Write 3 Done Sent Done false
6726 2545 Acked Write 7 Done Sent Done false
6727 2546 Acked Write 4 Done Sent Done false
6728 2547 Acked Flush 0 Done Sent Done false
6729 2548 Acked Write 8 Done Sent Done false
6730 2549 Acked Read 6 Done Sent Done false
6731 2550 Acked Read 10 Done Sent Done false
6732 2551 Acked Read 2 Done Sent Done false
6733 2552 Acked Write 4 Done Sent Done false
6734 2553 Acked Read 3 Done Sent Done false
6735 2554 Acked Read 6 Done Sent Done false
6736 2555 Acked Write 8 Done Sent Done false
6737 2556 Acked Read 6 Done Sent Done false
6738 2557 Acked Write 7 Done Sent Done false
6739 2558 Acked Write 9 Done Sent Done false
6740 2559 Acked Write 6 Done Sent Done false
6741 2560 Acked Read 2 Done Sent Done false
6742 2561 Acked Write 1 Done Sent Done false
6743 2562 Acked Read 6 Done Sent Done false
6744 2563 Acked Read 10 Done Sent Done false
6745 2564 Acked Read 7 Done Sent Done false
6746 2565 Acked Flush 0 Done Sent Done false
6747 2566 Acked Write 8 Done Sent Done false
6748 2567 Acked Read 7 Done Sent Done false
6749 2568 Acked Write 10 Done Sent Done false
6750 2569 Acked Flush 0 Done Sent Done false
6751 2570 Acked Write 5 Done Sent Done false
6752 2571 Acked Read 3 Done Sent Done false
6753 2572 Acked Read 1 Done Sent Done false
6754 2573 Acked Write 9 Done Sent Done false
6755 2574 Acked Flush 0 Done Sent Done false
6756 2575 Acked Write 4 Done Sent Done false
6757 2576 Acked Write 4 Done Sent Done false
6758 2577 Acked Read 10 Done Sent Done false
6759 2578 Acked Read 5 Done Sent Done false
6760 2579 Acked Read 8 Done Sent Done false
6761 2580 Acked Read 7 Done Sent Done false
6762 2581 Acked Read 7 Done Sent Done false
6763 2582 Acked Write 5 Done Sent Done false
6764 2583 Acked Write 2 Done Sent Done false
6765 2584 Acked Read 9 Done Sent Done false
6766 2585 Acked Read 8 Done Sent Done false
6767 2586 Acked Flush 0 Done Sent Done false
6768 2587 Acked Write 3 Done Sent Done false
6769 2588 Acked Read 2 Done Sent Done false
6770 2589 Acked Flush 0 Done Sent Done false
6771 2590 Acked Write 3 Done Sent Done false
6772 2591 Acked Write 4 Done Sent Done false
6773 2592 Acked Flush 0 Done Sent Done false
6774 2593 Acked Read 2 Done Sent Done false
6775 2594 Acked Flush 0 Done Sent Done false
6776 2595 Acked Read 7 Done Sent Done false
6777 2596 Acked Read 5 Done Sent Done false
6778 2597 Acked Write 2 Done Sent Done false
6779 2598 Acked Write 5 Done Sent Done false
6780 2599 Acked Write 5 Done Sent Done false
6781 2600 Acked Read 8 Done Sent Done false
6782 2601 Acked Flush 0 Done Sent Done false
6783 2602 Acked Write 9 Done Sent Done false
6784 2603 Acked Read 5 Done Sent Done false
6785 2604 Acked Write 10 Done Sent Done false
6786 2605 Acked Read 8 Done Sent Done false
6787 2606 Acked Write 8 Done Sent Done false
6788 2607 Acked Read 1 Done Sent Done false
6789 2608 Acked Read 8 Done Sent Done false
6790 2609 Acked Read 5 Done Sent Done false
6791 2610 Acked Flush 0 Done Sent Done false
6792 2611 Acked Write 5 Done Sent Done false
6793 2612 Acked Flush 0 Done Sent Done false
6794 2613 Acked Write 5 Done Sent Done false
6795 2614 Acked Read 9 Done Sent Done false
6796 2615 Acked Flush 0 Done Sent Done false
6797 2616 Acked Read 2 Done Sent Done false
6798 2617 Acked Write 6 Done Sent Done false
6799 2618 Acked Read 10 Done Sent Done false
6800 2619 Acked Read 3 Done Sent Done false
6801 2620 Acked Read 8 Done Sent Done false
6802 2621 Acked Read 6 Done Sent Done false
6803 2622 Acked Write 4 Done Sent Done false
6804 2623 Acked Write 5 Done Sent Done false
6805 2624 Acked Write 2 Done Sent Done false
6806 2625 Acked Read 9 Done Sent Done false
6807 2626 Acked Write 7 Done Sent Done false
6808 2627 Acked Read 3 Done Sent Done false
6809 2628 Acked Read 8 Done Sent Done false
6810 2629 Acked Read 2 Done Sent Done false
6811 2630 Acked Write 10 Done Sent Done false
6812 2631 Acked Write 5 Done Sent Done false
6813 2632 Acked Read 1 Done Sent Done false
6814 2633 Acked Read 4 Done Sent Done false
6815 2634 Acked Read 9 Done Sent Done false
6816 2635 Acked Read 6 Done Sent Done false
6817 2636 Acked Write 1 Done Sent Done false
6818 2637 Acked Read 7 Done Sent Done false
6819 2638 Acked Write 3 Done Sent Done false
6820 2639 Acked Read 3 Done Sent Done false
6821 2640 Acked Read 9 Done Sent Done false
6822 2641 Acked Write 10 Done Sent Done false
6823 2642 Acked Read 4 Done Sent Done false
6824 2643 Acked Write 9 Done Sent Done false
6825 2644 Acked Read 9 Done Sent Done false
6826 2645 Acked Write 1 Done Sent Done false
6827 2646 Acked Read 4 Done Sent Done false
6828 2647 Acked Write 3 Done Sent Done false
6829 2648 Acked Read 6 Done Sent Done false
6830 2649 Acked Flush 0 Done Sent Done false
6831 2650 Acked Write 7 Done Sent Done false
6832 2651 Acked Read 9 Done Sent Done false
6833 2652 Acked Read 5 Done Sent Done false
6834 2653 Acked Flush 0 Done Sent Done false
6835 2654 Acked Read 1 Done Sent Done false
6836 2655 Acked Read 1 Done Sent Done false
6837 2656 Acked Read 6 Done Sent Done false
6838 2657 Acked Read 9 Done Sent Done false
6839 2658 Acked Read 6 Done Sent Done false
6840 2659 Acked Flush 0 Done Sent Done false
6841 2660 Acked Read 6 Done Sent Done false
6842 2661 Acked Read 2 Done Sent Done false
6843 2662 Acked Read 6 Done Sent Done false
6844 2663 Acked Read 6 Done Sent Done false
6845 2664 Acked Read 4 Done Sent Done false
6846 2665 Acked Read 6 Done Sent Done false
6847 2666 Acked Write 3 Done Sent Done false
6848 2667 Acked Write 1 Done Sent Done false
6849 2668 Acked Read 3 Done Sent Done false
6850 2669 Acked Read 6 Done Sent Done false
6851 2670 Acked Read 1 Done Sent Done false
6852 2671 Acked Write 6 Done Sent Done false
6853 2672 Acked Read 7 Done Sent Done false
6854 2673 Acked Read 8 Done Sent Done false
6855 2674 Acked Write 2 Done Sent Done false
6856 2675 Acked Read 10 Done Sent Done false
6857 2676 Acked Write 9 Done Sent Done false
6858 2677 Acked Read 6 Done Sent Done false
6859 2678 Acked Write 1 Done Sent Done false
6860 2679 Acked Flush 0 Done Sent Done false
6861 2680 Acked Read 10 Done Sent Done false
6862 2681 Acked Read 10 Done Sent Done false
6863 2682 Acked Write 2 Done Sent Done false
6864 2683 Acked Write 4 Done Sent Done false
6865 2684 Acked Write 2 Done Sent Done false
6866 2685 Acked Write 10 Done Sent Done false
6867 2686 Acked Read 6 Done Sent Done false
6868 2687 Acked Read 10 Done Sent Done false
6869 2688 Acked Read 10 Done Sent Done false
6870 2689 Acked Read 3 Done Sent Done false
6871 2690 Acked Read 1 Done Sent Done false
6872 2691 Acked Write 8 Done Sent Done false
6873 2692 Acked Flush 0 Done Sent Done false
6874 2693 Acked Write 3 Done Sent Done false
6875 2694 Acked Write 8 Done Sent Done false
6876 2695 Acked Write 2 Done Sent Done false
6877 2696 Acked Read 6 Done Sent Done false
6878 2697 Acked Write 1 Done Sent Done false
6879 2698 Acked Write 6 Done Sent Done false
6880 2699 Acked Write 6 Done Sent Done false
6881 2700 Acked Write 7 Done Sent Done false
6882 2701 Acked Read 6 Done Sent Done false
6883 2702 Acked Write 5 Done Sent Done false
6884 2703 Acked Read 2 Done Sent Done false
6885 2704 Acked Read 1 Done Sent Done false
6886 2705 Acked Write 1 Done Sent Done false
6887 2706 Acked Write 5 Done Sent Done false
6888 2707 Acked Write 5 Done Sent Done false
6889 2708 Acked Write 2 Done Sent Done false
6890 2709 Acked Flush 0 Done Sent Done false
6891 2710 Acked Read 8 Done Sent Done false
6892 2711 Acked Write 8 Done Sent Done false
6893 2712 Acked Read 4 Done Sent Done false
6894 2713 Acked Write 5 Done Sent Done false
6895 2714 Acked Write 6 Done Sent Done false
6896 2715 Acked Read 7 Done Sent Done false
6897 2716 Acked Write 1 Done Sent Done false
6898 2717 Acked Read 10 Done Sent Done false
6899 2718 Acked Write 9 Done Sent Done false
6900 2719 Acked Write 4 Done Sent Done false
6901 2720 Acked Read 9 Done Sent Done false
6902 2721 Acked Read 5 Done Sent Done false
6903 2722 Acked Read 3 Done Sent Done false
6904 2723 Acked Flush 0 Done Sent Done false
6905 2724 Acked Write 2 Done Sent Done false
6906 2725 Acked Write 5 Done Sent Done false
6907 2726 Acked Read 3 Done Sent Done false
6908 2727 Acked Read 3 Done Sent Done false
6909 2728 Acked Write 1 Done Sent Done false
6910 2729 Acked Write 5 Done Sent Done false
6911 2730 Acked Write 3 Done Sent Done false
6912 2731 Acked Read 1 Done Sent Done false
6913 2732 Acked Read 7 Done Sent Done false
6914 2733 Acked Read 9 Done Sent Done false
6915 2734 Acked Write 6 Done Sent Done false
6916 2735 Acked Write 6 Done Sent Done false
6917 2736 Acked Write 10 Done Sent Done false
6918 2737 Acked Read 4 Done Sent Done false
6919 2738 Acked Write 2 Done Sent Done false
6920 2739 Acked Write 4 Done Sent Done false
6921 2740 Acked Read 4 Done Sent Done false
6922 2741 Acked Read 7 Done Sent Done false
6923 2742 Acked Read 10 Done Sent Done false
6924 2743 Acked Flush 0 Done Sent Done false
6925 2744 Acked Read 7 Done Sent Done false
6926 2745 Acked Flush 0 Done Sent Done false
6927 2746 Acked Read 2 Done Sent Done false
6928 2747 Acked Read 10 Done Sent Done false
6929 2748 Acked Read 9 Done Sent Done false
6930 2749 Acked Read 10 Done Sent Done false
6931 2750 Acked Read 9 Done Sent Done false
6932 2751 Acked Flush 0 Done Sent Done false
6933 2752 Acked Read 2 Done Sent Done false
6934 2753 Acked Read 1 Done Sent Done false
6935 2754 Acked Read 5 Done Sent Done false
6936 2755 Acked Write 6 Done Sent Done false
6937 2756 Acked Read 4 Done Sent Done false
6938 2757 Acked Flush 0 Done Sent Done false
6939 2758 Acked Read 7 Done Sent Done false
6940 2759 Acked Read 9 Done Sent Done false
6941 2760 Acked Write 8 Done Sent Done false
6942 2761 Acked Read 10 Done Sent Done false
6943 2762 Acked Read 1 Done Sent Done false
6944 2763 Acked Write 9 Done Sent Done false
6945 2764 Acked Read 5 Done Sent Done false
6946 2765 Acked Read 3 Done Sent Done false
6947 2766 Acked Write 9 Done Sent Done false
6948 2767 Acked Write 4 Done Sent Done false
6949 2768 Acked Read 6 Done Sent Done false
6950 2769 Acked Read 8 Done Sent Done false
6951 2770 Acked Write 2 Done Sent Done false
6952 2771 Acked Write 10 Done Sent Done false
6953 2772 Acked Read 8 Done Sent Done false
6954 2773 Acked Read 8 Done Sent Done false
6955 2774 Acked Read 7 Done Sent Done false
6956 2775 Acked Read 9 Done Sent Done false
6957 2776 Acked Write 9 Done Sent Done false
6958 2777 Acked Flush 0 Done Sent Done false
6959 2778 Acked Read 10 Done Sent Done false
6960 2779 Acked Read 1 Done Sent Done false
6961 2780 Acked Write 7 Done Sent Done false
6962 2781 Acked Read 8 Done Sent Done false
6963 2782 Acked Read 6 Done Sent Done false
6964 2783 Acked Read 7 Done Sent Done false
6965 2784 Acked Write 10 Done Sent Done false
6966 2785 Acked Flush 0 Done Sent Done false
6967 2786 Acked Write 2 Done Sent Done false
6968 2787 Acked Write 2 Done Sent Done false
6969 2788 Acked Read 4 Done Sent Done false
6970 2789 Acked Write 1 Done Sent Done false
6971 2790 Acked Read 5 Done Sent Done false
6972 2791 Acked Read 3 Done Sent Done false
6973 2792 Acked Write 2 Done Sent Done false
6974 2793 Acked Read 5 Done Sent Done false
6975 2794 Acked Write 3 Done Sent Done false
6976 2795 Acked Write 4 Done Sent Done false
6977 2796 Acked Write 2 Done Sent Done false
6978 2797 Acked Write 7 Done Sent Done false
6979 2798 Acked Read 5 Done Sent Done false
6980 2799 Acked Write 8 Done Sent Done false
6981 2800 Acked Read 9 Done Sent Done false
6982 2801 Acked Write 4 Done Sent Done false
6983 2802 Acked Write 3 Done Sent Done false
6984 2803 Acked Write 7 Done Sent Done false
6985 2804 Acked Write 3 Done Sent Done false
6986 2805 Acked Read 7 Done Sent Done false
6987 2806 Acked Read 4 Done Sent Done false
6988 2807 Acked Flush 0 Done Sent Done false
6989 2808 Acked Read 3 Done Sent Done false
6990 2809 Acked Read 2 Done Sent Done false
6991 STATES DS:0 DS:1 DS:2 TOTAL
6992 Sent 0 301 0 301
6993 Done 301 0 301 602
6994 Skipped 0 0 0 0
6995 Error 0 0 0 0
6996 Last Flush: 2807 2508 2807
6997 Downstairs last five completed: 2508 2507 2506 2505 2504
6998 Upstairs last five completed: 2809 2808 2807 2806 2805
6999 CLIENT: Up:0 ds:301 act:2
7000 [8820] got start action
7001 [8820] I am going to restart
7002 Starting downstairs at port 8820
7003 Make output file at "/tmp/dsc/downstairs-8820.txt"
7004 [8820][1] reports Starting
7005 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1211)
7006 [8820][1] reports Running
7007 ----------------------------------------------------------------
7008 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7009 States: Active Offline Active
7010 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7011 2509 Acked Read 4 Done Sent Done false
7012 2510 Acked Read 4 Done Sent Done false
7013 2511 Acked Read 5 Done Sent Done false
7014 2512 Acked Read 10 Done Sent Done false
7015 2513 Acked Read 4 Done Sent Done false
7016 2514 Acked Read 5 Done Sent Done false
7017 2515 Acked Write 4 Done Sent Done false
7018 2516 Acked Write 7 Done Sent Done false
7019 2517 Acked Write 4 Done Sent Done false
7020 2518 Acked Flush 0 Done Sent Done false
7021 2519 Acked Read 5 Done Sent Done false
7022 2520 Acked Write 2 Done Sent Done false
7023 2521 Acked Write 1 Done Sent Done false
7024 2522 Acked Read 8 Done Sent Done false
7025 2523 Acked Write 7 Done Sent Done false
7026 2524 Acked Write 5 Done Sent Done false
7027 2525 Acked Write 5 Done Sent Done false
7028 2526 Acked Read 1 Done Sent Done false
7029 2527 Acked Write 1 Done Sent Done false
7030 2528 Acked Flush 0 Done Sent Done false
7031 2529 Acked Read 8 Done Sent Done false
7032 2530 Acked Read 5 Done Sent Done false
7033 2531 Acked Write 2 Done Sent Done false
7034 2532 Acked Read 5 Done Sent Done false
7035 2533 Acked Read 4 Done Sent Done false
7036 2534 Acked Write 5 Done Sent Done false
7037 2535 Acked Write 5 Done Sent Done false
7038 2536 Acked Read 7 Done Sent Done false
7039 2537 Acked Read 2 Done Sent Done false
7040 2538 Acked Read 1 Done Sent Done false
7041 2539 Acked Read 3 Done Sent Done false
7042 2540 Acked Write 5 Done Sent Done false
7043 2541 Acked Read 7 Done Sent Done false
7044 2542 Acked Write 1 Done Sent Done false
7045 2543 Acked Read 6 Done Sent Done false
7046 2544 Acked Write 3 Done Sent Done false
7047 2545 Acked Write 7 Done Sent Done false
7048 2546 Acked Write 4 Done Sent Done false
7049 2547 Acked Flush 0 Done Sent Done false
7050 2548 Acked Write 8 Done Sent Done false
7051 2549 Acked Read 6 Done Sent Done false
7052 2550 Acked Read 10 Done Sent Done false
7053 2551 Acked Read 2 Done Sent Done false
7054 2552 Acked Write 4 Done Sent Done false
7055 2553 Acked Read 3 Done Sent Done false
7056 2554 Acked Read 6 Done Sent Done false
7057 2555 Acked Write 8 Done Sent Done false
7058 2556 Acked Read 6 Done Sent Done false
7059 2557 Acked Write 7 Done Sent Done false
7060 2558 Acked Write 9 Done Sent Done false
7061 2559 Acked Write 6 Done Sent Done false
7062 2560 Acked Read 2 Done Sent Done false
7063 2561 Acked Write 1 Done Sent Done false
7064 2562 Acked Read 6 Done Sent Done false
7065 2563 Acked Read 10 Done Sent Done false
7066 2564 Acked Read 7 Done Sent Done false
7067 2565 Acked Flush 0 Done Sent Done false
7068 2566 Acked Write 8 Done Sent Done false
7069 2567 Acked Read 7 Done Sent Done false
7070 2568 Acked Write 10 Done Sent Done false
7071 2569 Acked Flush 0 Done Sent Done false
7072 2570 Acked Write 5 Done Sent Done false
7073 2571 Acked Read 3 Done Sent Done false
7074 2572 Acked Read 1 Done Sent Done false
7075 2573 Acked Write 9 Done Sent Done false
7076 2574 Acked Flush 0 Done Sent Done false
7077 2575 Acked Write 4 Done Sent Done false
7078 2576 Acked Write 4 Done Sent Done false
7079 2577 Acked Read 10 Done Sent Done false
7080 2578 Acked Read 5 Done Sent Done false
7081 2579 Acked Read 8 Done Sent Done false
7082 2580 Acked Read 7 Done Sent Done false
7083 2581 Acked Read 7 Done Sent Done false
7084 2582 Acked Write 5 Done Sent Done false
7085 2583 Acked Write 2 Done Sent Done false
7086 2584 Acked Read 9 Done Sent Done false
7087 2585 Acked Read 8 Done Sent Done false
7088 2586 Acked Flush 0 Done Sent Done false
7089 2587 Acked Write 3 Done Sent Done false
7090 2588 Acked Read 2 Done Sent Done false
7091 2589 Acked Flush 0 Done Sent Done false
7092 2590 Acked Write 3 Done Sent Done false
7093 2591 Acked Write 4 Done Sent Done false
7094 2592 Acked Flush 0 Done Sent Done false
7095 2593 Acked Read 2 Done Sent Done false
7096 2594 Acked Flush 0 Done Sent Done false
7097 2595 Acked Read 7 Done Sent Done false
7098 2596 Acked Read 5 Done Sent Done false
7099 2597 Acked Write 2 Done Sent Done false
7100 2598 Acked Write 5 Done Sent Done false
7101 2599 Acked Write 5 Done Sent Done false
7102 2600 Acked Read 8 Done Sent Done false
7103 2601 Acked Flush 0 Done Sent Done false
7104 2602 Acked Write 9 Done Sent Done false
7105 2603 Acked Read 5 Done Sent Done false
7106 2604 Acked Write 10 Done Sent Done false
7107 2605 Acked Read 8 Done Sent Done false
7108 2606 Acked Write 8 Done Sent Done false
7109 2607 Acked Read 1 Done Sent Done false
7110 2608 Acked Read 8 Done Sent Done false
7111 2609 Acked Read 5 Done Sent Done false
7112 2610 Acked Flush 0 Done Sent Done false
7113 2611 Acked Write 5 Done Sent Done false
7114 2612 Acked Flush 0 Done Sent Done false
7115 2613 Acked Write 5 Done Sent Done false
7116 2614 Acked Read 9 Done Sent Done false
7117 2615 Acked Flush 0 Done Sent Done false
7118 2616 Acked Read 2 Done Sent Done false
7119 2617 Acked Write 6 Done Sent Done false
7120 2618 Acked Read 10 Done Sent Done false
7121 2619 Acked Read 3 Done Sent Done false
7122 2620 Acked Read 8 Done Sent Done false
7123 2621 Acked Read 6 Done Sent Done false
7124 2622 Acked Write 4 Done Sent Done false
7125 2623 Acked Write 5 Done Sent Done false
7126 2624 Acked Write 2 Done Sent Done false
7127 2625 Acked Read 9 Done Sent Done false
7128 2626 Acked Write 7 Done Sent Done false
7129 2627 Acked Read 3 Done Sent Done false
7130 2628 Acked Read 8 Done Sent Done false
7131 2629 Acked Read 2 Done Sent Done false
7132 2630 Acked Write 10 Done Sent Done false
7133 2631 Acked Write 5 Done Sent Done false
7134 2632 Acked Read 1 Done Sent Done false
7135 2633 Acked Read 4 Done Sent Done false
7136 2634 Acked Read 9 Done Sent Done false
7137 2635 Acked Read 6 Done Sent Done false
7138 2636 Acked Write 1 Done Sent Done false
7139 2637 Acked Read 7 Done Sent Done false
7140 2638 Acked Write 3 Done Sent Done false
7141 2639 Acked Read 3 Done Sent Done false
7142 2640 Acked Read 9 Done Sent Done false
7143 2641 Acked Write 10 Done Sent Done false
7144 2642 Acked Read 4 Done Sent Done false
7145 2643 Acked Write 9 Done Sent Done false
7146 2644 Acked Read 9 Done Sent Done false
7147 2645 Acked Write 1 Done Sent Done false
7148 2646 Acked Read 4 Done Sent Done false
7149 2647 Acked Write 3 Done Sent Done false
7150 2648 Acked Read 6 Done Sent Done false
7151 2649 Acked Flush 0 Done Sent Done false
7152 2650 Acked Write 7 Done Sent Done false
7153 2651 Acked Read 9 Done Sent Done false
7154 2652 Acked Read 5 Done Sent Done false
7155 2653 Acked Flush 0 Done Sent Done false
7156 2654 Acked Read 1 Done Sent Done false
7157 2655 Acked Read 1 Done Sent Done false
7158 2656 Acked Read 6 Done Sent Done false
7159 2657 Acked Read 9 Done Sent Done false
7160 2658 Acked Read 6 Done Sent Done false
7161 2659 Acked Flush 0 Done Sent Done false
7162 2660 Acked Read 6 Done Sent Done false
7163 2661 Acked Read 2 Done Sent Done false
7164 2662 Acked Read 6 Done Sent Done false
7165 2663 Acked Read 6 Done Sent Done false
7166 2664 Acked Read 4 Done Sent Done false
7167 2665 Acked Read 6 Done Sent Done false
7168 2666 Acked Write 3 Done Sent Done false
7169 2667 Acked Write 1 Done Sent Done false
7170 2668 Acked Read 3 Done Sent Done false
7171 2669 Acked Read 6 Done Sent Done false
7172 2670 Acked Read 1 Done Sent Done false
7173 2671 Acked Write 6 Done Sent Done false
7174 2672 Acked Read 7 Done Sent Done false
7175 2673 Acked Read 8 Done Sent Done false
7176 2674 Acked Write 2 Done Sent Done false
7177 2675 Acked Read 10 Done Sent Done false
7178 2676 Acked Write 9 Done Sent Done false
7179 2677 Acked Read 6 Done Sent Done false
7180 2678 Acked Write 1 Done Sent Done false
7181 2679 Acked Flush 0 Done Sent Done false
7182 2680 Acked Read 10 Done Sent Done false
7183 2681 Acked Read 10 Done Sent Done false
7184 2682 Acked Write 2 Done Sent Done false
7185 2683 Acked Write 4 Done Sent Done false
7186 2684 Acked Write 2 Done Sent Done false
7187 2685 Acked Write 10 Done Sent Done false
7188 2686 Acked Read 6 Done Sent Done false
7189 2687 Acked Read 10 Done Sent Done false
7190 2688 Acked Read 10 Done Sent Done false
7191 2689 Acked Read 3 Done Sent Done false
7192 2690 Acked Read 1 Done Sent Done false
7193 2691 Acked Write 8 Done Sent Done false
7194 2692 Acked Flush 0 Done Sent Done false
7195 2693 Acked Write 3 Done Sent Done false
7196 2694 Acked Write 8 Done Sent Done false
7197 2695 Acked Write 2 Done Sent Done false
7198 2696 Acked Read 6 Done Sent Done false
7199 2697 Acked Write 1 Done Sent Done false
7200 2698 Acked Write 6 Done Sent Done false
7201 2699 Acked Write 6 Done Sent Done false
7202 2700 Acked Write 7 Done Sent Done false
7203 2701 Acked Read 6 Done Sent Done false
7204 2702 Acked Write 5 Done Sent Done false
7205 2703 Acked Read 2 Done Sent Done false
7206 2704 Acked Read 1 Done Sent Done false
7207 2705 Acked Write 1 Done Sent Done false
7208 2706 Acked Write 5 Done Sent Done false
7209 2707 Acked Write 5 Done Sent Done false
7210 2708 Acked Write 2 Done Sent Done false
7211 2709 Acked Flush 0 Done Sent Done false
7212 2710 Acked Read 8 Done Sent Done false
7213 2711 Acked Write 8 Done Sent Done false
7214 2712 Acked Read 4 Done Sent Done false
7215 2713 Acked Write 5 Done Sent Done false
7216 2714 Acked Write 6 Done Sent Done false
7217 2715 Acked Read 7 Done Sent Done false
7218 2716 Acked Write 1 Done Sent Done false
7219 2717 Acked Read 10 Done Sent Done false
7220 2718 Acked Write 9 Done Sent Done false
7221 2719 Acked Write 4 Done Sent Done false
7222 2720 Acked Read 9 Done Sent Done false
7223 2721 Acked Read 5 Done Sent Done false
7224 2722 Acked Read 3 Done Sent Done false
7225 2723 Acked Flush 0 Done Sent Done false
7226 2724 Acked Write 2 Done Sent Done false
7227 2725 Acked Write 5 Done Sent Done false
7228 2726 Acked Read 3 Done Sent Done false
7229 2727 Acked Read 3 Done Sent Done false
7230 2728 Acked Write 1 Done Sent Done false
7231 2729 Acked Write 5 Done Sent Done false
7232 2730 Acked Write 3 Done Sent Done false
7233 2731 Acked Read 1 Done Sent Done false
7234 2732 Acked Read 7 Done Sent Done false
7235 2733 Acked Read 9 Done Sent Done false
7236 2734 Acked Write 6 Done Sent Done false
7237 2735 Acked Write 6 Done Sent Done false
7238 2736 Acked Write 10 Done Sent Done false
7239 2737 Acked Read 4 Done Sent Done false
7240 2738 Acked Write 2 Done Sent Done false
7241 2739 Acked Write 4 Done Sent Done false
7242 2740 Acked Read 4 Done Sent Done false
7243 2741 Acked Read 7 Done Sent Done false
7244 2742 Acked Read 10 Done Sent Done false
7245 2743 Acked Flush 0 Done Sent Done false
7246 2744 Acked Read 7 Done Sent Done false
7247 2745 Acked Flush 0 Done Sent Done false
7248 2746 Acked Read 2 Done Sent Done false
7249 2747 Acked Read 10 Done Sent Done false
7250 2748 Acked Read 9 Done Sent Done false
7251 2749 Acked Read 10 Done Sent Done false
7252 2750 Acked Read 9 Done Sent Done false
7253 2751 Acked Flush 0 Done Sent Done false
7254 2752 Acked Read 2 Done Sent Done false
7255 2753 Acked Read 1 Done Sent Done false
7256 2754 Acked Read 5 Done Sent Done false
7257 2755 Acked Write 6 Done Sent Done false
7258 2756 Acked Read 4 Done Sent Done false
7259 2757 Acked Flush 0 Done Sent Done false
7260 2758 Acked Read 7 Done Sent Done false
7261 2759 Acked Read 9 Done Sent Done false
7262 2760 Acked Write 8 Done Sent Done false
7263 2761 Acked Read 10 Done Sent Done false
7264 2762 Acked Read 1 Done Sent Done false
7265 2763 Acked Write 9 Done Sent Done false
7266 2764 Acked Read 5 Done Sent Done false
7267 2765 Acked Read 3 Done Sent Done false
7268 2766 Acked Write 9 Done Sent Done false
7269 2767 Acked Write 4 Done Sent Done false
7270 2768 Acked Read 6 Done Sent Done false
7271 2769 Acked Read 8 Done Sent Done false
7272 2770 Acked Write 2 Done Sent Done false
7273 2771 Acked Write 10 Done Sent Done false
7274 2772 Acked Read 8 Done Sent Done false
7275 2773 Acked Read 8 Done Sent Done false
7276 2774 Acked Read 7 Done Sent Done false
7277 2775 Acked Read 9 Done Sent Done false
7278 2776 Acked Write 9 Done Sent Done false
7279 2777 Acked Flush 0 Done Sent Done false
7280 2778 Acked Read 10 Done Sent Done false
7281 2779 Acked Read 1 Done Sent Done false
7282 2780 Acked Write 7 Done Sent Done false
7283 2781 Acked Read 8 Done Sent Done false
7284 2782 Acked Read 6 Done Sent Done false
7285 2783 Acked Read 7 Done Sent Done false
7286 2784 Acked Write 10 Done Sent Done false
7287 2785 Acked Flush 0 Done Sent Done false
7288 2786 Acked Write 2 Done Sent Done false
7289 2787 Acked Write 2 Done Sent Done false
7290 2788 Acked Read 4 Done Sent Done false
7291 2789 Acked Write 1 Done Sent Done false
7292 2790 Acked Read 5 Done Sent Done false
7293 2791 Acked Read 3 Done Sent Done false
7294 2792 Acked Write 2 Done Sent Done false
7295 2793 Acked Read 5 Done Sent Done false
7296 2794 Acked Write 3 Done Sent Done false
7297 2795 Acked Write 4 Done Sent Done false
7298 2796 Acked Write 2 Done Sent Done false
7299 2797 Acked Write 7 Done Sent Done false
7300 2798 Acked Read 5 Done Sent Done false
7301 2799 Acked Write 8 Done Sent Done false
7302 2800 Acked Read 9 Done Sent Done false
7303 2801 Acked Write 4 Done Sent Done false
7304 2802 Acked Write 3 Done Sent Done false
7305 2803 Acked Write 7 Done Sent Done false
7306 2804 Acked Write 3 Done Sent Done false
7307 2805 Acked Read 7 Done Sent Done false
7308 2806 Acked Read 4 Done Sent Done false
7309 2807 Acked Flush 0 Done Sent Done false
7310 2808 Acked Read 3 Done Sent Done false
7311 2809 Acked Read 2 Done Sent Done false
7312 2810 Acked Flush 0 Done Sent Done false
7313 STATES DS:0 DS:1 DS:2 TOTAL
7314 Sent 0 302 0 302
7315 Done 302 0 302 604
7316 Skipped 0 0 0 0
7317 Error 0 0 0 0
7318 Last Flush: 2810 2508 2810
7319 Downstairs last five completed: 2508 2507 2506 2505 2504
7320 Upstairs last five completed: 2810 2809 2808 2807 2806
7321 CLIENT: Up:0 ds:302 act:2
73222025-01-03T08:32:24.802ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73232025-01-03T08:32:24.802ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53199) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73242025-01-03T08:32:24.803ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73252025-01-03T08:32:24.804ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73262025-01-03T08:32:24.804ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73272025-01-03T08:32:24.804ZINFOcrucible: send last flush ID to this DS: 2508 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73282025-01-03T08:32:24.805ZINFOcrucible: Replied this last flush ID: 2508 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73292025-01-03T08:32:24.805ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73302025-01-03T08:32:24.805ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73312025-01-03T08:32:24.805ZINFOcrucible: [1] client re-new 302 jobs since flush 2508 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73322025-01-03T08:32:24.805ZINFOcrucible: [1] Replayed 302 jobs since flush: 2508 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
7333 ----------------------------------------------------------------
7334 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7335 Downstairs last five completed: 2810 2809 2808 2807 2806
7336 Upstairs last five completed: 2810 2809 2808 2807 2806
7337 CLIENT: Up:0 ds:0 act:3
7338 Replay: All jobs finished
7339 Main task has work to do, go find it
7340 got dsc Stop(2)
7341 stop 2
7342 Jan 03 08:32:27.050 INFO request completed, [8830] Got stop action so:false kr:false
7343 latency_us: 343, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 65d0dbaa-da67-4af2-b0c8-2af460a471f7, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
73442025-01-03T08:32:27.051ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73452025-01-03T08:32:27.051ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73462025-01-03T08:32:27.051ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73472025-01-03T08:32:27.051ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73482025-01-03T08:32:27.051ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
73492025-01-03T08:32:27.051ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
7350 Jan 03 08:32:27.051 INFO request completed, latency_us: 181, response_code: 200, uri: /state/cid/2, method: GET, req_id: 1c806497-efa8-4ced-909b-490a3e24c7ea, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
7351 [8830] Exited with: ExitStatus(unix_wait_status(9))
7352 [8830][2] reports Exit
7353 Jan 03 08:32:31.054 INFO request completed, latency_us: 197, response_code: 200, uri: /state/cid/2, method: GET, req_id: dfd36f4c-27ee-421b-b633-9ffa5fd56cea, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
7354 001/300 Read block 1566 len 20480
7355 002/300 Flush
7356 003/300 Read block 37 len 12288
7357 004/300 Write block 757 len 28672 data: 2 2 4 4 3 3 3
7358 005/300 Read block 2096 len 24576
7359 006/300 Write block 4067 len 28672 data: 2 2 2 2 2 2 3
7360 007/300 Read block 808 len 24576
7361 008/300 Write block 5 len 12288 data: 2 2 2
7362 009/300 Flush
7363 010/300 Read block 4654 len 4096
7364 011/300 Read block 1348 len 24576
7365 012/300 Write block 378 len 8192 data: 3 2
7366 013/300 Read block 1885 len 8192
7367 014/300 Read block 3452 len 32768
7368 015/300 Write block 223 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7369 016/300 Read block 3062 len 12288
7370 017/300 Read block 881 len 16384
7371 018/300 Write block 3730 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7372 019/300 Read block 1587 len 40960
7373 020/300 Write block 1834 len 36864 data: 2 2 2 2 3 4 4 5 6
7374 021/300 Read block 2073 len 12288
7375 022/300 Flush
7376 023/300 Read block 1823 len 36864
7377 024/300 Flush
7378 025/300 Read block 3519 len 8192
7379 026/300 Read block 1286 len 32768
7380 027/300 Read block 262 len 36864
7381 028/300 Read block 1353 len 4096
7382 029/300 Read block 3903 len 20480
7383 030/300 Read block 3164 len 24576
7384 031/300 Write block 1398 len 8192 data: 2 2
7385 032/300 Read block 2385 len 36864
7386 033/300 Write block 2854 len 32768 data: 2 2 2 3 3 3 3 3
7387 034/300 Write block 1626 len 16384 data: 2 2 3 3
7388 035/300 Write block 4930 len 16384 data: 4 3 3 3
7389 036/300 Write block 4613 len 24576 data: 2 2 3 3 3 3
7390 037/300 Read block 4651 len 16384
7391 038/300 Write block 317 len 12288 data: 3 2 2
7392 039/300 Write block 3385 len 12288 data: 2 2 2
7393 040/300 Read block 4970 len 16384
7394 041/300 Write block 54 len 12288 data: 4 4 4
7395 042/300 Read block 3434 len 36864
7396 043/300 Write block 1271 len 16384 data: 3 3 3 5
7397 044/300 Read block 4153 len 28672
7398 045/300 Write block 1137 len 28672 data: 3 3 3 3 2 3 3
7399 046/300 Read block 220 len 12288
7400 047/300 Read block 862 len 8192
7401 048/300 Read block 1621 len 40960
7402 049/300 Write block 292 len 8192 data: 2 2
7403 050/300 Write block 2708 len 40960 data: 4 4 5 5 5 5 5 4 3 2
7404 051/300 Read block 3069 len 8192
7405 052/300 Write block 4303 len 36864 data: 2 2 2 2 2 3 3 3 4
7406 053/300 Read block 2216 len 12288
7407 054/300 Flush
7408 055/300 Write block 1370 len 32768 data: 3 2 2 2 2 2 2 2
7409 056/300 Read block 4729 len 24576
7410 057/300 Flush
7411 058/300 Read block 4907 len 40960
7412 059/300 Write block 4886 len 40960 data: 3 4 4 4 2 2 2 2 2 2
7413 060/300 Write block 308 len 28672 data: 3 4 4 3 3 3 3
7414 061/300 Read block 2852 len 20480
7415 062/300 Read block 3224 len 12288
7416 063/300 Write block 155 len 32768 data: 3 3 3 3 3 4 3 3
7417 064/300 Write block 2204 len 32768 data: 2 2 2 2 2 2 2 2
7418 065/300 Write block 1565 len 8192 data: 2 2
7419 066/300 Write block 2247 len 32768 data: 3 3 3 4 4 3 3 3
7420 067/300 Write block 1391 len 28672 data: 2 2 2 2 2 2 2
7421 068/300 Write block 1100 len 12288 data: 3 3 2
7422 069/300 Read block 1864 len 16384
7423 070/300 Write block 917 len 24576 data: 3 3 3 4 4 5
7424 071/300 Write block 3361 len 16384 data: 3 2 2 2
7425 072/300 Read block 3456 len 28672
7426 073/300 Write block 1108 len 8192 data: 2 2
7427 074/300 Read block 3865 len 8192
7428 075/300 Read block 4804 len 28672
7429 076/300 Write block 2516 len 32768 data: 2 3 3 3 3 3 3 3
7430 077/300 Write block 812 len 36864 data: 3 3 3 3 3 4 3 3 4
7431 078/300 Write block 1675 len 20480 data: 4 4 4 4 4
7432 079/300 Read block 3091 len 16384
7433 080/300 Read block 4088 len 36864
7434 081/300 Flush
7435 082/300 Read block 458 len 16384
7436 083/300 Write block 3696 len 40960 data: 2 2 2 2 2 2 2 3 3 3
7437 084/300 Flush
7438 085/300 Read block 4489 len 8192
7439 086/300 Read block 2214 len 4096
7440 087/300 Write block 1793 len 40960 data: 3 2 2 2 2 2 2 2 2 2
7441 088/300 Read block 3007 len 12288
7442 089/300 Read block 4041 len 32768
7443 090/300 Write block 649 len 28672 data: 2 2 2 2 2 4 4
7444 091/300 Read block 1820 len 32768
7445 092/300 Write block 302 len 32768 data: 3 3 3 4 3 3 4 5
7446 093/300 Read block 3083 len 16384
7447 094/300 Write block 4439 len 40960 data: 3 3 4 4 4 4 4 4 4 4
7448 095/300 Read block 3524 len 24576
7449 096/300 Write block 2639 len 12288 data: 3 3 3
7450 097/300 Read block 2710 len 32768
7451 098/300 Read block 1099 len 28672
7452 099/300 Read block 1693 len 20480
7453 100/300 Read block 1256 len 4096
7454 101/300 Write block 96 len 28672 data: 2 2 2 2 2 2 2
7455 102/300 Write block 2677 len 16384 data: 2 2 2 2
7456 103/300 Flush
7457 104/300 Write block 3664 len 4096 data: 4
7458 105/300 Read block 3196 len 40960
7459 106/300 Flush
7460 107/300 Read block 610 len 24576
7461 108/300 Write block 3610 len 12288 data: 2 2 3
7462 109/300 Write block 764 len 4096 data: 3
7463 110/300 Write block 2153 len 40960 data: 3 3 3 3 3 3 2 2 5 4
7464 111/300 Flush
7465 112/300 Write block 631 len 40960 data: 3 3 4 3 3 3 3 4 4 4
7466 113/300 Read block 1589 len 8192
7467 114/300 Read block 2453 len 40960
7468 115/300 Write block 2252 len 20480 data: 4 4 4 3 3
7469 116/300 Flush
7470 117/300 Read block 2357 len 24576
7471 118/300 Write block 4693 len 20480 data: 4 4 4 3 3
7472 119/300 Write block 1254 len 4096 data: 3
7473 120/300 Write block 1205 len 12288 data: 2 2 2
7474 121/300 Read block 417 len 40960
7475 122/300 Read block 364 len 24576
7476 123/300 Write block 996 len 24576 data: 3 3 3 3 3 3
7477 124/300 Read block 3418 len 36864
7478 125/300 Write block 3999 len 28672 data: 3 4 5 4 4 4 3
7479 126/300 Write block 2382 len 40960 data: 2 3 3 5 2 2 2 2 2 2
7480 127/300 Read block 4707 len 40960
7481 128/300 Read block 3657 len 32768
7482 129/300 Read block 3245 len 36864
7483 130/300 Write block 2400 len 12288 data: 2 2 2
7484 131/300 Write block 3999 len 40960 data: 4 5 6 5 5 5 4 3 2 2
7485 132/300 Write block 3842 len 12288 data: 2 3 3
7486 133/300 Read block 1879 len 36864
7487 134/300 Read block 4227 len 16384
7488 135/300 Read block 3659 len 16384
7489 136/300 Read block 2736 len 28672
7490 137/300 Read block 1180 len 40960
7491 138/300 Flush
7492 139/300 Flush
7493 140/300 Read block 1903 len 28672
7494 141/300 Write block 2538 len 32768 data: 2 2 2 2 2 2 2 2
7495 142/300 Write block 1183 len 12288 data: 3 3 3
7496 143/300 Read block 897 len 4096
7497 144/300 Write block 3824 len 16384 data: 4 4 4 4
7498 145/300 Read block 684 len 40960
7499 146/300 Write block 3636 len 20480 data: 3 3 3 3 2
7500 147/300 Write block 310 len 40960 data: 5 4 4 4 4 3 3 4 3 3
7501 148/300 Flush
7502 149/300 Read block 4962 len 16384
7503 150/300 Read block 454 len 24576
7504 151/300 Flush
7505 152/300 Write block 681 len 28672 data: 3 2 2 2 2 2 2
7506 153/300 Read block 2287 len 40960
7507 154/300 Read block 2508 len 32768
7508 155/300 Read block 1718 len 24576
7509 156/300 Write block 3862 len 40960 data: 3 3 2 2 3 3 3 3 3 4
7510 157/300 Flush
7511 158/300 Write block 635 len 36864 data: 4 4 4 5 5 5 4 3 2
7512 159/300 Read block 2465 len 12288
7513 160/300 Read block 2962 len 12288
7514 161/300 Write block 1011 len 24576 data: 2 2 2 2 2 2
7515 162/300 Read block 3961 len 20480
7516 163/300 Read block 973 len 36864
7517 164/300 Write block 2136 len 8192 data: 2 2
7518 165/300 Write block 3712 len 40960 data: 3 2 2 3 4 4 4 5 4 4
7519 166/300 Read block 1729 len 4096
7520 167/300 Write block 4578 len 16384 data: 4 4 4 4
7521 168/300 Read block 3478 len 24576
7522 169/300 Write block 849 len 24576 data: 4 4 4 2 3 3
7523 170/300 Write block 3758 len 8192 data: 3 3
7524 171/300 Write block 3467 len 16384 data: 4 4 4 4
7525 172/300 Write block 3569 len 32768 data: 2 2 2 2 2 2 3 3
7526 173/300 Write block 488 len 20480 data: 2 3 3 3 3
7527 174/300 Read block 3610 len 8192
7528 175/300 Read block 1661 len 16384
7529 176/300 Read block 3979 len 28672
7530 177/300 Write block 2682 len 24576 data: 2 2 2 2 2 4
7531 178/300 Write block 206 len 4096 data: 3
7532 179/300 Read block 72 len 40960
7533 180/300 Read block 377 len 8192
7534 181/300 Read block 83 len 4096
7535 182/300 Read block 131 len 32768
7536 183/300 Write block 3237 len 24576 data: 2 2 2 2 2 2
7537 184/300 Write block 257 len 4096 data: 2
7538 185/300 Read block 1462 len 16384
7539 186/300 Read block 617 len 32768
7540 187/300 Read block 2558 len 32768
7541 188/300 Write block 2900 len 20480 data: 2 2 2 2 2
7542 189/300 Flush
7543 190/300 Read block 2690 len 8192
7544 191/300 Flush
7545 192/300 Write block 2967 len 8192 data: 2 2
7546 193/300 Write block 4050 len 4096 data: 2
7547 194/300 Read block 3148 len 16384
7548 195/300 Write block 497 len 4096 data: 3
7549 196/300 Flush
7550 197/300 Read block 1413 len 40960
7551 198/300 Flush
7552 199/300 Read block 177 len 36864
7553 200/300 Write block 60 len 40960 data: 4 3 2 3 4 4 5 5 4 4
7554 201/300 Read block 4660 len 36864
7555 202/300 Write block 479 len 32768 data: 3 3 3 3 3 3 2 2
7556 203/300 Read block 2372 len 16384
7557 204/300 Write block 690 len 32768 data: 2 2 2 3 3 3 3 3
7558 205/300 Write block 4434 len 36864 data: 2 2 2 2 2 4 4 5 5
7559 206/300 Read block 338 len 20480
7560 207/300 Write block 508 len 36864 data: 2 2 2 2 2 2 3 3 3
7561 208/300 Write block 2069 len 40960 data: 3 3 3 3 3 3 3 2 2 2
7562 209/300 Read block 1051 len 12288
7563 210/300 Read block 4383 len 16384
7564 211/300 Read block 4257 len 16384
7565 212/300 Read block 1675 len 28672
7566 213/300 Write block 2053 len 20480 data: 3 2 2 2 2
7567 214/300 Write block 519 len 4096 data: 4
7568 215/300 Read block 4566 len 20480
7569 216/300 Flush
7570 217/300 Write block 1290 len 8192 data: 2 3
7571 218/300 Read block 2868 len 36864
7572 219/300 Read block 3369 len 20480
7573 220/300 Read block 3203 len 40960
7574 221/300 Write block 4242 len 32768 data: 2 2 2 2 3 3 3 3
7575 222/300 Write block 3701 len 8192 data: 3 3
7576 223/300 Read block 1723 len 8192
7577 224/300 Read block 4146 len 20480
7578 225/300 Read block 3734 len 32768
7579 226/300 Write block 792 len 28672 data: 3 3 3 3 3 3 3
7580 227/300 Read block 2792 len 32768
7581 228/300 Write block 4368 len 32768 data: 3 2 2 2 2 2 2 2
7582 229/300 Read block 3849 len 28672
7583 230/300 Write block 1600 len 8192 data: 3 3
7584 231/300 Flush
7585 232/300 Read block 2293 len 32768
7586 233/300 Write block 2593 len 20480 data: 2 2 3 3 4
7587 234/300 Write block 647 len 20480 data: 2 2 3 3 3
7588 235/300 Write block 3633 len 40960 data: 4 3 3 4 4 4 4 3 3 3
7589 236/300 Write block 4040 len 36864 data: 3 3 4 5 4 3 3 3 3
7590 237/300 Read block 1737 len 40960
7591 238/300 Read block 2399 len 8192
7592 239/300 Write block 2052 len 20480 data: 3 4 3 3 3
7593 240/300 Read block 46 len 24576
7594 241/300 Flush
7595 242/300 Write block 4192 len 8192 data: 2 2
7596 243/300 Read block 2484 len 32768
7597 244/300 Read block 1178 len 36864
7598 245/300 Read block 3648 len 20480
7599 246/300 Read block 4535 len 36864
7600 247/300 Read block 2719 len 16384
7601 248/300 Read block 1054 len 12288
7602 249/300 Read block 114 len 32768
7603 250/300 Read block 962 len 20480
7604 251/300 Write block 3142 len 24576 data: 3 2 2 3 3 3
7605 252/300 Read block 2584 len 40960
7606 253/300 Write block 3973 len 36864 data: 3 3 3 4 4 4 4 4 2
7607 254/300 Read block 3602 len 24576
7608 255/300 Read block 635 len 36864
7609 256/300 Read block 2766 len 12288
7610 257/300 Read block 4380 len 36864
7611 258/300 Read block 3402 len 24576
7612 259/300 Read block 1932 len 40960
7613 260/300 Read block 776 len 32768
7614 261/300 Read block 295 len 4096
7615 262/300 Flush
7616 263/300 Write block 3515 len 8192 data: 2 2
7617 264/300 Read block 3169 len 4096
7618 265/300 Write block 774 len 4096 data: 3
7619 266/300 Read block 4434 len 24576
7620 267/300 Write block 444 len 32768 data: 2 2 2 3 2 2 2 2
7621 268/300 Write block 3938 len 28672 data: 2 2 2 2 2 2 2
7622 269/300 Write block 2029 len 8192 data: 4 4
7623 270/300 Write block 1301 len 24576 data: 2 2 2 3 3 2
7624 271/300 Read block 745 len 36864
7625 272/300 Write block 3744 len 16384 data: 3 3 3 3
7626 273/300 Write block 3165 len 20480 data: 3 2 2 2 2
7627 274/300 Flush
7628 275/300 Read block 849 len 12288
7629 276/300 Read block 499 len 40960
7630 277/300 Flush
7631 278/300 Write block 1326 len 12288 data: 4 4 4
7632 279/300 Write block 2055 len 8192 data: 4 4
7633 280/300 Read block 4074 len 28672
7634 281/300 Write block 2414 len 20480 data: 2 2 2 2 2
7635 282/300 Write block 3621 len 36864 data: 2 2 2 3 3 3 3 3 3
7636 283/300 Write block 1519 len 28672 data: 3 3 2 3 5 5 5
7637 284/300 Read block 3862 len 4096
7638 285/300 Write block 1614 len 4096 data: 2
7639 286/300 Read block 1342 len 4096
7640 287/300 Write block 946 len 36864 data: 2 2 2 2 2 2 2 2 2
7641 288/300 Read block 986 len 40960
7642 289/300 Write block 1396 len 4096 data: 3
7643 290/300 Read block 2390 len 20480
7644 291/300 Write block 580 len 24576 data: 3 3 2 2 2 2
7645 292/300 Write block 2494 len 12288 data: 2 2 2
7646 293/300 Read block 803 len 40960
7647 294/300 Read block 3573 len 36864
7648 295/300 Flush
7649 296/300 Flush
7650 297/300 Write block 2919 len 4096 data: 2
7651 298/300 Write block 2067 len 28672 data: 3 3 4 4 4 4 4
7652 299/300 Read block 4316 len 28672
7653 300/300 Write block 433 len 40960 data: 2 2 3 3 3 3 3 2 2 2
7654 301/300 Read block 4064 len 40960
7655 Main task has work to do, go find it
7656 got dsc Start(2)
7657 start 2
7658 Jan 03 08:32:31.304 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/2, method: POST, req_id: 521d6ae8-43f1-495b-9d42-0123dc095e15, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
7659 Replay: started 2, returned:Ok(())
7660 ----------------------------------------------------------------
7661 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7662 States: Active Active Offline
7663 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7664 2811 Acked Read 5 Done Done Sent false
7665 2812 Acked Flush 0 Done Done Sent false
7666 2813 Acked Read 3 Done Done Sent false
7667 2814 Acked Write 7 Done Done Sent false
7668 2815 Acked Read 6 Done Done Sent false
7669 2816 Acked Write 7 Done Done Sent false
7670 2817 Acked Read 6 Done Done Sent false
7671 2818 Acked Write 3 Done Done Sent false
7672 2819 Acked Flush 0 Done Done Sent false
7673 2820 Acked Read 1 Done Done Sent false
7674 2821 Acked Read 6 Done Done Sent false
7675 2822 Acked Write 2 Done Done Sent false
7676 2823 Acked Read 2 Done Done Sent false
7677 2824 Acked Read 8 Done Done Sent false
7678 2825 Acked Write 10 Done Done Sent false
7679 2826 Acked Read 3 Done Done Sent false
7680 2827 Acked Read 4 Done Done Sent false
7681 2828 Acked Write 10 Done Done Sent false
7682 2829 Acked Read 10 Done Done Sent false
7683 2830 Acked Write 9 Done Done Sent false
7684 2831 Acked Read 3 Done Done Sent false
7685 2832 Acked Flush 0 Done Done Sent false
7686 2833 Acked Read 9 Done Done Sent false
7687 2834 Acked Flush 0 Done Done Sent false
7688 2835 Acked Read 2 Done Done Sent false
7689 2836 Acked Read 8 Done Done Sent false
7690 2837 Acked Read 9 Done Done Sent false
7691 2838 Acked Read 1 Done Done Sent false
7692 2839 Acked Read 5 Done Done Sent false
7693 2840 Acked Read 6 Done Done Sent false
7694 2841 Acked Write 2 Done Done Sent false
7695 2842 Acked Read 9 Done Done Sent false
7696 2843 Acked Write 8 Done Done Sent false
7697 2844 Acked Write 4 Done Done Sent false
7698 2845 Acked Write 4 Done Done Sent false
7699 2846 Acked Write 6 Done Done Sent false
7700 2847 Acked Read 4 Done Done Sent false
7701 2848 Acked Write 3 Done Done Sent false
7702 2849 Acked Write 3 Done Done Sent false
7703 2850 Acked Read 4 Done Done Sent false
7704 2851 Acked Write 3 Done Done Sent false
7705 2852 Acked Read 9 Done Done Sent false
7706 2853 Acked Write 4 Done Done Sent false
7707 2854 Acked Read 7 Done Done Sent false
7708 2855 Acked Write 7 Done Done Sent false
7709 2856 Acked Read 3 Done Done Sent false
7710 2857 Acked Read 2 Done Done Sent false
7711 2858 Acked Read 10 Done Done Sent false
7712 2859 Acked Write 2 Done Done Sent false
7713 2860 Acked Write 10 Done Done Sent false
7714 2861 Acked Read 2 Done Done Sent false
7715 2862 Acked Write 9 Done Done Sent false
7716 2863 Acked Read 3 Done Done Sent false
7717 2864 Acked Flush 0 Done Done Sent false
7718 2865 Acked Write 8 Done Done Sent false
7719 2866 Acked Read 6 Done Done Sent false
7720 2867 Acked Flush 0 Done Done Sent false
7721 2868 Acked Read 10 Done Done Sent false
7722 2869 Acked Write 10 Done Done Sent false
7723 2870 Acked Write 7 Done Done Sent false
7724 2871 Acked Read 5 Done Done Sent false
7725 2872 Acked Read 3 Done Done Sent false
7726 2873 Acked Write 8 Done Done Sent false
7727 2874 Acked Write 8 Done Done Sent false
7728 2875 Acked Write 2 Done Done Sent false
7729 2876 Acked Write 8 Done Done Sent false
7730 2877 Acked Write 7 Done Done Sent false
7731 2878 Acked Write 3 Done Done Sent false
7732 2879 Acked Read 4 Done Done Sent false
7733 2880 Acked Write 6 Done Done Sent false
7734 2881 Acked Write 4 Done Done Sent false
7735 2882 Acked Read 7 Done Done Sent false
7736 2883 Acked Write 2 Done Done Sent false
7737 2884 Acked Read 2 Done Done Sent false
7738 2885 Acked Read 7 Done Done Sent false
7739 2886 Acked Write 8 Done Done Sent false
7740 2887 Acked Write 9 Done Done Sent false
7741 2888 Acked Write 5 Done Done Sent false
7742 2889 Acked Read 4 Done Done Sent false
7743 2890 Acked Read 9 Done Done Sent false
7744 2891 Acked Flush 0 Done Done Sent false
7745 2892 Acked Read 4 Done Done Sent false
7746 2893 Acked Write 10 Done Done Sent false
7747 2894 Acked Flush 0 Done Done Sent false
7748 2895 Acked Read 2 Done Done Sent false
7749 2896 Acked Read 1 Done Done Sent false
7750 2897 Acked Write 10 Done Done Sent false
7751 2898 Acked Read 3 Done Done Sent false
7752 2899 Acked Read 8 Done Done Sent false
7753 2900 Acked Write 7 Done Done Sent false
7754 2901 Acked Read 8 Done Done Sent false
7755 2902 Acked Write 8 Done Done Sent false
7756 2903 Acked Read 4 Done Done Sent false
7757 2904 Acked Write 10 Done Done Sent false
7758 2905 Acked Read 6 Done Done Sent false
7759 2906 Acked Write 3 Done Done Sent false
7760 2907 Acked Read 8 Done Done Sent false
7761 2908 Acked Read 7 Done Done Sent false
7762 2909 Acked Read 5 Done Done Sent false
7763 2910 Acked Read 1 Done Done Sent false
7764 2911 Acked Write 7 Done Done Sent false
7765 2912 Acked Write 4 Done Done Sent false
7766 2913 Acked Flush 0 Done Done Sent false
7767 2914 Acked Write 1 Done Done Sent false
7768 2915 Acked Read 10 Done Done Sent false
7769 2916 Acked Flush 0 Done Done Sent false
7770 2917 Acked Read 6 Done Done Sent false
7771 2918 Acked Write 3 Done Done Sent false
7772 2919 Acked Write 1 Done Done Sent false
7773 2920 Acked Write 10 Done Done Sent false
7774 2921 Acked Flush 0 Done Done Sent false
7775 2922 Acked Write 10 Done Done Sent false
7776 2923 Acked Read 2 Done Done Sent false
7777 2924 Acked Read 10 Done Done Sent false
7778 2925 Acked Write 5 Done Done Sent false
7779 2926 Acked Flush 0 Done Done Sent false
7780 2927 Acked Read 6 Done Done Sent false
7781 2928 Acked Write 5 Done Done Sent false
7782 2929 Acked Write 1 Done Done Sent false
7783 2930 Acked Write 3 Done Done Sent false
7784 2931 Acked Read 10 Done Done Sent false
7785 2932 Acked Read 6 Done Done Sent false
7786 2933 Acked Write 6 Done Done Sent false
7787 2934 Acked Read 9 Done Done Sent false
7788 2935 Acked Write 7 Done Done Sent false
7789 2936 Acked Write 10 Done Done Sent false
7790 2937 Acked Read 10 Done Done Sent false
7791 2938 Acked Read 8 Done Done Sent false
7792 2939 Acked Read 9 Done Done Sent false
7793 2940 Acked Write 3 Done Done Sent false
7794 2941 Acked Write 10 Done Done Sent false
7795 2942 Acked Write 3 Done Done Sent false
7796 2943 Acked Read 9 Done Done Sent false
7797 2944 Acked Read 4 Done Done Sent false
7798 2945 Acked Read 4 Done Done Sent false
7799 2946 Acked Read 7 Done Done Sent false
7800 2947 Acked Read 10 Done Done Sent false
7801 2948 Acked Flush 0 Done Done Sent false
7802 2949 Acked Flush 0 Done Done Sent false
7803 2950 Acked Read 7 Done Done Sent false
7804 2951 Acked Write 8 Done Done Sent false
7805 2952 Acked Write 3 Done Done Sent false
7806 2953 Acked Read 1 Done Done Sent false
7807 2954 Acked Write 4 Done Done Sent false
7808 2955 Acked Read 10 Done Done Sent false
7809 2956 Acked Write 5 Done Done Sent false
7810 2957 Acked Write 10 Done Done Sent false
7811 2958 Acked Flush 0 Done Done Sent false
7812 2959 Acked Read 4 Done Done Sent false
7813 2960 Acked Read 6 Done Done Sent false
7814 2961 Acked Flush 0 Done Done Sent false
7815 2962 Acked Write 7 Done Done Sent false
7816 2963 Acked Read 10 Done Done Sent false
7817 2964 Acked Read 8 Done Done Sent false
7818 2965 Acked Read 6 Done Done Sent false
7819 2966 Acked Write 10 Done Done Sent false
7820 2967 Acked Flush 0 Done Done Sent false
7821 2968 Acked Write 9 Done Done Sent false
7822 2969 Acked Read 3 Done Done Sent false
7823 2970 Acked Read 3 Done Done Sent false
7824 2971 Acked Write 6 Done Done Sent false
7825 2972 Acked Read 5 Done Done Sent false
7826 2973 Acked Read 9 Done Done Sent false
7827 2974 Acked Write 2 Done Done Sent false
7828 2975 Acked Write 10 Done Done Sent false
7829 2976 Acked Read 1 Done Done Sent false
7830 2977 Acked Write 4 Done Done Sent false
7831 2978 Acked Read 6 Done Done Sent false
7832 2979 Acked Write 6 Done Done Sent false
7833 2980 Acked Write 2 Done Done Sent false
7834 2981 Acked Write 4 Done Done Sent false
7835 2982 Acked Write 8 Done Done Sent false
7836 2983 Acked Write 5 Done Done Sent false
7837 2984 Acked Read 2 Done Done Sent false
7838 2985 Acked Read 4 Done Done Sent false
7839 2986 Acked Read 7 Done Done Sent false
7840 2987 Acked Write 6 Done Done Sent false
7841 2988 Acked Write 1 Done Done Sent false
7842 2989 Acked Read 10 Done Done Sent false
7843 2990 Acked Read 2 Done Done Sent false
7844 2991 Acked Read 1 Done Done Sent false
7845 2992 Acked Read 8 Done Done Sent false
7846 2993 Acked Write 6 Done Done Sent false
7847 2994 Acked Write 1 Done Done Sent false
7848 2995 Acked Read 4 Done Done Sent false
7849 2996 Acked Read 8 Done Done Sent false
7850 2997 Acked Read 8 Done Done Sent false
7851 2998 Acked Write 5 Done Done Sent false
7852 2999 Acked Flush 0 Done Done Sent false
7853 3000 Acked Read 2 Done Done Sent false
7854 3001 Acked Flush 0 Done Done Sent false
7855 3002 Acked Write 2 Done Done Sent false
7856 3003 Acked Write 1 Done Done Sent false
7857 3004 Acked Read 4 Done Done Sent false
7858 3005 Acked Write 1 Done Done Sent false
7859 3006 Acked Flush 0 Done Done Sent false
7860 3007 Acked Read 10 Done Done Sent false
7861 3008 Acked Flush 0 Done Done Sent false
7862 3009 Acked Read 9 Done Done Sent false
7863 3010 Acked Write 10 Done Done Sent false
7864 3011 Acked Read 9 Done Done Sent false
7865 3012 Acked Write 8 Done Done Sent false
7866 3013 Acked Read 4 Done Done Sent false
7867 3014 Acked Write 8 Done Done Sent false
7868 3015 Acked Write 9 Done Done Sent false
7869 3016 Acked Read 5 Done Done Sent false
7870 3017 Acked Write 9 Done Done Sent false
7871 3018 Acked Write 10 Done Done Sent false
7872 3019 Acked Read 3 Done Done Sent false
7873 3020 Acked Read 4 Done Done Sent false
7874 3021 Acked Read 4 Done Done Sent false
7875 3022 Acked Read 7 Done Done Sent false
7876 3023 Acked Write 5 Done Done Sent false
7877 3024 Acked Write 1 Done Done Sent false
7878 3025 Acked Read 5 Done Done Sent false
7879 3026 Acked Flush 0 Done Done Sent false
7880 3027 Acked Write 2 Done Done Sent false
7881 3028 Acked Read 9 Done Done Sent false
7882 3029 Acked Read 5 Done Done Sent false
7883 3030 Acked Read 10 Done Done Sent false
7884 3031 Acked Write 8 Done Done Sent false
7885 3032 Acked Write 2 Done Done Sent false
7886 3033 Acked Read 2 Done Done Sent false
7887 3034 Acked Read 5 Done Done Sent false
7888 3035 Acked Read 8 Done Done Sent false
7889 3036 Acked Write 7 Done Done Sent false
7890 3037 Acked Read 8 Done Done Sent false
7891 3038 Acked Write 8 Done Done Sent false
7892 3039 Acked Read 7 Done Done Sent false
7893 3040 Acked Write 2 Done Done Sent false
7894 3041 Acked Flush 0 Done Done Sent false
7895 3042 Acked Read 8 Done Done Sent false
7896 3043 Acked Write 5 Done Done Sent false
7897 3044 Acked Write 5 Done Done Sent false
7898 3045 Acked Write 10 Done Done Sent false
7899 3046 Acked Write 9 Done Done Sent false
7900 3047 Acked Read 10 Done Done Sent false
7901 3048 Acked Read 2 Done Done Sent false
7902 3049 Acked Write 5 Done Done Sent false
7903 3050 Acked Read 6 Done Done Sent false
7904 3051 Acked Flush 0 Done Done Sent false
7905 3052 Acked Write 2 Done Done Sent false
7906 3053 Acked Read 8 Done Done Sent false
7907 3054 Acked Read 9 Done Done Sent false
7908 3055 Acked Read 5 Done Done Sent false
7909 3056 Acked Read 9 Done Done Sent false
7910 3057 Acked Read 4 Done Done Sent false
7911 3058 Acked Read 3 Done Done Sent false
7912 3059 Acked Read 8 Done Done Sent false
7913 3060 Acked Read 5 Done Done Sent false
7914 3061 Acked Write 6 Done Done Sent false
7915 3062 Acked Read 10 Done Done Sent false
7916 3063 Acked Write 9 Done Done Sent false
7917 3064 Acked Read 6 Done Done Sent false
7918 3065 Acked Read 9 Done Done Sent false
7919 3066 Acked Read 3 Done Done Sent false
7920 3067 Acked Read 9 Done Done Sent false
7921 3068 Acked Read 6 Done Done Sent false
7922 3069 Acked Read 10 Done Done Sent false
7923 3070 Acked Read 8 Done Done Sent false
7924 3071 Acked Read 1 Done Done Sent false
7925 3072 Acked Flush 0 Done Done Sent false
7926 3073 Acked Write 2 Done Done Sent false
7927 3074 Acked Read 1 Done Done Sent false
7928 3075 Acked Write 1 Done Done Sent false
7929 3076 Acked Read 6 Done Done Sent false
7930 3077 Acked Write 8 Done Done Sent false
7931 3078 Acked Write 7 Done Done Sent false
7932 3079 Acked Write 2 Done Done Sent false
7933 3080 Acked Write 6 Done Done Sent false
7934 3081 Acked Read 9 Done Done Sent false
7935 3082 Acked Write 4 Done Done Sent false
7936 3083 Acked Write 5 Done Done Sent false
7937 3084 Acked Flush 0 Done Done Sent false
7938 3085 Acked Read 3 Done Done Sent false
7939 3086 Acked Read 10 Done Done Sent false
7940 3087 Acked Flush 0 Done Done Sent false
7941 3088 Acked Write 3 Done Done Sent false
7942 3089 Acked Write 2 Done Done Sent false
7943 3090 Acked Read 7 Done Done Sent false
7944 3091 Acked Write 5 Done Done Sent false
7945 3092 Acked Write 9 Done Done Sent false
7946 3093 Acked Write 7 Done Done Sent false
7947 3094 Acked Read 1 Done Done Sent false
7948 3095 Acked Write 1 Done Done Sent false
7949 3096 Acked Read 1 Done Done Sent false
7950 3097 Acked Write 9 Done Done Sent false
7951 3098 Acked Read 10 Done Done Sent false
7952 3099 Acked Write 1 Done Done Sent false
7953 3100 Acked Read 5 Done Done Sent false
7954 3101 Acked Write 6 Done Done Sent false
7955 3102 Acked Write 3 Done Done Sent false
7956 3103 Acked Read 10 Done Done Sent false
7957 3104 Acked Read 9 Done Done Sent false
7958 3105 Acked Flush 0 Done Done Sent false
7959 3106 Acked Flush 0 Done Done Sent false
7960 3107 Acked Write 1 Done Done Sent false
7961 3108 Acked Write 7 Done Done Sent false
7962 3109 Acked Read 7 Done Done Sent false
7963 3110 Acked Write 10 Done Done Sent false
7964 3111 Acked Read 10 Done Done Sent false
7965 STATES DS:0 DS:1 DS:2 TOTAL
7966 Sent 0 0 301 301
7967 Done 301 301 0 602
7968 Skipped 0 0 0 0
7969 Error 0 0 0 0
7970 Last Flush: 3106 3106 2810
7971 Downstairs last five completed: 2810 2809 2808 2807 2806
7972 Upstairs last five completed: 3111 3110 3109 3108 3107
7973 CLIENT: Up:0 ds:301 act:2
7974 ----------------------------------------------------------------
7975 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7976 States: Active Active Offline
7977 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7978 2811 Acked Read 5 Done Done Sent false
7979 2812 Acked Flush 0 Done Done Sent false
7980 2813 Acked Read 3 Done Done Sent false
7981 2814 Acked Write 7 Done Done Sent false
7982 2815 Acked Read 6 Done Done Sent false
7983 2816 Acked Write 7 Done Done Sent false
7984 2817 Acked Read 6 Done Done Sent false
7985 2818 Acked Write 3 Done Done Sent false
7986 2819 Acked Flush 0 Done Done Sent false
7987 2820 Acked Read 1 Done Done Sent false
7988 2821 Acked Read 6 Done Done Sent false
7989 2822 Acked Write 2 Done Done Sent false
7990 2823 Acked Read 2 Done Done Sent false
7991 2824 Acked Read 8 Done Done Sent false
7992 2825 Acked Write 10 Done Done Sent false
7993 2826 Acked Read 3 Done Done Sent false
7994 2827 Acked Read 4 Done Done Sent false
7995 2828 Acked Write 10 Done Done Sent false
7996 2829 Acked Read 10 Done Done Sent false
7997 2830 Acked Write 9 Done Done Sent false
7998 2831 Acked Read 3 Done Done Sent false
7999 2832 Acked Flush 0 Done Done Sent false
8000 2833 Acked Read 9 Done Done Sent false
8001 2834 Acked Flush 0 Done Done Sent false
8002 2835 Acked Read 2 Done Done Sent false
8003 2836 Acked Read 8 Done Done Sent false
8004 2837 Acked Read 9 Done Done Sent false
8005 2838 Acked Read 1 Done Done Sent false
8006 2839 Acked Read 5 Done Done Sent false
8007 2840 Acked Read 6 Done Done Sent false
8008 2841 Acked Write 2 Done Done Sent false
8009 2842 Acked Read 9 Done Done Sent false
8010 2843 Acked Write 8 Done Done Sent false
8011 2844 Acked Write 4 Done Done Sent false
8012 2845 Acked Write 4 Done Done Sent false
8013 2846 Acked Write 6 Done Done Sent false
8014 2847 Acked Read 4 Done Done Sent false
8015 2848 Acked Write 3 Done Done Sent false
8016 2849 Acked Write 3 Done Done Sent false
8017 2850 Acked Read 4 Done Done Sent false
8018 2851 Acked Write 3 Done Done Sent false
8019 2852 Acked Read 9 Done Done Sent false
8020 2853 Acked Write 4 Done Done Sent false
8021 2854 Acked Read 7 Done Done Sent false
8022 2855 Acked Write 7 Done Done Sent false
8023 2856 Acked Read 3 Done Done Sent false
8024 2857 Acked Read 2 Done Done Sent false
8025 2858 Acked Read 10 Done Done Sent false
8026 2859 Acked Write 2 Done Done Sent false
8027 2860 Acked Write 10 Done Done Sent false
8028 2861 Acked Read 2 Done Done Sent false
8029 2862 Acked Write 9 Done Done Sent false
8030 2863 Acked Read 3 Done Done Sent false
8031 2864 Acked Flush 0 Done Done Sent false
8032 2865 Acked Write 8 Done Done Sent false
8033 2866 Acked Read 6 Done Done Sent false
8034 2867 Acked Flush 0 Done Done Sent false
8035 2868 Acked Read 10 Done Done Sent false
8036 2869 Acked Write 10 Done Done Sent false
8037 2870 Acked Write 7 Done Done Sent false
8038 2871 Acked Read 5 Done Done Sent false
8039 2872 Acked Read 3 Done Done Sent false
8040 2873 Acked Write 8 Done Done Sent false
8041 2874 Acked Write 8 Done Done Sent false
8042 2875 Acked Write 2 Done Done Sent false
8043 2876 Acked Write 8 Done Done Sent false
8044 2877 Acked Write 7 Done Done Sent false
8045 2878 Acked Write 3 Done Done Sent false
8046 2879 Acked Read 4 Done Done Sent false
8047 2880 Acked Write 6 Done Done Sent false
8048 2881 Acked Write 4 Done Done Sent false
8049 2882 Acked Read 7 Done Done Sent false
8050 2883 Acked Write 2 Done Done Sent false
8051 2884 Acked Read 2 Done Done Sent false
8052 2885 Acked Read 7 Done Done Sent false
8053 2886 Acked Write 8 Done Done Sent false
8054 2887 Acked Write 9 Done Done Sent false
8055 2888 Acked Write 5 Done Done Sent false
8056 2889 Acked Read 4 Done Done Sent false
8057 2890 Acked Read 9 Done Done Sent false
8058 2891 Acked Flush 0 Done Done Sent false
8059 2892 Acked Read 4 Done Done Sent false
8060 2893 Acked Write 10 Done Done Sent false
8061 2894 Acked Flush 0 Done Done Sent false
8062 2895 Acked Read 2 Done Done Sent false
8063 2896 Acked Read 1 Done Done Sent false
8064 2897 Acked Write 10 Done Done Sent false
8065 2898 Acked Read 3 Done Done Sent false
8066 2899 Acked Read 8 Done Done Sent false
8067 2900 Acked Write 7 Done Done Sent false
8068 2901 Acked Read 8 Done Done Sent false
8069 2902 Acked Write 8 Done Done Sent false
8070 2903 Acked Read 4 Done Done Sent false
8071 2904 Acked Write 10 Done Done Sent false
8072 2905 Acked Read 6 Done Done Sent false
8073 2906 Acked Write 3 Done Done Sent false
8074 2907 Acked Read 8 Done Done Sent false
8075 2908 Acked Read 7 Done Done Sent false
8076 2909 Acked Read 5 Done Done Sent false
8077 2910 Acked Read 1 Done Done Sent false
8078 2911 Acked Write 7 Done Done Sent false
8079 2912 Acked Write 4 Done Done Sent false
8080 2913 Acked Flush 0 Done Done Sent false
8081 2914 Acked Write 1 Done Done Sent false
8082 2915 Acked Read 10 Done Done Sent false
8083 2916 Acked Flush 0 Done Done Sent false
8084 2917 Acked Read 6 Done Done Sent false
8085 2918 Acked Write 3 Done Done Sent false
8086 2919 Acked Write 1 Done Done Sent false
8087 2920 Acked Write 10 Done Done Sent false
8088 2921 Acked Flush 0 Done Done Sent false
8089 2922 Acked Write 10 Done Done Sent false
8090 2923 Acked Read 2 Done Done Sent false
8091 2924 Acked Read 10 Done Done Sent false
8092 2925 Acked Write 5 Done Done Sent false
8093 2926 Acked Flush 0 Done Done Sent false
8094 2927 Acked Read 6 Done Done Sent false
8095 2928 Acked Write 5 Done Done Sent false
8096 2929 Acked Write 1 Done Done Sent false
8097 2930 Acked Write 3 Done Done Sent false
8098 2931 Acked Read 10 Done Done Sent false
8099 2932 Acked Read 6 Done Done Sent false
8100 2933 Acked Write 6 Done Done Sent false
8101 2934 Acked Read 9 Done Done Sent false
8102 2935 Acked Write 7 Done Done Sent false
8103 2936 Acked Write 10 Done Done Sent false
8104 2937 Acked Read 10 Done Done Sent false
8105 2938 Acked Read 8 Done Done Sent false
8106 2939 Acked Read 9 Done Done Sent false
8107 2940 Acked Write 3 Done Done Sent false
8108 2941 Acked Write 10 Done Done Sent false
8109 2942 Acked Write 3 Done Done Sent false
8110 2943 Acked Read 9 Done Done Sent false
8111 2944 Acked Read 4 Done Done Sent false
8112 2945 Acked Read 4 Done Done Sent false
8113 2946 Acked Read 7 Done Done Sent false
8114 2947 Acked Read 10 Done Done Sent false
8115 2948 Acked Flush 0 Done Done Sent false
8116 2949 Acked Flush 0 Done Done Sent false
8117 2950 Acked Read 7 Done Done Sent false
8118 2951 Acked Write 8 Done Done Sent false
8119 2952 Acked Write 3 Done Done Sent false
8120 2953 Acked Read 1 Done Done Sent false
8121 2954 Acked Write 4 Done Done Sent false
8122 2955 Acked Read 10 Done Done Sent false
8123 2956 Acked Write 5 Done Done Sent false
8124 2957 Acked Write 10 Done Done Sent false
8125 2958 Acked Flush 0 Done Done Sent false
8126 2959 Acked Read 4 Done Done Sent false
8127 2960 Acked Read 6 Done Done Sent false
8128 2961 Acked Flush 0 Done Done Sent false
8129 2962 Acked Write 7 Done Done Sent false
8130 2963 Acked Read 10 Done Done Sent false
8131 2964 Acked Read 8 Done Done Sent false
8132 2965 Acked Read 6 Done Done Sent false
8133 2966 Acked Write 10 Done Done Sent false
8134 2967 Acked Flush 0 Done Done Sent false
8135 2968 Acked Write 9 Done Done Sent false
8136 2969 Acked Read 3 Done Done Sent false
8137 2970 Acked Read 3 Done Done Sent false
8138 2971 Acked Write 6 Done Done Sent false
8139 2972 Acked Read 5 Done Done Sent false
8140 2973 Acked Read 9 Done Done Sent false
8141 2974 Acked Write 2 Done Done Sent false
8142 2975 Acked Write 10 Done Done Sent false
8143 2976 Acked Read 1 Done Done Sent false
8144 2977 Acked Write 4 Done Done Sent false
8145 2978 Acked Read 6 Done Done Sent false
8146 2979 Acked Write 6 Done Done Sent false
8147 2980 Acked Write 2 Done Done Sent false
8148 2981 Acked Write 4 Done Done Sent false
8149 2982 Acked Write 8 Done Done Sent false
8150 2983 Acked Write 5 Done Done Sent false
8151 2984 Acked Read 2 Done Done Sent false
8152 2985 Acked Read 4 Done Done Sent false
8153 2986 Acked Read 7 Done Done Sent false
8154 2987 Acked Write 6 Done Done Sent false
8155 2988 Acked Write 1 Done Done Sent false
8156 2989 Acked Read 10 Done Done Sent false
8157 2990 Acked Read 2 Done Done Sent false
8158 2991 Acked Read 1 Done Done Sent false
8159 2992 Acked Read 8 Done Done Sent false
8160 2993 Acked Write 6 Done Done Sent false
8161 2994 Acked Write 1 Done Done Sent false
8162 2995 Acked Read 4 Done Done Sent false
8163 2996 Acked Read 8 Done Done Sent false
8164 2997 Acked Read 8 Done Done Sent false
8165 2998 Acked Write 5 Done Done Sent false
8166 2999 Acked Flush 0 Done Done Sent false
8167 3000 Acked Read 2 Done Done Sent false
8168 3001 Acked Flush 0 Done Done Sent false
8169 3002 Acked Write 2 Done Done Sent false
8170 3003 Acked Write 1 Done Done Sent false
8171 3004 Acked Read 4 Done Done Sent false
8172 3005 Acked Write 1 Done Done Sent false
8173 3006 Acked Flush 0 Done Done Sent false
8174 3007 Acked Read 10 Done Done Sent false
8175 3008 Acked Flush 0 Done Done Sent false
8176 3009 Acked Read 9 Done Done Sent false
8177 3010 Acked Write 10 Done Done Sent false
8178 3011 Acked Read 9 Done Done Sent false
8179 3012 Acked Write 8 Done Done Sent false
8180 3013 Acked Read 4 Done Done Sent false
8181 3014 Acked Write 8 Done Done Sent false
8182 3015 Acked Write 9 Done Done Sent false
8183 3016 Acked Read 5 Done Done Sent false
8184 3017 Acked Write 9 Done Done Sent false
8185 3018 Acked Write 10 Done Done Sent false
8186 3019 Acked Read 3 Done Done Sent false
8187 3020 Acked Read 4 Done Done Sent false
8188 3021 Acked Read 4 Done Done Sent false
8189 3022 Acked Read 7 Done Done Sent false
8190 3023 Acked Write 5 Done Done Sent false
8191 3024 Acked Write 1 Done Done Sent false
8192 3025 Acked Read 5 Done Done Sent false
8193 3026 Acked Flush 0 Done Done Sent false
8194 3027 Acked Write 2 Done Done Sent false
8195 3028 Acked Read 9 Done Done Sent false
8196 3029 Acked Read 5 Done Done Sent false
8197 3030 Acked Read 10 Done Done Sent false
8198 3031 Acked Write 8 Done Done Sent false
8199 3032 Acked Write 2 Done Done Sent false
8200 3033 Acked Read 2 Done Done Sent false
8201 3034 Acked Read 5 Done Done Sent false
8202 3035 Acked Read 8 Done Done Sent false
8203 3036 Acked Write 7 Done Done Sent false
8204 3037 Acked Read 8 Done Done Sent false
8205 3038 Acked Write 8 Done Done Sent false
8206 3039 Acked Read 7 Done Done Sent false
8207 3040 Acked Write 2 Done Done Sent false
8208 3041 Acked Flush 0 Done Done Sent false
8209 3042 Acked Read 8 Done Done Sent false
8210 3043 Acked Write 5 Done Done Sent false
8211 3044 Acked Write 5 Done Done Sent false
8212 3045 Acked Write 10 Done Done Sent false
8213 3046 Acked Write 9 Done Done Sent false
8214 3047 Acked Read 10 Done Done Sent false
8215 3048 Acked Read 2 Done Done Sent false
8216 3049 Acked Write 5 Done Done Sent false
8217 3050 Acked Read 6 Done Done Sent false
8218 3051 Acked Flush 0 Done Done Sent false
8219 3052 Acked Write 2 Done Done Sent false
8220 3053 Acked Read 8 Done Done Sent false
8221 3054 Acked Read 9 Done Done Sent false
8222 3055 Acked Read 5 Done Done Sent false
8223 3056 Acked Read 9 Done Done Sent false
8224 3057 Acked Read 4 Done Done Sent false
8225 3058 Acked Read 3 Done Done Sent false
8226 3059 Acked Read 8 Done Done Sent false
8227 3060 Acked Read 5 Done Done Sent false
8228 3061 Acked Write 6 Done Done Sent false
8229 3062 Acked Read 10 Done Done Sent false
8230 3063 Acked Write 9 Done Done Sent false
8231 3064 Acked Read 6 Done Done Sent false
8232 3065 Acked Read 9 Done Done Sent false
8233 3066 Acked Read 3 Done Done Sent false
8234 3067 Acked Read 9 Done Done Sent false
8235 3068 Acked Read 6 Done Done Sent false
8236 3069 Acked Read 10 Done Done Sent false
8237 3070 Acked Read 8 Done Done Sent false
8238 3071 Acked Read 1 Done Done Sent false
8239 3072 Acked Flush 0 Done Done Sent false
8240 3073 Acked Write 2 Done Done Sent false
8241 3074 Acked Read 1 Done Done Sent false
8242 3075 Acked Write 1 Done Done Sent false
8243 3076 Acked Read 6 Done Done Sent false
8244 3077 Acked Write 8 Done Done Sent false
8245 3078 Acked Write 7 Done Done Sent false
8246 3079 Acked Write 2 Done Done Sent false
8247 3080 Acked Write 6 Done Done Sent false
8248 3081 Acked Read 9 Done Done Sent false
8249 3082 Acked Write 4 Done Done Sent false
8250 3083 Acked Write 5 Done Done Sent false
8251 3084 Acked Flush 0 Done Done Sent false
8252 3085 Acked Read 3 Done Done Sent false
8253 3086 Acked Read 10 Done Done Sent false
8254 3087 Acked Flush 0 Done Done Sent false
8255 3088 Acked Write 3 Done Done Sent false
8256 3089 Acked Write 2 Done Done Sent false
8257 3090 Acked Read 7 Done Done Sent false
8258 3091 Acked Write 5 Done Done Sent false
8259 3092 Acked Write 9 Done Done Sent false
8260 3093 Acked Write 7 Done Done Sent false
8261 3094 Acked Read 1 Done Done Sent false
8262 3095 Acked Write 1 Done Done Sent false
8263 3096 Acked Read 1 Done Done Sent false
8264 3097 Acked Write 9 Done Done Sent false
8265 3098 Acked Read 10 Done Done Sent false
8266 3099 Acked Write 1 Done Done Sent false
8267 3100 Acked Read 5 Done Done Sent false
8268 3101 Acked Write 6 Done Done Sent false
8269 3102 Acked Write 3 Done Done Sent false
8270 3103 Acked Read 10 Done Done Sent false
8271 3104 Acked Read 9 Done Done Sent false
8272 3105 Acked Flush 0 Done Done Sent false
8273 3106 Acked Flush 0 Done Done Sent false
8274 3107 Acked Write 1 Done Done Sent false
8275 3108 Acked Write 7 Done Done Sent false
8276 3109 Acked Read 7 Done Done Sent false
8277 3110 Acked Write 10 Done Done Sent false
8278 3111 Acked Read 10 Done Done Sent false
8279 3112 Acked Flush 0 Done Done Sent false
8280 STATES DS:0 DS:1 DS:2 TOTAL
8281 Sent 0 0 302 302
8282 Done 302 302 0 604
8283 Skipped 0 0 0 0
8284 Error 0 0 0 0
8285 Last Flush: 3112 3112 2810
8286 Downstairs last five completed: 2810 2809 2808 2807 2806
8287 Upstairs last five completed: 3112 3111 3110 3109 3108
8288 CLIENT: Up:0 ds:302 act:2
82892025-01-03T08:32:37.052ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
82902025-01-03T08:32:37.052ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
82912025-01-03T08:32:37.052ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
82922025-01-03T08:32:37.052ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
82932025-01-03T08:32:37.052ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
8294 ----------------------------------------------------------------
8295 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8296 States: Active Active Offline
8297 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8298 2811 Acked Read 5 Done Done Sent false
8299 2812 Acked Flush 0 Done Done Sent false
8300 2813 Acked Read 3 Done Done Sent false
8301 2814 Acked Write 7 Done Done Sent false
8302 2815 Acked Read 6 Done Done Sent false
8303 2816 Acked Write 7 Done Done Sent false
8304 2817 Acked Read 6 Done Done Sent false
8305 2818 Acked Write 3 Done Done Sent false
8306 2819 Acked Flush 0 Done Done Sent false
8307 2820 Acked Read 1 Done Done Sent false
8308 2821 Acked Read 6 Done Done Sent false
8309 2822 Acked Write 2 Done Done Sent false
8310 2823 Acked Read 2 Done Done Sent false
8311 2824 Acked Read 8 Done Done Sent false
8312 2825 Acked Write 10 Done Done Sent false
8313 2826 Acked Read 3 Done Done Sent false
8314 2827 Acked Read 4 Done Done Sent false
8315 2828 Acked Write 10 Done Done Sent false
8316 2829 Acked Read 10 Done Done Sent false
8317 2830 Acked Write 9 Done Done Sent false
8318 2831 Acked Read 3 Done Done Sent false
8319 2832 Acked Flush 0 Done Done Sent false
8320 2833 Acked Read 9 Done Done Sent false
8321 2834 Acked Flush 0 Done Done Sent false
8322 2835 Acked Read 2 Done Done Sent false
8323 2836 Acked Read 8 Done Done Sent false
8324 2837 Acked Read 9 Done Done Sent false
8325 2838 Acked Read 1 Done Done Sent false
8326 2839 Acked Read 5 Done Done Sent false
8327 2840 Acked Read 6 Done Done Sent false
8328 2841 Acked Write 2 Done Done Sent false
8329 2842 Acked Read 9 Done Done Sent false
8330 2843 Acked Write 8 Done Done Sent false
8331 2844 Acked Write 4 Done Done Sent false
8332 2845 Acked Write 4 Done Done Sent false
8333 2846 Acked Write 6 Done Done Sent false
8334 2847 Acked Read 4 Done Done Sent false
8335 2848 Acked Write 3 Done Done Sent false
8336 2849 Acked Write 3 Done Done Sent false
8337 2850 Acked Read 4 Done Done Sent false
8338 2851 Acked Write 3 Done Done Sent false
8339 2852 Acked Read 9 Done Done Sent false
8340 2853 Acked Write 4 Done Done Sent false
8341 2854 Acked Read 7 Done Done Sent false
8342 2855 Acked Write 7 Done Done Sent false
8343 2856 Acked Read 3 Done Done Sent false
8344 2857 Acked Read 2 Done Done Sent false
8345 2858 Acked Read 10 Done Done Sent false
8346 2859 Acked Write 2 Done Done Sent false
8347 2860 Acked Write 10 Done Done Sent false
8348 2861 Acked Read 2 Done Done Sent false
8349 2862 Acked Write 9 Done Done Sent false
8350 2863 Acked Read 3 Done Done Sent false
8351 2864 Acked Flush 0 Done Done Sent false
8352 2865 Acked Write 8 Done Done Sent false
8353 2866 Acked Read 6 Done Done Sent false
8354 2867 Acked Flush 0 Done Done Sent false
8355 2868 Acked Read 10 Done Done Sent false
8356 2869 Acked Write 10 Done Done Sent false
8357 2870 Acked Write 7 Done Done Sent false
8358 2871 Acked Read 5 Done Done Sent false
8359 2872 Acked Read 3 Done Done Sent false
8360 2873 Acked Write 8 Done Done Sent false
8361 2874 Acked Write 8 Done Done Sent false
8362 2875 Acked Write 2 Done Done Sent false
8363 2876 Acked Write 8 Done Done Sent false
8364 2877 Acked Write 7 Done Done Sent false
8365 2878 Acked Write 3 Done Done Sent false
8366 2879 Acked Read 4 Done Done Sent false
8367 2880 Acked Write 6 Done Done Sent false
8368 2881 Acked Write 4 Done Done Sent false
8369 2882 Acked Read 7 Done Done Sent false
8370 2883 Acked Write 2 Done Done Sent false
8371 2884 Acked Read 2 Done Done Sent false
8372 2885 Acked Read 7 Done Done Sent false
8373 2886 Acked Write 8 Done Done Sent false
8374 2887 Acked Write 9 Done Done Sent false
8375 2888 Acked Write 5 Done Done Sent false
8376 2889 Acked Read 4 Done Done Sent false
8377 2890 Acked Read 9 Done Done Sent false
8378 2891 Acked Flush 0 Done Done Sent false
8379 2892 Acked Read 4 Done Done Sent false
8380 2893 Acked Write 10 Done Done Sent false
8381 2894 Acked Flush 0 Done Done Sent false
8382 2895 Acked Read 2 Done Done Sent false
8383 2896 Acked Read 1 Done Done Sent false
8384 2897 Acked Write 10 Done Done Sent false
8385 2898 Acked Read 3 Done Done Sent false
8386 2899 Acked Read 8 Done Done Sent false
8387 2900 Acked Write 7 Done Done Sent false
8388 2901 Acked Read 8 Done Done Sent false
8389 2902 Acked Write 8 Done Done Sent false
8390 2903 Acked Read 4 Done Done Sent false
8391 2904 Acked Write 10 Done Done Sent false
8392 2905 Acked Read 6 Done Done Sent false
8393 2906 Acked Write 3 Done Done Sent false
8394 2907 Acked Read 8 Done Done Sent false
8395 2908 Acked Read 7 Done Done Sent false
8396 2909 Acked Read 5 Done Done Sent false
8397 2910 Acked Read 1 Done Done Sent false
8398 2911 Acked Write 7 Done Done Sent false
8399 2912 Acked Write 4 Done Done Sent false
8400 2913 Acked Flush 0 Done Done Sent false
8401 2914 Acked Write 1 Done Done Sent false
8402 2915 Acked Read 10 Done Done Sent false
8403 2916 Acked Flush 0 Done Done Sent false
8404 2917 Acked Read 6 Done Done Sent false
8405 2918 Acked Write 3 Done Done Sent false
8406 2919 Acked Write 1 Done Done Sent false
8407 2920 Acked Write 10 Done Done Sent false
8408 2921 Acked Flush 0 Done Done Sent false
8409 2922 Acked Write 10 Done Done Sent false
8410 2923 Acked Read 2 Done Done Sent false
8411 2924 Acked Read 10 Done Done Sent false
8412 2925 Acked Write 5 Done Done Sent false
8413 2926 Acked Flush 0 Done Done Sent false
8414 2927 Acked Read 6 Done Done Sent false
8415 2928 Acked Write 5 Done Done Sent false
8416 2929 Acked Write 1 Done Done Sent false
8417 2930 Acked Write 3 Done Done Sent false
8418 2931 Acked Read 10 Done Done Sent false
8419 2932 Acked Read 6 Done Done Sent false
8420 2933 Acked Write 6 Done Done Sent false
8421 2934 Acked Read 9 Done Done Sent false
8422 2935 Acked Write 7 Done Done Sent false
8423 2936 Acked Write 10 Done Done Sent false
8424 2937 Acked Read 10 Done Done Sent false
8425 2938 Acked Read 8 Done Done Sent false
8426 2939 Acked Read 9 Done Done Sent false
8427 2940 Acked Write 3 Done Done Sent false
8428 2941 Acked Write 10 Done Done Sent false
8429 2942 Acked Write 3 Done Done Sent false
8430 2943 Acked Read 9 Done Done Sent false
8431 2944 Acked Read 4 Done Done Sent false
8432 2945 Acked Read 4 Done Done Sent false
8433 2946 Acked Read 7 Done Done Sent false
8434 2947 Acked Read 10 Done Done Sent false
8435 2948 Acked Flush 0 Done Done Sent false
8436 2949 Acked Flush 0 Done Done Sent false
8437 2950 Acked Read 7 Done Done Sent false
8438 2951 Acked Write 8 Done Done Sent false
8439 2952 Acked Write 3 Done Done Sent false
8440 2953 Acked Read 1 Done Done Sent false
8441 2954 Acked Write 4 Done Done Sent false
8442 2955 Acked Read 10 Done Done Sent false
8443 2956 Acked Write 5 Done Done Sent false
8444 2957 Acked Write 10 Done Done Sent false
8445 2958 Acked Flush 0 Done Done Sent false
8446 2959 Acked Read 4 Done Done Sent false
8447 2960 Acked Read 6 Done Done Sent false
8448 2961 Acked Flush 0 Done Done Sent false
8449 2962 Acked Write 7 Done Done Sent false
8450 2963 Acked Read 10 Done Done Sent false
8451 2964 Acked Read 8 Done Done Sent false
8452 2965 Acked Read 6 Done Done Sent false
8453 2966 Acked Write 10 Done Done Sent false
8454 2967 Acked Flush 0 Done Done Sent false
8455 2968 Acked Write 9 Done Done Sent false
8456 2969 Acked Read 3 Done Done Sent false
8457 2970 Acked Read 3 Done Done Sent false
8458 2971 Acked Write 6 Done Done Sent false
8459 2972 Acked Read 5 Done Done Sent false
8460 2973 Acked Read 9 Done Done Sent false
8461 2974 Acked Write 2 Done Done Sent false
8462 2975 Acked Write 10 Done Done Sent false
8463 2976 Acked Read 1 Done Done Sent false
8464 2977 Acked Write 4 Done Done Sent false
8465 2978 Acked Read 6 Done Done Sent false
8466 2979 Acked Write 6 Done Done Sent false
8467 2980 Acked Write 2 Done Done Sent false
8468 2981 Acked Write 4 Done Done Sent false
8469 2982 Acked Write 8 Done Done Sent false
8470 2983 Acked Write 5 Done Done Sent false
8471 2984 Acked Read 2 Done Done Sent false
8472 2985 Acked Read 4 Done Done Sent false
8473 2986 Acked Read 7 Done Done Sent false
8474 2987 Acked Write 6 Done Done Sent false
8475 2988 Acked Write 1 Done Done Sent false
8476 2989 Acked Read 10 Done Done Sent false
8477 2990 Acked Read 2 Done Done Sent false
8478 2991 Acked Read 1 Done Done Sent false
8479 2992 Acked Read 8 Done Done Sent false
8480 2993 Acked Write 6 Done Done Sent false
8481 2994 Acked Write 1 Done Done Sent false
8482 2995 Acked Read 4 Done Done Sent false
8483 2996 Acked Read 8 Done Done Sent false
8484 2997 Acked Read 8 Done Done Sent false
8485 2998 Acked Write 5 Done Done Sent false
8486 2999 Acked Flush 0 Done Done Sent false
8487 3000 Acked Read 2 Done Done Sent false
8488 3001 Acked Flush 0 Done Done Sent false
8489 3002 Acked Write 2 Done Done Sent false
8490 3003 Acked Write 1 Done Done Sent false
8491 3004 Acked Read 4 Done Done Sent false
8492 3005 Acked Write 1 Done Done Sent false
8493 3006 Acked Flush 0 Done Done Sent false
8494 3007 Acked Read 10 Done Done Sent false
8495 3008 Acked Flush 0 Done Done Sent false
8496 3009 Acked Read 9 Done Done Sent false
8497 3010 Acked Write 10 Done Done Sent false
8498 3011 Acked Read 9 Done Done Sent false
8499 3012 Acked Write 8 Done Done Sent false
8500 3013 Acked Read 4 Done Done Sent false
8501 3014 Acked Write 8 Done Done Sent false
8502 3015 Acked Write 9 Done Done Sent false
8503 3016 Acked Read 5 Done Done Sent false
8504 3017 Acked Write 9 Done Done Sent false
8505 3018 Acked Write 10 Done Done Sent false
8506 3019 Acked Read 3 Done Done Sent false
8507 3020 Acked Read 4 Done Done Sent false
8508 3021 Acked Read 4 Done Done Sent false
8509 3022 Acked Read 7 Done Done Sent false
8510 3023 Acked Write 5 Done Done Sent false
8511 3024 Acked Write 1 Done Done Sent false
8512 3025 Acked Read 5 Done Done Sent false
8513 3026 Acked Flush 0 Done Done Sent false
8514 3027 Acked Write 2 Done Done Sent false
8515 3028 Acked Read 9 Done Done Sent false
8516 3029 Acked Read 5 Done Done Sent false
8517 3030 Acked Read 10 Done Done Sent false
8518 3031 Acked Write 8 Done Done Sent false
8519 3032 Acked Write 2 Done Done Sent false
8520 3033 Acked Read 2 Done Done Sent false
8521 3034 Acked Read 5 Done Done Sent false
8522 3035 Acked Read 8 Done Done Sent false
8523 3036 Acked Write 7 Done Done Sent false
8524 3037 Acked Read 8 Done Done Sent false
8525 3038 Acked Write 8 Done Done Sent false
8526 3039 Acked Read 7 Done Done Sent false
8527 3040 Acked Write 2 Done Done Sent false
8528 3041 Acked Flush 0 Done Done Sent false
8529 3042 Acked Read 8 Done Done Sent false
8530 3043 Acked Write 5 Done Done Sent false
8531 3044 Acked Write 5 Done Done Sent false
8532 3045 Acked Write 10 Done Done Sent false
8533 3046 Acked Write 9 Done Done Sent false
8534 3047 Acked Read 10 Done Done Sent false
8535 3048 Acked Read 2 Done Done Sent false
8536 3049 Acked Write 5 Done Done Sent false
8537 3050 Acked Read 6 Done Done Sent false
8538 3051 Acked Flush 0 Done Done Sent false
8539 3052 Acked Write 2 Done Done Sent false
8540 3053 Acked Read 8 Done Done Sent false
8541 3054 Acked Read 9 Done Done Sent false
8542 3055 Acked Read 5 Done Done Sent false
8543 3056 Acked Read 9 Done Done Sent false
8544 3057 Acked Read 4 Done Done Sent false
8545 3058 Acked Read 3 Done Done Sent false
8546 3059 Acked Read 8 Done Done Sent false
8547 3060 Acked Read 5 Done Done Sent false
8548 3061 Acked Write 6 Done Done Sent false
8549 3062 Acked Read 10 Done Done Sent false
8550 3063 Acked Write 9 Done Done Sent false
8551 3064 Acked Read 6 Done Done Sent false
8552 3065 Acked Read 9 Done Done Sent false
8553 3066 Acked Read 3 Done Done Sent false
8554 3067 Acked Read 9 Done Done Sent false
8555 3068 Acked Read 6 Done Done Sent false
8556 3069 Acked Read 10 Done Done Sent false
8557 3070 Acked Read 8 Done Done Sent false
8558 3071 Acked Read 1 Done Done Sent false
8559 3072 Acked Flush 0 Done Done Sent false
8560 3073 Acked Write 2 Done Done Sent false
8561 3074 Acked Read 1 Done Done Sent false
8562 3075 Acked Write 1 Done Done Sent false
8563 3076 Acked Read 6 Done Done Sent false
8564 3077 Acked Write 8 Done Done Sent false
8565 3078 Acked Write 7 Done Done Sent false
8566 3079 Acked Write 2 Done Done Sent false
8567 3080 Acked Write 6 Done Done Sent false
8568 3081 Acked Read 9 Done Done Sent false
8569 3082 Acked Write 4 Done Done Sent false
8570 3083 Acked Write 5 Done Done Sent false
8571 3084 Acked Flush 0 Done Done Sent false
8572 3085 Acked Read 3 Done Done Sent false
8573 3086 Acked Read 10 Done Done Sent false
8574 3087 Acked Flush 0 Done Done Sent false
8575 3088 Acked Write 3 Done Done Sent false
8576 3089 Acked Write 2 Done Done Sent false
8577 3090 Acked Read 7 Done Done Sent false
8578 3091 Acked Write 5 Done Done Sent false
8579 3092 Acked Write 9 Done Done Sent false
8580 3093 Acked Write 7 Done Done Sent false
8581 3094 Acked Read 1 Done Done Sent false
8582 3095 Acked Write 1 Done Done Sent false
8583 3096 Acked Read 1 Done Done Sent false
8584 3097 Acked Write 9 Done Done Sent false
8585 3098 Acked Read 10 Done Done Sent false
8586 3099 Acked Write 1 Done Done Sent false
8587 3100 Acked Read 5 Done Done Sent false
8588 3101 Acked Write 6 Done Done Sent false
8589 3102 Acked Write 3 Done Done Sent false
8590 3103 Acked Read 10 Done Done Sent false
8591 3104 Acked Read 9 Done Done Sent false
8592 3105 Acked Flush 0 Done Done Sent false
8593 3106 Acked Flush 0 Done Done Sent false
8594 3107 Acked Write 1 Done Done Sent false
8595 3108 Acked Write 7 Done Done Sent false
8596 3109 Acked Read 7 Done Done Sent false
8597 3110 Acked Write 10 Done Done Sent false
8598 3111 Acked Read 10 Done Done Sent false
8599 3112 Acked Flush 0 Done Done Sent false
8600 STATES DS:0 DS:1 DS:2 TOTAL
8601 Sent 0 0 302 302
8602 Done 302 302 0 604
8603 Skipped 0 0 0 0
8604 Error 0 0 0 0
8605 Last Flush: 3112 3112 2810
8606 Downstairs last five completed: 2810 2809 2808 2807 2806
8607 Upstairs last five completed: 3112 3111 3110 3109 3108
8608 CLIENT: Up:0 ds:302 act:2
8609 ----------------------------------------------------------------
8610 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8611 States: Active Active Offline
8612 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8613 2811 Acked Read 5 Done Done Sent false
8614 2812 Acked Flush 0 Done Done Sent false
8615 2813 Acked Read 3 Done Done Sent false
8616 2814 Acked Write 7 Done Done Sent false
8617 2815 Acked Read 6 Done Done Sent false
8618 2816 Acked Write 7 Done Done Sent false
8619 2817 Acked Read 6 Done Done Sent false
8620 2818 Acked Write 3 Done Done Sent false
8621 2819 Acked Flush 0 Done Done Sent false
8622 2820 Acked Read 1 Done Done Sent false
8623 2821 Acked Read 6 Done Done Sent false
8624 2822 Acked Write 2 Done Done Sent false
8625 2823 Acked Read 2 Done Done Sent false
8626 2824 Acked Read 8 Done Done Sent false
8627 2825 Acked Write 10 Done Done Sent false
8628 2826 Acked Read 3 Done Done Sent false
8629 2827 Acked Read 4 Done Done Sent false
8630 2828 Acked Write 10 Done Done Sent false
8631 2829 Acked Read 10 Done Done Sent false
8632 2830 Acked Write 9 Done Done Sent false
8633 2831 Acked Read 3 Done Done Sent false
8634 2832 Acked Flush 0 Done Done Sent false
8635 2833 Acked Read 9 Done Done Sent false
8636 2834 Acked Flush 0 Done Done Sent false
8637 2835 Acked Read 2 Done Done Sent false
8638 2836 Acked Read 8 Done Done Sent false
8639 2837 Acked Read 9 Done Done Sent false
8640 2838 Acked Read 1 Done Done Sent false
8641 2839 Acked Read 5 Done Done Sent false
8642 2840 Acked Read 6 Done Done Sent false
8643 2841 Acked Write 2 Done Done Sent false
8644 2842 Acked Read 9 Done Done Sent false
8645 2843 Acked Write 8 Done Done Sent false
8646 2844 Acked Write 4 Done Done Sent false
8647 2845 Acked Write 4 Done Done Sent false
8648 2846 Acked Write 6 Done Done Sent false
8649 2847 Acked Read 4 Done Done Sent false
8650 2848 Acked Write 3 Done Done Sent false
8651 2849 Acked Write 3 Done Done Sent false
8652 2850 Acked Read 4 Done Done Sent false
8653 2851 Acked Write 3 Done Done Sent false
8654 2852 Acked Read 9 Done Done Sent false
8655 2853 Acked Write 4 Done Done Sent false
8656 2854 Acked Read 7 Done Done Sent false
8657 2855 Acked Write 7 Done Done Sent false
8658 2856 Acked Read 3 Done Done Sent false
8659 2857 Acked Read 2 Done Done Sent false
8660 2858 Acked Read 10 Done Done Sent false
8661 2859 Acked Write 2 Done Done Sent false
8662 2860 Acked Write 10 Done Done Sent false
8663 2861 Acked Read 2 Done Done Sent false
8664 2862 Acked Write 9 Done Done Sent false
8665 2863 Acked Read 3 Done Done Sent false
8666 2864 Acked Flush 0 Done Done Sent false
8667 2865 Acked Write 8 Done Done Sent false
8668 2866 Acked Read 6 Done Done Sent false
8669 2867 Acked Flush 0 Done Done Sent false
8670 2868 Acked Read 10 Done Done Sent false
8671 2869 Acked Write 10 Done Done Sent false
8672 2870 Acked Write 7 Done Done Sent false
8673 2871 Acked Read 5 Done Done Sent false
8674 2872 Acked Read 3 Done Done Sent false
8675 2873 Acked Write 8 Done Done Sent false
8676 2874 Acked Write 8 Done Done Sent false
8677 2875 Acked Write 2 Done Done Sent false
8678 2876 Acked Write 8 Done Done Sent false
8679 2877 Acked Write 7 Done Done Sent false
8680 2878 Acked Write 3 Done Done Sent false
8681 2879 Acked Read 4 Done Done Sent false
8682 2880 Acked Write 6 Done Done Sent false
8683 2881 Acked Write 4 Done Done Sent false
8684 2882 Acked Read 7 Done Done Sent false
8685 2883 Acked Write 2 Done Done Sent false
8686 2884 Acked Read 2 Done Done Sent false
8687 2885 Acked Read 7 Done Done Sent false
8688 2886 Acked Write 8 Done Done Sent false
8689 2887 Acked Write 9 Done Done Sent false
8690 2888 Acked Write 5 Done Done Sent false
8691 2889 Acked Read 4 Done Done Sent false
8692 2890 Acked Read 9 Done Done Sent false
8693 2891 Acked Flush 0 Done Done Sent false
8694 2892 Acked Read 4 Done Done Sent false
8695 2893 Acked Write 10 Done Done Sent false
8696 2894 Acked Flush 0 Done Done Sent false
8697 2895 Acked Read 2 Done Done Sent false
8698 2896 Acked Read 1 Done Done Sent false
8699 2897 Acked Write 10 Done Done Sent false
8700 2898 Acked Read 3 Done Done Sent false
8701 2899 Acked Read 8 Done Done Sent false
8702 2900 Acked Write 7 Done Done Sent false
8703 2901 Acked Read 8 Done Done Sent false
8704 2902 Acked Write 8 Done Done Sent false
8705 2903 Acked Read 4 Done Done Sent false
8706 2904 Acked Write 10 Done Done Sent false
8707 2905 Acked Read 6 Done Done Sent false
8708 2906 Acked Write 3 Done Done Sent false
8709 2907 Acked Read 8 Done Done Sent false
8710 2908 Acked Read 7 Done Done Sent false
8711 2909 Acked Read 5 Done Done Sent false
8712 2910 Acked Read 1 Done Done Sent false
8713 2911 Acked Write 7 Done Done Sent false
8714 2912 Acked Write 4 Done Done Sent false
8715 2913 Acked Flush 0 Done Done Sent false
8716 2914 Acked Write 1 Done Done Sent false
8717 2915 Acked Read 10 Done Done Sent false
8718 2916 Acked Flush 0 Done Done Sent false
8719 2917 Acked Read 6 Done Done Sent false
8720 2918 Acked Write 3 Done Done Sent false
8721 2919 Acked Write 1 Done Done Sent false
8722 2920 Acked Write 10 Done Done Sent false
8723 2921 Acked Flush 0 Done Done Sent false
8724 2922 Acked Write 10 Done Done Sent false
8725 2923 Acked Read 2 Done Done Sent false
8726 2924 Acked Read 10 Done Done Sent false
8727 2925 Acked Write 5 Done Done Sent false
8728 2926 Acked Flush 0 Done Done Sent false
8729 2927 Acked Read 6 Done Done Sent false
8730 2928 Acked Write 5 Done Done Sent false
8731 2929 Acked Write 1 Done Done Sent false
8732 2930 Acked Write 3 Done Done Sent false
8733 2931 Acked Read 10 Done Done Sent false
8734 2932 Acked Read 6 Done Done Sent false
8735 2933 Acked Write 6 Done Done Sent false
8736 2934 Acked Read 9 Done Done Sent false
8737 2935 Acked Write 7 Done Done Sent false
8738 2936 Acked Write 10 Done Done Sent false
8739 2937 Acked Read 10 Done Done Sent false
8740 2938 Acked Read 8 Done Done Sent false
8741 2939 Acked Read 9 Done Done Sent false
8742 2940 Acked Write 3 Done Done Sent false
8743 2941 Acked Write 10 Done Done Sent false
8744 2942 Acked Write 3 Done Done Sent false
8745 2943 Acked Read 9 Done Done Sent false
8746 2944 Acked Read 4 Done Done Sent false
8747 2945 Acked Read 4 Done Done Sent false
8748 2946 Acked Read 7 Done Done Sent false
8749 2947 Acked Read 10 Done Done Sent false
8750 2948 Acked Flush 0 Done Done Sent false
8751 2949 Acked Flush 0 Done Done Sent false
8752 2950 Acked Read 7 Done Done Sent false
8753 2951 Acked Write 8 Done Done Sent false
8754 2952 Acked Write 3 Done Done Sent false
8755 2953 Acked Read 1 Done Done Sent false
8756 2954 Acked Write 4 Done Done Sent false
8757 2955 Acked Read 10 Done Done Sent false
8758 2956 Acked Write 5 Done Done Sent false
8759 2957 Acked Write 10 Done Done Sent false
8760 2958 Acked Flush 0 Done Done Sent false
8761 2959 Acked Read 4 Done Done Sent false
8762 2960 Acked Read 6 Done Done Sent false
8763 2961 Acked Flush 0 Done Done Sent false
8764 2962 Acked Write 7 Done Done Sent false
8765 2963 Acked Read 10 Done Done Sent false
8766 2964 Acked Read 8 Done Done Sent false
8767 2965 Acked Read 6 Done Done Sent false
8768 2966 Acked Write 10 Done Done Sent false
8769 2967 Acked Flush 0 Done Done Sent false
8770 2968 Acked Write 9 Done Done Sent false
8771 2969 Acked Read 3 Done Done Sent false
8772 2970 Acked Read 3 Done Done Sent false
8773 2971 Acked Write 6 Done Done Sent false
8774 2972 Acked Read 5 Done Done Sent false
8775 2973 Acked Read 9 Done Done Sent false
8776 2974 Acked Write 2 Done Done Sent false
8777 2975 Acked Write 10 Done Done Sent false
8778 2976 Acked Read 1 Done Done Sent false
8779 2977 Acked Write 4 Done Done Sent false
8780 2978 Acked Read 6 Done Done Sent false
8781 2979 Acked Write 6 Done Done Sent false
8782 2980 Acked Write 2 Done Done Sent false
8783 2981 Acked Write 4 Done Done Sent false
8784 2982 Acked Write 8 Done Done Sent false
8785 2983 Acked Write 5 Done Done Sent false
8786 2984 Acked Read 2 Done Done Sent false
8787 2985 Acked Read 4 Done Done Sent false
8788 2986 Acked Read 7 Done Done Sent false
8789 2987 Acked Write 6 Done Done Sent false
8790 2988 Acked Write 1 Done Done Sent false
8791 2989 Acked Read 10 Done Done Sent false
8792 2990 Acked Read 2 Done Done Sent false
8793 2991 Acked Read 1 Done Done Sent false
8794 2992 Acked Read 8 Done Done Sent false
8795 2993 Acked Write 6 Done Done Sent false
8796 2994 Acked Write 1 Done Done Sent false
8797 2995 Acked Read 4 Done Done Sent false
8798 2996 Acked Read 8 Done Done Sent false
8799 2997 Acked Read 8 Done Done Sent false
8800 2998 Acked Write 5 Done Done Sent false
8801 2999 Acked Flush 0 Done Done Sent false
8802 3000 Acked Read 2 Done Done Sent false
8803 3001 Acked Flush 0 Done Done Sent false
8804 3002 Acked Write 2 Done Done Sent false
8805 3003 Acked Write 1 Done Done Sent false
8806 3004 Acked Read 4 Done Done Sent false
8807 3005 Acked Write 1 Done Done Sent false
8808 3006 Acked Flush 0 Done Done Sent false
8809 3007 Acked Read 10 Done Done Sent false
8810 3008 Acked Flush 0 Done Done Sent false
8811 3009 Acked Read 9 Done Done Sent false
8812 3010 Acked Write 10 Done Done Sent false
8813 3011 Acked Read 9 Done Done Sent false
8814 3012 Acked Write 8 Done Done Sent false
8815 3013 Acked Read 4 Done Done Sent false
8816 3014 Acked Write 8 Done Done Sent false
8817 3015 Acked Write 9 Done Done Sent false
8818 3016 Acked Read 5 Done Done Sent false
8819 3017 Acked Write 9 Done Done Sent false
8820 3018 Acked Write 10 Done Done Sent false
8821 3019 Acked Read 3 Done Done Sent false
8822 3020 Acked Read 4 Done Done Sent false
8823 3021 Acked Read 4 Done Done Sent false
8824 3022 Acked Read 7 Done Done Sent false
8825 3023 Acked Write 5 Done Done Sent false
8826 3024 Acked Write 1 Done Done Sent false
8827 3025 Acked Read 5 Done Done Sent false
8828 3026 Acked Flush 0 Done Done Sent false
8829 3027 Acked Write 2 Done Done Sent false
8830 3028 Acked Read 9 Done Done Sent false
8831 3029 Acked Read 5 Done Done Sent false
8832 3030 Acked Read 10 Done Done Sent false
8833 3031 Acked Write 8 Done Done Sent false
8834 3032 Acked Write 2 Done Done Sent false
8835 3033 Acked Read 2 Done Done Sent false
8836 3034 Acked Read 5 Done Done Sent false
8837 3035 Acked Read 8 Done Done Sent false
8838 3036 Acked Write 7 Done Done Sent false
8839 3037 Acked Read 8 Done Done Sent false
8840 3038 Acked Write 8 Done Done Sent false
8841 3039 Acked Read 7 Done Done Sent false
8842 3040 Acked Write 2 Done Done Sent false
8843 3041 Acked Flush 0 Done Done Sent false
8844 3042 Acked Read 8 Done Done Sent false
8845 3043 Acked Write 5 Done Done Sent false
8846 3044 Acked Write 5 Done Done Sent false
8847 3045 Acked Write 10 Done Done Sent false
8848 3046 Acked Write 9 Done Done Sent false
8849 3047 Acked Read 10 Done Done Sent false
8850 3048 Acked Read 2 Done Done Sent false
8851 3049 Acked Write 5 Done Done Sent false
8852 3050 Acked Read 6 Done Done Sent false
8853 3051 Acked Flush 0 Done Done Sent false
8854 3052 Acked Write 2 Done Done Sent false
8855 3053 Acked Read 8 Done Done Sent false
8856 3054 Acked Read 9 Done Done Sent false
8857 3055 Acked Read 5 Done Done Sent false
8858 3056 Acked Read 9 Done Done Sent false
8859 3057 Acked Read 4 Done Done Sent false
8860 3058 Acked Read 3 Done Done Sent false
8861 3059 Acked Read 8 Done Done Sent false
8862 3060 Acked Read 5 Done Done Sent false
8863 3061 Acked Write 6 Done Done Sent false
8864 3062 Acked Read 10 Done Done Sent false
8865 3063 Acked Write 9 Done Done Sent false
8866 3064 Acked Read 6 Done Done Sent false
8867 3065 Acked Read 9 Done Done Sent false
8868 3066 Acked Read 3 Done Done Sent false
8869 3067 Acked Read 9 Done Done Sent false
8870 3068 Acked Read 6 Done Done Sent false
8871 3069 Acked Read 10 Done Done Sent false
8872 3070 Acked Read 8 Done Done Sent false
8873 3071 Acked Read 1 Done Done Sent false
8874 3072 Acked Flush 0 Done Done Sent false
8875 3073 Acked Write 2 Done Done Sent false
8876 3074 Acked Read 1 Done Done Sent false
8877 3075 Acked Write 1 Done Done Sent false
8878 3076 Acked Read 6 Done Done Sent false
8879 3077 Acked Write 8 Done Done Sent false
8880 3078 Acked Write 7 Done Done Sent false
8881 3079 Acked Write 2 Done Done Sent false
8882 3080 Acked Write 6 Done Done Sent false
8883 3081 Acked Read 9 Done Done Sent false
8884 3082 Acked Write 4 Done Done Sent false
8885 3083 Acked Write 5 Done Done Sent false
8886 3084 Acked Flush 0 Done Done Sent false
8887 3085 Acked Read 3 Done Done Sent false
8888 3086 Acked Read 10 Done Done Sent false
8889 3087 Acked Flush 0 Done Done Sent false
8890 3088 Acked Write 3 Done Done Sent false
8891 3089 Acked Write 2 Done Done Sent false
8892 3090 Acked Read 7 Done Done Sent false
8893 3091 Acked Write 5 Done Done Sent false
8894 3092 Acked Write 9 Done Done Sent false
8895 3093 Acked Write 7 Done Done Sent false
8896 3094 Acked Read 1 Done Done Sent false
8897 3095 Acked Write 1 Done Done Sent false
8898 3096 Acked Read 1 Done Done Sent false
8899 3097 Acked Write 9 Done Done Sent false
8900 3098 Acked Read 10 Done Done Sent false
8901 3099 Acked Write 1 Done Done Sent false
8902 3100 Acked Read 5 Done Done Sent false
8903 3101 Acked Write 6 Done Done Sent false
8904 3102 Acked Write 3 Done Done Sent false
8905 3103 Acked Read 10 Done Done Sent false
8906 3104 Acked Read 9 Done Done Sent false
8907 3105 Acked Flush 0 Done Done Sent false
8908 3106 Acked Flush 0 Done Done Sent false
8909 3107 Acked Write 1 Done Done Sent false
8910 3108 Acked Write 7 Done Done Sent false
8911 3109 Acked Read 7 Done Done Sent false
8912 3110 Acked Write 10 Done Done Sent false
8913 3111 Acked Read 10 Done Done Sent false
8914 3112 Acked Flush 0 Done Done Sent false
8915 STATES DS:0 DS:1 DS:2 TOTAL
8916 Sent 0 0 302 302
8917 Done 302 302 0 604
8918 Skipped 0 0 0 0
8919 Error 0 0 0 0
8920 Last Flush: 3112 3112 2810
8921 Downstairs last five completed: 2810 2809 2808 2807 2806
8922 Upstairs last five completed: 3112 3111 3110 3109 3108
8923 CLIENT: Up:0 ds:302 act:2
8924 [8830] got start action
8925 [8830] I am going to restart
8926 Starting downstairs at port 8830
8927 Make output file at "/tmp/dsc/downstairs-8830.txt"
8928 [8830][2] reports Starting
8929 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1212)
8930 [8830][2] reports Running
89312025-01-03T08:32:47.053ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89322025-01-03T08:32:47.053ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50476) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89332025-01-03T08:32:47.054ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89342025-01-03T08:32:47.055ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89352025-01-03T08:32:47.055ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89362025-01-03T08:32:47.055ZINFOcrucible: send last flush ID to this DS: 2810 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89372025-01-03T08:32:47.055ZINFOcrucible: Replied this last flush ID: 2810 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89382025-01-03T08:32:47.055ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89392025-01-03T08:32:47.055ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89402025-01-03T08:32:47.055ZINFOcrucible: [2] client re-new 302 jobs since flush 2810 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89412025-01-03T08:32:47.055ZINFOcrucible: [2] Replayed 302 jobs since flush: 2810 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
8942 ----------------------------------------------------------------
8943 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8944 Downstairs last five completed: 3112 3111 3110 3109 3108
8945 Upstairs last five completed: 3112 3111 3110 3109 3108
8946 CLIENT: Up:0 ds:0 act:3
8947 Replay: All jobs finished
8948 Main task has work to do, go find it
8949 got dsc Stop(2)
8950 stop 2
8951 Jan 03 08:32:47.313 INFO request completed, latency_us: 367, response_code: 204, uri: /stop/cid/2, method: POST, [8830] Got stop action so:false kr:false
8952 req_id: 9c77bc73-ee65-41ec-a016-a86dfac942e2, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
89532025-01-03T08:32:47.314ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89542025-01-03T08:32:47.314ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
89552025-01-03T08:32:47.314ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
8956 Jan 03 08:32:47.314 INFO request completed{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:32:47.314401477Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
8957 , latency_us: 202, response_code: 200, uri: /state/cid/2, method: GET, req_id: 81436bbb-b262-442e-9173-338754c58b05, {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:32:47.314435886Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
8958 remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
89592025-01-03T08:32:47.314ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
8960 [8830] Exited with: ExitStatus(unix_wait_status(9))
8961 [8830][2] reports Exit
8962 Jan 03 08:32:51.316 INFO request completed, latency_us: 211, response_code: 200, uri: /state/cid/2, method: GET, req_id: 380154c1-0889-45a7-9eb4-3aee2b4df282, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
8963 001/300 Read block 2660 len 36864
8964 002/300 Write block 3544 len 24576 data: 5 5 5 4 4 3
8965 003/300 Read block 1380 len 28672
8966 004/300 Read block 45 len 4096
8967 005/300 Read block 4087 len 12288
8968 006/300 Read block 4385 len 32768
8969 007/300 Flush
8970 008/300 Read block 4133 len 16384
8971 009/300 Read block 3692 len 4096
8972 010/300 Write block 1911 len 24576 data: 3 3 3 3 3 3
8973 011/300 Write block 2099 len 4096 data: 2
8974 012/300 Write block 2298 len 28672 data: 2 3 4 4 4 3 3
8975 013/300 Write block 4074 len 40960 data: 3 3 3 3 2 2 2 2 2 2
8976 014/300 Read block 2621 len 32768
8977 015/300 Flush
8978 016/300 Read block 3054 len 40960
8979 017/300 Read block 3591 len 12288
8980 018/300 Read block 2621 len 12288
8981 019/300 Write block 339 len 20480 data: 2 2 3 3 3
8982 020/300 Write block 4512 len 20480 data: 3 3 3 3 2
8983 021/300 Read block 4806 len 12288
8984 022/300 Read block 756 len 40960
8985 023/300 Read block 331 len 28672
8986 024/300 Write block 2520 len 16384 data: 4 4 4 4
8987 025/300 Write block 2407 len 12288 data: 2 2 2
8988 026/300 Write block 3466 len 16384 data: 4 5 5 5
8989 027/300 Write block 3519 len 16384 data: 2 2 2 2
8990 028/300 Flush
8991 029/300 Read block 1899 len 40960
8992 030/300 Read block 2363 len 32768
8993 031/300 Write block 742 len 20480 data: 2 2 2 4 4
8994 032/300 Read block 1221 len 40960
8995 033/300 Read block 2523 len 36864
8996 034/300 Read block 2553 len 32768
8997 035/300 Write block 2097 len 8192 data: 2 2
8998 036/300 Read block 1826 len 16384
8999 037/300 Write block 2523 len 20480 data: 5 3 3 3 2
9000 038/300 Read block 19 len 4096
9001 039/300 Write block 2284 len 12288 data: 2 2 2
9002 040/300 Write block 241 len 12288 data: 3 2 2
9003 041/300 Read block 3917 len 8192
9004 042/300 Read block 3251 len 40960
9005 043/300 Read block 2574 len 4096
9006 044/300 Flush
9007 045/300 Write block 638 len 28672 data: 6 6 6 5 4 3 2
9008 046/300 Read block 568 len 32768
9009 047/300 Write block 3856 len 36864 data: 2 2 2 2 3 3 4 4 3
9010 048/300 Write block 1959 len 24576 data: 3 2 2 2 2 2
9011 049/300 Write block 651 len 40960 data: 4 3 3 5 5 4 4 4 3 2
9012 050/300 Flush
9013 051/300 Write block 2908 len 12288 data: 2 3 3
9014 052/300 Read block 1645 len 28672
9015 053/300 Read block 2011 len 28672
9016 054/300 Read block 1868 len 12288
9017 055/300 Read block 3899 len 40960
9018 056/300 Read block 2976 len 4096
9019 057/300 Write block 1728 len 36864 data: 2 2 2 2 2 2 2 3 5
9020 058/300 Read block 3201 len 24576
9021 059/300 Read block 646 len 24576
9022 060/300 Read block 3163 len 8192
9023 061/300 Write block 3230 len 40960 data: 2 2 3 3 3 3 2 3 3 3
9024 062/300 Write block 4333 len 8192 data: 2 3
9025 063/300 Write block 255 len 12288 data: 3 2 3
9026 064/300 Read block 1322 len 20480
9027 065/300 Write block 2721 len 40960 data: 2 2 2 2 2 2 2 2 2 2
9028 066/300 Read block 571 len 12288
9029 067/300 Flush
9030 068/300 Write block 2034 len 12288 data: 2 2 2
9031 069/300 Write block 1989 len 20480 data: 3 3 3 4 3
9032 070/300 Read block 222 len 12288
9033 071/300 Read block 1661 len 12288
9034 072/300 Read block 3654 len 16384
9035 073/300 Write block 3956 len 12288 data: 3 3 3
9036 074/300 Write block 286 len 32768 data: 3 3 3 3 3 3 3 3
9037 075/300 Flush
9038 076/300 Write block 2836 len 24576 data: 2 2 2 3 4 4
9039 077/300 Read block 2407 len 20480
9040 078/300 Read block 3493 len 16384
9041 079/300 Write block 2634 len 8192 data: 2 2
9042 080/300 Read block 26 len 40960
9043 081/300 Write block 3454 len 28672 data: 2 2 3 3 3 3 3
9044 082/300 Read block 4720 len 12288
9045 083/300 Read block 604 len 28672
9046 084/300 Read block 3943 len 8192
9047 085/300 Write block 1396 len 20480 data: 4 3 3 3 2
9048 086/300 Read block 1551 len 32768
9049 087/300 Read block 4274 len 24576
9050 088/300 Write block 3469 len 8192 data: 6 5
9051 089/300 Write block 295 len 12288 data: 2 2 2
9052 090/300 Read block 1778 len 16384
9053 091/300 Write block 307 len 40960 data: 4 5 6 6 5 5 5 5 4 4
9054 092/300 Write block 3587 len 4096 data: 3
9055 093/300 Read block 3967 len 40960
9056 094/300 Read block 3946 len 24576
9057 095/300 Write block 1182 len 16384 data: 4 4 4 4
9058 096/300 Read block 4670 len 12288
9059 097/300 Read block 4565 len 12288
9060 098/300 Write block 295 len 8192 data: 3 3
9061 099/300 Read block 1543 len 28672
9062 100/300 Read block 404 len 20480
9063 101/300 Read block 1616 len 28672
9064 102/300 Read block 1110 len 32768
9065 103/300 Flush
9066 104/300 Write block 361 len 40960 data: 2 2 2 3 4 4 4 4 4 3
9067 105/300 Read block 3154 len 40960
9068 106/300 Read block 4965 len 40960
9069 107/300 Write block 2169 len 32768 data: 2 2 2 2 2 2 2 2
9070 108/300 Read block 2050 len 4096
9071 109/300 Write block 592 len 40960 data: 3 2 2 2 2 2 2 2 2 2
9072 110/300 Flush
9073 111/300 Read block 1818 len 24576
9074 112/300 Write block 183 len 8192 data: 2 2
9075 113/300 Write block 3336 len 4096 data: 2
9076 114/300 Write block 3046 len 32768 data: 4 4 4 3 2 2 2 3
9077 115/300 Read block 1620 len 4096
9078 116/300 Read block 3602 len 40960
9079 117/300 Read block 2874 len 40960
9080 118/300 Flush
9081 119/300 Write block 4908 len 28672 data: 4 5 4 4 3 2 2
9082 120/300 Flush
9083 121/300 Flush
9084 122/300 Read block 2878 len 28672
9085 123/300 Read block 1450 len 40960
9086 124/300 Write block 2255 len 8192 data: 4 4
9087 125/300 Read block 1335 len 40960
9088 126/300 Write block 3448 len 16384 data: 2 2 2 2
9089 127/300 Read block 4777 len 16384
9090 128/300 Read block 1963 len 32768
9091 129/300 Write block 1317 len 28672 data: 3 2 2 2 2 2 3
9092 130/300 Read block 2786 len 36864
9093 131/300 Read block 2081 len 24576
9094 132/300 Write block 1446 len 4096 data: 5
9095 133/300 Read block 2191 len 20480
9096 134/300 Read block 33 len 8192
9097 135/300 Write block 405 len 36864 data: 3 4 4 4 4 4 3 3 4
9098 136/300 Flush
9099 137/300 Read block 2771 len 28672
9100 138/300 Flush
9101 139/300 Write block 2450 len 36864 data: 2 2 2 4 5 5 5 4 4
9102 140/300 Flush
9103 141/300 Write block 4195 len 32768 data: 2 2 3 3 5 5 5 5
9104 142/300 Write block 3681 len 12288 data: 2 2 2
9105 143/300 Write block 4753 len 36864 data: 2 2 2 3 3 3 3 3 3
9106 144/300 Read block 2147 len 16384
9107 145/300 Write block 2708 len 4096 data: 5
9108 146/300 Read block 4686 len 32768
9109 147/300 Read block 3234 len 32768
9110 148/300 Write block 1195 len 16384 data: 4 3 3 3
9111 149/300 Write block 4165 len 8192 data: 3 4
9112 150/300 Write block 1346 len 36864 data: 3 3 3 2 2 2 3 3 3
9113 151/300 Write block 2598 len 40960 data: 4 4 4 4 4 3 2 2 2 2
9114 152/300 Read block 1931 len 16384
9115 153/300 Write block 2073 len 24576 data: 5 4 4 3 3 3
9116 154/300 Write block 1265 len 20480 data: 3 3 3 3 3
9117 155/300 Read block 3840 len 40960
9118 156/300 Read block 3942 len 12288
9119 157/300 Read block 510 len 28672
9120 158/300 Read block 4309 len 16384
9121 159/300 Read block 2891 len 8192
9122 160/300 Flush
9123 161/300 Read block 2507 len 8192
9124 162/300 Read block 4911 len 12288
9125 163/300 Write block 1882 len 4096 data: 4
9126 164/300 Write block 3929 len 20480 data: 3 3 4 3 3
9127 165/300 Write block 1473 len 4096 data: 2
9128 166/300 Read block 393 len 28672
9129 167/300 Write block 4687 len 28672 data: 2 2 2 4 4 4 5
9130 168/300 Read block 975 len 8192
9131 169/300 Flush
9132 170/300 Flush
9133 171/300 Write block 2682 len 36864 data: 3 3 3 3 3 5 4 4 4
9134 172/300 Write block 2168 len 4096 data: 3
9135 173/300 Read block 150 len 36864
9136 174/300 Write block 4535 len 40960 data: 4 2 2 2 2 2 2 2 2 2
9137 175/300 Read block 2569 len 16384
9138 176/300 Write block 3177 len 36864 data: 4 4 4 3 2 2 3 3 3
9139 177/300 Write block 4593 len 12288 data: 2 2 2
9140 178/300 Write block 1818 len 36864 data: 3 3 3 3 3 2 3 3 3
9141 179/300 Write block 345 len 32768 data: 4 4 4 4 4 4 4 4
9142 180/300 Write block 1498 len 20480 data: 2 2 2 2 2
9143 181/300 Write block 1511 len 8192 data: 2 2
9144 182/300 Read block 4442 len 16384
9145 183/300 Read block 2748 len 40960
9146 184/300 Write block 3246 len 32768 data: 3 3 3 3 3 2 2 2
9147 185/300 Write block 2641 len 8192 data: 4 3
9148 186/300 Read block 903 len 24576
9149 187/300 Write block 1419 len 8192 data: 3 3
9150 188/300 Read block 1352 len 28672
9151 189/300 Write block 2958 len 24576 data: 2 2 2 2 2 2
9152 190/300 Read block 2340 len 32768
9153 191/300 Write block 3024 len 4096 data: 2
9154 192/300 Write block 3535 len 20480 data: 3 2 2 2 2
9155 193/300 Read block 4117 len 4096
9156 194/300 Read block 4165 len 32768
9157 195/300 Read block 1053 len 8192
9158 196/300 Write block 3245 len 4096 data: 3
9159 197/300 Write block 3115 len 20480 data: 2 2 3 3 2
9160 198/300 Read block 4792 len 24576
9161 199/300 Read block 2345 len 4096
9162 200/300 Write block 1014 len 28672 data: 3 3 3 2 3 3 3
9163 201/300 Read block 1267 len 24576
9164 202/300 Write block 4114 len 16384 data: 3 3 4 4
9165 203/300 Write block 947 len 20480 data: 3 3 3 3 3
9166 204/300 Flush
9167 205/300 Read block 4448 len 20480
9168 206/300 Flush
9169 207/300 Write block 2050 len 16384 data: 4 3 4 5
9170 208/300 Write block 4500 len 32768 data: 4 4 3 3 3 3 3 2
9171 209/300 Write block 1385 len 28672 data: 2 2 3 3 2 2 3
9172 210/300 Write block 779 len 28672 data: 2 2 2 2 3 3 3
9173 211/300 Write block 4482 len 32768 data: 2 2 2 2 2 2 2 2
9174 212/300 Write block 553 len 24576 data: 3 3 3 3 3 3
9175 213/300 Read block 510 len 12288
9176 214/300 Write block 4116 len 16384 data: 5 5 4 4
9177 215/300 Read block 3463 len 12288
9178 216/300 Write block 1916 len 36864 data: 4 2 2 4 3 3 4 4 4
9179 217/300 Read block 3366 len 40960
9180 218/300 Flush
9181 219/300 Read block 4391 len 4096
9182 220/300 Read block 715 len 24576
9183 221/300 Write block 3385 len 20480 data: 3 3 3 2 3
9184 222/300 Read block 130 len 8192
9185 223/300 Read block 2763 len 4096
9186 224/300 Write block 2540 len 40960 data: 3 3 3 3 3 3 2 2 2 2
9187 225/300 Write block 4755 len 28672 data: 3 4 4 4 4 4 4
9188 226/300 Read block 1659 len 32768
9189 227/300 Read block 4954 len 4096
9190 228/300 Write block 2520 len 36864 data: 5 5 5 6 4 4 4 3 2
9191 229/300 Write block 215 len 20480 data: 3 2 3 4 3
9192 230/300 Read block 4528 len 20480
9193 231/300 Read block 57 len 16384
9194 232/300 Write block 2624 len 40960 data: 4 5 5 4 4 5 3 4 4 2
9195 233/300 Flush
9196 234/300 Read block 4077 len 20480
9197 235/300 Read block 583 len 24576
9198 236/300 Write block 2749 len 28672 data: 4 3 3 3 2 2 2
9199 237/300 Write block 1668 len 32768 data: 2 2 2 3 3 3 3 5
9200 238/300 Read block 484 len 28672
9201 239/300 Write block 1449 len 28672 data: 3 2 3 3 3 4 4
9202 240/300 Write block 319 len 8192 data: 4 2
9203 241/300 Write block 2969 len 40960 data: 3 4 5 5 5 6 7 6 5 4
9204 242/300 Read block 881 len 32768
9205 243/300 Read block 1566 len 12288
9206 244/300 Read block 3572 len 40960
9207 245/300 Write block 1940 len 16384 data: 3 3 3 3
9208 246/300 Read block 333 len 32768
9209 247/300 Flush
9210 248/300 Write block 4491 len 12288 data: 3 3 3
9211 249/300 Flush
9212 250/300 Read block 964 len 40960
9213 251/300 Read block 4483 len 36864
9214 252/300 Read block 3076 len 40960
9215 253/300 Read block 4318 len 36864
9216 254/300 Write block 1574 len 32768 data: 3 3 3 4 3 2 2 2
9217 255/300 Flush
9218 256/300 Write block 872 len 8192 data: 2 2
9219 257/300 Write block 4183 len 8192 data: 2 2
9220 258/300 Write block 4240 len 8192 data: 2 2
9221 259/300 Write block 4156 len 8192 data: 2 2
9222 260/300 Read block 875 len 4096
9223 261/300 Read block 3055 len 20480
9224 262/300 Read block 1826 len 16384
9225 263/300 Flush
9226 264/300 Read block 3954 len 32768
9227 265/300 Read block 1927 len 4096
9228 266/300 Read block 3022 len 28672
9229 267/300 Read block 3123 len 36864
9230 268/300 Read block 1618 len 40960
9231 269/300 Read block 2349 len 36864
9232 270/300 Write block 3566 len 40960 data: 2 2 2 3 3 3 3 3 3 4
9233 271/300 Read block 1963 len 16384
9234 272/300 Write block 2700 len 40960 data: 3 3 2 2 2 2 2 2 6 5
9235 273/300 Read block 3418 len 12288
9236 274/300 Write block 3569 len 8192 data: 4 4
9237 275/300 Read block 3753 len 8192
9238 276/300 Read block 1486 len 4096
9239 277/300 Write block 1247 len 12288 data: 2 3 4
9240 278/300 Read block 2817 len 40960
9241 279/300 Read block 1830 len 40960
9242 280/300 Write block 4340 len 8192 data: 4 3
9243 281/300 Read block 3590 len 12288
9244 282/300 Read block 1866 len 36864
9245 283/300 Flush
9246 284/300 Read block 3733 len 32768
9247 285/300 Write block 422 len 20480 data: 3 2 2 2 2
9248 286/300 Write block 2288 len 4096 data: 2
9249 287/300 Read block 1949 len 28672
9250 288/300 Read block 580 len 4096
9251 289/300 Write block 3083 len 12288 data: 4 3 4
9252 290/300 Read block 2347 len 4096
9253 291/300 Write block 3703 len 32768 data: 4 4 4 3 3 3 3 3
9254 292/300 Write block 4758 len 12288 data: 5 5 5
9255 293/300 Flush
9256 294/300 Flush
9257 295/300 Write block 2086 len 12288 data: 3 3 3
9258 296/300 Read block 3153 len 24576
9259 297/300 Write block 1265 len 4096 data: 4
9260 298/300 Write block 4538 len 36864 data: 3 3 3 3 3 3 3 2 2
9261 299/300 Read block 3368 len 40960
9262 300/300 Read block 3408 len 36864
9263 301/300 Write block 4957 len 40960 data: 2 2 2 2 2 2 2 2 3 3
9264 Main task has work to do, go find it
9265 got dsc Start(2)
9266 start 2
9267 Jan 03 08:32:51.555 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/2, method: POST, req_id: 9367029c-884c-4cbf-b489-ebe991075559, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
9268 Replay: started 2, returned:Ok(())
9269 ----------------------------------------------------------------
9270 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9271 States: Active Active Offline
9272 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9273 3113 Acked Read 9 Done Done Sent false
9274 3114 Acked Write 6 Done Done Sent false
9275 3115 Acked Read 7 Done Done Sent false
9276 3116 Acked Read 1 Done Done Sent false
9277 3117 Acked Read 3 Done Done Sent false
9278 3118 Acked Read 8 Done Done Sent false
9279 3119 Acked Flush 0 Done Done Sent false
9280 3120 Acked Read 4 Done Done Sent false
9281 3121 Acked Read 1 Done Done Sent false
9282 3122 Acked Write 6 Done Done Sent false
9283 3123 Acked Write 1 Done Done Sent false
9284 3124 Acked Write 7 Done Done Sent false
9285 3125 Acked Write 10 Done Done Sent false
9286 3126 Acked Read 8 Done Done Sent false
9287 3127 Acked Flush 0 Done Done Sent false
9288 3128 Acked Read 10 Done Done Sent false
9289 3129 Acked Read 3 Done Done Sent false
9290 3130 Acked Read 3 Done Done Sent false
9291 3131 Acked Write 5 Done Done Sent false
9292 3132 Acked Write 5 Done Done Sent false
9293 3133 Acked Read 3 Done Done Sent false
9294 3134 Acked Read 10 Done Done Sent false
9295 3135 Acked Read 7 Done Done Sent false
9296 3136 Acked Write 4 Done Done Sent false
9297 3137 Acked Write 3 Done Done Sent false
9298 3138 Acked Write 4 Done Done Sent false
9299 3139 Acked Write 4 Done Done Sent false
9300 3140 Acked Flush 0 Done Done Sent false
9301 3141 Acked Read 10 Done Done Sent false
9302 3142 Acked Read 8 Done Done Sent false
9303 3143 Acked Write 5 Done Done Sent false
9304 3144 Acked Read 10 Done Done Sent false
9305 3145 Acked Read 9 Done Done Sent false
9306 3146 Acked Read 8 Done Done Sent false
9307 3147 Acked Write 2 Done Done Sent false
9308 3148 Acked Read 4 Done Done Sent false
9309 3149 Acked Write 5 Done Done Sent false
9310 3150 Acked Read 1 Done Done Sent false
9311 3151 Acked Write 3 Done Done Sent false
9312 3152 Acked Write 3 Done Done Sent false
9313 3153 Acked Read 2 Done Done Sent false
9314 3154 Acked Read 10 Done Done Sent false
9315 3155 Acked Read 1 Done Done Sent false
9316 3156 Acked Flush 0 Done Done Sent false
9317 3157 Acked Write 7 Done Done Sent false
9318 3158 Acked Read 8 Done Done Sent false
9319 3159 Acked Write 9 Done Done Sent false
9320 3160 Acked Write 6 Done Done Sent false
9321 3161 Acked Write 10 Done Done Sent false
9322 3162 Acked Flush 0 Done Done Sent false
9323 3163 Acked Write 3 Done Done Sent false
9324 3164 Acked Read 7 Done Done Sent false
9325 3165 Acked Read 7 Done Done Sent false
9326 3166 Acked Read 3 Done Done Sent false
9327 3167 Acked Read 10 Done Done Sent false
9328 3168 Acked Read 1 Done Done Sent false
9329 3169 Acked Write 9 Done Done Sent false
9330 3170 Acked Read 6 Done Done Sent false
9331 3171 Acked Read 6 Done Done Sent false
9332 3172 Acked Read 2 Done Done Sent false
9333 3173 Acked Write 10 Done Done Sent false
9334 3174 Acked Write 2 Done Done Sent false
9335 3175 Acked Write 3 Done Done Sent false
9336 3176 Acked Read 5 Done Done Sent false
9337 3177 Acked Write 10 Done Done Sent false
9338 3178 Acked Read 3 Done Done Sent false
9339 3179 Acked Flush 0 Done Done Sent false
9340 3180 Acked Write 3 Done Done Sent false
9341 3181 Acked Write 5 Done Done Sent false
9342 3182 Acked Read 3 Done Done Sent false
9343 3183 Acked Read 3 Done Done Sent false
9344 3184 Acked Read 4 Done Done Sent false
9345 3185 Acked Write 3 Done Done Sent false
9346 3186 Acked Write 8 Done Done Sent false
9347 3187 Acked Flush 0 Done Done Sent false
9348 3188 Acked Write 6 Done Done Sent false
9349 3189 Acked Read 5 Done Done Sent false
9350 3190 Acked Read 4 Done Done Sent false
9351 3191 Acked Write 2 Done Done Sent false
9352 3192 Acked Read 10 Done Done Sent false
9353 3193 Acked Write 7 Done Done Sent false
9354 3194 Acked Read 3 Done Done Sent false
9355 3195 Acked Read 7 Done Done Sent false
9356 3196 Acked Read 2 Done Done Sent false
9357 3197 Acked Write 5 Done Done Sent false
9358 3198 Acked Read 8 Done Done Sent false
9359 3199 Acked Read 6 Done Done Sent false
9360 3200 Acked Write 2 Done Done Sent false
9361 3201 Acked Write 3 Done Done Sent false
9362 3202 Acked Read 4 Done Done Sent false
9363 3203 Acked Write 10 Done Done Sent false
9364 3204 Acked Write 1 Done Done Sent false
9365 3205 Acked Read 10 Done Done Sent false
9366 3206 Acked Read 6 Done Done Sent false
9367 3207 Acked Write 4 Done Done Sent false
9368 3208 Acked Read 3 Done Done Sent false
9369 3209 Acked Read 3 Done Done Sent false
9370 3210 Acked Write 2 Done Done Sent false
9371 3211 Acked Read 7 Done Done Sent false
9372 3212 Acked Read 5 Done Done Sent false
9373 3213 Acked Read 7 Done Done Sent false
9374 3214 Acked Read 8 Done Done Sent false
9375 3215 Acked Flush 0 Done Done Sent false
9376 3216 Acked Write 10 Done Done Sent false
9377 3217 Acked Read 10 Done Done Sent false
9378 3218 Acked Read 10 Done Done Sent false
9379 3219 Acked Write 8 Done Done Sent false
9380 3220 Acked Read 1 Done Done Sent false
9381 3221 Acked Write 10 Done Done Sent false
9382 3222 Acked Flush 0 Done Done Sent false
9383 3223 Acked Read 6 Done Done Sent false
9384 3224 Acked Write 2 Done Done Sent false
9385 3225 Acked Write 1 Done Done Sent false
9386 3226 Acked Write 8 Done Done Sent false
9387 3227 Acked Read 1 Done Done Sent false
9388 3228 Acked Read 10 Done Done Sent false
9389 3229 Acked Read 10 Done Done Sent false
9390 3230 Acked Flush 0 Done Done Sent false
9391 3231 Acked Write 7 Done Done Sent false
9392 3232 Acked Flush 0 Done Done Sent false
9393 3233 Acked Flush 0 Done Done Sent false
9394 3234 Acked Read 7 Done Done Sent false
9395 3235 Acked Read 10 Done Done Sent false
9396 3236 Acked Write 2 Done Done Sent false
9397 3237 Acked Read 10 Done Done Sent false
9398 3238 Acked Write 4 Done Done Sent false
9399 3239 Acked Read 4 Done Done Sent false
9400 3240 Acked Read 8 Done Done Sent false
9401 3241 Acked Write 7 Done Done Sent false
9402 3242 Acked Read 9 Done Done Sent false
9403 3243 Acked Read 6 Done Done Sent false
9404 3244 Acked Write 1 Done Done Sent false
9405 3245 Acked Read 5 Done Done Sent false
9406 3246 Acked Read 2 Done Done Sent false
9407 3247 Acked Write 9 Done Done Sent false
9408 3248 Acked Flush 0 Done Done Sent false
9409 3249 Acked Read 7 Done Done Sent false
9410 3250 Acked Flush 0 Done Done Sent false
9411 3251 Acked Write 9 Done Done Sent false
9412 3252 Acked Flush 0 Done Done Sent false
9413 3253 Acked Write 8 Done Done Sent false
9414 3254 Acked Write 3 Done Done Sent false
9415 3255 Acked Write 9 Done Done Sent false
9416 3256 Acked Read 4 Done Done Sent false
9417 3257 Acked Write 1 Done Done Sent false
9418 3258 Acked Read 8 Done Done Sent false
9419 3259 Acked Read 8 Done Done Sent false
9420 3260 Acked Write 4 Done Done Sent false
9421 3261 Acked Write 2 Done Done Sent false
9422 3262 Acked Write 9 Done Done Sent false
9423 3263 Acked Write 10 Done Done Sent false
9424 3264 Acked Read 4 Done Done Sent false
9425 3265 Acked Write 6 Done Done Sent false
9426 3266 Acked Write 5 Done Done Sent false
9427 3267 Acked Read 10 Done Done Sent false
9428 3268 Acked Read 3 Done Done Sent false
9429 3269 Acked Read 7 Done Done Sent false
9430 3270 Acked Read 4 Done Done Sent false
9431 3271 Acked Read 2 Done Done Sent false
9432 3272 Acked Flush 0 Done Done Sent false
9433 3273 Acked Read 2 Done Done Sent false
9434 3274 Acked Read 3 Done Done Sent false
9435 3275 Acked Write 1 Done Done Sent false
9436 3276 Acked Write 5 Done Done Sent false
9437 3277 Acked Write 1 Done Done Sent false
9438 3278 Acked Read 7 Done Done Sent false
9439 3279 Acked Write 7 Done Done Sent false
9440 3280 Acked Read 2 Done Done Sent false
9441 3281 Acked Flush 0 Done Done Sent false
9442 3282 Acked Flush 0 Done Done Sent false
9443 3283 Acked Write 9 Done Done Sent false
9444 3284 Acked Write 1 Done Done Sent false
9445 3285 Acked Read 9 Done Done Sent false
9446 3286 Acked Write 10 Done Done Sent false
9447 3287 Acked Read 4 Done Done Sent false
9448 3288 Acked Write 9 Done Done Sent false
9449 3289 Acked Write 3 Done Done Sent false
9450 3290 Acked Write 9 Done Done Sent false
9451 3291 Acked Write 8 Done Done Sent false
9452 3292 Acked Write 5 Done Done Sent false
9453 3293 Acked Write 2 Done Done Sent false
9454 3294 Acked Read 4 Done Done Sent false
9455 3295 Acked Read 10 Done Done Sent false
9456 3296 Acked Write 8 Done Done Sent false
9457 3297 Acked Write 2 Done Done Sent false
9458 3298 Acked Read 6 Done Done Sent false
9459 3299 Acked Write 2 Done Done Sent false
9460 3300 Acked Read 7 Done Done Sent false
9461 3301 Acked Write 6 Done Done Sent false
9462 3302 Acked Read 8 Done Done Sent false
9463 3303 Acked Write 1 Done Done Sent false
9464 3304 Acked Write 5 Done Done Sent false
9465 3305 Acked Read 1 Done Done Sent false
9466 3306 Acked Read 8 Done Done Sent false
9467 3307 Acked Read 2 Done Done Sent false
9468 3308 Acked Write 1 Done Done Sent false
9469 3309 Acked Write 5 Done Done Sent false
9470 3310 Acked Read 6 Done Done Sent false
9471 3311 Acked Read 1 Done Done Sent false
9472 3312 Acked Write 7 Done Done Sent false
9473 3313 Acked Read 6 Done Done Sent false
9474 3314 Acked Write 4 Done Done Sent false
9475 3315 Acked Write 5 Done Done Sent false
9476 3316 Acked Flush 0 Done Done Sent false
9477 3317 Acked Read 5 Done Done Sent false
9478 3318 Acked Flush 0 Done Done Sent false
9479 3319 Acked Write 4 Done Done Sent false
9480 3320 Acked Write 8 Done Done Sent false
9481 3321 Acked Write 7 Done Done Sent false
9482 3322 Acked Write 7 Done Done Sent false
9483 3323 Acked Write 8 Done Done Sent false
9484 3324 Acked Write 6 Done Done Sent false
9485 3325 Acked Read 3 Done Done Sent false
9486 3326 Acked Write 4 Done Done Sent false
9487 3327 Acked Read 3 Done Done Sent false
9488 3328 Acked Write 9 Done Done Sent false
9489 3329 Acked Read 10 Done Done Sent false
9490 3330 Acked Flush 0 Done Done Sent false
9491 3331 Acked Read 1 Done Done Sent false
9492 3332 Acked Read 6 Done Done Sent false
9493 3333 Acked Write 5 Done Done Sent false
9494 3334 Acked Read 2 Done Done Sent false
9495 3335 Acked Read 1 Done Done Sent false
9496 3336 Acked Write 10 Done Done Sent false
9497 3337 Acked Write 7 Done Done Sent false
9498 3338 Acked Read 8 Done Done Sent false
9499 3339 Acked Read 1 Done Done Sent false
9500 3340 Acked Write 9 Done Done Sent false
9501 3341 Acked Write 5 Done Done Sent false
9502 3342 Acked Read 5 Done Done Sent false
9503 3343 Acked Read 4 Done Done Sent false
9504 3344 Acked Write 10 Done Done Sent false
9505 3345 Acked Flush 0 Done Done Sent false
9506 3346 Acked Read 5 Done Done Sent false
9507 3347 Acked Read 6 Done Done Sent false
9508 3348 Acked Write 7 Done Done Sent false
9509 3349 Acked Write 8 Done Done Sent false
9510 3350 Acked Read 7 Done Done Sent false
9511 3351 Acked Write 7 Done Done Sent false
9512 3352 Acked Write 2 Done Done Sent false
9513 3353 Acked Write 10 Done Done Sent false
9514 3354 Acked Read 8 Done Done Sent false
9515 3355 Acked Read 3 Done Done Sent false
9516 3356 Acked Read 10 Done Done Sent false
9517 3357 Acked Write 4 Done Done Sent false
9518 3358 Acked Read 8 Done Done Sent false
9519 3359 Acked Flush 0 Done Done Sent false
9520 3360 Acked Write 3 Done Done Sent false
9521 3361 Acked Flush 0 Done Done Sent false
9522 3362 Acked Read 10 Done Done Sent false
9523 3363 Acked Read 9 Done Done Sent false
9524 3364 Acked Read 10 Done Done Sent false
9525 3365 Acked Read 9 Done Done Sent false
9526 3366 Acked Write 8 Done Done Sent false
9527 3367 Acked Flush 0 Done Done Sent false
9528 3368 Acked Write 2 Done Done Sent false
9529 3369 Acked Write 2 Done Done Sent false
9530 3370 Acked Write 2 Done Done Sent false
9531 3371 Acked Write 2 Done Done Sent false
9532 3372 Acked Read 1 Done Done Sent false
9533 3373 Acked Read 5 Done Done Sent false
9534 3374 Acked Read 4 Done Done Sent false
9535 3375 Acked Flush 0 Done Done Sent false
9536 3376 Acked Read 8 Done Done Sent false
9537 3377 Acked Read 1 Done Done Sent false
9538 3378 Acked Read 7 Done Done Sent false
9539 3379 Acked Read 9 Done Done Sent false
9540 3380 Acked Read 10 Done Done Sent false
9541 3381 Acked Read 9 Done Done Sent false
9542 3382 Acked Write 10 Done Done Sent false
9543 3383 Acked Read 4 Done Done Sent false
9544 3384 Acked Write 10 Done Done Sent false
9545 3385 Acked Read 3 Done Done Sent false
9546 3386 Acked Write 2 Done Done Sent false
9547 3387 Acked Read 2 Done Done Sent false
9548 3388 Acked Read 1 Done Done Sent false
9549 3389 Acked Write 3 Done Done Sent false
9550 3390 Acked Read 10 Done Done Sent false
9551 3391 Acked Read 10 Done Done Sent false
9552 3392 Acked Write 2 Done Done Sent false
9553 3393 Acked Read 3 Done Done Sent false
9554 3394 Acked Read 9 Done Done Sent false
9555 3395 Acked Flush 0 Done Done Sent false
9556 3396 Acked Read 8 Done Done Sent false
9557 3397 Acked Write 5 Done Done Sent false
9558 3398 Acked Write 1 Done Done Sent false
9559 3399 Acked Read 7 Done Done Sent false
9560 3400 Acked Read 1 Done Done Sent false
9561 3401 Acked Write 3 Done Done Sent false
9562 3402 Acked Read 1 Done Done Sent false
9563 3403 Acked Write 8 Done Done Sent false
9564 3404 Acked Write 3 Done Done Sent false
9565 3405 Acked Flush 0 Done Done Sent false
9566 3406 Acked Flush 0 Done Done Sent false
9567 3407 Acked Write 3 Done Done Sent false
9568 3408 Acked Read 6 Done Done Sent false
9569 3409 Acked Write 1 Done Done Sent false
9570 3410 Acked Write 9 Done Done Sent false
9571 3411 Acked Read 10 Done Done Sent false
9572 3412 Acked Read 9 Done Done Sent false
9573 3413 Acked Write 10 Done Done Sent false
9574 STATES DS:0 DS:1 DS:2 TOTAL
9575 Sent 0 0 301 301
9576 Done 301 301 0 602
9577 Skipped 0 0 0 0
9578 Error 0 0 0 0
9579 Last Flush: 3406 3406 3112
9580 Downstairs last five completed: 3112 3111 3110 3109 3108
9581 Upstairs last five completed: 3413 3412 3411 3410 3409
9582 CLIENT: Up:0 ds:301 act:2
9583 [8830] got start action
9584 [8830] I am going to restart
9585 Starting downstairs at port 8830
9586 Make output file at "/tmp/dsc/downstairs-8830.txt"
9587 [8830][2] reports Starting
9588 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1213)
9589 [8830][2] reports Running
9590 ----------------------------------------------------------------
9591 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9592 States: Active Active Offline
9593 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9594 3113 Acked Read 9 Done Done Sent false
9595 3114 Acked Write 6 Done Done Sent false
9596 3115 Acked Read 7 Done Done Sent false
9597 3116 Acked Read 1 Done Done Sent false
9598 3117 Acked Read 3 Done Done Sent false
9599 3118 Acked Read 8 Done Done Sent false
9600 3119 Acked Flush 0 Done Done Sent false
9601 3120 Acked Read 4 Done Done Sent false
9602 3121 Acked Read 1 Done Done Sent false
9603 3122 Acked Write 6 Done Done Sent false
9604 3123 Acked Write 1 Done Done Sent false
9605 3124 Acked Write 7 Done Done Sent false
9606 3125 Acked Write 10 Done Done Sent false
9607 3126 Acked Read 8 Done Done Sent false
9608 3127 Acked Flush 0 Done Done Sent false
9609 3128 Acked Read 10 Done Done Sent false
9610 3129 Acked Read 3 Done Done Sent false
9611 3130 Acked Read 3 Done Done Sent false
9612 3131 Acked Write 5 Done Done Sent false
9613 3132 Acked Write 5 Done Done Sent false
9614 3133 Acked Read 3 Done Done Sent false
9615 3134 Acked Read 10 Done Done Sent false
9616 3135 Acked Read 7 Done Done Sent false
9617 3136 Acked Write 4 Done Done Sent false
9618 3137 Acked Write 3 Done Done Sent false
9619 3138 Acked Write 4 Done Done Sent false
9620 3139 Acked Write 4 Done Done Sent false
9621 3140 Acked Flush 0 Done Done Sent false
9622 3141 Acked Read 10 Done Done Sent false
9623 3142 Acked Read 8 Done Done Sent false
9624 3143 Acked Write 5 Done Done Sent false
9625 3144 Acked Read 10 Done Done Sent false
9626 3145 Acked Read 9 Done Done Sent false
9627 3146 Acked Read 8 Done Done Sent false
9628 3147 Acked Write 2 Done Done Sent false
9629 3148 Acked Read 4 Done Done Sent false
9630 3149 Acked Write 5 Done Done Sent false
9631 3150 Acked Read 1 Done Done Sent false
9632 3151 Acked Write 3 Done Done Sent false
9633 3152 Acked Write 3 Done Done Sent false
9634 3153 Acked Read 2 Done Done Sent false
9635 3154 Acked Read 10 Done Done Sent false
9636 3155 Acked Read 1 Done Done Sent false
9637 3156 Acked Flush 0 Done Done Sent false
9638 3157 Acked Write 7 Done Done Sent false
9639 3158 Acked Read 8 Done Done Sent false
9640 3159 Acked Write 9 Done Done Sent false
9641 3160 Acked Write 6 Done Done Sent false
9642 3161 Acked Write 10 Done Done Sent false
9643 3162 Acked Flush 0 Done Done Sent false
9644 3163 Acked Write 3 Done Done Sent false
9645 3164 Acked Read 7 Done Done Sent false
9646 3165 Acked Read 7 Done Done Sent false
9647 3166 Acked Read 3 Done Done Sent false
9648 3167 Acked Read 10 Done Done Sent false
9649 3168 Acked Read 1 Done Done Sent false
9650 3169 Acked Write 9 Done Done Sent false
9651 3170 Acked Read 6 Done Done Sent false
9652 3171 Acked Read 6 Done Done Sent false
9653 3172 Acked Read 2 Done Done Sent false
9654 3173 Acked Write 10 Done Done Sent false
9655 3174 Acked Write 2 Done Done Sent false
9656 3175 Acked Write 3 Done Done Sent false
9657 3176 Acked Read 5 Done Done Sent false
9658 3177 Acked Write 10 Done Done Sent false
9659 3178 Acked Read 3 Done Done Sent false
9660 3179 Acked Flush 0 Done Done Sent false
9661 3180 Acked Write 3 Done Done Sent false
9662 3181 Acked Write 5 Done Done Sent false
9663 3182 Acked Read 3 Done Done Sent false
9664 3183 Acked Read 3 Done Done Sent false
9665 3184 Acked Read 4 Done Done Sent false
9666 3185 Acked Write 3 Done Done Sent false
9667 3186 Acked Write 8 Done Done Sent false
9668 3187 Acked Flush 0 Done Done Sent false
9669 3188 Acked Write 6 Done Done Sent false
9670 3189 Acked Read 5 Done Done Sent false
9671 3190 Acked Read 4 Done Done Sent false
9672 3191 Acked Write 2 Done Done Sent false
9673 3192 Acked Read 10 Done Done Sent false
9674 3193 Acked Write 7 Done Done Sent false
9675 3194 Acked Read 3 Done Done Sent false
9676 3195 Acked Read 7 Done Done Sent false
9677 3196 Acked Read 2 Done Done Sent false
9678 3197 Acked Write 5 Done Done Sent false
9679 3198 Acked Read 8 Done Done Sent false
9680 3199 Acked Read 6 Done Done Sent false
9681 3200 Acked Write 2 Done Done Sent false
9682 3201 Acked Write 3 Done Done Sent false
9683 3202 Acked Read 4 Done Done Sent false
9684 3203 Acked Write 10 Done Done Sent false
9685 3204 Acked Write 1 Done Done Sent false
9686 3205 Acked Read 10 Done Done Sent false
9687 3206 Acked Read 6 Done Done Sent false
9688 3207 Acked Write 4 Done Done Sent false
9689 3208 Acked Read 3 Done Done Sent false
9690 3209 Acked Read 3 Done Done Sent false
9691 3210 Acked Write 2 Done Done Sent false
9692 3211 Acked Read 7 Done Done Sent false
9693 3212 Acked Read 5 Done Done Sent false
9694 3213 Acked Read 7 Done Done Sent false
9695 3214 Acked Read 8 Done Done Sent false
9696 3215 Acked Flush 0 Done Done Sent false
9697 3216 Acked Write 10 Done Done Sent false
9698 3217 Acked Read 10 Done Done Sent false
9699 3218 Acked Read 10 Done Done Sent false
9700 3219 Acked Write 8 Done Done Sent false
9701 3220 Acked Read 1 Done Done Sent false
9702 3221 Acked Write 10 Done Done Sent false
9703 3222 Acked Flush 0 Done Done Sent false
9704 3223 Acked Read 6 Done Done Sent false
9705 3224 Acked Write 2 Done Done Sent false
9706 3225 Acked Write 1 Done Done Sent false
9707 3226 Acked Write 8 Done Done Sent false
9708 3227 Acked Read 1 Done Done Sent false
9709 3228 Acked Read 10 Done Done Sent false
9710 3229 Acked Read 10 Done Done Sent false
9711 3230 Acked Flush 0 Done Done Sent false
9712 3231 Acked Write 7 Done Done Sent false
9713 3232 Acked Flush 0 Done Done Sent false
9714 3233 Acked Flush 0 Done Done Sent false
9715 3234 Acked Read 7 Done Done Sent false
9716 3235 Acked Read 10 Done Done Sent false
9717 3236 Acked Write 2 Done Done Sent false
9718 3237 Acked Read 10 Done Done Sent false
9719 3238 Acked Write 4 Done Done Sent false
9720 3239 Acked Read 4 Done Done Sent false
9721 3240 Acked Read 8 Done Done Sent false
9722 3241 Acked Write 7 Done Done Sent false
9723 3242 Acked Read 9 Done Done Sent false
9724 3243 Acked Read 6 Done Done Sent false
9725 3244 Acked Write 1 Done Done Sent false
9726 3245 Acked Read 5 Done Done Sent false
9727 3246 Acked Read 2 Done Done Sent false
9728 3247 Acked Write 9 Done Done Sent false
9729 3248 Acked Flush 0 Done Done Sent false
9730 3249 Acked Read 7 Done Done Sent false
9731 3250 Acked Flush 0 Done Done Sent false
9732 3251 Acked Write 9 Done Done Sent false
9733 3252 Acked Flush 0 Done Done Sent false
9734 3253 Acked Write 8 Done Done Sent false
9735 3254 Acked Write 3 Done Done Sent false
9736 3255 Acked Write 9 Done Done Sent false
9737 3256 Acked Read 4 Done Done Sent false
9738 3257 Acked Write 1 Done Done Sent false
9739 3258 Acked Read 8 Done Done Sent false
9740 3259 Acked Read 8 Done Done Sent false
9741 3260 Acked Write 4 Done Done Sent false
9742 3261 Acked Write 2 Done Done Sent false
9743 3262 Acked Write 9 Done Done Sent false
9744 3263 Acked Write 10 Done Done Sent false
9745 3264 Acked Read 4 Done Done Sent false
9746 3265 Acked Write 6 Done Done Sent false
9747 3266 Acked Write 5 Done Done Sent false
9748 3267 Acked Read 10 Done Done Sent false
9749 3268 Acked Read 3 Done Done Sent false
9750 3269 Acked Read 7 Done Done Sent false
9751 3270 Acked Read 4 Done Done Sent false
9752 3271 Acked Read 2 Done Done Sent false
9753 3272 Acked Flush 0 Done Done Sent false
9754 3273 Acked Read 2 Done Done Sent false
9755 3274 Acked Read 3 Done Done Sent false
9756 3275 Acked Write 1 Done Done Sent false
9757 3276 Acked Write 5 Done Done Sent false
9758 3277 Acked Write 1 Done Done Sent false
9759 3278 Acked Read 7 Done Done Sent false
9760 3279 Acked Write 7 Done Done Sent false
9761 3280 Acked Read 2 Done Done Sent false
9762 3281 Acked Flush 0 Done Done Sent false
9763 3282 Acked Flush 0 Done Done Sent false
9764 3283 Acked Write 9 Done Done Sent false
9765 3284 Acked Write 1 Done Done Sent false
9766 3285 Acked Read 9 Done Done Sent false
9767 3286 Acked Write 10 Done Done Sent false
9768 3287 Acked Read 4 Done Done Sent false
9769 3288 Acked Write 9 Done Done Sent false
9770 3289 Acked Write 3 Done Done Sent false
9771 3290 Acked Write 9 Done Done Sent false
9772 3291 Acked Write 8 Done Done Sent false
9773 3292 Acked Write 5 Done Done Sent false
9774 3293 Acked Write 2 Done Done Sent false
9775 3294 Acked Read 4 Done Done Sent false
9776 3295 Acked Read 10 Done Done Sent false
9777 3296 Acked Write 8 Done Done Sent false
9778 3297 Acked Write 2 Done Done Sent false
9779 3298 Acked Read 6 Done Done Sent false
9780 3299 Acked Write 2 Done Done Sent false
9781 3300 Acked Read 7 Done Done Sent false
9782 3301 Acked Write 6 Done Done Sent false
9783 3302 Acked Read 8 Done Done Sent false
9784 3303 Acked Write 1 Done Done Sent false
9785 3304 Acked Write 5 Done Done Sent false
9786 3305 Acked Read 1 Done Done Sent false
9787 3306 Acked Read 8 Done Done Sent false
9788 3307 Acked Read 2 Done Done Sent false
9789 3308 Acked Write 1 Done Done Sent false
9790 3309 Acked Write 5 Done Done Sent false
9791 3310 Acked Read 6 Done Done Sent false
9792 3311 Acked Read 1 Done Done Sent false
9793 3312 Acked Write 7 Done Done Sent false
9794 3313 Acked Read 6 Done Done Sent false
9795 3314 Acked Write 4 Done Done Sent false
9796 3315 Acked Write 5 Done Done Sent false
9797 3316 Acked Flush 0 Done Done Sent false
9798 3317 Acked Read 5 Done Done Sent false
9799 3318 Acked Flush 0 Done Done Sent false
9800 3319 Acked Write 4 Done Done Sent false
9801 3320 Acked Write 8 Done Done Sent false
9802 3321 Acked Write 7 Done Done Sent false
9803 3322 Acked Write 7 Done Done Sent false
9804 3323 Acked Write 8 Done Done Sent false
9805 3324 Acked Write 6 Done Done Sent false
9806 3325 Acked Read 3 Done Done Sent false
9807 3326 Acked Write 4 Done Done Sent false
9808 3327 Acked Read 3 Done Done Sent false
9809 3328 Acked Write 9 Done Done Sent false
9810 3329 Acked Read 10 Done Done Sent false
9811 3330 Acked Flush 0 Done Done Sent false
9812 3331 Acked Read 1 Done Done Sent false
9813 3332 Acked Read 6 Done Done Sent false
9814 3333 Acked Write 5 Done Done Sent false
9815 3334 Acked Read 2 Done Done Sent false
9816 3335 Acked Read 1 Done Done Sent false
9817 3336 Acked Write 10 Done Done Sent false
9818 3337 Acked Write 7 Done Done Sent false
9819 3338 Acked Read 8 Done Done Sent false
9820 3339 Acked Read 1 Done Done Sent false
9821 3340 Acked Write 9 Done Done Sent false
9822 3341 Acked Write 5 Done Done Sent false
9823 3342 Acked Read 5 Done Done Sent false
9824 3343 Acked Read 4 Done Done Sent false
9825 3344 Acked Write 10 Done Done Sent false
9826 3345 Acked Flush 0 Done Done Sent false
9827 3346 Acked Read 5 Done Done Sent false
9828 3347 Acked Read 6 Done Done Sent false
9829 3348 Acked Write 7 Done Done Sent false
9830 3349 Acked Write 8 Done Done Sent false
9831 3350 Acked Read 7 Done Done Sent false
9832 3351 Acked Write 7 Done Done Sent false
9833 3352 Acked Write 2 Done Done Sent false
9834 3353 Acked Write 10 Done Done Sent false
9835 3354 Acked Read 8 Done Done Sent false
9836 3355 Acked Read 3 Done Done Sent false
9837 3356 Acked Read 10 Done Done Sent false
9838 3357 Acked Write 4 Done Done Sent false
9839 3358 Acked Read 8 Done Done Sent false
9840 3359 Acked Flush 0 Done Done Sent false
9841 3360 Acked Write 3 Done Done Sent false
9842 3361 Acked Flush 0 Done Done Sent false
9843 3362 Acked Read 10 Done Done Sent false
9844 3363 Acked Read 9 Done Done Sent false
9845 3364 Acked Read 10 Done Done Sent false
9846 3365 Acked Read 9 Done Done Sent false
9847 3366 Acked Write 8 Done Done Sent false
9848 3367 Acked Flush 0 Done Done Sent false
9849 3368 Acked Write 2 Done Done Sent false
9850 3369 Acked Write 2 Done Done Sent false
9851 3370 Acked Write 2 Done Done Sent false
9852 3371 Acked Write 2 Done Done Sent false
9853 3372 Acked Read 1 Done Done Sent false
9854 3373 Acked Read 5 Done Done Sent false
9855 3374 Acked Read 4 Done Done Sent false
9856 3375 Acked Flush 0 Done Done Sent false
9857 3376 Acked Read 8 Done Done Sent false
9858 3377 Acked Read 1 Done Done Sent false
9859 3378 Acked Read 7 Done Done Sent false
9860 3379 Acked Read 9 Done Done Sent false
9861 3380 Acked Read 10 Done Done Sent false
9862 3381 Acked Read 9 Done Done Sent false
9863 3382 Acked Write 10 Done Done Sent false
9864 3383 Acked Read 4 Done Done Sent false
9865 3384 Acked Write 10 Done Done Sent false
9866 3385 Acked Read 3 Done Done Sent false
9867 3386 Acked Write 2 Done Done Sent false
9868 3387 Acked Read 2 Done Done Sent false
9869 3388 Acked Read 1 Done Done Sent false
9870 3389 Acked Write 3 Done Done Sent false
9871 3390 Acked Read 10 Done Done Sent false
9872 3391 Acked Read 10 Done Done Sent false
9873 3392 Acked Write 2 Done Done Sent false
9874 3393 Acked Read 3 Done Done Sent false
9875 3394 Acked Read 9 Done Done Sent false
9876 3395 Acked Flush 0 Done Done Sent false
9877 3396 Acked Read 8 Done Done Sent false
9878 3397 Acked Write 5 Done Done Sent false
9879 3398 Acked Write 1 Done Done Sent false
9880 3399 Acked Read 7 Done Done Sent false
9881 3400 Acked Read 1 Done Done Sent false
9882 3401 Acked Write 3 Done Done Sent false
9883 3402 Acked Read 1 Done Done Sent false
9884 3403 Acked Write 8 Done Done Sent false
9885 3404 Acked Write 3 Done Done Sent false
9886 3405 Acked Flush 0 Done Done Sent false
9887 3406 Acked Flush 0 Done Done Sent false
9888 3407 Acked Write 3 Done Done Sent false
9889 3408 Acked Read 6 Done Done Sent false
9890 3409 Acked Write 1 Done Done Sent false
9891 3410 Acked Write 9 Done Done Sent false
9892 3411 Acked Read 10 Done Done Sent false
9893 3412 Acked Read 9 Done Done Sent false
9894 3413 Acked Write 10 Done Done Sent false
9895 3414 Acked Flush 0 Done Done Sent false
9896 STATES DS:0 DS:1 DS:2 TOTAL
9897 Sent 0 0 302 302
9898 Done 302 302 0 604
9899 Skipped 0 0 0 0
9900 Error 0 0 0 0
9901 Last Flush: 3414 3414 3112
9902 Downstairs last five completed: 3112 3111 3110 3109 3108
9903 Upstairs last five completed: 3414 3413 3412 3411 3410
9904 CLIENT: Up:0 ds:302 act:2
99052025-01-03T08:32:57.315ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99062025-01-03T08:32:57.315ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45693) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99072025-01-03T08:32:57.316ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99082025-01-03T08:32:57.317ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99092025-01-03T08:32:57.317ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99102025-01-03T08:32:57.317ZINFOcrucible: send last flush ID to this DS: 3112 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99112025-01-03T08:32:57.317ZINFOcrucible: Replied this last flush ID: 3112 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99122025-01-03T08:32:57.317ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99132025-01-03T08:32:57.317ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99142025-01-03T08:32:57.317ZINFOcrucible: [2] client re-new 302 jobs since flush 3112 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99152025-01-03T08:32:57.317ZINFOcrucible: [2] Replayed 302 jobs since flush: 3112 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
9916 ----------------------------------------------------------------
9917 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9918 Downstairs last five completed: 3414 3413 3412 3411 3410
9919 Upstairs last five completed: 3414 3413 3412 3411 3410
9920 CLIENT: Up:0 ds:0 act:3
9921 Replay: All jobs finished
9922 Main task has work to do, go find it
9923 got dsc Stop(0)
9924 stop 0
9925 Jan 03 08:32:59.561 INFO [8810] Got stop action so:false kr:false
9926 request completed, latency_us: 307, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 52787f64-3450-4e5e-b17f-d6029cff8162, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
9927 Jan 03 08:32:59.561 INFO request completed, latency_us: 158, response_code: 200, uri: /state/cid/0, method: GET, req_id: 2821bfab-3506-495d-b266-21803bdc8618, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
99282025-01-03T08:32:59.561ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99292025-01-03T08:32:59.561ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99302025-01-03T08:32:59.561ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99312025-01-03T08:32:59.561ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99322025-01-03T08:32:59.561ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
99332025-01-03T08:32:59.562ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
9934 [8810] Exited with: ExitStatus(unix_wait_status(9))
9935 [8810][0] reports Exit
9936 Jan 03 08:33:03.564 INFO request completed, latency_us: 194, response_code: 200, uri: /state/cid/0, method: GET, req_id: f884005c-2c93-4c0f-a640-d45f9be825bb, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
9937 001/300 Read block 3377 len 20480
9938 002/300 Write block 370 len 16384 data: 4 3 3 2
9939 003/300 Write block 2205 len 28672 data: 3 3 3 3 3 3 3
9940 004/300 Read block 1234 len 40960
9941 005/300 Read block 2791 len 12288
9942 006/300 Write block 244 len 16384 data: 2 2 2 2
9943 007/300 Read block 581 len 20480
9944 008/300 Read block 4308 len 36864
9945 009/300 Read block 1945 len 4096
9946 010/300 Read block 336 len 8192
9947 011/300 Flush
9948 012/300 Write block 4795 len 12288 data: 2 2 2
9949 013/300 Write block 193 len 8192 data: 2 3
9950 014/300 Read block 4130 len 32768
9951 015/300 Write block 1546 len 36864 data: 2 2 2 2 2 2 2 2 2
9952 016/300 Write block 435 len 4096 data: 4
9953 017/300 Write block 3864 len 20480 data: 4 3 4 4 4
9954 018/300 Read block 2243 len 28672
9955 019/300 Read block 3520 len 40960
9956 020/300 Write block 2919 len 32768 data: 3 2 2 2 2 2 2 2
9957 021/300 Write block 1318 len 16384 data: 3 3 3 3
9958 022/300 Read block 3114 len 12288
9959 023/300 Read block 654 len 4096
9960 024/300 Flush
9961 025/300 Write block 4932 len 20480 data: 4 4 3 3 2
9962 026/300 Write block 1519 len 12288 data: 4 4 3
9963 027/300 Read block 2295 len 20480
9964 028/300 Read block 3567 len 8192
9965 029/300 Read block 985 len 8192
9966 030/300 Read block 70 len 20480
9967 031/300 Read block 3550 len 28672
9968 032/300 Read block 1023 len 32768
9969 033/300 Read block 4816 len 40960
9970 034/300 Write block 3176 len 32768 data: 4 5 5 5 4 3 3 4
9971 035/300 Read block 1036 len 20480
9972 036/300 Read block 3879 len 16384
9973 037/300 Read block 213 len 36864
9974 038/300 Read block 3405 len 24576
9975 039/300 Write block 3528 len 20480 data: 2 2 2 2 2
9976 040/300 Read block 763 len 32768
9977 041/300 Flush
9978 042/300 Read block 1132 len 16384
9979 043/300 Write block 745 len 36864 data: 5 5 4 4 4 4 4 4 3
9980 044/300 Read block 4521 len 20480
9981 045/300 Write block 1288 len 8192 data: 2 2
9982 046/300 Read block 3745 len 24576
9983 047/300 Read block 1594 len 24576
9984 048/300 Write block 4753 len 20480 data: 3 3 4 5 5
9985 049/300 Read block 4005 len 24576
9986 050/300 Write block 4955 len 12288 data: 3 3 3
9987 051/300 Write block 1336 len 24576 data: 2 2 2 3 3 4
9988 052/300 Read block 1270 len 4096
9989 053/300 Write block 2793 len 20480 data: 2 2 3 3 3
9990 054/300 Read block 4865 len 8192
9991 055/300 Write block 4930 len 12288 data: 5 4 5
9992 056/300 Read block 4178 len 16384
9993 057/300 Write block 798 len 28672 data: 4 4 3 3 3 2 4
9994 058/300 Write block 817 len 16384 data: 5 4 4 5
9995 059/300 Write block 719 len 8192 data: 2 2
9996 060/300 Read block 2236 len 12288
9997 061/300 Read block 3629 len 36864
9998 062/300 Write block 4058 len 36864 data: 3 3 3 3 3 3 2 2 2
9999 063/300 Write block 439 len 8192 data: 4 3
10000 064/300 Read block 2413 len 28672
10001 065/300 Flush
10002 066/300 Write block 2765 len 28672 data: 3 4 3 2 2 2 2
10003 067/300 Read block 3883 len 32768
10004 068/300 Write block 4290 len 8192 data: 2 2
10005 069/300 Read block 103 len 20480
10006 070/300 Read block 4708 len 24576
10007 071/300 Read block 2741 len 40960
10008 072/300 Read block 2968 len 24576
10009 073/300 Write block 837 len 40960 data: 3 3 3 3 3 3 5 5 4 4
10010 074/300 Write block 2761 len 36864 data: 5 5 5 4 4 5 4 3 3
10011 075/300 Read block 3959 len 16384
10012 076/300 Read block 1763 len 12288
10013 077/300 Read block 4192 len 16384
10014 078/300 Write block 757 len 32768 data: 3 3 5 5 4 4 4 4
10015 079/300 Flush
10016 080/300 Write block 4863 len 28672 data: 2 2 2 2 2 2 4
10017 081/300 Read block 1491 len 4096
10018 082/300 Write block 1215 len 24576 data: 4 4 4 3 3 3
10019 083/300 Read block 3424 len 12288
10020 084/300 Read block 3735 len 12288
10021 085/300 Read block 726 len 20480
10022 086/300 Read block 1367 len 4096
10023 087/300 Write block 4223 len 4096 data: 3
10024 088/300 Write block 440 len 40960 data: 4 3 3 2 3 3 3 4 3 3
10025 089/300 Write block 1917 len 40960 data: 3 3 5 4 4 5 5 5 4 4
10026 090/300 Read block 1203 len 8192
10027 091/300 Read block 3711 len 24576
10028 092/300 Read block 651 len 4096
10029 093/300 Read block 1294 len 4096
10030 094/300 Write block 1198 len 40960 data: 4 3 3 2 2 2 2 3 3 3
10031 095/300 Read block 18 len 32768
10032 096/300 Flush
10033 097/300 Write block 4154 len 24576 data: 2 2 3 3 2 2
10034 098/300 Write block 4828 len 40960 data: 3 2 2 3 3 3 3 3 3 3
10035 099/300 Flush
10036 100/300 Write block 962 len 32768 data: 3 2 2 2 2 2 2 2
10037 101/300 Flush
10038 102/300 Write block 236 len 12288 data: 3 3 3
10039 103/300 Write block 948 len 20480 data: 4 4 4 4 3
10040 104/300 Write block 4080 len 12288 data: 3 3 3
10041 105/300 Read block 4528 len 32768
10042 106/300 Read block 4183 len 24576
10043 107/300 Write block 2511 len 8192 data: 2 2
10044 108/300 Flush
10045 109/300 Write block 1248 len 16384 data: 4 5 4 4
10046 110/300 Read block 736 len 16384
10047 111/300 Read block 4469 len 16384
10048 112/300 Write block 3611 len 24576 data: 3 4 3 3 3 3
10049 113/300 Read block 919 len 16384
10050 114/300 Write block 894 len 16384 data: 2 4 4 4
10051 115/300 Read block 2877 len 12288
10052 116/300 Write block 3596 len 12288 data: 2 2 2
10053 117/300 Write block 1418 len 12288 data: 3 4 4
10054 118/300 Read block 3566 len 4096
10055 119/300 Flush
10056 120/300 Write block 3077 len 16384 data: 2 2 2 2
10057 121/300 Read block 716 len 8192
10058 122/300 Read block 57 len 20480
10059 123/300 Flush
10060 124/300 Read block 3131 len 12288
10061 125/300 Read block 3360 len 12288
10062 126/300 Write block 4620 len 32768 data: 3 3 4 4 3 3 3 3
10063 127/300 Write block 3452 len 40960 data: 2 2 3 3 4 4 4 4 4 2
10064 128/300 Read block 2605 len 12288
10065 129/300 Read block 960 len 36864
10066 130/300 Write block 4092 len 24576 data: 2 2 2 2 3 3
10067 131/300 Read block 1161 len 4096
10068 132/300 Write block 2297 len 24576 data: 2 3 4 5 5 5
10069 133/300 Read block 361 len 20480
10070 134/300 Read block 298 len 20480
10071 135/300 Read block 651 len 4096
10072 136/300 Read block 4934 len 24576
10073 137/300 Read block 2081 len 12288
10074 138/300 Read block 4225 len 12288
10075 139/300 Read block 3462 len 20480
10076 140/300 Read block 3390 len 28672
10077 141/300 Read block 4415 len 24576
10078 142/300 Write block 732 len 12288 data: 3 3 3
10079 143/300 Read block 4414 len 40960
10080 144/300 Write block 210 len 24576 data: 2 2 2 2 2 4
10081 145/300 Write block 69 len 16384 data: 5 4 3 4
10082 146/300 Read block 2096 len 16384
10083 147/300 Write block 4202 len 28672 data: 6 5 6 6 5 3 2
10084 148/300 Read block 3882 len 36864
10085 149/300 Write block 4647 len 12288 data: 4 4 3
10086 150/300 Read block 3910 len 16384
10087 151/300 Read block 2199 len 4096
10088 152/300 Write block 4935 len 36864 data: 4 3 2 3 3 3 3 4 5
10089 153/300 Write block 872 len 16384 data: 3 3 2 2
10090 154/300 Flush
10091 155/300 Write block 4932 len 32768 data: 6 5 4 5 4 3 4 4
10092 156/300 Read block 2325 len 32768
10093 157/300 Read block 1161 len 8192
10094 158/300 Read block 2265 len 20480
10095 159/300 Read block 3968 len 40960
10096 160/300 Read block 2037 len 40960
10097 161/300 Read block 628 len 4096
10098 162/300 Write block 3542 len 28672 data: 4 4 6 6 6 5 5
10099 163/300 Write block 4590 len 20480 data: 4 3 3 3 3
10100 164/300 Read block 3037 len 32768
10101 165/300 Read block 4907 len 12288
10102 166/300 Read block 958 len 36864
10103 167/300 Read block 1085 len 28672
10104 168/300 Flush
10105 169/300 Flush
10106 170/300 Write block 2796 len 4096 data: 4
10107 171/300 Write block 399 len 20480 data: 3 2 2 2 2
10108 172/300 Read block 4747 len 24576
10109 173/300 Read block 2733 len 20480
10110 174/300 Flush
10111 175/300 Read block 4317 len 20480
10112 176/300 Read block 677 len 28672
10113 177/300 Read block 1057 len 28672
10114 178/300 Read block 2736 len 8192
10115 179/300 Read block 1419 len 32768
10116 180/300 Write block 2569 len 4096 data: 4
10117 181/300 Flush
10118 182/300 Write block 3867 len 24576 data: 5 5 4 4 5 5
10119 183/300 Write block 1679 len 28672 data: 5 4 4 4 3 2 2
10120 184/300 Write block 4299 len 32768 data: 2 2 2 2 3 3 3 3
10121 185/300 Read block 4507 len 32768
10122 186/300 Write block 4664 len 4096 data: 4
10123 187/300 Write block 907 len 28672 data: 2 2 2 2 2 3 3
10124 188/300 Flush
10125 189/300 Read block 510 len 40960
10126 190/300 Flush
10127 191/300 Flush
10128 192/300 Flush
10129 193/300 Read block 3342 len 32768
10130 194/300 Write block 954 len 32768 data: 3 2 3 3 3 3 3 3
10131 195/300 Read block 4602 len 36864
10132 196/300 Write block 907 len 16384 data: 3 3 3 3
10133 197/300 Read block 3820 len 16384
10134 198/300 Write block 3555 len 8192 data: 2 2
10135 199/300 Read block 2305 len 36864
10136 200/300 Write block 3291 len 36864 data: 4 4 3 3 3 2 2 2 2
10137 201/300 Write block 1223 len 20480 data: 3 4 4 4 4
10138 202/300 Read block 4116 len 24576
10139 203/300 Read block 1574 len 20480
10140 204/300 Read block 2808 len 36864
10141 205/300 Read block 2733 len 16384
10142 206/300 Write block 2042 len 36864 data: 2 2 2 2 3 4 4 4 5
10143 207/300 Write block 3129 len 4096 data: 3
10144 208/300 Read block 3304 len 16384
10145 209/300 Write block 4366 len 28672 data: 4 4 4 3 3 3 3
10146 210/300 Read block 4769 len 20480
10147 211/300 Read block 107 len 4096
10148 212/300 Write block 2954 len 4096 data: 2
10149 213/300 Write block 4521 len 8192 data: 2 2
10150 214/300 Read block 2501 len 32768
10151 215/300 Write block 2719 len 40960 data: 2 2 3 3 3 3 3 3 3 3
10152 216/300 Read block 4412 len 28672
10153 217/300 Write block 4216 len 20480 data: 3 3 3 3 3
10154 218/300 Read block 4606 len 32768
10155 219/300 Write block 475 len 4096 data: 2
10156 220/300 Flush
10157 221/300 Read block 1378 len 4096
10158 222/300 Read block 2154 len 36864
10159 223/300 Write block 935 len 20480 data: 3 4 4 4 2
10160 224/300 Write block 775 len 24576 data: 3 2 2 2 3 3
10161 225/300 Write block 1594 len 20480 data: 4 3 3 3 3
10162 226/300 Read block 3316 len 28672
10163 227/300 Read block 1174 len 24576
10164 228/300 Read block 2507 len 20480
10165 229/300 Read block 729 len 32768
10166 230/300 Write block 2631 len 20480 data: 5 5 3 3 3
10167 231/300 Read block 258 len 20480
10168 232/300 Read block 1457 len 28672
10169 233/300 Write block 4661 len 24576 data: 2 4 4 5 4 4
10170 234/300 Read block 1958 len 40960
10171 235/300 Read block 4210 len 36864
10172 236/300 Read block 4841 len 4096
10173 237/300 Read block 3975 len 36864
10174 238/300 Write block 2414 len 28672 data: 3 3 3 3 3 2 2
10175 239/300 Read block 4067 len 24576
10176 240/300 Write block 3802 len 20480 data: 3 3 4 3 2
10177 241/300 Read block 3671 len 12288
10178 242/300 Write block 2487 len 28672 data: 5 5 3 3 3 2 2
10179 243/300 Flush
10180 244/300 Flush
10181 245/300 Read block 2737 len 24576
10182 246/300 Read block 3166 len 36864
10183 247/300 Write block 4906 len 24576 data: 3 4 5 6 5 5
10184 248/300 Write block 4318 len 40960 data: 2 3 3 3 3 4 4 3 3 2
10185 249/300 Write block 3792 len 24576 data: 4 4 4 4 3 3
10186 250/300 Read block 2729 len 40960
10187 251/300 Write block 1026 len 20480 data: 2 2 2 2 2
10188 252/300 Read block 203 len 16384
10189 253/300 Flush
10190 254/300 Write block 4672 len 40960 data: 2 2 2 2 2 2 2 2 3 3
10191 255/300 Write block 4398 len 20480 data: 2 2 2 2 2
10192 256/300 Write block 692 len 32768 data: 3 4 4 4 4 4 3 2
10193 257/300 Read block 1526 len 28672
10194 258/300 Write block 2199 len 36864 data: 2 2 2 2 2 3 4 4 4
10195 259/300 Write block 1053 len 16384 data: 3 4 4 4
10196 260/300 Read block 1326 len 16384
10197 261/300 Read block 4855 len 24576
10198 262/300 Write block 2142 len 4096 data: 2
10199 263/300 Read block 4394 len 8192
10200 264/300 Read block 4698 len 12288
10201 265/300 Read block 2717 len 8192
10202 266/300 Read block 1991 len 36864
10203 267/300 Read block 2359 len 8192
10204 268/300 Write block 344 len 16384 data: 3 5 5 5
10205 269/300 Flush
10206 270/300 Read block 3689 len 24576
10207 271/300 Write block 3629 len 28672 data: 4 3 3 4 5 4 4
10208 272/300 Read block 1834 len 40960
10209 273/300 Write block 2014 len 20480 data: 2 2 2 3 3
10210 274/300 Write block 3968 len 24576 data: 3 2 2 2 2 4
10211 275/300 Write block 738 len 20480 data: 2 2 2 2 3
10212 276/300 Read block 3526 len 12288
10213 277/300 Write block 2263 len 36864 data: 2 2 2 2 2 2 2 2 2
10214 278/300 Read block 2115 len 28672
10215 279/300 Write block 3863 len 20480 data: 5 5 4 5 6
10216 280/300 Read block 1908 len 16384
10217 281/300 Read block 1607 len 16384
10218 282/300 Read block 4206 len 32768
10219 283/300 Read block 1357 len 32768
10220 284/300 Read block 135 len 24576
10221 285/300 Flush
10222 286/300 Read block 1256 len 32768
10223 287/300 Flush
10224 288/300 Write block 3454 len 24576 data: 4 4 5 5 5 5
10225 289/300 Read block 1667 len 32768
10226 290/300 Read block 2406 len 36864
10227 291/300 Read block 879 len 40960
10228 292/300 Flush
10229 293/300 Write block 1317 len 32768 data: 4 4 4 4 4 3 4 3
10230 294/300 Write block 2731 len 20480 data: 2 3 3 3 3
10231 295/300 Write block 4743 len 4096 data: 2
10232 296/300 Read block 1341 len 20480
10233 297/300 Read block 2329 len 4096
10234 298/300 Read block 692 len 32768
10235 299/300 Read block 2312 len 8192
10236 300/300 Write block 3428 len 8192 data: 5 5
10237 301/300 Write block 340 len 32768 data: 3 4 4 4 4 6 6 6
10238 Main task has work to do, go find it
10239 got dsc Start(0)
10240 start 0
10241 Jan 03 08:33:03.808 INFO request completed, latency_us: 250, response_code: 204, uri: /start/cid/0, method: POST, req_id: 7df83fc6-fc23-457a-b268-8597896b16f1, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
10242 Replay: started 0, returned:Ok(())
10243 ----------------------------------------------------------------
10244 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10245 States: Offline Active Active
10246 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10247 3415 Acked Read 5 Sent Done Done false
10248 3416 Acked Write 4 Sent Done Done false
10249 3417 Acked Write 7 Sent Done Done false
10250 3418 Acked Read 10 Sent Done Done false
10251 3419 Acked Read 3 Sent Done Done false
10252 3420 Acked Write 4 Sent Done Done false
10253 3421 Acked Read 5 Sent Done Done false
10254 3422 Acked Read 9 Sent Done Done false
10255 3423 Acked Read 1 Sent Done Done false
10256 3424 Acked Read 2 Sent Done Done false
10257 3425 Acked Flush 0 Sent Done Done false
10258 3426 Acked Write 3 Sent Done Done false
10259 3427 Acked Write 2 Sent Done Done false
10260 3428 Acked Read 8 Sent Done Done false
10261 3429 Acked Write 9 Sent Done Done false
10262 3430 Acked Write 1 Sent Done Done false
10263 3431 Acked Write 5 Sent Done Done false
10264 3432 Acked Read 7 Sent Done Done false
10265 3433 Acked Read 10 Sent Done Done false
10266 3434 Acked Write 8 Sent Done Done false
10267 3435 Acked Write 4 Sent Done Done false
10268 3436 Acked Read 3 Sent Done Done false
10269 3437 Acked Read 1 Sent Done Done false
10270 3438 Acked Flush 0 Sent Done Done false
10271 3439 Acked Write 5 Sent Done Done false
10272 3440 Acked Write 3 Sent Done Done false
10273 3441 Acked Read 5 Sent Done Done false
10274 3442 Acked Read 2 Sent Done Done false
10275 3443 Acked Read 2 Sent Done Done false
10276 3444 Acked Read 5 Sent Done Done false
10277 3445 Acked Read 7 Sent Done Done false
10278 3446 Acked Read 8 Sent Done Done false
10279 3447 Acked Read 10 Sent Done Done false
10280 3448 Acked Write 8 Sent Done Done false
10281 3449 Acked Read 5 Sent Done Done false
10282 3450 Acked Read 4 Sent Done Done false
10283 3451 Acked Read 9 Sent Done Done false
10284 3452 Acked Read 6 Sent Done Done false
10285 3453 Acked Write 5 Sent Done Done false
10286 3454 Acked Read 8 Sent Done Done false
10287 3455 Acked Flush 0 Sent Done Done false
10288 3456 Acked Read 4 Sent Done Done false
10289 3457 Acked Write 9 Sent Done Done false
10290 3458 Acked Read 5 Sent Done Done false
10291 3459 Acked Write 2 Sent Done Done false
10292 3460 Acked Read 6 Sent Done Done false
10293 3461 Acked Read 6 Sent Done Done false
10294 3462 Acked Write 5 Sent Done Done false
10295 3463 Acked Read 6 Sent Done Done false
10296 3464 Acked Write 3 Sent Done Done false
10297 3465 Acked Write 6 Sent Done Done false
10298 3466 Acked Read 1 Sent Done Done false
10299 3467 Acked Write 5 Sent Done Done false
10300 3468 Acked Read 2 Sent Done Done false
10301 3469 Acked Write 3 Sent Done Done false
10302 3470 Acked Read 4 Sent Done Done false
10303 3471 Acked Write 7 Sent Done Done false
10304 3472 Acked Write 4 Sent Done Done false
10305 3473 Acked Write 2 Sent Done Done false
10306 3474 Acked Read 3 Sent Done Done false
10307 3475 Acked Read 9 Sent Done Done false
10308 3476 Acked Write 9 Sent Done Done false
10309 3477 Acked Write 2 Sent Done Done false
10310 3478 Acked Read 7 Sent Done Done false
10311 3479 Acked Flush 0 Sent Done Done false
10312 3480 Acked Write 7 Sent Done Done false
10313 3481 Acked Read 8 Sent Done Done false
10314 3482 Acked Write 2 Sent Done Done false
10315 3483 Acked Read 5 Sent Done Done false
10316 3484 Acked Read 6 Sent Done Done false
10317 3485 Acked Read 10 Sent Done Done false
10318 3486 Acked Read 6 Sent Done Done false
10319 3487 Acked Write 10 Sent Done Done false
10320 3488 Acked Write 9 Sent Done Done false
10321 3489 Acked Read 4 Sent Done Done false
10322 3490 Acked Read 3 Sent Done Done false
10323 3491 Acked Read 4 Sent Done Done false
10324 3492 Acked Write 8 Sent Done Done false
10325 3493 Acked Flush 0 Sent Done Done false
10326 3494 Acked Write 7 Sent Done Done false
10327 3495 Acked Read 1 Sent Done Done false
10328 3496 Acked Write 6 Sent Done Done false
10329 3497 Acked Read 3 Sent Done Done false
10330 3498 Acked Read 3 Sent Done Done false
10331 3499 Acked Read 5 Sent Done Done false
10332 3500 Acked Read 1 Sent Done Done false
10333 3501 Acked Write 1 Sent Done Done false
10334 3502 Acked Write 10 Sent Done Done false
10335 3503 Acked Write 10 Sent Done Done false
10336 3504 Acked Read 2 Sent Done Done false
10337 3505 Acked Read 6 Sent Done Done false
10338 3506 Acked Read 1 Sent Done Done false
10339 3507 Acked Read 1 Sent Done Done false
10340 3508 Acked Write 10 Sent Done Done false
10341 3509 Acked Read 8 Sent Done Done false
10342 3510 Acked Flush 0 Sent Done Done false
10343 3511 Acked Write 6 Sent Done Done false
10344 3512 Acked Write 10 Sent Done Done false
10345 3513 Acked Flush 0 Sent Done Done false
10346 3514 Acked Write 8 Sent Done Done false
10347 3515 Acked Flush 0 Sent Done Done false
10348 3516 Acked Write 3 Sent Done Done false
10349 3517 Acked Write 5 Sent Done Done false
10350 3518 Acked Write 3 Sent Done Done false
10351 3519 Acked Read 8 Sent Done Done false
10352 3520 Acked Read 6 Sent Done Done false
10353 3521 Acked Write 2 Sent Done Done false
10354 3522 Acked Flush 0 Sent Done Done false
10355 3523 Acked Write 4 Sent Done Done false
10356 3524 Acked Read 4 Sent Done Done false
10357 3525 Acked Read 4 Sent Done Done false
10358 3526 Acked Write 6 Sent Done Done false
10359 3527 Acked Read 4 Sent Done Done false
10360 3528 Acked Write 4 Sent Done Done false
10361 3529 Acked Read 3 Sent Done Done false
10362 3530 Acked Write 3 Sent Done Done false
10363 3531 Acked Write 3 Sent Done Done false
10364 3532 Acked Read 1 Sent Done Done false
10365 3533 Acked Flush 0 Sent Done Done false
10366 3534 Acked Write 4 Sent Done Done false
10367 3535 Acked Read 2 Sent Done Done false
10368 3536 Acked Read 5 Sent Done Done false
10369 3537 Acked Flush 0 Sent Done Done false
10370 3538 Acked Read 3 Sent Done Done false
10371 3539 Acked Read 3 Sent Done Done false
10372 3540 Acked Write 8 Sent Done Done false
10373 3541 Acked Write 10 Sent Done Done false
10374 3542 Acked Read 3 Sent Done Done false
10375 3543 Acked Read 9 Sent Done Done false
10376 3544 Acked Write 6 Sent Done Done false
10377 3545 Acked Read 1 Sent Done Done false
10378 3546 Acked Write 6 Sent Done Done false
10379 3547 Acked Read 5 Sent Done Done false
10380 3548 Acked Read 5 Sent Done Done false
10381 3549 Acked Read 1 Sent Done Done false
10382 3550 Acked Read 6 Sent Done Done false
10383 3551 Acked Read 3 Sent Done Done false
10384 3552 Acked Read 3 Sent Done Done false
10385 3553 Acked Read 5 Sent Done Done false
10386 3554 Acked Read 7 Sent Done Done false
10387 3555 Acked Read 6 Sent Done Done false
10388 3556 Acked Write 3 Sent Done Done false
10389 3557 Acked Read 10 Sent Done Done false
10390 3558 Acked Write 6 Sent Done Done false
10391 3559 Acked Write 4 Sent Done Done false
10392 3560 Acked Read 4 Sent Done Done false
10393 3561 Acked Write 7 Sent Done Done false
10394 3562 Acked Read 9 Sent Done Done false
10395 3563 Acked Write 3 Sent Done Done false
10396 3564 Acked Read 4 Sent Done Done false
10397 3565 Acked Read 1 Sent Done Done false
10398 3566 Acked Write 9 Sent Done Done false
10399 3567 Acked Write 4 Sent Done Done false
10400 3568 Acked Flush 0 Sent Done Done false
10401 3569 Acked Write 8 Sent Done Done false
10402 3570 Acked Read 8 Sent Done Done false
10403 3571 Acked Read 2 Sent Done Done false
10404 3572 Acked Read 5 Sent Done Done false
10405 3573 Acked Read 10 Sent Done Done false
10406 3574 Acked Read 10 Sent Done Done false
10407 3575 Acked Read 1 Sent Done Done false
10408 3576 Acked Write 7 Sent Done Done false
10409 3577 Acked Write 5 Sent Done Done false
10410 3578 Acked Read 8 Sent Done Done false
10411 3579 Acked Read 3 Sent Done Done false
10412 3580 Acked Read 9 Sent Done Done false
10413 3581 Acked Read 7 Sent Done Done false
10414 3582 Acked Flush 0 Sent Done Done false
10415 3583 Acked Flush 0 Sent Done Done false
10416 3584 Acked Write 1 Sent Done Done false
10417 3585 Acked Write 5 Sent Done Done false
10418 3586 Acked Read 6 Sent Done Done false
10419 3587 Acked Read 5 Sent Done Done false
10420 3588 Acked Flush 0 Sent Done Done false
10421 3589 Acked Read 5 Sent Done Done false
10422 3590 Acked Read 7 Sent Done Done false
10423 3591 Acked Read 7 Sent Done Done false
10424 3592 Acked Read 2 Sent Done Done false
10425 3593 Acked Read 8 Sent Done Done false
10426 3594 Acked Write 1 Sent Done Done false
10427 3595 Acked Flush 0 Sent Done Done false
10428 3596 Acked Write 6 Sent Done Done false
10429 3597 Acked Write 7 Sent Done Done false
10430 3598 Acked Write 8 Sent Done Done false
10431 3599 Acked Read 8 Sent Done Done false
10432 3600 Acked Write 1 Sent Done Done false
10433 3601 Acked Write 7 Sent Done Done false
10434 3602 Acked Flush 0 Sent Done Done false
10435 3603 Acked Read 10 Sent Done Done false
10436 3604 Acked Flush 0 Sent Done Done false
10437 3605 Acked Flush 0 Sent Done Done false
10438 3606 Acked Flush 0 Sent Done Done false
10439 3607 Acked Read 8 Sent Done Done false
10440 3608 Acked Write 8 Sent Done Done false
10441 3609 Acked Read 9 Sent Done Done false
10442 3610 Acked Write 4 Sent Done Done false
10443 3611 Acked Read 4 Sent Done Done false
10444 3612 Acked Write 2 Sent Done Done false
10445 3613 Acked Read 9 Sent Done Done false
10446 3614 Acked Write 9 Sent Done Done false
10447 3615 Acked Write 5 Sent Done Done false
10448 3616 Acked Read 6 Sent Done Done false
10449 3617 Acked Read 5 Sent Done Done false
10450 3618 Acked Read 9 Sent Done Done false
10451 3619 Acked Read 4 Sent Done Done false
10452 3620 Acked Write 9 Sent Done Done false
10453 3621 Acked Write 1 Sent Done Done false
10454 3622 Acked Read 4 Sent Done Done false
10455 3623 Acked Write 7 Sent Done Done false
10456 3624 Acked Read 5 Sent Done Done false
10457 3625 Acked Read 1 Sent Done Done false
10458 3626 Acked Write 1 Sent Done Done false
10459 3627 Acked Write 2 Sent Done Done false
10460 3628 Acked Read 8 Sent Done Done false
10461 3629 Acked Write 10 Sent Done Done false
10462 3630 Acked Read 7 Sent Done Done false
10463 3631 Acked Write 5 Sent Done Done false
10464 3632 Acked Read 8 Sent Done Done false
10465 3633 Acked Write 1 Sent Done Done false
10466 3634 Acked Flush 0 Sent Done Done false
10467 3635 Acked Read 1 Sent Done Done false
10468 3636 Acked Read 9 Sent Done Done false
10469 3637 Acked Write 5 Sent Done Done false
10470 3638 Acked Write 6 Sent Done Done false
10471 3639 Acked Write 5 Sent Done Done false
10472 3640 Acked Read 7 Sent Done Done false
10473 3641 Acked Read 6 Sent Done Done false
10474 3642 Acked Read 5 Sent Done Done false
10475 3643 Acked Read 8 Sent Done Done false
10476 3644 Acked Write 5 Sent Done Done false
10477 3645 Acked Read 5 Sent Done Done false
10478 3646 Acked Read 7 Sent Done Done false
10479 3647 Acked Write 6 Sent Done Done false
10480 3648 Acked Read 10 Sent Done Done false
10481 3649 Acked Read 9 Sent Done Done false
10482 3650 Acked Read 1 Sent Done Done false
10483 3651 Acked Read 9 Sent Done Done false
10484 3652 Acked Write 7 Sent Done Done false
10485 3653 Acked Read 6 Sent Done Done false
10486 3654 Acked Write 5 Sent Done Done false
10487 3655 Acked Read 3 Sent Done Done false
10488 3656 Acked Write 7 Sent Done Done false
10489 3657 Acked Flush 0 Sent Done Done false
10490 3658 Acked Flush 0 Sent Done Done false
10491 3659 Acked Read 6 Sent Done Done false
10492 3660 Acked Read 9 Sent Done Done false
10493 3661 Acked Write 6 Sent Done Done false
10494 3662 Acked Write 10 Sent Done Done false
10495 3663 Acked Write 6 Sent Done Done false
10496 3664 Acked Read 10 Sent Done Done false
10497 3665 Acked Write 5 Sent Done Done false
10498 3666 Acked Read 4 Sent Done Done false
10499 3667 Acked Flush 0 Sent Done Done false
10500 3668 Acked Write 10 Sent Done Done false
10501 3669 Acked Write 5 Sent Done Done false
10502 3670 Acked Write 8 Sent Done Done false
10503 3671 Acked Read 7 Sent Done Done false
10504 3672 Acked Write 9 Sent Done Done false
10505 3673 Acked Write 4 Sent Done Done false
10506 3674 Acked Read 4 Sent Done Done false
10507 3675 Acked Read 6 Sent Done Done false
10508 3676 Acked Write 1 Sent Done Done false
10509 3677 Acked Read 2 Sent Done Done false
10510 3678 Acked Read 3 Sent Done Done false
10511 3679 Acked Read 2 Sent Done Done false
10512 3680 Acked Read 9 Sent Done Done false
10513 3681 Acked Read 2 Sent Done Done false
10514 3682 Acked Write 4 Sent Done Done false
10515 3683 Acked Flush 0 Sent Done Done false
10516 3684 Acked Read 6 Sent Done Done false
10517 3685 Acked Write 7 Sent Done Done false
10518 3686 Acked Read 10 Sent Done Done false
10519 3687 Acked Write 5 Sent Done Done false
10520 3688 Acked Write 6 Sent Done Done false
10521 3689 Acked Write 5 Sent Done Done false
10522 3690 Acked Read 3 Sent Done Done false
10523 3691 Acked Write 9 Sent Done Done false
10524 3692 Acked Read 7 Sent Done Done false
10525 3693 Acked Write 5 Sent Done Done false
10526 3694 Acked Read 4 Sent Done Done false
10527 3695 Acked Read 4 Sent Done Done false
10528 3696 Acked Read 8 Sent Done Done false
10529 3697 Acked Read 8 Sent Done Done false
10530 3698 Acked Read 6 Sent Done Done false
10531 3699 Acked Flush 0 Sent Done Done false
10532 3700 Acked Read 8 Sent Done Done false
10533 3701 Acked Flush 0 Sent Done Done false
10534 3702 Acked Write 6 Sent Done Done false
10535 3703 Acked Read 8 Sent Done Done false
10536 3704 Acked Read 9 Sent Done Done false
10537 3705 Acked Read 10 Sent Done Done false
10538 3706 Acked Flush 0 Sent Done Done false
10539 3707 Acked Write 8 Sent Done Done false
10540 3708 Acked Write 5 Sent Done Done false
10541 3709 Acked Write 1 Sent Done Done false
10542 3710 Acked Read 5 Sent Done Done false
10543 3711 Acked Read 1 Sent Done Done false
10544 3712 Acked Read 8 Sent Done Done false
10545 3713 Acked Read 2 Sent Done Done false
10546 3714 Acked Write 2 Sent Done Done false
10547 3715 Acked Write 8 Sent Done Done false
10548 STATES DS:0 DS:1 DS:2 TOTAL
10549 Sent 301 0 0 301
10550 Done 0 301 301 602
10551 Skipped 0 0 0 0
10552 Error 0 0 0 0
10553 Last Flush: 3414 3706 3706
10554 Downstairs last five completed: 3414 3413 3412 3411 3410
10555 Upstairs last five completed: 3715 3714 3713 3712 3711
10556 CLIENT: Up:0 ds:301 act:2
10557 [8810] got start action
10558 [8810] I am going to restart
10559 Starting downstairs at port 8810
10560 Make output file at "/tmp/dsc/downstairs-8810.txt"
10561 [8810][0] reports Starting
10562 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1214)
10563 [8810][0] reports Running
10564 ----------------------------------------------------------------
10565 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10566 States: Offline Active Active
10567 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10568 3415 Acked Read 5 Sent Done Done false
10569 3416 Acked Write 4 Sent Done Done false
10570 3417 Acked Write 7 Sent Done Done false
10571 3418 Acked Read 10 Sent Done Done false
10572 3419 Acked Read 3 Sent Done Done false
10573 3420 Acked Write 4 Sent Done Done false
10574 3421 Acked Read 5 Sent Done Done false
10575 3422 Acked Read 9 Sent Done Done false
10576 3423 Acked Read 1 Sent Done Done false
10577 3424 Acked Read 2 Sent Done Done false
10578 3425 Acked Flush 0 Sent Done Done false
10579 3426 Acked Write 3 Sent Done Done false
10580 3427 Acked Write 2 Sent Done Done false
10581 3428 Acked Read 8 Sent Done Done false
10582 3429 Acked Write 9 Sent Done Done false
10583 3430 Acked Write 1 Sent Done Done false
10584 3431 Acked Write 5 Sent Done Done false
10585 3432 Acked Read 7 Sent Done Done false
10586 3433 Acked Read 10 Sent Done Done false
10587 3434 Acked Write 8 Sent Done Done false
10588 3435 Acked Write 4 Sent Done Done false
10589 3436 Acked Read 3 Sent Done Done false
10590 3437 Acked Read 1 Sent Done Done false
10591 3438 Acked Flush 0 Sent Done Done false
10592 3439 Acked Write 5 Sent Done Done false
10593 3440 Acked Write 3 Sent Done Done false
10594 3441 Acked Read 5 Sent Done Done false
10595 3442 Acked Read 2 Sent Done Done false
10596 3443 Acked Read 2 Sent Done Done false
10597 3444 Acked Read 5 Sent Done Done false
10598 3445 Acked Read 7 Sent Done Done false
10599 3446 Acked Read 8 Sent Done Done false
10600 3447 Acked Read 10 Sent Done Done false
10601 3448 Acked Write 8 Sent Done Done false
10602 3449 Acked Read 5 Sent Done Done false
10603 3450 Acked Read 4 Sent Done Done false
10604 3451 Acked Read 9 Sent Done Done false
10605 3452 Acked Read 6 Sent Done Done false
10606 3453 Acked Write 5 Sent Done Done false
10607 3454 Acked Read 8 Sent Done Done false
10608 3455 Acked Flush 0 Sent Done Done false
10609 3456 Acked Read 4 Sent Done Done false
10610 3457 Acked Write 9 Sent Done Done false
10611 3458 Acked Read 5 Sent Done Done false
10612 3459 Acked Write 2 Sent Done Done false
10613 3460 Acked Read 6 Sent Done Done false
10614 3461 Acked Read 6 Sent Done Done false
10615 3462 Acked Write 5 Sent Done Done false
10616 3463 Acked Read 6 Sent Done Done false
10617 3464 Acked Write 3 Sent Done Done false
10618 3465 Acked Write 6 Sent Done Done false
10619 3466 Acked Read 1 Sent Done Done false
10620 3467 Acked Write 5 Sent Done Done false
10621 3468 Acked Read 2 Sent Done Done false
10622 3469 Acked Write 3 Sent Done Done false
10623 3470 Acked Read 4 Sent Done Done false
10624 3471 Acked Write 7 Sent Done Done false
10625 3472 Acked Write 4 Sent Done Done false
10626 3473 Acked Write 2 Sent Done Done false
10627 3474 Acked Read 3 Sent Done Done false
10628 3475 Acked Read 9 Sent Done Done false
10629 3476 Acked Write 9 Sent Done Done false
10630 3477 Acked Write 2 Sent Done Done false
10631 3478 Acked Read 7 Sent Done Done false
10632 3479 Acked Flush 0 Sent Done Done false
10633 3480 Acked Write 7 Sent Done Done false
10634 3481 Acked Read 8 Sent Done Done false
10635 3482 Acked Write 2 Sent Done Done false
10636 3483 Acked Read 5 Sent Done Done false
10637 3484 Acked Read 6 Sent Done Done false
10638 3485 Acked Read 10 Sent Done Done false
10639 3486 Acked Read 6 Sent Done Done false
10640 3487 Acked Write 10 Sent Done Done false
10641 3488 Acked Write 9 Sent Done Done false
10642 3489 Acked Read 4 Sent Done Done false
10643 3490 Acked Read 3 Sent Done Done false
10644 3491 Acked Read 4 Sent Done Done false
10645 3492 Acked Write 8 Sent Done Done false
10646 3493 Acked Flush 0 Sent Done Done false
10647 3494 Acked Write 7 Sent Done Done false
10648 3495 Acked Read 1 Sent Done Done false
10649 3496 Acked Write 6 Sent Done Done false
10650 3497 Acked Read 3 Sent Done Done false
10651 3498 Acked Read 3 Sent Done Done false
10652 3499 Acked Read 5 Sent Done Done false
10653 3500 Acked Read 1 Sent Done Done false
10654 3501 Acked Write 1 Sent Done Done false
10655 3502 Acked Write 10 Sent Done Done false
10656 3503 Acked Write 10 Sent Done Done false
10657 3504 Acked Read 2 Sent Done Done false
10658 3505 Acked Read 6 Sent Done Done false
10659 3506 Acked Read 1 Sent Done Done false
10660 3507 Acked Read 1 Sent Done Done false
10661 3508 Acked Write 10 Sent Done Done false
10662 3509 Acked Read 8 Sent Done Done false
10663 3510 Acked Flush 0 Sent Done Done false
10664 3511 Acked Write 6 Sent Done Done false
10665 3512 Acked Write 10 Sent Done Done false
10666 3513 Acked Flush 0 Sent Done Done false
10667 3514 Acked Write 8 Sent Done Done false
10668 3515 Acked Flush 0 Sent Done Done false
10669 3516 Acked Write 3 Sent Done Done false
10670 3517 Acked Write 5 Sent Done Done false
10671 3518 Acked Write 3 Sent Done Done false
10672 3519 Acked Read 8 Sent Done Done false
10673 3520 Acked Read 6 Sent Done Done false
10674 3521 Acked Write 2 Sent Done Done false
10675 3522 Acked Flush 0 Sent Done Done false
10676 3523 Acked Write 4 Sent Done Done false
10677 3524 Acked Read 4 Sent Done Done false
10678 3525 Acked Read 4 Sent Done Done false
10679 3526 Acked Write 6 Sent Done Done false
10680 3527 Acked Read 4 Sent Done Done false
10681 3528 Acked Write 4 Sent Done Done false
10682 3529 Acked Read 3 Sent Done Done false
10683 3530 Acked Write 3 Sent Done Done false
10684 3531 Acked Write 3 Sent Done Done false
10685 3532 Acked Read 1 Sent Done Done false
10686 3533 Acked Flush 0 Sent Done Done false
10687 3534 Acked Write 4 Sent Done Done false
10688 3535 Acked Read 2 Sent Done Done false
10689 3536 Acked Read 5 Sent Done Done false
10690 3537 Acked Flush 0 Sent Done Done false
10691 3538 Acked Read 3 Sent Done Done false
10692 3539 Acked Read 3 Sent Done Done false
10693 3540 Acked Write 8 Sent Done Done false
10694 3541 Acked Write 10 Sent Done Done false
10695 3542 Acked Read 3 Sent Done Done false
10696 3543 Acked Read 9 Sent Done Done false
10697 3544 Acked Write 6 Sent Done Done false
10698 3545 Acked Read 1 Sent Done Done false
10699 3546 Acked Write 6 Sent Done Done false
10700 3547 Acked Read 5 Sent Done Done false
10701 3548 Acked Read 5 Sent Done Done false
10702 3549 Acked Read 1 Sent Done Done false
10703 3550 Acked Read 6 Sent Done Done false
10704 3551 Acked Read 3 Sent Done Done false
10705 3552 Acked Read 3 Sent Done Done false
10706 3553 Acked Read 5 Sent Done Done false
10707 3554 Acked Read 7 Sent Done Done false
10708 3555 Acked Read 6 Sent Done Done false
10709 3556 Acked Write 3 Sent Done Done false
10710 3557 Acked Read 10 Sent Done Done false
10711 3558 Acked Write 6 Sent Done Done false
10712 3559 Acked Write 4 Sent Done Done false
10713 3560 Acked Read 4 Sent Done Done false
10714 3561 Acked Write 7 Sent Done Done false
10715 3562 Acked Read 9 Sent Done Done false
10716 3563 Acked Write 3 Sent Done Done false
10717 3564 Acked Read 4 Sent Done Done false
10718 3565 Acked Read 1 Sent Done Done false
10719 3566 Acked Write 9 Sent Done Done false
10720 3567 Acked Write 4 Sent Done Done false
10721 3568 Acked Flush 0 Sent Done Done false
10722 3569 Acked Write 8 Sent Done Done false
10723 3570 Acked Read 8 Sent Done Done false
10724 3571 Acked Read 2 Sent Done Done false
10725 3572 Acked Read 5 Sent Done Done false
10726 3573 Acked Read 10 Sent Done Done false
10727 3574 Acked Read 10 Sent Done Done false
10728 3575 Acked Read 1 Sent Done Done false
10729 3576 Acked Write 7 Sent Done Done false
10730 3577 Acked Write 5 Sent Done Done false
10731 3578 Acked Read 8 Sent Done Done false
10732 3579 Acked Read 3 Sent Done Done false
10733 3580 Acked Read 9 Sent Done Done false
10734 3581 Acked Read 7 Sent Done Done false
10735 3582 Acked Flush 0 Sent Done Done false
10736 3583 Acked Flush 0 Sent Done Done false
10737 3584 Acked Write 1 Sent Done Done false
10738 3585 Acked Write 5 Sent Done Done false
10739 3586 Acked Read 6 Sent Done Done false
10740 3587 Acked Read 5 Sent Done Done false
10741 3588 Acked Flush 0 Sent Done Done false
10742 3589 Acked Read 5 Sent Done Done false
10743 3590 Acked Read 7 Sent Done Done false
10744 3591 Acked Read 7 Sent Done Done false
10745 3592 Acked Read 2 Sent Done Done false
10746 3593 Acked Read 8 Sent Done Done false
10747 3594 Acked Write 1 Sent Done Done false
10748 3595 Acked Flush 0 Sent Done Done false
10749 3596 Acked Write 6 Sent Done Done false
10750 3597 Acked Write 7 Sent Done Done false
10751 3598 Acked Write 8 Sent Done Done false
10752 3599 Acked Read 8 Sent Done Done false
10753 3600 Acked Write 1 Sent Done Done false
10754 3601 Acked Write 7 Sent Done Done false
10755 3602 Acked Flush 0 Sent Done Done false
10756 3603 Acked Read 10 Sent Done Done false
10757 3604 Acked Flush 0 Sent Done Done false
10758 3605 Acked Flush 0 Sent Done Done false
10759 3606 Acked Flush 0 Sent Done Done false
10760 3607 Acked Read 8 Sent Done Done false
10761 3608 Acked Write 8 Sent Done Done false
10762 3609 Acked Read 9 Sent Done Done false
10763 3610 Acked Write 4 Sent Done Done false
10764 3611 Acked Read 4 Sent Done Done false
10765 3612 Acked Write 2 Sent Done Done false
10766 3613 Acked Read 9 Sent Done Done false
10767 3614 Acked Write 9 Sent Done Done false
10768 3615 Acked Write 5 Sent Done Done false
10769 3616 Acked Read 6 Sent Done Done false
10770 3617 Acked Read 5 Sent Done Done false
10771 3618 Acked Read 9 Sent Done Done false
10772 3619 Acked Read 4 Sent Done Done false
10773 3620 Acked Write 9 Sent Done Done false
10774 3621 Acked Write 1 Sent Done Done false
10775 3622 Acked Read 4 Sent Done Done false
10776 3623 Acked Write 7 Sent Done Done false
10777 3624 Acked Read 5 Sent Done Done false
10778 3625 Acked Read 1 Sent Done Done false
10779 3626 Acked Write 1 Sent Done Done false
10780 3627 Acked Write 2 Sent Done Done false
10781 3628 Acked Read 8 Sent Done Done false
10782 3629 Acked Write 10 Sent Done Done false
10783 3630 Acked Read 7 Sent Done Done false
10784 3631 Acked Write 5 Sent Done Done false
10785 3632 Acked Read 8 Sent Done Done false
10786 3633 Acked Write 1 Sent Done Done false
10787 3634 Acked Flush 0 Sent Done Done false
10788 3635 Acked Read 1 Sent Done Done false
10789 3636 Acked Read 9 Sent Done Done false
10790 3637 Acked Write 5 Sent Done Done false
10791 3638 Acked Write 6 Sent Done Done false
10792 3639 Acked Write 5 Sent Done Done false
10793 3640 Acked Read 7 Sent Done Done false
10794 3641 Acked Read 6 Sent Done Done false
10795 3642 Acked Read 5 Sent Done Done false
10796 3643 Acked Read 8 Sent Done Done false
10797 3644 Acked Write 5 Sent Done Done false
10798 3645 Acked Read 5 Sent Done Done false
10799 3646 Acked Read 7 Sent Done Done false
10800 3647 Acked Write 6 Sent Done Done false
10801 3648 Acked Read 10 Sent Done Done false
10802 3649 Acked Read 9 Sent Done Done false
10803 3650 Acked Read 1 Sent Done Done false
10804 3651 Acked Read 9 Sent Done Done false
10805 3652 Acked Write 7 Sent Done Done false
10806 3653 Acked Read 6 Sent Done Done false
10807 3654 Acked Write 5 Sent Done Done false
10808 3655 Acked Read 3 Sent Done Done false
10809 3656 Acked Write 7 Sent Done Done false
10810 3657 Acked Flush 0 Sent Done Done false
10811 3658 Acked Flush 0 Sent Done Done false
10812 3659 Acked Read 6 Sent Done Done false
10813 3660 Acked Read 9 Sent Done Done false
10814 3661 Acked Write 6 Sent Done Done false
10815 3662 Acked Write 10 Sent Done Done false
10816 3663 Acked Write 6 Sent Done Done false
10817 3664 Acked Read 10 Sent Done Done false
10818 3665 Acked Write 5 Sent Done Done false
10819 3666 Acked Read 4 Sent Done Done false
10820 3667 Acked Flush 0 Sent Done Done false
10821 3668 Acked Write 10 Sent Done Done false
10822 3669 Acked Write 5 Sent Done Done false
10823 3670 Acked Write 8 Sent Done Done false
10824 3671 Acked Read 7 Sent Done Done false
10825 3672 Acked Write 9 Sent Done Done false
10826 3673 Acked Write 4 Sent Done Done false
10827 3674 Acked Read 4 Sent Done Done false
10828 3675 Acked Read 6 Sent Done Done false
10829 3676 Acked Write 1 Sent Done Done false
10830 3677 Acked Read 2 Sent Done Done false
10831 3678 Acked Read 3 Sent Done Done false
10832 3679 Acked Read 2 Sent Done Done false
10833 3680 Acked Read 9 Sent Done Done false
10834 3681 Acked Read 2 Sent Done Done false
10835 3682 Acked Write 4 Sent Done Done false
10836 3683 Acked Flush 0 Sent Done Done false
10837 3684 Acked Read 6 Sent Done Done false
10838 3685 Acked Write 7 Sent Done Done false
10839 3686 Acked Read 10 Sent Done Done false
10840 3687 Acked Write 5 Sent Done Done false
10841 3688 Acked Write 6 Sent Done Done false
10842 3689 Acked Write 5 Sent Done Done false
10843 3690 Acked Read 3 Sent Done Done false
10844 3691 Acked Write 9 Sent Done Done false
10845 3692 Acked Read 7 Sent Done Done false
10846 3693 Acked Write 5 Sent Done Done false
10847 3694 Acked Read 4 Sent Done Done false
10848 3695 Acked Read 4 Sent Done Done false
10849 3696 Acked Read 8 Sent Done Done false
10850 3697 Acked Read 8 Sent Done Done false
10851 3698 Acked Read 6 Sent Done Done false
10852 3699 Acked Flush 0 Sent Done Done false
10853 3700 Acked Read 8 Sent Done Done false
10854 3701 Acked Flush 0 Sent Done Done false
10855 3702 Acked Write 6 Sent Done Done false
10856 3703 Acked Read 8 Sent Done Done false
10857 3704 Acked Read 9 Sent Done Done false
10858 3705 Acked Read 10 Sent Done Done false
10859 3706 Acked Flush 0 Sent Done Done false
10860 3707 Acked Write 8 Sent Done Done false
10861 3708 Acked Write 5 Sent Done Done false
10862 3709 Acked Write 1 Sent Done Done false
10863 3710 Acked Read 5 Sent Done Done false
10864 3711 Acked Read 1 Sent Done Done false
10865 3712 Acked Read 8 Sent Done Done false
10866 3713 Acked Read 2 Sent Done Done false
10867 3714 Acked Write 2 Sent Done Done false
10868 3715 Acked Write 8 Sent Done Done false
10869 3716 Acked Flush 0 Sent Done Done false
10870 STATES DS:0 DS:1 DS:2 TOTAL
10871 Sent 302 0 0 302
10872 Done 0 302 302 604
10873 Skipped 0 0 0 0
10874 Error 0 0 0 0
10875 Last Flush: 3414 3716 3716
10876 Downstairs last five completed: 3414 3413 3412 3411 3410
10877 Upstairs last five completed: 3716 3715 3714 3713 3712
10878 CLIENT: Up:0 ds:302 act:2
108792025-01-03T08:33:09.562ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108802025-01-03T08:33:09.562ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38297) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108812025-01-03T08:33:09.563ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108822025-01-03T08:33:09.564ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108832025-01-03T08:33:09.564ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108842025-01-03T08:33:09.564ZINFOcrucible: send last flush ID to this DS: 3414 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108852025-01-03T08:33:09.564ZINFOcrucible: Replied this last flush ID: 3414 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108862025-01-03T08:33:09.565ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108872025-01-03T08:33:09.565ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108882025-01-03T08:33:09.565ZINFOcrucible: [0] client re-new 302 jobs since flush 3414 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
108892025-01-03T08:33:09.565ZINFOcrucible: [0] Replayed 302 jobs since flush: 3414 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
10890 ----------------------------------------------------------------
10891 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10892 Downstairs last five completed: 3716 3715 3714 3713 3712
10893 Upstairs last five completed: 3716 3715 3714 3713 3712
10894 CLIENT: Up:0 ds:0 act:3
10895 Replay: All jobs finished
10896 Main task has work to do, go find it
10897 got dsc Stop(0)
10898 stop 0
10899 Jan 03 08:33:11.814 INFO request completed, latency_us[8810] Got stop action so:false kr:false
10900 : 338, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 28b399dc-db0e-4375-a253-aeb6c00136b4, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
10901 Jan 03 08:33:11.815 INFO request completed, latency_us: 132, response_code: 200, uri: /state/cid/0, method: GET, req_id: 4c3dc062-2a3c-4f05-ac1a-12e0cc3a7595, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
109022025-01-03T08:33:11.815ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
109032025-01-03T08:33:11.815ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
109042025-01-03T08:33:11.815ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
109052025-01-03T08:33:11.815ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
109062025-01-03T08:33:11.815ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
109072025-01-03T08:33:11.815ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
10908 [8810] Exited with: ExitStatus(unix_wait_status(9))
10909 [8810][0] reports Exit
10910 Jan 03 08:33:15.817 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3bc10634-2572-416a-9f50-9c5d7bacd3fd, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
10911 001/300 Write block 2439 len 32768 data: 2 2 2 2 3 3 3 3
10912 002/300 Write block 3228 len 40960 data: 2 2 3 3 4 4 4 4 3 4
10913 003/300 Flush
10914 004/300 Read block 4264 len 28672
10915 005/300 Flush
10916 006/300 Read block 2540 len 12288
10917 007/300 Read block 222 len 4096
10918 008/300 Write block 2151 len 4096 data: 2
10919 009/300 Write block 668 len 36864 data: 4 5 2 2 3 3 3 3 3
10920 010/300 Read block 869 len 12288
10921 011/300 Read block 1772 len 16384
10922 012/300 Read block 2678 len 12288
10923 013/300 Read block 564 len 8192
10924 014/300 Write block 4754 len 4096 data: 4
10925 015/300 Read block 2264 len 36864
10926 016/300 Write block 1419 len 32768 data: 5 5 3 3 3 3 3 3
10927 017/300 Write block 4441 len 4096 data: 6
10928 018/300 Flush
10929 019/300 Write block 3395 len 12288 data: 3 3 4
10930 020/300 Read block 2007 len 12288
10931 021/300 Write block 2320 len 36864 data: 3 3 3 3 3 3 3 4 4
10932 022/300 Flush
10933 023/300 Write block 608 len 36864 data: 3 3 2 3 3 3 3 3 4
10934 024/300 Flush
10935 025/300 Flush
10936 026/300 Read block 3610 len 32768
10937 027/300 Write block 1169 len 36864 data: 4 3 3 2 2 3 3 5 5
10938 028/300 Write block 4887 len 20480 data: 5 5 5 3 3
10939 029/300 Write block 2078 len 24576 data: 4 2 2 2 2 2
10940 030/300 Read block 3895 len 32768
10941 031/300 Read block 4834 len 16384
10942 032/300 Read block 1530 len 36864
10943 033/300 Write block 2308 len 4096 data: 3
10944 034/300 Write block 2413 len 16384 data: 2 4 4 4
10945 035/300 Write block 1939 len 28672 data: 3 4 4 4 4 4 4
10946 036/300 Write block 3104 len 8192 data: 2 2
10947 037/300 Read block 4354 len 40960
10948 038/300 Read block 4692 len 32768
10949 039/300 Write block 3243 len 40960 data: 2 2 4 4 4 4 4 4 3 3
10950 040/300 Write block 1420 len 20480 data: 6 4 4 4 4
10951 041/300 Write block 3532 len 40960 data: 3 3 3 4 3 3 3 3 3 3
10952 042/300 Read block 914 len 20480
10953 043/300 Read block 4805 len 16384
10954 044/300 Write block 4938 len 20480 data: 5 5 4 4 5
10955 045/300 Read block 1142 len 4096
10956 046/300 Flush
10957 047/300 Write block 598 len 28672 data: 3 3 3 3 2 3 3
10958 048/300 Read block 1857 len 32768
10959 049/300 Write block 2133 len 32768 data: 3 3 2 3 3 2 2 2
10960 050/300 Write block 1367 len 16384 data: 3 3 3 4
10961 051/300 Write block 1923 len 40960 data: 6 6 5 5 3 2 2 2 2 2
10962 052/300 Write block 417 len 40960 data: 3 3 4 4 4 4 3 3 3 3
10963 053/300 Write block 2091 len 28672 data: 2 2 2 2 2 2 3
10964 054/300 Flush
10965 055/300 Write block 3581 len 8192 data: 3 2
10966 056/300 Read block 1009 len 16384
10967 057/300 Read block 231 len 28672
10968 058/300 Write block 1779 len 40960 data: 2 2 3 3 3 3 2 3 3 3
10969 059/300 Flush
10970 060/300 Read block 1257 len 32768
10971 061/300 Write block 3404 len 16384 data: 3 3 3 4
10972 062/300 Read block 2713 len 8192
10973 063/300 Read block 267 len 24576
10974 064/300 Write block 4768 len 28672 data: 3 5 4 4 2 2 2
10975 065/300 Write block 857 len 4096 data: 3
10976 066/300 Write block 3204 len 8192 data: 3 3
10977 067/300 Write block 4762 len 36864 data: 2 2 2 4 3 3 4 6 5
10978 068/300 Write block 790 len 4096 data: 3
10979 069/300 Write block 3276 len 12288 data: 3 4 3
10980 070/300 Read block 2181 len 24576
10981 071/300 Write block 784 len 32768 data: 4 4 4 3 3 3 4 3
10982 072/300 Write block 1585 len 40960 data: 3 3 3 3 3 3 3 2 4 5
10983 073/300 Read block 3587 len 40960
10984 074/300 Read block 2209 len 4096
10985 075/300 Read block 3094 len 4096
10986 076/300 Read block 2621 len 40960
10987 077/300 Read block 2353 len 40960
10988 078/300 Write block 1501 len 24576 data: 3 3 3 3 3 3
10989 079/300 Write block 2694 len 36864 data: 3 4 4 4 4 3 4 4 3
10990 080/300 Read block 4412 len 4096
10991 081/300 Read block 2945 len 16384
10992 082/300 Write block 248 len 40960 data: 2 3 3 3 3 3 3 4 3 4
10993 083/300 Write block 797 len 40960 data: 4 5 5 4 4 4 3 5 3 3
10994 084/300 Read block 1072 len 12288
10995 085/300 Write block 2812 len 20480 data: 6 6 6 4 4
10996 086/300 Read block 3968 len 8192
10997 087/300 Write block 4551 len 4096 data: 2
10998 088/300 Read block 2994 len 16384
10999 089/300 Write block 4705 len 16384 data: 5 5 6 6
11000 090/300 Read block 1523 len 36864
11001 091/300 Read block 2964 len 40960
11002 092/300 Write block 4564 len 36864 data: 3 3 2 2 2 2 2 2 2
11003 093/300 Write block 491 len 16384 data: 4 4 3 3
11004 094/300 Read block 3394 len 8192
11005 095/300 Read block 606 len 24576
11006 096/300 Read block 1846 len 36864
11007 097/300 Flush
11008 098/300 Read block 4376 len 28672
11009 099/300 Write block 784 len 28672 data: 5 5 5 4 4 4 5
11010 100/300 Flush
11011 101/300 Read block 3351 len 20480
11012 102/300 Write block 1961 len 12288 data: 3 3 3
11013 103/300 Read block 1079 len 24576
11014 104/300 Write block 3919 len 36864 data: 2 2 2 2 2 2 2 2 2
11015 105/300 Flush
11016 106/300 Read block 1181 len 8192
11017 107/300 Read block 3811 len 36864
11018 108/300 Read block 3291 len 24576
11019 109/300 Read block 3909 len 36864
11020 110/300 Read block 3006 len 20480
11021 111/300 Write block 434 len 28672 data: 3 5 4 4 4 5 5
11022 112/300 Read block 3973 len 24576
11023 113/300 Write block 2716 len 8192 data: 4 3
11024 114/300 Read block 339 len 4096
11025 115/300 Write block 3936 len 4096 data: 2
11026 116/300 Write block 4853 len 20480 data: 3 3 3 2 3
11027 117/300 Write block 4331 len 16384 data: 2 2 3 4
11028 118/300 Read block 1021 len 20480
11029 119/300 Write block 3431 len 40960 data: 4 5 5 5 5 4 4 4 3 3
11030 120/300 Read block 1463 len 8192
11031 121/300 Write block 3108 len 16384 data: 2 2 2 2
11032 122/300 Write block 4342 len 16384 data: 4 3 3 3
11033 123/300 Write block 1731 len 8192 data: 3 3
11034 124/300 Write block 4201 len 4096 data: 6
11035 125/300 Write block 817 len 40960 data: 6 5 5 6 4 4 3 4 3 4
11036 126/300 Flush
11037 127/300 Write block 3605 len 4096 data: 5
11038 128/300 Write block 3156 len 32768 data: 2 2 2 2 3 3 3 3
11039 129/300 Flush
11040 130/300 Write block 3375 len 4096 data: 2
11041 131/300 Read block 77 len 24576
11042 132/300 Write block 2289 len 4096 data: 3
11043 133/300 Read block 2485 len 24576
11044 134/300 Write block 1218 len 32768 data: 4 4 4 4 3 4 5 5
11045 135/300 Write block 2878 len 20480 data: 2 2 2 2 2
11046 136/300 Flush
11047 137/300 Read block 2853 len 16384
11048 138/300 Write block 3286 len 24576 data: 3 2 3 3 4 5
11049 139/300 Flush
11050 140/300 Write block 2421 len 24576 data: 2 2 2 2 2 2
11051 141/300 Read block 3077 len 12288
11052 142/300 Write block 723 len 20480 data: 3 3 3 3 3
11053 143/300 Read block 946 len 40960
11054 144/300 Flush
11055 145/300 Write block 4119 len 36864 data: 5 3 4 4 4 4 3 3 3
11056 146/300 Write block 4027 len 8192 data: 2 2
11057 147/300 Read block 1918 len 8192
11058 148/300 Write block 139 len 24576 data: 4 3 3 3 3 3
11059 149/300 Read block 1489 len 24576
11060 150/300 Flush
11061 151/300 Read block 2877 len 28672
11062 152/300 Read block 4826 len 16384
11063 153/300 Write block 2567 len 24576 data: 2 5 5 4 4 4
11064 154/300 Write block 2676 len 4096 data: 2
11065 155/300 Write block 3270 len 16384 data: 4 4 3 3
11066 156/300 Flush
11067 157/300 Write block 2741 len 12288 data: 3 2 4
11068 158/300 Write block 4879 len 12288 data: 4 4 5
11069 159/300 Write block 1808 len 36864 data: 2 2 2 2 2 2 2 2 2
11070 160/300 Read block 666 len 16384
11071 161/300 Write block 563 len 32768 data: 3 3 2 2 2 2 2 2
11072 162/300 Read block 3192 len 20480
11073 163/300 Write block 4552 len 28672 data: 2 2 2 2 2 3 3
11074 164/300 Read block 3954 len 16384
11075 165/300 Read block 4802 len 4096
11076 166/300 Read block 1376 len 12288
11077 167/300 Flush
11078 168/300 Read block 1371 len 20480
11079 169/300 Read block 4688 len 16384
11080 170/300 Read block 566 len 40960
11081 171/300 Read block 666 len 24576
11082 172/300 Read block 2772 len 32768
11083 173/300 Read block 2755 len 24576
11084 174/300 Read block 4661 len 24576
11085 175/300 Read block 4176 len 36864
11086 176/300 Read block 4239 len 36864
11087 177/300 Flush
11088 178/300 Read block 4015 len 8192
11089 179/300 Write block 2513 len 16384 data: 2 2 2 3
11090 180/300 Write block 1135 len 8192 data: 3 3
11091 181/300 Read block 3740 len 16384
11092 182/300 Read block 1162 len 32768
11093 183/300 Read block 2317 len 20480
11094 184/300 Read block 1242 len 16384
11095 185/300 Read block 1663 len 12288
11096 186/300 Read block 3764 len 28672
11097 187/300 Write block 96 len 24576 data: 3 3 3 3 3 3
11098 188/300 Read block 4189 len 40960
11099 189/300 Write block 710 len 12288 data: 6 6 5
11100 190/300 Read block 2877 len 20480
11101 191/300 Write block 3241 len 12288 data: 3 3 3
11102 192/300 Write block 2380 len 24576 data: 2 2 3 4 4 6
11103 193/300 Read block 1382 len 32768
11104 194/300 Read block 2926 len 12288
11105 195/300 Write block 3838 len 12288 data: 2 2 2
11106 196/300 Write block 2520 len 24576 data: 6 6 6 7 5 5
11107 197/300 Write block 3331 len 16384 data: 2 2 2 2
11108 198/300 Write block 619 len 12288 data: 4 3 3
11109 199/300 Flush
11110 200/300 Read block 4991 len 32768
11111 201/300 Read block 890 len 8192
11112 202/300 Flush
11113 203/300 Read block 1678 len 4096
11114 204/300 Read block 2613 len 40960
11115 205/300 Write block 346 len 36864 data: 7 7 5 5 5 5 5 3 3
11116 206/300 Write block 1896 len 36864 data: 4 3 3 3 2 2 2 2 3
11117 207/300 Write block 1623 len 24576 data: 2 2 2 3 3 4
11118 208/300 Read block 1059 len 24576
11119 209/300 Read block 4733 len 4096
11120 210/300 Write block 1388 len 36864 data: 4 3 3 4 3 3 3 3 5
11121 211/300 Read block 2229 len 16384
11122 212/300 Read block 4899 len 16384
11123 213/300 Read block 2750 len 12288
11124 214/300 Read block 840 len 24576
11125 215/300 Flush
11126 216/300 Write block 2475 len 40960 data: 3 3 2 2 3 3 3 4 4 4
11127 217/300 Write block 1878 len 20480 data: 4 3 2 3 5
11128 218/300 Read block 195 len 20480
11129 219/300 Read block 182 len 4096
11130 220/300 Read block 1206 len 40960
11131 221/300 Read block 743 len 8192
11132 222/300 Write block 1154 len 28672 data: 2 2 2 2 2 2 3
11133 223/300 Read block 3930 len 24576
11134 224/300 Read block 2578 len 16384
11135 225/300 Read block 4325 len 24576
11136 226/300 Flush
11137 227/300 Read block 1972 len 4096
11138 228/300 Write block 3083 len 40960 data: 5 4 5 3 3 3 4 4 4 3
11139 229/300 Write block 4848 len 20480 data: 2 2 2 2 2
11140 230/300 Write block 2849 len 36864 data: 3 2 2 2 2 3 3 3 4
11141 231/300 Write block 4117 len 12288 data: 6 5 6
11142 232/300 Write block 612 len 16384 data: 4 4 4 4
11143 233/300 Flush
11144 234/300 Read block 4099 len 8192
11145 235/300 Read block 3094 len 24576
11146 236/300 Write block 271 len 4096 data: 2
11147 237/300 Write block 2813 len 24576 data: 7 7 5 5 4 4
11148 238/300 Write block 1903 len 4096 data: 3
11149 239/300 Read block 3181 len 32768
11150 240/300 Read block 876 len 28672
11151 241/300 Flush
11152 242/300 Write block 3861 len 32768 data: 4 5 6 6 5 6 7 6
11153 243/300 Read block 2366 len 4096
11154 244/300 Flush
11155 245/300 Read block 3875 len 12288
11156 246/300 Read block 3011 len 40960
11157 247/300 Write block 876 len 28672 data: 3 3 3 3 3 3 3
11158 248/300 Read block 1371 len 20480
11159 249/300 Read block 2541 len 32768
11160 250/300 Read block 4496 len 24576
11161 251/300 Read block 3185 len 24576
11162 252/300 Flush
11163 253/300 Read block 605 len 20480
11164 254/300 Read block 966 len 20480
11165 255/300 Write block 1354 len 28672 data: 4 4 4 4 4 3 3
11166 256/300 Read block 75 len 4096
11167 257/300 Read block 2545 len 20480
11168 258/300 Read block 801 len 24576
11169 259/300 Flush
11170 260/300 Read block 2438 len 20480
11171 261/300 Flush
11172 262/300 Read block 1169 len 40960
11173 263/300 Flush
11174 264/300 Read block 4217 len 24576
11175 265/300 Flush
11176 266/300 Write block 4396 len 8192 data: 2 2
11177 267/300 Read block 3307 len 28672
11178 268/300 Write block 4597 len 12288 data: 2 2 3
11179 269/300 Read block 3587 len 20480
11180 270/300 Flush
11181 271/300 Read block 3569 len 16384
11182 272/300 Flush
11183 273/300 Read block 2154 len 20480
11184 274/300 Write block 1434 len 32768 data: 3 3 3 3 2 2 3 4
11185 275/300 Read block 3512 len 20480
11186 276/300 Write block 4872 len 4096 data: 4
11187 277/300 Read block 1680 len 28672
11188 278/300 Write block 4680 len 36864 data: 4 4 3 3 3 3 2 3 3
11189 279/300 Write block 1800 len 32768 data: 3 3 3 2 2 2 2 2
11190 280/300 Read block 332 len 28672
11191 281/300 Read block 2613 len 36864
11192 282/300 Flush
11193 283/300 Write block 2834 len 16384 data: 4 3 3 3
11194 284/300 Write block 724 len 8192 data: 4 4
11195 285/300 Read block 3207 len 28672
11196 286/300 Read block 3064 len 8192
11197 287/300 Read block 848 len 8192
11198 288/300 Read block 4480 len 28672
11199 289/300 Flush
11200 290/300 Read block 35 len 8192
11201 291/300 Read block 2494 len 28672
11202 292/300 Read block 1706 len 16384
11203 293/300 Read block 1482 len 36864
11204 294/300 Read block 11 len 4096
11205 295/300 Write block 920 len 40960 data: 5 5 6 5 3 4 4 3 3 2
11206 296/300 Write block 2492 len 4096 data: 3
11207 297/300 Read block 4984 len 4096
11208 298/300 Read block 1869 len 28672
11209 299/300 Read block 2970 len 24576
11210 300/300 Write block 4207 len 40960 data: 4 3 2 2 3 3 4 4 3 4
11211 301/300 Read block 3785 len 8192
11212 Main task has work to do, go find it
11213 got dsc Start(0)
11214 start 0
11215 Jan 03 08:33:16.052 INFO request completed, latency_us: 214, response_code: 204, uri: /start/cid/0, method: POST, req_id: c196261e-ccef-4dc0-a384-81aa75370b97, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
11216 Replay: started 0, returned:Ok(())
11217 ----------------------------------------------------------------
11218 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
11219 States: Offline Active Active
11220 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11221 3717 Acked Write 8 Sent Done Done false
11222 3718 Acked Write 10 Sent Done Done false
11223 3719 Acked Flush 0 Sent Done Done false
11224 3720 Acked Read 7 Sent Done Done false
11225 3721 Acked Flush 0 Sent Done Done false
11226 3722 Acked Read 3 Sent Done Done false
11227 3723 Acked Read 1 Sent Done Done false
11228 3724 Acked Write 1 Sent Done Done false
11229 3725 Acked Write 9 Sent Done Done false
11230 3726 Acked Read 3 Sent Done Done false
11231 3727 Acked Read 4 Sent Done Done false
11232 3728 Acked Read 3 Sent Done Done false
11233 3729 Acked Read 2 Sent Done Done false
11234 3730 Acked Write 1 Sent Done Done false
11235 3731 Acked Read 9 Sent Done Done false
11236 3732 Acked Write 8 Sent Done Done false
11237 3733 Acked Write 1 Sent Done Done false
11238 3734 Acked Flush 0 Sent Done Done false
11239 3735 Acked Write 3 Sent Done Done false
11240 3736 Acked Read 3 Sent Done Done false
11241 3737 Acked Write 9 Sent Done Done false
11242 3738 Acked Flush 0 Sent Done Done false
11243 3739 Acked Write 9 Sent Done Done false
11244 3740 Acked Flush 0 Sent Done Done false
11245 3741 Acked Flush 0 Sent Done Done false
11246 3742 Acked Read 8 Sent Done Done false
11247 3743 Acked Write 9 Sent Done Done false
11248 3744 Acked Write 5 Sent Done Done false
11249 3745 Acked Write 6 Sent Done Done false
11250 3746 Acked Read 8 Sent Done Done false
11251 3747 Acked Read 4 Sent Done Done false
11252 3748 Acked Read 9 Sent Done Done false
11253 3749 Acked Write 1 Sent Done Done false
11254 3750 Acked Write 4 Sent Done Done false
11255 3751 Acked Write 7 Sent Done Done false
11256 3752 Acked Write 2 Sent Done Done false
11257 3753 Acked Read 10 Sent Done Done false
11258 3754 Acked Read 8 Sent Done Done false
11259 3755 Acked Write 10 Sent Done Done false
11260 3756 Acked Write 5 Sent Done Done false
11261 3757 Acked Write 10 Sent Done Done false
11262 3758 Acked Read 5 Sent Done Done false
11263 3759 Acked Read 4 Sent Done Done false
11264 3760 Acked Write 5 Sent Done Done false
11265 3761 Acked Read 1 Sent Done Done false
11266 3762 Acked Flush 0 Sent Done Done false
11267 3763 Acked Write 7 Sent Done Done false
11268 3764 Acked Read 8 Sent Done Done false
11269 3765 Acked Write 8 Sent Done Done false
11270 3766 Acked Write 4 Sent Done Done false
11271 3767 Acked Write 10 Sent Done Done false
11272 3768 Acked Write 10 Sent Done Done false
11273 3769 Acked Write 7 Sent Done Done false
11274 3770 Acked Flush 0 Sent Done Done false
11275 3771 Acked Write 2 Sent Done Done false
11276 3772 Acked Read 4 Sent Done Done false
11277 3773 Acked Read 7 Sent Done Done false
11278 3774 Acked Write 10 Sent Done Done false
11279 3775 Acked Flush 0 Sent Done Done false
11280 3776 Acked Read 8 Sent Done Done false
11281 3777 Acked Write 4 Sent Done Done false
11282 3778 Acked Read 2 Sent Done Done false
11283 3779 Acked Read 6 Sent Done Done false
11284 3780 Acked Write 7 Sent Done Done false
11285 3781 Acked Write 1 Sent Done Done false
11286 3782 Acked Write 2 Sent Done Done false
11287 3783 Acked Write 9 Sent Done Done false
11288 3784 Acked Write 1 Sent Done Done false
11289 3785 Acked Write 3 Sent Done Done false
11290 3786 Acked Read 6 Sent Done Done false
11291 3787 Acked Write 8 Sent Done Done false
11292 3788 Acked Write 10 Sent Done Done false
11293 3789 Acked Read 10 Sent Done Done false
11294 3790 Acked Read 1 Sent Done Done false
11295 3791 Acked Read 1 Sent Done Done false
11296 3792 Acked Read 10 Sent Done Done false
11297 3793 Acked Read 10 Sent Done Done false
11298 3794 Acked Write 6 Sent Done Done false
11299 3795 Acked Write 9 Sent Done Done false
11300 3796 Acked Read 1 Sent Done Done false
11301 3797 Acked Read 4 Sent Done Done false
11302 3798 Acked Write 10 Sent Done Done false
11303 3799 Acked Write 10 Sent Done Done false
11304 3800 Acked Read 3 Sent Done Done false
11305 3801 Acked Write 5 Sent Done Done false
11306 3802 Acked Read 2 Sent Done Done false
11307 3803 Acked Write 1 Sent Done Done false
11308 3804 Acked Read 4 Sent Done Done false
11309 3805 Acked Write 4 Sent Done Done false
11310 3806 Acked Read 9 Sent Done Done false
11311 3807 Acked Read 10 Sent Done Done false
11312 3808 Acked Write 9 Sent Done Done false
11313 3809 Acked Write 4 Sent Done Done false
11314 3810 Acked Read 2 Sent Done Done false
11315 3811 Acked Read 6 Sent Done Done false
11316 3812 Acked Read 9 Sent Done Done false
11317 3813 Acked Flush 0 Sent Done Done false
11318 3814 Acked Read 7 Sent Done Done false
11319 3815 Acked Write 7 Sent Done Done false
11320 3816 Acked Flush 0 Sent Done Done false
11321 3817 Acked Read 5 Sent Done Done false
11322 3818 Acked Write 3 Sent Done Done false
11323 3819 Acked Read 6 Sent Done Done false
11324 3820 Acked Write 9 Sent Done Done false
11325 3821 Acked Flush 0 Sent Done Done false
11326 3822 Acked Read 2 Sent Done Done false
11327 3823 Acked Read 9 Sent Done Done false
11328 3824 Acked Read 6 Sent Done Done false
11329 3825 Acked Read 9 Sent Done Done false
11330 3826 Acked Read 5 Sent Done Done false
11331 3827 Acked Write 7 Sent Done Done false
11332 3828 Acked Read 6 Sent Done Done false
11333 3829 Acked Write 2 Sent Done Done false
11334 3830 Acked Read 1 Sent Done Done false
11335 3831 Acked Write 1 Sent Done Done false
11336 3832 Acked Write 5 Sent Done Done false
11337 3833 Acked Write 4 Sent Done Done false
11338 3834 Acked Read 5 Sent Done Done false
11339 3835 Acked Write 10 Sent Done Done false
11340 3836 Acked Read 2 Sent Done Done false
11341 3837 Acked Write 4 Sent Done Done false
11342 3838 Acked Write 4 Sent Done Done false
11343 3839 Acked Write 2 Sent Done Done false
11344 3840 Acked Write 1 Sent Done Done false
11345 3841 Acked Write 10 Sent Done Done false
11346 3842 Acked Flush 0 Sent Done Done false
11347 3843 Acked Write 1 Sent Done Done false
11348 3844 Acked Write 8 Sent Done Done false
11349 3845 Acked Flush 0 Sent Done Done false
11350 3846 Acked Write 1 Sent Done Done false
11351 3847 Acked Read 6 Sent Done Done false
11352 3848 Acked Write 1 Sent Done Done false
11353 3849 Acked Read 6 Sent Done Done false
11354 3850 Acked Write 8 Sent Done Done false
11355 3851 Acked Write 5 Sent Done Done false
11356 3852 Acked Flush 0 Sent Done Done false
11357 3853 Acked Read 4 Sent Done Done false
11358 3854 Acked Write 6 Sent Done Done false
11359 3855 Acked Flush 0 Sent Done Done false
11360 3856 Acked Write 6 Sent Done Done false
11361 3857 Acked Read 3 Sent Done Done false
11362 3858 Acked Write 5 Sent Done Done false
11363 3859 Acked Read 10 Sent Done Done false
11364 3860 Acked Flush 0 Sent Done Done false
11365 3861 Acked Write 9 Sent Done Done false
11366 3862 Acked Write 2 Sent Done Done false
11367 3863 Acked Read 2 Sent Done Done false
11368 3864 Acked Write 6 Sent Done Done false
11369 3865 Acked Read 6 Sent Done Done false
11370 3866 Acked Flush 0 Sent Done Done false
11371 3867 Acked Read 7 Sent Done Done false
11372 3868 Acked Read 4 Sent Done Done false
11373 3869 Acked Write 6 Sent Done Done false
11374 3870 Acked Write 1 Sent Done Done false
11375 3871 Acked Write 4 Sent Done Done false
11376 3872 Acked Flush 0 Sent Done Done false
11377 3873 Acked Write 3 Sent Done Done false
11378 3874 Acked Write 3 Sent Done Done false
11379 3875 Acked Write 9 Sent Done Done false
11380 3876 Acked Read 4 Sent Done Done false
11381 3877 Acked Write 8 Sent Done Done false
11382 3878 Acked Read 5 Sent Done Done false
11383 3879 Acked Write 7 Sent Done Done false
11384 3880 Acked Read 4 Sent Done Done false
11385 3881 Acked Read 1 Sent Done Done false
11386 3882 Acked Read 3 Sent Done Done false
11387 3883 Acked Flush 0 Sent Done Done false
11388 3884 Acked Read 5 Sent Done Done false
11389 3885 Acked Read 4 Sent Done Done false
11390 3886 Acked Read 10 Sent Done Done false
11391 3887 Acked Read 6 Sent Done Done false
11392 3888 Acked Read 8 Sent Done Done false
11393 3889 Acked Read 6 Sent Done Done false
11394 3890 Acked Read 6 Sent Done Done false
11395 3891 Acked Read 9 Sent Done Done false
11396 3892 Acked Read 9 Sent Done Done false
11397 3893 Acked Flush 0 Sent Done Done false
11398 3894 Acked Read 2 Sent Done Done false
11399 3895 Acked Write 4 Sent Done Done false
11400 3896 Acked Write 2 Sent Done Done false
11401 3897 Acked Read 4 Sent Done Done false
11402 3898 Acked Read 8 Sent Done Done false
11403 3899 Acked Read 5 Sent Done Done false
11404 3900 Acked Read 4 Sent Done Done false
11405 3901 Acked Read 3 Sent Done Done false
11406 3902 Acked Read 7 Sent Done Done false
11407 3903 Acked Write 6 Sent Done Done false
11408 3904 Acked Read 10 Sent Done Done false
11409 3905 Acked Write 3 Sent Done Done false
11410 3906 Acked Read 5 Sent Done Done false
11411 3907 Acked Write 3 Sent Done Done false
11412 3908 Acked Write 6 Sent Done Done false
11413 3909 Acked Read 8 Sent Done Done false
11414 3910 Acked Read 3 Sent Done Done false
11415 3911 Acked Write 3 Sent Done Done false
11416 3912 Acked Write 6 Sent Done Done false
11417 3913 Acked Write 4 Sent Done Done false
11418 3914 Acked Write 3 Sent Done Done false
11419 3915 Acked Flush 0 Sent Done Done false
11420 3916 Acked Read 8 Sent Done Done false
11421 3917 Acked Read 2 Sent Done Done false
11422 3918 Acked Flush 0 Sent Done Done false
11423 3919 Acked Read 1 Sent Done Done false
11424 3920 Acked Read 10 Sent Done Done false
11425 3921 Acked Write 9 Sent Done Done false
11426 3922 Acked Write 9 Sent Done Done false
11427 3923 Acked Write 6 Sent Done Done false
11428 3924 Acked Read 6 Sent Done Done false
11429 3925 Acked Read 1 Sent Done Done false
11430 3926 Acked Write 9 Sent Done Done false
11431 3927 Acked Read 4 Sent Done Done false
11432 3928 Acked Read 4 Sent Done Done false
11433 3929 Acked Read 3 Sent Done Done false
11434 3930 Acked Read 6 Sent Done Done false
11435 3931 Acked Flush 0 Sent Done Done false
11436 3932 Acked Write 10 Sent Done Done false
11437 3933 Acked Write 5 Sent Done Done false
11438 3934 Acked Read 5 Sent Done Done false
11439 3935 Acked Read 1 Sent Done Done false
11440 3936 Acked Read 10 Sent Done Done false
11441 3937 Acked Read 2 Sent Done Done false
11442 3938 Acked Write 7 Sent Done Done false
11443 3939 Acked Read 6 Sent Done Done false
11444 3940 Acked Read 4 Sent Done Done false
11445 3941 Acked Read 6 Sent Done Done false
11446 3942 Acked Flush 0 Sent Done Done false
11447 3943 Acked Read 1 Sent Done Done false
11448 3944 Acked Write 10 Sent Done Done false
11449 3945 Acked Write 5 Sent Done Done false
11450 3946 Acked Write 9 Sent Done Done false
11451 3947 Acked Write 3 Sent Done Done false
11452 3948 Acked Write 4 Sent Done Done false
11453 3949 Acked Flush 0 Sent Done Done false
11454 3950 Acked Read 2 Sent Done Done false
11455 3951 Acked Read 6 Sent Done Done false
11456 3952 Acked Write 1 Sent Done Done false
11457 3953 Acked Write 6 Sent Done Done false
11458 3954 Acked Write 1 Sent Done Done false
11459 3955 Acked Read 8 Sent Done Done false
11460 3956 Acked Read 7 Sent Done Done false
11461 3957 Acked Flush 0 Sent Done Done false
11462 3958 Acked Write 8 Sent Done Done false
11463 3959 Acked Read 1 Sent Done Done false
11464 3960 Acked Flush 0 Sent Done Done false
11465 3961 Acked Read 3 Sent Done Done false
11466 3962 Acked Read 10 Sent Done Done false
11467 3963 Acked Write 7 Sent Done Done false
11468 3964 Acked Read 5 Sent Done Done false
11469 3965 Acked Read 8 Sent Done Done false
11470 3966 Acked Read 6 Sent Done Done false
11471 3967 Acked Read 6 Sent Done Done false
11472 3968 Acked Flush 0 Sent Done Done false
11473 3969 Acked Read 5 Sent Done Done false
11474 3970 Acked Read 5 Sent Done Done false
11475 3971 Acked Write 7 Sent Done Done false
11476 3972 Acked Read 1 Sent Done Done false
11477 3973 Acked Read 5 Sent Done Done false
11478 3974 Acked Read 6 Sent Done Done false
11479 3975 Acked Flush 0 Sent Done Done false
11480 3976 Acked Read 5 Sent Done Done false
11481 3977 Acked Flush 0 Sent Done Done false
11482 3978 Acked Read 10 Sent Done Done false
11483 3979 Acked Flush 0 Sent Done Done false
11484 3980 Acked Read 6 Sent Done Done false
11485 3981 Acked Flush 0 Sent Done Done false
11486 3982 Acked Write 2 Sent Done Done false
11487 3983 Acked Read 7 Sent Done Done false
11488 3984 Acked Write 3 Sent Done Done false
11489 3985 Acked Read 5 Sent Done Done false
11490 3986 Acked Flush 0 Sent Done Done false
11491 3987 Acked Read 4 Sent Done Done false
11492 3988 Acked Flush 0 Sent Done Done false
11493 3989 Acked Read 5 Sent Done Done false
11494 3990 Acked Write 8 Sent Done Done false
11495 3991 Acked Read 5 Sent Done Done false
11496 3992 Acked Write 1 Sent Done Done false
11497 3993 Acked Read 7 Sent Done Done false
11498 3994 Acked Write 9 Sent Done Done false
11499 3995 Acked Write 8 Sent Done Done false
11500 3996 Acked Read 7 Sent Done Done false
11501 3997 Acked Read 9 Sent Done Done false
11502 3998 Acked Flush 0 Sent Done Done false
11503 3999 Acked Write 4 Sent Done Done false
11504 4000 Acked Write 2 Sent Done Done false
11505 4001 Acked Read 7 Sent Done Done false
11506 4002 Acked Read 2 Sent Done Done false
11507 4003 Acked Read 2 Sent Done Done false
11508 4004 Acked Read 7 Sent Done Done false
11509 4005 Acked Flush 0 Sent Done Done false
11510 4006 Acked Read 2 Sent Done Done false
11511 4007 Acked Read 7 Sent Done Done false
11512 4008 Acked Read 4 Sent Done Done false
11513 4009 Acked Read 9 Sent Done Done false
11514 4010 Acked Read 1 Sent Done Done false
11515 4011 Acked Write 10 Sent Done Done false
11516 4012 Acked Write 1 Sent Done Done false
11517 4013 Acked Read 1 Sent Done Done false
11518 4014 Acked Read 7 Sent Done Done false
11519 4015 Acked Read 6 Sent Done Done false
11520 4016 Acked Write 10 Sent Done Done false
11521 4017 Acked Read 2 Sent Done Done false
11522 STATES DS:0 DS:1 DS:2 TOTAL
11523 Sent 301 0 0 301
11524 Done 0 301 301 602
11525 Skipped 0 0 0 0
11526 Error 0 0 0 0
11527 Last Flush: 3716 4005 4005
11528 Downstairs last five completed: 3716 3715 3714 3713 3712
11529 Upstairs last five completed: 4017 4016 4015 4014 4013
11530 CLIENT: Up:0 ds:301 act:2
11531 [8810] got start action
11532 [8810] I am going to restart
11533 Starting downstairs at port 8810
11534 Make output file at "/tmp/dsc/downstairs-8810.txt"
11535 [8810][0] reports Starting
11536 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1215)
11537 [8810][0] reports Running
11538 ----------------------------------------------------------------
11539 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11540 States: Offline Active Active
11541 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11542 3717 Acked Write 8 Sent Done Done false
11543 3718 Acked Write 10 Sent Done Done false
11544 3719 Acked Flush 0 Sent Done Done false
11545 3720 Acked Read 7 Sent Done Done false
11546 3721 Acked Flush 0 Sent Done Done false
11547 3722 Acked Read 3 Sent Done Done false
11548 3723 Acked Read 1 Sent Done Done false
11549 3724 Acked Write 1 Sent Done Done false
11550 3725 Acked Write 9 Sent Done Done false
11551 3726 Acked Read 3 Sent Done Done false
11552 3727 Acked Read 4 Sent Done Done false
11553 3728 Acked Read 3 Sent Done Done false
11554 3729 Acked Read 2 Sent Done Done false
11555 3730 Acked Write 1 Sent Done Done false
11556 3731 Acked Read 9 Sent Done Done false
11557 3732 Acked Write 8 Sent Done Done false
11558 3733 Acked Write 1 Sent Done Done false
11559 3734 Acked Flush 0 Sent Done Done false
11560 3735 Acked Write 3 Sent Done Done false
11561 3736 Acked Read 3 Sent Done Done false
11562 3737 Acked Write 9 Sent Done Done false
11563 3738 Acked Flush 0 Sent Done Done false
11564 3739 Acked Write 9 Sent Done Done false
11565 3740 Acked Flush 0 Sent Done Done false
11566 3741 Acked Flush 0 Sent Done Done false
11567 3742 Acked Read 8 Sent Done Done false
11568 3743 Acked Write 9 Sent Done Done false
11569 3744 Acked Write 5 Sent Done Done false
11570 3745 Acked Write 6 Sent Done Done false
11571 3746 Acked Read 8 Sent Done Done false
11572 3747 Acked Read 4 Sent Done Done false
11573 3748 Acked Read 9 Sent Done Done false
11574 3749 Acked Write 1 Sent Done Done false
11575 3750 Acked Write 4 Sent Done Done false
11576 3751 Acked Write 7 Sent Done Done false
11577 3752 Acked Write 2 Sent Done Done false
11578 3753 Acked Read 10 Sent Done Done false
11579 3754 Acked Read 8 Sent Done Done false
11580 3755 Acked Write 10 Sent Done Done false
11581 3756 Acked Write 5 Sent Done Done false
11582 3757 Acked Write 10 Sent Done Done false
11583 3758 Acked Read 5 Sent Done Done false
11584 3759 Acked Read 4 Sent Done Done false
11585 3760 Acked Write 5 Sent Done Done false
11586 3761 Acked Read 1 Sent Done Done false
11587 3762 Acked Flush 0 Sent Done Done false
11588 3763 Acked Write 7 Sent Done Done false
11589 3764 Acked Read 8 Sent Done Done false
11590 3765 Acked Write 8 Sent Done Done false
11591 3766 Acked Write 4 Sent Done Done false
11592 3767 Acked Write 10 Sent Done Done false
11593 3768 Acked Write 10 Sent Done Done false
11594 3769 Acked Write 7 Sent Done Done false
11595 3770 Acked Flush 0 Sent Done Done false
11596 3771 Acked Write 2 Sent Done Done false
11597 3772 Acked Read 4 Sent Done Done false
11598 3773 Acked Read 7 Sent Done Done false
11599 3774 Acked Write 10 Sent Done Done false
11600 3775 Acked Flush 0 Sent Done Done false
11601 3776 Acked Read 8 Sent Done Done false
11602 3777 Acked Write 4 Sent Done Done false
11603 3778 Acked Read 2 Sent Done Done false
11604 3779 Acked Read 6 Sent Done Done false
11605 3780 Acked Write 7 Sent Done Done false
11606 3781 Acked Write 1 Sent Done Done false
11607 3782 Acked Write 2 Sent Done Done false
11608 3783 Acked Write 9 Sent Done Done false
11609 3784 Acked Write 1 Sent Done Done false
11610 3785 Acked Write 3 Sent Done Done false
11611 3786 Acked Read 6 Sent Done Done false
11612 3787 Acked Write 8 Sent Done Done false
11613 3788 Acked Write 10 Sent Done Done false
11614 3789 Acked Read 10 Sent Done Done false
11615 3790 Acked Read 1 Sent Done Done false
11616 3791 Acked Read 1 Sent Done Done false
11617 3792 Acked Read 10 Sent Done Done false
11618 3793 Acked Read 10 Sent Done Done false
11619 3794 Acked Write 6 Sent Done Done false
11620 3795 Acked Write 9 Sent Done Done false
11621 3796 Acked Read 1 Sent Done Done false
11622 3797 Acked Read 4 Sent Done Done false
11623 3798 Acked Write 10 Sent Done Done false
11624 3799 Acked Write 10 Sent Done Done false
11625 3800 Acked Read 3 Sent Done Done false
11626 3801 Acked Write 5 Sent Done Done false
11627 3802 Acked Read 2 Sent Done Done false
11628 3803 Acked Write 1 Sent Done Done false
11629 3804 Acked Read 4 Sent Done Done false
11630 3805 Acked Write 4 Sent Done Done false
11631 3806 Acked Read 9 Sent Done Done false
11632 3807 Acked Read 10 Sent Done Done false
11633 3808 Acked Write 9 Sent Done Done false
11634 3809 Acked Write 4 Sent Done Done false
11635 3810 Acked Read 2 Sent Done Done false
11636 3811 Acked Read 6 Sent Done Done false
11637 3812 Acked Read 9 Sent Done Done false
11638 3813 Acked Flush 0 Sent Done Done false
11639 3814 Acked Read 7 Sent Done Done false
11640 3815 Acked Write 7 Sent Done Done false
11641 3816 Acked Flush 0 Sent Done Done false
11642 3817 Acked Read 5 Sent Done Done false
11643 3818 Acked Write 3 Sent Done Done false
11644 3819 Acked Read 6 Sent Done Done false
11645 3820 Acked Write 9 Sent Done Done false
11646 3821 Acked Flush 0 Sent Done Done false
11647 3822 Acked Read 2 Sent Done Done false
11648 3823 Acked Read 9 Sent Done Done false
11649 3824 Acked Read 6 Sent Done Done false
11650 3825 Acked Read 9 Sent Done Done false
11651 3826 Acked Read 5 Sent Done Done false
11652 3827 Acked Write 7 Sent Done Done false
11653 3828 Acked Read 6 Sent Done Done false
11654 3829 Acked Write 2 Sent Done Done false
11655 3830 Acked Read 1 Sent Done Done false
11656 3831 Acked Write 1 Sent Done Done false
11657 3832 Acked Write 5 Sent Done Done false
11658 3833 Acked Write 4 Sent Done Done false
11659 3834 Acked Read 5 Sent Done Done false
11660 3835 Acked Write 10 Sent Done Done false
11661 3836 Acked Read 2 Sent Done Done false
11662 3837 Acked Write 4 Sent Done Done false
11663 3838 Acked Write 4 Sent Done Done false
11664 3839 Acked Write 2 Sent Done Done false
11665 3840 Acked Write 1 Sent Done Done false
11666 3841 Acked Write 10 Sent Done Done false
11667 3842 Acked Flush 0 Sent Done Done false
11668 3843 Acked Write 1 Sent Done Done false
11669 3844 Acked Write 8 Sent Done Done false
11670 3845 Acked Flush 0 Sent Done Done false
11671 3846 Acked Write 1 Sent Done Done false
11672 3847 Acked Read 6 Sent Done Done false
11673 3848 Acked Write 1 Sent Done Done false
11674 3849 Acked Read 6 Sent Done Done false
11675 3850 Acked Write 8 Sent Done Done false
11676 3851 Acked Write 5 Sent Done Done false
11677 3852 Acked Flush 0 Sent Done Done false
11678 3853 Acked Read 4 Sent Done Done false
11679 3854 Acked Write 6 Sent Done Done false
11680 3855 Acked Flush 0 Sent Done Done false
11681 3856 Acked Write 6 Sent Done Done false
11682 3857 Acked Read 3 Sent Done Done false
11683 3858 Acked Write 5 Sent Done Done false
11684 3859 Acked Read 10 Sent Done Done false
11685 3860 Acked Flush 0 Sent Done Done false
11686 3861 Acked Write 9 Sent Done Done false
11687 3862 Acked Write 2 Sent Done Done false
11688 3863 Acked Read 2 Sent Done Done false
11689 3864 Acked Write 6 Sent Done Done false
11690 3865 Acked Read 6 Sent Done Done false
11691 3866 Acked Flush 0 Sent Done Done false
11692 3867 Acked Read 7 Sent Done Done false
11693 3868 Acked Read 4 Sent Done Done false
11694 3869 Acked Write 6 Sent Done Done false
11695 3870 Acked Write 1 Sent Done Done false
11696 3871 Acked Write 4 Sent Done Done false
11697 3872 Acked Flush 0 Sent Done Done false
11698 3873 Acked Write 3 Sent Done Done false
11699 3874 Acked Write 3 Sent Done Done false
11700 3875 Acked Write 9 Sent Done Done false
11701 3876 Acked Read 4 Sent Done Done false
11702 3877 Acked Write 8 Sent Done Done false
11703 3878 Acked Read 5 Sent Done Done false
11704 3879 Acked Write 7 Sent Done Done false
11705 3880 Acked Read 4 Sent Done Done false
11706 3881 Acked Read 1 Sent Done Done false
11707 3882 Acked Read 3 Sent Done Done false
11708 3883 Acked Flush 0 Sent Done Done false
11709 3884 Acked Read 5 Sent Done Done false
11710 3885 Acked Read 4 Sent Done Done false
11711 3886 Acked Read 10 Sent Done Done false
11712 3887 Acked Read 6 Sent Done Done false
11713 3888 Acked Read 8 Sent Done Done false
11714 3889 Acked Read 6 Sent Done Done false
11715 3890 Acked Read 6 Sent Done Done false
11716 3891 Acked Read 9 Sent Done Done false
11717 3892 Acked Read 9 Sent Done Done false
11718 3893 Acked Flush 0 Sent Done Done false
11719 3894 Acked Read 2 Sent Done Done false
11720 3895 Acked Write 4 Sent Done Done false
11721 3896 Acked Write 2 Sent Done Done false
11722 3897 Acked Read 4 Sent Done Done false
11723 3898 Acked Read 8 Sent Done Done false
11724 3899 Acked Read 5 Sent Done Done false
11725 3900 Acked Read 4 Sent Done Done false
11726 3901 Acked Read 3 Sent Done Done false
11727 3902 Acked Read 7 Sent Done Done false
11728 3903 Acked Write 6 Sent Done Done false
11729 3904 Acked Read 10 Sent Done Done false
11730 3905 Acked Write 3 Sent Done Done false
11731 3906 Acked Read 5 Sent Done Done false
11732 3907 Acked Write 3 Sent Done Done false
11733 3908 Acked Write 6 Sent Done Done false
11734 3909 Acked Read 8 Sent Done Done false
11735 3910 Acked Read 3 Sent Done Done false
11736 3911 Acked Write 3 Sent Done Done false
11737 3912 Acked Write 6 Sent Done Done false
11738 3913 Acked Write 4 Sent Done Done false
11739 3914 Acked Write 3 Sent Done Done false
11740 3915 Acked Flush 0 Sent Done Done false
11741 3916 Acked Read 8 Sent Done Done false
11742 3917 Acked Read 2 Sent Done Done false
11743 3918 Acked Flush 0 Sent Done Done false
11744 3919 Acked Read 1 Sent Done Done false
11745 3920 Acked Read 10 Sent Done Done false
11746 3921 Acked Write 9 Sent Done Done false
11747 3922 Acked Write 9 Sent Done Done false
11748 3923 Acked Write 6 Sent Done Done false
11749 3924 Acked Read 6 Sent Done Done false
11750 3925 Acked Read 1 Sent Done Done false
11751 3926 Acked Write 9 Sent Done Done false
11752 3927 Acked Read 4 Sent Done Done false
11753 3928 Acked Read 4 Sent Done Done false
11754 3929 Acked Read 3 Sent Done Done false
11755 3930 Acked Read 6 Sent Done Done false
11756 3931 Acked Flush 0 Sent Done Done false
11757 3932 Acked Write 10 Sent Done Done false
11758 3933 Acked Write 5 Sent Done Done false
11759 3934 Acked Read 5 Sent Done Done false
11760 3935 Acked Read 1 Sent Done Done false
11761 3936 Acked Read 10 Sent Done Done false
11762 3937 Acked Read 2 Sent Done Done false
11763 3938 Acked Write 7 Sent Done Done false
11764 3939 Acked Read 6 Sent Done Done false
11765 3940 Acked Read 4 Sent Done Done false
11766 3941 Acked Read 6 Sent Done Done false
11767 3942 Acked Flush 0 Sent Done Done false
11768 3943 Acked Read 1 Sent Done Done false
11769 3944 Acked Write 10 Sent Done Done false
11770 3945 Acked Write 5 Sent Done Done false
11771 3946 Acked Write 9 Sent Done Done false
11772 3947 Acked Write 3 Sent Done Done false
11773 3948 Acked Write 4 Sent Done Done false
11774 3949 Acked Flush 0 Sent Done Done false
11775 3950 Acked Read 2 Sent Done Done false
11776 3951 Acked Read 6 Sent Done Done false
11777 3952 Acked Write 1 Sent Done Done false
11778 3953 Acked Write 6 Sent Done Done false
11779 3954 Acked Write 1 Sent Done Done false
11780 3955 Acked Read 8 Sent Done Done false
11781 3956 Acked Read 7 Sent Done Done false
11782 3957 Acked Flush 0 Sent Done Done false
11783 3958 Acked Write 8 Sent Done Done false
11784 3959 Acked Read 1 Sent Done Done false
11785 3960 Acked Flush 0 Sent Done Done false
11786 3961 Acked Read 3 Sent Done Done false
11787 3962 Acked Read 10 Sent Done Done false
11788 3963 Acked Write 7 Sent Done Done false
11789 3964 Acked Read 5 Sent Done Done false
11790 3965 Acked Read 8 Sent Done Done false
11791 3966 Acked Read 6 Sent Done Done false
11792 3967 Acked Read 6 Sent Done Done false
11793 3968 Acked Flush 0 Sent Done Done false
11794 3969 Acked Read 5 Sent Done Done false
11795 3970 Acked Read 5 Sent Done Done false
11796 3971 Acked Write 7 Sent Done Done false
11797 3972 Acked Read 1 Sent Done Done false
11798 3973 Acked Read 5 Sent Done Done false
11799 3974 Acked Read 6 Sent Done Done false
11800 3975 Acked Flush 0 Sent Done Done false
11801 3976 Acked Read 5 Sent Done Done false
11802 3977 Acked Flush 0 Sent Done Done false
11803 3978 Acked Read 10 Sent Done Done false
11804 3979 Acked Flush 0 Sent Done Done false
11805 3980 Acked Read 6 Sent Done Done false
11806 3981 Acked Flush 0 Sent Done Done false
11807 3982 Acked Write 2 Sent Done Done false
11808 3983 Acked Read 7 Sent Done Done false
11809 3984 Acked Write 3 Sent Done Done false
11810 3985 Acked Read 5 Sent Done Done false
11811 3986 Acked Flush 0 Sent Done Done false
11812 3987 Acked Read 4 Sent Done Done false
11813 3988 Acked Flush 0 Sent Done Done false
11814 3989 Acked Read 5 Sent Done Done false
11815 3990 Acked Write 8 Sent Done Done false
11816 3991 Acked Read 5 Sent Done Done false
11817 3992 Acked Write 1 Sent Done Done false
11818 3993 Acked Read 7 Sent Done Done false
11819 3994 Acked Write 9 Sent Done Done false
11820 3995 Acked Write 8 Sent Done Done false
11821 3996 Acked Read 7 Sent Done Done false
11822 3997 Acked Read 9 Sent Done Done false
11823 3998 Acked Flush 0 Sent Done Done false
11824 3999 Acked Write 4 Sent Done Done false
11825 4000 Acked Write 2 Sent Done Done false
11826 4001 Acked Read 7 Sent Done Done false
11827 4002 Acked Read 2 Sent Done Done false
11828 4003 Acked Read 2 Sent Done Done false
11829 4004 Acked Read 7 Sent Done Done false
11830 4005 Acked Flush 0 Sent Done Done false
11831 4006 Acked Read 2 Sent Done Done false
11832 4007 Acked Read 7 Sent Done Done false
11833 4008 Acked Read 4 Sent Done Done false
11834 4009 Acked Read 9 Sent Done Done false
11835 4010 Acked Read 1 Sent Done Done false
11836 4011 Acked Write 10 Sent Done Done false
11837 4012 Acked Write 1 Sent Done Done false
11838 4013 Acked Read 1 Sent Done Done false
11839 4014 Acked Read 7 Sent Done Done false
11840 4015 Acked Read 6 Sent Done Done false
11841 4016 Acked Write 10 Sent Done Done false
11842 4017 Acked Read 2 Sent Done Done false
11843 4018 Acked Flush 0 Sent Done Done false
11844 STATES DS:0 DS:1 DS:2 TOTAL
11845 Sent 302 0 0 302
11846 Done 0 302 302 604
11847 Skipped 0 0 0 0
11848 Error 0 0 0 0
11849 Last Flush: 3716 4018 4018
11850 Downstairs last five completed: 3716 3715 3714 3713 3712
11851 Upstairs last five completed: 4018 4017 4016 4015 4014
11852 CLIENT: Up:0 ds:302 act:2
118532025-01-03T08:33:21.816ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118542025-01-03T08:33:21.816ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58501) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118552025-01-03T08:33:21.817ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118562025-01-03T08:33:21.818ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118572025-01-03T08:33:21.818ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118582025-01-03T08:33:21.818ZINFOcrucible: send last flush ID to this DS: 3716 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118592025-01-03T08:33:21.818ZINFOcrucible: Replied this last flush ID: 3716 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118602025-01-03T08:33:21.818ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118612025-01-03T08:33:21.818ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118622025-01-03T08:33:21.818ZINFOcrucible: [0] client re-new 302 jobs since flush 3716 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118632025-01-03T08:33:21.819ZINFOcrucible: [0] Replayed 302 jobs since flush: 3716 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11864 ----------------------------------------------------------------
11865 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
11866 Downstairs last five completed: 4018 4017 4016 4015 4014
11867 Upstairs last five completed: 4018 4017 4016 4015 4014
11868 CLIENT: Up:0 ds:0 act:3
11869 Replay: All jobs finished
11870 Main task has work to do, go find it
11871 got dsc Stop(0)
11872 stop 0
11873 Jan 03 08:33:24.058 INFO request completed, latency_us: 373, response_code: 204, [8810] Got stop action so:false kr:false
11874 uri: /stop/cid/0, method: POST, req_id: 51c78cc1-996a-48b8-8b1b-a8f0039c0c16, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
118752025-01-03T08:33:24.059ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118762025-01-03T08:33:24.059ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118772025-01-03T08:33:24.059ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118782025-01-03T08:33:24.059ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
118792025-01-03T08:33:24.059ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11880 Jan 03 08:33:24.059 INFO request completed, latency_us: 180, response_code: 200, uri: /state/cid/0, method: GET, req_id: 88f02f85-a6a2-4502-a401-490920f501c2, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
118812025-01-03T08:33:24.059ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
11882 [8810] Exited with: ExitStatus(unix_wait_status(9))
11883 [8810][0] reports Exit
11884 Jan 03 08:33:28.061 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/0, method: GET, req_id: 560db8c7-4aba-4cc4-b109-f7c2f59416ba, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
11885 001/300 Write block 488 len 8192 data: 3 4
11886 002/300 Read block 3305 len 4096
11887 003/300 Write block 2182 len 20480 data: 2 2 3 3 3
11888 004/300 Read block 2090 len 8192
11889 005/300 Write block 790 len 32768 data: 6 4 4 4 4 4 4 5
11890 006/300 Read block 3079 len 40960
11891 007/300 Read block 3138 len 28672
11892 008/300 Write block 1046 len 4096 data: 3
11893 009/300 Write block 2191 len 8192 data: 2 2
11894 010/300 Write block 2771 len 36864 data: 3 2 2 2 2 2 2 2 3
11895 011/300 Read block 4970 len 8192
11896 012/300 Write block 1256 len 32768 data: 2 2 2 2 2 2 2 3
11897 013/300 Read block 1395 len 40960
11898 014/300 Read block 4428 len 16384
11899 015/300 Read block 454 len 4096
11900 016/300 Read block 2294 len 8192
11901 017/300 Write block 981 len 16384 data: 2 2 2 2
11902 018/300 Read block 1929 len 40960
11903 019/300 Read block 523 len 16384
11904 020/300 Read block 4626 len 24576
11905 021/300 Write block 2720 len 16384 data: 3 4 4 4
11906 022/300 Read block 802 len 12288
11907 023/300 Write block 2707 len 20480 data: 3 7 6 6 6
11908 024/300 Read block 2800 len 40960
11909 025/300 Read block 4319 len 4096
11910 026/300 Read block 2221 len 36864
11911 027/300 Write block 1282 len 24576 data: 3 3 3 3 3 3
11912 028/300 Read block 4535 len 24576
11913 029/300 Write block 942 len 40960 data: 2 2 2 2 3 4 5 5 5 5
11914 030/300 Read block 1066 len 20480
11915 031/300 Read block 2382 len 16384
11916 032/300 Read block 3809 len 12288
11917 033/300 Flush
11918 034/300 Write block 61 len 24576 data: 4 3 4 5 5 6
11919 035/300 Flush
11920 036/300 Read block 1987 len 24576
11921 037/300 Read block 736 len 8192
11922 038/300 Read block 1145 len 40960
11923 039/300 Flush
11924 040/300 Write block 3196 len 28672 data: 4 4 4 4 5 4 4
11925 041/300 Write block 3901 len 36864 data: 3 3 3 3 3 3 3 3 4
11926 042/300 Write block 3723 len 40960 data: 4 4 3 3 2 2 2 3 3 3
11927 043/300 Write block 3604 len 16384 data: 3 6 3 3
11928 044/300 Write block 2484 len 12288 data: 5 5 5
11929 045/300 Read block 4551 len 28672
11930 046/300 Read block 2912 len 20480
11931 047/300 Flush
11932 048/300 Write block 1566 len 32768 data: 3 2 2 2 2 2 3 3
11933 049/300 Read block 4067 len 20480
11934 050/300 Write block 912 len 32768 data: 4 4 4 4 4 4 4 4
11935 051/300 Read block 4357 len 12288
11936 052/300 Read block 3352 len 40960
11937 053/300 Read block 4112 len 20480
11938 054/300 Write block 2731 len 20480 data: 3 4 4 4 4
11939 055/300 Write block 4205 len 20480 data: 7 6 5 4 3
11940 056/300 Write block 1798 len 4096 data: 3
11941 057/300 Write block 2336 len 20480 data: 2 2 2 2 2
11942 058/300 Write block 2833 len 40960 data: 4 5 4 4 4 3 4 5 5 3
11943 059/300 Write block 4902 len 24576 data: 2 3 3 4 4 5
11944 060/300 Read block 814 len 12288
11945 061/300 Write block 3877 len 32768 data: 3 2 2 2 3 4 4 3
11946 062/300 Write block 657 len 24576 data: 5 5 4 3 2 4
11947 063/300 Read block 3597 len 8192
11948 064/300 Read block 4329 len 16384
11949 065/300 Write block 3388 len 32768 data: 3 4 3 3 3 2 2 4
11950 066/300 Read block 464 len 32768
11951 067/300 Read block 4574 len 20480
11952 068/300 Write block 1589 len 16384 data: 4 4 4 3
11953 069/300 Write block 689 len 24576 data: 2 3 3 4 5 5
11954 070/300 Read block 1306 len 12288
11955 071/300 Write block 3620 len 40960 data: 3 3 3 3 4 4 4 4 4 5
11956 072/300 Read block 548 len 32768
11957 073/300 Read block 4964 len 36864
11958 074/300 Flush
11959 075/300 Write block 3402 len 28672 data: 4 4 4 4 4 5 3
11960 076/300 Read block 666 len 24576
11961 077/300 Read block 199 len 16384
11962 078/300 Write block 3400 len 16384 data: 4 4 5 5
11963 079/300 Write block 60 len 12288 data: 5 5 4
11964 080/300 Read block 3743 len 12288
11965 081/300 Flush
11966 082/300 Flush
11967 083/300 Read block 295 len 28672
11968 084/300 Write block 2111 len 24576 data: 3 3 3 3 3 3
11969 085/300 Read block 4069 len 12288
11970 086/300 Write block 4172 len 36864 data: 4 4 3 3 2 2 2 2 2
11971 087/300 Write block 272 len 4096 data: 2
11972 088/300 Read block 3600 len 36864
11973 089/300 Read block 1970 len 16384
11974 090/300 Read block 41 len 28672
11975 091/300 Read block 552 len 12288
11976 092/300 Write block 3120 len 8192 data: 2 2
11977 093/300 Read block 965 len 8192
11978 094/300 Read block 2779 len 16384
11979 095/300 Read block 1108 len 36864
11980 096/300 Read block 401 len 24576
11981 097/300 Write block 2723 len 20480 data: 5 4 4 4 4
11982 098/300 Read block 4904 len 12288
11983 099/300 Write block 4015 len 24576 data: 2 2 2 2 2 2
11984 100/300 Read block 1179 len 12288
11985 101/300 Write block 2651 len 40960 data: 2 2 2 2 2 2 2 2 2 4
11986 102/300 Read block 4252 len 28672
11987 103/300 Write block 3729 len 20480 data: 3 4 4 4 3
11988 104/300 Write block 3891 len 28672 data: 3 3 2 2 2 2 2
11989 105/300 Read block 543 len 28672
11990 106/300 Read block 4447 len 8192
11991 107/300 Read block 4622 len 12288
11992 108/300 Read block 3369 len 20480
11993 109/300 Write block 3541 len 12288 data: 4 5 5
11994 110/300 Read block 1913 len 36864
11995 111/300 Write block 466 len 12288 data: 4 4 4
11996 112/300 Write block 4941 len 36864 data: 5 6 6 4 3 3 3 3 2
11997 113/300 Write block 4176 len 12288 data: 3 3 3
11998 114/300 Read block 2642 len 12288
11999 115/300 Read block 4914 len 24576
12000 116/300 Write block 4774 len 28672 data: 3 3 3 4 4 4 4
12001 117/300 Read block 4922 len 12288
12002 118/300 Read block 3085 len 24576
12003 119/300 Write block 2454 len 24576 data: 6 6 6 5 5 4
12004 120/300 Read block 2345 len 28672
12005 121/300 Flush
12006 122/300 Read block 4245 len 28672
12007 123/300 Write block 2695 len 16384 data: 5 5 5 5
12008 124/300 Write block 936 len 12288 data: 5 5 5
12009 125/300 Read block 2915 len 16384
12010 126/300 Read block 2410 len 36864
12011 127/300 Read block 2419 len 32768
12012 128/300 Write block 3103 len 12288 data: 3 3 3
12013 129/300 Write block 2492 len 20480 data: 4 3 3 3 3
12014 130/300 Write block 130 len 36864 data: 2 2 2 2 2 2 2 4 4
12015 131/300 Write block 928 len 32768 data: 4 3 2 2 2 2 2 4
12016 132/300 Write block 640 len 40960 data: 7 6 5 4 3 2 2 3 3 4
12017 133/300 Flush
12018 134/300 Flush
12019 135/300 Read block 4607 len 4096
12020 136/300 Write block 3075 len 20480 data: 3 2 3 3 3
12021 137/300 Write block 374 len 24576 data: 2 2 2 2 4 3
12022 138/300 Read block 4384 len 40960
12023 139/300 Write block 3786 len 28672 data: 4 3 4 4 4 4 5
12024 140/300 Read block 1753 len 20480
12025 141/300 Write block 2650 len 20480 data: 2 3 3 3 3
12026 142/300 Read block 4782 len 4096
12027 143/300 Write block 4879 len 16384 data: 5 5 6 5
12028 144/300 Write block 3366 len 32768 data: 2 2 2 2 2 2 2 2
12029 145/300 Write block 2804 len 20480 data: 2 3 4 5 5
12030 146/300 Read block 589 len 16384
12031 147/300 Write block 1703 len 20480 data: 2 2 2 2 2
12032 148/300 Write block 2661 len 28672 data: 3 3 3 3 2 2 2
12033 149/300 Read block 2182 len 32768
12034 150/300 Write block 519 len 24576 data: 5 4 5 4 4 4
12035 151/300 Flush
12036 152/300 Read block 550 len 12288
12037 153/300 Write block 3322 len 20480 data: 2 2 2 2 2
12038 154/300 Write block 429 len 20480 data: 3 3 3 2 3
12039 155/300 Read block 1520 len 16384
12040 156/300 Flush
12041 157/300 Read block 2908 len 40960
12042 158/300 Read block 3239 len 40960
12043 159/300 Read block 3542 len 8192
12044 160/300 Read block 4202 len 40960
12045 161/300 Read block 1710 len 40960
12046 162/300 Write block 4448 len 20480 data: 5 3 3 2 2
12047 163/300 Read block 3649 len 24576
12048 164/300 Write block 2900 len 40960 data: 3 3 3 3 3 2 2 2 3 4
12049 165/300 Read block 4205 len 32768
12050 166/300 Read block 3135 len 16384
12051 167/300 Write block 1598 len 40960 data: 4 4 4 4 3 4 3 3 3 3
12052 168/300 Flush
12053 169/300 Read block 4244 len 40960
12054 170/300 Write block 1359 len 40960 data: 4 4 3 3 4 3 4 3 4 4
12055 171/300 Flush
12056 172/300 Write block 3738 len 4096 data: 3
12057 173/300 Read block 4016 len 28672
12058 174/300 Write block 3383 len 24576 data: 3 2 4 4 4 4
12059 175/300 Write block 1307 len 12288 data: 2 2 2
12060 176/300 Read block 905 len 24576
12061 177/300 Write block 4581 len 8192 data: 5 4
12062 178/300 Write block 4266 len 20480 data: 3 3 3 3 3
12063 179/300 Write block 1827 len 20480 data: 3 3 3 3 2
12064 180/300 Write block 1947 len 16384 data: 5 4 4 4
12065 181/300 Read block 2561 len 40960
12066 182/300 Read block 4821 len 8192
12067 183/300 Read block 490 len 40960
12068 184/300 Flush
12069 185/300 Write block 3692 len 40960 data: 3 2 2 2 3 3 3 3 3 4
12070 186/300 Write block 4971 len 36864 data: 3 4 3 3 2 2 2 3 2
12071 187/300 Read block 191 len 16384
12072 188/300 Read block 4983 len 36864
12073 189/300 Read block 113 len 8192
12074 190/300 Read block 834 len 24576
12075 191/300 Flush
12076 192/300 Read block 617 len 20480
12077 193/300 Write block 2779 len 12288 data: 4 3 2
12078 194/300 Write block 4732 len 8192 data: 2 2
12079 195/300 Write block 3567 len 12288 data: 3 3 5
12080 196/300 Flush
12081 197/300 Read block 4798 len 40960
12082 198/300 Read block 3535 len 40960
12083 199/300 Read block 4108 len 20480
12084 200/300 Read block 1379 len 8192
12085 201/300 Read block 4911 len 12288
12086 202/300 Read block 2774 len 16384
12087 203/300 Read block 759 len 32768
12088 204/300 Write block 4105 len 12288 data: 2 2 2
12089 205/300 Write block 4958 len 20480 data: 3 3 3 3 3
12090 206/300 Write block 1978 len 28672 data: 2 2 2 2 3 2 2
12091 207/300 Write block 1529 len 28672 data: 3 3 2 2 2 2 4
12092 208/300 Read block 640 len 12288
12093 209/300 Read block 3280 len 16384
12094 210/300 Read block 290 len 40960
12095 211/300 Flush
12096 212/300 Write block 3938 len 12288 data: 3 3 3
12097 213/300 Read block 1550 len 32768
12098 214/300 Read block 1882 len 40960
12099 215/300 Read block 4209 len 8192
12100 216/300 Flush
12101 217/300 Read block 4514 len 20480
12102 218/300 Read block 4486 len 36864
12103 219/300 Write block 2249 len 4096 data: 4
12104 220/300 Write block 1830 len 4096 data: 4
12105 221/300 Read block 1249 len 24576
12106 222/300 Read block 521 len 16384
12107 223/300 Read block 2540 len 36864
12108 224/300 Write block 3219 len 28672 data: 3 3 3 3 3 3 3
12109 225/300 Write block 1073 len 20480 data: 2 2 2 2 2
12110 226/300 Write block 692 len 24576 data: 5 6 6 5 5 5
12111 227/300 Write block 3302 len 24576 data: 2 2 2 2 3 4
12112 228/300 Flush
12113 229/300 Write block 1147 len 12288 data: 5 4 4
12114 230/300 Read block 1840 len 20480
12115 231/300 Write block 191 len 24576 data: 2 2 3 4 4 4
12116 232/300 Flush
12117 233/300 Read block 4863 len 32768
12118 234/300 Read block 3679 len 28672
12119 235/300 Write block 1919 len 36864 data: 6 5 5 6 7 7 6 6 4
12120 236/300 Write block 4330 len 32768 data: 2 3 3 4 5 3 3 4
12121 237/300 Read block 2662 len 24576
12122 238/300 Read block 1585 len 12288
12123 239/300 Read block 748 len 40960
12124 240/300 Read block 3692 len 12288
12125 241/300 Read block 915 len 12288
12126 242/300 Write block 1581 len 32768 data: 3 2 2 3 4 4 4 4
12127 243/300 Write block 2729 len 16384 data: 3 3 4 5
12128 244/300 Read block 3735 len 8192
12129 245/300 Write block 1532 len 24576 data: 3 3 3 5 4 5
12130 246/300 Read block 2407 len 36864
12131 247/300 Write block 1713 len 4096 data: 2
12132 248/300 Read block 586 len 12288
12133 249/300 Read block 1445 len 40960
12134 250/300 Read block 3683 len 24576
12135 251/300 Write block 4197 len 8192 data: 4 4
12136 252/300 Write block 113 len 12288 data: 2 2 2
12137 253/300 Read block 1226 len 28672
12138 254/300 Read block 3062 len 16384
12139 255/300 Write block 819 len 8192 data: 6 7
12140 256/300 Read block 3513 len 40960
12141 257/300 Read block 292 len 24576
12142 258/300 Read block 251 len 4096
12143 259/300 Read block 63 len 36864
12144 260/300 Write block 1884 len 12288 data: 4 4 4
12145 261/300 Write block 2984 len 4096 data: 2
12146 262/300 Write block 1325 len 40960 data: 3 5 5 5 4 3 3 4 4 4
12147 263/300 Write block 888 len 28672 data: 3 2 2 2 2 2 3
12148 264/300 Read block 4003 len 20480
12149 265/300 Write block 1023 len 20480 data: 2 2 2 3 3
12150 266/300 Read block 1808 len 16384
12151 267/300 Write block 3852 len 20480 data: 5 6 5 5 3
12152 268/300 Read block 2338 len 32768
12153 269/300 Write block 3509 len 40960 data: 2 2 2 2 2 2 3 3 2 2
12154 270/300 Read block 4458 len 4096
12155 271/300 Write block 1231 len 12288 data: 3 3 3
12156 272/300 Read block 297 len 24576
12157 273/300 Read block 329 len 12288
12158 274/300 Write block 4843 len 20480 data: 3 3 3 2 2
12159 275/300 Read block 4517 len 28672
12160 276/300 Read block 3579 len 24576
12161 277/300 Read block 3638 len 4096
12162 278/300 Read block 1131 len 12288
12163 279/300 Write block 3633 len 8192 data: 6 5
12164 280/300 Write block 1999 len 16384 data: 3 3 3 2
12165 281/300 Read block 2008 len 36864
12166 282/300 Write block 4711 len 12288 data: 6 4 3
12167 283/300 Read block 3403 len 8192
12168 284/300 Read block 1663 len 4096
12169 285/300 Read block 41 len 20480
12170 286/300 Read block 982 len 40960
12171 287/300 Read block 3214 len 32768
12172 288/300 Read block 945 len 4096
12173 289/300 Read block 3896 len 20480
12174 290/300 Write block 764 len 28672 data: 5 3 4 4 3 3 3
12175 291/300 Write block 1750 len 40960 data: 2 3 4 5 5 4 3 4 2 2
12176 292/300 Write block 2214 len 20480 data: 2 2 2 2 3
12177 293/300 Write block 2716 len 28672 data: 5 4 2 3 4 5 5
12178 294/300 Write block 3768 len 20480 data: 2 2 2 2 3
12179 295/300 Read block 4769 len 12288
12180 296/300 Read block 4600 len 8192
12181 297/300 Read block 1539 len 12288
12182 298/300 Flush
12183 299/300 Write block 2582 len 28672 data: 3 3 3 3 3 2 2
12184 300/300 Write block 4528 len 16384 data: 2 2 2 2
12185 301/300 Write block 2307 len 16384 data: 3 4 3 3
12186 Main task has work to do, go find it
12187 got dsc Start(0)
12188 start 0
12189 Jan 03 08:33:28.304 INFO request completed, latency_us: 223, response_code: 204, uri: /start/cid/0, method: POST, req_id: a7dad7e4-d022-4533-8477-579f3689cde0, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
12190 Replay: started 0, returned:Ok(())
12191 ----------------------------------------------------------------
12192 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12193 States: Offline Active Active
12194 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12195 4019 Acked Write 2 Sent Done Done false
12196 4020 Acked Read 1 Sent Done Done false
12197 4021 Acked Write 5 Sent Done Done false
12198 4022 Acked Read 2 Sent Done Done false
12199 4023 Acked Write 8 Sent Done Done false
12200 4024 Acked Read 10 Sent Done Done false
12201 4025 Acked Read 7 Sent Done Done false
12202 4026 Acked Write 1 Sent Done Done false
12203 4027 Acked Write 2 Sent Done Done false
12204 4028 Acked Write 9 Sent Done Done false
12205 4029 Acked Read 2 Sent Done Done false
12206 4030 Acked Write 8 Sent Done Done false
12207 4031 Acked Read 10 Sent Done Done false
12208 4032 Acked Read 4 Sent Done Done false
12209 4033 Acked Read 1 Sent Done Done false
12210 4034 Acked Read 2 Sent Done Done false
12211 4035 Acked Write 4 Sent Done Done false
12212 4036 Acked Read 10 Sent Done Done false
12213 4037 Acked Read 4 Sent Done Done false
12214 4038 Acked Read 6 Sent Done Done false
12215 4039 Acked Write 4 Sent Done Done false
12216 4040 Acked Read 3 Sent Done Done false
12217 4041 Acked Write 5 Sent Done Done false
12218 4042 Acked Read 10 Sent Done Done false
12219 4043 Acked Read 1 Sent Done Done false
12220 4044 Acked Read 9 Sent Done Done false
12221 4045 Acked Write 6 Sent Done Done false
12222 4046 Acked Read 6 Sent Done Done false
12223 4047 Acked Write 10 Sent Done Done false
12224 4048 Acked Read 5 Sent Done Done false
12225 4049 Acked Read 4 Sent Done Done false
12226 4050 Acked Read 3 Sent Done Done false
12227 4051 Acked Flush 0 Sent Done Done false
12228 4052 Acked Write 6 Sent Done Done false
12229 4053 Acked Flush 0 Sent Done Done false
12230 4054 Acked Read 6 Sent Done Done false
12231 4055 Acked Read 2 Sent Done Done false
12232 4056 Acked Read 10 Sent Done Done false
12233 4057 Acked Flush 0 Sent Done Done false
12234 4058 Acked Write 7 Sent Done Done false
12235 4059 Acked Write 9 Sent Done Done false
12236 4060 Acked Write 10 Sent Done Done false
12237 4061 Acked Write 4 Sent Done Done false
12238 4062 Acked Write 3 Sent Done Done false
12239 4063 Acked Read 7 Sent Done Done false
12240 4064 Acked Read 5 Sent Done Done false
12241 4065 Acked Flush 0 Sent Done Done false
12242 4066 Acked Write 8 Sent Done Done false
12243 4067 Acked Read 5 Sent Done Done false
12244 4068 Acked Write 8 Sent Done Done false
12245 4069 Acked Read 3 Sent Done Done false
12246 4070 Acked Read 10 Sent Done Done false
12247 4071 Acked Read 5 Sent Done Done false
12248 4072 Acked Write 5 Sent Done Done false
12249 4073 Acked Write 5 Sent Done Done false
12250 4074 Acked Write 1 Sent Done Done false
12251 4075 Acked Write 5 Sent Done Done false
12252 4076 Acked Write 10 Sent Done Done false
12253 4077 Acked Write 6 Sent Done Done false
12254 4078 Acked Read 3 Sent Done Done false
12255 4079 Acked Write 8 Sent Done Done false
12256 4080 Acked Write 6 Sent Done Done false
12257 4081 Acked Read 2 Sent Done Done false
12258 4082 Acked Read 4 Sent Done Done false
12259 4083 Acked Write 8 Sent Done Done false
12260 4084 Acked Read 8 Sent Done Done false
12261 4085 Acked Read 5 Sent Done Done false
12262 4086 Acked Write 4 Sent Done Done false
12263 4087 Acked Write 6 Sent Done Done false
12264 4088 Acked Read 3 Sent Done Done false
12265 4089 Acked Write 10 Sent Done Done false
12266 4090 Acked Read 8 Sent Done Done false
12267 4091 Acked Read 9 Sent Done Done false
12268 4092 Acked Flush 0 Sent Done Done false
12269 4093 Acked Write 7 Sent Done Done false
12270 4094 Acked Read 6 Sent Done Done false
12271 4095 Acked Read 4 Sent Done Done false
12272 4096 Acked Write 4 Sent Done Done false
12273 4097 Acked Write 3 Sent Done Done false
12274 4098 Acked Read 3 Sent Done Done false
12275 4099 Acked Flush 0 Sent Done Done false
12276 4100 Acked Flush 0 Sent Done Done false
12277 4101 Acked Read 7 Sent Done Done false
12278 4102 Acked Write 6 Sent Done Done false
12279 4103 Acked Read 3 Sent Done Done false
12280 4104 Acked Write 9 Sent Done Done false
12281 4105 Acked Write 1 Sent Done Done false
12282 4106 Acked Read 9 Sent Done Done false
12283 4107 Acked Read 4 Sent Done Done false
12284 4108 Acked Read 7 Sent Done Done false
12285 4109 Acked Read 3 Sent Done Done false
12286 4110 Acked Write 2 Sent Done Done false
12287 4111 Acked Read 2 Sent Done Done false
12288 4112 Acked Read 4 Sent Done Done false
12289 4113 Acked Read 9 Sent Done Done false
12290 4114 Acked Read 6 Sent Done Done false
12291 4115 Acked Write 5 Sent Done Done false
12292 4116 Acked Read 3 Sent Done Done false
12293 4117 Acked Write 6 Sent Done Done false
12294 4118 Acked Read 3 Sent Done Done false
12295 4119 Acked Write 10 Sent Done Done false
12296 4120 Acked Read 7 Sent Done Done false
12297 4121 Acked Write 5 Sent Done Done false
12298 4122 Acked Write 7 Sent Done Done false
12299 4123 Acked Read 7 Sent Done Done false
12300 4124 Acked Read 2 Sent Done Done false
12301 4125 Acked Read 3 Sent Done Done false
12302 4126 Acked Read 5 Sent Done Done false
12303 4127 Acked Write 3 Sent Done Done false
12304 4128 Acked Read 9 Sent Done Done false
12305 4129 Acked Write 3 Sent Done Done false
12306 4130 Acked Write 9 Sent Done Done false
12307 4131 Acked Write 3 Sent Done Done false
12308 4132 Acked Read 3 Sent Done Done false
12309 4133 Acked Read 6 Sent Done Done false
12310 4134 Acked Write 7 Sent Done Done false
12311 4135 Acked Read 3 Sent Done Done false
12312 4136 Acked Read 6 Sent Done Done false
12313 4137 Acked Write 6 Sent Done Done false
12314 4138 Acked Read 7 Sent Done Done false
12315 4139 Acked Flush 0 Sent Done Done false
12316 4140 Acked Read 7 Sent Done Done false
12317 4141 Acked Write 4 Sent Done Done false
12318 4142 Acked Write 3 Sent Done Done false
12319 4143 Acked Read 4 Sent Done Done false
12320 4144 Acked Read 9 Sent Done Done false
12321 4145 Acked Read 8 Sent Done Done false
12322 4146 Acked Write 3 Sent Done Done false
12323 4147 Acked Write 5 Sent Done Done false
12324 4148 Acked Write 9 Sent Done Done false
12325 4149 Acked Write 8 Sent Done Done false
12326 4150 Acked Write 10 Sent Done Done false
12327 4151 Acked Flush 0 Sent Done Done false
12328 4152 Acked Flush 0 Sent Done Done false
12329 4153 Acked Read 1 Sent Done Done false
12330 4154 Acked Write 5 Sent Done Done false
12331 4155 Acked Write 6 Sent Done Done false
12332 4156 Acked Read 10 Sent Done Done false
12333 4157 Acked Write 7 Sent Done Done false
12334 4158 Acked Read 5 Sent Done Done false
12335 4159 Acked Write 5 Sent Done Done false
12336 4160 Acked Read 1 Sent Done Done false
12337 4161 Acked Write 4 Sent Done Done false
12338 4162 Acked Write 8 Sent Done Done false
12339 4163 Acked Write 5 Sent Done Done false
12340 4164 Acked Read 4 Sent Done Done false
12341 4165 Acked Write 5 Sent Done Done false
12342 4166 Acked Write 7 Sent Done Done false
12343 4167 Acked Read 8 Sent Done Done false
12344 4168 Acked Write 6 Sent Done Done false
12345 4169 Acked Flush 0 Sent Done Done false
12346 4170 Acked Read 3 Sent Done Done false
12347 4171 Acked Write 5 Sent Done Done false
12348 4172 Acked Write 5 Sent Done Done false
12349 4173 Acked Read 4 Sent Done Done false
12350 4174 Acked Flush 0 Sent Done Done false
12351 4175 Acked Read 10 Sent Done Done false
12352 4176 Acked Read 10 Sent Done Done false
12353 4177 Acked Read 2 Sent Done Done false
12354 4178 Acked Read 10 Sent Done Done false
12355 4179 Acked Read 10 Sent Done Done false
12356 4180 Acked Write 5 Sent Done Done false
12357 4181 Acked Read 6 Sent Done Done false
12358 4182 Acked Write 10 Sent Done Done false
12359 4183 Acked Read 8 Sent Done Done false
12360 4184 Acked Read 4 Sent Done Done false
12361 4185 Acked Write 10 Sent Done Done false
12362 4186 Acked Flush 0 Sent Done Done false
12363 4187 Acked Read 10 Sent Done Done false
12364 4188 Acked Write 10 Sent Done Done false
12365 4189 Acked Flush 0 Sent Done Done false
12366 4190 Acked Write 1 Sent Done Done false
12367 4191 Acked Read 7 Sent Done Done false
12368 4192 Acked Write 6 Sent Done Done false
12369 4193 Acked Write 3 Sent Done Done false
12370 4194 Acked Read 6 Sent Done Done false
12371 4195 Acked Write 2 Sent Done Done false
12372 4196 Acked Write 5 Sent Done Done false
12373 4197 Acked Write 5 Sent Done Done false
12374 4198 Acked Write 4 Sent Done Done false
12375 4199 Acked Read 10 Sent Done Done false
12376 4200 Acked Read 2 Sent Done Done false
12377 4201 Acked Read 10 Sent Done Done false
12378 4202 Acked Flush 0 Sent Done Done false
12379 4203 Acked Write 10 Sent Done Done false
12380 4204 Acked Write 9 Sent Done Done false
12381 4205 Acked Read 4 Sent Done Done false
12382 4206 Acked Read 9 Sent Done Done false
12383 4207 Acked Read 2 Sent Done Done false
12384 4208 Acked Read 6 Sent Done Done false
12385 4209 Acked Flush 0 Sent Done Done false
12386 4210 Acked Read 5 Sent Done Done false
12387 4211 Acked Write 3 Sent Done Done false
12388 4212 Acked Write 2 Sent Done Done false
12389 4213 Acked Write 3 Sent Done Done false
12390 4214 Acked Flush 0 Sent Done Done false
12391 4215 Acked Read 10 Sent Done Done false
12392 4216 Acked Read 10 Sent Done Done false
12393 4217 Acked Read 5 Sent Done Done false
12394 4218 Acked Read 2 Sent Done Done false
12395 4219 Acked Read 3 Sent Done Done false
12396 4220 Acked Read 4 Sent Done Done false
12397 4221 Acked Read 8 Sent Done Done false
12398 4222 Acked Write 3 Sent Done Done false
12399 4223 Acked Write 5 Sent Done Done false
12400 4224 Acked Write 7 Sent Done Done false
12401 4225 Acked Write 7 Sent Done Done false
12402 4226 Acked Read 3 Sent Done Done false
12403 4227 Acked Read 4 Sent Done Done false
12404 4228 Acked Read 10 Sent Done Done false
12405 4229 Acked Flush 0 Sent Done Done false
12406 4230 Acked Write 3 Sent Done Done false
12407 4231 Acked Read 8 Sent Done Done false
12408 4232 Acked Read 10 Sent Done Done false
12409 4233 Acked Read 2 Sent Done Done false
12410 4234 Acked Flush 0 Sent Done Done false
12411 4235 Acked Read 5 Sent Done Done false
12412 4236 Acked Read 9 Sent Done Done false
12413 4237 Acked Write 1 Sent Done Done false
12414 4238 Acked Write 1 Sent Done Done false
12415 4239 Acked Read 6 Sent Done Done false
12416 4240 Acked Read 4 Sent Done Done false
12417 4241 Acked Read 9 Sent Done Done false
12418 4242 Acked Write 7 Sent Done Done false
12419 4243 Acked Write 5 Sent Done Done false
12420 4244 Acked Write 6 Sent Done Done false
12421 4245 Acked Write 6 Sent Done Done false
12422 4246 Acked Flush 0 Sent Done Done false
12423 4247 Acked Write 3 Sent Done Done false
12424 4248 Acked Read 5 Sent Done Done false
12425 4249 Acked Write 6 Sent Done Done false
12426 4250 Acked Flush 0 Sent Done Done false
12427 4251 Acked Read 8 Sent Done Done false
12428 4252 Acked Read 7 Sent Done Done false
12429 4253 Acked Write 9 Sent Done Done false
12430 4254 Acked Write 8 Sent Done Done false
12431 4255 Acked Read 6 Sent Done Done false
12432 4256 Acked Read 3 Sent Done Done false
12433 4257 Acked Read 10 Sent Done Done false
12434 4258 Acked Read 3 Sent Done Done false
12435 4259 Acked Read 3 Sent Done Done false
12436 4260 Acked Write 8 Sent Done Done false
12437 4261 Acked Write 4 Sent Done Done false
12438 4262 Acked Read 2 Sent Done Done false
12439 4263 Acked Write 6 Sent Done Done false
12440 4264 Acked Read 9 Sent Done Done false
12441 4265 Acked Write 1 Sent Done Done false
12442 4266 Acked Read 3 Sent Done Done false
12443 4267 Acked Read 10 Sent Done Done false
12444 4268 Acked Read 6 Sent Done Done false
12445 4269 Acked Write 2 Sent Done Done false
12446 4270 Acked Write 3 Sent Done Done false
12447 4271 Acked Read 7 Sent Done Done false
12448 4272 Acked Read 4 Sent Done Done false
12449 4273 Acked Write 2 Sent Done Done false
12450 4274 Acked Read 10 Sent Done Done false
12451 4275 Acked Read 6 Sent Done Done false
12452 4276 Acked Read 1 Sent Done Done false
12453 4277 Acked Read 9 Sent Done Done false
12454 4278 Acked Write 3 Sent Done Done false
12455 4279 Acked Write 1 Sent Done Done false
12456 4280 Acked Write 10 Sent Done Done false
12457 4281 Acked Write 7 Sent Done Done false
12458 4282 Acked Read 5 Sent Done Done false
12459 4283 Acked Write 5 Sent Done Done false
12460 4284 Acked Read 4 Sent Done Done false
12461 4285 Acked Write 5 Sent Done Done false
12462 4286 Acked Read 8 Sent Done Done false
12463 4287 Acked Write 10 Sent Done Done false
12464 4288 Acked Read 1 Sent Done Done false
12465 4289 Acked Write 3 Sent Done Done false
12466 4290 Acked Read 6 Sent Done Done false
12467 4291 Acked Read 3 Sent Done Done false
12468 4292 Acked Write 5 Sent Done Done false
12469 4293 Acked Read 7 Sent Done Done false
12470 4294 Acked Read 6 Sent Done Done false
12471 4295 Acked Read 1 Sent Done Done false
12472 4296 Acked Read 3 Sent Done Done false
12473 4297 Acked Write 2 Sent Done Done false
12474 4298 Acked Write 4 Sent Done Done false
12475 4299 Acked Read 9 Sent Done Done false
12476 4300 Acked Write 3 Sent Done Done false
12477 4301 Acked Read 2 Sent Done Done false
12478 4302 Acked Read 1 Sent Done Done false
12479 4303 Acked Read 5 Sent Done Done false
12480 4304 Acked Read 10 Sent Done Done false
12481 4305 Acked Read 8 Sent Done Done false
12482 4306 Acked Read 1 Sent Done Done false
12483 4307 Acked Read 5 Sent Done Done false
12484 4308 Acked Write 7 Sent Done Done false
12485 4309 Acked Write 10 Sent Done Done false
12486 4310 Acked Write 5 Sent Done Done false
12487 4311 Acked Write 7 Sent Done Done false
12488 4312 Acked Write 5 Sent Done Done false
12489 4313 Acked Read 3 Sent Done Done false
12490 4314 Acked Read 2 Sent Done Done false
12491 4315 Acked Read 3 Sent Done Done false
12492 4316 Acked Flush 0 Sent Done Done false
12493 4317 Acked Write 7 Sent Done Done false
12494 4318 Acked Write 4 Sent Done Done false
12495 4319 Acked Write 4 Sent Done Done false
12496 STATES DS:0 DS:1 DS:2 TOTAL
12497 Sent 301 0 0 301
12498 Done 0 301 301 602
12499 Skipped 0 0 0 0
12500 Error 0 0 0 0
12501 Last Flush: 4018 4316 4316
12502 Downstairs last five completed: 4018 4017 4016 4015 4014
12503 Upstairs last five completed: 4319 4318 4317 4316 4315
12504 CLIENT: Up:0 ds:301 act:2
12505 [8810] got start action
12506 [8810] I am going to restart
12507 Starting downstairs at port 8810
12508 Make output file at "/tmp/dsc/downstairs-8810.txt"
12509 [8810][0] reports Starting
12510 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1216)
12511 [8810][0] reports Running
12512 ----------------------------------------------------------------
12513 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12514 States: Offline Active Active
12515 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12516 4019 Acked Write 2 Sent Done Done false
12517 4020 Acked Read 1 Sent Done Done false
12518 4021 Acked Write 5 Sent Done Done false
12519 4022 Acked Read 2 Sent Done Done false
12520 4023 Acked Write 8 Sent Done Done false
12521 4024 Acked Read 10 Sent Done Done false
12522 4025 Acked Read 7 Sent Done Done false
12523 4026 Acked Write 1 Sent Done Done false
12524 4027 Acked Write 2 Sent Done Done false
12525 4028 Acked Write 9 Sent Done Done false
12526 4029 Acked Read 2 Sent Done Done false
12527 4030 Acked Write 8 Sent Done Done false
12528 4031 Acked Read 10 Sent Done Done false
12529 4032 Acked Read 4 Sent Done Done false
12530 4033 Acked Read 1 Sent Done Done false
12531 4034 Acked Read 2 Sent Done Done false
12532 4035 Acked Write 4 Sent Done Done false
12533 4036 Acked Read 10 Sent Done Done false
12534 4037 Acked Read 4 Sent Done Done false
12535 4038 Acked Read 6 Sent Done Done false
12536 4039 Acked Write 4 Sent Done Done false
12537 4040 Acked Read 3 Sent Done Done false
12538 4041 Acked Write 5 Sent Done Done false
12539 4042 Acked Read 10 Sent Done Done false
12540 4043 Acked Read 1 Sent Done Done false
12541 4044 Acked Read 9 Sent Done Done false
12542 4045 Acked Write 6 Sent Done Done false
12543 4046 Acked Read 6 Sent Done Done false
12544 4047 Acked Write 10 Sent Done Done false
12545 4048 Acked Read 5 Sent Done Done false
12546 4049 Acked Read 4 Sent Done Done false
12547 4050 Acked Read 3 Sent Done Done false
12548 4051 Acked Flush 0 Sent Done Done false
12549 4052 Acked Write 6 Sent Done Done false
12550 4053 Acked Flush 0 Sent Done Done false
12551 4054 Acked Read 6 Sent Done Done false
12552 4055 Acked Read 2 Sent Done Done false
12553 4056 Acked Read 10 Sent Done Done false
12554 4057 Acked Flush 0 Sent Done Done false
12555 4058 Acked Write 7 Sent Done Done false
12556 4059 Acked Write 9 Sent Done Done false
12557 4060 Acked Write 10 Sent Done Done false
12558 4061 Acked Write 4 Sent Done Done false
12559 4062 Acked Write 3 Sent Done Done false
12560 4063 Acked Read 7 Sent Done Done false
12561 4064 Acked Read 5 Sent Done Done false
12562 4065 Acked Flush 0 Sent Done Done false
12563 4066 Acked Write 8 Sent Done Done false
12564 4067 Acked Read 5 Sent Done Done false
12565 4068 Acked Write 8 Sent Done Done false
12566 4069 Acked Read 3 Sent Done Done false
12567 4070 Acked Read 10 Sent Done Done false
12568 4071 Acked Read 5 Sent Done Done false
12569 4072 Acked Write 5 Sent Done Done false
12570 4073 Acked Write 5 Sent Done Done false
12571 4074 Acked Write 1 Sent Done Done false
12572 4075 Acked Write 5 Sent Done Done false
12573 4076 Acked Write 10 Sent Done Done false
12574 4077 Acked Write 6 Sent Done Done false
12575 4078 Acked Read 3 Sent Done Done false
12576 4079 Acked Write 8 Sent Done Done false
12577 4080 Acked Write 6 Sent Done Done false
12578 4081 Acked Read 2 Sent Done Done false
12579 4082 Acked Read 4 Sent Done Done false
12580 4083 Acked Write 8 Sent Done Done false
12581 4084 Acked Read 8 Sent Done Done false
12582 4085 Acked Read 5 Sent Done Done false
12583 4086 Acked Write 4 Sent Done Done false
12584 4087 Acked Write 6 Sent Done Done false
12585 4088 Acked Read 3 Sent Done Done false
12586 4089 Acked Write 10 Sent Done Done false
12587 4090 Acked Read 8 Sent Done Done false
12588 4091 Acked Read 9 Sent Done Done false
12589 4092 Acked Flush 0 Sent Done Done false
12590 4093 Acked Write 7 Sent Done Done false
12591 4094 Acked Read 6 Sent Done Done false
12592 4095 Acked Read 4 Sent Done Done false
12593 4096 Acked Write 4 Sent Done Done false
12594 4097 Acked Write 3 Sent Done Done false
12595 4098 Acked Read 3 Sent Done Done false
12596 4099 Acked Flush 0 Sent Done Done false
12597 4100 Acked Flush 0 Sent Done Done false
12598 4101 Acked Read 7 Sent Done Done false
12599 4102 Acked Write 6 Sent Done Done false
12600 4103 Acked Read 3 Sent Done Done false
12601 4104 Acked Write 9 Sent Done Done false
12602 4105 Acked Write 1 Sent Done Done false
12603 4106 Acked Read 9 Sent Done Done false
12604 4107 Acked Read 4 Sent Done Done false
12605 4108 Acked Read 7 Sent Done Done false
12606 4109 Acked Read 3 Sent Done Done false
12607 4110 Acked Write 2 Sent Done Done false
12608 4111 Acked Read 2 Sent Done Done false
12609 4112 Acked Read 4 Sent Done Done false
12610 4113 Acked Read 9 Sent Done Done false
12611 4114 Acked Read 6 Sent Done Done false
12612 4115 Acked Write 5 Sent Done Done false
12613 4116 Acked Read 3 Sent Done Done false
12614 4117 Acked Write 6 Sent Done Done false
12615 4118 Acked Read 3 Sent Done Done false
12616 4119 Acked Write 10 Sent Done Done false
12617 4120 Acked Read 7 Sent Done Done false
12618 4121 Acked Write 5 Sent Done Done false
12619 4122 Acked Write 7 Sent Done Done false
12620 4123 Acked Read 7 Sent Done Done false
12621 4124 Acked Read 2 Sent Done Done false
12622 4125 Acked Read 3 Sent Done Done false
12623 4126 Acked Read 5 Sent Done Done false
12624 4127 Acked Write 3 Sent Done Done false
12625 4128 Acked Read 9 Sent Done Done false
12626 4129 Acked Write 3 Sent Done Done false
12627 4130 Acked Write 9 Sent Done Done false
12628 4131 Acked Write 3 Sent Done Done false
12629 4132 Acked Read 3 Sent Done Done false
12630 4133 Acked Read 6 Sent Done Done false
12631 4134 Acked Write 7 Sent Done Done false
12632 4135 Acked Read 3 Sent Done Done false
12633 4136 Acked Read 6 Sent Done Done false
12634 4137 Acked Write 6 Sent Done Done false
12635 4138 Acked Read 7 Sent Done Done false
12636 4139 Acked Flush 0 Sent Done Done false
12637 4140 Acked Read 7 Sent Done Done false
12638 4141 Acked Write 4 Sent Done Done false
12639 4142 Acked Write 3 Sent Done Done false
12640 4143 Acked Read 4 Sent Done Done false
12641 4144 Acked Read 9 Sent Done Done false
12642 4145 Acked Read 8 Sent Done Done false
12643 4146 Acked Write 3 Sent Done Done false
12644 4147 Acked Write 5 Sent Done Done false
12645 4148 Acked Write 9 Sent Done Done false
12646 4149 Acked Write 8 Sent Done Done false
12647 4150 Acked Write 10 Sent Done Done false
12648 4151 Acked Flush 0 Sent Done Done false
12649 4152 Acked Flush 0 Sent Done Done false
12650 4153 Acked Read 1 Sent Done Done false
12651 4154 Acked Write 5 Sent Done Done false
12652 4155 Acked Write 6 Sent Done Done false
12653 4156 Acked Read 10 Sent Done Done false
12654 4157 Acked Write 7 Sent Done Done false
12655 4158 Acked Read 5 Sent Done Done false
12656 4159 Acked Write 5 Sent Done Done false
12657 4160 Acked Read 1 Sent Done Done false
12658 4161 Acked Write 4 Sent Done Done false
12659 4162 Acked Write 8 Sent Done Done false
12660 4163 Acked Write 5 Sent Done Done false
12661 4164 Acked Read 4 Sent Done Done false
12662 4165 Acked Write 5 Sent Done Done false
12663 4166 Acked Write 7 Sent Done Done false
12664 4167 Acked Read 8 Sent Done Done false
12665 4168 Acked Write 6 Sent Done Done false
12666 4169 Acked Flush 0 Sent Done Done false
12667 4170 Acked Read 3 Sent Done Done false
12668 4171 Acked Write 5 Sent Done Done false
12669 4172 Acked Write 5 Sent Done Done false
12670 4173 Acked Read 4 Sent Done Done false
12671 4174 Acked Flush 0 Sent Done Done false
12672 4175 Acked Read 10 Sent Done Done false
12673 4176 Acked Read 10 Sent Done Done false
12674 4177 Acked Read 2 Sent Done Done false
12675 4178 Acked Read 10 Sent Done Done false
12676 4179 Acked Read 10 Sent Done Done false
12677 4180 Acked Write 5 Sent Done Done false
12678 4181 Acked Read 6 Sent Done Done false
12679 4182 Acked Write 10 Sent Done Done false
12680 4183 Acked Read 8 Sent Done Done false
12681 4184 Acked Read 4 Sent Done Done false
12682 4185 Acked Write 10 Sent Done Done false
12683 4186 Acked Flush 0 Sent Done Done false
12684 4187 Acked Read 10 Sent Done Done false
12685 4188 Acked Write 10 Sent Done Done false
12686 4189 Acked Flush 0 Sent Done Done false
12687 4190 Acked Write 1 Sent Done Done false
12688 4191 Acked Read 7 Sent Done Done false
12689 4192 Acked Write 6 Sent Done Done false
12690 4193 Acked Write 3 Sent Done Done false
12691 4194 Acked Read 6 Sent Done Done false
12692 4195 Acked Write 2 Sent Done Done false
12693 4196 Acked Write 5 Sent Done Done false
12694 4197 Acked Write 5 Sent Done Done false
12695 4198 Acked Write 4 Sent Done Done false
12696 4199 Acked Read 10 Sent Done Done false
12697 4200 Acked Read 2 Sent Done Done false
12698 4201 Acked Read 10 Sent Done Done false
12699 4202 Acked Flush 0 Sent Done Done false
12700 4203 Acked Write 10 Sent Done Done false
12701 4204 Acked Write 9 Sent Done Done false
12702 4205 Acked Read 4 Sent Done Done false
12703 4206 Acked Read 9 Sent Done Done false
12704 4207 Acked Read 2 Sent Done Done false
12705 4208 Acked Read 6 Sent Done Done false
12706 4209 Acked Flush 0 Sent Done Done false
12707 4210 Acked Read 5 Sent Done Done false
12708 4211 Acked Write 3 Sent Done Done false
12709 4212 Acked Write 2 Sent Done Done false
12710 4213 Acked Write 3 Sent Done Done false
12711 4214 Acked Flush 0 Sent Done Done false
12712 4215 Acked Read 10 Sent Done Done false
12713 4216 Acked Read 10 Sent Done Done false
12714 4217 Acked Read 5 Sent Done Done false
12715 4218 Acked Read 2 Sent Done Done false
12716 4219 Acked Read 3 Sent Done Done false
12717 4220 Acked Read 4 Sent Done Done false
12718 4221 Acked Read 8 Sent Done Done false
12719 4222 Acked Write 3 Sent Done Done false
12720 4223 Acked Write 5 Sent Done Done false
12721 4224 Acked Write 7 Sent Done Done false
12722 4225 Acked Write 7 Sent Done Done false
12723 4226 Acked Read 3 Sent Done Done false
12724 4227 Acked Read 4 Sent Done Done false
12725 4228 Acked Read 10 Sent Done Done false
12726 4229 Acked Flush 0 Sent Done Done false
12727 4230 Acked Write 3 Sent Done Done false
12728 4231 Acked Read 8 Sent Done Done false
12729 4232 Acked Read 10 Sent Done Done false
12730 4233 Acked Read 2 Sent Done Done false
12731 4234 Acked Flush 0 Sent Done Done false
12732 4235 Acked Read 5 Sent Done Done false
12733 4236 Acked Read 9 Sent Done Done false
12734 4237 Acked Write 1 Sent Done Done false
12735 4238 Acked Write 1 Sent Done Done false
12736 4239 Acked Read 6 Sent Done Done false
12737 4240 Acked Read 4 Sent Done Done false
12738 4241 Acked Read 9 Sent Done Done false
12739 4242 Acked Write 7 Sent Done Done false
12740 4243 Acked Write 5 Sent Done Done false
12741 4244 Acked Write 6 Sent Done Done false
12742 4245 Acked Write 6 Sent Done Done false
12743 4246 Acked Flush 0 Sent Done Done false
12744 4247 Acked Write 3 Sent Done Done false
12745 4248 Acked Read 5 Sent Done Done false
12746 4249 Acked Write 6 Sent Done Done false
12747 4250 Acked Flush 0 Sent Done Done false
12748 4251 Acked Read 8 Sent Done Done false
12749 4252 Acked Read 7 Sent Done Done false
12750 4253 Acked Write 9 Sent Done Done false
12751 4254 Acked Write 8 Sent Done Done false
12752 4255 Acked Read 6 Sent Done Done false
12753 4256 Acked Read 3 Sent Done Done false
12754 4257 Acked Read 10 Sent Done Done false
12755 4258 Acked Read 3 Sent Done Done false
12756 4259 Acked Read 3 Sent Done Done false
12757 4260 Acked Write 8 Sent Done Done false
12758 4261 Acked Write 4 Sent Done Done false
12759 4262 Acked Read 2 Sent Done Done false
12760 4263 Acked Write 6 Sent Done Done false
12761 4264 Acked Read 9 Sent Done Done false
12762 4265 Acked Write 1 Sent Done Done false
12763 4266 Acked Read 3 Sent Done Done false
12764 4267 Acked Read 10 Sent Done Done false
12765 4268 Acked Read 6 Sent Done Done false
12766 4269 Acked Write 2 Sent Done Done false
12767 4270 Acked Write 3 Sent Done Done false
12768 4271 Acked Read 7 Sent Done Done false
12769 4272 Acked Read 4 Sent Done Done false
12770 4273 Acked Write 2 Sent Done Done false
12771 4274 Acked Read 10 Sent Done Done false
12772 4275 Acked Read 6 Sent Done Done false
12773 4276 Acked Read 1 Sent Done Done false
12774 4277 Acked Read 9 Sent Done Done false
12775 4278 Acked Write 3 Sent Done Done false
12776 4279 Acked Write 1 Sent Done Done false
12777 4280 Acked Write 10 Sent Done Done false
12778 4281 Acked Write 7 Sent Done Done false
12779 4282 Acked Read 5 Sent Done Done false
12780 4283 Acked Write 5 Sent Done Done false
12781 4284 Acked Read 4 Sent Done Done false
12782 4285 Acked Write 5 Sent Done Done false
12783 4286 Acked Read 8 Sent Done Done false
12784 4287 Acked Write 10 Sent Done Done false
12785 4288 Acked Read 1 Sent Done Done false
12786 4289 Acked Write 3 Sent Done Done false
12787 4290 Acked Read 6 Sent Done Done false
12788 4291 Acked Read 3 Sent Done Done false
12789 4292 Acked Write 5 Sent Done Done false
12790 4293 Acked Read 7 Sent Done Done false
12791 4294 Acked Read 6 Sent Done Done false
12792 4295 Acked Read 1 Sent Done Done false
12793 4296 Acked Read 3 Sent Done Done false
12794 4297 Acked Write 2 Sent Done Done false
12795 4298 Acked Write 4 Sent Done Done false
12796 4299 Acked Read 9 Sent Done Done false
12797 4300 Acked Write 3 Sent Done Done false
12798 4301 Acked Read 2 Sent Done Done false
12799 4302 Acked Read 1 Sent Done Done false
12800 4303 Acked Read 5 Sent Done Done false
12801 4304 Acked Read 10 Sent Done Done false
12802 4305 Acked Read 8 Sent Done Done false
12803 4306 Acked Read 1 Sent Done Done false
12804 4307 Acked Read 5 Sent Done Done false
12805 4308 Acked Write 7 Sent Done Done false
12806 4309 Acked Write 10 Sent Done Done false
12807 4310 Acked Write 5 Sent Done Done false
12808 4311 Acked Write 7 Sent Done Done false
12809 4312 Acked Write 5 Sent Done Done false
12810 4313 Acked Read 3 Sent Done Done false
12811 4314 Acked Read 2 Sent Done Done false
12812 4315 Acked Read 3 Sent Done Done false
12813 4316 Acked Flush 0 Sent Done Done false
12814 4317 Acked Write 7 Sent Done Done false
12815 4318 Acked Write 4 Sent Done Done false
12816 4319 Acked Write 4 Sent Done Done false
12817 4320 Acked Flush 0 Sent Done Done false
12818 STATES DS:0 DS:1 DS:2 TOTAL
12819 Sent 302 0 0 302
12820 Done 0 302 302 604
12821 Skipped 0 0 0 0
12822 Error 0 0 0 0
12823 Last Flush: 4018 4320 4320
12824 Downstairs last five completed: 4018 4017 4016 4015 4014
12825 Upstairs last five completed: 4320 4319 4318 4317 4316
12826 CLIENT: Up:0 ds:302 act:2
128272025-01-03T08:33:34.060ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128282025-01-03T08:33:34.060ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43146) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128292025-01-03T08:33:34.061ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128302025-01-03T08:33:34.062ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128312025-01-03T08:33:34.062ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128322025-01-03T08:33:34.062ZINFOcrucible: send last flush ID to this DS: 4018 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128332025-01-03T08:33:34.062ZINFOcrucible: Replied this last flush ID: 4018 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128342025-01-03T08:33:34.062ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128352025-01-03T08:33:34.062ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128362025-01-03T08:33:34.062ZINFOcrucible: [0] client re-new 302 jobs since flush 4018 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128372025-01-03T08:33:34.062ZINFOcrucible: [0] Replayed 302 jobs since flush: 4018 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
12838 ----------------------------------------------------------------
12839 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12840 Downstairs last five completed: 4320 4319 4318 4317 4316
12841 Upstairs last five completed: 4320 4319 4318 4317 4316
12842 CLIENT: Up:0 ds:0 act:3
12843 Replay: All jobs finished
12844 Main task has work to do, go find it
12845 got dsc Stop(1)
12846 stop 1
12847 Jan 03 08:33:36.308 INFO request completed, latency_us: 337, response_code: 204, uri: /stop/cid/1, method: POST, req_id: e5e6e69c-c8c3-4e05-b6fc-cf70d1c797ba, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
12848 [8820] Got stop action so:false kr:false
12849 Jan 03 08:33:36.309 INFO request completed, latency_us: 138, response_code: 200, uri: /state/cid/1, method: GET, req_id: 0f273932-412e-46fb-8c33-a6141dcc0904, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
128502025-01-03T08:33:36.309ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128512025-01-03T08:33:36.309ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128522025-01-03T08:33:36.309ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128532025-01-03T08:33:36.309ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128542025-01-03T08:33:36.309ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
128552025-01-03T08:33:36.309ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
12856 [8820] Exited with: ExitStatus(unix_wait_status(9))
12857 [8820][1] reports Exit
12858 Jan 03 08:33:40.311 INFO request completed, latency_us: 190, response_code: 200, uri: /state/cid/1, method: GET, req_id: 88d09856-1d2d-41f1-88b0-742d50c12aab, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
12859 001/300 Read block 3433 len 24576
12860 002/300 Read block 1618 len 24576
12861 003/300 Read block 65 len 40960
12862 004/300 Flush
12863 005/300 Read block 3891 len 20480
12864 006/300 Read block 1110 len 28672
12865 007/300 Read block 3680 len 8192
12866 008/300 Read block 3635 len 4096
12867 009/300 Write block 663 len 36864 data: 4 5 5 5 4 5 6 3 3
12868 010/300 Read block 2534 len 40960
12869 011/300 Write block 2532 len 16384 data: 2 2 2 2
12870 012/300 Read block 933 len 12288
12871 013/300 Write block 3012 len 24576 data: 2 2 2 3 3 3
12872 014/300 Read block 1996 len 4096
12873 015/300 Write block 438 len 16384 data: 5 6 6 4
12874 016/300 Read block 4113 len 20480
12875 017/300 Flush
12876 018/300 Read block 3301 len 20480
12877 019/300 Read block 1398 len 20480
12878 020/300 Flush
12879 021/300 Read block 4734 len 32768
12880 022/300 Flush
12881 023/300 Write block 1046 len 24576 data: 4 3 2 3 3 3
12882 024/300 Write block 140 len 20480 data: 4 4 4 4 4
12883 025/300 Write block 3230 len 28672 data: 4 4 5 5 5 5 4
12884 026/300 Write block 2285 len 28672 data: 3 3 2 3 4 3 3
12885 027/300 Read block 3720 len 24576
12886 028/300 Write block 1290 len 16384 data: 3 4 3 3
12887 029/300 Read block 3520 len 32768
12888 030/300 Read block 4840 len 40960
12889 031/300 Read block 2249 len 36864
12890 032/300 Write block 258 len 32768 data: 2 2 2 2 2 2 2 2
12891 033/300 Write block 2782 len 8192 data: 3 3
12892 034/300 Write block 387 len 20480 data: 2 2 2 2 2
12893 035/300 Write block 2165 len 36864 data: 3 2 2 4 3 3 3 3 3
12894 036/300 Write block 2842 len 16384 data: 4 3 3 3
12895 037/300 Write block 2114 len 28672 data: 4 4 4 3 2 2 2
12896 038/300 Read block 2084 len 12288
12897 039/300 Read block 2454 len 12288
12898 040/300 Write block 3544 len 40960 data: 7 7 7 6 6 4 3 3 2 2
12899 041/300 Write block 2671 len 8192 data: 2 2
12900 042/300 Write block 4448 len 16384 data: 6 4 4 3
12901 043/300 Write block 1709 len 40960 data: 2 2 2 2 3 2 2 2 2 2
12902 044/300 Read block 276 len 32768
12903 045/300 Flush
12904 046/300 Write block 3220 len 36864 data: 4 4 4 4 4 4 2 2 3
12905 047/300 Read block 4336 len 20480
12906 048/300 Write block 2720 len 24576 data: 5 6 6 6 5 5
12907 049/300 Read block 1508 len 16384
12908 050/300 Read block 2858 len 32768
12909 051/300 Write block 4005 len 16384 data: 5 4 3 3
12910 052/300 Read block 4176 len 16384
12911 053/300 Read block 2146 len 28672
12912 054/300 Read block 1621 len 4096
12913 055/300 Read block 1249 len 20480
12914 056/300 Write block 4412 len 36864 data: 4 4 3 3 3 2 2 3 3
12915 057/300 Read block 4366 len 4096
12916 058/300 Write block 3173 len 20480 data: 4 4 4 5 6
12917 059/300 Write block 3820 len 28672 data: 4 4 4 4 5 5 5
12918 060/300 Read block 2251 len 8192
12919 061/300 Read block 2139 len 8192
12920 062/300 Read block 668 len 20480
12921 063/300 Write block 4661 len 4096 data: 3
12922 064/300 Write block 2748 len 16384 data: 4 5 4 4
12923 065/300 Read block 3808 len 16384
12924 066/300 Flush
12925 067/300 Read block 4972 len 20480
12926 068/300 Write block 2677 len 24576 data: 3 3 3 3 2 4
12927 069/300 Read block 2204 len 28672
12928 070/300 Read block 1388 len 40960
12929 071/300 Flush
12930 072/300 Flush
12931 073/300 Write block 3028 len 32768 data: 2 2 2 2 2 2 2 2
12932 074/300 Read block 1984 len 24576
12933 075/300 Read block 3797 len 36864
12934 076/300 Read block 907 len 24576
12935 077/300 Read block 1356 len 28672
12936 078/300 Write block 4980 len 12288 data: 2 3 2
12937 079/300 Flush
12938 080/300 Read block 3109 len 40960
12939 081/300 Read block 4028 len 36864
12940 082/300 Write block 361 len 16384 data: 3 3 3 4
12941 083/300 Write block 2418 len 36864 data: 4 3 3 3 3 3 3 3 3
12942 084/300 Read block 2653 len 28672
12943 085/300 Write block 94 len 16384 data: 2 2 4 4
12944 086/300 Read block 72 len 4096
12945 087/300 Read block 258 len 40960
12946 088/300 Read block 4303 len 8192
12947 089/300 Flush
12948 090/300 Read block 1689 len 28672
12949 091/300 Read block 1257 len 8192
12950 092/300 Read block 1847 len 24576
12951 093/300 Write block 1360 len 16384 data: 5 4 4 5
12952 094/300 Write block 2525 len 8192 data: 6 5
12953 095/300 Flush
12954 096/300 Write block 4312 len 8192 data: 5 4
12955 097/300 Read block 4701 len 4096
12956 098/300 Read block 2868 len 16384
12957 099/300 Read block 1693 len 8192
12958 100/300 Read block 3958 len 12288
12959 101/300 Read block 498 len 12288
12960 102/300 Read block 950 len 32768
12961 103/300 Write block 3258 len 8192 data: 4 4
12962 104/300 Read block 14 len 32768
12963 105/300 Write block 539 len 40960 data: 4 4 4 4 4 3 3 2 2 2
12964 106/300 Write block 458 len 20480 data: 4 4 4 4 4
12965 107/300 Read block 1295 len 4096
12966 108/300 Read block 519 len 32768
12967 109/300 Write block 3855 len 36864 data: 6 4 3 3 3 4 5 6 7
12968 110/300 Write block 2418 len 24576 data: 5 4 4 4 4 4
12969 111/300 Read block 4582 len 32768
12970 112/300 Read block 1177 len 4096
12971 113/300 Read block 457 len 8192
12972 114/300 Write block 4883 len 28672 data: 4 3 3 4 6 6 6
12973 115/300 Read block 447 len 24576
12974 116/300 Read block 3806 len 36864
12975 117/300 Write block 2134 len 12288 data: 4 3 4
12976 118/300 Flush
12977 119/300 Write block 1718 len 4096 data: 3
12978 120/300 Write block 2964 len 8192 data: 2 2
12979 121/300 Write block 3419 len 28672 data: 2 2 3 5 5 5 5
12980 122/300 Write block 883 len 28672 data: 3 2 3 3 3 4 3
12981 123/300 Flush
12982 124/300 Write block 3511 len 32768 data: 3 3 3 3 4 4 3 3
12983 125/300 Read block 1540 len 12288
12984 126/300 Read block 1817 len 8192
12985 127/300 Write block 1464 len 16384 data: 4 4 4 3
12986 128/300 Write block 2388 len 28672 data: 3 3 3 3 2 2 2
12987 129/300 Read block 2642 len 4096
12988 130/300 Read block 3936 len 36864
12989 131/300 Read block 633 len 16384
12990 132/300 Write block 3839 len 4096 data: 3
12991 133/300 Write block 432 len 4096 data: 3
12992 134/300 Write block 4603 len 40960 data: 2 2 2 2 2 2 2 2 2 2
12993 135/300 Read block 4287 len 12288
12994 136/300 Read block 498 len 8192
12995 137/300 Read block 749 len 12288
12996 138/300 Read block 805 len 36864
12997 139/300 Flush
12998 140/300 Read block 65 len 4096
12999 141/300 Read block 3128 len 40960
13000 142/300 Read block 2459 len 28672
13001 143/300 Read block 3026 len 16384
13002 144/300 Flush
13003 145/300 Read block 2665 len 4096
13004 146/300 Write block 2504 len 36864 data: 2 2 2 2 2 2 2 3 3
13005 147/300 Read block 2696 len 40960
13006 148/300 Write block 4954 len 36864 data: 3 4 4 4 4 4 4 4 4
13007 149/300 Flush
13008 150/300 Write block 4948 len 12288 data: 4 3 2
13009 151/300 Read block 3058 len 16384
13010 152/300 Read block 2677 len 8192
13011 153/300 Write block 4638 len 4096 data: 3
13012 154/300 Read block 3813 len 8192
13013 155/300 Read block 2403 len 32768
13014 156/300 Read block 3715 len 20480
13015 157/300 Read block 3798 len 4096
13016 158/300 Write block 2132 len 24576 data: 4 4 5 4 5 4
13017 159/300 Write block 4432 len 16384 data: 3 3 3 3
13018 160/300 Write block 350 len 36864 data: 6 6 6 4 4 2 2 2 2
13019 161/300 Flush
13020 162/300 Read block 1616 len 16384
13021 163/300 Read block 3856 len 28672
13022 164/300 Read block 3721 len 4096
13023 165/300 Read block 4937 len 36864
13024 166/300 Read block 3660 len 28672
13025 167/300 Read block 3667 len 24576
13026 168/300 Write block 4947 len 32768 data: 4 5 4 3 2 3 3 4
13027 169/300 Read block 1139 len 24576
13028 170/300 Write block 652 len 36864 data: 4 4 6 6 5 6 6 5 4
13029 171/300 Write block 4819 len 8192 data: 3 3
13030 172/300 Read block 3973 len 20480
13031 173/300 Write block 3401 len 8192 data: 5 6
13032 174/300 Write block 3327 len 36864 data: 2 2 2 2 3 3 3 3 2
13033 175/300 Flush
13034 176/300 Read block 1969 len 36864
13035 177/300 Read block 1377 len 4096
13036 178/300 Read block 3751 len 8192
13037 179/300 Read block 2207 len 20480
13038 180/300 Write block 713 len 32768 data: 5 5 3 3 3 2 3 3
13039 181/300 Read block 2647 len 12288
13040 182/300 Write block 2899 len 12288 data: 3 4 4
13041 183/300 Read block 2012 len 8192
13042 184/300 Write block 245 len 24576 data: 3 3 3 3 4 4
13043 185/300 Flush
13044 186/300 Read block 1397 len 40960
13045 187/300 Read block 2430 len 12288
13046 188/300 Write block 1735 len 20480 data: 4 6 5 5 4
13047 189/300 Write block 4781 len 28672 data: 3 4 5 4 4 3 3
13048 190/300 Write block 4552 len 36864 data: 3 3 3 3 3 4 4 4 4
13049 191/300 Read block 3622 len 8192
13050 192/300 Write block 4503 len 20480 data: 4 4 4 4 3
13051 193/300 Flush
13052 194/300 Flush
13053 195/300 Read block 1638 len 8192
13054 196/300 Read block 4604 len 4096
13055 197/300 Write block 2087 len 28672 data: 4 4 3 3 3 3 3
13056 198/300 Read block 2305 len 16384
13057 199/300 Read block 429 len 32768
13058 200/300 Write block 187 len 16384 data: 2 2 2 2
13059 201/300 Read block 2382 len 36864
13060 202/300 Read block 538 len 20480
13061 203/300 Flush
13062 204/300 Read block 4131 len 16384
13063 205/300 Flush
13064 206/300 Read block 730 len 16384
13065 207/300 Read block 4025 len 28672
13066 208/300 Write block 1242 len 8192 data: 2 2
13067 209/300 Read block 2197 len 36864
13068 210/300 Write block 4666 len 36864 data: 5 4 4 4 4 3 3 3 3
13069 211/300 Read block 2700 len 4096
13070 212/300 Write block 1098 len 24576 data: 4 3 4 4 3 2
13071 213/300 Read block 2767 len 16384
13072 214/300 Read block 3169 len 24576
13073 215/300 Read block 3976 len 4096
13074 216/300 Write block 4712 len 4096 data: 5
13075 217/300 Read block 655 len 40960
13076 218/300 Read block 172 len 16384
13077 219/300 Read block 4934 len 20480
13078 220/300 Read block 3002 len 20480
13079 221/300 Read block 230 len 4096
13080 222/300 Flush
13081 223/300 Read block 679 len 24576
13082 224/300 Write block 4846 len 24576 data: 3 3 3 3 3 3
13083 225/300 Flush
13084 226/300 Read block 3991 len 36864
13085 227/300 Read block 4306 len 36864
13086 228/300 Read block 18 len 8192
13087 229/300 Read block 2077 len 12288
13088 230/300 Read block 282 len 40960
13089 231/300 Read block 71 len 40960
13090 232/300 Write block 329 len 20480 data: 4 4 4 4 3
13091 233/300 Read block 3627 len 4096
13092 234/300 Read block 2935 len 28672
13093 235/300 Write block 2142 len 16384 data: 3 2 2 2
13094 236/300 Write block 1128 len 36864 data: 2 2 3 2 3 4 3 4 4
13095 237/300 Read block 3789 len 32768
13096 238/300 Read block 2569 len 16384
13097 239/300 Read block 887 len 12288
13098 240/300 Write block 1585 len 4096 data: 5
13099 241/300 Read block 1747 len 20480
13100 242/300 Write block 2199 len 28672 data: 3 3 3 3 3 4 5
13101 243/300 Flush
13102 244/300 Read block 1393 len 8192
13103 245/300 Write block 1013 len 20480 data: 3 4 4 4 3
13104 246/300 Read block 1031 len 40960
13105 247/300 Read block 3018 len 12288
13106 248/300 Read block 1839 len 32768
13107 249/300 Write block 1881 len 40960 data: 4 6 4 5 5 5 4 4 4 4
13108 250/300 Write block 74 len 4096 data: 3
13109 251/300 Read block 4438 len 8192
13110 252/300 Read block 1415 len 36864
13111 253/300 Write block 2041 len 24576 data: 2 3 3 3 3 4
13112 254/300 Read block 2467 len 40960
13113 255/300 Read block 2134 len 36864
13114 256/300 Flush
13115 257/300 Read block 499 len 32768
13116 258/300 Flush
13117 259/300 Read block 2468 len 36864
13118 260/300 Write block 2795 len 4096 data: 4
13119 261/300 Write block 153 len 28672 data: 2 2 4 4 4 4 4
13120 262/300 Write block 1171 len 20480 data: 4 3 3 4 4
13121 263/300 Read block 3202 len 4096
13122 264/300 Write block 3015 len 36864 data: 4 4 4 3 3 3 3 3 3
13123 265/300 Read block 1675 len 4096
13124 266/300 Write block 195 len 12288 data: 5 5 5
13125 267/300 Flush
13126 268/300 Read block 41 len 12288
13127 269/300 Read block 81 len 12288
13128 270/300 Read block 1656 len 36864
13129 271/300 Read block 2026 len 8192
13130 272/300 Read block 378 len 36864
13131 273/300 Write block 4548 len 20480 data: 2 2 2 3 4
13132 274/300 Read block 4170 len 8192
13133 275/300 Flush
13134 276/300 Write block 4591 len 16384 data: 4 4 4 4
13135 277/300 Read block 2098 len 4096
13136 278/300 Read block 3592 len 8192
13137 279/300 Read block 963 len 32768
13138 280/300 Read block 71 len 40960
13139 281/300 Write block 501 len 20480 data: 4 3 3 3 3
13140 282/300 Write block 1115 len 40960 data: 2 2 4 4 4 4 5 5 4 3
13141 283/300 Read block 4599 len 20480
13142 284/300 Write block 4573 len 36864 data: 2 3 3 3 3 5 5 5 6
13143 285/300 Read block 4742 len 16384
13144 286/300 Write block 1119 len 40960 data: 5 5 6 6 5 4 2 2 2 3
13145 287/300 Read block 348 len 20480
13146 288/300 Read block 1656 len 4096
13147 289/300 Read block 176 len 16384
13148 290/300 Read block 239 len 28672
13149 291/300 Write block 1847 len 32768 data: 2 2 2 2 2 2 2 2
13150 292/300 Read block 2528 len 28672
13151 293/300 Read block 2142 len 4096
13152 294/300 Read block 185 len 8192
13153 295/300 Flush
13154 296/300 Read block 1140 len 16384
13155 297/300 Write block 4748 len 8192 data: 4 4
13156 298/300 Write block 837 len 40960 data: 4 4 4 4 4 4 6 6 5 5
13157 299/300 Flush
13158 300/300 Read block 4239 len 28672
13159 301/300 Read block 4657 len 16384
13160 Main task has work to do, go find it
13161 got dsc Start(1)
13162 start 1
13163 Jan 03 08:33:40.557 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/1, method: POST, req_id: f985f323-9cf0-40b0-8bd5-7a1c09381d21, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
13164 Replay: started 1, returned:Ok(())
13165 ----------------------------------------------------------------
13166 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13167 States: Active Offline Active
13168 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13169 4321 Acked Read 6 Done Sent Done false
13170 4322 Acked Read 6 Done Sent Done false
13171 4323 Acked Read 10 Done Sent Done false
13172 4324 Acked Flush 0 Done Sent Done false
13173 4325 Acked Read 5 Done Sent Done false
13174 4326 Acked Read 7 Done Sent Done false
13175 4327 Acked Read 2 Done Sent Done false
13176 4328 Acked Read 1 Done Sent Done false
13177 4329 Acked Write 9 Done Sent Done false
13178 4330 Acked Read 10 Done Sent Done false
13179 4331 Acked Write 4 Done Sent Done false
13180 4332 Acked Read 3 Done Sent Done false
13181 4333 Acked Write 6 Done Sent Done false
13182 4334 Acked Read 1 Done Sent Done false
13183 4335 Acked Write 4 Done Sent Done false
13184 4336 Acked Read 5 Done Sent Done false
13185 4337 Acked Flush 0 Done Sent Done false
13186 4338 Acked Read 5 Done Sent Done false
13187 4339 Acked Read 5 Done Sent Done false
13188 4340 Acked Flush 0 Done Sent Done false
13189 4341 Acked Read 8 Done Sent Done false
13190 4342 Acked Flush 0 Done Sent Done false
13191 4343 Acked Write 6 Done Sent Done false
13192 4344 Acked Write 5 Done Sent Done false
13193 4345 Acked Write 7 Done Sent Done false
13194 4346 Acked Write 7 Done Sent Done false
13195 4347 Acked Read 6 Done Sent Done false
13196 4348 Acked Write 4 Done Sent Done false
13197 4349 Acked Read 8 Done Sent Done false
13198 4350 Acked Read 10 Done Sent Done false
13199 4351 Acked Read 9 Done Sent Done false
13200 4352 Acked Write 8 Done Sent Done false
13201 4353 Acked Write 2 Done Sent Done false
13202 4354 Acked Write 5 Done Sent Done false
13203 4355 Acked Write 9 Done Sent Done false
13204 4356 Acked Write 4 Done Sent Done false
13205 4357 Acked Write 7 Done Sent Done false
13206 4358 Acked Read 3 Done Sent Done false
13207 4359 Acked Read 3 Done Sent Done false
13208 4360 Acked Write 10 Done Sent Done false
13209 4361 Acked Write 2 Done Sent Done false
13210 4362 Acked Write 4 Done Sent Done false
13211 4363 Acked Write 10 Done Sent Done false
13212 4364 Acked Read 8 Done Sent Done false
13213 4365 Acked Flush 0 Done Sent Done false
13214 4366 Acked Write 9 Done Sent Done false
13215 4367 Acked Read 5 Done Sent Done false
13216 4368 Acked Write 6 Done Sent Done false
13217 4369 Acked Read 4 Done Sent Done false
13218 4370 Acked Read 8 Done Sent Done false
13219 4371 Acked Write 4 Done Sent Done false
13220 4372 Acked Read 4 Done Sent Done false
13221 4373 Acked Read 7 Done Sent Done false
13222 4374 Acked Read 1 Done Sent Done false
13223 4375 Acked Read 5 Done Sent Done false
13224 4376 Acked Write 9 Done Sent Done false
13225 4377 Acked Read 1 Done Sent Done false
13226 4378 Acked Write 5 Done Sent Done false
13227 4379 Acked Write 7 Done Sent Done false
13228 4380 Acked Read 2 Done Sent Done false
13229 4381 Acked Read 2 Done Sent Done false
13230 4382 Acked Read 5 Done Sent Done false
13231 4383 Acked Write 1 Done Sent Done false
13232 4384 Acked Write 4 Done Sent Done false
13233 4385 Acked Read 4 Done Sent Done false
13234 4386 Acked Flush 0 Done Sent Done false
13235 4387 Acked Read 5 Done Sent Done false
13236 4388 Acked Write 6 Done Sent Done false
13237 4389 Acked Read 7 Done Sent Done false
13238 4390 Acked Read 10 Done Sent Done false
13239 4391 Acked Flush 0 Done Sent Done false
13240 4392 Acked Flush 0 Done Sent Done false
13241 4393 Acked Write 8 Done Sent Done false
13242 4394 Acked Read 6 Done Sent Done false
13243 4395 Acked Read 9 Done Sent Done false
13244 4396 Acked Read 6 Done Sent Done false
13245 4397 Acked Read 7 Done Sent Done false
13246 4398 Acked Write 3 Done Sent Done false
13247 4399 Acked Flush 0 Done Sent Done false
13248 4400 Acked Read 10 Done Sent Done false
13249 4401 Acked Read 9 Done Sent Done false
13250 4402 Acked Write 4 Done Sent Done false
13251 4403 Acked Write 9 Done Sent Done false
13252 4404 Acked Read 7 Done Sent Done false
13253 4405 Acked Write 4 Done Sent Done false
13254 4406 Acked Read 1 Done Sent Done false
13255 4407 Acked Read 10 Done Sent Done false
13256 4408 Acked Read 2 Done Sent Done false
13257 4409 Acked Flush 0 Done Sent Done false
13258 4410 Acked Read 7 Done Sent Done false
13259 4411 Acked Read 2 Done Sent Done false
13260 4412 Acked Read 6 Done Sent Done false
13261 4413 Acked Write 4 Done Sent Done false
13262 4414 Acked Write 2 Done Sent Done false
13263 4415 Acked Flush 0 Done Sent Done false
13264 4416 Acked Write 2 Done Sent Done false
13265 4417 Acked Read 1 Done Sent Done false
13266 4418 Acked Read 4 Done Sent Done false
13267 4419 Acked Read 2 Done Sent Done false
13268 4420 Acked Read 3 Done Sent Done false
13269 4421 Acked Read 3 Done Sent Done false
13270 4422 Acked Read 8 Done Sent Done false
13271 4423 Acked Write 2 Done Sent Done false
13272 4424 Acked Read 8 Done Sent Done false
13273 4425 Acked Write 10 Done Sent Done false
13274 4426 Acked Write 5 Done Sent Done false
13275 4427 Acked Read 1 Done Sent Done false
13276 4428 Acked Read 8 Done Sent Done false
13277 4429 Acked Write 9 Done Sent Done false
13278 4430 Acked Write 6 Done Sent Done false
13279 4431 Acked Read 8 Done Sent Done false
13280 4432 Acked Read 1 Done Sent Done false
13281 4433 Acked Read 2 Done Sent Done false
13282 4434 Acked Write 7 Done Sent Done false
13283 4435 Acked Read 6 Done Sent Done false
13284 4436 Acked Read 9 Done Sent Done false
13285 4437 Acked Write 3 Done Sent Done false
13286 4438 Acked Flush 0 Done Sent Done false
13287 4439 Acked Write 1 Done Sent Done false
13288 4440 Acked Write 2 Done Sent Done false
13289 4441 Acked Write 7 Done Sent Done false
13290 4442 Acked Write 7 Done Sent Done false
13291 4443 Acked Flush 0 Done Sent Done false
13292 4444 Acked Write 8 Done Sent Done false
13293 4445 Acked Read 3 Done Sent Done false
13294 4446 Acked Read 2 Done Sent Done false
13295 4447 Acked Write 4 Done Sent Done false
13296 4448 Acked Write 7 Done Sent Done false
13297 4449 Acked Read 1 Done Sent Done false
13298 4450 Acked Read 9 Done Sent Done false
13299 4451 Acked Read 4 Done Sent Done false
13300 4452 Acked Write 1 Done Sent Done false
13301 4453 Acked Write 1 Done Sent Done false
13302 4454 Acked Write 10 Done Sent Done false
13303 4455 Acked Read 3 Done Sent Done false
13304 4456 Acked Read 2 Done Sent Done false
13305 4457 Acked Read 3 Done Sent Done false
13306 4458 Acked Read 9 Done Sent Done false
13307 4459 Acked Flush 0 Done Sent Done false
13308 4460 Acked Read 1 Done Sent Done false
13309 4461 Acked Read 10 Done Sent Done false
13310 4462 Acked Read 7 Done Sent Done false
13311 4463 Acked Read 4 Done Sent Done false
13312 4464 Acked Flush 0 Done Sent Done false
13313 4465 Acked Read 1 Done Sent Done false
13314 4466 Acked Write 9 Done Sent Done false
13315 4467 Acked Read 10 Done Sent Done false
13316 4468 Acked Write 9 Done Sent Done false
13317 4469 Acked Flush 0 Done Sent Done false
13318 4470 Acked Write 3 Done Sent Done false
13319 4471 Acked Read 4 Done Sent Done false
13320 4472 Acked Read 2 Done Sent Done false
13321 4473 Acked Write 1 Done Sent Done false
13322 4474 Acked Read 2 Done Sent Done false
13323 4475 Acked Read 8 Done Sent Done false
13324 4476 Acked Read 5 Done Sent Done false
13325 4477 Acked Read 1 Done Sent Done false
13326 4478 Acked Write 6 Done Sent Done false
13327 4479 Acked Write 4 Done Sent Done false
13328 4480 Acked Write 9 Done Sent Done false
13329 4481 Acked Flush 0 Done Sent Done false
13330 4482 Acked Read 4 Done Sent Done false
13331 4483 Acked Read 7 Done Sent Done false
13332 4484 Acked Read 1 Done Sent Done false
13333 4485 Acked Read 9 Done Sent Done false
13334 4486 Acked Read 7 Done Sent Done false
13335 4487 Acked Read 6 Done Sent Done false
13336 4488 Acked Write 8 Done Sent Done false
13337 4489 Acked Read 6 Done Sent Done false
13338 4490 Acked Write 9 Done Sent Done false
13339 4491 Acked Write 2 Done Sent Done false
13340 4492 Acked Read 5 Done Sent Done false
13341 4493 Acked Write 2 Done Sent Done false
13342 4494 Acked Write 9 Done Sent Done false
13343 4495 Acked Flush 0 Done Sent Done false
13344 4496 Acked Read 9 Done Sent Done false
13345 4497 Acked Read 1 Done Sent Done false
13346 4498 Acked Read 2 Done Sent Done false
13347 4499 Acked Read 5 Done Sent Done false
13348 4500 Acked Write 8 Done Sent Done false
13349 4501 Acked Read 3 Done Sent Done false
13350 4502 Acked Write 3 Done Sent Done false
13351 4503 Acked Read 2 Done Sent Done false
13352 4504 Acked Write 6 Done Sent Done false
13353 4505 Acked Flush 0 Done Sent Done false
13354 4506 Acked Read 10 Done Sent Done false
13355 4507 Acked Read 3 Done Sent Done false
13356 4508 Acked Write 5 Done Sent Done false
13357 4509 Acked Write 7 Done Sent Done false
13358 4510 Acked Write 9 Done Sent Done false
13359 4511 Acked Read 2 Done Sent Done false
13360 4512 Acked Write 5 Done Sent Done false
13361 4513 Acked Flush 0 Done Sent Done false
13362 4514 Acked Flush 0 Done Sent Done false
13363 4515 Acked Read 2 Done Sent Done false
13364 4516 Acked Read 1 Done Sent Done false
13365 4517 Acked Write 7 Done Sent Done false
13366 4518 Acked Read 4 Done Sent Done false
13367 4519 Acked Read 8 Done Sent Done false
13368 4520 Acked Write 4 Done Sent Done false
13369 4521 Acked Read 9 Done Sent Done false
13370 4522 Acked Read 5 Done Sent Done false
13371 4523 Acked Flush 0 Done Sent Done false
13372 4524 Acked Read 4 Done Sent Done false
13373 4525 Acked Flush 0 Done Sent Done false
13374 4526 Acked Read 4 Done Sent Done false
13375 4527 Acked Read 7 Done Sent Done false
13376 4528 Acked Write 2 Done Sent Done false
13377 4529 Acked Read 9 Done Sent Done false
13378 4530 Acked Write 9 Done Sent Done false
13379 4531 Acked Read 1 Done Sent Done false
13380 4532 Acked Write 6 Done Sent Done false
13381 4533 Acked Read 4 Done Sent Done false
13382 4534 Acked Read 6 Done Sent Done false
13383 4535 Acked Read 1 Done Sent Done false
13384 4536 Acked Write 1 Done Sent Done false
13385 4537 Acked Read 10 Done Sent Done false
13386 4538 Acked Read 4 Done Sent Done false
13387 4539 Acked Read 5 Done Sent Done false
13388 4540 Acked Read 5 Done Sent Done false
13389 4541 Acked Read 1 Done Sent Done false
13390 4542 Acked Flush 0 Done Sent Done false
13391 4543 Acked Read 6 Done Sent Done false
13392 4544 Acked Write 6 Done Sent Done false
13393 4545 Acked Flush 0 Done Sent Done false
13394 4546 Acked Read 9 Done Sent Done false
13395 4547 Acked Read 9 Done Sent Done false
13396 4548 Acked Read 2 Done Sent Done false
13397 4549 Acked Read 3 Done Sent Done false
13398 4550 Acked Read 10 Done Sent Done false
13399 4551 Acked Read 10 Done Sent Done false
13400 4552 Acked Write 5 Done Sent Done false
13401 4553 Acked Read 1 Done Sent Done false
13402 4554 Acked Read 7 Done Sent Done false
13403 4555 Acked Write 4 Done Sent Done false
13404 4556 Acked Write 9 Done Sent Done false
13405 4557 Acked Read 8 Done Sent Done false
13406 4558 Acked Read 4 Done Sent Done false
13407 4559 Acked Read 3 Done Sent Done false
13408 4560 Acked Write 1 Done Sent Done false
13409 4561 Acked Read 5 Done Sent Done false
13410 4562 Acked Write 7 Done Sent Done false
13411 4563 Acked Flush 0 Done Sent Done false
13412 4564 Acked Read 2 Done Sent Done false
13413 4565 Acked Write 5 Done Sent Done false
13414 4566 Acked Read 10 Done Sent Done false
13415 4567 Acked Read 3 Done Sent Done false
13416 4568 Acked Read 8 Done Sent Done false
13417 4569 Acked Write 10 Done Sent Done false
13418 4570 Acked Write 1 Done Sent Done false
13419 4571 Acked Read 2 Done Sent Done false
13420 4572 Acked Read 9 Done Sent Done false
13421 4573 Acked Write 6 Done Sent Done false
13422 4574 Acked Read 10 Done Sent Done false
13423 4575 Acked Read 9 Done Sent Done false
13424 4576 Acked Flush 0 Done Sent Done false
13425 4577 Acked Read 8 Done Sent Done false
13426 4578 Acked Flush 0 Done Sent Done false
13427 4579 Acked Read 9 Done Sent Done false
13428 4580 Acked Write 1 Done Sent Done false
13429 4581 Acked Write 7 Done Sent Done false
13430 4582 Acked Write 5 Done Sent Done false
13431 4583 Acked Read 1 Done Sent Done false
13432 4584 Acked Write 9 Done Sent Done false
13433 4585 Acked Read 1 Done Sent Done false
13434 4586 Acked Write 3 Done Sent Done false
13435 4587 Acked Flush 0 Done Sent Done false
13436 4588 Acked Read 3 Done Sent Done false
13437 4589 Acked Read 3 Done Sent Done false
13438 4590 Acked Read 9 Done Sent Done false
13439 4591 Acked Read 2 Done Sent Done false
13440 4592 Acked Read 9 Done Sent Done false
13441 4593 Acked Write 5 Done Sent Done false
13442 4594 Acked Read 2 Done Sent Done false
13443 4595 Acked Flush 0 Done Sent Done false
13444 4596 Acked Write 4 Done Sent Done false
13445 4597 Acked Read 1 Done Sent Done false
13446 4598 Acked Read 2 Done Sent Done false
13447 4599 Acked Read 8 Done Sent Done false
13448 4600 Acked Read 10 Done Sent Done false
13449 4601 Acked Write 5 Done Sent Done false
13450 4602 Acked Write 10 Done Sent Done false
13451 4603 Acked Read 5 Done Sent Done false
13452 4604 Acked Write 9 Done Sent Done false
13453 4605 Acked Read 4 Done Sent Done false
13454 4606 Acked Write 10 Done Sent Done false
13455 4607 Acked Read 5 Done Sent Done false
13456 4608 Acked Read 1 Done Sent Done false
13457 4609 Acked Read 4 Done Sent Done false
13458 4610 Acked Read 7 Done Sent Done false
13459 4611 Acked Write 8 Done Sent Done false
13460 4612 Acked Read 7 Done Sent Done false
13461 4613 Acked Read 1 Done Sent Done false
13462 4614 Acked Read 2 Done Sent Done false
13463 4615 Acked Flush 0 Done Sent Done false
13464 4616 Acked Read 4 Done Sent Done false
13465 4617 Acked Write 2 Done Sent Done false
13466 4618 Acked Write 10 Done Sent Done false
13467 4619 Acked Flush 0 Done Sent Done false
13468 4620 Acked Read 7 Done Sent Done false
13469 4621 Acked Read 4 Done Sent Done false
13470 STATES DS:0 DS:1 DS:2 TOTAL
13471 Sent 0 301 0 301
13472 Done 301 0 301 602
13473 Skipped 0 0 0 0
13474 Error 0 0 0 0
13475 Last Flush: 4619 4320 4619
13476 Downstairs last five completed: 4320 4319 4318 4317 4316
13477 Upstairs last five completed: 4621 4620 4619 4618 4617
13478 CLIENT: Up:0 ds:301 act:2
13479 [8820] got start action
13480 [8820] I am going to restart
13481 Starting downstairs at port 8820
13482 Make output file at "/tmp/dsc/downstairs-8820.txt"
13483 [8820][1] reports Starting
13484 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1217)
13485 [8820][1] reports Running
13486 ----------------------------------------------------------------
13487 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13488 States: Active Offline Active
13489 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13490 4321 Acked Read 6 Done Sent Done false
13491 4322 Acked Read 6 Done Sent Done false
13492 4323 Acked Read 10 Done Sent Done false
13493 4324 Acked Flush 0 Done Sent Done false
13494 4325 Acked Read 5 Done Sent Done false
13495 4326 Acked Read 7 Done Sent Done false
13496 4327 Acked Read 2 Done Sent Done false
13497 4328 Acked Read 1 Done Sent Done false
13498 4329 Acked Write 9 Done Sent Done false
13499 4330 Acked Read 10 Done Sent Done false
13500 4331 Acked Write 4 Done Sent Done false
13501 4332 Acked Read 3 Done Sent Done false
13502 4333 Acked Write 6 Done Sent Done false
13503 4334 Acked Read 1 Done Sent Done false
13504 4335 Acked Write 4 Done Sent Done false
13505 4336 Acked Read 5 Done Sent Done false
13506 4337 Acked Flush 0 Done Sent Done false
13507 4338 Acked Read 5 Done Sent Done false
13508 4339 Acked Read 5 Done Sent Done false
13509 4340 Acked Flush 0 Done Sent Done false
13510 4341 Acked Read 8 Done Sent Done false
13511 4342 Acked Flush 0 Done Sent Done false
13512 4343 Acked Write 6 Done Sent Done false
13513 4344 Acked Write 5 Done Sent Done false
13514 4345 Acked Write 7 Done Sent Done false
13515 4346 Acked Write 7 Done Sent Done false
13516 4347 Acked Read 6 Done Sent Done false
13517 4348 Acked Write 4 Done Sent Done false
13518 4349 Acked Read 8 Done Sent Done false
13519 4350 Acked Read 10 Done Sent Done false
13520 4351 Acked Read 9 Done Sent Done false
13521 4352 Acked Write 8 Done Sent Done false
13522 4353 Acked Write 2 Done Sent Done false
13523 4354 Acked Write 5 Done Sent Done false
13524 4355 Acked Write 9 Done Sent Done false
13525 4356 Acked Write 4 Done Sent Done false
13526 4357 Acked Write 7 Done Sent Done false
13527 4358 Acked Read 3 Done Sent Done false
13528 4359 Acked Read 3 Done Sent Done false
13529 4360 Acked Write 10 Done Sent Done false
13530 4361 Acked Write 2 Done Sent Done false
13531 4362 Acked Write 4 Done Sent Done false
13532 4363 Acked Write 10 Done Sent Done false
13533 4364 Acked Read 8 Done Sent Done false
13534 4365 Acked Flush 0 Done Sent Done false
13535 4366 Acked Write 9 Done Sent Done false
13536 4367 Acked Read 5 Done Sent Done false
13537 4368 Acked Write 6 Done Sent Done false
13538 4369 Acked Read 4 Done Sent Done false
13539 4370 Acked Read 8 Done Sent Done false
13540 4371 Acked Write 4 Done Sent Done false
13541 4372 Acked Read 4 Done Sent Done false
13542 4373 Acked Read 7 Done Sent Done false
13543 4374 Acked Read 1 Done Sent Done false
13544 4375 Acked Read 5 Done Sent Done false
13545 4376 Acked Write 9 Done Sent Done false
13546 4377 Acked Read 1 Done Sent Done false
13547 4378 Acked Write 5 Done Sent Done false
13548 4379 Acked Write 7 Done Sent Done false
13549 4380 Acked Read 2 Done Sent Done false
13550 4381 Acked Read 2 Done Sent Done false
13551 4382 Acked Read 5 Done Sent Done false
13552 4383 Acked Write 1 Done Sent Done false
13553 4384 Acked Write 4 Done Sent Done false
13554 4385 Acked Read 4 Done Sent Done false
13555 4386 Acked Flush 0 Done Sent Done false
13556 4387 Acked Read 5 Done Sent Done false
13557 4388 Acked Write 6 Done Sent Done false
13558 4389 Acked Read 7 Done Sent Done false
13559 4390 Acked Read 10 Done Sent Done false
13560 4391 Acked Flush 0 Done Sent Done false
13561 4392 Acked Flush 0 Done Sent Done false
13562 4393 Acked Write 8 Done Sent Done false
13563 4394 Acked Read 6 Done Sent Done false
13564 4395 Acked Read 9 Done Sent Done false
13565 4396 Acked Read 6 Done Sent Done false
13566 4397 Acked Read 7 Done Sent Done false
13567 4398 Acked Write 3 Done Sent Done false
13568 4399 Acked Flush 0 Done Sent Done false
13569 4400 Acked Read 10 Done Sent Done false
13570 4401 Acked Read 9 Done Sent Done false
13571 4402 Acked Write 4 Done Sent Done false
13572 4403 Acked Write 9 Done Sent Done false
13573 4404 Acked Read 7 Done Sent Done false
13574 4405 Acked Write 4 Done Sent Done false
13575 4406 Acked Read 1 Done Sent Done false
13576 4407 Acked Read 10 Done Sent Done false
13577 4408 Acked Read 2 Done Sent Done false
13578 4409 Acked Flush 0 Done Sent Done false
13579 4410 Acked Read 7 Done Sent Done false
13580 4411 Acked Read 2 Done Sent Done false
13581 4412 Acked Read 6 Done Sent Done false
13582 4413 Acked Write 4 Done Sent Done false
13583 4414 Acked Write 2 Done Sent Done false
13584 4415 Acked Flush 0 Done Sent Done false
13585 4416 Acked Write 2 Done Sent Done false
13586 4417 Acked Read 1 Done Sent Done false
13587 4418 Acked Read 4 Done Sent Done false
13588 4419 Acked Read 2 Done Sent Done false
13589 4420 Acked Read 3 Done Sent Done false
13590 4421 Acked Read 3 Done Sent Done false
13591 4422 Acked Read 8 Done Sent Done false
13592 4423 Acked Write 2 Done Sent Done false
13593 4424 Acked Read 8 Done Sent Done false
13594 4425 Acked Write 10 Done Sent Done false
13595 4426 Acked Write 5 Done Sent Done false
13596 4427 Acked Read 1 Done Sent Done false
13597 4428 Acked Read 8 Done Sent Done false
13598 4429 Acked Write 9 Done Sent Done false
13599 4430 Acked Write 6 Done Sent Done false
13600 4431 Acked Read 8 Done Sent Done false
13601 4432 Acked Read 1 Done Sent Done false
13602 4433 Acked Read 2 Done Sent Done false
13603 4434 Acked Write 7 Done Sent Done false
13604 4435 Acked Read 6 Done Sent Done false
13605 4436 Acked Read 9 Done Sent Done false
13606 4437 Acked Write 3 Done Sent Done false
13607 4438 Acked Flush 0 Done Sent Done false
13608 4439 Acked Write 1 Done Sent Done false
13609 4440 Acked Write 2 Done Sent Done false
13610 4441 Acked Write 7 Done Sent Done false
13611 4442 Acked Write 7 Done Sent Done false
13612 4443 Acked Flush 0 Done Sent Done false
13613 4444 Acked Write 8 Done Sent Done false
13614 4445 Acked Read 3 Done Sent Done false
13615 4446 Acked Read 2 Done Sent Done false
13616 4447 Acked Write 4 Done Sent Done false
13617 4448 Acked Write 7 Done Sent Done false
13618 4449 Acked Read 1 Done Sent Done false
13619 4450 Acked Read 9 Done Sent Done false
13620 4451 Acked Read 4 Done Sent Done false
13621 4452 Acked Write 1 Done Sent Done false
13622 4453 Acked Write 1 Done Sent Done false
13623 4454 Acked Write 10 Done Sent Done false
13624 4455 Acked Read 3 Done Sent Done false
13625 4456 Acked Read 2 Done Sent Done false
13626 4457 Acked Read 3 Done Sent Done false
13627 4458 Acked Read 9 Done Sent Done false
13628 4459 Acked Flush 0 Done Sent Done false
13629 4460 Acked Read 1 Done Sent Done false
13630 4461 Acked Read 10 Done Sent Done false
13631 4462 Acked Read 7 Done Sent Done false
13632 4463 Acked Read 4 Done Sent Done false
13633 4464 Acked Flush 0 Done Sent Done false
13634 4465 Acked Read 1 Done Sent Done false
13635 4466 Acked Write 9 Done Sent Done false
13636 4467 Acked Read 10 Done Sent Done false
13637 4468 Acked Write 9 Done Sent Done false
13638 4469 Acked Flush 0 Done Sent Done false
13639 4470 Acked Write 3 Done Sent Done false
13640 4471 Acked Read 4 Done Sent Done false
13641 4472 Acked Read 2 Done Sent Done false
13642 4473 Acked Write 1 Done Sent Done false
13643 4474 Acked Read 2 Done Sent Done false
13644 4475 Acked Read 8 Done Sent Done false
13645 4476 Acked Read 5 Done Sent Done false
13646 4477 Acked Read 1 Done Sent Done false
13647 4478 Acked Write 6 Done Sent Done false
13648 4479 Acked Write 4 Done Sent Done false
13649 4480 Acked Write 9 Done Sent Done false
13650 4481 Acked Flush 0 Done Sent Done false
13651 4482 Acked Read 4 Done Sent Done false
13652 4483 Acked Read 7 Done Sent Done false
13653 4484 Acked Read 1 Done Sent Done false
13654 4485 Acked Read 9 Done Sent Done false
13655 4486 Acked Read 7 Done Sent Done false
13656 4487 Acked Read 6 Done Sent Done false
13657 4488 Acked Write 8 Done Sent Done false
13658 4489 Acked Read 6 Done Sent Done false
13659 4490 Acked Write 9 Done Sent Done false
13660 4491 Acked Write 2 Done Sent Done false
13661 4492 Acked Read 5 Done Sent Done false
13662 4493 Acked Write 2 Done Sent Done false
13663 4494 Acked Write 9 Done Sent Done false
13664 4495 Acked Flush 0 Done Sent Done false
13665 4496 Acked Read 9 Done Sent Done false
13666 4497 Acked Read 1 Done Sent Done false
13667 4498 Acked Read 2 Done Sent Done false
13668 4499 Acked Read 5 Done Sent Done false
13669 4500 Acked Write 8 Done Sent Done false
13670 4501 Acked Read 3 Done Sent Done false
13671 4502 Acked Write 3 Done Sent Done false
13672 4503 Acked Read 2 Done Sent Done false
13673 4504 Acked Write 6 Done Sent Done false
13674 4505 Acked Flush 0 Done Sent Done false
13675 4506 Acked Read 10 Done Sent Done false
13676 4507 Acked Read 3 Done Sent Done false
13677 4508 Acked Write 5 Done Sent Done false
13678 4509 Acked Write 7 Done Sent Done false
13679 4510 Acked Write 9 Done Sent Done false
13680 4511 Acked Read 2 Done Sent Done false
13681 4512 Acked Write 5 Done Sent Done false
13682 4513 Acked Flush 0 Done Sent Done false
13683 4514 Acked Flush 0 Done Sent Done false
13684 4515 Acked Read 2 Done Sent Done false
13685 4516 Acked Read 1 Done Sent Done false
13686 4517 Acked Write 7 Done Sent Done false
13687 4518 Acked Read 4 Done Sent Done false
13688 4519 Acked Read 8 Done Sent Done false
13689 4520 Acked Write 4 Done Sent Done false
13690 4521 Acked Read 9 Done Sent Done false
13691 4522 Acked Read 5 Done Sent Done false
13692 4523 Acked Flush 0 Done Sent Done false
13693 4524 Acked Read 4 Done Sent Done false
13694 4525 Acked Flush 0 Done Sent Done false
13695 4526 Acked Read 4 Done Sent Done false
13696 4527 Acked Read 7 Done Sent Done false
13697 4528 Acked Write 2 Done Sent Done false
13698 4529 Acked Read 9 Done Sent Done false
13699 4530 Acked Write 9 Done Sent Done false
13700 4531 Acked Read 1 Done Sent Done false
13701 4532 Acked Write 6 Done Sent Done false
13702 4533 Acked Read 4 Done Sent Done false
13703 4534 Acked Read 6 Done Sent Done false
13704 4535 Acked Read 1 Done Sent Done false
13705 4536 Acked Write 1 Done Sent Done false
13706 4537 Acked Read 10 Done Sent Done false
13707 4538 Acked Read 4 Done Sent Done false
13708 4539 Acked Read 5 Done Sent Done false
13709 4540 Acked Read 5 Done Sent Done false
13710 4541 Acked Read 1 Done Sent Done false
13711 4542 Acked Flush 0 Done Sent Done false
13712 4543 Acked Read 6 Done Sent Done false
13713 4544 Acked Write 6 Done Sent Done false
13714 4545 Acked Flush 0 Done Sent Done false
13715 4546 Acked Read 9 Done Sent Done false
13716 4547 Acked Read 9 Done Sent Done false
13717 4548 Acked Read 2 Done Sent Done false
13718 4549 Acked Read 3 Done Sent Done false
13719 4550 Acked Read 10 Done Sent Done false
13720 4551 Acked Read 10 Done Sent Done false
13721 4552 Acked Write 5 Done Sent Done false
13722 4553 Acked Read 1 Done Sent Done false
13723 4554 Acked Read 7 Done Sent Done false
13724 4555 Acked Write 4 Done Sent Done false
13725 4556 Acked Write 9 Done Sent Done false
13726 4557 Acked Read 8 Done Sent Done false
13727 4558 Acked Read 4 Done Sent Done false
13728 4559 Acked Read 3 Done Sent Done false
13729 4560 Acked Write 1 Done Sent Done false
13730 4561 Acked Read 5 Done Sent Done false
13731 4562 Acked Write 7 Done Sent Done false
13732 4563 Acked Flush 0 Done Sent Done false
13733 4564 Acked Read 2 Done Sent Done false
13734 4565 Acked Write 5 Done Sent Done false
13735 4566 Acked Read 10 Done Sent Done false
13736 4567 Acked Read 3 Done Sent Done false
13737 4568 Acked Read 8 Done Sent Done false
13738 4569 Acked Write 10 Done Sent Done false
13739 4570 Acked Write 1 Done Sent Done false
13740 4571 Acked Read 2 Done Sent Done false
13741 4572 Acked Read 9 Done Sent Done false
13742 4573 Acked Write 6 Done Sent Done false
13743 4574 Acked Read 10 Done Sent Done false
13744 4575 Acked Read 9 Done Sent Done false
13745 4576 Acked Flush 0 Done Sent Done false
13746 4577 Acked Read 8 Done Sent Done false
13747 4578 Acked Flush 0 Done Sent Done false
13748 4579 Acked Read 9 Done Sent Done false
13749 4580 Acked Write 1 Done Sent Done false
13750 4581 Acked Write 7 Done Sent Done false
13751 4582 Acked Write 5 Done Sent Done false
13752 4583 Acked Read 1 Done Sent Done false
13753 4584 Acked Write 9 Done Sent Done false
13754 4585 Acked Read 1 Done Sent Done false
13755 4586 Acked Write 3 Done Sent Done false
13756 4587 Acked Flush 0 Done Sent Done false
13757 4588 Acked Read 3 Done Sent Done false
13758 4589 Acked Read 3 Done Sent Done false
13759 4590 Acked Read 9 Done Sent Done false
13760 4591 Acked Read 2 Done Sent Done false
13761 4592 Acked Read 9 Done Sent Done false
13762 4593 Acked Write 5 Done Sent Done false
13763 4594 Acked Read 2 Done Sent Done false
13764 4595 Acked Flush 0 Done Sent Done false
13765 4596 Acked Write 4 Done Sent Done false
13766 4597 Acked Read 1 Done Sent Done false
13767 4598 Acked Read 2 Done Sent Done false
13768 4599 Acked Read 8 Done Sent Done false
13769 4600 Acked Read 10 Done Sent Done false
13770 4601 Acked Write 5 Done Sent Done false
13771 4602 Acked Write 10 Done Sent Done false
13772 4603 Acked Read 5 Done Sent Done false
13773 4604 Acked Write 9 Done Sent Done false
13774 4605 Acked Read 4 Done Sent Done false
13775 4606 Acked Write 10 Done Sent Done false
13776 4607 Acked Read 5 Done Sent Done false
13777 4608 Acked Read 1 Done Sent Done false
13778 4609 Acked Read 4 Done Sent Done false
13779 4610 Acked Read 7 Done Sent Done false
13780 4611 Acked Write 8 Done Sent Done false
13781 4612 Acked Read 7 Done Sent Done false
13782 4613 Acked Read 1 Done Sent Done false
13783 4614 Acked Read 2 Done Sent Done false
13784 4615 Acked Flush 0 Done Sent Done false
13785 4616 Acked Read 4 Done Sent Done false
13786 4617 Acked Write 2 Done Sent Done false
13787 4618 Acked Write 10 Done Sent Done false
13788 4619 Acked Flush 0 Done Sent Done false
13789 4620 Acked Read 7 Done Sent Done false
13790 4621 Acked Read 4 Done Sent Done false
13791 4622 Acked Flush 0 Done Sent Done false
13792 STATES DS:0 DS:1 DS:2 TOTAL
13793 Sent 0 302 0 302
13794 Done 302 0 302 604
13795 Skipped 0 0 0 0
13796 Error 0 0 0 0
13797 Last Flush: 4622 4320 4622
13798 Downstairs last five completed: 4320 4319 4318 4317 4316
13799 Upstairs last five completed: 4622 4621 4620 4619 4618
13800 CLIENT: Up:0 ds:302 act:2
138012025-01-03T08:33:46.310ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138022025-01-03T08:33:46.310ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35710) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138032025-01-03T08:33:46.311ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138042025-01-03T08:33:46.312ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138052025-01-03T08:33:46.312ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138062025-01-03T08:33:46.312ZINFOcrucible: send last flush ID to this DS: 4320 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138072025-01-03T08:33:46.312ZINFOcrucible: Replied this last flush ID: 4320 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138082025-01-03T08:33:46.312ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138092025-01-03T08:33:46.312ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138102025-01-03T08:33:46.312ZINFOcrucible: [1] client re-new 302 jobs since flush 4320 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138112025-01-03T08:33:46.313ZINFOcrucible: [1] Replayed 302 jobs since flush: 4320 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
13812 ----------------------------------------------------------------
13813 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13814 Downstairs last five completed: 4622 4621 4620 4619 4618
13815 Upstairs last five completed: 4622 4621 4620 4619 4618
13816 CLIENT: Up:0 ds:0 act:3
13817 Replay: All jobs finished
13818 Main task has work to do, go find it
13819 got dsc Stop(1)
13820 stop 1
13821 [8820] Got stop action so:false kr:false
13822 Jan 03 08:33:48.563 INFO request completed, latency_us: 341, response_code: 204, uri: /stop/cid/1, method: POST, req_id: dacd2e00-87ea-49f7-9c62-2018bf73a67c, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
138232025-01-03T08:33:48.563ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138242025-01-03T08:33:48.563ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
138252025-01-03T08:33:48.563ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
13826 Jan 03 08:33:48.563 INFO request completed, latency_us: 130, response_code: 200, uri: /state/cid/1, method: GET, req_id: c5541a37-090d-495d-a0ab-9eb6fecb115d, remote_addr: 127.0.0.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:33:48.563989303Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
13827 1:49669, local_addr: 127.0.{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:33:48.564012593Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
13828 0.1:9998
138292025-01-03T08:33:48.564ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
13830 [8820] Exited with: ExitStatus(unix_wait_status(9))
13831 [8820][1] reports Exit
13832 Jan 03 08:33:52.566 INFO request completed, latency_us: 198, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4ae6571f-ed43-4e14-ba5e-b8c79bd1b48d, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
13833 001/300 Read block 456 len 40960
13834 002/300 Write block 2345 len 36864 data: 2 2 2 2 2 2 2 2 2
13835 003/300 Read block 2969 len 16384
13836 004/300 Read block 3429 len 40960
13837 005/300 Write block 868 len 28672 data: 2 2 2 2 4 4 3
13838 006/300 Write block 4143 len 16384 data: 5 5 4 3
13839 007/300 Read block 3157 len 4096
13840 008/300 Read block 1575 len 8192
13841 009/300 Read block 759 len 28672
13842 010/300 Read block 1379 len 32768
13843 011/300 Write block 4529 len 40960 data: 3 3 3 3 3 4 5 3 3 4
13844 012/300 Flush
13845 013/300 Write block 140 len 4096 data: 5
13846 014/300 Flush
13847 015/300 Read block 1762 len 40960
13848 016/300 Flush
13849 017/300 Flush
13850 018/300 Read block 623 len 12288
13851 019/300 Flush
13852 020/300 Flush
13853 021/300 Write block 4630 len 16384 data: 3 3 3 4
13854 022/300 Write block 1827 len 28672 data: 4 4 4 5 3 2 2
13855 023/300 Read block 3841 len 24576
13856 024/300 Read block 1546 len 36864
13857 025/300 Read block 1450 len 12288
13858 026/300 Write block 68 len 28672 data: 5 6 5 4 5 4 4
13859 027/300 Flush
13860 028/300 Write block 4967 len 8192 data: 3 3
13861 029/300 Flush
13862 030/300 Write block 4145 len 28672 data: 5 4 3 3 3 3 2
13863 031/300 Read block 2887 len 36864
13864 032/300 Read block 2468 len 16384
13865 033/300 Write block 229 len 36864 data: 3 3 3 3 2 2 2 4 4
13866 034/300 Write block 4238 len 24576 data: 2 2 3 3 3 3
13867 035/300 Read block 1563 len 40960
13868 036/300 Write block 2221 len 32768 data: 3 3 3 3 3 3 2 2
13869 037/300 Write block 4352 len 4096 data: 2
13870 038/300 Read block 2885 len 12288
13871 039/300 Flush
13872 040/300 Write block 2114 len 32768 data: 5 5 5 4 3 3 3 3
13873 041/300 Write block 2890 len 32768 data: 2 2 2 2 2 2 2 3
13874 042/300 Read block 4095 len 20480
13875 043/300 Read block 4776 len 24576
13876 044/300 Read block 2147 len 40960
13877 045/300 Read block 3334 len 20480
13878 046/300 Write block 3601 len 32768 data: 3 3 3 4 7 4 4 3
13879 047/300 Write block 3231 len 40960 data: 5 6 6 6 6 5 5 4 4 3
13880 048/300 Read block 2564 len 12288
13881 049/300 Write block 357 len 16384 data: 3 3 2 2
13882 050/300 Write block 3596 len 36864 data: 3 3 3 3 3 4 4 4 5
13883 051/300 Read block 4538 len 36864
13884 052/300 Write block 3782 len 28672 data: 3 3 3 3 5 4 5
13885 053/300 Write block 2935 len 4096 data: 3
13886 054/300 Read block 2267 len 24576
13887 055/300 Read block 2617 len 12288
13888 056/300 Read block 4507 len 24576
13889 057/300 Read block 4248 len 28672
13890 058/300 Write block 342 len 24576 data: 5 5 5 7 8 8
13891 059/300 Read block 660 len 12288
13892 060/300 Write block 3485 len 32768 data: 2 2 2 2 2 2 2 3
13893 061/300 Read block 2317 len 28672
13894 062/300 Write block 457 len 12288 data: 3 5 5
13895 063/300 Flush
13896 064/300 Read block 4069 len 12288
13897 065/300 Write block 1288 len 36864 data: 3 3 4 5 4 4 3 3 3
13898 066/300 Write block 938 len 16384 data: 6 3 2 2
13899 067/300 Flush
13900 068/300 Read block 3498 len 4096
13901 069/300 Read block 1511 len 36864
13902 070/300 Read block 4019 len 28672
13903 071/300 Flush
13904 072/300 Read block 3392 len 32768
13905 073/300 Read block 3087 len 8192
13906 074/300 Read block 4668 len 16384
13907 075/300 Read block 594 len 40960
13908 076/300 Write block 909 len 24576 data: 4 4 3 5 5 5
13909 077/300 Write block 3656 len 28672 data: 4 4 4 2 3 3 4
13910 078/300 Read block 4363 len 24576
13911 079/300 Read block 2411 len 24576
13912 080/300 Read block 837 len 28672
13913 081/300 Write block 3999 len 40960 data: 5 6 7 6 6 6 6 5 4 4
13914 082/300 Write block 2924 len 16384 data: 3 3 3 3
13915 083/300 Write block 3576 len 8192 data: 4 3
13916 084/300 Flush
13917 085/300 Read block 3799 len 8192
13918 086/300 Read block 2560 len 12288
13919 087/300 Read block 3567 len 40960
13920 088/300 Write block 4262 len 20480 data: 2 3 3 3 4
13921 089/300 Read block 180 len 24576
13922 090/300 Write block 4119 len 24576 data: 7 4 5 5 5 5
13923 091/300 Write block 3224 len 20480 data: 5 5 3 3 4
13924 092/300 Read block 2527 len 36864
13925 093/300 Read block 1457 len 24576
13926 094/300 Read block 228 len 12288
13927 095/300 Write block 4597 len 36864 data: 3 3 4 3 2 2 3 3 3
13928 096/300 Write block 2697 len 4096 data: 6
13929 097/300 Write block 2227 len 12288 data: 3 3 3
13930 098/300 Write block 1848 len 20480 data: 3 3 3 3 3
13931 099/300 Read block 4811 len 28672
13932 100/300 Read block 3128 len 36864
13933 101/300 Read block 2519 len 40960
13934 102/300 Read block 2095 len 40960
13935 103/300 Write block 2163 len 16384 data: 4 4 4 3
13936 104/300 Read block 2527 len 20480
13937 105/300 Read block 4455 len 20480
13938 106/300 Write block 380 len 24576 data: 2 2 2 3 3 3
13939 107/300 Write block 1734 len 12288 data: 3 5 7
13940 108/300 Read block 3387 len 8192
13941 109/300 Write block 4179 len 40960 data: 3 3 2 2 3 3 2 2 2 2
13942 110/300 Write block 3236 len 32768 data: 6 6 5 5 4 4 4 4
13943 111/300 Flush
13944 112/300 Write block 408 len 40960 data: 5 5 5 4 4 5 3 3 3 4
13945 113/300 Write block 2477 len 8192 data: 3 3
13946 114/300 Write block 462 len 24576 data: 5 5 4 4 5 5
13947 115/300 Read block 1551 len 20480
13948 116/300 Read block 3612 len 24576
13949 117/300 Write block 4237 len 4096 data: 2
13950 118/300 Read block 1903 len 16384
13951 119/300 Write block 1593 len 4096 data: 5
13952 120/300 Read block 3157 len 40960
13953 121/300 Read block 4327 len 28672
13954 122/300 Write block 4845 len 8192 data: 4 4
13955 123/300 Write block 2392 len 20480 data: 3 3 3 2 2
13956 124/300 Read block 1723 len 8192
13957 125/300 Write block 4263 len 16384 data: 4 4 4 5
13958 126/300 Write block 2731 len 16384 data: 5 6 5 5
13959 127/300 Read block 1194 len 4096
13960 128/300 Read block 4696 len 16384
13961 129/300 Write block 1507 len 40960 data: 3 3 2 2 3 3 2 2 2 2
13962 130/300 Read block 207 len 12288
13963 131/300 Flush
13964 132/300 Write block 4198 len 16384 data: 5 6 6 7
13965 133/300 Write block 840 len 36864 data: 5 5 5 7 7 6 6 4 4
13966 134/300 Read block 4230 len 12288
13967 135/300 Read block 4718 len 40960
13968 136/300 Read block 2682 len 16384
13969 137/300 Read block 66 len 40960
13970 138/300 Write block 2407 len 24576 data: 3 3 3 2 2 2
13971 139/300 Read block 449 len 40960
13972 140/300 Flush
13973 141/300 Write block 2015 len 28672 data: 3 3 4 4 3 3 3
13974 142/300 Read block 1143 len 20480
13975 143/300 Read block 1144 len 28672
13976 144/300 Write block 1472 len 16384 data: 3 3 2 2
13977 145/300 Read block 1860 len 12288
13978 146/300 Read block 3364 len 32768
13979 147/300 Read block 4796 len 8192
13980 148/300 Flush
13981 149/300 Write block 282 len 4096 data: 2
13982 150/300 Write block 3776 len 8192 data: 2 2
13983 151/300 Read block 2447 len 40960
13984 152/300 Read block 3745 len 24576
13985 153/300 Read block 4043 len 12288
13986 154/300 Read block 4681 len 8192
13987 155/300 Write block 2617 len 12288 data: 2 2 3
13988 156/300 Write block 1802 len 40960 data: 4 3 3 3 3 3 3 3 3 3
13989 157/300 Write block 455 len 4096 data: 2
13990 158/300 Flush
13991 159/300 Read block 4607 len 28672
13992 160/300 Read block 3860 len 40960
13993 161/300 Read block 3605 len 40960
13994 162/300 Read block 497 len 36864
13995 163/300 Write block 2915 len 40960 data: 3 3 3 3 4 3 3 3 3 4
13996 164/300 Read block 1543 len 40960
13997 165/300 Read block 3791 len 40960
13998 166/300 Flush
13999 167/300 Write block 2879 len 16384 data: 3 3 3 3
14000 168/300 Write block 1267 len 40960 data: 4 4 4 3 4 4 4 6 5 5
14001 169/300 Read block 4736 len 40960
14002 170/300 Write block 1153 len 32768 data: 2 3 3 3 3 3 3 4
14003 171/300 Read block 3667 len 20480
14004 172/300 Read block 3966 len 32768
14005 173/300 Read block 4511 len 40960
14006 174/300 Read block 3447 len 40960
14007 175/300 Flush
14008 176/300 Write block 1510 len 8192 data: 3 4
14009 177/300 Write block 2563 len 32768 data: 4 4 3 2 3 6 6 5
14010 178/300 Flush
14011 179/300 Flush
14012 180/300 Write block 2988 len 32768 data: 2 2 2 2 2 2 2 2
14013 181/300 Read block 3364 len 8192
14014 182/300 Write block 318 len 4096 data: 4
14015 183/300 Write block 3799 len 36864 data: 3 3 3 4 4 5 4 3 2
14016 184/300 Read block 20 len 32768
14017 185/300 Flush
14018 186/300 Read block 2532 len 8192
14019 187/300 Write block 150 len 12288 data: 3 2 2
14020 188/300 Write block 3309 len 32768 data: 4 4 4 4 3 3 3 3
14021 189/300 Read block 4174 len 4096
14022 190/300 Read block 969 len 36864
14023 191/300 Write block 1393 len 36864 data: 4 4 4 6 4 4 4 3 2
14024 192/300 Read block 2956 len 12288
14025 193/300 Write block 4523 len 20480 data: 3 3 3 2 2
14026 194/300 Read block 1918 len 24576
14027 195/300 Write block 1283 len 8192 data: 4 4
14028 196/300 Read block 2211 len 28672
14029 197/300 Write block 2565 len 36864 data: 4 3 4 7 7 6 5 5 3
14030 198/300 Read block 2863 len 4096
14031 199/300 Write block 4166 len 12288 data: 5 5 5
14032 200/300 Read block 3877 len 20480
14033 201/300 Write block 4261 len 36864 data: 3 3 5 5 5 6 4 4 4
14034 202/300 Write block 4073 len 36864 data: 4 4 4 4 4 3 3 4 4
14035 203/300 Read block 4016 len 16384
14036 204/300 Write block 658 len 16384 data: 7 6 5 3
14037 205/300 Read block 263 len 36864
14038 206/300 Read block 2572 len 20480
14039 207/300 Write block 936 len 36864 data: 6 6 7 4 3 3 3 3 3
14040 208/300 Read block 2626 len 40960
14041 209/300 Write block 4476 len 40960 data: 3 3 4 4 3 3 3 3 3 3
14042 210/300 Write block 3821 len 8192 data: 5 5
14043 211/300 Write block 4381 len 28672 data: 2 2 2 2 3 4 4
14044 212/300 Read block 3590 len 20480
14045 213/300 Read block 4849 len 20480
14046 214/300 Write block 366 len 28672 data: 5 5 5 5 5 4 4
14047 215/300 Read block 2789 len 28672
14048 216/300 Write block 790 len 12288 data: 7 5 5
14049 217/300 Flush
14050 218/300 Write block 2021 len 4096 data: 4
14051 219/300 Read block 3834 len 16384
14052 220/300 Flush
14053 221/300 Read block 4710 len 24576
14054 222/300 Read block 499 len 24576
14055 223/300 Write block 698 len 28672 data: 4 3 2 2 2 2 3
14056 224/300 Read block 4853 len 32768
14057 225/300 Write block 854 len 32768 data: 4 3 3 4 3 3 3 3
14058 226/300 Read block 3957 len 24576
14059 227/300 Read block 4561 len 28672
14060 228/300 Flush
14061 229/300 Flush
14062 230/300 Read block 4742 len 20480
14063 231/300 Read block 1566 len 28672
14064 232/300 Write block 626 len 28672 data: 3 4 3 3 3 4 4
14065 233/300 Read block 1234 len 24576
14066 234/300 Write block 4681 len 40960 data: 5 4 4 4 4 3 4 4 3 5
14067 235/300 Write block 1893 len 4096 data: 3
14068 236/300 Read block 3556 len 16384
14069 237/300 Read block 822 len 16384
14070 238/300 Read block 3548 len 28672
14071 239/300 Write block 525 len 12288 data: 2 2 3
14072 240/300 Read block 4960 len 8192
14073 241/300 Read block 3232 len 8192
14074 242/300 Read block 1389 len 28672
14075 243/300 Write block 2365 len 40960 data: 4 4 4 3 3 5 5 4 4 4
14076 244/300 Read block 40 len 12288
14077 245/300 Write block 610 len 4096 data: 3
14078 246/300 Read block 1230 len 20480
14079 247/300 Write block 4168 len 24576 data: 6 5 5 4 5 5
14080 248/300 Read block 3301 len 40960
14081 249/300 Flush
14082 250/300 Write block 530 len 40960 data: 4 4 2 2 3 4 4 3 4 5
14083 251/300 Flush
14084 252/300 Read block 1928 len 36864
14085 253/300 Read block 2359 len 8192
14086 254/300 Read block 1552 len 16384
14087 255/300 Read block 413 len 4096
14088 256/300 Read block 1955 len 4096
14089 257/300 Read block 2337 len 40960
14090 258/300 Write block 3021 len 8192 data: 4 4
14091 259/300 Write block 3143 len 28672 data: 3 3 4 4 4 3 3
14092 260/300 Read block 1366 len 24576
14093 261/300 Read block 673 len 24576
14094 262/300 Write block 623 len 8192 data: 3 3
14095 263/300 Read block 2175 len 36864
14096 264/300 Write block 1510 len 36864 data: 4 5 4 3 3 3 3 3 3
14097 265/300 Read block 1654 len 32768
14098 266/300 Write block 2742 len 32768 data: 3 5 4 4 4 4 5 6
14099 267/300 Read block 1686 len 8192
14100 268/300 Read block 627 len 28672
14101 269/300 Read block 3446 len 40960
14102 270/300 Flush
14103 271/300 Read block 1920 len 8192
14104 272/300 Read block 4199 len 4096
14105 273/300 Read block 1949 len 36864
14106 274/300 Read block 2437 len 24576
14107 275/300 Write block 2530 len 20480 data: 2 2 3 3 3
14108 276/300 Read block 4332 len 20480
14109 277/300 Read block 3930 len 20480
14110 278/300 Read block 4263 len 24576
14111 279/300 Read block 1481 len 4096
14112 280/300 Write block 1244 len 12288 data: 2 2 2
14113 281/300 Read block 1865 len 40960
14114 282/300 Read block 62 len 32768
14115 283/300 Write block 4657 len 24576 data: 2 2 2 2 4 5
14116 284/300 Read block 91 len 36864
14117 285/300 Read block 4198 len 36864
14118 286/300 Write block 3695 len 8192 data: 3 4
14119 287/300 Write block 4304 len 16384 data: 4 4 4 3
14120 288/300 Write block 887 len 40960 data: 4 5 4 3 3 3 3 4 5 5
14121 289/300 Write block 2128 len 12288 data: 3 3 4
14122 290/300 Read block 874 len 12288
14123 291/300 Read block 4439 len 8192
14124 292/300 Read block 3223 len 28672
14125 293/300 Read block 3490 len 28672
14126 294/300 Read block 2869 len 28672
14127 295/300 Write block 3962 len 8192 data: 3 3
14128 296/300 Flush
14129 297/300 Read block 1625 len 28672
14130 298/300 Read block 2799 len 8192
14131 299/300 Write block 628 len 16384 data: 4 4 4 5
14132 300/300 Read block 1997 len 16384
14133 301/300 Read block 4495 len 20480
14134 Main task has work to do, go find it
14135 got dsc Start(1)
14136 start 1
14137 Jan 03 08:33:52.819 INFO request completed, latency_us: 211, response_code: 204, uri: /start/cid/1, method: POST, req_id: 3ca00ca8-dda8-43cc-97e4-d7e953763767, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
14138 Replay: started 1, returned:Ok(())
14139 ----------------------------------------------------------------
14140 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14141 States: Active Offline Active
14142 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14143 4623 Acked Read 10 Done Sent Done false
14144 4624 Acked Write 9 Done Sent Done false
14145 4625 Acked Read 4 Done Sent Done false
14146 4626 Acked Read 10 Done Sent Done false
14147 4627 Acked Write 7 Done Sent Done false
14148 4628 Acked Write 4 Done Sent Done false
14149 4629 Acked Read 1 Done Sent Done false
14150 4630 Acked Read 2 Done Sent Done false
14151 4631 Acked Read 7 Done Sent Done false
14152 4632 Acked Read 8 Done Sent Done false
14153 4633 Acked Write 10 Done Sent Done false
14154 4634 Acked Flush 0 Done Sent Done false
14155 4635 Acked Write 1 Done Sent Done false
14156 4636 Acked Flush 0 Done Sent Done false
14157 4637 Acked Read 10 Done Sent Done false
14158 4638 Acked Flush 0 Done Sent Done false
14159 4639 Acked Flush 0 Done Sent Done false
14160 4640 Acked Read 3 Done Sent Done false
14161 4641 Acked Flush 0 Done Sent Done false
14162 4642 Acked Flush 0 Done Sent Done false
14163 4643 Acked Write 4 Done Sent Done false
14164 4644 Acked Write 7 Done Sent Done false
14165 4645 Acked Read 6 Done Sent Done false
14166 4646 Acked Read 9 Done Sent Done false
14167 4647 Acked Read 3 Done Sent Done false
14168 4648 Acked Write 7 Done Sent Done false
14169 4649 Acked Flush 0 Done Sent Done false
14170 4650 Acked Write 2 Done Sent Done false
14171 4651 Acked Flush 0 Done Sent Done false
14172 4652 Acked Write 7 Done Sent Done false
14173 4653 Acked Read 9 Done Sent Done false
14174 4654 Acked Read 4 Done Sent Done false
14175 4655 Acked Write 9 Done Sent Done false
14176 4656 Acked Write 6 Done Sent Done false
14177 4657 Acked Read 10 Done Sent Done false
14178 4658 Acked Write 8 Done Sent Done false
14179 4659 Acked Write 1 Done Sent Done false
14180 4660 Acked Read 3 Done Sent Done false
14181 4661 Acked Flush 0 Done Sent Done false
14182 4662 Acked Write 8 Done Sent Done false
14183 4663 Acked Write 8 Done Sent Done false
14184 4664 Acked Read 5 Done Sent Done false
14185 4665 Acked Read 6 Done Sent Done false
14186 4666 Acked Read 10 Done Sent Done false
14187 4667 Acked Read 5 Done Sent Done false
14188 4668 Acked Write 8 Done Sent Done false
14189 4669 Acked Write 10 Done Sent Done false
14190 4670 Acked Read 3 Done Sent Done false
14191 4671 Acked Write 4 Done Sent Done false
14192 4672 Acked Write 9 Done Sent Done false
14193 4673 Acked Read 9 Done Sent Done false
14194 4674 Acked Write 7 Done Sent Done false
14195 4675 Acked Write 1 Done Sent Done false
14196 4676 Acked Read 6 Done Sent Done false
14197 4677 Acked Read 3 Done Sent Done false
14198 4678 Acked Read 6 Done Sent Done false
14199 4679 Acked Read 7 Done Sent Done false
14200 4680 Acked Write 6 Done Sent Done false
14201 4681 Acked Read 3 Done Sent Done false
14202 4682 Acked Write 8 Done Sent Done false
14203 4683 Acked Read 7 Done Sent Done false
14204 4684 Acked Write 3 Done Sent Done false
14205 4685 Acked Flush 0 Done Sent Done false
14206 4686 Acked Read 3 Done Sent Done false
14207 4687 Acked Write 9 Done Sent Done false
14208 4688 Acked Write 4 Done Sent Done false
14209 4689 Acked Flush 0 Done Sent Done false
14210 4690 Acked Read 1 Done Sent Done false
14211 4691 Acked Read 9 Done Sent Done false
14212 4692 Acked Read 7 Done Sent Done false
14213 4693 Acked Flush 0 Done Sent Done false
14214 4694 Acked Read 8 Done Sent Done false
14215 4695 Acked Read 2 Done Sent Done false
14216 4696 Acked Read 4 Done Sent Done false
14217 4697 Acked Read 10 Done Sent Done false
14218 4698 Acked Write 6 Done Sent Done false
14219 4699 Acked Write 7 Done Sent Done false
14220 4700 Acked Read 6 Done Sent Done false
14221 4701 Acked Read 6 Done Sent Done false
14222 4702 Acked Read 7 Done Sent Done false
14223 4703 Acked Write 10 Done Sent Done false
14224 4704 Acked Write 4 Done Sent Done false
14225 4705 Acked Write 2 Done Sent Done false
14226 4706 Acked Flush 0 Done Sent Done false
14227 4707 Acked Read 2 Done Sent Done false
14228 4708 Acked Read 3 Done Sent Done false
14229 4709 Acked Read 10 Done Sent Done false
14230 4710 Acked Write 5 Done Sent Done false
14231 4711 Acked Read 6 Done Sent Done false
14232 4712 Acked Write 6 Done Sent Done false
14233 4713 Acked Write 5 Done Sent Done false
14234 4714 Acked Read 9 Done Sent Done false
14235 4715 Acked Read 6 Done Sent Done false
14236 4716 Acked Read 3 Done Sent Done false
14237 4717 Acked Write 9 Done Sent Done false
14238 4718 Acked Write 1 Done Sent Done false
14239 4719 Acked Write 3 Done Sent Done false
14240 4720 Acked Write 5 Done Sent Done false
14241 4721 Acked Read 7 Done Sent Done false
14242 4722 Acked Read 9 Done Sent Done false
14243 4723 Acked Read 10 Done Sent Done false
14244 4724 Acked Read 10 Done Sent Done false
14245 4725 Acked Write 4 Done Sent Done false
14246 4726 Acked Read 5 Done Sent Done false
14247 4727 Acked Read 5 Done Sent Done false
14248 4728 Acked Write 6 Done Sent Done false
14249 4729 Acked Write 3 Done Sent Done false
14250 4730 Acked Read 2 Done Sent Done false
14251 4731 Acked Write 10 Done Sent Done false
14252 4732 Acked Write 8 Done Sent Done false
14253 4733 Acked Flush 0 Done Sent Done false
14254 4734 Acked Write 10 Done Sent Done false
14255 4735 Acked Write 2 Done Sent Done false
14256 4736 Acked Write 6 Done Sent Done false
14257 4737 Acked Read 5 Done Sent Done false
14258 4738 Acked Read 6 Done Sent Done false
14259 4739 Acked Write 1 Done Sent Done false
14260 4740 Acked Read 4 Done Sent Done false
14261 4741 Acked Write 1 Done Sent Done false
14262 4742 Acked Read 10 Done Sent Done false
14263 4743 Acked Read 7 Done Sent Done false
14264 4744 Acked Write 2 Done Sent Done false
14265 4745 Acked Write 5 Done Sent Done false
14266 4746 Acked Read 2 Done Sent Done false
14267 4747 Acked Write 4 Done Sent Done false
14268 4748 Acked Write 4 Done Sent Done false
14269 4749 Acked Read 1 Done Sent Done false
14270 4750 Acked Read 4 Done Sent Done false
14271 4751 Acked Write 10 Done Sent Done false
14272 4752 Acked Read 3 Done Sent Done false
14273 4753 Acked Flush 0 Done Sent Done false
14274 4754 Acked Write 4 Done Sent Done false
14275 4755 Acked Write 9 Done Sent Done false
14276 4756 Acked Read 3 Done Sent Done false
14277 4757 Acked Read 10 Done Sent Done false
14278 4758 Acked Read 4 Done Sent Done false
14279 4759 Acked Read 10 Done Sent Done false
14280 4760 Acked Write 6 Done Sent Done false
14281 4761 Acked Read 10 Done Sent Done false
14282 4762 Acked Flush 0 Done Sent Done false
14283 4763 Acked Write 7 Done Sent Done false
14284 4764 Acked Read 5 Done Sent Done false
14285 4765 Acked Read 7 Done Sent Done false
14286 4766 Acked Write 4 Done Sent Done false
14287 4767 Acked Read 3 Done Sent Done false
14288 4768 Acked Read 8 Done Sent Done false
14289 4769 Acked Read 2 Done Sent Done false
14290 4770 Acked Flush 0 Done Sent Done false
14291 4771 Acked Write 1 Done Sent Done false
14292 4772 Acked Write 2 Done Sent Done false
14293 4773 Acked Read 10 Done Sent Done false
14294 4774 Acked Read 6 Done Sent Done false
14295 4775 Acked Read 3 Done Sent Done false
14296 4776 Acked Read 2 Done Sent Done false
14297 4777 Acked Write 3 Done Sent Done false
14298 4778 Acked Write 10 Done Sent Done false
14299 4779 Acked Write 1 Done Sent Done false
14300 4780 Acked Flush 0 Done Sent Done false
14301 4781 Acked Read 7 Done Sent Done false
14302 4782 Acked Read 10 Done Sent Done false
14303 4783 Acked Read 10 Done Sent Done false
14304 4784 Acked Read 9 Done Sent Done false
14305 4785 Acked Write 10 Done Sent Done false
14306 4786 Acked Read 10 Done Sent Done false
14307 4787 Acked Read 10 Done Sent Done false
14308 4788 Acked Flush 0 Done Sent Done false
14309 4789 Acked Write 4 Done Sent Done false
14310 4790 Acked Write 10 Done Sent Done false
14311 4791 Acked Read 10 Done Sent Done false
14312 4792 Acked Write 8 Done Sent Done false
14313 4793 Acked Read 5 Done Sent Done false
14314 4794 Acked Read 8 Done Sent Done false
14315 4795 Acked Read 10 Done Sent Done false
14316 4796 Acked Read 10 Done Sent Done false
14317 4797 Acked Flush 0 Done Sent Done false
14318 4798 Acked Write 2 Done Sent Done false
14319 4799 Acked Write 8 Done Sent Done false
14320 4800 Acked Flush 0 Done Sent Done false
14321 4801 Acked Flush 0 Done Sent Done false
14322 4802 Acked Write 8 Done Sent Done false
14323 4803 Acked Read 2 Done Sent Done false
14324 4804 Acked Write 1 Done Sent Done false
14325 4805 Acked Write 9 Done Sent Done false
14326 4806 Acked Read 8 Done Sent Done false
14327 4807 Acked Flush 0 Done Sent Done false
14328 4808 Acked Read 2 Done Sent Done false
14329 4809 Acked Write 3 Done Sent Done false
14330 4810 Acked Write 8 Done Sent Done false
14331 4811 Acked Read 1 Done Sent Done false
14332 4812 Acked Read 9 Done Sent Done false
14333 4813 Acked Write 9 Done Sent Done false
14334 4814 Acked Read 3 Done Sent Done false
14335 4815 Acked Write 5 Done Sent Done false
14336 4816 Acked Read 6 Done Sent Done false
14337 4817 Acked Write 2 Done Sent Done false
14338 4818 Acked Read 7 Done Sent Done false
14339 4819 Acked Write 9 Done Sent Done false
14340 4820 Acked Read 1 Done Sent Done false
14341 4821 Acked Write 3 Done Sent Done false
14342 4822 Acked Read 5 Done Sent Done false
14343 4823 Acked Write 9 Done Sent Done false
14344 4824 Acked Write 9 Done Sent Done false
14345 4825 Acked Read 4 Done Sent Done false
14346 4826 Acked Write 4 Done Sent Done false
14347 4827 Acked Read 9 Done Sent Done false
14348 4828 Acked Read 5 Done Sent Done false
14349 4829 Acked Write 9 Done Sent Done false
14350 4830 Acked Read 10 Done Sent Done false
14351 4831 Acked Write 10 Done Sent Done false
14352 4832 Acked Write 2 Done Sent Done false
14353 4833 Acked Write 7 Done Sent Done false
14354 4834 Acked Read 5 Done Sent Done false
14355 4835 Acked Read 5 Done Sent Done false
14356 4836 Acked Write 7 Done Sent Done false
14357 4837 Acked Read 7 Done Sent Done false
14358 4838 Acked Write 3 Done Sent Done false
14359 4839 Acked Flush 0 Done Sent Done false
14360 4840 Acked Write 1 Done Sent Done false
14361 4841 Acked Read 4 Done Sent Done false
14362 4842 Acked Flush 0 Done Sent Done false
14363 4843 Acked Read 6 Done Sent Done false
14364 4844 Acked Read 6 Done Sent Done false
14365 4845 Acked Write 7 Done Sent Done false
14366 4846 Acked Read 8 Done Sent Done false
14367 4847 Acked Write 8 Done Sent Done false
14368 4848 Acked Read 6 Done Sent Done false
14369 4849 Acked Read 7 Done Sent Done false
14370 4850 Acked Flush 0 Done Sent Done false
14371 4851 Acked Flush 0 Done Sent Done false
14372 4852 Acked Read 5 Done Sent Done false
14373 4853 Acked Read 7 Done Sent Done false
14374 4854 Acked Write 7 Done Sent Done false
14375 4855 Acked Read 6 Done Sent Done false
14376 4856 Acked Write 10 Done Sent Done false
14377 4857 Acked Write 1 Done Sent Done false
14378 4858 Acked Read 4 Done Sent Done false
14379 4859 Acked Read 4 Done Sent Done false
14380 4860 Acked Read 7 Done Sent Done false
14381 4861 Acked Write 3 Done Sent Done false
14382 4862 Acked Read 2 Done Sent Done false
14383 4863 Acked Read 2 Done Sent Done false
14384 4864 Acked Read 7 Done Sent Done false
14385 4865 Acked Write 10 Done Sent Done false
14386 4866 Acked Read 3 Done Sent Done false
14387 4867 Acked Write 1 Done Sent Done false
14388 4868 Acked Read 5 Done Sent Done false
14389 4869 Acked Write 6 Done Sent Done false
14390 4870 Acked Read 10 Done Sent Done false
14391 4871 Acked Flush 0 Done Sent Done false
14392 4872 Acked Write 10 Done Sent Done false
14393 4873 Acked Flush 0 Done Sent Done false
14394 4874 Acked Read 9 Done Sent Done false
14395 4875 Acked Read 2 Done Sent Done false
14396 4876 Acked Read 4 Done Sent Done false
14397 4877 Acked Read 1 Done Sent Done false
14398 4878 Acked Read 1 Done Sent Done false
14399 4879 Acked Read 10 Done Sent Done false
14400 4880 Acked Write 2 Done Sent Done false
14401 4881 Acked Write 7 Done Sent Done false
14402 4882 Acked Read 6 Done Sent Done false
14403 4883 Acked Read 6 Done Sent Done false
14404 4884 Acked Write 2 Done Sent Done false
14405 4885 Acked Read 9 Done Sent Done false
14406 4886 Acked Write 9 Done Sent Done false
14407 4887 Acked Read 8 Done Sent Done false
14408 4888 Acked Write 8 Done Sent Done false
14409 4889 Acked Read 2 Done Sent Done false
14410 4890 Acked Read 7 Done Sent Done false
14411 4891 Acked Read 10 Done Sent Done false
14412 4892 Acked Flush 0 Done Sent Done false
14413 4893 Acked Read 2 Done Sent Done false
14414 4894 Acked Read 1 Done Sent Done false
14415 4895 Acked Read 9 Done Sent Done false
14416 4896 Acked Read 6 Done Sent Done false
14417 4897 Acked Write 5 Done Sent Done false
14418 4898 Acked Read 5 Done Sent Done false
14419 4899 Acked Read 5 Done Sent Done false
14420 4900 Acked Read 6 Done Sent Done false
14421 4901 Acked Read 1 Done Sent Done false
14422 4902 Acked Write 3 Done Sent Done false
14423 4903 Acked Read 10 Done Sent Done false
14424 4904 Acked Read 8 Done Sent Done false
14425 4905 Acked Write 6 Done Sent Done false
14426 4906 Acked Read 9 Done Sent Done false
14427 4907 Acked Read 9 Done Sent Done false
14428 4908 Acked Write 2 Done Sent Done false
14429 4909 Acked Write 4 Done Sent Done false
14430 4910 Acked Write 10 Done Sent Done false
14431 4911 Acked Write 3 Done Sent Done false
14432 4912 Acked Read 3 Done Sent Done false
14433 4913 Acked Read 2 Done Sent Done false
14434 4914 Acked Read 7 Done Sent Done false
14435 4915 Acked Read 7 Done Sent Done false
14436 4916 Acked Read 7 Done Sent Done false
14437 4917 Acked Write 2 Done Sent Done false
14438 4918 Acked Flush 0 Done Sent Done false
14439 4919 Acked Read 7 Done Sent Done false
14440 4920 Acked Read 2 Done Sent Done false
14441 4921 Acked Write 4 Done Sent Done false
14442 4922 Acked Read 4 Done Sent Done false
14443 4923 Acked Read 5 Done Sent Done false
14444 STATES DS:0 DS:1 DS:2 TOTAL
14445 Sent 0 301 0 301
14446 Done 301 0 301 602
14447 Skipped 0 0 0 0
14448 Error 0 0 0 0
14449 Last Flush: 4918 4622 4918
14450 Downstairs last five completed: 4622 4621 4620 4619 4618
14451 Upstairs last five completed: 4923 4922 4921 4920 4919
14452 CLIENT: Up:0 ds:301 act:2
14453 [8820] got start action
14454 [8820] I am going to restart
14455 Starting downstairs at port 8820
14456 Make output file at "/tmp/dsc/downstairs-8820.txt"
14457 [8820][1] reports Starting
14458 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
14459 [8820][1] reports Running
14460 ----------------------------------------------------------------
14461 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14462 States: Active Offline Active
14463 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14464 4623 Acked Read 10 Done Sent Done false
14465 4624 Acked Write 9 Done Sent Done false
14466 4625 Acked Read 4 Done Sent Done false
14467 4626 Acked Read 10 Done Sent Done false
14468 4627 Acked Write 7 Done Sent Done false
14469 4628 Acked Write 4 Done Sent Done false
14470 4629 Acked Read 1 Done Sent Done false
14471 4630 Acked Read 2 Done Sent Done false
14472 4631 Acked Read 7 Done Sent Done false
14473 4632 Acked Read 8 Done Sent Done false
14474 4633 Acked Write 10 Done Sent Done false
14475 4634 Acked Flush 0 Done Sent Done false
14476 4635 Acked Write 1 Done Sent Done false
14477 4636 Acked Flush 0 Done Sent Done false
14478 4637 Acked Read 10 Done Sent Done false
14479 4638 Acked Flush 0 Done Sent Done false
14480 4639 Acked Flush 0 Done Sent Done false
14481 4640 Acked Read 3 Done Sent Done false
14482 4641 Acked Flush 0 Done Sent Done false
14483 4642 Acked Flush 0 Done Sent Done false
14484 4643 Acked Write 4 Done Sent Done false
14485 4644 Acked Write 7 Done Sent Done false
14486 4645 Acked Read 6 Done Sent Done false
14487 4646 Acked Read 9 Done Sent Done false
14488 4647 Acked Read 3 Done Sent Done false
14489 4648 Acked Write 7 Done Sent Done false
14490 4649 Acked Flush 0 Done Sent Done false
14491 4650 Acked Write 2 Done Sent Done false
14492 4651 Acked Flush 0 Done Sent Done false
14493 4652 Acked Write 7 Done Sent Done false
14494 4653 Acked Read 9 Done Sent Done false
14495 4654 Acked Read 4 Done Sent Done false
14496 4655 Acked Write 9 Done Sent Done false
14497 4656 Acked Write 6 Done Sent Done false
14498 4657 Acked Read 10 Done Sent Done false
14499 4658 Acked Write 8 Done Sent Done false
14500 4659 Acked Write 1 Done Sent Done false
14501 4660 Acked Read 3 Done Sent Done false
14502 4661 Acked Flush 0 Done Sent Done false
14503 4662 Acked Write 8 Done Sent Done false
14504 4663 Acked Write 8 Done Sent Done false
14505 4664 Acked Read 5 Done Sent Done false
14506 4665 Acked Read 6 Done Sent Done false
14507 4666 Acked Read 10 Done Sent Done false
14508 4667 Acked Read 5 Done Sent Done false
14509 4668 Acked Write 8 Done Sent Done false
14510 4669 Acked Write 10 Done Sent Done false
14511 4670 Acked Read 3 Done Sent Done false
14512 4671 Acked Write 4 Done Sent Done false
14513 4672 Acked Write 9 Done Sent Done false
14514 4673 Acked Read 9 Done Sent Done false
14515 4674 Acked Write 7 Done Sent Done false
14516 4675 Acked Write 1 Done Sent Done false
14517 4676 Acked Read 6 Done Sent Done false
14518 4677 Acked Read 3 Done Sent Done false
14519 4678 Acked Read 6 Done Sent Done false
14520 4679 Acked Read 7 Done Sent Done false
14521 4680 Acked Write 6 Done Sent Done false
14522 4681 Acked Read 3 Done Sent Done false
14523 4682 Acked Write 8 Done Sent Done false
14524 4683 Acked Read 7 Done Sent Done false
14525 4684 Acked Write 3 Done Sent Done false
14526 4685 Acked Flush 0 Done Sent Done false
14527 4686 Acked Read 3 Done Sent Done false
14528 4687 Acked Write 9 Done Sent Done false
14529 4688 Acked Write 4 Done Sent Done false
14530 4689 Acked Flush 0 Done Sent Done false
14531 4690 Acked Read 1 Done Sent Done false
14532 4691 Acked Read 9 Done Sent Done false
14533 4692 Acked Read 7 Done Sent Done false
14534 4693 Acked Flush 0 Done Sent Done false
14535 4694 Acked Read 8 Done Sent Done false
14536 4695 Acked Read 2 Done Sent Done false
14537 4696 Acked Read 4 Done Sent Done false
14538 4697 Acked Read 10 Done Sent Done false
14539 4698 Acked Write 6 Done Sent Done false
14540 4699 Acked Write 7 Done Sent Done false
14541 4700 Acked Read 6 Done Sent Done false
14542 4701 Acked Read 6 Done Sent Done false
14543 4702 Acked Read 7 Done Sent Done false
14544 4703 Acked Write 10 Done Sent Done false
14545 4704 Acked Write 4 Done Sent Done false
14546 4705 Acked Write 2 Done Sent Done false
14547 4706 Acked Flush 0 Done Sent Done false
14548 4707 Acked Read 2 Done Sent Done false
14549 4708 Acked Read 3 Done Sent Done false
14550 4709 Acked Read 10 Done Sent Done false
14551 4710 Acked Write 5 Done Sent Done false
14552 4711 Acked Read 6 Done Sent Done false
14553 4712 Acked Write 6 Done Sent Done false
14554 4713 Acked Write 5 Done Sent Done false
14555 4714 Acked Read 9 Done Sent Done false
14556 4715 Acked Read 6 Done Sent Done false
14557 4716 Acked Read 3 Done Sent Done false
14558 4717 Acked Write 9 Done Sent Done false
14559 4718 Acked Write 1 Done Sent Done false
14560 4719 Acked Write 3 Done Sent Done false
14561 4720 Acked Write 5 Done Sent Done false
14562 4721 Acked Read 7 Done Sent Done false
14563 4722 Acked Read 9 Done Sent Done false
14564 4723 Acked Read 10 Done Sent Done false
14565 4724 Acked Read 10 Done Sent Done false
14566 4725 Acked Write 4 Done Sent Done false
14567 4726 Acked Read 5 Done Sent Done false
14568 4727 Acked Read 5 Done Sent Done false
14569 4728 Acked Write 6 Done Sent Done false
14570 4729 Acked Write 3 Done Sent Done false
14571 4730 Acked Read 2 Done Sent Done false
14572 4731 Acked Write 10 Done Sent Done false
14573 4732 Acked Write 8 Done Sent Done false
14574 4733 Acked Flush 0 Done Sent Done false
14575 4734 Acked Write 10 Done Sent Done false
14576 4735 Acked Write 2 Done Sent Done false
14577 4736 Acked Write 6 Done Sent Done false
14578 4737 Acked Read 5 Done Sent Done false
14579 4738 Acked Read 6 Done Sent Done false
14580 4739 Acked Write 1 Done Sent Done false
14581 4740 Acked Read 4 Done Sent Done false
14582 4741 Acked Write 1 Done Sent Done false
14583 4742 Acked Read 10 Done Sent Done false
14584 4743 Acked Read 7 Done Sent Done false
14585 4744 Acked Write 2 Done Sent Done false
14586 4745 Acked Write 5 Done Sent Done false
14587 4746 Acked Read 2 Done Sent Done false
14588 4747 Acked Write 4 Done Sent Done false
14589 4748 Acked Write 4 Done Sent Done false
14590 4749 Acked Read 1 Done Sent Done false
14591 4750 Acked Read 4 Done Sent Done false
14592 4751 Acked Write 10 Done Sent Done false
14593 4752 Acked Read 3 Done Sent Done false
14594 4753 Acked Flush 0 Done Sent Done false
14595 4754 Acked Write 4 Done Sent Done false
14596 4755 Acked Write 9 Done Sent Done false
14597 4756 Acked Read 3 Done Sent Done false
14598 4757 Acked Read 10 Done Sent Done false
14599 4758 Acked Read 4 Done Sent Done false
14600 4759 Acked Read 10 Done Sent Done false
14601 4760 Acked Write 6 Done Sent Done false
14602 4761 Acked Read 10 Done Sent Done false
14603 4762 Acked Flush 0 Done Sent Done false
14604 4763 Acked Write 7 Done Sent Done false
14605 4764 Acked Read 5 Done Sent Done false
14606 4765 Acked Read 7 Done Sent Done false
14607 4766 Acked Write 4 Done Sent Done false
14608 4767 Acked Read 3 Done Sent Done false
14609 4768 Acked Read 8 Done Sent Done false
14610 4769 Acked Read 2 Done Sent Done false
14611 4770 Acked Flush 0 Done Sent Done false
14612 4771 Acked Write 1 Done Sent Done false
14613 4772 Acked Write 2 Done Sent Done false
14614 4773 Acked Read 10 Done Sent Done false
14615 4774 Acked Read 6 Done Sent Done false
14616 4775 Acked Read 3 Done Sent Done false
14617 4776 Acked Read 2 Done Sent Done false
14618 4777 Acked Write 3 Done Sent Done false
14619 4778 Acked Write 10 Done Sent Done false
14620 4779 Acked Write 1 Done Sent Done false
14621 4780 Acked Flush 0 Done Sent Done false
14622 4781 Acked Read 7 Done Sent Done false
14623 4782 Acked Read 10 Done Sent Done false
14624 4783 Acked Read 10 Done Sent Done false
14625 4784 Acked Read 9 Done Sent Done false
14626 4785 Acked Write 10 Done Sent Done false
14627 4786 Acked Read 10 Done Sent Done false
14628 4787 Acked Read 10 Done Sent Done false
14629 4788 Acked Flush 0 Done Sent Done false
14630 4789 Acked Write 4 Done Sent Done false
14631 4790 Acked Write 10 Done Sent Done false
14632 4791 Acked Read 10 Done Sent Done false
14633 4792 Acked Write 8 Done Sent Done false
14634 4793 Acked Read 5 Done Sent Done false
14635 4794 Acked Read 8 Done Sent Done false
14636 4795 Acked Read 10 Done Sent Done false
14637 4796 Acked Read 10 Done Sent Done false
14638 4797 Acked Flush 0 Done Sent Done false
14639 4798 Acked Write 2 Done Sent Done false
14640 4799 Acked Write 8 Done Sent Done false
14641 4800 Acked Flush 0 Done Sent Done false
14642 4801 Acked Flush 0 Done Sent Done false
14643 4802 Acked Write 8 Done Sent Done false
14644 4803 Acked Read 2 Done Sent Done false
14645 4804 Acked Write 1 Done Sent Done false
14646 4805 Acked Write 9 Done Sent Done false
14647 4806 Acked Read 8 Done Sent Done false
14648 4807 Acked Flush 0 Done Sent Done false
14649 4808 Acked Read 2 Done Sent Done false
14650 4809 Acked Write 3 Done Sent Done false
14651 4810 Acked Write 8 Done Sent Done false
14652 4811 Acked Read 1 Done Sent Done false
14653 4812 Acked Read 9 Done Sent Done false
14654 4813 Acked Write 9 Done Sent Done false
14655 4814 Acked Read 3 Done Sent Done false
14656 4815 Acked Write 5 Done Sent Done false
14657 4816 Acked Read 6 Done Sent Done false
14658 4817 Acked Write 2 Done Sent Done false
14659 4818 Acked Read 7 Done Sent Done false
14660 4819 Acked Write 9 Done Sent Done false
14661 4820 Acked Read 1 Done Sent Done false
14662 4821 Acked Write 3 Done Sent Done false
14663 4822 Acked Read 5 Done Sent Done false
14664 4823 Acked Write 9 Done Sent Done false
14665 4824 Acked Write 9 Done Sent Done false
14666 4825 Acked Read 4 Done Sent Done false
14667 4826 Acked Write 4 Done Sent Done false
14668 4827 Acked Read 9 Done Sent Done false
14669 4828 Acked Read 5 Done Sent Done false
14670 4829 Acked Write 9 Done Sent Done false
14671 4830 Acked Read 10 Done Sent Done false
14672 4831 Acked Write 10 Done Sent Done false
14673 4832 Acked Write 2 Done Sent Done false
14674 4833 Acked Write 7 Done Sent Done false
14675 4834 Acked Read 5 Done Sent Done false
14676 4835 Acked Read 5 Done Sent Done false
14677 4836 Acked Write 7 Done Sent Done false
14678 4837 Acked Read 7 Done Sent Done false
14679 4838 Acked Write 3 Done Sent Done false
14680 4839 Acked Flush 0 Done Sent Done false
14681 4840 Acked Write 1 Done Sent Done false
14682 4841 Acked Read 4 Done Sent Done false
14683 4842 Acked Flush 0 Done Sent Done false
14684 4843 Acked Read 6 Done Sent Done false
14685 4844 Acked Read 6 Done Sent Done false
14686 4845 Acked Write 7 Done Sent Done false
14687 4846 Acked Read 8 Done Sent Done false
14688 4847 Acked Write 8 Done Sent Done false
14689 4848 Acked Read 6 Done Sent Done false
14690 4849 Acked Read 7 Done Sent Done false
14691 4850 Acked Flush 0 Done Sent Done false
14692 4851 Acked Flush 0 Done Sent Done false
14693 4852 Acked Read 5 Done Sent Done false
14694 4853 Acked Read 7 Done Sent Done false
14695 4854 Acked Write 7 Done Sent Done false
14696 4855 Acked Read 6 Done Sent Done false
14697 4856 Acked Write 10 Done Sent Done false
14698 4857 Acked Write 1 Done Sent Done false
14699 4858 Acked Read 4 Done Sent Done false
14700 4859 Acked Read 4 Done Sent Done false
14701 4860 Acked Read 7 Done Sent Done false
14702 4861 Acked Write 3 Done Sent Done false
14703 4862 Acked Read 2 Done Sent Done false
14704 4863 Acked Read 2 Done Sent Done false
14705 4864 Acked Read 7 Done Sent Done false
14706 4865 Acked Write 10 Done Sent Done false
14707 4866 Acked Read 3 Done Sent Done false
14708 4867 Acked Write 1 Done Sent Done false
14709 4868 Acked Read 5 Done Sent Done false
14710 4869 Acked Write 6 Done Sent Done false
14711 4870 Acked Read 10 Done Sent Done false
14712 4871 Acked Flush 0 Done Sent Done false
14713 4872 Acked Write 10 Done Sent Done false
14714 4873 Acked Flush 0 Done Sent Done false
14715 4874 Acked Read 9 Done Sent Done false
14716 4875 Acked Read 2 Done Sent Done false
14717 4876 Acked Read 4 Done Sent Done false
14718 4877 Acked Read 1 Done Sent Done false
14719 4878 Acked Read 1 Done Sent Done false
14720 4879 Acked Read 10 Done Sent Done false
14721 4880 Acked Write 2 Done Sent Done false
14722 4881 Acked Write 7 Done Sent Done false
14723 4882 Acked Read 6 Done Sent Done false
14724 4883 Acked Read 6 Done Sent Done false
14725 4884 Acked Write 2 Done Sent Done false
14726 4885 Acked Read 9 Done Sent Done false
14727 4886 Acked Write 9 Done Sent Done false
14728 4887 Acked Read 8 Done Sent Done false
14729 4888 Acked Write 8 Done Sent Done false
14730 4889 Acked Read 2 Done Sent Done false
14731 4890 Acked Read 7 Done Sent Done false
14732 4891 Acked Read 10 Done Sent Done false
14733 4892 Acked Flush 0 Done Sent Done false
14734 4893 Acked Read 2 Done Sent Done false
14735 4894 Acked Read 1 Done Sent Done false
14736 4895 Acked Read 9 Done Sent Done false
14737 4896 Acked Read 6 Done Sent Done false
14738 4897 Acked Write 5 Done Sent Done false
14739 4898 Acked Read 5 Done Sent Done false
14740 4899 Acked Read 5 Done Sent Done false
14741 4900 Acked Read 6 Done Sent Done false
14742 4901 Acked Read 1 Done Sent Done false
14743 4902 Acked Write 3 Done Sent Done false
14744 4903 Acked Read 10 Done Sent Done false
14745 4904 Acked Read 8 Done Sent Done false
14746 4905 Acked Write 6 Done Sent Done false
14747 4906 Acked Read 9 Done Sent Done false
14748 4907 Acked Read 9 Done Sent Done false
14749 4908 Acked Write 2 Done Sent Done false
14750 4909 Acked Write 4 Done Sent Done false
14751 4910 Acked Write 10 Done Sent Done false
14752 4911 Acked Write 3 Done Sent Done false
14753 4912 Acked Read 3 Done Sent Done false
14754 4913 Acked Read 2 Done Sent Done false
14755 4914 Acked Read 7 Done Sent Done false
14756 4915 Acked Read 7 Done Sent Done false
14757 4916 Acked Read 7 Done Sent Done false
14758 4917 Acked Write 2 Done Sent Done false
14759 4918 Acked Flush 0 Done Sent Done false
14760 4919 Acked Read 7 Done Sent Done false
14761 4920 Acked Read 2 Done Sent Done false
14762 4921 Acked Write 4 Done Sent Done false
14763 4922 Acked Read 4 Done Sent Done false
14764 4923 Acked Read 5 Done Sent Done false
14765 4924 Acked Flush 0 Done Sent Done false
14766 STATES DS:0 DS:1 DS:2 TOTAL
14767 Sent 0 302 0 302
14768 Done 302 0 302 604
14769 Skipped 0 0 0 0
14770 Error 0 0 0 0
14771 Last Flush: 4924 4622 4924
14772 Downstairs last five completed: 4622 4621 4620 4619 4618
14773 Upstairs last five completed: 4924 4923 4922 4921 4920
14774 CLIENT: Up:0 ds:302 act:2
147752025-01-03T08:33:58.565ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147762025-01-03T08:33:58.565ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58435) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147772025-01-03T08:33:58.566ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147782025-01-03T08:33:58.567ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147792025-01-03T08:33:58.567ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147802025-01-03T08:33:58.567ZINFOcrucible: send last flush ID to this DS: 4622 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147812025-01-03T08:33:58.567ZINFOcrucible: Replied this last flush ID: 4622 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147822025-01-03T08:33:58.567ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147832025-01-03T08:33:58.568ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147842025-01-03T08:33:58.568ZINFOcrucible: [1] client re-new 302 jobs since flush 4622 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147852025-01-03T08:33:58.568ZINFOcrucible: [1] Replayed 302 jobs since flush: 4622 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
14786 ----------------------------------------------------------------
14787 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14788 Downstairs last five completed: 4924 4923 4922 4921 4920
14789 Upstairs last five completed: 4924 4923 4922 4921 4920
14790 CLIENT: Up:0 ds:0 act:3
14791 Replay: All jobs finished
14792 Main task has work to do, go find it
14793 got dsc Stop(0)
14794 stop 0
14795 [8810] Got stop action so:false kr:false
14796 Jan 03 08:34:00.825 INFO request completed, latency_us: 339, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 471465d7-cda9-456e-a6cd-7780df11ea78, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
147972025-01-03T08:34:00.826ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147982025-01-03T08:34:00.826ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
147992025-01-03T08:34:00.826ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
148002025-01-03T08:34:00.826ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
148012025-01-03T08:34:00.826ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
148022025-01-03T08:34:00.826ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
14803 Jan 03 08:34:00.826 INFO request completed, latency_us: 246, response_code: 200, uri: /state/cid/0, method: GET, req_id: c00b5cc9-b8ce-4b63-8e10-9f95eaebc6cc, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
14804 [8810] Exited with: ExitStatus(unix_wait_status(9))
14805 [8810][0] reports Exit
14806 Jan 03 08:34:04.828 INFO request completed, latency_us: 194, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8412e057-1d1f-4976-ac86-b26d71adbe54, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
14807 001/300 Read block 701 len 12288
14808 002/300 Write block 4344 len 28672 data: 4 4 3 3 3 3 3
14809 003/300 Read block 4935 len 36864
14810 004/300 Read block 81 len 36864
14811 005/300 Write block 1358 len 4096 data: 5
14812 006/300 Read block 4989 len 12288
14813 007/300 Write block 4733 len 8192 data: 3 2
14814 008/300 Read block 2892 len 24576
14815 009/300 Write block 3167 len 16384 data: 3 3 3 2
14816 010/300 Write block 1387 len 12288 data: 4 5 4
14817 011/300 Read block 4210 len 4096
14818 012/300 Flush
14819 013/300 Read block 2540 len 8192
14820 014/300 Read block 882 len 4096
14821 015/300 Read block 3726 len 8192
14822 016/300 Read block 4708 len 32768
14823 017/300 Write block 587 len 4096 data: 3
14824 018/300 Read block 3162 len 8192
14825 019/300 Flush
14826 020/300 Read block 2384 len 4096
14827 021/300 Write block 2580 len 28672 data: 2 3 4 4 4 4 4
14828 022/300 Write block 3384 len 32768 data: 3 5 5 5 5 5 4 4
14829 023/300 Write block 2060 len 36864 data: 2 2 2 2 2 2 3 4 4
14830 024/300 Write block 2718 len 36864 data: 3 4 6 7 7 7 6 6 5
14831 025/300 Read block 1266 len 20480
14832 026/300 Read block 2690 len 8192
14833 027/300 Read block 2333 len 8192
14834 028/300 Write block 3148 len 4096 data: 4
14835 029/300 Write block 3198 len 12288 data: 5 5 6
14836 030/300 Write block 331 len 8192 data: 5 5
14837 031/300 Write block 4955 len 28672 data: 5 5 5 5 5 5 5
14838 032/300 Read block 343 len 20480
14839 033/300 Write block 1746 len 40960 data: 2 2 2 2 3 4 5 6 6 5
14840 034/300 Write block 1558 len 20480 data: 2 2 2 2 2
14841 035/300 Write block 1534 len 8192 data: 4 6
14842 036/300 Read block 3052 len 4096
14843 037/300 Read block 4629 len 32768
14844 038/300 Read block 4291 len 24576
14845 039/300 Write block 915 len 8192 data: 5 5
14846 040/300 Read block 4646 len 12288
14847 041/300 Write block 1700 len 32768 data: 2 2 2 3 3 3 3 3
14848 042/300 Read block 2878 len 20480
14849 043/300 Read block 656 len 40960
14850 044/300 Write block 1620 len 20480 data: 3 3 3 3 3
14851 045/300 Flush
14852 046/300 Read block 2315 len 32768
14853 047/300 Read block 2604 len 36864
14854 048/300 Write block 3180 len 8192 data: 5 4
14855 049/300 Write block 2217 len 24576 data: 3 4 3 3 4 4
14856 050/300 Write block 1582 len 12288 data: 3 3 4
14857 051/300 Write block 2503 len 40960 data: 2 3 3 3 3 3 3 3 4 4
14858 052/300 Write block 3393 len 28672 data: 3 3 5 4 5 5 4
14859 053/300 Read block 409 len 40960
14860 054/300 Write block 4295 len 4096 data: 2
14861 055/300 Read block 377 len 8192
14862 056/300 Read block 131 len 40960
14863 057/300 Read block 3853 len 40960
14864 058/300 Read block 933 len 40960
14865 059/300 Write block 3255 len 36864 data: 3 3 3 5 5 4 4 4 4
14866 060/300 Read block 2390 len 4096
14867 061/300 Read block 4094 len 16384
14868 062/300 Write block 4737 len 4096 data: 2
14869 063/300 Flush
14870 064/300 Write block 4276 len 32768 data: 3 3 3 2 2 2 2 2
14871 065/300 Write block 4506 len 8192 data: 5 4
14872 066/300 Write block 1826 len 36864 data: 4 5 5 5 6 4 3 3 3
14873 067/300 Write block 4198 len 36864 data: 6 7 7 8 7 6 7 8 7
14874 068/300 Flush
14875 069/300 Read block 1742 len 40960
14876 070/300 Read block 386 len 8192
14877 071/300 Read block 2742 len 8192
14878 072/300 Read block 3204 len 32768
14879 073/300 Read block 3119 len 20480
14880 074/300 Write block 842 len 4096 data: 6
14881 075/300 Read block 3843 len 36864
14882 076/300 Write block 4862 len 8192 data: 3 3
14883 077/300 Flush
14884 078/300 Read block 3915 len 4096
14885 079/300 Read block 950 len 32768
14886 080/300 Read block 4766 len 4096
14887 081/300 Write block 2288 len 24576 data: 4 5 4 4 3 3
14888 082/300 Read block 4351 len 4096
14889 083/300 Read block 655 len 24576
14890 084/300 Write block 4469 len 32768 data: 3 3 3 3 3 3 3 4
14891 085/300 Read block 1760 len 8192
14892 086/300 Flush
14893 087/300 Read block 1910 len 28672
14894 088/300 Write block 3419 len 4096 data: 3
14895 089/300 Read block 4478 len 40960
14896 090/300 Write block 2137 len 4096 data: 5
14897 091/300 Read block 4982 len 40960
14898 092/300 Read block 3502 len 32768
14899 093/300 Read block 1753 len 40960
14900 094/300 Read block 4528 len 16384
14901 095/300 Read block 1451 len 36864
14902 096/300 Write block 2678 len 24576 data: 4 4 4 3 5 4
14903 097/300 Write block 4601 len 32768 data: 3 3 4 4 4 3 3 3
14904 098/300 Write block 3671 len 24576 data: 3 3 3 3 3 3
14905 099/300 Write block 4878 len 36864 data: 5 6 6 7 6 5 4 4 5
14906 100/300 Write block 1228 len 24576 data: 3 3 3 4 4 4
14907 101/300 Read block 3555 len 28672
14908 102/300 Read block 1800 len 40960
14909 103/300 Write block 3599 len 12288 data: 4 4 5
14910 104/300 Flush
14911 105/300 Read block 4725 len 36864
14912 106/300 Write block 2600 len 8192 data: 5 5
14913 107/300 Read block 1065 len 16384
14914 108/300 Read block 191 len 12288
14915 109/300 Read block 1588 len 28672
14916 110/300 Write block 4701 len 32768 data: 2 3 3 4 6 6 7 7
14917 111/300 Write block 3926 len 16384 data: 3 3 2 4
14918 112/300 Read block 1857 len 20480
14919 113/300 Write block 2411 len 8192 data: 3 3
14920 114/300 Write block 4918 len 24576 data: 3 4 4 4 3 3
14921 115/300 Write block 3717 len 12288 data: 5 5 6
14922 116/300 Flush
14923 117/300 Read block 1567 len 16384
14924 118/300 Read block 1127 len 4096
14925 119/300 Write block 4067 len 28672 data: 3 3 3 3 3 3 5
14926 120/300 Write block 4467 len 8192 data: 3 3
14927 121/300 Flush
14928 122/300 Read block 1587 len 24576
14929 123/300 Flush
14930 124/300 Write block 3216 len 32768 data: 2 2 3 4 5 5 5 5
14931 125/300 Flush
14932 126/300 Read block 592 len 40960
14933 127/300 Write block 2483 len 12288 data: 5 6 6
14934 128/300 Flush
14935 129/300 Read block 322 len 12288
14936 130/300 Write block 4004 len 32768 data: 7 7 6 5 5 2 2 2
14937 131/300 Read block 2601 len 12288
14938 132/300 Write block 2904 len 4096 data: 4
14939 133/300 Read block 1773 len 20480
14940 134/300 Read block 4203 len 32768
14941 135/300 Write block 3549 len 4096 data: 5
14942 136/300 Flush
14943 137/300 Read block 3614 len 12288
14944 138/300 Read block 4388 len 4096
14945 139/300 Flush
14946 140/300 Write block 74 len 20480 data: 5 3 3 3 3
14947 141/300 Read block 3806 len 24576
14948 142/300 Write block 4044 len 24576 data: 5 4 4 4 4 2
14949 143/300 Write block 999 len 4096 data: 4
14950 144/300 Read block 2088 len 20480
14951 145/300 Write block 1557 len 28672 data: 2 3 3 3 3 3 2
14952 146/300 Write block 620 len 36864 data: 4 4 3 4 4 3 4 5 5
14953 147/300 Read block 56 len 28672
14954 148/300 Read block 1816 len 28672
14955 149/300 Write block 1642 len 20480 data: 2 3 3 3 3
14956 150/300 Read block 473 len 12288
14957 151/300 Write block 1948 len 12288 data: 5 5 5
14958 152/300 Read block 1859 len 40960
14959 153/300 Read block 1835 len 24576
14960 154/300 Write block 2402 len 32768 data: 3 2 2 2 2 4 4 4
14961 155/300 Write block 2316 len 16384 data: 3 4 3 3
14962 156/300 Read block 1633 len 36864
14963 157/300 Read block 3098 len 4096
14964 158/300 Read block 2994 len 16384
14965 159/300 Read block 2006 len 12288
14966 160/300 Write block 4868 len 28672 data: 3 5 4 4 5 3 3
14967 161/300 Read block 3316 len 12288
14968 162/300 Write block 3534 len 28672 data: 4 5 4 4 4 4 4
14969 163/300 Read block 2775 len 36864
14970 164/300 Read block 2243 len 24576
14971 165/300 Read block 3333 len 4096
14972 166/300 Read block 719 len 12288
14973 167/300 Read block 2216 len 24576
14974 168/300 Write block 510 len 24576 data: 3 3 3 3 4 4
14975 169/300 Read block 3223 len 36864
14976 170/300 Write block 1161 len 20480 data: 3 2 2 3 3
14977 171/300 Write block 4335 len 8192 data: 4 4
14978 172/300 Write block 2309 len 24576 data: 4 4 2 2 2 3
14979 173/300 Write block 2253 len 20480 data: 5 5 5 5 3
14980 174/300 Write block 95 len 40960 data: 3 5 5 4 4 4 4 3 3 3
14981 175/300 Write block 1090 len 8192 data: 4 5
14982 176/300 Write block 3231 len 28672 data: 6 7 7 7 7 7 7
14983 177/300 Flush
14984 178/300 Read block 1098 len 36864
14985 179/300 Write block 1086 len 40960 data: 2 2 2 4 5 6 5 6 6 6
14986 180/300 Read block 1304 len 36864
14987 181/300 Write block 4358 len 8192 data: 4 4
14988 182/300 Read block 3197 len 16384
14989 183/300 Write block 3284 len 4096 data: 3
14990 184/300 Write block 554 len 16384 data: 4 4 4 4
14991 185/300 Write block 399 len 8192 data: 4 3
14992 186/300 Read block 3616 len 20480
14993 187/300 Write block 3957 len 20480 data: 4 4 2 3 4
14994 188/300 Write block 3367 len 36864 data: 3 3 3 3 3 3 3 2 3
14995 189/300 Read block 2887 len 8192
14996 190/300 Read block 3182 len 12288
14997 191/300 Write block 1753 len 8192 data: 7 7
14998 192/300 Write block 4465 len 28672 data: 3 4 4 4 4 4 4
14999 193/300 Flush
15000 194/300 Flush
15001 195/300 Read block 661 len 40960
15002 196/300 Read block 4965 len 4096
15003 197/300 Write block 3506 len 12288 data: 2 2 2
15004 198/300 Write block 2120 len 16384 data: 4 4 3 3
15005 199/300 Read block 2246 len 28672
15006 200/300 Write block 998 len 4096 data: 4
15007 201/300 Write block 2419 len 28672 data: 5 5 5 5 5 4 4
15008 202/300 Flush
15009 203/300 Write block 894 len 36864 data: 5 6 6 5 4 4 4 4 4
15010 204/300 Read block 1588 len 24576
15011 205/300 Read block 3429 len 28672
15012 206/300 Read block 1543 len 8192
15013 207/300 Read block 1833 len 16384
15014 208/300 Read block 2175 len 40960
15015 209/300 Write block 3650 len 4096 data: 3
15016 210/300 Write block 3932 len 24576 data: 4 4 3 2 3 2
15017 211/300 Read block 2098 len 8192
15018 212/300 Read block 2392 len 20480
15019 213/300 Write block 3360 len 24576 data: 4 4 3 3 3 2
15020 214/300 Read block 1342 len 32768
15021 215/300 Read block 1227 len 12288
15022 216/300 Write block 2732 len 20480 data: 7 6 6 5 4
15023 217/300 Write block 3348 len 32768 data: 3 4 4 4 3 3 3 3
15024 218/300 Write block 2986 len 4096 data: 2
15025 219/300 Read block 2934 len 24576
15026 220/300 Read block 1359 len 8192
15027 221/300 Read block 4589 len 40960
15028 222/300 Read block 3057 len 32768
15029 223/300 Write block 3901 len 28672 data: 4 4 4 4 4 4 4
15030 224/300 Write block 1497 len 8192 data: 2 3
15031 225/300 Read block 411 len 24576
15032 226/300 Read block 2556 len 32768
15033 227/300 Write block 2054 len 20480 data: 4 5 5 3 2
15034 228/300 Read block 2163 len 40960
15035 229/300 Flush
15036 230/300 Flush
15037 231/300 Write block 3494 len 20480 data: 3 3 3 3 3
15038 232/300 Write block 1096 len 24576 data: 5 5 5 4 5 5
15039 233/300 Read block 4670 len 28672
15040 234/300 Read block 4071 len 24576
15041 235/300 Write block 2660 len 36864 data: 5 4 4 4 4 3 3 3 2
15042 236/300 Write block 3350 len 20480 data: 5 5 4 4 4
15043 237/300 Write block 74 len 12288 data: 6 4 4
15044 238/300 Read block 1315 len 20480
15045 239/300 Read block 325 len 4096
15046 240/300 Write block 4003 len 36864 data: 7 8 8 7 6 6 3 3 3
15047 241/300 Write block 2011 len 8192 data: 2 2
15048 242/300 Flush
15049 243/300 Read block 1803 len 20480
15050 244/300 Read block 2968 len 16384
15051 245/300 Read block 3333 len 36864
15052 246/300 Write block 3194 len 16384 data: 4 4 5 5
15053 247/300 Read block 1019 len 36864
15054 248/300 Flush
15055 249/300 Read block 131 len 28672
15056 250/300 Write block 4858 len 4096 data: 3
15057 251/300 Read block 3377 len 36864
15058 252/300 Write block 1510 len 8192 data: 5 6
15059 253/300 Write block 3411 len 24576 data: 2 3 3 3 3 3
15060 254/300 Write block 2459 len 4096 data: 5
15061 255/300 Read block 3358 len 4096
15062 256/300 Read block 2981 len 24576
15063 257/300 Read block 2727 len 24576
15064 258/300 Read block 22 len 16384
15065 259/300 Write block 3124 len 16384 data: 2 2 3 3
15066 260/300 Read block 1041 len 20480
15067 261/300 Read block 885 len 24576
15068 262/300 Write block 2303 len 28672 data: 4 4 3 3 4 5 5
15069 263/300 Write block 682 len 16384 data: 3 3 3 3
15070 264/300 Flush
15071 265/300 Write block 4619 len 12288 data: 3 4 4
15072 266/300 Read block 2610 len 12288
15073 267/300 Write block 4239 len 28672 data: 3 4 4 4 4 3 3
15074 268/300 Read block 1359 len 16384
15075 269/300 Write block 2527 len 20480 data: 4 3 2 3 3
15076 270/300 Write block 4416 len 36864 data: 4 3 3 4 4 3 4 4 4
15077 271/300 Write block 758 len 40960 data: 4 6 6 5 5 5 6 4 5 5
15078 272/300 Write block 1158 len 16384 data: 4 4 5 4
15079 273/300 Read block 4054 len 32768
15080 274/300 Read block 1697 len 4096
15081 275/300 Read block 280 len 36864
15082 276/300 Read block 4161 len 32768
15083 277/300 Write block 3733 len 16384 data: 4 3 3 3
15084 278/300 Read block 4339 len 20480
15085 279/300 Read block 652 len 24576
15086 280/300 Write block 1774 len 8192 data: 2 2
15087 281/300 Read block 321 len 40960
15088 282/300 Flush
15089 283/300 Write block 2945 len 40960 data: 3 4 4 3 3 2 2 2 2 3
15090 284/300 Write block 731 len 40960 data: 4 4 4 4 3 3 2 3 3 3
15091 285/300 Write block 4648 len 32768 data: 5 4 2 2 2 2 2 2
15092 286/300 Write block 113 len 32768 data: 3 3 3 2 2 2 2 2
15093 287/300 Read block 792 len 36864
15094 288/300 Read block 3469 len 32768
15095 289/300 Read block 3135 len 12288
15096 290/300 Read block 1685 len 4096
15097 291/300 Read block 1189 len 12288
15098 292/300 Write block 2123 len 20480 data: 4 4 4 3 3
15099 293/300 Read block 60 len 28672
15100 294/300 Write block 494 len 24576 data: 4 3 3 4 3 2
15101 295/300 Write block 2924 len 36864 data: 5 4 4 4 3 2 2 2 2
15102 296/300 Write block 4167 len 36864 data: 6 7 6 6 5 6 6 4 4
15103 297/300 Read block 214 len 20480
15104 298/300 Write block 889 len 16384 data: 5 4 4 4
15105 299/300 Flush
15106 300/300 Read block 3237 len 4096
15107 301/300 Read block 1163 len 32768
15108 Main task has work to do, go find it
15109 got dsc Start(0)
15110 start 0
15111 Jan 03 08:34:05.060 INFO request completed, latency_us: 238, response_code: 204, uri: /start/cid/0, method: POST, req_id: ce3d8fce-5bdf-4844-a460-7ef255126b1b, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
15112 Replay: started 0, returned:Ok(())
15113 ----------------------------------------------------------------
15114 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15115 States: Offline Active Active
15116 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15117 4925 Acked Read 3 Sent Done Done false
15118 4926 Acked Write 7 Sent Done Done false
15119 4927 Acked Read 9 Sent Done Done false
15120 4928 Acked Read 9 Sent Done Done false
15121 4929 Acked Write 1 Sent Done Done false
15122 4930 Acked Read 3 Sent Done Done false
15123 4931 Acked Write 2 Sent Done Done false
15124 4932 Acked Read 6 Sent Done Done false
15125 4933 Acked Write 4 Sent Done Done false
15126 4934 Acked Write 3 Sent Done Done false
15127 4935 Acked Read 1 Sent Done Done false
15128 4936 Acked Flush 0 Sent Done Done false
15129 4937 Acked Read 2 Sent Done Done false
15130 4938 Acked Read 1 Sent Done Done false
15131 4939 Acked Read 2 Sent Done Done false
15132 4940 Acked Read 8 Sent Done Done false
15133 4941 Acked Write 1 Sent Done Done false
15134 4942 Acked Read 2 Sent Done Done false
15135 4943 Acked Flush 0 Sent Done Done false
15136 4944 Acked Read 1 Sent Done Done false
15137 4945 Acked Write 7 Sent Done Done false
15138 4946 Acked Write 8 Sent Done Done false
15139 4947 Acked Write 9 Sent Done Done false
15140 4948 Acked Write 9 Sent Done Done false
15141 4949 Acked Read 5 Sent Done Done false
15142 4950 Acked Read 2 Sent Done Done false
15143 4951 Acked Read 2 Sent Done Done false
15144 4952 Acked Write 1 Sent Done Done false
15145 4953 Acked Write 3 Sent Done Done false
15146 4954 Acked Write 2 Sent Done Done false
15147 4955 Acked Write 7 Sent Done Done false
15148 4956 Acked Read 5 Sent Done Done false
15149 4957 Acked Write 10 Sent Done Done false
15150 4958 Acked Write 5 Sent Done Done false
15151 4959 Acked Write 2 Sent Done Done false
15152 4960 Acked Read 1 Sent Done Done false
15153 4961 Acked Read 8 Sent Done Done false
15154 4962 Acked Read 6 Sent Done Done false
15155 4963 Acked Write 2 Sent Done Done false
15156 4964 Acked Read 3 Sent Done Done false
15157 4965 Acked Write 8 Sent Done Done false
15158 4966 Acked Read 5 Sent Done Done false
15159 4967 Acked Read 10 Sent Done Done false
15160 4968 Acked Write 5 Sent Done Done false
15161 4969 Acked Flush 0 Sent Done Done false
15162 4970 Acked Read 8 Sent Done Done false
15163 4971 Acked Read 9 Sent Done Done false
15164 4972 Acked Write 2 Sent Done Done false
15165 4973 Acked Write 6 Sent Done Done false
15166 4974 Acked Write 3 Sent Done Done false
15167 4975 Acked Write 10 Sent Done Done false
15168 4976 Acked Write 7 Sent Done Done false
15169 4977 Acked Read 10 Sent Done Done false
15170 4978 Acked Write 1 Sent Done Done false
15171 4979 Acked Read 2 Sent Done Done false
15172 4980 Acked Read 10 Sent Done Done false
15173 4981 Acked Read 10 Sent Done Done false
15174 4982 Acked Read 10 Sent Done Done false
15175 4983 Acked Write 9 Sent Done Done false
15176 4984 Acked Read 1 Sent Done Done false
15177 4985 Acked Read 4 Sent Done Done false
15178 4986 Acked Write 1 Sent Done Done false
15179 4987 Acked Flush 0 Sent Done Done false
15180 4988 Acked Write 8 Sent Done Done false
15181 4989 Acked Write 2 Sent Done Done false
15182 4990 Acked Write 9 Sent Done Done false
15183 4991 Acked Write 9 Sent Done Done false
15184 4992 Acked Flush 0 Sent Done Done false
15185 4993 Acked Read 10 Sent Done Done false
15186 4994 Acked Read 2 Sent Done Done false
15187 4995 Acked Read 2 Sent Done Done false
15188 4996 Acked Read 8 Sent Done Done false
15189 4997 Acked Read 5 Sent Done Done false
15190 4998 Acked Write 1 Sent Done Done false
15191 4999 Acked Read 9 Sent Done Done false
15192 5000 Acked Write 2 Sent Done Done false
15193 5001 Acked Flush 0 Sent Done Done false
15194 5002 Acked Read 1 Sent Done Done false
15195 5003 Acked Read 8 Sent Done Done false
15196 5004 Acked Read 1 Sent Done Done false
15197 5005 Acked Write 6 Sent Done Done false
15198 5006 Acked Read 1 Sent Done Done false
15199 5007 Acked Read 6 Sent Done Done false
15200 5008 Acked Write 8 Sent Done Done false
15201 5009 Acked Read 2 Sent Done Done false
15202 5010 Acked Flush 0 Sent Done Done false
15203 5011 Acked Read 7 Sent Done Done false
15204 5012 Acked Write 1 Sent Done Done false
15205 5013 Acked Read 10 Sent Done Done false
15206 5014 Acked Write 1 Sent Done Done false
15207 5015 Acked Read 10 Sent Done Done false
15208 5016 Acked Read 8 Sent Done Done false
15209 5017 Acked Read 10 Sent Done Done false
15210 5018 Acked Read 4 Sent Done Done false
15211 5019 Acked Read 9 Sent Done Done false
15212 5020 Acked Write 6 Sent Done Done false
15213 5021 Acked Write 8 Sent Done Done false
15214 5022 Acked Write 6 Sent Done Done false
15215 5023 Acked Write 9 Sent Done Done false
15216 5024 Acked Write 6 Sent Done Done false
15217 5025 Acked Read 7 Sent Done Done false
15218 5026 Acked Read 10 Sent Done Done false
15219 5027 Acked Write 3 Sent Done Done false
15220 5028 Acked Flush 0 Sent Done Done false
15221 5029 Acked Read 9 Sent Done Done false
15222 5030 Acked Write 2 Sent Done Done false
15223 5031 Acked Read 4 Sent Done Done false
15224 5032 Acked Read 3 Sent Done Done false
15225 5033 Acked Read 7 Sent Done Done false
15226 5034 Acked Write 8 Sent Done Done false
15227 5035 Acked Write 4 Sent Done Done false
15228 5036 Acked Read 5 Sent Done Done false
15229 5037 Acked Write 2 Sent Done Done false
15230 5038 Acked Write 6 Sent Done Done false
15231 5039 Acked Write 3 Sent Done Done false
15232 5040 Acked Flush 0 Sent Done Done false
15233 5041 Acked Read 4 Sent Done Done false
15234 5042 Acked Read 1 Sent Done Done false
15235 5043 Acked Write 7 Sent Done Done false
15236 5044 Acked Write 2 Sent Done Done false
15237 5045 Acked Flush 0 Sent Done Done false
15238 5046 Acked Read 6 Sent Done Done false
15239 5047 Acked Flush 0 Sent Done Done false
15240 5048 Acked Write 8 Sent Done Done false
15241 5049 Acked Flush 0 Sent Done Done false
15242 5050 Acked Read 10 Sent Done Done false
15243 5051 Acked Write 3 Sent Done Done false
15244 5052 Acked Flush 0 Sent Done Done false
15245 5053 Acked Read 3 Sent Done Done false
15246 5054 Acked Write 8 Sent Done Done false
15247 5055 Acked Read 3 Sent Done Done false
15248 5056 Acked Write 1 Sent Done Done false
15249 5057 Acked Read 5 Sent Done Done false
15250 5058 Acked Read 8 Sent Done Done false
15251 5059 Acked Write 1 Sent Done Done false
15252 5060 Acked Flush 0 Sent Done Done false
15253 5061 Acked Read 3 Sent Done Done false
15254 5062 Acked Read 1 Sent Done Done false
15255 5063 Acked Flush 0 Sent Done Done false
15256 5064 Acked Write 5 Sent Done Done false
15257 5065 Acked Read 6 Sent Done Done false
15258 5066 Acked Write 6 Sent Done Done false
15259 5067 Acked Write 1 Sent Done Done false
15260 5068 Acked Read 5 Sent Done Done false
15261 5069 Acked Write 7 Sent Done Done false
15262 5070 Acked Write 9 Sent Done Done false
15263 5071 Acked Read 7 Sent Done Done false
15264 5072 Acked Read 7 Sent Done Done false
15265 5073 Acked Write 5 Sent Done Done false
15266 5074 Acked Read 3 Sent Done Done false
15267 5075 Acked Write 3 Sent Done Done false
15268 5076 Acked Read 10 Sent Done Done false
15269 5077 Acked Read 6 Sent Done Done false
15270 5078 Acked Write 8 Sent Done Done false
15271 5079 Acked Write 4 Sent Done Done false
15272 5080 Acked Read 9 Sent Done Done false
15273 5081 Acked Read 1 Sent Done Done false
15274 5082 Acked Read 4 Sent Done Done false
15275 5083 Acked Read 3 Sent Done Done false
15276 5084 Acked Write 7 Sent Done Done false
15277 5085 Acked Read 3 Sent Done Done false
15278 5086 Acked Write 7 Sent Done Done false
15279 5087 Acked Read 9 Sent Done Done false
15280 5088 Acked Read 6 Sent Done Done false
15281 5089 Acked Read 1 Sent Done Done false
15282 5090 Acked Read 3 Sent Done Done false
15283 5091 Acked Read 6 Sent Done Done false
15284 5092 Acked Write 6 Sent Done Done false
15285 5093 Acked Read 9 Sent Done Done false
15286 5094 Acked Write 5 Sent Done Done false
15287 5095 Acked Write 2 Sent Done Done false
15288 5096 Acked Write 6 Sent Done Done false
15289 5097 Acked Write 5 Sent Done Done false
15290 5098 Acked Write 10 Sent Done Done false
15291 5099 Acked Write 2 Sent Done Done false
15292 5100 Acked Write 7 Sent Done Done false
15293 5101 Acked Flush 0 Sent Done Done false
15294 5102 Acked Read 9 Sent Done Done false
15295 5103 Acked Write 10 Sent Done Done false
15296 5104 Acked Read 9 Sent Done Done false
15297 5105 Acked Write 2 Sent Done Done false
15298 5106 Acked Read 4 Sent Done Done false
15299 5107 Acked Write 1 Sent Done Done false
15300 5108 Acked Write 4 Sent Done Done false
15301 5109 Acked Write 2 Sent Done Done false
15302 5110 Acked Read 5 Sent Done Done false
15303 5111 Acked Write 5 Sent Done Done false
15304 5112 Acked Write 9 Sent Done Done false
15305 5113 Acked Read 2 Sent Done Done false
15306 5114 Acked Read 3 Sent Done Done false
15307 5115 Acked Write 2 Sent Done Done false
15308 5116 Acked Write 7 Sent Done Done false
15309 5117 Acked Flush 0 Sent Done Done false
15310 5118 Acked Flush 0 Sent Done Done false
15311 5119 Acked Read 10 Sent Done Done false
15312 5120 Acked Read 1 Sent Done Done false
15313 5121 Acked Write 3 Sent Done Done false
15314 5122 Acked Write 4 Sent Done Done false
15315 5123 Acked Read 7 Sent Done Done false
15316 5124 Acked Write 1 Sent Done Done false
15317 5125 Acked Write 7 Sent Done Done false
15318 5126 Acked Flush 0 Sent Done Done false
15319 5127 Acked Write 9 Sent Done Done false
15320 5128 Acked Read 6 Sent Done Done false
15321 5129 Acked Read 7 Sent Done Done false
15322 5130 Acked Read 2 Sent Done Done false
15323 5131 Acked Read 4 Sent Done Done false
15324 5132 Acked Read 10 Sent Done Done false
15325 5133 Acked Write 1 Sent Done Done false
15326 5134 Acked Write 6 Sent Done Done false
15327 5135 Acked Read 2 Sent Done Done false
15328 5136 Acked Read 5 Sent Done Done false
15329 5137 Acked Write 6 Sent Done Done false
15330 5138 Acked Read 8 Sent Done Done false
15331 5139 Acked Read 3 Sent Done Done false
15332 5140 Acked Write 5 Sent Done Done false
15333 5141 Acked Write 8 Sent Done Done false
15334 5142 Acked Write 1 Sent Done Done false
15335 5143 Acked Read 6 Sent Done Done false
15336 5144 Acked Read 2 Sent Done Done false
15337 5145 Acked Read 10 Sent Done Done false
15338 5146 Acked Read 8 Sent Done Done false
15339 5147 Acked Write 7 Sent Done Done false
15340 5148 Acked Write 2 Sent Done Done false
15341 5149 Acked Read 6 Sent Done Done false
15342 5150 Acked Read 8 Sent Done Done false
15343 5151 Acked Write 5 Sent Done Done false
15344 5152 Acked Read 10 Sent Done Done false
15345 5153 Acked Flush 0 Sent Done Done false
15346 5154 Acked Flush 0 Sent Done Done false
15347 5155 Acked Write 5 Sent Done Done false
15348 5156 Acked Write 6 Sent Done Done false
15349 5157 Acked Read 7 Sent Done Done false
15350 5158 Acked Read 6 Sent Done Done false
15351 5159 Acked Write 9 Sent Done Done false
15352 5160 Acked Write 5 Sent Done Done false
15353 5161 Acked Write 3 Sent Done Done false
15354 5162 Acked Read 5 Sent Done Done false
15355 5163 Acked Read 1 Sent Done Done false
15356 5164 Acked Write 9 Sent Done Done false
15357 5165 Acked Write 2 Sent Done Done false
15358 5166 Acked Flush 0 Sent Done Done false
15359 5167 Acked Read 5 Sent Done Done false
15360 5168 Acked Read 4 Sent Done Done false
15361 5169 Acked Read 9 Sent Done Done false
15362 5170 Acked Write 4 Sent Done Done false
15363 5171 Acked Read 9 Sent Done Done false
15364 5172 Acked Flush 0 Sent Done Done false
15365 5173 Acked Read 7 Sent Done Done false
15366 5174 Acked Write 1 Sent Done Done false
15367 5175 Acked Read 9 Sent Done Done false
15368 5176 Acked Write 2 Sent Done Done false
15369 5177 Acked Write 6 Sent Done Done false
15370 5178 Acked Write 1 Sent Done Done false
15371 5179 Acked Read 1 Sent Done Done false
15372 5180 Acked Read 6 Sent Done Done false
15373 5181 Acked Read 6 Sent Done Done false
15374 5182 Acked Read 4 Sent Done Done false
15375 5183 Acked Write 4 Sent Done Done false
15376 5184 Acked Read 5 Sent Done Done false
15377 5185 Acked Read 6 Sent Done Done false
15378 5186 Acked Write 7 Sent Done Done false
15379 5187 Acked Write 4 Sent Done Done false
15380 5188 Acked Flush 0 Sent Done Done false
15381 5189 Acked Write 3 Sent Done Done false
15382 5190 Acked Read 3 Sent Done Done false
15383 5191 Acked Write 7 Sent Done Done false
15384 5192 Acked Read 4 Sent Done Done false
15385 5193 Acked Write 5 Sent Done Done false
15386 5194 Acked Write 9 Sent Done Done false
15387 5195 Acked Write 10 Sent Done Done false
15388 5196 Acked Write 4 Sent Done Done false
15389 5197 Acked Read 8 Sent Done Done false
15390 5198 Acked Read 1 Sent Done Done false
15391 5199 Acked Read 9 Sent Done Done false
15392 5200 Acked Read 8 Sent Done Done false
15393 5201 Acked Write 4 Sent Done Done false
15394 5202 Acked Read 5 Sent Done Done false
15395 5203 Acked Read 6 Sent Done Done false
15396 5204 Acked Write 2 Sent Done Done false
15397 5205 Acked Read 10 Sent Done Done false
15398 5206 Acked Flush 0 Sent Done Done false
15399 5207 Acked Write 10 Sent Done Done false
15400 5208 Acked Write 10 Sent Done Done false
15401 5209 Acked Write 8 Sent Done Done false
15402 5210 Acked Write 8 Sent Done Done false
15403 5211 Acked Read 9 Sent Done Done false
15404 5212 Acked Read 8 Sent Done Done false
15405 5213 Acked Read 3 Sent Done Done false
15406 5214 Acked Read 1 Sent Done Done false
15407 5215 Acked Read 3 Sent Done Done false
15408 5216 Acked Write 5 Sent Done Done false
15409 5217 Acked Read 7 Sent Done Done false
15410 5218 Acked Write 6 Sent Done Done false
15411 5219 Acked Write 9 Sent Done Done false
15412 5220 Acked Write 9 Sent Done Done false
15413 5221 Acked Read 5 Sent Done Done false
15414 5222 Acked Write 4 Sent Done Done false
15415 5223 Acked Flush 0 Sent Done Done false
15416 5224 Acked Read 1 Sent Done Done false
15417 5225 Acked Read 8 Sent Done Done false
15418 STATES DS:0 DS:1 DS:2 TOTAL
15419 Sent 301 0 0 301
15420 Done 0 301 301 602
15421 Skipped 0 0 0 0
15422 Error 0 0 0 0
15423 Last Flush: 4924 5223 5223
15424 Downstairs last five completed: 4924 4923 4922 4921 4920
15425 Upstairs last five completed: 5225 5224 5223 5222 5221
15426 CLIENT: Up:0 ds:301 act:2
15427 [8810] got start action
15428 [8810] I am going to restart
15429 Starting downstairs at port 8810
15430 Make output file at "/tmp/dsc/downstairs-8810.txt"
15431 [8810][0] reports Starting
15432 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
15433 [8810][0] reports Running
15434 ----------------------------------------------------------------
15435 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15436 States: Offline Active Active
15437 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15438 4925 Acked Read 3 Sent Done Done false
15439 4926 Acked Write 7 Sent Done Done false
15440 4927 Acked Read 9 Sent Done Done false
15441 4928 Acked Read 9 Sent Done Done false
15442 4929 Acked Write 1 Sent Done Done false
15443 4930 Acked Read 3 Sent Done Done false
15444 4931 Acked Write 2 Sent Done Done false
15445 4932 Acked Read 6 Sent Done Done false
15446 4933 Acked Write 4 Sent Done Done false
15447 4934 Acked Write 3 Sent Done Done false
15448 4935 Acked Read 1 Sent Done Done false
15449 4936 Acked Flush 0 Sent Done Done false
15450 4937 Acked Read 2 Sent Done Done false
15451 4938 Acked Read 1 Sent Done Done false
15452 4939 Acked Read 2 Sent Done Done false
15453 4940 Acked Read 8 Sent Done Done false
15454 4941 Acked Write 1 Sent Done Done false
15455 4942 Acked Read 2 Sent Done Done false
15456 4943 Acked Flush 0 Sent Done Done false
15457 4944 Acked Read 1 Sent Done Done false
15458 4945 Acked Write 7 Sent Done Done false
15459 4946 Acked Write 8 Sent Done Done false
15460 4947 Acked Write 9 Sent Done Done false
15461 4948 Acked Write 9 Sent Done Done false
15462 4949 Acked Read 5 Sent Done Done false
15463 4950 Acked Read 2 Sent Done Done false
15464 4951 Acked Read 2 Sent Done Done false
15465 4952 Acked Write 1 Sent Done Done false
15466 4953 Acked Write 3 Sent Done Done false
15467 4954 Acked Write 2 Sent Done Done false
15468 4955 Acked Write 7 Sent Done Done false
15469 4956 Acked Read 5 Sent Done Done false
15470 4957 Acked Write 10 Sent Done Done false
15471 4958 Acked Write 5 Sent Done Done false
15472 4959 Acked Write 2 Sent Done Done false
15473 4960 Acked Read 1 Sent Done Done false
15474 4961 Acked Read 8 Sent Done Done false
15475 4962 Acked Read 6 Sent Done Done false
15476 4963 Acked Write 2 Sent Done Done false
15477 4964 Acked Read 3 Sent Done Done false
15478 4965 Acked Write 8 Sent Done Done false
15479 4966 Acked Read 5 Sent Done Done false
15480 4967 Acked Read 10 Sent Done Done false
15481 4968 Acked Write 5 Sent Done Done false
15482 4969 Acked Flush 0 Sent Done Done false
15483 4970 Acked Read 8 Sent Done Done false
15484 4971 Acked Read 9 Sent Done Done false
15485 4972 Acked Write 2 Sent Done Done false
15486 4973 Acked Write 6 Sent Done Done false
15487 4974 Acked Write 3 Sent Done Done false
15488 4975 Acked Write 10 Sent Done Done false
15489 4976 Acked Write 7 Sent Done Done false
15490 4977 Acked Read 10 Sent Done Done false
15491 4978 Acked Write 1 Sent Done Done false
15492 4979 Acked Read 2 Sent Done Done false
15493 4980 Acked Read 10 Sent Done Done false
15494 4981 Acked Read 10 Sent Done Done false
15495 4982 Acked Read 10 Sent Done Done false
15496 4983 Acked Write 9 Sent Done Done false
15497 4984 Acked Read 1 Sent Done Done false
15498 4985 Acked Read 4 Sent Done Done false
15499 4986 Acked Write 1 Sent Done Done false
15500 4987 Acked Flush 0 Sent Done Done false
15501 4988 Acked Write 8 Sent Done Done false
15502 4989 Acked Write 2 Sent Done Done false
15503 4990 Acked Write 9 Sent Done Done false
15504 4991 Acked Write 9 Sent Done Done false
15505 4992 Acked Flush 0 Sent Done Done false
15506 4993 Acked Read 10 Sent Done Done false
15507 4994 Acked Read 2 Sent Done Done false
15508 4995 Acked Read 2 Sent Done Done false
15509 4996 Acked Read 8 Sent Done Done false
15510 4997 Acked Read 5 Sent Done Done false
15511 4998 Acked Write 1 Sent Done Done false
15512 4999 Acked Read 9 Sent Done Done false
15513 5000 Acked Write 2 Sent Done Done false
15514 5001 Acked Flush 0 Sent Done Done false
15515 5002 Acked Read 1 Sent Done Done false
15516 5003 Acked Read 8 Sent Done Done false
15517 5004 Acked Read 1 Sent Done Done false
15518 5005 Acked Write 6 Sent Done Done false
15519 5006 Acked Read 1 Sent Done Done false
15520 5007 Acked Read 6 Sent Done Done false
15521 5008 Acked Write 8 Sent Done Done false
15522 5009 Acked Read 2 Sent Done Done false
15523 5010 Acked Flush 0 Sent Done Done false
15524 5011 Acked Read 7 Sent Done Done false
15525 5012 Acked Write 1 Sent Done Done false
15526 5013 Acked Read 10 Sent Done Done false
15527 5014 Acked Write 1 Sent Done Done false
15528 5015 Acked Read 10 Sent Done Done false
15529 5016 Acked Read 8 Sent Done Done false
15530 5017 Acked Read 10 Sent Done Done false
15531 5018 Acked Read 4 Sent Done Done false
15532 5019 Acked Read 9 Sent Done Done false
15533 5020 Acked Write 6 Sent Done Done false
15534 5021 Acked Write 8 Sent Done Done false
15535 5022 Acked Write 6 Sent Done Done false
15536 5023 Acked Write 9 Sent Done Done false
15537 5024 Acked Write 6 Sent Done Done false
15538 5025 Acked Read 7 Sent Done Done false
15539 5026 Acked Read 10 Sent Done Done false
15540 5027 Acked Write 3 Sent Done Done false
15541 5028 Acked Flush 0 Sent Done Done false
15542 5029 Acked Read 9 Sent Done Done false
15543 5030 Acked Write 2 Sent Done Done false
15544 5031 Acked Read 4 Sent Done Done false
15545 5032 Acked Read 3 Sent Done Done false
15546 5033 Acked Read 7 Sent Done Done false
15547 5034 Acked Write 8 Sent Done Done false
15548 5035 Acked Write 4 Sent Done Done false
15549 5036 Acked Read 5 Sent Done Done false
15550 5037 Acked Write 2 Sent Done Done false
15551 5038 Acked Write 6 Sent Done Done false
15552 5039 Acked Write 3 Sent Done Done false
15553 5040 Acked Flush 0 Sent Done Done false
15554 5041 Acked Read 4 Sent Done Done false
15555 5042 Acked Read 1 Sent Done Done false
15556 5043 Acked Write 7 Sent Done Done false
15557 5044 Acked Write 2 Sent Done Done false
15558 5045 Acked Flush 0 Sent Done Done false
15559 5046 Acked Read 6 Sent Done Done false
15560 5047 Acked Flush 0 Sent Done Done false
15561 5048 Acked Write 8 Sent Done Done false
15562 5049 Acked Flush 0 Sent Done Done false
15563 5050 Acked Read 10 Sent Done Done false
15564 5051 Acked Write 3 Sent Done Done false
15565 5052 Acked Flush 0 Sent Done Done false
15566 5053 Acked Read 3 Sent Done Done false
15567 5054 Acked Write 8 Sent Done Done false
15568 5055 Acked Read 3 Sent Done Done false
15569 5056 Acked Write 1 Sent Done Done false
15570 5057 Acked Read 5 Sent Done Done false
15571 5058 Acked Read 8 Sent Done Done false
15572 5059 Acked Write 1 Sent Done Done false
15573 5060 Acked Flush 0 Sent Done Done false
15574 5061 Acked Read 3 Sent Done Done false
15575 5062 Acked Read 1 Sent Done Done false
15576 5063 Acked Flush 0 Sent Done Done false
15577 5064 Acked Write 5 Sent Done Done false
15578 5065 Acked Read 6 Sent Done Done false
15579 5066 Acked Write 6 Sent Done Done false
15580 5067 Acked Write 1 Sent Done Done false
15581 5068 Acked Read 5 Sent Done Done false
15582 5069 Acked Write 7 Sent Done Done false
15583 5070 Acked Write 9 Sent Done Done false
15584 5071 Acked Read 7 Sent Done Done false
15585 5072 Acked Read 7 Sent Done Done false
15586 5073 Acked Write 5 Sent Done Done false
15587 5074 Acked Read 3 Sent Done Done false
15588 5075 Acked Write 3 Sent Done Done false
15589 5076 Acked Read 10 Sent Done Done false
15590 5077 Acked Read 6 Sent Done Done false
15591 5078 Acked Write 8 Sent Done Done false
15592 5079 Acked Write 4 Sent Done Done false
15593 5080 Acked Read 9 Sent Done Done false
15594 5081 Acked Read 1 Sent Done Done false
15595 5082 Acked Read 4 Sent Done Done false
15596 5083 Acked Read 3 Sent Done Done false
15597 5084 Acked Write 7 Sent Done Done false
15598 5085 Acked Read 3 Sent Done Done false
15599 5086 Acked Write 7 Sent Done Done false
15600 5087 Acked Read 9 Sent Done Done false
15601 5088 Acked Read 6 Sent Done Done false
15602 5089 Acked Read 1 Sent Done Done false
15603 5090 Acked Read 3 Sent Done Done false
15604 5091 Acked Read 6 Sent Done Done false
15605 5092 Acked Write 6 Sent Done Done false
15606 5093 Acked Read 9 Sent Done Done false
15607 5094 Acked Write 5 Sent Done Done false
15608 5095 Acked Write 2 Sent Done Done false
15609 5096 Acked Write 6 Sent Done Done false
15610 5097 Acked Write 5 Sent Done Done false
15611 5098 Acked Write 10 Sent Done Done false
15612 5099 Acked Write 2 Sent Done Done false
15613 5100 Acked Write 7 Sent Done Done false
15614 5101 Acked Flush 0 Sent Done Done false
15615 5102 Acked Read 9 Sent Done Done false
15616 5103 Acked Write 10 Sent Done Done false
15617 5104 Acked Read 9 Sent Done Done false
15618 5105 Acked Write 2 Sent Done Done false
15619 5106 Acked Read 4 Sent Done Done false
15620 5107 Acked Write 1 Sent Done Done false
15621 5108 Acked Write 4 Sent Done Done false
15622 5109 Acked Write 2 Sent Done Done false
15623 5110 Acked Read 5 Sent Done Done false
15624 5111 Acked Write 5 Sent Done Done false
15625 5112 Acked Write 9 Sent Done Done false
15626 5113 Acked Read 2 Sent Done Done false
15627 5114 Acked Read 3 Sent Done Done false
15628 5115 Acked Write 2 Sent Done Done false
15629 5116 Acked Write 7 Sent Done Done false
15630 5117 Acked Flush 0 Sent Done Done false
15631 5118 Acked Flush 0 Sent Done Done false
15632 5119 Acked Read 10 Sent Done Done false
15633 5120 Acked Read 1 Sent Done Done false
15634 5121 Acked Write 3 Sent Done Done false
15635 5122 Acked Write 4 Sent Done Done false
15636 5123 Acked Read 7 Sent Done Done false
15637 5124 Acked Write 1 Sent Done Done false
15638 5125 Acked Write 7 Sent Done Done false
15639 5126 Acked Flush 0 Sent Done Done false
15640 5127 Acked Write 9 Sent Done Done false
15641 5128 Acked Read 6 Sent Done Done false
15642 5129 Acked Read 7 Sent Done Done false
15643 5130 Acked Read 2 Sent Done Done false
15644 5131 Acked Read 4 Sent Done Done false
15645 5132 Acked Read 10 Sent Done Done false
15646 5133 Acked Write 1 Sent Done Done false
15647 5134 Acked Write 6 Sent Done Done false
15648 5135 Acked Read 2 Sent Done Done false
15649 5136 Acked Read 5 Sent Done Done false
15650 5137 Acked Write 6 Sent Done Done false
15651 5138 Acked Read 8 Sent Done Done false
15652 5139 Acked Read 3 Sent Done Done false
15653 5140 Acked Write 5 Sent Done Done false
15654 5141 Acked Write 8 Sent Done Done false
15655 5142 Acked Write 1 Sent Done Done false
15656 5143 Acked Read 6 Sent Done Done false
15657 5144 Acked Read 2 Sent Done Done false
15658 5145 Acked Read 10 Sent Done Done false
15659 5146 Acked Read 8 Sent Done Done false
15660 5147 Acked Write 7 Sent Done Done false
15661 5148 Acked Write 2 Sent Done Done false
15662 5149 Acked Read 6 Sent Done Done false
15663 5150 Acked Read 8 Sent Done Done false
15664 5151 Acked Write 5 Sent Done Done false
15665 5152 Acked Read 10 Sent Done Done false
15666 5153 Acked Flush 0 Sent Done Done false
15667 5154 Acked Flush 0 Sent Done Done false
15668 5155 Acked Write 5 Sent Done Done false
15669 5156 Acked Write 6 Sent Done Done false
15670 5157 Acked Read 7 Sent Done Done false
15671 5158 Acked Read 6 Sent Done Done false
15672 5159 Acked Write 9 Sent Done Done false
15673 5160 Acked Write 5 Sent Done Done false
15674 5161 Acked Write 3 Sent Done Done false
15675 5162 Acked Read 5 Sent Done Done false
15676 5163 Acked Read 1 Sent Done Done false
15677 5164 Acked Write 9 Sent Done Done false
15678 5165 Acked Write 2 Sent Done Done false
15679 5166 Acked Flush 0 Sent Done Done false
15680 5167 Acked Read 5 Sent Done Done false
15681 5168 Acked Read 4 Sent Done Done false
15682 5169 Acked Read 9 Sent Done Done false
15683 5170 Acked Write 4 Sent Done Done false
15684 5171 Acked Read 9 Sent Done Done false
15685 5172 Acked Flush 0 Sent Done Done false
15686 5173 Acked Read 7 Sent Done Done false
15687 5174 Acked Write 1 Sent Done Done false
15688 5175 Acked Read 9 Sent Done Done false
15689 5176 Acked Write 2 Sent Done Done false
15690 5177 Acked Write 6 Sent Done Done false
15691 5178 Acked Write 1 Sent Done Done false
15692 5179 Acked Read 1 Sent Done Done false
15693 5180 Acked Read 6 Sent Done Done false
15694 5181 Acked Read 6 Sent Done Done false
15695 5182 Acked Read 4 Sent Done Done false
15696 5183 Acked Write 4 Sent Done Done false
15697 5184 Acked Read 5 Sent Done Done false
15698 5185 Acked Read 6 Sent Done Done false
15699 5186 Acked Write 7 Sent Done Done false
15700 5187 Acked Write 4 Sent Done Done false
15701 5188 Acked Flush 0 Sent Done Done false
15702 5189 Acked Write 3 Sent Done Done false
15703 5190 Acked Read 3 Sent Done Done false
15704 5191 Acked Write 7 Sent Done Done false
15705 5192 Acked Read 4 Sent Done Done false
15706 5193 Acked Write 5 Sent Done Done false
15707 5194 Acked Write 9 Sent Done Done false
15708 5195 Acked Write 10 Sent Done Done false
15709 5196 Acked Write 4 Sent Done Done false
15710 5197 Acked Read 8 Sent Done Done false
15711 5198 Acked Read 1 Sent Done Done false
15712 5199 Acked Read 9 Sent Done Done false
15713 5200 Acked Read 8 Sent Done Done false
15714 5201 Acked Write 4 Sent Done Done false
15715 5202 Acked Read 5 Sent Done Done false
15716 5203 Acked Read 6 Sent Done Done false
15717 5204 Acked Write 2 Sent Done Done false
15718 5205 Acked Read 10 Sent Done Done false
15719 5206 Acked Flush 0 Sent Done Done false
15720 5207 Acked Write 10 Sent Done Done false
15721 5208 Acked Write 10 Sent Done Done false
15722 5209 Acked Write 8 Sent Done Done false
15723 5210 Acked Write 8 Sent Done Done false
15724 5211 Acked Read 9 Sent Done Done false
15725 5212 Acked Read 8 Sent Done Done false
15726 5213 Acked Read 3 Sent Done Done false
15727 5214 Acked Read 1 Sent Done Done false
15728 5215 Acked Read 3 Sent Done Done false
15729 5216 Acked Write 5 Sent Done Done false
15730 5217 Acked Read 7 Sent Done Done false
15731 5218 Acked Write 6 Sent Done Done false
15732 5219 Acked Write 9 Sent Done Done false
15733 5220 Acked Write 9 Sent Done Done false
15734 5221 Acked Read 5 Sent Done Done false
15735 5222 Acked Write 4 Sent Done Done false
15736 5223 Acked Flush 0 Sent Done Done false
15737 5224 Acked Read 1 Sent Done Done false
15738 5225 Acked Read 8 Sent Done Done false
15739 5226 Acked Flush 0 Sent Done Done false
15740 STATES DS:0 DS:1 DS:2 TOTAL
15741 Sent 302 0 0 302
15742 Done 0 302 302 604
15743 Skipped 0 0 0 0
15744 Error 0 0 0 0
15745 Last Flush: 4924 5226 5226
15746 Downstairs last five completed: 4924 4923 4922 4921 4920
15747 Upstairs last five completed: 5226 5225 5224 5223 5222
15748 CLIENT: Up:0 ds:302 act:2
157492025-01-03T08:34:10.827ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157502025-01-03T08:34:10.827ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54262) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157512025-01-03T08:34:10.828ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157522025-01-03T08:34:10.829ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157532025-01-03T08:34:10.829ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157542025-01-03T08:34:10.829ZINFOcrucible: send last flush ID to this DS: 4924 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157552025-01-03T08:34:10.829ZINFOcrucible: Replied this last flush ID: 4924 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157562025-01-03T08:34:10.829ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157572025-01-03T08:34:10.829ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157582025-01-03T08:34:10.829ZINFOcrucible: [0] client re-new 302 jobs since flush 4924 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157592025-01-03T08:34:10.829ZINFOcrucible: [0] Replayed 302 jobs since flush: 4924 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
15760 ----------------------------------------------------------------
15761 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15762 Downstairs last five completed: 5226 5225 5224 5223 5222
15763 Upstairs last five completed: 5226 5225 5224 5223 5222
15764 CLIENT: Up:0 ds:0 act:3
15765 Replay: All jobs finished
15766 Main task has work to do, go find it
15767 got dsc Stop(2)
15768 stop 2
15769 Jan 03 08:34:13.064 INFO request completed, latency_us: [8830] Got stop action so:false kr:false
15770 317, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 74ead4f2-fb1e-4eef-9c69-60a917454563, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
157712025-01-03T08:34:13.065ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157722025-01-03T08:34:13.065ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157732025-01-03T08:34:13.065ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157742025-01-03T08:34:13.065ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157752025-01-03T08:34:13.065ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
157762025-01-03T08:34:13.065ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
15777 Jan 03 08:34:13.065 INFO request completed, latency_us: 160, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4291b384-0009-45a4-96b8-5c0312ef8a36, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
15778 [8830] Exited with: ExitStatus(unix_wait_status(9))
15779 [8830][2] reports Exit
15780 Jan 03 08:34:17.068 INFO request completed, latency_us: 186, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8496e8a1-322f-46e9-a7d0-0555a74fe327, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
15781 001/300 Write block 1304 len 28672 data: 4 4 3 3 3 3 3
15782 002/300 Read block 3745 len 12288
15783 003/300 Read block 1245 len 32768
15784 004/300 Write block 4361 len 36864 data: 5 5 5 4 5 5 5 5 4
15785 005/300 Read block 379 len 8192
15786 006/300 Read block 1337 len 16384
15787 007/300 Write block 3557 len 4096 data: 2
15788 008/300 Read block 219 len 36864
15789 009/300 Read block 3788 len 20480
15790 010/300 Read block 1514 len 28672
15791 011/300 Read block 2202 len 12288
15792 012/300 Write block 2068 len 32768 data: 5 5 5 5 5 6 5 5
15793 013/300 Write block 2815 len 40960 data: 6 6 5 5 3 3 3 3 3 2
15794 014/300 Write block 4965 len 36864 data: 4 4 4 4 3 3 4 5 4
15795 015/300 Write block 2558 len 32768 data: 2 3 3 3 3 5 5 5
15796 016/300 Read block 876 len 40960
15797 017/300 Read block 4446 len 12288
15798 018/300 Write block 4802 len 24576 data: 4 4 4 4 4 4
15799 019/300 Write block 4086 len 12288 data: 2 2 2
15800 020/300 Read block 1359 len 40960
15801 021/300 Write block 2925 len 12288 data: 5 5 5
15802 022/300 Read block 2218 len 24576
15803 023/300 Read block 3414 len 36864
15804 024/300 Read block 3753 len 24576
15805 025/300 Read block 4127 len 36864
15806 026/300 Read block 1773 len 32768
15807 027/300 Write block 3045 len 40960 data: 4 5 5 5 4 3 3 3 4 4
15808 028/300 Write block 1598 len 40960 data: 5 5 5 5 4 5 4 4 4 4
15809 029/300 Write block 4603 len 16384 data: 5 5 5 4
15810 030/300 Read block 2074 len 20480
15811 031/300 Read block 3910 len 12288
15812 032/300 Flush
15813 033/300 Write block 4191 len 40960 data: 2 3 3 2 3 3 5 7 8 8
15814 034/300 Read block 1148 len 40960
15815 035/300 Write block 2856 len 24576 data: 4 5 4 4 4 4
15816 036/300 Write block 1910 len 20480 data: 4 4 4 4 4
15817 037/300 Read block 3427 len 20480
15818 038/300 Read block 4812 len 20480
15819 039/300 Write block 205 len 20480 data: 2 4 2 2 2
15820 040/300 Read block 4095 len 16384
15821 041/300 Write block 1630 len 4096 data: 3
15822 042/300 Read block 43 len 4096
15823 043/300 Flush
15824 044/300 Write block 3053 len 16384 data: 5 5 2 2
15825 045/300 Write block 1108 len 40960 data: 3 3 2 2 2 2 2 3 3 5
15826 046/300 Write block 1825 len 32768 data: 4 5 6 6 6 7 5 4
15827 047/300 Write block 4067 len 32768 data: 4 4 4 4 4 4 6 5
15828 048/300 Read block 556 len 20480
15829 049/300 Write block 3476 len 36864 data: 4 4 4 4 3 2 2 2 2
15830 050/300 Read block 4971 len 40960
15831 051/300 Read block 2702 len 36864
15832 052/300 Write block 4252 len 36864 data: 5 5 5 6 5 4 4 4 3
15833 053/300 Read block 2668 len 4096
15834 054/300 Read block 2362 len 40960
15835 055/300 Read block 1149 len 16384
15836 056/300 Write block 4378 len 24576 data: 2 2 2 3 3 3
15837 057/300 Write block 3253 len 8192 data: 3 2
15838 058/300 Flush
15839 059/300 Write block 3327 len 32768 data: 3 3 3 3 4 4 4 4
15840 060/300 Write block 4825 len 24576 data: 3 3 3 4 3 3
15841 061/300 Read block 1499 len 32768
15842 062/300 Read block 4030 len 36864
15843 063/300 Write block 945 len 8192 data: 3 4
15844 064/300 Write block 1141 len 16384 data: 3 4 4 4
15845 065/300 Read block 2777 len 20480
15846 066/300 Read block 3507 len 8192
15847 067/300 Write block 1119 len 16384 data: 6 6 7 7
15848 068/300 Read block 4808 len 28672
15849 069/300 Write block 999 len 28672 data: 5 4 4 2 2 3 3
15850 070/300 Read block 3295 len 24576
15851 071/300 Read block 4494 len 28672
15852 072/300 Read block 3457 len 4096
15853 073/300 Write block 1108 len 40960 data: 4 4 3 3 3 3 3 4 4 6
15854 074/300 Write block 1773 len 28672 data: 2 3 3 2 2 2 3
15855 075/300 Read block 3374 len 32768
15856 076/300 Flush
15857 077/300 Read block 4407 len 40960
15858 078/300 Read block 1359 len 8192
15859 079/300 Read block 3029 len 4096
15860 080/300 Read block 4518 len 40960
15861 081/300 Read block 2213 len 8192
15862 082/300 Read block 1578 len 24576
15863 083/300 Read block 689 len 12288
15864 084/300 Write block 4177 len 20480 data: 4 4 4 4 3
15865 085/300 Write block 3554 len 28672 data: 2 3 3 3 2 3 3
15866 086/300 Read block 4368 len 12288
15867 087/300 Write block 3125 len 28672 data: 3 4 4 3 4 3 3
15868 088/300 Write block 1649 len 28672 data: 4 4 5 4 4 3 3
15869 089/300 Write block 2154 len 28672 data: 4 4 4 4 4 3 3
15870 090/300 Flush
15871 091/300 Write block 1106 len 36864 data: 3 3 5 5 4 4 4 4 4
15872 092/300 Write block 4225 len 36864 data: 3 3 3 3 3 2 2 2 2
15873 093/300 Write block 4345 len 28672 data: 5 4 4 4 4 4 2
15874 094/300 Read block 2738 len 8192
15875 095/300 Read block 998 len 16384
15876 096/300 Flush
15877 097/300 Write block 1421 len 12288 data: 5 5 5
15878 098/300 Write block 3145 len 16384 data: 5 5 5 5
15879 099/300 Write block 2193 len 16384 data: 2 2 2 2
15880 100/300 Read block 4580 len 24576
15881 101/300 Write block 4156 len 4096 data: 4
15882 102/300 Read block 4444 len 12288
15883 103/300 Read block 3921 len 24576
15884 104/300 Read block 619 len 24576
15885 105/300 Read block 4758 len 16384
15886 106/300 Write block 416 len 36864 data: 4 5 4 5 5 5 5 4 4
15887 107/300 Write block 1422 len 36864 data: 6 6 5 4 4 3 2 2 2
15888 108/300 Read block 3846 len 12288
15889 109/300 Read block 1571 len 28672
15890 110/300 Write block 2168 len 4096 data: 5
15891 111/300 Flush
15892 112/300 Flush
15893 113/300 Write block 1569 len 20480 data: 3 3 3 4 4
15894 114/300 Write block 1706 len 40960 data: 4 4 2 3 3 3 3 4 3 3
15895 115/300 Write block 4176 len 40960 data: 4 5 5 5 5 4 3 4 4 3
15896 116/300 Flush
15897 117/300 Write block 2734 len 4096 data: 7
15898 118/300 Write block 2219 len 24576 data: 4 4 5 5 4 4
15899 119/300 Read block 4493 len 36864
15900 120/300 Write block 999 len 8192 data: 6 5
15901 121/300 Write block 322 len 36864 data: 2 2 2 2 3 3 4 5 5
15902 122/300 Read block 2539 len 4096
15903 123/300 Write block 4222 len 32768 data: 3 4 3 4 4 4 4 4
15904 124/300 Write block 864 len 28672 data: 2 3 3 3 3 3 3
15905 125/300 Read block 3332 len 20480
15906 126/300 Write block 2466 len 16384 data: 2 3 3 3
15907 127/300 Read block 1615 len 8192
15908 128/300 Read block 2723 len 24576
15909 129/300 Write block 348 len 8192 data: 6 6
15910 130/300 Write block 3023 len 20480 data: 4 3 2 2 2
15911 131/300 Write block 1204 len 32768 data: 3 4 4 4 2 2 2 2
15912 132/300 Write block 3190 len 16384 data: 3 4 4 4
15913 133/300 Flush
15914 134/300 Read block 4374 len 28672
15915 135/300 Write block 1084 len 20480 data: 2 2 3 3 3
15916 136/300 Flush
15917 137/300 Write block 2660 len 16384 data: 6 5 5 5
15918 138/300 Write block 310 len 28672 data: 7 6 6 6 6 5 5
15919 139/300 Read block 263 len 28672
15920 140/300 Read block 2156 len 28672
15921 141/300 Write block 4189 len 36864 data: 2 2 3 4 4 3 4 4 6
15922 142/300 Flush
15923 143/300 Read block 1615 len 12288
15924 144/300 Write block 4482 len 4096 data: 4
15925 145/300 Flush
15926 146/300 Flush
15927 147/300 Read block 4129 len 24576
15928 148/300 Write block 918 len 40960 data: 5 5 6 6 7 6 4 5 5 4
15929 149/300 Read block 3616 len 40960
15930 150/300 Write block 3046 len 8192 data: 6 6
15931 151/300 Read block 1583 len 24576
15932 152/300 Write block 3671 len 20480 data: 4 4 4 4 4
15933 153/300 Read block 4322 len 16384
15934 154/300 Write block 3435 len 36864 data: 6 5 5 5 4 4 3 3 3
15935 155/300 Write block 3892 len 20480 data: 4 3 3 3 3
15936 156/300 Write block 3949 len 16384 data: 4 4 5 4
15937 157/300 Read block 3928 len 16384
15938 158/300 Read block 4556 len 32768
15939 159/300 Write block 1920 len 4096 data: 6
15940 160/300 Write block 2481 len 40960 data: 4 5 6 7 7 6 6 6 4 4
15941 161/300 Write block 1949 len 36864 data: 6 6 3 2 2 2 3 3 3
15942 162/300 Read block 2184 len 16384
15943 163/300 Write block 1797 len 28672 data: 3 4 3 4 4 5 4
15944 164/300 Read block 4121 len 8192
15945 165/300 Read block 1424 len 4096
15946 166/300 Write block 973 len 32768 data: 2 3 3 3 3 3 3 2
15947 167/300 Write block 1251 len 20480 data: 5 4 3 4 3
15948 168/300 Read block 4079 len 4096
15949 169/300 Write block 3869 len 28672 data: 5 5 6 6 4 4 3
15950 170/300 Read block 4675 len 24576
15951 171/300 Read block 4696 len 16384
15952 172/300 Read block 1732 len 24576
15953 173/300 Write block 984 len 40960 data: 3 2 3 3 3 3 4 4 3 2
15954 174/300 Write block 3054 len 20480 data: 6 3 3 2 2
15955 175/300 Read block 3202 len 4096
15956 176/300 Flush
15957 177/300 Read block 2149 len 32768
15958 178/300 Read block 4840 len 8192
15959 179/300 Read block 2177 len 4096
15960 180/300 Write block 2871 len 36864 data: 3 3 3 2 2 2 2 3 4
15961 181/300 Flush
15962 182/300 Read block 3573 len 8192
15963 183/300 Write block 2267 len 4096 data: 3
15964 184/300 Read block 1397 len 28672
15965 185/300 Read block 2857 len 12288
15966 186/300 Read block 499 len 16384
15967 187/300 Read block 3354 len 32768
15968 188/300 Write block 3211 len 24576 data: 2 2 2 2 2 3
15969 189/300 Write block 521 len 16384 data: 6 5 5 5
15970 190/300 Read block 195 len 36864
15971 191/300 Read block 2525 len 4096
15972 192/300 Read block 3516 len 36864
15973 193/300 Write block 3469 len 28672 data: 7 6 4 3 3 3 2
15974 194/300 Read block 1897 len 24576
15975 195/300 Write block 718 len 12288 data: 3 4 4
15976 196/300 Read block 863 len 4096
15977 197/300 Read block 3115 len 40960
15978 198/300 Write block 2231 len 20480 data: 2 2 2 2 2
15979 199/300 Read block 4743 len 24576
15980 200/300 Write block 3396 len 16384 data: 5 6 6 5
15981 201/300 Read block 2319 len 20480
15982 202/300 Read block 141 len 40960
15983 203/300 Read block 4191 len 40960
15984 204/300 Read block 4613 len 8192
15985 205/300 Read block 4549 len 20480
15986 206/300 Read block 2507 len 20480
15987 207/300 Read block 2861 len 8192
15988 208/300 Write block 4017 len 28672 data: 3 3 3 3 3 3 3
15989 209/300 Flush
15990 210/300 Write block 3038 len 36864 data: 2 3 3 3 4 4 4 5 7
15991 211/300 Read block 4177 len 40960
15992 212/300 Write block 4843 len 16384 data: 4 4 5 5
15993 213/300 Write block 2081 len 40960 data: 3 3 3 3 3 4 5 5 4 4
15994 214/300 Write block 2395 len 12288 data: 3 3 2
15995 215/300 Read block 4794 len 28672
15996 216/300 Write block 862 len 4096 data: 2
15997 217/300 Write block 1479 len 20480 data: 2 2 2 2 3
15998 218/300 Write block 3612 len 4096 data: 5
15999 219/300 Read block 2216 len 24576
16000 220/300 Write block 278 len 32768 data: 2 2 2 2 3 2 3 3
16001 221/300 Read block 4806 len 32768
16002 222/300 Read block 2325 len 12288
16003 223/300 Read block 1024 len 32768
16004 224/300 Write block 2774 len 32768 data: 3 3 3 3 3 5 4 3
16005 225/300 Read block 3104 len 40960
16006 226/300 Read block 481 len 36864
16007 227/300 Flush
16008 228/300 Write block 3852 len 24576 data: 6 7 6 7 5 4
16009 229/300 Read block 418 len 4096
16010 230/300 Read block 2045 len 8192
16011 231/300 Flush
16012 232/300 Read block 2062 len 32768
16013 233/300 Write block 4736 len 20480 data: 2 3 2 2 2
16014 234/300 Read block 2536 len 24576
16015 235/300 Flush
16016 236/300 Write block 2473 len 12288 data: 3 3 4
16017 237/300 Read block 1048 len 8192
16018 238/300 Read block 2041 len 28672
16019 239/300 Read block 2293 len 4096
16020 240/300 Write block 246 len 24576 data: 4 4 4 5 5 4
16021 241/300 Write block 3894 len 32768 data: 4 4 4 3 2 2 3 5
16022 242/300 Write block 485 len 20480 data: 3 3 2 4 5
16023 243/300 Write block 2194 len 16384 data: 3 3 3 2
16024 244/300 Read block 3888 len 32768
16025 245/300 Read block 4184 len 24576
16026 246/300 Read block 526 len 28672
16027 247/300 Read block 4470 len 16384
16028 248/300 Write block 1669 len 40960 data: 3 3 4 4 4 4 6 5 5 5
16029 249/300 Read block 3005 len 24576
16030 250/300 Read block 2869 len 28672
16031 251/300 Write block 4887 len 24576 data: 7 7 7 4 4 3
16032 252/300 Write block 2768 len 16384 data: 4 4 3 4
16033 253/300 Write block 2244 len 16384 data: 2 2 3 4
16034 254/300 Write block 1670 len 32768 data: 4 5 5 5 5 7 6 6
16035 255/300 Write block 3491 len 20480 data: 3 4 4 4 4
16036 256/300 Read block 3142 len 36864
16037 257/300 Write block 4392 len 8192 data: 5 4
16038 258/300 Read block 4105 len 24576
16039 259/300 Write block 40 len 40960 data: 2 2 2 2 2 2 2 2 2 2
16040 260/300 Read block 1374 len 32768
16041 261/300 Read block 366 len 40960
16042 262/300 Read block 168 len 8192
16043 263/300 Read block 4552 len 36864
16044 264/300 Read block 2123 len 32768
16045 265/300 Read block 1050 len 32768
16046 266/300 Write block 3517 len 20480 data: 4 4 3 3 3
16047 267/300 Read block 1622 len 36864
16048 268/300 Read block 3023 len 32768
16049 269/300 Write block 2510 len 4096 data: 4
16050 270/300 Write block 4810 len 36864 data: 2 2 3 3 3 3 3 3 3
16051 271/300 Read block 323 len 28672
16052 272/300 Flush
16053 273/300 Write block 1904 len 28672 data: 4 3 3 4 4 4 5
16054 274/300 Read block 2291 len 40960
16055 275/300 Read block 2022 len 36864
16056 276/300 Write block 3948 len 36864 data: 4 5 5 6 5 4 4 3 4
16057 277/300 Read block 2396 len 32768
16058 278/300 Write block 3034 len 8192 data: 3 3
16059 279/300 Read block 3384 len 16384
16060 280/300 Read block 4772 len 16384
16061 281/300 Read block 3796 len 4096
16062 282/300 Read block 499 len 8192
16063 283/300 Read block 1732 len 32768
16064 284/300 Read block 598 len 24576
16065 285/300 Write block 2558 len 4096 data: 3
16066 286/300 Write block 3715 len 4096 data: 4
16067 287/300 Read block 2683 len 40960
16068 288/300 Flush
16069 289/300 Flush
16070 290/300 Read block 2762 len 20480
16071 291/300 Read block 2177 len 12288
16072 292/300 Flush
16073 293/300 Read block 3082 len 16384
16074 294/300 Read block 2187 len 20480
16075 295/300 Write block 4608 len 20480 data: 4 3 3 3 3
16076 296/300 Read block 4586 len 20480
16077 297/300 Read block 3626 len 28672
16078 298/300 Write block 1159 len 8192 data: 5 6
16079 299/300 Read block 2404 len 32768
16080 300/300 Read block 472 len 20480
16081 301/300 Write block 1946 len 24576 data: 5 6 6 7 7 4
16082 Main task has work to do, go find it
16083 got dsc Start(2)
16084 start 2
16085 Jan 03 08:34:17.314 INFO request completed, latency_us: 221, response_code: 204, uri: /start/cid/2, method: POST, req_id: 67d586f0-2d2a-41ab-90f0-2fb06da98549, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
16086 Replay: started 2, returned:Ok(())
16087 ----------------------------------------------------------------
16088 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16089 States: Active Active Offline
16090 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16091 5227 Acked Write 7 Done Done Sent false
16092 5228 Acked Read 3 Done Done Sent false
16093 5229 Acked Read 8 Done Done Sent false
16094 5230 Acked Write 9 Done Done Sent false
16095 5231 Acked Read 2 Done Done Sent false
16096 5232 Acked Read 4 Done Done Sent false
16097 5233 Acked Write 1 Done Done Sent false
16098 5234 Acked Read 9 Done Done Sent false
16099 5235 Acked Read 5 Done Done Sent false
16100 5236 Acked Read 7 Done Done Sent false
16101 5237 Acked Read 3 Done Done Sent false
16102 5238 Acked Write 8 Done Done Sent false
16103 5239 Acked Write 10 Done Done Sent false
16104 5240 Acked Write 9 Done Done Sent false
16105 5241 Acked Write 8 Done Done Sent false
16106 5242 Acked Read 10 Done Done Sent false
16107 5243 Acked Read 3 Done Done Sent false
16108 5244 Acked Write 6 Done Done Sent false
16109 5245 Acked Write 3 Done Done Sent false
16110 5246 Acked Read 10 Done Done Sent false
16111 5247 Acked Write 3 Done Done Sent false
16112 5248 Acked Read 6 Done Done Sent false
16113 5249 Acked Read 9 Done Done Sent false
16114 5250 Acked Read 6 Done Done Sent false
16115 5251 Acked Read 9 Done Done Sent false
16116 5252 Acked Read 8 Done Done Sent false
16117 5253 Acked Write 10 Done Done Sent false
16118 5254 Acked Write 10 Done Done Sent false
16119 5255 Acked Write 4 Done Done Sent false
16120 5256 Acked Read 5 Done Done Sent false
16121 5257 Acked Read 3 Done Done Sent false
16122 5258 Acked Flush 0 Done Done Sent false
16123 5259 Acked Write 10 Done Done Sent false
16124 5260 Acked Read 10 Done Done Sent false
16125 5261 Acked Write 6 Done Done Sent false
16126 5262 Acked Write 5 Done Done Sent false
16127 5263 Acked Read 5 Done Done Sent false
16128 5264 Acked Read 5 Done Done Sent false
16129 5265 Acked Write 5 Done Done Sent false
16130 5266 Acked Read 4 Done Done Sent false
16131 5267 Acked Write 1 Done Done Sent false
16132 5268 Acked Read 1 Done Done Sent false
16133 5269 Acked Flush 0 Done Done Sent false
16134 5270 Acked Write 4 Done Done Sent false
16135 5271 Acked Write 10 Done Done Sent false
16136 5272 Acked Write 8 Done Done Sent false
16137 5273 Acked Write 8 Done Done Sent false
16138 5274 Acked Read 5 Done Done Sent false
16139 5275 Acked Write 9 Done Done Sent false
16140 5276 Acked Read 10 Done Done Sent false
16141 5277 Acked Read 9 Done Done Sent false
16142 5278 Acked Write 9 Done Done Sent false
16143 5279 Acked Read 1 Done Done Sent false
16144 5280 Acked Read 10 Done Done Sent false
16145 5281 Acked Read 4 Done Done Sent false
16146 5282 Acked Write 6 Done Done Sent false
16147 5283 Acked Write 2 Done Done Sent false
16148 5284 Acked Flush 0 Done Done Sent false
16149 5285 Acked Write 8 Done Done Sent false
16150 5286 Acked Write 6 Done Done Sent false
16151 5287 Acked Read 8 Done Done Sent false
16152 5288 Acked Read 9 Done Done Sent false
16153 5289 Acked Write 2 Done Done Sent false
16154 5290 Acked Write 4 Done Done Sent false
16155 5291 Acked Read 5 Done Done Sent false
16156 5292 Acked Read 2 Done Done Sent false
16157 5293 Acked Write 4 Done Done Sent false
16158 5294 Acked Read 7 Done Done Sent false
16159 5295 Acked Write 7 Done Done Sent false
16160 5296 Acked Read 6 Done Done Sent false
16161 5297 Acked Read 7 Done Done Sent false
16162 5298 Acked Read 1 Done Done Sent false
16163 5299 Acked Write 10 Done Done Sent false
16164 5300 Acked Write 7 Done Done Sent false
16165 5301 Acked Read 8 Done Done Sent false
16166 5302 Acked Flush 0 Done Done Sent false
16167 5303 Acked Read 10 Done Done Sent false
16168 5304 Acked Read 2 Done Done Sent false
16169 5305 Acked Read 1 Done Done Sent false
16170 5306 Acked Read 10 Done Done Sent false
16171 5307 Acked Read 2 Done Done Sent false
16172 5308 Acked Read 6 Done Done Sent false
16173 5309 Acked Read 3 Done Done Sent false
16174 5310 Acked Write 5 Done Done Sent false
16175 5311 Acked Write 7 Done Done Sent false
16176 5312 Acked Read 3 Done Done Sent false
16177 5313 Acked Write 7 Done Done Sent false
16178 5314 Acked Write 7 Done Done Sent false
16179 5315 Acked Write 7 Done Done Sent false
16180 5316 Acked Flush 0 Done Done Sent false
16181 5317 Acked Write 9 Done Done Sent false
16182 5318 Acked Write 9 Done Done Sent false
16183 5319 Acked Write 7 Done Done Sent false
16184 5320 Acked Read 2 Done Done Sent false
16185 5321 Acked Read 4 Done Done Sent false
16186 5322 Acked Flush 0 Done Done Sent false
16187 5323 Acked Write 3 Done Done Sent false
16188 5324 Acked Write 4 Done Done Sent false
16189 5325 Acked Write 4 Done Done Sent false
16190 5326 Acked Read 6 Done Done Sent false
16191 5327 Acked Write 1 Done Done Sent false
16192 5328 Acked Read 3 Done Done Sent false
16193 5329 Acked Read 6 Done Done Sent false
16194 5330 Acked Read 6 Done Done Sent false
16195 5331 Acked Read 4 Done Done Sent false
16196 5332 Acked Write 9 Done Done Sent false
16197 5333 Acked Write 9 Done Done Sent false
16198 5334 Acked Read 3 Done Done Sent false
16199 5335 Acked Read 7 Done Done Sent false
16200 5336 Acked Write 1 Done Done Sent false
16201 5337 Acked Flush 0 Done Done Sent false
16202 5338 Acked Flush 0 Done Done Sent false
16203 5339 Acked Write 5 Done Done Sent false
16204 5340 Acked Write 10 Done Done Sent false
16205 5341 Acked Write 10 Done Done Sent false
16206 5342 Acked Flush 0 Done Done Sent false
16207 5343 Acked Write 1 Done Done Sent false
16208 5344 Acked Write 6 Done Done Sent false
16209 5345 Acked Read 9 Done Done Sent false
16210 5346 Acked Write 2 Done Done Sent false
16211 5347 Acked Write 9 Done Done Sent false
16212 5348 Acked Read 1 Done Done Sent false
16213 5349 Acked Write 8 Done Done Sent false
16214 5350 Acked Write 7 Done Done Sent false
16215 5351 Acked Read 5 Done Done Sent false
16216 5352 Acked Write 4 Done Done Sent false
16217 5353 Acked Read 2 Done Done Sent false
16218 5354 Acked Read 6 Done Done Sent false
16219 5355 Acked Write 2 Done Done Sent false
16220 5356 Acked Write 5 Done Done Sent false
16221 5357 Acked Write 8 Done Done Sent false
16222 5358 Acked Write 4 Done Done Sent false
16223 5359 Acked Flush 0 Done Done Sent false
16224 5360 Acked Read 7 Done Done Sent false
16225 5361 Acked Write 5 Done Done Sent false
16226 5362 Acked Flush 0 Done Done Sent false
16227 5363 Acked Write 4 Done Done Sent false
16228 5364 Acked Write 7 Done Done Sent false
16229 5365 Acked Read 7 Done Done Sent false
16230 5366 Acked Read 7 Done Done Sent false
16231 5367 Acked Write 9 Done Done Sent false
16232 5368 Acked Flush 0 Done Done Sent false
16233 5369 Acked Read 3 Done Done Sent false
16234 5370 Acked Write 1 Done Done Sent false
16235 5371 Acked Flush 0 Done Done Sent false
16236 5372 Acked Flush 0 Done Done Sent false
16237 5373 Acked Read 6 Done Done Sent false
16238 5374 Acked Write 10 Done Done Sent false
16239 5375 Acked Read 10 Done Done Sent false
16240 5376 Acked Write 2 Done Done Sent false
16241 5377 Acked Read 6 Done Done Sent false
16242 5378 Acked Write 5 Done Done Sent false
16243 5379 Acked Read 4 Done Done Sent false
16244 5380 Acked Write 9 Done Done Sent false
16245 5381 Acked Write 5 Done Done Sent false
16246 5382 Acked Write 4 Done Done Sent false
16247 5383 Acked Read 4 Done Done Sent false
16248 5384 Acked Read 8 Done Done Sent false
16249 5385 Acked Write 1 Done Done Sent false
16250 5386 Acked Write 10 Done Done Sent false
16251 5387 Acked Write 9 Done Done Sent false
16252 5388 Acked Read 4 Done Done Sent false
16253 5389 Acked Write 7 Done Done Sent false
16254 5390 Acked Read 2 Done Done Sent false
16255 5391 Acked Read 1 Done Done Sent false
16256 5392 Acked Write 8 Done Done Sent false
16257 5393 Acked Write 5 Done Done Sent false
16258 5394 Acked Read 1 Done Done Sent false
16259 5395 Acked Write 7 Done Done Sent false
16260 5396 Acked Read 6 Done Done Sent false
16261 5397 Acked Read 4 Done Done Sent false
16262 5398 Acked Read 6 Done Done Sent false
16263 5399 Acked Write 10 Done Done Sent false
16264 5400 Acked Write 5 Done Done Sent false
16265 5401 Acked Read 1 Done Done Sent false
16266 5402 Acked Flush 0 Done Done Sent false
16267 5403 Acked Read 8 Done Done Sent false
16268 5404 Acked Read 2 Done Done Sent false
16269 5405 Acked Read 1 Done Done Sent false
16270 5406 Acked Write 9 Done Done Sent false
16271 5407 Acked Flush 0 Done Done Sent false
16272 5408 Acked Read 2 Done Done Sent false
16273 5409 Acked Write 1 Done Done Sent false
16274 5410 Acked Read 7 Done Done Sent false
16275 5411 Acked Read 3 Done Done Sent false
16276 5412 Acked Read 4 Done Done Sent false
16277 5413 Acked Read 8 Done Done Sent false
16278 5414 Acked Write 6 Done Done Sent false
16279 5415 Acked Write 4 Done Done Sent false
16280 5416 Acked Read 9 Done Done Sent false
16281 5417 Acked Read 1 Done Done Sent false
16282 5418 Acked Read 9 Done Done Sent false
16283 5419 Acked Write 7 Done Done Sent false
16284 5420 Acked Read 6 Done Done Sent false
16285 5421 Acked Write 3 Done Done Sent false
16286 5422 Acked Read 1 Done Done Sent false
16287 5423 Acked Read 10 Done Done Sent false
16288 5424 Acked Write 5 Done Done Sent false
16289 5425 Acked Read 6 Done Done Sent false
16290 5426 Acked Write 4 Done Done Sent false
16291 5427 Acked Read 5 Done Done Sent false
16292 5428 Acked Read 10 Done Done Sent false
16293 5429 Acked Read 10 Done Done Sent false
16294 5430 Acked Read 2 Done Done Sent false
16295 5431 Acked Read 5 Done Done Sent false
16296 5432 Acked Read 5 Done Done Sent false
16297 5433 Acked Read 2 Done Done Sent false
16298 5434 Acked Write 7 Done Done Sent false
16299 5435 Acked Flush 0 Done Done Sent false
16300 5436 Acked Write 9 Done Done Sent false
16301 5437 Acked Read 10 Done Done Sent false
16302 5438 Acked Write 4 Done Done Sent false
16303 5439 Acked Write 10 Done Done Sent false
16304 5440 Acked Write 3 Done Done Sent false
16305 5441 Acked Read 7 Done Done Sent false
16306 5442 Acked Write 1 Done Done Sent false
16307 5443 Acked Write 5 Done Done Sent false
16308 5444 Acked Write 1 Done Done Sent false
16309 5445 Acked Read 6 Done Done Sent false
16310 5446 Acked Write 8 Done Done Sent false
16311 5447 Acked Read 8 Done Done Sent false
16312 5448 Acked Read 3 Done Done Sent false
16313 5449 Acked Read 8 Done Done Sent false
16314 5450 Acked Write 8 Done Done Sent false
16315 5451 Acked Read 10 Done Done Sent false
16316 5452 Acked Read 9 Done Done Sent false
16317 5453 Acked Flush 0 Done Done Sent false
16318 5454 Acked Write 6 Done Done Sent false
16319 5455 Acked Read 1 Done Done Sent false
16320 5456 Acked Read 2 Done Done Sent false
16321 5457 Acked Flush 0 Done Done Sent false
16322 5458 Acked Read 8 Done Done Sent false
16323 5459 Acked Write 5 Done Done Sent false
16324 5460 Acked Read 6 Done Done Sent false
16325 5461 Acked Flush 0 Done Done Sent false
16326 5462 Acked Write 3 Done Done Sent false
16327 5463 Acked Read 2 Done Done Sent false
16328 5464 Acked Read 7 Done Done Sent false
16329 5465 Acked Read 1 Done Done Sent false
16330 5466 Acked Write 6 Done Done Sent false
16331 5467 Acked Write 8 Done Done Sent false
16332 5468 Acked Write 5 Done Done Sent false
16333 5469 Acked Write 4 Done Done Sent false
16334 5470 Acked Read 8 Done Done Sent false
16335 5471 Acked Read 6 Done Done Sent false
16336 5472 Acked Read 7 Done Done Sent false
16337 5473 Acked Read 4 Done Done Sent false
16338 5474 Acked Write 10 Done Done Sent false
16339 5475 Acked Read 6 Done Done Sent false
16340 5476 Acked Read 7 Done Done Sent false
16341 5477 Acked Write 6 Done Done Sent false
16342 5478 Acked Write 4 Done Done Sent false
16343 5479 Acked Write 4 Done Done Sent false
16344 5480 Acked Write 8 Done Done Sent false
16345 5481 Acked Write 5 Done Done Sent false
16346 5482 Acked Read 9 Done Done Sent false
16347 5483 Acked Write 2 Done Done Sent false
16348 5484 Acked Read 6 Done Done Sent false
16349 5485 Acked Write 10 Done Done Sent false
16350 5486 Acked Read 8 Done Done Sent false
16351 5487 Acked Read 10 Done Done Sent false
16352 5488 Acked Read 2 Done Done Sent false
16353 5489 Acked Read 9 Done Done Sent false
16354 5490 Acked Read 8 Done Done Sent false
16355 5491 Acked Read 8 Done Done Sent false
16356 5492 Acked Write 5 Done Done Sent false
16357 5493 Acked Read 9 Done Done Sent false
16358 5494 Acked Read 8 Done Done Sent false
16359 5495 Acked Write 1 Done Done Sent false
16360 5496 Acked Write 9 Done Done Sent false
16361 5497 Acked Read 7 Done Done Sent false
16362 5498 Acked Flush 0 Done Done Sent false
16363 5499 Acked Write 7 Done Done Sent false
16364 5500 Acked Read 10 Done Done Sent false
16365 5501 Acked Read 9 Done Done Sent false
16366 5502 Acked Write 9 Done Done Sent false
16367 5503 Acked Read 8 Done Done Sent false
16368 5504 Acked Write 2 Done Done Sent false
16369 5505 Acked Read 4 Done Done Sent false
16370 5506 Acked Read 4 Done Done Sent false
16371 5507 Acked Read 1 Done Done Sent false
16372 5508 Acked Read 2 Done Done Sent false
16373 5509 Acked Read 8 Done Done Sent false
16374 5510 Acked Read 6 Done Done Sent false
16375 5511 Acked Write 1 Done Done Sent false
16376 5512 Acked Write 1 Done Done Sent false
16377 5513 Acked Read 10 Done Done Sent false
16378 5514 Acked Flush 0 Done Done Sent false
16379 5515 Acked Flush 0 Done Done Sent false
16380 5516 Acked Read 5 Done Done Sent false
16381 5517 Acked Read 3 Done Done Sent false
16382 5518 Acked Flush 0 Done Done Sent false
16383 5519 Acked Read 4 Done Done Sent false
16384 5520 Acked Read 5 Done Done Sent false
16385 5521 Acked Write 5 Done Done Sent false
16386 5522 Acked Read 5 Done Done Sent false
16387 5523 Acked Read 7 Done Done Sent false
16388 5524 Acked Write 2 Done Done Sent false
16389 5525 Acked Read 8 Done Done Sent false
16390 5526 Acked Read 5 Done Done Sent false
16391 5527 Acked Write 6 Done Done Sent false
16392 STATES DS:0 DS:1 DS:2 TOTAL
16393 Sent 0 0 301 301
16394 Done 301 301 0 602
16395 Skipped 0 0 0 0
16396 Error 0 0 0 0
16397 Last Flush: 5518 5518 5226
16398 Downstairs last five completed: 5226 5225 5224 5223 5222
16399 Upstairs last five completed: 5527 5526 5525 5524 5523
16400 CLIENT: Up:0 ds:301 act:2
16401 [8830] got start action
16402 [8830] I am going to restart
16403 Starting downstairs at port 8830
16404 Make output file at "/tmp/dsc/downstairs-8830.txt"
16405 [8830][2] reports Starting
16406 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1220)
16407 [8830][2] reports Running
16408 ----------------------------------------------------------------
16409 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16410 States: Active Active Offline
16411 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16412 5227 Acked Write 7 Done Done Sent false
16413 5228 Acked Read 3 Done Done Sent false
16414 5229 Acked Read 8 Done Done Sent false
16415 5230 Acked Write 9 Done Done Sent false
16416 5231 Acked Read 2 Done Done Sent false
16417 5232 Acked Read 4 Done Done Sent false
16418 5233 Acked Write 1 Done Done Sent false
16419 5234 Acked Read 9 Done Done Sent false
16420 5235 Acked Read 5 Done Done Sent false
16421 5236 Acked Read 7 Done Done Sent false
16422 5237 Acked Read 3 Done Done Sent false
16423 5238 Acked Write 8 Done Done Sent false
16424 5239 Acked Write 10 Done Done Sent false
16425 5240 Acked Write 9 Done Done Sent false
16426 5241 Acked Write 8 Done Done Sent false
16427 5242 Acked Read 10 Done Done Sent false
16428 5243 Acked Read 3 Done Done Sent false
16429 5244 Acked Write 6 Done Done Sent false
16430 5245 Acked Write 3 Done Done Sent false
16431 5246 Acked Read 10 Done Done Sent false
16432 5247 Acked Write 3 Done Done Sent false
16433 5248 Acked Read 6 Done Done Sent false
16434 5249 Acked Read 9 Done Done Sent false
16435 5250 Acked Read 6 Done Done Sent false
16436 5251 Acked Read 9 Done Done Sent false
16437 5252 Acked Read 8 Done Done Sent false
16438 5253 Acked Write 10 Done Done Sent false
16439 5254 Acked Write 10 Done Done Sent false
16440 5255 Acked Write 4 Done Done Sent false
16441 5256 Acked Read 5 Done Done Sent false
16442 5257 Acked Read 3 Done Done Sent false
16443 5258 Acked Flush 0 Done Done Sent false
16444 5259 Acked Write 10 Done Done Sent false
16445 5260 Acked Read 10 Done Done Sent false
16446 5261 Acked Write 6 Done Done Sent false
16447 5262 Acked Write 5 Done Done Sent false
16448 5263 Acked Read 5 Done Done Sent false
16449 5264 Acked Read 5 Done Done Sent false
16450 5265 Acked Write 5 Done Done Sent false
16451 5266 Acked Read 4 Done Done Sent false
16452 5267 Acked Write 1 Done Done Sent false
16453 5268 Acked Read 1 Done Done Sent false
16454 5269 Acked Flush 0 Done Done Sent false
16455 5270 Acked Write 4 Done Done Sent false
16456 5271 Acked Write 10 Done Done Sent false
16457 5272 Acked Write 8 Done Done Sent false
16458 5273 Acked Write 8 Done Done Sent false
16459 5274 Acked Read 5 Done Done Sent false
16460 5275 Acked Write 9 Done Done Sent false
16461 5276 Acked Read 10 Done Done Sent false
16462 5277 Acked Read 9 Done Done Sent false
16463 5278 Acked Write 9 Done Done Sent false
16464 5279 Acked Read 1 Done Done Sent false
16465 5280 Acked Read 10 Done Done Sent false
16466 5281 Acked Read 4 Done Done Sent false
16467 5282 Acked Write 6 Done Done Sent false
16468 5283 Acked Write 2 Done Done Sent false
16469 5284 Acked Flush 0 Done Done Sent false
16470 5285 Acked Write 8 Done Done Sent false
16471 5286 Acked Write 6 Done Done Sent false
16472 5287 Acked Read 8 Done Done Sent false
16473 5288 Acked Read 9 Done Done Sent false
16474 5289 Acked Write 2 Done Done Sent false
16475 5290 Acked Write 4 Done Done Sent false
16476 5291 Acked Read 5 Done Done Sent false
16477 5292 Acked Read 2 Done Done Sent false
16478 5293 Acked Write 4 Done Done Sent false
16479 5294 Acked Read 7 Done Done Sent false
16480 5295 Acked Write 7 Done Done Sent false
16481 5296 Acked Read 6 Done Done Sent false
16482 5297 Acked Read 7 Done Done Sent false
16483 5298 Acked Read 1 Done Done Sent false
16484 5299 Acked Write 10 Done Done Sent false
16485 5300 Acked Write 7 Done Done Sent false
16486 5301 Acked Read 8 Done Done Sent false
16487 5302 Acked Flush 0 Done Done Sent false
16488 5303 Acked Read 10 Done Done Sent false
16489 5304 Acked Read 2 Done Done Sent false
16490 5305 Acked Read 1 Done Done Sent false
16491 5306 Acked Read 10 Done Done Sent false
16492 5307 Acked Read 2 Done Done Sent false
16493 5308 Acked Read 6 Done Done Sent false
16494 5309 Acked Read 3 Done Done Sent false
16495 5310 Acked Write 5 Done Done Sent false
16496 5311 Acked Write 7 Done Done Sent false
16497 5312 Acked Read 3 Done Done Sent false
16498 5313 Acked Write 7 Done Done Sent false
16499 5314 Acked Write 7 Done Done Sent false
16500 5315 Acked Write 7 Done Done Sent false
16501 5316 Acked Flush 0 Done Done Sent false
16502 5317 Acked Write 9 Done Done Sent false
16503 5318 Acked Write 9 Done Done Sent false
16504 5319 Acked Write 7 Done Done Sent false
16505 5320 Acked Read 2 Done Done Sent false
16506 5321 Acked Read 4 Done Done Sent false
16507 5322 Acked Flush 0 Done Done Sent false
16508 5323 Acked Write 3 Done Done Sent false
16509 5324 Acked Write 4 Done Done Sent false
16510 5325 Acked Write 4 Done Done Sent false
16511 5326 Acked Read 6 Done Done Sent false
16512 5327 Acked Write 1 Done Done Sent false
16513 5328 Acked Read 3 Done Done Sent false
16514 5329 Acked Read 6 Done Done Sent false
16515 5330 Acked Read 6 Done Done Sent false
16516 5331 Acked Read 4 Done Done Sent false
16517 5332 Acked Write 9 Done Done Sent false
16518 5333 Acked Write 9 Done Done Sent false
16519 5334 Acked Read 3 Done Done Sent false
16520 5335 Acked Read 7 Done Done Sent false
16521 5336 Acked Write 1 Done Done Sent false
16522 5337 Acked Flush 0 Done Done Sent false
16523 5338 Acked Flush 0 Done Done Sent false
16524 5339 Acked Write 5 Done Done Sent false
16525 5340 Acked Write 10 Done Done Sent false
16526 5341 Acked Write 10 Done Done Sent false
16527 5342 Acked Flush 0 Done Done Sent false
16528 5343 Acked Write 1 Done Done Sent false
16529 5344 Acked Write 6 Done Done Sent false
16530 5345 Acked Read 9 Done Done Sent false
16531 5346 Acked Write 2 Done Done Sent false
16532 5347 Acked Write 9 Done Done Sent false
16533 5348 Acked Read 1 Done Done Sent false
16534 5349 Acked Write 8 Done Done Sent false
16535 5350 Acked Write 7 Done Done Sent false
16536 5351 Acked Read 5 Done Done Sent false
16537 5352 Acked Write 4 Done Done Sent false
16538 5353 Acked Read 2 Done Done Sent false
16539 5354 Acked Read 6 Done Done Sent false
16540 5355 Acked Write 2 Done Done Sent false
16541 5356 Acked Write 5 Done Done Sent false
16542 5357 Acked Write 8 Done Done Sent false
16543 5358 Acked Write 4 Done Done Sent false
16544 5359 Acked Flush 0 Done Done Sent false
16545 5360 Acked Read 7 Done Done Sent false
16546 5361 Acked Write 5 Done Done Sent false
16547 5362 Acked Flush 0 Done Done Sent false
16548 5363 Acked Write 4 Done Done Sent false
16549 5364 Acked Write 7 Done Done Sent false
16550 5365 Acked Read 7 Done Done Sent false
16551 5366 Acked Read 7 Done Done Sent false
16552 5367 Acked Write 9 Done Done Sent false
16553 5368 Acked Flush 0 Done Done Sent false
16554 5369 Acked Read 3 Done Done Sent false
16555 5370 Acked Write 1 Done Done Sent false
16556 5371 Acked Flush 0 Done Done Sent false
16557 5372 Acked Flush 0 Done Done Sent false
16558 5373 Acked Read 6 Done Done Sent false
16559 5374 Acked Write 10 Done Done Sent false
16560 5375 Acked Read 10 Done Done Sent false
16561 5376 Acked Write 2 Done Done Sent false
16562 5377 Acked Read 6 Done Done Sent false
16563 5378 Acked Write 5 Done Done Sent false
16564 5379 Acked Read 4 Done Done Sent false
16565 5380 Acked Write 9 Done Done Sent false
16566 5381 Acked Write 5 Done Done Sent false
16567 5382 Acked Write 4 Done Done Sent false
16568 5383 Acked Read 4 Done Done Sent false
16569 5384 Acked Read 8 Done Done Sent false
16570 5385 Acked Write 1 Done Done Sent false
16571 5386 Acked Write 10 Done Done Sent false
16572 5387 Acked Write 9 Done Done Sent false
16573 5388 Acked Read 4 Done Done Sent false
16574 5389 Acked Write 7 Done Done Sent false
16575 5390 Acked Read 2 Done Done Sent false
16576 5391 Acked Read 1 Done Done Sent false
16577 5392 Acked Write 8 Done Done Sent false
16578 5393 Acked Write 5 Done Done Sent false
16579 5394 Acked Read 1 Done Done Sent false
16580 5395 Acked Write 7 Done Done Sent false
16581 5396 Acked Read 6 Done Done Sent false
16582 5397 Acked Read 4 Done Done Sent false
16583 5398 Acked Read 6 Done Done Sent false
16584 5399 Acked Write 10 Done Done Sent false
16585 5400 Acked Write 5 Done Done Sent false
16586 5401 Acked Read 1 Done Done Sent false
16587 5402 Acked Flush 0 Done Done Sent false
16588 5403 Acked Read 8 Done Done Sent false
16589 5404 Acked Read 2 Done Done Sent false
16590 5405 Acked Read 1 Done Done Sent false
16591 5406 Acked Write 9 Done Done Sent false
16592 5407 Acked Flush 0 Done Done Sent false
16593 5408 Acked Read 2 Done Done Sent false
16594 5409 Acked Write 1 Done Done Sent false
16595 5410 Acked Read 7 Done Done Sent false
16596 5411 Acked Read 3 Done Done Sent false
16597 5412 Acked Read 4 Done Done Sent false
16598 5413 Acked Read 8 Done Done Sent false
16599 5414 Acked Write 6 Done Done Sent false
16600 5415 Acked Write 4 Done Done Sent false
16601 5416 Acked Read 9 Done Done Sent false
16602 5417 Acked Read 1 Done Done Sent false
16603 5418 Acked Read 9 Done Done Sent false
16604 5419 Acked Write 7 Done Done Sent false
16605 5420 Acked Read 6 Done Done Sent false
16606 5421 Acked Write 3 Done Done Sent false
16607 5422 Acked Read 1 Done Done Sent false
16608 5423 Acked Read 10 Done Done Sent false
16609 5424 Acked Write 5 Done Done Sent false
16610 5425 Acked Read 6 Done Done Sent false
16611 5426 Acked Write 4 Done Done Sent false
16612 5427 Acked Read 5 Done Done Sent false
16613 5428 Acked Read 10 Done Done Sent false
16614 5429 Acked Read 10 Done Done Sent false
16615 5430 Acked Read 2 Done Done Sent false
16616 5431 Acked Read 5 Done Done Sent false
16617 5432 Acked Read 5 Done Done Sent false
16618 5433 Acked Read 2 Done Done Sent false
16619 5434 Acked Write 7 Done Done Sent false
16620 5435 Acked Flush 0 Done Done Sent false
16621 5436 Acked Write 9 Done Done Sent false
16622 5437 Acked Read 10 Done Done Sent false
16623 5438 Acked Write 4 Done Done Sent false
16624 5439 Acked Write 10 Done Done Sent false
16625 5440 Acked Write 3 Done Done Sent false
16626 5441 Acked Read 7 Done Done Sent false
16627 5442 Acked Write 1 Done Done Sent false
16628 5443 Acked Write 5 Done Done Sent false
16629 5444 Acked Write 1 Done Done Sent false
16630 5445 Acked Read 6 Done Done Sent false
16631 5446 Acked Write 8 Done Done Sent false
16632 5447 Acked Read 8 Done Done Sent false
16633 5448 Acked Read 3 Done Done Sent false
16634 5449 Acked Read 8 Done Done Sent false
16635 5450 Acked Write 8 Done Done Sent false
16636 5451 Acked Read 10 Done Done Sent false
16637 5452 Acked Read 9 Done Done Sent false
16638 5453 Acked Flush 0 Done Done Sent false
16639 5454 Acked Write 6 Done Done Sent false
16640 5455 Acked Read 1 Done Done Sent false
16641 5456 Acked Read 2 Done Done Sent false
16642 5457 Acked Flush 0 Done Done Sent false
16643 5458 Acked Read 8 Done Done Sent false
16644 5459 Acked Write 5 Done Done Sent false
16645 5460 Acked Read 6 Done Done Sent false
16646 5461 Acked Flush 0 Done Done Sent false
16647 5462 Acked Write 3 Done Done Sent false
16648 5463 Acked Read 2 Done Done Sent false
16649 5464 Acked Read 7 Done Done Sent false
16650 5465 Acked Read 1 Done Done Sent false
16651 5466 Acked Write 6 Done Done Sent false
16652 5467 Acked Write 8 Done Done Sent false
16653 5468 Acked Write 5 Done Done Sent false
16654 5469 Acked Write 4 Done Done Sent false
16655 5470 Acked Read 8 Done Done Sent false
16656 5471 Acked Read 6 Done Done Sent false
16657 5472 Acked Read 7 Done Done Sent false
16658 5473 Acked Read 4 Done Done Sent false
16659 5474 Acked Write 10 Done Done Sent false
16660 5475 Acked Read 6 Done Done Sent false
16661 5476 Acked Read 7 Done Done Sent false
16662 5477 Acked Write 6 Done Done Sent false
16663 5478 Acked Write 4 Done Done Sent false
16664 5479 Acked Write 4 Done Done Sent false
16665 5480 Acked Write 8 Done Done Sent false
16666 5481 Acked Write 5 Done Done Sent false
16667 5482 Acked Read 9 Done Done Sent false
16668 5483 Acked Write 2 Done Done Sent false
16669 5484 Acked Read 6 Done Done Sent false
16670 5485 Acked Write 10 Done Done Sent false
16671 5486 Acked Read 8 Done Done Sent false
16672 5487 Acked Read 10 Done Done Sent false
16673 5488 Acked Read 2 Done Done Sent false
16674 5489 Acked Read 9 Done Done Sent false
16675 5490 Acked Read 8 Done Done Sent false
16676 5491 Acked Read 8 Done Done Sent false
16677 5492 Acked Write 5 Done Done Sent false
16678 5493 Acked Read 9 Done Done Sent false
16679 5494 Acked Read 8 Done Done Sent false
16680 5495 Acked Write 1 Done Done Sent false
16681 5496 Acked Write 9 Done Done Sent false
16682 5497 Acked Read 7 Done Done Sent false
16683 5498 Acked Flush 0 Done Done Sent false
16684 5499 Acked Write 7 Done Done Sent false
16685 5500 Acked Read 10 Done Done Sent false
16686 5501 Acked Read 9 Done Done Sent false
16687 5502 Acked Write 9 Done Done Sent false
16688 5503 Acked Read 8 Done Done Sent false
16689 5504 Acked Write 2 Done Done Sent false
16690 5505 Acked Read 4 Done Done Sent false
16691 5506 Acked Read 4 Done Done Sent false
16692 5507 Acked Read 1 Done Done Sent false
16693 5508 Acked Read 2 Done Done Sent false
16694 5509 Acked Read 8 Done Done Sent false
16695 5510 Acked Read 6 Done Done Sent false
16696 5511 Acked Write 1 Done Done Sent false
16697 5512 Acked Write 1 Done Done Sent false
16698 5513 Acked Read 10 Done Done Sent false
16699 5514 Acked Flush 0 Done Done Sent false
16700 5515 Acked Flush 0 Done Done Sent false
16701 5516 Acked Read 5 Done Done Sent false
16702 5517 Acked Read 3 Done Done Sent false
16703 5518 Acked Flush 0 Done Done Sent false
16704 5519 Acked Read 4 Done Done Sent false
16705 5520 Acked Read 5 Done Done Sent false
16706 5521 Acked Write 5 Done Done Sent false
16707 5522 Acked Read 5 Done Done Sent false
16708 5523 Acked Read 7 Done Done Sent false
16709 5524 Acked Write 2 Done Done Sent false
16710 5525 Acked Read 8 Done Done Sent false
16711 5526 Acked Read 5 Done Done Sent false
16712 5527 Acked Write 6 Done Done Sent false
16713 5528 Acked Flush 0 Done Done Sent false
16714 STATES DS:0 DS:1 DS:2 TOTAL
16715 Sent 0 0 302 302
16716 Done 302 302 0 604
16717 Skipped 0 0 0 0
16718 Error 0 0 0 0
16719 Last Flush: 5528 5528 5226
16720 Downstairs last five completed: 5226 5225 5224 5223 5222
16721 Upstairs last five completed: 5528 5527 5526 5525 5524
16722 CLIENT: Up:0 ds:302 act:2
167232025-01-03T08:34:23.066ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167242025-01-03T08:34:23.066ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40096) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167252025-01-03T08:34:23.067ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167262025-01-03T08:34:23.068ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167272025-01-03T08:34:23.068ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167282025-01-03T08:34:23.068ZINFOcrucible: send last flush ID to this DS: 5226 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167292025-01-03T08:34:23.068ZINFOcrucible: Replied this last flush ID: 5226 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167302025-01-03T08:34:23.068ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167312025-01-03T08:34:23.068ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167322025-01-03T08:34:23.068ZINFOcrucible: [2] client re-new 302 jobs since flush 5226 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167332025-01-03T08:34:23.068ZINFOcrucible: [2] Replayed 302 jobs since flush: 5226 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
16734 ----------------------------------------------------------------
16735 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16736 Downstairs last five completed: 5528 5527 5526 5525 5524
16737 Upstairs last five completed: 5528 5527 5526 5525 5524
16738 CLIENT: Up:0 ds:0 act:3
16739 Replay: All jobs finished
16740 Main task has work to do, go find it
16741 got dsc Stop(1)
16742 stop 1
16743 Jan 03 08:34:25.319[8820] Got stop action so:false kr:false
16744 INFO request completed, latency_us: 367, response_code: 204, uri: /stop/cid/1, method: POST, req_id: c652a3ae-ab22-4ee2-8317-1798f99d128f, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
167452025-01-03T08:34:25.320ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
16746 Jan 03 08:34:25.320 INFO request completed, latency_us: 154{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:34:25.320235828Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
16747 , response_code: 200, uri: /state/cid/1, method: GET, req_id: 704d2aa6-2c90-40cb-8366-3f46606bccb3, remote_addr: {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:34:25.320275128Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
16748 127.0.0.1:49669, local_addr: 127.{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:34:25.320306827Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
16749 0.0.1:9998
167502025-01-03T08:34:25.320ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
167512025-01-03T08:34:25.320ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
16752 [8820] Exited with: ExitStatus(unix_wait_status(9))
16753 [8820][1] reports Exit
16754 Jan 03 08:34:29.322 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: 6226d06f-1511-4744-99c2-d582cfd036c8, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
16755 001/300 Write block 336 len 16384 data: 2 2 2 3
16756 002/300 Read block 123 len 32768
16757 003/300 Read block 3817 len 32768
16758 004/300 Read block 562 len 8192
16759 005/300 Write block 2324 len 32768 data: 4 4 4 5 5 4 4 4
16760 006/300 Read block 230 len 4096
16761 007/300 Write block 3924 len 40960 data: 3 3 4 4 3 5 4 5 5 5
16762 008/300 Write block 4259 len 8192 data: 5 4
16763 009/300 Read block 3112 len 8192
16764 010/300 Write block 4059 len 40960 data: 4 4 4 4 4 3 3 3 5 5
16765 011/300 Write block 883 len 36864 data: 4 3 4 4 5 6 6 5 5
16766 012/300 Flush
16767 013/300 Flush
16768 014/300 Read block 2570 len 8192
16769 015/300 Write block 2351 len 28672 data: 3 3 3 2 2 2 2
16770 016/300 Read block 2537 len 16384
16771 017/300 Read block 3451 len 28672
16772 018/300 Flush
16773 019/300 Read block 110 len 8192
16774 020/300 Read block 125 len 16384
16775 021/300 Write block 4199 len 16384 data: 9 9 9 8
16776 022/300 Write block 470 len 8192 data: 4 4
16777 023/300 Read block 969 len 40960
16778 024/300 Read block 4197 len 36864
16779 025/300 Write block 3084 len 12288 data: 5 6 4
16780 026/300 Read block 3731 len 32768
16781 027/300 Write block 1384 len 40960 data: 2 3 3 5 6 5 4 5 4 5
16782 028/300 Read block 330 len 32768
16783 029/300 Read block 3050 len 20480
16784 030/300 Read block 3808 len 24576
16785 031/300 Write block 95 len 24576 data: 4 6 6 5 5 5
16786 032/300 Read block 119 len 36864
16787 033/300 Write block 1706 len 8192 data: 5 5
16788 034/300 Read block 1569 len 24576
16789 035/300 Read block 4137 len 32768
16790 036/300 Write block 3421 len 28672 data: 4 6 6 6 6 5 5
16791 037/300 Read block 1917 len 36864
16792 038/300 Read block 1133 len 16384
16793 039/300 Read block 4896 len 24576
16794 040/300 Read block 2290 len 12288
16795 041/300 Read block 794 len 28672
16796 042/300 Read block 4362 len 20480
16797 043/300 Flush
16798 044/300 Write block 3895 len 12288 data: 5 5 4
16799 045/300 Read block 661 len 16384
16800 046/300 Read block 1619 len 20480
16801 047/300 Write block 4482 len 28672 data: 5 4 4 4 3 3 3
16802 048/300 Write block 3301 len 24576 data: 2 3 3 3 3 4
16803 049/300 Read block 1302 len 36864
16804 050/300 Flush
16805 051/300 Write block 2111 len 24576 data: 4 4 4 6 6 6
16806 052/300 Flush
16807 053/300 Read block 2359 len 32768
16808 054/300 Write block 1455 len 32768 data: 5 4 4 4 3 2 2 2
16809 055/300 Write block 3760 len 12288 data: 3 4 4
16810 056/300 Read block 4642 len 28672
16811 057/300 Write block 252 len 16384 data: 4 4 4 5
16812 058/300 Write block 4790 len 20480 data: 3 3 2 2 2
16813 059/300 Flush
16814 060/300 Write block 2460 len 40960 data: 2 2 2 2 2 2 3 4 4 4
16815 061/300 Read block 3105 len 8192
16816 062/300 Read block 730 len 36864
16817 063/300 Read block 836 len 32768
16818 064/300 Flush
16819 065/300 Read block 4066 len 32768
16820 066/300 Read block 2083 len 8192
16821 067/300 Flush
16822 068/300 Read block 4152 len 20480
16823 069/300 Read block 4809 len 4096
16824 070/300 Read block 4381 len 28672
16825 071/300 Read block 1952 len 36864
16826 072/300 Read block 1148 len 16384
16827 073/300 Read block 644 len 4096
16828 074/300 Read block 4544 len 40960
16829 075/300 Read block 1980 len 16384
16830 076/300 Read block 1174 len 4096
16831 077/300 Write block 1775 len 28672 data: 4 3 3 3 4 3 4
16832 078/300 Flush
16833 079/300 Flush
16834 080/300 Write block 918 len 12288 data: 6 6 7
16835 081/300 Write block 1264 len 32768 data: 3 5 4 5 5 5 4 5
16836 082/300 Write block 3819 len 12288 data: 4 5 6
16837 083/300 Read block 4018 len 16384
16838 084/300 Write block 3565 len 20480 data: 3 3 4 4 6
16839 085/300 Read block 3149 len 28672
16840 086/300 Read block 195 len 24576
16841 087/300 Read block 1130 len 24576
16842 088/300 Read block 2286 len 20480
16843 089/300 Read block 3195 len 24576
16844 090/300 Write block 2197 len 32768 data: 3 2 4 4 4 4 4 5
16845 091/300 Write block 3554 len 36864 data: 3 4 4 4 3 4 4 3 3
16846 092/300 Flush
16847 093/300 Read block 1560 len 32768
16848 094/300 Read block 2833 len 40960
16849 095/300 Read block 1099 len 12288
16850 096/300 Write block 1159 len 24576 data: 6 7 5 3 3 4
16851 097/300 Read block 3891 len 36864
16852 098/300 Read block 4995 len 12288
16853 099/300 Read block 2325 len 12288
16854 100/300 Read block 4482 len 32768
16855 101/300 Write block 2637 len 40960 data: 3 3 4 4 5 4 3 3 3 3
16856 102/300 Write block 2262 len 8192 data: 2 3
16857 103/300 Write block 793 len 12288 data: 5 5 5
16858 104/300 Read block 1065 len 4096
16859 105/300 Write block 4767 len 16384 data: 4 5 7 6
16860 106/300 Write block 3843 len 4096 data: 4
16861 107/300 Read block 141 len 28672
16862 108/300 Read block 3888 len 12288
16863 109/300 Read block 322 len 36864
16864 110/300 Write block 4308 len 4096 data: 4
16865 111/300 Read block 3694 len 24576
16866 112/300 Write block 614 len 32768 data: 5 5 5 4 3 5 5 5
16867 113/300 Read block 4885 len 24576
16868 114/300 Read block 888 len 32768
16869 115/300 Read block 2633 len 8192
16870 116/300 Write block 2469 len 36864 data: 5 3 2 3 4 4 5 4 4
16871 117/300 Write block 3767 len 28672 data: 2 3 3 3 3 4 3
16872 118/300 Read block 3422 len 32768
16873 119/300 Write block 679 len 28672 data: 3 3 4 4 4 4 4
16874 120/300 Flush
16875 121/300 Read block 3753 len 12288
16876 122/300 Read block 621 len 8192
16877 123/300 Read block 3108 len 24576
16878 124/300 Read block 3627 len 4096
16879 125/300 Read block 4444 len 20480
16880 126/300 Read block 638 len 24576
16881 127/300 Write block 2093 len 36864 data: 4 3 3 3 4 3 3 2 2
16882 128/300 Read block 2516 len 16384
16883 129/300 Read block 4752 len 12288
16884 130/300 Write block 4944 len 28672 data: 5 4 4 5 6 5 4
16885 131/300 Read block 197 len 28672
16886 132/300 Read block 1972 len 16384
16887 133/300 Write block 2920 len 40960 data: 4 4 4 4 6 6 6 6 4 3
16888 134/300 Read block 4954 len 20480
16889 135/300 Write block 276 len 32768 data: 2 2 3 3 3 3 4 3
16890 136/300 Write block 2585 len 4096 data: 5
16891 137/300 Read block 3360 len 40960
16892 138/300 Flush
16893 139/300 Read block 2864 len 12288
16894 140/300 Read block 3263 len 12288
16895 141/300 Write block 93 len 16384 data: 2 3 5 7
16896 142/300 Read block 4449 len 40960
16897 143/300 Write block 3828 len 12288 data: 4 3 3
16898 144/300 Write block 1592 len 24576 data: 4 6 6 4 4 4
16899 145/300 Write block 4371 len 32768 data: 4 4 3 3 3 2 2 3
16900 146/300 Write block 4229 len 8192 data: 5 3
16901 147/300 Write block 736 len 28672 data: 4 3 4 4 4 3 4
16902 148/300 Write block 496 len 28672 data: 4 5 4 3 3 5 4
16903 149/300 Write block 3439 len 32768 data: 5 5 4 4 4 2 2 2
16904 150/300 Write block 4370 len 16384 data: 4 5 5 4
16905 151/300 Write block 1339 len 40960 data: 4 4 5 3 3 2 3 4 4 4
16906 152/300 Flush
16907 153/300 Write block 3596 len 4096 data: 4
16908 154/300 Write block 3799 len 12288 data: 4 4 4
16909 155/300 Write block 626 len 12288 data: 5 6 6
16910 156/300 Write block 3830 len 36864 data: 4 3 4 4 2 2 2 2 3
16911 157/300 Flush
16912 158/300 Write block 1362 len 4096 data: 5
16913 159/300 Read block 745 len 40960
16914 160/300 Write block 363 len 16384 data: 4 5 5 6
16915 161/300 Read block 2382 len 40960
16916 162/300 Write block 2944 len 4096 data: 3
16917 163/300 Write block 4127 len 12288 data: 4 3 3
16918 164/300 Write block 1378 len 24576 data: 2 2 2 2 2 2
16919 165/300 Read block 4045 len 32768
16920 166/300 Write block 3629 len 36864 data: 6 4 4 5 7 6 5 5 5
16921 167/300 Write block 2303 len 40960 data: 5 5 4 4 5 6 6 5 3 3
16922 168/300 Write block 3155 len 8192 data: 2 3
16923 169/300 Write block 32 len 40960 data: 2 2 2 2 2 3 3 3 3 3
16924 170/300 Write block 1741 len 8192 data: 4 2
16925 171/300 Read block 3840 len 40960
16926 172/300 Write block 735 len 12288 data: 4 5 4
16927 173/300 Read block 4758 len 36864
16928 174/300 Read block 861 len 20480
16929 175/300 Read block 31 len 36864
16930 176/300 Read block 2620 len 40960
16931 177/300 Write block 4863 len 24576 data: 4 3 3 3 3 4
16932 178/300 Write block 2420 len 40960 data: 6 6 6 6 5 5 4 2 2 2
16933 179/300 Read block 65 len 28672
16934 180/300 Flush
16935 181/300 Read block 4346 len 36864
16936 182/300 Flush
16937 183/300 Write block 1308 len 12288 data: 4 4 4
16938 184/300 Read block 4822 len 20480
16939 185/300 Write block 2367 len 40960 data: 5 4 4 6 6 5 5 5 3 3
16940 186/300 Read block 841 len 12288
16941 187/300 Flush
16942 188/300 Read block 3119 len 28672
16943 189/300 Flush
16944 190/300 Read block 4914 len 24576
16945 191/300 Read block 185 len 8192
16946 192/300 Read block 2268 len 36864
16947 193/300 Write block 3055 len 20480 data: 4 4 3 3 2
16948 194/300 Read block 1327 len 20480
16949 195/300 Read block 4499 len 32768
16950 196/300 Write block 1887 len 12288 data: 5 5 5
16951 197/300 Read block 2932 len 28672
16952 198/300 Read block 2220 len 8192
16953 199/300 Write block 3730 len 16384 data: 5 5 5 5
16954 200/300 Write block 4566 len 28672 data: 3 3 3 3 3 3 3
16955 201/300 Read block 3947 len 36864
16956 202/300 Write block 4708 len 4096 data: 8
16957 203/300 Write block 301 len 20480 data: 3 4 4 4 5
16958 204/300 Read block 145 len 24576
16959 205/300 Flush
16960 206/300 Write block 3361 len 12288 data: 5 4 4
16961 207/300 Write block 4281 len 16384 data: 3 3 3 2
16962 208/300 Read block 1448 len 36864
16963 209/300 Write block 3386 len 40960 data: 6 6 6 6 5 5 4 4 4 6
16964 210/300 Read block 2033 len 40960
16965 211/300 Read block 2454 len 4096
16966 212/300 Read block 4511 len 20480
16967 213/300 Write block 3917 len 32768 data: 2 2 3 3 3 3 3 4
16968 214/300 Write block 2481 len 28672 data: 5 6 7 8 8 7 7
16969 215/300 Write block 3869 len 4096 data: 6
16970 216/300 Read block 1049 len 4096
16971 217/300 Write block 4386 len 40960 data: 5 5 4 4 4 4 6 5 4 3
16972 218/300 Read block 997 len 8192
16973 219/300 Write block 2599 len 4096 data: 5
16974 220/300 Flush
16975 221/300 Write block 803 len 20480 data: 4 6 4 4 3
16976 222/300 Write block 1309 len 24576 data: 5 5 3 3 3 3
16977 223/300 Flush
16978 224/300 Write block 2498 len 40960 data: 2 2 2 2 2 3 4 4 4 4
16979 225/300 Write block 2313 len 12288 data: 3 4 3
16980 226/300 Write block 4622 len 32768 data: 5 5 4 4 4 4 3 3
16981 227/300 Flush
16982 228/300 Flush
16983 229/300 Write block 2868 len 36864 data: 2 2 3 4 4 4 3 3 3
16984 230/300 Read block 2775 len 16384
16985 231/300 Read block 4219 len 4096
16986 232/300 Write block 1021 len 24576 data: 3 3 3 3 3 4
16987 233/300 Read block 4171 len 28672
16988 234/300 Read block 347 len 28672
16989 235/300 Read block 1274 len 40960
16990 236/300 Write block 2687 len 16384 data: 6 5 5 5
16991 237/300 Write block 1396 len 4096 data: 7
16992 238/300 Write block 2304 len 32768 data: 6 5 5 6 7 7 6 4
16993 239/300 Write block 4520 len 4096 data: 2
16994 240/300 Read block 617 len 24576
16995 241/300 Write block 3442 len 12288 data: 5 5 3
16996 242/300 Read block 1707 len 40960
16997 243/300 Write block 4018 len 36864 data: 4 4 4 4 4 4 3 3 3
16998 244/300 Read block 4197 len 12288
16999 245/300 Read block 3241 len 20480
17000 246/300 Read block 1246 len 32768
17001 247/300 Read block 2038 len 40960
17002 248/300 Read block 189 len 8192
17003 249/300 Write block 2367 len 16384 data: 6 5 5 7
17004 250/300 Write block 4249 len 16384 data: 4 3 4 6
17005 251/300 Read block 694 len 24576
17006 252/300 Write block 4862 len 40960 data: 4 5 4 4 4 4 5 6 5 5
17007 253/300 Read block 2707 len 8192
17008 254/300 Write block 2802 len 28672 data: 3 2 3 4 5 6 6
17009 255/300 Read block 4180 len 40960
17010 256/300 Read block 47 len 20480
17011 257/300 Read block 392 len 40960
17012 258/300 Write block 4426 len 40960 data: 3 3 4 3 3 3 4 4 4 4
17013 259/300 Read block 2875 len 40960
17014 260/300 Write block 2072 len 32768 data: 6 7 6 6 4 4 5 3
17015 261/300 Read block 1022 len 32768
17016 262/300 Read block 1277 len 4096
17017 263/300 Write block 2927 len 28672 data: 7 5 4 3 3 3 2
17018 264/300 Read block 4847 len 24576
17019 265/300 Read block 4925 len 32768
17020 266/300 Read block 1323 len 24576
17021 267/300 Read block 1746 len 12288
17022 268/300 Write block 3600 len 4096 data: 5
17023 269/300 Flush
17024 270/300 Write block 2196 len 16384 data: 4 4 3 5
17025 271/300 Write block 1573 len 40960 data: 5 4 4 4 5 4 3 3 4 4
17026 272/300 Read block 3352 len 28672
17027 273/300 Read block 3999 len 32768
17028 274/300 Flush
17029 275/300 Read block 2322 len 12288
17030 276/300 Flush
17031 277/300 Read block 1197 len 16384
17032 278/300 Read block 3842 len 40960
17033 279/300 Read block 2691 len 4096
17034 280/300 Write block 972 len 40960 data: 2 3 4 4 4 4 4 4 3 3
17035 281/300 Read block 2045 len 40960
17036 282/300 Write block 4263 len 32768 data: 6 6 6 7 5 5 5 4
17037 283/300 Read block 675 len 12288
17038 284/300 Write block 2268 len 8192 data: 3 3
17039 285/300 Read block 2097 len 28672
17040 286/300 Read block 1464 len 24576
17041 287/300 Write block 4502 len 36864 data: 4 5 5 5 6 5 2 3 3
17042 288/300 Read block 4015 len 16384
17043 289/300 Flush
17044 290/300 Read block 1747 len 20480
17045 291/300 Read block 3372 len 16384
17046 292/300 Flush
17047 293/300 Read block 4817 len 24576
17048 294/300 Read block 1780 len 32768
17049 295/300 Write block 4236 len 32768 data: 2 3 3 4 5 5 5 5
17050 296/300 Read block 1794 len 16384
17051 297/300 Write block 1167 len 16384 data: 4 4 5 4
17052 298/300 Read block 1535 len 24576
17053 299/300 Write block 67 len 8192 data: 6 6
17054 300/300 Write block 811 len 8192 data: 3 4
17055 301/300 Write block 4689 len 4096 data: 4
17056 Main task has work to do, go find it
17057 got dsc Start(1)
17058 start 1
17059 Jan 03 08:34:29.570 INFO request completed, latency_us: 241, response_code: 204, uri: /start/cid/1, method: POST, req_id: 7b2bbda1-fbd9-482c-893f-8b0ef1a1ded7, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
17060 Replay: started 1, returned:Ok(())
17061 ----------------------------------------------------------------
17062 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17063 States: Active Offline Active
17064 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17065 5529 Acked Write 4 Done Sent Done false
17066 5530 Acked Read 8 Done Sent Done false
17067 5531 Acked Read 8 Done Sent Done false
17068 5532 Acked Read 2 Done Sent Done false
17069 5533 Acked Write 8 Done Sent Done false
17070 5534 Acked Read 1 Done Sent Done false
17071 5535 Acked Write 10 Done Sent Done false
17072 5536 Acked Write 2 Done Sent Done false
17073 5537 Acked Read 2 Done Sent Done false
17074 5538 Acked Write 10 Done Sent Done false
17075 5539 Acked Write 9 Done Sent Done false
17076 5540 Acked Flush 0 Done Sent Done false
17077 5541 Acked Flush 0 Done Sent Done false
17078 5542 Acked Read 2 Done Sent Done false
17079 5543 Acked Write 7 Done Sent Done false
17080 5544 Acked Read 4 Done Sent Done false
17081 5545 Acked Read 7 Done Sent Done false
17082 5546 Acked Flush 0 Done Sent Done false
17083 5547 Acked Read 2 Done Sent Done false
17084 5548 Acked Read 4 Done Sent Done false
17085 5549 Acked Write 4 Done Sent Done false
17086 5550 Acked Write 2 Done Sent Done false
17087 5551 Acked Read 10 Done Sent Done false
17088 5552 Acked Read 9 Done Sent Done false
17089 5553 Acked Write 3 Done Sent Done false
17090 5554 Acked Read 8 Done Sent Done false
17091 5555 Acked Write 10 Done Sent Done false
17092 5556 Acked Read 8 Done Sent Done false
17093 5557 Acked Read 5 Done Sent Done false
17094 5558 Acked Read 6 Done Sent Done false
17095 5559 Acked Write 6 Done Sent Done false
17096 5560 Acked Read 9 Done Sent Done false
17097 5561 Acked Write 2 Done Sent Done false
17098 5562 Acked Read 6 Done Sent Done false
17099 5563 Acked Read 8 Done Sent Done false
17100 5564 Acked Write 7 Done Sent Done false
17101 5565 Acked Read 9 Done Sent Done false
17102 5566 Acked Read 4 Done Sent Done false
17103 5567 Acked Read 6 Done Sent Done false
17104 5568 Acked Read 3 Done Sent Done false
17105 5569 Acked Read 7 Done Sent Done false
17106 5570 Acked Read 5 Done Sent Done false
17107 5571 Acked Flush 0 Done Sent Done false
17108 5572 Acked Write 3 Done Sent Done false
17109 5573 Acked Read 4 Done Sent Done false
17110 5574 Acked Read 5 Done Sent Done false
17111 5575 Acked Write 7 Done Sent Done false
17112 5576 Acked Write 6 Done Sent Done false
17113 5577 Acked Read 9 Done Sent Done false
17114 5578 Acked Flush 0 Done Sent Done false
17115 5579 Acked Write 6 Done Sent Done false
17116 5580 Acked Flush 0 Done Sent Done false
17117 5581 Acked Read 8 Done Sent Done false
17118 5582 Acked Write 8 Done Sent Done false
17119 5583 Acked Write 3 Done Sent Done false
17120 5584 Acked Read 7 Done Sent Done false
17121 5585 Acked Write 4 Done Sent Done false
17122 5586 Acked Write 5 Done Sent Done false
17123 5587 Acked Flush 0 Done Sent Done false
17124 5588 Acked Write 10 Done Sent Done false
17125 5589 Acked Read 2 Done Sent Done false
17126 5590 Acked Read 9 Done Sent Done false
17127 5591 Acked Read 8 Done Sent Done false
17128 5592 Acked Flush 0 Done Sent Done false
17129 5593 Acked Read 8 Done Sent Done false
17130 5594 Acked Read 2 Done Sent Done false
17131 5595 Acked Flush 0 Done Sent Done false
17132 5596 Acked Read 5 Done Sent Done false
17133 5597 Acked Read 1 Done Sent Done false
17134 5598 Acked Read 7 Done Sent Done false
17135 5599 Acked Read 9 Done Sent Done false
17136 5600 Acked Read 4 Done Sent Done false
17137 5601 Acked Read 1 Done Sent Done false
17138 5602 Acked Read 10 Done Sent Done false
17139 5603 Acked Read 4 Done Sent Done false
17140 5604 Acked Read 1 Done Sent Done false
17141 5605 Acked Write 7 Done Sent Done false
17142 5606 Acked Flush 0 Done Sent Done false
17143 5607 Acked Flush 0 Done Sent Done false
17144 5608 Acked Write 3 Done Sent Done false
17145 5609 Acked Write 8 Done Sent Done false
17146 5610 Acked Write 3 Done Sent Done false
17147 5611 Acked Read 4 Done Sent Done false
17148 5612 Acked Write 5 Done Sent Done false
17149 5613 Acked Read 7 Done Sent Done false
17150 5614 Acked Read 6 Done Sent Done false
17151 5615 Acked Read 6 Done Sent Done false
17152 5616 Acked Read 5 Done Sent Done false
17153 5617 Acked Read 6 Done Sent Done false
17154 5618 Acked Write 8 Done Sent Done false
17155 5619 Acked Write 9 Done Sent Done false
17156 5620 Acked Flush 0 Done Sent Done false
17157 5621 Acked Read 8 Done Sent Done false
17158 5622 Acked Read 10 Done Sent Done false
17159 5623 Acked Read 3 Done Sent Done false
17160 5624 Acked Write 6 Done Sent Done false
17161 5625 Acked Read 9 Done Sent Done false
17162 5626 Acked Read 3 Done Sent Done false
17163 5627 Acked Read 3 Done Sent Done false
17164 5628 Acked Read 8 Done Sent Done false
17165 5629 Acked Write 10 Done Sent Done false
17166 5630 Acked Write 2 Done Sent Done false
17167 5631 Acked Write 3 Done Sent Done false
17168 5632 Acked Read 1 Done Sent Done false
17169 5633 Acked Write 4 Done Sent Done false
17170 5634 Acked Write 1 Done Sent Done false
17171 5635 Acked Read 7 Done Sent Done false
17172 5636 Acked Read 3 Done Sent Done false
17173 5637 Acked Read 9 Done Sent Done false
17174 5638 Acked Write 1 Done Sent Done false
17175 5639 Acked Read 6 Done Sent Done false
17176 5640 Acked Write 8 Done Sent Done false
17177 5641 Acked Read 6 Done Sent Done false
17178 5642 Acked Read 8 Done Sent Done false
17179 5643 Acked Read 2 Done Sent Done false
17180 5644 Acked Write 9 Done Sent Done false
17181 5645 Acked Write 7 Done Sent Done false
17182 5646 Acked Read 8 Done Sent Done false
17183 5647 Acked Write 7 Done Sent Done false
17184 5648 Acked Flush 0 Done Sent Done false
17185 5649 Acked Read 3 Done Sent Done false
17186 5650 Acked Read 2 Done Sent Done false
17187 5651 Acked Read 6 Done Sent Done false
17188 5652 Acked Read 1 Done Sent Done false
17189 5653 Acked Read 5 Done Sent Done false
17190 5654 Acked Read 6 Done Sent Done false
17191 5655 Acked Write 9 Done Sent Done false
17192 5656 Acked Read 4 Done Sent Done false
17193 5657 Acked Read 3 Done Sent Done false
17194 5658 Acked Write 7 Done Sent Done false
17195 5659 Acked Read 7 Done Sent Done false
17196 5660 Acked Read 4 Done Sent Done false
17197 5661 Acked Write 10 Done Sent Done false
17198 5662 Acked Read 5 Done Sent Done false
17199 5663 Acked Write 8 Done Sent Done false
17200 5664 Acked Write 1 Done Sent Done false
17201 5665 Acked Read 10 Done Sent Done false
17202 5666 Acked Flush 0 Done Sent Done false
17203 5667 Acked Read 3 Done Sent Done false
17204 5668 Acked Read 3 Done Sent Done false
17205 5669 Acked Write 4 Done Sent Done false
17206 5670 Acked Read 10 Done Sent Done false
17207 5671 Acked Write 3 Done Sent Done false
17208 5672 Acked Write 6 Done Sent Done false
17209 5673 Acked Write 8 Done Sent Done false
17210 5674 Acked Write 2 Done Sent Done false
17211 5675 Acked Write 7 Done Sent Done false
17212 5676 Acked Write 7 Done Sent Done false
17213 5677 Acked Write 8 Done Sent Done false
17214 5678 Acked Write 4 Done Sent Done false
17215 5679 Acked Write 10 Done Sent Done false
17216 5680 Acked Flush 0 Done Sent Done false
17217 5681 Acked Write 1 Done Sent Done false
17218 5682 Acked Write 3 Done Sent Done false
17219 5683 Acked Write 3 Done Sent Done false
17220 5684 Acked Write 9 Done Sent Done false
17221 5685 Acked Flush 0 Done Sent Done false
17222 5686 Acked Write 1 Done Sent Done false
17223 5687 Acked Read 10 Done Sent Done false
17224 5688 Acked Write 4 Done Sent Done false
17225 5689 Acked Read 10 Done Sent Done false
17226 5690 Acked Write 1 Done Sent Done false
17227 5691 Acked Write 3 Done Sent Done false
17228 5692 Acked Write 6 Done Sent Done false
17229 5693 Acked Read 8 Done Sent Done false
17230 5694 Acked Write 9 Done Sent Done false
17231 5695 Acked Write 10 Done Sent Done false
17232 5696 Acked Write 2 Done Sent Done false
17233 5697 Acked Write 10 Done Sent Done false
17234 5698 Acked Write 2 Done Sent Done false
17235 5699 Acked Read 10 Done Sent Done false
17236 5700 Acked Write 3 Done Sent Done false
17237 5701 Acked Read 9 Done Sent Done false
17238 5702 Acked Read 5 Done Sent Done false
17239 5703 Acked Read 9 Done Sent Done false
17240 5704 Acked Read 10 Done Sent Done false
17241 5705 Acked Write 6 Done Sent Done false
17242 5706 Acked Write 10 Done Sent Done false
17243 5707 Acked Read 7 Done Sent Done false
17244 5708 Acked Flush 0 Done Sent Done false
17245 5709 Acked Read 9 Done Sent Done false
17246 5710 Acked Flush 0 Done Sent Done false
17247 5711 Acked Write 3 Done Sent Done false
17248 5712 Acked Read 5 Done Sent Done false
17249 5713 Acked Write 10 Done Sent Done false
17250 5714 Acked Read 3 Done Sent Done false
17251 5715 Acked Flush 0 Done Sent Done false
17252 5716 Acked Read 7 Done Sent Done false
17253 5717 Acked Flush 0 Done Sent Done false
17254 5718 Acked Read 6 Done Sent Done false
17255 5719 Acked Read 2 Done Sent Done false
17256 5720 Acked Read 9 Done Sent Done false
17257 5721 Acked Write 5 Done Sent Done false
17258 5722 Acked Read 5 Done Sent Done false
17259 5723 Acked Read 8 Done Sent Done false
17260 5724 Acked Write 3 Done Sent Done false
17261 5725 Acked Read 7 Done Sent Done false
17262 5726 Acked Read 2 Done Sent Done false
17263 5727 Acked Write 4 Done Sent Done false
17264 5728 Acked Write 7 Done Sent Done false
17265 5729 Acked Read 9 Done Sent Done false
17266 5730 Acked Write 1 Done Sent Done false
17267 5731 Acked Write 5 Done Sent Done false
17268 5732 Acked Read 6 Done Sent Done false
17269 5733 Acked Flush 0 Done Sent Done false
17270 5734 Acked Write 3 Done Sent Done false
17271 5735 Acked Write 4 Done Sent Done false
17272 5736 Acked Read 9 Done Sent Done false
17273 5737 Acked Write 10 Done Sent Done false
17274 5738 Acked Read 10 Done Sent Done false
17275 5739 Acked Read 1 Done Sent Done false
17276 5740 Acked Read 5 Done Sent Done false
17277 5741 Acked Write 8 Done Sent Done false
17278 5742 Acked Write 7 Done Sent Done false
17279 5743 Acked Write 1 Done Sent Done false
17280 5744 Acked Read 1 Done Sent Done false
17281 5745 Acked Write 10 Done Sent Done false
17282 5746 Acked Read 2 Done Sent Done false
17283 5747 Acked Write 1 Done Sent Done false
17284 5748 Acked Flush 0 Done Sent Done false
17285 5749 Acked Write 5 Done Sent Done false
17286 5750 Acked Write 6 Done Sent Done false
17287 5751 Acked Flush 0 Done Sent Done false
17288 5752 Acked Write 10 Done Sent Done false
17289 5753 Acked Write 3 Done Sent Done false
17290 5754 Acked Write 8 Done Sent Done false
17291 5755 Acked Flush 0 Done Sent Done false
17292 5756 Acked Flush 0 Done Sent Done false
17293 5757 Acked Write 9 Done Sent Done false
17294 5758 Acked Read 4 Done Sent Done false
17295 5759 Acked Read 1 Done Sent Done false
17296 5760 Acked Write 6 Done Sent Done false
17297 5761 Acked Read 7 Done Sent Done false
17298 5762 Acked Read 7 Done Sent Done false
17299 5763 Acked Read 10 Done Sent Done false
17300 5764 Acked Write 4 Done Sent Done false
17301 5765 Acked Write 1 Done Sent Done false
17302 5766 Acked Write 8 Done Sent Done false
17303 5767 Acked Write 1 Done Sent Done false
17304 5768 Acked Read 6 Done Sent Done false
17305 5769 Acked Write 3 Done Sent Done false
17306 5770 Acked Read 10 Done Sent Done false
17307 5771 Acked Write 9 Done Sent Done false
17308 5772 Acked Read 3 Done Sent Done false
17309 5773 Acked Read 5 Done Sent Done false
17310 5774 Acked Read 8 Done Sent Done false
17311 5775 Acked Read 10 Done Sent Done false
17312 5776 Acked Read 2 Done Sent Done false
17313 5777 Acked Write 4 Done Sent Done false
17314 5778 Acked Write 4 Done Sent Done false
17315 5779 Acked Read 6 Done Sent Done false
17316 5780 Acked Write 10 Done Sent Done false
17317 5781 Acked Read 2 Done Sent Done false
17318 5782 Acked Write 7 Done Sent Done false
17319 5783 Acked Read 10 Done Sent Done false
17320 5784 Acked Read 5 Done Sent Done false
17321 5785 Acked Read 10 Done Sent Done false
17322 5786 Acked Write 10 Done Sent Done false
17323 5787 Acked Read 10 Done Sent Done false
17324 5788 Acked Write 8 Done Sent Done false
17325 5789 Acked Read 8 Done Sent Done false
17326 5790 Acked Read 1 Done Sent Done false
17327 5791 Acked Write 7 Done Sent Done false
17328 5792 Acked Read 6 Done Sent Done false
17329 5793 Acked Read 8 Done Sent Done false
17330 5794 Acked Read 6 Done Sent Done false
17331 5795 Acked Read 3 Done Sent Done false
17332 5796 Acked Write 1 Done Sent Done false
17333 5797 Acked Flush 0 Done Sent Done false
17334 5798 Acked Write 4 Done Sent Done false
17335 5799 Acked Write 10 Done Sent Done false
17336 5800 Acked Read 7 Done Sent Done false
17337 5801 Acked Read 8 Done Sent Done false
17338 5802 Acked Flush 0 Done Sent Done false
17339 5803 Acked Read 3 Done Sent Done false
17340 5804 Acked Flush 0 Done Sent Done false
17341 5805 Acked Read 4 Done Sent Done false
17342 5806 Acked Read 10 Done Sent Done false
17343 5807 Acked Read 1 Done Sent Done false
17344 5808 Acked Write 10 Done Sent Done false
17345 5809 Acked Read 10 Done Sent Done false
17346 5810 Acked Write 8 Done Sent Done false
17347 5811 Acked Read 3 Done Sent Done false
17348 5812 Acked Write 2 Done Sent Done false
17349 5813 Acked Read 7 Done Sent Done false
17350 5814 Acked Read 6 Done Sent Done false
17351 5815 Acked Write 9 Done Sent Done false
17352 5816 Acked Read 4 Done Sent Done false
17353 5817 Acked Flush 0 Done Sent Done false
17354 5818 Acked Read 5 Done Sent Done false
17355 5819 Acked Read 4 Done Sent Done false
17356 5820 Acked Flush 0 Done Sent Done false
17357 5821 Acked Read 6 Done Sent Done false
17358 5822 Acked Read 8 Done Sent Done false
17359 5823 Acked Write 8 Done Sent Done false
17360 5824 Acked Read 4 Done Sent Done false
17361 5825 Acked Write 4 Done Sent Done false
17362 5826 Acked Read 6 Done Sent Done false
17363 5827 Acked Write 2 Done Sent Done false
17364 5828 Acked Write 2 Done Sent Done false
17365 5829 Acked Write 1 Done Sent Done false
17366 STATES DS:0 DS:1 DS:2 TOTAL
17367 Sent 0 301 0 301
17368 Done 301 0 301 602
17369 Skipped 0 0 0 0
17370 Error 0 0 0 0
17371 Last Flush: 5820 5528 5820
17372 Downstairs last five completed: 5528 5527 5526 5525 5524
17373 Upstairs last five completed: 5829 5828 5827 5826 5825
17374 CLIENT: Up:0 ds:301 act:2
17375 [8820] got start action
17376 [8820] I am going to restart
17377 Starting downstairs at port 8820
17378 Make output file at "/tmp/dsc/downstairs-8820.txt"
17379 [8820][1] reports Starting
17380 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1221)
17381 [8820][1] reports Running
17382 ----------------------------------------------------------------
17383 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17384 States: Active Offline Active
17385 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17386 5529 Acked Write 4 Done Sent Done false
17387 5530 Acked Read 8 Done Sent Done false
17388 5531 Acked Read 8 Done Sent Done false
17389 5532 Acked Read 2 Done Sent Done false
17390 5533 Acked Write 8 Done Sent Done false
17391 5534 Acked Read 1 Done Sent Done false
17392 5535 Acked Write 10 Done Sent Done false
17393 5536 Acked Write 2 Done Sent Done false
17394 5537 Acked Read 2 Done Sent Done false
17395 5538 Acked Write 10 Done Sent Done false
17396 5539 Acked Write 9 Done Sent Done false
17397 5540 Acked Flush 0 Done Sent Done false
17398 5541 Acked Flush 0 Done Sent Done false
17399 5542 Acked Read 2 Done Sent Done false
17400 5543 Acked Write 7 Done Sent Done false
17401 5544 Acked Read 4 Done Sent Done false
17402 5545 Acked Read 7 Done Sent Done false
17403 5546 Acked Flush 0 Done Sent Done false
17404 5547 Acked Read 2 Done Sent Done false
17405 5548 Acked Read 4 Done Sent Done false
17406 5549 Acked Write 4 Done Sent Done false
17407 5550 Acked Write 2 Done Sent Done false
17408 5551 Acked Read 10 Done Sent Done false
17409 5552 Acked Read 9 Done Sent Done false
17410 5553 Acked Write 3 Done Sent Done false
17411 5554 Acked Read 8 Done Sent Done false
17412 5555 Acked Write 10 Done Sent Done false
17413 5556 Acked Read 8 Done Sent Done false
17414 5557 Acked Read 5 Done Sent Done false
17415 5558 Acked Read 6 Done Sent Done false
17416 5559 Acked Write 6 Done Sent Done false
17417 5560 Acked Read 9 Done Sent Done false
17418 5561 Acked Write 2 Done Sent Done false
17419 5562 Acked Read 6 Done Sent Done false
17420 5563 Acked Read 8 Done Sent Done false
17421 5564 Acked Write 7 Done Sent Done false
17422 5565 Acked Read 9 Done Sent Done false
17423 5566 Acked Read 4 Done Sent Done false
17424 5567 Acked Read 6 Done Sent Done false
17425 5568 Acked Read 3 Done Sent Done false
17426 5569 Acked Read 7 Done Sent Done false
17427 5570 Acked Read 5 Done Sent Done false
17428 5571 Acked Flush 0 Done Sent Done false
17429 5572 Acked Write 3 Done Sent Done false
17430 5573 Acked Read 4 Done Sent Done false
17431 5574 Acked Read 5 Done Sent Done false
17432 5575 Acked Write 7 Done Sent Done false
17433 5576 Acked Write 6 Done Sent Done false
17434 5577 Acked Read 9 Done Sent Done false
17435 5578 Acked Flush 0 Done Sent Done false
17436 5579 Acked Write 6 Done Sent Done false
17437 5580 Acked Flush 0 Done Sent Done false
17438 5581 Acked Read 8 Done Sent Done false
17439 5582 Acked Write 8 Done Sent Done false
17440 5583 Acked Write 3 Done Sent Done false
17441 5584 Acked Read 7 Done Sent Done false
17442 5585 Acked Write 4 Done Sent Done false
17443 5586 Acked Write 5 Done Sent Done false
17444 5587 Acked Flush 0 Done Sent Done false
17445 5588 Acked Write 10 Done Sent Done false
17446 5589 Acked Read 2 Done Sent Done false
17447 5590 Acked Read 9 Done Sent Done false
17448 5591 Acked Read 8 Done Sent Done false
17449 5592 Acked Flush 0 Done Sent Done false
17450 5593 Acked Read 8 Done Sent Done false
17451 5594 Acked Read 2 Done Sent Done false
17452 5595 Acked Flush 0 Done Sent Done false
17453 5596 Acked Read 5 Done Sent Done false
17454 5597 Acked Read 1 Done Sent Done false
17455 5598 Acked Read 7 Done Sent Done false
17456 5599 Acked Read 9 Done Sent Done false
17457 5600 Acked Read 4 Done Sent Done false
17458 5601 Acked Read 1 Done Sent Done false
17459 5602 Acked Read 10 Done Sent Done false
17460 5603 Acked Read 4 Done Sent Done false
17461 5604 Acked Read 1 Done Sent Done false
17462 5605 Acked Write 7 Done Sent Done false
17463 5606 Acked Flush 0 Done Sent Done false
17464 5607 Acked Flush 0 Done Sent Done false
17465 5608 Acked Write 3 Done Sent Done false
17466 5609 Acked Write 8 Done Sent Done false
17467 5610 Acked Write 3 Done Sent Done false
17468 5611 Acked Read 4 Done Sent Done false
17469 5612 Acked Write 5 Done Sent Done false
17470 5613 Acked Read 7 Done Sent Done false
17471 5614 Acked Read 6 Done Sent Done false
17472 5615 Acked Read 6 Done Sent Done false
17473 5616 Acked Read 5 Done Sent Done false
17474 5617 Acked Read 6 Done Sent Done false
17475 5618 Acked Write 8 Done Sent Done false
17476 5619 Acked Write 9 Done Sent Done false
17477 5620 Acked Flush 0 Done Sent Done false
17478 5621 Acked Read 8 Done Sent Done false
17479 5622 Acked Read 10 Done Sent Done false
17480 5623 Acked Read 3 Done Sent Done false
17481 5624 Acked Write 6 Done Sent Done false
17482 5625 Acked Read 9 Done Sent Done false
17483 5626 Acked Read 3 Done Sent Done false
17484 5627 Acked Read 3 Done Sent Done false
17485 5628 Acked Read 8 Done Sent Done false
17486 5629 Acked Write 10 Done Sent Done false
17487 5630 Acked Write 2 Done Sent Done false
17488 5631 Acked Write 3 Done Sent Done false
17489 5632 Acked Read 1 Done Sent Done false
17490 5633 Acked Write 4 Done Sent Done false
17491 5634 Acked Write 1 Done Sent Done false
17492 5635 Acked Read 7 Done Sent Done false
17493 5636 Acked Read 3 Done Sent Done false
17494 5637 Acked Read 9 Done Sent Done false
17495 5638 Acked Write 1 Done Sent Done false
17496 5639 Acked Read 6 Done Sent Done false
17497 5640 Acked Write 8 Done Sent Done false
17498 5641 Acked Read 6 Done Sent Done false
17499 5642 Acked Read 8 Done Sent Done false
17500 5643 Acked Read 2 Done Sent Done false
17501 5644 Acked Write 9 Done Sent Done false
17502 5645 Acked Write 7 Done Sent Done false
17503 5646 Acked Read 8 Done Sent Done false
17504 5647 Acked Write 7 Done Sent Done false
17505 5648 Acked Flush 0 Done Sent Done false
17506 5649 Acked Read 3 Done Sent Done false
17507 5650 Acked Read 2 Done Sent Done false
17508 5651 Acked Read 6 Done Sent Done false
17509 5652 Acked Read 1 Done Sent Done false
17510 5653 Acked Read 5 Done Sent Done false
17511 5654 Acked Read 6 Done Sent Done false
17512 5655 Acked Write 9 Done Sent Done false
17513 5656 Acked Read 4 Done Sent Done false
17514 5657 Acked Read 3 Done Sent Done false
17515 5658 Acked Write 7 Done Sent Done false
17516 5659 Acked Read 7 Done Sent Done false
17517 5660 Acked Read 4 Done Sent Done false
17518 5661 Acked Write 10 Done Sent Done false
17519 5662 Acked Read 5 Done Sent Done false
17520 5663 Acked Write 8 Done Sent Done false
17521 5664 Acked Write 1 Done Sent Done false
17522 5665 Acked Read 10 Done Sent Done false
17523 5666 Acked Flush 0 Done Sent Done false
17524 5667 Acked Read 3 Done Sent Done false
17525 5668 Acked Read 3 Done Sent Done false
17526 5669 Acked Write 4 Done Sent Done false
17527 5670 Acked Read 10 Done Sent Done false
17528 5671 Acked Write 3 Done Sent Done false
17529 5672 Acked Write 6 Done Sent Done false
17530 5673 Acked Write 8 Done Sent Done false
17531 5674 Acked Write 2 Done Sent Done false
17532 5675 Acked Write 7 Done Sent Done false
17533 5676 Acked Write 7 Done Sent Done false
17534 5677 Acked Write 8 Done Sent Done false
17535 5678 Acked Write 4 Done Sent Done false
17536 5679 Acked Write 10 Done Sent Done false
17537 5680 Acked Flush 0 Done Sent Done false
17538 5681 Acked Write 1 Done Sent Done false
17539 5682 Acked Write 3 Done Sent Done false
17540 5683 Acked Write 3 Done Sent Done false
17541 5684 Acked Write 9 Done Sent Done false
17542 5685 Acked Flush 0 Done Sent Done false
17543 5686 Acked Write 1 Done Sent Done false
17544 5687 Acked Read 10 Done Sent Done false
17545 5688 Acked Write 4 Done Sent Done false
17546 5689 Acked Read 10 Done Sent Done false
17547 5690 Acked Write 1 Done Sent Done false
17548 5691 Acked Write 3 Done Sent Done false
17549 5692 Acked Write 6 Done Sent Done false
17550 5693 Acked Read 8 Done Sent Done false
17551 5694 Acked Write 9 Done Sent Done false
17552 5695 Acked Write 10 Done Sent Done false
17553 5696 Acked Write 2 Done Sent Done false
17554 5697 Acked Write 10 Done Sent Done false
17555 5698 Acked Write 2 Done Sent Done false
17556 5699 Acked Read 10 Done Sent Done false
17557 5700 Acked Write 3 Done Sent Done false
17558 5701 Acked Read 9 Done Sent Done false
17559 5702 Acked Read 5 Done Sent Done false
17560 5703 Acked Read 9 Done Sent Done false
17561 5704 Acked Read 10 Done Sent Done false
17562 5705 Acked Write 6 Done Sent Done false
17563 5706 Acked Write 10 Done Sent Done false
17564 5707 Acked Read 7 Done Sent Done false
17565 5708 Acked Flush 0 Done Sent Done false
17566 5709 Acked Read 9 Done Sent Done false
17567 5710 Acked Flush 0 Done Sent Done false
17568 5711 Acked Write 3 Done Sent Done false
17569 5712 Acked Read 5 Done Sent Done false
17570 5713 Acked Write 10 Done Sent Done false
17571 5714 Acked Read 3 Done Sent Done false
17572 5715 Acked Flush 0 Done Sent Done false
17573 5716 Acked Read 7 Done Sent Done false
17574 5717 Acked Flush 0 Done Sent Done false
17575 5718 Acked Read 6 Done Sent Done false
17576 5719 Acked Read 2 Done Sent Done false
17577 5720 Acked Read 9 Done Sent Done false
17578 5721 Acked Write 5 Done Sent Done false
17579 5722 Acked Read 5 Done Sent Done false
17580 5723 Acked Read 8 Done Sent Done false
17581 5724 Acked Write 3 Done Sent Done false
17582 5725 Acked Read 7 Done Sent Done false
17583 5726 Acked Read 2 Done Sent Done false
17584 5727 Acked Write 4 Done Sent Done false
17585 5728 Acked Write 7 Done Sent Done false
17586 5729 Acked Read 9 Done Sent Done false
17587 5730 Acked Write 1 Done Sent Done false
17588 5731 Acked Write 5 Done Sent Done false
17589 5732 Acked Read 6 Done Sent Done false
17590 5733 Acked Flush 0 Done Sent Done false
17591 5734 Acked Write 3 Done Sent Done false
17592 5735 Acked Write 4 Done Sent Done false
17593 5736 Acked Read 9 Done Sent Done false
17594 5737 Acked Write 10 Done Sent Done false
17595 5738 Acked Read 10 Done Sent Done false
17596 5739 Acked Read 1 Done Sent Done false
17597 5740 Acked Read 5 Done Sent Done false
17598 5741 Acked Write 8 Done Sent Done false
17599 5742 Acked Write 7 Done Sent Done false
17600 5743 Acked Write 1 Done Sent Done false
17601 5744 Acked Read 1 Done Sent Done false
17602 5745 Acked Write 10 Done Sent Done false
17603 5746 Acked Read 2 Done Sent Done false
17604 5747 Acked Write 1 Done Sent Done false
17605 5748 Acked Flush 0 Done Sent Done false
17606 5749 Acked Write 5 Done Sent Done false
17607 5750 Acked Write 6 Done Sent Done false
17608 5751 Acked Flush 0 Done Sent Done false
17609 5752 Acked Write 10 Done Sent Done false
17610 5753 Acked Write 3 Done Sent Done false
17611 5754 Acked Write 8 Done Sent Done false
17612 5755 Acked Flush 0 Done Sent Done false
17613 5756 Acked Flush 0 Done Sent Done false
17614 5757 Acked Write 9 Done Sent Done false
17615 5758 Acked Read 4 Done Sent Done false
17616 5759 Acked Read 1 Done Sent Done false
17617 5760 Acked Write 6 Done Sent Done false
17618 5761 Acked Read 7 Done Sent Done false
17619 5762 Acked Read 7 Done Sent Done false
17620 5763 Acked Read 10 Done Sent Done false
17621 5764 Acked Write 4 Done Sent Done false
17622 5765 Acked Write 1 Done Sent Done false
17623 5766 Acked Write 8 Done Sent Done false
17624 5767 Acked Write 1 Done Sent Done false
17625 5768 Acked Read 6 Done Sent Done false
17626 5769 Acked Write 3 Done Sent Done false
17627 5770 Acked Read 10 Done Sent Done false
17628 5771 Acked Write 9 Done Sent Done false
17629 5772 Acked Read 3 Done Sent Done false
17630 5773 Acked Read 5 Done Sent Done false
17631 5774 Acked Read 8 Done Sent Done false
17632 5775 Acked Read 10 Done Sent Done false
17633 5776 Acked Read 2 Done Sent Done false
17634 5777 Acked Write 4 Done Sent Done false
17635 5778 Acked Write 4 Done Sent Done false
17636 5779 Acked Read 6 Done Sent Done false
17637 5780 Acked Write 10 Done Sent Done false
17638 5781 Acked Read 2 Done Sent Done false
17639 5782 Acked Write 7 Done Sent Done false
17640 5783 Acked Read 10 Done Sent Done false
17641 5784 Acked Read 5 Done Sent Done false
17642 5785 Acked Read 10 Done Sent Done false
17643 5786 Acked Write 10 Done Sent Done false
17644 5787 Acked Read 10 Done Sent Done false
17645 5788 Acked Write 8 Done Sent Done false
17646 5789 Acked Read 8 Done Sent Done false
17647 5790 Acked Read 1 Done Sent Done false
17648 5791 Acked Write 7 Done Sent Done false
17649 5792 Acked Read 6 Done Sent Done false
17650 5793 Acked Read 8 Done Sent Done false
17651 5794 Acked Read 6 Done Sent Done false
17652 5795 Acked Read 3 Done Sent Done false
17653 5796 Acked Write 1 Done Sent Done false
17654 5797 Acked Flush 0 Done Sent Done false
17655 5798 Acked Write 4 Done Sent Done false
17656 5799 Acked Write 10 Done Sent Done false
17657 5800 Acked Read 7 Done Sent Done false
17658 5801 Acked Read 8 Done Sent Done false
17659 5802 Acked Flush 0 Done Sent Done false
17660 5803 Acked Read 3 Done Sent Done false
17661 5804 Acked Flush 0 Done Sent Done false
17662 5805 Acked Read 4 Done Sent Done false
17663 5806 Acked Read 10 Done Sent Done false
17664 5807 Acked Read 1 Done Sent Done false
17665 5808 Acked Write 10 Done Sent Done false
17666 5809 Acked Read 10 Done Sent Done false
17667 5810 Acked Write 8 Done Sent Done false
17668 5811 Acked Read 3 Done Sent Done false
17669 5812 Acked Write 2 Done Sent Done false
17670 5813 Acked Read 7 Done Sent Done false
17671 5814 Acked Read 6 Done Sent Done false
17672 5815 Acked Write 9 Done Sent Done false
17673 5816 Acked Read 4 Done Sent Done false
17674 5817 Acked Flush 0 Done Sent Done false
17675 5818 Acked Read 5 Done Sent Done false
17676 5819 Acked Read 4 Done Sent Done false
17677 5820 Acked Flush 0 Done Sent Done false
17678 5821 Acked Read 6 Done Sent Done false
17679 5822 Acked Read 8 Done Sent Done false
17680 5823 Acked Write 8 Done Sent Done false
17681 5824 Acked Read 4 Done Sent Done false
17682 5825 Acked Write 4 Done Sent Done false
17683 5826 Acked Read 6 Done Sent Done false
17684 5827 Acked Write 2 Done Sent Done false
17685 5828 Acked Write 2 Done Sent Done false
17686 5829 Acked Write 1 Done Sent Done false
17687 5830 Acked Flush 0 Done Sent Done false
17688 STATES DS:0 DS:1 DS:2 TOTAL
17689 Sent 0 302 0 302
17690 Done 302 0 302 604
17691 Skipped 0 0 0 0
17692 Error 0 0 0 0
17693 Last Flush: 5830 5528 5830
17694 Downstairs last five completed: 5528 5527 5526 5525 5524
17695 Upstairs last five completed: 5830 5829 5828 5827 5826
17696 CLIENT: Up:0 ds:302 act:2
176972025-01-03T08:34:35.321ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
176982025-01-03T08:34:35.322ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40787) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
176992025-01-03T08:34:35.323ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177002025-01-03T08:34:35.323ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177012025-01-03T08:34:35.323ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177022025-01-03T08:34:35.323ZINFOcrucible: send last flush ID to this DS: 5528 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177032025-01-03T08:34:35.324ZINFOcrucible: Replied this last flush ID: 5528 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177042025-01-03T08:34:35.324ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177052025-01-03T08:34:35.324ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177062025-01-03T08:34:35.324ZINFOcrucible: [1] client re-new 302 jobs since flush 5528 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177072025-01-03T08:34:35.324ZINFOcrucible: [1] Replayed 302 jobs since flush: 5528 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
17708 ----------------------------------------------------------------
17709 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17710 Downstairs last five completed: 5830 5829 5828 5827 5826
17711 Upstairs last five completed: 5830 5829 5828 5827 5826
17712 CLIENT: Up:0 ds:0 act:3
17713 Replay: All jobs finished
17714 Main task has work to do, go find it
17715 got dsc Stop(1)
17716 stop 1
17717 Jan 03 08:34:37.575 INFO [8820] Got stop action so:false kr:false
17718 request completed, latency_us: 322, response_code: 204, uri: /stop/cid/1, method: POST, req_id: a92c6395-9509-4da8-88aa-d47bf6d31352, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
177192025-01-03T08:34:37.576ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177202025-01-03T08:34:37.576ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
177212025-01-03T08:34:37.576ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
17722 Jan 03 08:34:37.576 INFO request completed, latency_us: 154, response_code: 200{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:34:37.576476357Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
17723 , uri: /state/cid/1, method: GET, req_id: 0e104c42-abd3-408a-b85d-56aa73df93fe, remote_addr: 127.0.0.{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:34:37.576502606Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
17724 1:49669, local_addr: 127.0.0.1:9998
177252025-01-03T08:34:37.576ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
17726 [8820] Exited with: ExitStatus(unix_wait_status(9))
17727 [8820][1] reports Exit
17728 Jan 03 08:34:41.579 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/1, method: GET, req_id: 2fa655c0-fcae-43c2-9062-333411be203c, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
17729 001/300 Write block 353 len 4096 data: 5
17730 002/300 Read block 209 len 36864
17731 003/300 Read block 2663 len 36864
17732 004/300 Write block 308 len 4096 data: 6
17733 005/300 Read block 4575 len 36864
17734 006/300 Write block 1111 len 20480 data: 5 5 5 5 5
17735 007/300 Write block 2391 len 40960 data: 4 4 4 4 4 4 3 2 2 3
17736 008/300 Flush
17737 009/300 Read block 4333 len 24576
17738 010/300 Read block 2870 len 32768
17739 011/300 Flush
17740 012/300 Write block 170 len 12288 data: 3 3 5
17741 013/300 Write block 2842 len 8192 data: 5 4
17742 014/300 Read block 1788 len 16384
17743 015/300 Read block 4446 len 40960
17744 016/300 Write block 4830 len 36864 data: 4 4 4 4 4 4 4 4 4
17745 017/300 Write block 4865 len 28672 data: 5 5 5 6 7 6 6
17746 018/300 Write block 935 len 12288 data: 5 7 7
17747 019/300 Read block 1471 len 16384
17748 020/300 Read block 1131 len 28672
17749 021/300 Read block 2137 len 16384
17750 022/300 Write block 3301 len 12288 data: 3 4 4
17751 023/300 Flush
17752 024/300 Read block 2238 len 16384
17753 025/300 Write block 2313 len 40960 data: 4 5 4 4 5 4 4 4 4 4
17754 026/300 Write block 4295 len 28672 data: 3 2 2 2 3 3 3
17755 027/300 Write block 4843 len 40960 data: 5 5 6 6 4 4 4 4 4 3
17756 028/300 Write block 1424 len 20480 data: 6 5 5 4 3
17757 029/300 Read block 4647 len 36864
17758 030/300 Write block 4282 len 32768 data: 4 4 3 2 2 2 2 2
17759 031/300 Flush
17760 032/300 Write block 1355 len 40960 data: 5 5 5 6 5 6 5 6 6 4
17761 033/300 Read block 3042 len 36864
17762 034/300 Write block 3418 len 28672 data: 2 4 3 5 7 7 7
17763 035/300 Read block 56 len 12288
17764 036/300 Read block 144 len 20480
17765 037/300 Read block 3254 len 28672
17766 038/300 Write block 2441 len 12288 data: 3 3 4
17767 039/300 Read block 3850 len 40960
17768 040/300 Flush
17769 041/300 Write block 1774 len 16384 data: 4 5 4 4
17770 042/300 Read block 1472 len 8192
17771 043/300 Write block 4858 len 24576 data: 4 3 3 3 5 6
17772 044/300 Flush
17773 045/300 Read block 609 len 20480
17774 046/300 Write block 2239 len 4096 data: 3
17775 047/300 Write block 4649 len 4096 data: 5
17776 048/300 Read block 4417 len 36864
17777 049/300 Flush
17778 050/300 Read block 3946 len 24576
17779 051/300 Flush
17780 052/300 Read block 461 len 4096
17781 053/300 Write block 2328 len 12288 data: 6 5 5
17782 054/300 Write block 875 len 12288 data: 3 4 4
17783 055/300 Read block 4567 len 40960
17784 056/300 Read block 1826 len 32768
17785 057/300 Write block 3357 len 32768 data: 4 4 4 5 6 5 5 4
17786 058/300 Read block 4684 len 16384
17787 059/300 Write block 4765 len 36864 data: 5 4 5 6 8 7 5 3 3
17788 060/300 Read block 29 len 12288
17789 061/300 Read block 773 len 36864
17790 062/300 Read block 2763 len 8192
17791 063/300 Write block 776 len 28672 data: 3 3 3 4 4 3 3
17792 064/300 Write block 3703 len 36864 data: 5 5 5 4 4 4 4 4 3
17793 065/300 Read block 380 len 16384
17794 066/300 Read block 4656 len 20480
17795 067/300 Read block 4762 len 24576
17796 068/300 Read block 1368 len 40960
17797 069/300 Read block 1378 len 12288
17798 070/300 Read block 843 len 40960
17799 071/300 Read block 158 len 28672
17800 072/300 Write block 4051 len 12288 data: 2 2 2
17801 073/300 Read block 2606 len 40960
17802 074/300 Read block 4599 len 20480
17803 075/300 Flush
17804 076/300 Read block 1617 len 24576
17805 077/300 Read block 1936 len 16384
17806 078/300 Flush
17807 079/300 Write block 2160 len 4096 data: 4
17808 080/300 Write block 4472 len 28672 data: 4 4 4 4 5 4 5
17809 081/300 Read block 1023 len 36864
17810 082/300 Write block 887 len 32768 data: 6 7 7 6 6 5 4 6
17811 083/300 Flush
17812 084/300 Write block 842 len 12288 data: 7 8 8
17813 085/300 Read block 4597 len 32768
17814 086/300 Read block 1919 len 4096
17815 087/300 Write block 2031 len 32768 data: 4 3 2 3 3 3 2 2
17816 088/300 Flush
17817 089/300 Read block 4024 len 32768
17818 090/300 Write block 1980 len 12288 data: 3 3 4
17819 091/300 Read block 2330 len 16384
17820 092/300 Read block 4266 len 36864
17821 093/300 Flush
17822 094/300 Write block 3894 len 36864 data: 5 6 6 5 3 3 4 6 5
17823 095/300 Read block 4577 len 28672
17824 096/300 Read block 2935 len 20480
17825 097/300 Read block 2683 len 8192
17826 098/300 Read block 1188 len 40960
17827 099/300 Write block 2853 len 20480 data: 3 4 4 5 6
17828 100/300 Write block 4716 len 24576 data: 3 3 2 2 3 3
17829 101/300 Read block 1805 len 32768
17830 102/300 Read block 772 len 28672
17831 103/300 Read block 1206 len 28672
17832 104/300 Write block 3613 len 32768 data: 4 4 4 4 3 3 3 4
17833 105/300 Write block 4910 len 8192 data: 6 6
17834 106/300 Read block 18 len 16384
17835 107/300 Write block 4370 len 24576 data: 5 6 6 5 4 4
17836 108/300 Flush
17837 109/300 Write block 4277 len 36864 data: 4 4 3 3 4 5 5 4 3
17838 110/300 Write block 3178 len 28672 data: 6 6 6 5 4 5 4
17839 111/300 Read block 3608 len 20480
17840 112/300 Write block 71 len 4096 data: 5
17841 113/300 Read block 3596 len 20480
17842 114/300 Write block 3411 len 12288 data: 3 4 4
17843 115/300 Write block 4043 len 36864 data: 6 6 5 5 5 5 3 3 3
17844 116/300 Read block 4093 len 40960
17845 117/300 Write block 2187 len 20480 data: 3 3 3 2 3
17846 118/300 Write block 4465 len 36864 data: 4 5 5 5 5 5 5 5 5
17847 119/300 Read block 611 len 20480
17848 120/300 Read block 2560 len 24576
17849 121/300 Read block 2573 len 8192
17850 122/300 Write block 3328 len 36864 data: 4 4 4 5 5 5 5 3 3
17851 123/300 Read block 4411 len 24576
17852 124/300 Write block 2839 len 28672 data: 5 6 6 6 5 4 4
17853 125/300 Read block 1842 len 16384
17854 126/300 Read block 4956 len 28672
17855 127/300 Read block 2539 len 36864
17856 128/300 Flush
17857 129/300 Flush
17858 130/300 Read block 3753 len 28672
17859 131/300 Write block 1152 len 16384 data: 2 3 4 4
17860 132/300 Write block 4134 len 20480 data: 2 3 3 4 4
17861 133/300 Write block 2659 len 36864 data: 3 7 6 6 6 5 4 4 4
17862 134/300 Write block 114 len 16384 data: 4 4 3 3
17863 135/300 Flush
17864 136/300 Read block 4331 len 36864
17865 137/300 Read block 3461 len 32768
17866 138/300 Read block 4986 len 36864
17867 139/300 Write block 1636 len 36864 data: 3 2 2 2 2 2 3 4 4
17868 140/300 Write block 3297 len 8192 data: 3 3
17869 141/300 Flush
17870 142/300 Write block 486 len 28672 data: 4 3 5 6 4 5 5
17871 143/300 Write block 267 len 36864 data: 2 2 2 2 3 3 2 2 2
17872 144/300 Write block 2227 len 24576 data: 4 4 4 2 3 3
17873 145/300 Write block 4335 len 24576 data: 5 5 5 4 4 5
17874 146/300 Read block 2482 len 8192
17875 147/300 Read block 3124 len 32768
17876 148/300 Write block 4020 len 24576 data: 5 5 5 5 4 4
17877 149/300 Flush
17878 150/300 Write block 4702 len 12288 data: 4 4 5
17879 151/300 Read block 960 len 8192
17880 152/300 Read block 1087 len 12288
17881 153/300 Write block 4354 len 16384 data: 2 4 4 4
17882 154/300 Read block 1520 len 36864
17883 155/300 Write block 4914 len 32768 data: 3 2 2 2 4 5 5 5
17884 156/300 Read block 3874 len 24576
17885 157/300 Read block 3320 len 40960
17886 158/300 Read block 745 len 20480
17887 159/300 Write block 1169 len 24576 data: 6 5 5 4 4 5
17888 160/300 Write block 1612 len 32768 data: 2 2 3 2 2 2 3 3
17889 161/300 Read block 1351 len 4096
17890 162/300 Read block 1516 len 24576
17891 163/300 Read block 3457 len 24576
17892 164/300 Write block 3422 len 32768 data: 8 8 8 7 6 6 6 6
17893 165/300 Read block 4578 len 20480
17894 166/300 Write block 1890 len 40960 data: 5 4 4 4 3 3 5 4 4 4
17895 167/300 Read block 750 len 20480
17896 168/300 Read block 4102 len 32768
17897 169/300 Write block 3424 len 12288 data: 9 8 7
17898 170/300 Write block 1887 len 4096 data: 6
17899 171/300 Read block 2882 len 24576
17900 172/300 Read block 3833 len 40960
17901 173/300 Read block 425 len 8192
17902 174/300 Write block 4927 len 36864 data: 3 3 4 6 5 7 6 5 6
17903 175/300 Write block 3484 len 20480 data: 3 3 3 3 3
17904 176/300 Write block 3065 len 32768 data: 4 3 2 3 3 3 3 3
17905 177/300 Write block 1823 len 8192 data: 3 4
17906 178/300 Read block 3332 len 36864
17907 179/300 Read block 538 len 16384
17908 180/300 Write block 244 len 12288 data: 3 4 5
17909 181/300 Read block 3861 len 12288
17910 182/300 Write block 4148 len 8192 data: 4 4
17911 183/300 Write block 1686 len 20480 data: 3 3 3 3 3
17912 184/300 Read block 368 len 16384
17913 185/300 Write block 3043 len 12288 data: 5 5 6
17914 186/300 Read block 4269 len 32768
17915 187/300 Read block 4413 len 16384
17916 188/300 Read block 3068 len 36864
17917 189/300 Read block 1271 len 8192
17918 190/300 Write block 4690 len 24576 data: 6 5 5 6 5 5
17919 191/300 Flush
17920 192/300 Read block 3143 len 20480
17921 193/300 Flush
17922 194/300 Write block 2881 len 12288 data: 4 4 2
17923 195/300 Read block 4015 len 24576
17924 196/300 Read block 688 len 4096
17925 197/300 Read block 2111 len 8192
17926 198/300 Write block 1629 len 32768 data: 4 4 3 4 2 2 3 4
17927 199/300 Read block 2540 len 4096
17928 200/300 Read block 3767 len 28672
17929 201/300 Read block 397 len 32768
17930 202/300 Write block 72 len 4096 data: 6
17931 203/300 Flush
17932 204/300 Read block 2431 len 24576
17933 205/300 Read block 4201 len 16384
17934 206/300 Read block 4135 len 40960
17935 207/300 Read block 4286 len 40960
17936 208/300 Read block 1319 len 36864
17937 209/300 Read block 2186 len 12288
17938 210/300 Read block 72 len 20480
17939 211/300 Read block 114 len 4096
17940 212/300 Flush
17941 213/300 Flush
17942 214/300 Read block 4365 len 4096
17943 215/300 Read block 3388 len 4096
17944 216/300 Read block 3975 len 8192
17945 217/300 Write block 3167 len 12288 data: 4 4 4
17946 218/300 Write block 3600 len 36864 data: 6 6 5 5 6 8 5 5 4
17947 219/300 Write block 1446 len 36864 data: 6 5 5 4 3 4 4 4 5
17948 220/300 Read block 2032 len 24576
17949 221/300 Flush
17950 222/300 Flush
17951 223/300 Write block 1073 len 32768 data: 3 3 3 3 3 2 2 2
17952 224/300 Flush
17953 225/300 Write block 1585 len 20480 data: 6 5 5 5 5
17954 226/300 Write block 2306 len 40960 data: 6 7 8 8 7 5 4 5 6 5
17955 227/300 Write block 4076 len 20480 data: 5 5 4 4 5
17956 228/300 Write block 1450 len 40960 data: 4 5 5 5 6 6 5 5 5 4
17957 229/300 Flush
17958 230/300 Write block 1773 len 28672 data: 3 5 6 5 5 4 5
17959 231/300 Read block 618 len 32768
17960 232/300 Read block 1732 len 40960
17961 233/300 Read block 4404 len 4096
17962 234/300 Read block 3050 len 28672
17963 235/300 Read block 3408 len 20480
17964 236/300 Read block 111 len 8192
17965 237/300 Write block 3883 len 28672 data: 5 4 3 4 4 4 3
17966 238/300 Write block 402 len 24576 data: 3 3 2 4 5 5
17967 239/300 Write block 407 len 8192 data: 6 6
17968 240/300 Read block 1537 len 32768
17969 241/300 Read block 3663 len 28672
17970 242/300 Read block 3683 len 8192
17971 243/300 Read block 115 len 8192
17972 244/300 Read block 3130 len 32768
17973 245/300 Read block 1862 len 12288
17974 246/300 Flush
17975 247/300 Write block 2054 len 16384 data: 5 6 6 4
17976 248/300 Write block 2572 len 16384 data: 6 4 3 3
17977 249/300 Read block 2017 len 28672
17978 250/300 Write block 3315 len 4096 data: 4
17979 251/300 Write block 10 len 20480 data: 2 3 3 3 3
17980 252/300 Read block 402 len 4096
17981 253/300 Write block 3866 len 16384 data: 7 8 7 7
17982 254/300 Read block 659 len 8192
17983 255/300 Read block 2302 len 24576
17984 256/300 Write block 3672 len 20480 data: 5 5 5 5 4
17985 257/300 Read block 2172 len 36864
17986 258/300 Read block 3593 len 36864
17987 259/300 Write block 182 len 4096 data: 2
17988 260/300 Read block 2806 len 12288
17989 261/300 Write block 1550 len 8192 data: 3 3
17990 262/300 Write block 4916 len 28672 data: 3 3 5 6 6 6 4
17991 263/300 Write block 3430 len 32768 data: 4 5 6 6 6 7 6 6
17992 264/300 Read block 4375 len 20480
17993 265/300 Read block 4585 len 16384
17994 266/300 Write block 3846 len 24576 data: 2 2 3 3 4 4
17995 267/300 Read block 2336 len 24576
17996 268/300 Write block 906 len 32768 data: 2 4 4 5 5 4 6 6
17997 269/300 Read block 205 len 20480
17998 270/300 Write block 902 len 32768 data: 5 4 2 2 3 5 5 6
17999 271/300 Read block 2803 len 24576
18000 272/300 Write block 2202 len 12288 data: 5 5 6
18001 273/300 Read block 3974 len 12288
18002 274/300 Write block 2175 len 8192 data: 3 3
18003 275/300 Read block 1549 len 36864
18004 276/300 Read block 3741 len 12288
18005 277/300 Write block 1121 len 8192 data: 8 8
18006 278/300 Write block 577 len 16384 data: 3 3 3 4
18007 279/300 Write block 2656 len 24576 data: 3 3 3 4 8 7
18008 280/300 Read block 2367 len 40960
18009 281/300 Write block 2607 len 40960 data: 3 2 3 3 3 3 2 2 2 2
18010 282/300 Read block 1916 len 28672
18011 283/300 Read block 20 len 16384
18012 284/300 Write block 4156 len 36864 data: 5 4 3 3 2 2 2 2 3
18013 285/300 Write block 4230 len 36864 data: 4 3 3 3 2 2 3 4 4
18014 286/300 Write block 3368 len 32768 data: 4 4 4 4 4 4 3 4
18015 287/300 Write block 4237 len 12288 data: 5 5 5
18016 288/300 Read block 925 len 40960
18017 289/300 Write block 1375 len 4096 data: 3
18018 290/300 Write block 4703 len 28672 data: 5 6 7 7 8 9 6
18019 291/300 Read block 2315 len 40960
18020 292/300 Read block 2711 len 16384
18021 293/300 Write block 4164 len 8192 data: 4 4
18022 294/300 Write block 1879 len 24576 data: 4 3 5 7 5 6
18023 295/300 Read block 4898 len 36864
18024 296/300 Write block 4311 len 40960 data: 5 6 5 4 3 3 3 3 4 4
18025 297/300 Read block 4384 len 4096
18026 298/300 Flush
18027 299/300 Write block 3416 len 20480 data: 4 3 3 5 4
18028 300/300 Read block 2791 len 8192
18029 301/300 Read block 3344 len 28672
18030 Main task has work to do, go find it
18031 got dsc Start(1)
18032 start 1
18033 Jan 03 08:34:41.830 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/1, method: POST, req_id: 85f1c48d-896d-431f-b688-137065db5fd5, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
18034 Replay: started 1, returned:Ok(())
18035 ----------------------------------------------------------------
18036 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18037 States: Active Offline Active
18038 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18039 5831 Acked Write 1 Done Sent Done false
18040 5832 Acked Read 9 Done Sent Done false
18041 5833 Acked Read 9 Done Sent Done false
18042 5834 Acked Write 1 Done Sent Done false
18043 5835 Acked Read 9 Done Sent Done false
18044 5836 Acked Write 5 Done Sent Done false
18045 5837 Acked Write 10 Done Sent Done false
18046 5838 Acked Flush 0 Done Sent Done false
18047 5839 Acked Read 6 Done Sent Done false
18048 5840 Acked Read 8 Done Sent Done false
18049 5841 Acked Flush 0 Done Sent Done false
18050 5842 Acked Write 3 Done Sent Done false
18051 5843 Acked Write 2 Done Sent Done false
18052 5844 Acked Read 4 Done Sent Done false
18053 5845 Acked Read 10 Done Sent Done false
18054 5846 Acked Write 9 Done Sent Done false
18055 5847 Acked Write 7 Done Sent Done false
18056 5848 Acked Write 3 Done Sent Done false
18057 5849 Acked Read 4 Done Sent Done false
18058 5850 Acked Read 7 Done Sent Done false
18059 5851 Acked Read 4 Done Sent Done false
18060 5852 Acked Write 3 Done Sent Done false
18061 5853 Acked Flush 0 Done Sent Done false
18062 5854 Acked Read 4 Done Sent Done false
18063 5855 Acked Write 10 Done Sent Done false
18064 5856 Acked Write 7 Done Sent Done false
18065 5857 Acked Write 10 Done Sent Done false
18066 5858 Acked Write 5 Done Sent Done false
18067 5859 Acked Read 9 Done Sent Done false
18068 5860 Acked Write 8 Done Sent Done false
18069 5861 Acked Flush 0 Done Sent Done false
18070 5862 Acked Write 10 Done Sent Done false
18071 5863 Acked Read 9 Done Sent Done false
18072 5864 Acked Write 7 Done Sent Done false
18073 5865 Acked Read 3 Done Sent Done false
18074 5866 Acked Read 5 Done Sent Done false
18075 5867 Acked Read 7 Done Sent Done false
18076 5868 Acked Write 3 Done Sent Done false
18077 5869 Acked Read 10 Done Sent Done false
18078 5870 Acked Flush 0 Done Sent Done false
18079 5871 Acked Write 4 Done Sent Done false
18080 5872 Acked Read 2 Done Sent Done false
18081 5873 Acked Write 6 Done Sent Done false
18082 5874 Acked Flush 0 Done Sent Done false
18083 5875 Acked Read 5 Done Sent Done false
18084 5876 Acked Write 1 Done Sent Done false
18085 5877 Acked Write 1 Done Sent Done false
18086 5878 Acked Read 9 Done Sent Done false
18087 5879 Acked Flush 0 Done Sent Done false
18088 5880 Acked Read 6 Done Sent Done false
18089 5881 Acked Flush 0 Done Sent Done false
18090 5882 Acked Read 1 Done Sent Done false
18091 5883 Acked Write 3 Done Sent Done false
18092 5884 Acked Write 3 Done Sent Done false
18093 5885 Acked Read 10 Done Sent Done false
18094 5886 Acked Read 8 Done Sent Done false
18095 5887 Acked Write 8 Done Sent Done false
18096 5888 Acked Read 4 Done Sent Done false
18097 5889 Acked Write 9 Done Sent Done false
18098 5890 Acked Read 3 Done Sent Done false
18099 5891 Acked Read 9 Done Sent Done false
18100 5892 Acked Read 2 Done Sent Done false
18101 5893 Acked Write 7 Done Sent Done false
18102 5894 Acked Write 9 Done Sent Done false
18103 5895 Acked Read 4 Done Sent Done false
18104 5896 Acked Read 5 Done Sent Done false
18105 5897 Acked Read 6 Done Sent Done false
18106 5898 Acked Read 10 Done Sent Done false
18107 5899 Acked Read 3 Done Sent Done false
18108 5900 Acked Read 10 Done Sent Done false
18109 5901 Acked Read 7 Done Sent Done false
18110 5902 Acked Write 3 Done Sent Done false
18111 5903 Acked Read 10 Done Sent Done false
18112 5904 Acked Read 5 Done Sent Done false
18113 5905 Acked Flush 0 Done Sent Done false
18114 5906 Acked Read 6 Done Sent Done false
18115 5907 Acked Read 4 Done Sent Done false
18116 5908 Acked Flush 0 Done Sent Done false
18117 5909 Acked Write 1 Done Sent Done false
18118 5910 Acked Write 7 Done Sent Done false
18119 5911 Acked Read 9 Done Sent Done false
18120 5912 Acked Write 8 Done Sent Done false
18121 5913 Acked Flush 0 Done Sent Done false
18122 5914 Acked Write 3 Done Sent Done false
18123 5915 Acked Read 8 Done Sent Done false
18124 5916 Acked Read 1 Done Sent Done false
18125 5917 Acked Write 8 Done Sent Done false
18126 5918 Acked Flush 0 Done Sent Done false
18127 5919 Acked Read 8 Done Sent Done false
18128 5920 Acked Write 3 Done Sent Done false
18129 5921 Acked Read 4 Done Sent Done false
18130 5922 Acked Read 9 Done Sent Done false
18131 5923 Acked Flush 0 Done Sent Done false
18132 5924 Acked Write 9 Done Sent Done false
18133 5925 Acked Read 7 Done Sent Done false
18134 5926 Acked Read 5 Done Sent Done false
18135 5927 Acked Read 2 Done Sent Done false
18136 5928 Acked Read 10 Done Sent Done false
18137 5929 Acked Write 5 Done Sent Done false
18138 5930 Acked Write 6 Done Sent Done false
18139 5931 Acked Read 8 Done Sent Done false
18140 5932 Acked Read 7 Done Sent Done false
18141 5933 Acked Read 7 Done Sent Done false
18142 5934 Acked Write 8 Done Sent Done false
18143 5935 Acked Write 2 Done Sent Done false
18144 5936 Acked Read 4 Done Sent Done false
18145 5937 Acked Write 6 Done Sent Done false
18146 5938 Acked Flush 0 Done Sent Done false
18147 5939 Acked Write 9 Done Sent Done false
18148 5940 Acked Write 7 Done Sent Done false
18149 5941 Acked Read 5 Done Sent Done false
18150 5942 Acked Write 1 Done Sent Done false
18151 5943 Acked Read 5 Done Sent Done false
18152 5944 Acked Write 3 Done Sent Done false
18153 5945 Acked Write 9 Done Sent Done false
18154 5946 Acked Read 10 Done Sent Done false
18155 5947 Acked Write 5 Done Sent Done false
18156 5948 Acked Write 9 Done Sent Done false
18157 5949 Acked Read 5 Done Sent Done false
18158 5950 Acked Read 6 Done Sent Done false
18159 5951 Acked Read 2 Done Sent Done false
18160 5952 Acked Write 9 Done Sent Done false
18161 5953 Acked Read 6 Done Sent Done false
18162 5954 Acked Write 7 Done Sent Done false
18163 5955 Acked Read 4 Done Sent Done false
18164 5956 Acked Read 7 Done Sent Done false
18165 5957 Acked Read 9 Done Sent Done false
18166 5958 Acked Flush 0 Done Sent Done false
18167 5959 Acked Flush 0 Done Sent Done false
18168 5960 Acked Read 7 Done Sent Done false
18169 5961 Acked Write 4 Done Sent Done false
18170 5962 Acked Write 5 Done Sent Done false
18171 5963 Acked Write 9 Done Sent Done false
18172 5964 Acked Write 4 Done Sent Done false
18173 5965 Acked Flush 0 Done Sent Done false
18174 5966 Acked Read 9 Done Sent Done false
18175 5967 Acked Read 8 Done Sent Done false
18176 5968 Acked Read 9 Done Sent Done false
18177 5969 Acked Write 9 Done Sent Done false
18178 5970 Acked Write 2 Done Sent Done false
18179 5971 Acked Flush 0 Done Sent Done false
18180 5972 Acked Write 7 Done Sent Done false
18181 5973 Acked Write 9 Done Sent Done false
18182 5974 Acked Write 6 Done Sent Done false
18183 5975 Acked Write 6 Done Sent Done false
18184 5976 Acked Read 2 Done Sent Done false
18185 5977 Acked Read 8 Done Sent Done false
18186 5978 Acked Write 6 Done Sent Done false
18187 5979 Acked Flush 0 Done Sent Done false
18188 5980 Acked Write 3 Done Sent Done false
18189 5981 Acked Read 2 Done Sent Done false
18190 5982 Acked Read 3 Done Sent Done false
18191 5983 Acked Write 4 Done Sent Done false
18192 5984 Acked Read 9 Done Sent Done false
18193 5985 Acked Write 8 Done Sent Done false
18194 5986 Acked Read 6 Done Sent Done false
18195 5987 Acked Read 10 Done Sent Done false
18196 5988 Acked Read 5 Done Sent Done false
18197 5989 Acked Write 6 Done Sent Done false
18198 5990 Acked Write 8 Done Sent Done false
18199 5991 Acked Read 1 Done Sent Done false
18200 5992 Acked Read 6 Done Sent Done false
18201 5993 Acked Read 6 Done Sent Done false
18202 5994 Acked Write 8 Done Sent Done false
18203 5995 Acked Read 5 Done Sent Done false
18204 5996 Acked Write 10 Done Sent Done false
18205 5997 Acked Read 5 Done Sent Done false
18206 5998 Acked Read 8 Done Sent Done false
18207 5999 Acked Write 3 Done Sent Done false
18208 6000 Acked Write 1 Done Sent Done false
18209 6001 Acked Read 6 Done Sent Done false
18210 6002 Acked Read 10 Done Sent Done false
18211 6003 Acked Read 2 Done Sent Done false
18212 6004 Acked Write 9 Done Sent Done false
18213 6005 Acked Write 5 Done Sent Done false
18214 6006 Acked Write 8 Done Sent Done false
18215 6007 Acked Write 2 Done Sent Done false
18216 6008 Acked Read 9 Done Sent Done false
18217 6009 Acked Read 4 Done Sent Done false
18218 6010 Acked Write 3 Done Sent Done false
18219 6011 Acked Read 3 Done Sent Done false
18220 6012 Acked Write 2 Done Sent Done false
18221 6013 Acked Write 5 Done Sent Done false
18222 6014 Acked Read 4 Done Sent Done false
18223 6015 Acked Write 3 Done Sent Done false
18224 6016 Acked Read 8 Done Sent Done false
18225 6017 Acked Read 4 Done Sent Done false
18226 6018 Acked Read 9 Done Sent Done false
18227 6019 Acked Read 2 Done Sent Done false
18228 6020 Acked Write 6 Done Sent Done false
18229 6021 Acked Flush 0 Done Sent Done false
18230 6022 Acked Read 5 Done Sent Done false
18231 6023 Acked Flush 0 Done Sent Done false
18232 6024 Acked Write 3 Done Sent Done false
18233 6025 Acked Read 6 Done Sent Done false
18234 6026 Acked Read 1 Done Sent Done false
18235 6027 Acked Read 2 Done Sent Done false
18236 6028 Acked Write 8 Done Sent Done false
18237 6029 Acked Read 1 Done Sent Done false
18238 6030 Acked Read 7 Done Sent Done false
18239 6031 Acked Read 8 Done Sent Done false
18240 6032 Acked Write 1 Done Sent Done false
18241 6033 Acked Flush 0 Done Sent Done false
18242 6034 Acked Read 6 Done Sent Done false
18243 6035 Acked Read 4 Done Sent Done false
18244 6036 Acked Read 10 Done Sent Done false
18245 6037 Acked Read 10 Done Sent Done false
18246 6038 Acked Read 9 Done Sent Done false
18247 6039 Acked Read 3 Done Sent Done false
18248 6040 Acked Read 5 Done Sent Done false
18249 6041 Acked Read 1 Done Sent Done false
18250 6042 Acked Flush 0 Done Sent Done false
18251 6043 Acked Flush 0 Done Sent Done false
18252 6044 Acked Read 1 Done Sent Done false
18253 6045 Acked Read 1 Done Sent Done false
18254 6046 Acked Read 2 Done Sent Done false
18255 6047 Acked Write 3 Done Sent Done false
18256 6048 Acked Write 9 Done Sent Done false
18257 6049 Acked Write 9 Done Sent Done false
18258 6050 Acked Read 6 Done Sent Done false
18259 6051 Acked Flush 0 Done Sent Done false
18260 6052 Acked Flush 0 Done Sent Done false
18261 6053 Acked Write 8 Done Sent Done false
18262 6054 Acked Flush 0 Done Sent Done false
18263 6055 Acked Write 5 Done Sent Done false
18264 6056 Acked Write 10 Done Sent Done false
18265 6057 Acked Write 5 Done Sent Done false
18266 6058 Acked Write 10 Done Sent Done false
18267 6059 Acked Flush 0 Done Sent Done false
18268 6060 Acked Write 7 Done Sent Done false
18269 6061 Acked Read 8 Done Sent Done false
18270 6062 Acked Read 10 Done Sent Done false
18271 6063 Acked Read 1 Done Sent Done false
18272 6064 Acked Read 7 Done Sent Done false
18273 6065 Acked Read 5 Done Sent Done false
18274 6066 Acked Read 2 Done Sent Done false
18275 6067 Acked Write 7 Done Sent Done false
18276 6068 Acked Write 6 Done Sent Done false
18277 6069 Acked Write 2 Done Sent Done false
18278 6070 Acked Read 8 Done Sent Done false
18279 6071 Acked Read 7 Done Sent Done false
18280 6072 Acked Read 2 Done Sent Done false
18281 6073 Acked Read 2 Done Sent Done false
18282 6074 Acked Read 8 Done Sent Done false
18283 6075 Acked Read 3 Done Sent Done false
18284 6076 Acked Flush 0 Done Sent Done false
18285 6077 Acked Write 4 Done Sent Done false
18286 6078 Acked Write 4 Done Sent Done false
18287 6079 Acked Read 7 Done Sent Done false
18288 6080 Acked Write 1 Done Sent Done false
18289 6081 Acked Write 5 Done Sent Done false
18290 6082 Acked Read 1 Done Sent Done false
18291 6083 Acked Write 4 Done Sent Done false
18292 6084 Acked Read 2 Done Sent Done false
18293 6085 Acked Read 6 Done Sent Done false
18294 6086 Acked Write 5 Done Sent Done false
18295 6087 Acked Read 9 Done Sent Done false
18296 6088 Acked Read 9 Done Sent Done false
18297 6089 Acked Write 1 Done Sent Done false
18298 6090 Acked Read 3 Done Sent Done false
18299 6091 Acked Write 2 Done Sent Done false
18300 6092 Acked Write 7 Done Sent Done false
18301 6093 Acked Write 8 Done Sent Done false
18302 6094 Acked Read 5 Done Sent Done false
18303 6095 Acked Read 4 Done Sent Done false
18304 6096 Acked Write 6 Done Sent Done false
18305 6097 Acked Read 6 Done Sent Done false
18306 6098 Acked Write 8 Done Sent Done false
18307 6099 Acked Read 5 Done Sent Done false
18308 6100 Acked Write 8 Done Sent Done false
18309 6101 Acked Read 6 Done Sent Done false
18310 6102 Acked Write 3 Done Sent Done false
18311 6103 Acked Read 3 Done Sent Done false
18312 6104 Acked Write 2 Done Sent Done false
18313 6105 Acked Read 9 Done Sent Done false
18314 6106 Acked Read 3 Done Sent Done false
18315 6107 Acked Write 2 Done Sent Done false
18316 6108 Acked Write 4 Done Sent Done false
18317 6109 Acked Write 6 Done Sent Done false
18318 6110 Acked Read 10 Done Sent Done false
18319 6111 Acked Write 10 Done Sent Done false
18320 6112 Acked Read 7 Done Sent Done false
18321 6113 Acked Read 4 Done Sent Done false
18322 6114 Acked Write 9 Done Sent Done false
18323 6115 Acked Write 9 Done Sent Done false
18324 6116 Acked Write 8 Done Sent Done false
18325 6117 Acked Write 3 Done Sent Done false
18326 6118 Acked Read 10 Done Sent Done false
18327 6119 Acked Write 1 Done Sent Done false
18328 6120 Acked Write 7 Done Sent Done false
18329 6121 Acked Read 10 Done Sent Done false
18330 6122 Acked Read 4 Done Sent Done false
18331 6123 Acked Write 2 Done Sent Done false
18332 6124 Acked Write 6 Done Sent Done false
18333 6125 Acked Read 9 Done Sent Done false
18334 6126 Acked Write 10 Done Sent Done false
18335 6127 Acked Read 1 Done Sent Done false
18336 6128 Acked Flush 0 Done Sent Done false
18337 6129 Acked Write 5 Done Sent Done false
18338 6130 Acked Read 2 Done Sent Done false
18339 6131 Acked Read 7 Done Sent Done false
18340 STATES DS:0 DS:1 DS:2 TOTAL
18341 Sent 0 301 0 301
18342 Done 301 0 301 602
18343 Skipped 0 0 0 0
18344 Error 0 0 0 0
18345 Last Flush: 6128 5830 6128
18346 Downstairs last five completed: 5830 5829 5828 5827 5826
18347 Upstairs last five completed: 6131 6130 6129 6128 6127
18348 CLIENT: Up:0 ds:301 act:2
18349 ----------------------------------------------------------------
18350 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18351 States: Active Offline Active
18352 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18353 5831 Acked Write 1 Done Sent Done false
18354 5832 Acked Read 9 Done Sent Done false
18355 5833 Acked Read 9 Done Sent Done false
18356 5834 Acked Write 1 Done Sent Done false
18357 5835 Acked Read 9 Done Sent Done false
18358 5836 Acked Write 5 Done Sent Done false
18359 5837 Acked Write 10 Done Sent Done false
18360 5838 Acked Flush 0 Done Sent Done false
18361 5839 Acked Read 6 Done Sent Done false
18362 5840 Acked Read 8 Done Sent Done false
18363 5841 Acked Flush 0 Done Sent Done false
18364 5842 Acked Write 3 Done Sent Done false
18365 5843 Acked Write 2 Done Sent Done false
18366 5844 Acked Read 4 Done Sent Done false
18367 5845 Acked Read 10 Done Sent Done false
18368 5846 Acked Write 9 Done Sent Done false
18369 5847 Acked Write 7 Done Sent Done false
18370 5848 Acked Write 3 Done Sent Done false
18371 5849 Acked Read 4 Done Sent Done false
18372 5850 Acked Read 7 Done Sent Done false
18373 5851 Acked Read 4 Done Sent Done false
18374 5852 Acked Write 3 Done Sent Done false
18375 5853 Acked Flush 0 Done Sent Done false
18376 5854 Acked Read 4 Done Sent Done false
18377 5855 Acked Write 10 Done Sent Done false
18378 5856 Acked Write 7 Done Sent Done false
18379 5857 Acked Write 10 Done Sent Done false
18380 5858 Acked Write 5 Done Sent Done false
18381 5859 Acked Read 9 Done Sent Done false
18382 5860 Acked Write 8 Done Sent Done false
18383 5861 Acked Flush 0 Done Sent Done false
18384 5862 Acked Write 10 Done Sent Done false
18385 5863 Acked Read 9 Done Sent Done false
18386 5864 Acked Write 7 Done Sent Done false
18387 5865 Acked Read 3 Done Sent Done false
18388 5866 Acked Read 5 Done Sent Done false
18389 5867 Acked Read 7 Done Sent Done false
18390 5868 Acked Write 3 Done Sent Done false
18391 5869 Acked Read 10 Done Sent Done false
18392 5870 Acked Flush 0 Done Sent Done false
18393 5871 Acked Write 4 Done Sent Done false
18394 5872 Acked Read 2 Done Sent Done false
18395 5873 Acked Write 6 Done Sent Done false
18396 5874 Acked Flush 0 Done Sent Done false
18397 5875 Acked Read 5 Done Sent Done false
18398 5876 Acked Write 1 Done Sent Done false
18399 5877 Acked Write 1 Done Sent Done false
18400 5878 Acked Read 9 Done Sent Done false
18401 5879 Acked Flush 0 Done Sent Done false
18402 5880 Acked Read 6 Done Sent Done false
18403 5881 Acked Flush 0 Done Sent Done false
18404 5882 Acked Read 1 Done Sent Done false
18405 5883 Acked Write 3 Done Sent Done false
18406 5884 Acked Write 3 Done Sent Done false
18407 5885 Acked Read 10 Done Sent Done false
18408 5886 Acked Read 8 Done Sent Done false
18409 5887 Acked Write 8 Done Sent Done false
18410 5888 Acked Read 4 Done Sent Done false
18411 5889 Acked Write 9 Done Sent Done false
18412 5890 Acked Read 3 Done Sent Done false
18413 5891 Acked Read 9 Done Sent Done false
18414 5892 Acked Read 2 Done Sent Done false
18415 5893 Acked Write 7 Done Sent Done false
18416 5894 Acked Write 9 Done Sent Done false
18417 5895 Acked Read 4 Done Sent Done false
18418 5896 Acked Read 5 Done Sent Done false
18419 5897 Acked Read 6 Done Sent Done false
18420 5898 Acked Read 10 Done Sent Done false
18421 5899 Acked Read 3 Done Sent Done false
18422 5900 Acked Read 10 Done Sent Done false
18423 5901 Acked Read 7 Done Sent Done false
18424 5902 Acked Write 3 Done Sent Done false
18425 5903 Acked Read 10 Done Sent Done false
18426 5904 Acked Read 5 Done Sent Done false
18427 5905 Acked Flush 0 Done Sent Done false
18428 5906 Acked Read 6 Done Sent Done false
18429 5907 Acked Read 4 Done Sent Done false
18430 5908 Acked Flush 0 Done Sent Done false
18431 5909 Acked Write 1 Done Sent Done false
18432 5910 Acked Write 7 Done Sent Done false
18433 5911 Acked Read 9 Done Sent Done false
18434 5912 Acked Write 8 Done Sent Done false
18435 5913 Acked Flush 0 Done Sent Done false
18436 5914 Acked Write 3 Done Sent Done false
18437 5915 Acked Read 8 Done Sent Done false
18438 5916 Acked Read 1 Done Sent Done false
18439 5917 Acked Write 8 Done Sent Done false
18440 5918 Acked Flush 0 Done Sent Done false
18441 5919 Acked Read 8 Done Sent Done false
18442 5920 Acked Write 3 Done Sent Done false
18443 5921 Acked Read 4 Done Sent Done false
18444 5922 Acked Read 9 Done Sent Done false
18445 5923 Acked Flush 0 Done Sent Done false
18446 5924 Acked Write 9 Done Sent Done false
18447 5925 Acked Read 7 Done Sent Done false
18448 5926 Acked Read 5 Done Sent Done false
18449 5927 Acked Read 2 Done Sent Done false
18450 5928 Acked Read 10 Done Sent Done false
18451 5929 Acked Write 5 Done Sent Done false
18452 5930 Acked Write 6 Done Sent Done false
18453 5931 Acked Read 8 Done Sent Done false
18454 5932 Acked Read 7 Done Sent Done false
18455 5933 Acked Read 7 Done Sent Done false
18456 5934 Acked Write 8 Done Sent Done false
18457 5935 Acked Write 2 Done Sent Done false
18458 5936 Acked Read 4 Done Sent Done false
18459 5937 Acked Write 6 Done Sent Done false
18460 5938 Acked Flush 0 Done Sent Done false
18461 5939 Acked Write 9 Done Sent Done false
18462 5940 Acked Write 7 Done Sent Done false
18463 5941 Acked Read 5 Done Sent Done false
18464 5942 Acked Write 1 Done Sent Done false
18465 5943 Acked Read 5 Done Sent Done false
18466 5944 Acked Write 3 Done Sent Done false
18467 5945 Acked Write 9 Done Sent Done false
18468 5946 Acked Read 10 Done Sent Done false
18469 5947 Acked Write 5 Done Sent Done false
18470 5948 Acked Write 9 Done Sent Done false
18471 5949 Acked Read 5 Done Sent Done false
18472 5950 Acked Read 6 Done Sent Done false
18473 5951 Acked Read 2 Done Sent Done false
18474 5952 Acked Write 9 Done Sent Done false
18475 5953 Acked Read 6 Done Sent Done false
18476 5954 Acked Write 7 Done Sent Done false
18477 5955 Acked Read 4 Done Sent Done false
18478 5956 Acked Read 7 Done Sent Done false
18479 5957 Acked Read 9 Done Sent Done false
18480 5958 Acked Flush 0 Done Sent Done false
18481 5959 Acked Flush 0 Done Sent Done false
18482 5960 Acked Read 7 Done Sent Done false
18483 5961 Acked Write 4 Done Sent Done false
18484 5962 Acked Write 5 Done Sent Done false
18485 5963 Acked Write 9 Done Sent Done false
18486 5964 Acked Write 4 Done Sent Done false
18487 5965 Acked Flush 0 Done Sent Done false
18488 5966 Acked Read 9 Done Sent Done false
18489 5967 Acked Read 8 Done Sent Done false
18490 5968 Acked Read 9 Done Sent Done false
18491 5969 Acked Write 9 Done Sent Done false
18492 5970 Acked Write 2 Done Sent Done false
18493 5971 Acked Flush 0 Done Sent Done false
18494 5972 Acked Write 7 Done Sent Done false
18495 5973 Acked Write 9 Done Sent Done false
18496 5974 Acked Write 6 Done Sent Done false
18497 5975 Acked Write 6 Done Sent Done false
18498 5976 Acked Read 2 Done Sent Done false
18499 5977 Acked Read 8 Done Sent Done false
18500 5978 Acked Write 6 Done Sent Done false
18501 5979 Acked Flush 0 Done Sent Done false
18502 5980 Acked Write 3 Done Sent Done false
18503 5981 Acked Read 2 Done Sent Done false
18504 5982 Acked Read 3 Done Sent Done false
18505 5983 Acked Write 4 Done Sent Done false
18506 5984 Acked Read 9 Done Sent Done false
18507 5985 Acked Write 8 Done Sent Done false
18508 5986 Acked Read 6 Done Sent Done false
18509 5987 Acked Read 10 Done Sent Done false
18510 5988 Acked Read 5 Done Sent Done false
18511 5989 Acked Write 6 Done Sent Done false
18512 5990 Acked Write 8 Done Sent Done false
18513 5991 Acked Read 1 Done Sent Done false
18514 5992 Acked Read 6 Done Sent Done false
18515 5993 Acked Read 6 Done Sent Done false
18516 5994 Acked Write 8 Done Sent Done false
18517 5995 Acked Read 5 Done Sent Done false
18518 5996 Acked Write 10 Done Sent Done false
18519 5997 Acked Read 5 Done Sent Done false
18520 5998 Acked Read 8 Done Sent Done false
18521 5999 Acked Write 3 Done Sent Done false
18522 6000 Acked Write 1 Done Sent Done false
18523 6001 Acked Read 6 Done Sent Done false
18524 6002 Acked Read 10 Done Sent Done false
18525 6003 Acked Read 2 Done Sent Done false
18526 6004 Acked Write 9 Done Sent Done false
18527 6005 Acked Write 5 Done Sent Done false
18528 6006 Acked Write 8 Done Sent Done false
18529 6007 Acked Write 2 Done Sent Done false
18530 6008 Acked Read 9 Done Sent Done false
18531 6009 Acked Read 4 Done Sent Done false
18532 6010 Acked Write 3 Done Sent Done false
18533 6011 Acked Read 3 Done Sent Done false
18534 6012 Acked Write 2 Done Sent Done false
18535 6013 Acked Write 5 Done Sent Done false
18536 6014 Acked Read 4 Done Sent Done false
18537 6015 Acked Write 3 Done Sent Done false
18538 6016 Acked Read 8 Done Sent Done false
18539 6017 Acked Read 4 Done Sent Done false
18540 6018 Acked Read 9 Done Sent Done false
18541 6019 Acked Read 2 Done Sent Done false
18542 6020 Acked Write 6 Done Sent Done false
18543 6021 Acked Flush 0 Done Sent Done false
18544 6022 Acked Read 5 Done Sent Done false
18545 6023 Acked Flush 0 Done Sent Done false
18546 6024 Acked Write 3 Done Sent Done false
18547 6025 Acked Read 6 Done Sent Done false
18548 6026 Acked Read 1 Done Sent Done false
18549 6027 Acked Read 2 Done Sent Done false
18550 6028 Acked Write 8 Done Sent Done false
18551 6029 Acked Read 1 Done Sent Done false
18552 6030 Acked Read 7 Done Sent Done false
18553 6031 Acked Read 8 Done Sent Done false
18554 6032 Acked Write 1 Done Sent Done false
18555 6033 Acked Flush 0 Done Sent Done false
18556 6034 Acked Read 6 Done Sent Done false
18557 6035 Acked Read 4 Done Sent Done false
18558 6036 Acked Read 10 Done Sent Done false
18559 6037 Acked Read 10 Done Sent Done false
18560 6038 Acked Read 9 Done Sent Done false
18561 6039 Acked Read 3 Done Sent Done false
18562 6040 Acked Read 5 Done Sent Done false
18563 6041 Acked Read 1 Done Sent Done false
18564 6042 Acked Flush 0 Done Sent Done false
18565 6043 Acked Flush 0 Done Sent Done false
18566 6044 Acked Read 1 Done Sent Done false
18567 6045 Acked Read 1 Done Sent Done false
18568 6046 Acked Read 2 Done Sent Done false
18569 6047 Acked Write 3 Done Sent Done false
18570 6048 Acked Write 9 Done Sent Done false
18571 6049 Acked Write 9 Done Sent Done false
18572 6050 Acked Read 6 Done Sent Done false
18573 6051 Acked Flush 0 Done Sent Done false
18574 6052 Acked Flush 0 Done Sent Done false
18575 6053 Acked Write 8 Done Sent Done false
18576 6054 Acked Flush 0 Done Sent Done false
18577 6055 Acked Write 5 Done Sent Done false
18578 6056 Acked Write 10 Done Sent Done false
18579 6057 Acked Write 5 Done Sent Done false
18580 6058 Acked Write 10 Done Sent Done false
18581 6059 Acked Flush 0 Done Sent Done false
18582 6060 Acked Write 7 Done Sent Done false
18583 6061 Acked Read 8 Done Sent Done false
18584 6062 Acked Read 10 Done Sent Done false
18585 6063 Acked Read 1 Done Sent Done false
18586 6064 Acked Read 7 Done Sent Done false
18587 6065 Acked Read 5 Done Sent Done false
18588 6066 Acked Read 2 Done Sent Done false
18589 6067 Acked Write 7 Done Sent Done false
18590 6068 Acked Write 6 Done Sent Done false
18591 6069 Acked Write 2 Done Sent Done false
18592 6070 Acked Read 8 Done Sent Done false
18593 6071 Acked Read 7 Done Sent Done false
18594 6072 Acked Read 2 Done Sent Done false
18595 6073 Acked Read 2 Done Sent Done false
18596 6074 Acked Read 8 Done Sent Done false
18597 6075 Acked Read 3 Done Sent Done false
18598 6076 Acked Flush 0 Done Sent Done false
18599 6077 Acked Write 4 Done Sent Done false
18600 6078 Acked Write 4 Done Sent Done false
18601 6079 Acked Read 7 Done Sent Done false
18602 6080 Acked Write 1 Done Sent Done false
18603 6081 Acked Write 5 Done Sent Done false
18604 6082 Acked Read 1 Done Sent Done false
18605 6083 Acked Write 4 Done Sent Done false
18606 6084 Acked Read 2 Done Sent Done false
18607 6085 Acked Read 6 Done Sent Done false
18608 6086 Acked Write 5 Done Sent Done false
18609 6087 Acked Read 9 Done Sent Done false
18610 6088 Acked Read 9 Done Sent Done false
18611 6089 Acked Write 1 Done Sent Done false
18612 6090 Acked Read 3 Done Sent Done false
18613 6091 Acked Write 2 Done Sent Done false
18614 6092 Acked Write 7 Done Sent Done false
18615 6093 Acked Write 8 Done Sent Done false
18616 6094 Acked Read 5 Done Sent Done false
18617 6095 Acked Read 4 Done Sent Done false
18618 6096 Acked Write 6 Done Sent Done false
18619 6097 Acked Read 6 Done Sent Done false
18620 6098 Acked Write 8 Done Sent Done false
18621 6099 Acked Read 5 Done Sent Done false
18622 6100 Acked Write 8 Done Sent Done false
18623 6101 Acked Read 6 Done Sent Done false
18624 6102 Acked Write 3 Done Sent Done false
18625 6103 Acked Read 3 Done Sent Done false
18626 6104 Acked Write 2 Done Sent Done false
18627 6105 Acked Read 9 Done Sent Done false
18628 6106 Acked Read 3 Done Sent Done false
18629 6107 Acked Write 2 Done Sent Done false
18630 6108 Acked Write 4 Done Sent Done false
18631 6109 Acked Write 6 Done Sent Done false
18632 6110 Acked Read 10 Done Sent Done false
18633 6111 Acked Write 10 Done Sent Done false
18634 6112 Acked Read 7 Done Sent Done false
18635 6113 Acked Read 4 Done Sent Done false
18636 6114 Acked Write 9 Done Sent Done false
18637 6115 Acked Write 9 Done Sent Done false
18638 6116 Acked Write 8 Done Sent Done false
18639 6117 Acked Write 3 Done Sent Done false
18640 6118 Acked Read 10 Done Sent Done false
18641 6119 Acked Write 1 Done Sent Done false
18642 6120 Acked Write 7 Done Sent Done false
18643 6121 Acked Read 10 Done Sent Done false
18644 6122 Acked Read 4 Done Sent Done false
18645 6123 Acked Write 2 Done Sent Done false
18646 6124 Acked Write 6 Done Sent Done false
18647 6125 Acked Read 9 Done Sent Done false
18648 6126 Acked Write 10 Done Sent Done false
18649 6127 Acked Read 1 Done Sent Done false
18650 6128 Acked Flush 0 Done Sent Done false
18651 6129 Acked Write 5 Done Sent Done false
18652 6130 Acked Read 2 Done Sent Done false
18653 6131 Acked Read 7 Done Sent Done false
18654 6132 Acked Flush 0 Done Sent Done false
18655 STATES DS:0 DS:1 DS:2 TOTAL
18656 Sent 0 302 0 302
18657 Done 302 0 302 604
18658 Skipped 0 0 0 0
18659 Error 0 0 0 0
18660 Last Flush: 6132 5830 6132
18661 Downstairs last five completed: 5830 5829 5828 5827 5826
18662 Upstairs last five completed: 6132 6131 6130 6129 6128
18663 CLIENT: Up:0 ds:302 act:2
186642025-01-03T08:34:47.577ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
186652025-01-03T08:34:47.577ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
186662025-01-03T08:34:47.577ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
186672025-01-03T08:34:47.577ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
186682025-01-03T08:34:47.577ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
18669 [8820] got start action
18670 [8820] I am going to restart
18671 Starting downstairs at port 8820
18672 Make output file at "/tmp/dsc/downstairs-8820.txt"
18673 [8820][1] reports Starting
18674 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1222)
18675 [8820][1] reports Running
18676 ----------------------------------------------------------------
18677 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18678 States: Active Offline Active
18679 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18680 5831 Acked Write 1 Done Sent Done false
18681 5832 Acked Read 9 Done Sent Done false
18682 5833 Acked Read 9 Done Sent Done false
18683 5834 Acked Write 1 Done Sent Done false
18684 5835 Acked Read 9 Done Sent Done false
18685 5836 Acked Write 5 Done Sent Done false
18686 5837 Acked Write 10 Done Sent Done false
18687 5838 Acked Flush 0 Done Sent Done false
18688 5839 Acked Read 6 Done Sent Done false
18689 5840 Acked Read 8 Done Sent Done false
18690 5841 Acked Flush 0 Done Sent Done false
18691 5842 Acked Write 3 Done Sent Done false
18692 5843 Acked Write 2 Done Sent Done false
18693 5844 Acked Read 4 Done Sent Done false
18694 5845 Acked Read 10 Done Sent Done false
18695 5846 Acked Write 9 Done Sent Done false
18696 5847 Acked Write 7 Done Sent Done false
18697 5848 Acked Write 3 Done Sent Done false
18698 5849 Acked Read 4 Done Sent Done false
18699 5850 Acked Read 7 Done Sent Done false
18700 5851 Acked Read 4 Done Sent Done false
18701 5852 Acked Write 3 Done Sent Done false
18702 5853 Acked Flush 0 Done Sent Done false
18703 5854 Acked Read 4 Done Sent Done false
18704 5855 Acked Write 10 Done Sent Done false
18705 5856 Acked Write 7 Done Sent Done false
18706 5857 Acked Write 10 Done Sent Done false
18707 5858 Acked Write 5 Done Sent Done false
18708 5859 Acked Read 9 Done Sent Done false
18709 5860 Acked Write 8 Done Sent Done false
18710 5861 Acked Flush 0 Done Sent Done false
18711 5862 Acked Write 10 Done Sent Done false
18712 5863 Acked Read 9 Done Sent Done false
18713 5864 Acked Write 7 Done Sent Done false
18714 5865 Acked Read 3 Done Sent Done false
18715 5866 Acked Read 5 Done Sent Done false
18716 5867 Acked Read 7 Done Sent Done false
18717 5868 Acked Write 3 Done Sent Done false
18718 5869 Acked Read 10 Done Sent Done false
18719 5870 Acked Flush 0 Done Sent Done false
18720 5871 Acked Write 4 Done Sent Done false
18721 5872 Acked Read 2 Done Sent Done false
18722 5873 Acked Write 6 Done Sent Done false
18723 5874 Acked Flush 0 Done Sent Done false
18724 5875 Acked Read 5 Done Sent Done false
18725 5876 Acked Write 1 Done Sent Done false
18726 5877 Acked Write 1 Done Sent Done false
18727 5878 Acked Read 9 Done Sent Done false
18728 5879 Acked Flush 0 Done Sent Done false
18729 5880 Acked Read 6 Done Sent Done false
18730 5881 Acked Flush 0 Done Sent Done false
18731 5882 Acked Read 1 Done Sent Done false
18732 5883 Acked Write 3 Done Sent Done false
18733 5884 Acked Write 3 Done Sent Done false
18734 5885 Acked Read 10 Done Sent Done false
18735 5886 Acked Read 8 Done Sent Done false
18736 5887 Acked Write 8 Done Sent Done false
18737 5888 Acked Read 4 Done Sent Done false
18738 5889 Acked Write 9 Done Sent Done false
18739 5890 Acked Read 3 Done Sent Done false
18740 5891 Acked Read 9 Done Sent Done false
18741 5892 Acked Read 2 Done Sent Done false
18742 5893 Acked Write 7 Done Sent Done false
18743 5894 Acked Write 9 Done Sent Done false
18744 5895 Acked Read 4 Done Sent Done false
18745 5896 Acked Read 5 Done Sent Done false
18746 5897 Acked Read 6 Done Sent Done false
18747 5898 Acked Read 10 Done Sent Done false
18748 5899 Acked Read 3 Done Sent Done false
18749 5900 Acked Read 10 Done Sent Done false
18750 5901 Acked Read 7 Done Sent Done false
18751 5902 Acked Write 3 Done Sent Done false
18752 5903 Acked Read 10 Done Sent Done false
18753 5904 Acked Read 5 Done Sent Done false
18754 5905 Acked Flush 0 Done Sent Done false
18755 5906 Acked Read 6 Done Sent Done false
18756 5907 Acked Read 4 Done Sent Done false
18757 5908 Acked Flush 0 Done Sent Done false
18758 5909 Acked Write 1 Done Sent Done false
18759 5910 Acked Write 7 Done Sent Done false
18760 5911 Acked Read 9 Done Sent Done false
18761 5912 Acked Write 8 Done Sent Done false
18762 5913 Acked Flush 0 Done Sent Done false
18763 5914 Acked Write 3 Done Sent Done false
18764 5915 Acked Read 8 Done Sent Done false
18765 5916 Acked Read 1 Done Sent Done false
18766 5917 Acked Write 8 Done Sent Done false
18767 5918 Acked Flush 0 Done Sent Done false
18768 5919 Acked Read 8 Done Sent Done false
18769 5920 Acked Write 3 Done Sent Done false
18770 5921 Acked Read 4 Done Sent Done false
18771 5922 Acked Read 9 Done Sent Done false
18772 5923 Acked Flush 0 Done Sent Done false
18773 5924 Acked Write 9 Done Sent Done false
18774 5925 Acked Read 7 Done Sent Done false
18775 5926 Acked Read 5 Done Sent Done false
18776 5927 Acked Read 2 Done Sent Done false
18777 5928 Acked Read 10 Done Sent Done false
18778 5929 Acked Write 5 Done Sent Done false
18779 5930 Acked Write 6 Done Sent Done false
18780 5931 Acked Read 8 Done Sent Done false
18781 5932 Acked Read 7 Done Sent Done false
18782 5933 Acked Read 7 Done Sent Done false
18783 5934 Acked Write 8 Done Sent Done false
18784 5935 Acked Write 2 Done Sent Done false
18785 5936 Acked Read 4 Done Sent Done false
18786 5937 Acked Write 6 Done Sent Done false
18787 5938 Acked Flush 0 Done Sent Done false
18788 5939 Acked Write 9 Done Sent Done false
18789 5940 Acked Write 7 Done Sent Done false
18790 5941 Acked Read 5 Done Sent Done false
18791 5942 Acked Write 1 Done Sent Done false
18792 5943 Acked Read 5 Done Sent Done false
18793 5944 Acked Write 3 Done Sent Done false
18794 5945 Acked Write 9 Done Sent Done false
18795 5946 Acked Read 10 Done Sent Done false
18796 5947 Acked Write 5 Done Sent Done false
18797 5948 Acked Write 9 Done Sent Done false
18798 5949 Acked Read 5 Done Sent Done false
18799 5950 Acked Read 6 Done Sent Done false
18800 5951 Acked Read 2 Done Sent Done false
18801 5952 Acked Write 9 Done Sent Done false
18802 5953 Acked Read 6 Done Sent Done false
18803 5954 Acked Write 7 Done Sent Done false
18804 5955 Acked Read 4 Done Sent Done false
18805 5956 Acked Read 7 Done Sent Done false
18806 5957 Acked Read 9 Done Sent Done false
18807 5958 Acked Flush 0 Done Sent Done false
18808 5959 Acked Flush 0 Done Sent Done false
18809 5960 Acked Read 7 Done Sent Done false
18810 5961 Acked Write 4 Done Sent Done false
18811 5962 Acked Write 5 Done Sent Done false
18812 5963 Acked Write 9 Done Sent Done false
18813 5964 Acked Write 4 Done Sent Done false
18814 5965 Acked Flush 0 Done Sent Done false
18815 5966 Acked Read 9 Done Sent Done false
18816 5967 Acked Read 8 Done Sent Done false
18817 5968 Acked Read 9 Done Sent Done false
18818 5969 Acked Write 9 Done Sent Done false
18819 5970 Acked Write 2 Done Sent Done false
18820 5971 Acked Flush 0 Done Sent Done false
18821 5972 Acked Write 7 Done Sent Done false
18822 5973 Acked Write 9 Done Sent Done false
18823 5974 Acked Write 6 Done Sent Done false
18824 5975 Acked Write 6 Done Sent Done false
18825 5976 Acked Read 2 Done Sent Done false
18826 5977 Acked Read 8 Done Sent Done false
18827 5978 Acked Write 6 Done Sent Done false
18828 5979 Acked Flush 0 Done Sent Done false
18829 5980 Acked Write 3 Done Sent Done false
18830 5981 Acked Read 2 Done Sent Done false
18831 5982 Acked Read 3 Done Sent Done false
18832 5983 Acked Write 4 Done Sent Done false
18833 5984 Acked Read 9 Done Sent Done false
18834 5985 Acked Write 8 Done Sent Done false
18835 5986 Acked Read 6 Done Sent Done false
18836 5987 Acked Read 10 Done Sent Done false
18837 5988 Acked Read 5 Done Sent Done false
18838 5989 Acked Write 6 Done Sent Done false
18839 5990 Acked Write 8 Done Sent Done false
18840 5991 Acked Read 1 Done Sent Done false
18841 5992 Acked Read 6 Done Sent Done false
18842 5993 Acked Read 6 Done Sent Done false
18843 5994 Acked Write 8 Done Sent Done false
18844 5995 Acked Read 5 Done Sent Done false
18845 5996 Acked Write 10 Done Sent Done false
18846 5997 Acked Read 5 Done Sent Done false
18847 5998 Acked Read 8 Done Sent Done false
18848 5999 Acked Write 3 Done Sent Done false
18849 6000 Acked Write 1 Done Sent Done false
18850 6001 Acked Read 6 Done Sent Done false
18851 6002 Acked Read 10 Done Sent Done false
18852 6003 Acked Read 2 Done Sent Done false
18853 6004 Acked Write 9 Done Sent Done false
18854 6005 Acked Write 5 Done Sent Done false
18855 6006 Acked Write 8 Done Sent Done false
18856 6007 Acked Write 2 Done Sent Done false
18857 6008 Acked Read 9 Done Sent Done false
18858 6009 Acked Read 4 Done Sent Done false
18859 6010 Acked Write 3 Done Sent Done false
18860 6011 Acked Read 3 Done Sent Done false
18861 6012 Acked Write 2 Done Sent Done false
18862 6013 Acked Write 5 Done Sent Done false
18863 6014 Acked Read 4 Done Sent Done false
18864 6015 Acked Write 3 Done Sent Done false
18865 6016 Acked Read 8 Done Sent Done false
18866 6017 Acked Read 4 Done Sent Done false
18867 6018 Acked Read 9 Done Sent Done false
18868 6019 Acked Read 2 Done Sent Done false
18869 6020 Acked Write 6 Done Sent Done false
18870 6021 Acked Flush 0 Done Sent Done false
18871 6022 Acked Read 5 Done Sent Done false
18872 6023 Acked Flush 0 Done Sent Done false
18873 6024 Acked Write 3 Done Sent Done false
18874 6025 Acked Read 6 Done Sent Done false
18875 6026 Acked Read 1 Done Sent Done false
18876 6027 Acked Read 2 Done Sent Done false
18877 6028 Acked Write 8 Done Sent Done false
18878 6029 Acked Read 1 Done Sent Done false
18879 6030 Acked Read 7 Done Sent Done false
18880 6031 Acked Read 8 Done Sent Done false
18881 6032 Acked Write 1 Done Sent Done false
18882 6033 Acked Flush 0 Done Sent Done false
18883 6034 Acked Read 6 Done Sent Done false
18884 6035 Acked Read 4 Done Sent Done false
18885 6036 Acked Read 10 Done Sent Done false
18886 6037 Acked Read 10 Done Sent Done false
18887 6038 Acked Read 9 Done Sent Done false
18888 6039 Acked Read 3 Done Sent Done false
18889 6040 Acked Read 5 Done Sent Done false
18890 6041 Acked Read 1 Done Sent Done false
18891 6042 Acked Flush 0 Done Sent Done false
18892 6043 Acked Flush 0 Done Sent Done false
18893 6044 Acked Read 1 Done Sent Done false
18894 6045 Acked Read 1 Done Sent Done false
18895 6046 Acked Read 2 Done Sent Done false
18896 6047 Acked Write 3 Done Sent Done false
18897 6048 Acked Write 9 Done Sent Done false
18898 6049 Acked Write 9 Done Sent Done false
18899 6050 Acked Read 6 Done Sent Done false
18900 6051 Acked Flush 0 Done Sent Done false
18901 6052 Acked Flush 0 Done Sent Done false
18902 6053 Acked Write 8 Done Sent Done false
18903 6054 Acked Flush 0 Done Sent Done false
18904 6055 Acked Write 5 Done Sent Done false
18905 6056 Acked Write 10 Done Sent Done false
18906 6057 Acked Write 5 Done Sent Done false
18907 6058 Acked Write 10 Done Sent Done false
18908 6059 Acked Flush 0 Done Sent Done false
18909 6060 Acked Write 7 Done Sent Done false
18910 6061 Acked Read 8 Done Sent Done false
18911 6062 Acked Read 10 Done Sent Done false
18912 6063 Acked Read 1 Done Sent Done false
18913 6064 Acked Read 7 Done Sent Done false
18914 6065 Acked Read 5 Done Sent Done false
18915 6066 Acked Read 2 Done Sent Done false
18916 6067 Acked Write 7 Done Sent Done false
18917 6068 Acked Write 6 Done Sent Done false
18918 6069 Acked Write 2 Done Sent Done false
18919 6070 Acked Read 8 Done Sent Done false
18920 6071 Acked Read 7 Done Sent Done false
18921 6072 Acked Read 2 Done Sent Done false
18922 6073 Acked Read 2 Done Sent Done false
18923 6074 Acked Read 8 Done Sent Done false
18924 6075 Acked Read 3 Done Sent Done false
18925 6076 Acked Flush 0 Done Sent Done false
18926 6077 Acked Write 4 Done Sent Done false
18927 6078 Acked Write 4 Done Sent Done false
18928 6079 Acked Read 7 Done Sent Done false
18929 6080 Acked Write 1 Done Sent Done false
18930 6081 Acked Write 5 Done Sent Done false
18931 6082 Acked Read 1 Done Sent Done false
18932 6083 Acked Write 4 Done Sent Done false
18933 6084 Acked Read 2 Done Sent Done false
18934 6085 Acked Read 6 Done Sent Done false
18935 6086 Acked Write 5 Done Sent Done false
18936 6087 Acked Read 9 Done Sent Done false
18937 6088 Acked Read 9 Done Sent Done false
18938 6089 Acked Write 1 Done Sent Done false
18939 6090 Acked Read 3 Done Sent Done false
18940 6091 Acked Write 2 Done Sent Done false
18941 6092 Acked Write 7 Done Sent Done false
18942 6093 Acked Write 8 Done Sent Done false
18943 6094 Acked Read 5 Done Sent Done false
18944 6095 Acked Read 4 Done Sent Done false
18945 6096 Acked Write 6 Done Sent Done false
18946 6097 Acked Read 6 Done Sent Done false
18947 6098 Acked Write 8 Done Sent Done false
18948 6099 Acked Read 5 Done Sent Done false
18949 6100 Acked Write 8 Done Sent Done false
18950 6101 Acked Read 6 Done Sent Done false
18951 6102 Acked Write 3 Done Sent Done false
18952 6103 Acked Read 3 Done Sent Done false
18953 6104 Acked Write 2 Done Sent Done false
18954 6105 Acked Read 9 Done Sent Done false
18955 6106 Acked Read 3 Done Sent Done false
18956 6107 Acked Write 2 Done Sent Done false
18957 6108 Acked Write 4 Done Sent Done false
18958 6109 Acked Write 6 Done Sent Done false
18959 6110 Acked Read 10 Done Sent Done false
18960 6111 Acked Write 10 Done Sent Done false
18961 6112 Acked Read 7 Done Sent Done false
18962 6113 Acked Read 4 Done Sent Done false
18963 6114 Acked Write 9 Done Sent Done false
18964 6115 Acked Write 9 Done Sent Done false
18965 6116 Acked Write 8 Done Sent Done false
18966 6117 Acked Write 3 Done Sent Done false
18967 6118 Acked Read 10 Done Sent Done false
18968 6119 Acked Write 1 Done Sent Done false
18969 6120 Acked Write 7 Done Sent Done false
18970 6121 Acked Read 10 Done Sent Done false
18971 6122 Acked Read 4 Done Sent Done false
18972 6123 Acked Write 2 Done Sent Done false
18973 6124 Acked Write 6 Done Sent Done false
18974 6125 Acked Read 9 Done Sent Done false
18975 6126 Acked Write 10 Done Sent Done false
18976 6127 Acked Read 1 Done Sent Done false
18977 6128 Acked Flush 0 Done Sent Done false
18978 6129 Acked Write 5 Done Sent Done false
18979 6130 Acked Read 2 Done Sent Done false
18980 6131 Acked Read 7 Done Sent Done false
18981 6132 Acked Flush 0 Done Sent Done false
18982 STATES DS:0 DS:1 DS:2 TOTAL
18983 Sent 0 302 0 302
18984 Done 302 0 302 604
18985 Skipped 0 0 0 0
18986 Error 0 0 0 0
18987 Last Flush: 6132 5830 6132
18988 Downstairs last five completed: 5830 5829 5828 5827 5826
18989 Upstairs last five completed: 6132 6131 6130 6129 6128
18990 CLIENT: Up:0 ds:302 act:2
18991 ----------------------------------------------------------------
18992 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18993 States: Active Offline Active
18994 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18995 5831 Acked Write 1 Done Sent Done false
18996 5832 Acked Read 9 Done Sent Done false
18997 5833 Acked Read 9 Done Sent Done false
18998 5834 Acked Write 1 Done Sent Done false
18999 5835 Acked Read 9 Done Sent Done false
19000 5836 Acked Write 5 Done Sent Done false
19001 5837 Acked Write 10 Done Sent Done false
19002 5838 Acked Flush 0 Done Sent Done false
19003 5839 Acked Read 6 Done Sent Done false
19004 5840 Acked Read 8 Done Sent Done false
19005 5841 Acked Flush 0 Done Sent Done false
19006 5842 Acked Write 3 Done Sent Done false
19007 5843 Acked Write 2 Done Sent Done false
19008 5844 Acked Read 4 Done Sent Done false
19009 5845 Acked Read 10 Done Sent Done false
19010 5846 Acked Write 9 Done Sent Done false
19011 5847 Acked Write 7 Done Sent Done false
19012 5848 Acked Write 3 Done Sent Done false
19013 5849 Acked Read 4 Done Sent Done false
19014 5850 Acked Read 7 Done Sent Done false
19015 5851 Acked Read 4 Done Sent Done false
19016 5852 Acked Write 3 Done Sent Done false
19017 5853 Acked Flush 0 Done Sent Done false
19018 5854 Acked Read 4 Done Sent Done false
19019 5855 Acked Write 10 Done Sent Done false
19020 5856 Acked Write 7 Done Sent Done false
19021 5857 Acked Write 10 Done Sent Done false
19022 5858 Acked Write 5 Done Sent Done false
19023 5859 Acked Read 9 Done Sent Done false
19024 5860 Acked Write 8 Done Sent Done false
19025 5861 Acked Flush 0 Done Sent Done false
19026 5862 Acked Write 10 Done Sent Done false
19027 5863 Acked Read 9 Done Sent Done false
19028 5864 Acked Write 7 Done Sent Done false
19029 5865 Acked Read 3 Done Sent Done false
19030 5866 Acked Read 5 Done Sent Done false
19031 5867 Acked Read 7 Done Sent Done false
19032 5868 Acked Write 3 Done Sent Done false
19033 5869 Acked Read 10 Done Sent Done false
19034 5870 Acked Flush 0 Done Sent Done false
19035 5871 Acked Write 4 Done Sent Done false
19036 5872 Acked Read 2 Done Sent Done false
19037 5873 Acked Write 6 Done Sent Done false
19038 5874 Acked Flush 0 Done Sent Done false
19039 5875 Acked Read 5 Done Sent Done false
19040 5876 Acked Write 1 Done Sent Done false
19041 5877 Acked Write 1 Done Sent Done false
19042 5878 Acked Read 9 Done Sent Done false
19043 5879 Acked Flush 0 Done Sent Done false
19044 5880 Acked Read 6 Done Sent Done false
19045 5881 Acked Flush 0 Done Sent Done false
19046 5882 Acked Read 1 Done Sent Done false
19047 5883 Acked Write 3 Done Sent Done false
19048 5884 Acked Write 3 Done Sent Done false
19049 5885 Acked Read 10 Done Sent Done false
19050 5886 Acked Read 8 Done Sent Done false
19051 5887 Acked Write 8 Done Sent Done false
19052 5888 Acked Read 4 Done Sent Done false
19053 5889 Acked Write 9 Done Sent Done false
19054 5890 Acked Read 3 Done Sent Done false
19055 5891 Acked Read 9 Done Sent Done false
19056 5892 Acked Read 2 Done Sent Done false
19057 5893 Acked Write 7 Done Sent Done false
19058 5894 Acked Write 9 Done Sent Done false
19059 5895 Acked Read 4 Done Sent Done false
19060 5896 Acked Read 5 Done Sent Done false
19061 5897 Acked Read 6 Done Sent Done false
19062 5898 Acked Read 10 Done Sent Done false
19063 5899 Acked Read 3 Done Sent Done false
19064 5900 Acked Read 10 Done Sent Done false
19065 5901 Acked Read 7 Done Sent Done false
19066 5902 Acked Write 3 Done Sent Done false
19067 5903 Acked Read 10 Done Sent Done false
19068 5904 Acked Read 5 Done Sent Done false
19069 5905 Acked Flush 0 Done Sent Done false
19070 5906 Acked Read 6 Done Sent Done false
19071 5907 Acked Read 4 Done Sent Done false
19072 5908 Acked Flush 0 Done Sent Done false
19073 5909 Acked Write 1 Done Sent Done false
19074 5910 Acked Write 7 Done Sent Done false
19075 5911 Acked Read 9 Done Sent Done false
19076 5912 Acked Write 8 Done Sent Done false
19077 5913 Acked Flush 0 Done Sent Done false
19078 5914 Acked Write 3 Done Sent Done false
19079 5915 Acked Read 8 Done Sent Done false
19080 5916 Acked Read 1 Done Sent Done false
19081 5917 Acked Write 8 Done Sent Done false
19082 5918 Acked Flush 0 Done Sent Done false
19083 5919 Acked Read 8 Done Sent Done false
19084 5920 Acked Write 3 Done Sent Done false
19085 5921 Acked Read 4 Done Sent Done false
19086 5922 Acked Read 9 Done Sent Done false
19087 5923 Acked Flush 0 Done Sent Done false
19088 5924 Acked Write 9 Done Sent Done false
19089 5925 Acked Read 7 Done Sent Done false
19090 5926 Acked Read 5 Done Sent Done false
19091 5927 Acked Read 2 Done Sent Done false
19092 5928 Acked Read 10 Done Sent Done false
19093 5929 Acked Write 5 Done Sent Done false
19094 5930 Acked Write 6 Done Sent Done false
19095 5931 Acked Read 8 Done Sent Done false
19096 5932 Acked Read 7 Done Sent Done false
19097 5933 Acked Read 7 Done Sent Done false
19098 5934 Acked Write 8 Done Sent Done false
19099 5935 Acked Write 2 Done Sent Done false
19100 5936 Acked Read 4 Done Sent Done false
19101 5937 Acked Write 6 Done Sent Done false
19102 5938 Acked Flush 0 Done Sent Done false
19103 5939 Acked Write 9 Done Sent Done false
19104 5940 Acked Write 7 Done Sent Done false
19105 5941 Acked Read 5 Done Sent Done false
19106 5942 Acked Write 1 Done Sent Done false
19107 5943 Acked Read 5 Done Sent Done false
19108 5944 Acked Write 3 Done Sent Done false
19109 5945 Acked Write 9 Done Sent Done false
19110 5946 Acked Read 10 Done Sent Done false
19111 5947 Acked Write 5 Done Sent Done false
19112 5948 Acked Write 9 Done Sent Done false
19113 5949 Acked Read 5 Done Sent Done false
19114 5950 Acked Read 6 Done Sent Done false
19115 5951 Acked Read 2 Done Sent Done false
19116 5952 Acked Write 9 Done Sent Done false
19117 5953 Acked Read 6 Done Sent Done false
19118 5954 Acked Write 7 Done Sent Done false
19119 5955 Acked Read 4 Done Sent Done false
19120 5956 Acked Read 7 Done Sent Done false
19121 5957 Acked Read 9 Done Sent Done false
19122 5958 Acked Flush 0 Done Sent Done false
19123 5959 Acked Flush 0 Done Sent Done false
19124 5960 Acked Read 7 Done Sent Done false
19125 5961 Acked Write 4 Done Sent Done false
19126 5962 Acked Write 5 Done Sent Done false
19127 5963 Acked Write 9 Done Sent Done false
19128 5964 Acked Write 4 Done Sent Done false
19129 5965 Acked Flush 0 Done Sent Done false
19130 5966 Acked Read 9 Done Sent Done false
19131 5967 Acked Read 8 Done Sent Done false
19132 5968 Acked Read 9 Done Sent Done false
19133 5969 Acked Write 9 Done Sent Done false
19134 5970 Acked Write 2 Done Sent Done false
19135 5971 Acked Flush 0 Done Sent Done false
19136 5972 Acked Write 7 Done Sent Done false
19137 5973 Acked Write 9 Done Sent Done false
19138 5974 Acked Write 6 Done Sent Done false
19139 5975 Acked Write 6 Done Sent Done false
19140 5976 Acked Read 2 Done Sent Done false
19141 5977 Acked Read 8 Done Sent Done false
19142 5978 Acked Write 6 Done Sent Done false
19143 5979 Acked Flush 0 Done Sent Done false
19144 5980 Acked Write 3 Done Sent Done false
19145 5981 Acked Read 2 Done Sent Done false
19146 5982 Acked Read 3 Done Sent Done false
19147 5983 Acked Write 4 Done Sent Done false
19148 5984 Acked Read 9 Done Sent Done false
19149 5985 Acked Write 8 Done Sent Done false
19150 5986 Acked Read 6 Done Sent Done false
19151 5987 Acked Read 10 Done Sent Done false
19152 5988 Acked Read 5 Done Sent Done false
19153 5989 Acked Write 6 Done Sent Done false
19154 5990 Acked Write 8 Done Sent Done false
19155 5991 Acked Read 1 Done Sent Done false
19156 5992 Acked Read 6 Done Sent Done false
19157 5993 Acked Read 6 Done Sent Done false
19158 5994 Acked Write 8 Done Sent Done false
19159 5995 Acked Read 5 Done Sent Done false
19160 5996 Acked Write 10 Done Sent Done false
19161 5997 Acked Read 5 Done Sent Done false
19162 5998 Acked Read 8 Done Sent Done false
19163 5999 Acked Write 3 Done Sent Done false
19164 6000 Acked Write 1 Done Sent Done false
19165 6001 Acked Read 6 Done Sent Done false
19166 6002 Acked Read 10 Done Sent Done false
19167 6003 Acked Read 2 Done Sent Done false
19168 6004 Acked Write 9 Done Sent Done false
19169 6005 Acked Write 5 Done Sent Done false
19170 6006 Acked Write 8 Done Sent Done false
19171 6007 Acked Write 2 Done Sent Done false
19172 6008 Acked Read 9 Done Sent Done false
19173 6009 Acked Read 4 Done Sent Done false
19174 6010 Acked Write 3 Done Sent Done false
19175 6011 Acked Read 3 Done Sent Done false
19176 6012 Acked Write 2 Done Sent Done false
19177 6013 Acked Write 5 Done Sent Done false
19178 6014 Acked Read 4 Done Sent Done false
19179 6015 Acked Write 3 Done Sent Done false
19180 6016 Acked Read 8 Done Sent Done false
19181 6017 Acked Read 4 Done Sent Done false
19182 6018 Acked Read 9 Done Sent Done false
19183 6019 Acked Read 2 Done Sent Done false
19184 6020 Acked Write 6 Done Sent Done false
19185 6021 Acked Flush 0 Done Sent Done false
19186 6022 Acked Read 5 Done Sent Done false
19187 6023 Acked Flush 0 Done Sent Done false
19188 6024 Acked Write 3 Done Sent Done false
19189 6025 Acked Read 6 Done Sent Done false
19190 6026 Acked Read 1 Done Sent Done false
19191 6027 Acked Read 2 Done Sent Done false
19192 6028 Acked Write 8 Done Sent Done false
19193 6029 Acked Read 1 Done Sent Done false
19194 6030 Acked Read 7 Done Sent Done false
19195 6031 Acked Read 8 Done Sent Done false
19196 6032 Acked Write 1 Done Sent Done false
19197 6033 Acked Flush 0 Done Sent Done false
19198 6034 Acked Read 6 Done Sent Done false
19199 6035 Acked Read 4 Done Sent Done false
19200 6036 Acked Read 10 Done Sent Done false
19201 6037 Acked Read 10 Done Sent Done false
19202 6038 Acked Read 9 Done Sent Done false
19203 6039 Acked Read 3 Done Sent Done false
19204 6040 Acked Read 5 Done Sent Done false
19205 6041 Acked Read 1 Done Sent Done false
19206 6042 Acked Flush 0 Done Sent Done false
19207 6043 Acked Flush 0 Done Sent Done false
19208 6044 Acked Read 1 Done Sent Done false
19209 6045 Acked Read 1 Done Sent Done false
19210 6046 Acked Read 2 Done Sent Done false
19211 6047 Acked Write 3 Done Sent Done false
19212 6048 Acked Write 9 Done Sent Done false
19213 6049 Acked Write 9 Done Sent Done false
19214 6050 Acked Read 6 Done Sent Done false
19215 6051 Acked Flush 0 Done Sent Done false
19216 6052 Acked Flush 0 Done Sent Done false
19217 6053 Acked Write 8 Done Sent Done false
19218 6054 Acked Flush 0 Done Sent Done false
19219 6055 Acked Write 5 Done Sent Done false
19220 6056 Acked Write 10 Done Sent Done false
19221 6057 Acked Write 5 Done Sent Done false
19222 6058 Acked Write 10 Done Sent Done false
19223 6059 Acked Flush 0 Done Sent Done false
19224 6060 Acked Write 7 Done Sent Done false
19225 6061 Acked Read 8 Done Sent Done false
19226 6062 Acked Read 10 Done Sent Done false
19227 6063 Acked Read 1 Done Sent Done false
19228 6064 Acked Read 7 Done Sent Done false
19229 6065 Acked Read 5 Done Sent Done false
19230 6066 Acked Read 2 Done Sent Done false
19231 6067 Acked Write 7 Done Sent Done false
19232 6068 Acked Write 6 Done Sent Done false
19233 6069 Acked Write 2 Done Sent Done false
19234 6070 Acked Read 8 Done Sent Done false
19235 6071 Acked Read 7 Done Sent Done false
19236 6072 Acked Read 2 Done Sent Done false
19237 6073 Acked Read 2 Done Sent Done false
19238 6074 Acked Read 8 Done Sent Done false
19239 6075 Acked Read 3 Done Sent Done false
19240 6076 Acked Flush 0 Done Sent Done false
19241 6077 Acked Write 4 Done Sent Done false
19242 6078 Acked Write 4 Done Sent Done false
19243 6079 Acked Read 7 Done Sent Done false
19244 6080 Acked Write 1 Done Sent Done false
19245 6081 Acked Write 5 Done Sent Done false
19246 6082 Acked Read 1 Done Sent Done false
19247 6083 Acked Write 4 Done Sent Done false
19248 6084 Acked Read 2 Done Sent Done false
19249 6085 Acked Read 6 Done Sent Done false
19250 6086 Acked Write 5 Done Sent Done false
19251 6087 Acked Read 9 Done Sent Done false
19252 6088 Acked Read 9 Done Sent Done false
19253 6089 Acked Write 1 Done Sent Done false
19254 6090 Acked Read 3 Done Sent Done false
19255 6091 Acked Write 2 Done Sent Done false
19256 6092 Acked Write 7 Done Sent Done false
19257 6093 Acked Write 8 Done Sent Done false
19258 6094 Acked Read 5 Done Sent Done false
19259 6095 Acked Read 4 Done Sent Done false
19260 6096 Acked Write 6 Done Sent Done false
19261 6097 Acked Read 6 Done Sent Done false
19262 6098 Acked Write 8 Done Sent Done false
19263 6099 Acked Read 5 Done Sent Done false
19264 6100 Acked Write 8 Done Sent Done false
19265 6101 Acked Read 6 Done Sent Done false
19266 6102 Acked Write 3 Done Sent Done false
19267 6103 Acked Read 3 Done Sent Done false
19268 6104 Acked Write 2 Done Sent Done false
19269 6105 Acked Read 9 Done Sent Done false
19270 6106 Acked Read 3 Done Sent Done false
19271 6107 Acked Write 2 Done Sent Done false
19272 6108 Acked Write 4 Done Sent Done false
19273 6109 Acked Write 6 Done Sent Done false
19274 6110 Acked Read 10 Done Sent Done false
19275 6111 Acked Write 10 Done Sent Done false
19276 6112 Acked Read 7 Done Sent Done false
19277 6113 Acked Read 4 Done Sent Done false
19278 6114 Acked Write 9 Done Sent Done false
19279 6115 Acked Write 9 Done Sent Done false
19280 6116 Acked Write 8 Done Sent Done false
19281 6117 Acked Write 3 Done Sent Done false
19282 6118 Acked Read 10 Done Sent Done false
19283 6119 Acked Write 1 Done Sent Done false
19284 6120 Acked Write 7 Done Sent Done false
19285 6121 Acked Read 10 Done Sent Done false
19286 6122 Acked Read 4 Done Sent Done false
19287 6123 Acked Write 2 Done Sent Done false
19288 6124 Acked Write 6 Done Sent Done false
19289 6125 Acked Read 9 Done Sent Done false
19290 6126 Acked Write 10 Done Sent Done false
19291 6127 Acked Read 1 Done Sent Done false
19292 6128 Acked Flush 0 Done Sent Done false
19293 6129 Acked Write 5 Done Sent Done false
19294 6130 Acked Read 2 Done Sent Done false
19295 6131 Acked Read 7 Done Sent Done false
19296 6132 Acked Flush 0 Done Sent Done false
19297 STATES DS:0 DS:1 DS:2 TOTAL
19298 Sent 0 302 0 302
19299 Done 302 0 302 604
19300 Skipped 0 0 0 0
19301 Error 0 0 0 0
19302 Last Flush: 6132 5830 6132
19303 Downstairs last five completed: 5830 5829 5828 5827 5826
19304 Upstairs last five completed: 6132 6131 6130 6129 6128
19305 CLIENT: Up:0 ds:302 act:2
193062025-01-03T08:34:57.578ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193072025-01-03T08:34:57.579ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:46220) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193082025-01-03T08:34:57.580ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193092025-01-03T08:34:57.580ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193102025-01-03T08:34:57.580ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193112025-01-03T08:34:57.580ZINFOcrucible: send last flush ID to this DS: 5830 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193122025-01-03T08:34:57.581ZINFOcrucible: Replied this last flush ID: 5830 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193132025-01-03T08:34:57.581ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193142025-01-03T08:34:57.581ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193152025-01-03T08:34:57.581ZINFOcrucible: [1] client re-new 302 jobs since flush 5830 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
193162025-01-03T08:34:57.581ZINFOcrucible: [1] Replayed 302 jobs since flush: 5830 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
19317 ----------------------------------------------------------------
19318 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
19319 Downstairs last five completed: 6132 6131 6130 6129 6128
19320 Upstairs last five completed: 6132 6131 6130 6129 6128
19321 CLIENT: Up:0 ds:0 act:3
19322 Replay: All jobs finished
19323 Main task has work to do, go find it
19324 got dsc Stop(1)
19325 stop 1
19326 Jan 03 08:34:57.840 INFO request completed, latency_us: 326, response_code: 204[8820] Got stop action so:false kr:false
19327 , uri: /stop/cid/1, method: POST, req_id: 5ae30b78-044c-41cd-ab9b-6c3591cdedaa, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
19328 Jan 03 08:34:57.841{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:34:57.841115628Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
19329 INFO request completed, latency_us: 134, response_code: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:34:57.841191476Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
19330 200{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:34:57.841230726Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
19331 , uri: /state/cid/1, method: GET, req_id: 22ee6372-141e-4c41-a4f3-04ebf1c5e456, remote_addr: 127{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:34:57.841253215Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
19332 .0.0.1:49669{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:34:57.841287415Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
19333 , local_addr: 127.0.0.1:9998{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:34:57.841311604Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
19334 
19335 [8820] Exited with: ExitStatus(unix_wait_status(9))
19336 [8820][1] reports Exit
19337 Jan 03 08:35:01.843 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: a30c3206-d4c0-4374-b9bf-10aad721a851, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
19338 001/300 Read block 1060 len 20480
19339 002/300 Write block 2211 len 28672 data: 4 2 2 3 3 3 4
19340 003/300 Flush
19341 004/300 Read block 3957 len 20480
19342 005/300 Read block 662 len 4096
19343 006/300 Read block 89 len 20480
19344 007/300 Read block 1297 len 32768
19345 008/300 Write block 4712 len 24576 data: 6 4 3 3 4 4
19346 009/300 Flush
19347 010/300 Read block 2366 len 40960
19348 011/300 Write block 3860 len 16384 data: 5 6 7 8
19349 012/300 Read block 2341 len 36864
19350 013/300 Write block 4618 len 28672 data: 4 4 5 5 6 6 5
19351 014/300 Write block 512 len 12288 data: 4 4 5
19352 015/300 Write block 2262 len 40960 data: 3 4 3 3 3 4 4 4 3 3
19353 016/300 Read block 4131 len 12288
19354 017/300 Write block 1607 len 36864 data: 5 2 2 2 2 3 3 4 3
19355 018/300 Read block 4759 len 16384
19356 019/300 Read block 3277 len 28672
19357 020/300 Read block 595 len 28672
19358 021/300 Read block 2703 len 20480
19359 022/300 Read block 1392 len 32768
19360 023/300 Read block 75 len 28672
19361 024/300 Read block 2429 len 28672
19362 025/300 Read block 1696 len 28672
19363 026/300 Write block 643 len 12288 data: 5 4 3
19364 027/300 Read block 2102 len 24576
19365 028/300 Read block 1769 len 40960
19366 029/300 Write block 1826 len 24576 data: 6 7 7 7 8 6
19367 030/300 Flush
19368 031/300 Read block 2193 len 32768
19369 032/300 Read block 1579 len 32768
19370 033/300 Write block 484 len 32768 data: 4 4 5 4 6 7 5 6
19371 034/300 Write block 1491 len 32768 data: 4 3 3 2 2 2 3 4
19372 035/300 Flush
19373 036/300 Read block 4300 len 16384
19374 037/300 Flush
19375 038/300 Write block 164 len 12288 data: 2 2 2
19376 039/300 Read block 3003 len 12288
19377 040/300 Read block 1568 len 20480
19378 041/300 Write block 3690 len 24576 data: 3 3 4 3 3 4
19379 042/300 Write block 3442 len 40960 data: 6 6 4 3 3 3 3 3 3 3
19380 043/300 Read block 4622 len 4096
19381 044/300 Write block 4915 len 36864 data: 3 4 4 6 7 7 7 5 4
19382 045/300 Read block 1970 len 28672
19383 046/300 Write block 4910 len 24576 data: 7 7 4 3 4 4
19384 047/300 Read block 3247 len 24576
19385 048/300 Read block 2704 len 40960
19386 049/300 Write block 2824 len 12288 data: 3 4 5
19387 050/300 Read block 3604 len 4096
19388 051/300 Read block 4661 len 28672
19389 052/300 Read block 137 len 40960
19390 053/300 Flush
19391 054/300 Read block 1034 len 32768
19392 055/300 Flush
19393 056/300 Write block 2009 len 20480 data: 2 2 3 3 2
19394 057/300 Read block 2123 len 24576
19395 058/300 Flush
19396 059/300 Read block 3242 len 32768
19397 060/300 Read block 3934 len 28672
19398 061/300 Write block 151 len 12288 data: 3 3 3
19399 062/300 Read block 3342 len 40960
19400 063/300 Write block 2662 len 36864 data: 7 7 6 5 5 5 3 2 2
19401 064/300 Read block 429 len 24576
19402 065/300 Read block 3154 len 4096
19403 066/300 Write block 2213 len 36864 data: 3 4 4 4 5 5 5 5 6
19404 067/300 Read block 392 len 12288
19405 068/300 Read block 2197 len 36864
19406 069/300 Read block 4885 len 28672
19407 070/300 Write block 2779 len 8192 data: 6 5
19408 071/300 Read block 988 len 20480
19409 072/300 Write block 1133 len 36864 data: 5 4 5 5 4 4 4 4 4
19410 073/300 Read block 3655 len 8192
19411 074/300 Read block 176 len 32768
19412 075/300 Flush
19413 076/300 Write block 2388 len 28672 data: 4 4 4 5 5 5 5
19414 077/300 Write block 1829 len 12288 data: 8 9 7
19415 078/300 Read block 889 len 36864
19416 079/300 Read block 3486 len 16384
19417 080/300 Read block 2189 len 32768
19418 081/300 Read block 2531 len 12288
19419 082/300 Read block 894 len 20480
19420 083/300 Write block 1714 len 4096 data: 4
19421 084/300 Write block 2014 len 32768 data: 3 4 4 5 5 4 4 5
19422 085/300 Write block 3327 len 24576 data: 4 5 5 5 6 6
19423 086/300 Read block 3434 len 16384
19424 087/300 Read block 2616 len 24576
19425 088/300 Write block 1037 len 8192 data: 2 2
19426 089/300 Read block 1276 len 12288
19427 090/300 Write block 3880 len 8192 data: 3 4
19428 091/300 Write block 3386 len 8192 data: 7 7
19429 092/300 Read block 1809 len 12288
19430 093/300 Read block 3219 len 24576
19431 094/300 Write block 4995 len 16384 data: 3 3 3 2
19432 095/300 Write block 2214 len 8192 data: 5 5
19433 096/300 Write block 4831 len 36864 data: 5 5 5 5 5 5 5 5 3
19434 097/300 Write block 931 len 16384 data: 3 3 3 3
19435 098/300 Read block 1467 len 28672
19436 099/300 Write block 3866 len 40960 data: 8 9 8 8 6 7 7 5 5 4
19437 100/300 Read block 4002 len 32768
19438 101/300 Read block 186 len 36864
19439 102/300 Read block 294 len 32768
19440 103/300 Write block 181 len 4096 data: 2
19441 104/300 Write block 4348 len 20480 data: 5 5 5 3 3
19442 105/300 Read block 2442 len 20480
19443 106/300 Write block 1668 len 20480 data: 3 4 5 6 6
19444 107/300 Write block 4140 len 24576 data: 3 4 4 6 6 6
19445 108/300 Read block 3443 len 40960
19446 109/300 Read block 4544 len 4096
19447 110/300 Read block 3679 len 8192
19448 111/300 Write block 4016 len 8192 data: 3 4
19449 112/300 Read block 1738 len 32768
19450 113/300 Flush
19451 114/300 Read block 2285 len 20480
19452 115/300 Write block 2295 len 16384 data: 3 2 3 4
19453 116/300 Read block 4584 len 40960
19454 117/300 Read block 3391 len 4096
19455 118/300 Read block 1919 len 32768
19456 119/300 Write block 4317 len 4096 data: 4
19457 120/300 Write block 4669 len 40960 data: 5 5 4 4 4 4 3 3 3 3
19458 121/300 Read block 4008 len 32768
19459 122/300 Write block 1803 len 32768 data: 5 4 4 4 4 4 4 4
19460 123/300 Read block 3138 len 12288
19461 124/300 Write block 1665 len 8192 data: 2 2
19462 125/300 Write block 2229 len 4096 data: 5
19463 126/300 Write block 2099 len 16384 data: 4 3 3 2
19464 127/300 Write block 4017 len 36864 data: 5 5 5 6 6 6 6 5 5
19465 128/300 Write block 4250 len 36864 data: 4 5 7 6 6 7 6 5 5
19466 129/300 Read block 4258 len 32768
19467 130/300 Read block 3445 len 8192
19468 131/300 Write block 1129 len 8192 data: 3 4
19469 132/300 Write block 963 len 40960 data: 3 3 3 3 3 3 3 2 2 3
19470 133/300 Read block 353 len 40960
19471 134/300 Write block 1174 len 36864 data: 6 5 6 6 5 5 4 5 5
19472 135/300 Read block 4529 len 4096
19473 136/300 Write block 3081 len 28672 data: 3 3 6 6 7 5 4
19474 137/300 Read block 1223 len 4096
19475 138/300 Read block 1676 len 36864
19476 139/300 Read block 4533 len 40960
19477 140/300 Read block 4258 len 28672
19478 141/300 Write block 4251 len 4096 data: 6
19479 142/300 Write block 3112 len 12288 data: 2 2 2
19480 143/300 Read block 4903 len 16384
19481 144/300 Read block 3044 len 8192
19482 145/300 Read block 3097 len 40960
19483 146/300 Read block 1302 len 24576
19484 147/300 Write block 2284 len 12288 data: 3 4 4
19485 148/300 Flush
19486 149/300 Read block 1192 len 4096
19487 150/300 Read block 320 len 40960
19488 151/300 Write block 4446 len 12288 data: 5 5 7
19489 152/300 Read block 2794 len 16384
19490 153/300 Write block 4475 len 20480 data: 5 6 5 6 5
19491 154/300 Read block 3996 len 32768
19492 155/300 Write block 4190 len 24576 data: 3 4 5 5 4 5
19493 156/300 Write block 1475 len 24576 data: 3 2 2 3 3 3
19494 157/300 Read block 3679 len 12288
19495 158/300 Read block 4648 len 16384
19496 159/300 Read block 4030 len 20480
19497 160/300 Read block 3998 len 32768
19498 161/300 Write block 1731 len 16384 data: 4 4 3 4
19499 162/300 Write block 4035 len 12288 data: 2 2 2
19500 163/300 Write block 4130 len 24576 data: 2 2 2 2 3 4
19501 164/300 Write block 2480 len 16384 data: 4 6 7 8
19502 165/300 Read block 1917 len 36864
19503 166/300 Write block 3268 len 16384 data: 3 3 5 5
19504 167/300 Read block 2879 len 28672
19505 168/300 Read block 4905 len 32768
19506 169/300 Write block 1725 len 40960 data: 2 2 2 3 3 3 5 5 4 5
19507 170/300 Write block 4067 len 32768 data: 6 6 5 5 5 5 7 6
19508 171/300 Read block 1713 len 8192
19509 172/300 Read block 4437 len 16384
19510 173/300 Read block 108 len 16384
19511 174/300 Read block 3736 len 36864
19512 175/300 Read block 1704 len 4096
19513 176/300 Write block 4945 len 36864 data: 5 5 6 7 6 5 3 4 4
19514 177/300 Write block 3453 len 36864 data: 3 5 5 6 6 6 6 5 3
19515 178/300 Read block 1676 len 28672
19516 179/300 Read block 3249 len 4096
19517 180/300 Flush
19518 181/300 Write block 383 len 28672 data: 4 4 4 3 3 3 3
19519 182/300 Write block 3832 len 28672 data: 5 5 3 3 3 3 4
19520 183/300 Read block 4481 len 4096
19521 184/300 Flush
19522 185/300 Write block 3016 len 20480 data: 5 5 4 4 4
19523 186/300 Read block 1399 len 4096
19524 187/300 Read block 828 len 16384
19525 188/300 Write block 2755 len 24576 data: 3 3 5 5 5 5
19526 189/300 Read block 249 len 20480
19527 190/300 Read block 4906 len 28672
19528 191/300 Write block 1995 len 32768 data: 3 3 3 3 4 4 4 3
19529 192/300 Read block 2018 len 12288
19530 193/300 Write block 183 len 36864 data: 3 3 2 2 3 3 3 3 3
19531 194/300 Read block 744 len 32768
19532 195/300 Read block 1301 len 36864
19533 196/300 Read block 4709 len 12288
19534 197/300 Read block 976 len 8192
19535 198/300 Write block 4874 len 4096 data: 4
19536 199/300 Read block 3502 len 32768
19537 200/300 Write block 1991 len 24576 data: 4 5 4 3 4 4
19538 201/300 Write block 310 len 4096 data: 8
19539 202/300 Write block 2641 len 20480 data: 6 5 4 4 4
19540 203/300 Read block 3067 len 40960
19541 204/300 Write block 2728 len 16384 data: 4 4 4 6
19542 205/300 Read block 4230 len 28672
19543 206/300 Write block 885 len 36864 data: 5 5 7 8 8 7 7 6 5
19544 207/300 Read block 2736 len 8192
19545 208/300 Read block 3640 len 28672
19546 209/300 Read block 3692 len 32768
19547 210/300 Write block 2463 len 8192 data: 3 3
19548 211/300 Read block 234 len 20480
19549 212/300 Write block 4580 len 12288 data: 6 7 5
19550 213/300 Write block 4467 len 28672 data: 6 6 6 6 6 6 6
19551 214/300 Read block 224 len 20480
19552 215/300 Write block 4313 len 28672 data: 6 5 4 4 5 4 5
19553 216/300 Write block 559 len 16384 data: 3 3 3 3
19554 217/300 Write block 4858 len 4096 data: 5
19555 218/300 Write block 683 len 28672 data: 5 5 5 3 3 2 3
19556 219/300 Flush
19557 220/300 Write block 318 len 8192 data: 5 5
19558 221/300 Write block 1503 len 8192 data: 4 4
19559 222/300 Read block 2457 len 28672
19560 223/300 Read block 162 len 28672
19561 224/300 Read block 3268 len 8192
19562 225/300 Read block 374 len 28672
19563 226/300 Write block 4105 len 24576 data: 3 3 3 2 2 2
19564 227/300 Read block 2057 len 20480
19565 228/300 Write block 1927 len 4096 data: 5
19566 229/300 Write block 3511 len 4096 data: 4
19567 230/300 Flush
19568 231/300 Write block 3881 len 28672 data: 5 5 6 5 4 5 5
19569 232/300 Read block 1679 len 28672
19570 233/300 Write block 638 len 12288 data: 7 7 8
19571 234/300 Read block 3574 len 24576
19572 235/300 Read block 1218 len 20480
19573 236/300 Write block 1468 len 4096 data: 3
19574 237/300 Read block 2994 len 16384
19575 238/300 Read block 1657 len 8192
19576 239/300 Read block 929 len 32768
19577 240/300 Read block 3143 len 20480
19578 241/300 Write block 1187 len 32768 data: 3 3 3 4 4 4 4 4
19579 242/300 Write block 2884 len 28672 data: 2 2 2 2 2 2 3
19580 243/300 Write block 1589 len 12288 data: 6 5 5
19581 244/300 Write block 1409 len 32768 data: 4 3 3 3 2 3 3 3
19582 245/300 Read block 4475 len 28672
19583 246/300 Write block 4559 len 28672 data: 5 5 3 3 3 4 4
19584 247/300 Read block 1247 len 40960
19585 248/300 Read block 4514 len 16384
19586 249/300 Read block 1013 len 32768
19587 250/300 Flush
19588 251/300 Read block 4352 len 16384
19589 252/300 Flush
19590 253/300 Write block 522 len 32768 data: 6 6 6 3 3 4 3 4
19591 254/300 Write block 305 len 36864 data: 6 4 5 7 7 9 7 7 7
19592 255/300 Write block 1619 len 16384 data: 4 4 4 4
19593 256/300 Read block 1015 len 8192
19594 257/300 Read block 532 len 16384
19595 258/300 Read block 4525 len 20480
19596 259/300 Read block 1501 len 8192
19597 260/300 Read block 3921 len 16384
19598 261/300 Read block 3396 len 16384
19599 262/300 Read block 3723 len 36864
19600 263/300 Write block 2048 len 32768 data: 5 5 6 4 5 6 6 7
19601 264/300 Flush
19602 265/300 Read block 526 len 28672
19603 266/300 Write block 4547 len 4096 data: 2
19604 267/300 Flush
19605 268/300 Write block 126 len 8192 data: 2 2
19606 269/300 Read block 4808 len 36864
19607 270/300 Write block 1788 len 32768 data: 4 3 3 3 3 4 3 3
19608 271/300 Write block 4834 len 40960 data: 6 6 6 6 6 4 3 3 3 6
19609 272/300 Flush
19610 273/300 Read block 2089 len 28672
19611 274/300 Write block 201 len 16384 data: 2 2 2 2
19612 275/300 Write block 319 len 24576 data: 6 3 2 3 3 3
19613 276/300 Write block 4244 len 20480 data: 4 4 4 4 4
19614 277/300 Read block 3970 len 20480
19615 278/300 Read block 2942 len 20480
19616 279/300 Write block 4884 len 12288 data: 5 5 6
19617 280/300 Read block 4647 len 40960
19618 281/300 Read block 2591 len 8192
19619 282/300 Read block 1966 len 16384
19620 283/300 Flush
19621 284/300 Write block 4884 len 12288 data: 6 6 7
19622 285/300 Read block 3154 len 20480
19623 286/300 Flush
19624 287/300 Write block 2995 len 28672 data: 3 2 2 2 2 2 2
19625 288/300 Write block 4642 len 8192 data: 5 4
19626 289/300 Write block 583 len 20480 data: 3 3 3 2 4
19627 290/300 Read block 4193 len 4096
19628 291/300 Write block 3889 len 4096 data: 4
19629 292/300 Write block 1478 len 36864 data: 4 4 4 3 3 4 4 4 4
19630 293/300 Read block 1510 len 36864
19631 294/300 Write block 750 len 12288 data: 5 5 5
19632 295/300 Write block 1263 len 28672 data: 4 4 6 5 6 6 6
19633 296/300 Read block 3894 len 12288
19634 297/300 Write block 2969 len 36864 data: 4 5 6 6 6 7 8 7 6
19635 298/300 Write block 4103 len 16384 data: 3 3 4 4
19636 299/300 Write block 4980 len 28672 data: 3 4 3 3 3 2 2
19637 300/300 Flush
19638 301/300 Write block 3234 len 36864 data: 8 8 8 8 6 6 5 5 5
19639 Main task has work to do, go find it
19640 got dsc Start(1)
19641 start 1
19642 Jan 03 08:35:02.087 INFO request completed, latency_us: 268, response_code: 204, uri: /start/cid/1, method: POST, req_id: 1b1b150e-276a-4b4c-bd7c-90ee859ae51f, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
19643 Replay: started 1, returned:Ok(())
19644 ----------------------------------------------------------------
19645 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
19646 States: Active Offline Active
19647 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19648 6133 Acked Read 5 Done Sent Done false
19649 6134 Acked Write 7 Done Sent Done false
19650 6135 Acked Flush 0 Done Sent Done false
19651 6136 Acked Read 5 Done Sent Done false
19652 6137 Acked Read 1 Done Sent Done false
19653 6138 Acked Read 5 Done Sent Done false
19654 6139 Acked Read 8 Done Sent Done false
19655 6140 Acked Write 6 Done Sent Done false
19656 6141 Acked Flush 0 Done Sent Done false
19657 6142 Acked Read 10 Done Sent Done false
19658 6143 Acked Write 4 Done Sent Done false
19659 6144 Acked Read 9 Done Sent Done false
19660 6145 Acked Write 7 Done Sent Done false
19661 6146 Acked Write 3 Done Sent Done false
19662 6147 Acked Write 10 Done Sent Done false
19663 6148 Acked Read 3 Done Sent Done false
19664 6149 Acked Write 9 Done Sent Done false
19665 6150 Acked Read 4 Done Sent Done false
19666 6151 Acked Read 7 Done Sent Done false
19667 6152 Acked Read 7 Done Sent Done false
19668 6153 Acked Read 5 Done Sent Done false
19669 6154 Acked Read 8 Done Sent Done false
19670 6155 Acked Read 7 Done Sent Done false
19671 6156 Acked Read 7 Done Sent Done false
19672 6157 Acked Read 7 Done Sent Done false
19673 6158 Acked Write 3 Done Sent Done false
19674 6159 Acked Read 6 Done Sent Done false
19675 6160 Acked Read 10 Done Sent Done false
19676 6161 Acked Write 6 Done Sent Done false
19677 6162 Acked Flush 0 Done Sent Done false
19678 6163 Acked Read 8 Done Sent Done false
19679 6164 Acked Read 8 Done Sent Done false
19680 6165 Acked Write 8 Done Sent Done false
19681 6166 Acked Write 8 Done Sent Done false
19682 6167 Acked Flush 0 Done Sent Done false
19683 6168 Acked Read 4 Done Sent Done false
19684 6169 Acked Flush 0 Done Sent Done false
19685 6170 Acked Write 3 Done Sent Done false
19686 6171 Acked Read 3 Done Sent Done false
19687 6172 Acked Read 5 Done Sent Done false
19688 6173 Acked Write 6 Done Sent Done false
19689 6174 Acked Write 10 Done Sent Done false
19690 6175 Acked Read 1 Done Sent Done false
19691 6176 Acked Write 9 Done Sent Done false
19692 6177 Acked Read 7 Done Sent Done false
19693 6178 Acked Write 6 Done Sent Done false
19694 6179 Acked Read 6 Done Sent Done false
19695 6180 Acked Read 10 Done Sent Done false
19696 6181 Acked Write 3 Done Sent Done false
19697 6182 Acked Read 1 Done Sent Done false
19698 6183 Acked Read 7 Done Sent Done false
19699 6184 Acked Read 10 Done Sent Done false
19700 6185 Acked Flush 0 Done Sent Done false
19701 6186 Acked Read 8 Done Sent Done false
19702 6187 Acked Flush 0 Done Sent Done false
19703 6188 Acked Write 5 Done Sent Done false
19704 6189 Acked Read 6 Done Sent Done false
19705 6190 Acked Flush 0 Done Sent Done false
19706 6191 Acked Read 8 Done Sent Done false
19707 6192 Acked Read 7 Done Sent Done false
19708 6193 Acked Write 3 Done Sent Done false
19709 6194 Acked Read 10 Done Sent Done false
19710 6195 Acked Write 9 Done Sent Done false
19711 6196 Acked Read 6 Done Sent Done false
19712 6197 Acked Read 1 Done Sent Done false
19713 6198 Acked Write 9 Done Sent Done false
19714 6199 Acked Read 3 Done Sent Done false
19715 6200 Acked Read 9 Done Sent Done false
19716 6201 Acked Read 7 Done Sent Done false
19717 6202 Acked Write 2 Done Sent Done false
19718 6203 Acked Read 5 Done Sent Done false
19719 6204 Acked Write 9 Done Sent Done false
19720 6205 Acked Read 2 Done Sent Done false
19721 6206 Acked Read 8 Done Sent Done false
19722 6207 Acked Flush 0 Done Sent Done false
19723 6208 Acked Write 7 Done Sent Done false
19724 6209 Acked Write 3 Done Sent Done false
19725 6210 Acked Read 9 Done Sent Done false
19726 6211 Acked Read 4 Done Sent Done false
19727 6212 Acked Read 8 Done Sent Done false
19728 6213 Acked Read 3 Done Sent Done false
19729 6214 Acked Read 5 Done Sent Done false
19730 6215 Acked Write 1 Done Sent Done false
19731 6216 Acked Write 8 Done Sent Done false
19732 6217 Acked Write 6 Done Sent Done false
19733 6218 Acked Read 4 Done Sent Done false
19734 6219 Acked Read 6 Done Sent Done false
19735 6220 Acked Write 2 Done Sent Done false
19736 6221 Acked Read 3 Done Sent Done false
19737 6222 Acked Write 2 Done Sent Done false
19738 6223 Acked Write 2 Done Sent Done false
19739 6224 Acked Read 3 Done Sent Done false
19740 6225 Acked Read 6 Done Sent Done false
19741 6226 Acked Write 4 Done Sent Done false
19742 6227 Acked Write 2 Done Sent Done false
19743 6228 Acked Write 9 Done Sent Done false
19744 6229 Acked Write 4 Done Sent Done false
19745 6230 Acked Read 7 Done Sent Done false
19746 6231 Acked Write 10 Done Sent Done false
19747 6232 Acked Read 8 Done Sent Done false
19748 6233 Acked Read 9 Done Sent Done false
19749 6234 Acked Read 8 Done Sent Done false
19750 6235 Acked Write 1 Done Sent Done false
19751 6236 Acked Write 5 Done Sent Done false
19752 6237 Acked Read 5 Done Sent Done false
19753 6238 Acked Write 5 Done Sent Done false
19754 6239 Acked Write 6 Done Sent Done false
19755 6240 Acked Read 10 Done Sent Done false
19756 6241 Acked Read 1 Done Sent Done false
19757 6242 Acked Read 2 Done Sent Done false
19758 6243 Acked Write 2 Done Sent Done false
19759 6244 Acked Read 8 Done Sent Done false
19760 6245 Acked Flush 0 Done Sent Done false
19761 6246 Acked Read 5 Done Sent Done false
19762 6247 Acked Write 4 Done Sent Done false
19763 6248 Acked Read 10 Done Sent Done false
19764 6249 Acked Read 1 Done Sent Done false
19765 6250 Acked Read 8 Done Sent Done false
19766 6251 Acked Write 1 Done Sent Done false
19767 6252 Acked Write 10 Done Sent Done false
19768 6253 Acked Read 8 Done Sent Done false
19769 6254 Acked Write 8 Done Sent Done false
19770 6255 Acked Read 3 Done Sent Done false
19771 6256 Acked Write 2 Done Sent Done false
19772 6257 Acked Write 1 Done Sent Done false
19773 6258 Acked Write 4 Done Sent Done false
19774 6259 Acked Write 9 Done Sent Done false
19775 6260 Acked Write 9 Done Sent Done false
19776 6261 Acked Read 8 Done Sent Done false
19777 6262 Acked Read 2 Done Sent Done false
19778 6263 Acked Write 2 Done Sent Done false
19779 6264 Acked Write 10 Done Sent Done false
19780 6265 Acked Read 10 Done Sent Done false
19781 6266 Acked Write 9 Done Sent Done false
19782 6267 Acked Read 1 Done Sent Done false
19783 6268 Acked Write 7 Done Sent Done false
19784 6269 Acked Read 1 Done Sent Done false
19785 6270 Acked Read 9 Done Sent Done false
19786 6271 Acked Read 10 Done Sent Done false
19787 6272 Acked Read 7 Done Sent Done false
19788 6273 Acked Write 1 Done Sent Done false
19789 6274 Acked Write 3 Done Sent Done false
19790 6275 Acked Read 4 Done Sent Done false
19791 6276 Acked Read 2 Done Sent Done false
19792 6277 Acked Read 10 Done Sent Done false
19793 6278 Acked Read 6 Done Sent Done false
19794 6279 Acked Write 3 Done Sent Done false
19795 6280 Acked Flush 0 Done Sent Done false
19796 6281 Acked Read 1 Done Sent Done false
19797 6282 Acked Read 10 Done Sent Done false
19798 6283 Acked Write 3 Done Sent Done false
19799 6284 Acked Read 4 Done Sent Done false
19800 6285 Acked Write 5 Done Sent Done false
19801 6286 Acked Read 8 Done Sent Done false
19802 6287 Acked Write 6 Done Sent Done false
19803 6288 Acked Write 6 Done Sent Done false
19804 6289 Acked Read 3 Done Sent Done false
19805 6290 Acked Read 4 Done Sent Done false
19806 6291 Acked Read 5 Done Sent Done false
19807 6292 Acked Read 8 Done Sent Done false
19808 6293 Acked Write 4 Done Sent Done false
19809 6294 Acked Write 3 Done Sent Done false
19810 6295 Acked Write 6 Done Sent Done false
19811 6296 Acked Write 4 Done Sent Done false
19812 6297 Acked Read 9 Done Sent Done false
19813 6298 Acked Write 4 Done Sent Done false
19814 6299 Acked Read 7 Done Sent Done false
19815 6300 Acked Read 8 Done Sent Done false
19816 6301 Acked Write 10 Done Sent Done false
19817 6302 Acked Write 8 Done Sent Done false
19818 6303 Acked Read 2 Done Sent Done false
19819 6304 Acked Read 4 Done Sent Done false
19820 6305 Acked Read 4 Done Sent Done false
19821 6306 Acked Read 9 Done Sent Done false
19822 6307 Acked Read 1 Done Sent Done false
19823 6308 Acked Write 9 Done Sent Done false
19824 6309 Acked Write 9 Done Sent Done false
19825 6310 Acked Read 7 Done Sent Done false
19826 6311 Acked Read 1 Done Sent Done false
19827 6312 Acked Flush 0 Done Sent Done false
19828 6313 Acked Write 7 Done Sent Done false
19829 6314 Acked Write 7 Done Sent Done false
19830 6315 Acked Read 1 Done Sent Done false
19831 6316 Acked Flush 0 Done Sent Done false
19832 6317 Acked Write 5 Done Sent Done false
19833 6318 Acked Read 1 Done Sent Done false
19834 6319 Acked Read 4 Done Sent Done false
19835 6320 Acked Write 6 Done Sent Done false
19836 6321 Acked Read 5 Done Sent Done false
19837 6322 Acked Read 7 Done Sent Done false
19838 6323 Acked Write 8 Done Sent Done false
19839 6324 Acked Read 3 Done Sent Done false
19840 6325 Acked Write 9 Done Sent Done false
19841 6326 Acked Read 8 Done Sent Done false
19842 6327 Acked Read 9 Done Sent Done false
19843 6328 Acked Read 3 Done Sent Done false
19844 6329 Acked Read 2 Done Sent Done false
19845 6330 Acked Write 1 Done Sent Done false
19846 6331 Acked Read 8 Done Sent Done false
19847 6332 Acked Write 6 Done Sent Done false
19848 6333 Acked Write 1 Done Sent Done false
19849 6334 Acked Write 5 Done Sent Done false
19850 6335 Acked Read 10 Done Sent Done false
19851 6336 Acked Write 4 Done Sent Done false
19852 6337 Acked Read 7 Done Sent Done false
19853 6338 Acked Write 9 Done Sent Done false
19854 6339 Acked Read 2 Done Sent Done false
19855 6340 Acked Read 7 Done Sent Done false
19856 6341 Acked Read 8 Done Sent Done false
19857 6342 Acked Write 2 Done Sent Done false
19858 6343 Acked Read 5 Done Sent Done false
19859 6344 Acked Write 3 Done Sent Done false
19860 6345 Acked Write 7 Done Sent Done false
19861 6346 Acked Read 5 Done Sent Done false
19862 6347 Acked Write 7 Done Sent Done false
19863 6348 Acked Write 4 Done Sent Done false
19864 6349 Acked Write 1 Done Sent Done false
19865 6350 Acked Write 7 Done Sent Done false
19866 6351 Acked Flush 0 Done Sent Done false
19867 6352 Acked Write 2 Done Sent Done false
19868 6353 Acked Write 2 Done Sent Done false
19869 6354 Acked Read 7 Done Sent Done false
19870 6355 Acked Read 7 Done Sent Done false
19871 6356 Acked Read 2 Done Sent Done false
19872 6357 Acked Read 7 Done Sent Done false
19873 6358 Acked Write 6 Done Sent Done false
19874 6359 Acked Read 5 Done Sent Done false
19875 6360 Acked Write 1 Done Sent Done false
19876 6361 Acked Write 1 Done Sent Done false
19877 6362 Acked Flush 0 Done Sent Done false
19878 6363 Acked Write 7 Done Sent Done false
19879 6364 Acked Read 7 Done Sent Done false
19880 6365 Acked Write 3 Done Sent Done false
19881 6366 Acked Read 6 Done Sent Done false
19882 6367 Acked Read 5 Done Sent Done false
19883 6368 Acked Write 1 Done Sent Done false
19884 6369 Acked Read 4 Done Sent Done false
19885 6370 Acked Read 2 Done Sent Done false
19886 6371 Acked Read 8 Done Sent Done false
19887 6372 Acked Read 5 Done Sent Done false
19888 6373 Acked Write 8 Done Sent Done false
19889 6374 Acked Write 7 Done Sent Done false
19890 6375 Acked Write 3 Done Sent Done false
19891 6376 Acked Write 8 Done Sent Done false
19892 6377 Acked Read 7 Done Sent Done false
19893 6378 Acked Write 7 Done Sent Done false
19894 6379 Acked Read 10 Done Sent Done false
19895 6380 Acked Read 4 Done Sent Done false
19896 6381 Acked Read 8 Done Sent Done false
19897 6382 Acked Flush 0 Done Sent Done false
19898 6383 Acked Read 4 Done Sent Done false
19899 6384 Acked Flush 0 Done Sent Done false
19900 6385 Acked Write 8 Done Sent Done false
19901 6386 Acked Write 9 Done Sent Done false
19902 6387 Acked Write 4 Done Sent Done false
19903 6388 Acked Read 2 Done Sent Done false
19904 6389 Acked Read 4 Done Sent Done false
19905 6390 Acked Read 5 Done Sent Done false
19906 6391 Acked Read 2 Done Sent Done false
19907 6392 Acked Read 4 Done Sent Done false
19908 6393 Acked Read 4 Done Sent Done false
19909 6394 Acked Read 9 Done Sent Done false
19910 6395 Acked Write 8 Done Sent Done false
19911 6396 Acked Flush 0 Done Sent Done false
19912 6397 Acked Read 7 Done Sent Done false
19913 6398 Acked Write 1 Done Sent Done false
19914 6399 Acked Flush 0 Done Sent Done false
19915 6400 Acked Write 2 Done Sent Done false
19916 6401 Acked Read 9 Done Sent Done false
19917 6402 Acked Write 8 Done Sent Done false
19918 6403 Acked Write 10 Done Sent Done false
19919 6404 Acked Flush 0 Done Sent Done false
19920 6405 Acked Read 7 Done Sent Done false
19921 6406 Acked Write 4 Done Sent Done false
19922 6407 Acked Write 6 Done Sent Done false
19923 6408 Acked Write 5 Done Sent Done false
19924 6409 Acked Read 5 Done Sent Done false
19925 6410 Acked Read 5 Done Sent Done false
19926 6411 Acked Write 3 Done Sent Done false
19927 6412 Acked Read 10 Done Sent Done false
19928 6413 Acked Read 2 Done Sent Done false
19929 6414 Acked Read 4 Done Sent Done false
19930 6415 Acked Flush 0 Done Sent Done false
19931 6416 Acked Write 3 Done Sent Done false
19932 6417 Acked Read 5 Done Sent Done false
19933 6418 Acked Flush 0 Done Sent Done false
19934 6419 Acked Write 7 Done Sent Done false
19935 6420 Acked Write 2 Done Sent Done false
19936 6421 Acked Write 5 Done Sent Done false
19937 6422 Acked Read 1 Done Sent Done false
19938 6423 Acked Write 1 Done Sent Done false
19939 6424 Acked Write 9 Done Sent Done false
19940 6425 Acked Read 9 Done Sent Done false
19941 6426 Acked Write 3 Done Sent Done false
19942 6427 Acked Write 7 Done Sent Done false
19943 6428 Acked Read 3 Done Sent Done false
19944 6429 Acked Write 9 Done Sent Done false
19945 6430 Acked Write 4 Done Sent Done false
19946 6431 Acked Write 7 Done Sent Done false
19947 6432 Acked Flush 0 Done Sent Done false
19948 6433 Acked Write 9 Done Sent Done false
19949 STATES DS:0 DS:1 DS:2 TOTAL
19950 Sent 0 301 0 301
19951 Done 301 0 301 602
19952 Skipped 0 0 0 0
19953 Error 0 0 0 0
19954 Last Flush: 6432 6132 6432
19955 Downstairs last five completed: 6132 6131 6130 6129 6128
19956 Upstairs last five completed: 6433 6432 6431 6430 6429
19957 CLIENT: Up:0 ds:301 act:2
19958 [8820] got start action
19959 [8820] I am going to restart
19960 Starting downstairs at port 8820
19961 Make output file at "/tmp/dsc/downstairs-8820.txt"
19962 [8820][1] reports Starting
19963 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1223)
19964 [8820][1] reports Running
19965 ----------------------------------------------------------------
19966 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19967 States: Active Offline Active
19968 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19969 6133 Acked Read 5 Done Sent Done false
19970 6134 Acked Write 7 Done Sent Done false
19971 6135 Acked Flush 0 Done Sent Done false
19972 6136 Acked Read 5 Done Sent Done false
19973 6137 Acked Read 1 Done Sent Done false
19974 6138 Acked Read 5 Done Sent Done false
19975 6139 Acked Read 8 Done Sent Done false
19976 6140 Acked Write 6 Done Sent Done false
19977 6141 Acked Flush 0 Done Sent Done false
19978 6142 Acked Read 10 Done Sent Done false
19979 6143 Acked Write 4 Done Sent Done false
19980 6144 Acked Read 9 Done Sent Done false
19981 6145 Acked Write 7 Done Sent Done false
19982 6146 Acked Write 3 Done Sent Done false
19983 6147 Acked Write 10 Done Sent Done false
19984 6148 Acked Read 3 Done Sent Done false
19985 6149 Acked Write 9 Done Sent Done false
19986 6150 Acked Read 4 Done Sent Done false
19987 6151 Acked Read 7 Done Sent Done false
19988 6152 Acked Read 7 Done Sent Done false
19989 6153 Acked Read 5 Done Sent Done false
19990 6154 Acked Read 8 Done Sent Done false
19991 6155 Acked Read 7 Done Sent Done false
19992 6156 Acked Read 7 Done Sent Done false
19993 6157 Acked Read 7 Done Sent Done false
19994 6158 Acked Write 3 Done Sent Done false
19995 6159 Acked Read 6 Done Sent Done false
19996 6160 Acked Read 10 Done Sent Done false
19997 6161 Acked Write 6 Done Sent Done false
19998 6162 Acked Flush 0 Done Sent Done false
19999 6163 Acked Read 8 Done Sent Done false
20000 6164 Acked Read 8 Done Sent Done false
20001 6165 Acked Write 8 Done Sent Done false
20002 6166 Acked Write 8 Done Sent Done false
20003 6167 Acked Flush 0 Done Sent Done false
20004 6168 Acked Read 4 Done Sent Done false
20005 6169 Acked Flush 0 Done Sent Done false
20006 6170 Acked Write 3 Done Sent Done false
20007 6171 Acked Read 3 Done Sent Done false
20008 6172 Acked Read 5 Done Sent Done false
20009 6173 Acked Write 6 Done Sent Done false
20010 6174 Acked Write 10 Done Sent Done false
20011 6175 Acked Read 1 Done Sent Done false
20012 6176 Acked Write 9 Done Sent Done false
20013 6177 Acked Read 7 Done Sent Done false
20014 6178 Acked Write 6 Done Sent Done false
20015 6179 Acked Read 6 Done Sent Done false
20016 6180 Acked Read 10 Done Sent Done false
20017 6181 Acked Write 3 Done Sent Done false
20018 6182 Acked Read 1 Done Sent Done false
20019 6183 Acked Read 7 Done Sent Done false
20020 6184 Acked Read 10 Done Sent Done false
20021 6185 Acked Flush 0 Done Sent Done false
20022 6186 Acked Read 8 Done Sent Done false
20023 6187 Acked Flush 0 Done Sent Done false
20024 6188 Acked Write 5 Done Sent Done false
20025 6189 Acked Read 6 Done Sent Done false
20026 6190 Acked Flush 0 Done Sent Done false
20027 6191 Acked Read 8 Done Sent Done false
20028 6192 Acked Read 7 Done Sent Done false
20029 6193 Acked Write 3 Done Sent Done false
20030 6194 Acked Read 10 Done Sent Done false
20031 6195 Acked Write 9 Done Sent Done false
20032 6196 Acked Read 6 Done Sent Done false
20033 6197 Acked Read 1 Done Sent Done false
20034 6198 Acked Write 9 Done Sent Done false
20035 6199 Acked Read 3 Done Sent Done false
20036 6200 Acked Read 9 Done Sent Done false
20037 6201 Acked Read 7 Done Sent Done false
20038 6202 Acked Write 2 Done Sent Done false
20039 6203 Acked Read 5 Done Sent Done false
20040 6204 Acked Write 9 Done Sent Done false
20041 6205 Acked Read 2 Done Sent Done false
20042 6206 Acked Read 8 Done Sent Done false
20043 6207 Acked Flush 0 Done Sent Done false
20044 6208 Acked Write 7 Done Sent Done false
20045 6209 Acked Write 3 Done Sent Done false
20046 6210 Acked Read 9 Done Sent Done false
20047 6211 Acked Read 4 Done Sent Done false
20048 6212 Acked Read 8 Done Sent Done false
20049 6213 Acked Read 3 Done Sent Done false
20050 6214 Acked Read 5 Done Sent Done false
20051 6215 Acked Write 1 Done Sent Done false
20052 6216 Acked Write 8 Done Sent Done false
20053 6217 Acked Write 6 Done Sent Done false
20054 6218 Acked Read 4 Done Sent Done false
20055 6219 Acked Read 6 Done Sent Done false
20056 6220 Acked Write 2 Done Sent Done false
20057 6221 Acked Read 3 Done Sent Done false
20058 6222 Acked Write 2 Done Sent Done false
20059 6223 Acked Write 2 Done Sent Done false
20060 6224 Acked Read 3 Done Sent Done false
20061 6225 Acked Read 6 Done Sent Done false
20062 6226 Acked Write 4 Done Sent Done false
20063 6227 Acked Write 2 Done Sent Done false
20064 6228 Acked Write 9 Done Sent Done false
20065 6229 Acked Write 4 Done Sent Done false
20066 6230 Acked Read 7 Done Sent Done false
20067 6231 Acked Write 10 Done Sent Done false
20068 6232 Acked Read 8 Done Sent Done false
20069 6233 Acked Read 9 Done Sent Done false
20070 6234 Acked Read 8 Done Sent Done false
20071 6235 Acked Write 1 Done Sent Done false
20072 6236 Acked Write 5 Done Sent Done false
20073 6237 Acked Read 5 Done Sent Done false
20074 6238 Acked Write 5 Done Sent Done false
20075 6239 Acked Write 6 Done Sent Done false
20076 6240 Acked Read 10 Done Sent Done false
20077 6241 Acked Read 1 Done Sent Done false
20078 6242 Acked Read 2 Done Sent Done false
20079 6243 Acked Write 2 Done Sent Done false
20080 6244 Acked Read 8 Done Sent Done false
20081 6245 Acked Flush 0 Done Sent Done false
20082 6246 Acked Read 5 Done Sent Done false
20083 6247 Acked Write 4 Done Sent Done false
20084 6248 Acked Read 10 Done Sent Done false
20085 6249 Acked Read 1 Done Sent Done false
20086 6250 Acked Read 8 Done Sent Done false
20087 6251 Acked Write 1 Done Sent Done false
20088 6252 Acked Write 10 Done Sent Done false
20089 6253 Acked Read 8 Done Sent Done false
20090 6254 Acked Write 8 Done Sent Done false
20091 6255 Acked Read 3 Done Sent Done false
20092 6256 Acked Write 2 Done Sent Done false
20093 6257 Acked Write 1 Done Sent Done false
20094 6258 Acked Write 4 Done Sent Done false
20095 6259 Acked Write 9 Done Sent Done false
20096 6260 Acked Write 9 Done Sent Done false
20097 6261 Acked Read 8 Done Sent Done false
20098 6262 Acked Read 2 Done Sent Done false
20099 6263 Acked Write 2 Done Sent Done false
20100 6264 Acked Write 10 Done Sent Done false
20101 6265 Acked Read 10 Done Sent Done false
20102 6266 Acked Write 9 Done Sent Done false
20103 6267 Acked Read 1 Done Sent Done false
20104 6268 Acked Write 7 Done Sent Done false
20105 6269 Acked Read 1 Done Sent Done false
20106 6270 Acked Read 9 Done Sent Done false
20107 6271 Acked Read 10 Done Sent Done false
20108 6272 Acked Read 7 Done Sent Done false
20109 6273 Acked Write 1 Done Sent Done false
20110 6274 Acked Write 3 Done Sent Done false
20111 6275 Acked Read 4 Done Sent Done false
20112 6276 Acked Read 2 Done Sent Done false
20113 6277 Acked Read 10 Done Sent Done false
20114 6278 Acked Read 6 Done Sent Done false
20115 6279 Acked Write 3 Done Sent Done false
20116 6280 Acked Flush 0 Done Sent Done false
20117 6281 Acked Read 1 Done Sent Done false
20118 6282 Acked Read 10 Done Sent Done false
20119 6283 Acked Write 3 Done Sent Done false
20120 6284 Acked Read 4 Done Sent Done false
20121 6285 Acked Write 5 Done Sent Done false
20122 6286 Acked Read 8 Done Sent Done false
20123 6287 Acked Write 6 Done Sent Done false
20124 6288 Acked Write 6 Done Sent Done false
20125 6289 Acked Read 3 Done Sent Done false
20126 6290 Acked Read 4 Done Sent Done false
20127 6291 Acked Read 5 Done Sent Done false
20128 6292 Acked Read 8 Done Sent Done false
20129 6293 Acked Write 4 Done Sent Done false
20130 6294 Acked Write 3 Done Sent Done false
20131 6295 Acked Write 6 Done Sent Done false
20132 6296 Acked Write 4 Done Sent Done false
20133 6297 Acked Read 9 Done Sent Done false
20134 6298 Acked Write 4 Done Sent Done false
20135 6299 Acked Read 7 Done Sent Done false
20136 6300 Acked Read 8 Done Sent Done false
20137 6301 Acked Write 10 Done Sent Done false
20138 6302 Acked Write 8 Done Sent Done false
20139 6303 Acked Read 2 Done Sent Done false
20140 6304 Acked Read 4 Done Sent Done false
20141 6305 Acked Read 4 Done Sent Done false
20142 6306 Acked Read 9 Done Sent Done false
20143 6307 Acked Read 1 Done Sent Done false
20144 6308 Acked Write 9 Done Sent Done false
20145 6309 Acked Write 9 Done Sent Done false
20146 6310 Acked Read 7 Done Sent Done false
20147 6311 Acked Read 1 Done Sent Done false
20148 6312 Acked Flush 0 Done Sent Done false
20149 6313 Acked Write 7 Done Sent Done false
20150 6314 Acked Write 7 Done Sent Done false
20151 6315 Acked Read 1 Done Sent Done false
20152 6316 Acked Flush 0 Done Sent Done false
20153 6317 Acked Write 5 Done Sent Done false
20154 6318 Acked Read 1 Done Sent Done false
20155 6319 Acked Read 4 Done Sent Done false
20156 6320 Acked Write 6 Done Sent Done false
20157 6321 Acked Read 5 Done Sent Done false
20158 6322 Acked Read 7 Done Sent Done false
20159 6323 Acked Write 8 Done Sent Done false
20160 6324 Acked Read 3 Done Sent Done false
20161 6325 Acked Write 9 Done Sent Done false
20162 6326 Acked Read 8 Done Sent Done false
20163 6327 Acked Read 9 Done Sent Done false
20164 6328 Acked Read 3 Done Sent Done false
20165 6329 Acked Read 2 Done Sent Done false
20166 6330 Acked Write 1 Done Sent Done false
20167 6331 Acked Read 8 Done Sent Done false
20168 6332 Acked Write 6 Done Sent Done false
20169 6333 Acked Write 1 Done Sent Done false
20170 6334 Acked Write 5 Done Sent Done false
20171 6335 Acked Read 10 Done Sent Done false
20172 6336 Acked Write 4 Done Sent Done false
20173 6337 Acked Read 7 Done Sent Done false
20174 6338 Acked Write 9 Done Sent Done false
20175 6339 Acked Read 2 Done Sent Done false
20176 6340 Acked Read 7 Done Sent Done false
20177 6341 Acked Read 8 Done Sent Done false
20178 6342 Acked Write 2 Done Sent Done false
20179 6343 Acked Read 5 Done Sent Done false
20180 6344 Acked Write 3 Done Sent Done false
20181 6345 Acked Write 7 Done Sent Done false
20182 6346 Acked Read 5 Done Sent Done false
20183 6347 Acked Write 7 Done Sent Done false
20184 6348 Acked Write 4 Done Sent Done false
20185 6349 Acked Write 1 Done Sent Done false
20186 6350 Acked Write 7 Done Sent Done false
20187 6351 Acked Flush 0 Done Sent Done false
20188 6352 Acked Write 2 Done Sent Done false
20189 6353 Acked Write 2 Done Sent Done false
20190 6354 Acked Read 7 Done Sent Done false
20191 6355 Acked Read 7 Done Sent Done false
20192 6356 Acked Read 2 Done Sent Done false
20193 6357 Acked Read 7 Done Sent Done false
20194 6358 Acked Write 6 Done Sent Done false
20195 6359 Acked Read 5 Done Sent Done false
20196 6360 Acked Write 1 Done Sent Done false
20197 6361 Acked Write 1 Done Sent Done false
20198 6362 Acked Flush 0 Done Sent Done false
20199 6363 Acked Write 7 Done Sent Done false
20200 6364 Acked Read 7 Done Sent Done false
20201 6365 Acked Write 3 Done Sent Done false
20202 6366 Acked Read 6 Done Sent Done false
20203 6367 Acked Read 5 Done Sent Done false
20204 6368 Acked Write 1 Done Sent Done false
20205 6369 Acked Read 4 Done Sent Done false
20206 6370 Acked Read 2 Done Sent Done false
20207 6371 Acked Read 8 Done Sent Done false
20208 6372 Acked Read 5 Done Sent Done false
20209 6373 Acked Write 8 Done Sent Done false
20210 6374 Acked Write 7 Done Sent Done false
20211 6375 Acked Write 3 Done Sent Done false
20212 6376 Acked Write 8 Done Sent Done false
20213 6377 Acked Read 7 Done Sent Done false
20214 6378 Acked Write 7 Done Sent Done false
20215 6379 Acked Read 10 Done Sent Done false
20216 6380 Acked Read 4 Done Sent Done false
20217 6381 Acked Read 8 Done Sent Done false
20218 6382 Acked Flush 0 Done Sent Done false
20219 6383 Acked Read 4 Done Sent Done false
20220 6384 Acked Flush 0 Done Sent Done false
20221 6385 Acked Write 8 Done Sent Done false
20222 6386 Acked Write 9 Done Sent Done false
20223 6387 Acked Write 4 Done Sent Done false
20224 6388 Acked Read 2 Done Sent Done false
20225 6389 Acked Read 4 Done Sent Done false
20226 6390 Acked Read 5 Done Sent Done false
20227 6391 Acked Read 2 Done Sent Done false
20228 6392 Acked Read 4 Done Sent Done false
20229 6393 Acked Read 4 Done Sent Done false
20230 6394 Acked Read 9 Done Sent Done false
20231 6395 Acked Write 8 Done Sent Done false
20232 6396 Acked Flush 0 Done Sent Done false
20233 6397 Acked Read 7 Done Sent Done false
20234 6398 Acked Write 1 Done Sent Done false
20235 6399 Acked Flush 0 Done Sent Done false
20236 6400 Acked Write 2 Done Sent Done false
20237 6401 Acked Read 9 Done Sent Done false
20238 6402 Acked Write 8 Done Sent Done false
20239 6403 Acked Write 10 Done Sent Done false
20240 6404 Acked Flush 0 Done Sent Done false
20241 6405 Acked Read 7 Done Sent Done false
20242 6406 Acked Write 4 Done Sent Done false
20243 6407 Acked Write 6 Done Sent Done false
20244 6408 Acked Write 5 Done Sent Done false
20245 6409 Acked Read 5 Done Sent Done false
20246 6410 Acked Read 5 Done Sent Done false
20247 6411 Acked Write 3 Done Sent Done false
20248 6412 Acked Read 10 Done Sent Done false
20249 6413 Acked Read 2 Done Sent Done false
20250 6414 Acked Read 4 Done Sent Done false
20251 6415 Acked Flush 0 Done Sent Done false
20252 6416 Acked Write 3 Done Sent Done false
20253 6417 Acked Read 5 Done Sent Done false
20254 6418 Acked Flush 0 Done Sent Done false
20255 6419 Acked Write 7 Done Sent Done false
20256 6420 Acked Write 2 Done Sent Done false
20257 6421 Acked Write 5 Done Sent Done false
20258 6422 Acked Read 1 Done Sent Done false
20259 6423 Acked Write 1 Done Sent Done false
20260 6424 Acked Write 9 Done Sent Done false
20261 6425 Acked Read 9 Done Sent Done false
20262 6426 Acked Write 3 Done Sent Done false
20263 6427 Acked Write 7 Done Sent Done false
20264 6428 Acked Read 3 Done Sent Done false
20265 6429 Acked Write 9 Done Sent Done false
20266 6430 Acked Write 4 Done Sent Done false
20267 6431 Acked Write 7 Done Sent Done false
20268 6432 Acked Flush 0 Done Sent Done false
20269 6433 Acked Write 9 Done Sent Done false
20270 6434 Acked Flush 0 Done Sent Done false
20271 STATES DS:0 DS:1 DS:2 TOTAL
20272 Sent 0 302 0 302
20273 Done 302 0 302 604
20274 Skipped 0 0 0 0
20275 Error 0 0 0 0
20276 Last Flush: 6434 6132 6434
20277 Downstairs last five completed: 6132 6131 6130 6129 6128
20278 Upstairs last five completed: 6434 6433 6432 6431 6430
20279 CLIENT: Up:0 ds:302 act:2
202802025-01-03T08:35:07.843ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202812025-01-03T08:35:07.843ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51763) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202822025-01-03T08:35:07.844ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202832025-01-03T08:35:07.844ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202842025-01-03T08:35:07.844ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202852025-01-03T08:35:07.844ZINFOcrucible: send last flush ID to this DS: 6132 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202862025-01-03T08:35:07.845ZINFOcrucible: Replied this last flush ID: 6132 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202872025-01-03T08:35:07.845ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202882025-01-03T08:35:07.845ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202892025-01-03T08:35:07.845ZINFOcrucible: [1] client re-new 302 jobs since flush 6132 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
202902025-01-03T08:35:07.845ZINFOcrucible: [1] Replayed 302 jobs since flush: 6132 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
20291 ----------------------------------------------------------------
20292 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20293 Downstairs last five completed: 6434 6433 6432 6431 6430
20294 Upstairs last five completed: 6434 6433 6432 6431 6430
20295 CLIENT: Up:0 ds:0 act:3
20296 Replay: All jobs finished
20297 Main task has work to do, go find it
20298 got dsc Stop(0)
20299 stop 0
20300 Jan 03 08:35:10.092 INFO request completed, latency_us: 348, response_code: 204, uri: /stop/cid/0, method: POST, req_id: ebe0fce8-c583-416c-94b6-e009c60411aa, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
20301 [8810] Got stop action so:false kr:false
20302 Jan 03 08:35:10.093 INFO request completed, latency_us: 130, response_code: 200, uri: /state/cid/0, method: GET, req_id: ef189dbd-3114-40d0-ba49-83764dd98ae0, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
203032025-01-03T08:35:10.093ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
203042025-01-03T08:35:10.093ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
203052025-01-03T08:35:10.093ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
203062025-01-03T08:35:10.093ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
203072025-01-03T08:35:10.093ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
203082025-01-03T08:35:10.093ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
20309 [8810] Exited with: ExitStatus(unix_wait_status(9))
20310 [8810][0] reports Exit
20311 Jan 03 08:35:14.095 INFO request completed, latency_us: 224, response_code: 200, uri: /state/cid/0, method: GET, req_id: e5532926-bcde-4265-a008-4f2df339bfa0, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
20312 001/300 Read block 2690 len 20480
20313 002/300 Read block 267 len 28672
20314 003/300 Read block 4896 len 36864
20315 004/300 Write block 708 len 28672 data: 4 6 7 7 6 6 6
20316 005/300 Write block 4836 len 16384 data: 7 7 7 5
20317 006/300 Write block 4062 len 40960 data: 5 5 4 4 4 7 7 6 6 6
20318 007/300 Write block 2316 len 12288 data: 5 6 5
20319 008/300 Write block 4509 len 28672 data: 4 4 3 4 4 4 4
20320 009/300 Read block 1944 len 16384
20321 010/300 Read block 1914 len 32768
20322 011/300 Write block 4762 len 16384 data: 3 3 3 6
20323 012/300 Write block 2451 len 40960 data: 3 3 5 7 7 7 6 6 6 3
20324 013/300 Flush
20325 014/300 Write block 3276 len 24576 data: 4 5 4 3 4 3
20326 015/300 Write block 3769 len 32768 data: 4 4 4 5 4 3 3 3
20327 016/300 Write block 4973 len 24576 data: 5 4 3 3 3 4
20328 017/300 Write block 2072 len 12288 data: 7 8 7
20329 018/300 Read block 1373 len 12288
20330 019/300 Read block 859 len 28672
20331 020/300 Flush
20332 021/300 Flush
20333 022/300 Read block 3506 len 32768
20334 023/300 Write block 1444 len 32768 data: 5 5 7 6 6 5 5 6
20335 024/300 Flush
20336 025/300 Read block 1495 len 24576
20337 026/300 Read block 1588 len 36864
20338 027/300 Write block 4096 len 20480 data: 4 4 2 2 2
20339 028/300 Flush
20340 029/300 Write block 1278 len 16384 data: 5 4 4 4
20341 030/300 Write block 96 len 36864 data: 8 7 6 6 6 5 4 4 4
20342 031/300 Read block 1145 len 28672
20343 032/300 Read block 1525 len 12288
20344 033/300 Read block 2535 len 40960
20345 034/300 Write block 1268 len 28672 data: 7 7 5 6 5 5 7
20346 035/300 Write block 4201 len 8192 data: 10 9
20347 036/300 Read block 307 len 40960
20348 037/300 Write block 2485 len 28672 data: 9 8 8 7 5 5 4
20349 038/300 Read block 315 len 24576
20350 039/300 Write block 3097 len 24576 data: 2 2 2 2 2 2
20351 040/300 Write block 853 len 36864 data: 4 5 4 4 5 4 4 4 4
20352 041/300 Write block 1409 len 24576 data: 5 4 4 4 3 4
20353 042/300 Read block 2122 len 12288
20354 043/300 Flush
20355 044/300 Write block 1699 len 8192 data: 2 3
20356 045/300 Write block 1818 len 32768 data: 4 4 4 4 4 4 5 5
20357 046/300 Read block 4052 len 32768
20358 047/300 Write block 1242 len 36864 data: 3 3 3 3 3 3 5 6 5
20359 048/300 Flush
20360 049/300 Read block 224 len 12288
20361 050/300 Write block 1787 len 40960 data: 4 5 4 4 4 4 5 4 4 3
20362 051/300 Read block 2638 len 28672
20363 052/300 Read block 334 len 4096
20364 053/300 Read block 1527 len 32768
20365 054/300 Write block 3235 len 36864 data: 9 9 9 7 7 6 6 6 5
20366 055/300 Read block 4278 len 36864
20367 056/300 Flush
20368 057/300 Write block 3694 len 36864 data: 4 5 5 4 4 4 4 5 4
20369 058/300 Write block 3194 len 12288 data: 5 5 6
20370 059/300 Read block 3420 len 40960
20371 060/300 Read block 2479 len 36864
20372 061/300 Write block 2711 len 12288 data: 7 6 6
20373 062/300 Read block 1519 len 20480
20374 063/300 Write block 4648 len 8192 data: 6 6
20375 064/300 Write block 175 len 16384 data: 5 4 3 3
20376 065/300 Flush
20377 066/300 Flush
20378 067/300 Read block 1319 len 40960
20379 068/300 Read block 3373 len 28672
20380 069/300 Write block 3969 len 4096 data: 3
20381 070/300 Write block 1760 len 12288 data: 2 2 2
20382 071/300 Write block 1823 len 16384 data: 5 6 6 7
20383 072/300 Read block 2728 len 36864
20384 073/300 Read block 1618 len 36864
20385 074/300 Write block 123 len 40960 data: 2 2 2 3 3 2 2 3 3 3
20386 075/300 Flush
20387 076/300 Read block 4874 len 20480
20388 077/300 Read block 2798 len 16384
20389 078/300 Write block 3809 len 28672 data: 3 3 2 2 2 3 3
20390 079/300 Flush
20391 080/300 Flush
20392 081/300 Read block 944 len 16384
20393 082/300 Flush
20394 083/300 Read block 3808 len 28672
20395 084/300 Read block 840 len 24576
20396 085/300 Read block 3147 len 4096
20397 086/300 Read block 1926 len 40960
20398 087/300 Read block 1461 len 32768
20399 088/300 Write block 3672 len 28672 data: 6 6 6 6 5 3 2
20400 089/300 Read block 1107 len 36864
20401 090/300 Write block 2227 len 16384 data: 5 5 6 3
20402 091/300 Flush
20403 092/300 Read block 3700 len 24576
20404 093/300 Read block 1171 len 16384
20405 094/300 Write block 3354 len 4096 data: 5
20406 095/300 Read block 517 len 16384
20407 096/300 Read block 860 len 24576
20408 097/300 Write block 4067 len 4096 data: 8
20409 098/300 Write block 2495 len 16384 data: 4 4 2 3
20410 099/300 Read block 2157 len 4096
20411 100/300 Write block 1485 len 12288 data: 5 5 5
20412 101/300 Read block 3392 len 24576
20413 102/300 Read block 3742 len 20480
20414 103/300 Read block 4745 len 20480
20415 104/300 Read block 4156 len 8192
20416 105/300 Read block 994 len 28672
20417 106/300 Read block 1127 len 36864
20418 107/300 Read block 2983 len 16384
20419 108/300 Flush
20420 109/300 Flush
20421 110/300 Read block 448 len 32768
20422 111/300 Write block 2946 len 16384 data: 5 5 4 4
20423 112/300 Read block 2371 len 32768
20424 113/300 Read block 348 len 32768
20425 114/300 Read block 3721 len 24576
20426 115/300 Read block 807 len 4096
20427 116/300 Write block 3668 len 12288 data: 4 3 2
20428 117/300 Flush
20429 118/300 Flush
20430 119/300 Read block 1099 len 16384
20431 120/300 Read block 3464 len 20480
20432 121/300 Read block 3478 len 20480
20433 122/300 Write block 3507 len 20480 data: 3 3 3 3 5
20434 123/300 Read block 4315 len 40960
20435 124/300 Write block 2835 len 12288 data: 5 5 5
20436 125/300 Write block 3653 len 12288 data: 5 4 4
20437 126/300 Flush
20438 127/300 Write block 2457 len 24576 data: 7 7 7 4 3 3
20439 128/300 Write block 497 len 4096 data: 6
20440 129/300 Write block 4323 len 16384 data: 5 5 4 4
20441 130/300 Write block 2963 len 16384 data: 3 3 3 2
20442 131/300 Read block 4022 len 40960
20443 132/300 Write block 2212 len 12288 data: 3 4 6
20444 133/300 Read block 1724 len 32768
20445 134/300 Write block 4340 len 32768 data: 6 4 5 4 5 6 5 5
20446 135/300 Write block 4862 len 12288 data: 6 7 5
20447 136/300 Read block 3575 len 16384
20448 137/300 Read block 3702 len 4096
20449 138/300 Write block 4108 len 32768 data: 3 3 3 3 3 3 4 4
20450 139/300 Flush
20451 140/300 Read block 4611 len 8192
20452 141/300 Read block 4216 len 40960
20453 142/300 Read block 4841 len 8192
20454 143/300 Write block 2126 len 24576 data: 4 4 4 4 5 4
20455 144/300 Write block 4612 len 36864 data: 4 3 3 4 4 4 5 5 6
20456 145/300 Write block 2561 len 28672 data: 4 4 6 6 6 4 5
20457 146/300 Read block 3091 len 32768
20458 147/300 Read block 4192 len 28672
20459 148/300 Flush
20460 149/300 Read block 816 len 20480
20461 150/300 Read block 1210 len 16384
20462 151/300 Read block 2337 len 4096
20463 152/300 Read block 1663 len 16384
20464 153/300 Read block 2572 len 36864
20465 154/300 Read block 1629 len 24576
20466 155/300 Read block 2754 len 16384
20467 156/300 Read block 161 len 36864
20468 157/300 Write block 3525 len 20480 data: 2 2 2 3 3
20469 158/300 Flush
20470 159/300 Write block 2449 len 4096 data: 2
20471 160/300 Write block 4066 len 4096 data: 5
20472 161/300 Write block 1465 len 16384 data: 5 5 4 4
20473 162/300 Write block 1483 len 32768 data: 5 5 6 6 6 5 5 5
20474 163/300 Read block 1359 len 40960
20475 164/300 Write block 4335 len 20480 data: 6 6 6 5 5
20476 165/300 Read block 2346 len 24576
20477 166/300 Flush
20478 167/300 Write block 3309 len 4096 data: 5
20479 168/300 Read block 4120 len 20480
20480 169/300 Write block 917 len 16384 data: 5 7 7 8
20481 170/300 Write block 396 len 12288 data: 3 3 3
20482 171/300 Write block 4979 len 24576 data: 3 4 5 4 4 4
20483 172/300 Write block 2757 len 28672 data: 6 6 6 6 6 6 6
20484 173/300 Read block 2665 len 16384
20485 174/300 Read block 3119 len 8192
20486 175/300 Write block 3018 len 20480 data: 5 5 5 5 5
20487 176/300 Read block 1649 len 40960
20488 177/300 Read block 3972 len 16384
20489 178/300 Write block 4398 len 12288 data: 3 3 3
20490 179/300 Read block 4400 len 4096
20491 180/300 Read block 1712 len 40960
20492 181/300 Read block 1030 len 24576
20493 182/300 Read block 702 len 24576
20494 183/300 Write block 772 len 40960 data: 3 3 4 4 4 4 4 5 5 4
20495 184/300 Read block 1121 len 12288
20496 185/300 Write block 3994 len 8192 data: 3 3
20497 186/300 Write block 878 len 12288 data: 4 4 4
20498 187/300 Read block 4021 len 24576
20499 188/300 Write block 403 len 24576 data: 4 3 5 6 7 7
20500 189/300 Write block 2909 len 36864 data: 5 4 3 3 3 3 4 4 4
20501 190/300 Read block 4695 len 16384
20502 191/300 Write block 4865 len 40960 data: 6 6 6 7 8 7 7 6 4 5
20503 192/300 Write block 4128 len 24576 data: 4 4 3 3 3 3
20504 193/300 Flush
20505 194/300 Write block 1817 len 24576 data: 2 5 5 5 5 5
20506 195/300 Read block 3308 len 40960
20507 196/300 Write block 3508 len 24576 data: 4 4 4 6 4 4
20508 197/300 Read block 3933 len 40960
20509 198/300 Write block 3511 len 24576 data: 7 5 5 4 5 5
20510 199/300 Flush
20511 200/300 Write block 3543 len 4096 data: 6
20512 201/300 Read block 49 len 16384
20513 202/300 Read block 2225 len 28672
20514 203/300 Write block 3759 len 16384 data: 4 4 5 5
20515 204/300 Read block 984 len 12288
20516 205/300 Flush
20517 206/300 Read block 608 len 32768
20518 207/300 Read block 2825 len 12288
20519 208/300 Read block 3925 len 28672
20520 209/300 Write block 2252 len 40960 data: 5 6 6 6 6 4 3 3 2 2
20521 210/300 Read block 4201 len 4096
20522 211/300 Read block 3053 len 24576
20523 212/300 Read block 3259 len 16384
20524 213/300 Write block 4492 len 36864 data: 4 4 4 6 6 7 7 6 5
20525 214/300 Write block 3233 len 8192 data: 8 9
20526 215/300 Read block 1083 len 4096
20527 216/300 Write block 3159 len 12288 data: 3 4 4
20528 217/300 Write block 2487 len 28672 data: 9 8 6 6 5 5 4
20529 218/300 Write block 3827 len 24576 data: 5 5 4 5 4 6
20530 219/300 Write block 2027 len 4096 data: 3
20531 220/300 Read block 3284 len 16384
20532 221/300 Write block 3235 len 4096 data: 10
20533 222/300 Write block 2508 len 8192 data: 4 4
20534 223/300 Read block 544 len 8192
20535 224/300 Read block 4962 len 12288
20536 225/300 Write block 873 len 40960 data: 5 4 4 5 5 5 5 5 4 4
20537 226/300 Write block 78 len 12288 data: 4 3 3
20538 227/300 Read block 4050 len 16384
20539 228/300 Read block 3515 len 12288
20540 229/300 Read block 1274 len 32768
20541 230/300 Flush
20542 231/300 Flush
20543 232/300 Read block 1469 len 16384
20544 233/300 Read block 1214 len 28672
20545 234/300 Read block 2706 len 16384
20546 235/300 Write block 1567 len 40960 data: 3 3 4 4 4 5 6 5 5 5
20547 236/300 Read block 4264 len 8192
20548 237/300 Read block 4051 len 24576
20549 238/300 Write block 405 len 16384 data: 6 7 8 8
20550 239/300 Read block 4661 len 36864
20551 240/300 Read block 1298 len 12288
20552 241/300 Write block 4500 len 20480 data: 6 5 5 6 6
20553 242/300 Flush
20554 243/300 Read block 2561 len 12288
20555 244/300 Read block 2817 len 4096
20556 245/300 Read block 1099 len 12288
20557 246/300 Flush
20558 247/300 Write block 4883 len 24576 data: 6 7 7 8 8 8
20559 248/300 Read block 2083 len 12288
20560 249/300 Read block 491 len 36864
20561 250/300 Write block 1710 len 32768 data: 4 4 4 5 5 4 3 3
20562 251/300 Write block 3266 len 20480 data: 2 3 4 4 6
20563 252/300 Read block 4171 len 16384
20564 253/300 Read block 629 len 12288
20565 254/300 Write block 2298 len 32768 data: 5 5 6 6 6 6 7 6
20566 255/300 Read block 1960 len 32768
20567 256/300 Read block 4494 len 36864
20568 257/300 Write block 3850 len 36864 data: 5 5 7 8 7 8 6 5 4
20569 258/300 Write block 2365 len 16384 data: 5 5 7 6
20570 259/300 Flush
20571 260/300 Write block 3607 len 16384 data: 6 5 2 3
20572 261/300 Read block 1080 len 16384
20573 262/300 Read block 2194 len 40960
20574 263/300 Read block 1882 len 8192
20575 264/300 Read block 2782 len 40960
20576 265/300 Read block 4041 len 4096
20577 266/300 Read block 3945 len 8192
20578 267/300 Write block 3937 len 20480 data: 3 4 4 4 3
20579 268/300 Read block 2537 len 24576
20580 269/300 Read block 569 len 36864
20581 270/300 Write block 3747 len 24576 data: 4 3 3 3 2 2
20582 271/300 Read block 3531 len 32768
20583 272/300 Read block 2577 len 20480
20584 273/300 Write block 228 len 12288 data: 3 4 4
20585 274/300 Read block 3871 len 28672
20586 275/300 Read block 1090 len 4096
20587 276/300 Read block 4259 len 4096
20588 277/300 Write block 83 len 20480 data: 4 4 4 4 4
20589 278/300 Write block 2665 len 20480 data: 6 6 6 4 3
20590 279/300 Read block 4960 len 36864
20591 280/300 Flush
20592 281/300 Write block 4542 len 40960 data: 4 4 4 3 3 3 3 3 3 4
20593 282/300 Read block 2556 len 36864
20594 283/300 Read block 1847 len 40960
20595 284/300 Write block 4027 len 4096 data: 3
20596 285/300 Read block 3688 len 40960
20597 286/300 Write block 4270 len 16384 data: 5 3 2 2
20598 287/300 Read block 527 len 28672
20599 288/300 Write block 2380 len 20480 data: 3 3 4 5 5
20600 289/300 Write block 3758 len 40960 data: 4 5 5 6 6 4 3 2 2 3
20601 290/300 Read block 35 len 20480
20602 291/300 Write block 3100 len 32768 data: 3 3 3 4 4 4 2 2
20603 292/300 Read block 2981 len 12288
20604 293/300 Read block 4163 len 24576
20605 294/300 Flush
20606 295/300 Flush
20607 296/300 Flush
20608 297/300 Read block 3469 len 24576
20609 298/300 Write block 3320 len 8192 data: 2 2
20610 299/300 Read block 847 len 28672
20611 300/300 Read block 1539 len 12288
20612 301/300 Write block 3370 len 4096 data: 5
20613 Main task has work to do, go find it
20614 got dsc Start(0)
20615 start 0
20616 Jan 03 08:35:14.341 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/0, method: POST, req_id: 235689b0-0030-4b4a-a6a1-4c1500af0db1, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
20617 Replay: started 0, returned:Ok(())
20618 ----------------------------------------------------------------
20619 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20620 States: Offline Active Active
20621 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20622 6435 Acked Read 5 Sent Done Done false
20623 6436 Acked Read 7 Sent Done Done false
20624 6437 Acked Read 9 Sent Done Done false
20625 6438 Acked Write 7 Sent Done Done false
20626 6439 Acked Write 4 Sent Done Done false
20627 6440 Acked Write 10 Sent Done Done false
20628 6441 Acked Write 3 Sent Done Done false
20629 6442 Acked Write 7 Sent Done Done false
20630 6443 Acked Read 4 Sent Done Done false
20631 6444 Acked Read 8 Sent Done Done false
20632 6445 Acked Write 4 Sent Done Done false
20633 6446 Acked Write 10 Sent Done Done false
20634 6447 Acked Flush 0 Sent Done Done false
20635 6448 Acked Write 6 Sent Done Done false
20636 6449 Acked Write 8 Sent Done Done false
20637 6450 Acked Write 6 Sent Done Done false
20638 6451 Acked Write 3 Sent Done Done false
20639 6452 Acked Read 3 Sent Done Done false
20640 6453 Acked Read 7 Sent Done Done false
20641 6454 Acked Flush 0 Sent Done Done false
20642 6455 Acked Flush 0 Sent Done Done false
20643 6456 Acked Read 8 Sent Done Done false
20644 6457 Acked Write 8 Sent Done Done false
20645 6458 Acked Flush 0 Sent Done Done false
20646 6459 Acked Read 6 Sent Done Done false
20647 6460 Acked Read 9 Sent Done Done false
20648 6461 Acked Write 5 Sent Done Done false
20649 6462 Acked Flush 0 Sent Done Done false
20650 6463 Acked Write 4 Sent Done Done false
20651 6464 Acked Write 9 Sent Done Done false
20652 6465 Acked Read 7 Sent Done Done false
20653 6466 Acked Read 3 Sent Done Done false
20654 6467 Acked Read 10 Sent Done Done false
20655 6468 Acked Write 7 Sent Done Done false
20656 6469 Acked Write 2 Sent Done Done false
20657 6470 Acked Read 10 Sent Done Done false
20658 6471 Acked Write 7 Sent Done Done false
20659 6472 Acked Read 6 Sent Done Done false
20660 6473 Acked Write 6 Sent Done Done false
20661 6474 Acked Write 9 Sent Done Done false
20662 6475 Acked Write 6 Sent Done Done false
20663 6476 Acked Read 3 Sent Done Done false
20664 6477 Acked Flush 0 Sent Done Done false
20665 6478 Acked Write 2 Sent Done Done false
20666 6479 Acked Write 8 Sent Done Done false
20667 6480 Acked Read 8 Sent Done Done false
20668 6481 Acked Write 9 Sent Done Done false
20669 6482 Acked Flush 0 Sent Done Done false
20670 6483 Acked Read 3 Sent Done Done false
20671 6484 Acked Write 10 Sent Done Done false
20672 6485 Acked Read 7 Sent Done Done false
20673 6486 Acked Read 1 Sent Done Done false
20674 6487 Acked Read 8 Sent Done Done false
20675 6488 Acked Write 9 Sent Done Done false
20676 6489 Acked Read 9 Sent Done Done false
20677 6490 Acked Flush 0 Sent Done Done false
20678 6491 Acked Write 9 Sent Done Done false
20679 6492 Acked Write 3 Sent Done Done false
20680 6493 Acked Read 10 Sent Done Done false
20681 6494 Acked Read 9 Sent Done Done false
20682 6495 Acked Write 3 Sent Done Done false
20683 6496 Acked Read 5 Sent Done Done false
20684 6497 Acked Write 2 Sent Done Done false
20685 6498 Acked Write 4 Sent Done Done false
20686 6499 Acked Flush 0 Sent Done Done false
20687 6500 Acked Flush 0 Sent Done Done false
20688 6501 Acked Read 10 Sent Done Done false
20689 6502 Acked Read 7 Sent Done Done false
20690 6503 Acked Write 1 Sent Done Done false
20691 6504 Acked Write 3 Sent Done Done false
20692 6505 Acked Write 4 Sent Done Done false
20693 6506 Acked Read 9 Sent Done Done false
20694 6507 Acked Read 9 Sent Done Done false
20695 6508 Acked Write 10 Sent Done Done false
20696 6509 Acked Flush 0 Sent Done Done false
20697 6510 Acked Read 5 Sent Done Done false
20698 6511 Acked Read 4 Sent Done Done false
20699 6512 Acked Write 7 Sent Done Done false
20700 6513 Acked Flush 0 Sent Done Done false
20701 6514 Acked Flush 0 Sent Done Done false
20702 6515 Acked Read 4 Sent Done Done false
20703 6516 Acked Flush 0 Sent Done Done false
20704 6517 Acked Read 7 Sent Done Done false
20705 6518 Acked Read 6 Sent Done Done false
20706 6519 Acked Read 1 Sent Done Done false
20707 6520 Acked Read 10 Sent Done Done false
20708 6521 Acked Read 8 Sent Done Done false
20709 6522 Acked Write 7 Sent Done Done false
20710 6523 Acked Read 9 Sent Done Done false
20711 6524 Acked Write 4 Sent Done Done false
20712 6525 Acked Flush 0 Sent Done Done false
20713 6526 Acked Read 6 Sent Done Done false
20714 6527 Acked Read 4 Sent Done Done false
20715 6528 Acked Write 1 Sent Done Done false
20716 6529 Acked Read 4 Sent Done Done false
20717 6530 Acked Read 6 Sent Done Done false
20718 6531 Acked Write 1 Sent Done Done false
20719 6532 Acked Write 4 Sent Done Done false
20720 6533 Acked Read 1 Sent Done Done false
20721 6534 Acked Write 3 Sent Done Done false
20722 6535 Acked Read 6 Sent Done Done false
20723 6536 Acked Read 5 Sent Done Done false
20724 6537 Acked Read 5 Sent Done Done false
20725 6538 Acked Read 2 Sent Done Done false
20726 6539 Acked Read 7 Sent Done Done false
20727 6540 Acked Read 9 Sent Done Done false
20728 6541 Acked Read 4 Sent Done Done false
20729 6542 Acked Flush 0 Sent Done Done false
20730 6543 Acked Flush 0 Sent Done Done false
20731 6544 Acked Read 8 Sent Done Done false
20732 6545 Acked Write 4 Sent Done Done false
20733 6546 Acked Read 8 Sent Done Done false
20734 6547 Acked Read 8 Sent Done Done false
20735 6548 Acked Read 6 Sent Done Done false
20736 6549 Acked Read 1 Sent Done Done false
20737 6550 Acked Write 3 Sent Done Done false
20738 6551 Acked Flush 0 Sent Done Done false
20739 6552 Acked Flush 0 Sent Done Done false
20740 6553 Acked Read 4 Sent Done Done false
20741 6554 Acked Read 5 Sent Done Done false
20742 6555 Acked Read 5 Sent Done Done false
20743 6556 Acked Write 5 Sent Done Done false
20744 6557 Acked Read 10 Sent Done Done false
20745 6558 Acked Write 3 Sent Done Done false
20746 6559 Acked Write 3 Sent Done Done false
20747 6560 Acked Flush 0 Sent Done Done false
20748 6561 Acked Write 6 Sent Done Done false
20749 6562 Acked Write 1 Sent Done Done false
20750 6563 Acked Write 4 Sent Done Done false
20751 6564 Acked Write 4 Sent Done Done false
20752 6565 Acked Read 10 Sent Done Done false
20753 6566 Acked Write 3 Sent Done Done false
20754 6567 Acked Read 8 Sent Done Done false
20755 6568 Acked Write 8 Sent Done Done false
20756 6569 Acked Write 3 Sent Done Done false
20757 6570 Acked Read 4 Sent Done Done false
20758 6571 Acked Read 1 Sent Done Done false
20759 6572 Acked Write 8 Sent Done Done false
20760 6573 Acked Flush 0 Sent Done Done false
20761 6574 Acked Read 2 Sent Done Done false
20762 6575 Acked Read 10 Sent Done Done false
20763 6576 Acked Read 2 Sent Done Done false
20764 6577 Acked Write 6 Sent Done Done false
20765 6578 Acked Write 9 Sent Done Done false
20766 6579 Acked Write 7 Sent Done Done false
20767 6580 Acked Read 8 Sent Done Done false
20768 6581 Acked Read 7 Sent Done Done false
20769 6582 Acked Flush 0 Sent Done Done false
20770 6583 Acked Read 5 Sent Done Done false
20771 6584 Acked Read 4 Sent Done Done false
20772 6585 Acked Read 1 Sent Done Done false
20773 6586 Acked Read 4 Sent Done Done false
20774 6587 Acked Read 9 Sent Done Done false
20775 6588 Acked Read 6 Sent Done Done false
20776 6589 Acked Read 4 Sent Done Done false
20777 6590 Acked Read 9 Sent Done Done false
20778 6591 Acked Write 5 Sent Done Done false
20779 6592 Acked Flush 0 Sent Done Done false
20780 6593 Acked Write 1 Sent Done Done false
20781 6594 Acked Write 1 Sent Done Done false
20782 6595 Acked Write 4 Sent Done Done false
20783 6596 Acked Write 8 Sent Done Done false
20784 6597 Acked Read 10 Sent Done Done false
20785 6598 Acked Write 5 Sent Done Done false
20786 6599 Acked Read 6 Sent Done Done false
20787 6600 Acked Flush 0 Sent Done Done false
20788 6601 Acked Write 1 Sent Done Done false
20789 6602 Acked Read 5 Sent Done Done false
20790 6603 Acked Write 4 Sent Done Done false
20791 6604 Acked Write 3 Sent Done Done false
20792 6605 Acked Write 6 Sent Done Done false
20793 6606 Acked Write 7 Sent Done Done false
20794 6607 Acked Read 4 Sent Done Done false
20795 6608 Acked Read 2 Sent Done Done false
20796 6609 Acked Write 5 Sent Done Done false
20797 6610 Acked Read 10 Sent Done Done false
20798 6611 Acked Read 4 Sent Done Done false
20799 6612 Acked Write 3 Sent Done Done false
20800 6613 Acked Read 1 Sent Done Done false
20801 6614 Acked Read 10 Sent Done Done false
20802 6615 Acked Read 6 Sent Done Done false
20803 6616 Acked Read 6 Sent Done Done false
20804 6617 Acked Write 10 Sent Done Done false
20805 6618 Acked Read 3 Sent Done Done false
20806 6619 Acked Write 2 Sent Done Done false
20807 6620 Acked Write 3 Sent Done Done false
20808 6621 Acked Read 6 Sent Done Done false
20809 6622 Acked Write 6 Sent Done Done false
20810 6623 Acked Write 9 Sent Done Done false
20811 6624 Acked Read 4 Sent Done Done false
20812 6625 Acked Write 10 Sent Done Done false
20813 6626 Acked Write 6 Sent Done Done false
20814 6627 Acked Flush 0 Sent Done Done false
20815 6628 Acked Write 6 Sent Done Done false
20816 6629 Acked Read 10 Sent Done Done false
20817 6630 Acked Write 6 Sent Done Done false
20818 6631 Acked Read 10 Sent Done Done false
20819 6632 Acked Write 6 Sent Done Done false
20820 6633 Acked Flush 0 Sent Done Done false
20821 6634 Acked Write 1 Sent Done Done false
20822 6635 Acked Read 4 Sent Done Done false
20823 6636 Acked Read 7 Sent Done Done false
20824 6637 Acked Write 4 Sent Done Done false
20825 6638 Acked Read 3 Sent Done Done false
20826 6639 Acked Flush 0 Sent Done Done false
20827 6640 Acked Read 8 Sent Done Done false
20828 6641 Acked Read 3 Sent Done Done false
20829 6642 Acked Read 7 Sent Done Done false
20830 6643 Acked Write 10 Sent Done Done false
20831 6644 Acked Read 1 Sent Done Done false
20832 6645 Acked Read 6 Sent Done Done false
20833 6646 Acked Read 4 Sent Done Done false
20834 6647 Acked Write 9 Sent Done Done false
20835 6648 Acked Write 2 Sent Done Done false
20836 6649 Acked Read 1 Sent Done Done false
20837 6650 Acked Write 3 Sent Done Done false
20838 6651 Acked Write 7 Sent Done Done false
20839 6652 Acked Write 6 Sent Done Done false
20840 6653 Acked Write 1 Sent Done Done false
20841 6654 Acked Read 4 Sent Done Done false
20842 6655 Acked Write 1 Sent Done Done false
20843 6656 Acked Write 2 Sent Done Done false
20844 6657 Acked Read 2 Sent Done Done false
20845 6658 Acked Read 3 Sent Done Done false
20846 6659 Acked Write 10 Sent Done Done false
20847 6660 Acked Write 3 Sent Done Done false
20848 6661 Acked Read 4 Sent Done Done false
20849 6662 Acked Read 3 Sent Done Done false
20850 6663 Acked Read 8 Sent Done Done false
20851 6664 Acked Flush 0 Sent Done Done false
20852 6665 Acked Flush 0 Sent Done Done false
20853 6666 Acked Read 4 Sent Done Done false
20854 6667 Acked Read 7 Sent Done Done false
20855 6668 Acked Read 4 Sent Done Done false
20856 6669 Acked Write 10 Sent Done Done false
20857 6670 Acked Read 2 Sent Done Done false
20858 6671 Acked Read 6 Sent Done Done false
20859 6672 Acked Write 4 Sent Done Done false
20860 6673 Acked Read 9 Sent Done Done false
20861 6674 Acked Read 3 Sent Done Done false
20862 6675 Acked Write 5 Sent Done Done false
20863 6676 Acked Flush 0 Sent Done Done false
20864 6677 Acked Read 3 Sent Done Done false
20865 6678 Acked Read 1 Sent Done Done false
20866 6679 Acked Read 3 Sent Done Done false
20867 6680 Acked Flush 0 Sent Done Done false
20868 6681 Acked Write 6 Sent Done Done false
20869 6682 Acked Read 3 Sent Done Done false
20870 6683 Acked Read 9 Sent Done Done false
20871 6684 Acked Write 8 Sent Done Done false
20872 6685 Acked Write 5 Sent Done Done false
20873 6686 Acked Read 4 Sent Done Done false
20874 6687 Acked Read 3 Sent Done Done false
20875 6688 Acked Write 8 Sent Done Done false
20876 6689 Acked Read 8 Sent Done Done false
20877 6690 Acked Read 9 Sent Done Done false
20878 6691 Acked Write 9 Sent Done Done false
20879 6692 Acked Write 4 Sent Done Done false
20880 6693 Acked Flush 0 Sent Done Done false
20881 6694 Acked Write 4 Sent Done Done false
20882 6695 Acked Read 4 Sent Done Done false
20883 6696 Acked Read 10 Sent Done Done false
20884 6697 Acked Read 2 Sent Done Done false
20885 6698 Acked Read 10 Sent Done Done false
20886 6699 Acked Read 1 Sent Done Done false
20887 6700 Acked Read 2 Sent Done Done false
20888 6701 Acked Write 5 Sent Done Done false
20889 6702 Acked Read 6 Sent Done Done false
20890 6703 Acked Read 9 Sent Done Done false
20891 6704 Acked Write 6 Sent Done Done false
20892 6705 Acked Read 8 Sent Done Done false
20893 6706 Acked Read 5 Sent Done Done false
20894 6707 Acked Write 3 Sent Done Done false
20895 6708 Acked Read 7 Sent Done Done false
20896 6709 Acked Read 1 Sent Done Done false
20897 6710 Acked Read 1 Sent Done Done false
20898 6711 Acked Write 5 Sent Done Done false
20899 6712 Acked Write 5 Sent Done Done false
20900 6713 Acked Read 9 Sent Done Done false
20901 6714 Acked Flush 0 Sent Done Done false
20902 6715 Acked Write 10 Sent Done Done false
20903 6716 Acked Read 9 Sent Done Done false
20904 6717 Acked Read 10 Sent Done Done false
20905 6718 Acked Write 1 Sent Done Done false
20906 6719 Acked Read 10 Sent Done Done false
20907 6720 Acked Write 4 Sent Done Done false
20908 6721 Acked Read 7 Sent Done Done false
20909 6722 Acked Write 5 Sent Done Done false
20910 6723 Acked Write 10 Sent Done Done false
20911 6724 Acked Read 5 Sent Done Done false
20912 6725 Acked Write 8 Sent Done Done false
20913 6726 Acked Read 3 Sent Done Done false
20914 6727 Acked Read 6 Sent Done Done false
20915 6728 Acked Flush 0 Sent Done Done false
20916 6729 Acked Flush 0 Sent Done Done false
20917 6730 Acked Flush 0 Sent Done Done false
20918 6731 Acked Read 6 Sent Done Done false
20919 6732 Acked Write 2 Sent Done Done false
20920 6733 Acked Read 7 Sent Done Done false
20921 6734 Acked Read 3 Sent Done Done false
20922 6735 Acked Write 1 Sent Done Done false
20923 STATES DS:0 DS:1 DS:2 TOTAL
20924 Sent 301 0 0 301
20925 Done 0 301 301 602
20926 Skipped 0 0 0 0
20927 Error 0 0 0 0
20928 Last Flush: 6434 6730 6730
20929 Downstairs last five completed: 6434 6433 6432 6431 6430
20930 Upstairs last five completed: 6735 6734 6733 6732 6731
20931 CLIENT: Up:0 ds:301 act:2
20932 [8810] got start action
20933 [8810] I am going to restart
20934 Starting downstairs at port 8810
20935 Make output file at "/tmp/dsc/downstairs-8810.txt"
20936 [8810][0] reports Starting
20937 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1224)
20938 [8810][0] reports Running
20939 ----------------------------------------------------------------
20940 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20941 States: Offline Active Active
20942 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20943 6435 Acked Read 5 Sent Done Done false
20944 6436 Acked Read 7 Sent Done Done false
20945 6437 Acked Read 9 Sent Done Done false
20946 6438 Acked Write 7 Sent Done Done false
20947 6439 Acked Write 4 Sent Done Done false
20948 6440 Acked Write 10 Sent Done Done false
20949 6441 Acked Write 3 Sent Done Done false
20950 6442 Acked Write 7 Sent Done Done false
20951 6443 Acked Read 4 Sent Done Done false
20952 6444 Acked Read 8 Sent Done Done false
20953 6445 Acked Write 4 Sent Done Done false
20954 6446 Acked Write 10 Sent Done Done false
20955 6447 Acked Flush 0 Sent Done Done false
20956 6448 Acked Write 6 Sent Done Done false
20957 6449 Acked Write 8 Sent Done Done false
20958 6450 Acked Write 6 Sent Done Done false
20959 6451 Acked Write 3 Sent Done Done false
20960 6452 Acked Read 3 Sent Done Done false
20961 6453 Acked Read 7 Sent Done Done false
20962 6454 Acked Flush 0 Sent Done Done false
20963 6455 Acked Flush 0 Sent Done Done false
20964 6456 Acked Read 8 Sent Done Done false
20965 6457 Acked Write 8 Sent Done Done false
20966 6458 Acked Flush 0 Sent Done Done false
20967 6459 Acked Read 6 Sent Done Done false
20968 6460 Acked Read 9 Sent Done Done false
20969 6461 Acked Write 5 Sent Done Done false
20970 6462 Acked Flush 0 Sent Done Done false
20971 6463 Acked Write 4 Sent Done Done false
20972 6464 Acked Write 9 Sent Done Done false
20973 6465 Acked Read 7 Sent Done Done false
20974 6466 Acked Read 3 Sent Done Done false
20975 6467 Acked Read 10 Sent Done Done false
20976 6468 Acked Write 7 Sent Done Done false
20977 6469 Acked Write 2 Sent Done Done false
20978 6470 Acked Read 10 Sent Done Done false
20979 6471 Acked Write 7 Sent Done Done false
20980 6472 Acked Read 6 Sent Done Done false
20981 6473 Acked Write 6 Sent Done Done false
20982 6474 Acked Write 9 Sent Done Done false
20983 6475 Acked Write 6 Sent Done Done false
20984 6476 Acked Read 3 Sent Done Done false
20985 6477 Acked Flush 0 Sent Done Done false
20986 6478 Acked Write 2 Sent Done Done false
20987 6479 Acked Write 8 Sent Done Done false
20988 6480 Acked Read 8 Sent Done Done false
20989 6481 Acked Write 9 Sent Done Done false
20990 6482 Acked Flush 0 Sent Done Done false
20991 6483 Acked Read 3 Sent Done Done false
20992 6484 Acked Write 10 Sent Done Done false
20993 6485 Acked Read 7 Sent Done Done false
20994 6486 Acked Read 1 Sent Done Done false
20995 6487 Acked Read 8 Sent Done Done false
20996 6488 Acked Write 9 Sent Done Done false
20997 6489 Acked Read 9 Sent Done Done false
20998 6490 Acked Flush 0 Sent Done Done false
20999 6491 Acked Write 9 Sent Done Done false
21000 6492 Acked Write 3 Sent Done Done false
21001 6493 Acked Read 10 Sent Done Done false
21002 6494 Acked Read 9 Sent Done Done false
21003 6495 Acked Write 3 Sent Done Done false
21004 6496 Acked Read 5 Sent Done Done false
21005 6497 Acked Write 2 Sent Done Done false
21006 6498 Acked Write 4 Sent Done Done false
21007 6499 Acked Flush 0 Sent Done Done false
21008 6500 Acked Flush 0 Sent Done Done false
21009 6501 Acked Read 10 Sent Done Done false
21010 6502 Acked Read 7 Sent Done Done false
21011 6503 Acked Write 1 Sent Done Done false
21012 6504 Acked Write 3 Sent Done Done false
21013 6505 Acked Write 4 Sent Done Done false
21014 6506 Acked Read 9 Sent Done Done false
21015 6507 Acked Read 9 Sent Done Done false
21016 6508 Acked Write 10 Sent Done Done false
21017 6509 Acked Flush 0 Sent Done Done false
21018 6510 Acked Read 5 Sent Done Done false
21019 6511 Acked Read 4 Sent Done Done false
21020 6512 Acked Write 7 Sent Done Done false
21021 6513 Acked Flush 0 Sent Done Done false
21022 6514 Acked Flush 0 Sent Done Done false
21023 6515 Acked Read 4 Sent Done Done false
21024 6516 Acked Flush 0 Sent Done Done false
21025 6517 Acked Read 7 Sent Done Done false
21026 6518 Acked Read 6 Sent Done Done false
21027 6519 Acked Read 1 Sent Done Done false
21028 6520 Acked Read 10 Sent Done Done false
21029 6521 Acked Read 8 Sent Done Done false
21030 6522 Acked Write 7 Sent Done Done false
21031 6523 Acked Read 9 Sent Done Done false
21032 6524 Acked Write 4 Sent Done Done false
21033 6525 Acked Flush 0 Sent Done Done false
21034 6526 Acked Read 6 Sent Done Done false
21035 6527 Acked Read 4 Sent Done Done false
21036 6528 Acked Write 1 Sent Done Done false
21037 6529 Acked Read 4 Sent Done Done false
21038 6530 Acked Read 6 Sent Done Done false
21039 6531 Acked Write 1 Sent Done Done false
21040 6532 Acked Write 4 Sent Done Done false
21041 6533 Acked Read 1 Sent Done Done false
21042 6534 Acked Write 3 Sent Done Done false
21043 6535 Acked Read 6 Sent Done Done false
21044 6536 Acked Read 5 Sent Done Done false
21045 6537 Acked Read 5 Sent Done Done false
21046 6538 Acked Read 2 Sent Done Done false
21047 6539 Acked Read 7 Sent Done Done false
21048 6540 Acked Read 9 Sent Done Done false
21049 6541 Acked Read 4 Sent Done Done false
21050 6542 Acked Flush 0 Sent Done Done false
21051 6543 Acked Flush 0 Sent Done Done false
21052 6544 Acked Read 8 Sent Done Done false
21053 6545 Acked Write 4 Sent Done Done false
21054 6546 Acked Read 8 Sent Done Done false
21055 6547 Acked Read 8 Sent Done Done false
21056 6548 Acked Read 6 Sent Done Done false
21057 6549 Acked Read 1 Sent Done Done false
21058 6550 Acked Write 3 Sent Done Done false
21059 6551 Acked Flush 0 Sent Done Done false
21060 6552 Acked Flush 0 Sent Done Done false
21061 6553 Acked Read 4 Sent Done Done false
21062 6554 Acked Read 5 Sent Done Done false
21063 6555 Acked Read 5 Sent Done Done false
21064 6556 Acked Write 5 Sent Done Done false
21065 6557 Acked Read 10 Sent Done Done false
21066 6558 Acked Write 3 Sent Done Done false
21067 6559 Acked Write 3 Sent Done Done false
21068 6560 Acked Flush 0 Sent Done Done false
21069 6561 Acked Write 6 Sent Done Done false
21070 6562 Acked Write 1 Sent Done Done false
21071 6563 Acked Write 4 Sent Done Done false
21072 6564 Acked Write 4 Sent Done Done false
21073 6565 Acked Read 10 Sent Done Done false
21074 6566 Acked Write 3 Sent Done Done false
21075 6567 Acked Read 8 Sent Done Done false
21076 6568 Acked Write 8 Sent Done Done false
21077 6569 Acked Write 3 Sent Done Done false
21078 6570 Acked Read 4 Sent Done Done false
21079 6571 Acked Read 1 Sent Done Done false
21080 6572 Acked Write 8 Sent Done Done false
21081 6573 Acked Flush 0 Sent Done Done false
21082 6574 Acked Read 2 Sent Done Done false
21083 6575 Acked Read 10 Sent Done Done false
21084 6576 Acked Read 2 Sent Done Done false
21085 6577 Acked Write 6 Sent Done Done false
21086 6578 Acked Write 9 Sent Done Done false
21087 6579 Acked Write 7 Sent Done Done false
21088 6580 Acked Read 8 Sent Done Done false
21089 6581 Acked Read 7 Sent Done Done false
21090 6582 Acked Flush 0 Sent Done Done false
21091 6583 Acked Read 5 Sent Done Done false
21092 6584 Acked Read 4 Sent Done Done false
21093 6585 Acked Read 1 Sent Done Done false
21094 6586 Acked Read 4 Sent Done Done false
21095 6587 Acked Read 9 Sent Done Done false
21096 6588 Acked Read 6 Sent Done Done false
21097 6589 Acked Read 4 Sent Done Done false
21098 6590 Acked Read 9 Sent Done Done false
21099 6591 Acked Write 5 Sent Done Done false
21100 6592 Acked Flush 0 Sent Done Done false
21101 6593 Acked Write 1 Sent Done Done false
21102 6594 Acked Write 1 Sent Done Done false
21103 6595 Acked Write 4 Sent Done Done false
21104 6596 Acked Write 8 Sent Done Done false
21105 6597 Acked Read 10 Sent Done Done false
21106 6598 Acked Write 5 Sent Done Done false
21107 6599 Acked Read 6 Sent Done Done false
21108 6600 Acked Flush 0 Sent Done Done false
21109 6601 Acked Write 1 Sent Done Done false
21110 6602 Acked Read 5 Sent Done Done false
21111 6603 Acked Write 4 Sent Done Done false
21112 6604 Acked Write 3 Sent Done Done false
21113 6605 Acked Write 6 Sent Done Done false
21114 6606 Acked Write 7 Sent Done Done false
21115 6607 Acked Read 4 Sent Done Done false
21116 6608 Acked Read 2 Sent Done Done false
21117 6609 Acked Write 5 Sent Done Done false
21118 6610 Acked Read 10 Sent Done Done false
21119 6611 Acked Read 4 Sent Done Done false
21120 6612 Acked Write 3 Sent Done Done false
21121 6613 Acked Read 1 Sent Done Done false
21122 6614 Acked Read 10 Sent Done Done false
21123 6615 Acked Read 6 Sent Done Done false
21124 6616 Acked Read 6 Sent Done Done false
21125 6617 Acked Write 10 Sent Done Done false
21126 6618 Acked Read 3 Sent Done Done false
21127 6619 Acked Write 2 Sent Done Done false
21128 6620 Acked Write 3 Sent Done Done false
21129 6621 Acked Read 6 Sent Done Done false
21130 6622 Acked Write 6 Sent Done Done false
21131 6623 Acked Write 9 Sent Done Done false
21132 6624 Acked Read 4 Sent Done Done false
21133 6625 Acked Write 10 Sent Done Done false
21134 6626 Acked Write 6 Sent Done Done false
21135 6627 Acked Flush 0 Sent Done Done false
21136 6628 Acked Write 6 Sent Done Done false
21137 6629 Acked Read 10 Sent Done Done false
21138 6630 Acked Write 6 Sent Done Done false
21139 6631 Acked Read 10 Sent Done Done false
21140 6632 Acked Write 6 Sent Done Done false
21141 6633 Acked Flush 0 Sent Done Done false
21142 6634 Acked Write 1 Sent Done Done false
21143 6635 Acked Read 4 Sent Done Done false
21144 6636 Acked Read 7 Sent Done Done false
21145 6637 Acked Write 4 Sent Done Done false
21146 6638 Acked Read 3 Sent Done Done false
21147 6639 Acked Flush 0 Sent Done Done false
21148 6640 Acked Read 8 Sent Done Done false
21149 6641 Acked Read 3 Sent Done Done false
21150 6642 Acked Read 7 Sent Done Done false
21151 6643 Acked Write 10 Sent Done Done false
21152 6644 Acked Read 1 Sent Done Done false
21153 6645 Acked Read 6 Sent Done Done false
21154 6646 Acked Read 4 Sent Done Done false
21155 6647 Acked Write 9 Sent Done Done false
21156 6648 Acked Write 2 Sent Done Done false
21157 6649 Acked Read 1 Sent Done Done false
21158 6650 Acked Write 3 Sent Done Done false
21159 6651 Acked Write 7 Sent Done Done false
21160 6652 Acked Write 6 Sent Done Done false
21161 6653 Acked Write 1 Sent Done Done false
21162 6654 Acked Read 4 Sent Done Done false
21163 6655 Acked Write 1 Sent Done Done false
21164 6656 Acked Write 2 Sent Done Done false
21165 6657 Acked Read 2 Sent Done Done false
21166 6658 Acked Read 3 Sent Done Done false
21167 6659 Acked Write 10 Sent Done Done false
21168 6660 Acked Write 3 Sent Done Done false
21169 6661 Acked Read 4 Sent Done Done false
21170 6662 Acked Read 3 Sent Done Done false
21171 6663 Acked Read 8 Sent Done Done false
21172 6664 Acked Flush 0 Sent Done Done false
21173 6665 Acked Flush 0 Sent Done Done false
21174 6666 Acked Read 4 Sent Done Done false
21175 6667 Acked Read 7 Sent Done Done false
21176 6668 Acked Read 4 Sent Done Done false
21177 6669 Acked Write 10 Sent Done Done false
21178 6670 Acked Read 2 Sent Done Done false
21179 6671 Acked Read 6 Sent Done Done false
21180 6672 Acked Write 4 Sent Done Done false
21181 6673 Acked Read 9 Sent Done Done false
21182 6674 Acked Read 3 Sent Done Done false
21183 6675 Acked Write 5 Sent Done Done false
21184 6676 Acked Flush 0 Sent Done Done false
21185 6677 Acked Read 3 Sent Done Done false
21186 6678 Acked Read 1 Sent Done Done false
21187 6679 Acked Read 3 Sent Done Done false
21188 6680 Acked Flush 0 Sent Done Done false
21189 6681 Acked Write 6 Sent Done Done false
21190 6682 Acked Read 3 Sent Done Done false
21191 6683 Acked Read 9 Sent Done Done false
21192 6684 Acked Write 8 Sent Done Done false
21193 6685 Acked Write 5 Sent Done Done false
21194 6686 Acked Read 4 Sent Done Done false
21195 6687 Acked Read 3 Sent Done Done false
21196 6688 Acked Write 8 Sent Done Done false
21197 6689 Acked Read 8 Sent Done Done false
21198 6690 Acked Read 9 Sent Done Done false
21199 6691 Acked Write 9 Sent Done Done false
21200 6692 Acked Write 4 Sent Done Done false
21201 6693 Acked Flush 0 Sent Done Done false
21202 6694 Acked Write 4 Sent Done Done false
21203 6695 Acked Read 4 Sent Done Done false
21204 6696 Acked Read 10 Sent Done Done false
21205 6697 Acked Read 2 Sent Done Done false
21206 6698 Acked Read 10 Sent Done Done false
21207 6699 Acked Read 1 Sent Done Done false
21208 6700 Acked Read 2 Sent Done Done false
21209 6701 Acked Write 5 Sent Done Done false
21210 6702 Acked Read 6 Sent Done Done false
21211 6703 Acked Read 9 Sent Done Done false
21212 6704 Acked Write 6 Sent Done Done false
21213 6705 Acked Read 8 Sent Done Done false
21214 6706 Acked Read 5 Sent Done Done false
21215 6707 Acked Write 3 Sent Done Done false
21216 6708 Acked Read 7 Sent Done Done false
21217 6709 Acked Read 1 Sent Done Done false
21218 6710 Acked Read 1 Sent Done Done false
21219 6711 Acked Write 5 Sent Done Done false
21220 6712 Acked Write 5 Sent Done Done false
21221 6713 Acked Read 9 Sent Done Done false
21222 6714 Acked Flush 0 Sent Done Done false
21223 6715 Acked Write 10 Sent Done Done false
21224 6716 Acked Read 9 Sent Done Done false
21225 6717 Acked Read 10 Sent Done Done false
21226 6718 Acked Write 1 Sent Done Done false
21227 6719 Acked Read 10 Sent Done Done false
21228 6720 Acked Write 4 Sent Done Done false
21229 6721 Acked Read 7 Sent Done Done false
21230 6722 Acked Write 5 Sent Done Done false
21231 6723 Acked Write 10 Sent Done Done false
21232 6724 Acked Read 5 Sent Done Done false
21233 6725 Acked Write 8 Sent Done Done false
21234 6726 Acked Read 3 Sent Done Done false
21235 6727 Acked Read 6 Sent Done Done false
21236 6728 Acked Flush 0 Sent Done Done false
21237 6729 Acked Flush 0 Sent Done Done false
21238 6730 Acked Flush 0 Sent Done Done false
21239 6731 Acked Read 6 Sent Done Done false
21240 6732 Acked Write 2 Sent Done Done false
21241 6733 Acked Read 7 Sent Done Done false
21242 6734 Acked Read 3 Sent Done Done false
21243 6735 Acked Write 1 Sent Done Done false
21244 6736 Acked Flush 0 Sent Done Done false
21245 STATES DS:0 DS:1 DS:2 TOTAL
21246 Sent 302 0 0 302
21247 Done 0 302 302 604
21248 Skipped 0 0 0 0
21249 Error 0 0 0 0
21250 Last Flush: 6434 6736 6736
21251 Downstairs last five completed: 6434 6433 6432 6431 6430
21252 Upstairs last five completed: 6736 6735 6734 6733 6732
21253 CLIENT: Up:0 ds:302 act:2
212542025-01-03T08:35:20.094ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212552025-01-03T08:35:20.095ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47365) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212562025-01-03T08:35:20.096ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212572025-01-03T08:35:20.096ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212582025-01-03T08:35:20.096ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212592025-01-03T08:35:20.096ZINFOcrucible: send last flush ID to this DS: 6434 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212602025-01-03T08:35:20.097ZINFOcrucible: Replied this last flush ID: 6434 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212612025-01-03T08:35:20.097ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212622025-01-03T08:35:20.097ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212632025-01-03T08:35:20.097ZINFOcrucible: [0] client re-new 302 jobs since flush 6434 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212642025-01-03T08:35:20.097ZINFOcrucible: [0] Replayed 302 jobs since flush: 6434 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
21265 ----------------------------------------------------------------
21266 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21267 Downstairs last five completed: 6736 6735 6734 6733 6732
21268 Upstairs last five completed: 6736 6735 6734 6733 6732
21269 CLIENT: Up:0 ds:0 act:3
21270 Replay: All jobs finished
21271 Main task has work to do, go find it
21272 got dsc Stop(1)
21273 Jan 03 08:35:22.345 INFO stop 1
21274 request completed, latency_us: 337, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 11b42450-4d72-4771-affe-d96c5ff97626, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
21275 [8820] Got stop action so:false kr:false
21276 Jan 03 08:35:22.346 INFO request completed, latency_us: 125, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7955bad3-560a-4dd5-932b-820476eb6d52, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
212772025-01-03T08:35:22.346ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212782025-01-03T08:35:22.346ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212792025-01-03T08:35:22.346ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212802025-01-03T08:35:22.346ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212812025-01-03T08:35:22.346ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
212822025-01-03T08:35:22.346ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
21283 [8820] Exited with: ExitStatus(unix_wait_status(9))
21284 [8820][1] reports Exit
21285 Jan 03 08:35:26.348 INFO request completed, latency_us: 231, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4a1a0bc8-746d-4a24-bcb7-a39217d1b24b, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
21286 001/300 Write block 1097 len 12288 data: 6 6 5
21287 002/300 Flush
21288 003/300 Write block 2157 len 40960 data: 5 5 4 5 6 5 5 5 5 4
21289 004/300 Write block 4961 len 40960 data: 6 5 3 3 5 5 5 5 4 4
21290 005/300 Read block 869 len 40960
21291 006/300 Read block 2190 len 32768
21292 007/300 Flush
21293 008/300 Read block 2056 len 24576
21294 009/300 Read block 2734 len 4096
21295 010/300 Write block 1150 len 40960 data: 3 3 3 4 5 5 4 4 5 7
21296 011/300 Read block 1269 len 20480
21297 012/300 Write block 4205 len 32768 data: 9 8 6 5 4 3 4 4
21298 013/300 Write block 1494 len 4096 data: 3
21299 014/300 Write block 429 len 32768 data: 4 4 4 4 4 4 6 5
21300 015/300 Read block 1330 len 28672
21301 016/300 Read block 4788 len 32768
21302 017/300 Write block 1080 len 16384 data: 3 2 2 2
21303 018/300 Write block 576 len 8192 data: 3 4
21304 019/300 Write block 480 len 36864 data: 4 4 4 4 5 5 6 5 7
21305 020/300 Read block 1599 len 16384
21306 021/300 Write block 3367 len 8192 data: 4 5
21307 022/300 Read block 4083 len 32768
21308 023/300 Read block 154 len 36864
21309 024/300 Read block 3015 len 28672
21310 025/300 Read block 2076 len 32768
21311 026/300 Flush
21312 027/300 Write block 2306 len 28672 data: 7 8 9 9 8 6 5
21313 028/300 Write block 1150 len 20480 data: 4 4 4 5 6
21314 029/300 Read block 3166 len 16384
21315 030/300 Read block 3920 len 36864
21316 031/300 Write block 4807 len 4096 data: 5
21317 032/300 Write block 2225 len 24576 data: 4 4 6 6 7 4
21318 033/300 Read block 4043 len 8192
21319 034/300 Read block 890 len 4096
21320 035/300 Write block 1817 len 4096 data: 3
21321 036/300 Write block 2253 len 28672 data: 7 7 7 7 5 4 4
21322 037/300 Read block 1322 len 16384
21323 038/300 Read block 3039 len 40960
21324 039/300 Write block 2111 len 16384 data: 5 5 5 7
21325 040/300 Write block 1169 len 32768 data: 7 6 6 5 5 7 6 7
21326 041/300 Read block 3654 len 4096
21327 042/300 Write block 3400 len 20480 data: 5 6 7 6 5
21328 043/300 Read block 996 len 16384
21329 044/300 Read block 3998 len 40960
21330 045/300 Write block 3663 len 8192 data: 3 5
21331 046/300 Write block 1485 len 8192 data: 7 7
21332 047/300 Read block 4508 len 12288
21333 048/300 Read block 3201 len 36864
21334 049/300 Write block 3704 len 16384 data: 6 6 5 5
21335 050/300 Read block 3080 len 36864
21336 051/300 Flush
21337 052/300 Read block 4077 len 12288
21338 053/300 Read block 4109 len 40960
21339 054/300 Read block 2677 len 40960
21340 055/300 Read block 628 len 12288
21341 056/300 Flush
21342 057/300 Write block 755 len 28672 data: 2 2 4 5 7 7 6
21343 058/300 Read block 2080 len 36864
21344 059/300 Write block 2095 len 4096 data: 4
21345 060/300 Write block 2827 len 4096 data: 4
21346 061/300 Write block 16 len 28672 data: 3 3 2 3 3 4 4
21347 062/300 Read block 2791 len 16384
21348 063/300 Write block 87 len 16384 data: 5 4 4 3
21349 064/300 Write block 3747 len 16384 data: 5 4 4 4
21350 065/300 Write block 2440 len 24576 data: 3 4 4 5 4 4
21351 066/300 Write block 3835 len 12288 data: 4 4 4
21352 067/300 Read block 2355 len 24576
21353 068/300 Read block 4518 len 16384
21354 069/300 Write block 424 len 36864 data: 5 4 4 3 3 5 5 5 5
21355 070/300 Write block 1541 len 20480 data: 3 3 3 3 2
21356 071/300 Read block 1043 len 8192
21357 072/300 Read block 590 len 32768
21358 073/300 Write block 1752 len 36864 data: 6 8 8 6 4 5 3 3 3
21359 074/300 Read block 3220 len 12288
21360 075/300 Read block 3684 len 24576
21361 076/300 Read block 4909 len 12288
21362 077/300 Read block 1352 len 12288
21363 078/300 Write block 587 len 40960 data: 5 3 3 3 3 4 3 3 3 3
21364 079/300 Read block 4293 len 36864
21365 080/300 Read block 833 len 36864
21366 081/300 Read block 1967 len 12288
21367 082/300 Read block 1538 len 4096
21368 083/300 Read block 3204 len 8192
21369 084/300 Write block 3024 len 12288 data: 4 3 3
21370 085/300 Read block 1996 len 12288
21371 086/300 Write block 3693 len 28672 data: 4 5 6 6 5 5 5
21372 087/300 Write block 1425 len 24576 data: 6 6 5 4 3 3
21373 088/300 Read block 1341 len 32768
21374 089/300 Write block 4188 len 28672 data: 3 3 4 5 6 6 5
21375 090/300 Read block 3327 len 12288
21376 091/300 Read block 4400 len 4096
21377 092/300 Write block 4954 len 20480 data: 5 6 6 6 6
21378 093/300 Flush
21379 094/300 Flush
21380 095/300 Read block 1952 len 36864
21381 096/300 Write block 447 len 40960 data: 5 4 4 3 3 3 3 3 3 2
21382 097/300 Read block 3386 len 24576
21383 098/300 Flush
21384 099/300 Read block 138 len 24576
21385 100/300 Write block 4192 len 16384 data: 7 7 6 6
21386 101/300 Write block 1584 len 28672 data: 5 7 6 6 6 7 6
21387 102/300 Read block 4304 len 16384
21388 103/300 Flush
21389 104/300 Write block 164 len 24576 data: 3 3 3 2 2 2
21390 105/300 Read block 3884 len 24576
21391 106/300 Read block 1785 len 28672
21392 107/300 Read block 673 len 12288
21393 108/300 Read block 2343 len 24576
21394 109/300 Read block 3525 len 4096
21395 110/300 Read block 4177 len 20480
21396 111/300 Read block 2934 len 12288
21397 112/300 Read block 2113 len 24576
21398 113/300 Write block 2344 len 40960 data: 2 3 3 3 3 3 3 4 4 4
21399 114/300 Write block 1488 len 12288 data: 6 6 6
21400 115/300 Read block 503 len 36864
21401 116/300 Read block 1004 len 36864
21402 117/300 Read block 4133 len 12288
21403 118/300 Write block 4095 len 8192 data: 3 5
21404 119/300 Write block 1889 len 28672 data: 6 6 5 5 5 4 4
21405 120/300 Flush
21406 121/300 Write block 2111 len 16384 data: 6 6 6 8
21407 122/300 Read block 4619 len 4096
21408 123/300 Write block 2516 len 8192 data: 4 4
21409 124/300 Read block 4436 len 24576
21410 125/300 Write block 2557 len 12288 data: 2 4 4
21411 126/300 Write block 1617 len 40960 data: 3 4 5 5 5 5 4 4 3 4
21412 127/300 Write block 3138 len 16384 data: 3 3 3 3
21413 128/300 Read block 1307 len 8192
21414 129/300 Read block 1472 len 20480
21415 130/300 Write block 3867 len 36864 data: 10 9 9 7 8 8 6 6 5
21416 131/300 Read block 4966 len 32768
21417 132/300 Write block 1101 len 40960 data: 6 4 3 2 2 4 4 6 6 5
21418 133/300 Flush
21419 134/300 Read block 1632 len 12288
21420 135/300 Write block 2022 len 12288 data: 3 3 4
21421 136/300 Read block 1240 len 24576
21422 137/300 Read block 2815 len 36864
21423 138/300 Read block 3582 len 4096
21424 139/300 Read block 4862 len 20480
21425 140/300 Read block 2034 len 36864
21426 141/300 Write block 2951 len 16384 data: 3 3 3 4
21427 142/300 Read block 4353 len 28672
21428 143/300 Write block 782 len 40960 data: 4 4 6 6 6 5 5 5 8 6
21429 144/300 Read block 4905 len 32768
21430 145/300 Read block 3469 len 40960
21431 146/300 Read block 1371 len 36864
21432 147/300 Read block 128 len 40960
21433 148/300 Read block 142 len 12288
21434 149/300 Read block 4430 len 20480
21435 150/300 Read block 3059 len 40960
21436 151/300 Read block 3 len 12288
21437 152/300 Write block 2042 len 32768 data: 4 4 4 4 5 5 6 6
21438 153/300 Flush
21439 154/300 Write block 1370 len 4096 data: 5
21440 155/300 Read block 3919 len 20480
21441 156/300 Write block 1312 len 20480 data: 4 4 4 3 3
21442 157/300 Read block 2693 len 32768
21443 158/300 Write block 419 len 32768 data: 6 6 6 6 5 6 5 5
21444 159/300 Write block 2579 len 24576 data: 3 3 4 5 5 5
21445 160/300 Write block 4415 len 32768 data: 4 5 4 4 5 5 4 5
21446 161/300 Write block 1733 len 16384 data: 5 6 6 8
21447 162/300 Read block 4672 len 36864
21448 163/300 Write block 1945 len 20480 data: 5 6 7 7 8
21449 164/300 Write block 1596 len 12288 data: 5 5 6
21450 165/300 Read block 1942 len 28672
21451 166/300 Flush
21452 167/300 Read block 116 len 12288
21453 168/300 Read block 871 len 4096
21454 169/300 Read block 2646 len 4096
21455 170/300 Read block 1922 len 8192
21456 171/300 Write block 3825 len 8192 data: 6 6
21457 172/300 Flush
21458 173/300 Read block 3408 len 4096
21459 174/300 Write block 3968 len 12288 data: 4 4 3
21460 175/300 Flush
21461 176/300 Read block 623 len 28672
21462 177/300 Write block 319 len 24576 data: 7 4 3 4 4 4
21463 178/300 Write block 3662 len 8192 data: 5 4
21464 179/300 Write block 1001 len 4096 data: 5
21465 180/300 Write block 4566 len 40960 data: 4 4 4 4 4 4 4 3 4 4
21466 181/300 Write block 180 len 16384 data: 3 3 3 4
21467 182/300 Read block 553 len 32768
21468 183/300 Read block 1158 len 12288
21469 184/300 Read block 55 len 32768
21470 185/300 Write block 1066 len 8192 data: 2 2
21471 186/300 Write block 4293 len 24576 data: 2 2 4 3 3 3
21472 187/300 Read block 3658 len 8192
21473 188/300 Write block 2545 len 12288 data: 4 3 3
21474 189/300 Write block 1130 len 36864 data: 5 3 4 6 5 6 6 5 5
21475 190/300 Read block 2468 len 24576
21476 191/300 Read block 1456 len 32768
21477 192/300 Read block 4709 len 4096
21478 193/300 Read block 1988 len 4096
21479 194/300 Read block 3934 len 32768
21480 195/300 Read block 3195 len 16384
21481 196/300 Write block 4128 len 36864 data: 5 5 4 4 4 4 4 5 4
21482 197/300 Read block 496 len 36864
21483 198/300 Write block 657 len 28672 data: 7 8 7 6 4 5 5
21484 199/300 Write block 4744 len 24576 data: 3 3 3 3 5 5
21485 200/300 Write block 1851 len 20480 data: 4 4 3 3 2
21486 201/300 Flush
21487 202/300 Flush
21488 203/300 Write block 3896 len 16384 data: 7 6 4 4
21489 204/300 Write block 3469 len 16384 data: 8 7 5 4
21490 205/300 Flush
21491 206/300 Read block 230 len 16384
21492 207/300 Write block 612 len 16384 data: 5 5 6 6
21493 208/300 Read block 3609 len 20480
21494 209/300 Read block 3012 len 16384
21495 210/300 Write block 301 len 40960 data: 4 5 5 5 7 5 6 8 8 10
21496 211/300 Read block 3345 len 36864
21497 212/300 Write block 3003 len 28672 data: 2 2 2 3 2 2 2
21498 213/300 Write block 3031 len 40960 data: 3 3 3 4 4 2 2 3 4 4
21499 214/300 Read block 4143 len 28672
21500 215/300 Flush
21501 216/300 Read block 206 len 36864
21502 217/300 Write block 2826 len 36864 data: 6 5 3 3 3 3 3 5 6
21503 218/300 Read block 812 len 36864
21504 219/300 Read block 1527 len 28672
21505 220/300 Write block 4854 len 36864 data: 4 4 3 4 6 4 4 4 7
21506 221/300 Flush
21507 222/300 Write block 3531 len 40960 data: 3 4 4 5 6 5 5 5 5 5
21508 223/300 Write block 2278 len 16384 data: 2 2 2 2
21509 224/300 Read block 1392 len 32768
21510 225/300 Write block 4365 len 8192 data: 6 6
21511 226/300 Read block 4382 len 32768
21512 227/300 Write block 4720 len 36864 data: 4 4 3 4 5 4 4 4 4
21513 228/300 Read block 3598 len 32768
21514 229/300 Write block 4917 len 40960 data: 5 7 8 8 8 6 5 2 2 3
21515 230/300 Read block 612 len 16384
21516 231/300 Read block 2896 len 24576
21517 232/300 Read block 2741 len 24576
21518 233/300 Read block 1546 len 8192
21519 234/300 Read block 2660 len 16384
21520 235/300 Write block 1456 len 8192 data: 6 6
21521 236/300 Write block 39 len 40960 data: 4 4 4 3 3 3 3 3 3 3
21522 237/300 Flush
21523 238/300 Read block 2426 len 16384
21524 239/300 Read block 4569 len 20480
21525 240/300 Read block 1141 len 32768
21526 241/300 Write block 482 len 28672 data: 5 5 6 6 7 6 8
21527 242/300 Read block 4967 len 36864
21528 243/300 Read block 1969 len 4096
21529 244/300 Read block 1832 len 40960
21530 245/300 Write block 1864 len 36864 data: 2 2 2 2 2 2 2 2 2
21531 246/300 Read block 2083 len 28672
21532 247/300 Read block 3665 len 28672
21533 248/300 Flush
21534 249/300 Read block 2006 len 28672
21535 250/300 Flush
21536 251/300 Read block 1151 len 12288
21537 252/300 Read block 350 len 12288
21538 253/300 Flush
21539 254/300 Write block 4709 len 20480 data: 7 6 7 7 5
21540 255/300 Write block 838 len 32768 data: 5 5 6 6 8 9 9 7
21541 256/300 Write block 890 len 8192 data: 8 8
21542 257/300 Read block 2250 len 16384
21543 258/300 Read block 1941 len 36864
21544 259/300 Read block 2271 len 16384
21545 260/300 Write block 3366 len 8192 data: 3 5
21546 261/300 Write block 3178 len 24576 data: 7 7 7 6 5 6
21547 262/300 Flush
21548 263/300 Read block 1149 len 12288
21549 264/300 Write block 3909 len 40960 data: 5 3 3 3 3 3 2 2 3 3
21550 265/300 Write block 2535 len 24576 data: 3 2 2 3 3 4
21551 266/300 Write block 2940 len 32768 data: 4 4 4 3 4 4 6 6
21552 267/300 Write block 3446 len 4096 data: 4
21553 268/300 Read block 575 len 40960
21554 269/300 Read block 3902 len 24576
21555 270/300 Write block 1315 len 32768 data: 4 4 5 5 5 5 5 4
21556 271/300 Read block 443 len 4096
21557 272/300 Read block 2829 len 36864
21558 273/300 Read block 3778 len 4096
21559 274/300 Write block 4793 len 16384 data: 3 3 3 3
21560 275/300 Read block 816 len 32768
21561 276/300 Write block 2018 len 20480 data: 6 5 5 6 4
21562 277/300 Flush
21563 278/300 Write block 2153 len 16384 data: 4 5 5 5
21564 279/300 Write block 4290 len 32768 data: 3 3 2 3 3 5 4 4
21565 280/300 Read block 1196 len 20480
21566 281/300 Read block 3393 len 24576
21567 282/300 Write block 3713 len 24576 data: 3 3 5 5 6 6
21568 283/300 Write block 3450 len 28672 data: 4 4 3 4 6 6 7
21569 284/300 Write block 4066 len 40960 data: 6 9 8 7 7 7 6 8 7 5
21570 285/300 Write block 3675 len 16384 data: 7 6 4 3
21571 286/300 Flush
21572 287/300 Write block 731 len 36864 data: 5 5 5 5 5 6 5 5 5
21573 288/300 Read block 1021 len 24576
21574 289/300 Write block 4136 len 40960 data: 5 5 5 3 4 5 5 7 7 7
21575 290/300 Flush
21576 291/300 Write block 4292 len 36864 data: 3 4 4 6 5 5 4 4 4
21577 292/300 Write block 2099 len 40960 data: 5 4 4 3 3 3 3 3 3 2
21578 293/300 Write block 2823 len 36864 data: 4 4 5 7 6 4 4 4 4
21579 294/300 Flush
21580 295/300 Write block 3281 len 36864 data: 4 2 2 4 3 4 3 4 4
21581 296/300 Write block 3636 len 4096 data: 6
21582 297/300 Read block 3255 len 28672
21583 298/300 Write block 2163 len 12288 data: 6 6 6
21584 299/300 Read block 1483 len 8192
21585 300/300 Read block 4493 len 24576
21586 301/300 Read block 4467 len 20480
21587 Main task has work to do, go find it
21588 got dsc Start(1)
21589 start 1
21590 Jan 03 08:35:26.594 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/1, method: POST, req_id: 29b6660f-ef44-47f9-ae7d-64bac504cc0a, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
21591 Replay: started 1, returned:Ok(())
21592 ----------------------------------------------------------------
21593 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21594 States: Active Offline Active
21595 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21596 6737 Acked Write 3 Done Sent Done false
21597 6738 Acked Flush 0 Done Sent Done false
21598 6739 Acked Write 10 Done Sent Done false
21599 6740 Acked Write 10 Done Sent Done false
21600 6741 Acked Read 10 Done Sent Done false
21601 6742 Acked Read 8 Done Sent Done false
21602 6743 Acked Flush 0 Done Sent Done false
21603 6744 Acked Read 6 Done Sent Done false
21604 6745 Acked Read 1 Done Sent Done false
21605 6746 Acked Write 10 Done Sent Done false
21606 6747 Acked Read 5 Done Sent Done false
21607 6748 Acked Write 8 Done Sent Done false
21608 6749 Acked Write 1 Done Sent Done false
21609 6750 Acked Write 8 Done Sent Done false
21610 6751 Acked Read 7 Done Sent Done false
21611 6752 Acked Read 8 Done Sent Done false
21612 6753 Acked Write 4 Done Sent Done false
21613 6754 Acked Write 2 Done Sent Done false
21614 6755 Acked Write 9 Done Sent Done false
21615 6756 Acked Read 4 Done Sent Done false
21616 6757 Acked Write 2 Done Sent Done false
21617 6758 Acked Read 8 Done Sent Done false
21618 6759 Acked Read 9 Done Sent Done false
21619 6760 Acked Read 7 Done Sent Done false
21620 6761 Acked Read 8 Done Sent Done false
21621 6762 Acked Flush 0 Done Sent Done false
21622 6763 Acked Write 7 Done Sent Done false
21623 6764 Acked Write 5 Done Sent Done false
21624 6765 Acked Read 4 Done Sent Done false
21625 6766 Acked Read 9 Done Sent Done false
21626 6767 Acked Write 1 Done Sent Done false
21627 6768 Acked Write 6 Done Sent Done false
21628 6769 Acked Read 2 Done Sent Done false
21629 6770 Acked Read 1 Done Sent Done false
21630 6771 Acked Write 1 Done Sent Done false
21631 6772 Acked Write 7 Done Sent Done false
21632 6773 Acked Read 4 Done Sent Done false
21633 6774 Acked Read 10 Done Sent Done false
21634 6775 Acked Write 4 Done Sent Done false
21635 6776 Acked Write 8 Done Sent Done false
21636 6777 Acked Read 1 Done Sent Done false
21637 6778 Acked Write 5 Done Sent Done false
21638 6779 Acked Read 4 Done Sent Done false
21639 6780 Acked Read 10 Done Sent Done false
21640 6781 Acked Write 2 Done Sent Done false
21641 6782 Acked Write 2 Done Sent Done false
21642 6783 Acked Read 3 Done Sent Done false
21643 6784 Acked Read 9 Done Sent Done false
21644 6785 Acked Write 4 Done Sent Done false
21645 6786 Acked Read 9 Done Sent Done false
21646 6787 Acked Flush 0 Done Sent Done false
21647 6788 Acked Read 3 Done Sent Done false
21648 6789 Acked Read 10 Done Sent Done false
21649 6790 Acked Read 10 Done Sent Done false
21650 6791 Acked Read 3 Done Sent Done false
21651 6792 Acked Flush 0 Done Sent Done false
21652 6793 Acked Write 7 Done Sent Done false
21653 6794 Acked Read 9 Done Sent Done false
21654 6795 Acked Write 1 Done Sent Done false
21655 6796 Acked Write 1 Done Sent Done false
21656 6797 Acked Write 7 Done Sent Done false
21657 6798 Acked Read 4 Done Sent Done false
21658 6799 Acked Write 4 Done Sent Done false
21659 6800 Acked Write 4 Done Sent Done false
21660 6801 Acked Write 6 Done Sent Done false
21661 6802 Acked Write 3 Done Sent Done false
21662 6803 Acked Read 6 Done Sent Done false
21663 6804 Acked Read 4 Done Sent Done false
21664 6805 Acked Write 9 Done Sent Done false
21665 6806 Acked Write 5 Done Sent Done false
21666 6807 Acked Read 2 Done Sent Done false
21667 6808 Acked Read 8 Done Sent Done false
21668 6809 Acked Write 9 Done Sent Done false
21669 6810 Acked Read 3 Done Sent Done false
21670 6811 Acked Read 6 Done Sent Done false
21671 6812 Acked Read 3 Done Sent Done false
21672 6813 Acked Read 3 Done Sent Done false
21673 6814 Acked Write 10 Done Sent Done false
21674 6815 Acked Read 9 Done Sent Done false
21675 6816 Acked Read 9 Done Sent Done false
21676 6817 Acked Read 3 Done Sent Done false
21677 6818 Acked Read 1 Done Sent Done false
21678 6819 Acked Read 2 Done Sent Done false
21679 6820 Acked Write 3 Done Sent Done false
21680 6821 Acked Read 3 Done Sent Done false
21681 6822 Acked Write 7 Done Sent Done false
21682 6823 Acked Write 6 Done Sent Done false
21683 6824 Acked Read 8 Done Sent Done false
21684 6825 Acked Write 7 Done Sent Done false
21685 6826 Acked Read 3 Done Sent Done false
21686 6827 Acked Read 1 Done Sent Done false
21687 6828 Acked Write 5 Done Sent Done false
21688 6829 Acked Flush 0 Done Sent Done false
21689 6830 Acked Flush 0 Done Sent Done false
21690 6831 Acked Read 9 Done Sent Done false
21691 6832 Acked Write 10 Done Sent Done false
21692 6833 Acked Read 6 Done Sent Done false
21693 6834 Acked Flush 0 Done Sent Done false
21694 6835 Acked Read 6 Done Sent Done false
21695 6836 Acked Write 4 Done Sent Done false
21696 6837 Acked Write 7 Done Sent Done false
21697 6838 Acked Read 4 Done Sent Done false
21698 6839 Acked Flush 0 Done Sent Done false
21699 6840 Acked Write 6 Done Sent Done false
21700 6841 Acked Read 6 Done Sent Done false
21701 6842 Acked Read 7 Done Sent Done false
21702 6843 Acked Read 3 Done Sent Done false
21703 6844 Acked Read 6 Done Sent Done false
21704 6845 Acked Read 1 Done Sent Done false
21705 6846 Acked Read 5 Done Sent Done false
21706 6847 Acked Read 3 Done Sent Done false
21707 6848 Acked Read 6 Done Sent Done false
21708 6849 Acked Write 10 Done Sent Done false
21709 6850 Acked Write 3 Done Sent Done false
21710 6851 Acked Read 9 Done Sent Done false
21711 6852 Acked Read 9 Done Sent Done false
21712 6853 Acked Read 3 Done Sent Done false
21713 6854 Acked Write 2 Done Sent Done false
21714 6855 Acked Write 7 Done Sent Done false
21715 6856 Acked Flush 0 Done Sent Done false
21716 6857 Acked Write 4 Done Sent Done false
21717 6858 Acked Read 1 Done Sent Done false
21718 6859 Acked Write 2 Done Sent Done false
21719 6860 Acked Read 6 Done Sent Done false
21720 6861 Acked Write 3 Done Sent Done false
21721 6862 Acked Write 10 Done Sent Done false
21722 6863 Acked Write 4 Done Sent Done false
21723 6864 Acked Read 2 Done Sent Done false
21724 6865 Acked Read 5 Done Sent Done false
21725 6866 Acked Write 9 Done Sent Done false
21726 6867 Acked Read 8 Done Sent Done false
21727 6868 Acked Write 10 Done Sent Done false
21728 6869 Acked Flush 0 Done Sent Done false
21729 6870 Acked Read 3 Done Sent Done false
21730 6871 Acked Write 3 Done Sent Done false
21731 6872 Acked Read 6 Done Sent Done false
21732 6873 Acked Read 9 Done Sent Done false
21733 6874 Acked Read 1 Done Sent Done false
21734 6875 Acked Read 5 Done Sent Done false
21735 6876 Acked Read 9 Done Sent Done false
21736 6877 Acked Write 4 Done Sent Done false
21737 6878 Acked Read 7 Done Sent Done false
21738 6879 Acked Write 10 Done Sent Done false
21739 6880 Acked Read 8 Done Sent Done false
21740 6881 Acked Read 10 Done Sent Done false
21741 6882 Acked Read 9 Done Sent Done false
21742 6883 Acked Read 10 Done Sent Done false
21743 6884 Acked Read 3 Done Sent Done false
21744 6885 Acked Read 5 Done Sent Done false
21745 6886 Acked Read 10 Done Sent Done false
21746 6887 Acked Read 3 Done Sent Done false
21747 6888 Acked Write 8 Done Sent Done false
21748 6889 Acked Flush 0 Done Sent Done false
21749 6890 Acked Write 1 Done Sent Done false
21750 6891 Acked Read 5 Done Sent Done false
21751 6892 Acked Write 5 Done Sent Done false
21752 6893 Acked Read 8 Done Sent Done false
21753 6894 Acked Write 8 Done Sent Done false
21754 6895 Acked Write 6 Done Sent Done false
21755 6896 Acked Write 8 Done Sent Done false
21756 6897 Acked Write 4 Done Sent Done false
21757 6898 Acked Read 9 Done Sent Done false
21758 6899 Acked Write 5 Done Sent Done false
21759 6900 Acked Write 3 Done Sent Done false
21760 6901 Acked Read 7 Done Sent Done false
21761 6902 Acked Flush 0 Done Sent Done false
21762 6903 Acked Read 3 Done Sent Done false
21763 6904 Acked Read 1 Done Sent Done false
21764 6905 Acked Read 1 Done Sent Done false
21765 6906 Acked Read 2 Done Sent Done false
21766 6907 Acked Write 2 Done Sent Done false
21767 6908 Acked Flush 0 Done Sent Done false
21768 6909 Acked Read 1 Done Sent Done false
21769 6910 Acked Write 3 Done Sent Done false
21770 6911 Acked Flush 0 Done Sent Done false
21771 6912 Acked Read 7 Done Sent Done false
21772 6913 Acked Write 6 Done Sent Done false
21773 6914 Acked Write 2 Done Sent Done false
21774 6915 Acked Write 1 Done Sent Done false
21775 6916 Acked Write 10 Done Sent Done false
21776 6917 Acked Write 4 Done Sent Done false
21777 6918 Acked Read 8 Done Sent Done false
21778 6919 Acked Read 3 Done Sent Done false
21779 6920 Acked Read 8 Done Sent Done false
21780 6921 Acked Write 2 Done Sent Done false
21781 6922 Acked Write 6 Done Sent Done false
21782 6923 Acked Read 2 Done Sent Done false
21783 6924 Acked Write 3 Done Sent Done false
21784 6925 Acked Write 9 Done Sent Done false
21785 6926 Acked Read 6 Done Sent Done false
21786 6927 Acked Read 8 Done Sent Done false
21787 6928 Acked Read 1 Done Sent Done false
21788 6929 Acked Read 1 Done Sent Done false
21789 6930 Acked Read 8 Done Sent Done false
21790 6931 Acked Read 4 Done Sent Done false
21791 6932 Acked Write 9 Done Sent Done false
21792 6933 Acked Read 9 Done Sent Done false
21793 6934 Acked Write 7 Done Sent Done false
21794 6935 Acked Write 6 Done Sent Done false
21795 6936 Acked Write 5 Done Sent Done false
21796 6937 Acked Flush 0 Done Sent Done false
21797 6938 Acked Flush 0 Done Sent Done false
21798 6939 Acked Write 4 Done Sent Done false
21799 6940 Acked Write 4 Done Sent Done false
21800 6941 Acked Flush 0 Done Sent Done false
21801 6942 Acked Read 4 Done Sent Done false
21802 6943 Acked Write 4 Done Sent Done false
21803 6944 Acked Read 5 Done Sent Done false
21804 6945 Acked Read 4 Done Sent Done false
21805 6946 Acked Write 10 Done Sent Done false
21806 6947 Acked Read 9 Done Sent Done false
21807 6948 Acked Write 7 Done Sent Done false
21808 6949 Acked Write 10 Done Sent Done false
21809 6950 Acked Read 7 Done Sent Done false
21810 6951 Acked Flush 0 Done Sent Done false
21811 6952 Acked Read 9 Done Sent Done false
21812 6953 Acked Write 9 Done Sent Done false
21813 6954 Acked Read 9 Done Sent Done false
21814 6955 Acked Read 7 Done Sent Done false
21815 6956 Acked Write 9 Done Sent Done false
21816 6957 Acked Flush 0 Done Sent Done false
21817 6958 Acked Write 10 Done Sent Done false
21818 6959 Acked Write 4 Done Sent Done false
21819 6960 Acked Read 8 Done Sent Done false
21820 6961 Acked Write 2 Done Sent Done false
21821 6962 Acked Read 8 Done Sent Done false
21822 6963 Acked Write 9 Done Sent Done false
21823 6964 Acked Read 8 Done Sent Done false
21824 6965 Acked Write 10 Done Sent Done false
21825 6966 Acked Read 4 Done Sent Done false
21826 6967 Acked Read 6 Done Sent Done false
21827 6968 Acked Read 6 Done Sent Done false
21828 6969 Acked Read 2 Done Sent Done false
21829 6970 Acked Read 4 Done Sent Done false
21830 6971 Acked Write 2 Done Sent Done false
21831 6972 Acked Write 10 Done Sent Done false
21832 6973 Acked Flush 0 Done Sent Done false
21833 6974 Acked Read 4 Done Sent Done false
21834 6975 Acked Read 5 Done Sent Done false
21835 6976 Acked Read 8 Done Sent Done false
21836 6977 Acked Write 7 Done Sent Done false
21837 6978 Acked Read 9 Done Sent Done false
21838 6979 Acked Read 1 Done Sent Done false
21839 6980 Acked Read 10 Done Sent Done false
21840 6981 Acked Write 9 Done Sent Done false
21841 6982 Acked Read 7 Done Sent Done false
21842 6983 Acked Read 7 Done Sent Done false
21843 6984 Acked Flush 0 Done Sent Done false
21844 6985 Acked Read 7 Done Sent Done false
21845 6986 Acked Flush 0 Done Sent Done false
21846 6987 Acked Read 3 Done Sent Done false
21847 6988 Acked Read 3 Done Sent Done false
21848 6989 Acked Flush 0 Done Sent Done false
21849 6990 Acked Write 5 Done Sent Done false
21850 6991 Acked Write 8 Done Sent Done false
21851 6992 Acked Write 2 Done Sent Done false
21852 6993 Acked Read 4 Done Sent Done false
21853 6994 Acked Read 9 Done Sent Done false
21854 6995 Acked Read 4 Done Sent Done false
21855 6996 Acked Write 2 Done Sent Done false
21856 6997 Acked Write 6 Done Sent Done false
21857 6998 Acked Flush 0 Done Sent Done false
21858 6999 Acked Read 3 Done Sent Done false
21859 7000 Acked Write 10 Done Sent Done false
21860 7001 Acked Write 6 Done Sent Done false
21861 7002 Acked Write 8 Done Sent Done false
21862 7003 Acked Write 1 Done Sent Done false
21863 7004 Acked Read 10 Done Sent Done false
21864 7005 Acked Read 6 Done Sent Done false
21865 7006 Acked Write 8 Done Sent Done false
21866 7007 Acked Read 1 Done Sent Done false
21867 7008 Acked Read 9 Done Sent Done false
21868 7009 Acked Read 1 Done Sent Done false
21869 7010 Acked Write 4 Done Sent Done false
21870 7011 Acked Read 8 Done Sent Done false
21871 7012 Acked Write 5 Done Sent Done false
21872 7013 Acked Flush 0 Done Sent Done false
21873 7014 Acked Write 4 Done Sent Done false
21874 7015 Acked Write 8 Done Sent Done false
21875 7016 Acked Read 5 Done Sent Done false
21876 7017 Acked Read 6 Done Sent Done false
21877 7018 Acked Write 6 Done Sent Done false
21878 7019 Acked Write 7 Done Sent Done false
21879 7020 Acked Write 10 Done Sent Done false
21880 7021 Acked Write 4 Done Sent Done false
21881 7022 Acked Flush 0 Done Sent Done false
21882 7023 Acked Write 9 Done Sent Done false
21883 7024 Acked Read 6 Done Sent Done false
21884 7025 Acked Write 10 Done Sent Done false
21885 7026 Acked Flush 0 Done Sent Done false
21886 7027 Acked Write 9 Done Sent Done false
21887 7028 Acked Write 10 Done Sent Done false
21888 7029 Acked Write 9 Done Sent Done false
21889 7030 Acked Flush 0 Done Sent Done false
21890 7031 Acked Write 9 Done Sent Done false
21891 7032 Acked Write 1 Done Sent Done false
21892 7033 Acked Read 7 Done Sent Done false
21893 7034 Acked Write 3 Done Sent Done false
21894 7035 Acked Read 2 Done Sent Done false
21895 7036 Acked Read 6 Done Sent Done false
21896 7037 Acked Read 5 Done Sent Done false
21897 STATES DS:0 DS:1 DS:2 TOTAL
21898 Sent 0 301 0 301
21899 Done 301 0 301 602
21900 Skipped 0 0 0 0
21901 Error 0 0 0 0
21902 Last Flush: 7030 6736 7030
21903 Downstairs last five completed: 6736 6735 6734 6733 6732
21904 Upstairs last five completed: 7037 7036 7035 7034 7033
21905 CLIENT: Up:0 ds:301 act:2
21906 [8820] got start action
21907 [8820] I am going to restart
21908 Starting downstairs at port 8820
21909 Make output file at "/tmp/dsc/downstairs-8820.txt"
21910 [8820][1] reports Starting
21911 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1225)
21912 [8820][1] reports Running
21913 ----------------------------------------------------------------
21914 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21915 States: Active Offline Active
21916 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21917 6737 Acked Write 3 Done Sent Done false
21918 6738 Acked Flush 0 Done Sent Done false
21919 6739 Acked Write 10 Done Sent Done false
21920 6740 Acked Write 10 Done Sent Done false
21921 6741 Acked Read 10 Done Sent Done false
21922 6742 Acked Read 8 Done Sent Done false
21923 6743 Acked Flush 0 Done Sent Done false
21924 6744 Acked Read 6 Done Sent Done false
21925 6745 Acked Read 1 Done Sent Done false
21926 6746 Acked Write 10 Done Sent Done false
21927 6747 Acked Read 5 Done Sent Done false
21928 6748 Acked Write 8 Done Sent Done false
21929 6749 Acked Write 1 Done Sent Done false
21930 6750 Acked Write 8 Done Sent Done false
21931 6751 Acked Read 7 Done Sent Done false
21932 6752 Acked Read 8 Done Sent Done false
21933 6753 Acked Write 4 Done Sent Done false
21934 6754 Acked Write 2 Done Sent Done false
21935 6755 Acked Write 9 Done Sent Done false
21936 6756 Acked Read 4 Done Sent Done false
21937 6757 Acked Write 2 Done Sent Done false
21938 6758 Acked Read 8 Done Sent Done false
21939 6759 Acked Read 9 Done Sent Done false
21940 6760 Acked Read 7 Done Sent Done false
21941 6761 Acked Read 8 Done Sent Done false
21942 6762 Acked Flush 0 Done Sent Done false
21943 6763 Acked Write 7 Done Sent Done false
21944 6764 Acked Write 5 Done Sent Done false
21945 6765 Acked Read 4 Done Sent Done false
21946 6766 Acked Read 9 Done Sent Done false
21947 6767 Acked Write 1 Done Sent Done false
21948 6768 Acked Write 6 Done Sent Done false
21949 6769 Acked Read 2 Done Sent Done false
21950 6770 Acked Read 1 Done Sent Done false
21951 6771 Acked Write 1 Done Sent Done false
21952 6772 Acked Write 7 Done Sent Done false
21953 6773 Acked Read 4 Done Sent Done false
21954 6774 Acked Read 10 Done Sent Done false
21955 6775 Acked Write 4 Done Sent Done false
21956 6776 Acked Write 8 Done Sent Done false
21957 6777 Acked Read 1 Done Sent Done false
21958 6778 Acked Write 5 Done Sent Done false
21959 6779 Acked Read 4 Done Sent Done false
21960 6780 Acked Read 10 Done Sent Done false
21961 6781 Acked Write 2 Done Sent Done false
21962 6782 Acked Write 2 Done Sent Done false
21963 6783 Acked Read 3 Done Sent Done false
21964 6784 Acked Read 9 Done Sent Done false
21965 6785 Acked Write 4 Done Sent Done false
21966 6786 Acked Read 9 Done Sent Done false
21967 6787 Acked Flush 0 Done Sent Done false
21968 6788 Acked Read 3 Done Sent Done false
21969 6789 Acked Read 10 Done Sent Done false
21970 6790 Acked Read 10 Done Sent Done false
21971 6791 Acked Read 3 Done Sent Done false
21972 6792 Acked Flush 0 Done Sent Done false
21973 6793 Acked Write 7 Done Sent Done false
21974 6794 Acked Read 9 Done Sent Done false
21975 6795 Acked Write 1 Done Sent Done false
21976 6796 Acked Write 1 Done Sent Done false
21977 6797 Acked Write 7 Done Sent Done false
21978 6798 Acked Read 4 Done Sent Done false
21979 6799 Acked Write 4 Done Sent Done false
21980 6800 Acked Write 4 Done Sent Done false
21981 6801 Acked Write 6 Done Sent Done false
21982 6802 Acked Write 3 Done Sent Done false
21983 6803 Acked Read 6 Done Sent Done false
21984 6804 Acked Read 4 Done Sent Done false
21985 6805 Acked Write 9 Done Sent Done false
21986 6806 Acked Write 5 Done Sent Done false
21987 6807 Acked Read 2 Done Sent Done false
21988 6808 Acked Read 8 Done Sent Done false
21989 6809 Acked Write 9 Done Sent Done false
21990 6810 Acked Read 3 Done Sent Done false
21991 6811 Acked Read 6 Done Sent Done false
21992 6812 Acked Read 3 Done Sent Done false
21993 6813 Acked Read 3 Done Sent Done false
21994 6814 Acked Write 10 Done Sent Done false
21995 6815 Acked Read 9 Done Sent Done false
21996 6816 Acked Read 9 Done Sent Done false
21997 6817 Acked Read 3 Done Sent Done false
21998 6818 Acked Read 1 Done Sent Done false
21999 6819 Acked Read 2 Done Sent Done false
22000 6820 Acked Write 3 Done Sent Done false
22001 6821 Acked Read 3 Done Sent Done false
22002 6822 Acked Write 7 Done Sent Done false
22003 6823 Acked Write 6 Done Sent Done false
22004 6824 Acked Read 8 Done Sent Done false
22005 6825 Acked Write 7 Done Sent Done false
22006 6826 Acked Read 3 Done Sent Done false
22007 6827 Acked Read 1 Done Sent Done false
22008 6828 Acked Write 5 Done Sent Done false
22009 6829 Acked Flush 0 Done Sent Done false
22010 6830 Acked Flush 0 Done Sent Done false
22011 6831 Acked Read 9 Done Sent Done false
22012 6832 Acked Write 10 Done Sent Done false
22013 6833 Acked Read 6 Done Sent Done false
22014 6834 Acked Flush 0 Done Sent Done false
22015 6835 Acked Read 6 Done Sent Done false
22016 6836 Acked Write 4 Done Sent Done false
22017 6837 Acked Write 7 Done Sent Done false
22018 6838 Acked Read 4 Done Sent Done false
22019 6839 Acked Flush 0 Done Sent Done false
22020 6840 Acked Write 6 Done Sent Done false
22021 6841 Acked Read 6 Done Sent Done false
22022 6842 Acked Read 7 Done Sent Done false
22023 6843 Acked Read 3 Done Sent Done false
22024 6844 Acked Read 6 Done Sent Done false
22025 6845 Acked Read 1 Done Sent Done false
22026 6846 Acked Read 5 Done Sent Done false
22027 6847 Acked Read 3 Done Sent Done false
22028 6848 Acked Read 6 Done Sent Done false
22029 6849 Acked Write 10 Done Sent Done false
22030 6850 Acked Write 3 Done Sent Done false
22031 6851 Acked Read 9 Done Sent Done false
22032 6852 Acked Read 9 Done Sent Done false
22033 6853 Acked Read 3 Done Sent Done false
22034 6854 Acked Write 2 Done Sent Done false
22035 6855 Acked Write 7 Done Sent Done false
22036 6856 Acked Flush 0 Done Sent Done false
22037 6857 Acked Write 4 Done Sent Done false
22038 6858 Acked Read 1 Done Sent Done false
22039 6859 Acked Write 2 Done Sent Done false
22040 6860 Acked Read 6 Done Sent Done false
22041 6861 Acked Write 3 Done Sent Done false
22042 6862 Acked Write 10 Done Sent Done false
22043 6863 Acked Write 4 Done Sent Done false
22044 6864 Acked Read 2 Done Sent Done false
22045 6865 Acked Read 5 Done Sent Done false
22046 6866 Acked Write 9 Done Sent Done false
22047 6867 Acked Read 8 Done Sent Done false
22048 6868 Acked Write 10 Done Sent Done false
22049 6869 Acked Flush 0 Done Sent Done false
22050 6870 Acked Read 3 Done Sent Done false
22051 6871 Acked Write 3 Done Sent Done false
22052 6872 Acked Read 6 Done Sent Done false
22053 6873 Acked Read 9 Done Sent Done false
22054 6874 Acked Read 1 Done Sent Done false
22055 6875 Acked Read 5 Done Sent Done false
22056 6876 Acked Read 9 Done Sent Done false
22057 6877 Acked Write 4 Done Sent Done false
22058 6878 Acked Read 7 Done Sent Done false
22059 6879 Acked Write 10 Done Sent Done false
22060 6880 Acked Read 8 Done Sent Done false
22061 6881 Acked Read 10 Done Sent Done false
22062 6882 Acked Read 9 Done Sent Done false
22063 6883 Acked Read 10 Done Sent Done false
22064 6884 Acked Read 3 Done Sent Done false
22065 6885 Acked Read 5 Done Sent Done false
22066 6886 Acked Read 10 Done Sent Done false
22067 6887 Acked Read 3 Done Sent Done false
22068 6888 Acked Write 8 Done Sent Done false
22069 6889 Acked Flush 0 Done Sent Done false
22070 6890 Acked Write 1 Done Sent Done false
22071 6891 Acked Read 5 Done Sent Done false
22072 6892 Acked Write 5 Done Sent Done false
22073 6893 Acked Read 8 Done Sent Done false
22074 6894 Acked Write 8 Done Sent Done false
22075 6895 Acked Write 6 Done Sent Done false
22076 6896 Acked Write 8 Done Sent Done false
22077 6897 Acked Write 4 Done Sent Done false
22078 6898 Acked Read 9 Done Sent Done false
22079 6899 Acked Write 5 Done Sent Done false
22080 6900 Acked Write 3 Done Sent Done false
22081 6901 Acked Read 7 Done Sent Done false
22082 6902 Acked Flush 0 Done Sent Done false
22083 6903 Acked Read 3 Done Sent Done false
22084 6904 Acked Read 1 Done Sent Done false
22085 6905 Acked Read 1 Done Sent Done false
22086 6906 Acked Read 2 Done Sent Done false
22087 6907 Acked Write 2 Done Sent Done false
22088 6908 Acked Flush 0 Done Sent Done false
22089 6909 Acked Read 1 Done Sent Done false
22090 6910 Acked Write 3 Done Sent Done false
22091 6911 Acked Flush 0 Done Sent Done false
22092 6912 Acked Read 7 Done Sent Done false
22093 6913 Acked Write 6 Done Sent Done false
22094 6914 Acked Write 2 Done Sent Done false
22095 6915 Acked Write 1 Done Sent Done false
22096 6916 Acked Write 10 Done Sent Done false
22097 6917 Acked Write 4 Done Sent Done false
22098 6918 Acked Read 8 Done Sent Done false
22099 6919 Acked Read 3 Done Sent Done false
22100 6920 Acked Read 8 Done Sent Done false
22101 6921 Acked Write 2 Done Sent Done false
22102 6922 Acked Write 6 Done Sent Done false
22103 6923 Acked Read 2 Done Sent Done false
22104 6924 Acked Write 3 Done Sent Done false
22105 6925 Acked Write 9 Done Sent Done false
22106 6926 Acked Read 6 Done Sent Done false
22107 6927 Acked Read 8 Done Sent Done false
22108 6928 Acked Read 1 Done Sent Done false
22109 6929 Acked Read 1 Done Sent Done false
22110 6930 Acked Read 8 Done Sent Done false
22111 6931 Acked Read 4 Done Sent Done false
22112 6932 Acked Write 9 Done Sent Done false
22113 6933 Acked Read 9 Done Sent Done false
22114 6934 Acked Write 7 Done Sent Done false
22115 6935 Acked Write 6 Done Sent Done false
22116 6936 Acked Write 5 Done Sent Done false
22117 6937 Acked Flush 0 Done Sent Done false
22118 6938 Acked Flush 0 Done Sent Done false
22119 6939 Acked Write 4 Done Sent Done false
22120 6940 Acked Write 4 Done Sent Done false
22121 6941 Acked Flush 0 Done Sent Done false
22122 6942 Acked Read 4 Done Sent Done false
22123 6943 Acked Write 4 Done Sent Done false
22124 6944 Acked Read 5 Done Sent Done false
22125 6945 Acked Read 4 Done Sent Done false
22126 6946 Acked Write 10 Done Sent Done false
22127 6947 Acked Read 9 Done Sent Done false
22128 6948 Acked Write 7 Done Sent Done false
22129 6949 Acked Write 10 Done Sent Done false
22130 6950 Acked Read 7 Done Sent Done false
22131 6951 Acked Flush 0 Done Sent Done false
22132 6952 Acked Read 9 Done Sent Done false
22133 6953 Acked Write 9 Done Sent Done false
22134 6954 Acked Read 9 Done Sent Done false
22135 6955 Acked Read 7 Done Sent Done false
22136 6956 Acked Write 9 Done Sent Done false
22137 6957 Acked Flush 0 Done Sent Done false
22138 6958 Acked Write 10 Done Sent Done false
22139 6959 Acked Write 4 Done Sent Done false
22140 6960 Acked Read 8 Done Sent Done false
22141 6961 Acked Write 2 Done Sent Done false
22142 6962 Acked Read 8 Done Sent Done false
22143 6963 Acked Write 9 Done Sent Done false
22144 6964 Acked Read 8 Done Sent Done false
22145 6965 Acked Write 10 Done Sent Done false
22146 6966 Acked Read 4 Done Sent Done false
22147 6967 Acked Read 6 Done Sent Done false
22148 6968 Acked Read 6 Done Sent Done false
22149 6969 Acked Read 2 Done Sent Done false
22150 6970 Acked Read 4 Done Sent Done false
22151 6971 Acked Write 2 Done Sent Done false
22152 6972 Acked Write 10 Done Sent Done false
22153 6973 Acked Flush 0 Done Sent Done false
22154 6974 Acked Read 4 Done Sent Done false
22155 6975 Acked Read 5 Done Sent Done false
22156 6976 Acked Read 8 Done Sent Done false
22157 6977 Acked Write 7 Done Sent Done false
22158 6978 Acked Read 9 Done Sent Done false
22159 6979 Acked Read 1 Done Sent Done false
22160 6980 Acked Read 10 Done Sent Done false
22161 6981 Acked Write 9 Done Sent Done false
22162 6982 Acked Read 7 Done Sent Done false
22163 6983 Acked Read 7 Done Sent Done false
22164 6984 Acked Flush 0 Done Sent Done false
22165 6985 Acked Read 7 Done Sent Done false
22166 6986 Acked Flush 0 Done Sent Done false
22167 6987 Acked Read 3 Done Sent Done false
22168 6988 Acked Read 3 Done Sent Done false
22169 6989 Acked Flush 0 Done Sent Done false
22170 6990 Acked Write 5 Done Sent Done false
22171 6991 Acked Write 8 Done Sent Done false
22172 6992 Acked Write 2 Done Sent Done false
22173 6993 Acked Read 4 Done Sent Done false
22174 6994 Acked Read 9 Done Sent Done false
22175 6995 Acked Read 4 Done Sent Done false
22176 6996 Acked Write 2 Done Sent Done false
22177 6997 Acked Write 6 Done Sent Done false
22178 6998 Acked Flush 0 Done Sent Done false
22179 6999 Acked Read 3 Done Sent Done false
22180 7000 Acked Write 10 Done Sent Done false
22181 7001 Acked Write 6 Done Sent Done false
22182 7002 Acked Write 8 Done Sent Done false
22183 7003 Acked Write 1 Done Sent Done false
22184 7004 Acked Read 10 Done Sent Done false
22185 7005 Acked Read 6 Done Sent Done false
22186 7006 Acked Write 8 Done Sent Done false
22187 7007 Acked Read 1 Done Sent Done false
22188 7008 Acked Read 9 Done Sent Done false
22189 7009 Acked Read 1 Done Sent Done false
22190 7010 Acked Write 4 Done Sent Done false
22191 7011 Acked Read 8 Done Sent Done false
22192 7012 Acked Write 5 Done Sent Done false
22193 7013 Acked Flush 0 Done Sent Done false
22194 7014 Acked Write 4 Done Sent Done false
22195 7015 Acked Write 8 Done Sent Done false
22196 7016 Acked Read 5 Done Sent Done false
22197 7017 Acked Read 6 Done Sent Done false
22198 7018 Acked Write 6 Done Sent Done false
22199 7019 Acked Write 7 Done Sent Done false
22200 7020 Acked Write 10 Done Sent Done false
22201 7021 Acked Write 4 Done Sent Done false
22202 7022 Acked Flush 0 Done Sent Done false
22203 7023 Acked Write 9 Done Sent Done false
22204 7024 Acked Read 6 Done Sent Done false
22205 7025 Acked Write 10 Done Sent Done false
22206 7026 Acked Flush 0 Done Sent Done false
22207 7027 Acked Write 9 Done Sent Done false
22208 7028 Acked Write 10 Done Sent Done false
22209 7029 Acked Write 9 Done Sent Done false
22210 7030 Acked Flush 0 Done Sent Done false
22211 7031 Acked Write 9 Done Sent Done false
22212 7032 Acked Write 1 Done Sent Done false
22213 7033 Acked Read 7 Done Sent Done false
22214 7034 Acked Write 3 Done Sent Done false
22215 7035 Acked Read 2 Done Sent Done false
22216 7036 Acked Read 6 Done Sent Done false
22217 7037 Acked Read 5 Done Sent Done false
22218 7038 Acked Flush 0 Done Sent Done false
22219 STATES DS:0 DS:1 DS:2 TOTAL
22220 Sent 0 302 0 302
22221 Done 302 0 302 604
22222 Skipped 0 0 0 0
22223 Error 0 0 0 0
22224 Last Flush: 7038 6736 7038
22225 Downstairs last five completed: 6736 6735 6734 6733 6732
22226 Upstairs last five completed: 7038 7037 7036 7035 7034
22227 CLIENT: Up:0 ds:302 act:2
222282025-01-03T08:35:32.347ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222292025-01-03T08:35:32.347ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35422) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222302025-01-03T08:35:32.348ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222312025-01-03T08:35:32.349ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222322025-01-03T08:35:32.349ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222332025-01-03T08:35:32.349ZINFOcrucible: send last flush ID to this DS: 6736 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222342025-01-03T08:35:32.349ZINFOcrucible: Replied this last flush ID: 6736 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222352025-01-03T08:35:32.349ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222362025-01-03T08:35:32.349ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222372025-01-03T08:35:32.349ZINFOcrucible: [1] client re-new 302 jobs since flush 6736 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222382025-01-03T08:35:32.349ZINFOcrucible: [1] Replayed 302 jobs since flush: 6736 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
22239 ----------------------------------------------------------------
22240 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22241 Downstairs last five completed: 7038 7037 7036 7035 7034
22242 Upstairs last five completed: 7038 7037 7036 7035 7034
22243 CLIENT: Up:0 ds:0 act:3
22244 Replay: All jobs finished
22245 Main task has work to do, go find it
22246 got dsc Stop(2)
22247 stop 2
22248 Jan 03 08:35:34.599[8830] Got stop action so:false kr:false
22249 INFO request completed, latency_us: 377, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 1717c956-2e3c-4376-b2cb-8c5757736a40, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
222502025-01-03T08:35:34.600ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222512025-01-03T08:35:34.600ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222522025-01-03T08:35:34.600ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222532025-01-03T08:35:34.600ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
222542025-01-03T08:35:34.600ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
22255 Jan 03 08:35:34.600 INFO request completed, latency_us: 130, response_code: 200, uri: /state/cid/2, method: GET{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:35:34.600617603Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
22256 , req_id: cfbca2b5-45b7-48ba-93e9-c6cf261382d9, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
22257 [8830] Exited with: ExitStatus(unix_wait_status(9))
22258 [8830][2] reports Exit
22259 Jan 03 08:35:38.602 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/2, method: GET, req_id: b6ffeb42-fbad-474a-ae7b-044dc35a3743, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
22260 001/300 Flush
22261 002/300 Read block 4079 len 28672
22262 003/300 Read block 1962 len 8192
22263 004/300 Write block 3766 len 36864 data: 3 4 4 5 5 5 6 5 4
22264 005/300 Write block 2332 len 12288 data: 4 3 3
22265 006/300 Read block 4750 len 8192
22266 007/300 Flush
22267 008/300 Read block 3476 len 40960
22268 009/300 Read block 972 len 12288
22269 010/300 Write block 2874 len 8192 data: 4 4
22270 011/300 Write block 3191 len 36864 data: 5 5 5 6 6 7 6 6 6
22271 012/300 Write block 1848 len 20480 data: 4 4 4 5 5
22272 013/300 Write block 1518 len 36864 data: 4 5 5 4 4 6 6 6 5
22273 014/300 Flush
22274 015/300 Read block 3778 len 24576
22275 016/300 Read block 4670 len 28672
22276 017/300 Flush
22277 018/300 Read block 1928 len 36864
22278 019/300 Write block 4387 len 12288 data: 6 5 5
22279 020/300 Read block 205 len 4096
22280 021/300 Write block 50 len 32768 data: 2 2 2 4 5 5 5 4
22281 022/300 Write block 3297 len 24576 data: 4 4 3 2 4 5
22282 023/300 Write block 3684 len 12288 data: 2 2 2
22283 024/300 Write block 1523 len 40960 data: 7 7 7 6 4 3 4 4 3 4
22284 025/300 Flush
22285 026/300 Read block 1671 len 32768
22286 027/300 Read block 1966 len 8192
22287 028/300 Write block 2095 len 4096 data: 5
22288 029/300 Flush
22289 030/300 Read block 2991 len 16384
22290 031/300 Write block 542 len 12288 data: 5 5 4
22291 032/300 Read block 3452 len 28672
22292 033/300 Read block 2502 len 16384
22293 034/300 Write block 3284 len 40960 data: 5 4 5 4 5 5 5 6 5 4
22294 035/300 Read block 2698 len 36864
22295 036/300 Flush
22296 037/300 Read block 1122 len 32768
22297 038/300 Write block 4274 len 12288 data: 3 3 4
22298 039/300 Read block 3736 len 12288
22299 040/300 Read block 653 len 8192
22300 041/300 Write block 2007 len 4096 data: 2
22301 042/300 Read block 2170 len 12288
22302 043/300 Read block 1377 len 40960
22303 044/300 Read block 3137 len 40960
22304 045/300 Write block 3123 len 24576 data: 2 3 4 5 5 4
22305 046/300 Read block 3523 len 4096
22306 047/300 Write block 3763 len 4096 data: 5
22307 048/300 Flush
22308 049/300 Write block 4380 len 20480 data: 3 4 4 4 3
22309 050/300 Read block 4452 len 16384
22310 051/300 Read block 4421 len 28672
22311 052/300 Write block 2661 len 4096 data: 8
22312 053/300 Read block 4078 len 36864
22313 054/300 Write block 3154 len 8192 data: 3 3
22314 055/300 Read block 1782 len 8192
22315 056/300 Read block 262 len 16384
22316 057/300 Write block 4752 len 28672 data: 2 4 5 5 6 6 6
22317 058/300 Read block 4059 len 4096
22318 059/300 Write block 576 len 40960 data: 4 5 4 4 5 4 3 4 4 4
22319 060/300 Read block 2282 len 28672
22320 061/300 Read block 160 len 32768
22321 062/300 Write block 774 len 36864 data: 5 5 5 5 5 6 6 5 5
22322 063/300 Write block 4429 len 32768 data: 4 4 4 5 5 5 5 3
22323 064/300 Read block 2812 len 24576
22324 065/300 Read block 1416 len 28672
22325 066/300 Read block 2119 len 36864
22326 067/300 Flush
22327 068/300 Read block 584 len 32768
22328 069/300 Read block 954 len 36864
22329 070/300 Read block 4134 len 20480
22330 071/300 Read block 4722 len 12288
22331 072/300 Write block 4335 len 40960 data: 7 7 7 6 6 7 5 6 5 6
22332 073/300 Read block 413 len 24576
22333 074/300 Write block 1011 len 12288 data: 3 3 4
22334 075/300 Write block 557 len 4096 data: 5
22335 076/300 Read block 706 len 8192
22336 077/300 Write block 1038 len 40960 data: 3 2 2 2 2 3 3 3 5 4
22337 078/300 Read block 2945 len 36864
22338 079/300 Write block 4943 len 4096 data: 7
22339 080/300 Read block 4192 len 28672
22340 081/300 Read block 4262 len 12288
22341 082/300 Read block 3927 len 12288
22342 083/300 Flush
22343 084/300 Read block 2491 len 12288
22344 085/300 Flush
22345 086/300 Flush
22346 087/300 Read block 2989 len 36864
22347 088/300 Read block 1402 len 28672
22348 089/300 Read block 916 len 4096
22349 090/300 Flush
22350 091/300 Read block 2216 len 32768
22351 092/300 Write block 2693 len 16384 data: 4 4 6 6
22352 093/300 Write block 3585 len 24576 data: 3 3 4 3 4 3
22353 094/300 Read block 4835 len 36864
22354 095/300 Write block 3393 len 36864 data: 5 5 7 6 7 7 6 6 7
22355 096/300 Flush
22356 097/300 Read block 3742 len 16384
22357 098/300 Write block 3026 len 32768 data: 4 3 3 3 3 4 4 4
22358 099/300 Write block 2057 len 24576 data: 5 3 2 3 3 3
22359 100/300 Read block 4370 len 28672
22360 101/300 Write block 3764 len 24576 data: 4 3 4 5 5 6
22361 102/300 Read block 1817 len 32768
22362 103/300 Read block 2293 len 20480
22363 104/300 Write block 1649 len 40960 data: 5 5 6 5 5 4 4 2 3 2
22364 105/300 Flush
22365 106/300 Write block 85 len 40960 data: 5 5 6 5 5 4 2 2 3 4
22366 107/300 Read block 4393 len 28672
22367 108/300 Read block 1208 len 20480
22368 109/300 Read block 444 len 12288
22369 110/300 Write block 4649 len 28672 data: 7 3 3 3 3 3 3
22370 111/300 Flush
22371 112/300 Write block 3139 len 20480 data: 4 4 4 4 4
22372 113/300 Read block 3741 len 40960
22373 114/300 Read block 2631 len 40960
22374 115/300 Read block 2806 len 32768
22375 116/300 Write block 1731 len 4096 data: 6
22376 117/300 Write block 4541 len 40960 data: 4 5 5 5 4 4 4 4 4 4
22377 118/300 Write block 2211 len 32768 data: 5 4 5 7 6 5 6 6
22378 119/300 Read block 804 len 32768
22379 120/300 Write block 1552 len 12288 data: 3 3 3
22380 121/300 Read block 3275 len 36864
22381 122/300 Read block 4370 len 16384
22382 123/300 Flush
22383 124/300 Write block 3123 len 4096 data: 3
22384 125/300 Write block 14 len 16384 data: 4 3 4 4
22385 126/300 Read block 3724 len 20480
22386 127/300 Read block 4232 len 32768
22387 128/300 Flush
22388 129/300 Write block 4826 len 20480 data: 4 4 5 4 5
22389 130/300 Read block 219 len 16384
22390 131/300 Read block 3437 len 40960
22391 132/300 Read block 1728 len 32768
22392 133/300 Read block 2861 len 4096
22393 134/300 Read block 3521 len 20480
22394 135/300 Write block 2142 len 24576 data: 4 3 3 3 2 2
22395 136/300 Write block 3731 len 20480 data: 6 6 6 4 4
22396 137/300 Read block 901 len 20480
22397 138/300 Write block 3354 len 12288 data: 6 4 4
22398 139/300 Read block 242 len 28672
22399 140/300 Write block 4105 len 36864 data: 5 5 4 4 4 4 4 4 4
22400 141/300 Read block 32 len 12288
22401 142/300 Read block 1630 len 24576
22402 143/300 Read block 4433 len 20480
22403 144/300 Read block 867 len 40960
22404 145/300 Write block 1068 len 20480 data: 2 2 2 2 2
22405 146/300 Read block 3196 len 40960
22406 147/300 Read block 1638 len 16384
22407 148/300 Read block 4874 len 8192
22408 149/300 Write block 2494 len 24576 data: 4 5 5 3 4 3
22409 150/300 Write block 3725 len 32768 data: 4 4 3 3 4 6 7 7
22410 151/300 Write block 3218 len 28672 data: 4 5 6 6 6 6 6
22411 152/300 Read block 925 len 16384
22412 153/300 Write block 3382 len 24576 data: 3 4 4 6 8 8
22413 154/300 Read block 4167 len 8192
22414 155/300 Read block 4890 len 4096
22415 156/300 Read block 3965 len 36864
22416 157/300 Write block 2332 len 4096 data: 5
22417 158/300 Write block 4334 len 40960 data: 6 8 8 8 7 7 8 6 7 6
22418 159/300 Write block 3905 len 16384 data: 5 5 5 4
22419 160/300 Write block 2808 len 16384 data: 7 6 6 6
22420 161/300 Write block 3119 len 28672 data: 3 3 3 2 4 4 5
22421 162/300 Read block 3110 len 8192
22422 163/300 Write block 1090 len 32768 data: 6 7 6 7 7 7 6 7
22423 164/300 Write block 1417 len 20480 data: 3 4 6 7 6
22424 165/300 Read block 4454 len 8192
22425 166/300 Write block 3983 len 28672 data: 2 2 3 3 3 3 3
22426 167/300 Write block 3296 len 4096 data: 3
22427 168/300 Write block 3990 len 24576 data: 2 2 2 2 4 4
22428 169/300 Flush
22429 170/300 Write block 1768 len 12288 data: 2 2 2
22430 171/300 Read block 1430 len 20480
22431 172/300 Write block 690 len 12288 data: 4 4 6
22432 173/300 Read block 4341 len 40960
22433 174/300 Write block 3887 len 12288 data: 6 5 5
22434 175/300 Read block 4758 len 8192
22435 176/300 Read block 824 len 16384
22436 177/300 Write block 4822 len 24576 data: 5 3 4 4 5 5
22437 178/300 Write block 2710 len 20480 data: 7 8 7 7 6
22438 179/300 Write block 2496 len 32768 data: 6 4 5 4 3 3 3 4
22439 180/300 Read block 2792 len 12288
22440 181/300 Write block 2687 len 40960 data: 7 6 6 6 4 4 5 5 7 7
22441 182/300 Read block 343 len 28672
22442 183/300 Flush
22443 184/300 Read block 204 len 12288
22444 185/300 Read block 1424 len 28672
22445 186/300 Write block 3255 len 8192 data: 4 4
22446 187/300 Write block 1154 len 32768 data: 7 6 5 5 6 8 8 6
22447 188/300 Write block 1762 len 40960 data: 3 2 2 2 2 2 3 3 3 2
22448 189/300 Read block 75 len 24576
22449 190/300 Write block 2892 len 28672 data: 3 3 3 3 3 4 3
22450 191/300 Read block 592 len 16384
22451 192/300 Read block 1332 len 20480
22452 193/300 Read block 1543 len 28672
22453 194/300 Read block 4880 len 8192
22454 195/300 Read block 1150 len 8192
22455 196/300 Write block 2124 len 36864 data: 5 5 5 5 5 5 6 5 5
22456 197/300 Read block 2181 len 16384
22457 198/300 Read block 2416 len 20480
22458 199/300 Write block 358 len 4096 data: 4
22459 200/300 Write block 730 len 32768 data: 4 6 6 6 6 6 7 6
22460 201/300 Flush
22461 202/300 Read block 2175 len 32768
22462 203/300 Read block 4409 len 36864
22463 204/300 Read block 4393 len 24576
22464 205/300 Read block 3547 len 36864
22465 206/300 Read block 2467 len 8192
22466 207/300 Write block 904 len 36864 data: 3 3 4 6 6 7 6 5 7
22467 208/300 Read block 3476 len 8192
22468 209/300 Write block 948 len 20480 data: 6 6 6 6 4
22469 210/300 Read block 2136 len 40960
22470 211/300 Read block 687 len 20480
22471 212/300 Write block 4964 len 32768 data: 4 6 6 6 6 5 5 5
22472 213/300 Read block 1650 len 12288
22473 214/300 Write block 4571 len 32768 data: 5 5 4 5 5 4 4 6
22474 215/300 Write block 296 len 16384 data: 4 3 2 2
22475 216/300 Write block 226 len 40960 data: 3 3 4 5 5 4 4 3 3 3
22476 217/300 Write block 3024 len 8192 data: 5 4
22477 218/300 Write block 4815 len 16384 data: 4 4 4 4
22478 219/300 Write block 3979 len 4096 data: 5
22479 220/300 Write block 4431 len 16384 data: 5 6 6 6
22480 221/300 Write block 1945 len 24576 data: 6 7 8 8 9 8
22481 222/300 Read block 4963 len 8192
22482 223/300 Write block 1279 len 36864 data: 5 5 5 4 5 5 4 4 4
22483 224/300 Read block 985 len 8192
22484 225/300 Read block 1771 len 40960
22485 226/300 Read block 942 len 40960
22486 227/300 Write block 4756 len 24576 data: 7 7 7 6 6 5
22487 228/300 Write block 3206 len 28672 data: 3 3 3 3 2 3 3
22488 229/300 Read block 4143 len 8192
22489 230/300 Read block 4858 len 32768
22490 231/300 Flush
22491 232/300 Write block 3109 len 32768 data: 3 3 3 3 3 3 3 3
22492 233/300 Read block 3489 len 28672
22493 234/300 Write block 3771 len 36864 data: 6 7 6 5 4 4 3 2 3
22494 235/300 Read block 3752 len 32768
22495 236/300 Write block 1058 len 16384 data: 2 2 2 2
22496 237/300 Write block 4812 len 32768 data: 4 4 4 5 5 5 5 4
22497 238/300 Flush
22498 239/300 Write block 1975 len 24576 data: 2 2 2 3 3 4
22499 240/300 Flush
22500 241/300 Flush
22501 242/300 Write block 2323 len 36864 data: 4 5 5 5 6 7 6 6 5
22502 243/300 Write block 4842 len 32768 data: 4 7 6 7 7 5 5 5
22503 244/300 Write block 4260 len 28672 data: 5 4 4 7 7 7 8
22504 245/300 Write block 3058 len 12288 data: 4 3 3
22505 246/300 Write block 2313 len 12288 data: 6 7 6
22506 247/300 Write block 4334 len 40960 data: 7 9 9 9 8 8 9 7 8 7
22507 248/300 Read block 3246 len 8192
22508 249/300 Read block 1465 len 8192
22509 250/300 Write block 4927 len 4096 data: 4
22510 251/300 Read block 1633 len 20480
22511 252/300 Write block 1797 len 4096 data: 4
22512 253/300 Read block 3733 len 20480
22513 254/300 Write block 3132 len 40960 data: 3 3 3 3 3 3 4 5 5 5
22514 255/300 Read block 3816 len 4096
22515 256/300 Read block 3847 len 8192
22516 257/300 Write block 2468 len 28672 data: 5 6 4 3 4 5 5
22517 258/300 Read block 3389 len 12288
22518 259/300 Read block 380 len 36864
22519 260/300 Read block 4187 len 20480
22520 261/300 Write block 3825 len 28672 data: 7 7 6 6 5 6 5
22521 262/300 Read block 1382 len 8192
22522 263/300 Flush
22523 264/300 Read block 479 len 24576
22524 265/300 Flush
22525 266/300 Read block 2278 len 40960
22526 267/300 Write block 440 len 32768 data: 7 5 4 3 4 4 4 6
22527 268/300 Write block 2132 len 8192 data: 6 5
22528 269/300 Read block 3773 len 32768
22529 270/300 Flush
22530 271/300 Write block 3071 len 12288 data: 4 4 4
22531 272/300 Read block 4637 len 20480
22532 273/300 Read block 2162 len 12288
22533 274/300 Read block 4914 len 20480
22534 275/300 Flush
22535 276/300 Write block 629 len 12288 data: 5 5 6
22536 277/300 Write block 3556 len 32768 data: 5 5 4 5 5 4 4 2
22537 278/300 Write block 497 len 32768 data: 7 5 4 4 6 5 4 4
22538 279/300 Read block 4863 len 40960
22539 280/300 Write block 171 len 8192 data: 4 6
22540 281/300 Read block 3085 len 24576
22541 282/300 Write block 1145 len 12288 data: 5 5 6
22542 283/300 Write block 4288 len 4096 data: 3
22543 284/300 Write block 171 len 28672 data: 5 7 5 5 6 5 4
22544 285/300 Flush
22545 286/300 Read block 674 len 12288
22546 287/300 Write block 3582 len 8192 data: 3 2
22547 288/300 Write block 2025 len 36864 data: 4 4 4 3 5 5 5 4 3
22548 289/300 Read block 3669 len 4096
22549 290/300 Write block 4701 len 20480 data: 3 5 6 7 8
22550 291/300 Write block 3681 len 20480 data: 3 3 3 3 3
22551 292/300 Read block 4116 len 12288
22552 293/300 Write block 2711 len 36864 data: 9 8 8 7 5 6 5 4 5
22553 294/300 Write block 4313 len 28672 data: 7 6 5 5 6 5 6
22554 295/300 Write block 4028 len 36864 data: 3 2 2 2 2 2 2 3 3
22555 296/300 Write block 1370 len 32768 data: 6 3 3 3 3 4 3 3
22556 297/300 Read block 4294 len 4096
22557 298/300 Read block 4758 len 28672
22558 299/300 Write block 3903 len 24576 data: 5 5 6 6 6 5
22559 300/300 Write block 1134 len 8192 data: 6 7
22560 301/300 Read block 2758 len 8192
22561 Main task has work to do, go find it
22562 got dsc Start(2)
22563 start 2
22564 Jan 03 08:35:38.840 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/2, method: POST, req_id: 007d80de-1b06-457d-830d-5c6477ee4745, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
22565 Replay: started 2, returned:Ok(())
22566 ----------------------------------------------------------------
22567 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22568 States: Active Active Offline
22569 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22570 7039 Acked Flush 0 Done Done Sent false
22571 7040 Acked Read 7 Done Done Sent false
22572 7041 Acked Read 2 Done Done Sent false
22573 7042 Acked Write 9 Done Done Sent false
22574 7043 Acked Write 3 Done Done Sent false
22575 7044 Acked Read 2 Done Done Sent false
22576 7045 Acked Flush 0 Done Done Sent false
22577 7046 Acked Read 10 Done Done Sent false
22578 7047 Acked Read 3 Done Done Sent false
22579 7048 Acked Write 2 Done Done Sent false
22580 7049 Acked Write 9 Done Done Sent false
22581 7050 Acked Write 5 Done Done Sent false
22582 7051 Acked Write 9 Done Done Sent false
22583 7052 Acked Flush 0 Done Done Sent false
22584 7053 Acked Read 6 Done Done Sent false
22585 7054 Acked Read 7 Done Done Sent false
22586 7055 Acked Flush 0 Done Done Sent false
22587 7056 Acked Read 9 Done Done Sent false
22588 7057 Acked Write 3 Done Done Sent false
22589 7058 Acked Read 1 Done Done Sent false
22590 7059 Acked Write 8 Done Done Sent false
22591 7060 Acked Write 6 Done Done Sent false
22592 7061 Acked Write 3 Done Done Sent false
22593 7062 Acked Write 10 Done Done Sent false
22594 7063 Acked Flush 0 Done Done Sent false
22595 7064 Acked Read 8 Done Done Sent false
22596 7065 Acked Read 2 Done Done Sent false
22597 7066 Acked Write 1 Done Done Sent false
22598 7067 Acked Flush 0 Done Done Sent false
22599 7068 Acked Read 4 Done Done Sent false
22600 7069 Acked Write 3 Done Done Sent false
22601 7070 Acked Read 7 Done Done Sent false
22602 7071 Acked Read 4 Done Done Sent false
22603 7072 Acked Write 10 Done Done Sent false
22604 7073 Acked Read 9 Done Done Sent false
22605 7074 Acked Flush 0 Done Done Sent false
22606 7075 Acked Read 8 Done Done Sent false
22607 7076 Acked Write 3 Done Done Sent false
22608 7077 Acked Read 3 Done Done Sent false
22609 7078 Acked Read 2 Done Done Sent false
22610 7079 Acked Write 1 Done Done Sent false
22611 7080 Acked Read 3 Done Done Sent false
22612 7081 Acked Read 10 Done Done Sent false
22613 7082 Acked Read 10 Done Done Sent false
22614 7083 Acked Write 6 Done Done Sent false
22615 7084 Acked Read 1 Done Done Sent false
22616 7085 Acked Write 1 Done Done Sent false
22617 7086 Acked Flush 0 Done Done Sent false
22618 7087 Acked Write 5 Done Done Sent false
22619 7088 Acked Read 4 Done Done Sent false
22620 7089 Acked Read 7 Done Done Sent false
22621 7090 Acked Write 1 Done Done Sent false
22622 7091 Acked Read 9 Done Done Sent false
22623 7092 Acked Write 2 Done Done Sent false
22624 7093 Acked Read 2 Done Done Sent false
22625 7094 Acked Read 4 Done Done Sent false
22626 7095 Acked Write 7 Done Done Sent false
22627 7096 Acked Read 1 Done Done Sent false
22628 7097 Acked Write 10 Done Done Sent false
22629 7098 Acked Read 7 Done Done Sent false
22630 7099 Acked Read 8 Done Done Sent false
22631 7100 Acked Write 9 Done Done Sent false
22632 7101 Acked Write 8 Done Done Sent false
22633 7102 Acked Read 6 Done Done Sent false
22634 7103 Acked Read 7 Done Done Sent false
22635 7104 Acked Read 9 Done Done Sent false
22636 7105 Acked Flush 0 Done Done Sent false
22637 7106 Acked Read 8 Done Done Sent false
22638 7107 Acked Read 9 Done Done Sent false
22639 7108 Acked Read 5 Done Done Sent false
22640 7109 Acked Read 3 Done Done Sent false
22641 7110 Acked Write 10 Done Done Sent false
22642 7111 Acked Read 6 Done Done Sent false
22643 7112 Acked Write 3 Done Done Sent false
22644 7113 Acked Write 1 Done Done Sent false
22645 7114 Acked Read 2 Done Done Sent false
22646 7115 Acked Write 10 Done Done Sent false
22647 7116 Acked Read 9 Done Done Sent false
22648 7117 Acked Write 1 Done Done Sent false
22649 7118 Acked Read 7 Done Done Sent false
22650 7119 Acked Read 3 Done Done Sent false
22651 7120 Acked Read 3 Done Done Sent false
22652 7121 Acked Flush 0 Done Done Sent false
22653 7122 Acked Read 3 Done Done Sent false
22654 7123 Acked Flush 0 Done Done Sent false
22655 7124 Acked Flush 0 Done Done Sent false
22656 7125 Acked Read 9 Done Done Sent false
22657 7126 Acked Read 7 Done Done Sent false
22658 7127 Acked Read 1 Done Done Sent false
22659 7128 Acked Flush 0 Done Done Sent false
22660 7129 Acked Read 8 Done Done Sent false
22661 7130 Acked Write 4 Done Done Sent false
22662 7131 Acked Write 6 Done Done Sent false
22663 7132 Acked Read 9 Done Done Sent false
22664 7133 Acked Write 9 Done Done Sent false
22665 7134 Acked Flush 0 Done Done Sent false
22666 7135 Acked Read 4 Done Done Sent false
22667 7136 Acked Write 8 Done Done Sent false
22668 7137 Acked Write 6 Done Done Sent false
22669 7138 Acked Read 7 Done Done Sent false
22670 7139 Acked Write 6 Done Done Sent false
22671 7140 Acked Read 8 Done Done Sent false
22672 7141 Acked Read 5 Done Done Sent false
22673 7142 Acked Write 10 Done Done Sent false
22674 7143 Acked Flush 0 Done Done Sent false
22675 7144 Acked Write 10 Done Done Sent false
22676 7145 Acked Read 7 Done Done Sent false
22677 7146 Acked Read 5 Done Done Sent false
22678 7147 Acked Read 3 Done Done Sent false
22679 7148 Acked Write 7 Done Done Sent false
22680 7149 Acked Flush 0 Done Done Sent false
22681 7150 Acked Write 5 Done Done Sent false
22682 7151 Acked Read 10 Done Done Sent false
22683 7152 Acked Read 10 Done Done Sent false
22684 7153 Acked Read 8 Done Done Sent false
22685 7154 Acked Write 1 Done Done Sent false
22686 7155 Acked Write 10 Done Done Sent false
22687 7156 Acked Write 8 Done Done Sent false
22688 7157 Acked Read 8 Done Done Sent false
22689 7158 Acked Write 3 Done Done Sent false
22690 7159 Acked Read 9 Done Done Sent false
22691 7160 Acked Read 4 Done Done Sent false
22692 7161 Acked Flush 0 Done Done Sent false
22693 7162 Acked Write 1 Done Done Sent false
22694 7163 Acked Write 4 Done Done Sent false
22695 7164 Acked Read 5 Done Done Sent false
22696 7165 Acked Read 8 Done Done Sent false
22697 7166 Acked Flush 0 Done Done Sent false
22698 7167 Acked Write 5 Done Done Sent false
22699 7168 Acked Read 4 Done Done Sent false
22700 7169 Acked Read 10 Done Done Sent false
22701 7170 Acked Read 8 Done Done Sent false
22702 7171 Acked Read 1 Done Done Sent false
22703 7172 Acked Read 5 Done Done Sent false
22704 7173 Acked Write 6 Done Done Sent false
22705 7174 Acked Write 5 Done Done Sent false
22706 7175 Acked Read 5 Done Done Sent false
22707 7176 Acked Write 3 Done Done Sent false
22708 7177 Acked Read 7 Done Done Sent false
22709 7178 Acked Write 9 Done Done Sent false
22710 7179 Acked Read 3 Done Done Sent false
22711 7180 Acked Read 6 Done Done Sent false
22712 7181 Acked Read 5 Done Done Sent false
22713 7182 Acked Read 10 Done Done Sent false
22714 7183 Acked Write 5 Done Done Sent false
22715 7184 Acked Read 10 Done Done Sent false
22716 7185 Acked Read 4 Done Done Sent false
22717 7186 Acked Read 2 Done Done Sent false
22718 7187 Acked Write 6 Done Done Sent false
22719 7188 Acked Write 8 Done Done Sent false
22720 7189 Acked Write 7 Done Done Sent false
22721 7190 Acked Read 4 Done Done Sent false
22722 7191 Acked Write 6 Done Done Sent false
22723 7192 Acked Read 2 Done Done Sent false
22724 7193 Acked Read 1 Done Done Sent false
22725 7194 Acked Read 9 Done Done Sent false
22726 7195 Acked Write 1 Done Done Sent false
22727 7196 Acked Write 10 Done Done Sent false
22728 7197 Acked Write 4 Done Done Sent false
22729 7198 Acked Write 4 Done Done Sent false
22730 7199 Acked Write 7 Done Done Sent false
22731 7200 Acked Read 2 Done Done Sent false
22732 7201 Acked Write 8 Done Done Sent false
22733 7202 Acked Write 5 Done Done Sent false
22734 7203 Acked Read 2 Done Done Sent false
22735 7204 Acked Write 7 Done Done Sent false
22736 7205 Acked Write 1 Done Done Sent false
22737 7206 Acked Write 6 Done Done Sent false
22738 7207 Acked Flush 0 Done Done Sent false
22739 7208 Acked Write 3 Done Done Sent false
22740 7209 Acked Read 5 Done Done Sent false
22741 7210 Acked Write 3 Done Done Sent false
22742 7211 Acked Read 10 Done Done Sent false
22743 7212 Acked Write 3 Done Done Sent false
22744 7213 Acked Read 2 Done Done Sent false
22745 7214 Acked Read 4 Done Done Sent false
22746 7215 Acked Write 6 Done Done Sent false
22747 7216 Acked Write 5 Done Done Sent false
22748 7217 Acked Write 8 Done Done Sent false
22749 7218 Acked Read 3 Done Done Sent false
22750 7219 Acked Write 10 Done Done Sent false
22751 7220 Acked Read 7 Done Done Sent false
22752 7221 Acked Flush 0 Done Done Sent false
22753 7222 Acked Read 3 Done Done Sent false
22754 7223 Acked Read 7 Done Done Sent false
22755 7224 Acked Write 2 Done Done Sent false
22756 7225 Acked Write 8 Done Done Sent false
22757 7226 Acked Write 10 Done Done Sent false
22758 7227 Acked Read 6 Done Done Sent false
22759 7228 Acked Write 7 Done Done Sent false
22760 7229 Acked Read 4 Done Done Sent false
22761 7230 Acked Read 5 Done Done Sent false
22762 7231 Acked Read 7 Done Done Sent false
22763 7232 Acked Read 2 Done Done Sent false
22764 7233 Acked Read 2 Done Done Sent false
22765 7234 Acked Write 9 Done Done Sent false
22766 7235 Acked Read 4 Done Done Sent false
22767 7236 Acked Read 5 Done Done Sent false
22768 7237 Acked Write 1 Done Done Sent false
22769 7238 Acked Write 8 Done Done Sent false
22770 7239 Acked Flush 0 Done Done Sent false
22771 7240 Acked Read 8 Done Done Sent false
22772 7241 Acked Read 9 Done Done Sent false
22773 7242 Acked Read 6 Done Done Sent false
22774 7243 Acked Read 9 Done Done Sent false
22775 7244 Acked Read 2 Done Done Sent false
22776 7245 Acked Write 9 Done Done Sent false
22777 7246 Acked Read 2 Done Done Sent false
22778 7247 Acked Write 5 Done Done Sent false
22779 7248 Acked Read 10 Done Done Sent false
22780 7249 Acked Read 5 Done Done Sent false
22781 7250 Acked Write 8 Done Done Sent false
22782 7251 Acked Read 3 Done Done Sent false
22783 7252 Acked Write 8 Done Done Sent false
22784 7253 Acked Write 4 Done Done Sent false
22785 7254 Acked Write 10 Done Done Sent false
22786 7255 Acked Write 2 Done Done Sent false
22787 7256 Acked Write 4 Done Done Sent false
22788 7257 Acked Write 1 Done Done Sent false
22789 7258 Acked Write 4 Done Done Sent false
22790 7259 Acked Write 6 Done Done Sent false
22791 7260 Acked Read 2 Done Done Sent false
22792 7261 Acked Write 9 Done Done Sent false
22793 7262 Acked Read 2 Done Done Sent false
22794 7263 Acked Read 10 Done Done Sent false
22795 7264 Acked Read 10 Done Done Sent false
22796 7265 Acked Write 6 Done Done Sent false
22797 7266 Acked Write 7 Done Done Sent false
22798 7267 Acked Read 2 Done Done Sent false
22799 7268 Acked Read 8 Done Done Sent false
22800 7269 Acked Flush 0 Done Done Sent false
22801 7270 Acked Write 8 Done Done Sent false
22802 7271 Acked Read 7 Done Done Sent false
22803 7272 Acked Write 9 Done Done Sent false
22804 7273 Acked Read 8 Done Done Sent false
22805 7274 Acked Write 4 Done Done Sent false
22806 7275 Acked Write 8 Done Done Sent false
22807 7276 Acked Flush 0 Done Done Sent false
22808 7277 Acked Write 6 Done Done Sent false
22809 7278 Acked Flush 0 Done Done Sent false
22810 7279 Acked Flush 0 Done Done Sent false
22811 7280 Acked Write 9 Done Done Sent false
22812 7281 Acked Write 8 Done Done Sent false
22813 7282 Acked Write 7 Done Done Sent false
22814 7283 Acked Write 3 Done Done Sent false
22815 7284 Acked Write 3 Done Done Sent false
22816 7285 Acked Write 10 Done Done Sent false
22817 7286 Acked Read 2 Done Done Sent false
22818 7287 Acked Read 2 Done Done Sent false
22819 7288 Acked Write 1 Done Done Sent false
22820 7289 Acked Read 5 Done Done Sent false
22821 7290 Acked Write 1 Done Done Sent false
22822 7291 Acked Read 5 Done Done Sent false
22823 7292 Acked Write 10 Done Done Sent false
22824 7293 Acked Read 1 Done Done Sent false
22825 7294 Acked Read 2 Done Done Sent false
22826 7295 Acked Write 7 Done Done Sent false
22827 7296 Acked Read 3 Done Done Sent false
22828 7297 Acked Read 9 Done Done Sent false
22829 7298 Acked Read 5 Done Done Sent false
22830 7299 Acked Write 7 Done Done Sent false
22831 7300 Acked Read 2 Done Done Sent false
22832 7301 Acked Flush 0 Done Done Sent false
22833 7302 Acked Read 6 Done Done Sent false
22834 7303 Acked Flush 0 Done Done Sent false
22835 7304 Acked Read 10 Done Done Sent false
22836 7305 Acked Write 8 Done Done Sent false
22837 7306 Acked Write 2 Done Done Sent false
22838 7307 Acked Read 8 Done Done Sent false
22839 7308 Acked Flush 0 Done Done Sent false
22840 7309 Acked Write 3 Done Done Sent false
22841 7310 Acked Read 5 Done Done Sent false
22842 7311 Acked Read 3 Done Done Sent false
22843 7312 Acked Read 5 Done Done Sent false
22844 7313 Acked Flush 0 Done Done Sent false
22845 7314 Acked Write 3 Done Done Sent false
22846 7315 Acked Write 8 Done Done Sent false
22847 7316 Acked Write 8 Done Done Sent false
22848 7317 Acked Read 10 Done Done Sent false
22849 7318 Acked Write 2 Done Done Sent false
22850 7319 Acked Read 6 Done Done Sent false
22851 7320 Acked Write 3 Done Done Sent false
22852 7321 Acked Write 1 Done Done Sent false
22853 7322 Acked Write 7 Done Done Sent false
22854 7323 Acked Flush 0 Done Done Sent false
22855 7324 Acked Read 3 Done Done Sent false
22856 7325 Acked Write 2 Done Done Sent false
22857 7326 Acked Write 9 Done Done Sent false
22858 7327 Acked Read 1 Done Done Sent false
22859 7328 Acked Write 5 Done Done Sent false
22860 7329 Acked Write 5 Done Done Sent false
22861 7330 Acked Read 3 Done Done Sent false
22862 7331 Acked Write 9 Done Done Sent false
22863 7332 Acked Write 7 Done Done Sent false
22864 7333 Acked Write 9 Done Done Sent false
22865 7334 Acked Write 8 Done Done Sent false
22866 7335 Acked Read 1 Done Done Sent false
22867 7336 Acked Read 7 Done Done Sent false
22868 7337 Acked Write 6 Done Done Sent false
22869 7338 Acked Write 2 Done Done Sent false
22870 7339 Acked Read 2 Done Done Sent false
22871 STATES DS:0 DS:1 DS:2 TOTAL
22872 Sent 0 0 301 301
22873 Done 301 301 0 602
22874 Skipped 0 0 0 0
22875 Error 0 0 0 0
22876 Last Flush: 7323 7323 7038
22877 Downstairs last five completed: 7038 7037 7036 7035 7034
22878 Upstairs last five completed: 7339 7338 7337 7336 7335
22879 CLIENT: Up:0 ds:301 act:2
22880 ----------------------------------------------------------------
22881 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22882 States: Active Active Offline
22883 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22884 7039 Acked Flush 0 Done Done Sent false
22885 7040 Acked Read 7 Done Done Sent false
22886 7041 Acked Read 2 Done Done Sent false
22887 7042 Acked Write 9 Done Done Sent false
22888 7043 Acked Write 3 Done Done Sent false
22889 7044 Acked Read 2 Done Done Sent false
22890 7045 Acked Flush 0 Done Done Sent false
22891 7046 Acked Read 10 Done Done Sent false
22892 7047 Acked Read 3 Done Done Sent false
22893 7048 Acked Write 2 Done Done Sent false
22894 7049 Acked Write 9 Done Done Sent false
22895 7050 Acked Write 5 Done Done Sent false
22896 7051 Acked Write 9 Done Done Sent false
22897 7052 Acked Flush 0 Done Done Sent false
22898 7053 Acked Read 6 Done Done Sent false
22899 7054 Acked Read 7 Done Done Sent false
22900 7055 Acked Flush 0 Done Done Sent false
22901 7056 Acked Read 9 Done Done Sent false
22902 7057 Acked Write 3 Done Done Sent false
22903 7058 Acked Read 1 Done Done Sent false
22904 7059 Acked Write 8 Done Done Sent false
22905 7060 Acked Write 6 Done Done Sent false
22906 7061 Acked Write 3 Done Done Sent false
22907 7062 Acked Write 10 Done Done Sent false
22908 7063 Acked Flush 0 Done Done Sent false
22909 7064 Acked Read 8 Done Done Sent false
22910 7065 Acked Read 2 Done Done Sent false
22911 7066 Acked Write 1 Done Done Sent false
22912 7067 Acked Flush 0 Done Done Sent false
22913 7068 Acked Read 4 Done Done Sent false
22914 7069 Acked Write 3 Done Done Sent false
22915 7070 Acked Read 7 Done Done Sent false
22916 7071 Acked Read 4 Done Done Sent false
22917 7072 Acked Write 10 Done Done Sent false
22918 7073 Acked Read 9 Done Done Sent false
22919 7074 Acked Flush 0 Done Done Sent false
22920 7075 Acked Read 8 Done Done Sent false
22921 7076 Acked Write 3 Done Done Sent false
22922 7077 Acked Read 3 Done Done Sent false
22923 7078 Acked Read 2 Done Done Sent false
22924 7079 Acked Write 1 Done Done Sent false
22925 7080 Acked Read 3 Done Done Sent false
22926 7081 Acked Read 10 Done Done Sent false
22927 7082 Acked Read 10 Done Done Sent false
22928 7083 Acked Write 6 Done Done Sent false
22929 7084 Acked Read 1 Done Done Sent false
22930 7085 Acked Write 1 Done Done Sent false
22931 7086 Acked Flush 0 Done Done Sent false
22932 7087 Acked Write 5 Done Done Sent false
22933 7088 Acked Read 4 Done Done Sent false
22934 7089 Acked Read 7 Done Done Sent false
22935 7090 Acked Write 1 Done Done Sent false
22936 7091 Acked Read 9 Done Done Sent false
22937 7092 Acked Write 2 Done Done Sent false
22938 7093 Acked Read 2 Done Done Sent false
22939 7094 Acked Read 4 Done Done Sent false
22940 7095 Acked Write 7 Done Done Sent false
22941 7096 Acked Read 1 Done Done Sent false
22942 7097 Acked Write 10 Done Done Sent false
22943 7098 Acked Read 7 Done Done Sent false
22944 7099 Acked Read 8 Done Done Sent false
22945 7100 Acked Write 9 Done Done Sent false
22946 7101 Acked Write 8 Done Done Sent false
22947 7102 Acked Read 6 Done Done Sent false
22948 7103 Acked Read 7 Done Done Sent false
22949 7104 Acked Read 9 Done Done Sent false
22950 7105 Acked Flush 0 Done Done Sent false
22951 7106 Acked Read 8 Done Done Sent false
22952 7107 Acked Read 9 Done Done Sent false
22953 7108 Acked Read 5 Done Done Sent false
22954 7109 Acked Read 3 Done Done Sent false
22955 7110 Acked Write 10 Done Done Sent false
22956 7111 Acked Read 6 Done Done Sent false
22957 7112 Acked Write 3 Done Done Sent false
22958 7113 Acked Write 1 Done Done Sent false
22959 7114 Acked Read 2 Done Done Sent false
22960 7115 Acked Write 10 Done Done Sent false
22961 7116 Acked Read 9 Done Done Sent false
22962 7117 Acked Write 1 Done Done Sent false
22963 7118 Acked Read 7 Done Done Sent false
22964 7119 Acked Read 3 Done Done Sent false
22965 7120 Acked Read 3 Done Done Sent false
22966 7121 Acked Flush 0 Done Done Sent false
22967 7122 Acked Read 3 Done Done Sent false
22968 7123 Acked Flush 0 Done Done Sent false
22969 7124 Acked Flush 0 Done Done Sent false
22970 7125 Acked Read 9 Done Done Sent false
22971 7126 Acked Read 7 Done Done Sent false
22972 7127 Acked Read 1 Done Done Sent false
22973 7128 Acked Flush 0 Done Done Sent false
22974 7129 Acked Read 8 Done Done Sent false
22975 7130 Acked Write 4 Done Done Sent false
22976 7131 Acked Write 6 Done Done Sent false
22977 7132 Acked Read 9 Done Done Sent false
22978 7133 Acked Write 9 Done Done Sent false
22979 7134 Acked Flush 0 Done Done Sent false
22980 7135 Acked Read 4 Done Done Sent false
22981 7136 Acked Write 8 Done Done Sent false
22982 7137 Acked Write 6 Done Done Sent false
22983 7138 Acked Read 7 Done Done Sent false
22984 7139 Acked Write 6 Done Done Sent false
22985 7140 Acked Read 8 Done Done Sent false
22986 7141 Acked Read 5 Done Done Sent false
22987 7142 Acked Write 10 Done Done Sent false
22988 7143 Acked Flush 0 Done Done Sent false
22989 7144 Acked Write 10 Done Done Sent false
22990 7145 Acked Read 7 Done Done Sent false
22991 7146 Acked Read 5 Done Done Sent false
22992 7147 Acked Read 3 Done Done Sent false
22993 7148 Acked Write 7 Done Done Sent false
22994 7149 Acked Flush 0 Done Done Sent false
22995 7150 Acked Write 5 Done Done Sent false
22996 7151 Acked Read 10 Done Done Sent false
22997 7152 Acked Read 10 Done Done Sent false
22998 7153 Acked Read 8 Done Done Sent false
22999 7154 Acked Write 1 Done Done Sent false
23000 7155 Acked Write 10 Done Done Sent false
23001 7156 Acked Write 8 Done Done Sent false
23002 7157 Acked Read 8 Done Done Sent false
23003 7158 Acked Write 3 Done Done Sent false
23004 7159 Acked Read 9 Done Done Sent false
23005 7160 Acked Read 4 Done Done Sent false
23006 7161 Acked Flush 0 Done Done Sent false
23007 7162 Acked Write 1 Done Done Sent false
23008 7163 Acked Write 4 Done Done Sent false
23009 7164 Acked Read 5 Done Done Sent false
23010 7165 Acked Read 8 Done Done Sent false
23011 7166 Acked Flush 0 Done Done Sent false
23012 7167 Acked Write 5 Done Done Sent false
23013 7168 Acked Read 4 Done Done Sent false
23014 7169 Acked Read 10 Done Done Sent false
23015 7170 Acked Read 8 Done Done Sent false
23016 7171 Acked Read 1 Done Done Sent false
23017 7172 Acked Read 5 Done Done Sent false
23018 7173 Acked Write 6 Done Done Sent false
23019 7174 Acked Write 5 Done Done Sent false
23020 7175 Acked Read 5 Done Done Sent false
23021 7176 Acked Write 3 Done Done Sent false
23022 7177 Acked Read 7 Done Done Sent false
23023 7178 Acked Write 9 Done Done Sent false
23024 7179 Acked Read 3 Done Done Sent false
23025 7180 Acked Read 6 Done Done Sent false
23026 7181 Acked Read 5 Done Done Sent false
23027 7182 Acked Read 10 Done Done Sent false
23028 7183 Acked Write 5 Done Done Sent false
23029 7184 Acked Read 10 Done Done Sent false
23030 7185 Acked Read 4 Done Done Sent false
23031 7186 Acked Read 2 Done Done Sent false
23032 7187 Acked Write 6 Done Done Sent false
23033 7188 Acked Write 8 Done Done Sent false
23034 7189 Acked Write 7 Done Done Sent false
23035 7190 Acked Read 4 Done Done Sent false
23036 7191 Acked Write 6 Done Done Sent false
23037 7192 Acked Read 2 Done Done Sent false
23038 7193 Acked Read 1 Done Done Sent false
23039 7194 Acked Read 9 Done Done Sent false
23040 7195 Acked Write 1 Done Done Sent false
23041 7196 Acked Write 10 Done Done Sent false
23042 7197 Acked Write 4 Done Done Sent false
23043 7198 Acked Write 4 Done Done Sent false
23044 7199 Acked Write 7 Done Done Sent false
23045 7200 Acked Read 2 Done Done Sent false
23046 7201 Acked Write 8 Done Done Sent false
23047 7202 Acked Write 5 Done Done Sent false
23048 7203 Acked Read 2 Done Done Sent false
23049 7204 Acked Write 7 Done Done Sent false
23050 7205 Acked Write 1 Done Done Sent false
23051 7206 Acked Write 6 Done Done Sent false
23052 7207 Acked Flush 0 Done Done Sent false
23053 7208 Acked Write 3 Done Done Sent false
23054 7209 Acked Read 5 Done Done Sent false
23055 7210 Acked Write 3 Done Done Sent false
23056 7211 Acked Read 10 Done Done Sent false
23057 7212 Acked Write 3 Done Done Sent false
23058 7213 Acked Read 2 Done Done Sent false
23059 7214 Acked Read 4 Done Done Sent false
23060 7215 Acked Write 6 Done Done Sent false
23061 7216 Acked Write 5 Done Done Sent false
23062 7217 Acked Write 8 Done Done Sent false
23063 7218 Acked Read 3 Done Done Sent false
23064 7219 Acked Write 10 Done Done Sent false
23065 7220 Acked Read 7 Done Done Sent false
23066 7221 Acked Flush 0 Done Done Sent false
23067 7222 Acked Read 3 Done Done Sent false
23068 7223 Acked Read 7 Done Done Sent false
23069 7224 Acked Write 2 Done Done Sent false
23070 7225 Acked Write 8 Done Done Sent false
23071 7226 Acked Write 10 Done Done Sent false
23072 7227 Acked Read 6 Done Done Sent false
23073 7228 Acked Write 7 Done Done Sent false
23074 7229 Acked Read 4 Done Done Sent false
23075 7230 Acked Read 5 Done Done Sent false
23076 7231 Acked Read 7 Done Done Sent false
23077 7232 Acked Read 2 Done Done Sent false
23078 7233 Acked Read 2 Done Done Sent false
23079 7234 Acked Write 9 Done Done Sent false
23080 7235 Acked Read 4 Done Done Sent false
23081 7236 Acked Read 5 Done Done Sent false
23082 7237 Acked Write 1 Done Done Sent false
23083 7238 Acked Write 8 Done Done Sent false
23084 7239 Acked Flush 0 Done Done Sent false
23085 7240 Acked Read 8 Done Done Sent false
23086 7241 Acked Read 9 Done Done Sent false
23087 7242 Acked Read 6 Done Done Sent false
23088 7243 Acked Read 9 Done Done Sent false
23089 7244 Acked Read 2 Done Done Sent false
23090 7245 Acked Write 9 Done Done Sent false
23091 7246 Acked Read 2 Done Done Sent false
23092 7247 Acked Write 5 Done Done Sent false
23093 7248 Acked Read 10 Done Done Sent false
23094 7249 Acked Read 5 Done Done Sent false
23095 7250 Acked Write 8 Done Done Sent false
23096 7251 Acked Read 3 Done Done Sent false
23097 7252 Acked Write 8 Done Done Sent false
23098 7253 Acked Write 4 Done Done Sent false
23099 7254 Acked Write 10 Done Done Sent false
23100 7255 Acked Write 2 Done Done Sent false
23101 7256 Acked Write 4 Done Done Sent false
23102 7257 Acked Write 1 Done Done Sent false
23103 7258 Acked Write 4 Done Done Sent false
23104 7259 Acked Write 6 Done Done Sent false
23105 7260 Acked Read 2 Done Done Sent false
23106 7261 Acked Write 9 Done Done Sent false
23107 7262 Acked Read 2 Done Done Sent false
23108 7263 Acked Read 10 Done Done Sent false
23109 7264 Acked Read 10 Done Done Sent false
23110 7265 Acked Write 6 Done Done Sent false
23111 7266 Acked Write 7 Done Done Sent false
23112 7267 Acked Read 2 Done Done Sent false
23113 7268 Acked Read 8 Done Done Sent false
23114 7269 Acked Flush 0 Done Done Sent false
23115 7270 Acked Write 8 Done Done Sent false
23116 7271 Acked Read 7 Done Done Sent false
23117 7272 Acked Write 9 Done Done Sent false
23118 7273 Acked Read 8 Done Done Sent false
23119 7274 Acked Write 4 Done Done Sent false
23120 7275 Acked Write 8 Done Done Sent false
23121 7276 Acked Flush 0 Done Done Sent false
23122 7277 Acked Write 6 Done Done Sent false
23123 7278 Acked Flush 0 Done Done Sent false
23124 7279 Acked Flush 0 Done Done Sent false
23125 7280 Acked Write 9 Done Done Sent false
23126 7281 Acked Write 8 Done Done Sent false
23127 7282 Acked Write 7 Done Done Sent false
23128 7283 Acked Write 3 Done Done Sent false
23129 7284 Acked Write 3 Done Done Sent false
23130 7285 Acked Write 10 Done Done Sent false
23131 7286 Acked Read 2 Done Done Sent false
23132 7287 Acked Read 2 Done Done Sent false
23133 7288 Acked Write 1 Done Done Sent false
23134 7289 Acked Read 5 Done Done Sent false
23135 7290 Acked Write 1 Done Done Sent false
23136 7291 Acked Read 5 Done Done Sent false
23137 7292 Acked Write 10 Done Done Sent false
23138 7293 Acked Read 1 Done Done Sent false
23139 7294 Acked Read 2 Done Done Sent false
23140 7295 Acked Write 7 Done Done Sent false
23141 7296 Acked Read 3 Done Done Sent false
23142 7297 Acked Read 9 Done Done Sent false
23143 7298 Acked Read 5 Done Done Sent false
23144 7299 Acked Write 7 Done Done Sent false
23145 7300 Acked Read 2 Done Done Sent false
23146 7301 Acked Flush 0 Done Done Sent false
23147 7302 Acked Read 6 Done Done Sent false
23148 7303 Acked Flush 0 Done Done Sent false
23149 7304 Acked Read 10 Done Done Sent false
23150 7305 Acked Write 8 Done Done Sent false
23151 7306 Acked Write 2 Done Done Sent false
23152 7307 Acked Read 8 Done Done Sent false
23153 7308 Acked Flush 0 Done Done Sent false
23154 7309 Acked Write 3 Done Done Sent false
23155 7310 Acked Read 5 Done Done Sent false
23156 7311 Acked Read 3 Done Done Sent false
23157 7312 Acked Read 5 Done Done Sent false
23158 7313 Acked Flush 0 Done Done Sent false
23159 7314 Acked Write 3 Done Done Sent false
23160 7315 Acked Write 8 Done Done Sent false
23161 7316 Acked Write 8 Done Done Sent false
23162 7317 Acked Read 10 Done Done Sent false
23163 7318 Acked Write 2 Done Done Sent false
23164 7319 Acked Read 6 Done Done Sent false
23165 7320 Acked Write 3 Done Done Sent false
23166 7321 Acked Write 1 Done Done Sent false
23167 7322 Acked Write 7 Done Done Sent false
23168 7323 Acked Flush 0 Done Done Sent false
23169 7324 Acked Read 3 Done Done Sent false
23170 7325 Acked Write 2 Done Done Sent false
23171 7326 Acked Write 9 Done Done Sent false
23172 7327 Acked Read 1 Done Done Sent false
23173 7328 Acked Write 5 Done Done Sent false
23174 7329 Acked Write 5 Done Done Sent false
23175 7330 Acked Read 3 Done Done Sent false
23176 7331 Acked Write 9 Done Done Sent false
23177 7332 Acked Write 7 Done Done Sent false
23178 7333 Acked Write 9 Done Done Sent false
23179 7334 Acked Write 8 Done Done Sent false
23180 7335 Acked Read 1 Done Done Sent false
23181 7336 Acked Read 7 Done Done Sent false
23182 7337 Acked Write 6 Done Done Sent false
23183 7338 Acked Write 2 Done Done Sent false
23184 7339 Acked Read 2 Done Done Sent false
23185 7340 Acked Flush 0 Done Done Sent false
23186 STATES DS:0 DS:1 DS:2 TOTAL
23187 Sent 0 0 302 302
23188 Done 302 302 0 604
23189 Skipped 0 0 0 0
23190 Error 0 0 0 0
23191 Last Flush: 7340 7340 7038
23192 Downstairs last five completed: 7038 7037 7036 7035 7034
23193 Upstairs last five completed: 7340 7339 7338 7337 7336
23194 CLIENT: Up:0 ds:302 act:2
231952025-01-03T08:35:44.602ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
231962025-01-03T08:35:44.602ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
231972025-01-03T08:35:44.602ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
231982025-01-03T08:35:44.602ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
231992025-01-03T08:35:44.602ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
23200 ----------------------------------------------------------------
23201 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23202 States: Active Active Offline
23203 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23204 7039 Acked Flush 0 Done Done Sent false
23205 7040 Acked Read 7 Done Done Sent false
23206 7041 Acked Read 2 Done Done Sent false
23207 7042 Acked Write 9 Done Done Sent false
23208 7043 Acked Write 3 Done Done Sent false
23209 7044 Acked Read 2 Done Done Sent false
23210 7045 Acked Flush 0 Done Done Sent false
23211 7046 Acked Read 10 Done Done Sent false
23212 7047 Acked Read 3 Done Done Sent false
23213 7048 Acked Write 2 Done Done Sent false
23214 7049 Acked Write 9 Done Done Sent false
23215 7050 Acked Write 5 Done Done Sent false
23216 7051 Acked Write 9 Done Done Sent false
23217 7052 Acked Flush 0 Done Done Sent false
23218 7053 Acked Read 6 Done Done Sent false
23219 7054 Acked Read 7 Done Done Sent false
23220 7055 Acked Flush 0 Done Done Sent false
23221 7056 Acked Read 9 Done Done Sent false
23222 7057 Acked Write 3 Done Done Sent false
23223 7058 Acked Read 1 Done Done Sent false
23224 7059 Acked Write 8 Done Done Sent false
23225 7060 Acked Write 6 Done Done Sent false
23226 7061 Acked Write 3 Done Done Sent false
23227 7062 Acked Write 10 Done Done Sent false
23228 7063 Acked Flush 0 Done Done Sent false
23229 7064 Acked Read 8 Done Done Sent false
23230 7065 Acked Read 2 Done Done Sent false
23231 7066 Acked Write 1 Done Done Sent false
23232 7067 Acked Flush 0 Done Done Sent false
23233 7068 Acked Read 4 Done Done Sent false
23234 7069 Acked Write 3 Done Done Sent false
23235 7070 Acked Read 7 Done Done Sent false
23236 7071 Acked Read 4 Done Done Sent false
23237 7072 Acked Write 10 Done Done Sent false
23238 7073 Acked Read 9 Done Done Sent false
23239 7074 Acked Flush 0 Done Done Sent false
23240 7075 Acked Read 8 Done Done Sent false
23241 7076 Acked Write 3 Done Done Sent false
23242 7077 Acked Read 3 Done Done Sent false
23243 7078 Acked Read 2 Done Done Sent false
23244 7079 Acked Write 1 Done Done Sent false
23245 7080 Acked Read 3 Done Done Sent false
23246 7081 Acked Read 10 Done Done Sent false
23247 7082 Acked Read 10 Done Done Sent false
23248 7083 Acked Write 6 Done Done Sent false
23249 7084 Acked Read 1 Done Done Sent false
23250 7085 Acked Write 1 Done Done Sent false
23251 7086 Acked Flush 0 Done Done Sent false
23252 7087 Acked Write 5 Done Done Sent false
23253 7088 Acked Read 4 Done Done Sent false
23254 7089 Acked Read 7 Done Done Sent false
23255 7090 Acked Write 1 Done Done Sent false
23256 7091 Acked Read 9 Done Done Sent false
23257 7092 Acked Write 2 Done Done Sent false
23258 7093 Acked Read 2 Done Done Sent false
23259 7094 Acked Read 4 Done Done Sent false
23260 7095 Acked Write 7 Done Done Sent false
23261 7096 Acked Read 1 Done Done Sent false
23262 7097 Acked Write 10 Done Done Sent false
23263 7098 Acked Read 7 Done Done Sent false
23264 7099 Acked Read 8 Done Done Sent false
23265 7100 Acked Write 9 Done Done Sent false
23266 7101 Acked Write 8 Done Done Sent false
23267 7102 Acked Read 6 Done Done Sent false
23268 7103 Acked Read 7 Done Done Sent false
23269 7104 Acked Read 9 Done Done Sent false
23270 7105 Acked Flush 0 Done Done Sent false
23271 7106 Acked Read 8 Done Done Sent false
23272 7107 Acked Read 9 Done Done Sent false
23273 7108 Acked Read 5 Done Done Sent false
23274 7109 Acked Read 3 Done Done Sent false
23275 7110 Acked Write 10 Done Done Sent false
23276 7111 Acked Read 6 Done Done Sent false
23277 7112 Acked Write 3 Done Done Sent false
23278 7113 Acked Write 1 Done Done Sent false
23279 7114 Acked Read 2 Done Done Sent false
23280 7115 Acked Write 10 Done Done Sent false
23281 7116 Acked Read 9 Done Done Sent false
23282 7117 Acked Write 1 Done Done Sent false
23283 7118 Acked Read 7 Done Done Sent false
23284 7119 Acked Read 3 Done Done Sent false
23285 7120 Acked Read 3 Done Done Sent false
23286 7121 Acked Flush 0 Done Done Sent false
23287 7122 Acked Read 3 Done Done Sent false
23288 7123 Acked Flush 0 Done Done Sent false
23289 7124 Acked Flush 0 Done Done Sent false
23290 7125 Acked Read 9 Done Done Sent false
23291 7126 Acked Read 7 Done Done Sent false
23292 7127 Acked Read 1 Done Done Sent false
23293 7128 Acked Flush 0 Done Done Sent false
23294 7129 Acked Read 8 Done Done Sent false
23295 7130 Acked Write 4 Done Done Sent false
23296 7131 Acked Write 6 Done Done Sent false
23297 7132 Acked Read 9 Done Done Sent false
23298 7133 Acked Write 9 Done Done Sent false
23299 7134 Acked Flush 0 Done Done Sent false
23300 7135 Acked Read 4 Done Done Sent false
23301 7136 Acked Write 8 Done Done Sent false
23302 7137 Acked Write 6 Done Done Sent false
23303 7138 Acked Read 7 Done Done Sent false
23304 7139 Acked Write 6 Done Done Sent false
23305 7140 Acked Read 8 Done Done Sent false
23306 7141 Acked Read 5 Done Done Sent false
23307 7142 Acked Write 10 Done Done Sent false
23308 7143 Acked Flush 0 Done Done Sent false
23309 7144 Acked Write 10 Done Done Sent false
23310 7145 Acked Read 7 Done Done Sent false
23311 7146 Acked Read 5 Done Done Sent false
23312 7147 Acked Read 3 Done Done Sent false
23313 7148 Acked Write 7 Done Done Sent false
23314 7149 Acked Flush 0 Done Done Sent false
23315 7150 Acked Write 5 Done Done Sent false
23316 7151 Acked Read 10 Done Done Sent false
23317 7152 Acked Read 10 Done Done Sent false
23318 7153 Acked Read 8 Done Done Sent false
23319 7154 Acked Write 1 Done Done Sent false
23320 7155 Acked Write 10 Done Done Sent false
23321 7156 Acked Write 8 Done Done Sent false
23322 7157 Acked Read 8 Done Done Sent false
23323 7158 Acked Write 3 Done Done Sent false
23324 7159 Acked Read 9 Done Done Sent false
23325 7160 Acked Read 4 Done Done Sent false
23326 7161 Acked Flush 0 Done Done Sent false
23327 7162 Acked Write 1 Done Done Sent false
23328 7163 Acked Write 4 Done Done Sent false
23329 7164 Acked Read 5 Done Done Sent false
23330 7165 Acked Read 8 Done Done Sent false
23331 7166 Acked Flush 0 Done Done Sent false
23332 7167 Acked Write 5 Done Done Sent false
23333 7168 Acked Read 4 Done Done Sent false
23334 7169 Acked Read 10 Done Done Sent false
23335 7170 Acked Read 8 Done Done Sent false
23336 7171 Acked Read 1 Done Done Sent false
23337 7172 Acked Read 5 Done Done Sent false
23338 7173 Acked Write 6 Done Done Sent false
23339 7174 Acked Write 5 Done Done Sent false
23340 7175 Acked Read 5 Done Done Sent false
23341 7176 Acked Write 3 Done Done Sent false
23342 7177 Acked Read 7 Done Done Sent false
23343 7178 Acked Write 9 Done Done Sent false
23344 7179 Acked Read 3 Done Done Sent false
23345 7180 Acked Read 6 Done Done Sent false
23346 7181 Acked Read 5 Done Done Sent false
23347 7182 Acked Read 10 Done Done Sent false
23348 7183 Acked Write 5 Done Done Sent false
23349 7184 Acked Read 10 Done Done Sent false
23350 7185 Acked Read 4 Done Done Sent false
23351 7186 Acked Read 2 Done Done Sent false
23352 7187 Acked Write 6 Done Done Sent false
23353 7188 Acked Write 8 Done Done Sent false
23354 7189 Acked Write 7 Done Done Sent false
23355 7190 Acked Read 4 Done Done Sent false
23356 7191 Acked Write 6 Done Done Sent false
23357 7192 Acked Read 2 Done Done Sent false
23358 7193 Acked Read 1 Done Done Sent false
23359 7194 Acked Read 9 Done Done Sent false
23360 7195 Acked Write 1 Done Done Sent false
23361 7196 Acked Write 10 Done Done Sent false
23362 7197 Acked Write 4 Done Done Sent false
23363 7198 Acked Write 4 Done Done Sent false
23364 7199 Acked Write 7 Done Done Sent false
23365 7200 Acked Read 2 Done Done Sent false
23366 7201 Acked Write 8 Done Done Sent false
23367 7202 Acked Write 5 Done Done Sent false
23368 7203 Acked Read 2 Done Done Sent false
23369 7204 Acked Write 7 Done Done Sent false
23370 7205 Acked Write 1 Done Done Sent false
23371 7206 Acked Write 6 Done Done Sent false
23372 7207 Acked Flush 0 Done Done Sent false
23373 7208 Acked Write 3 Done Done Sent false
23374 7209 Acked Read 5 Done Done Sent false
23375 7210 Acked Write 3 Done Done Sent false
23376 7211 Acked Read 10 Done Done Sent false
23377 7212 Acked Write 3 Done Done Sent false
23378 7213 Acked Read 2 Done Done Sent false
23379 7214 Acked Read 4 Done Done Sent false
23380 7215 Acked Write 6 Done Done Sent false
23381 7216 Acked Write 5 Done Done Sent false
23382 7217 Acked Write 8 Done Done Sent false
23383 7218 Acked Read 3 Done Done Sent false
23384 7219 Acked Write 10 Done Done Sent false
23385 7220 Acked Read 7 Done Done Sent false
23386 7221 Acked Flush 0 Done Done Sent false
23387 7222 Acked Read 3 Done Done Sent false
23388 7223 Acked Read 7 Done Done Sent false
23389 7224 Acked Write 2 Done Done Sent false
23390 7225 Acked Write 8 Done Done Sent false
23391 7226 Acked Write 10 Done Done Sent false
23392 7227 Acked Read 6 Done Done Sent false
23393 7228 Acked Write 7 Done Done Sent false
23394 7229 Acked Read 4 Done Done Sent false
23395 7230 Acked Read 5 Done Done Sent false
23396 7231 Acked Read 7 Done Done Sent false
23397 7232 Acked Read 2 Done Done Sent false
23398 7233 Acked Read 2 Done Done Sent false
23399 7234 Acked Write 9 Done Done Sent false
23400 7235 Acked Read 4 Done Done Sent false
23401 7236 Acked Read 5 Done Done Sent false
23402 7237 Acked Write 1 Done Done Sent false
23403 7238 Acked Write 8 Done Done Sent false
23404 7239 Acked Flush 0 Done Done Sent false
23405 7240 Acked Read 8 Done Done Sent false
23406 7241 Acked Read 9 Done Done Sent false
23407 7242 Acked Read 6 Done Done Sent false
23408 7243 Acked Read 9 Done Done Sent false
23409 7244 Acked Read 2 Done Done Sent false
23410 7245 Acked Write 9 Done Done Sent false
23411 7246 Acked Read 2 Done Done Sent false
23412 7247 Acked Write 5 Done Done Sent false
23413 7248 Acked Read 10 Done Done Sent false
23414 7249 Acked Read 5 Done Done Sent false
23415 7250 Acked Write 8 Done Done Sent false
23416 7251 Acked Read 3 Done Done Sent false
23417 7252 Acked Write 8 Done Done Sent false
23418 7253 Acked Write 4 Done Done Sent false
23419 7254 Acked Write 10 Done Done Sent false
23420 7255 Acked Write 2 Done Done Sent false
23421 7256 Acked Write 4 Done Done Sent false
23422 7257 Acked Write 1 Done Done Sent false
23423 7258 Acked Write 4 Done Done Sent false
23424 7259 Acked Write 6 Done Done Sent false
23425 7260 Acked Read 2 Done Done Sent false
23426 7261 Acked Write 9 Done Done Sent false
23427 7262 Acked Read 2 Done Done Sent false
23428 7263 Acked Read 10 Done Done Sent false
23429 7264 Acked Read 10 Done Done Sent false
23430 7265 Acked Write 6 Done Done Sent false
23431 7266 Acked Write 7 Done Done Sent false
23432 7267 Acked Read 2 Done Done Sent false
23433 7268 Acked Read 8 Done Done Sent false
23434 7269 Acked Flush 0 Done Done Sent false
23435 7270 Acked Write 8 Done Done Sent false
23436 7271 Acked Read 7 Done Done Sent false
23437 7272 Acked Write 9 Done Done Sent false
23438 7273 Acked Read 8 Done Done Sent false
23439 7274 Acked Write 4 Done Done Sent false
23440 7275 Acked Write 8 Done Done Sent false
23441 7276 Acked Flush 0 Done Done Sent false
23442 7277 Acked Write 6 Done Done Sent false
23443 7278 Acked Flush 0 Done Done Sent false
23444 7279 Acked Flush 0 Done Done Sent false
23445 7280 Acked Write 9 Done Done Sent false
23446 7281 Acked Write 8 Done Done Sent false
23447 7282 Acked Write 7 Done Done Sent false
23448 7283 Acked Write 3 Done Done Sent false
23449 7284 Acked Write 3 Done Done Sent false
23450 7285 Acked Write 10 Done Done Sent false
23451 7286 Acked Read 2 Done Done Sent false
23452 7287 Acked Read 2 Done Done Sent false
23453 7288 Acked Write 1 Done Done Sent false
23454 7289 Acked Read 5 Done Done Sent false
23455 7290 Acked Write 1 Done Done Sent false
23456 7291 Acked Read 5 Done Done Sent false
23457 7292 Acked Write 10 Done Done Sent false
23458 7293 Acked Read 1 Done Done Sent false
23459 7294 Acked Read 2 Done Done Sent false
23460 7295 Acked Write 7 Done Done Sent false
23461 7296 Acked Read 3 Done Done Sent false
23462 7297 Acked Read 9 Done Done Sent false
23463 7298 Acked Read 5 Done Done Sent false
23464 7299 Acked Write 7 Done Done Sent false
23465 7300 Acked Read 2 Done Done Sent false
23466 7301 Acked Flush 0 Done Done Sent false
23467 7302 Acked Read 6 Done Done Sent false
23468 7303 Acked Flush 0 Done Done Sent false
23469 7304 Acked Read 10 Done Done Sent false
23470 7305 Acked Write 8 Done Done Sent false
23471 7306 Acked Write 2 Done Done Sent false
23472 7307 Acked Read 8 Done Done Sent false
23473 7308 Acked Flush 0 Done Done Sent false
23474 7309 Acked Write 3 Done Done Sent false
23475 7310 Acked Read 5 Done Done Sent false
23476 7311 Acked Read 3 Done Done Sent false
23477 7312 Acked Read 5 Done Done Sent false
23478 7313 Acked Flush 0 Done Done Sent false
23479 7314 Acked Write 3 Done Done Sent false
23480 7315 Acked Write 8 Done Done Sent false
23481 7316 Acked Write 8 Done Done Sent false
23482 7317 Acked Read 10 Done Done Sent false
23483 7318 Acked Write 2 Done Done Sent false
23484 7319 Acked Read 6 Done Done Sent false
23485 7320 Acked Write 3 Done Done Sent false
23486 7321 Acked Write 1 Done Done Sent false
23487 7322 Acked Write 7 Done Done Sent false
23488 7323 Acked Flush 0 Done Done Sent false
23489 7324 Acked Read 3 Done Done Sent false
23490 7325 Acked Write 2 Done Done Sent false
23491 7326 Acked Write 9 Done Done Sent false
23492 7327 Acked Read 1 Done Done Sent false
23493 7328 Acked Write 5 Done Done Sent false
23494 7329 Acked Write 5 Done Done Sent false
23495 7330 Acked Read 3 Done Done Sent false
23496 7331 Acked Write 9 Done Done Sent false
23497 7332 Acked Write 7 Done Done Sent false
23498 7333 Acked Write 9 Done Done Sent false
23499 7334 Acked Write 8 Done Done Sent false
23500 7335 Acked Read 1 Done Done Sent false
23501 7336 Acked Read 7 Done Done Sent false
23502 7337 Acked Write 6 Done Done Sent false
23503 7338 Acked Write 2 Done Done Sent false
23504 7339 Acked Read 2 Done Done Sent false
23505 7340 Acked Flush 0 Done Done Sent false
23506 STATES DS:0 DS:1 DS:2 TOTAL
23507 Sent 0 0 302 302
23508 Done 302 302 0 604
23509 Skipped 0 0 0 0
23510 Error 0 0 0 0
23511 Last Flush: 7340 7340 7038
23512 Downstairs last five completed: 7038 7037 7036 7035 7034
23513 Upstairs last five completed: 7340 7339 7338 7337 7336
23514 CLIENT: Up:0 ds:302 act:2
23515 ----------------------------------------------------------------
23516 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23517 States: Active Active Offline
23518 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23519 7039 Acked Flush 0 Done Done Sent false
23520 7040 Acked Read 7 Done Done Sent false
23521 7041 Acked Read 2 Done Done Sent false
23522 7042 Acked Write 9 Done Done Sent false
23523 7043 Acked Write 3 Done Done Sent false
23524 7044 Acked Read 2 Done Done Sent false
23525 7045 Acked Flush 0 Done Done Sent false
23526 7046 Acked Read 10 Done Done Sent false
23527 7047 Acked Read 3 Done Done Sent false
23528 7048 Acked Write 2 Done Done Sent false
23529 7049 Acked Write 9 Done Done Sent false
23530 7050 Acked Write 5 Done Done Sent false
23531 7051 Acked Write 9 Done Done Sent false
23532 7052 Acked Flush 0 Done Done Sent false
23533 7053 Acked Read 6 Done Done Sent false
23534 7054 Acked Read 7 Done Done Sent false
23535 7055 Acked Flush 0 Done Done Sent false
23536 7056 Acked Read 9 Done Done Sent false
23537 7057 Acked Write 3 Done Done Sent false
23538 7058 Acked Read 1 Done Done Sent false
23539 7059 Acked Write 8 Done Done Sent false
23540 7060 Acked Write 6 Done Done Sent false
23541 7061 Acked Write 3 Done Done Sent false
23542 7062 Acked Write 10 Done Done Sent false
23543 7063 Acked Flush 0 Done Done Sent false
23544 7064 Acked Read 8 Done Done Sent false
23545 7065 Acked Read 2 Done Done Sent false
23546 7066 Acked Write 1 Done Done Sent false
23547 7067 Acked Flush 0 Done Done Sent false
23548 7068 Acked Read 4 Done Done Sent false
23549 7069 Acked Write 3 Done Done Sent false
23550 7070 Acked Read 7 Done Done Sent false
23551 7071 Acked Read 4 Done Done Sent false
23552 7072 Acked Write 10 Done Done Sent false
23553 7073 Acked Read 9 Done Done Sent false
23554 7074 Acked Flush 0 Done Done Sent false
23555 7075 Acked Read 8 Done Done Sent false
23556 7076 Acked Write 3 Done Done Sent false
23557 7077 Acked Read 3 Done Done Sent false
23558 7078 Acked Read 2 Done Done Sent false
23559 7079 Acked Write 1 Done Done Sent false
23560 7080 Acked Read 3 Done Done Sent false
23561 7081 Acked Read 10 Done Done Sent false
23562 7082 Acked Read 10 Done Done Sent false
23563 7083 Acked Write 6 Done Done Sent false
23564 7084 Acked Read 1 Done Done Sent false
23565 7085 Acked Write 1 Done Done Sent false
23566 7086 Acked Flush 0 Done Done Sent false
23567 7087 Acked Write 5 Done Done Sent false
23568 7088 Acked Read 4 Done Done Sent false
23569 7089 Acked Read 7 Done Done Sent false
23570 7090 Acked Write 1 Done Done Sent false
23571 7091 Acked Read 9 Done Done Sent false
23572 7092 Acked Write 2 Done Done Sent false
23573 7093 Acked Read 2 Done Done Sent false
23574 7094 Acked Read 4 Done Done Sent false
23575 7095 Acked Write 7 Done Done Sent false
23576 7096 Acked Read 1 Done Done Sent false
23577 7097 Acked Write 10 Done Done Sent false
23578 7098 Acked Read 7 Done Done Sent false
23579 7099 Acked Read 8 Done Done Sent false
23580 7100 Acked Write 9 Done Done Sent false
23581 7101 Acked Write 8 Done Done Sent false
23582 7102 Acked Read 6 Done Done Sent false
23583 7103 Acked Read 7 Done Done Sent false
23584 7104 Acked Read 9 Done Done Sent false
23585 7105 Acked Flush 0 Done Done Sent false
23586 7106 Acked Read 8 Done Done Sent false
23587 7107 Acked Read 9 Done Done Sent false
23588 7108 Acked Read 5 Done Done Sent false
23589 7109 Acked Read 3 Done Done Sent false
23590 7110 Acked Write 10 Done Done Sent false
23591 7111 Acked Read 6 Done Done Sent false
23592 7112 Acked Write 3 Done Done Sent false
23593 7113 Acked Write 1 Done Done Sent false
23594 7114 Acked Read 2 Done Done Sent false
23595 7115 Acked Write 10 Done Done Sent false
23596 7116 Acked Read 9 Done Done Sent false
23597 7117 Acked Write 1 Done Done Sent false
23598 7118 Acked Read 7 Done Done Sent false
23599 7119 Acked Read 3 Done Done Sent false
23600 7120 Acked Read 3 Done Done Sent false
23601 7121 Acked Flush 0 Done Done Sent false
23602 7122 Acked Read 3 Done Done Sent false
23603 7123 Acked Flush 0 Done Done Sent false
23604 7124 Acked Flush 0 Done Done Sent false
23605 7125 Acked Read 9 Done Done Sent false
23606 7126 Acked Read 7 Done Done Sent false
23607 7127 Acked Read 1 Done Done Sent false
23608 7128 Acked Flush 0 Done Done Sent false
23609 7129 Acked Read 8 Done Done Sent false
23610 7130 Acked Write 4 Done Done Sent false
23611 7131 Acked Write 6 Done Done Sent false
23612 7132 Acked Read 9 Done Done Sent false
23613 7133 Acked Write 9 Done Done Sent false
23614 7134 Acked Flush 0 Done Done Sent false
23615 7135 Acked Read 4 Done Done Sent false
23616 7136 Acked Write 8 Done Done Sent false
23617 7137 Acked Write 6 Done Done Sent false
23618 7138 Acked Read 7 Done Done Sent false
23619 7139 Acked Write 6 Done Done Sent false
23620 7140 Acked Read 8 Done Done Sent false
23621 7141 Acked Read 5 Done Done Sent false
23622 7142 Acked Write 10 Done Done Sent false
23623 7143 Acked Flush 0 Done Done Sent false
23624 7144 Acked Write 10 Done Done Sent false
23625 7145 Acked Read 7 Done Done Sent false
23626 7146 Acked Read 5 Done Done Sent false
23627 7147 Acked Read 3 Done Done Sent false
23628 7148 Acked Write 7 Done Done Sent false
23629 7149 Acked Flush 0 Done Done Sent false
23630 7150 Acked Write 5 Done Done Sent false
23631 7151 Acked Read 10 Done Done Sent false
23632 7152 Acked Read 10 Done Done Sent false
23633 7153 Acked Read 8 Done Done Sent false
23634 7154 Acked Write 1 Done Done Sent false
23635 7155 Acked Write 10 Done Done Sent false
23636 7156 Acked Write 8 Done Done Sent false
23637 7157 Acked Read 8 Done Done Sent false
23638 7158 Acked Write 3 Done Done Sent false
23639 7159 Acked Read 9 Done Done Sent false
23640 7160 Acked Read 4 Done Done Sent false
23641 7161 Acked Flush 0 Done Done Sent false
23642 7162 Acked Write 1 Done Done Sent false
23643 7163 Acked Write 4 Done Done Sent false
23644 7164 Acked Read 5 Done Done Sent false
23645 7165 Acked Read 8 Done Done Sent false
23646 7166 Acked Flush 0 Done Done Sent false
23647 7167 Acked Write 5 Done Done Sent false
23648 7168 Acked Read 4 Done Done Sent false
23649 7169 Acked Read 10 Done Done Sent false
23650 7170 Acked Read 8 Done Done Sent false
23651 7171 Acked Read 1 Done Done Sent false
23652 7172 Acked Read 5 Done Done Sent false
23653 7173 Acked Write 6 Done Done Sent false
23654 7174 Acked Write 5 Done Done Sent false
23655 7175 Acked Read 5 Done Done Sent false
23656 7176 Acked Write 3 Done Done Sent false
23657 7177 Acked Read 7 Done Done Sent false
23658 7178 Acked Write 9 Done Done Sent false
23659 7179 Acked Read 3 Done Done Sent false
23660 7180 Acked Read 6 Done Done Sent false
23661 7181 Acked Read 5 Done Done Sent false
23662 7182 Acked Read 10 Done Done Sent false
23663 7183 Acked Write 5 Done Done Sent false
23664 7184 Acked Read 10 Done Done Sent false
23665 7185 Acked Read 4 Done Done Sent false
23666 7186 Acked Read 2 Done Done Sent false
23667 7187 Acked Write 6 Done Done Sent false
23668 7188 Acked Write 8 Done Done Sent false
23669 7189 Acked Write 7 Done Done Sent false
23670 7190 Acked Read 4 Done Done Sent false
23671 7191 Acked Write 6 Done Done Sent false
23672 7192 Acked Read 2 Done Done Sent false
23673 7193 Acked Read 1 Done Done Sent false
23674 7194 Acked Read 9 Done Done Sent false
23675 7195 Acked Write 1 Done Done Sent false
23676 7196 Acked Write 10 Done Done Sent false
23677 7197 Acked Write 4 Done Done Sent false
23678 7198 Acked Write 4 Done Done Sent false
23679 7199 Acked Write 7 Done Done Sent false
23680 7200 Acked Read 2 Done Done Sent false
23681 7201 Acked Write 8 Done Done Sent false
23682 7202 Acked Write 5 Done Done Sent false
23683 7203 Acked Read 2 Done Done Sent false
23684 7204 Acked Write 7 Done Done Sent false
23685 7205 Acked Write 1 Done Done Sent false
23686 7206 Acked Write 6 Done Done Sent false
23687 7207 Acked Flush 0 Done Done Sent false
23688 7208 Acked Write 3 Done Done Sent false
23689 7209 Acked Read 5 Done Done Sent false
23690 7210 Acked Write 3 Done Done Sent false
23691 7211 Acked Read 10 Done Done Sent false
23692 7212 Acked Write 3 Done Done Sent false
23693 7213 Acked Read 2 Done Done Sent false
23694 7214 Acked Read 4 Done Done Sent false
23695 7215 Acked Write 6 Done Done Sent false
23696 7216 Acked Write 5 Done Done Sent false
23697 7217 Acked Write 8 Done Done Sent false
23698 7218 Acked Read 3 Done Done Sent false
23699 7219 Acked Write 10 Done Done Sent false
23700 7220 Acked Read 7 Done Done Sent false
23701 7221 Acked Flush 0 Done Done Sent false
23702 7222 Acked Read 3 Done Done Sent false
23703 7223 Acked Read 7 Done Done Sent false
23704 7224 Acked Write 2 Done Done Sent false
23705 7225 Acked Write 8 Done Done Sent false
23706 7226 Acked Write 10 Done Done Sent false
23707 7227 Acked Read 6 Done Done Sent false
23708 7228 Acked Write 7 Done Done Sent false
23709 7229 Acked Read 4 Done Done Sent false
23710 7230 Acked Read 5 Done Done Sent false
23711 7231 Acked Read 7 Done Done Sent false
23712 7232 Acked Read 2 Done Done Sent false
23713 7233 Acked Read 2 Done Done Sent false
23714 7234 Acked Write 9 Done Done Sent false
23715 7235 Acked Read 4 Done Done Sent false
23716 7236 Acked Read 5 Done Done Sent false
23717 7237 Acked Write 1 Done Done Sent false
23718 7238 Acked Write 8 Done Done Sent false
23719 7239 Acked Flush 0 Done Done Sent false
23720 7240 Acked Read 8 Done Done Sent false
23721 7241 Acked Read 9 Done Done Sent false
23722 7242 Acked Read 6 Done Done Sent false
23723 7243 Acked Read 9 Done Done Sent false
23724 7244 Acked Read 2 Done Done Sent false
23725 7245 Acked Write 9 Done Done Sent false
23726 7246 Acked Read 2 Done Done Sent false
23727 7247 Acked Write 5 Done Done Sent false
23728 7248 Acked Read 10 Done Done Sent false
23729 7249 Acked Read 5 Done Done Sent false
23730 7250 Acked Write 8 Done Done Sent false
23731 7251 Acked Read 3 Done Done Sent false
23732 7252 Acked Write 8 Done Done Sent false
23733 7253 Acked Write 4 Done Done Sent false
23734 7254 Acked Write 10 Done Done Sent false
23735 7255 Acked Write 2 Done Done Sent false
23736 7256 Acked Write 4 Done Done Sent false
23737 7257 Acked Write 1 Done Done Sent false
23738 7258 Acked Write 4 Done Done Sent false
23739 7259 Acked Write 6 Done Done Sent false
23740 7260 Acked Read 2 Done Done Sent false
23741 7261 Acked Write 9 Done Done Sent false
23742 7262 Acked Read 2 Done Done Sent false
23743 7263 Acked Read 10 Done Done Sent false
23744 7264 Acked Read 10 Done Done Sent false
23745 7265 Acked Write 6 Done Done Sent false
23746 7266 Acked Write 7 Done Done Sent false
23747 7267 Acked Read 2 Done Done Sent false
23748 7268 Acked Read 8 Done Done Sent false
23749 7269 Acked Flush 0 Done Done Sent false
23750 7270 Acked Write 8 Done Done Sent false
23751 7271 Acked Read 7 Done Done Sent false
23752 7272 Acked Write 9 Done Done Sent false
23753 7273 Acked Read 8 Done Done Sent false
23754 7274 Acked Write 4 Done Done Sent false
23755 7275 Acked Write 8 Done Done Sent false
23756 7276 Acked Flush 0 Done Done Sent false
23757 7277 Acked Write 6 Done Done Sent false
23758 7278 Acked Flush 0 Done Done Sent false
23759 7279 Acked Flush 0 Done Done Sent false
23760 7280 Acked Write 9 Done Done Sent false
23761 7281 Acked Write 8 Done Done Sent false
23762 7282 Acked Write 7 Done Done Sent false
23763 7283 Acked Write 3 Done Done Sent false
23764 7284 Acked Write 3 Done Done Sent false
23765 7285 Acked Write 10 Done Done Sent false
23766 7286 Acked Read 2 Done Done Sent false
23767 7287 Acked Read 2 Done Done Sent false
23768 7288 Acked Write 1 Done Done Sent false
23769 7289 Acked Read 5 Done Done Sent false
23770 7290 Acked Write 1 Done Done Sent false
23771 7291 Acked Read 5 Done Done Sent false
23772 7292 Acked Write 10 Done Done Sent false
23773 7293 Acked Read 1 Done Done Sent false
23774 7294 Acked Read 2 Done Done Sent false
23775 7295 Acked Write 7 Done Done Sent false
23776 7296 Acked Read 3 Done Done Sent false
23777 7297 Acked Read 9 Done Done Sent false
23778 7298 Acked Read 5 Done Done Sent false
23779 7299 Acked Write 7 Done Done Sent false
23780 7300 Acked Read 2 Done Done Sent false
23781 7301 Acked Flush 0 Done Done Sent false
23782 7302 Acked Read 6 Done Done Sent false
23783 7303 Acked Flush 0 Done Done Sent false
23784 7304 Acked Read 10 Done Done Sent false
23785 7305 Acked Write 8 Done Done Sent false
23786 7306 Acked Write 2 Done Done Sent false
23787 7307 Acked Read 8 Done Done Sent false
23788 7308 Acked Flush 0 Done Done Sent false
23789 7309 Acked Write 3 Done Done Sent false
23790 7310 Acked Read 5 Done Done Sent false
23791 7311 Acked Read 3 Done Done Sent false
23792 7312 Acked Read 5 Done Done Sent false
23793 7313 Acked Flush 0 Done Done Sent false
23794 7314 Acked Write 3 Done Done Sent false
23795 7315 Acked Write 8 Done Done Sent false
23796 7316 Acked Write 8 Done Done Sent false
23797 7317 Acked Read 10 Done Done Sent false
23798 7318 Acked Write 2 Done Done Sent false
23799 7319 Acked Read 6 Done Done Sent false
23800 7320 Acked Write 3 Done Done Sent false
23801 7321 Acked Write 1 Done Done Sent false
23802 7322 Acked Write 7 Done Done Sent false
23803 7323 Acked Flush 0 Done Done Sent false
23804 7324 Acked Read 3 Done Done Sent false
23805 7325 Acked Write 2 Done Done Sent false
23806 7326 Acked Write 9 Done Done Sent false
23807 7327 Acked Read 1 Done Done Sent false
23808 7328 Acked Write 5 Done Done Sent false
23809 7329 Acked Write 5 Done Done Sent false
23810 7330 Acked Read 3 Done Done Sent false
23811 7331 Acked Write 9 Done Done Sent false
23812 7332 Acked Write 7 Done Done Sent false
23813 7333 Acked Write 9 Done Done Sent false
23814 7334 Acked Write 8 Done Done Sent false
23815 7335 Acked Read 1 Done Done Sent false
23816 7336 Acked Read 7 Done Done Sent false
23817 7337 Acked Write 6 Done Done Sent false
23818 7338 Acked Write 2 Done Done Sent false
23819 7339 Acked Read 2 Done Done Sent false
23820 7340 Acked Flush 0 Done Done Sent false
23821 STATES DS:0 DS:1 DS:2 TOTAL
23822 Sent 0 0 302 302
23823 Done 302 302 0 604
23824 Skipped 0 0 0 0
23825 Error 0 0 0 0
23826 Last Flush: 7340 7340 7038
23827 Downstairs last five completed: 7038 7037 7036 7035 7034
23828 Upstairs last five completed: 7340 7339 7338 7337 7336
23829 CLIENT: Up:0 ds:302 act:2
238302025-01-03T08:35:54.603ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
238312025-01-03T08:35:54.603ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
238322025-01-03T08:35:54.603ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
238332025-01-03T08:35:54.603ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
238342025-01-03T08:35:54.603ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
23835 ----------------------------------------------------------------
23836 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23837 States: Active Active Offline
23838 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23839 7039 Acked Flush 0 Done Done Sent false
23840 7040 Acked Read 7 Done Done Sent false
23841 7041 Acked Read 2 Done Done Sent false
23842 7042 Acked Write 9 Done Done Sent false
23843 7043 Acked Write 3 Done Done Sent false
23844 7044 Acked Read 2 Done Done Sent false
23845 7045 Acked Flush 0 Done Done Sent false
23846 7046 Acked Read 10 Done Done Sent false
23847 7047 Acked Read 3 Done Done Sent false
23848 7048 Acked Write 2 Done Done Sent false
23849 7049 Acked Write 9 Done Done Sent false
23850 7050 Acked Write 5 Done Done Sent false
23851 7051 Acked Write 9 Done Done Sent false
23852 7052 Acked Flush 0 Done Done Sent false
23853 7053 Acked Read 6 Done Done Sent false
23854 7054 Acked Read 7 Done Done Sent false
23855 7055 Acked Flush 0 Done Done Sent false
23856 7056 Acked Read 9 Done Done Sent false
23857 7057 Acked Write 3 Done Done Sent false
23858 7058 Acked Read 1 Done Done Sent false
23859 7059 Acked Write 8 Done Done Sent false
23860 7060 Acked Write 6 Done Done Sent false
23861 7061 Acked Write 3 Done Done Sent false
23862 7062 Acked Write 10 Done Done Sent false
23863 7063 Acked Flush 0 Done Done Sent false
23864 7064 Acked Read 8 Done Done Sent false
23865 7065 Acked Read 2 Done Done Sent false
23866 7066 Acked Write 1 Done Done Sent false
23867 7067 Acked Flush 0 Done Done Sent false
23868 7068 Acked Read 4 Done Done Sent false
23869 7069 Acked Write 3 Done Done Sent false
23870 7070 Acked Read 7 Done Done Sent false
23871 7071 Acked Read 4 Done Done Sent false
23872 7072 Acked Write 10 Done Done Sent false
23873 7073 Acked Read 9 Done Done Sent false
23874 7074 Acked Flush 0 Done Done Sent false
23875 7075 Acked Read 8 Done Done Sent false
23876 7076 Acked Write 3 Done Done Sent false
23877 7077 Acked Read 3 Done Done Sent false
23878 7078 Acked Read 2 Done Done Sent false
23879 7079 Acked Write 1 Done Done Sent false
23880 7080 Acked Read 3 Done Done Sent false
23881 7081 Acked Read 10 Done Done Sent false
23882 7082 Acked Read 10 Done Done Sent false
23883 7083 Acked Write 6 Done Done Sent false
23884 7084 Acked Read 1 Done Done Sent false
23885 7085 Acked Write 1 Done Done Sent false
23886 7086 Acked Flush 0 Done Done Sent false
23887 7087 Acked Write 5 Done Done Sent false
23888 7088 Acked Read 4 Done Done Sent false
23889 7089 Acked Read 7 Done Done Sent false
23890 7090 Acked Write 1 Done Done Sent false
23891 7091 Acked Read 9 Done Done Sent false
23892 7092 Acked Write 2 Done Done Sent false
23893 7093 Acked Read 2 Done Done Sent false
23894 7094 Acked Read 4 Done Done Sent false
23895 7095 Acked Write 7 Done Done Sent false
23896 7096 Acked Read 1 Done Done Sent false
23897 7097 Acked Write 10 Done Done Sent false
23898 7098 Acked Read 7 Done Done Sent false
23899 7099 Acked Read 8 Done Done Sent false
23900 7100 Acked Write 9 Done Done Sent false
23901 7101 Acked Write 8 Done Done Sent false
23902 7102 Acked Read 6 Done Done Sent false
23903 7103 Acked Read 7 Done Done Sent false
23904 7104 Acked Read 9 Done Done Sent false
23905 7105 Acked Flush 0 Done Done Sent false
23906 7106 Acked Read 8 Done Done Sent false
23907 7107 Acked Read 9 Done Done Sent false
23908 7108 Acked Read 5 Done Done Sent false
23909 7109 Acked Read 3 Done Done Sent false
23910 7110 Acked Write 10 Done Done Sent false
23911 7111 Acked Read 6 Done Done Sent false
23912 7112 Acked Write 3 Done Done Sent false
23913 7113 Acked Write 1 Done Done Sent false
23914 7114 Acked Read 2 Done Done Sent false
23915 7115 Acked Write 10 Done Done Sent false
23916 7116 Acked Read 9 Done Done Sent false
23917 7117 Acked Write 1 Done Done Sent false
23918 7118 Acked Read 7 Done Done Sent false
23919 7119 Acked Read 3 Done Done Sent false
23920 7120 Acked Read 3 Done Done Sent false
23921 7121 Acked Flush 0 Done Done Sent false
23922 7122 Acked Read 3 Done Done Sent false
23923 7123 Acked Flush 0 Done Done Sent false
23924 7124 Acked Flush 0 Done Done Sent false
23925 7125 Acked Read 9 Done Done Sent false
23926 7126 Acked Read 7 Done Done Sent false
23927 7127 Acked Read 1 Done Done Sent false
23928 7128 Acked Flush 0 Done Done Sent false
23929 7129 Acked Read 8 Done Done Sent false
23930 7130 Acked Write 4 Done Done Sent false
23931 7131 Acked Write 6 Done Done Sent false
23932 7132 Acked Read 9 Done Done Sent false
23933 7133 Acked Write 9 Done Done Sent false
23934 7134 Acked Flush 0 Done Done Sent false
23935 7135 Acked Read 4 Done Done Sent false
23936 7136 Acked Write 8 Done Done Sent false
23937 7137 Acked Write 6 Done Done Sent false
23938 7138 Acked Read 7 Done Done Sent false
23939 7139 Acked Write 6 Done Done Sent false
23940 7140 Acked Read 8 Done Done Sent false
23941 7141 Acked Read 5 Done Done Sent false
23942 7142 Acked Write 10 Done Done Sent false
23943 7143 Acked Flush 0 Done Done Sent false
23944 7144 Acked Write 10 Done Done Sent false
23945 7145 Acked Read 7 Done Done Sent false
23946 7146 Acked Read 5 Done Done Sent false
23947 7147 Acked Read 3 Done Done Sent false
23948 7148 Acked Write 7 Done Done Sent false
23949 7149 Acked Flush 0 Done Done Sent false
23950 7150 Acked Write 5 Done Done Sent false
23951 7151 Acked Read 10 Done Done Sent false
23952 7152 Acked Read 10 Done Done Sent false
23953 7153 Acked Read 8 Done Done Sent false
23954 7154 Acked Write 1 Done Done Sent false
23955 7155 Acked Write 10 Done Done Sent false
23956 7156 Acked Write 8 Done Done Sent false
23957 7157 Acked Read 8 Done Done Sent false
23958 7158 Acked Write 3 Done Done Sent false
23959 7159 Acked Read 9 Done Done Sent false
23960 7160 Acked Read 4 Done Done Sent false
23961 7161 Acked Flush 0 Done Done Sent false
23962 7162 Acked Write 1 Done Done Sent false
23963 7163 Acked Write 4 Done Done Sent false
23964 7164 Acked Read 5 Done Done Sent false
23965 7165 Acked Read 8 Done Done Sent false
23966 7166 Acked Flush 0 Done Done Sent false
23967 7167 Acked Write 5 Done Done Sent false
23968 7168 Acked Read 4 Done Done Sent false
23969 7169 Acked Read 10 Done Done Sent false
23970 7170 Acked Read 8 Done Done Sent false
23971 7171 Acked Read 1 Done Done Sent false
23972 7172 Acked Read 5 Done Done Sent false
23973 7173 Acked Write 6 Done Done Sent false
23974 7174 Acked Write 5 Done Done Sent false
23975 7175 Acked Read 5 Done Done Sent false
23976 7176 Acked Write 3 Done Done Sent false
23977 7177 Acked Read 7 Done Done Sent false
23978 7178 Acked Write 9 Done Done Sent false
23979 7179 Acked Read 3 Done Done Sent false
23980 7180 Acked Read 6 Done Done Sent false
23981 7181 Acked Read 5 Done Done Sent false
23982 7182 Acked Read 10 Done Done Sent false
23983 7183 Acked Write 5 Done Done Sent false
23984 7184 Acked Read 10 Done Done Sent false
23985 7185 Acked Read 4 Done Done Sent false
23986 7186 Acked Read 2 Done Done Sent false
23987 7187 Acked Write 6 Done Done Sent false
23988 7188 Acked Write 8 Done Done Sent false
23989 7189 Acked Write 7 Done Done Sent false
23990 7190 Acked Read 4 Done Done Sent false
23991 7191 Acked Write 6 Done Done Sent false
23992 7192 Acked Read 2 Done Done Sent false
23993 7193 Acked Read 1 Done Done Sent false
23994 7194 Acked Read 9 Done Done Sent false
23995 7195 Acked Write 1 Done Done Sent false
23996 7196 Acked Write 10 Done Done Sent false
23997 7197 Acked Write 4 Done Done Sent false
23998 7198 Acked Write 4 Done Done Sent false
23999 7199 Acked Write 7 Done Done Sent false
24000 7200 Acked Read 2 Done Done Sent false
24001 7201 Acked Write 8 Done Done Sent false
24002 7202 Acked Write 5 Done Done Sent false
24003 7203 Acked Read 2 Done Done Sent false
24004 7204 Acked Write 7 Done Done Sent false
24005 7205 Acked Write 1 Done Done Sent false
24006 7206 Acked Write 6 Done Done Sent false
24007 7207 Acked Flush 0 Done Done Sent false
24008 7208 Acked Write 3 Done Done Sent false
24009 7209 Acked Read 5 Done Done Sent false
24010 7210 Acked Write 3 Done Done Sent false
24011 7211 Acked Read 10 Done Done Sent false
24012 7212 Acked Write 3 Done Done Sent false
24013 7213 Acked Read 2 Done Done Sent false
24014 7214 Acked Read 4 Done Done Sent false
24015 7215 Acked Write 6 Done Done Sent false
24016 7216 Acked Write 5 Done Done Sent false
24017 7217 Acked Write 8 Done Done Sent false
24018 7218 Acked Read 3 Done Done Sent false
24019 7219 Acked Write 10 Done Done Sent false
24020 7220 Acked Read 7 Done Done Sent false
24021 7221 Acked Flush 0 Done Done Sent false
24022 7222 Acked Read 3 Done Done Sent false
24023 7223 Acked Read 7 Done Done Sent false
24024 7224 Acked Write 2 Done Done Sent false
24025 7225 Acked Write 8 Done Done Sent false
24026 7226 Acked Write 10 Done Done Sent false
24027 7227 Acked Read 6 Done Done Sent false
24028 7228 Acked Write 7 Done Done Sent false
24029 7229 Acked Read 4 Done Done Sent false
24030 7230 Acked Read 5 Done Done Sent false
24031 7231 Acked Read 7 Done Done Sent false
24032 7232 Acked Read 2 Done Done Sent false
24033 7233 Acked Read 2 Done Done Sent false
24034 7234 Acked Write 9 Done Done Sent false
24035 7235 Acked Read 4 Done Done Sent false
24036 7236 Acked Read 5 Done Done Sent false
24037 7237 Acked Write 1 Done Done Sent false
24038 7238 Acked Write 8 Done Done Sent false
24039 7239 Acked Flush 0 Done Done Sent false
24040 7240 Acked Read 8 Done Done Sent false
24041 7241 Acked Read 9 Done Done Sent false
24042 7242 Acked Read 6 Done Done Sent false
24043 7243 Acked Read 9 Done Done Sent false
24044 7244 Acked Read 2 Done Done Sent false
24045 7245 Acked Write 9 Done Done Sent false
24046 7246 Acked Read 2 Done Done Sent false
24047 7247 Acked Write 5 Done Done Sent false
24048 7248 Acked Read 10 Done Done Sent false
24049 7249 Acked Read 5 Done Done Sent false
24050 7250 Acked Write 8 Done Done Sent false
24051 7251 Acked Read 3 Done Done Sent false
24052 7252 Acked Write 8 Done Done Sent false
24053 7253 Acked Write 4 Done Done Sent false
24054 7254 Acked Write 10 Done Done Sent false
24055 7255 Acked Write 2 Done Done Sent false
24056 7256 Acked Write 4 Done Done Sent false
24057 7257 Acked Write 1 Done Done Sent false
24058 7258 Acked Write 4 Done Done Sent false
24059 7259 Acked Write 6 Done Done Sent false
24060 7260 Acked Read 2 Done Done Sent false
24061 7261 Acked Write 9 Done Done Sent false
24062 7262 Acked Read 2 Done Done Sent false
24063 7263 Acked Read 10 Done Done Sent false
24064 7264 Acked Read 10 Done Done Sent false
24065 7265 Acked Write 6 Done Done Sent false
24066 7266 Acked Write 7 Done Done Sent false
24067 7267 Acked Read 2 Done Done Sent false
24068 7268 Acked Read 8 Done Done Sent false
24069 7269 Acked Flush 0 Done Done Sent false
24070 7270 Acked Write 8 Done Done Sent false
24071 7271 Acked Read 7 Done Done Sent false
24072 7272 Acked Write 9 Done Done Sent false
24073 7273 Acked Read 8 Done Done Sent false
24074 7274 Acked Write 4 Done Done Sent false
24075 7275 Acked Write 8 Done Done Sent false
24076 7276 Acked Flush 0 Done Done Sent false
24077 7277 Acked Write 6 Done Done Sent false
24078 7278 Acked Flush 0 Done Done Sent false
24079 7279 Acked Flush 0 Done Done Sent false
24080 7280 Acked Write 9 Done Done Sent false
24081 7281 Acked Write 8 Done Done Sent false
24082 7282 Acked Write 7 Done Done Sent false
24083 7283 Acked Write 3 Done Done Sent false
24084 7284 Acked Write 3 Done Done Sent false
24085 7285 Acked Write 10 Done Done Sent false
24086 7286 Acked Read 2 Done Done Sent false
24087 7287 Acked Read 2 Done Done Sent false
24088 7288 Acked Write 1 Done Done Sent false
24089 7289 Acked Read 5 Done Done Sent false
24090 7290 Acked Write 1 Done Done Sent false
24091 7291 Acked Read 5 Done Done Sent false
24092 7292 Acked Write 10 Done Done Sent false
24093 7293 Acked Read 1 Done Done Sent false
24094 7294 Acked Read 2 Done Done Sent false
24095 7295 Acked Write 7 Done Done Sent false
24096 7296 Acked Read 3 Done Done Sent false
24097 7297 Acked Read 9 Done Done Sent false
24098 7298 Acked Read 5 Done Done Sent false
24099 7299 Acked Write 7 Done Done Sent false
24100 7300 Acked Read 2 Done Done Sent false
24101 7301 Acked Flush 0 Done Done Sent false
24102 7302 Acked Read 6 Done Done Sent false
24103 7303 Acked Flush 0 Done Done Sent false
24104 7304 Acked Read 10 Done Done Sent false
24105 7305 Acked Write 8 Done Done Sent false
24106 7306 Acked Write 2 Done Done Sent false
24107 7307 Acked Read 8 Done Done Sent false
24108 7308 Acked Flush 0 Done Done Sent false
24109 7309 Acked Write 3 Done Done Sent false
24110 7310 Acked Read 5 Done Done Sent false
24111 7311 Acked Read 3 Done Done Sent false
24112 7312 Acked Read 5 Done Done Sent false
24113 7313 Acked Flush 0 Done Done Sent false
24114 7314 Acked Write 3 Done Done Sent false
24115 7315 Acked Write 8 Done Done Sent false
24116 7316 Acked Write 8 Done Done Sent false
24117 7317 Acked Read 10 Done Done Sent false
24118 7318 Acked Write 2 Done Done Sent false
24119 7319 Acked Read 6 Done Done Sent false
24120 7320 Acked Write 3 Done Done Sent false
24121 7321 Acked Write 1 Done Done Sent false
24122 7322 Acked Write 7 Done Done Sent false
24123 7323 Acked Flush 0 Done Done Sent false
24124 7324 Acked Read 3 Done Done Sent false
24125 7325 Acked Write 2 Done Done Sent false
24126 7326 Acked Write 9 Done Done Sent false
24127 7327 Acked Read 1 Done Done Sent false
24128 7328 Acked Write 5 Done Done Sent false
24129 7329 Acked Write 5 Done Done Sent false
24130 7330 Acked Read 3 Done Done Sent false
24131 7331 Acked Write 9 Done Done Sent false
24132 7332 Acked Write 7 Done Done Sent false
24133 7333 Acked Write 9 Done Done Sent false
24134 7334 Acked Write 8 Done Done Sent false
24135 7335 Acked Read 1 Done Done Sent false
24136 7336 Acked Read 7 Done Done Sent false
24137 7337 Acked Write 6 Done Done Sent false
24138 7338 Acked Write 2 Done Done Sent false
24139 7339 Acked Read 2 Done Done Sent false
24140 7340 Acked Flush 0 Done Done Sent false
24141 STATES DS:0 DS:1 DS:2 TOTAL
24142 Sent 0 0 302 302
24143 Done 302 302 0 604
24144 Skipped 0 0 0 0
24145 Error 0 0 0 0
24146 Last Flush: 7340 7340 7038
24147 Downstairs last five completed: 7038 7037 7036 7035 7034
24148 Upstairs last five completed: 7340 7339 7338 7337 7336
24149 CLIENT: Up:0 ds:302 act:2
24150 [8830] got start action
24151 [8830] I am going to restart
24152 Starting downstairs at port 8830
24153 Make output file at "/tmp/dsc/downstairs-8830.txt"
24154 [8830][2] reports Starting
24155 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1226)
24156 [8830][2] reports Running
24157 ----------------------------------------------------------------
24158 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24159 States: Active Active Offline
24160 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24161 7039 Acked Flush 0 Done Done Sent false
24162 7040 Acked Read 7 Done Done Sent false
24163 7041 Acked Read 2 Done Done Sent false
24164 7042 Acked Write 9 Done Done Sent false
24165 7043 Acked Write 3 Done Done Sent false
24166 7044 Acked Read 2 Done Done Sent false
24167 7045 Acked Flush 0 Done Done Sent false
24168 7046 Acked Read 10 Done Done Sent false
24169 7047 Acked Read 3 Done Done Sent false
24170 7048 Acked Write 2 Done Done Sent false
24171 7049 Acked Write 9 Done Done Sent false
24172 7050 Acked Write 5 Done Done Sent false
24173 7051 Acked Write 9 Done Done Sent false
24174 7052 Acked Flush 0 Done Done Sent false
24175 7053 Acked Read 6 Done Done Sent false
24176 7054 Acked Read 7 Done Done Sent false
24177 7055 Acked Flush 0 Done Done Sent false
24178 7056 Acked Read 9 Done Done Sent false
24179 7057 Acked Write 3 Done Done Sent false
24180 7058 Acked Read 1 Done Done Sent false
24181 7059 Acked Write 8 Done Done Sent false
24182 7060 Acked Write 6 Done Done Sent false
24183 7061 Acked Write 3 Done Done Sent false
24184 7062 Acked Write 10 Done Done Sent false
24185 7063 Acked Flush 0 Done Done Sent false
24186 7064 Acked Read 8 Done Done Sent false
24187 7065 Acked Read 2 Done Done Sent false
24188 7066 Acked Write 1 Done Done Sent false
24189 7067 Acked Flush 0 Done Done Sent false
24190 7068 Acked Read 4 Done Done Sent false
24191 7069 Acked Write 3 Done Done Sent false
24192 7070 Acked Read 7 Done Done Sent false
24193 7071 Acked Read 4 Done Done Sent false
24194 7072 Acked Write 10 Done Done Sent false
24195 7073 Acked Read 9 Done Done Sent false
24196 7074 Acked Flush 0 Done Done Sent false
24197 7075 Acked Read 8 Done Done Sent false
24198 7076 Acked Write 3 Done Done Sent false
24199 7077 Acked Read 3 Done Done Sent false
24200 7078 Acked Read 2 Done Done Sent false
24201 7079 Acked Write 1 Done Done Sent false
24202 7080 Acked Read 3 Done Done Sent false
24203 7081 Acked Read 10 Done Done Sent false
24204 7082 Acked Read 10 Done Done Sent false
24205 7083 Acked Write 6 Done Done Sent false
24206 7084 Acked Read 1 Done Done Sent false
24207 7085 Acked Write 1 Done Done Sent false
24208 7086 Acked Flush 0 Done Done Sent false
24209 7087 Acked Write 5 Done Done Sent false
24210 7088 Acked Read 4 Done Done Sent false
24211 7089 Acked Read 7 Done Done Sent false
24212 7090 Acked Write 1 Done Done Sent false
24213 7091 Acked Read 9 Done Done Sent false
24214 7092 Acked Write 2 Done Done Sent false
24215 7093 Acked Read 2 Done Done Sent false
24216 7094 Acked Read 4 Done Done Sent false
24217 7095 Acked Write 7 Done Done Sent false
24218 7096 Acked Read 1 Done Done Sent false
24219 7097 Acked Write 10 Done Done Sent false
24220 7098 Acked Read 7 Done Done Sent false
24221 7099 Acked Read 8 Done Done Sent false
24222 7100 Acked Write 9 Done Done Sent false
24223 7101 Acked Write 8 Done Done Sent false
24224 7102 Acked Read 6 Done Done Sent false
24225 7103 Acked Read 7 Done Done Sent false
24226 7104 Acked Read 9 Done Done Sent false
24227 7105 Acked Flush 0 Done Done Sent false
24228 7106 Acked Read 8 Done Done Sent false
24229 7107 Acked Read 9 Done Done Sent false
24230 7108 Acked Read 5 Done Done Sent false
24231 7109 Acked Read 3 Done Done Sent false
24232 7110 Acked Write 10 Done Done Sent false
24233 7111 Acked Read 6 Done Done Sent false
24234 7112 Acked Write 3 Done Done Sent false
24235 7113 Acked Write 1 Done Done Sent false
24236 7114 Acked Read 2 Done Done Sent false
24237 7115 Acked Write 10 Done Done Sent false
24238 7116 Acked Read 9 Done Done Sent false
24239 7117 Acked Write 1 Done Done Sent false
24240 7118 Acked Read 7 Done Done Sent false
24241 7119 Acked Read 3 Done Done Sent false
24242 7120 Acked Read 3 Done Done Sent false
24243 7121 Acked Flush 0 Done Done Sent false
24244 7122 Acked Read 3 Done Done Sent false
24245 7123 Acked Flush 0 Done Done Sent false
24246 7124 Acked Flush 0 Done Done Sent false
24247 7125 Acked Read 9 Done Done Sent false
24248 7126 Acked Read 7 Done Done Sent false
24249 7127 Acked Read 1 Done Done Sent false
24250 7128 Acked Flush 0 Done Done Sent false
24251 7129 Acked Read 8 Done Done Sent false
24252 7130 Acked Write 4 Done Done Sent false
24253 7131 Acked Write 6 Done Done Sent false
24254 7132 Acked Read 9 Done Done Sent false
24255 7133 Acked Write 9 Done Done Sent false
24256 7134 Acked Flush 0 Done Done Sent false
24257 7135 Acked Read 4 Done Done Sent false
24258 7136 Acked Write 8 Done Done Sent false
24259 7137 Acked Write 6 Done Done Sent false
24260 7138 Acked Read 7 Done Done Sent false
24261 7139 Acked Write 6 Done Done Sent false
24262 7140 Acked Read 8 Done Done Sent false
24263 7141 Acked Read 5 Done Done Sent false
24264 7142 Acked Write 10 Done Done Sent false
24265 7143 Acked Flush 0 Done Done Sent false
24266 7144 Acked Write 10 Done Done Sent false
24267 7145 Acked Read 7 Done Done Sent false
24268 7146 Acked Read 5 Done Done Sent false
24269 7147 Acked Read 3 Done Done Sent false
24270 7148 Acked Write 7 Done Done Sent false
24271 7149 Acked Flush 0 Done Done Sent false
24272 7150 Acked Write 5 Done Done Sent false
24273 7151 Acked Read 10 Done Done Sent false
24274 7152 Acked Read 10 Done Done Sent false
24275 7153 Acked Read 8 Done Done Sent false
24276 7154 Acked Write 1 Done Done Sent false
24277 7155 Acked Write 10 Done Done Sent false
24278 7156 Acked Write 8 Done Done Sent false
24279 7157 Acked Read 8 Done Done Sent false
24280 7158 Acked Write 3 Done Done Sent false
24281 7159 Acked Read 9 Done Done Sent false
24282 7160 Acked Read 4 Done Done Sent false
24283 7161 Acked Flush 0 Done Done Sent false
24284 7162 Acked Write 1 Done Done Sent false
24285 7163 Acked Write 4 Done Done Sent false
24286 7164 Acked Read 5 Done Done Sent false
24287 7165 Acked Read 8 Done Done Sent false
24288 7166 Acked Flush 0 Done Done Sent false
24289 7167 Acked Write 5 Done Done Sent false
24290 7168 Acked Read 4 Done Done Sent false
24291 7169 Acked Read 10 Done Done Sent false
24292 7170 Acked Read 8 Done Done Sent false
24293 7171 Acked Read 1 Done Done Sent false
24294 7172 Acked Read 5 Done Done Sent false
24295 7173 Acked Write 6 Done Done Sent false
24296 7174 Acked Write 5 Done Done Sent false
24297 7175 Acked Read 5 Done Done Sent false
24298 7176 Acked Write 3 Done Done Sent false
24299 7177 Acked Read 7 Done Done Sent false
24300 7178 Acked Write 9 Done Done Sent false
24301 7179 Acked Read 3 Done Done Sent false
24302 7180 Acked Read 6 Done Done Sent false
24303 7181 Acked Read 5 Done Done Sent false
24304 7182 Acked Read 10 Done Done Sent false
24305 7183 Acked Write 5 Done Done Sent false
24306 7184 Acked Read 10 Done Done Sent false
24307 7185 Acked Read 4 Done Done Sent false
24308 7186 Acked Read 2 Done Done Sent false
24309 7187 Acked Write 6 Done Done Sent false
24310 7188 Acked Write 8 Done Done Sent false
24311 7189 Acked Write 7 Done Done Sent false
24312 7190 Acked Read 4 Done Done Sent false
24313 7191 Acked Write 6 Done Done Sent false
24314 7192 Acked Read 2 Done Done Sent false
24315 7193 Acked Read 1 Done Done Sent false
24316 7194 Acked Read 9 Done Done Sent false
24317 7195 Acked Write 1 Done Done Sent false
24318 7196 Acked Write 10 Done Done Sent false
24319 7197 Acked Write 4 Done Done Sent false
24320 7198 Acked Write 4 Done Done Sent false
24321 7199 Acked Write 7 Done Done Sent false
24322 7200 Acked Read 2 Done Done Sent false
24323 7201 Acked Write 8 Done Done Sent false
24324 7202 Acked Write 5 Done Done Sent false
24325 7203 Acked Read 2 Done Done Sent false
24326 7204 Acked Write 7 Done Done Sent false
24327 7205 Acked Write 1 Done Done Sent false
24328 7206 Acked Write 6 Done Done Sent false
24329 7207 Acked Flush 0 Done Done Sent false
24330 7208 Acked Write 3 Done Done Sent false
24331 7209 Acked Read 5 Done Done Sent false
24332 7210 Acked Write 3 Done Done Sent false
24333 7211 Acked Read 10 Done Done Sent false
24334 7212 Acked Write 3 Done Done Sent false
24335 7213 Acked Read 2 Done Done Sent false
24336 7214 Acked Read 4 Done Done Sent false
24337 7215 Acked Write 6 Done Done Sent false
24338 7216 Acked Write 5 Done Done Sent false
24339 7217 Acked Write 8 Done Done Sent false
24340 7218 Acked Read 3 Done Done Sent false
24341 7219 Acked Write 10 Done Done Sent false
24342 7220 Acked Read 7 Done Done Sent false
24343 7221 Acked Flush 0 Done Done Sent false
24344 7222 Acked Read 3 Done Done Sent false
24345 7223 Acked Read 7 Done Done Sent false
24346 7224 Acked Write 2 Done Done Sent false
24347 7225 Acked Write 8 Done Done Sent false
24348 7226 Acked Write 10 Done Done Sent false
24349 7227 Acked Read 6 Done Done Sent false
24350 7228 Acked Write 7 Done Done Sent false
24351 7229 Acked Read 4 Done Done Sent false
24352 7230 Acked Read 5 Done Done Sent false
24353 7231 Acked Read 7 Done Done Sent false
24354 7232 Acked Read 2 Done Done Sent false
24355 7233 Acked Read 2 Done Done Sent false
24356 7234 Acked Write 9 Done Done Sent false
24357 7235 Acked Read 4 Done Done Sent false
24358 7236 Acked Read 5 Done Done Sent false
24359 7237 Acked Write 1 Done Done Sent false
24360 7238 Acked Write 8 Done Done Sent false
24361 7239 Acked Flush 0 Done Done Sent false
24362 7240 Acked Read 8 Done Done Sent false
24363 7241 Acked Read 9 Done Done Sent false
24364 7242 Acked Read 6 Done Done Sent false
24365 7243 Acked Read 9 Done Done Sent false
24366 7244 Acked Read 2 Done Done Sent false
24367 7245 Acked Write 9 Done Done Sent false
24368 7246 Acked Read 2 Done Done Sent false
24369 7247 Acked Write 5 Done Done Sent false
24370 7248 Acked Read 10 Done Done Sent false
24371 7249 Acked Read 5 Done Done Sent false
24372 7250 Acked Write 8 Done Done Sent false
24373 7251 Acked Read 3 Done Done Sent false
24374 7252 Acked Write 8 Done Done Sent false
24375 7253 Acked Write 4 Done Done Sent false
24376 7254 Acked Write 10 Done Done Sent false
24377 7255 Acked Write 2 Done Done Sent false
24378 7256 Acked Write 4 Done Done Sent false
24379 7257 Acked Write 1 Done Done Sent false
24380 7258 Acked Write 4 Done Done Sent false
24381 7259 Acked Write 6 Done Done Sent false
24382 7260 Acked Read 2 Done Done Sent false
24383 7261 Acked Write 9 Done Done Sent false
24384 7262 Acked Read 2 Done Done Sent false
24385 7263 Acked Read 10 Done Done Sent false
24386 7264 Acked Read 10 Done Done Sent false
24387 7265 Acked Write 6 Done Done Sent false
24388 7266 Acked Write 7 Done Done Sent false
24389 7267 Acked Read 2 Done Done Sent false
24390 7268 Acked Read 8 Done Done Sent false
24391 7269 Acked Flush 0 Done Done Sent false
24392 7270 Acked Write 8 Done Done Sent false
24393 7271 Acked Read 7 Done Done Sent false
24394 7272 Acked Write 9 Done Done Sent false
24395 7273 Acked Read 8 Done Done Sent false
24396 7274 Acked Write 4 Done Done Sent false
24397 7275 Acked Write 8 Done Done Sent false
24398 7276 Acked Flush 0 Done Done Sent false
24399 7277 Acked Write 6 Done Done Sent false
24400 7278 Acked Flush 0 Done Done Sent false
24401 7279 Acked Flush 0 Done Done Sent false
24402 7280 Acked Write 9 Done Done Sent false
24403 7281 Acked Write 8 Done Done Sent false
24404 7282 Acked Write 7 Done Done Sent false
24405 7283 Acked Write 3 Done Done Sent false
24406 7284 Acked Write 3 Done Done Sent false
24407 7285 Acked Write 10 Done Done Sent false
24408 7286 Acked Read 2 Done Done Sent false
24409 7287 Acked Read 2 Done Done Sent false
24410 7288 Acked Write 1 Done Done Sent false
24411 7289 Acked Read 5 Done Done Sent false
24412 7290 Acked Write 1 Done Done Sent false
24413 7291 Acked Read 5 Done Done Sent false
24414 7292 Acked Write 10 Done Done Sent false
24415 7293 Acked Read 1 Done Done Sent false
24416 7294 Acked Read 2 Done Done Sent false
24417 7295 Acked Write 7 Done Done Sent false
24418 7296 Acked Read 3 Done Done Sent false
24419 7297 Acked Read 9 Done Done Sent false
24420 7298 Acked Read 5 Done Done Sent false
24421 7299 Acked Write 7 Done Done Sent false
24422 7300 Acked Read 2 Done Done Sent false
24423 7301 Acked Flush 0 Done Done Sent false
24424 7302 Acked Read 6 Done Done Sent false
24425 7303 Acked Flush 0 Done Done Sent false
24426 7304 Acked Read 10 Done Done Sent false
24427 7305 Acked Write 8 Done Done Sent false
24428 7306 Acked Write 2 Done Done Sent false
24429 7307 Acked Read 8 Done Done Sent false
24430 7308 Acked Flush 0 Done Done Sent false
24431 7309 Acked Write 3 Done Done Sent false
24432 7310 Acked Read 5 Done Done Sent false
24433 7311 Acked Read 3 Done Done Sent false
24434 7312 Acked Read 5 Done Done Sent false
24435 7313 Acked Flush 0 Done Done Sent false
24436 7314 Acked Write 3 Done Done Sent false
24437 7315 Acked Write 8 Done Done Sent false
24438 7316 Acked Write 8 Done Done Sent false
24439 7317 Acked Read 10 Done Done Sent false
24440 7318 Acked Write 2 Done Done Sent false
24441 7319 Acked Read 6 Done Done Sent false
24442 7320 Acked Write 3 Done Done Sent false
24443 7321 Acked Write 1 Done Done Sent false
24444 7322 Acked Write 7 Done Done Sent false
24445 7323 Acked Flush 0 Done Done Sent false
24446 7324 Acked Read 3 Done Done Sent false
24447 7325 Acked Write 2 Done Done Sent false
24448 7326 Acked Write 9 Done Done Sent false
24449 7327 Acked Read 1 Done Done Sent false
24450 7328 Acked Write 5 Done Done Sent false
24451 7329 Acked Write 5 Done Done Sent false
24452 7330 Acked Read 3 Done Done Sent false
24453 7331 Acked Write 9 Done Done Sent false
24454 7332 Acked Write 7 Done Done Sent false
24455 7333 Acked Write 9 Done Done Sent false
24456 7334 Acked Write 8 Done Done Sent false
24457 7335 Acked Read 1 Done Done Sent false
24458 7336 Acked Read 7 Done Done Sent false
24459 7337 Acked Write 6 Done Done Sent false
24460 7338 Acked Write 2 Done Done Sent false
24461 7339 Acked Read 2 Done Done Sent false
24462 7340 Acked Flush 0 Done Done Sent false
24463 STATES DS:0 DS:1 DS:2 TOTAL
24464 Sent 0 0 302 302
24465 Done 302 302 0 604
24466 Skipped 0 0 0 0
24467 Error 0 0 0 0
24468 Last Flush: 7340 7340 7038
24469 Downstairs last five completed: 7038 7037 7036 7035 7034
24470 Upstairs last five completed: 7340 7339 7338 7337 7336
24471 CLIENT: Up:0 ds:302 act:2
24472 ----------------------------------------------------------------
24473 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24474 States: Active Active Offline
24475 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24476 7039 Acked Flush 0 Done Done Sent false
24477 7040 Acked Read 7 Done Done Sent false
24478 7041 Acked Read 2 Done Done Sent false
24479 7042 Acked Write 9 Done Done Sent false
24480 7043 Acked Write 3 Done Done Sent false
24481 7044 Acked Read 2 Done Done Sent false
24482 7045 Acked Flush 0 Done Done Sent false
24483 7046 Acked Read 10 Done Done Sent false
24484 7047 Acked Read 3 Done Done Sent false
24485 7048 Acked Write 2 Done Done Sent false
24486 7049 Acked Write 9 Done Done Sent false
24487 7050 Acked Write 5 Done Done Sent false
24488 7051 Acked Write 9 Done Done Sent false
24489 7052 Acked Flush 0 Done Done Sent false
24490 7053 Acked Read 6 Done Done Sent false
24491 7054 Acked Read 7 Done Done Sent false
24492 7055 Acked Flush 0 Done Done Sent false
24493 7056 Acked Read 9 Done Done Sent false
24494 7057 Acked Write 3 Done Done Sent false
24495 7058 Acked Read 1 Done Done Sent false
24496 7059 Acked Write 8 Done Done Sent false
24497 7060 Acked Write 6 Done Done Sent false
24498 7061 Acked Write 3 Done Done Sent false
24499 7062 Acked Write 10 Done Done Sent false
24500 7063 Acked Flush 0 Done Done Sent false
24501 7064 Acked Read 8 Done Done Sent false
24502 7065 Acked Read 2 Done Done Sent false
24503 7066 Acked Write 1 Done Done Sent false
24504 7067 Acked Flush 0 Done Done Sent false
24505 7068 Acked Read 4 Done Done Sent false
24506 7069 Acked Write 3 Done Done Sent false
24507 7070 Acked Read 7 Done Done Sent false
24508 7071 Acked Read 4 Done Done Sent false
24509 7072 Acked Write 10 Done Done Sent false
24510 7073 Acked Read 9 Done Done Sent false
24511 7074 Acked Flush 0 Done Done Sent false
24512 7075 Acked Read 8 Done Done Sent false
24513 7076 Acked Write 3 Done Done Sent false
24514 7077 Acked Read 3 Done Done Sent false
24515 7078 Acked Read 2 Done Done Sent false
24516 7079 Acked Write 1 Done Done Sent false
24517 7080 Acked Read 3 Done Done Sent false
24518 7081 Acked Read 10 Done Done Sent false
24519 7082 Acked Read 10 Done Done Sent false
24520 7083 Acked Write 6 Done Done Sent false
24521 7084 Acked Read 1 Done Done Sent false
24522 7085 Acked Write 1 Done Done Sent false
24523 7086 Acked Flush 0 Done Done Sent false
24524 7087 Acked Write 5 Done Done Sent false
24525 7088 Acked Read 4 Done Done Sent false
24526 7089 Acked Read 7 Done Done Sent false
24527 7090 Acked Write 1 Done Done Sent false
24528 7091 Acked Read 9 Done Done Sent false
24529 7092 Acked Write 2 Done Done Sent false
24530 7093 Acked Read 2 Done Done Sent false
24531 7094 Acked Read 4 Done Done Sent false
24532 7095 Acked Write 7 Done Done Sent false
24533 7096 Acked Read 1 Done Done Sent false
24534 7097 Acked Write 10 Done Done Sent false
24535 7098 Acked Read 7 Done Done Sent false
24536 7099 Acked Read 8 Done Done Sent false
24537 7100 Acked Write 9 Done Done Sent false
24538 7101 Acked Write 8 Done Done Sent false
24539 7102 Acked Read 6 Done Done Sent false
24540 7103 Acked Read 7 Done Done Sent false
24541 7104 Acked Read 9 Done Done Sent false
24542 7105 Acked Flush 0 Done Done Sent false
24543 7106 Acked Read 8 Done Done Sent false
24544 7107 Acked Read 9 Done Done Sent false
24545 7108 Acked Read 5 Done Done Sent false
24546 7109 Acked Read 3 Done Done Sent false
24547 7110 Acked Write 10 Done Done Sent false
24548 7111 Acked Read 6 Done Done Sent false
24549 7112 Acked Write 3 Done Done Sent false
24550 7113 Acked Write 1 Done Done Sent false
24551 7114 Acked Read 2 Done Done Sent false
24552 7115 Acked Write 10 Done Done Sent false
24553 7116 Acked Read 9 Done Done Sent false
24554 7117 Acked Write 1 Done Done Sent false
24555 7118 Acked Read 7 Done Done Sent false
24556 7119 Acked Read 3 Done Done Sent false
24557 7120 Acked Read 3 Done Done Sent false
24558 7121 Acked Flush 0 Done Done Sent false
24559 7122 Acked Read 3 Done Done Sent false
24560 7123 Acked Flush 0 Done Done Sent false
24561 7124 Acked Flush 0 Done Done Sent false
24562 7125 Acked Read 9 Done Done Sent false
24563 7126 Acked Read 7 Done Done Sent false
24564 7127 Acked Read 1 Done Done Sent false
24565 7128 Acked Flush 0 Done Done Sent false
24566 7129 Acked Read 8 Done Done Sent false
24567 7130 Acked Write 4 Done Done Sent false
24568 7131 Acked Write 6 Done Done Sent false
24569 7132 Acked Read 9 Done Done Sent false
24570 7133 Acked Write 9 Done Done Sent false
24571 7134 Acked Flush 0 Done Done Sent false
24572 7135 Acked Read 4 Done Done Sent false
24573 7136 Acked Write 8 Done Done Sent false
24574 7137 Acked Write 6 Done Done Sent false
24575 7138 Acked Read 7 Done Done Sent false
24576 7139 Acked Write 6 Done Done Sent false
24577 7140 Acked Read 8 Done Done Sent false
24578 7141 Acked Read 5 Done Done Sent false
24579 7142 Acked Write 10 Done Done Sent false
24580 7143 Acked Flush 0 Done Done Sent false
24581 7144 Acked Write 10 Done Done Sent false
24582 7145 Acked Read 7 Done Done Sent false
24583 7146 Acked Read 5 Done Done Sent false
24584 7147 Acked Read 3 Done Done Sent false
24585 7148 Acked Write 7 Done Done Sent false
24586 7149 Acked Flush 0 Done Done Sent false
24587 7150 Acked Write 5 Done Done Sent false
24588 7151 Acked Read 10 Done Done Sent false
24589 7152 Acked Read 10 Done Done Sent false
24590 7153 Acked Read 8 Done Done Sent false
24591 7154 Acked Write 1 Done Done Sent false
24592 7155 Acked Write 10 Done Done Sent false
24593 7156 Acked Write 8 Done Done Sent false
24594 7157 Acked Read 8 Done Done Sent false
24595 7158 Acked Write 3 Done Done Sent false
24596 7159 Acked Read 9 Done Done Sent false
24597 7160 Acked Read 4 Done Done Sent false
24598 7161 Acked Flush 0 Done Done Sent false
24599 7162 Acked Write 1 Done Done Sent false
24600 7163 Acked Write 4 Done Done Sent false
24601 7164 Acked Read 5 Done Done Sent false
24602 7165 Acked Read 8 Done Done Sent false
24603 7166 Acked Flush 0 Done Done Sent false
24604 7167 Acked Write 5 Done Done Sent false
24605 7168 Acked Read 4 Done Done Sent false
24606 7169 Acked Read 10 Done Done Sent false
24607 7170 Acked Read 8 Done Done Sent false
24608 7171 Acked Read 1 Done Done Sent false
24609 7172 Acked Read 5 Done Done Sent false
24610 7173 Acked Write 6 Done Done Sent false
24611 7174 Acked Write 5 Done Done Sent false
24612 7175 Acked Read 5 Done Done Sent false
24613 7176 Acked Write 3 Done Done Sent false
24614 7177 Acked Read 7 Done Done Sent false
24615 7178 Acked Write 9 Done Done Sent false
24616 7179 Acked Read 3 Done Done Sent false
24617 7180 Acked Read 6 Done Done Sent false
24618 7181 Acked Read 5 Done Done Sent false
24619 7182 Acked Read 10 Done Done Sent false
24620 7183 Acked Write 5 Done Done Sent false
24621 7184 Acked Read 10 Done Done Sent false
24622 7185 Acked Read 4 Done Done Sent false
24623 7186 Acked Read 2 Done Done Sent false
24624 7187 Acked Write 6 Done Done Sent false
24625 7188 Acked Write 8 Done Done Sent false
24626 7189 Acked Write 7 Done Done Sent false
24627 7190 Acked Read 4 Done Done Sent false
24628 7191 Acked Write 6 Done Done Sent false
24629 7192 Acked Read 2 Done Done Sent false
24630 7193 Acked Read 1 Done Done Sent false
24631 7194 Acked Read 9 Done Done Sent false
24632 7195 Acked Write 1 Done Done Sent false
24633 7196 Acked Write 10 Done Done Sent false
24634 7197 Acked Write 4 Done Done Sent false
24635 7198 Acked Write 4 Done Done Sent false
24636 7199 Acked Write 7 Done Done Sent false
24637 7200 Acked Read 2 Done Done Sent false
24638 7201 Acked Write 8 Done Done Sent false
24639 7202 Acked Write 5 Done Done Sent false
24640 7203 Acked Read 2 Done Done Sent false
24641 7204 Acked Write 7 Done Done Sent false
24642 7205 Acked Write 1 Done Done Sent false
24643 7206 Acked Write 6 Done Done Sent false
24644 7207 Acked Flush 0 Done Done Sent false
24645 7208 Acked Write 3 Done Done Sent false
24646 7209 Acked Read 5 Done Done Sent false
24647 7210 Acked Write 3 Done Done Sent false
24648 7211 Acked Read 10 Done Done Sent false
24649 7212 Acked Write 3 Done Done Sent false
24650 7213 Acked Read 2 Done Done Sent false
24651 7214 Acked Read 4 Done Done Sent false
24652 7215 Acked Write 6 Done Done Sent false
24653 7216 Acked Write 5 Done Done Sent false
24654 7217 Acked Write 8 Done Done Sent false
24655 7218 Acked Read 3 Done Done Sent false
24656 7219 Acked Write 10 Done Done Sent false
24657 7220 Acked Read 7 Done Done Sent false
24658 7221 Acked Flush 0 Done Done Sent false
24659 7222 Acked Read 3 Done Done Sent false
24660 7223 Acked Read 7 Done Done Sent false
24661 7224 Acked Write 2 Done Done Sent false
24662 7225 Acked Write 8 Done Done Sent false
24663 7226 Acked Write 10 Done Done Sent false
24664 7227 Acked Read 6 Done Done Sent false
24665 7228 Acked Write 7 Done Done Sent false
24666 7229 Acked Read 4 Done Done Sent false
24667 7230 Acked Read 5 Done Done Sent false
24668 7231 Acked Read 7 Done Done Sent false
24669 7232 Acked Read 2 Done Done Sent false
24670 7233 Acked Read 2 Done Done Sent false
24671 7234 Acked Write 9 Done Done Sent false
24672 7235 Acked Read 4 Done Done Sent false
24673 7236 Acked Read 5 Done Done Sent false
24674 7237 Acked Write 1 Done Done Sent false
24675 7238 Acked Write 8 Done Done Sent false
24676 7239 Acked Flush 0 Done Done Sent false
24677 7240 Acked Read 8 Done Done Sent false
24678 7241 Acked Read 9 Done Done Sent false
24679 7242 Acked Read 6 Done Done Sent false
24680 7243 Acked Read 9 Done Done Sent false
24681 7244 Acked Read 2 Done Done Sent false
24682 7245 Acked Write 9 Done Done Sent false
24683 7246 Acked Read 2 Done Done Sent false
24684 7247 Acked Write 5 Done Done Sent false
24685 7248 Acked Read 10 Done Done Sent false
24686 7249 Acked Read 5 Done Done Sent false
24687 7250 Acked Write 8 Done Done Sent false
24688 7251 Acked Read 3 Done Done Sent false
24689 7252 Acked Write 8 Done Done Sent false
24690 7253 Acked Write 4 Done Done Sent false
24691 7254 Acked Write 10 Done Done Sent false
24692 7255 Acked Write 2 Done Done Sent false
24693 7256 Acked Write 4 Done Done Sent false
24694 7257 Acked Write 1 Done Done Sent false
24695 7258 Acked Write 4 Done Done Sent false
24696 7259 Acked Write 6 Done Done Sent false
24697 7260 Acked Read 2 Done Done Sent false
24698 7261 Acked Write 9 Done Done Sent false
24699 7262 Acked Read 2 Done Done Sent false
24700 7263 Acked Read 10 Done Done Sent false
24701 7264 Acked Read 10 Done Done Sent false
24702 7265 Acked Write 6 Done Done Sent false
24703 7266 Acked Write 7 Done Done Sent false
24704 7267 Acked Read 2 Done Done Sent false
24705 7268 Acked Read 8 Done Done Sent false
24706 7269 Acked Flush 0 Done Done Sent false
24707 7270 Acked Write 8 Done Done Sent false
24708 7271 Acked Read 7 Done Done Sent false
24709 7272 Acked Write 9 Done Done Sent false
24710 7273 Acked Read 8 Done Done Sent false
24711 7274 Acked Write 4 Done Done Sent false
24712 7275 Acked Write 8 Done Done Sent false
24713 7276 Acked Flush 0 Done Done Sent false
24714 7277 Acked Write 6 Done Done Sent false
24715 7278 Acked Flush 0 Done Done Sent false
24716 7279 Acked Flush 0 Done Done Sent false
24717 7280 Acked Write 9 Done Done Sent false
24718 7281 Acked Write 8 Done Done Sent false
24719 7282 Acked Write 7 Done Done Sent false
24720 7283 Acked Write 3 Done Done Sent false
24721 7284 Acked Write 3 Done Done Sent false
24722 7285 Acked Write 10 Done Done Sent false
24723 7286 Acked Read 2 Done Done Sent false
24724 7287 Acked Read 2 Done Done Sent false
24725 7288 Acked Write 1 Done Done Sent false
24726 7289 Acked Read 5 Done Done Sent false
24727 7290 Acked Write 1 Done Done Sent false
24728 7291 Acked Read 5 Done Done Sent false
24729 7292 Acked Write 10 Done Done Sent false
24730 7293 Acked Read 1 Done Done Sent false
24731 7294 Acked Read 2 Done Done Sent false
24732 7295 Acked Write 7 Done Done Sent false
24733 7296 Acked Read 3 Done Done Sent false
24734 7297 Acked Read 9 Done Done Sent false
24735 7298 Acked Read 5 Done Done Sent false
24736 7299 Acked Write 7 Done Done Sent false
24737 7300 Acked Read 2 Done Done Sent false
24738 7301 Acked Flush 0 Done Done Sent false
24739 7302 Acked Read 6 Done Done Sent false
24740 7303 Acked Flush 0 Done Done Sent false
24741 7304 Acked Read 10 Done Done Sent false
24742 7305 Acked Write 8 Done Done Sent false
24743 7306 Acked Write 2 Done Done Sent false
24744 7307 Acked Read 8 Done Done Sent false
24745 7308 Acked Flush 0 Done Done Sent false
24746 7309 Acked Write 3 Done Done Sent false
24747 7310 Acked Read 5 Done Done Sent false
24748 7311 Acked Read 3 Done Done Sent false
24749 7312 Acked Read 5 Done Done Sent false
24750 7313 Acked Flush 0 Done Done Sent false
24751 7314 Acked Write 3 Done Done Sent false
24752 7315 Acked Write 8 Done Done Sent false
24753 7316 Acked Write 8 Done Done Sent false
24754 7317 Acked Read 10 Done Done Sent false
24755 7318 Acked Write 2 Done Done Sent false
24756 7319 Acked Read 6 Done Done Sent false
24757 7320 Acked Write 3 Done Done Sent false
24758 7321 Acked Write 1 Done Done Sent false
24759 7322 Acked Write 7 Done Done Sent false
24760 7323 Acked Flush 0 Done Done Sent false
24761 7324 Acked Read 3 Done Done Sent false
24762 7325 Acked Write 2 Done Done Sent false
24763 7326 Acked Write 9 Done Done Sent false
24764 7327 Acked Read 1 Done Done Sent false
24765 7328 Acked Write 5 Done Done Sent false
24766 7329 Acked Write 5 Done Done Sent false
24767 7330 Acked Read 3 Done Done Sent false
24768 7331 Acked Write 9 Done Done Sent false
24769 7332 Acked Write 7 Done Done Sent false
24770 7333 Acked Write 9 Done Done Sent false
24771 7334 Acked Write 8 Done Done Sent false
24772 7335 Acked Read 1 Done Done Sent false
24773 7336 Acked Read 7 Done Done Sent false
24774 7337 Acked Write 6 Done Done Sent false
24775 7338 Acked Write 2 Done Done Sent false
24776 7339 Acked Read 2 Done Done Sent false
24777 7340 Acked Flush 0 Done Done Sent false
24778 STATES DS:0 DS:1 DS:2 TOTAL
24779 Sent 0 0 302 302
24780 Done 302 302 0 604
24781 Skipped 0 0 0 0
24782 Error 0 0 0 0
24783 Last Flush: 7340 7340 7038
24784 Downstairs last five completed: 7038 7037 7036 7035 7034
24785 Upstairs last five completed: 7340 7339 7338 7337 7336
24786 CLIENT: Up:0 ds:302 act:2
247872025-01-03T08:36:04.604ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247882025-01-03T08:36:04.604ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41885) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247892025-01-03T08:36:04.605ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247902025-01-03T08:36:04.606ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247912025-01-03T08:36:04.606ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247922025-01-03T08:36:04.606ZINFOcrucible: send last flush ID to this DS: 7038 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247932025-01-03T08:36:04.606ZINFOcrucible: Replied this last flush ID: 7038 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247942025-01-03T08:36:04.606ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247952025-01-03T08:36:04.606ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247962025-01-03T08:36:04.606ZINFOcrucible: [2] client re-new 302 jobs since flush 7038 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
247972025-01-03T08:36:04.607ZINFOcrucible: [2] Replayed 302 jobs since flush: 7038 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
24798 ----------------------------------------------------------------
24799 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24800 Downstairs last five completed: 7340 7339 7338 7337 7336
24801 Upstairs last five completed: 7340 7339 7338 7337 7336
24802 CLIENT: Up:0 ds:0 act:3
24803 Replay: All jobs finished
24804 Main task has work to do, go find it
24805 got dsc Stop(0)
24806 stop 0
24807 Jan 03 08:36:06.855 INFO [8810] Got stop action so:false kr:false
24808 request completed, latency_us: 360, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 35cefa2a-2173-4921-9d3d-f6b5ee05be91, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
24809 Jan 03 08:36:06.856 INFO {"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:36:06.856779862Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
24810 request completed, latency_us: 150, response_code: 200, uri: /state/cid/0, method: GET, req_id: 3ea40225-4a03-411e-ac5b-54e151b420b1, remote_addr: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:36:06.85685909Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"0","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
24811 127.0.0.1:49669, local_addr: 127.0.0.1:9998{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:36:06.85688637Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
24812 
248132025-01-03T08:36:06.856ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
248142025-01-03T08:36:06.856ZINFOcrucible: [0] Transition from Active to Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
248152025-01-03T08:36:06.856ZINFOcrucible: client task is exiting = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
24816 [8810] Exited with: ExitStatus(unix_wait_status(9))
24817 [8810][0] reports Exit
24818 Jan 03 08:36:10.859 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/0, method: GET, req_id: 72956c23-67ec-4af1-80f3-e9b558fdb631, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
24819 001/300 Flush
24820 002/300 Write block 2972 len 32768 data: 7 7 8 9 8 7 5 2
24821 003/300 Read block 3408 len 20480
24822 004/300 Write block 132 len 40960 data: 4 3 3 3 3 5 5 5 6 5
24823 005/300 Read block 1891 len 16384
24824 006/300 Read block 356 len 8192
24825 007/300 Write block 3151 len 4096 data: 3
24826 008/300 Read block 2806 len 20480
24827 009/300 Read block 1045 len 40960
24828 010/300 Read block 3212 len 28672
24829 011/300 Read block 38 len 32768
24830 012/300 Write block 3983 len 28672 data: 3 3 4 4 4 4 4
24831 013/300 Read block 2110 len 40960
24832 014/300 Read block 1544 len 24576
24833 015/300 Flush
24834 016/300 Write block 2136 len 12288 data: 6 6 3
24835 017/300 Read block 385 len 40960
24836 018/300 Write block 157 len 40960 data: 5 5 5 5 4 4 3 4 4 4
24837 019/300 Flush
24838 020/300 Read block 3775 len 40960
24839 021/300 Write block 2664 len 40960 data: 7 7 7 7 5 4 3 3 3 2
24840 022/300 Write block 454 len 36864 data: 4 4 3 4 6 6 5 5 6
24841 023/300 Read block 217 len 28672
24842 024/300 Write block 3660 len 24576 data: 4 4 6 5 6 3
24843 025/300 Read block 4702 len 24576
24844 026/300 Write block 2530 len 36864 data: 4 4 4 4 4 4 3 3 4
24845 027/300 Write block 422 len 28672 data: 7 6 7 6 6 4 4
24846 028/300 Read block 1891 len 32768
24847 029/300 Read block 815 len 40960
24848 030/300 Read block 3455 len 8192
24849 031/300 Read block 472 len 36864
24850 032/300 Write block 474 len 20480 data: 3 3 3 3 3
24851 033/300 Read block 1628 len 20480
24852 034/300 Write block 2595 len 32768 data: 4 4 5 5 6 6 6 5
24853 035/300 Write block 4263 len 36864 data: 8 8 8 9 6 6 6 6 4
24854 036/300 Flush
24855 037/300 Read block 4671 len 24576
24856 038/300 Read block 2855 len 28672
24857 039/300 Write block 896 len 24576 data: 7 6 5 5 5 5
24858 040/300 Write block 2910 len 8192 data: 5 4
24859 041/300 Read block 2025 len 8192
24860 042/300 Write block 2956 len 12288 data: 2 2 3
24861 043/300 Flush
24862 044/300 Write block 2459 len 36864 data: 8 5 4 4 4 4 3 4 5
24863 045/300 Write block 924 len 20480 data: 5 6 6 5 5
24864 046/300 Read block 562 len 12288
24865 047/300 Write block 3573 len 4096 data: 4
24866 048/300 Read block 2883 len 4096
24867 049/300 Write block 303 len 12288 data: 6 6 8
24868 050/300 Read block 1872 len 32768
24869 051/300 Write block 345 len 4096 data: 8
24870 052/300 Write block 1178 len 8192 data: 6 6
24871 053/300 Read block 128 len 36864
24872 054/300 Write block 4249 len 24576 data: 5 5 7 8 7 7
24873 055/300 Write block 769 len 12288 data: 4 4 3
24874 056/300 Write block 2003 len 28672 data: 2 2 2 2 3 2 3
24875 057/300 Write block 682 len 4096 data: 5
24876 058/300 Read block 1902 len 16384
24877 059/300 Flush
24878 060/300 Write block 0 len 8192 data: 2 2
24879 061/300 Read block 2951 len 16384
24880 062/300 Read block 4418 len 20480
24881 063/300 Write block 3489 len 24576 data: 3 3 4 5 5 5
24882 064/300 Flush
24883 065/300 Write block 3825 len 40960 data: 8 8 7 7 6 7 6 7 6 4
24884 066/300 Read block 3309 len 20480
24885 067/300 Read block 4379 len 24576
24886 068/300 Read block 2768 len 24576
24887 069/300 Read block 636 len 4096
24888 070/300 Read block 4560 len 20480
24889 071/300 Write block 4687 len 8192 data: 5 5
24890 072/300 Write block 2007 len 20480 data: 4 3 4 3 4
24891 073/300 Write block 3836 len 36864 data: 5 5 5 4 3 2 3 5 4
24892 074/300 Write block 1979 len 32768 data: 4 5 4 5 3 3 2 2
24893 075/300 Write block 4583 len 8192 data: 3 3
24894 076/300 Read block 730 len 8192
24895 077/300 Write block 3399 len 20480 data: 7 7 8 8 7
24896 078/300 Read block 3468 len 40960
24897 079/300 Write block 1221 len 12288 data: 5 4 5
24898 080/300 Read block 1031 len 32768
24899 081/300 Write block 691 len 4096 data: 5
24900 082/300 Write block 80 len 16384 data: 4 4 3 5
24901 083/300 Read block 4349 len 24576
24902 084/300 Read block 463 len 40960
24903 085/300 Write block 3405 len 8192 data: 5 5
24904 086/300 Read block 4055 len 28672
24905 087/300 Read block 2993 len 12288
24906 088/300 Write block 3352 len 16384 data: 5 5 7 5
24907 089/300 Write block 4207 len 20480 data: 7 6 5 4 5
24908 090/300 Read block 2090 len 4096
24909 091/300 Read block 4012 len 16384
24910 092/300 Read block 4702 len 36864
24911 093/300 Write block 937 len 28672 data: 8 8 5 4 4 4 4
24912 094/300 Write block 1468 len 16384 data: 5 3 3 3
24913 095/300 Flush
24914 096/300 Write block 3422 len 16384 data: 9 9 10 9
24915 097/300 Read block 3386 len 24576
24916 098/300 Write block 3314 len 8192 data: 4 5
24917 099/300 Write block 3880 len 20480 data: 4 6 6 7 6
24918 100/300 Read block 3159 len 36864
24919 101/300 Read block 3664 len 4096
24920 102/300 Write block 1517 len 40960 data: 4 5 6 6 5 5 8 8 8 7
24921 103/300 Read block 4380 len 16384
24922 104/300 Write block 1525 len 8192 data: 9 8
24923 105/300 Flush
24924 106/300 Flush
24925 107/300 Read block 4326 len 40960
24926 108/300 Write block 1011 len 28672 data: 4 4 5 5 5 5 4
24927 109/300 Write block 2791 len 4096 data: 3
24928 110/300 Read block 2536 len 20480
24929 111/300 Read block 2654 len 24576
24930 112/300 Write block 565 len 36864 data: 3 3 3 3 3 3 2 2 2
24931 113/300 Write block 2823 len 40960 data: 5 5 6 8 7 5 5 5 5 4
24932 114/300 Write block 4436 len 24576 data: 4 3 3 5 5 7
24933 115/300 Read block 2042 len 28672
24934 116/300 Read block 3647 len 36864
24935 117/300 Read block 3463 len 36864
24936 118/300 Write block 4374 len 24576 data: 5 5 3 3 4 3
24937 119/300 Write block 373 len 36864 data: 3 3 3 3 3 5 4 3 3
24938 120/300 Write block 3896 len 20480 data: 8 7 5 5 5
24939 121/300 Write block 4028 len 4096 data: 4
24940 122/300 Write block 891 len 32768 data: 9 7 6 7 7 8 7 6
24941 123/300 Write block 1220 len 24576 data: 5 6 5 6 6 6
24942 124/300 Read block 2868 len 12288
24943 125/300 Write block 1493 len 32768 data: 4 4 3 3 4 5 3 3
24944 126/300 Write block 309 len 24576 data: 9 11 8 8 8 7
24945 127/300 Read block 1062 len 32768
24946 128/300 Read block 3347 len 28672
24947 129/300 Read block 1177 len 4096
24948 130/300 Read block 3260 len 40960
24949 131/300 Read block 1466 len 12288
24950 132/300 Read block 2807 len 36864
24951 133/300 Write block 1369 len 16384 data: 4 7 4 4
24952 134/300 Write block 3742 len 8192 data: 2 4
24953 135/300 Read block 193 len 20480
24954 136/300 Read block 4020 len 16384
24955 137/300 Read block 4595 len 36864
24956 138/300 Flush
24957 139/300 Read block 3922 len 24576
24958 140/300 Write block 1929 len 36864 data: 3 3 3 3 2 2 2 3 3
24959 141/300 Read block 1059 len 28672
24960 142/300 Write block 1868 len 4096 data: 3
24961 143/300 Read block 1967 len 28672
24962 144/300 Write block 3861 len 16384 data: 7 8 9 7
24963 145/300 Read block 393 len 20480
24964 146/300 Read block 1009 len 8192
24965 147/300 Read block 3680 len 8192
24966 148/300 Write block 3944 len 16384 data: 3 2 4 4
24967 149/300 Read block 230 len 20480
24968 150/300 Read block 102 len 20480
24969 151/300 Flush
24970 152/300 Write block 4269 len 12288 data: 7 7 5
24971 153/300 Read block 732 len 20480
24972 154/300 Read block 1195 len 36864
24973 155/300 Read block 2698 len 36864
24974 156/300 Write block 2409 len 8192 data: 5 3
24975 157/300 Write block 1291 len 16384 data: 6 5 5 4
24976 158/300 Write block 867 len 16384 data: 4 4 4 4
24977 159/300 Read block 1092 len 32768
24978 160/300 Read block 1073 len 36864
24979 161/300 Write block 1050 len 20480 data: 4 4 3 4 5
24980 162/300 Read block 333 len 20480
24981 163/300 Read block 1092 len 40960
24982 164/300 Write block 799 len 20480 data: 6 5 5 5 5
24983 165/300 Write block 2030 len 4096 data: 6
24984 166/300 Write block 1690 len 8192 data: 4 3
24985 167/300 Write block 2575 len 16384 data: 4 3 3 2
24986 168/300 Read block 1608 len 36864
24987 169/300 Read block 1562 len 36864
24988 170/300 Write block 2250 len 16384 data: 5 5 6 8
24989 171/300 Write block 3764 len 20480 data: 5 4 5 6 6
24990 172/300 Read block 2556 len 32768
24991 173/300 Write block 1289 len 36864 data: 4 5 7 6 6 5 4 4 3
24992 174/300 Read block 1577 len 8192
24993 175/300 Read block 4993 len 16384
24994 176/300 Write block 3539 len 40960 data: 6 6 5 6 7 8 8 8 7 7
24995 177/300 Write block 1103 len 20480 data: 4 3 3 5 5
24996 178/300 Write block 2450 len 12288 data: 3 4 4
24997 179/300 Flush
24998 180/300 Write block 1471 len 16384 data: 4 4 4 3
24999 181/300 Read block 1594 len 40960
25000 182/300 Read block 1459 len 28672
25001 183/300 Write block 4947 len 24576 data: 7 8 7 6 4 5
25002 184/300 Read block 3255 len 16384
25003 185/300 Read block 130 len 28672
25004 186/300 Write block 2938 len 16384 data: 4 4 5 5
25005 187/300 Write block 1668 len 20480 data: 4 5 6 7 7
25006 188/300 Read block 1157 len 40960
25007 189/300 Read block 2543 len 12288
25008 190/300 Read block 2890 len 40960
25009 191/300 Write block 3216 len 8192 data: 4 3
25010 192/300 Read block 311 len 16384
25011 193/300 Read block 896 len 4096
25012 194/300 Read block 4062 len 20480
25013 195/300 Write block 1471 len 36864 data: 5 5 5 4 4 3 3 5 5
25014 196/300 Read block 4111 len 24576
25015 197/300 Write block 4432 len 8192 data: 7 7
25016 198/300 Read block 2319 len 36864
25017 199/300 Read block 4412 len 40960
25018 200/300 Write block 3600 len 36864 data: 7 7 6 6 7 9 6 7 6
25019 201/300 Read block 3828 len 16384
25020 202/300 Write block 1813 len 24576 data: 3 3 3 3 4 6
25021 203/300 Write block 1388 len 20480 data: 7 6 5 6 5
25022 204/300 Read block 4597 len 28672
25023 205/300 Write block 507 len 8192 data: 2 3
25024 206/300 Read block 4123 len 20480
25025 207/300 Write block 3495 len 36864 data: 5 4 4 4 3 2 2 2 2
25026 208/300 Write block 2848 len 28672 data: 3 4 3 3 3 4 5
25027 209/300 Write block 2175 len 40960 data: 4 4 2 2 2 2 2 3 3 4
25028 210/300 Write block 1461 len 4096 data: 3
25029 211/300 Read block 2790 len 32768
25030 212/300 Write block 2994 len 40960 data: 3 4 3 3 3 3 3 3 2 3
25031 213/300 Flush
25032 214/300 Flush
25033 215/300 Read block 1787 len 36864
25034 216/300 Read block 3066 len 24576
25035 217/300 Read block 4111 len 20480
25036 218/300 Write block 298 len 8192 data: 3 3
25037 219/300 Read block 3592 len 8192
25038 220/300 Write block 4253 len 4096 data: 8
25039 221/300 Read block 4509 len 4096
25040 222/300 Read block 3074 len 8192
25041 223/300 Write block 3937 len 40960 data: 4 5 5 5 4 3 3 4 3 5
25042 224/300 Read block 501 len 12288
25043 225/300 Write block 1124 len 32768 data: 5 3 3 3 4 4 6 4
25044 226/300 Read block 1445 len 4096
25045 227/300 Read block 2247 len 12288
25046 228/300 Write block 4943 len 4096 data: 8
25047 229/300 Read block 4695 len 4096
25048 230/300 Write block 4162 len 8192 data: 3 3
25049 231/300 Read block 1375 len 28672
25050 232/300 Write block 3440 len 40960 data: 6 5 7 7 5 4 5 4 4 4
25051 233/300 Read block 2858 len 12288
25052 234/300 Write block 3698 len 32768 data: 6 6 5 6 5 6 7 7
25053 235/300 Read block 318 len 4096
25054 236/300 Read block 1351 len 24576
25055 237/300 Read block 1148 len 28672
25056 238/300 Read block 4659 len 40960
25057 239/300 Read block 2430 len 16384
25058 240/300 Read block 1029 len 8192
25059 241/300 Write block 3984 len 24576 data: 4 5 5 5 5 5
25060 242/300 Read block 577 len 4096
25061 243/300 Read block 1205 len 24576
25062 244/300 Write block 4509 len 8192 data: 5 5
25063 245/300 Write block 65 len 4096 data: 6
25064 246/300 Read block 3494 len 24576
25065 247/300 Read block 1173 len 24576
25066 248/300 Write block 4809 len 28672 data: 2 3 3 5 5 5 6
25067 249/300 Read block 1798 len 4096
25068 250/300 Write block 1497 len 32768 data: 5 6 4 4 4 4 5 5
25069 251/300 Read block 1561 len 4096
25070 252/300 Read block 4799 len 40960
25071 253/300 Write block 629 len 20480 data: 6 6 7 5 5
25072 254/300 Read block 1076 len 8192
25073 255/300 Write block 2516 len 40960 data: 5 5 4 4 7 7 7 8 6 7
25074 256/300 Write block 3316 len 28672 data: 4 2 2 2 3 3 3
25075 257/300 Read block 4382 len 24576
25076 258/300 Write block 1426 len 40960 data: 7 6 5 4 4 2 2 3 4 4
25077 259/300 Read block 2221 len 24576
25078 260/300 Read block 699 len 32768
25079 261/300 Flush
25080 262/300 Read block 2449 len 16384
25081 263/300 Write block 2403 len 28672 data: 3 3 3 3 5 5 6
25082 264/300 Read block 1055 len 24576
25083 265/300 Write block 1481 len 8192 data: 4 4
25084 266/300 Read block 650 len 40960
25085 267/300 Read block 4663 len 20480
25086 268/300 Read block 755 len 32768
25087 269/300 Read block 119 len 32768
25088 270/300 Write block 2706 len 36864 data: 3 4 8 7 8 10 9 9 8
25089 271/300 Write block 1615 len 20480 data: 4 3 4 5 6
25090 272/300 Write block 1154 len 32768 data: 8 7 6 6 7 9 9 7
25091 273/300 Write block 4532 len 32768 data: 4 4 5 6 4 4 5 4
25092 274/300 Read block 2913 len 32768
25093 275/300 Read block 3565 len 32768
25094 276/300 Flush
25095 277/300 Write block 4747 len 36864 data: 4 6 6 3 3 3 5 6 6
25096 278/300 Write block 4445 len 16384 data: 5 6 6 8
25097 279/300 Flush
25098 280/300 Read block 4629 len 40960
25099 281/300 Read block 3113 len 24576
25100 282/300 Write block 228 len 12288 data: 5 6 6
25101 283/300 Write block 1180 len 16384 data: 5 6 6 5
25102 284/300 Flush
25103 285/300 Write block 2051 len 40960 data: 5 6 7 7 8 7 6 4 3 4
25104 286/300 Write block 583 len 32768 data: 5 5 5 3 6 4 4 4
25105 287/300 Write block 2095 len 8192 data: 6 4
25106 288/300 Read block 4539 len 4096
25107 289/300 Read block 4353 len 32768
25108 290/300 Write block 172 len 8192 data: 8 6
25109 291/300 Write block 2284 len 40960 data: 4 5 5 3 5 6 5 5 4 4
25110 292/300 Read block 4796 len 4096
25111 293/300 Read block 3118 len 36864
25112 294/300 Read block 1762 len 12288
25113 295/300 Read block 2227 len 16384
25114 296/300 Read block 4182 len 24576
25115 297/300 Read block 772 len 32768
25116 298/300 Write block 2449 len 28672 data: 3 4 5 5 6 8 8
25117 299/300 Read block 372 len 16384
25118 300/300 Write block 2784 len 32768 data: 3 3 3 3 3 3 3 4
25119 301/300 Flush
25120 Main task has work to do, go find it
25121 got dsc Start(0)
25122 start 0
25123 Jan 03 08:36:11.106 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/0, method: POST, req_id: 46fa05b2-ee47-43ce-a778-916b516b2537, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
25124 Replay: started 0, returned:Ok(())
25125 ----------------------------------------------------------------
25126 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25127 States: Offline Active Active
25128 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25129 7341 Acked Flush 0 Sent Done Done false
25130 7342 Acked Write 8 Sent Done Done false
25131 7343 Acked Read 5 Sent Done Done false
25132 7344 Acked Write 10 Sent Done Done false
25133 7345 Acked Read 4 Sent Done Done false
25134 7346 Acked Read 2 Sent Done Done false
25135 7347 Acked Write 1 Sent Done Done false
25136 7348 Acked Read 5 Sent Done Done false
25137 7349 Acked Read 10 Sent Done Done false
25138 7350 Acked Read 7 Sent Done Done false
25139 7351 Acked Read 8 Sent Done Done false
25140 7352 Acked Write 7 Sent Done Done false
25141 7353 Acked Read 10 Sent Done Done false
25142 7354 Acked Read 6 Sent Done Done false
25143 7355 Acked Flush 0 Sent Done Done false
25144 7356 Acked Write 3 Sent Done Done false
25145 7357 Acked Read 10 Sent Done Done false
25146 7358 Acked Write 10 Sent Done Done false
25147 7359 Acked Flush 0 Sent Done Done false
25148 7360 Acked Read 10 Sent Done Done false
25149 7361 Acked Write 10 Sent Done Done false
25150 7362 Acked Write 9 Sent Done Done false
25151 7363 Acked Read 7 Sent Done Done false
25152 7364 Acked Write 6 Sent Done Done false
25153 7365 Acked Read 6 Sent Done Done false
25154 7366 Acked Write 9 Sent Done Done false
25155 7367 Acked Write 7 Sent Done Done false
25156 7368 Acked Read 8 Sent Done Done false
25157 7369 Acked Read 10 Sent Done Done false
25158 7370 Acked Read 2 Sent Done Done false
25159 7371 Acked Read 9 Sent Done Done false
25160 7372 Acked Write 5 Sent Done Done false
25161 7373 Acked Read 5 Sent Done Done false
25162 7374 Acked Write 8 Sent Done Done false
25163 7375 Acked Write 9 Sent Done Done false
25164 7376 Acked Flush 0 Sent Done Done false
25165 7377 Acked Read 6 Sent Done Done false
25166 7378 Acked Read 7 Sent Done Done false
25167 7379 Acked Write 6 Sent Done Done false
25168 7380 Acked Write 2 Sent Done Done false
25169 7381 Acked Read 2 Sent Done Done false
25170 7382 Acked Write 3 Sent Done Done false
25171 7383 Acked Flush 0 Sent Done Done false
25172 7384 Acked Write 9 Sent Done Done false
25173 7385 Acked Write 5 Sent Done Done false
25174 7386 Acked Read 3 Sent Done Done false
25175 7387 Acked Write 1 Sent Done Done false
25176 7388 Acked Read 1 Sent Done Done false
25177 7389 Acked Write 3 Sent Done Done false
25178 7390 Acked Read 8 Sent Done Done false
25179 7391 Acked Write 1 Sent Done Done false
25180 7392 Acked Write 2 Sent Done Done false
25181 7393 Acked Read 9 Sent Done Done false
25182 7394 Acked Write 6 Sent Done Done false
25183 7395 Acked Write 3 Sent Done Done false
25184 7396 Acked Write 7 Sent Done Done false
25185 7397 Acked Write 1 Sent Done Done false
25186 7398 Acked Read 4 Sent Done Done false
25187 7399 Acked Flush 0 Sent Done Done false
25188 7400 Acked Write 2 Sent Done Done false
25189 7401 Acked Read 4 Sent Done Done false
25190 7402 Acked Read 5 Sent Done Done false
25191 7403 Acked Write 6 Sent Done Done false
25192 7404 Acked Flush 0 Sent Done Done false
25193 7405 Acked Write 10 Sent Done Done false
25194 7406 Acked Read 5 Sent Done Done false
25195 7407 Acked Read 6 Sent Done Done false
25196 7408 Acked Read 6 Sent Done Done false
25197 7409 Acked Read 1 Sent Done Done false
25198 7410 Acked Read 5 Sent Done Done false
25199 7411 Acked Write 2 Sent Done Done false
25200 7412 Acked Write 5 Sent Done Done false
25201 7413 Acked Write 9 Sent Done Done false
25202 7414 Acked Write 8 Sent Done Done false
25203 7415 Acked Write 2 Sent Done Done false
25204 7416 Acked Read 2 Sent Done Done false
25205 7417 Acked Write 5 Sent Done Done false
25206 7418 Acked Read 10 Sent Done Done false
25207 7419 Acked Write 3 Sent Done Done false
25208 7420 Acked Read 8 Sent Done Done false
25209 7421 Acked Write 1 Sent Done Done false
25210 7422 Acked Write 4 Sent Done Done false
25211 7423 Acked Read 6 Sent Done Done false
25212 7424 Acked Read 10 Sent Done Done false
25213 7425 Acked Write 2 Sent Done Done false
25214 7426 Acked Read 7 Sent Done Done false
25215 7427 Acked Read 3 Sent Done Done false
25216 7428 Acked Write 4 Sent Done Done false
25217 7429 Acked Write 5 Sent Done Done false
25218 7430 Acked Read 1 Sent Done Done false
25219 7431 Acked Read 4 Sent Done Done false
25220 7432 Acked Read 9 Sent Done Done false
25221 7433 Acked Write 7 Sent Done Done false
25222 7434 Acked Write 4 Sent Done Done false
25223 7435 Acked Flush 0 Sent Done Done false
25224 7436 Acked Write 4 Sent Done Done false
25225 7437 Acked Read 6 Sent Done Done false
25226 7438 Acked Write 2 Sent Done Done false
25227 7439 Acked Write 5 Sent Done Done false
25228 7440 Acked Read 9 Sent Done Done false
25229 7441 Acked Read 1 Sent Done Done false
25230 7442 Acked Write 10 Sent Done Done false
25231 7443 Acked Read 4 Sent Done Done false
25232 7444 Acked Write 2 Sent Done Done false
25233 7445 Acked Flush 0 Sent Done Done false
25234 7446 Acked Flush 0 Sent Done Done false
25235 7447 Acked Read 10 Sent Done Done false
25236 7448 Acked Write 7 Sent Done Done false
25237 7449 Acked Write 1 Sent Done Done false
25238 7450 Acked Read 5 Sent Done Done false
25239 7451 Acked Read 6 Sent Done Done false
25240 7452 Acked Write 9 Sent Done Done false
25241 7453 Acked Write 10 Sent Done Done false
25242 7454 Acked Write 6 Sent Done Done false
25243 7455 Acked Read 7 Sent Done Done false
25244 7456 Acked Read 9 Sent Done Done false
25245 7457 Acked Read 9 Sent Done Done false
25246 7458 Acked Write 6 Sent Done Done false
25247 7459 Acked Write 9 Sent Done Done false
25248 7460 Acked Write 5 Sent Done Done false
25249 7461 Acked Write 1 Sent Done Done false
25250 7462 Acked Write 8 Sent Done Done false
25251 7463 Acked Write 6 Sent Done Done false
25252 7464 Acked Read 3 Sent Done Done false
25253 7465 Acked Write 8 Sent Done Done false
25254 7466 Acked Write 6 Sent Done Done false
25255 7467 Acked Read 8 Sent Done Done false
25256 7468 Acked Read 7 Sent Done Done false
25257 7469 Acked Read 1 Sent Done Done false
25258 7470 Acked Read 10 Sent Done Done false
25259 7471 Acked Read 3 Sent Done Done false
25260 7472 Acked Read 9 Sent Done Done false
25261 7473 Acked Write 4 Sent Done Done false
25262 7474 Acked Write 2 Sent Done Done false
25263 7475 Acked Read 5 Sent Done Done false
25264 7476 Acked Read 4 Sent Done Done false
25265 7477 Acked Read 9 Sent Done Done false
25266 7478 Acked Flush 0 Sent Done Done false
25267 7479 Acked Read 6 Sent Done Done false
25268 7480 Acked Write 9 Sent Done Done false
25269 7481 Acked Read 7 Sent Done Done false
25270 7482 Acked Write 1 Sent Done Done false
25271 7483 Acked Read 7 Sent Done Done false
25272 7484 Acked Write 4 Sent Done Done false
25273 7485 Acked Read 5 Sent Done Done false
25274 7486 Acked Read 2 Sent Done Done false
25275 7487 Acked Read 2 Sent Done Done false
25276 7488 Acked Write 4 Sent Done Done false
25277 7489 Acked Read 5 Sent Done Done false
25278 7490 Acked Read 5 Sent Done Done false
25279 7491 Acked Flush 0 Sent Done Done false
25280 7492 Acked Write 3 Sent Done Done false
25281 7493 Acked Read 5 Sent Done Done false
25282 7494 Acked Read 9 Sent Done Done false
25283 7495 Acked Read 9 Sent Done Done false
25284 7496 Acked Write 2 Sent Done Done false
25285 7497 Acked Write 4 Sent Done Done false
25286 7498 Acked Write 4 Sent Done Done false
25287 7499 Acked Read 8 Sent Done Done false
25288 7500 Acked Read 9 Sent Done Done false
25289 7501 Acked Write 5 Sent Done Done false
25290 7502 Acked Read 5 Sent Done Done false
25291 7503 Acked Read 10 Sent Done Done false
25292 7504 Acked Write 5 Sent Done Done false
25293 7505 Acked Write 1 Sent Done Done false
25294 7506 Acked Write 2 Sent Done Done false
25295 7507 Acked Write 4 Sent Done Done false
25296 7508 Acked Read 9 Sent Done Done false
25297 7509 Acked Read 9 Sent Done Done false
25298 7510 Acked Write 4 Sent Done Done false
25299 7511 Acked Write 5 Sent Done Done false
25300 7512 Acked Read 8 Sent Done Done false
25301 7513 Acked Write 9 Sent Done Done false
25302 7514 Acked Read 2 Sent Done Done false
25303 7515 Acked Read 4 Sent Done Done false
25304 7516 Acked Write 10 Sent Done Done false
25305 7517 Acked Write 5 Sent Done Done false
25306 7518 Acked Write 3 Sent Done Done false
25307 7519 Acked Flush 0 Sent Done Done false
25308 7520 Acked Write 4 Sent Done Done false
25309 7521 Acked Read 10 Sent Done Done false
25310 7522 Acked Read 7 Sent Done Done false
25311 7523 Acked Write 6 Sent Done Done false
25312 7524 Acked Read 4 Sent Done Done false
25313 7525 Acked Read 7 Sent Done Done false
25314 7526 Acked Write 4 Sent Done Done false
25315 7527 Acked Write 5 Sent Done Done false
25316 7528 Acked Read 10 Sent Done Done false
25317 7529 Acked Read 3 Sent Done Done false
25318 7530 Acked Read 10 Sent Done Done false
25319 7531 Acked Write 2 Sent Done Done false
25320 7532 Acked Read 4 Sent Done Done false
25321 7533 Acked Read 1 Sent Done Done false
25322 7534 Acked Read 5 Sent Done Done false
25323 7535 Acked Write 9 Sent Done Done false
25324 7536 Acked Read 6 Sent Done Done false
25325 7537 Acked Write 2 Sent Done Done false
25326 7538 Acked Read 9 Sent Done Done false
25327 7539 Acked Read 10 Sent Done Done false
25328 7540 Acked Write 9 Sent Done Done false
25329 7541 Acked Read 4 Sent Done Done false
25330 7542 Acked Write 6 Sent Done Done false
25331 7543 Acked Write 5 Sent Done Done false
25332 7544 Acked Read 7 Sent Done Done false
25333 7545 Acked Write 2 Sent Done Done false
25334 7546 Acked Read 5 Sent Done Done false
25335 7547 Acked Write 9 Sent Done Done false
25336 7548 Acked Write 7 Sent Done Done false
25337 7549 Acked Write 10 Sent Done Done false
25338 7550 Acked Write 1 Sent Done Done false
25339 7551 Acked Read 8 Sent Done Done false
25340 7552 Acked Write 10 Sent Done Done false
25341 7553 Acked Flush 0 Sent Done Done false
25342 7554 Acked Flush 0 Sent Done Done false
25343 7555 Acked Read 9 Sent Done Done false
25344 7556 Acked Read 6 Sent Done Done false
25345 7557 Acked Read 5 Sent Done Done false
25346 7558 Acked Write 2 Sent Done Done false
25347 7559 Acked Read 2 Sent Done Done false
25348 7560 Acked Write 1 Sent Done Done false
25349 7561 Acked Read 1 Sent Done Done false
25350 7562 Acked Read 2 Sent Done Done false
25351 7563 Acked Write 10 Sent Done Done false
25352 7564 Acked Read 3 Sent Done Done false
25353 7565 Acked Write 8 Sent Done Done false
25354 7566 Acked Read 1 Sent Done Done false
25355 7567 Acked Read 3 Sent Done Done false
25356 7568 Acked Write 1 Sent Done Done false
25357 7569 Acked Read 1 Sent Done Done false
25358 7570 Acked Write 2 Sent Done Done false
25359 7571 Acked Read 7 Sent Done Done false
25360 7572 Acked Write 10 Sent Done Done false
25361 7573 Acked Read 3 Sent Done Done false
25362 7574 Acked Write 8 Sent Done Done false
25363 7575 Acked Read 1 Sent Done Done false
25364 7576 Acked Read 6 Sent Done Done false
25365 7577 Acked Read 7 Sent Done Done false
25366 7578 Acked Read 10 Sent Done Done false
25367 7579 Acked Read 4 Sent Done Done false
25368 7580 Acked Read 2 Sent Done Done false
25369 7581 Acked Write 6 Sent Done Done false
25370 7582 Acked Read 1 Sent Done Done false
25371 7583 Acked Read 6 Sent Done Done false
25372 7584 Acked Write 2 Sent Done Done false
25373 7585 Acked Write 1 Sent Done Done false
25374 7586 Acked Read 6 Sent Done Done false
25375 7587 Acked Read 6 Sent Done Done false
25376 7588 Acked Write 7 Sent Done Done false
25377 7589 Acked Read 1 Sent Done Done false
25378 7590 Acked Write 8 Sent Done Done false
25379 7591 Acked Read 1 Sent Done Done false
25380 7592 Acked Read 10 Sent Done Done false
25381 7593 Acked Write 5 Sent Done Done false
25382 7594 Acked Read 2 Sent Done Done false
25383 7595 Acked Write 10 Sent Done Done false
25384 7596 Acked Write 7 Sent Done Done false
25385 7597 Acked Read 6 Sent Done Done false
25386 7598 Acked Write 10 Sent Done Done false
25387 7599 Acked Read 6 Sent Done Done false
25388 7600 Acked Read 8 Sent Done Done false
25389 7601 Acked Flush 0 Sent Done Done false
25390 7602 Acked Read 4 Sent Done Done false
25391 7603 Acked Write 7 Sent Done Done false
25392 7604 Acked Read 6 Sent Done Done false
25393 7605 Acked Write 2 Sent Done Done false
25394 7606 Acked Read 10 Sent Done Done false
25395 7607 Acked Read 5 Sent Done Done false
25396 7608 Acked Read 8 Sent Done Done false
25397 7609 Acked Read 8 Sent Done Done false
25398 7610 Acked Write 9 Sent Done Done false
25399 7611 Acked Write 5 Sent Done Done false
25400 7612 Acked Write 8 Sent Done Done false
25401 7613 Acked Write 8 Sent Done Done false
25402 7614 Acked Read 8 Sent Done Done false
25403 7615 Acked Read 8 Sent Done Done false
25404 7616 Acked Flush 0 Sent Done Done false
25405 7617 Acked Write 9 Sent Done Done false
25406 7618 Acked Write 4 Sent Done Done false
25407 7619 Acked Flush 0 Sent Done Done false
25408 7620 Acked Read 10 Sent Done Done false
25409 7621 Acked Read 6 Sent Done Done false
25410 7622 Acked Write 3 Sent Done Done false
25411 7623 Acked Write 4 Sent Done Done false
25412 7624 Acked Flush 0 Sent Done Done false
25413 7625 Acked Write 10 Sent Done Done false
25414 7626 Acked Write 8 Sent Done Done false
25415 7627 Acked Write 2 Sent Done Done false
25416 7628 Acked Read 1 Sent Done Done false
25417 7629 Acked Read 8 Sent Done Done false
25418 7630 Acked Write 2 Sent Done Done false
25419 7631 Acked Write 10 Sent Done Done false
25420 7632 Acked Read 1 Sent Done Done false
25421 7633 Acked Read 9 Sent Done Done false
25422 7634 Acked Read 3 Sent Done Done false
25423 7635 Acked Read 4 Sent Done Done false
25424 7636 Acked Read 6 Sent Done Done false
25425 7637 Acked Read 8 Sent Done Done false
25426 7638 Acked Write 7 Sent Done Done false
25427 7639 Acked Read 4 Sent Done Done false
25428 7640 Acked Write 8 Sent Done Done false
25429 7641 Acked Flush 0 Sent Done Done false
25430 STATES DS:0 DS:1 DS:2 TOTAL
25431 Sent 301 0 0 301
25432 Done 0 301 301 602
25433 Skipped 0 0 0 0
25434 Error 0 0 0 0
25435 Last Flush: 7340 7641 7641
25436 Downstairs last five completed: 7340 7339 7338 7337 7336
25437 Upstairs last five completed: 7641 7640 7639 7638 7637
25438 CLIENT: Up:0 ds:301 act:2
25439 [8810] got start action
25440 [8810] I am going to restart
25441 Starting downstairs at port 8810
25442 Make output file at "/tmp/dsc/downstairs-8810.txt"
25443 [8810][0] reports Starting
25444 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1227)
25445 [8810][0] reports Running
25446 ----------------------------------------------------------------
25447 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25448 States: Offline Active Active
25449 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25450 7341 Acked Flush 0 Sent Done Done false
25451 7342 Acked Write 8 Sent Done Done false
25452 7343 Acked Read 5 Sent Done Done false
25453 7344 Acked Write 10 Sent Done Done false
25454 7345 Acked Read 4 Sent Done Done false
25455 7346 Acked Read 2 Sent Done Done false
25456 7347 Acked Write 1 Sent Done Done false
25457 7348 Acked Read 5 Sent Done Done false
25458 7349 Acked Read 10 Sent Done Done false
25459 7350 Acked Read 7 Sent Done Done false
25460 7351 Acked Read 8 Sent Done Done false
25461 7352 Acked Write 7 Sent Done Done false
25462 7353 Acked Read 10 Sent Done Done false
25463 7354 Acked Read 6 Sent Done Done false
25464 7355 Acked Flush 0 Sent Done Done false
25465 7356 Acked Write 3 Sent Done Done false
25466 7357 Acked Read 10 Sent Done Done false
25467 7358 Acked Write 10 Sent Done Done false
25468 7359 Acked Flush 0 Sent Done Done false
25469 7360 Acked Read 10 Sent Done Done false
25470 7361 Acked Write 10 Sent Done Done false
25471 7362 Acked Write 9 Sent Done Done false
25472 7363 Acked Read 7 Sent Done Done false
25473 7364 Acked Write 6 Sent Done Done false
25474 7365 Acked Read 6 Sent Done Done false
25475 7366 Acked Write 9 Sent Done Done false
25476 7367 Acked Write 7 Sent Done Done false
25477 7368 Acked Read 8 Sent Done Done false
25478 7369 Acked Read 10 Sent Done Done false
25479 7370 Acked Read 2 Sent Done Done false
25480 7371 Acked Read 9 Sent Done Done false
25481 7372 Acked Write 5 Sent Done Done false
25482 7373 Acked Read 5 Sent Done Done false
25483 7374 Acked Write 8 Sent Done Done false
25484 7375 Acked Write 9 Sent Done Done false
25485 7376 Acked Flush 0 Sent Done Done false
25486 7377 Acked Read 6 Sent Done Done false
25487 7378 Acked Read 7 Sent Done Done false
25488 7379 Acked Write 6 Sent Done Done false
25489 7380 Acked Write 2 Sent Done Done false
25490 7381 Acked Read 2 Sent Done Done false
25491 7382 Acked Write 3 Sent Done Done false
25492 7383 Acked Flush 0 Sent Done Done false
25493 7384 Acked Write 9 Sent Done Done false
25494 7385 Acked Write 5 Sent Done Done false
25495 7386 Acked Read 3 Sent Done Done false
25496 7387 Acked Write 1 Sent Done Done false
25497 7388 Acked Read 1 Sent Done Done false
25498 7389 Acked Write 3 Sent Done Done false
25499 7390 Acked Read 8 Sent Done Done false
25500 7391 Acked Write 1 Sent Done Done false
25501 7392 Acked Write 2 Sent Done Done false
25502 7393 Acked Read 9 Sent Done Done false
25503 7394 Acked Write 6 Sent Done Done false
25504 7395 Acked Write 3 Sent Done Done false
25505 7396 Acked Write 7 Sent Done Done false
25506 7397 Acked Write 1 Sent Done Done false
25507 7398 Acked Read 4 Sent Done Done false
25508 7399 Acked Flush 0 Sent Done Done false
25509 7400 Acked Write 2 Sent Done Done false
25510 7401 Acked Read 4 Sent Done Done false
25511 7402 Acked Read 5 Sent Done Done false
25512 7403 Acked Write 6 Sent Done Done false
25513 7404 Acked Flush 0 Sent Done Done false
25514 7405 Acked Write 10 Sent Done Done false
25515 7406 Acked Read 5 Sent Done Done false
25516 7407 Acked Read 6 Sent Done Done false
25517 7408 Acked Read 6 Sent Done Done false
25518 7409 Acked Read 1 Sent Done Done false
25519 7410 Acked Read 5 Sent Done Done false
25520 7411 Acked Write 2 Sent Done Done false
25521 7412 Acked Write 5 Sent Done Done false
25522 7413 Acked Write 9 Sent Done Done false
25523 7414 Acked Write 8 Sent Done Done false
25524 7415 Acked Write 2 Sent Done Done false
25525 7416 Acked Read 2 Sent Done Done false
25526 7417 Acked Write 5 Sent Done Done false
25527 7418 Acked Read 10 Sent Done Done false
25528 7419 Acked Write 3 Sent Done Done false
25529 7420 Acked Read 8 Sent Done Done false
25530 7421 Acked Write 1 Sent Done Done false
25531 7422 Acked Write 4 Sent Done Done false
25532 7423 Acked Read 6 Sent Done Done false
25533 7424 Acked Read 10 Sent Done Done false
25534 7425 Acked Write 2 Sent Done Done false
25535 7426 Acked Read 7 Sent Done Done false
25536 7427 Acked Read 3 Sent Done Done false
25537 7428 Acked Write 4 Sent Done Done false
25538 7429 Acked Write 5 Sent Done Done false
25539 7430 Acked Read 1 Sent Done Done false
25540 7431 Acked Read 4 Sent Done Done false
25541 7432 Acked Read 9 Sent Done Done false
25542 7433 Acked Write 7 Sent Done Done false
25543 7434 Acked Write 4 Sent Done Done false
25544 7435 Acked Flush 0 Sent Done Done false
25545 7436 Acked Write 4 Sent Done Done false
25546 7437 Acked Read 6 Sent Done Done false
25547 7438 Acked Write 2 Sent Done Done false
25548 7439 Acked Write 5 Sent Done Done false
25549 7440 Acked Read 9 Sent Done Done false
25550 7441 Acked Read 1 Sent Done Done false
25551 7442 Acked Write 10 Sent Done Done false
25552 7443 Acked Read 4 Sent Done Done false
25553 7444 Acked Write 2 Sent Done Done false
25554 7445 Acked Flush 0 Sent Done Done false
25555 7446 Acked Flush 0 Sent Done Done false
25556 7447 Acked Read 10 Sent Done Done false
25557 7448 Acked Write 7 Sent Done Done false
25558 7449 Acked Write 1 Sent Done Done false
25559 7450 Acked Read 5 Sent Done Done false
25560 7451 Acked Read 6 Sent Done Done false
25561 7452 Acked Write 9 Sent Done Done false
25562 7453 Acked Write 10 Sent Done Done false
25563 7454 Acked Write 6 Sent Done Done false
25564 7455 Acked Read 7 Sent Done Done false
25565 7456 Acked Read 9 Sent Done Done false
25566 7457 Acked Read 9 Sent Done Done false
25567 7458 Acked Write 6 Sent Done Done false
25568 7459 Acked Write 9 Sent Done Done false
25569 7460 Acked Write 5 Sent Done Done false
25570 7461 Acked Write 1 Sent Done Done false
25571 7462 Acked Write 8 Sent Done Done false
25572 7463 Acked Write 6 Sent Done Done false
25573 7464 Acked Read 3 Sent Done Done false
25574 7465 Acked Write 8 Sent Done Done false
25575 7466 Acked Write 6 Sent Done Done false
25576 7467 Acked Read 8 Sent Done Done false
25577 7468 Acked Read 7 Sent Done Done false
25578 7469 Acked Read 1 Sent Done Done false
25579 7470 Acked Read 10 Sent Done Done false
25580 7471 Acked Read 3 Sent Done Done false
25581 7472 Acked Read 9 Sent Done Done false
25582 7473 Acked Write 4 Sent Done Done false
25583 7474 Acked Write 2 Sent Done Done false
25584 7475 Acked Read 5 Sent Done Done false
25585 7476 Acked Read 4 Sent Done Done false
25586 7477 Acked Read 9 Sent Done Done false
25587 7478 Acked Flush 0 Sent Done Done false
25588 7479 Acked Read 6 Sent Done Done false
25589 7480 Acked Write 9 Sent Done Done false
25590 7481 Acked Read 7 Sent Done Done false
25591 7482 Acked Write 1 Sent Done Done false
25592 7483 Acked Read 7 Sent Done Done false
25593 7484 Acked Write 4 Sent Done Done false
25594 7485 Acked Read 5 Sent Done Done false
25595 7486 Acked Read 2 Sent Done Done false
25596 7487 Acked Read 2 Sent Done Done false
25597 7488 Acked Write 4 Sent Done Done false
25598 7489 Acked Read 5 Sent Done Done false
25599 7490 Acked Read 5 Sent Done Done false
25600 7491 Acked Flush 0 Sent Done Done false
25601 7492 Acked Write 3 Sent Done Done false
25602 7493 Acked Read 5 Sent Done Done false
25603 7494 Acked Read 9 Sent Done Done false
25604 7495 Acked Read 9 Sent Done Done false
25605 7496 Acked Write 2 Sent Done Done false
25606 7497 Acked Write 4 Sent Done Done false
25607 7498 Acked Write 4 Sent Done Done false
25608 7499 Acked Read 8 Sent Done Done false
25609 7500 Acked Read 9 Sent Done Done false
25610 7501 Acked Write 5 Sent Done Done false
25611 7502 Acked Read 5 Sent Done Done false
25612 7503 Acked Read 10 Sent Done Done false
25613 7504 Acked Write 5 Sent Done Done false
25614 7505 Acked Write 1 Sent Done Done false
25615 7506 Acked Write 2 Sent Done Done false
25616 7507 Acked Write 4 Sent Done Done false
25617 7508 Acked Read 9 Sent Done Done false
25618 7509 Acked Read 9 Sent Done Done false
25619 7510 Acked Write 4 Sent Done Done false
25620 7511 Acked Write 5 Sent Done Done false
25621 7512 Acked Read 8 Sent Done Done false
25622 7513 Acked Write 9 Sent Done Done false
25623 7514 Acked Read 2 Sent Done Done false
25624 7515 Acked Read 4 Sent Done Done false
25625 7516 Acked Write 10 Sent Done Done false
25626 7517 Acked Write 5 Sent Done Done false
25627 7518 Acked Write 3 Sent Done Done false
25628 7519 Acked Flush 0 Sent Done Done false
25629 7520 Acked Write 4 Sent Done Done false
25630 7521 Acked Read 10 Sent Done Done false
25631 7522 Acked Read 7 Sent Done Done false
25632 7523 Acked Write 6 Sent Done Done false
25633 7524 Acked Read 4 Sent Done Done false
25634 7525 Acked Read 7 Sent Done Done false
25635 7526 Acked Write 4 Sent Done Done false
25636 7527 Acked Write 5 Sent Done Done false
25637 7528 Acked Read 10 Sent Done Done false
25638 7529 Acked Read 3 Sent Done Done false
25639 7530 Acked Read 10 Sent Done Done false
25640 7531 Acked Write 2 Sent Done Done false
25641 7532 Acked Read 4 Sent Done Done false
25642 7533 Acked Read 1 Sent Done Done false
25643 7534 Acked Read 5 Sent Done Done false
25644 7535 Acked Write 9 Sent Done Done false
25645 7536 Acked Read 6 Sent Done Done false
25646 7537 Acked Write 2 Sent Done Done false
25647 7538 Acked Read 9 Sent Done Done false
25648 7539 Acked Read 10 Sent Done Done false
25649 7540 Acked Write 9 Sent Done Done false
25650 7541 Acked Read 4 Sent Done Done false
25651 7542 Acked Write 6 Sent Done Done false
25652 7543 Acked Write 5 Sent Done Done false
25653 7544 Acked Read 7 Sent Done Done false
25654 7545 Acked Write 2 Sent Done Done false
25655 7546 Acked Read 5 Sent Done Done false
25656 7547 Acked Write 9 Sent Done Done false
25657 7548 Acked Write 7 Sent Done Done false
25658 7549 Acked Write 10 Sent Done Done false
25659 7550 Acked Write 1 Sent Done Done false
25660 7551 Acked Read 8 Sent Done Done false
25661 7552 Acked Write 10 Sent Done Done false
25662 7553 Acked Flush 0 Sent Done Done false
25663 7554 Acked Flush 0 Sent Done Done false
25664 7555 Acked Read 9 Sent Done Done false
25665 7556 Acked Read 6 Sent Done Done false
25666 7557 Acked Read 5 Sent Done Done false
25667 7558 Acked Write 2 Sent Done Done false
25668 7559 Acked Read 2 Sent Done Done false
25669 7560 Acked Write 1 Sent Done Done false
25670 7561 Acked Read 1 Sent Done Done false
25671 7562 Acked Read 2 Sent Done Done false
25672 7563 Acked Write 10 Sent Done Done false
25673 7564 Acked Read 3 Sent Done Done false
25674 7565 Acked Write 8 Sent Done Done false
25675 7566 Acked Read 1 Sent Done Done false
25676 7567 Acked Read 3 Sent Done Done false
25677 7568 Acked Write 1 Sent Done Done false
25678 7569 Acked Read 1 Sent Done Done false
25679 7570 Acked Write 2 Sent Done Done false
25680 7571 Acked Read 7 Sent Done Done false
25681 7572 Acked Write 10 Sent Done Done false
25682 7573 Acked Read 3 Sent Done Done false
25683 7574 Acked Write 8 Sent Done Done false
25684 7575 Acked Read 1 Sent Done Done false
25685 7576 Acked Read 6 Sent Done Done false
25686 7577 Acked Read 7 Sent Done Done false
25687 7578 Acked Read 10 Sent Done Done false
25688 7579 Acked Read 4 Sent Done Done false
25689 7580 Acked Read 2 Sent Done Done false
25690 7581 Acked Write 6 Sent Done Done false
25691 7582 Acked Read 1 Sent Done Done false
25692 7583 Acked Read 6 Sent Done Done false
25693 7584 Acked Write 2 Sent Done Done false
25694 7585 Acked Write 1 Sent Done Done false
25695 7586 Acked Read 6 Sent Done Done false
25696 7587 Acked Read 6 Sent Done Done false
25697 7588 Acked Write 7 Sent Done Done false
25698 7589 Acked Read 1 Sent Done Done false
25699 7590 Acked Write 8 Sent Done Done false
25700 7591 Acked Read 1 Sent Done Done false
25701 7592 Acked Read 10 Sent Done Done false
25702 7593 Acked Write 5 Sent Done Done false
25703 7594 Acked Read 2 Sent Done Done false
25704 7595 Acked Write 10 Sent Done Done false
25705 7596 Acked Write 7 Sent Done Done false
25706 7597 Acked Read 6 Sent Done Done false
25707 7598 Acked Write 10 Sent Done Done false
25708 7599 Acked Read 6 Sent Done Done false
25709 7600 Acked Read 8 Sent Done Done false
25710 7601 Acked Flush 0 Sent Done Done false
25711 7602 Acked Read 4 Sent Done Done false
25712 7603 Acked Write 7 Sent Done Done false
25713 7604 Acked Read 6 Sent Done Done false
25714 7605 Acked Write 2 Sent Done Done false
25715 7606 Acked Read 10 Sent Done Done false
25716 7607 Acked Read 5 Sent Done Done false
25717 7608 Acked Read 8 Sent Done Done false
25718 7609 Acked Read 8 Sent Done Done false
25719 7610 Acked Write 9 Sent Done Done false
25720 7611 Acked Write 5 Sent Done Done false
25721 7612 Acked Write 8 Sent Done Done false
25722 7613 Acked Write 8 Sent Done Done false
25723 7614 Acked Read 8 Sent Done Done false
25724 7615 Acked Read 8 Sent Done Done false
25725 7616 Acked Flush 0 Sent Done Done false
25726 7617 Acked Write 9 Sent Done Done false
25727 7618 Acked Write 4 Sent Done Done false
25728 7619 Acked Flush 0 Sent Done Done false
25729 7620 Acked Read 10 Sent Done Done false
25730 7621 Acked Read 6 Sent Done Done false
25731 7622 Acked Write 3 Sent Done Done false
25732 7623 Acked Write 4 Sent Done Done false
25733 7624 Acked Flush 0 Sent Done Done false
25734 7625 Acked Write 10 Sent Done Done false
25735 7626 Acked Write 8 Sent Done Done false
25736 7627 Acked Write 2 Sent Done Done false
25737 7628 Acked Read 1 Sent Done Done false
25738 7629 Acked Read 8 Sent Done Done false
25739 7630 Acked Write 2 Sent Done Done false
25740 7631 Acked Write 10 Sent Done Done false
25741 7632 Acked Read 1 Sent Done Done false
25742 7633 Acked Read 9 Sent Done Done false
25743 7634 Acked Read 3 Sent Done Done false
25744 7635 Acked Read 4 Sent Done Done false
25745 7636 Acked Read 6 Sent Done Done false
25746 7637 Acked Read 8 Sent Done Done false
25747 7638 Acked Write 7 Sent Done Done false
25748 7639 Acked Read 4 Sent Done Done false
25749 7640 Acked Write 8 Sent Done Done false
25750 7641 Acked Flush 0 Sent Done Done false
25751 STATES DS:0 DS:1 DS:2 TOTAL
25752 Sent 301 0 0 301
25753 Done 0 301 301 602
25754 Skipped 0 0 0 0
25755 Error 0 0 0 0
25756 Last Flush: 7340 7641 7641
25757 Downstairs last five completed: 7340 7339 7338 7337 7336
25758 Upstairs last five completed: 7641 7640 7639 7638 7637
25759 CLIENT: Up:0 ds:301 act:2
257602025-01-03T08:36:16.857ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257612025-01-03T08:36:16.857ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56471) = io task client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257622025-01-03T08:36:16.858ZWARNcrucible: version negotiation from state Offline = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257632025-01-03T08:36:16.859ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257642025-01-03T08:36:16.859ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257652025-01-03T08:36:16.859ZINFOcrucible: send last flush ID to this DS: 7340 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257662025-01-03T08:36:16.860ZINFOcrucible: Replied this last flush ID: 7340 = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257672025-01-03T08:36:16.860ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257682025-01-03T08:36:16.860ZINFOcrucible: [0] Transition from Offline to Active = downstairs client = 0 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257692025-01-03T08:36:16.860ZINFOcrucible: [0] client re-new 301 jobs since flush 7340 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257702025-01-03T08:36:16.860ZINFOcrucible: [0] Replayed 301 jobs since flush: 7340 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
25771 ----------------------------------------------------------------
25772 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25773 Downstairs last five completed: 7641 7640 7639 7638 7637
25774 Upstairs last five completed: 7641 7640 7639 7638 7637
25775 CLIENT: Up:0 ds:0 act:3
25776 Replay: All jobs finished
25777 Main task has work to do, go find it
25778 got dsc Stop(1)
25779 stop 1
25780 Jan 03 08:36:19.112[8820] Got stop action so:false kr:false
25781 INFO request completed, latency_us: 362, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 619dcbe1-9fb3-455a-b22e-0f9fe6dc08b3, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
257822025-01-03T08:36:19.113ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257832025-01-03T08:36:19.113ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257842025-01-03T08:36:19.113ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257852025-01-03T08:36:19.113ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257862025-01-03T08:36:19.113ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
257872025-01-03T08:36:19.113ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
25788 Jan 03 08:36:19.113 INFO request completed, latency_us: 275, response_code: 200, uri: /state/cid/1, method: GET, req_id: cb48e75a-a43d-4f2e-8eb8-9f162a05a0a4, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
25789 [8820] Exited with: ExitStatus(unix_wait_status(9))
25790 [8820][1] reports Exit
25791 Jan 03 08:36:23.115 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/1, method: GET, req_id: 24b40170-fd7a-4e9b-a4fa-27b378dc5184, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
25792 001/300 Read block 3727 len 40960
25793 002/300 Write block 3653 len 32768 data: 6 5 5 5 5 5 3 5
25794 003/300 Write block 4191 len 40960 data: 6 8 8 7 7 5 7 8 10 10
25795 004/300 Flush
25796 005/300 Read block 2498 len 12288
25797 006/300 Write block 295 len 4096 data: 4
25798 007/300 Flush
25799 008/300 Read block 1526 len 12288
25800 009/300 Write block 2717 len 24576 data: 6 5 6 7 8 8
25801 010/300 Read block 4592 len 36864
25802 011/300 Flush
25803 012/300 Write block 12 len 24576 data: 4 4 5 4 5 5
25804 013/300 Read block 2341 len 8192
25805 014/300 Read block 696 len 32768
25806 015/300 Write block 1586 len 36864 data: 7 7 7 8 7 6 5 7 7
25807 016/300 Write block 2551 len 20480 data: 2 2 2 2 2
25808 017/300 Write block 4995 len 4096 data: 4
25809 018/300 Flush
25810 019/300 Write block 3164 len 8192 data: 3 4
25811 020/300 Write block 7 len 4096 data: 3
25812 021/300 Write block 333 len 24576 data: 4 3 3 3 3 3
25813 022/300 Write block 3153 len 40960 data: 3 4 4 4 3 3 4 5 5 4
25814 023/300 Read block 3 len 8192
25815 024/300 Read block 2397 len 20480
25816 025/300 Flush
25817 026/300 Read block 2867 len 32768
25818 027/300 Write block 1096 len 16384 data: 7 8 7 6
25819 028/300 Read block 2477 len 40960
25820 029/300 Write block 2035 len 28672 data: 4 4 3 3 2 2 3
25821 030/300 Write block 161 len 28672 data: 5 5 4 5 5 5 3
25822 031/300 Read block 2703 len 40960
25823 032/300 Read block 3837 len 36864
25824 033/300 Write block 4817 len 16384 data: 6 6 5 4
25825 034/300 Write block 229 len 20480 data: 7 7 5 5 4
25826 035/300 Write block 1830 len 32768 data: 10 8 5 4 4 3 3 3
25827 036/300 Write block 2539 len 12288 data: 4 5 4
25828 037/300 Read block 580 len 32768
25829 038/300 Read block 1558 len 8192
25830 039/300 Read block 902 len 8192
25831 040/300 Write block 4173 len 20480 data: 7 5 5 5 6
25832 041/300 Read block 4799 len 12288
25833 042/300 Read block 103 len 16384
25834 043/300 Read block 660 len 32768
25835 044/300 Read block 2148 len 40960
25836 045/300 Write block 3544 len 4096 data: 9
25837 046/300 Flush
25838 047/300 Read block 1240 len 12288
25839 048/300 Flush
25840 049/300 Write block 3359 len 40960 data: 5 6 7 6 6 5 3 4 6 6
25841 050/300 Write block 193 len 32768 data: 4 5 6 6 6 5 4 3
25842 051/300 Read block 1669 len 40960
25843 052/300 Read block 3162 len 24576
25844 053/300 Read block 4478 len 40960
25845 054/300 Read block 1525 len 16384
25846 055/300 Read block 4384 len 24576
25847 056/300 Write block 1905 len 40960 data: 4 4 5 5 5 6 5 5 5 5
25848 057/300 Read block 247 len 24576
25849 058/300 Flush
25850 059/300 Write block 3937 len 20480 data: 5 6 6 6 5
25851 060/300 Write block 3977 len 8192 data: 5 5
25852 061/300 Read block 488 len 36864
25853 062/300 Read block 2887 len 4096
25854 063/300 Write block 3226 len 40960 data: 4 4 5 3 5 7 8 9 10 11
25855 064/300 Write block 4216 len 20480 data: 5 4 4 4 4
25856 065/300 Read block 4456 len 40960
25857 066/300 Write block 1239 len 40960 data: 2 2 2 4 4 4 4 4 4 6
25858 067/300 Read block 568 len 24576
25859 068/300 Write block 812 len 12288 data: 5 4 4
25860 069/300 Read block 4316 len 36864
25861 070/300 Read block 4895 len 16384
25862 071/300 Write block 2011 len 24576 data: 5 4 3 4 5 5
25863 072/300 Read block 1223 len 32768
25864 073/300 Read block 3791 len 28672
25865 074/300 Read block 1112 len 40960
25866 075/300 Write block 2700 len 24576 data: 5 5 4 3 3 3
25867 076/300 Flush
25868 077/300 Read block 222 len 16384
25869 078/300 Write block 461 len 8192 data: 6 7
25870 079/300 Write block 3132 len 36864 data: 4 4 4 4 4 4 5 6 6
25871 080/300 Flush
25872 081/300 Write block 3388 len 28672 data: 7 7 6 6 5 6 6
25873 082/300 Read block 2528 len 16384
25874 083/300 Write block 265 len 16384 data: 3 2 3 3
25875 084/300 Read block 1105 len 40960
25876 085/300 Write block 1034 len 20480 data: 2 2 2 3 4
25877 086/300 Write block 634 len 12288 data: 4 5 5
25878 087/300 Write block 1084 len 4096 data: 3
25879 088/300 Write block 3311 len 28672 data: 5 5 4 5 6 5 3
25880 089/300 Write block 4225 len 32768 data: 5 5 5 5 6 5 4 4
25881 090/300 Write block 473 len 40960 data: 4 4 4 4 4 4 4 5 5 6
25882 091/300 Write block 2493 len 8192 data: 5 5
25883 092/300 Read block 3400 len 12288
25884 093/300 Write block 2954 len 40960 data: 5 2 3 3 4 3 3 3 3 4
25885 094/300 Write block 2379 len 12288 data: 3 4 4
25886 095/300 Write block 2018 len 12288 data: 7 6 6
25887 096/300 Read block 2010 len 8192
25888 097/300 Write block 1920 len 24576 data: 7 6 7 8 8 7
25889 098/300 Read block 2738 len 32768
25890 099/300 Read block 1903 len 36864
25891 100/300 Write block 2858 len 28672 data: 5 5 5 5 3 3 3
25892 101/300 Read block 2698 len 36864
25893 102/300 Write block 2526 len 32768 data: 6 5 4 3 5 5 5 5
25894 103/300 Read block 79 len 8192
25895 104/300 Read block 2382 len 40960
25896 105/300 Flush
25897 106/300 Write block 4518 len 40960 data: 2 2 3 3 3 4 4 4 3 3
25898 107/300 Read block 222 len 40960
25899 108/300 Write block 1540 len 32768 data: 3 4 4 4 4 3 3 3
25900 109/300 Flush
25901 110/300 Write block 59 len 12288 data: 4 6 6
25902 111/300 Read block 4708 len 16384
25903 112/300 Read block 291 len 20480
25904 113/300 Write block 3635 len 12288 data: 6 7 6
25905 114/300 Write block 4581 len 12288 data: 8 6 4
25906 115/300 Write block 227 len 40960 data: 4 6 8 8 6 6 5 4 4 5
25907 116/300 Read block 4010 len 20480
25908 117/300 Write block 932 len 28672 data: 4 4 4 6 8 9 9
25909 118/300 Read block 2768 len 28672
25910 119/300 Read block 504 len 28672
25911 120/300 Flush
25912 121/300 Read block 1223 len 36864
25913 122/300 Read block 2193 len 28672
25914 123/300 Write block 562 len 36864 data: 4 4 4 4 4 4 4 4 4
25915 124/300 Read block 3686 len 32768
25916 125/300 Read block 448 len 28672
25917 126/300 Write block 3876 len 16384 data: 3 4 3 3
25918 127/300 Read block 2331 len 8192
25919 128/300 Read block 212 len 16384
25920 129/300 Read block 3444 len 24576
25921 130/300 Flush
25922 131/300 Write block 3204 len 40960 data: 4 4 4 4 4 4 3 4 4 3
25923 132/300 Read block 3745 len 36864
25924 133/300 Read block 2464 len 16384
25925 134/300 Flush
25926 135/300 Write block 4425 len 32768 data: 4 4 4 5 5 5 6 8
25927 136/300 Read block 1920 len 24576
25928 137/300 Read block 60 len 4096
25929 138/300 Write block 2552 len 24576 data: 3 3 3 3 2 3
25930 139/300 Write block 506 len 24576 data: 2 3 4 3 4 4
25931 140/300 Flush
25932 141/300 Write block 1287 len 36864 data: 5 4 5 6 8 7 7 6 5
25933 142/300 Read block 4137 len 36864
25934 143/300 Read block 294 len 20480
25935 144/300 Read block 1519 len 4096
25936 145/300 Read block 3576 len 12288
25937 146/300 Write block 3439 len 16384 data: 6 7 6 8
25938 147/300 Write block 768 len 8192 data: 4 5
25939 148/300 Write block 556 len 8192 data: 5 6
25940 149/300 Read block 115 len 32768
25941 150/300 Write block 2665 len 20480 data: 8 8 8 6 5
25942 151/300 Flush
25943 152/300 Write block 2641 len 36864 data: 7 6 5 5 5 4 2 2 2
25944 153/300 Write block 4421 len 16384 data: 5 6 5 5
25945 154/300 Write block 3809 len 40960 data: 4 4 3 3 3 4 4 4 4 3
25946 155/300 Read block 3077 len 40960
25947 156/300 Read block 1025 len 20480
25948 157/300 Write block 2168 len 28672 data: 6 4 4 4 4 4 3
25949 158/300 Read block 4840 len 36864
25950 159/300 Write block 1301 len 40960 data: 3 3 3 5 5 4 4 5 6 6
25951 160/300 Write block 1135 len 12288 data: 8 7 6
25952 161/300 Write block 2322 len 8192 data: 5 5
25953 162/300 Write block 4729 len 4096 data: 5
25954 163/300 Read block 3516 len 40960
25955 164/300 Write block 963 len 36864 data: 4 4 4 4 4 4 4 3 3
25956 165/300 Write block 4024 len 36864 data: 6 6 4 4 5 3 3 3 3
25957 166/300 Read block 3693 len 24576
25958 167/300 Write block 4352 len 8192 data: 4 2
25959 168/300 Write block 4520 len 32768 data: 4 4 4 5 5 5 4 4
25960 169/300 Read block 896 len 40960
25961 170/300 Read block 4149 len 24576
25962 171/300 Read block 4734 len 28672
25963 172/300 Read block 389 len 20480
25964 173/300 Write block 726 len 12288 data: 4 4 3
25965 174/300 Write block 2598 len 12288 data: 6 7 7
25966 175/300 Write block 1802 len 32768 data: 6 6 5 5 5 5 5 5
25967 176/300 Write block 807 len 36864 data: 4 3 3 3 4 6 5 5 4
25968 177/300 Read block 3900 len 12288
25969 178/300 Read block 2144 len 16384
25970 179/300 Read block 1306 len 40960
25971 180/300 Write block 4341 len 28672 data: 8 9 8 7 7 6 6
25972 181/300 Write block 1984 len 28672 data: 4 3 3 3 4 4 4
25973 182/300 Write block 112 len 12288 data: 2 4 5
25974 183/300 Read block 1924 len 16384
25975 184/300 Read block 3472 len 20480
25976 185/300 Write block 462 len 40960 data: 8 6 5 5 6 6 5 4 5 5
25977 186/300 Read block 3723 len 8192
25978 187/300 Write block 2909 len 40960 data: 6 6 5 4 4 4 5 5 5 4
25979 188/300 Write block 2441 len 16384 data: 5 5 6 5
25980 189/300 Write block 2355 len 32768 data: 3 3 3 2 2 3 4 4
25981 190/300 Read block 3868 len 40960
25982 191/300 Write block 4207 len 12288 data: 8 7 6
25983 192/300 Read block 3211 len 16384
25984 193/300 Read block 4797 len 12288
25985 194/300 Write block 88 len 4096 data: 6
25986 195/300 Read block 4716 len 24576
25987 196/300 Write block 1110 len 20480 data: 6 6 6 6 6
25988 197/300 Read block 1387 len 40960
25989 198/300 Read block 3086 len 4096
25990 199/300 Read block 2882 len 12288
25991 200/300 Read block 1112 len 8192
25992 201/300 Read block 2815 len 12288
25993 202/300 Read block 916 len 28672
25994 203/300 Write block 2381 len 40960 data: 5 5 6 6 7 3 3 5 5 5
25995 204/300 Write block 4689 len 4096 data: 5
25996 205/300 Write block 2059 len 20480 data: 4 5 4 4 3
25997 206/300 Write block 3358 len 40960 data: 5 6 7 8 7 7 6 4 5 7
25998 207/300 Read block 2731 len 4096
25999 208/300 Read block 971 len 24576
26000 209/300 Read block 207 len 4096
26001 210/300 Read block 3575 len 20480
26002 211/300 Read block 1395 len 4096
26003 212/300 Write block 4227 len 24576 data: 6 6 7 6 5 5
26004 213/300 Write block 4670 len 24576 data: 6 5 5 5 5 4
26005 214/300 Read block 2949 len 28672
26006 215/300 Flush
26007 216/300 Write block 3867 len 20480 data: 11 10 10 8 9
26008 217/300 Read block 3545 len 12288
26009 218/300 Write block 738 len 20480 data: 6 6 5 4 5
26010 219/300 Flush
26011 220/300 Read block 4469 len 16384
26012 221/300 Read block 4945 len 36864
26013 222/300 Flush
26014 223/300 Flush
26015 224/300 Read block 3443 len 12288
26016 225/300 Write block 2387 len 4096 data: 4
26017 226/300 Write block 2940 len 4096 data: 6
26018 227/300 Write block 899 len 36864 data: 6 6 6 6 5 4 4 5 7
26019 228/300 Read block 3156 len 16384
26020 229/300 Write block 3651 len 4096 data: 4
26021 230/300 Read block 4520 len 20480
26022 231/300 Flush
26023 232/300 Read block 3403 len 32768
26024 233/300 Read block 893 len 12288
26025 234/300 Read block 547 len 4096
26026 235/300 Read block 2441 len 16384
26027 236/300 Write block 3023 len 40960 data: 5 6 5 5 4 4 4 4 5 5
26028 237/300 Read block 1570 len 20480
26029 238/300 Read block 2323 len 8192
26030 239/300 Read block 4168 len 8192
26031 240/300 Flush
26032 241/300 Read block 4110 len 40960
26033 242/300 Write block 1767 len 40960 data: 3 4 4 4 3 2 4 6 7 6
26034 243/300 Write block 3422 len 32768 data: 10 10 11 10 8 7 7 7
26035 244/300 Read block 1908 len 28672
26036 245/300 Read block 4917 len 28672
26037 246/300 Read block 2926 len 32768
26038 247/300 Read block 2672 len 36864
26039 248/300 Write block 1562 len 8192 data: 4 3
26040 249/300 Write block 1715 len 12288 data: 5 4 4
26041 250/300 Write block 3422 len 40960 data: 11 11 12 11 9 8 8 8 5 6
26042 251/300 Read block 2481 len 28672
26043 252/300 Write block 1098 len 24576 data: 8 7 6 7 5 5
26044 253/300 Flush
26045 254/300 Read block 3033 len 24576
26046 255/300 Read block 4402 len 24576
26047 256/300 Write block 2429 len 20480 data: 3 2 2 2 2
26048 257/300 Flush
26049 258/300 Flush
26050 259/300 Write block 3044 len 20480 data: 6 7 8 7 6
26051 260/300 Read block 2447 len 40960
26052 261/300 Read block 1220 len 16384
26053 262/300 Write block 4485 len 28672 data: 5 4 4 4 3 3 4
26054 263/300 Read block 3017 len 28672
26055 264/300 Write block 4456 len 16384 data: 2 3 3 2
26056 265/300 Read block 564 len 20480
26057 266/300 Write block 2386 len 36864 data: 4 5 6 6 6 6 6 6 6
26058 267/300 Write block 2132 len 36864 data: 7 6 6 5 7 7 4 3 3
26059 268/300 Read block 4433 len 24576
26060 269/300 Write block 4645 len 28672 data: 5 5 5 7 8 4 4
26061 270/300 Write block 2282 len 12288 data: 2 2 5
26062 271/300 Write block 2784 len 20480 data: 4 4 4 4 4
26063 272/300 Read block 1583 len 24576
26064 273/300 Read block 967 len 4096
26065 274/300 Read block 3304 len 28672
26066 275/300 Write block 1197 len 40960 data: 4 5 4 4 3 3 3 4 5 5
26067 276/300 Read block 1837 len 20480
26068 277/300 Read block 1876 len 24576
26069 278/300 Write block 1575 len 12288 data: 6 6 6
26070 279/300 Read block 3983 len 28672
26071 280/300 Read block 4306 len 28672
26072 281/300 Write block 320 len 16384 data: 5 4 5 5
26073 282/300 Read block 2583 len 28672
26074 283/300 Write block 2109 len 32768 data: 2 2 7 7 7 9 7 7
26075 284/300 Read block 2459 len 32768
26076 285/300 Read block 2277 len 16384
26077 286/300 Read block 965 len 32768
26078 287/300 Read block 1237 len 4096
26079 288/300 Write block 2257 len 16384 data: 6 5 5 3
26080 289/300 Read block 2025 len 28672
26081 290/300 Flush
26082 291/300 Write block 4575 len 12288 data: 6 5 5
26083 292/300 Read block 3045 len 12288
26084 293/300 Read block 3787 len 20480
26085 294/300 Write block 2205 len 20480 data: 6 5 5 4 4
26086 295/300 Read block 3580 len 20480
26087 296/300 Write block 3043 len 36864 data: 6 7 8 9 8 7 5 4 4
26088 297/300 Read block 3966 len 8192
26089 298/300 Write block 1790 len 36864 data: 5 5 5 6 5 5 4 5 5
26090 299/300 Write block 3890 len 40960 data: 3 4 5 4 6 7 9 8 6 6
26091 300/300 Read block 2681 len 40960
26092 301/300 Read block 1264 len 4096
26093 Main task has work to do, go find it
26094 got dsc Start(1)
26095 start 1
26096 Jan 03 08:36:23.362 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/1, method: POST, req_id: bdfcbf8a-5436-4c31-a3f0-e717a922efd0, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
26097 Replay: started 1, returned:Ok(())
26098 ----------------------------------------------------------------
26099 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26100 States: Active Offline Active
26101 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26102 7642 Acked Read 10 Done Sent Done false
26103 7643 Acked Write 8 Done Sent Done false
26104 7644 Acked Write 10 Done Sent Done false
26105 7645 Acked Flush 0 Done Sent Done false
26106 7646 Acked Read 3 Done Sent Done false
26107 7647 Acked Write 1 Done Sent Done false
26108 7648 Acked Flush 0 Done Sent Done false
26109 7649 Acked Read 3 Done Sent Done false
26110 7650 Acked Write 6 Done Sent Done false
26111 7651 Acked Read 9 Done Sent Done false
26112 7652 Acked Flush 0 Done Sent Done false
26113 7653 Acked Write 6 Done Sent Done false
26114 7654 Acked Read 2 Done Sent Done false
26115 7655 Acked Read 8 Done Sent Done false
26116 7656 Acked Write 9 Done Sent Done false
26117 7657 Acked Write 5 Done Sent Done false
26118 7658 Acked Write 1 Done Sent Done false
26119 7659 Acked Flush 0 Done Sent Done false
26120 7660 Acked Write 2 Done Sent Done false
26121 7661 Acked Write 1 Done Sent Done false
26122 7662 Acked Write 6 Done Sent Done false
26123 7663 Acked Write 10 Done Sent Done false
26124 7664 Acked Read 2 Done Sent Done false
26125 7665 Acked Read 5 Done Sent Done false
26126 7666 Acked Flush 0 Done Sent Done false
26127 7667 Acked Read 8 Done Sent Done false
26128 7668 Acked Write 4 Done Sent Done false
26129 7669 Acked Read 10 Done Sent Done false
26130 7670 Acked Write 7 Done Sent Done false
26131 7671 Acked Write 7 Done Sent Done false
26132 7672 Acked Read 10 Done Sent Done false
26133 7673 Acked Read 9 Done Sent Done false
26134 7674 Acked Write 4 Done Sent Done false
26135 7675 Acked Write 5 Done Sent Done false
26136 7676 Acked Write 8 Done Sent Done false
26137 7677 Acked Write 3 Done Sent Done false
26138 7678 Acked Read 8 Done Sent Done false
26139 7679 Acked Read 2 Done Sent Done false
26140 7680 Acked Read 2 Done Sent Done false
26141 7681 Acked Write 5 Done Sent Done false
26142 7682 Acked Read 3 Done Sent Done false
26143 7683 Acked Read 4 Done Sent Done false
26144 7684 Acked Read 8 Done Sent Done false
26145 7685 Acked Read 10 Done Sent Done false
26146 7686 Acked Write 1 Done Sent Done false
26147 7687 Acked Flush 0 Done Sent Done false
26148 7688 Acked Read 3 Done Sent Done false
26149 7689 Acked Flush 0 Done Sent Done false
26150 7690 Acked Write 10 Done Sent Done false
26151 7691 Acked Write 8 Done Sent Done false
26152 7692 Acked Read 10 Done Sent Done false
26153 7693 Acked Read 6 Done Sent Done false
26154 7694 Acked Read 10 Done Sent Done false
26155 7695 Acked Read 4 Done Sent Done false
26156 7696 Acked Read 6 Done Sent Done false
26157 7697 Acked Write 10 Done Sent Done false
26158 7698 Acked Read 6 Done Sent Done false
26159 7699 Acked Flush 0 Done Sent Done false
26160 7700 Acked Write 5 Done Sent Done false
26161 7701 Acked Write 2 Done Sent Done false
26162 7702 Acked Read 9 Done Sent Done false
26163 7703 Acked Read 1 Done Sent Done false
26164 7704 Acked Write 10 Done Sent Done false
26165 7705 Acked Write 5 Done Sent Done false
26166 7706 Acked Read 10 Done Sent Done false
26167 7707 Acked Write 10 Done Sent Done false
26168 7708 Acked Read 6 Done Sent Done false
26169 7709 Acked Write 3 Done Sent Done false
26170 7710 Acked Read 9 Done Sent Done false
26171 7711 Acked Read 4 Done Sent Done false
26172 7712 Acked Write 6 Done Sent Done false
26173 7713 Acked Read 8 Done Sent Done false
26174 7714 Acked Read 7 Done Sent Done false
26175 7715 Acked Read 10 Done Sent Done false
26176 7716 Acked Write 6 Done Sent Done false
26177 7717 Acked Flush 0 Done Sent Done false
26178 7718 Acked Read 4 Done Sent Done false
26179 7719 Acked Write 2 Done Sent Done false
26180 7720 Acked Write 9 Done Sent Done false
26181 7721 Acked Flush 0 Done Sent Done false
26182 7722 Acked Write 7 Done Sent Done false
26183 7723 Acked Read 4 Done Sent Done false
26184 7724 Acked Write 4 Done Sent Done false
26185 7725 Acked Read 10 Done Sent Done false
26186 7726 Acked Write 5 Done Sent Done false
26187 7727 Acked Write 3 Done Sent Done false
26188 7728 Acked Write 1 Done Sent Done false
26189 7729 Acked Write 7 Done Sent Done false
26190 7730 Acked Write 8 Done Sent Done false
26191 7731 Acked Write 10 Done Sent Done false
26192 7732 Acked Write 2 Done Sent Done false
26193 7733 Acked Read 3 Done Sent Done false
26194 7734 Acked Write 10 Done Sent Done false
26195 7735 Acked Write 3 Done Sent Done false
26196 7736 Acked Write 3 Done Sent Done false
26197 7737 Acked Read 2 Done Sent Done false
26198 7738 Acked Write 6 Done Sent Done false
26199 7739 Acked Read 8 Done Sent Done false
26200 7740 Acked Read 9 Done Sent Done false
26201 7741 Acked Write 7 Done Sent Done false
26202 7742 Acked Read 9 Done Sent Done false
26203 7743 Acked Write 8 Done Sent Done false
26204 7744 Acked Read 2 Done Sent Done false
26205 7745 Acked Read 10 Done Sent Done false
26206 7746 Acked Flush 0 Done Sent Done false
26207 7747 Acked Write 10 Done Sent Done false
26208 7748 Acked Read 10 Done Sent Done false
26209 7749 Acked Write 8 Done Sent Done false
26210 7750 Acked Flush 0 Done Sent Done false
26211 7751 Acked Write 3 Done Sent Done false
26212 7752 Acked Read 4 Done Sent Done false
26213 7753 Acked Read 5 Done Sent Done false
26214 7754 Acked Write 3 Done Sent Done false
26215 7755 Acked Write 3 Done Sent Done false
26216 7756 Acked Write 10 Done Sent Done false
26217 7757 Acked Read 5 Done Sent Done false
26218 7758 Acked Write 7 Done Sent Done false
26219 7759 Acked Read 7 Done Sent Done false
26220 7760 Acked Read 7 Done Sent Done false
26221 7761 Acked Flush 0 Done Sent Done false
26222 7762 Acked Read 9 Done Sent Done false
26223 7763 Acked Read 7 Done Sent Done false
26224 7764 Acked Write 9 Done Sent Done false
26225 7765 Acked Read 8 Done Sent Done false
26226 7766 Acked Read 7 Done Sent Done false
26227 7767 Acked Write 4 Done Sent Done false
26228 7768 Acked Read 2 Done Sent Done false
26229 7769 Acked Read 4 Done Sent Done false
26230 7770 Acked Read 6 Done Sent Done false
26231 7771 Acked Flush 0 Done Sent Done false
26232 7772 Acked Write 10 Done Sent Done false
26233 7773 Acked Read 9 Done Sent Done false
26234 7774 Acked Read 4 Done Sent Done false
26235 7775 Acked Flush 0 Done Sent Done false
26236 7776 Acked Write 8 Done Sent Done false
26237 7777 Acked Read 6 Done Sent Done false
26238 7778 Acked Read 1 Done Sent Done false
26239 7779 Acked Write 6 Done Sent Done false
26240 7780 Acked Write 6 Done Sent Done false
26241 7781 Acked Flush 0 Done Sent Done false
26242 7782 Acked Write 9 Done Sent Done false
26243 7783 Acked Read 9 Done Sent Done false
26244 7784 Acked Read 5 Done Sent Done false
26245 7785 Acked Read 1 Done Sent Done false
26246 7786 Acked Read 3 Done Sent Done false
26247 7787 Acked Write 4 Done Sent Done false
26248 7788 Acked Write 2 Done Sent Done false
26249 7789 Acked Write 2 Done Sent Done false
26250 7790 Acked Read 8 Done Sent Done false
26251 7791 Acked Write 5 Done Sent Done false
26252 7792 Acked Flush 0 Done Sent Done false
26253 7793 Acked Write 9 Done Sent Done false
26254 7794 Acked Write 4 Done Sent Done false
26255 7795 Acked Write 10 Done Sent Done false
26256 7796 Acked Read 10 Done Sent Done false
26257 7797 Acked Read 5 Done Sent Done false
26258 7798 Acked Write 7 Done Sent Done false
26259 7799 Acked Read 9 Done Sent Done false
26260 7800 Acked Write 10 Done Sent Done false
26261 7801 Acked Write 3 Done Sent Done false
26262 7802 Acked Write 2 Done Sent Done false
26263 7803 Acked Write 1 Done Sent Done false
26264 7804 Acked Read 10 Done Sent Done false
26265 7805 Acked Write 9 Done Sent Done false
26266 7806 Acked Write 9 Done Sent Done false
26267 7807 Acked Read 6 Done Sent Done false
26268 7808 Acked Write 2 Done Sent Done false
26269 7809 Acked Write 8 Done Sent Done false
26270 7810 Acked Read 10 Done Sent Done false
26271 7811 Acked Read 6 Done Sent Done false
26272 7812 Acked Read 7 Done Sent Done false
26273 7813 Acked Read 5 Done Sent Done false
26274 7814 Acked Write 3 Done Sent Done false
26275 7815 Acked Write 3 Done Sent Done false
26276 7816 Acked Write 8 Done Sent Done false
26277 7817 Acked Write 9 Done Sent Done false
26278 7818 Acked Read 3 Done Sent Done false
26279 7819 Acked Read 4 Done Sent Done false
26280 7820 Acked Read 10 Done Sent Done false
26281 7821 Acked Write 7 Done Sent Done false
26282 7822 Acked Write 7 Done Sent Done false
26283 7823 Acked Write 3 Done Sent Done false
26284 7824 Acked Read 4 Done Sent Done false
26285 7825 Acked Read 5 Done Sent Done false
26286 7826 Acked Write 10 Done Sent Done false
26287 7827 Acked Read 2 Done Sent Done false
26288 7828 Acked Write 10 Done Sent Done false
26289 7829 Acked Write 4 Done Sent Done false
26290 7830 Acked Write 8 Done Sent Done false
26291 7831 Acked Read 10 Done Sent Done false
26292 7832 Acked Write 3 Done Sent Done false
26293 7833 Acked Read 4 Done Sent Done false
26294 7834 Acked Read 3 Done Sent Done false
26295 7835 Acked Write 1 Done Sent Done false
26296 7836 Acked Read 6 Done Sent Done false
26297 7837 Acked Write 5 Done Sent Done false
26298 7838 Acked Read 10 Done Sent Done false
26299 7839 Acked Read 1 Done Sent Done false
26300 7840 Acked Read 3 Done Sent Done false
26301 7841 Acked Read 2 Done Sent Done false
26302 7842 Acked Read 3 Done Sent Done false
26303 7843 Acked Read 7 Done Sent Done false
26304 7844 Acked Write 10 Done Sent Done false
26305 7845 Acked Write 1 Done Sent Done false
26306 7846 Acked Write 5 Done Sent Done false
26307 7847 Acked Write 10 Done Sent Done false
26308 7848 Acked Read 1 Done Sent Done false
26309 7849 Acked Read 6 Done Sent Done false
26310 7850 Acked Read 1 Done Sent Done false
26311 7851 Acked Read 5 Done Sent Done false
26312 7852 Acked Read 1 Done Sent Done false
26313 7853 Acked Write 6 Done Sent Done false
26314 7854 Acked Write 6 Done Sent Done false
26315 7855 Acked Read 7 Done Sent Done false
26316 7856 Acked Flush 0 Done Sent Done false
26317 7857 Acked Write 5 Done Sent Done false
26318 7858 Acked Read 3 Done Sent Done false
26319 7859 Acked Write 5 Done Sent Done false
26320 7860 Acked Flush 0 Done Sent Done false
26321 7861 Acked Read 4 Done Sent Done false
26322 7862 Acked Read 9 Done Sent Done false
26323 7863 Acked Flush 0 Done Sent Done false
26324 7864 Acked Flush 0 Done Sent Done false
26325 7865 Acked Read 3 Done Sent Done false
26326 7866 Acked Write 1 Done Sent Done false
26327 7867 Acked Write 1 Done Sent Done false
26328 7868 Acked Write 9 Done Sent Done false
26329 7869 Acked Read 4 Done Sent Done false
26330 7870 Acked Write 1 Done Sent Done false
26331 7871 Acked Read 5 Done Sent Done false
26332 7872 Acked Flush 0 Done Sent Done false
26333 7873 Acked Read 8 Done Sent Done false
26334 7874 Acked Read 3 Done Sent Done false
26335 7875 Acked Read 1 Done Sent Done false
26336 7876 Acked Read 4 Done Sent Done false
26337 7877 Acked Write 10 Done Sent Done false
26338 7878 Acked Read 5 Done Sent Done false
26339 7879 Acked Read 2 Done Sent Done false
26340 7880 Acked Read 2 Done Sent Done false
26341 7881 Acked Flush 0 Done Sent Done false
26342 7882 Acked Read 10 Done Sent Done false
26343 7883 Acked Write 10 Done Sent Done false
26344 7884 Acked Write 8 Done Sent Done false
26345 7885 Acked Read 7 Done Sent Done false
26346 7886 Acked Read 7 Done Sent Done false
26347 7887 Acked Read 8 Done Sent Done false
26348 7888 Acked Read 9 Done Sent Done false
26349 7889 Acked Write 2 Done Sent Done false
26350 7890 Acked Write 3 Done Sent Done false
26351 7891 Acked Write 10 Done Sent Done false
26352 7892 Acked Read 7 Done Sent Done false
26353 7893 Acked Write 6 Done Sent Done false
26354 7894 Acked Flush 0 Done Sent Done false
26355 7895 Acked Read 6 Done Sent Done false
26356 7896 Acked Read 6 Done Sent Done false
26357 7897 Acked Write 5 Done Sent Done false
26358 7898 Acked Flush 0 Done Sent Done false
26359 7899 Acked Flush 0 Done Sent Done false
26360 7900 Acked Write 5 Done Sent Done false
26361 7901 Acked Read 10 Done Sent Done false
26362 7902 Acked Read 4 Done Sent Done false
26363 7903 Acked Write 7 Done Sent Done false
26364 7904 Acked Read 7 Done Sent Done false
26365 7905 Acked Write 4 Done Sent Done false
26366 7906 Acked Read 5 Done Sent Done false
26367 7907 Acked Write 9 Done Sent Done false
26368 7908 Acked Write 9 Done Sent Done false
26369 7909 Acked Read 6 Done Sent Done false
26370 7910 Acked Write 7 Done Sent Done false
26371 7911 Acked Write 3 Done Sent Done false
26372 7912 Acked Write 5 Done Sent Done false
26373 7913 Acked Read 6 Done Sent Done false
26374 7914 Acked Read 1 Done Sent Done false
26375 7915 Acked Read 7 Done Sent Done false
26376 7916 Acked Write 10 Done Sent Done false
26377 7917 Acked Read 5 Done Sent Done false
26378 7918 Acked Read 6 Done Sent Done false
26379 7919 Acked Write 3 Done Sent Done false
26380 7920 Acked Read 7 Done Sent Done false
26381 7921 Acked Read 7 Done Sent Done false
26382 7922 Acked Write 4 Done Sent Done false
26383 7923 Acked Read 7 Done Sent Done false
26384 7924 Acked Write 8 Done Sent Done false
26385 7925 Acked Read 8 Done Sent Done false
26386 7926 Acked Read 4 Done Sent Done false
26387 7927 Acked Read 8 Done Sent Done false
26388 7928 Acked Read 1 Done Sent Done false
26389 7929 Acked Write 4 Done Sent Done false
26390 7930 Acked Read 7 Done Sent Done false
26391 7931 Acked Flush 0 Done Sent Done false
26392 7932 Acked Write 3 Done Sent Done false
26393 7933 Acked Read 3 Done Sent Done false
26394 7934 Acked Read 5 Done Sent Done false
26395 7935 Acked Write 5 Done Sent Done false
26396 7936 Acked Read 5 Done Sent Done false
26397 7937 Acked Write 9 Done Sent Done false
26398 7938 Acked Read 2 Done Sent Done false
26399 7939 Acked Write 9 Done Sent Done false
26400 7940 Acked Write 10 Done Sent Done false
26401 7941 Acked Read 10 Done Sent Done false
26402 7942 Acked Read 1 Done Sent Done false
26403 STATES DS:0 DS:1 DS:2 TOTAL
26404 Sent 0 301 0 301
26405 Done 301 0 301 602
26406 Skipped 0 0 0 0
26407 Error 0 0 0 0
26408 Last Flush: 7931 7641 7931
26409 Downstairs last five completed: 7641 7640 7639 7638 7637
26410 Upstairs last five completed: 7942 7941 7940 7939 7938
26411 CLIENT: Up:0 ds:301 act:2
26412 ----------------------------------------------------------------
26413 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26414 States: Active Offline Active
26415 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26416 7642 Acked Read 10 Done Sent Done false
26417 7643 Acked Write 8 Done Sent Done false
26418 7644 Acked Write 10 Done Sent Done false
26419 7645 Acked Flush 0 Done Sent Done false
26420 7646 Acked Read 3 Done Sent Done false
26421 7647 Acked Write 1 Done Sent Done false
26422 7648 Acked Flush 0 Done Sent Done false
26423 7649 Acked Read 3 Done Sent Done false
26424 7650 Acked Write 6 Done Sent Done false
26425 7651 Acked Read 9 Done Sent Done false
26426 7652 Acked Flush 0 Done Sent Done false
26427 7653 Acked Write 6 Done Sent Done false
26428 7654 Acked Read 2 Done Sent Done false
26429 7655 Acked Read 8 Done Sent Done false
26430 7656 Acked Write 9 Done Sent Done false
26431 7657 Acked Write 5 Done Sent Done false
26432 7658 Acked Write 1 Done Sent Done false
26433 7659 Acked Flush 0 Done Sent Done false
26434 7660 Acked Write 2 Done Sent Done false
26435 7661 Acked Write 1 Done Sent Done false
26436 7662 Acked Write 6 Done Sent Done false
26437 7663 Acked Write 10 Done Sent Done false
26438 7664 Acked Read 2 Done Sent Done false
26439 7665 Acked Read 5 Done Sent Done false
26440 7666 Acked Flush 0 Done Sent Done false
26441 7667 Acked Read 8 Done Sent Done false
26442 7668 Acked Write 4 Done Sent Done false
26443 7669 Acked Read 10 Done Sent Done false
26444 7670 Acked Write 7 Done Sent Done false
26445 7671 Acked Write 7 Done Sent Done false
26446 7672 Acked Read 10 Done Sent Done false
26447 7673 Acked Read 9 Done Sent Done false
26448 7674 Acked Write 4 Done Sent Done false
26449 7675 Acked Write 5 Done Sent Done false
26450 7676 Acked Write 8 Done Sent Done false
26451 7677 Acked Write 3 Done Sent Done false
26452 7678 Acked Read 8 Done Sent Done false
26453 7679 Acked Read 2 Done Sent Done false
26454 7680 Acked Read 2 Done Sent Done false
26455 7681 Acked Write 5 Done Sent Done false
26456 7682 Acked Read 3 Done Sent Done false
26457 7683 Acked Read 4 Done Sent Done false
26458 7684 Acked Read 8 Done Sent Done false
26459 7685 Acked Read 10 Done Sent Done false
26460 7686 Acked Write 1 Done Sent Done false
26461 7687 Acked Flush 0 Done Sent Done false
26462 7688 Acked Read 3 Done Sent Done false
26463 7689 Acked Flush 0 Done Sent Done false
26464 7690 Acked Write 10 Done Sent Done false
26465 7691 Acked Write 8 Done Sent Done false
26466 7692 Acked Read 10 Done Sent Done false
26467 7693 Acked Read 6 Done Sent Done false
26468 7694 Acked Read 10 Done Sent Done false
26469 7695 Acked Read 4 Done Sent Done false
26470 7696 Acked Read 6 Done Sent Done false
26471 7697 Acked Write 10 Done Sent Done false
26472 7698 Acked Read 6 Done Sent Done false
26473 7699 Acked Flush 0 Done Sent Done false
26474 7700 Acked Write 5 Done Sent Done false
26475 7701 Acked Write 2 Done Sent Done false
26476 7702 Acked Read 9 Done Sent Done false
26477 7703 Acked Read 1 Done Sent Done false
26478 7704 Acked Write 10 Done Sent Done false
26479 7705 Acked Write 5 Done Sent Done false
26480 7706 Acked Read 10 Done Sent Done false
26481 7707 Acked Write 10 Done Sent Done false
26482 7708 Acked Read 6 Done Sent Done false
26483 7709 Acked Write 3 Done Sent Done false
26484 7710 Acked Read 9 Done Sent Done false
26485 7711 Acked Read 4 Done Sent Done false
26486 7712 Acked Write 6 Done Sent Done false
26487 7713 Acked Read 8 Done Sent Done false
26488 7714 Acked Read 7 Done Sent Done false
26489 7715 Acked Read 10 Done Sent Done false
26490 7716 Acked Write 6 Done Sent Done false
26491 7717 Acked Flush 0 Done Sent Done false
26492 7718 Acked Read 4 Done Sent Done false
26493 7719 Acked Write 2 Done Sent Done false
26494 7720 Acked Write 9 Done Sent Done false
26495 7721 Acked Flush 0 Done Sent Done false
26496 7722 Acked Write 7 Done Sent Done false
26497 7723 Acked Read 4 Done Sent Done false
26498 7724 Acked Write 4 Done Sent Done false
26499 7725 Acked Read 10 Done Sent Done false
26500 7726 Acked Write 5 Done Sent Done false
26501 7727 Acked Write 3 Done Sent Done false
26502 7728 Acked Write 1 Done Sent Done false
26503 7729 Acked Write 7 Done Sent Done false
26504 7730 Acked Write 8 Done Sent Done false
26505 7731 Acked Write 10 Done Sent Done false
26506 7732 Acked Write 2 Done Sent Done false
26507 7733 Acked Read 3 Done Sent Done false
26508 7734 Acked Write 10 Done Sent Done false
26509 7735 Acked Write 3 Done Sent Done false
26510 7736 Acked Write 3 Done Sent Done false
26511 7737 Acked Read 2 Done Sent Done false
26512 7738 Acked Write 6 Done Sent Done false
26513 7739 Acked Read 8 Done Sent Done false
26514 7740 Acked Read 9 Done Sent Done false
26515 7741 Acked Write 7 Done Sent Done false
26516 7742 Acked Read 9 Done Sent Done false
26517 7743 Acked Write 8 Done Sent Done false
26518 7744 Acked Read 2 Done Sent Done false
26519 7745 Acked Read 10 Done Sent Done false
26520 7746 Acked Flush 0 Done Sent Done false
26521 7747 Acked Write 10 Done Sent Done false
26522 7748 Acked Read 10 Done Sent Done false
26523 7749 Acked Write 8 Done Sent Done false
26524 7750 Acked Flush 0 Done Sent Done false
26525 7751 Acked Write 3 Done Sent Done false
26526 7752 Acked Read 4 Done Sent Done false
26527 7753 Acked Read 5 Done Sent Done false
26528 7754 Acked Write 3 Done Sent Done false
26529 7755 Acked Write 3 Done Sent Done false
26530 7756 Acked Write 10 Done Sent Done false
26531 7757 Acked Read 5 Done Sent Done false
26532 7758 Acked Write 7 Done Sent Done false
26533 7759 Acked Read 7 Done Sent Done false
26534 7760 Acked Read 7 Done Sent Done false
26535 7761 Acked Flush 0 Done Sent Done false
26536 7762 Acked Read 9 Done Sent Done false
26537 7763 Acked Read 7 Done Sent Done false
26538 7764 Acked Write 9 Done Sent Done false
26539 7765 Acked Read 8 Done Sent Done false
26540 7766 Acked Read 7 Done Sent Done false
26541 7767 Acked Write 4 Done Sent Done false
26542 7768 Acked Read 2 Done Sent Done false
26543 7769 Acked Read 4 Done Sent Done false
26544 7770 Acked Read 6 Done Sent Done false
26545 7771 Acked Flush 0 Done Sent Done false
26546 7772 Acked Write 10 Done Sent Done false
26547 7773 Acked Read 9 Done Sent Done false
26548 7774 Acked Read 4 Done Sent Done false
26549 7775 Acked Flush 0 Done Sent Done false
26550 7776 Acked Write 8 Done Sent Done false
26551 7777 Acked Read 6 Done Sent Done false
26552 7778 Acked Read 1 Done Sent Done false
26553 7779 Acked Write 6 Done Sent Done false
26554 7780 Acked Write 6 Done Sent Done false
26555 7781 Acked Flush 0 Done Sent Done false
26556 7782 Acked Write 9 Done Sent Done false
26557 7783 Acked Read 9 Done Sent Done false
26558 7784 Acked Read 5 Done Sent Done false
26559 7785 Acked Read 1 Done Sent Done false
26560 7786 Acked Read 3 Done Sent Done false
26561 7787 Acked Write 4 Done Sent Done false
26562 7788 Acked Write 2 Done Sent Done false
26563 7789 Acked Write 2 Done Sent Done false
26564 7790 Acked Read 8 Done Sent Done false
26565 7791 Acked Write 5 Done Sent Done false
26566 7792 Acked Flush 0 Done Sent Done false
26567 7793 Acked Write 9 Done Sent Done false
26568 7794 Acked Write 4 Done Sent Done false
26569 7795 Acked Write 10 Done Sent Done false
26570 7796 Acked Read 10 Done Sent Done false
26571 7797 Acked Read 5 Done Sent Done false
26572 7798 Acked Write 7 Done Sent Done false
26573 7799 Acked Read 9 Done Sent Done false
26574 7800 Acked Write 10 Done Sent Done false
26575 7801 Acked Write 3 Done Sent Done false
26576 7802 Acked Write 2 Done Sent Done false
26577 7803 Acked Write 1 Done Sent Done false
26578 7804 Acked Read 10 Done Sent Done false
26579 7805 Acked Write 9 Done Sent Done false
26580 7806 Acked Write 9 Done Sent Done false
26581 7807 Acked Read 6 Done Sent Done false
26582 7808 Acked Write 2 Done Sent Done false
26583 7809 Acked Write 8 Done Sent Done false
26584 7810 Acked Read 10 Done Sent Done false
26585 7811 Acked Read 6 Done Sent Done false
26586 7812 Acked Read 7 Done Sent Done false
26587 7813 Acked Read 5 Done Sent Done false
26588 7814 Acked Write 3 Done Sent Done false
26589 7815 Acked Write 3 Done Sent Done false
26590 7816 Acked Write 8 Done Sent Done false
26591 7817 Acked Write 9 Done Sent Done false
26592 7818 Acked Read 3 Done Sent Done false
26593 7819 Acked Read 4 Done Sent Done false
26594 7820 Acked Read 10 Done Sent Done false
26595 7821 Acked Write 7 Done Sent Done false
26596 7822 Acked Write 7 Done Sent Done false
26597 7823 Acked Write 3 Done Sent Done false
26598 7824 Acked Read 4 Done Sent Done false
26599 7825 Acked Read 5 Done Sent Done false
26600 7826 Acked Write 10 Done Sent Done false
26601 7827 Acked Read 2 Done Sent Done false
26602 7828 Acked Write 10 Done Sent Done false
26603 7829 Acked Write 4 Done Sent Done false
26604 7830 Acked Write 8 Done Sent Done false
26605 7831 Acked Read 10 Done Sent Done false
26606 7832 Acked Write 3 Done Sent Done false
26607 7833 Acked Read 4 Done Sent Done false
26608 7834 Acked Read 3 Done Sent Done false
26609 7835 Acked Write 1 Done Sent Done false
26610 7836 Acked Read 6 Done Sent Done false
26611 7837 Acked Write 5 Done Sent Done false
26612 7838 Acked Read 10 Done Sent Done false
26613 7839 Acked Read 1 Done Sent Done false
26614 7840 Acked Read 3 Done Sent Done false
26615 7841 Acked Read 2 Done Sent Done false
26616 7842 Acked Read 3 Done Sent Done false
26617 7843 Acked Read 7 Done Sent Done false
26618 7844 Acked Write 10 Done Sent Done false
26619 7845 Acked Write 1 Done Sent Done false
26620 7846 Acked Write 5 Done Sent Done false
26621 7847 Acked Write 10 Done Sent Done false
26622 7848 Acked Read 1 Done Sent Done false
26623 7849 Acked Read 6 Done Sent Done false
26624 7850 Acked Read 1 Done Sent Done false
26625 7851 Acked Read 5 Done Sent Done false
26626 7852 Acked Read 1 Done Sent Done false
26627 7853 Acked Write 6 Done Sent Done false
26628 7854 Acked Write 6 Done Sent Done false
26629 7855 Acked Read 7 Done Sent Done false
26630 7856 Acked Flush 0 Done Sent Done false
26631 7857 Acked Write 5 Done Sent Done false
26632 7858 Acked Read 3 Done Sent Done false
26633 7859 Acked Write 5 Done Sent Done false
26634 7860 Acked Flush 0 Done Sent Done false
26635 7861 Acked Read 4 Done Sent Done false
26636 7862 Acked Read 9 Done Sent Done false
26637 7863 Acked Flush 0 Done Sent Done false
26638 7864 Acked Flush 0 Done Sent Done false
26639 7865 Acked Read 3 Done Sent Done false
26640 7866 Acked Write 1 Done Sent Done false
26641 7867 Acked Write 1 Done Sent Done false
26642 7868 Acked Write 9 Done Sent Done false
26643 7869 Acked Read 4 Done Sent Done false
26644 7870 Acked Write 1 Done Sent Done false
26645 7871 Acked Read 5 Done Sent Done false
26646 7872 Acked Flush 0 Done Sent Done false
26647 7873 Acked Read 8 Done Sent Done false
26648 7874 Acked Read 3 Done Sent Done false
26649 7875 Acked Read 1 Done Sent Done false
26650 7876 Acked Read 4 Done Sent Done false
26651 7877 Acked Write 10 Done Sent Done false
26652 7878 Acked Read 5 Done Sent Done false
26653 7879 Acked Read 2 Done Sent Done false
26654 7880 Acked Read 2 Done Sent Done false
26655 7881 Acked Flush 0 Done Sent Done false
26656 7882 Acked Read 10 Done Sent Done false
26657 7883 Acked Write 10 Done Sent Done false
26658 7884 Acked Write 8 Done Sent Done false
26659 7885 Acked Read 7 Done Sent Done false
26660 7886 Acked Read 7 Done Sent Done false
26661 7887 Acked Read 8 Done Sent Done false
26662 7888 Acked Read 9 Done Sent Done false
26663 7889 Acked Write 2 Done Sent Done false
26664 7890 Acked Write 3 Done Sent Done false
26665 7891 Acked Write 10 Done Sent Done false
26666 7892 Acked Read 7 Done Sent Done false
26667 7893 Acked Write 6 Done Sent Done false
26668 7894 Acked Flush 0 Done Sent Done false
26669 7895 Acked Read 6 Done Sent Done false
26670 7896 Acked Read 6 Done Sent Done false
26671 7897 Acked Write 5 Done Sent Done false
26672 7898 Acked Flush 0 Done Sent Done false
26673 7899 Acked Flush 0 Done Sent Done false
26674 7900 Acked Write 5 Done Sent Done false
26675 7901 Acked Read 10 Done Sent Done false
26676 7902 Acked Read 4 Done Sent Done false
26677 7903 Acked Write 7 Done Sent Done false
26678 7904 Acked Read 7 Done Sent Done false
26679 7905 Acked Write 4 Done Sent Done false
26680 7906 Acked Read 5 Done Sent Done false
26681 7907 Acked Write 9 Done Sent Done false
26682 7908 Acked Write 9 Done Sent Done false
26683 7909 Acked Read 6 Done Sent Done false
26684 7910 Acked Write 7 Done Sent Done false
26685 7911 Acked Write 3 Done Sent Done false
26686 7912 Acked Write 5 Done Sent Done false
26687 7913 Acked Read 6 Done Sent Done false
26688 7914 Acked Read 1 Done Sent Done false
26689 7915 Acked Read 7 Done Sent Done false
26690 7916 Acked Write 10 Done Sent Done false
26691 7917 Acked Read 5 Done Sent Done false
26692 7918 Acked Read 6 Done Sent Done false
26693 7919 Acked Write 3 Done Sent Done false
26694 7920 Acked Read 7 Done Sent Done false
26695 7921 Acked Read 7 Done Sent Done false
26696 7922 Acked Write 4 Done Sent Done false
26697 7923 Acked Read 7 Done Sent Done false
26698 7924 Acked Write 8 Done Sent Done false
26699 7925 Acked Read 8 Done Sent Done false
26700 7926 Acked Read 4 Done Sent Done false
26701 7927 Acked Read 8 Done Sent Done false
26702 7928 Acked Read 1 Done Sent Done false
26703 7929 Acked Write 4 Done Sent Done false
26704 7930 Acked Read 7 Done Sent Done false
26705 7931 Acked Flush 0 Done Sent Done false
26706 7932 Acked Write 3 Done Sent Done false
26707 7933 Acked Read 3 Done Sent Done false
26708 7934 Acked Read 5 Done Sent Done false
26709 7935 Acked Write 5 Done Sent Done false
26710 7936 Acked Read 5 Done Sent Done false
26711 7937 Acked Write 9 Done Sent Done false
26712 7938 Acked Read 2 Done Sent Done false
26713 7939 Acked Write 9 Done Sent Done false
26714 7940 Acked Write 10 Done Sent Done false
26715 7941 Acked Read 10 Done Sent Done false
26716 7942 Acked Read 1 Done Sent Done false
26717 7943 Acked Flush 0 Done Sent Done false
26718 STATES DS:0 DS:1 DS:2 TOTAL
26719 Sent 0 302 0 302
26720 Done 302 0 302 604
26721 Skipped 0 0 0 0
26722 Error 0 0 0 0
26723 Last Flush: 7943 7641 7943
26724 Downstairs last five completed: 7641 7640 7639 7638 7637
26725 Upstairs last five completed: 7943 7942 7941 7940 7939
26726 CLIENT: Up:0 ds:302 act:2
267272025-01-03T08:36:29.114ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
267282025-01-03T08:36:29.114ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
267292025-01-03T08:36:29.114ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
267302025-01-03T08:36:29.114ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
267312025-01-03T08:36:29.115ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
26732 [8820] got start action
26733 [8820] I am going to restart
26734 Starting downstairs at port 8820
26735 Make output file at "/tmp/dsc/downstairs-8820.txt"
26736 [8820][1] reports Starting
26737 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1228)
26738 [8820][1] reports Running
26739 ----------------------------------------------------------------
26740 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26741 States: Active Offline Active
26742 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26743 7642 Acked Read 10 Done Sent Done false
26744 7643 Acked Write 8 Done Sent Done false
26745 7644 Acked Write 10 Done Sent Done false
26746 7645 Acked Flush 0 Done Sent Done false
26747 7646 Acked Read 3 Done Sent Done false
26748 7647 Acked Write 1 Done Sent Done false
26749 7648 Acked Flush 0 Done Sent Done false
26750 7649 Acked Read 3 Done Sent Done false
26751 7650 Acked Write 6 Done Sent Done false
26752 7651 Acked Read 9 Done Sent Done false
26753 7652 Acked Flush 0 Done Sent Done false
26754 7653 Acked Write 6 Done Sent Done false
26755 7654 Acked Read 2 Done Sent Done false
26756 7655 Acked Read 8 Done Sent Done false
26757 7656 Acked Write 9 Done Sent Done false
26758 7657 Acked Write 5 Done Sent Done false
26759 7658 Acked Write 1 Done Sent Done false
26760 7659 Acked Flush 0 Done Sent Done false
26761 7660 Acked Write 2 Done Sent Done false
26762 7661 Acked Write 1 Done Sent Done false
26763 7662 Acked Write 6 Done Sent Done false
26764 7663 Acked Write 10 Done Sent Done false
26765 7664 Acked Read 2 Done Sent Done false
26766 7665 Acked Read 5 Done Sent Done false
26767 7666 Acked Flush 0 Done Sent Done false
26768 7667 Acked Read 8 Done Sent Done false
26769 7668 Acked Write 4 Done Sent Done false
26770 7669 Acked Read 10 Done Sent Done false
26771 7670 Acked Write 7 Done Sent Done false
26772 7671 Acked Write 7 Done Sent Done false
26773 7672 Acked Read 10 Done Sent Done false
26774 7673 Acked Read 9 Done Sent Done false
26775 7674 Acked Write 4 Done Sent Done false
26776 7675 Acked Write 5 Done Sent Done false
26777 7676 Acked Write 8 Done Sent Done false
26778 7677 Acked Write 3 Done Sent Done false
26779 7678 Acked Read 8 Done Sent Done false
26780 7679 Acked Read 2 Done Sent Done false
26781 7680 Acked Read 2 Done Sent Done false
26782 7681 Acked Write 5 Done Sent Done false
26783 7682 Acked Read 3 Done Sent Done false
26784 7683 Acked Read 4 Done Sent Done false
26785 7684 Acked Read 8 Done Sent Done false
26786 7685 Acked Read 10 Done Sent Done false
26787 7686 Acked Write 1 Done Sent Done false
26788 7687 Acked Flush 0 Done Sent Done false
26789 7688 Acked Read 3 Done Sent Done false
26790 7689 Acked Flush 0 Done Sent Done false
26791 7690 Acked Write 10 Done Sent Done false
26792 7691 Acked Write 8 Done Sent Done false
26793 7692 Acked Read 10 Done Sent Done false
26794 7693 Acked Read 6 Done Sent Done false
26795 7694 Acked Read 10 Done Sent Done false
26796 7695 Acked Read 4 Done Sent Done false
26797 7696 Acked Read 6 Done Sent Done false
26798 7697 Acked Write 10 Done Sent Done false
26799 7698 Acked Read 6 Done Sent Done false
26800 7699 Acked Flush 0 Done Sent Done false
26801 7700 Acked Write 5 Done Sent Done false
26802 7701 Acked Write 2 Done Sent Done false
26803 7702 Acked Read 9 Done Sent Done false
26804 7703 Acked Read 1 Done Sent Done false
26805 7704 Acked Write 10 Done Sent Done false
26806 7705 Acked Write 5 Done Sent Done false
26807 7706 Acked Read 10 Done Sent Done false
26808 7707 Acked Write 10 Done Sent Done false
26809 7708 Acked Read 6 Done Sent Done false
26810 7709 Acked Write 3 Done Sent Done false
26811 7710 Acked Read 9 Done Sent Done false
26812 7711 Acked Read 4 Done Sent Done false
26813 7712 Acked Write 6 Done Sent Done false
26814 7713 Acked Read 8 Done Sent Done false
26815 7714 Acked Read 7 Done Sent Done false
26816 7715 Acked Read 10 Done Sent Done false
26817 7716 Acked Write 6 Done Sent Done false
26818 7717 Acked Flush 0 Done Sent Done false
26819 7718 Acked Read 4 Done Sent Done false
26820 7719 Acked Write 2 Done Sent Done false
26821 7720 Acked Write 9 Done Sent Done false
26822 7721 Acked Flush 0 Done Sent Done false
26823 7722 Acked Write 7 Done Sent Done false
26824 7723 Acked Read 4 Done Sent Done false
26825 7724 Acked Write 4 Done Sent Done false
26826 7725 Acked Read 10 Done Sent Done false
26827 7726 Acked Write 5 Done Sent Done false
26828 7727 Acked Write 3 Done Sent Done false
26829 7728 Acked Write 1 Done Sent Done false
26830 7729 Acked Write 7 Done Sent Done false
26831 7730 Acked Write 8 Done Sent Done false
26832 7731 Acked Write 10 Done Sent Done false
26833 7732 Acked Write 2 Done Sent Done false
26834 7733 Acked Read 3 Done Sent Done false
26835 7734 Acked Write 10 Done Sent Done false
26836 7735 Acked Write 3 Done Sent Done false
26837 7736 Acked Write 3 Done Sent Done false
26838 7737 Acked Read 2 Done Sent Done false
26839 7738 Acked Write 6 Done Sent Done false
26840 7739 Acked Read 8 Done Sent Done false
26841 7740 Acked Read 9 Done Sent Done false
26842 7741 Acked Write 7 Done Sent Done false
26843 7742 Acked Read 9 Done Sent Done false
26844 7743 Acked Write 8 Done Sent Done false
26845 7744 Acked Read 2 Done Sent Done false
26846 7745 Acked Read 10 Done Sent Done false
26847 7746 Acked Flush 0 Done Sent Done false
26848 7747 Acked Write 10 Done Sent Done false
26849 7748 Acked Read 10 Done Sent Done false
26850 7749 Acked Write 8 Done Sent Done false
26851 7750 Acked Flush 0 Done Sent Done false
26852 7751 Acked Write 3 Done Sent Done false
26853 7752 Acked Read 4 Done Sent Done false
26854 7753 Acked Read 5 Done Sent Done false
26855 7754 Acked Write 3 Done Sent Done false
26856 7755 Acked Write 3 Done Sent Done false
26857 7756 Acked Write 10 Done Sent Done false
26858 7757 Acked Read 5 Done Sent Done false
26859 7758 Acked Write 7 Done Sent Done false
26860 7759 Acked Read 7 Done Sent Done false
26861 7760 Acked Read 7 Done Sent Done false
26862 7761 Acked Flush 0 Done Sent Done false
26863 7762 Acked Read 9 Done Sent Done false
26864 7763 Acked Read 7 Done Sent Done false
26865 7764 Acked Write 9 Done Sent Done false
26866 7765 Acked Read 8 Done Sent Done false
26867 7766 Acked Read 7 Done Sent Done false
26868 7767 Acked Write 4 Done Sent Done false
26869 7768 Acked Read 2 Done Sent Done false
26870 7769 Acked Read 4 Done Sent Done false
26871 7770 Acked Read 6 Done Sent Done false
26872 7771 Acked Flush 0 Done Sent Done false
26873 7772 Acked Write 10 Done Sent Done false
26874 7773 Acked Read 9 Done Sent Done false
26875 7774 Acked Read 4 Done Sent Done false
26876 7775 Acked Flush 0 Done Sent Done false
26877 7776 Acked Write 8 Done Sent Done false
26878 7777 Acked Read 6 Done Sent Done false
26879 7778 Acked Read 1 Done Sent Done false
26880 7779 Acked Write 6 Done Sent Done false
26881 7780 Acked Write 6 Done Sent Done false
26882 7781 Acked Flush 0 Done Sent Done false
26883 7782 Acked Write 9 Done Sent Done false
26884 7783 Acked Read 9 Done Sent Done false
26885 7784 Acked Read 5 Done Sent Done false
26886 7785 Acked Read 1 Done Sent Done false
26887 7786 Acked Read 3 Done Sent Done false
26888 7787 Acked Write 4 Done Sent Done false
26889 7788 Acked Write 2 Done Sent Done false
26890 7789 Acked Write 2 Done Sent Done false
26891 7790 Acked Read 8 Done Sent Done false
26892 7791 Acked Write 5 Done Sent Done false
26893 7792 Acked Flush 0 Done Sent Done false
26894 7793 Acked Write 9 Done Sent Done false
26895 7794 Acked Write 4 Done Sent Done false
26896 7795 Acked Write 10 Done Sent Done false
26897 7796 Acked Read 10 Done Sent Done false
26898 7797 Acked Read 5 Done Sent Done false
26899 7798 Acked Write 7 Done Sent Done false
26900 7799 Acked Read 9 Done Sent Done false
26901 7800 Acked Write 10 Done Sent Done false
26902 7801 Acked Write 3 Done Sent Done false
26903 7802 Acked Write 2 Done Sent Done false
26904 7803 Acked Write 1 Done Sent Done false
26905 7804 Acked Read 10 Done Sent Done false
26906 7805 Acked Write 9 Done Sent Done false
26907 7806 Acked Write 9 Done Sent Done false
26908 7807 Acked Read 6 Done Sent Done false
26909 7808 Acked Write 2 Done Sent Done false
26910 7809 Acked Write 8 Done Sent Done false
26911 7810 Acked Read 10 Done Sent Done false
26912 7811 Acked Read 6 Done Sent Done false
26913 7812 Acked Read 7 Done Sent Done false
26914 7813 Acked Read 5 Done Sent Done false
26915 7814 Acked Write 3 Done Sent Done false
26916 7815 Acked Write 3 Done Sent Done false
26917 7816 Acked Write 8 Done Sent Done false
26918 7817 Acked Write 9 Done Sent Done false
26919 7818 Acked Read 3 Done Sent Done false
26920 7819 Acked Read 4 Done Sent Done false
26921 7820 Acked Read 10 Done Sent Done false
26922 7821 Acked Write 7 Done Sent Done false
26923 7822 Acked Write 7 Done Sent Done false
26924 7823 Acked Write 3 Done Sent Done false
26925 7824 Acked Read 4 Done Sent Done false
26926 7825 Acked Read 5 Done Sent Done false
26927 7826 Acked Write 10 Done Sent Done false
26928 7827 Acked Read 2 Done Sent Done false
26929 7828 Acked Write 10 Done Sent Done false
26930 7829 Acked Write 4 Done Sent Done false
26931 7830 Acked Write 8 Done Sent Done false
26932 7831 Acked Read 10 Done Sent Done false
26933 7832 Acked Write 3 Done Sent Done false
26934 7833 Acked Read 4 Done Sent Done false
26935 7834 Acked Read 3 Done Sent Done false
26936 7835 Acked Write 1 Done Sent Done false
26937 7836 Acked Read 6 Done Sent Done false
26938 7837 Acked Write 5 Done Sent Done false
26939 7838 Acked Read 10 Done Sent Done false
26940 7839 Acked Read 1 Done Sent Done false
26941 7840 Acked Read 3 Done Sent Done false
26942 7841 Acked Read 2 Done Sent Done false
26943 7842 Acked Read 3 Done Sent Done false
26944 7843 Acked Read 7 Done Sent Done false
26945 7844 Acked Write 10 Done Sent Done false
26946 7845 Acked Write 1 Done Sent Done false
26947 7846 Acked Write 5 Done Sent Done false
26948 7847 Acked Write 10 Done Sent Done false
26949 7848 Acked Read 1 Done Sent Done false
26950 7849 Acked Read 6 Done Sent Done false
26951 7850 Acked Read 1 Done Sent Done false
26952 7851 Acked Read 5 Done Sent Done false
26953 7852 Acked Read 1 Done Sent Done false
26954 7853 Acked Write 6 Done Sent Done false
26955 7854 Acked Write 6 Done Sent Done false
26956 7855 Acked Read 7 Done Sent Done false
26957 7856 Acked Flush 0 Done Sent Done false
26958 7857 Acked Write 5 Done Sent Done false
26959 7858 Acked Read 3 Done Sent Done false
26960 7859 Acked Write 5 Done Sent Done false
26961 7860 Acked Flush 0 Done Sent Done false
26962 7861 Acked Read 4 Done Sent Done false
26963 7862 Acked Read 9 Done Sent Done false
26964 7863 Acked Flush 0 Done Sent Done false
26965 7864 Acked Flush 0 Done Sent Done false
26966 7865 Acked Read 3 Done Sent Done false
26967 7866 Acked Write 1 Done Sent Done false
26968 7867 Acked Write 1 Done Sent Done false
26969 7868 Acked Write 9 Done Sent Done false
26970 7869 Acked Read 4 Done Sent Done false
26971 7870 Acked Write 1 Done Sent Done false
26972 7871 Acked Read 5 Done Sent Done false
26973 7872 Acked Flush 0 Done Sent Done false
26974 7873 Acked Read 8 Done Sent Done false
26975 7874 Acked Read 3 Done Sent Done false
26976 7875 Acked Read 1 Done Sent Done false
26977 7876 Acked Read 4 Done Sent Done false
26978 7877 Acked Write 10 Done Sent Done false
26979 7878 Acked Read 5 Done Sent Done false
26980 7879 Acked Read 2 Done Sent Done false
26981 7880 Acked Read 2 Done Sent Done false
26982 7881 Acked Flush 0 Done Sent Done false
26983 7882 Acked Read 10 Done Sent Done false
26984 7883 Acked Write 10 Done Sent Done false
26985 7884 Acked Write 8 Done Sent Done false
26986 7885 Acked Read 7 Done Sent Done false
26987 7886 Acked Read 7 Done Sent Done false
26988 7887 Acked Read 8 Done Sent Done false
26989 7888 Acked Read 9 Done Sent Done false
26990 7889 Acked Write 2 Done Sent Done false
26991 7890 Acked Write 3 Done Sent Done false
26992 7891 Acked Write 10 Done Sent Done false
26993 7892 Acked Read 7 Done Sent Done false
26994 7893 Acked Write 6 Done Sent Done false
26995 7894 Acked Flush 0 Done Sent Done false
26996 7895 Acked Read 6 Done Sent Done false
26997 7896 Acked Read 6 Done Sent Done false
26998 7897 Acked Write 5 Done Sent Done false
26999 7898 Acked Flush 0 Done Sent Done false
27000 7899 Acked Flush 0 Done Sent Done false
27001 7900 Acked Write 5 Done Sent Done false
27002 7901 Acked Read 10 Done Sent Done false
27003 7902 Acked Read 4 Done Sent Done false
27004 7903 Acked Write 7 Done Sent Done false
27005 7904 Acked Read 7 Done Sent Done false
27006 7905 Acked Write 4 Done Sent Done false
27007 7906 Acked Read 5 Done Sent Done false
27008 7907 Acked Write 9 Done Sent Done false
27009 7908 Acked Write 9 Done Sent Done false
27010 7909 Acked Read 6 Done Sent Done false
27011 7910 Acked Write 7 Done Sent Done false
27012 7911 Acked Write 3 Done Sent Done false
27013 7912 Acked Write 5 Done Sent Done false
27014 7913 Acked Read 6 Done Sent Done false
27015 7914 Acked Read 1 Done Sent Done false
27016 7915 Acked Read 7 Done Sent Done false
27017 7916 Acked Write 10 Done Sent Done false
27018 7917 Acked Read 5 Done Sent Done false
27019 7918 Acked Read 6 Done Sent Done false
27020 7919 Acked Write 3 Done Sent Done false
27021 7920 Acked Read 7 Done Sent Done false
27022 7921 Acked Read 7 Done Sent Done false
27023 7922 Acked Write 4 Done Sent Done false
27024 7923 Acked Read 7 Done Sent Done false
27025 7924 Acked Write 8 Done Sent Done false
27026 7925 Acked Read 8 Done Sent Done false
27027 7926 Acked Read 4 Done Sent Done false
27028 7927 Acked Read 8 Done Sent Done false
27029 7928 Acked Read 1 Done Sent Done false
27030 7929 Acked Write 4 Done Sent Done false
27031 7930 Acked Read 7 Done Sent Done false
27032 7931 Acked Flush 0 Done Sent Done false
27033 7932 Acked Write 3 Done Sent Done false
27034 7933 Acked Read 3 Done Sent Done false
27035 7934 Acked Read 5 Done Sent Done false
27036 7935 Acked Write 5 Done Sent Done false
27037 7936 Acked Read 5 Done Sent Done false
27038 7937 Acked Write 9 Done Sent Done false
27039 7938 Acked Read 2 Done Sent Done false
27040 7939 Acked Write 9 Done Sent Done false
27041 7940 Acked Write 10 Done Sent Done false
27042 7941 Acked Read 10 Done Sent Done false
27043 7942 Acked Read 1 Done Sent Done false
27044 7943 Acked Flush 0 Done Sent Done false
27045 STATES DS:0 DS:1 DS:2 TOTAL
27046 Sent 0 302 0 302
27047 Done 302 0 302 604
27048 Skipped 0 0 0 0
27049 Error 0 0 0 0
27050 Last Flush: 7943 7641 7943
27051 Downstairs last five completed: 7641 7640 7639 7638 7637
27052 Upstairs last five completed: 7943 7942 7941 7940 7939
27053 CLIENT: Up:0 ds:302 act:2
27054 ----------------------------------------------------------------
27055 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27056 States: Active Offline Active
27057 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27058 7642 Acked Read 10 Done Sent Done false
27059 7643 Acked Write 8 Done Sent Done false
27060 7644 Acked Write 10 Done Sent Done false
27061 7645 Acked Flush 0 Done Sent Done false
27062 7646 Acked Read 3 Done Sent Done false
27063 7647 Acked Write 1 Done Sent Done false
27064 7648 Acked Flush 0 Done Sent Done false
27065 7649 Acked Read 3 Done Sent Done false
27066 7650 Acked Write 6 Done Sent Done false
27067 7651 Acked Read 9 Done Sent Done false
27068 7652 Acked Flush 0 Done Sent Done false
27069 7653 Acked Write 6 Done Sent Done false
27070 7654 Acked Read 2 Done Sent Done false
27071 7655 Acked Read 8 Done Sent Done false
27072 7656 Acked Write 9 Done Sent Done false
27073 7657 Acked Write 5 Done Sent Done false
27074 7658 Acked Write 1 Done Sent Done false
27075 7659 Acked Flush 0 Done Sent Done false
27076 7660 Acked Write 2 Done Sent Done false
27077 7661 Acked Write 1 Done Sent Done false
27078 7662 Acked Write 6 Done Sent Done false
27079 7663 Acked Write 10 Done Sent Done false
27080 7664 Acked Read 2 Done Sent Done false
27081 7665 Acked Read 5 Done Sent Done false
27082 7666 Acked Flush 0 Done Sent Done false
27083 7667 Acked Read 8 Done Sent Done false
27084 7668 Acked Write 4 Done Sent Done false
27085 7669 Acked Read 10 Done Sent Done false
27086 7670 Acked Write 7 Done Sent Done false
27087 7671 Acked Write 7 Done Sent Done false
27088 7672 Acked Read 10 Done Sent Done false
27089 7673 Acked Read 9 Done Sent Done false
27090 7674 Acked Write 4 Done Sent Done false
27091 7675 Acked Write 5 Done Sent Done false
27092 7676 Acked Write 8 Done Sent Done false
27093 7677 Acked Write 3 Done Sent Done false
27094 7678 Acked Read 8 Done Sent Done false
27095 7679 Acked Read 2 Done Sent Done false
27096 7680 Acked Read 2 Done Sent Done false
27097 7681 Acked Write 5 Done Sent Done false
27098 7682 Acked Read 3 Done Sent Done false
27099 7683 Acked Read 4 Done Sent Done false
27100 7684 Acked Read 8 Done Sent Done false
27101 7685 Acked Read 10 Done Sent Done false
27102 7686 Acked Write 1 Done Sent Done false
27103 7687 Acked Flush 0 Done Sent Done false
27104 7688 Acked Read 3 Done Sent Done false
27105 7689 Acked Flush 0 Done Sent Done false
27106 7690 Acked Write 10 Done Sent Done false
27107 7691 Acked Write 8 Done Sent Done false
27108 7692 Acked Read 10 Done Sent Done false
27109 7693 Acked Read 6 Done Sent Done false
27110 7694 Acked Read 10 Done Sent Done false
27111 7695 Acked Read 4 Done Sent Done false
27112 7696 Acked Read 6 Done Sent Done false
27113 7697 Acked Write 10 Done Sent Done false
27114 7698 Acked Read 6 Done Sent Done false
27115 7699 Acked Flush 0 Done Sent Done false
27116 7700 Acked Write 5 Done Sent Done false
27117 7701 Acked Write 2 Done Sent Done false
27118 7702 Acked Read 9 Done Sent Done false
27119 7703 Acked Read 1 Done Sent Done false
27120 7704 Acked Write 10 Done Sent Done false
27121 7705 Acked Write 5 Done Sent Done false
27122 7706 Acked Read 10 Done Sent Done false
27123 7707 Acked Write 10 Done Sent Done false
27124 7708 Acked Read 6 Done Sent Done false
27125 7709 Acked Write 3 Done Sent Done false
27126 7710 Acked Read 9 Done Sent Done false
27127 7711 Acked Read 4 Done Sent Done false
27128 7712 Acked Write 6 Done Sent Done false
27129 7713 Acked Read 8 Done Sent Done false
27130 7714 Acked Read 7 Done Sent Done false
27131 7715 Acked Read 10 Done Sent Done false
27132 7716 Acked Write 6 Done Sent Done false
27133 7717 Acked Flush 0 Done Sent Done false
27134 7718 Acked Read 4 Done Sent Done false
27135 7719 Acked Write 2 Done Sent Done false
27136 7720 Acked Write 9 Done Sent Done false
27137 7721 Acked Flush 0 Done Sent Done false
27138 7722 Acked Write 7 Done Sent Done false
27139 7723 Acked Read 4 Done Sent Done false
27140 7724 Acked Write 4 Done Sent Done false
27141 7725 Acked Read 10 Done Sent Done false
27142 7726 Acked Write 5 Done Sent Done false
27143 7727 Acked Write 3 Done Sent Done false
27144 7728 Acked Write 1 Done Sent Done false
27145 7729 Acked Write 7 Done Sent Done false
27146 7730 Acked Write 8 Done Sent Done false
27147 7731 Acked Write 10 Done Sent Done false
27148 7732 Acked Write 2 Done Sent Done false
27149 7733 Acked Read 3 Done Sent Done false
27150 7734 Acked Write 10 Done Sent Done false
27151 7735 Acked Write 3 Done Sent Done false
27152 7736 Acked Write 3 Done Sent Done false
27153 7737 Acked Read 2 Done Sent Done false
27154 7738 Acked Write 6 Done Sent Done false
27155 7739 Acked Read 8 Done Sent Done false
27156 7740 Acked Read 9 Done Sent Done false
27157 7741 Acked Write 7 Done Sent Done false
27158 7742 Acked Read 9 Done Sent Done false
27159 7743 Acked Write 8 Done Sent Done false
27160 7744 Acked Read 2 Done Sent Done false
27161 7745 Acked Read 10 Done Sent Done false
27162 7746 Acked Flush 0 Done Sent Done false
27163 7747 Acked Write 10 Done Sent Done false
27164 7748 Acked Read 10 Done Sent Done false
27165 7749 Acked Write 8 Done Sent Done false
27166 7750 Acked Flush 0 Done Sent Done false
27167 7751 Acked Write 3 Done Sent Done false
27168 7752 Acked Read 4 Done Sent Done false
27169 7753 Acked Read 5 Done Sent Done false
27170 7754 Acked Write 3 Done Sent Done false
27171 7755 Acked Write 3 Done Sent Done false
27172 7756 Acked Write 10 Done Sent Done false
27173 7757 Acked Read 5 Done Sent Done false
27174 7758 Acked Write 7 Done Sent Done false
27175 7759 Acked Read 7 Done Sent Done false
27176 7760 Acked Read 7 Done Sent Done false
27177 7761 Acked Flush 0 Done Sent Done false
27178 7762 Acked Read 9 Done Sent Done false
27179 7763 Acked Read 7 Done Sent Done false
27180 7764 Acked Write 9 Done Sent Done false
27181 7765 Acked Read 8 Done Sent Done false
27182 7766 Acked Read 7 Done Sent Done false
27183 7767 Acked Write 4 Done Sent Done false
27184 7768 Acked Read 2 Done Sent Done false
27185 7769 Acked Read 4 Done Sent Done false
27186 7770 Acked Read 6 Done Sent Done false
27187 7771 Acked Flush 0 Done Sent Done false
27188 7772 Acked Write 10 Done Sent Done false
27189 7773 Acked Read 9 Done Sent Done false
27190 7774 Acked Read 4 Done Sent Done false
27191 7775 Acked Flush 0 Done Sent Done false
27192 7776 Acked Write 8 Done Sent Done false
27193 7777 Acked Read 6 Done Sent Done false
27194 7778 Acked Read 1 Done Sent Done false
27195 7779 Acked Write 6 Done Sent Done false
27196 7780 Acked Write 6 Done Sent Done false
27197 7781 Acked Flush 0 Done Sent Done false
27198 7782 Acked Write 9 Done Sent Done false
27199 7783 Acked Read 9 Done Sent Done false
27200 7784 Acked Read 5 Done Sent Done false
27201 7785 Acked Read 1 Done Sent Done false
27202 7786 Acked Read 3 Done Sent Done false
27203 7787 Acked Write 4 Done Sent Done false
27204 7788 Acked Write 2 Done Sent Done false
27205 7789 Acked Write 2 Done Sent Done false
27206 7790 Acked Read 8 Done Sent Done false
27207 7791 Acked Write 5 Done Sent Done false
27208 7792 Acked Flush 0 Done Sent Done false
27209 7793 Acked Write 9 Done Sent Done false
27210 7794 Acked Write 4 Done Sent Done false
27211 7795 Acked Write 10 Done Sent Done false
27212 7796 Acked Read 10 Done Sent Done false
27213 7797 Acked Read 5 Done Sent Done false
27214 7798 Acked Write 7 Done Sent Done false
27215 7799 Acked Read 9 Done Sent Done false
27216 7800 Acked Write 10 Done Sent Done false
27217 7801 Acked Write 3 Done Sent Done false
27218 7802 Acked Write 2 Done Sent Done false
27219 7803 Acked Write 1 Done Sent Done false
27220 7804 Acked Read 10 Done Sent Done false
27221 7805 Acked Write 9 Done Sent Done false
27222 7806 Acked Write 9 Done Sent Done false
27223 7807 Acked Read 6 Done Sent Done false
27224 7808 Acked Write 2 Done Sent Done false
27225 7809 Acked Write 8 Done Sent Done false
27226 7810 Acked Read 10 Done Sent Done false
27227 7811 Acked Read 6 Done Sent Done false
27228 7812 Acked Read 7 Done Sent Done false
27229 7813 Acked Read 5 Done Sent Done false
27230 7814 Acked Write 3 Done Sent Done false
27231 7815 Acked Write 3 Done Sent Done false
27232 7816 Acked Write 8 Done Sent Done false
27233 7817 Acked Write 9 Done Sent Done false
27234 7818 Acked Read 3 Done Sent Done false
27235 7819 Acked Read 4 Done Sent Done false
27236 7820 Acked Read 10 Done Sent Done false
27237 7821 Acked Write 7 Done Sent Done false
27238 7822 Acked Write 7 Done Sent Done false
27239 7823 Acked Write 3 Done Sent Done false
27240 7824 Acked Read 4 Done Sent Done false
27241 7825 Acked Read 5 Done Sent Done false
27242 7826 Acked Write 10 Done Sent Done false
27243 7827 Acked Read 2 Done Sent Done false
27244 7828 Acked Write 10 Done Sent Done false
27245 7829 Acked Write 4 Done Sent Done false
27246 7830 Acked Write 8 Done Sent Done false
27247 7831 Acked Read 10 Done Sent Done false
27248 7832 Acked Write 3 Done Sent Done false
27249 7833 Acked Read 4 Done Sent Done false
27250 7834 Acked Read 3 Done Sent Done false
27251 7835 Acked Write 1 Done Sent Done false
27252 7836 Acked Read 6 Done Sent Done false
27253 7837 Acked Write 5 Done Sent Done false
27254 7838 Acked Read 10 Done Sent Done false
27255 7839 Acked Read 1 Done Sent Done false
27256 7840 Acked Read 3 Done Sent Done false
27257 7841 Acked Read 2 Done Sent Done false
27258 7842 Acked Read 3 Done Sent Done false
27259 7843 Acked Read 7 Done Sent Done false
27260 7844 Acked Write 10 Done Sent Done false
27261 7845 Acked Write 1 Done Sent Done false
27262 7846 Acked Write 5 Done Sent Done false
27263 7847 Acked Write 10 Done Sent Done false
27264 7848 Acked Read 1 Done Sent Done false
27265 7849 Acked Read 6 Done Sent Done false
27266 7850 Acked Read 1 Done Sent Done false
27267 7851 Acked Read 5 Done Sent Done false
27268 7852 Acked Read 1 Done Sent Done false
27269 7853 Acked Write 6 Done Sent Done false
27270 7854 Acked Write 6 Done Sent Done false
27271 7855 Acked Read 7 Done Sent Done false
27272 7856 Acked Flush 0 Done Sent Done false
27273 7857 Acked Write 5 Done Sent Done false
27274 7858 Acked Read 3 Done Sent Done false
27275 7859 Acked Write 5 Done Sent Done false
27276 7860 Acked Flush 0 Done Sent Done false
27277 7861 Acked Read 4 Done Sent Done false
27278 7862 Acked Read 9 Done Sent Done false
27279 7863 Acked Flush 0 Done Sent Done false
27280 7864 Acked Flush 0 Done Sent Done false
27281 7865 Acked Read 3 Done Sent Done false
27282 7866 Acked Write 1 Done Sent Done false
27283 7867 Acked Write 1 Done Sent Done false
27284 7868 Acked Write 9 Done Sent Done false
27285 7869 Acked Read 4 Done Sent Done false
27286 7870 Acked Write 1 Done Sent Done false
27287 7871 Acked Read 5 Done Sent Done false
27288 7872 Acked Flush 0 Done Sent Done false
27289 7873 Acked Read 8 Done Sent Done false
27290 7874 Acked Read 3 Done Sent Done false
27291 7875 Acked Read 1 Done Sent Done false
27292 7876 Acked Read 4 Done Sent Done false
27293 7877 Acked Write 10 Done Sent Done false
27294 7878 Acked Read 5 Done Sent Done false
27295 7879 Acked Read 2 Done Sent Done false
27296 7880 Acked Read 2 Done Sent Done false
27297 7881 Acked Flush 0 Done Sent Done false
27298 7882 Acked Read 10 Done Sent Done false
27299 7883 Acked Write 10 Done Sent Done false
27300 7884 Acked Write 8 Done Sent Done false
27301 7885 Acked Read 7 Done Sent Done false
27302 7886 Acked Read 7 Done Sent Done false
27303 7887 Acked Read 8 Done Sent Done false
27304 7888 Acked Read 9 Done Sent Done false
27305 7889 Acked Write 2 Done Sent Done false
27306 7890 Acked Write 3 Done Sent Done false
27307 7891 Acked Write 10 Done Sent Done false
27308 7892 Acked Read 7 Done Sent Done false
27309 7893 Acked Write 6 Done Sent Done false
27310 7894 Acked Flush 0 Done Sent Done false
27311 7895 Acked Read 6 Done Sent Done false
27312 7896 Acked Read 6 Done Sent Done false
27313 7897 Acked Write 5 Done Sent Done false
27314 7898 Acked Flush 0 Done Sent Done false
27315 7899 Acked Flush 0 Done Sent Done false
27316 7900 Acked Write 5 Done Sent Done false
27317 7901 Acked Read 10 Done Sent Done false
27318 7902 Acked Read 4 Done Sent Done false
27319 7903 Acked Write 7 Done Sent Done false
27320 7904 Acked Read 7 Done Sent Done false
27321 7905 Acked Write 4 Done Sent Done false
27322 7906 Acked Read 5 Done Sent Done false
27323 7907 Acked Write 9 Done Sent Done false
27324 7908 Acked Write 9 Done Sent Done false
27325 7909 Acked Read 6 Done Sent Done false
27326 7910 Acked Write 7 Done Sent Done false
27327 7911 Acked Write 3 Done Sent Done false
27328 7912 Acked Write 5 Done Sent Done false
27329 7913 Acked Read 6 Done Sent Done false
27330 7914 Acked Read 1 Done Sent Done false
27331 7915 Acked Read 7 Done Sent Done false
27332 7916 Acked Write 10 Done Sent Done false
27333 7917 Acked Read 5 Done Sent Done false
27334 7918 Acked Read 6 Done Sent Done false
27335 7919 Acked Write 3 Done Sent Done false
27336 7920 Acked Read 7 Done Sent Done false
27337 7921 Acked Read 7 Done Sent Done false
27338 7922 Acked Write 4 Done Sent Done false
27339 7923 Acked Read 7 Done Sent Done false
27340 7924 Acked Write 8 Done Sent Done false
27341 7925 Acked Read 8 Done Sent Done false
27342 7926 Acked Read 4 Done Sent Done false
27343 7927 Acked Read 8 Done Sent Done false
27344 7928 Acked Read 1 Done Sent Done false
27345 7929 Acked Write 4 Done Sent Done false
27346 7930 Acked Read 7 Done Sent Done false
27347 7931 Acked Flush 0 Done Sent Done false
27348 7932 Acked Write 3 Done Sent Done false
27349 7933 Acked Read 3 Done Sent Done false
27350 7934 Acked Read 5 Done Sent Done false
27351 7935 Acked Write 5 Done Sent Done false
27352 7936 Acked Read 5 Done Sent Done false
27353 7937 Acked Write 9 Done Sent Done false
27354 7938 Acked Read 2 Done Sent Done false
27355 7939 Acked Write 9 Done Sent Done false
27356 7940 Acked Write 10 Done Sent Done false
27357 7941 Acked Read 10 Done Sent Done false
27358 7942 Acked Read 1 Done Sent Done false
27359 7943 Acked Flush 0 Done Sent Done false
27360 STATES DS:0 DS:1 DS:2 TOTAL
27361 Sent 0 302 0 302
27362 Done 302 0 302 604
27363 Skipped 0 0 0 0
27364 Error 0 0 0 0
27365 Last Flush: 7943 7641 7943
27366 Downstairs last five completed: 7641 7640 7639 7638 7637
27367 Upstairs last five completed: 7943 7942 7941 7940 7939
27368 CLIENT: Up:0 ds:302 act:2
273692025-01-03T08:36:39.116ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273702025-01-03T08:36:39.117ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63353) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273712025-01-03T08:36:39.118ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273722025-01-03T08:36:39.118ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273732025-01-03T08:36:39.118ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273742025-01-03T08:36:39.118ZINFOcrucible: send last flush ID to this DS: 7641 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273752025-01-03T08:36:39.119ZINFOcrucible: Replied this last flush ID: 7641 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273762025-01-03T08:36:39.119ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273772025-01-03T08:36:39.119ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273782025-01-03T08:36:39.119ZINFOcrucible: [1] client re-new 302 jobs since flush 7641 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273792025-01-03T08:36:39.119ZINFOcrucible: [1] Replayed 302 jobs since flush: 7641 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
27380 ----------------------------------------------------------------
27381 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27382 Downstairs last five completed: 7943 7942 7941 7940 7939
27383 Upstairs last five completed: 7943 7942 7941 7940 7939
27384 CLIENT: Up:0 ds:0 act:3
27385 Replay: All jobs finished
27386 Main task has work to do, go find it
27387 got dsc Stop(2)
27388 stop 2
27389 Jan 03 08:36:39.372 INFO request completed, latency_us: 346, response_code: 204, uri: /stop/cid/2, method: POST, req_id: aebde23b-bb93-4e95-b5dc-0bb7bdc09993, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
27390 [8830] Got stop action so:false kr:false
273912025-01-03T08:36:39.373ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273922025-01-03T08:36:39.373ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
273932025-01-03T08:36:39.373ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
27394 Jan 03 08:36:39.373 INFO request completed, latency_us: 209, {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:36:39.37343529Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
27395 response_code: 200, uri: /state/cid/2, method: GET, req_id: 8c40397d-0121-43ed-ae56-771ba74b1a9f, remote_addr: {"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:36:39.373483629Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
27396 127.0.0.1:49669, local_addr: 127.0.0{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:36:39.373508259Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
27397 .1:9998
27398 [8830] Exited with: ExitStatus(unix_wait_status(9))
27399 [8830][2] reports Exit
27400 Jan 03 08:36:43.375 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/2, method: GET, req_id: d6c1ede3-232c-4865-a304-954a2251c69b, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
27401 001/300 Read block 3829 len 4096
27402 002/300 Flush
27403 003/300 Flush
27404 004/300 Read block 774 len 20480
27405 005/300 Read block 3466 len 8192
27406 006/300 Read block 4040 len 16384
27407 007/300 Write block 3487 len 20480 data: 4 4 4 4 5
27408 008/300 Write block 1186 len 24576 data: 3 4 4 4 5 5
27409 009/300 Write block 2227 len 4096 data: 7
27410 010/300 Read block 3552 len 40960
27411 011/300 Write block 525 len 32768 data: 4 4 5 4 5 5 5 3
27412 012/300 Write block 1317 len 4096 data: 6
27413 013/300 Read block 1 len 24576
27414 014/300 Read block 2148 len 20480
27415 015/300 Read block 4868 len 4096
27416 016/300 Write block 3849 len 24576 data: 4 6 6 8 9 8
27417 017/300 Write block 306 len 8192 data: 6 7
27418 018/300 Read block 1116 len 8192
27419 019/300 Read block 3408 len 12288
27420 020/300 Read block 3678 len 32768
27421 021/300 Read block 787 len 32768
27422 022/300 Read block 1721 len 16384
27423 023/300 Read block 4003 len 28672
27424 024/300 Write block 3455 len 32768 data: 7 8 7 7 7 6 4 2
27425 025/300 Read block 4991 len 16384
27426 026/300 Read block 62 len 12288
27427 027/300 Flush
27428 028/300 Read block 4119 len 20480
27429 029/300 Write block 3361 len 40960 data: 9 8 8 7 5 6 8 7 5 6
27430 030/300 Write block 468 len 40960 data: 6 5 6 6 4 5 5 5 5 5
27431 031/300 Flush
27432 032/300 Read block 3006 len 12288
27433 033/300 Read block 3827 len 16384
27434 034/300 Read block 4545 len 40960
27435 035/300 Read block 1251 len 20480
27436 036/300 Read block 4722 len 40960
27437 037/300 Write block 4445 len 24576 data: 6 7 7 9 5 5
27438 038/300 Write block 4297 len 36864 data: 6 5 5 5 4 3 4 5 5
27439 039/300 Write block 3008 len 28672 data: 3 3 2 2 3 3 3
27440 040/300 Write block 2809 len 24576 data: 7 7 7 7 8 8
27441 041/300 Read block 904 len 12288
27442 042/300 Flush
27443 043/300 Write block 381 len 32768 data: 4 3 5 5 5 4 4 4
27444 044/300 Read block 2962 len 24576
27445 045/300 Write block 785 len 28672 data: 7 7 6 6 6 9 7
27446 046/300 Write block 1857 len 24576 data: 2 2 2 2 2 2
27447 047/300 Read block 613 len 8192
27448 048/300 Write block 2263 len 8192 data: 5 4
27449 049/300 Write block 3683 len 32768 data: 4 4 4 3 3 3 3 4
27450 050/300 Write block 522 len 20480 data: 7 7 7 5 5
27451 051/300 Write block 4805 len 20480 data: 5 5 6 2 3
27452 052/300 Flush
27453 053/300 Read block 1955 len 12288
27454 054/300 Read block 1259 len 36864
27455 055/300 Flush
27456 056/300 Read block 3658 len 12288
27457 057/300 Flush
27458 058/300 Write block 129 len 32768 data: 3 4 4 5 4 4 4 4
27459 059/300 Read block 2130 len 28672
27460 060/300 Write block 3669 len 28672 data: 4 3 5 7 7 7 8
27461 061/300 Write block 707 len 28672 data: 2 5 7 8 8 7 7
27462 062/300 Read block 614 len 8192
27463 063/300 Flush
27464 064/300 Read block 4273 len 16384
27465 065/300 Flush
27466 066/300 Write block 2166 len 4096 data: 5
27467 067/300 Read block 487 len 28672
27468 068/300 Read block 801 len 16384
27469 069/300 Read block 900 len 20480
27470 070/300 Read block 2653 len 4096
27471 071/300 Read block 4353 len 20480
27472 072/300 Read block 4284 len 36864
27473 073/300 Write block 4793 len 28672 data: 4 4 4 4 3 2 2
27474 074/300 Write block 2793 len 12288 data: 3 3 5
27475 075/300 Write block 1245 len 28672 data: 5 5 5 7 7 6 6
27476 076/300 Read block 3583 len 36864
27477 077/300 Write block 4362 len 24576 data: 6 6 5 7 7 6
27478 078/300 Read block 2831 len 28672
27479 079/300 Read block 3991 len 4096
27480 080/300 Read block 601 len 8192
27481 081/300 Write block 2838 len 12288 data: 4 6 7
27482 082/300 Flush
27483 083/300 Flush
27484 084/300 Write block 3167 len 20480 data: 5 5 5 3 3
27485 085/300 Read block 4896 len 16384
27486 086/300 Write block 3194 len 4096 data: 7
27487 087/300 Write block 144 len 16384 data: 5 3 2 3
27488 088/300 Read block 1393 len 16384
27489 089/300 Write block 4094 len 16384 data: 3 4 6 5
27490 090/300 Read block 1475 len 32768
27491 091/300 Read block 2921 len 36864
27492 092/300 Read block 3325 len 32768
27493 093/300 Write block 3765 len 16384 data: 5 6 7 7
27494 094/300 Read block 2601 len 28672
27495 095/300 Write block 4439 len 40960 data: 6 6 8 6 5 5 7 8 8 10
27496 096/300 Write block 3763 len 24576 data: 6 6 6 7 8 8
27497 097/300 Flush
27498 098/300 Read block 4987 len 12288
27499 099/300 Read block 4303 len 24576
27500 100/300 Read block 477 len 32768
27501 101/300 Write block 75 len 28672 data: 5 5 4 5 4 5 5
27502 102/300 Write block 1836 len 4096 data: 4
27503 103/300 Write block 4741 len 36864 data: 3 2 3 4 4 4 5 7 7
27504 104/300 Read block 2392 len 32768
27505 105/300 Write block 2692 len 4096 data: 5
27506 106/300 Read block 3505 len 28672
27507 107/300 Read block 4545 len 16384
27508 108/300 Read block 2639 len 20480
27509 109/300 Read block 4277 len 8192
27510 110/300 Flush
27511 111/300 Write block 2202 len 40960 data: 6 6 7 7 6 6 5 5 4 6
27512 112/300 Flush
27513 113/300 Write block 2864 len 20480 data: 4 2 2 2 3
27514 114/300 Read block 1983 len 20480
27515 115/300 Read block 3458 len 16384
27516 116/300 Write block 1090 len 12288 data: 7 8 7
27517 117/300 Flush
27518 118/300 Flush
27519 119/300 Write block 3400 len 20480 data: 8 9 9 8 6
27520 120/300 Flush
27521 121/300 Read block 1357 len 8192
27522 122/300 Read block 3540 len 24576
27523 123/300 Read block 3541 len 4096
27524 124/300 Write block 1414 len 24576 data: 5 4 4 4 5 7
27525 125/300 Read block 4356 len 36864
27526 126/300 Write block 2630 len 12288 data: 4 6 6
27527 127/300 Read block 4053 len 32768
27528 128/300 Read block 1926 len 28672
27529 129/300 Read block 2123 len 20480
27530 130/300 Read block 4485 len 12288
27531 131/300 Read block 476 len 36864
27532 132/300 Read block 4039 len 4096
27533 133/300 Flush
27534 134/300 Read block 3524 len 40960
27535 135/300 Flush
27536 136/300 Flush
27537 137/300 Read block 1117 len 24576
27538 138/300 Flush
27539 139/300 Flush
27540 140/300 Write block 1522 len 32768 data: 6 9 9 10 9 5 4 5
27541 141/300 Write block 3206 len 24576 data: 5 5 5 5 4 5
27542 142/300 Write block 1680 len 8192 data: 5 5
27543 143/300 Read block 4157 len 24576
27544 144/300 Read block 3176 len 36864
27545 145/300 Read block 4812 len 8192
27546 146/300 Read block 1503 len 4096
27547 147/300 Read block 104 len 12288
27548 148/300 Write block 4918 len 16384 data: 8 9 9 9
27549 149/300 Read block 4704 len 12288
27550 150/300 Read block 755 len 4096
27551 151/300 Read block 1619 len 16384
27552 152/300 Read block 2798 len 40960
27553 153/300 Read block 1540 len 24576
27554 154/300 Write block 3060 len 8192 data: 4 3
27555 155/300 Read block 3793 len 32768
27556 156/300 Read block 4796 len 8192
27557 157/300 Write block 1653 len 20480 data: 6 5 5 3 4
27558 158/300 Read block 4386 len 32768
27559 159/300 Read block 815 len 32768
27560 160/300 Read block 1113 len 36864
27561 161/300 Read block 1979 len 12288
27562 162/300 Read block 178 len 4096
27563 163/300 Flush
27564 164/300 Write block 1570 len 16384 data: 5 5 6 7
27565 165/300 Write block 2033 len 16384 data: 4 4 5 5
27566 166/300 Read block 4811 len 20480
27567 167/300 Read block 1445 len 8192
27568 168/300 Read block 3644 len 16384
27569 169/300 Read block 2576 len 12288
27570 170/300 Write block 3007 len 40960 data: 3 4 4 3 3 4 4 4 5 6
27571 171/300 Read block 450 len 28672
27572 172/300 Flush
27573 173/300 Flush
27574 174/300 Read block 3703 len 4096
27575 175/300 Read block 1900 len 8192
27576 176/300 Read block 4112 len 20480
27577 177/300 Read block 1680 len 8192
27578 178/300 Read block 2290 len 28672
27579 179/300 Read block 1767 len 12288
27580 180/300 Read block 2501 len 12288
27581 181/300 Read block 4592 len 4096
27582 182/300 Read block 449 len 36864
27583 183/300 Read block 3295 len 4096
27584 184/300 Read block 2191 len 12288
27585 185/300 Read block 2287 len 16384
27586 186/300 Read block 2226 len 36864
27587 187/300 Read block 4753 len 16384
27588 188/300 Read block 2426 len 40960
27589 189/300 Read block 3350 len 16384
27590 190/300 Read block 1843 len 20480
27591 191/300 Flush
27592 192/300 Read block 3767 len 16384
27593 193/300 Write block 867 len 24576 data: 5 5 5 5 3 5
27594 194/300 Write block 119 len 24576 data: 3 3 2 2 3 3
27595 195/300 Read block 711 len 8192
27596 196/300 Read block 2318 len 16384
27597 197/300 Read block 4157 len 12288
27598 198/300 Write block 4138 len 8192 data: 6 4
27599 199/300 Read block 2821 len 4096
27600 200/300 Flush
27601 201/300 Read block 850 len 20480
27602 202/300 Write block 493 len 32768 data: 4 5 4 5 8 6 5 5
27603 203/300 Read block 181 len 24576
27604 204/300 Write block 1191 len 24576 data: 6 5 5 5 5 4
27605 205/300 Read block 2035 len 4096
27606 206/300 Read block 778 len 8192
27607 207/300 Read block 3970 len 36864
27608 208/300 Write block 1761 len 40960 data: 3 4 3 3 3 3 4 5 5 5
27609 209/300 Read block 4188 len 36864
27610 210/300 Read block 1871 len 28672
27611 211/300 Read block 3507 len 4096
27612 212/300 Write block 2857 len 20480 data: 7 6 6 6 6
27613 213/300 Write block 3769 len 40960 data: 7 6 7 8 7 6 5 5 4 3
27614 214/300 Write block 633 len 12288 data: 6 5 6
27615 215/300 Read block 3329 len 40960
27616 216/300 Read block 1724 len 16384
27617 217/300 Read block 1074 len 36864
27618 218/300 Flush
27619 219/300 Write block 3968 len 36864 data: 5 5 4 3 3 5 4 4 5
27620 220/300 Write block 3736 len 40960 data: 4 3 4 3 2 2 3 5 4 4
27621 221/300 Read block 739 len 24576
27622 222/300 Read block 1328 len 4096
27623 223/300 Flush
27624 224/300 Write block 4855 len 40960 data: 5 4 5 7 5 5 5 8 8 6
27625 225/300 Read block 3536 len 32768
27626 226/300 Read block 2051 len 28672
27627 227/300 Write block 2294 len 8192 data: 3 4
27628 228/300 Write block 2280 len 4096 data: 3
27629 229/300 Read block 408 len 36864
27630 230/300 Read block 4173 len 20480
27631 231/300 Read block 949 len 16384
27632 232/300 Read block 2258 len 32768
27633 233/300 Read block 3937 len 28672
27634 234/300 Write block 2123 len 24576 data: 5 6 6 6 6 6
27635 235/300 Read block 4271 len 32768
27636 236/300 Read block 1782 len 16384
27637 237/300 Write block 3324 len 32768 data: 3 3 3 5 6 6 6 7
27638 238/300 Write block 4644 len 28672 data: 5 6 6 6 8 9 5
27639 239/300 Write block 1896 len 32768 data: 6 5 5 5 3 3 3 4
27640 240/300 Read block 4892 len 12288
27641 241/300 Read block 951 len 40960
27642 242/300 Read block 4083 len 40960
27643 243/300 Read block 2989 len 12288
27644 244/300 Read block 623 len 40960
27645 245/300 Flush
27646 246/300 Write block 283 len 24576 data: 4 4 4 4 4 4
27647 247/300 Write block 2836 len 24576 data: 6 6 5 7 8 7
27648 248/300 Write block 2717 len 40960 data: 7 6 7 8 9 9 8 7 7 6
27649 249/300 Write block 4486 len 20480 data: 5 5 5 4 4
27650 250/300 Write block 3253 len 16384 data: 4 3 5 5
27651 251/300 Write block 1761 len 40960 data: 4 5 4 4 4 4 5 6 6 6
27652 252/300 Write block 2057 len 24576 data: 7 5 5 6 5 5
27653 253/300 Write block 4660 len 12288 data: 3 5 6
27654 254/300 Write block 3037 len 36864 data: 3 4 5 5 4 5 7 8 9
27655 255/300 Write block 596 len 4096 data: 4
27656 256/300 Read block 3837 len 12288
27657 257/300 Read block 967 len 40960
27658 258/300 Read block 4435 len 16384
27659 259/300 Write block 3976 len 32768 data: 6 6 6 6 5 3 2 4
27660 260/300 Read block 774 len 20480
27661 261/300 Read block 940 len 40960
27662 262/300 Flush
27663 263/300 Write block 2316 len 24576 data: 6 7 6 5 5 5
27664 264/300 Read block 929 len 28672
27665 265/300 Read block 1408 len 24576
27666 266/300 Read block 4906 len 20480
27667 267/300 Write block 4455 len 24576 data: 2 3 4 4 3 2
27668 268/300 Read block 2924 len 8192
27669 269/300 Read block 2505 len 32768
27670 270/300 Flush
27671 271/300 Read block 2794 len 4096
27672 272/300 Write block 2202 len 12288 data: 7 7 8
27673 273/300 Read block 700 len 12288
27674 274/300 Read block 464 len 8192
27675 275/300 Write block 2205 len 20480 data: 8 7 7 6 6
27676 276/300 Write block 2400 len 28672 data: 4 3 4 4 4 4 4
27677 277/300 Write block 3671 len 36864 data: 6 8 8 8 9 7 5 4 2
27678 278/300 Write block 608 len 28672 data: 4 4 4 4 6 6 7
27679 279/300 Write block 791 len 8192 data: 8 6
27680 280/300 Write block 412 len 20480 data: 5 6 4 4 5
27681 281/300 Read block 808 len 16384
27682 282/300 Write block 2690 len 4096 data: 7
27683 283/300 Read block 2352 len 40960
27684 284/300 Read block 3820 len 40960
27685 285/300 Read block 1310 len 4096
27686 286/300 Write block 2555 len 8192 data: 4 3
27687 287/300 Write block 1830 len 12288 data: 11 9 6
27688 288/300 Read block 22 len 20480
27689 289/300 Read block 1052 len 28672
27690 290/300 Read block 4448 len 36864
27691 291/300 Write block 3452 len 16384 data: 4 5 7 8
27692 292/300 Write block 4772 len 4096 data: 4
27693 293/300 Read block 4824 len 12288
27694 294/300 Write block 2983 len 16384 data: 2 3 2 3
27695 295/300 Write block 4917 len 36864 data: 6 9 10 10 10 7 6 3 3
27696 296/300 Flush
27697 297/300 Write block 772 len 12288 data: 4 4 6
27698 298/300 Flush
27699 299/300 Flush
27700 300/300 Read block 1369 len 8192
27701 301/300 Write block 4311 len 20480 data: 6 7 8 7 6
27702 Main task has work to do, go find it
27703 got dsc Start(2)
27704 start 2
27705 Jan 03 08:36:43.612 INFO request completed, latency_us: 243, response_code: 204, uri: /start/cid/2, method: POST, req_id: 7d72b63a-3294-4cf9-b178-2391a7d01b13, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
27706 Replay: started 2, returned:Ok(())
27707 ----------------------------------------------------------------
27708 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27709 States: Active Active Offline
27710 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27711 7944 Acked Read 1 Done Done Sent false
27712 7945 Acked Flush 0 Done Done Sent false
27713 7946 Acked Flush 0 Done Done Sent false
27714 7947 Acked Read 5 Done Done Sent false
27715 7948 Acked Read 2 Done Done Sent false
27716 7949 Acked Read 4 Done Done Sent false
27717 7950 Acked Write 5 Done Done Sent false
27718 7951 Acked Write 6 Done Done Sent false
27719 7952 Acked Write 1 Done Done Sent false
27720 7953 Acked Read 10 Done Done Sent false
27721 7954 Acked Write 8 Done Done Sent false
27722 7955 Acked Write 1 Done Done Sent false
27723 7956 Acked Read 6 Done Done Sent false
27724 7957 Acked Read 5 Done Done Sent false
27725 7958 Acked Read 1 Done Done Sent false
27726 7959 Acked Write 6 Done Done Sent false
27727 7960 Acked Write 2 Done Done Sent false
27728 7961 Acked Read 2 Done Done Sent false
27729 7962 Acked Read 3 Done Done Sent false
27730 7963 Acked Read 8 Done Done Sent false
27731 7964 Acked Read 8 Done Done Sent false
27732 7965 Acked Read 4 Done Done Sent false
27733 7966 Acked Read 7 Done Done Sent false
27734 7967 Acked Write 8 Done Done Sent false
27735 7968 Acked Read 4 Done Done Sent false
27736 7969 Acked Read 3 Done Done Sent false
27737 7970 Acked Flush 0 Done Done Sent false
27738 7971 Acked Read 5 Done Done Sent false
27739 7972 Acked Write 10 Done Done Sent false
27740 7973 Acked Write 10 Done Done Sent false
27741 7974 Acked Flush 0 Done Done Sent false
27742 7975 Acked Read 3 Done Done Sent false
27743 7976 Acked Read 4 Done Done Sent false
27744 7977 Acked Read 10 Done Done Sent false
27745 7978 Acked Read 5 Done Done Sent false
27746 7979 Acked Read 10 Done Done Sent false
27747 7980 Acked Write 6 Done Done Sent false
27748 7981 Acked Write 9 Done Done Sent false
27749 7982 Acked Write 7 Done Done Sent false
27750 7983 Acked Write 6 Done Done Sent false
27751 7984 Acked Read 3 Done Done Sent false
27752 7985 Acked Flush 0 Done Done Sent false
27753 7986 Acked Write 8 Done Done Sent false
27754 7987 Acked Read 6 Done Done Sent false
27755 7988 Acked Write 7 Done Done Sent false
27756 7989 Acked Write 6 Done Done Sent false
27757 7990 Acked Read 2 Done Done Sent false
27758 7991 Acked Write 2 Done Done Sent false
27759 7992 Acked Write 8 Done Done Sent false
27760 7993 Acked Write 5 Done Done Sent false
27761 7994 Acked Write 5 Done Done Sent false
27762 7995 Acked Flush 0 Done Done Sent false
27763 7996 Acked Read 3 Done Done Sent false
27764 7997 Acked Read 9 Done Done Sent false
27765 7998 Acked Flush 0 Done Done Sent false
27766 7999 Acked Read 3 Done Done Sent false
27767 8000 Acked Flush 0 Done Done Sent false
27768 8001 Acked Write 8 Done Done Sent false
27769 8002 Acked Read 7 Done Done Sent false
27770 8003 Acked Write 7 Done Done Sent false
27771 8004 Acked Write 7 Done Done Sent false
27772 8005 Acked Read 2 Done Done Sent false
27773 8006 Acked Flush 0 Done Done Sent false
27774 8007 Acked Read 4 Done Done Sent false
27775 8008 Acked Flush 0 Done Done Sent false
27776 8009 Acked Write 1 Done Done Sent false
27777 8010 Acked Read 7 Done Done Sent false
27778 8011 Acked Read 4 Done Done Sent false
27779 8012 Acked Read 5 Done Done Sent false
27780 8013 Acked Read 1 Done Done Sent false
27781 8014 Acked Read 5 Done Done Sent false
27782 8015 Acked Read 9 Done Done Sent false
27783 8016 Acked Write 7 Done Done Sent false
27784 8017 Acked Write 3 Done Done Sent false
27785 8018 Acked Write 7 Done Done Sent false
27786 8019 Acked Read 9 Done Done Sent false
27787 8020 Acked Write 6 Done Done Sent false
27788 8021 Acked Read 7 Done Done Sent false
27789 8022 Acked Read 1 Done Done Sent false
27790 8023 Acked Read 2 Done Done Sent false
27791 8024 Acked Write 3 Done Done Sent false
27792 8025 Acked Flush 0 Done Done Sent false
27793 8026 Acked Flush 0 Done Done Sent false
27794 8027 Acked Write 5 Done Done Sent false
27795 8028 Acked Read 4 Done Done Sent false
27796 8029 Acked Write 1 Done Done Sent false
27797 8030 Acked Write 4 Done Done Sent false
27798 8031 Acked Read 4 Done Done Sent false
27799 8032 Acked Write 4 Done Done Sent false
27800 8033 Acked Read 8 Done Done Sent false
27801 8034 Acked Read 9 Done Done Sent false
27802 8035 Acked Read 8 Done Done Sent false
27803 8036 Acked Write 4 Done Done Sent false
27804 8037 Acked Read 7 Done Done Sent false
27805 8038 Acked Write 10 Done Done Sent false
27806 8039 Acked Write 6 Done Done Sent false
27807 8040 Acked Flush 0 Done Done Sent false
27808 8041 Acked Read 3 Done Done Sent false
27809 8042 Acked Read 6 Done Done Sent false
27810 8043 Acked Read 8 Done Done Sent false
27811 8044 Acked Write 7 Done Done Sent false
27812 8045 Acked Write 1 Done Done Sent false
27813 8046 Acked Write 9 Done Done Sent false
27814 8047 Acked Read 8 Done Done Sent false
27815 8048 Acked Write 1 Done Done Sent false
27816 8049 Acked Read 7 Done Done Sent false
27817 8050 Acked Read 4 Done Done Sent false
27818 8051 Acked Read 5 Done Done Sent false
27819 8052 Acked Read 2 Done Done Sent false
27820 8053 Acked Flush 0 Done Done Sent false
27821 8054 Acked Write 10 Done Done Sent false
27822 8055 Acked Flush 0 Done Done Sent false
27823 8056 Acked Write 5 Done Done Sent false
27824 8057 Acked Read 5 Done Done Sent false
27825 8058 Acked Read 4 Done Done Sent false
27826 8059 Acked Write 3 Done Done Sent false
27827 8060 Acked Flush 0 Done Done Sent false
27828 8061 Acked Flush 0 Done Done Sent false
27829 8062 Acked Write 5 Done Done Sent false
27830 8063 Acked Flush 0 Done Done Sent false
27831 8064 Acked Read 2 Done Done Sent false
27832 8065 Acked Read 6 Done Done Sent false
27833 8066 Acked Read 1 Done Done Sent false
27834 8067 Acked Write 6 Done Done Sent false
27835 8068 Acked Read 9 Done Done Sent false
27836 8069 Acked Write 3 Done Done Sent false
27837 8070 Acked Read 8 Done Done Sent false
27838 8071 Acked Read 7 Done Done Sent false
27839 8072 Acked Read 5 Done Done Sent false
27840 8073 Acked Read 3 Done Done Sent false
27841 8074 Acked Read 9 Done Done Sent false
27842 8075 Acked Read 1 Done Done Sent false
27843 8076 Acked Flush 0 Done Done Sent false
27844 8077 Acked Read 10 Done Done Sent false
27845 8078 Acked Flush 0 Done Done Sent false
27846 8079 Acked Flush 0 Done Done Sent false
27847 8080 Acked Read 6 Done Done Sent false
27848 8081 Acked Flush 0 Done Done Sent false
27849 8082 Acked Flush 0 Done Done Sent false
27850 8083 Acked Write 8 Done Done Sent false
27851 8084 Acked Write 6 Done Done Sent false
27852 8085 Acked Write 2 Done Done Sent false
27853 8086 Acked Read 6 Done Done Sent false
27854 8087 Acked Read 9 Done Done Sent false
27855 8088 Acked Read 2 Done Done Sent false
27856 8089 Acked Read 1 Done Done Sent false
27857 8090 Acked Read 3 Done Done Sent false
27858 8091 Acked Write 4 Done Done Sent false
27859 8092 Acked Read 3 Done Done Sent false
27860 8093 Acked Read 1 Done Done Sent false
27861 8094 Acked Read 4 Done Done Sent false
27862 8095 Acked Read 10 Done Done Sent false
27863 8096 Acked Read 6 Done Done Sent false
27864 8097 Acked Write 2 Done Done Sent false
27865 8098 Acked Read 8 Done Done Sent false
27866 8099 Acked Read 2 Done Done Sent false
27867 8100 Acked Write 5 Done Done Sent false
27868 8101 Acked Read 8 Done Done Sent false
27869 8102 Acked Read 8 Done Done Sent false
27870 8103 Acked Read 9 Done Done Sent false
27871 8104 Acked Read 3 Done Done Sent false
27872 8105 Acked Read 1 Done Done Sent false
27873 8106 Acked Flush 0 Done Done Sent false
27874 8107 Acked Write 4 Done Done Sent false
27875 8108 Acked Write 4 Done Done Sent false
27876 8109 Acked Read 5 Done Done Sent false
27877 8110 Acked Read 2 Done Done Sent false
27878 8111 Acked Read 4 Done Done Sent false
27879 8112 Acked Read 3 Done Done Sent false
27880 8113 Acked Write 10 Done Done Sent false
27881 8114 Acked Read 7 Done Done Sent false
27882 8115 Acked Flush 0 Done Done Sent false
27883 8116 Acked Flush 0 Done Done Sent false
27884 8117 Acked Read 1 Done Done Sent false
27885 8118 Acked Read 2 Done Done Sent false
27886 8119 Acked Read 5 Done Done Sent false
27887 8120 Acked Read 2 Done Done Sent false
27888 8121 Acked Read 7 Done Done Sent false
27889 8122 Acked Read 3 Done Done Sent false
27890 8123 Acked Read 3 Done Done Sent false
27891 8124 Acked Read 1 Done Done Sent false
27892 8125 Acked Read 9 Done Done Sent false
27893 8126 Acked Read 1 Done Done Sent false
27894 8127 Acked Read 3 Done Done Sent false
27895 8128 Acked Read 4 Done Done Sent false
27896 8129 Acked Read 9 Done Done Sent false
27897 8130 Acked Read 4 Done Done Sent false
27898 8131 Acked Read 10 Done Done Sent false
27899 8132 Acked Read 4 Done Done Sent false
27900 8133 Acked Read 5 Done Done Sent false
27901 8134 Acked Flush 0 Done Done Sent false
27902 8135 Acked Read 4 Done Done Sent false
27903 8136 Acked Write 6 Done Done Sent false
27904 8137 Acked Write 6 Done Done Sent false
27905 8138 Acked Read 2 Done Done Sent false
27906 8139 Acked Read 4 Done Done Sent false
27907 8140 Acked Read 3 Done Done Sent false
27908 8141 Acked Write 2 Done Done Sent false
27909 8142 Acked Read 1 Done Done Sent false
27910 8143 Acked Flush 0 Done Done Sent false
27911 8144 Acked Read 5 Done Done Sent false
27912 8145 Acked Write 8 Done Done Sent false
27913 8146 Acked Read 6 Done Done Sent false
27914 8147 Acked Write 6 Done Done Sent false
27915 8148 Acked Read 1 Done Done Sent false
27916 8149 Acked Read 2 Done Done Sent false
27917 8150 Acked Read 9 Done Done Sent false
27918 8151 Acked Write 10 Done Done Sent false
27919 8152 Acked Read 9 Done Done Sent false
27920 8153 Acked Read 7 Done Done Sent false
27921 8154 Acked Read 1 Done Done Sent false
27922 8155 Acked Write 5 Done Done Sent false
27923 8156 Acked Write 10 Done Done Sent false
27924 8157 Acked Write 3 Done Done Sent false
27925 8158 Acked Read 10 Done Done Sent false
27926 8159 Acked Read 4 Done Done Sent false
27927 8160 Acked Read 9 Done Done Sent false
27928 8161 Acked Flush 0 Done Done Sent false
27929 8162 Acked Write 9 Done Done Sent false
27930 8163 Acked Write 10 Done Done Sent false
27931 8164 Acked Read 6 Done Done Sent false
27932 8165 Acked Read 1 Done Done Sent false
27933 8166 Acked Flush 0 Done Done Sent false
27934 8167 Acked Write 10 Done Done Sent false
27935 8168 Acked Read 8 Done Done Sent false
27936 8169 Acked Read 7 Done Done Sent false
27937 8170 Acked Write 2 Done Done Sent false
27938 8171 Acked Write 1 Done Done Sent false
27939 8172 Acked Read 9 Done Done Sent false
27940 8173 Acked Read 5 Done Done Sent false
27941 8174 Acked Read 4 Done Done Sent false
27942 8175 Acked Read 8 Done Done Sent false
27943 8176 Acked Read 7 Done Done Sent false
27944 8177 Acked Write 6 Done Done Sent false
27945 8178 Acked Read 8 Done Done Sent false
27946 8179 Acked Read 4 Done Done Sent false
27947 8180 Acked Write 8 Done Done Sent false
27948 8181 Acked Write 7 Done Done Sent false
27949 8182 Acked Write 8 Done Done Sent false
27950 8183 Acked Read 3 Done Done Sent false
27951 8184 Acked Read 10 Done Done Sent false
27952 8185 Acked Read 10 Done Done Sent false
27953 8186 Acked Read 3 Done Done Sent false
27954 8187 Acked Read 10 Done Done Sent false
27955 8188 Acked Flush 0 Done Done Sent false
27956 8189 Acked Write 6 Done Done Sent false
27957 8190 Acked Write 6 Done Done Sent false
27958 8191 Acked Write 10 Done Done Sent false
27959 8192 Acked Write 5 Done Done Sent false
27960 8193 Acked Write 4 Done Done Sent false
27961 8194 Acked Write 10 Done Done Sent false
27962 8195 Acked Write 6 Done Done Sent false
27963 8196 Acked Write 3 Done Done Sent false
27964 8197 Acked Write 9 Done Done Sent false
27965 8198 Acked Write 1 Done Done Sent false
27966 8199 Acked Read 3 Done Done Sent false
27967 8200 Acked Read 10 Done Done Sent false
27968 8201 Acked Read 4 Done Done Sent false
27969 8202 Acked Write 8 Done Done Sent false
27970 8203 Acked Read 5 Done Done Sent false
27971 8204 Acked Read 10 Done Done Sent false
27972 8205 Acked Flush 0 Done Done Sent false
27973 8206 Acked Write 6 Done Done Sent false
27974 8207 Acked Read 7 Done Done Sent false
27975 8208 Acked Read 6 Done Done Sent false
27976 8209 Acked Read 5 Done Done Sent false
27977 8210 Acked Write 6 Done Done Sent false
27978 8211 Acked Read 2 Done Done Sent false
27979 8212 Acked Read 8 Done Done Sent false
27980 8213 Acked Flush 0 Done Done Sent false
27981 8214 Acked Read 1 Done Done Sent false
27982 8215 Acked Write 3 Done Done Sent false
27983 8216 Acked Read 3 Done Done Sent false
27984 8217 Acked Read 2 Done Done Sent false
27985 8218 Acked Write 5 Done Done Sent false
27986 8219 Acked Write 7 Done Done Sent false
27987 8220 Acked Write 9 Done Done Sent false
27988 8221 Acked Write 7 Done Done Sent false
27989 8222 Acked Write 2 Done Done Sent false
27990 8223 Acked Write 5 Done Done Sent false
27991 8224 Acked Read 4 Done Done Sent false
27992 8225 Acked Write 1 Done Done Sent false
27993 8226 Acked Read 10 Done Done Sent false
27994 8227 Acked Read 10 Done Done Sent false
27995 8228 Acked Read 1 Done Done Sent false
27996 8229 Acked Write 2 Done Done Sent false
27997 8230 Acked Write 3 Done Done Sent false
27998 8231 Acked Read 5 Done Done Sent false
27999 8232 Acked Read 7 Done Done Sent false
28000 8233 Acked Read 9 Done Done Sent false
28001 8234 Acked Write 4 Done Done Sent false
28002 8235 Acked Write 1 Done Done Sent false
28003 8236 Acked Read 3 Done Done Sent false
28004 8237 Acked Write 4 Done Done Sent false
28005 8238 Acked Write 9 Done Done Sent false
28006 8239 Acked Flush 0 Done Done Sent false
28007 8240 Acked Write 3 Done Done Sent false
28008 8241 Acked Flush 0 Done Done Sent false
28009 8242 Acked Flush 0 Done Done Sent false
28010 8243 Acked Read 2 Done Done Sent false
28011 8244 Acked Write 5 Done Done Sent false
28012 STATES DS:0 DS:1 DS:2 TOTAL
28013 Sent 0 0 301 301
28014 Done 301 301 0 602
28015 Skipped 0 0 0 0
28016 Error 0 0 0 0
28017 Last Flush: 8242 8242 7943
28018 Downstairs last five completed: 7943 7942 7941 7940 7939
28019 Upstairs last five completed: 8244 8243 8242 8241 8240
28020 CLIENT: Up:0 ds:301 act:2
28021 [8830] got start action
28022 [8830] I am going to restart
28023 Starting downstairs at port 8830
28024 Make output file at "/tmp/dsc/downstairs-8830.txt"
28025 [8830][2] reports Starting
28026 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1229)
28027 [8830][2] reports Running
28028 ----------------------------------------------------------------
28029 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28030 States: Active Active Offline
28031 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28032 7944 Acked Read 1 Done Done Sent false
28033 7945 Acked Flush 0 Done Done Sent false
28034 7946 Acked Flush 0 Done Done Sent false
28035 7947 Acked Read 5 Done Done Sent false
28036 7948 Acked Read 2 Done Done Sent false
28037 7949 Acked Read 4 Done Done Sent false
28038 7950 Acked Write 5 Done Done Sent false
28039 7951 Acked Write 6 Done Done Sent false
28040 7952 Acked Write 1 Done Done Sent false
28041 7953 Acked Read 10 Done Done Sent false
28042 7954 Acked Write 8 Done Done Sent false
28043 7955 Acked Write 1 Done Done Sent false
28044 7956 Acked Read 6 Done Done Sent false
28045 7957 Acked Read 5 Done Done Sent false
28046 7958 Acked Read 1 Done Done Sent false
28047 7959 Acked Write 6 Done Done Sent false
28048 7960 Acked Write 2 Done Done Sent false
28049 7961 Acked Read 2 Done Done Sent false
28050 7962 Acked Read 3 Done Done Sent false
28051 7963 Acked Read 8 Done Done Sent false
28052 7964 Acked Read 8 Done Done Sent false
28053 7965 Acked Read 4 Done Done Sent false
28054 7966 Acked Read 7 Done Done Sent false
28055 7967 Acked Write 8 Done Done Sent false
28056 7968 Acked Read 4 Done Done Sent false
28057 7969 Acked Read 3 Done Done Sent false
28058 7970 Acked Flush 0 Done Done Sent false
28059 7971 Acked Read 5 Done Done Sent false
28060 7972 Acked Write 10 Done Done Sent false
28061 7973 Acked Write 10 Done Done Sent false
28062 7974 Acked Flush 0 Done Done Sent false
28063 7975 Acked Read 3 Done Done Sent false
28064 7976 Acked Read 4 Done Done Sent false
28065 7977 Acked Read 10 Done Done Sent false
28066 7978 Acked Read 5 Done Done Sent false
28067 7979 Acked Read 10 Done Done Sent false
28068 7980 Acked Write 6 Done Done Sent false
28069 7981 Acked Write 9 Done Done Sent false
28070 7982 Acked Write 7 Done Done Sent false
28071 7983 Acked Write 6 Done Done Sent false
28072 7984 Acked Read 3 Done Done Sent false
28073 7985 Acked Flush 0 Done Done Sent false
28074 7986 Acked Write 8 Done Done Sent false
28075 7987 Acked Read 6 Done Done Sent false
28076 7988 Acked Write 7 Done Done Sent false
28077 7989 Acked Write 6 Done Done Sent false
28078 7990 Acked Read 2 Done Done Sent false
28079 7991 Acked Write 2 Done Done Sent false
28080 7992 Acked Write 8 Done Done Sent false
28081 7993 Acked Write 5 Done Done Sent false
28082 7994 Acked Write 5 Done Done Sent false
28083 7995 Acked Flush 0 Done Done Sent false
28084 7996 Acked Read 3 Done Done Sent false
28085 7997 Acked Read 9 Done Done Sent false
28086 7998 Acked Flush 0 Done Done Sent false
28087 7999 Acked Read 3 Done Done Sent false
28088 8000 Acked Flush 0 Done Done Sent false
28089 8001 Acked Write 8 Done Done Sent false
28090 8002 Acked Read 7 Done Done Sent false
28091 8003 Acked Write 7 Done Done Sent false
28092 8004 Acked Write 7 Done Done Sent false
28093 8005 Acked Read 2 Done Done Sent false
28094 8006 Acked Flush 0 Done Done Sent false
28095 8007 Acked Read 4 Done Done Sent false
28096 8008 Acked Flush 0 Done Done Sent false
28097 8009 Acked Write 1 Done Done Sent false
28098 8010 Acked Read 7 Done Done Sent false
28099 8011 Acked Read 4 Done Done Sent false
28100 8012 Acked Read 5 Done Done Sent false
28101 8013 Acked Read 1 Done Done Sent false
28102 8014 Acked Read 5 Done Done Sent false
28103 8015 Acked Read 9 Done Done Sent false
28104 8016 Acked Write 7 Done Done Sent false
28105 8017 Acked Write 3 Done Done Sent false
28106 8018 Acked Write 7 Done Done Sent false
28107 8019 Acked Read 9 Done Done Sent false
28108 8020 Acked Write 6 Done Done Sent false
28109 8021 Acked Read 7 Done Done Sent false
28110 8022 Acked Read 1 Done Done Sent false
28111 8023 Acked Read 2 Done Done Sent false
28112 8024 Acked Write 3 Done Done Sent false
28113 8025 Acked Flush 0 Done Done Sent false
28114 8026 Acked Flush 0 Done Done Sent false
28115 8027 Acked Write 5 Done Done Sent false
28116 8028 Acked Read 4 Done Done Sent false
28117 8029 Acked Write 1 Done Done Sent false
28118 8030 Acked Write 4 Done Done Sent false
28119 8031 Acked Read 4 Done Done Sent false
28120 8032 Acked Write 4 Done Done Sent false
28121 8033 Acked Read 8 Done Done Sent false
28122 8034 Acked Read 9 Done Done Sent false
28123 8035 Acked Read 8 Done Done Sent false
28124 8036 Acked Write 4 Done Done Sent false
28125 8037 Acked Read 7 Done Done Sent false
28126 8038 Acked Write 10 Done Done Sent false
28127 8039 Acked Write 6 Done Done Sent false
28128 8040 Acked Flush 0 Done Done Sent false
28129 8041 Acked Read 3 Done Done Sent false
28130 8042 Acked Read 6 Done Done Sent false
28131 8043 Acked Read 8 Done Done Sent false
28132 8044 Acked Write 7 Done Done Sent false
28133 8045 Acked Write 1 Done Done Sent false
28134 8046 Acked Write 9 Done Done Sent false
28135 8047 Acked Read 8 Done Done Sent false
28136 8048 Acked Write 1 Done Done Sent false
28137 8049 Acked Read 7 Done Done Sent false
28138 8050 Acked Read 4 Done Done Sent false
28139 8051 Acked Read 5 Done Done Sent false
28140 8052 Acked Read 2 Done Done Sent false
28141 8053 Acked Flush 0 Done Done Sent false
28142 8054 Acked Write 10 Done Done Sent false
28143 8055 Acked Flush 0 Done Done Sent false
28144 8056 Acked Write 5 Done Done Sent false
28145 8057 Acked Read 5 Done Done Sent false
28146 8058 Acked Read 4 Done Done Sent false
28147 8059 Acked Write 3 Done Done Sent false
28148 8060 Acked Flush 0 Done Done Sent false
28149 8061 Acked Flush 0 Done Done Sent false
28150 8062 Acked Write 5 Done Done Sent false
28151 8063 Acked Flush 0 Done Done Sent false
28152 8064 Acked Read 2 Done Done Sent false
28153 8065 Acked Read 6 Done Done Sent false
28154 8066 Acked Read 1 Done Done Sent false
28155 8067 Acked Write 6 Done Done Sent false
28156 8068 Acked Read 9 Done Done Sent false
28157 8069 Acked Write 3 Done Done Sent false
28158 8070 Acked Read 8 Done Done Sent false
28159 8071 Acked Read 7 Done Done Sent false
28160 8072 Acked Read 5 Done Done Sent false
28161 8073 Acked Read 3 Done Done Sent false
28162 8074 Acked Read 9 Done Done Sent false
28163 8075 Acked Read 1 Done Done Sent false
28164 8076 Acked Flush 0 Done Done Sent false
28165 8077 Acked Read 10 Done Done Sent false
28166 8078 Acked Flush 0 Done Done Sent false
28167 8079 Acked Flush 0 Done Done Sent false
28168 8080 Acked Read 6 Done Done Sent false
28169 8081 Acked Flush 0 Done Done Sent false
28170 8082 Acked Flush 0 Done Done Sent false
28171 8083 Acked Write 8 Done Done Sent false
28172 8084 Acked Write 6 Done Done Sent false
28173 8085 Acked Write 2 Done Done Sent false
28174 8086 Acked Read 6 Done Done Sent false
28175 8087 Acked Read 9 Done Done Sent false
28176 8088 Acked Read 2 Done Done Sent false
28177 8089 Acked Read 1 Done Done Sent false
28178 8090 Acked Read 3 Done Done Sent false
28179 8091 Acked Write 4 Done Done Sent false
28180 8092 Acked Read 3 Done Done Sent false
28181 8093 Acked Read 1 Done Done Sent false
28182 8094 Acked Read 4 Done Done Sent false
28183 8095 Acked Read 10 Done Done Sent false
28184 8096 Acked Read 6 Done Done Sent false
28185 8097 Acked Write 2 Done Done Sent false
28186 8098 Acked Read 8 Done Done Sent false
28187 8099 Acked Read 2 Done Done Sent false
28188 8100 Acked Write 5 Done Done Sent false
28189 8101 Acked Read 8 Done Done Sent false
28190 8102 Acked Read 8 Done Done Sent false
28191 8103 Acked Read 9 Done Done Sent false
28192 8104 Acked Read 3 Done Done Sent false
28193 8105 Acked Read 1 Done Done Sent false
28194 8106 Acked Flush 0 Done Done Sent false
28195 8107 Acked Write 4 Done Done Sent false
28196 8108 Acked Write 4 Done Done Sent false
28197 8109 Acked Read 5 Done Done Sent false
28198 8110 Acked Read 2 Done Done Sent false
28199 8111 Acked Read 4 Done Done Sent false
28200 8112 Acked Read 3 Done Done Sent false
28201 8113 Acked Write 10 Done Done Sent false
28202 8114 Acked Read 7 Done Done Sent false
28203 8115 Acked Flush 0 Done Done Sent false
28204 8116 Acked Flush 0 Done Done Sent false
28205 8117 Acked Read 1 Done Done Sent false
28206 8118 Acked Read 2 Done Done Sent false
28207 8119 Acked Read 5 Done Done Sent false
28208 8120 Acked Read 2 Done Done Sent false
28209 8121 Acked Read 7 Done Done Sent false
28210 8122 Acked Read 3 Done Done Sent false
28211 8123 Acked Read 3 Done Done Sent false
28212 8124 Acked Read 1 Done Done Sent false
28213 8125 Acked Read 9 Done Done Sent false
28214 8126 Acked Read 1 Done Done Sent false
28215 8127 Acked Read 3 Done Done Sent false
28216 8128 Acked Read 4 Done Done Sent false
28217 8129 Acked Read 9 Done Done Sent false
28218 8130 Acked Read 4 Done Done Sent false
28219 8131 Acked Read 10 Done Done Sent false
28220 8132 Acked Read 4 Done Done Sent false
28221 8133 Acked Read 5 Done Done Sent false
28222 8134 Acked Flush 0 Done Done Sent false
28223 8135 Acked Read 4 Done Done Sent false
28224 8136 Acked Write 6 Done Done Sent false
28225 8137 Acked Write 6 Done Done Sent false
28226 8138 Acked Read 2 Done Done Sent false
28227 8139 Acked Read 4 Done Done Sent false
28228 8140 Acked Read 3 Done Done Sent false
28229 8141 Acked Write 2 Done Done Sent false
28230 8142 Acked Read 1 Done Done Sent false
28231 8143 Acked Flush 0 Done Done Sent false
28232 8144 Acked Read 5 Done Done Sent false
28233 8145 Acked Write 8 Done Done Sent false
28234 8146 Acked Read 6 Done Done Sent false
28235 8147 Acked Write 6 Done Done Sent false
28236 8148 Acked Read 1 Done Done Sent false
28237 8149 Acked Read 2 Done Done Sent false
28238 8150 Acked Read 9 Done Done Sent false
28239 8151 Acked Write 10 Done Done Sent false
28240 8152 Acked Read 9 Done Done Sent false
28241 8153 Acked Read 7 Done Done Sent false
28242 8154 Acked Read 1 Done Done Sent false
28243 8155 Acked Write 5 Done Done Sent false
28244 8156 Acked Write 10 Done Done Sent false
28245 8157 Acked Write 3 Done Done Sent false
28246 8158 Acked Read 10 Done Done Sent false
28247 8159 Acked Read 4 Done Done Sent false
28248 8160 Acked Read 9 Done Done Sent false
28249 8161 Acked Flush 0 Done Done Sent false
28250 8162 Acked Write 9 Done Done Sent false
28251 8163 Acked Write 10 Done Done Sent false
28252 8164 Acked Read 6 Done Done Sent false
28253 8165 Acked Read 1 Done Done Sent false
28254 8166 Acked Flush 0 Done Done Sent false
28255 8167 Acked Write 10 Done Done Sent false
28256 8168 Acked Read 8 Done Done Sent false
28257 8169 Acked Read 7 Done Done Sent false
28258 8170 Acked Write 2 Done Done Sent false
28259 8171 Acked Write 1 Done Done Sent false
28260 8172 Acked Read 9 Done Done Sent false
28261 8173 Acked Read 5 Done Done Sent false
28262 8174 Acked Read 4 Done Done Sent false
28263 8175 Acked Read 8 Done Done Sent false
28264 8176 Acked Read 7 Done Done Sent false
28265 8177 Acked Write 6 Done Done Sent false
28266 8178 Acked Read 8 Done Done Sent false
28267 8179 Acked Read 4 Done Done Sent false
28268 8180 Acked Write 8 Done Done Sent false
28269 8181 Acked Write 7 Done Done Sent false
28270 8182 Acked Write 8 Done Done Sent false
28271 8183 Acked Read 3 Done Done Sent false
28272 8184 Acked Read 10 Done Done Sent false
28273 8185 Acked Read 10 Done Done Sent false
28274 8186 Acked Read 3 Done Done Sent false
28275 8187 Acked Read 10 Done Done Sent false
28276 8188 Acked Flush 0 Done Done Sent false
28277 8189 Acked Write 6 Done Done Sent false
28278 8190 Acked Write 6 Done Done Sent false
28279 8191 Acked Write 10 Done Done Sent false
28280 8192 Acked Write 5 Done Done Sent false
28281 8193 Acked Write 4 Done Done Sent false
28282 8194 Acked Write 10 Done Done Sent false
28283 8195 Acked Write 6 Done Done Sent false
28284 8196 Acked Write 3 Done Done Sent false
28285 8197 Acked Write 9 Done Done Sent false
28286 8198 Acked Write 1 Done Done Sent false
28287 8199 Acked Read 3 Done Done Sent false
28288 8200 Acked Read 10 Done Done Sent false
28289 8201 Acked Read 4 Done Done Sent false
28290 8202 Acked Write 8 Done Done Sent false
28291 8203 Acked Read 5 Done Done Sent false
28292 8204 Acked Read 10 Done Done Sent false
28293 8205 Acked Flush 0 Done Done Sent false
28294 8206 Acked Write 6 Done Done Sent false
28295 8207 Acked Read 7 Done Done Sent false
28296 8208 Acked Read 6 Done Done Sent false
28297 8209 Acked Read 5 Done Done Sent false
28298 8210 Acked Write 6 Done Done Sent false
28299 8211 Acked Read 2 Done Done Sent false
28300 8212 Acked Read 8 Done Done Sent false
28301 8213 Acked Flush 0 Done Done Sent false
28302 8214 Acked Read 1 Done Done Sent false
28303 8215 Acked Write 3 Done Done Sent false
28304 8216 Acked Read 3 Done Done Sent false
28305 8217 Acked Read 2 Done Done Sent false
28306 8218 Acked Write 5 Done Done Sent false
28307 8219 Acked Write 7 Done Done Sent false
28308 8220 Acked Write 9 Done Done Sent false
28309 8221 Acked Write 7 Done Done Sent false
28310 8222 Acked Write 2 Done Done Sent false
28311 8223 Acked Write 5 Done Done Sent false
28312 8224 Acked Read 4 Done Done Sent false
28313 8225 Acked Write 1 Done Done Sent false
28314 8226 Acked Read 10 Done Done Sent false
28315 8227 Acked Read 10 Done Done Sent false
28316 8228 Acked Read 1 Done Done Sent false
28317 8229 Acked Write 2 Done Done Sent false
28318 8230 Acked Write 3 Done Done Sent false
28319 8231 Acked Read 5 Done Done Sent false
28320 8232 Acked Read 7 Done Done Sent false
28321 8233 Acked Read 9 Done Done Sent false
28322 8234 Acked Write 4 Done Done Sent false
28323 8235 Acked Write 1 Done Done Sent false
28324 8236 Acked Read 3 Done Done Sent false
28325 8237 Acked Write 4 Done Done Sent false
28326 8238 Acked Write 9 Done Done Sent false
28327 8239 Acked Flush 0 Done Done Sent false
28328 8240 Acked Write 3 Done Done Sent false
28329 8241 Acked Flush 0 Done Done Sent false
28330 8242 Acked Flush 0 Done Done Sent false
28331 8243 Acked Read 2 Done Done Sent false
28332 8244 Acked Write 5 Done Done Sent false
28333 8245 Acked Flush 0 Done Done Sent false
28334 STATES DS:0 DS:1 DS:2 TOTAL
28335 Sent 0 0 302 302
28336 Done 302 302 0 604
28337 Skipped 0 0 0 0
28338 Error 0 0 0 0
28339 Last Flush: 8245 8245 7943
28340 Downstairs last five completed: 7943 7942 7941 7940 7939
28341 Upstairs last five completed: 8245 8244 8243 8242 8241
28342 CLIENT: Up:0 ds:302 act:2
283432025-01-03T08:36:49.374ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283442025-01-03T08:36:49.374ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51551) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283452025-01-03T08:36:49.375ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283462025-01-03T08:36:49.376ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283472025-01-03T08:36:49.376ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283482025-01-03T08:36:49.376ZINFOcrucible: send last flush ID to this DS: 7943 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283492025-01-03T08:36:49.376ZINFOcrucible: Replied this last flush ID: 7943 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283502025-01-03T08:36:49.376ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283512025-01-03T08:36:49.376ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283522025-01-03T08:36:49.376ZINFOcrucible: [2] client re-new 302 jobs since flush 7943 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283532025-01-03T08:36:49.376ZINFOcrucible: [2] Replayed 302 jobs since flush: 7943 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28354 ----------------------------------------------------------------
28355 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28356 Downstairs last five completed: 8245 8244 8243 8242 8241
28357 Upstairs last five completed: 8245 8244 8243 8242 8241
28358 CLIENT: Up:0 ds:0 act:3
28359 Replay: All jobs finished
28360 Main task has work to do, go find it
28361 got dsc Stop(2)
28362 stop 2
28363 [8830] Got stop action so:false kr:false
28364 Jan 03 08:36:51.617 INFO request completed, latency_us: 346, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 44264283-64fd-4935-a502-cb3614b01315, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
283652025-01-03T08:36:51.618ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
283662025-01-03T08:36:51.618ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28367 Jan 03 08:36:51.618 INFO {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:36:51.618383748Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
28368 request completed, latency_us: 131, response_code: 200, uri: /state/cid/2, method: GET, req_id{"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:36:51.618409177Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
28369 : a3d2c396-ea59-411e-8d9c-1e9cc3d735f5, remote_addr: 127.0.0.1:{"msg":"[2] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:36:51.618436067Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"2","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
28370 49669, local_addr: 127.0.0.1:9998
283712025-01-03T08:36:51.618ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
28372 [8830] Exited with: ExitStatus(unix_wait_status(9))
28373 [8830][2] reports Exit
28374 Jan 03 08:36:55.620 INFO request completed, latency_us: 225, response_code: 200, uri: /state/cid/2, method: GET, req_id: 3545bf76-470b-4014-b27a-e9f301a54fdb, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
28375 001/300 Read block 810 len 4096
28376 002/300 Write block 4625 len 24576 data: 5 5 5 4 4 4
28377 003/300 Read block 2754 len 40960
28378 004/300 Read block 3378 len 24576
28379 005/300 Read block 2308 len 16384
28380 006/300 Read block 4365 len 12288
28381 007/300 Write block 1550 len 24576 data: 4 4 4 4 4 2
28382 008/300 Write block 1732 len 8192 data: 6 6
28383 009/300 Flush
28384 010/300 Write block 1028 len 36864 data: 3 3 3 2 2 2 3 3 3
28385 011/300 Read block 2911 len 40960
28386 012/300 Read block 90 len 40960
28387 013/300 Write block 4403 len 12288 data: 2 2 2
28388 014/300 Read block 2736 len 24576
28389 015/300 Read block 4481 len 36864
28390 016/300 Write block 3008 len 28672 data: 5 5 4 4 5 5 5
28391 017/300 Write block 2087 len 24576 data: 6 6 5 5 4 4
28392 018/300 Read block 2752 len 24576
28393 019/300 Read block 4858 len 32768
28394 020/300 Write block 4176 len 36864 data: 6 7 6 6 6 5 4 5 5
28395 021/300 Write block 703 len 8192 data: 3 4
28396 022/300 Write block 3630 len 20480 data: 5 5 6 8 7
28397 023/300 Flush
28398 024/300 Read block 1216 len 32768
28399 025/300 Write block 1897 len 16384 data: 6 6 6 4
28400 026/300 Write block 1825 len 4096 data: 7
28401 027/300 Read block 3532 len 24576
28402 028/300 Read block 2603 len 28672
28403 029/300 Read block 95 len 4096
28404 030/300 Write block 2371 len 24576 data: 7 6 6 6 4 4
28405 031/300 Write block 3214 len 12288 data: 3 3 5
28406 032/300 Read block 583 len 4096
28407 033/300 Flush
28408 034/300 Write block 125 len 12288 data: 3 4 4
28409 035/300 Read block 4289 len 24576
28410 036/300 Write block 3050 len 20480 data: 5 5 4 6 7
28411 037/300 Write block 50 len 20480 data: 3 3 3 5 6
28412 038/300 Write block 2075 len 32768 data: 7 5 5 6 4 3 4 4
28413 039/300 Write block 227 len 12288 data: 5 7 9
28414 040/300 Write block 2921 len 28672 data: 5 5 5 7 7 7 8
28415 041/300 Read block 783 len 8192
28416 042/300 Write block 1845 len 28672 data: 4 2 3 5 5 5 6
28417 043/300 Read block 4404 len 36864
28418 044/300 Read block 4038 len 24576
28419 045/300 Read block 3936 len 4096
28420 046/300 Read block 2027 len 32768
28421 047/300 Read block 1579 len 20480
28422 048/300 Write block 1057 len 20480 data: 4 3 3 3 3
28423 049/300 Read block 2522 len 32768
28424 050/300 Write block 1934 len 32768 data: 3 3 4 4 3 4 5 5
28425 051/300 Write block 3766 len 8192 data: 8 9
28426 052/300 Read block 2854 len 12288
28427 053/300 Read block 4178 len 4096
28428 054/300 Read block 3503 len 40960
28429 055/300 Write block 2812 len 28672 data: 8 9 9 7 7 6 6
28430 056/300 Flush
28431 057/300 Read block 2596 len 28672
28432 058/300 Write block 3993 len 8192 data: 3 5
28433 059/300 Write block 1132 len 28672 data: 5 7 7 9 8 7 6
28434 060/300 Read block 1349 len 20480
28435 061/300 Read block 777 len 32768
28436 062/300 Write block 4400 len 28672 data: 4 3 3 3 3 3 2
28437 063/300 Read block 2327 len 16384
28438 064/300 Read block 1713 len 28672
28439 065/300 Write block 849 len 20480 data: 5 5 5 3 5
28440 066/300 Write block 3461 len 16384 data: 5 3 3 3
28441 067/300 Read block 3973 len 12288
28442 068/300 Flush
28443 069/300 Read block 1268 len 40960
28444 070/300 Write block 4011 len 8192 data: 4 2
28445 071/300 Read block 2279 len 32768
28446 072/300 Write block 4854 len 12288 data: 5 6 5
28447 073/300 Write block 4699 len 12288 data: 2 2 4
28448 074/300 Write block 2176 len 16384 data: 5 3 3 3
28449 075/300 Write block 1632 len 28672 data: 5 3 3 4 5 3 3
28450 076/300 Read block 4629 len 20480
28451 077/300 Read block 4905 len 28672
28452 078/300 Write block 1344 len 40960 data: 3 4 5 5 5 3 3 3 4 4
28453 079/300 Read block 2724 len 28672
28454 080/300 Write block 1173 len 32768 data: 6 8 7 8 7 7 7 6
28455 081/300 Flush
28456 082/300 Write block 4177 len 8192 data: 8 7
28457 083/300 Read block 1755 len 4096
28458 084/300 Read block 1100 len 36864
28459 085/300 Write block 3935 len 4096 data: 3
28460 086/300 Write block 2639 len 24576 data: 5 5 8 7 6 6
28461 087/300 Write block 1711 len 28672 data: 5 5 6 6 6 5 5
28462 088/300 Read block 1931 len 20480
28463 089/300 Read block 4616 len 20480
28464 090/300 Read block 3212 len 12288
28465 091/300 Read block 4774 len 8192
28466 092/300 Write block 2530 len 28672 data: 6 6 6 6 5 5 4
28467 093/300 Read block 913 len 8192
28468 094/300 Read block 2021 len 8192
28469 095/300 Read block 3671 len 4096
28470 096/300 Read block 1504 len 32768
28471 097/300 Write block 4751 len 40960 data: 4 4 6 7 7 8 8 8 7 7
28472 098/300 Read block 1322 len 16384
28473 099/300 Write block 2818 len 4096 data: 7
28474 100/300 Read block 722 len 32768
28475 101/300 Read block 54 len 40960
28476 102/300 Flush
28477 103/300 Read block 3585 len 36864
28478 104/300 Read block 2023 len 8192
28479 105/300 Read block 4009 len 28672
28480 106/300 Read block 3997 len 12288
28481 107/300 Read block 3659 len 12288
28482 108/300 Write block 3169 len 20480 data: 6 4 4 3 5
28483 109/300 Read block 1101 len 24576
28484 110/300 Read block 761 len 32768
28485 111/300 Read block 1790 len 20480
28486 112/300 Read block 575 len 40960
28487 113/300 Write block 550 len 32768 data: 3 3 3 4 5 5 6 7
28488 114/300 Read block 2226 len 36864
28489 115/300 Read block 4170 len 36864
28490 116/300 Write block 1744 len 36864 data: 2 2 3 3 3 3 4 5 7
28491 117/300 Read block 1993 len 24576
28492 118/300 Write block 3221 len 36864 data: 7 7 7 7 6 5 5 6 4
28493 119/300 Read block 3817 len 4096
28494 120/300 Write block 4742 len 40960 data: 3 4 5 5 5 6 8 8 4 5
28495 121/300 Read block 880 len 32768
28496 122/300 Read block 4167 len 32768
28497 123/300 Write block 198 len 16384 data: 6 5 4 3
28498 124/300 Read block 3302 len 36864
28499 125/300 Read block 3369 len 40960
28500 126/300 Write block 4093 len 16384 data: 3 4 5 7
28501 127/300 Read block 3589 len 20480
28502 128/300 Write block 2943 len 24576 data: 4 5 5 7 7 5
28503 129/300 Write block 2065 len 24576 data: 3 4 5 6 6 6
28504 130/300 Write block 3904 len 16384 data: 6 7 7 7
28505 131/300 Read block 950 len 32768
28506 132/300 Write block 661 len 32768 data: 5 6 6 6 6 6 5 6
28507 133/300 Write block 1224 len 24576 data: 7 7 5 5 4 4
28508 134/300 Read block 2551 len 12288
28509 135/300 Read block 2374 len 4096
28510 136/300 Read block 1657 len 12288
28511 137/300 Write block 4713 len 24576 data: 6 4 4 5 5 3
28512 138/300 Write block 3483 len 36864 data: 3 4 4 4 5 5 5 5 6
28513 139/300 Read block 3264 len 12288
28514 140/300 Read block 2333 len 16384
28515 141/300 Flush
28516 142/300 Read block 1636 len 32768
28517 143/300 Read block 3825 len 20480
28518 144/300 Read block 781 len 4096
28519 145/300 Write block 691 len 28672 data: 6 7 7 7 6 6 6
28520 146/300 Read block 4818 len 36864
28521 147/300 Read block 432 len 36864
28522 148/300 Read block 940 len 12288
28523 149/300 Write block 3928 len 40960 data: 4 6 5 6 6 6 4 4 4 6
28524 150/300 Write block 1858 len 32768 data: 3 3 3 3 3 2 3 3
28525 151/300 Read block 3414 len 4096
28526 152/300 Write block 1949 len 8192 data: 10 9
28527 153/300 Read block 4911 len 32768
28528 154/300 Write block 209 len 8192 data: 3 3
28529 155/300 Read block 2910 len 24576
28530 156/300 Write block 3297 len 12288 data: 5 5 4
28531 157/300 Read block 2884 len 20480
28532 158/300 Read block 1566 len 20480
28533 159/300 Write block 4599 len 36864 data: 5 4 4 4 6 6 6 5 4
28534 160/300 Write block 2820 len 40960 data: 4 4 4 6 6 7 9 8 6 6
28535 161/300 Flush
28536 162/300 Write block 3606 len 12288 data: 7 8 7
28537 163/300 Read block 4031 len 36864
28538 164/300 Write block 3767 len 36864 data: 10 9 8 7 8 9 8 7 6
28539 165/300 Write block 747 len 32768 data: 5 5 5 6 6 6 4 2
28540 166/300 Read block 710 len 40960
28541 167/300 Write block 3578 len 36864 data: 3 3 3 4 4 3 2 4 4
28542 168/300 Read block 803 len 40960
28543 169/300 Read block 456 len 8192
28544 170/300 Read block 2839 len 24576
28545 171/300 Read block 4303 len 20480
28546 172/300 Write block 2061 len 24576 data: 6 6 4 3 4 5
28547 173/300 Read block 2358 len 20480
28548 174/300 Read block 2531 len 32768
28549 175/300 Read block 710 len 12288
28550 176/300 Read block 4000 len 40960
28551 177/300 Read block 4325 len 24576
28552 178/300 Read block 40 len 4096
28553 179/300 Write block 2052 len 16384 data: 7 8 8 9
28554 180/300 Write block 349 len 28672 data: 7 7 7 7 6 5 3
28555 181/300 Write block 4781 len 4096 data: 4
28556 182/300 Read block 40 len 24576
28557 183/300 Read block 271 len 16384
28558 184/300 Write block 4739 len 28672 data: 3 3 4 4 5 6 6
28559 185/300 Read block 4053 len 20480
28560 186/300 Read block 4397 len 32768
28561 187/300 Read block 825 len 40960
28562 188/300 Read block 1489 len 40960
28563 189/300 Write block 1452 len 20480 data: 6 6 7 7 7
28564 190/300 Write block 4036 len 36864 data: 4 3 2 2 4 4 5 7 7
28565 191/300 Read block 4705 len 16384
28566 192/300 Write block 4557 len 4096 data: 5
28567 193/300 Flush
28568 194/300 Read block 2595 len 12288
28569 195/300 Write block 2607 len 4096 data: 4
28570 196/300 Read block 2730 len 36864
28571 197/300 Write block 2042 len 32768 data: 5 5 5 5 6 6 7 7
28572 198/300 Write block 1548 len 8192 data: 3 3
28573 199/300 Read block 855 len 24576
28574 200/300 Read block 493 len 12288
28575 201/300 Read block 2248 len 4096
28576 202/300 Write block 2467 len 24576 data: 6 6 7 5 4 5
28577 203/300 Write block 611 len 24576 data: 5 7 7 8 7 6
28578 204/300 Read block 2028 len 28672
28579 205/300 Read block 3912 len 8192
28580 206/300 Flush
28581 207/300 Write block 2062 len 28672 data: 7 5 4 5 6 6 7
28582 208/300 Read block 4525 len 4096
28583 209/300 Flush
28584 210/300 Read block 2417 len 12288
28585 211/300 Write block 2739 len 28672 data: 3 3 4 4 6 5 5
28586 212/300 Read block 3192 len 20480
28587 213/300 Write block 4491 len 12288 data: 5 5 5
28588 214/300 Read block 4496 len 4096
28589 215/300 Read block 4367 len 40960
28590 216/300 Write block 1029 len 32768 data: 4 4 3 3 3 4 4 4
28591 217/300 Read block 505 len 4096
28592 218/300 Write block 1817 len 28672 data: 5 7 6 6 6 6 6
28593 219/300 Read block 1413 len 28672
28594 220/300 Flush
28595 221/300 Write block 1613 len 16384 data: 4 5 5 4
28596 222/300 Write block 4684 len 4096 data: 5
28597 223/300 Read block 690 len 16384
28598 224/300 Write block 1220 len 32768 data: 6 7 6 7 8 8 6 6
28599 225/300 Write block 2132 len 4096 data: 8
28600 226/300 Read block 3065 len 32768
28601 227/300 Read block 4962 len 32768
28602 228/300 Write block 476 len 12288 data: 6 6 5
28603 229/300 Write block 3598 len 36864 data: 4 5 8 8 7 7 8 10 8
28604 230/300 Write block 1024 len 40960 data: 4 4 5 4 4 5 5 4 4 4
28605 231/300 Read block 631 len 20480
28606 232/300 Read block 2191 len 40960
28607 233/300 Read block 3650 len 24576
28608 234/300 Write block 3884 len 40960 data: 7 5 6 7 6 6 4 5 6 5
28609 235/300 Write block 3751 len 28672 data: 3 3 2 2 2 2 2
28610 236/300 Write block 3476 len 4096 data: 5
28611 237/300 Write block 867 len 28672 data: 6 6 6 6 4 6 6
28612 238/300 Read block 4966 len 16384
28613 239/300 Read block 2066 len 16384
28614 240/300 Write block 558 len 32768 data: 4 4 4 4 5 5 5 5
28615 241/300 Read block 954 len 8192
28616 242/300 Read block 4335 len 16384
28617 243/300 Read block 2250 len 36864
28618 244/300 Write block 1301 len 32768 data: 4 4 4 6 6 5 5 6
28619 245/300 Read block 4939 len 40960
28620 246/300 Read block 3608 len 8192
28621 247/300 Write block 183 len 36864 data: 5 4 3 3 4 4 4 4 4
28622 248/300 Read block 517 len 4096
28623 249/300 Read block 4885 len 20480
28624 250/300 Read block 3010 len 4096
28625 251/300 Read block 303 len 16384
28626 252/300 Read block 434 len 24576
28627 253/300 Flush
28628 254/300 Read block 3117 len 40960
28629 255/300 Read block 851 len 12288
28630 256/300 Read block 2893 len 24576
28631 257/300 Write block 4191 len 4096 data: 7
28632 258/300 Read block 2920 len 20480
28633 259/300 Flush
28634 260/300 Read block 2127 len 32768
28635 261/300 Write block 1504 len 40960 data: 6 4 4 4 4 3 6 7 5 4
28636 262/300 Write block 3254 len 36864 data: 4 6 6 4 6 6 5 5 5
28637 263/300 Write block 1959 len 4096 data: 4
28638 264/300 Write block 4706 len 36864 data: 8 9 10 8 7 8 8 7 5
28639 265/300 Write block 702 len 28672 data: 3 4 5 2 2 3 6
28640 266/300 Write block 610 len 12288 data: 5 6 8
28641 267/300 Read block 4514 len 36864
28642 268/300 Read block 2495 len 40960
28643 269/300 Read block 2059 len 8192
28644 270/300 Read block 1496 len 20480
28645 271/300 Write block 791 len 36864 data: 9 7 6 6 6 5 6 6 7
28646 272/300 Write block 185 len 28672 data: 4 4 5 5 5 5 5
28647 273/300 Read block 3410 len 24576
28648 274/300 Write block 364 len 16384 data: 6 6 7 6
28649 275/300 Write block 2302 len 40960 data: 7 7 8 7 8 9 10 10 9 7
28650 276/300 Read block 2652 len 24576
28651 277/300 Flush
28652 278/300 Flush
28653 279/300 Read block 3190 len 40960
28654 280/300 Write block 3355 len 36864 data: 6 5 5 6 7 8 10 9 9
28655 281/300 Read block 575 len 36864
28656 282/300 Read block 3820 len 24576
28657 283/300 Read block 1360 len 32768
28658 284/300 Flush
28659 285/300 Write block 1086 len 4096 data: 4
28660 286/300 Write block 745 len 24576 data: 6 6 6 6 6 7
28661 287/300 Read block 420 len 20480
28662 288/300 Read block 3411 len 40960
28663 289/300 Read block 3419 len 16384
28664 290/300 Read block 4218 len 40960
28665 291/300 Write block 515 len 20480 data: 5 4 4 4 6
28666 292/300 Write block 3365 len 20480 data: 6 7 9 8 6
28667 293/300 Read block 3716 len 32768
28668 294/300 Read block 2948 len 32768
28669 295/300 Write block 810 len 16384 data: 4 5 7 6
28670 296/300 Read block 4385 len 40960
28671 297/300 Flush
28672 298/300 Read block 1926 len 20480
28673 299/300 Read block 213 len 40960
28674 300/300 Read block 1198 len 4096
28675 301/300 Write block 3197 len 8192 data: 7 7
28676 Main task has work to do, go find it
28677 got dsc Start(2)
28678 start 2
28679 Jan 03 08:36:55.873 INFO request completed, latency_us: 229, response_code: 204, uri: /start/cid/2, method: POST, req_id: 49fa68b9-1cd4-41b3-90a3-4b9f7da3180e, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
28680 Replay: started 2, returned:Ok(())
28681 ----------------------------------------------------------------
28682 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28683 States: Active Active Offline
28684 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28685 8246 Acked Read 1 Done Done Sent false
28686 8247 Acked Write 6 Done Done Sent false
28687 8248 Acked Read 10 Done Done Sent false
28688 8249 Acked Read 6 Done Done Sent false
28689 8250 Acked Read 4 Done Done Sent false
28690 8251 Acked Read 3 Done Done Sent false
28691 8252 Acked Write 6 Done Done Sent false
28692 8253 Acked Write 2 Done Done Sent false
28693 8254 Acked Flush 0 Done Done Sent false
28694 8255 Acked Write 9 Done Done Sent false
28695 8256 Acked Read 10 Done Done Sent false
28696 8257 Acked Read 10 Done Done Sent false
28697 8258 Acked Write 3 Done Done Sent false
28698 8259 Acked Read 6 Done Done Sent false
28699 8260 Acked Read 9 Done Done Sent false
28700 8261 Acked Write 7 Done Done Sent false
28701 8262 Acked Write 6 Done Done Sent false
28702 8263 Acked Read 6 Done Done Sent false
28703 8264 Acked Read 8 Done Done Sent false
28704 8265 Acked Write 9 Done Done Sent false
28705 8266 Acked Write 2 Done Done Sent false
28706 8267 Acked Write 5 Done Done Sent false
28707 8268 Acked Flush 0 Done Done Sent false
28708 8269 Acked Read 8 Done Done Sent false
28709 8270 Acked Write 4 Done Done Sent false
28710 8271 Acked Write 1 Done Done Sent false
28711 8272 Acked Read 6 Done Done Sent false
28712 8273 Acked Read 7 Done Done Sent false
28713 8274 Acked Read 1 Done Done Sent false
28714 8275 Acked Write 6 Done Done Sent false
28715 8276 Acked Write 3 Done Done Sent false
28716 8277 Acked Read 1 Done Done Sent false
28717 8278 Acked Flush 0 Done Done Sent false
28718 8279 Acked Write 3 Done Done Sent false
28719 8280 Acked Read 6 Done Done Sent false
28720 8281 Acked Write 5 Done Done Sent false
28721 8282 Acked Write 5 Done Done Sent false
28722 8283 Acked Write 8 Done Done Sent false
28723 8284 Acked Write 3 Done Done Sent false
28724 8285 Acked Write 7 Done Done Sent false
28725 8286 Acked Read 2 Done Done Sent false
28726 8287 Acked Write 7 Done Done Sent false
28727 8288 Acked Read 9 Done Done Sent false
28728 8289 Acked Read 6 Done Done Sent false
28729 8290 Acked Read 1 Done Done Sent false
28730 8291 Acked Read 8 Done Done Sent false
28731 8292 Acked Read 5 Done Done Sent false
28732 8293 Acked Write 5 Done Done Sent false
28733 8294 Acked Read 8 Done Done Sent false
28734 8295 Acked Write 8 Done Done Sent false
28735 8296 Acked Write 2 Done Done Sent false
28736 8297 Acked Read 3 Done Done Sent false
28737 8298 Acked Read 1 Done Done Sent false
28738 8299 Acked Read 10 Done Done Sent false
28739 8300 Acked Write 7 Done Done Sent false
28740 8301 Acked Flush 0 Done Done Sent false
28741 8302 Acked Read 7 Done Done Sent false
28742 8303 Acked Write 2 Done Done Sent false
28743 8304 Acked Write 7 Done Done Sent false
28744 8305 Acked Read 5 Done Done Sent false
28745 8306 Acked Read 8 Done Done Sent false
28746 8307 Acked Write 7 Done Done Sent false
28747 8308 Acked Read 4 Done Done Sent false
28748 8309 Acked Read 7 Done Done Sent false
28749 8310 Acked Write 5 Done Done Sent false
28750 8311 Acked Write 4 Done Done Sent false
28751 8312 Acked Read 3 Done Done Sent false
28752 8313 Acked Flush 0 Done Done Sent false
28753 8314 Acked Read 10 Done Done Sent false
28754 8315 Acked Write 2 Done Done Sent false
28755 8316 Acked Read 8 Done Done Sent false
28756 8317 Acked Write 3 Done Done Sent false
28757 8318 Acked Write 3 Done Done Sent false
28758 8319 Acked Write 4 Done Done Sent false
28759 8320 Acked Write 7 Done Done Sent false
28760 8321 Acked Read 5 Done Done Sent false
28761 8322 Acked Read 7 Done Done Sent false
28762 8323 Acked Write 10 Done Done Sent false
28763 8324 Acked Read 7 Done Done Sent false
28764 8325 Acked Write 8 Done Done Sent false
28765 8326 Acked Flush 0 Done Done Sent false
28766 8327 Acked Write 2 Done Done Sent false
28767 8328 Acked Read 1 Done Done Sent false
28768 8329 Acked Read 9 Done Done Sent false
28769 8330 Acked Write 1 Done Done Sent false
28770 8331 Acked Write 6 Done Done Sent false
28771 8332 Acked Write 7 Done Done Sent false
28772 8333 Acked Read 5 Done Done Sent false
28773 8334 Acked Read 5 Done Done Sent false
28774 8335 Acked Read 3 Done Done Sent false
28775 8336 Acked Read 2 Done Done Sent false
28776 8337 Acked Write 7 Done Done Sent false
28777 8338 Acked Read 2 Done Done Sent false
28778 8339 Acked Read 2 Done Done Sent false
28779 8340 Acked Read 1 Done Done Sent false
28780 8341 Acked Read 8 Done Done Sent false
28781 8342 Acked Write 10 Done Done Sent false
28782 8343 Acked Read 4 Done Done Sent false
28783 8344 Acked Write 1 Done Done Sent false
28784 8345 Acked Read 8 Done Done Sent false
28785 8346 Acked Read 10 Done Done Sent false
28786 8347 Acked Flush 0 Done Done Sent false
28787 8348 Acked Read 9 Done Done Sent false
28788 8349 Acked Read 2 Done Done Sent false
28789 8350 Acked Read 7 Done Done Sent false
28790 8351 Acked Read 3 Done Done Sent false
28791 8352 Acked Read 3 Done Done Sent false
28792 8353 Acked Write 5 Done Done Sent false
28793 8354 Acked Read 6 Done Done Sent false
28794 8355 Acked Read 8 Done Done Sent false
28795 8356 Acked Read 5 Done Done Sent false
28796 8357 Acked Read 10 Done Done Sent false
28797 8358 Acked Write 8 Done Done Sent false
28798 8359 Acked Read 9 Done Done Sent false
28799 8360 Acked Read 9 Done Done Sent false
28800 8361 Acked Write 9 Done Done Sent false
28801 8362 Acked Read 6 Done Done Sent false
28802 8363 Acked Write 9 Done Done Sent false
28803 8364 Acked Read 1 Done Done Sent false
28804 8365 Acked Write 10 Done Done Sent false
28805 8366 Acked Read 8 Done Done Sent false
28806 8367 Acked Read 8 Done Done Sent false
28807 8368 Acked Write 4 Done Done Sent false
28808 8369 Acked Read 9 Done Done Sent false
28809 8370 Acked Read 10 Done Done Sent false
28810 8371 Acked Write 4 Done Done Sent false
28811 8372 Acked Read 5 Done Done Sent false
28812 8373 Acked Write 6 Done Done Sent false
28813 8374 Acked Write 6 Done Done Sent false
28814 8375 Acked Write 4 Done Done Sent false
28815 8376 Acked Read 8 Done Done Sent false
28816 8377 Acked Write 8 Done Done Sent false
28817 8378 Acked Write 6 Done Done Sent false
28818 8379 Acked Read 3 Done Done Sent false
28819 8380 Acked Read 1 Done Done Sent false
28820 8381 Acked Read 3 Done Done Sent false
28821 8382 Acked Write 6 Done Done Sent false
28822 8383 Acked Write 9 Done Done Sent false
28823 8384 Acked Read 3 Done Done Sent false
28824 8385 Acked Read 4 Done Done Sent false
28825 8386 Acked Flush 0 Done Done Sent false
28826 8387 Acked Read 8 Done Done Sent false
28827 8388 Acked Read 5 Done Done Sent false
28828 8389 Acked Read 1 Done Done Sent false
28829 8390 Acked Write 7 Done Done Sent false
28830 8391 Acked Read 9 Done Done Sent false
28831 8392 Acked Read 9 Done Done Sent false
28832 8393 Acked Read 3 Done Done Sent false
28833 8394 Acked Write 10 Done Done Sent false
28834 8395 Acked Write 8 Done Done Sent false
28835 8396 Acked Read 1 Done Done Sent false
28836 8397 Acked Write 2 Done Done Sent false
28837 8398 Acked Read 8 Done Done Sent false
28838 8399 Acked Write 2 Done Done Sent false
28839 8400 Acked Read 6 Done Done Sent false
28840 8401 Acked Write 3 Done Done Sent false
28841 8402 Acked Read 5 Done Done Sent false
28842 8403 Acked Read 5 Done Done Sent false
28843 8404 Acked Write 9 Done Done Sent false
28844 8405 Acked Write 10 Done Done Sent false
28845 8406 Acked Flush 0 Done Done Sent false
28846 8407 Acked Write 3 Done Done Sent false
28847 8408 Acked Read 9 Done Done Sent false
28848 8409 Acked Write 9 Done Done Sent false
28849 8410 Acked Write 8 Done Done Sent false
28850 8411 Acked Read 10 Done Done Sent false
28851 8412 Acked Write 9 Done Done Sent false
28852 8413 Acked Read 10 Done Done Sent false
28853 8414 Acked Read 2 Done Done Sent false
28854 8415 Acked Read 6 Done Done Sent false
28855 8416 Acked Read 5 Done Done Sent false
28856 8417 Acked Write 6 Done Done Sent false
28857 8418 Acked Read 5 Done Done Sent false
28858 8419 Acked Read 8 Done Done Sent false
28859 8420 Acked Read 3 Done Done Sent false
28860 8421 Acked Read 10 Done Done Sent false
28861 8422 Acked Read 6 Done Done Sent false
28862 8423 Acked Read 1 Done Done Sent false
28863 8424 Acked Write 4 Done Done Sent false
28864 8425 Acked Write 7 Done Done Sent false
28865 8426 Acked Write 1 Done Done Sent false
28866 8427 Acked Read 6 Done Done Sent false
28867 8428 Acked Read 4 Done Done Sent false
28868 8429 Acked Write 7 Done Done Sent false
28869 8430 Acked Read 5 Done Done Sent false
28870 8431 Acked Read 8 Done Done Sent false
28871 8432 Acked Read 10 Done Done Sent false
28872 8433 Acked Read 10 Done Done Sent false
28873 8434 Acked Write 5 Done Done Sent false
28874 8435 Acked Write 9 Done Done Sent false
28875 8436 Acked Read 4 Done Done Sent false
28876 8437 Acked Write 1 Done Done Sent false
28877 8438 Acked Flush 0 Done Done Sent false
28878 8439 Acked Read 3 Done Done Sent false
28879 8440 Acked Write 1 Done Done Sent false
28880 8441 Acked Read 9 Done Done Sent false
28881 8442 Acked Write 8 Done Done Sent false
28882 8443 Acked Write 2 Done Done Sent false
28883 8444 Acked Read 6 Done Done Sent false
28884 8445 Acked Read 3 Done Done Sent false
28885 8446 Acked Read 1 Done Done Sent false
28886 8447 Acked Write 6 Done Done Sent false
28887 8448 Acked Write 6 Done Done Sent false
28888 8449 Acked Read 7 Done Done Sent false
28889 8450 Acked Read 2 Done Done Sent false
28890 8451 Acked Flush 0 Done Done Sent false
28891 8452 Acked Write 7 Done Done Sent false
28892 8453 Acked Read 1 Done Done Sent false
28893 8454 Acked Flush 0 Done Done Sent false
28894 8455 Acked Read 3 Done Done Sent false
28895 8456 Acked Write 7 Done Done Sent false
28896 8457 Acked Read 5 Done Done Sent false
28897 8458 Acked Write 3 Done Done Sent false
28898 8459 Acked Read 1 Done Done Sent false
28899 8460 Acked Read 10 Done Done Sent false
28900 8461 Acked Write 8 Done Done Sent false
28901 8462 Acked Read 1 Done Done Sent false
28902 8463 Acked Write 7 Done Done Sent false
28903 8464 Acked Read 7 Done Done Sent false
28904 8465 Acked Flush 0 Done Done Sent false
28905 8466 Acked Write 4 Done Done Sent false
28906 8467 Acked Write 1 Done Done Sent false
28907 8468 Acked Read 4 Done Done Sent false
28908 8469 Acked Write 8 Done Done Sent false
28909 8470 Acked Write 1 Done Done Sent false
28910 8471 Acked Read 8 Done Done Sent false
28911 8472 Acked Read 8 Done Done Sent false
28912 8473 Acked Write 3 Done Done Sent false
28913 8474 Acked Write 9 Done Done Sent false
28914 8475 Acked Write 10 Done Done Sent false
28915 8476 Acked Read 5 Done Done Sent false
28916 8477 Acked Read 10 Done Done Sent false
28917 8478 Acked Read 6 Done Done Sent false
28918 8479 Acked Write 10 Done Done Sent false
28919 8480 Acked Write 7 Done Done Sent false
28920 8481 Acked Write 1 Done Done Sent false
28921 8482 Acked Write 7 Done Done Sent false
28922 8483 Acked Read 4 Done Done Sent false
28923 8484 Acked Read 4 Done Done Sent false
28924 8485 Acked Write 8 Done Done Sent false
28925 8486 Acked Read 2 Done Done Sent false
28926 8487 Acked Read 4 Done Done Sent false
28927 8488 Acked Read 9 Done Done Sent false
28928 8489 Acked Write 8 Done Done Sent false
28929 8490 Acked Read 10 Done Done Sent false
28930 8491 Acked Read 2 Done Done Sent false
28931 8492 Acked Write 9 Done Done Sent false
28932 8493 Acked Read 1 Done Done Sent false
28933 8494 Acked Read 5 Done Done Sent false
28934 8495 Acked Read 1 Done Done Sent false
28935 8496 Acked Read 4 Done Done Sent false
28936 8497 Acked Read 6 Done Done Sent false
28937 8498 Acked Flush 0 Done Done Sent false
28938 8499 Acked Read 10 Done Done Sent false
28939 8500 Acked Read 3 Done Done Sent false
28940 8501 Acked Read 6 Done Done Sent false
28941 8502 Acked Write 1 Done Done Sent false
28942 8503 Acked Read 5 Done Done Sent false
28943 8504 Acked Flush 0 Done Done Sent false
28944 8505 Acked Read 8 Done Done Sent false
28945 8506 Acked Write 10 Done Done Sent false
28946 8507 Acked Write 9 Done Done Sent false
28947 8508 Acked Write 1 Done Done Sent false
28948 8509 Acked Write 9 Done Done Sent false
28949 8510 Acked Write 7 Done Done Sent false
28950 8511 Acked Write 3 Done Done Sent false
28951 8512 Acked Read 9 Done Done Sent false
28952 8513 Acked Read 10 Done Done Sent false
28953 8514 Acked Read 2 Done Done Sent false
28954 8515 Acked Read 5 Done Done Sent false
28955 8516 Acked Write 9 Done Done Sent false
28956 8517 Acked Write 7 Done Done Sent false
28957 8518 Acked Read 6 Done Done Sent false
28958 8519 Acked Write 4 Done Done Sent false
28959 8520 Acked Write 10 Done Done Sent false
28960 8521 Acked Read 6 Done Done Sent false
28961 8522 Acked Flush 0 Done Done Sent false
28962 8523 Acked Flush 0 Done Done Sent false
28963 8524 Acked Read 10 Done Done Sent false
28964 8525 Acked Write 9 Done Done Sent false
28965 8526 Acked Read 9 Done Done Sent false
28966 8527 Acked Read 6 Done Done Sent false
28967 8528 Acked Read 8 Done Done Sent false
28968 8529 Acked Flush 0 Done Done Sent false
28969 8530 Acked Write 1 Done Done Sent false
28970 8531 Acked Write 6 Done Done Sent false
28971 8532 Acked Read 5 Done Done Sent false
28972 8533 Acked Read 10 Done Done Sent false
28973 8534 Acked Read 4 Done Done Sent false
28974 8535 Acked Read 10 Done Done Sent false
28975 8536 Acked Write 5 Done Done Sent false
28976 8537 Acked Write 5 Done Done Sent false
28977 8538 Acked Read 8 Done Done Sent false
28978 8539 Acked Read 8 Done Done Sent false
28979 8540 Acked Write 4 Done Done Sent false
28980 8541 Acked Read 10 Done Done Sent false
28981 8542 Acked Flush 0 Done Done Sent false
28982 8543 Acked Read 5 Done Done Sent false
28983 8544 Acked Read 10 Done Done Sent false
28984 8545 Acked Read 1 Done Done Sent false
28985 8546 Acked Write 2 Done Done Sent false
28986 STATES DS:0 DS:1 DS:2 TOTAL
28987 Sent 0 0 301 301
28988 Done 301 301 0 602
28989 Skipped 0 0 0 0
28990 Error 0 0 0 0
28991 Last Flush: 8542 8542 8245
28992 Downstairs last five completed: 8245 8244 8243 8242 8241
28993 Upstairs last five completed: 8546 8545 8544 8543 8542
28994 CLIENT: Up:0 ds:301 act:2
28995 [8830] got start action
28996 [8830] I am going to restart
28997 Starting downstairs at port 8830
28998 Make output file at "/tmp/dsc/downstairs-8830.txt"
28999 [8830][2] reports Starting
29000 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1230)
29001 [8830][2] reports Running
29002 ----------------------------------------------------------------
29003 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29004 States: Active Active Offline
29005 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29006 8246 Acked Read 1 Done Done Sent false
29007 8247 Acked Write 6 Done Done Sent false
29008 8248 Acked Read 10 Done Done Sent false
29009 8249 Acked Read 6 Done Done Sent false
29010 8250 Acked Read 4 Done Done Sent false
29011 8251 Acked Read 3 Done Done Sent false
29012 8252 Acked Write 6 Done Done Sent false
29013 8253 Acked Write 2 Done Done Sent false
29014 8254 Acked Flush 0 Done Done Sent false
29015 8255 Acked Write 9 Done Done Sent false
29016 8256 Acked Read 10 Done Done Sent false
29017 8257 Acked Read 10 Done Done Sent false
29018 8258 Acked Write 3 Done Done Sent false
29019 8259 Acked Read 6 Done Done Sent false
29020 8260 Acked Read 9 Done Done Sent false
29021 8261 Acked Write 7 Done Done Sent false
29022 8262 Acked Write 6 Done Done Sent false
29023 8263 Acked Read 6 Done Done Sent false
29024 8264 Acked Read 8 Done Done Sent false
29025 8265 Acked Write 9 Done Done Sent false
29026 8266 Acked Write 2 Done Done Sent false
29027 8267 Acked Write 5 Done Done Sent false
29028 8268 Acked Flush 0 Done Done Sent false
29029 8269 Acked Read 8 Done Done Sent false
29030 8270 Acked Write 4 Done Done Sent false
29031 8271 Acked Write 1 Done Done Sent false
29032 8272 Acked Read 6 Done Done Sent false
29033 8273 Acked Read 7 Done Done Sent false
29034 8274 Acked Read 1 Done Done Sent false
29035 8275 Acked Write 6 Done Done Sent false
29036 8276 Acked Write 3 Done Done Sent false
29037 8277 Acked Read 1 Done Done Sent false
29038 8278 Acked Flush 0 Done Done Sent false
29039 8279 Acked Write 3 Done Done Sent false
29040 8280 Acked Read 6 Done Done Sent false
29041 8281 Acked Write 5 Done Done Sent false
29042 8282 Acked Write 5 Done Done Sent false
29043 8283 Acked Write 8 Done Done Sent false
29044 8284 Acked Write 3 Done Done Sent false
29045 8285 Acked Write 7 Done Done Sent false
29046 8286 Acked Read 2 Done Done Sent false
29047 8287 Acked Write 7 Done Done Sent false
29048 8288 Acked Read 9 Done Done Sent false
29049 8289 Acked Read 6 Done Done Sent false
29050 8290 Acked Read 1 Done Done Sent false
29051 8291 Acked Read 8 Done Done Sent false
29052 8292 Acked Read 5 Done Done Sent false
29053 8293 Acked Write 5 Done Done Sent false
29054 8294 Acked Read 8 Done Done Sent false
29055 8295 Acked Write 8 Done Done Sent false
29056 8296 Acked Write 2 Done Done Sent false
29057 8297 Acked Read 3 Done Done Sent false
29058 8298 Acked Read 1 Done Done Sent false
29059 8299 Acked Read 10 Done Done Sent false
29060 8300 Acked Write 7 Done Done Sent false
29061 8301 Acked Flush 0 Done Done Sent false
29062 8302 Acked Read 7 Done Done Sent false
29063 8303 Acked Write 2 Done Done Sent false
29064 8304 Acked Write 7 Done Done Sent false
29065 8305 Acked Read 5 Done Done Sent false
29066 8306 Acked Read 8 Done Done Sent false
29067 8307 Acked Write 7 Done Done Sent false
29068 8308 Acked Read 4 Done Done Sent false
29069 8309 Acked Read 7 Done Done Sent false
29070 8310 Acked Write 5 Done Done Sent false
29071 8311 Acked Write 4 Done Done Sent false
29072 8312 Acked Read 3 Done Done Sent false
29073 8313 Acked Flush 0 Done Done Sent false
29074 8314 Acked Read 10 Done Done Sent false
29075 8315 Acked Write 2 Done Done Sent false
29076 8316 Acked Read 8 Done Done Sent false
29077 8317 Acked Write 3 Done Done Sent false
29078 8318 Acked Write 3 Done Done Sent false
29079 8319 Acked Write 4 Done Done Sent false
29080 8320 Acked Write 7 Done Done Sent false
29081 8321 Acked Read 5 Done Done Sent false
29082 8322 Acked Read 7 Done Done Sent false
29083 8323 Acked Write 10 Done Done Sent false
29084 8324 Acked Read 7 Done Done Sent false
29085 8325 Acked Write 8 Done Done Sent false
29086 8326 Acked Flush 0 Done Done Sent false
29087 8327 Acked Write 2 Done Done Sent false
29088 8328 Acked Read 1 Done Done Sent false
29089 8329 Acked Read 9 Done Done Sent false
29090 8330 Acked Write 1 Done Done Sent false
29091 8331 Acked Write 6 Done Done Sent false
29092 8332 Acked Write 7 Done Done Sent false
29093 8333 Acked Read 5 Done Done Sent false
29094 8334 Acked Read 5 Done Done Sent false
29095 8335 Acked Read 3 Done Done Sent false
29096 8336 Acked Read 2 Done Done Sent false
29097 8337 Acked Write 7 Done Done Sent false
29098 8338 Acked Read 2 Done Done Sent false
29099 8339 Acked Read 2 Done Done Sent false
29100 8340 Acked Read 1 Done Done Sent false
29101 8341 Acked Read 8 Done Done Sent false
29102 8342 Acked Write 10 Done Done Sent false
29103 8343 Acked Read 4 Done Done Sent false
29104 8344 Acked Write 1 Done Done Sent false
29105 8345 Acked Read 8 Done Done Sent false
29106 8346 Acked Read 10 Done Done Sent false
29107 8347 Acked Flush 0 Done Done Sent false
29108 8348 Acked Read 9 Done Done Sent false
29109 8349 Acked Read 2 Done Done Sent false
29110 8350 Acked Read 7 Done Done Sent false
29111 8351 Acked Read 3 Done Done Sent false
29112 8352 Acked Read 3 Done Done Sent false
29113 8353 Acked Write 5 Done Done Sent false
29114 8354 Acked Read 6 Done Done Sent false
29115 8355 Acked Read 8 Done Done Sent false
29116 8356 Acked Read 5 Done Done Sent false
29117 8357 Acked Read 10 Done Done Sent false
29118 8358 Acked Write 8 Done Done Sent false
29119 8359 Acked Read 9 Done Done Sent false
29120 8360 Acked Read 9 Done Done Sent false
29121 8361 Acked Write 9 Done Done Sent false
29122 8362 Acked Read 6 Done Done Sent false
29123 8363 Acked Write 9 Done Done Sent false
29124 8364 Acked Read 1 Done Done Sent false
29125 8365 Acked Write 10 Done Done Sent false
29126 8366 Acked Read 8 Done Done Sent false
29127 8367 Acked Read 8 Done Done Sent false
29128 8368 Acked Write 4 Done Done Sent false
29129 8369 Acked Read 9 Done Done Sent false
29130 8370 Acked Read 10 Done Done Sent false
29131 8371 Acked Write 4 Done Done Sent false
29132 8372 Acked Read 5 Done Done Sent false
29133 8373 Acked Write 6 Done Done Sent false
29134 8374 Acked Write 6 Done Done Sent false
29135 8375 Acked Write 4 Done Done Sent false
29136 8376 Acked Read 8 Done Done Sent false
29137 8377 Acked Write 8 Done Done Sent false
29138 8378 Acked Write 6 Done Done Sent false
29139 8379 Acked Read 3 Done Done Sent false
29140 8380 Acked Read 1 Done Done Sent false
29141 8381 Acked Read 3 Done Done Sent false
29142 8382 Acked Write 6 Done Done Sent false
29143 8383 Acked Write 9 Done Done Sent false
29144 8384 Acked Read 3 Done Done Sent false
29145 8385 Acked Read 4 Done Done Sent false
29146 8386 Acked Flush 0 Done Done Sent false
29147 8387 Acked Read 8 Done Done Sent false
29148 8388 Acked Read 5 Done Done Sent false
29149 8389 Acked Read 1 Done Done Sent false
29150 8390 Acked Write 7 Done Done Sent false
29151 8391 Acked Read 9 Done Done Sent false
29152 8392 Acked Read 9 Done Done Sent false
29153 8393 Acked Read 3 Done Done Sent false
29154 8394 Acked Write 10 Done Done Sent false
29155 8395 Acked Write 8 Done Done Sent false
29156 8396 Acked Read 1 Done Done Sent false
29157 8397 Acked Write 2 Done Done Sent false
29158 8398 Acked Read 8 Done Done Sent false
29159 8399 Acked Write 2 Done Done Sent false
29160 8400 Acked Read 6 Done Done Sent false
29161 8401 Acked Write 3 Done Done Sent false
29162 8402 Acked Read 5 Done Done Sent false
29163 8403 Acked Read 5 Done Done Sent false
29164 8404 Acked Write 9 Done Done Sent false
29165 8405 Acked Write 10 Done Done Sent false
29166 8406 Acked Flush 0 Done Done Sent false
29167 8407 Acked Write 3 Done Done Sent false
29168 8408 Acked Read 9 Done Done Sent false
29169 8409 Acked Write 9 Done Done Sent false
29170 8410 Acked Write 8 Done Done Sent false
29171 8411 Acked Read 10 Done Done Sent false
29172 8412 Acked Write 9 Done Done Sent false
29173 8413 Acked Read 10 Done Done Sent false
29174 8414 Acked Read 2 Done Done Sent false
29175 8415 Acked Read 6 Done Done Sent false
29176 8416 Acked Read 5 Done Done Sent false
29177 8417 Acked Write 6 Done Done Sent false
29178 8418 Acked Read 5 Done Done Sent false
29179 8419 Acked Read 8 Done Done Sent false
29180 8420 Acked Read 3 Done Done Sent false
29181 8421 Acked Read 10 Done Done Sent false
29182 8422 Acked Read 6 Done Done Sent false
29183 8423 Acked Read 1 Done Done Sent false
29184 8424 Acked Write 4 Done Done Sent false
29185 8425 Acked Write 7 Done Done Sent false
29186 8426 Acked Write 1 Done Done Sent false
29187 8427 Acked Read 6 Done Done Sent false
29188 8428 Acked Read 4 Done Done Sent false
29189 8429 Acked Write 7 Done Done Sent false
29190 8430 Acked Read 5 Done Done Sent false
29191 8431 Acked Read 8 Done Done Sent false
29192 8432 Acked Read 10 Done Done Sent false
29193 8433 Acked Read 10 Done Done Sent false
29194 8434 Acked Write 5 Done Done Sent false
29195 8435 Acked Write 9 Done Done Sent false
29196 8436 Acked Read 4 Done Done Sent false
29197 8437 Acked Write 1 Done Done Sent false
29198 8438 Acked Flush 0 Done Done Sent false
29199 8439 Acked Read 3 Done Done Sent false
29200 8440 Acked Write 1 Done Done Sent false
29201 8441 Acked Read 9 Done Done Sent false
29202 8442 Acked Write 8 Done Done Sent false
29203 8443 Acked Write 2 Done Done Sent false
29204 8444 Acked Read 6 Done Done Sent false
29205 8445 Acked Read 3 Done Done Sent false
29206 8446 Acked Read 1 Done Done Sent false
29207 8447 Acked Write 6 Done Done Sent false
29208 8448 Acked Write 6 Done Done Sent false
29209 8449 Acked Read 7 Done Done Sent false
29210 8450 Acked Read 2 Done Done Sent false
29211 8451 Acked Flush 0 Done Done Sent false
29212 8452 Acked Write 7 Done Done Sent false
29213 8453 Acked Read 1 Done Done Sent false
29214 8454 Acked Flush 0 Done Done Sent false
29215 8455 Acked Read 3 Done Done Sent false
29216 8456 Acked Write 7 Done Done Sent false
29217 8457 Acked Read 5 Done Done Sent false
29218 8458 Acked Write 3 Done Done Sent false
29219 8459 Acked Read 1 Done Done Sent false
29220 8460 Acked Read 10 Done Done Sent false
29221 8461 Acked Write 8 Done Done Sent false
29222 8462 Acked Read 1 Done Done Sent false
29223 8463 Acked Write 7 Done Done Sent false
29224 8464 Acked Read 7 Done Done Sent false
29225 8465 Acked Flush 0 Done Done Sent false
29226 8466 Acked Write 4 Done Done Sent false
29227 8467 Acked Write 1 Done Done Sent false
29228 8468 Acked Read 4 Done Done Sent false
29229 8469 Acked Write 8 Done Done Sent false
29230 8470 Acked Write 1 Done Done Sent false
29231 8471 Acked Read 8 Done Done Sent false
29232 8472 Acked Read 8 Done Done Sent false
29233 8473 Acked Write 3 Done Done Sent false
29234 8474 Acked Write 9 Done Done Sent false
29235 8475 Acked Write 10 Done Done Sent false
29236 8476 Acked Read 5 Done Done Sent false
29237 8477 Acked Read 10 Done Done Sent false
29238 8478 Acked Read 6 Done Done Sent false
29239 8479 Acked Write 10 Done Done Sent false
29240 8480 Acked Write 7 Done Done Sent false
29241 8481 Acked Write 1 Done Done Sent false
29242 8482 Acked Write 7 Done Done Sent false
29243 8483 Acked Read 4 Done Done Sent false
29244 8484 Acked Read 4 Done Done Sent false
29245 8485 Acked Write 8 Done Done Sent false
29246 8486 Acked Read 2 Done Done Sent false
29247 8487 Acked Read 4 Done Done Sent false
29248 8488 Acked Read 9 Done Done Sent false
29249 8489 Acked Write 8 Done Done Sent false
29250 8490 Acked Read 10 Done Done Sent false
29251 8491 Acked Read 2 Done Done Sent false
29252 8492 Acked Write 9 Done Done Sent false
29253 8493 Acked Read 1 Done Done Sent false
29254 8494 Acked Read 5 Done Done Sent false
29255 8495 Acked Read 1 Done Done Sent false
29256 8496 Acked Read 4 Done Done Sent false
29257 8497 Acked Read 6 Done Done Sent false
29258 8498 Acked Flush 0 Done Done Sent false
29259 8499 Acked Read 10 Done Done Sent false
29260 8500 Acked Read 3 Done Done Sent false
29261 8501 Acked Read 6 Done Done Sent false
29262 8502 Acked Write 1 Done Done Sent false
29263 8503 Acked Read 5 Done Done Sent false
29264 8504 Acked Flush 0 Done Done Sent false
29265 8505 Acked Read 8 Done Done Sent false
29266 8506 Acked Write 10 Done Done Sent false
29267 8507 Acked Write 9 Done Done Sent false
29268 8508 Acked Write 1 Done Done Sent false
29269 8509 Acked Write 9 Done Done Sent false
29270 8510 Acked Write 7 Done Done Sent false
29271 8511 Acked Write 3 Done Done Sent false
29272 8512 Acked Read 9 Done Done Sent false
29273 8513 Acked Read 10 Done Done Sent false
29274 8514 Acked Read 2 Done Done Sent false
29275 8515 Acked Read 5 Done Done Sent false
29276 8516 Acked Write 9 Done Done Sent false
29277 8517 Acked Write 7 Done Done Sent false
29278 8518 Acked Read 6 Done Done Sent false
29279 8519 Acked Write 4 Done Done Sent false
29280 8520 Acked Write 10 Done Done Sent false
29281 8521 Acked Read 6 Done Done Sent false
29282 8522 Acked Flush 0 Done Done Sent false
29283 8523 Acked Flush 0 Done Done Sent false
29284 8524 Acked Read 10 Done Done Sent false
29285 8525 Acked Write 9 Done Done Sent false
29286 8526 Acked Read 9 Done Done Sent false
29287 8527 Acked Read 6 Done Done Sent false
29288 8528 Acked Read 8 Done Done Sent false
29289 8529 Acked Flush 0 Done Done Sent false
29290 8530 Acked Write 1 Done Done Sent false
29291 8531 Acked Write 6 Done Done Sent false
29292 8532 Acked Read 5 Done Done Sent false
29293 8533 Acked Read 10 Done Done Sent false
29294 8534 Acked Read 4 Done Done Sent false
29295 8535 Acked Read 10 Done Done Sent false
29296 8536 Acked Write 5 Done Done Sent false
29297 8537 Acked Write 5 Done Done Sent false
29298 8538 Acked Read 8 Done Done Sent false
29299 8539 Acked Read 8 Done Done Sent false
29300 8540 Acked Write 4 Done Done Sent false
29301 8541 Acked Read 10 Done Done Sent false
29302 8542 Acked Flush 0 Done Done Sent false
29303 8543 Acked Read 5 Done Done Sent false
29304 8544 Acked Read 10 Done Done Sent false
29305 8545 Acked Read 1 Done Done Sent false
29306 8546 Acked Write 2 Done Done Sent false
29307 8547 Acked Flush 0 Done Done Sent false
29308 STATES DS:0 DS:1 DS:2 TOTAL
29309 Sent 0 0 302 302
29310 Done 302 302 0 604
29311 Skipped 0 0 0 0
29312 Error 0 0 0 0
29313 Last Flush: 8547 8547 8245
29314 Downstairs last five completed: 8245 8244 8243 8242 8241
29315 Upstairs last five completed: 8547 8546 8545 8544 8543
29316 CLIENT: Up:0 ds:302 act:2
293172025-01-03T08:37:01.619ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293182025-01-03T08:37:01.619ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50731) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293192025-01-03T08:37:01.620ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293202025-01-03T08:37:01.621ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293212025-01-03T08:37:01.621ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293222025-01-03T08:37:01.621ZINFOcrucible: send last flush ID to this DS: 8245 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293232025-01-03T08:37:01.621ZINFOcrucible: Replied this last flush ID: 8245 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293242025-01-03T08:37:01.621ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293252025-01-03T08:37:01.621ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293262025-01-03T08:37:01.621ZINFOcrucible: [2] client re-new 302 jobs since flush 8245 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293272025-01-03T08:37:01.621ZINFOcrucible: [2] Replayed 302 jobs since flush: 8245 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
29328 ----------------------------------------------------------------
29329 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29330 Downstairs last five completed: 8547 8546 8545 8544 8543
29331 Upstairs last five completed: 8547 8546 8545 8544 8543
29332 CLIENT: Up:0 ds:0 act:3
29333 Replay: All jobs finished
29334 Main task has work to do, go find it
29335 got dsc Stop(1)
29336 Jan 03 08:37:03.877 INFO request completed, latency_us: stop 1
29337 314, response_code: 204, uri: /stop/cid/1, method: POST, req_id: a65dbb13-c23c-44f3-a9e5-5892b536e9df, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
29338 [8820] Got stop action so:false kr:false
29339 Jan 03 08:37:03.878 INFO request completed, latency_us: 175, response_code: 200, uri: /state/cid/1, method: GET, req_id: 0ee8641f-2ec5-42b8-83b9-6eaee4f94e04, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
293402025-01-03T08:37:03.878ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293412025-01-03T08:37:03.878ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293422025-01-03T08:37:03.879ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293432025-01-03T08:37:03.879ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293442025-01-03T08:37:03.879ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
293452025-01-03T08:37:03.879ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
29346 [8820] Exited with: ExitStatus(unix_wait_status(9))
29347 [8820][1] reports Exit
29348 Jan 03 08:37:07.880 INFO request completed, latency_us: 205, response_code: 200, uri: /state/cid/1, method: GET, req_id: a528fc00-36bc-441a-8e9a-e88a7b764dd9, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
29349 001/300 Read block 3491 len 36864
29350 002/300 Flush
29351 003/300 Read block 4361 len 28672
29352 004/300 Write block 849 len 8192 data: 6 6
29353 005/300 Write block 1084 len 36864 data: 4 3 5 4 4 5 8 9 8
29354 006/300 Read block 4060 len 4096
29355 007/300 Read block 1119 len 40960
29356 008/300 Read block 1985 len 20480
29357 009/300 Write block 3199 len 40960 data: 7 7 5 5 4 5 5 6 6 6
29358 010/300 Write block 242 len 40960 data: 3 3 4 5 6 5 5 6 6 5
29359 011/300 Write block 2151 len 24576 data: 3 2 5 6 6 6
29360 012/300 Write block 3774 len 28672 data: 8 7 6 5 4 4 4
29361 013/300 Flush
29362 014/300 Read block 3853 len 12288
29363 015/300 Write block 284 len 8192 data: 5 5
29364 016/300 Write block 3608 len 16384 data: 8 3 4 4
29365 017/300 Write block 3647 len 36864 data: 4 4 4 4 5 4 7 6 6
29366 018/300 Read block 3737 len 20480
29367 019/300 Read block 3185 len 4096
29368 020/300 Read block 2581 len 8192
29369 021/300 Read block 735 len 4096
29370 022/300 Write block 554 len 12288 data: 6 6 7
29371 023/300 Flush
29372 024/300 Write block 4366 len 32768 data: 8 7 6 5 6 7 7 6
29373 025/300 Flush
29374 026/300 Write block 3374 len 12288 data: 4 5 2
29375 027/300 Read block 3656 len 28672
29376 028/300 Read block 3572 len 16384
29377 029/300 Read block 2736 len 12288
29378 030/300 Read block 2472 len 32768
29379 031/300 Read block 2687 len 16384
29380 032/300 Write block 3273 len 8192 data: 4 3
29381 033/300 Write block 4030 len 24576 data: 4 4 4 3 3 4
29382 034/300 Read block 4516 len 20480
29383 035/300 Write block 2318 len 20480 data: 7 6 6 6 6
29384 036/300 Write block 4289 len 40960 data: 3 4 4 4 5 5 7 6 7 6
29385 037/300 Read block 236 len 16384
29386 038/300 Write block 1235 len 4096 data: 2
29387 039/300 Read block 29 len 12288
29388 040/300 Write block 513 len 40960 data: 5 6 6 5 5 5 7 5 7 8
29389 041/300 Read block 132 len 4096
29390 042/300 Read block 470 len 40960
29391 043/300 Read block 4742 len 40960
29392 044/300 Write block 182 len 32768 data: 4 6 5 5 5 6 6 6
29393 045/300 Write block 454 len 32768 data: 5 5 4 5 7 7 6 7
29394 046/300 Read block 179 len 24576
29395 047/300 Read block 163 len 20480
29396 048/300 Read block 806 len 12288
29397 049/300 Read block 2949 len 32768
29398 050/300 Flush
29399 051/300 Flush
29400 052/300 Write block 3879 len 16384 data: 4 5 7 7
29401 053/300 Write block 3551 len 36864 data: 4 3 3 4 5 6 6 5 6
29402 054/300 Read block 874 len 20480
29403 055/300 Write block 2924 len 28672 data: 8 8 8 9 6 5 4
29404 056/300 Flush
29405 057/300 Read block 1665 len 8192
29406 058/300 Write block 3205 len 20480 data: 6 7 7 7 6
29407 059/300 Write block 1788 len 12288 data: 6 5 6
29408 060/300 Write block 821 len 8192 data: 5 5
29409 061/300 Read block 4149 len 24576
29410 062/300 Flush
29411 063/300 Write block 2942 len 4096 data: 5
29412 064/300 Write block 3042 len 28672 data: 6 8 9 10 10 9 8
29413 065/300 Read block 2123 len 24576
29414 066/300 Write block 4747 len 32768 data: 7 9 9 5 6 5 7 8
29415 067/300 Write block 2309 len 20480 data: 11 10 8 6 7
29416 068/300 Read block 3518 len 20480
29417 069/300 Read block 2528 len 4096
29418 070/300 Read block 2765 len 24576
29419 071/300 Flush
29420 072/300 Write block 4828 len 4096 data: 6
29421 073/300 Write block 537 len 16384 data: 4 5 6 5
29422 074/300 Read block 291 len 24576
29423 075/300 Write block 4477 len 20480 data: 6 7 6 4 4
29424 076/300 Read block 3967 len 32768
29425 077/300 Read block 3799 len 24576
29426 078/300 Write block 363 len 40960 data: 5 7 7 8 7 6 6 6 5 5
29427 079/300 Read block 4014 len 4096
29428 080/300 Write block 1018 len 24576 data: 4 4 4 4 4 4
29429 081/300 Write block 866 len 8192 data: 4 7
29430 082/300 Write block 3909 len 36864 data: 6 4 4 4 4 4 3 3 4
29431 083/300 Flush
29432 084/300 Read block 1113 len 40960
29433 085/300 Read block 4403 len 24576
29434 086/300 Write block 2730 len 24576 data: 5 7 8 7 8 6
29435 087/300 Read block 1549 len 20480
29436 088/300 Read block 3653 len 8192
29437 089/300 Read block 1559 len 28672
29438 090/300 Write block 2157 len 8192 data: 6 6
29439 091/300 Read block 705 len 40960
29440 092/300 Write block 1962 len 40960 data: 4 4 3 2 2 2 2 2 2 2
29441 093/300 Write block 4173 len 32768 data: 8 6 6 7 9 8 7 7
29442 094/300 Write block 3857 len 20480 data: 6 5 4 6 8
29443 095/300 Read block 3166 len 40960
29444 096/300 Write block 3184 len 40960 data: 5 4 3 3 3 3 4 6 6 6
29445 097/300 Read block 2658 len 36864
29446 098/300 Write block 3718 len 20480 data: 7 7 5 5 4
29447 099/300 Read block 984 len 12288
29448 100/300 Read block 3293 len 8192
29449 101/300 Read block 991 len 32768
29450 102/300 Write block 3557 len 16384 data: 7 6 7 6
29451 103/300 Flush
29452 104/300 Read block 2480 len 28672
29453 105/300 Read block 490 len 24576
29454 106/300 Write block 127 len 24576 data: 5 3 4 5 5 6
29455 107/300 Write block 3443 len 8192 data: 8 6
29456 108/300 Write block 1924 len 32768 data: 9 8 7 6 3 4 4 4
29457 109/300 Write block 3682 len 8192 data: 4 5
29458 110/300 Read block 2326 len 12288
29459 111/300 Flush
29460 112/300 Read block 4602 len 16384
29461 113/300 Read block 3945 len 4096
29462 114/300 Write block 2588 len 40960 data: 3 2 2 2 2 3 3 5 5 6
29463 115/300 Write block 4630 len 28672 data: 5 4 4 5 4 4 4
29464 116/300 Write block 1284 len 16384 data: 6 5 5 6
29465 117/300 Read block 4210 len 8192
29466 118/300 Write block 2569 len 40960 data: 8 7 6 7 5 4 5 4 4 3
29467 119/300 Write block 4642 len 4096 data: 6
29468 120/300 Read block 1213 len 4096
29469 121/300 Flush
29470 122/300 Read block 3933 len 32768
29471 123/300 Write block 1652 len 12288 data: 6 7 6
29472 124/300 Write block 3834 len 28672 data: 5 5 6 6 6 5 4
29473 125/300 Flush
29474 126/300 Write block 2222 len 36864 data: 6 5 5 5 5 8 7 8 5
29475 127/300 Read block 2720 len 32768
29476 128/300 Read block 3246 len 12288
29477 129/300 Flush
29478 130/300 Flush
29479 131/300 Read block 1731 len 24576
29480 132/300 Read block 1145 len 24576
29481 133/300 Flush
29482 134/300 Flush
29483 135/300 Write block 1025 len 32768 data: 5 6 5 5 6 6 5 5
29484 136/300 Write block 2424 len 12288 data: 6 6 5
29485 137/300 Read block 4101 len 12288
29486 138/300 Write block 4653 len 40960 data: 4 4 4 2 3 3 3 4 6 7
29487 139/300 Flush
29488 140/300 Flush
29489 141/300 Write block 4888 len 20480 data: 9 8 5 5 4
29490 142/300 Write block 3129 len 20480 data: 5 4 4 5 5
29491 143/300 Write block 2840 len 28672 data: 9 8 7 6 5 5 2
29492 144/300 Read block 3212 len 8192
29493 145/300 Flush
29494 146/300 Flush
29495 147/300 Write block 2954 len 24576 data: 6 3 4 4 5 4
29496 148/300 Read block 1689 len 40960
29497 149/300 Write block 2547 len 20480 data: 4 3 3 2 3
29498 150/300 Write block 77 len 24576 data: 5 6 5 6 6 4
29499 151/300 Write block 1200 len 24576 data: 5 4 4 4 5 6
29500 152/300 Write block 1242 len 40960 data: 5 5 5 6 6 6 8 8 7 7
29501 153/300 Read block 461 len 36864
29502 154/300 Write block 2744 len 40960 data: 6 6 5 5 6 7 5 5 4 3
29503 155/300 Read block 500 len 28672
29504 156/300 Read block 3876 len 4096
29505 157/300 Read block 215 len 24576
29506 158/300 Flush
29507 159/300 Read block 3710 len 36864
29508 160/300 Read block 3286 len 4096
29509 161/300 Read block 4569 len 40960
29510 162/300 Write block 3657 len 20480 data: 6 6 4 6 5
29511 163/300 Read block 3642 len 8192
29512 164/300 Flush
29513 165/300 Write block 3389 len 4096 data: 8
29514 166/300 Write block 3804 len 12288 data: 6 5 4
29515 167/300 Write block 3552 len 12288 data: 4 4 5
29516 168/300 Flush
29517 169/300 Read block 4978 len 12288
29518 170/300 Flush
29519 171/300 Write block 3630 len 40960 data: 6 6 7 9 8 7 8 7 5 5
29520 172/300 Flush
29521 173/300 Read block 2578 len 40960
29522 174/300 Write block 1447 len 4096 data: 7
29523 175/300 Write block 3761 len 40960 data: 7 7 7 7 7 9 11 10 9 8
29524 176/300 Write block 4529 len 4096 data: 4
29525 177/300 Read block 151 len 40960
29526 178/300 Read block 4704 len 20480
29527 179/300 Read block 304 len 28672
29528 180/300 Read block 3447 len 4096
29529 181/300 Read block 4210 len 24576
29530 182/300 Write block 4461 len 28672 data: 2 3 3 3 5 6 7
29531 183/300 Read block 334 len 4096
29532 184/300 Read block 2429 len 12288
29533 185/300 Read block 2924 len 4096
29534 186/300 Read block 2569 len 36864
29535 187/300 Write block 2105 len 40960 data: 4 4 4 3 3 3 8 8 8 10
29536 188/300 Read block 1168 len 32768
29537 189/300 Read block 4696 len 24576
29538 190/300 Write block 3147 len 4096 data: 6
29539 191/300 Read block 2673 len 12288
29540 192/300 Read block 4091 len 28672
29541 193/300 Read block 4922 len 20480
29542 194/300 Write block 1283 len 28672 data: 6 7 6 6 7 5 6
29543 195/300 Write block 3649 len 20480 data: 5 5 6 5 8
29544 196/300 Write block 4619 len 20480 data: 6 7 6 7 7
29545 197/300 Write block 4080 len 16384 data: 6 5 4 3
29546 198/300 Read block 2434 len 8192
29547 199/300 Write block 925 len 32768 data: 7 7 6 6 4 3 4 5
29548 200/300 Flush
29549 201/300 Read block 3663 len 8192
29550 202/300 Read block 3025 len 36864
29551 203/300 Read block 1977 len 36864
29552 204/300 Write block 2917 len 4096 data: 6
29553 205/300 Write block 2810 len 20480 data: 8 8 9 10 10
29554 206/300 Read block 4564 len 8192
29555 207/300 Write block 3047 len 20480 data: 10 9 6 6 6
29556 208/300 Read block 1868 len 20480
29557 209/300 Read block 1426 len 16384
29558 210/300 Read block 3859 len 4096
29559 211/300 Read block 2462 len 40960
29560 212/300 Flush
29561 213/300 Write block 3231 len 28672 data: 8 9 10 11 12 10 10
29562 214/300 Read block 4021 len 20480
29563 215/300 Read block 3527 len 20480
29564 216/300 Read block 1528 len 28672
29565 217/300 Read block 4626 len 16384
29566 218/300 Read block 3601 len 28672
29567 219/300 Write block 1556 len 40960 data: 2 3 4 4 4 4 5 4 2 3
29568 220/300 Read block 2817 len 20480
29569 221/300 Write block 2479 len 20480 data: 4 5 7 8 9
29570 222/300 Read block 4441 len 36864
29571 223/300 Write block 4154 len 12288 data: 3 3 6
29572 224/300 Write block 3930 len 4096 data: 6
29573 225/300 Write block 724 len 16384 data: 5 5 5 5
29574 226/300 Read block 1710 len 28672
29575 227/300 Write block 2577 len 36864 data: 5 4 4 4 5 6 6 6 6
29576 228/300 Flush
29577 229/300 Read block 445 len 28672
29578 230/300 Write block 1871 len 28672 data: 3 3 2 3 3 4 4
29579 231/300 Read block 4632 len 28672
29580 232/300 Read block 264 len 28672
29581 233/300 Read block 1187 len 24576
29582 234/300 Read block 3361 len 32768
29583 235/300 Write block 1449 len 12288 data: 6 6 7
29584 236/300 Write block 1960 len 16384 data: 3 4 5 5
29585 237/300 Read block 1143 len 8192
29586 238/300 Read block 3371 len 16384
29587 239/300 Read block 3126 len 24576
29588 240/300 Read block 4462 len 16384
29589 241/300 Read block 4753 len 32768
29590 242/300 Read block 2717 len 24576
29591 243/300 Write block 3902 len 4096 data: 6
29592 244/300 Write block 2907 len 40960 data: 3 4 7 7 6 5 5 5 6 6
29593 245/300 Read block 2627 len 40960
29594 246/300 Read block 2843 len 32768
29595 247/300 Write block 2959 len 32768 data: 5 4 4 4 5 4 4 3
29596 248/300 Write block 2424 len 8192 data: 7 7
29597 249/300 Write block 2074 len 20480 data: 8 8 6 6 7
29598 250/300 Read block 4802 len 28672
29599 251/300 Read block 306 len 40960
29600 252/300 Read block 3012 len 24576
29601 253/300 Read block 680 len 20480
29602 254/300 Write block 3296 len 12288 data: 4 6 6
29603 255/300 Flush
29604 256/300 Write block 3867 len 28672 data: 12 11 11 9 10 9 7
29605 257/300 Write block 2473 len 4096 data: 6
29606 258/300 Write block 2109 len 32768 data: 4 4 9 9 9 11 8 8
29607 259/300 Write block 4478 len 28672 data: 8 7 5 5 6 5 5
29608 260/300 Read block 4800 len 32768
29609 261/300 Write block 1668 len 8192 data: 5 6
29610 262/300 Read block 3209 len 20480
29611 263/300 Write block 3399 len 16384 data: 8 9 10 10
29612 264/300 Read block 812 len 8192
29613 265/300 Write block 1187 len 32768 data: 5 5 5 6 7 6 6 6
29614 266/300 Read block 248 len 8192
29615 267/300 Read block 2953 len 32768
29616 268/300 Read block 4150 len 4096
29617 269/300 Read block 270 len 20480
29618 270/300 Read block 2806 len 40960
29619 271/300 Read block 1093 len 8192
29620 272/300 Write block 1898 len 4096 data: 7
29621 273/300 Write block 1585 len 12288 data: 8 8 8
29622 274/300 Read block 4490 len 28672
29623 275/300 Write block 1467 len 4096 data: 5
29624 276/300 Read block 1505 len 12288
29625 277/300 Read block 2898 len 36864
29626 278/300 Read block 1542 len 12288
29627 279/300 Read block 1170 len 12288
29628 280/300 Read block 1243 len 12288
29629 281/300 Write block 4600 len 8192 data: 5 5
29630 282/300 Read block 4799 len 36864
29631 283/300 Read block 2664 len 4096
29632 284/300 Write block 3659 len 28672 data: 5 7 6 7 6 7 4
29633 285/300 Read block 3460 len 32768
29634 286/300 Write block 785 len 4096 data: 8
29635 287/300 Write block 3372 len 20480 data: 5 5 5 6 3
29636 288/300 Read block 2819 len 24576
29637 289/300 Read block 1088 len 40960
29638 290/300 Write block 4858 len 12288 data: 8 6 6
29639 291/300 Write block 1013 len 20480 data: 6 6 6 6 5
29640 292/300 Read block 694 len 32768
29641 293/300 Read block 4352 len 40960
29642 294/300 Write block 4616 len 40960 data: 5 5 6 7 8 7 8 8 6 6
29643 295/300 Read block 4730 len 8192
29644 296/300 Read block 4155 len 12288
29645 297/300 Read block 3197 len 36864
29646 298/300 Flush
29647 299/300 Read block 570 len 16384
29648 300/300 Read block 2013 len 8192
29649 301/300 Read block 3046 len 28672
29650 Main task has work to do, go find it
29651 got dsc Start(1)
29652 start 1
29653 Jan 03 08:37:08.121 INFO request completed, latency_us: 201, response_code: 204, uri: /start/cid/1, method: POST, req_id: 12d86dd6-aa4f-484b-b573-98147fe52889, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
29654 Replay: started 1, returned:Ok(())
29655 ----------------------------------------------------------------
29656 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29657 States: Active Offline Active
29658 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29659 8548 Acked Read 9 Done Sent Done false
29660 8549 Acked Flush 0 Done Sent Done false
29661 8550 Acked Read 7 Done Sent Done false
29662 8551 Acked Write 2 Done Sent Done false
29663 8552 Acked Write 9 Done Sent Done false
29664 8553 Acked Read 1 Done Sent Done false
29665 8554 Acked Read 10 Done Sent Done false
29666 8555 Acked Read 5 Done Sent Done false
29667 8556 Acked Write 10 Done Sent Done false
29668 8557 Acked Write 10 Done Sent Done false
29669 8558 Acked Write 6 Done Sent Done false
29670 8559 Acked Write 7 Done Sent Done false
29671 8560 Acked Flush 0 Done Sent Done false
29672 8561 Acked Read 3 Done Sent Done false
29673 8562 Acked Write 2 Done Sent Done false
29674 8563 Acked Write 4 Done Sent Done false
29675 8564 Acked Write 9 Done Sent Done false
29676 8565 Acked Read 5 Done Sent Done false
29677 8566 Acked Read 1 Done Sent Done false
29678 8567 Acked Read 2 Done Sent Done false
29679 8568 Acked Read 1 Done Sent Done false
29680 8569 Acked Write 3 Done Sent Done false
29681 8570 Acked Flush 0 Done Sent Done false
29682 8571 Acked Write 8 Done Sent Done false
29683 8572 Acked Flush 0 Done Sent Done false
29684 8573 Acked Write 3 Done Sent Done false
29685 8574 Acked Read 7 Done Sent Done false
29686 8575 Acked Read 4 Done Sent Done false
29687 8576 Acked Read 3 Done Sent Done false
29688 8577 Acked Read 8 Done Sent Done false
29689 8578 Acked Read 4 Done Sent Done false
29690 8579 Acked Write 2 Done Sent Done false
29691 8580 Acked Write 6 Done Sent Done false
29692 8581 Acked Read 5 Done Sent Done false
29693 8582 Acked Write 5 Done Sent Done false
29694 8583 Acked Write 10 Done Sent Done false
29695 8584 Acked Read 4 Done Sent Done false
29696 8585 Acked Write 1 Done Sent Done false
29697 8586 Acked Read 3 Done Sent Done false
29698 8587 Acked Write 10 Done Sent Done false
29699 8588 Acked Read 1 Done Sent Done false
29700 8589 Acked Read 10 Done Sent Done false
29701 8590 Acked Read 10 Done Sent Done false
29702 8591 Acked Write 8 Done Sent Done false
29703 8592 Acked Write 8 Done Sent Done false
29704 8593 Acked Read 6 Done Sent Done false
29705 8594 Acked Read 5 Done Sent Done false
29706 8595 Acked Read 3 Done Sent Done false
29707 8596 Acked Read 8 Done Sent Done false
29708 8597 Acked Flush 0 Done Sent Done false
29709 8598 Acked Flush 0 Done Sent Done false
29710 8599 Acked Write 4 Done Sent Done false
29711 8600 Acked Write 9 Done Sent Done false
29712 8601 Acked Read 5 Done Sent Done false
29713 8602 Acked Write 7 Done Sent Done false
29714 8603 Acked Flush 0 Done Sent Done false
29715 8604 Acked Read 2 Done Sent Done false
29716 8605 Acked Write 5 Done Sent Done false
29717 8606 Acked Write 3 Done Sent Done false
29718 8607 Acked Write 2 Done Sent Done false
29719 8608 Acked Read 6 Done Sent Done false
29720 8609 Acked Flush 0 Done Sent Done false
29721 8610 Acked Write 1 Done Sent Done false
29722 8611 Acked Write 7 Done Sent Done false
29723 8612 Acked Read 6 Done Sent Done false
29724 8613 Acked Write 8 Done Sent Done false
29725 8614 Acked Write 5 Done Sent Done false
29726 8615 Acked Read 5 Done Sent Done false
29727 8616 Acked Read 1 Done Sent Done false
29728 8617 Acked Read 6 Done Sent Done false
29729 8618 Acked Flush 0 Done Sent Done false
29730 8619 Acked Write 1 Done Sent Done false
29731 8620 Acked Write 4 Done Sent Done false
29732 8621 Acked Read 6 Done Sent Done false
29733 8622 Acked Write 5 Done Sent Done false
29734 8623 Acked Read 8 Done Sent Done false
29735 8624 Acked Read 6 Done Sent Done false
29736 8625 Acked Write 10 Done Sent Done false
29737 8626 Acked Read 1 Done Sent Done false
29738 8627 Acked Write 6 Done Sent Done false
29739 8628 Acked Write 2 Done Sent Done false
29740 8629 Acked Write 9 Done Sent Done false
29741 8630 Acked Flush 0 Done Sent Done false
29742 8631 Acked Read 10 Done Sent Done false
29743 8632 Acked Read 6 Done Sent Done false
29744 8633 Acked Write 6 Done Sent Done false
29745 8634 Acked Read 5 Done Sent Done false
29746 8635 Acked Read 2 Done Sent Done false
29747 8636 Acked Read 7 Done Sent Done false
29748 8637 Acked Write 2 Done Sent Done false
29749 8638 Acked Read 10 Done Sent Done false
29750 8639 Acked Write 10 Done Sent Done false
29751 8640 Acked Write 8 Done Sent Done false
29752 8641 Acked Write 5 Done Sent Done false
29753 8642 Acked Read 10 Done Sent Done false
29754 8643 Acked Write 10 Done Sent Done false
29755 8644 Acked Read 9 Done Sent Done false
29756 8645 Acked Write 5 Done Sent Done false
29757 8646 Acked Read 3 Done Sent Done false
29758 8647 Acked Read 2 Done Sent Done false
29759 8648 Acked Read 8 Done Sent Done false
29760 8649 Acked Write 4 Done Sent Done false
29761 8650 Acked Flush 0 Done Sent Done false
29762 8651 Acked Read 7 Done Sent Done false
29763 8652 Acked Read 6 Done Sent Done false
29764 8653 Acked Write 6 Done Sent Done false
29765 8654 Acked Write 2 Done Sent Done false
29766 8655 Acked Write 8 Done Sent Done false
29767 8656 Acked Write 2 Done Sent Done false
29768 8657 Acked Read 3 Done Sent Done false
29769 8658 Acked Flush 0 Done Sent Done false
29770 8659 Acked Read 4 Done Sent Done false
29771 8660 Acked Read 1 Done Sent Done false
29772 8661 Acked Write 10 Done Sent Done false
29773 8662 Acked Write 7 Done Sent Done false
29774 8663 Acked Write 4 Done Sent Done false
29775 8664 Acked Read 2 Done Sent Done false
29776 8665 Acked Write 10 Done Sent Done false
29777 8666 Acked Write 1 Done Sent Done false
29778 8667 Acked Read 1 Done Sent Done false
29779 8668 Acked Flush 0 Done Sent Done false
29780 8669 Acked Read 8 Done Sent Done false
29781 8670 Acked Write 3 Done Sent Done false
29782 8671 Acked Write 7 Done Sent Done false
29783 8672 Acked Flush 0 Done Sent Done false
29784 8673 Acked Write 9 Done Sent Done false
29785 8674 Acked Read 8 Done Sent Done false
29786 8675 Acked Read 3 Done Sent Done false
29787 8676 Acked Flush 0 Done Sent Done false
29788 8677 Acked Flush 0 Done Sent Done false
29789 8678 Acked Read 6 Done Sent Done false
29790 8679 Acked Read 6 Done Sent Done false
29791 8680 Acked Flush 0 Done Sent Done false
29792 8681 Acked Flush 0 Done Sent Done false
29793 8682 Acked Write 8 Done Sent Done false
29794 8683 Acked Write 3 Done Sent Done false
29795 8684 Acked Read 3 Done Sent Done false
29796 8685 Acked Write 10 Done Sent Done false
29797 8686 Acked Flush 0 Done Sent Done false
29798 8687 Acked Flush 0 Done Sent Done false
29799 8688 Acked Write 5 Done Sent Done false
29800 8689 Acked Write 5 Done Sent Done false
29801 8690 Acked Write 7 Done Sent Done false
29802 8691 Acked Read 2 Done Sent Done false
29803 8692 Acked Flush 0 Done Sent Done false
29804 8693 Acked Flush 0 Done Sent Done false
29805 8694 Acked Write 6 Done Sent Done false
29806 8695 Acked Read 10 Done Sent Done false
29807 8696 Acked Write 5 Done Sent Done false
29808 8697 Acked Write 6 Done Sent Done false
29809 8698 Acked Write 6 Done Sent Done false
29810 8699 Acked Write 10 Done Sent Done false
29811 8700 Acked Read 9 Done Sent Done false
29812 8701 Acked Write 10 Done Sent Done false
29813 8702 Acked Read 7 Done Sent Done false
29814 8703 Acked Read 1 Done Sent Done false
29815 8704 Acked Read 6 Done Sent Done false
29816 8705 Acked Flush 0 Done Sent Done false
29817 8706 Acked Read 9 Done Sent Done false
29818 8707 Acked Read 1 Done Sent Done false
29819 8708 Acked Read 10 Done Sent Done false
29820 8709 Acked Write 5 Done Sent Done false
29821 8710 Acked Read 2 Done Sent Done false
29822 8711 Acked Flush 0 Done Sent Done false
29823 8712 Acked Write 1 Done Sent Done false
29824 8713 Acked Write 3 Done Sent Done false
29825 8714 Acked Write 3 Done Sent Done false
29826 8715 Acked Flush 0 Done Sent Done false
29827 8716 Acked Read 3 Done Sent Done false
29828 8717 Acked Flush 0 Done Sent Done false
29829 8718 Acked Write 10 Done Sent Done false
29830 8719 Acked Flush 0 Done Sent Done false
29831 8720 Acked Read 10 Done Sent Done false
29832 8721 Acked Write 1 Done Sent Done false
29833 8722 Acked Write 10 Done Sent Done false
29834 8723 Acked Write 1 Done Sent Done false
29835 8724 Acked Read 10 Done Sent Done false
29836 8725 Acked Read 5 Done Sent Done false
29837 8726 Acked Read 7 Done Sent Done false
29838 8727 Acked Read 1 Done Sent Done false
29839 8728 Acked Read 6 Done Sent Done false
29840 8729 Acked Write 7 Done Sent Done false
29841 8730 Acked Read 1 Done Sent Done false
29842 8731 Acked Read 3 Done Sent Done false
29843 8732 Acked Read 1 Done Sent Done false
29844 8733 Acked Read 9 Done Sent Done false
29845 8734 Acked Write 10 Done Sent Done false
29846 8735 Acked Read 8 Done Sent Done false
29847 8736 Acked Read 6 Done Sent Done false
29848 8737 Acked Write 1 Done Sent Done false
29849 8738 Acked Read 3 Done Sent Done false
29850 8739 Acked Read 7 Done Sent Done false
29851 8740 Acked Read 5 Done Sent Done false
29852 8741 Acked Write 7 Done Sent Done false
29853 8742 Acked Write 5 Done Sent Done false
29854 8743 Acked Write 5 Done Sent Done false
29855 8744 Acked Write 4 Done Sent Done false
29856 8745 Acked Read 2 Done Sent Done false
29857 8746 Acked Write 8 Done Sent Done false
29858 8747 Acked Flush 0 Done Sent Done false
29859 8748 Acked Read 2 Done Sent Done false
29860 8749 Acked Read 9 Done Sent Done false
29861 8750 Acked Read 9 Done Sent Done false
29862 8751 Acked Write 1 Done Sent Done false
29863 8752 Acked Write 5 Done Sent Done false
29864 8753 Acked Read 2 Done Sent Done false
29865 8754 Acked Write 5 Done Sent Done false
29866 8755 Acked Read 5 Done Sent Done false
29867 8756 Acked Read 4 Done Sent Done false
29868 8757 Acked Read 1 Done Sent Done false
29869 8758 Acked Read 10 Done Sent Done false
29870 8759 Acked Flush 0 Done Sent Done false
29871 8760 Acked Write 7 Done Sent Done false
29872 8761 Acked Read 5 Done Sent Done false
29873 8762 Acked Read 5 Done Sent Done false
29874 8763 Acked Read 7 Done Sent Done false
29875 8764 Acked Read 4 Done Sent Done false
29876 8765 Acked Read 7 Done Sent Done false
29877 8766 Acked Write 10 Done Sent Done false
29878 8767 Acked Read 5 Done Sent Done false
29879 8768 Acked Write 5 Done Sent Done false
29880 8769 Acked Read 9 Done Sent Done false
29881 8770 Acked Write 3 Done Sent Done false
29882 8771 Acked Write 1 Done Sent Done false
29883 8772 Acked Write 4 Done Sent Done false
29884 8773 Acked Read 7 Done Sent Done false
29885 8774 Acked Write 9 Done Sent Done false
29886 8775 Acked Flush 0 Done Sent Done false
29887 8776 Acked Read 7 Done Sent Done false
29888 8777 Acked Write 7 Done Sent Done false
29889 8778 Acked Read 7 Done Sent Done false
29890 8779 Acked Read 7 Done Sent Done false
29891 8780 Acked Read 6 Done Sent Done false
29892 8781 Acked Read 8 Done Sent Done false
29893 8782 Acked Write 3 Done Sent Done false
29894 8783 Acked Write 4 Done Sent Done false
29895 8784 Acked Read 2 Done Sent Done false
29896 8785 Acked Read 4 Done Sent Done false
29897 8786 Acked Read 6 Done Sent Done false
29898 8787 Acked Read 4 Done Sent Done false
29899 8788 Acked Read 8 Done Sent Done false
29900 8789 Acked Read 6 Done Sent Done false
29901 8790 Acked Write 1 Done Sent Done false
29902 8791 Acked Write 10 Done Sent Done false
29903 8792 Acked Read 10 Done Sent Done false
29904 8793 Acked Read 8 Done Sent Done false
29905 8794 Acked Write 8 Done Sent Done false
29906 8795 Acked Write 2 Done Sent Done false
29907 8796 Acked Write 5 Done Sent Done false
29908 8797 Acked Read 7 Done Sent Done false
29909 8798 Acked Read 10 Done Sent Done false
29910 8799 Acked Read 6 Done Sent Done false
29911 8800 Acked Read 5 Done Sent Done false
29912 8801 Acked Write 3 Done Sent Done false
29913 8802 Acked Flush 0 Done Sent Done false
29914 8803 Acked Write 7 Done Sent Done false
29915 8804 Acked Write 1 Done Sent Done false
29916 8805 Acked Write 8 Done Sent Done false
29917 8806 Acked Write 7 Done Sent Done false
29918 8807 Acked Read 8 Done Sent Done false
29919 8808 Acked Write 2 Done Sent Done false
29920 8809 Acked Read 5 Done Sent Done false
29921 8810 Acked Write 4 Done Sent Done false
29922 8811 Acked Read 2 Done Sent Done false
29923 8812 Acked Write 8 Done Sent Done false
29924 8813 Acked Read 2 Done Sent Done false
29925 8814 Acked Read 8 Done Sent Done false
29926 8815 Acked Read 1 Done Sent Done false
29927 8816 Acked Read 5 Done Sent Done false
29928 8817 Acked Read 10 Done Sent Done false
29929 8818 Acked Read 2 Done Sent Done false
29930 8819 Acked Write 1 Done Sent Done false
29931 8820 Acked Write 3 Done Sent Done false
29932 8821 Acked Read 7 Done Sent Done false
29933 8822 Acked Write 1 Done Sent Done false
29934 8823 Acked Read 3 Done Sent Done false
29935 8824 Acked Read 9 Done Sent Done false
29936 8825 Acked Read 3 Done Sent Done false
29937 8826 Acked Read 3 Done Sent Done false
29938 8827 Acked Read 3 Done Sent Done false
29939 8828 Acked Write 2 Done Sent Done false
29940 8829 Acked Read 9 Done Sent Done false
29941 8830 Acked Read 1 Done Sent Done false
29942 8831 Acked Write 7 Done Sent Done false
29943 8832 Acked Read 8 Done Sent Done false
29944 8833 Acked Write 1 Done Sent Done false
29945 8834 Acked Write 5 Done Sent Done false
29946 8835 Acked Read 6 Done Sent Done false
29947 8836 Acked Read 10 Done Sent Done false
29948 8837 Acked Write 3 Done Sent Done false
29949 8838 Acked Write 5 Done Sent Done false
29950 8839 Acked Read 8 Done Sent Done false
29951 8840 Acked Read 10 Done Sent Done false
29952 8841 Acked Write 10 Done Sent Done false
29953 8842 Acked Read 2 Done Sent Done false
29954 8843 Acked Read 3 Done Sent Done false
29955 8844 Acked Read 9 Done Sent Done false
29956 8845 Acked Flush 0 Done Sent Done false
29957 8846 Acked Read 4 Done Sent Done false
29958 8847 Acked Read 2 Done Sent Done false
29959 8848 Acked Read 7 Done Sent Done false
29960 STATES DS:0 DS:1 DS:2 TOTAL
29961 Sent 0 301 0 301
29962 Done 301 0 301 602
29963 Skipped 0 0 0 0
29964 Error 0 0 0 0
29965 Last Flush: 8845 8547 8845
29966 Downstairs last five completed: 8547 8546 8545 8544 8543
29967 Upstairs last five completed: 8848 8847 8846 8845 8844
29968 CLIENT: Up:0 ds:301 act:2
29969 [8820] got start action
29970 [8820] I am going to restart
29971 Starting downstairs at port 8820
29972 Make output file at "/tmp/dsc/downstairs-8820.txt"
29973 [8820][1] reports Starting
29974 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1231)
29975 [8820][1] reports Running
29976 ----------------------------------------------------------------
29977 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29978 States: Active Offline Active
29979 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29980 8548 Acked Read 9 Done Sent Done false
29981 8549 Acked Flush 0 Done Sent Done false
29982 8550 Acked Read 7 Done Sent Done false
29983 8551 Acked Write 2 Done Sent Done false
29984 8552 Acked Write 9 Done Sent Done false
29985 8553 Acked Read 1 Done Sent Done false
29986 8554 Acked Read 10 Done Sent Done false
29987 8555 Acked Read 5 Done Sent Done false
29988 8556 Acked Write 10 Done Sent Done false
29989 8557 Acked Write 10 Done Sent Done false
29990 8558 Acked Write 6 Done Sent Done false
29991 8559 Acked Write 7 Done Sent Done false
29992 8560 Acked Flush 0 Done Sent Done false
29993 8561 Acked Read 3 Done Sent Done false
29994 8562 Acked Write 2 Done Sent Done false
29995 8563 Acked Write 4 Done Sent Done false
29996 8564 Acked Write 9 Done Sent Done false
29997 8565 Acked Read 5 Done Sent Done false
29998 8566 Acked Read 1 Done Sent Done false
29999 8567 Acked Read 2 Done Sent Done false
30000 8568 Acked Read 1 Done Sent Done false
30001 8569 Acked Write 3 Done Sent Done false
30002 8570 Acked Flush 0 Done Sent Done false
30003 8571 Acked Write 8 Done Sent Done false
30004 8572 Acked Flush 0 Done Sent Done false
30005 8573 Acked Write 3 Done Sent Done false
30006 8574 Acked Read 7 Done Sent Done false
30007 8575 Acked Read 4 Done Sent Done false
30008 8576 Acked Read 3 Done Sent Done false
30009 8577 Acked Read 8 Done Sent Done false
30010 8578 Acked Read 4 Done Sent Done false
30011 8579 Acked Write 2 Done Sent Done false
30012 8580 Acked Write 6 Done Sent Done false
30013 8581 Acked Read 5 Done Sent Done false
30014 8582 Acked Write 5 Done Sent Done false
30015 8583 Acked Write 10 Done Sent Done false
30016 8584 Acked Read 4 Done Sent Done false
30017 8585 Acked Write 1 Done Sent Done false
30018 8586 Acked Read 3 Done Sent Done false
30019 8587 Acked Write 10 Done Sent Done false
30020 8588 Acked Read 1 Done Sent Done false
30021 8589 Acked Read 10 Done Sent Done false
30022 8590 Acked Read 10 Done Sent Done false
30023 8591 Acked Write 8 Done Sent Done false
30024 8592 Acked Write 8 Done Sent Done false
30025 8593 Acked Read 6 Done Sent Done false
30026 8594 Acked Read 5 Done Sent Done false
30027 8595 Acked Read 3 Done Sent Done false
30028 8596 Acked Read 8 Done Sent Done false
30029 8597 Acked Flush 0 Done Sent Done false
30030 8598 Acked Flush 0 Done Sent Done false
30031 8599 Acked Write 4 Done Sent Done false
30032 8600 Acked Write 9 Done Sent Done false
30033 8601 Acked Read 5 Done Sent Done false
30034 8602 Acked Write 7 Done Sent Done false
30035 8603 Acked Flush 0 Done Sent Done false
30036 8604 Acked Read 2 Done Sent Done false
30037 8605 Acked Write 5 Done Sent Done false
30038 8606 Acked Write 3 Done Sent Done false
30039 8607 Acked Write 2 Done Sent Done false
30040 8608 Acked Read 6 Done Sent Done false
30041 8609 Acked Flush 0 Done Sent Done false
30042 8610 Acked Write 1 Done Sent Done false
30043 8611 Acked Write 7 Done Sent Done false
30044 8612 Acked Read 6 Done Sent Done false
30045 8613 Acked Write 8 Done Sent Done false
30046 8614 Acked Write 5 Done Sent Done false
30047 8615 Acked Read 5 Done Sent Done false
30048 8616 Acked Read 1 Done Sent Done false
30049 8617 Acked Read 6 Done Sent Done false
30050 8618 Acked Flush 0 Done Sent Done false
30051 8619 Acked Write 1 Done Sent Done false
30052 8620 Acked Write 4 Done Sent Done false
30053 8621 Acked Read 6 Done Sent Done false
30054 8622 Acked Write 5 Done Sent Done false
30055 8623 Acked Read 8 Done Sent Done false
30056 8624 Acked Read 6 Done Sent Done false
30057 8625 Acked Write 10 Done Sent Done false
30058 8626 Acked Read 1 Done Sent Done false
30059 8627 Acked Write 6 Done Sent Done false
30060 8628 Acked Write 2 Done Sent Done false
30061 8629 Acked Write 9 Done Sent Done false
30062 8630 Acked Flush 0 Done Sent Done false
30063 8631 Acked Read 10 Done Sent Done false
30064 8632 Acked Read 6 Done Sent Done false
30065 8633 Acked Write 6 Done Sent Done false
30066 8634 Acked Read 5 Done Sent Done false
30067 8635 Acked Read 2 Done Sent Done false
30068 8636 Acked Read 7 Done Sent Done false
30069 8637 Acked Write 2 Done Sent Done false
30070 8638 Acked Read 10 Done Sent Done false
30071 8639 Acked Write 10 Done Sent Done false
30072 8640 Acked Write 8 Done Sent Done false
30073 8641 Acked Write 5 Done Sent Done false
30074 8642 Acked Read 10 Done Sent Done false
30075 8643 Acked Write 10 Done Sent Done false
30076 8644 Acked Read 9 Done Sent Done false
30077 8645 Acked Write 5 Done Sent Done false
30078 8646 Acked Read 3 Done Sent Done false
30079 8647 Acked Read 2 Done Sent Done false
30080 8648 Acked Read 8 Done Sent Done false
30081 8649 Acked Write 4 Done Sent Done false
30082 8650 Acked Flush 0 Done Sent Done false
30083 8651 Acked Read 7 Done Sent Done false
30084 8652 Acked Read 6 Done Sent Done false
30085 8653 Acked Write 6 Done Sent Done false
30086 8654 Acked Write 2 Done Sent Done false
30087 8655 Acked Write 8 Done Sent Done false
30088 8656 Acked Write 2 Done Sent Done false
30089 8657 Acked Read 3 Done Sent Done false
30090 8658 Acked Flush 0 Done Sent Done false
30091 8659 Acked Read 4 Done Sent Done false
30092 8660 Acked Read 1 Done Sent Done false
30093 8661 Acked Write 10 Done Sent Done false
30094 8662 Acked Write 7 Done Sent Done false
30095 8663 Acked Write 4 Done Sent Done false
30096 8664 Acked Read 2 Done Sent Done false
30097 8665 Acked Write 10 Done Sent Done false
30098 8666 Acked Write 1 Done Sent Done false
30099 8667 Acked Read 1 Done Sent Done false
30100 8668 Acked Flush 0 Done Sent Done false
30101 8669 Acked Read 8 Done Sent Done false
30102 8670 Acked Write 3 Done Sent Done false
30103 8671 Acked Write 7 Done Sent Done false
30104 8672 Acked Flush 0 Done Sent Done false
30105 8673 Acked Write 9 Done Sent Done false
30106 8674 Acked Read 8 Done Sent Done false
30107 8675 Acked Read 3 Done Sent Done false
30108 8676 Acked Flush 0 Done Sent Done false
30109 8677 Acked Flush 0 Done Sent Done false
30110 8678 Acked Read 6 Done Sent Done false
30111 8679 Acked Read 6 Done Sent Done false
30112 8680 Acked Flush 0 Done Sent Done false
30113 8681 Acked Flush 0 Done Sent Done false
30114 8682 Acked Write 8 Done Sent Done false
30115 8683 Acked Write 3 Done Sent Done false
30116 8684 Acked Read 3 Done Sent Done false
30117 8685 Acked Write 10 Done Sent Done false
30118 8686 Acked Flush 0 Done Sent Done false
30119 8687 Acked Flush 0 Done Sent Done false
30120 8688 Acked Write 5 Done Sent Done false
30121 8689 Acked Write 5 Done Sent Done false
30122 8690 Acked Write 7 Done Sent Done false
30123 8691 Acked Read 2 Done Sent Done false
30124 8692 Acked Flush 0 Done Sent Done false
30125 8693 Acked Flush 0 Done Sent Done false
30126 8694 Acked Write 6 Done Sent Done false
30127 8695 Acked Read 10 Done Sent Done false
30128 8696 Acked Write 5 Done Sent Done false
30129 8697 Acked Write 6 Done Sent Done false
30130 8698 Acked Write 6 Done Sent Done false
30131 8699 Acked Write 10 Done Sent Done false
30132 8700 Acked Read 9 Done Sent Done false
30133 8701 Acked Write 10 Done Sent Done false
30134 8702 Acked Read 7 Done Sent Done false
30135 8703 Acked Read 1 Done Sent Done false
30136 8704 Acked Read 6 Done Sent Done false
30137 8705 Acked Flush 0 Done Sent Done false
30138 8706 Acked Read 9 Done Sent Done false
30139 8707 Acked Read 1 Done Sent Done false
30140 8708 Acked Read 10 Done Sent Done false
30141 8709 Acked Write 5 Done Sent Done false
30142 8710 Acked Read 2 Done Sent Done false
30143 8711 Acked Flush 0 Done Sent Done false
30144 8712 Acked Write 1 Done Sent Done false
30145 8713 Acked Write 3 Done Sent Done false
30146 8714 Acked Write 3 Done Sent Done false
30147 8715 Acked Flush 0 Done Sent Done false
30148 8716 Acked Read 3 Done Sent Done false
30149 8717 Acked Flush 0 Done Sent Done false
30150 8718 Acked Write 10 Done Sent Done false
30151 8719 Acked Flush 0 Done Sent Done false
30152 8720 Acked Read 10 Done Sent Done false
30153 8721 Acked Write 1 Done Sent Done false
30154 8722 Acked Write 10 Done Sent Done false
30155 8723 Acked Write 1 Done Sent Done false
30156 8724 Acked Read 10 Done Sent Done false
30157 8725 Acked Read 5 Done Sent Done false
30158 8726 Acked Read 7 Done Sent Done false
30159 8727 Acked Read 1 Done Sent Done false
30160 8728 Acked Read 6 Done Sent Done false
30161 8729 Acked Write 7 Done Sent Done false
30162 8730 Acked Read 1 Done Sent Done false
30163 8731 Acked Read 3 Done Sent Done false
30164 8732 Acked Read 1 Done Sent Done false
30165 8733 Acked Read 9 Done Sent Done false
30166 8734 Acked Write 10 Done Sent Done false
30167 8735 Acked Read 8 Done Sent Done false
30168 8736 Acked Read 6 Done Sent Done false
30169 8737 Acked Write 1 Done Sent Done false
30170 8738 Acked Read 3 Done Sent Done false
30171 8739 Acked Read 7 Done Sent Done false
30172 8740 Acked Read 5 Done Sent Done false
30173 8741 Acked Write 7 Done Sent Done false
30174 8742 Acked Write 5 Done Sent Done false
30175 8743 Acked Write 5 Done Sent Done false
30176 8744 Acked Write 4 Done Sent Done false
30177 8745 Acked Read 2 Done Sent Done false
30178 8746 Acked Write 8 Done Sent Done false
30179 8747 Acked Flush 0 Done Sent Done false
30180 8748 Acked Read 2 Done Sent Done false
30181 8749 Acked Read 9 Done Sent Done false
30182 8750 Acked Read 9 Done Sent Done false
30183 8751 Acked Write 1 Done Sent Done false
30184 8752 Acked Write 5 Done Sent Done false
30185 8753 Acked Read 2 Done Sent Done false
30186 8754 Acked Write 5 Done Sent Done false
30187 8755 Acked Read 5 Done Sent Done false
30188 8756 Acked Read 4 Done Sent Done false
30189 8757 Acked Read 1 Done Sent Done false
30190 8758 Acked Read 10 Done Sent Done false
30191 8759 Acked Flush 0 Done Sent Done false
30192 8760 Acked Write 7 Done Sent Done false
30193 8761 Acked Read 5 Done Sent Done false
30194 8762 Acked Read 5 Done Sent Done false
30195 8763 Acked Read 7 Done Sent Done false
30196 8764 Acked Read 4 Done Sent Done false
30197 8765 Acked Read 7 Done Sent Done false
30198 8766 Acked Write 10 Done Sent Done false
30199 8767 Acked Read 5 Done Sent Done false
30200 8768 Acked Write 5 Done Sent Done false
30201 8769 Acked Read 9 Done Sent Done false
30202 8770 Acked Write 3 Done Sent Done false
30203 8771 Acked Write 1 Done Sent Done false
30204 8772 Acked Write 4 Done Sent Done false
30205 8773 Acked Read 7 Done Sent Done false
30206 8774 Acked Write 9 Done Sent Done false
30207 8775 Acked Flush 0 Done Sent Done false
30208 8776 Acked Read 7 Done Sent Done false
30209 8777 Acked Write 7 Done Sent Done false
30210 8778 Acked Read 7 Done Sent Done false
30211 8779 Acked Read 7 Done Sent Done false
30212 8780 Acked Read 6 Done Sent Done false
30213 8781 Acked Read 8 Done Sent Done false
30214 8782 Acked Write 3 Done Sent Done false
30215 8783 Acked Write 4 Done Sent Done false
30216 8784 Acked Read 2 Done Sent Done false
30217 8785 Acked Read 4 Done Sent Done false
30218 8786 Acked Read 6 Done Sent Done false
30219 8787 Acked Read 4 Done Sent Done false
30220 8788 Acked Read 8 Done Sent Done false
30221 8789 Acked Read 6 Done Sent Done false
30222 8790 Acked Write 1 Done Sent Done false
30223 8791 Acked Write 10 Done Sent Done false
30224 8792 Acked Read 10 Done Sent Done false
30225 8793 Acked Read 8 Done Sent Done false
30226 8794 Acked Write 8 Done Sent Done false
30227 8795 Acked Write 2 Done Sent Done false
30228 8796 Acked Write 5 Done Sent Done false
30229 8797 Acked Read 7 Done Sent Done false
30230 8798 Acked Read 10 Done Sent Done false
30231 8799 Acked Read 6 Done Sent Done false
30232 8800 Acked Read 5 Done Sent Done false
30233 8801 Acked Write 3 Done Sent Done false
30234 8802 Acked Flush 0 Done Sent Done false
30235 8803 Acked Write 7 Done Sent Done false
30236 8804 Acked Write 1 Done Sent Done false
30237 8805 Acked Write 8 Done Sent Done false
30238 8806 Acked Write 7 Done Sent Done false
30239 8807 Acked Read 8 Done Sent Done false
30240 8808 Acked Write 2 Done Sent Done false
30241 8809 Acked Read 5 Done Sent Done false
30242 8810 Acked Write 4 Done Sent Done false
30243 8811 Acked Read 2 Done Sent Done false
30244 8812 Acked Write 8 Done Sent Done false
30245 8813 Acked Read 2 Done Sent Done false
30246 8814 Acked Read 8 Done Sent Done false
30247 8815 Acked Read 1 Done Sent Done false
30248 8816 Acked Read 5 Done Sent Done false
30249 8817 Acked Read 10 Done Sent Done false
30250 8818 Acked Read 2 Done Sent Done false
30251 8819 Acked Write 1 Done Sent Done false
30252 8820 Acked Write 3 Done Sent Done false
30253 8821 Acked Read 7 Done Sent Done false
30254 8822 Acked Write 1 Done Sent Done false
30255 8823 Acked Read 3 Done Sent Done false
30256 8824 Acked Read 9 Done Sent Done false
30257 8825 Acked Read 3 Done Sent Done false
30258 8826 Acked Read 3 Done Sent Done false
30259 8827 Acked Read 3 Done Sent Done false
30260 8828 Acked Write 2 Done Sent Done false
30261 8829 Acked Read 9 Done Sent Done false
30262 8830 Acked Read 1 Done Sent Done false
30263 8831 Acked Write 7 Done Sent Done false
30264 8832 Acked Read 8 Done Sent Done false
30265 8833 Acked Write 1 Done Sent Done false
30266 8834 Acked Write 5 Done Sent Done false
30267 8835 Acked Read 6 Done Sent Done false
30268 8836 Acked Read 10 Done Sent Done false
30269 8837 Acked Write 3 Done Sent Done false
30270 8838 Acked Write 5 Done Sent Done false
30271 8839 Acked Read 8 Done Sent Done false
30272 8840 Acked Read 10 Done Sent Done false
30273 8841 Acked Write 10 Done Sent Done false
30274 8842 Acked Read 2 Done Sent Done false
30275 8843 Acked Read 3 Done Sent Done false
30276 8844 Acked Read 9 Done Sent Done false
30277 8845 Acked Flush 0 Done Sent Done false
30278 8846 Acked Read 4 Done Sent Done false
30279 8847 Acked Read 2 Done Sent Done false
30280 8848 Acked Read 7 Done Sent Done false
30281 8849 Acked Flush 0 Done Sent Done false
30282 STATES DS:0 DS:1 DS:2 TOTAL
30283 Sent 0 302 0 302
30284 Done 302 0 302 604
30285 Skipped 0 0 0 0
30286 Error 0 0 0 0
30287 Last Flush: 8849 8547 8849
30288 Downstairs last five completed: 8547 8546 8545 8544 8543
30289 Upstairs last five completed: 8849 8848 8847 8846 8845
30290 CLIENT: Up:0 ds:302 act:2
302912025-01-03T08:37:13.880ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
302922025-01-03T08:37:13.880ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:34785) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
302932025-01-03T08:37:13.881ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
302942025-01-03T08:37:13.882ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
302952025-01-03T08:37:13.882ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
302962025-01-03T08:37:13.882ZINFOcrucible: send last flush ID to this DS: 8547 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
302972025-01-03T08:37:13.882ZINFOcrucible: Replied this last flush ID: 8547 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
302982025-01-03T08:37:13.882ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
302992025-01-03T08:37:13.882ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
303002025-01-03T08:37:13.882ZINFOcrucible: [1] client re-new 302 jobs since flush 8547 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
303012025-01-03T08:37:13.882ZINFOcrucible: [1] Replayed 302 jobs since flush: 8547 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
30302 ----------------------------------------------------------------
30303 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30304 Downstairs last five completed: 8849 8848 8847 8846 8845
30305 Upstairs last five completed: 8849 8848 8847 8846 8845
30306 CLIENT: Up:0 ds:0 act:3
30307 Replay: All jobs finished
30308 Main task has work to do, go find it
30309 got dsc Stop(2)
30310 Jan 03 08:37:16.125 INFO request completed, latency_us: 360stop 2
30311 , response_code: 204, uri: /stop/cid/2, method: POST, req_id: d861e725-81a4-4d84-8921-55035b13d931, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
30312 [8830] Got stop action so:false kr:false
30313 Jan 03 08:37:16.126 INFO request completed, latency_us: 179, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2ffd9521-a00a-4879-9be1-fa805ae03e13, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
303142025-01-03T08:37:16.127ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
303152025-01-03T08:37:16.127ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
303162025-01-03T08:37:16.127ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
303172025-01-03T08:37:16.127ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
303182025-01-03T08:37:16.127ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
303192025-01-03T08:37:16.127ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
30320 [8830] Exited with: ExitStatus(unix_wait_status(9))
30321 [8830][2] reports Exit
30322 Jan 03 08:37:20.129 INFO request completed, latency_us: 204, response_code: 200, uri: /state/cid/2, method: GET, req_id: bf4ff9ef-f9a1-4b96-a398-78a7d6d9fde9, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
30323 001/300 Write block 782 len 8192 data: 6 5
30324 002/300 Read block 955 len 36864
30325 003/300 Read block 3644 len 8192
30326 004/300 Read block 1534 len 12288
30327 005/300 Read block 3975 len 28672
30328 006/300 Write block 291 len 12288 data: 4 4 4
30329 007/300 Flush
30330 008/300 Write block 2117 len 20480 data: 5 4 4 5 5
30331 009/300 Read block 1893 len 32768
30332 010/300 Write block 1607 len 12288 data: 6 3 3
30333 011/300 Write block 1 len 20480 data: 3 2 2 2 3
30334 012/300 Read block 695 len 40960
30335 013/300 Write block 2571 len 4096 data: 7
30336 014/300 Read block 3451 len 36864
30337 015/300 Read block 4941 len 28672
30338 016/300 Read block 1522 len 20480
30339 017/300 Read block 4601 len 16384
30340 018/300 Read block 1719 len 36864
30341 019/300 Read block 421 len 12288
30342 020/300 Flush
30343 021/300 Read block 2806 len 4096
30344 022/300 Read block 3062 len 24576
30345 023/300 Read block 549 len 20480
30346 024/300 Read block 1917 len 32768
30347 025/300 Write block 117 len 36864 data: 4 3 4 4 3 3 4 4 4
30348 026/300 Write block 1328 len 20480 data: 6 5 4 4 5
30349 027/300 Write block 1259 len 8192 data: 3 3
30350 028/300 Write block 4514 len 36864 data: 5 5 3 2 3 3 5 5 5
30351 029/300 Flush
30352 030/300 Read block 3859 len 40960
30353 031/300 Read block 184 len 20480
30354 032/300 Read block 2881 len 32768
30355 033/300 Read block 256 len 40960
30356 034/300 Write block 1939 len 36864 data: 5 6 6 5 5 5 7 8 9
30357 035/300 Read block 2218 len 24576
30358 036/300 Write block 1985 len 8192 data: 4 4
30359 037/300 Write block 3217 len 16384 data: 4 5 6 7
30360 038/300 Read block 3530 len 28672
30361 039/300 Write block 1512 len 32768 data: 6 5 4 4 4 5 6 7
30362 040/300 Read block 4967 len 32768
30363 041/300 Read block 1698 len 40960
30364 042/300 Read block 3411 len 16384
30365 043/300 Write block 1768 len 12288 data: 7 7 7
30366 044/300 Read block 3941 len 28672
30367 045/300 Write block 1766 len 32768 data: 5 6 8 8 8 4 3 5
30368 046/300 Read block 2198 len 28672
30369 047/300 Read block 3878 len 32768
30370 048/300 Read block 442 len 4096
30371 049/300 Read block 935 len 12288
30372 050/300 Flush
30373 051/300 Read block 2853 len 4096
30374 052/300 Read block 3910 len 28672
30375 053/300 Write block 2605 len 8192 data: 3 3
30376 054/300 Write block 2030 len 8192 data: 7 6
30377 055/300 Read block 4577 len 32768
30378 056/300 Read block 4636 len 36864
30379 057/300 Read block 872 len 32768
30380 058/300 Write block 186 len 4096 data: 6
30381 059/300 Read block 1182 len 20480
30382 060/300 Read block 4990 len 4096
30383 061/300 Write block 2495 len 24576 data: 6 7 5 6 5 4
30384 062/300 Flush
30385 063/300 Read block 2264 len 8192
30386 064/300 Read block 3114 len 16384
30387 065/300 Write block 4575 len 16384 data: 7 6 6 7
30388 066/300 Read block 4844 len 4096
30389 067/300 Write block 2996 len 4096 data: 4
30390 068/300 Read block 4029 len 40960
30391 069/300 Read block 352 len 32768
30392 070/300 Read block 88 len 16384
30393 071/300 Flush
30394 072/300 Read block 3847 len 36864
30395 073/300 Write block 4027 len 12288 data: 5 6 4
30396 074/300 Read block 3071 len 20480
30397 075/300 Flush
30398 076/300 Flush
30399 077/300 Read block 3581 len 40960
30400 078/300 Read block 701 len 40960
30401 079/300 Write block 4979 len 32768 data: 4 5 6 5 5 5 3 3
30402 080/300 Write block 1846 len 12288 data: 3 4 6
30403 081/300 Write block 1479 len 24576 data: 6 5 5 5 6 6
30404 082/300 Flush
30405 083/300 Write block 3929 len 40960 data: 7 7 7 7 7 5 5 5 7 7
30406 084/300 Read block 2283 len 12288
30407 085/300 Read block 3147 len 36864
30408 086/300 Write block 4721 len 16384 data: 5 4 5 6
30409 087/300 Write block 3935 len 20480 data: 6 6 8 8 7
30410 088/300 Read block 2592 len 24576
30411 089/300 Write block 4631 len 8192 data: 5 5
30412 090/300 Write block 4753 len 36864 data: 8 9 8 9 9 9 8 8 6
30413 091/300 Write block 3500 len 24576 data: 3 3 3 3 2 2
30414 092/300 Read block 4894 len 40960
30415 093/300 Read block 399 len 20480
30416 094/300 Flush
30417 095/300 Read block 949 len 24576
30418 096/300 Flush
30419 097/300 Write block 2931 len 36864 data: 4 4 3 3 4 3 4 5 5
30420 098/300 Write block 1642 len 32768 data: 4 5 5 4 4 3 4 6
30421 099/300 Write block 2871 len 8192 data: 5 5
30422 100/300 Flush
30423 101/300 Write block 2238 len 24576 data: 2 4 3 2 2 2
30424 102/300 Read block 4514 len 20480
30425 103/300 Write block 1264 len 32768 data: 5 7 6 7 8 8 6 7
30426 104/300 Write block 3261 len 12288 data: 6 6 5
30427 105/300 Read block 4051 len 4096
30428 106/300 Read block 2572 len 24576
30429 107/300 Read block 4869 len 24576
30430 108/300 Read block 4333 len 4096
30431 109/300 Read block 3394 len 32768
30432 110/300 Read block 3621 len 16384
30433 111/300 Read block 1497 len 28672
30434 112/300 Write block 2750 len 24576 data: 6 6 5 4 3 4
30435 113/300 Write block 1569 len 16384 data: 5 6 6 7
30436 114/300 Read block 538 len 36864
30437 115/300 Flush
30438 116/300 Write block 2367 len 12288 data: 8 7 6
30439 117/300 Read block 1980 len 20480
30440 118/300 Write block 281 len 28672 data: 4 5 5 6 6 5 5
30441 119/300 Write block 1089 len 24576 data: 6 9 10 9 8 8
30442 120/300 Write block 4724 len 24576 data: 7 5 5 5 5 6
30443 121/300 Write block 2985 len 20480 data: 3 4 2 3 3
30444 122/300 Flush
30445 123/300 Write block 2642 len 40960 data: 8 7 7 6 5 3 3 3 3 4
30446 124/300 Read block 4418 len 32768
30447 125/300 Flush
30448 126/300 Read block 4285 len 24576
30449 127/300 Read block 1162 len 12288
30450 128/300 Read block 792 len 32768
30451 129/300 Flush
30452 130/300 Read block 2953 len 28672
30453 131/300 Write block 4017 len 12288 data: 6 6 6
30454 132/300 Flush
30455 133/300 Read block 947 len 8192
30456 134/300 Write block 984 len 32768 data: 4 3 4 4 4 4 5 5
30457 135/300 Read block 2259 len 12288
30458 136/300 Read block 1562 len 8192
30459 137/300 Read block 254 len 12288
30460 138/300 Read block 1249 len 36864
30461 139/300 Write block 4772 len 8192 data: 5 4
30462 140/300 Read block 3156 len 32768
30463 141/300 Read block 2071 len 24576
30464 142/300 Write block 1098 len 20480 data: 9 8 7 8 6
30465 143/300 Write block 3065 len 16384 data: 5 4 3 4
30466 144/300 Write block 1059 len 4096 data: 4
30467 145/300 Read block 1895 len 8192
30468 146/300 Write block 4959 len 28672 data: 6 6 7 6 4 5 7
30469 147/300 Write block 3183 len 24576 data: 7 6 5 4 4 4
30470 148/300 Read block 1416 len 40960
30471 149/300 Read block 4745 len 8192
30472 150/300 Write block 4105 len 16384 data: 6 6 5 5
30473 151/300 Write block 4820 len 32768 data: 5 5 6 4 5 5 6 6
30474 152/300 Read block 2125 len 20480
30475 153/300 Read block 2897 len 16384
30476 154/300 Read block 4320 len 28672
30477 155/300 Write block 3255 len 16384 data: 7 7 5 7
30478 156/300 Write block 3392 len 28672 data: 6 7 7 8 7 8 8
30479 157/300 Read block 518 len 4096
30480 158/300 Write block 4374 len 28672 data: 6 6 4 4 5 4 4
30481 159/300 Read block 4332 len 24576
30482 160/300 Read block 2010 len 20480
30483 161/300 Read block 3961 len 40960
30484 162/300 Write block 3053 len 28672 data: 7 8 5 5 4 5 4
30485 163/300 Read block 900 len 28672
30486 164/300 Read block 1263 len 36864
30487 165/300 Write block 531 len 40960 data: 6 4 3 4 5 5 5 6 7 6
30488 166/300 Write block 1343 len 20480 data: 4 4 5 6 6
30489 167/300 Read block 193 len 8192
30490 168/300 Write block 2982 len 16384 data: 2 3 4 4
30491 169/300 Flush
30492 170/300 Read block 3829 len 24576
30493 171/300 Write block 29 len 40960 data: 2 3 3 3 3 3 3 3 4 4
30494 172/300 Write block 2171 len 12288 data: 5 5 5
30495 173/300 Read block 1661 len 36864
30496 174/300 Write block 476 len 24576 data: 7 7 6 5 6 6
30497 175/300 Write block 4130 len 36864 data: 5 5 5 5 5 6 6 6 7
30498 176/300 Read block 1782 len 32768
30499 177/300 Read block 2382 len 28672
30500 178/300 Read block 798 len 20480
30501 179/300 Read block 4672 len 40960
30502 180/300 Read block 1769 len 24576
30503 181/300 Write block 1700 len 24576 data: 4 3 3 4 4 4
30504 182/300 Write block 3152 len 40960 data: 3 4 5 5 5 4 4 5 6 6
30505 183/300 Write block 2168 len 20480 data: 7 5 5 6 6
30506 184/300 Write block 2516 len 28672 data: 6 6 5 5 8 8 8
30507 185/300 Read block 2511 len 4096
30508 186/300 Flush
30509 187/300 Read block 77 len 28672
30510 188/300 Flush
30511 189/300 Read block 3469 len 16384
30512 190/300 Read block 3215 len 32768
30513 191/300 Flush
30514 192/300 Read block 1353 len 32768
30515 193/300 Write block 2359 len 28672 data: 3 4 5 5 4 4 6
30516 194/300 Read block 3494 len 8192
30517 195/300 Write block 4232 len 8192 data: 6 4
30518 196/300 Write block 3312 len 12288 data: 6 5 6
30519 197/300 Write block 4003 len 28672 data: 8 9 9 8 7 7 4
30520 198/300 Read block 1852 len 12288
30521 199/300 Read block 4479 len 4096
30522 200/300 Flush
30523 201/300 Write block 3964 len 8192 data: 3 3
30524 202/300 Write block 3141 len 36864 data: 6 5 5 4 6 6 7 6 4
30525 203/300 Read block 3968 len 8192
30526 204/300 Read block 3830 len 12288
30527 205/300 Write block 552 len 8192 data: 4 5
30528 206/300 Flush
30529 207/300 Write block 1740 len 24576 data: 4 5 3 2 3 3
30530 208/300 Flush
30531 209/300 Read block 28 len 32768
30532 210/300 Write block 1386 len 24576 data: 4 6 8 7 6 7
30533 211/300 Read block 1356 len 4096
30534 212/300 Write block 3913 len 16384 data: 5 5 4 4
30535 213/300 Read block 128 len 40960
30536 214/300 Read block 2622 len 8192
30537 215/300 Read block 1527 len 40960
30538 216/300 Read block 4023 len 32768
30539 217/300 Read block 1087 len 40960
30540 218/300 Read block 3847 len 24576
30541 219/300 Write block 3430 len 36864 data: 6 7 7 7 7 8 7 7 6
30542 220/300 Flush
30543 221/300 Flush
30544 222/300 Read block 1153 len 28672
30545 223/300 Read block 2635 len 32768
30546 224/300 Read block 4145 len 16384
30547 225/300 Read block 2113 len 12288
30548 226/300 Read block 3595 len 36864
30549 227/300 Read block 4301 len 16384
30550 228/300 Read block 2406 len 16384
30551 229/300 Write block 2977 len 24576 data: 8 6 3 2 2 3
30552 230/300 Write block 686 len 40960 data: 4 4 3 4 5 7 8 8 8 7
30553 231/300 Read block 4197 len 16384
30554 232/300 Read block 3231 len 12288
30555 233/300 Read block 713 len 36864
30556 234/300 Write block 3437 len 32768 data: 8 7 7 8 7 9 9 7
30557 235/300 Read block 3029 len 36864
30558 236/300 Read block 38 len 12288
30559 237/300 Read block 3823 len 32768
30560 238/300 Write block 4498 len 40960 data: 8 7 7 6 6 7 7 6 7 6
30561 239/300 Read block 1875 len 32768
30562 240/300 Write block 1938 len 28672 data: 4 6 7 7 6 6 6
30563 241/300 Read block 20 len 20480
30564 242/300 Read block 4682 len 12288
30565 243/300 Read block 1849 len 24576
30566 244/300 Flush
30567 245/300 Read block 3946 len 4096
30568 246/300 Read block 601 len 8192
30569 247/300 Read block 2086 len 28672
30570 248/300 Read block 3738 len 8192
30571 249/300 Read block 4265 len 28672
30572 250/300 Read block 3855 len 4096
30573 251/300 Flush
30574 252/300 Write block 2242 len 40960 data: 3 3 3 3 4 5 4 5 6 6
30575 253/300 Read block 172 len 20480
30576 254/300 Write block 3672 len 24576 data: 9 9 9 10 8 6
30577 255/300 Write block 2259 len 40960 data: 6 4 3 4 6 5 4 4 5 5
30578 256/300 Read block 1369 len 32768
30579 257/300 Write block 2414 len 36864 data: 5 5 5 4 6 6 7 7 7
30580 258/300 Read block 4045 len 36864
30581 259/300 Write block 2966 len 36864 data: 4 3 3 5 6 7 8 8 9
30582 260/300 Read block 4235 len 36864
30583 261/300 Write block 690 len 40960 data: 6 8 9 9 9 8 7 7 5 4
30584 262/300 Write block 3650 len 24576 data: 6 7 6 9 7 7
30585 263/300 Read block 962 len 12288
30586 264/300 Read block 1961 len 20480
30587 265/300 Read block 1435 len 16384
30588 266/300 Read block 2568 len 16384
30589 267/300 Write block 3302 len 12288 data: 6 5 4
30590 268/300 Write block 4767 len 28672 data: 6 7 9 8 6 6 5
30591 269/300 Write block 795 len 40960 data: 7 6 7 7 8 6 6 6 6 7
30592 270/300 Write block 4795 len 12288 data: 5 5 4
30593 271/300 Write block 4569 len 20480 data: 5 5 6 6 5
30594 272/300 Read block 2503 len 32768
30595 273/300 Write block 401 len 28672 data: 3 4 5 4 7 8 9
30596 274/300 Read block 412 len 16384
30597 275/300 Write block 246 len 40960 data: 7 6 6 7 7 6 5 5 5 6
30598 276/300 Read block 4121 len 24576
30599 277/300 Read block 4063 len 4096
30600 278/300 Read block 696 len 20480
30601 279/300 Write block 2752 len 40960 data: 6 5 4 5 4 7 7 7 7 7
30602 280/300 Read block 999 len 16384
30603 281/300 Read block 2771 len 40960
30604 282/300 Read block 1047 len 32768
30605 283/300 Read block 2989 len 12288
30606 284/300 Write block 2941 len 12288 data: 6 6 5
30607 285/300 Write block 947 len 20480 data: 5 7 7 7 7
30608 286/300 Write block 461 len 8192 data: 8 9
30609 287/300 Write block 4952 len 16384 data: 6 5 6 7
30610 288/300 Write block 3895 len 12288 data: 8 10 9
30611 289/300 Read block 1769 len 24576
30612 290/300 Read block 4071 len 28672
30613 291/300 Write block 2231 len 8192 data: 4 4
30614 292/300 Read block 1475 len 28672
30615 293/300 Read block 3455 len 8192
30616 294/300 Read block 3919 len 36864
30617 295/300 Read block 4332 len 20480
30618 296/300 Write block 4135 len 20480 data: 7 7 7 8 5
30619 297/300 Read block 3197 len 12288
30620 298/300 Read block 4712 len 16384
30621 299/300 Write block 3618 len 12288 data: 4 4 5
30622 300/300 Write block 1696 len 8192 data: 2 2
30623 301/300 Read block 1794 len 4096
30624 Main task has work to do, go find it
30625 got dsc Start(2)
30626 start 2
30627 Jan 03 08:37:20.380 INFO request completed, latency_us: 217, response_code: 204, uri: /start/cid/2, method: POST, req_id: 02ed9070-d2e1-4009-b8d2-67ea65a295e3, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
30628 Replay: started 2, returned:Ok(())
30629 ----------------------------------------------------------------
30630 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30631 States: Active Active Offline
30632 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30633 8850 Acked Write 2 Done Done Sent false
30634 8851 Acked Read 9 Done Done Sent false
30635 8852 Acked Read 2 Done Done Sent false
30636 8853 Acked Read 3 Done Done Sent false
30637 8854 Acked Read 7 Done Done Sent false
30638 8855 Acked Write 3 Done Done Sent false
30639 8856 Acked Flush 0 Done Done Sent false
30640 8857 Acked Write 5 Done Done Sent false
30641 8858 Acked Read 8 Done Done Sent false
30642 8859 Acked Write 3 Done Done Sent false
30643 8860 Acked Write 5 Done Done Sent false
30644 8861 Acked Read 10 Done Done Sent false
30645 8862 Acked Write 1 Done Done Sent false
30646 8863 Acked Read 9 Done Done Sent false
30647 8864 Acked Read 7 Done Done Sent false
30648 8865 Acked Read 5 Done Done Sent false
30649 8866 Acked Read 4 Done Done Sent false
30650 8867 Acked Read 9 Done Done Sent false
30651 8868 Acked Read 3 Done Done Sent false
30652 8869 Acked Flush 0 Done Done Sent false
30653 8870 Acked Read 1 Done Done Sent false
30654 8871 Acked Read 6 Done Done Sent false
30655 8872 Acked Read 5 Done Done Sent false
30656 8873 Acked Read 8 Done Done Sent false
30657 8874 Acked Write 9 Done Done Sent false
30658 8875 Acked Write 5 Done Done Sent false
30659 8876 Acked Write 2 Done Done Sent false
30660 8877 Acked Write 9 Done Done Sent false
30661 8878 Acked Flush 0 Done Done Sent false
30662 8879 Acked Read 10 Done Done Sent false
30663 8880 Acked Read 5 Done Done Sent false
30664 8881 Acked Read 8 Done Done Sent false
30665 8882 Acked Read 10 Done Done Sent false
30666 8883 Acked Write 9 Done Done Sent false
30667 8884 Acked Read 6 Done Done Sent false
30668 8885 Acked Write 2 Done Done Sent false
30669 8886 Acked Write 4 Done Done Sent false
30670 8887 Acked Read 7 Done Done Sent false
30671 8888 Acked Write 8 Done Done Sent false
30672 8889 Acked Read 8 Done Done Sent false
30673 8890 Acked Read 10 Done Done Sent false
30674 8891 Acked Read 4 Done Done Sent false
30675 8892 Acked Write 3 Done Done Sent false
30676 8893 Acked Read 7 Done Done Sent false
30677 8894 Acked Write 8 Done Done Sent false
30678 8895 Acked Read 7 Done Done Sent false
30679 8896 Acked Read 8 Done Done Sent false
30680 8897 Acked Read 1 Done Done Sent false
30681 8898 Acked Read 3 Done Done Sent false
30682 8899 Acked Flush 0 Done Done Sent false
30683 8900 Acked Read 1 Done Done Sent false
30684 8901 Acked Read 7 Done Done Sent false
30685 8902 Acked Write 2 Done Done Sent false
30686 8903 Acked Write 2 Done Done Sent false
30687 8904 Acked Read 8 Done Done Sent false
30688 8905 Acked Read 9 Done Done Sent false
30689 8906 Acked Read 8 Done Done Sent false
30690 8907 Acked Write 1 Done Done Sent false
30691 8908 Acked Read 5 Done Done Sent false
30692 8909 Acked Read 1 Done Done Sent false
30693 8910 Acked Write 6 Done Done Sent false
30694 8911 Acked Flush 0 Done Done Sent false
30695 8912 Acked Read 2 Done Done Sent false
30696 8913 Acked Read 4 Done Done Sent false
30697 8914 Acked Write 4 Done Done Sent false
30698 8915 Acked Read 1 Done Done Sent false
30699 8916 Acked Write 1 Done Done Sent false
30700 8917 Acked Read 10 Done Done Sent false
30701 8918 Acked Read 8 Done Done Sent false
30702 8919 Acked Read 4 Done Done Sent false
30703 8920 Acked Flush 0 Done Done Sent false
30704 8921 Acked Read 9 Done Done Sent false
30705 8922 Acked Write 3 Done Done Sent false
30706 8923 Acked Read 5 Done Done Sent false
30707 8924 Acked Flush 0 Done Done Sent false
30708 8925 Acked Flush 0 Done Done Sent false
30709 8926 Acked Read 10 Done Done Sent false
30710 8927 Acked Read 10 Done Done Sent false
30711 8928 Acked Write 8 Done Done Sent false
30712 8929 Acked Write 3 Done Done Sent false
30713 8930 Acked Write 6 Done Done Sent false
30714 8931 Acked Flush 0 Done Done Sent false
30715 8932 Acked Write 10 Done Done Sent false
30716 8933 Acked Read 3 Done Done Sent false
30717 8934 Acked Read 9 Done Done Sent false
30718 8935 Acked Write 4 Done Done Sent false
30719 8936 Acked Write 5 Done Done Sent false
30720 8937 Acked Read 6 Done Done Sent false
30721 8938 Acked Write 2 Done Done Sent false
30722 8939 Acked Write 9 Done Done Sent false
30723 8940 Acked Write 6 Done Done Sent false
30724 8941 Acked Read 10 Done Done Sent false
30725 8942 Acked Read 5 Done Done Sent false
30726 8943 Acked Flush 0 Done Done Sent false
30727 8944 Acked Read 6 Done Done Sent false
30728 8945 Acked Flush 0 Done Done Sent false
30729 8946 Acked Write 9 Done Done Sent false
30730 8947 Acked Write 8 Done Done Sent false
30731 8948 Acked Write 2 Done Done Sent false
30732 8949 Acked Flush 0 Done Done Sent false
30733 8950 Acked Write 6 Done Done Sent false
30734 8951 Acked Read 5 Done Done Sent false
30735 8952 Acked Write 8 Done Done Sent false
30736 8953 Acked Write 3 Done Done Sent false
30737 8954 Acked Read 1 Done Done Sent false
30738 8955 Acked Read 6 Done Done Sent false
30739 8956 Acked Read 6 Done Done Sent false
30740 8957 Acked Read 1 Done Done Sent false
30741 8958 Acked Read 8 Done Done Sent false
30742 8959 Acked Read 4 Done Done Sent false
30743 8960 Acked Read 7 Done Done Sent false
30744 8961 Acked Write 6 Done Done Sent false
30745 8962 Acked Write 4 Done Done Sent false
30746 8963 Acked Read 9 Done Done Sent false
30747 8964 Acked Flush 0 Done Done Sent false
30748 8965 Acked Write 3 Done Done Sent false
30749 8966 Acked Read 5 Done Done Sent false
30750 8967 Acked Write 7 Done Done Sent false
30751 8968 Acked Write 6 Done Done Sent false
30752 8969 Acked Write 6 Done Done Sent false
30753 8970 Acked Write 5 Done Done Sent false
30754 8971 Acked Flush 0 Done Done Sent false
30755 8972 Acked Write 10 Done Done Sent false
30756 8973 Acked Read 8 Done Done Sent false
30757 8974 Acked Flush 0 Done Done Sent false
30758 8975 Acked Read 6 Done Done Sent false
30759 8976 Acked Read 3 Done Done Sent false
30760 8977 Acked Read 8 Done Done Sent false
30761 8978 Acked Flush 0 Done Done Sent false
30762 8979 Acked Read 7 Done Done Sent false
30763 8980 Acked Write 3 Done Done Sent false
30764 8981 Acked Flush 0 Done Done Sent false
30765 8982 Acked Read 2 Done Done Sent false
30766 8983 Acked Write 8 Done Done Sent false
30767 8984 Acked Read 3 Done Done Sent false
30768 8985 Acked Read 2 Done Done Sent false
30769 8986 Acked Read 3 Done Done Sent false
30770 8987 Acked Read 9 Done Done Sent false
30771 8988 Acked Write 2 Done Done Sent false
30772 8989 Acked Read 8 Done Done Sent false
30773 8990 Acked Read 6 Done Done Sent false
30774 8991 Acked Write 5 Done Done Sent false
30775 8992 Acked Write 4 Done Done Sent false
30776 8993 Acked Write 1 Done Done Sent false
30777 8994 Acked Read 2 Done Done Sent false
30778 8995 Acked Write 7 Done Done Sent false
30779 8996 Acked Write 6 Done Done Sent false
30780 8997 Acked Read 10 Done Done Sent false
30781 8998 Acked Read 2 Done Done Sent false
30782 8999 Acked Write 4 Done Done Sent false
30783 9000 Acked Write 8 Done Done Sent false
30784 9001 Acked Read 5 Done Done Sent false
30785 9002 Acked Read 4 Done Done Sent false
30786 9003 Acked Read 7 Done Done Sent false
30787 9004 Acked Write 4 Done Done Sent false
30788 9005 Acked Write 7 Done Done Sent false
30789 9006 Acked Read 1 Done Done Sent false
30790 9007 Acked Write 7 Done Done Sent false
30791 9008 Acked Read 6 Done Done Sent false
30792 9009 Acked Read 5 Done Done Sent false
30793 9010 Acked Read 10 Done Done Sent false
30794 9011 Acked Write 7 Done Done Sent false
30795 9012 Acked Read 7 Done Done Sent false
30796 9013 Acked Read 9 Done Done Sent false
30797 9014 Acked Write 10 Done Done Sent false
30798 9015 Acked Write 5 Done Done Sent false
30799 9016 Acked Read 2 Done Done Sent false
30800 9017 Acked Write 4 Done Done Sent false
30801 9018 Acked Flush 0 Done Done Sent false
30802 9019 Acked Read 6 Done Done Sent false
30803 9020 Acked Write 10 Done Done Sent false
30804 9021 Acked Write 3 Done Done Sent false
30805 9022 Acked Read 9 Done Done Sent false
30806 9023 Acked Write 6 Done Done Sent false
30807 9024 Acked Write 9 Done Done Sent false
30808 9025 Acked Read 8 Done Done Sent false
30809 9026 Acked Read 7 Done Done Sent false
30810 9027 Acked Read 5 Done Done Sent false
30811 9028 Acked Read 10 Done Done Sent false
30812 9029 Acked Read 6 Done Done Sent false
30813 9030 Acked Write 6 Done Done Sent false
30814 9031 Acked Write 10 Done Done Sent false
30815 9032 Acked Write 5 Done Done Sent false
30816 9033 Acked Write 7 Done Done Sent false
30817 9034 Acked Read 1 Done Done Sent false
30818 9035 Acked Flush 0 Done Done Sent false
30819 9036 Acked Read 7 Done Done Sent false
30820 9037 Acked Flush 0 Done Done Sent false
30821 9038 Acked Read 4 Done Done Sent false
30822 9039 Acked Read 8 Done Done Sent false
30823 9040 Acked Flush 0 Done Done Sent false
30824 9041 Acked Read 8 Done Done Sent false
30825 9042 Acked Write 7 Done Done Sent false
30826 9043 Acked Read 2 Done Done Sent false
30827 9044 Acked Write 2 Done Done Sent false
30828 9045 Acked Write 3 Done Done Sent false
30829 9046 Acked Write 7 Done Done Sent false
30830 9047 Acked Read 3 Done Done Sent false
30831 9048 Acked Read 1 Done Done Sent false
30832 9049 Acked Flush 0 Done Done Sent false
30833 9050 Acked Write 2 Done Done Sent false
30834 9051 Acked Write 9 Done Done Sent false
30835 9052 Acked Read 2 Done Done Sent false
30836 9053 Acked Read 3 Done Done Sent false
30837 9054 Acked Write 2 Done Done Sent false
30838 9055 Acked Flush 0 Done Done Sent false
30839 9056 Acked Write 6 Done Done Sent false
30840 9057 Acked Flush 0 Done Done Sent false
30841 9058 Acked Read 8 Done Done Sent false
30842 9059 Acked Write 6 Done Done Sent false
30843 9060 Acked Read 1 Done Done Sent false
30844 9061 Acked Write 4 Done Done Sent false
30845 9062 Acked Read 10 Done Done Sent false
30846 9063 Acked Read 2 Done Done Sent false
30847 9064 Acked Read 10 Done Done Sent false
30848 9065 Acked Read 8 Done Done Sent false
30849 9066 Acked Read 10 Done Done Sent false
30850 9067 Acked Read 6 Done Done Sent false
30851 9068 Acked Write 9 Done Done Sent false
30852 9069 Acked Flush 0 Done Done Sent false
30853 9070 Acked Flush 0 Done Done Sent false
30854 9071 Acked Read 7 Done Done Sent false
30855 9072 Acked Read 8 Done Done Sent false
30856 9073 Acked Read 4 Done Done Sent false
30857 9074 Acked Read 3 Done Done Sent false
30858 9075 Acked Read 9 Done Done Sent false
30859 9076 Acked Read 4 Done Done Sent false
30860 9077 Acked Read 4 Done Done Sent false
30861 9078 Acked Write 6 Done Done Sent false
30862 9079 Acked Write 10 Done Done Sent false
30863 9080 Acked Read 4 Done Done Sent false
30864 9081 Acked Read 3 Done Done Sent false
30865 9082 Acked Read 9 Done Done Sent false
30866 9083 Acked Write 8 Done Done Sent false
30867 9084 Acked Read 9 Done Done Sent false
30868 9085 Acked Read 3 Done Done Sent false
30869 9086 Acked Read 8 Done Done Sent false
30870 9087 Acked Write 10 Done Done Sent false
30871 9088 Acked Read 8 Done Done Sent false
30872 9089 Acked Write 7 Done Done Sent false
30873 9090 Acked Read 5 Done Done Sent false
30874 9091 Acked Read 3 Done Done Sent false
30875 9092 Acked Read 6 Done Done Sent false
30876 9093 Acked Flush 0 Done Done Sent false
30877 9094 Acked Read 1 Done Done Sent false
30878 9095 Acked Read 2 Done Done Sent false
30879 9096 Acked Read 7 Done Done Sent false
30880 9097 Acked Read 2 Done Done Sent false
30881 9098 Acked Read 7 Done Done Sent false
30882 9099 Acked Read 1 Done Done Sent false
30883 9100 Acked Flush 0 Done Done Sent false
30884 9101 Acked Write 10 Done Done Sent false
30885 9102 Acked Read 5 Done Done Sent false
30886 9103 Acked Write 6 Done Done Sent false
30887 9104 Acked Write 10 Done Done Sent false
30888 9105 Acked Read 8 Done Done Sent false
30889 9106 Acked Write 9 Done Done Sent false
30890 9107 Acked Read 9 Done Done Sent false
30891 9108 Acked Write 9 Done Done Sent false
30892 9109 Acked Read 9 Done Done Sent false
30893 9110 Acked Write 10 Done Done Sent false
30894 9111 Acked Write 6 Done Done Sent false
30895 9112 Acked Read 3 Done Done Sent false
30896 9113 Acked Read 5 Done Done Sent false
30897 9114 Acked Read 4 Done Done Sent false
30898 9115 Acked Read 4 Done Done Sent false
30899 9116 Acked Write 3 Done Done Sent false
30900 9117 Acked Write 7 Done Done Sent false
30901 9118 Acked Write 10 Done Done Sent false
30902 9119 Acked Write 3 Done Done Sent false
30903 9120 Acked Write 5 Done Done Sent false
30904 9121 Acked Read 8 Done Done Sent false
30905 9122 Acked Write 7 Done Done Sent false
30906 9123 Acked Read 4 Done Done Sent false
30907 9124 Acked Write 10 Done Done Sent false
30908 9125 Acked Read 6 Done Done Sent false
30909 9126 Acked Read 1 Done Done Sent false
30910 9127 Acked Read 5 Done Done Sent false
30911 9128 Acked Write 10 Done Done Sent false
30912 9129 Acked Read 4 Done Done Sent false
30913 9130 Acked Read 10 Done Done Sent false
30914 9131 Acked Read 8 Done Done Sent false
30915 9132 Acked Read 3 Done Done Sent false
30916 9133 Acked Write 3 Done Done Sent false
30917 9134 Acked Write 5 Done Done Sent false
30918 9135 Acked Write 2 Done Done Sent false
30919 9136 Acked Write 4 Done Done Sent false
30920 9137 Acked Write 3 Done Done Sent false
30921 9138 Acked Read 6 Done Done Sent false
30922 9139 Acked Read 7 Done Done Sent false
30923 9140 Acked Write 2 Done Done Sent false
30924 9141 Acked Read 7 Done Done Sent false
30925 9142 Acked Read 2 Done Done Sent false
30926 9143 Acked Read 9 Done Done Sent false
30927 9144 Acked Read 5 Done Done Sent false
30928 9145 Acked Write 5 Done Done Sent false
30929 9146 Acked Read 3 Done Done Sent false
30930 9147 Acked Read 4 Done Done Sent false
30931 9148 Acked Write 3 Done Done Sent false
30932 9149 Acked Write 2 Done Done Sent false
30933 9150 Acked Read 1 Done Done Sent false
30934 STATES DS:0 DS:1 DS:2 TOTAL
30935 Sent 0 0 301 301
30936 Done 301 301 0 602
30937 Skipped 0 0 0 0
30938 Error 0 0 0 0
30939 Last Flush: 9100 9100 8849
30940 Downstairs last five completed: 8849 8848 8847 8846 8845
30941 Upstairs last five completed: 9150 9149 9148 9147 9146
30942 CLIENT: Up:0 ds:301 act:2
30943 [8830] got start action
30944 [8830] I am going to restart
30945 Starting downstairs at port 8830
30946 Make output file at "/tmp/dsc/downstairs-8830.txt"
30947 [8830][2] reports Starting
30948 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1232)
30949 [8830][2] reports Running
30950 ----------------------------------------------------------------
30951 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30952 States: Active Active Offline
30953 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30954 8850 Acked Write 2 Done Done Sent false
30955 8851 Acked Read 9 Done Done Sent false
30956 8852 Acked Read 2 Done Done Sent false
30957 8853 Acked Read 3 Done Done Sent false
30958 8854 Acked Read 7 Done Done Sent false
30959 8855 Acked Write 3 Done Done Sent false
30960 8856 Acked Flush 0 Done Done Sent false
30961 8857 Acked Write 5 Done Done Sent false
30962 8858 Acked Read 8 Done Done Sent false
30963 8859 Acked Write 3 Done Done Sent false
30964 8860 Acked Write 5 Done Done Sent false
30965 8861 Acked Read 10 Done Done Sent false
30966 8862 Acked Write 1 Done Done Sent false
30967 8863 Acked Read 9 Done Done Sent false
30968 8864 Acked Read 7 Done Done Sent false
30969 8865 Acked Read 5 Done Done Sent false
30970 8866 Acked Read 4 Done Done Sent false
30971 8867 Acked Read 9 Done Done Sent false
30972 8868 Acked Read 3 Done Done Sent false
30973 8869 Acked Flush 0 Done Done Sent false
30974 8870 Acked Read 1 Done Done Sent false
30975 8871 Acked Read 6 Done Done Sent false
30976 8872 Acked Read 5 Done Done Sent false
30977 8873 Acked Read 8 Done Done Sent false
30978 8874 Acked Write 9 Done Done Sent false
30979 8875 Acked Write 5 Done Done Sent false
30980 8876 Acked Write 2 Done Done Sent false
30981 8877 Acked Write 9 Done Done Sent false
30982 8878 Acked Flush 0 Done Done Sent false
30983 8879 Acked Read 10 Done Done Sent false
30984 8880 Acked Read 5 Done Done Sent false
30985 8881 Acked Read 8 Done Done Sent false
30986 8882 Acked Read 10 Done Done Sent false
30987 8883 Acked Write 9 Done Done Sent false
30988 8884 Acked Read 6 Done Done Sent false
30989 8885 Acked Write 2 Done Done Sent false
30990 8886 Acked Write 4 Done Done Sent false
30991 8887 Acked Read 7 Done Done Sent false
30992 8888 Acked Write 8 Done Done Sent false
30993 8889 Acked Read 8 Done Done Sent false
30994 8890 Acked Read 10 Done Done Sent false
30995 8891 Acked Read 4 Done Done Sent false
30996 8892 Acked Write 3 Done Done Sent false
30997 8893 Acked Read 7 Done Done Sent false
30998 8894 Acked Write 8 Done Done Sent false
30999 8895 Acked Read 7 Done Done Sent false
31000 8896 Acked Read 8 Done Done Sent false
31001 8897 Acked Read 1 Done Done Sent false
31002 8898 Acked Read 3 Done Done Sent false
31003 8899 Acked Flush 0 Done Done Sent false
31004 8900 Acked Read 1 Done Done Sent false
31005 8901 Acked Read 7 Done Done Sent false
31006 8902 Acked Write 2 Done Done Sent false
31007 8903 Acked Write 2 Done Done Sent false
31008 8904 Acked Read 8 Done Done Sent false
31009 8905 Acked Read 9 Done Done Sent false
31010 8906 Acked Read 8 Done Done Sent false
31011 8907 Acked Write 1 Done Done Sent false
31012 8908 Acked Read 5 Done Done Sent false
31013 8909 Acked Read 1 Done Done Sent false
31014 8910 Acked Write 6 Done Done Sent false
31015 8911 Acked Flush 0 Done Done Sent false
31016 8912 Acked Read 2 Done Done Sent false
31017 8913 Acked Read 4 Done Done Sent false
31018 8914 Acked Write 4 Done Done Sent false
31019 8915 Acked Read 1 Done Done Sent false
31020 8916 Acked Write 1 Done Done Sent false
31021 8917 Acked Read 10 Done Done Sent false
31022 8918 Acked Read 8 Done Done Sent false
31023 8919 Acked Read 4 Done Done Sent false
31024 8920 Acked Flush 0 Done Done Sent false
31025 8921 Acked Read 9 Done Done Sent false
31026 8922 Acked Write 3 Done Done Sent false
31027 8923 Acked Read 5 Done Done Sent false
31028 8924 Acked Flush 0 Done Done Sent false
31029 8925 Acked Flush 0 Done Done Sent false
31030 8926 Acked Read 10 Done Done Sent false
31031 8927 Acked Read 10 Done Done Sent false
31032 8928 Acked Write 8 Done Done Sent false
31033 8929 Acked Write 3 Done Done Sent false
31034 8930 Acked Write 6 Done Done Sent false
31035 8931 Acked Flush 0 Done Done Sent false
31036 8932 Acked Write 10 Done Done Sent false
31037 8933 Acked Read 3 Done Done Sent false
31038 8934 Acked Read 9 Done Done Sent false
31039 8935 Acked Write 4 Done Done Sent false
31040 8936 Acked Write 5 Done Done Sent false
31041 8937 Acked Read 6 Done Done Sent false
31042 8938 Acked Write 2 Done Done Sent false
31043 8939 Acked Write 9 Done Done Sent false
31044 8940 Acked Write 6 Done Done Sent false
31045 8941 Acked Read 10 Done Done Sent false
31046 8942 Acked Read 5 Done Done Sent false
31047 8943 Acked Flush 0 Done Done Sent false
31048 8944 Acked Read 6 Done Done Sent false
31049 8945 Acked Flush 0 Done Done Sent false
31050 8946 Acked Write 9 Done Done Sent false
31051 8947 Acked Write 8 Done Done Sent false
31052 8948 Acked Write 2 Done Done Sent false
31053 8949 Acked Flush 0 Done Done Sent false
31054 8950 Acked Write 6 Done Done Sent false
31055 8951 Acked Read 5 Done Done Sent false
31056 8952 Acked Write 8 Done Done Sent false
31057 8953 Acked Write 3 Done Done Sent false
31058 8954 Acked Read 1 Done Done Sent false
31059 8955 Acked Read 6 Done Done Sent false
31060 8956 Acked Read 6 Done Done Sent false
31061 8957 Acked Read 1 Done Done Sent false
31062 8958 Acked Read 8 Done Done Sent false
31063 8959 Acked Read 4 Done Done Sent false
31064 8960 Acked Read 7 Done Done Sent false
31065 8961 Acked Write 6 Done Done Sent false
31066 8962 Acked Write 4 Done Done Sent false
31067 8963 Acked Read 9 Done Done Sent false
31068 8964 Acked Flush 0 Done Done Sent false
31069 8965 Acked Write 3 Done Done Sent false
31070 8966 Acked Read 5 Done Done Sent false
31071 8967 Acked Write 7 Done Done Sent false
31072 8968 Acked Write 6 Done Done Sent false
31073 8969 Acked Write 6 Done Done Sent false
31074 8970 Acked Write 5 Done Done Sent false
31075 8971 Acked Flush 0 Done Done Sent false
31076 8972 Acked Write 10 Done Done Sent false
31077 8973 Acked Read 8 Done Done Sent false
31078 8974 Acked Flush 0 Done Done Sent false
31079 8975 Acked Read 6 Done Done Sent false
31080 8976 Acked Read 3 Done Done Sent false
31081 8977 Acked Read 8 Done Done Sent false
31082 8978 Acked Flush 0 Done Done Sent false
31083 8979 Acked Read 7 Done Done Sent false
31084 8980 Acked Write 3 Done Done Sent false
31085 8981 Acked Flush 0 Done Done Sent false
31086 8982 Acked Read 2 Done Done Sent false
31087 8983 Acked Write 8 Done Done Sent false
31088 8984 Acked Read 3 Done Done Sent false
31089 8985 Acked Read 2 Done Done Sent false
31090 8986 Acked Read 3 Done Done Sent false
31091 8987 Acked Read 9 Done Done Sent false
31092 8988 Acked Write 2 Done Done Sent false
31093 8989 Acked Read 8 Done Done Sent false
31094 8990 Acked Read 6 Done Done Sent false
31095 8991 Acked Write 5 Done Done Sent false
31096 8992 Acked Write 4 Done Done Sent false
31097 8993 Acked Write 1 Done Done Sent false
31098 8994 Acked Read 2 Done Done Sent false
31099 8995 Acked Write 7 Done Done Sent false
31100 8996 Acked Write 6 Done Done Sent false
31101 8997 Acked Read 10 Done Done Sent false
31102 8998 Acked Read 2 Done Done Sent false
31103 8999 Acked Write 4 Done Done Sent false
31104 9000 Acked Write 8 Done Done Sent false
31105 9001 Acked Read 5 Done Done Sent false
31106 9002 Acked Read 4 Done Done Sent false
31107 9003 Acked Read 7 Done Done Sent false
31108 9004 Acked Write 4 Done Done Sent false
31109 9005 Acked Write 7 Done Done Sent false
31110 9006 Acked Read 1 Done Done Sent false
31111 9007 Acked Write 7 Done Done Sent false
31112 9008 Acked Read 6 Done Done Sent false
31113 9009 Acked Read 5 Done Done Sent false
31114 9010 Acked Read 10 Done Done Sent false
31115 9011 Acked Write 7 Done Done Sent false
31116 9012 Acked Read 7 Done Done Sent false
31117 9013 Acked Read 9 Done Done Sent false
31118 9014 Acked Write 10 Done Done Sent false
31119 9015 Acked Write 5 Done Done Sent false
31120 9016 Acked Read 2 Done Done Sent false
31121 9017 Acked Write 4 Done Done Sent false
31122 9018 Acked Flush 0 Done Done Sent false
31123 9019 Acked Read 6 Done Done Sent false
31124 9020 Acked Write 10 Done Done Sent false
31125 9021 Acked Write 3 Done Done Sent false
31126 9022 Acked Read 9 Done Done Sent false
31127 9023 Acked Write 6 Done Done Sent false
31128 9024 Acked Write 9 Done Done Sent false
31129 9025 Acked Read 8 Done Done Sent false
31130 9026 Acked Read 7 Done Done Sent false
31131 9027 Acked Read 5 Done Done Sent false
31132 9028 Acked Read 10 Done Done Sent false
31133 9029 Acked Read 6 Done Done Sent false
31134 9030 Acked Write 6 Done Done Sent false
31135 9031 Acked Write 10 Done Done Sent false
31136 9032 Acked Write 5 Done Done Sent false
31137 9033 Acked Write 7 Done Done Sent false
31138 9034 Acked Read 1 Done Done Sent false
31139 9035 Acked Flush 0 Done Done Sent false
31140 9036 Acked Read 7 Done Done Sent false
31141 9037 Acked Flush 0 Done Done Sent false
31142 9038 Acked Read 4 Done Done Sent false
31143 9039 Acked Read 8 Done Done Sent false
31144 9040 Acked Flush 0 Done Done Sent false
31145 9041 Acked Read 8 Done Done Sent false
31146 9042 Acked Write 7 Done Done Sent false
31147 9043 Acked Read 2 Done Done Sent false
31148 9044 Acked Write 2 Done Done Sent false
31149 9045 Acked Write 3 Done Done Sent false
31150 9046 Acked Write 7 Done Done Sent false
31151 9047 Acked Read 3 Done Done Sent false
31152 9048 Acked Read 1 Done Done Sent false
31153 9049 Acked Flush 0 Done Done Sent false
31154 9050 Acked Write 2 Done Done Sent false
31155 9051 Acked Write 9 Done Done Sent false
31156 9052 Acked Read 2 Done Done Sent false
31157 9053 Acked Read 3 Done Done Sent false
31158 9054 Acked Write 2 Done Done Sent false
31159 9055 Acked Flush 0 Done Done Sent false
31160 9056 Acked Write 6 Done Done Sent false
31161 9057 Acked Flush 0 Done Done Sent false
31162 9058 Acked Read 8 Done Done Sent false
31163 9059 Acked Write 6 Done Done Sent false
31164 9060 Acked Read 1 Done Done Sent false
31165 9061 Acked Write 4 Done Done Sent false
31166 9062 Acked Read 10 Done Done Sent false
31167 9063 Acked Read 2 Done Done Sent false
31168 9064 Acked Read 10 Done Done Sent false
31169 9065 Acked Read 8 Done Done Sent false
31170 9066 Acked Read 10 Done Done Sent false
31171 9067 Acked Read 6 Done Done Sent false
31172 9068 Acked Write 9 Done Done Sent false
31173 9069 Acked Flush 0 Done Done Sent false
31174 9070 Acked Flush 0 Done Done Sent false
31175 9071 Acked Read 7 Done Done Sent false
31176 9072 Acked Read 8 Done Done Sent false
31177 9073 Acked Read 4 Done Done Sent false
31178 9074 Acked Read 3 Done Done Sent false
31179 9075 Acked Read 9 Done Done Sent false
31180 9076 Acked Read 4 Done Done Sent false
31181 9077 Acked Read 4 Done Done Sent false
31182 9078 Acked Write 6 Done Done Sent false
31183 9079 Acked Write 10 Done Done Sent false
31184 9080 Acked Read 4 Done Done Sent false
31185 9081 Acked Read 3 Done Done Sent false
31186 9082 Acked Read 9 Done Done Sent false
31187 9083 Acked Write 8 Done Done Sent false
31188 9084 Acked Read 9 Done Done Sent false
31189 9085 Acked Read 3 Done Done Sent false
31190 9086 Acked Read 8 Done Done Sent false
31191 9087 Acked Write 10 Done Done Sent false
31192 9088 Acked Read 8 Done Done Sent false
31193 9089 Acked Write 7 Done Done Sent false
31194 9090 Acked Read 5 Done Done Sent false
31195 9091 Acked Read 3 Done Done Sent false
31196 9092 Acked Read 6 Done Done Sent false
31197 9093 Acked Flush 0 Done Done Sent false
31198 9094 Acked Read 1 Done Done Sent false
31199 9095 Acked Read 2 Done Done Sent false
31200 9096 Acked Read 7 Done Done Sent false
31201 9097 Acked Read 2 Done Done Sent false
31202 9098 Acked Read 7 Done Done Sent false
31203 9099 Acked Read 1 Done Done Sent false
31204 9100 Acked Flush 0 Done Done Sent false
31205 9101 Acked Write 10 Done Done Sent false
31206 9102 Acked Read 5 Done Done Sent false
31207 9103 Acked Write 6 Done Done Sent false
31208 9104 Acked Write 10 Done Done Sent false
31209 9105 Acked Read 8 Done Done Sent false
31210 9106 Acked Write 9 Done Done Sent false
31211 9107 Acked Read 9 Done Done Sent false
31212 9108 Acked Write 9 Done Done Sent false
31213 9109 Acked Read 9 Done Done Sent false
31214 9110 Acked Write 10 Done Done Sent false
31215 9111 Acked Write 6 Done Done Sent false
31216 9112 Acked Read 3 Done Done Sent false
31217 9113 Acked Read 5 Done Done Sent false
31218 9114 Acked Read 4 Done Done Sent false
31219 9115 Acked Read 4 Done Done Sent false
31220 9116 Acked Write 3 Done Done Sent false
31221 9117 Acked Write 7 Done Done Sent false
31222 9118 Acked Write 10 Done Done Sent false
31223 9119 Acked Write 3 Done Done Sent false
31224 9120 Acked Write 5 Done Done Sent false
31225 9121 Acked Read 8 Done Done Sent false
31226 9122 Acked Write 7 Done Done Sent false
31227 9123 Acked Read 4 Done Done Sent false
31228 9124 Acked Write 10 Done Done Sent false
31229 9125 Acked Read 6 Done Done Sent false
31230 9126 Acked Read 1 Done Done Sent false
31231 9127 Acked Read 5 Done Done Sent false
31232 9128 Acked Write 10 Done Done Sent false
31233 9129 Acked Read 4 Done Done Sent false
31234 9130 Acked Read 10 Done Done Sent false
31235 9131 Acked Read 8 Done Done Sent false
31236 9132 Acked Read 3 Done Done Sent false
31237 9133 Acked Write 3 Done Done Sent false
31238 9134 Acked Write 5 Done Done Sent false
31239 9135 Acked Write 2 Done Done Sent false
31240 9136 Acked Write 4 Done Done Sent false
31241 9137 Acked Write 3 Done Done Sent false
31242 9138 Acked Read 6 Done Done Sent false
31243 9139 Acked Read 7 Done Done Sent false
31244 9140 Acked Write 2 Done Done Sent false
31245 9141 Acked Read 7 Done Done Sent false
31246 9142 Acked Read 2 Done Done Sent false
31247 9143 Acked Read 9 Done Done Sent false
31248 9144 Acked Read 5 Done Done Sent false
31249 9145 Acked Write 5 Done Done Sent false
31250 9146 Acked Read 3 Done Done Sent false
31251 9147 Acked Read 4 Done Done Sent false
31252 9148 Acked Write 3 Done Done Sent false
31253 9149 Acked Write 2 Done Done Sent false
31254 9150 Acked Read 1 Done Done Sent false
31255 9151 Acked Flush 0 Done Done Sent false
31256 STATES DS:0 DS:1 DS:2 TOTAL
31257 Sent 0 0 302 302
31258 Done 302 302 0 604
31259 Skipped 0 0 0 0
31260 Error 0 0 0 0
31261 Last Flush: 9151 9151 8849
31262 Downstairs last five completed: 8849 8848 8847 8846 8845
31263 Upstairs last five completed: 9151 9150 9149 9148 9147
31264 CLIENT: Up:0 ds:302 act:2
312652025-01-03T08:37:26.128ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312662025-01-03T08:37:26.128ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62309) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312672025-01-03T08:37:26.129ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312682025-01-03T08:37:26.130ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312692025-01-03T08:37:26.130ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312702025-01-03T08:37:26.130ZINFOcrucible: send last flush ID to this DS: 8849 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312712025-01-03T08:37:26.130ZINFOcrucible: Replied this last flush ID: 8849 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312722025-01-03T08:37:26.130ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312732025-01-03T08:37:26.130ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312742025-01-03T08:37:26.130ZINFOcrucible: [2] client re-new 302 jobs since flush 8849 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312752025-01-03T08:37:26.130ZINFOcrucible: [2] Replayed 302 jobs since flush: 8849 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
31276 ----------------------------------------------------------------
31277 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31278 Downstairs last five completed: 9151 9150 9149 9148 9147
31279 Upstairs last five completed: 9151 9150 9149 9148 9147
31280 CLIENT: Up:0 ds:0 act:3
31281 Replay: All jobs finished
31282 Main task has work to do, go find it
31283 got dsc Stop(2)
31284 Jan 03 08:37:28.386stop 2
31285 INFO request completed, latency_us: 321, response_code: 204, uri: /stop/cid/2, method: POST, req_id: adc7c195-55ce-4746-bb7d-7afd7e542ec1, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
31286 [8830] Got stop action so:false kr:false
31287 Jan 03 08:37:28.386 INFO request completed, latency_us: 153, response_code: 200, uri: /state/cid/2, method: GET, req_id: 98d8bd01-e97a-4068-b16e-92684e7f8e05, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
312882025-01-03T08:37:28.386ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312892025-01-03T08:37:28.387ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312902025-01-03T08:37:28.387ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312912025-01-03T08:37:28.387ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312922025-01-03T08:37:28.387ZINFOcrucible: [2] Transition from Active to Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
312932025-01-03T08:37:28.387ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
31294 [8830] Exited with: ExitStatus(unix_wait_status(9))
31295 [8830][2] reports Exit
31296 Jan 03 08:37:32.389 INFO request completed, latency_us: 223, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4bb22616-9683-4380-b28d-9f7883b2894e, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
31297 001/300 Read block 2244 len 20480
31298 002/300 Write block 2470 len 36864 data: 6 5 6 7 6 6 5 5 4
31299 003/300 Read block 3372 len 36864
31300 004/300 Flush
31301 005/300 Write block 170 len 28672 data: 4 6 9 7 6 7 6
31302 006/300 Read block 1671 len 4096
31303 007/300 Write block 1669 len 8192 data: 7 7
31304 008/300 Write block 149 len 32768 data: 3 4 4 4 4 3 5 5
31305 009/300 Read block 4785 len 8192
31306 010/300 Write block 1352 len 36864 data: 5 5 5 6 6 6 7 6 7
31307 011/300 Read block 3306 len 36864
31308 012/300 Read block 4646 len 16384
31309 013/300 Read block 3150 len 24576
31310 014/300 Write block 2567 len 40960 data: 6 8 9 8 8 8 6 5 6 5
31311 015/300 Read block 2054 len 20480
31312 016/300 Flush
31313 017/300 Write block 1140 len 8192 data: 5 5
31314 018/300 Read block 3870 len 12288
31315 019/300 Write block 2767 len 32768 data: 5 5 5 4 5 3 3 4
31316 020/300 Flush
31317 021/300 Flush
31318 022/300 Read block 232 len 20480
31319 023/300 Read block 3050 len 20480
31320 024/300 Write block 4485 len 8192 data: 6 6
31321 025/300 Read block 4672 len 4096
31322 026/300 Read block 3657 len 20480
31323 027/300 Read block 1340 len 24576
31324 028/300 Read block 2727 len 4096
31325 029/300 Read block 4483 len 16384
31326 030/300 Read block 1608 len 32768
31327 031/300 Read block 82 len 36864
31328 032/300 Write block 1917 len 32768 data: 4 4 7 8 7 8 9 10
31329 033/300 Read block 2303 len 36864
31330 034/300 Write block 4093 len 36864 data: 4 5 6 8 6 3 3 3 2
31331 035/300 Write block 213 len 32768 data: 3 3 5 3 4 5 4 3
31332 036/300 Write block 1957 len 36864 data: 4 3 5 4 5 6 6 4 3
31333 037/300 Read block 1020 len 24576
31334 038/300 Write block 4374 len 20480 data: 7 7 5 5 6
31335 039/300 Read block 112 len 36864
31336 040/300 Write block 4254 len 32768 data: 8 8 7 6 6 6 6 5
31337 041/300 Read block 4869 len 16384
31338 042/300 Flush
31339 043/300 Write block 4794 len 8192 data: 5 6
31340 044/300 Read block 1121 len 4096
31341 045/300 Write block 254 len 36864 data: 6 7 4 5 3 3 3 3 3
31342 046/300 Read block 3158 len 16384
31343 047/300 Read block 1736 len 8192
31344 048/300 Read block 3026 len 12288
31345 049/300 Write block 1288 len 24576 data: 6 7 7 9 8 8
31346 050/300 Read block 2472 len 32768
31347 051/300 Write block 1300 len 28672 data: 2 5 5 5 7 7 6
31348 052/300 Write block 3451 len 16384 data: 5 5 6 8
31349 053/300 Read block 938 len 12288
31350 054/300 Write block 3950 len 40960 data: 6 7 6 5 5 4 5 5 5 3
31351 055/300 Read block 594 len 36864
31352 056/300 Read block 2226 len 4096
31353 057/300 Read block 2593 len 40960
31354 058/300 Read block 1461 len 4096
31355 059/300 Read block 53 len 36864
31356 060/300 Read block 1323 len 16384
31357 061/300 Read block 1914 len 8192
31358 062/300 Write block 2290 len 40960 data: 6 6 5 5 4 5 3 4 6 6
31359 063/300 Write block 3732 len 16384 data: 8 7 5 5
31360 064/300 Write block 4945 len 24576 data: 6 6 8 9 8 7
31361 065/300 Write block 331 len 28672 data: 6 6 5 4 4 4 4
31362 066/300 Write block 2801 len 4096 data: 3
31363 067/300 Read block 4491 len 20480
31364 068/300 Write block 829 len 32768 data: 4 4 4 2 2 2 2 3
31365 069/300 Read block 1392 len 24576
31366 070/300 Write block 1690 len 40960 data: 5 4 2 2 2 2 3 3 2 3
31367 071/300 Flush
31368 072/300 Write block 437 len 16384 data: 5 6 7 8
31369 073/300 Flush
31370 074/300 Read block 4590 len 4096
31371 075/300 Flush
31372 076/300 Write block 2279 len 40960 data: 3 4 3 3 3 6 6 6 4 6
31373 077/300 Write block 1430 len 8192 data: 5 3
31374 078/300 Write block 212 len 12288 data: 3 4 4
31375 079/300 Read block 1218 len 36864
31376 080/300 Read block 1875 len 8192
31377 081/300 Flush
31378 082/300 Read block 4964 len 32768
31379 083/300 Write block 4905 len 8192 data: 5 5
31380 084/300 Write block 3339 len 32768 data: 2 2 2 2 3 3 3 3
31381 085/300 Write block 1098 len 8192 data: 10 9
31382 086/300 Read block 4268 len 16384
31383 087/300 Flush
31384 088/300 Read block 2857 len 32768
31385 089/300 Read block 3268 len 20480
31386 090/300 Write block 2850 len 24576 data: 4 4 4 5 6 5
31387 091/300 Flush
31388 092/300 Write block 2307 len 16384 data: 10 11 12 11
31389 093/300 Write block 198 len 16384 data: 7 6 5 4
31390 094/300 Write block 2584 len 40960 data: 7 7 5 3 4 3 3 3 3 4
31391 095/300 Write block 460 len 36864 data: 7 9 10 7 6 6 7 7 7
31392 096/300 Read block 933 len 40960
31393 097/300 Read block 882 len 32768
31394 098/300 Write block 1017 len 8192 data: 6 5
31395 099/300 Flush
31396 100/300 Write block 3214 len 8192 data: 4 4
31397 101/300 Read block 2213 len 8192
31398 102/300 Read block 2530 len 16384
31399 103/300 Write block 592 len 20480 data: 5 4 4 4 5
31400 104/300 Write block 4529 len 32768 data: 5 4 4 5 5 6 7 5
31401 105/300 Read block 1944 len 36864
31402 106/300 Flush
31403 107/300 Read block 3698 len 36864
31404 108/300 Read block 760 len 8192
31405 109/300 Read block 1045 len 36864
31406 110/300 Write block 1253 len 8192 data: 4 5
31407 111/300 Write block 1571 len 16384 data: 7 8 8 6
31408 112/300 Read block 1927 len 40960
31409 113/300 Read block 5 len 8192
31410 114/300 Read block 2640 len 8192
31411 115/300 Write block 19 len 36864 data: 4 4 5 5 4 4 3 3 3
31412 116/300 Write block 2754 len 40960 data: 5 6 5 8 8 8 8 8 7 7
31413 117/300 Flush
31414 118/300 Write block 2675 len 40960 data: 2 3 4 5 5 5 4 6 5 4
31415 119/300 Read block 1762 len 24576
31416 120/300 Read block 1849 len 24576
31417 121/300 Read block 422 len 8192
31418 122/300 Read block 4828 len 20480
31419 123/300 Write block 1816 len 12288 data: 4 6 8
31420 124/300 Write block 3576 len 8192 data: 5 4
31421 125/300 Read block 1733 len 16384
31422 126/300 Flush
31423 127/300 Read block 867 len 20480
31424 128/300 Read block 496 len 28672
31425 129/300 Read block 8 len 36864
31426 130/300 Read block 4983 len 20480
31427 131/300 Write block 3847 len 16384 data: 3 4 5 7
31428 132/300 Read block 4930 len 24576
31429 133/300 Read block 4944 len 32768
31430 134/300 Write block 1712 len 4096 data: 6
31431 135/300 Read block 4502 len 20480
31432 136/300 Read block 1421 len 32768
31433 137/300 Flush
31434 138/300 Read block 3912 len 36864
31435 139/300 Write block 1121 len 8192 data: 9 9
31436 140/300 Flush
31437 141/300 Read block 2233 len 28672
31438 142/300 Read block 551 len 4096
31439 143/300 Read block 955 len 20480
31440 144/300 Write block 2483 len 20480 data: 10 9 10 9 10
31441 145/300 Read block 4512 len 40960
31442 146/300 Flush
31443 147/300 Read block 359 len 32768
31444 148/300 Write block 417 len 12288 data: 6 5 7
31445 149/300 Read block 229 len 36864
31446 150/300 Write block 1898 len 4096 data: 8
31447 151/300 Read block 3602 len 4096
31448 152/300 Write block 4419 len 12288 data: 6 6 6
31449 153/300 Read block 4067 len 20480
31450 154/300 Write block 3742 len 28672 data: 4 6 5 5 4 6 5
31451 155/300 Read block 3073 len 32768
31452 156/300 Read block 4305 len 12288
31453 157/300 Read block 899 len 20480
31454 158/300 Write block 642 len 36864 data: 6 6 5 4 3 4 4 5 4
31455 159/300 Read block 4250 len 32768
31456 160/300 Read block 4114 len 4096
31457 161/300 Write block 2697 len 16384 data: 7 6 4 6
31458 162/300 Read block 2519 len 36864
31459 163/300 Read block 4706 len 20480
31460 164/300 Flush
31461 165/300 Read block 2718 len 32768
31462 166/300 Write block 1444 len 12288 data: 6 6 8
31463 167/300 Write block 3106 len 20480 data: 3 3 3 4 4
31464 168/300 Write block 762 len 24576 data: 6 6 7 5 6 6
31465 169/300 Read block 845 len 28672
31466 170/300 Read block 4970 len 12288
31467 171/300 Write block 1538 len 4096 data: 4
31468 172/300 Read block 388 len 36864
31469 173/300 Write block 1315 len 4096 data: 5
31470 174/300 Write block 3182 len 40960 data: 6 8 7 6 5 5 5 4 5 7
31471 175/300 Read block 529 len 32768
31472 176/300 Write block 3013 len 16384 data: 6 6 6 7
31473 177/300 Flush
31474 178/300 Flush
31475 179/300 Read block 3441 len 40960
31476 180/300 Write block 4090 len 40960 data: 2 2 3 5 6 7 9 7 4 4
31477 181/300 Flush
31478 182/300 Flush
31479 183/300 Write block 1277 len 36864 data: 4 6 6 6 6 5 7 8 7
31480 184/300 Write block 2641 len 36864 data: 9 9 8 8 7 6 4 4 4
31481 185/300 Write block 4865 len 32768 data: 7 7 7 8 9 8 8 7
31482 186/300 Read block 933 len 20480
31483 187/300 Write block 112 len 8192 data: 3 5
31484 188/300 Read block 1852 len 32768
31485 189/300 Read block 22 len 36864
31486 190/300 Write block 2682 len 28672 data: 7 6 5 4 4 8 7
31487 191/300 Write block 4643 len 4096 data: 5
31488 192/300 Write block 2553 len 36864 data: 4 4 5 4 4 5 5 4 5
31489 193/300 Write block 2099 len 4096 data: 6
31490 194/300 Read block 4120 len 16384
31491 195/300 Write block 1798 len 16384 data: 6 4 5 5
31492 196/300 Read block 4204 len 40960
31493 197/300 Write block 4004 len 32768 data: 10 10 9 8 8 5 4 5
31494 198/300 Read block 1867 len 20480
31495 199/300 Write block 3225 len 4096 data: 7
31496 200/300 Flush
31497 201/300 Write block 2201 len 8192 data: 5 8
31498 202/300 Write block 4669 len 28672 data: 6 7 6 6 6 6 5
31499 203/300 Read block 609 len 8192
31500 204/300 Write block 1186 len 4096 data: 4
31501 205/300 Read block 1701 len 4096
31502 206/300 Read block 1950 len 4096
31503 207/300 Read block 4229 len 24576
31504 208/300 Write block 158 len 32768 data: 6 6 6 6 6 5 6 6
31505 209/300 Flush
31506 210/300 Read block 4010 len 20480
31507 211/300 Write block 3123 len 24576 data: 5 5 6 6 6 5
31508 212/300 Read block 823 len 20480
31509 213/300 Read block 4065 len 24576
31510 214/300 Write block 1910 len 24576 data: 7 6 6 6 6 4
31511 215/300 Read block 2470 len 24576
31512 216/300 Read block 500 len 12288
31513 217/300 Flush
31514 218/300 Write block 4320 len 32768 data: 5 4 4 6 6 5 5 3
31515 219/300 Write block 1418 len 20480 data: 6 8 8 7 7
31516 220/300 Read block 4417 len 24576
31517 221/300 Read block 1991 len 20480
31518 222/300 Write block 4772 len 20480 data: 7 6 4 4 4
31519 223/300 Read block 3451 len 40960
31520 224/300 Write block 2682 len 28672 data: 8 7 6 5 5 9 8
31521 225/300 Read block 3920 len 32768
31522 226/300 Write block 2964 len 24576 data: 5 5 5 4 4 6
31523 227/300 Read block 2083 len 12288
31524 228/300 Write block 2733 len 16384 data: 8 9 7 5
31525 229/300 Read block 3139 len 28672
31526 230/300 Write block 1970 len 32768 data: 3 3 2 3 2 3 3 3
31527 231/300 Flush
31528 232/300 Write block 3743 len 36864 data: 7 6 6 5 7 6 5 5 4
31529 233/300 Write block 3820 len 20480 data: 6 7 6 5 6
31530 234/300 Write block 2955 len 32768 data: 4 5 5 6 6 5 5 5
31531 235/300 Write block 1162 len 12288 data: 4 4 5
31532 236/300 Read block 325 len 32768
31533 237/300 Write block 3700 len 12288 data: 6 7 6
31534 238/300 Read block 4576 len 8192
31535 239/300 Read block 1650 len 40960
31536 240/300 Write block 782 len 20480 data: 7 6 7 9 8
31537 241/300 Write block 4887 len 28672 data: 9 10 9 6 6 5 3
31538 242/300 Read block 4851 len 32768
31539 243/300 Read block 4199 len 8192
31540 244/300 Write block 3520 len 20480 data: 4 4 3 2 2
31541 245/300 Read block 151 len 4096
31542 246/300 Write block 4008 len 36864 data: 9 6 5 6 3 2 2 3 4
31543 247/300 Read block 1636 len 12288
31544 248/300 Flush
31545 249/300 Write block 2392 len 20480 data: 7 7 7 5 5
31546 250/300 Flush
31547 251/300 Write block 3081 len 20480 data: 4 4 7 7 8
31548 252/300 Flush
31549 253/300 Read block 3547 len 8192
31550 254/300 Read block 2191 len 16384
31551 255/300 Read block 2083 len 40960
31552 256/300 Read block 2035 len 24576
31553 257/300 Write block 81 len 24576 data: 7 5 6 5 6 6
31554 258/300 Read block 4206 len 8192
31555 259/300 Flush
31556 260/300 Write block 2073 len 4096 data: 9
31557 261/300 Flush
31558 262/300 Flush
31559 263/300 Read block 1919 len 12288
31560 264/300 Read block 4337 len 4096
31561 265/300 Flush
31562 266/300 Write block 2692 len 24576 data: 6 6 6 8 8 8
31563 267/300 Write block 4820 len 32768 data: 6 6 7 5 6 6 7 7
31564 268/300 Read block 2903 len 32768
31565 269/300 Read block 1679 len 4096
31566 270/300 Write block 1195 len 28672 data: 6 5 5 6 5 6 5
31567 271/300 Read block 3602 len 20480
31568 272/300 Write block 110 len 16384 data: 4 3 4 6
31569 273/300 Read block 1794 len 4096
31570 274/300 Write block 2679 len 24576 data: 6 6 5 9 8 7
31571 275/300 Read block 1300 len 32768
31572 276/300 Flush
31573 277/300 Write block 4340 len 32768 data: 10 9 10 9 8 8 7 7
31574 278/300 Write block 4112 len 36864 data: 5 5 5 5 6 7 6 8 5
31575 279/300 Read block 2500 len 24576
31576 280/300 Write block 3938 len 20480 data: 9 8 7 6 4
31577 281/300 Write block 4846 len 20480 data: 8 6 6 6 5
31578 282/300 Read block 3812 len 12288
31579 283/300 Read block 1547 len 4096
31580 284/300 Read block 2022 len 8192
31581 285/300 Read block 876 len 32768
31582 286/300 Write block 609 len 24576 data: 5 6 7 9 8 9
31583 287/300 Read block 3781 len 4096
31584 288/300 Flush
31585 289/300 Read block 2272 len 16384
31586 290/300 Write block 4017 len 20480 data: 7 7 7 7 7
31587 291/300 Read block 513 len 16384
31588 292/300 Write block 1229 len 24576 data: 5 4 5 5 5 2
31589 293/300 Read block 1447 len 20480
31590 294/300 Write block 2151 len 32768 data: 4 3 6 7 7 7 7 7
31591 295/300 Write block 619 len 4096 data: 6
31592 296/300 Flush
31593 297/300 Write block 1967 len 4096 data: 3
31594 298/300 Flush
31595 299/300 Read block 3647 len 32768
31596 300/300 Write block 4491 len 28672 data: 6 6 6 5 7 7 8
31597 301/300 Write block 3753 len 8192 data: 3 3
31598 Main task has work to do, go find it
31599 got dsc Start(2)
31600 start 2
31601 Jan 03 08:37:32.623 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/2, method: POST, req_id: 00d039a2-e51a-4eb4-a5b0-7ce1f0bd7c7a, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
31602 Replay: started 2, returned:Ok(())
31603 ----------------------------------------------------------------
31604 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31605 States: Active Active Offline
31606 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31607 9152 Acked Read 5 Done Done Sent false
31608 9153 Acked Write 9 Done Done Sent false
31609 9154 Acked Read 9 Done Done Sent false
31610 9155 Acked Flush 0 Done Done Sent false
31611 9156 Acked Write 7 Done Done Sent false
31612 9157 Acked Read 1 Done Done Sent false
31613 9158 Acked Write 2 Done Done Sent false
31614 9159 Acked Write 8 Done Done Sent false
31615 9160 Acked Read 2 Done Done Sent false
31616 9161 Acked Write 9 Done Done Sent false
31617 9162 Acked Read 9 Done Done Sent false
31618 9163 Acked Read 4 Done Done Sent false
31619 9164 Acked Read 6 Done Done Sent false
31620 9165 Acked Write 10 Done Done Sent false
31621 9166 Acked Read 5 Done Done Sent false
31622 9167 Acked Flush 0 Done Done Sent false
31623 9168 Acked Write 2 Done Done Sent false
31624 9169 Acked Read 3 Done Done Sent false
31625 9170 Acked Write 8 Done Done Sent false
31626 9171 Acked Flush 0 Done Done Sent false
31627 9172 Acked Flush 0 Done Done Sent false
31628 9173 Acked Read 5 Done Done Sent false
31629 9174 Acked Read 5 Done Done Sent false
31630 9175 Acked Write 2 Done Done Sent false
31631 9176 Acked Read 1 Done Done Sent false
31632 9177 Acked Read 5 Done Done Sent false
31633 9178 Acked Read 6 Done Done Sent false
31634 9179 Acked Read 1 Done Done Sent false
31635 9180 Acked Read 4 Done Done Sent false
31636 9181 Acked Read 8 Done Done Sent false
31637 9182 Acked Read 9 Done Done Sent false
31638 9183 Acked Write 8 Done Done Sent false
31639 9184 Acked Read 9 Done Done Sent false
31640 9185 Acked Write 9 Done Done Sent false
31641 9186 Acked Write 8 Done Done Sent false
31642 9187 Acked Write 9 Done Done Sent false
31643 9188 Acked Read 6 Done Done Sent false
31644 9189 Acked Write 5 Done Done Sent false
31645 9190 Acked Read 9 Done Done Sent false
31646 9191 Acked Write 8 Done Done Sent false
31647 9192 Acked Read 4 Done Done Sent false
31648 9193 Acked Flush 0 Done Done Sent false
31649 9194 Acked Write 2 Done Done Sent false
31650 9195 Acked Read 1 Done Done Sent false
31651 9196 Acked Write 9 Done Done Sent false
31652 9197 Acked Read 4 Done Done Sent false
31653 9198 Acked Read 2 Done Done Sent false
31654 9199 Acked Read 3 Done Done Sent false
31655 9200 Acked Write 6 Done Done Sent false
31656 9201 Acked Read 8 Done Done Sent false
31657 9202 Acked Write 7 Done Done Sent false
31658 9203 Acked Write 4 Done Done Sent false
31659 9204 Acked Read 3 Done Done Sent false
31660 9205 Acked Write 10 Done Done Sent false
31661 9206 Acked Read 9 Done Done Sent false
31662 9207 Acked Read 1 Done Done Sent false
31663 9208 Acked Read 10 Done Done Sent false
31664 9209 Acked Read 1 Done Done Sent false
31665 9210 Acked Read 9 Done Done Sent false
31666 9211 Acked Read 4 Done Done Sent false
31667 9212 Acked Read 2 Done Done Sent false
31668 9213 Acked Write 10 Done Done Sent false
31669 9214 Acked Write 4 Done Done Sent false
31670 9215 Acked Write 6 Done Done Sent false
31671 9216 Acked Write 7 Done Done Sent false
31672 9217 Acked Write 1 Done Done Sent false
31673 9218 Acked Read 5 Done Done Sent false
31674 9219 Acked Write 8 Done Done Sent false
31675 9220 Acked Read 6 Done Done Sent false
31676 9221 Acked Write 10 Done Done Sent false
31677 9222 Acked Flush 0 Done Done Sent false
31678 9223 Acked Write 4 Done Done Sent false
31679 9224 Acked Flush 0 Done Done Sent false
31680 9225 Acked Read 1 Done Done Sent false
31681 9226 Acked Flush 0 Done Done Sent false
31682 9227 Acked Write 10 Done Done Sent false
31683 9228 Acked Write 2 Done Done Sent false
31684 9229 Acked Write 3 Done Done Sent false
31685 9230 Acked Read 9 Done Done Sent false
31686 9231 Acked Read 2 Done Done Sent false
31687 9232 Acked Flush 0 Done Done Sent false
31688 9233 Acked Read 8 Done Done Sent false
31689 9234 Acked Write 2 Done Done Sent false
31690 9235 Acked Write 8 Done Done Sent false
31691 9236 Acked Write 2 Done Done Sent false
31692 9237 Acked Read 4 Done Done Sent false
31693 9238 Acked Flush 0 Done Done Sent false
31694 9239 Acked Read 8 Done Done Sent false
31695 9240 Acked Read 5 Done Done Sent false
31696 9241 Acked Write 6 Done Done Sent false
31697 9242 Acked Flush 0 Done Done Sent false
31698 9243 Acked Write 4 Done Done Sent false
31699 9244 Acked Write 4 Done Done Sent false
31700 9245 Acked Write 10 Done Done Sent false
31701 9246 Acked Write 9 Done Done Sent false
31702 9247 Acked Read 10 Done Done Sent false
31703 9248 Acked Read 8 Done Done Sent false
31704 9249 Acked Write 2 Done Done Sent false
31705 9250 Acked Flush 0 Done Done Sent false
31706 9251 Acked Write 2 Done Done Sent false
31707 9252 Acked Read 2 Done Done Sent false
31708 9253 Acked Read 4 Done Done Sent false
31709 9254 Acked Write 5 Done Done Sent false
31710 9255 Acked Write 8 Done Done Sent false
31711 9256 Acked Read 9 Done Done Sent false
31712 9257 Acked Flush 0 Done Done Sent false
31713 9258 Acked Read 9 Done Done Sent false
31714 9259 Acked Read 2 Done Done Sent false
31715 9260 Acked Read 9 Done Done Sent false
31716 9261 Acked Write 2 Done Done Sent false
31717 9262 Acked Write 4 Done Done Sent false
31718 9263 Acked Read 10 Done Done Sent false
31719 9264 Acked Read 2 Done Done Sent false
31720 9265 Acked Read 2 Done Done Sent false
31721 9266 Acked Write 9 Done Done Sent false
31722 9267 Acked Write 10 Done Done Sent false
31723 9268 Acked Flush 0 Done Done Sent false
31724 9269 Acked Write 10 Done Done Sent false
31725 9270 Acked Read 6 Done Done Sent false
31726 9271 Acked Read 6 Done Done Sent false
31727 9272 Acked Read 2 Done Done Sent false
31728 9273 Acked Read 5 Done Done Sent false
31729 9274 Acked Write 3 Done Done Sent false
31730 9275 Acked Write 2 Done Done Sent false
31731 9276 Acked Read 4 Done Done Sent false
31732 9277 Acked Flush 0 Done Done Sent false
31733 9278 Acked Read 5 Done Done Sent false
31734 9279 Acked Read 7 Done Done Sent false
31735 9280 Acked Read 9 Done Done Sent false
31736 9281 Acked Read 5 Done Done Sent false
31737 9282 Acked Write 4 Done Done Sent false
31738 9283 Acked Read 6 Done Done Sent false
31739 9284 Acked Read 8 Done Done Sent false
31740 9285 Acked Write 1 Done Done Sent false
31741 9286 Acked Read 5 Done Done Sent false
31742 9287 Acked Read 8 Done Done Sent false
31743 9288 Acked Flush 0 Done Done Sent false
31744 9289 Acked Read 9 Done Done Sent false
31745 9290 Acked Write 2 Done Done Sent false
31746 9291 Acked Flush 0 Done Done Sent false
31747 9292 Acked Read 7 Done Done Sent false
31748 9293 Acked Read 1 Done Done Sent false
31749 9294 Acked Read 5 Done Done Sent false
31750 9295 Acked Write 5 Done Done Sent false
31751 9296 Acked Read 10 Done Done Sent false
31752 9297 Acked Flush 0 Done Done Sent false
31753 9298 Acked Read 8 Done Done Sent false
31754 9299 Acked Write 3 Done Done Sent false
31755 9300 Acked Read 9 Done Done Sent false
31756 9301 Acked Write 1 Done Done Sent false
31757 9302 Acked Read 1 Done Done Sent false
31758 9303 Acked Write 3 Done Done Sent false
31759 9304 Acked Read 5 Done Done Sent false
31760 9305 Acked Write 7 Done Done Sent false
31761 9306 Acked Read 8 Done Done Sent false
31762 9307 Acked Read 3 Done Done Sent false
31763 9308 Acked Read 5 Done Done Sent false
31764 9309 Acked Write 9 Done Done Sent false
31765 9310 Acked Read 8 Done Done Sent false
31766 9311 Acked Read 1 Done Done Sent false
31767 9312 Acked Write 4 Done Done Sent false
31768 9313 Acked Read 9 Done Done Sent false
31769 9314 Acked Read 5 Done Done Sent false
31770 9315 Acked Flush 0 Done Done Sent false
31771 9316 Acked Read 8 Done Done Sent false
31772 9317 Acked Write 3 Done Done Sent false
31773 9318 Acked Write 5 Done Done Sent false
31774 9319 Acked Write 6 Done Done Sent false
31775 9320 Acked Read 7 Done Done Sent false
31776 9321 Acked Read 3 Done Done Sent false
31777 9322 Acked Write 1 Done Done Sent false
31778 9323 Acked Read 9 Done Done Sent false
31779 9324 Acked Write 1 Done Done Sent false
31780 9325 Acked Write 10 Done Done Sent false
31781 9326 Acked Read 8 Done Done Sent false
31782 9327 Acked Write 4 Done Done Sent false
31783 9328 Acked Flush 0 Done Done Sent false
31784 9329 Acked Flush 0 Done Done Sent false
31785 9330 Acked Read 10 Done Done Sent false
31786 9331 Acked Write 10 Done Done Sent false
31787 9332 Acked Flush 0 Done Done Sent false
31788 9333 Acked Flush 0 Done Done Sent false
31789 9334 Acked Write 9 Done Done Sent false
31790 9335 Acked Write 9 Done Done Sent false
31791 9336 Acked Write 8 Done Done Sent false
31792 9337 Acked Read 5 Done Done Sent false
31793 9338 Acked Write 2 Done Done Sent false
31794 9339 Acked Read 8 Done Done Sent false
31795 9340 Acked Read 9 Done Done Sent false
31796 9341 Acked Write 7 Done Done Sent false
31797 9342 Acked Write 1 Done Done Sent false
31798 9343 Acked Write 9 Done Done Sent false
31799 9344 Acked Write 1 Done Done Sent false
31800 9345 Acked Read 4 Done Done Sent false
31801 9346 Acked Write 4 Done Done Sent false
31802 9347 Acked Read 10 Done Done Sent false
31803 9348 Acked Write 8 Done Done Sent false
31804 9349 Acked Read 5 Done Done Sent false
31805 9350 Acked Write 1 Done Done Sent false
31806 9351 Acked Flush 0 Done Done Sent false
31807 9352 Acked Write 2 Done Done Sent false
31808 9353 Acked Write 7 Done Done Sent false
31809 9354 Acked Read 2 Done Done Sent false
31810 9355 Acked Write 1 Done Done Sent false
31811 9356 Acked Read 1 Done Done Sent false
31812 9357 Acked Read 1 Done Done Sent false
31813 9358 Acked Read 6 Done Done Sent false
31814 9359 Acked Write 8 Done Done Sent false
31815 9360 Acked Flush 0 Done Done Sent false
31816 9361 Acked Read 5 Done Done Sent false
31817 9362 Acked Write 6 Done Done Sent false
31818 9363 Acked Read 5 Done Done Sent false
31819 9364 Acked Read 6 Done Done Sent false
31820 9365 Acked Write 6 Done Done Sent false
31821 9366 Acked Read 6 Done Done Sent false
31822 9367 Acked Read 3 Done Done Sent false
31823 9368 Acked Flush 0 Done Done Sent false
31824 9369 Acked Write 8 Done Done Sent false
31825 9370 Acked Write 5 Done Done Sent false
31826 9371 Acked Read 6 Done Done Sent false
31827 9372 Acked Read 5 Done Done Sent false
31828 9373 Acked Write 5 Done Done Sent false
31829 9374 Acked Read 10 Done Done Sent false
31830 9375 Acked Write 7 Done Done Sent false
31831 9376 Acked Read 8 Done Done Sent false
31832 9377 Acked Write 6 Done Done Sent false
31833 9378 Acked Read 3 Done Done Sent false
31834 9379 Acked Write 4 Done Done Sent false
31835 9380 Acked Read 7 Done Done Sent false
31836 9381 Acked Write 8 Done Done Sent false
31837 9382 Acked Flush 0 Done Done Sent false
31838 9383 Acked Write 9 Done Done Sent false
31839 9384 Acked Write 5 Done Done Sent false
31840 9385 Acked Write 8 Done Done Sent false
31841 9386 Acked Write 3 Done Done Sent false
31842 9387 Acked Read 8 Done Done Sent false
31843 9388 Acked Write 3 Done Done Sent false
31844 9389 Acked Read 2 Done Done Sent false
31845 9390 Acked Read 10 Done Done Sent false
31846 9391 Acked Write 5 Done Done Sent false
31847 9392 Acked Write 7 Done Done Sent false
31848 9393 Acked Read 8 Done Done Sent false
31849 9394 Acked Read 2 Done Done Sent false
31850 9395 Acked Write 5 Done Done Sent false
31851 9396 Acked Read 1 Done Done Sent false
31852 9397 Acked Write 9 Done Done Sent false
31853 9398 Acked Read 3 Done Done Sent false
31854 9399 Acked Flush 0 Done Done Sent false
31855 9400 Acked Write 5 Done Done Sent false
31856 9401 Acked Flush 0 Done Done Sent false
31857 9402 Acked Write 5 Done Done Sent false
31858 9403 Acked Flush 0 Done Done Sent false
31859 9404 Acked Read 2 Done Done Sent false
31860 9405 Acked Read 4 Done Done Sent false
31861 9406 Acked Read 10 Done Done Sent false
31862 9407 Acked Read 6 Done Done Sent false
31863 9408 Acked Write 6 Done Done Sent false
31864 9409 Acked Read 2 Done Done Sent false
31865 9410 Acked Flush 0 Done Done Sent false
31866 9411 Acked Write 1 Done Done Sent false
31867 9412 Acked Flush 0 Done Done Sent false
31868 9413 Acked Flush 0 Done Done Sent false
31869 9414 Acked Read 3 Done Done Sent false
31870 9415 Acked Read 1 Done Done Sent false
31871 9416 Acked Flush 0 Done Done Sent false
31872 9417 Acked Write 6 Done Done Sent false
31873 9418 Acked Write 8 Done Done Sent false
31874 9419 Acked Read 8 Done Done Sent false
31875 9420 Acked Read 1 Done Done Sent false
31876 9421 Acked Write 7 Done Done Sent false
31877 9422 Acked Read 5 Done Done Sent false
31878 9423 Acked Write 4 Done Done Sent false
31879 9424 Acked Read 1 Done Done Sent false
31880 9425 Acked Write 6 Done Done Sent false
31881 9426 Acked Read 8 Done Done Sent false
31882 9427 Acked Flush 0 Done Done Sent false
31883 9428 Acked Write 8 Done Done Sent false
31884 9429 Acked Write 9 Done Done Sent false
31885 9430 Acked Read 6 Done Done Sent false
31886 9431 Acked Write 5 Done Done Sent false
31887 9432 Acked Write 5 Done Done Sent false
31888 9433 Acked Read 3 Done Done Sent false
31889 9434 Acked Read 1 Done Done Sent false
31890 9435 Acked Read 2 Done Done Sent false
31891 9436 Acked Read 8 Done Done Sent false
31892 9437 Acked Write 6 Done Done Sent false
31893 9438 Acked Read 1 Done Done Sent false
31894 9439 Acked Flush 0 Done Done Sent false
31895 9440 Acked Read 4 Done Done Sent false
31896 9441 Acked Write 5 Done Done Sent false
31897 9442 Acked Read 4 Done Done Sent false
31898 9443 Acked Write 6 Done Done Sent false
31899 9444 Acked Read 5 Done Done Sent false
31900 9445 Acked Write 8 Done Done Sent false
31901 9446 Acked Write 1 Done Done Sent false
31902 9447 Acked Flush 0 Done Done Sent false
31903 9448 Acked Write 1 Done Done Sent false
31904 9449 Acked Flush 0 Done Done Sent false
31905 9450 Acked Read 8 Done Done Sent false
31906 9451 Acked Write 7 Done Done Sent false
31907 9452 Acked Write 2 Done Done Sent false
31908 STATES DS:0 DS:1 DS:2 TOTAL
31909 Sent 0 0 301 301
31910 Done 301 301 0 602
31911 Skipped 0 0 0 0
31912 Error 0 0 0 0
31913 Last Flush: 9449 9449 9151
31914 Downstairs last five completed: 9151 9150 9149 9148 9147
31915 Upstairs last five completed: 9452 9451 9450 9449 9448
31916 CLIENT: Up:0 ds:301 act:2
31917 ----------------------------------------------------------------
31918 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31919 States: Active Active Offline
31920 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31921 9152 Acked Read 5 Done Done Sent false
31922 9153 Acked Write 9 Done Done Sent false
31923 9154 Acked Read 9 Done Done Sent false
31924 9155 Acked Flush 0 Done Done Sent false
31925 9156 Acked Write 7 Done Done Sent false
31926 9157 Acked Read 1 Done Done Sent false
31927 9158 Acked Write 2 Done Done Sent false
31928 9159 Acked Write 8 Done Done Sent false
31929 9160 Acked Read 2 Done Done Sent false
31930 9161 Acked Write 9 Done Done Sent false
31931 9162 Acked Read 9 Done Done Sent false
31932 9163 Acked Read 4 Done Done Sent false
31933 9164 Acked Read 6 Done Done Sent false
31934 9165 Acked Write 10 Done Done Sent false
31935 9166 Acked Read 5 Done Done Sent false
31936 9167 Acked Flush 0 Done Done Sent false
31937 9168 Acked Write 2 Done Done Sent false
31938 9169 Acked Read 3 Done Done Sent false
31939 9170 Acked Write 8 Done Done Sent false
31940 9171 Acked Flush 0 Done Done Sent false
31941 9172 Acked Flush 0 Done Done Sent false
31942 9173 Acked Read 5 Done Done Sent false
31943 9174 Acked Read 5 Done Done Sent false
31944 9175 Acked Write 2 Done Done Sent false
31945 9176 Acked Read 1 Done Done Sent false
31946 9177 Acked Read 5 Done Done Sent false
31947 9178 Acked Read 6 Done Done Sent false
31948 9179 Acked Read 1 Done Done Sent false
31949 9180 Acked Read 4 Done Done Sent false
31950 9181 Acked Read 8 Done Done Sent false
31951 9182 Acked Read 9 Done Done Sent false
31952 9183 Acked Write 8 Done Done Sent false
31953 9184 Acked Read 9 Done Done Sent false
31954 9185 Acked Write 9 Done Done Sent false
31955 9186 Acked Write 8 Done Done Sent false
31956 9187 Acked Write 9 Done Done Sent false
31957 9188 Acked Read 6 Done Done Sent false
31958 9189 Acked Write 5 Done Done Sent false
31959 9190 Acked Read 9 Done Done Sent false
31960 9191 Acked Write 8 Done Done Sent false
31961 9192 Acked Read 4 Done Done Sent false
31962 9193 Acked Flush 0 Done Done Sent false
31963 9194 Acked Write 2 Done Done Sent false
31964 9195 Acked Read 1 Done Done Sent false
31965 9196 Acked Write 9 Done Done Sent false
31966 9197 Acked Read 4 Done Done Sent false
31967 9198 Acked Read 2 Done Done Sent false
31968 9199 Acked Read 3 Done Done Sent false
31969 9200 Acked Write 6 Done Done Sent false
31970 9201 Acked Read 8 Done Done Sent false
31971 9202 Acked Write 7 Done Done Sent false
31972 9203 Acked Write 4 Done Done Sent false
31973 9204 Acked Read 3 Done Done Sent false
31974 9205 Acked Write 10 Done Done Sent false
31975 9206 Acked Read 9 Done Done Sent false
31976 9207 Acked Read 1 Done Done Sent false
31977 9208 Acked Read 10 Done Done Sent false
31978 9209 Acked Read 1 Done Done Sent false
31979 9210 Acked Read 9 Done Done Sent false
31980 9211 Acked Read 4 Done Done Sent false
31981 9212 Acked Read 2 Done Done Sent false
31982 9213 Acked Write 10 Done Done Sent false
31983 9214 Acked Write 4 Done Done Sent false
31984 9215 Acked Write 6 Done Done Sent false
31985 9216 Acked Write 7 Done Done Sent false
31986 9217 Acked Write 1 Done Done Sent false
31987 9218 Acked Read 5 Done Done Sent false
31988 9219 Acked Write 8 Done Done Sent false
31989 9220 Acked Read 6 Done Done Sent false
31990 9221 Acked Write 10 Done Done Sent false
31991 9222 Acked Flush 0 Done Done Sent false
31992 9223 Acked Write 4 Done Done Sent false
31993 9224 Acked Flush 0 Done Done Sent false
31994 9225 Acked Read 1 Done Done Sent false
31995 9226 Acked Flush 0 Done Done Sent false
31996 9227 Acked Write 10 Done Done Sent false
31997 9228 Acked Write 2 Done Done Sent false
31998 9229 Acked Write 3 Done Done Sent false
31999 9230 Acked Read 9 Done Done Sent false
32000 9231 Acked Read 2 Done Done Sent false
32001 9232 Acked Flush 0 Done Done Sent false
32002 9233 Acked Read 8 Done Done Sent false
32003 9234 Acked Write 2 Done Done Sent false
32004 9235 Acked Write 8 Done Done Sent false
32005 9236 Acked Write 2 Done Done Sent false
32006 9237 Acked Read 4 Done Done Sent false
32007 9238 Acked Flush 0 Done Done Sent false
32008 9239 Acked Read 8 Done Done Sent false
32009 9240 Acked Read 5 Done Done Sent false
32010 9241 Acked Write 6 Done Done Sent false
32011 9242 Acked Flush 0 Done Done Sent false
32012 9243 Acked Write 4 Done Done Sent false
32013 9244 Acked Write 4 Done Done Sent false
32014 9245 Acked Write 10 Done Done Sent false
32015 9246 Acked Write 9 Done Done Sent false
32016 9247 Acked Read 10 Done Done Sent false
32017 9248 Acked Read 8 Done Done Sent false
32018 9249 Acked Write 2 Done Done Sent false
32019 9250 Acked Flush 0 Done Done Sent false
32020 9251 Acked Write 2 Done Done Sent false
32021 9252 Acked Read 2 Done Done Sent false
32022 9253 Acked Read 4 Done Done Sent false
32023 9254 Acked Write 5 Done Done Sent false
32024 9255 Acked Write 8 Done Done Sent false
32025 9256 Acked Read 9 Done Done Sent false
32026 9257 Acked Flush 0 Done Done Sent false
32027 9258 Acked Read 9 Done Done Sent false
32028 9259 Acked Read 2 Done Done Sent false
32029 9260 Acked Read 9 Done Done Sent false
32030 9261 Acked Write 2 Done Done Sent false
32031 9262 Acked Write 4 Done Done Sent false
32032 9263 Acked Read 10 Done Done Sent false
32033 9264 Acked Read 2 Done Done Sent false
32034 9265 Acked Read 2 Done Done Sent false
32035 9266 Acked Write 9 Done Done Sent false
32036 9267 Acked Write 10 Done Done Sent false
32037 9268 Acked Flush 0 Done Done Sent false
32038 9269 Acked Write 10 Done Done Sent false
32039 9270 Acked Read 6 Done Done Sent false
32040 9271 Acked Read 6 Done Done Sent false
32041 9272 Acked Read 2 Done Done Sent false
32042 9273 Acked Read 5 Done Done Sent false
32043 9274 Acked Write 3 Done Done Sent false
32044 9275 Acked Write 2 Done Done Sent false
32045 9276 Acked Read 4 Done Done Sent false
32046 9277 Acked Flush 0 Done Done Sent false
32047 9278 Acked Read 5 Done Done Sent false
32048 9279 Acked Read 7 Done Done Sent false
32049 9280 Acked Read 9 Done Done Sent false
32050 9281 Acked Read 5 Done Done Sent false
32051 9282 Acked Write 4 Done Done Sent false
32052 9283 Acked Read 6 Done Done Sent false
32053 9284 Acked Read 8 Done Done Sent false
32054 9285 Acked Write 1 Done Done Sent false
32055 9286 Acked Read 5 Done Done Sent false
32056 9287 Acked Read 8 Done Done Sent false
32057 9288 Acked Flush 0 Done Done Sent false
32058 9289 Acked Read 9 Done Done Sent false
32059 9290 Acked Write 2 Done Done Sent false
32060 9291 Acked Flush 0 Done Done Sent false
32061 9292 Acked Read 7 Done Done Sent false
32062 9293 Acked Read 1 Done Done Sent false
32063 9294 Acked Read 5 Done Done Sent false
32064 9295 Acked Write 5 Done Done Sent false
32065 9296 Acked Read 10 Done Done Sent false
32066 9297 Acked Flush 0 Done Done Sent false
32067 9298 Acked Read 8 Done Done Sent false
32068 9299 Acked Write 3 Done Done Sent false
32069 9300 Acked Read 9 Done Done Sent false
32070 9301 Acked Write 1 Done Done Sent false
32071 9302 Acked Read 1 Done Done Sent false
32072 9303 Acked Write 3 Done Done Sent false
32073 9304 Acked Read 5 Done Done Sent false
32074 9305 Acked Write 7 Done Done Sent false
32075 9306 Acked Read 8 Done Done Sent false
32076 9307 Acked Read 3 Done Done Sent false
32077 9308 Acked Read 5 Done Done Sent false
32078 9309 Acked Write 9 Done Done Sent false
32079 9310 Acked Read 8 Done Done Sent false
32080 9311 Acked Read 1 Done Done Sent false
32081 9312 Acked Write 4 Done Done Sent false
32082 9313 Acked Read 9 Done Done Sent false
32083 9314 Acked Read 5 Done Done Sent false
32084 9315 Acked Flush 0 Done Done Sent false
32085 9316 Acked Read 8 Done Done Sent false
32086 9317 Acked Write 3 Done Done Sent false
32087 9318 Acked Write 5 Done Done Sent false
32088 9319 Acked Write 6 Done Done Sent false
32089 9320 Acked Read 7 Done Done Sent false
32090 9321 Acked Read 3 Done Done Sent false
32091 9322 Acked Write 1 Done Done Sent false
32092 9323 Acked Read 9 Done Done Sent false
32093 9324 Acked Write 1 Done Done Sent false
32094 9325 Acked Write 10 Done Done Sent false
32095 9326 Acked Read 8 Done Done Sent false
32096 9327 Acked Write 4 Done Done Sent false
32097 9328 Acked Flush 0 Done Done Sent false
32098 9329 Acked Flush 0 Done Done Sent false
32099 9330 Acked Read 10 Done Done Sent false
32100 9331 Acked Write 10 Done Done Sent false
32101 9332 Acked Flush 0 Done Done Sent false
32102 9333 Acked Flush 0 Done Done Sent false
32103 9334 Acked Write 9 Done Done Sent false
32104 9335 Acked Write 9 Done Done Sent false
32105 9336 Acked Write 8 Done Done Sent false
32106 9337 Acked Read 5 Done Done Sent false
32107 9338 Acked Write 2 Done Done Sent false
32108 9339 Acked Read 8 Done Done Sent false
32109 9340 Acked Read 9 Done Done Sent false
32110 9341 Acked Write 7 Done Done Sent false
32111 9342 Acked Write 1 Done Done Sent false
32112 9343 Acked Write 9 Done Done Sent false
32113 9344 Acked Write 1 Done Done Sent false
32114 9345 Acked Read 4 Done Done Sent false
32115 9346 Acked Write 4 Done Done Sent false
32116 9347 Acked Read 10 Done Done Sent false
32117 9348 Acked Write 8 Done Done Sent false
32118 9349 Acked Read 5 Done Done Sent false
32119 9350 Acked Write 1 Done Done Sent false
32120 9351 Acked Flush 0 Done Done Sent false
32121 9352 Acked Write 2 Done Done Sent false
32122 9353 Acked Write 7 Done Done Sent false
32123 9354 Acked Read 2 Done Done Sent false
32124 9355 Acked Write 1 Done Done Sent false
32125 9356 Acked Read 1 Done Done Sent false
32126 9357 Acked Read 1 Done Done Sent false
32127 9358 Acked Read 6 Done Done Sent false
32128 9359 Acked Write 8 Done Done Sent false
32129 9360 Acked Flush 0 Done Done Sent false
32130 9361 Acked Read 5 Done Done Sent false
32131 9362 Acked Write 6 Done Done Sent false
32132 9363 Acked Read 5 Done Done Sent false
32133 9364 Acked Read 6 Done Done Sent false
32134 9365 Acked Write 6 Done Done Sent false
32135 9366 Acked Read 6 Done Done Sent false
32136 9367 Acked Read 3 Done Done Sent false
32137 9368 Acked Flush 0 Done Done Sent false
32138 9369 Acked Write 8 Done Done Sent false
32139 9370 Acked Write 5 Done Done Sent false
32140 9371 Acked Read 6 Done Done Sent false
32141 9372 Acked Read 5 Done Done Sent false
32142 9373 Acked Write 5 Done Done Sent false
32143 9374 Acked Read 10 Done Done Sent false
32144 9375 Acked Write 7 Done Done Sent false
32145 9376 Acked Read 8 Done Done Sent false
32146 9377 Acked Write 6 Done Done Sent false
32147 9378 Acked Read 3 Done Done Sent false
32148 9379 Acked Write 4 Done Done Sent false
32149 9380 Acked Read 7 Done Done Sent false
32150 9381 Acked Write 8 Done Done Sent false
32151 9382 Acked Flush 0 Done Done Sent false
32152 9383 Acked Write 9 Done Done Sent false
32153 9384 Acked Write 5 Done Done Sent false
32154 9385 Acked Write 8 Done Done Sent false
32155 9386 Acked Write 3 Done Done Sent false
32156 9387 Acked Read 8 Done Done Sent false
32157 9388 Acked Write 3 Done Done Sent false
32158 9389 Acked Read 2 Done Done Sent false
32159 9390 Acked Read 10 Done Done Sent false
32160 9391 Acked Write 5 Done Done Sent false
32161 9392 Acked Write 7 Done Done Sent false
32162 9393 Acked Read 8 Done Done Sent false
32163 9394 Acked Read 2 Done Done Sent false
32164 9395 Acked Write 5 Done Done Sent false
32165 9396 Acked Read 1 Done Done Sent false
32166 9397 Acked Write 9 Done Done Sent false
32167 9398 Acked Read 3 Done Done Sent false
32168 9399 Acked Flush 0 Done Done Sent false
32169 9400 Acked Write 5 Done Done Sent false
32170 9401 Acked Flush 0 Done Done Sent false
32171 9402 Acked Write 5 Done Done Sent false
32172 9403 Acked Flush 0 Done Done Sent false
32173 9404 Acked Read 2 Done Done Sent false
32174 9405 Acked Read 4 Done Done Sent false
32175 9406 Acked Read 10 Done Done Sent false
32176 9407 Acked Read 6 Done Done Sent false
32177 9408 Acked Write 6 Done Done Sent false
32178 9409 Acked Read 2 Done Done Sent false
32179 9410 Acked Flush 0 Done Done Sent false
32180 9411 Acked Write 1 Done Done Sent false
32181 9412 Acked Flush 0 Done Done Sent false
32182 9413 Acked Flush 0 Done Done Sent false
32183 9414 Acked Read 3 Done Done Sent false
32184 9415 Acked Read 1 Done Done Sent false
32185 9416 Acked Flush 0 Done Done Sent false
32186 9417 Acked Write 6 Done Done Sent false
32187 9418 Acked Write 8 Done Done Sent false
32188 9419 Acked Read 8 Done Done Sent false
32189 9420 Acked Read 1 Done Done Sent false
32190 9421 Acked Write 7 Done Done Sent false
32191 9422 Acked Read 5 Done Done Sent false
32192 9423 Acked Write 4 Done Done Sent false
32193 9424 Acked Read 1 Done Done Sent false
32194 9425 Acked Write 6 Done Done Sent false
32195 9426 Acked Read 8 Done Done Sent false
32196 9427 Acked Flush 0 Done Done Sent false
32197 9428 Acked Write 8 Done Done Sent false
32198 9429 Acked Write 9 Done Done Sent false
32199 9430 Acked Read 6 Done Done Sent false
32200 9431 Acked Write 5 Done Done Sent false
32201 9432 Acked Write 5 Done Done Sent false
32202 9433 Acked Read 3 Done Done Sent false
32203 9434 Acked Read 1 Done Done Sent false
32204 9435 Acked Read 2 Done Done Sent false
32205 9436 Acked Read 8 Done Done Sent false
32206 9437 Acked Write 6 Done Done Sent false
32207 9438 Acked Read 1 Done Done Sent false
32208 9439 Acked Flush 0 Done Done Sent false
32209 9440 Acked Read 4 Done Done Sent false
32210 9441 Acked Write 5 Done Done Sent false
32211 9442 Acked Read 4 Done Done Sent false
32212 9443 Acked Write 6 Done Done Sent false
32213 9444 Acked Read 5 Done Done Sent false
32214 9445 Acked Write 8 Done Done Sent false
32215 9446 Acked Write 1 Done Done Sent false
32216 9447 Acked Flush 0 Done Done Sent false
32217 9448 Acked Write 1 Done Done Sent false
32218 9449 Acked Flush 0 Done Done Sent false
32219 9450 Acked Read 8 Done Done Sent false
32220 9451 Acked Write 7 Done Done Sent false
32221 9452 Acked Write 2 Done Done Sent false
32222 9453 Acked Flush 0 Done Done Sent false
32223 STATES DS:0 DS:1 DS:2 TOTAL
32224 Sent 0 0 302 302
32225 Done 302 302 0 604
32226 Skipped 0 0 0 0
32227 Error 0 0 0 0
32228 Last Flush: 9453 9453 9151
32229 Downstairs last five completed: 9151 9150 9149 9148 9147
32230 Upstairs last five completed: 9453 9452 9451 9450 9449
32231 CLIENT: Up:0 ds:302 act:2
322322025-01-03T08:37:38.389ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
322332025-01-03T08:37:38.389ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
322342025-01-03T08:37:38.389ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
322352025-01-03T08:37:38.389ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
322362025-01-03T08:37:38.389ZINFOcrucible: client task is exiting = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
32237 ----------------------------------------------------------------
32238 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32239 States: Active Active Offline
32240 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32241 9152 Acked Read 5 Done Done Sent false
32242 9153 Acked Write 9 Done Done Sent false
32243 9154 Acked Read 9 Done Done Sent false
32244 9155 Acked Flush 0 Done Done Sent false
32245 9156 Acked Write 7 Done Done Sent false
32246 9157 Acked Read 1 Done Done Sent false
32247 9158 Acked Write 2 Done Done Sent false
32248 9159 Acked Write 8 Done Done Sent false
32249 9160 Acked Read 2 Done Done Sent false
32250 9161 Acked Write 9 Done Done Sent false
32251 9162 Acked Read 9 Done Done Sent false
32252 9163 Acked Read 4 Done Done Sent false
32253 9164 Acked Read 6 Done Done Sent false
32254 9165 Acked Write 10 Done Done Sent false
32255 9166 Acked Read 5 Done Done Sent false
32256 9167 Acked Flush 0 Done Done Sent false
32257 9168 Acked Write 2 Done Done Sent false
32258 9169 Acked Read 3 Done Done Sent false
32259 9170 Acked Write 8 Done Done Sent false
32260 9171 Acked Flush 0 Done Done Sent false
32261 9172 Acked Flush 0 Done Done Sent false
32262 9173 Acked Read 5 Done Done Sent false
32263 9174 Acked Read 5 Done Done Sent false
32264 9175 Acked Write 2 Done Done Sent false
32265 9176 Acked Read 1 Done Done Sent false
32266 9177 Acked Read 5 Done Done Sent false
32267 9178 Acked Read 6 Done Done Sent false
32268 9179 Acked Read 1 Done Done Sent false
32269 9180 Acked Read 4 Done Done Sent false
32270 9181 Acked Read 8 Done Done Sent false
32271 9182 Acked Read 9 Done Done Sent false
32272 9183 Acked Write 8 Done Done Sent false
32273 9184 Acked Read 9 Done Done Sent false
32274 9185 Acked Write 9 Done Done Sent false
32275 9186 Acked Write 8 Done Done Sent false
32276 9187 Acked Write 9 Done Done Sent false
32277 9188 Acked Read 6 Done Done Sent false
32278 9189 Acked Write 5 Done Done Sent false
32279 9190 Acked Read 9 Done Done Sent false
32280 9191 Acked Write 8 Done Done Sent false
32281 9192 Acked Read 4 Done Done Sent false
32282 9193 Acked Flush 0 Done Done Sent false
32283 9194 Acked Write 2 Done Done Sent false
32284 9195 Acked Read 1 Done Done Sent false
32285 9196 Acked Write 9 Done Done Sent false
32286 9197 Acked Read 4 Done Done Sent false
32287 9198 Acked Read 2 Done Done Sent false
32288 9199 Acked Read 3 Done Done Sent false
32289 9200 Acked Write 6 Done Done Sent false
32290 9201 Acked Read 8 Done Done Sent false
32291 9202 Acked Write 7 Done Done Sent false
32292 9203 Acked Write 4 Done Done Sent false
32293 9204 Acked Read 3 Done Done Sent false
32294 9205 Acked Write 10 Done Done Sent false
32295 9206 Acked Read 9 Done Done Sent false
32296 9207 Acked Read 1 Done Done Sent false
32297 9208 Acked Read 10 Done Done Sent false
32298 9209 Acked Read 1 Done Done Sent false
32299 9210 Acked Read 9 Done Done Sent false
32300 9211 Acked Read 4 Done Done Sent false
32301 9212 Acked Read 2 Done Done Sent false
32302 9213 Acked Write 10 Done Done Sent false
32303 9214 Acked Write 4 Done Done Sent false
32304 9215 Acked Write 6 Done Done Sent false
32305 9216 Acked Write 7 Done Done Sent false
32306 9217 Acked Write 1 Done Done Sent false
32307 9218 Acked Read 5 Done Done Sent false
32308 9219 Acked Write 8 Done Done Sent false
32309 9220 Acked Read 6 Done Done Sent false
32310 9221 Acked Write 10 Done Done Sent false
32311 9222 Acked Flush 0 Done Done Sent false
32312 9223 Acked Write 4 Done Done Sent false
32313 9224 Acked Flush 0 Done Done Sent false
32314 9225 Acked Read 1 Done Done Sent false
32315 9226 Acked Flush 0 Done Done Sent false
32316 9227 Acked Write 10 Done Done Sent false
32317 9228 Acked Write 2 Done Done Sent false
32318 9229 Acked Write 3 Done Done Sent false
32319 9230 Acked Read 9 Done Done Sent false
32320 9231 Acked Read 2 Done Done Sent false
32321 9232 Acked Flush 0 Done Done Sent false
32322 9233 Acked Read 8 Done Done Sent false
32323 9234 Acked Write 2 Done Done Sent false
32324 9235 Acked Write 8 Done Done Sent false
32325 9236 Acked Write 2 Done Done Sent false
32326 9237 Acked Read 4 Done Done Sent false
32327 9238 Acked Flush 0 Done Done Sent false
32328 9239 Acked Read 8 Done Done Sent false
32329 9240 Acked Read 5 Done Done Sent false
32330 9241 Acked Write 6 Done Done Sent false
32331 9242 Acked Flush 0 Done Done Sent false
32332 9243 Acked Write 4 Done Done Sent false
32333 9244 Acked Write 4 Done Done Sent false
32334 9245 Acked Write 10 Done Done Sent false
32335 9246 Acked Write 9 Done Done Sent false
32336 9247 Acked Read 10 Done Done Sent false
32337 9248 Acked Read 8 Done Done Sent false
32338 9249 Acked Write 2 Done Done Sent false
32339 9250 Acked Flush 0 Done Done Sent false
32340 9251 Acked Write 2 Done Done Sent false
32341 9252 Acked Read 2 Done Done Sent false
32342 9253 Acked Read 4 Done Done Sent false
32343 9254 Acked Write 5 Done Done Sent false
32344 9255 Acked Write 8 Done Done Sent false
32345 9256 Acked Read 9 Done Done Sent false
32346 9257 Acked Flush 0 Done Done Sent false
32347 9258 Acked Read 9 Done Done Sent false
32348 9259 Acked Read 2 Done Done Sent false
32349 9260 Acked Read 9 Done Done Sent false
32350 9261 Acked Write 2 Done Done Sent false
32351 9262 Acked Write 4 Done Done Sent false
32352 9263 Acked Read 10 Done Done Sent false
32353 9264 Acked Read 2 Done Done Sent false
32354 9265 Acked Read 2 Done Done Sent false
32355 9266 Acked Write 9 Done Done Sent false
32356 9267 Acked Write 10 Done Done Sent false
32357 9268 Acked Flush 0 Done Done Sent false
32358 9269 Acked Write 10 Done Done Sent false
32359 9270 Acked Read 6 Done Done Sent false
32360 9271 Acked Read 6 Done Done Sent false
32361 9272 Acked Read 2 Done Done Sent false
32362 9273 Acked Read 5 Done Done Sent false
32363 9274 Acked Write 3 Done Done Sent false
32364 9275 Acked Write 2 Done Done Sent false
32365 9276 Acked Read 4 Done Done Sent false
32366 9277 Acked Flush 0 Done Done Sent false
32367 9278 Acked Read 5 Done Done Sent false
32368 9279 Acked Read 7 Done Done Sent false
32369 9280 Acked Read 9 Done Done Sent false
32370 9281 Acked Read 5 Done Done Sent false
32371 9282 Acked Write 4 Done Done Sent false
32372 9283 Acked Read 6 Done Done Sent false
32373 9284 Acked Read 8 Done Done Sent false
32374 9285 Acked Write 1 Done Done Sent false
32375 9286 Acked Read 5 Done Done Sent false
32376 9287 Acked Read 8 Done Done Sent false
32377 9288 Acked Flush 0 Done Done Sent false
32378 9289 Acked Read 9 Done Done Sent false
32379 9290 Acked Write 2 Done Done Sent false
32380 9291 Acked Flush 0 Done Done Sent false
32381 9292 Acked Read 7 Done Done Sent false
32382 9293 Acked Read 1 Done Done Sent false
32383 9294 Acked Read 5 Done Done Sent false
32384 9295 Acked Write 5 Done Done Sent false
32385 9296 Acked Read 10 Done Done Sent false
32386 9297 Acked Flush 0 Done Done Sent false
32387 9298 Acked Read 8 Done Done Sent false
32388 9299 Acked Write 3 Done Done Sent false
32389 9300 Acked Read 9 Done Done Sent false
32390 9301 Acked Write 1 Done Done Sent false
32391 9302 Acked Read 1 Done Done Sent false
32392 9303 Acked Write 3 Done Done Sent false
32393 9304 Acked Read 5 Done Done Sent false
32394 9305 Acked Write 7 Done Done Sent false
32395 9306 Acked Read 8 Done Done Sent false
32396 9307 Acked Read 3 Done Done Sent false
32397 9308 Acked Read 5 Done Done Sent false
32398 9309 Acked Write 9 Done Done Sent false
32399 9310 Acked Read 8 Done Done Sent false
32400 9311 Acked Read 1 Done Done Sent false
32401 9312 Acked Write 4 Done Done Sent false
32402 9313 Acked Read 9 Done Done Sent false
32403 9314 Acked Read 5 Done Done Sent false
32404 9315 Acked Flush 0 Done Done Sent false
32405 9316 Acked Read 8 Done Done Sent false
32406 9317 Acked Write 3 Done Done Sent false
32407 9318 Acked Write 5 Done Done Sent false
32408 9319 Acked Write 6 Done Done Sent false
32409 9320 Acked Read 7 Done Done Sent false
32410 9321 Acked Read 3 Done Done Sent false
32411 9322 Acked Write 1 Done Done Sent false
32412 9323 Acked Read 9 Done Done Sent false
32413 9324 Acked Write 1 Done Done Sent false
32414 9325 Acked Write 10 Done Done Sent false
32415 9326 Acked Read 8 Done Done Sent false
32416 9327 Acked Write 4 Done Done Sent false
32417 9328 Acked Flush 0 Done Done Sent false
32418 9329 Acked Flush 0 Done Done Sent false
32419 9330 Acked Read 10 Done Done Sent false
32420 9331 Acked Write 10 Done Done Sent false
32421 9332 Acked Flush 0 Done Done Sent false
32422 9333 Acked Flush 0 Done Done Sent false
32423 9334 Acked Write 9 Done Done Sent false
32424 9335 Acked Write 9 Done Done Sent false
32425 9336 Acked Write 8 Done Done Sent false
32426 9337 Acked Read 5 Done Done Sent false
32427 9338 Acked Write 2 Done Done Sent false
32428 9339 Acked Read 8 Done Done Sent false
32429 9340 Acked Read 9 Done Done Sent false
32430 9341 Acked Write 7 Done Done Sent false
32431 9342 Acked Write 1 Done Done Sent false
32432 9343 Acked Write 9 Done Done Sent false
32433 9344 Acked Write 1 Done Done Sent false
32434 9345 Acked Read 4 Done Done Sent false
32435 9346 Acked Write 4 Done Done Sent false
32436 9347 Acked Read 10 Done Done Sent false
32437 9348 Acked Write 8 Done Done Sent false
32438 9349 Acked Read 5 Done Done Sent false
32439 9350 Acked Write 1 Done Done Sent false
32440 9351 Acked Flush 0 Done Done Sent false
32441 9352 Acked Write 2 Done Done Sent false
32442 9353 Acked Write 7 Done Done Sent false
32443 9354 Acked Read 2 Done Done Sent false
32444 9355 Acked Write 1 Done Done Sent false
32445 9356 Acked Read 1 Done Done Sent false
32446 9357 Acked Read 1 Done Done Sent false
32447 9358 Acked Read 6 Done Done Sent false
32448 9359 Acked Write 8 Done Done Sent false
32449 9360 Acked Flush 0 Done Done Sent false
32450 9361 Acked Read 5 Done Done Sent false
32451 9362 Acked Write 6 Done Done Sent false
32452 9363 Acked Read 5 Done Done Sent false
32453 9364 Acked Read 6 Done Done Sent false
32454 9365 Acked Write 6 Done Done Sent false
32455 9366 Acked Read 6 Done Done Sent false
32456 9367 Acked Read 3 Done Done Sent false
32457 9368 Acked Flush 0 Done Done Sent false
32458 9369 Acked Write 8 Done Done Sent false
32459 9370 Acked Write 5 Done Done Sent false
32460 9371 Acked Read 6 Done Done Sent false
32461 9372 Acked Read 5 Done Done Sent false
32462 9373 Acked Write 5 Done Done Sent false
32463 9374 Acked Read 10 Done Done Sent false
32464 9375 Acked Write 7 Done Done Sent false
32465 9376 Acked Read 8 Done Done Sent false
32466 9377 Acked Write 6 Done Done Sent false
32467 9378 Acked Read 3 Done Done Sent false
32468 9379 Acked Write 4 Done Done Sent false
32469 9380 Acked Read 7 Done Done Sent false
32470 9381 Acked Write 8 Done Done Sent false
32471 9382 Acked Flush 0 Done Done Sent false
32472 9383 Acked Write 9 Done Done Sent false
32473 9384 Acked Write 5 Done Done Sent false
32474 9385 Acked Write 8 Done Done Sent false
32475 9386 Acked Write 3 Done Done Sent false
32476 9387 Acked Read 8 Done Done Sent false
32477 9388 Acked Write 3 Done Done Sent false
32478 9389 Acked Read 2 Done Done Sent false
32479 9390 Acked Read 10 Done Done Sent false
32480 9391 Acked Write 5 Done Done Sent false
32481 9392 Acked Write 7 Done Done Sent false
32482 9393 Acked Read 8 Done Done Sent false
32483 9394 Acked Read 2 Done Done Sent false
32484 9395 Acked Write 5 Done Done Sent false
32485 9396 Acked Read 1 Done Done Sent false
32486 9397 Acked Write 9 Done Done Sent false
32487 9398 Acked Read 3 Done Done Sent false
32488 9399 Acked Flush 0 Done Done Sent false
32489 9400 Acked Write 5 Done Done Sent false
32490 9401 Acked Flush 0 Done Done Sent false
32491 9402 Acked Write 5 Done Done Sent false
32492 9403 Acked Flush 0 Done Done Sent false
32493 9404 Acked Read 2 Done Done Sent false
32494 9405 Acked Read 4 Done Done Sent false
32495 9406 Acked Read 10 Done Done Sent false
32496 9407 Acked Read 6 Done Done Sent false
32497 9408 Acked Write 6 Done Done Sent false
32498 9409 Acked Read 2 Done Done Sent false
32499 9410 Acked Flush 0 Done Done Sent false
32500 9411 Acked Write 1 Done Done Sent false
32501 9412 Acked Flush 0 Done Done Sent false
32502 9413 Acked Flush 0 Done Done Sent false
32503 9414 Acked Read 3 Done Done Sent false
32504 9415 Acked Read 1 Done Done Sent false
32505 9416 Acked Flush 0 Done Done Sent false
32506 9417 Acked Write 6 Done Done Sent false
32507 9418 Acked Write 8 Done Done Sent false
32508 9419 Acked Read 8 Done Done Sent false
32509 9420 Acked Read 1 Done Done Sent false
32510 9421 Acked Write 7 Done Done Sent false
32511 9422 Acked Read 5 Done Done Sent false
32512 9423 Acked Write 4 Done Done Sent false
32513 9424 Acked Read 1 Done Done Sent false
32514 9425 Acked Write 6 Done Done Sent false
32515 9426 Acked Read 8 Done Done Sent false
32516 9427 Acked Flush 0 Done Done Sent false
32517 9428 Acked Write 8 Done Done Sent false
32518 9429 Acked Write 9 Done Done Sent false
32519 9430 Acked Read 6 Done Done Sent false
32520 9431 Acked Write 5 Done Done Sent false
32521 9432 Acked Write 5 Done Done Sent false
32522 9433 Acked Read 3 Done Done Sent false
32523 9434 Acked Read 1 Done Done Sent false
32524 9435 Acked Read 2 Done Done Sent false
32525 9436 Acked Read 8 Done Done Sent false
32526 9437 Acked Write 6 Done Done Sent false
32527 9438 Acked Read 1 Done Done Sent false
32528 9439 Acked Flush 0 Done Done Sent false
32529 9440 Acked Read 4 Done Done Sent false
32530 9441 Acked Write 5 Done Done Sent false
32531 9442 Acked Read 4 Done Done Sent false
32532 9443 Acked Write 6 Done Done Sent false
32533 9444 Acked Read 5 Done Done Sent false
32534 9445 Acked Write 8 Done Done Sent false
32535 9446 Acked Write 1 Done Done Sent false
32536 9447 Acked Flush 0 Done Done Sent false
32537 9448 Acked Write 1 Done Done Sent false
32538 9449 Acked Flush 0 Done Done Sent false
32539 9450 Acked Read 8 Done Done Sent false
32540 9451 Acked Write 7 Done Done Sent false
32541 9452 Acked Write 2 Done Done Sent false
32542 9453 Acked Flush 0 Done Done Sent false
32543 STATES DS:0 DS:1 DS:2 TOTAL
32544 Sent 0 0 302 302
32545 Done 302 302 0 604
32546 Skipped 0 0 0 0
32547 Error 0 0 0 0
32548 Last Flush: 9453 9453 9151
32549 Downstairs last five completed: 9151 9150 9149 9148 9147
32550 Upstairs last five completed: 9453 9452 9451 9450 9449
32551 CLIENT: Up:0 ds:302 act:2
32552 [8830] got start action
32553 [8830] I am going to restart
32554 Starting downstairs at port 8830
32555 Make output file at "/tmp/dsc/downstairs-8830.txt"
32556 [8830][2] reports Starting
32557 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1233)
32558 [8830][2] reports Running
32559 ----------------------------------------------------------------
32560 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32561 States: Active Active Offline
32562 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32563 9152 Acked Read 5 Done Done Sent false
32564 9153 Acked Write 9 Done Done Sent false
32565 9154 Acked Read 9 Done Done Sent false
32566 9155 Acked Flush 0 Done Done Sent false
32567 9156 Acked Write 7 Done Done Sent false
32568 9157 Acked Read 1 Done Done Sent false
32569 9158 Acked Write 2 Done Done Sent false
32570 9159 Acked Write 8 Done Done Sent false
32571 9160 Acked Read 2 Done Done Sent false
32572 9161 Acked Write 9 Done Done Sent false
32573 9162 Acked Read 9 Done Done Sent false
32574 9163 Acked Read 4 Done Done Sent false
32575 9164 Acked Read 6 Done Done Sent false
32576 9165 Acked Write 10 Done Done Sent false
32577 9166 Acked Read 5 Done Done Sent false
32578 9167 Acked Flush 0 Done Done Sent false
32579 9168 Acked Write 2 Done Done Sent false
32580 9169 Acked Read 3 Done Done Sent false
32581 9170 Acked Write 8 Done Done Sent false
32582 9171 Acked Flush 0 Done Done Sent false
32583 9172 Acked Flush 0 Done Done Sent false
32584 9173 Acked Read 5 Done Done Sent false
32585 9174 Acked Read 5 Done Done Sent false
32586 9175 Acked Write 2 Done Done Sent false
32587 9176 Acked Read 1 Done Done Sent false
32588 9177 Acked Read 5 Done Done Sent false
32589 9178 Acked Read 6 Done Done Sent false
32590 9179 Acked Read 1 Done Done Sent false
32591 9180 Acked Read 4 Done Done Sent false
32592 9181 Acked Read 8 Done Done Sent false
32593 9182 Acked Read 9 Done Done Sent false
32594 9183 Acked Write 8 Done Done Sent false
32595 9184 Acked Read 9 Done Done Sent false
32596 9185 Acked Write 9 Done Done Sent false
32597 9186 Acked Write 8 Done Done Sent false
32598 9187 Acked Write 9 Done Done Sent false
32599 9188 Acked Read 6 Done Done Sent false
32600 9189 Acked Write 5 Done Done Sent false
32601 9190 Acked Read 9 Done Done Sent false
32602 9191 Acked Write 8 Done Done Sent false
32603 9192 Acked Read 4 Done Done Sent false
32604 9193 Acked Flush 0 Done Done Sent false
32605 9194 Acked Write 2 Done Done Sent false
32606 9195 Acked Read 1 Done Done Sent false
32607 9196 Acked Write 9 Done Done Sent false
32608 9197 Acked Read 4 Done Done Sent false
32609 9198 Acked Read 2 Done Done Sent false
32610 9199 Acked Read 3 Done Done Sent false
32611 9200 Acked Write 6 Done Done Sent false
32612 9201 Acked Read 8 Done Done Sent false
32613 9202 Acked Write 7 Done Done Sent false
32614 9203 Acked Write 4 Done Done Sent false
32615 9204 Acked Read 3 Done Done Sent false
32616 9205 Acked Write 10 Done Done Sent false
32617 9206 Acked Read 9 Done Done Sent false
32618 9207 Acked Read 1 Done Done Sent false
32619 9208 Acked Read 10 Done Done Sent false
32620 9209 Acked Read 1 Done Done Sent false
32621 9210 Acked Read 9 Done Done Sent false
32622 9211 Acked Read 4 Done Done Sent false
32623 9212 Acked Read 2 Done Done Sent false
32624 9213 Acked Write 10 Done Done Sent false
32625 9214 Acked Write 4 Done Done Sent false
32626 9215 Acked Write 6 Done Done Sent false
32627 9216 Acked Write 7 Done Done Sent false
32628 9217 Acked Write 1 Done Done Sent false
32629 9218 Acked Read 5 Done Done Sent false
32630 9219 Acked Write 8 Done Done Sent false
32631 9220 Acked Read 6 Done Done Sent false
32632 9221 Acked Write 10 Done Done Sent false
32633 9222 Acked Flush 0 Done Done Sent false
32634 9223 Acked Write 4 Done Done Sent false
32635 9224 Acked Flush 0 Done Done Sent false
32636 9225 Acked Read 1 Done Done Sent false
32637 9226 Acked Flush 0 Done Done Sent false
32638 9227 Acked Write 10 Done Done Sent false
32639 9228 Acked Write 2 Done Done Sent false
32640 9229 Acked Write 3 Done Done Sent false
32641 9230 Acked Read 9 Done Done Sent false
32642 9231 Acked Read 2 Done Done Sent false
32643 9232 Acked Flush 0 Done Done Sent false
32644 9233 Acked Read 8 Done Done Sent false
32645 9234 Acked Write 2 Done Done Sent false
32646 9235 Acked Write 8 Done Done Sent false
32647 9236 Acked Write 2 Done Done Sent false
32648 9237 Acked Read 4 Done Done Sent false
32649 9238 Acked Flush 0 Done Done Sent false
32650 9239 Acked Read 8 Done Done Sent false
32651 9240 Acked Read 5 Done Done Sent false
32652 9241 Acked Write 6 Done Done Sent false
32653 9242 Acked Flush 0 Done Done Sent false
32654 9243 Acked Write 4 Done Done Sent false
32655 9244 Acked Write 4 Done Done Sent false
32656 9245 Acked Write 10 Done Done Sent false
32657 9246 Acked Write 9 Done Done Sent false
32658 9247 Acked Read 10 Done Done Sent false
32659 9248 Acked Read 8 Done Done Sent false
32660 9249 Acked Write 2 Done Done Sent false
32661 9250 Acked Flush 0 Done Done Sent false
32662 9251 Acked Write 2 Done Done Sent false
32663 9252 Acked Read 2 Done Done Sent false
32664 9253 Acked Read 4 Done Done Sent false
32665 9254 Acked Write 5 Done Done Sent false
32666 9255 Acked Write 8 Done Done Sent false
32667 9256 Acked Read 9 Done Done Sent false
32668 9257 Acked Flush 0 Done Done Sent false
32669 9258 Acked Read 9 Done Done Sent false
32670 9259 Acked Read 2 Done Done Sent false
32671 9260 Acked Read 9 Done Done Sent false
32672 9261 Acked Write 2 Done Done Sent false
32673 9262 Acked Write 4 Done Done Sent false
32674 9263 Acked Read 10 Done Done Sent false
32675 9264 Acked Read 2 Done Done Sent false
32676 9265 Acked Read 2 Done Done Sent false
32677 9266 Acked Write 9 Done Done Sent false
32678 9267 Acked Write 10 Done Done Sent false
32679 9268 Acked Flush 0 Done Done Sent false
32680 9269 Acked Write 10 Done Done Sent false
32681 9270 Acked Read 6 Done Done Sent false
32682 9271 Acked Read 6 Done Done Sent false
32683 9272 Acked Read 2 Done Done Sent false
32684 9273 Acked Read 5 Done Done Sent false
32685 9274 Acked Write 3 Done Done Sent false
32686 9275 Acked Write 2 Done Done Sent false
32687 9276 Acked Read 4 Done Done Sent false
32688 9277 Acked Flush 0 Done Done Sent false
32689 9278 Acked Read 5 Done Done Sent false
32690 9279 Acked Read 7 Done Done Sent false
32691 9280 Acked Read 9 Done Done Sent false
32692 9281 Acked Read 5 Done Done Sent false
32693 9282 Acked Write 4 Done Done Sent false
32694 9283 Acked Read 6 Done Done Sent false
32695 9284 Acked Read 8 Done Done Sent false
32696 9285 Acked Write 1 Done Done Sent false
32697 9286 Acked Read 5 Done Done Sent false
32698 9287 Acked Read 8 Done Done Sent false
32699 9288 Acked Flush 0 Done Done Sent false
32700 9289 Acked Read 9 Done Done Sent false
32701 9290 Acked Write 2 Done Done Sent false
32702 9291 Acked Flush 0 Done Done Sent false
32703 9292 Acked Read 7 Done Done Sent false
32704 9293 Acked Read 1 Done Done Sent false
32705 9294 Acked Read 5 Done Done Sent false
32706 9295 Acked Write 5 Done Done Sent false
32707 9296 Acked Read 10 Done Done Sent false
32708 9297 Acked Flush 0 Done Done Sent false
32709 9298 Acked Read 8 Done Done Sent false
32710 9299 Acked Write 3 Done Done Sent false
32711 9300 Acked Read 9 Done Done Sent false
32712 9301 Acked Write 1 Done Done Sent false
32713 9302 Acked Read 1 Done Done Sent false
32714 9303 Acked Write 3 Done Done Sent false
32715 9304 Acked Read 5 Done Done Sent false
32716 9305 Acked Write 7 Done Done Sent false
32717 9306 Acked Read 8 Done Done Sent false
32718 9307 Acked Read 3 Done Done Sent false
32719 9308 Acked Read 5 Done Done Sent false
32720 9309 Acked Write 9 Done Done Sent false
32721 9310 Acked Read 8 Done Done Sent false
32722 9311 Acked Read 1 Done Done Sent false
32723 9312 Acked Write 4 Done Done Sent false
32724 9313 Acked Read 9 Done Done Sent false
32725 9314 Acked Read 5 Done Done Sent false
32726 9315 Acked Flush 0 Done Done Sent false
32727 9316 Acked Read 8 Done Done Sent false
32728 9317 Acked Write 3 Done Done Sent false
32729 9318 Acked Write 5 Done Done Sent false
32730 9319 Acked Write 6 Done Done Sent false
32731 9320 Acked Read 7 Done Done Sent false
32732 9321 Acked Read 3 Done Done Sent false
32733 9322 Acked Write 1 Done Done Sent false
32734 9323 Acked Read 9 Done Done Sent false
32735 9324 Acked Write 1 Done Done Sent false
32736 9325 Acked Write 10 Done Done Sent false
32737 9326 Acked Read 8 Done Done Sent false
32738 9327 Acked Write 4 Done Done Sent false
32739 9328 Acked Flush 0 Done Done Sent false
32740 9329 Acked Flush 0 Done Done Sent false
32741 9330 Acked Read 10 Done Done Sent false
32742 9331 Acked Write 10 Done Done Sent false
32743 9332 Acked Flush 0 Done Done Sent false
32744 9333 Acked Flush 0 Done Done Sent false
32745 9334 Acked Write 9 Done Done Sent false
32746 9335 Acked Write 9 Done Done Sent false
32747 9336 Acked Write 8 Done Done Sent false
32748 9337 Acked Read 5 Done Done Sent false
32749 9338 Acked Write 2 Done Done Sent false
32750 9339 Acked Read 8 Done Done Sent false
32751 9340 Acked Read 9 Done Done Sent false
32752 9341 Acked Write 7 Done Done Sent false
32753 9342 Acked Write 1 Done Done Sent false
32754 9343 Acked Write 9 Done Done Sent false
32755 9344 Acked Write 1 Done Done Sent false
32756 9345 Acked Read 4 Done Done Sent false
32757 9346 Acked Write 4 Done Done Sent false
32758 9347 Acked Read 10 Done Done Sent false
32759 9348 Acked Write 8 Done Done Sent false
32760 9349 Acked Read 5 Done Done Sent false
32761 9350 Acked Write 1 Done Done Sent false
32762 9351 Acked Flush 0 Done Done Sent false
32763 9352 Acked Write 2 Done Done Sent false
32764 9353 Acked Write 7 Done Done Sent false
32765 9354 Acked Read 2 Done Done Sent false
32766 9355 Acked Write 1 Done Done Sent false
32767 9356 Acked Read 1 Done Done Sent false
32768 9357 Acked Read 1 Done Done Sent false
32769 9358 Acked Read 6 Done Done Sent false
32770 9359 Acked Write 8 Done Done Sent false
32771 9360 Acked Flush 0 Done Done Sent false
32772 9361 Acked Read 5 Done Done Sent false
32773 9362 Acked Write 6 Done Done Sent false
32774 9363 Acked Read 5 Done Done Sent false
32775 9364 Acked Read 6 Done Done Sent false
32776 9365 Acked Write 6 Done Done Sent false
32777 9366 Acked Read 6 Done Done Sent false
32778 9367 Acked Read 3 Done Done Sent false
32779 9368 Acked Flush 0 Done Done Sent false
32780 9369 Acked Write 8 Done Done Sent false
32781 9370 Acked Write 5 Done Done Sent false
32782 9371 Acked Read 6 Done Done Sent false
32783 9372 Acked Read 5 Done Done Sent false
32784 9373 Acked Write 5 Done Done Sent false
32785 9374 Acked Read 10 Done Done Sent false
32786 9375 Acked Write 7 Done Done Sent false
32787 9376 Acked Read 8 Done Done Sent false
32788 9377 Acked Write 6 Done Done Sent false
32789 9378 Acked Read 3 Done Done Sent false
32790 9379 Acked Write 4 Done Done Sent false
32791 9380 Acked Read 7 Done Done Sent false
32792 9381 Acked Write 8 Done Done Sent false
32793 9382 Acked Flush 0 Done Done Sent false
32794 9383 Acked Write 9 Done Done Sent false
32795 9384 Acked Write 5 Done Done Sent false
32796 9385 Acked Write 8 Done Done Sent false
32797 9386 Acked Write 3 Done Done Sent false
32798 9387 Acked Read 8 Done Done Sent false
32799 9388 Acked Write 3 Done Done Sent false
32800 9389 Acked Read 2 Done Done Sent false
32801 9390 Acked Read 10 Done Done Sent false
32802 9391 Acked Write 5 Done Done Sent false
32803 9392 Acked Write 7 Done Done Sent false
32804 9393 Acked Read 8 Done Done Sent false
32805 9394 Acked Read 2 Done Done Sent false
32806 9395 Acked Write 5 Done Done Sent false
32807 9396 Acked Read 1 Done Done Sent false
32808 9397 Acked Write 9 Done Done Sent false
32809 9398 Acked Read 3 Done Done Sent false
32810 9399 Acked Flush 0 Done Done Sent false
32811 9400 Acked Write 5 Done Done Sent false
32812 9401 Acked Flush 0 Done Done Sent false
32813 9402 Acked Write 5 Done Done Sent false
32814 9403 Acked Flush 0 Done Done Sent false
32815 9404 Acked Read 2 Done Done Sent false
32816 9405 Acked Read 4 Done Done Sent false
32817 9406 Acked Read 10 Done Done Sent false
32818 9407 Acked Read 6 Done Done Sent false
32819 9408 Acked Write 6 Done Done Sent false
32820 9409 Acked Read 2 Done Done Sent false
32821 9410 Acked Flush 0 Done Done Sent false
32822 9411 Acked Write 1 Done Done Sent false
32823 9412 Acked Flush 0 Done Done Sent false
32824 9413 Acked Flush 0 Done Done Sent false
32825 9414 Acked Read 3 Done Done Sent false
32826 9415 Acked Read 1 Done Done Sent false
32827 9416 Acked Flush 0 Done Done Sent false
32828 9417 Acked Write 6 Done Done Sent false
32829 9418 Acked Write 8 Done Done Sent false
32830 9419 Acked Read 8 Done Done Sent false
32831 9420 Acked Read 1 Done Done Sent false
32832 9421 Acked Write 7 Done Done Sent false
32833 9422 Acked Read 5 Done Done Sent false
32834 9423 Acked Write 4 Done Done Sent false
32835 9424 Acked Read 1 Done Done Sent false
32836 9425 Acked Write 6 Done Done Sent false
32837 9426 Acked Read 8 Done Done Sent false
32838 9427 Acked Flush 0 Done Done Sent false
32839 9428 Acked Write 8 Done Done Sent false
32840 9429 Acked Write 9 Done Done Sent false
32841 9430 Acked Read 6 Done Done Sent false
32842 9431 Acked Write 5 Done Done Sent false
32843 9432 Acked Write 5 Done Done Sent false
32844 9433 Acked Read 3 Done Done Sent false
32845 9434 Acked Read 1 Done Done Sent false
32846 9435 Acked Read 2 Done Done Sent false
32847 9436 Acked Read 8 Done Done Sent false
32848 9437 Acked Write 6 Done Done Sent false
32849 9438 Acked Read 1 Done Done Sent false
32850 9439 Acked Flush 0 Done Done Sent false
32851 9440 Acked Read 4 Done Done Sent false
32852 9441 Acked Write 5 Done Done Sent false
32853 9442 Acked Read 4 Done Done Sent false
32854 9443 Acked Write 6 Done Done Sent false
32855 9444 Acked Read 5 Done Done Sent false
32856 9445 Acked Write 8 Done Done Sent false
32857 9446 Acked Write 1 Done Done Sent false
32858 9447 Acked Flush 0 Done Done Sent false
32859 9448 Acked Write 1 Done Done Sent false
32860 9449 Acked Flush 0 Done Done Sent false
32861 9450 Acked Read 8 Done Done Sent false
32862 9451 Acked Write 7 Done Done Sent false
32863 9452 Acked Write 2 Done Done Sent false
32864 9453 Acked Flush 0 Done Done Sent false
32865 STATES DS:0 DS:1 DS:2 TOTAL
32866 Sent 0 0 302 302
32867 Done 302 302 0 604
32868 Skipped 0 0 0 0
32869 Error 0 0 0 0
32870 Last Flush: 9453 9453 9151
32871 Downstairs last five completed: 9151 9150 9149 9148 9147
32872 Upstairs last five completed: 9453 9452 9451 9450 9449
32873 CLIENT: Up:0 ds:302 act:2
328742025-01-03T08:37:48.391ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328752025-01-03T08:37:48.391ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55683) = io task client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328762025-01-03T08:37:48.392ZWARNcrucible: version negotiation from state Offline = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328772025-01-03T08:37:48.392ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328782025-01-03T08:37:48.392ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328792025-01-03T08:37:48.392ZINFOcrucible: send last flush ID to this DS: 9151 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328802025-01-03T08:37:48.393ZINFOcrucible: Replied this last flush ID: 9151 = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328812025-01-03T08:37:48.393ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328822025-01-03T08:37:48.393ZINFOcrucible: [2] Transition from Offline to Active = downstairs client = 2 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328832025-01-03T08:37:48.393ZINFOcrucible: [2] client re-new 302 jobs since flush 9151 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328842025-01-03T08:37:48.393ZINFOcrucible: [2] Replayed 302 jobs since flush: 9151 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
32885 ----------------------------------------------------------------
32886 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32887 Downstairs last five completed: 9453 9452 9451 9450 9449
32888 Upstairs last five completed: 9453 9452 9451 9450 9449
32889 CLIENT: Up:0 ds:0 act:3
32890 Replay: All jobs finished
32891 Main task has work to do, go find it
32892 got dsc Stop(1)
32893 stop 1
32894 Jan 03 08:37:48.632 INFO [8820] Got stop action so:false kr:false
32895 request completed, latency_us: 378, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 7fae252b-9a38-4fb7-8e0f-c5e874d28141, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
32896 Jan 03 08:37:48.633 INFO request completed, latency_us: 132, response_code: 200, uri: /state/cid/1, method: GET, req_id: 455d3a01-7c20-4d41-97e8-6898b987e955, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
328972025-01-03T08:37:48.633ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328982025-01-03T08:37:48.633ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
328992025-01-03T08:37:48.633ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
329002025-01-03T08:37:48.633ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
329012025-01-03T08:37:48.633ZINFOcrucible: [1] Transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
329022025-01-03T08:37:48.633ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
32903 [8820] Exited with: ExitStatus(unix_wait_status(9))
32904 [8820][1] reports Exit
32905 Jan 03 08:37:52.636 INFO request completed, latency_us: 215, response_code: 200, uri: /state/cid/1, method: GET, req_id: bf95dd96-5829-4456-b28a-40b3d01e3340, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
32906 001/300 Write block 417 len 40960 data: 7 6 8 7 7 8 7 8 7 7
32907 002/300 Flush
32908 003/300 Read block 4904 len 16384
32909 004/300 Write block 4705 len 12288 data: 9 9 10
32910 005/300 Write block 1036 len 28672 data: 5 4 5 3 3 3 3
32911 006/300 Read block 2519 len 8192
32912 007/300 Read block 94 len 4096
32913 008/300 Write block 4777 len 8192 data: 5 5
32914 009/300 Write block 4927 len 40960 data: 5 4 5 7 6 8 7 6 7 5
32915 010/300 Read block 3098 len 28672
32916 011/300 Read block 3256 len 32768
32917 012/300 Read block 747 len 28672
32918 013/300 Read block 1842 len 20480
32919 014/300 Write block 3171 len 36864 data: 5 4 6 5 5 6 7 8 8
32920 015/300 Read block 2953 len 8192
32921 016/300 Write block 2754 len 32768 data: 6 7 6 9 9 9 9 9
32922 017/300 Read block 3732 len 12288
32923 018/300 Write block 2819 len 40960 data: 4 5 5 5 7 7 8 10 9 7
32924 019/300 Flush
32925 020/300 Write block 389 len 4096 data: 4
32926 021/300 Read block 1235 len 12288
32927 022/300 Write block 2217 len 36864 data: 7 7 6 6 7 7 6 6 6
32928 023/300 Read block 3848 len 8192
32929 024/300 Write block 3493 len 32768 data: 6 6 6 5 5 5 4 4
32930 025/300 Read block 1578 len 12288
32931 026/300 Read block 1155 len 36864
32932 027/300 Read block 4157 len 16384
32933 028/300 Write block 1485 len 28672 data: 8 8 7 7 7 7 5
32934 029/300 Write block 533 len 4096 data: 4
32935 030/300 Write block 923 len 28672 data: 7 6 8 8 7 7 5
32936 031/300 Read block 858 len 24576
32937 032/300 Read block 2186 len 12288
32938 033/300 Write block 4934 len 36864 data: 7 8 6 4 6 6 5 6 7
32939 034/300 Read block 4009 len 16384
32940 035/300 Write block 584 len 28672 data: 6 6 4 7 5 5 5
32941 036/300 Read block 3308 len 36864
32942 037/300 Write block 1565 len 32768 data: 4 4 4 4 6 7 8 9
32943 038/300 Write block 173 len 4096 data: 8
32944 039/300 Write block 1914 len 12288 data: 7 5 5
32945 040/300 Write block 3953 len 16384 data: 6 6 5 6
32946 041/300 Write block 474 len 28672 data: 6 6 8 8 7 6 7
32947 042/300 Read block 4583 len 4096
32948 043/300 Read block 2371 len 36864
32949 044/300 Write block 4684 len 36864 data: 6 5 4 6 6 6 7 6 6
32950 045/300 Read block 1421 len 4096
32951 046/300 Read block 423 len 28672
32952 047/300 Flush
32953 048/300 Write block 2856 len 16384 data: 6 8 7 7
32954 049/300 Flush
32955 050/300 Read block 4821 len 40960
32956 051/300 Write block 656 len 28672 data: 6 8 9 8 7 6 7
32957 052/300 Read block 2051 len 16384
32958 053/300 Write block 4719 len 32768 data: 3 5 6 5 6 8 6 6
32959 054/300 Write block 579 len 32768 data: 5 6 5 4 6 7 7 5
32960 055/300 Write block 3077 len 8192 data: 4 4
32961 056/300 Write block 1299 len 8192 data: 2 3
32962 057/300 Read block 245 len 36864
32963 058/300 Write block 708 len 12288 data: 7 8 9
32964 059/300 Write block 4232 len 16384 data: 7 5 3 3
32965 060/300 Write block 3949 len 24576 data: 6 7 8 7 7 7
32966 061/300 Write block 4274 len 32768 data: 4 4 5 5 5 4 4 5
32967 062/300 Read block 2211 len 4096
32968 063/300 Read block 1909 len 24576
32969 064/300 Read block 2652 len 20480
32970 065/300 Write block 3781 len 40960 data: 3 4 4 4 4 6 5 6 5 5
32971 066/300 Read block 3473 len 4096
32972 067/300 Write block 4227 len 12288 data: 7 7 8
32973 068/300 Read block 1120 len 4096
32974 069/300 Flush
32975 070/300 Read block 184 len 24576
32976 071/300 Read block 4296 len 4096
32977 072/300 Flush
32978 073/300 Read block 2473 len 40960
32979 074/300 Read block 4194 len 12288
32980 075/300 Write block 1905 len 8192 data: 5 5
32981 076/300 Write block 3303 len 20480 data: 6 5 4 5 5
32982 077/300 Write block 1534 len 20480 data: 5 7 5 6 5
32983 078/300 Read block 323 len 32768
32984 079/300 Write block 2184 len 32768 data: 5 4 4 4 4 4 3 4
32985 080/300 Read block 1137 len 28672
32986 081/300 Write block 4198 len 28672 data: 9 11 11 11 10 7 8
32987 082/300 Read block 2924 len 20480
32988 083/300 Write block 4786 len 8192 data: 4 4
32989 084/300 Read block 1783 len 16384
32990 085/300 Read block 1178 len 24576
32991 086/300 Write block 1996 len 20480 data: 5 4 4 5 5
32992 087/300 Write block 816 len 16384 data: 4 7 6 7
32993 088/300 Flush
32994 089/300 Read block 3953 len 36864
32995 090/300 Write block 849 len 40960 data: 7 7 6 4 6 6 5 5 6 5
32996 091/300 Write block 1329 len 4096 data: 6
32997 092/300 Read block 4001 len 36864
32998 093/300 Read block 1045 len 32768
32999 094/300 Flush
33000 095/300 Read block 4193 len 40960
33001 096/300 Read block 2715 len 20480
33002 097/300 Read block 3452 len 24576
33003 098/300 Flush
33004 099/300 Read block 4194 len 28672
33005 100/300 Flush
33006 101/300 Read block 524 len 12288
33007 102/300 Write block 1391 len 8192 data: 8 6
33008 103/300 Read block 1767 len 20480
33009 104/300 Write block 4549 len 40960 data: 5 5 5 5 4 4 4 4 6 5
33010 105/300 Write block 2919 len 12288 data: 5 5 6
33011 106/300 Read block 2679 len 16384
33012 107/300 Write block 1572 len 12288 data: 10 9 7
33013 108/300 Read block 313 len 20480
33014 109/300 Read block 2928 len 12288
33015 110/300 Read block 3422 len 32768
33016 111/300 Write block 2301 len 32768 data: 7 8 8 9 8 9 11 12
33017 112/300 Read block 536 len 12288
33018 113/300 Read block 3270 len 36864
33019 114/300 Flush
33020 115/300 Read block 655 len 4096
33021 116/300 Flush
33022 117/300 Read block 316 len 8192
33023 118/300 Write block 1634 len 24576 data: 4 5 6 4 4 3
33024 119/300 Read block 2883 len 4096
33025 120/300 Read block 4255 len 12288
33026 121/300 Read block 2082 len 12288
33027 122/300 Read block 1939 len 40960
33028 123/300 Read block 4015 len 24576
33029 124/300 Read block 378 len 8192
33030 125/300 Write block 2826 len 36864 data: 11 10 8 7 6 6 5 6 7
33031 126/300 Read block 702 len 20480
33032 127/300 Write block 1132 len 32768 data: 6 8 8 10 9 8 7 5
33033 128/300 Read block 2173 len 24576
33034 129/300 Read block 3259 len 16384
33035 130/300 Write block 1529 len 4096 data: 6
33036 131/300 Write block 4671 len 40960 data: 7 7 7 7 6 4 4 4 3 5
33037 132/300 Write block 4602 len 28672 data: 5 7 7 7 6 5 5
33038 133/300 Read block 748 len 20480
33039 134/300 Write block 4006 len 8192 data: 10 9
33040 135/300 Read block 366 len 32768
33041 136/300 Flush
33042 137/300 Write block 1625 len 4096 data: 4
33043 138/300 Read block 1518 len 32768
33044 139/300 Write block 4366 len 20480 data: 9 8 7 6 7
33045 140/300 Write block 3971 len 36864 data: 4 4 6 5 5 7 7 7 7
33046 141/300 Write block 655 len 32768 data: 7 7 9 10 9 8 7 8
33047 142/300 Write block 669 len 28672 data: 7 4 4 4 4 4 4
33048 143/300 Read block 2451 len 12288
33049 144/300 Write block 359 len 8192 data: 3 3
33050 145/300 Read block 4987 len 36864
33051 146/300 Write block 928 len 28672 data: 8 6 4 5 6 5 5
33052 147/300 Read block 4241 len 36864
33053 148/300 Read block 2732 len 12288
33054 149/300 Write block 3406 len 32768 data: 6 6 4 2 2 4 5 5
33055 150/300 Write block 2129 len 24576 data: 6 7 6 9 7 7
33056 151/300 Read block 2943 len 36864
33057 152/300 Read block 959 len 40960
33058 153/300 Read block 4648 len 40960
33059 154/300 Write block 2479 len 24576 data: 5 6 8 9 11 10
33060 155/300 Write block 4513 len 8192 data: 5 6
33061 156/300 Read block 3674 len 28672
33062 157/300 Read block 1095 len 8192
33063 158/300 Read block 292 len 4096
33064 159/300 Write block 2393 len 28672 data: 8 8 6 6 4 3 3
33065 160/300 Read block 69 len 32768
33066 161/300 Read block 4647 len 16384
33067 162/300 Read block 2936 len 28672
33068 163/300 Write block 4928 len 40960 data: 5 6 8 7 9 8 8 9 7 5
33069 164/300 Write block 2948 len 24576 data: 6 5 3 4 4 4
33070 165/300 Write block 3051 len 32768 data: 7 5 8 9 6 6 5 6
33071 166/300 Read block 1968 len 4096
33072 167/300 Flush
33073 168/300 Read block 4418 len 28672
33074 169/300 Read block 4755 len 12288
33075 170/300 Read block 3920 len 16384
33076 171/300 Write block 3515 len 28672 data: 6 6 5 5 4 5 5
33077 172/300 Flush
33078 173/300 Read block 3754 len 24576
33079 174/300 Read block 702 len 8192
33080 175/300 Read block 905 len 20480
33081 176/300 Read block 3157 len 4096
33082 177/300 Read block 4393 len 8192
33083 178/300 Read block 902 len 24576
33084 179/300 Write block 2745 len 40960 data: 7 6 6 7 8 7 7 7 6 7
33085 180/300 Write block 1447 len 20480 data: 8 7 7 7 8
33086 181/300 Read block 4417 len 16384
33087 182/300 Read block 3497 len 8192
33088 183/300 Write block 4326 len 20480 data: 6 4 2 2 3
33089 184/300 Read block 2649 len 4096
33090 185/300 Flush
33091 186/300 Write block 2999 len 36864 data: 4 4 4 3 4 3 3 4 4
33092 187/300 Read block 552 len 8192
33093 188/300 Write block 3376 len 4096 data: 4
33094 189/300 Read block 1216 len 4096
33095 190/300 Read block 3104 len 24576
33096 191/300 Read block 4090 len 40960
33097 192/300 Read block 2429 len 40960
33098 193/300 Read block 3975 len 20480
33099 194/300 Write block 3954 len 16384 data: 8 6 7 6
33100 195/300 Read block 2819 len 4096
33101 196/300 Write block 3651 len 16384 data: 8 7 10 8
33102 197/300 Write block 1068 len 8192 data: 3 3
33103 198/300 Write block 3474 len 12288 data: 4 3 6
33104 199/300 Read block 4244 len 32768
33105 200/300 Write block 3714 len 40960 data: 4 6 6 7 8 8 6 6 5 5
33106 201/300 Write block 1228 len 8192 data: 5 6
33107 202/300 Write block 3419 len 12288 data: 6 5 6
33108 203/300 Write block 2034 len 4096 data: 5
33109 204/300 Read block 4470 len 20480
33110 205/300 Write block 3425 len 24576 data: 12 10 9 9 9 7
33111 206/300 Read block 986 len 40960
33112 207/300 Write block 3308 len 12288 data: 4 6 5
33113 208/300 Read block 1930 len 20480
33114 209/300 Read block 3330 len 40960
33115 210/300 Read block 3869 len 8192
33116 211/300 Read block 974 len 32768
33117 212/300 Read block 798 len 16384
33118 213/300 Read block 30 len 16384
33119 214/300 Write block 1928 len 32768 data: 4 5 5 5 4 3 4 4
33120 215/300 Flush
33121 216/300 Write block 2331 len 32768 data: 6 6 4 4 3 3 3 3
33122 217/300 Read block 4599 len 36864
33123 218/300 Write block 1153 len 32768 data: 6 9 8 7 7 8 10 10
33124 219/300 Write block 3855 len 40960 data: 9 7 7 6 5 7 9 9 10 8
33125 220/300 Write block 3590 len 20480 data: 4 3 3 3 2
33126 221/300 Write block 2343 len 20480 data: 2 3 4 4 4
33127 222/300 Read block 4705 len 16384
33128 223/300 Write block 4794 len 16384 data: 6 7 6 5
33129 224/300 Write block 981 len 24576 data: 4 3 3 5 4 5
33130 225/300 Write block 3869 len 20480 data: 12 10 11 10 8
33131 226/300 Read block 4910 len 28672
33132 227/300 Read block 129 len 8192
33133 228/300 Write block 3391 len 24576 data: 7 7 8 8 9 8
33134 229/300 Read block 4688 len 12288
33135 230/300 Write block 844 len 28672 data: 10 8 7 5 5 8 8
33136 231/300 Write block 31 len 28672 data: 4 4 4 4 4 4 5
33137 232/300 Write block 3368 len 16384 data: 9 7 7 5
33138 233/300 Read block 1585 len 12288
33139 234/300 Write block 260 len 12288 data: 4 4 4
33140 235/300 Write block 1336 len 4096 data: 3
33141 236/300 Read block 2522 len 28672
33142 237/300 Write block 2050 len 16384 data: 7 6 8 9
33143 238/300 Flush
33144 239/300 Write block 453 len 28672 data: 4 6 6 5 6 8 8
33145 240/300 Read block 4063 len 8192
33146 241/300 Read block 3143 len 28672
33147 242/300 Read block 3037 len 4096
33148 243/300 Write block 3892 len 28672 data: 7 6 7 9 11 10 7
33149 244/300 Write block 2425 len 40960 data: 8 6 3 3 4 3 3 3 3 2
33150 245/300 Read block 4640 len 4096
33151 246/300 Read block 3450 len 12288
33152 247/300 Read block 2094 len 24576
33153 248/300 Write block 503 len 4096 data: 5
33154 249/300 Read block 3346 len 36864
33155 250/300 Read block 3729 len 40960
33156 251/300 Write block 54 len 24576 data: 7 6 6 5 4 5
33157 252/300 Flush
33158 253/300 Read block 4531 len 16384
33159 254/300 Write block 964 len 32768 data: 5 5 5 5 5 5 4 4
33160 255/300 Write block 4624 len 28672 data: 7 7 6 6 5 5 6
33161 256/300 Flush
33162 257/300 Write block 4687 len 40960 data: 7 7 7 8 7 7 7 6 6 4
33163 258/300 Read block 2711 len 16384
33164 259/300 Write block 662 len 40960 data: 9 7 7 7 7 6 7 8 5 5
33165 260/300 Read block 2581 len 32768
33166 261/300 Write block 1012 len 40960 data: 5 7 7 7 7 7 6 5 5 5
33167 262/300 Write block 4679 len 36864 data: 4 6 6 5 5 7 6 5 8
33168 263/300 Read block 2056 len 4096
33169 264/300 Read block 1126 len 4096
33170 265/300 Read block 639 len 28672
33171 266/300 Read block 2533 len 28672
33172 267/300 Flush
33173 268/300 Write block 4176 len 40960 data: 8 10 9 8 8 6 5 6 6 4
33174 269/300 Write block 309 len 20480 data: 10 12 9 9 9
33175 270/300 Read block 4816 len 36864
33176 271/300 Write block 4130 len 16384 data: 6 6 6 6
33177 272/300 Read block 3614 len 4096
33178 273/300 Read block 1253 len 24576
33179 274/300 Write block 362 len 12288 data: 4 6 8
33180 275/300 Write block 3312 len 32768 data: 7 6 7 7 6 4 3 3
33181 276/300 Read block 1504 len 16384
33182 277/300 Write block 936 len 40960 data: 9 10 10 6 5 5 5 5 4 4
33183 278/300 Read block 102 len 12288
33184 279/300 Write block 1020 len 12288 data: 6 6 5
33185 280/300 Read block 4126 len 24576
33186 281/300 Flush
33187 282/300 Read block 965 len 24576
33188 283/300 Read block 624 len 28672
33189 284/300 Write block 4784 len 36864 data: 5 5 5 5 3 4 4 4 3
33190 285/300 Read block 1663 len 24576
33191 286/300 Write block 1219 len 12288 data: 5 7 8
33192 287/300 Read block 258 len 4096
33193 288/300 Read block 4733 len 32768
33194 289/300 Write block 2687 len 36864 data: 10 9 7 8 5 7 7 7 9
33195 290/300 Write block 4297 len 20480 data: 8 7 6 6 5
33196 291/300 Write block 4370 len 40960 data: 8 8 8 7 8 8 6 6 7 5
33197 292/300 Write block 1772 len 40960 data: 4 6 7 8 7 6 5 6 4 5
33198 293/300 Read block 1339 len 40960
33199 294/300 Write block 2108 len 16384 data: 4 5 5 10
33200 295/300 Write block 4968 len 4096 data: 7
33201 296/300 Flush
33202 297/300 Write block 593 len 32768 data: 5 5 5 6 3 4 4 4
33203 298/300 Write block 1798 len 32768 data: 7 5 6 6 7 7 6 6
33204 299/300 Read block 4793 len 8192
33205 300/300 Flush
33206 301/300 Write block 1930 len 16384 data: 6 6 5 4
33207 Main task has work to do, go find it
33208 got dsc Start(1)
33209 start 1
33210 Jan 03 08:37:52.873 INFO request completed, latency_us: 273, response_code: 204, uri: /start/cid/1, method: POST, req_id: 5e5e8118-b5da-43c7-a596-0f11aea42466, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
33211 Replay: started 1, returned:Ok(())
33212 ----------------------------------------------------------------
33213 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33214 States: Active Offline Active
33215 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33216 9454 Acked Write 10 Done Sent Done false
33217 9455 Acked Flush 0 Done Sent Done false
33218 9456 Acked Read 4 Done Sent Done false
33219 9457 Acked Write 3 Done Sent Done false
33220 9458 Acked Write 7 Done Sent Done false
33221 9459 Acked Read 2 Done Sent Done false
33222 9460 Acked Read 1 Done Sent Done false
33223 9461 Acked Write 2 Done Sent Done false
33224 9462 Acked Write 10 Done Sent Done false
33225 9463 Acked Read 7 Done Sent Done false
33226 9464 Acked Read 8 Done Sent Done false
33227 9465 Acked Read 7 Done Sent Done false
33228 9466 Acked Read 5 Done Sent Done false
33229 9467 Acked Write 9 Done Sent Done false
33230 9468 Acked Read 2 Done Sent Done false
33231 9469 Acked Write 8 Done Sent Done false
33232 9470 Acked Read 3 Done Sent Done false
33233 9471 Acked Write 10 Done Sent Done false
33234 9472 Acked Flush 0 Done Sent Done false
33235 9473 Acked Write 1 Done Sent Done false
33236 9474 Acked Read 3 Done Sent Done false
33237 9475 Acked Write 9 Done Sent Done false
33238 9476 Acked Read 2 Done Sent Done false
33239 9477 Acked Write 8 Done Sent Done false
33240 9478 Acked Read 3 Done Sent Done false
33241 9479 Acked Read 9 Done Sent Done false
33242 9480 Acked Read 4 Done Sent Done false
33243 9481 Acked Write 7 Done Sent Done false
33244 9482 Acked Write 1 Done Sent Done false
33245 9483 Acked Write 7 Done Sent Done false
33246 9484 Acked Read 6 Done Sent Done false
33247 9485 Acked Read 3 Done Sent Done false
33248 9486 Acked Write 9 Done Sent Done false
33249 9487 Acked Read 4 Done Sent Done false
33250 9488 Acked Write 7 Done Sent Done false
33251 9489 Acked Read 9 Done Sent Done false
33252 9490 Acked Write 8 Done Sent Done false
33253 9491 Acked Write 1 Done Sent Done false
33254 9492 Acked Write 3 Done Sent Done false
33255 9493 Acked Write 4 Done Sent Done false
33256 9494 Acked Write 7 Done Sent Done false
33257 9495 Acked Read 1 Done Sent Done false
33258 9496 Acked Read 9 Done Sent Done false
33259 9497 Acked Write 9 Done Sent Done false
33260 9498 Acked Read 1 Done Sent Done false
33261 9499 Acked Read 7 Done Sent Done false
33262 9500 Acked Flush 0 Done Sent Done false
33263 9501 Acked Write 4 Done Sent Done false
33264 9502 Acked Flush 0 Done Sent Done false
33265 9503 Acked Read 10 Done Sent Done false
33266 9504 Acked Write 7 Done Sent Done false
33267 9505 Acked Read 4 Done Sent Done false
33268 9506 Acked Write 8 Done Sent Done false
33269 9507 Acked Write 8 Done Sent Done false
33270 9508 Acked Write 2 Done Sent Done false
33271 9509 Acked Write 2 Done Sent Done false
33272 9510 Acked Read 9 Done Sent Done false
33273 9511 Acked Write 3 Done Sent Done false
33274 9512 Acked Write 4 Done Sent Done false
33275 9513 Acked Write 6 Done Sent Done false
33276 9514 Acked Write 8 Done Sent Done false
33277 9515 Acked Read 1 Done Sent Done false
33278 9516 Acked Read 6 Done Sent Done false
33279 9517 Acked Read 5 Done Sent Done false
33280 9518 Acked Write 10 Done Sent Done false
33281 9519 Acked Read 1 Done Sent Done false
33282 9520 Acked Write 3 Done Sent Done false
33283 9521 Acked Read 1 Done Sent Done false
33284 9522 Acked Flush 0 Done Sent Done false
33285 9523 Acked Read 6 Done Sent Done false
33286 9524 Acked Read 1 Done Sent Done false
33287 9525 Acked Flush 0 Done Sent Done false
33288 9526 Acked Read 10 Done Sent Done false
33289 9527 Acked Read 3 Done Sent Done false
33290 9528 Acked Write 2 Done Sent Done false
33291 9529 Acked Write 5 Done Sent Done false
33292 9530 Acked Write 5 Done Sent Done false
33293 9531 Acked Read 8 Done Sent Done false
33294 9532 Acked Write 8 Done Sent Done false
33295 9533 Acked Read 7 Done Sent Done false
33296 9534 Acked Write 7 Done Sent Done false
33297 9535 Acked Read 5 Done Sent Done false
33298 9536 Acked Write 2 Done Sent Done false
33299 9537 Acked Read 4 Done Sent Done false
33300 9538 Acked Read 6 Done Sent Done false
33301 9539 Acked Write 5 Done Sent Done false
33302 9540 Acked Write 4 Done Sent Done false
33303 9541 Acked Flush 0 Done Sent Done false
33304 9542 Acked Read 9 Done Sent Done false
33305 9543 Acked Write 10 Done Sent Done false
33306 9544 Acked Write 1 Done Sent Done false
33307 9545 Acked Read 9 Done Sent Done false
33308 9546 Acked Read 8 Done Sent Done false
33309 9547 Acked Flush 0 Done Sent Done false
33310 9548 Acked Read 10 Done Sent Done false
33311 9549 Acked Read 5 Done Sent Done false
33312 9550 Acked Read 6 Done Sent Done false
33313 9551 Acked Flush 0 Done Sent Done false
33314 9552 Acked Read 7 Done Sent Done false
33315 9553 Acked Flush 0 Done Sent Done false
33316 9554 Acked Read 3 Done Sent Done false
33317 9555 Acked Write 2 Done Sent Done false
33318 9556 Acked Read 5 Done Sent Done false
33319 9557 Acked Write 10 Done Sent Done false
33320 9558 Acked Write 3 Done Sent Done false
33321 9559 Acked Read 4 Done Sent Done false
33322 9560 Acked Write 3 Done Sent Done false
33323 9561 Acked Read 5 Done Sent Done false
33324 9562 Acked Read 3 Done Sent Done false
33325 9563 Acked Read 8 Done Sent Done false
33326 9564 Acked Write 8 Done Sent Done false
33327 9565 Acked Read 3 Done Sent Done false
33328 9566 Acked Read 9 Done Sent Done false
33329 9567 Acked Flush 0 Done Sent Done false
33330 9568 Acked Read 1 Done Sent Done false
33331 9569 Acked Flush 0 Done Sent Done false
33332 9570 Acked Read 2 Done Sent Done false
33333 9571 Acked Write 6 Done Sent Done false
33334 9572 Acked Read 1 Done Sent Done false
33335 9573 Acked Read 3 Done Sent Done false
33336 9574 Acked Read 3 Done Sent Done false
33337 9575 Acked Read 10 Done Sent Done false
33338 9576 Acked Read 6 Done Sent Done false
33339 9577 Acked Read 2 Done Sent Done false
33340 9578 Acked Write 9 Done Sent Done false
33341 9579 Acked Read 5 Done Sent Done false
33342 9580 Acked Write 8 Done Sent Done false
33343 9581 Acked Read 6 Done Sent Done false
33344 9582 Acked Read 4 Done Sent Done false
33345 9583 Acked Write 1 Done Sent Done false
33346 9584 Acked Write 10 Done Sent Done false
33347 9585 Acked Write 7 Done Sent Done false
33348 9586 Acked Read 5 Done Sent Done false
33349 9587 Acked Write 2 Done Sent Done false
33350 9588 Acked Read 8 Done Sent Done false
33351 9589 Acked Flush 0 Done Sent Done false
33352 9590 Acked Write 1 Done Sent Done false
33353 9591 Acked Read 8 Done Sent Done false
33354 9592 Acked Write 5 Done Sent Done false
33355 9593 Acked Write 9 Done Sent Done false
33356 9594 Acked Write 8 Done Sent Done false
33357 9595 Acked Write 7 Done Sent Done false
33358 9596 Acked Read 3 Done Sent Done false
33359 9597 Acked Write 2 Done Sent Done false
33360 9598 Acked Read 9 Done Sent Done false
33361 9599 Acked Write 7 Done Sent Done false
33362 9600 Acked Read 9 Done Sent Done false
33363 9601 Acked Read 3 Done Sent Done false
33364 9602 Acked Write 8 Done Sent Done false
33365 9603 Acked Write 6 Done Sent Done false
33366 9604 Acked Read 9 Done Sent Done false
33367 9605 Acked Read 10 Done Sent Done false
33368 9606 Acked Read 10 Done Sent Done false
33369 9607 Acked Write 6 Done Sent Done false
33370 9608 Acked Write 2 Done Sent Done false
33371 9609 Acked Read 7 Done Sent Done false
33372 9610 Acked Read 2 Done Sent Done false
33373 9611 Acked Read 1 Done Sent Done false
33374 9612 Acked Write 7 Done Sent Done false
33375 9613 Acked Read 8 Done Sent Done false
33376 9614 Acked Read 4 Done Sent Done false
33377 9615 Acked Read 7 Done Sent Done false
33378 9616 Acked Write 10 Done Sent Done false
33379 9617 Acked Write 6 Done Sent Done false
33380 9618 Acked Write 8 Done Sent Done false
33381 9619 Acked Read 1 Done Sent Done false
33382 9620 Acked Flush 0 Done Sent Done false
33383 9621 Acked Read 7 Done Sent Done false
33384 9622 Acked Read 3 Done Sent Done false
33385 9623 Acked Read 4 Done Sent Done false
33386 9624 Acked Write 7 Done Sent Done false
33387 9625 Acked Flush 0 Done Sent Done false
33388 9626 Acked Read 6 Done Sent Done false
33389 9627 Acked Read 2 Done Sent Done false
33390 9628 Acked Read 5 Done Sent Done false
33391 9629 Acked Read 1 Done Sent Done false
33392 9630 Acked Read 2 Done Sent Done false
33393 9631 Acked Read 6 Done Sent Done false
33394 9632 Acked Write 10 Done Sent Done false
33395 9633 Acked Write 5 Done Sent Done false
33396 9634 Acked Read 4 Done Sent Done false
33397 9635 Acked Read 2 Done Sent Done false
33398 9636 Acked Write 5 Done Sent Done false
33399 9637 Acked Read 1 Done Sent Done false
33400 9638 Acked Flush 0 Done Sent Done false
33401 9639 Acked Write 9 Done Sent Done false
33402 9640 Acked Read 2 Done Sent Done false
33403 9641 Acked Write 1 Done Sent Done false
33404 9642 Acked Read 1 Done Sent Done false
33405 9643 Acked Read 6 Done Sent Done false
33406 9644 Acked Read 10 Done Sent Done false
33407 9645 Acked Read 10 Done Sent Done false
33408 9646 Acked Read 5 Done Sent Done false
33409 9647 Acked Write 4 Done Sent Done false
33410 9648 Acked Read 1 Done Sent Done false
33411 9649 Acked Write 4 Done Sent Done false
33412 9650 Acked Write 2 Done Sent Done false
33413 9651 Acked Write 3 Done Sent Done false
33414 9652 Acked Read 8 Done Sent Done false
33415 9653 Acked Write 10 Done Sent Done false
33416 9654 Acked Write 2 Done Sent Done false
33417 9655 Acked Write 3 Done Sent Done false
33418 9656 Acked Write 1 Done Sent Done false
33419 9657 Acked Read 5 Done Sent Done false
33420 9658 Acked Write 6 Done Sent Done false
33421 9659 Acked Read 10 Done Sent Done false
33422 9660 Acked Write 3 Done Sent Done false
33423 9661 Acked Read 5 Done Sent Done false
33424 9662 Acked Read 10 Done Sent Done false
33425 9663 Acked Read 2 Done Sent Done false
33426 9664 Acked Read 8 Done Sent Done false
33427 9665 Acked Read 4 Done Sent Done false
33428 9666 Acked Read 4 Done Sent Done false
33429 9667 Acked Write 8 Done Sent Done false
33430 9668 Acked Flush 0 Done Sent Done false
33431 9669 Acked Write 8 Done Sent Done false
33432 9670 Acked Read 9 Done Sent Done false
33433 9671 Acked Write 8 Done Sent Done false
33434 9672 Acked Write 10 Done Sent Done false
33435 9673 Acked Write 5 Done Sent Done false
33436 9674 Acked Write 5 Done Sent Done false
33437 9675 Acked Read 4 Done Sent Done false
33438 9676 Acked Write 4 Done Sent Done false
33439 9677 Acked Write 6 Done Sent Done false
33440 9678 Acked Write 5 Done Sent Done false
33441 9679 Acked Read 7 Done Sent Done false
33442 9680 Acked Read 2 Done Sent Done false
33443 9681 Acked Write 6 Done Sent Done false
33444 9682 Acked Read 3 Done Sent Done false
33445 9683 Acked Write 7 Done Sent Done false
33446 9684 Acked Write 7 Done Sent Done false
33447 9685 Acked Write 4 Done Sent Done false
33448 9686 Acked Read 3 Done Sent Done false
33449 9687 Acked Write 3 Done Sent Done false
33450 9688 Acked Write 1 Done Sent Done false
33451 9689 Acked Read 7 Done Sent Done false
33452 9690 Acked Write 4 Done Sent Done false
33453 9691 Acked Flush 0 Done Sent Done false
33454 9692 Acked Write 7 Done Sent Done false
33455 9693 Acked Read 2 Done Sent Done false
33456 9694 Acked Read 7 Done Sent Done false
33457 9695 Acked Read 1 Done Sent Done false
33458 9696 Acked Write 7 Done Sent Done false
33459 9697 Acked Write 10 Done Sent Done false
33460 9698 Acked Read 1 Done Sent Done false
33461 9699 Acked Read 3 Done Sent Done false
33462 9700 Acked Read 6 Done Sent Done false
33463 9701 Acked Write 1 Done Sent Done false
33464 9702 Acked Read 9 Done Sent Done false
33465 9703 Acked Read 10 Done Sent Done false
33466 9704 Acked Write 6 Done Sent Done false
33467 9705 Acked Flush 0 Done Sent Done false
33468 9706 Acked Read 4 Done Sent Done false
33469 9707 Acked Write 8 Done Sent Done false
33470 9708 Acked Write 7 Done Sent Done false
33471 9709 Acked Flush 0 Done Sent Done false
33472 9710 Acked Write 10 Done Sent Done false
33473 9711 Acked Read 4 Done Sent Done false
33474 9712 Acked Write 10 Done Sent Done false
33475 9713 Acked Read 8 Done Sent Done false
33476 9714 Acked Write 10 Done Sent Done false
33477 9715 Acked Write 9 Done Sent Done false
33478 9716 Acked Read 1 Done Sent Done false
33479 9717 Acked Read 1 Done Sent Done false
33480 9718 Acked Read 7 Done Sent Done false
33481 9719 Acked Read 7 Done Sent Done false
33482 9720 Acked Flush 0 Done Sent Done false
33483 9721 Acked Write 10 Done Sent Done false
33484 9722 Acked Write 5 Done Sent Done false
33485 9723 Acked Read 9 Done Sent Done false
33486 9724 Acked Write 4 Done Sent Done false
33487 9725 Acked Read 1 Done Sent Done false
33488 9726 Acked Read 6 Done Sent Done false
33489 9727 Acked Write 3 Done Sent Done false
33490 9728 Acked Write 8 Done Sent Done false
33491 9729 Acked Read 4 Done Sent Done false
33492 9730 Acked Write 10 Done Sent Done false
33493 9731 Acked Read 3 Done Sent Done false
33494 9732 Acked Write 3 Done Sent Done false
33495 9733 Acked Read 6 Done Sent Done false
33496 9734 Acked Flush 0 Done Sent Done false
33497 9735 Acked Read 6 Done Sent Done false
33498 9736 Acked Read 7 Done Sent Done false
33499 9737 Acked Write 9 Done Sent Done false
33500 9738 Acked Read 6 Done Sent Done false
33501 9739 Acked Write 3 Done Sent Done false
33502 9740 Acked Read 1 Done Sent Done false
33503 9741 Acked Read 8 Done Sent Done false
33504 9742 Acked Write 9 Done Sent Done false
33505 9743 Acked Write 5 Done Sent Done false
33506 9744 Acked Write 10 Done Sent Done false
33507 9745 Acked Write 10 Done Sent Done false
33508 9746 Acked Read 10 Done Sent Done false
33509 9747 Acked Write 4 Done Sent Done false
33510 9748 Acked Write 1 Done Sent Done false
33511 9749 Acked Flush 0 Done Sent Done false
33512 9750 Acked Write 8 Done Sent Done false
33513 9751 Acked Write 8 Done Sent Done false
33514 9752 Acked Read 2 Done Sent Done false
33515 9753 Acked Flush 0 Done Sent Done false
33516 9754 Acked Write 4 Done Sent Done false
33517 STATES DS:0 DS:1 DS:2 TOTAL
33518 Sent 0 301 0 301
33519 Done 301 0 301 602
33520 Skipped 0 0 0 0
33521 Error 0 0 0 0
33522 Last Flush: 9753 9453 9753
33523 Downstairs last five completed: 9453 9452 9451 9450 9449
33524 Upstairs last five completed: 9754 9753 9752 9751 9750
33525 CLIENT: Up:0 ds:301 act:2
33526 [8820] got start action
33527 [8820] I am going to restart
33528 Starting downstairs at port 8820
33529 Make output file at "/tmp/dsc/downstairs-8820.txt"
33530 [8820][1] reports Starting
33531 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1234)
33532 [8820][1] reports Running
33533 ----------------------------------------------------------------
33534 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33535 States: Active Offline Active
33536 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33537 9454 Acked Write 10 Done Sent Done false
33538 9455 Acked Flush 0 Done Sent Done false
33539 9456 Acked Read 4 Done Sent Done false
33540 9457 Acked Write 3 Done Sent Done false
33541 9458 Acked Write 7 Done Sent Done false
33542 9459 Acked Read 2 Done Sent Done false
33543 9460 Acked Read 1 Done Sent Done false
33544 9461 Acked Write 2 Done Sent Done false
33545 9462 Acked Write 10 Done Sent Done false
33546 9463 Acked Read 7 Done Sent Done false
33547 9464 Acked Read 8 Done Sent Done false
33548 9465 Acked Read 7 Done Sent Done false
33549 9466 Acked Read 5 Done Sent Done false
33550 9467 Acked Write 9 Done Sent Done false
33551 9468 Acked Read 2 Done Sent Done false
33552 9469 Acked Write 8 Done Sent Done false
33553 9470 Acked Read 3 Done Sent Done false
33554 9471 Acked Write 10 Done Sent Done false
33555 9472 Acked Flush 0 Done Sent Done false
33556 9473 Acked Write 1 Done Sent Done false
33557 9474 Acked Read 3 Done Sent Done false
33558 9475 Acked Write 9 Done Sent Done false
33559 9476 Acked Read 2 Done Sent Done false
33560 9477 Acked Write 8 Done Sent Done false
33561 9478 Acked Read 3 Done Sent Done false
33562 9479 Acked Read 9 Done Sent Done false
33563 9480 Acked Read 4 Done Sent Done false
33564 9481 Acked Write 7 Done Sent Done false
33565 9482 Acked Write 1 Done Sent Done false
33566 9483 Acked Write 7 Done Sent Done false
33567 9484 Acked Read 6 Done Sent Done false
33568 9485 Acked Read 3 Done Sent Done false
33569 9486 Acked Write 9 Done Sent Done false
33570 9487 Acked Read 4 Done Sent Done false
33571 9488 Acked Write 7 Done Sent Done false
33572 9489 Acked Read 9 Done Sent Done false
33573 9490 Acked Write 8 Done Sent Done false
33574 9491 Acked Write 1 Done Sent Done false
33575 9492 Acked Write 3 Done Sent Done false
33576 9493 Acked Write 4 Done Sent Done false
33577 9494 Acked Write 7 Done Sent Done false
33578 9495 Acked Read 1 Done Sent Done false
33579 9496 Acked Read 9 Done Sent Done false
33580 9497 Acked Write 9 Done Sent Done false
33581 9498 Acked Read 1 Done Sent Done false
33582 9499 Acked Read 7 Done Sent Done false
33583 9500 Acked Flush 0 Done Sent Done false
33584 9501 Acked Write 4 Done Sent Done false
33585 9502 Acked Flush 0 Done Sent Done false
33586 9503 Acked Read 10 Done Sent Done false
33587 9504 Acked Write 7 Done Sent Done false
33588 9505 Acked Read 4 Done Sent Done false
33589 9506 Acked Write 8 Done Sent Done false
33590 9507 Acked Write 8 Done Sent Done false
33591 9508 Acked Write 2 Done Sent Done false
33592 9509 Acked Write 2 Done Sent Done false
33593 9510 Acked Read 9 Done Sent Done false
33594 9511 Acked Write 3 Done Sent Done false
33595 9512 Acked Write 4 Done Sent Done false
33596 9513 Acked Write 6 Done Sent Done false
33597 9514 Acked Write 8 Done Sent Done false
33598 9515 Acked Read 1 Done Sent Done false
33599 9516 Acked Read 6 Done Sent Done false
33600 9517 Acked Read 5 Done Sent Done false
33601 9518 Acked Write 10 Done Sent Done false
33602 9519 Acked Read 1 Done Sent Done false
33603 9520 Acked Write 3 Done Sent Done false
33604 9521 Acked Read 1 Done Sent Done false
33605 9522 Acked Flush 0 Done Sent Done false
33606 9523 Acked Read 6 Done Sent Done false
33607 9524 Acked Read 1 Done Sent Done false
33608 9525 Acked Flush 0 Done Sent Done false
33609 9526 Acked Read 10 Done Sent Done false
33610 9527 Acked Read 3 Done Sent Done false
33611 9528 Acked Write 2 Done Sent Done false
33612 9529 Acked Write 5 Done Sent Done false
33613 9530 Acked Write 5 Done Sent Done false
33614 9531 Acked Read 8 Done Sent Done false
33615 9532 Acked Write 8 Done Sent Done false
33616 9533 Acked Read 7 Done Sent Done false
33617 9534 Acked Write 7 Done Sent Done false
33618 9535 Acked Read 5 Done Sent Done false
33619 9536 Acked Write 2 Done Sent Done false
33620 9537 Acked Read 4 Done Sent Done false
33621 9538 Acked Read 6 Done Sent Done false
33622 9539 Acked Write 5 Done Sent Done false
33623 9540 Acked Write 4 Done Sent Done false
33624 9541 Acked Flush 0 Done Sent Done false
33625 9542 Acked Read 9 Done Sent Done false
33626 9543 Acked Write 10 Done Sent Done false
33627 9544 Acked Write 1 Done Sent Done false
33628 9545 Acked Read 9 Done Sent Done false
33629 9546 Acked Read 8 Done Sent Done false
33630 9547 Acked Flush 0 Done Sent Done false
33631 9548 Acked Read 10 Done Sent Done false
33632 9549 Acked Read 5 Done Sent Done false
33633 9550 Acked Read 6 Done Sent Done false
33634 9551 Acked Flush 0 Done Sent Done false
33635 9552 Acked Read 7 Done Sent Done false
33636 9553 Acked Flush 0 Done Sent Done false
33637 9554 Acked Read 3 Done Sent Done false
33638 9555 Acked Write 2 Done Sent Done false
33639 9556 Acked Read 5 Done Sent Done false
33640 9557 Acked Write 10 Done Sent Done false
33641 9558 Acked Write 3 Done Sent Done false
33642 9559 Acked Read 4 Done Sent Done false
33643 9560 Acked Write 3 Done Sent Done false
33644 9561 Acked Read 5 Done Sent Done false
33645 9562 Acked Read 3 Done Sent Done false
33646 9563 Acked Read 8 Done Sent Done false
33647 9564 Acked Write 8 Done Sent Done false
33648 9565 Acked Read 3 Done Sent Done false
33649 9566 Acked Read 9 Done Sent Done false
33650 9567 Acked Flush 0 Done Sent Done false
33651 9568 Acked Read 1 Done Sent Done false
33652 9569 Acked Flush 0 Done Sent Done false
33653 9570 Acked Read 2 Done Sent Done false
33654 9571 Acked Write 6 Done Sent Done false
33655 9572 Acked Read 1 Done Sent Done false
33656 9573 Acked Read 3 Done Sent Done false
33657 9574 Acked Read 3 Done Sent Done false
33658 9575 Acked Read 10 Done Sent Done false
33659 9576 Acked Read 6 Done Sent Done false
33660 9577 Acked Read 2 Done Sent Done false
33661 9578 Acked Write 9 Done Sent Done false
33662 9579 Acked Read 5 Done Sent Done false
33663 9580 Acked Write 8 Done Sent Done false
33664 9581 Acked Read 6 Done Sent Done false
33665 9582 Acked Read 4 Done Sent Done false
33666 9583 Acked Write 1 Done Sent Done false
33667 9584 Acked Write 10 Done Sent Done false
33668 9585 Acked Write 7 Done Sent Done false
33669 9586 Acked Read 5 Done Sent Done false
33670 9587 Acked Write 2 Done Sent Done false
33671 9588 Acked Read 8 Done Sent Done false
33672 9589 Acked Flush 0 Done Sent Done false
33673 9590 Acked Write 1 Done Sent Done false
33674 9591 Acked Read 8 Done Sent Done false
33675 9592 Acked Write 5 Done Sent Done false
33676 9593 Acked Write 9 Done Sent Done false
33677 9594 Acked Write 8 Done Sent Done false
33678 9595 Acked Write 7 Done Sent Done false
33679 9596 Acked Read 3 Done Sent Done false
33680 9597 Acked Write 2 Done Sent Done false
33681 9598 Acked Read 9 Done Sent Done false
33682 9599 Acked Write 7 Done Sent Done false
33683 9600 Acked Read 9 Done Sent Done false
33684 9601 Acked Read 3 Done Sent Done false
33685 9602 Acked Write 8 Done Sent Done false
33686 9603 Acked Write 6 Done Sent Done false
33687 9604 Acked Read 9 Done Sent Done false
33688 9605 Acked Read 10 Done Sent Done false
33689 9606 Acked Read 10 Done Sent Done false
33690 9607 Acked Write 6 Done Sent Done false
33691 9608 Acked Write 2 Done Sent Done false
33692 9609 Acked Read 7 Done Sent Done false
33693 9610 Acked Read 2 Done Sent Done false
33694 9611 Acked Read 1 Done Sent Done false
33695 9612 Acked Write 7 Done Sent Done false
33696 9613 Acked Read 8 Done Sent Done false
33697 9614 Acked Read 4 Done Sent Done false
33698 9615 Acked Read 7 Done Sent Done false
33699 9616 Acked Write 10 Done Sent Done false
33700 9617 Acked Write 6 Done Sent Done false
33701 9618 Acked Write 8 Done Sent Done false
33702 9619 Acked Read 1 Done Sent Done false
33703 9620 Acked Flush 0 Done Sent Done false
33704 9621 Acked Read 7 Done Sent Done false
33705 9622 Acked Read 3 Done Sent Done false
33706 9623 Acked Read 4 Done Sent Done false
33707 9624 Acked Write 7 Done Sent Done false
33708 9625 Acked Flush 0 Done Sent Done false
33709 9626 Acked Read 6 Done Sent Done false
33710 9627 Acked Read 2 Done Sent Done false
33711 9628 Acked Read 5 Done Sent Done false
33712 9629 Acked Read 1 Done Sent Done false
33713 9630 Acked Read 2 Done Sent Done false
33714 9631 Acked Read 6 Done Sent Done false
33715 9632 Acked Write 10 Done Sent Done false
33716 9633 Acked Write 5 Done Sent Done false
33717 9634 Acked Read 4 Done Sent Done false
33718 9635 Acked Read 2 Done Sent Done false
33719 9636 Acked Write 5 Done Sent Done false
33720 9637 Acked Read 1 Done Sent Done false
33721 9638 Acked Flush 0 Done Sent Done false
33722 9639 Acked Write 9 Done Sent Done false
33723 9640 Acked Read 2 Done Sent Done false
33724 9641 Acked Write 1 Done Sent Done false
33725 9642 Acked Read 1 Done Sent Done false
33726 9643 Acked Read 6 Done Sent Done false
33727 9644 Acked Read 10 Done Sent Done false
33728 9645 Acked Read 10 Done Sent Done false
33729 9646 Acked Read 5 Done Sent Done false
33730 9647 Acked Write 4 Done Sent Done false
33731 9648 Acked Read 1 Done Sent Done false
33732 9649 Acked Write 4 Done Sent Done false
33733 9650 Acked Write 2 Done Sent Done false
33734 9651 Acked Write 3 Done Sent Done false
33735 9652 Acked Read 8 Done Sent Done false
33736 9653 Acked Write 10 Done Sent Done false
33737 9654 Acked Write 2 Done Sent Done false
33738 9655 Acked Write 3 Done Sent Done false
33739 9656 Acked Write 1 Done Sent Done false
33740 9657 Acked Read 5 Done Sent Done false
33741 9658 Acked Write 6 Done Sent Done false
33742 9659 Acked Read 10 Done Sent Done false
33743 9660 Acked Write 3 Done Sent Done false
33744 9661 Acked Read 5 Done Sent Done false
33745 9662 Acked Read 10 Done Sent Done false
33746 9663 Acked Read 2 Done Sent Done false
33747 9664 Acked Read 8 Done Sent Done false
33748 9665 Acked Read 4 Done Sent Done false
33749 9666 Acked Read 4 Done Sent Done false
33750 9667 Acked Write 8 Done Sent Done false
33751 9668 Acked Flush 0 Done Sent Done false
33752 9669 Acked Write 8 Done Sent Done false
33753 9670 Acked Read 9 Done Sent Done false
33754 9671 Acked Write 8 Done Sent Done false
33755 9672 Acked Write 10 Done Sent Done false
33756 9673 Acked Write 5 Done Sent Done false
33757 9674 Acked Write 5 Done Sent Done false
33758 9675 Acked Read 4 Done Sent Done false
33759 9676 Acked Write 4 Done Sent Done false
33760 9677 Acked Write 6 Done Sent Done false
33761 9678 Acked Write 5 Done Sent Done false
33762 9679 Acked Read 7 Done Sent Done false
33763 9680 Acked Read 2 Done Sent Done false
33764 9681 Acked Write 6 Done Sent Done false
33765 9682 Acked Read 3 Done Sent Done false
33766 9683 Acked Write 7 Done Sent Done false
33767 9684 Acked Write 7 Done Sent Done false
33768 9685 Acked Write 4 Done Sent Done false
33769 9686 Acked Read 3 Done Sent Done false
33770 9687 Acked Write 3 Done Sent Done false
33771 9688 Acked Write 1 Done Sent Done false
33772 9689 Acked Read 7 Done Sent Done false
33773 9690 Acked Write 4 Done Sent Done false
33774 9691 Acked Flush 0 Done Sent Done false
33775 9692 Acked Write 7 Done Sent Done false
33776 9693 Acked Read 2 Done Sent Done false
33777 9694 Acked Read 7 Done Sent Done false
33778 9695 Acked Read 1 Done Sent Done false
33779 9696 Acked Write 7 Done Sent Done false
33780 9697 Acked Write 10 Done Sent Done false
33781 9698 Acked Read 1 Done Sent Done false
33782 9699 Acked Read 3 Done Sent Done false
33783 9700 Acked Read 6 Done Sent Done false
33784 9701 Acked Write 1 Done Sent Done false
33785 9702 Acked Read 9 Done Sent Done false
33786 9703 Acked Read 10 Done Sent Done false
33787 9704 Acked Write 6 Done Sent Done false
33788 9705 Acked Flush 0 Done Sent Done false
33789 9706 Acked Read 4 Done Sent Done false
33790 9707 Acked Write 8 Done Sent Done false
33791 9708 Acked Write 7 Done Sent Done false
33792 9709 Acked Flush 0 Done Sent Done false
33793 9710 Acked Write 10 Done Sent Done false
33794 9711 Acked Read 4 Done Sent Done false
33795 9712 Acked Write 10 Done Sent Done false
33796 9713 Acked Read 8 Done Sent Done false
33797 9714 Acked Write 10 Done Sent Done false
33798 9715 Acked Write 9 Done Sent Done false
33799 9716 Acked Read 1 Done Sent Done false
33800 9717 Acked Read 1 Done Sent Done false
33801 9718 Acked Read 7 Done Sent Done false
33802 9719 Acked Read 7 Done Sent Done false
33803 9720 Acked Flush 0 Done Sent Done false
33804 9721 Acked Write 10 Done Sent Done false
33805 9722 Acked Write 5 Done Sent Done false
33806 9723 Acked Read 9 Done Sent Done false
33807 9724 Acked Write 4 Done Sent Done false
33808 9725 Acked Read 1 Done Sent Done false
33809 9726 Acked Read 6 Done Sent Done false
33810 9727 Acked Write 3 Done Sent Done false
33811 9728 Acked Write 8 Done Sent Done false
33812 9729 Acked Read 4 Done Sent Done false
33813 9730 Acked Write 10 Done Sent Done false
33814 9731 Acked Read 3 Done Sent Done false
33815 9732 Acked Write 3 Done Sent Done false
33816 9733 Acked Read 6 Done Sent Done false
33817 9734 Acked Flush 0 Done Sent Done false
33818 9735 Acked Read 6 Done Sent Done false
33819 9736 Acked Read 7 Done Sent Done false
33820 9737 Acked Write 9 Done Sent Done false
33821 9738 Acked Read 6 Done Sent Done false
33822 9739 Acked Write 3 Done Sent Done false
33823 9740 Acked Read 1 Done Sent Done false
33824 9741 Acked Read 8 Done Sent Done false
33825 9742 Acked Write 9 Done Sent Done false
33826 9743 Acked Write 5 Done Sent Done false
33827 9744 Acked Write 10 Done Sent Done false
33828 9745 Acked Write 10 Done Sent Done false
33829 9746 Acked Read 10 Done Sent Done false
33830 9747 Acked Write 4 Done Sent Done false
33831 9748 Acked Write 1 Done Sent Done false
33832 9749 Acked Flush 0 Done Sent Done false
33833 9750 Acked Write 8 Done Sent Done false
33834 9751 Acked Write 8 Done Sent Done false
33835 9752 Acked Read 2 Done Sent Done false
33836 9753 Acked Flush 0 Done Sent Done false
33837 9754 Acked Write 4 Done Sent Done false
33838 9755 Acked Flush 0 Done Sent Done false
33839 STATES DS:0 DS:1 DS:2 TOTAL
33840 Sent 0 302 0 302
33841 Done 302 0 302 604
33842 Skipped 0 0 0 0
33843 Error 0 0 0 0
33844 Last Flush: 9755 9453 9755
33845 Downstairs last five completed: 9453 9452 9451 9450 9449
33846 Upstairs last five completed: 9755 9754 9753 9752 9751
33847 CLIENT: Up:0 ds:302 act:2
338482025-01-03T08:37:58.634ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338492025-01-03T08:37:58.634ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51001) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338502025-01-03T08:37:58.635ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338512025-01-03T08:37:58.636ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338522025-01-03T08:37:58.636ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338532025-01-03T08:37:58.636ZINFOcrucible: send last flush ID to this DS: 9453 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338542025-01-03T08:37:58.636ZINFOcrucible: Replied this last flush ID: 9453 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338552025-01-03T08:37:58.636ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338562025-01-03T08:37:58.636ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338572025-01-03T08:37:58.636ZINFOcrucible: [1] client re-new 302 jobs since flush 9453 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
338582025-01-03T08:37:58.637ZINFOcrucible: [1] Replayed 302 jobs since flush: 9453 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
33859 ----------------------------------------------------------------
33860 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33861 Downstairs last five completed: 9755 9754 9753 9752 9751
33862 Upstairs last five completed: 9755 9754 9753 9752 9751
33863 CLIENT: Up:0 ds:0 act:3
33864 Replay: All jobs finished
33865 Main task has work to do, go find it
33866 got dsc Stop(1)
33867 stop 1
33868 [8820] Got stop action so:false kr:false
33869 Jan 03 08:38:00.879 INFO request completed, latency_us: 373, response_code: 204, uri: /stop/cid/1, method: POST, req_id: c5cb87a7-a25b-46fc-8e4e-d969aeff74e0, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
338702025-01-03T08:38:00.880ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
33871 Jan 03 08:38:00.880 INFO request completed, latency_us: 138, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:38:00.880126378Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
33872 response_code: 200, uri{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-03T08:38:00.880162447Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
33873 : /state/cid/1, method: {"msg":"ds_transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:38:00.880202236Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
33874 GET, req_id: 4ac8bc33-bbb0-44ff-aaa8-7bc445b24494, remote_addr: 127{"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:38:00.880226496Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
33875 .0.0.1:49669, local_addr: 127.0.0.1:9998
338762025-01-03T08:38:00.880ZINFOcrucible: client task is exiting = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
33877 [8820] Exited with: ExitStatus(unix_wait_status(9))
33878 [8820][1] reports Exit
33879 Jan 03 08:38:04.882 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/1, method: GET, req_id: 0d844db9-8dc3-4131-8efe-70e4c8e51f3b, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
33880 001/300 Write block 4213 len 4096 data: 5
33881 002/300 Read block 3964 len 20480
33882 003/300 Write block 4814 len 24576 data: 6 7 6 7 7 6
33883 004/300 Write block 3290 len 16384 data: 6 7 6 5
33884 005/300 Write block 1212 len 36864 data: 2 4 4 5 5 5 5 6 8
33885 006/300 Write block 2619 len 40960 data: 4 4 4 4 4 5 6 6 5 5
33886 007/300 Read block 823 len 20480
33887 008/300 Read block 1867 len 16384
33888 009/300 Flush
33889 010/300 Read block 1703 len 32768
33890 011/300 Write block 2962 len 32768 data: 6 6 6 6 6 5 5 7
33891 012/300 Read block 2064 len 28672
33892 013/300 Read block 2268 len 12288
33893 014/300 Read block 4946 len 24576
33894 015/300 Write block 4362 len 20480 data: 7 7 6 8 10
33895 016/300 Read block 4232 len 20480
33896 017/300 Read block 2777 len 8192
33897 018/300 Read block 4931 len 4096
33898 019/300 Read block 124 len 24576
33899 020/300 Write block 3341 len 32768 data: 3 3 4 4 4 4 4 4
33900 021/300 Write block 1957 len 36864 data: 5 4 6 5 6 7 7 5 4
33901 022/300 Read block 3798 len 8192
33902 023/300 Write block 3148 len 4096 data: 7
33903 024/300 Write block 28 len 32768 data: 2 3 4 5 5 5 5 5
33904 025/300 Read block 3011 len 28672
33905 026/300 Flush
33906 027/300 Read block 2222 len 40960
33907 028/300 Read block 1931 len 24576
33908 029/300 Write block 1615 len 32768 data: 6 5 5 6 7 6 6 6
33909 030/300 Read block 1657 len 8192
33910 031/300 Write block 2876 len 8192 data: 4 3
33911 032/300 Write block 4942 len 24576 data: 8 9 6 7 7 9
33912 033/300 Read block 4175 len 20480
33913 034/300 Read block 467 len 12288
33914 035/300 Flush
33915 036/300 Read block 2455 len 12288
33916 037/300 Read block 587 len 12288
33917 038/300 Read block 632 len 16384
33918 039/300 Write block 3749 len 40960 data: 6 6 5 4 4 4 3 3 3 5
33919 040/300 Write block 3063 len 40960 data: 4 4 6 5 4 5 4 4 5 5
33920 041/300 Write block 3784 len 8192 data: 5 5
33921 042/300 Read block 1565 len 36864
33922 043/300 Write block 1864 len 32768 data: 4 4 3 3 4 3 3 4
33923 044/300 Read block 91 len 40960
33924 045/300 Read block 1712 len 40960
33925 046/300 Write block 1673 len 24576 data: 6 6 8 7 7 6
33926 047/300 Read block 871 len 4096
33927 048/300 Read block 2346 len 40960
33928 049/300 Flush
33929 050/300 Read block 4212 len 24576
33930 051/300 Read block 2431 len 32768
33931 052/300 Read block 4396 len 16384
33932 053/300 Flush
33933 054/300 Flush
33934 055/300 Write block 2767 len 28672 data: 6 6 6 5 6 4 4
33935 056/300 Read block 2217 len 16384
33936 057/300 Read block 2815 len 40960
33937 058/300 Read block 3444 len 40960
33938 059/300 Write block 4806 len 16384 data: 6 7 3 4
33939 060/300 Read block 1171 len 36864
33940 061/300 Write block 4964 len 24576 data: 6 8 7 7 8 6
33941 062/300 Read block 4515 len 8192
33942 063/300 Flush
33943 064/300 Write block 3944 len 16384 data: 5 4 6 5
33944 065/300 Write block 856 len 36864 data: 6 7 6 5 5 5 3 2 3
33945 066/300 Read block 4491 len 40960
33946 067/300 Write block 654 len 8192 data: 7 8
33947 068/300 Read block 3593 len 40960
33948 069/300 Read block 1368 len 12288
33949 070/300 Write block 4746 len 16384 data: 6 8 10 10
33950 071/300 Read block 1001 len 20480
33951 072/300 Read block 67 len 36864
33952 073/300 Read block 4117 len 32768
33953 074/300 Flush
33954 075/300 Write block 3939 len 24576 data: 9 8 7 5 4 6
33955 076/300 Write block 2049 len 40960 data: 8 8 7 9 10 9 10 8 8 6
33956 077/300 Write block 1426 len 20480 data: 8 7 6 5 6
33957 078/300 Read block 4241 len 20480
33958 079/300 Write block 2866 len 8192 data: 3 3
33959 080/300 Write block 2188 len 32768 data: 5 5 4 5 3 3 4 4
33960 081/300 Read block 113 len 32768
33961 082/300 Write block 565 len 4096 data: 6
33962 083/300 Read block 4332 len 32768
33963 084/300 Read block 1832 len 20480
33964 085/300 Write block 772 len 28672 data: 5 5 7 6 6 6 6
33965 086/300 Read block 591 len 8192
33966 087/300 Write block 4004 len 28672 data: 11 11 11 10 10 7 6
33967 088/300 Flush
33968 089/300 Write block 430 len 4096 data: 6
33969 090/300 Write block 1034 len 16384 data: 5 5 6 5
33970 091/300 Read block 3934 len 4096
33971 092/300 Read block 2365 len 36864
33972 093/300 Read block 3884 len 32768
33973 094/300 Flush
33974 095/300 Write block 4098 len 36864 data: 5 5 4 3 3 4 4 7 7
33975 096/300 Read block 1668 len 16384
33976 097/300 Read block 4234 len 36864
33977 098/300 Read block 4288 len 36864
33978 099/300 Flush
33979 100/300 Read block 3505 len 12288
33980 101/300 Write block 3839 len 4096 data: 6
33981 102/300 Write block 261 len 16384 data: 5 5 3 3
33982 103/300 Write block 4539 len 20480 data: 5 4 5 6 6
33983 104/300 Flush
33984 105/300 Write block 3852 len 8192 data: 9 10
33985 106/300 Read block 961 len 28672
33986 107/300 Write block 2244 len 32768 data: 4 4 5 6 5 6 7 7
33987 108/300 Flush
33988 109/300 Write block 3616 len 40960 data: 5 4 5 5 6 4 4 4 5 5
33989 110/300 Write block 3253 len 32768 data: 5 5 8 8 6 8 7 6
33990 111/300 Write block 60 len 8192 data: 7 7
33991 112/300 Write block 4657 len 20480 data: 4 4 4 5 7
33992 113/300 Write block 1704 len 4096 data: 5
33993 114/300 Write block 2462 len 4096 data: 5
33994 115/300 Read block 2398 len 8192
33995 116/300 Read block 4440 len 12288
33996 117/300 Write block 244 len 20480 data: 5 6 8 7 7
33997 118/300 Flush
33998 119/300 Read block 2100 len 20480
33999 120/300 Write block 2134 len 24576 data: 8 6 8 8 5 4
34000 121/300 Write block 1591 len 32768 data: 7 6 8 8 5 6 6 7
34001 122/300 Read block 1336 len 12288
34002 123/300 Write block 2264 len 32768 data: 6 5 5 6 6 5 4 4
34003 124/300 Write block 449 len 16384 data: 5 4 4 4
34004 125/300 Write block 215 len 32768 data: 6 4 5 6 5 4 3 3
34005 126/300 Read block 4946 len 12288
34006 127/300 Read block 4675 len 16384
34007 128/300 Read block 2392 len 24576
34008 129/300 Read block 4489 len 12288
34009 130/300 Flush
34010 131/300 Read block 254 len 8192
34011 132/300 Write block 8 len 16384 data: 2 2 3 4
34012 133/300 Flush
34013 134/300 Write block 3270 len 40960 data: 7 6 4 5 4 3 5 6 5 4
34014 135/300 Write block 2115 len 36864 data: 9 9 6 5 5 6 6 4 6
34015 136/300 Read block 4439 len 16384
34016 137/300 Read block 91 len 16384
34017 138/300 Write block 3712 len 32768 data: 4 4 5 7 7 8 9 9
34018 139/300 Read block 534 len 36864
34019 140/300 Read block 1520 len 4096
34020 141/300 Read block 4089 len 12288
34021 142/300 Read block 3844 len 24576
34022 143/300 Read block 386 len 16384
34023 144/300 Write block 4311 len 20480 data: 7 8 9 8 7
34024 145/300 Read block 4553 len 28672
34025 146/300 Write block 2404 len 16384 data: 5 5 5 6
34026 147/300 Flush
34027 148/300 Read block 3827 len 4096
34028 149/300 Read block 52 len 32768
34029 150/300 Flush
34030 151/300 Write block 947 len 8192 data: 6 8
34031 152/300 Read block 2828 len 28672
34032 153/300 Read block 4800 len 24576
34033 154/300 Write block 1630 len 16384 data: 5 4 6 4
34034 155/300 Read block 2713 len 8192
34035 156/300 Write block 3383 len 4096 data: 5
34036 157/300 Flush
34037 158/300 Write block 4319 len 40960 data: 7 6 5 5 7 7 6 7 5 3
34038 159/300 Write block 1610 len 24576 data: 3 3 4 5 6 7
34039 160/300 Read block 1766 len 28672
34040 161/300 Read block 3868 len 16384
34041 162/300 Write block 3591 len 32768 data: 4 4 4 3 2 5 4 5
34042 163/300 Read block 1675 len 20480
34043 164/300 Read block 4731 len 4096
34044 165/300 Read block 429 len 8192
34045 166/300 Write block 525 len 16384 data: 6 6 6 5
34046 167/300 Write block 3702 len 40960 data: 7 7 8 8 6 6 5 5 5 4
34047 168/300 Flush
34048 169/300 Write block 1847 len 16384 data: 5 7 6 6
34049 170/300 Flush
34050 171/300 Write block 2684 len 32768 data: 8 6 6 11 10 8 9 6
34051 172/300 Read block 1822 len 12288
34052 173/300 Read block 2655 len 36864
34053 174/300 Write block 3450 len 8192 data: 5 6
34054 175/300 Read block 2630 len 12288
34055 176/300 Write block 2038 len 20480 data: 4 3 3 4 6
34056 177/300 Flush
34057 178/300 Read block 4738 len 12288
34058 179/300 Write block 912 len 8192 data: 8 7
34059 180/300 Write block 4230 len 16384 data: 7 6 8 6
34060 181/300 Read block 3692 len 8192
34061 182/300 Read block 3462 len 36864
34062 183/300 Write block 3759 len 32768 data: 6 6 8 8 8 8 8 10
34063 184/300 Read block 1883 len 16384
34064 185/300 Read block 1935 len 40960
34065 186/300 Read block 1218 len 8192
34066 187/300 Read block 517 len 40960
34067 188/300 Write block 1115 len 28672 data: 6 5 7 5 7 7 10
34068 189/300 Write block 1233 len 32768 data: 6 3 3 2 3 2 3 3
34069 190/300 Read block 3983 len 4096
34070 191/300 Write block 2183 len 36864 data: 4 6 5 5 5 6 6 5 6
34071 192/300 Flush
34072 193/300 Read block 1302 len 8192
34073 194/300 Write block 3247 len 16384 data: 5 5 5 5
34074 195/300 Read block 4347 len 8192
34075 196/300 Write block 176 len 36864 data: 7 5 4 3 4 4 5 7 6
34076 197/300 Read block 4003 len 40960
34077 198/300 Write block 253 len 36864 data: 6 7 8 5 6 4 4 5 6
34078 199/300 Write block 4211 len 8192 data: 6 5
34079 200/300 Write block 2152 len 36864 data: 4 7 8 8 8 8 8 5 6
34080 201/300 Flush
34081 202/300 Read block 2089 len 24576
34082 203/300 Read block 1534 len 4096
34083 204/300 Read block 684 len 28672
34084 205/300 Write block 1943 len 32768 data: 7 7 8 9 10 9 11 10
34085 206/300 Write block 3002 len 40960 data: 4 5 4 4 5 5 6 6 5 5
34086 207/300 Write block 675 len 36864 data: 5 4 3 3 4 4 5 6 6
34087 208/300 Read block 4218 len 32768
34088 209/300 Read block 4220 len 36864
34089 210/300 Flush
34090 211/300 Read block 326 len 28672
34091 212/300 Read block 2307 len 12288
34092 213/300 Write block 1100 len 12288 data: 8 9 7
34093 214/300 Write block 111 len 4096 data: 4
34094 215/300 Write block 253 len 28672 data: 7 8 9 6 7 5 5
34095 216/300 Read block 2279 len 4096
34096 217/300 Read block 2926 len 28672
34097 218/300 Read block 2253 len 28672
34098 219/300 Write block 2826 len 8192 data: 12 11
34099 220/300 Flush
34100 221/300 Write block 3255 len 36864 data: 9 9 7 9 8 7 7 7 6
34101 222/300 Write block 646 len 4096 data: 4
34102 223/300 Write block 1390 len 24576 data: 7 9 7 6 5 5
34103 224/300 Read block 3974 len 12288
34104 225/300 Read block 4272 len 32768
34105 226/300 Flush
34106 227/300 Write block 1322 len 16384 data: 5 5 4 4
34107 228/300 Write block 3084 len 12288 data: 8 9 6
34108 229/300 Write block 2128 len 8192 data: 7 7
34109 230/300 Read block 4505 len 24576
34110 231/300 Write block 4244 len 8192 data: 5 5
34111 232/300 Write block 236 len 24576 data: 6 5 4 3 3 4
34112 233/300 Read block 495 len 32768
34113 234/300 Read block 434 len 4096
34114 235/300 Read block 1173 len 36864
34115 236/300 Read block 57 len 12288
34116 237/300 Write block 3074 len 40960 data: 4 4 3 5 5 4 3 5 5 8
34117 238/300 Read block 2966 len 40960
34118 239/300 Read block 518 len 32768
34119 240/300 Read block 2546 len 24576
34120 241/300 Write block 4401 len 20480 data: 4 4 4 4 4
34121 242/300 Write block 4457 len 36864 data: 5 5 4 3 3 4 4 4 6
34122 243/300 Write block 3656 len 24576 data: 6 7 7 6 8 7
34123 244/300 Write block 2453 len 12288 data: 7 9 9
34124 245/300 Write block 4392 len 36864 data: 7 6 5 4 3 3 4 4 5
34125 246/300 Write block 3118 len 36864 data: 4 4 4 4 3 6 6 7 7
34126 247/300 Read block 193 len 20480
34127 248/300 Read block 3738 len 36864
34128 249/300 Write block 3417 len 32768 data: 4 4 7 6 7 12 12 13
34129 250/300 Read block 4460 len 16384
34130 251/300 Read block 4568 len 28672
34131 252/300 Write block 1381 len 8192 data: 3 3
34132 253/300 Flush
34133 254/300 Read block 635 len 16384
34134 255/300 Flush
34135 256/300 Read block 1020 len 24576
34136 257/300 Read block 86 len 4096
34137 258/300 Read block 1314 len 36864
34138 259/300 Read block 1128 len 24576
34139 260/300 Read block 3601 len 12288
34140 261/300 Read block 3276 len 24576
34141 262/300 Write block 4454 len 24576 data: 2 3 4 6 6 5
34142 263/300 Write block 279 len 20480 data: 4 4 5 6 6
34143 264/300 Write block 1082 len 28672 data: 3 3 5 4 6 5 5
34144 265/300 Write block 2489 len 12288 data: 7 7 6
34145 266/300 Read block 4671 len 8192
34146 267/300 Write block 204 len 28672 data: 3 3 5 3 3 4 4
34147 268/300 Read block 2001 len 20480
34148 269/300 Read block 1377 len 36864
34149 270/300 Write block 2357 len 32768 data: 4 3 4 5 6 6 5 5
34150 271/300 Write block 1770 len 16384 data: 9 5 5 7
34151 272/300 Write block 3481 len 40960 data: 3 3 4 5 5 5 6 6 6 6
34152 273/300 Write block 4396 len 16384 data: 4 4 5 5
34153 274/300 Write block 3599 len 28672 data: 6 9 9 8 8 9 11
34154 275/300 Write block 630 len 8192 data: 7 8
34155 276/300 Read block 1549 len 12288
34156 277/300 Read block 4739 len 12288
34157 278/300 Flush
34158 279/300 Read block 3100 len 16384
34159 280/300 Read block 4377 len 36864
34160 281/300 Write block 25 len 24576 data: 4 4 4 3 4 5
34161 282/300 Flush
34162 283/300 Read block 3052 len 12288
34163 284/300 Read block 428 len 40960
34164 285/300 Read block 1417 len 40960
34165 286/300 Read block 2298 len 20480
34166 287/300 Write block 1565 len 28672 data: 5 5 5 5 7 8 9
34167 288/300 Read block 4494 len 28672
34168 289/300 Flush
34169 290/300 Write block 1191 len 24576 data: 8 7 7 7 7 6
34170 291/300 Flush
34171 292/300 Read block 3204 len 24576
34172 293/300 Write block 3696 len 8192 data: 7 6
34173 294/300 Flush
34174 295/300 Read block 2962 len 24576
34175 296/300 Read block 2034 len 12288
34176 297/300 Read block 1174 len 16384
34177 298/300 Read block 3537 len 16384
34178 299/300 Write block 4921 len 12288 data: 11 8 7
34179 300/300 Read block 2162 len 8192
34180 301/300 Write block 1690 len 4096 data: 6
34181 Main task has work to do, go find it
34182 got dsc Start(1)
34183 start 1
34184 Jan 03 08:38:05.119 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/1, method: POST, req_id: 0cebdf84-bd69-4152-a2b4-bdbbd0a2d7d1, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
34185 Replay: started 1, returned:Ok(())
34186 ----------------------------------------------------------------
34187 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34188 States: Active Offline Active
34189 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34190 9756 Acked Write 1 Done Sent Done false
34191 9757 Acked Read 5 Done Sent Done false
34192 9758 Acked Write 6 Done Sent Done false
34193 9759 Acked Write 4 Done Sent Done false
34194 9760 Acked Write 9 Done Sent Done false
34195 9761 Acked Write 10 Done Sent Done false
34196 9762 Acked Read 5 Done Sent Done false
34197 9763 Acked Read 4 Done Sent Done false
34198 9764 Acked Flush 0 Done Sent Done false
34199 9765 Acked Read 8 Done Sent Done false
34200 9766 Acked Write 8 Done Sent Done false
34201 9767 Acked Read 7 Done Sent Done false
34202 9768 Acked Read 3 Done Sent Done false
34203 9769 Acked Read 6 Done Sent Done false
34204 9770 Acked Write 5 Done Sent Done false
34205 9771 Acked Read 5 Done Sent Done false
34206 9772 Acked Read 2 Done Sent Done false
34207 9773 Acked Read 1 Done Sent Done false
34208 9774 Acked Read 6 Done Sent Done false
34209 9775 Acked Write 8 Done Sent Done false
34210 9776 Acked Write 9 Done Sent Done false
34211 9777 Acked Read 2 Done Sent Done false
34212 9778 Acked Write 1 Done Sent Done false
34213 9779 Acked Write 8 Done Sent Done false
34214 9780 Acked Read 7 Done Sent Done false
34215 9781 Acked Flush 0 Done Sent Done false
34216 9782 Acked Read 10 Done Sent Done false
34217 9783 Acked Read 6 Done Sent Done false
34218 9784 Acked Write 8 Done Sent Done false
34219 9785 Acked Read 2 Done Sent Done false
34220 9786 Acked Write 2 Done Sent Done false
34221 9787 Acked Write 6 Done Sent Done false
34222 9788 Acked Read 5 Done Sent Done false
34223 9789 Acked Read 3 Done Sent Done false
34224 9790 Acked Flush 0 Done Sent Done false
34225 9791 Acked Read 3 Done Sent Done false
34226 9792 Acked Read 3 Done Sent Done false
34227 9793 Acked Read 4 Done Sent Done false
34228 9794 Acked Write 10 Done Sent Done false
34229 9795 Acked Write 10 Done Sent Done false
34230 9796 Acked Write 2 Done Sent Done false
34231 9797 Acked Read 9 Done Sent Done false
34232 9798 Acked Write 8 Done Sent Done false
34233 9799 Acked Read 10 Done Sent Done false
34234 9800 Acked Read 10 Done Sent Done false
34235 9801 Acked Write 6 Done Sent Done false
34236 9802 Acked Read 1 Done Sent Done false
34237 9803 Acked Read 10 Done Sent Done false
34238 9804 Acked Flush 0 Done Sent Done false
34239 9805 Acked Read 6 Done Sent Done false
34240 9806 Acked Read 8 Done Sent Done false
34241 9807 Acked Read 4 Done Sent Done false
34242 9808 Acked Flush 0 Done Sent Done false
34243 9809 Acked Flush 0 Done Sent Done false
34244 9810 Acked Write 7 Done Sent Done false
34245 9811 Acked Read 4 Done Sent Done false
34246 9812 Acked Read 10 Done Sent Done false
34247 9813 Acked Read 10 Done Sent Done false
34248 9814 Acked Write 4 Done Sent Done false
34249 9815 Acked Read 9 Done Sent Done false
34250 9816 Acked Write 6 Done Sent Done false
34251 9817 Acked Read 2 Done Sent Done false
34252 9818 Acked Flush 0 Done Sent Done false
34253 9819 Acked Write 4 Done Sent Done false
34254 9820 Acked Write 9 Done Sent Done false
34255 9821 Acked Read 10 Done Sent Done false
34256 9822 Acked Write 2 Done Sent Done false
34257 9823 Acked Read 10 Done Sent Done false
34258 9824 Acked Read 3 Done Sent Done false
34259 9825 Acked Write 4 Done Sent Done false
34260 9826 Acked Read 5 Done Sent Done false
34261 9827 Acked Read 9 Done Sent Done false
34262 9828 Acked Read 8 Done Sent Done false
34263 9829 Acked Flush 0 Done Sent Done false
34264 9830 Acked Write 6 Done Sent Done false
34265 9831 Acked Write 10 Done Sent Done false
34266 9832 Acked Write 5 Done Sent Done false
34267 9833 Acked Read 5 Done Sent Done false
34268 9834 Acked Write 2 Done Sent Done false
34269 9835 Acked Write 8 Done Sent Done false
34270 9836 Acked Read 8 Done Sent Done false
34271 9837 Acked Write 1 Done Sent Done false
34272 9838 Acked Read 8 Done Sent Done false
34273 9839 Acked Read 5 Done Sent Done false
34274 9840 Acked Write 7 Done Sent Done false
34275 9841 Acked Read 2 Done Sent Done false
34276 9842 Acked Write 7 Done Sent Done false
34277 9843 Acked Flush 0 Done Sent Done false
34278 9844 Acked Write 1 Done Sent Done false
34279 9845 Acked Write 4 Done Sent Done false
34280 9846 Acked Read 1 Done Sent Done false
34281 9847 Acked Read 9 Done Sent Done false
34282 9848 Acked Read 8 Done Sent Done false
34283 9849 Acked Flush 0 Done Sent Done false
34284 9850 Acked Write 9 Done Sent Done false
34285 9851 Acked Read 4 Done Sent Done false
34286 9852 Acked Read 9 Done Sent Done false
34287 9853 Acked Read 9 Done Sent Done false
34288 9854 Acked Flush 0 Done Sent Done false
34289 9855 Acked Read 3 Done Sent Done false
34290 9856 Acked Write 1 Done Sent Done false
34291 9857 Acked Write 4 Done Sent Done false
34292 9858 Acked Write 5 Done Sent Done false
34293 9859 Acked Flush 0 Done Sent Done false
34294 9860 Acked Write 2 Done Sent Done false
34295 9861 Acked Read 7 Done Sent Done false
34296 9862 Acked Write 8 Done Sent Done false
34297 9863 Acked Flush 0 Done Sent Done false
34298 9864 Acked Write 10 Done Sent Done false
34299 9865 Acked Write 8 Done Sent Done false
34300 9866 Acked Write 2 Done Sent Done false
34301 9867 Acked Write 5 Done Sent Done false
34302 9868 Acked Write 1 Done Sent Done false
34303 9869 Acked Write 1 Done Sent Done false
34304 9870 Acked Read 2 Done Sent Done false
34305 9871 Acked Read 3 Done Sent Done false
34306 9872 Acked Write 5 Done Sent Done false
34307 9873 Acked Flush 0 Done Sent Done false
34308 9874 Acked Read 5 Done Sent Done false
34309 9875 Acked Write 6 Done Sent Done false
34310 9876 Acked Write 8 Done Sent Done false
34311 9877 Acked Read 3 Done Sent Done false
34312 9878 Acked Write 8 Done Sent Done false
34313 9879 Acked Write 4 Done Sent Done false
34314 9880 Acked Write 8 Done Sent Done false
34315 9881 Acked Read 3 Done Sent Done false
34316 9882 Acked Read 4 Done Sent Done false
34317 9883 Acked Read 6 Done Sent Done false
34318 9884 Acked Read 3 Done Sent Done false
34319 9885 Acked Flush 0 Done Sent Done false
34320 9886 Acked Read 2 Done Sent Done false
34321 9887 Acked Write 4 Done Sent Done false
34322 9888 Acked Flush 0 Done Sent Done false
34323 9889 Acked Write 10 Done Sent Done false
34324 9890 Acked Write 9 Done Sent Done false
34325 9891 Acked Read 4 Done Sent Done false
34326 9892 Acked Read 4 Done Sent Done false
34327 9893 Acked Write 8 Done Sent Done false
34328 9894 Acked Read 9 Done Sent Done false
34329 9895 Acked Read 1 Done Sent Done false
34330 9896 Acked Read 3 Done Sent Done false
34331 9897 Acked Read 6 Done Sent Done false
34332 9898 Acked Read 4 Done Sent Done false
34333 9899 Acked Write 5 Done Sent Done false
34334 9900 Acked Read 7 Done Sent Done false
34335 9901 Acked Write 4 Done Sent Done false
34336 9902 Acked Flush 0 Done Sent Done false
34337 9903 Acked Read 1 Done Sent Done false
34338 9904 Acked Read 8 Done Sent Done false
34339 9905 Acked Flush 0 Done Sent Done false
34340 9906 Acked Write 2 Done Sent Done false
34341 9907 Acked Read 7 Done Sent Done false
34342 9908 Acked Read 6 Done Sent Done false
34343 9909 Acked Write 4 Done Sent Done false
34344 9910 Acked Read 2 Done Sent Done false
34345 9911 Acked Write 1 Done Sent Done false
34346 9912 Acked Flush 0 Done Sent Done false
34347 9913 Acked Write 10 Done Sent Done false
34348 9914 Acked Write 6 Done Sent Done false
34349 9915 Acked Read 7 Done Sent Done false
34350 9916 Acked Read 4 Done Sent Done false
34351 9917 Acked Write 8 Done Sent Done false
34352 9918 Acked Read 5 Done Sent Done false
34353 9919 Acked Read 1 Done Sent Done false
34354 9920 Acked Read 2 Done Sent Done false
34355 9921 Acked Write 4 Done Sent Done false
34356 9922 Acked Write 10 Done Sent Done false
34357 9923 Acked Flush 0 Done Sent Done false
34358 9924 Acked Write 4 Done Sent Done false
34359 9925 Acked Flush 0 Done Sent Done false
34360 9926 Acked Write 8 Done Sent Done false
34361 9927 Acked Read 3 Done Sent Done false
34362 9928 Acked Read 9 Done Sent Done false
34363 9929 Acked Write 2 Done Sent Done false
34364 9930 Acked Read 3 Done Sent Done false
34365 9931 Acked Write 5 Done Sent Done false
34366 9932 Acked Flush 0 Done Sent Done false
34367 9933 Acked Read 3 Done Sent Done false
34368 9934 Acked Write 2 Done Sent Done false
34369 9935 Acked Write 4 Done Sent Done false
34370 9936 Acked Read 2 Done Sent Done false
34371 9937 Acked Read 9 Done Sent Done false
34372 9938 Acked Write 8 Done Sent Done false
34373 9939 Acked Read 4 Done Sent Done false
34374 9940 Acked Read 10 Done Sent Done false
34375 9941 Acked Read 2 Done Sent Done false
34376 9942 Acked Read 10 Done Sent Done false
34377 9943 Acked Write 7 Done Sent Done false
34378 9944 Acked Write 8 Done Sent Done false
34379 9945 Acked Read 1 Done Sent Done false
34380 9946 Acked Write 9 Done Sent Done false
34381 9947 Acked Flush 0 Done Sent Done false
34382 9948 Acked Read 2 Done Sent Done false
34383 9949 Acked Write 4 Done Sent Done false
34384 9950 Acked Read 2 Done Sent Done false
34385 9951 Acked Write 9 Done Sent Done false
34386 9952 Acked Read 10 Done Sent Done false
34387 9953 Acked Write 9 Done Sent Done false
34388 9954 Acked Write 2 Done Sent Done false
34389 9955 Acked Write 9 Done Sent Done false
34390 9956 Acked Flush 0 Done Sent Done false
34391 9957 Acked Read 6 Done Sent Done false
34392 9958 Acked Read 1 Done Sent Done false
34393 9959 Acked Read 7 Done Sent Done false
34394 9960 Acked Write 8 Done Sent Done false
34395 9961 Acked Write 10 Done Sent Done false
34396 9962 Acked Write 9 Done Sent Done false
34397 9963 Acked Read 8 Done Sent Done false
34398 9964 Acked Read 9 Done Sent Done false
34399 9965 Acked Flush 0 Done Sent Done false
34400 9966 Acked Read 7 Done Sent Done false
34401 9967 Acked Read 3 Done Sent Done false
34402 9968 Acked Write 3 Done Sent Done false
34403 9969 Acked Write 1 Done Sent Done false
34404 9970 Acked Write 7 Done Sent Done false
34405 9971 Acked Read 1 Done Sent Done false
34406 9972 Acked Read 7 Done Sent Done false
34407 9973 Acked Read 7 Done Sent Done false
34408 9974 Acked Write 2 Done Sent Done false
34409 9975 Acked Flush 0 Done Sent Done false
34410 9976 Acked Write 9 Done Sent Done false
34411 9977 Acked Write 1 Done Sent Done false
34412 9978 Acked Write 6 Done Sent Done false
34413 9979 Acked Read 3 Done Sent Done false
34414 9980 Acked Read 8 Done Sent Done false
34415 9981 Acked Flush 0 Done Sent Done false
34416 9982 Acked Write 4 Done Sent Done false
34417 9983 Acked Write 3 Done Sent Done false
34418 9984 Acked Write 2 Done Sent Done false
34419 9985 Acked Read 6 Done Sent Done false
34420 9986 Acked Write 2 Done Sent Done false
34421 9987 Acked Write 6 Done Sent Done false
34422 9988 Acked Read 8 Done Sent Done false
34423 9989 Acked Read 1 Done Sent Done false
34424 9990 Acked Read 9 Done Sent Done false
34425 9991 Acked Read 3 Done Sent Done false
34426 9992 Acked Write 10 Done Sent Done false
34427 9993 Acked Read 10 Done Sent Done false
34428 9994 Acked Read 8 Done Sent Done false
34429 9995 Acked Read 6 Done Sent Done false
34430 9996 Acked Write 5 Done Sent Done false
34431 9997 Acked Write 9 Done Sent Done false
34432 9998 Acked Write 6 Done Sent Done false
34433 9999 Acked Write 3 Done Sent Done false
34434 10000 Acked Write 9 Done Sent Done false
34435 10001 Acked Write 9 Done Sent Done false
34436 10002 Acked Read 5 Done Sent Done false
34437 10003 Acked Read 9 Done Sent Done false
34438 10004 Acked Write 8 Done Sent Done false
34439 10005 Acked Read 4 Done Sent Done false
34440 10006 Acked Read 7 Done Sent Done false
34441 10007 Acked Write 2 Done Sent Done false
34442 10008 Acked Flush 0 Done Sent Done false
34443 10009 Acked Read 4 Done Sent Done false
34444 10010 Acked Flush 0 Done Sent Done false
34445 10011 Acked Read 6 Done Sent Done false
34446 10012 Acked Read 1 Done Sent Done false
34447 10013 Acked Read 9 Done Sent Done false
34448 10014 Acked Read 6 Done Sent Done false
34449 10015 Acked Read 3 Done Sent Done false
34450 10016 Acked Read 6 Done Sent Done false
34451 10017 Acked Write 6 Done Sent Done false
34452 10018 Acked Write 5 Done Sent Done false
34453 10019 Acked Write 7 Done Sent Done false
34454 10020 Acked Write 3 Done Sent Done false
34455 10021 Acked Read 2 Done Sent Done false
34456 10022 Acked Write 7 Done Sent Done false
34457 10023 Acked Read 5 Done Sent Done false
34458 10024 Acked Read 9 Done Sent Done false
34459 10025 Acked Write 8 Done Sent Done false
34460 10026 Acked Write 4 Done Sent Done false
34461 10027 Acked Write 10 Done Sent Done false
34462 10028 Acked Write 4 Done Sent Done false
34463 10029 Acked Write 7 Done Sent Done false
34464 10030 Acked Write 2 Done Sent Done false
34465 10031 Acked Read 3 Done Sent Done false
34466 10032 Acked Read 3 Done Sent Done false
34467 10033 Acked Flush 0 Done Sent Done false
34468 10034 Acked Read 4 Done Sent Done false
34469 10035 Acked Read 9 Done Sent Done false
34470 10036 Acked Write 6 Done Sent Done false
34471 10037 Acked Flush 0 Done Sent Done false
34472 10038 Acked Read 3 Done Sent Done false
34473 10039 Acked Read 10 Done Sent Done false
34474 10040 Acked Read 10 Done Sent Done false
34475 10041 Acked Read 5 Done Sent Done false
34476 10042 Acked Write 7 Done Sent Done false
34477 10043 Acked Read 7 Done Sent Done false
34478 10044 Acked Flush 0 Done Sent Done false
34479 10045 Acked Write 6 Done Sent Done false
34480 10046 Acked Flush 0 Done Sent Done false
34481 10047 Acked Read 6 Done Sent Done false
34482 10048 Acked Write 2 Done Sent Done false
34483 10049 Acked Flush 0 Done Sent Done false
34484 10050 Acked Read 6 Done Sent Done false
34485 10051 Acked Read 3 Done Sent Done false
34486 10052 Acked Read 4 Done Sent Done false
34487 10053 Acked Read 4 Done Sent Done false
34488 10054 Acked Write 3 Done Sent Done false
34489 10055 Acked Read 2 Done Sent Done false
34490 10056 Acked Write 1 Done Sent Done false
34491 STATES DS:0 DS:1 DS:2 TOTAL
34492 Sent 0 301 0 301
34493 Done 301 0 301 602
34494 Skipped 0 0 0 0
34495 Error 0 0 0 0
34496 Last Flush: 10049 9755 10049
34497 Downstairs last five completed: 9755 9754 9753 9752 9751
34498 Upstairs last five completed: 10056 10055 10054 10053 10052
34499 CLIENT: Up:0 ds:301 act:2
34500 [8820] got start action
34501 [8820] I am going to restart
34502 Starting downstairs at port 8820
34503 Make output file at "/tmp/dsc/downstairs-8820.txt"
34504 [8820][1] reports Starting
34505 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1235)
34506 [8820][1] reports Running
34507 ----------------------------------------------------------------
34508 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34509 States: Active Offline Active
34510 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34511 9756 Acked Write 1 Done Sent Done false
34512 9757 Acked Read 5 Done Sent Done false
34513 9758 Acked Write 6 Done Sent Done false
34514 9759 Acked Write 4 Done Sent Done false
34515 9760 Acked Write 9 Done Sent Done false
34516 9761 Acked Write 10 Done Sent Done false
34517 9762 Acked Read 5 Done Sent Done false
34518 9763 Acked Read 4 Done Sent Done false
34519 9764 Acked Flush 0 Done Sent Done false
34520 9765 Acked Read 8 Done Sent Done false
34521 9766 Acked Write 8 Done Sent Done false
34522 9767 Acked Read 7 Done Sent Done false
34523 9768 Acked Read 3 Done Sent Done false
34524 9769 Acked Read 6 Done Sent Done false
34525 9770 Acked Write 5 Done Sent Done false
34526 9771 Acked Read 5 Done Sent Done false
34527 9772 Acked Read 2 Done Sent Done false
34528 9773 Acked Read 1 Done Sent Done false
34529 9774 Acked Read 6 Done Sent Done false
34530 9775 Acked Write 8 Done Sent Done false
34531 9776 Acked Write 9 Done Sent Done false
34532 9777 Acked Read 2 Done Sent Done false
34533 9778 Acked Write 1 Done Sent Done false
34534 9779 Acked Write 8 Done Sent Done false
34535 9780 Acked Read 7 Done Sent Done false
34536 9781 Acked Flush 0 Done Sent Done false
34537 9782 Acked Read 10 Done Sent Done false
34538 9783 Acked Read 6 Done Sent Done false
34539 9784 Acked Write 8 Done Sent Done false
34540 9785 Acked Read 2 Done Sent Done false
34541 9786 Acked Write 2 Done Sent Done false
34542 9787 Acked Write 6 Done Sent Done false
34543 9788 Acked Read 5 Done Sent Done false
34544 9789 Acked Read 3 Done Sent Done false
34545 9790 Acked Flush 0 Done Sent Done false
34546 9791 Acked Read 3 Done Sent Done false
34547 9792 Acked Read 3 Done Sent Done false
34548 9793 Acked Read 4 Done Sent Done false
34549 9794 Acked Write 10 Done Sent Done false
34550 9795 Acked Write 10 Done Sent Done false
34551 9796 Acked Write 2 Done Sent Done false
34552 9797 Acked Read 9 Done Sent Done false
34553 9798 Acked Write 8 Done Sent Done false
34554 9799 Acked Read 10 Done Sent Done false
34555 9800 Acked Read 10 Done Sent Done false
34556 9801 Acked Write 6 Done Sent Done false
34557 9802 Acked Read 1 Done Sent Done false
34558 9803 Acked Read 10 Done Sent Done false
34559 9804 Acked Flush 0 Done Sent Done false
34560 9805 Acked Read 6 Done Sent Done false
34561 9806 Acked Read 8 Done Sent Done false
34562 9807 Acked Read 4 Done Sent Done false
34563 9808 Acked Flush 0 Done Sent Done false
34564 9809 Acked Flush 0 Done Sent Done false
34565 9810 Acked Write 7 Done Sent Done false
34566 9811 Acked Read 4 Done Sent Done false
34567 9812 Acked Read 10 Done Sent Done false
34568 9813 Acked Read 10 Done Sent Done false
34569 9814 Acked Write 4 Done Sent Done false
34570 9815 Acked Read 9 Done Sent Done false
34571 9816 Acked Write 6 Done Sent Done false
34572 9817 Acked Read 2 Done Sent Done false
34573 9818 Acked Flush 0 Done Sent Done false
34574 9819 Acked Write 4 Done Sent Done false
34575 9820 Acked Write 9 Done Sent Done false
34576 9821 Acked Read 10 Done Sent Done false
34577 9822 Acked Write 2 Done Sent Done false
34578 9823 Acked Read 10 Done Sent Done false
34579 9824 Acked Read 3 Done Sent Done false
34580 9825 Acked Write 4 Done Sent Done false
34581 9826 Acked Read 5 Done Sent Done false
34582 9827 Acked Read 9 Done Sent Done false
34583 9828 Acked Read 8 Done Sent Done false
34584 9829 Acked Flush 0 Done Sent Done false
34585 9830 Acked Write 6 Done Sent Done false
34586 9831 Acked Write 10 Done Sent Done false
34587 9832 Acked Write 5 Done Sent Done false
34588 9833 Acked Read 5 Done Sent Done false
34589 9834 Acked Write 2 Done Sent Done false
34590 9835 Acked Write 8 Done Sent Done false
34591 9836 Acked Read 8 Done Sent Done false
34592 9837 Acked Write 1 Done Sent Done false
34593 9838 Acked Read 8 Done Sent Done false
34594 9839 Acked Read 5 Done Sent Done false
34595 9840 Acked Write 7 Done Sent Done false
34596 9841 Acked Read 2 Done Sent Done false
34597 9842 Acked Write 7 Done Sent Done false
34598 9843 Acked Flush 0 Done Sent Done false
34599 9844 Acked Write 1 Done Sent Done false
34600 9845 Acked Write 4 Done Sent Done false
34601 9846 Acked Read 1 Done Sent Done false
34602 9847 Acked Read 9 Done Sent Done false
34603 9848 Acked Read 8 Done Sent Done false
34604 9849 Acked Flush 0 Done Sent Done false
34605 9850 Acked Write 9 Done Sent Done false
34606 9851 Acked Read 4 Done Sent Done false
34607 9852 Acked Read 9 Done Sent Done false
34608 9853 Acked Read 9 Done Sent Done false
34609 9854 Acked Flush 0 Done Sent Done false
34610 9855 Acked Read 3 Done Sent Done false
34611 9856 Acked Write 1 Done Sent Done false
34612 9857 Acked Write 4 Done Sent Done false
34613 9858 Acked Write 5 Done Sent Done false
34614 9859 Acked Flush 0 Done Sent Done false
34615 9860 Acked Write 2 Done Sent Done false
34616 9861 Acked Read 7 Done Sent Done false
34617 9862 Acked Write 8 Done Sent Done false
34618 9863 Acked Flush 0 Done Sent Done false
34619 9864 Acked Write 10 Done Sent Done false
34620 9865 Acked Write 8 Done Sent Done false
34621 9866 Acked Write 2 Done Sent Done false
34622 9867 Acked Write 5 Done Sent Done false
34623 9868 Acked Write 1 Done Sent Done false
34624 9869 Acked Write 1 Done Sent Done false
34625 9870 Acked Read 2 Done Sent Done false
34626 9871 Acked Read 3 Done Sent Done false
34627 9872 Acked Write 5 Done Sent Done false
34628 9873 Acked Flush 0 Done Sent Done false
34629 9874 Acked Read 5 Done Sent Done false
34630 9875 Acked Write 6 Done Sent Done false
34631 9876 Acked Write 8 Done Sent Done false
34632 9877 Acked Read 3 Done Sent Done false
34633 9878 Acked Write 8 Done Sent Done false
34634 9879 Acked Write 4 Done Sent Done false
34635 9880 Acked Write 8 Done Sent Done false
34636 9881 Acked Read 3 Done Sent Done false
34637 9882 Acked Read 4 Done Sent Done false
34638 9883 Acked Read 6 Done Sent Done false
34639 9884 Acked Read 3 Done Sent Done false
34640 9885 Acked Flush 0 Done Sent Done false
34641 9886 Acked Read 2 Done Sent Done false
34642 9887 Acked Write 4 Done Sent Done false
34643 9888 Acked Flush 0 Done Sent Done false
34644 9889 Acked Write 10 Done Sent Done false
34645 9890 Acked Write 9 Done Sent Done false
34646 9891 Acked Read 4 Done Sent Done false
34647 9892 Acked Read 4 Done Sent Done false
34648 9893 Acked Write 8 Done Sent Done false
34649 9894 Acked Read 9 Done Sent Done false
34650 9895 Acked Read 1 Done Sent Done false
34651 9896 Acked Read 3 Done Sent Done false
34652 9897 Acked Read 6 Done Sent Done false
34653 9898 Acked Read 4 Done Sent Done false
34654 9899 Acked Write 5 Done Sent Done false
34655 9900 Acked Read 7 Done Sent Done false
34656 9901 Acked Write 4 Done Sent Done false
34657 9902 Acked Flush 0 Done Sent Done false
34658 9903 Acked Read 1 Done Sent Done false
34659 9904 Acked Read 8 Done Sent Done false
34660 9905 Acked Flush 0 Done Sent Done false
34661 9906 Acked Write 2 Done Sent Done false
34662 9907 Acked Read 7 Done Sent Done false
34663 9908 Acked Read 6 Done Sent Done false
34664 9909 Acked Write 4 Done Sent Done false
34665 9910 Acked Read 2 Done Sent Done false
34666 9911 Acked Write 1 Done Sent Done false
34667 9912 Acked Flush 0 Done Sent Done false
34668 9913 Acked Write 10 Done Sent Done false
34669 9914 Acked Write 6 Done Sent Done false
34670 9915 Acked Read 7 Done Sent Done false
34671 9916 Acked Read 4 Done Sent Done false
34672 9917 Acked Write 8 Done Sent Done false
34673 9918 Acked Read 5 Done Sent Done false
34674 9919 Acked Read 1 Done Sent Done false
34675 9920 Acked Read 2 Done Sent Done false
34676 9921 Acked Write 4 Done Sent Done false
34677 9922 Acked Write 10 Done Sent Done false
34678 9923 Acked Flush 0 Done Sent Done false
34679 9924 Acked Write 4 Done Sent Done false
34680 9925 Acked Flush 0 Done Sent Done false
34681 9926 Acked Write 8 Done Sent Done false
34682 9927 Acked Read 3 Done Sent Done false
34683 9928 Acked Read 9 Done Sent Done false
34684 9929 Acked Write 2 Done Sent Done false
34685 9930 Acked Read 3 Done Sent Done false
34686 9931 Acked Write 5 Done Sent Done false
34687 9932 Acked Flush 0 Done Sent Done false
34688 9933 Acked Read 3 Done Sent Done false
34689 9934 Acked Write 2 Done Sent Done false
34690 9935 Acked Write 4 Done Sent Done false
34691 9936 Acked Read 2 Done Sent Done false
34692 9937 Acked Read 9 Done Sent Done false
34693 9938 Acked Write 8 Done Sent Done false
34694 9939 Acked Read 4 Done Sent Done false
34695 9940 Acked Read 10 Done Sent Done false
34696 9941 Acked Read 2 Done Sent Done false
34697 9942 Acked Read 10 Done Sent Done false
34698 9943 Acked Write 7 Done Sent Done false
34699 9944 Acked Write 8 Done Sent Done false
34700 9945 Acked Read 1 Done Sent Done false
34701 9946 Acked Write 9 Done Sent Done false
34702 9947 Acked Flush 0 Done Sent Done false
34703 9948 Acked Read 2 Done Sent Done false
34704 9949 Acked Write 4 Done Sent Done false
34705 9950 Acked Read 2 Done Sent Done false
34706 9951 Acked Write 9 Done Sent Done false
34707 9952 Acked Read 10 Done Sent Done false
34708 9953 Acked Write 9 Done Sent Done false
34709 9954 Acked Write 2 Done Sent Done false
34710 9955 Acked Write 9 Done Sent Done false
34711 9956 Acked Flush 0 Done Sent Done false
34712 9957 Acked Read 6 Done Sent Done false
34713 9958 Acked Read 1 Done Sent Done false
34714 9959 Acked Read 7 Done Sent Done false
34715 9960 Acked Write 8 Done Sent Done false
34716 9961 Acked Write 10 Done Sent Done false
34717 9962 Acked Write 9 Done Sent Done false
34718 9963 Acked Read 8 Done Sent Done false
34719 9964 Acked Read 9 Done Sent Done false
34720 9965 Acked Flush 0 Done Sent Done false
34721 9966 Acked Read 7 Done Sent Done false
34722 9967 Acked Read 3 Done Sent Done false
34723 9968 Acked Write 3 Done Sent Done false
34724 9969 Acked Write 1 Done Sent Done false
34725 9970 Acked Write 7 Done Sent Done false
34726 9971 Acked Read 1 Done Sent Done false
34727 9972 Acked Read 7 Done Sent Done false
34728 9973 Acked Read 7 Done Sent Done false
34729 9974 Acked Write 2 Done Sent Done false
34730 9975 Acked Flush 0 Done Sent Done false
34731 9976 Acked Write 9 Done Sent Done false
34732 9977 Acked Write 1 Done Sent Done false
34733 9978 Acked Write 6 Done Sent Done false
34734 9979 Acked Read 3 Done Sent Done false
34735 9980 Acked Read 8 Done Sent Done false
34736 9981 Acked Flush 0 Done Sent Done false
34737 9982 Acked Write 4 Done Sent Done false
34738 9983 Acked Write 3 Done Sent Done false
34739 9984 Acked Write 2 Done Sent Done false
34740 9985 Acked Read 6 Done Sent Done false
34741 9986 Acked Write 2 Done Sent Done false
34742 9987 Acked Write 6 Done Sent Done false
34743 9988 Acked Read 8 Done Sent Done false
34744 9989 Acked Read 1 Done Sent Done false
34745 9990 Acked Read 9 Done Sent Done false
34746 9991 Acked Read 3 Done Sent Done false
34747 9992 Acked Write 10 Done Sent Done false
34748 9993 Acked Read 10 Done Sent Done false
34749 9994 Acked Read 8 Done Sent Done false
34750 9995 Acked Read 6 Done Sent Done false
34751 9996 Acked Write 5 Done Sent Done false
34752 9997 Acked Write 9 Done Sent Done false
34753 9998 Acked Write 6 Done Sent Done false
34754 9999 Acked Write 3 Done Sent Done false
34755 10000 Acked Write 9 Done Sent Done false
34756 10001 Acked Write 9 Done Sent Done false
34757 10002 Acked Read 5 Done Sent Done false
34758 10003 Acked Read 9 Done Sent Done false
34759 10004 Acked Write 8 Done Sent Done false
34760 10005 Acked Read 4 Done Sent Done false
34761 10006 Acked Read 7 Done Sent Done false
34762 10007 Acked Write 2 Done Sent Done false
34763 10008 Acked Flush 0 Done Sent Done false
34764 10009 Acked Read 4 Done Sent Done false
34765 10010 Acked Flush 0 Done Sent Done false
34766 10011 Acked Read 6 Done Sent Done false
34767 10012 Acked Read 1 Done Sent Done false
34768 10013 Acked Read 9 Done Sent Done false
34769 10014 Acked Read 6 Done Sent Done false
34770 10015 Acked Read 3 Done Sent Done false
34771 10016 Acked Read 6 Done Sent Done false
34772 10017 Acked Write 6 Done Sent Done false
34773 10018 Acked Write 5 Done Sent Done false
34774 10019 Acked Write 7 Done Sent Done false
34775 10020 Acked Write 3 Done Sent Done false
34776 10021 Acked Read 2 Done Sent Done false
34777 10022 Acked Write 7 Done Sent Done false
34778 10023 Acked Read 5 Done Sent Done false
34779 10024 Acked Read 9 Done Sent Done false
34780 10025 Acked Write 8 Done Sent Done false
34781 10026 Acked Write 4 Done Sent Done false
34782 10027 Acked Write 10 Done Sent Done false
34783 10028 Acked Write 4 Done Sent Done false
34784 10029 Acked Write 7 Done Sent Done false
34785 10030 Acked Write 2 Done Sent Done false
34786 10031 Acked Read 3 Done Sent Done false
34787 10032 Acked Read 3 Done Sent Done false
34788 10033 Acked Flush 0 Done Sent Done false
34789 10034 Acked Read 4 Done Sent Done false
34790 10035 Acked Read 9 Done Sent Done false
34791 10036 Acked Write 6 Done Sent Done false
34792 10037 Acked Flush 0 Done Sent Done false
34793 10038 Acked Read 3 Done Sent Done false
34794 10039 Acked Read 10 Done Sent Done false
34795 10040 Acked Read 10 Done Sent Done false
34796 10041 Acked Read 5 Done Sent Done false
34797 10042 Acked Write 7 Done Sent Done false
34798 10043 Acked Read 7 Done Sent Done false
34799 10044 Acked Flush 0 Done Sent Done false
34800 10045 Acked Write 6 Done Sent Done false
34801 10046 Acked Flush 0 Done Sent Done false
34802 10047 Acked Read 6 Done Sent Done false
34803 10048 Acked Write 2 Done Sent Done false
34804 10049 Acked Flush 0 Done Sent Done false
34805 10050 Acked Read 6 Done Sent Done false
34806 10051 Acked Read 3 Done Sent Done false
34807 10052 Acked Read 4 Done Sent Done false
34808 10053 Acked Read 4 Done Sent Done false
34809 10054 Acked Write 3 Done Sent Done false
34810 10055 Acked Read 2 Done Sent Done false
34811 10056 Acked Write 1 Done Sent Done false
34812 10057 Acked Flush 0 Done Sent Done false
34813 STATES DS:0 DS:1 DS:2 TOTAL
34814 Sent 0 302 0 302
34815 Done 302 0 302 604
34816 Skipped 0 0 0 0
34817 Error 0 0 0 0
34818 Last Flush: 10057 9755 10057
34819 Downstairs last five completed: 9755 9754 9753 9752 9751
34820 Upstairs last five completed: 10057 10056 10055 10054 10053
34821 CLIENT: Up:0 ds:302 act:2
348222025-01-03T08:38:10.882ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348232025-01-03T08:38:10.882ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42194) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348242025-01-03T08:38:10.883ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348252025-01-03T08:38:10.884ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348262025-01-03T08:38:10.884ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348272025-01-03T08:38:10.884ZINFOcrucible: send last flush ID to this DS: 9755 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348282025-01-03T08:38:10.884ZINFOcrucible: Replied this last flush ID: 9755 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348292025-01-03T08:38:10.884ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348302025-01-03T08:38:10.884ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348312025-01-03T08:38:10.884ZINFOcrucible: [1] client re-new 302 jobs since flush 9755 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348322025-01-03T08:38:10.884ZINFOcrucible: [1] Replayed 302 jobs since flush: 9755 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
34833 ----------------------------------------------------------------
34834 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34835 Downstairs last five completed: 10057 10056 10055 10054 10053
34836 Upstairs last five completed: 10057 10056 10055 10054 10053
34837 CLIENT: Up:0 ds:0 act:3
34838 Replay: All jobs finished
34839 Main task has work to do, go find it
34840 got dsc Stop(1)
34841 stop 1
34842 Jan 03 08:38:13.124 INFO request completed[8820] Got stop action so:false kr:false
34843 , latency_us: 353, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 41ff8ddf-29de-4e30-ab49-b98237395bdb, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
348442025-01-03T08:38:13.125ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348452025-01-03T08:38:13.125ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348462025-01-03T08:38:13.125ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
348472025-01-03T08:38:13.125ZINFOcrucible: ds_transition from Active to Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
34848 Jan 03 08:38:13.125 INFO request completed, latency_us: 131, response_code: 200, uri: /state/cid/1, method: GET, req_id: {"msg":"[1] Transition from Active to Offline","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:38:13.125524007Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
34849 c5794901-1a6a-4a7b-b93e-fae7509b8bc5, remote_addr: 127.0.0.1:49669, local_addr: 127.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-03T08:38:13.125608926Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1205,"":"io task","client":"1","":"downstairs","session_id":"4d73cd98-4f0b-450f-bfa8-8a9ca6991800"}
34850 0.1:9998
34851 [8820] Exited with: ExitStatus(unix_wait_status(9))
34852 [8820][1] reports Exit
34853 Jan 03 08:38:17.127 INFO request completed, latency_us: 200, response_code: 200, uri: /state/cid/1, method: GET, req_id: c8666c1b-5988-4512-a52b-6f041c7700e1, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
34854 001/300 Write block 204 len 12288 data: 4 4 6
34855 002/300 Read block 2154 len 32768
34856 003/300 Write block 250 len 32768 data: 8 7 6 8 9 10 7 8
34857 004/300 Write block 1047 len 40960 data: 5 3 4 5 5 4 5 6 5 5
34858 005/300 Read block 1949 len 20480
34859 006/300 Read block 3093 len 8192
34860 007/300 Write block 1988 len 16384 data: 5 5 5 5
34861 008/300 Write block 2373 len 8192 data: 7 7
34862 009/300 Write block 3767 len 24576 data: 12 11 10 9 9 10
34863 010/300 Write block 1139 len 8192 data: 6 6
34864 011/300 Flush
34865 012/300 Write block 3731 len 12288 data: 8 9 8
34866 013/300 Write block 971 len 12288 data: 5 4 4
34867 014/300 Write block 4501 len 16384 data: 7 7 8 8
34868 015/300 Read block 3045 len 20480
34869 016/300 Read block 480 len 20480
34870 017/300 Write block 258 len 4096 data: 6
34871 018/300 Write block 249 len 4096 data: 8
34872 019/300 Write block 746 len 32768 data: 7 7 7 7 8 7 7 5
34873 020/300 Read block 1351 len 12288
34874 021/300 Read block 2849 len 36864
34875 022/300 Read block 620 len 20480
34876 023/300 Read block 717 len 32768
34877 024/300 Flush
34878 025/300 Read block 963 len 24576
34879 026/300 Flush
34880 027/300 Read block 4773 len 4096
34881 028/300 Read block 2495 len 4096
34882 029/300 Read block 3411 len 16384
34883 030/300 Write block 787 len 4096 data: 7
34884 031/300 Read block 433 len 12288
34885 032/300 Read block 4430 len 36864
34886 033/300 Write block 3712 len 8192 data: 5 5
34887 034/300 Read block 2191 len 8192
34888 035/300 Read block 1514 len 24576
34889 036/300 Read block 3112 len 8192
34890 037/300 Write block 441 len 8192 data: 6 5
34891 038/300 Write block 281 len 36864 data: 6 7 7 7 7 6 6 5 4
34892 039/300 Read block 1874 len 4096
34893 040/300 Read block 1884 len 12288
34894 041/300 Write block 3454 len 20480 data: 9 9 9 8 8
34895 042/300 Write block 2035 len 36864 data: 6 6 4 5 4 4 5 7 6
34896 043/300 Write block 639 len 8192 data: 8 9
34897 044/300 Read block 448 len 12288
34898 045/300 Write block 341 len 24576 data: 5 6 6 6 9 9
34899 046/300 Read block 3719 len 24576
34900 047/300 Read block 1213 len 36864
34901 048/300 Flush
34902 049/300 Read block 170 len 24576
34903 050/300 Write block 6 len 28672 data: 3 4 3 3 4 5 5
34904 051/300 Read block 3214 len 36864
34905 052/300 Read block 4763 len 32768
34906 053/300 Flush
34907 054/300 Read block 3846 len 20480
34908 055/300 Read block 1577 len 36864
34909 056/300 Read block 1438 len 24576
34910 057/300 Read block 741 len 28672
34911 058/300 Flush
34912 059/300 Write block 862 len 32768 data: 4 3 4 4 5 8 7 7
34913 060/300 Write block 2941 len 16384 data: 7 7 6 6
34914 061/300 Write block 3306 len 24576 data: 6 6 5 7 6 6
34915 062/300 Write block 2061 len 28672 data: 7 8 6 5 6 7 7
34916 063/300 Write block 2212 len 40960 data: 5 6 8 7 6 8 8 7 7 8
34917 064/300 Read block 2942 len 24576
34918 065/300 Read block 3270 len 28672
34919 066/300 Write block 278 len 32768 data: 4 5 5 7 8 8 8 8
34920 067/300 Read block 4265 len 4096
34921 068/300 Read block 3032 len 24576
34922 069/300 Read block 2307 len 20480
34923 070/300 Read block 2352 len 36864
34924 071/300 Write block 1917 len 36864 data: 5 5 8 9 8 9 10 11 9
34925 072/300 Write block 4677 len 12288 data: 5 5 5
34926 073/300 Write block 1999 len 8192 data: 6 6
34927 074/300 Read block 4424 len 32768
34928 075/300 Read block 495 len 32768
34929 076/300 Read block 4540 len 20480
34930 077/300 Read block 3336 len 12288
34931 078/300 Read block 1611 len 24576
34932 079/300 Read block 3809 len 24576
34933 080/300 Read block 2435 len 8192
34934 081/300 Read block 57 len 40960
34935 082/300 Flush
34936 083/300 Read block 2999 len 24576
34937 084/300 Write block 205 len 8192 data: 5 7
34938 085/300 Flush
34939 086/300 Read block 4648 len 8192
34940 087/300 Read block 3933 len 32768
34941 088/300 Write block 2508 len 28672 data: 5 5 5 5 5 3 3
34942 089/300 Write block 4243 len 24576 data: 6 6 6 5 5 5
34943 090/300 Write block 1154 len 16384 data: 10 9 8 8
34944 091/300 Write block 494 len 12288 data: 6 5 6
34945 092/300 Write block 437 len 36864 data: 6 7 8 9 7 6 4 5 5
34946 093/300 Read block 4634 len 4096
34947 094/300 Read block 1486 len 8192
34948 095/300 Write block 233 len 4096 data: 6
34949 096/300 Flush
34950 097/300 Flush
34951 098/300 Write block 3783 len 32768 data: 5 6 6 7 6 7 6 6
34952 099/300 Write block 2542 len 16384 data: 4 4 4 5
34953 100/300 Read block 3617 len 4096
34954 101/300 Write block 4644 len 28672 data: 6 7 7 7 9 10 6
34955 102/300 Write block 4953 len 20480 data: 6 7 8 7 7
34956 103/300 Read block 4268 len 36864
34957 104/300 Flush
34958 105/300 Read block 4000 len 4096
34959 106/300 Write block 2392 len 20480 data: 8 9 9 7 7
34960 107/300 Read block 1055 len 8192
34961 108/300 Write block 4709 len 28672 data: 9 8 9 9 8 6 5
34962 109/300 Flush
34963 110/300 Read block 2131 len 36864
34964 111/300 Read block 3390 len 40960
34965 112/300 Read block 2567 len 40960
34966 113/300 Flush
34967 114/300 Write block 1652 len 16384 data: 7 8 7 6
34968 115/300 Write block 2072 len 40960 data: 8 10 9 9 7 7 8 5 4 5
34969 116/300 Write block 1783 len 40960 data: 4 4 3 4 5 7 6 7 6 6
34970 117/300 Read block 2410 len 12288
34971 118/300 Write block 4100 len 8192 data: 5 4
34972 119/300 Flush
34973 120/300 Read block 1151 len 24576
34974 121/300 Flush
34975 122/300 Write block 787 len 8192 data: 8 7
34976 123/300 Read block 683 len 28672
34977 124/300 Read block 4693 len 40960
34978 125/300 Flush
34979 126/300 Write block 3389 len 4096 data: 9
34980 127/300 Flush
34981 128/300 Write block 772 len 24576 data: 6 6 8 7 7 7
34982 129/300 Read block 3503 len 36864
34983 130/300 Read block 3448 len 12288
34984 131/300 Read block 1801 len 20480
34985 132/300 Read block 4052 len 20480
34986 133/300 Read block 1567 len 28672
34987 134/300 Write block 2933 len 28672 data: 4 4 5 4 5 6 6
34988 135/300 Read block 1366 len 4096
34989 136/300 Read block 867 len 40960
34990 137/300 Flush
34991 138/300 Write block 700 len 20480 data: 3 3 4 5 6
34992 139/300 Read block 3262 len 36864
34993 140/300 Write block 1487 len 12288 data: 8 8 8
34994 141/300 Read block 412 len 24576
34995 142/300 Read block 4804 len 20480
34996 143/300 Read block 2847 len 36864
34997 144/300 Read block 4864 len 8192
34998 145/300 Write block 939 len 28672 data: 7 6 6 6 6 5 5
34999 146/300 Write block 34 len 16384 data: 6 6 5 6
35000 147/300 Read block 1320 len 28672
35001 148/300 Read block 4724 len 16384
35002 149/300 Write block 2697 len 32768 data: 9 7 5 7 6 5 4 4
35003 150/300 Read block 1518 len 24576
35004 151/300 Read block 341 len 4096
35005 152/300 Flush
35006 153/300 Write block 4980 len 24576 data: 6 7 6 6 6 4
35007 154/300 Write block 3809 len 4096 data: 5
35008 155/300 Write block 2537 len 8192 data: 4 5
35009 156/300 Read block 4995 len 12288
35010 157/300 Write block 3342 len 20480 data: 4 5 5 5 5
35011 158/300 Read block 1399 len 20480
35012 159/300 Flush
35013 160/300 Read block 2331 len 20480
35014 161/300 Flush
35015 162/300 Write block 4752 len 16384 data: 6 9 10 9
35016 163/300 Write block 353 len 32768 data: 7 6 4 3 4 5 4 4
35017 164/300 Write block 4558 len 16384 data: 6 6 6 4
35018 165/300 Read block 1396 len 12288
35019 166/300 Flush
35020 167/300 Write block 4016 len 4096 data: 5
35021 168/300 Read block 1634 len 28672
35022 169/300 Read block 3421 len 28672
35023 170/300 Read block 210 len 12288
35024 171/300 Write block 965 len 28672 data: 6 6 6 6 6 5 6
35025 172/300 Read block 54 len 8192
35026 173/300 Read block 1718 len 28672
35027 174/300 Read block 2517 len 24576
35028 175/300 Read block 3039 len 24576
35029 176/300 Read block 709 len 32768
35030 177/300 Write block 2878 len 12288 data: 4 5 4
35031 178/300 Flush
35032 179/300 Read block 2009 len 28672
35033 180/300 Read block 4034 len 24576
35034 181/300 Write block 3632 len 8192 data: 8 10
35035 182/300 Write block 3851 len 8192 data: 7 10
35036 183/300 Read block 3854 len 12288
35037 184/300 Read block 134 len 32768
35038 185/300 Read block 4969 len 40960
35039 186/300 Read block 263 len 24576
35040 187/300 Write block 3985 len 28672 data: 6 6 6 6 6 3 3
35041 188/300 Write block 2566 len 32768 data: 5 7 9 10 9 9 9 7
35042 189/300 Write block 4062 len 32768 data: 6 6 5 5 7 10 9 8
35043 190/300 Read block 601 len 4096
35044 191/300 Read block 3047 len 40960
35045 192/300 Write block 666 len 32768 data: 8 7 8 9 6 6 5 5
35046 193/300 Write block 2559 len 8192 data: 6 5
35047 194/300 Flush
35048 195/300 Read block 548 len 36864
35049 196/300 Read block 1447 len 32768
35050 197/300 Write block 3138 len 12288 data: 6 7 7
35051 198/300 Read block 3904 len 36864
35052 199/300 Read block 4077 len 16384
35053 200/300 Flush
35054 201/300 Read block 4106 len 32768
35055 202/300 Read block 3621 len 36864
35056 203/300 Write block 4455 len 36864 data: 4 5 7 7 6 4 4 5 5
35057 204/300 Read block 2424 len 4096
35058 205/300 Write block 3402 len 28672 data: 11 9 7 6 7 7 5
35059 206/300 Write block 54 len 24576 data: 8 7 7 6 5 6
35060 207/300 Write block 2765 len 4096 data: 5
35061 208/300 Write block 774 len 20480 data: 9 8 8 8 7
35062 209/300 Flush
35063 210/300 Write block 980 len 28672 data: 4 5 4 4 6 5 6
35064 211/300 Write block 906 len 16384 data: 6 8 7 8
35065 212/300 Write block 4431 len 8192 data: 7 9
35066 213/300 Flush
35067 214/300 Write block 2260 len 16384 data: 5 4 5 7
35068 215/300 Write block 883 len 16384 data: 5 4 6 6
35069 216/300 Read block 327 len 20480
35070 217/300 Read block 1420 len 8192
35071 218/300 Read block 859 len 16384
35072 219/300 Read block 2537 len 16384
35073 220/300 Read block 706 len 28672
35074 221/300 Write block 1583 len 12288 data: 4 6 9
35075 222/300 Write block 4470 len 40960 data: 7 7 7 7 5 6 7 7 9 8
35076 223/300 Flush
35077 224/300 Read block 382 len 20480
35078 225/300 Read block 2667 len 40960
35079 226/300 Flush
35080 227/300 Flush
35081 228/300 Read block 3663 len 12288
35082 229/300 Write block 2964 len 32768 data: 7 7 7 6 6 8 7 8
35083 230/300 Read block 2902 len 16384
35084 231/300 Read block 192 len 12288
35085 232/300 Flush
35086 233/300 Write block 3375 len 24576 data: 7 5 2 2 3 3
35087 234/300 Read block 354 len 8192
35088 235/300 Read block 4727 len 12288
35089 236/300 Write block 4031 len 16384 data: 5 5 4 4
35090 237/300 Read block 4994 len 4096
35091 238/300 Write block 4573 len 28672 data: 6 6 8 7 7 8 6
35092 239/300 Read block 2713 len 40960
35093 240/300 Write block 4126 len 20480 data: 4 5 6 6 7
35094 241/300 Write block 4478 len 36864 data: 10 9 6 6 7 6 6 7 7
35095 242/300 Read block 1017 len 36864
35096 243/300 Write block 3683 len 40960 data: 6 5 5 4 4 4 4 5 4 5
35097 244/300 Write block 4091 len 28672 data: 3 4 6 7 8 10 8
35098 245/300 Write block 406 len 36864 data: 9 10 9 6 6 5 6 7 5
35099 246/300 Write block 169 len 36864 data: 3 5 7 10 9 7 8 8 6
35100 247/300 Write block 2451 len 4096 data: 6
35101 248/300 Write block 4151 len 8192 data: 3 2
35102 249/300 Write block 3130 len 40960 data: 5 5 6 6 5 5 5 5 7 8
35103 250/300 Flush
35104 251/300 Write block 4034 len 12288 data: 5 5 5
35105 252/300 Write block 4226 len 40960 data: 6 8 8 9 8 7 9 7 4 4
35106 253/300 Write block 2436 len 28672 data: 2 2 2 3 4 6 6
35107 254/300 Write block 4003 len 32768 data: 9 12 12 12 11 11 8 7
35108 255/300 Read block 4906 len 4096
35109 256/300 Read block 4279 len 24576
35110 257/300 Write block 2191 len 28672 data: 7 4 4 5 5 5 5
35111 258/300 Read block 1714 len 36864
35112 259/300 Write block 2987 len 8192 data: 3 4
35113 260/300 Read block 1592 len 40960
35114 261/300 Read block 4114 len 32768
35115 262/300 Flush
35116 263/300 Read block 207 len 20480
35117 264/300 Write block 1148 len 28672 data: 5 5 5 5 5 7 11
35118 265/300 Write block 4949 len 12288 data: 9 8 5
35119 266/300 Read block 186 len 36864
35120 267/300 Read block 882 len 4096
35121 268/300 Write block 4666 len 20480 data: 6 5 5 7 8
35122 269/300 Flush
35123 270/300 Flush
35124 271/300 Write block 2061 len 4096 data: 8
35125 272/300 Write block 181 len 8192 data: 5 6
35126 273/300 Read block 2734 len 28672
35127 274/300 Write block 2585 len 20480 data: 8 6 4 5 4
35128 275/300 Write block 1287 len 4096 data: 8
35129 276/300 Read block 624 len 36864
35130 277/300 Write block 2791 len 20480 data: 5 2 4 4 6
35131 278/300 Flush
35132 279/300 Write block 4384 len 40960 data: 4 4 6 7 6 6 5 5 8 7
35133 280/300 Read block 3741 len 28672
35134 281/300 Write block 1241 len 32768 data: 3 6 6 6 7 7 7 9
35135 282/300 Write block 3765 len 20480 data: 9 11 13 12 11
35136 283/300 Read block 1999 len 4096
35137 284/300 Read block 2706 len 28672
35138 285/300 Flush
35139 286/300 Read block 4960 len 40960
35140 287/300 Write block 190 len 8192 data: 6 6
35141 288/300 Write block 581 len 8192 data: 6 5
35142 289/300 Read block 189 len 24576
35143 290/300 Write block 4737 len 20480 data: 4 3 4 4 5
35144 291/300 Write block 2821 len 24576 data: 6 6 8 8 9 13
35145 292/300 Write block 838 len 16384 data: 6 6 7 7
35146 293/300 Write block 2912 len 4096 data: 6
35147 294/300 Write block 2180 len 20480 data: 3 3 4 5 7
35148 295/300 Read block 88 len 32768
35149 296/300 Read block 886 len 36864
35150 297/300 Write block 821 len 8192 data: 6 6
35151 298/300 Read block 507 len 28672
35152 299/300 Write block 2155 len 4096 data: 9
35153 300/300 Flush
35154 301/300 Read block 297 len 24576
35155 Main task has work to do, go find it
35156 got dsc Start(1)
35157 start 1
35158 Jan 03 08:38:17.359 INFO request completed, latency_us: 245, response_code: 204, uri: /start/cid/1, method: POST, req_id: b9ec3409-4323-4c8d-8083-71c342e84fdb, remote_addr: 127.0.0.1:49669, local_addr: 127.0.0.1:9998
35159 Replay: started 1, returned:Ok(())
35160 ----------------------------------------------------------------
35161 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
35162 States: Active Offline Active
35163 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35164 10058 Acked Write 3 Done Sent Done false
35165 10059 Acked Read 8 Done Sent Done false
35166 10060 Acked Write 8 Done Sent Done false
35167 10061 Acked Write 10 Done Sent Done false
35168 10062 Acked Read 5 Done Sent Done false
35169 10063 Acked Read 2 Done Sent Done false
35170 10064 Acked Write 4 Done Sent Done false
35171 10065 Acked Write 2 Done Sent Done false
35172 10066 Acked Write 6 Done Sent Done false
35173 10067 Acked Write 2 Done Sent Done false
35174 10068 Acked Flush 0 Done Sent Done false
35175 10069 Acked Write 3 Done Sent Done false
35176 10070 Acked Write 3 Done Sent Done false
35177 10071 Acked Write 4 Done Sent Done false
35178 10072 Acked Read 5 Done Sent Done false
35179 10073 Acked Read 5 Done Sent Done false
35180 10074 Acked Write 1 Done Sent Done false
35181 10075 Acked Write 1 Done Sent Done false
35182 10076 Acked Write 8 Done Sent Done false
35183 10077 Acked Read 3 Done Sent Done false
35184 10078 Acked Read 9 Done Sent Done false
35185 10079 Acked Read 5 Done Sent Done false
35186 10080 Acked Read 8 Done Sent Done false
35187 10081 Acked Flush 0 Done Sent Done false
35188 10082 Acked Read 6 Done Sent Done false
35189 10083 Acked Flush 0 Done Sent Done false
35190 10084 Acked Read 1 Done Sent Done false
35191 10085 Acked Read 1 Done Sent Done false
35192 10086 Acked Read 4 Done Sent Done false
35193 10087 Acked Write 1 Done Sent Done false
35194 10088 Acked Read 3 Done Sent Done false
35195 10089 Acked Read 9 Done Sent Done false
35196 10090 Acked Write 2 Done Sent Done false
35197 10091 Acked Read 2 Done Sent Done false
35198 10092 Acked Read 6 Done Sent Done false
35199 10093 Acked Read 2 Done Sent Done false
35200 10094 Acked Write 2 Done Sent Done false
35201 10095 Acked Write 9 Done Sent Done false
35202 10096 Acked Read 1 Done Sent Done false
35203 10097 Acked Read 3 Done Sent Done false
35204 10098 Acked Write 5 Done Sent Done false
35205 10099 Acked Write 9 Done Sent Done false
35206 10100 Acked Write 2 Done Sent Done false
35207 10101 Acked Read 3 Done Sent Done false
35208 10102 Acked Write 6 Done Sent Done false
35209 10103 Acked Read 6 Done Sent Done false
35210 10104 Acked Read 9 Done Sent Done false
35211 10105 Acked Flush 0 Done Sent Done false
35212 10106 Acked Read 6 Done Sent Done false
35213 10107 Acked Write 7 Done Sent Done false
35214 10108 Acked Read 9 Done Sent Done false
35215 10109 Acked Read 8 Done Sent Done false
35216 10110 Acked Flush 0 Done Sent Done false
35217 10111 Acked Read 5 Done Sent Done false
35218 10112 Acked Read 9 Done Sent Done false
35219 10113 Acked Read 6 Done Sent Done false
35220 10114 Acked Read 7 Done Sent Done false
35221 10115 Acked Flush 0 Done Sent Done false
35222 10116 Acked Write 8 Done Sent Done false
35223 10117 Acked Write 4 Done Sent Done false
35224 10118 Acked Write 6 Done Sent Done false
35225 10119 Acked Write 7 Done Sent Done false
35226 10120 Acked Write 10 Done Sent Done false
35227 10121 Acked Read 6 Done Sent Done false
35228 10122 Acked Read 7 Done Sent Done false
35229 10123 Acked Write 8 Done Sent Done false
35230 10124 Acked Read 1 Done Sent Done false
35231 10125 Acked Read 6 Done Sent Done false
35232 10126 Acked Read 5 Done Sent Done false
35233 10127 Acked Read 9 Done Sent Done false
35234 10128 Acked Write 9 Done Sent Done false
35235 10129 Acked Write 3 Done Sent Done false
35236 10130 Acked Write 2 Done Sent Done false
35237 10131 Acked Read 8 Done Sent Done false
35238 10132 Acked Read 8 Done Sent Done false
35239 10133 Acked Read 5 Done Sent Done false
35240 10134 Acked Read 3 Done Sent Done false
35241 10135 Acked Read 6 Done Sent Done false
35242 10136 Acked Read 6 Done Sent Done false
35243 10137 Acked Read 2 Done Sent Done false
35244 10138 Acked Read 10 Done Sent Done false
35245 10139 Acked Flush 0 Done Sent Done false
35246 10140 Acked Read 6 Done Sent Done false
35247 10141 Acked Write 2 Done Sent Done false
35248 10142 Acked Flush 0 Done Sent Done false
35249 10143 Acked Read 2 Done Sent Done false
35250 10144 Acked Read 8 Done Sent Done false
35251 10145 Acked Write 7 Done Sent Done false
35252 10146 Acked Write 6 Done Sent Done false
35253 10147 Acked Write 4 Done Sent Done false
35254 10148 Acked Write 3 Done Sent Done false
35255 10149 Acked Write 9 Done Sent Done false
35256 10150 Acked Read 1 Done Sent Done false
35257 10151 Acked Read 2 Done Sent Done false
35258 10152 Acked Write 1 Done Sent Done false
35259 10153 Acked Flush 0 Done Sent Done false
35260 10154 Acked Flush 0 Done Sent Done false
35261 10155 Acked Write 8 Done Sent Done false
35262 10156 Acked Write 4 Done Sent Done false
35263 10157 Acked Read 1 Done Sent Done false
35264 10158 Acked Write 7 Done Sent Done false
35265 10159 Acked Write 5 Done Sent Done false
35266 10160 Acked Read 9 Done Sent Done false
35267 10161 Acked Flush 0 Done Sent Done false
35268 10162 Acked Read 1 Done Sent Done false
35269 10163 Acked Write 5 Done Sent Done false
35270 10164 Acked Read 2 Done Sent Done false
35271 10165 Acked Write 7 Done Sent Done false
35272 10166 Acked Flush 0 Done Sent Done false
35273 10167 Acked Read 9 Done Sent Done false
35274 10168 Acked Read 10 Done Sent Done false
35275 10169 Acked Read 10 Done Sent Done false
35276 10170 Acked Flush 0 Done Sent Done false
35277 10171 Acked Write 4 Done Sent Done false
35278 10172 Acked Write 10 Done Sent Done false
35279 10173 Acked Write 10 Done Sent Done false
35280 10174 Acked Read 3 Done Sent Done false
35281 10175 Acked Write 2 Done Sent Done false
35282 10176 Acked Flush 0 Done Sent Done false
35283 10177 Acked Read 6 Done Sent Done false
35284 10178 Acked Flush 0 Done Sent Done false
35285 10179 Acked Write 2 Done Sent Done false
35286 10180 Acked Read 7 Done Sent Done false
35287 10181 Acked Read 10 Done Sent Done false
35288 10182 Acked Flush 0 Done Sent Done false
35289 10183 Acked Write 1 Done Sent Done false
35290 10184 Acked Flush 0 Done Sent Done false
35291 10185 Acked Write 6 Done Sent Done false
35292 10186 Acked Read 9 Done Sent Done false
35293 10187 Acked Read 3 Done Sent Done false
35294 10188 Acked Read 5 Done Sent Done false
35295 10189 Acked Read 5 Done Sent Done false
35296 10190 Acked Read 7 Done Sent Done false
35297 10191 Acked Write 7 Done Sent Done false
35298 10192 Acked Read 1 Done Sent Done false
35299 10193 Acked Read 10 Done Sent Done false
35300 10194 Acked Flush 0 Done Sent Done false
35301 10195 Acked Write 5 Done Sent Done false
35302 10196 Acked Read 9 Done Sent Done false
35303 10197 Acked Write 3 Done Sent Done false
35304 10198 Acked Read 6 Done Sent Done false
35305 10199 Acked Read 5 Done Sent Done false
35306 10200 Acked Read 9 Done Sent Done false
35307 10201 Acked Read 2 Done Sent Done false
35308 10202 Acked Write 7 Done Sent Done false
35309 10203 Acked Write 4 Done Sent Done false
35310 10204 Acked Read 7 Done Sent Done false
35311 10205 Acked Read 4 Done Sent Done false
35312 10206 Acked Write 8 Done Sent Done false
35313 10207 Acked Read 6 Done Sent Done false
35314 10208 Acked Read 1 Done Sent Done false
35315 10209 Acked Flush 0 Done Sent Done false
35316 10210 Acked Write 6 Done Sent Done false
35317 10211 Acked Write 1 Done Sent Done false
35318 10212 Acked Write 2 Done Sent Done false
35319 10213 Acked Read 3 Done Sent Done false
35320 10214 Acked Write 5 Done Sent Done false
35321 10215 Acked Read 5 Done Sent Done false
35322 10216 Acked Flush 0 Done Sent Done false
35323 10217 Acked Read 5 Done Sent Done false
35324 10218 Acked Flush 0 Done Sent Done false
35325 10219 Acked Write 4 Done Sent Done false
35326 10220 Acked Write 8 Done Sent Done false
35327 10221 Acked Write 4 Done Sent Done false
35328 10222 Acked Read 3 Done Sent Done false
35329 10223 Acked Flush 0 Done Sent Done false
35330 10224 Acked Write 1 Done Sent Done false
35331 10225 Acked Read 7 Done Sent Done false
35332 10226 Acked Read 7 Done Sent Done false
35333 10227 Acked Read 3 Done Sent Done false
35334 10228 Acked Write 7 Done Sent Done false
35335 10229 Acked Read 2 Done Sent Done false
35336 10230 Acked Read 7 Done Sent Done false
35337 10231 Acked Read 6 Done Sent Done false
35338 10232 Acked Read 6 Done Sent Done false
35339 10233 Acked Read 8 Done Sent Done false
35340 10234 Acked Write 3 Done Sent Done false
35341 10235 Acked Flush 0 Done Sent Done false
35342 10236 Acked Read 7 Done Sent Done false
35343 10237 Acked Read 6 Done Sent Done false
35344 10238 Acked Write 2 Done Sent Done false
35345 10239 Acked Write 2 Done Sent Done false
35346 10240 Acked Read 3 Done Sent Done false
35347 10241 Acked Read 8 Done Sent Done false
35348 10242 Acked Read 10 Done Sent Done false
35349 10243 Acked Read 6 Done Sent Done false
35350 10244 Acked Write 7 Done Sent Done false
35351 10245 Acked Write 8 Done Sent Done false
35352 10246 Acked Write 8 Done Sent Done false
35353 10247 Acked Read 1 Done Sent Done false
35354 10248 Acked Read 10 Done Sent Done false
35355 10249 Acked Write 8 Done Sent Done false
35356 10250 Acked Write 2 Done Sent Done false
35357 10251 Acked Flush 0 Done Sent Done false
35358 10252 Acked Read 9 Done Sent Done false
35359 10253 Acked Read 8 Done Sent Done false
35360 10254 Acked Write 3 Done Sent Done false
35361 10255 Acked Read 9 Done Sent Done false
35362 10256 Acked Read 4 Done Sent Done false
35363 10257 Acked Flush 0 Done Sent Done false
35364 10258 Acked Read 8 Done Sent Done false
35365 10259 Acked Read 9 Done Sent Done false
35366 10260 Acked Write 9 Done Sent Done false
35367 10261 Acked Read 1 Done Sent Done false
35368 10262 Acked Write 7 Done Sent Done false
35369 10263 Acked Write 6 Done Sent Done false
35370 10264 Acked Write 1 Done Sent Done false
35371 10265 Acked Write 5 Done Sent Done false
35372 10266 Acked Flush 0 Done Sent Done false
35373 10267 Acked Write 7 Done Sent Done false
35374 10268 Acked Write 4 Done Sent Done false
35375 10269 Acked Write 2 Done Sent Done false
35376 10270 Acked Flush 0 Done Sent Done false
35377 10271 Acked Write 4 Done Sent Done false
35378 10272 Acked Write 4 Done Sent Done false
35379 10273 Acked Read 5 Done Sent Done false
35380 10274 Acked Read 2 Done Sent Done false
35381 10275 Acked Read 4 Done Sent Done false
35382 10276 Acked Read 4 Done Sent Done false
35383 10277 Acked Read 7 Done Sent Done false
35384 10278 Acked Write 3 Done Sent Done false
35385 10279 Acked Write 10 Done Sent Done false
35386 10280 Acked Flush 0 Done Sent Done false
35387 10281 Acked Read 5 Done Sent Done false
35388 10282 Acked Read 10 Done Sent Done false
35389 10283 Acked Flush 0 Done Sent Done false
35390 10284 Acked Flush 0 Done Sent Done false
35391 10285 Acked Read 3 Done Sent Done false
35392 10286 Acked Write 8 Done Sent Done false
35393 10287 Acked Read 4 Done Sent Done false
35394 10288 Acked Read 3 Done Sent Done false
35395 10289 Acked Flush 0 Done Sent Done false
35396 10290 Acked Write 6 Done Sent Done false
35397 10291 Acked Read 2 Done Sent Done false
35398 10292 Acked Read 3 Done Sent Done false
35399 10293 Acked Write 4 Done Sent Done false
35400 10294 Acked Read 1 Done Sent Done false
35401 10295 Acked Write 7 Done Sent Done false
35402 10296 Acked Read 10 Done Sent Done false
35403 10297 Acked Write 5 Done Sent Done false
35404 10298 Acked Write 9 Done Sent Done false
35405 10299 Acked Read 9 Done Sent Done false
35406 10300 Acked Write 10 Done Sent Done false
35407 10301 Acked Write 7 Done Sent Done false
35408 10302 Acked Write 9 Done Sent Done false
35409 10303 Acked Write 9 Done Sent Done false
35410 10304 Acked Write 1 Done Sent Done false
35411 10305 Acked Write 2 Done Sent Done false
35412 10306 Acked Write 10 Done Sent Done false
35413 10307 Acked Flush 0 Done Sent Done false
35414 10308 Acked Write 3 Done Sent Done false
35415 10309 Acked Write 10 Done Sent Done false
35416 10310 Acked Write 7 Done Sent Done false
35417 10311 Acked Write 8 Done Sent Done false
35418 10312 Acked Read 1 Done Sent Done false
35419 10313 Acked Read 6 Done Sent Done false
35420 10314 Acked Write 7 Done Sent Done false
35421 10315 Acked Read 9 Done Sent Done false
35422 10316 Acked Write 2 Done Sent Done false
35423 10317 Acked Read 10 Done Sent Done false
35424 10318 Acked Read 8 Done Sent Done false
35425 10319 Acked Flush 0 Done Sent Done false
35426 10320 Acked Read 5 Done Sent Done false
35427 10321 Acked Write 7 Done Sent Done false
35428 10322 Acked Write 3 Done Sent Done false
35429 10323 Acked Read 9 Done Sent Done false
35430 10324 Acked Read 1 Done Sent Done false
35431 10325 Acked Write 5 Done Sent Done false
35432 10326 Acked Flush 0 Done Sent Done false
35433 10327 Acked Flush 0 Done Sent Done false
35434 10328 Acked Write 1 Done Sent Done false
35435 10329 Acked Write 2 Done Sent Done false
35436 10330 Acked Read 7 Done Sent Done false
35437 10331 Acked Write 5 Done Sent Done false
35438 10332 Acked Write 1 Done Sent Done false
35439 10333 Acked Read 9 Done Sent Done false
35440 10334 Acked Write 5 Done Sent Done false
35441 10335 Acked Flush 0 Done Sent Done false
35442 10336 Acked Write 10 Done Sent Done false
35443 10337 Acked Read 7 Done Sent Done false
35444 10338 Acked Write 8 Done Sent Done false
35445 10339 Acked Write 5 Done Sent Done false
35446 10340 Acked Read 1 Done Sent Done false
35447 10341 Acked Read 7 Done Sent Done false
35448 10342 Acked Flush 0 Done Sent Done false
35449 10343 Acked Read 10 Done Sent Done false
35450 10344 Acked Write 2 Done Sent Done false
35451 10345 Acked Write 2 Done Sent Done false
35452 10346 Acked Read 6 Done Sent Done false
35453 10347 Acked Write 5 Done Sent Done false
35454 10348 Acked Write 6 Done Sent Done false
35455 10349 Acked Write 4 Done Sent Done false
35456 10350 Acked Write 1 Done Sent Done false
35457 10351 Acked Write 5 Done Sent Done false
35458 10352 Acked Read 8 Done Sent Done false
35459 10353 Acked Read 9 Done Sent Done false
35460 10354 Acked Write 2 Done Sent Done false
35461 10355 Acked Read 7 Done Sent Done false
35462 10356 Acked Write 1 Done Sent Done false
35463 10357 Acked Flush 0 Done Sent Done false
35464 10358 Acked Read 6 Done Sent Done false
35465 STATES DS:0 DS:1 DS:2 TOTAL
35466 Sent 0 301 0 301
35467 Done 301 0 301 602
35468 Skipped 0 0 0 0
35469 Error 0 0 0 0
35470 Last Flush: 10357 10057 10357
35471 Downstairs last five completed: 10057 10056 10055 10054 10053
35472 Upstairs last five completed: 10358 10357 10356 10355 10354
35473 CLIENT: Up:0 ds:301 act:2
35474 [8820] got start action
35475 [8820] I am going to restart
35476 Starting downstairs at port 8820
35477 Make output file at "/tmp/dsc/downstairs-8820.txt"
35478 [8820][1] reports Starting
35479 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1236)
35480 [8820][1] reports Running
35481 ----------------------------------------------------------------
35482 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
35483 States: Active Offline Active
35484 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35485 10058 Acked Write 3 Done Sent Done false
35486 10059 Acked Read 8 Done Sent Done false
35487 10060 Acked Write 8 Done Sent Done false
35488 10061 Acked Write 10 Done Sent Done false
35489 10062 Acked Read 5 Done Sent Done false
35490 10063 Acked Read 2 Done Sent Done false
35491 10064 Acked Write 4 Done Sent Done false
35492 10065 Acked Write 2 Done Sent Done false
35493 10066 Acked Write 6 Done Sent Done false
35494 10067 Acked Write 2 Done Sent Done false
35495 10068 Acked Flush 0 Done Sent Done false
35496 10069 Acked Write 3 Done Sent Done false
35497 10070 Acked Write 3 Done Sent Done false
35498 10071 Acked Write 4 Done Sent Done false
35499 10072 Acked Read 5 Done Sent Done false
35500 10073 Acked Read 5 Done Sent Done false
35501 10074 Acked Write 1 Done Sent Done false
35502 10075 Acked Write 1 Done Sent Done false
35503 10076 Acked Write 8 Done Sent Done false
35504 10077 Acked Read 3 Done Sent Done false
35505 10078 Acked Read 9 Done Sent Done false
35506 10079 Acked Read 5 Done Sent Done false
35507 10080 Acked Read 8 Done Sent Done false
35508 10081 Acked Flush 0 Done Sent Done false
35509 10082 Acked Read 6 Done Sent Done false
35510 10083 Acked Flush 0 Done Sent Done false
35511 10084 Acked Read 1 Done Sent Done false
35512 10085 Acked Read 1 Done Sent Done false
35513 10086 Acked Read 4 Done Sent Done false
35514 10087 Acked Write 1 Done Sent Done false
35515 10088 Acked Read 3 Done Sent Done false
35516 10089 Acked Read 9 Done Sent Done false
35517 10090 Acked Write 2 Done Sent Done false
35518 10091 Acked Read 2 Done Sent Done false
35519 10092 Acked Read 6 Done Sent Done false
35520 10093 Acked Read 2 Done Sent Done false
35521 10094 Acked Write 2 Done Sent Done false
35522 10095 Acked Write 9 Done Sent Done false
35523 10096 Acked Read 1 Done Sent Done false
35524 10097 Acked Read 3 Done Sent Done false
35525 10098 Acked Write 5 Done Sent Done false
35526 10099 Acked Write 9 Done Sent Done false
35527 10100 Acked Write 2 Done Sent Done false
35528 10101 Acked Read 3 Done Sent Done false
35529 10102 Acked Write 6 Done Sent Done false
35530 10103 Acked Read 6 Done Sent Done false
35531 10104 Acked Read 9 Done Sent Done false
35532 10105 Acked Flush 0 Done Sent Done false
35533 10106 Acked Read 6 Done Sent Done false
35534 10107 Acked Write 7 Done Sent Done false
35535 10108 Acked Read 9 Done Sent Done false
35536 10109 Acked Read 8 Done Sent Done false
35537 10110 Acked Flush 0 Done Sent Done false
35538 10111 Acked Read 5 Done Sent Done false
35539 10112 Acked Read 9 Done Sent Done false
35540 10113 Acked Read 6 Done Sent Done false
35541 10114 Acked Read 7 Done Sent Done false
35542 10115 Acked Flush 0 Done Sent Done false
35543 10116 Acked Write 8 Done Sent Done false
35544 10117 Acked Write 4 Done Sent Done false
35545 10118 Acked Write 6 Done Sent Done false
35546 10119 Acked Write 7 Done Sent Done false
35547 10120 Acked Write 10 Done Sent Done false
35548 10121 Acked Read 6 Done Sent Done false
35549 10122 Acked Read 7 Done Sent Done false
35550 10123 Acked Write 8 Done Sent Done false
35551 10124 Acked Read 1 Done Sent Done false
35552 10125 Acked Read 6 Done Sent Done false
35553 10126 Acked Read 5 Done Sent Done false
35554 10127 Acked Read 9 Done Sent Done false
35555 10128 Acked Write 9 Done Sent Done false
35556 10129 Acked Write 3 Done Sent Done false
35557 10130 Acked Write 2 Done Sent Done false
35558 10131 Acked Read 8 Done Sent Done false
35559 10132 Acked Read 8 Done Sent Done false
35560 10133 Acked Read 5 Done Sent Done false
35561 10134 Acked Read 3 Done Sent Done false
35562 10135 Acked Read 6 Done Sent Done false
35563 10136 Acked Read 6 Done Sent Done false
35564 10137 Acked Read 2 Done Sent Done false
35565 10138 Acked Read 10 Done Sent Done false
35566 10139 Acked Flush 0 Done Sent Done false
35567 10140 Acked Read 6 Done Sent Done false
35568 10141 Acked Write 2 Done Sent Done false
35569 10142 Acked Flush 0 Done Sent Done false
35570 10143 Acked Read 2 Done Sent Done false
35571 10144 Acked Read 8 Done Sent Done false
35572 10145 Acked Write 7 Done Sent Done false
35573 10146 Acked Write 6 Done Sent Done false
35574 10147 Acked Write 4 Done Sent Done false
35575 10148 Acked Write 3 Done Sent Done false
35576 10149 Acked Write 9 Done Sent Done false
35577 10150 Acked Read 1 Done Sent Done false
35578 10151 Acked Read 2 Done Sent Done false
35579 10152 Acked Write 1 Done Sent Done false
35580 10153 Acked Flush 0 Done Sent Done false
35581 10154 Acked Flush 0 Done Sent Done false
35582 10155 Acked Write 8 Done Sent Done false
35583 10156 Acked Write 4 Done Sent Done false
35584 10157 Acked Read 1 Done Sent Done false
35585 10158 Acked Write 7 Done Sent Done false
35586 10159 Acked Write 5 Done Sent Done false
35587 10160 Acked Read 9 Done Sent Done false
35588 10161 Acked Flush 0 Done Sent Done false
35589 10162 Acked Read 1 Done Sent Done false
35590 10163 Acked Write 5 Done Sent Done false
35591 10164 Acked Read 2 Done Sent Done false
35592 10165 Acked Write 7 Done Sent Done false
35593 10166 Acked Flush 0 Done Sent Done false
35594 10167 Acked Read 9 Done Sent Done false
35595 10168 Acked Read 10 Done Sent Done false
35596 10169 Acked Read 10 Done Sent Done false
35597 10170 Acked Flush 0 Done Sent Done false
35598 10171 Acked Write 4 Done Sent Done false
35599 10172 Acked Write 10 Done Sent Done false
35600 10173 Acked Write 10 Done Sent Done false
35601 10174 Acked Read 3 Done Sent Done false
35602 10175 Acked Write 2 Done Sent Done false
35603 10176 Acked Flush 0 Done Sent Done false
35604 10177 Acked Read 6 Done Sent Done false
35605 10178 Acked Flush 0 Done Sent Done false
35606 10179 Acked Write 2 Done Sent Done false
35607 10180 Acked Read 7 Done Sent Done false
35608 10181 Acked Read 10 Done Sent Done false
35609 10182 Acked Flush 0 Done Sent Done false
35610 10183 Acked Write 1 Done Sent Done false
35611 10184 Acked Flush 0 Done Sent Done false
35612 10185 Acked Write 6 Done Sent Done false
35613 10186 Acked Read 9 Done Sent Done false
35614 10187 Acked Read 3 Done Sent Done false
35615 10188 Acked Read 5 Done Sent Done false
35616 10189 Acked Read 5 Done Sent Done false
35617 10190 Acked Read 7 Done Sent Done false
35618 10191 Acked Write 7 Done Sent Done false
35619 10192 Acked Read 1 Done Sent Done false
35620 10193 Acked Read 10 Done Sent Done false
35621 10194 Acked Flush 0 Done Sent Done false
35622 10195 Acked Write 5 Done Sent Done false
35623 10196 Acked Read 9 Done Sent Done false
35624 10197 Acked Write 3 Done Sent Done false
35625 10198 Acked Read 6 Done Sent Done false
35626 10199 Acked Read 5 Done Sent Done false
35627 10200 Acked Read 9 Done Sent Done false
35628 10201 Acked Read 2 Done Sent Done false
35629 10202 Acked Write 7 Done Sent Done false
35630 10203 Acked Write 4 Done Sent Done false
35631 10204 Acked Read 7 Done Sent Done false
35632 10205 Acked Read 4 Done Sent Done false
35633 10206 Acked Write 8 Done Sent Done false
35634 10207 Acked Read 6 Done Sent Done false
35635 10208 Acked Read 1 Done Sent Done false
35636 10209 Acked Flush 0 Done Sent Done false
35637 10210 Acked Write 6 Done Sent Done false
35638 10211 Acked Write 1 Done Sent Done false
35639 10212 Acked Write 2 Done Sent Done false
35640 10213 Acked Read 3 Done Sent Done false
35641 10214 Acked Write 5 Done Sent Done false
35642 10215 Acked Read 5 Done Sent Done false
35643 10216 Acked Flush 0 Done Sent Done false
35644 10217 Acked Read 5 Done Sent Done false
35645 10218 Acked Flush 0 Done Sent Done false
35646 10219 Acked Write 4 Done Sent Done false
35647 10220 Acked Write 8 Done Sent Done false
35648 10221 Acked Write 4 Done Sent Done false
35649 10222 Acked Read 3 Done Sent Done false
35650 10223 Acked Flush 0 Done Sent Done false
35651 10224 Acked Write 1 Done Sent Done false
35652 10225 Acked Read 7 Done Sent Done false
35653 10226 Acked Read 7 Done Sent Done false
35654 10227 Acked Read 3 Done Sent Done false
35655 10228 Acked Write 7 Done Sent Done false
35656 10229 Acked Read 2 Done Sent Done false
35657 10230 Acked Read 7 Done Sent Done false
35658 10231 Acked Read 6 Done Sent Done false
35659 10232 Acked Read 6 Done Sent Done false
35660 10233 Acked Read 8 Done Sent Done false
35661 10234 Acked Write 3 Done Sent Done false
35662 10235 Acked Flush 0 Done Sent Done false
35663 10236 Acked Read 7 Done Sent Done false
35664 10237 Acked Read 6 Done Sent Done false
35665 10238 Acked Write 2 Done Sent Done false
35666 10239 Acked Write 2 Done Sent Done false
35667 10240 Acked Read 3 Done Sent Done false
35668 10241 Acked Read 8 Done Sent Done false
35669 10242 Acked Read 10 Done Sent Done false
35670 10243 Acked Read 6 Done Sent Done false
35671 10244 Acked Write 7 Done Sent Done false
35672 10245 Acked Write 8 Done Sent Done false
35673 10246 Acked Write 8 Done Sent Done false
35674 10247 Acked Read 1 Done Sent Done false
35675 10248 Acked Read 10 Done Sent Done false
35676 10249 Acked Write 8 Done Sent Done false
35677 10250 Acked Write 2 Done Sent Done false
35678 10251 Acked Flush 0 Done Sent Done false
35679 10252 Acked Read 9 Done Sent Done false
35680 10253 Acked Read 8 Done Sent Done false
35681 10254 Acked Write 3 Done Sent Done false
35682 10255 Acked Read 9 Done Sent Done false
35683 10256 Acked Read 4 Done Sent Done false
35684 10257 Acked Flush 0 Done Sent Done false
35685 10258 Acked Read 8 Done Sent Done false
35686 10259 Acked Read 9 Done Sent Done false
35687 10260 Acked Write 9 Done Sent Done false
35688 10261 Acked Read 1 Done Sent Done false
35689 10262 Acked Write 7 Done Sent Done false
35690 10263 Acked Write 6 Done Sent Done false
35691 10264 Acked Write 1 Done Sent Done false
35692 10265 Acked Write 5 Done Sent Done false
35693 10266 Acked Flush 0 Done Sent Done false
35694 10267 Acked Write 7 Done Sent Done false
35695 10268 Acked Write 4 Done Sent Done false
35696 10269 Acked Write 2 Done Sent Done false
35697 10270 Acked Flush 0 Done Sent Done false
35698 10271 Acked Write 4 Done Sent Done false
35699 10272 Acked Write 4 Done Sent Done false
35700 10273 Acked Read 5 Done Sent Done false
35701 10274 Acked Read 2 Done Sent Done false
35702 10275 Acked Read 4 Done Sent Done false
35703 10276 Acked Read 4 Done Sent Done false
35704 10277 Acked Read 7 Done Sent Done false
35705 10278 Acked Write 3 Done Sent Done false
35706 10279 Acked Write 10 Done Sent Done false
35707 10280 Acked Flush 0 Done Sent Done false
35708 10281 Acked Read 5 Done Sent Done false
35709 10282 Acked Read 10 Done Sent Done false
35710 10283 Acked Flush 0 Done Sent Done false
35711 10284 Acked Flush 0 Done Sent Done false
35712 10285 Acked Read 3 Done Sent Done false
35713 10286 Acked Write 8 Done Sent Done false
35714 10287 Acked Read 4 Done Sent Done false
35715 10288 Acked Read 3 Done Sent Done false
35716 10289 Acked Flush 0 Done Sent Done false
35717 10290 Acked Write 6 Done Sent Done false
35718 10291 Acked Read 2 Done Sent Done false
35719 10292 Acked Read 3 Done Sent Done false
35720 10293 Acked Write 4 Done Sent Done false
35721 10294 Acked Read 1 Done Sent Done false
35722 10295 Acked Write 7 Done Sent Done false
35723 10296 Acked Read 10 Done Sent Done false
35724 10297 Acked Write 5 Done Sent Done false
35725 10298 Acked Write 9 Done Sent Done false
35726 10299 Acked Read 9 Done Sent Done false
35727 10300 Acked Write 10 Done Sent Done false
35728 10301 Acked Write 7 Done Sent Done false
35729 10302 Acked Write 9 Done Sent Done false
35730 10303 Acked Write 9 Done Sent Done false
35731 10304 Acked Write 1 Done Sent Done false
35732 10305 Acked Write 2 Done Sent Done false
35733 10306 Acked Write 10 Done Sent Done false
35734 10307 Acked Flush 0 Done Sent Done false
35735 10308 Acked Write 3 Done Sent Done false
35736 10309 Acked Write 10 Done Sent Done false
35737 10310 Acked Write 7 Done Sent Done false
35738 10311 Acked Write 8 Done Sent Done false
35739 10312 Acked Read 1 Done Sent Done false
35740 10313 Acked Read 6 Done Sent Done false
35741 10314 Acked Write 7 Done Sent Done false
35742 10315 Acked Read 9 Done Sent Done false
35743 10316 Acked Write 2 Done Sent Done false
35744 10317 Acked Read 10 Done Sent Done false
35745 10318 Acked Read 8 Done Sent Done false
35746 10319 Acked Flush 0 Done Sent Done false
35747 10320 Acked Read 5 Done Sent Done false
35748 10321 Acked Write 7 Done Sent Done false
35749 10322 Acked Write 3 Done Sent Done false
35750 10323 Acked Read 9 Done Sent Done false
35751 10324 Acked Read 1 Done Sent Done false
35752 10325 Acked Write 5 Done Sent Done false
35753 10326 Acked Flush 0 Done Sent Done false
35754 10327 Acked Flush 0 Done Sent Done false
35755 10328 Acked Write 1 Done Sent Done false
35756 10329 Acked Write 2 Done Sent Done false
35757 10330 Acked Read 7 Done Sent Done false
35758 10331 Acked Write 5 Done Sent Done false
35759 10332 Acked Write 1 Done Sent Done false
35760 10333 Acked Read 9 Done Sent Done false
35761 10334 Acked Write 5 Done Sent Done false
35762 10335 Acked Flush 0 Done Sent Done false
35763 10336 Acked Write 10 Done Sent Done false
35764 10337 Acked Read 7 Done Sent Done false
35765 10338 Acked Write 8 Done Sent Done false
35766 10339 Acked Write 5 Done Sent Done false
35767 10340 Acked Read 1 Done Sent Done false
35768 10341 Acked Read 7 Done Sent Done false
35769 10342 Acked Flush 0 Done Sent Done false
35770 10343 Acked Read 10 Done Sent Done false
35771 10344 Acked Write 2 Done Sent Done false
35772 10345 Acked Write 2 Done Sent Done false
35773 10346 Acked Read 6 Done Sent Done false
35774 10347 Acked Write 5 Done Sent Done false
35775 10348 Acked Write 6 Done Sent Done false
35776 10349 Acked Write 4 Done Sent Done false
35777 10350 Acked Write 1 Done Sent Done false
35778 10351 Acked Write 5 Done Sent Done false
35779 10352 Acked Read 8 Done Sent Done false
35780 10353 Acked Read 9 Done Sent Done false
35781 10354 Acked Write 2 Done Sent Done false
35782 10355 Acked Read 7 Done Sent Done false
35783 10356 Acked Write 1 Done Sent Done false
35784 10357 Acked Flush 0 Done Sent Done false
35785 10358 Acked Read 6 Done Sent Done false
35786 10359 Acked Flush 0 Done Sent Done false
35787 STATES DS:0 DS:1 DS:2 TOTAL
35788 Sent 0 302 0 302
35789 Done 302 0 302 604
35790 Skipped 0 0 0 0
35791 Error 0 0 0 0
35792 Last Flush: 10359 10057 10359
35793 Downstairs last five completed: 10057 10056 10055 10054 10053
35794 Upstairs last five completed: 10359 10358 10357 10356 10355
35795 CLIENT: Up:0 ds:302 act:2
357962025-01-03T08:38:23.126ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
357972025-01-03T08:38:23.126ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50881) = io task client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
357982025-01-03T08:38:23.127ZWARNcrucible: version negotiation from state Offline = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
357992025-01-03T08:38:23.128ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
358002025-01-03T08:38:23.128ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
358012025-01-03T08:38:23.128ZINFOcrucible: send last flush ID to this DS: 10057 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
358022025-01-03T08:38:23.128ZINFOcrucible: Replied this last flush ID: 10057 = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
358032025-01-03T08:38:23.128ZINFOcrucible: ds_transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
358042025-01-03T08:38:23.128ZINFOcrucible: [1] Transition from Offline to Active = downstairs client = 1 session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
358052025-01-03T08:38:23.128ZINFOcrucible: [1] client re-new 302 jobs since flush 10057 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
358062025-01-03T08:38:23.129ZINFOcrucible: [1] Replayed 302 jobs since flush: 10057 = downstairs session_id = 4d73cd98-4f0b-450f-bfa8-8a9ca6991800
35807 ----------------------------------------------------------------
35808 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35809 Downstairs last five completed: 10359 10358 10357 10356 10355
35810 Upstairs last five completed: 10359 10358 10357 10356 10355
35811 CLIENT: Up:0 ds:0 act:3
35812 Replay: All jobs finished
35813 Test replay has completed
35814 Wrote out file "/tmp/test_replay_verify.log"
35815 CLIENT: Tests done. All submitted work has been ACK'd
35816 ----------------------------------------------------------------
35817 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
35818 Downstairs last five completed: 10359 10358 10357 10356 10355
35819 Upstairs last five completed: 10359 10358 10357 10356 10355
35820 CLIENT: Up:0 ds:0 act:3
35821 CLIENT: All jobs finished, all DS active.
35822 Replays:30 time: 7:28
35823 Do final verify
35824 Jan 03 08:38:25.444 INFO accepted connection, remote_addr: 127.0.0.1:62599, local_addr: 127.0.0.1:9998
35825 Jan 03 08:38:25.444 INFO request completed, latency_us: 206, response_code: 200, uri: /regioninfo, method: GET, req_id: 9bbf85c1-494c-46e2-9f42-65af65aa2485, remote_addr: 127.0.0.1:62599, local_addr: 127.0.0.1:9998
358262025-01-03T08:38:25.445ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
35827 Jan 03 08:38:25.446 INFO request completed, latency_us: 127, response_code: 200, uri: /regioncount, method: GET, req_id: 4b7d3700-d16d-4e98-90ce-ad9c6b0680a9, remote_addr: 127.0.0.1:62599, local_addr: 127.0.0.1:9998
358282025-01-03T08:38:25.446ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
35829 Jan 03 08:38:25.446 INFO request completed, latency_us: 137, response_code: 200, uri: /port/cid/0, method: GET, req_id: 203c4d8d-4c7d-421d-981f-fb31b693e3a0, remote_addr: 127.0.0.1:62599, local_addr: 127.0.0.1:9998
35830 Jan 03 08:38:25.447 INFO request completed, latency_us: 130, response_code: 200, uri: /port/cid/1, method: GET, req_id: e13ae828-07bb-467a-945a-5a64778cc730, remote_addr: 127.0.0.1:62599, local_addr: 127.0.0.1:9998
35831 Jan 03 08:38:25.448 INFO request completed, latency_us: 99, response_code: 200, uri: /port/cid/2, method: GET, req_id: 499038a3-2943-4215-9a58-743d8b5cf504, remote_addr: 127.0.0.1:62599, local_addr: 127.0.0.1:9998
358322025-01-03T08:38:25.448ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
358332025-01-03T08:38:25.448ZINFOcrucible: Upstairs starts
358342025-01-03T08:38:25.448ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "94736ad28ed33108b1c88fccc3caea693da35514", git_commit_timestamp: "2025-01-03T08:14:58.000000000Z", git_branch: "renovate/tempfile-3.x-lockfile", rustc_semver: "1.82.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "f6e511eec7342f59a25f7c0534f1dbea00d01b14", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
358352025-01-03T08:38:25.448ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
358362025-01-03T08:38:25.448ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
358372025-01-03T08:38:25.449ZINFOcrucible: Crucible 079d3998-657d-441e-a534-97aab3e34d05 has session id: c812c356-4a54-441f-97ec-b3b446f0ea43 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358382025-01-03T08:38:25.449ZINFOcrucible: Upstairs opts: Upstairs UUID: 079d3998-657d-441e-a534-97aab3e34d05, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358392025-01-03T08:38:25.449ZINFOcrucible: Crucible stats registered with UUID: 079d3998-657d-441e-a534-97aab3e34d05 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358402025-01-03T08:38:25.483ZINFOcrucible: Downstairs in volume = 3 task = crutest
358412025-01-03T08:38:25.483ZINFOcrucible: The guest has requested activation with gen:3
358422025-01-03T08:38:25.483ZINFOcrucible: 079d3998-657d-441e-a534-97aab3e34d05 active request set session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358432025-01-03T08:38:25.483ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358442025-01-03T08:38:25.483ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358452025-01-03T08:38:25.483ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358462025-01-03T08:38:25.483ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358472025-01-03T08:38:25.483ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358482025-01-03T08:38:25.483ZINFOcrucible: client set_active_request while in New; waiting... = downstairs client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358492025-01-03T08:38:25.483ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358502025-01-03T08:38:25.483ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358512025-01-03T08:38:25.483ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358522025-01-03T08:38:25.483ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41289) = io task client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358532025-01-03T08:38:25.483ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35200) = io task client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358542025-01-03T08:38:25.483ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58650) = io task client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358552025-01-03T08:38:25.484ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358562025-01-03T08:38:25.484ZINFOcrucible: [1] Transition from New to WaitActive = downstairs client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358572025-01-03T08:38:25.484ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358582025-01-03T08:38:25.484ZINFOcrucible: [2] Transition from New to WaitActive = downstairs client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358592025-01-03T08:38:25.484ZINFOcrucible: ds_transition from New to WaitActive = downstairs client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358602025-01-03T08:38:25.484ZINFOcrucible: [0] Transition from New to WaitActive = downstairs client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358612025-01-03T08:38:25.485ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358622025-01-03T08:38:25.485ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358632025-01-03T08:38:25.485ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358642025-01-03T08:38:25.486ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358652025-01-03T08:38:25.486ZINFOcrucible: [0] Transition from WaitActive to WaitQuorum = downstairs client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358662025-01-03T08:38:25.486ZINFOcrucible: Waiting for more clients to be ready session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358672025-01-03T08:38:25.486ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358682025-01-03T08:38:25.486ZINFOcrucible: [1] Transition from WaitActive to WaitQuorum = downstairs client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358692025-01-03T08:38:25.486ZINFOcrucible: Waiting for more clients to be ready session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358702025-01-03T08:38:25.486ZINFOcrucible: ds_transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358712025-01-03T08:38:25.486ZINFOcrucible: [2] Transition from WaitActive to WaitQuorum = downstairs client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358722025-01-03T08:38:25.486ZINFOcrucible: [0]R flush_numbers[0..12]: [913, 925, 896, 909, 919, 925, 911, 913, 925, 914, 888, 921] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358732025-01-03T08:38:25.486ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358742025-01-03T08:38:25.486ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358752025-01-03T08:38:25.486ZINFOcrucible: [1]R flush_numbers[0..12]: [913, 925, 896, 909, 919, 925, 911, 913, 925, 914, 888, 921] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358762025-01-03T08:38:25.486ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358772025-01-03T08:38:25.486ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358782025-01-03T08:38:25.486ZINFOcrucible: [2]R flush_numbers[0..12]: [913, 925, 896, 909, 919, 925, 911, 913, 925, 914, 888, 921] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358792025-01-03T08:38:25.486ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358802025-01-03T08:38:25.486ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358812025-01-03T08:38:25.486ZINFOcrucible: Max found gen is 3 = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358822025-01-03T08:38:25.486ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358832025-01-03T08:38:25.486ZINFOcrucible: Next flush: 926 = downstairs session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
35884 {"msg":"All extents match","v":0,"name":"crucible","level":30,"time":"Wait for a query_work_queue command to finish before sending IO
35885 2025-01-03T08:38:25.486395491Z","hostname":"w-01JGNMHA0PYC9VDZ90ECYGD6YD","pid":1241,"":"downstairs","":"downstairs","session_id":"c812c356-4a54-441f-97ec-b3b446f0ea43"}
358862025-01-03T08:38:25.486ZINFOcrucible: No downstairs reconciliation required session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358872025-01-03T08:38:25.486ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358882025-01-03T08:38:25.486ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358892025-01-03T08:38:25.486ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358902025-01-03T08:38:25.486ZINFOcrucible: All required reconciliation work is completed session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358912025-01-03T08:38:25.486ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358922025-01-03T08:38:25.486ZINFOcrucible: 079d3998-657d-441e-a534-97aab3e34d05 is now active with session: c812c356-4a54-441f-97ec-b3b446f0ea43 session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358932025-01-03T08:38:25.486ZINFOcrucible: Set Active after no reconciliation session_id = c812c356-4a54-441f-97ec-b3b446f0ea43
358942025-01-03T08:38:25.486ZINFOcrucible: The guest has finished waiting for activation with:3
35895 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
35896 Loading write count information from file "/tmp/test_replay_verify.log"
35897 Read and Verify all blocks (0..5000 range:false)
35898 Wrote out file "/tmp/test_replay_verify.log"
35899 Verify test completed
35900 Wrote out file "/tmp/test_replay_verify.log"
35901 CLIENT: Tests done. All submitted work has been ACK'd
35902 ----------------------------------------------------------------
35903 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
35904 States: Active Active Active
35905 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
35906 1000 Acked Read 256 Done Done Done false
35907 1001 Acked Read 256 Done Done Done false
35908 1002 Acked Read 256 Done Done Done false
35909 1003 Acked Read 256 Done Done Done false
35910 1004 Acked Read 256 Done Done Done false
35911 1005 Acked Read 256 Done Done Done false
35912 1006 Acked Read 256 Done Done Done false
35913 1007 Acked Read 256 Done Done Done false
35914 1008 Acked Read 256 Done Done Done false
35915 1009 Acked Read 256 Done Done Done false
35916 1010 Acked Read 256 Done Done Done false
35917 1011 Acked Read 256 Done Done Done false
35918 1012 Acked Read 256 Done Done Done false
35919 1013 Acked Read 256 Done Done Done false
35920 1014 Acked Read 256 Done Done Done false
35921 1015 Acked Read 256 Done Done Done false
35922 1016 Acked Read 136 Done Done Done false
35923 1017 Acked Read 256 Done Done Done false
35924 1018 Acked Read 256 Done Done Done false
35925 1019 Acked Read 256 Done Done Done false
35926 STATES DS:0 DS:1 DS:2 TOTAL
35927 Sent 0 0 0 0
35928 Done 20 20 20 60
35929 Skipped 0 0 0 0
35930 Error 0 0 0 0
35931 Last Flush: 0 0 0
35932 Downstairs last five completed:
35933 Upstairs last five completed: 1019 1018 1017 1016 1015
35934 CLIENT: Up:0 ds:20 act:3
35935 ----------------------------------------------------------------
35936 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
35937 Downstairs last five completed: 1020 1019 1018 1017 1016
35938 Upstairs last five completed: 1020 1019 1018 1017 1016
35939 CLIENT: Up:0 ds:0 act:3
35940 CLIENT: All crucible jobs finished, exiting program
359412025-01-03T08:38:29.900ZWARNcrucible: Guest handle has been dropped
35942 Jan 03 08:38:29.973 INFO accepted connection, remote_addr: 127.0.0.1:42802, local_addr: 127.0.0.1:9998
35943 Main task has work to do, go find it
35944 got dsc Shutdown
35945 Shutdown
35946 Jan 03 08:38:29.974 INFO request completed, latency_us: 160, response_code: Shut it down
35947 204, uri: /shutdown, [8810] Disable keep_running
35948 method[8810] Got stop action so:false kr:false
35949 : POST, req_id: b9638431-cb23-4660-8cf2-c5210e2e4845, remote_addr: 127.0.0.1:42802, local_addr: 127.0.0.1:9998
35950 [8820] Disable keep_running
35951 [8820] Got stop action so:false kr:false
35952 [8830] Disable keep_running
35953 [8830] Got stop action so:false kr:false
35954 [8820] Exited with: ExitStatus(unix_wait_status(9))
35955 [8830] Exited with: ExitStatus(unix_wait_status(9))
35956 [8820][1] reports Exit
35957 [8830][2] reports Exit
35958 [8810] Exited with: ExitStatus(unix_wait_status(9))
35959 [8810][0] reports Exit
35960 January 3, 2025 at 08:38:34 AM UTC Test ends with 0