Max level shown:
1 starting January 21, 2025 at 03:47:03 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.32298616
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.020784952
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.022722982
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start ds: 8830
15 start access at:127.0.0.1:9998
16 Starting downstairs at port 8820
17 Starting downstairs at port 8810
18 Starting downstairs at port 8830
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Make output file at "/tmp/dsc/downstairs-8830.txt"
21 Make output file at "/tmp/dsc/downstairs-8810.txt"
22 [8820][1] initial start wait reports Starting
23 [8810][0] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1190)
26 [8820][1] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1191)
28 [8830][2] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1192)
30 [8810][0] initial start wait reports Running
31 All downstairs are running
32 Jan 21 15:47:03.472 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Jan 21 15:47:08.576 INFO accepted connection, remote_addr: 127.0.0.1:45366, local_addr: 127.0.0.1:9998
36 Jan 21 15:47:08.578 INFO request completed, latency_us: 420, response_code: 200, uri: /regioninfo, method: GET, req_id: 3ff3d3e5-87d1-4784-98cc-3e94ffcfd2ce, remote_addr: 127.0.0.1:45366, local_addr: 127.0.0.1:9998
37 Jan 21 15:47:08.580 INFO request completed, latency_us: 126, response_code: 200, uri: /regioncount, method: GET, req_id: 5a3019b3-48c4-4695-893f-0a546853cf7c, remote_addr: 127.0.0.1:45366, local_addr: 127.0.0.1:9998
382025-01-21T15:47:08.580ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392025-01-21T15:47:08.581ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Jan 21 15:47:08.581 INFO request completed, latency_us: 162, response_code: 200, uri: /port/cid/0, method: GET, req_id: df30803c-4c06-4f16-8e4c-27cb289a20f9, remote_addr: 127.0.0.1:45366, local_addr: 127.0.0.1:9998
41 Jan 21 15:47:08.582 INFO request completed, latency_us: 106, response_code: 200, uri: /port/cid/1, method: GET, req_id: 6c736b1d-7e52-4a73-a3fe-1db341e12541, remote_addr: 127.0.0.1:45366, local_addr: 127.0.0.1:9998
42 Jan 21 15:47:08.582 INFO request completed, latency_us: 104, response_code: 200, uri: /port/cid/2, method: GET, req_id: 2e7fc7fe-51c5-4903-941a-24deae8a3d88, remote_addr: 127.0.0.1:45366, local_addr: 127.0.0.1:9998
432025-01-21T15:47:08.583ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442025-01-21T15:47:08.584ZINFOcrucible: Upstairs starts
452025-01-21T15:47:08.584ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "5c2b227300b758c153f33da870809d67a5160dae", git_commit_timestamp: "2025-01-21T15:31:20.000000000Z", git_branch: "mkeeter/always-check-live-repair", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462025-01-21T15:47:08.584ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472025-01-21T15:47:08.584ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482025-01-21T15:47:08.586ZINFOcrucible: Crucible 5f22cdcf-4871-4f4d-ae45-5a27f568c3d6 has session id: c92739a0-b5de-4795-a55c-7b3ca4ca2347 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
492025-01-21T15:47:08.586ZINFOcrucible: Upstairs opts: Upstairs UUID: 5f22cdcf-4871-4f4d-ae45-5a27f568c3d6, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
502025-01-21T15:47:08.586ZINFOcrucible: Crucible stats registered with UUID: 5f22cdcf-4871-4f4d-ae45-5a27f568c3d6 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
512025-01-21T15:47:08.586ZWARNcrucible: could not find Downstairs address for Nexus session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
522025-01-21T15:47:08.588ZINFOcrucible: Downstairs in volume = 3 task = crutest
532025-01-21T15:47:08.588ZINFOcrucible: The guest has requested activation with gen:1
542025-01-21T15:47:08.588ZINFOcrucible: 5f22cdcf-4871-4f4d-ae45-5a27f568c3d6 active request set session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
552025-01-21T15:47:08.588ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
562025-01-21T15:47:08.588ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
572025-01-21T15:47:08.588ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
582025-01-21T15:47:08.588ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
592025-01-21T15:47:08.588ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
602025-01-21T15:47:08.588ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
612025-01-21T15:47:08.588ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
622025-01-21T15:47:08.588ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
632025-01-21T15:47:08.588ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
642025-01-21T15:47:08.588ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49516) = io task client = 2 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
652025-01-21T15:47:08.589ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48634) = io task client = 0 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
662025-01-21T15:47:08.589ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51013) = io task client = 1 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
672025-01-21T15:47:08.591ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
682025-01-21T15:47:08.591ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
692025-01-21T15:47:08.591ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
702025-01-21T15:47:08.592ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
712025-01-21T15:47:08.592ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
722025-01-21T15:47:08.592ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
732025-01-21T15:47:08.593ZINFOcrucible: Waiting for more clients to be ready session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
742025-01-21T15:47:08.593ZINFOcrucible: Waiting for more clients to be ready session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
752025-01-21T15:47:08.593ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
762025-01-21T15:47:08.593ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
772025-01-21T15:47:08.593ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
782025-01-21T15:47:08.593ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
792025-01-21T15:47:08.593ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
802025-01-21T15:47:08.593ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
812025-01-21T15:47:08.593ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
822025-01-21T15:47:08.593ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
832025-01-21T15:47:08.593ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
842025-01-21T15:47:08.593ZINFOcrucible: Max found gen is 1 = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
852025-01-21T15:47:08.593ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
862025-01-21T15:47:08.593ZINFOcrucible: Next flush: 1 = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
872025-01-21T15:47:08.593ZINFOcrucible: All extents match = downstairs session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
882025-01-21T15:47:08.594ZINFOcrucible: No downstairs reconciliation required session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
892025-01-21T15:47:08.594ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
902025-01-21T15:47:08.594ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
912025-01-21T15:47:08.594ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
922025-01-21T15:47:08.594ZINFOcrucible: All required reconciliation work is completed session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
932025-01-21T15:47:08.594ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
942025-01-21T15:47:08.594ZINFOcrucible: 5f22cdcf-4871-4f4d-ae45-5a27f568c3d6 is now active with session: c92739a0-b5de-4795-a55c-7b3ca4ca2347 session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
952025-01-21T15:47:08.594ZINFOcrucible: Set Active after no reconciliation session_id = c92739a0-b5de-4795-a55c-7b3ca4ca2347
962025-01-21T15:47:08.594ZINFOcrucible: The guest has finished waiting for activation with:1
97 Activate successful
98 Wait for a query_work_queue command to finish before sending IO
99 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
100 Fill test
101 Read and Verify all blocks (0..5000 range:false)
102 Wrote out file "/tmp/test_replay_verify.log"
103 CLIENT: Tests done. All submitted work has been ACK'd
104 ----------------------------------------------------------------
105 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
106 States: Active Active Active
107 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
108 1021 Acked Read 256 Done Done Done false
109 1022 Acked Read 256 Done Done Done false
110 1023 Acked Read 256 Done Done Done false
111 1024 Acked Read 256 Done Done Done false
112 1025 Acked Read 256 Done Done Done false
113 1026 Acked Read 256 Done Done Done false
114 1027 Acked Read 256 Done Done Done false
115 1028 Acked Read 256 Done Done Done false
116 1029 Acked Read 256 Done Done Done false
117 1030 Acked Read 256 Done Done Done false
118 1031 Acked Read 256 Done Done Done false
119 1032 Acked Read 256 Done Done Done false
120 1033 Acked Read 256 Done Done Done false
121 1034 Acked Read 256 Done Done Done false
122 1035 Acked Read 256 Done Done Done false
123 1036 Acked Read 256 Done Done Done false
124 1037 Acked Read 256 Done Done Done false
125 1038 Acked Read 256 Done Done Done false
126 1039 Acked Read 136 Done Done Done false
127 1040 Acked Read 256 Done Done Done false
128 STATES DS:0 DS:1 DS:2 TOTAL
129 Sent 0 0 0 0
130 Done 20 20 20 60
131 Skipped 0 0 0 0
132 Error 0 0 0 0
133 Last Flush: 1020 1020 1020
134 Downstairs last five completed: 1020 1019 1018 1017 1016
135 Upstairs last five completed: 1040 1039 1038 1037 1036
136 CLIENT: Up:0 ds:20 act:3
137 ----------------------------------------------------------------
138 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
139 Downstairs last five completed: 1041 1040 1039 1038 1037
140 Upstairs last five completed: 1041 1040 1039 1038 1037
141 CLIENT: Up:0 ds:0 act:3
142 CLIENT: All crucible jobs finished, exiting program
1432025-01-21T15:47:13.177ZWARNcrucible: Guest handle has been dropped
144 Replay loop starts now January 21, 2025 at 03:47:13 PM UTC
145 Jan 21 15:47:13.253 INFO accepted connection, remote_addr: 127.0.0.1:39067, local_addr: 127.0.0.1:9998
146 Jan 21 15:47:13.254 INFO request completed, latency_us: 172, response_code: 200, uri: /regioninfo, method: GET, req_id: 9d7131ec-363f-45f9-a150-e42ea4f52ed1, remote_addr: 127.0.0.1:39067, local_addr: 127.0.0.1:9998
1472025-01-21T15:47:13.255ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
148 Jan 21 15:47:13.255 INFO request completed, latency_us: 94, response_code: 200, uri: /regioncount, method: GET, req_id: b7b75307-5d0b-4c7d-9e02-87bec7eb166d, remote_addr: 127.0.0.1:39067, local_addr: 127.0.0.1:9998
1492025-01-21T15:47:13.255ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
150 Jan 21 15:47:13.255 INFO request completed, latency_us: 115, response_code: 200, uri: /port/cid/0, method: GET, req_id: 22410e85-4498-47c3-9a5b-e38bd0f9aa8f, remote_addr: 127.0.0.1:39067, local_addr: 127.0.0.1:9998
151 Jan 21 15:47:13.256 INFO request completed, latency_us: 100, response_code: 200, uri: /port/cid/1, method: GET, req_id: ee16724a-7c43-479d-b130-3906fd8d70e6, remote_addr: 127.0.0.1:39067, local_addr: 127.0.0.1:9998
152 Jan 21 15:47:13.257 INFO request completed, latency_us: 99, response_code: 200, uri: /port/cid/2, method: GET, req_id: 462d2aca-a610-4add-867c-20e0a93c9bb3, remote_addr: 127.0.0.1:39067, local_addr: 127.0.0.1:9998
1532025-01-21T15:47:13.257ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1542025-01-21T15:47:13.257ZINFOcrucible: Upstairs starts
1552025-01-21T15:47:13.257ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "5c2b227300b758c153f33da870809d67a5160dae", git_commit_timestamp: "2025-01-21T15:31:20.000000000Z", git_branch: "mkeeter/always-check-live-repair", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1562025-01-21T15:47:13.257ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1572025-01-21T15:47:13.257ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1582025-01-21T15:47:13.258ZINFOcrucible: Crucible 081a58d3-5115-4e46-9ed0-3f4e1b963f6f has session id: 5d7111f5-ffb3-4e9c-baab-0950e41aedfd session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1592025-01-21T15:47:13.258ZINFOcrucible: Upstairs opts: Upstairs UUID: 081a58d3-5115-4e46-9ed0-3f4e1b963f6f, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1602025-01-21T15:47:13.258ZINFOcrucible: Crucible stats registered with UUID: 081a58d3-5115-4e46-9ed0-3f4e1b963f6f session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1612025-01-21T15:47:13.258ZWARNcrucible: could not find Downstairs address for Nexus session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1622025-01-21T15:47:13.259ZINFOcrucible: Downstairs in volume = 3 task = crutest
1632025-01-21T15:47:13.259ZINFOcrucible: The guest has requested activation with gen:2
1642025-01-21T15:47:13.259ZINFOcrucible: 081a58d3-5115-4e46-9ed0-3f4e1b963f6f active request set session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1652025-01-21T15:47:13.259ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1662025-01-21T15:47:13.259ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1672025-01-21T15:47:13.259ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1682025-01-21T15:47:13.259ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1692025-01-21T15:47:13.259ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1702025-01-21T15:47:13.259ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1712025-01-21T15:47:13.259ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1722025-01-21T15:47:13.259ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1732025-01-21T15:47:13.259ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1742025-01-21T15:47:13.259ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53949) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1752025-01-21T15:47:13.259ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60802) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1762025-01-21T15:47:13.259ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48394) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1772025-01-21T15:47:13.260ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1782025-01-21T15:47:13.260ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1792025-01-21T15:47:13.260ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1802025-01-21T15:47:13.261ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1812025-01-21T15:47:13.261ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1822025-01-21T15:47:13.261ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1832025-01-21T15:47:13.261ZINFOcrucible: Waiting for more clients to be ready session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1842025-01-21T15:47:13.261ZINFOcrucible: Waiting for more clients to be ready session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1852025-01-21T15:47:13.262ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1862025-01-21T15:47:13.262ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1872025-01-21T15:47:13.262ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1882025-01-21T15:47:13.262ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1892025-01-21T15:47:13.262ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1902025-01-21T15:47:13.262ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1912025-01-21T15:47:13.262ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1922025-01-21T15:47:13.262ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1932025-01-21T15:47:13.262ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
194 {"Activate successful
195 msg":"Wait for a query_work_queue command to finish before sending IO
196 Max found gen is 2","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:47:13.262160572Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"downstairs","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
1972025-01-21T15:47:13.262ZINFOcrucible: Generation requested: 2 >= found:2 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1982025-01-21T15:47:13.262ZINFOcrucible: Next flush: 2 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1992025-01-21T15:47:13.262ZINFOcrucible: All extents match = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2002025-01-21T15:47:13.262ZINFOcrucible: No downstairs reconciliation required session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2012025-01-21T15:47:13.262ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2022025-01-21T15:47:13.262ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2032025-01-21T15:47:13.262ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2042025-01-21T15:47:13.262ZINFOcrucible: All required reconciliation work is completed session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2052025-01-21T15:47:13.262ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2062025-01-21T15:47:13.262ZINFOcrucible: 081a58d3-5115-4e46-9ed0-3f4e1b963f6f is now active with session: 5d7111f5-ffb3-4e9c-baab-0950e41aedfd session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2072025-01-21T15:47:13.262ZINFOcrucible: Set Active after no reconciliation session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2082025-01-21T15:47:13.262ZINFOcrucible: The guest has finished waiting for activation with:2
209 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
210 Loading write count information from file "/tmp/test_replay_verify.log"
211 Jan 21 15:47:13.296 INFO accepted connection, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
212 Main task has work to do, go find it
213 got dsc Stop(2)
214 Jan 21 15:47:13.297 INFO request completedstop 2
215 , latency_us: 186, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 7d5c9498-df02-4527-a010-29be313f72a4, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
216 [8830] Got stop action so:false kr:false
2172025-01-21T15:47:13.297ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2182025-01-21T15:47:13.297ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2192025-01-21T15:47:13.298ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2202025-01-21T15:47:13.298ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
221 Jan 21 15:47:13.298 INFO request completed, latency_us: 263, response_code: 200, uri: /state/cid/2, method: GET, req_id: 1fa9ee58-58b9-49eb-bb03-65f9f6b4b76d, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
222 [8830] Exited with: ExitStatus(unix_wait_status(9))
223 [8830][2] reports Exit
224 Jan 21 15:47:17.300 INFO request completed, latency_us: 208, response_code: 200, uri: /state/cid/2, method: GET, req_id: fa5f23a5-3910-4854-b44a-1e411e8ee9a7, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
225 001/300 Read block 739 len 40960
226 002/300 Flush
227 003/300 Read block 2240 len 8192
228 004/300 Write block 3866 len 24576 data: 2 2 2 2 2 2
229 005/300 Write block 2018 len 4096 data: 2
230 006/300 Flush
231 007/300 Read block 1144 len 32768
232 008/300 Read block 1813 len 16384
233 009/300 Write block 2410 len 16384 data: 2 2 2 2
234 010/300 Write block 2790 len 40960 data: 2 2 2 2 2 2 2 2 2 2
235 011/300 Flush
236 012/300 Write block 345 len 36864 data: 2 2 2 2 2 2 2 2 2
237 013/300 Read block 4061 len 12288
238 014/300 Read block 3102 len 40960
239 015/300 Read block 4993 len 24576
240 016/300 Read block 2761 len 24576
241 017/300 Read block 4055 len 16384
242 018/300 Write block 4168 len 32768 data: 2 2 2 2 2 2 2 2
243 019/300 Read block 2168 len 36864
244 020/300 Write block 4683 len 4096 data: 2
245 021/300 Flush
246 022/300 Read block 2313 len 8192
247 023/300 Write block 1702 len 24576 data: 2 2 2 2 2 2
248 024/300 Read block 1132 len 20480
249 025/300 Write block 2544 len 28672 data: 2 2 2 2 2 2 2
250 026/300 Write block 1974 len 32768 data: 2 2 2 2 2 2 2 2
251 027/300 Read block 3393 len 20480
252 028/300 Flush
253 029/300 Read block 2560 len 12288
254 030/300 Read block 460 len 28672
255 031/300 Read block 4573 len 40960
256 032/300 Flush
257 033/300 Read block 3390 len 32768
258 034/300 Write block 2909 len 4096 data: 2
259 035/300 Flush
260 036/300 Read block 1239 len 24576
261 037/300 Read block 1191 len 40960
262 038/300 Read block 2668 len 16384
263 039/300 Read block 1715 len 36864
264 040/300 Read block 1321 len 28672
265 041/300 Write block 3629 len 4096 data: 2
266 042/300 Flush
267 043/300 Write block 37 len 8192 data: 2 2
268 044/300 Write block 1998 len 32768 data: 2 2 2 2 2 2 2 2
269 045/300 Flush
270 046/300 Write block 2073 len 16384 data: 2 2 2 2
271 047/300 Read block 1695 len 4096
272 048/300 Write block 1333 len 36864 data: 2 2 2 2 2 2 2 2 2
273 049/300 Read block 3195 len 40960
274 050/300 Read block 777 len 24576
275 051/300 Read block 4280 len 32768
276 052/300 Read block 927 len 28672
277 053/300 Write block 2327 len 32768 data: 2 2 2 2 2 2 2 2
278 054/300 Read block 1335 len 36864
279 055/300 Write block 1438 len 40960 data: 2 2 2 2 2 2 2 2 2 2
280 056/300 Flush
281 057/300 Read block 112 len 8192
282 058/300 Read block 1721 len 32768
283 059/300 Flush
284 060/300 Write block 4925 len 28672 data: 2 2 2 2 2 2 2
285 061/300 Read block 4132 len 40960
286 062/300 Read block 1198 len 24576
287 063/300 Write block 2224 len 28672 data: 2 2 2 2 2 2 2
288 064/300 Write block 2457 len 16384 data: 2 2 2 2
289 065/300 Read block 4692 len 8192
290 066/300 Write block 4774 len 20480 data: 2 2 2 2 2
291 067/300 Write block 3925 len 24576 data: 2 2 2 2 2 2
292 068/300 Read block 4893 len 20480
293 069/300 Write block 983 len 12288 data: 2 2 2
294 070/300 Flush
295 071/300 Write block 213 len 40960 data: 2 2 2 2 2 2 2 2 2 2
296 072/300 Read block 1129 len 8192
297 073/300 Read block 3838 len 20480
298 074/300 Read block 2397 len 16384
299 075/300 Write block 2807 len 40960 data: 2 2 2 2 2 2 2 2 2 2
300 076/300 Write block 4778 len 20480 data: 3 2 2 2 2
301 077/300 Write block 4923 len 32768 data: 2 2 3 3 3 3 3 3
302 078/300 Read block 3976 len 24576
303 079/300 Write block 3748 len 8192 data: 2 2
304 080/300 Write block 358 len 16384 data: 2 2 2 2
305 081/300 Read block 693 len 32768
306 082/300 Read block 2857 len 16384
307 083/300 Read block 3724 len 40960
308 084/300 Write block 529 len 12288 data: 2 2 2
309 085/300 Read block 4336 len 20480
310 086/300 Flush
311 087/300 Write block 2703 len 4096 data: 2
312 088/300 Read block 475 len 24576
313 089/300 Read block 4337 len 8192
314 090/300 Read block 740 len 36864
315 091/300 Write block 3985 len 16384 data: 2 2 2 2
316 092/300 Write block 1186 len 32768 data: 2 2 2 2 2 2 2 2
317 093/300 Write block 2575 len 28672 data: 2 2 2 2 2 2 2
318 094/300 Read block 3860 len 36864
319 095/300 Write block 1483 len 20480 data: 2 2 2 2 2
320 096/300 Flush
321 097/300 Read block 4394 len 20480
322 098/300 Write block 4720 len 36864 data: 2 2 2 2 2 2 2 2 2
323 099/300 Read block 3631 len 24576
324 100/300 Write block 3589 len 20480 data: 2 2 2 2 2
325 101/300 Write block 2229 len 40960 data: 3 3 2 2 2 2 2 2 2 2
326 102/300 Write block 3890 len 28672 data: 2 2 2 2 2 2 2
327 103/300 Read block 324 len 32768
328 104/300 Flush
329 105/300 Read block 3684 len 12288
330 106/300 Write block 1737 len 8192 data: 2 2
331 107/300 Flush
332 108/300 Read block 79 len 24576
333 109/300 Read block 1579 len 20480
334 110/300 Read block 1413 len 40960
335 111/300 Write block 2070 len 32768 data: 2 2 2 3 3 3 3 2
336 112/300 Flush
337 113/300 Read block 1026 len 12288
338 114/300 Read block 578 len 4096
339 115/300 Write block 4749 len 8192 data: 2 2
340 116/300 Flush
341 117/300 Write block 4573 len 28672 data: 2 2 2 2 2 2 2
342 118/300 Read block 4552 len 24576
343 119/300 Read block 2008 len 24576
344 120/300 Flush
345 121/300 Write block 1313 len 24576 data: 2 2 2 2 2 2
346 122/300 Read block 1755 len 28672
347 123/300 Read block 2669 len 8192
348 124/300 Read block 4923 len 32768
349 125/300 Write block 1362 len 28672 data: 2 2 2 2 2 2 2
350 126/300 Write block 3984 len 12288 data: 2 3 3
351 127/300 Write block 546 len 12288 data: 2 2 2
352 128/300 Read block 4446 len 4096
353 129/300 Flush
354 130/300 Write block 4128 len 20480 data: 2 2 2 2 2
355 131/300 Read block 4056 len 24576
356 132/300 Read block 2281 len 20480
357 133/300 Read block 3971 len 16384
358 134/300 Read block 2655 len 40960
359 135/300 Write block 1406 len 40960 data: 2 2 2 2 2 2 2 2 2 2
360 136/300 Read block 4438 len 36864
361 137/300 Write block 1989 len 4096 data: 2
362 138/300 Read block 4100 len 24576
363 139/300 Write block 754 len 40960 data: 2 2 2 2 2 2 2 2 2 2
364 140/300 Read block 2214 len 8192
365 141/300 Read block 3688 len 12288
366 142/300 Flush
367 143/300 Read block 19 len 12288
368 144/300 Write block 4814 len 28672 data: 2 2 2 2 2 2 2
369 145/300 Read block 4939 len 12288
370 146/300 Read block 4032 len 40960
371 147/300 Flush
372 148/300 Write block 4909 len 32768 data: 2 2 2 2 2 2 2 2
373 149/300 Write block 200 len 40960 data: 2 2 2 2 2 2 2 2 2 2
374 150/300 Read block 3478 len 32768
375 151/300 Flush
376 152/300 Write block 1037 len 16384 data: 2 2 2 2
377 153/300 Read block 4574 len 4096
378 154/300 Flush
379 155/300 Read block 3398 len 8192
380 156/300 Read block 2282 len 12288
381 157/300 Write block 4620 len 20480 data: 2 2 2 2 2
382 158/300 Write block 1420 len 40960 data: 2 2 2 2 2 2 2 2 2 2
383 159/300 Write block 553 len 24576 data: 2 2 2 2 2 2
384 160/300 Read block 4741 len 8192
385 161/300 Write block 593 len 20480 data: 2 2 2 2 2
386 162/300 Read block 2132 len 24576
387 163/300 Write block 1692 len 24576 data: 2 2 2 2 2 2
388 164/300 Read block 1451 len 16384
389 165/300 Read block 3768 len 28672
390 166/300 Write block 769 len 24576 data: 2 2 2 2 2 2
391 167/300 Flush
392 168/300 Read block 425 len 32768
393 169/300 Read block 59 len 4096
394 170/300 Flush
395 171/300 Write block 4519 len 4096 data: 2
396 172/300 Read block 4191 len 28672
397 173/300 Write block 2670 len 16384 data: 2 2 2 2
398 174/300 Write block 2565 len 40960 data: 2 2 2 2 2 2 2 2 2 2
399 175/300 Write block 1809 len 40960 data: 2 2 2 2 2 2 2 2 2 2
400 176/300 Flush
401 177/300 Write block 468 len 16384 data: 2 2 2 2
402 178/300 Write block 4331 len 8192 data: 2 2
403 179/300 Flush
404 180/300 Write block 1505 len 4096 data: 2
405 181/300 Read block 3065 len 20480
406 182/300 Read block 4854 len 16384
407 183/300 Write block 2107 len 12288 data: 2 2 2
408 184/300 Write block 3863 len 20480 data: 2 2 2 3 3
409 185/300 Read block 2054 len 16384
410 186/300 Write block 3975 len 4096 data: 2
411 187/300 Read block 1515 len 36864
412 188/300 Write block 4424 len 12288 data: 2 2 2
413 189/300 Read block 376 len 8192
414 190/300 Read block 3693 len 8192
415 191/300 Read block 1523 len 40960
416 192/300 Read block 1534 len 20480
417 193/300 Flush
418 194/300 Write block 270 len 24576 data: 2 2 2 2 2 2
419 195/300 Write block 1857 len 36864 data: 2 2 2 2 2 2 2 2 2
420 196/300 Read block 491 len 8192
421 197/300 Write block 130 len 32768 data: 2 2 2 2 2 2 2 2
422 198/300 Write block 3552 len 36864 data: 2 2 2 2 2 2 2 2 2
423 199/300 Read block 4279 len 4096
424 200/300 Write block 4004 len 28672 data: 2 2 2 2 2 2 2
425 201/300 Write block 1923 len 36864 data: 2 2 2 2 2 2 2 2 2
426 202/300 Read block 1644 len 20480
427 203/300 Write block 2468 len 36864 data: 2 2 2 2 2 2 2 2 2
428 204/300 Write block 2441 len 16384 data: 2 2 2 2
429 205/300 Write block 2288 len 32768 data: 2 2 2 2 2 2 2 2
430 206/300 Read block 1350 len 32768
431 207/300 Read block 886 len 8192
432 208/300 Write block 1756 len 20480 data: 2 2 2 2 2
433 209/300 Read block 3667 len 8192
434 210/300 Read block 2831 len 20480
435 211/300 Read block 3282 len 28672
436 212/300 Read block 3656 len 12288
437 213/300 Write block 3177 len 16384 data: 2 2 2 2
438 214/300 Read block 549 len 32768
439 215/300 Write block 4632 len 28672 data: 2 2 2 2 2 2 2
440 216/300 Read block 99 len 24576
441 217/300 Write block 3612 len 4096 data: 2
442 218/300 Read block 4649 len 20480
443 219/300 Write block 3366 len 16384 data: 2 2 2 2
444 220/300 Read block 784 len 40960
445 221/300 Read block 3040 len 40960
446 222/300 Write block 156 len 8192 data: 2 2
447 223/300 Read block 3639 len 32768
448 224/300 Read block 3032 len 4096
449 225/300 Read block 1557 len 40960
450 226/300 Write block 498 len 28672 data: 2 2 2 2 2 2 2
451 227/300 Read block 169 len 32768
452 228/300 Read block 2819 len 24576
453 229/300 Read block 1897 len 12288
454 230/300 Read block 487 len 32768
455 231/300 Read block 2032 len 12288
456 232/300 Write block 2457 len 32768 data: 3 3 3 3 2 2 2 2
457 233/300 Write block 4942 len 16384 data: 2 2 2 2
458 234/300 Read block 4926 len 24576
459 235/300 Write block 233 len 40960 data: 2 2 2 2 2 2 2 2 2 2
460 236/300 Read block 2298 len 28672
461 237/300 Read block 3739 len 8192
462 238/300 Write block 461 len 24576 data: 2 2 2 2 2 2
463 239/300 Write block 728 len 36864 data: 2 2 2 2 2 2 2 2 2
464 240/300 Read block 737 len 24576
465 241/300 Write block 2426 len 16384 data: 2 2 2 2
466 242/300 Write block 861 len 36864 data: 2 2 2 2 2 2 2 2 2
467 243/300 Write block 4797 len 16384 data: 2 2 2 2
468 244/300 Read block 3302 len 4096
469 245/300 Read block 86 len 28672
470 246/300 Read block 1061 len 32768
471 247/300 Write block 3939 len 4096 data: 2
472 248/300 Write block 2189 len 16384 data: 2 2 2 2
473 249/300 Write block 2552 len 40960 data: 2 2 2 2 2 2 2 2 2 2
474 250/300 Read block 2716 len 16384
475 251/300 Flush
476 252/300 Flush
477 253/300 Read block 2697 len 8192
478 254/300 Write block 4468 len 12288 data: 2 2 2
479 255/300 Read block 871 len 32768
480 256/300 Write block 3154 len 28672 data: 2 2 2 2 2 2 2
481 257/300 Write block 2424 len 8192 data: 2 2
482 258/300 Write block 2886 len 32768 data: 2 2 2 2 2 2 2 2
483 259/300 Read block 232 len 4096
484 260/300 Flush
485 261/300 Read block 1546 len 32768
486 262/300 Write block 4626 len 12288 data: 2 2 2
487 263/300 Write block 3137 len 36864 data: 2 2 2 2 2 2 2 2 2
488 264/300 Flush
489 265/300 Read block 4696 len 12288
490 266/300 Read block 2087 len 36864
491 267/300 Write block 4389 len 36864 data: 2 2 2 2 2 2 2 2 2
492 268/300 Read block 3175 len 36864
493 269/300 Write block 2822 len 8192 data: 2 2
494 270/300 Read block 3749 len 40960
495 271/300 Write block 301 len 24576 data: 2 2 2 2 2 2
496 272/300 Read block 4886 len 4096
497 273/300 Write block 3196 len 24576 data: 2 2 2 2 2 2
498 274/300 Flush
499 275/300 Read block 818 len 12288
500 276/300 Write block 2250 len 16384 data: 2 2 2 2
501 277/300 Read block 1598 len 24576
502 278/300 Write block 4652 len 8192 data: 2 2
503 279/300 Read block 1522 len 28672
504 280/300 Flush
505 281/300 Read block 4971 len 32768
506 282/300 Write block 2697 len 20480 data: 2 2 2 2 2
507 283/300 Write block 1115 len 24576 data: 2 2 2 2 2 2
508 284/300 Write block 4215 len 28672 data: 2 2 2 2 2 2 2
509 285/300 Read block 3601 len 8192
510 286/300 Read block 573 len 20480
511 287/300 Read block 2442 len 32768
512 288/300 Read block 3150 len 32768
513 289/300 Read block 4390 len 36864
514 290/300 Read block 3638 len 24576
515 291/300 Write block 2139 len 8192 data: 2 2
516 292/300 Read block 1995 len 8192
517 293/300 Write block 4986 len 12288 data: 2 2 2
518 294/300 Write block 967 len 36864 data: 2 2 2 2 2 2 2 2 2
519 295/300 Write block 1933 len 20480 data: 2 2 2 2 2
520 296/300 Read block 3005 len 40960
521 297/300 Read block 1302 len 32768
522 298/300 Read block 3766 len 24576
523 299/300 Read block 216 len 36864
524 300/300 Read block 3554 len 28672
525 301/300 Read block 2773 len 40960
526 Main task has work to do, go find it
527 got dsc Start(2)
528 start 2
529 Jan 21 15:47:17.546 INFO request completed, latency_us: 223, response_code: 204, uri: /start/cid/2, method: POST, req_id: 9a8ea242-e26f-40d6-af40-1ad114aac440, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
530 Replay: started 2, returned:Ok(())
531 ----------------------------------------------------------------
532 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
533 States: Active Active Offline
534 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
535 1000 Acked Read 10 Done Done Sent false
536 1001 Acked Flush 0 Done Done Sent false
537 1002 Acked Read 2 Done Done Sent false
538 1003 Acked Write 6 Done Done Sent false
539 1004 Acked Write 1 Done Done Sent false
540 1005 Acked Flush 0 Done Done Sent false
541 1006 Acked Read 8 Done Done Sent false
542 1007 Acked Read 4 Done Done Sent false
543 1008 Acked Write 4 Done Done Sent false
544 1009 Acked Write 10 Done Done Sent false
545 1010 Acked Flush 0 Done Done Sent false
546 1011 Acked Write 9 Done Done Sent false
547 1012 Acked Read 3 Done Done Sent false
548 1013 Acked Read 10 Done Done Sent false
549 1014 Acked Read 6 Done Done Sent false
550 1015 Acked Read 6 Done Done Sent false
551 1016 Acked Read 4 Done Done Sent false
552 1017 Acked Write 8 Done Done Sent false
553 1018 Acked Read 9 Done Done Sent false
554 1019 Acked Write 1 Done Done Sent false
555 1020 Acked Flush 0 Done Done Sent false
556 1021 Acked Read 2 Done Done Sent false
557 1022 Acked Write 6 Done Done Sent false
558 1023 Acked Read 5 Done Done Sent false
559 1024 Acked Write 7 Done Done Sent false
560 1025 Acked Write 8 Done Done Sent false
561 1026 Acked Read 5 Done Done Sent false
562 1027 Acked Flush 0 Done Done Sent false
563 1028 Acked Read 3 Done Done Sent false
564 1029 Acked Read 7 Done Done Sent false
565 1030 Acked Read 10 Done Done Sent false
566 1031 Acked Flush 0 Done Done Sent false
567 1032 Acked Read 8 Done Done Sent false
568 1033 Acked Write 1 Done Done Sent false
569 1034 Acked Flush 0 Done Done Sent false
570 1035 Acked Read 6 Done Done Sent false
571 1036 Acked Read 10 Done Done Sent false
572 1037 Acked Read 4 Done Done Sent false
573 1038 Acked Read 9 Done Done Sent false
574 1039 Acked Read 7 Done Done Sent false
575 1040 Acked Write 1 Done Done Sent false
576 1041 Acked Flush 0 Done Done Sent false
577 1042 Acked Write 2 Done Done Sent false
578 1043 Acked Write 8 Done Done Sent false
579 1044 Acked Flush 0 Done Done Sent false
580 1045 Acked Write 4 Done Done Sent false
581 1046 Acked Read 1 Done Done Sent false
582 1047 Acked Write 9 Done Done Sent false
583 1048 Acked Read 10 Done Done Sent false
584 1049 Acked Read 6 Done Done Sent false
585 1050 Acked Read 8 Done Done Sent false
586 1051 Acked Read 7 Done Done Sent false
587 1052 Acked Write 8 Done Done Sent false
588 1053 Acked Read 9 Done Done Sent false
589 1054 Acked Write 10 Done Done Sent false
590 1055 Acked Flush 0 Done Done Sent false
591 1056 Acked Read 2 Done Done Sent false
592 1057 Acked Read 8 Done Done Sent false
593 1058 Acked Flush 0 Done Done Sent false
594 1059 Acked Write 7 Done Done Sent false
595 1060 Acked Read 10 Done Done Sent false
596 1061 Acked Read 6 Done Done Sent false
597 1062 Acked Write 7 Done Done Sent false
598 1063 Acked Write 4 Done Done Sent false
599 1064 Acked Read 2 Done Done Sent false
600 1065 Acked Write 5 Done Done Sent false
601 1066 Acked Write 6 Done Done Sent false
602 1067 Acked Read 5 Done Done Sent false
603 1068 Acked Write 3 Done Done Sent false
604 1069 Acked Flush 0 Done Done Sent false
605 1070 Acked Write 10 Done Done Sent false
606 1071 Acked Read 2 Done Done Sent false
607 1072 Acked Read 5 Done Done Sent false
608 1073 Acked Read 4 Done Done Sent false
609 1074 Acked Write 10 Done Done Sent false
610 1075 Acked Write 5 Done Done Sent false
611 1076 Acked Write 8 Done Done Sent false
612 1077 Acked Read 6 Done Done Sent false
613 1078 Acked Write 2 Done Done Sent false
614 1079 Acked Write 4 Done Done Sent false
615 1080 Acked Read 8 Done Done Sent false
616 1081 Acked Read 4 Done Done Sent false
617 1082 Acked Read 10 Done Done Sent false
618 1083 Acked Write 3 Done Done Sent false
619 1084 Acked Read 5 Done Done Sent false
620 1085 Acked Flush 0 Done Done Sent false
621 1086 Acked Write 1 Done Done Sent false
622 1087 Acked Read 6 Done Done Sent false
623 1088 Acked Read 2 Done Done Sent false
624 1089 Acked Read 9 Done Done Sent false
625 1090 Acked Write 4 Done Done Sent false
626 1091 Acked Write 8 Done Done Sent false
627 1092 Acked Write 7 Done Done Sent false
628 1093 Acked Read 9 Done Done Sent false
629 1094 Acked Write 5 Done Done Sent false
630 1095 Acked Flush 0 Done Done Sent false
631 1096 Acked Read 5 Done Done Sent false
632 1097 Acked Write 9 Done Done Sent false
633 1098 Acked Read 6 Done Done Sent false
634 1099 Acked Write 5 Done Done Sent false
635 1100 Acked Write 10 Done Done Sent false
636 1101 Acked Write 7 Done Done Sent false
637 1102 Acked Read 8 Done Done Sent false
638 1103 Acked Flush 0 Done Done Sent false
639 1104 Acked Read 3 Done Done Sent false
640 1105 Acked Write 2 Done Done Sent false
641 1106 Acked Flush 0 Done Done Sent false
642 1107 Acked Read 6 Done Done Sent false
643 1108 Acked Read 5 Done Done Sent false
644 1109 Acked Read 10 Done Done Sent false
645 1110 Acked Write 8 Done Done Sent false
646 1111 Acked Flush 0 Done Done Sent false
647 1112 Acked Read 3 Done Done Sent false
648 1113 Acked Read 1 Done Done Sent false
649 1114 Acked Write 2 Done Done Sent false
650 1115 Acked Flush 0 Done Done Sent false
651 1116 Acked Write 7 Done Done Sent false
652 1117 Acked Read 6 Done Done Sent false
653 1118 Acked Read 6 Done Done Sent false
654 1119 Acked Flush 0 Done Done Sent false
655 1120 Acked Write 6 Done Done Sent false
656 1121 Acked Read 7 Done Done Sent false
657 1122 Acked Read 2 Done Done Sent false
658 1123 Acked Read 8 Done Done Sent false
659 1124 Acked Write 7 Done Done Sent false
660 1125 Acked Write 3 Done Done Sent false
661 1126 Acked Write 3 Done Done Sent false
662 1127 Acked Read 1 Done Done Sent false
663 1128 Acked Flush 0 Done Done Sent false
664 1129 Acked Write 5 Done Done Sent false
665 1130 Acked Read 6 Done Done Sent false
666 1131 Acked Read 5 Done Done Sent false
667 1132 Acked Read 4 Done Done Sent false
668 1133 Acked Read 10 Done Done Sent false
669 1134 Acked Write 10 Done Done Sent false
670 1135 Acked Read 9 Done Done Sent false
671 1136 Acked Write 1 Done Done Sent false
672 1137 Acked Read 6 Done Done Sent false
673 1138 Acked Write 10 Done Done Sent false
674 1139 Acked Read 2 Done Done Sent false
675 1140 Acked Read 3 Done Done Sent false
676 1141 Acked Flush 0 Done Done Sent false
677 1142 Acked Read 3 Done Done Sent false
678 1143 Acked Write 7 Done Done Sent false
679 1144 Acked Read 3 Done Done Sent false
680 1145 Acked Read 10 Done Done Sent false
681 1146 Acked Flush 0 Done Done Sent false
682 1147 Acked Write 8 Done Done Sent false
683 1148 Acked Write 10 Done Done Sent false
684 1149 Acked Read 8 Done Done Sent false
685 1150 Acked Flush 0 Done Done Sent false
686 1151 Acked Write 4 Done Done Sent false
687 1152 Acked Read 1 Done Done Sent false
688 1153 Acked Flush 0 Done Done Sent false
689 1154 Acked Read 2 Done Done Sent false
690 1155 Acked Read 3 Done Done Sent false
691 1156 Acked Write 5 Done Done Sent false
692 1157 Acked Write 10 Done Done Sent false
693 1158 Acked Write 6 Done Done Sent false
694 1159 Acked Read 2 Done Done Sent false
695 1160 Acked Write 5 Done Done Sent false
696 1161 Acked Read 6 Done Done Sent false
697 1162 Acked Write 6 Done Done Sent false
698 1163 Acked Read 4 Done Done Sent false
699 1164 Acked Read 7 Done Done Sent false
700 1165 Acked Write 6 Done Done Sent false
701 1166 Acked Flush 0 Done Done Sent false
702 1167 Acked Read 8 Done Done Sent false
703 1168 Acked Read 1 Done Done Sent false
704 1169 Acked Flush 0 Done Done Sent false
705 1170 Acked Write 1 Done Done Sent false
706 1171 Acked Read 7 Done Done Sent false
707 1172 Acked Write 4 Done Done Sent false
708 1173 Acked Write 10 Done Done Sent false
709 1174 Acked Write 10 Done Done Sent false
710 1175 Acked Flush 0 Done Done Sent false
711 1176 Acked Write 4 Done Done Sent false
712 1177 Acked Write 2 Done Done Sent false
713 1178 Acked Flush 0 Done Done Sent false
714 1179 Acked Write 1 Done Done Sent false
715 1180 Acked Read 5 Done Done Sent false
716 1181 Acked Read 4 Done Done Sent false
717 1182 Acked Write 3 Done Done Sent false
718 1183 Acked Write 5 Done Done Sent false
719 1184 Acked Read 4 Done Done Sent false
720 1185 Acked Write 1 Done Done Sent false
721 1186 Acked Read 9 Done Done Sent false
722 1187 Acked Write 3 Done Done Sent false
723 1188 Acked Read 2 Done Done Sent false
724 1189 Acked Read 2 Done Done Sent false
725 1190 Acked Read 10 Done Done Sent false
726 1191 Acked Read 5 Done Done Sent false
727 1192 Acked Flush 0 Done Done Sent false
728 1193 Acked Write 6 Done Done Sent false
729 1194 Acked Write 9 Done Done Sent false
730 1195 Acked Read 2 Done Done Sent false
731 1196 Acked Write 8 Done Done Sent false
732 1197 Acked Write 9 Done Done Sent false
733 1198 Acked Read 1 Done Done Sent false
734 1199 Acked Write 7 Done Done Sent false
735 1200 Acked Write 9 Done Done Sent false
736 1201 Acked Read 5 Done Done Sent false
737 1202 Acked Write 9 Done Done Sent false
738 1203 Acked Write 4 Done Done Sent false
739 1204 Acked Write 8 Done Done Sent false
740 1205 Acked Read 8 Done Done Sent false
741 1206 Acked Read 2 Done Done Sent false
742 1207 Acked Write 5 Done Done Sent false
743 1208 Acked Read 2 Done Done Sent false
744 1209 Acked Read 5 Done Done Sent false
745 1210 Acked Read 7 Done Done Sent false
746 1211 Acked Read 3 Done Done Sent false
747 1212 Acked Write 4 Done Done Sent false
748 1213 Acked Read 8 Done Done Sent false
749 1214 Acked Write 7 Done Done Sent false
750 1215 Acked Read 6 Done Done Sent false
751 1216 Acked Write 1 Done Done Sent false
752 1217 Acked Read 5 Done Done Sent false
753 1218 Acked Write 4 Done Done Sent false
754 1219 Acked Read 10 Done Done Sent false
755 1220 Acked Read 10 Done Done Sent false
756 1221 Acked Write 2 Done Done Sent false
757 1222 Acked Read 8 Done Done Sent false
758 1223 Acked Read 1 Done Done Sent false
759 1224 Acked Read 10 Done Done Sent false
760 1225 Acked Write 7 Done Done Sent false
761 1226 Acked Read 8 Done Done Sent false
762 1227 Acked Read 6 Done Done Sent false
763 1228 Acked Read 3 Done Done Sent false
764 1229 Acked Read 8 Done Done Sent false
765 1230 Acked Read 3 Done Done Sent false
766 1231 Acked Write 8 Done Done Sent false
767 1232 Acked Write 4 Done Done Sent false
768 1233 Acked Read 6 Done Done Sent false
769 1234 Acked Write 10 Done Done Sent false
770 1235 Acked Read 7 Done Done Sent false
771 1236 Acked Read 2 Done Done Sent false
772 1237 Acked Write 6 Done Done Sent false
773 1238 Acked Write 9 Done Done Sent false
774 1239 Acked Read 6 Done Done Sent false
775 1240 Acked Write 4 Done Done Sent false
776 1241 Acked Write 9 Done Done Sent false
777 1242 Acked Write 4 Done Done Sent false
778 1243 Acked Read 1 Done Done Sent false
779 1244 Acked Read 7 Done Done Sent false
780 1245 Acked Read 8 Done Done Sent false
781 1246 Acked Write 1 Done Done Sent false
782 1247 Acked Write 4 Done Done Sent false
783 1248 Acked Write 10 Done Done Sent false
784 1249 Acked Read 4 Done Done Sent false
785 1250 Acked Flush 0 Done Done Sent false
786 1251 Acked Flush 0 Done Done Sent false
787 1252 Acked Read 2 Done Done Sent false
788 1253 Acked Write 3 Done Done Sent false
789 1254 Acked Read 8 Done Done Sent false
790 1255 Acked Write 7 Done Done Sent false
791 1256 Acked Write 2 Done Done Sent false
792 1257 Acked Write 8 Done Done Sent false
793 1258 Acked Read 1 Done Done Sent false
794 1259 Acked Flush 0 Done Done Sent false
795 1260 Acked Read 8 Done Done Sent false
796 1261 Acked Write 3 Done Done Sent false
797 1262 Acked Write 9 Done Done Sent false
798 1263 Acked Flush 0 Done Done Sent false
799 1264 Acked Read 3 Done Done Sent false
800 1265 Acked Read 9 Done Done Sent false
801 1266 Acked Write 9 Done Done Sent false
802 1267 Acked Read 9 Done Done Sent false
803 1268 Acked Write 2 Done Done Sent false
804 1269 Acked Read 10 Done Done Sent false
805 1270 Acked Write 6 Done Done Sent false
806 1271 Acked Read 1 Done Done Sent false
807 1272 Acked Write 6 Done Done Sent false
808 1273 Acked Flush 0 Done Done Sent false
809 1274 Acked Read 3 Done Done Sent false
810 1275 Acked Write 4 Done Done Sent false
811 1276 Acked Read 6 Done Done Sent false
812 1277 Acked Write 2 Done Done Sent false
813 1278 Acked Read 7 Done Done Sent false
814 1279 Acked Flush 0 Done Done Sent false
815 1280 Acked Read 8 Done Done Sent false
816 1281 Acked Write 5 Done Done Sent false
817 1282 Acked Write 6 Done Done Sent false
818 1283 Acked Write 7 Done Done Sent false
819 1284 Acked Read 2 Done Done Sent false
820 1285 Acked Read 5 Done Done Sent false
821 1286 Acked Read 8 Done Done Sent false
822 1287 Acked Read 8 Done Done Sent false
823 1288 Acked Read 9 Done Done Sent false
824 1289 Acked Read 6 Done Done Sent false
825 1290 Acked Write 2 Done Done Sent false
826 1291 Acked Read 2 Done Done Sent false
827 1292 Acked Write 3 Done Done Sent false
828 1293 Acked Write 9 Done Done Sent false
829 1294 Acked Write 5 Done Done Sent false
830 1295 Acked Read 10 Done Done Sent false
831 1296 Acked Read 8 Done Done Sent false
832 1297 Acked Read 6 Done Done Sent false
833 1298 Acked Read 9 Done Done Sent false
834 1299 Acked Read 7 Done Done Sent false
835 1300 Acked Read 10 Done Done Sent false
836 STATES DS:0 DS:1 DS:2 TOTAL
837 Sent 0 0 301 301
838 Done 301 301 0 602
839 Skipped 0 0 0 0
840 Error 0 0 0 0
841 Last Flush: 1279 1279 0
842 Downstairs last five completed:
843 Upstairs last five completed: 1300 1299 1298 1297 1296
844 CLIENT: Up:0 ds:301 act:2
845 ----------------------------------------------------------------
846 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
847 States: Active Active Offline
848 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
849 1000 Acked Read 10 Done Done Sent false
850 1001 Acked Flush 0 Done Done Sent false
851 1002 Acked Read 2 Done Done Sent false
852 1003 Acked Write 6 Done Done Sent false
853 1004 Acked Write 1 Done Done Sent false
854 1005 Acked Flush 0 Done Done Sent false
855 1006 Acked Read 8 Done Done Sent false
856 1007 Acked Read 4 Done Done Sent false
857 1008 Acked Write 4 Done Done Sent false
858 1009 Acked Write 10 Done Done Sent false
859 1010 Acked Flush 0 Done Done Sent false
860 1011 Acked Write 9 Done Done Sent false
861 1012 Acked Read 3 Done Done Sent false
862 1013 Acked Read 10 Done Done Sent false
863 1014 Acked Read 6 Done Done Sent false
864 1015 Acked Read 6 Done Done Sent false
865 1016 Acked Read 4 Done Done Sent false
866 1017 Acked Write 8 Done Done Sent false
867 1018 Acked Read 9 Done Done Sent false
868 1019 Acked Write 1 Done Done Sent false
869 1020 Acked Flush 0 Done Done Sent false
870 1021 Acked Read 2 Done Done Sent false
871 1022 Acked Write 6 Done Done Sent false
872 1023 Acked Read 5 Done Done Sent false
873 1024 Acked Write 7 Done Done Sent false
874 1025 Acked Write 8 Done Done Sent false
875 1026 Acked Read 5 Done Done Sent false
876 1027 Acked Flush 0 Done Done Sent false
877 1028 Acked Read 3 Done Done Sent false
878 1029 Acked Read 7 Done Done Sent false
879 1030 Acked Read 10 Done Done Sent false
880 1031 Acked Flush 0 Done Done Sent false
881 1032 Acked Read 8 Done Done Sent false
882 1033 Acked Write 1 Done Done Sent false
883 1034 Acked Flush 0 Done Done Sent false
884 1035 Acked Read 6 Done Done Sent false
885 1036 Acked Read 10 Done Done Sent false
886 1037 Acked Read 4 Done Done Sent false
887 1038 Acked Read 9 Done Done Sent false
888 1039 Acked Read 7 Done Done Sent false
889 1040 Acked Write 1 Done Done Sent false
890 1041 Acked Flush 0 Done Done Sent false
891 1042 Acked Write 2 Done Done Sent false
892 1043 Acked Write 8 Done Done Sent false
893 1044 Acked Flush 0 Done Done Sent false
894 1045 Acked Write 4 Done Done Sent false
895 1046 Acked Read 1 Done Done Sent false
896 1047 Acked Write 9 Done Done Sent false
897 1048 Acked Read 10 Done Done Sent false
898 1049 Acked Read 6 Done Done Sent false
899 1050 Acked Read 8 Done Done Sent false
900 1051 Acked Read 7 Done Done Sent false
901 1052 Acked Write 8 Done Done Sent false
902 1053 Acked Read 9 Done Done Sent false
903 1054 Acked Write 10 Done Done Sent false
904 1055 Acked Flush 0 Done Done Sent false
905 1056 Acked Read 2 Done Done Sent false
906 1057 Acked Read 8 Done Done Sent false
907 1058 Acked Flush 0 Done Done Sent false
908 1059 Acked Write 7 Done Done Sent false
909 1060 Acked Read 10 Done Done Sent false
910 1061 Acked Read 6 Done Done Sent false
911 1062 Acked Write 7 Done Done Sent false
912 1063 Acked Write 4 Done Done Sent false
913 1064 Acked Read 2 Done Done Sent false
914 1065 Acked Write 5 Done Done Sent false
915 1066 Acked Write 6 Done Done Sent false
916 1067 Acked Read 5 Done Done Sent false
917 1068 Acked Write 3 Done Done Sent false
918 1069 Acked Flush 0 Done Done Sent false
919 1070 Acked Write 10 Done Done Sent false
920 1071 Acked Read 2 Done Done Sent false
921 1072 Acked Read 5 Done Done Sent false
922 1073 Acked Read 4 Done Done Sent false
923 1074 Acked Write 10 Done Done Sent false
924 1075 Acked Write 5 Done Done Sent false
925 1076 Acked Write 8 Done Done Sent false
926 1077 Acked Read 6 Done Done Sent false
927 1078 Acked Write 2 Done Done Sent false
928 1079 Acked Write 4 Done Done Sent false
929 1080 Acked Read 8 Done Done Sent false
930 1081 Acked Read 4 Done Done Sent false
931 1082 Acked Read 10 Done Done Sent false
932 1083 Acked Write 3 Done Done Sent false
933 1084 Acked Read 5 Done Done Sent false
934 1085 Acked Flush 0 Done Done Sent false
935 1086 Acked Write 1 Done Done Sent false
936 1087 Acked Read 6 Done Done Sent false
937 1088 Acked Read 2 Done Done Sent false
938 1089 Acked Read 9 Done Done Sent false
939 1090 Acked Write 4 Done Done Sent false
940 1091 Acked Write 8 Done Done Sent false
941 1092 Acked Write 7 Done Done Sent false
942 1093 Acked Read 9 Done Done Sent false
943 1094 Acked Write 5 Done Done Sent false
944 1095 Acked Flush 0 Done Done Sent false
945 1096 Acked Read 5 Done Done Sent false
946 1097 Acked Write 9 Done Done Sent false
947 1098 Acked Read 6 Done Done Sent false
948 1099 Acked Write 5 Done Done Sent false
949 1100 Acked Write 10 Done Done Sent false
950 1101 Acked Write 7 Done Done Sent false
951 1102 Acked Read 8 Done Done Sent false
952 1103 Acked Flush 0 Done Done Sent false
953 1104 Acked Read 3 Done Done Sent false
954 1105 Acked Write 2 Done Done Sent false
955 1106 Acked Flush 0 Done Done Sent false
956 1107 Acked Read 6 Done Done Sent false
957 1108 Acked Read 5 Done Done Sent false
958 1109 Acked Read 10 Done Done Sent false
959 1110 Acked Write 8 Done Done Sent false
960 1111 Acked Flush 0 Done Done Sent false
961 1112 Acked Read 3 Done Done Sent false
962 1113 Acked Read 1 Done Done Sent false
963 1114 Acked Write 2 Done Done Sent false
964 1115 Acked Flush 0 Done Done Sent false
965 1116 Acked Write 7 Done Done Sent false
966 1117 Acked Read 6 Done Done Sent false
967 1118 Acked Read 6 Done Done Sent false
968 1119 Acked Flush 0 Done Done Sent false
969 1120 Acked Write 6 Done Done Sent false
970 1121 Acked Read 7 Done Done Sent false
971 1122 Acked Read 2 Done Done Sent false
972 1123 Acked Read 8 Done Done Sent false
973 1124 Acked Write 7 Done Done Sent false
974 1125 Acked Write 3 Done Done Sent false
975 1126 Acked Write 3 Done Done Sent false
976 1127 Acked Read 1 Done Done Sent false
977 1128 Acked Flush 0 Done Done Sent false
978 1129 Acked Write 5 Done Done Sent false
979 1130 Acked Read 6 Done Done Sent false
980 1131 Acked Read 5 Done Done Sent false
981 1132 Acked Read 4 Done Done Sent false
982 1133 Acked Read 10 Done Done Sent false
983 1134 Acked Write 10 Done Done Sent false
984 1135 Acked Read 9 Done Done Sent false
985 1136 Acked Write 1 Done Done Sent false
986 1137 Acked Read 6 Done Done Sent false
987 1138 Acked Write 10 Done Done Sent false
988 1139 Acked Read 2 Done Done Sent false
989 1140 Acked Read 3 Done Done Sent false
990 1141 Acked Flush 0 Done Done Sent false
991 1142 Acked Read 3 Done Done Sent false
992 1143 Acked Write 7 Done Done Sent false
993 1144 Acked Read 3 Done Done Sent false
994 1145 Acked Read 10 Done Done Sent false
995 1146 Acked Flush 0 Done Done Sent false
996 1147 Acked Write 8 Done Done Sent false
997 1148 Acked Write 10 Done Done Sent false
998 1149 Acked Read 8 Done Done Sent false
999 1150 Acked Flush 0 Done Done Sent false
1000 1151 Acked Write 4 Done Done Sent false
1001 1152 Acked Read 1 Done Done Sent false
1002 1153 Acked Flush 0 Done Done Sent false
1003 1154 Acked Read 2 Done Done Sent false
1004 1155 Acked Read 3 Done Done Sent false
1005 1156 Acked Write 5 Done Done Sent false
1006 1157 Acked Write 10 Done Done Sent false
1007 1158 Acked Write 6 Done Done Sent false
1008 1159 Acked Read 2 Done Done Sent false
1009 1160 Acked Write 5 Done Done Sent false
1010 1161 Acked Read 6 Done Done Sent false
1011 1162 Acked Write 6 Done Done Sent false
1012 1163 Acked Read 4 Done Done Sent false
1013 1164 Acked Read 7 Done Done Sent false
1014 1165 Acked Write 6 Done Done Sent false
1015 1166 Acked Flush 0 Done Done Sent false
1016 1167 Acked Read 8 Done Done Sent false
1017 1168 Acked Read 1 Done Done Sent false
1018 1169 Acked Flush 0 Done Done Sent false
1019 1170 Acked Write 1 Done Done Sent false
1020 1171 Acked Read 7 Done Done Sent false
1021 1172 Acked Write 4 Done Done Sent false
1022 1173 Acked Write 10 Done Done Sent false
1023 1174 Acked Write 10 Done Done Sent false
1024 1175 Acked Flush 0 Done Done Sent false
1025 1176 Acked Write 4 Done Done Sent false
1026 1177 Acked Write 2 Done Done Sent false
1027 1178 Acked Flush 0 Done Done Sent false
1028 1179 Acked Write 1 Done Done Sent false
1029 1180 Acked Read 5 Done Done Sent false
1030 1181 Acked Read 4 Done Done Sent false
1031 1182 Acked Write 3 Done Done Sent false
1032 1183 Acked Write 5 Done Done Sent false
1033 1184 Acked Read 4 Done Done Sent false
1034 1185 Acked Write 1 Done Done Sent false
1035 1186 Acked Read 9 Done Done Sent false
1036 1187 Acked Write 3 Done Done Sent false
1037 1188 Acked Read 2 Done Done Sent false
1038 1189 Acked Read 2 Done Done Sent false
1039 1190 Acked Read 10 Done Done Sent false
1040 1191 Acked Read 5 Done Done Sent false
1041 1192 Acked Flush 0 Done Done Sent false
1042 1193 Acked Write 6 Done Done Sent false
1043 1194 Acked Write 9 Done Done Sent false
1044 1195 Acked Read 2 Done Done Sent false
1045 1196 Acked Write 8 Done Done Sent false
1046 1197 Acked Write 9 Done Done Sent false
1047 1198 Acked Read 1 Done Done Sent false
1048 1199 Acked Write 7 Done Done Sent false
1049 1200 Acked Write 9 Done Done Sent false
1050 1201 Acked Read 5 Done Done Sent false
1051 1202 Acked Write 9 Done Done Sent false
1052 1203 Acked Write 4 Done Done Sent false
1053 1204 Acked Write 8 Done Done Sent false
1054 1205 Acked Read 8 Done Done Sent false
1055 1206 Acked Read 2 Done Done Sent false
1056 1207 Acked Write 5 Done Done Sent false
1057 1208 Acked Read 2 Done Done Sent false
1058 1209 Acked Read 5 Done Done Sent false
1059 1210 Acked Read 7 Done Done Sent false
1060 1211 Acked Read 3 Done Done Sent false
1061 1212 Acked Write 4 Done Done Sent false
1062 1213 Acked Read 8 Done Done Sent false
1063 1214 Acked Write 7 Done Done Sent false
1064 1215 Acked Read 6 Done Done Sent false
1065 1216 Acked Write 1 Done Done Sent false
1066 1217 Acked Read 5 Done Done Sent false
1067 1218 Acked Write 4 Done Done Sent false
1068 1219 Acked Read 10 Done Done Sent false
1069 1220 Acked Read 10 Done Done Sent false
1070 1221 Acked Write 2 Done Done Sent false
1071 1222 Acked Read 8 Done Done Sent false
1072 1223 Acked Read 1 Done Done Sent false
1073 1224 Acked Read 10 Done Done Sent false
1074 1225 Acked Write 7 Done Done Sent false
1075 1226 Acked Read 8 Done Done Sent false
1076 1227 Acked Read 6 Done Done Sent false
1077 1228 Acked Read 3 Done Done Sent false
1078 1229 Acked Read 8 Done Done Sent false
1079 1230 Acked Read 3 Done Done Sent false
1080 1231 Acked Write 8 Done Done Sent false
1081 1232 Acked Write 4 Done Done Sent false
1082 1233 Acked Read 6 Done Done Sent false
1083 1234 Acked Write 10 Done Done Sent false
1084 1235 Acked Read 7 Done Done Sent false
1085 1236 Acked Read 2 Done Done Sent false
1086 1237 Acked Write 6 Done Done Sent false
1087 1238 Acked Write 9 Done Done Sent false
1088 1239 Acked Read 6 Done Done Sent false
1089 1240 Acked Write 4 Done Done Sent false
1090 1241 Acked Write 9 Done Done Sent false
1091 1242 Acked Write 4 Done Done Sent false
1092 1243 Acked Read 1 Done Done Sent false
1093 1244 Acked Read 7 Done Done Sent false
1094 1245 Acked Read 8 Done Done Sent false
1095 1246 Acked Write 1 Done Done Sent false
1096 1247 Acked Write 4 Done Done Sent false
1097 1248 Acked Write 10 Done Done Sent false
1098 1249 Acked Read 4 Done Done Sent false
1099 1250 Acked Flush 0 Done Done Sent false
1100 1251 Acked Flush 0 Done Done Sent false
1101 1252 Acked Read 2 Done Done Sent false
1102 1253 Acked Write 3 Done Done Sent false
1103 1254 Acked Read 8 Done Done Sent false
1104 1255 Acked Write 7 Done Done Sent false
1105 1256 Acked Write 2 Done Done Sent false
1106 1257 Acked Write 8 Done Done Sent false
1107 1258 Acked Read 1 Done Done Sent false
1108 1259 Acked Flush 0 Done Done Sent false
1109 1260 Acked Read 8 Done Done Sent false
1110 1261 Acked Write 3 Done Done Sent false
1111 1262 Acked Write 9 Done Done Sent false
1112 1263 Acked Flush 0 Done Done Sent false
1113 1264 Acked Read 3 Done Done Sent false
1114 1265 Acked Read 9 Done Done Sent false
1115 1266 Acked Write 9 Done Done Sent false
1116 1267 Acked Read 9 Done Done Sent false
1117 1268 Acked Write 2 Done Done Sent false
1118 1269 Acked Read 10 Done Done Sent false
1119 1270 Acked Write 6 Done Done Sent false
1120 1271 Acked Read 1 Done Done Sent false
1121 1272 Acked Write 6 Done Done Sent false
1122 1273 Acked Flush 0 Done Done Sent false
1123 1274 Acked Read 3 Done Done Sent false
1124 1275 Acked Write 4 Done Done Sent false
1125 1276 Acked Read 6 Done Done Sent false
1126 1277 Acked Write 2 Done Done Sent false
1127 1278 Acked Read 7 Done Done Sent false
1128 1279 Acked Flush 0 Done Done Sent false
1129 1280 Acked Read 8 Done Done Sent false
1130 1281 Acked Write 5 Done Done Sent false
1131 1282 Acked Write 6 Done Done Sent false
1132 1283 Acked Write 7 Done Done Sent false
1133 1284 Acked Read 2 Done Done Sent false
1134 1285 Acked Read 5 Done Done Sent false
1135 1286 Acked Read 8 Done Done Sent false
1136 1287 Acked Read 8 Done Done Sent false
1137 1288 Acked Read 9 Done Done Sent false
1138 1289 Acked Read 6 Done Done Sent false
1139 1290 Acked Write 2 Done Done Sent false
1140 1291 Acked Read 2 Done Done Sent false
1141 1292 Acked Write 3 Done Done Sent false
1142 1293 Acked Write 9 Done Done Sent false
1143 1294 Acked Write 5 Done Done Sent false
1144 1295 Acked Read 10 Done Done Sent false
1145 1296 Acked Read 8 Done Done Sent false
1146 1297 Acked Read 6 Done Done Sent false
1147 1298 Acked Read 9 Done Done Sent false
1148 1299 Acked Read 7 Done Done Sent false
1149 1300 Acked Read 10 Done Done Sent false
1150 1301 Acked Flush 0 Done Done Sent false
1151 STATES DS:0 DS:1 DS:2 TOTAL
1152 Sent 0 0 302 302
1153 Done 302 302 0 604
1154 Skipped 0 0 0 0
1155 Error 0 0 0 0
1156 Last Flush: 1301 1301 0
1157 Downstairs last five completed:
1158 Upstairs last five completed: 1301 1300 1299 1298 1297
1159 CLIENT: Up:0 ds:302 act:2
11602025-01-21T15:47:23.299ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
11612025-01-21T15:47:23.299ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
11622025-01-21T15:47:23.299ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
11632025-01-21T15:47:23.299ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
11642025-01-21T15:47:23.299ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1165 [8830] got start action
1166 [8830] I am going to restart
1167 Starting downstairs at port 8830
1168 Make output file at "/tmp/dsc/downstairs-8830.txt"
1169 [8830][2] reports Starting
1170 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1201)
1171 [8830][2] reports Running
1172 ----------------------------------------------------------------
1173 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1174 States: Active Active Offline
1175 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1176 1000 Acked Read 10 Done Done Sent false
1177 1001 Acked Flush 0 Done Done Sent false
1178 1002 Acked Read 2 Done Done Sent false
1179 1003 Acked Write 6 Done Done Sent false
1180 1004 Acked Write 1 Done Done Sent false
1181 1005 Acked Flush 0 Done Done Sent false
1182 1006 Acked Read 8 Done Done Sent false
1183 1007 Acked Read 4 Done Done Sent false
1184 1008 Acked Write 4 Done Done Sent false
1185 1009 Acked Write 10 Done Done Sent false
1186 1010 Acked Flush 0 Done Done Sent false
1187 1011 Acked Write 9 Done Done Sent false
1188 1012 Acked Read 3 Done Done Sent false
1189 1013 Acked Read 10 Done Done Sent false
1190 1014 Acked Read 6 Done Done Sent false
1191 1015 Acked Read 6 Done Done Sent false
1192 1016 Acked Read 4 Done Done Sent false
1193 1017 Acked Write 8 Done Done Sent false
1194 1018 Acked Read 9 Done Done Sent false
1195 1019 Acked Write 1 Done Done Sent false
1196 1020 Acked Flush 0 Done Done Sent false
1197 1021 Acked Read 2 Done Done Sent false
1198 1022 Acked Write 6 Done Done Sent false
1199 1023 Acked Read 5 Done Done Sent false
1200 1024 Acked Write 7 Done Done Sent false
1201 1025 Acked Write 8 Done Done Sent false
1202 1026 Acked Read 5 Done Done Sent false
1203 1027 Acked Flush 0 Done Done Sent false
1204 1028 Acked Read 3 Done Done Sent false
1205 1029 Acked Read 7 Done Done Sent false
1206 1030 Acked Read 10 Done Done Sent false
1207 1031 Acked Flush 0 Done Done Sent false
1208 1032 Acked Read 8 Done Done Sent false
1209 1033 Acked Write 1 Done Done Sent false
1210 1034 Acked Flush 0 Done Done Sent false
1211 1035 Acked Read 6 Done Done Sent false
1212 1036 Acked Read 10 Done Done Sent false
1213 1037 Acked Read 4 Done Done Sent false
1214 1038 Acked Read 9 Done Done Sent false
1215 1039 Acked Read 7 Done Done Sent false
1216 1040 Acked Write 1 Done Done Sent false
1217 1041 Acked Flush 0 Done Done Sent false
1218 1042 Acked Write 2 Done Done Sent false
1219 1043 Acked Write 8 Done Done Sent false
1220 1044 Acked Flush 0 Done Done Sent false
1221 1045 Acked Write 4 Done Done Sent false
1222 1046 Acked Read 1 Done Done Sent false
1223 1047 Acked Write 9 Done Done Sent false
1224 1048 Acked Read 10 Done Done Sent false
1225 1049 Acked Read 6 Done Done Sent false
1226 1050 Acked Read 8 Done Done Sent false
1227 1051 Acked Read 7 Done Done Sent false
1228 1052 Acked Write 8 Done Done Sent false
1229 1053 Acked Read 9 Done Done Sent false
1230 1054 Acked Write 10 Done Done Sent false
1231 1055 Acked Flush 0 Done Done Sent false
1232 1056 Acked Read 2 Done Done Sent false
1233 1057 Acked Read 8 Done Done Sent false
1234 1058 Acked Flush 0 Done Done Sent false
1235 1059 Acked Write 7 Done Done Sent false
1236 1060 Acked Read 10 Done Done Sent false
1237 1061 Acked Read 6 Done Done Sent false
1238 1062 Acked Write 7 Done Done Sent false
1239 1063 Acked Write 4 Done Done Sent false
1240 1064 Acked Read 2 Done Done Sent false
1241 1065 Acked Write 5 Done Done Sent false
1242 1066 Acked Write 6 Done Done Sent false
1243 1067 Acked Read 5 Done Done Sent false
1244 1068 Acked Write 3 Done Done Sent false
1245 1069 Acked Flush 0 Done Done Sent false
1246 1070 Acked Write 10 Done Done Sent false
1247 1071 Acked Read 2 Done Done Sent false
1248 1072 Acked Read 5 Done Done Sent false
1249 1073 Acked Read 4 Done Done Sent false
1250 1074 Acked Write 10 Done Done Sent false
1251 1075 Acked Write 5 Done Done Sent false
1252 1076 Acked Write 8 Done Done Sent false
1253 1077 Acked Read 6 Done Done Sent false
1254 1078 Acked Write 2 Done Done Sent false
1255 1079 Acked Write 4 Done Done Sent false
1256 1080 Acked Read 8 Done Done Sent false
1257 1081 Acked Read 4 Done Done Sent false
1258 1082 Acked Read 10 Done Done Sent false
1259 1083 Acked Write 3 Done Done Sent false
1260 1084 Acked Read 5 Done Done Sent false
1261 1085 Acked Flush 0 Done Done Sent false
1262 1086 Acked Write 1 Done Done Sent false
1263 1087 Acked Read 6 Done Done Sent false
1264 1088 Acked Read 2 Done Done Sent false
1265 1089 Acked Read 9 Done Done Sent false
1266 1090 Acked Write 4 Done Done Sent false
1267 1091 Acked Write 8 Done Done Sent false
1268 1092 Acked Write 7 Done Done Sent false
1269 1093 Acked Read 9 Done Done Sent false
1270 1094 Acked Write 5 Done Done Sent false
1271 1095 Acked Flush 0 Done Done Sent false
1272 1096 Acked Read 5 Done Done Sent false
1273 1097 Acked Write 9 Done Done Sent false
1274 1098 Acked Read 6 Done Done Sent false
1275 1099 Acked Write 5 Done Done Sent false
1276 1100 Acked Write 10 Done Done Sent false
1277 1101 Acked Write 7 Done Done Sent false
1278 1102 Acked Read 8 Done Done Sent false
1279 1103 Acked Flush 0 Done Done Sent false
1280 1104 Acked Read 3 Done Done Sent false
1281 1105 Acked Write 2 Done Done Sent false
1282 1106 Acked Flush 0 Done Done Sent false
1283 1107 Acked Read 6 Done Done Sent false
1284 1108 Acked Read 5 Done Done Sent false
1285 1109 Acked Read 10 Done Done Sent false
1286 1110 Acked Write 8 Done Done Sent false
1287 1111 Acked Flush 0 Done Done Sent false
1288 1112 Acked Read 3 Done Done Sent false
1289 1113 Acked Read 1 Done Done Sent false
1290 1114 Acked Write 2 Done Done Sent false
1291 1115 Acked Flush 0 Done Done Sent false
1292 1116 Acked Write 7 Done Done Sent false
1293 1117 Acked Read 6 Done Done Sent false
1294 1118 Acked Read 6 Done Done Sent false
1295 1119 Acked Flush 0 Done Done Sent false
1296 1120 Acked Write 6 Done Done Sent false
1297 1121 Acked Read 7 Done Done Sent false
1298 1122 Acked Read 2 Done Done Sent false
1299 1123 Acked Read 8 Done Done Sent false
1300 1124 Acked Write 7 Done Done Sent false
1301 1125 Acked Write 3 Done Done Sent false
1302 1126 Acked Write 3 Done Done Sent false
1303 1127 Acked Read 1 Done Done Sent false
1304 1128 Acked Flush 0 Done Done Sent false
1305 1129 Acked Write 5 Done Done Sent false
1306 1130 Acked Read 6 Done Done Sent false
1307 1131 Acked Read 5 Done Done Sent false
1308 1132 Acked Read 4 Done Done Sent false
1309 1133 Acked Read 10 Done Done Sent false
1310 1134 Acked Write 10 Done Done Sent false
1311 1135 Acked Read 9 Done Done Sent false
1312 1136 Acked Write 1 Done Done Sent false
1313 1137 Acked Read 6 Done Done Sent false
1314 1138 Acked Write 10 Done Done Sent false
1315 1139 Acked Read 2 Done Done Sent false
1316 1140 Acked Read 3 Done Done Sent false
1317 1141 Acked Flush 0 Done Done Sent false
1318 1142 Acked Read 3 Done Done Sent false
1319 1143 Acked Write 7 Done Done Sent false
1320 1144 Acked Read 3 Done Done Sent false
1321 1145 Acked Read 10 Done Done Sent false
1322 1146 Acked Flush 0 Done Done Sent false
1323 1147 Acked Write 8 Done Done Sent false
1324 1148 Acked Write 10 Done Done Sent false
1325 1149 Acked Read 8 Done Done Sent false
1326 1150 Acked Flush 0 Done Done Sent false
1327 1151 Acked Write 4 Done Done Sent false
1328 1152 Acked Read 1 Done Done Sent false
1329 1153 Acked Flush 0 Done Done Sent false
1330 1154 Acked Read 2 Done Done Sent false
1331 1155 Acked Read 3 Done Done Sent false
1332 1156 Acked Write 5 Done Done Sent false
1333 1157 Acked Write 10 Done Done Sent false
1334 1158 Acked Write 6 Done Done Sent false
1335 1159 Acked Read 2 Done Done Sent false
1336 1160 Acked Write 5 Done Done Sent false
1337 1161 Acked Read 6 Done Done Sent false
1338 1162 Acked Write 6 Done Done Sent false
1339 1163 Acked Read 4 Done Done Sent false
1340 1164 Acked Read 7 Done Done Sent false
1341 1165 Acked Write 6 Done Done Sent false
1342 1166 Acked Flush 0 Done Done Sent false
1343 1167 Acked Read 8 Done Done Sent false
1344 1168 Acked Read 1 Done Done Sent false
1345 1169 Acked Flush 0 Done Done Sent false
1346 1170 Acked Write 1 Done Done Sent false
1347 1171 Acked Read 7 Done Done Sent false
1348 1172 Acked Write 4 Done Done Sent false
1349 1173 Acked Write 10 Done Done Sent false
1350 1174 Acked Write 10 Done Done Sent false
1351 1175 Acked Flush 0 Done Done Sent false
1352 1176 Acked Write 4 Done Done Sent false
1353 1177 Acked Write 2 Done Done Sent false
1354 1178 Acked Flush 0 Done Done Sent false
1355 1179 Acked Write 1 Done Done Sent false
1356 1180 Acked Read 5 Done Done Sent false
1357 1181 Acked Read 4 Done Done Sent false
1358 1182 Acked Write 3 Done Done Sent false
1359 1183 Acked Write 5 Done Done Sent false
1360 1184 Acked Read 4 Done Done Sent false
1361 1185 Acked Write 1 Done Done Sent false
1362 1186 Acked Read 9 Done Done Sent false
1363 1187 Acked Write 3 Done Done Sent false
1364 1188 Acked Read 2 Done Done Sent false
1365 1189 Acked Read 2 Done Done Sent false
1366 1190 Acked Read 10 Done Done Sent false
1367 1191 Acked Read 5 Done Done Sent false
1368 1192 Acked Flush 0 Done Done Sent false
1369 1193 Acked Write 6 Done Done Sent false
1370 1194 Acked Write 9 Done Done Sent false
1371 1195 Acked Read 2 Done Done Sent false
1372 1196 Acked Write 8 Done Done Sent false
1373 1197 Acked Write 9 Done Done Sent false
1374 1198 Acked Read 1 Done Done Sent false
1375 1199 Acked Write 7 Done Done Sent false
1376 1200 Acked Write 9 Done Done Sent false
1377 1201 Acked Read 5 Done Done Sent false
1378 1202 Acked Write 9 Done Done Sent false
1379 1203 Acked Write 4 Done Done Sent false
1380 1204 Acked Write 8 Done Done Sent false
1381 1205 Acked Read 8 Done Done Sent false
1382 1206 Acked Read 2 Done Done Sent false
1383 1207 Acked Write 5 Done Done Sent false
1384 1208 Acked Read 2 Done Done Sent false
1385 1209 Acked Read 5 Done Done Sent false
1386 1210 Acked Read 7 Done Done Sent false
1387 1211 Acked Read 3 Done Done Sent false
1388 1212 Acked Write 4 Done Done Sent false
1389 1213 Acked Read 8 Done Done Sent false
1390 1214 Acked Write 7 Done Done Sent false
1391 1215 Acked Read 6 Done Done Sent false
1392 1216 Acked Write 1 Done Done Sent false
1393 1217 Acked Read 5 Done Done Sent false
1394 1218 Acked Write 4 Done Done Sent false
1395 1219 Acked Read 10 Done Done Sent false
1396 1220 Acked Read 10 Done Done Sent false
1397 1221 Acked Write 2 Done Done Sent false
1398 1222 Acked Read 8 Done Done Sent false
1399 1223 Acked Read 1 Done Done Sent false
1400 1224 Acked Read 10 Done Done Sent false
1401 1225 Acked Write 7 Done Done Sent false
1402 1226 Acked Read 8 Done Done Sent false
1403 1227 Acked Read 6 Done Done Sent false
1404 1228 Acked Read 3 Done Done Sent false
1405 1229 Acked Read 8 Done Done Sent false
1406 1230 Acked Read 3 Done Done Sent false
1407 1231 Acked Write 8 Done Done Sent false
1408 1232 Acked Write 4 Done Done Sent false
1409 1233 Acked Read 6 Done Done Sent false
1410 1234 Acked Write 10 Done Done Sent false
1411 1235 Acked Read 7 Done Done Sent false
1412 1236 Acked Read 2 Done Done Sent false
1413 1237 Acked Write 6 Done Done Sent false
1414 1238 Acked Write 9 Done Done Sent false
1415 1239 Acked Read 6 Done Done Sent false
1416 1240 Acked Write 4 Done Done Sent false
1417 1241 Acked Write 9 Done Done Sent false
1418 1242 Acked Write 4 Done Done Sent false
1419 1243 Acked Read 1 Done Done Sent false
1420 1244 Acked Read 7 Done Done Sent false
1421 1245 Acked Read 8 Done Done Sent false
1422 1246 Acked Write 1 Done Done Sent false
1423 1247 Acked Write 4 Done Done Sent false
1424 1248 Acked Write 10 Done Done Sent false
1425 1249 Acked Read 4 Done Done Sent false
1426 1250 Acked Flush 0 Done Done Sent false
1427 1251 Acked Flush 0 Done Done Sent false
1428 1252 Acked Read 2 Done Done Sent false
1429 1253 Acked Write 3 Done Done Sent false
1430 1254 Acked Read 8 Done Done Sent false
1431 1255 Acked Write 7 Done Done Sent false
1432 1256 Acked Write 2 Done Done Sent false
1433 1257 Acked Write 8 Done Done Sent false
1434 1258 Acked Read 1 Done Done Sent false
1435 1259 Acked Flush 0 Done Done Sent false
1436 1260 Acked Read 8 Done Done Sent false
1437 1261 Acked Write 3 Done Done Sent false
1438 1262 Acked Write 9 Done Done Sent false
1439 1263 Acked Flush 0 Done Done Sent false
1440 1264 Acked Read 3 Done Done Sent false
1441 1265 Acked Read 9 Done Done Sent false
1442 1266 Acked Write 9 Done Done Sent false
1443 1267 Acked Read 9 Done Done Sent false
1444 1268 Acked Write 2 Done Done Sent false
1445 1269 Acked Read 10 Done Done Sent false
1446 1270 Acked Write 6 Done Done Sent false
1447 1271 Acked Read 1 Done Done Sent false
1448 1272 Acked Write 6 Done Done Sent false
1449 1273 Acked Flush 0 Done Done Sent false
1450 1274 Acked Read 3 Done Done Sent false
1451 1275 Acked Write 4 Done Done Sent false
1452 1276 Acked Read 6 Done Done Sent false
1453 1277 Acked Write 2 Done Done Sent false
1454 1278 Acked Read 7 Done Done Sent false
1455 1279 Acked Flush 0 Done Done Sent false
1456 1280 Acked Read 8 Done Done Sent false
1457 1281 Acked Write 5 Done Done Sent false
1458 1282 Acked Write 6 Done Done Sent false
1459 1283 Acked Write 7 Done Done Sent false
1460 1284 Acked Read 2 Done Done Sent false
1461 1285 Acked Read 5 Done Done Sent false
1462 1286 Acked Read 8 Done Done Sent false
1463 1287 Acked Read 8 Done Done Sent false
1464 1288 Acked Read 9 Done Done Sent false
1465 1289 Acked Read 6 Done Done Sent false
1466 1290 Acked Write 2 Done Done Sent false
1467 1291 Acked Read 2 Done Done Sent false
1468 1292 Acked Write 3 Done Done Sent false
1469 1293 Acked Write 9 Done Done Sent false
1470 1294 Acked Write 5 Done Done Sent false
1471 1295 Acked Read 10 Done Done Sent false
1472 1296 Acked Read 8 Done Done Sent false
1473 1297 Acked Read 6 Done Done Sent false
1474 1298 Acked Read 9 Done Done Sent false
1475 1299 Acked Read 7 Done Done Sent false
1476 1300 Acked Read 10 Done Done Sent false
1477 1301 Acked Flush 0 Done Done Sent false
1478 STATES DS:0 DS:1 DS:2 TOTAL
1479 Sent 0 0 302 302
1480 Done 302 302 0 604
1481 Skipped 0 0 0 0
1482 Error 0 0 0 0
1483 Last Flush: 1301 1301 0
1484 Downstairs last five completed:
1485 Upstairs last five completed: 1301 1300 1299 1298 1297
1486 CLIENT: Up:0 ds:302 act:2
1487 ----------------------------------------------------------------
1488 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1489 States: Active Active Offline
1490 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1491 1000 Acked Read 10 Done Done Sent false
1492 1001 Acked Flush 0 Done Done Sent false
1493 1002 Acked Read 2 Done Done Sent false
1494 1003 Acked Write 6 Done Done Sent false
1495 1004 Acked Write 1 Done Done Sent false
1496 1005 Acked Flush 0 Done Done Sent false
1497 1006 Acked Read 8 Done Done Sent false
1498 1007 Acked Read 4 Done Done Sent false
1499 1008 Acked Write 4 Done Done Sent false
1500 1009 Acked Write 10 Done Done Sent false
1501 1010 Acked Flush 0 Done Done Sent false
1502 1011 Acked Write 9 Done Done Sent false
1503 1012 Acked Read 3 Done Done Sent false
1504 1013 Acked Read 10 Done Done Sent false
1505 1014 Acked Read 6 Done Done Sent false
1506 1015 Acked Read 6 Done Done Sent false
1507 1016 Acked Read 4 Done Done Sent false
1508 1017 Acked Write 8 Done Done Sent false
1509 1018 Acked Read 9 Done Done Sent false
1510 1019 Acked Write 1 Done Done Sent false
1511 1020 Acked Flush 0 Done Done Sent false
1512 1021 Acked Read 2 Done Done Sent false
1513 1022 Acked Write 6 Done Done Sent false
1514 1023 Acked Read 5 Done Done Sent false
1515 1024 Acked Write 7 Done Done Sent false
1516 1025 Acked Write 8 Done Done Sent false
1517 1026 Acked Read 5 Done Done Sent false
1518 1027 Acked Flush 0 Done Done Sent false
1519 1028 Acked Read 3 Done Done Sent false
1520 1029 Acked Read 7 Done Done Sent false
1521 1030 Acked Read 10 Done Done Sent false
1522 1031 Acked Flush 0 Done Done Sent false
1523 1032 Acked Read 8 Done Done Sent false
1524 1033 Acked Write 1 Done Done Sent false
1525 1034 Acked Flush 0 Done Done Sent false
1526 1035 Acked Read 6 Done Done Sent false
1527 1036 Acked Read 10 Done Done Sent false
1528 1037 Acked Read 4 Done Done Sent false
1529 1038 Acked Read 9 Done Done Sent false
1530 1039 Acked Read 7 Done Done Sent false
1531 1040 Acked Write 1 Done Done Sent false
1532 1041 Acked Flush 0 Done Done Sent false
1533 1042 Acked Write 2 Done Done Sent false
1534 1043 Acked Write 8 Done Done Sent false
1535 1044 Acked Flush 0 Done Done Sent false
1536 1045 Acked Write 4 Done Done Sent false
1537 1046 Acked Read 1 Done Done Sent false
1538 1047 Acked Write 9 Done Done Sent false
1539 1048 Acked Read 10 Done Done Sent false
1540 1049 Acked Read 6 Done Done Sent false
1541 1050 Acked Read 8 Done Done Sent false
1542 1051 Acked Read 7 Done Done Sent false
1543 1052 Acked Write 8 Done Done Sent false
1544 1053 Acked Read 9 Done Done Sent false
1545 1054 Acked Write 10 Done Done Sent false
1546 1055 Acked Flush 0 Done Done Sent false
1547 1056 Acked Read 2 Done Done Sent false
1548 1057 Acked Read 8 Done Done Sent false
1549 1058 Acked Flush 0 Done Done Sent false
1550 1059 Acked Write 7 Done Done Sent false
1551 1060 Acked Read 10 Done Done Sent false
1552 1061 Acked Read 6 Done Done Sent false
1553 1062 Acked Write 7 Done Done Sent false
1554 1063 Acked Write 4 Done Done Sent false
1555 1064 Acked Read 2 Done Done Sent false
1556 1065 Acked Write 5 Done Done Sent false
1557 1066 Acked Write 6 Done Done Sent false
1558 1067 Acked Read 5 Done Done Sent false
1559 1068 Acked Write 3 Done Done Sent false
1560 1069 Acked Flush 0 Done Done Sent false
1561 1070 Acked Write 10 Done Done Sent false
1562 1071 Acked Read 2 Done Done Sent false
1563 1072 Acked Read 5 Done Done Sent false
1564 1073 Acked Read 4 Done Done Sent false
1565 1074 Acked Write 10 Done Done Sent false
1566 1075 Acked Write 5 Done Done Sent false
1567 1076 Acked Write 8 Done Done Sent false
1568 1077 Acked Read 6 Done Done Sent false
1569 1078 Acked Write 2 Done Done Sent false
1570 1079 Acked Write 4 Done Done Sent false
1571 1080 Acked Read 8 Done Done Sent false
1572 1081 Acked Read 4 Done Done Sent false
1573 1082 Acked Read 10 Done Done Sent false
1574 1083 Acked Write 3 Done Done Sent false
1575 1084 Acked Read 5 Done Done Sent false
1576 1085 Acked Flush 0 Done Done Sent false
1577 1086 Acked Write 1 Done Done Sent false
1578 1087 Acked Read 6 Done Done Sent false
1579 1088 Acked Read 2 Done Done Sent false
1580 1089 Acked Read 9 Done Done Sent false
1581 1090 Acked Write 4 Done Done Sent false
1582 1091 Acked Write 8 Done Done Sent false
1583 1092 Acked Write 7 Done Done Sent false
1584 1093 Acked Read 9 Done Done Sent false
1585 1094 Acked Write 5 Done Done Sent false
1586 1095 Acked Flush 0 Done Done Sent false
1587 1096 Acked Read 5 Done Done Sent false
1588 1097 Acked Write 9 Done Done Sent false
1589 1098 Acked Read 6 Done Done Sent false
1590 1099 Acked Write 5 Done Done Sent false
1591 1100 Acked Write 10 Done Done Sent false
1592 1101 Acked Write 7 Done Done Sent false
1593 1102 Acked Read 8 Done Done Sent false
1594 1103 Acked Flush 0 Done Done Sent false
1595 1104 Acked Read 3 Done Done Sent false
1596 1105 Acked Write 2 Done Done Sent false
1597 1106 Acked Flush 0 Done Done Sent false
1598 1107 Acked Read 6 Done Done Sent false
1599 1108 Acked Read 5 Done Done Sent false
1600 1109 Acked Read 10 Done Done Sent false
1601 1110 Acked Write 8 Done Done Sent false
1602 1111 Acked Flush 0 Done Done Sent false
1603 1112 Acked Read 3 Done Done Sent false
1604 1113 Acked Read 1 Done Done Sent false
1605 1114 Acked Write 2 Done Done Sent false
1606 1115 Acked Flush 0 Done Done Sent false
1607 1116 Acked Write 7 Done Done Sent false
1608 1117 Acked Read 6 Done Done Sent false
1609 1118 Acked Read 6 Done Done Sent false
1610 1119 Acked Flush 0 Done Done Sent false
1611 1120 Acked Write 6 Done Done Sent false
1612 1121 Acked Read 7 Done Done Sent false
1613 1122 Acked Read 2 Done Done Sent false
1614 1123 Acked Read 8 Done Done Sent false
1615 1124 Acked Write 7 Done Done Sent false
1616 1125 Acked Write 3 Done Done Sent false
1617 1126 Acked Write 3 Done Done Sent false
1618 1127 Acked Read 1 Done Done Sent false
1619 1128 Acked Flush 0 Done Done Sent false
1620 1129 Acked Write 5 Done Done Sent false
1621 1130 Acked Read 6 Done Done Sent false
1622 1131 Acked Read 5 Done Done Sent false
1623 1132 Acked Read 4 Done Done Sent false
1624 1133 Acked Read 10 Done Done Sent false
1625 1134 Acked Write 10 Done Done Sent false
1626 1135 Acked Read 9 Done Done Sent false
1627 1136 Acked Write 1 Done Done Sent false
1628 1137 Acked Read 6 Done Done Sent false
1629 1138 Acked Write 10 Done Done Sent false
1630 1139 Acked Read 2 Done Done Sent false
1631 1140 Acked Read 3 Done Done Sent false
1632 1141 Acked Flush 0 Done Done Sent false
1633 1142 Acked Read 3 Done Done Sent false
1634 1143 Acked Write 7 Done Done Sent false
1635 1144 Acked Read 3 Done Done Sent false
1636 1145 Acked Read 10 Done Done Sent false
1637 1146 Acked Flush 0 Done Done Sent false
1638 1147 Acked Write 8 Done Done Sent false
1639 1148 Acked Write 10 Done Done Sent false
1640 1149 Acked Read 8 Done Done Sent false
1641 1150 Acked Flush 0 Done Done Sent false
1642 1151 Acked Write 4 Done Done Sent false
1643 1152 Acked Read 1 Done Done Sent false
1644 1153 Acked Flush 0 Done Done Sent false
1645 1154 Acked Read 2 Done Done Sent false
1646 1155 Acked Read 3 Done Done Sent false
1647 1156 Acked Write 5 Done Done Sent false
1648 1157 Acked Write 10 Done Done Sent false
1649 1158 Acked Write 6 Done Done Sent false
1650 1159 Acked Read 2 Done Done Sent false
1651 1160 Acked Write 5 Done Done Sent false
1652 1161 Acked Read 6 Done Done Sent false
1653 1162 Acked Write 6 Done Done Sent false
1654 1163 Acked Read 4 Done Done Sent false
1655 1164 Acked Read 7 Done Done Sent false
1656 1165 Acked Write 6 Done Done Sent false
1657 1166 Acked Flush 0 Done Done Sent false
1658 1167 Acked Read 8 Done Done Sent false
1659 1168 Acked Read 1 Done Done Sent false
1660 1169 Acked Flush 0 Done Done Sent false
1661 1170 Acked Write 1 Done Done Sent false
1662 1171 Acked Read 7 Done Done Sent false
1663 1172 Acked Write 4 Done Done Sent false
1664 1173 Acked Write 10 Done Done Sent false
1665 1174 Acked Write 10 Done Done Sent false
1666 1175 Acked Flush 0 Done Done Sent false
1667 1176 Acked Write 4 Done Done Sent false
1668 1177 Acked Write 2 Done Done Sent false
1669 1178 Acked Flush 0 Done Done Sent false
1670 1179 Acked Write 1 Done Done Sent false
1671 1180 Acked Read 5 Done Done Sent false
1672 1181 Acked Read 4 Done Done Sent false
1673 1182 Acked Write 3 Done Done Sent false
1674 1183 Acked Write 5 Done Done Sent false
1675 1184 Acked Read 4 Done Done Sent false
1676 1185 Acked Write 1 Done Done Sent false
1677 1186 Acked Read 9 Done Done Sent false
1678 1187 Acked Write 3 Done Done Sent false
1679 1188 Acked Read 2 Done Done Sent false
1680 1189 Acked Read 2 Done Done Sent false
1681 1190 Acked Read 10 Done Done Sent false
1682 1191 Acked Read 5 Done Done Sent false
1683 1192 Acked Flush 0 Done Done Sent false
1684 1193 Acked Write 6 Done Done Sent false
1685 1194 Acked Write 9 Done Done Sent false
1686 1195 Acked Read 2 Done Done Sent false
1687 1196 Acked Write 8 Done Done Sent false
1688 1197 Acked Write 9 Done Done Sent false
1689 1198 Acked Read 1 Done Done Sent false
1690 1199 Acked Write 7 Done Done Sent false
1691 1200 Acked Write 9 Done Done Sent false
1692 1201 Acked Read 5 Done Done Sent false
1693 1202 Acked Write 9 Done Done Sent false
1694 1203 Acked Write 4 Done Done Sent false
1695 1204 Acked Write 8 Done Done Sent false
1696 1205 Acked Read 8 Done Done Sent false
1697 1206 Acked Read 2 Done Done Sent false
1698 1207 Acked Write 5 Done Done Sent false
1699 1208 Acked Read 2 Done Done Sent false
1700 1209 Acked Read 5 Done Done Sent false
1701 1210 Acked Read 7 Done Done Sent false
1702 1211 Acked Read 3 Done Done Sent false
1703 1212 Acked Write 4 Done Done Sent false
1704 1213 Acked Read 8 Done Done Sent false
1705 1214 Acked Write 7 Done Done Sent false
1706 1215 Acked Read 6 Done Done Sent false
1707 1216 Acked Write 1 Done Done Sent false
1708 1217 Acked Read 5 Done Done Sent false
1709 1218 Acked Write 4 Done Done Sent false
1710 1219 Acked Read 10 Done Done Sent false
1711 1220 Acked Read 10 Done Done Sent false
1712 1221 Acked Write 2 Done Done Sent false
1713 1222 Acked Read 8 Done Done Sent false
1714 1223 Acked Read 1 Done Done Sent false
1715 1224 Acked Read 10 Done Done Sent false
1716 1225 Acked Write 7 Done Done Sent false
1717 1226 Acked Read 8 Done Done Sent false
1718 1227 Acked Read 6 Done Done Sent false
1719 1228 Acked Read 3 Done Done Sent false
1720 1229 Acked Read 8 Done Done Sent false
1721 1230 Acked Read 3 Done Done Sent false
1722 1231 Acked Write 8 Done Done Sent false
1723 1232 Acked Write 4 Done Done Sent false
1724 1233 Acked Read 6 Done Done Sent false
1725 1234 Acked Write 10 Done Done Sent false
1726 1235 Acked Read 7 Done Done Sent false
1727 1236 Acked Read 2 Done Done Sent false
1728 1237 Acked Write 6 Done Done Sent false
1729 1238 Acked Write 9 Done Done Sent false
1730 1239 Acked Read 6 Done Done Sent false
1731 1240 Acked Write 4 Done Done Sent false
1732 1241 Acked Write 9 Done Done Sent false
1733 1242 Acked Write 4 Done Done Sent false
1734 1243 Acked Read 1 Done Done Sent false
1735 1244 Acked Read 7 Done Done Sent false
1736 1245 Acked Read 8 Done Done Sent false
1737 1246 Acked Write 1 Done Done Sent false
1738 1247 Acked Write 4 Done Done Sent false
1739 1248 Acked Write 10 Done Done Sent false
1740 1249 Acked Read 4 Done Done Sent false
1741 1250 Acked Flush 0 Done Done Sent false
1742 1251 Acked Flush 0 Done Done Sent false
1743 1252 Acked Read 2 Done Done Sent false
1744 1253 Acked Write 3 Done Done Sent false
1745 1254 Acked Read 8 Done Done Sent false
1746 1255 Acked Write 7 Done Done Sent false
1747 1256 Acked Write 2 Done Done Sent false
1748 1257 Acked Write 8 Done Done Sent false
1749 1258 Acked Read 1 Done Done Sent false
1750 1259 Acked Flush 0 Done Done Sent false
1751 1260 Acked Read 8 Done Done Sent false
1752 1261 Acked Write 3 Done Done Sent false
1753 1262 Acked Write 9 Done Done Sent false
1754 1263 Acked Flush 0 Done Done Sent false
1755 1264 Acked Read 3 Done Done Sent false
1756 1265 Acked Read 9 Done Done Sent false
1757 1266 Acked Write 9 Done Done Sent false
1758 1267 Acked Read 9 Done Done Sent false
1759 1268 Acked Write 2 Done Done Sent false
1760 1269 Acked Read 10 Done Done Sent false
1761 1270 Acked Write 6 Done Done Sent false
1762 1271 Acked Read 1 Done Done Sent false
1763 1272 Acked Write 6 Done Done Sent false
1764 1273 Acked Flush 0 Done Done Sent false
1765 1274 Acked Read 3 Done Done Sent false
1766 1275 Acked Write 4 Done Done Sent false
1767 1276 Acked Read 6 Done Done Sent false
1768 1277 Acked Write 2 Done Done Sent false
1769 1278 Acked Read 7 Done Done Sent false
1770 1279 Acked Flush 0 Done Done Sent false
1771 1280 Acked Read 8 Done Done Sent false
1772 1281 Acked Write 5 Done Done Sent false
1773 1282 Acked Write 6 Done Done Sent false
1774 1283 Acked Write 7 Done Done Sent false
1775 1284 Acked Read 2 Done Done Sent false
1776 1285 Acked Read 5 Done Done Sent false
1777 1286 Acked Read 8 Done Done Sent false
1778 1287 Acked Read 8 Done Done Sent false
1779 1288 Acked Read 9 Done Done Sent false
1780 1289 Acked Read 6 Done Done Sent false
1781 1290 Acked Write 2 Done Done Sent false
1782 1291 Acked Read 2 Done Done Sent false
1783 1292 Acked Write 3 Done Done Sent false
1784 1293 Acked Write 9 Done Done Sent false
1785 1294 Acked Write 5 Done Done Sent false
1786 1295 Acked Read 10 Done Done Sent false
1787 1296 Acked Read 8 Done Done Sent false
1788 1297 Acked Read 6 Done Done Sent false
1789 1298 Acked Read 9 Done Done Sent false
1790 1299 Acked Read 7 Done Done Sent false
1791 1300 Acked Read 10 Done Done Sent false
1792 1301 Acked Flush 0 Done Done Sent false
1793 STATES DS:0 DS:1 DS:2 TOTAL
1794 Sent 0 0 302 302
1795 Done 302 302 0 604
1796 Skipped 0 0 0 0
1797 Error 0 0 0 0
1798 Last Flush: 1301 1301 0
1799 Downstairs last five completed:
1800 Upstairs last five completed: 1301 1300 1299 1298 1297
1801 CLIENT: Up:0 ds:302 act:2
18022025-01-21T15:47:33.301ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18032025-01-21T15:47:33.301ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40012) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18042025-01-21T15:47:33.302ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18052025-01-21T15:47:33.303ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18062025-01-21T15:47:33.303ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18072025-01-21T15:47:33.303ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18082025-01-21T15:47:33.303ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18092025-01-21T15:47:33.303ZINFOcrucible: [2] client re-new 302 jobs since flush 0 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18102025-01-21T15:47:33.303ZINFOcrucible: [2] Replayed 302 jobs since flush: 0 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1811 ----------------------------------------------------------------
1812 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1813 Downstairs last five completed: 1301 1300 1299 1298 1297
1814 Upstairs last five completed: 1301 1300 1299 1298 1297
1815 CLIENT: Up:0 ds:0 act:3
1816 Replay: All jobs finished
1817 Main task has work to do, go find it
1818 got dsc Stop(1)
1819 stop 1
1820 Jan 21 15:47:33.555 INFO request completed, latency_us: 317, response_code:[8820] Got stop action so:false kr:false
1821 204, uri: /stop/cid/1, method: POST, req_id: b86eb628-2ae0-48c0-97bb-144b80b6c3a6, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
18222025-01-21T15:47:33.556ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18232025-01-21T15:47:33.556ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18242025-01-21T15:47:33.556ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18252025-01-21T15:47:33.556ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
1826 Jan 21 15:47:33.556 INFO request completed, latency_us: 155, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3a46e6f9-5ee3-46de-a1f2-ecc87fa3dde6, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
1827 [8820] Exited with: ExitStatus(unix_wait_status(9))
1828 [8820][1] reports Exit
1829 Jan 21 15:47:37.559 INFO request completed, latency_us: 198, response_code: 200, uri: /state/cid/1, method: GET, req_id: f5b56b6f-8741-4d91-a0c4-9337f9e63159, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
1830 001/300 Write block 479 len 36864 data: 2 2 2 2 2 2 2 2 2
1831 002/300 Read block 1871 len 20480
1832 003/300 Read block 1459 len 16384
1833 004/300 Read block 4644 len 12288
1834 005/300 Write block 181 len 36864 data: 2 2 2 2 2 2 2 2 2
1835 006/300 Write block 618 len 28672 data: 2 2 2 2 2 2 2
1836 007/300 Read block 3165 len 40960
1837 008/300 Read block 1636 len 28672
1838 009/300 Write block 3622 len 40960 data: 2 2 2 2 2 2 2 3 2 2
1839 010/300 Read block 1470 len 8192
1840 011/300 Write block 2527 len 32768 data: 2 2 2 2 2 2 2 2
1841 012/300 Read block 2197 len 28672
1842 013/300 Read block 994 len 4096
1843 014/300 Flush
1844 015/300 Read block 708 len 8192
1845 016/300 Read block 1283 len 8192
1846 017/300 Write block 186 len 12288 data: 3 3 3
1847 018/300 Write block 1640 len 20480 data: 2 2 2 2 2
1848 019/300 Write block 2197 len 20480 data: 2 2 2 2 2
1849 020/300 Write block 3536 len 28672 data: 2 2 2 2 2 2 2
1850 021/300 Read block 872 len 28672
1851 022/300 Write block 541 len 4096 data: 2
1852 023/300 Write block 207 len 28672 data: 3 3 3 2 2 2 3
1853 024/300 Write block 37 len 4096 data: 3
1854 025/300 Read block 3305 len 16384
1855 026/300 Read block 3144 len 28672
1856 027/300 Read block 2024 len 36864
1857 028/300 Read block 695 len 36864
1858 029/300 Write block 3082 len 28672 data: 2 2 2 2 2 2 2
1859 030/300 Read block 571 len 8192
1860 031/300 Read block 338 len 4096
1861 032/300 Read block 1464 len 40960
1862 033/300 Read block 2732 len 12288
1863 034/300 Read block 1166 len 24576
1864 035/300 Read block 3092 len 40960
1865 036/300 Write block 3853 len 12288 data: 2 2 2
1866 037/300 Read block 3540 len 16384
1867 038/300 Write block 2521 len 24576 data: 2 2 2 2 2 2
1868 039/300 Read block 3860 len 16384
1869 040/300 Write block 2735 len 12288 data: 2 2 2
1870 041/300 Write block 1009 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1871 042/300 Read block 4840 len 32768
1872 043/300 Read block 2329 len 32768
1873 044/300 Write block 4649 len 20480 data: 2 2 2 3 3
1874 045/300 Write block 1467 len 32768 data: 2 2 2 2 2 2 2 2
1875 046/300 Write block 4179 len 8192 data: 2 2
1876 047/300 Write block 2191 len 4096 data: 3
1877 048/300 Write block 695 len 16384 data: 2 2 2 2
1878 049/300 Write block 3809 len 32768 data: 2 2 2 2 2 2 2 2
1879 050/300 Read block 3229 len 4096
1880 051/300 Write block 4178 len 12288 data: 2 3 3
1881 052/300 Write block 254 len 36864 data: 2 2 2 2 2 2 2 2 2
1882 053/300 Write block 3650 len 12288 data: 2 2 2
1883 054/300 Write block 4273 len 8192 data: 2 2
1884 055/300 Read block 3856 len 28672
1885 056/300 Write block 3148 len 20480 data: 2 2 2 2 2
1886 057/300 Write block 4410 len 36864 data: 2 2 2 2 2 2 2 2 2
1887 058/300 Write block 1433 len 16384 data: 2 2 2 2
1888 059/300 Flush
1889 060/300 Read block 3114 len 32768
1890 061/300 Read block 272 len 28672
1891 062/300 Read block 1246 len 28672
1892 063/300 Flush
1893 064/300 Flush
1894 065/300 Write block 770 len 36864 data: 3 3 3 3 3 2 2 2 2
1895 066/300 Read block 3863 len 40960
1896 067/300 Read block 1157 len 28672
1897 068/300 Flush
1898 069/300 Write block 3587 len 20480 data: 2 2 3 3 3
1899 070/300 Write block 788 len 24576 data: 2 2 2 2 2 2
1900 071/300 Flush
1901 072/300 Read block 3524 len 24576
1902 073/300 Read block 1495 len 20480
1903 074/300 Write block 1907 len 12288 data: 2 2 2
1904 075/300 Flush
1905 076/300 Read block 2760 len 8192
1906 077/300 Write block 45 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1907 078/300 Flush
1908 079/300 Write block 804 len 32768 data: 2 2 2 2 2 2 2 2
1909 080/300 Write block 4388 len 32768 data: 2 3 3 3 3 3 3 3
1910 081/300 Read block 744 len 24576
1911 082/300 Write block 2422 len 36864 data: 2 2 3 3 3 3 3 3 2
1912 083/300 Write block 2634 len 36864 data: 2 2 2 2 2 2 2 2 2
1913 084/300 Write block 1755 len 12288 data: 2 3 3
1914 085/300 Write block 3199 len 36864 data: 3 3 3 2 2 2 2 2 2
1915 086/300 Read block 2925 len 32768
1916 087/300 Write block 4231 len 28672 data: 2 2 2 2 2 2 2
1917 088/300 Write block 1847 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1918 089/300 Read block 3632 len 24576
1919 090/300 Write block 4226 len 32768 data: 2 2 2 2 2 3 3 3
1920 091/300 Read block 1927 len 4096
1921 092/300 Read block 318 len 8192
1922 093/300 Write block 1090 len 32768 data: 2 2 2 2 2 2 2 2
1923 094/300 Read block 3215 len 36864
1924 095/300 Write block 4086 len 32768 data: 2 2 2 2 2 2 2 2
1925 096/300 Write block 614 len 40960 data: 2 2 2 2 3 3 3 3 3 3
1926 097/300 Read block 4679 len 32768
1927 098/300 Write block 526 len 32768 data: 2 2 2 3 3 3 2 2
1928 099/300 Write block 2419 len 8192 data: 2 2
1929 100/300 Read block 1768 len 32768
1930 101/300 Flush
1931 102/300 Write block 911 len 32768 data: 2 2 2 2 2 2 2 2
1932 103/300 Read block 2940 len 4096
1933 104/300 Write block 144 len 32768 data: 2 2 2 2 2 2 2 2
1934 105/300 Read block 837 len 12288
1935 106/300 Read block 73 len 24576
1936 107/300 Write block 797 len 24576 data: 2 2 2 2 2 2
1937 108/300 Read block 1047 len 40960
1938 109/300 Read block 1971 len 24576
1939 110/300 Read block 1777 len 40960
1940 111/300 Write block 2446 len 8192 data: 2 2
1941 112/300 Write block 4981 len 12288 data: 2 2 2
1942 113/300 Flush
1943 114/300 Write block 2858 len 8192 data: 2 2
1944 115/300 Read block 3940 len 40960
1945 116/300 Read block 537 len 16384
1946 117/300 Read block 1835 len 16384
1947 118/300 Read block 2742 len 32768
1948 119/300 Read block 2276 len 16384
1949 120/300 Write block 4495 len 8192 data: 2 2
1950 121/300 Write block 4984 len 28672 data: 2 2 3 3 3 2 2
1951 122/300 Read block 863 len 4096
1952 123/300 Write block 1036 len 40960 data: 2 3 3 3 3 2 2 2 2 2
1953 124/300 Write block 1100 len 24576 data: 2 2 2 2 2 2
1954 125/300 Read block 2853 len 40960
1955 126/300 Write block 4309 len 12288 data: 2 2 2
1956 127/300 Write block 2894 len 12288 data: 2 2 2
1957 128/300 Read block 2069 len 4096
1958 129/300 Write block 2348 len 8192 data: 2 2
1959 130/300 Read block 4691 len 40960
1960 131/300 Read block 3461 len 40960
1961 132/300 Write block 2839 len 24576 data: 2 2 2 2 2 2
1962 133/300 Read block 1686 len 28672
1963 134/300 Read block 4694 len 36864
1964 135/300 Write block 483 len 32768 data: 3 3 3 3 3 2 2 2
1965 136/300 Write block 368 len 36864 data: 2 2 2 2 2 2 2 2 2
1966 137/300 Read block 1855 len 16384
1967 138/300 Read block 950 len 8192
1968 139/300 Write block 3459 len 24576 data: 2 2 2 2 2 2
1969 140/300 Read block 948 len 40960
1970 141/300 Read block 4376 len 28672
1971 142/300 Read block 534 len 16384
1972 143/300 Flush
1973 144/300 Read block 3587 len 28672
1974 145/300 Write block 3985 len 16384 data: 4 4 3 3
1975 146/300 Write block 4843 len 16384 data: 2 2 2 2
1976 147/300 Read block 2772 len 12288
1977 148/300 Read block 2910 len 4096
1978 149/300 Read block 3983 len 24576
1979 150/300 Read block 4273 len 4096
1980 151/300 Write block 1798 len 8192 data: 2 2
1981 152/300 Read block 3577 len 28672
1982 153/300 Write block 4583 len 12288 data: 2 2 2
1983 154/300 Flush
1984 155/300 Read block 441 len 32768
1985 156/300 Write block 4754 len 36864 data: 2 2 2 2 2 2 2 2 2
1986 157/300 Read block 3115 len 36864
1987 158/300 Read block 2645 len 8192
1988 159/300 Write block 1339 len 40960 data: 3 3 3 2 2 2 2 2 2 2
1989 160/300 Read block 3020 len 4096
1990 161/300 Read block 2245 len 8192
1991 162/300 Write block 3398 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1992 163/300 Read block 3091 len 20480
1993 164/300 Write block 3024 len 20480 data: 2 2 2 2 2
1994 165/300 Read block 3754 len 4096
1995 166/300 Read block 4930 len 8192
1996 167/300 Write block 3783 len 28672 data: 2 2 2 2 2 2 2
1997 168/300 Write block 4567 len 28672 data: 2 2 2 2 2 2 3
1998 169/300 Read block 4622 len 36864
1999 170/300 Write block 4152 len 36864 data: 2 2 2 2 2 2 2 2 2
2000 171/300 Read block 790 len 28672
2001 172/300 Read block 1953 len 32768
2002 173/300 Write block 3034 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2003 174/300 Write block 1428 len 8192 data: 3 3
2004 175/300 Read block 1027 len 4096
2005 176/300 Read block 4316 len 16384
2006 177/300 Read block 1672 len 28672
2007 178/300 Write block 380 len 20480 data: 2 2 2 2 2
2008 179/300 Write block 4999 len 4096 data: 2
2009 180/300 Read block 1482 len 4096
2010 181/300 Read block 2500 len 24576
2011 182/300 Read block 814 len 24576
2012 183/300 Write block 2951 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2013 184/300 Read block 1714 len 40960
2014 185/300 Write block 4799 len 32768 data: 3 3 2 2 2 2 2 2
2015 186/300 Write block 4181 len 28672 data: 2 2 2 2 2 2 2
2016 187/300 Write block 3252 len 32768 data: 2 2 2 2 2 2 2 2
2017 188/300 Write block 3468 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2018 189/300 Write block 404 len 24576 data: 2 2 2 2 2 2
2019 190/300 Read block 51 len 4096
2020 191/300 Read block 4254 len 12288
2021 192/300 Write block 2088 len 4096 data: 2
2022 193/300 Write block 3624 len 24576 data: 3 3 3 3 3 4
2023 194/300 Read block 3471 len 16384
2024 195/300 Write block 1591 len 16384 data: 2 2 2 2
2025 196/300 Write block 3110 len 32768 data: 2 2 2 2 2 2 2 2
2026 197/300 Read block 1936 len 24576
2027 198/300 Write block 588 len 20480 data: 2 2 2 2 2
2028 199/300 Write block 3042 len 4096 data: 3
2029 200/300 Read block 4181 len 24576
2030 201/300 Read block 4067 len 40960
2031 202/300 Read block 1377 len 8192
2032 203/300 Read block 2212 len 8192
2033 204/300 Write block 1200 len 16384 data: 2 2 2 2
2034 205/300 Write block 3490 len 20480 data: 2 2 2 2 2
2035 206/300 Write block 4597 len 4096 data: 2
2036 207/300 Write block 3208 len 8192 data: 2 2
2037 208/300 Write block 1477 len 4096 data: 2
2038 209/300 Write block 3401 len 28672 data: 3 3 3 3 3 3 3
2039 210/300 Flush
2040 211/300 Write block 3319 len 12288 data: 2 2 2
2041 212/300 Flush
2042 213/300 Write block 3497 len 16384 data: 2 2 2 2
2043 214/300 Read block 4858 len 36864
2044 215/300 Read block 2728 len 16384
2045 216/300 Write block 966 len 8192 data: 2 3
2046 217/300 Write block 1620 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2047 218/300 Write block 4388 len 20480 data: 3 4 4 4 4
2048 219/300 Read block 2855 len 16384
2049 220/300 Write block 215 len 20480 data: 3 3 3 3 3
2050 221/300 Read block 2932 len 32768
2051 222/300 Read block 736 len 20480
2052 223/300 Read block 548 len 12288
2053 224/300 Read block 4675 len 36864
2054 225/300 Write block 3574 len 36864 data: 2 2 2 2 2 2 2 2 2
2055 226/300 Read block 3759 len 4096
2056 227/300 Read block 4381 len 16384
2057 228/300 Read block 1338 len 16384
2058 229/300 Read block 502 len 24576
2059 230/300 Read block 4535 len 24576
2060 231/300 Read block 743 len 16384
2061 232/300 Write block 453 len 20480 data: 2 2 2 2 2
2062 233/300 Write block 4995 len 8192 data: 2 2
2063 234/300 Write block 1573 len 20480 data: 2 2 2 2 2
2064 235/300 Read block 2793 len 40960
2065 236/300 Read block 3102 len 16384
2066 237/300 Read block 514 len 36864
2067 238/300 Write block 76 len 36864 data: 2 2 2 2 2 2 2 2 2
2068 239/300 Read block 2800 len 16384
2069 240/300 Read block 2996 len 40960
2070 241/300 Flush
2071 242/300 Read block 3468 len 24576
2072 243/300 Read block 2228 len 36864
2073 244/300 Read block 3981 len 40960
2074 245/300 Read block 1142 len 16384
2075 246/300 Write block 1105 len 32768 data: 3 2 2 2 2 2 2 2
2076 247/300 Read block 2302 len 16384
2077 248/300 Write block 4720 len 32768 data: 3 3 3 3 3 3 3 3
2078 249/300 Write block 1195 len 36864 data: 2 2 2 2 2 3 3 3 3
2079 250/300 Write block 2794 len 8192 data: 3 3
2080 251/300 Read block 1208 len 24576
2081 252/300 Write block 452 len 32768 data: 2 3 3 3 3 3 2 2
2082 253/300 Write block 2027 len 28672 data: 2 2 2 2 2 2 2
2083 254/300 Read block 3095 len 16384
2084 255/300 Write block 4921 len 24576 data: 2 2 3 3 4 4
2085 256/300 Flush
2086 257/300 Read block 3707 len 12288
2087 258/300 Read block 173 len 12288
2088 259/300 Write block 4442 len 4096 data: 2
2089 260/300 Write block 4711 len 4096 data: 2
2090 261/300 Write block 3238 len 12288 data: 2 2 2
2091 262/300 Flush
2092 263/300 Read block 1074 len 16384
2093 264/300 Read block 3658 len 28672
2094 265/300 Write block 3386 len 24576 data: 2 2 2 2 2 2
2095 266/300 Write block 4726 len 36864 data: 4 4 3 2 2 2 2 2 2
2096 267/300 Write block 4585 len 16384 data: 3 2 2 2
2097 268/300 Write block 3226 len 4096 data: 2
2098 269/300 Read block 2026 len 24576
2099 270/300 Read block 2691 len 24576
2100 271/300 Write block 1183 len 16384 data: 2 2 2 3
2101 272/300 Read block 3212 len 28672
2102 273/300 Write block 1382 len 8192 data: 2 2
2103 274/300 Read block 905 len 8192
2104 275/300 Write block 3666 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2105 276/300 Write block 3212 len 4096 data: 2
2106 277/300 Read block 195 len 20480
2107 278/300 Flush
2108 279/300 Read block 475 len 40960
2109 280/300 Write block 4145 len 36864 data: 2 2 2 2 2 2 2 3 3
2110 281/300 Read block 2850 len 20480
2111 282/300 Flush
2112 283/300 Write block 4758 len 20480 data: 3 3 3 3 3
2113 284/300 Write block 3221 len 28672 data: 2 2 2 2 2 3 2
2114 285/300 Write block 4508 len 4096 data: 2
2115 286/300 Read block 2864 len 8192
2116 287/300 Write block 1317 len 36864 data: 3 3 2 2 2 2 2 2 2
2117 288/300 Write block 4586 len 24576 data: 3 3 3 2 2 2
2118 289/300 Read block 1997 len 32768
2119 290/300 Write block 1682 len 12288 data: 2 2 2
2120 291/300 Flush
2121 292/300 Flush
2122 293/300 Write block 2171 len 8192 data: 2 2
2123 294/300 Read block 4527 len 8192
2124 295/300 Read block 1938 len 40960
2125 296/300 Read block 637 len 12288
2126 297/300 Read block 1849 len 36864
2127 298/300 Write block 1489 len 12288 data: 2 2 2
2128 299/300 Flush
2129 300/300 Write block 2827 len 24576 data: 2 2 2 2 2 2
2130 301/300 Read block 1591 len 16384
2131 Main task has work to do, go find it
2132 got dsc Start(1)
2133 start 1
2134 Jan 21 15:47:37.795 INFO request completed, latency_us: 238, response_code: 204, uri: /start/cid/1, method: POST, req_id: 05eaebcd-bcb9-44dc-8f9c-3a236b1fed92, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
2135 Replay: started 1, returned:Ok(())
2136 ----------------------------------------------------------------
2137 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2138 States: Active Offline Active
2139 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2140 1302 Acked Write 9 Done Sent Done false
2141 1303 Acked Read 5 Done Sent Done false
2142 1304 Acked Read 4 Done Sent Done false
2143 1305 Acked Read 3 Done Sent Done false
2144 1306 Acked Write 9 Done Sent Done false
2145 1307 Acked Write 7 Done Sent Done false
2146 1308 Acked Read 10 Done Sent Done false
2147 1309 Acked Read 7 Done Sent Done false
2148 1310 Acked Write 10 Done Sent Done false
2149 1311 Acked Read 2 Done Sent Done false
2150 1312 Acked Write 8 Done Sent Done false
2151 1313 Acked Read 7 Done Sent Done false
2152 1314 Acked Read 1 Done Sent Done false
2153 1315 Acked Flush 0 Done Sent Done false
2154 1316 Acked Read 2 Done Sent Done false
2155 1317 Acked Read 2 Done Sent Done false
2156 1318 Acked Write 3 Done Sent Done false
2157 1319 Acked Write 5 Done Sent Done false
2158 1320 Acked Write 5 Done Sent Done false
2159 1321 Acked Write 7 Done Sent Done false
2160 1322 Acked Read 7 Done Sent Done false
2161 1323 Acked Write 1 Done Sent Done false
2162 1324 Acked Write 7 Done Sent Done false
2163 1325 Acked Write 1 Done Sent Done false
2164 1326 Acked Read 4 Done Sent Done false
2165 1327 Acked Read 7 Done Sent Done false
2166 1328 Acked Read 9 Done Sent Done false
2167 1329 Acked Read 9 Done Sent Done false
2168 1330 Acked Write 7 Done Sent Done false
2169 1331 Acked Read 2 Done Sent Done false
2170 1332 Acked Read 1 Done Sent Done false
2171 1333 Acked Read 10 Done Sent Done false
2172 1334 Acked Read 3 Done Sent Done false
2173 1335 Acked Read 6 Done Sent Done false
2174 1336 Acked Read 10 Done Sent Done false
2175 1337 Acked Write 3 Done Sent Done false
2176 1338 Acked Read 4 Done Sent Done false
2177 1339 Acked Write 6 Done Sent Done false
2178 1340 Acked Read 4 Done Sent Done false
2179 1341 Acked Write 3 Done Sent Done false
2180 1342 Acked Write 10 Done Sent Done false
2181 1343 Acked Read 8 Done Sent Done false
2182 1344 Acked Read 8 Done Sent Done false
2183 1345 Acked Write 5 Done Sent Done false
2184 1346 Acked Write 8 Done Sent Done false
2185 1347 Acked Write 2 Done Sent Done false
2186 1348 Acked Write 1 Done Sent Done false
2187 1349 Acked Write 4 Done Sent Done false
2188 1350 Acked Write 8 Done Sent Done false
2189 1351 Acked Read 1 Done Sent Done false
2190 1352 Acked Write 3 Done Sent Done false
2191 1353 Acked Write 9 Done Sent Done false
2192 1354 Acked Write 3 Done Sent Done false
2193 1355 Acked Write 2 Done Sent Done false
2194 1356 Acked Read 7 Done Sent Done false
2195 1357 Acked Write 5 Done Sent Done false
2196 1358 Acked Write 9 Done Sent Done false
2197 1359 Acked Write 4 Done Sent Done false
2198 1360 Acked Flush 0 Done Sent Done false
2199 1361 Acked Read 8 Done Sent Done false
2200 1362 Acked Read 7 Done Sent Done false
2201 1363 Acked Read 7 Done Sent Done false
2202 1364 Acked Flush 0 Done Sent Done false
2203 1365 Acked Flush 0 Done Sent Done false
2204 1366 Acked Write 9 Done Sent Done false
2205 1367 Acked Read 10 Done Sent Done false
2206 1368 Acked Read 7 Done Sent Done false
2207 1369 Acked Flush 0 Done Sent Done false
2208 1370 Acked Write 5 Done Sent Done false
2209 1371 Acked Write 6 Done Sent Done false
2210 1372 Acked Flush 0 Done Sent Done false
2211 1373 Acked Read 6 Done Sent Done false
2212 1374 Acked Read 5 Done Sent Done false
2213 1375 Acked Write 3 Done Sent Done false
2214 1376 Acked Flush 0 Done Sent Done false
2215 1377 Acked Read 2 Done Sent Done false
2216 1378 Acked Write 10 Done Sent Done false
2217 1379 Acked Flush 0 Done Sent Done false
2218 1380 Acked Write 8 Done Sent Done false
2219 1381 Acked Write 8 Done Sent Done false
2220 1382 Acked Read 6 Done Sent Done false
2221 1383 Acked Write 9 Done Sent Done false
2222 1384 Acked Write 9 Done Sent Done false
2223 1385 Acked Write 3 Done Sent Done false
2224 1386 Acked Write 9 Done Sent Done false
2225 1387 Acked Read 8 Done Sent Done false
2226 1388 Acked Write 7 Done Sent Done false
2227 1389 Acked Write 10 Done Sent Done false
2228 1390 Acked Read 6 Done Sent Done false
2229 1391 Acked Write 8 Done Sent Done false
2230 1392 Acked Read 1 Done Sent Done false
2231 1393 Acked Read 2 Done Sent Done false
2232 1394 Acked Write 8 Done Sent Done false
2233 1395 Acked Read 9 Done Sent Done false
2234 1396 Acked Write 8 Done Sent Done false
2235 1397 Acked Write 10 Done Sent Done false
2236 1398 Acked Read 8 Done Sent Done false
2237 1399 Acked Write 8 Done Sent Done false
2238 1400 Acked Write 2 Done Sent Done false
2239 1401 Acked Read 8 Done Sent Done false
2240 1402 Acked Flush 0 Done Sent Done false
2241 1403 Acked Write 8 Done Sent Done false
2242 1404 Acked Read 1 Done Sent Done false
2243 1405 Acked Write 8 Done Sent Done false
2244 1406 Acked Read 3 Done Sent Done false
2245 1407 Acked Read 6 Done Sent Done false
2246 1408 Acked Write 6 Done Sent Done false
2247 1409 Acked Read 10 Done Sent Done false
2248 1410 Acked Read 6 Done Sent Done false
2249 1411 Acked Read 10 Done Sent Done false
2250 1412 Acked Write 2 Done Sent Done false
2251 1413 Acked Write 3 Done Sent Done false
2252 1414 Acked Flush 0 Done Sent Done false
2253 1415 Acked Write 2 Done Sent Done false
2254 1416 Acked Read 10 Done Sent Done false
2255 1417 Acked Read 4 Done Sent Done false
2256 1418 Acked Read 4 Done Sent Done false
2257 1419 Acked Read 8 Done Sent Done false
2258 1420 Acked Read 4 Done Sent Done false
2259 1421 Acked Write 2 Done Sent Done false
2260 1422 Acked Write 7 Done Sent Done false
2261 1423 Acked Read 1 Done Sent Done false
2262 1424 Acked Write 10 Done Sent Done false
2263 1425 Acked Write 6 Done Sent Done false
2264 1426 Acked Read 10 Done Sent Done false
2265 1427 Acked Write 3 Done Sent Done false
2266 1428 Acked Write 3 Done Sent Done false
2267 1429 Acked Read 1 Done Sent Done false
2268 1430 Acked Write 2 Done Sent Done false
2269 1431 Acked Read 10 Done Sent Done false
2270 1432 Acked Read 10 Done Sent Done false
2271 1433 Acked Write 6 Done Sent Done false
2272 1434 Acked Read 7 Done Sent Done false
2273 1435 Acked Read 9 Done Sent Done false
2274 1436 Acked Write 8 Done Sent Done false
2275 1437 Acked Write 9 Done Sent Done false
2276 1438 Acked Read 4 Done Sent Done false
2277 1439 Acked Read 2 Done Sent Done false
2278 1440 Acked Write 6 Done Sent Done false
2279 1441 Acked Read 10 Done Sent Done false
2280 1442 Acked Read 7 Done Sent Done false
2281 1443 Acked Read 4 Done Sent Done false
2282 1444 Acked Flush 0 Done Sent Done false
2283 1445 Acked Read 7 Done Sent Done false
2284 1446 Acked Write 4 Done Sent Done false
2285 1447 Acked Write 4 Done Sent Done false
2286 1448 Acked Read 3 Done Sent Done false
2287 1449 Acked Read 1 Done Sent Done false
2288 1450 Acked Read 6 Done Sent Done false
2289 1451 Acked Read 1 Done Sent Done false
2290 1452 Acked Write 2 Done Sent Done false
2291 1453 Acked Read 7 Done Sent Done false
2292 1454 Acked Write 3 Done Sent Done false
2293 1455 Acked Flush 0 Done Sent Done false
2294 1456 Acked Read 8 Done Sent Done false
2295 1457 Acked Write 9 Done Sent Done false
2296 1458 Acked Read 9 Done Sent Done false
2297 1459 Acked Read 2 Done Sent Done false
2298 1460 Acked Write 10 Done Sent Done false
2299 1461 Acked Read 1 Done Sent Done false
2300 1462 Acked Read 2 Done Sent Done false
2301 1463 Acked Write 10 Done Sent Done false
2302 1464 Acked Read 5 Done Sent Done false
2303 1465 Acked Write 5 Done Sent Done false
2304 1466 Acked Read 1 Done Sent Done false
2305 1467 Acked Read 2 Done Sent Done false
2306 1468 Acked Write 7 Done Sent Done false
2307 1469 Acked Write 7 Done Sent Done false
2308 1470 Acked Read 9 Done Sent Done false
2309 1471 Acked Write 9 Done Sent Done false
2310 1472 Acked Read 7 Done Sent Done false
2311 1473 Acked Read 8 Done Sent Done false
2312 1474 Acked Write 10 Done Sent Done false
2313 1475 Acked Write 2 Done Sent Done false
2314 1476 Acked Read 1 Done Sent Done false
2315 1477 Acked Read 4 Done Sent Done false
2316 1478 Acked Read 7 Done Sent Done false
2317 1479 Acked Write 5 Done Sent Done false
2318 1480 Acked Write 1 Done Sent Done false
2319 1481 Acked Read 1 Done Sent Done false
2320 1482 Acked Read 6 Done Sent Done false
2321 1483 Acked Read 6 Done Sent Done false
2322 1484 Acked Write 10 Done Sent Done false
2323 1485 Acked Read 10 Done Sent Done false
2324 1486 Acked Write 8 Done Sent Done false
2325 1487 Acked Write 7 Done Sent Done false
2326 1488 Acked Write 8 Done Sent Done false
2327 1489 Acked Write 10 Done Sent Done false
2328 1490 Acked Write 6 Done Sent Done false
2329 1491 Acked Read 1 Done Sent Done false
2330 1492 Acked Read 3 Done Sent Done false
2331 1493 Acked Write 1 Done Sent Done false
2332 1494 Acked Write 6 Done Sent Done false
2333 1495 Acked Read 4 Done Sent Done false
2334 1496 Acked Write 4 Done Sent Done false
2335 1497 Acked Write 8 Done Sent Done false
2336 1498 Acked Read 6 Done Sent Done false
2337 1499 Acked Write 5 Done Sent Done false
2338 1500 Acked Write 1 Done Sent Done false
2339 1501 Acked Read 6 Done Sent Done false
2340 1502 Acked Read 10 Done Sent Done false
2341 1503 Acked Read 2 Done Sent Done false
2342 1504 Acked Read 2 Done Sent Done false
2343 1505 Acked Write 4 Done Sent Done false
2344 1506 Acked Write 5 Done Sent Done false
2345 1507 Acked Write 1 Done Sent Done false
2346 1508 Acked Write 2 Done Sent Done false
2347 1509 Acked Write 1 Done Sent Done false
2348 1510 Acked Write 7 Done Sent Done false
2349 1511 Acked Flush 0 Done Sent Done false
2350 1512 Acked Write 3 Done Sent Done false
2351 1513 Acked Flush 0 Done Sent Done false
2352 1514 Acked Write 4 Done Sent Done false
2353 1515 Acked Read 9 Done Sent Done false
2354 1516 Acked Read 4 Done Sent Done false
2355 1517 Acked Write 2 Done Sent Done false
2356 1518 Acked Write 10 Done Sent Done false
2357 1519 Acked Write 5 Done Sent Done false
2358 1520 Acked Read 4 Done Sent Done false
2359 1521 Acked Write 5 Done Sent Done false
2360 1522 Acked Read 8 Done Sent Done false
2361 1523 Acked Read 5 Done Sent Done false
2362 1524 Acked Read 3 Done Sent Done false
2363 1525 Acked Read 9 Done Sent Done false
2364 1526 Acked Write 9 Done Sent Done false
2365 1527 Acked Read 1 Done Sent Done false
2366 1528 Acked Read 4 Done Sent Done false
2367 1529 Acked Read 4 Done Sent Done false
2368 1530 Acked Read 6 Done Sent Done false
2369 1531 Acked Read 6 Done Sent Done false
2370 1532 Acked Read 4 Done Sent Done false
2371 1533 Acked Write 5 Done Sent Done false
2372 1534 Acked Write 2 Done Sent Done false
2373 1535 Acked Write 5 Done Sent Done false
2374 1536 Acked Read 10 Done Sent Done false
2375 1537 Acked Read 4 Done Sent Done false
2376 1538 Acked Read 9 Done Sent Done false
2377 1539 Acked Write 9 Done Sent Done false
2378 1540 Acked Read 4 Done Sent Done false
2379 1541 Acked Read 10 Done Sent Done false
2380 1542 Acked Flush 0 Done Sent Done false
2381 1543 Acked Read 6 Done Sent Done false
2382 1544 Acked Read 9 Done Sent Done false
2383 1545 Acked Read 10 Done Sent Done false
2384 1546 Acked Read 4 Done Sent Done false
2385 1547 Acked Write 8 Done Sent Done false
2386 1548 Acked Read 4 Done Sent Done false
2387 1549 Acked Write 8 Done Sent Done false
2388 1550 Acked Write 9 Done Sent Done false
2389 1551 Acked Write 2 Done Sent Done false
2390 1552 Acked Read 6 Done Sent Done false
2391 1553 Acked Write 8 Done Sent Done false
2392 1554 Acked Write 7 Done Sent Done false
2393 1555 Acked Read 4 Done Sent Done false
2394 1556 Acked Write 6 Done Sent Done false
2395 1557 Acked Flush 0 Done Sent Done false
2396 1558 Acked Read 3 Done Sent Done false
2397 1559 Acked Read 3 Done Sent Done false
2398 1560 Acked Write 1 Done Sent Done false
2399 1561 Acked Write 1 Done Sent Done false
2400 1562 Acked Write 3 Done Sent Done false
2401 1563 Acked Flush 0 Done Sent Done false
2402 1564 Acked Read 4 Done Sent Done false
2403 1565 Acked Read 7 Done Sent Done false
2404 1566 Acked Write 6 Done Sent Done false
2405 1567 Acked Write 9 Done Sent Done false
2406 1568 Acked Write 4 Done Sent Done false
2407 1569 Acked Write 1 Done Sent Done false
2408 1570 Acked Read 6 Done Sent Done false
2409 1571 Acked Read 6 Done Sent Done false
2410 1572 Acked Write 4 Done Sent Done false
2411 1573 Acked Read 7 Done Sent Done false
2412 1574 Acked Write 2 Done Sent Done false
2413 1575 Acked Read 2 Done Sent Done false
2414 1576 Acked Write 10 Done Sent Done false
2415 1577 Acked Write 1 Done Sent Done false
2416 1578 Acked Read 5 Done Sent Done false
2417 1579 Acked Flush 0 Done Sent Done false
2418 1580 Acked Read 10 Done Sent Done false
2419 1581 Acked Write 9 Done Sent Done false
2420 1582 Acked Read 5 Done Sent Done false
2421 1583 Acked Flush 0 Done Sent Done false
2422 1584 Acked Write 5 Done Sent Done false
2423 1585 Acked Write 7 Done Sent Done false
2424 1586 Acked Write 1 Done Sent Done false
2425 1587 Acked Read 2 Done Sent Done false
2426 1588 Acked Write 9 Done Sent Done false
2427 1589 Acked Write 6 Done Sent Done false
2428 1590 Acked Read 8 Done Sent Done false
2429 1591 Acked Write 3 Done Sent Done false
2430 1592 Acked Flush 0 Done Sent Done false
2431 1593 Acked Flush 0 Done Sent Done false
2432 1594 Acked Write 2 Done Sent Done false
2433 1595 Acked Read 2 Done Sent Done false
2434 1596 Acked Read 10 Done Sent Done false
2435 1597 Acked Read 3 Done Sent Done false
2436 1598 Acked Read 9 Done Sent Done false
2437 1599 Acked Write 3 Done Sent Done false
2438 1600 Acked Flush 0 Done Sent Done false
2439 1601 Acked Write 6 Done Sent Done false
2440 1602 Acked Read 4 Done Sent Done false
2441 STATES DS:0 DS:1 DS:2 TOTAL
2442 Sent 0 301 0 301
2443 Done 301 0 301 602
2444 Skipped 0 0 0 0
2445 Error 0 0 0 0
2446 Last Flush: 1600 1301 1600
2447 Downstairs last five completed: 1301 1300 1299 1298 1297
2448 Upstairs last five completed: 1602 1601 1600 1599 1598
2449 CLIENT: Up:0 ds:301 act:2
2450 [8820] got start action
2451 [8820] I am going to restart
2452 Starting downstairs at port 8820
2453 Make output file at "/tmp/dsc/downstairs-8820.txt"
2454 [8820][1] reports Starting
2455 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1202)
2456 [8820][1] reports Running
2457 ----------------------------------------------------------------
2458 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2459 States: Active Offline Active
2460 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2461 1302 Acked Write 9 Done Sent Done false
2462 1303 Acked Read 5 Done Sent Done false
2463 1304 Acked Read 4 Done Sent Done false
2464 1305 Acked Read 3 Done Sent Done false
2465 1306 Acked Write 9 Done Sent Done false
2466 1307 Acked Write 7 Done Sent Done false
2467 1308 Acked Read 10 Done Sent Done false
2468 1309 Acked Read 7 Done Sent Done false
2469 1310 Acked Write 10 Done Sent Done false
2470 1311 Acked Read 2 Done Sent Done false
2471 1312 Acked Write 8 Done Sent Done false
2472 1313 Acked Read 7 Done Sent Done false
2473 1314 Acked Read 1 Done Sent Done false
2474 1315 Acked Flush 0 Done Sent Done false
2475 1316 Acked Read 2 Done Sent Done false
2476 1317 Acked Read 2 Done Sent Done false
2477 1318 Acked Write 3 Done Sent Done false
2478 1319 Acked Write 5 Done Sent Done false
2479 1320 Acked Write 5 Done Sent Done false
2480 1321 Acked Write 7 Done Sent Done false
2481 1322 Acked Read 7 Done Sent Done false
2482 1323 Acked Write 1 Done Sent Done false
2483 1324 Acked Write 7 Done Sent Done false
2484 1325 Acked Write 1 Done Sent Done false
2485 1326 Acked Read 4 Done Sent Done false
2486 1327 Acked Read 7 Done Sent Done false
2487 1328 Acked Read 9 Done Sent Done false
2488 1329 Acked Read 9 Done Sent Done false
2489 1330 Acked Write 7 Done Sent Done false
2490 1331 Acked Read 2 Done Sent Done false
2491 1332 Acked Read 1 Done Sent Done false
2492 1333 Acked Read 10 Done Sent Done false
2493 1334 Acked Read 3 Done Sent Done false
2494 1335 Acked Read 6 Done Sent Done false
2495 1336 Acked Read 10 Done Sent Done false
2496 1337 Acked Write 3 Done Sent Done false
2497 1338 Acked Read 4 Done Sent Done false
2498 1339 Acked Write 6 Done Sent Done false
2499 1340 Acked Read 4 Done Sent Done false
2500 1341 Acked Write 3 Done Sent Done false
2501 1342 Acked Write 10 Done Sent Done false
2502 1343 Acked Read 8 Done Sent Done false
2503 1344 Acked Read 8 Done Sent Done false
2504 1345 Acked Write 5 Done Sent Done false
2505 1346 Acked Write 8 Done Sent Done false
2506 1347 Acked Write 2 Done Sent Done false
2507 1348 Acked Write 1 Done Sent Done false
2508 1349 Acked Write 4 Done Sent Done false
2509 1350 Acked Write 8 Done Sent Done false
2510 1351 Acked Read 1 Done Sent Done false
2511 1352 Acked Write 3 Done Sent Done false
2512 1353 Acked Write 9 Done Sent Done false
2513 1354 Acked Write 3 Done Sent Done false
2514 1355 Acked Write 2 Done Sent Done false
2515 1356 Acked Read 7 Done Sent Done false
2516 1357 Acked Write 5 Done Sent Done false
2517 1358 Acked Write 9 Done Sent Done false
2518 1359 Acked Write 4 Done Sent Done false
2519 1360 Acked Flush 0 Done Sent Done false
2520 1361 Acked Read 8 Done Sent Done false
2521 1362 Acked Read 7 Done Sent Done false
2522 1363 Acked Read 7 Done Sent Done false
2523 1364 Acked Flush 0 Done Sent Done false
2524 1365 Acked Flush 0 Done Sent Done false
2525 1366 Acked Write 9 Done Sent Done false
2526 1367 Acked Read 10 Done Sent Done false
2527 1368 Acked Read 7 Done Sent Done false
2528 1369 Acked Flush 0 Done Sent Done false
2529 1370 Acked Write 5 Done Sent Done false
2530 1371 Acked Write 6 Done Sent Done false
2531 1372 Acked Flush 0 Done Sent Done false
2532 1373 Acked Read 6 Done Sent Done false
2533 1374 Acked Read 5 Done Sent Done false
2534 1375 Acked Write 3 Done Sent Done false
2535 1376 Acked Flush 0 Done Sent Done false
2536 1377 Acked Read 2 Done Sent Done false
2537 1378 Acked Write 10 Done Sent Done false
2538 1379 Acked Flush 0 Done Sent Done false
2539 1380 Acked Write 8 Done Sent Done false
2540 1381 Acked Write 8 Done Sent Done false
2541 1382 Acked Read 6 Done Sent Done false
2542 1383 Acked Write 9 Done Sent Done false
2543 1384 Acked Write 9 Done Sent Done false
2544 1385 Acked Write 3 Done Sent Done false
2545 1386 Acked Write 9 Done Sent Done false
2546 1387 Acked Read 8 Done Sent Done false
2547 1388 Acked Write 7 Done Sent Done false
2548 1389 Acked Write 10 Done Sent Done false
2549 1390 Acked Read 6 Done Sent Done false
2550 1391 Acked Write 8 Done Sent Done false
2551 1392 Acked Read 1 Done Sent Done false
2552 1393 Acked Read 2 Done Sent Done false
2553 1394 Acked Write 8 Done Sent Done false
2554 1395 Acked Read 9 Done Sent Done false
2555 1396 Acked Write 8 Done Sent Done false
2556 1397 Acked Write 10 Done Sent Done false
2557 1398 Acked Read 8 Done Sent Done false
2558 1399 Acked Write 8 Done Sent Done false
2559 1400 Acked Write 2 Done Sent Done false
2560 1401 Acked Read 8 Done Sent Done false
2561 1402 Acked Flush 0 Done Sent Done false
2562 1403 Acked Write 8 Done Sent Done false
2563 1404 Acked Read 1 Done Sent Done false
2564 1405 Acked Write 8 Done Sent Done false
2565 1406 Acked Read 3 Done Sent Done false
2566 1407 Acked Read 6 Done Sent Done false
2567 1408 Acked Write 6 Done Sent Done false
2568 1409 Acked Read 10 Done Sent Done false
2569 1410 Acked Read 6 Done Sent Done false
2570 1411 Acked Read 10 Done Sent Done false
2571 1412 Acked Write 2 Done Sent Done false
2572 1413 Acked Write 3 Done Sent Done false
2573 1414 Acked Flush 0 Done Sent Done false
2574 1415 Acked Write 2 Done Sent Done false
2575 1416 Acked Read 10 Done Sent Done false
2576 1417 Acked Read 4 Done Sent Done false
2577 1418 Acked Read 4 Done Sent Done false
2578 1419 Acked Read 8 Done Sent Done false
2579 1420 Acked Read 4 Done Sent Done false
2580 1421 Acked Write 2 Done Sent Done false
2581 1422 Acked Write 7 Done Sent Done false
2582 1423 Acked Read 1 Done Sent Done false
2583 1424 Acked Write 10 Done Sent Done false
2584 1425 Acked Write 6 Done Sent Done false
2585 1426 Acked Read 10 Done Sent Done false
2586 1427 Acked Write 3 Done Sent Done false
2587 1428 Acked Write 3 Done Sent Done false
2588 1429 Acked Read 1 Done Sent Done false
2589 1430 Acked Write 2 Done Sent Done false
2590 1431 Acked Read 10 Done Sent Done false
2591 1432 Acked Read 10 Done Sent Done false
2592 1433 Acked Write 6 Done Sent Done false
2593 1434 Acked Read 7 Done Sent Done false
2594 1435 Acked Read 9 Done Sent Done false
2595 1436 Acked Write 8 Done Sent Done false
2596 1437 Acked Write 9 Done Sent Done false
2597 1438 Acked Read 4 Done Sent Done false
2598 1439 Acked Read 2 Done Sent Done false
2599 1440 Acked Write 6 Done Sent Done false
2600 1441 Acked Read 10 Done Sent Done false
2601 1442 Acked Read 7 Done Sent Done false
2602 1443 Acked Read 4 Done Sent Done false
2603 1444 Acked Flush 0 Done Sent Done false
2604 1445 Acked Read 7 Done Sent Done false
2605 1446 Acked Write 4 Done Sent Done false
2606 1447 Acked Write 4 Done Sent Done false
2607 1448 Acked Read 3 Done Sent Done false
2608 1449 Acked Read 1 Done Sent Done false
2609 1450 Acked Read 6 Done Sent Done false
2610 1451 Acked Read 1 Done Sent Done false
2611 1452 Acked Write 2 Done Sent Done false
2612 1453 Acked Read 7 Done Sent Done false
2613 1454 Acked Write 3 Done Sent Done false
2614 1455 Acked Flush 0 Done Sent Done false
2615 1456 Acked Read 8 Done Sent Done false
2616 1457 Acked Write 9 Done Sent Done false
2617 1458 Acked Read 9 Done Sent Done false
2618 1459 Acked Read 2 Done Sent Done false
2619 1460 Acked Write 10 Done Sent Done false
2620 1461 Acked Read 1 Done Sent Done false
2621 1462 Acked Read 2 Done Sent Done false
2622 1463 Acked Write 10 Done Sent Done false
2623 1464 Acked Read 5 Done Sent Done false
2624 1465 Acked Write 5 Done Sent Done false
2625 1466 Acked Read 1 Done Sent Done false
2626 1467 Acked Read 2 Done Sent Done false
2627 1468 Acked Write 7 Done Sent Done false
2628 1469 Acked Write 7 Done Sent Done false
2629 1470 Acked Read 9 Done Sent Done false
2630 1471 Acked Write 9 Done Sent Done false
2631 1472 Acked Read 7 Done Sent Done false
2632 1473 Acked Read 8 Done Sent Done false
2633 1474 Acked Write 10 Done Sent Done false
2634 1475 Acked Write 2 Done Sent Done false
2635 1476 Acked Read 1 Done Sent Done false
2636 1477 Acked Read 4 Done Sent Done false
2637 1478 Acked Read 7 Done Sent Done false
2638 1479 Acked Write 5 Done Sent Done false
2639 1480 Acked Write 1 Done Sent Done false
2640 1481 Acked Read 1 Done Sent Done false
2641 1482 Acked Read 6 Done Sent Done false
2642 1483 Acked Read 6 Done Sent Done false
2643 1484 Acked Write 10 Done Sent Done false
2644 1485 Acked Read 10 Done Sent Done false
2645 1486 Acked Write 8 Done Sent Done false
2646 1487 Acked Write 7 Done Sent Done false
2647 1488 Acked Write 8 Done Sent Done false
2648 1489 Acked Write 10 Done Sent Done false
2649 1490 Acked Write 6 Done Sent Done false
2650 1491 Acked Read 1 Done Sent Done false
2651 1492 Acked Read 3 Done Sent Done false
2652 1493 Acked Write 1 Done Sent Done false
2653 1494 Acked Write 6 Done Sent Done false
2654 1495 Acked Read 4 Done Sent Done false
2655 1496 Acked Write 4 Done Sent Done false
2656 1497 Acked Write 8 Done Sent Done false
2657 1498 Acked Read 6 Done Sent Done false
2658 1499 Acked Write 5 Done Sent Done false
2659 1500 Acked Write 1 Done Sent Done false
2660 1501 Acked Read 6 Done Sent Done false
2661 1502 Acked Read 10 Done Sent Done false
2662 1503 Acked Read 2 Done Sent Done false
2663 1504 Acked Read 2 Done Sent Done false
2664 1505 Acked Write 4 Done Sent Done false
2665 1506 Acked Write 5 Done Sent Done false
2666 1507 Acked Write 1 Done Sent Done false
2667 1508 Acked Write 2 Done Sent Done false
2668 1509 Acked Write 1 Done Sent Done false
2669 1510 Acked Write 7 Done Sent Done false
2670 1511 Acked Flush 0 Done Sent Done false
2671 1512 Acked Write 3 Done Sent Done false
2672 1513 Acked Flush 0 Done Sent Done false
2673 1514 Acked Write 4 Done Sent Done false
2674 1515 Acked Read 9 Done Sent Done false
2675 1516 Acked Read 4 Done Sent Done false
2676 1517 Acked Write 2 Done Sent Done false
2677 1518 Acked Write 10 Done Sent Done false
2678 1519 Acked Write 5 Done Sent Done false
2679 1520 Acked Read 4 Done Sent Done false
2680 1521 Acked Write 5 Done Sent Done false
2681 1522 Acked Read 8 Done Sent Done false
2682 1523 Acked Read 5 Done Sent Done false
2683 1524 Acked Read 3 Done Sent Done false
2684 1525 Acked Read 9 Done Sent Done false
2685 1526 Acked Write 9 Done Sent Done false
2686 1527 Acked Read 1 Done Sent Done false
2687 1528 Acked Read 4 Done Sent Done false
2688 1529 Acked Read 4 Done Sent Done false
2689 1530 Acked Read 6 Done Sent Done false
2690 1531 Acked Read 6 Done Sent Done false
2691 1532 Acked Read 4 Done Sent Done false
2692 1533 Acked Write 5 Done Sent Done false
2693 1534 Acked Write 2 Done Sent Done false
2694 1535 Acked Write 5 Done Sent Done false
2695 1536 Acked Read 10 Done Sent Done false
2696 1537 Acked Read 4 Done Sent Done false
2697 1538 Acked Read 9 Done Sent Done false
2698 1539 Acked Write 9 Done Sent Done false
2699 1540 Acked Read 4 Done Sent Done false
2700 1541 Acked Read 10 Done Sent Done false
2701 1542 Acked Flush 0 Done Sent Done false
2702 1543 Acked Read 6 Done Sent Done false
2703 1544 Acked Read 9 Done Sent Done false
2704 1545 Acked Read 10 Done Sent Done false
2705 1546 Acked Read 4 Done Sent Done false
2706 1547 Acked Write 8 Done Sent Done false
2707 1548 Acked Read 4 Done Sent Done false
2708 1549 Acked Write 8 Done Sent Done false
2709 1550 Acked Write 9 Done Sent Done false
2710 1551 Acked Write 2 Done Sent Done false
2711 1552 Acked Read 6 Done Sent Done false
2712 1553 Acked Write 8 Done Sent Done false
2713 1554 Acked Write 7 Done Sent Done false
2714 1555 Acked Read 4 Done Sent Done false
2715 1556 Acked Write 6 Done Sent Done false
2716 1557 Acked Flush 0 Done Sent Done false
2717 1558 Acked Read 3 Done Sent Done false
2718 1559 Acked Read 3 Done Sent Done false
2719 1560 Acked Write 1 Done Sent Done false
2720 1561 Acked Write 1 Done Sent Done false
2721 1562 Acked Write 3 Done Sent Done false
2722 1563 Acked Flush 0 Done Sent Done false
2723 1564 Acked Read 4 Done Sent Done false
2724 1565 Acked Read 7 Done Sent Done false
2725 1566 Acked Write 6 Done Sent Done false
2726 1567 Acked Write 9 Done Sent Done false
2727 1568 Acked Write 4 Done Sent Done false
2728 1569 Acked Write 1 Done Sent Done false
2729 1570 Acked Read 6 Done Sent Done false
2730 1571 Acked Read 6 Done Sent Done false
2731 1572 Acked Write 4 Done Sent Done false
2732 1573 Acked Read 7 Done Sent Done false
2733 1574 Acked Write 2 Done Sent Done false
2734 1575 Acked Read 2 Done Sent Done false
2735 1576 Acked Write 10 Done Sent Done false
2736 1577 Acked Write 1 Done Sent Done false
2737 1578 Acked Read 5 Done Sent Done false
2738 1579 Acked Flush 0 Done Sent Done false
2739 1580 Acked Read 10 Done Sent Done false
2740 1581 Acked Write 9 Done Sent Done false
2741 1582 Acked Read 5 Done Sent Done false
2742 1583 Acked Flush 0 Done Sent Done false
2743 1584 Acked Write 5 Done Sent Done false
2744 1585 Acked Write 7 Done Sent Done false
2745 1586 Acked Write 1 Done Sent Done false
2746 1587 Acked Read 2 Done Sent Done false
2747 1588 Acked Write 9 Done Sent Done false
2748 1589 Acked Write 6 Done Sent Done false
2749 1590 Acked Read 8 Done Sent Done false
2750 1591 Acked Write 3 Done Sent Done false
2751 1592 Acked Flush 0 Done Sent Done false
2752 1593 Acked Flush 0 Done Sent Done false
2753 1594 Acked Write 2 Done Sent Done false
2754 1595 Acked Read 2 Done Sent Done false
2755 1596 Acked Read 10 Done Sent Done false
2756 1597 Acked Read 3 Done Sent Done false
2757 1598 Acked Read 9 Done Sent Done false
2758 1599 Acked Write 3 Done Sent Done false
2759 1600 Acked Flush 0 Done Sent Done false
2760 1601 Acked Write 6 Done Sent Done false
2761 1602 Acked Read 4 Done Sent Done false
2762 1603 Acked Flush 0 Done Sent Done false
2763 STATES DS:0 DS:1 DS:2 TOTAL
2764 Sent 0 302 0 302
2765 Done 302 0 302 604
2766 Skipped 0 0 0 0
2767 Error 0 0 0 0
2768 Last Flush: 1603 1301 1603
2769 Downstairs last five completed: 1301 1300 1299 1298 1297
2770 Upstairs last five completed: 1603 1602 1601 1600 1599
2771 CLIENT: Up:0 ds:302 act:2
27722025-01-21T15:47:43.558ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27732025-01-21T15:47:43.558ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60640) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27742025-01-21T15:47:43.559ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27752025-01-21T15:47:43.559ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27762025-01-21T15:47:43.560ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27772025-01-21T15:47:43.560ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27782025-01-21T15:47:43.560ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27792025-01-21T15:47:43.560ZINFOcrucible: [1] client re-new 302 jobs since flush 1301 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27802025-01-21T15:47:43.560ZINFOcrucible: [1] Replayed 302 jobs since flush: 1301 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2781 ----------------------------------------------------------------
2782 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2783 Downstairs last five completed: 1603 1602 1601 1600 1599
2784 Upstairs last five completed: 1603 1602 1601 1600 1599
2785 CLIENT: Up:0 ds:0 act:3
2786 Replay: All jobs finished
2787 Main task has work to do, go find it
2788 got dsc Stop(2)
2789 stop 2
2790 Jan 21 15:47:45.800 INFO request completed, latency_us: 665, response_code: [8830] Got stop action so:false kr:false
2791 204, uri: /stop/cid/2, method: POST, req_id: 7d5deb9d-87d6-43f1-84af-4570a353cc0e, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
2792 Jan 21 15:47:45.801 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/2, method: GET, req_id: c26e4b32-5adf-406d-bcc1-575826a38b9d, remote_addr: 127.0.0.1:52945, local_addr{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:47:45.801465515Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
2793 : 127.0.0{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:47:45.80154814Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
2794 .{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:47:45.801582988Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
2795 1:9998
27962025-01-21T15:47:45.801ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
2797 [8830] Exited with: ExitStatus(unix_wait_status(9))
2798 [8830][2] reports Exit
2799 Jan 21 15:47:49.803 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/2, method: GET, req_id: 71a8e2a3-e545-45d2-a965-0d18c494b3e1, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
2800 001/300 Write block 4125 len 12288 data: 2 2 2
2801 002/300 Write block 1151 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2802 003/300 Flush
2803 004/300 Read block 1730 len 12288
2804 005/300 Read block 4225 len 28672
2805 006/300 Flush
2806 007/300 Write block 3572 len 16384 data: 2 2 3 3
2807 008/300 Read block 4291 len 28672
2808 009/300 Read block 275 len 12288
2809 010/300 Read block 86 len 4096
2810 011/300 Write block 2928 len 12288 data: 2 2 2
2811 012/300 Read block 25 len 20480
2812 013/300 Flush
2813 014/300 Write block 3544 len 16384 data: 2 2 2 2
2814 015/300 Read block 4781 len 28672
2815 016/300 Write block 1734 len 40960 data: 2 2 2 3 3 2 2 2 2 2
2816 017/300 Write block 3621 len 36864 data: 2 3 3 4 4 4 4 4 5
2817 018/300 Read block 2883 len 40960
2818 019/300 Read block 1468 len 12288
2819 020/300 Write block 830 len 12288 data: 2 2 2
2820 021/300 Read block 2450 len 40960
2821 022/300 Flush
2822 023/300 Read block 1672 len 36864
2823 024/300 Write block 1562 len 8192 data: 2 2
2824 025/300 Read block 2183 len 16384
2825 026/300 Read block 3235 len 12288
2826 027/300 Write block 85 len 20480 data: 2 2 2 2 2
2827 028/300 Write block 2074 len 24576 data: 4 4 4 3 2 2
2828 029/300 Read block 61 len 20480
2829 030/300 Read block 2405 len 40960
2830 031/300 Write block 2076 len 28672 data: 5 4 3 3 2 2 2
2831 032/300 Read block 3133 len 24576
2832 033/300 Write block 2147 len 12288 data: 2 2 2
2833 034/300 Write block 3149 len 4096 data: 3
2834 035/300 Flush
2835 036/300 Read block 389 len 8192
2836 037/300 Read block 4333 len 8192
2837 038/300 Read block 4992 len 24576
2838 039/300 Read block 903 len 24576
2839 040/300 Read block 1782 len 24576
2840 041/300 Read block 2207 len 36864
2841 042/300 Write block 2721 len 12288 data: 2 2 2
2842 043/300 Write block 4983 len 32768 data: 3 3 3 4 4 4 3 3
2843 044/300 Read block 4975 len 12288
2844 045/300 Read block 3701 len 8192
2845 046/300 Read block 3092 len 32768
2846 047/300 Read block 84 len 4096
2847 048/300 Write block 1048 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2848 049/300 Write block 4107 len 24576 data: 2 2 2 2 2 2
2849 050/300 Flush
2850 051/300 Read block 1707 len 24576
2851 052/300 Flush
2852 053/300 Read block 226 len 16384
2853 054/300 Read block 1381 len 16384
2854 055/300 Read block 972 len 40960
2855 056/300 Write block 3960 len 24576 data: 2 2 2 2 2 2
2856 057/300 Read block 4240 len 8192
2857 058/300 Flush
2858 059/300 Read block 817 len 16384
2859 060/300 Write block 889 len 28672 data: 2 2 2 2 2 2 2
2860 061/300 Read block 3290 len 12288
2861 062/300 Write block 2629 len 32768 data: 2 2 2 2 2 3 3 3
2862 063/300 Read block 2206 len 12288
2863 064/300 Write block 2017 len 8192 data: 2 3
2864 065/300 Write block 370 len 20480 data: 3 3 3 3 3
2865 066/300 Read block 4987 len 4096
2866 067/300 Read block 4710 len 12288
2867 068/300 Read block 4138 len 36864
2868 069/300 Read block 3444 len 20480
2869 070/300 Read block 2807 len 8192
2870 071/300 Write block 3297 len 4096 data: 2
2871 072/300 Write block 4580 len 12288 data: 2 2 2
2872 073/300 Write block 3799 len 16384 data: 2 2 2 2
2873 074/300 Write block 601 len 20480 data: 2 2 2 2 2
2874 075/300 Write block 4691 len 4096 data: 2
2875 076/300 Write block 2236 len 4096 data: 3
2876 077/300 Write block 2079 len 24576 data: 4 3 3 3 2 2
2877 078/300 Write block 2183 len 16384 data: 2 2 2 2
2878 079/300 Read block 3718 len 36864
2879 080/300 Read block 744 len 28672
2880 081/300 Write block 634 len 24576 data: 2 2 2 2 2 2
2881 082/300 Read block 3519 len 28672
2882 083/300 Write block 345 len 16384 data: 3 3 3 3
2883 084/300 Write block 860 len 8192 data: 2 3
2884 085/300 Write block 1307 len 24576 data: 2 2 2 2 2 2
2885 086/300 Read block 2156 len 8192
2886 087/300 Read block 2595 len 36864
2887 088/300 Write block 2596 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2888 089/300 Write block 2818 len 40960 data: 2 2 2 2 3 3 2 2 2 3
2889 090/300 Read block 942 len 8192
2890 091/300 Read block 1943 len 16384
2891 092/300 Flush
2892 093/300 Write block 483 len 36864 data: 4 4 4 4 4 3 3 3 2
2893 094/300 Read block 4530 len 4096
2894 095/300 Read block 3107 len 20480
2895 096/300 Write block 4118 len 28672 data: 2 2 2 2 2 2 2
2896 097/300 Write block 296 len 24576 data: 2 2 2 2 2 3
2897 098/300 Read block 1059 len 20480
2898 099/300 Write block 3994 len 16384 data: 2 2 2 2
2899 100/300 Write block 2759 len 32768 data: 2 2 2 2 2 2 2 2
2900 101/300 Read block 2440 len 8192
2901 102/300 Write block 2492 len 32768 data: 2 2 2 2 2 2 2 2
2902 103/300 Flush
2903 104/300 Read block 254 len 24576
2904 105/300 Read block 2536 len 40960
2905 106/300 Read block 802 len 4096
2906 107/300 Read block 3744 len 24576
2907 108/300 Flush
2908 109/300 Write block 1993 len 32768 data: 2 2 2 2 2 3 3 3
2909 110/300 Write block 3494 len 4096 data: 3
2910 111/300 Write block 42 len 32768 data: 2 2 2 3 3 3 3 3
2911 112/300 Read block 4950 len 36864
2912 113/300 Read block 3188 len 32768
2913 114/300 Write block 1137 len 32768 data: 2 2 2 2 2 2 2 2
2914 115/300 Read block 1992 len 12288
2915 116/300 Flush
2916 117/300 Write block 3266 len 24576 data: 2 2 2 2 2 2
2917 118/300 Write block 1770 len 16384 data: 2 2 2 2
2918 119/300 Write block 2057 len 32768 data: 2 2 2 2 2 2 2 2
2919 120/300 Flush
2920 121/300 Read block 2473 len 8192
2921 122/300 Read block 472 len 32768
2922 123/300 Read block 3806 len 40960
2923 124/300 Read block 1354 len 28672
2924 125/300 Write block 917 len 28672 data: 3 3 2 2 2 2 2
2925 126/300 Read block 4596 len 16384
2926 127/300 Read block 183 len 32768
2927 128/300 Write block 3274 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2928 129/300 Write block 1239 len 8192 data: 2 2
2929 130/300 Write block 2262 len 24576 data: 2 2 2 2 2 2
2930 131/300 Read block 3704 len 4096
2931 132/300 Read block 4468 len 12288
2932 133/300 Read block 1427 len 16384
2933 134/300 Read block 2194 len 12288
2934 135/300 Read block 3999 len 40960
2935 136/300 Read block 4189 len 32768
2936 137/300 Flush
2937 138/300 Write block 3628 len 8192 data: 5 6
2938 139/300 Read block 2819 len 40960
2939 140/300 Write block 2032 len 32768 data: 3 3 2 2 2 2 2 2
2940 141/300 Flush
2941 142/300 Write block 4841 len 32768 data: 2 2 3 3 3 3 2 2
2942 143/300 Write block 126 len 16384 data: 2 2 2 2
2943 144/300 Flush
2944 145/300 Write block 3837 len 32768 data: 2 2 2 2 2 2 2 2
2945 146/300 Read block 1812 len 8192
2946 147/300 Write block 194 len 12288 data: 2 2 2
2947 148/300 Flush
2948 149/300 Write block 3348 len 32768 data: 2 2 2 2 2 2 2 2
2949 150/300 Write block 4055 len 32768 data: 2 2 2 2 2 2 2 2
2950 151/300 Write block 3668 len 24576 data: 3 3 3 3 3 3
2951 152/300 Read block 860 len 28672
2952 153/300 Read block 4076 len 20480
2953 154/300 Read block 4043 len 28672
2954 155/300 Read block 618 len 8192
2955 156/300 Read block 2923 len 4096
2956 157/300 Read block 1840 len 16384
2957 158/300 Write block 2925 len 40960 data: 2 2 2 3 3 3 2 2 2 2
2958 159/300 Flush
2959 160/300 Write block 1067 len 28672 data: 2 2 2 2 2 2 2
2960 161/300 Read block 4710 len 4096
2961 162/300 Write block 3493 len 8192 data: 3 4
2962 163/300 Read block 879 len 16384
2963 164/300 Read block 290 len 4096
2964 165/300 Write block 434 len 24576 data: 2 2 2 2 2 2
2965 166/300 Write block 1980 len 36864 data: 3 3 2 2 2 2 2 2 2
2966 167/300 Read block 4776 len 12288
2967 168/300 Read block 4037 len 12288
2968 169/300 Read block 140 len 20480
2969 170/300 Flush
2970 171/300 Read block 4065 len 20480
2971 172/300 Read block 4003 len 32768
2972 173/300 Read block 375 len 4096
2973 174/300 Read block 198 len 4096
2974 175/300 Read block 4212 len 12288
2975 176/300 Read block 3120 len 28672
2976 177/300 Read block 545 len 8192
2977 178/300 Flush
2978 179/300 Write block 4618 len 40960 data: 2 2 3 3 3 3 3 2 3 3
2979 180/300 Write block 2164 len 4096 data: 2
2980 181/300 Flush
2981 182/300 Write block 2832 len 32768 data: 3 2 2 2 2 2 2 3
2982 183/300 Read block 4065 len 4096
2983 184/300 Read block 2584 len 40960
2984 185/300 Read block 3672 len 40960
2985 186/300 Read block 3935 len 12288
2986 187/300 Read block 2035 len 32768
2987 188/300 Read block 2362 len 32768
2988 189/300 Read block 205 len 8192
2989 190/300 Read block 1298 len 12288
2990 191/300 Write block 4406 len 24576 data: 2 2 2 2 3 3
2991 192/300 Write block 4217 len 24576 data: 3 3 3 3 3 2
2992 193/300 Write block 3735 len 8192 data: 2 2
2993 194/300 Write block 1811 len 12288 data: 3 3 3
2994 195/300 Write block 4755 len 20480 data: 3 3 3 4 4
2995 196/300 Read block 1219 len 36864
2996 197/300 Read block 2173 len 20480
2997 198/300 Read block 4607 len 20480
2998 199/300 Read block 4608 len 16384
2999 200/300 Read block 3811 len 40960
3000 201/300 Write block 1319 len 32768 data: 3 3 3 3 3 3 3 2
3001 202/300 Write block 4381 len 12288 data: 2 2 2
3002 203/300 Write block 72 len 16384 data: 2 2 2 2
3003 204/300 Write block 4585 len 20480 data: 4 4 4 4 3
3004 205/300 Read block 1060 len 20480
3005 206/300 Read block 802 len 32768
3006 207/300 Read block 4524 len 4096
3007 208/300 Read block 4861 len 16384
3008 209/300 Write block 1766 len 12288 data: 2 2 2
3009 210/300 Flush
3010 211/300 Write block 555 len 16384 data: 3 3 3 3
3011 212/300 Read block 3221 len 28672
3012 213/300 Write block 467 len 12288 data: 2 3 3
3013 214/300 Read block 3337 len 20480
3014 215/300 Write block 1837 len 32768 data: 2 2 2 2 2 2 2 2
3015 216/300 Write block 4601 len 8192 data: 2 2
3016 217/300 Write block 1618 len 16384 data: 2 2 3 3
3017 218/300 Write block 50 len 16384 data: 3 3 3 3
3018 219/300 Read block 3432 len 24576
3019 220/300 Read block 2533 len 16384
3020 221/300 Write block 3161 len 32768 data: 2 2 2 2 2 2 2 2
3021 222/300 Read block 3225 len 28672
3022 223/300 Write block 882 len 12288 data: 2 2 2
3023 224/300 Write block 1123 len 8192 data: 2 2
3024 225/300 Read block 4991 len 4096
3025 226/300 Flush
3026 227/300 Read block 3481 len 8192
3027 228/300 Read block 2255 len 12288
3028 229/300 Flush
3029 230/300 Write block 4468 len 28672 data: 3 3 3 2 2 2 2
3030 231/300 Write block 2351 len 24576 data: 2 2 2 2 2 2
3031 232/300 Write block 1778 len 36864 data: 2 2 2 2 2 2 2 2 2
3032 233/300 Read block 1167 len 32768
3033 234/300 Write block 4828 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3034 235/300 Write block 2822 len 28672 data: 4 4 3 3 3 4 3
3035 236/300 Read block 2502 len 8192
3036 237/300 Write block 953 len 8192 data: 2 2
3037 238/300 Read block 406 len 24576
3038 239/300 Read block 549 len 24576
3039 240/300 Read block 2162 len 36864
3040 241/300 Write block 4203 len 16384 data: 2 2 2 2
3041 242/300 Write block 2601 len 40960 data: 3 3 3 3 3 2 2 2 2 2
3042 243/300 Write block 666 len 4096 data: 2
3043 244/300 Write block 4853 len 32768 data: 2 2 2 2 2 2 2 2
3044 245/300 Read block 3155 len 32768
3045 246/300 Read block 702 len 16384
3046 247/300 Read block 190 len 24576
3047 248/300 Read block 2572 len 4096
3048 249/300 Read block 4168 len 4096
3049 250/300 Write block 4392 len 4096 data: 5
3050 251/300 Read block 3427 len 32768
3051 252/300 Write block 4247 len 4096 data: 2
3052 253/300 Write block 3243 len 24576 data: 2 2 2 2 2 2
3053 254/300 Read block 43 len 24576
3054 255/300 Read block 3190 len 8192
3055 256/300 Write block 2488 len 36864 data: 2 2 2 2 3 3 3 3 3
3056 257/300 Read block 993 len 40960
3057 258/300 Read block 46 len 32768
3058 259/300 Read block 894 len 20480
3059 260/300 Read block 2466 len 24576
3060 261/300 Write block 1900 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3061 262/300 Read block 4266 len 4096
3062 263/300 Read block 3105 len 40960
3063 264/300 Flush
3064 265/300 Read block 1401 len 20480
3065 266/300 Write block 1660 len 36864 data: 2 2 2 2 2 2 2 2 2
3066 267/300 Read block 4513 len 16384
3067 268/300 Read block 3848 len 4096
3068 269/300 Read block 4521 len 20480
3069 270/300 Read block 4124 len 28672
3070 271/300 Read block 2294 len 8192
3071 272/300 Write block 839 len 36864 data: 2 2 2 2 2 2 2 2 2
3072 273/300 Write block 3079 len 4096 data: 2
3073 274/300 Write block 3238 len 16384 data: 3 3 3 2
3074 275/300 Write block 791 len 4096 data: 3
3075 276/300 Write block 2179 len 36864 data: 2 2 2 2 3 3 3 3 2
3076 277/300 Write block 3768 len 28672 data: 2 2 2 2 2 2 2
3077 278/300 Read block 3828 len 28672
3078 279/300 Read block 510 len 20480
3079 280/300 Write block 3691 len 8192 data: 2 2
3080 281/300 Read block 712 len 36864
3081 282/300 Write block 1341 len 8192 data: 4 3
3082 283/300 Write block 2609 len 36864 data: 3 3 2 2 2 2 2 2 2
3083 284/300 Write block 262 len 36864 data: 3 2 2 2 2 2 2 2 3
3084 285/300 Write block 243 len 4096 data: 2
3085 286/300 Read block 3240 len 16384
3086 287/300 Flush
3087 288/300 Write block 4482 len 28672 data: 2 2 2 2 2 2 2
3088 289/300 Write block 480 len 40960 data: 3 3 3 5 5 5 5 5 4 4
3089 290/300 Write block 4036 len 12288 data: 2 2 2
3090 291/300 Write block 2910 len 32768 data: 2 2 2 2 2 2 2 2
3091 292/300 Write block 2447 len 40960 data: 3 2 2 2 2 2 2 2 2 2
3092 293/300 Read block 3964 len 32768
3093 294/300 Read block 3576 len 28672
3094 295/300 Write block 3145 len 40960 data: 3 2 2 3 4 3 3 3 2 3
3095 296/300 Read block 1019 len 4096
3096 297/300 Flush
3097 298/300 Write block 937 len 12288 data: 2 2 2
3098 299/300 Read block 4691 len 8192
3099 300/300 Read block 2388 len 20480
3100 301/300 Flush
3101 Main task has work to do, go find it
3102 got dsc Start(2)
3103 start 2
3104 Jan 21 15:47:50.034 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/2, method: POST, req_id: c52d413a-3e13-4d97-b670-132933ecd8ef, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
3105 Replay: started 2, returned:Ok(())
3106 ----------------------------------------------------------------
3107 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3108 States: Active Active Offline
3109 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3110 1604 Acked Write 3 Done Done Sent false
3111 1605 Acked Write 10 Done Done Sent false
3112 1606 Acked Flush 0 Done Done Sent false
3113 1607 Acked Read 3 Done Done Sent false
3114 1608 Acked Read 7 Done Done Sent false
3115 1609 Acked Flush 0 Done Done Sent false
3116 1610 Acked Write 4 Done Done Sent false
3117 1611 Acked Read 7 Done Done Sent false
3118 1612 Acked Read 3 Done Done Sent false
3119 1613 Acked Read 1 Done Done Sent false
3120 1614 Acked Write 3 Done Done Sent false
3121 1615 Acked Read 5 Done Done Sent false
3122 1616 Acked Flush 0 Done Done Sent false
3123 1617 Acked Write 4 Done Done Sent false
3124 1618 Acked Read 7 Done Done Sent false
3125 1619 Acked Write 10 Done Done Sent false
3126 1620 Acked Write 9 Done Done Sent false
3127 1621 Acked Read 10 Done Done Sent false
3128 1622 Acked Read 3 Done Done Sent false
3129 1623 Acked Write 3 Done Done Sent false
3130 1624 Acked Read 10 Done Done Sent false
3131 1625 Acked Flush 0 Done Done Sent false
3132 1626 Acked Read 9 Done Done Sent false
3133 1627 Acked Write 2 Done Done Sent false
3134 1628 Acked Read 4 Done Done Sent false
3135 1629 Acked Read 3 Done Done Sent false
3136 1630 Acked Write 5 Done Done Sent false
3137 1631 Acked Write 6 Done Done Sent false
3138 1632 Acked Read 5 Done Done Sent false
3139 1633 Acked Read 10 Done Done Sent false
3140 1634 Acked Write 7 Done Done Sent false
3141 1635 Acked Read 6 Done Done Sent false
3142 1636 Acked Write 3 Done Done Sent false
3143 1637 Acked Write 1 Done Done Sent false
3144 1638 Acked Flush 0 Done Done Sent false
3145 1639 Acked Read 2 Done Done Sent false
3146 1640 Acked Read 2 Done Done Sent false
3147 1641 Acked Read 6 Done Done Sent false
3148 1642 Acked Read 6 Done Done Sent false
3149 1643 Acked Read 6 Done Done Sent false
3150 1644 Acked Read 9 Done Done Sent false
3151 1645 Acked Write 3 Done Done Sent false
3152 1646 Acked Write 8 Done Done Sent false
3153 1647 Acked Read 3 Done Done Sent false
3154 1648 Acked Read 2 Done Done Sent false
3155 1649 Acked Read 8 Done Done Sent false
3156 1650 Acked Read 1 Done Done Sent false
3157 1651 Acked Write 10 Done Done Sent false
3158 1652 Acked Write 6 Done Done Sent false
3159 1653 Acked Flush 0 Done Done Sent false
3160 1654 Acked Read 6 Done Done Sent false
3161 1655 Acked Flush 0 Done Done Sent false
3162 1656 Acked Read 4 Done Done Sent false
3163 1657 Acked Read 4 Done Done Sent false
3164 1658 Acked Read 10 Done Done Sent false
3165 1659 Acked Write 6 Done Done Sent false
3166 1660 Acked Read 2 Done Done Sent false
3167 1661 Acked Flush 0 Done Done Sent false
3168 1662 Acked Read 4 Done Done Sent false
3169 1663 Acked Write 7 Done Done Sent false
3170 1664 Acked Read 3 Done Done Sent false
3171 1665 Acked Write 8 Done Done Sent false
3172 1666 Acked Read 3 Done Done Sent false
3173 1667 Acked Write 2 Done Done Sent false
3174 1668 Acked Write 5 Done Done Sent false
3175 1669 Acked Read 1 Done Done Sent false
3176 1670 Acked Read 3 Done Done Sent false
3177 1671 Acked Read 9 Done Done Sent false
3178 1672 Acked Read 5 Done Done Sent false
3179 1673 Acked Read 2 Done Done Sent false
3180 1674 Acked Write 1 Done Done Sent false
3181 1675 Acked Write 3 Done Done Sent false
3182 1676 Acked Write 4 Done Done Sent false
3183 1677 Acked Write 5 Done Done Sent false
3184 1678 Acked Write 1 Done Done Sent false
3185 1679 Acked Write 1 Done Done Sent false
3186 1680 Acked Write 6 Done Done Sent false
3187 1681 Acked Write 4 Done Done Sent false
3188 1682 Acked Read 9 Done Done Sent false
3189 1683 Acked Read 7 Done Done Sent false
3190 1684 Acked Write 6 Done Done Sent false
3191 1685 Acked Read 7 Done Done Sent false
3192 1686 Acked Write 4 Done Done Sent false
3193 1687 Acked Write 2 Done Done Sent false
3194 1688 Acked Write 6 Done Done Sent false
3195 1689 Acked Read 2 Done Done Sent false
3196 1690 Acked Read 9 Done Done Sent false
3197 1691 Acked Write 10 Done Done Sent false
3198 1692 Acked Write 10 Done Done Sent false
3199 1693 Acked Read 2 Done Done Sent false
3200 1694 Acked Read 4 Done Done Sent false
3201 1695 Acked Flush 0 Done Done Sent false
3202 1696 Acked Write 9 Done Done Sent false
3203 1697 Acked Read 1 Done Done Sent false
3204 1698 Acked Read 5 Done Done Sent false
3205 1699 Acked Write 7 Done Done Sent false
3206 1700 Acked Write 6 Done Done Sent false
3207 1701 Acked Read 5 Done Done Sent false
3208 1702 Acked Write 4 Done Done Sent false
3209 1703 Acked Write 8 Done Done Sent false
3210 1704 Acked Read 2 Done Done Sent false
3211 1705 Acked Write 8 Done Done Sent false
3212 1706 Acked Flush 0 Done Done Sent false
3213 1707 Acked Read 6 Done Done Sent false
3214 1708 Acked Read 10 Done Done Sent false
3215 1709 Acked Read 1 Done Done Sent false
3216 1710 Acked Read 6 Done Done Sent false
3217 1711 Acked Flush 0 Done Done Sent false
3218 1712 Acked Write 8 Done Done Sent false
3219 1713 Acked Write 1 Done Done Sent false
3220 1714 Acked Write 8 Done Done Sent false
3221 1715 Acked Read 9 Done Done Sent false
3222 1716 Acked Read 8 Done Done Sent false
3223 1717 Acked Write 8 Done Done Sent false
3224 1718 Acked Read 3 Done Done Sent false
3225 1719 Acked Flush 0 Done Done Sent false
3226 1720 Acked Write 6 Done Done Sent false
3227 1721 Acked Write 4 Done Done Sent false
3228 1722 Acked Write 8 Done Done Sent false
3229 1723 Acked Flush 0 Done Done Sent false
3230 1724 Acked Read 2 Done Done Sent false
3231 1725 Acked Read 8 Done Done Sent false
3232 1726 Acked Read 10 Done Done Sent false
3233 1727 Acked Read 7 Done Done Sent false
3234 1728 Acked Write 7 Done Done Sent false
3235 1729 Acked Read 4 Done Done Sent false
3236 1730 Acked Read 8 Done Done Sent false
3237 1731 Acked Write 10 Done Done Sent false
3238 1732 Acked Write 2 Done Done Sent false
3239 1733 Acked Write 6 Done Done Sent false
3240 1734 Acked Read 1 Done Done Sent false
3241 1735 Acked Read 3 Done Done Sent false
3242 1736 Acked Read 4 Done Done Sent false
3243 1737 Acked Read 3 Done Done Sent false
3244 1738 Acked Read 10 Done Done Sent false
3245 1739 Acked Read 8 Done Done Sent false
3246 1740 Acked Flush 0 Done Done Sent false
3247 1741 Acked Write 2 Done Done Sent false
3248 1742 Acked Read 10 Done Done Sent false
3249 1743 Acked Write 8 Done Done Sent false
3250 1744 Acked Flush 0 Done Done Sent false
3251 1745 Acked Write 8 Done Done Sent false
3252 1746 Acked Write 4 Done Done Sent false
3253 1747 Acked Flush 0 Done Done Sent false
3254 1748 Acked Write 8 Done Done Sent false
3255 1749 Acked Read 2 Done Done Sent false
3256 1750 Acked Write 3 Done Done Sent false
3257 1751 Acked Flush 0 Done Done Sent false
3258 1752 Acked Write 8 Done Done Sent false
3259 1753 Acked Write 8 Done Done Sent false
3260 1754 Acked Write 6 Done Done Sent false
3261 1755 Acked Read 7 Done Done Sent false
3262 1756 Acked Read 5 Done Done Sent false
3263 1757 Acked Read 7 Done Done Sent false
3264 1758 Acked Read 2 Done Done Sent false
3265 1759 Acked Read 1 Done Done Sent false
3266 1760 Acked Read 4 Done Done Sent false
3267 1761 Acked Write 10 Done Done Sent false
3268 1762 Acked Flush 0 Done Done Sent false
3269 1763 Acked Write 7 Done Done Sent false
3270 1764 Acked Read 1 Done Done Sent false
3271 1765 Acked Write 2 Done Done Sent false
3272 1766 Acked Read 4 Done Done Sent false
3273 1767 Acked Read 1 Done Done Sent false
3274 1768 Acked Write 6 Done Done Sent false
3275 1769 Acked Write 9 Done Done Sent false
3276 1770 Acked Read 3 Done Done Sent false
3277 1771 Acked Read 3 Done Done Sent false
3278 1772 Acked Read 5 Done Done Sent false
3279 1773 Acked Flush 0 Done Done Sent false
3280 1774 Acked Read 5 Done Done Sent false
3281 1775 Acked Read 8 Done Done Sent false
3282 1776 Acked Read 1 Done Done Sent false
3283 1777 Acked Read 1 Done Done Sent false
3284 1778 Acked Read 3 Done Done Sent false
3285 1779 Acked Read 7 Done Done Sent false
3286 1780 Acked Read 2 Done Done Sent false
3287 1781 Acked Flush 0 Done Done Sent false
3288 1782 Acked Write 10 Done Done Sent false
3289 1783 Acked Write 1 Done Done Sent false
3290 1784 Acked Flush 0 Done Done Sent false
3291 1785 Acked Write 8 Done Done Sent false
3292 1786 Acked Read 1 Done Done Sent false
3293 1787 Acked Read 10 Done Done Sent false
3294 1788 Acked Read 10 Done Done Sent false
3295 1789 Acked Read 3 Done Done Sent false
3296 1790 Acked Read 8 Done Done Sent false
3297 1791 Acked Read 8 Done Done Sent false
3298 1792 Acked Read 2 Done Done Sent false
3299 1793 Acked Read 3 Done Done Sent false
3300 1794 Acked Write 6 Done Done Sent false
3301 1795 Acked Write 6 Done Done Sent false
3302 1796 Acked Write 2 Done Done Sent false
3303 1797 Acked Write 3 Done Done Sent false
3304 1798 Acked Write 5 Done Done Sent false
3305 1799 Acked Read 9 Done Done Sent false
3306 1800 Acked Read 5 Done Done Sent false
3307 1801 Acked Read 5 Done Done Sent false
3308 1802 Acked Read 4 Done Done Sent false
3309 1803 Acked Read 10 Done Done Sent false
3310 1804 Acked Write 8 Done Done Sent false
3311 1805 Acked Write 3 Done Done Sent false
3312 1806 Acked Write 4 Done Done Sent false
3313 1807 Acked Write 5 Done Done Sent false
3314 1808 Acked Read 5 Done Done Sent false
3315 1809 Acked Read 8 Done Done Sent false
3316 1810 Acked Read 1 Done Done Sent false
3317 1811 Acked Read 4 Done Done Sent false
3318 1812 Acked Write 3 Done Done Sent false
3319 1813 Acked Flush 0 Done Done Sent false
3320 1814 Acked Write 4 Done Done Sent false
3321 1815 Acked Read 7 Done Done Sent false
3322 1816 Acked Write 3 Done Done Sent false
3323 1817 Acked Read 5 Done Done Sent false
3324 1818 Acked Write 8 Done Done Sent false
3325 1819 Acked Write 2 Done Done Sent false
3326 1820 Acked Write 4 Done Done Sent false
3327 1821 Acked Write 4 Done Done Sent false
3328 1822 Acked Read 6 Done Done Sent false
3329 1823 Acked Read 4 Done Done Sent false
3330 1824 Acked Write 8 Done Done Sent false
3331 1825 Acked Read 7 Done Done Sent false
3332 1826 Acked Write 3 Done Done Sent false
3333 1827 Acked Write 2 Done Done Sent false
3334 1828 Acked Read 1 Done Done Sent false
3335 1829 Acked Flush 0 Done Done Sent false
3336 1830 Acked Read 2 Done Done Sent false
3337 1831 Acked Read 3 Done Done Sent false
3338 1832 Acked Flush 0 Done Done Sent false
3339 1833 Acked Write 7 Done Done Sent false
3340 1834 Acked Write 6 Done Done Sent false
3341 1835 Acked Write 9 Done Done Sent false
3342 1836 Acked Read 8 Done Done Sent false
3343 1837 Acked Write 10 Done Done Sent false
3344 1838 Acked Write 7 Done Done Sent false
3345 1839 Acked Read 2 Done Done Sent false
3346 1840 Acked Write 2 Done Done Sent false
3347 1841 Acked Read 6 Done Done Sent false
3348 1842 Acked Read 6 Done Done Sent false
3349 1843 Acked Read 9 Done Done Sent false
3350 1844 Acked Write 4 Done Done Sent false
3351 1845 Acked Write 10 Done Done Sent false
3352 1846 Acked Write 1 Done Done Sent false
3353 1847 Acked Write 8 Done Done Sent false
3354 1848 Acked Read 8 Done Done Sent false
3355 1849 Acked Read 4 Done Done Sent false
3356 1850 Acked Read 6 Done Done Sent false
3357 1851 Acked Read 1 Done Done Sent false
3358 1852 Acked Read 1 Done Done Sent false
3359 1853 Acked Write 1 Done Done Sent false
3360 1854 Acked Read 8 Done Done Sent false
3361 1855 Acked Write 1 Done Done Sent false
3362 1856 Acked Write 6 Done Done Sent false
3363 1857 Acked Read 6 Done Done Sent false
3364 1858 Acked Read 2 Done Done Sent false
3365 1859 Acked Write 9 Done Done Sent false
3366 1860 Acked Read 10 Done Done Sent false
3367 1861 Acked Read 8 Done Done Sent false
3368 1862 Acked Read 5 Done Done Sent false
3369 1863 Acked Read 6 Done Done Sent false
3370 1864 Acked Write 10 Done Done Sent false
3371 1865 Acked Read 1 Done Done Sent false
3372 1866 Acked Read 10 Done Done Sent false
3373 1867 Acked Flush 0 Done Done Sent false
3374 1868 Acked Read 5 Done Done Sent false
3375 1869 Acked Write 9 Done Done Sent false
3376 1870 Acked Read 4 Done Done Sent false
3377 1871 Acked Read 1 Done Done Sent false
3378 1872 Acked Read 5 Done Done Sent false
3379 1873 Acked Read 7 Done Done Sent false
3380 1874 Acked Read 2 Done Done Sent false
3381 1875 Acked Write 9 Done Done Sent false
3382 1876 Acked Write 1 Done Done Sent false
3383 1877 Acked Write 4 Done Done Sent false
3384 1878 Acked Write 1 Done Done Sent false
3385 1879 Acked Write 9 Done Done Sent false
3386 1880 Acked Write 7 Done Done Sent false
3387 1881 Acked Read 7 Done Done Sent false
3388 1882 Acked Read 5 Done Done Sent false
3389 1883 Acked Write 2 Done Done Sent false
3390 1884 Acked Read 9 Done Done Sent false
3391 1885 Acked Write 2 Done Done Sent false
3392 1886 Acked Write 9 Done Done Sent false
3393 1887 Acked Write 9 Done Done Sent false
3394 1888 Acked Write 1 Done Done Sent false
3395 1889 Acked Read 4 Done Done Sent false
3396 1890 Acked Flush 0 Done Done Sent false
3397 1891 Acked Write 7 Done Done Sent false
3398 1892 Acked Write 10 Done Done Sent false
3399 1893 Acked Write 3 Done Done Sent false
3400 1894 Acked Write 8 Done Done Sent false
3401 1895 Acked Write 10 Done Done Sent false
3402 1896 Acked Read 8 Done Done Sent false
3403 1897 Acked Read 7 Done Done Sent false
3404 1898 Acked Write 10 Done Done Sent false
3405 1899 Acked Read 1 Done Done Sent false
3406 1900 Acked Flush 0 Done Done Sent false
3407 1901 Acked Write 3 Done Done Sent false
3408 1902 Acked Read 2 Done Done Sent false
3409 1903 Acked Read 5 Done Done Sent false
3410 1904 Acked Flush 0 Done Done Sent false
3411 STATES DS:0 DS:1 DS:2 TOTAL
3412 Sent 0 0 301 301
3413 Done 301 301 0 602
3414 Skipped 0 0 0 0
3415 Error 0 0 0 0
3416 Last Flush: 1904 1904 1603
3417 Downstairs last five completed: 1603 1602 1601 1600 1599
3418 Upstairs last five completed: 1904 1903 1902 1901 1900
3419 CLIENT: Up:0 ds:301 act:2
3420 [8830] got start action
3421 [8830] I am going to restart
3422 Starting downstairs at port 8830
3423 Make output file at "/tmp/dsc/downstairs-8830.txt"
3424 [8830][2] reports Starting
3425 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1203)
3426 [8830][2] reports Running
3427 ----------------------------------------------------------------
3428 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3429 States: Active Active Offline
3430 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3431 1604 Acked Write 3 Done Done Sent false
3432 1605 Acked Write 10 Done Done Sent false
3433 1606 Acked Flush 0 Done Done Sent false
3434 1607 Acked Read 3 Done Done Sent false
3435 1608 Acked Read 7 Done Done Sent false
3436 1609 Acked Flush 0 Done Done Sent false
3437 1610 Acked Write 4 Done Done Sent false
3438 1611 Acked Read 7 Done Done Sent false
3439 1612 Acked Read 3 Done Done Sent false
3440 1613 Acked Read 1 Done Done Sent false
3441 1614 Acked Write 3 Done Done Sent false
3442 1615 Acked Read 5 Done Done Sent false
3443 1616 Acked Flush 0 Done Done Sent false
3444 1617 Acked Write 4 Done Done Sent false
3445 1618 Acked Read 7 Done Done Sent false
3446 1619 Acked Write 10 Done Done Sent false
3447 1620 Acked Write 9 Done Done Sent false
3448 1621 Acked Read 10 Done Done Sent false
3449 1622 Acked Read 3 Done Done Sent false
3450 1623 Acked Write 3 Done Done Sent false
3451 1624 Acked Read 10 Done Done Sent false
3452 1625 Acked Flush 0 Done Done Sent false
3453 1626 Acked Read 9 Done Done Sent false
3454 1627 Acked Write 2 Done Done Sent false
3455 1628 Acked Read 4 Done Done Sent false
3456 1629 Acked Read 3 Done Done Sent false
3457 1630 Acked Write 5 Done Done Sent false
3458 1631 Acked Write 6 Done Done Sent false
3459 1632 Acked Read 5 Done Done Sent false
3460 1633 Acked Read 10 Done Done Sent false
3461 1634 Acked Write 7 Done Done Sent false
3462 1635 Acked Read 6 Done Done Sent false
3463 1636 Acked Write 3 Done Done Sent false
3464 1637 Acked Write 1 Done Done Sent false
3465 1638 Acked Flush 0 Done Done Sent false
3466 1639 Acked Read 2 Done Done Sent false
3467 1640 Acked Read 2 Done Done Sent false
3468 1641 Acked Read 6 Done Done Sent false
3469 1642 Acked Read 6 Done Done Sent false
3470 1643 Acked Read 6 Done Done Sent false
3471 1644 Acked Read 9 Done Done Sent false
3472 1645 Acked Write 3 Done Done Sent false
3473 1646 Acked Write 8 Done Done Sent false
3474 1647 Acked Read 3 Done Done Sent false
3475 1648 Acked Read 2 Done Done Sent false
3476 1649 Acked Read 8 Done Done Sent false
3477 1650 Acked Read 1 Done Done Sent false
3478 1651 Acked Write 10 Done Done Sent false
3479 1652 Acked Write 6 Done Done Sent false
3480 1653 Acked Flush 0 Done Done Sent false
3481 1654 Acked Read 6 Done Done Sent false
3482 1655 Acked Flush 0 Done Done Sent false
3483 1656 Acked Read 4 Done Done Sent false
3484 1657 Acked Read 4 Done Done Sent false
3485 1658 Acked Read 10 Done Done Sent false
3486 1659 Acked Write 6 Done Done Sent false
3487 1660 Acked Read 2 Done Done Sent false
3488 1661 Acked Flush 0 Done Done Sent false
3489 1662 Acked Read 4 Done Done Sent false
3490 1663 Acked Write 7 Done Done Sent false
3491 1664 Acked Read 3 Done Done Sent false
3492 1665 Acked Write 8 Done Done Sent false
3493 1666 Acked Read 3 Done Done Sent false
3494 1667 Acked Write 2 Done Done Sent false
3495 1668 Acked Write 5 Done Done Sent false
3496 1669 Acked Read 1 Done Done Sent false
3497 1670 Acked Read 3 Done Done Sent false
3498 1671 Acked Read 9 Done Done Sent false
3499 1672 Acked Read 5 Done Done Sent false
3500 1673 Acked Read 2 Done Done Sent false
3501 1674 Acked Write 1 Done Done Sent false
3502 1675 Acked Write 3 Done Done Sent false
3503 1676 Acked Write 4 Done Done Sent false
3504 1677 Acked Write 5 Done Done Sent false
3505 1678 Acked Write 1 Done Done Sent false
3506 1679 Acked Write 1 Done Done Sent false
3507 1680 Acked Write 6 Done Done Sent false
3508 1681 Acked Write 4 Done Done Sent false
3509 1682 Acked Read 9 Done Done Sent false
3510 1683 Acked Read 7 Done Done Sent false
3511 1684 Acked Write 6 Done Done Sent false
3512 1685 Acked Read 7 Done Done Sent false
3513 1686 Acked Write 4 Done Done Sent false
3514 1687 Acked Write 2 Done Done Sent false
3515 1688 Acked Write 6 Done Done Sent false
3516 1689 Acked Read 2 Done Done Sent false
3517 1690 Acked Read 9 Done Done Sent false
3518 1691 Acked Write 10 Done Done Sent false
3519 1692 Acked Write 10 Done Done Sent false
3520 1693 Acked Read 2 Done Done Sent false
3521 1694 Acked Read 4 Done Done Sent false
3522 1695 Acked Flush 0 Done Done Sent false
3523 1696 Acked Write 9 Done Done Sent false
3524 1697 Acked Read 1 Done Done Sent false
3525 1698 Acked Read 5 Done Done Sent false
3526 1699 Acked Write 7 Done Done Sent false
3527 1700 Acked Write 6 Done Done Sent false
3528 1701 Acked Read 5 Done Done Sent false
3529 1702 Acked Write 4 Done Done Sent false
3530 1703 Acked Write 8 Done Done Sent false
3531 1704 Acked Read 2 Done Done Sent false
3532 1705 Acked Write 8 Done Done Sent false
3533 1706 Acked Flush 0 Done Done Sent false
3534 1707 Acked Read 6 Done Done Sent false
3535 1708 Acked Read 10 Done Done Sent false
3536 1709 Acked Read 1 Done Done Sent false
3537 1710 Acked Read 6 Done Done Sent false
3538 1711 Acked Flush 0 Done Done Sent false
3539 1712 Acked Write 8 Done Done Sent false
3540 1713 Acked Write 1 Done Done Sent false
3541 1714 Acked Write 8 Done Done Sent false
3542 1715 Acked Read 9 Done Done Sent false
3543 1716 Acked Read 8 Done Done Sent false
3544 1717 Acked Write 8 Done Done Sent false
3545 1718 Acked Read 3 Done Done Sent false
3546 1719 Acked Flush 0 Done Done Sent false
3547 1720 Acked Write 6 Done Done Sent false
3548 1721 Acked Write 4 Done Done Sent false
3549 1722 Acked Write 8 Done Done Sent false
3550 1723 Acked Flush 0 Done Done Sent false
3551 1724 Acked Read 2 Done Done Sent false
3552 1725 Acked Read 8 Done Done Sent false
3553 1726 Acked Read 10 Done Done Sent false
3554 1727 Acked Read 7 Done Done Sent false
3555 1728 Acked Write 7 Done Done Sent false
3556 1729 Acked Read 4 Done Done Sent false
3557 1730 Acked Read 8 Done Done Sent false
3558 1731 Acked Write 10 Done Done Sent false
3559 1732 Acked Write 2 Done Done Sent false
3560 1733 Acked Write 6 Done Done Sent false
3561 1734 Acked Read 1 Done Done Sent false
3562 1735 Acked Read 3 Done Done Sent false
3563 1736 Acked Read 4 Done Done Sent false
3564 1737 Acked Read 3 Done Done Sent false
3565 1738 Acked Read 10 Done Done Sent false
3566 1739 Acked Read 8 Done Done Sent false
3567 1740 Acked Flush 0 Done Done Sent false
3568 1741 Acked Write 2 Done Done Sent false
3569 1742 Acked Read 10 Done Done Sent false
3570 1743 Acked Write 8 Done Done Sent false
3571 1744 Acked Flush 0 Done Done Sent false
3572 1745 Acked Write 8 Done Done Sent false
3573 1746 Acked Write 4 Done Done Sent false
3574 1747 Acked Flush 0 Done Done Sent false
3575 1748 Acked Write 8 Done Done Sent false
3576 1749 Acked Read 2 Done Done Sent false
3577 1750 Acked Write 3 Done Done Sent false
3578 1751 Acked Flush 0 Done Done Sent false
3579 1752 Acked Write 8 Done Done Sent false
3580 1753 Acked Write 8 Done Done Sent false
3581 1754 Acked Write 6 Done Done Sent false
3582 1755 Acked Read 7 Done Done Sent false
3583 1756 Acked Read 5 Done Done Sent false
3584 1757 Acked Read 7 Done Done Sent false
3585 1758 Acked Read 2 Done Done Sent false
3586 1759 Acked Read 1 Done Done Sent false
3587 1760 Acked Read 4 Done Done Sent false
3588 1761 Acked Write 10 Done Done Sent false
3589 1762 Acked Flush 0 Done Done Sent false
3590 1763 Acked Write 7 Done Done Sent false
3591 1764 Acked Read 1 Done Done Sent false
3592 1765 Acked Write 2 Done Done Sent false
3593 1766 Acked Read 4 Done Done Sent false
3594 1767 Acked Read 1 Done Done Sent false
3595 1768 Acked Write 6 Done Done Sent false
3596 1769 Acked Write 9 Done Done Sent false
3597 1770 Acked Read 3 Done Done Sent false
3598 1771 Acked Read 3 Done Done Sent false
3599 1772 Acked Read 5 Done Done Sent false
3600 1773 Acked Flush 0 Done Done Sent false
3601 1774 Acked Read 5 Done Done Sent false
3602 1775 Acked Read 8 Done Done Sent false
3603 1776 Acked Read 1 Done Done Sent false
3604 1777 Acked Read 1 Done Done Sent false
3605 1778 Acked Read 3 Done Done Sent false
3606 1779 Acked Read 7 Done Done Sent false
3607 1780 Acked Read 2 Done Done Sent false
3608 1781 Acked Flush 0 Done Done Sent false
3609 1782 Acked Write 10 Done Done Sent false
3610 1783 Acked Write 1 Done Done Sent false
3611 1784 Acked Flush 0 Done Done Sent false
3612 1785 Acked Write 8 Done Done Sent false
3613 1786 Acked Read 1 Done Done Sent false
3614 1787 Acked Read 10 Done Done Sent false
3615 1788 Acked Read 10 Done Done Sent false
3616 1789 Acked Read 3 Done Done Sent false
3617 1790 Acked Read 8 Done Done Sent false
3618 1791 Acked Read 8 Done Done Sent false
3619 1792 Acked Read 2 Done Done Sent false
3620 1793 Acked Read 3 Done Done Sent false
3621 1794 Acked Write 6 Done Done Sent false
3622 1795 Acked Write 6 Done Done Sent false
3623 1796 Acked Write 2 Done Done Sent false
3624 1797 Acked Write 3 Done Done Sent false
3625 1798 Acked Write 5 Done Done Sent false
3626 1799 Acked Read 9 Done Done Sent false
3627 1800 Acked Read 5 Done Done Sent false
3628 1801 Acked Read 5 Done Done Sent false
3629 1802 Acked Read 4 Done Done Sent false
3630 1803 Acked Read 10 Done Done Sent false
3631 1804 Acked Write 8 Done Done Sent false
3632 1805 Acked Write 3 Done Done Sent false
3633 1806 Acked Write 4 Done Done Sent false
3634 1807 Acked Write 5 Done Done Sent false
3635 1808 Acked Read 5 Done Done Sent false
3636 1809 Acked Read 8 Done Done Sent false
3637 1810 Acked Read 1 Done Done Sent false
3638 1811 Acked Read 4 Done Done Sent false
3639 1812 Acked Write 3 Done Done Sent false
3640 1813 Acked Flush 0 Done Done Sent false
3641 1814 Acked Write 4 Done Done Sent false
3642 1815 Acked Read 7 Done Done Sent false
3643 1816 Acked Write 3 Done Done Sent false
3644 1817 Acked Read 5 Done Done Sent false
3645 1818 Acked Write 8 Done Done Sent false
3646 1819 Acked Write 2 Done Done Sent false
3647 1820 Acked Write 4 Done Done Sent false
3648 1821 Acked Write 4 Done Done Sent false
3649 1822 Acked Read 6 Done Done Sent false
3650 1823 Acked Read 4 Done Done Sent false
3651 1824 Acked Write 8 Done Done Sent false
3652 1825 Acked Read 7 Done Done Sent false
3653 1826 Acked Write 3 Done Done Sent false
3654 1827 Acked Write 2 Done Done Sent false
3655 1828 Acked Read 1 Done Done Sent false
3656 1829 Acked Flush 0 Done Done Sent false
3657 1830 Acked Read 2 Done Done Sent false
3658 1831 Acked Read 3 Done Done Sent false
3659 1832 Acked Flush 0 Done Done Sent false
3660 1833 Acked Write 7 Done Done Sent false
3661 1834 Acked Write 6 Done Done Sent false
3662 1835 Acked Write 9 Done Done Sent false
3663 1836 Acked Read 8 Done Done Sent false
3664 1837 Acked Write 10 Done Done Sent false
3665 1838 Acked Write 7 Done Done Sent false
3666 1839 Acked Read 2 Done Done Sent false
3667 1840 Acked Write 2 Done Done Sent false
3668 1841 Acked Read 6 Done Done Sent false
3669 1842 Acked Read 6 Done Done Sent false
3670 1843 Acked Read 9 Done Done Sent false
3671 1844 Acked Write 4 Done Done Sent false
3672 1845 Acked Write 10 Done Done Sent false
3673 1846 Acked Write 1 Done Done Sent false
3674 1847 Acked Write 8 Done Done Sent false
3675 1848 Acked Read 8 Done Done Sent false
3676 1849 Acked Read 4 Done Done Sent false
3677 1850 Acked Read 6 Done Done Sent false
3678 1851 Acked Read 1 Done Done Sent false
3679 1852 Acked Read 1 Done Done Sent false
3680 1853 Acked Write 1 Done Done Sent false
3681 1854 Acked Read 8 Done Done Sent false
3682 1855 Acked Write 1 Done Done Sent false
3683 1856 Acked Write 6 Done Done Sent false
3684 1857 Acked Read 6 Done Done Sent false
3685 1858 Acked Read 2 Done Done Sent false
3686 1859 Acked Write 9 Done Done Sent false
3687 1860 Acked Read 10 Done Done Sent false
3688 1861 Acked Read 8 Done Done Sent false
3689 1862 Acked Read 5 Done Done Sent false
3690 1863 Acked Read 6 Done Done Sent false
3691 1864 Acked Write 10 Done Done Sent false
3692 1865 Acked Read 1 Done Done Sent false
3693 1866 Acked Read 10 Done Done Sent false
3694 1867 Acked Flush 0 Done Done Sent false
3695 1868 Acked Read 5 Done Done Sent false
3696 1869 Acked Write 9 Done Done Sent false
3697 1870 Acked Read 4 Done Done Sent false
3698 1871 Acked Read 1 Done Done Sent false
3699 1872 Acked Read 5 Done Done Sent false
3700 1873 Acked Read 7 Done Done Sent false
3701 1874 Acked Read 2 Done Done Sent false
3702 1875 Acked Write 9 Done Done Sent false
3703 1876 Acked Write 1 Done Done Sent false
3704 1877 Acked Write 4 Done Done Sent false
3705 1878 Acked Write 1 Done Done Sent false
3706 1879 Acked Write 9 Done Done Sent false
3707 1880 Acked Write 7 Done Done Sent false
3708 1881 Acked Read 7 Done Done Sent false
3709 1882 Acked Read 5 Done Done Sent false
3710 1883 Acked Write 2 Done Done Sent false
3711 1884 Acked Read 9 Done Done Sent false
3712 1885 Acked Write 2 Done Done Sent false
3713 1886 Acked Write 9 Done Done Sent false
3714 1887 Acked Write 9 Done Done Sent false
3715 1888 Acked Write 1 Done Done Sent false
3716 1889 Acked Read 4 Done Done Sent false
3717 1890 Acked Flush 0 Done Done Sent false
3718 1891 Acked Write 7 Done Done Sent false
3719 1892 Acked Write 10 Done Done Sent false
3720 1893 Acked Write 3 Done Done Sent false
3721 1894 Acked Write 8 Done Done Sent false
3722 1895 Acked Write 10 Done Done Sent false
3723 1896 Acked Read 8 Done Done Sent false
3724 1897 Acked Read 7 Done Done Sent false
3725 1898 Acked Write 10 Done Done Sent false
3726 1899 Acked Read 1 Done Done Sent false
3727 1900 Acked Flush 0 Done Done Sent false
3728 1901 Acked Write 3 Done Done Sent false
3729 1902 Acked Read 2 Done Done Sent false
3730 1903 Acked Read 5 Done Done Sent false
3731 1904 Acked Flush 0 Done Done Sent false
3732 STATES DS:0 DS:1 DS:2 TOTAL
3733 Sent 0 0 301 301
3734 Done 301 301 0 602
3735 Skipped 0 0 0 0
3736 Error 0 0 0 0
3737 Last Flush: 1904 1904 1603
3738 Downstairs last five completed: 1603 1602 1601 1600 1599
3739 Upstairs last five completed: 1904 1903 1902 1901 1900
3740 CLIENT: Up:0 ds:301 act:2
37412025-01-21T15:47:55.803ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
37422025-01-21T15:47:55.803ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35805) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
37432025-01-21T15:47:55.804ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
37442025-01-21T15:47:55.805ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
37452025-01-21T15:47:55.805ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
37462025-01-21T15:47:55.805ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
37472025-01-21T15:47:55.805ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
37482025-01-21T15:47:55.805ZINFOcrucible: [2] client re-new 301 jobs since flush 1603 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
37492025-01-21T15:47:55.806ZINFOcrucible: [2] Replayed 301 jobs since flush: 1603 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
3750 ----------------------------------------------------------------
3751 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3752 Downstairs last five completed: 1904 1903 1902 1901 1900
3753 Upstairs last five completed: 1904 1903 1902 1901 1900
3754 CLIENT: Up:0 ds:0 act:3
3755 Replay: All jobs finished
3756 Main task has work to do, go find it
3757 got dsc Stop(1)
3758 stop 1
3759 Jan 21 15:47:58.040 INFO request completed, latency_us: 325, response_code: 204, uri: /stop/cid/1[8820] Got stop action so:false kr:false
3760 , method: POST, req_id: ff6504b4-9af4-4b4d-8287-ebdc8590b015, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
37612025-01-21T15:47:58.041ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
3762 Jan 21 15:47:58.041 INFO request completed, latency_us: 152{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:47:58.041659132Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
3763 , {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:47:58.04169685Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
3764 response_code: 200, uri: /state/cid/1, method:{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:47:58.041727498Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
3765 GET, req_id: a9bb2ca2-043e-4751-82b3-94f0046d9810, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
3766 [8820] Exited with: ExitStatus(unix_wait_status(9))
3767 [8820][1] reports Exit
3768 Jan 21 15:48:02.044 INFO request completed, latency_us: 250, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4503e2c8-a25e-48ec-a61f-53ecaea609ba, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
3769 001/300 Read block 3821 len 32768
3770 002/300 Write block 1809 len 20480 data: 3 3 4 4 4
3771 003/300 Write block 625 len 12288 data: 2 2 2
3772 004/300 Read block 2421 len 40960
3773 005/300 Write block 2675 len 36864 data: 2 2 2 2 2 2 2 2 2
3774 006/300 Read block 4213 len 24576
3775 007/300 Write block 4389 len 8192 data: 5 5
3776 008/300 Write block 2456 len 40960 data: 3 4 4 4 4 3 3 3 3 2
3777 009/300 Read block 160 len 8192
3778 010/300 Read block 2643 len 20480
3779 011/300 Write block 3344 len 20480 data: 2 2 2 2 3
3780 012/300 Read block 3972 len 40960
3781 013/300 Write block 1905 len 36864 data: 3 3 4 4 4 2 2 2 2
3782 014/300 Write block 664 len 28672 data: 2 2 3 2 2 2 2
3783 015/300 Write block 4751 len 4096 data: 2
3784 016/300 Write block 3913 len 28672 data: 2 2 2 2 2 2 2
3785 017/300 Read block 3200 len 4096
3786 018/300 Read block 2449 len 16384
3787 019/300 Write block 1187 len 36864 data: 3 3 3 3 3 3 3 2 3
3788 020/300 Write block 1489 len 24576 data: 3 3 3 2 2 2
3789 021/300 Read block 2358 len 32768
3790 022/300 Write block 1542 len 36864 data: 2 2 2 2 2 2 2 2 2
3791 023/300 Write block 839 len 4096 data: 3
3792 024/300 Write block 1041 len 24576 data: 3 3 3 3 3 2
3793 025/300 Read block 2493 len 16384
3794 026/300 Read block 2481 len 28672
3795 027/300 Read block 802 len 4096
3796 028/300 Read block 1952 len 36864
3797 029/300 Read block 4169 len 32768
3798 030/300 Read block 1675 len 28672
3799 031/300 Read block 2137 len 4096
3800 032/300 Read block 2611 len 28672
3801 033/300 Read block 4529 len 40960
3802 034/300 Write block 1962 len 4096 data: 2
3803 035/300 Write block 599 len 20480 data: 2 2 3 3 3
3804 036/300 Flush
3805 037/300 Write block 1070 len 36864 data: 3 3 3 3 2 2 2 2 2
3806 038/300 Read block 1355 len 28672
3807 039/300 Read block 583 len 32768
3808 040/300 Read block 4765 len 24576
3809 041/300 Write block 438 len 12288 data: 3 3 2
3810 042/300 Write block 1428 len 32768 data: 4 4 2 2 2 3 3 3
3811 043/300 Read block 1545 len 4096
3812 044/300 Write block 1157 len 20480 data: 3 3 3 3 2
3813 045/300 Write block 4467 len 20480 data: 2 4 4 4 3
3814 046/300 Read block 3571 len 4096
3815 047/300 Flush
3816 048/300 Write block 1275 len 4096 data: 2
3817 049/300 Read block 556 len 4096
3818 050/300 Read block 903 len 20480
3819 051/300 Write block 4422 len 32768 data: 2 2 3 3 3 2 2 2
3820 052/300 Read block 111 len 4096
3821 053/300 Write block 4112 len 16384 data: 3 2 2 2
3822 054/300 Write block 2873 len 4096 data: 2
3823 055/300 Flush
3824 056/300 Write block 1749 len 40960 data: 2 2 2 2 2 2 3 4 4 3
3825 057/300 Write block 2939 len 4096 data: 2
3826 058/300 Read block 1265 len 4096
3827 059/300 Write block 4406 len 24576 data: 3 3 3 3 4 4
3828 060/300 Write block 113 len 8192 data: 2 2
3829 061/300 Write block 2707 len 32768 data: 2 2 2 2 2 2 2 2
3830 062/300 Write block 2105 len 12288 data: 2 2 3
3831 063/300 Read block 240 len 24576
3832 064/300 Read block 2356 len 40960
3833 065/300 Read block 1311 len 40960
3834 066/300 Read block 4090 len 28672
3835 067/300 Read block 2705 len 20480
3836 068/300 Read block 1844 len 4096
3837 069/300 Write block 4482 len 12288 data: 3 3 3
3838 070/300 Write block 1620 len 16384 data: 4 4 3 3
3839 071/300 Write block 2145 len 8192 data: 2 2
3840 072/300 Write block 2352 len 36864 data: 3 3 3 3 3 2 2 2 2
3841 073/300 Write block 2019 len 36864 data: 2 2 2 2 2 2 2 2 3
3842 074/300 Flush
3843 075/300 Read block 216 len 36864
3844 076/300 Read block 2833 len 28672
3845 077/300 Read block 2053 len 24576
3846 078/300 Write block 3045 len 28672 data: 2 2 2 2 2 2 2
3847 079/300 Read block 3346 len 32768
3848 080/300 Write block 593 len 12288 data: 3 3 3
3849 081/300 Read block 1951 len 16384
3850 082/300 Write block 1946 len 20480 data: 2 2 2 2 2
3851 083/300 Write block 4669 len 32768 data: 2 2 2 2 2 2 2 2
3852 084/300 Read block 4721 len 36864
3853 085/300 Read block 4034 len 36864
3854 086/300 Write block 4624 len 8192 data: 4 3
3855 087/300 Write block 4398 len 12288 data: 2 2 2
3856 088/300 Write block 1576 len 40960 data: 3 3 2 2 2 2 2 2 2 2
3857 089/300 Write block 4850 len 36864 data: 2 2 2 3 3 3 3 3 3
3858 090/300 Write block 2040 len 32768 data: 2 2 2 2 2 2 2 2
3859 091/300 Read block 4581 len 28672
3860 092/300 Read block 490 len 36864
3861 093/300 Read block 3707 len 12288
3862 094/300 Flush
3863 095/300 Read block 3639 len 28672
3864 096/300 Read block 2336 len 40960
3865 097/300 Write block 1580 len 8192 data: 3 3
3866 098/300 Read block 4811 len 16384
3867 099/300 Write block 1237 len 36864 data: 2 2 3 3 2 2 2 2 2
3868 100/300 Write block 2353 len 24576 data: 4 4 4 4 3 3
3869 101/300 Write block 1535 len 36864 data: 2 2 2 2 2 2 2 3 3
3870 102/300 Flush
3871 103/300 Write block 2248 len 16384 data: 2 2 3 3
3872 104/300 Write block 4515 len 40960 data: 2 2 2 2 3 2 2 2 2 2
3873 105/300 Read block 1261 len 20480
3874 106/300 Write block 4312 len 28672 data: 2 2 2 2 2 2 2
3875 107/300 Read block 712 len 36864
3876 108/300 Read block 3925 len 16384
3877 109/300 Flush
3878 110/300 Read block 2419 len 12288
3879 111/300 Read block 491 len 16384
3880 112/300 Write block 3184 len 16384 data: 2 2 2 2
3881 113/300 Flush
3882 114/300 Read block 3313 len 24576
3883 115/300 Read block 3214 len 4096
3884 116/300 Read block 378 len 8192
3885 117/300 Read block 2419 len 16384
3886 118/300 Read block 2140 len 20480
3887 119/300 Write block 4638 len 36864 data: 3 2 2 2 2 2 2 2 2
3888 120/300 Flush
3889 121/300 Flush
3890 122/300 Flush
3891 123/300 Read block 4321 len 4096
3892 124/300 Write block 3814 len 32768 data: 3 3 3 2 2 2 2 2
3893 125/300 Write block 1433 len 8192 data: 4 4
3894 126/300 Read block 2176 len 4096
3895 127/300 Write block 3664 len 32768 data: 2 2 3 3 4 4 4 4
3896 128/300 Write block 1803 len 40960 data: 2 2 2 2 2 2 4 4 5 5
3897 129/300 Flush
3898 130/300 Flush
3899 131/300 Read block 3297 len 36864
3900 132/300 Read block 3428 len 20480
3901 133/300 Read block 210 len 4096
3902 134/300 Read block 3812 len 32768
3903 135/300 Write block 3089 len 8192 data: 2 2
3904 136/300 Write block 1188 len 36864 data: 4 4 4 4 4 4 3 4 3
3905 137/300 Write block 1932 len 24576 data: 2 3 3 3 3 3
3906 138/300 Read block 4933 len 4096
3907 139/300 Write block 4556 len 28672 data: 2 2 2 2 2 2 2
3908 140/300 Read block 3032 len 12288
3909 141/300 Flush
3910 142/300 Write block 3553 len 12288 data: 3 3 3
3911 143/300 Read block 22 len 28672
3912 144/300 Write block 3645 len 4096 data: 2
3913 145/300 Write block 4987 len 36864 data: 5 5 4 4 2 2 2 2 3
3914 146/300 Read block 387 len 12288
3915 147/300 Read block 72 len 36864
3916 148/300 Write block 809 len 16384 data: 3 3 3 2
3917 149/300 Write block 3296 len 36864 data: 2 3 2 2 2 2 2 2 2
3918 150/300 Write block 4251 len 24576 data: 2 2 2 2 2 2
3919 151/300 Write block 2548 len 8192 data: 3 3
3920 152/300 Read block 1724 len 28672
3921 153/300 Flush
3922 154/300 Write block 2392 len 20480 data: 2 2 2 2 2
3923 155/300 Read block 3190 len 24576
3924 156/300 Read block 636 len 36864
3925 157/300 Write block 345 len 20480 data: 4 4 4 4 3
3926 158/300 Write block 4309 len 4096 data: 3
3927 159/300 Write block 711 len 16384 data: 2 2 2 2
3928 160/300 Write block 3707 len 36864 data: 2 2 2 2 2 2 2 2 2
3929 161/300 Read block 4804 len 24576
3930 162/300 Write block 4987 len 20480 data: 6 6 5 5 3
3931 163/300 Read block 1646 len 16384
3932 164/300 Flush
3933 165/300 Write block 3417 len 24576 data: 2 2 2 2 2 2
3934 166/300 Write block 2874 len 32768 data: 2 2 2 2 2 2 2 2
3935 167/300 Write block 4821 len 40960 data: 2 2 2 2 2 2 2 3 3 3
3936 168/300 Write block 4217 len 16384 data: 4 4 4 4
3937 169/300 Read block 4014 len 32768
3938 170/300 Read block 1805 len 16384
3939 171/300 Read block 2425 len 28672
3940 172/300 Write block 1096 len 40960 data: 3 3 2 2 3 3 3 3 3 4
3941 173/300 Read block 2750 len 24576
3942 174/300 Read block 3755 len 20480
3943 175/300 Read block 2650 len 16384
3944 176/300 Write block 4572 len 20480 data: 3 4 3 3 3
3945 177/300 Read block 2629 len 36864
3946 178/300 Read block 4253 len 32768
3947 179/300 Read block 1126 len 12288
3948 180/300 Write block 4653 len 28672 data: 4 2 2 2 2 2 2
3949 181/300 Read block 4973 len 28672
3950 182/300 Flush
3951 183/300 Read block 4096 len 28672
3952 184/300 Write block 4189 len 12288 data: 2 2 2
3953 185/300 Write block 2170 len 40960 data: 2 3 3 2 2 2 2 2 2 3
3954 186/300 Write block 2923 len 20480 data: 2 2 3 3 3
3955 187/300 Read block 3270 len 36864
3956 188/300 Write block 4115 len 32768 data: 3 2 2 3 3 3 3 3
3957 189/300 Write block 524 len 28672 data: 2 2 3 3 3 4 4
3958 190/300 Write block 636 len 36864 data: 3 3 3 3 2 2 2 2 2
3959 191/300 Flush
3960 192/300 Read block 3661 len 12288
3961 193/300 Write block 3487 len 8192 data: 2 2
3962 194/300 Write block 3296 len 8192 data: 3 4
3963 195/300 Write block 4029 len 36864 data: 2 2 2 2 2 2 2 3 3
3964 196/300 Read block 1540 len 40960
3965 197/300 Flush
3966 198/300 Write block 3305 len 4096 data: 2
3967 199/300 Write block 1822 len 16384 data: 2 2 2 2
3968 200/300 Read block 1288 len 16384
3969 201/300 Read block 4540 len 4096
3970 202/300 Write block 1104 len 36864 data: 4 5 3 3 3 3 3 3 3
3971 203/300 Read block 1102 len 24576
3972 204/300 Read block 1866 len 40960
3973 205/300 Read block 1403 len 16384
3974 206/300 Read block 3909 len 20480
3975 207/300 Flush
3976 208/300 Write block 3417 len 36864 data: 3 3 3 3 3 3 2 2 2
3977 209/300 Write block 4720 len 20480 data: 4 4 4 4 4
3978 210/300 Write block 3844 len 24576 data: 3 2 2 2 2 2
3979 211/300 Read block 4851 len 32768
3980 212/300 Read block 2035 len 24576
3981 213/300 Read block 4953 len 12288
3982 214/300 Write block 1352 len 4096 data: 2
3983 215/300 Read block 2164 len 16384
3984 216/300 Write block 772 len 40960 data: 4 4 4 3 3 3 3 2 2 2
3985 217/300 Write block 4604 len 24576 data: 2 2 2 2 2 2
3986 218/300 Read block 3401 len 40960
3987 219/300 Read block 3061 len 16384
3988 220/300 Flush
3989 221/300 Read block 4071 len 16384
3990 222/300 Read block 2410 len 12288
3991 223/300 Write block 1911 len 32768 data: 3 3 3 2 2 2 2 2
3992 224/300 Write block 3808 len 40960 data: 2 3 3 3 3 3 4 4 4 3
3993 225/300 Read block 1041 len 24576
3994 226/300 Read block 3930 len 32768
3995 227/300 Write block 3025 len 40960 data: 3 3 3 3 2 2 2 2 2 3
3996 228/300 Read block 542 len 40960
3997 229/300 Read block 1341 len 32768
3998 230/300 Write block 2458 len 40960 data: 5 5 5 4 4 4 4 3 2 2
3999 231/300 Read block 3251 len 36864
4000 232/300 Write block 3265 len 4096 data: 2
4001 233/300 Read block 3928 len 40960
4002 234/300 Read block 1244 len 12288
4003 235/300 Read block 2295 len 20480
4004 236/300 Read block 102 len 16384
4005 237/300 Write block 881 len 40960 data: 2 3 3 3 2 2 2 2 3 3
4006 238/300 Read block 2346 len 32768
4007 239/300 Write block 2465 len 28672 data: 4 3 3 3 3 3 3
4008 240/300 Read block 1963 len 24576
4009 241/300 Flush
4010 242/300 Read block 3662 len 40960
4011 243/300 Read block 4369 len 8192
4012 244/300 Write block 2834 len 8192 data: 3 3
4013 245/300 Flush
4014 246/300 Read block 3994 len 4096
4015 247/300 Read block 1278 len 32768
4016 248/300 Read block 3350 len 36864
4017 249/300 Read block 4017 len 24576
4018 250/300 Write block 4668 len 32768 data: 2 3 3 3 3 3 3 3
4019 251/300 Read block 4534 len 40960
4020 252/300 Flush
4021 253/300 Read block 2818 len 32768
4022 254/300 Write block 1590 len 12288 data: 2 3 3
4023 255/300 Read block 3712 len 36864
4024 256/300 Flush
4025 257/300 Read block 2903 len 28672
4026 258/300 Read block 2459 len 24576
4027 259/300 Read block 1449 len 8192
4028 260/300 Write block 3911 len 28672 data: 2 2 3 3 3 3 3
4029 261/300 Write block 339 len 8192 data: 2 2
4030 262/300 Read block 4948 len 24576
4031 263/300 Flush
4032 264/300 Write block 2106 len 20480 data: 3 4 3 3 2
4033 265/300 Read block 4840 len 40960
4034 266/300 Flush
4035 267/300 Read block 3947 len 36864
4036 268/300 Read block 2531 len 28672
4037 269/300 Read block 3119 len 20480
4038 270/300 Write block 683 len 32768 data: 2 2 2 2 2 2 2 2
4039 271/300 Read block 2951 len 4096
4040 272/300 Read block 1485 len 20480
4041 273/300 Read block 2671 len 24576
4042 274/300 Read block 1448 len 12288
4043 275/300 Read block 30 len 20480
4044 276/300 Read block 1906 len 4096
4045 277/300 Read block 4110 len 24576
4046 278/300 Read block 4807 len 40960
4047 279/300 Read block 1063 len 12288
4048 280/300 Flush
4049 281/300 Write block 3764 len 16384 data: 2 2 2 2
4050 282/300 Write block 4910 len 24576 data: 3 3 3 3 3 3
4051 283/300 Read block 1155 len 16384
4052 284/300 Write block 3745 len 36864 data: 2 2 2 3 3 2 2 2 2
4053 285/300 Read block 3863 len 40960
4054 286/300 Write block 732 len 32768 data: 3 3 3 3 3 2 2 2
4055 287/300 Write block 1371 len 24576 data: 2 2 2 2 2 2
4056 288/300 Read block 1415 len 4096
4057 289/300 Write block 3905 len 20480 data: 2 2 2 2 2
4058 290/300 Write block 4242 len 4096 data: 2
4059 291/300 Write block 3909 len 8192 data: 3 2
4060 292/300 Write block 1768 len 20480 data: 3 2 3 3 3
4061 293/300 Read block 1769 len 36864
4062 294/300 Read block 498 len 8192
4063 295/300 Flush
4064 296/300 Flush
4065 297/300 Write block 2668 len 36864 data: 2 2 3 3 3 3 2 3 3
4066 298/300 Read block 3983 len 36864
4067 299/300 Write block 3378 len 36864 data: 2 2 2 2 2 2 2 2 3
4068 300/300 Write block 4871 len 16384 data: 2 2 2 2
4069 301/300 Read block 1659 len 16384
4070 Main task has work to do, go find it
4071 got dsc Start(1)
4072 start 1
4073 Jan 21 15:48:02.285 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/1, method: POST, req_id: edadaa3e-35ff-452d-b92d-02da8562614d, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
4074 Replay: started 1, returned:Ok(())
4075 ----------------------------------------------------------------
4076 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4077 States: Active Offline Active
4078 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4079 1905 Acked Read 8 Done Sent Done false
4080 1906 Acked Write 5 Done Sent Done false
4081 1907 Acked Write 3 Done Sent Done false
4082 1908 Acked Read 10 Done Sent Done false
4083 1909 Acked Write 9 Done Sent Done false
4084 1910 Acked Read 6 Done Sent Done false
4085 1911 Acked Write 2 Done Sent Done false
4086 1912 Acked Write 10 Done Sent Done false
4087 1913 Acked Read 2 Done Sent Done false
4088 1914 Acked Read 5 Done Sent Done false
4089 1915 Acked Write 5 Done Sent Done false
4090 1916 Acked Read 10 Done Sent Done false
4091 1917 Acked Write 9 Done Sent Done false
4092 1918 Acked Write 7 Done Sent Done false
4093 1919 Acked Write 1 Done Sent Done false
4094 1920 Acked Write 7 Done Sent Done false
4095 1921 Acked Read 1 Done Sent Done false
4096 1922 Acked Read 4 Done Sent Done false
4097 1923 Acked Write 9 Done Sent Done false
4098 1924 Acked Write 6 Done Sent Done false
4099 1925 Acked Read 8 Done Sent Done false
4100 1926 Acked Write 9 Done Sent Done false
4101 1927 Acked Write 1 Done Sent Done false
4102 1928 Acked Write 6 Done Sent Done false
4103 1929 Acked Read 4 Done Sent Done false
4104 1930 Acked Read 7 Done Sent Done false
4105 1931 Acked Read 1 Done Sent Done false
4106 1932 Acked Read 9 Done Sent Done false
4107 1933 Acked Read 8 Done Sent Done false
4108 1934 Acked Read 7 Done Sent Done false
4109 1935 Acked Read 1 Done Sent Done false
4110 1936 Acked Read 7 Done Sent Done false
4111 1937 Acked Read 10 Done Sent Done false
4112 1938 Acked Write 1 Done Sent Done false
4113 1939 Acked Write 5 Done Sent Done false
4114 1940 Acked Flush 0 Done Sent Done false
4115 1941 Acked Write 9 Done Sent Done false
4116 1942 Acked Read 7 Done Sent Done false
4117 1943 Acked Read 8 Done Sent Done false
4118 1944 Acked Read 6 Done Sent Done false
4119 1945 Acked Write 3 Done Sent Done false
4120 1946 Acked Write 8 Done Sent Done false
4121 1947 Acked Read 1 Done Sent Done false
4122 1948 Acked Write 5 Done Sent Done false
4123 1949 Acked Write 5 Done Sent Done false
4124 1950 Acked Read 1 Done Sent Done false
4125 1951 Acked Flush 0 Done Sent Done false
4126 1952 Acked Write 1 Done Sent Done false
4127 1953 Acked Read 1 Done Sent Done false
4128 1954 Acked Read 5 Done Sent Done false
4129 1955 Acked Write 8 Done Sent Done false
4130 1956 Acked Read 1 Done Sent Done false
4131 1957 Acked Write 4 Done Sent Done false
4132 1958 Acked Write 1 Done Sent Done false
4133 1959 Acked Flush 0 Done Sent Done false
4134 1960 Acked Write 10 Done Sent Done false
4135 1961 Acked Write 1 Done Sent Done false
4136 1962 Acked Read 1 Done Sent Done false
4137 1963 Acked Write 6 Done Sent Done false
4138 1964 Acked Write 2 Done Sent Done false
4139 1965 Acked Write 8 Done Sent Done false
4140 1966 Acked Write 3 Done Sent Done false
4141 1967 Acked Read 6 Done Sent Done false
4142 1968 Acked Read 10 Done Sent Done false
4143 1969 Acked Read 10 Done Sent Done false
4144 1970 Acked Read 7 Done Sent Done false
4145 1971 Acked Read 5 Done Sent Done false
4146 1972 Acked Read 1 Done Sent Done false
4147 1973 Acked Write 3 Done Sent Done false
4148 1974 Acked Write 4 Done Sent Done false
4149 1975 Acked Write 2 Done Sent Done false
4150 1976 Acked Write 9 Done Sent Done false
4151 1977 Acked Write 9 Done Sent Done false
4152 1978 Acked Flush 0 Done Sent Done false
4153 1979 Acked Read 9 Done Sent Done false
4154 1980 Acked Read 7 Done Sent Done false
4155 1981 Acked Read 6 Done Sent Done false
4156 1982 Acked Write 7 Done Sent Done false
4157 1983 Acked Read 8 Done Sent Done false
4158 1984 Acked Write 3 Done Sent Done false
4159 1985 Acked Read 4 Done Sent Done false
4160 1986 Acked Write 5 Done Sent Done false
4161 1987 Acked Write 8 Done Sent Done false
4162 1988 Acked Read 9 Done Sent Done false
4163 1989 Acked Read 9 Done Sent Done false
4164 1990 Acked Write 2 Done Sent Done false
4165 1991 Acked Write 3 Done Sent Done false
4166 1992 Acked Write 10 Done Sent Done false
4167 1993 Acked Write 9 Done Sent Done false
4168 1994 Acked Write 8 Done Sent Done false
4169 1995 Acked Read 7 Done Sent Done false
4170 1996 Acked Read 9 Done Sent Done false
4171 1997 Acked Read 3 Done Sent Done false
4172 1998 Acked Flush 0 Done Sent Done false
4173 1999 Acked Read 7 Done Sent Done false
4174 2000 Acked Read 10 Done Sent Done false
4175 2001 Acked Write 2 Done Sent Done false
4176 2002 Acked Read 4 Done Sent Done false
4177 2003 Acked Write 9 Done Sent Done false
4178 2004 Acked Write 6 Done Sent Done false
4179 2005 Acked Write 9 Done Sent Done false
4180 2006 Acked Flush 0 Done Sent Done false
4181 2007 Acked Write 4 Done Sent Done false
4182 2008 Acked Write 10 Done Sent Done false
4183 2009 Acked Read 5 Done Sent Done false
4184 2010 Acked Write 7 Done Sent Done false
4185 2011 Acked Read 9 Done Sent Done false
4186 2012 Acked Read 4 Done Sent Done false
4187 2013 Acked Flush 0 Done Sent Done false
4188 2014 Acked Read 3 Done Sent Done false
4189 2015 Acked Read 4 Done Sent Done false
4190 2016 Acked Write 4 Done Sent Done false
4191 2017 Acked Flush 0 Done Sent Done false
4192 2018 Acked Read 6 Done Sent Done false
4193 2019 Acked Read 1 Done Sent Done false
4194 2020 Acked Read 2 Done Sent Done false
4195 2021 Acked Read 4 Done Sent Done false
4196 2022 Acked Read 5 Done Sent Done false
4197 2023 Acked Write 9 Done Sent Done false
4198 2024 Acked Flush 0 Done Sent Done false
4199 2025 Acked Flush 0 Done Sent Done false
4200 2026 Acked Flush 0 Done Sent Done false
4201 2027 Acked Read 1 Done Sent Done false
4202 2028 Acked Write 8 Done Sent Done false
4203 2029 Acked Write 2 Done Sent Done false
4204 2030 Acked Read 1 Done Sent Done false
4205 2031 Acked Write 8 Done Sent Done false
4206 2032 Acked Write 10 Done Sent Done false
4207 2033 Acked Flush 0 Done Sent Done false
4208 2034 Acked Flush 0 Done Sent Done false
4209 2035 Acked Read 9 Done Sent Done false
4210 2036 Acked Read 5 Done Sent Done false
4211 2037 Acked Read 1 Done Sent Done false
4212 2038 Acked Read 8 Done Sent Done false
4213 2039 Acked Write 2 Done Sent Done false
4214 2040 Acked Write 9 Done Sent Done false
4215 2041 Acked Write 6 Done Sent Done false
4216 2042 Acked Read 1 Done Sent Done false
4217 2043 Acked Write 7 Done Sent Done false
4218 2044 Acked Read 3 Done Sent Done false
4219 2045 Acked Flush 0 Done Sent Done false
4220 2046 Acked Write 3 Done Sent Done false
4221 2047 Acked Read 7 Done Sent Done false
4222 2048 Acked Write 1 Done Sent Done false
4223 2049 Acked Write 9 Done Sent Done false
4224 2050 Acked Read 3 Done Sent Done false
4225 2051 Acked Read 9 Done Sent Done false
4226 2052 Acked Write 4 Done Sent Done false
4227 2053 Acked Write 9 Done Sent Done false
4228 2054 Acked Write 6 Done Sent Done false
4229 2055 Acked Write 2 Done Sent Done false
4230 2056 Acked Read 7 Done Sent Done false
4231 2057 Acked Flush 0 Done Sent Done false
4232 2058 Acked Write 5 Done Sent Done false
4233 2059 Acked Read 6 Done Sent Done false
4234 2060 Acked Read 9 Done Sent Done false
4235 2061 Acked Write 5 Done Sent Done false
4236 2062 Acked Write 1 Done Sent Done false
4237 2063 Acked Write 4 Done Sent Done false
4238 2064 Acked Write 9 Done Sent Done false
4239 2065 Acked Read 6 Done Sent Done false
4240 2066 Acked Write 5 Done Sent Done false
4241 2067 Acked Read 4 Done Sent Done false
4242 2068 Acked Flush 0 Done Sent Done false
4243 2069 Acked Write 6 Done Sent Done false
4244 2070 Acked Write 8 Done Sent Done false
4245 2071 Acked Write 10 Done Sent Done false
4246 2072 Acked Write 4 Done Sent Done false
4247 2073 Acked Read 8 Done Sent Done false
4248 2074 Acked Read 4 Done Sent Done false
4249 2075 Acked Read 7 Done Sent Done false
4250 2076 Acked Write 10 Done Sent Done false
4251 2077 Acked Read 6 Done Sent Done false
4252 2078 Acked Read 5 Done Sent Done false
4253 2079 Acked Read 4 Done Sent Done false
4254 2080 Acked Write 5 Done Sent Done false
4255 2081 Acked Read 9 Done Sent Done false
4256 2082 Acked Read 8 Done Sent Done false
4257 2083 Acked Read 3 Done Sent Done false
4258 2084 Acked Write 7 Done Sent Done false
4259 2085 Acked Read 7 Done Sent Done false
4260 2086 Acked Flush 0 Done Sent Done false
4261 2087 Acked Read 7 Done Sent Done false
4262 2088 Acked Write 3 Done Sent Done false
4263 2089 Acked Write 10 Done Sent Done false
4264 2090 Acked Write 5 Done Sent Done false
4265 2091 Acked Read 9 Done Sent Done false
4266 2092 Acked Write 8 Done Sent Done false
4267 2093 Acked Write 7 Done Sent Done false
4268 2094 Acked Write 9 Done Sent Done false
4269 2095 Acked Flush 0 Done Sent Done false
4270 2096 Acked Read 3 Done Sent Done false
4271 2097 Acked Write 2 Done Sent Done false
4272 2098 Acked Write 2 Done Sent Done false
4273 2099 Acked Write 9 Done Sent Done false
4274 2100 Acked Read 10 Done Sent Done false
4275 2101 Acked Flush 0 Done Sent Done false
4276 2102 Acked Write 1 Done Sent Done false
4277 2103 Acked Write 4 Done Sent Done false
4278 2104 Acked Read 4 Done Sent Done false
4279 2105 Acked Read 1 Done Sent Done false
4280 2106 Acked Write 9 Done Sent Done false
4281 2107 Acked Read 6 Done Sent Done false
4282 2108 Acked Read 10 Done Sent Done false
4283 2109 Acked Read 4 Done Sent Done false
4284 2110 Acked Read 5 Done Sent Done false
4285 2111 Acked Flush 0 Done Sent Done false
4286 2112 Acked Write 9 Done Sent Done false
4287 2113 Acked Write 5 Done Sent Done false
4288 2114 Acked Write 6 Done Sent Done false
4289 2115 Acked Read 8 Done Sent Done false
4290 2116 Acked Read 6 Done Sent Done false
4291 2117 Acked Read 3 Done Sent Done false
4292 2118 Acked Write 1 Done Sent Done false
4293 2119 Acked Read 4 Done Sent Done false
4294 2120 Acked Write 10 Done Sent Done false
4295 2121 Acked Write 6 Done Sent Done false
4296 2122 Acked Read 10 Done Sent Done false
4297 2123 Acked Read 4 Done Sent Done false
4298 2124 Acked Flush 0 Done Sent Done false
4299 2125 Acked Read 4 Done Sent Done false
4300 2126 Acked Read 3 Done Sent Done false
4301 2127 Acked Write 8 Done Sent Done false
4302 2128 Acked Write 10 Done Sent Done false
4303 2129 Acked Read 6 Done Sent Done false
4304 2130 Acked Read 8 Done Sent Done false
4305 2131 Acked Write 10 Done Sent Done false
4306 2132 Acked Read 10 Done Sent Done false
4307 2133 Acked Read 8 Done Sent Done false
4308 2134 Acked Write 10 Done Sent Done false
4309 2135 Acked Read 9 Done Sent Done false
4310 2136 Acked Write 1 Done Sent Done false
4311 2137 Acked Read 10 Done Sent Done false
4312 2138 Acked Read 3 Done Sent Done false
4313 2139 Acked Read 5 Done Sent Done false
4314 2140 Acked Read 4 Done Sent Done false
4315 2141 Acked Write 10 Done Sent Done false
4316 2142 Acked Read 8 Done Sent Done false
4317 2143 Acked Write 7 Done Sent Done false
4318 2144 Acked Read 6 Done Sent Done false
4319 2145 Acked Flush 0 Done Sent Done false
4320 2146 Acked Read 10 Done Sent Done false
4321 2147 Acked Read 2 Done Sent Done false
4322 2148 Acked Write 2 Done Sent Done false
4323 2149 Acked Flush 0 Done Sent Done false
4324 2150 Acked Read 1 Done Sent Done false
4325 2151 Acked Read 8 Done Sent Done false
4326 2152 Acked Read 9 Done Sent Done false
4327 2153 Acked Read 6 Done Sent Done false
4328 2154 Acked Write 8 Done Sent Done false
4329 2155 Acked Read 10 Done Sent Done false
4330 2156 Acked Flush 0 Done Sent Done false
4331 2157 Acked Read 8 Done Sent Done false
4332 2158 Acked Write 3 Done Sent Done false
4333 2159 Acked Read 9 Done Sent Done false
4334 2160 Acked Flush 0 Done Sent Done false
4335 2161 Acked Read 7 Done Sent Done false
4336 2162 Acked Read 6 Done Sent Done false
4337 2163 Acked Read 2 Done Sent Done false
4338 2164 Acked Write 7 Done Sent Done false
4339 2165 Acked Write 2 Done Sent Done false
4340 2166 Acked Read 6 Done Sent Done false
4341 2167 Acked Flush 0 Done Sent Done false
4342 2168 Acked Write 5 Done Sent Done false
4343 2169 Acked Read 10 Done Sent Done false
4344 2170 Acked Flush 0 Done Sent Done false
4345 2171 Acked Read 9 Done Sent Done false
4346 2172 Acked Read 7 Done Sent Done false
4347 2173 Acked Read 5 Done Sent Done false
4348 2174 Acked Write 8 Done Sent Done false
4349 2175 Acked Read 1 Done Sent Done false
4350 2176 Acked Read 5 Done Sent Done false
4351 2177 Acked Read 6 Done Sent Done false
4352 2178 Acked Read 3 Done Sent Done false
4353 2179 Acked Read 5 Done Sent Done false
4354 2180 Acked Read 1 Done Sent Done false
4355 2181 Acked Read 6 Done Sent Done false
4356 2182 Acked Read 10 Done Sent Done false
4357 2183 Acked Read 3 Done Sent Done false
4358 2184 Acked Flush 0 Done Sent Done false
4359 2185 Acked Write 4 Done Sent Done false
4360 2186 Acked Write 6 Done Sent Done false
4361 2187 Acked Read 4 Done Sent Done false
4362 2188 Acked Write 9 Done Sent Done false
4363 2189 Acked Read 10 Done Sent Done false
4364 2190 Acked Write 8 Done Sent Done false
4365 2191 Acked Write 6 Done Sent Done false
4366 2192 Acked Read 1 Done Sent Done false
4367 2193 Acked Write 5 Done Sent Done false
4368 2194 Acked Write 1 Done Sent Done false
4369 2195 Acked Write 2 Done Sent Done false
4370 2196 Acked Write 5 Done Sent Done false
4371 2197 Acked Read 9 Done Sent Done false
4372 2198 Acked Read 2 Done Sent Done false
4373 2199 Acked Flush 0 Done Sent Done false
4374 2200 Acked Flush 0 Done Sent Done false
4375 2201 Acked Write 9 Done Sent Done false
4376 2202 Acked Read 9 Done Sent Done false
4377 2203 Acked Write 9 Done Sent Done false
4378 2204 Acked Write 4 Done Sent Done false
4379 2205 Acked Read 4 Done Sent Done false
4380 STATES DS:0 DS:1 DS:2 TOTAL
4381 Sent 0 301 0 301
4382 Done 301 0 301 602
4383 Skipped 0 0 0 0
4384 Error 0 0 0 0
4385 Last Flush: 2200 1904 2200
4386 Downstairs last five completed: 1904 1903 1902 1901 1900
4387 Upstairs last five completed: 2205 2204 2203 2202 2201
4388 CLIENT: Up:0 ds:301 act:2
4389 [8820] got start action
4390 [8820] I am going to restart
4391 Starting downstairs at port 8820
4392 Make output file at "/tmp/dsc/downstairs-8820.txt"
4393 [8820][1] reports Starting
4394 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1204)
4395 [8820][1] reports Running
4396 ----------------------------------------------------------------
4397 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4398 States: Active Offline Active
4399 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4400 1905 Acked Read 8 Done Sent Done false
4401 1906 Acked Write 5 Done Sent Done false
4402 1907 Acked Write 3 Done Sent Done false
4403 1908 Acked Read 10 Done Sent Done false
4404 1909 Acked Write 9 Done Sent Done false
4405 1910 Acked Read 6 Done Sent Done false
4406 1911 Acked Write 2 Done Sent Done false
4407 1912 Acked Write 10 Done Sent Done false
4408 1913 Acked Read 2 Done Sent Done false
4409 1914 Acked Read 5 Done Sent Done false
4410 1915 Acked Write 5 Done Sent Done false
4411 1916 Acked Read 10 Done Sent Done false
4412 1917 Acked Write 9 Done Sent Done false
4413 1918 Acked Write 7 Done Sent Done false
4414 1919 Acked Write 1 Done Sent Done false
4415 1920 Acked Write 7 Done Sent Done false
4416 1921 Acked Read 1 Done Sent Done false
4417 1922 Acked Read 4 Done Sent Done false
4418 1923 Acked Write 9 Done Sent Done false
4419 1924 Acked Write 6 Done Sent Done false
4420 1925 Acked Read 8 Done Sent Done false
4421 1926 Acked Write 9 Done Sent Done false
4422 1927 Acked Write 1 Done Sent Done false
4423 1928 Acked Write 6 Done Sent Done false
4424 1929 Acked Read 4 Done Sent Done false
4425 1930 Acked Read 7 Done Sent Done false
4426 1931 Acked Read 1 Done Sent Done false
4427 1932 Acked Read 9 Done Sent Done false
4428 1933 Acked Read 8 Done Sent Done false
4429 1934 Acked Read 7 Done Sent Done false
4430 1935 Acked Read 1 Done Sent Done false
4431 1936 Acked Read 7 Done Sent Done false
4432 1937 Acked Read 10 Done Sent Done false
4433 1938 Acked Write 1 Done Sent Done false
4434 1939 Acked Write 5 Done Sent Done false
4435 1940 Acked Flush 0 Done Sent Done false
4436 1941 Acked Write 9 Done Sent Done false
4437 1942 Acked Read 7 Done Sent Done false
4438 1943 Acked Read 8 Done Sent Done false
4439 1944 Acked Read 6 Done Sent Done false
4440 1945 Acked Write 3 Done Sent Done false
4441 1946 Acked Write 8 Done Sent Done false
4442 1947 Acked Read 1 Done Sent Done false
4443 1948 Acked Write 5 Done Sent Done false
4444 1949 Acked Write 5 Done Sent Done false
4445 1950 Acked Read 1 Done Sent Done false
4446 1951 Acked Flush 0 Done Sent Done false
4447 1952 Acked Write 1 Done Sent Done false
4448 1953 Acked Read 1 Done Sent Done false
4449 1954 Acked Read 5 Done Sent Done false
4450 1955 Acked Write 8 Done Sent Done false
4451 1956 Acked Read 1 Done Sent Done false
4452 1957 Acked Write 4 Done Sent Done false
4453 1958 Acked Write 1 Done Sent Done false
4454 1959 Acked Flush 0 Done Sent Done false
4455 1960 Acked Write 10 Done Sent Done false
4456 1961 Acked Write 1 Done Sent Done false
4457 1962 Acked Read 1 Done Sent Done false
4458 1963 Acked Write 6 Done Sent Done false
4459 1964 Acked Write 2 Done Sent Done false
4460 1965 Acked Write 8 Done Sent Done false
4461 1966 Acked Write 3 Done Sent Done false
4462 1967 Acked Read 6 Done Sent Done false
4463 1968 Acked Read 10 Done Sent Done false
4464 1969 Acked Read 10 Done Sent Done false
4465 1970 Acked Read 7 Done Sent Done false
4466 1971 Acked Read 5 Done Sent Done false
4467 1972 Acked Read 1 Done Sent Done false
4468 1973 Acked Write 3 Done Sent Done false
4469 1974 Acked Write 4 Done Sent Done false
4470 1975 Acked Write 2 Done Sent Done false
4471 1976 Acked Write 9 Done Sent Done false
4472 1977 Acked Write 9 Done Sent Done false
4473 1978 Acked Flush 0 Done Sent Done false
4474 1979 Acked Read 9 Done Sent Done false
4475 1980 Acked Read 7 Done Sent Done false
4476 1981 Acked Read 6 Done Sent Done false
4477 1982 Acked Write 7 Done Sent Done false
4478 1983 Acked Read 8 Done Sent Done false
4479 1984 Acked Write 3 Done Sent Done false
4480 1985 Acked Read 4 Done Sent Done false
4481 1986 Acked Write 5 Done Sent Done false
4482 1987 Acked Write 8 Done Sent Done false
4483 1988 Acked Read 9 Done Sent Done false
4484 1989 Acked Read 9 Done Sent Done false
4485 1990 Acked Write 2 Done Sent Done false
4486 1991 Acked Write 3 Done Sent Done false
4487 1992 Acked Write 10 Done Sent Done false
4488 1993 Acked Write 9 Done Sent Done false
4489 1994 Acked Write 8 Done Sent Done false
4490 1995 Acked Read 7 Done Sent Done false
4491 1996 Acked Read 9 Done Sent Done false
4492 1997 Acked Read 3 Done Sent Done false
4493 1998 Acked Flush 0 Done Sent Done false
4494 1999 Acked Read 7 Done Sent Done false
4495 2000 Acked Read 10 Done Sent Done false
4496 2001 Acked Write 2 Done Sent Done false
4497 2002 Acked Read 4 Done Sent Done false
4498 2003 Acked Write 9 Done Sent Done false
4499 2004 Acked Write 6 Done Sent Done false
4500 2005 Acked Write 9 Done Sent Done false
4501 2006 Acked Flush 0 Done Sent Done false
4502 2007 Acked Write 4 Done Sent Done false
4503 2008 Acked Write 10 Done Sent Done false
4504 2009 Acked Read 5 Done Sent Done false
4505 2010 Acked Write 7 Done Sent Done false
4506 2011 Acked Read 9 Done Sent Done false
4507 2012 Acked Read 4 Done Sent Done false
4508 2013 Acked Flush 0 Done Sent Done false
4509 2014 Acked Read 3 Done Sent Done false
4510 2015 Acked Read 4 Done Sent Done false
4511 2016 Acked Write 4 Done Sent Done false
4512 2017 Acked Flush 0 Done Sent Done false
4513 2018 Acked Read 6 Done Sent Done false
4514 2019 Acked Read 1 Done Sent Done false
4515 2020 Acked Read 2 Done Sent Done false
4516 2021 Acked Read 4 Done Sent Done false
4517 2022 Acked Read 5 Done Sent Done false
4518 2023 Acked Write 9 Done Sent Done false
4519 2024 Acked Flush 0 Done Sent Done false
4520 2025 Acked Flush 0 Done Sent Done false
4521 2026 Acked Flush 0 Done Sent Done false
4522 2027 Acked Read 1 Done Sent Done false
4523 2028 Acked Write 8 Done Sent Done false
4524 2029 Acked Write 2 Done Sent Done false
4525 2030 Acked Read 1 Done Sent Done false
4526 2031 Acked Write 8 Done Sent Done false
4527 2032 Acked Write 10 Done Sent Done false
4528 2033 Acked Flush 0 Done Sent Done false
4529 2034 Acked Flush 0 Done Sent Done false
4530 2035 Acked Read 9 Done Sent Done false
4531 2036 Acked Read 5 Done Sent Done false
4532 2037 Acked Read 1 Done Sent Done false
4533 2038 Acked Read 8 Done Sent Done false
4534 2039 Acked Write 2 Done Sent Done false
4535 2040 Acked Write 9 Done Sent Done false
4536 2041 Acked Write 6 Done Sent Done false
4537 2042 Acked Read 1 Done Sent Done false
4538 2043 Acked Write 7 Done Sent Done false
4539 2044 Acked Read 3 Done Sent Done false
4540 2045 Acked Flush 0 Done Sent Done false
4541 2046 Acked Write 3 Done Sent Done false
4542 2047 Acked Read 7 Done Sent Done false
4543 2048 Acked Write 1 Done Sent Done false
4544 2049 Acked Write 9 Done Sent Done false
4545 2050 Acked Read 3 Done Sent Done false
4546 2051 Acked Read 9 Done Sent Done false
4547 2052 Acked Write 4 Done Sent Done false
4548 2053 Acked Write 9 Done Sent Done false
4549 2054 Acked Write 6 Done Sent Done false
4550 2055 Acked Write 2 Done Sent Done false
4551 2056 Acked Read 7 Done Sent Done false
4552 2057 Acked Flush 0 Done Sent Done false
4553 2058 Acked Write 5 Done Sent Done false
4554 2059 Acked Read 6 Done Sent Done false
4555 2060 Acked Read 9 Done Sent Done false
4556 2061 Acked Write 5 Done Sent Done false
4557 2062 Acked Write 1 Done Sent Done false
4558 2063 Acked Write 4 Done Sent Done false
4559 2064 Acked Write 9 Done Sent Done false
4560 2065 Acked Read 6 Done Sent Done false
4561 2066 Acked Write 5 Done Sent Done false
4562 2067 Acked Read 4 Done Sent Done false
4563 2068 Acked Flush 0 Done Sent Done false
4564 2069 Acked Write 6 Done Sent Done false
4565 2070 Acked Write 8 Done Sent Done false
4566 2071 Acked Write 10 Done Sent Done false
4567 2072 Acked Write 4 Done Sent Done false
4568 2073 Acked Read 8 Done Sent Done false
4569 2074 Acked Read 4 Done Sent Done false
4570 2075 Acked Read 7 Done Sent Done false
4571 2076 Acked Write 10 Done Sent Done false
4572 2077 Acked Read 6 Done Sent Done false
4573 2078 Acked Read 5 Done Sent Done false
4574 2079 Acked Read 4 Done Sent Done false
4575 2080 Acked Write 5 Done Sent Done false
4576 2081 Acked Read 9 Done Sent Done false
4577 2082 Acked Read 8 Done Sent Done false
4578 2083 Acked Read 3 Done Sent Done false
4579 2084 Acked Write 7 Done Sent Done false
4580 2085 Acked Read 7 Done Sent Done false
4581 2086 Acked Flush 0 Done Sent Done false
4582 2087 Acked Read 7 Done Sent Done false
4583 2088 Acked Write 3 Done Sent Done false
4584 2089 Acked Write 10 Done Sent Done false
4585 2090 Acked Write 5 Done Sent Done false
4586 2091 Acked Read 9 Done Sent Done false
4587 2092 Acked Write 8 Done Sent Done false
4588 2093 Acked Write 7 Done Sent Done false
4589 2094 Acked Write 9 Done Sent Done false
4590 2095 Acked Flush 0 Done Sent Done false
4591 2096 Acked Read 3 Done Sent Done false
4592 2097 Acked Write 2 Done Sent Done false
4593 2098 Acked Write 2 Done Sent Done false
4594 2099 Acked Write 9 Done Sent Done false
4595 2100 Acked Read 10 Done Sent Done false
4596 2101 Acked Flush 0 Done Sent Done false
4597 2102 Acked Write 1 Done Sent Done false
4598 2103 Acked Write 4 Done Sent Done false
4599 2104 Acked Read 4 Done Sent Done false
4600 2105 Acked Read 1 Done Sent Done false
4601 2106 Acked Write 9 Done Sent Done false
4602 2107 Acked Read 6 Done Sent Done false
4603 2108 Acked Read 10 Done Sent Done false
4604 2109 Acked Read 4 Done Sent Done false
4605 2110 Acked Read 5 Done Sent Done false
4606 2111 Acked Flush 0 Done Sent Done false
4607 2112 Acked Write 9 Done Sent Done false
4608 2113 Acked Write 5 Done Sent Done false
4609 2114 Acked Write 6 Done Sent Done false
4610 2115 Acked Read 8 Done Sent Done false
4611 2116 Acked Read 6 Done Sent Done false
4612 2117 Acked Read 3 Done Sent Done false
4613 2118 Acked Write 1 Done Sent Done false
4614 2119 Acked Read 4 Done Sent Done false
4615 2120 Acked Write 10 Done Sent Done false
4616 2121 Acked Write 6 Done Sent Done false
4617 2122 Acked Read 10 Done Sent Done false
4618 2123 Acked Read 4 Done Sent Done false
4619 2124 Acked Flush 0 Done Sent Done false
4620 2125 Acked Read 4 Done Sent Done false
4621 2126 Acked Read 3 Done Sent Done false
4622 2127 Acked Write 8 Done Sent Done false
4623 2128 Acked Write 10 Done Sent Done false
4624 2129 Acked Read 6 Done Sent Done false
4625 2130 Acked Read 8 Done Sent Done false
4626 2131 Acked Write 10 Done Sent Done false
4627 2132 Acked Read 10 Done Sent Done false
4628 2133 Acked Read 8 Done Sent Done false
4629 2134 Acked Write 10 Done Sent Done false
4630 2135 Acked Read 9 Done Sent Done false
4631 2136 Acked Write 1 Done Sent Done false
4632 2137 Acked Read 10 Done Sent Done false
4633 2138 Acked Read 3 Done Sent Done false
4634 2139 Acked Read 5 Done Sent Done false
4635 2140 Acked Read 4 Done Sent Done false
4636 2141 Acked Write 10 Done Sent Done false
4637 2142 Acked Read 8 Done Sent Done false
4638 2143 Acked Write 7 Done Sent Done false
4639 2144 Acked Read 6 Done Sent Done false
4640 2145 Acked Flush 0 Done Sent Done false
4641 2146 Acked Read 10 Done Sent Done false
4642 2147 Acked Read 2 Done Sent Done false
4643 2148 Acked Write 2 Done Sent Done false
4644 2149 Acked Flush 0 Done Sent Done false
4645 2150 Acked Read 1 Done Sent Done false
4646 2151 Acked Read 8 Done Sent Done false
4647 2152 Acked Read 9 Done Sent Done false
4648 2153 Acked Read 6 Done Sent Done false
4649 2154 Acked Write 8 Done Sent Done false
4650 2155 Acked Read 10 Done Sent Done false
4651 2156 Acked Flush 0 Done Sent Done false
4652 2157 Acked Read 8 Done Sent Done false
4653 2158 Acked Write 3 Done Sent Done false
4654 2159 Acked Read 9 Done Sent Done false
4655 2160 Acked Flush 0 Done Sent Done false
4656 2161 Acked Read 7 Done Sent Done false
4657 2162 Acked Read 6 Done Sent Done false
4658 2163 Acked Read 2 Done Sent Done false
4659 2164 Acked Write 7 Done Sent Done false
4660 2165 Acked Write 2 Done Sent Done false
4661 2166 Acked Read 6 Done Sent Done false
4662 2167 Acked Flush 0 Done Sent Done false
4663 2168 Acked Write 5 Done Sent Done false
4664 2169 Acked Read 10 Done Sent Done false
4665 2170 Acked Flush 0 Done Sent Done false
4666 2171 Acked Read 9 Done Sent Done false
4667 2172 Acked Read 7 Done Sent Done false
4668 2173 Acked Read 5 Done Sent Done false
4669 2174 Acked Write 8 Done Sent Done false
4670 2175 Acked Read 1 Done Sent Done false
4671 2176 Acked Read 5 Done Sent Done false
4672 2177 Acked Read 6 Done Sent Done false
4673 2178 Acked Read 3 Done Sent Done false
4674 2179 Acked Read 5 Done Sent Done false
4675 2180 Acked Read 1 Done Sent Done false
4676 2181 Acked Read 6 Done Sent Done false
4677 2182 Acked Read 10 Done Sent Done false
4678 2183 Acked Read 3 Done Sent Done false
4679 2184 Acked Flush 0 Done Sent Done false
4680 2185 Acked Write 4 Done Sent Done false
4681 2186 Acked Write 6 Done Sent Done false
4682 2187 Acked Read 4 Done Sent Done false
4683 2188 Acked Write 9 Done Sent Done false
4684 2189 Acked Read 10 Done Sent Done false
4685 2190 Acked Write 8 Done Sent Done false
4686 2191 Acked Write 6 Done Sent Done false
4687 2192 Acked Read 1 Done Sent Done false
4688 2193 Acked Write 5 Done Sent Done false
4689 2194 Acked Write 1 Done Sent Done false
4690 2195 Acked Write 2 Done Sent Done false
4691 2196 Acked Write 5 Done Sent Done false
4692 2197 Acked Read 9 Done Sent Done false
4693 2198 Acked Read 2 Done Sent Done false
4694 2199 Acked Flush 0 Done Sent Done false
4695 2200 Acked Flush 0 Done Sent Done false
4696 2201 Acked Write 9 Done Sent Done false
4697 2202 Acked Read 9 Done Sent Done false
4698 2203 Acked Write 9 Done Sent Done false
4699 2204 Acked Write 4 Done Sent Done false
4700 2205 Acked Read 4 Done Sent Done false
4701 2206 Acked Flush 0 Done Sent Done false
4702 STATES DS:0 DS:1 DS:2 TOTAL
4703 Sent 0 302 0 302
4704 Done 302 0 302 604
4705 Skipped 0 0 0 0
4706 Error 0 0 0 0
4707 Last Flush: 2206 1904 2206
4708 Downstairs last five completed: 1904 1903 1902 1901 1900
4709 Upstairs last five completed: 2206 2205 2204 2203 2202
4710 CLIENT: Up:0 ds:302 act:2
47112025-01-21T15:48:08.043ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
47122025-01-21T15:48:08.043ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:56732) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
47132025-01-21T15:48:08.044ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
47142025-01-21T15:48:08.045ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
47152025-01-21T15:48:08.045ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
47162025-01-21T15:48:08.045ZINFOcrucible: send last flush ID to this DS: 1904 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
47172025-01-21T15:48:08.045ZINFOcrucible: Replied this last flush ID: 1904 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
47182025-01-21T15:48:08.045ZINFOcrucible: [1] client re-new 302 jobs since flush 1904 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
47192025-01-21T15:48:08.045ZINFOcrucible: [1] Replayed 302 jobs since flush: 1904 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
4720 ----------------------------------------------------------------
4721 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4722 Downstairs last five completed: 2206 2205 2204 2203 2202
4723 Upstairs last five completed: 2206 2205 2204 2203 2202
4724 CLIENT: Up:0 ds:0 act:3
4725 Replay: All jobs finished
4726 Main task has work to do, go find it
4727 got dsc Stop(1)
4728 stop 1
4729 [8820] Got stop action so:false kr:false
4730 Jan 21 15:48:10.290 INFO request completed, latency_us: 358, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 4891ccc1-4310-4f41-bb25-c7fb737234cc, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
47312025-01-21T15:48:10.291ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
4732 Jan 21 15:48:10.291 {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:48:10.291544564Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
4733 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/1, method{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:48:10.291573172Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
4734 : GET, req_id: c54af500-1980-431f-af31-bfd86b0fdbbc, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
47352025-01-21T15:48:10.291ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
4736 [8820] Exited with: ExitStatus(unix_wait_status(9))
4737 [8820][1] reports Exit
4738 Jan 21 15:48:14.294 INFO request completed, latency_us: 368, response_code: 200, uri: /state/cid/1, method: GET, req_id: ad01e4b4-f035-4dd2-9e47-98c1a8fb913a, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
4739 001/300 Read block 4423 len 28672
4740 002/300 Read block 2607 len 20480
4741 003/300 Write block 3245 len 12288 data: 3 3 3
4742 004/300 Write block 1873 len 36864 data: 2 2 2 2 2 2 2 2 2
4743 005/300 Read block 2032 len 24576
4744 006/300 Read block 3557 len 28672
4745 007/300 Read block 2421 len 16384
4746 008/300 Write block 367 len 8192 data: 2 3
4747 009/300 Write block 4739 len 16384 data: 2 2 2 2
4748 010/300 Read block 4900 len 36864
4749 011/300 Read block 3221 len 16384
4750 012/300 Read block 2176 len 20480
4751 013/300 Read block 920 len 40960
4752 014/300 Read block 82 len 36864
4753 015/300 Flush
4754 016/300 Write block 3878 len 24576 data: 2 2 2 2 2 2
4755 017/300 Read block 1290 len 40960
4756 018/300 Flush
4757 019/300 Write block 3173 len 36864 data: 2 2 2 2 3 3 3 3 2
4758 020/300 Write block 2907 len 20480 data: 2 2 3 3 3
4759 021/300 Read block 2962 len 4096
4760 022/300 Read block 1366 len 36864
4761 023/300 Read block 690 len 36864
4762 024/300 Write block 1429 len 4096 data: 5
4763 025/300 Read block 4228 len 16384
4764 026/300 Write block 3600 len 8192 data: 2 2
4765 027/300 Flush
4766 028/300 Write block 3719 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4767 029/300 Flush
4768 030/300 Write block 685 len 16384 data: 3 3 3 3
4769 031/300 Write block 4296 len 20480 data: 2 2 2 2 2
4770 032/300 Read block 4166 len 4096
4771 033/300 Flush
4772 034/300 Read block 2592 len 32768
4773 035/300 Read block 153 len 28672
4774 036/300 Read block 4731 len 40960
4775 037/300 Read block 1152 len 12288
4776 038/300 Read block 987 len 16384
4777 039/300 Read block 2974 len 4096
4778 040/300 Read block 1680 len 12288
4779 041/300 Write block 824 len 12288 data: 2 2 2
4780 042/300 Read block 3096 len 32768
4781 043/300 Read block 2632 len 36864
4782 044/300 Write block 1918 len 24576 data: 3 2 2 2 2 3
4783 045/300 Write block 2172 len 4096 data: 4
4784 046/300 Read block 146 len 24576
4785 047/300 Read block 4697 len 4096
4786 048/300 Read block 3223 len 32768
4787 049/300 Read block 4906 len 20480
4788 050/300 Read block 3756 len 20480
4789 051/300 Write block 879 len 12288 data: 2 2 3
4790 052/300 Read block 2470 len 20480
4791 053/300 Read block 2482 len 4096
4792 054/300 Read block 4096 len 24576
4793 055/300 Write block 818 len 32768 data: 2 2 2 2 2 2 3 3
4794 056/300 Flush
4795 057/300 Flush
4796 058/300 Read block 4010 len 32768
4797 059/300 Write block 1901 len 36864 data: 3 3 3 3 4 4 5 5 5
4798 060/300 Read block 378 len 36864
4799 061/300 Read block 4215 len 36864
4800 062/300 Flush
4801 063/300 Write block 4083 len 4096 data: 2
4802 064/300 Write block 696 len 20480 data: 3 3 3 2 2
4803 065/300 Write block 2347 len 36864 data: 2 3 3 2 3 4 5 5 5
4804 066/300 Write block 2150 len 12288 data: 2 2 2
4805 067/300 Read block 3999 len 36864
4806 068/300 Flush
4807 069/300 Write block 172 len 32768 data: 2 2 2 2 2 2 2 2
4808 070/300 Read block 2781 len 24576
4809 071/300 Write block 2025 len 4096 data: 3
4810 072/300 Write block 4071 len 16384 data: 2 2 2 2
4811 073/300 Write block 1223 len 20480 data: 2 2 2 2 2
4812 074/300 Write block 3219 len 8192 data: 2 2
4813 075/300 Read block 3339 len 36864
4814 076/300 Write block 842 len 20480 data: 3 3 3 3 3
4815 077/300 Write block 1066 len 36864 data: 2 3 3 3 4 4 4 4 3
4816 078/300 Read block 3368 len 36864
4817 079/300 Read block 4819 len 12288
4818 080/300 Read block 3144 len 16384
4819 081/300 Read block 3762 len 12288
4820 082/300 Write block 3288 len 28672 data: 2 2 2 2 2 2 2
4821 083/300 Write block 4181 len 40960 data: 3 3 3 3 3 3 3 2 3 3
4822 084/300 Flush
4823 085/300 Write block 2172 len 12288 data: 5 3 3
4824 086/300 Write block 4680 len 4096 data: 2
4825 087/300 Flush
4826 088/300 Write block 1748 len 24576 data: 2 3 3 3 3 3
4827 089/300 Write block 1927 len 8192 data: 3 3
4828 090/300 Flush
4829 091/300 Read block 2271 len 4096
4830 092/300 Read block 1080 len 12288
4831 093/300 Read block 4072 len 36864
4832 094/300 Read block 2264 len 28672
4833 095/300 Write block 4345 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4834 096/300 Write block 4045 len 36864 data: 2 2 2 2 2 2 2 2 2
4835 097/300 Write block 64 len 20480 data: 2 2 2 2 2
4836 098/300 Write block 485 len 20480 data: 6 6 6 5 5
4837 099/300 Write block 3094 len 4096 data: 2
4838 100/300 Write block 1244 len 4096 data: 3
4839 101/300 Read block 1866 len 12288
4840 102/300 Read block 3561 len 16384
4841 103/300 Write block 202 len 16384 data: 3 3 3 3
4842 104/300 Flush
4843 105/300 Read block 1232 len 40960
4844 106/300 Read block 277 len 40960
4845 107/300 Write block 819 len 24576 data: 3 3 3 3 3 4
4846 108/300 Read block 3677 len 8192
4847 109/300 Write block 3362 len 28672 data: 2 2 2 2 3 3 3
4848 110/300 Read block 798 len 32768
4849 111/300 Write block 999 len 12288 data: 2 2 2
4850 112/300 Write block 1292 len 24576 data: 2 2 2 2 2 2
4851 113/300 Write block 4393 len 12288 data: 4 4 4
4852 114/300 Write block 4229 len 24576 data: 3 3 4 4 4 3
4853 115/300 Write block 2539 len 12288 data: 2 2 2
4854 116/300 Read block 62 len 12288
4855 117/300 Read block 639 len 28672
4856 118/300 Read block 455 len 8192
4857 119/300 Read block 664 len 4096
4858 120/300 Flush
4859 121/300 Write block 324 len 28672 data: 2 2 2 2 2 2 2
4860 122/300 Read block 3641 len 8192
4861 123/300 Write block 2522 len 32768 data: 3 3 3 3 3 3 3 3
4862 124/300 Write block 224 len 24576 data: 2 2 2 2 2 2
4863 125/300 Write block 1299 len 28672 data: 2 2 2 2 2 2 2
4864 126/300 Read block 327 len 16384
4865 127/300 Read block 3330 len 24576
4866 128/300 Read block 1219 len 24576
4867 129/300 Write block 4990 len 32768 data: 6 4 3 3 3 4 3 2
4868 130/300 Read block 3199 len 12288
4869 131/300 Read block 3922 len 32768
4870 132/300 Read block 4138 len 28672
4871 133/300 Read block 1746 len 12288
4872 134/300 Read block 2322 len 12288
4873 135/300 Read block 1820 len 20480
4874 136/300 Write block 527 len 16384 data: 4 4 5 5
4875 137/300 Flush
4876 138/300 Write block 4709 len 40960 data: 2 2 3 2 2 2 2 2 2 2
4877 139/300 Flush
4878 140/300 Read block 3950 len 32768
4879 141/300 Read block 1310 len 4096
4880 142/300 Write block 4837 len 4096 data: 3
4881 143/300 Write block 1346 len 20480 data: 3 3 3 2 2
4882 144/300 Read block 1901 len 28672
4883 145/300 Read block 2385 len 40960
4884 146/300 Write block 1702 len 32768 data: 3 3 3 3 3 3 2 2
4885 147/300 Read block 4174 len 20480
4886 148/300 Read block 1607 len 28672
4887 149/300 Read block 4029 len 4096
4888 150/300 Write block 616 len 16384 data: 3 3 4 4
4889 151/300 Write block 945 len 20480 data: 2 2 2 2 2
4890 152/300 Write block 2187 len 24576 data: 3 2 3 3 4 3
4891 153/300 Flush
4892 154/300 Write block 4762 len 36864 data: 4 2 2 2 2 2 2 2 2
4893 155/300 Read block 30 len 24576
4894 156/300 Read block 1779 len 8192
4895 157/300 Read block 4383 len 36864
4896 158/300 Read block 1682 len 36864
4897 159/300 Read block 1739 len 36864
4898 160/300 Read block 3292 len 8192
4899 161/300 Write block 3657 len 8192 data: 2 2
4900 162/300 Read block 3327 len 36864
4901 163/300 Read block 2050 len 16384
4902 164/300 Flush
4903 165/300 Write block 2027 len 40960 data: 4 3 3 3 3 4 4 3 3 3
4904 166/300 Read block 407 len 28672
4905 167/300 Read block 4648 len 24576
4906 168/300 Write block 3189 len 20480 data: 2 2 2 2 2
4907 169/300 Read block 1177 len 12288
4908 170/300 Read block 1095 len 8192
4909 171/300 Read block 4614 len 20480
4910 172/300 Write block 3202 len 16384 data: 3 3 3 3
4911 173/300 Read block 1285 len 20480
4912 174/300 Read block 3713 len 24576
4913 175/300 Write block 4428 len 4096 data: 3
4914 176/300 Read block 3216 len 12288
4915 177/300 Read block 1177 len 36864
4916 178/300 Write block 1574 len 24576 data: 3 3 4 4 3 3
4917 179/300 Read block 4830 len 20480
4918 180/300 Read block 3174 len 32768
4919 181/300 Read block 1486 len 32768
4920 182/300 Flush
4921 183/300 Write block 630 len 32768 data: 2 2 2 2 3 3 4 4
4922 184/300 Read block 929 len 8192
4923 185/300 Write block 2463 len 40960 data: 5 5 5 4 4 4 4 4 4 3
4924 186/300 Read block 4765 len 12288
4925 187/300 Read block 3647 len 20480
4926 188/300 Write block 3093 len 40960 data: 2 3 2 2 2 2 2 2 2 2
4927 189/300 Read block 2441 len 4096
4928 190/300 Read block 4338 len 28672
4929 191/300 Read block 3927 len 4096
4930 192/300 Write block 2753 len 20480 data: 2 2 2 2 2
4931 193/300 Write block 2542 len 12288 data: 2 2 3
4932 194/300 Read block 2212 len 28672
4933 195/300 Write block 2684 len 24576 data: 2 2 2 2 2 2
4934 196/300 Read block 2168 len 28672
4935 197/300 Write block 223 len 32768 data: 2 3 3 3 3 3 3 2
4936 198/300 Read block 2584 len 4096
4937 199/300 Write block 3711 len 16384 data: 3 3 3 3
4938 200/300 Flush
4939 201/300 Read block 2404 len 16384
4940 202/300 Read block 1438 len 40960
4941 203/300 Write block 1766 len 24576 data: 3 3 4 3 4 4
4942 204/300 Read block 1846 len 4096
4943 205/300 Read block 743 len 36864
4944 206/300 Read block 3529 len 32768
4945 207/300 Flush
4946 208/300 Read block 2915 len 28672
4947 209/300 Read block 3914 len 12288
4948 210/300 Flush
4949 211/300 Read block 4037 len 24576
4950 212/300 Write block 3875 len 20480 data: 2 2 2 3 3
4951 213/300 Read block 2501 len 36864
4952 214/300 Read block 4065 len 4096
4953 215/300 Write block 3308 len 4096 data: 2
4954 216/300 Read block 2249 len 36864
4955 217/300 Read block 2509 len 12288
4956 218/300 Write block 887 len 20480 data: 3 3 4 4 3
4957 219/300 Read block 541 len 24576
4958 220/300 Read block 3630 len 40960
4959 221/300 Write block 3854 len 20480 data: 3 3 2 2 2
4960 222/300 Write block 967 len 4096 data: 4
4961 223/300 Flush
4962 224/300 Read block 1952 len 12288
4963 225/300 Write block 4795 len 32768 data: 2 2 3 3 4 4 3 3
4964 226/300 Read block 1282 len 20480
4965 227/300 Read block 4793 len 28672
4966 228/300 Write block 3939 len 40960 data: 3 2 2 2 2 2 2 2 2 2
4967 229/300 Read block 116 len 36864
4968 230/300 Read block 4675 len 28672
4969 231/300 Write block 731 len 40960 data: 3 4 4 4 4 4 3 3 3 2
4970 232/300 Read block 1938 len 40960
4971 233/300 Write block 3440 len 4096 data: 2
4972 234/300 Read block 3446 len 4096
4973 235/300 Flush
4974 236/300 Read block 1343 len 40960
4975 237/300 Read block 2246 len 8192
4976 238/300 Read block 569 len 32768
4977 239/300 Read block 892 len 16384
4978 240/300 Read block 3833 len 16384
4979 241/300 Write block 4318 len 24576 data: 3 2 2 2 2 2
4980 242/300 Write block 4366 len 4096 data: 2
4981 243/300 Read block 4209 len 28672
4982 244/300 Read block 675 len 8192
4983 245/300 Read block 1155 len 4096
4984 246/300 Read block 1419 len 8192
4985 247/300 Read block 1274 len 32768
4986 248/300 Read block 4461 len 40960
4987 249/300 Write block 1781 len 28672 data: 3 3 3 3 3 3 2
4988 250/300 Write block 1390 len 4096 data: 2
4989 251/300 Read block 441 len 32768
4990 252/300 Read block 2794 len 12288
4991 253/300 Write block 4132 len 8192 data: 3 2
4992 254/300 Read block 1900 len 4096
4993 255/300 Read block 3325 len 36864
4994 256/300 Write block 2654 len 16384 data: 2 2 2 2
4995 257/300 Read block 199 len 8192
4996 258/300 Write block 3199 len 8192 data: 4 4
4997 259/300 Read block 657 len 12288
4998 260/300 Read block 2909 len 36864
4999 261/300 Read block 3000 len 24576
5000 262/300 Write block 2046 len 36864 data: 3 3 2 2 2 2 2 2 2
5001 263/300 Read block 1963 len 28672
5002 264/300 Write block 3263 len 36864 data: 2 2 3 3 3 3 3 3 3
5003 265/300 Read block 4956 len 16384
5004 266/300 Read block 377 len 28672
5005 267/300 Read block 1640 len 20480
5006 268/300 Read block 2378 len 12288
5007 269/300 Write block 1057 len 40960 data: 3 2 2 2 2 2 2 2 2 3
5008 270/300 Write block 3677 len 4096 data: 2
5009 271/300 Read block 2536 len 32768
5010 272/300 Read block 894 len 12288
5011 273/300 Read block 2090 len 12288
5012 274/300 Write block 3771 len 28672 data: 3 3 3 3 2 2 2
5013 275/300 Read block 812 len 8192
5014 276/300 Write block 3028 len 32768 data: 4 3 3 3 3 3 4 3
5015 277/300 Write block 746 len 20480 data: 2 2 2 2 2
5016 278/300 Read block 4942 len 32768
5017 279/300 Read block 814 len 8192
5018 280/300 Write block 683 len 4096 data: 3
5019 281/300 Read block 3704 len 8192
5020 282/300 Read block 3461 len 28672
5021 283/300 Read block 4264 len 32768
5022 284/300 Write block 4965 len 20480 data: 2 2 2 2 2
5023 285/300 Write block 2668 len 40960 data: 3 3 4 4 4 4 3 4 4 3
5024 286/300 Read block 653 len 4096
5025 287/300 Write block 3534 len 32768 data: 2 2 3 3 3 3 3 3
5026 288/300 Read block 4778 len 24576
5027 289/300 Read block 1606 len 32768
5028 290/300 Flush
5029 291/300 Read block 4836 len 8192
5030 292/300 Read block 4446 len 4096
5031 293/300 Read block 1502 len 36864
5032 294/300 Read block 2326 len 28672
5033 295/300 Write block 3698 len 24576 data: 2 2 2 2 2 2
5034 296/300 Read block 2626 len 24576
5035 297/300 Write block 3888 len 40960 data: 2 2 3 3 3 3 3 3 3 2
5036 298/300 Write block 259 len 20480 data: 3 3 3 4 3
5037 299/300 Read block 738 len 16384
5038 300/300 Read block 1145 len 28672
5039 301/300 Write block 2721 len 8192 data: 3 3
5040 Main task has work to do, go find it
5041 got dsc Start(1)
5042 start 1
5043 Jan 21 15:48:14.545 INFO request completed, latency_us: 215, response_code: 204, uri: /start/cid/1, method: POST, req_id: eabcf443-a6b3-4359-8937-ad413ca5b974, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
5044 Replay: started 1, returned:Ok(())
5045 ----------------------------------------------------------------
5046 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
5047 States: Active Offline Active
5048 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5049 2207 Acked Read 7 Done Sent Done false
5050 2208 Acked Read 5 Done Sent Done false
5051 2209 Acked Write 3 Done Sent Done false
5052 2210 Acked Write 9 Done Sent Done false
5053 2211 Acked Read 6 Done Sent Done false
5054 2212 Acked Read 7 Done Sent Done false
5055 2213 Acked Read 4 Done Sent Done false
5056 2214 Acked Write 2 Done Sent Done false
5057 2215 Acked Write 4 Done Sent Done false
5058 2216 Acked Read 9 Done Sent Done false
5059 2217 Acked Read 4 Done Sent Done false
5060 2218 Acked Read 5 Done Sent Done false
5061 2219 Acked Read 10 Done Sent Done false
5062 2220 Acked Read 9 Done Sent Done false
5063 2221 Acked Flush 0 Done Sent Done false
5064 2222 Acked Write 6 Done Sent Done false
5065 2223 Acked Read 10 Done Sent Done false
5066 2224 Acked Flush 0 Done Sent Done false
5067 2225 Acked Write 9 Done Sent Done false
5068 2226 Acked Write 5 Done Sent Done false
5069 2227 Acked Read 1 Done Sent Done false
5070 2228 Acked Read 9 Done Sent Done false
5071 2229 Acked Read 9 Done Sent Done false
5072 2230 Acked Write 1 Done Sent Done false
5073 2231 Acked Read 4 Done Sent Done false
5074 2232 Acked Write 2 Done Sent Done false
5075 2233 Acked Flush 0 Done Sent Done false
5076 2234 Acked Write 10 Done Sent Done false
5077 2235 Acked Flush 0 Done Sent Done false
5078 2236 Acked Write 4 Done Sent Done false
5079 2237 Acked Write 5 Done Sent Done false
5080 2238 Acked Read 1 Done Sent Done false
5081 2239 Acked Flush 0 Done Sent Done false
5082 2240 Acked Read 8 Done Sent Done false
5083 2241 Acked Read 7 Done Sent Done false
5084 2242 Acked Read 10 Done Sent Done false
5085 2243 Acked Read 3 Done Sent Done false
5086 2244 Acked Read 4 Done Sent Done false
5087 2245 Acked Read 1 Done Sent Done false
5088 2246 Acked Read 3 Done Sent Done false
5089 2247 Acked Write 3 Done Sent Done false
5090 2248 Acked Read 8 Done Sent Done false
5091 2249 Acked Read 9 Done Sent Done false
5092 2250 Acked Write 6 Done Sent Done false
5093 2251 Acked Write 1 Done Sent Done false
5094 2252 Acked Read 6 Done Sent Done false
5095 2253 Acked Read 1 Done Sent Done false
5096 2254 Acked Read 8 Done Sent Done false
5097 2255 Acked Read 5 Done Sent Done false
5098 2256 Acked Read 5 Done Sent Done false
5099 2257 Acked Write 3 Done Sent Done false
5100 2258 Acked Read 5 Done Sent Done false
5101 2259 Acked Read 1 Done Sent Done false
5102 2260 Acked Read 6 Done Sent Done false
5103 2261 Acked Write 8 Done Sent Done false
5104 2262 Acked Flush 0 Done Sent Done false
5105 2263 Acked Flush 0 Done Sent Done false
5106 2264 Acked Read 8 Done Sent Done false
5107 2265 Acked Write 9 Done Sent Done false
5108 2266 Acked Read 9 Done Sent Done false
5109 2267 Acked Read 9 Done Sent Done false
5110 2268 Acked Flush 0 Done Sent Done false
5111 2269 Acked Write 1 Done Sent Done false
5112 2270 Acked Write 5 Done Sent Done false
5113 2271 Acked Write 9 Done Sent Done false
5114 2272 Acked Write 3 Done Sent Done false
5115 2273 Acked Read 9 Done Sent Done false
5116 2274 Acked Flush 0 Done Sent Done false
5117 2275 Acked Write 8 Done Sent Done false
5118 2276 Acked Read 6 Done Sent Done false
5119 2277 Acked Write 1 Done Sent Done false
5120 2278 Acked Write 4 Done Sent Done false
5121 2279 Acked Write 5 Done Sent Done false
5122 2280 Acked Write 2 Done Sent Done false
5123 2281 Acked Read 9 Done Sent Done false
5124 2282 Acked Write 5 Done Sent Done false
5125 2283 Acked Write 9 Done Sent Done false
5126 2284 Acked Read 9 Done Sent Done false
5127 2285 Acked Read 3 Done Sent Done false
5128 2286 Acked Read 4 Done Sent Done false
5129 2287 Acked Read 3 Done Sent Done false
5130 2288 Acked Write 7 Done Sent Done false
5131 2289 Acked Write 10 Done Sent Done false
5132 2290 Acked Flush 0 Done Sent Done false
5133 2291 Acked Write 3 Done Sent Done false
5134 2292 Acked Write 1 Done Sent Done false
5135 2293 Acked Flush 0 Done Sent Done false
5136 2294 Acked Write 6 Done Sent Done false
5137 2295 Acked Write 2 Done Sent Done false
5138 2296 Acked Flush 0 Done Sent Done false
5139 2297 Acked Read 1 Done Sent Done false
5140 2298 Acked Read 3 Done Sent Done false
5141 2299 Acked Read 9 Done Sent Done false
5142 2300 Acked Read 7 Done Sent Done false
5143 2301 Acked Write 10 Done Sent Done false
5144 2302 Acked Write 9 Done Sent Done false
5145 2303 Acked Write 5 Done Sent Done false
5146 2304 Acked Write 5 Done Sent Done false
5147 2305 Acked Write 1 Done Sent Done false
5148 2306 Acked Write 1 Done Sent Done false
5149 2307 Acked Read 3 Done Sent Done false
5150 2308 Acked Read 4 Done Sent Done false
5151 2309 Acked Write 4 Done Sent Done false
5152 2310 Acked Flush 0 Done Sent Done false
5153 2311 Acked Read 10 Done Sent Done false
5154 2312 Acked Read 10 Done Sent Done false
5155 2313 Acked Write 6 Done Sent Done false
5156 2314 Acked Read 2 Done Sent Done false
5157 2315 Acked Write 7 Done Sent Done false
5158 2316 Acked Read 8 Done Sent Done false
5159 2317 Acked Write 3 Done Sent Done false
5160 2318 Acked Write 6 Done Sent Done false
5161 2319 Acked Write 3 Done Sent Done false
5162 2320 Acked Write 6 Done Sent Done false
5163 2321 Acked Write 3 Done Sent Done false
5164 2322 Acked Read 3 Done Sent Done false
5165 2323 Acked Read 7 Done Sent Done false
5166 2324 Acked Read 2 Done Sent Done false
5167 2325 Acked Read 1 Done Sent Done false
5168 2326 Acked Flush 0 Done Sent Done false
5169 2327 Acked Write 7 Done Sent Done false
5170 2328 Acked Read 2 Done Sent Done false
5171 2329 Acked Write 8 Done Sent Done false
5172 2330 Acked Write 6 Done Sent Done false
5173 2331 Acked Write 7 Done Sent Done false
5174 2332 Acked Read 4 Done Sent Done false
5175 2333 Acked Read 6 Done Sent Done false
5176 2334 Acked Read 6 Done Sent Done false
5177 2335 Acked Write 8 Done Sent Done false
5178 2336 Acked Read 3 Done Sent Done false
5179 2337 Acked Read 8 Done Sent Done false
5180 2338 Acked Read 7 Done Sent Done false
5181 2339 Acked Read 3 Done Sent Done false
5182 2340 Acked Read 3 Done Sent Done false
5183 2341 Acked Read 5 Done Sent Done false
5184 2342 Acked Write 4 Done Sent Done false
5185 2343 Acked Flush 0 Done Sent Done false
5186 2344 Acked Write 10 Done Sent Done false
5187 2345 Acked Flush 0 Done Sent Done false
5188 2346 Acked Read 8 Done Sent Done false
5189 2347 Acked Read 1 Done Sent Done false
5190 2348 Acked Write 1 Done Sent Done false
5191 2349 Acked Write 5 Done Sent Done false
5192 2350 Acked Read 7 Done Sent Done false
5193 2351 Acked Read 10 Done Sent Done false
5194 2352 Acked Write 8 Done Sent Done false
5195 2353 Acked Read 5 Done Sent Done false
5196 2354 Acked Read 7 Done Sent Done false
5197 2355 Acked Read 1 Done Sent Done false
5198 2356 Acked Write 4 Done Sent Done false
5199 2357 Acked Write 5 Done Sent Done false
5200 2358 Acked Write 6 Done Sent Done false
5201 2359 Acked Flush 0 Done Sent Done false
5202 2360 Acked Write 9 Done Sent Done false
5203 2361 Acked Read 6 Done Sent Done false
5204 2362 Acked Read 2 Done Sent Done false
5205 2363 Acked Read 9 Done Sent Done false
5206 2364 Acked Read 9 Done Sent Done false
5207 2365 Acked Read 9 Done Sent Done false
5208 2366 Acked Read 2 Done Sent Done false
5209 2367 Acked Write 2 Done Sent Done false
5210 2368 Acked Read 9 Done Sent Done false
5211 2369 Acked Read 4 Done Sent Done false
5212 2370 Acked Flush 0 Done Sent Done false
5213 2371 Acked Write 10 Done Sent Done false
5214 2372 Acked Read 7 Done Sent Done false
5215 2373 Acked Read 6 Done Sent Done false
5216 2374 Acked Write 5 Done Sent Done false
5217 2375 Acked Read 3 Done Sent Done false
5218 2376 Acked Read 2 Done Sent Done false
5219 2377 Acked Read 5 Done Sent Done false
5220 2378 Acked Write 4 Done Sent Done false
5221 2379 Acked Read 5 Done Sent Done false
5222 2380 Acked Read 6 Done Sent Done false
5223 2381 Acked Write 1 Done Sent Done false
5224 2382 Acked Read 3 Done Sent Done false
5225 2383 Acked Read 9 Done Sent Done false
5226 2384 Acked Write 6 Done Sent Done false
5227 2385 Acked Read 5 Done Sent Done false
5228 2386 Acked Read 8 Done Sent Done false
5229 2387 Acked Read 8 Done Sent Done false
5230 2388 Acked Flush 0 Done Sent Done false
5231 2389 Acked Write 8 Done Sent Done false
5232 2390 Acked Read 2 Done Sent Done false
5233 2391 Acked Write 10 Done Sent Done false
5234 2392 Acked Read 3 Done Sent Done false
5235 2393 Acked Read 5 Done Sent Done false
5236 2394 Acked Write 10 Done Sent Done false
5237 2395 Acked Read 1 Done Sent Done false
5238 2396 Acked Read 7 Done Sent Done false
5239 2397 Acked Read 1 Done Sent Done false
5240 2398 Acked Write 5 Done Sent Done false
5241 2399 Acked Write 3 Done Sent Done false
5242 2400 Acked Read 7 Done Sent Done false
5243 2401 Acked Write 6 Done Sent Done false
5244 2402 Acked Read 7 Done Sent Done false
5245 2403 Acked Write 8 Done Sent Done false
5246 2404 Acked Read 1 Done Sent Done false
5247 2405 Acked Write 4 Done Sent Done false
5248 2406 Acked Flush 0 Done Sent Done false
5249 2407 Acked Read 4 Done Sent Done false
5250 2408 Acked Read 10 Done Sent Done false
5251 2409 Acked Write 6 Done Sent Done false
5252 2410 Acked Read 1 Done Sent Done false
5253 2411 Acked Read 9 Done Sent Done false
5254 2412 Acked Read 8 Done Sent Done false
5255 2413 Acked Flush 0 Done Sent Done false
5256 2414 Acked Read 7 Done Sent Done false
5257 2415 Acked Read 3 Done Sent Done false
5258 2416 Acked Flush 0 Done Sent Done false
5259 2417 Acked Read 6 Done Sent Done false
5260 2418 Acked Write 5 Done Sent Done false
5261 2419 Acked Read 9 Done Sent Done false
5262 2420 Acked Read 1 Done Sent Done false
5263 2421 Acked Write 1 Done Sent Done false
5264 2422 Acked Read 9 Done Sent Done false
5265 2423 Acked Read 3 Done Sent Done false
5266 2424 Acked Write 5 Done Sent Done false
5267 2425 Acked Read 6 Done Sent Done false
5268 2426 Acked Read 10 Done Sent Done false
5269 2427 Acked Write 5 Done Sent Done false
5270 2428 Acked Write 1 Done Sent Done false
5271 2429 Acked Flush 0 Done Sent Done false
5272 2430 Acked Read 3 Done Sent Done false
5273 2431 Acked Write 8 Done Sent Done false
5274 2432 Acked Read 5 Done Sent Done false
5275 2433 Acked Read 7 Done Sent Done false
5276 2434 Acked Write 10 Done Sent Done false
5277 2435 Acked Read 9 Done Sent Done false
5278 2436 Acked Read 7 Done Sent Done false
5279 2437 Acked Write 10 Done Sent Done false
5280 2438 Acked Read 10 Done Sent Done false
5281 2439 Acked Write 1 Done Sent Done false
5282 2440 Acked Read 1 Done Sent Done false
5283 2441 Acked Flush 0 Done Sent Done false
5284 2442 Acked Read 10 Done Sent Done false
5285 2443 Acked Read 2 Done Sent Done false
5286 2444 Acked Read 8 Done Sent Done false
5287 2445 Acked Read 4 Done Sent Done false
5288 2446 Acked Read 4 Done Sent Done false
5289 2447 Acked Write 6 Done Sent Done false
5290 2448 Acked Write 1 Done Sent Done false
5291 2449 Acked Read 7 Done Sent Done false
5292 2450 Acked Read 2 Done Sent Done false
5293 2451 Acked Read 1 Done Sent Done false
5294 2452 Acked Read 2 Done Sent Done false
5295 2453 Acked Read 8 Done Sent Done false
5296 2454 Acked Read 10 Done Sent Done false
5297 2455 Acked Write 7 Done Sent Done false
5298 2456 Acked Write 1 Done Sent Done false
5299 2457 Acked Read 8 Done Sent Done false
5300 2458 Acked Read 3 Done Sent Done false
5301 2459 Acked Write 2 Done Sent Done false
5302 2460 Acked Read 1 Done Sent Done false
5303 2461 Acked Read 9 Done Sent Done false
5304 2462 Acked Write 4 Done Sent Done false
5305 2463 Acked Read 2 Done Sent Done false
5306 2464 Acked Write 2 Done Sent Done false
5307 2465 Acked Read 3 Done Sent Done false
5308 2466 Acked Read 9 Done Sent Done false
5309 2467 Acked Read 6 Done Sent Done false
5310 2468 Acked Write 9 Done Sent Done false
5311 2469 Acked Read 7 Done Sent Done false
5312 2470 Acked Write 9 Done Sent Done false
5313 2471 Acked Read 4 Done Sent Done false
5314 2472 Acked Read 7 Done Sent Done false
5315 2473 Acked Read 5 Done Sent Done false
5316 2474 Acked Read 3 Done Sent Done false
5317 2475 Acked Write 10 Done Sent Done false
5318 2476 Acked Write 1 Done Sent Done false
5319 2477 Acked Read 8 Done Sent Done false
5320 2478 Acked Read 3 Done Sent Done false
5321 2479 Acked Read 3 Done Sent Done false
5322 2480 Acked Write 7 Done Sent Done false
5323 2481 Acked Read 2 Done Sent Done false
5324 2482 Acked Write 8 Done Sent Done false
5325 2483 Acked Write 5 Done Sent Done false
5326 2484 Acked Read 8 Done Sent Done false
5327 2485 Acked Read 2 Done Sent Done false
5328 2486 Acked Write 1 Done Sent Done false
5329 2487 Acked Read 2 Done Sent Done false
5330 2488 Acked Read 7 Done Sent Done false
5331 2489 Acked Read 8 Done Sent Done false
5332 2490 Acked Write 5 Done Sent Done false
5333 2491 Acked Write 10 Done Sent Done false
5334 2492 Acked Read 1 Done Sent Done false
5335 2493 Acked Write 8 Done Sent Done false
5336 2494 Acked Read 6 Done Sent Done false
5337 2495 Acked Read 8 Done Sent Done false
5338 2496 Acked Flush 0 Done Sent Done false
5339 2497 Acked Read 2 Done Sent Done false
5340 2498 Acked Read 1 Done Sent Done false
5341 2499 Acked Read 9 Done Sent Done false
5342 2500 Acked Read 7 Done Sent Done false
5343 2501 Acked Write 6 Done Sent Done false
5344 2502 Acked Read 6 Done Sent Done false
5345 2503 Acked Write 10 Done Sent Done false
5346 2504 Acked Write 5 Done Sent Done false
5347 2505 Acked Read 4 Done Sent Done false
5348 2506 Acked Read 7 Done Sent Done false
5349 2507 Acked Write 2 Done Sent Done false
5350 STATES DS:0 DS:1 DS:2 TOTAL
5351 Sent 0 301 0 301
5352 Done 301 0 301 602
5353 Skipped 0 0 0 0
5354 Error 0 0 0 0
5355 Last Flush: 2496 2206 2496
5356 Downstairs last five completed: 2206 2205 2204 2203 2202
5357 Upstairs last five completed: 2507 2506 2505 2504 2503
5358 CLIENT: Up:0 ds:301 act:2
5359 [8820] got start action
5360 [8820] I am going to restart
5361 Starting downstairs at port 8820
5362 Make output file at "/tmp/dsc/downstairs-8820.txt"
5363 [8820][1] reports Starting
5364 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1205)
5365 [8820][1] reports Running
5366 ----------------------------------------------------------------
5367 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5368 States: Active Offline Active
5369 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5370 2207 Acked Read 7 Done Sent Done false
5371 2208 Acked Read 5 Done Sent Done false
5372 2209 Acked Write 3 Done Sent Done false
5373 2210 Acked Write 9 Done Sent Done false
5374 2211 Acked Read 6 Done Sent Done false
5375 2212 Acked Read 7 Done Sent Done false
5376 2213 Acked Read 4 Done Sent Done false
5377 2214 Acked Write 2 Done Sent Done false
5378 2215 Acked Write 4 Done Sent Done false
5379 2216 Acked Read 9 Done Sent Done false
5380 2217 Acked Read 4 Done Sent Done false
5381 2218 Acked Read 5 Done Sent Done false
5382 2219 Acked Read 10 Done Sent Done false
5383 2220 Acked Read 9 Done Sent Done false
5384 2221 Acked Flush 0 Done Sent Done false
5385 2222 Acked Write 6 Done Sent Done false
5386 2223 Acked Read 10 Done Sent Done false
5387 2224 Acked Flush 0 Done Sent Done false
5388 2225 Acked Write 9 Done Sent Done false
5389 2226 Acked Write 5 Done Sent Done false
5390 2227 Acked Read 1 Done Sent Done false
5391 2228 Acked Read 9 Done Sent Done false
5392 2229 Acked Read 9 Done Sent Done false
5393 2230 Acked Write 1 Done Sent Done false
5394 2231 Acked Read 4 Done Sent Done false
5395 2232 Acked Write 2 Done Sent Done false
5396 2233 Acked Flush 0 Done Sent Done false
5397 2234 Acked Write 10 Done Sent Done false
5398 2235 Acked Flush 0 Done Sent Done false
5399 2236 Acked Write 4 Done Sent Done false
5400 2237 Acked Write 5 Done Sent Done false
5401 2238 Acked Read 1 Done Sent Done false
5402 2239 Acked Flush 0 Done Sent Done false
5403 2240 Acked Read 8 Done Sent Done false
5404 2241 Acked Read 7 Done Sent Done false
5405 2242 Acked Read 10 Done Sent Done false
5406 2243 Acked Read 3 Done Sent Done false
5407 2244 Acked Read 4 Done Sent Done false
5408 2245 Acked Read 1 Done Sent Done false
5409 2246 Acked Read 3 Done Sent Done false
5410 2247 Acked Write 3 Done Sent Done false
5411 2248 Acked Read 8 Done Sent Done false
5412 2249 Acked Read 9 Done Sent Done false
5413 2250 Acked Write 6 Done Sent Done false
5414 2251 Acked Write 1 Done Sent Done false
5415 2252 Acked Read 6 Done Sent Done false
5416 2253 Acked Read 1 Done Sent Done false
5417 2254 Acked Read 8 Done Sent Done false
5418 2255 Acked Read 5 Done Sent Done false
5419 2256 Acked Read 5 Done Sent Done false
5420 2257 Acked Write 3 Done Sent Done false
5421 2258 Acked Read 5 Done Sent Done false
5422 2259 Acked Read 1 Done Sent Done false
5423 2260 Acked Read 6 Done Sent Done false
5424 2261 Acked Write 8 Done Sent Done false
5425 2262 Acked Flush 0 Done Sent Done false
5426 2263 Acked Flush 0 Done Sent Done false
5427 2264 Acked Read 8 Done Sent Done false
5428 2265 Acked Write 9 Done Sent Done false
5429 2266 Acked Read 9 Done Sent Done false
5430 2267 Acked Read 9 Done Sent Done false
5431 2268 Acked Flush 0 Done Sent Done false
5432 2269 Acked Write 1 Done Sent Done false
5433 2270 Acked Write 5 Done Sent Done false
5434 2271 Acked Write 9 Done Sent Done false
5435 2272 Acked Write 3 Done Sent Done false
5436 2273 Acked Read 9 Done Sent Done false
5437 2274 Acked Flush 0 Done Sent Done false
5438 2275 Acked Write 8 Done Sent Done false
5439 2276 Acked Read 6 Done Sent Done false
5440 2277 Acked Write 1 Done Sent Done false
5441 2278 Acked Write 4 Done Sent Done false
5442 2279 Acked Write 5 Done Sent Done false
5443 2280 Acked Write 2 Done Sent Done false
5444 2281 Acked Read 9 Done Sent Done false
5445 2282 Acked Write 5 Done Sent Done false
5446 2283 Acked Write 9 Done Sent Done false
5447 2284 Acked Read 9 Done Sent Done false
5448 2285 Acked Read 3 Done Sent Done false
5449 2286 Acked Read 4 Done Sent Done false
5450 2287 Acked Read 3 Done Sent Done false
5451 2288 Acked Write 7 Done Sent Done false
5452 2289 Acked Write 10 Done Sent Done false
5453 2290 Acked Flush 0 Done Sent Done false
5454 2291 Acked Write 3 Done Sent Done false
5455 2292 Acked Write 1 Done Sent Done false
5456 2293 Acked Flush 0 Done Sent Done false
5457 2294 Acked Write 6 Done Sent Done false
5458 2295 Acked Write 2 Done Sent Done false
5459 2296 Acked Flush 0 Done Sent Done false
5460 2297 Acked Read 1 Done Sent Done false
5461 2298 Acked Read 3 Done Sent Done false
5462 2299 Acked Read 9 Done Sent Done false
5463 2300 Acked Read 7 Done Sent Done false
5464 2301 Acked Write 10 Done Sent Done false
5465 2302 Acked Write 9 Done Sent Done false
5466 2303 Acked Write 5 Done Sent Done false
5467 2304 Acked Write 5 Done Sent Done false
5468 2305 Acked Write 1 Done Sent Done false
5469 2306 Acked Write 1 Done Sent Done false
5470 2307 Acked Read 3 Done Sent Done false
5471 2308 Acked Read 4 Done Sent Done false
5472 2309 Acked Write 4 Done Sent Done false
5473 2310 Acked Flush 0 Done Sent Done false
5474 2311 Acked Read 10 Done Sent Done false
5475 2312 Acked Read 10 Done Sent Done false
5476 2313 Acked Write 6 Done Sent Done false
5477 2314 Acked Read 2 Done Sent Done false
5478 2315 Acked Write 7 Done Sent Done false
5479 2316 Acked Read 8 Done Sent Done false
5480 2317 Acked Write 3 Done Sent Done false
5481 2318 Acked Write 6 Done Sent Done false
5482 2319 Acked Write 3 Done Sent Done false
5483 2320 Acked Write 6 Done Sent Done false
5484 2321 Acked Write 3 Done Sent Done false
5485 2322 Acked Read 3 Done Sent Done false
5486 2323 Acked Read 7 Done Sent Done false
5487 2324 Acked Read 2 Done Sent Done false
5488 2325 Acked Read 1 Done Sent Done false
5489 2326 Acked Flush 0 Done Sent Done false
5490 2327 Acked Write 7 Done Sent Done false
5491 2328 Acked Read 2 Done Sent Done false
5492 2329 Acked Write 8 Done Sent Done false
5493 2330 Acked Write 6 Done Sent Done false
5494 2331 Acked Write 7 Done Sent Done false
5495 2332 Acked Read 4 Done Sent Done false
5496 2333 Acked Read 6 Done Sent Done false
5497 2334 Acked Read 6 Done Sent Done false
5498 2335 Acked Write 8 Done Sent Done false
5499 2336 Acked Read 3 Done Sent Done false
5500 2337 Acked Read 8 Done Sent Done false
5501 2338 Acked Read 7 Done Sent Done false
5502 2339 Acked Read 3 Done Sent Done false
5503 2340 Acked Read 3 Done Sent Done false
5504 2341 Acked Read 5 Done Sent Done false
5505 2342 Acked Write 4 Done Sent Done false
5506 2343 Acked Flush 0 Done Sent Done false
5507 2344 Acked Write 10 Done Sent Done false
5508 2345 Acked Flush 0 Done Sent Done false
5509 2346 Acked Read 8 Done Sent Done false
5510 2347 Acked Read 1 Done Sent Done false
5511 2348 Acked Write 1 Done Sent Done false
5512 2349 Acked Write 5 Done Sent Done false
5513 2350 Acked Read 7 Done Sent Done false
5514 2351 Acked Read 10 Done Sent Done false
5515 2352 Acked Write 8 Done Sent Done false
5516 2353 Acked Read 5 Done Sent Done false
5517 2354 Acked Read 7 Done Sent Done false
5518 2355 Acked Read 1 Done Sent Done false
5519 2356 Acked Write 4 Done Sent Done false
5520 2357 Acked Write 5 Done Sent Done false
5521 2358 Acked Write 6 Done Sent Done false
5522 2359 Acked Flush 0 Done Sent Done false
5523 2360 Acked Write 9 Done Sent Done false
5524 2361 Acked Read 6 Done Sent Done false
5525 2362 Acked Read 2 Done Sent Done false
5526 2363 Acked Read 9 Done Sent Done false
5527 2364 Acked Read 9 Done Sent Done false
5528 2365 Acked Read 9 Done Sent Done false
5529 2366 Acked Read 2 Done Sent Done false
5530 2367 Acked Write 2 Done Sent Done false
5531 2368 Acked Read 9 Done Sent Done false
5532 2369 Acked Read 4 Done Sent Done false
5533 2370 Acked Flush 0 Done Sent Done false
5534 2371 Acked Write 10 Done Sent Done false
5535 2372 Acked Read 7 Done Sent Done false
5536 2373 Acked Read 6 Done Sent Done false
5537 2374 Acked Write 5 Done Sent Done false
5538 2375 Acked Read 3 Done Sent Done false
5539 2376 Acked Read 2 Done Sent Done false
5540 2377 Acked Read 5 Done Sent Done false
5541 2378 Acked Write 4 Done Sent Done false
5542 2379 Acked Read 5 Done Sent Done false
5543 2380 Acked Read 6 Done Sent Done false
5544 2381 Acked Write 1 Done Sent Done false
5545 2382 Acked Read 3 Done Sent Done false
5546 2383 Acked Read 9 Done Sent Done false
5547 2384 Acked Write 6 Done Sent Done false
5548 2385 Acked Read 5 Done Sent Done false
5549 2386 Acked Read 8 Done Sent Done false
5550 2387 Acked Read 8 Done Sent Done false
5551 2388 Acked Flush 0 Done Sent Done false
5552 2389 Acked Write 8 Done Sent Done false
5553 2390 Acked Read 2 Done Sent Done false
5554 2391 Acked Write 10 Done Sent Done false
5555 2392 Acked Read 3 Done Sent Done false
5556 2393 Acked Read 5 Done Sent Done false
5557 2394 Acked Write 10 Done Sent Done false
5558 2395 Acked Read 1 Done Sent Done false
5559 2396 Acked Read 7 Done Sent Done false
5560 2397 Acked Read 1 Done Sent Done false
5561 2398 Acked Write 5 Done Sent Done false
5562 2399 Acked Write 3 Done Sent Done false
5563 2400 Acked Read 7 Done Sent Done false
5564 2401 Acked Write 6 Done Sent Done false
5565 2402 Acked Read 7 Done Sent Done false
5566 2403 Acked Write 8 Done Sent Done false
5567 2404 Acked Read 1 Done Sent Done false
5568 2405 Acked Write 4 Done Sent Done false
5569 2406 Acked Flush 0 Done Sent Done false
5570 2407 Acked Read 4 Done Sent Done false
5571 2408 Acked Read 10 Done Sent Done false
5572 2409 Acked Write 6 Done Sent Done false
5573 2410 Acked Read 1 Done Sent Done false
5574 2411 Acked Read 9 Done Sent Done false
5575 2412 Acked Read 8 Done Sent Done false
5576 2413 Acked Flush 0 Done Sent Done false
5577 2414 Acked Read 7 Done Sent Done false
5578 2415 Acked Read 3 Done Sent Done false
5579 2416 Acked Flush 0 Done Sent Done false
5580 2417 Acked Read 6 Done Sent Done false
5581 2418 Acked Write 5 Done Sent Done false
5582 2419 Acked Read 9 Done Sent Done false
5583 2420 Acked Read 1 Done Sent Done false
5584 2421 Acked Write 1 Done Sent Done false
5585 2422 Acked Read 9 Done Sent Done false
5586 2423 Acked Read 3 Done Sent Done false
5587 2424 Acked Write 5 Done Sent Done false
5588 2425 Acked Read 6 Done Sent Done false
5589 2426 Acked Read 10 Done Sent Done false
5590 2427 Acked Write 5 Done Sent Done false
5591 2428 Acked Write 1 Done Sent Done false
5592 2429 Acked Flush 0 Done Sent Done false
5593 2430 Acked Read 3 Done Sent Done false
5594 2431 Acked Write 8 Done Sent Done false
5595 2432 Acked Read 5 Done Sent Done false
5596 2433 Acked Read 7 Done Sent Done false
5597 2434 Acked Write 10 Done Sent Done false
5598 2435 Acked Read 9 Done Sent Done false
5599 2436 Acked Read 7 Done Sent Done false
5600 2437 Acked Write 10 Done Sent Done false
5601 2438 Acked Read 10 Done Sent Done false
5602 2439 Acked Write 1 Done Sent Done false
5603 2440 Acked Read 1 Done Sent Done false
5604 2441 Acked Flush 0 Done Sent Done false
5605 2442 Acked Read 10 Done Sent Done false
5606 2443 Acked Read 2 Done Sent Done false
5607 2444 Acked Read 8 Done Sent Done false
5608 2445 Acked Read 4 Done Sent Done false
5609 2446 Acked Read 4 Done Sent Done false
5610 2447 Acked Write 6 Done Sent Done false
5611 2448 Acked Write 1 Done Sent Done false
5612 2449 Acked Read 7 Done Sent Done false
5613 2450 Acked Read 2 Done Sent Done false
5614 2451 Acked Read 1 Done Sent Done false
5615 2452 Acked Read 2 Done Sent Done false
5616 2453 Acked Read 8 Done Sent Done false
5617 2454 Acked Read 10 Done Sent Done false
5618 2455 Acked Write 7 Done Sent Done false
5619 2456 Acked Write 1 Done Sent Done false
5620 2457 Acked Read 8 Done Sent Done false
5621 2458 Acked Read 3 Done Sent Done false
5622 2459 Acked Write 2 Done Sent Done false
5623 2460 Acked Read 1 Done Sent Done false
5624 2461 Acked Read 9 Done Sent Done false
5625 2462 Acked Write 4 Done Sent Done false
5626 2463 Acked Read 2 Done Sent Done false
5627 2464 Acked Write 2 Done Sent Done false
5628 2465 Acked Read 3 Done Sent Done false
5629 2466 Acked Read 9 Done Sent Done false
5630 2467 Acked Read 6 Done Sent Done false
5631 2468 Acked Write 9 Done Sent Done false
5632 2469 Acked Read 7 Done Sent Done false
5633 2470 Acked Write 9 Done Sent Done false
5634 2471 Acked Read 4 Done Sent Done false
5635 2472 Acked Read 7 Done Sent Done false
5636 2473 Acked Read 5 Done Sent Done false
5637 2474 Acked Read 3 Done Sent Done false
5638 2475 Acked Write 10 Done Sent Done false
5639 2476 Acked Write 1 Done Sent Done false
5640 2477 Acked Read 8 Done Sent Done false
5641 2478 Acked Read 3 Done Sent Done false
5642 2479 Acked Read 3 Done Sent Done false
5643 2480 Acked Write 7 Done Sent Done false
5644 2481 Acked Read 2 Done Sent Done false
5645 2482 Acked Write 8 Done Sent Done false
5646 2483 Acked Write 5 Done Sent Done false
5647 2484 Acked Read 8 Done Sent Done false
5648 2485 Acked Read 2 Done Sent Done false
5649 2486 Acked Write 1 Done Sent Done false
5650 2487 Acked Read 2 Done Sent Done false
5651 2488 Acked Read 7 Done Sent Done false
5652 2489 Acked Read 8 Done Sent Done false
5653 2490 Acked Write 5 Done Sent Done false
5654 2491 Acked Write 10 Done Sent Done false
5655 2492 Acked Read 1 Done Sent Done false
5656 2493 Acked Write 8 Done Sent Done false
5657 2494 Acked Read 6 Done Sent Done false
5658 2495 Acked Read 8 Done Sent Done false
5659 2496 Acked Flush 0 Done Sent Done false
5660 2497 Acked Read 2 Done Sent Done false
5661 2498 Acked Read 1 Done Sent Done false
5662 2499 Acked Read 9 Done Sent Done false
5663 2500 Acked Read 7 Done Sent Done false
5664 2501 Acked Write 6 Done Sent Done false
5665 2502 Acked Read 6 Done Sent Done false
5666 2503 Acked Write 10 Done Sent Done false
5667 2504 Acked Write 5 Done Sent Done false
5668 2505 Acked Read 4 Done Sent Done false
5669 2506 Acked Read 7 Done Sent Done false
5670 2507 Acked Write 2 Done Sent Done false
5671 2508 Acked Flush 0 Done Sent Done false
5672 STATES DS:0 DS:1 DS:2 TOTAL
5673 Sent 0 302 0 302
5674 Done 302 0 302 604
5675 Skipped 0 0 0 0
5676 Error 0 0 0 0
5677 Last Flush: 2508 2206 2508
5678 Downstairs last five completed: 2206 2205 2204 2203 2202
5679 Upstairs last five completed: 2508 2507 2506 2505 2504
5680 CLIENT: Up:0 ds:302 act:2
56812025-01-21T15:48:20.293ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
56822025-01-21T15:48:20.293ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59530) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
56832025-01-21T15:48:20.294ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
56842025-01-21T15:48:20.295ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
56852025-01-21T15:48:20.295ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
56862025-01-21T15:48:20.295ZINFOcrucible: send last flush ID to this DS: 2206 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
56872025-01-21T15:48:20.295ZINFOcrucible: Replied this last flush ID: 2206 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
56882025-01-21T15:48:20.295ZINFOcrucible: [1] client re-new 302 jobs since flush 2206 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
56892025-01-21T15:48:20.295ZINFOcrucible: [1] Replayed 302 jobs since flush: 2206 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
5690 ----------------------------------------------------------------
5691 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5692 Downstairs last five completed: 2508 2507 2506 2505 2504
5693 Upstairs last five completed: 2508 2507 2506 2505 2504
5694 CLIENT: Up:0 ds:0 act:3
5695 Replay: All jobs finished
5696 Main task has work to do, go find it
5697 got dsc Stop(0)
5698 stop 0
5699 [8810] Got stop action so:false kr:false
5700 Jan 21 15:48:22.551 INFO request completed, latency_us: 391, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 7092abc6-2ec1-4cff-804c-0c144ef2a591, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
57012025-01-21T15:48:22.552ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
57022025-01-21T15:48:22.552ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
57032025-01-21T15:48:22.552ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
57042025-01-21T15:48:22.552ZINFOcrucible: client task is exiting = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
5705 Jan 21 15:48:22.552 INFO request completed, latency_us: 248, response_code: 200, uri: /state/cid/0, method: GET, req_id: 1e259f4b-5dcd-4e70-a9d2-74c8789ecb73, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
5706 [8810] Exited with: ExitStatus(unix_wait_status(9))
5707 [8810][0] reports Exit
5708 Jan 21 15:48:26.555 INFO request completed, latency_us: 257, response_code: 200, uri: /state/cid/0, method: GET, req_id: dbb34677-15f5-438f-a19c-ce9c347c99c4, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
5709 001/300 Write block 3725 len 32768 data: 3 3 3 3 2 2 2 2
5710 002/300 Read block 2601 len 28672
5711 003/300 Flush
5712 004/300 Read block 4018 len 4096
5713 005/300 Read block 3691 len 8192
5714 006/300 Read block 3454 len 20480
5715 007/300 Read block 4692 len 20480
5716 008/300 Write block 4386 len 20480 data: 2 2 4 6 6
5717 009/300 Read block 1076 len 24576
5718 010/300 Read block 3506 len 32768
5719 011/300 Read block 2480 len 24576
5720 012/300 Write block 2311 len 8192 data: 2 2
5721 013/300 Read block 955 len 16384
5722 014/300 Read block 4596 len 32768
5723 015/300 Write block 3085 len 8192 data: 3 3
5724 016/300 Flush
5725 017/300 Read block 1152 len 20480
5726 018/300 Write block 3439 len 28672 data: 2 3 2 2 2 2 2
5727 019/300 Read block 3663 len 4096
5728 020/300 Read block 112 len 4096
5729 021/300 Read block 4822 len 24576
5730 022/300 Read block 2403 len 36864
5731 023/300 Write block 2541 len 32768 data: 3 3 3 4 3 3 3 4
5732 024/300 Read block 1188 len 32768
5733 025/300 Read block 4528 len 20480
5734 026/300 Write block 1019 len 20480 data: 2 2 2 2 2
5735 027/300 Write block 2583 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5736 028/300 Write block 2621 len 40960 data: 2 2 2 2 2 2 2 2 3 3
5737 029/300 Read block 2016 len 16384
5738 030/300 Read block 437 len 20480
5739 031/300 Read block 3309 len 32768
5740 032/300 Read block 1635 len 40960
5741 033/300 Read block 1709 len 4096
5742 034/300 Write block 3230 len 12288 data: 2 2 2
5743 035/300 Flush
5744 036/300 Read block 64 len 4096
5745 037/300 Write block 1038 len 28672 data: 4 4 4 4 4 4 4
5746 038/300 Write block 3355 len 36864 data: 3 2 2 2 2 2 2 3 3
5747 039/300 Flush
5748 040/300 Read block 326 len 28672
5749 041/300 Read block 3398 len 4096
5750 042/300 Read block 1880 len 36864
5751 043/300 Write block 2254 len 40960 data: 2 2 2 2 2 2 2 2 3 3
5752 044/300 Read block 244 len 36864
5753 045/300 Write block 2569 len 8192 data: 3 3
5754 046/300 Write block 2004 len 20480 data: 3 3 2 2 2
5755 047/300 Read block 2013 len 16384
5756 048/300 Read block 1515 len 28672
5757 049/300 Read block 552 len 20480
5758 050/300 Write block 2013 len 28672 data: 2 2 2 2 3 4 3
5759 051/300 Read block 279 len 8192
5760 052/300 Read block 1 len 40960
5761 053/300 Read block 1350 len 8192
5762 054/300 Write block 443 len 24576 data: 2 2 2 2 2 2
5763 055/300 Write block 1896 len 40960 data: 2 2 2 2 3 4 4 4 4 5
5764 056/300 Write block 4870 len 4096 data: 2
5765 057/300 Read block 4483 len 28672
5766 058/300 Write block 4151 len 20480 data: 3 4 4 3 3
5767 059/300 Read block 810 len 4096
5768 060/300 Write block 4630 len 16384 data: 2 2 3 3
5769 061/300 Read block 902 len 36864
5770 062/300 Read block 1047 len 16384
5771 063/300 Flush
5772 064/300 Flush
5773 065/300 Read block 4293 len 24576
5774 066/300 Write block 1144 len 32768 data: 3 2 2 2 2 2 2 3
5775 067/300 Read block 3960 len 4096
5776 068/300 Read block 4858 len 4096
5777 069/300 Read block 1249 len 24576
5778 070/300 Flush
5779 071/300 Write block 3781 len 28672 data: 2 2 3 3 3 3 3
5780 072/300 Write block 4438 len 28672 data: 2 2 2 2 3 2 2
5781 073/300 Read block 2535 len 36864
5782 074/300 Read block 2467 len 24576
5783 075/300 Write block 1323 len 8192 data: 4 4
5784 076/300 Write block 3350 len 16384 data: 3 3 3 3
5785 077/300 Read block 4062 len 16384
5786 078/300 Write block 3248 len 20480 data: 3 2 2 2 3
5787 079/300 Read block 1408 len 36864
5788 080/300 Flush
5789 081/300 Write block 2368 len 12288 data: 2 2 2
5790 082/300 Write block 2897 len 32768 data: 2 2 2 2 2 2 2 2
5791 083/300 Read block 62 len 16384
5792 084/300 Write block 1352 len 16384 data: 3 2 2 2
5793 085/300 Read block 3148 len 24576
5794 086/300 Write block 1777 len 8192 data: 2 3
5795 087/300 Read block 2319 len 12288
5796 088/300 Flush
5797 089/300 Flush
5798 090/300 Read block 4557 len 24576
5799 091/300 Write block 4525 len 4096 data: 2
5800 092/300 Read block 981 len 16384
5801 093/300 Write block 3198 len 16384 data: 3 5 5 4
5802 094/300 Read block 3673 len 16384
5803 095/300 Write block 155 len 16384 data: 2 3 3 2
5804 096/300 Read block 1605 len 36864
5805 097/300 Write block 3627 len 24576 data: 5 6 7 3 3 2
5806 098/300 Read block 683 len 32768
5807 099/300 Write block 2210 len 20480 data: 2 2 2 2 2
5808 100/300 Write block 646 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5809 101/300 Write block 4903 len 36864 data: 2 2 2 2 2 2 3 4 4
5810 102/300 Read block 1438 len 4096
5811 103/300 Read block 653 len 28672
5812 104/300 Write block 3283 len 8192 data: 3 2
5813 105/300 Write block 2858 len 4096 data: 3
5814 106/300 Write block 3346 len 24576 data: 3 3 4 3 4 4
5815 107/300 Read block 2047 len 32768
5816 108/300 Read block 3204 len 24576
5817 109/300 Flush
5818 110/300 Flush
5819 111/300 Read block 3153 len 36864
5820 112/300 Write block 4170 len 40960 data: 3 3 3 3 3 3 2 2 3 4
5821 113/300 Write block 2468 len 20480 data: 5 5 5 5 4
5822 114/300 Write block 187 len 32768 data: 4 4 3 2 2 2 2 3
5823 115/300 Flush
5824 116/300 Read block 1646 len 4096
5825 117/300 Flush
5826 118/300 Write block 3133 len 16384 data: 2 2 2 2
5827 119/300 Write block 298 len 36864 data: 3 3 3 4 3 3 3 3 3
5828 120/300 Flush
5829 121/300 Read block 4453 len 16384
5830 122/300 Write block 3081 len 4096 data: 2
5831 123/300 Read block 3820 len 12288
5832 124/300 Write block 4985 len 28672 data: 4 5 7 7 6 7 5
5833 125/300 Read block 4889 len 8192
5834 126/300 Flush
5835 127/300 Read block 4643 len 28672
5836 128/300 Read block 4695 len 20480
5837 129/300 Write block 638 len 16384 data: 4 4 3 3
5838 130/300 Flush
5839 131/300 Read block 4704 len 4096
5840 132/300 Read block 1440 len 24576
5841 133/300 Flush
5842 134/300 Write block 1872 len 8192 data: 2 3
5843 135/300 Read block 3067 len 16384
5844 136/300 Write block 3062 len 36864 data: 2 2 2 2 2 2 2 2 2
5845 137/300 Read block 4123 len 20480
5846 138/300 Read block 3167 len 8192
5847 139/300 Read block 750 len 36864
5848 140/300 Read block 3505 len 40960
5849 141/300 Flush
5850 142/300 Read block 929 len 32768
5851 143/300 Read block 2302 len 40960
5852 144/300 Write block 4482 len 20480 data: 4 4 4 3 3
5853 145/300 Write block 3450 len 28672 data: 2 2 2 2 2 2 2
5854 146/300 Write block 1259 len 36864 data: 2 2 2 2 2 2 2 2 2
5855 147/300 Write block 277 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5856 148/300 Write block 4432 len 36864 data: 2 2 2 2 2 2 3 3 3
5857 149/300 Write block 1267 len 4096 data: 3
5858 150/300 Read block 2997 len 4096
5859 151/300 Write block 737 len 8192 data: 4 4
5860 152/300 Read block 3370 len 12288
5861 153/300 Write block 570 len 8192 data: 2 2
5862 154/300 Read block 2545 len 12288
5863 155/300 Write block 3791 len 32768 data: 2 2 2 2 2 2 2 2
5864 156/300 Read block 2216 len 36864
5865 157/300 Read block 3033 len 12288
5866 158/300 Write block 1529 len 28672 data: 2 2 2 2 2 2 3
5867 159/300 Read block 1583 len 12288
5868 160/300 Flush
5869 161/300 Read block 1420 len 16384
5870 162/300 Flush
5871 163/300 Read block 3748 len 24576
5872 164/300 Read block 1305 len 20480
5873 165/300 Read block 1642 len 24576
5874 166/300 Read block 4017 len 12288
5875 167/300 Write block 2236 len 28672 data: 4 3 3 2 2 2 2
5876 168/300 Read block 2336 len 12288
5877 169/300 Read block 1443 len 12288
5878 170/300 Read block 3837 len 32768
5879 171/300 Read block 706 len 32768
5880 172/300 Write block 4477 len 4096 data: 2
5881 173/300 Read block 1011 len 20480
5882 174/300 Flush
5883 175/300 Read block 2925 len 4096
5884 176/300 Write block 1761 len 36864 data: 2 2 2 2 2 4 4 5 4
5885 177/300 Read block 1775 len 32768
5886 178/300 Write block 337 len 32768 data: 2 2 3 3 2 2 2 2
5887 179/300 Write block 2242 len 36864 data: 3 2 2 2 2 2 3 3 4
5888 180/300 Read block 825 len 16384
5889 181/300 Read block 677 len 12288
5890 182/300 Write block 1050 len 36864 data: 3 3 3 3 3 3 3 4 3
5891 183/300 Write block 1204 len 12288 data: 2 2 2
5892 184/300 Write block 4290 len 4096 data: 2
5893 185/300 Write block 2035 len 8192 data: 4 4
5894 186/300 Write block 2893 len 20480 data: 3 3 3 3 3
5895 187/300 Write block 1607 len 20480 data: 2 2 2 2 2
5896 188/300 Read block 4743 len 8192
5897 189/300 Read block 2585 len 40960
5898 190/300 Flush
5899 191/300 Write block 354 len 36864 data: 2 2 2 2 3 3 3 3 2
5900 192/300 Write block 807 len 24576 data: 3 3 4 4 4 3
5901 193/300 Read block 4853 len 4096
5902 194/300 Read block 3464 len 4096
5903 195/300 Read block 2321 len 28672
5904 196/300 Flush
5905 197/300 Read block 1464 len 24576
5906 198/300 Read block 3840 len 12288
5907 199/300 Read block 495 len 28672
5908 200/300 Read block 3296 len 4096
5909 201/300 Write block 2699 len 20480 data: 3 3 3 2 3
5910 202/300 Write block 2913 len 20480 data: 3 3 3 3 3
5911 203/300 Read block 4220 len 24576
5912 204/300 Write block 3973 len 24576 data: 2 2 3 2 2 2
5913 205/300 Read block 1763 len 20480
5914 206/300 Read block 1980 len 12288
5915 207/300 Read block 3317 len 24576
5916 208/300 Write block 4108 len 12288 data: 3 3 3
5917 209/300 Write block 2483 len 8192 data: 2 2
5918 210/300 Read block 3878 len 16384
5919 211/300 Read block 3399 len 4096
5920 212/300 Write block 798 len 12288 data: 3 3 3
5921 213/300 Write block 4994 len 12288 data: 4 5 4
5922 214/300 Write block 995 len 32768 data: 2 2 2 2 3 3 3 2
5923 215/300 Write block 4028 len 4096 data: 2
5924 216/300 Write block 2167 len 36864 data: 2 2 2 3 4 6 4 4 3
5925 217/300 Write block 1070 len 20480 data: 5 5 5 5 4
5926 218/300 Write block 1843 len 12288 data: 3 3 2
5927 219/300 Read block 2063 len 24576
5928 220/300 Read block 2807 len 16384
5929 221/300 Write block 787 len 4096 data: 2
5930 222/300 Write block 1903 len 8192 data: 5 5
5931 223/300 Write block 3076 len 32768 data: 2 2 2 3 2 3 3 3
5932 224/300 Read block 2031 len 4096
5933 225/300 Read block 602 len 36864
5934 226/300 Read block 987 len 36864
5935 227/300 Write block 4177 len 36864 data: 3 4 5 4 4 4 4 4 4
5936 228/300 Read block 1407 len 28672
5937 229/300 Read block 2704 len 8192
5938 230/300 Read block 1131 len 12288
5939 231/300 Write block 2280 len 4096 data: 2
5940 232/300 Read block 3621 len 40960
5941 233/300 Write block 1056 len 16384 data: 4 5 4 3
5942 234/300 Read block 1459 len 8192
5943 235/300 Write block 2198 len 12288 data: 3 3 3
5944 236/300 Read block 3474 len 32768
5945 237/300 Read block 4540 len 24576
5946 238/300 Read block 2607 len 4096
5947 239/300 Write block 188 len 40960 data: 5 4 3 3 3 3 4 3 3 2
5948 240/300 Read block 2705 len 20480
5949 241/300 Write block 3266 len 40960 data: 4 4 4 4 4 4 2 2 3 3
5950 242/300 Read block 2970 len 24576
5951 243/300 Read block 3457 len 24576
5952 244/300 Read block 2591 len 8192
5953 245/300 Write block 3698 len 28672 data: 3 3 3 3 3 3 2
5954 246/300 Read block 3139 len 12288
5955 247/300 Write block 1241 len 32768 data: 3 3 3 4 3 2 2 2
5956 248/300 Read block 4180 len 24576
5957 249/300 Flush
5958 250/300 Write block 1564 len 36864 data: 2 2 2 2 2 2 2 2 2
5959 251/300 Read block 2072 len 32768
5960 252/300 Flush
5961 253/300 Flush
5962 254/300 Write block 2459 len 36864 data: 6 6 5 5 6 6 6 5 5
5963 255/300 Read block 3573 len 36864
5964 256/300 Write block 407 len 32768 data: 3 3 3 2 2 2 2 2
5965 257/300 Flush
5966 258/300 Write block 38 len 40960 data: 3 2 2 2 3 3 3 4 4 4
5967 259/300 Read block 1118 len 4096
5968 260/300 Read block 20 len 24576
5969 261/300 Read block 3952 len 4096
5970 262/300 Write block 969 len 16384 data: 3 3 3 3
5971 263/300 Read block 1885 len 20480
5972 264/300 Read block 30 len 28672
5973 265/300 Flush
5974 266/300 Read block 105 len 12288
5975 267/300 Read block 4100 len 8192
5976 268/300 Read block 700 len 12288
5977 269/300 Read block 4779 len 36864
5978 270/300 Flush
5979 271/300 Read block 3533 len 8192
5980 272/300 Write block 2040 len 36864 data: 3 3 3 3 3 3 4 4 3
5981 273/300 Write block 1511 len 32768 data: 2 2 2 2 2 2 2 2
5982 274/300 Read block 2227 len 36864
5983 275/300 Read block 3892 len 16384
5984 276/300 Write block 2501 len 4096 data: 2
5985 277/300 Read block 2884 len 36864
5986 278/300 Read block 3751 len 36864
5987 279/300 Read block 2710 len 40960
5988 280/300 Read block 656 len 12288
5989 281/300 Read block 947 len 32768
5990 282/300 Read block 2352 len 40960
5991 283/300 Read block 1788 len 28672
5992 284/300 Read block 1499 len 36864
5993 285/300 Read block 56 len 4096
5994 286/300 Write block 1312 len 36864 data: 3 3 3 3 3 4 4 4 4
5995 287/300 Read block 349 len 40960
5996 288/300 Read block 3598 len 20480
5997 289/300 Read block 972 len 28672
5998 290/300 Read block 1943 len 40960
5999 291/300 Read block 2239 len 8192
6000 292/300 Write block 3758 len 36864 data: 2 2 2 2 2 2 3 3 3
6001 293/300 Write block 538 len 8192 data: 2 2
6002 294/300 Write block 4923 len 8192 data: 4 4
6003 295/300 Write block 3877 len 4096 data: 3
6004 296/300 Write block 3841 len 24576 data: 3 3 3 4 3 3
6005 297/300 Read block 112 len 40960
6006 298/300 Read block 3401 len 32768
6007 299/300 Write block 2169 len 4096 data: 3
6008 300/300 Read block 173 len 4096
6009 301/300 Read block 3260 len 36864
6010 Main task has work to do, go find it
6011 got dsc Start(0)
6012 start 0
6013 Jan 21 15:48:26.794 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/0, method: POST, req_id: 2f6d3b85-e14e-4c0d-b350-0d34a9ad089b, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
6014 Replay: started 0, returned:Ok(())
6015 ----------------------------------------------------------------
6016 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6017 States: Offline Active Active
6018 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6019 2509 Acked Write 8 Sent Done Done false
6020 2510 Acked Read 7 Sent Done Done false
6021 2511 Acked Flush 0 Sent Done Done false
6022 2512 Acked Read 1 Sent Done Done false
6023 2513 Acked Read 2 Sent Done Done false
6024 2514 Acked Read 5 Sent Done Done false
6025 2515 Acked Read 5 Sent Done Done false
6026 2516 Acked Write 5 Sent Done Done false
6027 2517 Acked Read 6 Sent Done Done false
6028 2518 Acked Read 8 Sent Done Done false
6029 2519 Acked Read 6 Sent Done Done false
6030 2520 Acked Write 2 Sent Done Done false
6031 2521 Acked Read 4 Sent Done Done false
6032 2522 Acked Read 8 Sent Done Done false
6033 2523 Acked Write 2 Sent Done Done false
6034 2524 Acked Flush 0 Sent Done Done false
6035 2525 Acked Read 5 Sent Done Done false
6036 2526 Acked Write 7 Sent Done Done false
6037 2527 Acked Read 1 Sent Done Done false
6038 2528 Acked Read 1 Sent Done Done false
6039 2529 Acked Read 6 Sent Done Done false
6040 2530 Acked Read 9 Sent Done Done false
6041 2531 Acked Write 8 Sent Done Done false
6042 2532 Acked Read 8 Sent Done Done false
6043 2533 Acked Read 5 Sent Done Done false
6044 2534 Acked Write 5 Sent Done Done false
6045 2535 Acked Write 10 Sent Done Done false
6046 2536 Acked Write 10 Sent Done Done false
6047 2537 Acked Read 4 Sent Done Done false
6048 2538 Acked Read 5 Sent Done Done false
6049 2539 Acked Read 8 Sent Done Done false
6050 2540 Acked Read 10 Sent Done Done false
6051 2541 Acked Read 1 Sent Done Done false
6052 2542 Acked Write 3 Sent Done Done false
6053 2543 Acked Flush 0 Sent Done Done false
6054 2544 Acked Read 1 Sent Done Done false
6055 2545 Acked Write 7 Sent Done Done false
6056 2546 Acked Write 9 Sent Done Done false
6057 2547 Acked Flush 0 Sent Done Done false
6058 2548 Acked Read 7 Sent Done Done false
6059 2549 Acked Read 1 Sent Done Done false
6060 2550 Acked Read 9 Sent Done Done false
6061 2551 Acked Write 10 Sent Done Done false
6062 2552 Acked Read 9 Sent Done Done false
6063 2553 Acked Write 2 Sent Done Done false
6064 2554 Acked Write 5 Sent Done Done false
6065 2555 Acked Read 4 Sent Done Done false
6066 2556 Acked Read 7 Sent Done Done false
6067 2557 Acked Read 5 Sent Done Done false
6068 2558 Acked Write 7 Sent Done Done false
6069 2559 Acked Read 2 Sent Done Done false
6070 2560 Acked Read 10 Sent Done Done false
6071 2561 Acked Read 2 Sent Done Done false
6072 2562 Acked Write 6 Sent Done Done false
6073 2563 Acked Write 10 Sent Done Done false
6074 2564 Acked Write 1 Sent Done Done false
6075 2565 Acked Read 7 Sent Done Done false
6076 2566 Acked Write 5 Sent Done Done false
6077 2567 Acked Read 1 Sent Done Done false
6078 2568 Acked Write 4 Sent Done Done false
6079 2569 Acked Read 9 Sent Done Done false
6080 2570 Acked Read 4 Sent Done Done false
6081 2571 Acked Flush 0 Sent Done Done false
6082 2572 Acked Flush 0 Sent Done Done false
6083 2573 Acked Read 6 Sent Done Done false
6084 2574 Acked Write 8 Sent Done Done false
6085 2575 Acked Read 1 Sent Done Done false
6086 2576 Acked Read 1 Sent Done Done false
6087 2577 Acked Read 6 Sent Done Done false
6088 2578 Acked Flush 0 Sent Done Done false
6089 2579 Acked Write 7 Sent Done Done false
6090 2580 Acked Write 7 Sent Done Done false
6091 2581 Acked Read 9 Sent Done Done false
6092 2582 Acked Read 6 Sent Done Done false
6093 2583 Acked Write 2 Sent Done Done false
6094 2584 Acked Write 4 Sent Done Done false
6095 2585 Acked Read 4 Sent Done Done false
6096 2586 Acked Write 5 Sent Done Done false
6097 2587 Acked Read 9 Sent Done Done false
6098 2588 Acked Flush 0 Sent Done Done false
6099 2589 Acked Write 3 Sent Done Done false
6100 2590 Acked Write 8 Sent Done Done false
6101 2591 Acked Read 4 Sent Done Done false
6102 2592 Acked Write 4 Sent Done Done false
6103 2593 Acked Read 6 Sent Done Done false
6104 2594 Acked Write 2 Sent Done Done false
6105 2595 Acked Read 3 Sent Done Done false
6106 2596 Acked Flush 0 Sent Done Done false
6107 2597 Acked Flush 0 Sent Done Done false
6108 2598 Acked Read 6 Sent Done Done false
6109 2599 Acked Write 1 Sent Done Done false
6110 2600 Acked Read 4 Sent Done Done false
6111 2601 Acked Write 4 Sent Done Done false
6112 2602 Acked Read 4 Sent Done Done false
6113 2603 Acked Write 4 Sent Done Done false
6114 2604 Acked Read 9 Sent Done Done false
6115 2605 Acked Write 6 Sent Done Done false
6116 2606 Acked Read 8 Sent Done Done false
6117 2607 Acked Write 5 Sent Done Done false
6118 2608 Acked Write 10 Sent Done Done false
6119 2609 Acked Write 9 Sent Done Done false
6120 2610 Acked Read 1 Sent Done Done false
6121 2611 Acked Read 7 Sent Done Done false
6122 2612 Acked Write 2 Sent Done Done false
6123 2613 Acked Write 1 Sent Done Done false
6124 2614 Acked Write 6 Sent Done Done false
6125 2615 Acked Read 8 Sent Done Done false
6126 2616 Acked Read 6 Sent Done Done false
6127 2617 Acked Flush 0 Sent Done Done false
6128 2618 Acked Flush 0 Sent Done Done false
6129 2619 Acked Read 9 Sent Done Done false
6130 2620 Acked Write 10 Sent Done Done false
6131 2621 Acked Write 5 Sent Done Done false
6132 2622 Acked Write 8 Sent Done Done false
6133 2623 Acked Flush 0 Sent Done Done false
6134 2624 Acked Read 1 Sent Done Done false
6135 2625 Acked Flush 0 Sent Done Done false
6136 2626 Acked Write 4 Sent Done Done false
6137 2627 Acked Write 9 Sent Done Done false
6138 2628 Acked Flush 0 Sent Done Done false
6139 2629 Acked Read 4 Sent Done Done false
6140 2630 Acked Write 1 Sent Done Done false
6141 2631 Acked Read 3 Sent Done Done false
6142 2632 Acked Write 7 Sent Done Done false
6143 2633 Acked Read 2 Sent Done Done false
6144 2634 Acked Flush 0 Sent Done Done false
6145 2635 Acked Read 7 Sent Done Done false
6146 2636 Acked Read 5 Sent Done Done false
6147 2637 Acked Write 4 Sent Done Done false
6148 2638 Acked Flush 0 Sent Done Done false
6149 2639 Acked Read 1 Sent Done Done false
6150 2640 Acked Read 6 Sent Done Done false
6151 2641 Acked Flush 0 Sent Done Done false
6152 2642 Acked Write 2 Sent Done Done false
6153 2643 Acked Read 4 Sent Done Done false
6154 2644 Acked Write 9 Sent Done Done false
6155 2645 Acked Read 5 Sent Done Done false
6156 2646 Acked Read 2 Sent Done Done false
6157 2647 Acked Read 9 Sent Done Done false
6158 2648 Acked Read 10 Sent Done Done false
6159 2649 Acked Flush 0 Sent Done Done false
6160 2650 Acked Read 8 Sent Done Done false
6161 2651 Acked Read 10 Sent Done Done false
6162 2652 Acked Write 5 Sent Done Done false
6163 2653 Acked Write 7 Sent Done Done false
6164 2654 Acked Write 9 Sent Done Done false
6165 2655 Acked Write 10 Sent Done Done false
6166 2656 Acked Write 9 Sent Done Done false
6167 2657 Acked Write 1 Sent Done Done false
6168 2658 Acked Read 1 Sent Done Done false
6169 2659 Acked Write 2 Sent Done Done false
6170 2660 Acked Read 3 Sent Done Done false
6171 2661 Acked Write 2 Sent Done Done false
6172 2662 Acked Read 3 Sent Done Done false
6173 2663 Acked Write 8 Sent Done Done false
6174 2664 Acked Read 9 Sent Done Done false
6175 2665 Acked Read 3 Sent Done Done false
6176 2666 Acked Write 7 Sent Done Done false
6177 2667 Acked Read 3 Sent Done Done false
6178 2668 Acked Flush 0 Sent Done Done false
6179 2669 Acked Read 4 Sent Done Done false
6180 2670 Acked Flush 0 Sent Done Done false
6181 2671 Acked Read 6 Sent Done Done false
6182 2672 Acked Read 5 Sent Done Done false
6183 2673 Acked Read 6 Sent Done Done false
6184 2674 Acked Read 3 Sent Done Done false
6185 2675 Acked Write 7 Sent Done Done false
6186 2676 Acked Read 3 Sent Done Done false
6187 2677 Acked Read 3 Sent Done Done false
6188 2678 Acked Read 8 Sent Done Done false
6189 2679 Acked Read 8 Sent Done Done false
6190 2680 Acked Write 1 Sent Done Done false
6191 2681 Acked Read 5 Sent Done Done false
6192 2682 Acked Flush 0 Sent Done Done false
6193 2683 Acked Read 1 Sent Done Done false
6194 2684 Acked Write 9 Sent Done Done false
6195 2685 Acked Read 8 Sent Done Done false
6196 2686 Acked Write 8 Sent Done Done false
6197 2687 Acked Write 9 Sent Done Done false
6198 2688 Acked Read 4 Sent Done Done false
6199 2689 Acked Read 3 Sent Done Done false
6200 2690 Acked Write 9 Sent Done Done false
6201 2691 Acked Write 3 Sent Done Done false
6202 2692 Acked Write 1 Sent Done Done false
6203 2693 Acked Write 2 Sent Done Done false
6204 2694 Acked Write 5 Sent Done Done false
6205 2695 Acked Write 5 Sent Done Done false
6206 2696 Acked Read 2 Sent Done Done false
6207 2697 Acked Read 10 Sent Done Done false
6208 2698 Acked Flush 0 Sent Done Done false
6209 2699 Acked Write 9 Sent Done Done false
6210 2700 Acked Write 6 Sent Done Done false
6211 2701 Acked Read 1 Sent Done Done false
6212 2702 Acked Read 1 Sent Done Done false
6213 2703 Acked Read 7 Sent Done Done false
6214 2704 Acked Flush 0 Sent Done Done false
6215 2705 Acked Read 6 Sent Done Done false
6216 2706 Acked Read 3 Sent Done Done false
6217 2707 Acked Read 7 Sent Done Done false
6218 2708 Acked Read 1 Sent Done Done false
6219 2709 Acked Write 5 Sent Done Done false
6220 2710 Acked Write 5 Sent Done Done false
6221 2711 Acked Read 6 Sent Done Done false
6222 2712 Acked Write 6 Sent Done Done false
6223 2713 Acked Read 5 Sent Done Done false
6224 2714 Acked Read 3 Sent Done Done false
6225 2715 Acked Read 6 Sent Done Done false
6226 2716 Acked Write 3 Sent Done Done false
6227 2717 Acked Write 2 Sent Done Done false
6228 2718 Acked Read 4 Sent Done Done false
6229 2719 Acked Read 1 Sent Done Done false
6230 2720 Acked Write 3 Sent Done Done false
6231 2721 Acked Write 3 Sent Done Done false
6232 2722 Acked Write 8 Sent Done Done false
6233 2723 Acked Write 1 Sent Done Done false
6234 2724 Acked Write 9 Sent Done Done false
6235 2725 Acked Write 5 Sent Done Done false
6236 2726 Acked Write 3 Sent Done Done false
6237 2727 Acked Read 6 Sent Done Done false
6238 2728 Acked Read 4 Sent Done Done false
6239 2729 Acked Write 1 Sent Done Done false
6240 2730 Acked Write 2 Sent Done Done false
6241 2731 Acked Write 8 Sent Done Done false
6242 2732 Acked Read 1 Sent Done Done false
6243 2733 Acked Read 9 Sent Done Done false
6244 2734 Acked Read 9 Sent Done Done false
6245 2735 Acked Write 9 Sent Done Done false
6246 2736 Acked Read 7 Sent Done Done false
6247 2737 Acked Read 2 Sent Done Done false
6248 2738 Acked Read 3 Sent Done Done false
6249 2739 Acked Write 1 Sent Done Done false
6250 2740 Acked Read 10 Sent Done Done false
6251 2741 Acked Write 4 Sent Done Done false
6252 2742 Acked Read 2 Sent Done Done false
6253 2743 Acked Write 3 Sent Done Done false
6254 2744 Acked Read 8 Sent Done Done false
6255 2745 Acked Read 6 Sent Done Done false
6256 2746 Acked Read 1 Sent Done Done false
6257 2747 Acked Write 10 Sent Done Done false
6258 2748 Acked Read 5 Sent Done Done false
6259 2749 Acked Write 10 Sent Done Done false
6260 2750 Acked Read 6 Sent Done Done false
6261 2751 Acked Read 6 Sent Done Done false
6262 2752 Acked Read 2 Sent Done Done false
6263 2753 Acked Write 7 Sent Done Done false
6264 2754 Acked Read 3 Sent Done Done false
6265 2755 Acked Write 8 Sent Done Done false
6266 2756 Acked Read 6 Sent Done Done false
6267 2757 Acked Flush 0 Sent Done Done false
6268 2758 Acked Write 9 Sent Done Done false
6269 2759 Acked Read 8 Sent Done Done false
6270 2760 Acked Flush 0 Sent Done Done false
6271 2761 Acked Flush 0 Sent Done Done false
6272 2762 Acked Write 9 Sent Done Done false
6273 2763 Acked Read 9 Sent Done Done false
6274 2764 Acked Write 8 Sent Done Done false
6275 2765 Acked Flush 0 Sent Done Done false
6276 2766 Acked Write 10 Sent Done Done false
6277 2767 Acked Read 1 Sent Done Done false
6278 2768 Acked Read 6 Sent Done Done false
6279 2769 Acked Read 1 Sent Done Done false
6280 2770 Acked Write 4 Sent Done Done false
6281 2771 Acked Read 5 Sent Done Done false
6282 2772 Acked Read 7 Sent Done Done false
6283 2773 Acked Flush 0 Sent Done Done false
6284 2774 Acked Read 3 Sent Done Done false
6285 2775 Acked Read 2 Sent Done Done false
6286 2776 Acked Read 3 Sent Done Done false
6287 2777 Acked Read 9 Sent Done Done false
6288 2778 Acked Flush 0 Sent Done Done false
6289 2779 Acked Read 2 Sent Done Done false
6290 2780 Acked Write 9 Sent Done Done false
6291 2781 Acked Write 8 Sent Done Done false
6292 2782 Acked Read 9 Sent Done Done false
6293 2783 Acked Read 4 Sent Done Done false
6294 2784 Acked Write 1 Sent Done Done false
6295 2785 Acked Read 9 Sent Done Done false
6296 2786 Acked Read 9 Sent Done Done false
6297 2787 Acked Read 10 Sent Done Done false
6298 2788 Acked Read 3 Sent Done Done false
6299 2789 Acked Read 8 Sent Done Done false
6300 2790 Acked Read 10 Sent Done Done false
6301 2791 Acked Read 7 Sent Done Done false
6302 2792 Acked Read 9 Sent Done Done false
6303 2793 Acked Read 1 Sent Done Done false
6304 2794 Acked Write 9 Sent Done Done false
6305 2795 Acked Read 10 Sent Done Done false
6306 2796 Acked Read 5 Sent Done Done false
6307 2797 Acked Read 7 Sent Done Done false
6308 2798 Acked Read 10 Sent Done Done false
6309 2799 Acked Read 2 Sent Done Done false
6310 2800 Acked Write 9 Sent Done Done false
6311 2801 Acked Write 2 Sent Done Done false
6312 2802 Acked Write 2 Sent Done Done false
6313 2803 Acked Write 1 Sent Done Done false
6314 2804 Acked Write 6 Sent Done Done false
6315 2805 Acked Read 10 Sent Done Done false
6316 2806 Acked Read 8 Sent Done Done false
6317 2807 Acked Write 1 Sent Done Done false
6318 2808 Acked Read 1 Sent Done Done false
6319 2809 Acked Read 9 Sent Done Done false
6320 STATES DS:0 DS:1 DS:2 TOTAL
6321 Sent 301 0 0 301
6322 Done 0 301 301 602
6323 Skipped 0 0 0 0
6324 Error 0 0 0 0
6325 Last Flush: 2508 2778 2778
6326 Downstairs last five completed: 2508 2507 2506 2505 2504
6327 Upstairs last five completed: 2809 2808 2807 2806 2805
6328 CLIENT: Up:0 ds:301 act:2
6329 [8810] got start action
6330 [8810] I am going to restart
6331 Starting downstairs at port 8810
6332 Make output file at "/tmp/dsc/downstairs-8810.txt"
6333 [8810][0] reports Starting
6334 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1206)
6335 [8810][0] reports Running
6336 ----------------------------------------------------------------
6337 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
6338 States: Offline Active Active
6339 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6340 2509 Acked Write 8 Sent Done Done false
6341 2510 Acked Read 7 Sent Done Done false
6342 2511 Acked Flush 0 Sent Done Done false
6343 2512 Acked Read 1 Sent Done Done false
6344 2513 Acked Read 2 Sent Done Done false
6345 2514 Acked Read 5 Sent Done Done false
6346 2515 Acked Read 5 Sent Done Done false
6347 2516 Acked Write 5 Sent Done Done false
6348 2517 Acked Read 6 Sent Done Done false
6349 2518 Acked Read 8 Sent Done Done false
6350 2519 Acked Read 6 Sent Done Done false
6351 2520 Acked Write 2 Sent Done Done false
6352 2521 Acked Read 4 Sent Done Done false
6353 2522 Acked Read 8 Sent Done Done false
6354 2523 Acked Write 2 Sent Done Done false
6355 2524 Acked Flush 0 Sent Done Done false
6356 2525 Acked Read 5 Sent Done Done false
6357 2526 Acked Write 7 Sent Done Done false
6358 2527 Acked Read 1 Sent Done Done false
6359 2528 Acked Read 1 Sent Done Done false
6360 2529 Acked Read 6 Sent Done Done false
6361 2530 Acked Read 9 Sent Done Done false
6362 2531 Acked Write 8 Sent Done Done false
6363 2532 Acked Read 8 Sent Done Done false
6364 2533 Acked Read 5 Sent Done Done false
6365 2534 Acked Write 5 Sent Done Done false
6366 2535 Acked Write 10 Sent Done Done false
6367 2536 Acked Write 10 Sent Done Done false
6368 2537 Acked Read 4 Sent Done Done false
6369 2538 Acked Read 5 Sent Done Done false
6370 2539 Acked Read 8 Sent Done Done false
6371 2540 Acked Read 10 Sent Done Done false
6372 2541 Acked Read 1 Sent Done Done false
6373 2542 Acked Write 3 Sent Done Done false
6374 2543 Acked Flush 0 Sent Done Done false
6375 2544 Acked Read 1 Sent Done Done false
6376 2545 Acked Write 7 Sent Done Done false
6377 2546 Acked Write 9 Sent Done Done false
6378 2547 Acked Flush 0 Sent Done Done false
6379 2548 Acked Read 7 Sent Done Done false
6380 2549 Acked Read 1 Sent Done Done false
6381 2550 Acked Read 9 Sent Done Done false
6382 2551 Acked Write 10 Sent Done Done false
6383 2552 Acked Read 9 Sent Done Done false
6384 2553 Acked Write 2 Sent Done Done false
6385 2554 Acked Write 5 Sent Done Done false
6386 2555 Acked Read 4 Sent Done Done false
6387 2556 Acked Read 7 Sent Done Done false
6388 2557 Acked Read 5 Sent Done Done false
6389 2558 Acked Write 7 Sent Done Done false
6390 2559 Acked Read 2 Sent Done Done false
6391 2560 Acked Read 10 Sent Done Done false
6392 2561 Acked Read 2 Sent Done Done false
6393 2562 Acked Write 6 Sent Done Done false
6394 2563 Acked Write 10 Sent Done Done false
6395 2564 Acked Write 1 Sent Done Done false
6396 2565 Acked Read 7 Sent Done Done false
6397 2566 Acked Write 5 Sent Done Done false
6398 2567 Acked Read 1 Sent Done Done false
6399 2568 Acked Write 4 Sent Done Done false
6400 2569 Acked Read 9 Sent Done Done false
6401 2570 Acked Read 4 Sent Done Done false
6402 2571 Acked Flush 0 Sent Done Done false
6403 2572 Acked Flush 0 Sent Done Done false
6404 2573 Acked Read 6 Sent Done Done false
6405 2574 Acked Write 8 Sent Done Done false
6406 2575 Acked Read 1 Sent Done Done false
6407 2576 Acked Read 1 Sent Done Done false
6408 2577 Acked Read 6 Sent Done Done false
6409 2578 Acked Flush 0 Sent Done Done false
6410 2579 Acked Write 7 Sent Done Done false
6411 2580 Acked Write 7 Sent Done Done false
6412 2581 Acked Read 9 Sent Done Done false
6413 2582 Acked Read 6 Sent Done Done false
6414 2583 Acked Write 2 Sent Done Done false
6415 2584 Acked Write 4 Sent Done Done false
6416 2585 Acked Read 4 Sent Done Done false
6417 2586 Acked Write 5 Sent Done Done false
6418 2587 Acked Read 9 Sent Done Done false
6419 2588 Acked Flush 0 Sent Done Done false
6420 2589 Acked Write 3 Sent Done Done false
6421 2590 Acked Write 8 Sent Done Done false
6422 2591 Acked Read 4 Sent Done Done false
6423 2592 Acked Write 4 Sent Done Done false
6424 2593 Acked Read 6 Sent Done Done false
6425 2594 Acked Write 2 Sent Done Done false
6426 2595 Acked Read 3 Sent Done Done false
6427 2596 Acked Flush 0 Sent Done Done false
6428 2597 Acked Flush 0 Sent Done Done false
6429 2598 Acked Read 6 Sent Done Done false
6430 2599 Acked Write 1 Sent Done Done false
6431 2600 Acked Read 4 Sent Done Done false
6432 2601 Acked Write 4 Sent Done Done false
6433 2602 Acked Read 4 Sent Done Done false
6434 2603 Acked Write 4 Sent Done Done false
6435 2604 Acked Read 9 Sent Done Done false
6436 2605 Acked Write 6 Sent Done Done false
6437 2606 Acked Read 8 Sent Done Done false
6438 2607 Acked Write 5 Sent Done Done false
6439 2608 Acked Write 10 Sent Done Done false
6440 2609 Acked Write 9 Sent Done Done false
6441 2610 Acked Read 1 Sent Done Done false
6442 2611 Acked Read 7 Sent Done Done false
6443 2612 Acked Write 2 Sent Done Done false
6444 2613 Acked Write 1 Sent Done Done false
6445 2614 Acked Write 6 Sent Done Done false
6446 2615 Acked Read 8 Sent Done Done false
6447 2616 Acked Read 6 Sent Done Done false
6448 2617 Acked Flush 0 Sent Done Done false
6449 2618 Acked Flush 0 Sent Done Done false
6450 2619 Acked Read 9 Sent Done Done false
6451 2620 Acked Write 10 Sent Done Done false
6452 2621 Acked Write 5 Sent Done Done false
6453 2622 Acked Write 8 Sent Done Done false
6454 2623 Acked Flush 0 Sent Done Done false
6455 2624 Acked Read 1 Sent Done Done false
6456 2625 Acked Flush 0 Sent Done Done false
6457 2626 Acked Write 4 Sent Done Done false
6458 2627 Acked Write 9 Sent Done Done false
6459 2628 Acked Flush 0 Sent Done Done false
6460 2629 Acked Read 4 Sent Done Done false
6461 2630 Acked Write 1 Sent Done Done false
6462 2631 Acked Read 3 Sent Done Done false
6463 2632 Acked Write 7 Sent Done Done false
6464 2633 Acked Read 2 Sent Done Done false
6465 2634 Acked Flush 0 Sent Done Done false
6466 2635 Acked Read 7 Sent Done Done false
6467 2636 Acked Read 5 Sent Done Done false
6468 2637 Acked Write 4 Sent Done Done false
6469 2638 Acked Flush 0 Sent Done Done false
6470 2639 Acked Read 1 Sent Done Done false
6471 2640 Acked Read 6 Sent Done Done false
6472 2641 Acked Flush 0 Sent Done Done false
6473 2642 Acked Write 2 Sent Done Done false
6474 2643 Acked Read 4 Sent Done Done false
6475 2644 Acked Write 9 Sent Done Done false
6476 2645 Acked Read 5 Sent Done Done false
6477 2646 Acked Read 2 Sent Done Done false
6478 2647 Acked Read 9 Sent Done Done false
6479 2648 Acked Read 10 Sent Done Done false
6480 2649 Acked Flush 0 Sent Done Done false
6481 2650 Acked Read 8 Sent Done Done false
6482 2651 Acked Read 10 Sent Done Done false
6483 2652 Acked Write 5 Sent Done Done false
6484 2653 Acked Write 7 Sent Done Done false
6485 2654 Acked Write 9 Sent Done Done false
6486 2655 Acked Write 10 Sent Done Done false
6487 2656 Acked Write 9 Sent Done Done false
6488 2657 Acked Write 1 Sent Done Done false
6489 2658 Acked Read 1 Sent Done Done false
6490 2659 Acked Write 2 Sent Done Done false
6491 2660 Acked Read 3 Sent Done Done false
6492 2661 Acked Write 2 Sent Done Done false
6493 2662 Acked Read 3 Sent Done Done false
6494 2663 Acked Write 8 Sent Done Done false
6495 2664 Acked Read 9 Sent Done Done false
6496 2665 Acked Read 3 Sent Done Done false
6497 2666 Acked Write 7 Sent Done Done false
6498 2667 Acked Read 3 Sent Done Done false
6499 2668 Acked Flush 0 Sent Done Done false
6500 2669 Acked Read 4 Sent Done Done false
6501 2670 Acked Flush 0 Sent Done Done false
6502 2671 Acked Read 6 Sent Done Done false
6503 2672 Acked Read 5 Sent Done Done false
6504 2673 Acked Read 6 Sent Done Done false
6505 2674 Acked Read 3 Sent Done Done false
6506 2675 Acked Write 7 Sent Done Done false
6507 2676 Acked Read 3 Sent Done Done false
6508 2677 Acked Read 3 Sent Done Done false
6509 2678 Acked Read 8 Sent Done Done false
6510 2679 Acked Read 8 Sent Done Done false
6511 2680 Acked Write 1 Sent Done Done false
6512 2681 Acked Read 5 Sent Done Done false
6513 2682 Acked Flush 0 Sent Done Done false
6514 2683 Acked Read 1 Sent Done Done false
6515 2684 Acked Write 9 Sent Done Done false
6516 2685 Acked Read 8 Sent Done Done false
6517 2686 Acked Write 8 Sent Done Done false
6518 2687 Acked Write 9 Sent Done Done false
6519 2688 Acked Read 4 Sent Done Done false
6520 2689 Acked Read 3 Sent Done Done false
6521 2690 Acked Write 9 Sent Done Done false
6522 2691 Acked Write 3 Sent Done Done false
6523 2692 Acked Write 1 Sent Done Done false
6524 2693 Acked Write 2 Sent Done Done false
6525 2694 Acked Write 5 Sent Done Done false
6526 2695 Acked Write 5 Sent Done Done false
6527 2696 Acked Read 2 Sent Done Done false
6528 2697 Acked Read 10 Sent Done Done false
6529 2698 Acked Flush 0 Sent Done Done false
6530 2699 Acked Write 9 Sent Done Done false
6531 2700 Acked Write 6 Sent Done Done false
6532 2701 Acked Read 1 Sent Done Done false
6533 2702 Acked Read 1 Sent Done Done false
6534 2703 Acked Read 7 Sent Done Done false
6535 2704 Acked Flush 0 Sent Done Done false
6536 2705 Acked Read 6 Sent Done Done false
6537 2706 Acked Read 3 Sent Done Done false
6538 2707 Acked Read 7 Sent Done Done false
6539 2708 Acked Read 1 Sent Done Done false
6540 2709 Acked Write 5 Sent Done Done false
6541 2710 Acked Write 5 Sent Done Done false
6542 2711 Acked Read 6 Sent Done Done false
6543 2712 Acked Write 6 Sent Done Done false
6544 2713 Acked Read 5 Sent Done Done false
6545 2714 Acked Read 3 Sent Done Done false
6546 2715 Acked Read 6 Sent Done Done false
6547 2716 Acked Write 3 Sent Done Done false
6548 2717 Acked Write 2 Sent Done Done false
6549 2718 Acked Read 4 Sent Done Done false
6550 2719 Acked Read 1 Sent Done Done false
6551 2720 Acked Write 3 Sent Done Done false
6552 2721 Acked Write 3 Sent Done Done false
6553 2722 Acked Write 8 Sent Done Done false
6554 2723 Acked Write 1 Sent Done Done false
6555 2724 Acked Write 9 Sent Done Done false
6556 2725 Acked Write 5 Sent Done Done false
6557 2726 Acked Write 3 Sent Done Done false
6558 2727 Acked Read 6 Sent Done Done false
6559 2728 Acked Read 4 Sent Done Done false
6560 2729 Acked Write 1 Sent Done Done false
6561 2730 Acked Write 2 Sent Done Done false
6562 2731 Acked Write 8 Sent Done Done false
6563 2732 Acked Read 1 Sent Done Done false
6564 2733 Acked Read 9 Sent Done Done false
6565 2734 Acked Read 9 Sent Done Done false
6566 2735 Acked Write 9 Sent Done Done false
6567 2736 Acked Read 7 Sent Done Done false
6568 2737 Acked Read 2 Sent Done Done false
6569 2738 Acked Read 3 Sent Done Done false
6570 2739 Acked Write 1 Sent Done Done false
6571 2740 Acked Read 10 Sent Done Done false
6572 2741 Acked Write 4 Sent Done Done false
6573 2742 Acked Read 2 Sent Done Done false
6574 2743 Acked Write 3 Sent Done Done false
6575 2744 Acked Read 8 Sent Done Done false
6576 2745 Acked Read 6 Sent Done Done false
6577 2746 Acked Read 1 Sent Done Done false
6578 2747 Acked Write 10 Sent Done Done false
6579 2748 Acked Read 5 Sent Done Done false
6580 2749 Acked Write 10 Sent Done Done false
6581 2750 Acked Read 6 Sent Done Done false
6582 2751 Acked Read 6 Sent Done Done false
6583 2752 Acked Read 2 Sent Done Done false
6584 2753 Acked Write 7 Sent Done Done false
6585 2754 Acked Read 3 Sent Done Done false
6586 2755 Acked Write 8 Sent Done Done false
6587 2756 Acked Read 6 Sent Done Done false
6588 2757 Acked Flush 0 Sent Done Done false
6589 2758 Acked Write 9 Sent Done Done false
6590 2759 Acked Read 8 Sent Done Done false
6591 2760 Acked Flush 0 Sent Done Done false
6592 2761 Acked Flush 0 Sent Done Done false
6593 2762 Acked Write 9 Sent Done Done false
6594 2763 Acked Read 9 Sent Done Done false
6595 2764 Acked Write 8 Sent Done Done false
6596 2765 Acked Flush 0 Sent Done Done false
6597 2766 Acked Write 10 Sent Done Done false
6598 2767 Acked Read 1 Sent Done Done false
6599 2768 Acked Read 6 Sent Done Done false
6600 2769 Acked Read 1 Sent Done Done false
6601 2770 Acked Write 4 Sent Done Done false
6602 2771 Acked Read 5 Sent Done Done false
6603 2772 Acked Read 7 Sent Done Done false
6604 2773 Acked Flush 0 Sent Done Done false
6605 2774 Acked Read 3 Sent Done Done false
6606 2775 Acked Read 2 Sent Done Done false
6607 2776 Acked Read 3 Sent Done Done false
6608 2777 Acked Read 9 Sent Done Done false
6609 2778 Acked Flush 0 Sent Done Done false
6610 2779 Acked Read 2 Sent Done Done false
6611 2780 Acked Write 9 Sent Done Done false
6612 2781 Acked Write 8 Sent Done Done false
6613 2782 Acked Read 9 Sent Done Done false
6614 2783 Acked Read 4 Sent Done Done false
6615 2784 Acked Write 1 Sent Done Done false
6616 2785 Acked Read 9 Sent Done Done false
6617 2786 Acked Read 9 Sent Done Done false
6618 2787 Acked Read 10 Sent Done Done false
6619 2788 Acked Read 3 Sent Done Done false
6620 2789 Acked Read 8 Sent Done Done false
6621 2790 Acked Read 10 Sent Done Done false
6622 2791 Acked Read 7 Sent Done Done false
6623 2792 Acked Read 9 Sent Done Done false
6624 2793 Acked Read 1 Sent Done Done false
6625 2794 Acked Write 9 Sent Done Done false
6626 2795 Acked Read 10 Sent Done Done false
6627 2796 Acked Read 5 Sent Done Done false
6628 2797 Acked Read 7 Sent Done Done false
6629 2798 Acked Read 10 Sent Done Done false
6630 2799 Acked Read 2 Sent Done Done false
6631 2800 Acked Write 9 Sent Done Done false
6632 2801 Acked Write 2 Sent Done Done false
6633 2802 Acked Write 2 Sent Done Done false
6634 2803 Acked Write 1 Sent Done Done false
6635 2804 Acked Write 6 Sent Done Done false
6636 2805 Acked Read 10 Sent Done Done false
6637 2806 Acked Read 8 Sent Done Done false
6638 2807 Acked Write 1 Sent Done Done false
6639 2808 Acked Read 1 Sent Done Done false
6640 2809 Acked Read 9 Sent Done Done false
6641 2810 Acked Flush 0 Sent Done Done false
6642 STATES DS:0 DS:1 DS:2 TOTAL
6643 Sent 302 0 0 302
6644 Done 0 302 302 604
6645 Skipped 0 0 0 0
6646 Error 0 0 0 0
6647 Last Flush: 2508 2810 2810
6648 Downstairs last five completed: 2508 2507 2506 2505 2504
6649 Upstairs last five completed: 2810 2809 2808 2807 2806
6650 CLIENT: Up:0 ds:302 act:2
66512025-01-21T15:48:32.553ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66522025-01-21T15:48:32.554ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51774) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66532025-01-21T15:48:32.555ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66542025-01-21T15:48:32.555ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66552025-01-21T15:48:32.555ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66562025-01-21T15:48:32.556ZINFOcrucible: send last flush ID to this DS: 2508 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66572025-01-21T15:48:32.556ZINFOcrucible: Replied this last flush ID: 2508 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66582025-01-21T15:48:32.556ZINFOcrucible: [0] client re-new 302 jobs since flush 2508 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66592025-01-21T15:48:32.556ZINFOcrucible: [0] Replayed 302 jobs since flush: 2508 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
6660 ----------------------------------------------------------------
6661 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6662 Downstairs last five completed: 2810 2809 2808 2807 2806
6663 Upstairs last five completed: 2810 2809 2808 2807 2806
6664 CLIENT: Up:0 ds:0 act:3
6665 Replay: All jobs finished
6666 Main task has work to do, go find it
6667 got dsc Stop(1)
6668 stop 1
6669 Jan 21 15:48:34.800 INFO request completed[8820] Got stop action so:false kr:false
6670 , latency_us: 352, response_code: 204, uri: /stop/cid/1, method: POST, req_id: c2df9706-0d0a-43eb-afeb-7278b44d4ad1, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
66712025-01-21T15:48:34.801ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66722025-01-21T15:48:34.801ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
66732025-01-21T15:48:34.801ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
6674 Jan 21 15:48:34.801 INFO request completed, latency_us: 150, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7e480192-a5ba-4ac7-a5e7-37b52e09d770, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
66752025-01-21T15:48:34.801ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
6676 [8820] Exited with: ExitStatus(unix_wait_status(9))
6677 [8820][1] reports Exit
6678 Jan 21 15:48:38.803 INFO request completed, latency_us: 201, response_code: 200, uri: /state/cid/1, method: GET, req_id: 791f483e-3f58-40a6-95bb-7d149a04d46f, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
6679 001/300 Write block 3663 len 16384 data: 2 3 3 4
6680 002/300 Flush
6681 003/300 Read block 1800 len 32768
6682 004/300 Read block 2053 len 8192
6683 005/300 Read block 4104 len 12288
6684 006/300 Read block 4295 len 36864
6685 007/300 Read block 2086 len 12288
6686 008/300 Read block 1614 len 20480
6687 009/300 Read block 4771 len 4096
6688 010/300 Read block 2035 len 16384
6689 011/300 Read block 4309 len 4096
6690 012/300 Read block 2619 len 16384
6691 013/300 Read block 3521 len 8192
6692 014/300 Read block 4005 len 24576
6693 015/300 Read block 1400 len 12288
6694 016/300 Flush
6695 017/300 Read block 4034 len 16384
6696 018/300 Write block 2628 len 40960 data: 3 4 4 3 3 3 4 4 4 3
6697 019/300 Write block 1765 len 4096 data: 3
6698 020/300 Write block 2956 len 16384 data: 3 3 3 3
6699 021/300 Read block 768 len 20480
6700 022/300 Read block 4732 len 28672
6701 023/300 Write block 4465 len 28672 data: 2 2 3 5 5 5 4
6702 024/300 Read block 3534 len 16384
6703 025/300 Write block 945 len 20480 data: 3 3 3 3 3
6704 026/300 Write block 2224 len 24576 data: 3 3 3 3 3 4
6705 027/300 Read block 370 len 32768
6706 028/300 Flush
6707 029/300 Read block 2964 len 8192
6708 030/300 Flush
6709 031/300 Write block 3886 len 32768 data: 2 2 3 3 4 4 4 4
6710 032/300 Flush
6711 033/300 Read block 1795 len 8192
6712 034/300 Flush
6713 035/300 Write block 1510 len 32768 data: 2 3 3 3 3 3 3 3
6714 036/300 Write block 2031 len 36864 data: 4 5 5 4 5 5 3 3 3
6715 037/300 Write block 4872 len 20480 data: 3 3 3 2 2
6716 038/300 Flush
6717 039/300 Read block 2239 len 24576
6718 040/300 Read block 3550 len 36864
6719 041/300 Flush
6720 042/300 Write block 811 len 20480 data: 5 4 2 2 2
6721 043/300 Flush
6722 044/300 Write block 2498 len 28672 data: 3 3 2 3 2 2 2
6723 045/300 Write block 3648 len 24576 data: 2 2 3 3 3 2
6724 046/300 Flush
6725 047/300 Read block 217 len 24576
6726 048/300 Write block 1804 len 36864 data: 3 3 3 3 3 5 5 6 6
6727 049/300 Write block 1468 len 32768 data: 3 3 3 3 3 3 3 2
6728 050/300 Read block 4107 len 36864
6729 051/300 Write block 1446 len 40960 data: 3 3 2 2 2 2 2 2 2 2
6730 052/300 Write block 2780 len 32768 data: 2 2 2 2 2 2 2 2
6731 053/300 Flush
6732 054/300 Write block 467 len 20480 data: 3 4 4 3 3
6733 055/300 Write block 3645 len 28672 data: 3 2 2 3 3 4 4
6734 056/300 Flush
6735 057/300 Write block 1116 len 12288 data: 3 3 3
6736 058/300 Write block 497 len 40960 data: 2 3 3 3 3 3 3 3 2 2
6737 059/300 Write block 315 len 16384 data: 2 2 2 2
6738 060/300 Read block 1064 len 40960
6739 061/300 Read block 502 len 32768
6740 062/300 Read block 4813 len 20480
6741 063/300 Read block 1338 len 36864
6742 064/300 Write block 1846 len 28672 data: 2 3 3 3 3 3 3
6743 065/300 Write block 3519 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6744 066/300 Read block 3732 len 32768
6745 067/300 Read block 2864 len 4096
6746 068/300 Read block 1707 len 8192
6747 069/300 Write block 2178 len 4096 data: 3
6748 070/300 Write block 3036 len 40960 data: 3 3 3 3 3 3 4 3 2 3
6749 071/300 Write block 1726 len 28672 data: 2 2 2 2 2 2 2
6750 072/300 Read block 4645 len 24576
6751 073/300 Write block 2707 len 28672 data: 3 3 3 3 3 3 3
6752 074/300 Flush
6753 075/300 Flush
6754 076/300 Write block 1806 len 20480 data: 4 4 4 6 6
6755 077/300 Write block 4239 len 4096 data: 2
6756 078/300 Write block 3946 len 32768 data: 3 3 3 2 2 2 2 2
6757 079/300 Read block 3230 len 20480
6758 080/300 Write block 2908 len 20480 data: 3 4 4 4 3
6759 081/300 Read block 675 len 16384
6760 082/300 Read block 2561 len 40960
6761 083/300 Write block 855 len 28672 data: 2 2 2 2 2 3 4
6762 084/300 Read block 138 len 36864
6763 085/300 Write block 2732 len 20480 data: 2 2 2 3 3
6764 086/300 Read block 3243 len 28672
6765 087/300 Write block 129 len 32768 data: 3 3 3 3 3 3 3 3
6766 088/300 Read block 256 len 24576
6767 089/300 Write block 3721 len 32768 data: 3 3 3 3 4 4 4 4
6768 090/300 Write block 3718 len 20480 data: 2 3 3 4 4
6769 091/300 Read block 4110 len 40960
6770 092/300 Read block 3885 len 8192
6771 093/300 Read block 2728 len 32768
6772 094/300 Write block 461 len 32768 data: 3 3 3 3 3 3 4 5
6773 095/300 Write block 3856 len 24576 data: 3 3 3 2 2 2
6774 096/300 Flush
6775 097/300 Read block 3261 len 12288
6776 098/300 Write block 1524 len 40960 data: 2 2 2 2 2 3 3 3 3 3
6777 099/300 Read block 1290 len 16384
6778 100/300 Write block 737 len 32768 data: 5 5 4 3 2 2 2 2
6779 101/300 Read block 3131 len 28672
6780 102/300 Write block 2776 len 36864 data: 2 2 2 2 3 3 3 3 3
6781 103/300 Write block 1379 len 16384 data: 2 2 2 3
6782 104/300 Read block 4032 len 28672
6783 105/300 Flush
6784 106/300 Read block 711 len 36864
6785 107/300 Write block 4533 len 32768 data: 2 2 2 2 2 2 2 2
6786 108/300 Read block 3737 len 8192
6787 109/300 Write block 1698 len 4096 data: 2
6788 110/300 Write block 1121 len 8192 data: 2 2
6789 111/300 Read block 4534 len 20480
6790 112/300 Read block 4686 len 36864
6791 113/300 Write block 3864 len 12288 data: 3 3 4
6792 114/300 Write block 3952 len 20480 data: 3 3 2 2 2
6793 115/300 Write block 930 len 12288 data: 2 2 2
6794 116/300 Write block 1014 len 20480 data: 3 3 3 3 3
6795 117/300 Write block 2188 len 12288 data: 3 4 4
6796 118/300 Flush
6797 119/300 Read block 1475 len 8192
6798 120/300 Write block 1191 len 40960 data: 5 5 5 4 5 4 3 3 3 4
6799 121/300 Flush
6800 122/300 Write block 4268 len 20480 data: 2 2 2 2 2
6801 123/300 Write block 1876 len 32768 data: 3 3 3 3 3 3 2 2
6802 124/300 Read block 4410 len 36864
6803 125/300 Read block 4081 len 32768
6804 126/300 Write block 3071 len 12288 data: 2 2 2
6805 127/300 Write block 4451 len 28672 data: 2 2 2 2 2 2 2
6806 128/300 Read block 4266 len 16384
6807 129/300 Read block 3633 len 40960
6808 130/300 Read block 510 len 4096
6809 131/300 Write block 3440 len 24576 data: 4 3 3 3 3 3
6810 132/300 Write block 2620 len 40960 data: 2 3 3 3 3 3 3 3 4 5
6811 133/300 Read block 4991 len 32768
6812 134/300 Write block 15 len 32768 data: 2 2 2 2 2 2 2 2
6813 135/300 Read block 2652 len 12288
6814 136/300 Flush
6815 137/300 Read block 333 len 24576
6816 138/300 Read block 572 len 20480
6817 139/300 Read block 4941 len 4096
6818 140/300 Read block 1571 len 8192
6819 141/300 Read block 740 len 24576
6820 142/300 Write block 47 len 28672 data: 5 4 4 4 4 4 4
6821 143/300 Read block 200 len 36864
6822 144/300 Write block 3958 len 36864 data: 2 2 3 3 3 3 3 3 2
6823 145/300 Read block 1037 len 32768
6824 146/300 Write block 1010 len 16384 data: 3 3 3 3
6825 147/300 Flush
6826 148/300 Read block 3835 len 36864
6827 149/300 Write block 3850 len 36864 data: 2 2 2 3 4 4 4 4 4
6828 150/300 Write block 2094 len 36864 data: 2 2 2 2 2 2 2 2 2
6829 151/300 Write block 4840 len 12288 data: 2 3 3
6830 152/300 Read block 4014 len 40960
6831 153/300 Write block 1156 len 16384 data: 3 4 4 4
6832 154/300 Flush
6833 155/300 Read block 1236 len 24576
6834 156/300 Flush
6835 157/300 Write block 728 len 24576 data: 3 3 3 4 5 5
6836 158/300 Write block 2984 len 28672 data: 2 2 2 2 2 2 2
6837 159/300 Write block 30 len 12288 data: 2 2 2
6838 160/300 Read block 1932 len 40960
6839 161/300 Read block 3618 len 28672
6840 162/300 Read block 3088 len 32768
6841 163/300 Write block 1090 len 32768 data: 3 3 3 3 3 3 4 4
6842 164/300 Write block 1346 len 36864 data: 4 4 4 3 3 2 4 3 3
6843 165/300 Read block 1322 len 8192
6844 166/300 Write block 4105 len 8192 data: 2 2
6845 167/300 Read block 4166 len 28672
6846 168/300 Read block 1708 len 36864
6847 169/300 Write block 1558 len 36864 data: 2 2 2 2 3 3 3 3 3
6848 170/300 Read block 137 len 28672
6849 171/300 Read block 4804 len 12288
6850 172/300 Write block 4952 len 28672 data: 2 2 2 2 2 2 2
6851 173/300 Write block 3041 len 12288 data: 4 5 4
6852 174/300 Read block 1872 len 40960
6853 175/300 Read block 4523 len 4096
6854 176/300 Write block 4970 len 16384 data: 2 2 2 2
6855 177/300 Read block 1107 len 20480
6856 178/300 Read block 4622 len 8192
6857 179/300 Read block 867 len 20480
6858 180/300 Write block 687 len 40960 data: 4 4 3 3 2 2 2 2 3 4
6859 181/300 Write block 573 len 32768 data: 2 2 2 2 2 2 2 2
6860 182/300 Read block 4509 len 8192
6861 183/300 Write block 3338 len 32768 data: 2 2 2 2 2 2 3 3
6862 184/300 Write block 4927 len 36864 data: 4 4 4 4 3 2 2 2 2
6863 185/300 Write block 1782 len 28672 data: 4 4 4 4 4 3 2
6864 186/300 Write block 248 len 24576 data: 2 2 2 2 2 2
6865 187/300 Write block 1891 len 8192 data: 2 2
6866 188/300 Read block 3634 len 8192
6867 189/300 Write block 4743 len 32768 data: 2 2 2 2 2 2 3 3
6868 190/300 Write block 3297 len 16384 data: 5 3 3 3
6869 191/300 Read block 4302 len 20480
6870 192/300 Flush
6871 193/300 Read block 3638 len 40960
6872 194/300 Read block 4882 len 8192
6873 195/300 Write block 3360 len 40960 data: 3 3 4 4 3 3 4 4 4 3
6874 196/300 Read block 4045 len 16384
6875 197/300 Write block 1577 len 32768 data: 5 4 4 4 4 3 3 3
6876 198/300 Write block 3475 len 12288 data: 3 3 3
6877 199/300 Read block 4491 len 16384
6878 200/300 Write block 1751 len 32768 data: 4 4 4 3 4 5 5 4
6879 201/300 Write block 4100 len 16384 data: 2 2 2 2
6880 202/300 Write block 3334 len 4096 data: 2
6881 203/300 Read block 1989 len 40960
6882 204/300 Write block 4268 len 20480 data: 3 3 3 3 3
6883 205/300 Write block 1036 len 16384 data: 3 4 5 5
6884 206/300 Write block 4030 len 40960 data: 3 3 3 3 3 3 4 4 3 2
6885 207/300 Read block 2793 len 28672
6886 208/300 Read block 4064 len 8192
6887 209/300 Read block 2319 len 8192
6888 210/300 Read block 4746 len 40960
6889 211/300 Read block 741 len 28672
6890 212/300 Read block 814 len 24576
6891 213/300 Read block 4173 len 16384
6892 214/300 Read block 570 len 20480
6893 215/300 Write block 703 len 16384 data: 2 2 2 2
6894 216/300 Read block 3133 len 16384
6895 217/300 Write block 3414 len 28672 data: 2 2 2 4 4 4 4
6896 218/300 Read block 1954 len 4096
6897 219/300 Read block 3222 len 36864
6898 220/300 Read block 4580 len 28672
6899 221/300 Write block 1883 len 32768 data: 3 2 2 2 2 2 2 2
6900 222/300 Write block 4158 len 20480 data: 3 3 3 2 2
6901 223/300 Write block 377 len 4096 data: 2
6902 224/300 Read block 1121 len 20480
6903 225/300 Read block 1223 len 12288
6904 226/300 Read block 2674 len 32768
6905 227/300 Read block 2331 len 24576
6906 228/300 Read block 3477 len 12288
6907 229/300 Write block 67 len 8192 data: 3 3
6908 230/300 Read block 1371 len 40960
6909 231/300 Read block 4968 len 24576
6910 232/300 Write block 3349 len 40960 data: 4 5 5 4 4 3 4 3 3 3
6911 233/300 Write block 4380 len 40960 data: 2 3 3 3 2 2 3 3 5 7
6912 234/300 Read block 2190 len 16384
6913 235/300 Read block 3982 len 20480
6914 236/300 Write block 928 len 16384 data: 2 2 3 3
6915 237/300 Write block 4397 len 20480 data: 3 3 3 3 2
6916 238/300 Write block 790 len 20480 data: 3 4 3 3 2
6917 239/300 Read block 1023 len 40960
6918 240/300 Write block 2020 len 8192 data: 3 3
6919 241/300 Read block 4231 len 4096
6920 242/300 Read block 4583 len 12288
6921 243/300 Write block 1514 len 24576 data: 4 4 4 4 3 2
6922 244/300 Read block 3163 len 24576
6923 245/300 Write block 1286 len 40960 data: 2 2 2 2 2 2 3 3 3 3
6924 246/300 Read block 2899 len 40960
6925 247/300 Read block 633 len 36864
6926 248/300 Read block 3757 len 40960
6927 249/300 Read block 3633 len 8192
6928 250/300 Write block 2210 len 12288 data: 3 3 3
6929 251/300 Read block 3305 len 4096
6930 252/300 Write block 4952 len 36864 data: 3 3 3 3 3 3 3 2 2
6931 253/300 Flush
6932 254/300 Read block 4906 len 16384
6933 255/300 Write block 156 len 40960 data: 4 4 3 2 2 2 2 2 2 2
6934 256/300 Flush
6935 257/300 Read block 3531 len 8192
6936 258/300 Write block 413 len 28672 data: 3 3 2 2 2 2 2
6937 259/300 Read block 3341 len 16384
6938 260/300 Write block 2188 len 24576 data: 4 5 5 5 4 2
6939 261/300 Write block 913 len 24576 data: 3 3 3 3 4 4
6940 262/300 Read block 1001 len 36864
6941 263/300 Read block 2986 len 36864
6942 264/300 Read block 2472 len 36864
6943 265/300 Read block 727 len 32768
6944 266/300 Read block 4959 len 24576
6945 267/300 Write block 722 len 40960 data: 2 2 2 2 2 2 4 4 4 5
6946 268/300 Write block 1018 len 4096 data: 4
6947 269/300 Read block 2142 len 36864
6948 270/300 Flush
6949 271/300 Flush
6950 272/300 Write block 3171 len 32768 data: 2 2 3 3 3 3 4 4
6951 273/300 Read block 2522 len 16384
6952 274/300 Write block 2604 len 24576 data: 4 4 3 3 3 4
6953 275/300 Write block 2314 len 20480 data: 2 2 2 2 2
6954 276/300 Read block 3836 len 28672
6955 277/300 Read block 2173 len 32768
6956 278/300 Read block 240 len 32768
6957 279/300 Write block 637 len 28672 data: 5 5 5 4 4 3 3
6958 280/300 Read block 185 len 36864
6959 281/300 Write block 198 len 28672 data: 2 2 3 3 4 4 4
6960 282/300 Read block 124 len 28672
6961 283/300 Write block 792 len 4096 data: 4
6962 284/300 Write block 4340 len 16384 data: 2 2 2 2
6963 285/300 Read block 1273 len 12288
6964 286/300 Flush
6965 287/300 Read block 602 len 36864
6966 288/300 Write block 3681 len 12288 data: 2 2 2
6967 289/300 Read block 3565 len 12288
6968 290/300 Read block 970 len 12288
6969 291/300 Flush
6970 292/300 Flush
6971 293/300 Read block 424 len 12288
6972 294/300 Write block 319 len 16384 data: 2 2 2 2
6973 295/300 Write block 3937 len 40960 data: 2 2 4 3 3 3 3 3 3 4
6974 296/300 Flush
6975 297/300 Read block 2115 len 20480
6976 298/300 Read block 1553 len 4096
6977 299/300 Read block 2464 len 4096
6978 300/300 Flush
6979 301/300 Read block 2418 len 12288
6980 Main task has work to do, go find it
6981 got dsc Start(1)
6982 start 1
6983 Jan 21 15:48:39.047 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/1, method: POST, req_id: 1ab37bb9-a6c5-4d8e-b510-fa863f29450b, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
6984 Replay: started 1, returned:Ok(())
6985 ----------------------------------------------------------------
6986 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6987 States: Active Offline Active
6988 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6989 2811 Acked Write 4 Done Sent Done false
6990 2812 Acked Flush 0 Done Sent Done false
6991 2813 Acked Read 8 Done Sent Done false
6992 2814 Acked Read 2 Done Sent Done false
6993 2815 Acked Read 3 Done Sent Done false
6994 2816 Acked Read 9 Done Sent Done false
6995 2817 Acked Read 3 Done Sent Done false
6996 2818 Acked Read 5 Done Sent Done false
6997 2819 Acked Read 1 Done Sent Done false
6998 2820 Acked Read 4 Done Sent Done false
6999 2821 Acked Read 1 Done Sent Done false
7000 2822 Acked Read 4 Done Sent Done false
7001 2823 Acked Read 2 Done Sent Done false
7002 2824 Acked Read 6 Done Sent Done false
7003 2825 Acked Read 3 Done Sent Done false
7004 2826 Acked Flush 0 Done Sent Done false
7005 2827 Acked Read 4 Done Sent Done false
7006 2828 Acked Write 10 Done Sent Done false
7007 2829 Acked Write 1 Done Sent Done false
7008 2830 Acked Write 4 Done Sent Done false
7009 2831 Acked Read 5 Done Sent Done false
7010 2832 Acked Read 7 Done Sent Done false
7011 2833 Acked Write 7 Done Sent Done false
7012 2834 Acked Read 4 Done Sent Done false
7013 2835 Acked Write 5 Done Sent Done false
7014 2836 Acked Write 6 Done Sent Done false
7015 2837 Acked Read 8 Done Sent Done false
7016 2838 Acked Flush 0 Done Sent Done false
7017 2839 Acked Read 2 Done Sent Done false
7018 2840 Acked Flush 0 Done Sent Done false
7019 2841 Acked Write 8 Done Sent Done false
7020 2842 Acked Flush 0 Done Sent Done false
7021 2843 Acked Read 2 Done Sent Done false
7022 2844 Acked Flush 0 Done Sent Done false
7023 2845 Acked Write 8 Done Sent Done false
7024 2846 Acked Write 9 Done Sent Done false
7025 2847 Acked Write 5 Done Sent Done false
7026 2848 Acked Flush 0 Done Sent Done false
7027 2849 Acked Read 6 Done Sent Done false
7028 2850 Acked Read 9 Done Sent Done false
7029 2851 Acked Flush 0 Done Sent Done false
7030 2852 Acked Write 5 Done Sent Done false
7031 2853 Acked Flush 0 Done Sent Done false
7032 2854 Acked Write 7 Done Sent Done false
7033 2855 Acked Write 6 Done Sent Done false
7034 2856 Acked Flush 0 Done Sent Done false
7035 2857 Acked Read 6 Done Sent Done false
7036 2858 Acked Write 9 Done Sent Done false
7037 2859 Acked Write 8 Done Sent Done false
7038 2860 Acked Read 9 Done Sent Done false
7039 2861 Acked Write 10 Done Sent Done false
7040 2862 Acked Write 8 Done Sent Done false
7041 2863 Acked Flush 0 Done Sent Done false
7042 2864 Acked Write 5 Done Sent Done false
7043 2865 Acked Write 7 Done Sent Done false
7044 2866 Acked Flush 0 Done Sent Done false
7045 2867 Acked Write 3 Done Sent Done false
7046 2868 Acked Write 10 Done Sent Done false
7047 2869 Acked Write 4 Done Sent Done false
7048 2870 Acked Read 10 Done Sent Done false
7049 2871 Acked Read 8 Done Sent Done false
7050 2872 Acked Read 5 Done Sent Done false
7051 2873 Acked Read 9 Done Sent Done false
7052 2874 Acked Write 7 Done Sent Done false
7053 2875 Acked Write 10 Done Sent Done false
7054 2876 Acked Read 8 Done Sent Done false
7055 2877 Acked Read 1 Done Sent Done false
7056 2878 Acked Read 2 Done Sent Done false
7057 2879 Acked Write 1 Done Sent Done false
7058 2880 Acked Write 10 Done Sent Done false
7059 2881 Acked Write 7 Done Sent Done false
7060 2882 Acked Read 6 Done Sent Done false
7061 2883 Acked Write 7 Done Sent Done false
7062 2884 Acked Flush 0 Done Sent Done false
7063 2885 Acked Flush 0 Done Sent Done false
7064 2886 Acked Write 5 Done Sent Done false
7065 2887 Acked Write 1 Done Sent Done false
7066 2888 Acked Write 8 Done Sent Done false
7067 2889 Acked Read 5 Done Sent Done false
7068 2890 Acked Write 5 Done Sent Done false
7069 2891 Acked Read 4 Done Sent Done false
7070 2892 Acked Read 10 Done Sent Done false
7071 2893 Acked Write 7 Done Sent Done false
7072 2894 Acked Read 9 Done Sent Done false
7073 2895 Acked Write 5 Done Sent Done false
7074 2896 Acked Read 7 Done Sent Done false
7075 2897 Acked Write 8 Done Sent Done false
7076 2898 Acked Read 6 Done Sent Done false
7077 2899 Acked Write 8 Done Sent Done false
7078 2900 Acked Write 5 Done Sent Done false
7079 2901 Acked Read 10 Done Sent Done false
7080 2902 Acked Read 2 Done Sent Done false
7081 2903 Acked Read 8 Done Sent Done false
7082 2904 Acked Write 8 Done Sent Done false
7083 2905 Acked Write 6 Done Sent Done false
7084 2906 Acked Flush 0 Done Sent Done false
7085 2907 Acked Read 3 Done Sent Done false
7086 2908 Acked Write 10 Done Sent Done false
7087 2909 Acked Read 4 Done Sent Done false
7088 2910 Acked Write 8 Done Sent Done false
7089 2911 Acked Read 7 Done Sent Done false
7090 2912 Acked Write 9 Done Sent Done false
7091 2913 Acked Write 4 Done Sent Done false
7092 2914 Acked Read 7 Done Sent Done false
7093 2915 Acked Flush 0 Done Sent Done false
7094 2916 Acked Read 9 Done Sent Done false
7095 2917 Acked Write 8 Done Sent Done false
7096 2918 Acked Read 2 Done Sent Done false
7097 2919 Acked Write 1 Done Sent Done false
7098 2920 Acked Write 2 Done Sent Done false
7099 2921 Acked Read 5 Done Sent Done false
7100 2922 Acked Read 9 Done Sent Done false
7101 2923 Acked Write 3 Done Sent Done false
7102 2924 Acked Write 5 Done Sent Done false
7103 2925 Acked Write 3 Done Sent Done false
7104 2926 Acked Write 5 Done Sent Done false
7105 2927 Acked Write 3 Done Sent Done false
7106 2928 Acked Flush 0 Done Sent Done false
7107 2929 Acked Read 2 Done Sent Done false
7108 2930 Acked Write 10 Done Sent Done false
7109 2931 Acked Flush 0 Done Sent Done false
7110 2932 Acked Write 5 Done Sent Done false
7111 2933 Acked Write 8 Done Sent Done false
7112 2934 Acked Read 9 Done Sent Done false
7113 2935 Acked Read 8 Done Sent Done false
7114 2936 Acked Write 3 Done Sent Done false
7115 2937 Acked Write 7 Done Sent Done false
7116 2938 Acked Read 4 Done Sent Done false
7117 2939 Acked Read 10 Done Sent Done false
7118 2940 Acked Read 1 Done Sent Done false
7119 2941 Acked Write 6 Done Sent Done false
7120 2942 Acked Write 10 Done Sent Done false
7121 2943 Acked Read 8 Done Sent Done false
7122 2944 Acked Write 8 Done Sent Done false
7123 2945 Acked Read 3 Done Sent Done false
7124 2946 Acked Flush 0 Done Sent Done false
7125 2947 Acked Read 6 Done Sent Done false
7126 2948 Acked Read 5 Done Sent Done false
7127 2949 Acked Read 1 Done Sent Done false
7128 2950 Acked Read 2 Done Sent Done false
7129 2951 Acked Read 6 Done Sent Done false
7130 2952 Acked Write 7 Done Sent Done false
7131 2953 Acked Read 9 Done Sent Done false
7132 2954 Acked Write 9 Done Sent Done false
7133 2955 Acked Read 8 Done Sent Done false
7134 2956 Acked Write 4 Done Sent Done false
7135 2957 Acked Flush 0 Done Sent Done false
7136 2958 Acked Read 9 Done Sent Done false
7137 2959 Acked Write 9 Done Sent Done false
7138 2960 Acked Write 9 Done Sent Done false
7139 2961 Acked Write 3 Done Sent Done false
7140 2962 Acked Read 10 Done Sent Done false
7141 2963 Acked Write 4 Done Sent Done false
7142 2964 Acked Flush 0 Done Sent Done false
7143 2965 Acked Read 6 Done Sent Done false
7144 2966 Acked Flush 0 Done Sent Done false
7145 2967 Acked Write 6 Done Sent Done false
7146 2968 Acked Write 7 Done Sent Done false
7147 2969 Acked Write 3 Done Sent Done false
7148 2970 Acked Read 10 Done Sent Done false
7149 2971 Acked Read 7 Done Sent Done false
7150 2972 Acked Read 8 Done Sent Done false
7151 2973 Acked Write 8 Done Sent Done false
7152 2974 Acked Write 9 Done Sent Done false
7153 2975 Acked Read 2 Done Sent Done false
7154 2976 Acked Write 2 Done Sent Done false
7155 2977 Acked Read 7 Done Sent Done false
7156 2978 Acked Read 9 Done Sent Done false
7157 2979 Acked Write 9 Done Sent Done false
7158 2980 Acked Read 7 Done Sent Done false
7159 2981 Acked Read 3 Done Sent Done false
7160 2982 Acked Write 7 Done Sent Done false
7161 2983 Acked Write 3 Done Sent Done false
7162 2984 Acked Read 10 Done Sent Done false
7163 2985 Acked Read 1 Done Sent Done false
7164 2986 Acked Write 4 Done Sent Done false
7165 2987 Acked Read 5 Done Sent Done false
7166 2988 Acked Read 2 Done Sent Done false
7167 2989 Acked Read 5 Done Sent Done false
7168 2990 Acked Write 10 Done Sent Done false
7169 2991 Acked Write 8 Done Sent Done false
7170 2992 Acked Read 2 Done Sent Done false
7171 2993 Acked Write 8 Done Sent Done false
7172 2994 Acked Write 9 Done Sent Done false
7173 2995 Acked Write 7 Done Sent Done false
7174 2996 Acked Write 6 Done Sent Done false
7175 2997 Acked Write 2 Done Sent Done false
7176 2998 Acked Read 2 Done Sent Done false
7177 2999 Acked Write 8 Done Sent Done false
7178 3000 Acked Write 4 Done Sent Done false
7179 3001 Acked Read 5 Done Sent Done false
7180 3002 Acked Flush 0 Done Sent Done false
7181 3003 Acked Read 10 Done Sent Done false
7182 3004 Acked Read 2 Done Sent Done false
7183 3005 Acked Write 10 Done Sent Done false
7184 3006 Acked Read 4 Done Sent Done false
7185 3007 Acked Write 8 Done Sent Done false
7186 3008 Acked Write 3 Done Sent Done false
7187 3009 Acked Read 4 Done Sent Done false
7188 3010 Acked Write 8 Done Sent Done false
7189 3011 Acked Write 4 Done Sent Done false
7190 3012 Acked Write 1 Done Sent Done false
7191 3013 Acked Read 10 Done Sent Done false
7192 3014 Acked Write 5 Done Sent Done false
7193 3015 Acked Write 4 Done Sent Done false
7194 3016 Acked Write 10 Done Sent Done false
7195 3017 Acked Read 7 Done Sent Done false
7196 3018 Acked Read 2 Done Sent Done false
7197 3019 Acked Read 2 Done Sent Done false
7198 3020 Acked Read 10 Done Sent Done false
7199 3021 Acked Read 7 Done Sent Done false
7200 3022 Acked Read 6 Done Sent Done false
7201 3023 Acked Read 4 Done Sent Done false
7202 3024 Acked Read 5 Done Sent Done false
7203 3025 Acked Write 4 Done Sent Done false
7204 3026 Acked Read 4 Done Sent Done false
7205 3027 Acked Write 7 Done Sent Done false
7206 3028 Acked Read 1 Done Sent Done false
7207 3029 Acked Read 9 Done Sent Done false
7208 3030 Acked Read 7 Done Sent Done false
7209 3031 Acked Write 8 Done Sent Done false
7210 3032 Acked Write 5 Done Sent Done false
7211 3033 Acked Write 1 Done Sent Done false
7212 3034 Acked Read 5 Done Sent Done false
7213 3035 Acked Read 3 Done Sent Done false
7214 3036 Acked Read 8 Done Sent Done false
7215 3037 Acked Read 6 Done Sent Done false
7216 3038 Acked Read 3 Done Sent Done false
7217 3039 Acked Write 2 Done Sent Done false
7218 3040 Acked Read 10 Done Sent Done false
7219 3041 Acked Read 6 Done Sent Done false
7220 3042 Acked Write 10 Done Sent Done false
7221 3043 Acked Write 10 Done Sent Done false
7222 3044 Acked Read 4 Done Sent Done false
7223 3045 Acked Read 5 Done Sent Done false
7224 3046 Acked Write 4 Done Sent Done false
7225 3047 Acked Write 5 Done Sent Done false
7226 3048 Acked Write 5 Done Sent Done false
7227 3049 Acked Read 10 Done Sent Done false
7228 3050 Acked Write 2 Done Sent Done false
7229 3051 Acked Read 1 Done Sent Done false
7230 3052 Acked Read 3 Done Sent Done false
7231 3053 Acked Write 6 Done Sent Done false
7232 3054 Acked Read 6 Done Sent Done false
7233 3055 Acked Write 10 Done Sent Done false
7234 3056 Acked Read 10 Done Sent Done false
7235 3057 Acked Read 9 Done Sent Done false
7236 3058 Acked Read 10 Done Sent Done false
7237 3059 Acked Read 2 Done Sent Done false
7238 3060 Acked Write 3 Done Sent Done false
7239 3061 Acked Read 1 Done Sent Done false
7240 3062 Acked Write 9 Done Sent Done false
7241 3063 Acked Flush 0 Done Sent Done false
7242 3064 Acked Read 4 Done Sent Done false
7243 3065 Acked Write 10 Done Sent Done false
7244 3066 Acked Flush 0 Done Sent Done false
7245 3067 Acked Read 2 Done Sent Done false
7246 3068 Acked Write 7 Done Sent Done false
7247 3069 Acked Read 4 Done Sent Done false
7248 3070 Acked Write 6 Done Sent Done false
7249 3071 Acked Write 6 Done Sent Done false
7250 3072 Acked Read 9 Done Sent Done false
7251 3073 Acked Read 9 Done Sent Done false
7252 3074 Acked Read 9 Done Sent Done false
7253 3075 Acked Read 8 Done Sent Done false
7254 3076 Acked Read 6 Done Sent Done false
7255 3077 Acked Write 10 Done Sent Done false
7256 3078 Acked Write 1 Done Sent Done false
7257 3079 Acked Read 9 Done Sent Done false
7258 3080 Acked Flush 0 Done Sent Done false
7259 3081 Acked Flush 0 Done Sent Done false
7260 3082 Acked Write 8 Done Sent Done false
7261 3083 Acked Read 4 Done Sent Done false
7262 3084 Acked Write 6 Done Sent Done false
7263 3085 Acked Write 5 Done Sent Done false
7264 3086 Acked Read 7 Done Sent Done false
7265 3087 Acked Read 8 Done Sent Done false
7266 3088 Acked Read 8 Done Sent Done false
7267 3089 Acked Write 7 Done Sent Done false
7268 3090 Acked Read 9 Done Sent Done false
7269 3091 Acked Write 7 Done Sent Done false
7270 3092 Acked Read 7 Done Sent Done false
7271 3093 Acked Write 1 Done Sent Done false
7272 3094 Acked Write 4 Done Sent Done false
7273 3095 Acked Read 3 Done Sent Done false
7274 3096 Acked Flush 0 Done Sent Done false
7275 3097 Acked Read 9 Done Sent Done false
7276 3098 Acked Write 3 Done Sent Done false
7277 3099 Acked Read 3 Done Sent Done false
7278 3100 Acked Read 3 Done Sent Done false
7279 3101 Acked Flush 0 Done Sent Done false
7280 3102 Acked Flush 0 Done Sent Done false
7281 3103 Acked Read 3 Done Sent Done false
7282 3104 Acked Write 4 Done Sent Done false
7283 3105 Acked Write 10 Done Sent Done false
7284 3106 Acked Flush 0 Done Sent Done false
7285 3107 Acked Read 5 Done Sent Done false
7286 3108 Acked Read 1 Done Sent Done false
7287 3109 Acked Read 1 Done Sent Done false
7288 3110 Acked Flush 0 Done Sent Done false
7289 3111 Acked Read 3 Done Sent Done false
7290 STATES DS:0 DS:1 DS:2 TOTAL
7291 Sent 0 301 0 301
7292 Done 301 0 301 602
7293 Skipped 0 0 0 0
7294 Error 0 0 0 0
7295 Last Flush: 3110 2810 3110
7296 Downstairs last five completed: 2810 2809 2808 2807 2806
7297 Upstairs last five completed: 3111 3110 3109 3108 3107
7298 CLIENT: Up:0 ds:301 act:2
7299 [8820] got start action
7300 [8820] I am going to restart
7301 Starting downstairs at port 8820
7302 Make output file at "/tmp/dsc/downstairs-8820.txt"
7303 [8820][1] reports Starting
7304 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1207)
7305 [8820][1] reports Running
7306 ----------------------------------------------------------------
7307 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
7308 States: Active Offline Active
7309 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7310 2811 Acked Write 4 Done Sent Done false
7311 2812 Acked Flush 0 Done Sent Done false
7312 2813 Acked Read 8 Done Sent Done false
7313 2814 Acked Read 2 Done Sent Done false
7314 2815 Acked Read 3 Done Sent Done false
7315 2816 Acked Read 9 Done Sent Done false
7316 2817 Acked Read 3 Done Sent Done false
7317 2818 Acked Read 5 Done Sent Done false
7318 2819 Acked Read 1 Done Sent Done false
7319 2820 Acked Read 4 Done Sent Done false
7320 2821 Acked Read 1 Done Sent Done false
7321 2822 Acked Read 4 Done Sent Done false
7322 2823 Acked Read 2 Done Sent Done false
7323 2824 Acked Read 6 Done Sent Done false
7324 2825 Acked Read 3 Done Sent Done false
7325 2826 Acked Flush 0 Done Sent Done false
7326 2827 Acked Read 4 Done Sent Done false
7327 2828 Acked Write 10 Done Sent Done false
7328 2829 Acked Write 1 Done Sent Done false
7329 2830 Acked Write 4 Done Sent Done false
7330 2831 Acked Read 5 Done Sent Done false
7331 2832 Acked Read 7 Done Sent Done false
7332 2833 Acked Write 7 Done Sent Done false
7333 2834 Acked Read 4 Done Sent Done false
7334 2835 Acked Write 5 Done Sent Done false
7335 2836 Acked Write 6 Done Sent Done false
7336 2837 Acked Read 8 Done Sent Done false
7337 2838 Acked Flush 0 Done Sent Done false
7338 2839 Acked Read 2 Done Sent Done false
7339 2840 Acked Flush 0 Done Sent Done false
7340 2841 Acked Write 8 Done Sent Done false
7341 2842 Acked Flush 0 Done Sent Done false
7342 2843 Acked Read 2 Done Sent Done false
7343 2844 Acked Flush 0 Done Sent Done false
7344 2845 Acked Write 8 Done Sent Done false
7345 2846 Acked Write 9 Done Sent Done false
7346 2847 Acked Write 5 Done Sent Done false
7347 2848 Acked Flush 0 Done Sent Done false
7348 2849 Acked Read 6 Done Sent Done false
7349 2850 Acked Read 9 Done Sent Done false
7350 2851 Acked Flush 0 Done Sent Done false
7351 2852 Acked Write 5 Done Sent Done false
7352 2853 Acked Flush 0 Done Sent Done false
7353 2854 Acked Write 7 Done Sent Done false
7354 2855 Acked Write 6 Done Sent Done false
7355 2856 Acked Flush 0 Done Sent Done false
7356 2857 Acked Read 6 Done Sent Done false
7357 2858 Acked Write 9 Done Sent Done false
7358 2859 Acked Write 8 Done Sent Done false
7359 2860 Acked Read 9 Done Sent Done false
7360 2861 Acked Write 10 Done Sent Done false
7361 2862 Acked Write 8 Done Sent Done false
7362 2863 Acked Flush 0 Done Sent Done false
7363 2864 Acked Write 5 Done Sent Done false
7364 2865 Acked Write 7 Done Sent Done false
7365 2866 Acked Flush 0 Done Sent Done false
7366 2867 Acked Write 3 Done Sent Done false
7367 2868 Acked Write 10 Done Sent Done false
7368 2869 Acked Write 4 Done Sent Done false
7369 2870 Acked Read 10 Done Sent Done false
7370 2871 Acked Read 8 Done Sent Done false
7371 2872 Acked Read 5 Done Sent Done false
7372 2873 Acked Read 9 Done Sent Done false
7373 2874 Acked Write 7 Done Sent Done false
7374 2875 Acked Write 10 Done Sent Done false
7375 2876 Acked Read 8 Done Sent Done false
7376 2877 Acked Read 1 Done Sent Done false
7377 2878 Acked Read 2 Done Sent Done false
7378 2879 Acked Write 1 Done Sent Done false
7379 2880 Acked Write 10 Done Sent Done false
7380 2881 Acked Write 7 Done Sent Done false
7381 2882 Acked Read 6 Done Sent Done false
7382 2883 Acked Write 7 Done Sent Done false
7383 2884 Acked Flush 0 Done Sent Done false
7384 2885 Acked Flush 0 Done Sent Done false
7385 2886 Acked Write 5 Done Sent Done false
7386 2887 Acked Write 1 Done Sent Done false
7387 2888 Acked Write 8 Done Sent Done false
7388 2889 Acked Read 5 Done Sent Done false
7389 2890 Acked Write 5 Done Sent Done false
7390 2891 Acked Read 4 Done Sent Done false
7391 2892 Acked Read 10 Done Sent Done false
7392 2893 Acked Write 7 Done Sent Done false
7393 2894 Acked Read 9 Done Sent Done false
7394 2895 Acked Write 5 Done Sent Done false
7395 2896 Acked Read 7 Done Sent Done false
7396 2897 Acked Write 8 Done Sent Done false
7397 2898 Acked Read 6 Done Sent Done false
7398 2899 Acked Write 8 Done Sent Done false
7399 2900 Acked Write 5 Done Sent Done false
7400 2901 Acked Read 10 Done Sent Done false
7401 2902 Acked Read 2 Done Sent Done false
7402 2903 Acked Read 8 Done Sent Done false
7403 2904 Acked Write 8 Done Sent Done false
7404 2905 Acked Write 6 Done Sent Done false
7405 2906 Acked Flush 0 Done Sent Done false
7406 2907 Acked Read 3 Done Sent Done false
7407 2908 Acked Write 10 Done Sent Done false
7408 2909 Acked Read 4 Done Sent Done false
7409 2910 Acked Write 8 Done Sent Done false
7410 2911 Acked Read 7 Done Sent Done false
7411 2912 Acked Write 9 Done Sent Done false
7412 2913 Acked Write 4 Done Sent Done false
7413 2914 Acked Read 7 Done Sent Done false
7414 2915 Acked Flush 0 Done Sent Done false
7415 2916 Acked Read 9 Done Sent Done false
7416 2917 Acked Write 8 Done Sent Done false
7417 2918 Acked Read 2 Done Sent Done false
7418 2919 Acked Write 1 Done Sent Done false
7419 2920 Acked Write 2 Done Sent Done false
7420 2921 Acked Read 5 Done Sent Done false
7421 2922 Acked Read 9 Done Sent Done false
7422 2923 Acked Write 3 Done Sent Done false
7423 2924 Acked Write 5 Done Sent Done false
7424 2925 Acked Write 3 Done Sent Done false
7425 2926 Acked Write 5 Done Sent Done false
7426 2927 Acked Write 3 Done Sent Done false
7427 2928 Acked Flush 0 Done Sent Done false
7428 2929 Acked Read 2 Done Sent Done false
7429 2930 Acked Write 10 Done Sent Done false
7430 2931 Acked Flush 0 Done Sent Done false
7431 2932 Acked Write 5 Done Sent Done false
7432 2933 Acked Write 8 Done Sent Done false
7433 2934 Acked Read 9 Done Sent Done false
7434 2935 Acked Read 8 Done Sent Done false
7435 2936 Acked Write 3 Done Sent Done false
7436 2937 Acked Write 7 Done Sent Done false
7437 2938 Acked Read 4 Done Sent Done false
7438 2939 Acked Read 10 Done Sent Done false
7439 2940 Acked Read 1 Done Sent Done false
7440 2941 Acked Write 6 Done Sent Done false
7441 2942 Acked Write 10 Done Sent Done false
7442 2943 Acked Read 8 Done Sent Done false
7443 2944 Acked Write 8 Done Sent Done false
7444 2945 Acked Read 3 Done Sent Done false
7445 2946 Acked Flush 0 Done Sent Done false
7446 2947 Acked Read 6 Done Sent Done false
7447 2948 Acked Read 5 Done Sent Done false
7448 2949 Acked Read 1 Done Sent Done false
7449 2950 Acked Read 2 Done Sent Done false
7450 2951 Acked Read 6 Done Sent Done false
7451 2952 Acked Write 7 Done Sent Done false
7452 2953 Acked Read 9 Done Sent Done false
7453 2954 Acked Write 9 Done Sent Done false
7454 2955 Acked Read 8 Done Sent Done false
7455 2956 Acked Write 4 Done Sent Done false
7456 2957 Acked Flush 0 Done Sent Done false
7457 2958 Acked Read 9 Done Sent Done false
7458 2959 Acked Write 9 Done Sent Done false
7459 2960 Acked Write 9 Done Sent Done false
7460 2961 Acked Write 3 Done Sent Done false
7461 2962 Acked Read 10 Done Sent Done false
7462 2963 Acked Write 4 Done Sent Done false
7463 2964 Acked Flush 0 Done Sent Done false
7464 2965 Acked Read 6 Done Sent Done false
7465 2966 Acked Flush 0 Done Sent Done false
7466 2967 Acked Write 6 Done Sent Done false
7467 2968 Acked Write 7 Done Sent Done false
7468 2969 Acked Write 3 Done Sent Done false
7469 2970 Acked Read 10 Done Sent Done false
7470 2971 Acked Read 7 Done Sent Done false
7471 2972 Acked Read 8 Done Sent Done false
7472 2973 Acked Write 8 Done Sent Done false
7473 2974 Acked Write 9 Done Sent Done false
7474 2975 Acked Read 2 Done Sent Done false
7475 2976 Acked Write 2 Done Sent Done false
7476 2977 Acked Read 7 Done Sent Done false
7477 2978 Acked Read 9 Done Sent Done false
7478 2979 Acked Write 9 Done Sent Done false
7479 2980 Acked Read 7 Done Sent Done false
7480 2981 Acked Read 3 Done Sent Done false
7481 2982 Acked Write 7 Done Sent Done false
7482 2983 Acked Write 3 Done Sent Done false
7483 2984 Acked Read 10 Done Sent Done false
7484 2985 Acked Read 1 Done Sent Done false
7485 2986 Acked Write 4 Done Sent Done false
7486 2987 Acked Read 5 Done Sent Done false
7487 2988 Acked Read 2 Done Sent Done false
7488 2989 Acked Read 5 Done Sent Done false
7489 2990 Acked Write 10 Done Sent Done false
7490 2991 Acked Write 8 Done Sent Done false
7491 2992 Acked Read 2 Done Sent Done false
7492 2993 Acked Write 8 Done Sent Done false
7493 2994 Acked Write 9 Done Sent Done false
7494 2995 Acked Write 7 Done Sent Done false
7495 2996 Acked Write 6 Done Sent Done false
7496 2997 Acked Write 2 Done Sent Done false
7497 2998 Acked Read 2 Done Sent Done false
7498 2999 Acked Write 8 Done Sent Done false
7499 3000 Acked Write 4 Done Sent Done false
7500 3001 Acked Read 5 Done Sent Done false
7501 3002 Acked Flush 0 Done Sent Done false
7502 3003 Acked Read 10 Done Sent Done false
7503 3004 Acked Read 2 Done Sent Done false
7504 3005 Acked Write 10 Done Sent Done false
7505 3006 Acked Read 4 Done Sent Done false
7506 3007 Acked Write 8 Done Sent Done false
7507 3008 Acked Write 3 Done Sent Done false
7508 3009 Acked Read 4 Done Sent Done false
7509 3010 Acked Write 8 Done Sent Done false
7510 3011 Acked Write 4 Done Sent Done false
7511 3012 Acked Write 1 Done Sent Done false
7512 3013 Acked Read 10 Done Sent Done false
7513 3014 Acked Write 5 Done Sent Done false
7514 3015 Acked Write 4 Done Sent Done false
7515 3016 Acked Write 10 Done Sent Done false
7516 3017 Acked Read 7 Done Sent Done false
7517 3018 Acked Read 2 Done Sent Done false
7518 3019 Acked Read 2 Done Sent Done false
7519 3020 Acked Read 10 Done Sent Done false
7520 3021 Acked Read 7 Done Sent Done false
7521 3022 Acked Read 6 Done Sent Done false
7522 3023 Acked Read 4 Done Sent Done false
7523 3024 Acked Read 5 Done Sent Done false
7524 3025 Acked Write 4 Done Sent Done false
7525 3026 Acked Read 4 Done Sent Done false
7526 3027 Acked Write 7 Done Sent Done false
7527 3028 Acked Read 1 Done Sent Done false
7528 3029 Acked Read 9 Done Sent Done false
7529 3030 Acked Read 7 Done Sent Done false
7530 3031 Acked Write 8 Done Sent Done false
7531 3032 Acked Write 5 Done Sent Done false
7532 3033 Acked Write 1 Done Sent Done false
7533 3034 Acked Read 5 Done Sent Done false
7534 3035 Acked Read 3 Done Sent Done false
7535 3036 Acked Read 8 Done Sent Done false
7536 3037 Acked Read 6 Done Sent Done false
7537 3038 Acked Read 3 Done Sent Done false
7538 3039 Acked Write 2 Done Sent Done false
7539 3040 Acked Read 10 Done Sent Done false
7540 3041 Acked Read 6 Done Sent Done false
7541 3042 Acked Write 10 Done Sent Done false
7542 3043 Acked Write 10 Done Sent Done false
7543 3044 Acked Read 4 Done Sent Done false
7544 3045 Acked Read 5 Done Sent Done false
7545 3046 Acked Write 4 Done Sent Done false
7546 3047 Acked Write 5 Done Sent Done false
7547 3048 Acked Write 5 Done Sent Done false
7548 3049 Acked Read 10 Done Sent Done false
7549 3050 Acked Write 2 Done Sent Done false
7550 3051 Acked Read 1 Done Sent Done false
7551 3052 Acked Read 3 Done Sent Done false
7552 3053 Acked Write 6 Done Sent Done false
7553 3054 Acked Read 6 Done Sent Done false
7554 3055 Acked Write 10 Done Sent Done false
7555 3056 Acked Read 10 Done Sent Done false
7556 3057 Acked Read 9 Done Sent Done false
7557 3058 Acked Read 10 Done Sent Done false
7558 3059 Acked Read 2 Done Sent Done false
7559 3060 Acked Write 3 Done Sent Done false
7560 3061 Acked Read 1 Done Sent Done false
7561 3062 Acked Write 9 Done Sent Done false
7562 3063 Acked Flush 0 Done Sent Done false
7563 3064 Acked Read 4 Done Sent Done false
7564 3065 Acked Write 10 Done Sent Done false
7565 3066 Acked Flush 0 Done Sent Done false
7566 3067 Acked Read 2 Done Sent Done false
7567 3068 Acked Write 7 Done Sent Done false
7568 3069 Acked Read 4 Done Sent Done false
7569 3070 Acked Write 6 Done Sent Done false
7570 3071 Acked Write 6 Done Sent Done false
7571 3072 Acked Read 9 Done Sent Done false
7572 3073 Acked Read 9 Done Sent Done false
7573 3074 Acked Read 9 Done Sent Done false
7574 3075 Acked Read 8 Done Sent Done false
7575 3076 Acked Read 6 Done Sent Done false
7576 3077 Acked Write 10 Done Sent Done false
7577 3078 Acked Write 1 Done Sent Done false
7578 3079 Acked Read 9 Done Sent Done false
7579 3080 Acked Flush 0 Done Sent Done false
7580 3081 Acked Flush 0 Done Sent Done false
7581 3082 Acked Write 8 Done Sent Done false
7582 3083 Acked Read 4 Done Sent Done false
7583 3084 Acked Write 6 Done Sent Done false
7584 3085 Acked Write 5 Done Sent Done false
7585 3086 Acked Read 7 Done Sent Done false
7586 3087 Acked Read 8 Done Sent Done false
7587 3088 Acked Read 8 Done Sent Done false
7588 3089 Acked Write 7 Done Sent Done false
7589 3090 Acked Read 9 Done Sent Done false
7590 3091 Acked Write 7 Done Sent Done false
7591 3092 Acked Read 7 Done Sent Done false
7592 3093 Acked Write 1 Done Sent Done false
7593 3094 Acked Write 4 Done Sent Done false
7594 3095 Acked Read 3 Done Sent Done false
7595 3096 Acked Flush 0 Done Sent Done false
7596 3097 Acked Read 9 Done Sent Done false
7597 3098 Acked Write 3 Done Sent Done false
7598 3099 Acked Read 3 Done Sent Done false
7599 3100 Acked Read 3 Done Sent Done false
7600 3101 Acked Flush 0 Done Sent Done false
7601 3102 Acked Flush 0 Done Sent Done false
7602 3103 Acked Read 3 Done Sent Done false
7603 3104 Acked Write 4 Done Sent Done false
7604 3105 Acked Write 10 Done Sent Done false
7605 3106 Acked Flush 0 Done Sent Done false
7606 3107 Acked Read 5 Done Sent Done false
7607 3108 Acked Read 1 Done Sent Done false
7608 3109 Acked Read 1 Done Sent Done false
7609 3110 Acked Flush 0 Done Sent Done false
7610 3111 Acked Read 3 Done Sent Done false
7611 3112 Acked Flush 0 Done Sent Done false
7612 STATES DS:0 DS:1 DS:2 TOTAL
7613 Sent 0 302 0 302
7614 Done 302 0 302 604
7615 Skipped 0 0 0 0
7616 Error 0 0 0 0
7617 Last Flush: 3112 2810 3112
7618 Downstairs last five completed: 2810 2809 2808 2807 2806
7619 Upstairs last five completed: 3112 3111 3110 3109 3108
7620 CLIENT: Up:0 ds:302 act:2
76212025-01-21T15:48:44.802ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76222025-01-21T15:48:44.803ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47688) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76232025-01-21T15:48:44.804ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76242025-01-21T15:48:44.804ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76252025-01-21T15:48:44.804ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76262025-01-21T15:48:44.804ZINFOcrucible: send last flush ID to this DS: 2810 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76272025-01-21T15:48:44.804ZINFOcrucible: Replied this last flush ID: 2810 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76282025-01-21T15:48:44.804ZINFOcrucible: [1] client re-new 302 jobs since flush 2810 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76292025-01-21T15:48:44.805ZINFOcrucible: [1] Replayed 302 jobs since flush: 2810 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
7630 ----------------------------------------------------------------
7631 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7632 Downstairs last five completed: 3112 3111 3110 3109 3108
7633 Upstairs last five completed: 3112 3111 3110 3109 3108
7634 CLIENT: Up:0 ds:0 act:3
7635 Replay: All jobs finished
7636 Main task has work to do, go find it
7637 got dsc Stop(2)
7638 stop 2
7639 [8830] Got stop action so:false kr:false
7640 Jan 21 15:48:47.052 INFO request completed, latency_us: 375, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 9a72a4be-9e23-4f58-864a-a2062e6a0028, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
76412025-01-21T15:48:47.053ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
76422025-01-21T15:48:47.053ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
7643 Jan 21 15:48:47.053{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:48:47.053422511Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
7644 INFO request completed, latency_us: 141, response_code: 200, uri: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:48:47.05344846Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
7645 /state/cid/2, method: GET, req_id: 6b9d1408-1cf8-4ffe-9f80-e2d79a00a989, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
7646 [8830] Exited with: ExitStatus(unix_wait_status(9))
7647 [8830][2] reports Exit
7648 Jan 21 15:48:51.056 INFO request completed, latency_us: 203, response_code: 200, uri: /state/cid/2, method: GET, req_id: 3489317d-b1b3-4213-b2ca-9f191ae99c3c, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
7649 001/300 Read block 3182 len 16384
7650 002/300 Write block 410 len 8192 data: 3 3
7651 003/300 Flush
7652 004/300 Write block 1782 len 32768 data: 5 5 5 5 5 4 3 2
7653 005/300 Read block 3309 len 8192
7654 006/300 Read block 2004 len 40960
7655 007/300 Read block 1771 len 40960
7656 008/300 Read block 285 len 40960
7657 009/300 Read block 4915 len 32768
7658 010/300 Flush
7659 011/300 Write block 2009 len 32768 data: 2 2 2 2 3 3 3 3
7660 012/300 Read block 982 len 28672
7661 013/300 Read block 2614 len 32768
7662 014/300 Write block 2431 len 4096 data: 2
7663 015/300 Flush
7664 016/300 Read block 2244 len 4096
7665 017/300 Write block 1979 len 28672 data: 3 4 4 3 3 3 3
7666 018/300 Read block 620 len 20480
7667 019/300 Write block 746 len 20480 data: 3 3 3 3 3
7668 020/300 Write block 3029 len 32768 data: 4 4 4 4 4 5 4 4
7669 021/300 Read block 2545 len 8192
7670 022/300 Read block 2898 len 12288
7671 023/300 Read block 1425 len 8192
7672 024/300 Flush
7673 025/300 Write block 2839 len 16384 data: 4 3 3 3
7674 026/300 Flush
7675 027/300 Write block 3567 len 20480 data: 2 2 2 2 2
7676 028/300 Write block 512 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7677 029/300 Read block 4617 len 36864
7678 030/300 Read block 1354 len 4096
7679 031/300 Read block 1853 len 36864
7680 032/300 Read block 4297 len 20480
7681 033/300 Write block 3248 len 4096 data: 4
7682 034/300 Write block 646 len 24576 data: 3 3 3 3 3 3
7683 035/300 Read block 2833 len 28672
7684 036/300 Write block 3090 len 20480 data: 3 2 2 3 4
7685 037/300 Read block 4292 len 16384
7686 038/300 Read block 1836 len 28672
7687 039/300 Read block 390 len 12288
7688 040/300 Read block 4113 len 20480
7689 041/300 Read block 571 len 12288
7690 042/300 Write block 3580 len 8192 data: 3 3
7691 043/300 Write block 1340 len 24576 data: 4 5 4 3 3 3
7692 044/300 Write block 4569 len 32768 data: 3 3 3 4 5 4 4 4
7693 045/300 Read block 4111 len 40960
7694 046/300 Read block 1730 len 32768
7695 047/300 Read block 3116 len 40960
7696 048/300 Read block 1733 len 12288
7697 049/300 Read block 1120 len 36864
7698 050/300 Flush
7699 051/300 Read block 1580 len 28672
7700 052/300 Read block 4096 len 8192
7701 053/300 Read block 945 len 20480
7702 054/300 Read block 1732 len 12288
7703 055/300 Read block 2953 len 32768
7704 056/300 Write block 1045 len 4096 data: 4
7705 057/300 Read block 3943 len 36864
7706 058/300 Read block 4479 len 16384
7707 059/300 Read block 3031 len 16384
7708 060/300 Read block 759 len 8192
7709 061/300 Write block 1839 len 40960 data: 3 3 3 3 4 4 3 3 4 4
7710 062/300 Read block 555 len 40960
7711 063/300 Write block 4331 len 24576 data: 3 3 2 2 2 2
7712 064/300 Write block 3992 len 16384 data: 2 2 3 3
7713 065/300 Read block 2333 len 24576
7714 066/300 Read block 4370 len 20480
7715 067/300 Write block 2899 len 24576 data: 3 3 3 3 3 3
7716 068/300 Read block 1134 len 28672
7717 069/300 Read block 2952 len 8192
7718 070/300 Read block 4952 len 36864
7719 071/300 Write block 4137 len 28672 data: 2 2 2 2 2 2 2
7720 072/300 Flush
7721 073/300 Read block 2108 len 24576
7722 074/300 Read block 477 len 8192
7723 075/300 Write block 55 len 32768 data: 2 2 2 2 2 2 2 2
7724 076/300 Write block 1153 len 36864 data: 3 3 3 4 5 5 5 4 3
7725 077/300 Write block 18 len 12288 data: 3 3 3
7726 078/300 Flush
7727 079/300 Read block 3172 len 40960
7728 080/300 Read block 3459 len 12288
7729 081/300 Read block 1703 len 12288
7730 082/300 Read block 1842 len 8192
7731 083/300 Read block 4766 len 36864
7732 084/300 Read block 4993 len 20480
7733 085/300 Read block 2212 len 40960
7734 086/300 Read block 439 len 24576
7735 087/300 Read block 3999 len 40960
7736 088/300 Read block 4243 len 16384
7737 089/300 Write block 1645 len 28672 data: 2 2 2 2 2 2 2
7738 090/300 Read block 1789 len 40960
7739 091/300 Read block 1299 len 40960
7740 092/300 Read block 3913 len 16384
7741 093/300 Read block 2791 len 20480
7742 094/300 Write block 888 len 36864 data: 4 5 5 4 3 3 3 3 2
7743 095/300 Write block 2899 len 16384 data: 4 4 4 4
7744 096/300 Flush
7745 097/300 Write block 1669 len 32768 data: 2 2 2 2 2 2 2 2
7746 098/300 Write block 715 len 32768 data: 2 2 2 2 2 2 2 3
7747 099/300 Write block 1167 len 36864 data: 2 2 2 2 2 2 2 2 2
7748 100/300 Write block 646 len 32768 data: 4 4 4 4 4 4 3 3
7749 101/300 Read block 4018 len 4096
7750 102/300 Read block 3015 len 16384
7751 103/300 Write block 826 len 24576 data: 3 2 2 2 3 3
7752 104/300 Flush
7753 105/300 Write block 2117 len 32768 data: 2 2 2 2 2 2 2 2
7754 106/300 Read block 1375 len 20480
7755 107/300 Write block 3477 len 28672 data: 4 2 2 2 2 2 2
7756 108/300 Write block 2748 len 40960 data: 2 2 2 2 2 3 3 3 3 3
7757 109/300 Write block 4985 len 20480 data: 5 6 8 8 7
7758 110/300 Write block 2921 len 12288 data: 2 2 3
7759 111/300 Read block 2445 len 4096
7760 112/300 Write block 1900 len 4096 data: 4
7761 113/300 Read block 4302 len 20480
7762 114/300 Write block 626 len 24576 data: 3 3 2 2 3 3
7763 115/300 Flush
7764 116/300 Read block 1403 len 4096
7765 117/300 Read block 3884 len 20480
7766 118/300 Read block 3161 len 32768
7767 119/300 Write block 2132 len 32768 data: 2 2 2 2 2 2 2 3
7768 120/300 Read block 463 len 12288
7769 121/300 Write block 2042 len 16384 data: 4 4 4 4
7770 122/300 Write block 4094 len 28672 data: 2 2 2 2 2 2 3
7771 123/300 Read block 1047 len 20480
7772 124/300 Read block 3864 len 8192
7773 125/300 Read block 2158 len 40960
7774 126/300 Read block 1684 len 4096
7775 127/300 Write block 1552 len 40960 data: 2 2 2 2 2 2 3 3 3 3
7776 128/300 Read block 4928 len 4096
7777 129/300 Write block 1456 len 20480 data: 2 2 2 2 2
7778 130/300 Write block 4957 len 28672 data: 4 4 3 3 2 2 2
7779 131/300 Read block 2840 len 24576
7780 132/300 Write block 1864 len 16384 data: 3 3 2 2
7781 133/300 Write block 2661 len 40960 data: 2 2 2 2 2 2 2 4 4 5
7782 134/300 Write block 404 len 20480 data: 3 3 3 4 4
7783 135/300 Write block 4446 len 4096 data: 2
7784 136/300 Write block 1362 len 8192 data: 3 3
7785 137/300 Write block 427 len 8192 data: 2 2
7786 138/300 Read block 2897 len 36864
7787 139/300 Read block 4339 len 8192
7788 140/300 Read block 4613 len 4096
7789 141/300 Write block 993 len 36864 data: 2 2 3 3 3 3 4 4 4
7790 142/300 Write block 3081 len 36864 data: 4 4 4 3 4 4 3 3 3
7791 143/300 Flush
7792 144/300 Read block 4392 len 4096
7793 145/300 Write block 892 len 8192 data: 4 4
7794 146/300 Read block 3155 len 24576
7795 147/300 Write block 3152 len 8192 data: 4 3
7796 148/300 Write block 2417 len 36864 data: 2 2 3 3 2 3 3 4 4
7797 149/300 Read block 3274 len 4096
7798 150/300 Read block 4387 len 32768
7799 151/300 Write block 552 len 4096 data: 2
7800 152/300 Read block 1036 len 4096
7801 153/300 Read block 4319 len 12288
7802 154/300 Write block 4315 len 8192 data: 3 3
7803 155/300 Read block 933 len 40960
7804 156/300 Read block 2929 len 24576
7805 157/300 Read block 945 len 24576
7806 158/300 Write block 3835 len 4096 data: 2
7807 159/300 Read block 348 len 16384
7808 160/300 Read block 1506 len 28672
7809 161/300 Read block 4711 len 24576
7810 162/300 Write block 666 len 8192 data: 4 3
7811 163/300 Read block 2453 len 40960
7812 164/300 Write block 1978 len 40960 data: 3 4 5 5 4 4 4 4 3 3
7813 165/300 Read block 4261 len 24576
7814 166/300 Read block 1180 len 12288
7815 167/300 Write block 858 len 40960 data: 3 3 4 5 3 3 3 3 3 3
7816 168/300 Write block 136 len 20480 data: 4 3 2 2 2
7817 169/300 Write block 1381 len 36864 data: 3 4 3 2 2 2 2 2 2
7818 170/300 Read block 2492 len 20480
7819 171/300 Read block 1502 len 12288
7820 172/300 Read block 3612 len 36864
7821 173/300 Write block 2634 len 24576 data: 5 5 5 4 3 3
7822 174/300 Write block 4253 len 40960 data: 3 3 3 3 2 2 2 2 2 2
7823 175/300 Read block 4023 len 40960
7824 176/300 Read block 3247 len 32768
7825 177/300 Write block 435 len 28672 data: 3 3 3 4 4 3 2
7826 178/300 Write block 1897 len 4096 data: 3
7827 179/300 Write block 3369 len 28672 data: 4 2 2 2 2 2 2
7828 180/300 Write block 2333 len 32768 data: 3 3 2 2 2 2 2 2
7829 181/300 Read block 4154 len 28672
7830 182/300 Write block 4216 len 20480 data: 3 5 5 5 5
7831 183/300 Read block 1751 len 24576
7832 184/300 Flush
7833 185/300 Read block 4824 len 16384
7834 186/300 Write block 1342 len 12288 data: 5 4 4
7835 187/300 Read block 1732 len 8192
7836 188/300 Read block 3017 len 40960
7837 189/300 Write block 3517 len 12288 data: 2 2 3
7838 190/300 Write block 2015 len 16384 data: 4 4 4 5
7839 191/300 Read block 1430 len 8192
7840 192/300 Read block 3294 len 32768
7841 193/300 Read block 181 len 24576
7842 194/300 Write block 4891 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7843 195/300 Write block 283 len 32768 data: 3 3 3 3 2 2 2 2
7844 196/300 Read block 2628 len 36864
7845 197/300 Flush
7846 198/300 Flush
7847 199/300 Read block 371 len 24576
7848 200/300 Write block 3389 len 20480 data: 3 3 3 2 2
7849 201/300 Write block 37 len 36864 data: 4 4 3 3 3 4 4 4 5
7850 202/300 Write block 2034 len 12288 data: 5 6 6
7851 203/300 Write block 1156 len 28672 data: 5 6 6 6 5 4 2
7852 204/300 Write block 2712 len 24576 data: 4 4 3 2 2 2
7853 205/300 Read block 3130 len 40960
7854 206/300 Flush
7855 207/300 Read block 2655 len 32768
7856 208/300 Write block 4538 len 16384 data: 3 3 3 2
7857 209/300 Flush
7858 210/300 Read block 2532 len 12288
7859 211/300 Write block 4408 len 36864 data: 4 4 5 5 3 3 3 3 3
7860 212/300 Write block 2865 len 4096 data: 2
7861 213/300 Read block 3110 len 28672
7862 214/300 Write block 3703 len 12288 data: 4 3 2
7863 215/300 Read block 492 len 8192
7864 216/300 Read block 1336 len 24576
7865 217/300 Flush
7866 218/300 Flush
7867 219/300 Flush
7868 220/300 Write block 2619 len 4096 data: 2
7869 221/300 Read block 1178 len 36864
7870 222/300 Flush
7871 223/300 Write block 4671 len 16384 data: 4 4 4 4
7872 224/300 Read block 4061 len 12288
7873 225/300 Read block 3344 len 28672
7874 226/300 Write block 4226 len 28672 data: 3 3 3 4 4 5 5
7875 227/300 Write block 693 len 40960 data: 3 3 4 5 4 4 3 3 2 2
7876 228/300 Write block 4251 len 28672 data: 3 3 4 4 4 4 3
7877 229/300 Write block 1650 len 16384 data: 3 3 2 2
7878 230/300 Write block 3607 len 16384 data: 2 2 2 2
7879 231/300 Read block 131 len 28672
7880 232/300 Write block 2281 len 12288 data: 2 2 2
7881 233/300 Read block 2781 len 20480
7882 234/300 Write block 3855 len 32768 data: 5 5 5 5 3 3 3 2
7883 235/300 Read block 1377 len 24576
7884 236/300 Read block 867 len 12288
7885 237/300 Write block 4541 len 24576 data: 3 2 2 2 2 2
7886 238/300 Read block 1078 len 16384
7887 239/300 Write block 2324 len 24576 data: 2 2 2 3 3 3
7888 240/300 Read block 3174 len 8192
7889 241/300 Read block 266 len 36864
7890 242/300 Read block 4009 len 28672
7891 243/300 Flush
7892 244/300 Read block 3642 len 32768
7893 245/300 Read block 4405 len 40960
7894 246/300 Write block 1144 len 20480 data: 4 3 3 3 3
7895 247/300 Read block 3539 len 24576
7896 248/300 Write block 2203 len 28672 data: 2 2 2 2 2 2 2
7897 249/300 Read block 2986 len 32768
7898 250/300 Write block 2314 len 40960 data: 3 3 3 3 3 2 2 2 2 2
7899 251/300 Write block 2745 len 24576 data: 2 2 2 3 3 3
7900 252/300 Write block 4478 len 4096 data: 2
7901 253/300 Read block 2485 len 16384
7902 254/300 Read block 734 len 16384
7903 255/300 Read block 36 len 28672
7904 256/300 Write block 114 len 28672 data: 3 2 2 2 2 2 2
7905 257/300 Write block 209 len 24576 data: 4 3 3 3 4 3
7906 258/300 Read block 4123 len 12288
7907 259/300 Write block 3116 len 24576 data: 3 3 2 2 2 2
7908 260/300 Write block 3468 len 12288 data: 3 3 3
7909 261/300 Write block 1469 len 12288 data: 4 4 4
7910 262/300 Read block 1396 len 36864
7911 263/300 Read block 1809 len 8192
7912 264/300 Read block 518 len 40960
7913 265/300 Write block 2872 len 36864 data: 2 3 3 3 3 3 3 3 3
7914 266/300 Read block 4359 len 20480
7915 267/300 Read block 1501 len 8192
7916 268/300 Read block 1853 len 24576
7917 269/300 Write block 3579 len 32768 data: 3 4 4 3 2 2 2 2
7918 270/300 Read block 4696 len 28672
7919 271/300 Read block 3448 len 16384
7920 272/300 Read block 3550 len 36864
7921 273/300 Write block 1591 len 4096 data: 4
7922 274/300 Flush
7923 275/300 Write block 3876 len 28672 data: 3 4 4 4 3 3 3
7924 276/300 Write block 1984 len 40960 data: 5 5 4 4 3 3 2 2 2 3
7925 277/300 Read block 872 len 8192
7926 278/300 Read block 3160 len 8192
7927 279/300 Read block 1648 len 28672
7928 280/300 Read block 4025 len 40960
7929 281/300 Flush
7930 282/300 Read block 3229 len 36864
7931 283/300 Write block 1311 len 24576 data: 3 4 4 4 4 4
7932 284/300 Write block 2906 len 8192 data: 2 3
7933 285/300 Write block 4707 len 12288 data: 2 2 3
7934 286/300 Read block 3489 len 8192
7935 287/300 Read block 855 len 20480
7936 288/300 Read block 2435 len 16384
7937 289/300 Write block 1196 len 24576 data: 5 4 4 4 5 4
7938 290/300 Write block 3915 len 24576 data: 4 4 4 3 3 2
7939 291/300 Read block 3200 len 20480
7940 292/300 Flush
7941 293/300 Read block 2361 len 32768
7942 294/300 Write block 1578 len 40960 data: 5 5 5 5 4 4 4 3 2 2
7943 295/300 Write block 4572 len 16384 data: 5 6 5 5
7944 296/300 Read block 2994 len 4096
7945 297/300 Read block 1556 len 12288
7946 298/300 Write block 3526 len 28672 data: 3 3 3 2 2 2 2
7947 299/300 Write block 4442 len 36864 data: 4 3 3 2 3 2 2 2 2
7948 300/300 Read block 498 len 12288
7949 301/300 Read block 2573 len 12288
7950 Main task has work to do, go find it
7951 got dsc Start(2)
7952 start 2
7953 Jan 21 15:48:51.298 INFO request completed, latency_us: 222, response_code: 204, uri: /start/cid/2, method: POST, req_id: 9e396f90-2482-4b96-b421-a9ee32b5f082, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
7954 Replay: started 2, returned:Ok(())
7955 ----------------------------------------------------------------
7956 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7957 States: Active Active Offline
7958 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7959 3113 Acked Read 4 Done Done Sent false
7960 3114 Acked Write 2 Done Done Sent false
7961 3115 Acked Flush 0 Done Done Sent false
7962 3116 Acked Write 8 Done Done Sent false
7963 3117 Acked Read 2 Done Done Sent false
7964 3118 Acked Read 10 Done Done Sent false
7965 3119 Acked Read 10 Done Done Sent false
7966 3120 Acked Read 10 Done Done Sent false
7967 3121 Acked Read 8 Done Done Sent false
7968 3122 Acked Flush 0 Done Done Sent false
7969 3123 Acked Write 8 Done Done Sent false
7970 3124 Acked Read 7 Done Done Sent false
7971 3125 Acked Read 8 Done Done Sent false
7972 3126 Acked Write 1 Done Done Sent false
7973 3127 Acked Flush 0 Done Done Sent false
7974 3128 Acked Read 1 Done Done Sent false
7975 3129 Acked Write 7 Done Done Sent false
7976 3130 Acked Read 5 Done Done Sent false
7977 3131 Acked Write 5 Done Done Sent false
7978 3132 Acked Write 8 Done Done Sent false
7979 3133 Acked Read 2 Done Done Sent false
7980 3134 Acked Read 3 Done Done Sent false
7981 3135 Acked Read 2 Done Done Sent false
7982 3136 Acked Flush 0 Done Done Sent false
7983 3137 Acked Write 4 Done Done Sent false
7984 3138 Acked Flush 0 Done Done Sent false
7985 3139 Acked Write 5 Done Done Sent false
7986 3140 Acked Write 10 Done Done Sent false
7987 3141 Acked Read 9 Done Done Sent false
7988 3142 Acked Read 1 Done Done Sent false
7989 3143 Acked Read 9 Done Done Sent false
7990 3144 Acked Read 5 Done Done Sent false
7991 3145 Acked Write 1 Done Done Sent false
7992 3146 Acked Write 6 Done Done Sent false
7993 3147 Acked Read 7 Done Done Sent false
7994 3148 Acked Write 5 Done Done Sent false
7995 3149 Acked Read 4 Done Done Sent false
7996 3150 Acked Read 7 Done Done Sent false
7997 3151 Acked Read 3 Done Done Sent false
7998 3152 Acked Read 5 Done Done Sent false
7999 3153 Acked Read 3 Done Done Sent false
8000 3154 Acked Write 2 Done Done Sent false
8001 3155 Acked Write 6 Done Done Sent false
8002 3156 Acked Write 8 Done Done Sent false
8003 3157 Acked Read 10 Done Done Sent false
8004 3158 Acked Read 8 Done Done Sent false
8005 3159 Acked Read 10 Done Done Sent false
8006 3160 Acked Read 3 Done Done Sent false
8007 3161 Acked Read 9 Done Done Sent false
8008 3162 Acked Flush 0 Done Done Sent false
8009 3163 Acked Read 7 Done Done Sent false
8010 3164 Acked Read 2 Done Done Sent false
8011 3165 Acked Read 5 Done Done Sent false
8012 3166 Acked Read 3 Done Done Sent false
8013 3167 Acked Read 8 Done Done Sent false
8014 3168 Acked Write 1 Done Done Sent false
8015 3169 Acked Read 9 Done Done Sent false
8016 3170 Acked Read 4 Done Done Sent false
8017 3171 Acked Read 4 Done Done Sent false
8018 3172 Acked Read 2 Done Done Sent false
8019 3173 Acked Write 10 Done Done Sent false
8020 3174 Acked Read 10 Done Done Sent false
8021 3175 Acked Write 6 Done Done Sent false
8022 3176 Acked Write 4 Done Done Sent false
8023 3177 Acked Read 6 Done Done Sent false
8024 3178 Acked Read 5 Done Done Sent false
8025 3179 Acked Write 6 Done Done Sent false
8026 3180 Acked Read 7 Done Done Sent false
8027 3181 Acked Read 2 Done Done Sent false
8028 3182 Acked Read 9 Done Done Sent false
8029 3183 Acked Write 7 Done Done Sent false
8030 3184 Acked Flush 0 Done Done Sent false
8031 3185 Acked Read 6 Done Done Sent false
8032 3186 Acked Read 2 Done Done Sent false
8033 3187 Acked Write 8 Done Done Sent false
8034 3188 Acked Write 9 Done Done Sent false
8035 3189 Acked Write 3 Done Done Sent false
8036 3190 Acked Flush 0 Done Done Sent false
8037 3191 Acked Read 10 Done Done Sent false
8038 3192 Acked Read 3 Done Done Sent false
8039 3193 Acked Read 3 Done Done Sent false
8040 3194 Acked Read 2 Done Done Sent false
8041 3195 Acked Read 9 Done Done Sent false
8042 3196 Acked Read 5 Done Done Sent false
8043 3197 Acked Read 10 Done Done Sent false
8044 3198 Acked Read 6 Done Done Sent false
8045 3199 Acked Read 10 Done Done Sent false
8046 3200 Acked Read 4 Done Done Sent false
8047 3201 Acked Write 7 Done Done Sent false
8048 3202 Acked Read 10 Done Done Sent false
8049 3203 Acked Read 10 Done Done Sent false
8050 3204 Acked Read 4 Done Done Sent false
8051 3205 Acked Read 5 Done Done Sent false
8052 3206 Acked Write 9 Done Done Sent false
8053 3207 Acked Write 4 Done Done Sent false
8054 3208 Acked Flush 0 Done Done Sent false
8055 3209 Acked Write 8 Done Done Sent false
8056 3210 Acked Write 8 Done Done Sent false
8057 3211 Acked Write 9 Done Done Sent false
8058 3212 Acked Write 8 Done Done Sent false
8059 3213 Acked Read 1 Done Done Sent false
8060 3214 Acked Read 4 Done Done Sent false
8061 3215 Acked Write 6 Done Done Sent false
8062 3216 Acked Flush 0 Done Done Sent false
8063 3217 Acked Write 8 Done Done Sent false
8064 3218 Acked Read 5 Done Done Sent false
8065 3219 Acked Write 7 Done Done Sent false
8066 3220 Acked Write 10 Done Done Sent false
8067 3221 Acked Write 5 Done Done Sent false
8068 3222 Acked Write 3 Done Done Sent false
8069 3223 Acked Read 1 Done Done Sent false
8070 3224 Acked Write 1 Done Done Sent false
8071 3225 Acked Read 5 Done Done Sent false
8072 3226 Acked Write 6 Done Done Sent false
8073 3227 Acked Flush 0 Done Done Sent false
8074 3228 Acked Read 1 Done Done Sent false
8075 3229 Acked Read 5 Done Done Sent false
8076 3230 Acked Read 8 Done Done Sent false
8077 3231 Acked Write 8 Done Done Sent false
8078 3232 Acked Read 3 Done Done Sent false
8079 3233 Acked Write 4 Done Done Sent false
8080 3234 Acked Write 7 Done Done Sent false
8081 3235 Acked Read 5 Done Done Sent false
8082 3236 Acked Read 2 Done Done Sent false
8083 3237 Acked Read 10 Done Done Sent false
8084 3238 Acked Read 1 Done Done Sent false
8085 3239 Acked Write 10 Done Done Sent false
8086 3240 Acked Read 1 Done Done Sent false
8087 3241 Acked Write 5 Done Done Sent false
8088 3242 Acked Write 7 Done Done Sent false
8089 3243 Acked Read 6 Done Done Sent false
8090 3244 Acked Write 4 Done Done Sent false
8091 3245 Acked Write 10 Done Done Sent false
8092 3246 Acked Write 5 Done Done Sent false
8093 3247 Acked Write 1 Done Done Sent false
8094 3248 Acked Write 2 Done Done Sent false
8095 3249 Acked Write 2 Done Done Sent false
8096 3250 Acked Read 9 Done Done Sent false
8097 3251 Acked Read 2 Done Done Sent false
8098 3252 Acked Read 1 Done Done Sent false
8099 3253 Acked Write 9 Done Done Sent false
8100 3254 Acked Write 9 Done Done Sent false
8101 3255 Acked Flush 0 Done Done Sent false
8102 3256 Acked Read 1 Done Done Sent false
8103 3257 Acked Write 2 Done Done Sent false
8104 3258 Acked Read 6 Done Done Sent false
8105 3259 Acked Write 2 Done Done Sent false
8106 3260 Acked Write 9 Done Done Sent false
8107 3261 Acked Read 1 Done Done Sent false
8108 3262 Acked Read 8 Done Done Sent false
8109 3263 Acked Write 1 Done Done Sent false
8110 3264 Acked Read 1 Done Done Sent false
8111 3265 Acked Read 3 Done Done Sent false
8112 3266 Acked Write 2 Done Done Sent false
8113 3267 Acked Read 10 Done Done Sent false
8114 3268 Acked Read 6 Done Done Sent false
8115 3269 Acked Read 6 Done Done Sent false
8116 3270 Acked Write 1 Done Done Sent false
8117 3271 Acked Read 4 Done Done Sent false
8118 3272 Acked Read 7 Done Done Sent false
8119 3273 Acked Read 6 Done Done Sent false
8120 3274 Acked Write 2 Done Done Sent false
8121 3275 Acked Read 10 Done Done Sent false
8122 3276 Acked Write 10 Done Done Sent false
8123 3277 Acked Read 6 Done Done Sent false
8124 3278 Acked Read 3 Done Done Sent false
8125 3279 Acked Write 10 Done Done Sent false
8126 3280 Acked Write 5 Done Done Sent false
8127 3281 Acked Write 9 Done Done Sent false
8128 3282 Acked Read 5 Done Done Sent false
8129 3283 Acked Read 3 Done Done Sent false
8130 3284 Acked Read 9 Done Done Sent false
8131 3285 Acked Write 6 Done Done Sent false
8132 3286 Acked Write 10 Done Done Sent false
8133 3287 Acked Read 10 Done Done Sent false
8134 3288 Acked Read 8 Done Done Sent false
8135 3289 Acked Write 7 Done Done Sent false
8136 3290 Acked Write 1 Done Done Sent false
8137 3291 Acked Write 7 Done Done Sent false
8138 3292 Acked Write 8 Done Done Sent false
8139 3293 Acked Read 7 Done Done Sent false
8140 3294 Acked Write 5 Done Done Sent false
8141 3295 Acked Read 6 Done Done Sent false
8142 3296 Acked Flush 0 Done Done Sent false
8143 3297 Acked Read 4 Done Done Sent false
8144 3298 Acked Write 3 Done Done Sent false
8145 3299 Acked Read 2 Done Done Sent false
8146 3300 Acked Read 10 Done Done Sent false
8147 3301 Acked Write 3 Done Done Sent false
8148 3302 Acked Write 4 Done Done Sent false
8149 3303 Acked Read 2 Done Done Sent false
8150 3304 Acked Read 8 Done Done Sent false
8151 3305 Acked Read 6 Done Done Sent false
8152 3306 Acked Write 10 Done Done Sent false
8153 3307 Acked Write 8 Done Done Sent false
8154 3308 Acked Read 9 Done Done Sent false
8155 3309 Acked Flush 0 Done Done Sent false
8156 3310 Acked Flush 0 Done Done Sent false
8157 3311 Acked Read 6 Done Done Sent false
8158 3312 Acked Write 5 Done Done Sent false
8159 3313 Acked Write 9 Done Done Sent false
8160 3314 Acked Write 3 Done Done Sent false
8161 3315 Acked Write 7 Done Done Sent false
8162 3316 Acked Write 6 Done Done Sent false
8163 3317 Acked Read 10 Done Done Sent false
8164 3318 Acked Flush 0 Done Done Sent false
8165 3319 Acked Read 8 Done Done Sent false
8166 3320 Acked Write 4 Done Done Sent false
8167 3321 Acked Flush 0 Done Done Sent false
8168 3322 Acked Read 3 Done Done Sent false
8169 3323 Acked Write 9 Done Done Sent false
8170 3324 Acked Write 1 Done Done Sent false
8171 3325 Acked Read 7 Done Done Sent false
8172 3326 Acked Write 3 Done Done Sent false
8173 3327 Acked Read 2 Done Done Sent false
8174 3328 Acked Read 6 Done Done Sent false
8175 3329 Acked Flush 0 Done Done Sent false
8176 3330 Acked Flush 0 Done Done Sent false
8177 3331 Acked Flush 0 Done Done Sent false
8178 3332 Acked Write 1 Done Done Sent false
8179 3333 Acked Read 9 Done Done Sent false
8180 3334 Acked Flush 0 Done Done Sent false
8181 3335 Acked Write 4 Done Done Sent false
8182 3336 Acked Read 3 Done Done Sent false
8183 3337 Acked Read 7 Done Done Sent false
8184 3338 Acked Write 7 Done Done Sent false
8185 3339 Acked Write 10 Done Done Sent false
8186 3340 Acked Write 7 Done Done Sent false
8187 3341 Acked Write 4 Done Done Sent false
8188 3342 Acked Write 4 Done Done Sent false
8189 3343 Acked Read 7 Done Done Sent false
8190 3344 Acked Write 3 Done Done Sent false
8191 3345 Acked Read 5 Done Done Sent false
8192 3346 Acked Write 8 Done Done Sent false
8193 3347 Acked Read 6 Done Done Sent false
8194 3348 Acked Read 3 Done Done Sent false
8195 3349 Acked Write 6 Done Done Sent false
8196 3350 Acked Read 4 Done Done Sent false
8197 3351 Acked Write 6 Done Done Sent false
8198 3352 Acked Read 2 Done Done Sent false
8199 3353 Acked Read 9 Done Done Sent false
8200 3354 Acked Read 7 Done Done Sent false
8201 3355 Acked Flush 0 Done Done Sent false
8202 3356 Acked Read 8 Done Done Sent false
8203 3357 Acked Read 10 Done Done Sent false
8204 3358 Acked Write 5 Done Done Sent false
8205 3359 Acked Read 6 Done Done Sent false
8206 3360 Acked Write 7 Done Done Sent false
8207 3361 Acked Read 8 Done Done Sent false
8208 3362 Acked Write 10 Done Done Sent false
8209 3363 Acked Write 6 Done Done Sent false
8210 3364 Acked Write 1 Done Done Sent false
8211 3365 Acked Read 4 Done Done Sent false
8212 3366 Acked Read 4 Done Done Sent false
8213 3367 Acked Read 7 Done Done Sent false
8214 3368 Acked Write 7 Done Done Sent false
8215 3369 Acked Write 6 Done Done Sent false
8216 3370 Acked Read 3 Done Done Sent false
8217 3371 Acked Write 6 Done Done Sent false
8218 3372 Acked Write 3 Done Done Sent false
8219 3373 Acked Write 3 Done Done Sent false
8220 3374 Acked Read 9 Done Done Sent false
8221 3375 Acked Read 2 Done Done Sent false
8222 3376 Acked Read 10 Done Done Sent false
8223 3377 Acked Write 9 Done Done Sent false
8224 3378 Acked Read 5 Done Done Sent false
8225 3379 Acked Read 2 Done Done Sent false
8226 3380 Acked Read 6 Done Done Sent false
8227 3381 Acked Write 8 Done Done Sent false
8228 3382 Acked Read 7 Done Done Sent false
8229 3383 Acked Read 4 Done Done Sent false
8230 3384 Acked Read 9 Done Done Sent false
8231 3385 Acked Write 1 Done Done Sent false
8232 3386 Acked Flush 0 Done Done Sent false
8233 3387 Acked Write 7 Done Done Sent false
8234 3388 Acked Write 10 Done Done Sent false
8235 3389 Acked Read 2 Done Done Sent false
8236 3390 Acked Read 2 Done Done Sent false
8237 3391 Acked Read 7 Done Done Sent false
8238 3392 Acked Read 10 Done Done Sent false
8239 3393 Acked Flush 0 Done Done Sent false
8240 3394 Acked Read 9 Done Done Sent false
8241 3395 Acked Write 6 Done Done Sent false
8242 3396 Acked Write 2 Done Done Sent false
8243 3397 Acked Write 3 Done Done Sent false
8244 3398 Acked Read 2 Done Done Sent false
8245 3399 Acked Read 5 Done Done Sent false
8246 3400 Acked Read 4 Done Done Sent false
8247 3401 Acked Write 6 Done Done Sent false
8248 3402 Acked Write 6 Done Done Sent false
8249 3403 Acked Read 5 Done Done Sent false
8250 3404 Acked Flush 0 Done Done Sent false
8251 3405 Acked Read 8 Done Done Sent false
8252 3406 Acked Write 10 Done Done Sent false
8253 3407 Acked Write 4 Done Done Sent false
8254 3408 Acked Read 1 Done Done Sent false
8255 3409 Acked Read 3 Done Done Sent false
8256 3410 Acked Write 7 Done Done Sent false
8257 3411 Acked Write 9 Done Done Sent false
8258 3412 Acked Read 3 Done Done Sent false
8259 3413 Acked Read 3 Done Done Sent false
8260 STATES DS:0 DS:1 DS:2 TOTAL
8261 Sent 0 0 301 301
8262 Done 301 301 0 602
8263 Skipped 0 0 0 0
8264 Error 0 0 0 0
8265 Last Flush: 3404 3404 3112
8266 Downstairs last five completed: 3112 3111 3110 3109 3108
8267 Upstairs last five completed: 3413 3412 3411 3410 3409
8268 CLIENT: Up:0 ds:301 act:2
8269 [8830] got start action
8270 [8830] I am going to restart
8271 Starting downstairs at port 8830
8272 Make output file at "/tmp/dsc/downstairs-8830.txt"
8273 [8830][2] reports Starting
8274 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1208)
8275 [8830][2] reports Running
8276 ----------------------------------------------------------------
8277 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8278 States: Active Active Offline
8279 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8280 3113 Acked Read 4 Done Done Sent false
8281 3114 Acked Write 2 Done Done Sent false
8282 3115 Acked Flush 0 Done Done Sent false
8283 3116 Acked Write 8 Done Done Sent false
8284 3117 Acked Read 2 Done Done Sent false
8285 3118 Acked Read 10 Done Done Sent false
8286 3119 Acked Read 10 Done Done Sent false
8287 3120 Acked Read 10 Done Done Sent false
8288 3121 Acked Read 8 Done Done Sent false
8289 3122 Acked Flush 0 Done Done Sent false
8290 3123 Acked Write 8 Done Done Sent false
8291 3124 Acked Read 7 Done Done Sent false
8292 3125 Acked Read 8 Done Done Sent false
8293 3126 Acked Write 1 Done Done Sent false
8294 3127 Acked Flush 0 Done Done Sent false
8295 3128 Acked Read 1 Done Done Sent false
8296 3129 Acked Write 7 Done Done Sent false
8297 3130 Acked Read 5 Done Done Sent false
8298 3131 Acked Write 5 Done Done Sent false
8299 3132 Acked Write 8 Done Done Sent false
8300 3133 Acked Read 2 Done Done Sent false
8301 3134 Acked Read 3 Done Done Sent false
8302 3135 Acked Read 2 Done Done Sent false
8303 3136 Acked Flush 0 Done Done Sent false
8304 3137 Acked Write 4 Done Done Sent false
8305 3138 Acked Flush 0 Done Done Sent false
8306 3139 Acked Write 5 Done Done Sent false
8307 3140 Acked Write 10 Done Done Sent false
8308 3141 Acked Read 9 Done Done Sent false
8309 3142 Acked Read 1 Done Done Sent false
8310 3143 Acked Read 9 Done Done Sent false
8311 3144 Acked Read 5 Done Done Sent false
8312 3145 Acked Write 1 Done Done Sent false
8313 3146 Acked Write 6 Done Done Sent false
8314 3147 Acked Read 7 Done Done Sent false
8315 3148 Acked Write 5 Done Done Sent false
8316 3149 Acked Read 4 Done Done Sent false
8317 3150 Acked Read 7 Done Done Sent false
8318 3151 Acked Read 3 Done Done Sent false
8319 3152 Acked Read 5 Done Done Sent false
8320 3153 Acked Read 3 Done Done Sent false
8321 3154 Acked Write 2 Done Done Sent false
8322 3155 Acked Write 6 Done Done Sent false
8323 3156 Acked Write 8 Done Done Sent false
8324 3157 Acked Read 10 Done Done Sent false
8325 3158 Acked Read 8 Done Done Sent false
8326 3159 Acked Read 10 Done Done Sent false
8327 3160 Acked Read 3 Done Done Sent false
8328 3161 Acked Read 9 Done Done Sent false
8329 3162 Acked Flush 0 Done Done Sent false
8330 3163 Acked Read 7 Done Done Sent false
8331 3164 Acked Read 2 Done Done Sent false
8332 3165 Acked Read 5 Done Done Sent false
8333 3166 Acked Read 3 Done Done Sent false
8334 3167 Acked Read 8 Done Done Sent false
8335 3168 Acked Write 1 Done Done Sent false
8336 3169 Acked Read 9 Done Done Sent false
8337 3170 Acked Read 4 Done Done Sent false
8338 3171 Acked Read 4 Done Done Sent false
8339 3172 Acked Read 2 Done Done Sent false
8340 3173 Acked Write 10 Done Done Sent false
8341 3174 Acked Read 10 Done Done Sent false
8342 3175 Acked Write 6 Done Done Sent false
8343 3176 Acked Write 4 Done Done Sent false
8344 3177 Acked Read 6 Done Done Sent false
8345 3178 Acked Read 5 Done Done Sent false
8346 3179 Acked Write 6 Done Done Sent false
8347 3180 Acked Read 7 Done Done Sent false
8348 3181 Acked Read 2 Done Done Sent false
8349 3182 Acked Read 9 Done Done Sent false
8350 3183 Acked Write 7 Done Done Sent false
8351 3184 Acked Flush 0 Done Done Sent false
8352 3185 Acked Read 6 Done Done Sent false
8353 3186 Acked Read 2 Done Done Sent false
8354 3187 Acked Write 8 Done Done Sent false
8355 3188 Acked Write 9 Done Done Sent false
8356 3189 Acked Write 3 Done Done Sent false
8357 3190 Acked Flush 0 Done Done Sent false
8358 3191 Acked Read 10 Done Done Sent false
8359 3192 Acked Read 3 Done Done Sent false
8360 3193 Acked Read 3 Done Done Sent false
8361 3194 Acked Read 2 Done Done Sent false
8362 3195 Acked Read 9 Done Done Sent false
8363 3196 Acked Read 5 Done Done Sent false
8364 3197 Acked Read 10 Done Done Sent false
8365 3198 Acked Read 6 Done Done Sent false
8366 3199 Acked Read 10 Done Done Sent false
8367 3200 Acked Read 4 Done Done Sent false
8368 3201 Acked Write 7 Done Done Sent false
8369 3202 Acked Read 10 Done Done Sent false
8370 3203 Acked Read 10 Done Done Sent false
8371 3204 Acked Read 4 Done Done Sent false
8372 3205 Acked Read 5 Done Done Sent false
8373 3206 Acked Write 9 Done Done Sent false
8374 3207 Acked Write 4 Done Done Sent false
8375 3208 Acked Flush 0 Done Done Sent false
8376 3209 Acked Write 8 Done Done Sent false
8377 3210 Acked Write 8 Done Done Sent false
8378 3211 Acked Write 9 Done Done Sent false
8379 3212 Acked Write 8 Done Done Sent false
8380 3213 Acked Read 1 Done Done Sent false
8381 3214 Acked Read 4 Done Done Sent false
8382 3215 Acked Write 6 Done Done Sent false
8383 3216 Acked Flush 0 Done Done Sent false
8384 3217 Acked Write 8 Done Done Sent false
8385 3218 Acked Read 5 Done Done Sent false
8386 3219 Acked Write 7 Done Done Sent false
8387 3220 Acked Write 10 Done Done Sent false
8388 3221 Acked Write 5 Done Done Sent false
8389 3222 Acked Write 3 Done Done Sent false
8390 3223 Acked Read 1 Done Done Sent false
8391 3224 Acked Write 1 Done Done Sent false
8392 3225 Acked Read 5 Done Done Sent false
8393 3226 Acked Write 6 Done Done Sent false
8394 3227 Acked Flush 0 Done Done Sent false
8395 3228 Acked Read 1 Done Done Sent false
8396 3229 Acked Read 5 Done Done Sent false
8397 3230 Acked Read 8 Done Done Sent false
8398 3231 Acked Write 8 Done Done Sent false
8399 3232 Acked Read 3 Done Done Sent false
8400 3233 Acked Write 4 Done Done Sent false
8401 3234 Acked Write 7 Done Done Sent false
8402 3235 Acked Read 5 Done Done Sent false
8403 3236 Acked Read 2 Done Done Sent false
8404 3237 Acked Read 10 Done Done Sent false
8405 3238 Acked Read 1 Done Done Sent false
8406 3239 Acked Write 10 Done Done Sent false
8407 3240 Acked Read 1 Done Done Sent false
8408 3241 Acked Write 5 Done Done Sent false
8409 3242 Acked Write 7 Done Done Sent false
8410 3243 Acked Read 6 Done Done Sent false
8411 3244 Acked Write 4 Done Done Sent false
8412 3245 Acked Write 10 Done Done Sent false
8413 3246 Acked Write 5 Done Done Sent false
8414 3247 Acked Write 1 Done Done Sent false
8415 3248 Acked Write 2 Done Done Sent false
8416 3249 Acked Write 2 Done Done Sent false
8417 3250 Acked Read 9 Done Done Sent false
8418 3251 Acked Read 2 Done Done Sent false
8419 3252 Acked Read 1 Done Done Sent false
8420 3253 Acked Write 9 Done Done Sent false
8421 3254 Acked Write 9 Done Done Sent false
8422 3255 Acked Flush 0 Done Done Sent false
8423 3256 Acked Read 1 Done Done Sent false
8424 3257 Acked Write 2 Done Done Sent false
8425 3258 Acked Read 6 Done Done Sent false
8426 3259 Acked Write 2 Done Done Sent false
8427 3260 Acked Write 9 Done Done Sent false
8428 3261 Acked Read 1 Done Done Sent false
8429 3262 Acked Read 8 Done Done Sent false
8430 3263 Acked Write 1 Done Done Sent false
8431 3264 Acked Read 1 Done Done Sent false
8432 3265 Acked Read 3 Done Done Sent false
8433 3266 Acked Write 2 Done Done Sent false
8434 3267 Acked Read 10 Done Done Sent false
8435 3268 Acked Read 6 Done Done Sent false
8436 3269 Acked Read 6 Done Done Sent false
8437 3270 Acked Write 1 Done Done Sent false
8438 3271 Acked Read 4 Done Done Sent false
8439 3272 Acked Read 7 Done Done Sent false
8440 3273 Acked Read 6 Done Done Sent false
8441 3274 Acked Write 2 Done Done Sent false
8442 3275 Acked Read 10 Done Done Sent false
8443 3276 Acked Write 10 Done Done Sent false
8444 3277 Acked Read 6 Done Done Sent false
8445 3278 Acked Read 3 Done Done Sent false
8446 3279 Acked Write 10 Done Done Sent false
8447 3280 Acked Write 5 Done Done Sent false
8448 3281 Acked Write 9 Done Done Sent false
8449 3282 Acked Read 5 Done Done Sent false
8450 3283 Acked Read 3 Done Done Sent false
8451 3284 Acked Read 9 Done Done Sent false
8452 3285 Acked Write 6 Done Done Sent false
8453 3286 Acked Write 10 Done Done Sent false
8454 3287 Acked Read 10 Done Done Sent false
8455 3288 Acked Read 8 Done Done Sent false
8456 3289 Acked Write 7 Done Done Sent false
8457 3290 Acked Write 1 Done Done Sent false
8458 3291 Acked Write 7 Done Done Sent false
8459 3292 Acked Write 8 Done Done Sent false
8460 3293 Acked Read 7 Done Done Sent false
8461 3294 Acked Write 5 Done Done Sent false
8462 3295 Acked Read 6 Done Done Sent false
8463 3296 Acked Flush 0 Done Done Sent false
8464 3297 Acked Read 4 Done Done Sent false
8465 3298 Acked Write 3 Done Done Sent false
8466 3299 Acked Read 2 Done Done Sent false
8467 3300 Acked Read 10 Done Done Sent false
8468 3301 Acked Write 3 Done Done Sent false
8469 3302 Acked Write 4 Done Done Sent false
8470 3303 Acked Read 2 Done Done Sent false
8471 3304 Acked Read 8 Done Done Sent false
8472 3305 Acked Read 6 Done Done Sent false
8473 3306 Acked Write 10 Done Done Sent false
8474 3307 Acked Write 8 Done Done Sent false
8475 3308 Acked Read 9 Done Done Sent false
8476 3309 Acked Flush 0 Done Done Sent false
8477 3310 Acked Flush 0 Done Done Sent false
8478 3311 Acked Read 6 Done Done Sent false
8479 3312 Acked Write 5 Done Done Sent false
8480 3313 Acked Write 9 Done Done Sent false
8481 3314 Acked Write 3 Done Done Sent false
8482 3315 Acked Write 7 Done Done Sent false
8483 3316 Acked Write 6 Done Done Sent false
8484 3317 Acked Read 10 Done Done Sent false
8485 3318 Acked Flush 0 Done Done Sent false
8486 3319 Acked Read 8 Done Done Sent false
8487 3320 Acked Write 4 Done Done Sent false
8488 3321 Acked Flush 0 Done Done Sent false
8489 3322 Acked Read 3 Done Done Sent false
8490 3323 Acked Write 9 Done Done Sent false
8491 3324 Acked Write 1 Done Done Sent false
8492 3325 Acked Read 7 Done Done Sent false
8493 3326 Acked Write 3 Done Done Sent false
8494 3327 Acked Read 2 Done Done Sent false
8495 3328 Acked Read 6 Done Done Sent false
8496 3329 Acked Flush 0 Done Done Sent false
8497 3330 Acked Flush 0 Done Done Sent false
8498 3331 Acked Flush 0 Done Done Sent false
8499 3332 Acked Write 1 Done Done Sent false
8500 3333 Acked Read 9 Done Done Sent false
8501 3334 Acked Flush 0 Done Done Sent false
8502 3335 Acked Write 4 Done Done Sent false
8503 3336 Acked Read 3 Done Done Sent false
8504 3337 Acked Read 7 Done Done Sent false
8505 3338 Acked Write 7 Done Done Sent false
8506 3339 Acked Write 10 Done Done Sent false
8507 3340 Acked Write 7 Done Done Sent false
8508 3341 Acked Write 4 Done Done Sent false
8509 3342 Acked Write 4 Done Done Sent false
8510 3343 Acked Read 7 Done Done Sent false
8511 3344 Acked Write 3 Done Done Sent false
8512 3345 Acked Read 5 Done Done Sent false
8513 3346 Acked Write 8 Done Done Sent false
8514 3347 Acked Read 6 Done Done Sent false
8515 3348 Acked Read 3 Done Done Sent false
8516 3349 Acked Write 6 Done Done Sent false
8517 3350 Acked Read 4 Done Done Sent false
8518 3351 Acked Write 6 Done Done Sent false
8519 3352 Acked Read 2 Done Done Sent false
8520 3353 Acked Read 9 Done Done Sent false
8521 3354 Acked Read 7 Done Done Sent false
8522 3355 Acked Flush 0 Done Done Sent false
8523 3356 Acked Read 8 Done Done Sent false
8524 3357 Acked Read 10 Done Done Sent false
8525 3358 Acked Write 5 Done Done Sent false
8526 3359 Acked Read 6 Done Done Sent false
8527 3360 Acked Write 7 Done Done Sent false
8528 3361 Acked Read 8 Done Done Sent false
8529 3362 Acked Write 10 Done Done Sent false
8530 3363 Acked Write 6 Done Done Sent false
8531 3364 Acked Write 1 Done Done Sent false
8532 3365 Acked Read 4 Done Done Sent false
8533 3366 Acked Read 4 Done Done Sent false
8534 3367 Acked Read 7 Done Done Sent false
8535 3368 Acked Write 7 Done Done Sent false
8536 3369 Acked Write 6 Done Done Sent false
8537 3370 Acked Read 3 Done Done Sent false
8538 3371 Acked Write 6 Done Done Sent false
8539 3372 Acked Write 3 Done Done Sent false
8540 3373 Acked Write 3 Done Done Sent false
8541 3374 Acked Read 9 Done Done Sent false
8542 3375 Acked Read 2 Done Done Sent false
8543 3376 Acked Read 10 Done Done Sent false
8544 3377 Acked Write 9 Done Done Sent false
8545 3378 Acked Read 5 Done Done Sent false
8546 3379 Acked Read 2 Done Done Sent false
8547 3380 Acked Read 6 Done Done Sent false
8548 3381 Acked Write 8 Done Done Sent false
8549 3382 Acked Read 7 Done Done Sent false
8550 3383 Acked Read 4 Done Done Sent false
8551 3384 Acked Read 9 Done Done Sent false
8552 3385 Acked Write 1 Done Done Sent false
8553 3386 Acked Flush 0 Done Done Sent false
8554 3387 Acked Write 7 Done Done Sent false
8555 3388 Acked Write 10 Done Done Sent false
8556 3389 Acked Read 2 Done Done Sent false
8557 3390 Acked Read 2 Done Done Sent false
8558 3391 Acked Read 7 Done Done Sent false
8559 3392 Acked Read 10 Done Done Sent false
8560 3393 Acked Flush 0 Done Done Sent false
8561 3394 Acked Read 9 Done Done Sent false
8562 3395 Acked Write 6 Done Done Sent false
8563 3396 Acked Write 2 Done Done Sent false
8564 3397 Acked Write 3 Done Done Sent false
8565 3398 Acked Read 2 Done Done Sent false
8566 3399 Acked Read 5 Done Done Sent false
8567 3400 Acked Read 4 Done Done Sent false
8568 3401 Acked Write 6 Done Done Sent false
8569 3402 Acked Write 6 Done Done Sent false
8570 3403 Acked Read 5 Done Done Sent false
8571 3404 Acked Flush 0 Done Done Sent false
8572 3405 Acked Read 8 Done Done Sent false
8573 3406 Acked Write 10 Done Done Sent false
8574 3407 Acked Write 4 Done Done Sent false
8575 3408 Acked Read 1 Done Done Sent false
8576 3409 Acked Read 3 Done Done Sent false
8577 3410 Acked Write 7 Done Done Sent false
8578 3411 Acked Write 9 Done Done Sent false
8579 3412 Acked Read 3 Done Done Sent false
8580 3413 Acked Read 3 Done Done Sent false
8581 3414 Acked Flush 0 Done Done Sent false
8582 STATES DS:0 DS:1 DS:2 TOTAL
8583 Sent 0 0 302 302
8584 Done 302 302 0 604
8585 Skipped 0 0 0 0
8586 Error 0 0 0 0
8587 Last Flush: 3414 3414 3112
8588 Downstairs last five completed: 3112 3111 3110 3109 3108
8589 Upstairs last five completed: 3414 3413 3412 3411 3410
8590 CLIENT: Up:0 ds:302 act:2
85912025-01-21T15:48:57.054ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
85922025-01-21T15:48:57.054ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52753) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
85932025-01-21T15:48:57.055ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
85942025-01-21T15:48:57.055ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
85952025-01-21T15:48:57.056ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
85962025-01-21T15:48:57.056ZINFOcrucible: send last flush ID to this DS: 3112 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
85972025-01-21T15:48:57.056ZINFOcrucible: Replied this last flush ID: 3112 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
85982025-01-21T15:48:57.056ZINFOcrucible: [2] client re-new 302 jobs since flush 3112 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
85992025-01-21T15:48:57.056ZINFOcrucible: [2] Replayed 302 jobs since flush: 3112 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
8600 ----------------------------------------------------------------
8601 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8602 Downstairs last five completed: 3414 3413 3412 3411 3410
8603 Upstairs last five completed: 3414 3413 3412 3411 3410
8604 CLIENT: Up:0 ds:0 act:3
8605 Replay: All jobs finished
8606 Main task has work to do, go find it
8607 got dsc Stop(0)
8608 stop 0
8609 Jan 21 15:48:59.305 INFO request completed[8810] Got stop action so:false kr:false
8610 , latency_us: 346, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 8a250793-64a7-454b-915d-cba2737cf2a7, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
86112025-01-21T15:48:59.306ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
86122025-01-21T15:48:59.306ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
86132025-01-21T15:48:59.306ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
8614 Jan 21 15:48:59.306 INFO request completed, latency_us: 152, response_code: 200, uri: /state/cid/0, method: GET, req_id: 92f4e4f6-d4b2-4cbb-b235-d18031088d60{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:48:59.306408496Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
8615 , remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
8616 [8810] Exited with: ExitStatus(unix_wait_status(9))
8617 [8810][0] reports Exit
8618 Jan 21 15:49:03.308 INFO request completed, latency_us: 210, response_code: 200, uri: /state/cid/0, method: GET, req_id: 89f9a013-821e-40a3-801b-6281739cd8a1, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
8619 001/300 Read block 3369 len 28672
8620 002/300 Read block 2482 len 36864
8621 003/300 Write block 4472 len 28672 data: 3 3 3 2 2 3 3
8622 004/300 Read block 1696 len 16384
8623 005/300 Read block 2468 len 8192
8624 006/300 Write block 4777 len 4096 data: 3
8625 007/300 Write block 2139 len 16384 data: 4 3 2 2
8626 008/300 Read block 1815 len 12288
8627 009/300 Read block 2928 len 32768
8628 010/300 Write block 4896 len 8192 data: 3 3
8629 011/300 Flush
8630 012/300 Read block 1797 len 4096
8631 013/300 Read block 3820 len 20480
8632 014/300 Read block 4563 len 12288
8633 015/300 Write block 2298 len 36864 data: 2 2 2 2 2 2 2 2 2
8634 016/300 Read block 4220 len 28672
8635 017/300 Read block 2374 len 16384
8636 018/300 Read block 3543 len 24576
8637 019/300 Write block 2604 len 20480 data: 5 5 4 4 4
8638 020/300 Read block 1824 len 40960
8639 021/300 Write block 3468 len 40960 data: 4 4 4 3 3 3 3 4 4 5
8640 022/300 Write block 3171 len 28672 data: 3 3 4 4 4 4 5
8641 023/300 Flush
8642 024/300 Write block 1321 len 28672 data: 4 4 5 5 4 3 2
8643 025/300 Flush
8644 026/300 Read block 3367 len 24576
8645 027/300 Read block 2862 len 24576
8646 028/300 Read block 2645 len 4096
8647 029/300 Read block 1115 len 16384
8648 030/300 Write block 3907 len 8192 data: 3 3
8649 031/300 Write block 2939 len 20480 data: 3 2 2 2 2
8650 032/300 Read block 4619 len 8192
8651 033/300 Write block 1497 len 4096 data: 2
8652 034/300 Flush
8653 035/300 Read block 2785 len 16384
8654 036/300 Read block 1745 len 32768
8655 037/300 Write block 3874 len 12288 data: 2 3 4
8656 038/300 Read block 3967 len 32768
8657 039/300 Read block 2261 len 28672
8658 040/300 Read block 1788 len 20480
8659 041/300 Read block 118 len 40960
8660 042/300 Write block 1444 len 4096 data: 3
8661 043/300 Read block 2065 len 36864
8662 044/300 Read block 4664 len 40960
8663 045/300 Read block 1671 len 28672
8664 046/300 Write block 3124 len 4096 data: 2
8665 047/300 Read block 4925 len 8192
8666 048/300 Read block 620 len 4096
8667 049/300 Read block 4490 len 36864
8668 050/300 Write block 1933 len 12288 data: 4 4 4
8669 051/300 Write block 1906 len 28672 data: 5 6 6 6 3 4 4
8670 052/300 Read block 4446 len 36864
8671 053/300 Read block 1838 len 32768
8672 054/300 Flush
8673 055/300 Read block 1306 len 16384
8674 056/300 Write block 60 len 20480 data: 3 3 3 2 3
8675 057/300 Write block 1355 len 36864 data: 3 2 2 2 2 2 2 4 4
8676 058/300 Write block 1933 len 28672 data: 5 5 5 4 4 2 2
8677 059/300 Write block 1025 len 36864 data: 2 2 2 2 2 2 2 2 2
8678 060/300 Read block 1585 len 24576
8679 061/300 Read block 4080 len 16384
8680 062/300 Write block 1315 len 24576 data: 5 5 5 5 5 5
8681 063/300 Read block 5 len 36864
8682 064/300 Write block 3606 len 36864 data: 2 3 3 3 3 2 3 2 2
8683 065/300 Read block 4375 len 12288
8684 066/300 Read block 2976 len 24576
8685 067/300 Read block 4588 len 40960
8686 068/300 Write block 1844 len 4096 data: 5
8687 069/300 Flush
8688 070/300 Read block 2583 len 40960
8689 071/300 Write block 2055 len 8192 data: 2 2
8690 072/300 Flush
8691 073/300 Write block 2787 len 32768 data: 3 2 2 3 3 3 3 4
8692 074/300 Read block 4591 len 36864
8693 075/300 Read block 4148 len 16384
8694 076/300 Read block 4116 len 12288
8695 077/300 Read block 68 len 20480
8696 078/300 Write block 414 len 24576 data: 4 3 3 3 3 3
8697 079/300 Flush
8698 080/300 Read block 704 len 4096
8699 081/300 Flush
8700 082/300 Write block 899 len 36864 data: 2 2 2 2 2 2 2 2 2
8701 083/300 Flush
8702 084/300 Write block 1406 len 28672 data: 3 3 3 3 3 3 3
8703 085/300 Write block 1232 len 20480 data: 2 2 2 2 2
8704 086/300 Write block 1950 len 12288 data: 3 2 2
8705 087/300 Write block 2788 len 4096 data: 3
8706 088/300 Read block 4560 len 4096
8707 089/300 Read block 4389 len 28672
8708 090/300 Read block 2292 len 8192
8709 091/300 Read block 2709 len 12288
8710 092/300 Write block 2070 len 12288 data: 3 3 3
8711 093/300 Read block 4431 len 20480
8712 094/300 Read block 4924 len 36864
8713 095/300 Read block 460 len 16384
8714 096/300 Read block 4662 len 4096
8715 097/300 Write block 2444 len 32768 data: 3 2 3 4 3 3 3 3
8716 098/300 Read block 3834 len 32768
8717 099/300 Read block 585 len 32768
8718 100/300 Write block 2926 len 8192 data: 4 4
8719 101/300 Write block 3543 len 20480 data: 2 3 3 3 3
8720 102/300 Write block 4592 len 24576 data: 2 2 2 2 2 3
8721 103/300 Read block 4205 len 12288
8722 104/300 Write block 3593 len 20480 data: 3 2 2 2 2
8723 105/300 Read block 3269 len 32768
8724 106/300 Write block 2948 len 8192 data: 2 2
8725 107/300 Write block 2477 len 28672 data: 2 2 2 2 2 2 3
8726 108/300 Flush
8727 109/300 Flush
8728 110/300 Write block 2055 len 36864 data: 3 3 3 3 3 3 3 3 3
8729 111/300 Write block 4320 len 28672 data: 3 3 3 3 2 2 2
8730 112/300 Write block 1536 len 36864 data: 3 3 3 3 3 3 4 4 3
8731 113/300 Write block 3563 len 36864 data: 2 2 2 2 3 3 3 3 3
8732 114/300 Flush
8733 115/300 Write block 2951 len 8192 data: 3 3
8734 116/300 Flush
8735 117/300 Write block 2252 len 8192 data: 3 3
8736 118/300 Read block 3618 len 8192
8737 119/300 Read block 1507 len 16384
8738 120/300 Write block 1684 len 24576 data: 3 2 2 2 2 2
8739 121/300 Read block 4507 len 36864
8740 122/300 Read block 2531 len 40960
8741 123/300 Write block 4625 len 32768 data: 4 4 4 3 2 3 3 4
8742 124/300 Read block 94 len 20480
8743 125/300 Write block 430 len 8192 data: 2 2
8744 126/300 Read block 602 len 28672
8745 127/300 Read block 691 len 28672
8746 128/300 Read block 2799 len 28672
8747 129/300 Read block 2648 len 32768
8748 130/300 Read block 3557 len 12288
8749 131/300 Read block 1141 len 16384
8750 132/300 Write block 1587 len 36864 data: 3 2 2 3 5 4 3 3 2
8751 133/300 Read block 2726 len 8192
8752 134/300 Write block 941 len 20480 data: 2 2 2 2 4
8753 135/300 Read block 4693 len 8192
8754 136/300 Read block 3310 len 36864
8755 137/300 Read block 1244 len 16384
8756 138/300 Read block 3640 len 20480
8757 139/300 Write block 453 len 16384 data: 4 4 4 4
8758 140/300 Write block 4540 len 4096 data: 4
8759 141/300 Read block 2048 len 36864
8760 142/300 Write block 4301 len 24576 data: 2 2 2 2 2 2
8761 143/300 Flush
8762 144/300 Read block 4381 len 24576
8763 145/300 Write block 634 len 20480 data: 4 4 5 6 6
8764 146/300 Flush
8765 147/300 Read block 3050 len 36864
8766 148/300 Read block 1495 len 20480
8767 149/300 Read block 517 len 12288
8768 150/300 Read block 2547 len 12288
8769 151/300 Read block 1288 len 28672
8770 152/300 Read block 4815 len 20480
8771 153/300 Write block 2459 len 32768 data: 7 7 6 6 7 7 7 6
8772 154/300 Write block 1764 len 4096 data: 3
8773 155/300 Write block 2786 len 28672 data: 3 4 4 3 4 4 4
8774 156/300 Write block 2715 len 24576 data: 3 3 3 2 2 2
8775 157/300 Read block 2453 len 20480
8776 158/300 Write block 3964 len 8192 data: 4 4
8777 159/300 Write block 2613 len 8192 data: 3 3
8778 160/300 Flush
8779 161/300 Read block 1801 len 32768
8780 162/300 Read block 793 len 40960
8781 163/300 Read block 3381 len 28672
8782 164/300 Read block 4947 len 4096
8783 165/300 Flush
8784 166/300 Read block 2623 len 20480
8785 167/300 Write block 2929 len 24576 data: 4 4 3 3 3 3
8786 168/300 Write block 1705 len 40960 data: 4 4 4 3 3 2 2 2 2 2
8787 169/300 Write block 937 len 32768 data: 3 3 3 2 3 3 3 3
8788 170/300 Read block 2378 len 32768
8789 171/300 Read block 2739 len 4096
8790 172/300 Read block 3534 len 12288
8791 173/300 Flush
8792 174/300 Read block 868 len 16384
8793 175/300 Read block 4482 len 16384
8794 176/300 Read block 3303 len 8192
8795 177/300 Write block 864 len 28672 data: 4 4 4 4 3 3 2
8796 178/300 Read block 2703 len 36864
8797 179/300 Read block 3580 len 12288
8798 180/300 Write block 555 len 16384 data: 4 4 4 4
8799 181/300 Write block 1737 len 16384 data: 4 4 3 3
8800 182/300 Read block 2167 len 32768
8801 183/300 Write block 4107 len 8192 data: 3 4
8802 184/300 Read block 1734 len 32768
8803 185/300 Read block 2674 len 36864
8804 186/300 Read block 260 len 32768
8805 187/300 Read block 1480 len 40960
8806 188/300 Read block 4818 len 16384
8807 189/300 Read block 3628 len 36864
8808 190/300 Write block 890 len 16384 data: 6 5 5 5
8809 191/300 Read block 1526 len 4096
8810 192/300 Read block 3968 len 32768
8811 193/300 Read block 1928 len 20480
8812 194/300 Write block 1814 len 36864 data: 3 3 3 3 3 2 2 2 3
8813 195/300 Write block 2813 len 32768 data: 3 3 3 3 2 3 3 3
8814 196/300 Read block 3523 len 24576
8815 197/300 Read block 3177 len 12288
8816 198/300 Flush
8817 199/300 Write block 3757 len 12288 data: 2 3 3
8818 200/300 Write block 3835 len 28672 data: 3 2 3 3 3 3 4
8819 201/300 Write block 4850 len 32768 data: 3 3 3 4 4 4 4 4
8820 202/300 Write block 3698 len 16384 data: 4 4 4 4
8821 203/300 Read block 4119 len 36864
8822 204/300 Read block 4391 len 32768
8823 205/300 Write block 1997 len 4096 data: 3
8824 206/300 Read block 4097 len 40960
8825 207/300 Write block 2672 len 24576 data: 5 5 4 5 5 4
8826 208/300 Read block 3188 len 4096
8827 209/300 Read block 1818 len 20480
8828 210/300 Write block 321 len 24576 data: 3 3 2 3 3 3
8829 211/300 Read block 2834 len 4096
8830 212/300 Write block 2455 len 8192 data: 3 4
8831 213/300 Flush
8832 214/300 Write block 2513 len 8192 data: 2 2
8833 215/300 Write block 1325 len 36864 data: 5 4 3 2 2 2 2 2 3
8834 216/300 Read block 311 len 40960
8835 217/300 Write block 4173 len 12288 data: 4 4 4
8836 218/300 Read block 1611 len 16384
8837 219/300 Write block 3230 len 12288 data: 3 3 3
8838 220/300 Read block 2136 len 20480
8839 221/300 Write block 3885 len 28672 data: 2 3 3 4 4 5 5
8840 222/300 Read block 1027 len 28672
8841 223/300 Read block 2536 len 8192
8842 224/300 Read block 4821 len 24576
8843 225/300 Read block 1980 len 8192
8844 226/300 Read block 1405 len 32768
8845 227/300 Read block 3269 len 12288
8846 228/300 Read block 4777 len 24576
8847 229/300 Write block 2812 len 40960 data: 3 4 4 4 4 3 4 4 4 3
8848 230/300 Write block 2882 len 40960 data: 2 2 2 2 3 3 3 3 3 3
8849 231/300 Read block 2776 len 4096
8850 232/300 Write block 1266 len 8192 data: 3 4
8851 233/300 Read block 3473 len 28672
8852 234/300 Write block 1865 len 28672 data: 4 3 3 2 2 2 2
8853 235/300 Write block 1298 len 20480 data: 2 3 3 3 3
8854 236/300 Write block 1005 len 32768 data: 2 2 2 2 3 4 4 4
8855 237/300 Read block 921 len 16384
8856 238/300 Write block 4343 len 36864 data: 3 2 3 3 3 3 3 3 3
8857 239/300 Read block 1130 len 4096
8858 240/300 Read block 4756 len 8192
8859 241/300 Read block 605 len 36864
8860 242/300 Flush
8861 243/300 Write block 3225 len 40960 data: 3 4 3 2 2 4 4 4 2 2
8862 244/300 Read block 145 len 36864
8863 245/300 Read block 848 len 32768
8864 246/300 Read block 3160 len 36864
8865 247/300 Write block 4294 len 12288 data: 2 2 3
8866 248/300 Write block 3561 len 28672 data: 2 2 3 3 3 3 4
8867 249/300 Write block 3523 len 24576 data: 3 3 3 4 4 4
8868 250/300 Read block 3822 len 8192
8869 251/300 Read block 2611 len 40960
8870 252/300 Flush
8871 253/300 Write block 1983 len 36864 data: 5 6 6 5 5 4 4 3 3
8872 254/300 Read block 1134 len 16384
8873 255/300 Write block 1719 len 28672 data: 2 2 2 2 2 2 2
8874 256/300 Read block 1853 len 12288
8875 257/300 Read block 4339 len 40960
8876 258/300 Read block 523 len 32768
8877 259/300 Write block 2535 len 20480 data: 2 2 2 2 3
8878 260/300 Write block 4121 len 36864 data: 4 4 3 3 3 3 3 3 3
8879 261/300 Write block 441 len 40960 data: 3 2 3 3 3 3 3 3 2 2
8880 262/300 Write block 1683 len 32768 data: 3 4 3 3 3 3 3 2
8881 263/300 Read block 4524 len 16384
8882 264/300 Read block 844 len 28672
8883 265/300 Write block 1679 len 20480 data: 2 2 2 3 4
8884 266/300 Read block 161 len 12288
8885 267/300 Flush
8886 268/300 Write block 800 len 28672 data: 4 3 3 2 3 3 3
8887 269/300 Write block 4477 len 16384 data: 4 4 2 2
8888 270/300 Write block 2141 len 28672 data: 3 3 2 2 3 3 3
8889 271/300 Flush
8890 272/300 Write block 2753 len 28672 data: 4 4 4 4 4 2 3
8891 273/300 Read block 4109 len 20480
8892 274/300 Write block 855 len 36864 data: 3 3 3 4 4 5 6 4 4
8893 275/300 Read block 2722 len 24576
8894 276/300 Read block 3978 len 36864
8895 277/300 Write block 3707 len 28672 data: 3 3 3 3 4 4 4
8896 278/300 Write block 4809 len 36864 data: 2 2 2 2 2 3 3 3 3
8897 279/300 Read block 2498 len 4096
8898 280/300 Read block 4669 len 12288
8899 281/300 Write block 3463 len 36864 data: 3 3 2 2 2 5 5 5 4
8900 282/300 Read block 1147 len 32768
8901 283/300 Read block 4861 len 24576
8902 284/300 Read block 1386 len 40960
8903 285/300 Read block 337 len 8192
8904 286/300 Write block 3124 len 8192 data: 3 2
8905 287/300 Read block 1944 len 20480
8906 288/300 Write block 2390 len 24576 data: 2 2 3 3 3 3
8907 289/300 Read block 2079 len 32768
8908 290/300 Read block 3157 len 16384
8909 291/300 Read block 1491 len 40960
8910 292/300 Read block 2094 len 24576
8911 293/300 Write block 2458 len 4096 data: 6
8912 294/300 Read block 1647 len 12288
8913 295/300 Write block 1416 len 8192 data: 2 2
8914 296/300 Write block 3503 len 16384 data: 2 2 2 2
8915 297/300 Read block 3036 len 24576
8916 298/300 Write block 590 len 16384 data: 3 3 3 4
8917 299/300 Read block 2087 len 28672
8918 300/300 Write block 3369 len 32768 data: 5 3 3 3 3 3 3 2
8919 301/300 Read block 3848 len 24576
8920 Main task has work to do, go find it
8921 got dsc Start(0)
8922 start 0
8923 Jan 21 15:49:03.555 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/0, method: POST, req_id: 6a3198e6-7419-465e-a682-7f6313d4b27b, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
8924 Replay: started 0, returned:Ok(())
8925 ----------------------------------------------------------------
8926 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8927 States: Offline Active Active
8928 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8929 3415 Acked Read 7 Sent Done Done false
8930 3416 Acked Read 9 Sent Done Done false
8931 3417 Acked Write 7 Sent Done Done false
8932 3418 Acked Read 4 Sent Done Done false
8933 3419 Acked Read 2 Sent Done Done false
8934 3420 Acked Write 1 Sent Done Done false
8935 3421 Acked Write 4 Sent Done Done false
8936 3422 Acked Read 3 Sent Done Done false
8937 3423 Acked Read 8 Sent Done Done false
8938 3424 Acked Write 2 Sent Done Done false
8939 3425 Acked Flush 0 Sent Done Done false
8940 3426 Acked Read 1 Sent Done Done false
8941 3427 Acked Read 5 Sent Done Done false
8942 3428 Acked Read 3 Sent Done Done false
8943 3429 Acked Write 9 Sent Done Done false
8944 3430 Acked Read 7 Sent Done Done false
8945 3431 Acked Read 4 Sent Done Done false
8946 3432 Acked Read 6 Sent Done Done false
8947 3433 Acked Write 5 Sent Done Done false
8948 3434 Acked Read 10 Sent Done Done false
8949 3435 Acked Write 10 Sent Done Done false
8950 3436 Acked Write 7 Sent Done Done false
8951 3437 Acked Flush 0 Sent Done Done false
8952 3438 Acked Write 7 Sent Done Done false
8953 3439 Acked Flush 0 Sent Done Done false
8954 3440 Acked Read 6 Sent Done Done false
8955 3441 Acked Read 6 Sent Done Done false
8956 3442 Acked Read 1 Sent Done Done false
8957 3443 Acked Read 4 Sent Done Done false
8958 3444 Acked Write 2 Sent Done Done false
8959 3445 Acked Write 5 Sent Done Done false
8960 3446 Acked Read 2 Sent Done Done false
8961 3447 Acked Write 1 Sent Done Done false
8962 3448 Acked Flush 0 Sent Done Done false
8963 3449 Acked Read 4 Sent Done Done false
8964 3450 Acked Read 8 Sent Done Done false
8965 3451 Acked Write 3 Sent Done Done false
8966 3452 Acked Read 8 Sent Done Done false
8967 3453 Acked Read 7 Sent Done Done false
8968 3454 Acked Read 5 Sent Done Done false
8969 3455 Acked Read 10 Sent Done Done false
8970 3456 Acked Write 1 Sent Done Done false
8971 3457 Acked Read 9 Sent Done Done false
8972 3458 Acked Read 10 Sent Done Done false
8973 3459 Acked Read 7 Sent Done Done false
8974 3460 Acked Write 1 Sent Done Done false
8975 3461 Acked Read 2 Sent Done Done false
8976 3462 Acked Read 1 Sent Done Done false
8977 3463 Acked Read 9 Sent Done Done false
8978 3464 Acked Write 3 Sent Done Done false
8979 3465 Acked Write 7 Sent Done Done false
8980 3466 Acked Read 9 Sent Done Done false
8981 3467 Acked Read 8 Sent Done Done false
8982 3468 Acked Flush 0 Sent Done Done false
8983 3469 Acked Read 4 Sent Done Done false
8984 3470 Acked Write 5 Sent Done Done false
8985 3471 Acked Write 9 Sent Done Done false
8986 3472 Acked Write 7 Sent Done Done false
8987 3473 Acked Write 9 Sent Done Done false
8988 3474 Acked Read 6 Sent Done Done false
8989 3475 Acked Read 4 Sent Done Done false
8990 3476 Acked Write 6 Sent Done Done false
8991 3477 Acked Read 9 Sent Done Done false
8992 3478 Acked Write 9 Sent Done Done false
8993 3479 Acked Read 3 Sent Done Done false
8994 3480 Acked Read 6 Sent Done Done false
8995 3481 Acked Read 10 Sent Done Done false
8996 3482 Acked Write 1 Sent Done Done false
8997 3483 Acked Flush 0 Sent Done Done false
8998 3484 Acked Read 10 Sent Done Done false
8999 3485 Acked Write 2 Sent Done Done false
9000 3486 Acked Flush 0 Sent Done Done false
9001 3487 Acked Write 8 Sent Done Done false
9002 3488 Acked Read 9 Sent Done Done false
9003 3489 Acked Read 4 Sent Done Done false
9004 3490 Acked Read 3 Sent Done Done false
9005 3491 Acked Read 5 Sent Done Done false
9006 3492 Acked Write 6 Sent Done Done false
9007 3493 Acked Flush 0 Sent Done Done false
9008 3494 Acked Read 1 Sent Done Done false
9009 3495 Acked Flush 0 Sent Done Done false
9010 3496 Acked Write 9 Sent Done Done false
9011 3497 Acked Flush 0 Sent Done Done false
9012 3498 Acked Write 7 Sent Done Done false
9013 3499 Acked Write 5 Sent Done Done false
9014 3500 Acked Write 3 Sent Done Done false
9015 3501 Acked Write 1 Sent Done Done false
9016 3502 Acked Read 1 Sent Done Done false
9017 3503 Acked Read 7 Sent Done Done false
9018 3504 Acked Read 2 Sent Done Done false
9019 3505 Acked Read 3 Sent Done Done false
9020 3506 Acked Write 3 Sent Done Done false
9021 3507 Acked Read 5 Sent Done Done false
9022 3508 Acked Read 9 Sent Done Done false
9023 3509 Acked Read 4 Sent Done Done false
9024 3510 Acked Read 1 Sent Done Done false
9025 3511 Acked Write 8 Sent Done Done false
9026 3512 Acked Read 8 Sent Done Done false
9027 3513 Acked Read 8 Sent Done Done false
9028 3514 Acked Write 2 Sent Done Done false
9029 3515 Acked Write 5 Sent Done Done false
9030 3516 Acked Write 6 Sent Done Done false
9031 3517 Acked Read 3 Sent Done Done false
9032 3518 Acked Write 5 Sent Done Done false
9033 3519 Acked Read 8 Sent Done Done false
9034 3520 Acked Write 2 Sent Done Done false
9035 3521 Acked Write 7 Sent Done Done false
9036 3522 Acked Flush 0 Sent Done Done false
9037 3523 Acked Flush 0 Sent Done Done false
9038 3524 Acked Write 9 Sent Done Done false
9039 3525 Acked Write 7 Sent Done Done false
9040 3526 Acked Write 9 Sent Done Done false
9041 3527 Acked Write 9 Sent Done Done false
9042 3528 Acked Flush 0 Sent Done Done false
9043 3529 Acked Write 2 Sent Done Done false
9044 3530 Acked Flush 0 Sent Done Done false
9045 3531 Acked Write 2 Sent Done Done false
9046 3532 Acked Read 2 Sent Done Done false
9047 3533 Acked Read 4 Sent Done Done false
9048 3534 Acked Write 6 Sent Done Done false
9049 3535 Acked Read 9 Sent Done Done false
9050 3536 Acked Read 10 Sent Done Done false
9051 3537 Acked Write 8 Sent Done Done false
9052 3538 Acked Read 5 Sent Done Done false
9053 3539 Acked Write 2 Sent Done Done false
9054 3540 Acked Read 7 Sent Done Done false
9055 3541 Acked Read 7 Sent Done Done false
9056 3542 Acked Read 7 Sent Done Done false
9057 3543 Acked Read 8 Sent Done Done false
9058 3544 Acked Read 3 Sent Done Done false
9059 3545 Acked Read 4 Sent Done Done false
9060 3546 Acked Write 9 Sent Done Done false
9061 3547 Acked Read 2 Sent Done Done false
9062 3548 Acked Write 5 Sent Done Done false
9063 3549 Acked Read 2 Sent Done Done false
9064 3550 Acked Read 9 Sent Done Done false
9065 3551 Acked Read 4 Sent Done Done false
9066 3552 Acked Read 5 Sent Done Done false
9067 3553 Acked Write 4 Sent Done Done false
9068 3554 Acked Write 1 Sent Done Done false
9069 3555 Acked Read 9 Sent Done Done false
9070 3556 Acked Write 6 Sent Done Done false
9071 3557 Acked Flush 0 Sent Done Done false
9072 3558 Acked Read 6 Sent Done Done false
9073 3559 Acked Write 5 Sent Done Done false
9074 3560 Acked Flush 0 Sent Done Done false
9075 3561 Acked Read 9 Sent Done Done false
9076 3562 Acked Read 5 Sent Done Done false
9077 3563 Acked Read 3 Sent Done Done false
9078 3564 Acked Read 3 Sent Done Done false
9079 3565 Acked Read 7 Sent Done Done false
9080 3566 Acked Read 5 Sent Done Done false
9081 3567 Acked Write 8 Sent Done Done false
9082 3568 Acked Write 1 Sent Done Done false
9083 3569 Acked Write 7 Sent Done Done false
9084 3570 Acked Write 6 Sent Done Done false
9085 3571 Acked Read 5 Sent Done Done false
9086 3572 Acked Write 2 Sent Done Done false
9087 3573 Acked Write 2 Sent Done Done false
9088 3574 Acked Flush 0 Sent Done Done false
9089 3575 Acked Read 8 Sent Done Done false
9090 3576 Acked Read 10 Sent Done Done false
9091 3577 Acked Read 7 Sent Done Done false
9092 3578 Acked Read 1 Sent Done Done false
9093 3579 Acked Flush 0 Sent Done Done false
9094 3580 Acked Read 5 Sent Done Done false
9095 3581 Acked Write 6 Sent Done Done false
9096 3582 Acked Write 10 Sent Done Done false
9097 3583 Acked Write 8 Sent Done Done false
9098 3584 Acked Read 8 Sent Done Done false
9099 3585 Acked Read 1 Sent Done Done false
9100 3586 Acked Read 3 Sent Done Done false
9101 3587 Acked Flush 0 Sent Done Done false
9102 3588 Acked Read 4 Sent Done Done false
9103 3589 Acked Read 4 Sent Done Done false
9104 3590 Acked Read 2 Sent Done Done false
9105 3591 Acked Write 7 Sent Done Done false
9106 3592 Acked Read 9 Sent Done Done false
9107 3593 Acked Read 3 Sent Done Done false
9108 3594 Acked Write 4 Sent Done Done false
9109 3595 Acked Write 4 Sent Done Done false
9110 3596 Acked Read 8 Sent Done Done false
9111 3597 Acked Write 2 Sent Done Done false
9112 3598 Acked Read 8 Sent Done Done false
9113 3599 Acked Read 9 Sent Done Done false
9114 3600 Acked Read 8 Sent Done Done false
9115 3601 Acked Read 10 Sent Done Done false
9116 3602 Acked Read 4 Sent Done Done false
9117 3603 Acked Read 9 Sent Done Done false
9118 3604 Acked Write 4 Sent Done Done false
9119 3605 Acked Read 1 Sent Done Done false
9120 3606 Acked Read 8 Sent Done Done false
9121 3607 Acked Read 5 Sent Done Done false
9122 3608 Acked Write 9 Sent Done Done false
9123 3609 Acked Write 8 Sent Done Done false
9124 3610 Acked Read 6 Sent Done Done false
9125 3611 Acked Read 3 Sent Done Done false
9126 3612 Acked Flush 0 Sent Done Done false
9127 3613 Acked Write 3 Sent Done Done false
9128 3614 Acked Write 7 Sent Done Done false
9129 3615 Acked Write 8 Sent Done Done false
9130 3616 Acked Write 4 Sent Done Done false
9131 3617 Acked Read 9 Sent Done Done false
9132 3618 Acked Read 8 Sent Done Done false
9133 3619 Acked Write 1 Sent Done Done false
9134 3620 Acked Read 10 Sent Done Done false
9135 3621 Acked Write 6 Sent Done Done false
9136 3622 Acked Read 1 Sent Done Done false
9137 3623 Acked Read 5 Sent Done Done false
9138 3624 Acked Write 6 Sent Done Done false
9139 3625 Acked Read 1 Sent Done Done false
9140 3626 Acked Write 2 Sent Done Done false
9141 3627 Acked Flush 0 Sent Done Done false
9142 3628 Acked Write 2 Sent Done Done false
9143 3629 Acked Write 9 Sent Done Done false
9144 3630 Acked Read 10 Sent Done Done false
9145 3631 Acked Write 3 Sent Done Done false
9146 3632 Acked Read 4 Sent Done Done false
9147 3633 Acked Write 3 Sent Done Done false
9148 3634 Acked Read 5 Sent Done Done false
9149 3635 Acked Write 7 Sent Done Done false
9150 3636 Acked Read 7 Sent Done Done false
9151 3637 Acked Read 2 Sent Done Done false
9152 3638 Acked Read 6 Sent Done Done false
9153 3639 Acked Read 2 Sent Done Done false
9154 3640 Acked Read 8 Sent Done Done false
9155 3641 Acked Read 3 Sent Done Done false
9156 3642 Acked Read 6 Sent Done Done false
9157 3643 Acked Write 10 Sent Done Done false
9158 3644 Acked Write 10 Sent Done Done false
9159 3645 Acked Read 1 Sent Done Done false
9160 3646 Acked Write 2 Sent Done Done false
9161 3647 Acked Read 7 Sent Done Done false
9162 3648 Acked Write 7 Sent Done Done false
9163 3649 Acked Write 5 Sent Done Done false
9164 3650 Acked Write 8 Sent Done Done false
9165 3651 Acked Read 4 Sent Done Done false
9166 3652 Acked Write 9 Sent Done Done false
9167 3653 Acked Read 1 Sent Done Done false
9168 3654 Acked Read 2 Sent Done Done false
9169 3655 Acked Read 9 Sent Done Done false
9170 3656 Acked Flush 0 Sent Done Done false
9171 3657 Acked Write 10 Sent Done Done false
9172 3658 Acked Read 9 Sent Done Done false
9173 3659 Acked Read 8 Sent Done Done false
9174 3660 Acked Read 9 Sent Done Done false
9175 3661 Acked Write 3 Sent Done Done false
9176 3662 Acked Write 7 Sent Done Done false
9177 3663 Acked Write 6 Sent Done Done false
9178 3664 Acked Read 2 Sent Done Done false
9179 3665 Acked Read 10 Sent Done Done false
9180 3666 Acked Flush 0 Sent Done Done false
9181 3667 Acked Write 9 Sent Done Done false
9182 3668 Acked Read 4 Sent Done Done false
9183 3669 Acked Write 7 Sent Done Done false
9184 3670 Acked Read 3 Sent Done Done false
9185 3671 Acked Read 10 Sent Done Done false
9186 3672 Acked Read 8 Sent Done Done false
9187 3673 Acked Write 5 Sent Done Done false
9188 3674 Acked Write 9 Sent Done Done false
9189 3675 Acked Write 10 Sent Done Done false
9190 3676 Acked Write 8 Sent Done Done false
9191 3677 Acked Read 4 Sent Done Done false
9192 3678 Acked Read 7 Sent Done Done false
9193 3679 Acked Write 5 Sent Done Done false
9194 3680 Acked Read 3 Sent Done Done false
9195 3681 Acked Flush 0 Sent Done Done false
9196 3682 Acked Write 7 Sent Done Done false
9197 3683 Acked Write 4 Sent Done Done false
9198 3684 Acked Write 7 Sent Done Done false
9199 3685 Acked Flush 0 Sent Done Done false
9200 3686 Acked Write 7 Sent Done Done false
9201 3687 Acked Read 5 Sent Done Done false
9202 3688 Acked Write 9 Sent Done Done false
9203 3689 Acked Read 6 Sent Done Done false
9204 3690 Acked Read 9 Sent Done Done false
9205 3691 Acked Write 7 Sent Done Done false
9206 3692 Acked Write 9 Sent Done Done false
9207 3693 Acked Read 1 Sent Done Done false
9208 3694 Acked Read 3 Sent Done Done false
9209 3695 Acked Write 9 Sent Done Done false
9210 3696 Acked Read 8 Sent Done Done false
9211 3697 Acked Read 6 Sent Done Done false
9212 3698 Acked Read 10 Sent Done Done false
9213 3699 Acked Read 2 Sent Done Done false
9214 3700 Acked Write 2 Sent Done Done false
9215 3701 Acked Read 5 Sent Done Done false
9216 3702 Acked Write 6 Sent Done Done false
9217 3703 Acked Read 8 Sent Done Done false
9218 3704 Acked Read 4 Sent Done Done false
9219 3705 Acked Read 10 Sent Done Done false
9220 3706 Acked Read 6 Sent Done Done false
9221 3707 Acked Write 1 Sent Done Done false
9222 3708 Acked Read 3 Sent Done Done false
9223 3709 Acked Write 2 Sent Done Done false
9224 3710 Acked Write 4 Sent Done Done false
9225 3711 Acked Read 6 Sent Done Done false
9226 3712 Acked Write 4 Sent Done Done false
9227 3713 Acked Read 7 Sent Done Done false
9228 3714 Acked Write 8 Sent Done Done false
9229 3715 Acked Read 6 Sent Done Done false
9230 STATES DS:0 DS:1 DS:2 TOTAL
9231 Sent 301 0 0 301
9232 Done 0 301 301 602
9233 Skipped 0 0 0 0
9234 Error 0 0 0 0
9235 Last Flush: 3414 3685 3685
9236 Downstairs last five completed: 3414 3413 3412 3411 3410
9237 Upstairs last five completed: 3715 3714 3713 3712 3711
9238 CLIENT: Up:0 ds:301 act:2
9239 [8810] got start action
9240 [8810] I am going to restart
9241 Starting downstairs at port 8810
9242 Make output file at "/tmp/dsc/downstairs-8810.txt"
9243 [8810][0] reports Starting
9244 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1209)
9245 [8810][0] reports Running
9246 ----------------------------------------------------------------
9247 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
9248 States: Offline Active Active
9249 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9250 3415 Acked Read 7 Sent Done Done false
9251 3416 Acked Read 9 Sent Done Done false
9252 3417 Acked Write 7 Sent Done Done false
9253 3418 Acked Read 4 Sent Done Done false
9254 3419 Acked Read 2 Sent Done Done false
9255 3420 Acked Write 1 Sent Done Done false
9256 3421 Acked Write 4 Sent Done Done false
9257 3422 Acked Read 3 Sent Done Done false
9258 3423 Acked Read 8 Sent Done Done false
9259 3424 Acked Write 2 Sent Done Done false
9260 3425 Acked Flush 0 Sent Done Done false
9261 3426 Acked Read 1 Sent Done Done false
9262 3427 Acked Read 5 Sent Done Done false
9263 3428 Acked Read 3 Sent Done Done false
9264 3429 Acked Write 9 Sent Done Done false
9265 3430 Acked Read 7 Sent Done Done false
9266 3431 Acked Read 4 Sent Done Done false
9267 3432 Acked Read 6 Sent Done Done false
9268 3433 Acked Write 5 Sent Done Done false
9269 3434 Acked Read 10 Sent Done Done false
9270 3435 Acked Write 10 Sent Done Done false
9271 3436 Acked Write 7 Sent Done Done false
9272 3437 Acked Flush 0 Sent Done Done false
9273 3438 Acked Write 7 Sent Done Done false
9274 3439 Acked Flush 0 Sent Done Done false
9275 3440 Acked Read 6 Sent Done Done false
9276 3441 Acked Read 6 Sent Done Done false
9277 3442 Acked Read 1 Sent Done Done false
9278 3443 Acked Read 4 Sent Done Done false
9279 3444 Acked Write 2 Sent Done Done false
9280 3445 Acked Write 5 Sent Done Done false
9281 3446 Acked Read 2 Sent Done Done false
9282 3447 Acked Write 1 Sent Done Done false
9283 3448 Acked Flush 0 Sent Done Done false
9284 3449 Acked Read 4 Sent Done Done false
9285 3450 Acked Read 8 Sent Done Done false
9286 3451 Acked Write 3 Sent Done Done false
9287 3452 Acked Read 8 Sent Done Done false
9288 3453 Acked Read 7 Sent Done Done false
9289 3454 Acked Read 5 Sent Done Done false
9290 3455 Acked Read 10 Sent Done Done false
9291 3456 Acked Write 1 Sent Done Done false
9292 3457 Acked Read 9 Sent Done Done false
9293 3458 Acked Read 10 Sent Done Done false
9294 3459 Acked Read 7 Sent Done Done false
9295 3460 Acked Write 1 Sent Done Done false
9296 3461 Acked Read 2 Sent Done Done false
9297 3462 Acked Read 1 Sent Done Done false
9298 3463 Acked Read 9 Sent Done Done false
9299 3464 Acked Write 3 Sent Done Done false
9300 3465 Acked Write 7 Sent Done Done false
9301 3466 Acked Read 9 Sent Done Done false
9302 3467 Acked Read 8 Sent Done Done false
9303 3468 Acked Flush 0 Sent Done Done false
9304 3469 Acked Read 4 Sent Done Done false
9305 3470 Acked Write 5 Sent Done Done false
9306 3471 Acked Write 9 Sent Done Done false
9307 3472 Acked Write 7 Sent Done Done false
9308 3473 Acked Write 9 Sent Done Done false
9309 3474 Acked Read 6 Sent Done Done false
9310 3475 Acked Read 4 Sent Done Done false
9311 3476 Acked Write 6 Sent Done Done false
9312 3477 Acked Read 9 Sent Done Done false
9313 3478 Acked Write 9 Sent Done Done false
9314 3479 Acked Read 3 Sent Done Done false
9315 3480 Acked Read 6 Sent Done Done false
9316 3481 Acked Read 10 Sent Done Done false
9317 3482 Acked Write 1 Sent Done Done false
9318 3483 Acked Flush 0 Sent Done Done false
9319 3484 Acked Read 10 Sent Done Done false
9320 3485 Acked Write 2 Sent Done Done false
9321 3486 Acked Flush 0 Sent Done Done false
9322 3487 Acked Write 8 Sent Done Done false
9323 3488 Acked Read 9 Sent Done Done false
9324 3489 Acked Read 4 Sent Done Done false
9325 3490 Acked Read 3 Sent Done Done false
9326 3491 Acked Read 5 Sent Done Done false
9327 3492 Acked Write 6 Sent Done Done false
9328 3493 Acked Flush 0 Sent Done Done false
9329 3494 Acked Read 1 Sent Done Done false
9330 3495 Acked Flush 0 Sent Done Done false
9331 3496 Acked Write 9 Sent Done Done false
9332 3497 Acked Flush 0 Sent Done Done false
9333 3498 Acked Write 7 Sent Done Done false
9334 3499 Acked Write 5 Sent Done Done false
9335 3500 Acked Write 3 Sent Done Done false
9336 3501 Acked Write 1 Sent Done Done false
9337 3502 Acked Read 1 Sent Done Done false
9338 3503 Acked Read 7 Sent Done Done false
9339 3504 Acked Read 2 Sent Done Done false
9340 3505 Acked Read 3 Sent Done Done false
9341 3506 Acked Write 3 Sent Done Done false
9342 3507 Acked Read 5 Sent Done Done false
9343 3508 Acked Read 9 Sent Done Done false
9344 3509 Acked Read 4 Sent Done Done false
9345 3510 Acked Read 1 Sent Done Done false
9346 3511 Acked Write 8 Sent Done Done false
9347 3512 Acked Read 8 Sent Done Done false
9348 3513 Acked Read 8 Sent Done Done false
9349 3514 Acked Write 2 Sent Done Done false
9350 3515 Acked Write 5 Sent Done Done false
9351 3516 Acked Write 6 Sent Done Done false
9352 3517 Acked Read 3 Sent Done Done false
9353 3518 Acked Write 5 Sent Done Done false
9354 3519 Acked Read 8 Sent Done Done false
9355 3520 Acked Write 2 Sent Done Done false
9356 3521 Acked Write 7 Sent Done Done false
9357 3522 Acked Flush 0 Sent Done Done false
9358 3523 Acked Flush 0 Sent Done Done false
9359 3524 Acked Write 9 Sent Done Done false
9360 3525 Acked Write 7 Sent Done Done false
9361 3526 Acked Write 9 Sent Done Done false
9362 3527 Acked Write 9 Sent Done Done false
9363 3528 Acked Flush 0 Sent Done Done false
9364 3529 Acked Write 2 Sent Done Done false
9365 3530 Acked Flush 0 Sent Done Done false
9366 3531 Acked Write 2 Sent Done Done false
9367 3532 Acked Read 2 Sent Done Done false
9368 3533 Acked Read 4 Sent Done Done false
9369 3534 Acked Write 6 Sent Done Done false
9370 3535 Acked Read 9 Sent Done Done false
9371 3536 Acked Read 10 Sent Done Done false
9372 3537 Acked Write 8 Sent Done Done false
9373 3538 Acked Read 5 Sent Done Done false
9374 3539 Acked Write 2 Sent Done Done false
9375 3540 Acked Read 7 Sent Done Done false
9376 3541 Acked Read 7 Sent Done Done false
9377 3542 Acked Read 7 Sent Done Done false
9378 3543 Acked Read 8 Sent Done Done false
9379 3544 Acked Read 3 Sent Done Done false
9380 3545 Acked Read 4 Sent Done Done false
9381 3546 Acked Write 9 Sent Done Done false
9382 3547 Acked Read 2 Sent Done Done false
9383 3548 Acked Write 5 Sent Done Done false
9384 3549 Acked Read 2 Sent Done Done false
9385 3550 Acked Read 9 Sent Done Done false
9386 3551 Acked Read 4 Sent Done Done false
9387 3552 Acked Read 5 Sent Done Done false
9388 3553 Acked Write 4 Sent Done Done false
9389 3554 Acked Write 1 Sent Done Done false
9390 3555 Acked Read 9 Sent Done Done false
9391 3556 Acked Write 6 Sent Done Done false
9392 3557 Acked Flush 0 Sent Done Done false
9393 3558 Acked Read 6 Sent Done Done false
9394 3559 Acked Write 5 Sent Done Done false
9395 3560 Acked Flush 0 Sent Done Done false
9396 3561 Acked Read 9 Sent Done Done false
9397 3562 Acked Read 5 Sent Done Done false
9398 3563 Acked Read 3 Sent Done Done false
9399 3564 Acked Read 3 Sent Done Done false
9400 3565 Acked Read 7 Sent Done Done false
9401 3566 Acked Read 5 Sent Done Done false
9402 3567 Acked Write 8 Sent Done Done false
9403 3568 Acked Write 1 Sent Done Done false
9404 3569 Acked Write 7 Sent Done Done false
9405 3570 Acked Write 6 Sent Done Done false
9406 3571 Acked Read 5 Sent Done Done false
9407 3572 Acked Write 2 Sent Done Done false
9408 3573 Acked Write 2 Sent Done Done false
9409 3574 Acked Flush 0 Sent Done Done false
9410 3575 Acked Read 8 Sent Done Done false
9411 3576 Acked Read 10 Sent Done Done false
9412 3577 Acked Read 7 Sent Done Done false
9413 3578 Acked Read 1 Sent Done Done false
9414 3579 Acked Flush 0 Sent Done Done false
9415 3580 Acked Read 5 Sent Done Done false
9416 3581 Acked Write 6 Sent Done Done false
9417 3582 Acked Write 10 Sent Done Done false
9418 3583 Acked Write 8 Sent Done Done false
9419 3584 Acked Read 8 Sent Done Done false
9420 3585 Acked Read 1 Sent Done Done false
9421 3586 Acked Read 3 Sent Done Done false
9422 3587 Acked Flush 0 Sent Done Done false
9423 3588 Acked Read 4 Sent Done Done false
9424 3589 Acked Read 4 Sent Done Done false
9425 3590 Acked Read 2 Sent Done Done false
9426 3591 Acked Write 7 Sent Done Done false
9427 3592 Acked Read 9 Sent Done Done false
9428 3593 Acked Read 3 Sent Done Done false
9429 3594 Acked Write 4 Sent Done Done false
9430 3595 Acked Write 4 Sent Done Done false
9431 3596 Acked Read 8 Sent Done Done false
9432 3597 Acked Write 2 Sent Done Done false
9433 3598 Acked Read 8 Sent Done Done false
9434 3599 Acked Read 9 Sent Done Done false
9435 3600 Acked Read 8 Sent Done Done false
9436 3601 Acked Read 10 Sent Done Done false
9437 3602 Acked Read 4 Sent Done Done false
9438 3603 Acked Read 9 Sent Done Done false
9439 3604 Acked Write 4 Sent Done Done false
9440 3605 Acked Read 1 Sent Done Done false
9441 3606 Acked Read 8 Sent Done Done false
9442 3607 Acked Read 5 Sent Done Done false
9443 3608 Acked Write 9 Sent Done Done false
9444 3609 Acked Write 8 Sent Done Done false
9445 3610 Acked Read 6 Sent Done Done false
9446 3611 Acked Read 3 Sent Done Done false
9447 3612 Acked Flush 0 Sent Done Done false
9448 3613 Acked Write 3 Sent Done Done false
9449 3614 Acked Write 7 Sent Done Done false
9450 3615 Acked Write 8 Sent Done Done false
9451 3616 Acked Write 4 Sent Done Done false
9452 3617 Acked Read 9 Sent Done Done false
9453 3618 Acked Read 8 Sent Done Done false
9454 3619 Acked Write 1 Sent Done Done false
9455 3620 Acked Read 10 Sent Done Done false
9456 3621 Acked Write 6 Sent Done Done false
9457 3622 Acked Read 1 Sent Done Done false
9458 3623 Acked Read 5 Sent Done Done false
9459 3624 Acked Write 6 Sent Done Done false
9460 3625 Acked Read 1 Sent Done Done false
9461 3626 Acked Write 2 Sent Done Done false
9462 3627 Acked Flush 0 Sent Done Done false
9463 3628 Acked Write 2 Sent Done Done false
9464 3629 Acked Write 9 Sent Done Done false
9465 3630 Acked Read 10 Sent Done Done false
9466 3631 Acked Write 3 Sent Done Done false
9467 3632 Acked Read 4 Sent Done Done false
9468 3633 Acked Write 3 Sent Done Done false
9469 3634 Acked Read 5 Sent Done Done false
9470 3635 Acked Write 7 Sent Done Done false
9471 3636 Acked Read 7 Sent Done Done false
9472 3637 Acked Read 2 Sent Done Done false
9473 3638 Acked Read 6 Sent Done Done false
9474 3639 Acked Read 2 Sent Done Done false
9475 3640 Acked Read 8 Sent Done Done false
9476 3641 Acked Read 3 Sent Done Done false
9477 3642 Acked Read 6 Sent Done Done false
9478 3643 Acked Write 10 Sent Done Done false
9479 3644 Acked Write 10 Sent Done Done false
9480 3645 Acked Read 1 Sent Done Done false
9481 3646 Acked Write 2 Sent Done Done false
9482 3647 Acked Read 7 Sent Done Done false
9483 3648 Acked Write 7 Sent Done Done false
9484 3649 Acked Write 5 Sent Done Done false
9485 3650 Acked Write 8 Sent Done Done false
9486 3651 Acked Read 4 Sent Done Done false
9487 3652 Acked Write 9 Sent Done Done false
9488 3653 Acked Read 1 Sent Done Done false
9489 3654 Acked Read 2 Sent Done Done false
9490 3655 Acked Read 9 Sent Done Done false
9491 3656 Acked Flush 0 Sent Done Done false
9492 3657 Acked Write 10 Sent Done Done false
9493 3658 Acked Read 9 Sent Done Done false
9494 3659 Acked Read 8 Sent Done Done false
9495 3660 Acked Read 9 Sent Done Done false
9496 3661 Acked Write 3 Sent Done Done false
9497 3662 Acked Write 7 Sent Done Done false
9498 3663 Acked Write 6 Sent Done Done false
9499 3664 Acked Read 2 Sent Done Done false
9500 3665 Acked Read 10 Sent Done Done false
9501 3666 Acked Flush 0 Sent Done Done false
9502 3667 Acked Write 9 Sent Done Done false
9503 3668 Acked Read 4 Sent Done Done false
9504 3669 Acked Write 7 Sent Done Done false
9505 3670 Acked Read 3 Sent Done Done false
9506 3671 Acked Read 10 Sent Done Done false
9507 3672 Acked Read 8 Sent Done Done false
9508 3673 Acked Write 5 Sent Done Done false
9509 3674 Acked Write 9 Sent Done Done false
9510 3675 Acked Write 10 Sent Done Done false
9511 3676 Acked Write 8 Sent Done Done false
9512 3677 Acked Read 4 Sent Done Done false
9513 3678 Acked Read 7 Sent Done Done false
9514 3679 Acked Write 5 Sent Done Done false
9515 3680 Acked Read 3 Sent Done Done false
9516 3681 Acked Flush 0 Sent Done Done false
9517 3682 Acked Write 7 Sent Done Done false
9518 3683 Acked Write 4 Sent Done Done false
9519 3684 Acked Write 7 Sent Done Done false
9520 3685 Acked Flush 0 Sent Done Done false
9521 3686 Acked Write 7 Sent Done Done false
9522 3687 Acked Read 5 Sent Done Done false
9523 3688 Acked Write 9 Sent Done Done false
9524 3689 Acked Read 6 Sent Done Done false
9525 3690 Acked Read 9 Sent Done Done false
9526 3691 Acked Write 7 Sent Done Done false
9527 3692 Acked Write 9 Sent Done Done false
9528 3693 Acked Read 1 Sent Done Done false
9529 3694 Acked Read 3 Sent Done Done false
9530 3695 Acked Write 9 Sent Done Done false
9531 3696 Acked Read 8 Sent Done Done false
9532 3697 Acked Read 6 Sent Done Done false
9533 3698 Acked Read 10 Sent Done Done false
9534 3699 Acked Read 2 Sent Done Done false
9535 3700 Acked Write 2 Sent Done Done false
9536 3701 Acked Read 5 Sent Done Done false
9537 3702 Acked Write 6 Sent Done Done false
9538 3703 Acked Read 8 Sent Done Done false
9539 3704 Acked Read 4 Sent Done Done false
9540 3705 Acked Read 10 Sent Done Done false
9541 3706 Acked Read 6 Sent Done Done false
9542 3707 Acked Write 1 Sent Done Done false
9543 3708 Acked Read 3 Sent Done Done false
9544 3709 Acked Write 2 Sent Done Done false
9545 3710 Acked Write 4 Sent Done Done false
9546 3711 Acked Read 6 Sent Done Done false
9547 3712 Acked Write 4 Sent Done Done false
9548 3713 Acked Read 7 Sent Done Done false
9549 3714 Acked Write 8 Sent Done Done false
9550 3715 Acked Read 6 Sent Done Done false
9551 3716 Acked Flush 0 Sent Done Done false
9552 STATES DS:0 DS:1 DS:2 TOTAL
9553 Sent 302 0 0 302
9554 Done 0 302 302 604
9555 Skipped 0 0 0 0
9556 Error 0 0 0 0
9557 Last Flush: 3414 3716 3716
9558 Downstairs last five completed: 3414 3413 3412 3411 3410
9559 Upstairs last five completed: 3716 3715 3714 3713 3712
9560 CLIENT: Up:0 ds:302 act:2
95612025-01-21T15:49:09.306ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
95622025-01-21T15:49:09.307ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64356) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
95632025-01-21T15:49:09.308ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
95642025-01-21T15:49:09.308ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
95652025-01-21T15:49:09.308ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
95662025-01-21T15:49:09.308ZINFOcrucible: send last flush ID to this DS: 3414 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
95672025-01-21T15:49:09.308ZINFOcrucible: Replied this last flush ID: 3414 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
95682025-01-21T15:49:09.308ZINFOcrucible: [0] client re-new 302 jobs since flush 3414 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
95692025-01-21T15:49:09.309ZINFOcrucible: [0] Replayed 302 jobs since flush: 3414 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
9570 ----------------------------------------------------------------
9571 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9572 Downstairs last five completed: 3716 3715 3714 3713 3712
9573 Upstairs last five completed: 3716 3715 3714 3713 3712
9574 CLIENT: Up:0 ds:0 act:3
9575 Replay: All jobs finished
9576 Main task has work to do, go find it
9577 got dsc Stop(1)
9578 stop 1
9579 Jan 21 15:49:11.560[8820] Got stop action so:false kr:false
9580 INFO request completed, latency_us: 346, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 0c9db390-96db-491e-b8d5-04e9e16122ed, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
95812025-01-21T15:49:11.560ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
9582 Jan 21 15:49:11.560 INFO request completed, latency_us: 105, response_code: 200, uri: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:49:11.560839173Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
9583 /state/cid/1, method: GET, req_id: eb26657e-6e14-4bd7-9d03-14cc9874e815, remote_addr: 127{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:49:11.560864362Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
9584 .0.0.1:52945, local_addr: 127.0.0.1:9998
95852025-01-21T15:49:11.560ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
9586 [8820] Exited with: ExitStatus(unix_wait_status(9))
9587 [8820][1] reports Exit
9588 Jan 21 15:49:15.562 INFO request completed, latency_us: 223, response_code: 200, uri: /state/cid/1, method: GET, req_id: dd34d98d-dd93-49cb-9679-43ada9da9765, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
9589 001/300 Read block 3762 len 16384
9590 002/300 Write block 332 len 20480 data: 2 2 2 2 2
9591 003/300 Read block 1062 len 36864
9592 004/300 Read block 1716 len 24576
9593 005/300 Write block 3449 len 36864 data: 2 3 3 3 3 3 3 3 2
9594 006/300 Write block 1296 len 24576 data: 3 3 3 4 4 4
9595 007/300 Read block 1800 len 4096
9596 008/300 Read block 3474 len 16384
9597 009/300 Read block 4571 len 20480
9598 010/300 Read block 2422 len 28672
9599 011/300 Write block 3327 len 20480 data: 2 2 2 2 2
9600 012/300 Read block 757 len 32768
9601 013/300 Flush
9602 014/300 Write block 167 len 32768 data: 2 2 2 2 2 3 3 3
9603 015/300 Read block 1672 len 28672
9604 016/300 Read block 3774 len 12288
9605 017/300 Read block 4226 len 8192
9606 018/300 Read block 747 len 8192
9607 019/300 Write block 4897 len 36864 data: 4 3 3 3 2 2 3 3 3
9608 020/300 Write block 3519 len 40960 data: 4 3 3 3 4 4 4 5 5 5
9609 021/300 Write block 138 len 8192 data: 3 3
9610 022/300 Flush
9611 023/300 Read block 2498 len 8192
9612 024/300 Read block 4224 len 28672
9613 025/300 Write block 4873 len 32768 data: 4 4 3 3 2 2 2 2
9614 026/300 Write block 982 len 32768 data: 2 3 3 3 2 2 2 2
9615 027/300 Read block 2131 len 4096
9616 028/300 Read block 2960 len 32768
9617 029/300 Read block 2720 len 4096
9618 030/300 Write block 849 len 16384 data: 2 2 2 2
9619 031/300 Write block 1712 len 4096 data: 3
9620 032/300 Read block 1890 len 24576
9621 033/300 Read block 3841 len 12288
9622 034/300 Read block 2659 len 36864
9623 035/300 Write block 1230 len 8192 data: 2 2
9624 036/300 Flush
9625 037/300 Write block 2849 len 4096 data: 2
9626 038/300 Write block 2828 len 16384 data: 4 3 3 3
9627 039/300 Read block 3807 len 8192
9628 040/300 Write block 3382 len 8192 data: 3 3
9629 041/300 Write block 1813 len 28672 data: 5 4 4 4 4 4 3
9630 042/300 Read block 624 len 20480
9631 043/300 Write block 4733 len 4096 data: 3
9632 044/300 Write block 4986 len 4096 data: 7
9633 045/300 Write block 336 len 8192 data: 3 3
9634 046/300 Read block 1246 len 36864
9635 047/300 Write block 2286 len 40960 data: 2 2 3 3 3 3 3 3 3 3
9636 048/300 Write block 2952 len 32768 data: 4 3 3 3 4 4 4 4
9637 049/300 Read block 2014 len 32768
9638 050/300 Write block 4855 len 12288 data: 5 5 5
9639 051/300 Flush
9640 052/300 Read block 4376 len 28672
9641 053/300 Write block 1612 len 12288 data: 2 2 2
9642 054/300 Write block 4446 len 40960 data: 4 3 3 3 3 3 3 3 3 3
9643 055/300 Write block 284 len 32768 data: 4 4 4 3 3 3 3 2
9644 056/300 Write block 4416 len 28672 data: 4 3 3 2 2 2 3
9645 057/300 Write block 1224 len 24576 data: 3 3 3 3 2 2
9646 058/300 Write block 4646 len 16384 data: 3 2 2 3
9647 059/300 Read block 3092 len 12288
9648 060/300 Flush
9649 061/300 Read block 2991 len 8192
9650 062/300 Write block 2908 len 16384 data: 4 5 5 5
9651 063/300 Read block 995 len 24576
9652 064/300 Read block 877 len 16384
9653 065/300 Read block 4714 len 32768
9654 066/300 Write block 4499 len 32768 data: 2 2 2 2 2 2 2 2
9655 067/300 Flush
9656 068/300 Read block 3129 len 32768
9657 069/300 Read block 461 len 28672
9658 070/300 Write block 2366 len 24576 data: 2 2 3 3 3 2
9659 071/300 Write block 678 len 16384 data: 2 2 2 2
9660 072/300 Read block 2140 len 32768
9661 073/300 Read block 4693 len 36864
9662 074/300 Write block 3568 len 4096 data: 4
9663 075/300 Write block 940 len 12288 data: 3 4 4
9664 076/300 Write block 4637 len 4096 data: 3
9665 077/300 Read block 3478 len 24576
9666 078/300 Read block 4634 len 16384
9667 079/300 Write block 1682 len 32768 data: 4 5 5 4 4 4 4 4
9668 080/300 Write block 749 len 16384 data: 4 4 2 2
9669 081/300 Read block 2756 len 4096
9670 082/300 Write block 3087 len 20480 data: 4 4 4 4 3
9671 083/300 Read block 801 len 40960
9672 084/300 Write block 761 len 20480 data: 3 3 3 2 2
9673 085/300 Write block 2622 len 12288 data: 4 4 4
9674 086/300 Write block 1982 len 36864 data: 5 6 7 7 6 6 5 5 4
9675 087/300 Flush
9676 088/300 Read block 2535 len 32768
9677 089/300 Read block 4364 len 24576
9678 090/300 Write block 1849 len 12288 data: 4 4 4
9679 091/300 Write block 2095 len 20480 data: 3 3 3 3 3
9680 092/300 Read block 4907 len 4096
9681 093/300 Read block 2141 len 20480
9682 094/300 Read block 798 len 36864
9683 095/300 Write block 1505 len 28672 data: 3 2 2 2 2 3 4
9684 096/300 Write block 1757 len 32768 data: 6 5 3 3 3 3 3 4
9685 097/300 Write block 314 len 8192 data: 2 3
9686 098/300 Write block 4269 len 24576 data: 4 4 4 4 3 3
9687 099/300 Read block 3521 len 20480
9688 100/300 Flush
9689 101/300 Write block 39 len 16384 data: 4 4 4 5
9690 102/300 Read block 3149 len 20480
9691 103/300 Read block 3949 len 40960
9692 104/300 Flush
9693 105/300 Write block 2832 len 4096 data: 4
9694 106/300 Read block 3539 len 28672
9695 107/300 Flush
9696 108/300 Read block 3087 len 20480
9697 109/300 Write block 838 len 12288 data: 2 4 3
9698 110/300 Read block 2792 len 4096
9699 111/300 Write block 3082 len 8192 data: 5 5
9700 112/300 Read block 4960 len 32768
9701 113/300 Read block 821 len 16384
9702 114/300 Read block 1049 len 20480
9703 115/300 Read block 1320 len 28672
9704 116/300 Write block 3422 len 12288 data: 4 3 3
9705 117/300 Read block 578 len 4096
9706 118/300 Write block 3680 len 20480 data: 2 3 3 3 2
9707 119/300 Write block 592 len 20480 data: 4 5 4 4 3
9708 120/300 Write block 176 len 4096 data: 3
9709 121/300 Write block 3719 len 20480 data: 4 4 5 5 4
9710 122/300 Write block 2132 len 16384 data: 3 3 3 3
9711 123/300 Write block 183 len 32768 data: 3 3 3 4 5 6 5 4
9712 124/300 Read block 1535 len 28672
9713 125/300 Flush
9714 126/300 Write block 4876 len 4096 data: 4
9715 127/300 Flush
9716 128/300 Read block 2651 len 40960
9717 129/300 Write block 3422 len 16384 data: 5 4 4 3
9718 130/300 Flush
9719 131/300 Flush
9720 132/300 Read block 4481 len 24576
9721 133/300 Read block 4980 len 20480
9722 134/300 Write block 2365 len 8192 data: 2 3
9723 135/300 Write block 4608 len 8192 data: 3 3
9724 136/300 Read block 4648 len 40960
9725 137/300 Read block 3299 len 8192
9726 138/300 Read block 1709 len 36864
9727 139/300 Write block 3090 len 36864 data: 5 4 3 4 5 3 3 3 3
9728 140/300 Write block 1835 len 36864 data: 2 2 3 3 4 4 4 4 5
9729 141/300 Read block 1437 len 36864
9730 142/300 Write block 4017 len 24576 data: 2 2 2 2 2 2
9731 143/300 Write block 1102 len 32768 data: 4 4 5 6 4 4 4 4
9732 144/300 Read block 1759 len 20480
9733 145/300 Write block 15 len 4096 data: 3
9734 146/300 Read block 4017 len 8192
9735 147/300 Flush
9736 148/300 Read block 1298 len 12288
9737 149/300 Read block 3560 len 16384
9738 150/300 Read block 2423 len 20480
9739 151/300 Read block 3453 len 24576
9740 152/300 Read block 3280 len 28672
9741 153/300 Read block 3523 len 32768
9742 154/300 Read block 4152 len 36864
9743 155/300 Read block 1002 len 32768
9744 156/300 Read block 2517 len 20480
9745 157/300 Read block 3711 len 8192
9746 158/300 Read block 503 len 8192
9747 159/300 Read block 3067 len 12288
9748 160/300 Write block 2085 len 32768 data: 2 2 2 3 2 2 2 2
9749 161/300 Read block 1653 len 24576
9750 162/300 Read block 2371 len 24576
9751 163/300 Write block 603 len 28672 data: 4 3 3 2 2 2 2
9752 164/300 Read block 1386 len 8192
9753 165/300 Read block 2845 len 8192
9754 166/300 Write block 695 len 16384 data: 5 6 5 5
9755 167/300 Read block 677 len 36864
9756 168/300 Flush
9757 169/300 Write block 4955 len 8192 data: 4 4
9758 170/300 Write block 4621 len 28672 data: 4 4 4 5 5 5 5
9759 171/300 Read block 673 len 8192
9760 172/300 Write block 4754 len 40960 data: 3 4 4 4 5 5 4 4 5 3
9761 173/300 Write block 1452 len 16384 data: 3 3 3 3
9762 174/300 Read block 1944 len 16384
9763 175/300 Write block 4343 len 4096 data: 4
9764 176/300 Write block 2791 len 36864 data: 5 5 4 5 4 3 3 3 3
9765 177/300 Read block 2464 len 16384
9766 178/300 Read block 4229 len 32768
9767 179/300 Read block 3341 len 16384
9768 180/300 Read block 3522 len 24576
9769 181/300 Read block 4662 len 12288
9770 182/300 Write block 2643 len 32768 data: 2 2 2 2 2 2 2 2
9771 183/300 Read block 4590 len 12288
9772 184/300 Write block 4099 len 24576 data: 3 4 3 3 3 2
9773 185/300 Flush
9774 186/300 Write block 3827 len 8192 data: 2 2
9775 187/300 Write block 4836 len 8192 data: 3 4
9776 188/300 Read block 4530 len 24576
9777 189/300 Write block 517 len 28672 data: 3 3 3 3 3 2 2
9778 190/300 Write block 3147 len 20480 data: 3 4 5 4 4
9779 191/300 Write block 802 len 28672 data: 4 3 4 4 4 4 4
9780 192/300 Read block 2579 len 8192
9781 193/300 Flush
9782 194/300 Read block 4104 len 12288
9783 195/300 Write block 3 len 32768 data: 2 2 2 2 2 2 2 2
9784 196/300 Flush
9785 197/300 Read block 2300 len 8192
9786 198/300 Read block 4433 len 36864
9787 199/300 Write block 1154 len 32768 data: 4 4 6 7 7 7 6 5
9788 200/300 Read block 862 len 20480
9789 201/300 Write block 988 len 36864 data: 3 3 2 2 2 3 3 4 4
9790 202/300 Read block 1882 len 8192
9791 203/300 Read block 4964 len 28672
9792 204/300 Flush
9793 205/300 Read block 4561 len 36864
9794 206/300 Read block 230 len 4096
9795 207/300 Write block 2344 len 4096 data: 2
9796 208/300 Write block 4348 len 28672 data: 4 4 4 4 3 3 3
9797 209/300 Write block 3529 len 4096 data: 3
9798 210/300 Read block 334 len 36864
9799 211/300 Write block 3162 len 8192 data: 3 3
9800 212/300 Write block 1331 len 32768 data: 3 3 4 3 3 3 3 3
9801 213/300 Read block 3548 len 24576
9802 214/300 Read block 3202 len 36864
9803 215/300 Read block 2806 len 24576
9804 216/300 Read block 2250 len 4096
9805 217/300 Write block 4755 len 8192 data: 5 5
9806 218/300 Write block 4132 len 36864 data: 4 3 2 2 2 3 3 3 3
9807 219/300 Write block 508 len 4096 data: 2
9808 220/300 Flush
9809 221/300 Read block 2241 len 28672
9810 222/300 Flush
9811 223/300 Read block 2551 len 20480
9812 224/300 Read block 1741 len 20480
9813 225/300 Write block 2977 len 40960 data: 2 2 2 2 2 2 2 3 3 3
9814 226/300 Read block 2162 len 12288
9815 227/300 Write block 386 len 28672 data: 2 2 2 2 2 2 2
9816 228/300 Write block 606 len 40960 data: 3 3 3 3 2 2 2 2 3 3
9817 229/300 Write block 4933 len 40960 data: 3 3 3 2 2 2 2 2 2 3
9818 230/300 Write block 4631 len 12288 data: 4 5 4
9819 231/300 Read block 958 len 28672
9820 232/300 Flush
9821 233/300 Write block 1904 len 40960 data: 6 6 6 7 7 7 4 5 5 4
9822 234/300 Read block 1332 len 16384
9823 235/300 Write block 3304 len 4096 data: 3
9824 236/300 Write block 1325 len 32768 data: 6 5 4 3 3 3 4 4
9825 237/300 Flush
9826 238/300 Read block 1354 len 12288
9827 239/300 Read block 3005 len 16384
9828 240/300 Read block 2748 len 24576
9829 241/300 Read block 959 len 12288
9830 242/300 Read block 3210 len 20480
9831 243/300 Write block 2446 len 8192 data: 4 5
9832 244/300 Flush
9833 245/300 Read block 337 len 16384
9834 246/300 Read block 3916 len 24576
9835 247/300 Write block 933 len 28672 data: 2 2 2 2 4 4 4
9836 248/300 Write block 2429 len 36864 data: 4 3 3 2 2 2 2 2 2
9837 249/300 Read block 1416 len 32768
9838 250/300 Read block 3192 len 16384
9839 251/300 Write block 931 len 12288 data: 4 3 3
9840 252/300 Write block 1134 len 4096 data: 2
9841 253/300 Read block 1389 len 24576
9842 254/300 Write block 3575 len 12288 data: 4 3 3
9843 255/300 Write block 4631 len 36864 data: 5 6 5 3 3 3 4 4 3
9844 256/300 Read block 4629 len 4096
9845 257/300 Flush
9846 258/300 Write block 3187 len 4096 data: 3
9847 259/300 Read block 204 len 24576
9848 260/300 Write block 3721 len 20480 data: 6 6 5 4 5
9849 261/300 Write block 4607 len 36864 data: 3 4 4 2 2 2 2 2 2
9850 262/300 Flush
9851 263/300 Write block 1324 len 12288 data: 6 7 6
9852 264/300 Read block 652 len 16384
9853 265/300 Read block 4328 len 32768
9854 266/300 Read block 2069 len 24576
9855 267/300 Write block 671 len 28672 data: 2 2 2 2 2 2 2
9856 268/300 Read block 2070 len 36864
9857 269/300 Read block 1670 len 40960
9858 270/300 Read block 3568 len 32768
9859 271/300 Read block 4481 len 40960
9860 272/300 Write block 541 len 36864 data: 3 2 2 2 2 3 3 3 2
9861 273/300 Write block 2169 len 24576 data: 4 4 5 7 5 5
9862 274/300 Write block 1499 len 20480 data: 2 2 2 2 2
9863 275/300 Read block 4369 len 40960
9864 276/300 Read block 4202 len 28672
9865 277/300 Write block 4382 len 16384 data: 4 4 3 3
9866 278/300 Write block 1049 len 36864 data: 3 4 4 4 4 4 4 5 6
9867 279/300 Read block 2116 len 12288
9868 280/300 Read block 453 len 40960
9869 281/300 Read block 4344 len 4096
9870 282/300 Flush
9871 283/300 Read block 2850 len 40960
9872 284/300 Read block 2971 len 28672
9873 285/300 Read block 3307 len 24576
9874 286/300 Write block 1715 len 24576 data: 2 2 2 2 3 3
9875 287/300 Write block 3476 len 28672 data: 5 6 3 3 3 3 3
9876 288/300 Write block 4544 len 4096 data: 3
9877 289/300 Read block 693 len 32768
9878 290/300 Flush
9879 291/300 Write block 1986 len 36864 data: 7 7 6 6 5 4 3 4 3
9880 292/300 Write block 351 len 32768 data: 3 3 3 3 3 3 3 4
9881 293/300 Write block 2433 len 20480 data: 3 3 3 3 3
9882 294/300 Read block 3094 len 4096
9883 295/300 Write block 1616 len 36864 data: 2 2 3 3 5 5 4 4 3
9884 296/300 Write block 4515 len 40960 data: 3 3 3 3 4 3 3 3 3 3
9885 297/300 Read block 1371 len 32768
9886 298/300 Read block 2896 len 36864
9887 299/300 Flush
9888 300/300 Write block 1421 len 8192 data: 3 3
9889 301/300 Read block 1208 len 8192
9890 Main task has work to do, go find it
9891 got dsc Start(1)
9892 start 1
9893 Jan 21 15:49:15.794 INFO request completed, latency_us: 201, response_code: 204, uri: /start/cid/1, method: POST, req_id: 602ed865-35f3-42a2-a73c-52d56c0f6eeb, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
9894 Replay: started 1, returned:Ok(())
9895 ----------------------------------------------------------------
9896 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9897 States: Active Offline Active
9898 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9899 3717 Acked Read 4 Done Sent Done false
9900 3718 Acked Write 5 Done Sent Done false
9901 3719 Acked Read 9 Done Sent Done false
9902 3720 Acked Read 6 Done Sent Done false
9903 3721 Acked Write 9 Done Sent Done false
9904 3722 Acked Write 6 Done Sent Done false
9905 3723 Acked Read 1 Done Sent Done false
9906 3724 Acked Read 4 Done Sent Done false
9907 3725 Acked Read 5 Done Sent Done false
9908 3726 Acked Read 7 Done Sent Done false
9909 3727 Acked Write 5 Done Sent Done false
9910 3728 Acked Read 8 Done Sent Done false
9911 3729 Acked Flush 0 Done Sent Done false
9912 3730 Acked Write 8 Done Sent Done false
9913 3731 Acked Read 7 Done Sent Done false
9914 3732 Acked Read 3 Done Sent Done false
9915 3733 Acked Read 2 Done Sent Done false
9916 3734 Acked Read 2 Done Sent Done false
9917 3735 Acked Write 9 Done Sent Done false
9918 3736 Acked Write 10 Done Sent Done false
9919 3737 Acked Write 2 Done Sent Done false
9920 3738 Acked Flush 0 Done Sent Done false
9921 3739 Acked Read 2 Done Sent Done false
9922 3740 Acked Read 7 Done Sent Done false
9923 3741 Acked Write 8 Done Sent Done false
9924 3742 Acked Write 8 Done Sent Done false
9925 3743 Acked Read 1 Done Sent Done false
9926 3744 Acked Read 8 Done Sent Done false
9927 3745 Acked Read 1 Done Sent Done false
9928 3746 Acked Write 4 Done Sent Done false
9929 3747 Acked Write 1 Done Sent Done false
9930 3748 Acked Read 6 Done Sent Done false
9931 3749 Acked Read 3 Done Sent Done false
9932 3750 Acked Read 9 Done Sent Done false
9933 3751 Acked Write 2 Done Sent Done false
9934 3752 Acked Flush 0 Done Sent Done false
9935 3753 Acked Write 1 Done Sent Done false
9936 3754 Acked Write 4 Done Sent Done false
9937 3755 Acked Read 2 Done Sent Done false
9938 3756 Acked Write 2 Done Sent Done false
9939 3757 Acked Write 7 Done Sent Done false
9940 3758 Acked Read 5 Done Sent Done false
9941 3759 Acked Write 1 Done Sent Done false
9942 3760 Acked Write 1 Done Sent Done false
9943 3761 Acked Write 2 Done Sent Done false
9944 3762 Acked Read 9 Done Sent Done false
9945 3763 Acked Write 10 Done Sent Done false
9946 3764 Acked Write 8 Done Sent Done false
9947 3765 Acked Read 8 Done Sent Done false
9948 3766 Acked Write 3 Done Sent Done false
9949 3767 Acked Flush 0 Done Sent Done false
9950 3768 Acked Read 7 Done Sent Done false
9951 3769 Acked Write 3 Done Sent Done false
9952 3770 Acked Write 10 Done Sent Done false
9953 3771 Acked Write 8 Done Sent Done false
9954 3772 Acked Write 7 Done Sent Done false
9955 3773 Acked Write 6 Done Sent Done false
9956 3774 Acked Write 4 Done Sent Done false
9957 3775 Acked Read 3 Done Sent Done false
9958 3776 Acked Flush 0 Done Sent Done false
9959 3777 Acked Read 2 Done Sent Done false
9960 3778 Acked Write 4 Done Sent Done false
9961 3779 Acked Read 6 Done Sent Done false
9962 3780 Acked Read 4 Done Sent Done false
9963 3781 Acked Read 8 Done Sent Done false
9964 3782 Acked Write 8 Done Sent Done false
9965 3783 Acked Flush 0 Done Sent Done false
9966 3784 Acked Read 8 Done Sent Done false
9967 3785 Acked Read 7 Done Sent Done false
9968 3786 Acked Write 6 Done Sent Done false
9969 3787 Acked Write 4 Done Sent Done false
9970 3788 Acked Read 8 Done Sent Done false
9971 3789 Acked Read 9 Done Sent Done false
9972 3790 Acked Write 1 Done Sent Done false
9973 3791 Acked Write 3 Done Sent Done false
9974 3792 Acked Write 1 Done Sent Done false
9975 3793 Acked Read 6 Done Sent Done false
9976 3794 Acked Read 4 Done Sent Done false
9977 3795 Acked Write 8 Done Sent Done false
9978 3796 Acked Write 4 Done Sent Done false
9979 3797 Acked Read 1 Done Sent Done false
9980 3798 Acked Write 5 Done Sent Done false
9981 3799 Acked Read 10 Done Sent Done false
9982 3800 Acked Write 5 Done Sent Done false
9983 3801 Acked Write 3 Done Sent Done false
9984 3802 Acked Write 9 Done Sent Done false
9985 3803 Acked Flush 0 Done Sent Done false
9986 3804 Acked Read 8 Done Sent Done false
9987 3805 Acked Read 6 Done Sent Done false
9988 3806 Acked Write 3 Done Sent Done false
9989 3807 Acked Write 5 Done Sent Done false
9990 3808 Acked Read 1 Done Sent Done false
9991 3809 Acked Read 5 Done Sent Done false
9992 3810 Acked Read 9 Done Sent Done false
9993 3811 Acked Write 7 Done Sent Done false
9994 3812 Acked Write 8 Done Sent Done false
9995 3813 Acked Write 2 Done Sent Done false
9996 3814 Acked Write 6 Done Sent Done false
9997 3815 Acked Read 5 Done Sent Done false
9998 3816 Acked Flush 0 Done Sent Done false
9999 3817 Acked Write 4 Done Sent Done false
10000 3818 Acked Read 5 Done Sent Done false
10001 3819 Acked Read 10 Done Sent Done false
10002 3820 Acked Flush 0 Done Sent Done false
10003 3821 Acked Write 1 Done Sent Done false
10004 3822 Acked Read 7 Done Sent Done false
10005 3823 Acked Flush 0 Done Sent Done false
10006 3824 Acked Read 5 Done Sent Done false
10007 3825 Acked Write 3 Done Sent Done false
10008 3826 Acked Read 1 Done Sent Done false
10009 3827 Acked Write 2 Done Sent Done false
10010 3828 Acked Read 8 Done Sent Done false
10011 3829 Acked Read 4 Done Sent Done false
10012 3830 Acked Read 5 Done Sent Done false
10013 3831 Acked Read 7 Done Sent Done false
10014 3832 Acked Write 3 Done Sent Done false
10015 3833 Acked Read 1 Done Sent Done false
10016 3834 Acked Write 5 Done Sent Done false
10017 3835 Acked Write 5 Done Sent Done false
10018 3836 Acked Write 1 Done Sent Done false
10019 3837 Acked Write 5 Done Sent Done false
10020 3838 Acked Write 4 Done Sent Done false
10021 3839 Acked Write 8 Done Sent Done false
10022 3840 Acked Read 7 Done Sent Done false
10023 3841 Acked Flush 0 Done Sent Done false
10024 3842 Acked Write 1 Done Sent Done false
10025 3843 Acked Flush 0 Done Sent Done false
10026 3844 Acked Read 10 Done Sent Done false
10027 3845 Acked Write 4 Done Sent Done false
10028 3846 Acked Flush 0 Done Sent Done false
10029 3847 Acked Flush 0 Done Sent Done false
10030 3848 Acked Read 6 Done Sent Done false
10031 3849 Acked Read 5 Done Sent Done false
10032 3850 Acked Write 2 Done Sent Done false
10033 3851 Acked Write 2 Done Sent Done false
10034 3852 Acked Read 10 Done Sent Done false
10035 3853 Acked Read 2 Done Sent Done false
10036 3854 Acked Read 9 Done Sent Done false
10037 3855 Acked Write 9 Done Sent Done false
10038 3856 Acked Write 9 Done Sent Done false
10039 3857 Acked Read 9 Done Sent Done false
10040 3858 Acked Write 6 Done Sent Done false
10041 3859 Acked Write 8 Done Sent Done false
10042 3860 Acked Read 5 Done Sent Done false
10043 3861 Acked Write 1 Done Sent Done false
10044 3862 Acked Read 2 Done Sent Done false
10045 3863 Acked Flush 0 Done Sent Done false
10046 3864 Acked Read 3 Done Sent Done false
10047 3865 Acked Read 4 Done Sent Done false
10048 3866 Acked Read 5 Done Sent Done false
10049 3867 Acked Read 6 Done Sent Done false
10050 3868 Acked Read 7 Done Sent Done false
10051 3869 Acked Read 8 Done Sent Done false
10052 3870 Acked Read 9 Done Sent Done false
10053 3871 Acked Read 8 Done Sent Done false
10054 3872 Acked Read 5 Done Sent Done false
10055 3873 Acked Read 2 Done Sent Done false
10056 3874 Acked Read 2 Done Sent Done false
10057 3875 Acked Read 3 Done Sent Done false
10058 3876 Acked Write 8 Done Sent Done false
10059 3877 Acked Read 6 Done Sent Done false
10060 3878 Acked Read 6 Done Sent Done false
10061 3879 Acked Write 7 Done Sent Done false
10062 3880 Acked Read 2 Done Sent Done false
10063 3881 Acked Read 2 Done Sent Done false
10064 3882 Acked Write 4 Done Sent Done false
10065 3883 Acked Read 9 Done Sent Done false
10066 3884 Acked Flush 0 Done Sent Done false
10067 3885 Acked Write 2 Done Sent Done false
10068 3886 Acked Write 7 Done Sent Done false
10069 3887 Acked Read 2 Done Sent Done false
10070 3888 Acked Write 10 Done Sent Done false
10071 3889 Acked Write 4 Done Sent Done false
10072 3890 Acked Read 4 Done Sent Done false
10073 3891 Acked Write 1 Done Sent Done false
10074 3892 Acked Write 9 Done Sent Done false
10075 3893 Acked Read 4 Done Sent Done false
10076 3894 Acked Read 8 Done Sent Done false
10077 3895 Acked Read 4 Done Sent Done false
10078 3896 Acked Read 6 Done Sent Done false
10079 3897 Acked Read 3 Done Sent Done false
10080 3898 Acked Write 8 Done Sent Done false
10081 3899 Acked Read 3 Done Sent Done false
10082 3900 Acked Write 6 Done Sent Done false
10083 3901 Acked Flush 0 Done Sent Done false
10084 3902 Acked Write 2 Done Sent Done false
10085 3903 Acked Write 2 Done Sent Done false
10086 3904 Acked Read 6 Done Sent Done false
10087 3905 Acked Write 7 Done Sent Done false
10088 3906 Acked Write 5 Done Sent Done false
10089 3907 Acked Write 7 Done Sent Done false
10090 3908 Acked Read 2 Done Sent Done false
10091 3909 Acked Flush 0 Done Sent Done false
10092 3910 Acked Read 3 Done Sent Done false
10093 3911 Acked Write 8 Done Sent Done false
10094 3912 Acked Flush 0 Done Sent Done false
10095 3913 Acked Read 2 Done Sent Done false
10096 3914 Acked Read 9 Done Sent Done false
10097 3915 Acked Write 8 Done Sent Done false
10098 3916 Acked Read 5 Done Sent Done false
10099 3917 Acked Write 9 Done Sent Done false
10100 3918 Acked Read 2 Done Sent Done false
10101 3919 Acked Read 7 Done Sent Done false
10102 3920 Acked Flush 0 Done Sent Done false
10103 3921 Acked Read 9 Done Sent Done false
10104 3922 Acked Read 1 Done Sent Done false
10105 3923 Acked Write 1 Done Sent Done false
10106 3924 Acked Write 7 Done Sent Done false
10107 3925 Acked Write 1 Done Sent Done false
10108 3926 Acked Read 9 Done Sent Done false
10109 3927 Acked Write 2 Done Sent Done false
10110 3928 Acked Write 8 Done Sent Done false
10111 3929 Acked Read 6 Done Sent Done false
10112 3930 Acked Read 9 Done Sent Done false
10113 3931 Acked Read 6 Done Sent Done false
10114 3932 Acked Read 1 Done Sent Done false
10115 3933 Acked Write 2 Done Sent Done false
10116 3934 Acked Write 9 Done Sent Done false
10117 3935 Acked Write 1 Done Sent Done false
10118 3936 Acked Flush 0 Done Sent Done false
10119 3937 Acked Read 7 Done Sent Done false
10120 3938 Acked Flush 0 Done Sent Done false
10121 3939 Acked Read 5 Done Sent Done false
10122 3940 Acked Read 5 Done Sent Done false
10123 3941 Acked Write 10 Done Sent Done false
10124 3942 Acked Read 3 Done Sent Done false
10125 3943 Acked Write 7 Done Sent Done false
10126 3944 Acked Write 10 Done Sent Done false
10127 3945 Acked Write 10 Done Sent Done false
10128 3946 Acked Write 3 Done Sent Done false
10129 3947 Acked Read 7 Done Sent Done false
10130 3948 Acked Flush 0 Done Sent Done false
10131 3949 Acked Write 10 Done Sent Done false
10132 3950 Acked Read 4 Done Sent Done false
10133 3951 Acked Write 1 Done Sent Done false
10134 3952 Acked Write 8 Done Sent Done false
10135 3953 Acked Flush 0 Done Sent Done false
10136 3954 Acked Read 3 Done Sent Done false
10137 3955 Acked Read 4 Done Sent Done false
10138 3956 Acked Read 6 Done Sent Done false
10139 3957 Acked Read 3 Done Sent Done false
10140 3958 Acked Read 5 Done Sent Done false
10141 3959 Acked Write 2 Done Sent Done false
10142 3960 Acked Flush 0 Done Sent Done false
10143 3961 Acked Read 4 Done Sent Done false
10144 3962 Acked Read 6 Done Sent Done false
10145 3963 Acked Write 7 Done Sent Done false
10146 3964 Acked Write 9 Done Sent Done false
10147 3965 Acked Read 8 Done Sent Done false
10148 3966 Acked Read 4 Done Sent Done false
10149 3967 Acked Write 3 Done Sent Done false
10150 3968 Acked Write 1 Done Sent Done false
10151 3969 Acked Read 6 Done Sent Done false
10152 3970 Acked Write 3 Done Sent Done false
10153 3971 Acked Write 9 Done Sent Done false
10154 3972 Acked Read 1 Done Sent Done false
10155 3973 Acked Flush 0 Done Sent Done false
10156 3974 Acked Write 1 Done Sent Done false
10157 3975 Acked Read 6 Done Sent Done false
10158 3976 Acked Write 5 Done Sent Done false
10159 3977 Acked Write 9 Done Sent Done false
10160 3978 Acked Flush 0 Done Sent Done false
10161 3979 Acked Write 3 Done Sent Done false
10162 3980 Acked Read 4 Done Sent Done false
10163 3981 Acked Read 8 Done Sent Done false
10164 3982 Acked Read 6 Done Sent Done false
10165 3983 Acked Write 7 Done Sent Done false
10166 3984 Acked Read 9 Done Sent Done false
10167 3985 Acked Read 10 Done Sent Done false
10168 3986 Acked Read 8 Done Sent Done false
10169 3987 Acked Read 10 Done Sent Done false
10170 3988 Acked Write 9 Done Sent Done false
10171 3989 Acked Write 6 Done Sent Done false
10172 3990 Acked Write 5 Done Sent Done false
10173 3991 Acked Read 10 Done Sent Done false
10174 3992 Acked Read 7 Done Sent Done false
10175 3993 Acked Write 4 Done Sent Done false
10176 3994 Acked Write 9 Done Sent Done false
10177 3995 Acked Read 3 Done Sent Done false
10178 3996 Acked Read 10 Done Sent Done false
10179 3997 Acked Read 1 Done Sent Done false
10180 3998 Acked Flush 0 Done Sent Done false
10181 3999 Acked Read 10 Done Sent Done false
10182 4000 Acked Read 7 Done Sent Done false
10183 4001 Acked Read 6 Done Sent Done false
10184 4002 Acked Write 6 Done Sent Done false
10185 4003 Acked Write 7 Done Sent Done false
10186 4004 Acked Write 1 Done Sent Done false
10187 4005 Acked Read 8 Done Sent Done false
10188 4006 Acked Flush 0 Done Sent Done false
10189 4007 Acked Write 9 Done Sent Done false
10190 4008 Acked Write 8 Done Sent Done false
10191 4009 Acked Write 5 Done Sent Done false
10192 4010 Acked Read 1 Done Sent Done false
10193 4011 Acked Write 9 Done Sent Done false
10194 4012 Acked Write 10 Done Sent Done false
10195 4013 Acked Read 8 Done Sent Done false
10196 4014 Acked Read 9 Done Sent Done false
10197 4015 Acked Flush 0 Done Sent Done false
10198 4016 Acked Write 2 Done Sent Done false
10199 4017 Acked Read 2 Done Sent Done false
10200 STATES DS:0 DS:1 DS:2 TOTAL
10201 Sent 0 301 0 301
10202 Done 301 0 301 602
10203 Skipped 0 0 0 0
10204 Error 0 0 0 0
10205 Last Flush: 4015 3716 4015
10206 Downstairs last five completed: 3716 3715 3714 3713 3712
10207 Upstairs last five completed: 4017 4016 4015 4014 4013
10208 CLIENT: Up:0 ds:301 act:2
10209 [8820] got start action
10210 [8820] I am going to restart
10211 Starting downstairs at port 8820
10212 Make output file at "/tmp/dsc/downstairs-8820.txt"
10213 [8820][1] reports Starting
10214 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1210)
10215 [8820][1] reports Running
10216 ----------------------------------------------------------------
10217 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10218 States: Active Offline Active
10219 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10220 3717 Acked Read 4 Done Sent Done false
10221 3718 Acked Write 5 Done Sent Done false
10222 3719 Acked Read 9 Done Sent Done false
10223 3720 Acked Read 6 Done Sent Done false
10224 3721 Acked Write 9 Done Sent Done false
10225 3722 Acked Write 6 Done Sent Done false
10226 3723 Acked Read 1 Done Sent Done false
10227 3724 Acked Read 4 Done Sent Done false
10228 3725 Acked Read 5 Done Sent Done false
10229 3726 Acked Read 7 Done Sent Done false
10230 3727 Acked Write 5 Done Sent Done false
10231 3728 Acked Read 8 Done Sent Done false
10232 3729 Acked Flush 0 Done Sent Done false
10233 3730 Acked Write 8 Done Sent Done false
10234 3731 Acked Read 7 Done Sent Done false
10235 3732 Acked Read 3 Done Sent Done false
10236 3733 Acked Read 2 Done Sent Done false
10237 3734 Acked Read 2 Done Sent Done false
10238 3735 Acked Write 9 Done Sent Done false
10239 3736 Acked Write 10 Done Sent Done false
10240 3737 Acked Write 2 Done Sent Done false
10241 3738 Acked Flush 0 Done Sent Done false
10242 3739 Acked Read 2 Done Sent Done false
10243 3740 Acked Read 7 Done Sent Done false
10244 3741 Acked Write 8 Done Sent Done false
10245 3742 Acked Write 8 Done Sent Done false
10246 3743 Acked Read 1 Done Sent Done false
10247 3744 Acked Read 8 Done Sent Done false
10248 3745 Acked Read 1 Done Sent Done false
10249 3746 Acked Write 4 Done Sent Done false
10250 3747 Acked Write 1 Done Sent Done false
10251 3748 Acked Read 6 Done Sent Done false
10252 3749 Acked Read 3 Done Sent Done false
10253 3750 Acked Read 9 Done Sent Done false
10254 3751 Acked Write 2 Done Sent Done false
10255 3752 Acked Flush 0 Done Sent Done false
10256 3753 Acked Write 1 Done Sent Done false
10257 3754 Acked Write 4 Done Sent Done false
10258 3755 Acked Read 2 Done Sent Done false
10259 3756 Acked Write 2 Done Sent Done false
10260 3757 Acked Write 7 Done Sent Done false
10261 3758 Acked Read 5 Done Sent Done false
10262 3759 Acked Write 1 Done Sent Done false
10263 3760 Acked Write 1 Done Sent Done false
10264 3761 Acked Write 2 Done Sent Done false
10265 3762 Acked Read 9 Done Sent Done false
10266 3763 Acked Write 10 Done Sent Done false
10267 3764 Acked Write 8 Done Sent Done false
10268 3765 Acked Read 8 Done Sent Done false
10269 3766 Acked Write 3 Done Sent Done false
10270 3767 Acked Flush 0 Done Sent Done false
10271 3768 Acked Read 7 Done Sent Done false
10272 3769 Acked Write 3 Done Sent Done false
10273 3770 Acked Write 10 Done Sent Done false
10274 3771 Acked Write 8 Done Sent Done false
10275 3772 Acked Write 7 Done Sent Done false
10276 3773 Acked Write 6 Done Sent Done false
10277 3774 Acked Write 4 Done Sent Done false
10278 3775 Acked Read 3 Done Sent Done false
10279 3776 Acked Flush 0 Done Sent Done false
10280 3777 Acked Read 2 Done Sent Done false
10281 3778 Acked Write 4 Done Sent Done false
10282 3779 Acked Read 6 Done Sent Done false
10283 3780 Acked Read 4 Done Sent Done false
10284 3781 Acked Read 8 Done Sent Done false
10285 3782 Acked Write 8 Done Sent Done false
10286 3783 Acked Flush 0 Done Sent Done false
10287 3784 Acked Read 8 Done Sent Done false
10288 3785 Acked Read 7 Done Sent Done false
10289 3786 Acked Write 6 Done Sent Done false
10290 3787 Acked Write 4 Done Sent Done false
10291 3788 Acked Read 8 Done Sent Done false
10292 3789 Acked Read 9 Done Sent Done false
10293 3790 Acked Write 1 Done Sent Done false
10294 3791 Acked Write 3 Done Sent Done false
10295 3792 Acked Write 1 Done Sent Done false
10296 3793 Acked Read 6 Done Sent Done false
10297 3794 Acked Read 4 Done Sent Done false
10298 3795 Acked Write 8 Done Sent Done false
10299 3796 Acked Write 4 Done Sent Done false
10300 3797 Acked Read 1 Done Sent Done false
10301 3798 Acked Write 5 Done Sent Done false
10302 3799 Acked Read 10 Done Sent Done false
10303 3800 Acked Write 5 Done Sent Done false
10304 3801 Acked Write 3 Done Sent Done false
10305 3802 Acked Write 9 Done Sent Done false
10306 3803 Acked Flush 0 Done Sent Done false
10307 3804 Acked Read 8 Done Sent Done false
10308 3805 Acked Read 6 Done Sent Done false
10309 3806 Acked Write 3 Done Sent Done false
10310 3807 Acked Write 5 Done Sent Done false
10311 3808 Acked Read 1 Done Sent Done false
10312 3809 Acked Read 5 Done Sent Done false
10313 3810 Acked Read 9 Done Sent Done false
10314 3811 Acked Write 7 Done Sent Done false
10315 3812 Acked Write 8 Done Sent Done false
10316 3813 Acked Write 2 Done Sent Done false
10317 3814 Acked Write 6 Done Sent Done false
10318 3815 Acked Read 5 Done Sent Done false
10319 3816 Acked Flush 0 Done Sent Done false
10320 3817 Acked Write 4 Done Sent Done false
10321 3818 Acked Read 5 Done Sent Done false
10322 3819 Acked Read 10 Done Sent Done false
10323 3820 Acked Flush 0 Done Sent Done false
10324 3821 Acked Write 1 Done Sent Done false
10325 3822 Acked Read 7 Done Sent Done false
10326 3823 Acked Flush 0 Done Sent Done false
10327 3824 Acked Read 5 Done Sent Done false
10328 3825 Acked Write 3 Done Sent Done false
10329 3826 Acked Read 1 Done Sent Done false
10330 3827 Acked Write 2 Done Sent Done false
10331 3828 Acked Read 8 Done Sent Done false
10332 3829 Acked Read 4 Done Sent Done false
10333 3830 Acked Read 5 Done Sent Done false
10334 3831 Acked Read 7 Done Sent Done false
10335 3832 Acked Write 3 Done Sent Done false
10336 3833 Acked Read 1 Done Sent Done false
10337 3834 Acked Write 5 Done Sent Done false
10338 3835 Acked Write 5 Done Sent Done false
10339 3836 Acked Write 1 Done Sent Done false
10340 3837 Acked Write 5 Done Sent Done false
10341 3838 Acked Write 4 Done Sent Done false
10342 3839 Acked Write 8 Done Sent Done false
10343 3840 Acked Read 7 Done Sent Done false
10344 3841 Acked Flush 0 Done Sent Done false
10345 3842 Acked Write 1 Done Sent Done false
10346 3843 Acked Flush 0 Done Sent Done false
10347 3844 Acked Read 10 Done Sent Done false
10348 3845 Acked Write 4 Done Sent Done false
10349 3846 Acked Flush 0 Done Sent Done false
10350 3847 Acked Flush 0 Done Sent Done false
10351 3848 Acked Read 6 Done Sent Done false
10352 3849 Acked Read 5 Done Sent Done false
10353 3850 Acked Write 2 Done Sent Done false
10354 3851 Acked Write 2 Done Sent Done false
10355 3852 Acked Read 10 Done Sent Done false
10356 3853 Acked Read 2 Done Sent Done false
10357 3854 Acked Read 9 Done Sent Done false
10358 3855 Acked Write 9 Done Sent Done false
10359 3856 Acked Write 9 Done Sent Done false
10360 3857 Acked Read 9 Done Sent Done false
10361 3858 Acked Write 6 Done Sent Done false
10362 3859 Acked Write 8 Done Sent Done false
10363 3860 Acked Read 5 Done Sent Done false
10364 3861 Acked Write 1 Done Sent Done false
10365 3862 Acked Read 2 Done Sent Done false
10366 3863 Acked Flush 0 Done Sent Done false
10367 3864 Acked Read 3 Done Sent Done false
10368 3865 Acked Read 4 Done Sent Done false
10369 3866 Acked Read 5 Done Sent Done false
10370 3867 Acked Read 6 Done Sent Done false
10371 3868 Acked Read 7 Done Sent Done false
10372 3869 Acked Read 8 Done Sent Done false
10373 3870 Acked Read 9 Done Sent Done false
10374 3871 Acked Read 8 Done Sent Done false
10375 3872 Acked Read 5 Done Sent Done false
10376 3873 Acked Read 2 Done Sent Done false
10377 3874 Acked Read 2 Done Sent Done false
10378 3875 Acked Read 3 Done Sent Done false
10379 3876 Acked Write 8 Done Sent Done false
10380 3877 Acked Read 6 Done Sent Done false
10381 3878 Acked Read 6 Done Sent Done false
10382 3879 Acked Write 7 Done Sent Done false
10383 3880 Acked Read 2 Done Sent Done false
10384 3881 Acked Read 2 Done Sent Done false
10385 3882 Acked Write 4 Done Sent Done false
10386 3883 Acked Read 9 Done Sent Done false
10387 3884 Acked Flush 0 Done Sent Done false
10388 3885 Acked Write 2 Done Sent Done false
10389 3886 Acked Write 7 Done Sent Done false
10390 3887 Acked Read 2 Done Sent Done false
10391 3888 Acked Write 10 Done Sent Done false
10392 3889 Acked Write 4 Done Sent Done false
10393 3890 Acked Read 4 Done Sent Done false
10394 3891 Acked Write 1 Done Sent Done false
10395 3892 Acked Write 9 Done Sent Done false
10396 3893 Acked Read 4 Done Sent Done false
10397 3894 Acked Read 8 Done Sent Done false
10398 3895 Acked Read 4 Done Sent Done false
10399 3896 Acked Read 6 Done Sent Done false
10400 3897 Acked Read 3 Done Sent Done false
10401 3898 Acked Write 8 Done Sent Done false
10402 3899 Acked Read 3 Done Sent Done false
10403 3900 Acked Write 6 Done Sent Done false
10404 3901 Acked Flush 0 Done Sent Done false
10405 3902 Acked Write 2 Done Sent Done false
10406 3903 Acked Write 2 Done Sent Done false
10407 3904 Acked Read 6 Done Sent Done false
10408 3905 Acked Write 7 Done Sent Done false
10409 3906 Acked Write 5 Done Sent Done false
10410 3907 Acked Write 7 Done Sent Done false
10411 3908 Acked Read 2 Done Sent Done false
10412 3909 Acked Flush 0 Done Sent Done false
10413 3910 Acked Read 3 Done Sent Done false
10414 3911 Acked Write 8 Done Sent Done false
10415 3912 Acked Flush 0 Done Sent Done false
10416 3913 Acked Read 2 Done Sent Done false
10417 3914 Acked Read 9 Done Sent Done false
10418 3915 Acked Write 8 Done Sent Done false
10419 3916 Acked Read 5 Done Sent Done false
10420 3917 Acked Write 9 Done Sent Done false
10421 3918 Acked Read 2 Done Sent Done false
10422 3919 Acked Read 7 Done Sent Done false
10423 3920 Acked Flush 0 Done Sent Done false
10424 3921 Acked Read 9 Done Sent Done false
10425 3922 Acked Read 1 Done Sent Done false
10426 3923 Acked Write 1 Done Sent Done false
10427 3924 Acked Write 7 Done Sent Done false
10428 3925 Acked Write 1 Done Sent Done false
10429 3926 Acked Read 9 Done Sent Done false
10430 3927 Acked Write 2 Done Sent Done false
10431 3928 Acked Write 8 Done Sent Done false
10432 3929 Acked Read 6 Done Sent Done false
10433 3930 Acked Read 9 Done Sent Done false
10434 3931 Acked Read 6 Done Sent Done false
10435 3932 Acked Read 1 Done Sent Done false
10436 3933 Acked Write 2 Done Sent Done false
10437 3934 Acked Write 9 Done Sent Done false
10438 3935 Acked Write 1 Done Sent Done false
10439 3936 Acked Flush 0 Done Sent Done false
10440 3937 Acked Read 7 Done Sent Done false
10441 3938 Acked Flush 0 Done Sent Done false
10442 3939 Acked Read 5 Done Sent Done false
10443 3940 Acked Read 5 Done Sent Done false
10444 3941 Acked Write 10 Done Sent Done false
10445 3942 Acked Read 3 Done Sent Done false
10446 3943 Acked Write 7 Done Sent Done false
10447 3944 Acked Write 10 Done Sent Done false
10448 3945 Acked Write 10 Done Sent Done false
10449 3946 Acked Write 3 Done Sent Done false
10450 3947 Acked Read 7 Done Sent Done false
10451 3948 Acked Flush 0 Done Sent Done false
10452 3949 Acked Write 10 Done Sent Done false
10453 3950 Acked Read 4 Done Sent Done false
10454 3951 Acked Write 1 Done Sent Done false
10455 3952 Acked Write 8 Done Sent Done false
10456 3953 Acked Flush 0 Done Sent Done false
10457 3954 Acked Read 3 Done Sent Done false
10458 3955 Acked Read 4 Done Sent Done false
10459 3956 Acked Read 6 Done Sent Done false
10460 3957 Acked Read 3 Done Sent Done false
10461 3958 Acked Read 5 Done Sent Done false
10462 3959 Acked Write 2 Done Sent Done false
10463 3960 Acked Flush 0 Done Sent Done false
10464 3961 Acked Read 4 Done Sent Done false
10465 3962 Acked Read 6 Done Sent Done false
10466 3963 Acked Write 7 Done Sent Done false
10467 3964 Acked Write 9 Done Sent Done false
10468 3965 Acked Read 8 Done Sent Done false
10469 3966 Acked Read 4 Done Sent Done false
10470 3967 Acked Write 3 Done Sent Done false
10471 3968 Acked Write 1 Done Sent Done false
10472 3969 Acked Read 6 Done Sent Done false
10473 3970 Acked Write 3 Done Sent Done false
10474 3971 Acked Write 9 Done Sent Done false
10475 3972 Acked Read 1 Done Sent Done false
10476 3973 Acked Flush 0 Done Sent Done false
10477 3974 Acked Write 1 Done Sent Done false
10478 3975 Acked Read 6 Done Sent Done false
10479 3976 Acked Write 5 Done Sent Done false
10480 3977 Acked Write 9 Done Sent Done false
10481 3978 Acked Flush 0 Done Sent Done false
10482 3979 Acked Write 3 Done Sent Done false
10483 3980 Acked Read 4 Done Sent Done false
10484 3981 Acked Read 8 Done Sent Done false
10485 3982 Acked Read 6 Done Sent Done false
10486 3983 Acked Write 7 Done Sent Done false
10487 3984 Acked Read 9 Done Sent Done false
10488 3985 Acked Read 10 Done Sent Done false
10489 3986 Acked Read 8 Done Sent Done false
10490 3987 Acked Read 10 Done Sent Done false
10491 3988 Acked Write 9 Done Sent Done false
10492 3989 Acked Write 6 Done Sent Done false
10493 3990 Acked Write 5 Done Sent Done false
10494 3991 Acked Read 10 Done Sent Done false
10495 3992 Acked Read 7 Done Sent Done false
10496 3993 Acked Write 4 Done Sent Done false
10497 3994 Acked Write 9 Done Sent Done false
10498 3995 Acked Read 3 Done Sent Done false
10499 3996 Acked Read 10 Done Sent Done false
10500 3997 Acked Read 1 Done Sent Done false
10501 3998 Acked Flush 0 Done Sent Done false
10502 3999 Acked Read 10 Done Sent Done false
10503 4000 Acked Read 7 Done Sent Done false
10504 4001 Acked Read 6 Done Sent Done false
10505 4002 Acked Write 6 Done Sent Done false
10506 4003 Acked Write 7 Done Sent Done false
10507 4004 Acked Write 1 Done Sent Done false
10508 4005 Acked Read 8 Done Sent Done false
10509 4006 Acked Flush 0 Done Sent Done false
10510 4007 Acked Write 9 Done Sent Done false
10511 4008 Acked Write 8 Done Sent Done false
10512 4009 Acked Write 5 Done Sent Done false
10513 4010 Acked Read 1 Done Sent Done false
10514 4011 Acked Write 9 Done Sent Done false
10515 4012 Acked Write 10 Done Sent Done false
10516 4013 Acked Read 8 Done Sent Done false
10517 4014 Acked Read 9 Done Sent Done false
10518 4015 Acked Flush 0 Done Sent Done false
10519 4016 Acked Write 2 Done Sent Done false
10520 4017 Acked Read 2 Done Sent Done false
10521 4018 Acked Flush 0 Done Sent Done false
10522 STATES DS:0 DS:1 DS:2 TOTAL
10523 Sent 0 302 0 302
10524 Done 302 0 302 604
10525 Skipped 0 0 0 0
10526 Error 0 0 0 0
10527 Last Flush: 4018 3716 4018
10528 Downstairs last five completed: 3716 3715 3714 3713 3712
10529 Upstairs last five completed: 4018 4017 4016 4015 4014
10530 CLIENT: Up:0 ds:302 act:2
105312025-01-21T15:49:21.562ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105322025-01-21T15:49:21.562ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48111) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105332025-01-21T15:49:21.564ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105342025-01-21T15:49:21.564ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105352025-01-21T15:49:21.564ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105362025-01-21T15:49:21.564ZINFOcrucible: send last flush ID to this DS: 3716 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105372025-01-21T15:49:21.564ZINFOcrucible: Replied this last flush ID: 3716 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105382025-01-21T15:49:21.565ZINFOcrucible: [1] client re-new 302 jobs since flush 3716 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105392025-01-21T15:49:21.565ZINFOcrucible: [1] Replayed 302 jobs since flush: 3716 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
10540 ----------------------------------------------------------------
10541 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10542 Downstairs last five completed: 4018 4017 4016 4015 4014
10543 Upstairs last five completed: 4018 4017 4016 4015 4014
10544 CLIENT: Up:0 ds:0 act:3
10545 Replay: All jobs finished
10546 Main task has work to do, go find it
10547 got dsc Stop(1)
10548 stop 1
10549 Jan 21 15:49:23.799 INFO request completed, latency_us: 353, response_code: [8820] Got stop action so:false kr:false
10550 204, uri: /stop/cid/1, method: POST, req_id: 1dce96cc-3b4c-4529-9f9b-767c9f876568, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
10551 Jan 21 15:49:23.800 INFO request completed, latency_us: 144, response_code: 200, uri: /state/cid/1, method: GET, req_id: a17e1a5e-dee2-45f3-82e0-2f795a7acfc8, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
105522025-01-21T15:49:23.800ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105532025-01-21T15:49:23.800ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105542025-01-21T15:49:23.800ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
105552025-01-21T15:49:23.800ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
10556 [8820] Exited with: ExitStatus(unix_wait_status(9))
10557 [8820][1] reports Exit
10558 Jan 21 15:49:27.803 INFO request completed, latency_us: 207, response_code: 200, uri: /state/cid/1, method: GET, req_id: 312e67fb-c61b-4435-8b9d-295a2481bb19, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
10559 001/300 Write block 2620 len 36864 data: 3 4 5 5 5 4 4 4 5
10560 002/300 Write block 2133 len 28672 data: 4 4 4 3 3 3 5
10561 003/300 Read block 4819 len 36864
10562 004/300 Write block 648 len 40960 data: 5 5 5 5 4 4 3 3 2 2
10563 005/300 Read block 3750 len 8192
10564 006/300 Write block 3314 len 20480 data: 2 2 2 2 2
10565 007/300 Read block 3851 len 40960
10566 008/300 Read block 2789 len 28672
10567 009/300 Read block 1239 len 36864
10568 010/300 Write block 4516 len 32768 data: 4 4 4 5 4 4 4 4
10569 011/300 Flush
10570 012/300 Read block 1698 len 28672
10571 013/300 Write block 4094 len 24576 data: 3 3 3 3 3 4
10572 014/300 Write block 409 len 40960 data: 4 4 4 3 4 5 4 4 4 4
10573 015/300 Write block 1733 len 28672 data: 2 3 3 3 5 5 4
10574 016/300 Write block 1240 len 40960 data: 4 4 4 4 5 4 3 3 3 2
10575 017/300 Write block 1683 len 28672 data: 6 6 5 5 5 5 5
10576 018/300 Read block 1848 len 40960
10577 019/300 Write block 3240 len 40960 data: 4 3 2 3 3 4 4 4 5 3
10578 020/300 Read block 1039 len 16384
10579 021/300 Read block 537 len 40960
10580 022/300 Write block 2977 len 12288 data: 3 3 3
10581 023/300 Read block 1494 len 20480
10582 024/300 Read block 4359 len 16384
10583 025/300 Read block 3223 len 28672
10584 026/300 Write block 1191 len 4096 data: 6
10585 027/300 Write block 1712 len 8192 data: 4 3
10586 028/300 Read block 3871 len 12288
10587 029/300 Read block 795 len 4096
10588 030/300 Read block 640 len 36864
10589 031/300 Write block 2784 len 4096 data: 4
10590 032/300 Read block 2288 len 20480
10591 033/300 Read block 2932 len 16384
10592 034/300 Read block 761 len 12288
10593 035/300 Write block 1378 len 28672 data: 2 3 3 4 5 4 3
10594 036/300 Read block 1572 len 40960
10595 037/300 Read block 1318 len 4096
10596 038/300 Read block 2237 len 36864
10597 039/300 Read block 1915 len 20480
10598 040/300 Read block 1435 len 16384
10599 041/300 Write block 578 len 4096 data: 3
10600 042/300 Flush
10601 043/300 Read block 352 len 24576
10602 044/300 Read block 4124 len 8192
10603 045/300 Read block 3520 len 28672
10604 046/300 Read block 4792 len 16384
10605 047/300 Read block 3714 len 32768
10606 048/300 Write block 1504 len 36864 data: 2 4 3 3 3 3 4 5 4
10607 049/300 Flush
10608 050/300 Flush
10609 051/300 Write block 971 len 4096 data: 4
10610 052/300 Write block 1937 len 40960 data: 5 3 3 2 2 2 2 2 2 3
10611 053/300 Write block 163 len 16384 data: 3 3 3 2
10612 054/300 Write block 4079 len 32768 data: 2 2 2 2 3 2 2 3
10613 055/300 Read block 4423 len 8192
10614 056/300 Flush
10615 057/300 Write block 1132 len 4096 data: 2
10616 058/300 Flush
10617 059/300 Read block 431 len 8192
10618 060/300 Read block 947 len 16384
10619 061/300 Write block 1860 len 16384 data: 3 3 3 3
10620 062/300 Read block 3366 len 12288
10621 063/300 Flush
10622 064/300 Write block 4791 len 4096 data: 2
10623 065/300 Write block 224 len 32768 data: 4 4 4 4 4 4 3 2
10624 066/300 Read block 2378 len 36864
10625 067/300 Write block 3077 len 40960 data: 3 3 4 3 5 6 6 4 5 5
10626 068/300 Read block 534 len 12288
10627 069/300 Write block 854 len 8192 data: 2 4
10628 070/300 Read block 1641 len 36864
10629 071/300 Write block 3002 len 28672 data: 2 2 2 2 2 2 2
10630 072/300 Write block 4621 len 28672 data: 5 5 5 6 6 6 6
10631 073/300 Read block 2699 len 28672
10632 074/300 Read block 4700 len 16384
10633 075/300 Read block 2915 len 4096
10634 076/300 Flush
10635 077/300 Write block 1330 len 8192 data: 4 5
10636 078/300 Read block 1815 len 4096
10637 079/300 Write block 1185 len 32768 data: 3 4 4 5 5 5 7 6
10638 080/300 Flush
10639 081/300 Write block 2957 len 40960 data: 5 5 5 3 2 2 2 2 2 2
10640 082/300 Read block 1004 len 32768
10641 083/300 Flush
10642 084/300 Flush
10643 085/300 Flush
10644 086/300 Write block 3299 len 32768 data: 4 4 3 3 3 4 3 2
10645 087/300 Read block 2835 len 8192
10646 088/300 Flush
10647 089/300 Write block 3723 len 12288 data: 6 5 6
10648 090/300 Read block 906 len 20480
10649 091/300 Read block 2653 len 12288
10650 092/300 Write block 4695 len 40960 data: 2 2 2 2 2 2 2 2 2 2
10651 093/300 Write block 1112 len 8192 data: 4 2
10652 094/300 Read block 2541 len 4096
10653 095/300 Read block 3403 len 24576
10654 096/300 Read block 1538 len 40960
10655 097/300 Read block 162 len 16384
10656 098/300 Write block 1941 len 4096 data: 3
10657 099/300 Read block 191 len 16384
10658 100/300 Read block 3012 len 20480
10659 101/300 Read block 404 len 8192
10660 102/300 Flush
10661 103/300 Write block 1741 len 28672 data: 3 3 3 2 2 2 2
10662 104/300 Write block 1362 len 12288 data: 5 5 3
10663 105/300 Flush
10664 106/300 Flush
10665 107/300 Read block 706 len 12288
10666 108/300 Read block 3261 len 32768
10667 109/300 Write block 1025 len 12288 data: 3 3 3
10668 110/300 Write block 1915 len 16384 data: 3 3 3 4
10669 111/300 Read block 372 len 36864
10670 112/300 Read block 3324 len 36864
10671 113/300 Write block 2227 len 8192 data: 4 4
10672 114/300 Write block 2629 len 20480 data: 6 5 4 4 4
10673 115/300 Read block 1449 len 20480
10674 116/300 Write block 2657 len 20480 data: 3 2 2 2 3
10675 117/300 Write block 1247 len 36864 data: 4 4 3 2 2 2 2 2 2
10676 118/300 Write block 3160 len 24576 data: 3 3 4 4 3 3
10677 119/300 Read block 4882 len 4096
10678 120/300 Read block 1509 len 12288
10679 121/300 Read block 4354 len 32768
10680 122/300 Read block 2414 len 16384
10681 123/300 Read block 2639 len 8192
10682 124/300 Flush
10683 125/300 Write block 3948 len 4096 data: 4
10684 126/300 Read block 3996 len 28672
10685 127/300 Write block 459 len 28672 data: 3 2 4 4 4 4 4
10686 128/300 Write block 1367 len 20480 data: 3 3 2 2 3
10687 129/300 Read block 3657 len 12288
10688 130/300 Read block 4234 len 12288
10689 131/300 Read block 1225 len 12288
10690 132/300 Write block 1562 len 8192 data: 4 4
10691 133/300 Write block 1871 len 20480 data: 3 3 4 3 3
10692 134/300 Write block 3557 len 8192 data: 3 3
10693 135/300 Read block 462 len 12288
10694 136/300 Write block 84 len 4096 data: 3
10695 137/300 Read block 2902 len 40960
10696 138/300 Write block 1060 len 16384 data: 3 3 3 3
10697 139/300 Read block 2815 len 32768
10698 140/300 Read block 2618 len 24576
10699 141/300 Flush
10700 142/300 Read block 278 len 20480
10701 143/300 Write block 1078 len 24576 data: 3 2 2 2 2 2
10702 144/300 Read block 4585 len 32768
10703 145/300 Read block 1536 len 24576
10704 146/300 Write block 4745 len 16384 data: 3 3 3 3
10705 147/300 Write block 450 len 12288 data: 3 2 3
10706 148/300 Read block 4845 len 20480
10707 149/300 Read block 2008 len 24576
10708 150/300 Read block 845 len 40960
10709 151/300 Write block 2779 len 16384 data: 3 4 4 4
10710 152/300 Write block 4491 len 16384 data: 2 2 2 2
10711 153/300 Read block 4022 len 32768
10712 154/300 Read block 927 len 28672
10713 155/300 Write block 1621 len 8192 data: 6 5
10714 156/300 Read block 471 len 4096
10715 157/300 Read block 4307 len 16384
10716 158/300 Write block 4447 len 16384 data: 4 4 4 4
10717 159/300 Write block 4892 len 40960 data: 3 3 3 3 4 5 4 4 4 3
10718 160/300 Read block 2654 len 20480
10719 161/300 Read block 2636 len 12288
10720 162/300 Write block 793 len 24576 data: 4 3 2 2 3 4
10721 163/300 Write block 2144 len 4096 data: 3
10722 164/300 Flush
10723 165/300 Write block 212 len 32768 data: 4 5 4 4 4 4 4 4
10724 166/300 Read block 1238 len 36864
10725 167/300 Flush
10726 168/300 Write block 1882 len 32768 data: 3 4 3 3 3 3 3 3
10727 169/300 Read block 1313 len 16384
10728 170/300 Read block 581 len 20480
10729 171/300 Read block 3746 len 24576
10730 172/300 Write block 669 len 16384 data: 3 3 3 3
10731 173/300 Write block 2227 len 8192 data: 5 5
10732 174/300 Read block 2550 len 20480
10733 175/300 Write block 2870 len 12288 data: 2 2 3
10734 176/300 Read block 3830 len 12288
10735 177/300 Write block 2389 len 16384 data: 2 3 3 4
10736 178/300 Read block 4488 len 36864
10737 179/300 Write block 3377 len 32768 data: 2 3 3 3 3 4 4 3
10738 180/300 Read block 4194 len 12288
10739 181/300 Read block 659 len 40960
10740 182/300 Read block 1201 len 8192
10741 183/300 Write block 1363 len 32768 data: 6 4 3 3 4 4 3 3
10742 184/300 Read block 447 len 12288
10743 185/300 Read block 3207 len 28672
10744 186/300 Read block 4551 len 16384
10745 187/300 Read block 4632 len 40960
10746 188/300 Write block 2534 len 28672 data: 3 3 3 3 3 4 3
10747 189/300 Read block 584 len 4096
10748 190/300 Flush
10749 191/300 Write block 2644 len 36864 data: 3 3 3 3 3 3 3 2 2
10750 192/300 Write block 2796 len 12288 data: 4 4 4
10751 193/300 Write block 3379 len 36864 data: 4 4 4 5 5 4 3 4 3
10752 194/300 Flush
10753 195/300 Write block 4125 len 32768 data: 4 4 4 4 4 3 3 5
10754 196/300 Write block 611 len 12288 data: 3 3 3
10755 197/300 Read block 1714 len 24576
10756 198/300 Read block 3467 len 24576
10757 199/300 Write block 3784 len 24576 data: 4 4 4 4 3 3
10758 200/300 Read block 19 len 40960
10759 201/300 Read block 2630 len 32768
10760 202/300 Read block 3335 len 20480
10761 203/300 Read block 4830 len 20480
10762 204/300 Read block 2681 len 20480
10763 205/300 Read block 2509 len 4096
10764 206/300 Flush
10765 207/300 Read block 2487 len 4096
10766 208/300 Flush
10767 209/300 Read block 3308 len 28672
10768 210/300 Flush
10769 211/300 Read block 3472 len 40960
10770 212/300 Write block 599 len 8192 data: 3 3
10771 213/300 Read block 2666 len 20480
10772 214/300 Read block 1478 len 20480
10773 215/300 Write block 3735 len 24576 data: 3 3 2 2 2 2
10774 216/300 Write block 4006 len 16384 data: 3 3 3 3
10775 217/300 Write block 4800 len 40960 data: 5 4 4 3 3 3 3 2 2 3
10776 218/300 Write block 2848 len 40960 data: 2 3 2 2 2 2 2 2 2 2
10777 219/300 Write block 1380 len 16384 data: 4 5 6 5
10778 220/300 Flush
10779 221/300 Read block 3826 len 40960
10780 222/300 Write block 392 len 32768 data: 3 2 2 2 2 2 2 2
10781 223/300 Read block 444 len 36864
10782 224/300 Write block 3293 len 36864 data: 3 3 2 4 6 4 5 5 4
10783 225/300 Write block 166 len 32768 data: 3 3 3 3 3 3 4 4
10784 226/300 Write block 4541 len 12288 data: 4 3 3
10785 227/300 Read block 1693 len 8192
10786 228/300 Write block 3822 len 4096 data: 2
10787 229/300 Read block 3887 len 8192
10788 230/300 Read block 711 len 12288
10789 231/300 Write block 394 len 40960 data: 3 3 3 3 3 3 2 2 2 2
10790 232/300 Flush
10791 233/300 Write block 4143 len 40960 data: 3 2 3 3 3 3 3 3 4 5
10792 234/300 Read block 276 len 24576
10793 235/300 Read block 875 len 12288
10794 236/300 Read block 1497 len 24576
10795 237/300 Read block 3789 len 20480
10796 238/300 Write block 4205 len 28672 data: 3 3 2 2 2 2 2
10797 239/300 Read block 3423 len 16384
10798 240/300 Write block 4973 len 4096 data: 3
10799 241/300 Write block 2588 len 24576 data: 3 3 3 3 3 2
10800 242/300 Read block 3101 len 28672
10801 243/300 Read block 777 len 24576
10802 244/300 Read block 4450 len 40960
10803 245/300 Read block 2782 len 8192
10804 246/300 Write block 2396 len 8192 data: 3 2
10805 247/300 Write block 4485 len 28672 data: 4 4 3 3 2 2 3
10806 248/300 Flush
10807 249/300 Flush
10808 250/300 Write block 1446 len 24576 data: 4 4 3 3 3 3
10809 251/300 Read block 1028 len 40960
10810 252/300 Read block 3768 len 16384
10811 253/300 Read block 3719 len 8192
10812 254/300 Read block 4950 len 24576
10813 255/300 Read block 2630 len 4096
10814 256/300 Write block 3285 len 4096 data: 2
10815 257/300 Write block 3672 len 8192 data: 4 4
10816 258/300 Flush
10817 259/300 Read block 3707 len 8192
10818 260/300 Read block 1341 len 16384
10819 261/300 Read block 257 len 36864
10820 262/300 Read block 4549 len 20480
10821 263/300 Read block 1588 len 16384
10822 264/300 Read block 703 len 36864
10823 265/300 Write block 2354 len 12288 data: 6 6 5
10824 266/300 Write block 4476 len 12288 data: 3 5 5
10825 267/300 Write block 2794 len 36864 data: 6 5 5 5 5 4 2 2 2
10826 268/300 Flush
10827 269/300 Write block 2095 len 8192 data: 4 4
10828 270/300 Write block 3976 len 36864 data: 3 3 3 2 2 2 2 2 3
10829 271/300 Write block 2707 len 32768 data: 4 4 4 4 4 5 5 4
10830 272/300 Write block 197 len 12288 data: 3 3 3
10831 273/300 Read block 1993 len 16384
10832 274/300 Write block 3823 len 12288 data: 2 2 2
10833 275/300 Read block 2505 len 16384
10834 276/300 Flush
10835 277/300 Write block 3558 len 32768 data: 4 3 3 3 3 4 4 4
10836 278/300 Write block 587 len 12288 data: 2 3 3
10837 279/300 Read block 1533 len 16384
10838 280/300 Write block 1507 len 24576 data: 4 4 4 5 6 5
10839 281/300 Write block 4719 len 36864 data: 2 5 5 5 5 5 4 5 5
10840 282/300 Read block 443 len 28672
10841 283/300 Write block 675 len 4096 data: 3
10842 284/300 Write block 49 len 24576 data: 5 5 5 5 5 3
10843 285/300 Read block 3967 len 8192
10844 286/300 Write block 1126 len 28672 data: 2 2 2 2 2 2 3
10845 287/300 Flush
10846 288/300 Read block 2035 len 12288
10847 289/300 Read block 1254 len 4096
10848 290/300 Write block 3440 len 32768 data: 5 4 4 4 4 4 2 2
10849 291/300 Read block 2460 len 36864
10850 292/300 Write block 2164 len 40960 data: 3 2 2 3 3 5 5 6 8 6
10851 293/300 Read block 447 len 40960
10852 294/300 Read block 4694 len 32768
10853 295/300 Read block 4692 len 4096
10854 296/300 Write block 4795 len 4096 data: 3
10855 297/300 Flush
10856 298/300 Read block 2464 len 20480
10857 299/300 Read block 4695 len 8192
10858 300/300 Write block 233 len 36864 data: 3 3 3 3 3 3 3 3 3
10859 301/300 Write block 2791 len 16384 data: 6 6 5 7
10860 Main task has work to do, go find it
10861 got dsc Start(1)
10862 start 1
10863 Jan 21 15:49:28.041 INFO request completed, latency_us: 213, response_code: 204, uri: /start/cid/1, method: POST, req_id: 7c321f6b-e691-44b2-8d27-d56f291344bb, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
10864 Replay: started 1, returned:Ok(())
10865 ----------------------------------------------------------------
10866 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10867 States: Active Offline Active
10868 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10869 4019 Acked Write 9 Done Sent Done false
10870 4020 Acked Write 7 Done Sent Done false
10871 4021 Acked Read 9 Done Sent Done false
10872 4022 Acked Write 10 Done Sent Done false
10873 4023 Acked Read 2 Done Sent Done false
10874 4024 Acked Write 5 Done Sent Done false
10875 4025 Acked Read 10 Done Sent Done false
10876 4026 Acked Read 7 Done Sent Done false
10877 4027 Acked Read 9 Done Sent Done false
10878 4028 Acked Write 8 Done Sent Done false
10879 4029 Acked Flush 0 Done Sent Done false
10880 4030 Acked Read 7 Done Sent Done false
10881 4031 Acked Write 6 Done Sent Done false
10882 4032 Acked Write 10 Done Sent Done false
10883 4033 Acked Write 7 Done Sent Done false
10884 4034 Acked Write 10 Done Sent Done false
10885 4035 Acked Write 7 Done Sent Done false
10886 4036 Acked Read 10 Done Sent Done false
10887 4037 Acked Write 10 Done Sent Done false
10888 4038 Acked Read 4 Done Sent Done false
10889 4039 Acked Read 10 Done Sent Done false
10890 4040 Acked Write 3 Done Sent Done false
10891 4041 Acked Read 5 Done Sent Done false
10892 4042 Acked Read 4 Done Sent Done false
10893 4043 Acked Read 7 Done Sent Done false
10894 4044 Acked Write 1 Done Sent Done false
10895 4045 Acked Write 2 Done Sent Done false
10896 4046 Acked Read 3 Done Sent Done false
10897 4047 Acked Read 1 Done Sent Done false
10898 4048 Acked Read 9 Done Sent Done false
10899 4049 Acked Write 1 Done Sent Done false
10900 4050 Acked Read 5 Done Sent Done false
10901 4051 Acked Read 4 Done Sent Done false
10902 4052 Acked Read 3 Done Sent Done false
10903 4053 Acked Write 7 Done Sent Done false
10904 4054 Acked Read 10 Done Sent Done false
10905 4055 Acked Read 1 Done Sent Done false
10906 4056 Acked Read 9 Done Sent Done false
10907 4057 Acked Read 5 Done Sent Done false
10908 4058 Acked Read 4 Done Sent Done false
10909 4059 Acked Write 1 Done Sent Done false
10910 4060 Acked Flush 0 Done Sent Done false
10911 4061 Acked Read 6 Done Sent Done false
10912 4062 Acked Read 2 Done Sent Done false
10913 4063 Acked Read 7 Done Sent Done false
10914 4064 Acked Read 4 Done Sent Done false
10915 4065 Acked Read 8 Done Sent Done false
10916 4066 Acked Write 9 Done Sent Done false
10917 4067 Acked Flush 0 Done Sent Done false
10918 4068 Acked Flush 0 Done Sent Done false
10919 4069 Acked Write 1 Done Sent Done false
10920 4070 Acked Write 10 Done Sent Done false
10921 4071 Acked Write 4 Done Sent Done false
10922 4072 Acked Write 8 Done Sent Done false
10923 4073 Acked Read 2 Done Sent Done false
10924 4074 Acked Flush 0 Done Sent Done false
10925 4075 Acked Write 1 Done Sent Done false
10926 4076 Acked Flush 0 Done Sent Done false
10927 4077 Acked Read 2 Done Sent Done false
10928 4078 Acked Read 4 Done Sent Done false
10929 4079 Acked Write 4 Done Sent Done false
10930 4080 Acked Read 3 Done Sent Done false
10931 4081 Acked Flush 0 Done Sent Done false
10932 4082 Acked Write 1 Done Sent Done false
10933 4083 Acked Write 8 Done Sent Done false
10934 4084 Acked Read 9 Done Sent Done false
10935 4085 Acked Write 10 Done Sent Done false
10936 4086 Acked Read 3 Done Sent Done false
10937 4087 Acked Write 2 Done Sent Done false
10938 4088 Acked Read 9 Done Sent Done false
10939 4089 Acked Write 7 Done Sent Done false
10940 4090 Acked Write 7 Done Sent Done false
10941 4091 Acked Read 7 Done Sent Done false
10942 4092 Acked Read 4 Done Sent Done false
10943 4093 Acked Read 1 Done Sent Done false
10944 4094 Acked Flush 0 Done Sent Done false
10945 4095 Acked Write 2 Done Sent Done false
10946 4096 Acked Read 1 Done Sent Done false
10947 4097 Acked Write 8 Done Sent Done false
10948 4098 Acked Flush 0 Done Sent Done false
10949 4099 Acked Write 10 Done Sent Done false
10950 4100 Acked Read 8 Done Sent Done false
10951 4101 Acked Flush 0 Done Sent Done false
10952 4102 Acked Flush 0 Done Sent Done false
10953 4103 Acked Flush 0 Done Sent Done false
10954 4104 Acked Write 8 Done Sent Done false
10955 4105 Acked Read 2 Done Sent Done false
10956 4106 Acked Flush 0 Done Sent Done false
10957 4107 Acked Write 3 Done Sent Done false
10958 4108 Acked Read 5 Done Sent Done false
10959 4109 Acked Read 3 Done Sent Done false
10960 4110 Acked Write 10 Done Sent Done false
10961 4111 Acked Write 2 Done Sent Done false
10962 4112 Acked Read 1 Done Sent Done false
10963 4113 Acked Read 6 Done Sent Done false
10964 4114 Acked Read 10 Done Sent Done false
10965 4115 Acked Read 4 Done Sent Done false
10966 4116 Acked Write 1 Done Sent Done false
10967 4117 Acked Read 4 Done Sent Done false
10968 4118 Acked Read 5 Done Sent Done false
10969 4119 Acked Read 2 Done Sent Done false
10970 4120 Acked Flush 0 Done Sent Done false
10971 4121 Acked Write 7 Done Sent Done false
10972 4122 Acked Write 3 Done Sent Done false
10973 4123 Acked Flush 0 Done Sent Done false
10974 4124 Acked Flush 0 Done Sent Done false
10975 4125 Acked Read 3 Done Sent Done false
10976 4126 Acked Read 8 Done Sent Done false
10977 4127 Acked Write 3 Done Sent Done false
10978 4128 Acked Write 4 Done Sent Done false
10979 4129 Acked Read 9 Done Sent Done false
10980 4130 Acked Read 9 Done Sent Done false
10981 4131 Acked Write 2 Done Sent Done false
10982 4132 Acked Write 5 Done Sent Done false
10983 4133 Acked Read 5 Done Sent Done false
10984 4134 Acked Write 5 Done Sent Done false
10985 4135 Acked Write 9 Done Sent Done false
10986 4136 Acked Write 6 Done Sent Done false
10987 4137 Acked Read 1 Done Sent Done false
10988 4138 Acked Read 3 Done Sent Done false
10989 4139 Acked Read 8 Done Sent Done false
10990 4140 Acked Read 4 Done Sent Done false
10991 4141 Acked Read 2 Done Sent Done false
10992 4142 Acked Flush 0 Done Sent Done false
10993 4143 Acked Write 1 Done Sent Done false
10994 4144 Acked Read 7 Done Sent Done false
10995 4145 Acked Write 7 Done Sent Done false
10996 4146 Acked Write 5 Done Sent Done false
10997 4147 Acked Read 3 Done Sent Done false
10998 4148 Acked Read 3 Done Sent Done false
10999 4149 Acked Read 3 Done Sent Done false
11000 4150 Acked Write 2 Done Sent Done false
11001 4151 Acked Write 5 Done Sent Done false
11002 4152 Acked Write 2 Done Sent Done false
11003 4153 Acked Read 3 Done Sent Done false
11004 4154 Acked Write 1 Done Sent Done false
11005 4155 Acked Read 10 Done Sent Done false
11006 4156 Acked Write 4 Done Sent Done false
11007 4157 Acked Read 8 Done Sent Done false
11008 4158 Acked Read 6 Done Sent Done false
11009 4159 Acked Flush 0 Done Sent Done false
11010 4160 Acked Read 5 Done Sent Done false
11011 4161 Acked Write 6 Done Sent Done false
11012 4162 Acked Read 8 Done Sent Done false
11013 4163 Acked Read 6 Done Sent Done false
11014 4164 Acked Write 4 Done Sent Done false
11015 4165 Acked Write 3 Done Sent Done false
11016 4166 Acked Read 5 Done Sent Done false
11017 4167 Acked Read 6 Done Sent Done false
11018 4168 Acked Read 10 Done Sent Done false
11019 4169 Acked Write 4 Done Sent Done false
11020 4170 Acked Write 4 Done Sent Done false
11021 4171 Acked Read 8 Done Sent Done false
11022 4172 Acked Read 7 Done Sent Done false
11023 4173 Acked Write 2 Done Sent Done false
11024 4174 Acked Read 1 Done Sent Done false
11025 4175 Acked Read 4 Done Sent Done false
11026 4176 Acked Write 4 Done Sent Done false
11027 4177 Acked Write 10 Done Sent Done false
11028 4178 Acked Read 5 Done Sent Done false
11029 4179 Acked Read 3 Done Sent Done false
11030 4180 Acked Write 6 Done Sent Done false
11031 4181 Acked Write 1 Done Sent Done false
11032 4182 Acked Flush 0 Done Sent Done false
11033 4183 Acked Write 8 Done Sent Done false
11034 4184 Acked Read 9 Done Sent Done false
11035 4185 Acked Flush 0 Done Sent Done false
11036 4186 Acked Write 8 Done Sent Done false
11037 4187 Acked Read 4 Done Sent Done false
11038 4188 Acked Read 5 Done Sent Done false
11039 4189 Acked Read 6 Done Sent Done false
11040 4190 Acked Write 4 Done Sent Done false
11041 4191 Acked Write 2 Done Sent Done false
11042 4192 Acked Read 5 Done Sent Done false
11043 4193 Acked Write 3 Done Sent Done false
11044 4194 Acked Read 3 Done Sent Done false
11045 4195 Acked Write 4 Done Sent Done false
11046 4196 Acked Read 9 Done Sent Done false
11047 4197 Acked Write 8 Done Sent Done false
11048 4198 Acked Read 3 Done Sent Done false
11049 4199 Acked Read 10 Done Sent Done false
11050 4200 Acked Read 2 Done Sent Done false
11051 4201 Acked Write 8 Done Sent Done false
11052 4202 Acked Read 3 Done Sent Done false
11053 4203 Acked Read 7 Done Sent Done false
11054 4204 Acked Read 4 Done Sent Done false
11055 4205 Acked Read 10 Done Sent Done false
11056 4206 Acked Write 7 Done Sent Done false
11057 4207 Acked Read 1 Done Sent Done false
11058 4208 Acked Flush 0 Done Sent Done false
11059 4209 Acked Write 9 Done Sent Done false
11060 4210 Acked Write 3 Done Sent Done false
11061 4211 Acked Write 9 Done Sent Done false
11062 4212 Acked Flush 0 Done Sent Done false
11063 4213 Acked Write 8 Done Sent Done false
11064 4214 Acked Write 3 Done Sent Done false
11065 4215 Acked Read 6 Done Sent Done false
11066 4216 Acked Read 6 Done Sent Done false
11067 4217 Acked Write 6 Done Sent Done false
11068 4218 Acked Read 10 Done Sent Done false
11069 4219 Acked Read 8 Done Sent Done false
11070 4220 Acked Read 5 Done Sent Done false
11071 4221 Acked Read 5 Done Sent Done false
11072 4222 Acked Read 5 Done Sent Done false
11073 4223 Acked Read 1 Done Sent Done false
11074 4224 Acked Flush 0 Done Sent Done false
11075 4225 Acked Read 1 Done Sent Done false
11076 4226 Acked Flush 0 Done Sent Done false
11077 4227 Acked Read 7 Done Sent Done false
11078 4228 Acked Flush 0 Done Sent Done false
11079 4229 Acked Read 10 Done Sent Done false
11080 4230 Acked Write 2 Done Sent Done false
11081 4231 Acked Read 5 Done Sent Done false
11082 4232 Acked Read 5 Done Sent Done false
11083 4233 Acked Write 6 Done Sent Done false
11084 4234 Acked Write 4 Done Sent Done false
11085 4235 Acked Write 10 Done Sent Done false
11086 4236 Acked Write 10 Done Sent Done false
11087 4237 Acked Write 4 Done Sent Done false
11088 4238 Acked Flush 0 Done Sent Done false
11089 4239 Acked Read 10 Done Sent Done false
11090 4240 Acked Write 8 Done Sent Done false
11091 4241 Acked Read 9 Done Sent Done false
11092 4242 Acked Write 9 Done Sent Done false
11093 4243 Acked Write 8 Done Sent Done false
11094 4244 Acked Write 3 Done Sent Done false
11095 4245 Acked Read 2 Done Sent Done false
11096 4246 Acked Write 1 Done Sent Done false
11097 4247 Acked Read 2 Done Sent Done false
11098 4248 Acked Read 3 Done Sent Done false
11099 4249 Acked Write 10 Done Sent Done false
11100 4250 Acked Flush 0 Done Sent Done false
11101 4251 Acked Write 10 Done Sent Done false
11102 4252 Acked Read 6 Done Sent Done false
11103 4253 Acked Read 3 Done Sent Done false
11104 4254 Acked Read 6 Done Sent Done false
11105 4255 Acked Read 5 Done Sent Done false
11106 4256 Acked Write 7 Done Sent Done false
11107 4257 Acked Read 4 Done Sent Done false
11108 4258 Acked Write 1 Done Sent Done false
11109 4259 Acked Write 6 Done Sent Done false
11110 4260 Acked Read 7 Done Sent Done false
11111 4261 Acked Read 6 Done Sent Done false
11112 4262 Acked Read 10 Done Sent Done false
11113 4263 Acked Read 2 Done Sent Done false
11114 4264 Acked Write 2 Done Sent Done false
11115 4265 Acked Write 7 Done Sent Done false
11116 4266 Acked Flush 0 Done Sent Done false
11117 4267 Acked Flush 0 Done Sent Done false
11118 4268 Acked Write 6 Done Sent Done false
11119 4269 Acked Read 10 Done Sent Done false
11120 4270 Acked Read 4 Done Sent Done false
11121 4271 Acked Read 2 Done Sent Done false
11122 4272 Acked Read 6 Done Sent Done false
11123 4273 Acked Read 1 Done Sent Done false
11124 4274 Acked Write 1 Done Sent Done false
11125 4275 Acked Write 2 Done Sent Done false
11126 4276 Acked Flush 0 Done Sent Done false
11127 4277 Acked Read 2 Done Sent Done false
11128 4278 Acked Read 4 Done Sent Done false
11129 4279 Acked Read 9 Done Sent Done false
11130 4280 Acked Read 5 Done Sent Done false
11131 4281 Acked Read 4 Done Sent Done false
11132 4282 Acked Read 9 Done Sent Done false
11133 4283 Acked Write 3 Done Sent Done false
11134 4284 Acked Write 3 Done Sent Done false
11135 4285 Acked Write 9 Done Sent Done false
11136 4286 Acked Flush 0 Done Sent Done false
11137 4287 Acked Write 2 Done Sent Done false
11138 4288 Acked Write 9 Done Sent Done false
11139 4289 Acked Write 8 Done Sent Done false
11140 4290 Acked Write 3 Done Sent Done false
11141 4291 Acked Read 4 Done Sent Done false
11142 4292 Acked Write 3 Done Sent Done false
11143 4293 Acked Read 4 Done Sent Done false
11144 4294 Acked Flush 0 Done Sent Done false
11145 4295 Acked Write 8 Done Sent Done false
11146 4296 Acked Write 3 Done Sent Done false
11147 4297 Acked Read 4 Done Sent Done false
11148 4298 Acked Write 6 Done Sent Done false
11149 4299 Acked Write 9 Done Sent Done false
11150 4300 Acked Read 7 Done Sent Done false
11151 4301 Acked Write 1 Done Sent Done false
11152 4302 Acked Write 6 Done Sent Done false
11153 4303 Acked Read 2 Done Sent Done false
11154 4304 Acked Write 7 Done Sent Done false
11155 4305 Acked Flush 0 Done Sent Done false
11156 4306 Acked Read 3 Done Sent Done false
11157 4307 Acked Read 1 Done Sent Done false
11158 4308 Acked Write 8 Done Sent Done false
11159 4309 Acked Read 9 Done Sent Done false
11160 4310 Acked Write 10 Done Sent Done false
11161 4311 Acked Read 10 Done Sent Done false
11162 4312 Acked Read 8 Done Sent Done false
11163 4313 Acked Read 1 Done Sent Done false
11164 4314 Acked Write 1 Done Sent Done false
11165 4315 Acked Flush 0 Done Sent Done false
11166 4316 Acked Read 5 Done Sent Done false
11167 4317 Acked Read 2 Done Sent Done false
11168 4318 Acked Write 9 Done Sent Done false
11169 4319 Acked Write 4 Done Sent Done false
11170 STATES DS:0 DS:1 DS:2 TOTAL
11171 Sent 0 301 0 301
11172 Done 301 0 301 602
11173 Skipped 0 0 0 0
11174 Error 0 0 0 0
11175 Last Flush: 4315 4018 4315
11176 Downstairs last five completed: 4018 4017 4016 4015 4014
11177 Upstairs last five completed: 4319 4318 4317 4316 4315
11178 CLIENT: Up:0 ds:301 act:2
11179 ----------------------------------------------------------------
11180 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11181 States: Active Offline Active
11182 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11183 4019 Acked Write 9 Done Sent Done false
11184 4020 Acked Write 7 Done Sent Done false
11185 4021 Acked Read 9 Done Sent Done false
11186 4022 Acked Write 10 Done Sent Done false
11187 4023 Acked Read 2 Done Sent Done false
11188 4024 Acked Write 5 Done Sent Done false
11189 4025 Acked Read 10 Done Sent Done false
11190 4026 Acked Read 7 Done Sent Done false
11191 4027 Acked Read 9 Done Sent Done false
11192 4028 Acked Write 8 Done Sent Done false
11193 4029 Acked Flush 0 Done Sent Done false
11194 4030 Acked Read 7 Done Sent Done false
11195 4031 Acked Write 6 Done Sent Done false
11196 4032 Acked Write 10 Done Sent Done false
11197 4033 Acked Write 7 Done Sent Done false
11198 4034 Acked Write 10 Done Sent Done false
11199 4035 Acked Write 7 Done Sent Done false
11200 4036 Acked Read 10 Done Sent Done false
11201 4037 Acked Write 10 Done Sent Done false
11202 4038 Acked Read 4 Done Sent Done false
11203 4039 Acked Read 10 Done Sent Done false
11204 4040 Acked Write 3 Done Sent Done false
11205 4041 Acked Read 5 Done Sent Done false
11206 4042 Acked Read 4 Done Sent Done false
11207 4043 Acked Read 7 Done Sent Done false
11208 4044 Acked Write 1 Done Sent Done false
11209 4045 Acked Write 2 Done Sent Done false
11210 4046 Acked Read 3 Done Sent Done false
11211 4047 Acked Read 1 Done Sent Done false
11212 4048 Acked Read 9 Done Sent Done false
11213 4049 Acked Write 1 Done Sent Done false
11214 4050 Acked Read 5 Done Sent Done false
11215 4051 Acked Read 4 Done Sent Done false
11216 4052 Acked Read 3 Done Sent Done false
11217 4053 Acked Write 7 Done Sent Done false
11218 4054 Acked Read 10 Done Sent Done false
11219 4055 Acked Read 1 Done Sent Done false
11220 4056 Acked Read 9 Done Sent Done false
11221 4057 Acked Read 5 Done Sent Done false
11222 4058 Acked Read 4 Done Sent Done false
11223 4059 Acked Write 1 Done Sent Done false
11224 4060 Acked Flush 0 Done Sent Done false
11225 4061 Acked Read 6 Done Sent Done false
11226 4062 Acked Read 2 Done Sent Done false
11227 4063 Acked Read 7 Done Sent Done false
11228 4064 Acked Read 4 Done Sent Done false
11229 4065 Acked Read 8 Done Sent Done false
11230 4066 Acked Write 9 Done Sent Done false
11231 4067 Acked Flush 0 Done Sent Done false
11232 4068 Acked Flush 0 Done Sent Done false
11233 4069 Acked Write 1 Done Sent Done false
11234 4070 Acked Write 10 Done Sent Done false
11235 4071 Acked Write 4 Done Sent Done false
11236 4072 Acked Write 8 Done Sent Done false
11237 4073 Acked Read 2 Done Sent Done false
11238 4074 Acked Flush 0 Done Sent Done false
11239 4075 Acked Write 1 Done Sent Done false
11240 4076 Acked Flush 0 Done Sent Done false
11241 4077 Acked Read 2 Done Sent Done false
11242 4078 Acked Read 4 Done Sent Done false
11243 4079 Acked Write 4 Done Sent Done false
11244 4080 Acked Read 3 Done Sent Done false
11245 4081 Acked Flush 0 Done Sent Done false
11246 4082 Acked Write 1 Done Sent Done false
11247 4083 Acked Write 8 Done Sent Done false
11248 4084 Acked Read 9 Done Sent Done false
11249 4085 Acked Write 10 Done Sent Done false
11250 4086 Acked Read 3 Done Sent Done false
11251 4087 Acked Write 2 Done Sent Done false
11252 4088 Acked Read 9 Done Sent Done false
11253 4089 Acked Write 7 Done Sent Done false
11254 4090 Acked Write 7 Done Sent Done false
11255 4091 Acked Read 7 Done Sent Done false
11256 4092 Acked Read 4 Done Sent Done false
11257 4093 Acked Read 1 Done Sent Done false
11258 4094 Acked Flush 0 Done Sent Done false
11259 4095 Acked Write 2 Done Sent Done false
11260 4096 Acked Read 1 Done Sent Done false
11261 4097 Acked Write 8 Done Sent Done false
11262 4098 Acked Flush 0 Done Sent Done false
11263 4099 Acked Write 10 Done Sent Done false
11264 4100 Acked Read 8 Done Sent Done false
11265 4101 Acked Flush 0 Done Sent Done false
11266 4102 Acked Flush 0 Done Sent Done false
11267 4103 Acked Flush 0 Done Sent Done false
11268 4104 Acked Write 8 Done Sent Done false
11269 4105 Acked Read 2 Done Sent Done false
11270 4106 Acked Flush 0 Done Sent Done false
11271 4107 Acked Write 3 Done Sent Done false
11272 4108 Acked Read 5 Done Sent Done false
11273 4109 Acked Read 3 Done Sent Done false
11274 4110 Acked Write 10 Done Sent Done false
11275 4111 Acked Write 2 Done Sent Done false
11276 4112 Acked Read 1 Done Sent Done false
11277 4113 Acked Read 6 Done Sent Done false
11278 4114 Acked Read 10 Done Sent Done false
11279 4115 Acked Read 4 Done Sent Done false
11280 4116 Acked Write 1 Done Sent Done false
11281 4117 Acked Read 4 Done Sent Done false
11282 4118 Acked Read 5 Done Sent Done false
11283 4119 Acked Read 2 Done Sent Done false
11284 4120 Acked Flush 0 Done Sent Done false
11285 4121 Acked Write 7 Done Sent Done false
11286 4122 Acked Write 3 Done Sent Done false
11287 4123 Acked Flush 0 Done Sent Done false
11288 4124 Acked Flush 0 Done Sent Done false
11289 4125 Acked Read 3 Done Sent Done false
11290 4126 Acked Read 8 Done Sent Done false
11291 4127 Acked Write 3 Done Sent Done false
11292 4128 Acked Write 4 Done Sent Done false
11293 4129 Acked Read 9 Done Sent Done false
11294 4130 Acked Read 9 Done Sent Done false
11295 4131 Acked Write 2 Done Sent Done false
11296 4132 Acked Write 5 Done Sent Done false
11297 4133 Acked Read 5 Done Sent Done false
11298 4134 Acked Write 5 Done Sent Done false
11299 4135 Acked Write 9 Done Sent Done false
11300 4136 Acked Write 6 Done Sent Done false
11301 4137 Acked Read 1 Done Sent Done false
11302 4138 Acked Read 3 Done Sent Done false
11303 4139 Acked Read 8 Done Sent Done false
11304 4140 Acked Read 4 Done Sent Done false
11305 4141 Acked Read 2 Done Sent Done false
11306 4142 Acked Flush 0 Done Sent Done false
11307 4143 Acked Write 1 Done Sent Done false
11308 4144 Acked Read 7 Done Sent Done false
11309 4145 Acked Write 7 Done Sent Done false
11310 4146 Acked Write 5 Done Sent Done false
11311 4147 Acked Read 3 Done Sent Done false
11312 4148 Acked Read 3 Done Sent Done false
11313 4149 Acked Read 3 Done Sent Done false
11314 4150 Acked Write 2 Done Sent Done false
11315 4151 Acked Write 5 Done Sent Done false
11316 4152 Acked Write 2 Done Sent Done false
11317 4153 Acked Read 3 Done Sent Done false
11318 4154 Acked Write 1 Done Sent Done false
11319 4155 Acked Read 10 Done Sent Done false
11320 4156 Acked Write 4 Done Sent Done false
11321 4157 Acked Read 8 Done Sent Done false
11322 4158 Acked Read 6 Done Sent Done false
11323 4159 Acked Flush 0 Done Sent Done false
11324 4160 Acked Read 5 Done Sent Done false
11325 4161 Acked Write 6 Done Sent Done false
11326 4162 Acked Read 8 Done Sent Done false
11327 4163 Acked Read 6 Done Sent Done false
11328 4164 Acked Write 4 Done Sent Done false
11329 4165 Acked Write 3 Done Sent Done false
11330 4166 Acked Read 5 Done Sent Done false
11331 4167 Acked Read 6 Done Sent Done false
11332 4168 Acked Read 10 Done Sent Done false
11333 4169 Acked Write 4 Done Sent Done false
11334 4170 Acked Write 4 Done Sent Done false
11335 4171 Acked Read 8 Done Sent Done false
11336 4172 Acked Read 7 Done Sent Done false
11337 4173 Acked Write 2 Done Sent Done false
11338 4174 Acked Read 1 Done Sent Done false
11339 4175 Acked Read 4 Done Sent Done false
11340 4176 Acked Write 4 Done Sent Done false
11341 4177 Acked Write 10 Done Sent Done false
11342 4178 Acked Read 5 Done Sent Done false
11343 4179 Acked Read 3 Done Sent Done false
11344 4180 Acked Write 6 Done Sent Done false
11345 4181 Acked Write 1 Done Sent Done false
11346 4182 Acked Flush 0 Done Sent Done false
11347 4183 Acked Write 8 Done Sent Done false
11348 4184 Acked Read 9 Done Sent Done false
11349 4185 Acked Flush 0 Done Sent Done false
11350 4186 Acked Write 8 Done Sent Done false
11351 4187 Acked Read 4 Done Sent Done false
11352 4188 Acked Read 5 Done Sent Done false
11353 4189 Acked Read 6 Done Sent Done false
11354 4190 Acked Write 4 Done Sent Done false
11355 4191 Acked Write 2 Done Sent Done false
11356 4192 Acked Read 5 Done Sent Done false
11357 4193 Acked Write 3 Done Sent Done false
11358 4194 Acked Read 3 Done Sent Done false
11359 4195 Acked Write 4 Done Sent Done false
11360 4196 Acked Read 9 Done Sent Done false
11361 4197 Acked Write 8 Done Sent Done false
11362 4198 Acked Read 3 Done Sent Done false
11363 4199 Acked Read 10 Done Sent Done false
11364 4200 Acked Read 2 Done Sent Done false
11365 4201 Acked Write 8 Done Sent Done false
11366 4202 Acked Read 3 Done Sent Done false
11367 4203 Acked Read 7 Done Sent Done false
11368 4204 Acked Read 4 Done Sent Done false
11369 4205 Acked Read 10 Done Sent Done false
11370 4206 Acked Write 7 Done Sent Done false
11371 4207 Acked Read 1 Done Sent Done false
11372 4208 Acked Flush 0 Done Sent Done false
11373 4209 Acked Write 9 Done Sent Done false
11374 4210 Acked Write 3 Done Sent Done false
11375 4211 Acked Write 9 Done Sent Done false
11376 4212 Acked Flush 0 Done Sent Done false
11377 4213 Acked Write 8 Done Sent Done false
11378 4214 Acked Write 3 Done Sent Done false
11379 4215 Acked Read 6 Done Sent Done false
11380 4216 Acked Read 6 Done Sent Done false
11381 4217 Acked Write 6 Done Sent Done false
11382 4218 Acked Read 10 Done Sent Done false
11383 4219 Acked Read 8 Done Sent Done false
11384 4220 Acked Read 5 Done Sent Done false
11385 4221 Acked Read 5 Done Sent Done false
11386 4222 Acked Read 5 Done Sent Done false
11387 4223 Acked Read 1 Done Sent Done false
11388 4224 Acked Flush 0 Done Sent Done false
11389 4225 Acked Read 1 Done Sent Done false
11390 4226 Acked Flush 0 Done Sent Done false
11391 4227 Acked Read 7 Done Sent Done false
11392 4228 Acked Flush 0 Done Sent Done false
11393 4229 Acked Read 10 Done Sent Done false
11394 4230 Acked Write 2 Done Sent Done false
11395 4231 Acked Read 5 Done Sent Done false
11396 4232 Acked Read 5 Done Sent Done false
11397 4233 Acked Write 6 Done Sent Done false
11398 4234 Acked Write 4 Done Sent Done false
11399 4235 Acked Write 10 Done Sent Done false
11400 4236 Acked Write 10 Done Sent Done false
11401 4237 Acked Write 4 Done Sent Done false
11402 4238 Acked Flush 0 Done Sent Done false
11403 4239 Acked Read 10 Done Sent Done false
11404 4240 Acked Write 8 Done Sent Done false
11405 4241 Acked Read 9 Done Sent Done false
11406 4242 Acked Write 9 Done Sent Done false
11407 4243 Acked Write 8 Done Sent Done false
11408 4244 Acked Write 3 Done Sent Done false
11409 4245 Acked Read 2 Done Sent Done false
11410 4246 Acked Write 1 Done Sent Done false
11411 4247 Acked Read 2 Done Sent Done false
11412 4248 Acked Read 3 Done Sent Done false
11413 4249 Acked Write 10 Done Sent Done false
11414 4250 Acked Flush 0 Done Sent Done false
11415 4251 Acked Write 10 Done Sent Done false
11416 4252 Acked Read 6 Done Sent Done false
11417 4253 Acked Read 3 Done Sent Done false
11418 4254 Acked Read 6 Done Sent Done false
11419 4255 Acked Read 5 Done Sent Done false
11420 4256 Acked Write 7 Done Sent Done false
11421 4257 Acked Read 4 Done Sent Done false
11422 4258 Acked Write 1 Done Sent Done false
11423 4259 Acked Write 6 Done Sent Done false
11424 4260 Acked Read 7 Done Sent Done false
11425 4261 Acked Read 6 Done Sent Done false
11426 4262 Acked Read 10 Done Sent Done false
11427 4263 Acked Read 2 Done Sent Done false
11428 4264 Acked Write 2 Done Sent Done false
11429 4265 Acked Write 7 Done Sent Done false
11430 4266 Acked Flush 0 Done Sent Done false
11431 4267 Acked Flush 0 Done Sent Done false
11432 4268 Acked Write 6 Done Sent Done false
11433 4269 Acked Read 10 Done Sent Done false
11434 4270 Acked Read 4 Done Sent Done false
11435 4271 Acked Read 2 Done Sent Done false
11436 4272 Acked Read 6 Done Sent Done false
11437 4273 Acked Read 1 Done Sent Done false
11438 4274 Acked Write 1 Done Sent Done false
11439 4275 Acked Write 2 Done Sent Done false
11440 4276 Acked Flush 0 Done Sent Done false
11441 4277 Acked Read 2 Done Sent Done false
11442 4278 Acked Read 4 Done Sent Done false
11443 4279 Acked Read 9 Done Sent Done false
11444 4280 Acked Read 5 Done Sent Done false
11445 4281 Acked Read 4 Done Sent Done false
11446 4282 Acked Read 9 Done Sent Done false
11447 4283 Acked Write 3 Done Sent Done false
11448 4284 Acked Write 3 Done Sent Done false
11449 4285 Acked Write 9 Done Sent Done false
11450 4286 Acked Flush 0 Done Sent Done false
11451 4287 Acked Write 2 Done Sent Done false
11452 4288 Acked Write 9 Done Sent Done false
11453 4289 Acked Write 8 Done Sent Done false
11454 4290 Acked Write 3 Done Sent Done false
11455 4291 Acked Read 4 Done Sent Done false
11456 4292 Acked Write 3 Done Sent Done false
11457 4293 Acked Read 4 Done Sent Done false
11458 4294 Acked Flush 0 Done Sent Done false
11459 4295 Acked Write 8 Done Sent Done false
11460 4296 Acked Write 3 Done Sent Done false
11461 4297 Acked Read 4 Done Sent Done false
11462 4298 Acked Write 6 Done Sent Done false
11463 4299 Acked Write 9 Done Sent Done false
11464 4300 Acked Read 7 Done Sent Done false
11465 4301 Acked Write 1 Done Sent Done false
11466 4302 Acked Write 6 Done Sent Done false
11467 4303 Acked Read 2 Done Sent Done false
11468 4304 Acked Write 7 Done Sent Done false
11469 4305 Acked Flush 0 Done Sent Done false
11470 4306 Acked Read 3 Done Sent Done false
11471 4307 Acked Read 1 Done Sent Done false
11472 4308 Acked Write 8 Done Sent Done false
11473 4309 Acked Read 9 Done Sent Done false
11474 4310 Acked Write 10 Done Sent Done false
11475 4311 Acked Read 10 Done Sent Done false
11476 4312 Acked Read 8 Done Sent Done false
11477 4313 Acked Read 1 Done Sent Done false
11478 4314 Acked Write 1 Done Sent Done false
11479 4315 Acked Flush 0 Done Sent Done false
11480 4316 Acked Read 5 Done Sent Done false
11481 4317 Acked Read 2 Done Sent Done false
11482 4318 Acked Write 9 Done Sent Done false
11483 4319 Acked Write 4 Done Sent Done false
11484 4320 Acked Flush 0 Done Sent Done false
11485 STATES DS:0 DS:1 DS:2 TOTAL
11486 Sent 0 302 0 302
11487 Done 302 0 302 604
11488 Skipped 0 0 0 0
11489 Error 0 0 0 0
11490 Last Flush: 4320 4018 4320
11491 Downstairs last five completed: 4018 4017 4016 4015 4014
11492 Upstairs last five completed: 4320 4319 4318 4317 4316
11493 CLIENT: Up:0 ds:302 act:2
114942025-01-21T15:49:33.802ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
114952025-01-21T15:49:33.802ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
114962025-01-21T15:49:33.802ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
114972025-01-21T15:49:33.802ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
114982025-01-21T15:49:33.802ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
11499 [8820] got start action
11500 [8820] I am going to restart
11501 Starting downstairs at port 8820
11502 Make output file at "/tmp/dsc/downstairs-8820.txt"
11503 [8820][1] reports Starting
11504 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1211)
11505 [8820][1] reports Running
11506 ----------------------------------------------------------------
11507 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11508 States: Active Offline Active
11509 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11510 4019 Acked Write 9 Done Sent Done false
11511 4020 Acked Write 7 Done Sent Done false
11512 4021 Acked Read 9 Done Sent Done false
11513 4022 Acked Write 10 Done Sent Done false
11514 4023 Acked Read 2 Done Sent Done false
11515 4024 Acked Write 5 Done Sent Done false
11516 4025 Acked Read 10 Done Sent Done false
11517 4026 Acked Read 7 Done Sent Done false
11518 4027 Acked Read 9 Done Sent Done false
11519 4028 Acked Write 8 Done Sent Done false
11520 4029 Acked Flush 0 Done Sent Done false
11521 4030 Acked Read 7 Done Sent Done false
11522 4031 Acked Write 6 Done Sent Done false
11523 4032 Acked Write 10 Done Sent Done false
11524 4033 Acked Write 7 Done Sent Done false
11525 4034 Acked Write 10 Done Sent Done false
11526 4035 Acked Write 7 Done Sent Done false
11527 4036 Acked Read 10 Done Sent Done false
11528 4037 Acked Write 10 Done Sent Done false
11529 4038 Acked Read 4 Done Sent Done false
11530 4039 Acked Read 10 Done Sent Done false
11531 4040 Acked Write 3 Done Sent Done false
11532 4041 Acked Read 5 Done Sent Done false
11533 4042 Acked Read 4 Done Sent Done false
11534 4043 Acked Read 7 Done Sent Done false
11535 4044 Acked Write 1 Done Sent Done false
11536 4045 Acked Write 2 Done Sent Done false
11537 4046 Acked Read 3 Done Sent Done false
11538 4047 Acked Read 1 Done Sent Done false
11539 4048 Acked Read 9 Done Sent Done false
11540 4049 Acked Write 1 Done Sent Done false
11541 4050 Acked Read 5 Done Sent Done false
11542 4051 Acked Read 4 Done Sent Done false
11543 4052 Acked Read 3 Done Sent Done false
11544 4053 Acked Write 7 Done Sent Done false
11545 4054 Acked Read 10 Done Sent Done false
11546 4055 Acked Read 1 Done Sent Done false
11547 4056 Acked Read 9 Done Sent Done false
11548 4057 Acked Read 5 Done Sent Done false
11549 4058 Acked Read 4 Done Sent Done false
11550 4059 Acked Write 1 Done Sent Done false
11551 4060 Acked Flush 0 Done Sent Done false
11552 4061 Acked Read 6 Done Sent Done false
11553 4062 Acked Read 2 Done Sent Done false
11554 4063 Acked Read 7 Done Sent Done false
11555 4064 Acked Read 4 Done Sent Done false
11556 4065 Acked Read 8 Done Sent Done false
11557 4066 Acked Write 9 Done Sent Done false
11558 4067 Acked Flush 0 Done Sent Done false
11559 4068 Acked Flush 0 Done Sent Done false
11560 4069 Acked Write 1 Done Sent Done false
11561 4070 Acked Write 10 Done Sent Done false
11562 4071 Acked Write 4 Done Sent Done false
11563 4072 Acked Write 8 Done Sent Done false
11564 4073 Acked Read 2 Done Sent Done false
11565 4074 Acked Flush 0 Done Sent Done false
11566 4075 Acked Write 1 Done Sent Done false
11567 4076 Acked Flush 0 Done Sent Done false
11568 4077 Acked Read 2 Done Sent Done false
11569 4078 Acked Read 4 Done Sent Done false
11570 4079 Acked Write 4 Done Sent Done false
11571 4080 Acked Read 3 Done Sent Done false
11572 4081 Acked Flush 0 Done Sent Done false
11573 4082 Acked Write 1 Done Sent Done false
11574 4083 Acked Write 8 Done Sent Done false
11575 4084 Acked Read 9 Done Sent Done false
11576 4085 Acked Write 10 Done Sent Done false
11577 4086 Acked Read 3 Done Sent Done false
11578 4087 Acked Write 2 Done Sent Done false
11579 4088 Acked Read 9 Done Sent Done false
11580 4089 Acked Write 7 Done Sent Done false
11581 4090 Acked Write 7 Done Sent Done false
11582 4091 Acked Read 7 Done Sent Done false
11583 4092 Acked Read 4 Done Sent Done false
11584 4093 Acked Read 1 Done Sent Done false
11585 4094 Acked Flush 0 Done Sent Done false
11586 4095 Acked Write 2 Done Sent Done false
11587 4096 Acked Read 1 Done Sent Done false
11588 4097 Acked Write 8 Done Sent Done false
11589 4098 Acked Flush 0 Done Sent Done false
11590 4099 Acked Write 10 Done Sent Done false
11591 4100 Acked Read 8 Done Sent Done false
11592 4101 Acked Flush 0 Done Sent Done false
11593 4102 Acked Flush 0 Done Sent Done false
11594 4103 Acked Flush 0 Done Sent Done false
11595 4104 Acked Write 8 Done Sent Done false
11596 4105 Acked Read 2 Done Sent Done false
11597 4106 Acked Flush 0 Done Sent Done false
11598 4107 Acked Write 3 Done Sent Done false
11599 4108 Acked Read 5 Done Sent Done false
11600 4109 Acked Read 3 Done Sent Done false
11601 4110 Acked Write 10 Done Sent Done false
11602 4111 Acked Write 2 Done Sent Done false
11603 4112 Acked Read 1 Done Sent Done false
11604 4113 Acked Read 6 Done Sent Done false
11605 4114 Acked Read 10 Done Sent Done false
11606 4115 Acked Read 4 Done Sent Done false
11607 4116 Acked Write 1 Done Sent Done false
11608 4117 Acked Read 4 Done Sent Done false
11609 4118 Acked Read 5 Done Sent Done false
11610 4119 Acked Read 2 Done Sent Done false
11611 4120 Acked Flush 0 Done Sent Done false
11612 4121 Acked Write 7 Done Sent Done false
11613 4122 Acked Write 3 Done Sent Done false
11614 4123 Acked Flush 0 Done Sent Done false
11615 4124 Acked Flush 0 Done Sent Done false
11616 4125 Acked Read 3 Done Sent Done false
11617 4126 Acked Read 8 Done Sent Done false
11618 4127 Acked Write 3 Done Sent Done false
11619 4128 Acked Write 4 Done Sent Done false
11620 4129 Acked Read 9 Done Sent Done false
11621 4130 Acked Read 9 Done Sent Done false
11622 4131 Acked Write 2 Done Sent Done false
11623 4132 Acked Write 5 Done Sent Done false
11624 4133 Acked Read 5 Done Sent Done false
11625 4134 Acked Write 5 Done Sent Done false
11626 4135 Acked Write 9 Done Sent Done false
11627 4136 Acked Write 6 Done Sent Done false
11628 4137 Acked Read 1 Done Sent Done false
11629 4138 Acked Read 3 Done Sent Done false
11630 4139 Acked Read 8 Done Sent Done false
11631 4140 Acked Read 4 Done Sent Done false
11632 4141 Acked Read 2 Done Sent Done false
11633 4142 Acked Flush 0 Done Sent Done false
11634 4143 Acked Write 1 Done Sent Done false
11635 4144 Acked Read 7 Done Sent Done false
11636 4145 Acked Write 7 Done Sent Done false
11637 4146 Acked Write 5 Done Sent Done false
11638 4147 Acked Read 3 Done Sent Done false
11639 4148 Acked Read 3 Done Sent Done false
11640 4149 Acked Read 3 Done Sent Done false
11641 4150 Acked Write 2 Done Sent Done false
11642 4151 Acked Write 5 Done Sent Done false
11643 4152 Acked Write 2 Done Sent Done false
11644 4153 Acked Read 3 Done Sent Done false
11645 4154 Acked Write 1 Done Sent Done false
11646 4155 Acked Read 10 Done Sent Done false
11647 4156 Acked Write 4 Done Sent Done false
11648 4157 Acked Read 8 Done Sent Done false
11649 4158 Acked Read 6 Done Sent Done false
11650 4159 Acked Flush 0 Done Sent Done false
11651 4160 Acked Read 5 Done Sent Done false
11652 4161 Acked Write 6 Done Sent Done false
11653 4162 Acked Read 8 Done Sent Done false
11654 4163 Acked Read 6 Done Sent Done false
11655 4164 Acked Write 4 Done Sent Done false
11656 4165 Acked Write 3 Done Sent Done false
11657 4166 Acked Read 5 Done Sent Done false
11658 4167 Acked Read 6 Done Sent Done false
11659 4168 Acked Read 10 Done Sent Done false
11660 4169 Acked Write 4 Done Sent Done false
11661 4170 Acked Write 4 Done Sent Done false
11662 4171 Acked Read 8 Done Sent Done false
11663 4172 Acked Read 7 Done Sent Done false
11664 4173 Acked Write 2 Done Sent Done false
11665 4174 Acked Read 1 Done Sent Done false
11666 4175 Acked Read 4 Done Sent Done false
11667 4176 Acked Write 4 Done Sent Done false
11668 4177 Acked Write 10 Done Sent Done false
11669 4178 Acked Read 5 Done Sent Done false
11670 4179 Acked Read 3 Done Sent Done false
11671 4180 Acked Write 6 Done Sent Done false
11672 4181 Acked Write 1 Done Sent Done false
11673 4182 Acked Flush 0 Done Sent Done false
11674 4183 Acked Write 8 Done Sent Done false
11675 4184 Acked Read 9 Done Sent Done false
11676 4185 Acked Flush 0 Done Sent Done false
11677 4186 Acked Write 8 Done Sent Done false
11678 4187 Acked Read 4 Done Sent Done false
11679 4188 Acked Read 5 Done Sent Done false
11680 4189 Acked Read 6 Done Sent Done false
11681 4190 Acked Write 4 Done Sent Done false
11682 4191 Acked Write 2 Done Sent Done false
11683 4192 Acked Read 5 Done Sent Done false
11684 4193 Acked Write 3 Done Sent Done false
11685 4194 Acked Read 3 Done Sent Done false
11686 4195 Acked Write 4 Done Sent Done false
11687 4196 Acked Read 9 Done Sent Done false
11688 4197 Acked Write 8 Done Sent Done false
11689 4198 Acked Read 3 Done Sent Done false
11690 4199 Acked Read 10 Done Sent Done false
11691 4200 Acked Read 2 Done Sent Done false
11692 4201 Acked Write 8 Done Sent Done false
11693 4202 Acked Read 3 Done Sent Done false
11694 4203 Acked Read 7 Done Sent Done false
11695 4204 Acked Read 4 Done Sent Done false
11696 4205 Acked Read 10 Done Sent Done false
11697 4206 Acked Write 7 Done Sent Done false
11698 4207 Acked Read 1 Done Sent Done false
11699 4208 Acked Flush 0 Done Sent Done false
11700 4209 Acked Write 9 Done Sent Done false
11701 4210 Acked Write 3 Done Sent Done false
11702 4211 Acked Write 9 Done Sent Done false
11703 4212 Acked Flush 0 Done Sent Done false
11704 4213 Acked Write 8 Done Sent Done false
11705 4214 Acked Write 3 Done Sent Done false
11706 4215 Acked Read 6 Done Sent Done false
11707 4216 Acked Read 6 Done Sent Done false
11708 4217 Acked Write 6 Done Sent Done false
11709 4218 Acked Read 10 Done Sent Done false
11710 4219 Acked Read 8 Done Sent Done false
11711 4220 Acked Read 5 Done Sent Done false
11712 4221 Acked Read 5 Done Sent Done false
11713 4222 Acked Read 5 Done Sent Done false
11714 4223 Acked Read 1 Done Sent Done false
11715 4224 Acked Flush 0 Done Sent Done false
11716 4225 Acked Read 1 Done Sent Done false
11717 4226 Acked Flush 0 Done Sent Done false
11718 4227 Acked Read 7 Done Sent Done false
11719 4228 Acked Flush 0 Done Sent Done false
11720 4229 Acked Read 10 Done Sent Done false
11721 4230 Acked Write 2 Done Sent Done false
11722 4231 Acked Read 5 Done Sent Done false
11723 4232 Acked Read 5 Done Sent Done false
11724 4233 Acked Write 6 Done Sent Done false
11725 4234 Acked Write 4 Done Sent Done false
11726 4235 Acked Write 10 Done Sent Done false
11727 4236 Acked Write 10 Done Sent Done false
11728 4237 Acked Write 4 Done Sent Done false
11729 4238 Acked Flush 0 Done Sent Done false
11730 4239 Acked Read 10 Done Sent Done false
11731 4240 Acked Write 8 Done Sent Done false
11732 4241 Acked Read 9 Done Sent Done false
11733 4242 Acked Write 9 Done Sent Done false
11734 4243 Acked Write 8 Done Sent Done false
11735 4244 Acked Write 3 Done Sent Done false
11736 4245 Acked Read 2 Done Sent Done false
11737 4246 Acked Write 1 Done Sent Done false
11738 4247 Acked Read 2 Done Sent Done false
11739 4248 Acked Read 3 Done Sent Done false
11740 4249 Acked Write 10 Done Sent Done false
11741 4250 Acked Flush 0 Done Sent Done false
11742 4251 Acked Write 10 Done Sent Done false
11743 4252 Acked Read 6 Done Sent Done false
11744 4253 Acked Read 3 Done Sent Done false
11745 4254 Acked Read 6 Done Sent Done false
11746 4255 Acked Read 5 Done Sent Done false
11747 4256 Acked Write 7 Done Sent Done false
11748 4257 Acked Read 4 Done Sent Done false
11749 4258 Acked Write 1 Done Sent Done false
11750 4259 Acked Write 6 Done Sent Done false
11751 4260 Acked Read 7 Done Sent Done false
11752 4261 Acked Read 6 Done Sent Done false
11753 4262 Acked Read 10 Done Sent Done false
11754 4263 Acked Read 2 Done Sent Done false
11755 4264 Acked Write 2 Done Sent Done false
11756 4265 Acked Write 7 Done Sent Done false
11757 4266 Acked Flush 0 Done Sent Done false
11758 4267 Acked Flush 0 Done Sent Done false
11759 4268 Acked Write 6 Done Sent Done false
11760 4269 Acked Read 10 Done Sent Done false
11761 4270 Acked Read 4 Done Sent Done false
11762 4271 Acked Read 2 Done Sent Done false
11763 4272 Acked Read 6 Done Sent Done false
11764 4273 Acked Read 1 Done Sent Done false
11765 4274 Acked Write 1 Done Sent Done false
11766 4275 Acked Write 2 Done Sent Done false
11767 4276 Acked Flush 0 Done Sent Done false
11768 4277 Acked Read 2 Done Sent Done false
11769 4278 Acked Read 4 Done Sent Done false
11770 4279 Acked Read 9 Done Sent Done false
11771 4280 Acked Read 5 Done Sent Done false
11772 4281 Acked Read 4 Done Sent Done false
11773 4282 Acked Read 9 Done Sent Done false
11774 4283 Acked Write 3 Done Sent Done false
11775 4284 Acked Write 3 Done Sent Done false
11776 4285 Acked Write 9 Done Sent Done false
11777 4286 Acked Flush 0 Done Sent Done false
11778 4287 Acked Write 2 Done Sent Done false
11779 4288 Acked Write 9 Done Sent Done false
11780 4289 Acked Write 8 Done Sent Done false
11781 4290 Acked Write 3 Done Sent Done false
11782 4291 Acked Read 4 Done Sent Done false
11783 4292 Acked Write 3 Done Sent Done false
11784 4293 Acked Read 4 Done Sent Done false
11785 4294 Acked Flush 0 Done Sent Done false
11786 4295 Acked Write 8 Done Sent Done false
11787 4296 Acked Write 3 Done Sent Done false
11788 4297 Acked Read 4 Done Sent Done false
11789 4298 Acked Write 6 Done Sent Done false
11790 4299 Acked Write 9 Done Sent Done false
11791 4300 Acked Read 7 Done Sent Done false
11792 4301 Acked Write 1 Done Sent Done false
11793 4302 Acked Write 6 Done Sent Done false
11794 4303 Acked Read 2 Done Sent Done false
11795 4304 Acked Write 7 Done Sent Done false
11796 4305 Acked Flush 0 Done Sent Done false
11797 4306 Acked Read 3 Done Sent Done false
11798 4307 Acked Read 1 Done Sent Done false
11799 4308 Acked Write 8 Done Sent Done false
11800 4309 Acked Read 9 Done Sent Done false
11801 4310 Acked Write 10 Done Sent Done false
11802 4311 Acked Read 10 Done Sent Done false
11803 4312 Acked Read 8 Done Sent Done false
11804 4313 Acked Read 1 Done Sent Done false
11805 4314 Acked Write 1 Done Sent Done false
11806 4315 Acked Flush 0 Done Sent Done false
11807 4316 Acked Read 5 Done Sent Done false
11808 4317 Acked Read 2 Done Sent Done false
11809 4318 Acked Write 9 Done Sent Done false
11810 4319 Acked Write 4 Done Sent Done false
11811 4320 Acked Flush 0 Done Sent Done false
11812 STATES DS:0 DS:1 DS:2 TOTAL
11813 Sent 0 302 0 302
11814 Done 302 0 302 604
11815 Skipped 0 0 0 0
11816 Error 0 0 0 0
11817 Last Flush: 4320 4018 4320
11818 Downstairs last five completed: 4018 4017 4016 4015 4014
11819 Upstairs last five completed: 4320 4319 4318 4317 4316
11820 CLIENT: Up:0 ds:302 act:2
11821 ----------------------------------------------------------------
11822 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11823 States: Active Offline Active
11824 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11825 4019 Acked Write 9 Done Sent Done false
11826 4020 Acked Write 7 Done Sent Done false
11827 4021 Acked Read 9 Done Sent Done false
11828 4022 Acked Write 10 Done Sent Done false
11829 4023 Acked Read 2 Done Sent Done false
11830 4024 Acked Write 5 Done Sent Done false
11831 4025 Acked Read 10 Done Sent Done false
11832 4026 Acked Read 7 Done Sent Done false
11833 4027 Acked Read 9 Done Sent Done false
11834 4028 Acked Write 8 Done Sent Done false
11835 4029 Acked Flush 0 Done Sent Done false
11836 4030 Acked Read 7 Done Sent Done false
11837 4031 Acked Write 6 Done Sent Done false
11838 4032 Acked Write 10 Done Sent Done false
11839 4033 Acked Write 7 Done Sent Done false
11840 4034 Acked Write 10 Done Sent Done false
11841 4035 Acked Write 7 Done Sent Done false
11842 4036 Acked Read 10 Done Sent Done false
11843 4037 Acked Write 10 Done Sent Done false
11844 4038 Acked Read 4 Done Sent Done false
11845 4039 Acked Read 10 Done Sent Done false
11846 4040 Acked Write 3 Done Sent Done false
11847 4041 Acked Read 5 Done Sent Done false
11848 4042 Acked Read 4 Done Sent Done false
11849 4043 Acked Read 7 Done Sent Done false
11850 4044 Acked Write 1 Done Sent Done false
11851 4045 Acked Write 2 Done Sent Done false
11852 4046 Acked Read 3 Done Sent Done false
11853 4047 Acked Read 1 Done Sent Done false
11854 4048 Acked Read 9 Done Sent Done false
11855 4049 Acked Write 1 Done Sent Done false
11856 4050 Acked Read 5 Done Sent Done false
11857 4051 Acked Read 4 Done Sent Done false
11858 4052 Acked Read 3 Done Sent Done false
11859 4053 Acked Write 7 Done Sent Done false
11860 4054 Acked Read 10 Done Sent Done false
11861 4055 Acked Read 1 Done Sent Done false
11862 4056 Acked Read 9 Done Sent Done false
11863 4057 Acked Read 5 Done Sent Done false
11864 4058 Acked Read 4 Done Sent Done false
11865 4059 Acked Write 1 Done Sent Done false
11866 4060 Acked Flush 0 Done Sent Done false
11867 4061 Acked Read 6 Done Sent Done false
11868 4062 Acked Read 2 Done Sent Done false
11869 4063 Acked Read 7 Done Sent Done false
11870 4064 Acked Read 4 Done Sent Done false
11871 4065 Acked Read 8 Done Sent Done false
11872 4066 Acked Write 9 Done Sent Done false
11873 4067 Acked Flush 0 Done Sent Done false
11874 4068 Acked Flush 0 Done Sent Done false
11875 4069 Acked Write 1 Done Sent Done false
11876 4070 Acked Write 10 Done Sent Done false
11877 4071 Acked Write 4 Done Sent Done false
11878 4072 Acked Write 8 Done Sent Done false
11879 4073 Acked Read 2 Done Sent Done false
11880 4074 Acked Flush 0 Done Sent Done false
11881 4075 Acked Write 1 Done Sent Done false
11882 4076 Acked Flush 0 Done Sent Done false
11883 4077 Acked Read 2 Done Sent Done false
11884 4078 Acked Read 4 Done Sent Done false
11885 4079 Acked Write 4 Done Sent Done false
11886 4080 Acked Read 3 Done Sent Done false
11887 4081 Acked Flush 0 Done Sent Done false
11888 4082 Acked Write 1 Done Sent Done false
11889 4083 Acked Write 8 Done Sent Done false
11890 4084 Acked Read 9 Done Sent Done false
11891 4085 Acked Write 10 Done Sent Done false
11892 4086 Acked Read 3 Done Sent Done false
11893 4087 Acked Write 2 Done Sent Done false
11894 4088 Acked Read 9 Done Sent Done false
11895 4089 Acked Write 7 Done Sent Done false
11896 4090 Acked Write 7 Done Sent Done false
11897 4091 Acked Read 7 Done Sent Done false
11898 4092 Acked Read 4 Done Sent Done false
11899 4093 Acked Read 1 Done Sent Done false
11900 4094 Acked Flush 0 Done Sent Done false
11901 4095 Acked Write 2 Done Sent Done false
11902 4096 Acked Read 1 Done Sent Done false
11903 4097 Acked Write 8 Done Sent Done false
11904 4098 Acked Flush 0 Done Sent Done false
11905 4099 Acked Write 10 Done Sent Done false
11906 4100 Acked Read 8 Done Sent Done false
11907 4101 Acked Flush 0 Done Sent Done false
11908 4102 Acked Flush 0 Done Sent Done false
11909 4103 Acked Flush 0 Done Sent Done false
11910 4104 Acked Write 8 Done Sent Done false
11911 4105 Acked Read 2 Done Sent Done false
11912 4106 Acked Flush 0 Done Sent Done false
11913 4107 Acked Write 3 Done Sent Done false
11914 4108 Acked Read 5 Done Sent Done false
11915 4109 Acked Read 3 Done Sent Done false
11916 4110 Acked Write 10 Done Sent Done false
11917 4111 Acked Write 2 Done Sent Done false
11918 4112 Acked Read 1 Done Sent Done false
11919 4113 Acked Read 6 Done Sent Done false
11920 4114 Acked Read 10 Done Sent Done false
11921 4115 Acked Read 4 Done Sent Done false
11922 4116 Acked Write 1 Done Sent Done false
11923 4117 Acked Read 4 Done Sent Done false
11924 4118 Acked Read 5 Done Sent Done false
11925 4119 Acked Read 2 Done Sent Done false
11926 4120 Acked Flush 0 Done Sent Done false
11927 4121 Acked Write 7 Done Sent Done false
11928 4122 Acked Write 3 Done Sent Done false
11929 4123 Acked Flush 0 Done Sent Done false
11930 4124 Acked Flush 0 Done Sent Done false
11931 4125 Acked Read 3 Done Sent Done false
11932 4126 Acked Read 8 Done Sent Done false
11933 4127 Acked Write 3 Done Sent Done false
11934 4128 Acked Write 4 Done Sent Done false
11935 4129 Acked Read 9 Done Sent Done false
11936 4130 Acked Read 9 Done Sent Done false
11937 4131 Acked Write 2 Done Sent Done false
11938 4132 Acked Write 5 Done Sent Done false
11939 4133 Acked Read 5 Done Sent Done false
11940 4134 Acked Write 5 Done Sent Done false
11941 4135 Acked Write 9 Done Sent Done false
11942 4136 Acked Write 6 Done Sent Done false
11943 4137 Acked Read 1 Done Sent Done false
11944 4138 Acked Read 3 Done Sent Done false
11945 4139 Acked Read 8 Done Sent Done false
11946 4140 Acked Read 4 Done Sent Done false
11947 4141 Acked Read 2 Done Sent Done false
11948 4142 Acked Flush 0 Done Sent Done false
11949 4143 Acked Write 1 Done Sent Done false
11950 4144 Acked Read 7 Done Sent Done false
11951 4145 Acked Write 7 Done Sent Done false
11952 4146 Acked Write 5 Done Sent Done false
11953 4147 Acked Read 3 Done Sent Done false
11954 4148 Acked Read 3 Done Sent Done false
11955 4149 Acked Read 3 Done Sent Done false
11956 4150 Acked Write 2 Done Sent Done false
11957 4151 Acked Write 5 Done Sent Done false
11958 4152 Acked Write 2 Done Sent Done false
11959 4153 Acked Read 3 Done Sent Done false
11960 4154 Acked Write 1 Done Sent Done false
11961 4155 Acked Read 10 Done Sent Done false
11962 4156 Acked Write 4 Done Sent Done false
11963 4157 Acked Read 8 Done Sent Done false
11964 4158 Acked Read 6 Done Sent Done false
11965 4159 Acked Flush 0 Done Sent Done false
11966 4160 Acked Read 5 Done Sent Done false
11967 4161 Acked Write 6 Done Sent Done false
11968 4162 Acked Read 8 Done Sent Done false
11969 4163 Acked Read 6 Done Sent Done false
11970 4164 Acked Write 4 Done Sent Done false
11971 4165 Acked Write 3 Done Sent Done false
11972 4166 Acked Read 5 Done Sent Done false
11973 4167 Acked Read 6 Done Sent Done false
11974 4168 Acked Read 10 Done Sent Done false
11975 4169 Acked Write 4 Done Sent Done false
11976 4170 Acked Write 4 Done Sent Done false
11977 4171 Acked Read 8 Done Sent Done false
11978 4172 Acked Read 7 Done Sent Done false
11979 4173 Acked Write 2 Done Sent Done false
11980 4174 Acked Read 1 Done Sent Done false
11981 4175 Acked Read 4 Done Sent Done false
11982 4176 Acked Write 4 Done Sent Done false
11983 4177 Acked Write 10 Done Sent Done false
11984 4178 Acked Read 5 Done Sent Done false
11985 4179 Acked Read 3 Done Sent Done false
11986 4180 Acked Write 6 Done Sent Done false
11987 4181 Acked Write 1 Done Sent Done false
11988 4182 Acked Flush 0 Done Sent Done false
11989 4183 Acked Write 8 Done Sent Done false
11990 4184 Acked Read 9 Done Sent Done false
11991 4185 Acked Flush 0 Done Sent Done false
11992 4186 Acked Write 8 Done Sent Done false
11993 4187 Acked Read 4 Done Sent Done false
11994 4188 Acked Read 5 Done Sent Done false
11995 4189 Acked Read 6 Done Sent Done false
11996 4190 Acked Write 4 Done Sent Done false
11997 4191 Acked Write 2 Done Sent Done false
11998 4192 Acked Read 5 Done Sent Done false
11999 4193 Acked Write 3 Done Sent Done false
12000 4194 Acked Read 3 Done Sent Done false
12001 4195 Acked Write 4 Done Sent Done false
12002 4196 Acked Read 9 Done Sent Done false
12003 4197 Acked Write 8 Done Sent Done false
12004 4198 Acked Read 3 Done Sent Done false
12005 4199 Acked Read 10 Done Sent Done false
12006 4200 Acked Read 2 Done Sent Done false
12007 4201 Acked Write 8 Done Sent Done false
12008 4202 Acked Read 3 Done Sent Done false
12009 4203 Acked Read 7 Done Sent Done false
12010 4204 Acked Read 4 Done Sent Done false
12011 4205 Acked Read 10 Done Sent Done false
12012 4206 Acked Write 7 Done Sent Done false
12013 4207 Acked Read 1 Done Sent Done false
12014 4208 Acked Flush 0 Done Sent Done false
12015 4209 Acked Write 9 Done Sent Done false
12016 4210 Acked Write 3 Done Sent Done false
12017 4211 Acked Write 9 Done Sent Done false
12018 4212 Acked Flush 0 Done Sent Done false
12019 4213 Acked Write 8 Done Sent Done false
12020 4214 Acked Write 3 Done Sent Done false
12021 4215 Acked Read 6 Done Sent Done false
12022 4216 Acked Read 6 Done Sent Done false
12023 4217 Acked Write 6 Done Sent Done false
12024 4218 Acked Read 10 Done Sent Done false
12025 4219 Acked Read 8 Done Sent Done false
12026 4220 Acked Read 5 Done Sent Done false
12027 4221 Acked Read 5 Done Sent Done false
12028 4222 Acked Read 5 Done Sent Done false
12029 4223 Acked Read 1 Done Sent Done false
12030 4224 Acked Flush 0 Done Sent Done false
12031 4225 Acked Read 1 Done Sent Done false
12032 4226 Acked Flush 0 Done Sent Done false
12033 4227 Acked Read 7 Done Sent Done false
12034 4228 Acked Flush 0 Done Sent Done false
12035 4229 Acked Read 10 Done Sent Done false
12036 4230 Acked Write 2 Done Sent Done false
12037 4231 Acked Read 5 Done Sent Done false
12038 4232 Acked Read 5 Done Sent Done false
12039 4233 Acked Write 6 Done Sent Done false
12040 4234 Acked Write 4 Done Sent Done false
12041 4235 Acked Write 10 Done Sent Done false
12042 4236 Acked Write 10 Done Sent Done false
12043 4237 Acked Write 4 Done Sent Done false
12044 4238 Acked Flush 0 Done Sent Done false
12045 4239 Acked Read 10 Done Sent Done false
12046 4240 Acked Write 8 Done Sent Done false
12047 4241 Acked Read 9 Done Sent Done false
12048 4242 Acked Write 9 Done Sent Done false
12049 4243 Acked Write 8 Done Sent Done false
12050 4244 Acked Write 3 Done Sent Done false
12051 4245 Acked Read 2 Done Sent Done false
12052 4246 Acked Write 1 Done Sent Done false
12053 4247 Acked Read 2 Done Sent Done false
12054 4248 Acked Read 3 Done Sent Done false
12055 4249 Acked Write 10 Done Sent Done false
12056 4250 Acked Flush 0 Done Sent Done false
12057 4251 Acked Write 10 Done Sent Done false
12058 4252 Acked Read 6 Done Sent Done false
12059 4253 Acked Read 3 Done Sent Done false
12060 4254 Acked Read 6 Done Sent Done false
12061 4255 Acked Read 5 Done Sent Done false
12062 4256 Acked Write 7 Done Sent Done false
12063 4257 Acked Read 4 Done Sent Done false
12064 4258 Acked Write 1 Done Sent Done false
12065 4259 Acked Write 6 Done Sent Done false
12066 4260 Acked Read 7 Done Sent Done false
12067 4261 Acked Read 6 Done Sent Done false
12068 4262 Acked Read 10 Done Sent Done false
12069 4263 Acked Read 2 Done Sent Done false
12070 4264 Acked Write 2 Done Sent Done false
12071 4265 Acked Write 7 Done Sent Done false
12072 4266 Acked Flush 0 Done Sent Done false
12073 4267 Acked Flush 0 Done Sent Done false
12074 4268 Acked Write 6 Done Sent Done false
12075 4269 Acked Read 10 Done Sent Done false
12076 4270 Acked Read 4 Done Sent Done false
12077 4271 Acked Read 2 Done Sent Done false
12078 4272 Acked Read 6 Done Sent Done false
12079 4273 Acked Read 1 Done Sent Done false
12080 4274 Acked Write 1 Done Sent Done false
12081 4275 Acked Write 2 Done Sent Done false
12082 4276 Acked Flush 0 Done Sent Done false
12083 4277 Acked Read 2 Done Sent Done false
12084 4278 Acked Read 4 Done Sent Done false
12085 4279 Acked Read 9 Done Sent Done false
12086 4280 Acked Read 5 Done Sent Done false
12087 4281 Acked Read 4 Done Sent Done false
12088 4282 Acked Read 9 Done Sent Done false
12089 4283 Acked Write 3 Done Sent Done false
12090 4284 Acked Write 3 Done Sent Done false
12091 4285 Acked Write 9 Done Sent Done false
12092 4286 Acked Flush 0 Done Sent Done false
12093 4287 Acked Write 2 Done Sent Done false
12094 4288 Acked Write 9 Done Sent Done false
12095 4289 Acked Write 8 Done Sent Done false
12096 4290 Acked Write 3 Done Sent Done false
12097 4291 Acked Read 4 Done Sent Done false
12098 4292 Acked Write 3 Done Sent Done false
12099 4293 Acked Read 4 Done Sent Done false
12100 4294 Acked Flush 0 Done Sent Done false
12101 4295 Acked Write 8 Done Sent Done false
12102 4296 Acked Write 3 Done Sent Done false
12103 4297 Acked Read 4 Done Sent Done false
12104 4298 Acked Write 6 Done Sent Done false
12105 4299 Acked Write 9 Done Sent Done false
12106 4300 Acked Read 7 Done Sent Done false
12107 4301 Acked Write 1 Done Sent Done false
12108 4302 Acked Write 6 Done Sent Done false
12109 4303 Acked Read 2 Done Sent Done false
12110 4304 Acked Write 7 Done Sent Done false
12111 4305 Acked Flush 0 Done Sent Done false
12112 4306 Acked Read 3 Done Sent Done false
12113 4307 Acked Read 1 Done Sent Done false
12114 4308 Acked Write 8 Done Sent Done false
12115 4309 Acked Read 9 Done Sent Done false
12116 4310 Acked Write 10 Done Sent Done false
12117 4311 Acked Read 10 Done Sent Done false
12118 4312 Acked Read 8 Done Sent Done false
12119 4313 Acked Read 1 Done Sent Done false
12120 4314 Acked Write 1 Done Sent Done false
12121 4315 Acked Flush 0 Done Sent Done false
12122 4316 Acked Read 5 Done Sent Done false
12123 4317 Acked Read 2 Done Sent Done false
12124 4318 Acked Write 9 Done Sent Done false
12125 4319 Acked Write 4 Done Sent Done false
12126 4320 Acked Flush 0 Done Sent Done false
12127 STATES DS:0 DS:1 DS:2 TOTAL
12128 Sent 0 302 0 302
12129 Done 302 0 302 604
12130 Skipped 0 0 0 0
12131 Error 0 0 0 0
12132 Last Flush: 4320 4018 4320
12133 Downstairs last five completed: 4018 4017 4016 4015 4014
12134 Upstairs last five completed: 4320 4319 4318 4317 4316
12135 CLIENT: Up:0 ds:302 act:2
121362025-01-21T15:49:43.803ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121372025-01-21T15:49:43.803ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64660) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121382025-01-21T15:49:43.804ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121392025-01-21T15:49:43.805ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121402025-01-21T15:49:43.805ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121412025-01-21T15:49:43.805ZINFOcrucible: send last flush ID to this DS: 4018 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121422025-01-21T15:49:43.805ZINFOcrucible: Replied this last flush ID: 4018 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121432025-01-21T15:49:43.805ZINFOcrucible: [1] client re-new 302 jobs since flush 4018 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121442025-01-21T15:49:43.806ZINFOcrucible: [1] Replayed 302 jobs since flush: 4018 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
12145 ----------------------------------------------------------------
12146 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12147 Downstairs last five completed: 4320 4319 4318 4317 4316
12148 Upstairs last five completed: 4320 4319 4318 4317 4316
12149 CLIENT: Up:0 ds:0 act:3
12150 Replay: All jobs finished
12151 Main task has work to do, go find it
12152 got dsc Stop(1)
12153 stop 1
12154 Jan 21 15:49:44.052 INFO[8820] Got stop action so:false kr:false
12155 request completed, latency_us: 332, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 0c8e3446-7180-460d-8381-155960b7addb, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
121562025-01-21T15:49:44.052ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121572025-01-21T15:49:44.052ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121582025-01-21T15:49:44.052ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
121592025-01-21T15:49:44.052ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
12160 Jan 21 15:49:44.052 INFO request completed, latency_us: 171, response_code: 200, uri: /state/cid/1, method: GET, req_id: 59f71f76-37a5-4d2f-97c5-f022db7c198c, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
12161 [8820] Exited with: ExitStatus(unix_wait_status(9))
12162 [8820][1] reports Exit
12163 Jan 21 15:49:48.055 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/1, method: GET, req_id: 4fc81846-f76b-4fec-80fc-f4924728e308, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
12164 001/300 Read block 1457 len 8192
12165 002/300 Read block 1392 len 4096
12166 003/300 Read block 4942 len 8192
12167 004/300 Read block 315 len 28672
12168 005/300 Write block 1862 len 36864 data: 4 4 4 5 4 4 3 3 3
12169 006/300 Write block 4770 len 12288 data: 3 2 2
12170 007/300 Read block 2067 len 4096
12171 008/300 Read block 1518 len 28672
12172 009/300 Write block 2714 len 4096 data: 5
12173 010/300 Write block 1568 len 36864 data: 3 3 3 3 3 3 4 4 5
12174 011/300 Write block 2300 len 36864 data: 3 3 3 3 3 3 3 2 2
12175 012/300 Write block 3947 len 24576 data: 4 5 3 3 3 4
12176 013/300 Read block 2234 len 24576
12177 014/300 Write block 2166 len 28672 data: 3 4 4 6 6 7 9
12178 015/300 Write block 1137 len 40960 data: 3 3 3 3 3 3 3 5 4 4
12179 016/300 Flush
12180 017/300 Read block 4230 len 32768
12181 018/300 Read block 2477 len 32768
12182 019/300 Write block 2827 len 28672 data: 5 5 4 4 4 5 3
12183 020/300 Write block 2777 len 32768 data: 3 3 4 5 5 5 4 5
12184 021/300 Read block 538 len 40960
12185 022/300 Read block 3882 len 16384
12186 023/300 Flush
12187 024/300 Write block 3713 len 40960 data: 5 4 3 2 2 3 5 5 7 7
12188 025/300 Write block 4233 len 16384 data: 5 4 3 3
12189 026/300 Read block 1022 len 36864
12190 027/300 Read block 3039 len 28672
12191 028/300 Write block 609 len 4096 data: 4
12192 029/300 Read block 1449 len 8192
12193 030/300 Flush
12194 031/300 Read block 4477 len 40960
12195 032/300 Write block 4329 len 24576 data: 2 2 4 4 3 3
12196 033/300 Write block 4051 len 32768 data: 3 3 3 2 3 3 3 3
12197 034/300 Read block 2071 len 4096
12198 035/300 Read block 2408 len 20480
12199 036/300 Read block 1501 len 12288
12200 037/300 Read block 2362 len 20480
12201 038/300 Flush
12202 039/300 Read block 569 len 4096
12203 040/300 Read block 3625 len 12288
12204 041/300 Write block 2722 len 12288 data: 4 3 2
12205 042/300 Read block 4418 len 8192
12206 043/300 Read block 2657 len 20480
12207 044/300 Write block 3171 len 32768 data: 4 4 5 5 5 5 6 5
12208 045/300 Read block 2810 len 8192
12209 046/300 Write block 3720 len 16384 data: 6 8 8 7
12210 047/300 Read block 1160 len 4096
12211 048/300 Flush
12212 049/300 Read block 2930 len 24576
12213 050/300 Write block 3697 len 36864 data: 2 5 5 5 5 4 5 4 3
12214 051/300 Read block 2742 len 28672
12215 052/300 Read block 3259 len 28672
12216 053/300 Write block 3822 len 40960 data: 3 3 3 3 2 3 3 2 2 2
12217 054/300 Flush
12218 055/300 Write block 4292 len 8192 data: 2 2
12219 056/300 Write block 572 len 8192 data: 2 3
12220 057/300 Write block 591 len 40960 data: 4 5 6 5 5 4 3 2 4 4
12221 058/300 Write block 2706 len 20480 data: 2 5 5 5 5
12222 059/300 Read block 955 len 32768
12223 060/300 Write block 201 len 32768 data: 4 5 5 5 4 3 4 4
12224 061/300 Read block 3824 len 28672
12225 062/300 Read block 726 len 16384
12226 063/300 Read block 991 len 16384
12227 064/300 Read block 3945 len 32768
12228 065/300 Read block 589 len 36864
12229 066/300 Write block 963 len 24576 data: 2 2 2 3 5 3
12230 067/300 Write block 4803 len 32768 data: 4 4 4 4 3 3 4 3
12231 068/300 Read block 392 len 4096
12232 069/300 Write block 3475 len 40960 data: 5 6 7 4 4 4 4 4 3 2
12233 070/300 Read block 1748 len 40960
12234 071/300 Read block 4579 len 4096
12235 072/300 Read block 4957 len 28672
12236 073/300 Flush
12237 074/300 Write block 2578 len 20480 data: 3 3 3 3 2
12238 075/300 Read block 884 len 4096
12239 076/300 Read block 4161 len 4096
12240 077/300 Read block 1722 len 36864
12241 078/300 Flush
12242 079/300 Write block 625 len 12288 data: 3 4 4
12243 080/300 Write block 275 len 8192 data: 3 2
12244 081/300 Flush
12245 082/300 Write block 4430 len 40960 data: 2 2 3 3 3 3 3 3 4 4
12246 083/300 Read block 3084 len 24576
12247 084/300 Write block 4493 len 16384 data: 3 3 3 3
12248 085/300 Read block 1469 len 16384
12249 086/300 Write block 837 len 4096 data: 2
12250 087/300 Read block 44 len 16384
12251 088/300 Write block 121 len 4096 data: 2
12252 089/300 Read block 3910 len 4096
12253 090/300 Write block 903 len 12288 data: 3 3 3
12254 091/300 Read block 4957 len 4096
12255 092/300 Write block 2144 len 36864 data: 4 4 4 4 3 3 3 3 3
12256 093/300 Flush
12257 094/300 Read block 1611 len 28672
12258 095/300 Write block 4038 len 8192 data: 4 3
12259 096/300 Read block 1432 len 12288
12260 097/300 Write block 1381 len 32768 data: 6 7 6 4 3 3 3 3
12261 098/300 Read block 4657 len 16384
12262 099/300 Flush
12263 100/300 Write block 3344 len 4096 data: 4
12264 101/300 Read block 3507 len 4096
12265 102/300 Write block 4869 len 20480 data: 2 3 3 4 5
12266 103/300 Write block 682 len 32768 data: 2 4 3 4 4 5 5 4
12267 104/300 Read block 4592 len 16384
12268 105/300 Write block 4138 len 8192 data: 4 4
12269 106/300 Read block 1443 len 20480
12270 107/300 Write block 528 len 32768 data: 5 6 6 4 3 3 2 2
12271 108/300 Write block 4529 len 4096 data: 2
12272 109/300 Write block 4699 len 40960 data: 3 3 3 3 3 3 2 2 3 3
12273 110/300 Read block 1130 len 4096
12274 111/300 Read block 2009 len 24576
12275 112/300 Flush
12276 113/300 Read block 4668 len 28672
12277 114/300 Write block 181 len 28672 data: 3 3 4 4 4 5 6
12278 115/300 Read block 1830 len 40960
12279 116/300 Write block 1545 len 16384 data: 3 3 3 3
12280 117/300 Read block 931 len 8192
12281 118/300 Read block 4876 len 32768
12282 119/300 Read block 4257 len 32768
12283 120/300 Read block 1139 len 12288
12284 121/300 Write block 1637 len 32768 data: 2 2 2 3 3 3 3 3
12285 122/300 Write block 3390 len 36864 data: 4 4 3 3 2 2 2 2 3
12286 123/300 Write block 2268 len 36864 data: 2 2 2 2 2 2 2 2 2
12287 124/300 Write block 841 len 40960 data: 3 4 4 4 4 4 3 2 3 3
12288 125/300 Write block 1857 len 4096 data: 3
12289 126/300 Read block 392 len 36864
12290 127/300 Write block 3669 len 4096 data: 5
12291 128/300 Write block 4397 len 36864 data: 4 4 4 4 3 2 2 2 2
12292 129/300 Write block 2961 len 4096 data: 3
12293 130/300 Read block 4209 len 40960
12294 131/300 Read block 1697 len 24576
12295 132/300 Write block 124 len 32768 data: 2 2 3 3 3 4 4 4
12296 133/300 Read block 1935 len 40960
12297 134/300 Flush
12298 135/300 Read block 4516 len 20480
12299 136/300 Read block 3776 len 28672
12300 137/300 Read block 598 len 8192
12301 138/300 Write block 3269 len 20480 data: 5 5 5 3 3
12302 139/300 Write block 467 len 28672 data: 5 6 5 4 4 2 2
12303 140/300 Read block 2920 len 8192
12304 141/300 Write block 892 len 16384 data: 6 6 4 4
12305 142/300 Read block 904 len 20480
12306 143/300 Write block 1477 len 12288 data: 3 2 2
12307 144/300 Read block 1683 len 28672
12308 145/300 Read block 2724 len 40960
12309 146/300 Read block 3785 len 8192
12310 147/300 Read block 1013 len 12288
12311 148/300 Flush
12312 149/300 Read block 1448 len 28672
12313 150/300 Write block 304 len 40960 data: 4 4 4 2 2 2 2 2 2 2
12314 151/300 Flush
12315 152/300 Read block 2205 len 24576
12316 153/300 Write block 462 len 12288 data: 5 5 5
12317 154/300 Read block 388 len 32768
12318 155/300 Read block 1034 len 36864
12319 156/300 Write block 2866 len 8192 data: 2 2
12320 157/300 Read block 3588 len 40960
12321 158/300 Read block 435 len 12288
12322 159/300 Read block 4124 len 8192
12323 160/300 Write block 1350 len 16384 data: 4 3 5 4
12324 161/300 Write block 1615 len 4096 data: 2
12325 162/300 Write block 4038 len 24576 data: 5 4 2 2 2 2
12326 163/300 Flush
12327 164/300 Read block 3138 len 20480
12328 165/300 Flush
12329 166/300 Read block 1488 len 28672
12330 167/300 Write block 1832 len 24576 data: 2 2 2 3 3 4
12331 168/300 Flush
12332 169/300 Write block 715 len 24576 data: 3 3 3 3 3 3
12333 170/300 Read block 313 len 4096
12334 171/300 Flush
12335 172/300 Write block 1795 len 20480 data: 2 2 2 3 3
12336 173/300 Write block 2806 len 36864 data: 2 3 3 3 3 3 4 5 5
12337 174/300 Write block 2026 len 16384 data: 3 5 4 4
12338 175/300 Write block 3083 len 20480 data: 7 5 6 6 5
12339 176/300 Write block 2601 len 36864 data: 4 4 4 6 6 5 5 5 5
12340 177/300 Write block 2656 len 12288 data: 3 4 3
12341 178/300 Flush
12342 179/300 Write block 2553 len 20480 data: 3 3 3 3 3
12343 180/300 Read block 773 len 8192
12344 181/300 Read block 4284 len 12288
12345 182/300 Write block 3926 len 36864 data: 3 3 3 3 3 2 2 2 2
12346 183/300 Write block 180 len 32768 data: 2 4 4 5 5 5 6 7
12347 184/300 Read block 4170 len 40960
12348 185/300 Write block 1082 len 36864 data: 3 3 2 2 2 2 2 2 4
12349 186/300 Write block 3416 len 24576 data: 3 5 5 5 5 4
12350 187/300 Read block 1536 len 28672
12351 188/300 Read block 2549 len 16384
12352 189/300 Write block 4557 len 8192 data: 3 3
12353 190/300 Read block 339 len 4096
12354 191/300 Write block 2110 len 36864 data: 3 2 2 2 2 2 2 3 3
12355 192/300 Read block 4102 len 40960
12356 193/300 Write block 1428 len 28672 data: 5 6 3 3 3 5 5
12357 194/300 Write block 2165 len 20480 data: 3 4 5 5 7
12358 195/300 Read block 2384 len 16384
12359 196/300 Write block 4384 len 16384 data: 4 4 4 4
12360 197/300 Write block 4029 len 40960 data: 3 4 4 4 4 4 4 5 5 6
12361 198/300 Read block 2878 len 40960
12362 199/300 Write block 2742 len 40960 data: 2 2 2 3 3 3 4 4 4 3
12363 200/300 Read block 3073 len 28672
12364 201/300 Read block 2407 len 4096
12365 202/300 Write block 2541 len 32768 data: 4 4 4 5 4 4 4 5
12366 203/300 Write block 3114 len 28672 data: 3 3 4 4 3 3 3
12367 204/300 Read block 3766 len 8192
12368 205/300 Read block 39 len 12288
12369 206/300 Write block 4278 len 24576 data: 2 2 2 2 2 2
12370 207/300 Read block 4794 len 32768
12371 208/300 Read block 1248 len 8192
12372 209/300 Write block 381 len 12288 data: 3 3 3
12373 210/300 Read block 1277 len 16384
12374 211/300 Write block 789 len 32768 data: 3 4 5 5 5 4 3 3
12375 212/300 Write block 1105 len 12288 data: 7 5 5
12376 213/300 Flush
12377 214/300 Flush
12378 215/300 Read block 2280 len 16384
12379 216/300 Read block 3523 len 32768
12380 217/300 Read block 2928 len 32768
12381 218/300 Flush
12382 219/300 Write block 1738 len 28672 data: 6 5 4 4 4 4 3
12383 220/300 Read block 1742 len 4096
12384 221/300 Write block 4185 len 32768 data: 5 4 4 3 4 4 3 2
12385 222/300 Read block 359 len 4096
12386 223/300 Read block 1434 len 24576
12387 224/300 Read block 1417 len 40960
12388 225/300 Write block 35 len 36864 data: 2 2 5 5 5 5 5 6 5
12389 226/300 Write block 2357 len 12288 data: 4 4 3
12390 227/300 Write block 1349 len 4096 data: 4
12391 228/300 Read block 4615 len 4096
12392 229/300 Write block 3039 len 16384 data: 4 4 5 6
12393 230/300 Read block 3890 len 4096
12394 231/300 Write block 2269 len 20480 data: 3 3 3 3 3
12395 232/300 Read block 155 len 4096
12396 233/300 Read block 3293 len 32768
12397 234/300 Read block 3169 len 4096
12398 235/300 Read block 1262 len 24576
12399 236/300 Write block 4702 len 32768 data: 4 4 4 3 3 4 4 4
12400 237/300 Read block 4859 len 24576
12401 238/300 Read block 4875 len 28672
12402 239/300 Read block 3933 len 24576
12403 240/300 Write block 852 len 20480 data: 3 2 3 5 4
12404 241/300 Write block 1896 len 32768 data: 3 4 3 3 5 5 5 6
12405 242/300 Write block 4436 len 8192 data: 4 4
12406 243/300 Read block 3713 len 40960
12407 244/300 Read block 55 len 20480
12408 245/300 Write block 4492 len 12288 data: 3 4 4
12409 246/300 Read block 1471 len 8192
12410 247/300 Read block 229 len 4096
12411 248/300 Write block 1102 len 16384 data: 5 5 6 8
12412 249/300 Read block 2957 len 36864
12413 250/300 Write block 1791 len 36864 data: 2 2 2 2 3 3 3 4 4
12414 251/300 Write block 4691 len 28672 data: 3 2 2 2 3 3 3
12415 252/300 Write block 3869 len 40960 data: 3 3 3 2 2 3 4 5 5 5
12416 253/300 Read block 2019 len 16384
12417 254/300 Read block 3748 len 20480
12418 255/300 Write block 2332 len 16384 data: 3 4 4 3
12419 256/300 Write block 2319 len 20480 data: 3 3 3 3 3
12420 257/300 Read block 2042 len 12288
12421 258/300 Read block 3790 len 12288
12422 259/300 Write block 4362 len 36864 data: 2 2 2 2 3 2 2 2 2
12423 260/300 Write block 3984 len 20480 data: 4 5 5 4 4
12424 261/300 Read block 2265 len 32768
12425 262/300 Read block 3848 len 28672
12426 263/300 Write block 4570 len 24576 data: 4 4 6 7 6 6
12427 264/300 Read block 3687 len 24576
12428 265/300 Write block 694 len 16384 data: 4 6 7 6
12429 266/300 Write block 2696 len 36864 data: 2 3 3 4 4 4 3 4 2
12430 267/300 Flush
12431 268/300 Write block 513 len 12288 data: 3 3 3
12432 269/300 Read block 3628 len 32768
12433 270/300 Read block 4886 len 8192
12434 271/300 Read block 3736 len 40960
12435 272/300 Flush
12436 273/300 Read block 4286 len 28672
12437 274/300 Read block 4330 len 20480
12438 275/300 Read block 143 len 12288
12439 276/300 Write block 743 len 28672 data: 3 3 2 4 4 4 5
12440 277/300 Read block 105 len 4096
12441 278/300 Flush
12442 279/300 Read block 55 len 28672
12443 280/300 Write block 1772 len 36864 data: 4 3 2 2 2 3 4 3 3
12444 281/300 Read block 4267 len 40960
12445 282/300 Flush
12446 283/300 Read block 4544 len 32768
12447 284/300 Write block 1496 len 8192 data: 2 3
12448 285/300 Read block 3116 len 28672
12449 286/300 Read block 1726 len 8192
12450 287/300 Read block 3218 len 12288
12451 288/300 Write block 713 len 4096 data: 3
12452 289/300 Read block 4444 len 16384
12453 290/300 Read block 3728 len 12288
12454 291/300 Write block 2137 len 20480 data: 4 4 6 4 4
12455 292/300 Read block 3058 len 12288
12456 293/300 Write block 4098 len 32768 data: 4 5 5 4 4 4 3 3
12457 294/300 Read block 4695 len 32768
12458 295/300 Flush
12459 296/300 Read block 1299 len 32768
12460 297/300 Write block 1166 len 24576 data: 2 3 3 3 3 3
12461 298/300 Read block 2396 len 4096
12462 299/300 Write block 4497 len 8192 data: 2 2
12463 300/300 Flush
12464 301/300 Read block 1521 len 32768
12465 Main task has work to do, go find it
12466 got dsc Start(1)
12467 start 1
12468 Jan 21 15:49:48.294 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/1, method: POST, req_id: 7f03d3fe-a46f-46dd-a8cd-5f5735d209b8, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
12469 Replay: started 1, returned:Ok(())
12470 ----------------------------------------------------------------
12471 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12472 States: Active Offline Active
12473 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12474 4321 Acked Read 2 Done Sent Done false
12475 4322 Acked Read 1 Done Sent Done false
12476 4323 Acked Read 2 Done Sent Done false
12477 4324 Acked Read 7 Done Sent Done false
12478 4325 Acked Write 9 Done Sent Done false
12479 4326 Acked Write 3 Done Sent Done false
12480 4327 Acked Read 1 Done Sent Done false
12481 4328 Acked Read 7 Done Sent Done false
12482 4329 Acked Write 1 Done Sent Done false
12483 4330 Acked Write 9 Done Sent Done false
12484 4331 Acked Write 9 Done Sent Done false
12485 4332 Acked Write 6 Done Sent Done false
12486 4333 Acked Read 6 Done Sent Done false
12487 4334 Acked Write 7 Done Sent Done false
12488 4335 Acked Write 10 Done Sent Done false
12489 4336 Acked Flush 0 Done Sent Done false
12490 4337 Acked Read 8 Done Sent Done false
12491 4338 Acked Read 8 Done Sent Done false
12492 4339 Acked Write 7 Done Sent Done false
12493 4340 Acked Write 8 Done Sent Done false
12494 4341 Acked Read 10 Done Sent Done false
12495 4342 Acked Read 4 Done Sent Done false
12496 4343 Acked Flush 0 Done Sent Done false
12497 4344 Acked Write 10 Done Sent Done false
12498 4345 Acked Write 4 Done Sent Done false
12499 4346 Acked Read 9 Done Sent Done false
12500 4347 Acked Read 7 Done Sent Done false
12501 4348 Acked Write 1 Done Sent Done false
12502 4349 Acked Read 2 Done Sent Done false
12503 4350 Acked Flush 0 Done Sent Done false
12504 4351 Acked Read 10 Done Sent Done false
12505 4352 Acked Write 6 Done Sent Done false
12506 4353 Acked Write 8 Done Sent Done false
12507 4354 Acked Read 1 Done Sent Done false
12508 4355 Acked Read 5 Done Sent Done false
12509 4356 Acked Read 3 Done Sent Done false
12510 4357 Acked Read 5 Done Sent Done false
12511 4358 Acked Flush 0 Done Sent Done false
12512 4359 Acked Read 1 Done Sent Done false
12513 4360 Acked Read 3 Done Sent Done false
12514 4361 Acked Write 3 Done Sent Done false
12515 4362 Acked Read 2 Done Sent Done false
12516 4363 Acked Read 5 Done Sent Done false
12517 4364 Acked Write 8 Done Sent Done false
12518 4365 Acked Read 2 Done Sent Done false
12519 4366 Acked Write 4 Done Sent Done false
12520 4367 Acked Read 1 Done Sent Done false
12521 4368 Acked Flush 0 Done Sent Done false
12522 4369 Acked Read 6 Done Sent Done false
12523 4370 Acked Write 9 Done Sent Done false
12524 4371 Acked Read 7 Done Sent Done false
12525 4372 Acked Read 7 Done Sent Done false
12526 4373 Acked Write 10 Done Sent Done false
12527 4374 Acked Flush 0 Done Sent Done false
12528 4375 Acked Write 2 Done Sent Done false
12529 4376 Acked Write 2 Done Sent Done false
12530 4377 Acked Write 10 Done Sent Done false
12531 4378 Acked Write 5 Done Sent Done false
12532 4379 Acked Read 8 Done Sent Done false
12533 4380 Acked Write 8 Done Sent Done false
12534 4381 Acked Read 7 Done Sent Done false
12535 4382 Acked Read 4 Done Sent Done false
12536 4383 Acked Read 4 Done Sent Done false
12537 4384 Acked Read 8 Done Sent Done false
12538 4385 Acked Read 9 Done Sent Done false
12539 4386 Acked Write 6 Done Sent Done false
12540 4387 Acked Write 8 Done Sent Done false
12541 4388 Acked Read 1 Done Sent Done false
12542 4389 Acked Write 10 Done Sent Done false
12543 4390 Acked Read 10 Done Sent Done false
12544 4391 Acked Read 1 Done Sent Done false
12545 4392 Acked Read 7 Done Sent Done false
12546 4393 Acked Flush 0 Done Sent Done false
12547 4394 Acked Write 5 Done Sent Done false
12548 4395 Acked Read 1 Done Sent Done false
12549 4396 Acked Read 1 Done Sent Done false
12550 4397 Acked Read 9 Done Sent Done false
12551 4398 Acked Flush 0 Done Sent Done false
12552 4399 Acked Write 3 Done Sent Done false
12553 4400 Acked Write 2 Done Sent Done false
12554 4401 Acked Flush 0 Done Sent Done false
12555 4402 Acked Write 10 Done Sent Done false
12556 4403 Acked Read 6 Done Sent Done false
12557 4404 Acked Write 4 Done Sent Done false
12558 4405 Acked Read 4 Done Sent Done false
12559 4406 Acked Write 1 Done Sent Done false
12560 4407 Acked Read 4 Done Sent Done false
12561 4408 Acked Write 1 Done Sent Done false
12562 4409 Acked Read 1 Done Sent Done false
12563 4410 Acked Write 3 Done Sent Done false
12564 4411 Acked Read 1 Done Sent Done false
12565 4412 Acked Write 9 Done Sent Done false
12566 4413 Acked Flush 0 Done Sent Done false
12567 4414 Acked Read 7 Done Sent Done false
12568 4415 Acked Write 2 Done Sent Done false
12569 4416 Acked Read 3 Done Sent Done false
12570 4417 Acked Write 8 Done Sent Done false
12571 4418 Acked Read 4 Done Sent Done false
12572 4419 Acked Flush 0 Done Sent Done false
12573 4420 Acked Write 1 Done Sent Done false
12574 4421 Acked Read 1 Done Sent Done false
12575 4422 Acked Write 5 Done Sent Done false
12576 4423 Acked Write 8 Done Sent Done false
12577 4424 Acked Read 4 Done Sent Done false
12578 4425 Acked Write 2 Done Sent Done false
12579 4426 Acked Read 5 Done Sent Done false
12580 4427 Acked Write 8 Done Sent Done false
12581 4428 Acked Write 1 Done Sent Done false
12582 4429 Acked Write 10 Done Sent Done false
12583 4430 Acked Read 1 Done Sent Done false
12584 4431 Acked Read 6 Done Sent Done false
12585 4432 Acked Flush 0 Done Sent Done false
12586 4433 Acked Read 7 Done Sent Done false
12587 4434 Acked Write 7 Done Sent Done false
12588 4435 Acked Read 10 Done Sent Done false
12589 4436 Acked Write 4 Done Sent Done false
12590 4437 Acked Read 2 Done Sent Done false
12591 4438 Acked Read 8 Done Sent Done false
12592 4439 Acked Read 8 Done Sent Done false
12593 4440 Acked Read 3 Done Sent Done false
12594 4441 Acked Write 8 Done Sent Done false
12595 4442 Acked Write 9 Done Sent Done false
12596 4443 Acked Write 9 Done Sent Done false
12597 4444 Acked Write 10 Done Sent Done false
12598 4445 Acked Write 1 Done Sent Done false
12599 4446 Acked Read 9 Done Sent Done false
12600 4447 Acked Write 1 Done Sent Done false
12601 4448 Acked Write 9 Done Sent Done false
12602 4449 Acked Write 1 Done Sent Done false
12603 4450 Acked Read 10 Done Sent Done false
12604 4451 Acked Read 6 Done Sent Done false
12605 4452 Acked Write 8 Done Sent Done false
12606 4453 Acked Read 10 Done Sent Done false
12607 4454 Acked Flush 0 Done Sent Done false
12608 4455 Acked Read 5 Done Sent Done false
12609 4456 Acked Read 7 Done Sent Done false
12610 4457 Acked Read 2 Done Sent Done false
12611 4458 Acked Write 5 Done Sent Done false
12612 4459 Acked Write 7 Done Sent Done false
12613 4460 Acked Read 2 Done Sent Done false
12614 4461 Acked Write 4 Done Sent Done false
12615 4462 Acked Read 5 Done Sent Done false
12616 4463 Acked Write 3 Done Sent Done false
12617 4464 Acked Read 7 Done Sent Done false
12618 4465 Acked Read 10 Done Sent Done false
12619 4466 Acked Read 2 Done Sent Done false
12620 4467 Acked Read 3 Done Sent Done false
12621 4468 Acked Flush 0 Done Sent Done false
12622 4469 Acked Read 7 Done Sent Done false
12623 4470 Acked Write 10 Done Sent Done false
12624 4471 Acked Flush 0 Done Sent Done false
12625 4472 Acked Read 6 Done Sent Done false
12626 4473 Acked Write 3 Done Sent Done false
12627 4474 Acked Read 8 Done Sent Done false
12628 4475 Acked Read 9 Done Sent Done false
12629 4476 Acked Write 2 Done Sent Done false
12630 4477 Acked Read 10 Done Sent Done false
12631 4478 Acked Read 3 Done Sent Done false
12632 4479 Acked Read 2 Done Sent Done false
12633 4480 Acked Write 4 Done Sent Done false
12634 4481 Acked Write 1 Done Sent Done false
12635 4482 Acked Write 6 Done Sent Done false
12636 4483 Acked Flush 0 Done Sent Done false
12637 4484 Acked Read 5 Done Sent Done false
12638 4485 Acked Flush 0 Done Sent Done false
12639 4486 Acked Read 7 Done Sent Done false
12640 4487 Acked Write 6 Done Sent Done false
12641 4488 Acked Flush 0 Done Sent Done false
12642 4489 Acked Write 6 Done Sent Done false
12643 4490 Acked Read 1 Done Sent Done false
12644 4491 Acked Flush 0 Done Sent Done false
12645 4492 Acked Write 5 Done Sent Done false
12646 4493 Acked Write 9 Done Sent Done false
12647 4494 Acked Write 4 Done Sent Done false
12648 4495 Acked Write 5 Done Sent Done false
12649 4496 Acked Write 9 Done Sent Done false
12650 4497 Acked Write 3 Done Sent Done false
12651 4498 Acked Flush 0 Done Sent Done false
12652 4499 Acked Write 5 Done Sent Done false
12653 4500 Acked Read 2 Done Sent Done false
12654 4501 Acked Read 3 Done Sent Done false
12655 4502 Acked Write 9 Done Sent Done false
12656 4503 Acked Write 8 Done Sent Done false
12657 4504 Acked Read 10 Done Sent Done false
12658 4505 Acked Write 9 Done Sent Done false
12659 4506 Acked Write 6 Done Sent Done false
12660 4507 Acked Read 7 Done Sent Done false
12661 4508 Acked Read 4 Done Sent Done false
12662 4509 Acked Write 2 Done Sent Done false
12663 4510 Acked Read 1 Done Sent Done false
12664 4511 Acked Write 9 Done Sent Done false
12665 4512 Acked Read 10 Done Sent Done false
12666 4513 Acked Write 7 Done Sent Done false
12667 4514 Acked Write 5 Done Sent Done false
12668 4515 Acked Read 4 Done Sent Done false
12669 4516 Acked Write 4 Done Sent Done false
12670 4517 Acked Write 10 Done Sent Done false
12671 4518 Acked Read 10 Done Sent Done false
12672 4519 Acked Write 10 Done Sent Done false
12673 4520 Acked Read 7 Done Sent Done false
12674 4521 Acked Read 1 Done Sent Done false
12675 4522 Acked Write 8 Done Sent Done false
12676 4523 Acked Write 7 Done Sent Done false
12677 4524 Acked Read 2 Done Sent Done false
12678 4525 Acked Read 3 Done Sent Done false
12679 4526 Acked Write 6 Done Sent Done false
12680 4527 Acked Read 8 Done Sent Done false
12681 4528 Acked Read 2 Done Sent Done false
12682 4529 Acked Write 3 Done Sent Done false
12683 4530 Acked Read 4 Done Sent Done false
12684 4531 Acked Write 8 Done Sent Done false
12685 4532 Acked Write 3 Done Sent Done false
12686 4533 Acked Flush 0 Done Sent Done false
12687 4534 Acked Flush 0 Done Sent Done false
12688 4535 Acked Read 4 Done Sent Done false
12689 4536 Acked Read 8 Done Sent Done false
12690 4537 Acked Read 8 Done Sent Done false
12691 4538 Acked Flush 0 Done Sent Done false
12692 4539 Acked Write 7 Done Sent Done false
12693 4540 Acked Read 1 Done Sent Done false
12694 4541 Acked Write 8 Done Sent Done false
12695 4542 Acked Read 1 Done Sent Done false
12696 4543 Acked Read 6 Done Sent Done false
12697 4544 Acked Read 10 Done Sent Done false
12698 4545 Acked Write 9 Done Sent Done false
12699 4546 Acked Write 3 Done Sent Done false
12700 4547 Acked Write 1 Done Sent Done false
12701 4548 Acked Read 1 Done Sent Done false
12702 4549 Acked Write 4 Done Sent Done false
12703 4550 Acked Read 1 Done Sent Done false
12704 4551 Acked Write 5 Done Sent Done false
12705 4552 Acked Read 1 Done Sent Done false
12706 4553 Acked Read 8 Done Sent Done false
12707 4554 Acked Read 1 Done Sent Done false
12708 4555 Acked Read 6 Done Sent Done false
12709 4556 Acked Write 8 Done Sent Done false
12710 4557 Acked Read 6 Done Sent Done false
12711 4558 Acked Read 7 Done Sent Done false
12712 4559 Acked Read 6 Done Sent Done false
12713 4560 Acked Write 5 Done Sent Done false
12714 4561 Acked Write 8 Done Sent Done false
12715 4562 Acked Write 2 Done Sent Done false
12716 4563 Acked Read 10 Done Sent Done false
12717 4564 Acked Read 5 Done Sent Done false
12718 4565 Acked Write 3 Done Sent Done false
12719 4566 Acked Read 2 Done Sent Done false
12720 4567 Acked Read 1 Done Sent Done false
12721 4568 Acked Write 4 Done Sent Done false
12722 4569 Acked Read 9 Done Sent Done false
12723 4570 Acked Write 9 Done Sent Done false
12724 4571 Acked Write 7 Done Sent Done false
12725 4572 Acked Write 10 Done Sent Done false
12726 4573 Acked Read 4 Done Sent Done false
12727 4574 Acked Read 5 Done Sent Done false
12728 4575 Acked Write 4 Done Sent Done false
12729 4576 Acked Write 5 Done Sent Done false
12730 4577 Acked Read 3 Done Sent Done false
12731 4578 Acked Read 3 Done Sent Done false
12732 4579 Acked Write 9 Done Sent Done false
12733 4580 Acked Write 5 Done Sent Done false
12734 4581 Acked Read 8 Done Sent Done false
12735 4582 Acked Read 7 Done Sent Done false
12736 4583 Acked Write 6 Done Sent Done false
12737 4584 Acked Read 6 Done Sent Done false
12738 4585 Acked Write 4 Done Sent Done false
12739 4586 Acked Write 9 Done Sent Done false
12740 4587 Acked Flush 0 Done Sent Done false
12741 4588 Acked Write 3 Done Sent Done false
12742 4589 Acked Read 8 Done Sent Done false
12743 4590 Acked Read 2 Done Sent Done false
12744 4591 Acked Read 10 Done Sent Done false
12745 4592 Acked Flush 0 Done Sent Done false
12746 4593 Acked Read 7 Done Sent Done false
12747 4594 Acked Read 5 Done Sent Done false
12748 4595 Acked Read 3 Done Sent Done false
12749 4596 Acked Write 7 Done Sent Done false
12750 4597 Acked Read 1 Done Sent Done false
12751 4598 Acked Flush 0 Done Sent Done false
12752 4599 Acked Read 7 Done Sent Done false
12753 4600 Acked Write 9 Done Sent Done false
12754 4601 Acked Read 10 Done Sent Done false
12755 4602 Acked Flush 0 Done Sent Done false
12756 4603 Acked Read 8 Done Sent Done false
12757 4604 Acked Write 2 Done Sent Done false
12758 4605 Acked Read 7 Done Sent Done false
12759 4606 Acked Read 2 Done Sent Done false
12760 4607 Acked Read 3 Done Sent Done false
12761 4608 Acked Write 1 Done Sent Done false
12762 4609 Acked Read 4 Done Sent Done false
12763 4610 Acked Read 3 Done Sent Done false
12764 4611 Acked Write 5 Done Sent Done false
12765 4612 Acked Read 3 Done Sent Done false
12766 4613 Acked Write 8 Done Sent Done false
12767 4614 Acked Read 8 Done Sent Done false
12768 4615 Acked Flush 0 Done Sent Done false
12769 4616 Acked Read 8 Done Sent Done false
12770 4617 Acked Write 6 Done Sent Done false
12771 4618 Acked Read 1 Done Sent Done false
12772 4619 Acked Write 2 Done Sent Done false
12773 4620 Acked Flush 0 Done Sent Done false
12774 4621 Acked Read 8 Done Sent Done false
12775 STATES DS:0 DS:1 DS:2 TOTAL
12776 Sent 0 301 0 301
12777 Done 301 0 301 602
12778 Skipped 0 0 0 0
12779 Error 0 0 0 0
12780 Last Flush: 4620 4320 4620
12781 Downstairs last five completed: 4320 4319 4318 4317 4316
12782 Upstairs last five completed: 4621 4620 4619 4618 4617
12783 CLIENT: Up:0 ds:301 act:2
12784 ----------------------------------------------------------------
12785 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12786 States: Active Offline Active
12787 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12788 4321 Acked Read 2 Done Sent Done false
12789 4322 Acked Read 1 Done Sent Done false
12790 4323 Acked Read 2 Done Sent Done false
12791 4324 Acked Read 7 Done Sent Done false
12792 4325 Acked Write 9 Done Sent Done false
12793 4326 Acked Write 3 Done Sent Done false
12794 4327 Acked Read 1 Done Sent Done false
12795 4328 Acked Read 7 Done Sent Done false
12796 4329 Acked Write 1 Done Sent Done false
12797 4330 Acked Write 9 Done Sent Done false
12798 4331 Acked Write 9 Done Sent Done false
12799 4332 Acked Write 6 Done Sent Done false
12800 4333 Acked Read 6 Done Sent Done false
12801 4334 Acked Write 7 Done Sent Done false
12802 4335 Acked Write 10 Done Sent Done false
12803 4336 Acked Flush 0 Done Sent Done false
12804 4337 Acked Read 8 Done Sent Done false
12805 4338 Acked Read 8 Done Sent Done false
12806 4339 Acked Write 7 Done Sent Done false
12807 4340 Acked Write 8 Done Sent Done false
12808 4341 Acked Read 10 Done Sent Done false
12809 4342 Acked Read 4 Done Sent Done false
12810 4343 Acked Flush 0 Done Sent Done false
12811 4344 Acked Write 10 Done Sent Done false
12812 4345 Acked Write 4 Done Sent Done false
12813 4346 Acked Read 9 Done Sent Done false
12814 4347 Acked Read 7 Done Sent Done false
12815 4348 Acked Write 1 Done Sent Done false
12816 4349 Acked Read 2 Done Sent Done false
12817 4350 Acked Flush 0 Done Sent Done false
12818 4351 Acked Read 10 Done Sent Done false
12819 4352 Acked Write 6 Done Sent Done false
12820 4353 Acked Write 8 Done Sent Done false
12821 4354 Acked Read 1 Done Sent Done false
12822 4355 Acked Read 5 Done Sent Done false
12823 4356 Acked Read 3 Done Sent Done false
12824 4357 Acked Read 5 Done Sent Done false
12825 4358 Acked Flush 0 Done Sent Done false
12826 4359 Acked Read 1 Done Sent Done false
12827 4360 Acked Read 3 Done Sent Done false
12828 4361 Acked Write 3 Done Sent Done false
12829 4362 Acked Read 2 Done Sent Done false
12830 4363 Acked Read 5 Done Sent Done false
12831 4364 Acked Write 8 Done Sent Done false
12832 4365 Acked Read 2 Done Sent Done false
12833 4366 Acked Write 4 Done Sent Done false
12834 4367 Acked Read 1 Done Sent Done false
12835 4368 Acked Flush 0 Done Sent Done false
12836 4369 Acked Read 6 Done Sent Done false
12837 4370 Acked Write 9 Done Sent Done false
12838 4371 Acked Read 7 Done Sent Done false
12839 4372 Acked Read 7 Done Sent Done false
12840 4373 Acked Write 10 Done Sent Done false
12841 4374 Acked Flush 0 Done Sent Done false
12842 4375 Acked Write 2 Done Sent Done false
12843 4376 Acked Write 2 Done Sent Done false
12844 4377 Acked Write 10 Done Sent Done false
12845 4378 Acked Write 5 Done Sent Done false
12846 4379 Acked Read 8 Done Sent Done false
12847 4380 Acked Write 8 Done Sent Done false
12848 4381 Acked Read 7 Done Sent Done false
12849 4382 Acked Read 4 Done Sent Done false
12850 4383 Acked Read 4 Done Sent Done false
12851 4384 Acked Read 8 Done Sent Done false
12852 4385 Acked Read 9 Done Sent Done false
12853 4386 Acked Write 6 Done Sent Done false
12854 4387 Acked Write 8 Done Sent Done false
12855 4388 Acked Read 1 Done Sent Done false
12856 4389 Acked Write 10 Done Sent Done false
12857 4390 Acked Read 10 Done Sent Done false
12858 4391 Acked Read 1 Done Sent Done false
12859 4392 Acked Read 7 Done Sent Done false
12860 4393 Acked Flush 0 Done Sent Done false
12861 4394 Acked Write 5 Done Sent Done false
12862 4395 Acked Read 1 Done Sent Done false
12863 4396 Acked Read 1 Done Sent Done false
12864 4397 Acked Read 9 Done Sent Done false
12865 4398 Acked Flush 0 Done Sent Done false
12866 4399 Acked Write 3 Done Sent Done false
12867 4400 Acked Write 2 Done Sent Done false
12868 4401 Acked Flush 0 Done Sent Done false
12869 4402 Acked Write 10 Done Sent Done false
12870 4403 Acked Read 6 Done Sent Done false
12871 4404 Acked Write 4 Done Sent Done false
12872 4405 Acked Read 4 Done Sent Done false
12873 4406 Acked Write 1 Done Sent Done false
12874 4407 Acked Read 4 Done Sent Done false
12875 4408 Acked Write 1 Done Sent Done false
12876 4409 Acked Read 1 Done Sent Done false
12877 4410 Acked Write 3 Done Sent Done false
12878 4411 Acked Read 1 Done Sent Done false
12879 4412 Acked Write 9 Done Sent Done false
12880 4413 Acked Flush 0 Done Sent Done false
12881 4414 Acked Read 7 Done Sent Done false
12882 4415 Acked Write 2 Done Sent Done false
12883 4416 Acked Read 3 Done Sent Done false
12884 4417 Acked Write 8 Done Sent Done false
12885 4418 Acked Read 4 Done Sent Done false
12886 4419 Acked Flush 0 Done Sent Done false
12887 4420 Acked Write 1 Done Sent Done false
12888 4421 Acked Read 1 Done Sent Done false
12889 4422 Acked Write 5 Done Sent Done false
12890 4423 Acked Write 8 Done Sent Done false
12891 4424 Acked Read 4 Done Sent Done false
12892 4425 Acked Write 2 Done Sent Done false
12893 4426 Acked Read 5 Done Sent Done false
12894 4427 Acked Write 8 Done Sent Done false
12895 4428 Acked Write 1 Done Sent Done false
12896 4429 Acked Write 10 Done Sent Done false
12897 4430 Acked Read 1 Done Sent Done false
12898 4431 Acked Read 6 Done Sent Done false
12899 4432 Acked Flush 0 Done Sent Done false
12900 4433 Acked Read 7 Done Sent Done false
12901 4434 Acked Write 7 Done Sent Done false
12902 4435 Acked Read 10 Done Sent Done false
12903 4436 Acked Write 4 Done Sent Done false
12904 4437 Acked Read 2 Done Sent Done false
12905 4438 Acked Read 8 Done Sent Done false
12906 4439 Acked Read 8 Done Sent Done false
12907 4440 Acked Read 3 Done Sent Done false
12908 4441 Acked Write 8 Done Sent Done false
12909 4442 Acked Write 9 Done Sent Done false
12910 4443 Acked Write 9 Done Sent Done false
12911 4444 Acked Write 10 Done Sent Done false
12912 4445 Acked Write 1 Done Sent Done false
12913 4446 Acked Read 9 Done Sent Done false
12914 4447 Acked Write 1 Done Sent Done false
12915 4448 Acked Write 9 Done Sent Done false
12916 4449 Acked Write 1 Done Sent Done false
12917 4450 Acked Read 10 Done Sent Done false
12918 4451 Acked Read 6 Done Sent Done false
12919 4452 Acked Write 8 Done Sent Done false
12920 4453 Acked Read 10 Done Sent Done false
12921 4454 Acked Flush 0 Done Sent Done false
12922 4455 Acked Read 5 Done Sent Done false
12923 4456 Acked Read 7 Done Sent Done false
12924 4457 Acked Read 2 Done Sent Done false
12925 4458 Acked Write 5 Done Sent Done false
12926 4459 Acked Write 7 Done Sent Done false
12927 4460 Acked Read 2 Done Sent Done false
12928 4461 Acked Write 4 Done Sent Done false
12929 4462 Acked Read 5 Done Sent Done false
12930 4463 Acked Write 3 Done Sent Done false
12931 4464 Acked Read 7 Done Sent Done false
12932 4465 Acked Read 10 Done Sent Done false
12933 4466 Acked Read 2 Done Sent Done false
12934 4467 Acked Read 3 Done Sent Done false
12935 4468 Acked Flush 0 Done Sent Done false
12936 4469 Acked Read 7 Done Sent Done false
12937 4470 Acked Write 10 Done Sent Done false
12938 4471 Acked Flush 0 Done Sent Done false
12939 4472 Acked Read 6 Done Sent Done false
12940 4473 Acked Write 3 Done Sent Done false
12941 4474 Acked Read 8 Done Sent Done false
12942 4475 Acked Read 9 Done Sent Done false
12943 4476 Acked Write 2 Done Sent Done false
12944 4477 Acked Read 10 Done Sent Done false
12945 4478 Acked Read 3 Done Sent Done false
12946 4479 Acked Read 2 Done Sent Done false
12947 4480 Acked Write 4 Done Sent Done false
12948 4481 Acked Write 1 Done Sent Done false
12949 4482 Acked Write 6 Done Sent Done false
12950 4483 Acked Flush 0 Done Sent Done false
12951 4484 Acked Read 5 Done Sent Done false
12952 4485 Acked Flush 0 Done Sent Done false
12953 4486 Acked Read 7 Done Sent Done false
12954 4487 Acked Write 6 Done Sent Done false
12955 4488 Acked Flush 0 Done Sent Done false
12956 4489 Acked Write 6 Done Sent Done false
12957 4490 Acked Read 1 Done Sent Done false
12958 4491 Acked Flush 0 Done Sent Done false
12959 4492 Acked Write 5 Done Sent Done false
12960 4493 Acked Write 9 Done Sent Done false
12961 4494 Acked Write 4 Done Sent Done false
12962 4495 Acked Write 5 Done Sent Done false
12963 4496 Acked Write 9 Done Sent Done false
12964 4497 Acked Write 3 Done Sent Done false
12965 4498 Acked Flush 0 Done Sent Done false
12966 4499 Acked Write 5 Done Sent Done false
12967 4500 Acked Read 2 Done Sent Done false
12968 4501 Acked Read 3 Done Sent Done false
12969 4502 Acked Write 9 Done Sent Done false
12970 4503 Acked Write 8 Done Sent Done false
12971 4504 Acked Read 10 Done Sent Done false
12972 4505 Acked Write 9 Done Sent Done false
12973 4506 Acked Write 6 Done Sent Done false
12974 4507 Acked Read 7 Done Sent Done false
12975 4508 Acked Read 4 Done Sent Done false
12976 4509 Acked Write 2 Done Sent Done false
12977 4510 Acked Read 1 Done Sent Done false
12978 4511 Acked Write 9 Done Sent Done false
12979 4512 Acked Read 10 Done Sent Done false
12980 4513 Acked Write 7 Done Sent Done false
12981 4514 Acked Write 5 Done Sent Done false
12982 4515 Acked Read 4 Done Sent Done false
12983 4516 Acked Write 4 Done Sent Done false
12984 4517 Acked Write 10 Done Sent Done false
12985 4518 Acked Read 10 Done Sent Done false
12986 4519 Acked Write 10 Done Sent Done false
12987 4520 Acked Read 7 Done Sent Done false
12988 4521 Acked Read 1 Done Sent Done false
12989 4522 Acked Write 8 Done Sent Done false
12990 4523 Acked Write 7 Done Sent Done false
12991 4524 Acked Read 2 Done Sent Done false
12992 4525 Acked Read 3 Done Sent Done false
12993 4526 Acked Write 6 Done Sent Done false
12994 4527 Acked Read 8 Done Sent Done false
12995 4528 Acked Read 2 Done Sent Done false
12996 4529 Acked Write 3 Done Sent Done false
12997 4530 Acked Read 4 Done Sent Done false
12998 4531 Acked Write 8 Done Sent Done false
12999 4532 Acked Write 3 Done Sent Done false
13000 4533 Acked Flush 0 Done Sent Done false
13001 4534 Acked Flush 0 Done Sent Done false
13002 4535 Acked Read 4 Done Sent Done false
13003 4536 Acked Read 8 Done Sent Done false
13004 4537 Acked Read 8 Done Sent Done false
13005 4538 Acked Flush 0 Done Sent Done false
13006 4539 Acked Write 7 Done Sent Done false
13007 4540 Acked Read 1 Done Sent Done false
13008 4541 Acked Write 8 Done Sent Done false
13009 4542 Acked Read 1 Done Sent Done false
13010 4543 Acked Read 6 Done Sent Done false
13011 4544 Acked Read 10 Done Sent Done false
13012 4545 Acked Write 9 Done Sent Done false
13013 4546 Acked Write 3 Done Sent Done false
13014 4547 Acked Write 1 Done Sent Done false
13015 4548 Acked Read 1 Done Sent Done false
13016 4549 Acked Write 4 Done Sent Done false
13017 4550 Acked Read 1 Done Sent Done false
13018 4551 Acked Write 5 Done Sent Done false
13019 4552 Acked Read 1 Done Sent Done false
13020 4553 Acked Read 8 Done Sent Done false
13021 4554 Acked Read 1 Done Sent Done false
13022 4555 Acked Read 6 Done Sent Done false
13023 4556 Acked Write 8 Done Sent Done false
13024 4557 Acked Read 6 Done Sent Done false
13025 4558 Acked Read 7 Done Sent Done false
13026 4559 Acked Read 6 Done Sent Done false
13027 4560 Acked Write 5 Done Sent Done false
13028 4561 Acked Write 8 Done Sent Done false
13029 4562 Acked Write 2 Done Sent Done false
13030 4563 Acked Read 10 Done Sent Done false
13031 4564 Acked Read 5 Done Sent Done false
13032 4565 Acked Write 3 Done Sent Done false
13033 4566 Acked Read 2 Done Sent Done false
13034 4567 Acked Read 1 Done Sent Done false
13035 4568 Acked Write 4 Done Sent Done false
13036 4569 Acked Read 9 Done Sent Done false
13037 4570 Acked Write 9 Done Sent Done false
13038 4571 Acked Write 7 Done Sent Done false
13039 4572 Acked Write 10 Done Sent Done false
13040 4573 Acked Read 4 Done Sent Done false
13041 4574 Acked Read 5 Done Sent Done false
13042 4575 Acked Write 4 Done Sent Done false
13043 4576 Acked Write 5 Done Sent Done false
13044 4577 Acked Read 3 Done Sent Done false
13045 4578 Acked Read 3 Done Sent Done false
13046 4579 Acked Write 9 Done Sent Done false
13047 4580 Acked Write 5 Done Sent Done false
13048 4581 Acked Read 8 Done Sent Done false
13049 4582 Acked Read 7 Done Sent Done false
13050 4583 Acked Write 6 Done Sent Done false
13051 4584 Acked Read 6 Done Sent Done false
13052 4585 Acked Write 4 Done Sent Done false
13053 4586 Acked Write 9 Done Sent Done false
13054 4587 Acked Flush 0 Done Sent Done false
13055 4588 Acked Write 3 Done Sent Done false
13056 4589 Acked Read 8 Done Sent Done false
13057 4590 Acked Read 2 Done Sent Done false
13058 4591 Acked Read 10 Done Sent Done false
13059 4592 Acked Flush 0 Done Sent Done false
13060 4593 Acked Read 7 Done Sent Done false
13061 4594 Acked Read 5 Done Sent Done false
13062 4595 Acked Read 3 Done Sent Done false
13063 4596 Acked Write 7 Done Sent Done false
13064 4597 Acked Read 1 Done Sent Done false
13065 4598 Acked Flush 0 Done Sent Done false
13066 4599 Acked Read 7 Done Sent Done false
13067 4600 Acked Write 9 Done Sent Done false
13068 4601 Acked Read 10 Done Sent Done false
13069 4602 Acked Flush 0 Done Sent Done false
13070 4603 Acked Read 8 Done Sent Done false
13071 4604 Acked Write 2 Done Sent Done false
13072 4605 Acked Read 7 Done Sent Done false
13073 4606 Acked Read 2 Done Sent Done false
13074 4607 Acked Read 3 Done Sent Done false
13075 4608 Acked Write 1 Done Sent Done false
13076 4609 Acked Read 4 Done Sent Done false
13077 4610 Acked Read 3 Done Sent Done false
13078 4611 Acked Write 5 Done Sent Done false
13079 4612 Acked Read 3 Done Sent Done false
13080 4613 Acked Write 8 Done Sent Done false
13081 4614 Acked Read 8 Done Sent Done false
13082 4615 Acked Flush 0 Done Sent Done false
13083 4616 Acked Read 8 Done Sent Done false
13084 4617 Acked Write 6 Done Sent Done false
13085 4618 Acked Read 1 Done Sent Done false
13086 4619 Acked Write 2 Done Sent Done false
13087 4620 Acked Flush 0 Done Sent Done false
13088 4621 Acked Read 8 Done Sent Done false
13089 4622 Acked Flush 0 Done Sent Done false
13090 STATES DS:0 DS:1 DS:2 TOTAL
13091 Sent 0 302 0 302
13092 Done 302 0 302 604
13093 Skipped 0 0 0 0
13094 Error 0 0 0 0
13095 Last Flush: 4622 4320 4622
13096 Downstairs last five completed: 4320 4319 4318 4317 4316
13097 Upstairs last five completed: 4622 4621 4620 4619 4618
13098 CLIENT: Up:0 ds:302 act:2
130992025-01-21T15:49:54.054ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
131002025-01-21T15:49:54.054ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
131012025-01-21T15:49:54.054ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
131022025-01-21T15:49:54.054ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
131032025-01-21T15:49:54.054ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
13104 [8820] got start action
13105 [8820] I am going to restart
13106 Starting downstairs at port 8820
13107 Make output file at "/tmp/dsc/downstairs-8820.txt"
13108 [8820][1] reports Starting
13109 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
13110 [8820][1] reports Running
13111 ----------------------------------------------------------------
13112 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13113 States: Active Offline Active
13114 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13115 4321 Acked Read 2 Done Sent Done false
13116 4322 Acked Read 1 Done Sent Done false
13117 4323 Acked Read 2 Done Sent Done false
13118 4324 Acked Read 7 Done Sent Done false
13119 4325 Acked Write 9 Done Sent Done false
13120 4326 Acked Write 3 Done Sent Done false
13121 4327 Acked Read 1 Done Sent Done false
13122 4328 Acked Read 7 Done Sent Done false
13123 4329 Acked Write 1 Done Sent Done false
13124 4330 Acked Write 9 Done Sent Done false
13125 4331 Acked Write 9 Done Sent Done false
13126 4332 Acked Write 6 Done Sent Done false
13127 4333 Acked Read 6 Done Sent Done false
13128 4334 Acked Write 7 Done Sent Done false
13129 4335 Acked Write 10 Done Sent Done false
13130 4336 Acked Flush 0 Done Sent Done false
13131 4337 Acked Read 8 Done Sent Done false
13132 4338 Acked Read 8 Done Sent Done false
13133 4339 Acked Write 7 Done Sent Done false
13134 4340 Acked Write 8 Done Sent Done false
13135 4341 Acked Read 10 Done Sent Done false
13136 4342 Acked Read 4 Done Sent Done false
13137 4343 Acked Flush 0 Done Sent Done false
13138 4344 Acked Write 10 Done Sent Done false
13139 4345 Acked Write 4 Done Sent Done false
13140 4346 Acked Read 9 Done Sent Done false
13141 4347 Acked Read 7 Done Sent Done false
13142 4348 Acked Write 1 Done Sent Done false
13143 4349 Acked Read 2 Done Sent Done false
13144 4350 Acked Flush 0 Done Sent Done false
13145 4351 Acked Read 10 Done Sent Done false
13146 4352 Acked Write 6 Done Sent Done false
13147 4353 Acked Write 8 Done Sent Done false
13148 4354 Acked Read 1 Done Sent Done false
13149 4355 Acked Read 5 Done Sent Done false
13150 4356 Acked Read 3 Done Sent Done false
13151 4357 Acked Read 5 Done Sent Done false
13152 4358 Acked Flush 0 Done Sent Done false
13153 4359 Acked Read 1 Done Sent Done false
13154 4360 Acked Read 3 Done Sent Done false
13155 4361 Acked Write 3 Done Sent Done false
13156 4362 Acked Read 2 Done Sent Done false
13157 4363 Acked Read 5 Done Sent Done false
13158 4364 Acked Write 8 Done Sent Done false
13159 4365 Acked Read 2 Done Sent Done false
13160 4366 Acked Write 4 Done Sent Done false
13161 4367 Acked Read 1 Done Sent Done false
13162 4368 Acked Flush 0 Done Sent Done false
13163 4369 Acked Read 6 Done Sent Done false
13164 4370 Acked Write 9 Done Sent Done false
13165 4371 Acked Read 7 Done Sent Done false
13166 4372 Acked Read 7 Done Sent Done false
13167 4373 Acked Write 10 Done Sent Done false
13168 4374 Acked Flush 0 Done Sent Done false
13169 4375 Acked Write 2 Done Sent Done false
13170 4376 Acked Write 2 Done Sent Done false
13171 4377 Acked Write 10 Done Sent Done false
13172 4378 Acked Write 5 Done Sent Done false
13173 4379 Acked Read 8 Done Sent Done false
13174 4380 Acked Write 8 Done Sent Done false
13175 4381 Acked Read 7 Done Sent Done false
13176 4382 Acked Read 4 Done Sent Done false
13177 4383 Acked Read 4 Done Sent Done false
13178 4384 Acked Read 8 Done Sent Done false
13179 4385 Acked Read 9 Done Sent Done false
13180 4386 Acked Write 6 Done Sent Done false
13181 4387 Acked Write 8 Done Sent Done false
13182 4388 Acked Read 1 Done Sent Done false
13183 4389 Acked Write 10 Done Sent Done false
13184 4390 Acked Read 10 Done Sent Done false
13185 4391 Acked Read 1 Done Sent Done false
13186 4392 Acked Read 7 Done Sent Done false
13187 4393 Acked Flush 0 Done Sent Done false
13188 4394 Acked Write 5 Done Sent Done false
13189 4395 Acked Read 1 Done Sent Done false
13190 4396 Acked Read 1 Done Sent Done false
13191 4397 Acked Read 9 Done Sent Done false
13192 4398 Acked Flush 0 Done Sent Done false
13193 4399 Acked Write 3 Done Sent Done false
13194 4400 Acked Write 2 Done Sent Done false
13195 4401 Acked Flush 0 Done Sent Done false
13196 4402 Acked Write 10 Done Sent Done false
13197 4403 Acked Read 6 Done Sent Done false
13198 4404 Acked Write 4 Done Sent Done false
13199 4405 Acked Read 4 Done Sent Done false
13200 4406 Acked Write 1 Done Sent Done false
13201 4407 Acked Read 4 Done Sent Done false
13202 4408 Acked Write 1 Done Sent Done false
13203 4409 Acked Read 1 Done Sent Done false
13204 4410 Acked Write 3 Done Sent Done false
13205 4411 Acked Read 1 Done Sent Done false
13206 4412 Acked Write 9 Done Sent Done false
13207 4413 Acked Flush 0 Done Sent Done false
13208 4414 Acked Read 7 Done Sent Done false
13209 4415 Acked Write 2 Done Sent Done false
13210 4416 Acked Read 3 Done Sent Done false
13211 4417 Acked Write 8 Done Sent Done false
13212 4418 Acked Read 4 Done Sent Done false
13213 4419 Acked Flush 0 Done Sent Done false
13214 4420 Acked Write 1 Done Sent Done false
13215 4421 Acked Read 1 Done Sent Done false
13216 4422 Acked Write 5 Done Sent Done false
13217 4423 Acked Write 8 Done Sent Done false
13218 4424 Acked Read 4 Done Sent Done false
13219 4425 Acked Write 2 Done Sent Done false
13220 4426 Acked Read 5 Done Sent Done false
13221 4427 Acked Write 8 Done Sent Done false
13222 4428 Acked Write 1 Done Sent Done false
13223 4429 Acked Write 10 Done Sent Done false
13224 4430 Acked Read 1 Done Sent Done false
13225 4431 Acked Read 6 Done Sent Done false
13226 4432 Acked Flush 0 Done Sent Done false
13227 4433 Acked Read 7 Done Sent Done false
13228 4434 Acked Write 7 Done Sent Done false
13229 4435 Acked Read 10 Done Sent Done false
13230 4436 Acked Write 4 Done Sent Done false
13231 4437 Acked Read 2 Done Sent Done false
13232 4438 Acked Read 8 Done Sent Done false
13233 4439 Acked Read 8 Done Sent Done false
13234 4440 Acked Read 3 Done Sent Done false
13235 4441 Acked Write 8 Done Sent Done false
13236 4442 Acked Write 9 Done Sent Done false
13237 4443 Acked Write 9 Done Sent Done false
13238 4444 Acked Write 10 Done Sent Done false
13239 4445 Acked Write 1 Done Sent Done false
13240 4446 Acked Read 9 Done Sent Done false
13241 4447 Acked Write 1 Done Sent Done false
13242 4448 Acked Write 9 Done Sent Done false
13243 4449 Acked Write 1 Done Sent Done false
13244 4450 Acked Read 10 Done Sent Done false
13245 4451 Acked Read 6 Done Sent Done false
13246 4452 Acked Write 8 Done Sent Done false
13247 4453 Acked Read 10 Done Sent Done false
13248 4454 Acked Flush 0 Done Sent Done false
13249 4455 Acked Read 5 Done Sent Done false
13250 4456 Acked Read 7 Done Sent Done false
13251 4457 Acked Read 2 Done Sent Done false
13252 4458 Acked Write 5 Done Sent Done false
13253 4459 Acked Write 7 Done Sent Done false
13254 4460 Acked Read 2 Done Sent Done false
13255 4461 Acked Write 4 Done Sent Done false
13256 4462 Acked Read 5 Done Sent Done false
13257 4463 Acked Write 3 Done Sent Done false
13258 4464 Acked Read 7 Done Sent Done false
13259 4465 Acked Read 10 Done Sent Done false
13260 4466 Acked Read 2 Done Sent Done false
13261 4467 Acked Read 3 Done Sent Done false
13262 4468 Acked Flush 0 Done Sent Done false
13263 4469 Acked Read 7 Done Sent Done false
13264 4470 Acked Write 10 Done Sent Done false
13265 4471 Acked Flush 0 Done Sent Done false
13266 4472 Acked Read 6 Done Sent Done false
13267 4473 Acked Write 3 Done Sent Done false
13268 4474 Acked Read 8 Done Sent Done false
13269 4475 Acked Read 9 Done Sent Done false
13270 4476 Acked Write 2 Done Sent Done false
13271 4477 Acked Read 10 Done Sent Done false
13272 4478 Acked Read 3 Done Sent Done false
13273 4479 Acked Read 2 Done Sent Done false
13274 4480 Acked Write 4 Done Sent Done false
13275 4481 Acked Write 1 Done Sent Done false
13276 4482 Acked Write 6 Done Sent Done false
13277 4483 Acked Flush 0 Done Sent Done false
13278 4484 Acked Read 5 Done Sent Done false
13279 4485 Acked Flush 0 Done Sent Done false
13280 4486 Acked Read 7 Done Sent Done false
13281 4487 Acked Write 6 Done Sent Done false
13282 4488 Acked Flush 0 Done Sent Done false
13283 4489 Acked Write 6 Done Sent Done false
13284 4490 Acked Read 1 Done Sent Done false
13285 4491 Acked Flush 0 Done Sent Done false
13286 4492 Acked Write 5 Done Sent Done false
13287 4493 Acked Write 9 Done Sent Done false
13288 4494 Acked Write 4 Done Sent Done false
13289 4495 Acked Write 5 Done Sent Done false
13290 4496 Acked Write 9 Done Sent Done false
13291 4497 Acked Write 3 Done Sent Done false
13292 4498 Acked Flush 0 Done Sent Done false
13293 4499 Acked Write 5 Done Sent Done false
13294 4500 Acked Read 2 Done Sent Done false
13295 4501 Acked Read 3 Done Sent Done false
13296 4502 Acked Write 9 Done Sent Done false
13297 4503 Acked Write 8 Done Sent Done false
13298 4504 Acked Read 10 Done Sent Done false
13299 4505 Acked Write 9 Done Sent Done false
13300 4506 Acked Write 6 Done Sent Done false
13301 4507 Acked Read 7 Done Sent Done false
13302 4508 Acked Read 4 Done Sent Done false
13303 4509 Acked Write 2 Done Sent Done false
13304 4510 Acked Read 1 Done Sent Done false
13305 4511 Acked Write 9 Done Sent Done false
13306 4512 Acked Read 10 Done Sent Done false
13307 4513 Acked Write 7 Done Sent Done false
13308 4514 Acked Write 5 Done Sent Done false
13309 4515 Acked Read 4 Done Sent Done false
13310 4516 Acked Write 4 Done Sent Done false
13311 4517 Acked Write 10 Done Sent Done false
13312 4518 Acked Read 10 Done Sent Done false
13313 4519 Acked Write 10 Done Sent Done false
13314 4520 Acked Read 7 Done Sent Done false
13315 4521 Acked Read 1 Done Sent Done false
13316 4522 Acked Write 8 Done Sent Done false
13317 4523 Acked Write 7 Done Sent Done false
13318 4524 Acked Read 2 Done Sent Done false
13319 4525 Acked Read 3 Done Sent Done false
13320 4526 Acked Write 6 Done Sent Done false
13321 4527 Acked Read 8 Done Sent Done false
13322 4528 Acked Read 2 Done Sent Done false
13323 4529 Acked Write 3 Done Sent Done false
13324 4530 Acked Read 4 Done Sent Done false
13325 4531 Acked Write 8 Done Sent Done false
13326 4532 Acked Write 3 Done Sent Done false
13327 4533 Acked Flush 0 Done Sent Done false
13328 4534 Acked Flush 0 Done Sent Done false
13329 4535 Acked Read 4 Done Sent Done false
13330 4536 Acked Read 8 Done Sent Done false
13331 4537 Acked Read 8 Done Sent Done false
13332 4538 Acked Flush 0 Done Sent Done false
13333 4539 Acked Write 7 Done Sent Done false
13334 4540 Acked Read 1 Done Sent Done false
13335 4541 Acked Write 8 Done Sent Done false
13336 4542 Acked Read 1 Done Sent Done false
13337 4543 Acked Read 6 Done Sent Done false
13338 4544 Acked Read 10 Done Sent Done false
13339 4545 Acked Write 9 Done Sent Done false
13340 4546 Acked Write 3 Done Sent Done false
13341 4547 Acked Write 1 Done Sent Done false
13342 4548 Acked Read 1 Done Sent Done false
13343 4549 Acked Write 4 Done Sent Done false
13344 4550 Acked Read 1 Done Sent Done false
13345 4551 Acked Write 5 Done Sent Done false
13346 4552 Acked Read 1 Done Sent Done false
13347 4553 Acked Read 8 Done Sent Done false
13348 4554 Acked Read 1 Done Sent Done false
13349 4555 Acked Read 6 Done Sent Done false
13350 4556 Acked Write 8 Done Sent Done false
13351 4557 Acked Read 6 Done Sent Done false
13352 4558 Acked Read 7 Done Sent Done false
13353 4559 Acked Read 6 Done Sent Done false
13354 4560 Acked Write 5 Done Sent Done false
13355 4561 Acked Write 8 Done Sent Done false
13356 4562 Acked Write 2 Done Sent Done false
13357 4563 Acked Read 10 Done Sent Done false
13358 4564 Acked Read 5 Done Sent Done false
13359 4565 Acked Write 3 Done Sent Done false
13360 4566 Acked Read 2 Done Sent Done false
13361 4567 Acked Read 1 Done Sent Done false
13362 4568 Acked Write 4 Done Sent Done false
13363 4569 Acked Read 9 Done Sent Done false
13364 4570 Acked Write 9 Done Sent Done false
13365 4571 Acked Write 7 Done Sent Done false
13366 4572 Acked Write 10 Done Sent Done false
13367 4573 Acked Read 4 Done Sent Done false
13368 4574 Acked Read 5 Done Sent Done false
13369 4575 Acked Write 4 Done Sent Done false
13370 4576 Acked Write 5 Done Sent Done false
13371 4577 Acked Read 3 Done Sent Done false
13372 4578 Acked Read 3 Done Sent Done false
13373 4579 Acked Write 9 Done Sent Done false
13374 4580 Acked Write 5 Done Sent Done false
13375 4581 Acked Read 8 Done Sent Done false
13376 4582 Acked Read 7 Done Sent Done false
13377 4583 Acked Write 6 Done Sent Done false
13378 4584 Acked Read 6 Done Sent Done false
13379 4585 Acked Write 4 Done Sent Done false
13380 4586 Acked Write 9 Done Sent Done false
13381 4587 Acked Flush 0 Done Sent Done false
13382 4588 Acked Write 3 Done Sent Done false
13383 4589 Acked Read 8 Done Sent Done false
13384 4590 Acked Read 2 Done Sent Done false
13385 4591 Acked Read 10 Done Sent Done false
13386 4592 Acked Flush 0 Done Sent Done false
13387 4593 Acked Read 7 Done Sent Done false
13388 4594 Acked Read 5 Done Sent Done false
13389 4595 Acked Read 3 Done Sent Done false
13390 4596 Acked Write 7 Done Sent Done false
13391 4597 Acked Read 1 Done Sent Done false
13392 4598 Acked Flush 0 Done Sent Done false
13393 4599 Acked Read 7 Done Sent Done false
13394 4600 Acked Write 9 Done Sent Done false
13395 4601 Acked Read 10 Done Sent Done false
13396 4602 Acked Flush 0 Done Sent Done false
13397 4603 Acked Read 8 Done Sent Done false
13398 4604 Acked Write 2 Done Sent Done false
13399 4605 Acked Read 7 Done Sent Done false
13400 4606 Acked Read 2 Done Sent Done false
13401 4607 Acked Read 3 Done Sent Done false
13402 4608 Acked Write 1 Done Sent Done false
13403 4609 Acked Read 4 Done Sent Done false
13404 4610 Acked Read 3 Done Sent Done false
13405 4611 Acked Write 5 Done Sent Done false
13406 4612 Acked Read 3 Done Sent Done false
13407 4613 Acked Write 8 Done Sent Done false
13408 4614 Acked Read 8 Done Sent Done false
13409 4615 Acked Flush 0 Done Sent Done false
13410 4616 Acked Read 8 Done Sent Done false
13411 4617 Acked Write 6 Done Sent Done false
13412 4618 Acked Read 1 Done Sent Done false
13413 4619 Acked Write 2 Done Sent Done false
13414 4620 Acked Flush 0 Done Sent Done false
13415 4621 Acked Read 8 Done Sent Done false
13416 4622 Acked Flush 0 Done Sent Done false
13417 STATES DS:0 DS:1 DS:2 TOTAL
13418 Sent 0 302 0 302
13419 Done 302 0 302 604
13420 Skipped 0 0 0 0
13421 Error 0 0 0 0
13422 Last Flush: 4622 4320 4622
13423 Downstairs last five completed: 4320 4319 4318 4317 4316
13424 Upstairs last five completed: 4622 4621 4620 4619 4618
13425 CLIENT: Up:0 ds:302 act:2
13426 ----------------------------------------------------------------
13427 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13428 States: Active Offline Active
13429 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13430 4321 Acked Read 2 Done Sent Done false
13431 4322 Acked Read 1 Done Sent Done false
13432 4323 Acked Read 2 Done Sent Done false
13433 4324 Acked Read 7 Done Sent Done false
13434 4325 Acked Write 9 Done Sent Done false
13435 4326 Acked Write 3 Done Sent Done false
13436 4327 Acked Read 1 Done Sent Done false
13437 4328 Acked Read 7 Done Sent Done false
13438 4329 Acked Write 1 Done Sent Done false
13439 4330 Acked Write 9 Done Sent Done false
13440 4331 Acked Write 9 Done Sent Done false
13441 4332 Acked Write 6 Done Sent Done false
13442 4333 Acked Read 6 Done Sent Done false
13443 4334 Acked Write 7 Done Sent Done false
13444 4335 Acked Write 10 Done Sent Done false
13445 4336 Acked Flush 0 Done Sent Done false
13446 4337 Acked Read 8 Done Sent Done false
13447 4338 Acked Read 8 Done Sent Done false
13448 4339 Acked Write 7 Done Sent Done false
13449 4340 Acked Write 8 Done Sent Done false
13450 4341 Acked Read 10 Done Sent Done false
13451 4342 Acked Read 4 Done Sent Done false
13452 4343 Acked Flush 0 Done Sent Done false
13453 4344 Acked Write 10 Done Sent Done false
13454 4345 Acked Write 4 Done Sent Done false
13455 4346 Acked Read 9 Done Sent Done false
13456 4347 Acked Read 7 Done Sent Done false
13457 4348 Acked Write 1 Done Sent Done false
13458 4349 Acked Read 2 Done Sent Done false
13459 4350 Acked Flush 0 Done Sent Done false
13460 4351 Acked Read 10 Done Sent Done false
13461 4352 Acked Write 6 Done Sent Done false
13462 4353 Acked Write 8 Done Sent Done false
13463 4354 Acked Read 1 Done Sent Done false
13464 4355 Acked Read 5 Done Sent Done false
13465 4356 Acked Read 3 Done Sent Done false
13466 4357 Acked Read 5 Done Sent Done false
13467 4358 Acked Flush 0 Done Sent Done false
13468 4359 Acked Read 1 Done Sent Done false
13469 4360 Acked Read 3 Done Sent Done false
13470 4361 Acked Write 3 Done Sent Done false
13471 4362 Acked Read 2 Done Sent Done false
13472 4363 Acked Read 5 Done Sent Done false
13473 4364 Acked Write 8 Done Sent Done false
13474 4365 Acked Read 2 Done Sent Done false
13475 4366 Acked Write 4 Done Sent Done false
13476 4367 Acked Read 1 Done Sent Done false
13477 4368 Acked Flush 0 Done Sent Done false
13478 4369 Acked Read 6 Done Sent Done false
13479 4370 Acked Write 9 Done Sent Done false
13480 4371 Acked Read 7 Done Sent Done false
13481 4372 Acked Read 7 Done Sent Done false
13482 4373 Acked Write 10 Done Sent Done false
13483 4374 Acked Flush 0 Done Sent Done false
13484 4375 Acked Write 2 Done Sent Done false
13485 4376 Acked Write 2 Done Sent Done false
13486 4377 Acked Write 10 Done Sent Done false
13487 4378 Acked Write 5 Done Sent Done false
13488 4379 Acked Read 8 Done Sent Done false
13489 4380 Acked Write 8 Done Sent Done false
13490 4381 Acked Read 7 Done Sent Done false
13491 4382 Acked Read 4 Done Sent Done false
13492 4383 Acked Read 4 Done Sent Done false
13493 4384 Acked Read 8 Done Sent Done false
13494 4385 Acked Read 9 Done Sent Done false
13495 4386 Acked Write 6 Done Sent Done false
13496 4387 Acked Write 8 Done Sent Done false
13497 4388 Acked Read 1 Done Sent Done false
13498 4389 Acked Write 10 Done Sent Done false
13499 4390 Acked Read 10 Done Sent Done false
13500 4391 Acked Read 1 Done Sent Done false
13501 4392 Acked Read 7 Done Sent Done false
13502 4393 Acked Flush 0 Done Sent Done false
13503 4394 Acked Write 5 Done Sent Done false
13504 4395 Acked Read 1 Done Sent Done false
13505 4396 Acked Read 1 Done Sent Done false
13506 4397 Acked Read 9 Done Sent Done false
13507 4398 Acked Flush 0 Done Sent Done false
13508 4399 Acked Write 3 Done Sent Done false
13509 4400 Acked Write 2 Done Sent Done false
13510 4401 Acked Flush 0 Done Sent Done false
13511 4402 Acked Write 10 Done Sent Done false
13512 4403 Acked Read 6 Done Sent Done false
13513 4404 Acked Write 4 Done Sent Done false
13514 4405 Acked Read 4 Done Sent Done false
13515 4406 Acked Write 1 Done Sent Done false
13516 4407 Acked Read 4 Done Sent Done false
13517 4408 Acked Write 1 Done Sent Done false
13518 4409 Acked Read 1 Done Sent Done false
13519 4410 Acked Write 3 Done Sent Done false
13520 4411 Acked Read 1 Done Sent Done false
13521 4412 Acked Write 9 Done Sent Done false
13522 4413 Acked Flush 0 Done Sent Done false
13523 4414 Acked Read 7 Done Sent Done false
13524 4415 Acked Write 2 Done Sent Done false
13525 4416 Acked Read 3 Done Sent Done false
13526 4417 Acked Write 8 Done Sent Done false
13527 4418 Acked Read 4 Done Sent Done false
13528 4419 Acked Flush 0 Done Sent Done false
13529 4420 Acked Write 1 Done Sent Done false
13530 4421 Acked Read 1 Done Sent Done false
13531 4422 Acked Write 5 Done Sent Done false
13532 4423 Acked Write 8 Done Sent Done false
13533 4424 Acked Read 4 Done Sent Done false
13534 4425 Acked Write 2 Done Sent Done false
13535 4426 Acked Read 5 Done Sent Done false
13536 4427 Acked Write 8 Done Sent Done false
13537 4428 Acked Write 1 Done Sent Done false
13538 4429 Acked Write 10 Done Sent Done false
13539 4430 Acked Read 1 Done Sent Done false
13540 4431 Acked Read 6 Done Sent Done false
13541 4432 Acked Flush 0 Done Sent Done false
13542 4433 Acked Read 7 Done Sent Done false
13543 4434 Acked Write 7 Done Sent Done false
13544 4435 Acked Read 10 Done Sent Done false
13545 4436 Acked Write 4 Done Sent Done false
13546 4437 Acked Read 2 Done Sent Done false
13547 4438 Acked Read 8 Done Sent Done false
13548 4439 Acked Read 8 Done Sent Done false
13549 4440 Acked Read 3 Done Sent Done false
13550 4441 Acked Write 8 Done Sent Done false
13551 4442 Acked Write 9 Done Sent Done false
13552 4443 Acked Write 9 Done Sent Done false
13553 4444 Acked Write 10 Done Sent Done false
13554 4445 Acked Write 1 Done Sent Done false
13555 4446 Acked Read 9 Done Sent Done false
13556 4447 Acked Write 1 Done Sent Done false
13557 4448 Acked Write 9 Done Sent Done false
13558 4449 Acked Write 1 Done Sent Done false
13559 4450 Acked Read 10 Done Sent Done false
13560 4451 Acked Read 6 Done Sent Done false
13561 4452 Acked Write 8 Done Sent Done false
13562 4453 Acked Read 10 Done Sent Done false
13563 4454 Acked Flush 0 Done Sent Done false
13564 4455 Acked Read 5 Done Sent Done false
13565 4456 Acked Read 7 Done Sent Done false
13566 4457 Acked Read 2 Done Sent Done false
13567 4458 Acked Write 5 Done Sent Done false
13568 4459 Acked Write 7 Done Sent Done false
13569 4460 Acked Read 2 Done Sent Done false
13570 4461 Acked Write 4 Done Sent Done false
13571 4462 Acked Read 5 Done Sent Done false
13572 4463 Acked Write 3 Done Sent Done false
13573 4464 Acked Read 7 Done Sent Done false
13574 4465 Acked Read 10 Done Sent Done false
13575 4466 Acked Read 2 Done Sent Done false
13576 4467 Acked Read 3 Done Sent Done false
13577 4468 Acked Flush 0 Done Sent Done false
13578 4469 Acked Read 7 Done Sent Done false
13579 4470 Acked Write 10 Done Sent Done false
13580 4471 Acked Flush 0 Done Sent Done false
13581 4472 Acked Read 6 Done Sent Done false
13582 4473 Acked Write 3 Done Sent Done false
13583 4474 Acked Read 8 Done Sent Done false
13584 4475 Acked Read 9 Done Sent Done false
13585 4476 Acked Write 2 Done Sent Done false
13586 4477 Acked Read 10 Done Sent Done false
13587 4478 Acked Read 3 Done Sent Done false
13588 4479 Acked Read 2 Done Sent Done false
13589 4480 Acked Write 4 Done Sent Done false
13590 4481 Acked Write 1 Done Sent Done false
13591 4482 Acked Write 6 Done Sent Done false
13592 4483 Acked Flush 0 Done Sent Done false
13593 4484 Acked Read 5 Done Sent Done false
13594 4485 Acked Flush 0 Done Sent Done false
13595 4486 Acked Read 7 Done Sent Done false
13596 4487 Acked Write 6 Done Sent Done false
13597 4488 Acked Flush 0 Done Sent Done false
13598 4489 Acked Write 6 Done Sent Done false
13599 4490 Acked Read 1 Done Sent Done false
13600 4491 Acked Flush 0 Done Sent Done false
13601 4492 Acked Write 5 Done Sent Done false
13602 4493 Acked Write 9 Done Sent Done false
13603 4494 Acked Write 4 Done Sent Done false
13604 4495 Acked Write 5 Done Sent Done false
13605 4496 Acked Write 9 Done Sent Done false
13606 4497 Acked Write 3 Done Sent Done false
13607 4498 Acked Flush 0 Done Sent Done false
13608 4499 Acked Write 5 Done Sent Done false
13609 4500 Acked Read 2 Done Sent Done false
13610 4501 Acked Read 3 Done Sent Done false
13611 4502 Acked Write 9 Done Sent Done false
13612 4503 Acked Write 8 Done Sent Done false
13613 4504 Acked Read 10 Done Sent Done false
13614 4505 Acked Write 9 Done Sent Done false
13615 4506 Acked Write 6 Done Sent Done false
13616 4507 Acked Read 7 Done Sent Done false
13617 4508 Acked Read 4 Done Sent Done false
13618 4509 Acked Write 2 Done Sent Done false
13619 4510 Acked Read 1 Done Sent Done false
13620 4511 Acked Write 9 Done Sent Done false
13621 4512 Acked Read 10 Done Sent Done false
13622 4513 Acked Write 7 Done Sent Done false
13623 4514 Acked Write 5 Done Sent Done false
13624 4515 Acked Read 4 Done Sent Done false
13625 4516 Acked Write 4 Done Sent Done false
13626 4517 Acked Write 10 Done Sent Done false
13627 4518 Acked Read 10 Done Sent Done false
13628 4519 Acked Write 10 Done Sent Done false
13629 4520 Acked Read 7 Done Sent Done false
13630 4521 Acked Read 1 Done Sent Done false
13631 4522 Acked Write 8 Done Sent Done false
13632 4523 Acked Write 7 Done Sent Done false
13633 4524 Acked Read 2 Done Sent Done false
13634 4525 Acked Read 3 Done Sent Done false
13635 4526 Acked Write 6 Done Sent Done false
13636 4527 Acked Read 8 Done Sent Done false
13637 4528 Acked Read 2 Done Sent Done false
13638 4529 Acked Write 3 Done Sent Done false
13639 4530 Acked Read 4 Done Sent Done false
13640 4531 Acked Write 8 Done Sent Done false
13641 4532 Acked Write 3 Done Sent Done false
13642 4533 Acked Flush 0 Done Sent Done false
13643 4534 Acked Flush 0 Done Sent Done false
13644 4535 Acked Read 4 Done Sent Done false
13645 4536 Acked Read 8 Done Sent Done false
13646 4537 Acked Read 8 Done Sent Done false
13647 4538 Acked Flush 0 Done Sent Done false
13648 4539 Acked Write 7 Done Sent Done false
13649 4540 Acked Read 1 Done Sent Done false
13650 4541 Acked Write 8 Done Sent Done false
13651 4542 Acked Read 1 Done Sent Done false
13652 4543 Acked Read 6 Done Sent Done false
13653 4544 Acked Read 10 Done Sent Done false
13654 4545 Acked Write 9 Done Sent Done false
13655 4546 Acked Write 3 Done Sent Done false
13656 4547 Acked Write 1 Done Sent Done false
13657 4548 Acked Read 1 Done Sent Done false
13658 4549 Acked Write 4 Done Sent Done false
13659 4550 Acked Read 1 Done Sent Done false
13660 4551 Acked Write 5 Done Sent Done false
13661 4552 Acked Read 1 Done Sent Done false
13662 4553 Acked Read 8 Done Sent Done false
13663 4554 Acked Read 1 Done Sent Done false
13664 4555 Acked Read 6 Done Sent Done false
13665 4556 Acked Write 8 Done Sent Done false
13666 4557 Acked Read 6 Done Sent Done false
13667 4558 Acked Read 7 Done Sent Done false
13668 4559 Acked Read 6 Done Sent Done false
13669 4560 Acked Write 5 Done Sent Done false
13670 4561 Acked Write 8 Done Sent Done false
13671 4562 Acked Write 2 Done Sent Done false
13672 4563 Acked Read 10 Done Sent Done false
13673 4564 Acked Read 5 Done Sent Done false
13674 4565 Acked Write 3 Done Sent Done false
13675 4566 Acked Read 2 Done Sent Done false
13676 4567 Acked Read 1 Done Sent Done false
13677 4568 Acked Write 4 Done Sent Done false
13678 4569 Acked Read 9 Done Sent Done false
13679 4570 Acked Write 9 Done Sent Done false
13680 4571 Acked Write 7 Done Sent Done false
13681 4572 Acked Write 10 Done Sent Done false
13682 4573 Acked Read 4 Done Sent Done false
13683 4574 Acked Read 5 Done Sent Done false
13684 4575 Acked Write 4 Done Sent Done false
13685 4576 Acked Write 5 Done Sent Done false
13686 4577 Acked Read 3 Done Sent Done false
13687 4578 Acked Read 3 Done Sent Done false
13688 4579 Acked Write 9 Done Sent Done false
13689 4580 Acked Write 5 Done Sent Done false
13690 4581 Acked Read 8 Done Sent Done false
13691 4582 Acked Read 7 Done Sent Done false
13692 4583 Acked Write 6 Done Sent Done false
13693 4584 Acked Read 6 Done Sent Done false
13694 4585 Acked Write 4 Done Sent Done false
13695 4586 Acked Write 9 Done Sent Done false
13696 4587 Acked Flush 0 Done Sent Done false
13697 4588 Acked Write 3 Done Sent Done false
13698 4589 Acked Read 8 Done Sent Done false
13699 4590 Acked Read 2 Done Sent Done false
13700 4591 Acked Read 10 Done Sent Done false
13701 4592 Acked Flush 0 Done Sent Done false
13702 4593 Acked Read 7 Done Sent Done false
13703 4594 Acked Read 5 Done Sent Done false
13704 4595 Acked Read 3 Done Sent Done false
13705 4596 Acked Write 7 Done Sent Done false
13706 4597 Acked Read 1 Done Sent Done false
13707 4598 Acked Flush 0 Done Sent Done false
13708 4599 Acked Read 7 Done Sent Done false
13709 4600 Acked Write 9 Done Sent Done false
13710 4601 Acked Read 10 Done Sent Done false
13711 4602 Acked Flush 0 Done Sent Done false
13712 4603 Acked Read 8 Done Sent Done false
13713 4604 Acked Write 2 Done Sent Done false
13714 4605 Acked Read 7 Done Sent Done false
13715 4606 Acked Read 2 Done Sent Done false
13716 4607 Acked Read 3 Done Sent Done false
13717 4608 Acked Write 1 Done Sent Done false
13718 4609 Acked Read 4 Done Sent Done false
13719 4610 Acked Read 3 Done Sent Done false
13720 4611 Acked Write 5 Done Sent Done false
13721 4612 Acked Read 3 Done Sent Done false
13722 4613 Acked Write 8 Done Sent Done false
13723 4614 Acked Read 8 Done Sent Done false
13724 4615 Acked Flush 0 Done Sent Done false
13725 4616 Acked Read 8 Done Sent Done false
13726 4617 Acked Write 6 Done Sent Done false
13727 4618 Acked Read 1 Done Sent Done false
13728 4619 Acked Write 2 Done Sent Done false
13729 4620 Acked Flush 0 Done Sent Done false
13730 4621 Acked Read 8 Done Sent Done false
13731 4622 Acked Flush 0 Done Sent Done false
13732 STATES DS:0 DS:1 DS:2 TOTAL
13733 Sent 0 302 0 302
13734 Done 302 0 302 604
13735 Skipped 0 0 0 0
13736 Error 0 0 0 0
13737 Last Flush: 4622 4320 4622
13738 Downstairs last five completed: 4320 4319 4318 4317 4316
13739 Upstairs last five completed: 4622 4621 4620 4619 4618
13740 CLIENT: Up:0 ds:302 act:2
137412025-01-21T15:50:04.055ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137422025-01-21T15:50:04.055ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48386) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137432025-01-21T15:50:04.056ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137442025-01-21T15:50:04.057ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137452025-01-21T15:50:04.057ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137462025-01-21T15:50:04.057ZINFOcrucible: send last flush ID to this DS: 4320 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137472025-01-21T15:50:04.057ZINFOcrucible: Replied this last flush ID: 4320 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137482025-01-21T15:50:04.057ZINFOcrucible: [1] client re-new 302 jobs since flush 4320 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137492025-01-21T15:50:04.058ZINFOcrucible: [1] Replayed 302 jobs since flush: 4320 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
13750 ----------------------------------------------------------------
13751 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13752 Downstairs last five completed: 4622 4621 4620 4619 4618
13753 Upstairs last five completed: 4622 4621 4620 4619 4618
13754 CLIENT: Up:0 ds:0 act:3
13755 Replay: All jobs finished
13756 Main task has work to do, go find it
13757 got dsc Stop(1)
13758 stop 1
13759 Jan 21 15:50:04.304[8820] Got stop action so:false kr:false
13760 INFO request completed, latency_us: 663, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 6391d594-762e-4b1a-b147-2bd6a19288ff, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
13761 Jan 21 15:50:04.305 INFO request completed, latency_us: 124, response_code: 200, uri: /state/cid/1, method: GET, req_id: d16deec0-0c0f-4d98-9363-e9d1cddf5d40, remote_addr: 127.0.0.1:{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:50:04.305859222Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
13762 52945, local_addr: 127.0.0.1:9998
137632025-01-21T15:50:04.305ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137642025-01-21T15:50:04.305ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
137652025-01-21T15:50:04.306ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
13766 [8820] Exited with: ExitStatus(unix_wait_status(9))
13767 [8820][1] reports Exit
13768 Jan 21 15:50:08.307 INFO request completed, latency_us: 207, response_code: 200, uri: /state/cid/1, method: GET, req_id: 490dd8d9-8462-4e68-8304-fb40b8f2586b, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
13769 001/300 Read block 2197 len 24576
13770 002/300 Read block 2804 len 32768
13771 003/300 Write block 4524 len 16384 data: 4 3 2 2
13772 004/300 Read block 1865 len 32768
13773 005/300 Read block 4245 len 32768
13774 006/300 Read block 1913 len 40960
13775 007/300 Write block 1073 len 28672 data: 6 5 3 3 3 4 3
13776 008/300 Read block 2151 len 24576
13777 009/300 Write block 4353 len 40960 data: 4 4 2 2 2 2 2 2 2 3
13778 010/300 Write block 3692 len 24576 data: 3 2 2 2 2 3
13779 011/300 Read block 527 len 8192
13780 012/300 Flush
13781 013/300 Read block 1109 len 20480
13782 014/300 Write block 2076 len 28672 data: 6 5 4 5 4 4 4
13783 015/300 Read block 4049 len 8192
13784 016/300 Read block 4936 len 12288
13785 017/300 Write block 1265 len 36864 data: 3 4 5 2 2 2 2 2 2
13786 018/300 Read block 4219 len 4096
13787 019/300 Read block 3214 len 12288
13788 020/300 Write block 1449 len 16384 data: 4 4 4 4
13789 021/300 Write block 922 len 12288 data: 3 3 2
13790 022/300 Read block 1317 len 20480
13791 023/300 Read block 4711 len 40960
13792 024/300 Read block 3710 len 28672
13793 025/300 Write block 3892 len 16384 data: 5 5 4 4
13794 026/300 Write block 293 len 40960 data: 2 2 2 3 3 4 4 4 5 4
13795 027/300 Read block 3367 len 20480
13796 028/300 Read block 978 len 36864
13797 029/300 Read block 4009 len 24576
13798 030/300 Write block 4350 len 40960 data: 5 5 4 5 5 3 3 3 3 3
13799 031/300 Write block 1820 len 8192 data: 3 3
13800 032/300 Read block 3917 len 16384
13801 033/300 Read block 1331 len 16384
13802 034/300 Read block 2586 len 12288
13803 035/300 Write block 2250 len 8192 data: 5 4
13804 036/300 Read block 2475 len 24576
13805 037/300 Write block 762 len 28672 data: 4 4 3 3 2 2 2
13806 038/300 Write block 3326 len 12288 data: 2 3 3
13807 039/300 Write block 3519 len 20480 data: 5 4 4 4 5
13808 040/300 Read block 1191 len 28672
13809 041/300 Write block 2428 len 28672 data: 4 5 4 4 3 4 4
13810 042/300 Write block 703 len 32768 data: 3 3 3 3 2 2 2 2
13811 043/300 Read block 259 len 20480
13812 044/300 Write block 829 len 40960 data: 3 4 4 3 2 2 2 2 3 3
13813 045/300 Write block 2078 len 24576 data: 5 6 5 5 5 3
13814 046/300 Read block 1941 len 32768
13815 047/300 Read block 4481 len 12288
13816 048/300 Flush
13817 049/300 Read block 3976 len 40960
13818 050/300 Write block 1477 len 36864 data: 4 3 3 2 2 2 3 3 3
13819 051/300 Read block 906 len 8192
13820 052/300 Write block 71 len 16384 data: 2 3 3 3
13821 053/300 Read block 1248 len 16384
13822 054/300 Read block 2273 len 36864
13823 055/300 Read block 4315 len 24576
13824 056/300 Read block 4494 len 8192
13825 057/300 Write block 1310 len 16384 data: 3 4 5 5
13826 058/300 Read block 2220 len 32768
13827 059/300 Read block 3384 len 12288
13828 060/300 Read block 3151 len 20480
13829 061/300 Read block 118 len 40960
13830 062/300 Read block 4119 len 24576
13831 063/300 Read block 1846 len 4096
13832 064/300 Read block 3433 len 16384
13833 065/300 Flush
13834 066/300 Read block 2307 len 24576
13835 067/300 Read block 1200 len 36864
13836 068/300 Flush
13837 069/300 Read block 795 len 36864
13838 070/300 Read block 1723 len 4096
13839 071/300 Write block 4680 len 12288 data: 3 2 2
13840 072/300 Write block 753 len 12288 data: 2 3 3
13841 073/300 Flush
13842 074/300 Read block 2774 len 40960
13843 075/300 Read block 1452 len 36864
13844 076/300 Write block 3430 len 16384 data: 2 2 2 2
13845 077/300 Read block 846 len 28672
13846 078/300 Read block 87 len 4096
13847 079/300 Read block 2133 len 32768
13848 080/300 Read block 364 len 8192
13849 081/300 Read block 1752 len 24576
13850 082/300 Write block 3105 len 4096 data: 2
13851 083/300 Read block 2706 len 8192
13852 084/300 Read block 1299 len 16384
13853 085/300 Read block 3797 len 40960
13854 086/300 Read block 2186 len 40960
13855 087/300 Read block 3594 len 16384
13856 088/300 Write block 367 len 36864 data: 3 4 3 4 4 4 4 4 3
13857 089/300 Read block 446 len 36864
13858 090/300 Flush
13859 091/300 Flush
13860 092/300 Read block 4874 len 16384
13861 093/300 Read block 1410 len 40960
13862 094/300 Flush
13863 095/300 Read block 4631 len 4096
13864 096/300 Read block 2291 len 36864
13865 097/300 Read block 3972 len 16384
13866 098/300 Read block 2263 len 28672
13867 099/300 Flush
13868 100/300 Write block 1997 len 16384 data: 4 4 4 4
13869 101/300 Read block 3302 len 20480
13870 102/300 Read block 2514 len 32768
13871 103/300 Read block 499 len 32768
13872 104/300 Read block 2821 len 8192
13873 105/300 Read block 499 len 20480
13874 106/300 Write block 429 len 24576 data: 2 3 3 2 2 3
13875 107/300 Flush
13876 108/300 Write block 1184 len 16384 data: 3 4 5 5
13877 109/300 Read block 293 len 12288
13878 110/300 Flush
13879 111/300 Read block 2810 len 24576
13880 112/300 Write block 2526 len 40960 data: 4 4 4 4 3 3 3 3 4 4
13881 113/300 Read block 3777 len 32768
13882 114/300 Read block 4573 len 4096
13883 115/300 Flush
13884 116/300 Write block 4437 len 32768 data: 5 5 5 4 3 5 4 4
13885 117/300 Write block 4662 len 12288 data: 2 2 2
13886 118/300 Write block 2996 len 4096 data: 2
13887 119/300 Write block 1254 len 32768 data: 3 3 2 2 2 3 3 3
13888 120/300 Read block 3434 len 40960
13889 121/300 Write block 2271 len 40960 data: 4 4 4 3 3 3 2 2 2 3
13890 122/300 Read block 197 len 40960
13891 123/300 Write block 3106 len 40960 data: 2 2 2 2 3 3 3 3 4 4
13892 124/300 Read block 2472 len 12288
13893 125/300 Flush
13894 126/300 Read block 4264 len 36864
13895 127/300 Read block 1721 len 28672
13896 128/300 Write block 1003 len 8192 data: 2 2
13897 129/300 Write block 893 len 20480 data: 7 5 5 3 2
13898 130/300 Write block 4032 len 36864 data: 5 5 5 5 6 6 7 5 3
13899 131/300 Read block 1908 len 32768
13900 132/300 Read block 1381 len 4096
13901 133/300 Read block 2803 len 32768
13902 134/300 Read block 3944 len 32768
13903 135/300 Read block 3343 len 32768
13904 136/300 Write block 4713 len 40960 data: 3 3 3 3 3 3 3 6 6 6
13905 137/300 Read block 4955 len 40960
13906 138/300 Read block 4683 len 20480
13907 139/300 Write block 1611 len 24576 data: 3 3 3 3 3 3
13908 140/300 Read block 4408 len 12288
13909 141/300 Write block 3828 len 36864 data: 4 3 3 3 2 2 2 4 3
13910 142/300 Read block 2214 len 8192
13911 143/300 Flush
13912 144/300 Read block 3387 len 40960
13913 145/300 Read block 68 len 4096
13914 146/300 Write block 4347 len 16384 data: 4 5 5 6
13915 147/300 Flush
13916 148/300 Write block 119 len 4096 data: 3
13917 149/300 Read block 634 len 8192
13918 150/300 Read block 4979 len 8192
13919 151/300 Write block 3254 len 32768 data: 3 3 3 3 3 3 2 2
13920 152/300 Flush
13921 153/300 Flush
13922 154/300 Read block 4261 len 40960
13923 155/300 Read block 783 len 40960
13924 156/300 Write block 1320 len 16384 data: 6 5 5 6
13925 157/300 Read block 1439 len 16384
13926 158/300 Flush
13927 159/300 Flush
13928 160/300 Read block 4750 len 40960
13929 161/300 Flush
13930 162/300 Read block 4125 len 12288
13931 163/300 Write block 3911 len 40960 data: 3 3 4 4 5 5 5 4 4 3
13932 164/300 Write block 3594 len 36864 data: 3 3 3 3 2 2 3 3 2
13933 165/300 Read block 542 len 4096
13934 166/300 Read block 4142 len 8192
13935 167/300 Write block 1311 len 8192 data: 5 6
13936 168/300 Write block 701 len 16384 data: 3 3 4 4
13937 169/300 Read block 63 len 4096
13938 170/300 Flush
13939 171/300 Write block 2021 len 20480 data: 4 3 3 3 4
13940 172/300 Read block 2728 len 32768
13941 173/300 Write block 3069 len 12288 data: 3 3 3
13942 174/300 Read block 1404 len 40960
13943 175/300 Read block 4646 len 40960
13944 176/300 Flush
13945 177/300 Read block 410 len 8192
13946 178/300 Read block 4691 len 32768
13947 179/300 Write block 1484 len 8192 data: 4 4
13948 180/300 Read block 4456 len 40960
13949 181/300 Read block 4615 len 24576
13950 182/300 Read block 4285 len 12288
13951 183/300 Write block 214 len 24576 data: 5 5 5 5 5 5
13952 184/300 Write block 261 len 12288 data: 4 5 4
13953 185/300 Read block 3262 len 8192
13954 186/300 Read block 3764 len 32768
13955 187/300 Write block 1523 len 32768 data: 2 3 3 3 3 3 4 4
13956 188/300 Read block 4524 len 24576
13957 189/300 Read block 3864 len 24576
13958 190/300 Read block 3729 len 16384
13959 191/300 Read block 4915 len 4096
13960 192/300 Read block 4414 len 24576
13961 193/300 Write block 1917 len 36864 data: 4 5 3 3 3 3 4 3 3
13962 194/300 Flush
13963 195/300 Read block 2328 len 4096
13964 196/300 Read block 1341 len 28672
13965 197/300 Read block 2164 len 4096
13966 198/300 Read block 261 len 24576
13967 199/300 Flush
13968 200/300 Read block 661 len 12288
13969 201/300 Flush
13970 202/300 Write block 840 len 28672 data: 4 4 5 5 5 5 5
13971 203/300 Flush
13972 204/300 Write block 739 len 8192 data: 5 4
13973 205/300 Read block 4206 len 4096
13974 206/300 Read block 2773 len 4096
13975 207/300 Write block 210 len 36864 data: 4 4 5 6 6 6 6 6 6
13976 208/300 Read block 2642 len 4096
13977 209/300 Write block 71 len 16384 data: 3 4 4 4
13978 210/300 Read block 4524 len 24576
13979 211/300 Read block 3321 len 8192
13980 212/300 Write block 3124 len 4096 data: 4
13981 213/300 Read block 2384 len 36864
13982 214/300 Write block 74 len 4096 data: 5
13983 215/300 Read block 2496 len 12288
13984 216/300 Read block 4801 len 40960
13985 217/300 Write block 2747 len 32768 data: 4 5 5 5 4 3 5 5
13986 218/300 Read block 898 len 24576
13987 219/300 Write block 250 len 12288 data: 3 3 3
13988 220/300 Write block 4474 len 4096 data: 4
13989 221/300 Write block 1863 len 16384 data: 5 5 6 5
13990 222/300 Read block 4714 len 16384
13991 223/300 Read block 3635 len 28672
13992 224/300 Read block 1505 len 36864
13993 225/300 Read block 2911 len 28672
13994 226/300 Read block 3227 len 24576
13995 227/300 Write block 1816 len 4096 data: 5
13996 228/300 Write block 1845 len 28672 data: 4 4 5 5 5 5 5
13997 229/300 Read block 605 len 28672
13998 230/300 Flush
13999 231/300 Flush
14000 232/300 Flush
14001 233/300 Read block 218 len 20480
14002 234/300 Write block 914 len 32768 data: 4 4 4 5 5 3 3 3
14003 235/300 Write block 458 len 40960 data: 3 4 3 5 6 6 6 5 4 6
14004 236/300 Write block 746 len 12288 data: 5 5 5
14005 237/300 Flush
14006 238/300 Write block 4931 len 4096 data: 4
14007 239/300 Read block 3233 len 12288
14008 240/300 Write block 4110 len 8192 data: 4 3
14009 241/300 Write block 2800 len 16384 data: 3 3 3 2
14010 242/300 Read block 708 len 16384
14011 243/300 Write block 2711 len 12288 data: 5 6 6
14012 244/300 Read block 845 len 32768
14013 245/300 Read block 1524 len 28672
14014 246/300 Read block 1613 len 8192
14015 247/300 Read block 2327 len 32768
14016 248/300 Flush
14017 249/300 Flush
14018 250/300 Write block 676 len 8192 data: 3 3
14019 251/300 Write block 4287 len 12288 data: 2 2 2
14020 252/300 Read block 2149 len 36864
14021 253/300 Write block 1650 len 20480 data: 4 4 3 3 2
14022 254/300 Read block 4668 len 24576
14023 255/300 Read block 4173 len 8192
14024 256/300 Read block 1895 len 40960
14025 257/300 Write block 2245 len 24576 data: 3 3 3 4 4 6
14026 258/300 Write block 3708 len 16384 data: 4 4 4 5
14027 259/300 Write block 2351 len 40960 data: 4 5 6 7 7 6 5 5 4 3
14028 260/300 Read block 3349 len 16384
14029 261/300 Write block 4769 len 28672 data: 3 4 3 3 2 3 3
14030 262/300 Read block 2993 len 20480
14031 263/300 Flush
14032 264/300 Read block 2961 len 20480
14033 265/300 Write block 4665 len 12288 data: 2 2 2
14034 266/300 Read block 1399 len 16384
14035 267/300 Read block 764 len 40960
14036 268/300 Write block 4267 len 20480 data: 2 4 5 5 5
14037 269/300 Read block 3253 len 40960
14038 270/300 Read block 520 len 40960
14039 271/300 Read block 2162 len 36864
14040 272/300 Read block 4392 len 12288
14041 273/300 Write block 774 len 8192 data: 5 4
14042 274/300 Read block 2549 len 24576
14043 275/300 Write block 1332 len 16384 data: 5 5 4 4
14044 276/300 Read block 1595 len 20480
14045 277/300 Write block 3031 len 40960 data: 5 5 5 6 5 5 4 4 5 5
14046 278/300 Flush
14047 279/300 Write block 2860 len 4096 data: 2
14048 280/300 Read block 4683 len 28672
14049 281/300 Read block 3192 len 40960
14050 282/300 Write block 1789 len 36864 data: 3 2 3 3 3 3 4 4 4
14051 283/300 Read block 2906 len 8192
14052 284/300 Read block 2684 len 36864
14053 285/300 Write block 1265 len 4096 data: 4
14054 286/300 Write block 739 len 40960 data: 6 5 3 3 4 4 3 6 6 6
14055 287/300 Read block 173 len 24576
14056 288/300 Read block 3807 len 8192
14057 289/300 Read block 1539 len 8192
14058 290/300 Read block 237 len 12288
14059 291/300 Write block 4165 len 8192 data: 2 2
14060 292/300 Write block 4528 len 40960 data: 2 3 2 2 2 3 3 3 3 3
14061 293/300 Write block 2902 len 24576 data: 5 4 4 2 3 4
14062 294/300 Read block 4510 len 36864
14063 295/300 Read block 1446 len 16384
14064 296/300 Read block 2658 len 4096
14065 297/300 Read block 4494 len 36864
14066 298/300 Write block 3013 len 24576 data: 2 2 2 2 2 2
14067 299/300 Flush
14068 300/300 Read block 4756 len 28672
14069 301/300 Write block 3396 len 28672 data: 3 3 4 3 3 4 4
14070 Main task has work to do, go find it
14071 got dsc Start(1)
14072 start 1
14073 Jan 21 15:50:08.561 INFO request completed, latency_us: 255, response_code: 204, uri: /start/cid/1, method: POST, req_id: 0838e9ad-f826-4bc3-9df1-727756002df1, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
14074 Replay: started 1, returned:Ok(())
14075 ----------------------------------------------------------------
14076 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14077 States: Active Offline Active
14078 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14079 4623 Acked Read 6 Done Sent Done false
14080 4624 Acked Read 8 Done Sent Done false
14081 4625 Acked Write 4 Done Sent Done false
14082 4626 Acked Read 8 Done Sent Done false
14083 4627 Acked Read 8 Done Sent Done false
14084 4628 Acked Read 10 Done Sent Done false
14085 4629 Acked Write 7 Done Sent Done false
14086 4630 Acked Read 6 Done Sent Done false
14087 4631 Acked Write 10 Done Sent Done false
14088 4632 Acked Write 6 Done Sent Done false
14089 4633 Acked Read 2 Done Sent Done false
14090 4634 Acked Flush 0 Done Sent Done false
14091 4635 Acked Read 5 Done Sent Done false
14092 4636 Acked Write 7 Done Sent Done false
14093 4637 Acked Read 2 Done Sent Done false
14094 4638 Acked Read 3 Done Sent Done false
14095 4639 Acked Write 9 Done Sent Done false
14096 4640 Acked Read 1 Done Sent Done false
14097 4641 Acked Read 3 Done Sent Done false
14098 4642 Acked Write 4 Done Sent Done false
14099 4643 Acked Write 3 Done Sent Done false
14100 4644 Acked Read 5 Done Sent Done false
14101 4645 Acked Read 10 Done Sent Done false
14102 4646 Acked Read 7 Done Sent Done false
14103 4647 Acked Write 4 Done Sent Done false
14104 4648 Acked Write 10 Done Sent Done false
14105 4649 Acked Read 5 Done Sent Done false
14106 4650 Acked Read 9 Done Sent Done false
14107 4651 Acked Read 6 Done Sent Done false
14108 4652 Acked Write 10 Done Sent Done false
14109 4653 Acked Write 2 Done Sent Done false
14110 4654 Acked Read 4 Done Sent Done false
14111 4655 Acked Read 4 Done Sent Done false
14112 4656 Acked Read 3 Done Sent Done false
14113 4657 Acked Write 2 Done Sent Done false
14114 4658 Acked Read 6 Done Sent Done false
14115 4659 Acked Write 7 Done Sent Done false
14116 4660 Acked Write 3 Done Sent Done false
14117 4661 Acked Write 5 Done Sent Done false
14118 4662 Acked Read 7 Done Sent Done false
14119 4663 Acked Write 7 Done Sent Done false
14120 4664 Acked Write 8 Done Sent Done false
14121 4665 Acked Read 5 Done Sent Done false
14122 4666 Acked Write 10 Done Sent Done false
14123 4667 Acked Write 6 Done Sent Done false
14124 4668 Acked Read 8 Done Sent Done false
14125 4669 Acked Read 3 Done Sent Done false
14126 4670 Acked Flush 0 Done Sent Done false
14127 4671 Acked Read 10 Done Sent Done false
14128 4672 Acked Write 9 Done Sent Done false
14129 4673 Acked Read 2 Done Sent Done false
14130 4674 Acked Write 4 Done Sent Done false
14131 4675 Acked Read 4 Done Sent Done false
14132 4676 Acked Read 9 Done Sent Done false
14133 4677 Acked Read 6 Done Sent Done false
14134 4678 Acked Read 2 Done Sent Done false
14135 4679 Acked Write 4 Done Sent Done false
14136 4680 Acked Read 8 Done Sent Done false
14137 4681 Acked Read 3 Done Sent Done false
14138 4682 Acked Read 5 Done Sent Done false
14139 4683 Acked Read 10 Done Sent Done false
14140 4684 Acked Read 6 Done Sent Done false
14141 4685 Acked Read 1 Done Sent Done false
14142 4686 Acked Read 4 Done Sent Done false
14143 4687 Acked Flush 0 Done Sent Done false
14144 4688 Acked Read 6 Done Sent Done false
14145 4689 Acked Read 9 Done Sent Done false
14146 4690 Acked Flush 0 Done Sent Done false
14147 4691 Acked Read 9 Done Sent Done false
14148 4692 Acked Read 1 Done Sent Done false
14149 4693 Acked Write 3 Done Sent Done false
14150 4694 Acked Write 3 Done Sent Done false
14151 4695 Acked Flush 0 Done Sent Done false
14152 4696 Acked Read 10 Done Sent Done false
14153 4697 Acked Read 9 Done Sent Done false
14154 4698 Acked Write 4 Done Sent Done false
14155 4699 Acked Read 7 Done Sent Done false
14156 4700 Acked Read 1 Done Sent Done false
14157 4701 Acked Read 8 Done Sent Done false
14158 4702 Acked Read 2 Done Sent Done false
14159 4703 Acked Read 6 Done Sent Done false
14160 4704 Acked Write 1 Done Sent Done false
14161 4705 Acked Read 2 Done Sent Done false
14162 4706 Acked Read 4 Done Sent Done false
14163 4707 Acked Read 10 Done Sent Done false
14164 4708 Acked Read 10 Done Sent Done false
14165 4709 Acked Read 4 Done Sent Done false
14166 4710 Acked Write 9 Done Sent Done false
14167 4711 Acked Read 9 Done Sent Done false
14168 4712 Acked Flush 0 Done Sent Done false
14169 4713 Acked Flush 0 Done Sent Done false
14170 4714 Acked Read 4 Done Sent Done false
14171 4715 Acked Read 10 Done Sent Done false
14172 4716 Acked Flush 0 Done Sent Done false
14173 4717 Acked Read 1 Done Sent Done false
14174 4718 Acked Read 9 Done Sent Done false
14175 4719 Acked Read 4 Done Sent Done false
14176 4720 Acked Read 7 Done Sent Done false
14177 4721 Acked Flush 0 Done Sent Done false
14178 4722 Acked Write 4 Done Sent Done false
14179 4723 Acked Read 5 Done Sent Done false
14180 4724 Acked Read 8 Done Sent Done false
14181 4725 Acked Read 8 Done Sent Done false
14182 4726 Acked Read 2 Done Sent Done false
14183 4727 Acked Read 5 Done Sent Done false
14184 4728 Acked Write 6 Done Sent Done false
14185 4729 Acked Flush 0 Done Sent Done false
14186 4730 Acked Write 4 Done Sent Done false
14187 4731 Acked Read 3 Done Sent Done false
14188 4732 Acked Flush 0 Done Sent Done false
14189 4733 Acked Read 6 Done Sent Done false
14190 4734 Acked Write 10 Done Sent Done false
14191 4735 Acked Read 8 Done Sent Done false
14192 4736 Acked Read 1 Done Sent Done false
14193 4737 Acked Flush 0 Done Sent Done false
14194 4738 Acked Write 8 Done Sent Done false
14195 4739 Acked Write 3 Done Sent Done false
14196 4740 Acked Write 1 Done Sent Done false
14197 4741 Acked Write 8 Done Sent Done false
14198 4742 Acked Read 10 Done Sent Done false
14199 4743 Acked Write 10 Done Sent Done false
14200 4744 Acked Read 10 Done Sent Done false
14201 4745 Acked Write 10 Done Sent Done false
14202 4746 Acked Read 3 Done Sent Done false
14203 4747 Acked Flush 0 Done Sent Done false
14204 4748 Acked Read 9 Done Sent Done false
14205 4749 Acked Read 7 Done Sent Done false
14206 4750 Acked Write 2 Done Sent Done false
14207 4751 Acked Write 5 Done Sent Done false
14208 4752 Acked Write 9 Done Sent Done false
14209 4753 Acked Read 8 Done Sent Done false
14210 4754 Acked Read 1 Done Sent Done false
14211 4755 Acked Read 8 Done Sent Done false
14212 4756 Acked Read 8 Done Sent Done false
14213 4757 Acked Read 8 Done Sent Done false
14214 4758 Acked Write 10 Done Sent Done false
14215 4759 Acked Read 10 Done Sent Done false
14216 4760 Acked Read 5 Done Sent Done false
14217 4761 Acked Write 6 Done Sent Done false
14218 4762 Acked Read 3 Done Sent Done false
14219 4763 Acked Write 9 Done Sent Done false
14220 4764 Acked Read 2 Done Sent Done false
14221 4765 Acked Flush 0 Done Sent Done false
14222 4766 Acked Read 10 Done Sent Done false
14223 4767 Acked Read 1 Done Sent Done false
14224 4768 Acked Write 4 Done Sent Done false
14225 4769 Acked Flush 0 Done Sent Done false
14226 4770 Acked Write 1 Done Sent Done false
14227 4771 Acked Read 2 Done Sent Done false
14228 4772 Acked Read 2 Done Sent Done false
14229 4773 Acked Write 8 Done Sent Done false
14230 4774 Acked Flush 0 Done Sent Done false
14231 4775 Acked Flush 0 Done Sent Done false
14232 4776 Acked Read 10 Done Sent Done false
14233 4777 Acked Read 10 Done Sent Done false
14234 4778 Acked Write 4 Done Sent Done false
14235 4779 Acked Read 4 Done Sent Done false
14236 4780 Acked Flush 0 Done Sent Done false
14237 4781 Acked Flush 0 Done Sent Done false
14238 4782 Acked Read 10 Done Sent Done false
14239 4783 Acked Flush 0 Done Sent Done false
14240 4784 Acked Read 3 Done Sent Done false
14241 4785 Acked Write 10 Done Sent Done false
14242 4786 Acked Write 9 Done Sent Done false
14243 4787 Acked Read 1 Done Sent Done false
14244 4788 Acked Read 2 Done Sent Done false
14245 4789 Acked Write 2 Done Sent Done false
14246 4790 Acked Write 4 Done Sent Done false
14247 4791 Acked Read 1 Done Sent Done false
14248 4792 Acked Flush 0 Done Sent Done false
14249 4793 Acked Write 5 Done Sent Done false
14250 4794 Acked Read 8 Done Sent Done false
14251 4795 Acked Write 3 Done Sent Done false
14252 4796 Acked Read 10 Done Sent Done false
14253 4797 Acked Read 10 Done Sent Done false
14254 4798 Acked Flush 0 Done Sent Done false
14255 4799 Acked Read 2 Done Sent Done false
14256 4800 Acked Read 8 Done Sent Done false
14257 4801 Acked Write 2 Done Sent Done false
14258 4802 Acked Read 10 Done Sent Done false
14259 4803 Acked Read 6 Done Sent Done false
14260 4804 Acked Read 3 Done Sent Done false
14261 4805 Acked Write 6 Done Sent Done false
14262 4806 Acked Write 3 Done Sent Done false
14263 4807 Acked Read 2 Done Sent Done false
14264 4808 Acked Read 8 Done Sent Done false
14265 4809 Acked Write 8 Done Sent Done false
14266 4810 Acked Read 6 Done Sent Done false
14267 4811 Acked Read 6 Done Sent Done false
14268 4812 Acked Read 4 Done Sent Done false
14269 4813 Acked Read 1 Done Sent Done false
14270 4814 Acked Read 6 Done Sent Done false
14271 4815 Acked Write 9 Done Sent Done false
14272 4816 Acked Flush 0 Done Sent Done false
14273 4817 Acked Read 1 Done Sent Done false
14274 4818 Acked Read 7 Done Sent Done false
14275 4819 Acked Read 1 Done Sent Done false
14276 4820 Acked Read 6 Done Sent Done false
14277 4821 Acked Flush 0 Done Sent Done false
14278 4822 Acked Read 3 Done Sent Done false
14279 4823 Acked Flush 0 Done Sent Done false
14280 4824 Acked Write 7 Done Sent Done false
14281 4825 Acked Flush 0 Done Sent Done false
14282 4826 Acked Write 2 Done Sent Done false
14283 4827 Acked Read 1 Done Sent Done false
14284 4828 Acked Read 1 Done Sent Done false
14285 4829 Acked Write 9 Done Sent Done false
14286 4830 Acked Read 1 Done Sent Done false
14287 4831 Acked Write 4 Done Sent Done false
14288 4832 Acked Read 6 Done Sent Done false
14289 4833 Acked Read 2 Done Sent Done false
14290 4834 Acked Write 1 Done Sent Done false
14291 4835 Acked Read 9 Done Sent Done false
14292 4836 Acked Write 1 Done Sent Done false
14293 4837 Acked Read 3 Done Sent Done false
14294 4838 Acked Read 10 Done Sent Done false
14295 4839 Acked Write 8 Done Sent Done false
14296 4840 Acked Read 6 Done Sent Done false
14297 4841 Acked Write 3 Done Sent Done false
14298 4842 Acked Write 1 Done Sent Done false
14299 4843 Acked Write 4 Done Sent Done false
14300 4844 Acked Read 4 Done Sent Done false
14301 4845 Acked Read 7 Done Sent Done false
14302 4846 Acked Read 9 Done Sent Done false
14303 4847 Acked Read 7 Done Sent Done false
14304 4848 Acked Read 6 Done Sent Done false
14305 4849 Acked Write 1 Done Sent Done false
14306 4850 Acked Write 7 Done Sent Done false
14307 4851 Acked Read 7 Done Sent Done false
14308 4852 Acked Flush 0 Done Sent Done false
14309 4853 Acked Flush 0 Done Sent Done false
14310 4854 Acked Flush 0 Done Sent Done false
14311 4855 Acked Read 5 Done Sent Done false
14312 4856 Acked Write 8 Done Sent Done false
14313 4857 Acked Write 10 Done Sent Done false
14314 4858 Acked Write 3 Done Sent Done false
14315 4859 Acked Flush 0 Done Sent Done false
14316 4860 Acked Write 1 Done Sent Done false
14317 4861 Acked Read 3 Done Sent Done false
14318 4862 Acked Write 2 Done Sent Done false
14319 4863 Acked Write 4 Done Sent Done false
14320 4864 Acked Read 4 Done Sent Done false
14321 4865 Acked Write 3 Done Sent Done false
14322 4866 Acked Read 8 Done Sent Done false
14323 4867 Acked Read 7 Done Sent Done false
14324 4868 Acked Read 2 Done Sent Done false
14325 4869 Acked Read 8 Done Sent Done false
14326 4870 Acked Flush 0 Done Sent Done false
14327 4871 Acked Flush 0 Done Sent Done false
14328 4872 Acked Write 2 Done Sent Done false
14329 4873 Acked Write 3 Done Sent Done false
14330 4874 Acked Read 9 Done Sent Done false
14331 4875 Acked Write 5 Done Sent Done false
14332 4876 Acked Read 6 Done Sent Done false
14333 4877 Acked Read 2 Done Sent Done false
14334 4878 Acked Read 10 Done Sent Done false
14335 4879 Acked Write 6 Done Sent Done false
14336 4880 Acked Write 4 Done Sent Done false
14337 4881 Acked Write 10 Done Sent Done false
14338 4882 Acked Read 4 Done Sent Done false
14339 4883 Acked Write 7 Done Sent Done false
14340 4884 Acked Read 5 Done Sent Done false
14341 4885 Acked Flush 0 Done Sent Done false
14342 4886 Acked Read 5 Done Sent Done false
14343 4887 Acked Write 3 Done Sent Done false
14344 4888 Acked Read 4 Done Sent Done false
14345 4889 Acked Read 10 Done Sent Done false
14346 4890 Acked Write 5 Done Sent Done false
14347 4891 Acked Read 10 Done Sent Done false
14348 4892 Acked Read 10 Done Sent Done false
14349 4893 Acked Read 9 Done Sent Done false
14350 4894 Acked Read 3 Done Sent Done false
14351 4895 Acked Write 2 Done Sent Done false
14352 4896 Acked Read 6 Done Sent Done false
14353 4897 Acked Write 4 Done Sent Done false
14354 4898 Acked Read 5 Done Sent Done false
14355 4899 Acked Write 10 Done Sent Done false
14356 4900 Acked Flush 0 Done Sent Done false
14357 4901 Acked Write 1 Done Sent Done false
14358 4902 Acked Read 7 Done Sent Done false
14359 4903 Acked Read 10 Done Sent Done false
14360 4904 Acked Write 9 Done Sent Done false
14361 4905 Acked Read 2 Done Sent Done false
14362 4906 Acked Read 9 Done Sent Done false
14363 4907 Acked Write 1 Done Sent Done false
14364 4908 Acked Write 10 Done Sent Done false
14365 4909 Acked Read 6 Done Sent Done false
14366 4910 Acked Read 2 Done Sent Done false
14367 4911 Acked Read 2 Done Sent Done false
14368 4912 Acked Read 3 Done Sent Done false
14369 4913 Acked Write 2 Done Sent Done false
14370 4914 Acked Write 10 Done Sent Done false
14371 4915 Acked Write 6 Done Sent Done false
14372 4916 Acked Read 9 Done Sent Done false
14373 4917 Acked Read 4 Done Sent Done false
14374 4918 Acked Read 1 Done Sent Done false
14375 4919 Acked Read 9 Done Sent Done false
14376 4920 Acked Write 6 Done Sent Done false
14377 4921 Acked Flush 0 Done Sent Done false
14378 4922 Acked Read 7 Done Sent Done false
14379 4923 Acked Write 7 Done Sent Done false
14380 STATES DS:0 DS:1 DS:2 TOTAL
14381 Sent 0 301 0 301
14382 Done 301 0 301 602
14383 Skipped 0 0 0 0
14384 Error 0 0 0 0
14385 Last Flush: 4921 4622 4921
14386 Downstairs last five completed: 4622 4621 4620 4619 4618
14387 Upstairs last five completed: 4923 4922 4921 4920 4919
14388 CLIENT: Up:0 ds:301 act:2
14389 [8820] got start action
14390 [8820] I am going to restart
14391 Starting downstairs at port 8820
14392 Make output file at "/tmp/dsc/downstairs-8820.txt"
14393 [8820][1] reports Starting
14394 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1213)
14395 [8820][1] reports Running
14396 ----------------------------------------------------------------
14397 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14398 States: Active Offline Active
14399 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14400 4623 Acked Read 6 Done Sent Done false
14401 4624 Acked Read 8 Done Sent Done false
14402 4625 Acked Write 4 Done Sent Done false
14403 4626 Acked Read 8 Done Sent Done false
14404 4627 Acked Read 8 Done Sent Done false
14405 4628 Acked Read 10 Done Sent Done false
14406 4629 Acked Write 7 Done Sent Done false
14407 4630 Acked Read 6 Done Sent Done false
14408 4631 Acked Write 10 Done Sent Done false
14409 4632 Acked Write 6 Done Sent Done false
14410 4633 Acked Read 2 Done Sent Done false
14411 4634 Acked Flush 0 Done Sent Done false
14412 4635 Acked Read 5 Done Sent Done false
14413 4636 Acked Write 7 Done Sent Done false
14414 4637 Acked Read 2 Done Sent Done false
14415 4638 Acked Read 3 Done Sent Done false
14416 4639 Acked Write 9 Done Sent Done false
14417 4640 Acked Read 1 Done Sent Done false
14418 4641 Acked Read 3 Done Sent Done false
14419 4642 Acked Write 4 Done Sent Done false
14420 4643 Acked Write 3 Done Sent Done false
14421 4644 Acked Read 5 Done Sent Done false
14422 4645 Acked Read 10 Done Sent Done false
14423 4646 Acked Read 7 Done Sent Done false
14424 4647 Acked Write 4 Done Sent Done false
14425 4648 Acked Write 10 Done Sent Done false
14426 4649 Acked Read 5 Done Sent Done false
14427 4650 Acked Read 9 Done Sent Done false
14428 4651 Acked Read 6 Done Sent Done false
14429 4652 Acked Write 10 Done Sent Done false
14430 4653 Acked Write 2 Done Sent Done false
14431 4654 Acked Read 4 Done Sent Done false
14432 4655 Acked Read 4 Done Sent Done false
14433 4656 Acked Read 3 Done Sent Done false
14434 4657 Acked Write 2 Done Sent Done false
14435 4658 Acked Read 6 Done Sent Done false
14436 4659 Acked Write 7 Done Sent Done false
14437 4660 Acked Write 3 Done Sent Done false
14438 4661 Acked Write 5 Done Sent Done false
14439 4662 Acked Read 7 Done Sent Done false
14440 4663 Acked Write 7 Done Sent Done false
14441 4664 Acked Write 8 Done Sent Done false
14442 4665 Acked Read 5 Done Sent Done false
14443 4666 Acked Write 10 Done Sent Done false
14444 4667 Acked Write 6 Done Sent Done false
14445 4668 Acked Read 8 Done Sent Done false
14446 4669 Acked Read 3 Done Sent Done false
14447 4670 Acked Flush 0 Done Sent Done false
14448 4671 Acked Read 10 Done Sent Done false
14449 4672 Acked Write 9 Done Sent Done false
14450 4673 Acked Read 2 Done Sent Done false
14451 4674 Acked Write 4 Done Sent Done false
14452 4675 Acked Read 4 Done Sent Done false
14453 4676 Acked Read 9 Done Sent Done false
14454 4677 Acked Read 6 Done Sent Done false
14455 4678 Acked Read 2 Done Sent Done false
14456 4679 Acked Write 4 Done Sent Done false
14457 4680 Acked Read 8 Done Sent Done false
14458 4681 Acked Read 3 Done Sent Done false
14459 4682 Acked Read 5 Done Sent Done false
14460 4683 Acked Read 10 Done Sent Done false
14461 4684 Acked Read 6 Done Sent Done false
14462 4685 Acked Read 1 Done Sent Done false
14463 4686 Acked Read 4 Done Sent Done false
14464 4687 Acked Flush 0 Done Sent Done false
14465 4688 Acked Read 6 Done Sent Done false
14466 4689 Acked Read 9 Done Sent Done false
14467 4690 Acked Flush 0 Done Sent Done false
14468 4691 Acked Read 9 Done Sent Done false
14469 4692 Acked Read 1 Done Sent Done false
14470 4693 Acked Write 3 Done Sent Done false
14471 4694 Acked Write 3 Done Sent Done false
14472 4695 Acked Flush 0 Done Sent Done false
14473 4696 Acked Read 10 Done Sent Done false
14474 4697 Acked Read 9 Done Sent Done false
14475 4698 Acked Write 4 Done Sent Done false
14476 4699 Acked Read 7 Done Sent Done false
14477 4700 Acked Read 1 Done Sent Done false
14478 4701 Acked Read 8 Done Sent Done false
14479 4702 Acked Read 2 Done Sent Done false
14480 4703 Acked Read 6 Done Sent Done false
14481 4704 Acked Write 1 Done Sent Done false
14482 4705 Acked Read 2 Done Sent Done false
14483 4706 Acked Read 4 Done Sent Done false
14484 4707 Acked Read 10 Done Sent Done false
14485 4708 Acked Read 10 Done Sent Done false
14486 4709 Acked Read 4 Done Sent Done false
14487 4710 Acked Write 9 Done Sent Done false
14488 4711 Acked Read 9 Done Sent Done false
14489 4712 Acked Flush 0 Done Sent Done false
14490 4713 Acked Flush 0 Done Sent Done false
14491 4714 Acked Read 4 Done Sent Done false
14492 4715 Acked Read 10 Done Sent Done false
14493 4716 Acked Flush 0 Done Sent Done false
14494 4717 Acked Read 1 Done Sent Done false
14495 4718 Acked Read 9 Done Sent Done false
14496 4719 Acked Read 4 Done Sent Done false
14497 4720 Acked Read 7 Done Sent Done false
14498 4721 Acked Flush 0 Done Sent Done false
14499 4722 Acked Write 4 Done Sent Done false
14500 4723 Acked Read 5 Done Sent Done false
14501 4724 Acked Read 8 Done Sent Done false
14502 4725 Acked Read 8 Done Sent Done false
14503 4726 Acked Read 2 Done Sent Done false
14504 4727 Acked Read 5 Done Sent Done false
14505 4728 Acked Write 6 Done Sent Done false
14506 4729 Acked Flush 0 Done Sent Done false
14507 4730 Acked Write 4 Done Sent Done false
14508 4731 Acked Read 3 Done Sent Done false
14509 4732 Acked Flush 0 Done Sent Done false
14510 4733 Acked Read 6 Done Sent Done false
14511 4734 Acked Write 10 Done Sent Done false
14512 4735 Acked Read 8 Done Sent Done false
14513 4736 Acked Read 1 Done Sent Done false
14514 4737 Acked Flush 0 Done Sent Done false
14515 4738 Acked Write 8 Done Sent Done false
14516 4739 Acked Write 3 Done Sent Done false
14517 4740 Acked Write 1 Done Sent Done false
14518 4741 Acked Write 8 Done Sent Done false
14519 4742 Acked Read 10 Done Sent Done false
14520 4743 Acked Write 10 Done Sent Done false
14521 4744 Acked Read 10 Done Sent Done false
14522 4745 Acked Write 10 Done Sent Done false
14523 4746 Acked Read 3 Done Sent Done false
14524 4747 Acked Flush 0 Done Sent Done false
14525 4748 Acked Read 9 Done Sent Done false
14526 4749 Acked Read 7 Done Sent Done false
14527 4750 Acked Write 2 Done Sent Done false
14528 4751 Acked Write 5 Done Sent Done false
14529 4752 Acked Write 9 Done Sent Done false
14530 4753 Acked Read 8 Done Sent Done false
14531 4754 Acked Read 1 Done Sent Done false
14532 4755 Acked Read 8 Done Sent Done false
14533 4756 Acked Read 8 Done Sent Done false
14534 4757 Acked Read 8 Done Sent Done false
14535 4758 Acked Write 10 Done Sent Done false
14536 4759 Acked Read 10 Done Sent Done false
14537 4760 Acked Read 5 Done Sent Done false
14538 4761 Acked Write 6 Done Sent Done false
14539 4762 Acked Read 3 Done Sent Done false
14540 4763 Acked Write 9 Done Sent Done false
14541 4764 Acked Read 2 Done Sent Done false
14542 4765 Acked Flush 0 Done Sent Done false
14543 4766 Acked Read 10 Done Sent Done false
14544 4767 Acked Read 1 Done Sent Done false
14545 4768 Acked Write 4 Done Sent Done false
14546 4769 Acked Flush 0 Done Sent Done false
14547 4770 Acked Write 1 Done Sent Done false
14548 4771 Acked Read 2 Done Sent Done false
14549 4772 Acked Read 2 Done Sent Done false
14550 4773 Acked Write 8 Done Sent Done false
14551 4774 Acked Flush 0 Done Sent Done false
14552 4775 Acked Flush 0 Done Sent Done false
14553 4776 Acked Read 10 Done Sent Done false
14554 4777 Acked Read 10 Done Sent Done false
14555 4778 Acked Write 4 Done Sent Done false
14556 4779 Acked Read 4 Done Sent Done false
14557 4780 Acked Flush 0 Done Sent Done false
14558 4781 Acked Flush 0 Done Sent Done false
14559 4782 Acked Read 10 Done Sent Done false
14560 4783 Acked Flush 0 Done Sent Done false
14561 4784 Acked Read 3 Done Sent Done false
14562 4785 Acked Write 10 Done Sent Done false
14563 4786 Acked Write 9 Done Sent Done false
14564 4787 Acked Read 1 Done Sent Done false
14565 4788 Acked Read 2 Done Sent Done false
14566 4789 Acked Write 2 Done Sent Done false
14567 4790 Acked Write 4 Done Sent Done false
14568 4791 Acked Read 1 Done Sent Done false
14569 4792 Acked Flush 0 Done Sent Done false
14570 4793 Acked Write 5 Done Sent Done false
14571 4794 Acked Read 8 Done Sent Done false
14572 4795 Acked Write 3 Done Sent Done false
14573 4796 Acked Read 10 Done Sent Done false
14574 4797 Acked Read 10 Done Sent Done false
14575 4798 Acked Flush 0 Done Sent Done false
14576 4799 Acked Read 2 Done Sent Done false
14577 4800 Acked Read 8 Done Sent Done false
14578 4801 Acked Write 2 Done Sent Done false
14579 4802 Acked Read 10 Done Sent Done false
14580 4803 Acked Read 6 Done Sent Done false
14581 4804 Acked Read 3 Done Sent Done false
14582 4805 Acked Write 6 Done Sent Done false
14583 4806 Acked Write 3 Done Sent Done false
14584 4807 Acked Read 2 Done Sent Done false
14585 4808 Acked Read 8 Done Sent Done false
14586 4809 Acked Write 8 Done Sent Done false
14587 4810 Acked Read 6 Done Sent Done false
14588 4811 Acked Read 6 Done Sent Done false
14589 4812 Acked Read 4 Done Sent Done false
14590 4813 Acked Read 1 Done Sent Done false
14591 4814 Acked Read 6 Done Sent Done false
14592 4815 Acked Write 9 Done Sent Done false
14593 4816 Acked Flush 0 Done Sent Done false
14594 4817 Acked Read 1 Done Sent Done false
14595 4818 Acked Read 7 Done Sent Done false
14596 4819 Acked Read 1 Done Sent Done false
14597 4820 Acked Read 6 Done Sent Done false
14598 4821 Acked Flush 0 Done Sent Done false
14599 4822 Acked Read 3 Done Sent Done false
14600 4823 Acked Flush 0 Done Sent Done false
14601 4824 Acked Write 7 Done Sent Done false
14602 4825 Acked Flush 0 Done Sent Done false
14603 4826 Acked Write 2 Done Sent Done false
14604 4827 Acked Read 1 Done Sent Done false
14605 4828 Acked Read 1 Done Sent Done false
14606 4829 Acked Write 9 Done Sent Done false
14607 4830 Acked Read 1 Done Sent Done false
14608 4831 Acked Write 4 Done Sent Done false
14609 4832 Acked Read 6 Done Sent Done false
14610 4833 Acked Read 2 Done Sent Done false
14611 4834 Acked Write 1 Done Sent Done false
14612 4835 Acked Read 9 Done Sent Done false
14613 4836 Acked Write 1 Done Sent Done false
14614 4837 Acked Read 3 Done Sent Done false
14615 4838 Acked Read 10 Done Sent Done false
14616 4839 Acked Write 8 Done Sent Done false
14617 4840 Acked Read 6 Done Sent Done false
14618 4841 Acked Write 3 Done Sent Done false
14619 4842 Acked Write 1 Done Sent Done false
14620 4843 Acked Write 4 Done Sent Done false
14621 4844 Acked Read 4 Done Sent Done false
14622 4845 Acked Read 7 Done Sent Done false
14623 4846 Acked Read 9 Done Sent Done false
14624 4847 Acked Read 7 Done Sent Done false
14625 4848 Acked Read 6 Done Sent Done false
14626 4849 Acked Write 1 Done Sent Done false
14627 4850 Acked Write 7 Done Sent Done false
14628 4851 Acked Read 7 Done Sent Done false
14629 4852 Acked Flush 0 Done Sent Done false
14630 4853 Acked Flush 0 Done Sent Done false
14631 4854 Acked Flush 0 Done Sent Done false
14632 4855 Acked Read 5 Done Sent Done false
14633 4856 Acked Write 8 Done Sent Done false
14634 4857 Acked Write 10 Done Sent Done false
14635 4858 Acked Write 3 Done Sent Done false
14636 4859 Acked Flush 0 Done Sent Done false
14637 4860 Acked Write 1 Done Sent Done false
14638 4861 Acked Read 3 Done Sent Done false
14639 4862 Acked Write 2 Done Sent Done false
14640 4863 Acked Write 4 Done Sent Done false
14641 4864 Acked Read 4 Done Sent Done false
14642 4865 Acked Write 3 Done Sent Done false
14643 4866 Acked Read 8 Done Sent Done false
14644 4867 Acked Read 7 Done Sent Done false
14645 4868 Acked Read 2 Done Sent Done false
14646 4869 Acked Read 8 Done Sent Done false
14647 4870 Acked Flush 0 Done Sent Done false
14648 4871 Acked Flush 0 Done Sent Done false
14649 4872 Acked Write 2 Done Sent Done false
14650 4873 Acked Write 3 Done Sent Done false
14651 4874 Acked Read 9 Done Sent Done false
14652 4875 Acked Write 5 Done Sent Done false
14653 4876 Acked Read 6 Done Sent Done false
14654 4877 Acked Read 2 Done Sent Done false
14655 4878 Acked Read 10 Done Sent Done false
14656 4879 Acked Write 6 Done Sent Done false
14657 4880 Acked Write 4 Done Sent Done false
14658 4881 Acked Write 10 Done Sent Done false
14659 4882 Acked Read 4 Done Sent Done false
14660 4883 Acked Write 7 Done Sent Done false
14661 4884 Acked Read 5 Done Sent Done false
14662 4885 Acked Flush 0 Done Sent Done false
14663 4886 Acked Read 5 Done Sent Done false
14664 4887 Acked Write 3 Done Sent Done false
14665 4888 Acked Read 4 Done Sent Done false
14666 4889 Acked Read 10 Done Sent Done false
14667 4890 Acked Write 5 Done Sent Done false
14668 4891 Acked Read 10 Done Sent Done false
14669 4892 Acked Read 10 Done Sent Done false
14670 4893 Acked Read 9 Done Sent Done false
14671 4894 Acked Read 3 Done Sent Done false
14672 4895 Acked Write 2 Done Sent Done false
14673 4896 Acked Read 6 Done Sent Done false
14674 4897 Acked Write 4 Done Sent Done false
14675 4898 Acked Read 5 Done Sent Done false
14676 4899 Acked Write 10 Done Sent Done false
14677 4900 Acked Flush 0 Done Sent Done false
14678 4901 Acked Write 1 Done Sent Done false
14679 4902 Acked Read 7 Done Sent Done false
14680 4903 Acked Read 10 Done Sent Done false
14681 4904 Acked Write 9 Done Sent Done false
14682 4905 Acked Read 2 Done Sent Done false
14683 4906 Acked Read 9 Done Sent Done false
14684 4907 Acked Write 1 Done Sent Done false
14685 4908 Acked Write 10 Done Sent Done false
14686 4909 Acked Read 6 Done Sent Done false
14687 4910 Acked Read 2 Done Sent Done false
14688 4911 Acked Read 2 Done Sent Done false
14689 4912 Acked Read 3 Done Sent Done false
14690 4913 Acked Write 2 Done Sent Done false
14691 4914 Acked Write 10 Done Sent Done false
14692 4915 Acked Write 6 Done Sent Done false
14693 4916 Acked Read 9 Done Sent Done false
14694 4917 Acked Read 4 Done Sent Done false
14695 4918 Acked Read 1 Done Sent Done false
14696 4919 Acked Read 9 Done Sent Done false
14697 4920 Acked Write 6 Done Sent Done false
14698 4921 Acked Flush 0 Done Sent Done false
14699 4922 Acked Read 7 Done Sent Done false
14700 4923 Acked Write 7 Done Sent Done false
14701 4924 Acked Flush 0 Done Sent Done false
14702 STATES DS:0 DS:1 DS:2 TOTAL
14703 Sent 0 302 0 302
14704 Done 302 0 302 604
14705 Skipped 0 0 0 0
14706 Error 0 0 0 0
14707 Last Flush: 4924 4622 4924
14708 Downstairs last five completed: 4622 4621 4620 4619 4618
14709 Upstairs last five completed: 4924 4923 4922 4921 4920
14710 CLIENT: Up:0 ds:302 act:2
147112025-01-21T15:50:14.307ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147122025-01-21T15:50:14.307ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40526) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147132025-01-21T15:50:14.308ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147142025-01-21T15:50:14.309ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147152025-01-21T15:50:14.309ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147162025-01-21T15:50:14.309ZINFOcrucible: send last flush ID to this DS: 4622 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147172025-01-21T15:50:14.309ZINFOcrucible: Replied this last flush ID: 4622 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147182025-01-21T15:50:14.309ZINFOcrucible: [1] client re-new 302 jobs since flush 4622 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147192025-01-21T15:50:14.309ZINFOcrucible: [1] Replayed 302 jobs since flush: 4622 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
14720 ----------------------------------------------------------------
14721 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14722 Downstairs last five completed: 4924 4923 4922 4921 4920
14723 Upstairs last five completed: 4924 4923 4922 4921 4920
14724 CLIENT: Up:0 ds:0 act:3
14725 Replay: All jobs finished
14726 Main task has work to do, go find it
14727 got dsc Stop(2)
14728 stop 2
14729 Jan 21 15:50:16.566 INFO request completed, latency_us: 345, [8830] Got stop action so:false kr:false
14730 response_code: 204, uri: /stop/cid/2, method: POST, req_id: 617a94d5-a28c-45d3-a15d-16fb56ccddf3, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
147312025-01-21T15:50:16.567ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
14732 Jan 21 15:50:16.567 INFO request completed, latency_us: 143, response_code: 200, uri: /state/cid/2, method: GET, req_id: fe0837d4-2a25-44de-8282-d075463c87e6, remote_addr: 127.0.0.1:52945, local_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:50:16.567229589Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
14733 .0.0.1:9998
147342025-01-21T15:50:16.567ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
147352025-01-21T15:50:16.567ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
14736 [8830] Exited with: ExitStatus(unix_wait_status(9))
14737 [8830][2] reports Exit
14738 Jan 21 15:50:20.570 INFO request completed, latency_us: 220, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8a528ae7-394b-4934-8be3-622582b319f3, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
14739 001/300 Write block 2014 len 20480 data: 4 5 5 5 6
14740 002/300 Write block 373 len 40960 data: 5 5 4 3 3 2 2 3 4 4
14741 003/300 Write block 1841 len 28672 data: 5 5 6 6 5 5 6
14742 004/300 Write block 4607 len 24576 data: 4 5 5 3 3 3
14743 005/300 Write block 1071 len 12288 data: 6 6 7
14744 006/300 Read block 1095 len 4096
14745 007/300 Write block 751 len 8192 data: 3 3
14746 008/300 Flush
14747 009/300 Write block 1324 len 32768 data: 7 8 7 5 4 4 5 6
14748 010/300 Write block 2651 len 4096 data: 3
14749 011/300 Write block 4200 len 16384 data: 2 2 2 3
14750 012/300 Write block 2038 len 36864 data: 4 4 4 4 5 5 5 5 5
14751 013/300 Read block 3242 len 24576
14752 014/300 Write block 2530 len 24576 data: 4 4 4 4 5 5
14753 015/300 Read block 3478 len 12288
14754 016/300 Read block 4010 len 12288
14755 017/300 Flush
14756 018/300 Write block 3201 len 16384 data: 5 4 4 4
14757 019/300 Write block 309 len 12288 data: 3 3 3
14758 020/300 Read block 4181 len 20480
14759 021/300 Write block 1983 len 16384 data: 7 8 8 8
14760 022/300 Write block 607 len 12288 data: 4 4 5
14761 023/300 Write block 3928 len 36864 data: 4 4 4 3 3 3 3 2 2
14762 024/300 Write block 1011 len 16384 data: 5 5 4 4
14763 025/300 Read block 2128 len 4096
14764 026/300 Read block 4677 len 12288
14765 027/300 Write block 2305 len 4096 data: 4
14766 028/300 Read block 3209 len 8192
14767 029/300 Read block 4785 len 16384
14768 030/300 Flush
14769 031/300 Write block 2647 len 28672 data: 4 4 4 4 4 3 2
14770 032/300 Flush
14771 033/300 Read block 3214 len 8192
14772 034/300 Read block 3527 len 24576
14773 035/300 Write block 3483 len 20480 data: 4 3 2 2 3
14774 036/300 Write block 2614 len 36864 data: 4 3 3 3 2 3 4 5 6
14775 037/300 Read block 4525 len 8192
14776 038/300 Write block 54 len 36864 data: 4 3 3 3 3 3 4 4 4
14777 039/300 Write block 2837 len 12288 data: 3 3 5
14778 040/300 Write block 2934 len 32768 data: 4 2 2 2 2 4 3 3
14779 041/300 Read block 942 len 12288
14780 042/300 Read block 2809 len 40960
14781 043/300 Read block 1173 len 4096
14782 044/300 Write block 1053 len 8192 data: 5 5
14783 045/300 Read block 1749 len 4096
14784 046/300 Write block 1076 len 16384 data: 4 4 5 4
14785 047/300 Flush
14786 048/300 Read block 46 len 32768
14787 049/300 Read block 4177 len 28672
14788 050/300 Read block 3585 len 16384
14789 051/300 Read block 3686 len 32768
14790 052/300 Read block 1805 len 4096
14791 053/300 Write block 3472 len 32768 data: 4 4 4 6 7 8 5 5
14792 054/300 Write block 1758 len 24576 data: 6 4 4 4 4 4
14793 055/300 Write block 215 len 16384 data: 7 7 7 7
14794 056/300 Read block 4613 len 16384
14795 057/300 Read block 651 len 8192
14796 058/300 Write block 2856 len 32768 data: 3 3 4 3 3 2 2 2
14797 059/300 Write block 3369 len 16384 data: 6 4 4 4
14798 060/300 Write block 4457 len 32768 data: 3 2 2 2 2 2 2 2
14799 061/300 Read block 1760 len 20480
14800 062/300 Write block 3201 len 36864 data: 6 5 5 5 4 3 3 3 3
14801 063/300 Read block 3362 len 28672
14802 064/300 Read block 4591 len 32768
14803 065/300 Write block 710 len 40960 data: 3 3 3 4 3 4 4 4 4 4
14804 066/300 Read block 911 len 12288
14805 067/300 Read block 3924 len 24576
14806 068/300 Flush
14807 069/300 Write block 3759 len 28672 data: 4 3 3 3 3 4 4
14808 070/300 Write block 3443 len 16384 data: 5 5 5 3
14809 071/300 Read block 4223 len 24576
14810 072/300 Write block 529 len 12288 data: 7 7 5
14811 073/300 Read block 1251 len 28672
14812 074/300 Read block 4621 len 20480
14813 075/300 Read block 256 len 16384
14814 076/300 Read block 4411 len 40960
14815 077/300 Write block 4065 len 4096 data: 2
14816 078/300 Read block 4810 len 12288
14817 079/300 Read block 4270 len 36864
14818 080/300 Write block 3482 len 20480 data: 5 5 4 3 3
14819 081/300 Write block 4168 len 20480 data: 3 3 4 4 4
14820 082/300 Read block 2683 len 8192
14821 083/300 Write block 144 len 8192 data: 3 3
14822 084/300 Flush
14823 085/300 Read block 1390 len 28672
14824 086/300 Read block 4343 len 40960
14825 087/300 Read block 1609 len 28672
14826 088/300 Read block 1099 len 8192
14827 089/300 Write block 428 len 12288 data: 3 3 4
14828 090/300 Read block 3451 len 36864
14829 091/300 Flush
14830 092/300 Write block 4567 len 16384 data: 3 3 4 5
14831 093/300 Read block 2506 len 28672
14832 094/300 Flush
14833 095/300 Read block 2008 len 24576
14834 096/300 Write block 2597 len 32768 data: 3 3 3 3 5 5 5 7
14835 097/300 Read block 361 len 32768
14836 098/300 Flush
14837 099/300 Read block 4225 len 4096
14838 100/300 Write block 357 len 4096 data: 4
14839 101/300 Read block 1893 len 16384
14840 102/300 Write block 2162 len 40960 data: 2 2 4 4 5 6 6 8 7 8
14841 103/300 Read block 4724 len 8192
14842 104/300 Write block 2628 len 24576 data: 6 7 6 5 5 5
14843 105/300 Read block 3114 len 24576
14844 106/300 Write block 4810 len 28672 data: 4 3 3 3 4 4 4
14845 107/300 Write block 561 len 36864 data: 2 2 2 2 2 2 2 2 2
14846 108/300 Write block 3020 len 32768 data: 2 2 2 2 3 4 4 4
14847 109/300 Read block 2145 len 16384
14848 110/300 Read block 1252 len 4096
14849 111/300 Write block 1668 len 32768 data: 3 3 3 3 3 3 3 3
14850 112/300 Read block 900 len 12288
14851 113/300 Write block 3280 len 32768 data: 3 3 3 4 3 3 2 2
14852 114/300 Write block 614 len 8192 data: 4 4
14853 115/300 Write block 973 len 24576 data: 3 3 3 2 2 2
14854 116/300 Flush
14855 117/300 Read block 3488 len 20480
14856 118/300 Flush
14857 119/300 Write block 3418 len 12288 data: 6 6 6
14858 120/300 Read block 3886 len 4096
14859 121/300 Write block 1353 len 4096 data: 5
14860 122/300 Flush
14861 123/300 Write block 4343 len 16384 data: 5 3 4 4
14862 124/300 Write block 3982 len 36864 data: 3 3 5 6 6 5 5 2 2
14863 125/300 Write block 3049 len 16384 data: 3 3 3 2
14864 126/300 Read block 3334 len 20480
14865 127/300 Write block 1090 len 28672 data: 5 4 4 4 4 4 5
14866 128/300 Write block 1574 len 40960 data: 5 5 6 6 6 6 6 6 5 5
14867 129/300 Flush
14868 130/300 Write block 1453 len 4096 data: 4
14869 131/300 Read block 2706 len 24576
14870 132/300 Flush
14871 133/300 Read block 3620 len 16384
14872 134/300 Write block 2841 len 16384 data: 4 4 3 3
14873 135/300 Flush
14874 136/300 Read block 1439 len 36864
14875 137/300 Write block 2342 len 32768 data: 2 2 3 2 2 3 4 4
14876 138/300 Read block 2531 len 12288
14877 139/300 Read block 1290 len 12288
14878 140/300 Flush
14879 141/300 Read block 4848 len 24576
14880 142/300 Read block 3143 len 4096
14881 143/300 Read block 2139 len 32768
14882 144/300 Read block 1318 len 12288
14883 145/300 Read block 3551 len 4096
14884 146/300 Read block 401 len 24576
14885 147/300 Read block 2814 len 28672
14886 148/300 Write block 3411 len 28672 data: 2 2 2 3 3 4 6
14887 149/300 Read block 3751 len 16384
14888 150/300 Read block 402 len 28672
14889 151/300 Read block 13 len 32768
14890 152/300 Write block 2881 len 28672 data: 3 3 3 3 3 4 4
14891 153/300 Write block 1560 len 12288 data: 4 4 5
14892 154/300 Write block 2556 len 32768 data: 4 4 3 3 3 3 2 2
14893 155/300 Write block 4440 len 32768 data: 5 4 6 5 5 3 5 5
14894 156/300 Flush
14895 157/300 Flush
14896 158/300 Read block 1801 len 16384
14897 159/300 Read block 100 len 32768
14898 160/300 Read block 3529 len 32768
14899 161/300 Read block 2880 len 4096
14900 162/300 Write block 1519 len 8192 data: 3 2
14901 163/300 Flush
14902 164/300 Read block 1537 len 12288
14903 165/300 Read block 1412 len 8192
14904 166/300 Read block 1055 len 16384
14905 167/300 Read block 1995 len 16384
14906 168/300 Read block 1749 len 8192
14907 169/300 Read block 598 len 20480
14908 170/300 Read block 2029 len 20480
14909 171/300 Write block 981 len 40960 data: 2 3 4 4 4 3 3 4 4 3
14910 172/300 Flush
14911 173/300 Read block 1813 len 16384
14912 174/300 Write block 2759 len 40960 data: 4 3 3 3 3 3 3 3 2 2
14913 175/300 Read block 4764 len 28672
14914 176/300 Write block 1367 len 32768 data: 5 5 4 4 4 3 3 3
14915 177/300 Read block 4525 len 28672
14916 178/300 Write block 2486 len 8192 data: 2 2
14917 179/300 Read block 1343 len 20480
14918 180/300 Write block 479 len 8192 data: 3 4
14919 181/300 Read block 328 len 24576
14920 182/300 Write block 1518 len 4096 data: 4
14921 183/300 Read block 185 len 40960
14922 184/300 Write block 2494 len 24576 data: 4 4 4 3 4 4
14923 185/300 Read block 3380 len 40960
14924 186/300 Read block 2647 len 4096
14925 187/300 Write block 4908 len 40960 data: 3 4 5 5 4 4 4 4 3 2
14926 188/300 Write block 3230 len 32768 data: 5 5 5 3 3 2 2 2
14927 189/300 Read block 4511 len 32768
14928 190/300 Read block 2249 len 24576
14929 191/300 Read block 4612 len 20480
14930 192/300 Read block 770 len 24576
14931 193/300 Read block 4248 len 4096
14932 194/300 Write block 3398 len 32768 data: 5 4 4 5 5 4 4 4
14933 195/300 Read block 606 len 8192
14934 196/300 Flush
14935 197/300 Read block 4544 len 36864
14936 198/300 Write block 4121 len 16384 data: 5 5 4 4
14937 199/300 Read block 3494 len 4096
14938 200/300 Read block 1252 len 8192
14939 201/300 Write block 2631 len 36864 data: 6 6 6 6 6 6 5 4 4
14940 202/300 Read block 2330 len 28672
14941 203/300 Write block 4371 len 8192 data: 2 2
14942 204/300 Read block 3199 len 12288
14943 205/300 Read block 2978 len 32768
14944 206/300 Read block 2224 len 8192
14945 207/300 Read block 4878 len 12288
14946 208/300 Read block 1300 len 28672
14947 209/300 Read block 137 len 24576
14948 210/300 Read block 3431 len 12288
14949 211/300 Write block 783 len 12288 data: 2 2 2
14950 212/300 Write block 269 len 8192 data: 3 4
14951 213/300 Read block 1716 len 24576
14952 214/300 Write block 1776 len 8192 data: 3 4
14953 215/300 Flush
14954 216/300 Write block 2697 len 8192 data: 4 4
14955 217/300 Write block 480 len 24576 data: 5 4 4 6 6 7
14956 218/300 Write block 2979 len 4096 data: 4
14957 219/300 Read block 2637 len 32768
14958 220/300 Write block 4600 len 4096 data: 2
14959 221/300 Write block 4532 len 8192 data: 3 4
14960 222/300 Write block 1076 len 12288 data: 5 5 6
14961 223/300 Write block 2077 len 40960 data: 6 6 7 6 6 6 4 3 3 3
14962 224/300 Write block 4718 len 20480 data: 4 4 7 7 7
14963 225/300 Write block 1392 len 4096 data: 2
14964 226/300 Read block 3441 len 28672
14965 227/300 Write block 2370 len 4096 data: 4
14966 228/300 Write block 1776 len 12288 data: 4 5 5
14967 229/300 Read block 1341 len 4096
14968 230/300 Write block 4067 len 40960 data: 2 2 2 2 3 3 3 3 2 2
14969 231/300 Read block 393 len 28672
14970 232/300 Write block 1907 len 20480 data: 8 8 8 5 6
14971 233/300 Write block 813 len 16384 data: 3 3 3 2
14972 234/300 Write block 3895 len 16384 data: 5 4 3 2
14973 235/300 Read block 4851 len 12288
14974 236/300 Write block 2352 len 12288 data: 6 7 8
14975 237/300 Read block 719 len 36864
14976 238/300 Write block 1877 len 16384 data: 4 4 4 4
14977 239/300 Write block 2097 len 40960 data: 4 4 4 3 3 3 2 2 3 4
14978 240/300 Read block 3658 len 8192
14979 241/300 Write block 1372 len 12288 data: 4 4 4
14980 242/300 Write block 3936 len 4096 data: 3
14981 243/300 Write block 4733 len 36864 data: 4 3 2 2 2 2 3 3 3
14982 244/300 Write block 3180 len 4096 data: 4
14983 245/300 Read block 375 len 28672
14984 246/300 Write block 3617 len 40960 data: 2 2 2 2 3 4 4 5 5 5
14985 247/300 Read block 2106 len 8192
14986 248/300 Write block 186 len 36864 data: 7 8 7 6 5 4 4 4 5
14987 249/300 Read block 3465 len 16384
14988 250/300 Read block 4341 len 20480
14989 251/300 Read block 2475 len 4096
14990 252/300 Flush
14991 253/300 Read block 1329 len 8192
14992 254/300 Flush
14993 255/300 Read block 3284 len 4096
14994 256/300 Read block 3968 len 20480
14995 257/300 Write block 999 len 4096 data: 5
14996 258/300 Write block 3885 len 32768 data: 3 4 4 5 5 6 6 6
14997 259/300 Read block 3214 len 4096
14998 260/300 Read block 1220 len 28672
14999 261/300 Write block 4491 len 36864 data: 4 4 5 5 4 4 3 3 3
15000 262/300 Write block 2408 len 8192 data: 2 2
15001 263/300 Read block 956 len 36864
15002 264/300 Write block 3944 len 4096 data: 4
15003 265/300 Read block 2118 len 8192
15004 266/300 Write block 772 len 20480 data: 5 5 6 5 4
15005 267/300 Read block 4462 len 12288
15006 268/300 Read block 680 len 32768
15007 269/300 Write block 2580 len 12288 data: 4 4 3
15008 270/300 Read block 768 len 28672
15009 271/300 Flush
15010 272/300 Read block 888 len 4096
15011 273/300 Read block 1296 len 32768
15012 274/300 Read block 1509 len 36864
15013 275/300 Write block 2929 len 24576 data: 5 5 4 4 4 5
15014 276/300 Read block 1945 len 4096
15015 277/300 Write block 2973 len 20480 data: 2 2 2 2 4
15016 278/300 Read block 2111 len 16384
15017 279/300 Write block 3245 len 28672 data: 5 5 5 6 4 3 3
15018 280/300 Write block 1067 len 40960 data: 4 4 4 6 7 7 8 6 4 6
15019 281/300 Write block 2968 len 12288 data: 2 2 2
15020 282/300 Write block 1373 len 8192 data: 5 5
15021 283/300 Write block 2983 len 32768 data: 3 4 4 4 3 3 3 3
15022 284/300 Write block 4185 len 20480 data: 6 5 5 4 5
15023 285/300 Write block 3838 len 8192 data: 4 4
15024 286/300 Read block 2422 len 4096
15025 287/300 Read block 1051 len 28672
15026 288/300 Read block 1626 len 8192
15027 289/300 Read block 3476 len 16384
15028 290/300 Read block 2422 len 4096
15029 291/300 Flush
15030 292/300 Write block 3513 len 28672 data: 2 2 2 2 3 3 6
15031 293/300 Read block 1305 len 20480
15032 294/300 Write block 4622 len 28672 data: 6 6 7 7 7 7 4
15033 295/300 Write block 4307 len 40960 data: 2 2 4 3 3 3 3 3 4 4
15034 296/300 Read block 606 len 24576
15035 297/300 Write block 1952 len 40960 data: 3 2 2 2 2 2 2 2 2 2
15036 298/300 Read block 1659 len 16384
15037 299/300 Flush
15038 300/300 Read block 4750 len 24576
15039 301/300 Write block 1771 len 16384 data: 5 5 4 3
15040 Main task has work to do, go find it
15041 got dsc Start(2)
15042 start 2
15043 Jan 21 15:50:20.792 INFO request completed, latency_us: 199, response_code: 204, uri: /start/cid/2, method: POST, req_id: f616e6be-50e1-4bf7-a7c9-62e853d6b4ad, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
15044 Replay: started 2, returned:Ok(())
15045 ----------------------------------------------------------------
15046 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
15047 States: Active Active Offline
15048 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15049 4925 Acked Write 5 Done Done Sent false
15050 4926 Acked Write 10 Done Done Sent false
15051 4927 Acked Write 7 Done Done Sent false
15052 4928 Acked Write 6 Done Done Sent false
15053 4929 Acked Write 3 Done Done Sent false
15054 4930 Acked Read 1 Done Done Sent false
15055 4931 Acked Write 2 Done Done Sent false
15056 4932 Acked Flush 0 Done Done Sent false
15057 4933 Acked Write 8 Done Done Sent false
15058 4934 Acked Write 1 Done Done Sent false
15059 4935 Acked Write 4 Done Done Sent false
15060 4936 Acked Write 9 Done Done Sent false
15061 4937 Acked Read 6 Done Done Sent false
15062 4938 Acked Write 6 Done Done Sent false
15063 4939 Acked Read 3 Done Done Sent false
15064 4940 Acked Read 3 Done Done Sent false
15065 4941 Acked Flush 0 Done Done Sent false
15066 4942 Acked Write 4 Done Done Sent false
15067 4943 Acked Write 3 Done Done Sent false
15068 4944 Acked Read 5 Done Done Sent false
15069 4945 Acked Write 4 Done Done Sent false
15070 4946 Acked Write 3 Done Done Sent false
15071 4947 Acked Write 9 Done Done Sent false
15072 4948 Acked Write 4 Done Done Sent false
15073 4949 Acked Read 1 Done Done Sent false
15074 4950 Acked Read 3 Done Done Sent false
15075 4951 Acked Write 1 Done Done Sent false
15076 4952 Acked Read 2 Done Done Sent false
15077 4953 Acked Read 4 Done Done Sent false
15078 4954 Acked Flush 0 Done Done Sent false
15079 4955 Acked Write 7 Done Done Sent false
15080 4956 Acked Flush 0 Done Done Sent false
15081 4957 Acked Read 2 Done Done Sent false
15082 4958 Acked Read 6 Done Done Sent false
15083 4959 Acked Write 5 Done Done Sent false
15084 4960 Acked Write 9 Done Done Sent false
15085 4961 Acked Read 2 Done Done Sent false
15086 4962 Acked Write 9 Done Done Sent false
15087 4963 Acked Write 3 Done Done Sent false
15088 4964 Acked Write 8 Done Done Sent false
15089 4965 Acked Read 3 Done Done Sent false
15090 4966 Acked Read 10 Done Done Sent false
15091 4967 Acked Read 1 Done Done Sent false
15092 4968 Acked Write 2 Done Done Sent false
15093 4969 Acked Read 1 Done Done Sent false
15094 4970 Acked Write 4 Done Done Sent false
15095 4971 Acked Flush 0 Done Done Sent false
15096 4972 Acked Read 8 Done Done Sent false
15097 4973 Acked Read 7 Done Done Sent false
15098 4974 Acked Read 4 Done Done Sent false
15099 4975 Acked Read 8 Done Done Sent false
15100 4976 Acked Read 1 Done Done Sent false
15101 4977 Acked Write 8 Done Done Sent false
15102 4978 Acked Write 6 Done Done Sent false
15103 4979 Acked Write 4 Done Done Sent false
15104 4980 Acked Read 4 Done Done Sent false
15105 4981 Acked Read 2 Done Done Sent false
15106 4982 Acked Write 8 Done Done Sent false
15107 4983 Acked Write 4 Done Done Sent false
15108 4984 Acked Write 8 Done Done Sent false
15109 4985 Acked Read 5 Done Done Sent false
15110 4986 Acked Write 9 Done Done Sent false
15111 4987 Acked Read 7 Done Done Sent false
15112 4988 Acked Read 8 Done Done Sent false
15113 4989 Acked Write 10 Done Done Sent false
15114 4990 Acked Read 3 Done Done Sent false
15115 4991 Acked Read 6 Done Done Sent false
15116 4992 Acked Flush 0 Done Done Sent false
15117 4993 Acked Write 7 Done Done Sent false
15118 4994 Acked Write 4 Done Done Sent false
15119 4995 Acked Read 6 Done Done Sent false
15120 4996 Acked Write 3 Done Done Sent false
15121 4997 Acked Read 7 Done Done Sent false
15122 4998 Acked Read 5 Done Done Sent false
15123 4999 Acked Read 4 Done Done Sent false
15124 5000 Acked Read 10 Done Done Sent false
15125 5001 Acked Write 1 Done Done Sent false
15126 5002 Acked Read 3 Done Done Sent false
15127 5003 Acked Read 9 Done Done Sent false
15128 5004 Acked Write 5 Done Done Sent false
15129 5005 Acked Write 5 Done Done Sent false
15130 5006 Acked Read 2 Done Done Sent false
15131 5007 Acked Write 2 Done Done Sent false
15132 5008 Acked Flush 0 Done Done Sent false
15133 5009 Acked Read 7 Done Done Sent false
15134 5010 Acked Read 10 Done Done Sent false
15135 5011 Acked Read 7 Done Done Sent false
15136 5012 Acked Read 2 Done Done Sent false
15137 5013 Acked Write 3 Done Done Sent false
15138 5014 Acked Read 9 Done Done Sent false
15139 5015 Acked Flush 0 Done Done Sent false
15140 5016 Acked Write 4 Done Done Sent false
15141 5017 Acked Read 7 Done Done Sent false
15142 5018 Acked Flush 0 Done Done Sent false
15143 5019 Acked Read 6 Done Done Sent false
15144 5020 Acked Write 8 Done Done Sent false
15145 5021 Acked Read 8 Done Done Sent false
15146 5022 Acked Flush 0 Done Done Sent false
15147 5023 Acked Read 1 Done Done Sent false
15148 5024 Acked Write 1 Done Done Sent false
15149 5025 Acked Read 4 Done Done Sent false
15150 5026 Acked Write 10 Done Done Sent false
15151 5027 Acked Read 2 Done Done Sent false
15152 5028 Acked Write 6 Done Done Sent false
15153 5029 Acked Read 6 Done Done Sent false
15154 5030 Acked Write 7 Done Done Sent false
15155 5031 Acked Write 9 Done Done Sent false
15156 5032 Acked Write 8 Done Done Sent false
15157 5033 Acked Read 4 Done Done Sent false
15158 5034 Acked Read 1 Done Done Sent false
15159 5035 Acked Write 8 Done Done Sent false
15160 5036 Acked Read 3 Done Done Sent false
15161 5037 Acked Write 8 Done Done Sent false
15162 5038 Acked Write 2 Done Done Sent false
15163 5039 Acked Write 6 Done Done Sent false
15164 5040 Acked Flush 0 Done Done Sent false
15165 5041 Acked Read 5 Done Done Sent false
15166 5042 Acked Flush 0 Done Done Sent false
15167 5043 Acked Write 3 Done Done Sent false
15168 5044 Acked Read 1 Done Done Sent false
15169 5045 Acked Write 1 Done Done Sent false
15170 5046 Acked Flush 0 Done Done Sent false
15171 5047 Acked Write 4 Done Done Sent false
15172 5048 Acked Write 9 Done Done Sent false
15173 5049 Acked Write 4 Done Done Sent false
15174 5050 Acked Read 5 Done Done Sent false
15175 5051 Acked Write 7 Done Done Sent false
15176 5052 Acked Write 10 Done Done Sent false
15177 5053 Acked Flush 0 Done Done Sent false
15178 5054 Acked Write 1 Done Done Sent false
15179 5055 Acked Read 6 Done Done Sent false
15180 5056 Acked Flush 0 Done Done Sent false
15181 5057 Acked Read 4 Done Done Sent false
15182 5058 Acked Write 4 Done Done Sent false
15183 5059 Acked Flush 0 Done Done Sent false
15184 5060 Acked Read 9 Done Done Sent false
15185 5061 Acked Write 8 Done Done Sent false
15186 5062 Acked Read 3 Done Done Sent false
15187 5063 Acked Read 3 Done Done Sent false
15188 5064 Acked Flush 0 Done Done Sent false
15189 5065 Acked Read 6 Done Done Sent false
15190 5066 Acked Read 1 Done Done Sent false
15191 5067 Acked Read 8 Done Done Sent false
15192 5068 Acked Read 3 Done Done Sent false
15193 5069 Acked Read 1 Done Done Sent false
15194 5070 Acked Read 6 Done Done Sent false
15195 5071 Acked Read 7 Done Done Sent false
15196 5072 Acked Write 7 Done Done Sent false
15197 5073 Acked Read 4 Done Done Sent false
15198 5074 Acked Read 7 Done Done Sent false
15199 5075 Acked Read 8 Done Done Sent false
15200 5076 Acked Write 7 Done Done Sent false
15201 5077 Acked Write 3 Done Done Sent false
15202 5078 Acked Write 8 Done Done Sent false
15203 5079 Acked Write 8 Done Done Sent false
15204 5080 Acked Flush 0 Done Done Sent false
15205 5081 Acked Flush 0 Done Done Sent false
15206 5082 Acked Read 4 Done Done Sent false
15207 5083 Acked Read 8 Done Done Sent false
15208 5084 Acked Read 8 Done Done Sent false
15209 5085 Acked Read 1 Done Done Sent false
15210 5086 Acked Write 2 Done Done Sent false
15211 5087 Acked Flush 0 Done Done Sent false
15212 5088 Acked Read 3 Done Done Sent false
15213 5089 Acked Read 2 Done Done Sent false
15214 5090 Acked Read 4 Done Done Sent false
15215 5091 Acked Read 4 Done Done Sent false
15216 5092 Acked Read 2 Done Done Sent false
15217 5093 Acked Read 5 Done Done Sent false
15218 5094 Acked Read 5 Done Done Sent false
15219 5095 Acked Write 10 Done Done Sent false
15220 5096 Acked Flush 0 Done Done Sent false
15221 5097 Acked Read 4 Done Done Sent false
15222 5098 Acked Write 10 Done Done Sent false
15223 5099 Acked Read 7 Done Done Sent false
15224 5100 Acked Write 8 Done Done Sent false
15225 5101 Acked Read 7 Done Done Sent false
15226 5102 Acked Write 2 Done Done Sent false
15227 5103 Acked Read 5 Done Done Sent false
15228 5104 Acked Write 2 Done Done Sent false
15229 5105 Acked Read 6 Done Done Sent false
15230 5106 Acked Write 1 Done Done Sent false
15231 5107 Acked Read 10 Done Done Sent false
15232 5108 Acked Write 6 Done Done Sent false
15233 5109 Acked Read 10 Done Done Sent false
15234 5110 Acked Read 1 Done Done Sent false
15235 5111 Acked Write 10 Done Done Sent false
15236 5112 Acked Write 8 Done Done Sent false
15237 5113 Acked Read 8 Done Done Sent false
15238 5114 Acked Read 6 Done Done Sent false
15239 5115 Acked Read 5 Done Done Sent false
15240 5116 Acked Read 6 Done Done Sent false
15241 5117 Acked Read 1 Done Done Sent false
15242 5118 Acked Write 8 Done Done Sent false
15243 5119 Acked Read 2 Done Done Sent false
15244 5120 Acked Flush 0 Done Done Sent false
15245 5121 Acked Read 9 Done Done Sent false
15246 5122 Acked Write 4 Done Done Sent false
15247 5123 Acked Read 1 Done Done Sent false
15248 5124 Acked Read 2 Done Done Sent false
15249 5125 Acked Write 9 Done Done Sent false
15250 5126 Acked Read 7 Done Done Sent false
15251 5127 Acked Write 2 Done Done Sent false
15252 5128 Acked Read 3 Done Done Sent false
15253 5129 Acked Read 8 Done Done Sent false
15254 5130 Acked Read 2 Done Done Sent false
15255 5131 Acked Read 3 Done Done Sent false
15256 5132 Acked Read 7 Done Done Sent false
15257 5133 Acked Read 6 Done Done Sent false
15258 5134 Acked Read 3 Done Done Sent false
15259 5135 Acked Write 3 Done Done Sent false
15260 5136 Acked Write 2 Done Done Sent false
15261 5137 Acked Read 6 Done Done Sent false
15262 5138 Acked Write 2 Done Done Sent false
15263 5139 Acked Flush 0 Done Done Sent false
15264 5140 Acked Write 2 Done Done Sent false
15265 5141 Acked Write 6 Done Done Sent false
15266 5142 Acked Write 1 Done Done Sent false
15267 5143 Acked Read 8 Done Done Sent false
15268 5144 Acked Write 1 Done Done Sent false
15269 5145 Acked Write 2 Done Done Sent false
15270 5146 Acked Write 3 Done Done Sent false
15271 5147 Acked Write 10 Done Done Sent false
15272 5148 Acked Write 5 Done Done Sent false
15273 5149 Acked Write 1 Done Done Sent false
15274 5150 Acked Read 7 Done Done Sent false
15275 5151 Acked Write 1 Done Done Sent false
15276 5152 Acked Write 3 Done Done Sent false
15277 5153 Acked Read 1 Done Done Sent false
15278 5154 Acked Write 10 Done Done Sent false
15279 5155 Acked Read 7 Done Done Sent false
15280 5156 Acked Write 5 Done Done Sent false
15281 5157 Acked Write 4 Done Done Sent false
15282 5158 Acked Write 4 Done Done Sent false
15283 5159 Acked Read 3 Done Done Sent false
15284 5160 Acked Write 3 Done Done Sent false
15285 5161 Acked Read 9 Done Done Sent false
15286 5162 Acked Write 4 Done Done Sent false
15287 5163 Acked Write 10 Done Done Sent false
15288 5164 Acked Read 2 Done Done Sent false
15289 5165 Acked Write 3 Done Done Sent false
15290 5166 Acked Write 1 Done Done Sent false
15291 5167 Acked Write 9 Done Done Sent false
15292 5168 Acked Write 1 Done Done Sent false
15293 5169 Acked Read 7 Done Done Sent false
15294 5170 Acked Write 10 Done Done Sent false
15295 5171 Acked Read 2 Done Done Sent false
15296 5172 Acked Write 9 Done Done Sent false
15297 5173 Acked Read 4 Done Done Sent false
15298 5174 Acked Read 5 Done Done Sent false
15299 5175 Acked Read 1 Done Done Sent false
15300 5176 Acked Flush 0 Done Done Sent false
15301 5177 Acked Read 2 Done Done Sent false
15302 5178 Acked Flush 0 Done Done Sent false
15303 5179 Acked Read 1 Done Done Sent false
15304 5180 Acked Read 5 Done Done Sent false
15305 5181 Acked Write 1 Done Done Sent false
15306 5182 Acked Write 8 Done Done Sent false
15307 5183 Acked Read 1 Done Done Sent false
15308 5184 Acked Read 7 Done Done Sent false
15309 5185 Acked Write 9 Done Done Sent false
15310 5186 Acked Write 2 Done Done Sent false
15311 5187 Acked Read 9 Done Done Sent false
15312 5188 Acked Write 1 Done Done Sent false
15313 5189 Acked Read 2 Done Done Sent false
15314 5190 Acked Write 5 Done Done Sent false
15315 5191 Acked Read 3 Done Done Sent false
15316 5192 Acked Read 8 Done Done Sent false
15317 5193 Acked Write 3 Done Done Sent false
15318 5194 Acked Read 7 Done Done Sent false
15319 5195 Acked Flush 0 Done Done Sent false
15320 5196 Acked Read 1 Done Done Sent false
15321 5197 Acked Read 8 Done Done Sent false
15322 5198 Acked Read 9 Done Done Sent false
15323 5199 Acked Write 6 Done Done Sent false
15324 5200 Acked Read 1 Done Done Sent false
15325 5201 Acked Write 5 Done Done Sent false
15326 5202 Acked Read 4 Done Done Sent false
15327 5203 Acked Write 7 Done Done Sent false
15328 5204 Acked Write 10 Done Done Sent false
15329 5205 Acked Write 3 Done Done Sent false
15330 5206 Acked Write 2 Done Done Sent false
15331 5207 Acked Write 8 Done Done Sent false
15332 5208 Acked Write 5 Done Done Sent false
15333 5209 Acked Write 2 Done Done Sent false
15334 5210 Acked Read 1 Done Done Sent false
15335 5211 Acked Read 7 Done Done Sent false
15336 5212 Acked Read 2 Done Done Sent false
15337 5213 Acked Read 4 Done Done Sent false
15338 5214 Acked Read 1 Done Done Sent false
15339 5215 Acked Flush 0 Done Done Sent false
15340 5216 Acked Write 7 Done Done Sent false
15341 5217 Acked Read 5 Done Done Sent false
15342 5218 Acked Write 7 Done Done Sent false
15343 5219 Acked Write 10 Done Done Sent false
15344 5220 Acked Read 6 Done Done Sent false
15345 5221 Acked Write 10 Done Done Sent false
15346 5222 Acked Read 4 Done Done Sent false
15347 5223 Acked Flush 0 Done Done Sent false
15348 5224 Acked Read 6 Done Done Sent false
15349 5225 Acked Write 4 Done Done Sent false
15350 STATES DS:0 DS:1 DS:2 TOTAL
15351 Sent 0 0 301 301
15352 Done 301 301 0 602
15353 Skipped 0 0 0 0
15354 Error 0 0 0 0
15355 Last Flush: 5223 5223 4924
15356 Downstairs last five completed: 4924 4923 4922 4921 4920
15357 Upstairs last five completed: 5225 5224 5223 5222 5221
15358 CLIENT: Up:0 ds:301 act:2
15359 [8830] got start action
15360 [8830] I am going to restart
15361 Starting downstairs at port 8830
15362 Make output file at "/tmp/dsc/downstairs-8830.txt"
15363 [8830][2] reports Starting
15364 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1214)
15365 [8830][2] reports Running
15366 ----------------------------------------------------------------
15367 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15368 States: Active Active Offline
15369 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15370 4925 Acked Write 5 Done Done Sent false
15371 4926 Acked Write 10 Done Done Sent false
15372 4927 Acked Write 7 Done Done Sent false
15373 4928 Acked Write 6 Done Done Sent false
15374 4929 Acked Write 3 Done Done Sent false
15375 4930 Acked Read 1 Done Done Sent false
15376 4931 Acked Write 2 Done Done Sent false
15377 4932 Acked Flush 0 Done Done Sent false
15378 4933 Acked Write 8 Done Done Sent false
15379 4934 Acked Write 1 Done Done Sent false
15380 4935 Acked Write 4 Done Done Sent false
15381 4936 Acked Write 9 Done Done Sent false
15382 4937 Acked Read 6 Done Done Sent false
15383 4938 Acked Write 6 Done Done Sent false
15384 4939 Acked Read 3 Done Done Sent false
15385 4940 Acked Read 3 Done Done Sent false
15386 4941 Acked Flush 0 Done Done Sent false
15387 4942 Acked Write 4 Done Done Sent false
15388 4943 Acked Write 3 Done Done Sent false
15389 4944 Acked Read 5 Done Done Sent false
15390 4945 Acked Write 4 Done Done Sent false
15391 4946 Acked Write 3 Done Done Sent false
15392 4947 Acked Write 9 Done Done Sent false
15393 4948 Acked Write 4 Done Done Sent false
15394 4949 Acked Read 1 Done Done Sent false
15395 4950 Acked Read 3 Done Done Sent false
15396 4951 Acked Write 1 Done Done Sent false
15397 4952 Acked Read 2 Done Done Sent false
15398 4953 Acked Read 4 Done Done Sent false
15399 4954 Acked Flush 0 Done Done Sent false
15400 4955 Acked Write 7 Done Done Sent false
15401 4956 Acked Flush 0 Done Done Sent false
15402 4957 Acked Read 2 Done Done Sent false
15403 4958 Acked Read 6 Done Done Sent false
15404 4959 Acked Write 5 Done Done Sent false
15405 4960 Acked Write 9 Done Done Sent false
15406 4961 Acked Read 2 Done Done Sent false
15407 4962 Acked Write 9 Done Done Sent false
15408 4963 Acked Write 3 Done Done Sent false
15409 4964 Acked Write 8 Done Done Sent false
15410 4965 Acked Read 3 Done Done Sent false
15411 4966 Acked Read 10 Done Done Sent false
15412 4967 Acked Read 1 Done Done Sent false
15413 4968 Acked Write 2 Done Done Sent false
15414 4969 Acked Read 1 Done Done Sent false
15415 4970 Acked Write 4 Done Done Sent false
15416 4971 Acked Flush 0 Done Done Sent false
15417 4972 Acked Read 8 Done Done Sent false
15418 4973 Acked Read 7 Done Done Sent false
15419 4974 Acked Read 4 Done Done Sent false
15420 4975 Acked Read 8 Done Done Sent false
15421 4976 Acked Read 1 Done Done Sent false
15422 4977 Acked Write 8 Done Done Sent false
15423 4978 Acked Write 6 Done Done Sent false
15424 4979 Acked Write 4 Done Done Sent false
15425 4980 Acked Read 4 Done Done Sent false
15426 4981 Acked Read 2 Done Done Sent false
15427 4982 Acked Write 8 Done Done Sent false
15428 4983 Acked Write 4 Done Done Sent false
15429 4984 Acked Write 8 Done Done Sent false
15430 4985 Acked Read 5 Done Done Sent false
15431 4986 Acked Write 9 Done Done Sent false
15432 4987 Acked Read 7 Done Done Sent false
15433 4988 Acked Read 8 Done Done Sent false
15434 4989 Acked Write 10 Done Done Sent false
15435 4990 Acked Read 3 Done Done Sent false
15436 4991 Acked Read 6 Done Done Sent false
15437 4992 Acked Flush 0 Done Done Sent false
15438 4993 Acked Write 7 Done Done Sent false
15439 4994 Acked Write 4 Done Done Sent false
15440 4995 Acked Read 6 Done Done Sent false
15441 4996 Acked Write 3 Done Done Sent false
15442 4997 Acked Read 7 Done Done Sent false
15443 4998 Acked Read 5 Done Done Sent false
15444 4999 Acked Read 4 Done Done Sent false
15445 5000 Acked Read 10 Done Done Sent false
15446 5001 Acked Write 1 Done Done Sent false
15447 5002 Acked Read 3 Done Done Sent false
15448 5003 Acked Read 9 Done Done Sent false
15449 5004 Acked Write 5 Done Done Sent false
15450 5005 Acked Write 5 Done Done Sent false
15451 5006 Acked Read 2 Done Done Sent false
15452 5007 Acked Write 2 Done Done Sent false
15453 5008 Acked Flush 0 Done Done Sent false
15454 5009 Acked Read 7 Done Done Sent false
15455 5010 Acked Read 10 Done Done Sent false
15456 5011 Acked Read 7 Done Done Sent false
15457 5012 Acked Read 2 Done Done Sent false
15458 5013 Acked Write 3 Done Done Sent false
15459 5014 Acked Read 9 Done Done Sent false
15460 5015 Acked Flush 0 Done Done Sent false
15461 5016 Acked Write 4 Done Done Sent false
15462 5017 Acked Read 7 Done Done Sent false
15463 5018 Acked Flush 0 Done Done Sent false
15464 5019 Acked Read 6 Done Done Sent false
15465 5020 Acked Write 8 Done Done Sent false
15466 5021 Acked Read 8 Done Done Sent false
15467 5022 Acked Flush 0 Done Done Sent false
15468 5023 Acked Read 1 Done Done Sent false
15469 5024 Acked Write 1 Done Done Sent false
15470 5025 Acked Read 4 Done Done Sent false
15471 5026 Acked Write 10 Done Done Sent false
15472 5027 Acked Read 2 Done Done Sent false
15473 5028 Acked Write 6 Done Done Sent false
15474 5029 Acked Read 6 Done Done Sent false
15475 5030 Acked Write 7 Done Done Sent false
15476 5031 Acked Write 9 Done Done Sent false
15477 5032 Acked Write 8 Done Done Sent false
15478 5033 Acked Read 4 Done Done Sent false
15479 5034 Acked Read 1 Done Done Sent false
15480 5035 Acked Write 8 Done Done Sent false
15481 5036 Acked Read 3 Done Done Sent false
15482 5037 Acked Write 8 Done Done Sent false
15483 5038 Acked Write 2 Done Done Sent false
15484 5039 Acked Write 6 Done Done Sent false
15485 5040 Acked Flush 0 Done Done Sent false
15486 5041 Acked Read 5 Done Done Sent false
15487 5042 Acked Flush 0 Done Done Sent false
15488 5043 Acked Write 3 Done Done Sent false
15489 5044 Acked Read 1 Done Done Sent false
15490 5045 Acked Write 1 Done Done Sent false
15491 5046 Acked Flush 0 Done Done Sent false
15492 5047 Acked Write 4 Done Done Sent false
15493 5048 Acked Write 9 Done Done Sent false
15494 5049 Acked Write 4 Done Done Sent false
15495 5050 Acked Read 5 Done Done Sent false
15496 5051 Acked Write 7 Done Done Sent false
15497 5052 Acked Write 10 Done Done Sent false
15498 5053 Acked Flush 0 Done Done Sent false
15499 5054 Acked Write 1 Done Done Sent false
15500 5055 Acked Read 6 Done Done Sent false
15501 5056 Acked Flush 0 Done Done Sent false
15502 5057 Acked Read 4 Done Done Sent false
15503 5058 Acked Write 4 Done Done Sent false
15504 5059 Acked Flush 0 Done Done Sent false
15505 5060 Acked Read 9 Done Done Sent false
15506 5061 Acked Write 8 Done Done Sent false
15507 5062 Acked Read 3 Done Done Sent false
15508 5063 Acked Read 3 Done Done Sent false
15509 5064 Acked Flush 0 Done Done Sent false
15510 5065 Acked Read 6 Done Done Sent false
15511 5066 Acked Read 1 Done Done Sent false
15512 5067 Acked Read 8 Done Done Sent false
15513 5068 Acked Read 3 Done Done Sent false
15514 5069 Acked Read 1 Done Done Sent false
15515 5070 Acked Read 6 Done Done Sent false
15516 5071 Acked Read 7 Done Done Sent false
15517 5072 Acked Write 7 Done Done Sent false
15518 5073 Acked Read 4 Done Done Sent false
15519 5074 Acked Read 7 Done Done Sent false
15520 5075 Acked Read 8 Done Done Sent false
15521 5076 Acked Write 7 Done Done Sent false
15522 5077 Acked Write 3 Done Done Sent false
15523 5078 Acked Write 8 Done Done Sent false
15524 5079 Acked Write 8 Done Done Sent false
15525 5080 Acked Flush 0 Done Done Sent false
15526 5081 Acked Flush 0 Done Done Sent false
15527 5082 Acked Read 4 Done Done Sent false
15528 5083 Acked Read 8 Done Done Sent false
15529 5084 Acked Read 8 Done Done Sent false
15530 5085 Acked Read 1 Done Done Sent false
15531 5086 Acked Write 2 Done Done Sent false
15532 5087 Acked Flush 0 Done Done Sent false
15533 5088 Acked Read 3 Done Done Sent false
15534 5089 Acked Read 2 Done Done Sent false
15535 5090 Acked Read 4 Done Done Sent false
15536 5091 Acked Read 4 Done Done Sent false
15537 5092 Acked Read 2 Done Done Sent false
15538 5093 Acked Read 5 Done Done Sent false
15539 5094 Acked Read 5 Done Done Sent false
15540 5095 Acked Write 10 Done Done Sent false
15541 5096 Acked Flush 0 Done Done Sent false
15542 5097 Acked Read 4 Done Done Sent false
15543 5098 Acked Write 10 Done Done Sent false
15544 5099 Acked Read 7 Done Done Sent false
15545 5100 Acked Write 8 Done Done Sent false
15546 5101 Acked Read 7 Done Done Sent false
15547 5102 Acked Write 2 Done Done Sent false
15548 5103 Acked Read 5 Done Done Sent false
15549 5104 Acked Write 2 Done Done Sent false
15550 5105 Acked Read 6 Done Done Sent false
15551 5106 Acked Write 1 Done Done Sent false
15552 5107 Acked Read 10 Done Done Sent false
15553 5108 Acked Write 6 Done Done Sent false
15554 5109 Acked Read 10 Done Done Sent false
15555 5110 Acked Read 1 Done Done Sent false
15556 5111 Acked Write 10 Done Done Sent false
15557 5112 Acked Write 8 Done Done Sent false
15558 5113 Acked Read 8 Done Done Sent false
15559 5114 Acked Read 6 Done Done Sent false
15560 5115 Acked Read 5 Done Done Sent false
15561 5116 Acked Read 6 Done Done Sent false
15562 5117 Acked Read 1 Done Done Sent false
15563 5118 Acked Write 8 Done Done Sent false
15564 5119 Acked Read 2 Done Done Sent false
15565 5120 Acked Flush 0 Done Done Sent false
15566 5121 Acked Read 9 Done Done Sent false
15567 5122 Acked Write 4 Done Done Sent false
15568 5123 Acked Read 1 Done Done Sent false
15569 5124 Acked Read 2 Done Done Sent false
15570 5125 Acked Write 9 Done Done Sent false
15571 5126 Acked Read 7 Done Done Sent false
15572 5127 Acked Write 2 Done Done Sent false
15573 5128 Acked Read 3 Done Done Sent false
15574 5129 Acked Read 8 Done Done Sent false
15575 5130 Acked Read 2 Done Done Sent false
15576 5131 Acked Read 3 Done Done Sent false
15577 5132 Acked Read 7 Done Done Sent false
15578 5133 Acked Read 6 Done Done Sent false
15579 5134 Acked Read 3 Done Done Sent false
15580 5135 Acked Write 3 Done Done Sent false
15581 5136 Acked Write 2 Done Done Sent false
15582 5137 Acked Read 6 Done Done Sent false
15583 5138 Acked Write 2 Done Done Sent false
15584 5139 Acked Flush 0 Done Done Sent false
15585 5140 Acked Write 2 Done Done Sent false
15586 5141 Acked Write 6 Done Done Sent false
15587 5142 Acked Write 1 Done Done Sent false
15588 5143 Acked Read 8 Done Done Sent false
15589 5144 Acked Write 1 Done Done Sent false
15590 5145 Acked Write 2 Done Done Sent false
15591 5146 Acked Write 3 Done Done Sent false
15592 5147 Acked Write 10 Done Done Sent false
15593 5148 Acked Write 5 Done Done Sent false
15594 5149 Acked Write 1 Done Done Sent false
15595 5150 Acked Read 7 Done Done Sent false
15596 5151 Acked Write 1 Done Done Sent false
15597 5152 Acked Write 3 Done Done Sent false
15598 5153 Acked Read 1 Done Done Sent false
15599 5154 Acked Write 10 Done Done Sent false
15600 5155 Acked Read 7 Done Done Sent false
15601 5156 Acked Write 5 Done Done Sent false
15602 5157 Acked Write 4 Done Done Sent false
15603 5158 Acked Write 4 Done Done Sent false
15604 5159 Acked Read 3 Done Done Sent false
15605 5160 Acked Write 3 Done Done Sent false
15606 5161 Acked Read 9 Done Done Sent false
15607 5162 Acked Write 4 Done Done Sent false
15608 5163 Acked Write 10 Done Done Sent false
15609 5164 Acked Read 2 Done Done Sent false
15610 5165 Acked Write 3 Done Done Sent false
15611 5166 Acked Write 1 Done Done Sent false
15612 5167 Acked Write 9 Done Done Sent false
15613 5168 Acked Write 1 Done Done Sent false
15614 5169 Acked Read 7 Done Done Sent false
15615 5170 Acked Write 10 Done Done Sent false
15616 5171 Acked Read 2 Done Done Sent false
15617 5172 Acked Write 9 Done Done Sent false
15618 5173 Acked Read 4 Done Done Sent false
15619 5174 Acked Read 5 Done Done Sent false
15620 5175 Acked Read 1 Done Done Sent false
15621 5176 Acked Flush 0 Done Done Sent false
15622 5177 Acked Read 2 Done Done Sent false
15623 5178 Acked Flush 0 Done Done Sent false
15624 5179 Acked Read 1 Done Done Sent false
15625 5180 Acked Read 5 Done Done Sent false
15626 5181 Acked Write 1 Done Done Sent false
15627 5182 Acked Write 8 Done Done Sent false
15628 5183 Acked Read 1 Done Done Sent false
15629 5184 Acked Read 7 Done Done Sent false
15630 5185 Acked Write 9 Done Done Sent false
15631 5186 Acked Write 2 Done Done Sent false
15632 5187 Acked Read 9 Done Done Sent false
15633 5188 Acked Write 1 Done Done Sent false
15634 5189 Acked Read 2 Done Done Sent false
15635 5190 Acked Write 5 Done Done Sent false
15636 5191 Acked Read 3 Done Done Sent false
15637 5192 Acked Read 8 Done Done Sent false
15638 5193 Acked Write 3 Done Done Sent false
15639 5194 Acked Read 7 Done Done Sent false
15640 5195 Acked Flush 0 Done Done Sent false
15641 5196 Acked Read 1 Done Done Sent false
15642 5197 Acked Read 8 Done Done Sent false
15643 5198 Acked Read 9 Done Done Sent false
15644 5199 Acked Write 6 Done Done Sent false
15645 5200 Acked Read 1 Done Done Sent false
15646 5201 Acked Write 5 Done Done Sent false
15647 5202 Acked Read 4 Done Done Sent false
15648 5203 Acked Write 7 Done Done Sent false
15649 5204 Acked Write 10 Done Done Sent false
15650 5205 Acked Write 3 Done Done Sent false
15651 5206 Acked Write 2 Done Done Sent false
15652 5207 Acked Write 8 Done Done Sent false
15653 5208 Acked Write 5 Done Done Sent false
15654 5209 Acked Write 2 Done Done Sent false
15655 5210 Acked Read 1 Done Done Sent false
15656 5211 Acked Read 7 Done Done Sent false
15657 5212 Acked Read 2 Done Done Sent false
15658 5213 Acked Read 4 Done Done Sent false
15659 5214 Acked Read 1 Done Done Sent false
15660 5215 Acked Flush 0 Done Done Sent false
15661 5216 Acked Write 7 Done Done Sent false
15662 5217 Acked Read 5 Done Done Sent false
15663 5218 Acked Write 7 Done Done Sent false
15664 5219 Acked Write 10 Done Done Sent false
15665 5220 Acked Read 6 Done Done Sent false
15666 5221 Acked Write 10 Done Done Sent false
15667 5222 Acked Read 4 Done Done Sent false
15668 5223 Acked Flush 0 Done Done Sent false
15669 5224 Acked Read 6 Done Done Sent false
15670 5225 Acked Write 4 Done Done Sent false
15671 5226 Acked Flush 0 Done Done Sent false
15672 STATES DS:0 DS:1 DS:2 TOTAL
15673 Sent 0 0 302 302
15674 Done 302 302 0 604
15675 Skipped 0 0 0 0
15676 Error 0 0 0 0
15677 Last Flush: 5226 5226 4924
15678 Downstairs last five completed: 4924 4923 4922 4921 4920
15679 Upstairs last five completed: 5226 5225 5224 5223 5222
15680 CLIENT: Up:0 ds:302 act:2
156812025-01-21T15:50:26.568ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
156822025-01-21T15:50:26.568ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42246) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
156832025-01-21T15:50:26.569ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
156842025-01-21T15:50:26.570ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
156852025-01-21T15:50:26.570ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
156862025-01-21T15:50:26.570ZINFOcrucible: send last flush ID to this DS: 4924 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
156872025-01-21T15:50:26.570ZINFOcrucible: Replied this last flush ID: 4924 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
156882025-01-21T15:50:26.570ZINFOcrucible: [2] client re-new 302 jobs since flush 4924 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
156892025-01-21T15:50:26.570ZINFOcrucible: [2] Replayed 302 jobs since flush: 4924 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
15690 ----------------------------------------------------------------
15691 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15692 Downstairs last five completed: 5226 5225 5224 5223 5222
15693 Upstairs last five completed: 5226 5225 5224 5223 5222
15694 CLIENT: Up:0 ds:0 act:3
15695 Replay: All jobs finished
15696 Main task has work to do, go find it
15697 got dsc Stop(0)
15698 stop 0
15699 Jan 21 15:50:28.798 INFO request completed, latency_us: 377, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 8a5883d6-e68d-41d3-ab6b-cdae2f2550d4, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1[8810] Got stop action so:false kr:false
15700 :9998
157012025-01-21T15:50:28.799ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
157022025-01-21T15:50:28.799ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
15703 Jan 21 15:50:28.799 INFO request completed, {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:50:28.799644432Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
15704 latency_us: 123, response_code: 200, uri: /state/cid/0, method: GET, req_id: 0276c8f9-5b2d-4244-afc5-15811a38d0c4, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
157052025-01-21T15:50:28.799ZINFOcrucible: client task is exiting = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
15706 [8810] Exited with: ExitStatus(unix_wait_status(9))
15707 [8810][0] reports Exit
15708 Jan 21 15:50:32.802 INFO request completed, latency_us: 198, response_code: 200, uri: /state/cid/0, method: GET, req_id: 11722484-972a-478c-ad56-c57bea4b43d2, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
15709 001/300 Read block 4594 len 36864
15710 002/300 Read block 3985 len 4096
15711 003/300 Write block 4167 len 32768 data: 2 4 4 5 5 5 5 5
15712 004/300 Write block 625 len 20480 data: 4 5 5 3 3
15713 005/300 Read block 11 len 16384
15714 006/300 Write block 690 len 12288 data: 4 3 3
15715 007/300 Write block 301 len 16384 data: 6 5 4 5
15716 008/300 Write block 3679 len 28672 data: 2 3 4 4 4 3 2
15717 009/300 Write block 2263 len 36864 data: 4 3 3 3 3 3 4 4 5
15718 010/300 Flush
15719 011/300 Read block 2281 len 20480
15720 012/300 Read block 4723 len 16384
15721 013/300 Read block 861 len 24576
15722 014/300 Read block 2012 len 36864
15723 015/300 Read block 3102 len 24576
15724 016/300 Write block 4053 len 40960 data: 4 3 4 4 4 4 3 3 3 3
15725 017/300 Write block 3054 len 24576 data: 2 2 2 2 2 2
15726 018/300 Read block 3244 len 20480
15727 019/300 Write block 1670 len 16384 data: 4 4 4 4
15728 020/300 Read block 394 len 4096
15729 021/300 Read block 2901 len 28672
15730 022/300 Read block 3449 len 16384
15731 023/300 Read block 3417 len 24576
15732 024/300 Read block 4299 len 28672
15733 025/300 Write block 2259 len 32768 data: 3 3 3 4 5 4 4 4
15734 026/300 Read block 4735 len 4096
15735 027/300 Read block 2027 len 12288
15736 028/300 Read block 724 len 20480
15737 029/300 Read block 4158 len 16384
15738 030/300 Read block 487 len 12288
15739 031/300 Read block 2852 len 36864
15740 032/300 Write block 4159 len 8192 data: 4 4
15741 033/300 Read block 3735 len 32768
15742 034/300 Write block 961 len 4096 data: 2
15743 035/300 Read block 1557 len 24576
15744 036/300 Read block 4513 len 32768
15745 037/300 Read block 3663 len 24576
15746 038/300 Read block 2095 len 20480
15747 039/300 Write block 3025 len 28672 data: 5 5 5 5 5 5 6
15748 040/300 Read block 4436 len 28672
15749 041/300 Write block 3259 len 24576 data: 4 3 3 2 3 3
15750 042/300 Read block 4076 len 8192
15751 043/300 Read block 914 len 24576
15752 044/300 Write block 191 len 24576 data: 5 5 5 6 4 4
15753 045/300 Read block 326 len 8192
15754 046/300 Read block 4610 len 4096
15755 047/300 Write block 3780 len 40960 data: 2 3 3 4 5 5 5 5 4 4
15756 048/300 Write block 4677 len 36864 data: 2 2 2 4 3 3 3 2 2
15757 049/300 Read block 3092 len 24576
15758 050/300 Write block 630 len 12288 data: 4 4 3
15759 051/300 Write block 2423 len 24576 data: 4 5 5 4 4 5
15760 052/300 Read block 3840 len 24576
15761 053/300 Read block 2388 len 4096
15762 054/300 Read block 3879 len 20480
15763 055/300 Read block 4129 len 32768
15764 056/300 Flush
15765 057/300 Read block 1156 len 12288
15766 058/300 Write block 860 len 4096 data: 6
15767 059/300 Write block 4176 len 24576 data: 3 4 5 6 5 5
15768 060/300 Write block 4847 len 36864 data: 3 3 2 4 4 4 5 5 6
15769 061/300 Read block 4402 len 4096
15770 062/300 Write block 1304 len 28672 data: 3 3 2 3 3 3 4
15771 063/300 Read block 2579 len 20480
15772 064/300 Write block 2297 len 12288 data: 2 3 3
15773 065/300 Read block 2152 len 20480
15774 066/300 Write block 2530 len 16384 data: 5 5 5 5
15775 067/300 Read block 4773 len 4096
15776 068/300 Read block 720 len 28672
15777 069/300 Read block 3809 len 20480
15778 070/300 Read block 1467 len 28672
15779 071/300 Read block 129 len 20480
15780 072/300 Read block 4716 len 28672
15781 073/300 Read block 4298 len 20480
15782 074/300 Read block 4316 len 16384
15783 075/300 Write block 2659 len 36864 data: 3 3 4 3 3 3 3 3 3
15784 076/300 Read block 173 len 12288
15785 077/300 Write block 1778 len 20480 data: 6 4 4 4 6
15786 078/300 Read block 4436 len 12288
15787 079/300 Read block 2712 len 8192
15788 080/300 Write block 2292 len 4096 data: 4
15789 081/300 Flush
15790 082/300 Read block 2020 len 20480
15791 083/300 Write block 3507 len 40960 data: 2 2 2 2 2 2 3 3 3 3
15792 084/300 Write block 4285 len 24576 data: 2 2 3 3 3 3
15793 085/300 Read block 2372 len 12288
15794 086/300 Read block 4643 len 16384
15795 087/300 Read block 1898 len 40960
15796 088/300 Write block 1154 len 4096 data: 5
15797 089/300 Read block 1536 len 4096
15798 090/300 Read block 4215 len 20480
15799 091/300 Flush
15800 092/300 Read block 4145 len 32768
15801 093/300 Write block 3961 len 4096 data: 4
15802 094/300 Write block 4950 len 20480 data: 2 2 4 4 4
15803 095/300 Read block 1119 len 16384
15804 096/300 Write block 3349 len 12288 data: 5 6 6
15805 097/300 Read block 3946 len 40960
15806 098/300 Read block 1772 len 36864
15807 099/300 Write block 4323 len 40960 data: 4 3 3 3 2 2 3 3 5 5
15808 100/300 Read block 3695 len 36864
15809 101/300 Write block 629 len 8192 data: 4 5
15810 102/300 Read block 3255 len 32768
15811 103/300 Write block 92 len 16384 data: 2 2 2 2
15812 104/300 Write block 1462 len 20480 data: 2 2 2 2 2
15813 105/300 Read block 1864 len 32768
15814 106/300 Write block 1109 len 20480 data: 5 4 4 5 3
15815 107/300 Read block 3471 len 36864
15816 108/300 Read block 3927 len 24576
15817 109/300 Write block 3352 len 20480 data: 5 5 4 5 4
15818 110/300 Flush
15819 111/300 Read block 4865 len 12288
15820 112/300 Read block 2124 len 16384
15821 113/300 Write block 926 len 28672 data: 2 2 3 3 4 5 4
15822 114/300 Write block 1274 len 32768 data: 2 3 2 2 2 2 2 2
15823 115/300 Read block 4742 len 40960
15824 116/300 Flush
15825 117/300 Write block 3404 len 36864 data: 5 5 4 4 2 2 2 3 3
15826 118/300 Read block 3785 len 20480
15827 119/300 Read block 1258 len 20480
15828 120/300 Flush
15829 121/300 Write block 2058 len 20480 data: 4 4 4 4 4
15830 122/300 Flush
15831 123/300 Read block 186 len 20480
15832 124/300 Write block 1427 len 16384 data: 3 6 7 4
15833 125/300 Flush
15834 126/300 Write block 2015 len 28672 data: 6 6 6 7 4 4 5
15835 127/300 Read block 4245 len 40960
15836 128/300 Read block 4278 len 36864
15837 129/300 Write block 2784 len 28672 data: 6 3 4 5 5 4 5
15838 130/300 Write block 1291 len 24576 data: 3 4 4 4 4 4
15839 131/300 Read block 3452 len 24576
15840 132/300 Write block 1542 len 16384 data: 5 5 4 4
15841 133/300 Write block 644 len 8192 data: 3 2
15842 134/300 Read block 1861 len 12288
15843 135/300 Write block 3458 len 24576 data: 2 3 3 3 3 4
15844 136/300 Read block 3283 len 36864
15845 137/300 Write block 4756 len 36864 data: 6 5 6 6 5 5 6 4 3
15846 138/300 Write block 4484 len 12288 data: 5 5 5
15847 139/300 Read block 2430 len 20480
15848 140/300 Read block 1378 len 16384
15849 141/300 Write block 622 len 28672 data: 4 4 3 5 6 6 4
15850 142/300 Write block 4166 len 4096 data: 3
15851 143/300 Write block 3028 len 28672 data: 6 6 6 7 6 6 7
15852 144/300 Write block 253 len 20480 data: 3 3 3 3 3
15853 145/300 Read block 2363 len 4096
15854 146/300 Read block 3562 len 16384
15855 147/300 Read block 4438 len 16384
15856 148/300 Read block 3275 len 12288
15857 149/300 Write block 4954 len 16384 data: 5 5 5 5
15858 150/300 Read block 4831 len 24576
15859 151/300 Flush
15860 152/300 Write block 3627 len 20480 data: 6 7 8 4 4
15861 153/300 Read block 225 len 28672
15862 154/300 Read block 4560 len 36864
15863 155/300 Flush
15864 156/300 Read block 2454 len 28672
15865 157/300 Read block 397 len 40960
15866 158/300 Read block 1158 len 36864
15867 159/300 Write block 3375 len 24576 data: 4 3 3 4 5 5
15868 160/300 Read block 4269 len 16384
15869 161/300 Write block 3948 len 28672 data: 6 4 4 4 5 4 3
15870 162/300 Read block 1049 len 8192
15871 163/300 Flush
15872 164/300 Read block 1609 len 12288
15873 165/300 Flush
15874 166/300 Read block 3373 len 32768
15875 167/300 Write block 3127 len 12288 data: 2 2 2
15876 168/300 Read block 4635 len 4096
15877 169/300 Read block 22 len 4096
15878 170/300 Read block 899 len 28672
15879 171/300 Flush
15880 172/300 Read block 2363 len 8192
15881 173/300 Write block 798 len 40960 data: 5 4 5 4 5 4 5 5 5 5
15882 174/300 Read block 4546 len 40960
15883 175/300 Write block 552 len 16384 data: 3 3 3 5
15884 176/300 Read block 3982 len 36864
15885 177/300 Write block 380 len 16384 data: 4 5 5 4
15886 178/300 Read block 698 len 4096
15887 179/300 Read block 1010 len 24576
15888 180/300 Read block 1537 len 40960
15889 181/300 Write block 4315 len 4096 data: 5
15890 182/300 Read block 4696 len 40960
15891 183/300 Read block 3204 len 32768
15892 184/300 Read block 1668 len 40960
15893 185/300 Read block 4129 len 4096
15894 186/300 Read block 167 len 8192
15895 187/300 Write block 1847 len 16384 data: 7 6 6 6
15896 188/300 Flush
15897 189/300 Flush
15898 190/300 Read block 3530 len 16384
15899 191/300 Write block 902 len 20480 data: 3 4 4 4 3
15900 192/300 Read block 693 len 4096
15901 193/300 Read block 87 len 20480
15902 194/300 Write block 1670 len 24576 data: 5 5 5 5 4 4
15903 195/300 Write block 3832 len 28672 data: 3 3 3 5 4 4 5
15904 196/300 Read block 254 len 28672
15905 197/300 Write block 2121 len 4096 data: 3
15906 198/300 Write block 4098 len 16384 data: 5 6 6 5
15907 199/300 Write block 4914 len 36864 data: 5 5 4 3 2 2 2 3 3
15908 200/300 Write block 1363 len 40960 data: 7 5 4 4 6 6 5 5 5 5
15909 201/300 Read block 1048 len 24576
15910 202/300 Write block 4648 len 40960 data: 3 4 3 3 4 5 3 3 3 3
15911 203/300 Flush
15912 204/300 Write block 1044 len 36864 data: 5 5 3 2 3 4 5 5 5
15913 205/300 Flush
15914 206/300 Read block 4615 len 24576
15915 207/300 Read block 4136 len 12288
15916 208/300 Read block 2166 len 28672
15917 209/300 Write block 991 len 32768 data: 3 3 4 4 5 5 4 4
15918 210/300 Write block 1000 len 28672 data: 5 5 3 3 3 3 3
15919 211/300 Read block 1420 len 28672
15920 212/300 Write block 4542 len 8192 data: 4 4
15921 213/300 Read block 1588 len 24576
15922 214/300 Read block 3922 len 32768
15923 215/300 Read block 3514 len 36864
15924 216/300 Read block 1332 len 36864
15925 217/300 Read block 1739 len 24576
15926 218/300 Write block 1622 len 40960 data: 6 5 4 3 3 3 3 3 2 2
15927 219/300 Read block 2033 len 20480
15928 220/300 Read block 2985 len 24576
15929 221/300 Flush
15930 222/300 Write block 1119 len 16384 data: 3 3 3 3
15931 223/300 Write block 4068 len 20480 data: 3 3 3 4 4
15932 224/300 Read block 4698 len 16384
15933 225/300 Write block 979 len 8192 data: 2 2
15934 226/300 Read block 3630 len 28672
15935 227/300 Read block 4733 len 4096
15936 228/300 Read block 934 len 36864
15937 229/300 Read block 4864 len 20480
15938 230/300 Read block 872 len 4096
15939 231/300 Read block 4410 len 24576
15940 232/300 Read block 1400 len 16384
15941 233/300 Read block 3695 len 24576
15942 234/300 Read block 8 len 8192
15943 235/300 Write block 1844 len 8192 data: 7 6
15944 236/300 Write block 1985 len 28672 data: 9 9 8 7 7 6 5
15945 237/300 Read block 4909 len 28672
15946 238/300 Read block 1501 len 32768
15947 239/300 Write block 2931 len 32768 data: 5 5 5 6 3 3 3 3
15948 240/300 Write block 2252 len 36864 data: 4 4 3 3 3 3 3 4 4
15949 241/300 Read block 2320 len 12288
15950 242/300 Write block 3493 len 4096 data: 4
15951 243/300 Write block 527 len 12288 data: 5 6 8
15952 244/300 Write block 3227 len 16384 data: 4 3 3 6
15953 245/300 Flush
15954 246/300 Flush
15955 247/300 Write block 863 len 24576 data: 5 5 5 5 5 4
15956 248/300 Write block 666 len 12288 data: 5 4 3
15957 249/300 Read block 1159 len 4096
15958 250/300 Write block 116 len 4096 data: 3
15959 251/300 Write block 533 len 28672 data: 4 3 3 2 2 3 3
15960 252/300 Read block 1592 len 12288
15961 253/300 Write block 836 len 8192 data: 3 4
15962 254/300 Write block 2805 len 28672 data: 2 3 4 4 4 4 4
15963 255/300 Read block 3768 len 32768
15964 256/300 Read block 4052 len 20480
15965 257/300 Write block 2067 len 40960 data: 2 2 2 4 4 4 4 5 5 7
15966 258/300 Flush
15967 259/300 Write block 404 len 16384 data: 4 4 4 5
15968 260/300 Read block 2230 len 12288
15969 261/300 Read block 136 len 16384
15970 262/300 Read block 2503 len 16384
15971 263/300 Read block 3987 len 20480
15972 264/300 Write block 898 len 36864 data: 2 3 3 3 4 5 5 5 4
15973 265/300 Read block 1386 len 4096
15974 266/300 Flush
15975 267/300 Flush
15976 268/300 Write block 2898 len 36864 data: 3 5 5 5 6 5 5 3 4
15977 269/300 Write block 936 len 28672 data: 3 5 5 5 4 5 5
15978 270/300 Flush
15979 271/300 Flush
15980 272/300 Write block 3198 len 8192 data: 4 6
15981 273/300 Read block 3783 len 8192
15982 274/300 Read block 325 len 20480
15983 275/300 Read block 4636 len 8192
15984 276/300 Write block 1152 len 12288 data: 3 4 6
15985 277/300 Write block 4842 len 40960 data: 4 4 4 4 4 4 4 3 5 5
15986 278/300 Write block 1734 len 36864 data: 4 4 4 6 7 6 5 5 5
15987 279/300 Read block 3989 len 32768
15988 280/300 Write block 2842 len 8192 data: 5 4
15989 281/300 Flush
15990 282/300 Read block 217 len 32768
15991 283/300 Read block 4501 len 24576
15992 284/300 Read block 1132 len 28672
15993 285/300 Read block 1915 len 4096
15994 286/300 Write block 2007 len 40960 data: 3 3 3 3 3 3 4 5 7 7
15995 287/300 Read block 4815 len 12288
15996 288/300 Flush
15997 289/300 Write block 426 len 28672 data: 2 3 4 4 5 4 3
15998 290/300 Write block 3615 len 8192 data: 2 2
15999 291/300 Read block 2530 len 12288
16000 292/300 Read block 3252 len 12288
16001 293/300 Write block 178 len 16384 data: 3 3 3 5
16002 294/300 Read block 267 len 28672
16003 295/300 Read block 3309 len 40960
16004 296/300 Read block 3646 len 4096
16005 297/300 Write block 3210 len 28672 data: 2 2 3 2 2 2 2
16006 298/300 Read block 2172 len 40960
16007 299/300 Write block 4970 len 28672 data: 3 3 3 4 2 2 2
16008 300/300 Write block 4602 len 32768 data: 3 2 3 3 3 5 6 6
16009 301/300 Write block 3922 len 28672 data: 2 2 2 3 4 4 5
16010 Main task has work to do, go find it
16011 got dsc Start(0)
16012 start 0
16013 Jan 21 15:50:33.046 INFO request completed, latency_us: 219, response_code: 204, uri: /start/cid/0, method: POST, req_id: c759139e-0af9-4d6f-a3fa-2fd709e0b80d, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
16014 Replay: started 0, returned:Ok(())
16015 ----------------------------------------------------------------
16016 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16017 States: Offline Active Active
16018 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16019 5227 Acked Read 9 Sent Done Done false
16020 5228 Acked Read 1 Sent Done Done false
16021 5229 Acked Write 8 Sent Done Done false
16022 5230 Acked Write 5 Sent Done Done false
16023 5231 Acked Read 4 Sent Done Done false
16024 5232 Acked Write 3 Sent Done Done false
16025 5233 Acked Write 4 Sent Done Done false
16026 5234 Acked Write 7 Sent Done Done false
16027 5235 Acked Write 9 Sent Done Done false
16028 5236 Acked Flush 0 Sent Done Done false
16029 5237 Acked Read 5 Sent Done Done false
16030 5238 Acked Read 4 Sent Done Done false
16031 5239 Acked Read 6 Sent Done Done false
16032 5240 Acked Read 9 Sent Done Done false
16033 5241 Acked Read 6 Sent Done Done false
16034 5242 Acked Write 10 Sent Done Done false
16035 5243 Acked Write 6 Sent Done Done false
16036 5244 Acked Read 5 Sent Done Done false
16037 5245 Acked Write 4 Sent Done Done false
16038 5246 Acked Read 1 Sent Done Done false
16039 5247 Acked Read 7 Sent Done Done false
16040 5248 Acked Read 4 Sent Done Done false
16041 5249 Acked Read 6 Sent Done Done false
16042 5250 Acked Read 7 Sent Done Done false
16043 5251 Acked Write 8 Sent Done Done false
16044 5252 Acked Read 1 Sent Done Done false
16045 5253 Acked Read 3 Sent Done Done false
16046 5254 Acked Read 5 Sent Done Done false
16047 5255 Acked Read 4 Sent Done Done false
16048 5256 Acked Read 3 Sent Done Done false
16049 5257 Acked Read 9 Sent Done Done false
16050 5258 Acked Write 2 Sent Done Done false
16051 5259 Acked Read 8 Sent Done Done false
16052 5260 Acked Write 1 Sent Done Done false
16053 5261 Acked Read 6 Sent Done Done false
16054 5262 Acked Read 8 Sent Done Done false
16055 5263 Acked Read 6 Sent Done Done false
16056 5264 Acked Read 5 Sent Done Done false
16057 5265 Acked Write 7 Sent Done Done false
16058 5266 Acked Read 7 Sent Done Done false
16059 5267 Acked Write 6 Sent Done Done false
16060 5268 Acked Read 2 Sent Done Done false
16061 5269 Acked Read 6 Sent Done Done false
16062 5270 Acked Write 6 Sent Done Done false
16063 5271 Acked Read 2 Sent Done Done false
16064 5272 Acked Read 1 Sent Done Done false
16065 5273 Acked Write 10 Sent Done Done false
16066 5274 Acked Write 9 Sent Done Done false
16067 5275 Acked Read 6 Sent Done Done false
16068 5276 Acked Write 3 Sent Done Done false
16069 5277 Acked Write 6 Sent Done Done false
16070 5278 Acked Read 6 Sent Done Done false
16071 5279 Acked Read 1 Sent Done Done false
16072 5280 Acked Read 5 Sent Done Done false
16073 5281 Acked Read 8 Sent Done Done false
16074 5282 Acked Flush 0 Sent Done Done false
16075 5283 Acked Read 3 Sent Done Done false
16076 5284 Acked Write 1 Sent Done Done false
16077 5285 Acked Write 6 Sent Done Done false
16078 5286 Acked Write 9 Sent Done Done false
16079 5287 Acked Read 1 Sent Done Done false
16080 5288 Acked Write 7 Sent Done Done false
16081 5289 Acked Read 5 Sent Done Done false
16082 5290 Acked Write 3 Sent Done Done false
16083 5291 Acked Read 5 Sent Done Done false
16084 5292 Acked Write 4 Sent Done Done false
16085 5293 Acked Read 1 Sent Done Done false
16086 5294 Acked Read 7 Sent Done Done false
16087 5295 Acked Read 5 Sent Done Done false
16088 5296 Acked Read 7 Sent Done Done false
16089 5297 Acked Read 5 Sent Done Done false
16090 5298 Acked Read 7 Sent Done Done false
16091 5299 Acked Read 5 Sent Done Done false
16092 5300 Acked Read 4 Sent Done Done false
16093 5301 Acked Write 9 Sent Done Done false
16094 5302 Acked Read 3 Sent Done Done false
16095 5303 Acked Write 5 Sent Done Done false
16096 5304 Acked Read 3 Sent Done Done false
16097 5305 Acked Read 2 Sent Done Done false
16098 5306 Acked Write 1 Sent Done Done false
16099 5307 Acked Flush 0 Sent Done Done false
16100 5308 Acked Read 5 Sent Done Done false
16101 5309 Acked Write 10 Sent Done Done false
16102 5310 Acked Write 6 Sent Done Done false
16103 5311 Acked Read 3 Sent Done Done false
16104 5312 Acked Read 4 Sent Done Done false
16105 5313 Acked Read 10 Sent Done Done false
16106 5314 Acked Write 1 Sent Done Done false
16107 5315 Acked Read 1 Sent Done Done false
16108 5316 Acked Read 5 Sent Done Done false
16109 5317 Acked Flush 0 Sent Done Done false
16110 5318 Acked Read 8 Sent Done Done false
16111 5319 Acked Write 1 Sent Done Done false
16112 5320 Acked Write 5 Sent Done Done false
16113 5321 Acked Read 4 Sent Done Done false
16114 5322 Acked Write 3 Sent Done Done false
16115 5323 Acked Read 10 Sent Done Done false
16116 5324 Acked Read 9 Sent Done Done false
16117 5325 Acked Write 10 Sent Done Done false
16118 5326 Acked Read 9 Sent Done Done false
16119 5327 Acked Write 2 Sent Done Done false
16120 5328 Acked Read 8 Sent Done Done false
16121 5329 Acked Write 4 Sent Done Done false
16122 5330 Acked Write 5 Sent Done Done false
16123 5331 Acked Read 8 Sent Done Done false
16124 5332 Acked Write 5 Sent Done Done false
16125 5333 Acked Read 9 Sent Done Done false
16126 5334 Acked Read 6 Sent Done Done false
16127 5335 Acked Write 5 Sent Done Done false
16128 5336 Acked Flush 0 Sent Done Done false
16129 5337 Acked Read 3 Sent Done Done false
16130 5338 Acked Read 4 Sent Done Done false
16131 5339 Acked Write 7 Sent Done Done false
16132 5340 Acked Write 8 Sent Done Done false
16133 5341 Acked Read 10 Sent Done Done false
16134 5342 Acked Flush 0 Sent Done Done false
16135 5343 Acked Write 9 Sent Done Done false
16136 5344 Acked Read 5 Sent Done Done false
16137 5345 Acked Read 5 Sent Done Done false
16138 5346 Acked Flush 0 Sent Done Done false
16139 5347 Acked Write 5 Sent Done Done false
16140 5348 Acked Flush 0 Sent Done Done false
16141 5349 Acked Read 5 Sent Done Done false
16142 5350 Acked Write 4 Sent Done Done false
16143 5351 Acked Flush 0 Sent Done Done false
16144 5352 Acked Write 7 Sent Done Done false
16145 5353 Acked Read 10 Sent Done Done false
16146 5354 Acked Read 9 Sent Done Done false
16147 5355 Acked Write 7 Sent Done Done false
16148 5356 Acked Write 6 Sent Done Done false
16149 5357 Acked Read 6 Sent Done Done false
16150 5358 Acked Write 4 Sent Done Done false
16151 5359 Acked Write 2 Sent Done Done false
16152 5360 Acked Read 3 Sent Done Done false
16153 5361 Acked Write 6 Sent Done Done false
16154 5362 Acked Read 9 Sent Done Done false
16155 5363 Acked Write 9 Sent Done Done false
16156 5364 Acked Write 3 Sent Done Done false
16157 5365 Acked Read 5 Sent Done Done false
16158 5366 Acked Read 4 Sent Done Done false
16159 5367 Acked Write 7 Sent Done Done false
16160 5368 Acked Write 1 Sent Done Done false
16161 5369 Acked Write 7 Sent Done Done false
16162 5370 Acked Write 5 Sent Done Done false
16163 5371 Acked Read 1 Sent Done Done false
16164 5372 Acked Read 4 Sent Done Done false
16165 5373 Acked Read 4 Sent Done Done false
16166 5374 Acked Read 3 Sent Done Done false
16167 5375 Acked Write 4 Sent Done Done false
16168 5376 Acked Read 6 Sent Done Done false
16169 5377 Acked Flush 0 Sent Done Done false
16170 5378 Acked Write 5 Sent Done Done false
16171 5379 Acked Read 7 Sent Done Done false
16172 5380 Acked Read 9 Sent Done Done false
16173 5381 Acked Flush 0 Sent Done Done false
16174 5382 Acked Read 7 Sent Done Done false
16175 5383 Acked Read 10 Sent Done Done false
16176 5384 Acked Read 9 Sent Done Done false
16177 5385 Acked Write 6 Sent Done Done false
16178 5386 Acked Read 4 Sent Done Done false
16179 5387 Acked Write 7 Sent Done Done false
16180 5388 Acked Read 2 Sent Done Done false
16181 5389 Acked Flush 0 Sent Done Done false
16182 5390 Acked Read 3 Sent Done Done false
16183 5391 Acked Flush 0 Sent Done Done false
16184 5392 Acked Read 8 Sent Done Done false
16185 5393 Acked Write 3 Sent Done Done false
16186 5394 Acked Read 1 Sent Done Done false
16187 5395 Acked Read 1 Sent Done Done false
16188 5396 Acked Read 7 Sent Done Done false
16189 5397 Acked Flush 0 Sent Done Done false
16190 5398 Acked Read 2 Sent Done Done false
16191 5399 Acked Write 10 Sent Done Done false
16192 5400 Acked Read 10 Sent Done Done false
16193 5401 Acked Write 4 Sent Done Done false
16194 5402 Acked Read 9 Sent Done Done false
16195 5403 Acked Write 4 Sent Done Done false
16196 5404 Acked Read 1 Sent Done Done false
16197 5405 Acked Read 6 Sent Done Done false
16198 5406 Acked Read 10 Sent Done Done false
16199 5407 Acked Write 1 Sent Done Done false
16200 5408 Acked Read 10 Sent Done Done false
16201 5409 Acked Read 8 Sent Done Done false
16202 5410 Acked Read 10 Sent Done Done false
16203 5411 Acked Read 1 Sent Done Done false
16204 5412 Acked Read 2 Sent Done Done false
16205 5413 Acked Write 4 Sent Done Done false
16206 5414 Acked Flush 0 Sent Done Done false
16207 5415 Acked Flush 0 Sent Done Done false
16208 5416 Acked Read 4 Sent Done Done false
16209 5417 Acked Write 5 Sent Done Done false
16210 5418 Acked Read 1 Sent Done Done false
16211 5419 Acked Read 5 Sent Done Done false
16212 5420 Acked Write 6 Sent Done Done false
16213 5421 Acked Write 7 Sent Done Done false
16214 5422 Acked Read 7 Sent Done Done false
16215 5423 Acked Write 1 Sent Done Done false
16216 5424 Acked Write 4 Sent Done Done false
16217 5425 Acked Write 9 Sent Done Done false
16218 5426 Acked Write 10 Sent Done Done false
16219 5427 Acked Read 6 Sent Done Done false
16220 5428 Acked Write 10 Sent Done Done false
16221 5429 Acked Flush 0 Sent Done Done false
16222 5430 Acked Write 9 Sent Done Done false
16223 5431 Acked Flush 0 Sent Done Done false
16224 5432 Acked Read 6 Sent Done Done false
16225 5433 Acked Read 3 Sent Done Done false
16226 5434 Acked Read 7 Sent Done Done false
16227 5435 Acked Write 8 Sent Done Done false
16228 5436 Acked Write 7 Sent Done Done false
16229 5437 Acked Read 7 Sent Done Done false
16230 5438 Acked Write 2 Sent Done Done false
16231 5439 Acked Read 6 Sent Done Done false
16232 5440 Acked Read 8 Sent Done Done false
16233 5441 Acked Read 9 Sent Done Done false
16234 5442 Acked Read 9 Sent Done Done false
16235 5443 Acked Read 6 Sent Done Done false
16236 5444 Acked Write 10 Sent Done Done false
16237 5445 Acked Read 5 Sent Done Done false
16238 5446 Acked Read 6 Sent Done Done false
16239 5447 Acked Flush 0 Sent Done Done false
16240 5448 Acked Write 4 Sent Done Done false
16241 5449 Acked Write 5 Sent Done Done false
16242 5450 Acked Read 4 Sent Done Done false
16243 5451 Acked Write 2 Sent Done Done false
16244 5452 Acked Read 7 Sent Done Done false
16245 5453 Acked Read 1 Sent Done Done false
16246 5454 Acked Read 9 Sent Done Done false
16247 5455 Acked Read 5 Sent Done Done false
16248 5456 Acked Read 1 Sent Done Done false
16249 5457 Acked Read 6 Sent Done Done false
16250 5458 Acked Read 4 Sent Done Done false
16251 5459 Acked Read 6 Sent Done Done false
16252 5460 Acked Read 2 Sent Done Done false
16253 5461 Acked Write 2 Sent Done Done false
16254 5462 Acked Write 7 Sent Done Done false
16255 5463 Acked Read 7 Sent Done Done false
16256 5464 Acked Read 8 Sent Done Done false
16257 5465 Acked Write 8 Sent Done Done false
16258 5466 Acked Write 9 Sent Done Done false
16259 5467 Acked Read 3 Sent Done Done false
16260 5468 Acked Write 1 Sent Done Done false
16261 5469 Acked Write 3 Sent Done Done false
16262 5470 Acked Write 4 Sent Done Done false
16263 5471 Acked Flush 0 Sent Done Done false
16264 5472 Acked Flush 0 Sent Done Done false
16265 5473 Acked Write 6 Sent Done Done false
16266 5474 Acked Write 3 Sent Done Done false
16267 5475 Acked Read 1 Sent Done Done false
16268 5476 Acked Write 1 Sent Done Done false
16269 5477 Acked Write 7 Sent Done Done false
16270 5478 Acked Read 3 Sent Done Done false
16271 5479 Acked Write 2 Sent Done Done false
16272 5480 Acked Write 7 Sent Done Done false
16273 5481 Acked Read 8 Sent Done Done false
16274 5482 Acked Read 5 Sent Done Done false
16275 5483 Acked Write 10 Sent Done Done false
16276 5484 Acked Flush 0 Sent Done Done false
16277 5485 Acked Write 4 Sent Done Done false
16278 5486 Acked Read 3 Sent Done Done false
16279 5487 Acked Read 4 Sent Done Done false
16280 5488 Acked Read 4 Sent Done Done false
16281 5489 Acked Read 5 Sent Done Done false
16282 5490 Acked Write 9 Sent Done Done false
16283 5491 Acked Read 1 Sent Done Done false
16284 5492 Acked Flush 0 Sent Done Done false
16285 5493 Acked Flush 0 Sent Done Done false
16286 5494 Acked Write 9 Sent Done Done false
16287 5495 Acked Write 7 Sent Done Done false
16288 5496 Acked Flush 0 Sent Done Done false
16289 5497 Acked Flush 0 Sent Done Done false
16290 5498 Acked Write 2 Sent Done Done false
16291 5499 Acked Read 2 Sent Done Done false
16292 5500 Acked Read 5 Sent Done Done false
16293 5501 Acked Read 2 Sent Done Done false
16294 5502 Acked Write 3 Sent Done Done false
16295 5503 Acked Write 10 Sent Done Done false
16296 5504 Acked Write 9 Sent Done Done false
16297 5505 Acked Read 8 Sent Done Done false
16298 5506 Acked Write 2 Sent Done Done false
16299 5507 Acked Flush 0 Sent Done Done false
16300 5508 Acked Read 8 Sent Done Done false
16301 5509 Acked Read 6 Sent Done Done false
16302 5510 Acked Read 7 Sent Done Done false
16303 5511 Acked Read 1 Sent Done Done false
16304 5512 Acked Write 10 Sent Done Done false
16305 5513 Acked Read 3 Sent Done Done false
16306 5514 Acked Flush 0 Sent Done Done false
16307 5515 Acked Write 7 Sent Done Done false
16308 5516 Acked Write 2 Sent Done Done false
16309 5517 Acked Read 3 Sent Done Done false
16310 5518 Acked Read 3 Sent Done Done false
16311 5519 Acked Write 4 Sent Done Done false
16312 5520 Acked Read 7 Sent Done Done false
16313 5521 Acked Read 10 Sent Done Done false
16314 5522 Acked Read 1 Sent Done Done false
16315 5523 Acked Write 7 Sent Done Done false
16316 5524 Acked Read 10 Sent Done Done false
16317 5525 Acked Write 7 Sent Done Done false
16318 5526 Acked Write 8 Sent Done Done false
16319 5527 Acked Write 7 Sent Done Sent false
16320 STATES DS:0 DS:1 DS:2 TOTAL
16321 Sent 301 0 1 302
16322 Done 0 301 300 601
16323 Skipped 0 0 0 0
16324 Error 0 0 0 0
16325 Last Flush: 5226 5514 5514
16326 Downstairs last five completed: 5226 5225 5224 5223 5222
16327 Upstairs last five completed: 5527 5526 5525 5524 5523
16328 CLIENT: Up:0 ds:301 act:2
16329 [8810] got start action
16330 [8810] I am going to restart
16331 Starting downstairs at port 8810
16332 Make output file at "/tmp/dsc/downstairs-8810.txt"
16333 [8810][0] reports Starting
16334 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1215)
16335 [8810][0] reports Running
16336 ----------------------------------------------------------------
16337 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16338 States: Offline Active Active
16339 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16340 5227 Acked Read 9 Sent Done Done false
16341 5228 Acked Read 1 Sent Done Done false
16342 5229 Acked Write 8 Sent Done Done false
16343 5230 Acked Write 5 Sent Done Done false
16344 5231 Acked Read 4 Sent Done Done false
16345 5232 Acked Write 3 Sent Done Done false
16346 5233 Acked Write 4 Sent Done Done false
16347 5234 Acked Write 7 Sent Done Done false
16348 5235 Acked Write 9 Sent Done Done false
16349 5236 Acked Flush 0 Sent Done Done false
16350 5237 Acked Read 5 Sent Done Done false
16351 5238 Acked Read 4 Sent Done Done false
16352 5239 Acked Read 6 Sent Done Done false
16353 5240 Acked Read 9 Sent Done Done false
16354 5241 Acked Read 6 Sent Done Done false
16355 5242 Acked Write 10 Sent Done Done false
16356 5243 Acked Write 6 Sent Done Done false
16357 5244 Acked Read 5 Sent Done Done false
16358 5245 Acked Write 4 Sent Done Done false
16359 5246 Acked Read 1 Sent Done Done false
16360 5247 Acked Read 7 Sent Done Done false
16361 5248 Acked Read 4 Sent Done Done false
16362 5249 Acked Read 6 Sent Done Done false
16363 5250 Acked Read 7 Sent Done Done false
16364 5251 Acked Write 8 Sent Done Done false
16365 5252 Acked Read 1 Sent Done Done false
16366 5253 Acked Read 3 Sent Done Done false
16367 5254 Acked Read 5 Sent Done Done false
16368 5255 Acked Read 4 Sent Done Done false
16369 5256 Acked Read 3 Sent Done Done false
16370 5257 Acked Read 9 Sent Done Done false
16371 5258 Acked Write 2 Sent Done Done false
16372 5259 Acked Read 8 Sent Done Done false
16373 5260 Acked Write 1 Sent Done Done false
16374 5261 Acked Read 6 Sent Done Done false
16375 5262 Acked Read 8 Sent Done Done false
16376 5263 Acked Read 6 Sent Done Done false
16377 5264 Acked Read 5 Sent Done Done false
16378 5265 Acked Write 7 Sent Done Done false
16379 5266 Acked Read 7 Sent Done Done false
16380 5267 Acked Write 6 Sent Done Done false
16381 5268 Acked Read 2 Sent Done Done false
16382 5269 Acked Read 6 Sent Done Done false
16383 5270 Acked Write 6 Sent Done Done false
16384 5271 Acked Read 2 Sent Done Done false
16385 5272 Acked Read 1 Sent Done Done false
16386 5273 Acked Write 10 Sent Done Done false
16387 5274 Acked Write 9 Sent Done Done false
16388 5275 Acked Read 6 Sent Done Done false
16389 5276 Acked Write 3 Sent Done Done false
16390 5277 Acked Write 6 Sent Done Done false
16391 5278 Acked Read 6 Sent Done Done false
16392 5279 Acked Read 1 Sent Done Done false
16393 5280 Acked Read 5 Sent Done Done false
16394 5281 Acked Read 8 Sent Done Done false
16395 5282 Acked Flush 0 Sent Done Done false
16396 5283 Acked Read 3 Sent Done Done false
16397 5284 Acked Write 1 Sent Done Done false
16398 5285 Acked Write 6 Sent Done Done false
16399 5286 Acked Write 9 Sent Done Done false
16400 5287 Acked Read 1 Sent Done Done false
16401 5288 Acked Write 7 Sent Done Done false
16402 5289 Acked Read 5 Sent Done Done false
16403 5290 Acked Write 3 Sent Done Done false
16404 5291 Acked Read 5 Sent Done Done false
16405 5292 Acked Write 4 Sent Done Done false
16406 5293 Acked Read 1 Sent Done Done false
16407 5294 Acked Read 7 Sent Done Done false
16408 5295 Acked Read 5 Sent Done Done false
16409 5296 Acked Read 7 Sent Done Done false
16410 5297 Acked Read 5 Sent Done Done false
16411 5298 Acked Read 7 Sent Done Done false
16412 5299 Acked Read 5 Sent Done Done false
16413 5300 Acked Read 4 Sent Done Done false
16414 5301 Acked Write 9 Sent Done Done false
16415 5302 Acked Read 3 Sent Done Done false
16416 5303 Acked Write 5 Sent Done Done false
16417 5304 Acked Read 3 Sent Done Done false
16418 5305 Acked Read 2 Sent Done Done false
16419 5306 Acked Write 1 Sent Done Done false
16420 5307 Acked Flush 0 Sent Done Done false
16421 5308 Acked Read 5 Sent Done Done false
16422 5309 Acked Write 10 Sent Done Done false
16423 5310 Acked Write 6 Sent Done Done false
16424 5311 Acked Read 3 Sent Done Done false
16425 5312 Acked Read 4 Sent Done Done false
16426 5313 Acked Read 10 Sent Done Done false
16427 5314 Acked Write 1 Sent Done Done false
16428 5315 Acked Read 1 Sent Done Done false
16429 5316 Acked Read 5 Sent Done Done false
16430 5317 Acked Flush 0 Sent Done Done false
16431 5318 Acked Read 8 Sent Done Done false
16432 5319 Acked Write 1 Sent Done Done false
16433 5320 Acked Write 5 Sent Done Done false
16434 5321 Acked Read 4 Sent Done Done false
16435 5322 Acked Write 3 Sent Done Done false
16436 5323 Acked Read 10 Sent Done Done false
16437 5324 Acked Read 9 Sent Done Done false
16438 5325 Acked Write 10 Sent Done Done false
16439 5326 Acked Read 9 Sent Done Done false
16440 5327 Acked Write 2 Sent Done Done false
16441 5328 Acked Read 8 Sent Done Done false
16442 5329 Acked Write 4 Sent Done Done false
16443 5330 Acked Write 5 Sent Done Done false
16444 5331 Acked Read 8 Sent Done Done false
16445 5332 Acked Write 5 Sent Done Done false
16446 5333 Acked Read 9 Sent Done Done false
16447 5334 Acked Read 6 Sent Done Done false
16448 5335 Acked Write 5 Sent Done Done false
16449 5336 Acked Flush 0 Sent Done Done false
16450 5337 Acked Read 3 Sent Done Done false
16451 5338 Acked Read 4 Sent Done Done false
16452 5339 Acked Write 7 Sent Done Done false
16453 5340 Acked Write 8 Sent Done Done false
16454 5341 Acked Read 10 Sent Done Done false
16455 5342 Acked Flush 0 Sent Done Done false
16456 5343 Acked Write 9 Sent Done Done false
16457 5344 Acked Read 5 Sent Done Done false
16458 5345 Acked Read 5 Sent Done Done false
16459 5346 Acked Flush 0 Sent Done Done false
16460 5347 Acked Write 5 Sent Done Done false
16461 5348 Acked Flush 0 Sent Done Done false
16462 5349 Acked Read 5 Sent Done Done false
16463 5350 Acked Write 4 Sent Done Done false
16464 5351 Acked Flush 0 Sent Done Done false
16465 5352 Acked Write 7 Sent Done Done false
16466 5353 Acked Read 10 Sent Done Done false
16467 5354 Acked Read 9 Sent Done Done false
16468 5355 Acked Write 7 Sent Done Done false
16469 5356 Acked Write 6 Sent Done Done false
16470 5357 Acked Read 6 Sent Done Done false
16471 5358 Acked Write 4 Sent Done Done false
16472 5359 Acked Write 2 Sent Done Done false
16473 5360 Acked Read 3 Sent Done Done false
16474 5361 Acked Write 6 Sent Done Done false
16475 5362 Acked Read 9 Sent Done Done false
16476 5363 Acked Write 9 Sent Done Done false
16477 5364 Acked Write 3 Sent Done Done false
16478 5365 Acked Read 5 Sent Done Done false
16479 5366 Acked Read 4 Sent Done Done false
16480 5367 Acked Write 7 Sent Done Done false
16481 5368 Acked Write 1 Sent Done Done false
16482 5369 Acked Write 7 Sent Done Done false
16483 5370 Acked Write 5 Sent Done Done false
16484 5371 Acked Read 1 Sent Done Done false
16485 5372 Acked Read 4 Sent Done Done false
16486 5373 Acked Read 4 Sent Done Done false
16487 5374 Acked Read 3 Sent Done Done false
16488 5375 Acked Write 4 Sent Done Done false
16489 5376 Acked Read 6 Sent Done Done false
16490 5377 Acked Flush 0 Sent Done Done false
16491 5378 Acked Write 5 Sent Done Done false
16492 5379 Acked Read 7 Sent Done Done false
16493 5380 Acked Read 9 Sent Done Done false
16494 5381 Acked Flush 0 Sent Done Done false
16495 5382 Acked Read 7 Sent Done Done false
16496 5383 Acked Read 10 Sent Done Done false
16497 5384 Acked Read 9 Sent Done Done false
16498 5385 Acked Write 6 Sent Done Done false
16499 5386 Acked Read 4 Sent Done Done false
16500 5387 Acked Write 7 Sent Done Done false
16501 5388 Acked Read 2 Sent Done Done false
16502 5389 Acked Flush 0 Sent Done Done false
16503 5390 Acked Read 3 Sent Done Done false
16504 5391 Acked Flush 0 Sent Done Done false
16505 5392 Acked Read 8 Sent Done Done false
16506 5393 Acked Write 3 Sent Done Done false
16507 5394 Acked Read 1 Sent Done Done false
16508 5395 Acked Read 1 Sent Done Done false
16509 5396 Acked Read 7 Sent Done Done false
16510 5397 Acked Flush 0 Sent Done Done false
16511 5398 Acked Read 2 Sent Done Done false
16512 5399 Acked Write 10 Sent Done Done false
16513 5400 Acked Read 10 Sent Done Done false
16514 5401 Acked Write 4 Sent Done Done false
16515 5402 Acked Read 9 Sent Done Done false
16516 5403 Acked Write 4 Sent Done Done false
16517 5404 Acked Read 1 Sent Done Done false
16518 5405 Acked Read 6 Sent Done Done false
16519 5406 Acked Read 10 Sent Done Done false
16520 5407 Acked Write 1 Sent Done Done false
16521 5408 Acked Read 10 Sent Done Done false
16522 5409 Acked Read 8 Sent Done Done false
16523 5410 Acked Read 10 Sent Done Done false
16524 5411 Acked Read 1 Sent Done Done false
16525 5412 Acked Read 2 Sent Done Done false
16526 5413 Acked Write 4 Sent Done Done false
16527 5414 Acked Flush 0 Sent Done Done false
16528 5415 Acked Flush 0 Sent Done Done false
16529 5416 Acked Read 4 Sent Done Done false
16530 5417 Acked Write 5 Sent Done Done false
16531 5418 Acked Read 1 Sent Done Done false
16532 5419 Acked Read 5 Sent Done Done false
16533 5420 Acked Write 6 Sent Done Done false
16534 5421 Acked Write 7 Sent Done Done false
16535 5422 Acked Read 7 Sent Done Done false
16536 5423 Acked Write 1 Sent Done Done false
16537 5424 Acked Write 4 Sent Done Done false
16538 5425 Acked Write 9 Sent Done Done false
16539 5426 Acked Write 10 Sent Done Done false
16540 5427 Acked Read 6 Sent Done Done false
16541 5428 Acked Write 10 Sent Done Done false
16542 5429 Acked Flush 0 Sent Done Done false
16543 5430 Acked Write 9 Sent Done Done false
16544 5431 Acked Flush 0 Sent Done Done false
16545 5432 Acked Read 6 Sent Done Done false
16546 5433 Acked Read 3 Sent Done Done false
16547 5434 Acked Read 7 Sent Done Done false
16548 5435 Acked Write 8 Sent Done Done false
16549 5436 Acked Write 7 Sent Done Done false
16550 5437 Acked Read 7 Sent Done Done false
16551 5438 Acked Write 2 Sent Done Done false
16552 5439 Acked Read 6 Sent Done Done false
16553 5440 Acked Read 8 Sent Done Done false
16554 5441 Acked Read 9 Sent Done Done false
16555 5442 Acked Read 9 Sent Done Done false
16556 5443 Acked Read 6 Sent Done Done false
16557 5444 Acked Write 10 Sent Done Done false
16558 5445 Acked Read 5 Sent Done Done false
16559 5446 Acked Read 6 Sent Done Done false
16560 5447 Acked Flush 0 Sent Done Done false
16561 5448 Acked Write 4 Sent Done Done false
16562 5449 Acked Write 5 Sent Done Done false
16563 5450 Acked Read 4 Sent Done Done false
16564 5451 Acked Write 2 Sent Done Done false
16565 5452 Acked Read 7 Sent Done Done false
16566 5453 Acked Read 1 Sent Done Done false
16567 5454 Acked Read 9 Sent Done Done false
16568 5455 Acked Read 5 Sent Done Done false
16569 5456 Acked Read 1 Sent Done Done false
16570 5457 Acked Read 6 Sent Done Done false
16571 5458 Acked Read 4 Sent Done Done false
16572 5459 Acked Read 6 Sent Done Done false
16573 5460 Acked Read 2 Sent Done Done false
16574 5461 Acked Write 2 Sent Done Done false
16575 5462 Acked Write 7 Sent Done Done false
16576 5463 Acked Read 7 Sent Done Done false
16577 5464 Acked Read 8 Sent Done Done false
16578 5465 Acked Write 8 Sent Done Done false
16579 5466 Acked Write 9 Sent Done Done false
16580 5467 Acked Read 3 Sent Done Done false
16581 5468 Acked Write 1 Sent Done Done false
16582 5469 Acked Write 3 Sent Done Done false
16583 5470 Acked Write 4 Sent Done Done false
16584 5471 Acked Flush 0 Sent Done Done false
16585 5472 Acked Flush 0 Sent Done Done false
16586 5473 Acked Write 6 Sent Done Done false
16587 5474 Acked Write 3 Sent Done Done false
16588 5475 Acked Read 1 Sent Done Done false
16589 5476 Acked Write 1 Sent Done Done false
16590 5477 Acked Write 7 Sent Done Done false
16591 5478 Acked Read 3 Sent Done Done false
16592 5479 Acked Write 2 Sent Done Done false
16593 5480 Acked Write 7 Sent Done Done false
16594 5481 Acked Read 8 Sent Done Done false
16595 5482 Acked Read 5 Sent Done Done false
16596 5483 Acked Write 10 Sent Done Done false
16597 5484 Acked Flush 0 Sent Done Done false
16598 5485 Acked Write 4 Sent Done Done false
16599 5486 Acked Read 3 Sent Done Done false
16600 5487 Acked Read 4 Sent Done Done false
16601 5488 Acked Read 4 Sent Done Done false
16602 5489 Acked Read 5 Sent Done Done false
16603 5490 Acked Write 9 Sent Done Done false
16604 5491 Acked Read 1 Sent Done Done false
16605 5492 Acked Flush 0 Sent Done Done false
16606 5493 Acked Flush 0 Sent Done Done false
16607 5494 Acked Write 9 Sent Done Done false
16608 5495 Acked Write 7 Sent Done Done false
16609 5496 Acked Flush 0 Sent Done Done false
16610 5497 Acked Flush 0 Sent Done Done false
16611 5498 Acked Write 2 Sent Done Done false
16612 5499 Acked Read 2 Sent Done Done false
16613 5500 Acked Read 5 Sent Done Done false
16614 5501 Acked Read 2 Sent Done Done false
16615 5502 Acked Write 3 Sent Done Done false
16616 5503 Acked Write 10 Sent Done Done false
16617 5504 Acked Write 9 Sent Done Done false
16618 5505 Acked Read 8 Sent Done Done false
16619 5506 Acked Write 2 Sent Done Done false
16620 5507 Acked Flush 0 Sent Done Done false
16621 5508 Acked Read 8 Sent Done Done false
16622 5509 Acked Read 6 Sent Done Done false
16623 5510 Acked Read 7 Sent Done Done false
16624 5511 Acked Read 1 Sent Done Done false
16625 5512 Acked Write 10 Sent Done Done false
16626 5513 Acked Read 3 Sent Done Done false
16627 5514 Acked Flush 0 Sent Done Done false
16628 5515 Acked Write 7 Sent Done Done false
16629 5516 Acked Write 2 Sent Done Done false
16630 5517 Acked Read 3 Sent Done Done false
16631 5518 Acked Read 3 Sent Done Done false
16632 5519 Acked Write 4 Sent Done Done false
16633 5520 Acked Read 7 Sent Done Done false
16634 5521 Acked Read 10 Sent Done Done false
16635 5522 Acked Read 1 Sent Done Done false
16636 5523 Acked Write 7 Sent Done Done false
16637 5524 Acked Read 10 Sent Done Done false
16638 5525 Acked Write 7 Sent Done Done false
16639 5526 Acked Write 8 Sent Done Done false
16640 5527 Acked Write 7 Sent Done Done false
16641 5528 Acked Flush 0 Sent Done Done false
16642 STATES DS:0 DS:1 DS:2 TOTAL
16643 Sent 302 0 0 302
16644 Done 0 302 302 604
16645 Skipped 0 0 0 0
16646 Error 0 0 0 0
16647 Last Flush: 5226 5528 5528
16648 Downstairs last five completed: 5226 5225 5224 5223 5222
16649 Upstairs last five completed: 5528 5527 5526 5525 5524
16650 CLIENT: Up:0 ds:302 act:2
166512025-01-21T15:50:38.800ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166522025-01-21T15:50:38.800ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36168) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166532025-01-21T15:50:38.801ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166542025-01-21T15:50:38.802ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166552025-01-21T15:50:38.802ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166562025-01-21T15:50:38.802ZINFOcrucible: send last flush ID to this DS: 5226 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166572025-01-21T15:50:38.802ZINFOcrucible: Replied this last flush ID: 5226 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166582025-01-21T15:50:38.802ZINFOcrucible: [0] client re-new 302 jobs since flush 5226 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166592025-01-21T15:50:38.803ZINFOcrucible: [0] Replayed 302 jobs since flush: 5226 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
16660 ----------------------------------------------------------------
16661 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16662 Downstairs last five completed: 5528 5527 5526 5525 5524
16663 Upstairs last five completed: 5528 5527 5526 5525 5524
16664 CLIENT: Up:0 ds:0 act:3
16665 Replay: All jobs finished
16666 Main task has work to do, go find it
16667 got dsc Stop(1)
16668 stop 1
16669 Jan 21 15:50:41.052 INFO request completed, latency_us: 366, [8820] Got stop action so:false kr:false
16670 response_code: 204, uri: /stop/cid/1, method: POST, req_id: 1fd4823d-34a4-4c36-b405-c49c6a91f958, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
16671 Jan 21 15:50:41.053 INFO request completed, latency_us: 126, response_code: 200, uri: /state/cid/1, method: GET, req_id: 5b7c6f79-0b20-460f-838d-eb74d5e7435c, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
166722025-01-21T15:50:41.053ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166732025-01-21T15:50:41.053ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166742025-01-21T15:50:41.053ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
166752025-01-21T15:50:41.053ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
16676 [8820] Exited with: ExitStatus(unix_wait_status(9))
16677 [8820][1] reports Exit
16678 Jan 21 15:50:45.055 INFO request completed, latency_us: 214, response_code: 200, uri: /state/cid/1, method: GET, req_id: 468f4232-ec0c-46be-8108-a9588c3b2dd7, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
16679 001/300 Write block 2710 len 36864 data: 6 6 7 7 6 4 4 4 3
16680 002/300 Read block 1511 len 36864
16681 003/300 Write block 1484 len 8192 data: 5 5
16682 004/300 Read block 343 len 24576
16683 005/300 Read block 1029 len 16384
16684 006/300 Read block 2145 len 12288
16685 007/300 Write block 1129 len 4096 data: 3
16686 008/300 Write block 1936 len 28672 data: 5 6 4 4 3 4 3
16687 009/300 Write block 1205 len 20480 data: 3 3 2 2 2
16688 010/300 Read block 4960 len 20480
16689 011/300 Write block 4184 len 12288 data: 5 7 6
16690 012/300 Write block 4730 len 32768 data: 3 3 3 5 4 3 3 3
16691 013/300 Read block 1293 len 20480
16692 014/300 Read block 1504 len 12288
16693 015/300 Flush
16694 016/300 Flush
16695 017/300 Write block 3853 len 36864 data: 4 5 6 6 6 6 4 4 4
16696 018/300 Write block 103 len 24576 data: 2 2 2 2 2 2
16697 019/300 Write block 1548 len 20480 data: 4 3 3 2 3
16698 020/300 Flush
16699 021/300 Read block 4605 len 16384
16700 022/300 Read block 4628 len 8192
16701 023/300 Read block 1387 len 24576
16702 024/300 Read block 678 len 4096
16703 025/300 Read block 30 len 20480
16704 026/300 Write block 4581 len 4096 data: 3
16705 027/300 Write block 4941 len 8192 data: 3 4
16706 028/300 Write block 3701 len 8192 data: 6 5
16707 029/300 Read block 3402 len 32768
16708 030/300 Read block 2994 len 40960
16709 031/300 Read block 4029 len 24576
16710 032/300 Read block 4914 len 36864
16711 033/300 Write block 148 len 28672 data: 3 3 3 3 2 2 2
16712 034/300 Read block 4581 len 36864
16713 035/300 Read block 151 len 36864
16714 036/300 Write block 1512 len 20480 data: 6 4 5 5 5
16715 037/300 Read block 3867 len 20480
16716 038/300 Write block 770 len 28672 data: 4 4 6 6 7 6 5
16717 039/300 Write block 963 len 28672 data: 3 3 3 4 6 4 4
16718 040/300 Flush
16719 041/300 Write block 3129 len 16384 data: 3 2 2 2
16720 042/300 Write block 3702 len 32768 data: 6 6 5 4 2 4 5 5
16721 043/300 Write block 735 len 28672 data: 5 5 6 6 7 6 4
16722 044/300 Write block 3038 len 4096 data: 5
16723 045/300 Read block 4649 len 24576
16724 046/300 Read block 2338 len 32768
16725 047/300 Read block 4535 len 36864
16726 048/300 Read block 602 len 12288
16727 049/300 Read block 3776 len 40960
16728 050/300 Read block 653 len 16384
16729 051/300 Read block 2937 len 8192
16730 052/300 Read block 1313 len 8192
16731 053/300 Write block 3270 len 16384 data: 6 6 4 4
16732 054/300 Write block 1564 len 36864 data: 4 4 4 3 4 4 4 4 4
16733 055/300 Read block 2613 len 20480
16734 056/300 Read block 4133 len 32768
16735 057/300 Write block 3919 len 16384 data: 5 4 2 3
16736 058/300 Read block 4468 len 28672
16737 059/300 Write block 1477 len 12288 data: 5 4 4
16738 060/300 Read block 2398 len 8192
16739 061/300 Write block 1707 len 40960 data: 5 4 4 3 3 5 4 3 3 3
16740 062/300 Read block 2244 len 12288
16741 063/300 Read block 4060 len 20480
16742 064/300 Write block 3295 len 16384 data: 3 5 7 5
16743 065/300 Flush
16744 066/300 Flush
16745 067/300 Read block 3228 len 4096
16746 068/300 Flush
16747 069/300 Read block 2667 len 36864
16748 070/300 Read block 842 len 12288
16749 071/300 Read block 2744 len 28672
16750 072/300 Read block 1849 len 40960
16751 073/300 Write block 401 len 32768 data: 3 3 3 5 5 5 6 5
16752 074/300 Write block 2592 len 20480 data: 4 3 2 2 3
16753 075/300 Read block 3262 len 16384
16754 076/300 Write block 3913 len 8192 data: 5 5
16755 077/300 Read block 3427 len 40960
16756 078/300 Read block 4553 len 4096
16757 079/300 Flush
16758 080/300 Read block 4520 len 12288
16759 081/300 Read block 3808 len 8192
16760 082/300 Write block 4214 len 36864 data: 2 3 4 6 6 6 6 4 3
16761 083/300 Write block 3914 len 12288 data: 6 6 6
16762 084/300 Write block 1170 len 36864 data: 4 4 3 3 3 3 2 2 2
16763 085/300 Flush
16764 086/300 Flush
16765 087/300 Write block 4983 len 36864 data: 4 4 6 8 9 9 8 8 6
16766 088/300 Read block 874 len 20480
16767 089/300 Read block 4609 len 4096
16768 090/300 Flush
16769 091/300 Read block 2295 len 32768
16770 092/300 Write block 2334 len 12288 data: 5 4 3
16771 093/300 Write block 527 len 24576 data: 6 7 9 8 6 4
16772 094/300 Flush
16773 095/300 Read block 4476 len 28672
16774 096/300 Read block 4867 len 4096
16775 097/300 Read block 4954 len 20480
16776 098/300 Read block 3914 len 28672
16777 099/300 Flush
16778 100/300 Read block 4213 len 16384
16779 101/300 Read block 3774 len 16384
16780 102/300 Flush
16781 103/300 Write block 2921 len 8192 data: 3 3
16782 104/300 Write block 1540 len 40960 data: 4 4 6 6 5 5 4 4 5 4
16783 105/300 Write block 4826 len 40960 data: 3 3 4 4 4 3 3 3 3 3
16784 106/300 Read block 4925 len 32768
16785 107/300 Read block 2584 len 20480
16786 108/300 Write block 4504 len 28672 data: 3 3 3 2 3 2 2
16787 109/300 Flush
16788 110/300 Write block 4675 len 24576 data: 4 3 3 3 3 5
16789 111/300 Read block 1140 len 28672
16790 112/300 Read block 4189 len 16384
16791 113/300 Flush
16792 114/300 Read block 2565 len 12288
16793 115/300 Read block 1253 len 8192
16794 116/300 Write block 4597 len 12288 data: 4 2 2
16795 117/300 Flush
16796 118/300 Read block 3236 len 20480
16797 119/300 Read block 1673 len 4096
16798 120/300 Write block 3463 len 40960 data: 5 4 3 3 3 6 6 6 5 5
16799 121/300 Write block 570 len 24576 data: 3 3 3 4 3 3
16800 122/300 Flush
16801 123/300 Write block 2093 len 16384 data: 2 3 5 5
16802 124/300 Write block 3951 len 24576 data: 5 6 5 4 3 3
16803 125/300 Flush
16804 126/300 Write block 4584 len 40960 data: 3 5 5 5 5 4 3 3 3 3
16805 127/300 Flush
16806 128/300 Write block 863 len 16384 data: 6 6 6 6
16807 129/300 Read block 889 len 8192
16808 130/300 Read block 3279 len 32768
16809 131/300 Read block 1999 len 24576
16810 132/300 Read block 2889 len 28672
16811 133/300 Read block 3458 len 24576
16812 134/300 Read block 1299 len 8192
16813 135/300 Read block 2146 len 4096
16814 136/300 Write block 3280 len 16384 data: 4 4 4 5
16815 137/300 Read block 1232 len 36864
16816 138/300 Read block 1039 len 16384
16817 139/300 Read block 3695 len 24576
16818 140/300 Read block 3395 len 4096
16819 141/300 Read block 3707 len 36864
16820 142/300 Write block 1859 len 36864 data: 3 4 4 5 6 6 7 6 5
16821 143/300 Read block 1812 len 32768
16822 144/300 Read block 3523 len 8192
16823 145/300 Flush
16824 146/300 Write block 2529 len 24576 data: 5 6 6 6 6 6
16825 147/300 Read block 2635 len 4096
16826 148/300 Read block 4315 len 32768
16827 149/300 Write block 847 len 36864 data: 4 3 4 4 3 4 3 4 6
16828 150/300 Read block 4821 len 16384
16829 151/300 Read block 4032 len 20480
16830 152/300 Read block 2279 len 12288
16831 153/300 Read block 2607 len 28672
16832 154/300 Write block 3768 len 4096 data: 3
16833 155/300 Write block 3853 len 16384 data: 5 6 7 7
16834 156/300 Read block 2597 len 32768
16835 157/300 Read block 929 len 20480
16836 158/300 Write block 4618 len 12288 data: 3 3 4
16837 159/300 Flush
16838 160/300 Read block 2301 len 4096
16839 161/300 Read block 3007 len 16384
16840 162/300 Write block 281 len 16384 data: 3 3 4 5
16841 163/300 Read block 965 len 12288
16842 164/300 Read block 3738 len 12288
16843 165/300 Read block 3387 len 24576
16844 166/300 Write block 1304 len 12288 data: 4 4 3
16845 167/300 Read block 1569 len 32768
16846 168/300 Read block 2079 len 4096
16847 169/300 Read block 2173 len 40960
16848 170/300 Read block 812 len 32768
16849 171/300 Write block 4903 len 36864 data: 4 4 4 3 3 4 5 6 6
16850 172/300 Write block 1356 len 12288 data: 3 3 3
16851 173/300 Write block 804 len 32768 data: 6 6 6 6 5 5 5 6
16852 174/300 Write block 473 len 24576 data: 3 2 2 2 2 2
16853 175/300 Write block 163 len 32768 data: 4 4 4 4 4 4 4 4
16854 176/300 Read block 3674 len 40960
16855 177/300 Write block 2850 len 4096 data: 3
16856 178/300 Write block 3853 len 16384 data: 6 7 8 8
16857 179/300 Write block 718 len 40960 data: 5 5 4 3 4 3 3 3 3 3
16858 180/300 Write block 3809 len 40960 data: 4 4 4 4 4 5 5 5 4 3
16859 181/300 Write block 1746 len 36864 data: 3 3 3 4 4 5 5 5 4
16860 182/300 Flush
16861 183/300 Write block 1108 len 28672 data: 5 6 5 5 6 4 2
16862 184/300 Read block 1364 len 20480
16863 185/300 Read block 2108 len 28672
16864 186/300 Write block 1493 len 28672 data: 3 3 2 3 4 2 3
16865 187/300 Write block 1888 len 8192 data: 4 4
16866 188/300 Write block 2023 len 36864 data: 4 4 5 4 6 5 5 4 5
16867 189/300 Read block 282 len 16384
16868 190/300 Read block 2430 len 40960
16869 191/300 Read block 3571 len 28672
16870 192/300 Write block 4395 len 16384 data: 5 3 5 5
16871 193/300 Flush
16872 194/300 Read block 4262 len 28672
16873 195/300 Flush
16874 196/300 Write block 1870 len 28672 data: 4 4 4 5 4 4 4
16875 197/300 Write block 3768 len 16384 data: 4 3 3 4
16876 198/300 Write block 1833 len 32768 data: 3 3 4 4 5 4 5 5
16877 199/300 Write block 3605 len 28672 data: 2 3 4 4 4 4 3
16878 200/300 Read block 4792 len 16384
16879 201/300 Read block 3070 len 4096
16880 202/300 Write block 3667 len 12288 data: 4 5 6
16881 203/300 Write block 4723 len 8192 data: 6 6
16882 204/300 Read block 3550 len 4096
16883 205/300 Flush
16884 206/300 Read block 1427 len 32768
16885 207/300 Read block 4217 len 40960
16886 208/300 Write block 1226 len 24576 data: 4 4 3 3 3 3
16887 209/300 Flush
16888 210/300 Write block 4355 len 20480 data: 4 4 4 4 4
16889 211/300 Write block 1933 len 40960 data: 6 6 6 6 7 5 5 4 5 4
16890 212/300 Write block 4312 len 24576 data: 4 4 4 6 5 3
16891 213/300 Write block 4678 len 32768 data: 4 4 6 4 4 4 3 3
16892 214/300 Read block 4294 len 12288
16893 215/300 Flush
16894 216/300 Read block 4101 len 20480
16895 217/300 Read block 2423 len 8192
16896 218/300 Flush
16897 219/300 Read block 280 len 24576
16898 220/300 Flush
16899 221/300 Read block 682 len 8192
16900 222/300 Write block 2277 len 28672 data: 3 3 3 4 3 3 3
16901 223/300 Write block 4794 len 8192 data: 2 4
16902 224/300 Write block 4171 len 40960 data: 6 6 6 6 5 4 5 6 7 6
16903 225/300 Flush
16904 226/300 Read block 3024 len 16384
16905 227/300 Flush
16906 228/300 Write block 4944 len 28672 data: 3 3 2 2 2 2 3
16907 229/300 Read block 4040 len 4096
16908 230/300 Write block 4110 len 8192 data: 5 4
16909 231/300 Write block 4357 len 28672 data: 5 5 5 3 3 4 3
16910 232/300 Read block 3452 len 20480
16911 233/300 Write block 4775 len 8192 data: 4 3
16912 234/300 Read block 3593 len 28672
16913 235/300 Flush
16914 236/300 Read block 4725 len 28672
16915 237/300 Write block 2383 len 32768 data: 2 2 2 2 2 2 3 4
16916 238/300 Write block 1526 len 8192 data: 4 4
16917 239/300 Read block 8 len 36864
16918 240/300 Write block 3303 len 16384 data: 4 5 4 3
16919 241/300 Read block 1769 len 20480
16920 242/300 Flush
16921 243/300 Read block 4960 len 40960
16922 244/300 Write block 3471 len 32768 data: 6 6 5 5 7 8 9 6
16923 245/300 Read block 4407 len 20480
16924 246/300 Read block 1514 len 16384
16925 247/300 Write block 1510 len 28672 data: 6 7 7 5 6 6 6
16926 248/300 Write block 1055 len 36864 data: 5 6 7 5 4 4 4 4 4
16927 249/300 Write block 1361 len 4096 data: 3
16928 250/300 Write block 2751 len 12288 data: 5 4 6
16929 251/300 Write block 40 len 40960 data: 6 6 7 6 5 6 5 6 5 6
16930 252/300 Write block 661 len 8192 data: 2 2
16931 253/300 Read block 286 len 32768
16932 254/300 Read block 3435 len 36864
16933 255/300 Read block 4199 len 4096
16934 256/300 Write block 589 len 36864 data: 4 4 5 6 7 6 6 5 4
16935 257/300 Read block 1178 len 24576
16936 258/300 Flush
16937 259/300 Write block 2952 len 28672 data: 5 4 4 4 5 6 6
16938 260/300 Flush
16939 261/300 Write block 3736 len 40960 data: 4 3 3 3 3 2 2 2 2 3
16940 262/300 Write block 4214 len 24576 data: 3 4 5 7 7 7
16941 263/300 Write block 3512 len 28672 data: 3 4 4 4 4 4 4
16942 264/300 Write block 4362 len 8192 data: 5 4
16943 265/300 Read block 1422 len 12288
16944 266/300 Read block 3165 len 4096
16945 267/300 Write block 881 len 12288 data: 4 4 4
16946 268/300 Flush
16947 269/300 Read block 1518 len 32768
16948 270/300 Write block 3559 len 4096 data: 4
16949 271/300 Read block 275 len 16384
16950 272/300 Read block 4286 len 12288
16951 273/300 Flush
16952 274/300 Read block 4529 len 12288
16953 275/300 Write block 4268 len 20480 data: 5 6 6 6 5
16954 276/300 Write block 1736 len 28672 data: 5 7 8 7 6 6 6
16955 277/300 Write block 2344 len 20480 data: 4 3 3 4 5
16956 278/300 Read block 4508 len 4096
16957 279/300 Flush
16958 280/300 Read block 2806 len 40960
16959 281/300 Write block 1029 len 12288 data: 3 3 3
16960 282/300 Read block 453 len 28672
16961 283/300 Read block 3191 len 36864
16962 284/300 Read block 2575 len 32768
16963 285/300 Write block 1259 len 28672 data: 4 4 4 3 3 3 5
16964 286/300 Read block 4116 len 24576
16965 287/300 Read block 4869 len 32768
16966 288/300 Write block 2628 len 12288 data: 7 8 7
16967 289/300 Write block 2344 len 32768 data: 5 4 4 5 6 5 3 5
16968 290/300 Write block 3501 len 16384 data: 2 2 3 3
16969 291/300 Flush
16970 292/300 Write block 3580 len 12288 data: 5 5 4
16971 293/300 Write block 162 len 24576 data: 3 5 5 5 5 5
16972 294/300 Read block 1422 len 36864
16973 295/300 Write block 1914 len 24576 data: 3 4 4 5 6 4
16974 296/300 Read block 1036 len 20480
16975 297/300 Read block 2416 len 4096
16976 298/300 Write block 2515 len 20480 data: 2 2 2 2 2
16977 299/300 Write block 2714 len 32768 data: 7 5 5 5 4 3 3 4
16978 300/300 Flush
16979 301/300 Read block 3180 len 24576
16980 Main task has work to do, go find it
16981 got dsc Start(1)
16982 start 1
16983 Jan 21 15:50:45.287 INFO request completed, latency_us: 232, response_code: 204, uri: /start/cid/1, method: POST, req_id: f7eea589-5f3e-4102-9e85-2caf87f33197, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
16984 Replay: started 1, returned:Ok(())
16985 ----------------------------------------------------------------
16986 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16987 States: Active Offline Active
16988 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16989 5529 Acked Write 9 Done Sent Done false
16990 5530 Acked Read 9 Done Sent Done false
16991 5531 Acked Write 2 Done Sent Done false
16992 5532 Acked Read 6 Done Sent Done false
16993 5533 Acked Read 4 Done Sent Done false
16994 5534 Acked Read 3 Done Sent Done false
16995 5535 Acked Write 1 Done Sent Done false
16996 5536 Acked Write 7 Done Sent Done false
16997 5537 Acked Write 5 Done Sent Done false
16998 5538 Acked Read 5 Done Sent Done false
16999 5539 Acked Write 3 Done Sent Done false
17000 5540 Acked Write 8 Done Sent Done false
17001 5541 Acked Read 5 Done Sent Done false
17002 5542 Acked Read 3 Done Sent Done false
17003 5543 Acked Flush 0 Done Sent Done false
17004 5544 Acked Flush 0 Done Sent Done false
17005 5545 Acked Write 9 Done Sent Done false
17006 5546 Acked Write 6 Done Sent Done false
17007 5547 Acked Write 5 Done Sent Done false
17008 5548 Acked Flush 0 Done Sent Done false
17009 5549 Acked Read 4 Done Sent Done false
17010 5550 Acked Read 2 Done Sent Done false
17011 5551 Acked Read 6 Done Sent Done false
17012 5552 Acked Read 1 Done Sent Done false
17013 5553 Acked Read 5 Done Sent Done false
17014 5554 Acked Write 1 Done Sent Done false
17015 5555 Acked Write 2 Done Sent Done false
17016 5556 Acked Write 2 Done Sent Done false
17017 5557 Acked Read 8 Done Sent Done false
17018 5558 Acked Read 10 Done Sent Done false
17019 5559 Acked Read 6 Done Sent Done false
17020 5560 Acked Read 9 Done Sent Done false
17021 5561 Acked Write 7 Done Sent Done false
17022 5562 Acked Read 9 Done Sent Done false
17023 5563 Acked Read 9 Done Sent Done false
17024 5564 Acked Write 5 Done Sent Done false
17025 5565 Acked Read 5 Done Sent Done false
17026 5566 Acked Write 7 Done Sent Done false
17027 5567 Acked Write 7 Done Sent Done false
17028 5568 Acked Flush 0 Done Sent Done false
17029 5569 Acked Write 4 Done Sent Done false
17030 5570 Acked Write 8 Done Sent Done false
17031 5571 Acked Write 7 Done Sent Done false
17032 5572 Acked Write 1 Done Sent Done false
17033 5573 Acked Read 6 Done Sent Done false
17034 5574 Acked Read 8 Done Sent Done false
17035 5575 Acked Read 9 Done Sent Done false
17036 5576 Acked Read 3 Done Sent Done false
17037 5577 Acked Read 10 Done Sent Done false
17038 5578 Acked Read 4 Done Sent Done false
17039 5579 Acked Read 2 Done Sent Done false
17040 5580 Acked Read 2 Done Sent Done false
17041 5581 Acked Write 4 Done Sent Done false
17042 5582 Acked Write 9 Done Sent Done false
17043 5583 Acked Read 5 Done Sent Done false
17044 5584 Acked Read 8 Done Sent Done false
17045 5585 Acked Write 4 Done Sent Done false
17046 5586 Acked Read 7 Done Sent Done false
17047 5587 Acked Write 3 Done Sent Done false
17048 5588 Acked Read 2 Done Sent Done false
17049 5589 Acked Write 10 Done Sent Done false
17050 5590 Acked Read 3 Done Sent Done false
17051 5591 Acked Read 5 Done Sent Done false
17052 5592 Acked Write 4 Done Sent Done false
17053 5593 Acked Flush 0 Done Sent Done false
17054 5594 Acked Flush 0 Done Sent Done false
17055 5595 Acked Read 1 Done Sent Done false
17056 5596 Acked Flush 0 Done Sent Done false
17057 5597 Acked Read 9 Done Sent Done false
17058 5598 Acked Read 3 Done Sent Done false
17059 5599 Acked Read 7 Done Sent Done false
17060 5600 Acked Read 10 Done Sent Done false
17061 5601 Acked Write 8 Done Sent Done false
17062 5602 Acked Write 5 Done Sent Done false
17063 5603 Acked Read 4 Done Sent Done false
17064 5604 Acked Write 2 Done Sent Done false
17065 5605 Acked Read 10 Done Sent Done false
17066 5606 Acked Read 1 Done Sent Done false
17067 5607 Acked Flush 0 Done Sent Done false
17068 5608 Acked Read 3 Done Sent Done false
17069 5609 Acked Read 2 Done Sent Done false
17070 5610 Acked Write 9 Done Sent Done false
17071 5611 Acked Write 3 Done Sent Done false
17072 5612 Acked Write 9 Done Sent Done false
17073 5613 Acked Flush 0 Done Sent Done false
17074 5614 Acked Flush 0 Done Sent Done false
17075 5615 Acked Write 9 Done Sent Done false
17076 5616 Acked Read 5 Done Sent Done false
17077 5617 Acked Read 1 Done Sent Done false
17078 5618 Acked Flush 0 Done Sent Done false
17079 5619 Acked Read 8 Done Sent Done false
17080 5620 Acked Write 3 Done Sent Done false
17081 5621 Acked Write 6 Done Sent Done false
17082 5622 Acked Flush 0 Done Sent Done false
17083 5623 Acked Read 7 Done Sent Done false
17084 5624 Acked Read 1 Done Sent Done false
17085 5625 Acked Read 5 Done Sent Done false
17086 5626 Acked Read 7 Done Sent Done false
17087 5627 Acked Flush 0 Done Sent Done false
17088 5628 Acked Read 4 Done Sent Done false
17089 5629 Acked Read 4 Done Sent Done false
17090 5630 Acked Flush 0 Done Sent Done false
17091 5631 Acked Write 2 Done Sent Done false
17092 5632 Acked Write 10 Done Sent Done false
17093 5633 Acked Write 10 Done Sent Done false
17094 5634 Acked Read 8 Done Sent Done false
17095 5635 Acked Read 5 Done Sent Done false
17096 5636 Acked Write 7 Done Sent Done false
17097 5637 Acked Flush 0 Done Sent Done false
17098 5638 Acked Write 6 Done Sent Done false
17099 5639 Acked Read 7 Done Sent Done false
17100 5640 Acked Read 4 Done Sent Done false
17101 5641 Acked Flush 0 Done Sent Done false
17102 5642 Acked Read 3 Done Sent Done false
17103 5643 Acked Read 2 Done Sent Done false
17104 5644 Acked Write 3 Done Sent Done false
17105 5645 Acked Flush 0 Done Sent Done false
17106 5646 Acked Read 5 Done Sent Done false
17107 5647 Acked Read 1 Done Sent Done false
17108 5648 Acked Write 10 Done Sent Done false
17109 5649 Acked Write 6 Done Sent Done false
17110 5650 Acked Flush 0 Done Sent Done false
17111 5651 Acked Write 4 Done Sent Done false
17112 5652 Acked Write 6 Done Sent Done false
17113 5653 Acked Flush 0 Done Sent Done false
17114 5654 Acked Write 10 Done Sent Done false
17115 5655 Acked Flush 0 Done Sent Done false
17116 5656 Acked Write 4 Done Sent Done false
17117 5657 Acked Read 2 Done Sent Done false
17118 5658 Acked Read 8 Done Sent Done false
17119 5659 Acked Read 6 Done Sent Done false
17120 5660 Acked Read 7 Done Sent Done false
17121 5661 Acked Read 6 Done Sent Done false
17122 5662 Acked Read 2 Done Sent Done false
17123 5663 Acked Read 1 Done Sent Done false
17124 5664 Acked Write 4 Done Sent Done false
17125 5665 Acked Read 9 Done Sent Done false
17126 5666 Acked Read 4 Done Sent Done false
17127 5667 Acked Read 6 Done Sent Done false
17128 5668 Acked Read 1 Done Sent Done false
17129 5669 Acked Read 9 Done Sent Done false
17130 5670 Acked Write 9 Done Sent Done false
17131 5671 Acked Read 8 Done Sent Done false
17132 5672 Acked Read 2 Done Sent Done false
17133 5673 Acked Flush 0 Done Sent Done false
17134 5674 Acked Write 6 Done Sent Done false
17135 5675 Acked Read 1 Done Sent Done false
17136 5676 Acked Read 8 Done Sent Done false
17137 5677 Acked Write 9 Done Sent Done false
17138 5678 Acked Read 4 Done Sent Done false
17139 5679 Acked Read 5 Done Sent Done false
17140 5680 Acked Read 3 Done Sent Done false
17141 5681 Acked Read 7 Done Sent Done false
17142 5682 Acked Write 1 Done Sent Done false
17143 5683 Acked Write 4 Done Sent Done false
17144 5684 Acked Read 8 Done Sent Done false
17145 5685 Acked Read 5 Done Sent Done false
17146 5686 Acked Write 3 Done Sent Done false
17147 5687 Acked Flush 0 Done Sent Done false
17148 5688 Acked Read 1 Done Sent Done false
17149 5689 Acked Read 4 Done Sent Done false
17150 5690 Acked Write 4 Done Sent Done false
17151 5691 Acked Read 3 Done Sent Done false
17152 5692 Acked Read 3 Done Sent Done false
17153 5693 Acked Read 6 Done Sent Done false
17154 5694 Acked Write 3 Done Sent Done false
17155 5695 Acked Read 8 Done Sent Done false
17156 5696 Acked Read 1 Done Sent Done false
17157 5697 Acked Read 10 Done Sent Done false
17158 5698 Acked Read 8 Done Sent Done false
17159 5699 Acked Write 9 Done Sent Done false
17160 5700 Acked Write 3 Done Sent Done false
17161 5701 Acked Write 8 Done Sent Done false
17162 5702 Acked Write 6 Done Sent Done false
17163 5703 Acked Write 8 Done Sent Done false
17164 5704 Acked Read 10 Done Sent Done false
17165 5705 Acked Write 1 Done Sent Done false
17166 5706 Acked Write 4 Done Sent Done false
17167 5707 Acked Write 10 Done Sent Done false
17168 5708 Acked Write 10 Done Sent Done false
17169 5709 Acked Write 9 Done Sent Done false
17170 5710 Acked Flush 0 Done Sent Done false
17171 5711 Acked Write 7 Done Sent Done false
17172 5712 Acked Read 5 Done Sent Done false
17173 5713 Acked Read 7 Done Sent Done false
17174 5714 Acked Write 7 Done Sent Done false
17175 5715 Acked Write 2 Done Sent Done false
17176 5716 Acked Write 9 Done Sent Done false
17177 5717 Acked Read 4 Done Sent Done false
17178 5718 Acked Read 10 Done Sent Done false
17179 5719 Acked Read 7 Done Sent Done false
17180 5720 Acked Write 4 Done Sent Done false
17181 5721 Acked Flush 0 Done Sent Done false
17182 5722 Acked Read 7 Done Sent Done false
17183 5723 Acked Flush 0 Done Sent Done false
17184 5724 Acked Write 7 Done Sent Done false
17185 5725 Acked Write 4 Done Sent Done false
17186 5726 Acked Write 8 Done Sent Done false
17187 5727 Acked Write 7 Done Sent Done false
17188 5728 Acked Read 4 Done Sent Done false
17189 5729 Acked Read 1 Done Sent Done false
17190 5730 Acked Write 3 Done Sent Done false
17191 5731 Acked Write 2 Done Sent Done false
17192 5732 Acked Read 1 Done Sent Done false
17193 5733 Acked Flush 0 Done Sent Done false
17194 5734 Acked Read 8 Done Sent Done false
17195 5735 Acked Read 10 Done Sent Done false
17196 5736 Acked Write 6 Done Sent Done false
17197 5737 Acked Flush 0 Done Sent Done false
17198 5738 Acked Write 5 Done Sent Done false
17199 5739 Acked Write 10 Done Sent Done false
17200 5740 Acked Write 6 Done Sent Done false
17201 5741 Acked Write 8 Done Sent Done false
17202 5742 Acked Read 3 Done Sent Done false
17203 5743 Acked Flush 0 Done Sent Done false
17204 5744 Acked Read 5 Done Sent Done false
17205 5745 Acked Read 2 Done Sent Done false
17206 5746 Acked Flush 0 Done Sent Done false
17207 5747 Acked Read 6 Done Sent Done false
17208 5748 Acked Flush 0 Done Sent Done false
17209 5749 Acked Read 2 Done Sent Done false
17210 5750 Acked Write 7 Done Sent Done false
17211 5751 Acked Write 2 Done Sent Done false
17212 5752 Acked Write 10 Done Sent Done false
17213 5753 Acked Flush 0 Done Sent Done false
17214 5754 Acked Read 4 Done Sent Done false
17215 5755 Acked Flush 0 Done Sent Done false
17216 5756 Acked Write 7 Done Sent Done false
17217 5757 Acked Read 1 Done Sent Done false
17218 5758 Acked Write 2 Done Sent Done false
17219 5759 Acked Write 7 Done Sent Done false
17220 5760 Acked Read 5 Done Sent Done false
17221 5761 Acked Write 2 Done Sent Done false
17222 5762 Acked Read 7 Done Sent Done false
17223 5763 Acked Flush 0 Done Sent Done false
17224 5764 Acked Read 7 Done Sent Done false
17225 5765 Acked Write 8 Done Sent Done false
17226 5766 Acked Write 2 Done Sent Done false
17227 5767 Acked Read 9 Done Sent Done false
17228 5768 Acked Write 4 Done Sent Done false
17229 5769 Acked Read 5 Done Sent Done false
17230 5770 Acked Flush 0 Done Sent Done false
17231 5771 Acked Read 10 Done Sent Done false
17232 5772 Acked Write 8 Done Sent Done false
17233 5773 Acked Read 5 Done Sent Done false
17234 5774 Acked Read 4 Done Sent Done false
17235 5775 Acked Write 7 Done Sent Done false
17236 5776 Acked Write 9 Done Sent Done false
17237 5777 Acked Write 1 Done Sent Done false
17238 5778 Acked Write 3 Done Sent Done false
17239 5779 Acked Write 10 Done Sent Done false
17240 5780 Acked Write 2 Done Sent Done false
17241 5781 Acked Read 8 Done Sent Done false
17242 5782 Acked Read 9 Done Sent Done false
17243 5783 Acked Read 1 Done Sent Done false
17244 5784 Acked Write 9 Done Sent Done false
17245 5785 Acked Read 6 Done Sent Done false
17246 5786 Acked Flush 0 Done Sent Done false
17247 5787 Acked Write 7 Done Sent Done false
17248 5788 Acked Flush 0 Done Sent Done false
17249 5789 Acked Write 10 Done Sent Done false
17250 5790 Acked Write 6 Done Sent Done false
17251 5791 Acked Write 7 Done Sent Done false
17252 5792 Acked Write 2 Done Sent Done false
17253 5793 Acked Read 3 Done Sent Done false
17254 5794 Acked Read 1 Done Sent Done false
17255 5795 Acked Write 3 Done Sent Done false
17256 5796 Acked Flush 0 Done Sent Done false
17257 5797 Acked Read 8 Done Sent Done false
17258 5798 Acked Write 1 Done Sent Done false
17259 5799 Acked Read 4 Done Sent Done false
17260 5800 Acked Read 3 Done Sent Done false
17261 5801 Acked Flush 0 Done Sent Done false
17262 5802 Acked Read 3 Done Sent Done false
17263 5803 Acked Write 5 Done Sent Done false
17264 5804 Acked Write 7 Done Sent Done false
17265 5805 Acked Write 5 Done Sent Done false
17266 5806 Acked Read 1 Done Sent Done false
17267 5807 Acked Flush 0 Done Sent Done false
17268 5808 Acked Read 10 Done Sent Done false
17269 5809 Acked Write 3 Done Sent Done false
17270 5810 Acked Read 7 Done Sent Done false
17271 5811 Acked Read 9 Done Sent Done false
17272 5812 Acked Read 8 Done Sent Done false
17273 5813 Acked Write 7 Done Sent Done false
17274 5814 Acked Read 6 Done Sent Done false
17275 5815 Acked Read 8 Done Sent Done false
17276 5816 Acked Write 3 Done Sent Done false
17277 5817 Acked Write 8 Done Sent Done false
17278 5818 Acked Write 4 Done Sent Done false
17279 5819 Acked Flush 0 Done Sent Done false
17280 5820 Acked Write 3 Done Sent Done false
17281 5821 Acked Write 6 Done Sent Done false
17282 5822 Acked Read 9 Done Sent Done false
17283 5823 Acked Write 6 Done Sent Done false
17284 5824 Acked Read 5 Done Sent Done false
17285 5825 Acked Read 1 Done Sent Done false
17286 5826 Acked Write 5 Done Sent Done false
17287 5827 Acked Write 8 Done Sent Done false
17288 5828 Acked Flush 0 Done Sent Done false
17289 5829 Acked Read 6 Done Sent Done false
17290 STATES DS:0 DS:1 DS:2 TOTAL
17291 Sent 0 301 0 301
17292 Done 301 0 301 602
17293 Skipped 0 0 0 0
17294 Error 0 0 0 0
17295 Last Flush: 5828 5528 5828
17296 Downstairs last five completed: 5528 5527 5526 5525 5524
17297 Upstairs last five completed: 5829 5828 5827 5826 5825
17298 CLIENT: Up:0 ds:301 act:2
17299 [8820] got start action
17300 [8820] I am going to restart
17301 Starting downstairs at port 8820
17302 Make output file at "/tmp/dsc/downstairs-8820.txt"
17303 [8820][1] reports Starting
17304 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1216)
17305 [8820][1] reports Running
17306 ----------------------------------------------------------------
17307 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17308 States: Active Offline Active
17309 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17310 5529 Acked Write 9 Done Sent Done false
17311 5530 Acked Read 9 Done Sent Done false
17312 5531 Acked Write 2 Done Sent Done false
17313 5532 Acked Read 6 Done Sent Done false
17314 5533 Acked Read 4 Done Sent Done false
17315 5534 Acked Read 3 Done Sent Done false
17316 5535 Acked Write 1 Done Sent Done false
17317 5536 Acked Write 7 Done Sent Done false
17318 5537 Acked Write 5 Done Sent Done false
17319 5538 Acked Read 5 Done Sent Done false
17320 5539 Acked Write 3 Done Sent Done false
17321 5540 Acked Write 8 Done Sent Done false
17322 5541 Acked Read 5 Done Sent Done false
17323 5542 Acked Read 3 Done Sent Done false
17324 5543 Acked Flush 0 Done Sent Done false
17325 5544 Acked Flush 0 Done Sent Done false
17326 5545 Acked Write 9 Done Sent Done false
17327 5546 Acked Write 6 Done Sent Done false
17328 5547 Acked Write 5 Done Sent Done false
17329 5548 Acked Flush 0 Done Sent Done false
17330 5549 Acked Read 4 Done Sent Done false
17331 5550 Acked Read 2 Done Sent Done false
17332 5551 Acked Read 6 Done Sent Done false
17333 5552 Acked Read 1 Done Sent Done false
17334 5553 Acked Read 5 Done Sent Done false
17335 5554 Acked Write 1 Done Sent Done false
17336 5555 Acked Write 2 Done Sent Done false
17337 5556 Acked Write 2 Done Sent Done false
17338 5557 Acked Read 8 Done Sent Done false
17339 5558 Acked Read 10 Done Sent Done false
17340 5559 Acked Read 6 Done Sent Done false
17341 5560 Acked Read 9 Done Sent Done false
17342 5561 Acked Write 7 Done Sent Done false
17343 5562 Acked Read 9 Done Sent Done false
17344 5563 Acked Read 9 Done Sent Done false
17345 5564 Acked Write 5 Done Sent Done false
17346 5565 Acked Read 5 Done Sent Done false
17347 5566 Acked Write 7 Done Sent Done false
17348 5567 Acked Write 7 Done Sent Done false
17349 5568 Acked Flush 0 Done Sent Done false
17350 5569 Acked Write 4 Done Sent Done false
17351 5570 Acked Write 8 Done Sent Done false
17352 5571 Acked Write 7 Done Sent Done false
17353 5572 Acked Write 1 Done Sent Done false
17354 5573 Acked Read 6 Done Sent Done false
17355 5574 Acked Read 8 Done Sent Done false
17356 5575 Acked Read 9 Done Sent Done false
17357 5576 Acked Read 3 Done Sent Done false
17358 5577 Acked Read 10 Done Sent Done false
17359 5578 Acked Read 4 Done Sent Done false
17360 5579 Acked Read 2 Done Sent Done false
17361 5580 Acked Read 2 Done Sent Done false
17362 5581 Acked Write 4 Done Sent Done false
17363 5582 Acked Write 9 Done Sent Done false
17364 5583 Acked Read 5 Done Sent Done false
17365 5584 Acked Read 8 Done Sent Done false
17366 5585 Acked Write 4 Done Sent Done false
17367 5586 Acked Read 7 Done Sent Done false
17368 5587 Acked Write 3 Done Sent Done false
17369 5588 Acked Read 2 Done Sent Done false
17370 5589 Acked Write 10 Done Sent Done false
17371 5590 Acked Read 3 Done Sent Done false
17372 5591 Acked Read 5 Done Sent Done false
17373 5592 Acked Write 4 Done Sent Done false
17374 5593 Acked Flush 0 Done Sent Done false
17375 5594 Acked Flush 0 Done Sent Done false
17376 5595 Acked Read 1 Done Sent Done false
17377 5596 Acked Flush 0 Done Sent Done false
17378 5597 Acked Read 9 Done Sent Done false
17379 5598 Acked Read 3 Done Sent Done false
17380 5599 Acked Read 7 Done Sent Done false
17381 5600 Acked Read 10 Done Sent Done false
17382 5601 Acked Write 8 Done Sent Done false
17383 5602 Acked Write 5 Done Sent Done false
17384 5603 Acked Read 4 Done Sent Done false
17385 5604 Acked Write 2 Done Sent Done false
17386 5605 Acked Read 10 Done Sent Done false
17387 5606 Acked Read 1 Done Sent Done false
17388 5607 Acked Flush 0 Done Sent Done false
17389 5608 Acked Read 3 Done Sent Done false
17390 5609 Acked Read 2 Done Sent Done false
17391 5610 Acked Write 9 Done Sent Done false
17392 5611 Acked Write 3 Done Sent Done false
17393 5612 Acked Write 9 Done Sent Done false
17394 5613 Acked Flush 0 Done Sent Done false
17395 5614 Acked Flush 0 Done Sent Done false
17396 5615 Acked Write 9 Done Sent Done false
17397 5616 Acked Read 5 Done Sent Done false
17398 5617 Acked Read 1 Done Sent Done false
17399 5618 Acked Flush 0 Done Sent Done false
17400 5619 Acked Read 8 Done Sent Done false
17401 5620 Acked Write 3 Done Sent Done false
17402 5621 Acked Write 6 Done Sent Done false
17403 5622 Acked Flush 0 Done Sent Done false
17404 5623 Acked Read 7 Done Sent Done false
17405 5624 Acked Read 1 Done Sent Done false
17406 5625 Acked Read 5 Done Sent Done false
17407 5626 Acked Read 7 Done Sent Done false
17408 5627 Acked Flush 0 Done Sent Done false
17409 5628 Acked Read 4 Done Sent Done false
17410 5629 Acked Read 4 Done Sent Done false
17411 5630 Acked Flush 0 Done Sent Done false
17412 5631 Acked Write 2 Done Sent Done false
17413 5632 Acked Write 10 Done Sent Done false
17414 5633 Acked Write 10 Done Sent Done false
17415 5634 Acked Read 8 Done Sent Done false
17416 5635 Acked Read 5 Done Sent Done false
17417 5636 Acked Write 7 Done Sent Done false
17418 5637 Acked Flush 0 Done Sent Done false
17419 5638 Acked Write 6 Done Sent Done false
17420 5639 Acked Read 7 Done Sent Done false
17421 5640 Acked Read 4 Done Sent Done false
17422 5641 Acked Flush 0 Done Sent Done false
17423 5642 Acked Read 3 Done Sent Done false
17424 5643 Acked Read 2 Done Sent Done false
17425 5644 Acked Write 3 Done Sent Done false
17426 5645 Acked Flush 0 Done Sent Done false
17427 5646 Acked Read 5 Done Sent Done false
17428 5647 Acked Read 1 Done Sent Done false
17429 5648 Acked Write 10 Done Sent Done false
17430 5649 Acked Write 6 Done Sent Done false
17431 5650 Acked Flush 0 Done Sent Done false
17432 5651 Acked Write 4 Done Sent Done false
17433 5652 Acked Write 6 Done Sent Done false
17434 5653 Acked Flush 0 Done Sent Done false
17435 5654 Acked Write 10 Done Sent Done false
17436 5655 Acked Flush 0 Done Sent Done false
17437 5656 Acked Write 4 Done Sent Done false
17438 5657 Acked Read 2 Done Sent Done false
17439 5658 Acked Read 8 Done Sent Done false
17440 5659 Acked Read 6 Done Sent Done false
17441 5660 Acked Read 7 Done Sent Done false
17442 5661 Acked Read 6 Done Sent Done false
17443 5662 Acked Read 2 Done Sent Done false
17444 5663 Acked Read 1 Done Sent Done false
17445 5664 Acked Write 4 Done Sent Done false
17446 5665 Acked Read 9 Done Sent Done false
17447 5666 Acked Read 4 Done Sent Done false
17448 5667 Acked Read 6 Done Sent Done false
17449 5668 Acked Read 1 Done Sent Done false
17450 5669 Acked Read 9 Done Sent Done false
17451 5670 Acked Write 9 Done Sent Done false
17452 5671 Acked Read 8 Done Sent Done false
17453 5672 Acked Read 2 Done Sent Done false
17454 5673 Acked Flush 0 Done Sent Done false
17455 5674 Acked Write 6 Done Sent Done false
17456 5675 Acked Read 1 Done Sent Done false
17457 5676 Acked Read 8 Done Sent Done false
17458 5677 Acked Write 9 Done Sent Done false
17459 5678 Acked Read 4 Done Sent Done false
17460 5679 Acked Read 5 Done Sent Done false
17461 5680 Acked Read 3 Done Sent Done false
17462 5681 Acked Read 7 Done Sent Done false
17463 5682 Acked Write 1 Done Sent Done false
17464 5683 Acked Write 4 Done Sent Done false
17465 5684 Acked Read 8 Done Sent Done false
17466 5685 Acked Read 5 Done Sent Done false
17467 5686 Acked Write 3 Done Sent Done false
17468 5687 Acked Flush 0 Done Sent Done false
17469 5688 Acked Read 1 Done Sent Done false
17470 5689 Acked Read 4 Done Sent Done false
17471 5690 Acked Write 4 Done Sent Done false
17472 5691 Acked Read 3 Done Sent Done false
17473 5692 Acked Read 3 Done Sent Done false
17474 5693 Acked Read 6 Done Sent Done false
17475 5694 Acked Write 3 Done Sent Done false
17476 5695 Acked Read 8 Done Sent Done false
17477 5696 Acked Read 1 Done Sent Done false
17478 5697 Acked Read 10 Done Sent Done false
17479 5698 Acked Read 8 Done Sent Done false
17480 5699 Acked Write 9 Done Sent Done false
17481 5700 Acked Write 3 Done Sent Done false
17482 5701 Acked Write 8 Done Sent Done false
17483 5702 Acked Write 6 Done Sent Done false
17484 5703 Acked Write 8 Done Sent Done false
17485 5704 Acked Read 10 Done Sent Done false
17486 5705 Acked Write 1 Done Sent Done false
17487 5706 Acked Write 4 Done Sent Done false
17488 5707 Acked Write 10 Done Sent Done false
17489 5708 Acked Write 10 Done Sent Done false
17490 5709 Acked Write 9 Done Sent Done false
17491 5710 Acked Flush 0 Done Sent Done false
17492 5711 Acked Write 7 Done Sent Done false
17493 5712 Acked Read 5 Done Sent Done false
17494 5713 Acked Read 7 Done Sent Done false
17495 5714 Acked Write 7 Done Sent Done false
17496 5715 Acked Write 2 Done Sent Done false
17497 5716 Acked Write 9 Done Sent Done false
17498 5717 Acked Read 4 Done Sent Done false
17499 5718 Acked Read 10 Done Sent Done false
17500 5719 Acked Read 7 Done Sent Done false
17501 5720 Acked Write 4 Done Sent Done false
17502 5721 Acked Flush 0 Done Sent Done false
17503 5722 Acked Read 7 Done Sent Done false
17504 5723 Acked Flush 0 Done Sent Done false
17505 5724 Acked Write 7 Done Sent Done false
17506 5725 Acked Write 4 Done Sent Done false
17507 5726 Acked Write 8 Done Sent Done false
17508 5727 Acked Write 7 Done Sent Done false
17509 5728 Acked Read 4 Done Sent Done false
17510 5729 Acked Read 1 Done Sent Done false
17511 5730 Acked Write 3 Done Sent Done false
17512 5731 Acked Write 2 Done Sent Done false
17513 5732 Acked Read 1 Done Sent Done false
17514 5733 Acked Flush 0 Done Sent Done false
17515 5734 Acked Read 8 Done Sent Done false
17516 5735 Acked Read 10 Done Sent Done false
17517 5736 Acked Write 6 Done Sent Done false
17518 5737 Acked Flush 0 Done Sent Done false
17519 5738 Acked Write 5 Done Sent Done false
17520 5739 Acked Write 10 Done Sent Done false
17521 5740 Acked Write 6 Done Sent Done false
17522 5741 Acked Write 8 Done Sent Done false
17523 5742 Acked Read 3 Done Sent Done false
17524 5743 Acked Flush 0 Done Sent Done false
17525 5744 Acked Read 5 Done Sent Done false
17526 5745 Acked Read 2 Done Sent Done false
17527 5746 Acked Flush 0 Done Sent Done false
17528 5747 Acked Read 6 Done Sent Done false
17529 5748 Acked Flush 0 Done Sent Done false
17530 5749 Acked Read 2 Done Sent Done false
17531 5750 Acked Write 7 Done Sent Done false
17532 5751 Acked Write 2 Done Sent Done false
17533 5752 Acked Write 10 Done Sent Done false
17534 5753 Acked Flush 0 Done Sent Done false
17535 5754 Acked Read 4 Done Sent Done false
17536 5755 Acked Flush 0 Done Sent Done false
17537 5756 Acked Write 7 Done Sent Done false
17538 5757 Acked Read 1 Done Sent Done false
17539 5758 Acked Write 2 Done Sent Done false
17540 5759 Acked Write 7 Done Sent Done false
17541 5760 Acked Read 5 Done Sent Done false
17542 5761 Acked Write 2 Done Sent Done false
17543 5762 Acked Read 7 Done Sent Done false
17544 5763 Acked Flush 0 Done Sent Done false
17545 5764 Acked Read 7 Done Sent Done false
17546 5765 Acked Write 8 Done Sent Done false
17547 5766 Acked Write 2 Done Sent Done false
17548 5767 Acked Read 9 Done Sent Done false
17549 5768 Acked Write 4 Done Sent Done false
17550 5769 Acked Read 5 Done Sent Done false
17551 5770 Acked Flush 0 Done Sent Done false
17552 5771 Acked Read 10 Done Sent Done false
17553 5772 Acked Write 8 Done Sent Done false
17554 5773 Acked Read 5 Done Sent Done false
17555 5774 Acked Read 4 Done Sent Done false
17556 5775 Acked Write 7 Done Sent Done false
17557 5776 Acked Write 9 Done Sent Done false
17558 5777 Acked Write 1 Done Sent Done false
17559 5778 Acked Write 3 Done Sent Done false
17560 5779 Acked Write 10 Done Sent Done false
17561 5780 Acked Write 2 Done Sent Done false
17562 5781 Acked Read 8 Done Sent Done false
17563 5782 Acked Read 9 Done Sent Done false
17564 5783 Acked Read 1 Done Sent Done false
17565 5784 Acked Write 9 Done Sent Done false
17566 5785 Acked Read 6 Done Sent Done false
17567 5786 Acked Flush 0 Done Sent Done false
17568 5787 Acked Write 7 Done Sent Done false
17569 5788 Acked Flush 0 Done Sent Done false
17570 5789 Acked Write 10 Done Sent Done false
17571 5790 Acked Write 6 Done Sent Done false
17572 5791 Acked Write 7 Done Sent Done false
17573 5792 Acked Write 2 Done Sent Done false
17574 5793 Acked Read 3 Done Sent Done false
17575 5794 Acked Read 1 Done Sent Done false
17576 5795 Acked Write 3 Done Sent Done false
17577 5796 Acked Flush 0 Done Sent Done false
17578 5797 Acked Read 8 Done Sent Done false
17579 5798 Acked Write 1 Done Sent Done false
17580 5799 Acked Read 4 Done Sent Done false
17581 5800 Acked Read 3 Done Sent Done false
17582 5801 Acked Flush 0 Done Sent Done false
17583 5802 Acked Read 3 Done Sent Done false
17584 5803 Acked Write 5 Done Sent Done false
17585 5804 Acked Write 7 Done Sent Done false
17586 5805 Acked Write 5 Done Sent Done false
17587 5806 Acked Read 1 Done Sent Done false
17588 5807 Acked Flush 0 Done Sent Done false
17589 5808 Acked Read 10 Done Sent Done false
17590 5809 Acked Write 3 Done Sent Done false
17591 5810 Acked Read 7 Done Sent Done false
17592 5811 Acked Read 9 Done Sent Done false
17593 5812 Acked Read 8 Done Sent Done false
17594 5813 Acked Write 7 Done Sent Done false
17595 5814 Acked Read 6 Done Sent Done false
17596 5815 Acked Read 8 Done Sent Done false
17597 5816 Acked Write 3 Done Sent Done false
17598 5817 Acked Write 8 Done Sent Done false
17599 5818 Acked Write 4 Done Sent Done false
17600 5819 Acked Flush 0 Done Sent Done false
17601 5820 Acked Write 3 Done Sent Done false
17602 5821 Acked Write 6 Done Sent Done false
17603 5822 Acked Read 9 Done Sent Done false
17604 5823 Acked Write 6 Done Sent Done false
17605 5824 Acked Read 5 Done Sent Done false
17606 5825 Acked Read 1 Done Sent Done false
17607 5826 Acked Write 5 Done Sent Done false
17608 5827 Acked Write 8 Done Sent Done false
17609 5828 Acked Flush 0 Done Sent Done false
17610 5829 Acked Read 6 Done Sent Done false
17611 5830 Acked Flush 0 Done Sent Done false
17612 STATES DS:0 DS:1 DS:2 TOTAL
17613 Sent 0 302 0 302
17614 Done 302 0 302 604
17615 Skipped 0 0 0 0
17616 Error 0 0 0 0
17617 Last Flush: 5830 5528 5830
17618 Downstairs last five completed: 5528 5527 5526 5525 5524
17619 Upstairs last five completed: 5830 5829 5828 5827 5826
17620 CLIENT: Up:0 ds:302 act:2
176212025-01-21T15:50:51.054ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176222025-01-21T15:50:51.054ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54605) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176232025-01-21T15:50:51.055ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176242025-01-21T15:50:51.056ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176252025-01-21T15:50:51.056ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176262025-01-21T15:50:51.056ZINFOcrucible: send last flush ID to this DS: 5528 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176272025-01-21T15:50:51.056ZINFOcrucible: Replied this last flush ID: 5528 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176282025-01-21T15:50:51.056ZINFOcrucible: [1] client re-new 302 jobs since flush 5528 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176292025-01-21T15:50:51.057ZINFOcrucible: [1] Replayed 302 jobs since flush: 5528 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
17630 ----------------------------------------------------------------
17631 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17632 Downstairs last five completed: 5830 5829 5828 5827 5826
17633 Upstairs last five completed: 5830 5829 5828 5827 5826
17634 CLIENT: Up:0 ds:0 act:3
17635 Replay: All jobs finished
17636 Main task has work to do, go find it
17637 got dsc Stop(0)
17638 stop 0
17639 Jan 21 15:50:53.293 [8810] Got stop action so:false kr:false
17640 INFO request completed, latency_us: 416, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 43d6fc77-2a4c-4349-afbd-e974819a4923, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
176412025-01-21T15:50:53.294ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176422025-01-21T15:50:53.294ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176432025-01-21T15:50:53.294ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
176442025-01-21T15:50:53.294ZINFOcrucible: client task is exiting = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
17645 Jan 21 15:50:53.294 INFO request completed, latency_us: 164, response_code: 200, uri: /state/cid/0, method: GET, req_id: 66bc19ad-5fbe-4a76-b75b-dfa875d19423, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
17646 [8810] Exited with: ExitStatus(unix_wait_status(9))
17647 [8810][0] reports Exit
17648 Jan 21 15:50:57.296 INFO request completed, latency_us: 258, response_code: 200, uri: /state/cid/0, method: GET, req_id: 1a5e98d5-706f-420e-9b72-280169d36a43, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
17649 001/300 Read block 1994 len 8192
17650 002/300 Write block 4389 len 16384 data: 8 7 5 6
17651 003/300 Flush
17652 004/300 Read block 809 len 20480
17653 005/300 Write block 918 len 12288 data: 6 4 4
17654 006/300 Write block 1856 len 4096 data: 3
17655 007/300 Read block 1430 len 8192
17656 008/300 Write block 4751 len 24576 data: 3 2 2 4 6 7
17657 009/300 Read block 4238 len 4096
17658 010/300 Write block 1566 len 12288 data: 5 4 5
17659 011/300 Flush
17660 012/300 Write block 720 len 4096 data: 5
17661 013/300 Write block 4939 len 32768 data: 3 3 4 5 3 4 4 3
17662 014/300 Read block 3348 len 36864
17663 015/300 Flush
17664 016/300 Read block 4646 len 28672
17665 017/300 Flush
17666 018/300 Read block 426 len 20480
17667 019/300 Write block 77 len 8192 data: 3 3
17668 020/300 Read block 1651 len 40960
17669 021/300 Write block 3330 len 16384 data: 3 3 2 2
17670 022/300 Write block 3829 len 40960 data: 4 4 4 4 4 4 6 5 5 6
17671 023/300 Read block 3287 len 20480
17672 024/300 Read block 3084 len 28672
17673 025/300 Read block 673 len 24576
17674 026/300 Read block 2193 len 24576
17675 027/300 Read block 4097 len 16384
17676 028/300 Read block 1134 len 36864
17677 029/300 Write block 1994 len 28672 data: 4 3 3 5 5 5 5
17678 030/300 Write block 2566 len 24576 data: 3 3 3 4 4 3
17679 031/300 Write block 2147 len 36864 data: 5 4 4 4 4 4 2 2 2
17680 032/300 Write block 3730 len 24576 data: 3 3 3 2 2 4
17681 033/300 Write block 1362 len 8192 data: 6 8
17682 034/300 Read block 725 len 32768
17683 035/300 Write block 177 len 36864 data: 3 4 4 4 6 5 6 6 6
17684 036/300 Flush
17685 037/300 Write block 1018 len 24576 data: 5 3 3 3 3 3
17686 038/300 Read block 768 len 8192
17687 039/300 Write block 2490 len 12288 data: 3 3 4
17688 040/300 Read block 4393 len 40960
17689 041/300 Read block 2574 len 8192
17690 042/300 Write block 251 len 4096 data: 4
17691 043/300 Read block 3161 len 40960
17692 044/300 Write block 3793 len 28672 data: 3 3 3 3 3 3 3
17693 045/300 Write block 1412 len 20480 data: 4 3 3 3 3
17694 046/300 Write block 3662 len 4096 data: 2
17695 047/300 Write block 4038 len 16384 data: 8 6 4 3
17696 048/300 Flush
17697 049/300 Read block 2562 len 8192
17698 050/300 Read block 3298 len 32768
17699 051/300 Read block 4570 len 20480
17700 052/300 Write block 2286 len 8192 data: 3 3
17701 053/300 Write block 4305 len 32768 data: 3 3 3 3 5 4 4 5
17702 054/300 Write block 2266 len 8192 data: 5 4
17703 055/300 Write block 4439 len 12288 data: 6 6 5
17704 056/300 Write block 933 len 16384 data: 4 3 3 4
17705 057/300 Read block 3219 len 4096
17706 058/300 Write block 2641 len 16384 data: 3 3 3 4
17707 059/300 Read block 1365 len 32768
17708 060/300 Write block 3282 len 20480 data: 5 6 4 4 3
17709 061/300 Write block 3648 len 12288 data: 4 4 5
17710 062/300 Write block 4155 len 4096 data: 4
17711 063/300 Flush
17712 064/300 Read block 160 len 16384
17713 065/300 Read block 1930 len 20480
17714 066/300 Write block 4491 len 20480 data: 5 5 6 6 5
17715 067/300 Flush
17716 068/300 Write block 3103 len 32768 data: 2 2 3 3 3 3 3 4
17717 069/300 Write block 1912 len 36864 data: 6 5 4 5 5 6 7 5 4
17718 070/300 Read block 1390 len 24576
17719 071/300 Read block 1598 len 32768
17720 072/300 Write block 4452 len 28672 data: 4 4 4 4 3 4 3
17721 073/300 Write block 78 len 4096 data: 4
17722 074/300 Flush
17723 075/300 Read block 3314 len 28672
17724 076/300 Read block 1394 len 12288
17725 077/300 Read block 2996 len 20480
17726 078/300 Flush
17727 079/300 Read block 4174 len 4096
17728 080/300 Write block 460 len 20480 data: 4 6 7 7 7
17729 081/300 Write block 4295 len 28672 data: 3 4 3 3 3 3 3
17730 082/300 Read block 2264 len 12288
17731 083/300 Read block 279 len 4096
17732 084/300 Write block 10 len 24576 data: 3 2 2 2 2 4
17733 085/300 Write block 2850 len 36864 data: 4 3 3 3 3 3 4 4 5
17734 086/300 Write block 3507 len 32768 data: 3 3 3 3 3 4 5 5
17735 087/300 Read block 4630 len 12288
17736 088/300 Read block 2164 len 8192
17737 089/300 Write block 3569 len 16384 data: 4 4 4 3
17738 090/300 Flush
17739 091/300 Read block 2756 len 36864
17740 092/300 Write block 1437 len 32768 data: 2 3 3 3 3 3 3 4
17741 093/300 Read block 2929 len 12288
17742 094/300 Write block 1169 len 8192 data: 4 5
17743 095/300 Read block 4504 len 4096
17744 096/300 Read block 548 len 24576
17745 097/300 Read block 651 len 12288
17746 098/300 Flush
17747 099/300 Read block 4766 len 20480
17748 100/300 Read block 3506 len 32768
17749 101/300 Read block 3412 len 36864
17750 102/300 Read block 3669 len 4096
17751 103/300 Write block 422 len 8192 data: 2 2
17752 104/300 Read block 4804 len 40960
17753 105/300 Write block 2095 len 4096 data: 6
17754 106/300 Write block 2346 len 28672 data: 5 6 7 6 4 6 7
17755 107/300 Write block 5 len 8192 data: 3 3
17756 108/300 Write block 2236 len 36864 data: 5 4 4 3 3 3 4 3 3
17757 109/300 Read block 3856 len 4096
17758 110/300 Read block 1377 len 32768
17759 111/300 Read block 2063 len 24576
17760 112/300 Read block 3276 len 36864
17761 113/300 Flush
17762 114/300 Write block 4850 len 4096 data: 6
17763 115/300 Write block 3673 len 4096 data: 5
17764 116/300 Read block 3384 len 40960
17765 117/300 Write block 800 len 32768 data: 6 5 6 5 7 7 7 7
17766 118/300 Read block 3806 len 32768
17767 119/300 Write block 1356 len 20480 data: 4 4 4 3 3
17768 120/300 Write block 4205 len 4096 data: 4
17769 121/300 Flush
17770 122/300 Read block 3556 len 8192
17771 123/300 Write block 4488 len 40960 data: 4 3 3 6 6 7 7 6 5 4
17772 124/300 Read block 392 len 16384
17773 125/300 Read block 2638 len 40960
17774 126/300 Read block 4169 len 8192
17775 127/300 Flush
17776 128/300 Read block 4284 len 4096
17777 129/300 Read block 4382 len 32768
17778 130/300 Read block 4377 len 8192
17779 131/300 Write block 2447 len 24576 data: 6 4 4 4 4 3
17780 132/300 Read block 181 len 24576
17781 133/300 Write block 3725 len 4096 data: 7
17782 134/300 Read block 2978 len 8192
17783 135/300 Read block 3273 len 40960
17784 136/300 Write block 4538 len 16384 data: 4 4 5 5
17785 137/300 Write block 1129 len 12288 data: 4 3 3
17786 138/300 Write block 4040 len 8192 data: 5 4
17787 139/300 Flush
17788 140/300 Read block 256 len 36864
17789 141/300 Read block 4728 len 24576
17790 142/300 Read block 3601 len 4096
17791 143/300 Write block 2916 len 28672 data: 4 4 2 2 2 4 4
17792 144/300 Read block 3536 len 4096
17793 145/300 Flush
17794 146/300 Write block 1893 len 40960 data: 2 2 2 4 5 4 4 6 6 6
17795 147/300 Read block 3658 len 32768
17796 148/300 Read block 1148 len 12288
17797 149/300 Write block 1728 len 32768 data: 3 3 3 3 3 3 5 5
17798 150/300 Read block 4241 len 36864
17799 151/300 Write block 503 len 32768 data: 4 4 3 3 2 3 2 2
17800 152/300 Read block 1900 len 16384
17801 153/300 Read block 1028 len 16384
17802 154/300 Write block 2736 len 28672 data: 4 3 2 2 2 2 3
17803 155/300 Write block 1794 len 16384 data: 4 5 5 5
17804 156/300 Read block 4294 len 8192
17805 157/300 Read block 3410 len 40960
17806 158/300 Read block 4764 len 12288
17807 159/300 Read block 1229 len 40960
17808 160/300 Write block 3048 len 40960 data: 3 4 4 4 3 2 3 3 3 3
17809 161/300 Write block 4608 len 20480 data: 7 7 4 4 4
17810 162/300 Read block 272 len 20480
17811 163/300 Read block 3415 len 16384
17812 164/300 Read block 4483 len 40960
17813 165/300 Write block 1024 len 24576 data: 2 4 4 4 3 4
17814 166/300 Write block 4750 len 16384 data: 4 4 3 3
17815 167/300 Read block 4681 len 8192
17816 168/300 Flush
17817 169/300 Write block 787 len 28672 data: 3 3 4 5 6 6 6
17818 170/300 Flush
17819 171/300 Read block 2759 len 36864
17820 172/300 Read block 3391 len 4096
17821 173/300 Read block 665 len 28672
17822 174/300 Write block 403 len 40960 data: 4 6 6 6 7 6 5 5 5 4
17823 175/300 Write block 3811 len 28672 data: 5 5 5 6 6 6 5
17824 176/300 Read block 1313 len 20480
17825 177/300 Read block 1983 len 36864
17826 178/300 Read block 3604 len 12288
17827 179/300 Read block 309 len 24576
17828 180/300 Write block 251 len 28672 data: 5 4 4 4 4 4 4
17829 181/300 Read block 1069 len 36864
17830 182/300 Write block 1547 len 16384 data: 5 6 5 4
17831 183/300 Read block 4803 len 4096
17832 184/300 Write block 815 len 40960 data: 4 3 2 3 4 4 4 4 4 5
17833 185/300 Write block 2244 len 24576 data: 4 4 4 4 5 5
17834 186/300 Flush
17835 187/300 Write block 1160 len 16384 data: 7 6 3 2
17836 188/300 Read block 1021 len 40960
17837 189/300 Write block 59 len 16384 data: 4 5 5 5
17838 190/300 Read block 370 len 4096
17839 191/300 Write block 4852 len 4096 data: 5
17840 192/300 Read block 2134 len 8192
17841 193/300 Flush
17842 194/300 Read block 1811 len 8192
17843 195/300 Read block 1949 len 12288
17844 196/300 Read block 2151 len 40960
17845 197/300 Read block 2060 len 20480
17846 198/300 Read block 1279 len 20480
17847 199/300 Read block 1597 len 24576
17848 200/300 Read block 4937 len 40960
17849 201/300 Read block 646 len 16384
17850 202/300 Write block 3543 len 28672 data: 3 4 4 4 4 2 2
17851 203/300 Read block 1648 len 16384
17852 204/300 Read block 1919 len 16384
17853 205/300 Write block 501 len 16384 data: 4 4 5 5
17854 206/300 Read block 2997 len 16384
17855 207/300 Read block 2234 len 16384
17856 208/300 Read block 40 len 12288
17857 209/300 Read block 4121 len 16384
17858 210/300 Write block 726 len 28672 data: 4 4 5 5 5 6 6
17859 211/300 Read block 1761 len 40960
17860 212/300 Write block 1118 len 40960 data: 4 4 4 4 4 3 3 2 3 3
17861 213/300 Write block 3562 len 40960 data: 4 5 5 5 4 5 5 5 5 5
17862 214/300 Write block 4542 len 28672 data: 5 5 4 3 3 2 2
17863 215/300 Read block 2102 len 24576
17864 216/300 Write block 593 len 28672 data: 8 7 7 6 5 3 5
17865 217/300 Read block 746 len 32768
17866 218/300 Write block 2973 len 40960 data: 3 3 3 3 5 4 5 3 3 3
17867 219/300 Flush
17868 220/300 Flush
17869 221/300 Read block 517 len 24576
17870 222/300 Write block 3886 len 28672 data: 5 5 6 6 7 7 7
17871 223/300 Read block 4261 len 36864
17872 224/300 Read block 3048 len 40960
17873 225/300 Write block 2052 len 24576 data: 3 3 3 4 4 4
17874 226/300 Read block 1308 len 24576
17875 227/300 Flush
17876 228/300 Read block 3725 len 32768
17877 229/300 Read block 3637 len 32768
17878 230/300 Write block 653 len 24576 data: 5 4 4 3 3 2
17879 231/300 Write block 2965 len 40960 data: 3 3 2 3 3 3 2 2 4 4
17880 232/300 Read block 4787 len 16384
17881 233/300 Write block 3336 len 36864 data: 2 2 3 3 3 3 3 3 5
17882 234/300 Flush
17883 235/300 Read block 2903 len 16384
17884 236/300 Write block 3572 len 4096 data: 4
17885 237/300 Write block 3547 len 20480 data: 5 3 3 2 2
17886 238/300 Read block 894 len 8192
17887 239/300 Read block 684 len 8192
17888 240/300 Read block 3802 len 28672
17889 241/300 Read block 4865 len 12288
17890 242/300 Write block 709 len 16384 data: 3 4 4 4
17891 243/300 Write block 246 len 36864 data: 2 2 3 3 4 6 5 5 5
17892 244/300 Write block 4063 len 36864 data: 2 2 3 2 3 4 4 4 5
17893 245/300 Flush
17894 246/300 Read block 4014 len 32768
17895 247/300 Write block 1468 len 20480 data: 4 5 5 5 4
17896 248/300 Read block 1663 len 36864
17897 249/300 Flush
17898 250/300 Write block 114 len 28672 data: 4 3 4 3 3 4 3
17899 251/300 Write block 699 len 36864 data: 4 4 4 4 5 5 4 4 3
17900 252/300 Write block 1902 len 12288 data: 7 7 7
17901 253/300 Read block 1025 len 16384
17902 254/300 Read block 335 len 28672
17903 255/300 Read block 1206 len 8192
17904 256/300 Read block 3349 len 20480
17905 257/300 Flush
17906 258/300 Write block 1266 len 36864 data: 5 6 3 3 3 3 3 3 3
17907 259/300 Read block 2839 len 12288
17908 260/300 Read block 3325 len 40960
17909 261/300 Flush
17910 262/300 Write block 3162 len 16384 data: 5 5 4 4
17911 263/300 Write block 2268 len 16384 data: 4 5 5 6
17912 264/300 Read block 3372 len 28672
17913 265/300 Read block 2127 len 12288
17914 266/300 Read block 3184 len 24576
17915 267/300 Flush
17916 268/300 Read block 1806 len 12288
17917 269/300 Read block 3054 len 40960
17918 270/300 Write block 1616 len 12288 data: 4 3 4
17919 271/300 Read block 1142 len 32768
17920 272/300 Write block 4431 len 36864 data: 3 4 4 4 4 5 6 6 7
17921 273/300 Read block 3390 len 40960
17922 274/300 Read block 1061 len 8192
17923 275/300 Write block 4340 len 20480 data: 3 3 3 6 4
17924 276/300 Read block 2573 len 36864
17925 277/300 Write block 3664 len 40960 data: 4 4 5 5 6 7 5 5 5 6
17926 278/300 Read block 2486 len 40960
17927 279/300 Write block 71 len 24576 data: 4 5 5 6 3 3
17928 280/300 Read block 1368 len 16384
17929 281/300 Flush
17930 282/300 Read block 2917 len 24576
17931 283/300 Read block 1186 len 16384
17932 284/300 Read block 2614 len 24576
17933 285/300 Read block 4153 len 32768
17934 286/300 Read block 847 len 24576
17935 287/300 Read block 3107 len 32768
17936 288/300 Write block 2107 len 36864 data: 5 4 4 4 3 3 3 3 3
17937 289/300 Write block 3857 len 20480 data: 7 7 5 5 5
17938 290/300 Flush
17939 291/300 Write block 1415 len 12288 data: 4 4 3
17940 292/300 Write block 361 len 20480 data: 4 3 2 2 2
17941 293/300 Read block 2377 len 28672
17942 294/300 Write block 4421 len 24576 data: 3 4 3 4 4 4
17943 295/300 Write block 915 len 28672 data: 5 5 6 7 5 5 4
17944 296/300 Flush
17945 297/300 Read block 4693 len 24576
17946 298/300 Write block 4844 len 36864 data: 5 5 5 5 5 4 7 6 6
17947 299/300 Read block 2943 len 32768
17948 300/300 Write block 2491 len 20480 data: 4 5 4 5 5
17949 301/300 Write block 3063 len 36864 data: 3 3 3 3 3 3 4 4 4
17950 Main task has work to do, go find it
17951 got dsc Start(0)
17952 start 0
17953 Jan 21 15:50:57.537 INFO request completed, latency_us: 255, response_code: 204, uri: /start/cid/0, method: POST, req_id: 67527f2f-34a9-42c2-b4e4-ca6d7aa1271d, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
17954 Replay: started 0, returned:Ok(())
17955 ----------------------------------------------------------------
17956 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17957 States: Offline Active Active
17958 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17959 5831 Acked Read 2 Sent Done Done false
17960 5832 Acked Write 4 Sent Done Done false
17961 5833 Acked Flush 0 Sent Done Done false
17962 5834 Acked Read 5 Sent Done Done false
17963 5835 Acked Write 3 Sent Done Done false
17964 5836 Acked Write 1 Sent Done Done false
17965 5837 Acked Read 2 Sent Done Done false
17966 5838 Acked Write 6 Sent Done Done false
17967 5839 Acked Read 1 Sent Done Done false
17968 5840 Acked Write 3 Sent Done Done false
17969 5841 Acked Flush 0 Sent Done Done false
17970 5842 Acked Write 1 Sent Done Done false
17971 5843 Acked Write 8 Sent Done Done false
17972 5844 Acked Read 9 Sent Done Done false
17973 5845 Acked Flush 0 Sent Done Done false
17974 5846 Acked Read 7 Sent Done Done false
17975 5847 Acked Flush 0 Sent Done Done false
17976 5848 Acked Read 5 Sent Done Done false
17977 5849 Acked Write 2 Sent Done Done false
17978 5850 Acked Read 10 Sent Done Done false
17979 5851 Acked Write 4 Sent Done Done false
17980 5852 Acked Write 10 Sent Done Done false
17981 5853 Acked Read 5 Sent Done Done false
17982 5854 Acked Read 7 Sent Done Done false
17983 5855 Acked Read 6 Sent Done Done false
17984 5856 Acked Read 6 Sent Done Done false
17985 5857 Acked Read 4 Sent Done Done false
17986 5858 Acked Read 9 Sent Done Done false
17987 5859 Acked Write 7 Sent Done Done false
17988 5860 Acked Write 6 Sent Done Done false
17989 5861 Acked Write 9 Sent Done Done false
17990 5862 Acked Write 6 Sent Done Done false
17991 5863 Acked Write 2 Sent Done Done false
17992 5864 Acked Read 8 Sent Done Done false
17993 5865 Acked Write 9 Sent Done Done false
17994 5866 Acked Flush 0 Sent Done Done false
17995 5867 Acked Write 6 Sent Done Done false
17996 5868 Acked Read 2 Sent Done Done false
17997 5869 Acked Write 3 Sent Done Done false
17998 5870 Acked Read 10 Sent Done Done false
17999 5871 Acked Read 2 Sent Done Done false
18000 5872 Acked Write 1 Sent Done Done false
18001 5873 Acked Read 10 Sent Done Done false
18002 5874 Acked Write 7 Sent Done Done false
18003 5875 Acked Write 5 Sent Done Done false
18004 5876 Acked Write 1 Sent Done Done false
18005 5877 Acked Write 4 Sent Done Done false
18006 5878 Acked Flush 0 Sent Done Done false
18007 5879 Acked Read 2 Sent Done Done false
18008 5880 Acked Read 8 Sent Done Done false
18009 5881 Acked Read 5 Sent Done Done false
18010 5882 Acked Write 2 Sent Done Done false
18011 5883 Acked Write 8 Sent Done Done false
18012 5884 Acked Write 2 Sent Done Done false
18013 5885 Acked Write 3 Sent Done Done false
18014 5886 Acked Write 4 Sent Done Done false
18015 5887 Acked Read 1 Sent Done Done false
18016 5888 Acked Write 4 Sent Done Done false
18017 5889 Acked Read 8 Sent Done Done false
18018 5890 Acked Write 5 Sent Done Done false
18019 5891 Acked Write 3 Sent Done Done false
18020 5892 Acked Write 1 Sent Done Done false
18021 5893 Acked Flush 0 Sent Done Done false
18022 5894 Acked Read 4 Sent Done Done false
18023 5895 Acked Read 5 Sent Done Done false
18024 5896 Acked Write 5 Sent Done Done false
18025 5897 Acked Flush 0 Sent Done Done false
18026 5898 Acked Write 8 Sent Done Done false
18027 5899 Acked Write 9 Sent Done Done false
18028 5900 Acked Read 6 Sent Done Done false
18029 5901 Acked Read 8 Sent Done Done false
18030 5902 Acked Write 7 Sent Done Done false
18031 5903 Acked Write 1 Sent Done Done false
18032 5904 Acked Flush 0 Sent Done Done false
18033 5905 Acked Read 7 Sent Done Done false
18034 5906 Acked Read 3 Sent Done Done false
18035 5907 Acked Read 5 Sent Done Done false
18036 5908 Acked Flush 0 Sent Done Done false
18037 5909 Acked Read 1 Sent Done Done false
18038 5910 Acked Write 5 Sent Done Done false
18039 5911 Acked Write 7 Sent Done Done false
18040 5912 Acked Read 3 Sent Done Done false
18041 5913 Acked Read 1 Sent Done Done false
18042 5914 Acked Write 6 Sent Done Done false
18043 5915 Acked Write 9 Sent Done Done false
18044 5916 Acked Write 8 Sent Done Done false
18045 5917 Acked Read 3 Sent Done Done false
18046 5918 Acked Read 2 Sent Done Done false
18047 5919 Acked Write 4 Sent Done Done false
18048 5920 Acked Flush 0 Sent Done Done false
18049 5921 Acked Read 9 Sent Done Done false
18050 5922 Acked Write 8 Sent Done Done false
18051 5923 Acked Read 3 Sent Done Done false
18052 5924 Acked Write 2 Sent Done Done false
18053 5925 Acked Read 1 Sent Done Done false
18054 5926 Acked Read 6 Sent Done Done false
18055 5927 Acked Read 3 Sent Done Done false
18056 5928 Acked Flush 0 Sent Done Done false
18057 5929 Acked Read 5 Sent Done Done false
18058 5930 Acked Read 8 Sent Done Done false
18059 5931 Acked Read 9 Sent Done Done false
18060 5932 Acked Read 1 Sent Done Done false
18061 5933 Acked Write 2 Sent Done Done false
18062 5934 Acked Read 10 Sent Done Done false
18063 5935 Acked Write 1 Sent Done Done false
18064 5936 Acked Write 7 Sent Done Done false
18065 5937 Acked Write 2 Sent Done Done false
18066 5938 Acked Write 9 Sent Done Done false
18067 5939 Acked Read 1 Sent Done Done false
18068 5940 Acked Read 8 Sent Done Done false
18069 5941 Acked Read 6 Sent Done Done false
18070 5942 Acked Read 9 Sent Done Done false
18071 5943 Acked Flush 0 Sent Done Done false
18072 5944 Acked Write 1 Sent Done Done false
18073 5945 Acked Write 1 Sent Done Done false
18074 5946 Acked Read 10 Sent Done Done false
18075 5947 Acked Write 8 Sent Done Done false
18076 5948 Acked Read 8 Sent Done Done false
18077 5949 Acked Write 5 Sent Done Done false
18078 5950 Acked Write 1 Sent Done Done false
18079 5951 Acked Flush 0 Sent Done Done false
18080 5952 Acked Read 2 Sent Done Done false
18081 5953 Acked Write 10 Sent Done Done false
18082 5954 Acked Read 4 Sent Done Done false
18083 5955 Acked Read 10 Sent Done Done false
18084 5956 Acked Read 2 Sent Done Done false
18085 5957 Acked Flush 0 Sent Done Done false
18086 5958 Acked Read 1 Sent Done Done false
18087 5959 Acked Read 8 Sent Done Done false
18088 5960 Acked Read 2 Sent Done Done false
18089 5961 Acked Write 6 Sent Done Done false
18090 5962 Acked Read 6 Sent Done Done false
18091 5963 Acked Write 1 Sent Done Done false
18092 5964 Acked Read 2 Sent Done Done false
18093 5965 Acked Read 10 Sent Done Done false
18094 5966 Acked Write 4 Sent Done Done false
18095 5967 Acked Write 3 Sent Done Done false
18096 5968 Acked Write 2 Sent Done Done false
18097 5969 Acked Flush 0 Sent Done Done false
18098 5970 Acked Read 9 Sent Done Done false
18099 5971 Acked Read 6 Sent Done Done false
18100 5972 Acked Read 1 Sent Done Done false
18101 5973 Acked Write 7 Sent Done Done false
18102 5974 Acked Read 1 Sent Done Done false
18103 5975 Acked Flush 0 Sent Done Done false
18104 5976 Acked Write 10 Sent Done Done false
18105 5977 Acked Read 8 Sent Done Done false
18106 5978 Acked Read 3 Sent Done Done false
18107 5979 Acked Write 8 Sent Done Done false
18108 5980 Acked Read 9 Sent Done Done false
18109 5981 Acked Write 8 Sent Done Done false
18110 5982 Acked Read 4 Sent Done Done false
18111 5983 Acked Read 4 Sent Done Done false
18112 5984 Acked Write 7 Sent Done Done false
18113 5985 Acked Write 4 Sent Done Done false
18114 5986 Acked Read 2 Sent Done Done false
18115 5987 Acked Read 10 Sent Done Done false
18116 5988 Acked Read 3 Sent Done Done false
18117 5989 Acked Read 10 Sent Done Done false
18118 5990 Acked Write 10 Sent Done Done false
18119 5991 Acked Write 5 Sent Done Done false
18120 5992 Acked Read 5 Sent Done Done false
18121 5993 Acked Read 4 Sent Done Done false
18122 5994 Acked Read 10 Sent Done Done false
18123 5995 Acked Write 6 Sent Done Done false
18124 5996 Acked Write 4 Sent Done Done false
18125 5997 Acked Read 2 Sent Done Done false
18126 5998 Acked Flush 0 Sent Done Done false
18127 5999 Acked Write 7 Sent Done Done false
18128 6000 Acked Flush 0 Sent Done Done false
18129 6001 Acked Read 9 Sent Done Done false
18130 6002 Acked Read 1 Sent Done Done false
18131 6003 Acked Read 7 Sent Done Done false
18132 6004 Acked Write 10 Sent Done Done false
18133 6005 Acked Write 7 Sent Done Done false
18134 6006 Acked Read 5 Sent Done Done false
18135 6007 Acked Read 9 Sent Done Done false
18136 6008 Acked Read 3 Sent Done Done false
18137 6009 Acked Read 6 Sent Done Done false
18138 6010 Acked Write 7 Sent Done Done false
18139 6011 Acked Read 9 Sent Done Done false
18140 6012 Acked Write 4 Sent Done Done false
18141 6013 Acked Read 1 Sent Done Done false
18142 6014 Acked Write 10 Sent Done Done false
18143 6015 Acked Write 6 Sent Done Done false
18144 6016 Acked Flush 0 Sent Done Done false
18145 6017 Acked Write 4 Sent Done Done false
18146 6018 Acked Read 10 Sent Done Done false
18147 6019 Acked Write 4 Sent Done Done false
18148 6020 Acked Read 1 Sent Done Done false
18149 6021 Acked Write 1 Sent Done Done false
18150 6022 Acked Read 2 Sent Done Done false
18151 6023 Acked Flush 0 Sent Done Done false
18152 6024 Acked Read 2 Sent Done Done false
18153 6025 Acked Read 3 Sent Done Done false
18154 6026 Acked Read 10 Sent Done Done false
18155 6027 Acked Read 5 Sent Done Done false
18156 6028 Acked Read 5 Sent Done Done false
18157 6029 Acked Read 6 Sent Done Done false
18158 6030 Acked Read 10 Sent Done Done false
18159 6031 Acked Read 4 Sent Done Done false
18160 6032 Acked Write 7 Sent Done Done false
18161 6033 Acked Read 4 Sent Done Done false
18162 6034 Acked Read 4 Sent Done Done false
18163 6035 Acked Write 4 Sent Done Done false
18164 6036 Acked Read 4 Sent Done Done false
18165 6037 Acked Read 4 Sent Done Done false
18166 6038 Acked Read 3 Sent Done Done false
18167 6039 Acked Read 4 Sent Done Done false
18168 6040 Acked Write 7 Sent Done Done false
18169 6041 Acked Read 10 Sent Done Done false
18170 6042 Acked Write 10 Sent Done Done false
18171 6043 Acked Write 10 Sent Done Done false
18172 6044 Acked Write 7 Sent Done Done false
18173 6045 Acked Read 6 Sent Done Done false
18174 6046 Acked Write 7 Sent Done Done false
18175 6047 Acked Read 8 Sent Done Done false
18176 6048 Acked Write 10 Sent Done Done false
18177 6049 Acked Flush 0 Sent Done Done false
18178 6050 Acked Flush 0 Sent Done Done false
18179 6051 Acked Read 6 Sent Done Done false
18180 6052 Acked Write 7 Sent Done Done false
18181 6053 Acked Read 9 Sent Done Done false
18182 6054 Acked Read 10 Sent Done Done false
18183 6055 Acked Write 6 Sent Done Done false
18184 6056 Acked Read 6 Sent Done Done false
18185 6057 Acked Flush 0 Sent Done Done false
18186 6058 Acked Read 8 Sent Done Done false
18187 6059 Acked Read 8 Sent Done Done false
18188 6060 Acked Write 6 Sent Done Done false
18189 6061 Acked Write 10 Sent Done Done false
18190 6062 Acked Read 4 Sent Done Done false
18191 6063 Acked Write 9 Sent Done Done false
18192 6064 Acked Flush 0 Sent Done Done false
18193 6065 Acked Read 4 Sent Done Done false
18194 6066 Acked Write 1 Sent Done Done false
18195 6067 Acked Write 5 Sent Done Done false
18196 6068 Acked Read 2 Sent Done Done false
18197 6069 Acked Read 2 Sent Done Done false
18198 6070 Acked Read 7 Sent Done Done false
18199 6071 Acked Read 3 Sent Done Done false
18200 6072 Acked Write 4 Sent Done Done false
18201 6073 Acked Write 9 Sent Done Done false
18202 6074 Acked Write 9 Sent Done Done false
18203 6075 Acked Flush 0 Sent Done Done false
18204 6076 Acked Read 8 Sent Done Done false
18205 6077 Acked Write 5 Sent Done Done false
18206 6078 Acked Read 9 Sent Done Done false
18207 6079 Acked Flush 0 Sent Done Done false
18208 6080 Acked Write 7 Sent Done Done false
18209 6081 Acked Write 9 Sent Done Done false
18210 6082 Acked Write 3 Sent Done Done false
18211 6083 Acked Read 4 Sent Done Done false
18212 6084 Acked Read 7 Sent Done Done false
18213 6085 Acked Read 2 Sent Done Done false
18214 6086 Acked Read 5 Sent Done Done false
18215 6087 Acked Flush 0 Sent Done Done false
18216 6088 Acked Write 9 Sent Done Done false
18217 6089 Acked Read 3 Sent Done Done false
18218 6090 Acked Read 10 Sent Done Done false
18219 6091 Acked Flush 0 Sent Done Done false
18220 6092 Acked Write 4 Sent Done Done false
18221 6093 Acked Write 4 Sent Done Done false
18222 6094 Acked Read 7 Sent Done Done false
18223 6095 Acked Read 3 Sent Done Done false
18224 6096 Acked Read 6 Sent Done Done false
18225 6097 Acked Flush 0 Sent Done Done false
18226 6098 Acked Read 3 Sent Done Done false
18227 6099 Acked Read 10 Sent Done Done false
18228 6100 Acked Write 3 Sent Done Done false
18229 6101 Acked Read 8 Sent Done Done false
18230 6102 Acked Write 9 Sent Done Done false
18231 6103 Acked Read 10 Sent Done Done false
18232 6104 Acked Read 2 Sent Done Done false
18233 6105 Acked Write 5 Sent Done Done false
18234 6106 Acked Read 9 Sent Done Done false
18235 6107 Acked Write 10 Sent Done Done false
18236 6108 Acked Read 10 Sent Done Done false
18237 6109 Acked Write 6 Sent Done Done false
18238 6110 Acked Read 4 Sent Done Done false
18239 6111 Acked Flush 0 Sent Done Done false
18240 6112 Acked Read 6 Sent Done Done false
18241 6113 Acked Read 4 Sent Done Done false
18242 6114 Acked Read 6 Sent Done Done false
18243 6115 Acked Read 8 Sent Done Done false
18244 6116 Acked Read 6 Sent Done Done false
18245 6117 Acked Read 8 Sent Done Done false
18246 6118 Acked Write 9 Sent Done Done false
18247 6119 Acked Write 5 Sent Done Done false
18248 6120 Acked Flush 0 Sent Done Done false
18249 6121 Acked Write 3 Sent Done Done false
18250 6122 Acked Write 5 Sent Done Done false
18251 6123 Acked Read 7 Sent Done Done false
18252 6124 Acked Write 6 Sent Done Done false
18253 6125 Acked Write 7 Sent Done Done false
18254 6126 Acked Flush 0 Sent Done Done false
18255 6127 Acked Read 6 Sent Done Done false
18256 6128 Acked Write 9 Sent Done Done false
18257 6129 Acked Read 8 Sent Done Done false
18258 6130 Acked Write 5 Sent Done Done false
18259 6131 Acked Write 9 Sent Done Done false
18260 STATES DS:0 DS:1 DS:2 TOTAL
18261 Sent 301 0 0 301
18262 Done 0 301 301 602
18263 Skipped 0 0 0 0
18264 Error 0 0 0 0
18265 Last Flush: 5830 6126 6126
18266 Downstairs last five completed: 5830 5829 5828 5827 5826
18267 Upstairs last five completed: 6131 6130 6129 6128 6127
18268 CLIENT: Up:0 ds:301 act:2
18269 [8810] got start action
18270 [8810] I am going to restart
18271 Starting downstairs at port 8810
18272 Make output file at "/tmp/dsc/downstairs-8810.txt"
18273 [8810][0] reports Starting
18274 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1217)
18275 [8810][0] reports Running
18276 ----------------------------------------------------------------
18277 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18278 States: Offline Active Active
18279 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18280 5831 Acked Read 2 Sent Done Done false
18281 5832 Acked Write 4 Sent Done Done false
18282 5833 Acked Flush 0 Sent Done Done false
18283 5834 Acked Read 5 Sent Done Done false
18284 5835 Acked Write 3 Sent Done Done false
18285 5836 Acked Write 1 Sent Done Done false
18286 5837 Acked Read 2 Sent Done Done false
18287 5838 Acked Write 6 Sent Done Done false
18288 5839 Acked Read 1 Sent Done Done false
18289 5840 Acked Write 3 Sent Done Done false
18290 5841 Acked Flush 0 Sent Done Done false
18291 5842 Acked Write 1 Sent Done Done false
18292 5843 Acked Write 8 Sent Done Done false
18293 5844 Acked Read 9 Sent Done Done false
18294 5845 Acked Flush 0 Sent Done Done false
18295 5846 Acked Read 7 Sent Done Done false
18296 5847 Acked Flush 0 Sent Done Done false
18297 5848 Acked Read 5 Sent Done Done false
18298 5849 Acked Write 2 Sent Done Done false
18299 5850 Acked Read 10 Sent Done Done false
18300 5851 Acked Write 4 Sent Done Done false
18301 5852 Acked Write 10 Sent Done Done false
18302 5853 Acked Read 5 Sent Done Done false
18303 5854 Acked Read 7 Sent Done Done false
18304 5855 Acked Read 6 Sent Done Done false
18305 5856 Acked Read 6 Sent Done Done false
18306 5857 Acked Read 4 Sent Done Done false
18307 5858 Acked Read 9 Sent Done Done false
18308 5859 Acked Write 7 Sent Done Done false
18309 5860 Acked Write 6 Sent Done Done false
18310 5861 Acked Write 9 Sent Done Done false
18311 5862 Acked Write 6 Sent Done Done false
18312 5863 Acked Write 2 Sent Done Done false
18313 5864 Acked Read 8 Sent Done Done false
18314 5865 Acked Write 9 Sent Done Done false
18315 5866 Acked Flush 0 Sent Done Done false
18316 5867 Acked Write 6 Sent Done Done false
18317 5868 Acked Read 2 Sent Done Done false
18318 5869 Acked Write 3 Sent Done Done false
18319 5870 Acked Read 10 Sent Done Done false
18320 5871 Acked Read 2 Sent Done Done false
18321 5872 Acked Write 1 Sent Done Done false
18322 5873 Acked Read 10 Sent Done Done false
18323 5874 Acked Write 7 Sent Done Done false
18324 5875 Acked Write 5 Sent Done Done false
18325 5876 Acked Write 1 Sent Done Done false
18326 5877 Acked Write 4 Sent Done Done false
18327 5878 Acked Flush 0 Sent Done Done false
18328 5879 Acked Read 2 Sent Done Done false
18329 5880 Acked Read 8 Sent Done Done false
18330 5881 Acked Read 5 Sent Done Done false
18331 5882 Acked Write 2 Sent Done Done false
18332 5883 Acked Write 8 Sent Done Done false
18333 5884 Acked Write 2 Sent Done Done false
18334 5885 Acked Write 3 Sent Done Done false
18335 5886 Acked Write 4 Sent Done Done false
18336 5887 Acked Read 1 Sent Done Done false
18337 5888 Acked Write 4 Sent Done Done false
18338 5889 Acked Read 8 Sent Done Done false
18339 5890 Acked Write 5 Sent Done Done false
18340 5891 Acked Write 3 Sent Done Done false
18341 5892 Acked Write 1 Sent Done Done false
18342 5893 Acked Flush 0 Sent Done Done false
18343 5894 Acked Read 4 Sent Done Done false
18344 5895 Acked Read 5 Sent Done Done false
18345 5896 Acked Write 5 Sent Done Done false
18346 5897 Acked Flush 0 Sent Done Done false
18347 5898 Acked Write 8 Sent Done Done false
18348 5899 Acked Write 9 Sent Done Done false
18349 5900 Acked Read 6 Sent Done Done false
18350 5901 Acked Read 8 Sent Done Done false
18351 5902 Acked Write 7 Sent Done Done false
18352 5903 Acked Write 1 Sent Done Done false
18353 5904 Acked Flush 0 Sent Done Done false
18354 5905 Acked Read 7 Sent Done Done false
18355 5906 Acked Read 3 Sent Done Done false
18356 5907 Acked Read 5 Sent Done Done false
18357 5908 Acked Flush 0 Sent Done Done false
18358 5909 Acked Read 1 Sent Done Done false
18359 5910 Acked Write 5 Sent Done Done false
18360 5911 Acked Write 7 Sent Done Done false
18361 5912 Acked Read 3 Sent Done Done false
18362 5913 Acked Read 1 Sent Done Done false
18363 5914 Acked Write 6 Sent Done Done false
18364 5915 Acked Write 9 Sent Done Done false
18365 5916 Acked Write 8 Sent Done Done false
18366 5917 Acked Read 3 Sent Done Done false
18367 5918 Acked Read 2 Sent Done Done false
18368 5919 Acked Write 4 Sent Done Done false
18369 5920 Acked Flush 0 Sent Done Done false
18370 5921 Acked Read 9 Sent Done Done false
18371 5922 Acked Write 8 Sent Done Done false
18372 5923 Acked Read 3 Sent Done Done false
18373 5924 Acked Write 2 Sent Done Done false
18374 5925 Acked Read 1 Sent Done Done false
18375 5926 Acked Read 6 Sent Done Done false
18376 5927 Acked Read 3 Sent Done Done false
18377 5928 Acked Flush 0 Sent Done Done false
18378 5929 Acked Read 5 Sent Done Done false
18379 5930 Acked Read 8 Sent Done Done false
18380 5931 Acked Read 9 Sent Done Done false
18381 5932 Acked Read 1 Sent Done Done false
18382 5933 Acked Write 2 Sent Done Done false
18383 5934 Acked Read 10 Sent Done Done false
18384 5935 Acked Write 1 Sent Done Done false
18385 5936 Acked Write 7 Sent Done Done false
18386 5937 Acked Write 2 Sent Done Done false
18387 5938 Acked Write 9 Sent Done Done false
18388 5939 Acked Read 1 Sent Done Done false
18389 5940 Acked Read 8 Sent Done Done false
18390 5941 Acked Read 6 Sent Done Done false
18391 5942 Acked Read 9 Sent Done Done false
18392 5943 Acked Flush 0 Sent Done Done false
18393 5944 Acked Write 1 Sent Done Done false
18394 5945 Acked Write 1 Sent Done Done false
18395 5946 Acked Read 10 Sent Done Done false
18396 5947 Acked Write 8 Sent Done Done false
18397 5948 Acked Read 8 Sent Done Done false
18398 5949 Acked Write 5 Sent Done Done false
18399 5950 Acked Write 1 Sent Done Done false
18400 5951 Acked Flush 0 Sent Done Done false
18401 5952 Acked Read 2 Sent Done Done false
18402 5953 Acked Write 10 Sent Done Done false
18403 5954 Acked Read 4 Sent Done Done false
18404 5955 Acked Read 10 Sent Done Done false
18405 5956 Acked Read 2 Sent Done Done false
18406 5957 Acked Flush 0 Sent Done Done false
18407 5958 Acked Read 1 Sent Done Done false
18408 5959 Acked Read 8 Sent Done Done false
18409 5960 Acked Read 2 Sent Done Done false
18410 5961 Acked Write 6 Sent Done Done false
18411 5962 Acked Read 6 Sent Done Done false
18412 5963 Acked Write 1 Sent Done Done false
18413 5964 Acked Read 2 Sent Done Done false
18414 5965 Acked Read 10 Sent Done Done false
18415 5966 Acked Write 4 Sent Done Done false
18416 5967 Acked Write 3 Sent Done Done false
18417 5968 Acked Write 2 Sent Done Done false
18418 5969 Acked Flush 0 Sent Done Done false
18419 5970 Acked Read 9 Sent Done Done false
18420 5971 Acked Read 6 Sent Done Done false
18421 5972 Acked Read 1 Sent Done Done false
18422 5973 Acked Write 7 Sent Done Done false
18423 5974 Acked Read 1 Sent Done Done false
18424 5975 Acked Flush 0 Sent Done Done false
18425 5976 Acked Write 10 Sent Done Done false
18426 5977 Acked Read 8 Sent Done Done false
18427 5978 Acked Read 3 Sent Done Done false
18428 5979 Acked Write 8 Sent Done Done false
18429 5980 Acked Read 9 Sent Done Done false
18430 5981 Acked Write 8 Sent Done Done false
18431 5982 Acked Read 4 Sent Done Done false
18432 5983 Acked Read 4 Sent Done Done false
18433 5984 Acked Write 7 Sent Done Done false
18434 5985 Acked Write 4 Sent Done Done false
18435 5986 Acked Read 2 Sent Done Done false
18436 5987 Acked Read 10 Sent Done Done false
18437 5988 Acked Read 3 Sent Done Done false
18438 5989 Acked Read 10 Sent Done Done false
18439 5990 Acked Write 10 Sent Done Done false
18440 5991 Acked Write 5 Sent Done Done false
18441 5992 Acked Read 5 Sent Done Done false
18442 5993 Acked Read 4 Sent Done Done false
18443 5994 Acked Read 10 Sent Done Done false
18444 5995 Acked Write 6 Sent Done Done false
18445 5996 Acked Write 4 Sent Done Done false
18446 5997 Acked Read 2 Sent Done Done false
18447 5998 Acked Flush 0 Sent Done Done false
18448 5999 Acked Write 7 Sent Done Done false
18449 6000 Acked Flush 0 Sent Done Done false
18450 6001 Acked Read 9 Sent Done Done false
18451 6002 Acked Read 1 Sent Done Done false
18452 6003 Acked Read 7 Sent Done Done false
18453 6004 Acked Write 10 Sent Done Done false
18454 6005 Acked Write 7 Sent Done Done false
18455 6006 Acked Read 5 Sent Done Done false
18456 6007 Acked Read 9 Sent Done Done false
18457 6008 Acked Read 3 Sent Done Done false
18458 6009 Acked Read 6 Sent Done Done false
18459 6010 Acked Write 7 Sent Done Done false
18460 6011 Acked Read 9 Sent Done Done false
18461 6012 Acked Write 4 Sent Done Done false
18462 6013 Acked Read 1 Sent Done Done false
18463 6014 Acked Write 10 Sent Done Done false
18464 6015 Acked Write 6 Sent Done Done false
18465 6016 Acked Flush 0 Sent Done Done false
18466 6017 Acked Write 4 Sent Done Done false
18467 6018 Acked Read 10 Sent Done Done false
18468 6019 Acked Write 4 Sent Done Done false
18469 6020 Acked Read 1 Sent Done Done false
18470 6021 Acked Write 1 Sent Done Done false
18471 6022 Acked Read 2 Sent Done Done false
18472 6023 Acked Flush 0 Sent Done Done false
18473 6024 Acked Read 2 Sent Done Done false
18474 6025 Acked Read 3 Sent Done Done false
18475 6026 Acked Read 10 Sent Done Done false
18476 6027 Acked Read 5 Sent Done Done false
18477 6028 Acked Read 5 Sent Done Done false
18478 6029 Acked Read 6 Sent Done Done false
18479 6030 Acked Read 10 Sent Done Done false
18480 6031 Acked Read 4 Sent Done Done false
18481 6032 Acked Write 7 Sent Done Done false
18482 6033 Acked Read 4 Sent Done Done false
18483 6034 Acked Read 4 Sent Done Done false
18484 6035 Acked Write 4 Sent Done Done false
18485 6036 Acked Read 4 Sent Done Done false
18486 6037 Acked Read 4 Sent Done Done false
18487 6038 Acked Read 3 Sent Done Done false
18488 6039 Acked Read 4 Sent Done Done false
18489 6040 Acked Write 7 Sent Done Done false
18490 6041 Acked Read 10 Sent Done Done false
18491 6042 Acked Write 10 Sent Done Done false
18492 6043 Acked Write 10 Sent Done Done false
18493 6044 Acked Write 7 Sent Done Done false
18494 6045 Acked Read 6 Sent Done Done false
18495 6046 Acked Write 7 Sent Done Done false
18496 6047 Acked Read 8 Sent Done Done false
18497 6048 Acked Write 10 Sent Done Done false
18498 6049 Acked Flush 0 Sent Done Done false
18499 6050 Acked Flush 0 Sent Done Done false
18500 6051 Acked Read 6 Sent Done Done false
18501 6052 Acked Write 7 Sent Done Done false
18502 6053 Acked Read 9 Sent Done Done false
18503 6054 Acked Read 10 Sent Done Done false
18504 6055 Acked Write 6 Sent Done Done false
18505 6056 Acked Read 6 Sent Done Done false
18506 6057 Acked Flush 0 Sent Done Done false
18507 6058 Acked Read 8 Sent Done Done false
18508 6059 Acked Read 8 Sent Done Done false
18509 6060 Acked Write 6 Sent Done Done false
18510 6061 Acked Write 10 Sent Done Done false
18511 6062 Acked Read 4 Sent Done Done false
18512 6063 Acked Write 9 Sent Done Done false
18513 6064 Acked Flush 0 Sent Done Done false
18514 6065 Acked Read 4 Sent Done Done false
18515 6066 Acked Write 1 Sent Done Done false
18516 6067 Acked Write 5 Sent Done Done false
18517 6068 Acked Read 2 Sent Done Done false
18518 6069 Acked Read 2 Sent Done Done false
18519 6070 Acked Read 7 Sent Done Done false
18520 6071 Acked Read 3 Sent Done Done false
18521 6072 Acked Write 4 Sent Done Done false
18522 6073 Acked Write 9 Sent Done Done false
18523 6074 Acked Write 9 Sent Done Done false
18524 6075 Acked Flush 0 Sent Done Done false
18525 6076 Acked Read 8 Sent Done Done false
18526 6077 Acked Write 5 Sent Done Done false
18527 6078 Acked Read 9 Sent Done Done false
18528 6079 Acked Flush 0 Sent Done Done false
18529 6080 Acked Write 7 Sent Done Done false
18530 6081 Acked Write 9 Sent Done Done false
18531 6082 Acked Write 3 Sent Done Done false
18532 6083 Acked Read 4 Sent Done Done false
18533 6084 Acked Read 7 Sent Done Done false
18534 6085 Acked Read 2 Sent Done Done false
18535 6086 Acked Read 5 Sent Done Done false
18536 6087 Acked Flush 0 Sent Done Done false
18537 6088 Acked Write 9 Sent Done Done false
18538 6089 Acked Read 3 Sent Done Done false
18539 6090 Acked Read 10 Sent Done Done false
18540 6091 Acked Flush 0 Sent Done Done false
18541 6092 Acked Write 4 Sent Done Done false
18542 6093 Acked Write 4 Sent Done Done false
18543 6094 Acked Read 7 Sent Done Done false
18544 6095 Acked Read 3 Sent Done Done false
18545 6096 Acked Read 6 Sent Done Done false
18546 6097 Acked Flush 0 Sent Done Done false
18547 6098 Acked Read 3 Sent Done Done false
18548 6099 Acked Read 10 Sent Done Done false
18549 6100 Acked Write 3 Sent Done Done false
18550 6101 Acked Read 8 Sent Done Done false
18551 6102 Acked Write 9 Sent Done Done false
18552 6103 Acked Read 10 Sent Done Done false
18553 6104 Acked Read 2 Sent Done Done false
18554 6105 Acked Write 5 Sent Done Done false
18555 6106 Acked Read 9 Sent Done Done false
18556 6107 Acked Write 10 Sent Done Done false
18557 6108 Acked Read 10 Sent Done Done false
18558 6109 Acked Write 6 Sent Done Done false
18559 6110 Acked Read 4 Sent Done Done false
18560 6111 Acked Flush 0 Sent Done Done false
18561 6112 Acked Read 6 Sent Done Done false
18562 6113 Acked Read 4 Sent Done Done false
18563 6114 Acked Read 6 Sent Done Done false
18564 6115 Acked Read 8 Sent Done Done false
18565 6116 Acked Read 6 Sent Done Done false
18566 6117 Acked Read 8 Sent Done Done false
18567 6118 Acked Write 9 Sent Done Done false
18568 6119 Acked Write 5 Sent Done Done false
18569 6120 Acked Flush 0 Sent Done Done false
18570 6121 Acked Write 3 Sent Done Done false
18571 6122 Acked Write 5 Sent Done Done false
18572 6123 Acked Read 7 Sent Done Done false
18573 6124 Acked Write 6 Sent Done Done false
18574 6125 Acked Write 7 Sent Done Done false
18575 6126 Acked Flush 0 Sent Done Done false
18576 6127 Acked Read 6 Sent Done Done false
18577 6128 Acked Write 9 Sent Done Done false
18578 6129 Acked Read 8 Sent Done Done false
18579 6130 Acked Write 5 Sent Done Done false
18580 6131 Acked Write 9 Sent Done Done false
18581 6132 Acked Flush 0 Sent Done Done false
18582 STATES DS:0 DS:1 DS:2 TOTAL
18583 Sent 302 0 0 302
18584 Done 0 302 302 604
18585 Skipped 0 0 0 0
18586 Error 0 0 0 0
18587 Last Flush: 5830 6132 6132
18588 Downstairs last five completed: 5830 5829 5828 5827 5826
18589 Upstairs last five completed: 6132 6131 6130 6129 6128
18590 CLIENT: Up:0 ds:302 act:2
185912025-01-21T15:51:03.295ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
185922025-01-21T15:51:03.295ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51774) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
185932025-01-21T15:51:03.296ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
185942025-01-21T15:51:03.296ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
185952025-01-21T15:51:03.296ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
185962025-01-21T15:51:03.296ZINFOcrucible: send last flush ID to this DS: 5830 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
185972025-01-21T15:51:03.297ZINFOcrucible: Replied this last flush ID: 5830 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
185982025-01-21T15:51:03.297ZINFOcrucible: [0] client re-new 302 jobs since flush 5830 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
185992025-01-21T15:51:03.297ZINFOcrucible: [0] Replayed 302 jobs since flush: 5830 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18600 ----------------------------------------------------------------
18601 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18602 Downstairs last five completed: 6132 6131 6130 6129 6128
18603 Upstairs last five completed: 6132 6131 6130 6129 6128
18604 CLIENT: Up:0 ds:0 act:3
18605 Replay: All jobs finished
18606 Main task has work to do, go find it
18607 got dsc Stop(1)
18608 stop 1
18609 [8820] Got stop action so:false kr:false
18610 Jan 21 15:51:05.542 INFO request completed, latency_us: 365, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 889c5866-ecc5-483f-b5df-78277d63356b, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
186112025-01-21T15:51:05.543ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
186122025-01-21T15:51:05.543ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
186132025-01-21T15:51:05.543ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18614 Jan 21 15:51:05.543 INFO request completed, latency_us: 160, response_code: 200, uri: /state/cid/1, method: GET, req_id: 206e823d-a1ac-4c78-84ca-8385d8a5e910, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
186152025-01-21T15:51:05.543ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
18616 [8820] Exited with: ExitStatus(unix_wait_status(9))
18617 [8820][1] reports Exit
18618 Jan 21 15:51:09.546 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/1, method: GET, req_id: d4755006-f272-4c94-8b1f-49295ffd5f4f, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
18619 001/300 Read block 2569 len 24576
18620 002/300 Read block 86 len 28672
18621 003/300 Read block 2411 len 40960
18622 004/300 Read block 3463 len 36864
18623 005/300 Read block 84 len 16384
18624 006/300 Flush
18625 007/300 Read block 860 len 4096
18626 008/300 Write block 1973 len 28672 data: 2 3 3 3 3 4 5
18627 009/300 Read block 425 len 20480
18628 010/300 Write block 619 len 4096 data: 5
18629 011/300 Read block 162 len 8192
18630 012/300 Write block 1542 len 12288 data: 7 7 6
18631 013/300 Read block 4666 len 24576
18632 014/300 Write block 580 len 8192 data: 3 2
18633 015/300 Read block 4416 len 12288
18634 016/300 Write block 2237 len 16384 data: 5 5 4 4
18635 017/300 Read block 2027 len 28672
18636 018/300 Write block 3470 len 40960 data: 7 7 7 6 6 8 9 10 7 6
18637 019/300 Write block 1855 len 12288 data: 3 4 4
18638 020/300 Write block 3484 len 40960 data: 5 4 4 4 3 2 3 3 3 5
18639 021/300 Read block 4187 len 28672
18640 022/300 Write block 2160 len 16384 data: 2 2 3 3
18641 023/300 Write block 699 len 8192 data: 5 5
18642 024/300 Read block 620 len 20480
18643 025/300 Flush
18644 026/300 Read block 4910 len 8192
18645 027/300 Read block 3647 len 32768
18646 028/300 Flush
18647 029/300 Read block 703 len 12288
18648 030/300 Flush
18649 031/300 Read block 4754 len 20480
18650 032/300 Flush
18651 033/300 Read block 4683 len 28672
18652 034/300 Read block 4433 len 4096
18653 035/300 Read block 2777 len 32768
18654 036/300 Write block 3780 len 40960 data: 3 4 4 5 6 6 6 6 5 5
18655 037/300 Write block 2310 len 12288 data: 2 3 3
18656 038/300 Read block 2625 len 32768
18657 039/300 Read block 2296 len 8192
18658 040/300 Read block 3522 len 16384
18659 041/300 Read block 163 len 32768
18660 042/300 Write block 3466 len 4096 data: 4
18661 043/300 Write block 4543 len 20480 data: 6 5 4 4 3
18662 044/300 Write block 1501 len 12288 data: 3 3 3
18663 045/300 Write block 112 len 32768 data: 2 3 5 4 5 4 4 5
18664 046/300 Flush
18665 047/300 Write block 1587 len 16384 data: 4 3 3 4
18666 048/300 Read block 2115 len 20480
18667 049/300 Write block 1444 len 40960 data: 5 3 5 5 4 5 5 5 5 5
18668 050/300 Write block 1653 len 36864 data: 4 3 2 2 2 2 2 3 3
18669 051/300 Read block 4363 len 4096
18670 052/300 Write block 1540 len 16384 data: 5 5 8 8
18671 053/300 Flush
18672 054/300 Flush
18673 055/300 Write block 3880 len 12288 data: 4 4 4
18674 056/300 Write block 1042 len 28672 data: 5 5 6 6 4 3 4
18675 057/300 Read block 1110 len 16384
18676 058/300 Flush
18677 059/300 Write block 3560 len 28672 data: 4 4 5 6 6 6 5
18678 060/300 Read block 3081 len 20480
18679 061/300 Read block 1548 len 8192
18680 062/300 Flush
18681 063/300 Write block 171 len 12288 data: 4 5 5
18682 064/300 Write block 4230 len 12288 data: 5 6 6
18683 065/300 Read block 4490 len 16384
18684 066/300 Read block 155 len 36864
18685 067/300 Write block 752 len 8192 data: 4 3
18686 068/300 Read block 2134 len 12288
18687 069/300 Write block 1758 len 40960 data: 7 5 5 5 5 5 5 4 5 5
18688 070/300 Read block 4932 len 24576
18689 071/300 Flush
18690 072/300 Flush
18691 073/300 Write block 3903 len 24576 data: 2 2 3 3 4 4
18692 074/300 Read block 4752 len 32768
18693 075/300 Write block 3377 len 28672 data: 4 5 6 6 5 6 6
18694 076/300 Read block 2948 len 36864
18695 077/300 Write block 4688 len 28672 data: 2 2 2 4 3 3 3
18696 078/300 Write block 1390 len 4096 data: 3
18697 079/300 Write block 3019 len 8192 data: 2 3
18698 080/300 Read block 1564 len 36864
18699 081/300 Write block 2330 len 40960 data: 3 3 4 5 6 5 4 3 3 3
18700 082/300 Read block 2991 len 4096
18701 083/300 Read block 2412 len 4096
18702 084/300 Write block 2471 len 28672 data: 6 5 3 3 3 3 3
18703 085/300 Read block 2872 len 4096
18704 086/300 Flush
18705 087/300 Read block 3727 len 24576
18706 088/300 Write block 4103 len 28672 data: 5 4 4 3 4 5 4
18707 089/300 Read block 782 len 32768
18708 090/300 Read block 2061 len 20480
18709 091/300 Write block 859 len 36864 data: 5 7 7 5 7 7 7 7 6
18710 092/300 Flush
18711 093/300 Read block 3823 len 24576
18712 094/300 Write block 4545 len 12288 data: 5 5 4
18713 095/300 Write block 4480 len 28672 data: 3 2 5 5 6 6 6
18714 096/300 Read block 3575 len 24576
18715 097/300 Read block 4402 len 36864
18716 098/300 Write block 1393 len 24576 data: 2 2 2 2 2 2
18717 099/300 Read block 1863 len 36864
18718 100/300 Write block 774 len 32768 data: 8 7 6 4 4 3 3 3
18719 101/300 Read block 4043 len 36864
18720 102/300 Write block 1735 len 20480 data: 6 6 8 9 8
18721 103/300 Read block 4444 len 36864
18722 104/300 Write block 2303 len 24576 data: 4 4 5 4 3 3
18723 105/300 Write block 2198 len 40960 data: 4 4 4 3 2 3 3 3 3 3
18724 106/300 Write block 2390 len 28672 data: 5 4 5 4 4 4 4
18725 107/300 Read block 1342 len 32768
18726 108/300 Flush
18727 109/300 Read block 4534 len 20480
18728 110/300 Write block 798 len 4096 data: 6
18729 111/300 Read block 4621 len 4096
18730 112/300 Read block 1877 len 16384
18731 113/300 Read block 980 len 28672
18732 114/300 Read block 4823 len 40960
18733 115/300 Read block 3749 len 4096
18734 116/300 Read block 2313 len 12288
18735 117/300 Read block 4178 len 12288
18736 118/300 Read block 1790 len 16384
18737 119/300 Write block 4657 len 28672 data: 4 3 3 2 2 3 3
18738 120/300 Write block 1032 len 32768 data: 3 3 2 2 4 5 6 6
18739 121/300 Read block 4267 len 28672
18740 122/300 Flush
18741 123/300 Write block 1161 len 24576 data: 7 4 3 2 2 3
18742 124/300 Flush
18743 125/300 Read block 357 len 24576
18744 126/300 Read block 835 len 32768
18745 127/300 Read block 1554 len 4096
18746 128/300 Write block 3290 len 24576 data: 3 3 3 4 4 4
18747 129/300 Flush
18748 130/300 Read block 4284 len 36864
18749 131/300 Write block 502 len 8192 data: 5 6
18750 132/300 Read block 2389 len 40960
18751 133/300 Write block 1050 len 36864 data: 6 6 6 6 6 6 7 8 6
18752 134/300 Read block 4246 len 24576
18753 135/300 Flush
18754 136/300 Read block 2956 len 12288
18755 137/300 Write block 4682 len 8192 data: 5 5
18756 138/300 Write block 2080 len 4096 data: 7
18757 139/300 Read block 3443 len 8192
18758 140/300 Read block 4653 len 20480
18759 141/300 Write block 64 len 36864 data: 4 3 3 4 4 2 2 5 6
18760 142/300 Read block 572 len 12288
18761 143/300 Read block 4901 len 28672
18762 144/300 Read block 1856 len 32768
18763 145/300 Read block 2305 len 32768
18764 146/300 Write block 3433 len 24576 data: 3 2 2 2 2 2
18765 147/300 Read block 3635 len 24576
18766 148/300 Write block 1757 len 20480 data: 7 8 6 6 6
18767 149/300 Write block 3481 len 28672 data: 5 6 6 6 5 5 5
18768 150/300 Flush
18769 151/300 Write block 3956 len 40960 data: 4 2 3 3 4 5 4 4 5 5
18770 152/300 Read block 2696 len 4096
18771 153/300 Read block 1412 len 28672
18772 154/300 Read block 4199 len 12288
18773 155/300 Write block 17 len 40960 data: 3 4 4 4 3 3 2 2 2 2
18774 156/300 Read block 3333 len 20480
18775 157/300 Read block 952 len 40960
18776 158/300 Flush
18777 159/300 Write block 2389 len 32768 data: 4 6 5 6 5 5 5 5
18778 160/300 Read block 1815 len 16384
18779 161/300 Read block 3026 len 24576
18780 162/300 Write block 4369 len 28672 data: 3 3 3 3 2 2 2
18781 163/300 Read block 737 len 20480
18782 164/300 Read block 1552 len 20480
18783 165/300 Write block 410 len 12288 data: 6 6 5
18784 166/300 Read block 849 len 12288
18785 167/300 Read block 747 len 40960
18786 168/300 Write block 2655 len 32768 data: 3 4 5 4 4 4 5 4
18787 169/300 Write block 1788 len 8192 data: 4 4
18788 170/300 Read block 4456 len 24576
18789 171/300 Write block 3565 len 20480 data: 7 6 6 6 6
18790 172/300 Flush
18791 173/300 Write block 3164 len 4096 data: 5
18792 174/300 Read block 209 len 24576
18793 175/300 Read block 941 len 12288
18794 176/300 Read block 3183 len 24576
18795 177/300 Write block 3702 len 12288 data: 7 7 6
18796 178/300 Write block 3200 len 20480 data: 6 7 6 6 6
18797 179/300 Write block 536 len 20480 data: 3 3 4 4 2
18798 180/300 Read block 362 len 36864
18799 181/300 Write block 2940 len 4096 data: 4
18800 182/300 Write block 3496 len 40960 data: 2 3 3 3 3 3 3 4 4 3
18801 183/300 Read block 4975 len 32768
18802 184/300 Write block 338 len 40960 data: 3 4 4 3 3 3 3 5 5 5
18803 185/300 Read block 732 len 20480
18804 186/300 Read block 1043 len 8192
18805 187/300 Flush
18806 188/300 Write block 1383 len 12288 data: 7 5 4
18807 189/300 Read block 986 len 40960
18808 190/300 Write block 1863 len 36864 data: 7 7 8 7 6 4 4 5 5
18809 191/300 Read block 1351 len 24576
18810 192/300 Write block 383 len 36864 data: 5 3 2 3 3 3 3 3 3
18811 193/300 Read block 2151 len 20480
18812 194/300 Write block 4479 len 4096 data: 3
18813 195/300 Write block 141 len 12288 data: 2 2 2
18814 196/300 Flush
18815 197/300 Read block 1665 len 12288
18816 198/300 Write block 2213 len 32768 data: 3 3 2 2 2 2 2 2
18817 199/300 Write block 3742 len 8192 data: 3 3
18818 200/300 Flush
18819 201/300 Read block 3949 len 12288
18820 202/300 Read block 2794 len 20480
18821 203/300 Read block 625 len 20480
18822 204/300 Write block 1913 len 16384 data: 6 5 6 6
18823 205/300 Read block 772 len 4096
18824 206/300 Flush
18825 207/300 Read block 4499 len 40960
18826 208/300 Write block 3788 len 28672 data: 6 6 2 3 3 4 4
18827 209/300 Write block 1219 len 24576 data: 2 2 2 2 3 4
18828 210/300 Read block 1315 len 20480
18829 211/300 Read block 2275 len 16384
18830 212/300 Write block 2929 len 4096 data: 6
18831 213/300 Read block 1686 len 4096
18832 214/300 Flush
18833 215/300 Read block 4312 len 28672
18834 216/300 Read block 4055 len 40960
18835 217/300 Read block 4893 len 8192
18836 218/300 Read block 3825 len 8192
18837 219/300 Write block 3350 len 28672 data: 7 7 6 6 5 6 5
18838 220/300 Read block 719 len 12288
18839 221/300 Read block 1760 len 24576
18840 222/300 Read block 1807 len 24576
18841 223/300 Write block 192 len 8192 data: 6 6
18842 224/300 Flush
18843 225/300 Write block 4766 len 12288 data: 3 3 3
18844 226/300 Write block 3789 len 32768 data: 7 3 4 4 5 5 4 4
18845 227/300 Write block 2973 len 12288 data: 5 5 4
18846 228/300 Read block 1576 len 40960
18847 229/300 Read block 590 len 40960
18848 230/300 Read block 2456 len 20480
18849 231/300 Read block 668 len 20480
18850 232/300 Read block 3918 len 4096
18851 233/300 Flush
18852 234/300 Read block 2006 len 40960
18853 235/300 Write block 366 len 32768 data: 2 4 5 4 5 5 5 6
18854 236/300 Read block 4263 len 40960
18855 237/300 Write block 163 len 8192 data: 6 6
18856 238/300 Read block 4026 len 4096
18857 239/300 Write block 4507 len 4096 data: 3
18858 240/300 Write block 4756 len 8192 data: 8 6
18859 241/300 Write block 4123 len 20480 data: 5 5 5 5 5
18860 242/300 Write block 4623 len 20480 data: 7 8 8 8 8
18861 243/300 Flush
18862 244/300 Write block 3672 len 40960 data: 6 7 3 3 2 3 2 3 4 5
18863 245/300 Read block 1718 len 40960
18864 246/300 Read block 2524 len 40960
18865 247/300 Write block 4863 len 40960 data: 2 2 2 2 2 2 3 4 4 5
18866 248/300 Read block 4903 len 8192
18867 249/300 Read block 2081 len 36864
18868 250/300 Write block 3712 len 40960 data: 5 6 5 4 3 3 4 6 7 9
18869 251/300 Read block 2653 len 4096
18870 252/300 Write block 753 len 36864 data: 4 4 4 3 3 3 3 3 4
18871 253/300 Write block 2135 len 12288 data: 5 4 5
18872 254/300 Read block 4943 len 4096
18873 255/300 Read block 856 len 20480
18874 256/300 Write block 3203 len 12288 data: 7 7 5
18875 257/300 Read block 2365 len 16384
18876 258/300 Read block 337 len 12288
18877 259/300 Write block 1761 len 36864 data: 7 6 6 6 5 6 6 6 5
18878 260/300 Read block 3867 len 8192
18879 261/300 Read block 3293 len 32768
18880 262/300 Read block 1577 len 24576
18881 263/300 Read block 1500 len 28672
18882 264/300 Read block 4395 len 16384
18883 265/300 Write block 1741 len 12288 data: 7 7 5
18884 266/300 Read block 4970 len 40960
18885 267/300 Read block 1967 len 4096
18886 268/300 Write block 1808 len 32768 data: 5 7 7 7 7 6 5 5
18887 269/300 Read block 2418 len 4096
18888 270/300 Write block 954 len 28672 data: 3 2 2 2 2 2 2
18889 271/300 Write block 2528 len 8192 data: 5 6
18890 272/300 Write block 3714 len 16384 data: 6 5 4 4
18891 273/300 Read block 1169 len 40960
18892 274/300 Flush
18893 275/300 Write block 3378 len 8192 data: 6 7
18894 276/300 Write block 841 len 28672 data: 5 6 6 6 6 6 5
18895 277/300 Write block 1795 len 36864 data: 6 6 6 5 5 2 2 2 3
18896 278/300 Write block 1663 len 20480 data: 3 3 3 3 3
18897 279/300 Read block 3303 len 32768
18898 280/300 Read block 2309 len 16384
18899 281/300 Read block 4207 len 28672
18900 282/300 Read block 1617 len 16384
18901 283/300 Read block 2689 len 40960
18902 284/300 Flush
18903 285/300 Write block 614 len 24576 data: 5 5 4 4 5 6
18904 286/300 Flush
18905 287/300 Read block 1488 len 36864
18906 288/300 Read block 2453 len 16384
18907 289/300 Write block 503 len 32768 data: 7 6 4 4 3 4 3 3
18908 290/300 Flush
18909 291/300 Read block 2557 len 36864
18910 292/300 Write block 360 len 12288 data: 4 5 4
18911 293/300 Read block 2205 len 4096
18912 294/300 Read block 1815 len 8192
18913 295/300 Read block 3661 len 36864
18914 296/300 Read block 1935 len 4096
18915 297/300 Read block 3326 len 24576
18916 298/300 Read block 1197 len 12288
18917 299/300 Read block 2763 len 36864
18918 300/300 Read block 1883 len 8192
18919 301/300 Read block 654 len 28672
18920 Main task has work to do, go find it
18921 got dsc Start(1)
18922 start 1
18923 Jan 21 15:51:09.792 INFO request completed, latency_us: 210, response_code: 204, uri: /start/cid/1, method: POST, req_id: daca9c85-028b-4be1-a3bd-a1ab37394cf9, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
18924 Replay: started 1, returned:Ok(())
18925 ----------------------------------------------------------------
18926 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18927 States: Active Offline Active
18928 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18929 6133 Acked Read 6 Done Sent Done false
18930 6134 Acked Read 7 Done Sent Done false
18931 6135 Acked Read 10 Done Sent Done false
18932 6136 Acked Read 9 Done Sent Done false
18933 6137 Acked Read 4 Done Sent Done false
18934 6138 Acked Flush 0 Done Sent Done false
18935 6139 Acked Read 1 Done Sent Done false
18936 6140 Acked Write 7 Done Sent Done false
18937 6141 Acked Read 5 Done Sent Done false
18938 6142 Acked Write 1 Done Sent Done false
18939 6143 Acked Read 2 Done Sent Done false
18940 6144 Acked Write 3 Done Sent Done false
18941 6145 Acked Read 6 Done Sent Done false
18942 6146 Acked Write 2 Done Sent Done false
18943 6147 Acked Read 3 Done Sent Done false
18944 6148 Acked Write 4 Done Sent Done false
18945 6149 Acked Read 7 Done Sent Done false
18946 6150 Acked Write 10 Done Sent Done false
18947 6151 Acked Write 3 Done Sent Done false
18948 6152 Acked Write 10 Done Sent Done false
18949 6153 Acked Read 7 Done Sent Done false
18950 6154 Acked Write 4 Done Sent Done false
18951 6155 Acked Write 2 Done Sent Done false
18952 6156 Acked Read 5 Done Sent Done false
18953 6157 Acked Flush 0 Done Sent Done false
18954 6158 Acked Read 2 Done Sent Done false
18955 6159 Acked Read 8 Done Sent Done false
18956 6160 Acked Flush 0 Done Sent Done false
18957 6161 Acked Read 3 Done Sent Done false
18958 6162 Acked Flush 0 Done Sent Done false
18959 6163 Acked Read 5 Done Sent Done false
18960 6164 Acked Flush 0 Done Sent Done false
18961 6165 Acked Read 7 Done Sent Done false
18962 6166 Acked Read 1 Done Sent Done false
18963 6167 Acked Read 8 Done Sent Done false
18964 6168 Acked Write 10 Done Sent Done false
18965 6169 Acked Write 3 Done Sent Done false
18966 6170 Acked Read 8 Done Sent Done false
18967 6171 Acked Read 2 Done Sent Done false
18968 6172 Acked Read 4 Done Sent Done false
18969 6173 Acked Read 8 Done Sent Done false
18970 6174 Acked Write 1 Done Sent Done false
18971 6175 Acked Write 5 Done Sent Done false
18972 6176 Acked Write 3 Done Sent Done false
18973 6177 Acked Write 8 Done Sent Done false
18974 6178 Acked Flush 0 Done Sent Done false
18975 6179 Acked Write 4 Done Sent Done false
18976 6180 Acked Read 5 Done Sent Done false
18977 6181 Acked Write 10 Done Sent Done false
18978 6182 Acked Write 9 Done Sent Done false
18979 6183 Acked Read 1 Done Sent Done false
18980 6184 Acked Write 4 Done Sent Done false
18981 6185 Acked Flush 0 Done Sent Done false
18982 6186 Acked Flush 0 Done Sent Done false
18983 6187 Acked Write 3 Done Sent Done false
18984 6188 Acked Write 7 Done Sent Done false
18985 6189 Acked Read 4 Done Sent Done false
18986 6190 Acked Flush 0 Done Sent Done false
18987 6191 Acked Write 7 Done Sent Done false
18988 6192 Acked Read 5 Done Sent Done false
18989 6193 Acked Read 2 Done Sent Done false
18990 6194 Acked Flush 0 Done Sent Done false
18991 6195 Acked Write 3 Done Sent Done false
18992 6196 Acked Write 3 Done Sent Done false
18993 6197 Acked Read 4 Done Sent Done false
18994 6198 Acked Read 9 Done Sent Done false
18995 6199 Acked Write 2 Done Sent Done false
18996 6200 Acked Read 3 Done Sent Done false
18997 6201 Acked Write 10 Done Sent Done false
18998 6202 Acked Read 6 Done Sent Done false
18999 6203 Acked Flush 0 Done Sent Done false
19000 6204 Acked Flush 0 Done Sent Done false
19001 6205 Acked Write 6 Done Sent Done false
19002 6206 Acked Read 8 Done Sent Done false
19003 6207 Acked Write 7 Done Sent Done false
19004 6208 Acked Read 9 Done Sent Done false
19005 6209 Acked Write 7 Done Sent Done false
19006 6210 Acked Write 1 Done Sent Done false
19007 6211 Acked Write 2 Done Sent Done false
19008 6212 Acked Read 9 Done Sent Done false
19009 6213 Acked Write 10 Done Sent Done false
19010 6214 Acked Read 1 Done Sent Done false
19011 6215 Acked Read 1 Done Sent Done false
19012 6216 Acked Write 7 Done Sent Done false
19013 6217 Acked Read 1 Done Sent Done false
19014 6218 Acked Flush 0 Done Sent Done false
19015 6219 Acked Read 6 Done Sent Done false
19016 6220 Acked Write 7 Done Sent Done false
19017 6221 Acked Read 8 Done Sent Done false
19018 6222 Acked Read 5 Done Sent Done false
19019 6223 Acked Write 9 Done Sent Done false
19020 6224 Acked Flush 0 Done Sent Done false
19021 6225 Acked Read 6 Done Sent Done false
19022 6226 Acked Write 3 Done Sent Done false
19023 6227 Acked Write 7 Done Sent Done false
19024 6228 Acked Read 6 Done Sent Done false
19025 6229 Acked Read 9 Done Sent Done false
19026 6230 Acked Write 6 Done Sent Done false
19027 6231 Acked Read 9 Done Sent Done false
19028 6232 Acked Write 8 Done Sent Done false
19029 6233 Acked Read 9 Done Sent Done false
19030 6234 Acked Write 5 Done Sent Done false
19031 6235 Acked Read 9 Done Sent Done false
19032 6236 Acked Write 6 Done Sent Done false
19033 6237 Acked Write 10 Done Sent Done false
19034 6238 Acked Write 7 Done Sent Done false
19035 6239 Acked Read 8 Done Sent Done false
19036 6240 Acked Flush 0 Done Sent Done false
19037 6241 Acked Read 5 Done Sent Done false
19038 6242 Acked Write 1 Done Sent Done false
19039 6243 Acked Read 1 Done Sent Done false
19040 6244 Acked Read 4 Done Sent Done false
19041 6245 Acked Read 7 Done Sent Done false
19042 6246 Acked Read 10 Done Sent Done false
19043 6247 Acked Read 1 Done Sent Done false
19044 6248 Acked Read 3 Done Sent Done false
19045 6249 Acked Read 3 Done Sent Done false
19046 6250 Acked Read 4 Done Sent Done false
19047 6251 Acked Write 7 Done Sent Done false
19048 6252 Acked Write 8 Done Sent Done false
19049 6253 Acked Read 7 Done Sent Done false
19050 6254 Acked Flush 0 Done Sent Done false
19051 6255 Acked Write 6 Done Sent Done false
19052 6256 Acked Flush 0 Done Sent Done false
19053 6257 Acked Read 6 Done Sent Done false
19054 6258 Acked Read 8 Done Sent Done false
19055 6259 Acked Read 1 Done Sent Done false
19056 6260 Acked Write 6 Done Sent Done false
19057 6261 Acked Flush 0 Done Sent Done false
19058 6262 Acked Read 9 Done Sent Done false
19059 6263 Acked Write 2 Done Sent Done false
19060 6264 Acked Read 10 Done Sent Done false
19061 6265 Acked Write 9 Done Sent Done false
19062 6266 Acked Read 6 Done Sent Done false
19063 6267 Acked Flush 0 Done Sent Done false
19064 6268 Acked Read 3 Done Sent Done false
19065 6269 Acked Write 2 Done Sent Done false
19066 6270 Acked Write 1 Done Sent Done false
19067 6271 Acked Read 2 Done Sent Done false
19068 6272 Acked Read 5 Done Sent Done false
19069 6273 Acked Write 9 Done Sent Done false
19070 6274 Acked Read 3 Done Sent Done false
19071 6275 Acked Read 7 Done Sent Done false
19072 6276 Acked Read 8 Done Sent Done false
19073 6277 Acked Read 8 Done Sent Done false
19074 6278 Acked Write 6 Done Sent Done false
19075 6279 Acked Read 6 Done Sent Done false
19076 6280 Acked Write 5 Done Sent Done false
19077 6281 Acked Write 7 Done Sent Done false
19078 6282 Acked Flush 0 Done Sent Done false
19079 6283 Acked Write 10 Done Sent Done false
19080 6284 Acked Read 1 Done Sent Done false
19081 6285 Acked Read 7 Done Sent Done false
19082 6286 Acked Read 3 Done Sent Done false
19083 6287 Acked Write 10 Done Sent Done false
19084 6288 Acked Read 5 Done Sent Done false
19085 6289 Acked Read 10 Done Sent Done false
19086 6290 Acked Flush 0 Done Sent Done false
19087 6291 Acked Write 8 Done Sent Done false
19088 6292 Acked Read 4 Done Sent Done false
19089 6293 Acked Read 6 Done Sent Done false
19090 6294 Acked Write 7 Done Sent Done false
19091 6295 Acked Read 5 Done Sent Done false
19092 6296 Acked Read 5 Done Sent Done false
19093 6297 Acked Write 3 Done Sent Done false
19094 6298 Acked Read 3 Done Sent Done false
19095 6299 Acked Read 10 Done Sent Done false
19096 6300 Acked Write 8 Done Sent Done false
19097 6301 Acked Write 2 Done Sent Done false
19098 6302 Acked Read 6 Done Sent Done false
19099 6303 Acked Write 5 Done Sent Done false
19100 6304 Acked Flush 0 Done Sent Done false
19101 6305 Acked Write 1 Done Sent Done false
19102 6306 Acked Read 6 Done Sent Done false
19103 6307 Acked Read 3 Done Sent Done false
19104 6308 Acked Read 6 Done Sent Done false
19105 6309 Acked Write 3 Done Sent Done false
19106 6310 Acked Write 5 Done Sent Done false
19107 6311 Acked Write 5 Done Sent Done false
19108 6312 Acked Read 9 Done Sent Done false
19109 6313 Acked Write 1 Done Sent Done false
19110 6314 Acked Write 10 Done Sent Done false
19111 6315 Acked Read 8 Done Sent Done false
19112 6316 Acked Write 10 Done Sent Done false
19113 6317 Acked Read 5 Done Sent Done false
19114 6318 Acked Read 2 Done Sent Done false
19115 6319 Acked Flush 0 Done Sent Done false
19116 6320 Acked Write 3 Done Sent Done false
19117 6321 Acked Read 10 Done Sent Done false
19118 6322 Acked Write 9 Done Sent Done false
19119 6323 Acked Read 6 Done Sent Done false
19120 6324 Acked Write 9 Done Sent Done false
19121 6325 Acked Read 5 Done Sent Done false
19122 6326 Acked Write 1 Done Sent Done false
19123 6327 Acked Write 3 Done Sent Done false
19124 6328 Acked Flush 0 Done Sent Done false
19125 6329 Acked Read 3 Done Sent Done false
19126 6330 Acked Write 8 Done Sent Done false
19127 6331 Acked Write 2 Done Sent Done false
19128 6332 Acked Flush 0 Done Sent Done false
19129 6333 Acked Read 3 Done Sent Done false
19130 6334 Acked Read 5 Done Sent Done false
19131 6335 Acked Read 5 Done Sent Done false
19132 6336 Acked Write 4 Done Sent Done false
19133 6337 Acked Read 1 Done Sent Done false
19134 6338 Acked Flush 0 Done Sent Done false
19135 6339 Acked Read 10 Done Sent Done false
19136 6340 Acked Write 7 Done Sent Done false
19137 6341 Acked Write 6 Done Sent Done false
19138 6342 Acked Read 5 Done Sent Done false
19139 6343 Acked Read 4 Done Sent Done false
19140 6344 Acked Write 1 Done Sent Done false
19141 6345 Acked Read 1 Done Sent Done false
19142 6346 Acked Flush 0 Done Sent Done false
19143 6347 Acked Read 7 Done Sent Done false
19144 6348 Acked Read 10 Done Sent Done false
19145 6349 Acked Read 2 Done Sent Done false
19146 6350 Acked Read 2 Done Sent Done false
19147 6351 Acked Write 7 Done Sent Done false
19148 6352 Acked Read 3 Done Sent Done false
19149 6353 Acked Read 6 Done Sent Done false
19150 6354 Acked Read 6 Done Sent Done false
19151 6355 Acked Write 2 Done Sent Done false
19152 6356 Acked Flush 0 Done Sent Done false
19153 6357 Acked Write 3 Done Sent Done false
19154 6358 Acked Write 8 Done Sent Done false
19155 6359 Acked Write 3 Done Sent Done false
19156 6360 Acked Read 10 Done Sent Done false
19157 6361 Acked Read 10 Done Sent Done false
19158 6362 Acked Read 5 Done Sent Done false
19159 6363 Acked Read 5 Done Sent Done false
19160 6364 Acked Read 1 Done Sent Done false
19161 6365 Acked Flush 0 Done Sent Done false
19162 6366 Acked Read 10 Done Sent Done false
19163 6367 Acked Write 8 Done Sent Done false
19164 6368 Acked Read 10 Done Sent Done false
19165 6369 Acked Write 2 Done Sent Done false
19166 6370 Acked Read 1 Done Sent Done false
19167 6371 Acked Write 1 Done Sent Done false
19168 6372 Acked Write 2 Done Sent Done false
19169 6373 Acked Write 5 Done Sent Done false
19170 6374 Acked Write 5 Done Sent Done false
19171 6375 Acked Flush 0 Done Sent Done false
19172 6376 Acked Write 10 Done Sent Done false
19173 6377 Acked Read 10 Done Sent Done false
19174 6378 Acked Read 10 Done Sent Done false
19175 6379 Acked Write 10 Done Sent Done false
19176 6380 Acked Read 2 Done Sent Done false
19177 6381 Acked Read 9 Done Sent Done false
19178 6382 Acked Write 10 Done Sent Done false
19179 6383 Acked Read 1 Done Sent Done false
19180 6384 Acked Write 9 Done Sent Done false
19181 6385 Acked Write 3 Done Sent Done false
19182 6386 Acked Read 1 Done Sent Done false
19183 6387 Acked Read 5 Done Sent Done false
19184 6388 Acked Write 3 Done Sent Done false
19185 6389 Acked Read 4 Done Sent Done false
19186 6390 Acked Read 3 Done Sent Done false
19187 6391 Acked Write 9 Done Sent Done false
19188 6392 Acked Read 2 Done Sent Done false
19189 6393 Acked Read 8 Done Sent Done false
19190 6394 Acked Read 6 Done Sent Done false
19191 6395 Acked Read 7 Done Sent Done false
19192 6396 Acked Read 4 Done Sent Done false
19193 6397 Acked Write 3 Done Sent Done false
19194 6398 Acked Read 10 Done Sent Done false
19195 6399 Acked Read 1 Done Sent Done false
19196 6400 Acked Write 8 Done Sent Done false
19197 6401 Acked Read 1 Done Sent Done false
19198 6402 Acked Write 7 Done Sent Done false
19199 6403 Acked Write 2 Done Sent Done false
19200 6404 Acked Write 4 Done Sent Done false
19201 6405 Acked Read 10 Done Sent Done false
19202 6406 Acked Flush 0 Done Sent Done false
19203 6407 Acked Write 2 Done Sent Done false
19204 6408 Acked Write 7 Done Sent Done false
19205 6409 Acked Write 9 Done Sent Done false
19206 6410 Acked Write 5 Done Sent Done false
19207 6411 Acked Read 8 Done Sent Done false
19208 6412 Acked Read 4 Done Sent Done false
19209 6413 Acked Read 7 Done Sent Done false
19210 6414 Acked Read 4 Done Sent Done false
19211 6415 Acked Read 10 Done Sent Done false
19212 6416 Acked Flush 0 Done Sent Done false
19213 6417 Acked Write 6 Done Sent Done false
19214 6418 Acked Flush 0 Done Sent Done false
19215 6419 Acked Read 9 Done Sent Done false
19216 6420 Acked Read 4 Done Sent Done false
19217 6421 Acked Write 8 Done Sent Done false
19218 6422 Acked Flush 0 Done Sent Done false
19219 6423 Acked Read 9 Done Sent Done false
19220 6424 Acked Write 3 Done Sent Done false
19221 6425 Acked Read 1 Done Sent Done false
19222 6426 Acked Read 2 Done Sent Done false
19223 6427 Acked Read 9 Done Sent Done false
19224 6428 Acked Read 1 Done Sent Done false
19225 6429 Acked Read 6 Done Sent Done false
19226 6430 Acked Read 3 Done Sent Done false
19227 6431 Acked Read 9 Done Sent Done false
19228 6432 Acked Read 2 Done Sent Done false
19229 6433 Acked Read 7 Done Sent Done false
19230 STATES DS:0 DS:1 DS:2 TOTAL
19231 Sent 0 301 0 301
19232 Done 301 0 301 602
19233 Skipped 0 0 0 0
19234 Error 0 0 0 0
19235 Last Flush: 6422 6132 6422
19236 Downstairs last five completed: 6132 6131 6130 6129 6128
19237 Upstairs last five completed: 6433 6432 6431 6430 6429
19238 CLIENT: Up:0 ds:301 act:2
19239 ----------------------------------------------------------------
19240 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19241 States: Active Offline Active
19242 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19243 6133 Acked Read 6 Done Sent Done false
19244 6134 Acked Read 7 Done Sent Done false
19245 6135 Acked Read 10 Done Sent Done false
19246 6136 Acked Read 9 Done Sent Done false
19247 6137 Acked Read 4 Done Sent Done false
19248 6138 Acked Flush 0 Done Sent Done false
19249 6139 Acked Read 1 Done Sent Done false
19250 6140 Acked Write 7 Done Sent Done false
19251 6141 Acked Read 5 Done Sent Done false
19252 6142 Acked Write 1 Done Sent Done false
19253 6143 Acked Read 2 Done Sent Done false
19254 6144 Acked Write 3 Done Sent Done false
19255 6145 Acked Read 6 Done Sent Done false
19256 6146 Acked Write 2 Done Sent Done false
19257 6147 Acked Read 3 Done Sent Done false
19258 6148 Acked Write 4 Done Sent Done false
19259 6149 Acked Read 7 Done Sent Done false
19260 6150 Acked Write 10 Done Sent Done false
19261 6151 Acked Write 3 Done Sent Done false
19262 6152 Acked Write 10 Done Sent Done false
19263 6153 Acked Read 7 Done Sent Done false
19264 6154 Acked Write 4 Done Sent Done false
19265 6155 Acked Write 2 Done Sent Done false
19266 6156 Acked Read 5 Done Sent Done false
19267 6157 Acked Flush 0 Done Sent Done false
19268 6158 Acked Read 2 Done Sent Done false
19269 6159 Acked Read 8 Done Sent Done false
19270 6160 Acked Flush 0 Done Sent Done false
19271 6161 Acked Read 3 Done Sent Done false
19272 6162 Acked Flush 0 Done Sent Done false
19273 6163 Acked Read 5 Done Sent Done false
19274 6164 Acked Flush 0 Done Sent Done false
19275 6165 Acked Read 7 Done Sent Done false
19276 6166 Acked Read 1 Done Sent Done false
19277 6167 Acked Read 8 Done Sent Done false
19278 6168 Acked Write 10 Done Sent Done false
19279 6169 Acked Write 3 Done Sent Done false
19280 6170 Acked Read 8 Done Sent Done false
19281 6171 Acked Read 2 Done Sent Done false
19282 6172 Acked Read 4 Done Sent Done false
19283 6173 Acked Read 8 Done Sent Done false
19284 6174 Acked Write 1 Done Sent Done false
19285 6175 Acked Write 5 Done Sent Done false
19286 6176 Acked Write 3 Done Sent Done false
19287 6177 Acked Write 8 Done Sent Done false
19288 6178 Acked Flush 0 Done Sent Done false
19289 6179 Acked Write 4 Done Sent Done false
19290 6180 Acked Read 5 Done Sent Done false
19291 6181 Acked Write 10 Done Sent Done false
19292 6182 Acked Write 9 Done Sent Done false
19293 6183 Acked Read 1 Done Sent Done false
19294 6184 Acked Write 4 Done Sent Done false
19295 6185 Acked Flush 0 Done Sent Done false
19296 6186 Acked Flush 0 Done Sent Done false
19297 6187 Acked Write 3 Done Sent Done false
19298 6188 Acked Write 7 Done Sent Done false
19299 6189 Acked Read 4 Done Sent Done false
19300 6190 Acked Flush 0 Done Sent Done false
19301 6191 Acked Write 7 Done Sent Done false
19302 6192 Acked Read 5 Done Sent Done false
19303 6193 Acked Read 2 Done Sent Done false
19304 6194 Acked Flush 0 Done Sent Done false
19305 6195 Acked Write 3 Done Sent Done false
19306 6196 Acked Write 3 Done Sent Done false
19307 6197 Acked Read 4 Done Sent Done false
19308 6198 Acked Read 9 Done Sent Done false
19309 6199 Acked Write 2 Done Sent Done false
19310 6200 Acked Read 3 Done Sent Done false
19311 6201 Acked Write 10 Done Sent Done false
19312 6202 Acked Read 6 Done Sent Done false
19313 6203 Acked Flush 0 Done Sent Done false
19314 6204 Acked Flush 0 Done Sent Done false
19315 6205 Acked Write 6 Done Sent Done false
19316 6206 Acked Read 8 Done Sent Done false
19317 6207 Acked Write 7 Done Sent Done false
19318 6208 Acked Read 9 Done Sent Done false
19319 6209 Acked Write 7 Done Sent Done false
19320 6210 Acked Write 1 Done Sent Done false
19321 6211 Acked Write 2 Done Sent Done false
19322 6212 Acked Read 9 Done Sent Done false
19323 6213 Acked Write 10 Done Sent Done false
19324 6214 Acked Read 1 Done Sent Done false
19325 6215 Acked Read 1 Done Sent Done false
19326 6216 Acked Write 7 Done Sent Done false
19327 6217 Acked Read 1 Done Sent Done false
19328 6218 Acked Flush 0 Done Sent Done false
19329 6219 Acked Read 6 Done Sent Done false
19330 6220 Acked Write 7 Done Sent Done false
19331 6221 Acked Read 8 Done Sent Done false
19332 6222 Acked Read 5 Done Sent Done false
19333 6223 Acked Write 9 Done Sent Done false
19334 6224 Acked Flush 0 Done Sent Done false
19335 6225 Acked Read 6 Done Sent Done false
19336 6226 Acked Write 3 Done Sent Done false
19337 6227 Acked Write 7 Done Sent Done false
19338 6228 Acked Read 6 Done Sent Done false
19339 6229 Acked Read 9 Done Sent Done false
19340 6230 Acked Write 6 Done Sent Done false
19341 6231 Acked Read 9 Done Sent Done false
19342 6232 Acked Write 8 Done Sent Done false
19343 6233 Acked Read 9 Done Sent Done false
19344 6234 Acked Write 5 Done Sent Done false
19345 6235 Acked Read 9 Done Sent Done false
19346 6236 Acked Write 6 Done Sent Done false
19347 6237 Acked Write 10 Done Sent Done false
19348 6238 Acked Write 7 Done Sent Done false
19349 6239 Acked Read 8 Done Sent Done false
19350 6240 Acked Flush 0 Done Sent Done false
19351 6241 Acked Read 5 Done Sent Done false
19352 6242 Acked Write 1 Done Sent Done false
19353 6243 Acked Read 1 Done Sent Done false
19354 6244 Acked Read 4 Done Sent Done false
19355 6245 Acked Read 7 Done Sent Done false
19356 6246 Acked Read 10 Done Sent Done false
19357 6247 Acked Read 1 Done Sent Done false
19358 6248 Acked Read 3 Done Sent Done false
19359 6249 Acked Read 3 Done Sent Done false
19360 6250 Acked Read 4 Done Sent Done false
19361 6251 Acked Write 7 Done Sent Done false
19362 6252 Acked Write 8 Done Sent Done false
19363 6253 Acked Read 7 Done Sent Done false
19364 6254 Acked Flush 0 Done Sent Done false
19365 6255 Acked Write 6 Done Sent Done false
19366 6256 Acked Flush 0 Done Sent Done false
19367 6257 Acked Read 6 Done Sent Done false
19368 6258 Acked Read 8 Done Sent Done false
19369 6259 Acked Read 1 Done Sent Done false
19370 6260 Acked Write 6 Done Sent Done false
19371 6261 Acked Flush 0 Done Sent Done false
19372 6262 Acked Read 9 Done Sent Done false
19373 6263 Acked Write 2 Done Sent Done false
19374 6264 Acked Read 10 Done Sent Done false
19375 6265 Acked Write 9 Done Sent Done false
19376 6266 Acked Read 6 Done Sent Done false
19377 6267 Acked Flush 0 Done Sent Done false
19378 6268 Acked Read 3 Done Sent Done false
19379 6269 Acked Write 2 Done Sent Done false
19380 6270 Acked Write 1 Done Sent Done false
19381 6271 Acked Read 2 Done Sent Done false
19382 6272 Acked Read 5 Done Sent Done false
19383 6273 Acked Write 9 Done Sent Done false
19384 6274 Acked Read 3 Done Sent Done false
19385 6275 Acked Read 7 Done Sent Done false
19386 6276 Acked Read 8 Done Sent Done false
19387 6277 Acked Read 8 Done Sent Done false
19388 6278 Acked Write 6 Done Sent Done false
19389 6279 Acked Read 6 Done Sent Done false
19390 6280 Acked Write 5 Done Sent Done false
19391 6281 Acked Write 7 Done Sent Done false
19392 6282 Acked Flush 0 Done Sent Done false
19393 6283 Acked Write 10 Done Sent Done false
19394 6284 Acked Read 1 Done Sent Done false
19395 6285 Acked Read 7 Done Sent Done false
19396 6286 Acked Read 3 Done Sent Done false
19397 6287 Acked Write 10 Done Sent Done false
19398 6288 Acked Read 5 Done Sent Done false
19399 6289 Acked Read 10 Done Sent Done false
19400 6290 Acked Flush 0 Done Sent Done false
19401 6291 Acked Write 8 Done Sent Done false
19402 6292 Acked Read 4 Done Sent Done false
19403 6293 Acked Read 6 Done Sent Done false
19404 6294 Acked Write 7 Done Sent Done false
19405 6295 Acked Read 5 Done Sent Done false
19406 6296 Acked Read 5 Done Sent Done false
19407 6297 Acked Write 3 Done Sent Done false
19408 6298 Acked Read 3 Done Sent Done false
19409 6299 Acked Read 10 Done Sent Done false
19410 6300 Acked Write 8 Done Sent Done false
19411 6301 Acked Write 2 Done Sent Done false
19412 6302 Acked Read 6 Done Sent Done false
19413 6303 Acked Write 5 Done Sent Done false
19414 6304 Acked Flush 0 Done Sent Done false
19415 6305 Acked Write 1 Done Sent Done false
19416 6306 Acked Read 6 Done Sent Done false
19417 6307 Acked Read 3 Done Sent Done false
19418 6308 Acked Read 6 Done Sent Done false
19419 6309 Acked Write 3 Done Sent Done false
19420 6310 Acked Write 5 Done Sent Done false
19421 6311 Acked Write 5 Done Sent Done false
19422 6312 Acked Read 9 Done Sent Done false
19423 6313 Acked Write 1 Done Sent Done false
19424 6314 Acked Write 10 Done Sent Done false
19425 6315 Acked Read 8 Done Sent Done false
19426 6316 Acked Write 10 Done Sent Done false
19427 6317 Acked Read 5 Done Sent Done false
19428 6318 Acked Read 2 Done Sent Done false
19429 6319 Acked Flush 0 Done Sent Done false
19430 6320 Acked Write 3 Done Sent Done false
19431 6321 Acked Read 10 Done Sent Done false
19432 6322 Acked Write 9 Done Sent Done false
19433 6323 Acked Read 6 Done Sent Done false
19434 6324 Acked Write 9 Done Sent Done false
19435 6325 Acked Read 5 Done Sent Done false
19436 6326 Acked Write 1 Done Sent Done false
19437 6327 Acked Write 3 Done Sent Done false
19438 6328 Acked Flush 0 Done Sent Done false
19439 6329 Acked Read 3 Done Sent Done false
19440 6330 Acked Write 8 Done Sent Done false
19441 6331 Acked Write 2 Done Sent Done false
19442 6332 Acked Flush 0 Done Sent Done false
19443 6333 Acked Read 3 Done Sent Done false
19444 6334 Acked Read 5 Done Sent Done false
19445 6335 Acked Read 5 Done Sent Done false
19446 6336 Acked Write 4 Done Sent Done false
19447 6337 Acked Read 1 Done Sent Done false
19448 6338 Acked Flush 0 Done Sent Done false
19449 6339 Acked Read 10 Done Sent Done false
19450 6340 Acked Write 7 Done Sent Done false
19451 6341 Acked Write 6 Done Sent Done false
19452 6342 Acked Read 5 Done Sent Done false
19453 6343 Acked Read 4 Done Sent Done false
19454 6344 Acked Write 1 Done Sent Done false
19455 6345 Acked Read 1 Done Sent Done false
19456 6346 Acked Flush 0 Done Sent Done false
19457 6347 Acked Read 7 Done Sent Done false
19458 6348 Acked Read 10 Done Sent Done false
19459 6349 Acked Read 2 Done Sent Done false
19460 6350 Acked Read 2 Done Sent Done false
19461 6351 Acked Write 7 Done Sent Done false
19462 6352 Acked Read 3 Done Sent Done false
19463 6353 Acked Read 6 Done Sent Done false
19464 6354 Acked Read 6 Done Sent Done false
19465 6355 Acked Write 2 Done Sent Done false
19466 6356 Acked Flush 0 Done Sent Done false
19467 6357 Acked Write 3 Done Sent Done false
19468 6358 Acked Write 8 Done Sent Done false
19469 6359 Acked Write 3 Done Sent Done false
19470 6360 Acked Read 10 Done Sent Done false
19471 6361 Acked Read 10 Done Sent Done false
19472 6362 Acked Read 5 Done Sent Done false
19473 6363 Acked Read 5 Done Sent Done false
19474 6364 Acked Read 1 Done Sent Done false
19475 6365 Acked Flush 0 Done Sent Done false
19476 6366 Acked Read 10 Done Sent Done false
19477 6367 Acked Write 8 Done Sent Done false
19478 6368 Acked Read 10 Done Sent Done false
19479 6369 Acked Write 2 Done Sent Done false
19480 6370 Acked Read 1 Done Sent Done false
19481 6371 Acked Write 1 Done Sent Done false
19482 6372 Acked Write 2 Done Sent Done false
19483 6373 Acked Write 5 Done Sent Done false
19484 6374 Acked Write 5 Done Sent Done false
19485 6375 Acked Flush 0 Done Sent Done false
19486 6376 Acked Write 10 Done Sent Done false
19487 6377 Acked Read 10 Done Sent Done false
19488 6378 Acked Read 10 Done Sent Done false
19489 6379 Acked Write 10 Done Sent Done false
19490 6380 Acked Read 2 Done Sent Done false
19491 6381 Acked Read 9 Done Sent Done false
19492 6382 Acked Write 10 Done Sent Done false
19493 6383 Acked Read 1 Done Sent Done false
19494 6384 Acked Write 9 Done Sent Done false
19495 6385 Acked Write 3 Done Sent Done false
19496 6386 Acked Read 1 Done Sent Done false
19497 6387 Acked Read 5 Done Sent Done false
19498 6388 Acked Write 3 Done Sent Done false
19499 6389 Acked Read 4 Done Sent Done false
19500 6390 Acked Read 3 Done Sent Done false
19501 6391 Acked Write 9 Done Sent Done false
19502 6392 Acked Read 2 Done Sent Done false
19503 6393 Acked Read 8 Done Sent Done false
19504 6394 Acked Read 6 Done Sent Done false
19505 6395 Acked Read 7 Done Sent Done false
19506 6396 Acked Read 4 Done Sent Done false
19507 6397 Acked Write 3 Done Sent Done false
19508 6398 Acked Read 10 Done Sent Done false
19509 6399 Acked Read 1 Done Sent Done false
19510 6400 Acked Write 8 Done Sent Done false
19511 6401 Acked Read 1 Done Sent Done false
19512 6402 Acked Write 7 Done Sent Done false
19513 6403 Acked Write 2 Done Sent Done false
19514 6404 Acked Write 4 Done Sent Done false
19515 6405 Acked Read 10 Done Sent Done false
19516 6406 Acked Flush 0 Done Sent Done false
19517 6407 Acked Write 2 Done Sent Done false
19518 6408 Acked Write 7 Done Sent Done false
19519 6409 Acked Write 9 Done Sent Done false
19520 6410 Acked Write 5 Done Sent Done false
19521 6411 Acked Read 8 Done Sent Done false
19522 6412 Acked Read 4 Done Sent Done false
19523 6413 Acked Read 7 Done Sent Done false
19524 6414 Acked Read 4 Done Sent Done false
19525 6415 Acked Read 10 Done Sent Done false
19526 6416 Acked Flush 0 Done Sent Done false
19527 6417 Acked Write 6 Done Sent Done false
19528 6418 Acked Flush 0 Done Sent Done false
19529 6419 Acked Read 9 Done Sent Done false
19530 6420 Acked Read 4 Done Sent Done false
19531 6421 Acked Write 8 Done Sent Done false
19532 6422 Acked Flush 0 Done Sent Done false
19533 6423 Acked Read 9 Done Sent Done false
19534 6424 Acked Write 3 Done Sent Done false
19535 6425 Acked Read 1 Done Sent Done false
19536 6426 Acked Read 2 Done Sent Done false
19537 6427 Acked Read 9 Done Sent Done false
19538 6428 Acked Read 1 Done Sent Done false
19539 6429 Acked Read 6 Done Sent Done false
19540 6430 Acked Read 3 Done Sent Done false
19541 6431 Acked Read 9 Done Sent Done false
19542 6432 Acked Read 2 Done Sent Done false
19543 6433 Acked Read 7 Done Sent Done false
19544 6434 Acked Flush 0 Done Sent Done false
19545 STATES DS:0 DS:1 DS:2 TOTAL
19546 Sent 0 302 0 302
19547 Done 302 0 302 604
19548 Skipped 0 0 0 0
19549 Error 0 0 0 0
19550 Last Flush: 6434 6132 6434
19551 Downstairs last five completed: 6132 6131 6130 6129 6128
19552 Upstairs last five completed: 6434 6433 6432 6431 6430
19553 CLIENT: Up:0 ds:302 act:2
195542025-01-21T15:51:15.544ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
195552025-01-21T15:51:15.544ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
195562025-01-21T15:51:15.544ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
195572025-01-21T15:51:15.544ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
195582025-01-21T15:51:15.544ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
19559 [8820] got start action
19560 [8820] I am going to restart
19561 Starting downstairs at port 8820
19562 Make output file at "/tmp/dsc/downstairs-8820.txt"
19563 [8820][1] reports Starting
19564 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1218)
19565 [8820][1] reports Running
19566 ----------------------------------------------------------------
19567 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19568 States: Active Offline Active
19569 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19570 6133 Acked Read 6 Done Sent Done false
19571 6134 Acked Read 7 Done Sent Done false
19572 6135 Acked Read 10 Done Sent Done false
19573 6136 Acked Read 9 Done Sent Done false
19574 6137 Acked Read 4 Done Sent Done false
19575 6138 Acked Flush 0 Done Sent Done false
19576 6139 Acked Read 1 Done Sent Done false
19577 6140 Acked Write 7 Done Sent Done false
19578 6141 Acked Read 5 Done Sent Done false
19579 6142 Acked Write 1 Done Sent Done false
19580 6143 Acked Read 2 Done Sent Done false
19581 6144 Acked Write 3 Done Sent Done false
19582 6145 Acked Read 6 Done Sent Done false
19583 6146 Acked Write 2 Done Sent Done false
19584 6147 Acked Read 3 Done Sent Done false
19585 6148 Acked Write 4 Done Sent Done false
19586 6149 Acked Read 7 Done Sent Done false
19587 6150 Acked Write 10 Done Sent Done false
19588 6151 Acked Write 3 Done Sent Done false
19589 6152 Acked Write 10 Done Sent Done false
19590 6153 Acked Read 7 Done Sent Done false
19591 6154 Acked Write 4 Done Sent Done false
19592 6155 Acked Write 2 Done Sent Done false
19593 6156 Acked Read 5 Done Sent Done false
19594 6157 Acked Flush 0 Done Sent Done false
19595 6158 Acked Read 2 Done Sent Done false
19596 6159 Acked Read 8 Done Sent Done false
19597 6160 Acked Flush 0 Done Sent Done false
19598 6161 Acked Read 3 Done Sent Done false
19599 6162 Acked Flush 0 Done Sent Done false
19600 6163 Acked Read 5 Done Sent Done false
19601 6164 Acked Flush 0 Done Sent Done false
19602 6165 Acked Read 7 Done Sent Done false
19603 6166 Acked Read 1 Done Sent Done false
19604 6167 Acked Read 8 Done Sent Done false
19605 6168 Acked Write 10 Done Sent Done false
19606 6169 Acked Write 3 Done Sent Done false
19607 6170 Acked Read 8 Done Sent Done false
19608 6171 Acked Read 2 Done Sent Done false
19609 6172 Acked Read 4 Done Sent Done false
19610 6173 Acked Read 8 Done Sent Done false
19611 6174 Acked Write 1 Done Sent Done false
19612 6175 Acked Write 5 Done Sent Done false
19613 6176 Acked Write 3 Done Sent Done false
19614 6177 Acked Write 8 Done Sent Done false
19615 6178 Acked Flush 0 Done Sent Done false
19616 6179 Acked Write 4 Done Sent Done false
19617 6180 Acked Read 5 Done Sent Done false
19618 6181 Acked Write 10 Done Sent Done false
19619 6182 Acked Write 9 Done Sent Done false
19620 6183 Acked Read 1 Done Sent Done false
19621 6184 Acked Write 4 Done Sent Done false
19622 6185 Acked Flush 0 Done Sent Done false
19623 6186 Acked Flush 0 Done Sent Done false
19624 6187 Acked Write 3 Done Sent Done false
19625 6188 Acked Write 7 Done Sent Done false
19626 6189 Acked Read 4 Done Sent Done false
19627 6190 Acked Flush 0 Done Sent Done false
19628 6191 Acked Write 7 Done Sent Done false
19629 6192 Acked Read 5 Done Sent Done false
19630 6193 Acked Read 2 Done Sent Done false
19631 6194 Acked Flush 0 Done Sent Done false
19632 6195 Acked Write 3 Done Sent Done false
19633 6196 Acked Write 3 Done Sent Done false
19634 6197 Acked Read 4 Done Sent Done false
19635 6198 Acked Read 9 Done Sent Done false
19636 6199 Acked Write 2 Done Sent Done false
19637 6200 Acked Read 3 Done Sent Done false
19638 6201 Acked Write 10 Done Sent Done false
19639 6202 Acked Read 6 Done Sent Done false
19640 6203 Acked Flush 0 Done Sent Done false
19641 6204 Acked Flush 0 Done Sent Done false
19642 6205 Acked Write 6 Done Sent Done false
19643 6206 Acked Read 8 Done Sent Done false
19644 6207 Acked Write 7 Done Sent Done false
19645 6208 Acked Read 9 Done Sent Done false
19646 6209 Acked Write 7 Done Sent Done false
19647 6210 Acked Write 1 Done Sent Done false
19648 6211 Acked Write 2 Done Sent Done false
19649 6212 Acked Read 9 Done Sent Done false
19650 6213 Acked Write 10 Done Sent Done false
19651 6214 Acked Read 1 Done Sent Done false
19652 6215 Acked Read 1 Done Sent Done false
19653 6216 Acked Write 7 Done Sent Done false
19654 6217 Acked Read 1 Done Sent Done false
19655 6218 Acked Flush 0 Done Sent Done false
19656 6219 Acked Read 6 Done Sent Done false
19657 6220 Acked Write 7 Done Sent Done false
19658 6221 Acked Read 8 Done Sent Done false
19659 6222 Acked Read 5 Done Sent Done false
19660 6223 Acked Write 9 Done Sent Done false
19661 6224 Acked Flush 0 Done Sent Done false
19662 6225 Acked Read 6 Done Sent Done false
19663 6226 Acked Write 3 Done Sent Done false
19664 6227 Acked Write 7 Done Sent Done false
19665 6228 Acked Read 6 Done Sent Done false
19666 6229 Acked Read 9 Done Sent Done false
19667 6230 Acked Write 6 Done Sent Done false
19668 6231 Acked Read 9 Done Sent Done false
19669 6232 Acked Write 8 Done Sent Done false
19670 6233 Acked Read 9 Done Sent Done false
19671 6234 Acked Write 5 Done Sent Done false
19672 6235 Acked Read 9 Done Sent Done false
19673 6236 Acked Write 6 Done Sent Done false
19674 6237 Acked Write 10 Done Sent Done false
19675 6238 Acked Write 7 Done Sent Done false
19676 6239 Acked Read 8 Done Sent Done false
19677 6240 Acked Flush 0 Done Sent Done false
19678 6241 Acked Read 5 Done Sent Done false
19679 6242 Acked Write 1 Done Sent Done false
19680 6243 Acked Read 1 Done Sent Done false
19681 6244 Acked Read 4 Done Sent Done false
19682 6245 Acked Read 7 Done Sent Done false
19683 6246 Acked Read 10 Done Sent Done false
19684 6247 Acked Read 1 Done Sent Done false
19685 6248 Acked Read 3 Done Sent Done false
19686 6249 Acked Read 3 Done Sent Done false
19687 6250 Acked Read 4 Done Sent Done false
19688 6251 Acked Write 7 Done Sent Done false
19689 6252 Acked Write 8 Done Sent Done false
19690 6253 Acked Read 7 Done Sent Done false
19691 6254 Acked Flush 0 Done Sent Done false
19692 6255 Acked Write 6 Done Sent Done false
19693 6256 Acked Flush 0 Done Sent Done false
19694 6257 Acked Read 6 Done Sent Done false
19695 6258 Acked Read 8 Done Sent Done false
19696 6259 Acked Read 1 Done Sent Done false
19697 6260 Acked Write 6 Done Sent Done false
19698 6261 Acked Flush 0 Done Sent Done false
19699 6262 Acked Read 9 Done Sent Done false
19700 6263 Acked Write 2 Done Sent Done false
19701 6264 Acked Read 10 Done Sent Done false
19702 6265 Acked Write 9 Done Sent Done false
19703 6266 Acked Read 6 Done Sent Done false
19704 6267 Acked Flush 0 Done Sent Done false
19705 6268 Acked Read 3 Done Sent Done false
19706 6269 Acked Write 2 Done Sent Done false
19707 6270 Acked Write 1 Done Sent Done false
19708 6271 Acked Read 2 Done Sent Done false
19709 6272 Acked Read 5 Done Sent Done false
19710 6273 Acked Write 9 Done Sent Done false
19711 6274 Acked Read 3 Done Sent Done false
19712 6275 Acked Read 7 Done Sent Done false
19713 6276 Acked Read 8 Done Sent Done false
19714 6277 Acked Read 8 Done Sent Done false
19715 6278 Acked Write 6 Done Sent Done false
19716 6279 Acked Read 6 Done Sent Done false
19717 6280 Acked Write 5 Done Sent Done false
19718 6281 Acked Write 7 Done Sent Done false
19719 6282 Acked Flush 0 Done Sent Done false
19720 6283 Acked Write 10 Done Sent Done false
19721 6284 Acked Read 1 Done Sent Done false
19722 6285 Acked Read 7 Done Sent Done false
19723 6286 Acked Read 3 Done Sent Done false
19724 6287 Acked Write 10 Done Sent Done false
19725 6288 Acked Read 5 Done Sent Done false
19726 6289 Acked Read 10 Done Sent Done false
19727 6290 Acked Flush 0 Done Sent Done false
19728 6291 Acked Write 8 Done Sent Done false
19729 6292 Acked Read 4 Done Sent Done false
19730 6293 Acked Read 6 Done Sent Done false
19731 6294 Acked Write 7 Done Sent Done false
19732 6295 Acked Read 5 Done Sent Done false
19733 6296 Acked Read 5 Done Sent Done false
19734 6297 Acked Write 3 Done Sent Done false
19735 6298 Acked Read 3 Done Sent Done false
19736 6299 Acked Read 10 Done Sent Done false
19737 6300 Acked Write 8 Done Sent Done false
19738 6301 Acked Write 2 Done Sent Done false
19739 6302 Acked Read 6 Done Sent Done false
19740 6303 Acked Write 5 Done Sent Done false
19741 6304 Acked Flush 0 Done Sent Done false
19742 6305 Acked Write 1 Done Sent Done false
19743 6306 Acked Read 6 Done Sent Done false
19744 6307 Acked Read 3 Done Sent Done false
19745 6308 Acked Read 6 Done Sent Done false
19746 6309 Acked Write 3 Done Sent Done false
19747 6310 Acked Write 5 Done Sent Done false
19748 6311 Acked Write 5 Done Sent Done false
19749 6312 Acked Read 9 Done Sent Done false
19750 6313 Acked Write 1 Done Sent Done false
19751 6314 Acked Write 10 Done Sent Done false
19752 6315 Acked Read 8 Done Sent Done false
19753 6316 Acked Write 10 Done Sent Done false
19754 6317 Acked Read 5 Done Sent Done false
19755 6318 Acked Read 2 Done Sent Done false
19756 6319 Acked Flush 0 Done Sent Done false
19757 6320 Acked Write 3 Done Sent Done false
19758 6321 Acked Read 10 Done Sent Done false
19759 6322 Acked Write 9 Done Sent Done false
19760 6323 Acked Read 6 Done Sent Done false
19761 6324 Acked Write 9 Done Sent Done false
19762 6325 Acked Read 5 Done Sent Done false
19763 6326 Acked Write 1 Done Sent Done false
19764 6327 Acked Write 3 Done Sent Done false
19765 6328 Acked Flush 0 Done Sent Done false
19766 6329 Acked Read 3 Done Sent Done false
19767 6330 Acked Write 8 Done Sent Done false
19768 6331 Acked Write 2 Done Sent Done false
19769 6332 Acked Flush 0 Done Sent Done false
19770 6333 Acked Read 3 Done Sent Done false
19771 6334 Acked Read 5 Done Sent Done false
19772 6335 Acked Read 5 Done Sent Done false
19773 6336 Acked Write 4 Done Sent Done false
19774 6337 Acked Read 1 Done Sent Done false
19775 6338 Acked Flush 0 Done Sent Done false
19776 6339 Acked Read 10 Done Sent Done false
19777 6340 Acked Write 7 Done Sent Done false
19778 6341 Acked Write 6 Done Sent Done false
19779 6342 Acked Read 5 Done Sent Done false
19780 6343 Acked Read 4 Done Sent Done false
19781 6344 Acked Write 1 Done Sent Done false
19782 6345 Acked Read 1 Done Sent Done false
19783 6346 Acked Flush 0 Done Sent Done false
19784 6347 Acked Read 7 Done Sent Done false
19785 6348 Acked Read 10 Done Sent Done false
19786 6349 Acked Read 2 Done Sent Done false
19787 6350 Acked Read 2 Done Sent Done false
19788 6351 Acked Write 7 Done Sent Done false
19789 6352 Acked Read 3 Done Sent Done false
19790 6353 Acked Read 6 Done Sent Done false
19791 6354 Acked Read 6 Done Sent Done false
19792 6355 Acked Write 2 Done Sent Done false
19793 6356 Acked Flush 0 Done Sent Done false
19794 6357 Acked Write 3 Done Sent Done false
19795 6358 Acked Write 8 Done Sent Done false
19796 6359 Acked Write 3 Done Sent Done false
19797 6360 Acked Read 10 Done Sent Done false
19798 6361 Acked Read 10 Done Sent Done false
19799 6362 Acked Read 5 Done Sent Done false
19800 6363 Acked Read 5 Done Sent Done false
19801 6364 Acked Read 1 Done Sent Done false
19802 6365 Acked Flush 0 Done Sent Done false
19803 6366 Acked Read 10 Done Sent Done false
19804 6367 Acked Write 8 Done Sent Done false
19805 6368 Acked Read 10 Done Sent Done false
19806 6369 Acked Write 2 Done Sent Done false
19807 6370 Acked Read 1 Done Sent Done false
19808 6371 Acked Write 1 Done Sent Done false
19809 6372 Acked Write 2 Done Sent Done false
19810 6373 Acked Write 5 Done Sent Done false
19811 6374 Acked Write 5 Done Sent Done false
19812 6375 Acked Flush 0 Done Sent Done false
19813 6376 Acked Write 10 Done Sent Done false
19814 6377 Acked Read 10 Done Sent Done false
19815 6378 Acked Read 10 Done Sent Done false
19816 6379 Acked Write 10 Done Sent Done false
19817 6380 Acked Read 2 Done Sent Done false
19818 6381 Acked Read 9 Done Sent Done false
19819 6382 Acked Write 10 Done Sent Done false
19820 6383 Acked Read 1 Done Sent Done false
19821 6384 Acked Write 9 Done Sent Done false
19822 6385 Acked Write 3 Done Sent Done false
19823 6386 Acked Read 1 Done Sent Done false
19824 6387 Acked Read 5 Done Sent Done false
19825 6388 Acked Write 3 Done Sent Done false
19826 6389 Acked Read 4 Done Sent Done false
19827 6390 Acked Read 3 Done Sent Done false
19828 6391 Acked Write 9 Done Sent Done false
19829 6392 Acked Read 2 Done Sent Done false
19830 6393 Acked Read 8 Done Sent Done false
19831 6394 Acked Read 6 Done Sent Done false
19832 6395 Acked Read 7 Done Sent Done false
19833 6396 Acked Read 4 Done Sent Done false
19834 6397 Acked Write 3 Done Sent Done false
19835 6398 Acked Read 10 Done Sent Done false
19836 6399 Acked Read 1 Done Sent Done false
19837 6400 Acked Write 8 Done Sent Done false
19838 6401 Acked Read 1 Done Sent Done false
19839 6402 Acked Write 7 Done Sent Done false
19840 6403 Acked Write 2 Done Sent Done false
19841 6404 Acked Write 4 Done Sent Done false
19842 6405 Acked Read 10 Done Sent Done false
19843 6406 Acked Flush 0 Done Sent Done false
19844 6407 Acked Write 2 Done Sent Done false
19845 6408 Acked Write 7 Done Sent Done false
19846 6409 Acked Write 9 Done Sent Done false
19847 6410 Acked Write 5 Done Sent Done false
19848 6411 Acked Read 8 Done Sent Done false
19849 6412 Acked Read 4 Done Sent Done false
19850 6413 Acked Read 7 Done Sent Done false
19851 6414 Acked Read 4 Done Sent Done false
19852 6415 Acked Read 10 Done Sent Done false
19853 6416 Acked Flush 0 Done Sent Done false
19854 6417 Acked Write 6 Done Sent Done false
19855 6418 Acked Flush 0 Done Sent Done false
19856 6419 Acked Read 9 Done Sent Done false
19857 6420 Acked Read 4 Done Sent Done false
19858 6421 Acked Write 8 Done Sent Done false
19859 6422 Acked Flush 0 Done Sent Done false
19860 6423 Acked Read 9 Done Sent Done false
19861 6424 Acked Write 3 Done Sent Done false
19862 6425 Acked Read 1 Done Sent Done false
19863 6426 Acked Read 2 Done Sent Done false
19864 6427 Acked Read 9 Done Sent Done false
19865 6428 Acked Read 1 Done Sent Done false
19866 6429 Acked Read 6 Done Sent Done false
19867 6430 Acked Read 3 Done Sent Done false
19868 6431 Acked Read 9 Done Sent Done false
19869 6432 Acked Read 2 Done Sent Done false
19870 6433 Acked Read 7 Done Sent Done false
19871 6434 Acked Flush 0 Done Sent Done false
19872 STATES DS:0 DS:1 DS:2 TOTAL
19873 Sent 0 302 0 302
19874 Done 302 0 302 604
19875 Skipped 0 0 0 0
19876 Error 0 0 0 0
19877 Last Flush: 6434 6132 6434
19878 Downstairs last five completed: 6132 6131 6130 6129 6128
19879 Upstairs last five completed: 6434 6433 6432 6431 6430
19880 CLIENT: Up:0 ds:302 act:2
19881 ----------------------------------------------------------------
19882 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19883 States: Active Offline Active
19884 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19885 6133 Acked Read 6 Done Sent Done false
19886 6134 Acked Read 7 Done Sent Done false
19887 6135 Acked Read 10 Done Sent Done false
19888 6136 Acked Read 9 Done Sent Done false
19889 6137 Acked Read 4 Done Sent Done false
19890 6138 Acked Flush 0 Done Sent Done false
19891 6139 Acked Read 1 Done Sent Done false
19892 6140 Acked Write 7 Done Sent Done false
19893 6141 Acked Read 5 Done Sent Done false
19894 6142 Acked Write 1 Done Sent Done false
19895 6143 Acked Read 2 Done Sent Done false
19896 6144 Acked Write 3 Done Sent Done false
19897 6145 Acked Read 6 Done Sent Done false
19898 6146 Acked Write 2 Done Sent Done false
19899 6147 Acked Read 3 Done Sent Done false
19900 6148 Acked Write 4 Done Sent Done false
19901 6149 Acked Read 7 Done Sent Done false
19902 6150 Acked Write 10 Done Sent Done false
19903 6151 Acked Write 3 Done Sent Done false
19904 6152 Acked Write 10 Done Sent Done false
19905 6153 Acked Read 7 Done Sent Done false
19906 6154 Acked Write 4 Done Sent Done false
19907 6155 Acked Write 2 Done Sent Done false
19908 6156 Acked Read 5 Done Sent Done false
19909 6157 Acked Flush 0 Done Sent Done false
19910 6158 Acked Read 2 Done Sent Done false
19911 6159 Acked Read 8 Done Sent Done false
19912 6160 Acked Flush 0 Done Sent Done false
19913 6161 Acked Read 3 Done Sent Done false
19914 6162 Acked Flush 0 Done Sent Done false
19915 6163 Acked Read 5 Done Sent Done false
19916 6164 Acked Flush 0 Done Sent Done false
19917 6165 Acked Read 7 Done Sent Done false
19918 6166 Acked Read 1 Done Sent Done false
19919 6167 Acked Read 8 Done Sent Done false
19920 6168 Acked Write 10 Done Sent Done false
19921 6169 Acked Write 3 Done Sent Done false
19922 6170 Acked Read 8 Done Sent Done false
19923 6171 Acked Read 2 Done Sent Done false
19924 6172 Acked Read 4 Done Sent Done false
19925 6173 Acked Read 8 Done Sent Done false
19926 6174 Acked Write 1 Done Sent Done false
19927 6175 Acked Write 5 Done Sent Done false
19928 6176 Acked Write 3 Done Sent Done false
19929 6177 Acked Write 8 Done Sent Done false
19930 6178 Acked Flush 0 Done Sent Done false
19931 6179 Acked Write 4 Done Sent Done false
19932 6180 Acked Read 5 Done Sent Done false
19933 6181 Acked Write 10 Done Sent Done false
19934 6182 Acked Write 9 Done Sent Done false
19935 6183 Acked Read 1 Done Sent Done false
19936 6184 Acked Write 4 Done Sent Done false
19937 6185 Acked Flush 0 Done Sent Done false
19938 6186 Acked Flush 0 Done Sent Done false
19939 6187 Acked Write 3 Done Sent Done false
19940 6188 Acked Write 7 Done Sent Done false
19941 6189 Acked Read 4 Done Sent Done false
19942 6190 Acked Flush 0 Done Sent Done false
19943 6191 Acked Write 7 Done Sent Done false
19944 6192 Acked Read 5 Done Sent Done false
19945 6193 Acked Read 2 Done Sent Done false
19946 6194 Acked Flush 0 Done Sent Done false
19947 6195 Acked Write 3 Done Sent Done false
19948 6196 Acked Write 3 Done Sent Done false
19949 6197 Acked Read 4 Done Sent Done false
19950 6198 Acked Read 9 Done Sent Done false
19951 6199 Acked Write 2 Done Sent Done false
19952 6200 Acked Read 3 Done Sent Done false
19953 6201 Acked Write 10 Done Sent Done false
19954 6202 Acked Read 6 Done Sent Done false
19955 6203 Acked Flush 0 Done Sent Done false
19956 6204 Acked Flush 0 Done Sent Done false
19957 6205 Acked Write 6 Done Sent Done false
19958 6206 Acked Read 8 Done Sent Done false
19959 6207 Acked Write 7 Done Sent Done false
19960 6208 Acked Read 9 Done Sent Done false
19961 6209 Acked Write 7 Done Sent Done false
19962 6210 Acked Write 1 Done Sent Done false
19963 6211 Acked Write 2 Done Sent Done false
19964 6212 Acked Read 9 Done Sent Done false
19965 6213 Acked Write 10 Done Sent Done false
19966 6214 Acked Read 1 Done Sent Done false
19967 6215 Acked Read 1 Done Sent Done false
19968 6216 Acked Write 7 Done Sent Done false
19969 6217 Acked Read 1 Done Sent Done false
19970 6218 Acked Flush 0 Done Sent Done false
19971 6219 Acked Read 6 Done Sent Done false
19972 6220 Acked Write 7 Done Sent Done false
19973 6221 Acked Read 8 Done Sent Done false
19974 6222 Acked Read 5 Done Sent Done false
19975 6223 Acked Write 9 Done Sent Done false
19976 6224 Acked Flush 0 Done Sent Done false
19977 6225 Acked Read 6 Done Sent Done false
19978 6226 Acked Write 3 Done Sent Done false
19979 6227 Acked Write 7 Done Sent Done false
19980 6228 Acked Read 6 Done Sent Done false
19981 6229 Acked Read 9 Done Sent Done false
19982 6230 Acked Write 6 Done Sent Done false
19983 6231 Acked Read 9 Done Sent Done false
19984 6232 Acked Write 8 Done Sent Done false
19985 6233 Acked Read 9 Done Sent Done false
19986 6234 Acked Write 5 Done Sent Done false
19987 6235 Acked Read 9 Done Sent Done false
19988 6236 Acked Write 6 Done Sent Done false
19989 6237 Acked Write 10 Done Sent Done false
19990 6238 Acked Write 7 Done Sent Done false
19991 6239 Acked Read 8 Done Sent Done false
19992 6240 Acked Flush 0 Done Sent Done false
19993 6241 Acked Read 5 Done Sent Done false
19994 6242 Acked Write 1 Done Sent Done false
19995 6243 Acked Read 1 Done Sent Done false
19996 6244 Acked Read 4 Done Sent Done false
19997 6245 Acked Read 7 Done Sent Done false
19998 6246 Acked Read 10 Done Sent Done false
19999 6247 Acked Read 1 Done Sent Done false
20000 6248 Acked Read 3 Done Sent Done false
20001 6249 Acked Read 3 Done Sent Done false
20002 6250 Acked Read 4 Done Sent Done false
20003 6251 Acked Write 7 Done Sent Done false
20004 6252 Acked Write 8 Done Sent Done false
20005 6253 Acked Read 7 Done Sent Done false
20006 6254 Acked Flush 0 Done Sent Done false
20007 6255 Acked Write 6 Done Sent Done false
20008 6256 Acked Flush 0 Done Sent Done false
20009 6257 Acked Read 6 Done Sent Done false
20010 6258 Acked Read 8 Done Sent Done false
20011 6259 Acked Read 1 Done Sent Done false
20012 6260 Acked Write 6 Done Sent Done false
20013 6261 Acked Flush 0 Done Sent Done false
20014 6262 Acked Read 9 Done Sent Done false
20015 6263 Acked Write 2 Done Sent Done false
20016 6264 Acked Read 10 Done Sent Done false
20017 6265 Acked Write 9 Done Sent Done false
20018 6266 Acked Read 6 Done Sent Done false
20019 6267 Acked Flush 0 Done Sent Done false
20020 6268 Acked Read 3 Done Sent Done false
20021 6269 Acked Write 2 Done Sent Done false
20022 6270 Acked Write 1 Done Sent Done false
20023 6271 Acked Read 2 Done Sent Done false
20024 6272 Acked Read 5 Done Sent Done false
20025 6273 Acked Write 9 Done Sent Done false
20026 6274 Acked Read 3 Done Sent Done false
20027 6275 Acked Read 7 Done Sent Done false
20028 6276 Acked Read 8 Done Sent Done false
20029 6277 Acked Read 8 Done Sent Done false
20030 6278 Acked Write 6 Done Sent Done false
20031 6279 Acked Read 6 Done Sent Done false
20032 6280 Acked Write 5 Done Sent Done false
20033 6281 Acked Write 7 Done Sent Done false
20034 6282 Acked Flush 0 Done Sent Done false
20035 6283 Acked Write 10 Done Sent Done false
20036 6284 Acked Read 1 Done Sent Done false
20037 6285 Acked Read 7 Done Sent Done false
20038 6286 Acked Read 3 Done Sent Done false
20039 6287 Acked Write 10 Done Sent Done false
20040 6288 Acked Read 5 Done Sent Done false
20041 6289 Acked Read 10 Done Sent Done false
20042 6290 Acked Flush 0 Done Sent Done false
20043 6291 Acked Write 8 Done Sent Done false
20044 6292 Acked Read 4 Done Sent Done false
20045 6293 Acked Read 6 Done Sent Done false
20046 6294 Acked Write 7 Done Sent Done false
20047 6295 Acked Read 5 Done Sent Done false
20048 6296 Acked Read 5 Done Sent Done false
20049 6297 Acked Write 3 Done Sent Done false
20050 6298 Acked Read 3 Done Sent Done false
20051 6299 Acked Read 10 Done Sent Done false
20052 6300 Acked Write 8 Done Sent Done false
20053 6301 Acked Write 2 Done Sent Done false
20054 6302 Acked Read 6 Done Sent Done false
20055 6303 Acked Write 5 Done Sent Done false
20056 6304 Acked Flush 0 Done Sent Done false
20057 6305 Acked Write 1 Done Sent Done false
20058 6306 Acked Read 6 Done Sent Done false
20059 6307 Acked Read 3 Done Sent Done false
20060 6308 Acked Read 6 Done Sent Done false
20061 6309 Acked Write 3 Done Sent Done false
20062 6310 Acked Write 5 Done Sent Done false
20063 6311 Acked Write 5 Done Sent Done false
20064 6312 Acked Read 9 Done Sent Done false
20065 6313 Acked Write 1 Done Sent Done false
20066 6314 Acked Write 10 Done Sent Done false
20067 6315 Acked Read 8 Done Sent Done false
20068 6316 Acked Write 10 Done Sent Done false
20069 6317 Acked Read 5 Done Sent Done false
20070 6318 Acked Read 2 Done Sent Done false
20071 6319 Acked Flush 0 Done Sent Done false
20072 6320 Acked Write 3 Done Sent Done false
20073 6321 Acked Read 10 Done Sent Done false
20074 6322 Acked Write 9 Done Sent Done false
20075 6323 Acked Read 6 Done Sent Done false
20076 6324 Acked Write 9 Done Sent Done false
20077 6325 Acked Read 5 Done Sent Done false
20078 6326 Acked Write 1 Done Sent Done false
20079 6327 Acked Write 3 Done Sent Done false
20080 6328 Acked Flush 0 Done Sent Done false
20081 6329 Acked Read 3 Done Sent Done false
20082 6330 Acked Write 8 Done Sent Done false
20083 6331 Acked Write 2 Done Sent Done false
20084 6332 Acked Flush 0 Done Sent Done false
20085 6333 Acked Read 3 Done Sent Done false
20086 6334 Acked Read 5 Done Sent Done false
20087 6335 Acked Read 5 Done Sent Done false
20088 6336 Acked Write 4 Done Sent Done false
20089 6337 Acked Read 1 Done Sent Done false
20090 6338 Acked Flush 0 Done Sent Done false
20091 6339 Acked Read 10 Done Sent Done false
20092 6340 Acked Write 7 Done Sent Done false
20093 6341 Acked Write 6 Done Sent Done false
20094 6342 Acked Read 5 Done Sent Done false
20095 6343 Acked Read 4 Done Sent Done false
20096 6344 Acked Write 1 Done Sent Done false
20097 6345 Acked Read 1 Done Sent Done false
20098 6346 Acked Flush 0 Done Sent Done false
20099 6347 Acked Read 7 Done Sent Done false
20100 6348 Acked Read 10 Done Sent Done false
20101 6349 Acked Read 2 Done Sent Done false
20102 6350 Acked Read 2 Done Sent Done false
20103 6351 Acked Write 7 Done Sent Done false
20104 6352 Acked Read 3 Done Sent Done false
20105 6353 Acked Read 6 Done Sent Done false
20106 6354 Acked Read 6 Done Sent Done false
20107 6355 Acked Write 2 Done Sent Done false
20108 6356 Acked Flush 0 Done Sent Done false
20109 6357 Acked Write 3 Done Sent Done false
20110 6358 Acked Write 8 Done Sent Done false
20111 6359 Acked Write 3 Done Sent Done false
20112 6360 Acked Read 10 Done Sent Done false
20113 6361 Acked Read 10 Done Sent Done false
20114 6362 Acked Read 5 Done Sent Done false
20115 6363 Acked Read 5 Done Sent Done false
20116 6364 Acked Read 1 Done Sent Done false
20117 6365 Acked Flush 0 Done Sent Done false
20118 6366 Acked Read 10 Done Sent Done false
20119 6367 Acked Write 8 Done Sent Done false
20120 6368 Acked Read 10 Done Sent Done false
20121 6369 Acked Write 2 Done Sent Done false
20122 6370 Acked Read 1 Done Sent Done false
20123 6371 Acked Write 1 Done Sent Done false
20124 6372 Acked Write 2 Done Sent Done false
20125 6373 Acked Write 5 Done Sent Done false
20126 6374 Acked Write 5 Done Sent Done false
20127 6375 Acked Flush 0 Done Sent Done false
20128 6376 Acked Write 10 Done Sent Done false
20129 6377 Acked Read 10 Done Sent Done false
20130 6378 Acked Read 10 Done Sent Done false
20131 6379 Acked Write 10 Done Sent Done false
20132 6380 Acked Read 2 Done Sent Done false
20133 6381 Acked Read 9 Done Sent Done false
20134 6382 Acked Write 10 Done Sent Done false
20135 6383 Acked Read 1 Done Sent Done false
20136 6384 Acked Write 9 Done Sent Done false
20137 6385 Acked Write 3 Done Sent Done false
20138 6386 Acked Read 1 Done Sent Done false
20139 6387 Acked Read 5 Done Sent Done false
20140 6388 Acked Write 3 Done Sent Done false
20141 6389 Acked Read 4 Done Sent Done false
20142 6390 Acked Read 3 Done Sent Done false
20143 6391 Acked Write 9 Done Sent Done false
20144 6392 Acked Read 2 Done Sent Done false
20145 6393 Acked Read 8 Done Sent Done false
20146 6394 Acked Read 6 Done Sent Done false
20147 6395 Acked Read 7 Done Sent Done false
20148 6396 Acked Read 4 Done Sent Done false
20149 6397 Acked Write 3 Done Sent Done false
20150 6398 Acked Read 10 Done Sent Done false
20151 6399 Acked Read 1 Done Sent Done false
20152 6400 Acked Write 8 Done Sent Done false
20153 6401 Acked Read 1 Done Sent Done false
20154 6402 Acked Write 7 Done Sent Done false
20155 6403 Acked Write 2 Done Sent Done false
20156 6404 Acked Write 4 Done Sent Done false
20157 6405 Acked Read 10 Done Sent Done false
20158 6406 Acked Flush 0 Done Sent Done false
20159 6407 Acked Write 2 Done Sent Done false
20160 6408 Acked Write 7 Done Sent Done false
20161 6409 Acked Write 9 Done Sent Done false
20162 6410 Acked Write 5 Done Sent Done false
20163 6411 Acked Read 8 Done Sent Done false
20164 6412 Acked Read 4 Done Sent Done false
20165 6413 Acked Read 7 Done Sent Done false
20166 6414 Acked Read 4 Done Sent Done false
20167 6415 Acked Read 10 Done Sent Done false
20168 6416 Acked Flush 0 Done Sent Done false
20169 6417 Acked Write 6 Done Sent Done false
20170 6418 Acked Flush 0 Done Sent Done false
20171 6419 Acked Read 9 Done Sent Done false
20172 6420 Acked Read 4 Done Sent Done false
20173 6421 Acked Write 8 Done Sent Done false
20174 6422 Acked Flush 0 Done Sent Done false
20175 6423 Acked Read 9 Done Sent Done false
20176 6424 Acked Write 3 Done Sent Done false
20177 6425 Acked Read 1 Done Sent Done false
20178 6426 Acked Read 2 Done Sent Done false
20179 6427 Acked Read 9 Done Sent Done false
20180 6428 Acked Read 1 Done Sent Done false
20181 6429 Acked Read 6 Done Sent Done false
20182 6430 Acked Read 3 Done Sent Done false
20183 6431 Acked Read 9 Done Sent Done false
20184 6432 Acked Read 2 Done Sent Done false
20185 6433 Acked Read 7 Done Sent Done false
20186 6434 Acked Flush 0 Done Sent Done false
20187 STATES DS:0 DS:1 DS:2 TOTAL
20188 Sent 0 302 0 302
20189 Done 302 0 302 604
20190 Skipped 0 0 0 0
20191 Error 0 0 0 0
20192 Last Flush: 6434 6132 6434
20193 Downstairs last five completed: 6132 6131 6130 6129 6128
20194 Upstairs last five completed: 6434 6433 6432 6431 6430
20195 CLIENT: Up:0 ds:302 act:2
201962025-01-21T15:51:25.545ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
201972025-01-21T15:51:25.546ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53383) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
201982025-01-21T15:51:25.547ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
201992025-01-21T15:51:25.547ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
202002025-01-21T15:51:25.547ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
202012025-01-21T15:51:25.547ZINFOcrucible: send last flush ID to this DS: 6132 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
202022025-01-21T15:51:25.548ZINFOcrucible: Replied this last flush ID: 6132 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
202032025-01-21T15:51:25.548ZINFOcrucible: [1] client re-new 302 jobs since flush 6132 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
202042025-01-21T15:51:25.548ZINFOcrucible: [1] Replayed 302 jobs since flush: 6132 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
20205 ----------------------------------------------------------------
20206 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20207 Downstairs last five completed: 6434 6433 6432 6431 6430
20208 Upstairs last five completed: 6434 6433 6432 6431 6430
20209 CLIENT: Up:0 ds:0 act:3
20210 Replay: All jobs finished
20211 Main task has work to do, go find it
20212 got dsc Stop(0)
20213 stop 0
20214 [8810] Got stop action so:false kr:false
20215 Jan 21 15:51:25.801 INFO request completed, latency_us: 363, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 5cb9b508-352c-4094-8733-c76750e01358, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
202162025-01-21T15:51:25.802ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
202172025-01-21T15:51:25.802ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
202182025-01-21T15:51:25.802ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
20219 Jan 21 15:51:25.802 INFO request completed, latency_us: 187, response_code: 200, uri: /state/cid/0, method: GET, req_id: d6220dc3-47dd-41bb-a9d3-c823dee02a1a, remote_addr: 127.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:51:25.802347135Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
20220 0.1:52945, local_addr: 127.0.0.1:9998
20221 [8810] Exited with: ExitStatus(unix_wait_status(9))
20222 [8810][0] reports Exit
20223 Jan 21 15:51:29.804 INFO request completed, latency_us: 221, response_code: 200, uri: /state/cid/0, method: GET, req_id: a1df6f62-5217-4dac-b08c-1ed7931d0daf, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
20224 001/300 Read block 890 len 32768
20225 002/300 Write block 722 len 28672 data: 5 4 4 4 5 5 6
20226 003/300 Read block 711 len 8192
20227 004/300 Read block 3543 len 16384
20228 005/300 Read block 4029 len 20480
20229 006/300 Write block 3324 len 16384 data: 2 2 3 4
20230 007/300 Read block 2841 len 40960
20231 008/300 Write block 24 len 36864 data: 3 3 3 2 2 2 3 3 3
20232 009/300 Write block 3448 len 36864 data: 2 3 4 4 4 4 4 4 4
20233 010/300 Write block 2527 len 4096 data: 5
20234 011/300 Write block 456 len 24576 data: 5 4 4 5 5 7
20235 012/300 Read block 3434 len 4096
20236 013/300 Flush
20237 014/300 Read block 4622 len 20480
20238 015/300 Read block 292 len 28672
20239 016/300 Read block 2808 len 40960
20240 017/300 Read block 4066 len 12288
20241 018/300 Write block 3146 len 4096 data: 3
20242 019/300 Write block 3742 len 40960 data: 4 4 3 4 3 3 4 4 3 3
20243 020/300 Read block 1977 len 4096
20244 021/300 Read block 4241 len 32768
20245 022/300 Write block 602 len 20480 data: 4 5 4 4 4
20246 023/300 Write block 2640 len 32768 data: 3 4 4 4 5 4 4 5
20247 024/300 Read block 4738 len 28672
20248 025/300 Flush
20249 026/300 Write block 2430 len 36864 data: 5 5 4 5 5 4 4 4 2
20250 027/300 Write block 3524 len 40960 data: 5 5 6 6 6 4 3 3 3 2
20251 028/300 Write block 4031 len 32768 data: 5 6 6 6 6 7 7 9
20252 029/300 Read block 4879 len 4096
20253 030/300 Write block 2372 len 36864 data: 2 2 2 2 2 2 2 2 2
20254 031/300 Write block 2063 len 40960 data: 4 3 2 2 3 3 3 5 5 5
20255 032/300 Read block 2010 len 32768
20256 033/300 Read block 3592 len 20480
20257 034/300 Write block 2358 len 28672 data: 6 5 4 2 2 2 2
20258 035/300 Read block 3005 len 36864
20259 036/300 Read block 831 len 8192
20260 037/300 Write block 4446 len 32768 data: 6 6 5 5 5 4 5 5
20261 038/300 Read block 4866 len 4096
20262 039/300 Read block 4744 len 4096
20263 040/300 Read block 2918 len 8192
20264 041/300 Write block 4002 len 24576 data: 2 2 3 3 4 4
20265 042/300 Read block 2781 len 16384
20266 043/300 Read block 3434 len 24576
20267 044/300 Read block 3255 len 28672
20268 045/300 Read block 2447 len 24576
20269 046/300 Read block 959 len 12288
20270 047/300 Flush
20271 048/300 Write block 2982 len 28672 data: 4 4 5 5 5 4 4
20272 049/300 Read block 65 len 16384
20273 050/300 Write block 1449 len 36864 data: 6 6 6 6 6 4 4 3 3
20274 051/300 Write block 2707 len 4096 data: 6
20275 052/300 Read block 352 len 20480
20276 053/300 Read block 4343 len 12288
20277 054/300 Write block 1222 len 40960 data: 3 4 5 4 5 5 4 4 4 4
20278 055/300 Read block 4600 len 8192
20279 056/300 Write block 1964 len 16384 data: 2 2 2 2
20280 057/300 Write block 3254 len 12288 data: 4 4 4
20281 058/300 Read block 2345 len 32768
20282 059/300 Write block 1988 len 36864 data: 8 8 7 6 4 5 5 4 4
20283 060/300 Write block 596 len 24576 data: 7 6 4 6 5 4
20284 061/300 Write block 2061 len 40960 data: 5 5 5 4 3 3 4 4 4 6
20285 062/300 Write block 358 len 20480 data: 5 4 5 6 5
20286 063/300 Read block 8 len 8192
20287 064/300 Read block 578 len 32768
20288 065/300 Write block 1702 len 4096 data: 4
20289 066/300 Write block 3325 len 4096 data: 3
20290 067/300 Read block 3853 len 28672
20291 068/300 Write block 716 len 20480 data: 5 5 6 6 6
20292 069/300 Read block 1641 len 16384
20293 070/300 Read block 4253 len 40960
20294 071/300 Write block 2854 len 40960 data: 4 4 5 5 6 4 4 3 3 3
20295 072/300 Write block 4044 len 20480 data: 2 3 3 3 3
20296 073/300 Write block 4446 len 20480 data: 7 7 6 6 6
20297 074/300 Write block 1299 len 24576 data: 5 5 5 4 3 5
20298 075/300 Flush
20299 076/300 Write block 4200 len 40960 data: 3 3 3 4 3 5 4 3 3 3
20300 077/300 Read block 4132 len 20480
20301 078/300 Write block 3352 len 24576 data: 7 7 6 7 6 4
20302 079/300 Read block 2024 len 32768
20303 080/300 Write block 321 len 36864 data: 4 4 3 4 4 4 3 3 3
20304 081/300 Read block 64 len 16384
20305 082/300 Read block 3954 len 4096
20306 083/300 Write block 933 len 16384 data: 5 4 4 5
20307 084/300 Read block 2403 len 32768
20308 085/300 Write block 139 len 32768 data: 4 3 3 3 3 4 4 3
20309 086/300 Read block 2176 len 40960
20310 087/300 Read block 4112 len 24576
20311 088/300 Read block 1408 len 28672
20312 089/300 Read block 933 len 28672
20313 090/300 Read block 3254 len 40960
20314 091/300 Read block 1482 len 4096
20315 092/300 Read block 1293 len 40960
20316 093/300 Read block 406 len 8192
20317 094/300 Flush
20318 095/300 Read block 2350 len 24576
20319 096/300 Write block 431 len 24576 data: 5 4 3 4 4 4
20320 097/300 Write block 1527 len 16384 data: 5 4 5 5
20321 098/300 Write block 3536 len 12288 data: 4 4 4
20322 099/300 Read block 1304 len 32768
20323 100/300 Flush
20324 101/300 Write block 452 len 12288 data: 4 5 5
20325 102/300 Write block 1034 len 32768 data: 3 3 5 6 7 7 5 5
20326 103/300 Read block 4293 len 40960
20327 104/300 Flush
20328 105/300 Read block 2510 len 40960
20329 106/300 Write block 3390 len 40960 data: 5 5 4 4 3 3 4 4 6 5
20330 107/300 Read block 3093 len 24576
20331 108/300 Read block 4364 len 20480
20332 109/300 Read block 2637 len 4096
20333 110/300 Write block 1396 len 24576 data: 3 3 3 2 2 2
20334 111/300 Write block 3708 len 36864 data: 6 6 5 6 6 7 7 6 5
20335 112/300 Read block 1338 len 4096
20336 113/300 Write block 3408 len 36864 data: 3 3 3 4 4 3 4 4 5
20337 114/300 Flush
20338 115/300 Read block 333 len 28672
20339 116/300 Flush
20340 117/300 Read block 4532 len 32768
20341 118/300 Write block 77 len 36864 data: 4 5 3 3 3 3 3 4 3
20342 119/300 Write block 1658 len 4096 data: 3
20343 120/300 Write block 3175 len 20480 data: 6 6 7 6 4
20344 121/300 Read block 780 len 12288
20345 122/300 Write block 572 len 28672 data: 4 5 4 4 3 3 4
20346 123/300 Write block 3788 len 24576 data: 7 8 4 5 5 6
20347 124/300 Read block 4671 len 16384
20348 125/300 Read block 3400 len 4096
20349 126/300 Flush
20350 127/300 Write block 1658 len 24576 data: 4 3 4 4 3 4
20351 128/300 Write block 4138 len 8192 data: 5 5
20352 129/300 Read block 529 len 12288
20353 130/300 Write block 3554 len 24576 data: 4 4 3 4 5 5
20354 131/300 Write block 4859 len 32768 data: 3 3 2 2 3 3 3 3
20355 132/300 Read block 1536 len 36864
20356 133/300 Write block 3021 len 8192 data: 3 3
20357 134/300 Write block 1884 len 24576 data: 4 4 4 4 5 5
20358 135/300 Write block 313 len 16384 data: 3 3 4 3
20359 136/300 Write block 4977 len 4096 data: 2
20360 137/300 Write block 4513 len 28672 data: 2 2 4 5 5 5 6
20361 138/300 Read block 744 len 16384
20362 139/300 Write block 4591 len 36864 data: 4 4 4 3 3 3 5 3 3
20363 140/300 Read block 2140 len 24576
20364 141/300 Write block 2508 len 24576 data: 2 2 2 2 2 3
20365 142/300 Read block 396 len 40960
20366 143/300 Read block 1723 len 16384
20367 144/300 Flush
20368 145/300 Write block 1808 len 24576 data: 6 8 8 8 8 7
20369 146/300 Write block 3702 len 28672 data: 8 8 7 5 3 5 7
20370 147/300 Write block 4663 len 36864 data: 4 3 3 3 3 3 4 4 5
20371 148/300 Write block 4671 len 12288 data: 6 5 5
20372 149/300 Read block 4980 len 20480
20373 150/300 Read block 4194 len 8192
20374 151/300 Read block 4836 len 28672
20375 152/300 Read block 11 len 32768
20376 153/300 Read block 1549 len 16384
20377 154/300 Write block 2246 len 28672 data: 5 5 6 6 7 5 5
20378 155/300 Write block 4186 len 28672 data: 7 6 5 6 5 4 3
20379 156/300 Read block 3517 len 4096
20380 157/300 Write block 1669 len 36864 data: 4 6 6 6 6 5 5 3 2
20381 158/300 Read block 1017 len 28672
20382 159/300 Write block 1119 len 20480 data: 5 5 5 5 4
20383 160/300 Write block 3874 len 32768 data: 4 5 6 6 6 5 5 5
20384 161/300 Write block 3619 len 4096 data: 3
20385 162/300 Write block 4114 len 4096 data: 3
20386 163/300 Read block 1004 len 36864
20387 164/300 Read block 3199 len 4096
20388 165/300 Read block 849 len 32768
20389 166/300 Read block 3542 len 16384
20390 167/300 Write block 2560 len 40960 data: 4 4 3 3 2 3 4 4 4 5
20391 168/300 Read block 985 len 36864
20392 169/300 Write block 4246 len 4096 data: 2
20393 170/300 Read block 4779 len 16384
20394 171/300 Read block 229 len 16384
20395 172/300 Read block 3320 len 32768
20396 173/300 Read block 499 len 40960
20397 174/300 Read block 2327 len 28672
20398 175/300 Write block 89 len 24576 data: 3 2 2 3 3 3
20399 176/300 Write block 4598 len 36864 data: 4 4 3 3 4 3 4 4 4
20400 177/300 Write block 1880 len 20480 data: 5 4 4 5 5
20401 178/300 Read block 4075 len 16384
20402 179/300 Read block 4037 len 20480
20403 180/300 Read block 887 len 24576
20404 181/300 Write block 4396 len 20480 data: 4 6 6 5 5
20405 182/300 Write block 4580 len 28672 data: 3 4 3 3 4 6 6
20406 183/300 Read block 2299 len 12288
20407 184/300 Write block 2753 len 8192 data: 7 6
20408 185/300 Read block 4828 len 12288
20409 186/300 Read block 2985 len 8192
20410 187/300 Read block 2466 len 24576
20411 188/300 Flush
20412 189/300 Write block 3292 len 32768 data: 4 5 5 5 6 8 6 6
20413 190/300 Read block 960 len 36864
20414 191/300 Read block 2863 len 40960
20415 192/300 Read block 797 len 24576
20416 193/300 Write block 1752 len 16384 data: 6 6 5 5
20417 194/300 Write block 1138 len 36864 data: 4 4 4 4 4 4 6 5 5
20418 195/300 Flush
20419 196/300 Read block 1156 len 28672
20420 197/300 Write block 3164 len 16384 data: 6 5 3 3
20421 198/300 Write block 1702 len 20480 data: 5 4 4 5 5
20422 199/300 Write block 4847 len 16384 data: 6 6 5 8
20423 200/300 Read block 514 len 8192
20424 201/300 Write block 3327 len 4096 data: 5
20425 202/300 Write block 2125 len 40960 data: 2 2 2 2 2 2 2 4 5 5
20426 203/300 Write block 1712 len 40960 data: 6 5 4 4 4 3 3 4 4 3
20427 204/300 Read block 2829 len 8192
20428 205/300 Read block 2950 len 36864
20429 206/300 Write block 763 len 12288 data: 5 4 4
20430 207/300 Write block 463 len 24576 data: 8 8 6 5 7 7
20431 208/300 Read block 4545 len 8192
20432 209/300 Write block 3679 len 32768 data: 4 5 6 5 5 4 3 2
20433 210/300 Write block 2044 len 12288 data: 6 6 6
20434 211/300 Read block 4794 len 24576
20435 212/300 Write block 270 len 36864 data: 5 3 3 3 3 4 3 3 3
20436 213/300 Flush
20437 214/300 Write block 1687 len 20480 data: 6 6 6 3 2
20438 215/300 Read block 360 len 28672
20439 216/300 Write block 2925 len 20480 data: 4 5 5 4 7
20440 217/300 Read block 2378 len 4096
20441 218/300 Write block 1325 len 4096 data: 9
20442 219/300 Read block 2999 len 24576
20443 220/300 Write block 2621 len 20480 data: 6 7 6 6 5
20444 221/300 Write block 3649 len 32768 data: 5 6 5 4 3 2 2 2
20445 222/300 Read block 2734 len 28672
20446 223/300 Read block 4990 len 36864
20447 224/300 Write block 4607 len 16384 data: 6 8 8 5
20448 225/300 Read block 2796 len 32768
20449 226/300 Read block 3368 len 40960
20450 227/300 Write block 1830 len 4096 data: 2
20451 228/300 Write block 2967 len 8192 data: 3 4
20452 229/300 Read block 495 len 12288
20453 230/300 Read block 1468 len 16384
20454 231/300 Read block 203 len 40960
20455 232/300 Write block 3068 len 20480 data: 4 5 5 5 3
20456 233/300 Read block 1793 len 24576
20457 234/300 Flush
20458 235/300 Read block 257 len 20480
20459 236/300 Read block 3452 len 36864
20460 237/300 Read block 835 len 36864
20461 238/300 Read block 1668 len 12288
20462 239/300 Flush
20463 240/300 Write block 2796 len 40960 data: 6 6 6 5 4 4 4 3 2 3
20464 241/300 Read block 3716 len 20480
20465 242/300 Read block 3847 len 12288
20466 243/300 Read block 4086 len 4096
20467 244/300 Read block 3277 len 28672
20468 245/300 Read block 4751 len 28672
20469 246/300 Write block 4571 len 20480 data: 5 7 8 7 7
20470 247/300 Write block 2595 len 12288 data: 3 4 4
20471 248/300 Read block 2167 len 36864
20472 249/300 Write block 1585 len 28672 data: 4 3 5 4 4 5 6
20473 250/300 Write block 1720 len 36864 data: 5 4 3 3 3 3 3 3 4
20474 251/300 Read block 1031 len 20480
20475 252/300 Write block 1654 len 32768 data: 4 3 3 3 5 4 5 5
20476 253/300 Read block 1518 len 4096
20477 254/300 Write block 3523 len 36864 data: 6 6 6 7 7 7 5 4 4
20478 255/300 Write block 1358 len 24576 data: 5 4 4 4 7 9
20479 256/300 Read block 3940 len 24576
20480 257/300 Write block 4541 len 28672 data: 6 6 7 6 6 6 5
20481 258/300 Write block 1214 len 24576 data: 2 2 2 2 2 3
20482 259/300 Write block 4846 len 12288 data: 6 7 7
20483 260/300 Read block 3073 len 8192
20484 261/300 Write block 1215 len 24576 data: 3 3 3 3 4 3
20485 262/300 Read block 3888 len 32768
20486 263/300 Write block 1484 len 24576 data: 6 6 3 3 2 4
20487 264/300 Read block 1896 len 12288
20488 265/300 Read block 1337 len 32768
20489 266/300 Read block 1749 len 28672
20490 267/300 Read block 585 len 20480
20491 268/300 Write block 180 len 36864 data: 5 7 6 7 7 7 8 9 8
20492 269/300 Write block 1483 len 32768 data: 4 7 7 4 4 3 5 4
20493 270/300 Read block 2195 len 4096
20494 271/300 Write block 3617 len 8192 data: 3 3
20495 272/300 Write block 2536 len 8192 data: 4 4
20496 273/300 Read block 945 len 4096
20497 274/300 Write block 4542 len 8192 data: 7 8
20498 275/300 Write block 2006 len 24576 data: 3 4 4 4 4 4
20499 276/300 Read block 3943 len 32768
20500 277/300 Read block 1151 len 24576
20501 278/300 Read block 4635 len 4096
20502 279/300 Write block 4625 len 20480 data: 9 9 9 5 3
20503 280/300 Read block 3691 len 20480
20504 281/300 Read block 4917 len 4096
20505 282/300 Write block 1636 len 20480 data: 2 3 3 3 4
20506 283/300 Write block 3111 len 20480 data: 4 4 4 5 5
20507 284/300 Read block 4968 len 28672
20508 285/300 Flush
20509 286/300 Read block 3842 len 24576
20510 287/300 Flush
20511 288/300 Write block 4088 len 36864 data: 3 3 3 3 3 3 4 4 4
20512 289/300 Read block 3584 len 4096
20513 290/300 Write block 678 len 24576 data: 3 3 3 3 3 5
20514 291/300 Write block 3750 len 8192 data: 4 4
20515 292/300 Write block 3877 len 40960 data: 7 7 6 6 6 5 3 2 4 6
20516 293/300 Read block 1448 len 12288
20517 294/300 Read block 329 len 12288
20518 295/300 Read block 1591 len 16384
20519 296/300 Read block 3148 len 16384
20520 297/300 Write block 3152 len 16384 data: 5 4 4 3
20521 298/300 Read block 1120 len 40960
20522 299/300 Flush
20523 300/300 Write block 136 len 4096 data: 5
20524 301/300 Write block 4874 len 8192 data: 5 4
20525 Main task has work to do, go find it
20526 got dsc Start(0)
20527 start 0
20528 Jan 21 15:51:30.047 INFO request completed, latency_us: 236, response_code: 204, uri: /start/cid/0, method: POST, req_id: 7f49b7a3-370a-4c5c-b9f7-8538ca68db7a, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
20529 Replay: started 0, returned:Ok(())
20530 ----------------------------------------------------------------
20531 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20532 States: Offline Active Active
20533 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20534 6435 Acked Read 8 Sent Done Done false
20535 6436 Acked Write 7 Sent Done Done false
20536 6437 Acked Read 2 Sent Done Done false
20537 6438 Acked Read 4 Sent Done Done false
20538 6439 Acked Read 5 Sent Done Done false
20539 6440 Acked Write 4 Sent Done Done false
20540 6441 Acked Read 10 Sent Done Done false
20541 6442 Acked Write 9 Sent Done Done false
20542 6443 Acked Write 9 Sent Done Done false
20543 6444 Acked Write 1 Sent Done Done false
20544 6445 Acked Write 6 Sent Done Done false
20545 6446 Acked Read 1 Sent Done Done false
20546 6447 Acked Flush 0 Sent Done Done false
20547 6448 Acked Read 5 Sent Done Done false
20548 6449 Acked Read 7 Sent Done Done false
20549 6450 Acked Read 10 Sent Done Done false
20550 6451 Acked Read 3 Sent Done Done false
20551 6452 Acked Write 1 Sent Done Done false
20552 6453 Acked Write 10 Sent Done Done false
20553 6454 Acked Read 1 Sent Done Done false
20554 6455 Acked Read 8 Sent Done Done false
20555 6456 Acked Write 5 Sent Done Done false
20556 6457 Acked Write 8 Sent Done Done false
20557 6458 Acked Read 7 Sent Done Done false
20558 6459 Acked Flush 0 Sent Done Done false
20559 6460 Acked Write 9 Sent Done Done false
20560 6461 Acked Write 10 Sent Done Done false
20561 6462 Acked Write 8 Sent Done Done false
20562 6463 Acked Read 1 Sent Done Done false
20563 6464 Acked Write 9 Sent Done Done false
20564 6465 Acked Write 10 Sent Done Done false
20565 6466 Acked Read 8 Sent Done Done false
20566 6467 Acked Read 5 Sent Done Done false
20567 6468 Acked Write 7 Sent Done Done false
20568 6469 Acked Read 9 Sent Done Done false
20569 6470 Acked Read 2 Sent Done Done false
20570 6471 Acked Write 8 Sent Done Done false
20571 6472 Acked Read 1 Sent Done Done false
20572 6473 Acked Read 1 Sent Done Done false
20573 6474 Acked Read 2 Sent Done Done false
20574 6475 Acked Write 6 Sent Done Done false
20575 6476 Acked Read 4 Sent Done Done false
20576 6477 Acked Read 6 Sent Done Done false
20577 6478 Acked Read 7 Sent Done Done false
20578 6479 Acked Read 6 Sent Done Done false
20579 6480 Acked Read 3 Sent Done Done false
20580 6481 Acked Flush 0 Sent Done Done false
20581 6482 Acked Write 7 Sent Done Done false
20582 6483 Acked Read 4 Sent Done Done false
20583 6484 Acked Write 9 Sent Done Done false
20584 6485 Acked Write 1 Sent Done Done false
20585 6486 Acked Read 5 Sent Done Done false
20586 6487 Acked Read 3 Sent Done Done false
20587 6488 Acked Write 10 Sent Done Done false
20588 6489 Acked Read 2 Sent Done Done false
20589 6490 Acked Write 4 Sent Done Done false
20590 6491 Acked Write 3 Sent Done Done false
20591 6492 Acked Read 8 Sent Done Done false
20592 6493 Acked Write 9 Sent Done Done false
20593 6494 Acked Write 6 Sent Done Done false
20594 6495 Acked Write 10 Sent Done Done false
20595 6496 Acked Write 5 Sent Done Done false
20596 6497 Acked Read 2 Sent Done Done false
20597 6498 Acked Read 8 Sent Done Done false
20598 6499 Acked Write 1 Sent Done Done false
20599 6500 Acked Write 1 Sent Done Done false
20600 6501 Acked Read 7 Sent Done Done false
20601 6502 Acked Write 5 Sent Done Done false
20602 6503 Acked Read 4 Sent Done Done false
20603 6504 Acked Read 10 Sent Done Done false
20604 6505 Acked Write 10 Sent Done Done false
20605 6506 Acked Write 5 Sent Done Done false
20606 6507 Acked Write 5 Sent Done Done false
20607 6508 Acked Write 6 Sent Done Done false
20608 6509 Acked Flush 0 Sent Done Done false
20609 6510 Acked Write 10 Sent Done Done false
20610 6511 Acked Read 5 Sent Done Done false
20611 6512 Acked Write 6 Sent Done Done false
20612 6513 Acked Read 8 Sent Done Done false
20613 6514 Acked Write 9 Sent Done Done false
20614 6515 Acked Read 4 Sent Done Done false
20615 6516 Acked Read 1 Sent Done Done false
20616 6517 Acked Write 4 Sent Done Done false
20617 6518 Acked Read 8 Sent Done Done false
20618 6519 Acked Write 8 Sent Done Done false
20619 6520 Acked Read 10 Sent Done Done false
20620 6521 Acked Read 6 Sent Done Done false
20621 6522 Acked Read 7 Sent Done Done false
20622 6523 Acked Read 7 Sent Done Done false
20623 6524 Acked Read 10 Sent Done Done false
20624 6525 Acked Read 1 Sent Done Done false
20625 6526 Acked Read 10 Sent Done Done false
20626 6527 Acked Read 2 Sent Done Done false
20627 6528 Acked Flush 0 Sent Done Done false
20628 6529 Acked Read 6 Sent Done Done false
20629 6530 Acked Write 6 Sent Done Done false
20630 6531 Acked Write 4 Sent Done Done false
20631 6532 Acked Write 3 Sent Done Done false
20632 6533 Acked Read 8 Sent Done Done false
20633 6534 Acked Flush 0 Sent Done Done false
20634 6535 Acked Write 3 Sent Done Done false
20635 6536 Acked Write 8 Sent Done Done false
20636 6537 Acked Read 10 Sent Done Done false
20637 6538 Acked Flush 0 Sent Done Done false
20638 6539 Acked Read 10 Sent Done Done false
20639 6540 Acked Write 10 Sent Done Done false
20640 6541 Acked Read 6 Sent Done Done false
20641 6542 Acked Read 5 Sent Done Done false
20642 6543 Acked Read 1 Sent Done Done false
20643 6544 Acked Write 6 Sent Done Done false
20644 6545 Acked Write 9 Sent Done Done false
20645 6546 Acked Read 1 Sent Done Done false
20646 6547 Acked Write 9 Sent Done Done false
20647 6548 Acked Flush 0 Sent Done Done false
20648 6549 Acked Read 7 Sent Done Done false
20649 6550 Acked Flush 0 Sent Done Done false
20650 6551 Acked Read 8 Sent Done Done false
20651 6552 Acked Write 9 Sent Done Done false
20652 6553 Acked Write 1 Sent Done Done false
20653 6554 Acked Write 5 Sent Done Done false
20654 6555 Acked Read 3 Sent Done Done false
20655 6556 Acked Write 7 Sent Done Done false
20656 6557 Acked Write 6 Sent Done Done false
20657 6558 Acked Read 4 Sent Done Done false
20658 6559 Acked Read 1 Sent Done Done false
20659 6560 Acked Flush 0 Sent Done Done false
20660 6561 Acked Write 6 Sent Done Done false
20661 6562 Acked Write 2 Sent Done Done false
20662 6563 Acked Read 3 Sent Done Done false
20663 6564 Acked Write 6 Sent Done Done false
20664 6565 Acked Write 8 Sent Done Done false
20665 6566 Acked Read 9 Sent Done Done false
20666 6567 Acked Write 2 Sent Done Done false
20667 6568 Acked Write 6 Sent Done Done false
20668 6569 Acked Write 4 Sent Done Done false
20669 6570 Acked Write 1 Sent Done Done false
20670 6571 Acked Write 7 Sent Done Done false
20671 6572 Acked Read 4 Sent Done Done false
20672 6573 Acked Write 9 Sent Done Done false
20673 6574 Acked Read 6 Sent Done Done false
20674 6575 Acked Write 6 Sent Done Done false
20675 6576 Acked Read 10 Sent Done Done false
20676 6577 Acked Read 4 Sent Done Done false
20677 6578 Acked Flush 0 Sent Done Done false
20678 6579 Acked Write 6 Sent Done Done false
20679 6580 Acked Write 7 Sent Done Done false
20680 6581 Acked Write 9 Sent Done Done false
20681 6582 Acked Write 3 Sent Done Done false
20682 6583 Acked Read 5 Sent Done Done false
20683 6584 Acked Read 2 Sent Done Done false
20684 6585 Acked Read 7 Sent Done Done false
20685 6586 Acked Read 8 Sent Done Done false
20686 6587 Acked Read 4 Sent Done Done false
20687 6588 Acked Write 7 Sent Done Done false
20688 6589 Acked Write 7 Sent Done Done false
20689 6590 Acked Read 1 Sent Done Done false
20690 6591 Acked Write 9 Sent Done Done false
20691 6592 Acked Read 7 Sent Done Done false
20692 6593 Acked Write 5 Sent Done Done false
20693 6594 Acked Write 8 Sent Done Done false
20694 6595 Acked Write 1 Sent Done Done false
20695 6596 Acked Write 1 Sent Done Done false
20696 6597 Acked Read 9 Sent Done Done false
20697 6598 Acked Read 1 Sent Done Done false
20698 6599 Acked Read 8 Sent Done Done false
20699 6600 Acked Read 4 Sent Done Done false
20700 6601 Acked Write 10 Sent Done Done false
20701 6602 Acked Read 9 Sent Done Done false
20702 6603 Acked Write 1 Sent Done Done false
20703 6604 Acked Read 4 Sent Done Done false
20704 6605 Acked Read 4 Sent Done Done false
20705 6606 Acked Read 8 Sent Done Done false
20706 6607 Acked Read 10 Sent Done Done false
20707 6608 Acked Read 7 Sent Done Done false
20708 6609 Acked Write 6 Sent Done Done false
20709 6610 Acked Write 9 Sent Done Done false
20710 6611 Acked Write 5 Sent Done Done false
20711 6612 Acked Read 4 Sent Done Done false
20712 6613 Acked Read 5 Sent Done Done false
20713 6614 Acked Read 6 Sent Done Done false
20714 6615 Acked Write 5 Sent Done Done false
20715 6616 Acked Write 7 Sent Done Done false
20716 6617 Acked Read 3 Sent Done Done false
20717 6618 Acked Write 2 Sent Done Done false
20718 6619 Acked Read 3 Sent Done Done false
20719 6620 Acked Read 2 Sent Done Done false
20720 6621 Acked Read 6 Sent Done Done false
20721 6622 Acked Flush 0 Sent Done Done false
20722 6623 Acked Write 8 Sent Done Done false
20723 6624 Acked Read 9 Sent Done Done false
20724 6625 Acked Read 10 Sent Done Done false
20725 6626 Acked Read 6 Sent Done Done false
20726 6627 Acked Write 4 Sent Done Done false
20727 6628 Acked Write 9 Sent Done Done false
20728 6629 Acked Flush 0 Sent Done Done false
20729 6630 Acked Read 7 Sent Done Done false
20730 6631 Acked Write 4 Sent Done Done false
20731 6632 Acked Write 5 Sent Done Done false
20732 6633 Acked Write 4 Sent Done Done false
20733 6634 Acked Read 2 Sent Done Done false
20734 6635 Acked Write 1 Sent Done Done false
20735 6636 Acked Write 10 Sent Done Done false
20736 6637 Acked Write 10 Sent Done Done false
20737 6638 Acked Read 2 Sent Done Done false
20738 6639 Acked Read 9 Sent Done Done false
20739 6640 Acked Write 3 Sent Done Done false
20740 6641 Acked Write 6 Sent Done Done false
20741 6642 Acked Read 2 Sent Done Done false
20742 6643 Acked Write 8 Sent Done Done false
20743 6644 Acked Write 3 Sent Done Done false
20744 6645 Acked Read 6 Sent Done Done false
20745 6646 Acked Write 9 Sent Done Done false
20746 6647 Acked Flush 0 Sent Done Done false
20747 6648 Acked Write 5 Sent Done Done false
20748 6649 Acked Read 7 Sent Done Done false
20749 6650 Acked Write 5 Sent Done Done false
20750 6651 Acked Read 1 Sent Done Done false
20751 6652 Acked Write 1 Sent Done Done false
20752 6653 Acked Read 6 Sent Done Done false
20753 6654 Acked Write 5 Sent Done Done false
20754 6655 Acked Write 8 Sent Done Done false
20755 6656 Acked Read 7 Sent Done Done false
20756 6657 Acked Read 9 Sent Done Done false
20757 6658 Acked Write 4 Sent Done Done false
20758 6659 Acked Read 8 Sent Done Done false
20759 6660 Acked Read 10 Sent Done Done false
20760 6661 Acked Write 1 Sent Done Done false
20761 6662 Acked Write 2 Sent Done Done false
20762 6663 Acked Read 3 Sent Done Done false
20763 6664 Acked Read 4 Sent Done Done false
20764 6665 Acked Read 10 Sent Done Done false
20765 6666 Acked Write 5 Sent Done Done false
20766 6667 Acked Read 6 Sent Done Done false
20767 6668 Acked Flush 0 Sent Done Done false
20768 6669 Acked Read 5 Sent Done Done false
20769 6670 Acked Read 9 Sent Done Done false
20770 6671 Acked Read 9 Sent Done Done false
20771 6672 Acked Read 3 Sent Done Done false
20772 6673 Acked Flush 0 Sent Done Done false
20773 6674 Acked Write 10 Sent Done Done false
20774 6675 Acked Read 5 Sent Done Done false
20775 6676 Acked Read 3 Sent Done Done false
20776 6677 Acked Read 1 Sent Done Done false
20777 6678 Acked Read 7 Sent Done Done false
20778 6679 Acked Read 7 Sent Done Done false
20779 6680 Acked Write 5 Sent Done Done false
20780 6681 Acked Write 3 Sent Done Done false
20781 6682 Acked Read 9 Sent Done Done false
20782 6683 Acked Write 7 Sent Done Done false
20783 6684 Acked Write 9 Sent Done Done false
20784 6685 Acked Read 5 Sent Done Done false
20785 6686 Acked Write 8 Sent Done Done false
20786 6687 Acked Read 1 Sent Done Done false
20787 6688 Acked Write 9 Sent Done Done false
20788 6689 Acked Write 6 Sent Done Done false
20789 6690 Acked Read 6 Sent Done Done false
20790 6691 Acked Write 7 Sent Done Done false
20791 6692 Acked Write 6 Sent Done Done false
20792 6693 Acked Write 3 Sent Done Done false
20793 6694 Acked Read 2 Sent Done Done false
20794 6695 Acked Write 6 Sent Done Done false
20795 6696 Acked Read 8 Sent Done Done false
20796 6697 Acked Write 6 Sent Done Done false
20797 6698 Acked Read 3 Sent Done Done false
20798 6699 Acked Read 8 Sent Done Done false
20799 6700 Acked Read 7 Sent Done Done false
20800 6701 Acked Read 5 Sent Done Done false
20801 6702 Acked Write 9 Sent Done Done false
20802 6703 Acked Write 8 Sent Done Done false
20803 6704 Acked Read 1 Sent Done Done false
20804 6705 Acked Write 2 Sent Done Done false
20805 6706 Acked Write 2 Sent Done Done false
20806 6707 Acked Read 1 Sent Done Done false
20807 6708 Acked Write 2 Sent Done Done false
20808 6709 Acked Write 6 Sent Done Done false
20809 6710 Acked Read 8 Sent Done Done false
20810 6711 Acked Read 6 Sent Done Done false
20811 6712 Acked Read 1 Sent Done Done false
20812 6713 Acked Write 5 Sent Done Done false
20813 6714 Acked Read 5 Sent Done Done false
20814 6715 Acked Read 1 Sent Done Done false
20815 6716 Acked Write 5 Sent Done Done false
20816 6717 Acked Write 5 Sent Done Done false
20817 6718 Acked Read 7 Sent Done Done false
20818 6719 Acked Flush 0 Sent Done Done false
20819 6720 Acked Read 6 Sent Done Done false
20820 6721 Acked Flush 0 Sent Done Done false
20821 6722 Acked Write 9 Sent Done Done false
20822 6723 Acked Read 1 Sent Done Done false
20823 6724 Acked Write 6 Sent Done Done false
20824 6725 Acked Write 2 Sent Done Done false
20825 6726 Acked Write 10 Sent Done Done false
20826 6727 Acked Read 3 Sent Done Done false
20827 6728 Acked Read 3 Sent Done Done false
20828 6729 Acked Read 4 Sent Done Done false
20829 6730 Acked Read 4 Sent Done Done false
20830 6731 Acked Write 4 Sent Done Done false
20831 6732 Acked Read 10 Sent Done Done false
20832 6733 Acked Flush 0 Sent Done Done false
20833 6734 Acked Write 1 Sent Done Done false
20834 6735 Acked Write 2 Sent Done Done false
20835 STATES DS:0 DS:1 DS:2 TOTAL
20836 Sent 301 0 0 301
20837 Done 0 301 301 602
20838 Skipped 0 0 0 0
20839 Error 0 0 0 0
20840 Last Flush: 6434 6733 6733
20841 Downstairs last five completed: 6434 6433 6432 6431 6430
20842 Upstairs last five completed: 6735 6734 6733 6732 6731
20843 CLIENT: Up:0 ds:301 act:2
20844 [8810] got start action
20845 [8810] I am going to restart
20846 Starting downstairs at port 8810
20847 Make output file at "/tmp/dsc/downstairs-8810.txt"
20848 [8810][0] reports Starting
20849 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
20850 [8810][0] reports Running
20851 ----------------------------------------------------------------
20852 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20853 States: Offline Active Active
20854 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20855 6435 Acked Read 8 Sent Done Done false
20856 6436 Acked Write 7 Sent Done Done false
20857 6437 Acked Read 2 Sent Done Done false
20858 6438 Acked Read 4 Sent Done Done false
20859 6439 Acked Read 5 Sent Done Done false
20860 6440 Acked Write 4 Sent Done Done false
20861 6441 Acked Read 10 Sent Done Done false
20862 6442 Acked Write 9 Sent Done Done false
20863 6443 Acked Write 9 Sent Done Done false
20864 6444 Acked Write 1 Sent Done Done false
20865 6445 Acked Write 6 Sent Done Done false
20866 6446 Acked Read 1 Sent Done Done false
20867 6447 Acked Flush 0 Sent Done Done false
20868 6448 Acked Read 5 Sent Done Done false
20869 6449 Acked Read 7 Sent Done Done false
20870 6450 Acked Read 10 Sent Done Done false
20871 6451 Acked Read 3 Sent Done Done false
20872 6452 Acked Write 1 Sent Done Done false
20873 6453 Acked Write 10 Sent Done Done false
20874 6454 Acked Read 1 Sent Done Done false
20875 6455 Acked Read 8 Sent Done Done false
20876 6456 Acked Write 5 Sent Done Done false
20877 6457 Acked Write 8 Sent Done Done false
20878 6458 Acked Read 7 Sent Done Done false
20879 6459 Acked Flush 0 Sent Done Done false
20880 6460 Acked Write 9 Sent Done Done false
20881 6461 Acked Write 10 Sent Done Done false
20882 6462 Acked Write 8 Sent Done Done false
20883 6463 Acked Read 1 Sent Done Done false
20884 6464 Acked Write 9 Sent Done Done false
20885 6465 Acked Write 10 Sent Done Done false
20886 6466 Acked Read 8 Sent Done Done false
20887 6467 Acked Read 5 Sent Done Done false
20888 6468 Acked Write 7 Sent Done Done false
20889 6469 Acked Read 9 Sent Done Done false
20890 6470 Acked Read 2 Sent Done Done false
20891 6471 Acked Write 8 Sent Done Done false
20892 6472 Acked Read 1 Sent Done Done false
20893 6473 Acked Read 1 Sent Done Done false
20894 6474 Acked Read 2 Sent Done Done false
20895 6475 Acked Write 6 Sent Done Done false
20896 6476 Acked Read 4 Sent Done Done false
20897 6477 Acked Read 6 Sent Done Done false
20898 6478 Acked Read 7 Sent Done Done false
20899 6479 Acked Read 6 Sent Done Done false
20900 6480 Acked Read 3 Sent Done Done false
20901 6481 Acked Flush 0 Sent Done Done false
20902 6482 Acked Write 7 Sent Done Done false
20903 6483 Acked Read 4 Sent Done Done false
20904 6484 Acked Write 9 Sent Done Done false
20905 6485 Acked Write 1 Sent Done Done false
20906 6486 Acked Read 5 Sent Done Done false
20907 6487 Acked Read 3 Sent Done Done false
20908 6488 Acked Write 10 Sent Done Done false
20909 6489 Acked Read 2 Sent Done Done false
20910 6490 Acked Write 4 Sent Done Done false
20911 6491 Acked Write 3 Sent Done Done false
20912 6492 Acked Read 8 Sent Done Done false
20913 6493 Acked Write 9 Sent Done Done false
20914 6494 Acked Write 6 Sent Done Done false
20915 6495 Acked Write 10 Sent Done Done false
20916 6496 Acked Write 5 Sent Done Done false
20917 6497 Acked Read 2 Sent Done Done false
20918 6498 Acked Read 8 Sent Done Done false
20919 6499 Acked Write 1 Sent Done Done false
20920 6500 Acked Write 1 Sent Done Done false
20921 6501 Acked Read 7 Sent Done Done false
20922 6502 Acked Write 5 Sent Done Done false
20923 6503 Acked Read 4 Sent Done Done false
20924 6504 Acked Read 10 Sent Done Done false
20925 6505 Acked Write 10 Sent Done Done false
20926 6506 Acked Write 5 Sent Done Done false
20927 6507 Acked Write 5 Sent Done Done false
20928 6508 Acked Write 6 Sent Done Done false
20929 6509 Acked Flush 0 Sent Done Done false
20930 6510 Acked Write 10 Sent Done Done false
20931 6511 Acked Read 5 Sent Done Done false
20932 6512 Acked Write 6 Sent Done Done false
20933 6513 Acked Read 8 Sent Done Done false
20934 6514 Acked Write 9 Sent Done Done false
20935 6515 Acked Read 4 Sent Done Done false
20936 6516 Acked Read 1 Sent Done Done false
20937 6517 Acked Write 4 Sent Done Done false
20938 6518 Acked Read 8 Sent Done Done false
20939 6519 Acked Write 8 Sent Done Done false
20940 6520 Acked Read 10 Sent Done Done false
20941 6521 Acked Read 6 Sent Done Done false
20942 6522 Acked Read 7 Sent Done Done false
20943 6523 Acked Read 7 Sent Done Done false
20944 6524 Acked Read 10 Sent Done Done false
20945 6525 Acked Read 1 Sent Done Done false
20946 6526 Acked Read 10 Sent Done Done false
20947 6527 Acked Read 2 Sent Done Done false
20948 6528 Acked Flush 0 Sent Done Done false
20949 6529 Acked Read 6 Sent Done Done false
20950 6530 Acked Write 6 Sent Done Done false
20951 6531 Acked Write 4 Sent Done Done false
20952 6532 Acked Write 3 Sent Done Done false
20953 6533 Acked Read 8 Sent Done Done false
20954 6534 Acked Flush 0 Sent Done Done false
20955 6535 Acked Write 3 Sent Done Done false
20956 6536 Acked Write 8 Sent Done Done false
20957 6537 Acked Read 10 Sent Done Done false
20958 6538 Acked Flush 0 Sent Done Done false
20959 6539 Acked Read 10 Sent Done Done false
20960 6540 Acked Write 10 Sent Done Done false
20961 6541 Acked Read 6 Sent Done Done false
20962 6542 Acked Read 5 Sent Done Done false
20963 6543 Acked Read 1 Sent Done Done false
20964 6544 Acked Write 6 Sent Done Done false
20965 6545 Acked Write 9 Sent Done Done false
20966 6546 Acked Read 1 Sent Done Done false
20967 6547 Acked Write 9 Sent Done Done false
20968 6548 Acked Flush 0 Sent Done Done false
20969 6549 Acked Read 7 Sent Done Done false
20970 6550 Acked Flush 0 Sent Done Done false
20971 6551 Acked Read 8 Sent Done Done false
20972 6552 Acked Write 9 Sent Done Done false
20973 6553 Acked Write 1 Sent Done Done false
20974 6554 Acked Write 5 Sent Done Done false
20975 6555 Acked Read 3 Sent Done Done false
20976 6556 Acked Write 7 Sent Done Done false
20977 6557 Acked Write 6 Sent Done Done false
20978 6558 Acked Read 4 Sent Done Done false
20979 6559 Acked Read 1 Sent Done Done false
20980 6560 Acked Flush 0 Sent Done Done false
20981 6561 Acked Write 6 Sent Done Done false
20982 6562 Acked Write 2 Sent Done Done false
20983 6563 Acked Read 3 Sent Done Done false
20984 6564 Acked Write 6 Sent Done Done false
20985 6565 Acked Write 8 Sent Done Done false
20986 6566 Acked Read 9 Sent Done Done false
20987 6567 Acked Write 2 Sent Done Done false
20988 6568 Acked Write 6 Sent Done Done false
20989 6569 Acked Write 4 Sent Done Done false
20990 6570 Acked Write 1 Sent Done Done false
20991 6571 Acked Write 7 Sent Done Done false
20992 6572 Acked Read 4 Sent Done Done false
20993 6573 Acked Write 9 Sent Done Done false
20994 6574 Acked Read 6 Sent Done Done false
20995 6575 Acked Write 6 Sent Done Done false
20996 6576 Acked Read 10 Sent Done Done false
20997 6577 Acked Read 4 Sent Done Done false
20998 6578 Acked Flush 0 Sent Done Done false
20999 6579 Acked Write 6 Sent Done Done false
21000 6580 Acked Write 7 Sent Done Done false
21001 6581 Acked Write 9 Sent Done Done false
21002 6582 Acked Write 3 Sent Done Done false
21003 6583 Acked Read 5 Sent Done Done false
21004 6584 Acked Read 2 Sent Done Done false
21005 6585 Acked Read 7 Sent Done Done false
21006 6586 Acked Read 8 Sent Done Done false
21007 6587 Acked Read 4 Sent Done Done false
21008 6588 Acked Write 7 Sent Done Done false
21009 6589 Acked Write 7 Sent Done Done false
21010 6590 Acked Read 1 Sent Done Done false
21011 6591 Acked Write 9 Sent Done Done false
21012 6592 Acked Read 7 Sent Done Done false
21013 6593 Acked Write 5 Sent Done Done false
21014 6594 Acked Write 8 Sent Done Done false
21015 6595 Acked Write 1 Sent Done Done false
21016 6596 Acked Write 1 Sent Done Done false
21017 6597 Acked Read 9 Sent Done Done false
21018 6598 Acked Read 1 Sent Done Done false
21019 6599 Acked Read 8 Sent Done Done false
21020 6600 Acked Read 4 Sent Done Done false
21021 6601 Acked Write 10 Sent Done Done false
21022 6602 Acked Read 9 Sent Done Done false
21023 6603 Acked Write 1 Sent Done Done false
21024 6604 Acked Read 4 Sent Done Done false
21025 6605 Acked Read 4 Sent Done Done false
21026 6606 Acked Read 8 Sent Done Done false
21027 6607 Acked Read 10 Sent Done Done false
21028 6608 Acked Read 7 Sent Done Done false
21029 6609 Acked Write 6 Sent Done Done false
21030 6610 Acked Write 9 Sent Done Done false
21031 6611 Acked Write 5 Sent Done Done false
21032 6612 Acked Read 4 Sent Done Done false
21033 6613 Acked Read 5 Sent Done Done false
21034 6614 Acked Read 6 Sent Done Done false
21035 6615 Acked Write 5 Sent Done Done false
21036 6616 Acked Write 7 Sent Done Done false
21037 6617 Acked Read 3 Sent Done Done false
21038 6618 Acked Write 2 Sent Done Done false
21039 6619 Acked Read 3 Sent Done Done false
21040 6620 Acked Read 2 Sent Done Done false
21041 6621 Acked Read 6 Sent Done Done false
21042 6622 Acked Flush 0 Sent Done Done false
21043 6623 Acked Write 8 Sent Done Done false
21044 6624 Acked Read 9 Sent Done Done false
21045 6625 Acked Read 10 Sent Done Done false
21046 6626 Acked Read 6 Sent Done Done false
21047 6627 Acked Write 4 Sent Done Done false
21048 6628 Acked Write 9 Sent Done Done false
21049 6629 Acked Flush 0 Sent Done Done false
21050 6630 Acked Read 7 Sent Done Done false
21051 6631 Acked Write 4 Sent Done Done false
21052 6632 Acked Write 5 Sent Done Done false
21053 6633 Acked Write 4 Sent Done Done false
21054 6634 Acked Read 2 Sent Done Done false
21055 6635 Acked Write 1 Sent Done Done false
21056 6636 Acked Write 10 Sent Done Done false
21057 6637 Acked Write 10 Sent Done Done false
21058 6638 Acked Read 2 Sent Done Done false
21059 6639 Acked Read 9 Sent Done Done false
21060 6640 Acked Write 3 Sent Done Done false
21061 6641 Acked Write 6 Sent Done Done false
21062 6642 Acked Read 2 Sent Done Done false
21063 6643 Acked Write 8 Sent Done Done false
21064 6644 Acked Write 3 Sent Done Done false
21065 6645 Acked Read 6 Sent Done Done false
21066 6646 Acked Write 9 Sent Done Done false
21067 6647 Acked Flush 0 Sent Done Done false
21068 6648 Acked Write 5 Sent Done Done false
21069 6649 Acked Read 7 Sent Done Done false
21070 6650 Acked Write 5 Sent Done Done false
21071 6651 Acked Read 1 Sent Done Done false
21072 6652 Acked Write 1 Sent Done Done false
21073 6653 Acked Read 6 Sent Done Done false
21074 6654 Acked Write 5 Sent Done Done false
21075 6655 Acked Write 8 Sent Done Done false
21076 6656 Acked Read 7 Sent Done Done false
21077 6657 Acked Read 9 Sent Done Done false
21078 6658 Acked Write 4 Sent Done Done false
21079 6659 Acked Read 8 Sent Done Done false
21080 6660 Acked Read 10 Sent Done Done false
21081 6661 Acked Write 1 Sent Done Done false
21082 6662 Acked Write 2 Sent Done Done false
21083 6663 Acked Read 3 Sent Done Done false
21084 6664 Acked Read 4 Sent Done Done false
21085 6665 Acked Read 10 Sent Done Done false
21086 6666 Acked Write 5 Sent Done Done false
21087 6667 Acked Read 6 Sent Done Done false
21088 6668 Acked Flush 0 Sent Done Done false
21089 6669 Acked Read 5 Sent Done Done false
21090 6670 Acked Read 9 Sent Done Done false
21091 6671 Acked Read 9 Sent Done Done false
21092 6672 Acked Read 3 Sent Done Done false
21093 6673 Acked Flush 0 Sent Done Done false
21094 6674 Acked Write 10 Sent Done Done false
21095 6675 Acked Read 5 Sent Done Done false
21096 6676 Acked Read 3 Sent Done Done false
21097 6677 Acked Read 1 Sent Done Done false
21098 6678 Acked Read 7 Sent Done Done false
21099 6679 Acked Read 7 Sent Done Done false
21100 6680 Acked Write 5 Sent Done Done false
21101 6681 Acked Write 3 Sent Done Done false
21102 6682 Acked Read 9 Sent Done Done false
21103 6683 Acked Write 7 Sent Done Done false
21104 6684 Acked Write 9 Sent Done Done false
21105 6685 Acked Read 5 Sent Done Done false
21106 6686 Acked Write 8 Sent Done Done false
21107 6687 Acked Read 1 Sent Done Done false
21108 6688 Acked Write 9 Sent Done Done false
21109 6689 Acked Write 6 Sent Done Done false
21110 6690 Acked Read 6 Sent Done Done false
21111 6691 Acked Write 7 Sent Done Done false
21112 6692 Acked Write 6 Sent Done Done false
21113 6693 Acked Write 3 Sent Done Done false
21114 6694 Acked Read 2 Sent Done Done false
21115 6695 Acked Write 6 Sent Done Done false
21116 6696 Acked Read 8 Sent Done Done false
21117 6697 Acked Write 6 Sent Done Done false
21118 6698 Acked Read 3 Sent Done Done false
21119 6699 Acked Read 8 Sent Done Done false
21120 6700 Acked Read 7 Sent Done Done false
21121 6701 Acked Read 5 Sent Done Done false
21122 6702 Acked Write 9 Sent Done Done false
21123 6703 Acked Write 8 Sent Done Done false
21124 6704 Acked Read 1 Sent Done Done false
21125 6705 Acked Write 2 Sent Done Done false
21126 6706 Acked Write 2 Sent Done Done false
21127 6707 Acked Read 1 Sent Done Done false
21128 6708 Acked Write 2 Sent Done Done false
21129 6709 Acked Write 6 Sent Done Done false
21130 6710 Acked Read 8 Sent Done Done false
21131 6711 Acked Read 6 Sent Done Done false
21132 6712 Acked Read 1 Sent Done Done false
21133 6713 Acked Write 5 Sent Done Done false
21134 6714 Acked Read 5 Sent Done Done false
21135 6715 Acked Read 1 Sent Done Done false
21136 6716 Acked Write 5 Sent Done Done false
21137 6717 Acked Write 5 Sent Done Done false
21138 6718 Acked Read 7 Sent Done Done false
21139 6719 Acked Flush 0 Sent Done Done false
21140 6720 Acked Read 6 Sent Done Done false
21141 6721 Acked Flush 0 Sent Done Done false
21142 6722 Acked Write 9 Sent Done Done false
21143 6723 Acked Read 1 Sent Done Done false
21144 6724 Acked Write 6 Sent Done Done false
21145 6725 Acked Write 2 Sent Done Done false
21146 6726 Acked Write 10 Sent Done Done false
21147 6727 Acked Read 3 Sent Done Done false
21148 6728 Acked Read 3 Sent Done Done false
21149 6729 Acked Read 4 Sent Done Done false
21150 6730 Acked Read 4 Sent Done Done false
21151 6731 Acked Write 4 Sent Done Done false
21152 6732 Acked Read 10 Sent Done Done false
21153 6733 Acked Flush 0 Sent Done Done false
21154 6734 Acked Write 1 Sent Done Done false
21155 6735 Acked Write 2 Sent Done Done false
21156 6736 Acked Flush 0 Sent Done Done false
21157 STATES DS:0 DS:1 DS:2 TOTAL
21158 Sent 302 0 0 302
21159 Done 0 302 302 604
21160 Skipped 0 0 0 0
21161 Error 0 0 0 0
21162 Last Flush: 6434 6736 6736
21163 Downstairs last five completed: 6434 6433 6432 6431 6430
21164 Upstairs last five completed: 6736 6735 6734 6733 6732
21165 CLIENT: Up:0 ds:302 act:2
211662025-01-21T15:51:35.803ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211672025-01-21T15:51:35.803ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:48867) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211682025-01-21T15:51:35.804ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211692025-01-21T15:51:35.805ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211702025-01-21T15:51:35.805ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211712025-01-21T15:51:35.805ZINFOcrucible: send last flush ID to this DS: 6434 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211722025-01-21T15:51:35.805ZINFOcrucible: Replied this last flush ID: 6434 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211732025-01-21T15:51:35.805ZINFOcrucible: [0] client re-new 302 jobs since flush 6434 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211742025-01-21T15:51:35.805ZINFOcrucible: [0] Replayed 302 jobs since flush: 6434 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
21175 ----------------------------------------------------------------
21176 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21177 Downstairs last five completed: 6736 6735 6734 6733 6732
21178 Upstairs last five completed: 6736 6735 6734 6733 6732
21179 CLIENT: Up:0 ds:0 act:3
21180 Replay: All jobs finished
21181 Main task has work to do, go find it
21182 got dsc Stop(1)
21183 stop 1
21184 [8820] Got stop action so:false kr:false
21185 Jan 21 15:51:38.053 INFO request completed, latency_us: 623, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 59ebb512-2871-47ea-8182-7db29ca0c100, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
211862025-01-21T15:51:38.054ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211872025-01-21T15:51:38.054ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
211882025-01-21T15:51:38.054ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
21189 Jan 21 15:51:38.054 INFO request completed, latency_us: 149, response_code: 200, uri: /state/cid/1, method: GET, req_id: c3c50868-95e9-4b99-a9cb-a276127997f8, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
211902025-01-21T15:51:38.054ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
21191 [8820] Exited with: ExitStatus(unix_wait_status(9))
21192 [8820][1] reports Exit
21193 Jan 21 15:51:42.057 INFO request completed, latency_us: 398, response_code: 200, uri: /state/cid/1, method: GET, req_id: c9712628-ccd4-4b26-95a6-e4e5f8e3ec16, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
21194 001/300 Write block 2893 len 20480 data: 4 4 4 4 4
21195 002/300 Write block 3601 len 32768 data: 4 3 2 2 3 4 5 5
21196 003/300 Write block 75 len 4096 data: 4
21197 004/300 Flush
21198 005/300 Write block 72 len 16384 data: 7 6 7 5
21199 006/300 Flush
21200 007/300 Read block 3133 len 36864
21201 008/300 Read block 3459 len 28672
21202 009/300 Read block 3739 len 8192
21203 010/300 Read block 1181 len 28672
21204 011/300 Write block 4236 len 12288 data: 4 3 2
21205 012/300 Write block 4436 len 8192 data: 6 7
21206 013/300 Write block 3197 len 40960 data: 3 5 7 7 8 7 8 8 6 4
21207 014/300 Write block 3441 len 32768 data: 5 5 6 6 6 4 3 3
21208 015/300 Write block 4347 len 32768 data: 5 6 6 7 6 5 6 6
21209 016/300 Read block 723 len 8192
21210 017/300 Write block 3491 len 28672 data: 4 4 6 5 2 3 4
21211 018/300 Write block 2821 len 28672 data: 4 5 5 4 4 4 6
21212 019/300 Read block 2667 len 24576
21213 020/300 Write block 2223 len 40960 data: 2 4 4 4 6 6 5 4 3 3
21214 021/300 Write block 77 len 20480 data: 5 6 4 4 4
21215 022/300 Read block 981 len 28672
21216 023/300 Read block 4178 len 4096
21217 024/300 Read block 3397 len 8192
21218 025/300 Write block 648 len 8192 data: 6 6
21219 026/300 Read block 2693 len 32768
21220 027/300 Read block 3818 len 28672
21221 028/300 Read block 2762 len 12288
21222 029/300 Read block 2271 len 24576
21223 030/300 Read block 1635 len 28672
21224 031/300 Read block 648 len 32768
21225 032/300 Flush
21226 033/300 Write block 3554 len 20480 data: 5 5 4 5 6
21227 034/300 Write block 3848 len 36864 data: 3 3 3 3 3 7 8 9 9
21228 035/300 Read block 3678 len 24576
21229 036/300 Read block 4200 len 32768
21230 037/300 Write block 1108 len 16384 data: 6 7 6 6
21231 038/300 Flush
21232 039/300 Read block 1712 len 20480
21233 040/300 Write block 4430 len 40960 data: 3 4 5 5 5 5 7 8 7 8
21234 041/300 Read block 310 len 28672
21235 042/300 Read block 3059 len 36864
21236 043/300 Read block 2985 len 28672
21237 044/300 Write block 2041 len 28672 data: 5 6 6 7 7 7 5
21238 045/300 Write block 580 len 24576 data: 4 3 2 2 2 2
21239 046/300 Read block 1686 len 24576
21240 047/300 Read block 3706 len 16384
21241 048/300 Write block 647 len 36864 data: 5 7 7 6 6 5 6 5 5
21242 049/300 Write block 1037 len 40960 data: 7 8 8 6 6 6 6 7 7 5
21243 050/300 Flush
21244 051/300 Read block 4338 len 16384
21245 052/300 Write block 896 len 8192 data: 4 3
21246 053/300 Write block 277 len 12288 data: 4 4 3
21247 054/300 Write block 3951 len 40960 data: 6 7 6 5 4 5 3 4 4 5
21248 055/300 Read block 4300 len 16384
21249 056/300 Read block 2288 len 8192
21250 057/300 Read block 2965 len 8192
21251 058/300 Write block 1045 len 20480 data: 8 6 4 5 5
21252 059/300 Read block 1568 len 12288
21253 060/300 Write block 4786 len 8192 data: 2 2
21254 061/300 Read block 2935 len 4096
21255 062/300 Flush
21256 063/300 Read block 624 len 36864
21257 064/300 Read block 4432 len 32768
21258 065/300 Write block 4990 len 28672 data: 9 7 4 4 5 6 5
21259 066/300 Read block 3379 len 12288
21260 067/300 Write block 4483 len 20480 data: 6 7 7 7 4
21261 068/300 Read block 3782 len 36864
21262 069/300 Flush
21263 070/300 Read block 3544 len 24576
21264 071/300 Read block 1938 len 36864
21265 072/300 Read block 3984 len 4096
21266 073/300 Flush
21267 074/300 Flush
21268 075/300 Read block 4411 len 32768
21269 076/300 Read block 4425 len 36864
21270 077/300 Flush
21271 078/300 Write block 974 len 8192 data: 4 4
21272 079/300 Flush
21273 080/300 Write block 3527 len 16384 data: 8 8 6 5
21274 081/300 Write block 2642 len 28672 data: 5 5 6 5 5 6 5
21275 082/300 Read block 1620 len 32768
21276 083/300 Write block 135 len 8192 data: 4 6
21277 084/300 Read block 4051 len 8192
21278 085/300 Write block 3701 len 28672 data: 7 9 9 8 6 4 6
21279 086/300 Write block 4084 len 40960 data: 3 3 4 3 4 4 4 4 4 4
21280 087/300 Write block 4673 len 20480 data: 6 5 5 4 4
21281 088/300 Read block 2157 len 40960
21282 089/300 Write block 4240 len 20480 data: 2 2 3 2 2
21283 090/300 Write block 3628 len 12288 data: 8 9 5
21284 091/300 Read block 2800 len 4096
21285 092/300 Flush
21286 093/300 Read block 2731 len 12288
21287 094/300 Read block 1796 len 12288
21288 095/300 Write block 1618 len 24576 data: 5 4 6 7 7 6
21289 096/300 Read block 719 len 4096
21290 097/300 Read block 3007 len 28672
21291 098/300 Read block 4169 len 40960
21292 099/300 Write block 613 len 28672 data: 4 6 6 5 5 6 7
21293 100/300 Write block 3236 len 12288 data: 3 3 4
21294 101/300 Write block 914 len 12288 data: 5 6 6
21295 102/300 Read block 1915 len 32768
21296 103/300 Write block 2157 len 16384 data: 2 2 2 3
21297 104/300 Read block 4193 len 28672
21298 105/300 Write block 811 len 20480 data: 7 5 4 4 5
21299 106/300 Write block 3103 len 40960 data: 3 3 4 4 4 4 4 5 5 5
21300 107/300 Read block 2436 len 12288
21301 108/300 Read block 3361 len 16384
21302 109/300 Write block 4457 len 8192 data: 5 4
21303 110/300 Read block 4976 len 16384
21304 111/300 Read block 2457 len 20480
21305 112/300 Flush
21306 113/300 Write block 1715 len 20480 data: 5 5 4 4 5
21307 114/300 Write block 2780 len 4096 data: 6
21308 115/300 Read block 2182 len 8192
21309 116/300 Write block 4303 len 24576 data: 3 3 4 4 4 4
21310 117/300 Write block 2973 len 12288 data: 6 6 5
21311 118/300 Write block 1425 len 40960 data: 3 3 4 7 8 5 4 4 6 6
21312 119/300 Write block 622 len 16384 data: 5 5 4 6
21313 120/300 Read block 3667 len 24576
21314 121/300 Read block 4523 len 16384
21315 122/300 Read block 2657 len 16384
21316 123/300 Write block 1152 len 12288 data: 4 5 7
21317 124/300 Read block 849 len 36864
21318 125/300 Read block 529 len 16384
21319 126/300 Write block 237 len 32768 data: 4 4 4 4 4 3 3 2
21320 127/300 Write block 2600 len 12288 data: 4 6 6
21321 128/300 Write block 2490 len 12288 data: 4 5 6
21322 129/300 Read block 3072 len 8192
21323 130/300 Write block 3935 len 16384 data: 3 4 3 3
21324 131/300 Read block 2745 len 8192
21325 132/300 Flush
21326 133/300 Write block 1193 len 40960 data: 6 5 6 6 5 5 5 6 5 4
21327 134/300 Write block 627 len 28672 data: 7 5 5 6 5 4 3
21328 135/300 Read block 3963 len 28672
21329 136/300 Read block 2765 len 36864
21330 137/300 Flush
21331 138/300 Write block 3614 len 12288 data: 3 3 3
21332 139/300 Write block 1024 len 24576 data: 3 5 5 5 4 5
21333 140/300 Write block 4688 len 4096 data: 3
21334 141/300 Write block 208 len 4096 data: 5
21335 142/300 Read block 1476 len 16384
21336 143/300 Flush
21337 144/300 Flush
21338 145/300 Read block 1592 len 8192
21339 146/300 Read block 3786 len 24576
21340 147/300 Flush
21341 148/300 Write block 2796 len 28672 data: 7 7 7 6 5 5 5
21342 149/300 Write block 950 len 20480 data: 2 2 2 3 4
21343 150/300 Read block 1607 len 32768
21344 151/300 Write block 1507 len 40960 data: 5 5 5 7 8 8 6 7 7 7
21345 152/300 Read block 1752 len 32768
21346 153/300 Write block 3064 len 20480 data: 4 4 4 4 5
21347 154/300 Write block 809 len 12288 data: 6 6 8
21348 155/300 Write block 3282 len 12288 data: 6 7 5
21349 156/300 Read block 1734 len 28672
21350 157/300 Read block 4805 len 28672
21351 158/300 Read block 2931 len 12288
21352 159/300 Write block 1136 len 8192 data: 2 4
21353 160/300 Flush
21354 161/300 Read block 2365 len 36864
21355 162/300 Write block 4936 len 8192 data: 3 3
21356 163/300 Write block 2767 len 24576 data: 3 3 2 2 2 2
21357 164/300 Read block 544 len 40960
21358 165/300 Read block 3382 len 16384
21359 166/300 Write block 178 len 4096 data: 5
21360 167/300 Read block 74 len 28672
21361 168/300 Write block 2733 len 16384 data: 3 3 4 5
21362 169/300 Read block 2286 len 40960
21363 170/300 Write block 4204 len 36864 data: 4 6 5 4 4 4 3 3 2
21364 171/300 Write block 284 len 36864 data: 6 5 5 4 4 4 4 3 2
21365 172/300 Read block 1139 len 24576
21366 173/300 Write block 4431 len 24576 data: 5 6 6 6 6 8
21367 174/300 Read block 2809 len 36864
21368 175/300 Flush
21369 176/300 Write block 1045 len 16384 data: 9 7 5 6
21370 177/300 Write block 1287 len 32768 data: 3 3 3 3 4 5 5 5
21371 178/300 Write block 3566 len 20480 data: 7 7 7 7 6
21372 179/300 Write block 3812 len 36864 data: 6 6 7 7 7 6 4 3 3
21373 180/300 Flush
21374 181/300 Write block 4464 len 36864 data: 3 3 3 4 6 6 6 5 4
21375 182/300 Write block 4131 len 20480 data: 4 6 4 3 3
21376 183/300 Read block 3784 len 24576
21377 184/300 Read block 2734 len 24576
21378 185/300 Write block 3763 len 8192 data: 4 5
21379 186/300 Write block 2878 len 24576 data: 4 4 4 4 4 4
21380 187/300 Read block 16 len 12288
21381 188/300 Write block 1555 len 20480 data: 3 3 3 4 4
21382 189/300 Write block 4709 len 32768 data: 5 3 4 3 4 4 4 4
21383 190/300 Write block 1618 len 12288 data: 6 5 7
21384 191/300 Flush
21385 192/300 Read block 3317 len 16384
21386 193/300 Write block 1572 len 28672 data: 5 4 6 6 7 7 7
21387 194/300 Read block 3503 len 36864
21388 195/300 Write block 225 len 8192 data: 5 5
21389 196/300 Read block 2525 len 28672
21390 197/300 Flush
21391 198/300 Write block 2187 len 16384 data: 4 5 6 6
21392 199/300 Write block 1346 len 40960 data: 5 5 5 5 5 4 6 6 4 4
21393 200/300 Read block 2416 len 32768
21394 201/300 Write block 2718 len 8192 data: 5 4
21395 202/300 Flush
21396 203/300 Flush
21397 204/300 Read block 2306 len 24576
21398 205/300 Write block 2108 len 4096 data: 5
21399 206/300 Write block 3539 len 24576 data: 4 4 4 3 4 5
21400 207/300 Write block 2256 len 12288 data: 4 4 4
21401 208/300 Write block 3214 len 4096 data: 3
21402 209/300 Write block 2372 len 4096 data: 3
21403 210/300 Write block 31 len 12288 data: 4 4 2
21404 211/300 Read block 1868 len 36864
21405 212/300 Read block 326 len 28672
21406 213/300 Read block 2392 len 12288
21407 214/300 Write block 3381 len 4096 data: 6
21408 215/300 Flush
21409 216/300 Read block 1517 len 28672
21410 217/300 Read block 32 len 32768
21411 218/300 Flush
21412 219/300 Read block 1803 len 24576
21413 220/300 Write block 159 len 28672 data: 3 3 3 4 7 7 6
21414 221/300 Read block 2827 len 12288
21415 222/300 Write block 2007 len 28672 data: 5 5 5 5 5 4 5
21416 223/300 Write block 318 len 4096 data: 3
21417 224/300 Write block 2255 len 40960 data: 4 5 5 5 5 5 4 5 6 5
21418 225/300 Write block 3899 len 8192 data: 2 2
21419 226/300 Write block 1120 len 4096 data: 6
21420 227/300 Write block 2040 len 4096 data: 5
21421 228/300 Write block 4317 len 16384 data: 4 4 3 4
21422 229/300 Write block 785 len 4096 data: 3
21423 230/300 Write block 3203 len 8192 data: 9 9
21424 231/300 Write block 158 len 8192 data: 4 4
21425 232/300 Read block 917 len 40960
21426 233/300 Read block 605 len 32768
21427 234/300 Flush
21428 235/300 Write block 3832 len 36864 data: 5 5 5 7 6 6 7 5 4
21429 236/300 Read block 820 len 8192
21430 237/300 Read block 1663 len 4096
21431 238/300 Read block 2368 len 12288
21432 239/300 Write block 3767 len 40960 data: 3 5 4 4 5 4 4 4 3 3
21433 240/300 Read block 187 len 16384
21434 241/300 Write block 103 len 24576 data: 3 3 3 3 3 3
21435 242/300 Read block 3638 len 20480
21436 243/300 Read block 2972 len 8192
21437 244/300 Write block 3689 len 12288 data: 2 2 3
21438 245/300 Read block 2441 len 20480
21439 246/300 Write block 3925 len 32768 data: 4 5 5 6 5 5 4 4
21440 247/300 Write block 3335 len 4096 data: 2
21441 248/300 Read block 4461 len 4096
21442 249/300 Flush
21443 250/300 Write block 1042 len 36864 data: 7 7 8 10 8 6 7 6 7
21444 251/300 Read block 1675 len 20480
21445 252/300 Write block 4751 len 36864 data: 5 4 4 5 7 9 7 7 7
21446 253/300 Read block 1525 len 8192
21447 254/300 Read block 4880 len 36864
21448 255/300 Write block 1093 len 32768 data: 5 5 5 6 5 3 3 4
21449 256/300 Read block 1659 len 24576
21450 257/300 Flush
21451 258/300 Flush
21452 259/300 Write block 716 len 16384 data: 6 6 7 7
21453 260/300 Write block 1407 len 28672 data: 4 4 4 4 4 5 4
21454 261/300 Write block 4493 len 32768 data: 8 8 7 6 5 4 4 3
21455 262/300 Flush
21456 263/300 Read block 3510 len 20480
21457 264/300 Read block 337 len 24576
21458 265/300 Read block 2781 len 28672
21459 266/300 Write block 2340 len 12288 data: 3 2 3
21460 267/300 Read block 3771 len 12288
21461 268/300 Write block 3135 len 32768 data: 3 3 3 3 3 3 3 3
21462 269/300 Write block 995 len 36864 data: 6 6 5 5 6 6 6 4 4
21463 270/300 Write block 3773 len 40960 data: 5 5 4 4 3 2 2 4 5 5
21464 271/300 Flush
21465 272/300 Read block 696 len 32768
21466 273/300 Write block 1556 len 12288 data: 4 4 5
21467 274/300 Read block 47 len 12288
21468 275/300 Read block 1149 len 28672
21469 276/300 Write block 405 len 16384 data: 7 7 8 7
21470 277/300 Write block 726 len 24576 data: 6 6 7 6 6 7
21471 278/300 Write block 1254 len 16384 data: 4 4 3 3
21472 279/300 Read block 2102 len 12288
21473 280/300 Read block 4675 len 28672
21474 281/300 Write block 3253 len 12288 data: 3 5 5
21475 282/300 Read block 4843 len 32768
21476 283/300 Read block 4838 len 12288
21477 284/300 Read block 4736 len 24576
21478 285/300 Write block 4545 len 20480 data: 7 7 6 3 2
21479 286/300 Write block 2591 len 12288 data: 4 5 4
21480 287/300 Write block 844 len 36864 data: 7 7 7 6 4 5 5 4 5
21481 288/300 Write block 1081 len 16384 data: 3 4 4 3
21482 289/300 Write block 1062 len 20480 data: 5 5 3 3 4
21483 290/300 Write block 521 len 4096 data: 4
21484 291/300 Read block 1586 len 40960
21485 292/300 Read block 2286 len 40960
21486 293/300 Write block 3111 len 4096 data: 6
21487 294/300 Read block 424 len 4096
21488 295/300 Write block 2874 len 36864 data: 4 4 4 4 5 5 5 5 5
21489 296/300 Read block 3415 len 36864
21490 297/300 Write block 4539 len 12288 data: 5 6 7
21491 298/300 Read block 99 len 36864
21492 299/300 Read block 2911 len 24576
21493 300/300 Read block 1647 len 16384
21494 301/300 Write block 3018 len 28672 data: 3 3 4 4 4 3 4
21495 Main task has work to do, go find it
21496 got dsc Start(1)
21497 start 1
21498 Jan 21 15:51:42.282 INFO request completed, latency_us: 249, response_code: 204, uri: /start/cid/1, method: POST, req_id: 97a8cf77-4550-4ac5-9a7a-f2f9cf527f0e, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
21499 Replay: started 1, returned:Ok(())
21500 ----------------------------------------------------------------
21501 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
21502 States: Active Offline Active
21503 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21504 6737 Acked Write 5 Done Sent Done false
21505 6738 Acked Write 8 Done Sent Done false
21506 6739 Acked Write 1 Done Sent Done false
21507 6740 Acked Flush 0 Done Sent Done false
21508 6741 Acked Write 4 Done Sent Done false
21509 6742 Acked Flush 0 Done Sent Done false
21510 6743 Acked Read 9 Done Sent Done false
21511 6744 Acked Read 7 Done Sent Done false
21512 6745 Acked Read 2 Done Sent Done false
21513 6746 Acked Read 7 Done Sent Done false
21514 6747 Acked Write 3 Done Sent Done false
21515 6748 Acked Write 2 Done Sent Done false
21516 6749 Acked Write 10 Done Sent Done false
21517 6750 Acked Write 8 Done Sent Done false
21518 6751 Acked Write 8 Done Sent Done false
21519 6752 Acked Read 2 Done Sent Done false
21520 6753 Acked Write 7 Done Sent Done false
21521 6754 Acked Write 7 Done Sent Done false
21522 6755 Acked Read 6 Done Sent Done false
21523 6756 Acked Write 10 Done Sent Done false
21524 6757 Acked Write 5 Done Sent Done false
21525 6758 Acked Read 7 Done Sent Done false
21526 6759 Acked Read 1 Done Sent Done false
21527 6760 Acked Read 2 Done Sent Done false
21528 6761 Acked Write 2 Done Sent Done false
21529 6762 Acked Read 8 Done Sent Done false
21530 6763 Acked Read 7 Done Sent Done false
21531 6764 Acked Read 3 Done Sent Done false
21532 6765 Acked Read 6 Done Sent Done false
21533 6766 Acked Read 7 Done Sent Done false
21534 6767 Acked Read 8 Done Sent Done false
21535 6768 Acked Flush 0 Done Sent Done false
21536 6769 Acked Write 5 Done Sent Done false
21537 6770 Acked Write 9 Done Sent Done false
21538 6771 Acked Read 6 Done Sent Done false
21539 6772 Acked Read 8 Done Sent Done false
21540 6773 Acked Write 4 Done Sent Done false
21541 6774 Acked Flush 0 Done Sent Done false
21542 6775 Acked Read 5 Done Sent Done false
21543 6776 Acked Write 10 Done Sent Done false
21544 6777 Acked Read 7 Done Sent Done false
21545 6778 Acked Read 9 Done Sent Done false
21546 6779 Acked Read 7 Done Sent Done false
21547 6780 Acked Write 7 Done Sent Done false
21548 6781 Acked Write 6 Done Sent Done false
21549 6782 Acked Read 6 Done Sent Done false
21550 6783 Acked Read 4 Done Sent Done false
21551 6784 Acked Write 9 Done Sent Done false
21552 6785 Acked Write 10 Done Sent Done false
21553 6786 Acked Flush 0 Done Sent Done false
21554 6787 Acked Read 4 Done Sent Done false
21555 6788 Acked Write 2 Done Sent Done false
21556 6789 Acked Write 3 Done Sent Done false
21557 6790 Acked Write 10 Done Sent Done false
21558 6791 Acked Read 4 Done Sent Done false
21559 6792 Acked Read 2 Done Sent Done false
21560 6793 Acked Read 2 Done Sent Done false
21561 6794 Acked Write 5 Done Sent Done false
21562 6795 Acked Read 3 Done Sent Done false
21563 6796 Acked Write 2 Done Sent Done false
21564 6797 Acked Read 1 Done Sent Done false
21565 6798 Acked Flush 0 Done Sent Done false
21566 6799 Acked Read 9 Done Sent Done false
21567 6800 Acked Read 8 Done Sent Done false
21568 6801 Acked Write 7 Done Sent Done false
21569 6802 Acked Read 3 Done Sent Done false
21570 6803 Acked Write 5 Done Sent Done false
21571 6804 Acked Read 9 Done Sent Done false
21572 6805 Acked Flush 0 Done Sent Done false
21573 6806 Acked Read 6 Done Sent Done false
21574 6807 Acked Read 9 Done Sent Done false
21575 6808 Acked Read 1 Done Sent Done false
21576 6809 Acked Flush 0 Done Sent Done false
21577 6810 Acked Flush 0 Done Sent Done false
21578 6811 Acked Read 8 Done Sent Done false
21579 6812 Acked Read 9 Done Sent Done false
21580 6813 Acked Flush 0 Done Sent Done false
21581 6814 Acked Write 2 Done Sent Done false
21582 6815 Acked Flush 0 Done Sent Done false
21583 6816 Acked Write 4 Done Sent Done false
21584 6817 Acked Write 7 Done Sent Done false
21585 6818 Acked Read 8 Done Sent Done false
21586 6819 Acked Write 2 Done Sent Done false
21587 6820 Acked Read 2 Done Sent Done false
21588 6821 Acked Write 7 Done Sent Done false
21589 6822 Acked Write 10 Done Sent Done false
21590 6823 Acked Write 5 Done Sent Done false
21591 6824 Acked Read 10 Done Sent Done false
21592 6825 Acked Write 5 Done Sent Done false
21593 6826 Acked Write 3 Done Sent Done false
21594 6827 Acked Read 1 Done Sent Done false
21595 6828 Acked Flush 0 Done Sent Done false
21596 6829 Acked Read 3 Done Sent Done false
21597 6830 Acked Read 3 Done Sent Done false
21598 6831 Acked Write 6 Done Sent Done false
21599 6832 Acked Read 1 Done Sent Done false
21600 6833 Acked Read 7 Done Sent Done false
21601 6834 Acked Read 10 Done Sent Done false
21602 6835 Acked Write 7 Done Sent Done false
21603 6836 Acked Write 3 Done Sent Done false
21604 6837 Acked Write 3 Done Sent Done false
21605 6838 Acked Read 8 Done Sent Done false
21606 6839 Acked Write 4 Done Sent Done false
21607 6840 Acked Read 7 Done Sent Done false
21608 6841 Acked Write 5 Done Sent Done false
21609 6842 Acked Write 10 Done Sent Done false
21610 6843 Acked Read 3 Done Sent Done false
21611 6844 Acked Read 4 Done Sent Done false
21612 6845 Acked Write 2 Done Sent Done false
21613 6846 Acked Read 4 Done Sent Done false
21614 6847 Acked Read 5 Done Sent Done false
21615 6848 Acked Flush 0 Done Sent Done false
21616 6849 Acked Write 5 Done Sent Done false
21617 6850 Acked Write 1 Done Sent Done false
21618 6851 Acked Read 2 Done Sent Done false
21619 6852 Acked Write 6 Done Sent Done false
21620 6853 Acked Write 3 Done Sent Done false
21621 6854 Acked Write 10 Done Sent Done false
21622 6855 Acked Write 4 Done Sent Done false
21623 6856 Acked Read 6 Done Sent Done false
21624 6857 Acked Read 4 Done Sent Done false
21625 6858 Acked Read 4 Done Sent Done false
21626 6859 Acked Write 3 Done Sent Done false
21627 6860 Acked Read 9 Done Sent Done false
21628 6861 Acked Read 4 Done Sent Done false
21629 6862 Acked Write 8 Done Sent Done false
21630 6863 Acked Write 3 Done Sent Done false
21631 6864 Acked Write 3 Done Sent Done false
21632 6865 Acked Read 2 Done Sent Done false
21633 6866 Acked Write 4 Done Sent Done false
21634 6867 Acked Read 2 Done Sent Done false
21635 6868 Acked Flush 0 Done Sent Done false
21636 6869 Acked Write 10 Done Sent Done false
21637 6870 Acked Write 7 Done Sent Done false
21638 6871 Acked Read 7 Done Sent Done false
21639 6872 Acked Read 9 Done Sent Done false
21640 6873 Acked Flush 0 Done Sent Done false
21641 6874 Acked Write 3 Done Sent Done false
21642 6875 Acked Write 6 Done Sent Done false
21643 6876 Acked Write 1 Done Sent Done false
21644 6877 Acked Write 1 Done Sent Done false
21645 6878 Acked Read 4 Done Sent Done false
21646 6879 Acked Flush 0 Done Sent Done false
21647 6880 Acked Flush 0 Done Sent Done false
21648 6881 Acked Read 2 Done Sent Done false
21649 6882 Acked Read 6 Done Sent Done false
21650 6883 Acked Flush 0 Done Sent Done false
21651 6884 Acked Write 7 Done Sent Done false
21652 6885 Acked Write 5 Done Sent Done false
21653 6886 Acked Read 8 Done Sent Done false
21654 6887 Acked Write 10 Done Sent Done false
21655 6888 Acked Read 8 Done Sent Done false
21656 6889 Acked Write 5 Done Sent Done false
21657 6890 Acked Write 3 Done Sent Done false
21658 6891 Acked Write 3 Done Sent Done false
21659 6892 Acked Read 7 Done Sent Done false
21660 6893 Acked Read 7 Done Sent Done false
21661 6894 Acked Read 3 Done Sent Done false
21662 6895 Acked Write 2 Done Sent Done false
21663 6896 Acked Flush 0 Done Sent Done false
21664 6897 Acked Read 9 Done Sent Done false
21665 6898 Acked Write 2 Done Sent Done false
21666 6899 Acked Write 6 Done Sent Done false
21667 6900 Acked Read 10 Done Sent Done false
21668 6901 Acked Read 4 Done Sent Done false
21669 6902 Acked Write 1 Done Sent Done false
21670 6903 Acked Read 7 Done Sent Done false
21671 6904 Acked Write 4 Done Sent Done false
21672 6905 Acked Read 10 Done Sent Done false
21673 6906 Acked Write 9 Done Sent Done false
21674 6907 Acked Write 9 Done Sent Done false
21675 6908 Acked Read 6 Done Sent Done false
21676 6909 Acked Write 6 Done Sent Done false
21677 6910 Acked Read 9 Done Sent Done false
21678 6911 Acked Flush 0 Done Sent Done false
21679 6912 Acked Write 4 Done Sent Done false
21680 6913 Acked Write 8 Done Sent Done false
21681 6914 Acked Write 5 Done Sent Done false
21682 6915 Acked Write 9 Done Sent Done false
21683 6916 Acked Flush 0 Done Sent Done false
21684 6917 Acked Write 9 Done Sent Done false
21685 6918 Acked Write 5 Done Sent Done false
21686 6919 Acked Read 6 Done Sent Done false
21687 6920 Acked Read 6 Done Sent Done false
21688 6921 Acked Write 2 Done Sent Done false
21689 6922 Acked Write 6 Done Sent Done false
21690 6923 Acked Read 3 Done Sent Done false
21691 6924 Acked Write 5 Done Sent Done false
21692 6925 Acked Write 8 Done Sent Done false
21693 6926 Acked Write 3 Done Sent Done false
21694 6927 Acked Flush 0 Done Sent Done false
21695 6928 Acked Read 4 Done Sent Done false
21696 6929 Acked Write 7 Done Sent Done false
21697 6930 Acked Read 9 Done Sent Done false
21698 6931 Acked Write 2 Done Sent Done false
21699 6932 Acked Read 7 Done Sent Done false
21700 6933 Acked Flush 0 Done Sent Done false
21701 6934 Acked Write 4 Done Sent Done false
21702 6935 Acked Write 10 Done Sent Done false
21703 6936 Acked Read 8 Done Sent Done false
21704 6937 Acked Write 2 Done Sent Done false
21705 6938 Acked Flush 0 Done Sent Done false
21706 6939 Acked Flush 0 Done Sent Done false
21707 6940 Acked Read 6 Done Sent Done false
21708 6941 Acked Write 1 Done Sent Done false
21709 6942 Acked Write 6 Done Sent Done false
21710 6943 Acked Write 3 Done Sent Done false
21711 6944 Acked Write 1 Done Sent Done false
21712 6945 Acked Write 1 Done Sent Done false
21713 6946 Acked Write 3 Done Sent Done false
21714 6947 Acked Read 9 Done Sent Done false
21715 6948 Acked Read 7 Done Sent Done false
21716 6949 Acked Read 3 Done Sent Done false
21717 6950 Acked Write 1 Done Sent Done false
21718 6951 Acked Flush 0 Done Sent Done false
21719 6952 Acked Read 7 Done Sent Done false
21720 6953 Acked Read 8 Done Sent Done false
21721 6954 Acked Flush 0 Done Sent Done false
21722 6955 Acked Read 6 Done Sent Done false
21723 6956 Acked Write 7 Done Sent Done false
21724 6957 Acked Read 3 Done Sent Done false
21725 6958 Acked Write 7 Done Sent Done false
21726 6959 Acked Write 1 Done Sent Done false
21727 6960 Acked Write 10 Done Sent Done false
21728 6961 Acked Write 2 Done Sent Done false
21729 6962 Acked Write 1 Done Sent Done false
21730 6963 Acked Write 1 Done Sent Done false
21731 6964 Acked Write 4 Done Sent Done false
21732 6965 Acked Write 1 Done Sent Done false
21733 6966 Acked Write 2 Done Sent Done false
21734 6967 Acked Write 2 Done Sent Done false
21735 6968 Acked Read 10 Done Sent Done false
21736 6969 Acked Read 8 Done Sent Done false
21737 6970 Acked Flush 0 Done Sent Done false
21738 6971 Acked Write 9 Done Sent Done false
21739 6972 Acked Read 2 Done Sent Done false
21740 6973 Acked Read 1 Done Sent Done false
21741 6974 Acked Read 3 Done Sent Done false
21742 6975 Acked Write 10 Done Sent Done false
21743 6976 Acked Read 4 Done Sent Done false
21744 6977 Acked Write 6 Done Sent Done false
21745 6978 Acked Read 5 Done Sent Done false
21746 6979 Acked Read 2 Done Sent Done false
21747 6980 Acked Write 3 Done Sent Done false
21748 6981 Acked Read 5 Done Sent Done false
21749 6982 Acked Write 8 Done Sent Done false
21750 6983 Acked Write 1 Done Sent Done false
21751 6984 Acked Read 1 Done Sent Done false
21752 6985 Acked Flush 0 Done Sent Done false
21753 6986 Acked Write 9 Done Sent Done false
21754 6987 Acked Read 5 Done Sent Done false
21755 6988 Acked Write 9 Done Sent Done false
21756 6989 Acked Read 2 Done Sent Done false
21757 6990 Acked Read 9 Done Sent Done false
21758 6991 Acked Write 8 Done Sent Done false
21759 6992 Acked Read 6 Done Sent Done false
21760 6993 Acked Flush 0 Done Sent Done false
21761 6994 Acked Flush 0 Done Sent Done false
21762 6995 Acked Write 4 Done Sent Done false
21763 6996 Acked Write 7 Done Sent Done false
21764 6997 Acked Write 8 Done Sent Done false
21765 6998 Acked Flush 0 Done Sent Done false
21766 6999 Acked Read 5 Done Sent Done false
21767 7000 Acked Read 6 Done Sent Done false
21768 7001 Acked Read 7 Done Sent Done false
21769 7002 Acked Write 3 Done Sent Done false
21770 7003 Acked Read 3 Done Sent Done false
21771 7004 Acked Write 8 Done Sent Done false
21772 7005 Acked Write 9 Done Sent Done false
21773 7006 Acked Write 10 Done Sent Done false
21774 7007 Acked Flush 0 Done Sent Done false
21775 7008 Acked Read 8 Done Sent Done false
21776 7009 Acked Write 3 Done Sent Done false
21777 7010 Acked Read 3 Done Sent Done false
21778 7011 Acked Read 7 Done Sent Done false
21779 7012 Acked Write 4 Done Sent Done false
21780 7013 Acked Write 6 Done Sent Done false
21781 7014 Acked Write 4 Done Sent Done false
21782 7015 Acked Read 3 Done Sent Done false
21783 7016 Acked Read 7 Done Sent Done false
21784 7017 Acked Write 3 Done Sent Done false
21785 7018 Acked Read 8 Done Sent Done false
21786 7019 Acked Read 3 Done Sent Done false
21787 7020 Acked Read 6 Done Sent Done false
21788 7021 Acked Write 5 Done Sent Done false
21789 7022 Acked Write 3 Done Sent Done false
21790 7023 Acked Write 9 Done Sent Done false
21791 7024 Acked Write 4 Done Sent Done false
21792 7025 Acked Write 5 Done Sent Done false
21793 7026 Acked Write 1 Done Sent Done false
21794 7027 Acked Read 10 Done Sent Done false
21795 7028 Acked Read 10 Done Sent Done false
21796 7029 Acked Write 1 Done Sent Done false
21797 7030 Acked Read 1 Done Sent Done false
21798 7031 Acked Write 9 Done Sent Done false
21799 7032 Acked Read 9 Done Sent Done false
21800 7033 Acked Write 3 Done Sent Done false
21801 7034 Acked Read 9 Done Sent Done false
21802 7035 Acked Read 6 Done Sent Done false
21803 7036 Acked Read 4 Done Sent Done false
21804 7037 Acked Write 7 Done Sent Done false
21805 STATES DS:0 DS:1 DS:2 TOTAL
21806 Sent 0 301 0 301
21807 Done 301 0 301 602
21808 Skipped 0 0 0 0
21809 Error 0 0 0 0
21810 Last Flush: 7007 6736 7007
21811 Downstairs last five completed: 6736 6735 6734 6733 6732
21812 Upstairs last five completed: 7037 7036 7035 7034 7033
21813 CLIENT: Up:0 ds:301 act:2
21814 ----------------------------------------------------------------
21815 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21816 States: Active Offline Active
21817 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21818 6737 Acked Write 5 Done Sent Done false
21819 6738 Acked Write 8 Done Sent Done false
21820 6739 Acked Write 1 Done Sent Done false
21821 6740 Acked Flush 0 Done Sent Done false
21822 6741 Acked Write 4 Done Sent Done false
21823 6742 Acked Flush 0 Done Sent Done false
21824 6743 Acked Read 9 Done Sent Done false
21825 6744 Acked Read 7 Done Sent Done false
21826 6745 Acked Read 2 Done Sent Done false
21827 6746 Acked Read 7 Done Sent Done false
21828 6747 Acked Write 3 Done Sent Done false
21829 6748 Acked Write 2 Done Sent Done false
21830 6749 Acked Write 10 Done Sent Done false
21831 6750 Acked Write 8 Done Sent Done false
21832 6751 Acked Write 8 Done Sent Done false
21833 6752 Acked Read 2 Done Sent Done false
21834 6753 Acked Write 7 Done Sent Done false
21835 6754 Acked Write 7 Done Sent Done false
21836 6755 Acked Read 6 Done Sent Done false
21837 6756 Acked Write 10 Done Sent Done false
21838 6757 Acked Write 5 Done Sent Done false
21839 6758 Acked Read 7 Done Sent Done false
21840 6759 Acked Read 1 Done Sent Done false
21841 6760 Acked Read 2 Done Sent Done false
21842 6761 Acked Write 2 Done Sent Done false
21843 6762 Acked Read 8 Done Sent Done false
21844 6763 Acked Read 7 Done Sent Done false
21845 6764 Acked Read 3 Done Sent Done false
21846 6765 Acked Read 6 Done Sent Done false
21847 6766 Acked Read 7 Done Sent Done false
21848 6767 Acked Read 8 Done Sent Done false
21849 6768 Acked Flush 0 Done Sent Done false
21850 6769 Acked Write 5 Done Sent Done false
21851 6770 Acked Write 9 Done Sent Done false
21852 6771 Acked Read 6 Done Sent Done false
21853 6772 Acked Read 8 Done Sent Done false
21854 6773 Acked Write 4 Done Sent Done false
21855 6774 Acked Flush 0 Done Sent Done false
21856 6775 Acked Read 5 Done Sent Done false
21857 6776 Acked Write 10 Done Sent Done false
21858 6777 Acked Read 7 Done Sent Done false
21859 6778 Acked Read 9 Done Sent Done false
21860 6779 Acked Read 7 Done Sent Done false
21861 6780 Acked Write 7 Done Sent Done false
21862 6781 Acked Write 6 Done Sent Done false
21863 6782 Acked Read 6 Done Sent Done false
21864 6783 Acked Read 4 Done Sent Done false
21865 6784 Acked Write 9 Done Sent Done false
21866 6785 Acked Write 10 Done Sent Done false
21867 6786 Acked Flush 0 Done Sent Done false
21868 6787 Acked Read 4 Done Sent Done false
21869 6788 Acked Write 2 Done Sent Done false
21870 6789 Acked Write 3 Done Sent Done false
21871 6790 Acked Write 10 Done Sent Done false
21872 6791 Acked Read 4 Done Sent Done false
21873 6792 Acked Read 2 Done Sent Done false
21874 6793 Acked Read 2 Done Sent Done false
21875 6794 Acked Write 5 Done Sent Done false
21876 6795 Acked Read 3 Done Sent Done false
21877 6796 Acked Write 2 Done Sent Done false
21878 6797 Acked Read 1 Done Sent Done false
21879 6798 Acked Flush 0 Done Sent Done false
21880 6799 Acked Read 9 Done Sent Done false
21881 6800 Acked Read 8 Done Sent Done false
21882 6801 Acked Write 7 Done Sent Done false
21883 6802 Acked Read 3 Done Sent Done false
21884 6803 Acked Write 5 Done Sent Done false
21885 6804 Acked Read 9 Done Sent Done false
21886 6805 Acked Flush 0 Done Sent Done false
21887 6806 Acked Read 6 Done Sent Done false
21888 6807 Acked Read 9 Done Sent Done false
21889 6808 Acked Read 1 Done Sent Done false
21890 6809 Acked Flush 0 Done Sent Done false
21891 6810 Acked Flush 0 Done Sent Done false
21892 6811 Acked Read 8 Done Sent Done false
21893 6812 Acked Read 9 Done Sent Done false
21894 6813 Acked Flush 0 Done Sent Done false
21895 6814 Acked Write 2 Done Sent Done false
21896 6815 Acked Flush 0 Done Sent Done false
21897 6816 Acked Write 4 Done Sent Done false
21898 6817 Acked Write 7 Done Sent Done false
21899 6818 Acked Read 8 Done Sent Done false
21900 6819 Acked Write 2 Done Sent Done false
21901 6820 Acked Read 2 Done Sent Done false
21902 6821 Acked Write 7 Done Sent Done false
21903 6822 Acked Write 10 Done Sent Done false
21904 6823 Acked Write 5 Done Sent Done false
21905 6824 Acked Read 10 Done Sent Done false
21906 6825 Acked Write 5 Done Sent Done false
21907 6826 Acked Write 3 Done Sent Done false
21908 6827 Acked Read 1 Done Sent Done false
21909 6828 Acked Flush 0 Done Sent Done false
21910 6829 Acked Read 3 Done Sent Done false
21911 6830 Acked Read 3 Done Sent Done false
21912 6831 Acked Write 6 Done Sent Done false
21913 6832 Acked Read 1 Done Sent Done false
21914 6833 Acked Read 7 Done Sent Done false
21915 6834 Acked Read 10 Done Sent Done false
21916 6835 Acked Write 7 Done Sent Done false
21917 6836 Acked Write 3 Done Sent Done false
21918 6837 Acked Write 3 Done Sent Done false
21919 6838 Acked Read 8 Done Sent Done false
21920 6839 Acked Write 4 Done Sent Done false
21921 6840 Acked Read 7 Done Sent Done false
21922 6841 Acked Write 5 Done Sent Done false
21923 6842 Acked Write 10 Done Sent Done false
21924 6843 Acked Read 3 Done Sent Done false
21925 6844 Acked Read 4 Done Sent Done false
21926 6845 Acked Write 2 Done Sent Done false
21927 6846 Acked Read 4 Done Sent Done false
21928 6847 Acked Read 5 Done Sent Done false
21929 6848 Acked Flush 0 Done Sent Done false
21930 6849 Acked Write 5 Done Sent Done false
21931 6850 Acked Write 1 Done Sent Done false
21932 6851 Acked Read 2 Done Sent Done false
21933 6852 Acked Write 6 Done Sent Done false
21934 6853 Acked Write 3 Done Sent Done false
21935 6854 Acked Write 10 Done Sent Done false
21936 6855 Acked Write 4 Done Sent Done false
21937 6856 Acked Read 6 Done Sent Done false
21938 6857 Acked Read 4 Done Sent Done false
21939 6858 Acked Read 4 Done Sent Done false
21940 6859 Acked Write 3 Done Sent Done false
21941 6860 Acked Read 9 Done Sent Done false
21942 6861 Acked Read 4 Done Sent Done false
21943 6862 Acked Write 8 Done Sent Done false
21944 6863 Acked Write 3 Done Sent Done false
21945 6864 Acked Write 3 Done Sent Done false
21946 6865 Acked Read 2 Done Sent Done false
21947 6866 Acked Write 4 Done Sent Done false
21948 6867 Acked Read 2 Done Sent Done false
21949 6868 Acked Flush 0 Done Sent Done false
21950 6869 Acked Write 10 Done Sent Done false
21951 6870 Acked Write 7 Done Sent Done false
21952 6871 Acked Read 7 Done Sent Done false
21953 6872 Acked Read 9 Done Sent Done false
21954 6873 Acked Flush 0 Done Sent Done false
21955 6874 Acked Write 3 Done Sent Done false
21956 6875 Acked Write 6 Done Sent Done false
21957 6876 Acked Write 1 Done Sent Done false
21958 6877 Acked Write 1 Done Sent Done false
21959 6878 Acked Read 4 Done Sent Done false
21960 6879 Acked Flush 0 Done Sent Done false
21961 6880 Acked Flush 0 Done Sent Done false
21962 6881 Acked Read 2 Done Sent Done false
21963 6882 Acked Read 6 Done Sent Done false
21964 6883 Acked Flush 0 Done Sent Done false
21965 6884 Acked Write 7 Done Sent Done false
21966 6885 Acked Write 5 Done Sent Done false
21967 6886 Acked Read 8 Done Sent Done false
21968 6887 Acked Write 10 Done Sent Done false
21969 6888 Acked Read 8 Done Sent Done false
21970 6889 Acked Write 5 Done Sent Done false
21971 6890 Acked Write 3 Done Sent Done false
21972 6891 Acked Write 3 Done Sent Done false
21973 6892 Acked Read 7 Done Sent Done false
21974 6893 Acked Read 7 Done Sent Done false
21975 6894 Acked Read 3 Done Sent Done false
21976 6895 Acked Write 2 Done Sent Done false
21977 6896 Acked Flush 0 Done Sent Done false
21978 6897 Acked Read 9 Done Sent Done false
21979 6898 Acked Write 2 Done Sent Done false
21980 6899 Acked Write 6 Done Sent Done false
21981 6900 Acked Read 10 Done Sent Done false
21982 6901 Acked Read 4 Done Sent Done false
21983 6902 Acked Write 1 Done Sent Done false
21984 6903 Acked Read 7 Done Sent Done false
21985 6904 Acked Write 4 Done Sent Done false
21986 6905 Acked Read 10 Done Sent Done false
21987 6906 Acked Write 9 Done Sent Done false
21988 6907 Acked Write 9 Done Sent Done false
21989 6908 Acked Read 6 Done Sent Done false
21990 6909 Acked Write 6 Done Sent Done false
21991 6910 Acked Read 9 Done Sent Done false
21992 6911 Acked Flush 0 Done Sent Done false
21993 6912 Acked Write 4 Done Sent Done false
21994 6913 Acked Write 8 Done Sent Done false
21995 6914 Acked Write 5 Done Sent Done false
21996 6915 Acked Write 9 Done Sent Done false
21997 6916 Acked Flush 0 Done Sent Done false
21998 6917 Acked Write 9 Done Sent Done false
21999 6918 Acked Write 5 Done Sent Done false
22000 6919 Acked Read 6 Done Sent Done false
22001 6920 Acked Read 6 Done Sent Done false
22002 6921 Acked Write 2 Done Sent Done false
22003 6922 Acked Write 6 Done Sent Done false
22004 6923 Acked Read 3 Done Sent Done false
22005 6924 Acked Write 5 Done Sent Done false
22006 6925 Acked Write 8 Done Sent Done false
22007 6926 Acked Write 3 Done Sent Done false
22008 6927 Acked Flush 0 Done Sent Done false
22009 6928 Acked Read 4 Done Sent Done false
22010 6929 Acked Write 7 Done Sent Done false
22011 6930 Acked Read 9 Done Sent Done false
22012 6931 Acked Write 2 Done Sent Done false
22013 6932 Acked Read 7 Done Sent Done false
22014 6933 Acked Flush 0 Done Sent Done false
22015 6934 Acked Write 4 Done Sent Done false
22016 6935 Acked Write 10 Done Sent Done false
22017 6936 Acked Read 8 Done Sent Done false
22018 6937 Acked Write 2 Done Sent Done false
22019 6938 Acked Flush 0 Done Sent Done false
22020 6939 Acked Flush 0 Done Sent Done false
22021 6940 Acked Read 6 Done Sent Done false
22022 6941 Acked Write 1 Done Sent Done false
22023 6942 Acked Write 6 Done Sent Done false
22024 6943 Acked Write 3 Done Sent Done false
22025 6944 Acked Write 1 Done Sent Done false
22026 6945 Acked Write 1 Done Sent Done false
22027 6946 Acked Write 3 Done Sent Done false
22028 6947 Acked Read 9 Done Sent Done false
22029 6948 Acked Read 7 Done Sent Done false
22030 6949 Acked Read 3 Done Sent Done false
22031 6950 Acked Write 1 Done Sent Done false
22032 6951 Acked Flush 0 Done Sent Done false
22033 6952 Acked Read 7 Done Sent Done false
22034 6953 Acked Read 8 Done Sent Done false
22035 6954 Acked Flush 0 Done Sent Done false
22036 6955 Acked Read 6 Done Sent Done false
22037 6956 Acked Write 7 Done Sent Done false
22038 6957 Acked Read 3 Done Sent Done false
22039 6958 Acked Write 7 Done Sent Done false
22040 6959 Acked Write 1 Done Sent Done false
22041 6960 Acked Write 10 Done Sent Done false
22042 6961 Acked Write 2 Done Sent Done false
22043 6962 Acked Write 1 Done Sent Done false
22044 6963 Acked Write 1 Done Sent Done false
22045 6964 Acked Write 4 Done Sent Done false
22046 6965 Acked Write 1 Done Sent Done false
22047 6966 Acked Write 2 Done Sent Done false
22048 6967 Acked Write 2 Done Sent Done false
22049 6968 Acked Read 10 Done Sent Done false
22050 6969 Acked Read 8 Done Sent Done false
22051 6970 Acked Flush 0 Done Sent Done false
22052 6971 Acked Write 9 Done Sent Done false
22053 6972 Acked Read 2 Done Sent Done false
22054 6973 Acked Read 1 Done Sent Done false
22055 6974 Acked Read 3 Done Sent Done false
22056 6975 Acked Write 10 Done Sent Done false
22057 6976 Acked Read 4 Done Sent Done false
22058 6977 Acked Write 6 Done Sent Done false
22059 6978 Acked Read 5 Done Sent Done false
22060 6979 Acked Read 2 Done Sent Done false
22061 6980 Acked Write 3 Done Sent Done false
22062 6981 Acked Read 5 Done Sent Done false
22063 6982 Acked Write 8 Done Sent Done false
22064 6983 Acked Write 1 Done Sent Done false
22065 6984 Acked Read 1 Done Sent Done false
22066 6985 Acked Flush 0 Done Sent Done false
22067 6986 Acked Write 9 Done Sent Done false
22068 6987 Acked Read 5 Done Sent Done false
22069 6988 Acked Write 9 Done Sent Done false
22070 6989 Acked Read 2 Done Sent Done false
22071 6990 Acked Read 9 Done Sent Done false
22072 6991 Acked Write 8 Done Sent Done false
22073 6992 Acked Read 6 Done Sent Done false
22074 6993 Acked Flush 0 Done Sent Done false
22075 6994 Acked Flush 0 Done Sent Done false
22076 6995 Acked Write 4 Done Sent Done false
22077 6996 Acked Write 7 Done Sent Done false
22078 6997 Acked Write 8 Done Sent Done false
22079 6998 Acked Flush 0 Done Sent Done false
22080 6999 Acked Read 5 Done Sent Done false
22081 7000 Acked Read 6 Done Sent Done false
22082 7001 Acked Read 7 Done Sent Done false
22083 7002 Acked Write 3 Done Sent Done false
22084 7003 Acked Read 3 Done Sent Done false
22085 7004 Acked Write 8 Done Sent Done false
22086 7005 Acked Write 9 Done Sent Done false
22087 7006 Acked Write 10 Done Sent Done false
22088 7007 Acked Flush 0 Done Sent Done false
22089 7008 Acked Read 8 Done Sent Done false
22090 7009 Acked Write 3 Done Sent Done false
22091 7010 Acked Read 3 Done Sent Done false
22092 7011 Acked Read 7 Done Sent Done false
22093 7012 Acked Write 4 Done Sent Done false
22094 7013 Acked Write 6 Done Sent Done false
22095 7014 Acked Write 4 Done Sent Done false
22096 7015 Acked Read 3 Done Sent Done false
22097 7016 Acked Read 7 Done Sent Done false
22098 7017 Acked Write 3 Done Sent Done false
22099 7018 Acked Read 8 Done Sent Done false
22100 7019 Acked Read 3 Done Sent Done false
22101 7020 Acked Read 6 Done Sent Done false
22102 7021 Acked Write 5 Done Sent Done false
22103 7022 Acked Write 3 Done Sent Done false
22104 7023 Acked Write 9 Done Sent Done false
22105 7024 Acked Write 4 Done Sent Done false
22106 7025 Acked Write 5 Done Sent Done false
22107 7026 Acked Write 1 Done Sent Done false
22108 7027 Acked Read 10 Done Sent Done false
22109 7028 Acked Read 10 Done Sent Done false
22110 7029 Acked Write 1 Done Sent Done false
22111 7030 Acked Read 1 Done Sent Done false
22112 7031 Acked Write 9 Done Sent Done false
22113 7032 Acked Read 9 Done Sent Done false
22114 7033 Acked Write 3 Done Sent Done false
22115 7034 Acked Read 9 Done Sent Done false
22116 7035 Acked Read 6 Done Sent Done false
22117 7036 Acked Read 4 Done Sent Done false
22118 7037 Acked Write 7 Done Sent Done false
22119 7038 Acked Flush 0 Done Sent Done false
22120 STATES DS:0 DS:1 DS:2 TOTAL
22121 Sent 0 302 0 302
22122 Done 302 0 302 604
22123 Skipped 0 0 0 0
22124 Error 0 0 0 0
22125 Last Flush: 7038 6736 7038
22126 Downstairs last five completed: 6736 6735 6734 6733 6732
22127 Upstairs last five completed: 7038 7037 7036 7035 7034
22128 CLIENT: Up:0 ds:302 act:2
221292025-01-21T15:51:48.056ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
221302025-01-21T15:51:48.056ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
221312025-01-21T15:51:48.056ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
221322025-01-21T15:51:48.056ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
221332025-01-21T15:51:48.056ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
22134 ----------------------------------------------------------------
22135 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22136 States: Active Offline Active
22137 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22138 6737 Acked Write 5 Done Sent Done false
22139 6738 Acked Write 8 Done Sent Done false
22140 6739 Acked Write 1 Done Sent Done false
22141 6740 Acked Flush 0 Done Sent Done false
22142 6741 Acked Write 4 Done Sent Done false
22143 6742 Acked Flush 0 Done Sent Done false
22144 6743 Acked Read 9 Done Sent Done false
22145 6744 Acked Read 7 Done Sent Done false
22146 6745 Acked Read 2 Done Sent Done false
22147 6746 Acked Read 7 Done Sent Done false
22148 6747 Acked Write 3 Done Sent Done false
22149 6748 Acked Write 2 Done Sent Done false
22150 6749 Acked Write 10 Done Sent Done false
22151 6750 Acked Write 8 Done Sent Done false
22152 6751 Acked Write 8 Done Sent Done false
22153 6752 Acked Read 2 Done Sent Done false
22154 6753 Acked Write 7 Done Sent Done false
22155 6754 Acked Write 7 Done Sent Done false
22156 6755 Acked Read 6 Done Sent Done false
22157 6756 Acked Write 10 Done Sent Done false
22158 6757 Acked Write 5 Done Sent Done false
22159 6758 Acked Read 7 Done Sent Done false
22160 6759 Acked Read 1 Done Sent Done false
22161 6760 Acked Read 2 Done Sent Done false
22162 6761 Acked Write 2 Done Sent Done false
22163 6762 Acked Read 8 Done Sent Done false
22164 6763 Acked Read 7 Done Sent Done false
22165 6764 Acked Read 3 Done Sent Done false
22166 6765 Acked Read 6 Done Sent Done false
22167 6766 Acked Read 7 Done Sent Done false
22168 6767 Acked Read 8 Done Sent Done false
22169 6768 Acked Flush 0 Done Sent Done false
22170 6769 Acked Write 5 Done Sent Done false
22171 6770 Acked Write 9 Done Sent Done false
22172 6771 Acked Read 6 Done Sent Done false
22173 6772 Acked Read 8 Done Sent Done false
22174 6773 Acked Write 4 Done Sent Done false
22175 6774 Acked Flush 0 Done Sent Done false
22176 6775 Acked Read 5 Done Sent Done false
22177 6776 Acked Write 10 Done Sent Done false
22178 6777 Acked Read 7 Done Sent Done false
22179 6778 Acked Read 9 Done Sent Done false
22180 6779 Acked Read 7 Done Sent Done false
22181 6780 Acked Write 7 Done Sent Done false
22182 6781 Acked Write 6 Done Sent Done false
22183 6782 Acked Read 6 Done Sent Done false
22184 6783 Acked Read 4 Done Sent Done false
22185 6784 Acked Write 9 Done Sent Done false
22186 6785 Acked Write 10 Done Sent Done false
22187 6786 Acked Flush 0 Done Sent Done false
22188 6787 Acked Read 4 Done Sent Done false
22189 6788 Acked Write 2 Done Sent Done false
22190 6789 Acked Write 3 Done Sent Done false
22191 6790 Acked Write 10 Done Sent Done false
22192 6791 Acked Read 4 Done Sent Done false
22193 6792 Acked Read 2 Done Sent Done false
22194 6793 Acked Read 2 Done Sent Done false
22195 6794 Acked Write 5 Done Sent Done false
22196 6795 Acked Read 3 Done Sent Done false
22197 6796 Acked Write 2 Done Sent Done false
22198 6797 Acked Read 1 Done Sent Done false
22199 6798 Acked Flush 0 Done Sent Done false
22200 6799 Acked Read 9 Done Sent Done false
22201 6800 Acked Read 8 Done Sent Done false
22202 6801 Acked Write 7 Done Sent Done false
22203 6802 Acked Read 3 Done Sent Done false
22204 6803 Acked Write 5 Done Sent Done false
22205 6804 Acked Read 9 Done Sent Done false
22206 6805 Acked Flush 0 Done Sent Done false
22207 6806 Acked Read 6 Done Sent Done false
22208 6807 Acked Read 9 Done Sent Done false
22209 6808 Acked Read 1 Done Sent Done false
22210 6809 Acked Flush 0 Done Sent Done false
22211 6810 Acked Flush 0 Done Sent Done false
22212 6811 Acked Read 8 Done Sent Done false
22213 6812 Acked Read 9 Done Sent Done false
22214 6813 Acked Flush 0 Done Sent Done false
22215 6814 Acked Write 2 Done Sent Done false
22216 6815 Acked Flush 0 Done Sent Done false
22217 6816 Acked Write 4 Done Sent Done false
22218 6817 Acked Write 7 Done Sent Done false
22219 6818 Acked Read 8 Done Sent Done false
22220 6819 Acked Write 2 Done Sent Done false
22221 6820 Acked Read 2 Done Sent Done false
22222 6821 Acked Write 7 Done Sent Done false
22223 6822 Acked Write 10 Done Sent Done false
22224 6823 Acked Write 5 Done Sent Done false
22225 6824 Acked Read 10 Done Sent Done false
22226 6825 Acked Write 5 Done Sent Done false
22227 6826 Acked Write 3 Done Sent Done false
22228 6827 Acked Read 1 Done Sent Done false
22229 6828 Acked Flush 0 Done Sent Done false
22230 6829 Acked Read 3 Done Sent Done false
22231 6830 Acked Read 3 Done Sent Done false
22232 6831 Acked Write 6 Done Sent Done false
22233 6832 Acked Read 1 Done Sent Done false
22234 6833 Acked Read 7 Done Sent Done false
22235 6834 Acked Read 10 Done Sent Done false
22236 6835 Acked Write 7 Done Sent Done false
22237 6836 Acked Write 3 Done Sent Done false
22238 6837 Acked Write 3 Done Sent Done false
22239 6838 Acked Read 8 Done Sent Done false
22240 6839 Acked Write 4 Done Sent Done false
22241 6840 Acked Read 7 Done Sent Done false
22242 6841 Acked Write 5 Done Sent Done false
22243 6842 Acked Write 10 Done Sent Done false
22244 6843 Acked Read 3 Done Sent Done false
22245 6844 Acked Read 4 Done Sent Done false
22246 6845 Acked Write 2 Done Sent Done false
22247 6846 Acked Read 4 Done Sent Done false
22248 6847 Acked Read 5 Done Sent Done false
22249 6848 Acked Flush 0 Done Sent Done false
22250 6849 Acked Write 5 Done Sent Done false
22251 6850 Acked Write 1 Done Sent Done false
22252 6851 Acked Read 2 Done Sent Done false
22253 6852 Acked Write 6 Done Sent Done false
22254 6853 Acked Write 3 Done Sent Done false
22255 6854 Acked Write 10 Done Sent Done false
22256 6855 Acked Write 4 Done Sent Done false
22257 6856 Acked Read 6 Done Sent Done false
22258 6857 Acked Read 4 Done Sent Done false
22259 6858 Acked Read 4 Done Sent Done false
22260 6859 Acked Write 3 Done Sent Done false
22261 6860 Acked Read 9 Done Sent Done false
22262 6861 Acked Read 4 Done Sent Done false
22263 6862 Acked Write 8 Done Sent Done false
22264 6863 Acked Write 3 Done Sent Done false
22265 6864 Acked Write 3 Done Sent Done false
22266 6865 Acked Read 2 Done Sent Done false
22267 6866 Acked Write 4 Done Sent Done false
22268 6867 Acked Read 2 Done Sent Done false
22269 6868 Acked Flush 0 Done Sent Done false
22270 6869 Acked Write 10 Done Sent Done false
22271 6870 Acked Write 7 Done Sent Done false
22272 6871 Acked Read 7 Done Sent Done false
22273 6872 Acked Read 9 Done Sent Done false
22274 6873 Acked Flush 0 Done Sent Done false
22275 6874 Acked Write 3 Done Sent Done false
22276 6875 Acked Write 6 Done Sent Done false
22277 6876 Acked Write 1 Done Sent Done false
22278 6877 Acked Write 1 Done Sent Done false
22279 6878 Acked Read 4 Done Sent Done false
22280 6879 Acked Flush 0 Done Sent Done false
22281 6880 Acked Flush 0 Done Sent Done false
22282 6881 Acked Read 2 Done Sent Done false
22283 6882 Acked Read 6 Done Sent Done false
22284 6883 Acked Flush 0 Done Sent Done false
22285 6884 Acked Write 7 Done Sent Done false
22286 6885 Acked Write 5 Done Sent Done false
22287 6886 Acked Read 8 Done Sent Done false
22288 6887 Acked Write 10 Done Sent Done false
22289 6888 Acked Read 8 Done Sent Done false
22290 6889 Acked Write 5 Done Sent Done false
22291 6890 Acked Write 3 Done Sent Done false
22292 6891 Acked Write 3 Done Sent Done false
22293 6892 Acked Read 7 Done Sent Done false
22294 6893 Acked Read 7 Done Sent Done false
22295 6894 Acked Read 3 Done Sent Done false
22296 6895 Acked Write 2 Done Sent Done false
22297 6896 Acked Flush 0 Done Sent Done false
22298 6897 Acked Read 9 Done Sent Done false
22299 6898 Acked Write 2 Done Sent Done false
22300 6899 Acked Write 6 Done Sent Done false
22301 6900 Acked Read 10 Done Sent Done false
22302 6901 Acked Read 4 Done Sent Done false
22303 6902 Acked Write 1 Done Sent Done false
22304 6903 Acked Read 7 Done Sent Done false
22305 6904 Acked Write 4 Done Sent Done false
22306 6905 Acked Read 10 Done Sent Done false
22307 6906 Acked Write 9 Done Sent Done false
22308 6907 Acked Write 9 Done Sent Done false
22309 6908 Acked Read 6 Done Sent Done false
22310 6909 Acked Write 6 Done Sent Done false
22311 6910 Acked Read 9 Done Sent Done false
22312 6911 Acked Flush 0 Done Sent Done false
22313 6912 Acked Write 4 Done Sent Done false
22314 6913 Acked Write 8 Done Sent Done false
22315 6914 Acked Write 5 Done Sent Done false
22316 6915 Acked Write 9 Done Sent Done false
22317 6916 Acked Flush 0 Done Sent Done false
22318 6917 Acked Write 9 Done Sent Done false
22319 6918 Acked Write 5 Done Sent Done false
22320 6919 Acked Read 6 Done Sent Done false
22321 6920 Acked Read 6 Done Sent Done false
22322 6921 Acked Write 2 Done Sent Done false
22323 6922 Acked Write 6 Done Sent Done false
22324 6923 Acked Read 3 Done Sent Done false
22325 6924 Acked Write 5 Done Sent Done false
22326 6925 Acked Write 8 Done Sent Done false
22327 6926 Acked Write 3 Done Sent Done false
22328 6927 Acked Flush 0 Done Sent Done false
22329 6928 Acked Read 4 Done Sent Done false
22330 6929 Acked Write 7 Done Sent Done false
22331 6930 Acked Read 9 Done Sent Done false
22332 6931 Acked Write 2 Done Sent Done false
22333 6932 Acked Read 7 Done Sent Done false
22334 6933 Acked Flush 0 Done Sent Done false
22335 6934 Acked Write 4 Done Sent Done false
22336 6935 Acked Write 10 Done Sent Done false
22337 6936 Acked Read 8 Done Sent Done false
22338 6937 Acked Write 2 Done Sent Done false
22339 6938 Acked Flush 0 Done Sent Done false
22340 6939 Acked Flush 0 Done Sent Done false
22341 6940 Acked Read 6 Done Sent Done false
22342 6941 Acked Write 1 Done Sent Done false
22343 6942 Acked Write 6 Done Sent Done false
22344 6943 Acked Write 3 Done Sent Done false
22345 6944 Acked Write 1 Done Sent Done false
22346 6945 Acked Write 1 Done Sent Done false
22347 6946 Acked Write 3 Done Sent Done false
22348 6947 Acked Read 9 Done Sent Done false
22349 6948 Acked Read 7 Done Sent Done false
22350 6949 Acked Read 3 Done Sent Done false
22351 6950 Acked Write 1 Done Sent Done false
22352 6951 Acked Flush 0 Done Sent Done false
22353 6952 Acked Read 7 Done Sent Done false
22354 6953 Acked Read 8 Done Sent Done false
22355 6954 Acked Flush 0 Done Sent Done false
22356 6955 Acked Read 6 Done Sent Done false
22357 6956 Acked Write 7 Done Sent Done false
22358 6957 Acked Read 3 Done Sent Done false
22359 6958 Acked Write 7 Done Sent Done false
22360 6959 Acked Write 1 Done Sent Done false
22361 6960 Acked Write 10 Done Sent Done false
22362 6961 Acked Write 2 Done Sent Done false
22363 6962 Acked Write 1 Done Sent Done false
22364 6963 Acked Write 1 Done Sent Done false
22365 6964 Acked Write 4 Done Sent Done false
22366 6965 Acked Write 1 Done Sent Done false
22367 6966 Acked Write 2 Done Sent Done false
22368 6967 Acked Write 2 Done Sent Done false
22369 6968 Acked Read 10 Done Sent Done false
22370 6969 Acked Read 8 Done Sent Done false
22371 6970 Acked Flush 0 Done Sent Done false
22372 6971 Acked Write 9 Done Sent Done false
22373 6972 Acked Read 2 Done Sent Done false
22374 6973 Acked Read 1 Done Sent Done false
22375 6974 Acked Read 3 Done Sent Done false
22376 6975 Acked Write 10 Done Sent Done false
22377 6976 Acked Read 4 Done Sent Done false
22378 6977 Acked Write 6 Done Sent Done false
22379 6978 Acked Read 5 Done Sent Done false
22380 6979 Acked Read 2 Done Sent Done false
22381 6980 Acked Write 3 Done Sent Done false
22382 6981 Acked Read 5 Done Sent Done false
22383 6982 Acked Write 8 Done Sent Done false
22384 6983 Acked Write 1 Done Sent Done false
22385 6984 Acked Read 1 Done Sent Done false
22386 6985 Acked Flush 0 Done Sent Done false
22387 6986 Acked Write 9 Done Sent Done false
22388 6987 Acked Read 5 Done Sent Done false
22389 6988 Acked Write 9 Done Sent Done false
22390 6989 Acked Read 2 Done Sent Done false
22391 6990 Acked Read 9 Done Sent Done false
22392 6991 Acked Write 8 Done Sent Done false
22393 6992 Acked Read 6 Done Sent Done false
22394 6993 Acked Flush 0 Done Sent Done false
22395 6994 Acked Flush 0 Done Sent Done false
22396 6995 Acked Write 4 Done Sent Done false
22397 6996 Acked Write 7 Done Sent Done false
22398 6997 Acked Write 8 Done Sent Done false
22399 6998 Acked Flush 0 Done Sent Done false
22400 6999 Acked Read 5 Done Sent Done false
22401 7000 Acked Read 6 Done Sent Done false
22402 7001 Acked Read 7 Done Sent Done false
22403 7002 Acked Write 3 Done Sent Done false
22404 7003 Acked Read 3 Done Sent Done false
22405 7004 Acked Write 8 Done Sent Done false
22406 7005 Acked Write 9 Done Sent Done false
22407 7006 Acked Write 10 Done Sent Done false
22408 7007 Acked Flush 0 Done Sent Done false
22409 7008 Acked Read 8 Done Sent Done false
22410 7009 Acked Write 3 Done Sent Done false
22411 7010 Acked Read 3 Done Sent Done false
22412 7011 Acked Read 7 Done Sent Done false
22413 7012 Acked Write 4 Done Sent Done false
22414 7013 Acked Write 6 Done Sent Done false
22415 7014 Acked Write 4 Done Sent Done false
22416 7015 Acked Read 3 Done Sent Done false
22417 7016 Acked Read 7 Done Sent Done false
22418 7017 Acked Write 3 Done Sent Done false
22419 7018 Acked Read 8 Done Sent Done false
22420 7019 Acked Read 3 Done Sent Done false
22421 7020 Acked Read 6 Done Sent Done false
22422 7021 Acked Write 5 Done Sent Done false
22423 7022 Acked Write 3 Done Sent Done false
22424 7023 Acked Write 9 Done Sent Done false
22425 7024 Acked Write 4 Done Sent Done false
22426 7025 Acked Write 5 Done Sent Done false
22427 7026 Acked Write 1 Done Sent Done false
22428 7027 Acked Read 10 Done Sent Done false
22429 7028 Acked Read 10 Done Sent Done false
22430 7029 Acked Write 1 Done Sent Done false
22431 7030 Acked Read 1 Done Sent Done false
22432 7031 Acked Write 9 Done Sent Done false
22433 7032 Acked Read 9 Done Sent Done false
22434 7033 Acked Write 3 Done Sent Done false
22435 7034 Acked Read 9 Done Sent Done false
22436 7035 Acked Read 6 Done Sent Done false
22437 7036 Acked Read 4 Done Sent Done false
22438 7037 Acked Write 7 Done Sent Done false
22439 7038 Acked Flush 0 Done Sent Done false
22440 STATES DS:0 DS:1 DS:2 TOTAL
22441 Sent 0 302 0 302
22442 Done 302 0 302 604
22443 Skipped 0 0 0 0
22444 Error 0 0 0 0
22445 Last Flush: 7038 6736 7038
22446 Downstairs last five completed: 6736 6735 6734 6733 6732
22447 Upstairs last five completed: 7038 7037 7036 7035 7034
22448 CLIENT: Up:0 ds:302 act:2
22449 ----------------------------------------------------------------
22450 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22451 States: Active Offline Active
22452 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22453 6737 Acked Write 5 Done Sent Done false
22454 6738 Acked Write 8 Done Sent Done false
22455 6739 Acked Write 1 Done Sent Done false
22456 6740 Acked Flush 0 Done Sent Done false
22457 6741 Acked Write 4 Done Sent Done false
22458 6742 Acked Flush 0 Done Sent Done false
22459 6743 Acked Read 9 Done Sent Done false
22460 6744 Acked Read 7 Done Sent Done false
22461 6745 Acked Read 2 Done Sent Done false
22462 6746 Acked Read 7 Done Sent Done false
22463 6747 Acked Write 3 Done Sent Done false
22464 6748 Acked Write 2 Done Sent Done false
22465 6749 Acked Write 10 Done Sent Done false
22466 6750 Acked Write 8 Done Sent Done false
22467 6751 Acked Write 8 Done Sent Done false
22468 6752 Acked Read 2 Done Sent Done false
22469 6753 Acked Write 7 Done Sent Done false
22470 6754 Acked Write 7 Done Sent Done false
22471 6755 Acked Read 6 Done Sent Done false
22472 6756 Acked Write 10 Done Sent Done false
22473 6757 Acked Write 5 Done Sent Done false
22474 6758 Acked Read 7 Done Sent Done false
22475 6759 Acked Read 1 Done Sent Done false
22476 6760 Acked Read 2 Done Sent Done false
22477 6761 Acked Write 2 Done Sent Done false
22478 6762 Acked Read 8 Done Sent Done false
22479 6763 Acked Read 7 Done Sent Done false
22480 6764 Acked Read 3 Done Sent Done false
22481 6765 Acked Read 6 Done Sent Done false
22482 6766 Acked Read 7 Done Sent Done false
22483 6767 Acked Read 8 Done Sent Done false
22484 6768 Acked Flush 0 Done Sent Done false
22485 6769 Acked Write 5 Done Sent Done false
22486 6770 Acked Write 9 Done Sent Done false
22487 6771 Acked Read 6 Done Sent Done false
22488 6772 Acked Read 8 Done Sent Done false
22489 6773 Acked Write 4 Done Sent Done false
22490 6774 Acked Flush 0 Done Sent Done false
22491 6775 Acked Read 5 Done Sent Done false
22492 6776 Acked Write 10 Done Sent Done false
22493 6777 Acked Read 7 Done Sent Done false
22494 6778 Acked Read 9 Done Sent Done false
22495 6779 Acked Read 7 Done Sent Done false
22496 6780 Acked Write 7 Done Sent Done false
22497 6781 Acked Write 6 Done Sent Done false
22498 6782 Acked Read 6 Done Sent Done false
22499 6783 Acked Read 4 Done Sent Done false
22500 6784 Acked Write 9 Done Sent Done false
22501 6785 Acked Write 10 Done Sent Done false
22502 6786 Acked Flush 0 Done Sent Done false
22503 6787 Acked Read 4 Done Sent Done false
22504 6788 Acked Write 2 Done Sent Done false
22505 6789 Acked Write 3 Done Sent Done false
22506 6790 Acked Write 10 Done Sent Done false
22507 6791 Acked Read 4 Done Sent Done false
22508 6792 Acked Read 2 Done Sent Done false
22509 6793 Acked Read 2 Done Sent Done false
22510 6794 Acked Write 5 Done Sent Done false
22511 6795 Acked Read 3 Done Sent Done false
22512 6796 Acked Write 2 Done Sent Done false
22513 6797 Acked Read 1 Done Sent Done false
22514 6798 Acked Flush 0 Done Sent Done false
22515 6799 Acked Read 9 Done Sent Done false
22516 6800 Acked Read 8 Done Sent Done false
22517 6801 Acked Write 7 Done Sent Done false
22518 6802 Acked Read 3 Done Sent Done false
22519 6803 Acked Write 5 Done Sent Done false
22520 6804 Acked Read 9 Done Sent Done false
22521 6805 Acked Flush 0 Done Sent Done false
22522 6806 Acked Read 6 Done Sent Done false
22523 6807 Acked Read 9 Done Sent Done false
22524 6808 Acked Read 1 Done Sent Done false
22525 6809 Acked Flush 0 Done Sent Done false
22526 6810 Acked Flush 0 Done Sent Done false
22527 6811 Acked Read 8 Done Sent Done false
22528 6812 Acked Read 9 Done Sent Done false
22529 6813 Acked Flush 0 Done Sent Done false
22530 6814 Acked Write 2 Done Sent Done false
22531 6815 Acked Flush 0 Done Sent Done false
22532 6816 Acked Write 4 Done Sent Done false
22533 6817 Acked Write 7 Done Sent Done false
22534 6818 Acked Read 8 Done Sent Done false
22535 6819 Acked Write 2 Done Sent Done false
22536 6820 Acked Read 2 Done Sent Done false
22537 6821 Acked Write 7 Done Sent Done false
22538 6822 Acked Write 10 Done Sent Done false
22539 6823 Acked Write 5 Done Sent Done false
22540 6824 Acked Read 10 Done Sent Done false
22541 6825 Acked Write 5 Done Sent Done false
22542 6826 Acked Write 3 Done Sent Done false
22543 6827 Acked Read 1 Done Sent Done false
22544 6828 Acked Flush 0 Done Sent Done false
22545 6829 Acked Read 3 Done Sent Done false
22546 6830 Acked Read 3 Done Sent Done false
22547 6831 Acked Write 6 Done Sent Done false
22548 6832 Acked Read 1 Done Sent Done false
22549 6833 Acked Read 7 Done Sent Done false
22550 6834 Acked Read 10 Done Sent Done false
22551 6835 Acked Write 7 Done Sent Done false
22552 6836 Acked Write 3 Done Sent Done false
22553 6837 Acked Write 3 Done Sent Done false
22554 6838 Acked Read 8 Done Sent Done false
22555 6839 Acked Write 4 Done Sent Done false
22556 6840 Acked Read 7 Done Sent Done false
22557 6841 Acked Write 5 Done Sent Done false
22558 6842 Acked Write 10 Done Sent Done false
22559 6843 Acked Read 3 Done Sent Done false
22560 6844 Acked Read 4 Done Sent Done false
22561 6845 Acked Write 2 Done Sent Done false
22562 6846 Acked Read 4 Done Sent Done false
22563 6847 Acked Read 5 Done Sent Done false
22564 6848 Acked Flush 0 Done Sent Done false
22565 6849 Acked Write 5 Done Sent Done false
22566 6850 Acked Write 1 Done Sent Done false
22567 6851 Acked Read 2 Done Sent Done false
22568 6852 Acked Write 6 Done Sent Done false
22569 6853 Acked Write 3 Done Sent Done false
22570 6854 Acked Write 10 Done Sent Done false
22571 6855 Acked Write 4 Done Sent Done false
22572 6856 Acked Read 6 Done Sent Done false
22573 6857 Acked Read 4 Done Sent Done false
22574 6858 Acked Read 4 Done Sent Done false
22575 6859 Acked Write 3 Done Sent Done false
22576 6860 Acked Read 9 Done Sent Done false
22577 6861 Acked Read 4 Done Sent Done false
22578 6862 Acked Write 8 Done Sent Done false
22579 6863 Acked Write 3 Done Sent Done false
22580 6864 Acked Write 3 Done Sent Done false
22581 6865 Acked Read 2 Done Sent Done false
22582 6866 Acked Write 4 Done Sent Done false
22583 6867 Acked Read 2 Done Sent Done false
22584 6868 Acked Flush 0 Done Sent Done false
22585 6869 Acked Write 10 Done Sent Done false
22586 6870 Acked Write 7 Done Sent Done false
22587 6871 Acked Read 7 Done Sent Done false
22588 6872 Acked Read 9 Done Sent Done false
22589 6873 Acked Flush 0 Done Sent Done false
22590 6874 Acked Write 3 Done Sent Done false
22591 6875 Acked Write 6 Done Sent Done false
22592 6876 Acked Write 1 Done Sent Done false
22593 6877 Acked Write 1 Done Sent Done false
22594 6878 Acked Read 4 Done Sent Done false
22595 6879 Acked Flush 0 Done Sent Done false
22596 6880 Acked Flush 0 Done Sent Done false
22597 6881 Acked Read 2 Done Sent Done false
22598 6882 Acked Read 6 Done Sent Done false
22599 6883 Acked Flush 0 Done Sent Done false
22600 6884 Acked Write 7 Done Sent Done false
22601 6885 Acked Write 5 Done Sent Done false
22602 6886 Acked Read 8 Done Sent Done false
22603 6887 Acked Write 10 Done Sent Done false
22604 6888 Acked Read 8 Done Sent Done false
22605 6889 Acked Write 5 Done Sent Done false
22606 6890 Acked Write 3 Done Sent Done false
22607 6891 Acked Write 3 Done Sent Done false
22608 6892 Acked Read 7 Done Sent Done false
22609 6893 Acked Read 7 Done Sent Done false
22610 6894 Acked Read 3 Done Sent Done false
22611 6895 Acked Write 2 Done Sent Done false
22612 6896 Acked Flush 0 Done Sent Done false
22613 6897 Acked Read 9 Done Sent Done false
22614 6898 Acked Write 2 Done Sent Done false
22615 6899 Acked Write 6 Done Sent Done false
22616 6900 Acked Read 10 Done Sent Done false
22617 6901 Acked Read 4 Done Sent Done false
22618 6902 Acked Write 1 Done Sent Done false
22619 6903 Acked Read 7 Done Sent Done false
22620 6904 Acked Write 4 Done Sent Done false
22621 6905 Acked Read 10 Done Sent Done false
22622 6906 Acked Write 9 Done Sent Done false
22623 6907 Acked Write 9 Done Sent Done false
22624 6908 Acked Read 6 Done Sent Done false
22625 6909 Acked Write 6 Done Sent Done false
22626 6910 Acked Read 9 Done Sent Done false
22627 6911 Acked Flush 0 Done Sent Done false
22628 6912 Acked Write 4 Done Sent Done false
22629 6913 Acked Write 8 Done Sent Done false
22630 6914 Acked Write 5 Done Sent Done false
22631 6915 Acked Write 9 Done Sent Done false
22632 6916 Acked Flush 0 Done Sent Done false
22633 6917 Acked Write 9 Done Sent Done false
22634 6918 Acked Write 5 Done Sent Done false
22635 6919 Acked Read 6 Done Sent Done false
22636 6920 Acked Read 6 Done Sent Done false
22637 6921 Acked Write 2 Done Sent Done false
22638 6922 Acked Write 6 Done Sent Done false
22639 6923 Acked Read 3 Done Sent Done false
22640 6924 Acked Write 5 Done Sent Done false
22641 6925 Acked Write 8 Done Sent Done false
22642 6926 Acked Write 3 Done Sent Done false
22643 6927 Acked Flush 0 Done Sent Done false
22644 6928 Acked Read 4 Done Sent Done false
22645 6929 Acked Write 7 Done Sent Done false
22646 6930 Acked Read 9 Done Sent Done false
22647 6931 Acked Write 2 Done Sent Done false
22648 6932 Acked Read 7 Done Sent Done false
22649 6933 Acked Flush 0 Done Sent Done false
22650 6934 Acked Write 4 Done Sent Done false
22651 6935 Acked Write 10 Done Sent Done false
22652 6936 Acked Read 8 Done Sent Done false
22653 6937 Acked Write 2 Done Sent Done false
22654 6938 Acked Flush 0 Done Sent Done false
22655 6939 Acked Flush 0 Done Sent Done false
22656 6940 Acked Read 6 Done Sent Done false
22657 6941 Acked Write 1 Done Sent Done false
22658 6942 Acked Write 6 Done Sent Done false
22659 6943 Acked Write 3 Done Sent Done false
22660 6944 Acked Write 1 Done Sent Done false
22661 6945 Acked Write 1 Done Sent Done false
22662 6946 Acked Write 3 Done Sent Done false
22663 6947 Acked Read 9 Done Sent Done false
22664 6948 Acked Read 7 Done Sent Done false
22665 6949 Acked Read 3 Done Sent Done false
22666 6950 Acked Write 1 Done Sent Done false
22667 6951 Acked Flush 0 Done Sent Done false
22668 6952 Acked Read 7 Done Sent Done false
22669 6953 Acked Read 8 Done Sent Done false
22670 6954 Acked Flush 0 Done Sent Done false
22671 6955 Acked Read 6 Done Sent Done false
22672 6956 Acked Write 7 Done Sent Done false
22673 6957 Acked Read 3 Done Sent Done false
22674 6958 Acked Write 7 Done Sent Done false
22675 6959 Acked Write 1 Done Sent Done false
22676 6960 Acked Write 10 Done Sent Done false
22677 6961 Acked Write 2 Done Sent Done false
22678 6962 Acked Write 1 Done Sent Done false
22679 6963 Acked Write 1 Done Sent Done false
22680 6964 Acked Write 4 Done Sent Done false
22681 6965 Acked Write 1 Done Sent Done false
22682 6966 Acked Write 2 Done Sent Done false
22683 6967 Acked Write 2 Done Sent Done false
22684 6968 Acked Read 10 Done Sent Done false
22685 6969 Acked Read 8 Done Sent Done false
22686 6970 Acked Flush 0 Done Sent Done false
22687 6971 Acked Write 9 Done Sent Done false
22688 6972 Acked Read 2 Done Sent Done false
22689 6973 Acked Read 1 Done Sent Done false
22690 6974 Acked Read 3 Done Sent Done false
22691 6975 Acked Write 10 Done Sent Done false
22692 6976 Acked Read 4 Done Sent Done false
22693 6977 Acked Write 6 Done Sent Done false
22694 6978 Acked Read 5 Done Sent Done false
22695 6979 Acked Read 2 Done Sent Done false
22696 6980 Acked Write 3 Done Sent Done false
22697 6981 Acked Read 5 Done Sent Done false
22698 6982 Acked Write 8 Done Sent Done false
22699 6983 Acked Write 1 Done Sent Done false
22700 6984 Acked Read 1 Done Sent Done false
22701 6985 Acked Flush 0 Done Sent Done false
22702 6986 Acked Write 9 Done Sent Done false
22703 6987 Acked Read 5 Done Sent Done false
22704 6988 Acked Write 9 Done Sent Done false
22705 6989 Acked Read 2 Done Sent Done false
22706 6990 Acked Read 9 Done Sent Done false
22707 6991 Acked Write 8 Done Sent Done false
22708 6992 Acked Read 6 Done Sent Done false
22709 6993 Acked Flush 0 Done Sent Done false
22710 6994 Acked Flush 0 Done Sent Done false
22711 6995 Acked Write 4 Done Sent Done false
22712 6996 Acked Write 7 Done Sent Done false
22713 6997 Acked Write 8 Done Sent Done false
22714 6998 Acked Flush 0 Done Sent Done false
22715 6999 Acked Read 5 Done Sent Done false
22716 7000 Acked Read 6 Done Sent Done false
22717 7001 Acked Read 7 Done Sent Done false
22718 7002 Acked Write 3 Done Sent Done false
22719 7003 Acked Read 3 Done Sent Done false
22720 7004 Acked Write 8 Done Sent Done false
22721 7005 Acked Write 9 Done Sent Done false
22722 7006 Acked Write 10 Done Sent Done false
22723 7007 Acked Flush 0 Done Sent Done false
22724 7008 Acked Read 8 Done Sent Done false
22725 7009 Acked Write 3 Done Sent Done false
22726 7010 Acked Read 3 Done Sent Done false
22727 7011 Acked Read 7 Done Sent Done false
22728 7012 Acked Write 4 Done Sent Done false
22729 7013 Acked Write 6 Done Sent Done false
22730 7014 Acked Write 4 Done Sent Done false
22731 7015 Acked Read 3 Done Sent Done false
22732 7016 Acked Read 7 Done Sent Done false
22733 7017 Acked Write 3 Done Sent Done false
22734 7018 Acked Read 8 Done Sent Done false
22735 7019 Acked Read 3 Done Sent Done false
22736 7020 Acked Read 6 Done Sent Done false
22737 7021 Acked Write 5 Done Sent Done false
22738 7022 Acked Write 3 Done Sent Done false
22739 7023 Acked Write 9 Done Sent Done false
22740 7024 Acked Write 4 Done Sent Done false
22741 7025 Acked Write 5 Done Sent Done false
22742 7026 Acked Write 1 Done Sent Done false
22743 7027 Acked Read 10 Done Sent Done false
22744 7028 Acked Read 10 Done Sent Done false
22745 7029 Acked Write 1 Done Sent Done false
22746 7030 Acked Read 1 Done Sent Done false
22747 7031 Acked Write 9 Done Sent Done false
22748 7032 Acked Read 9 Done Sent Done false
22749 7033 Acked Write 3 Done Sent Done false
22750 7034 Acked Read 9 Done Sent Done false
22751 7035 Acked Read 6 Done Sent Done false
22752 7036 Acked Read 4 Done Sent Done false
22753 7037 Acked Write 7 Done Sent Done false
22754 7038 Acked Flush 0 Done Sent Done false
22755 STATES DS:0 DS:1 DS:2 TOTAL
22756 Sent 0 302 0 302
22757 Done 302 0 302 604
22758 Skipped 0 0 0 0
22759 Error 0 0 0 0
22760 Last Flush: 7038 6736 7038
22761 Downstairs last five completed: 6736 6735 6734 6733 6732
22762 Upstairs last five completed: 7038 7037 7036 7035 7034
22763 CLIENT: Up:0 ds:302 act:2
22764 [8820] got start action
22765 [8820] I am going to restart
22766 Starting downstairs at port 8820
22767 Make output file at "/tmp/dsc/downstairs-8820.txt"
22768 [8820][1] reports Starting
22769 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1220)
22770 [8820][1] reports Running
227712025-01-21T15:51:58.056ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227722025-01-21T15:51:58.057ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54711) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227732025-01-21T15:51:58.058ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227742025-01-21T15:51:58.058ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227752025-01-21T15:51:58.058ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227762025-01-21T15:51:58.058ZINFOcrucible: send last flush ID to this DS: 6736 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227772025-01-21T15:51:58.059ZINFOcrucible: Replied this last flush ID: 6736 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227782025-01-21T15:51:58.059ZINFOcrucible: [1] client re-new 302 jobs since flush 6736 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227792025-01-21T15:51:58.059ZINFOcrucible: [1] Replayed 302 jobs since flush: 6736 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
22780 ----------------------------------------------------------------
22781 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22782 Downstairs last five completed: 7038 7037 7036 7035 7034
22783 Upstairs last five completed: 7038 7037 7036 7035 7034
22784 CLIENT: Up:0 ds:0 act:3
22785 Replay: All jobs finished
22786 Main task has work to do, go find it
22787 got dsc Stop(2)
22788 stop 2
22789 Jan 21 15:51:58.291 INFO request completed[8830] Got stop action so:false kr:false
22790 , latency_us: 354, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 1679b0f9-ead0-4e9b-8f49-fab2f40ac8c0, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
227912025-01-21T15:51:58.291ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
227922025-01-21T15:51:58.292ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
22793 Jan 21 15:51:58.292 INFO request completed{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:51:58.292069368Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
22794 , latency_us: 134, response_code: 200, uri: /state/cid/2, method: GET, req_id: 584041d8-fcd5-45d1-930f-3c1246886c52, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
227952025-01-21T15:51:58.292ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
22796 [8830] Exited with: ExitStatus(unix_wait_status(9))
22797 [8830][2] reports Exit
22798 Jan 21 15:52:02.293 INFO request completed, latency_us: 207, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7440c746-9f29-48d9-8f35-f4c437f30054, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
22799 001/300 Read block 2761 len 40960
22800 002/300 Read block 1390 len 4096
22801 003/300 Read block 122 len 32768
22802 004/300 Read block 4484 len 4096
22803 005/300 Read block 4947 len 4096
22804 006/300 Write block 2129 len 24576 data: 3 3 3 5 6 6
22805 007/300 Write block 2152 len 12288 data: 5 3 3
22806 008/300 Flush
22807 009/300 Read block 3495 len 8192
22808 010/300 Write block 346 len 16384 data: 6 6 5 4
22809 011/300 Read block 1272 len 4096
22810 012/300 Read block 3263 len 20480
22811 013/300 Read block 4237 len 16384
22812 014/300 Read block 2514 len 16384
22813 015/300 Read block 3875 len 32768
22814 016/300 Flush
22815 017/300 Read block 666 len 32768
22816 018/300 Write block 3540 len 40960 data: 5 5 4 5 6 5 5 6 4 4
22817 019/300 Read block 1603 len 20480
22818 020/300 Read block 2078 len 40960
22819 021/300 Read block 2000 len 40960
22820 022/300 Read block 4305 len 28672
22821 023/300 Write block 3812 len 28672 data: 7 7 8 8 8 7 5
22822 024/300 Flush
22823 025/300 Write block 1246 len 4096 data: 4
22824 026/300 Read block 2935 len 8192
22825 027/300 Read block 1026 len 28672
22826 028/300 Read block 224 len 8192
22827 029/300 Write block 1051 len 24576 data: 7 7 7 7 7 8
22828 030/300 Write block 739 len 24576 data: 8 7 5 4 5 5
22829 031/300 Write block 2047 len 32768 data: 6 4 3 3 3 4 4 4
22830 032/300 Read block 2231 len 16384
22831 033/300 Read block 3399 len 8192
22832 034/300 Write block 223 len 32768 data: 3 5 6 6 5 5 5 4
22833 035/300 Read block 824 len 24576
22834 036/300 Read block 2972 len 16384
22835 037/300 Read block 3449 len 8192
22836 038/300 Flush
22837 039/300 Write block 1648 len 8192 data: 3 3
22838 040/300 Flush
22839 041/300 Read block 2424 len 40960
22840 042/300 Read block 426 len 36864
22841 043/300 Read block 240 len 24576
22842 044/300 Read block 694 len 12288
22843 045/300 Read block 752 len 24576
22844 046/300 Read block 3635 len 8192
22845 047/300 Read block 2498 len 12288
22846 048/300 Write block 304 len 36864 data: 6 5 5 3 3 4 4 4 3
22847 049/300 Flush
22848 050/300 Write block 2665 len 36864 data: 4 4 4 5 5 6 5 6 6
22849 051/300 Write block 1582 len 20480 data: 6 6 5 5 4
22850 052/300 Flush
22851 053/300 Write block 913 len 24576 data: 4 6 7 7 7 8
22852 054/300 Flush
22853 055/300 Write block 4131 len 36864 data: 5 7 5 4 4 3 4 6 6
22854 056/300 Flush
22855 057/300 Read block 1355 len 20480
22856 058/300 Read block 1831 len 20480
22857 059/300 Flush
22858 060/300 Read block 545 len 4096
22859 061/300 Write block 466 len 16384 data: 6 8 8 6
22860 062/300 Flush
22861 063/300 Read block 4252 len 16384
22862 064/300 Read block 4190 len 8192
22863 065/300 Write block 3889 len 8192 data: 7 8
22864 066/300 Flush
22865 067/300 Write block 4407 len 24576 data: 4 5 5 6 6 4
22866 068/300 Read block 3043 len 16384
22867 069/300 Read block 277 len 36864
22868 070/300 Flush
22869 071/300 Flush
22870 072/300 Write block 208 len 16384 data: 6 5 5 5
22871 073/300 Read block 3762 len 32768
22872 074/300 Write block 1817 len 40960 data: 5 5 4 4 4 4 3 3 3 2
22873 075/300 Read block 1290 len 40960
22874 076/300 Write block 1656 len 36864 data: 4 4 6 5 6 6 4 5 4
22875 077/300 Write block 139 len 40960 data: 5 4 4 4 4 5 5 4 3 4
22876 078/300 Read block 4735 len 24576
22877 079/300 Read block 131 len 20480
22878 080/300 Read block 722 len 40960
22879 081/300 Write block 1768 len 8192 data: 7 6
22880 082/300 Write block 2796 len 32768 data: 8 8 8 7 6 6 6 4
22881 083/300 Read block 1467 len 8192
22882 084/300 Flush
22883 085/300 Write block 685 len 16384 data: 5 5 6 6
22884 086/300 Write block 1283 len 20480 data: 2 2 2 3 4
22885 087/300 Write block 1402 len 8192 data: 2 2
22886 088/300 Flush
22887 089/300 Write block 761 len 4096 data: 5
22888 090/300 Read block 1741 len 36864
22889 091/300 Write block 3133 len 32768 data: 3 3 4 4 4 4 4 4
22890 092/300 Flush
22891 093/300 Write block 1545 len 4096 data: 6
22892 094/300 Read block 3267 len 12288
22893 095/300 Read block 4727 len 12288
22894 096/300 Read block 2233 len 16384
22895 097/300 Read block 3825 len 20480
22896 098/300 Read block 4533 len 20480
22897 099/300 Read block 2003 len 8192
22898 100/300 Write block 3103 len 20480 data: 4 4 5 5 5
22899 101/300 Read block 2859 len 36864
22900 102/300 Read block 1499 len 4096
22901 103/300 Flush
22902 104/300 Read block 968 len 32768
22903 105/300 Read block 2267 len 32768
22904 106/300 Write block 1253 len 32768 data: 3 5 5 4 4 3 5 5
22905 107/300 Write block 974 len 28672 data: 5 5 3 3 3 3 3
22906 108/300 Read block 2373 len 4096
22907 109/300 Write block 1531 len 40960 data: 4 4 4 3 4 4 4 4 4 6
22908 110/300 Read block 4036 len 4096
22909 111/300 Read block 3470 len 28672
22910 112/300 Write block 4003 len 40960 data: 3 4 4 5 5 4 4 3 2 2
22911 113/300 Write block 2196 len 12288 data: 2 3 5
22912 114/300 Write block 3898 len 12288 data: 3 3 3
22913 115/300 Write block 288 len 28672 data: 5 5 5 4 3 3 3
22914 116/300 Write block 2651 len 16384 data: 5 4 3 3
22915 117/300 Read block 1239 len 8192
22916 118/300 Write block 3390 len 36864 data: 6 6 5 5 4 4 5 5 7
22917 119/300 Write block 3533 len 4096 data: 3
22918 120/300 Write block 1747 len 32768 data: 4 4 5 5 6 7 7 6
22919 121/300 Read block 1157 len 36864
22920 122/300 Read block 4570 len 12288
22921 123/300 Read block 1855 len 36864
22922 124/300 Flush
22923 125/300 Write block 3226 len 32768 data: 5 5 4 4 7 6 6 4
22924 126/300 Read block 2052 len 20480
22925 127/300 Write block 3500 len 24576 data: 4 4 4 5 5 4
22926 128/300 Read block 2326 len 8192
22927 129/300 Write block 3017 len 40960 data: 3 4 4 5 5 5 4 5 6 6
22928 130/300 Write block 3150 len 32768 data: 5 5 6 5 5 4 3 3
22929 131/300 Read block 3842 len 20480
22930 132/300 Read block 555 len 36864
22931 133/300 Write block 1196 len 12288 data: 7 6 6
22932 134/300 Read block 1690 len 16384
22933 135/300 Write block 3900 len 28672 data: 4 2 2 3 3 4 4
22934 136/300 Read block 4441 len 4096
22935 137/300 Read block 2744 len 36864
22936 138/300 Write block 2087 len 20480 data: 3 4 3 3 3
22937 139/300 Read block 4168 len 28672
22938 140/300 Write block 2310 len 12288 data: 3 4 4
22939 141/300 Write block 1242 len 40960 data: 5 5 6 5 5 5 5 4 3 3
22940 142/300 Flush
22941 143/300 Read block 71 len 28672
22942 144/300 Read block 2811 len 20480
22943 145/300 Read block 3814 len 32768
22944 146/300 Write block 924 len 24576 data: 3 2 3 3 4 4
22945 147/300 Read block 4046 len 4096
22946 148/300 Write block 3464 len 24576 data: 5 4 5 4 7 7
22947 149/300 Write block 1908 len 28672 data: 9 9 6 7 7 7 6
22948 150/300 Read block 3594 len 8192
22949 151/300 Write block 2341 len 36864 data: 3 4 3 6 5 6 7 8 7
22950 152/300 Read block 4763 len 16384
22951 153/300 Write block 4084 len 36864 data: 4 4 5 4 5 5 5 5 5
22952 154/300 Read block 321 len 40960
22953 155/300 Read block 907 len 20480
22954 156/300 Read block 4945 len 16384
22955 157/300 Read block 390 len 4096
22956 158/300 Flush
22957 159/300 Write block 119 len 16384 data: 6 4 3 2
22958 160/300 Write block 4670 len 24576 data: 5 7 6 7 6 6
22959 161/300 Read block 1992 len 28672
22960 162/300 Write block 4694 len 8192 data: 4 4
22961 163/300 Write block 2300 len 24576 data: 4 4 4 5 5 6
22962 164/300 Write block 4973 len 4096 data: 5
22963 165/300 Write block 649 len 28672 data: 8 7 7 6 7 6 6
22964 166/300 Flush
22965 167/300 Read block 45 len 8192
22966 168/300 Write block 258 len 24576 data: 3 4 4 5 6 5
22967 169/300 Write block 4796 len 20480 data: 3 4 4 5 6
22968 170/300 Read block 1731 len 24576
22969 171/300 Read block 3732 len 8192
22970 172/300 Write block 1736 len 8192 data: 7 9
22971 173/300 Read block 2301 len 4096
22972 174/300 Flush
22973 175/300 Write block 2646 len 36864 data: 6 7 6 5 5 6 5 4 4
22974 176/300 Write block 836 len 16384 data: 4 5 4 5
22975 177/300 Write block 991 len 40960 data: 4 4 5 5 7 7 6 6 7 7
22976 178/300 Read block 1061 len 12288
22977 179/300 Write block 1867 len 20480 data: 7 5 5 6 6
22978 180/300 Write block 1071 len 24576 data: 8 8 9 7 5 7
22979 181/300 Flush
22980 182/300 Write block 1255 len 24576 data: 6 5 5 4 6 6
22981 183/300 Write block 229 len 28672 data: 6 5 3 2 4 4 4
22982 184/300 Read block 1639 len 12288
22983 185/300 Read block 3570 len 40960
22984 186/300 Write block 2519 len 28672 data: 3 2 3 4 4 4 4
22985 187/300 Write block 3131 len 24576 data: 3 3 4 4 5 5
22986 188/300 Write block 4207 len 8192 data: 5 5
22987 189/300 Write block 614 len 4096 data: 7
22988 190/300 Read block 548 len 12288
22989 191/300 Read block 977 len 12288
22990 192/300 Read block 756 len 24576
22991 193/300 Read block 85 len 32768
22992 194/300 Write block 4326 len 20480 data: 4 3 3 4 4
22993 195/300 Write block 3335 len 12288 data: 3 3 3
22994 196/300 Write block 1243 len 36864 data: 6 7 6 6 6 6 5 4 4
22995 197/300 Write block 1065 len 24576 data: 4 5 5 5 5 7
22996 198/300 Write block 2385 len 12288 data: 3 3 3
22997 199/300 Read block 1870 len 20480
22998 200/300 Read block 1302 len 16384
22999 201/300 Read block 2748 len 24576
23000 202/300 Read block 2216 len 24576
23001 203/300 Write block 1323 len 16384 data: 7 8 10 8
23002 204/300 Read block 2091 len 32768
23003 205/300 Read block 4030 len 16384
23004 206/300 Write block 2119 len 40960 data: 3 3 4 3 3 3 3 3 3 3
23005 207/300 Write block 4770 len 40960 data: 5 4 4 3 4 5 4 4 4 3
23006 208/300 Write block 4833 len 28672 data: 4 4 4 4 5 2 2
23007 209/300 Write block 1843 len 16384 data: 7 8 7 6
23008 210/300 Read block 3663 len 32768
23009 211/300 Read block 1362 len 20480
23010 212/300 Flush
23011 213/300 Write block 239 len 40960 data: 5 5 5 4 4 3 2 3 3 4
23012 214/300 Write block 3121 len 8192 data: 3 2
23013 215/300 Read block 1149 len 12288
23014 216/300 Flush
23015 217/300 Read block 3109 len 12288
23016 218/300 Read block 2261 len 16384
23017 219/300 Read block 3336 len 36864
23018 220/300 Read block 3382 len 4096
23019 221/300 Read block 4976 len 32768
23020 222/300 Flush
23021 223/300 Write block 2557 len 36864 data: 5 4 4 5 5 4 4 3 4
23022 224/300 Read block 969 len 28672
23023 225/300 Write block 3522 len 12288 data: 5 7 7
23024 226/300 Flush
23025 227/300 Read block 482 len 36864
23026 228/300 Read block 4828 len 16384
23027 229/300 Write block 771 len 40960 data: 5 7 7 9 8 7 5 5 4 4
23028 230/300 Flush
23029 231/300 Read block 1625 len 12288
23030 232/300 Flush
23031 233/300 Flush
23032 234/300 Read block 3365 len 4096
23033 235/300 Write block 338 len 20480 data: 4 5 5 4 4
23034 236/300 Read block 3028 len 40960
23035 237/300 Write block 3756 len 16384 data: 2 3 4 5
23036 238/300 Read block 2880 len 8192
23037 239/300 Flush
23038 240/300 Write block 884 len 16384 data: 4 3 3 4
23039 241/300 Write block 1978 len 24576 data: 5 6 6 6 6 8
23040 242/300 Read block 3192 len 16384
23041 243/300 Write block 3441 len 12288 data: 6 6 7
23042 244/300 Write block 2397 len 4096 data: 3
23043 245/300 Write block 3402 len 36864 data: 6 5 6 6 5 5 4 4 4
23044 246/300 Write block 4541 len 12288 data: 8 8 9
23045 247/300 Write block 1939 len 24576 data: 6 5 6 5 3 3
23046 248/300 Read block 2386 len 36864
23047 249/300 Write block 4030 len 16384 data: 5 6 7 7
23048 250/300 Read block 2081 len 32768
23049 251/300 Write block 4502 len 12288 data: 3 3 4
23050 252/300 Read block 425 len 8192
23051 253/300 Write block 3942 len 36864 data: 4 4 5 4 5 5 7 5 5
23052 254/300 Read block 1802 len 24576
23053 255/300 Write block 190 len 40960 data: 6 6 7 7 7 5 5 4 4 4
23054 256/300 Read block 4290 len 20480
23055 257/300 Read block 302 len 16384
23056 258/300 Write block 3513 len 20480 data: 6 6 5 5 5
23057 259/300 Write block 2026 len 36864 data: 5 7 6 6 5 6 6 6 6
23058 260/300 Flush
23059 261/300 Read block 3413 len 12288
23060 262/300 Read block 3013 len 16384
23061 263/300 Read block 819 len 24576
23062 264/300 Write block 3405 len 20480 data: 7 6 6 5 5
23063 265/300 Read block 1292 len 20480
23064 266/300 Write block 3975 len 24576 data: 4 4 4 4 3 3
23065 267/300 Write block 4050 len 24576 data: 3 4 4 5 4 5
23066 268/300 Read block 3553 len 40960
23067 269/300 Write block 267 len 32768 data: 3 3 4 6 4 4 4 4
23068 270/300 Write block 3071 len 12288 data: 6 4 3
23069 271/300 Write block 1353 len 4096 data: 7
23070 272/300 Read block 4719 len 32768
23071 273/300 Read block 3302 len 32768
23072 274/300 Write block 2089 len 32768 data: 4 4 4 3 3 4 7 6
23073 275/300 Flush
23074 276/300 Read block 157 len 20480
23075 277/300 Read block 363 len 28672
23076 278/300 Read block 2860 len 28672
23077 279/300 Flush
23078 280/300 Write block 3768 len 8192 data: 6 5
23079 281/300 Read block 4173 len 36864
23080 282/300 Read block 2049 len 16384
23081 283/300 Write block 628 len 8192 data: 6 6
23082 284/300 Write block 4590 len 36864 data: 4 5 5 5 4 4 4 6 5
23083 285/300 Read block 931 len 8192
23084 286/300 Read block 1186 len 32768
23085 287/300 Write block 45 len 8192 data: 7 6
23086 288/300 Read block 4854 len 16384
23087 289/300 Read block 2704 len 40960
23088 290/300 Read block 1136 len 40960
23089 291/300 Read block 737 len 28672
23090 292/300 Read block 1094 len 36864
23091 293/300 Flush
23092 294/300 Write block 1827 len 20480 data: 2 2 2 3 2
23093 295/300 Read block 4612 len 16384
23094 296/300 Write block 738 len 12288 data: 7 9 8
23095 297/300 Read block 2279 len 32768
23096 298/300 Write block 3206 len 8192 data: 5 4
23097 299/300 Read block 1904 len 12288
23098 300/300 Flush
23099 301/300 Read block 3424 len 16384
23100 Main task has work to do, go find it
23101 got dsc Start(2)
23102 start 2
23103 Jan 21 15:52:02.528 INFO request completed, latency_us: 204, response_code: 204, uri: /start/cid/2, method: POST, req_id: 61134df6-5819-4aed-a4fe-06ae8266750d, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
23104 Replay: started 2, returned:Ok(())
23105 ----------------------------------------------------------------
23106 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23107 States: Active Active Offline
23108 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23109 7039 Acked Read 10 Done Done Sent false
23110 7040 Acked Read 1 Done Done Sent false
23111 7041 Acked Read 8 Done Done Sent false
23112 7042 Acked Read 1 Done Done Sent false
23113 7043 Acked Read 1 Done Done Sent false
23114 7044 Acked Write 6 Done Done Sent false
23115 7045 Acked Write 3 Done Done Sent false
23116 7046 Acked Flush 0 Done Done Sent false
23117 7047 Acked Read 2 Done Done Sent false
23118 7048 Acked Write 4 Done Done Sent false
23119 7049 Acked Read 1 Done Done Sent false
23120 7050 Acked Read 5 Done Done Sent false
23121 7051 Acked Read 4 Done Done Sent false
23122 7052 Acked Read 4 Done Done Sent false
23123 7053 Acked Read 8 Done Done Sent false
23124 7054 Acked Flush 0 Done Done Sent false
23125 7055 Acked Read 8 Done Done Sent false
23126 7056 Acked Write 10 Done Done Sent false
23127 7057 Acked Read 5 Done Done Sent false
23128 7058 Acked Read 10 Done Done Sent false
23129 7059 Acked Read 10 Done Done Sent false
23130 7060 Acked Read 7 Done Done Sent false
23131 7061 Acked Write 7 Done Done Sent false
23132 7062 Acked Flush 0 Done Done Sent false
23133 7063 Acked Write 1 Done Done Sent false
23134 7064 Acked Read 2 Done Done Sent false
23135 7065 Acked Read 7 Done Done Sent false
23136 7066 Acked Read 2 Done Done Sent false
23137 7067 Acked Write 6 Done Done Sent false
23138 7068 Acked Write 6 Done Done Sent false
23139 7069 Acked Write 8 Done Done Sent false
23140 7070 Acked Read 4 Done Done Sent false
23141 7071 Acked Read 2 Done Done Sent false
23142 7072 Acked Write 8 Done Done Sent false
23143 7073 Acked Read 6 Done Done Sent false
23144 7074 Acked Read 4 Done Done Sent false
23145 7075 Acked Read 2 Done Done Sent false
23146 7076 Acked Flush 0 Done Done Sent false
23147 7077 Acked Write 2 Done Done Sent false
23148 7078 Acked Flush 0 Done Done Sent false
23149 7079 Acked Read 10 Done Done Sent false
23150 7080 Acked Read 9 Done Done Sent false
23151 7081 Acked Read 6 Done Done Sent false
23152 7082 Acked Read 3 Done Done Sent false
23153 7083 Acked Read 6 Done Done Sent false
23154 7084 Acked Read 2 Done Done Sent false
23155 7085 Acked Read 3 Done Done Sent false
23156 7086 Acked Write 9 Done Done Sent false
23157 7087 Acked Flush 0 Done Done Sent false
23158 7088 Acked Write 9 Done Done Sent false
23159 7089 Acked Write 5 Done Done Sent false
23160 7090 Acked Flush 0 Done Done Sent false
23161 7091 Acked Write 6 Done Done Sent false
23162 7092 Acked Flush 0 Done Done Sent false
23163 7093 Acked Write 9 Done Done Sent false
23164 7094 Acked Flush 0 Done Done Sent false
23165 7095 Acked Read 5 Done Done Sent false
23166 7096 Acked Read 5 Done Done Sent false
23167 7097 Acked Flush 0 Done Done Sent false
23168 7098 Acked Read 1 Done Done Sent false
23169 7099 Acked Write 4 Done Done Sent false
23170 7100 Acked Flush 0 Done Done Sent false
23171 7101 Acked Read 4 Done Done Sent false
23172 7102 Acked Read 2 Done Done Sent false
23173 7103 Acked Write 2 Done Done Sent false
23174 7104 Acked Flush 0 Done Done Sent false
23175 7105 Acked Write 6 Done Done Sent false
23176 7106 Acked Read 4 Done Done Sent false
23177 7107 Acked Read 9 Done Done Sent false
23178 7108 Acked Flush 0 Done Done Sent false
23179 7109 Acked Flush 0 Done Done Sent false
23180 7110 Acked Write 4 Done Done Sent false
23181 7111 Acked Read 8 Done Done Sent false
23182 7112 Acked Write 10 Done Done Sent false
23183 7113 Acked Read 10 Done Done Sent false
23184 7114 Acked Write 9 Done Done Sent false
23185 7115 Acked Write 10 Done Done Sent false
23186 7116 Acked Read 6 Done Done Sent false
23187 7117 Acked Read 5 Done Done Sent false
23188 7118 Acked Read 10 Done Done Sent false
23189 7119 Acked Write 2 Done Done Sent false
23190 7120 Acked Write 8 Done Done Sent false
23191 7121 Acked Read 2 Done Done Sent false
23192 7122 Acked Flush 0 Done Done Sent false
23193 7123 Acked Write 4 Done Done Sent false
23194 7124 Acked Write 5 Done Done Sent false
23195 7125 Acked Write 2 Done Done Sent false
23196 7126 Acked Flush 0 Done Done Sent false
23197 7127 Acked Write 1 Done Done Sent false
23198 7128 Acked Read 9 Done Done Sent false
23199 7129 Acked Write 8 Done Done Sent false
23200 7130 Acked Flush 0 Done Done Sent false
23201 7131 Acked Write 1 Done Done Sent false
23202 7132 Acked Read 3 Done Done Sent false
23203 7133 Acked Read 3 Done Done Sent false
23204 7134 Acked Read 4 Done Done Sent false
23205 7135 Acked Read 5 Done Done Sent false
23206 7136 Acked Read 5 Done Done Sent false
23207 7137 Acked Read 2 Done Done Sent false
23208 7138 Acked Write 5 Done Done Sent false
23209 7139 Acked Read 9 Done Done Sent false
23210 7140 Acked Read 1 Done Done Sent false
23211 7141 Acked Flush 0 Done Done Sent false
23212 7142 Acked Read 8 Done Done Sent false
23213 7143 Acked Read 8 Done Done Sent false
23214 7144 Acked Write 8 Done Done Sent false
23215 7145 Acked Write 7 Done Done Sent false
23216 7146 Acked Read 1 Done Done Sent false
23217 7147 Acked Write 10 Done Done Sent false
23218 7148 Acked Read 1 Done Done Sent false
23219 7149 Acked Read 7 Done Done Sent false
23220 7150 Acked Write 10 Done Done Sent false
23221 7151 Acked Write 3 Done Done Sent false
23222 7152 Acked Write 3 Done Done Sent false
23223 7153 Acked Write 7 Done Done Sent false
23224 7154 Acked Write 4 Done Done Sent false
23225 7155 Acked Read 2 Done Done Sent false
23226 7156 Acked Write 9 Done Done Sent false
23227 7157 Acked Write 1 Done Done Sent false
23228 7158 Acked Write 8 Done Done Sent false
23229 7159 Acked Read 9 Done Done Sent false
23230 7160 Acked Read 3 Done Done Sent false
23231 7161 Acked Read 9 Done Done Sent false
23232 7162 Acked Flush 0 Done Done Sent false
23233 7163 Acked Write 8 Done Done Sent false
23234 7164 Acked Read 5 Done Done Sent false
23235 7165 Acked Write 6 Done Done Sent false
23236 7166 Acked Read 2 Done Done Sent false
23237 7167 Acked Write 10 Done Done Sent false
23238 7168 Acked Write 8 Done Done Sent false
23239 7169 Acked Read 5 Done Done Sent false
23240 7170 Acked Read 9 Done Done Sent false
23241 7171 Acked Write 3 Done Done Sent false
23242 7172 Acked Read 4 Done Done Sent false
23243 7173 Acked Write 7 Done Done Sent false
23244 7174 Acked Read 1 Done Done Sent false
23245 7175 Acked Read 9 Done Done Sent false
23246 7176 Acked Write 5 Done Done Sent false
23247 7177 Acked Read 7 Done Done Sent false
23248 7178 Acked Write 3 Done Done Sent false
23249 7179 Acked Write 10 Done Done Sent false
23250 7180 Acked Flush 0 Done Done Sent false
23251 7181 Acked Read 7 Done Done Sent false
23252 7182 Acked Read 5 Done Done Sent false
23253 7183 Acked Read 8 Done Done Sent false
23254 7184 Acked Write 6 Done Done Sent false
23255 7185 Acked Read 1 Done Done Sent false
23256 7186 Acked Write 6 Done Done Sent false
23257 7187 Acked Write 7 Done Done Sent false
23258 7188 Acked Read 2 Done Done Sent false
23259 7189 Acked Write 9 Done Done Sent false
23260 7190 Acked Read 4 Done Done Sent false
23261 7191 Acked Write 9 Done Done Sent false
23262 7192 Acked Read 10 Done Done Sent false
23263 7193 Acked Read 5 Done Done Sent false
23264 7194 Acked Read 4 Done Done Sent false
23265 7195 Acked Read 1 Done Done Sent false
23266 7196 Acked Flush 0 Done Done Sent false
23267 7197 Acked Write 4 Done Done Sent false
23268 7198 Acked Write 6 Done Done Sent false
23269 7199 Acked Read 7 Done Done Sent false
23270 7200 Acked Write 2 Done Done Sent false
23271 7201 Acked Write 6 Done Done Sent false
23272 7202 Acked Write 1 Done Done Sent false
23273 7203 Acked Write 7 Done Done Sent false
23274 7204 Acked Flush 0 Done Done Sent false
23275 7205 Acked Read 2 Done Done Sent false
23276 7206 Acked Write 6 Done Done Sent false
23277 7207 Acked Write 5 Done Done Sent false
23278 7208 Acked Read 6 Done Done Sent false
23279 7209 Acked Read 2 Done Done Sent false
23280 7210 Acked Write 2 Done Done Sent false
23281 7211 Acked Read 1 Done Done Sent false
23282 7212 Acked Flush 0 Done Done Sent false
23283 7213 Acked Write 9 Done Done Sent false
23284 7214 Acked Write 4 Done Done Sent false
23285 7215 Acked Write 10 Done Done Sent false
23286 7216 Acked Read 3 Done Done Sent false
23287 7217 Acked Write 5 Done Done Sent false
23288 7218 Acked Write 6 Done Done Sent false
23289 7219 Acked Flush 0 Done Done Sent false
23290 7220 Acked Write 6 Done Done Sent false
23291 7221 Acked Write 7 Done Done Sent false
23292 7222 Acked Read 3 Done Done Sent false
23293 7223 Acked Read 10 Done Done Sent false
23294 7224 Acked Write 7 Done Done Sent false
23295 7225 Acked Write 6 Done Done Sent false
23296 7226 Acked Write 2 Done Done Sent false
23297 7227 Acked Write 1 Done Done Sent false
23298 7228 Acked Read 3 Done Done Sent false
23299 7229 Acked Read 3 Done Done Sent false
23300 7230 Acked Read 6 Done Done Sent false
23301 7231 Acked Read 8 Done Done Sent false
23302 7232 Acked Write 5 Done Done Sent false
23303 7233 Acked Write 3 Done Done Sent false
23304 7234 Acked Write 9 Done Done Sent false
23305 7235 Acked Write 6 Done Done Sent false
23306 7236 Acked Write 3 Done Done Sent false
23307 7237 Acked Read 5 Done Done Sent false
23308 7238 Acked Read 4 Done Done Sent false
23309 7239 Acked Read 6 Done Done Sent false
23310 7240 Acked Read 6 Done Done Sent false
23311 7241 Acked Write 4 Done Done Sent false
23312 7242 Acked Read 8 Done Done Sent false
23313 7243 Acked Read 4 Done Done Sent false
23314 7244 Acked Write 10 Done Done Sent false
23315 7245 Acked Write 10 Done Done Sent false
23316 7246 Acked Write 7 Done Done Sent false
23317 7247 Acked Write 4 Done Done Sent false
23318 7248 Acked Read 8 Done Done Sent false
23319 7249 Acked Read 5 Done Done Sent false
23320 7250 Acked Flush 0 Done Done Sent false
23321 7251 Acked Write 10 Done Done Sent false
23322 7252 Acked Write 2 Done Done Sent false
23323 7253 Acked Read 3 Done Done Sent false
23324 7254 Acked Flush 0 Done Done Sent false
23325 7255 Acked Read 3 Done Done Sent false
23326 7256 Acked Read 4 Done Done Sent false
23327 7257 Acked Read 9 Done Done Sent false
23328 7258 Acked Read 1 Done Done Sent false
23329 7259 Acked Read 8 Done Done Sent false
23330 7260 Acked Flush 0 Done Done Sent false
23331 7261 Acked Write 9 Done Done Sent false
23332 7262 Acked Read 7 Done Done Sent false
23333 7263 Acked Write 3 Done Done Sent false
23334 7264 Acked Flush 0 Done Done Sent false
23335 7265 Acked Read 9 Done Done Sent false
23336 7266 Acked Read 4 Done Done Sent false
23337 7267 Acked Write 10 Done Done Sent false
23338 7268 Acked Flush 0 Done Done Sent false
23339 7269 Acked Read 3 Done Done Sent false
23340 7270 Acked Flush 0 Done Done Sent false
23341 7271 Acked Flush 0 Done Done Sent false
23342 7272 Acked Read 1 Done Done Sent false
23343 7273 Acked Write 5 Done Done Sent false
23344 7274 Acked Read 10 Done Done Sent false
23345 7275 Acked Write 4 Done Done Sent false
23346 7276 Acked Read 2 Done Done Sent false
23347 7277 Acked Flush 0 Done Done Sent false
23348 7278 Acked Write 4 Done Done Sent false
23349 7279 Acked Write 6 Done Done Sent false
23350 7280 Acked Read 4 Done Done Sent false
23351 7281 Acked Write 3 Done Done Sent false
23352 7282 Acked Write 1 Done Done Sent false
23353 7283 Acked Write 9 Done Done Sent false
23354 7284 Acked Write 3 Done Done Sent false
23355 7285 Acked Write 6 Done Done Sent false
23356 7286 Acked Read 9 Done Done Sent false
23357 7287 Acked Write 4 Done Done Sent false
23358 7288 Acked Read 8 Done Done Sent false
23359 7289 Acked Write 3 Done Done Sent false
23360 7290 Acked Read 2 Done Done Sent false
23361 7291 Acked Write 9 Done Done Sent false
23362 7292 Acked Read 6 Done Done Sent false
23363 7293 Acked Write 10 Done Done Sent false
23364 7294 Acked Read 5 Done Done Sent false
23365 7295 Acked Read 4 Done Done Sent false
23366 7296 Acked Write 5 Done Done Sent false
23367 7297 Acked Write 9 Done Done Sent false
23368 7298 Acked Flush 0 Done Done Sent false
23369 7299 Acked Read 3 Done Done Sent false
23370 7300 Acked Read 4 Done Done Sent false
23371 7301 Acked Read 6 Done Done Sent false
23372 7302 Acked Write 5 Done Done Sent false
23373 7303 Acked Read 5 Done Done Sent false
23374 7304 Acked Write 6 Done Done Sent false
23375 7305 Acked Write 6 Done Done Sent false
23376 7306 Acked Read 10 Done Done Sent false
23377 7307 Acked Write 8 Done Done Sent false
23378 7308 Acked Write 3 Done Done Sent false
23379 7309 Acked Write 1 Done Done Sent false
23380 7310 Acked Read 8 Done Done Sent false
23381 7311 Acked Read 8 Done Done Sent false
23382 7312 Acked Write 8 Done Done Sent false
23383 7313 Acked Flush 0 Done Done Sent false
23384 7314 Acked Read 5 Done Done Sent false
23385 7315 Acked Read 7 Done Done Sent false
23386 7316 Acked Read 7 Done Done Sent false
23387 7317 Acked Flush 0 Done Done Sent false
23388 7318 Acked Write 2 Done Done Sent false
23389 7319 Acked Read 9 Done Done Sent false
23390 7320 Acked Read 4 Done Done Sent false
23391 7321 Acked Write 2 Done Done Sent false
23392 7322 Acked Write 9 Done Done Sent false
23393 7323 Acked Read 2 Done Done Sent false
23394 7324 Acked Read 8 Done Done Sent false
23395 7325 Acked Write 2 Done Done Sent false
23396 7326 Acked Read 4 Done Done Sent false
23397 7327 Acked Read 10 Done Done Sent false
23398 7328 Acked Read 10 Done Done Sent false
23399 7329 Acked Read 7 Done Done Sent false
23400 7330 Acked Read 9 Done Done Sent false
23401 7331 Acked Flush 0 Done Done Sent false
23402 7332 Acked Write 5 Done Done Sent false
23403 7333 Acked Read 4 Done Done Sent false
23404 7334 Acked Write 3 Done Done Sent false
23405 7335 Acked Read 8 Done Done Sent false
23406 7336 Acked Write 2 Done Done Sent false
23407 7337 Acked Read 3 Done Done Sent false
23408 7338 Acked Flush 0 Done Done Sent false
23409 7339 Acked Read 4 Done Done Sent false
23410 STATES DS:0 DS:1 DS:2 TOTAL
23411 Sent 0 0 301 301
23412 Done 301 301 0 602
23413 Skipped 0 0 0 0
23414 Error 0 0 0 0
23415 Last Flush: 7338 7338 7038
23416 Downstairs last five completed: 7038 7037 7036 7035 7034
23417 Upstairs last five completed: 7339 7338 7337 7336 7335
23418 CLIENT: Up:0 ds:301 act:2
23419 [8830] got start action
23420 [8830] I am going to restart
23421 Starting downstairs at port 8830
23422 Make output file at "/tmp/dsc/downstairs-8830.txt"
23423 [8830][2] reports Starting
23424 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1221)
23425 [8830][2] reports Running
23426 ----------------------------------------------------------------
23427 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
23428 States: Active Active Offline
23429 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23430 7039 Acked Read 10 Done Done Sent false
23431 7040 Acked Read 1 Done Done Sent false
23432 7041 Acked Read 8 Done Done Sent false
23433 7042 Acked Read 1 Done Done Sent false
23434 7043 Acked Read 1 Done Done Sent false
23435 7044 Acked Write 6 Done Done Sent false
23436 7045 Acked Write 3 Done Done Sent false
23437 7046 Acked Flush 0 Done Done Sent false
23438 7047 Acked Read 2 Done Done Sent false
23439 7048 Acked Write 4 Done Done Sent false
23440 7049 Acked Read 1 Done Done Sent false
23441 7050 Acked Read 5 Done Done Sent false
23442 7051 Acked Read 4 Done Done Sent false
23443 7052 Acked Read 4 Done Done Sent false
23444 7053 Acked Read 8 Done Done Sent false
23445 7054 Acked Flush 0 Done Done Sent false
23446 7055 Acked Read 8 Done Done Sent false
23447 7056 Acked Write 10 Done Done Sent false
23448 7057 Acked Read 5 Done Done Sent false
23449 7058 Acked Read 10 Done Done Sent false
23450 7059 Acked Read 10 Done Done Sent false
23451 7060 Acked Read 7 Done Done Sent false
23452 7061 Acked Write 7 Done Done Sent false
23453 7062 Acked Flush 0 Done Done Sent false
23454 7063 Acked Write 1 Done Done Sent false
23455 7064 Acked Read 2 Done Done Sent false
23456 7065 Acked Read 7 Done Done Sent false
23457 7066 Acked Read 2 Done Done Sent false
23458 7067 Acked Write 6 Done Done Sent false
23459 7068 Acked Write 6 Done Done Sent false
23460 7069 Acked Write 8 Done Done Sent false
23461 7070 Acked Read 4 Done Done Sent false
23462 7071 Acked Read 2 Done Done Sent false
23463 7072 Acked Write 8 Done Done Sent false
23464 7073 Acked Read 6 Done Done Sent false
23465 7074 Acked Read 4 Done Done Sent false
23466 7075 Acked Read 2 Done Done Sent false
23467 7076 Acked Flush 0 Done Done Sent false
23468 7077 Acked Write 2 Done Done Sent false
23469 7078 Acked Flush 0 Done Done Sent false
23470 7079 Acked Read 10 Done Done Sent false
23471 7080 Acked Read 9 Done Done Sent false
23472 7081 Acked Read 6 Done Done Sent false
23473 7082 Acked Read 3 Done Done Sent false
23474 7083 Acked Read 6 Done Done Sent false
23475 7084 Acked Read 2 Done Done Sent false
23476 7085 Acked Read 3 Done Done Sent false
23477 7086 Acked Write 9 Done Done Sent false
23478 7087 Acked Flush 0 Done Done Sent false
23479 7088 Acked Write 9 Done Done Sent false
23480 7089 Acked Write 5 Done Done Sent false
23481 7090 Acked Flush 0 Done Done Sent false
23482 7091 Acked Write 6 Done Done Sent false
23483 7092 Acked Flush 0 Done Done Sent false
23484 7093 Acked Write 9 Done Done Sent false
23485 7094 Acked Flush 0 Done Done Sent false
23486 7095 Acked Read 5 Done Done Sent false
23487 7096 Acked Read 5 Done Done Sent false
23488 7097 Acked Flush 0 Done Done Sent false
23489 7098 Acked Read 1 Done Done Sent false
23490 7099 Acked Write 4 Done Done Sent false
23491 7100 Acked Flush 0 Done Done Sent false
23492 7101 Acked Read 4 Done Done Sent false
23493 7102 Acked Read 2 Done Done Sent false
23494 7103 Acked Write 2 Done Done Sent false
23495 7104 Acked Flush 0 Done Done Sent false
23496 7105 Acked Write 6 Done Done Sent false
23497 7106 Acked Read 4 Done Done Sent false
23498 7107 Acked Read 9 Done Done Sent false
23499 7108 Acked Flush 0 Done Done Sent false
23500 7109 Acked Flush 0 Done Done Sent false
23501 7110 Acked Write 4 Done Done Sent false
23502 7111 Acked Read 8 Done Done Sent false
23503 7112 Acked Write 10 Done Done Sent false
23504 7113 Acked Read 10 Done Done Sent false
23505 7114 Acked Write 9 Done Done Sent false
23506 7115 Acked Write 10 Done Done Sent false
23507 7116 Acked Read 6 Done Done Sent false
23508 7117 Acked Read 5 Done Done Sent false
23509 7118 Acked Read 10 Done Done Sent false
23510 7119 Acked Write 2 Done Done Sent false
23511 7120 Acked Write 8 Done Done Sent false
23512 7121 Acked Read 2 Done Done Sent false
23513 7122 Acked Flush 0 Done Done Sent false
23514 7123 Acked Write 4 Done Done Sent false
23515 7124 Acked Write 5 Done Done Sent false
23516 7125 Acked Write 2 Done Done Sent false
23517 7126 Acked Flush 0 Done Done Sent false
23518 7127 Acked Write 1 Done Done Sent false
23519 7128 Acked Read 9 Done Done Sent false
23520 7129 Acked Write 8 Done Done Sent false
23521 7130 Acked Flush 0 Done Done Sent false
23522 7131 Acked Write 1 Done Done Sent false
23523 7132 Acked Read 3 Done Done Sent false
23524 7133 Acked Read 3 Done Done Sent false
23525 7134 Acked Read 4 Done Done Sent false
23526 7135 Acked Read 5 Done Done Sent false
23527 7136 Acked Read 5 Done Done Sent false
23528 7137 Acked Read 2 Done Done Sent false
23529 7138 Acked Write 5 Done Done Sent false
23530 7139 Acked Read 9 Done Done Sent false
23531 7140 Acked Read 1 Done Done Sent false
23532 7141 Acked Flush 0 Done Done Sent false
23533 7142 Acked Read 8 Done Done Sent false
23534 7143 Acked Read 8 Done Done Sent false
23535 7144 Acked Write 8 Done Done Sent false
23536 7145 Acked Write 7 Done Done Sent false
23537 7146 Acked Read 1 Done Done Sent false
23538 7147 Acked Write 10 Done Done Sent false
23539 7148 Acked Read 1 Done Done Sent false
23540 7149 Acked Read 7 Done Done Sent false
23541 7150 Acked Write 10 Done Done Sent false
23542 7151 Acked Write 3 Done Done Sent false
23543 7152 Acked Write 3 Done Done Sent false
23544 7153 Acked Write 7 Done Done Sent false
23545 7154 Acked Write 4 Done Done Sent false
23546 7155 Acked Read 2 Done Done Sent false
23547 7156 Acked Write 9 Done Done Sent false
23548 7157 Acked Write 1 Done Done Sent false
23549 7158 Acked Write 8 Done Done Sent false
23550 7159 Acked Read 9 Done Done Sent false
23551 7160 Acked Read 3 Done Done Sent false
23552 7161 Acked Read 9 Done Done Sent false
23553 7162 Acked Flush 0 Done Done Sent false
23554 7163 Acked Write 8 Done Done Sent false
23555 7164 Acked Read 5 Done Done Sent false
23556 7165 Acked Write 6 Done Done Sent false
23557 7166 Acked Read 2 Done Done Sent false
23558 7167 Acked Write 10 Done Done Sent false
23559 7168 Acked Write 8 Done Done Sent false
23560 7169 Acked Read 5 Done Done Sent false
23561 7170 Acked Read 9 Done Done Sent false
23562 7171 Acked Write 3 Done Done Sent false
23563 7172 Acked Read 4 Done Done Sent false
23564 7173 Acked Write 7 Done Done Sent false
23565 7174 Acked Read 1 Done Done Sent false
23566 7175 Acked Read 9 Done Done Sent false
23567 7176 Acked Write 5 Done Done Sent false
23568 7177 Acked Read 7 Done Done Sent false
23569 7178 Acked Write 3 Done Done Sent false
23570 7179 Acked Write 10 Done Done Sent false
23571 7180 Acked Flush 0 Done Done Sent false
23572 7181 Acked Read 7 Done Done Sent false
23573 7182 Acked Read 5 Done Done Sent false
23574 7183 Acked Read 8 Done Done Sent false
23575 7184 Acked Write 6 Done Done Sent false
23576 7185 Acked Read 1 Done Done Sent false
23577 7186 Acked Write 6 Done Done Sent false
23578 7187 Acked Write 7 Done Done Sent false
23579 7188 Acked Read 2 Done Done Sent false
23580 7189 Acked Write 9 Done Done Sent false
23581 7190 Acked Read 4 Done Done Sent false
23582 7191 Acked Write 9 Done Done Sent false
23583 7192 Acked Read 10 Done Done Sent false
23584 7193 Acked Read 5 Done Done Sent false
23585 7194 Acked Read 4 Done Done Sent false
23586 7195 Acked Read 1 Done Done Sent false
23587 7196 Acked Flush 0 Done Done Sent false
23588 7197 Acked Write 4 Done Done Sent false
23589 7198 Acked Write 6 Done Done Sent false
23590 7199 Acked Read 7 Done Done Sent false
23591 7200 Acked Write 2 Done Done Sent false
23592 7201 Acked Write 6 Done Done Sent false
23593 7202 Acked Write 1 Done Done Sent false
23594 7203 Acked Write 7 Done Done Sent false
23595 7204 Acked Flush 0 Done Done Sent false
23596 7205 Acked Read 2 Done Done Sent false
23597 7206 Acked Write 6 Done Done Sent false
23598 7207 Acked Write 5 Done Done Sent false
23599 7208 Acked Read 6 Done Done Sent false
23600 7209 Acked Read 2 Done Done Sent false
23601 7210 Acked Write 2 Done Done Sent false
23602 7211 Acked Read 1 Done Done Sent false
23603 7212 Acked Flush 0 Done Done Sent false
23604 7213 Acked Write 9 Done Done Sent false
23605 7214 Acked Write 4 Done Done Sent false
23606 7215 Acked Write 10 Done Done Sent false
23607 7216 Acked Read 3 Done Done Sent false
23608 7217 Acked Write 5 Done Done Sent false
23609 7218 Acked Write 6 Done Done Sent false
23610 7219 Acked Flush 0 Done Done Sent false
23611 7220 Acked Write 6 Done Done Sent false
23612 7221 Acked Write 7 Done Done Sent false
23613 7222 Acked Read 3 Done Done Sent false
23614 7223 Acked Read 10 Done Done Sent false
23615 7224 Acked Write 7 Done Done Sent false
23616 7225 Acked Write 6 Done Done Sent false
23617 7226 Acked Write 2 Done Done Sent false
23618 7227 Acked Write 1 Done Done Sent false
23619 7228 Acked Read 3 Done Done Sent false
23620 7229 Acked Read 3 Done Done Sent false
23621 7230 Acked Read 6 Done Done Sent false
23622 7231 Acked Read 8 Done Done Sent false
23623 7232 Acked Write 5 Done Done Sent false
23624 7233 Acked Write 3 Done Done Sent false
23625 7234 Acked Write 9 Done Done Sent false
23626 7235 Acked Write 6 Done Done Sent false
23627 7236 Acked Write 3 Done Done Sent false
23628 7237 Acked Read 5 Done Done Sent false
23629 7238 Acked Read 4 Done Done Sent false
23630 7239 Acked Read 6 Done Done Sent false
23631 7240 Acked Read 6 Done Done Sent false
23632 7241 Acked Write 4 Done Done Sent false
23633 7242 Acked Read 8 Done Done Sent false
23634 7243 Acked Read 4 Done Done Sent false
23635 7244 Acked Write 10 Done Done Sent false
23636 7245 Acked Write 10 Done Done Sent false
23637 7246 Acked Write 7 Done Done Sent false
23638 7247 Acked Write 4 Done Done Sent false
23639 7248 Acked Read 8 Done Done Sent false
23640 7249 Acked Read 5 Done Done Sent false
23641 7250 Acked Flush 0 Done Done Sent false
23642 7251 Acked Write 10 Done Done Sent false
23643 7252 Acked Write 2 Done Done Sent false
23644 7253 Acked Read 3 Done Done Sent false
23645 7254 Acked Flush 0 Done Done Sent false
23646 7255 Acked Read 3 Done Done Sent false
23647 7256 Acked Read 4 Done Done Sent false
23648 7257 Acked Read 9 Done Done Sent false
23649 7258 Acked Read 1 Done Done Sent false
23650 7259 Acked Read 8 Done Done Sent false
23651 7260 Acked Flush 0 Done Done Sent false
23652 7261 Acked Write 9 Done Done Sent false
23653 7262 Acked Read 7 Done Done Sent false
23654 7263 Acked Write 3 Done Done Sent false
23655 7264 Acked Flush 0 Done Done Sent false
23656 7265 Acked Read 9 Done Done Sent false
23657 7266 Acked Read 4 Done Done Sent false
23658 7267 Acked Write 10 Done Done Sent false
23659 7268 Acked Flush 0 Done Done Sent false
23660 7269 Acked Read 3 Done Done Sent false
23661 7270 Acked Flush 0 Done Done Sent false
23662 7271 Acked Flush 0 Done Done Sent false
23663 7272 Acked Read 1 Done Done Sent false
23664 7273 Acked Write 5 Done Done Sent false
23665 7274 Acked Read 10 Done Done Sent false
23666 7275 Acked Write 4 Done Done Sent false
23667 7276 Acked Read 2 Done Done Sent false
23668 7277 Acked Flush 0 Done Done Sent false
23669 7278 Acked Write 4 Done Done Sent false
23670 7279 Acked Write 6 Done Done Sent false
23671 7280 Acked Read 4 Done Done Sent false
23672 7281 Acked Write 3 Done Done Sent false
23673 7282 Acked Write 1 Done Done Sent false
23674 7283 Acked Write 9 Done Done Sent false
23675 7284 Acked Write 3 Done Done Sent false
23676 7285 Acked Write 6 Done Done Sent false
23677 7286 Acked Read 9 Done Done Sent false
23678 7287 Acked Write 4 Done Done Sent false
23679 7288 Acked Read 8 Done Done Sent false
23680 7289 Acked Write 3 Done Done Sent false
23681 7290 Acked Read 2 Done Done Sent false
23682 7291 Acked Write 9 Done Done Sent false
23683 7292 Acked Read 6 Done Done Sent false
23684 7293 Acked Write 10 Done Done Sent false
23685 7294 Acked Read 5 Done Done Sent false
23686 7295 Acked Read 4 Done Done Sent false
23687 7296 Acked Write 5 Done Done Sent false
23688 7297 Acked Write 9 Done Done Sent false
23689 7298 Acked Flush 0 Done Done Sent false
23690 7299 Acked Read 3 Done Done Sent false
23691 7300 Acked Read 4 Done Done Sent false
23692 7301 Acked Read 6 Done Done Sent false
23693 7302 Acked Write 5 Done Done Sent false
23694 7303 Acked Read 5 Done Done Sent false
23695 7304 Acked Write 6 Done Done Sent false
23696 7305 Acked Write 6 Done Done Sent false
23697 7306 Acked Read 10 Done Done Sent false
23698 7307 Acked Write 8 Done Done Sent false
23699 7308 Acked Write 3 Done Done Sent false
23700 7309 Acked Write 1 Done Done Sent false
23701 7310 Acked Read 8 Done Done Sent false
23702 7311 Acked Read 8 Done Done Sent false
23703 7312 Acked Write 8 Done Done Sent false
23704 7313 Acked Flush 0 Done Done Sent false
23705 7314 Acked Read 5 Done Done Sent false
23706 7315 Acked Read 7 Done Done Sent false
23707 7316 Acked Read 7 Done Done Sent false
23708 7317 Acked Flush 0 Done Done Sent false
23709 7318 Acked Write 2 Done Done Sent false
23710 7319 Acked Read 9 Done Done Sent false
23711 7320 Acked Read 4 Done Done Sent false
23712 7321 Acked Write 2 Done Done Sent false
23713 7322 Acked Write 9 Done Done Sent false
23714 7323 Acked Read 2 Done Done Sent false
23715 7324 Acked Read 8 Done Done Sent false
23716 7325 Acked Write 2 Done Done Sent false
23717 7326 Acked Read 4 Done Done Sent false
23718 7327 Acked Read 10 Done Done Sent false
23719 7328 Acked Read 10 Done Done Sent false
23720 7329 Acked Read 7 Done Done Sent false
23721 7330 Acked Read 9 Done Done Sent false
23722 7331 Acked Flush 0 Done Done Sent false
23723 7332 Acked Write 5 Done Done Sent false
23724 7333 Acked Read 4 Done Done Sent false
23725 7334 Acked Write 3 Done Done Sent false
23726 7335 Acked Read 8 Done Done Sent false
23727 7336 Acked Write 2 Done Done Sent false
23728 7337 Acked Read 3 Done Done Sent false
23729 7338 Acked Flush 0 Done Done Sent false
23730 7339 Acked Read 4 Done Done Sent false
23731 7340 Acked Flush 0 Done Done Sent false
23732 STATES DS:0 DS:1 DS:2 TOTAL
23733 Sent 0 0 302 302
23734 Done 302 302 0 604
23735 Skipped 0 0 0 0
23736 Error 0 0 0 0
23737 Last Flush: 7340 7340 7038
23738 Downstairs last five completed: 7038 7037 7036 7035 7034
23739 Upstairs last five completed: 7340 7339 7338 7337 7336
23740 CLIENT: Up:0 ds:302 act:2
237412025-01-21T15:52:08.293ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237422025-01-21T15:52:08.293ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38139) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237432025-01-21T15:52:08.294ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237442025-01-21T15:52:08.295ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237452025-01-21T15:52:08.295ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237462025-01-21T15:52:08.295ZINFOcrucible: send last flush ID to this DS: 7038 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237472025-01-21T15:52:08.295ZINFOcrucible: Replied this last flush ID: 7038 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237482025-01-21T15:52:08.295ZINFOcrucible: [2] client re-new 302 jobs since flush 7038 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237492025-01-21T15:52:08.296ZINFOcrucible: [2] Replayed 302 jobs since flush: 7038 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
23750 ----------------------------------------------------------------
23751 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23752 Downstairs last five completed: 7340 7339 7338 7337 7336
23753 Upstairs last five completed: 7340 7339 7338 7337 7336
23754 CLIENT: Up:0 ds:0 act:3
23755 Replay: All jobs finished
23756 Main task has work to do, go find it
23757 got dsc Stop(1)
23758 stop 1
23759 Jan 21 15:52:10.533[8820] Got stop action so:false kr:false
23760 INFO request completed, latency_us: 358, response_code: 204, uri: /stop/cid/1, method: POST, req_id: b99a386a-e982-4b28-948c-a50d95c77406, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
237612025-01-21T15:52:10.534ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
237622025-01-21T15:52:10.534ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
23763 Jan 21 15:52:10.534 INFO request completed, latency_us: 157, response_code: 200, {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:52:10.534410266Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
23764 uri: /state/cid/1, method: GET, req_id: 82c8d09c-0509-4978-abf2-6a12a46df1bb, remote_addr: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:52:10.534439634Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
23765 127.0.0.1:52945, local_addr: 127.0.0.1:9998
23766 [8820] Exited with: ExitStatus(unix_wait_status(9))
23767 [8820][1] reports Exit
23768 Jan 21 15:52:14.536 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/1, method: GET, req_id: e3a4564c-e856-4c76-bf33-c11d9c61ab9a, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
23769 001/300 Write block 915 len 24576 data: 8 8 8 9 6 6
23770 002/300 Write block 4238 len 28672 data: 3 3 3 3 4 3 3
23771 003/300 Write block 2518 len 32768 data: 3 4 3 4 5 5 5 5
23772 004/300 Flush
23773 005/300 Write block 2289 len 28672 data: 4 4 4 5 4 4 4
23774 006/300 Write block 4632 len 8192 data: 7 6
23775 007/300 Read block 1196 len 24576
23776 008/300 Write block 3787 len 40960 data: 7 8 9 5 6 6 7 6 5 5
23777 009/300 Write block 2604 len 32768 data: 8 7 6 6 6 6 4 3
23778 010/300 Read block 98 len 36864
23779 011/300 Read block 3489 len 16384
23780 012/300 Read block 2238 len 16384
23781 013/300 Read block 4190 len 24576
23782 014/300 Write block 803 len 32768 data: 6 8 8 8 8 6 7 7
23783 015/300 Read block 264 len 32768
23784 016/300 Flush
23785 017/300 Write block 2090 len 36864 data: 5 5 4 4 5 8 7 5 5
23786 018/300 Read block 847 len 16384
23787 019/300 Flush
23788 020/300 Read block 1009 len 8192
23789 021/300 Write block 2762 len 24576 data: 4 4 4 4 4 4
23790 022/300 Read block 2784 len 36864
23791 023/300 Write block 2988 len 28672 data: 5 4 4 2 2 2 2
23792 024/300 Write block 1567 len 4096 data: 5
23793 025/300 Write block 401 len 20480 data: 4 4 5 7 8
23794 026/300 Read block 1935 len 16384
23795 027/300 Read block 730 len 32768
23796 028/300 Flush
23797 029/300 Write block 3754 len 16384 data: 2 2 3 4
23798 030/300 Write block 2919 len 24576 data: 3 3 5 5 4 3
23799 031/300 Read block 1686 len 12288
23800 032/300 Read block 4969 len 24576
23801 033/300 Write block 2881 len 40960 data: 6 6 5 4 4 5 5 4 4 4
23802 034/300 Read block 1764 len 32768
23803 035/300 Flush
23804 036/300 Write block 1804 len 32768 data: 4 4 5 5 7 9 9 9
23805 037/300 Read block 3393 len 40960
23806 038/300 Read block 4401 len 12288
23807 039/300 Write block 14 len 8192 data: 3 5
23808 040/300 Read block 2212 len 24576
23809 041/300 Flush
23810 042/300 Write block 1703 len 4096 data: 5
23811 043/300 Read block 4834 len 12288
23812 044/300 Read block 389 len 36864
23813 045/300 Read block 2403 len 28672
23814 046/300 Write block 717 len 4096 data: 7
23815 047/300 Read block 2467 len 40960
23816 048/300 Write block 2730 len 8192 data: 2 2
23817 049/300 Read block 3818 len 16384
23818 050/300 Read block 597 len 20480
23819 051/300 Write block 3902 len 12288 data: 3 4 4
23820 052/300 Read block 1242 len 32768
23821 053/300 Read block 4792 len 36864
23822 054/300 Read block 1853 len 4096
23823 055/300 Read block 3702 len 8192
23824 056/300 Read block 1907 len 4096
23825 057/300 Read block 1014 len 24576
23826 058/300 Read block 1502 len 4096
23827 059/300 Flush
23828 060/300 Read block 2497 len 40960
23829 061/300 Read block 438 len 20480
23830 062/300 Flush
23831 063/300 Write block 2093 len 24576 data: 5 6 9 8 6 6
23832 064/300 Write block 2504 len 16384 data: 3 2 2 2
23833 065/300 Read block 2422 len 32768
23834 066/300 Write block 596 len 32768 data: 8 7 5 7 6 5 5 6
23835 067/300 Flush
23836 068/300 Read block 4101 len 20480
23837 069/300 Read block 4053 len 12288
23838 070/300 Write block 2339 len 4096 data: 4
23839 071/300 Read block 4614 len 8192
23840 072/300 Read block 4004 len 20480
23841 073/300 Write block 830 len 24576 data: 5 5 4 3 3 3
23842 074/300 Read block 4262 len 20480
23843 075/300 Read block 2530 len 36864
23844 076/300 Write block 3378 len 4096 data: 7
23845 077/300 Write block 3890 len 28672 data: 9 8 8 6 5 6 5
23846 078/300 Read block 4126 len 16384
23847 079/300 Read block 4679 len 16384
23848 080/300 Read block 2980 len 28672
23849 081/300 Read block 303 len 40960
23850 082/300 Read block 3672 len 24576
23851 083/300 Write block 1067 len 28672 data: 6 6 6 8 9 9 10
23852 084/300 Write block 2819 len 28672 data: 5 5 5 6 6 5 5
23853 085/300 Flush
23854 086/300 Read block 1327 len 32768
23855 087/300 Read block 1959 len 16384
23856 088/300 Write block 3322 len 20480 data: 2 2 3 4 4
23857 089/300 Read block 2132 len 24576
23858 090/300 Write block 4562 len 24576 data: 3 2 2 2 2 4
23859 091/300 Read block 1542 len 12288
23860 092/300 Read block 2181 len 24576
23861 093/300 Write block 2184 len 16384 data: 4 4 4 5
23862 094/300 Write block 4055 len 24576 data: 6 5 5 5 4 4
23863 095/300 Write block 1831 len 8192 data: 3 3
23864 096/300 Write block 84 len 20480 data: 5 4 3 3 3
23865 097/300 Write block 2839 len 12288 data: 6 4 5
23866 098/300 Read block 4646 len 4096
23867 099/300 Read block 4314 len 20480
23868 100/300 Write block 446 len 40960 data: 4 4 4 3 4 3 5 6 6 5
23869 101/300 Read block 4843 len 20480
23870 102/300 Read block 3144 len 36864
23871 103/300 Read block 3268 len 4096
23872 104/300 Read block 3640 len 4096
23873 105/300 Write block 3388 len 4096 data: 3
23874 106/300 Write block 523 len 28672 data: 3 3 3 4 7 8 10
23875 107/300 Read block 1328 len 40960
23876 108/300 Read block 2110 len 16384
23877 109/300 Read block 3987 len 12288
23878 110/300 Write block 1837 len 12288 data: 6 5 6
23879 111/300 Write block 4813 len 12288 data: 4 5 5
23880 112/300 Read block 2430 len 24576
23881 113/300 Write block 2558 len 24576 data: 5 5 6 6 5 5
23882 114/300 Read block 589 len 16384
23883 115/300 Read block 1648 len 36864
23884 116/300 Read block 4047 len 8192
23885 117/300 Write block 2288 len 16384 data: 4 5 5 5
23886 118/300 Read block 238 len 28672
23887 119/300 Write block 3878 len 4096 data: 8
23888 120/300 Write block 4803 len 20480 data: 5 5 5 5 4
23889 121/300 Read block 1797 len 4096
23890 122/300 Write block 968 len 40960 data: 5 5 4 5 4 4 6 6 4 4
23891 123/300 Write block 2698 len 32768 data: 5 5 5 5 4 5 3 2
23892 124/300 Write block 3747 len 16384 data: 4 5 5 5
23893 125/300 Read block 1019 len 20480
23894 126/300 Write block 1692 len 4096 data: 3
23895 127/300 Write block 2438 len 20480 data: 3 2 2 3 3
23896 128/300 Read block 4986 len 4096
23897 129/300 Write block 1540 len 4096 data: 7
23898 130/300 Read block 2266 len 40960
23899 131/300 Read block 3171 len 16384
23900 132/300 Write block 1972 len 16384 data: 2 3 4 4
23901 133/300 Flush
23902 134/300 Read block 1961 len 32768
23903 135/300 Write block 2113 len 20480 data: 4 4 4 3 4
23904 136/300 Write block 4450 len 16384 data: 7 5 6 6
23905 137/300 Write block 2990 len 36864 data: 5 3 3 3 3 2 3 2 2
23906 138/300 Write block 4928 len 4096 data: 5
23907 139/300 Read block 1907 len 36864
23908 140/300 Read block 1368 len 16384
23909 141/300 Read block 4271 len 28672
23910 142/300 Write block 3368 len 16384 data: 5 7 5 5
23911 143/300 Write block 3783 len 36864 data: 6 7 7 7 8 9 10 6 7
23912 144/300 Read block 1728 len 40960
23913 145/300 Read block 7 len 32768
23914 146/300 Write block 1907 len 12288 data: 9 10 10
23915 147/300 Write block 4702 len 12288 data: 5 5 5
23916 148/300 Read block 3478 len 36864
23917 149/300 Read block 92 len 28672
23918 150/300 Read block 4636 len 8192
23919 151/300 Read block 1682 len 16384
23920 152/300 Write block 775 len 40960 data: 9 8 6 6 5 5 4 2 3 3
23921 153/300 Read block 417 len 20480
23922 154/300 Read block 1347 len 8192
23923 155/300 Read block 1374 len 24576
23924 156/300 Write block 2026 len 16384 data: 6 8 7 7
23925 157/300 Flush
23926 158/300 Read block 2067 len 20480
23927 159/300 Read block 3191 len 16384
23928 160/300 Write block 4395 len 24576 data: 6 5 7 7 6 6
23929 161/300 Flush
23930 162/300 Write block 552 len 24576 data: 4 4 4 6 5 5
23931 163/300 Write block 3181 len 24576 data: 3 2 2 3 3 3
23932 164/300 Write block 578 len 8192 data: 5 3
23933 165/300 Flush
23934 166/300 Read block 355 len 32768
23935 167/300 Flush
23936 168/300 Read block 1337 len 40960
23937 169/300 Read block 3580 len 32768
23938 170/300 Write block 2885 len 28672 data: 5 6 6 5 5 5 4
23939 171/300 Read block 1348 len 36864
23940 172/300 Flush
23941 173/300 Read block 1404 len 20480
23942 174/300 Write block 269 len 28672 data: 5 7 5 5 5 5 5
23943 175/300 Write block 4948 len 40960 data: 3 3 4 3 5 5 6 6 6 6
23944 176/300 Read block 2327 len 40960
23945 177/300 Read block 3655 len 20480
23946 178/300 Write block 3087 len 16384 data: 6 5 5 6
23947 179/300 Read block 4069 len 20480
23948 180/300 Flush
23949 181/300 Read block 2323 len 28672
23950 182/300 Read block 1352 len 24576
23951 183/300 Write block 3899 len 8192 data: 4 5
23952 184/300 Read block 3722 len 36864
23953 185/300 Read block 787 len 4096
23954 186/300 Write block 2587 len 32768 data: 3 4 4 4 5 6 5 3
23955 187/300 Write block 3163 len 12288 data: 6 7 6
23956 188/300 Read block 78 len 28672
23957 189/300 Flush
23958 190/300 Write block 2990 len 32768 data: 6 4 4 4 4 3 4 3
23959 191/300 Write block 4521 len 36864 data: 5 5 5 5 4 3 3 3 4
23960 192/300 Read block 2495 len 32768
23961 193/300 Flush
23962 194/300 Read block 3852 len 16384
23963 195/300 Write block 4621 len 40960 data: 6 7 8 9 10 10 10 6 4 4
23964 196/300 Read block 658 len 40960
23965 197/300 Read block 4556 len 32768
23966 198/300 Read block 2931 len 16384
23967 199/300 Read block 344 len 40960
23968 200/300 Read block 2605 len 16384
23969 201/300 Flush
23970 202/300 Read block 710 len 16384
23971 203/300 Flush
23972 204/300 Read block 1586 len 36864
23973 205/300 Write block 3416 len 12288 data: 6 7 7
23974 206/300 Write block 1787 len 36864 data: 5 5 5 3 4 4 4 5 7
23975 207/300 Flush
23976 208/300 Flush
23977 209/300 Write block 1331 len 24576 data: 7 6 6 5 5 4
23978 210/300 Read block 1936 len 20480
23979 211/300 Write block 2334 len 24576 data: 7 6 5 4 4 5
23980 212/300 Read block 706 len 40960
23981 213/300 Flush
23982 214/300 Read block 3374 len 12288
23983 215/300 Write block 3787 len 40960 data: 9 10 11 7 8 7 8 7 6 6
23984 216/300 Read block 4220 len 16384
23985 217/300 Read block 772 len 4096
23986 218/300 Read block 614 len 24576
23987 219/300 Flush
23988 220/300 Read block 988 len 32768
23989 221/300 Read block 3898 len 36864
23990 222/300 Read block 2440 len 40960
23991 223/300 Read block 4957 len 40960
23992 224/300 Flush
23993 225/300 Read block 1093 len 40960
23994 226/300 Write block 1764 len 28672 data: 7 6 7 7 8 7 5
23995 227/300 Write block 3359 len 8192 data: 3 4
23996 228/300 Write block 2546 len 12288 data: 5 5 6
23997 229/300 Read block 4947 len 32768
23998 230/300 Write block 4519 len 40960 data: 7 5 6 6 6 6 5 4 4 4
23999 231/300 Write block 3448 len 36864 data: 4 4 5 5 5 5 5 5 5
24000 232/300 Flush
24001 233/300 Read block 1713 len 32768
24002 234/300 Flush
24003 235/300 Read block 2844 len 20480
24004 236/300 Write block 224 len 36864 data: 6 7 7 6 6 7 6 4 3
24005 237/300 Read block 628 len 20480
24006 238/300 Read block 717 len 20480
24007 239/300 Flush
24008 240/300 Write block 2854 len 16384 data: 5 5 6 6
24009 241/300 Write block 2314 len 16384 data: 4 4 4 4
24010 242/300 Flush
24011 243/300 Write block 452 len 20480 data: 6 7 7 6 6
24012 244/300 Read block 1959 len 16384
24013 245/300 Read block 2040 len 24576
24014 246/300 Write block 1472 len 16384 data: 5 4 4 3
24015 247/300 Read block 1752 len 8192
24016 248/300 Read block 2287 len 12288
24017 249/300 Read block 2869 len 12288
24018 250/300 Read block 3318 len 4096
24019 251/300 Read block 1081 len 16384
24020 252/300 Write block 688 len 12288 data: 7 5 5
24021 253/300 Read block 1713 len 12288
24022 254/300 Flush
24023 255/300 Flush
24024 256/300 Flush
24025 257/300 Read block 2118 len 16384
24026 258/300 Write block 817 len 24576 data: 3 4 5 5 5 5
24027 259/300 Read block 2073 len 40960
24028 260/300 Write block 3214 len 40960 data: 4 3 3 2 2 3 3 3 3 3
24029 261/300 Read block 4418 len 4096
24030 262/300 Write block 1064 len 40960 data: 4 5 6 7 7 7 9 10 10 11
24031 263/300 Read block 948 len 12288
24032 264/300 Read block 4531 len 24576
24033 265/300 Write block 2889 len 4096 data: 6
24034 266/300 Flush
24035 267/300 Read block 4413 len 28672
24036 268/300 Flush
24037 269/300 Read block 4210 len 4096
24038 270/300 Read block 454 len 4096
24039 271/300 Flush
24040 272/300 Write block 1687 len 4096 data: 7
24041 273/300 Read block 3639 len 8192
24042 274/300 Write block 432 len 32768 data: 5 4 5 5 5 4 5 5
24043 275/300 Flush
24044 276/300 Read block 3796 len 20480
24045 277/300 Read block 3884 len 12288
24046 278/300 Write block 4057 len 32768 data: 6 6 5 5 4 4 3 3
24047 279/300 Read block 1726 len 32768
24048 280/300 Read block 3660 len 32768
24049 281/300 Read block 1152 len 32768
24050 282/300 Read block 1178 len 4096
24051 283/300 Write block 522 len 36864 data: 3 4 4 4 5 8 9 11 9
24052 284/300 Flush
24053 285/300 Write block 2355 len 16384 data: 8 7 6 7
24054 286/300 Write block 3881 len 40960 data: 7 6 4 3 5 7 6 7 8 10
24055 287/300 Read block 2363 len 20480
24056 288/300 Read block 2898 len 24576
24057 289/300 Write block 1944 len 8192 data: 4 3
24058 290/300 Read block 748 len 8192
24059 291/300 Write block 3289 len 12288 data: 3 4 4
24060 292/300 Read block 2022 len 40960
24061 293/300 Read block 3439 len 16384
24062 294/300 Read block 842 len 8192
24063 295/300 Read block 1911 len 16384
24064 296/300 Flush
24065 297/300 Read block 2671 len 4096
24066 298/300 Read block 161 len 4096
24067 299/300 Read block 2500 len 8192
24068 300/300 Read block 1969 len 36864
24069 301/300 Write block 3797 len 36864 data: 4 4 4 3 3 3 2 2 2
24070 Main task has work to do, go find it
24071 got dsc Start(1)
24072 start 1
24073 Jan 21 15:52:14.780 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/1, method: POST, req_id: a05e6187-ae05-458a-ae62-0af24c62ec21, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
24074 Replay: started 1, returned:Ok(())
24075 ----------------------------------------------------------------
24076 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24077 States: Active Offline Active
24078 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24079 7341 Acked Write 6 Done Sent Done false
24080 7342 Acked Write 7 Done Sent Done false
24081 7343 Acked Write 8 Done Sent Done false
24082 7344 Acked Flush 0 Done Sent Done false
24083 7345 Acked Write 7 Done Sent Done false
24084 7346 Acked Write 2 Done Sent Done false
24085 7347 Acked Read 6 Done Sent Done false
24086 7348 Acked Write 10 Done Sent Done false
24087 7349 Acked Write 8 Done Sent Done false
24088 7350 Acked Read 9 Done Sent Done false
24089 7351 Acked Read 4 Done Sent Done false
24090 7352 Acked Read 4 Done Sent Done false
24091 7353 Acked Read 6 Done Sent Done false
24092 7354 Acked Write 8 Done Sent Done false
24093 7355 Acked Read 8 Done Sent Done false
24094 7356 Acked Flush 0 Done Sent Done false
24095 7357 Acked Write 9 Done Sent Done false
24096 7358 Acked Read 4 Done Sent Done false
24097 7359 Acked Flush 0 Done Sent Done false
24098 7360 Acked Read 2 Done Sent Done false
24099 7361 Acked Write 6 Done Sent Done false
24100 7362 Acked Read 9 Done Sent Done false
24101 7363 Acked Write 7 Done Sent Done false
24102 7364 Acked Write 1 Done Sent Done false
24103 7365 Acked Write 5 Done Sent Done false
24104 7366 Acked Read 4 Done Sent Done false
24105 7367 Acked Read 8 Done Sent Done false
24106 7368 Acked Flush 0 Done Sent Done false
24107 7369 Acked Write 4 Done Sent Done false
24108 7370 Acked Write 6 Done Sent Done false
24109 7371 Acked Read 3 Done Sent Done false
24110 7372 Acked Read 6 Done Sent Done false
24111 7373 Acked Write 10 Done Sent Done false
24112 7374 Acked Read 8 Done Sent Done false
24113 7375 Acked Flush 0 Done Sent Done false
24114 7376 Acked Write 8 Done Sent Done false
24115 7377 Acked Read 10 Done Sent Done false
24116 7378 Acked Read 3 Done Sent Done false
24117 7379 Acked Write 2 Done Sent Done false
24118 7380 Acked Read 6 Done Sent Done false
24119 7381 Acked Flush 0 Done Sent Done false
24120 7382 Acked Write 1 Done Sent Done false
24121 7383 Acked Read 3 Done Sent Done false
24122 7384 Acked Read 9 Done Sent Done false
24123 7385 Acked Read 7 Done Sent Done false
24124 7386 Acked Write 1 Done Sent Done false
24125 7387 Acked Read 10 Done Sent Done false
24126 7388 Acked Write 2 Done Sent Done false
24127 7389 Acked Read 4 Done Sent Done false
24128 7390 Acked Read 5 Done Sent Done false
24129 7391 Acked Write 3 Done Sent Done false
24130 7392 Acked Read 8 Done Sent Done false
24131 7393 Acked Read 9 Done Sent Done false
24132 7394 Acked Read 1 Done Sent Done false
24133 7395 Acked Read 2 Done Sent Done false
24134 7396 Acked Read 1 Done Sent Done false
24135 7397 Acked Read 6 Done Sent Done false
24136 7398 Acked Read 1 Done Sent Done false
24137 7399 Acked Flush 0 Done Sent Done false
24138 7400 Acked Read 10 Done Sent Done false
24139 7401 Acked Read 5 Done Sent Done false
24140 7402 Acked Flush 0 Done Sent Done false
24141 7403 Acked Write 6 Done Sent Done false
24142 7404 Acked Write 4 Done Sent Done false
24143 7405 Acked Read 8 Done Sent Done false
24144 7406 Acked Write 8 Done Sent Done false
24145 7407 Acked Flush 0 Done Sent Done false
24146 7408 Acked Read 5 Done Sent Done false
24147 7409 Acked Read 3 Done Sent Done false
24148 7410 Acked Write 1 Done Sent Done false
24149 7411 Acked Read 2 Done Sent Done false
24150 7412 Acked Read 5 Done Sent Done false
24151 7413 Acked Write 6 Done Sent Done false
24152 7414 Acked Read 5 Done Sent Done false
24153 7415 Acked Read 9 Done Sent Done false
24154 7416 Acked Write 1 Done Sent Done false
24155 7417 Acked Write 7 Done Sent Done false
24156 7418 Acked Read 4 Done Sent Done false
24157 7419 Acked Read 4 Done Sent Done false
24158 7420 Acked Read 7 Done Sent Done false
24159 7421 Acked Read 10 Done Sent Done false
24160 7422 Acked Read 6 Done Sent Done false
24161 7423 Acked Write 7 Done Sent Done false
24162 7424 Acked Write 7 Done Sent Done false
24163 7425 Acked Flush 0 Done Sent Done false
24164 7426 Acked Read 8 Done Sent Done false
24165 7427 Acked Read 4 Done Sent Done false
24166 7428 Acked Write 5 Done Sent Done false
24167 7429 Acked Read 6 Done Sent Done false
24168 7430 Acked Write 6 Done Sent Done false
24169 7431 Acked Read 3 Done Sent Done false
24170 7432 Acked Read 6 Done Sent Done false
24171 7433 Acked Write 4 Done Sent Done false
24172 7434 Acked Write 6 Done Sent Done false
24173 7435 Acked Write 2 Done Sent Done false
24174 7436 Acked Write 5 Done Sent Done false
24175 7437 Acked Write 3 Done Sent Done false
24176 7438 Acked Read 1 Done Sent Done false
24177 7439 Acked Read 5 Done Sent Done false
24178 7440 Acked Write 10 Done Sent Done false
24179 7441 Acked Read 5 Done Sent Done false
24180 7442 Acked Read 9 Done Sent Done false
24181 7443 Acked Read 1 Done Sent Done false
24182 7444 Acked Read 1 Done Sent Done false
24183 7445 Acked Write 1 Done Sent Done false
24184 7446 Acked Write 7 Done Sent Done false
24185 7447 Acked Read 10 Done Sent Done false
24186 7448 Acked Read 4 Done Sent Done false
24187 7449 Acked Read 3 Done Sent Done false
24188 7450 Acked Write 3 Done Sent Done false
24189 7451 Acked Write 3 Done Sent Done false
24190 7452 Acked Read 6 Done Sent Done false
24191 7453 Acked Write 6 Done Sent Done false
24192 7454 Acked Read 4 Done Sent Done false
24193 7455 Acked Read 9 Done Sent Done false
24194 7456 Acked Read 2 Done Sent Done false
24195 7457 Acked Write 4 Done Sent Done false
24196 7458 Acked Read 7 Done Sent Done false
24197 7459 Acked Write 1 Done Sent Done false
24198 7460 Acked Write 5 Done Sent Done false
24199 7461 Acked Read 1 Done Sent Done false
24200 7462 Acked Write 10 Done Sent Done false
24201 7463 Acked Write 8 Done Sent Done false
24202 7464 Acked Write 4 Done Sent Done false
24203 7465 Acked Read 5 Done Sent Done false
24204 7466 Acked Write 1 Done Sent Done false
24205 7467 Acked Write 5 Done Sent Done false
24206 7468 Acked Read 1 Done Sent Done false
24207 7469 Acked Write 1 Done Sent Done false
24208 7470 Acked Read 10 Done Sent Done false
24209 7471 Acked Read 4 Done Sent Done false
24210 7472 Acked Write 4 Done Sent Done false
24211 7473 Acked Flush 0 Done Sent Done false
24212 7474 Acked Read 8 Done Sent Done false
24213 7475 Acked Write 5 Done Sent Done false
24214 7476 Acked Write 4 Done Sent Done false
24215 7477 Acked Write 9 Done Sent Done false
24216 7478 Acked Write 1 Done Sent Done false
24217 7479 Acked Read 9 Done Sent Done false
24218 7480 Acked Read 4 Done Sent Done false
24219 7481 Acked Read 7 Done Sent Done false
24220 7482 Acked Write 4 Done Sent Done false
24221 7483 Acked Write 9 Done Sent Done false
24222 7484 Acked Read 10 Done Sent Done false
24223 7485 Acked Read 8 Done Sent Done false
24224 7486 Acked Write 3 Done Sent Done false
24225 7487 Acked Write 3 Done Sent Done false
24226 7488 Acked Read 9 Done Sent Done false
24227 7489 Acked Read 7 Done Sent Done false
24228 7490 Acked Read 2 Done Sent Done false
24229 7491 Acked Read 4 Done Sent Done false
24230 7492 Acked Write 10 Done Sent Done false
24231 7493 Acked Read 5 Done Sent Done false
24232 7494 Acked Read 2 Done Sent Done false
24233 7495 Acked Read 6 Done Sent Done false
24234 7496 Acked Write 4 Done Sent Done false
24235 7497 Acked Flush 0 Done Sent Done false
24236 7498 Acked Read 5 Done Sent Done false
24237 7499 Acked Read 4 Done Sent Done false
24238 7500 Acked Write 6 Done Sent Done false
24239 7501 Acked Flush 0 Done Sent Done false
24240 7502 Acked Write 6 Done Sent Done false
24241 7503 Acked Write 6 Done Sent Done false
24242 7504 Acked Write 2 Done Sent Done false
24243 7505 Acked Flush 0 Done Sent Done false
24244 7506 Acked Read 8 Done Sent Done false
24245 7507 Acked Flush 0 Done Sent Done false
24246 7508 Acked Read 10 Done Sent Done false
24247 7509 Acked Read 8 Done Sent Done false
24248 7510 Acked Write 7 Done Sent Done false
24249 7511 Acked Read 9 Done Sent Done false
24250 7512 Acked Flush 0 Done Sent Done false
24251 7513 Acked Read 5 Done Sent Done false
24252 7514 Acked Write 7 Done Sent Done false
24253 7515 Acked Write 10 Done Sent Done false
24254 7516 Acked Read 10 Done Sent Done false
24255 7517 Acked Read 5 Done Sent Done false
24256 7518 Acked Write 4 Done Sent Done false
24257 7519 Acked Read 5 Done Sent Done false
24258 7520 Acked Flush 0 Done Sent Done false
24259 7521 Acked Read 7 Done Sent Done false
24260 7522 Acked Read 6 Done Sent Done false
24261 7523 Acked Write 2 Done Sent Done false
24262 7524 Acked Read 9 Done Sent Done false
24263 7525 Acked Read 1 Done Sent Done false
24264 7526 Acked Write 8 Done Sent Done false
24265 7527 Acked Write 3 Done Sent Done false
24266 7528 Acked Read 7 Done Sent Done false
24267 7529 Acked Flush 0 Done Sent Done false
24268 7530 Acked Write 8 Done Sent Done false
24269 7531 Acked Write 9 Done Sent Done false
24270 7532 Acked Read 8 Done Sent Done false
24271 7533 Acked Flush 0 Done Sent Done false
24272 7534 Acked Read 4 Done Sent Done false
24273 7535 Acked Write 10 Done Sent Done false
24274 7536 Acked Read 10 Done Sent Done false
24275 7537 Acked Read 8 Done Sent Done false
24276 7538 Acked Read 4 Done Sent Done false
24277 7539 Acked Read 10 Done Sent Done false
24278 7540 Acked Read 4 Done Sent Done false
24279 7541 Acked Flush 0 Done Sent Done false
24280 7542 Acked Read 4 Done Sent Done false
24281 7543 Acked Flush 0 Done Sent Done false
24282 7544 Acked Read 9 Done Sent Done false
24283 7545 Acked Write 3 Done Sent Done false
24284 7546 Acked Write 9 Done Sent Done false
24285 7547 Acked Flush 0 Done Sent Done false
24286 7548 Acked Flush 0 Done Sent Done false
24287 7549 Acked Write 6 Done Sent Done false
24288 7550 Acked Read 5 Done Sent Done false
24289 7551 Acked Write 6 Done Sent Done false
24290 7552 Acked Read 10 Done Sent Done false
24291 7553 Acked Flush 0 Done Sent Done false
24292 7554 Acked Read 3 Done Sent Done false
24293 7555 Acked Write 10 Done Sent Done false
24294 7556 Acked Read 4 Done Sent Done false
24295 7557 Acked Read 1 Done Sent Done false
24296 7558 Acked Read 6 Done Sent Done false
24297 7559 Acked Flush 0 Done Sent Done false
24298 7560 Acked Read 8 Done Sent Done false
24299 7561 Acked Read 9 Done Sent Done false
24300 7562 Acked Read 10 Done Sent Done false
24301 7563 Acked Read 10 Done Sent Done false
24302 7564 Acked Flush 0 Done Sent Done false
24303 7565 Acked Read 10 Done Sent Done false
24304 7566 Acked Write 7 Done Sent Done false
24305 7567 Acked Write 2 Done Sent Done false
24306 7568 Acked Write 3 Done Sent Done false
24307 7569 Acked Read 8 Done Sent Done false
24308 7570 Acked Write 10 Done Sent Done false
24309 7571 Acked Write 9 Done Sent Done false
24310 7572 Acked Flush 0 Done Sent Done false
24311 7573 Acked Read 8 Done Sent Done false
24312 7574 Acked Flush 0 Done Sent Done false
24313 7575 Acked Read 5 Done Sent Done false
24314 7576 Acked Write 9 Done Sent Done false
24315 7577 Acked Read 5 Done Sent Done false
24316 7578 Acked Read 5 Done Sent Done false
24317 7579 Acked Flush 0 Done Sent Done false
24318 7580 Acked Write 4 Done Sent Done false
24319 7581 Acked Write 4 Done Sent Done false
24320 7582 Acked Flush 0 Done Sent Done false
24321 7583 Acked Write 5 Done Sent Done false
24322 7584 Acked Read 4 Done Sent Done false
24323 7585 Acked Read 6 Done Sent Done false
24324 7586 Acked Write 4 Done Sent Done false
24325 7587 Acked Read 2 Done Sent Done false
24326 7588 Acked Read 3 Done Sent Done false
24327 7589 Acked Read 3 Done Sent Done false
24328 7590 Acked Read 1 Done Sent Done false
24329 7591 Acked Read 4 Done Sent Done false
24330 7592 Acked Write 3 Done Sent Done false
24331 7593 Acked Read 3 Done Sent Done false
24332 7594 Acked Flush 0 Done Sent Done false
24333 7595 Acked Flush 0 Done Sent Done false
24334 7596 Acked Flush 0 Done Sent Done false
24335 7597 Acked Read 4 Done Sent Done false
24336 7598 Acked Write 6 Done Sent Done false
24337 7599 Acked Read 10 Done Sent Done false
24338 7600 Acked Write 10 Done Sent Done false
24339 7601 Acked Read 1 Done Sent Done false
24340 7602 Acked Write 10 Done Sent Done false
24341 7603 Acked Read 3 Done Sent Done false
24342 7604 Acked Read 6 Done Sent Done false
24343 7605 Acked Write 1 Done Sent Done false
24344 7606 Acked Flush 0 Done Sent Done false
24345 7607 Acked Read 7 Done Sent Done false
24346 7608 Acked Flush 0 Done Sent Done false
24347 7609 Acked Read 1 Done Sent Done false
24348 7610 Acked Read 1 Done Sent Done false
24349 7611 Acked Flush 0 Done Sent Done false
24350 7612 Acked Write 1 Done Sent Done false
24351 7613 Acked Read 2 Done Sent Done false
24352 7614 Acked Write 8 Done Sent Done false
24353 7615 Acked Flush 0 Done Sent Done false
24354 7616 Acked Read 5 Done Sent Done false
24355 7617 Acked Read 3 Done Sent Done false
24356 7618 Acked Write 8 Done Sent Done false
24357 7619 Acked Read 8 Done Sent Done false
24358 7620 Acked Read 8 Done Sent Done false
24359 7621 Acked Read 8 Done Sent Done false
24360 7622 Acked Read 1 Done Sent Done false
24361 7623 Acked Write 9 Done Sent Done false
24362 7624 Acked Flush 0 Done Sent Done false
24363 7625 Acked Write 4 Done Sent Done false
24364 7626 Acked Write 10 Done Sent Done false
24365 7627 Acked Read 5 Done Sent Done false
24366 7628 Acked Read 6 Done Sent Done false
24367 7629 Acked Write 2 Done Sent Done false
24368 7630 Acked Read 2 Done Sent Done false
24369 7631 Acked Write 3 Done Sent Done false
24370 7632 Acked Read 10 Done Sent Done false
24371 7633 Acked Read 4 Done Sent Done false
24372 7634 Acked Read 2 Done Sent Done false
24373 7635 Acked Read 4 Done Sent Done false
24374 7636 Acked Flush 0 Done Sent Done false
24375 7637 Acked Read 1 Done Sent Done false
24376 7638 Acked Read 1 Done Sent Done false
24377 7639 Acked Read 2 Done Sent Done false
24378 7640 Acked Read 9 Done Sent Done false
24379 7641 Acked Write 9 Done Sent Done false
24380 STATES DS:0 DS:1 DS:2 TOTAL
24381 Sent 0 301 0 301
24382 Done 301 0 301 602
24383 Skipped 0 0 0 0
24384 Error 0 0 0 0
24385 Last Flush: 7636 7340 7636
24386 Downstairs last five completed: 7340 7339 7338 7337 7336
24387 Upstairs last five completed: 7641 7640 7639 7638 7637
24388 CLIENT: Up:0 ds:301 act:2
24389 [8820] got start action
24390 [8820] I am going to restart
24391 Starting downstairs at port 8820
24392 Make output file at "/tmp/dsc/downstairs-8820.txt"
24393 [8820][1] reports Starting
24394 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1222)
24395 [8820][1] reports Running
24396 ----------------------------------------------------------------
24397 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24398 States: Active Offline Active
24399 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24400 7341 Acked Write 6 Done Sent Done false
24401 7342 Acked Write 7 Done Sent Done false
24402 7343 Acked Write 8 Done Sent Done false
24403 7344 Acked Flush 0 Done Sent Done false
24404 7345 Acked Write 7 Done Sent Done false
24405 7346 Acked Write 2 Done Sent Done false
24406 7347 Acked Read 6 Done Sent Done false
24407 7348 Acked Write 10 Done Sent Done false
24408 7349 Acked Write 8 Done Sent Done false
24409 7350 Acked Read 9 Done Sent Done false
24410 7351 Acked Read 4 Done Sent Done false
24411 7352 Acked Read 4 Done Sent Done false
24412 7353 Acked Read 6 Done Sent Done false
24413 7354 Acked Write 8 Done Sent Done false
24414 7355 Acked Read 8 Done Sent Done false
24415 7356 Acked Flush 0 Done Sent Done false
24416 7357 Acked Write 9 Done Sent Done false
24417 7358 Acked Read 4 Done Sent Done false
24418 7359 Acked Flush 0 Done Sent Done false
24419 7360 Acked Read 2 Done Sent Done false
24420 7361 Acked Write 6 Done Sent Done false
24421 7362 Acked Read 9 Done Sent Done false
24422 7363 Acked Write 7 Done Sent Done false
24423 7364 Acked Write 1 Done Sent Done false
24424 7365 Acked Write 5 Done Sent Done false
24425 7366 Acked Read 4 Done Sent Done false
24426 7367 Acked Read 8 Done Sent Done false
24427 7368 Acked Flush 0 Done Sent Done false
24428 7369 Acked Write 4 Done Sent Done false
24429 7370 Acked Write 6 Done Sent Done false
24430 7371 Acked Read 3 Done Sent Done false
24431 7372 Acked Read 6 Done Sent Done false
24432 7373 Acked Write 10 Done Sent Done false
24433 7374 Acked Read 8 Done Sent Done false
24434 7375 Acked Flush 0 Done Sent Done false
24435 7376 Acked Write 8 Done Sent Done false
24436 7377 Acked Read 10 Done Sent Done false
24437 7378 Acked Read 3 Done Sent Done false
24438 7379 Acked Write 2 Done Sent Done false
24439 7380 Acked Read 6 Done Sent Done false
24440 7381 Acked Flush 0 Done Sent Done false
24441 7382 Acked Write 1 Done Sent Done false
24442 7383 Acked Read 3 Done Sent Done false
24443 7384 Acked Read 9 Done Sent Done false
24444 7385 Acked Read 7 Done Sent Done false
24445 7386 Acked Write 1 Done Sent Done false
24446 7387 Acked Read 10 Done Sent Done false
24447 7388 Acked Write 2 Done Sent Done false
24448 7389 Acked Read 4 Done Sent Done false
24449 7390 Acked Read 5 Done Sent Done false
24450 7391 Acked Write 3 Done Sent Done false
24451 7392 Acked Read 8 Done Sent Done false
24452 7393 Acked Read 9 Done Sent Done false
24453 7394 Acked Read 1 Done Sent Done false
24454 7395 Acked Read 2 Done Sent Done false
24455 7396 Acked Read 1 Done Sent Done false
24456 7397 Acked Read 6 Done Sent Done false
24457 7398 Acked Read 1 Done Sent Done false
24458 7399 Acked Flush 0 Done Sent Done false
24459 7400 Acked Read 10 Done Sent Done false
24460 7401 Acked Read 5 Done Sent Done false
24461 7402 Acked Flush 0 Done Sent Done false
24462 7403 Acked Write 6 Done Sent Done false
24463 7404 Acked Write 4 Done Sent Done false
24464 7405 Acked Read 8 Done Sent Done false
24465 7406 Acked Write 8 Done Sent Done false
24466 7407 Acked Flush 0 Done Sent Done false
24467 7408 Acked Read 5 Done Sent Done false
24468 7409 Acked Read 3 Done Sent Done false
24469 7410 Acked Write 1 Done Sent Done false
24470 7411 Acked Read 2 Done Sent Done false
24471 7412 Acked Read 5 Done Sent Done false
24472 7413 Acked Write 6 Done Sent Done false
24473 7414 Acked Read 5 Done Sent Done false
24474 7415 Acked Read 9 Done Sent Done false
24475 7416 Acked Write 1 Done Sent Done false
24476 7417 Acked Write 7 Done Sent Done false
24477 7418 Acked Read 4 Done Sent Done false
24478 7419 Acked Read 4 Done Sent Done false
24479 7420 Acked Read 7 Done Sent Done false
24480 7421 Acked Read 10 Done Sent Done false
24481 7422 Acked Read 6 Done Sent Done false
24482 7423 Acked Write 7 Done Sent Done false
24483 7424 Acked Write 7 Done Sent Done false
24484 7425 Acked Flush 0 Done Sent Done false
24485 7426 Acked Read 8 Done Sent Done false
24486 7427 Acked Read 4 Done Sent Done false
24487 7428 Acked Write 5 Done Sent Done false
24488 7429 Acked Read 6 Done Sent Done false
24489 7430 Acked Write 6 Done Sent Done false
24490 7431 Acked Read 3 Done Sent Done false
24491 7432 Acked Read 6 Done Sent Done false
24492 7433 Acked Write 4 Done Sent Done false
24493 7434 Acked Write 6 Done Sent Done false
24494 7435 Acked Write 2 Done Sent Done false
24495 7436 Acked Write 5 Done Sent Done false
24496 7437 Acked Write 3 Done Sent Done false
24497 7438 Acked Read 1 Done Sent Done false
24498 7439 Acked Read 5 Done Sent Done false
24499 7440 Acked Write 10 Done Sent Done false
24500 7441 Acked Read 5 Done Sent Done false
24501 7442 Acked Read 9 Done Sent Done false
24502 7443 Acked Read 1 Done Sent Done false
24503 7444 Acked Read 1 Done Sent Done false
24504 7445 Acked Write 1 Done Sent Done false
24505 7446 Acked Write 7 Done Sent Done false
24506 7447 Acked Read 10 Done Sent Done false
24507 7448 Acked Read 4 Done Sent Done false
24508 7449 Acked Read 3 Done Sent Done false
24509 7450 Acked Write 3 Done Sent Done false
24510 7451 Acked Write 3 Done Sent Done false
24511 7452 Acked Read 6 Done Sent Done false
24512 7453 Acked Write 6 Done Sent Done false
24513 7454 Acked Read 4 Done Sent Done false
24514 7455 Acked Read 9 Done Sent Done false
24515 7456 Acked Read 2 Done Sent Done false
24516 7457 Acked Write 4 Done Sent Done false
24517 7458 Acked Read 7 Done Sent Done false
24518 7459 Acked Write 1 Done Sent Done false
24519 7460 Acked Write 5 Done Sent Done false
24520 7461 Acked Read 1 Done Sent Done false
24521 7462 Acked Write 10 Done Sent Done false
24522 7463 Acked Write 8 Done Sent Done false
24523 7464 Acked Write 4 Done Sent Done false
24524 7465 Acked Read 5 Done Sent Done false
24525 7466 Acked Write 1 Done Sent Done false
24526 7467 Acked Write 5 Done Sent Done false
24527 7468 Acked Read 1 Done Sent Done false
24528 7469 Acked Write 1 Done Sent Done false
24529 7470 Acked Read 10 Done Sent Done false
24530 7471 Acked Read 4 Done Sent Done false
24531 7472 Acked Write 4 Done Sent Done false
24532 7473 Acked Flush 0 Done Sent Done false
24533 7474 Acked Read 8 Done Sent Done false
24534 7475 Acked Write 5 Done Sent Done false
24535 7476 Acked Write 4 Done Sent Done false
24536 7477 Acked Write 9 Done Sent Done false
24537 7478 Acked Write 1 Done Sent Done false
24538 7479 Acked Read 9 Done Sent Done false
24539 7480 Acked Read 4 Done Sent Done false
24540 7481 Acked Read 7 Done Sent Done false
24541 7482 Acked Write 4 Done Sent Done false
24542 7483 Acked Write 9 Done Sent Done false
24543 7484 Acked Read 10 Done Sent Done false
24544 7485 Acked Read 8 Done Sent Done false
24545 7486 Acked Write 3 Done Sent Done false
24546 7487 Acked Write 3 Done Sent Done false
24547 7488 Acked Read 9 Done Sent Done false
24548 7489 Acked Read 7 Done Sent Done false
24549 7490 Acked Read 2 Done Sent Done false
24550 7491 Acked Read 4 Done Sent Done false
24551 7492 Acked Write 10 Done Sent Done false
24552 7493 Acked Read 5 Done Sent Done false
24553 7494 Acked Read 2 Done Sent Done false
24554 7495 Acked Read 6 Done Sent Done false
24555 7496 Acked Write 4 Done Sent Done false
24556 7497 Acked Flush 0 Done Sent Done false
24557 7498 Acked Read 5 Done Sent Done false
24558 7499 Acked Read 4 Done Sent Done false
24559 7500 Acked Write 6 Done Sent Done false
24560 7501 Acked Flush 0 Done Sent Done false
24561 7502 Acked Write 6 Done Sent Done false
24562 7503 Acked Write 6 Done Sent Done false
24563 7504 Acked Write 2 Done Sent Done false
24564 7505 Acked Flush 0 Done Sent Done false
24565 7506 Acked Read 8 Done Sent Done false
24566 7507 Acked Flush 0 Done Sent Done false
24567 7508 Acked Read 10 Done Sent Done false
24568 7509 Acked Read 8 Done Sent Done false
24569 7510 Acked Write 7 Done Sent Done false
24570 7511 Acked Read 9 Done Sent Done false
24571 7512 Acked Flush 0 Done Sent Done false
24572 7513 Acked Read 5 Done Sent Done false
24573 7514 Acked Write 7 Done Sent Done false
24574 7515 Acked Write 10 Done Sent Done false
24575 7516 Acked Read 10 Done Sent Done false
24576 7517 Acked Read 5 Done Sent Done false
24577 7518 Acked Write 4 Done Sent Done false
24578 7519 Acked Read 5 Done Sent Done false
24579 7520 Acked Flush 0 Done Sent Done false
24580 7521 Acked Read 7 Done Sent Done false
24581 7522 Acked Read 6 Done Sent Done false
24582 7523 Acked Write 2 Done Sent Done false
24583 7524 Acked Read 9 Done Sent Done false
24584 7525 Acked Read 1 Done Sent Done false
24585 7526 Acked Write 8 Done Sent Done false
24586 7527 Acked Write 3 Done Sent Done false
24587 7528 Acked Read 7 Done Sent Done false
24588 7529 Acked Flush 0 Done Sent Done false
24589 7530 Acked Write 8 Done Sent Done false
24590 7531 Acked Write 9 Done Sent Done false
24591 7532 Acked Read 8 Done Sent Done false
24592 7533 Acked Flush 0 Done Sent Done false
24593 7534 Acked Read 4 Done Sent Done false
24594 7535 Acked Write 10 Done Sent Done false
24595 7536 Acked Read 10 Done Sent Done false
24596 7537 Acked Read 8 Done Sent Done false
24597 7538 Acked Read 4 Done Sent Done false
24598 7539 Acked Read 10 Done Sent Done false
24599 7540 Acked Read 4 Done Sent Done false
24600 7541 Acked Flush 0 Done Sent Done false
24601 7542 Acked Read 4 Done Sent Done false
24602 7543 Acked Flush 0 Done Sent Done false
24603 7544 Acked Read 9 Done Sent Done false
24604 7545 Acked Write 3 Done Sent Done false
24605 7546 Acked Write 9 Done Sent Done false
24606 7547 Acked Flush 0 Done Sent Done false
24607 7548 Acked Flush 0 Done Sent Done false
24608 7549 Acked Write 6 Done Sent Done false
24609 7550 Acked Read 5 Done Sent Done false
24610 7551 Acked Write 6 Done Sent Done false
24611 7552 Acked Read 10 Done Sent Done false
24612 7553 Acked Flush 0 Done Sent Done false
24613 7554 Acked Read 3 Done Sent Done false
24614 7555 Acked Write 10 Done Sent Done false
24615 7556 Acked Read 4 Done Sent Done false
24616 7557 Acked Read 1 Done Sent Done false
24617 7558 Acked Read 6 Done Sent Done false
24618 7559 Acked Flush 0 Done Sent Done false
24619 7560 Acked Read 8 Done Sent Done false
24620 7561 Acked Read 9 Done Sent Done false
24621 7562 Acked Read 10 Done Sent Done false
24622 7563 Acked Read 10 Done Sent Done false
24623 7564 Acked Flush 0 Done Sent Done false
24624 7565 Acked Read 10 Done Sent Done false
24625 7566 Acked Write 7 Done Sent Done false
24626 7567 Acked Write 2 Done Sent Done false
24627 7568 Acked Write 3 Done Sent Done false
24628 7569 Acked Read 8 Done Sent Done false
24629 7570 Acked Write 10 Done Sent Done false
24630 7571 Acked Write 9 Done Sent Done false
24631 7572 Acked Flush 0 Done Sent Done false
24632 7573 Acked Read 8 Done Sent Done false
24633 7574 Acked Flush 0 Done Sent Done false
24634 7575 Acked Read 5 Done Sent Done false
24635 7576 Acked Write 9 Done Sent Done false
24636 7577 Acked Read 5 Done Sent Done false
24637 7578 Acked Read 5 Done Sent Done false
24638 7579 Acked Flush 0 Done Sent Done false
24639 7580 Acked Write 4 Done Sent Done false
24640 7581 Acked Write 4 Done Sent Done false
24641 7582 Acked Flush 0 Done Sent Done false
24642 7583 Acked Write 5 Done Sent Done false
24643 7584 Acked Read 4 Done Sent Done false
24644 7585 Acked Read 6 Done Sent Done false
24645 7586 Acked Write 4 Done Sent Done false
24646 7587 Acked Read 2 Done Sent Done false
24647 7588 Acked Read 3 Done Sent Done false
24648 7589 Acked Read 3 Done Sent Done false
24649 7590 Acked Read 1 Done Sent Done false
24650 7591 Acked Read 4 Done Sent Done false
24651 7592 Acked Write 3 Done Sent Done false
24652 7593 Acked Read 3 Done Sent Done false
24653 7594 Acked Flush 0 Done Sent Done false
24654 7595 Acked Flush 0 Done Sent Done false
24655 7596 Acked Flush 0 Done Sent Done false
24656 7597 Acked Read 4 Done Sent Done false
24657 7598 Acked Write 6 Done Sent Done false
24658 7599 Acked Read 10 Done Sent Done false
24659 7600 Acked Write 10 Done Sent Done false
24660 7601 Acked Read 1 Done Sent Done false
24661 7602 Acked Write 10 Done Sent Done false
24662 7603 Acked Read 3 Done Sent Done false
24663 7604 Acked Read 6 Done Sent Done false
24664 7605 Acked Write 1 Done Sent Done false
24665 7606 Acked Flush 0 Done Sent Done false
24666 7607 Acked Read 7 Done Sent Done false
24667 7608 Acked Flush 0 Done Sent Done false
24668 7609 Acked Read 1 Done Sent Done false
24669 7610 Acked Read 1 Done Sent Done false
24670 7611 Acked Flush 0 Done Sent Done false
24671 7612 Acked Write 1 Done Sent Done false
24672 7613 Acked Read 2 Done Sent Done false
24673 7614 Acked Write 8 Done Sent Done false
24674 7615 Acked Flush 0 Done Sent Done false
24675 7616 Acked Read 5 Done Sent Done false
24676 7617 Acked Read 3 Done Sent Done false
24677 7618 Acked Write 8 Done Sent Done false
24678 7619 Acked Read 8 Done Sent Done false
24679 7620 Acked Read 8 Done Sent Done false
24680 7621 Acked Read 8 Done Sent Done false
24681 7622 Acked Read 1 Done Sent Done false
24682 7623 Acked Write 9 Done Sent Done false
24683 7624 Acked Flush 0 Done Sent Done false
24684 7625 Acked Write 4 Done Sent Done false
24685 7626 Acked Write 10 Done Sent Done false
24686 7627 Acked Read 5 Done Sent Done false
24687 7628 Acked Read 6 Done Sent Done false
24688 7629 Acked Write 2 Done Sent Done false
24689 7630 Acked Read 2 Done Sent Done false
24690 7631 Acked Write 3 Done Sent Done false
24691 7632 Acked Read 10 Done Sent Done false
24692 7633 Acked Read 4 Done Sent Done false
24693 7634 Acked Read 2 Done Sent Done false
24694 7635 Acked Read 4 Done Sent Done false
24695 7636 Acked Flush 0 Done Sent Done false
24696 7637 Acked Read 1 Done Sent Done false
24697 7638 Acked Read 1 Done Sent Done false
24698 7639 Acked Read 2 Done Sent Done false
24699 7640 Acked Read 9 Done Sent Done false
24700 7641 Acked Write 9 Done Sent Done false
24701 7642 Acked Flush 0 Done Sent Done false
24702 STATES DS:0 DS:1 DS:2 TOTAL
24703 Sent 0 302 0 302
24704 Done 302 0 302 604
24705 Skipped 0 0 0 0
24706 Error 0 0 0 0
24707 Last Flush: 7642 7340 7642
24708 Downstairs last five completed: 7340 7339 7338 7337 7336
24709 Upstairs last five completed: 7642 7641 7640 7639 7638
24710 CLIENT: Up:0 ds:302 act:2
247112025-01-21T15:52:20.535ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247122025-01-21T15:52:20.535ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53151) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247132025-01-21T15:52:20.537ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247142025-01-21T15:52:20.537ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247152025-01-21T15:52:20.537ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247162025-01-21T15:52:20.537ZINFOcrucible: send last flush ID to this DS: 7340 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247172025-01-21T15:52:20.537ZINFOcrucible: Replied this last flush ID: 7340 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247182025-01-21T15:52:20.537ZINFOcrucible: [1] client re-new 302 jobs since flush 7340 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247192025-01-21T15:52:20.538ZINFOcrucible: [1] Replayed 302 jobs since flush: 7340 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
24720 ----------------------------------------------------------------
24721 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
24722 Downstairs last five completed: 7642 7641 7640 7639 7638
24723 Upstairs last five completed: 7642 7641 7640 7639 7638
24724 CLIENT: Up:0 ds:0 act:3
24725 Replay: All jobs finished
24726 Main task has work to do, go find it
24727 got dsc Stop(2)
24728 stop 2
24729 [8830] Got stop action so:false kr:false
24730 Jan 21 15:52:22.785 INFO request completed, latency_us: 361, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 176f2150-ed51-4155-b7a0-cd87f74dbe5e, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
24731 Jan 21 15:52:22.786{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:52:22.786269566Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
24732 INFO request completed, latency_us: 117, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4b036746-1132-45e2-a619-fea6a6f14299, remote_addr: 127.0.{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:52:22.78637178Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
24733 0.1:52945, local_addr: 127.0.0.1:9998
247342025-01-21T15:52:22.786ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
247352025-01-21T15:52:22.786ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
24736 [8830] Exited with: ExitStatus(unix_wait_status(9))
24737 [8830][2] reports Exit
24738 Jan 21 15:52:26.788 INFO request completed, latency_us: 215, response_code: 200, uri: /state/cid/2, method: GET, req_id: d7abb977-18c5-4bb4-8f02-d00c40f949c6, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
24739 001/300 Write block 2096 len 8192 data: 9 7
24740 002/300 Flush
24741 003/300 Write block 4477 len 24576 data: 6 6 4 4 3 6
24742 004/300 Read block 1826 len 40960
24743 005/300 Read block 409 len 40960
24744 006/300 Read block 1008 len 4096
24745 007/300 Write block 4853 len 24576 data: 6 6 7 6 6 4
24746 008/300 Write block 2116 len 40960 data: 4 5 4 4 4 5 4 4 4 4
24747 009/300 Write block 1672 len 20480 data: 7 7 6 6 4
24748 010/300 Read block 4251 len 20480
24749 011/300 Read block 3784 len 40960
24750 012/300 Write block 2404 len 32768 data: 2 2 2 2 3 3 3 3
24751 013/300 Read block 3342 len 40960
24752 014/300 Write block 4028 len 24576 data: 3 4 6 7 8 8
24753 015/300 Read block 2188 len 12288
24754 016/300 Read block 722 len 12288
24755 017/300 Read block 532 len 32768
24756 018/300 Read block 4094 len 40960
24757 019/300 Read block 4809 len 24576
24758 020/300 Read block 411 len 36864
24759 021/300 Write block 3446 len 24576 data: 5 4 5 5 6 6
24760 022/300 Write block 4720 len 16384 data: 8 8 8 7
24761 023/300 Read block 2766 len 8192
24762 024/300 Read block 3743 len 24576
24763 025/300 Read block 257 len 8192
24764 026/300 Read block 4426 len 12288
24765 027/300 Read block 488 len 4096
24766 028/300 Write block 1116 len 4096 data: 4
24767 029/300 Flush
24768 030/300 Write block 1025 len 28672 data: 6 6 6 5 6 4 4
24769 031/300 Write block 2993 len 12288 data: 5 5 4
24770 032/300 Read block 522 len 40960
24771 033/300 Flush
24772 034/300 Write block 2093 len 24576 data: 6 7 10 10 8 7
24773 035/300 Read block 4969 len 12288
24774 036/300 Read block 4462 len 24576
24775 037/300 Read block 2784 len 32768
24776 038/300 Write block 2578 len 8192 data: 4 4
24777 039/300 Read block 3055 len 32768
24778 040/300 Read block 779 len 32768
24779 041/300 Read block 3614 len 4096
24780 042/300 Read block 3227 len 32768
24781 043/300 Read block 2443 len 36864
24782 044/300 Flush
24783 045/300 Flush
24784 046/300 Read block 387 len 12288
24785 047/300 Write block 2811 len 12288 data: 5 5 6
24786 048/300 Flush
24787 049/300 Read block 2936 len 12288
24788 050/300 Write block 3167 len 40960 data: 4 3 2 2 5 5 6 6 7 7
24789 051/300 Flush
24790 052/300 Read block 3295 len 8192
24791 053/300 Read block 372 len 36864
24792 054/300 Read block 4607 len 32768
24793 055/300 Write block 4288 len 40960 data: 4 4 4 2 3 3 3 4 5 4
24794 056/300 Flush
24795 057/300 Flush
24796 058/300 Read block 1310 len 28672
24797 059/300 Read block 1630 len 12288
24798 060/300 Read block 3389 len 32768
24799 061/300 Write block 2323 len 20480 data: 4 3 3 3 4
24800 062/300 Write block 2430 len 36864 data: 6 6 5 6 6 5 5 5 4
24801 063/300 Write block 2855 len 28672 data: 6 7 7 7 5 5 4
24802 064/300 Write block 783 len 4096 data: 4
24803 065/300 Read block 1773 len 4096
24804 066/300 Flush
24805 067/300 Read block 2495 len 20480
24806 068/300 Write block 3877 len 12288 data: 8 9 7
24807 069/300 Read block 3990 len 4096
24808 070/300 Write block 2884 len 40960 data: 5 6 7 7 6 7 6 5 3 5
24809 071/300 Write block 2866 len 24576 data: 3 3 2 2 3 3
24810 072/300 Write block 1225 len 32768 data: 5 6 6 5 5 5 5 3
24811 073/300 Read block 3940 len 24576
24812 074/300 Write block 864 len 20480 data: 8 8 8 7 5
24813 075/300 Read block 671 len 12288
24814 076/300 Write block 4912 len 40960 data: 5 5 6 6 5 4 3 3 3 4
24815 077/300 Read block 4541 len 8192
24816 078/300 Read block 3964 len 40960
24817 079/300 Read block 2444 len 36864
24818 080/300 Read block 4248 len 40960
24819 081/300 Write block 3977 len 12288 data: 5 5 4
24820 082/300 Flush
24821 083/300 Read block 4212 len 40960
24822 084/300 Read block 3160 len 36864
24823 085/300 Read block 4348 len 20480
24824 086/300 Read block 1431 len 16384
24825 087/300 Write block 100 len 40960 data: 2 2 2 4 4 4 4 4 4 2
24826 088/300 Read block 2542 len 16384
24827 089/300 Read block 4235 len 20480
24828 090/300 Write block 3196 len 16384 data: 3 4 6 8
24829 091/300 Read block 2632 len 4096
24830 092/300 Read block 1091 len 8192
24831 093/300 Read block 2701 len 28672
24832 094/300 Read block 361 len 24576
24833 095/300 Write block 1194 len 24576 data: 6 7 8 7 7 6
24834 096/300 Read block 4099 len 28672
24835 097/300 Write block 2116 len 4096 data: 5
24836 098/300 Flush
24837 099/300 Write block 1862 len 16384 data: 6 8 8 9
24838 100/300 Write block 632 len 36864 data: 5 4 5 5 6 7 7 6 5
24839 101/300 Write block 638 len 12288 data: 8 7 6
24840 102/300 Write block 3808 len 36864 data: 3 5 5 6 8 8 9 9 9
24841 103/300 Read block 1177 len 32768
24842 104/300 Flush
24843 105/300 Read block 4405 len 24576
24844 106/300 Write block 3499 len 40960 data: 4 5 5 5 6 6 5 3 4 4
24845 107/300 Write block 933 len 24576 data: 6 5 5 6 6 6
24846 108/300 Read block 491 len 28672
24847 109/300 Write block 340 len 20480 data: 6 5 5 4 4
24848 110/300 Write block 1494 len 16384 data: 4 3 4 5
24849 111/300 Read block 133 len 16384
24850 112/300 Read block 1833 len 4096
24851 113/300 Read block 4979 len 20480
24852 114/300 Read block 3645 len 20480
24853 115/300 Write block 3966 len 40960 data: 3 2 2 2 2 2 2 3 3 5
24854 116/300 Write block 4716 len 20480 data: 5 4 5 5 9
24855 117/300 Flush
24856 118/300 Read block 3569 len 32768
24857 119/300 Read block 1923 len 40960
24858 120/300 Write block 4810 len 8192 data: 5 4
24859 121/300 Read block 306 len 40960
24860 122/300 Read block 2263 len 16384
24861 123/300 Write block 3485 len 40960 data: 6 6 6 4 3 4 5 5 7 6
24862 124/300 Flush
24863 125/300 Read block 2275 len 12288
24864 126/300 Write block 3275 len 40960 data: 4 3 3 3 3 5 5 7 8 6
24865 127/300 Read block 2759 len 40960
24866 128/300 Read block 2774 len 28672
24867 129/300 Read block 871 len 32768
24868 130/300 Write block 2737 len 8192 data: 4 3
24869 131/300 Read block 977 len 8192
24870 132/300 Read block 852 len 12288
24871 133/300 Read block 3632 len 40960
24872 134/300 Read block 446 len 8192
24873 135/300 Write block 3879 len 12288 data: 8 7 8
24874 136/300 Write block 2217 len 12288 data: 3 3 3
24875 137/300 Write block 3981 len 12288 data: 3 4 4
24876 138/300 Write block 41 len 16384 data: 7 8 7 6
24877 139/300 Read block 498 len 36864
24878 140/300 Read block 2393 len 16384
24879 141/300 Flush
24880 142/300 Write block 1891 len 28672 data: 3 3 3 3 3 5 6
24881 143/300 Read block 4632 len 40960
24882 144/300 Write block 2774 len 32768 data: 2 2 3 4 4 5 7 6
24883 145/300 Flush
24884 146/300 Write block 4553 len 20480 data: 2 2 2 3 4
24885 147/300 Write block 489 len 36864 data: 6 4 3 2 2 2 2 2 3
24886 148/300 Write block 4209 len 28672 data: 5 4 4 3 2 4 5
24887 149/300 Read block 3022 len 24576
24888 150/300 Read block 4825 len 36864
24889 151/300 Read block 3374 len 12288
24890 152/300 Read block 4525 len 4096
24891 153/300 Flush
24892 154/300 Write block 3379 len 28672 data: 8 7 7 7 7 5 4
24893 155/300 Write block 4391 len 20480 data: 6 7 5 5 7
24894 156/300 Write block 2708 len 20480 data: 6 6 7 7 8
24895 157/300 Read block 487 len 12288
24896 158/300 Read block 137 len 12288
24897 159/300 Read block 3321 len 12288
24898 160/300 Read block 1107 len 28672
24899 161/300 Read block 1772 len 40960
24900 162/300 Write block 447 len 28672 data: 5 5 4 5 4 7 8
24901 163/300 Write block 1019 len 12288 data: 4 4 4
24902 164/300 Write block 2659 len 40960 data: 5 5 6 5 4 4 5 5 5 6
24903 165/300 Flush
24904 166/300 Write block 4940 len 36864 data: 4 5 6 4 5 5 4 3 4
24905 167/300 Read block 2125 len 8192
24906 168/300 Read block 3763 len 24576
24907 169/300 Write block 3206 len 28672 data: 6 5 4 4 3 3 4
24908 170/300 Read block 1520 len 8192
24909 171/300 Read block 4076 len 24576
24910 172/300 Read block 3141 len 24576
24911 173/300 Write block 344 len 16384 data: 5 6 7 7
24912 174/300 Read block 3321 len 20480
24913 175/300 Flush
24914 176/300 Write block 786 len 12288 data: 2 4 4
24915 177/300 Write block 2945 len 36864 data: 2 2 2 3 3 2 4 6 5
24916 178/300 Read block 450 len 36864
24917 179/300 Write block 2182 len 32768 data: 3 4 5 5 5 6 6 7
24918 180/300 Read block 3175 len 28672
24919 181/300 Read block 94 len 20480
24920 182/300 Flush
24921 183/300 Write block 3765 len 24576 data: 5 4 4 7 6 5
24922 184/300 Read block 4263 len 20480
24923 185/300 Write block 4103 len 40960 data: 6 5 5 4 5 6 5 6 5 4
24924 186/300 Write block 1289 len 36864 data: 4 4 5 6 6 6 5 5 4
24925 187/300 Read block 3014 len 36864
24926 188/300 Write block 981 len 12288 data: 3 4 5
24927 189/300 Read block 1487 len 16384
24928 190/300 Read block 2617 len 8192
24929 191/300 Write block 2946 len 24576 data: 3 3 4 4 3 5
24930 192/300 Read block 647 len 32768
24931 193/300 Write block 555 len 8192 data: 7 6
24932 194/300 Read block 1006 len 36864
24933 195/300 Read block 4660 len 16384
24934 196/300 Write block 2604 len 24576 data: 9 8 7 7 7 7
24935 197/300 Write block 4379 len 24576 data: 2 3 4 5 5 5
24936 198/300 Read block 4831 len 40960
24937 199/300 Write block 2624 len 36864 data: 7 6 5 5 8 9 8 7 7
24938 200/300 Read block 1287 len 12288
24939 201/300 Read block 387 len 36864
24940 202/300 Read block 1234 len 32768
24941 203/300 Read block 4575 len 4096
24942 204/300 Write block 4052 len 32768 data: 5 6 5 7 6 7 7 6
24943 205/300 Write block 1553 len 20480 data: 3 3 4 5 5
24944 206/300 Write block 2339 len 16384 data: 6 4 4 5
24945 207/300 Write block 1800 len 36864 data: 3 3 3 4 5 5 6 6 8
24946 208/300 Read block 4936 len 36864
24947 209/300 Write block 3832 len 12288 data: 6 6 6
24948 210/300 Read block 4372 len 32768
24949 211/300 Write block 4841 len 20480 data: 4 5 5 6 6
24950 212/300 Write block 2088 len 24576 data: 5 5 6 6 5 7
24951 213/300 Write block 521 len 4096 data: 5
24952 214/300 Flush
24953 215/300 Write block 555 len 40960 data: 8 7 6 5 2 2 3 3 3 3
24954 216/300 Write block 502 len 40960 data: 6 8 7 5 5 4 5 4 4 2
24955 217/300 Flush
24956 218/300 Write block 2026 len 12288 data: 7 9 8
24957 219/300 Write block 4889 len 16384 data: 2 2 3 4
24958 220/300 Flush
24959 221/300 Write block 1112 len 32768 data: 7 5 3 3 5 4 5 6
24960 222/300 Write block 4363 len 24576 data: 5 3 3 4 3 3
24961 223/300 Read block 3859 len 20480
24962 224/300 Write block 2505 len 32768 data: 3 3 3 3 3 3 3 3
24963 225/300 Flush
24964 226/300 Write block 3085 len 8192 data: 7 7
24965 227/300 Flush
24966 228/300 Read block 1605 len 28672
24967 229/300 Read block 1330 len 36864
24968 230/300 Flush
24969 231/300 Write block 88 len 8192 data: 4 4
24970 232/300 Read block 3769 len 40960
24971 233/300 Write block 933 len 32768 data: 7 6 6 7 7 7 6 5
24972 234/300 Read block 4978 len 32768
24973 235/300 Read block 4063 len 24576
24974 236/300 Read block 1359 len 16384
24975 237/300 Read block 3841 len 12288
24976 238/300 Read block 2494 len 12288
24977 239/300 Write block 3461 len 36864 data: 4 4 6 6 5 6 5 8 8
24978 240/300 Flush
24979 241/300 Write block 3857 len 12288 data: 8 8 6
24980 242/300 Write block 111 len 28672 data: 2 3 4 6 5 6 5
24981 243/300 Write block 3478 len 4096 data: 8
24982 244/300 Read block 4313 len 12288
24983 245/300 Flush
24984 246/300 Read block 3388 len 12288
24985 247/300 Write block 3496 len 40960 data: 4 5 4 5 6 6 6 7 7 6
24986 248/300 Flush
24987 249/300 Read block 1954 len 8192
24988 250/300 Read block 4624 len 12288
24989 251/300 Write block 3038 len 32768 data: 6 6 6 6 7 5 3 4
24990 252/300 Read block 1448 len 36864
24991 253/300 Write block 4276 len 8192 data: 2 2
24992 254/300 Write block 4093 len 36864 data: 5 5 5 5 4 6 7 7 6
24993 255/300 Read block 4619 len 4096
24994 256/300 Read block 838 len 20480
24995 257/300 Read block 3004 len 20480
24996 258/300 Read block 2082 len 8192
24997 259/300 Read block 2352 len 24576
24998 260/300 Write block 4423 len 16384 data: 4 5 5 5
24999 261/300 Read block 2465 len 28672
25000 262/300 Read block 2569 len 20480
25001 263/300 Read block 4269 len 28672
25002 264/300 Write block 2319 len 32768 data: 4 4 4 4 5 4 4 4
25003 265/300 Write block 251 len 8192 data: 7 6
25004 266/300 Read block 3955 len 28672
25005 267/300 Read block 4973 len 24576
25006 268/300 Read block 1027 len 36864
25007 269/300 Read block 263 len 16384
25008 270/300 Write block 3955 len 36864 data: 5 6 4 5 5 6 6 5 5
25009 271/300 Read block 4596 len 16384
25010 272/300 Read block 3729 len 20480
25011 273/300 Write block 1534 len 8192 data: 4 5
25012 274/300 Write block 4005 len 8192 data: 5 6
25013 275/300 Write block 1394 len 24576 data: 3 3 4 4 4 3
25014 276/300 Read block 4787 len 28672
25015 277/300 Read block 4630 len 16384
25016 278/300 Read block 3910 len 24576
25017 279/300 Read block 1165 len 36864
25018 280/300 Write block 3077 len 36864 data: 4 4 5 4 6 7 8 6 8
25019 281/300 Read block 2393 len 12288
25020 282/300 Write block 263 len 40960 data: 6 3 3 3 4 4 6 8 6 6
25021 283/300 Write block 1513 len 24576 data: 7 8 8 8 5 5
25022 284/300 Read block 197 len 32768
25023 285/300 Read block 3000 len 16384
25024 286/300 Read block 4260 len 8192
25025 287/300 Flush
25026 288/300 Write block 3183 len 20480 data: 3 4 4 4 4
25027 289/300 Read block 4380 len 16384
25028 290/300 Read block 1448 len 36864
25029 291/300 Read block 2566 len 28672
25030 292/300 Read block 205 len 28672
25031 293/300 Flush
25032 294/300 Write block 2255 len 4096 data: 5
25033 295/300 Flush
25034 296/300 Write block 4045 len 40960 data: 4 4 4 4 3 4 5 6 7 6
25035 297/300 Read block 2007 len 20480
25036 298/300 Read block 2664 len 36864
25037 299/300 Read block 1137 len 12288
25038 300/300 Write block 3685 len 24576 data: 4 3 2 2 3 3
25039 301/300 Write block 158 len 24576 data: 5 5 4 4 5 8
25040 Main task has work to do, go find it
25041 got dsc Start(2)
25042 start 2
25043 Jan 21 15:52:27.037 INFO request completed, latency_us: 291, response_code: 204, uri: /start/cid/2, method: POST, req_id: f22e1892-4068-4fea-bc97-69453dc7bb2a, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
25044 Replay: started 2, returned:Ok(())
25045 ----------------------------------------------------------------
25046 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25047 States: Active Active Offline
25048 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25049 7643 Acked Write 2 Done Done Sent false
25050 7644 Acked Flush 0 Done Done Sent false
25051 7645 Acked Write 6 Done Done Sent false
25052 7646 Acked Read 10 Done Done Sent false
25053 7647 Acked Read 10 Done Done Sent false
25054 7648 Acked Read 1 Done Done Sent false
25055 7649 Acked Write 6 Done Done Sent false
25056 7650 Acked Write 10 Done Done Sent false
25057 7651 Acked Write 5 Done Done Sent false
25058 7652 Acked Read 5 Done Done Sent false
25059 7653 Acked Read 10 Done Done Sent false
25060 7654 Acked Write 8 Done Done Sent false
25061 7655 Acked Read 10 Done Done Sent false
25062 7656 Acked Write 6 Done Done Sent false
25063 7657 Acked Read 3 Done Done Sent false
25064 7658 Acked Read 3 Done Done Sent false
25065 7659 Acked Read 8 Done Done Sent false
25066 7660 Acked Read 10 Done Done Sent false
25067 7661 Acked Read 6 Done Done Sent false
25068 7662 Acked Read 9 Done Done Sent false
25069 7663 Acked Write 6 Done Done Sent false
25070 7664 Acked Write 4 Done Done Sent false
25071 7665 Acked Read 2 Done Done Sent false
25072 7666 Acked Read 6 Done Done Sent false
25073 7667 Acked Read 2 Done Done Sent false
25074 7668 Acked Read 3 Done Done Sent false
25075 7669 Acked Read 1 Done Done Sent false
25076 7670 Acked Write 1 Done Done Sent false
25077 7671 Acked Flush 0 Done Done Sent false
25078 7672 Acked Write 7 Done Done Sent false
25079 7673 Acked Write 3 Done Done Sent false
25080 7674 Acked Read 10 Done Done Sent false
25081 7675 Acked Flush 0 Done Done Sent false
25082 7676 Acked Write 6 Done Done Sent false
25083 7677 Acked Read 3 Done Done Sent false
25084 7678 Acked Read 6 Done Done Sent false
25085 7679 Acked Read 8 Done Done Sent false
25086 7680 Acked Write 2 Done Done Sent false
25087 7681 Acked Read 8 Done Done Sent false
25088 7682 Acked Read 8 Done Done Sent false
25089 7683 Acked Read 1 Done Done Sent false
25090 7684 Acked Read 8 Done Done Sent false
25091 7685 Acked Read 9 Done Done Sent false
25092 7686 Acked Flush 0 Done Done Sent false
25093 7687 Acked Flush 0 Done Done Sent false
25094 7688 Acked Read 3 Done Done Sent false
25095 7689 Acked Write 3 Done Done Sent false
25096 7690 Acked Flush 0 Done Done Sent false
25097 7691 Acked Read 3 Done Done Sent false
25098 7692 Acked Write 10 Done Done Sent false
25099 7693 Acked Flush 0 Done Done Sent false
25100 7694 Acked Read 2 Done Done Sent false
25101 7695 Acked Read 9 Done Done Sent false
25102 7696 Acked Read 8 Done Done Sent false
25103 7697 Acked Write 10 Done Done Sent false
25104 7698 Acked Flush 0 Done Done Sent false
25105 7699 Acked Flush 0 Done Done Sent false
25106 7700 Acked Read 7 Done Done Sent false
25107 7701 Acked Read 3 Done Done Sent false
25108 7702 Acked Read 8 Done Done Sent false
25109 7703 Acked Write 5 Done Done Sent false
25110 7704 Acked Write 9 Done Done Sent false
25111 7705 Acked Write 7 Done Done Sent false
25112 7706 Acked Write 1 Done Done Sent false
25113 7707 Acked Read 1 Done Done Sent false
25114 7708 Acked Flush 0 Done Done Sent false
25115 7709 Acked Read 5 Done Done Sent false
25116 7710 Acked Write 3 Done Done Sent false
25117 7711 Acked Read 1 Done Done Sent false
25118 7712 Acked Write 10 Done Done Sent false
25119 7713 Acked Write 6 Done Done Sent false
25120 7714 Acked Write 8 Done Done Sent false
25121 7715 Acked Read 6 Done Done Sent false
25122 7716 Acked Write 5 Done Done Sent false
25123 7717 Acked Read 3 Done Done Sent false
25124 7718 Acked Write 10 Done Done Sent false
25125 7719 Acked Read 2 Done Done Sent false
25126 7720 Acked Read 10 Done Done Sent false
25127 7721 Acked Read 9 Done Done Sent false
25128 7722 Acked Read 10 Done Done Sent false
25129 7723 Acked Write 3 Done Done Sent false
25130 7724 Acked Flush 0 Done Done Sent false
25131 7725 Acked Read 10 Done Done Sent false
25132 7726 Acked Read 9 Done Done Sent false
25133 7727 Acked Read 5 Done Done Sent false
25134 7728 Acked Read 4 Done Done Sent false
25135 7729 Acked Write 10 Done Done Sent false
25136 7730 Acked Read 4 Done Done Sent false
25137 7731 Acked Read 5 Done Done Sent false
25138 7732 Acked Write 4 Done Done Sent false
25139 7733 Acked Read 1 Done Done Sent false
25140 7734 Acked Read 2 Done Done Sent false
25141 7735 Acked Read 7 Done Done Sent false
25142 7736 Acked Read 6 Done Done Sent false
25143 7737 Acked Write 6 Done Done Sent false
25144 7738 Acked Read 7 Done Done Sent false
25145 7739 Acked Write 1 Done Done Sent false
25146 7740 Acked Flush 0 Done Done Sent false
25147 7741 Acked Write 4 Done Done Sent false
25148 7742 Acked Write 9 Done Done Sent false
25149 7743 Acked Write 3 Done Done Sent false
25150 7744 Acked Write 9 Done Done Sent false
25151 7745 Acked Read 8 Done Done Sent false
25152 7746 Acked Flush 0 Done Done Sent false
25153 7747 Acked Read 6 Done Done Sent false
25154 7748 Acked Write 10 Done Done Sent false
25155 7749 Acked Write 6 Done Done Sent false
25156 7750 Acked Read 7 Done Done Sent false
25157 7751 Acked Write 5 Done Done Sent false
25158 7752 Acked Write 4 Done Done Sent false
25159 7753 Acked Read 4 Done Done Sent false
25160 7754 Acked Read 1 Done Done Sent false
25161 7755 Acked Read 5 Done Done Sent false
25162 7756 Acked Read 5 Done Done Sent false
25163 7757 Acked Write 10 Done Done Sent false
25164 7758 Acked Write 5 Done Done Sent false
25165 7759 Acked Flush 0 Done Done Sent false
25166 7760 Acked Read 8 Done Done Sent false
25167 7761 Acked Read 10 Done Done Sent false
25168 7762 Acked Write 2 Done Done Sent false
25169 7763 Acked Read 10 Done Done Sent false
25170 7764 Acked Read 4 Done Done Sent false
25171 7765 Acked Write 10 Done Done Sent false
25172 7766 Acked Flush 0 Done Done Sent false
25173 7767 Acked Read 3 Done Done Sent false
25174 7768 Acked Write 10 Done Done Sent false
25175 7769 Acked Read 10 Done Done Sent false
25176 7770 Acked Read 7 Done Done Sent false
25177 7771 Acked Read 8 Done Done Sent false
25178 7772 Acked Write 2 Done Done Sent false
25179 7773 Acked Read 2 Done Done Sent false
25180 7774 Acked Read 3 Done Done Sent false
25181 7775 Acked Read 10 Done Done Sent false
25182 7776 Acked Read 2 Done Done Sent false
25183 7777 Acked Write 3 Done Done Sent false
25184 7778 Acked Write 3 Done Done Sent false
25185 7779 Acked Write 3 Done Done Sent false
25186 7780 Acked Write 4 Done Done Sent false
25187 7781 Acked Read 9 Done Done Sent false
25188 7782 Acked Read 4 Done Done Sent false
25189 7783 Acked Flush 0 Done Done Sent false
25190 7784 Acked Write 7 Done Done Sent false
25191 7785 Acked Read 10 Done Done Sent false
25192 7786 Acked Write 8 Done Done Sent false
25193 7787 Acked Flush 0 Done Done Sent false
25194 7788 Acked Write 5 Done Done Sent false
25195 7789 Acked Write 9 Done Done Sent false
25196 7790 Acked Write 7 Done Done Sent false
25197 7791 Acked Read 6 Done Done Sent false
25198 7792 Acked Read 9 Done Done Sent false
25199 7793 Acked Read 3 Done Done Sent false
25200 7794 Acked Read 1 Done Done Sent false
25201 7795 Acked Flush 0 Done Done Sent false
25202 7796 Acked Write 7 Done Done Sent false
25203 7797 Acked Write 5 Done Done Sent false
25204 7798 Acked Write 5 Done Done Sent false
25205 7799 Acked Read 3 Done Done Sent false
25206 7800 Acked Read 3 Done Done Sent false
25207 7801 Acked Read 3 Done Done Sent false
25208 7802 Acked Read 7 Done Done Sent false
25209 7803 Acked Read 10 Done Done Sent false
25210 7804 Acked Write 7 Done Done Sent false
25211 7805 Acked Write 3 Done Done Sent false
25212 7806 Acked Write 10 Done Done Sent false
25213 7807 Acked Flush 0 Done Done Sent false
25214 7808 Acked Write 9 Done Done Sent false
25215 7809 Acked Read 2 Done Done Sent false
25216 7810 Acked Read 6 Done Done Sent false
25217 7811 Acked Write 7 Done Done Sent false
25218 7812 Acked Read 2 Done Done Sent false
25219 7813 Acked Read 6 Done Done Sent false
25220 7814 Acked Read 6 Done Done Sent false
25221 7815 Acked Write 4 Done Done Sent false
25222 7816 Acked Read 5 Done Done Sent false
25223 7817 Acked Flush 0 Done Done Sent false
25224 7818 Acked Write 3 Done Done Sent false
25225 7819 Acked Write 9 Done Done Sent false
25226 7820 Acked Read 9 Done Done Sent false
25227 7821 Acked Write 8 Done Done Sent false
25228 7822 Acked Read 7 Done Done Sent false
25229 7823 Acked Read 5 Done Done Sent false
25230 7824 Acked Flush 0 Done Done Sent false
25231 7825 Acked Write 6 Done Done Sent false
25232 7826 Acked Read 5 Done Done Sent false
25233 7827 Acked Write 10 Done Done Sent false
25234 7828 Acked Write 9 Done Done Sent false
25235 7829 Acked Read 9 Done Done Sent false
25236 7830 Acked Write 3 Done Done Sent false
25237 7831 Acked Read 4 Done Done Sent false
25238 7832 Acked Read 2 Done Done Sent false
25239 7833 Acked Write 6 Done Done Sent false
25240 7834 Acked Read 8 Done Done Sent false
25241 7835 Acked Write 2 Done Done Sent false
25242 7836 Acked Read 9 Done Done Sent false
25243 7837 Acked Read 4 Done Done Sent false
25244 7838 Acked Write 6 Done Done Sent false
25245 7839 Acked Write 6 Done Done Sent false
25246 7840 Acked Read 10 Done Done Sent false
25247 7841 Acked Write 9 Done Done Sent false
25248 7842 Acked Read 3 Done Done Sent false
25249 7843 Acked Read 9 Done Done Sent false
25250 7844 Acked Read 8 Done Done Sent false
25251 7845 Acked Read 1 Done Done Sent false
25252 7846 Acked Write 8 Done Done Sent false
25253 7847 Acked Write 5 Done Done Sent false
25254 7848 Acked Write 4 Done Done Sent false
25255 7849 Acked Write 9 Done Done Sent false
25256 7850 Acked Read 9 Done Done Sent false
25257 7851 Acked Write 3 Done Done Sent false
25258 7852 Acked Read 8 Done Done Sent false
25259 7853 Acked Write 5 Done Done Sent false
25260 7854 Acked Write 6 Done Done Sent false
25261 7855 Acked Write 1 Done Done Sent false
25262 7856 Acked Flush 0 Done Done Sent false
25263 7857 Acked Write 10 Done Done Sent false
25264 7858 Acked Write 10 Done Done Sent false
25265 7859 Acked Flush 0 Done Done Sent false
25266 7860 Acked Write 3 Done Done Sent false
25267 7861 Acked Write 4 Done Done Sent false
25268 7862 Acked Flush 0 Done Done Sent false
25269 7863 Acked Write 8 Done Done Sent false
25270 7864 Acked Write 6 Done Done Sent false
25271 7865 Acked Read 5 Done Done Sent false
25272 7866 Acked Write 8 Done Done Sent false
25273 7867 Acked Flush 0 Done Done Sent false
25274 7868 Acked Write 2 Done Done Sent false
25275 7869 Acked Flush 0 Done Done Sent false
25276 7870 Acked Read 7 Done Done Sent false
25277 7871 Acked Read 9 Done Done Sent false
25278 7872 Acked Flush 0 Done Done Sent false
25279 7873 Acked Write 2 Done Done Sent false
25280 7874 Acked Read 10 Done Done Sent false
25281 7875 Acked Write 8 Done Done Sent false
25282 7876 Acked Read 8 Done Done Sent false
25283 7877 Acked Read 6 Done Done Sent false
25284 7878 Acked Read 4 Done Done Sent false
25285 7879 Acked Read 3 Done Done Sent false
25286 7880 Acked Read 3 Done Done Sent false
25287 7881 Acked Write 9 Done Done Sent false
25288 7882 Acked Flush 0 Done Done Sent false
25289 7883 Acked Write 3 Done Done Sent false
25290 7884 Acked Write 7 Done Done Sent false
25291 7885 Acked Write 1 Done Done Sent false
25292 7886 Acked Read 3 Done Done Sent false
25293 7887 Acked Flush 0 Done Done Sent false
25294 7888 Acked Read 3 Done Done Sent false
25295 7889 Acked Write 10 Done Done Sent false
25296 7890 Acked Flush 0 Done Done Sent false
25297 7891 Acked Read 2 Done Done Sent false
25298 7892 Acked Read 3 Done Done Sent false
25299 7893 Acked Write 8 Done Done Sent false
25300 7894 Acked Read 9 Done Done Sent false
25301 7895 Acked Write 2 Done Done Sent false
25302 7896 Acked Write 9 Done Done Sent false
25303 7897 Acked Read 1 Done Done Sent false
25304 7898 Acked Read 5 Done Done Sent false
25305 7899 Acked Read 5 Done Done Sent false
25306 7900 Acked Read 2 Done Done Sent false
25307 7901 Acked Read 6 Done Done Sent false
25308 7902 Acked Write 4 Done Done Sent false
25309 7903 Acked Read 7 Done Done Sent false
25310 7904 Acked Read 5 Done Done Sent false
25311 7905 Acked Read 7 Done Done Sent false
25312 7906 Acked Write 8 Done Done Sent false
25313 7907 Acked Write 2 Done Done Sent false
25314 7908 Acked Read 7 Done Done Sent false
25315 7909 Acked Read 6 Done Done Sent false
25316 7910 Acked Read 9 Done Done Sent false
25317 7911 Acked Read 4 Done Done Sent false
25318 7912 Acked Write 9 Done Done Sent false
25319 7913 Acked Read 4 Done Done Sent false
25320 7914 Acked Read 5 Done Done Sent false
25321 7915 Acked Write 2 Done Done Sent false
25322 7916 Acked Write 2 Done Done Sent false
25323 7917 Acked Write 6 Done Done Sent false
25324 7918 Acked Read 7 Done Done Sent false
25325 7919 Acked Read 4 Done Done Sent false
25326 7920 Acked Read 6 Done Done Sent false
25327 7921 Acked Read 9 Done Done Sent false
25328 7922 Acked Write 9 Done Done Sent false
25329 7923 Acked Read 3 Done Done Sent false
25330 7924 Acked Write 10 Done Done Sent false
25331 7925 Acked Write 6 Done Done Sent false
25332 7926 Acked Read 8 Done Done Sent false
25333 7927 Acked Read 4 Done Done Sent false
25334 7928 Acked Read 2 Done Done Sent false
25335 7929 Acked Flush 0 Done Done Sent false
25336 7930 Acked Write 5 Done Done Sent false
25337 7931 Acked Read 4 Done Done Sent false
25338 7932 Acked Read 9 Done Done Sent false
25339 7933 Acked Read 7 Done Done Sent false
25340 7934 Acked Read 7 Done Done Sent false
25341 7935 Acked Flush 0 Done Done Sent false
25342 7936 Acked Write 1 Done Done Sent false
25343 7937 Acked Flush 0 Done Done Sent false
25344 7938 Acked Write 10 Done Done Sent false
25345 7939 Acked Read 5 Done Done Sent false
25346 7940 Acked Read 9 Done Done Sent false
25347 7941 Acked Read 3 Done Done Sent false
25348 7942 Acked Write 6 Done Done Sent false
25349 7943 Acked Write 6 Done Done Sent false
25350 STATES DS:0 DS:1 DS:2 TOTAL
25351 Sent 0 0 301 301
25352 Done 301 301 0 602
25353 Skipped 0 0 0 0
25354 Error 0 0 0 0
25355 Last Flush: 7937 7937 7642
25356 Downstairs last five completed: 7642 7641 7640 7639 7638
25357 Upstairs last five completed: 7943 7942 7941 7940 7939
25358 CLIENT: Up:0 ds:301 act:2
25359 [8830] got start action
25360 [8830] I am going to restart
25361 Starting downstairs at port 8830
25362 Make output file at "/tmp/dsc/downstairs-8830.txt"
25363 [8830][2] reports Starting
25364 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1223)
25365 [8830][2] reports Running
25366 ----------------------------------------------------------------
25367 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25368 States: Active Active Offline
25369 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25370 7643 Acked Write 2 Done Done Sent false
25371 7644 Acked Flush 0 Done Done Sent false
25372 7645 Acked Write 6 Done Done Sent false
25373 7646 Acked Read 10 Done Done Sent false
25374 7647 Acked Read 10 Done Done Sent false
25375 7648 Acked Read 1 Done Done Sent false
25376 7649 Acked Write 6 Done Done Sent false
25377 7650 Acked Write 10 Done Done Sent false
25378 7651 Acked Write 5 Done Done Sent false
25379 7652 Acked Read 5 Done Done Sent false
25380 7653 Acked Read 10 Done Done Sent false
25381 7654 Acked Write 8 Done Done Sent false
25382 7655 Acked Read 10 Done Done Sent false
25383 7656 Acked Write 6 Done Done Sent false
25384 7657 Acked Read 3 Done Done Sent false
25385 7658 Acked Read 3 Done Done Sent false
25386 7659 Acked Read 8 Done Done Sent false
25387 7660 Acked Read 10 Done Done Sent false
25388 7661 Acked Read 6 Done Done Sent false
25389 7662 Acked Read 9 Done Done Sent false
25390 7663 Acked Write 6 Done Done Sent false
25391 7664 Acked Write 4 Done Done Sent false
25392 7665 Acked Read 2 Done Done Sent false
25393 7666 Acked Read 6 Done Done Sent false
25394 7667 Acked Read 2 Done Done Sent false
25395 7668 Acked Read 3 Done Done Sent false
25396 7669 Acked Read 1 Done Done Sent false
25397 7670 Acked Write 1 Done Done Sent false
25398 7671 Acked Flush 0 Done Done Sent false
25399 7672 Acked Write 7 Done Done Sent false
25400 7673 Acked Write 3 Done Done Sent false
25401 7674 Acked Read 10 Done Done Sent false
25402 7675 Acked Flush 0 Done Done Sent false
25403 7676 Acked Write 6 Done Done Sent false
25404 7677 Acked Read 3 Done Done Sent false
25405 7678 Acked Read 6 Done Done Sent false
25406 7679 Acked Read 8 Done Done Sent false
25407 7680 Acked Write 2 Done Done Sent false
25408 7681 Acked Read 8 Done Done Sent false
25409 7682 Acked Read 8 Done Done Sent false
25410 7683 Acked Read 1 Done Done Sent false
25411 7684 Acked Read 8 Done Done Sent false
25412 7685 Acked Read 9 Done Done Sent false
25413 7686 Acked Flush 0 Done Done Sent false
25414 7687 Acked Flush 0 Done Done Sent false
25415 7688 Acked Read 3 Done Done Sent false
25416 7689 Acked Write 3 Done Done Sent false
25417 7690 Acked Flush 0 Done Done Sent false
25418 7691 Acked Read 3 Done Done Sent false
25419 7692 Acked Write 10 Done Done Sent false
25420 7693 Acked Flush 0 Done Done Sent false
25421 7694 Acked Read 2 Done Done Sent false
25422 7695 Acked Read 9 Done Done Sent false
25423 7696 Acked Read 8 Done Done Sent false
25424 7697 Acked Write 10 Done Done Sent false
25425 7698 Acked Flush 0 Done Done Sent false
25426 7699 Acked Flush 0 Done Done Sent false
25427 7700 Acked Read 7 Done Done Sent false
25428 7701 Acked Read 3 Done Done Sent false
25429 7702 Acked Read 8 Done Done Sent false
25430 7703 Acked Write 5 Done Done Sent false
25431 7704 Acked Write 9 Done Done Sent false
25432 7705 Acked Write 7 Done Done Sent false
25433 7706 Acked Write 1 Done Done Sent false
25434 7707 Acked Read 1 Done Done Sent false
25435 7708 Acked Flush 0 Done Done Sent false
25436 7709 Acked Read 5 Done Done Sent false
25437 7710 Acked Write 3 Done Done Sent false
25438 7711 Acked Read 1 Done Done Sent false
25439 7712 Acked Write 10 Done Done Sent false
25440 7713 Acked Write 6 Done Done Sent false
25441 7714 Acked Write 8 Done Done Sent false
25442 7715 Acked Read 6 Done Done Sent false
25443 7716 Acked Write 5 Done Done Sent false
25444 7717 Acked Read 3 Done Done Sent false
25445 7718 Acked Write 10 Done Done Sent false
25446 7719 Acked Read 2 Done Done Sent false
25447 7720 Acked Read 10 Done Done Sent false
25448 7721 Acked Read 9 Done Done Sent false
25449 7722 Acked Read 10 Done Done Sent false
25450 7723 Acked Write 3 Done Done Sent false
25451 7724 Acked Flush 0 Done Done Sent false
25452 7725 Acked Read 10 Done Done Sent false
25453 7726 Acked Read 9 Done Done Sent false
25454 7727 Acked Read 5 Done Done Sent false
25455 7728 Acked Read 4 Done Done Sent false
25456 7729 Acked Write 10 Done Done Sent false
25457 7730 Acked Read 4 Done Done Sent false
25458 7731 Acked Read 5 Done Done Sent false
25459 7732 Acked Write 4 Done Done Sent false
25460 7733 Acked Read 1 Done Done Sent false
25461 7734 Acked Read 2 Done Done Sent false
25462 7735 Acked Read 7 Done Done Sent false
25463 7736 Acked Read 6 Done Done Sent false
25464 7737 Acked Write 6 Done Done Sent false
25465 7738 Acked Read 7 Done Done Sent false
25466 7739 Acked Write 1 Done Done Sent false
25467 7740 Acked Flush 0 Done Done Sent false
25468 7741 Acked Write 4 Done Done Sent false
25469 7742 Acked Write 9 Done Done Sent false
25470 7743 Acked Write 3 Done Done Sent false
25471 7744 Acked Write 9 Done Done Sent false
25472 7745 Acked Read 8 Done Done Sent false
25473 7746 Acked Flush 0 Done Done Sent false
25474 7747 Acked Read 6 Done Done Sent false
25475 7748 Acked Write 10 Done Done Sent false
25476 7749 Acked Write 6 Done Done Sent false
25477 7750 Acked Read 7 Done Done Sent false
25478 7751 Acked Write 5 Done Done Sent false
25479 7752 Acked Write 4 Done Done Sent false
25480 7753 Acked Read 4 Done Done Sent false
25481 7754 Acked Read 1 Done Done Sent false
25482 7755 Acked Read 5 Done Done Sent false
25483 7756 Acked Read 5 Done Done Sent false
25484 7757 Acked Write 10 Done Done Sent false
25485 7758 Acked Write 5 Done Done Sent false
25486 7759 Acked Flush 0 Done Done Sent false
25487 7760 Acked Read 8 Done Done Sent false
25488 7761 Acked Read 10 Done Done Sent false
25489 7762 Acked Write 2 Done Done Sent false
25490 7763 Acked Read 10 Done Done Sent false
25491 7764 Acked Read 4 Done Done Sent false
25492 7765 Acked Write 10 Done Done Sent false
25493 7766 Acked Flush 0 Done Done Sent false
25494 7767 Acked Read 3 Done Done Sent false
25495 7768 Acked Write 10 Done Done Sent false
25496 7769 Acked Read 10 Done Done Sent false
25497 7770 Acked Read 7 Done Done Sent false
25498 7771 Acked Read 8 Done Done Sent false
25499 7772 Acked Write 2 Done Done Sent false
25500 7773 Acked Read 2 Done Done Sent false
25501 7774 Acked Read 3 Done Done Sent false
25502 7775 Acked Read 10 Done Done Sent false
25503 7776 Acked Read 2 Done Done Sent false
25504 7777 Acked Write 3 Done Done Sent false
25505 7778 Acked Write 3 Done Done Sent false
25506 7779 Acked Write 3 Done Done Sent false
25507 7780 Acked Write 4 Done Done Sent false
25508 7781 Acked Read 9 Done Done Sent false
25509 7782 Acked Read 4 Done Done Sent false
25510 7783 Acked Flush 0 Done Done Sent false
25511 7784 Acked Write 7 Done Done Sent false
25512 7785 Acked Read 10 Done Done Sent false
25513 7786 Acked Write 8 Done Done Sent false
25514 7787 Acked Flush 0 Done Done Sent false
25515 7788 Acked Write 5 Done Done Sent false
25516 7789 Acked Write 9 Done Done Sent false
25517 7790 Acked Write 7 Done Done Sent false
25518 7791 Acked Read 6 Done Done Sent false
25519 7792 Acked Read 9 Done Done Sent false
25520 7793 Acked Read 3 Done Done Sent false
25521 7794 Acked Read 1 Done Done Sent false
25522 7795 Acked Flush 0 Done Done Sent false
25523 7796 Acked Write 7 Done Done Sent false
25524 7797 Acked Write 5 Done Done Sent false
25525 7798 Acked Write 5 Done Done Sent false
25526 7799 Acked Read 3 Done Done Sent false
25527 7800 Acked Read 3 Done Done Sent false
25528 7801 Acked Read 3 Done Done Sent false
25529 7802 Acked Read 7 Done Done Sent false
25530 7803 Acked Read 10 Done Done Sent false
25531 7804 Acked Write 7 Done Done Sent false
25532 7805 Acked Write 3 Done Done Sent false
25533 7806 Acked Write 10 Done Done Sent false
25534 7807 Acked Flush 0 Done Done Sent false
25535 7808 Acked Write 9 Done Done Sent false
25536 7809 Acked Read 2 Done Done Sent false
25537 7810 Acked Read 6 Done Done Sent false
25538 7811 Acked Write 7 Done Done Sent false
25539 7812 Acked Read 2 Done Done Sent false
25540 7813 Acked Read 6 Done Done Sent false
25541 7814 Acked Read 6 Done Done Sent false
25542 7815 Acked Write 4 Done Done Sent false
25543 7816 Acked Read 5 Done Done Sent false
25544 7817 Acked Flush 0 Done Done Sent false
25545 7818 Acked Write 3 Done Done Sent false
25546 7819 Acked Write 9 Done Done Sent false
25547 7820 Acked Read 9 Done Done Sent false
25548 7821 Acked Write 8 Done Done Sent false
25549 7822 Acked Read 7 Done Done Sent false
25550 7823 Acked Read 5 Done Done Sent false
25551 7824 Acked Flush 0 Done Done Sent false
25552 7825 Acked Write 6 Done Done Sent false
25553 7826 Acked Read 5 Done Done Sent false
25554 7827 Acked Write 10 Done Done Sent false
25555 7828 Acked Write 9 Done Done Sent false
25556 7829 Acked Read 9 Done Done Sent false
25557 7830 Acked Write 3 Done Done Sent false
25558 7831 Acked Read 4 Done Done Sent false
25559 7832 Acked Read 2 Done Done Sent false
25560 7833 Acked Write 6 Done Done Sent false
25561 7834 Acked Read 8 Done Done Sent false
25562 7835 Acked Write 2 Done Done Sent false
25563 7836 Acked Read 9 Done Done Sent false
25564 7837 Acked Read 4 Done Done Sent false
25565 7838 Acked Write 6 Done Done Sent false
25566 7839 Acked Write 6 Done Done Sent false
25567 7840 Acked Read 10 Done Done Sent false
25568 7841 Acked Write 9 Done Done Sent false
25569 7842 Acked Read 3 Done Done Sent false
25570 7843 Acked Read 9 Done Done Sent false
25571 7844 Acked Read 8 Done Done Sent false
25572 7845 Acked Read 1 Done Done Sent false
25573 7846 Acked Write 8 Done Done Sent false
25574 7847 Acked Write 5 Done Done Sent false
25575 7848 Acked Write 4 Done Done Sent false
25576 7849 Acked Write 9 Done Done Sent false
25577 7850 Acked Read 9 Done Done Sent false
25578 7851 Acked Write 3 Done Done Sent false
25579 7852 Acked Read 8 Done Done Sent false
25580 7853 Acked Write 5 Done Done Sent false
25581 7854 Acked Write 6 Done Done Sent false
25582 7855 Acked Write 1 Done Done Sent false
25583 7856 Acked Flush 0 Done Done Sent false
25584 7857 Acked Write 10 Done Done Sent false
25585 7858 Acked Write 10 Done Done Sent false
25586 7859 Acked Flush 0 Done Done Sent false
25587 7860 Acked Write 3 Done Done Sent false
25588 7861 Acked Write 4 Done Done Sent false
25589 7862 Acked Flush 0 Done Done Sent false
25590 7863 Acked Write 8 Done Done Sent false
25591 7864 Acked Write 6 Done Done Sent false
25592 7865 Acked Read 5 Done Done Sent false
25593 7866 Acked Write 8 Done Done Sent false
25594 7867 Acked Flush 0 Done Done Sent false
25595 7868 Acked Write 2 Done Done Sent false
25596 7869 Acked Flush 0 Done Done Sent false
25597 7870 Acked Read 7 Done Done Sent false
25598 7871 Acked Read 9 Done Done Sent false
25599 7872 Acked Flush 0 Done Done Sent false
25600 7873 Acked Write 2 Done Done Sent false
25601 7874 Acked Read 10 Done Done Sent false
25602 7875 Acked Write 8 Done Done Sent false
25603 7876 Acked Read 8 Done Done Sent false
25604 7877 Acked Read 6 Done Done Sent false
25605 7878 Acked Read 4 Done Done Sent false
25606 7879 Acked Read 3 Done Done Sent false
25607 7880 Acked Read 3 Done Done Sent false
25608 7881 Acked Write 9 Done Done Sent false
25609 7882 Acked Flush 0 Done Done Sent false
25610 7883 Acked Write 3 Done Done Sent false
25611 7884 Acked Write 7 Done Done Sent false
25612 7885 Acked Write 1 Done Done Sent false
25613 7886 Acked Read 3 Done Done Sent false
25614 7887 Acked Flush 0 Done Done Sent false
25615 7888 Acked Read 3 Done Done Sent false
25616 7889 Acked Write 10 Done Done Sent false
25617 7890 Acked Flush 0 Done Done Sent false
25618 7891 Acked Read 2 Done Done Sent false
25619 7892 Acked Read 3 Done Done Sent false
25620 7893 Acked Write 8 Done Done Sent false
25621 7894 Acked Read 9 Done Done Sent false
25622 7895 Acked Write 2 Done Done Sent false
25623 7896 Acked Write 9 Done Done Sent false
25624 7897 Acked Read 1 Done Done Sent false
25625 7898 Acked Read 5 Done Done Sent false
25626 7899 Acked Read 5 Done Done Sent false
25627 7900 Acked Read 2 Done Done Sent false
25628 7901 Acked Read 6 Done Done Sent false
25629 7902 Acked Write 4 Done Done Sent false
25630 7903 Acked Read 7 Done Done Sent false
25631 7904 Acked Read 5 Done Done Sent false
25632 7905 Acked Read 7 Done Done Sent false
25633 7906 Acked Write 8 Done Done Sent false
25634 7907 Acked Write 2 Done Done Sent false
25635 7908 Acked Read 7 Done Done Sent false
25636 7909 Acked Read 6 Done Done Sent false
25637 7910 Acked Read 9 Done Done Sent false
25638 7911 Acked Read 4 Done Done Sent false
25639 7912 Acked Write 9 Done Done Sent false
25640 7913 Acked Read 4 Done Done Sent false
25641 7914 Acked Read 5 Done Done Sent false
25642 7915 Acked Write 2 Done Done Sent false
25643 7916 Acked Write 2 Done Done Sent false
25644 7917 Acked Write 6 Done Done Sent false
25645 7918 Acked Read 7 Done Done Sent false
25646 7919 Acked Read 4 Done Done Sent false
25647 7920 Acked Read 6 Done Done Sent false
25648 7921 Acked Read 9 Done Done Sent false
25649 7922 Acked Write 9 Done Done Sent false
25650 7923 Acked Read 3 Done Done Sent false
25651 7924 Acked Write 10 Done Done Sent false
25652 7925 Acked Write 6 Done Done Sent false
25653 7926 Acked Read 8 Done Done Sent false
25654 7927 Acked Read 4 Done Done Sent false
25655 7928 Acked Read 2 Done Done Sent false
25656 7929 Acked Flush 0 Done Done Sent false
25657 7930 Acked Write 5 Done Done Sent false
25658 7931 Acked Read 4 Done Done Sent false
25659 7932 Acked Read 9 Done Done Sent false
25660 7933 Acked Read 7 Done Done Sent false
25661 7934 Acked Read 7 Done Done Sent false
25662 7935 Acked Flush 0 Done Done Sent false
25663 7936 Acked Write 1 Done Done Sent false
25664 7937 Acked Flush 0 Done Done Sent false
25665 7938 Acked Write 10 Done Done Sent false
25666 7939 Acked Read 5 Done Done Sent false
25667 7940 Acked Read 9 Done Done Sent false
25668 7941 Acked Read 3 Done Done Sent false
25669 7942 Acked Write 6 Done Done Sent false
25670 7943 Acked Write 6 Done Done Sent false
25671 7944 Acked Flush 0 Done Done Sent false
25672 STATES DS:0 DS:1 DS:2 TOTAL
25673 Sent 0 0 302 302
25674 Done 302 302 0 604
25675 Skipped 0 0 0 0
25676 Error 0 0 0 0
25677 Last Flush: 7944 7944 7642
25678 Downstairs last five completed: 7642 7641 7640 7639 7638
25679 Upstairs last five completed: 7944 7943 7942 7941 7940
25680 CLIENT: Up:0 ds:302 act:2
256812025-01-21T15:52:32.787ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
256822025-01-21T15:52:32.787ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50750) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
256832025-01-21T15:52:32.788ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
256842025-01-21T15:52:32.789ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
256852025-01-21T15:52:32.789ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
256862025-01-21T15:52:32.789ZINFOcrucible: send last flush ID to this DS: 7642 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
256872025-01-21T15:52:32.789ZINFOcrucible: Replied this last flush ID: 7642 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
256882025-01-21T15:52:32.789ZINFOcrucible: [2] client re-new 302 jobs since flush 7642 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
256892025-01-21T15:52:32.789ZINFOcrucible: [2] Replayed 302 jobs since flush: 7642 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
25690 ----------------------------------------------------------------
25691 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25692 Downstairs last five completed: 7944 7943 7942 7941 7940
25693 Upstairs last five completed: 7944 7943 7942 7941 7940
25694 CLIENT: Up:0 ds:0 act:3
25695 Replay: All jobs finished
25696 Main task has work to do, go find it
25697 got dsc Stop(1)
25698 stop 1
25699 Jan 21 15:52:35.042 INFO request completed, latency_us: [8820] Got stop action so:false kr:false
25700 364, response_code: 204, uri: /stop/cid/1, method: POST, req_id: cd2f4e4f-e804-496d-a7db-ced9810b6351, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
257012025-01-21T15:52:35.042ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
257022025-01-21T15:52:35.043ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
257032025-01-21T15:52:35.043ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
25704 Jan 21 15:52:35.043 INFO {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:52:35.043217178Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
25705 request completed, latency_us: 181, response_code: 200, uri: /state/cid/1, method: GET, req_id: 75b0a164-3e9f-4deb-b469-ec78eeb6da8a, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
25706 [8820] Exited with: ExitStatus(unix_wait_status(9))
25707 [8820][1] reports Exit
25708 Jan 21 15:52:39.045 INFO request completed, latency_us: 328, response_code: 200, uri: /state/cid/1, method: GET, req_id: f8f3d68c-9d74-436c-9436-43b312597b9d, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
25709 001/300 Flush
25710 002/300 Write block 3702 len 12288 data: 10 10 9
25711 003/300 Read block 69 len 8192
25712 004/300 Write block 4184 len 40960 data: 6 8 8 7 6 7 6 5 4 2
25713 005/300 Flush
25714 006/300 Read block 3241 len 28672
25715 007/300 Read block 162 len 36864
25716 008/300 Read block 911 len 12288
25717 009/300 Read block 3719 len 28672
25718 010/300 Read block 280 len 40960
25719 011/300 Read block 3860 len 8192
25720 012/300 Flush
25721 013/300 Read block 4398 len 16384
25722 014/300 Read block 2350 len 40960
25723 015/300 Read block 2964 len 12288
25724 016/300 Read block 3356 len 32768
25725 017/300 Read block 936 len 12288
25726 018/300 Read block 1312 len 36864
25727 019/300 Write block 2100 len 8192 data: 4 4
25728 020/300 Read block 2031 len 36864
25729 021/300 Read block 2335 len 8192
25730 022/300 Write block 2342 len 24576 data: 6 4 7 6 7 8
25731 023/300 Read block 3591 len 12288
25732 024/300 Read block 3913 len 8192
25733 025/300 Read block 1935 len 28672
25734 026/300 Write block 3142 len 40960 data: 4 3 3 4 4 4 5 6 6 6
25735 027/300 Flush
25736 028/300 Read block 3109 len 16384
25737 029/300 Write block 1110 len 4096 data: 7
25738 030/300 Write block 3970 len 36864 data: 3 3 3 4 4 6 5 6 6
25739 031/300 Write block 1594 len 36864 data: 4 3 2 2 2 2 2 2 2
25740 032/300 Write block 557 len 24576 data: 7 6 3 3 4 4
25741 033/300 Write block 2337 len 40960 data: 5 5 7 5 5 7 5 8 7 8
25742 034/300 Flush
25743 035/300 Write block 3976 len 4096 data: 6
25744 036/300 Write block 2497 len 24576 data: 4 5 5 3 4 3
25745 037/300 Flush
25746 038/300 Write block 4420 len 36864 data: 3 4 5 5 6 6 6 3 4
25747 039/300 Read block 4370 len 4096
25748 040/300 Read block 3103 len 36864
25749 041/300 Flush
25750 042/300 Read block 4542 len 28672
25751 043/300 Write block 914 len 16384 data: 7 9 9 9
25752 044/300 Read block 4785 len 4096
25753 045/300 Write block 4746 len 20480 data: 4 4 4 4 5
25754 046/300 Read block 1479 len 24576
25755 047/300 Read block 1472 len 8192
25756 048/300 Read block 3731 len 8192
25757 049/300 Read block 668 len 12288
25758 050/300 Write block 4150 len 16384 data: 4 5 6 5
25759 051/300 Read block 2208 len 40960
25760 052/300 Write block 4050 len 4096 data: 5
25761 053/300 Read block 3747 len 28672
25762 054/300 Write block 1187 len 20480 data: 6 6 6 6 8
25763 055/300 Write block 406 len 32768 data: 8 9 8 6 7 7 6 5
25764 056/300 Read block 1041 len 36864
25765 057/300 Write block 4635 len 24576 data: 4 4 5 5 4 3
25766 058/300 Write block 2220 len 4096 data: 3
25767 059/300 Write block 3452 len 28672 data: 6 6 6 6 6 3 3
25768 060/300 Read block 4205 len 40960
25769 061/300 Write block 2227 len 16384 data: 7 7 6 5
25770 062/300 Write block 2389 len 40960 data: 5 7 6 7 6 6 6 6 4 2
25771 063/300 Flush
25772 064/300 Write block 4417 len 28672 data: 4 4 3 4 5 6 6
25773 065/300 Write block 189 len 28672 data: 7 7 7 8 8 8 6
25774 066/300 Read block 3896 len 20480
25775 067/300 Write block 3314 len 12288 data: 3 3 3
25776 068/300 Read block 1487 len 12288
25777 069/300 Write block 4088 len 28672 data: 6 6 6 6 6 6 6
25778 070/300 Write block 2883 len 16384 data: 6 6 7 8
25779 071/300 Read block 1112 len 4096
25780 072/300 Read block 4751 len 40960
25781 073/300 Read block 930 len 16384
25782 074/300 Write block 1532 len 12288 data: 5 5 5
25783 075/300 Read block 254 len 36864
25784 076/300 Write block 3753 len 32768 data: 3 3 3 4 5 5 6 4
25785 077/300 Write block 1913 len 16384 data: 8 7 7 7
25786 078/300 Read block 4439 len 36864
25787 079/300 Read block 4125 len 4096
25788 080/300 Write block 1442 len 8192 data: 4 4
25789 081/300 Read block 2181 len 4096
25790 082/300 Write block 3700 len 32768 data: 6 8 11 11 10 7 5 7
25791 083/300 Read block 3118 len 40960
25792 084/300 Flush
25793 085/300 Write block 7 len 36864 data: 3 3 3 4 3 3 3 4 6
25794 086/300 Read block 4658 len 28672
25795 087/300 Read block 2958 len 28672
25796 088/300 Write block 3863 len 20480 data: 3 4 4 5 4
25797 089/300 Write block 3930 len 8192 data: 6 5
25798 090/300 Flush
25799 091/300 Write block 2174 len 40960 data: 6 4 3 3 4 4 3 3 4 5
25800 092/300 Read block 2074 len 16384
25801 093/300 Write block 4892 len 28672 data: 5 4 4 4 5 6 5
25802 094/300 Write block 3536 len 32768 data: 5 5 5 5 6 6 5 6
25803 095/300 Write block 701 len 8192 data: 5 5
25804 096/300 Write block 2232 len 32768 data: 4 3 3 3 6 6 6 5
25805 097/300 Write block 4722 len 36864 data: 9 8 7 5 6 6 4 3 4
25806 098/300 Read block 2605 len 4096
25807 099/300 Flush
25808 100/300 Write block 358 len 20480 data: 6 5 6 7 6
25809 101/300 Write block 689 len 4096 data: 6
25810 102/300 Read block 3185 len 32768
25811 103/300 Write block 4344 len 36864 data: 5 5 5 6 7 7 8 7 6
25812 104/300 Read block 3209 len 20480
25813 105/300 Write block 292 len 40960 data: 4 4 4 3 4 4 5 5 5 7
25814 106/300 Write block 4337 len 32768 data: 2 2 2 4 4 4 7 6
25815 107/300 Write block 4153 len 20480 data: 6 4 5 3 3
25816 108/300 Read block 1324 len 40960
25817 109/300 Read block 3660 len 20480
25818 110/300 Flush
25819 111/300 Read block 1577 len 12288
25820 112/300 Read block 1895 len 16384
25821 113/300 Write block 716 len 40960 data: 7 8 8 8 7 4 6 5 5 5
25822 114/300 Write block 2836 len 24576 data: 3 4 4 7 5 6
25823 115/300 Write block 3370 len 16384 data: 6 6 5 4
25824 116/300 Flush
25825 117/300 Read block 4658 len 16384
25826 118/300 Write block 2220 len 20480 data: 4 2 2 3 5
25827 119/300 Write block 3557 len 12288 data: 6 7 6
25828 120/300 Flush
25829 121/300 Write block 2400 len 32768 data: 2 2 2 2 3 3 3 3
25830 122/300 Read block 2079 len 40960
25831 123/300 Read block 3863 len 36864
25832 124/300 Read block 4836 len 40960
25833 125/300 Read block 2701 len 12288
25834 126/300 Write block 1161 len 16384 data: 8 5 4 3
25835 127/300 Read block 4974 len 28672
25836 128/300 Write block 2687 len 16384 data: 3 3 3 2
25837 129/300 Read block 377 len 32768
25838 130/300 Write block 313 len 28672 data: 4 4 5 4 3 4 3
25839 131/300 Write block 4193 len 4096 data: 3
25840 132/300 Flush
25841 133/300 Write block 1374 len 20480 data: 6 3 3 2 3
25842 134/300 Read block 2239 len 4096
25843 135/300 Read block 2978 len 12288
25844 136/300 Read block 2495 len 28672
25845 137/300 Write block 807 len 40960 data: 9 7 8 8 9 6 5 5 6 4
25846 138/300 Flush
25847 139/300 Read block 4460 len 8192
25848 140/300 Read block 3405 len 4096
25849 141/300 Write block 2240 len 24576 data: 5 4 5 4 5 5
25850 142/300 Read block 3129 len 8192
25851 143/300 Read block 2653 len 28672
25852 144/300 Read block 2243 len 32768
25853 145/300 Read block 4639 len 8192
25854 146/300 Flush
25855 147/300 Write block 3425 len 8192 data: 4 2
25856 148/300 Write block 689 len 40960 data: 7 6 4 4 4 5 7 8 7 6
25857 149/300 Read block 1687 len 20480
25858 150/300 Write block 3731 len 32768 data: 4 4 3 3 5 5 4 4
25859 151/300 Write block 2561 len 24576 data: 7 6 6 4 5 5
25860 152/300 Read block 497 len 28672
25861 153/300 Flush
25862 154/300 Write block 3230 len 16384 data: 8 7 7 5
25863 155/300 Read block 1901 len 28672
25864 156/300 Write block 2124 len 40960 data: 5 5 4 4 4 4 4 4 6 7
25865 157/300 Write block 1118 len 24576 data: 6 7 7 6 6 5
25866 158/300 Read block 4112 len 4096
25867 159/300 Read block 1878 len 32768
25868 160/300 Write block 141 len 4096 data: 5
25869 161/300 Read block 851 len 20480
25870 162/300 Write block 4798 len 36864 data: 5 6 7 5 5 6 6 6 6
25871 163/300 Write block 581 len 24576 data: 4 3 3 3 3 2
25872 164/300 Read block 3142 len 36864
25873 165/300 Read block 1280 len 4096
25874 166/300 Write block 3598 len 24576 data: 3 3 4 5 4 3
25875 167/300 Write block 3816 len 8192 data: 10 8
25876 168/300 Write block 1652 len 20480 data: 4 5 5 4 5
25877 169/300 Write block 1258 len 28672 data: 5 7 7 5 4 4 4
25878 170/300 Write block 4268 len 40960 data: 6 7 7 7 6 4 4 2 3 3
25879 171/300 Read block 3132 len 12288
25880 172/300 Write block 3246 len 16384 data: 6 6 7 5
25881 173/300 Write block 1661 len 32768 data: 7 5 6 5 4 4 4 4
25882 174/300 Write block 4808 len 20480 data: 4 5 6 5 4
25883 175/300 Write block 4671 len 32768 data: 8 7 8 7 7 5 5 5
25884 176/300 Write block 3222 len 28672 data: 4 4 3 4 6 6 5
25885 177/300 Write block 229 len 32768 data: 8 7 5 4 5 5 5 4
25886 178/300 Read block 3086 len 4096
25887 179/300 Read block 835 len 40960
25888 180/300 Read block 4303 len 36864
25889 181/300 Read block 4479 len 8192
25890 182/300 Read block 2384 len 32768
25891 183/300 Flush
25892 184/300 Read block 1896 len 28672
25893 185/300 Read block 2446 len 12288
25894 186/300 Read block 2326 len 8192
25895 187/300 Read block 179 len 32768
25896 188/300 Read block 366 len 4096
25897 189/300 Read block 4410 len 28672
25898 190/300 Read block 3986 len 32768
25899 191/300 Write block 4912 len 32768 data: 6 6 7 7 6 5 4 4
25900 192/300 Read block 4145 len 32768
25901 193/300 Write block 2986 len 36864 data: 6 5 6 5 7 5 5 6 6
25902 194/300 Write block 156 len 32768 data: 5 5 6 6 5 5 6 9
25903 195/300 Read block 3065 len 12288
25904 196/300 Flush
25905 197/300 Read block 4482 len 32768
25906 198/300 Read block 769 len 16384
25907 199/300 Write block 343 len 20480 data: 5 6 7 8 8
25908 200/300 Write block 3800 len 36864 data: 4 4 4 3 3 3 2 2 4
25909 201/300 Read block 2739 len 16384
25910 202/300 Read block 712 len 20480
25911 203/300 Read block 4139 len 16384
25912 204/300 Read block 2326 len 24576
25913 205/300 Read block 3302 len 36864
25914 206/300 Write block 4304 len 4096 data: 4
25915 207/300 Read block 3160 len 32768
25916 208/300 Read block 1827 len 36864
25917 209/300 Read block 3527 len 40960
25918 210/300 Read block 4564 len 40960
25919 211/300 Flush
25920 212/300 Write block 1919 len 32768 data: 6 5 4 4 5 4 4 3
25921 213/300 Read block 2163 len 16384
25922 214/300 Write block 234 len 36864 data: 6 6 5 5 5 6 6 6 5
25923 215/300 Read block 2140 len 4096
25924 216/300 Flush
25925 217/300 Write block 2894 len 24576 data: 5 5 5 5 4 6
25926 218/300 Write block 1076 len 20480 data: 8 6 7 5 3
25927 219/300 Read block 2413 len 28672
25928 220/300 Read block 2341 len 8192
25929 221/300 Write block 1505 len 40960 data: 5 4 6 6 6 8 9 9 8 9
25930 222/300 Write block 739 len 16384 data: 10 9 6 5
25931 223/300 Read block 3185 len 12288
25932 224/300 Write block 3396 len 12288 data: 6 6 8
25933 225/300 Read block 3684 len 16384
25934 226/300 Flush
25935 227/300 Read block 598 len 16384
25936 228/300 Write block 4788 len 36864 data: 2 2 2 3 2 2 3 5 4
25937 229/300 Read block 4367 len 16384
25938 230/300 Write block 3300 len 12288 data: 6 5 4
25939 231/300 Read block 2755 len 28672
25940 232/300 Write block 608 len 4096 data: 5
25941 233/300 Write block 4277 len 12288 data: 4 3 3
25942 234/300 Read block 1000 len 12288
25943 235/300 Write block 4098 len 24576 data: 7 8 8 7 5 7
25944 236/300 Read block 3152 len 20480
25945 237/300 Write block 2413 len 32768 data: 3 2 2 2 3 3 4 4
25946 238/300 Read block 3067 len 16384
25947 239/300 Read block 771 len 8192
25948 240/300 Flush
25949 241/300 Read block 2128 len 28672
25950 242/300 Read block 124 len 40960
25951 243/300 Flush
25952 244/300 Flush
25953 245/300 Read block 2653 len 4096
25954 246/300 Read block 4247 len 20480
25955 247/300 Read block 16 len 24576
25956 248/300 Write block 1529 len 8192 data: 6 6
25957 249/300 Read block 4958 len 20480
25958 250/300 Write block 2519 len 40960 data: 5 4 5 6 6 6 6 5 6 6
25959 251/300 Read block 4592 len 28672
25960 252/300 Write block 136 len 12288 data: 7 4 4
25961 253/300 Write block 1720 len 12288 data: 6 5 4
25962 254/300 Read block 3491 len 32768
25963 255/300 Read block 1511 len 8192
25964 256/300 Write block 2048 len 12288 data: 5 4 4
25965 257/300 Flush
25966 258/300 Write block 1661 len 36864 data: 8 6 7 6 5 5 5 5 5
25967 259/300 Read block 2822 len 40960
25968 260/300 Read block 4852 len 24576
25969 261/300 Read block 1801 len 36864
25970 262/300 Read block 429 len 32768
25971 263/300 Flush
25972 264/300 Write block 2726 len 32768 data: 2 2 2 2 3 3 3 4
25973 265/300 Flush
25974 266/300 Write block 2026 len 32768 data: 8 10 9 8 6 7 7 7
25975 267/300 Write block 2019 len 24576 data: 5 5 6 4 5 5
25976 268/300 Read block 4807 len 28672
25977 269/300 Write block 3747 len 20480 data: 5 6 6 6 5
25978 270/300 Write block 2873 len 32768 data: 4 5 5 5 5 6 6 6
25979 271/300 Flush
25980 272/300 Write block 4219 len 28672 data: 8 7 5 4 2 2 2
25981 273/300 Read block 2551 len 36864
25982 274/300 Write block 2235 len 28672 data: 4 7 7 7 6 6 5
25983 275/300 Read block 500 len 24576
25984 276/300 Write block 3545 len 24576 data: 6 6 7 5 5 3
25985 277/300 Read block 43 len 28672
25986 278/300 Read block 2532 len 28672
25987 279/300 Write block 1663 len 32768 data: 8 7 6 6 6 6 6 7
25988 280/300 Write block 222 len 20480 data: 3 4 7 8 8
25989 281/300 Read block 4739 len 24576
25990 282/300 Write block 4744 len 4096 data: 3
25991 283/300 Flush
25992 284/300 Flush
25993 285/300 Write block 3309 len 40960 data: 2 2 2 2 2 4 4 4 3 3
25994 286/300 Write block 3917 len 16384 data: 6 5 6 5
25995 287/300 Write block 383 len 8192 data: 6 4
25996 288/300 Flush
25997 289/300 Write block 3699 len 24576 data: 6 7 9 12 12 11
25998 290/300 Read block 4230 len 16384
25999 291/300 Write block 3026 len 28672 data: 7 6 7 7 7 8 7
26000 292/300 Write block 530 len 16384 data: 10 7 5 5
26001 293/300 Read block 3166 len 28672
26002 294/300 Read block 782 len 4096
26003 295/300 Write block 2420 len 28672 data: 5 3 4 5 6 6 5
26004 296/300 Read block 4120 len 40960
26005 297/300 Write block 4097 len 16384 data: 5 8 9 9
26006 298/300 Write block 2040 len 20480 data: 6 6 7 7 8
26007 299/300 Read block 782 len 40960
26008 300/300 Write block 2629 len 32768 data: 10 9 8 8 7 7 7 7
26009 301/300 Write block 4050 len 20480 data: 6 6 7 8 7
26010 Main task has work to do, go find it
26011 got dsc Start(1)
26012 start 1
26013 Jan 21 15:52:39.286 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/1, method: POST, req_id: c1f97d0b-ff40-41f4-b000-6f92612885c6, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
26014 Replay: started 1, returned:Ok(())
26015 ----------------------------------------------------------------
26016 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26017 States: Active Offline Active
26018 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26019 7945 Acked Flush 0 Done Sent Done false
26020 7946 Acked Write 3 Done Sent Done false
26021 7947 Acked Read 2 Done Sent Done false
26022 7948 Acked Write 10 Done Sent Done false
26023 7949 Acked Flush 0 Done Sent Done false
26024 7950 Acked Read 7 Done Sent Done false
26025 7951 Acked Read 9 Done Sent Done false
26026 7952 Acked Read 3 Done Sent Done false
26027 7953 Acked Read 7 Done Sent Done false
26028 7954 Acked Read 10 Done Sent Done false
26029 7955 Acked Read 2 Done Sent Done false
26030 7956 Acked Flush 0 Done Sent Done false
26031 7957 Acked Read 4 Done Sent Done false
26032 7958 Acked Read 10 Done Sent Done false
26033 7959 Acked Read 3 Done Sent Done false
26034 7960 Acked Read 8 Done Sent Done false
26035 7961 Acked Read 3 Done Sent Done false
26036 7962 Acked Read 9 Done Sent Done false
26037 7963 Acked Write 2 Done Sent Done false
26038 7964 Acked Read 9 Done Sent Done false
26039 7965 Acked Read 2 Done Sent Done false
26040 7966 Acked Write 6 Done Sent Done false
26041 7967 Acked Read 3 Done Sent Done false
26042 7968 Acked Read 2 Done Sent Done false
26043 7969 Acked Read 7 Done Sent Done false
26044 7970 Acked Write 10 Done Sent Done false
26045 7971 Acked Flush 0 Done Sent Done false
26046 7972 Acked Read 4 Done Sent Done false
26047 7973 Acked Write 1 Done Sent Done false
26048 7974 Acked Write 9 Done Sent Done false
26049 7975 Acked Write 9 Done Sent Done false
26050 7976 Acked Write 6 Done Sent Done false
26051 7977 Acked Write 10 Done Sent Done false
26052 7978 Acked Flush 0 Done Sent Done false
26053 7979 Acked Write 1 Done Sent Done false
26054 7980 Acked Write 6 Done Sent Done false
26055 7981 Acked Flush 0 Done Sent Done false
26056 7982 Acked Write 9 Done Sent Done false
26057 7983 Acked Read 1 Done Sent Done false
26058 7984 Acked Read 9 Done Sent Done false
26059 7985 Acked Flush 0 Done Sent Done false
26060 7986 Acked Read 7 Done Sent Done false
26061 7987 Acked Write 4 Done Sent Done false
26062 7988 Acked Read 1 Done Sent Done false
26063 7989 Acked Write 5 Done Sent Done false
26064 7990 Acked Read 6 Done Sent Done false
26065 7991 Acked Read 2 Done Sent Done false
26066 7992 Acked Read 2 Done Sent Done false
26067 7993 Acked Read 3 Done Sent Done false
26068 7994 Acked Write 4 Done Sent Done false
26069 7995 Acked Read 10 Done Sent Done false
26070 7996 Acked Write 1 Done Sent Done false
26071 7997 Acked Read 7 Done Sent Done false
26072 7998 Acked Write 5 Done Sent Done false
26073 7999 Acked Write 8 Done Sent Done false
26074 8000 Acked Read 9 Done Sent Done false
26075 8001 Acked Write 6 Done Sent Done false
26076 8002 Acked Write 1 Done Sent Done false
26077 8003 Acked Write 7 Done Sent Done false
26078 8004 Acked Read 10 Done Sent Done false
26079 8005 Acked Write 4 Done Sent Done false
26080 8006 Acked Write 10 Done Sent Done false
26081 8007 Acked Flush 0 Done Sent Done false
26082 8008 Acked Write 7 Done Sent Done false
26083 8009 Acked Write 7 Done Sent Done false
26084 8010 Acked Read 5 Done Sent Done false
26085 8011 Acked Write 3 Done Sent Done false
26086 8012 Acked Read 3 Done Sent Done false
26087 8013 Acked Write 7 Done Sent Done false
26088 8014 Acked Write 4 Done Sent Done false
26089 8015 Acked Read 1 Done Sent Done false
26090 8016 Acked Read 10 Done Sent Done false
26091 8017 Acked Read 4 Done Sent Done false
26092 8018 Acked Write 3 Done Sent Done false
26093 8019 Acked Read 9 Done Sent Done false
26094 8020 Acked Write 8 Done Sent Done false
26095 8021 Acked Write 4 Done Sent Done false
26096 8022 Acked Read 9 Done Sent Done false
26097 8023 Acked Read 1 Done Sent Done false
26098 8024 Acked Write 2 Done Sent Done false
26099 8025 Acked Read 1 Done Sent Done false
26100 8026 Acked Write 8 Done Sent Done false
26101 8027 Acked Read 10 Done Sent Done false
26102 8028 Acked Flush 0 Done Sent Done false
26103 8029 Acked Write 9 Done Sent Done false
26104 8030 Acked Read 7 Done Sent Done false
26105 8031 Acked Read 7 Done Sent Done false
26106 8032 Acked Write 5 Done Sent Done false
26107 8033 Acked Write 2 Done Sent Done false
26108 8034 Acked Flush 0 Done Sent Done false
26109 8035 Acked Write 10 Done Sent Done false
26110 8036 Acked Read 4 Done Sent Done false
26111 8037 Acked Write 7 Done Sent Done false
26112 8038 Acked Write 8 Done Sent Done false
26113 8039 Acked Write 2 Done Sent Done false
26114 8040 Acked Write 8 Done Sent Done false
26115 8041 Acked Write 9 Done Sent Done false
26116 8042 Acked Read 1 Done Sent Done false
26117 8043 Acked Flush 0 Done Sent Done false
26118 8044 Acked Write 5 Done Sent Done false
26119 8045 Acked Write 1 Done Sent Done false
26120 8046 Acked Read 8 Done Sent Done false
26121 8047 Acked Write 9 Done Sent Done false
26122 8048 Acked Read 5 Done Sent Done false
26123 8049 Acked Write 10 Done Sent Done false
26124 8050 Acked Write 8 Done Sent Done false
26125 8051 Acked Write 5 Done Sent Done false
26126 8052 Acked Read 10 Done Sent Done false
26127 8053 Acked Read 5 Done Sent Done false
26128 8054 Acked Flush 0 Done Sent Done false
26129 8055 Acked Read 3 Done Sent Done false
26130 8056 Acked Read 4 Done Sent Done false
26131 8057 Acked Write 10 Done Sent Done false
26132 8058 Acked Write 6 Done Sent Done false
26133 8059 Acked Write 4 Done Sent Done false
26134 8060 Acked Flush 0 Done Sent Done false
26135 8061 Acked Read 4 Done Sent Done false
26136 8062 Acked Write 5 Done Sent Done false
26137 8063 Acked Write 3 Done Sent Done false
26138 8064 Acked Flush 0 Done Sent Done false
26139 8065 Acked Write 8 Done Sent Done false
26140 8066 Acked Read 10 Done Sent Done false
26141 8067 Acked Read 9 Done Sent Done false
26142 8068 Acked Read 10 Done Sent Done false
26143 8069 Acked Read 3 Done Sent Done false
26144 8070 Acked Write 4 Done Sent Done false
26145 8071 Acked Read 7 Done Sent Done false
26146 8072 Acked Write 4 Done Sent Done false
26147 8073 Acked Read 8 Done Sent Done false
26148 8074 Acked Write 7 Done Sent Done false
26149 8075 Acked Write 1 Done Sent Done false
26150 8076 Acked Flush 0 Done Sent Done false
26151 8077 Acked Write 5 Done Sent Done false
26152 8078 Acked Read 1 Done Sent Done false
26153 8079 Acked Read 3 Done Sent Done false
26154 8080 Acked Read 7 Done Sent Done false
26155 8081 Acked Write 10 Done Sent Done false
26156 8082 Acked Flush 0 Done Sent Done false
26157 8083 Acked Read 2 Done Sent Done false
26158 8084 Acked Read 1 Done Sent Done false
26159 8085 Acked Write 6 Done Sent Done false
26160 8086 Acked Read 2 Done Sent Done false
26161 8087 Acked Read 7 Done Sent Done false
26162 8088 Acked Read 8 Done Sent Done false
26163 8089 Acked Read 2 Done Sent Done false
26164 8090 Acked Flush 0 Done Sent Done false
26165 8091 Acked Write 2 Done Sent Done false
26166 8092 Acked Write 10 Done Sent Done false
26167 8093 Acked Read 5 Done Sent Done false
26168 8094 Acked Write 8 Done Sent Done false
26169 8095 Acked Write 6 Done Sent Done false
26170 8096 Acked Read 7 Done Sent Done false
26171 8097 Acked Flush 0 Done Sent Done false
26172 8098 Acked Write 4 Done Sent Done false
26173 8099 Acked Read 7 Done Sent Done false
26174 8100 Acked Write 10 Done Sent Done false
26175 8101 Acked Write 6 Done Sent Done false
26176 8102 Acked Read 1 Done Sent Done false
26177 8103 Acked Read 8 Done Sent Done false
26178 8104 Acked Write 1 Done Sent Done false
26179 8105 Acked Read 5 Done Sent Done false
26180 8106 Acked Write 9 Done Sent Done false
26181 8107 Acked Write 6 Done Sent Done false
26182 8108 Acked Read 9 Done Sent Done false
26183 8109 Acked Read 1 Done Sent Done false
26184 8110 Acked Write 6 Done Sent Done false
26185 8111 Acked Write 2 Done Sent Done false
26186 8112 Acked Write 5 Done Sent Done false
26187 8113 Acked Write 7 Done Sent Done false
26188 8114 Acked Write 10 Done Sent Done false
26189 8115 Acked Read 3 Done Sent Done false
26190 8116 Acked Write 4 Done Sent Done false
26191 8117 Acked Write 8 Done Sent Done false
26192 8118 Acked Write 5 Done Sent Done false
26193 8119 Acked Write 8 Done Sent Done false
26194 8120 Acked Write 7 Done Sent Done false
26195 8121 Acked Write 8 Done Sent Done false
26196 8122 Acked Read 1 Done Sent Done false
26197 8123 Acked Read 10 Done Sent Done false
26198 8124 Acked Read 9 Done Sent Done false
26199 8125 Acked Read 2 Done Sent Done false
26200 8126 Acked Read 8 Done Sent Done false
26201 8127 Acked Flush 0 Done Sent Done false
26202 8128 Acked Read 7 Done Sent Done false
26203 8129 Acked Read 3 Done Sent Done false
26204 8130 Acked Read 2 Done Sent Done false
26205 8131 Acked Read 8 Done Sent Done false
26206 8132 Acked Read 1 Done Sent Done false
26207 8133 Acked Read 7 Done Sent Done false
26208 8134 Acked Read 8 Done Sent Done false
26209 8135 Acked Write 8 Done Sent Done false
26210 8136 Acked Read 8 Done Sent Done false
26211 8137 Acked Write 9 Done Sent Done false
26212 8138 Acked Write 8 Done Sent Done false
26213 8139 Acked Read 3 Done Sent Done false
26214 8140 Acked Flush 0 Done Sent Done false
26215 8141 Acked Read 8 Done Sent Done false
26216 8142 Acked Read 4 Done Sent Done false
26217 8143 Acked Write 5 Done Sent Done false
26218 8144 Acked Write 9 Done Sent Done false
26219 8145 Acked Read 4 Done Sent Done false
26220 8146 Acked Read 5 Done Sent Done false
26221 8147 Acked Read 4 Done Sent Done false
26222 8148 Acked Read 6 Done Sent Done false
26223 8149 Acked Read 9 Done Sent Done false
26224 8150 Acked Write 1 Done Sent Done false
26225 8151 Acked Read 8 Done Sent Done false
26226 8152 Acked Read 9 Done Sent Done false
26227 8153 Acked Read 10 Done Sent Done false
26228 8154 Acked Read 10 Done Sent Done false
26229 8155 Acked Flush 0 Done Sent Done false
26230 8156 Acked Write 8 Done Sent Done false
26231 8157 Acked Read 4 Done Sent Done false
26232 8158 Acked Write 9 Done Sent Done false
26233 8159 Acked Read 1 Done Sent Done false
26234 8160 Acked Flush 0 Done Sent Done false
26235 8161 Acked Write 6 Done Sent Done false
26236 8162 Acked Write 5 Done Sent Done false
26237 8163 Acked Read 7 Done Sent Done false
26238 8164 Acked Read 2 Done Sent Done false
26239 8165 Acked Write 10 Done Sent Done false
26240 8166 Acked Write 4 Done Sent Done false
26241 8167 Acked Read 3 Done Sent Done false
26242 8168 Acked Write 3 Done Sent Done false
26243 8169 Acked Read 4 Done Sent Done false
26244 8170 Acked Flush 0 Done Sent Done false
26245 8171 Acked Read 4 Done Sent Done false
26246 8172 Acked Write 9 Done Sent Done false
26247 8173 Acked Read 4 Done Sent Done false
26248 8174 Acked Write 3 Done Sent Done false
26249 8175 Acked Read 7 Done Sent Done false
26250 8176 Acked Write 1 Done Sent Done false
26251 8177 Acked Write 3 Done Sent Done false
26252 8178 Acked Read 3 Done Sent Done false
26253 8179 Acked Write 6 Done Sent Done false
26254 8180 Acked Read 5 Done Sent Done false
26255 8181 Acked Write 8 Done Sent Done false
26256 8182 Acked Read 4 Done Sent Done false
26257 8183 Acked Read 2 Done Sent Done false
26258 8184 Acked Flush 0 Done Sent Done false
26259 8185 Acked Read 7 Done Sent Done false
26260 8186 Acked Read 10 Done Sent Done false
26261 8187 Acked Flush 0 Done Sent Done false
26262 8188 Acked Flush 0 Done Sent Done false
26263 8189 Acked Read 1 Done Sent Done false
26264 8190 Acked Read 5 Done Sent Done false
26265 8191 Acked Read 6 Done Sent Done false
26266 8192 Acked Write 2 Done Sent Done false
26267 8193 Acked Read 5 Done Sent Done false
26268 8194 Acked Write 10 Done Sent Done false
26269 8195 Acked Read 7 Done Sent Done false
26270 8196 Acked Write 3 Done Sent Done false
26271 8197 Acked Write 3 Done Sent Done false
26272 8198 Acked Read 8 Done Sent Done false
26273 8199 Acked Read 2 Done Sent Done false
26274 8200 Acked Write 3 Done Sent Done false
26275 8201 Acked Flush 0 Done Sent Done false
26276 8202 Acked Write 9 Done Sent Done false
26277 8203 Acked Read 10 Done Sent Done false
26278 8204 Acked Read 6 Done Sent Done false
26279 8205 Acked Read 9 Done Sent Done false
26280 8206 Acked Read 8 Done Sent Done false
26281 8207 Acked Flush 0 Done Sent Done false
26282 8208 Acked Write 8 Done Sent Done false
26283 8209 Acked Flush 0 Done Sent Done false
26284 8210 Acked Write 8 Done Sent Done false
26285 8211 Acked Write 6 Done Sent Done false
26286 8212 Acked Read 7 Done Sent Done false
26287 8213 Acked Write 5 Done Sent Done false
26288 8214 Acked Write 8 Done Sent Done false
26289 8215 Acked Flush 0 Done Sent Done false
26290 8216 Acked Write 7 Done Sent Done false
26291 8217 Acked Read 9 Done Sent Done false
26292 8218 Acked Write 7 Done Sent Done false
26293 8219 Acked Read 6 Done Sent Done false
26294 8220 Acked Write 6 Done Sent Done false
26295 8221 Acked Read 7 Done Sent Done false
26296 8222 Acked Read 7 Done Sent Done false
26297 8223 Acked Write 8 Done Sent Done false
26298 8224 Acked Write 5 Done Sent Done false
26299 8225 Acked Read 6 Done Sent Done false
26300 8226 Acked Write 1 Done Sent Done false
26301 8227 Acked Flush 0 Done Sent Done false
26302 8228 Acked Flush 0 Done Sent Done false
26303 8229 Acked Write 10 Done Sent Done false
26304 8230 Acked Write 4 Done Sent Done false
26305 8231 Acked Write 2 Done Sent Done false
26306 8232 Acked Flush 0 Done Sent Done false
26307 8233 Acked Write 6 Done Sent Done false
26308 8234 Acked Read 4 Done Sent Done false
26309 8235 Acked Write 7 Done Sent Done false
26310 8236 Acked Write 4 Done Sent Done false
26311 8237 Acked Read 7 Done Sent Done false
26312 8238 Acked Read 1 Done Sent Done false
26313 8239 Acked Write 7 Done Sent Done false
26314 8240 Acked Read 10 Done Sent Done false
26315 8241 Acked Write 4 Done Sent Done false
26316 8242 Acked Write 5 Done Sent Done false
26317 8243 Acked Read 10 Done Sent Done false
26318 8244 Acked Write 8 Done Sent Done false
26319 8245 Acked Write 5 Sent Sent Sent false
26320 STATES DS:0 DS:1 DS:2 TOTAL
26321 Sent 1 301 1 303
26322 Done 300 0 300 600
26323 Skipped 0 0 0 0
26324 Error 0 0 0 0
26325 Last Flush: 8232 7944 8232
26326 Downstairs last five completed: 7944 7943 7942 7941 7940
26327 Upstairs last five completed: 8245 8244 8243 8242 8241
26328 CLIENT: Up:0 ds:301 act:2
26329 [8820] got start action
26330 [8820] I am going to restart
26331 Starting downstairs at port 8820
26332 Make output file at "/tmp/dsc/downstairs-8820.txt"
26333 [8820][1] reports Starting
26334 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1224)
26335 [8820][1] reports Running
26336 ----------------------------------------------------------------
26337 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26338 States: Active Offline Active
26339 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26340 7945 Acked Flush 0 Done Sent Done false
26341 7946 Acked Write 3 Done Sent Done false
26342 7947 Acked Read 2 Done Sent Done false
26343 7948 Acked Write 10 Done Sent Done false
26344 7949 Acked Flush 0 Done Sent Done false
26345 7950 Acked Read 7 Done Sent Done false
26346 7951 Acked Read 9 Done Sent Done false
26347 7952 Acked Read 3 Done Sent Done false
26348 7953 Acked Read 7 Done Sent Done false
26349 7954 Acked Read 10 Done Sent Done false
26350 7955 Acked Read 2 Done Sent Done false
26351 7956 Acked Flush 0 Done Sent Done false
26352 7957 Acked Read 4 Done Sent Done false
26353 7958 Acked Read 10 Done Sent Done false
26354 7959 Acked Read 3 Done Sent Done false
26355 7960 Acked Read 8 Done Sent Done false
26356 7961 Acked Read 3 Done Sent Done false
26357 7962 Acked Read 9 Done Sent Done false
26358 7963 Acked Write 2 Done Sent Done false
26359 7964 Acked Read 9 Done Sent Done false
26360 7965 Acked Read 2 Done Sent Done false
26361 7966 Acked Write 6 Done Sent Done false
26362 7967 Acked Read 3 Done Sent Done false
26363 7968 Acked Read 2 Done Sent Done false
26364 7969 Acked Read 7 Done Sent Done false
26365 7970 Acked Write 10 Done Sent Done false
26366 7971 Acked Flush 0 Done Sent Done false
26367 7972 Acked Read 4 Done Sent Done false
26368 7973 Acked Write 1 Done Sent Done false
26369 7974 Acked Write 9 Done Sent Done false
26370 7975 Acked Write 9 Done Sent Done false
26371 7976 Acked Write 6 Done Sent Done false
26372 7977 Acked Write 10 Done Sent Done false
26373 7978 Acked Flush 0 Done Sent Done false
26374 7979 Acked Write 1 Done Sent Done false
26375 7980 Acked Write 6 Done Sent Done false
26376 7981 Acked Flush 0 Done Sent Done false
26377 7982 Acked Write 9 Done Sent Done false
26378 7983 Acked Read 1 Done Sent Done false
26379 7984 Acked Read 9 Done Sent Done false
26380 7985 Acked Flush 0 Done Sent Done false
26381 7986 Acked Read 7 Done Sent Done false
26382 7987 Acked Write 4 Done Sent Done false
26383 7988 Acked Read 1 Done Sent Done false
26384 7989 Acked Write 5 Done Sent Done false
26385 7990 Acked Read 6 Done Sent Done false
26386 7991 Acked Read 2 Done Sent Done false
26387 7992 Acked Read 2 Done Sent Done false
26388 7993 Acked Read 3 Done Sent Done false
26389 7994 Acked Write 4 Done Sent Done false
26390 7995 Acked Read 10 Done Sent Done false
26391 7996 Acked Write 1 Done Sent Done false
26392 7997 Acked Read 7 Done Sent Done false
26393 7998 Acked Write 5 Done Sent Done false
26394 7999 Acked Write 8 Done Sent Done false
26395 8000 Acked Read 9 Done Sent Done false
26396 8001 Acked Write 6 Done Sent Done false
26397 8002 Acked Write 1 Done Sent Done false
26398 8003 Acked Write 7 Done Sent Done false
26399 8004 Acked Read 10 Done Sent Done false
26400 8005 Acked Write 4 Done Sent Done false
26401 8006 Acked Write 10 Done Sent Done false
26402 8007 Acked Flush 0 Done Sent Done false
26403 8008 Acked Write 7 Done Sent Done false
26404 8009 Acked Write 7 Done Sent Done false
26405 8010 Acked Read 5 Done Sent Done false
26406 8011 Acked Write 3 Done Sent Done false
26407 8012 Acked Read 3 Done Sent Done false
26408 8013 Acked Write 7 Done Sent Done false
26409 8014 Acked Write 4 Done Sent Done false
26410 8015 Acked Read 1 Done Sent Done false
26411 8016 Acked Read 10 Done Sent Done false
26412 8017 Acked Read 4 Done Sent Done false
26413 8018 Acked Write 3 Done Sent Done false
26414 8019 Acked Read 9 Done Sent Done false
26415 8020 Acked Write 8 Done Sent Done false
26416 8021 Acked Write 4 Done Sent Done false
26417 8022 Acked Read 9 Done Sent Done false
26418 8023 Acked Read 1 Done Sent Done false
26419 8024 Acked Write 2 Done Sent Done false
26420 8025 Acked Read 1 Done Sent Done false
26421 8026 Acked Write 8 Done Sent Done false
26422 8027 Acked Read 10 Done Sent Done false
26423 8028 Acked Flush 0 Done Sent Done false
26424 8029 Acked Write 9 Done Sent Done false
26425 8030 Acked Read 7 Done Sent Done false
26426 8031 Acked Read 7 Done Sent Done false
26427 8032 Acked Write 5 Done Sent Done false
26428 8033 Acked Write 2 Done Sent Done false
26429 8034 Acked Flush 0 Done Sent Done false
26430 8035 Acked Write 10 Done Sent Done false
26431 8036 Acked Read 4 Done Sent Done false
26432 8037 Acked Write 7 Done Sent Done false
26433 8038 Acked Write 8 Done Sent Done false
26434 8039 Acked Write 2 Done Sent Done false
26435 8040 Acked Write 8 Done Sent Done false
26436 8041 Acked Write 9 Done Sent Done false
26437 8042 Acked Read 1 Done Sent Done false
26438 8043 Acked Flush 0 Done Sent Done false
26439 8044 Acked Write 5 Done Sent Done false
26440 8045 Acked Write 1 Done Sent Done false
26441 8046 Acked Read 8 Done Sent Done false
26442 8047 Acked Write 9 Done Sent Done false
26443 8048 Acked Read 5 Done Sent Done false
26444 8049 Acked Write 10 Done Sent Done false
26445 8050 Acked Write 8 Done Sent Done false
26446 8051 Acked Write 5 Done Sent Done false
26447 8052 Acked Read 10 Done Sent Done false
26448 8053 Acked Read 5 Done Sent Done false
26449 8054 Acked Flush 0 Done Sent Done false
26450 8055 Acked Read 3 Done Sent Done false
26451 8056 Acked Read 4 Done Sent Done false
26452 8057 Acked Write 10 Done Sent Done false
26453 8058 Acked Write 6 Done Sent Done false
26454 8059 Acked Write 4 Done Sent Done false
26455 8060 Acked Flush 0 Done Sent Done false
26456 8061 Acked Read 4 Done Sent Done false
26457 8062 Acked Write 5 Done Sent Done false
26458 8063 Acked Write 3 Done Sent Done false
26459 8064 Acked Flush 0 Done Sent Done false
26460 8065 Acked Write 8 Done Sent Done false
26461 8066 Acked Read 10 Done Sent Done false
26462 8067 Acked Read 9 Done Sent Done false
26463 8068 Acked Read 10 Done Sent Done false
26464 8069 Acked Read 3 Done Sent Done false
26465 8070 Acked Write 4 Done Sent Done false
26466 8071 Acked Read 7 Done Sent Done false
26467 8072 Acked Write 4 Done Sent Done false
26468 8073 Acked Read 8 Done Sent Done false
26469 8074 Acked Write 7 Done Sent Done false
26470 8075 Acked Write 1 Done Sent Done false
26471 8076 Acked Flush 0 Done Sent Done false
26472 8077 Acked Write 5 Done Sent Done false
26473 8078 Acked Read 1 Done Sent Done false
26474 8079 Acked Read 3 Done Sent Done false
26475 8080 Acked Read 7 Done Sent Done false
26476 8081 Acked Write 10 Done Sent Done false
26477 8082 Acked Flush 0 Done Sent Done false
26478 8083 Acked Read 2 Done Sent Done false
26479 8084 Acked Read 1 Done Sent Done false
26480 8085 Acked Write 6 Done Sent Done false
26481 8086 Acked Read 2 Done Sent Done false
26482 8087 Acked Read 7 Done Sent Done false
26483 8088 Acked Read 8 Done Sent Done false
26484 8089 Acked Read 2 Done Sent Done false
26485 8090 Acked Flush 0 Done Sent Done false
26486 8091 Acked Write 2 Done Sent Done false
26487 8092 Acked Write 10 Done Sent Done false
26488 8093 Acked Read 5 Done Sent Done false
26489 8094 Acked Write 8 Done Sent Done false
26490 8095 Acked Write 6 Done Sent Done false
26491 8096 Acked Read 7 Done Sent Done false
26492 8097 Acked Flush 0 Done Sent Done false
26493 8098 Acked Write 4 Done Sent Done false
26494 8099 Acked Read 7 Done Sent Done false
26495 8100 Acked Write 10 Done Sent Done false
26496 8101 Acked Write 6 Done Sent Done false
26497 8102 Acked Read 1 Done Sent Done false
26498 8103 Acked Read 8 Done Sent Done false
26499 8104 Acked Write 1 Done Sent Done false
26500 8105 Acked Read 5 Done Sent Done false
26501 8106 Acked Write 9 Done Sent Done false
26502 8107 Acked Write 6 Done Sent Done false
26503 8108 Acked Read 9 Done Sent Done false
26504 8109 Acked Read 1 Done Sent Done false
26505 8110 Acked Write 6 Done Sent Done false
26506 8111 Acked Write 2 Done Sent Done false
26507 8112 Acked Write 5 Done Sent Done false
26508 8113 Acked Write 7 Done Sent Done false
26509 8114 Acked Write 10 Done Sent Done false
26510 8115 Acked Read 3 Done Sent Done false
26511 8116 Acked Write 4 Done Sent Done false
26512 8117 Acked Write 8 Done Sent Done false
26513 8118 Acked Write 5 Done Sent Done false
26514 8119 Acked Write 8 Done Sent Done false
26515 8120 Acked Write 7 Done Sent Done false
26516 8121 Acked Write 8 Done Sent Done false
26517 8122 Acked Read 1 Done Sent Done false
26518 8123 Acked Read 10 Done Sent Done false
26519 8124 Acked Read 9 Done Sent Done false
26520 8125 Acked Read 2 Done Sent Done false
26521 8126 Acked Read 8 Done Sent Done false
26522 8127 Acked Flush 0 Done Sent Done false
26523 8128 Acked Read 7 Done Sent Done false
26524 8129 Acked Read 3 Done Sent Done false
26525 8130 Acked Read 2 Done Sent Done false
26526 8131 Acked Read 8 Done Sent Done false
26527 8132 Acked Read 1 Done Sent Done false
26528 8133 Acked Read 7 Done Sent Done false
26529 8134 Acked Read 8 Done Sent Done false
26530 8135 Acked Write 8 Done Sent Done false
26531 8136 Acked Read 8 Done Sent Done false
26532 8137 Acked Write 9 Done Sent Done false
26533 8138 Acked Write 8 Done Sent Done false
26534 8139 Acked Read 3 Done Sent Done false
26535 8140 Acked Flush 0 Done Sent Done false
26536 8141 Acked Read 8 Done Sent Done false
26537 8142 Acked Read 4 Done Sent Done false
26538 8143 Acked Write 5 Done Sent Done false
26539 8144 Acked Write 9 Done Sent Done false
26540 8145 Acked Read 4 Done Sent Done false
26541 8146 Acked Read 5 Done Sent Done false
26542 8147 Acked Read 4 Done Sent Done false
26543 8148 Acked Read 6 Done Sent Done false
26544 8149 Acked Read 9 Done Sent Done false
26545 8150 Acked Write 1 Done Sent Done false
26546 8151 Acked Read 8 Done Sent Done false
26547 8152 Acked Read 9 Done Sent Done false
26548 8153 Acked Read 10 Done Sent Done false
26549 8154 Acked Read 10 Done Sent Done false
26550 8155 Acked Flush 0 Done Sent Done false
26551 8156 Acked Write 8 Done Sent Done false
26552 8157 Acked Read 4 Done Sent Done false
26553 8158 Acked Write 9 Done Sent Done false
26554 8159 Acked Read 1 Done Sent Done false
26555 8160 Acked Flush 0 Done Sent Done false
26556 8161 Acked Write 6 Done Sent Done false
26557 8162 Acked Write 5 Done Sent Done false
26558 8163 Acked Read 7 Done Sent Done false
26559 8164 Acked Read 2 Done Sent Done false
26560 8165 Acked Write 10 Done Sent Done false
26561 8166 Acked Write 4 Done Sent Done false
26562 8167 Acked Read 3 Done Sent Done false
26563 8168 Acked Write 3 Done Sent Done false
26564 8169 Acked Read 4 Done Sent Done false
26565 8170 Acked Flush 0 Done Sent Done false
26566 8171 Acked Read 4 Done Sent Done false
26567 8172 Acked Write 9 Done Sent Done false
26568 8173 Acked Read 4 Done Sent Done false
26569 8174 Acked Write 3 Done Sent Done false
26570 8175 Acked Read 7 Done Sent Done false
26571 8176 Acked Write 1 Done Sent Done false
26572 8177 Acked Write 3 Done Sent Done false
26573 8178 Acked Read 3 Done Sent Done false
26574 8179 Acked Write 6 Done Sent Done false
26575 8180 Acked Read 5 Done Sent Done false
26576 8181 Acked Write 8 Done Sent Done false
26577 8182 Acked Read 4 Done Sent Done false
26578 8183 Acked Read 2 Done Sent Done false
26579 8184 Acked Flush 0 Done Sent Done false
26580 8185 Acked Read 7 Done Sent Done false
26581 8186 Acked Read 10 Done Sent Done false
26582 8187 Acked Flush 0 Done Sent Done false
26583 8188 Acked Flush 0 Done Sent Done false
26584 8189 Acked Read 1 Done Sent Done false
26585 8190 Acked Read 5 Done Sent Done false
26586 8191 Acked Read 6 Done Sent Done false
26587 8192 Acked Write 2 Done Sent Done false
26588 8193 Acked Read 5 Done Sent Done false
26589 8194 Acked Write 10 Done Sent Done false
26590 8195 Acked Read 7 Done Sent Done false
26591 8196 Acked Write 3 Done Sent Done false
26592 8197 Acked Write 3 Done Sent Done false
26593 8198 Acked Read 8 Done Sent Done false
26594 8199 Acked Read 2 Done Sent Done false
26595 8200 Acked Write 3 Done Sent Done false
26596 8201 Acked Flush 0 Done Sent Done false
26597 8202 Acked Write 9 Done Sent Done false
26598 8203 Acked Read 10 Done Sent Done false
26599 8204 Acked Read 6 Done Sent Done false
26600 8205 Acked Read 9 Done Sent Done false
26601 8206 Acked Read 8 Done Sent Done false
26602 8207 Acked Flush 0 Done Sent Done false
26603 8208 Acked Write 8 Done Sent Done false
26604 8209 Acked Flush 0 Done Sent Done false
26605 8210 Acked Write 8 Done Sent Done false
26606 8211 Acked Write 6 Done Sent Done false
26607 8212 Acked Read 7 Done Sent Done false
26608 8213 Acked Write 5 Done Sent Done false
26609 8214 Acked Write 8 Done Sent Done false
26610 8215 Acked Flush 0 Done Sent Done false
26611 8216 Acked Write 7 Done Sent Done false
26612 8217 Acked Read 9 Done Sent Done false
26613 8218 Acked Write 7 Done Sent Done false
26614 8219 Acked Read 6 Done Sent Done false
26615 8220 Acked Write 6 Done Sent Done false
26616 8221 Acked Read 7 Done Sent Done false
26617 8222 Acked Read 7 Done Sent Done false
26618 8223 Acked Write 8 Done Sent Done false
26619 8224 Acked Write 5 Done Sent Done false
26620 8225 Acked Read 6 Done Sent Done false
26621 8226 Acked Write 1 Done Sent Done false
26622 8227 Acked Flush 0 Done Sent Done false
26623 8228 Acked Flush 0 Done Sent Done false
26624 8229 Acked Write 10 Done Sent Done false
26625 8230 Acked Write 4 Done Sent Done false
26626 8231 Acked Write 2 Done Sent Done false
26627 8232 Acked Flush 0 Done Sent Done false
26628 8233 Acked Write 6 Done Sent Done false
26629 8234 Acked Read 4 Done Sent Done false
26630 8235 Acked Write 7 Done Sent Done false
26631 8236 Acked Write 4 Done Sent Done false
26632 8237 Acked Read 7 Done Sent Done false
26633 8238 Acked Read 1 Done Sent Done false
26634 8239 Acked Write 7 Done Sent Done false
26635 8240 Acked Read 10 Done Sent Done false
26636 8241 Acked Write 4 Done Sent Done false
26637 8242 Acked Write 5 Done Sent Done false
26638 8243 Acked Read 10 Done Sent Done false
26639 8244 Acked Write 8 Done Sent Done false
26640 8245 Acked Write 5 Done Sent Done false
26641 8246 Acked Flush 0 Done Sent Done false
26642 STATES DS:0 DS:1 DS:2 TOTAL
26643 Sent 0 302 0 302
26644 Done 302 0 302 604
26645 Skipped 0 0 0 0
26646 Error 0 0 0 0
26647 Last Flush: 8246 7944 8246
26648 Downstairs last five completed: 7944 7943 7942 7941 7940
26649 Upstairs last five completed: 8246 8245 8244 8243 8242
26650 CLIENT: Up:0 ds:302 act:2
266512025-01-21T15:52:45.044ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
266522025-01-21T15:52:45.044ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38091) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
266532025-01-21T15:52:45.045ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
266542025-01-21T15:52:45.046ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
266552025-01-21T15:52:45.046ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
266562025-01-21T15:52:45.046ZINFOcrucible: send last flush ID to this DS: 7944 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
266572025-01-21T15:52:45.046ZINFOcrucible: Replied this last flush ID: 7944 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
266582025-01-21T15:52:45.046ZINFOcrucible: [1] client re-new 302 jobs since flush 7944 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
266592025-01-21T15:52:45.046ZINFOcrucible: [1] Replayed 302 jobs since flush: 7944 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
26660 ----------------------------------------------------------------
26661 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26662 Downstairs last five completed: 8246 8245 8244 8243 8242
26663 Upstairs last five completed: 8246 8245 8244 8243 8242
26664 CLIENT: Up:0 ds:0 act:3
26665 Replay: All jobs finished
26666 Main task has work to do, go find it
26667 got dsc Stop(0)
26668 stop 0
26669 Jan 21 15:52:47.292 INFO [8810] Got stop action so:false kr:false
26670 request completed, latency_us: 382, response_code: 204, uri: /stop/cid/0, method: POST, req_id: c9b85bf8-8234-48f7-acbd-faa859a4461a, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
266712025-01-21T15:52:47.293ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
26672 Jan 21 15:52:47.293 INFO request completed, latency_us: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:52:47.293138977Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
26673 135, response_code: 200, uri: /state/cid/0, method: GET, req_id: {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:52:47.293174885Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
26674 39eb1d37-75dc-49ff-a08c-e30b35ce5cbd, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:52:47.293201483Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
26675 1:9998
26676 [8810] Exited with: ExitStatus(unix_wait_status(9))
26677 [8810][0] reports Exit
26678 Jan 21 15:52:51.294 INFO request completed, latency_us: 250, response_code: 200, uri: /state/cid/0, method: GET, req_id: bfcd14bf-7a7d-4719-affc-dba5199ef163, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
26679 001/300 Read block 1732 len 40960
26680 002/300 Write block 1177 len 36864 data: 3 3 2 2 2 2 3 4 5
26681 003/300 Read block 1669 len 36864
26682 004/300 Write block 1790 len 20480 data: 4 5 5 5 6
26683 005/300 Write block 1179 len 8192 data: 3 3
26684 006/300 Write block 1102 len 32768 data: 6 6 7 9 6 6 7 8
26685 007/300 Flush
26686 008/300 Read block 2422 len 4096
26687 009/300 Write block 1566 len 24576 data: 6 6 6 5 5 5
26688 010/300 Write block 1658 len 20480 data: 7 6 7 9 7
26689 011/300 Write block 2101 len 32768 data: 5 4 3 3 4 5 6 6
26690 012/300 Read block 3014 len 20480
26691 013/300 Read block 3208 len 40960
26692 014/300 Write block 2419 len 16384 data: 5 6 4 5
26693 015/300 Read block 249 len 12288
26694 016/300 Write block 2054 len 20480 data: 5 5 5 5 5
26695 017/300 Read block 3607 len 36864
26696 018/300 Read block 4694 len 36864
26697 019/300 Write block 2888 len 24576 data: 7 8 7 6 4 6
26698 020/300 Write block 4179 len 16384 data: 8 7 6 5
26699 021/300 Read block 2741 len 16384
26700 022/300 Read block 4801 len 8192
26701 023/300 Read block 4169 len 8192
26702 024/300 Write block 3111 len 12288 data: 7 6 5
26703 025/300 Read block 1140 len 16384
26704 026/300 Write block 3262 len 4096 data: 3
26705 027/300 Write block 1407 len 24576 data: 5 5 5 5 5 6
26706 028/300 Read block 1200 len 28672
26707 029/300 Write block 3888 len 28672 data: 8 9 11 9 9 7 6
26708 030/300 Write block 2100 len 4096 data: 5
26709 031/300 Read block 2945 len 28672
26710 032/300 Read block 846 len 20480
26711 033/300 Write block 2516 len 36864 data: 3 3 4 6 5 6 7 7 7
26712 034/300 Read block 1802 len 4096
26713 035/300 Write block 4459 len 16384 data: 3 3 3 3
26714 036/300 Read block 1014 len 4096
26715 037/300 Read block 3971 len 24576
26716 038/300 Read block 3480 len 28672
26717 039/300 Read block 4274 len 24576
26718 040/300 Read block 625 len 32768
26719 041/300 Write block 1670 len 4096 data: 8
26720 042/300 Read block 4333 len 40960
26721 043/300 Read block 3410 len 4096
26722 044/300 Read block 4792 len 8192
26723 045/300 Read block 1825 len 28672
26724 046/300 Read block 556 len 4096
26725 047/300 Write block 3870 len 4096 data: 4
26726 048/300 Read block 814 len 24576
26727 049/300 Read block 1929 len 12288
26728 050/300 Read block 3924 len 40960
26729 051/300 Read block 1523 len 24576
26730 052/300 Write block 4257 len 16384 data: 4 3 3 3
26731 053/300 Write block 3815 len 32768 data: 10 11 9 6 4 4 3 4
26732 054/300 Read block 3800 len 32768
26733 055/300 Read block 154 len 12288
26734 056/300 Flush
26735 057/300 Read block 807 len 36864
26736 058/300 Read block 2412 len 4096
26737 059/300 Read block 1672 len 8192
26738 060/300 Read block 110 len 8192
26739 061/300 Write block 3165 len 8192 data: 7 4
26740 062/300 Write block 162 len 40960 data: 7 10 8 7 6 6 5 5 5 5
26741 063/300 Read block 862 len 24576
26742 064/300 Write block 1341 len 40960 data: 6 6 5 5 4 6 6 6 6 6
26743 065/300 Write block 470 len 40960 data: 5 5 3 4 3 3 3 3 3 4
26744 066/300 Write block 1665 len 36864 data: 7 7 7 7 7 9 7 8 8
26745 067/300 Flush
26746 068/300 Write block 964 len 4096 data: 4
26747 069/300 Write block 4976 len 24576 data: 3 3 2 2 2 3
26748 070/300 Write block 1668 len 32768 data: 8 8 10 8 9 9 7 7
26749 071/300 Write block 3085 len 32768 data: 9 8 7 6 6 7 5 4
26750 072/300 Flush
26751 073/300 Write block 644 len 12288 data: 4 3 5
26752 074/300 Read block 4527 len 20480
26753 075/300 Read block 1178 len 20480
26754 076/300 Flush
26755 077/300 Write block 925 len 20480 data: 3 4 4 5 5
26756 078/300 Write block 2772 len 28672 data: 3 2 3 3 4 5 5
26757 079/300 Read block 301 len 28672
26758 080/300 Read block 820 len 8192
26759 081/300 Read block 2184 len 12288
26760 082/300 Read block 3854 len 20480
26761 083/300 Flush
26762 084/300 Read block 4174 len 8192
26763 085/300 Read block 4491 len 40960
26764 086/300 Write block 3877 len 8192 data: 9 10
26765 087/300 Read block 3460 len 12288
26766 088/300 Read block 4011 len 8192
26767 089/300 Read block 2766 len 20480
26768 090/300 Write block 1796 len 4096 data: 7
26769 091/300 Flush
26770 092/300 Read block 4894 len 16384
26771 093/300 Write block 1704 len 24576 data: 5 6 6 6 5 5
26772 094/300 Write block 992 len 4096 data: 5
26773 095/300 Write block 1540 len 32768 data: 8 6 9 9 7 7 5 6
26774 096/300 Read block 847 len 32768
26775 097/300 Read block 3849 len 36864
26776 098/300 Read block 1194 len 4096
26777 099/300 Flush
26778 100/300 Flush
26779 101/300 Read block 4418 len 24576
26780 102/300 Read block 4099 len 28672
26781 103/300 Write block 766 len 8192 data: 3 3
26782 104/300 Read block 3556 len 16384
26783 105/300 Write block 4909 len 28672 data: 6 7 7 7 7 8 8
26784 106/300 Flush
26785 107/300 Read block 4144 len 40960
26786 108/300 Write block 25 len 4096 data: 4
26787 109/300 Read block 1986 len 28672
26788 110/300 Read block 3222 len 8192
26789 111/300 Read block 590 len 24576
26790 112/300 Read block 4455 len 16384
26791 113/300 Write block 2604 len 36864 data: 10 9 8 8 8 8 5 4 3
26792 114/300 Write block 2077 len 36864 data: 7 7 8 8 7 7 5 4 4
26793 115/300 Write block 3982 len 12288 data: 5 5 6
26794 116/300 Write block 1812 len 16384 data: 9 8 6 6
26795 117/300 Read block 3710 len 32768
26796 118/300 Read block 1221 len 24576
26797 119/300 Read block 3230 len 36864
26798 120/300 Read block 186 len 20480
26799 121/300 Write block 3417 len 4096 data: 8
26800 122/300 Write block 2392 len 32768 data: 8 7 7 7 7 5 3 2
26801 123/300 Write block 1982 len 20480 data: 7 9 9 10 10
26802 124/300 Read block 2660 len 12288
26803 125/300 Read block 1207 len 32768
26804 126/300 Read block 3097 len 40960
26805 127/300 Flush
26806 128/300 Read block 4682 len 20480
26807 129/300 Write block 920 len 40960 data: 7 5 4 4 4 4 5 5 6 6
26808 130/300 Read block 1592 len 28672
26809 131/300 Write block 1595 len 36864 data: 4 3 3 3 3 3 3 3 2
26810 132/300 Write block 2176 len 12288 data: 4 4 5
26811 133/300 Read block 408 len 28672
26812 134/300 Flush
26813 135/300 Flush
26814 136/300 Read block 1810 len 40960
26815 137/300 Read block 4043 len 24576
26816 138/300 Read block 4745 len 12288
26817 139/300 Write block 2083 len 8192 data: 6 5
26818 140/300 Flush
26819 141/300 Write block 2279 len 12288 data: 4 5 4
26820 142/300 Write block 4731 len 24576 data: 4 4 6 5 4 4
26821 143/300 Read block 4692 len 40960
26822 144/300 Read block 1892 len 20480
26823 145/300 Read block 594 len 4096
26824 146/300 Flush
26825 147/300 Read block 1822 len 32768
26826 148/300 Read block 1224 len 20480
26827 149/300 Flush
26828 150/300 Read block 2238 len 8192
26829 151/300 Read block 2892 len 32768
26830 152/300 Read block 2212 len 32768
26831 153/300 Read block 4967 len 4096
26832 154/300 Write block 3624 len 8192 data: 6 6
26833 155/300 Read block 3196 len 8192
26834 156/300 Read block 1886 len 40960
26835 157/300 Read block 706 len 20480
26836 158/300 Read block 819 len 32768
26837 159/300 Read block 1759 len 16384
26838 160/300 Flush
26839 161/300 Read block 3165 len 36864
26840 162/300 Flush
26841 163/300 Write block 1695 len 8192 data: 3 3
26842 164/300 Write block 4232 len 40960 data: 7 6 5 4 5 4 4 4 4 4
26843 165/300 Read block 3486 len 28672
26844 166/300 Read block 2975 len 20480
26845 167/300 Read block 185 len 8192
26846 168/300 Read block 2842 len 20480
26847 169/300 Write block 1144 len 20480 data: 7 6 6 4 4
26848 170/300 Read block 1075 len 20480
26849 171/300 Write block 2295 len 4096 data: 5
26850 172/300 Flush
26851 173/300 Flush
26852 174/300 Flush
26853 175/300 Read block 50 len 12288
26854 176/300 Read block 2499 len 40960
26855 177/300 Read block 4155 len 40960
26856 178/300 Read block 2287 len 32768
26857 179/300 Write block 2403 len 32768 data: 3 4 4 4 4 4 4 4
26858 180/300 Write block 533 len 36864 data: 6 4 4 4 4 5 5 3 4
26859 181/300 Write block 224 len 24576 data: 8 9 9 7 7 9
26860 182/300 Read block 3747 len 40960
26861 183/300 Write block 1340 len 40960 data: 5 7 7 6 6 5 7 7 7 7
26862 184/300 Write block 4324 len 32768 data: 4 4 5 4 4 5 5 6
26863 185/300 Read block 4839 len 24576
26864 186/300 Flush
26865 187/300 Read block 361 len 8192
26866 188/300 Read block 2826 len 24576
26867 189/300 Write block 1312 len 24576 data: 7 6 5 6 6 6
26868 190/300 Write block 1511 len 20480 data: 10 10 9 10 9
26869 191/300 Read block 1940 len 28672
26870 192/300 Write block 600 len 28672 data: 7 6 6 7 5 5 5
26871 193/300 Write block 2808 len 4096 data: 5
26872 194/300 Write block 3403 len 12288 data: 6 7 8
26873 195/300 Write block 4935 len 32768 data: 4 4 4 3 4 5 6 7
26874 196/300 Write block 2418 len 40960 data: 4 6 7 5 6 6 7 7 6 5
26875 197/300 Read block 4181 len 4096
26876 198/300 Read block 3191 len 12288
26877 199/300 Read block 6 len 24576
26878 200/300 Read block 4847 len 8192
26879 201/300 Write block 2573 len 12288 data: 3 3 3
26880 202/300 Read block 3643 len 8192
26881 203/300 Read block 4225 len 16384
26882 204/300 Write block 4316 len 20480 data: 6 5 5 4 5
26883 205/300 Write block 4396 len 40960 data: 6 8 8 7 7 4 3 3 3 3
26884 206/300 Read block 1782 len 36864
26885 207/300 Read block 151 len 36864
26886 208/300 Read block 3040 len 28672
26887 209/300 Flush
26888 210/300 Write block 2554 len 32768 data: 4 4 5 6 6 6 7 8
26889 211/300 Write block 1840 len 40960 data: 6 6 6 8 9 8 7 8 7 7
26890 212/300 Write block 3283 len 16384 data: 9 7 5 4
26891 213/300 Read block 1952 len 4096
26892 214/300 Write block 2419 len 40960 data: 7 8 6 7 7 8 8 7 6 6
26893 215/300 Read block 2839 len 12288
26894 216/300 Flush
26895 217/300 Read block 1102 len 4096
26896 218/300 Write block 601 len 12288 data: 7 7 8
26897 219/300 Read block 3501 len 28672
26898 220/300 Read block 1943 len 32768
26899 221/300 Write block 3416 len 40960 data: 7 9 8 7 7 5 6 5 5 5
26900 222/300 Write block 3499 len 40960 data: 6 7 7 7 8 8 7 4 5 5
26901 223/300 Write block 231 len 28672 data: 6 5 6 7 7 6 6
26902 224/300 Write block 2951 len 20480 data: 6 7 6 5 5
26903 225/300 Read block 87 len 16384
26904 226/300 Read block 3639 len 12288
26905 227/300 Write block 4884 len 4096 data: 2
26906 228/300 Write block 766 len 20480 data: 4 4 3 3 5
26907 229/300 Write block 3217 len 36864 data: 3 3 4 4 4 5 5 4 5
26908 230/300 Read block 2911 len 12288
26909 231/300 Read block 1480 len 8192
26910 232/300 Write block 961 len 24576 data: 3 2 4 5 4 5
26911 233/300 Read block 2059 len 24576
26912 234/300 Read block 4639 len 32768
26913 235/300 Read block 3525 len 28672
26914 236/300 Flush
26915 237/300 Write block 889 len 40960 data: 6 7 6 7 8 6 6 5 4 3
26916 238/300 Write block 4927 len 32768 data: 5 6 5 5 5 3 4 4
26917 239/300 Write block 611 len 20480 data: 4 4 5 8 7
26918 240/300 Read block 458 len 8192
26919 241/300 Write block 2645 len 12288 data: 6 7 8
26920 242/300 Read block 4620 len 36864
26921 243/300 Read block 1346 len 32768
26922 244/300 Read block 4335 len 8192
26923 245/300 Read block 3787 len 20480
26924 246/300 Write block 3820 len 16384 data: 5 4 5 4
26925 247/300 Read block 4283 len 4096
26926 248/300 Read block 3961 len 12288
26927 249/300 Write block 4699 len 4096 data: 4
26928 250/300 Read block 3350 len 32768
26929 251/300 Flush
26930 252/300 Read block 1475 len 40960
26931 253/300 Read block 791 len 4096
26932 254/300 Read block 4516 len 4096
26933 255/300 Read block 3889 len 28672
26934 256/300 Read block 3811 len 32768
26935 257/300 Write block 701 len 40960 data: 6 6 6 6 5 5 4 3 4 5
26936 258/300 Write block 104 len 12288 data: 5 5 5
26937 259/300 Flush
26938 260/300 Flush
26939 261/300 Write block 3973 len 28672 data: 5 5 7 7 7 7 5
26940 262/300 Write block 1628 len 36864 data: 4 4 3 3 2 2 2 2 3
26941 263/300 Read block 3154 len 24576
26942 264/300 Write block 2219 len 40960 data: 4 5 3 3 4 6 5 5 8 8
26943 265/300 Write block 3171 len 12288 data: 6 6 7
26944 266/300 Write block 1266 len 16384 data: 6 7 4 4
26945 267/300 Write block 1132 len 12288 data: 4 2 3
26946 268/300 Read block 4319 len 36864
26947 269/300 Write block 820 len 8192 data: 6 6
26948 270/300 Read block 3632 len 4096
26949 271/300 Flush
26950 272/300 Write block 4692 len 12288 data: 4 4 5
26951 273/300 Read block 458 len 16384
26952 274/300 Write block 852 len 28672 data: 6 4 5 7 5 4 5
26953 275/300 Read block 132 len 28672
26954 276/300 Read block 802 len 16384
26955 277/300 Read block 1712 len 12288
26956 278/300 Write block 3167 len 20480 data: 5 4 3 3 7
26957 279/300 Write block 459 len 32768 data: 6 6 8 8 9 9 7 7
26958 280/300 Read block 2095 len 12288
26959 281/300 Read block 774 len 4096
26960 282/300 Read block 605 len 16384
26961 283/300 Write block 3790 len 16384 data: 8 9 8 9
26962 284/300 Write block 4078 len 24576 data: 2 3 3 3 3 4
26963 285/300 Read block 1744 len 20480
26964 286/300 Read block 3389 len 36864
26965 287/300 Read block 81 len 32768
26966 288/300 Read block 1458 len 40960
26967 289/300 Read block 2172 len 36864
26968 290/300 Read block 4813 len 4096
26969 291/300 Read block 2859 len 4096
26970 292/300 Read block 584 len 28672
26971 293/300 Write block 4824 len 24576 data: 3 3 4 4 5 5
26972 294/300 Write block 3547 len 40960 data: 8 6 6 4 3 3 4 6 6 5
26973 295/300 Read block 2708 len 20480
26974 296/300 Read block 2806 len 8192
26975 297/300 Flush
26976 298/300 Write block 2852 len 32768 data: 4 4 6 7 8 8 8 6
26977 299/300 Write block 3125 len 28672 data: 3 2 3 3 4 3 4
26978 300/300 Read block 2383 len 24576
26979 301/300 Flush
26980 Main task has work to do, go find it
26981 got dsc Start(0)
26982 start 0
26983 Jan 21 15:52:51.537 INFO request completed, latency_us: 233, response_code: 204, uri: /start/cid/0, method: POST, req_id: 0cd67ef3-1ef3-4570-91db-d68bf0f805c7, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
26984 Replay: started 0, returned:Ok(())
26985 ----------------------------------------------------------------
26986 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26987 States: Offline Active Active
26988 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26989 8247 Acked Read 10 Sent Done Done false
26990 8248 Acked Write 9 Sent Done Done false
26991 8249 Acked Read 9 Sent Done Done false
26992 8250 Acked Write 5 Sent Done Done false
26993 8251 Acked Write 2 Sent Done Done false
26994 8252 Acked Write 8 Sent Done Done false
26995 8253 Acked Flush 0 Sent Done Done false
26996 8254 Acked Read 1 Sent Done Done false
26997 8255 Acked Write 6 Sent Done Done false
26998 8256 Acked Write 5 Sent Done Done false
26999 8257 Acked Write 8 Sent Done Done false
27000 8258 Acked Read 5 Sent Done Done false
27001 8259 Acked Read 10 Sent Done Done false
27002 8260 Acked Write 4 Sent Done Done false
27003 8261 Acked Read 3 Sent Done Done false
27004 8262 Acked Write 5 Sent Done Done false
27005 8263 Acked Read 9 Sent Done Done false
27006 8264 Acked Read 9 Sent Done Done false
27007 8265 Acked Write 6 Sent Done Done false
27008 8266 Acked Write 4 Sent Done Done false
27009 8267 Acked Read 4 Sent Done Done false
27010 8268 Acked Read 2 Sent Done Done false
27011 8269 Acked Read 2 Sent Done Done false
27012 8270 Acked Write 3 Sent Done Done false
27013 8271 Acked Read 4 Sent Done Done false
27014 8272 Acked Write 1 Sent Done Done false
27015 8273 Acked Write 6 Sent Done Done false
27016 8274 Acked Read 7 Sent Done Done false
27017 8275 Acked Write 7 Sent Done Done false
27018 8276 Acked Write 1 Sent Done Done false
27019 8277 Acked Read 7 Sent Done Done false
27020 8278 Acked Read 5 Sent Done Done false
27021 8279 Acked Write 9 Sent Done Done false
27022 8280 Acked Read 1 Sent Done Done false
27023 8281 Acked Write 4 Sent Done Done false
27024 8282 Acked Read 1 Sent Done Done false
27025 8283 Acked Read 6 Sent Done Done false
27026 8284 Acked Read 7 Sent Done Done false
27027 8285 Acked Read 6 Sent Done Done false
27028 8286 Acked Read 8 Sent Done Done false
27029 8287 Acked Write 1 Sent Done Done false
27030 8288 Acked Read 10 Sent Done Done false
27031 8289 Acked Read 1 Sent Done Done false
27032 8290 Acked Read 2 Sent Done Done false
27033 8291 Acked Read 7 Sent Done Done false
27034 8292 Acked Read 1 Sent Done Done false
27035 8293 Acked Write 1 Sent Done Done false
27036 8294 Acked Read 6 Sent Done Done false
27037 8295 Acked Read 3 Sent Done Done false
27038 8296 Acked Read 10 Sent Done Done false
27039 8297 Acked Read 6 Sent Done Done false
27040 8298 Acked Write 4 Sent Done Done false
27041 8299 Acked Write 8 Sent Done Done false
27042 8300 Acked Read 8 Sent Done Done false
27043 8301 Acked Read 3 Sent Done Done false
27044 8302 Acked Flush 0 Sent Done Done false
27045 8303 Acked Read 9 Sent Done Done false
27046 8304 Acked Read 1 Sent Done Done false
27047 8305 Acked Read 2 Sent Done Done false
27048 8306 Acked Read 2 Sent Done Done false
27049 8307 Acked Write 2 Sent Done Done false
27050 8308 Acked Write 10 Sent Done Done false
27051 8309 Acked Read 6 Sent Done Done false
27052 8310 Acked Write 10 Sent Done Done false
27053 8311 Acked Write 10 Sent Done Done false
27054 8312 Acked Write 9 Sent Done Done false
27055 8313 Acked Flush 0 Sent Done Done false
27056 8314 Acked Write 1 Sent Done Done false
27057 8315 Acked Write 6 Sent Done Done false
27058 8316 Acked Write 8 Sent Done Done false
27059 8317 Acked Write 8 Sent Done Done false
27060 8318 Acked Flush 0 Sent Done Done false
27061 8319 Acked Write 3 Sent Done Done false
27062 8320 Acked Read 5 Sent Done Done false
27063 8321 Acked Read 5 Sent Done Done false
27064 8322 Acked Flush 0 Sent Done Done false
27065 8323 Acked Write 5 Sent Done Done false
27066 8324 Acked Write 7 Sent Done Done false
27067 8325 Acked Read 7 Sent Done Done false
27068 8326 Acked Read 2 Sent Done Done false
27069 8327 Acked Read 3 Sent Done Done false
27070 8328 Acked Read 5 Sent Done Done false
27071 8329 Acked Flush 0 Sent Done Done false
27072 8330 Acked Read 2 Sent Done Done false
27073 8331 Acked Read 10 Sent Done Done false
27074 8332 Acked Write 2 Sent Done Done false
27075 8333 Acked Read 3 Sent Done Done false
27076 8334 Acked Read 2 Sent Done Done false
27077 8335 Acked Read 5 Sent Done Done false
27078 8336 Acked Write 1 Sent Done Done false
27079 8337 Acked Flush 0 Sent Done Done false
27080 8338 Acked Read 4 Sent Done Done false
27081 8339 Acked Write 6 Sent Done Done false
27082 8340 Acked Write 1 Sent Done Done false
27083 8341 Acked Write 8 Sent Done Done false
27084 8342 Acked Read 8 Sent Done Done false
27085 8343 Acked Read 9 Sent Done Done false
27086 8344 Acked Read 1 Sent Done Done false
27087 8345 Acked Flush 0 Sent Done Done false
27088 8346 Acked Flush 0 Sent Done Done false
27089 8347 Acked Read 6 Sent Done Done false
27090 8348 Acked Read 7 Sent Done Done false
27091 8349 Acked Write 2 Sent Done Done false
27092 8350 Acked Read 4 Sent Done Done false
27093 8351 Acked Write 7 Sent Done Done false
27094 8352 Acked Flush 0 Sent Done Done false
27095 8353 Acked Read 10 Sent Done Done false
27096 8354 Acked Write 1 Sent Done Done false
27097 8355 Acked Read 7 Sent Done Done false
27098 8356 Acked Read 2 Sent Done Done false
27099 8357 Acked Read 6 Sent Done Done false
27100 8358 Acked Read 4 Sent Done Done false
27101 8359 Acked Write 9 Sent Done Done false
27102 8360 Acked Write 9 Sent Done Done false
27103 8361 Acked Write 3 Sent Done Done false
27104 8362 Acked Write 4 Sent Done Done false
27105 8363 Acked Read 8 Sent Done Done false
27106 8364 Acked Read 6 Sent Done Done false
27107 8365 Acked Read 9 Sent Done Done false
27108 8366 Acked Read 5 Sent Done Done false
27109 8367 Acked Write 1 Sent Done Done false
27110 8368 Acked Write 8 Sent Done Done false
27111 8369 Acked Write 5 Sent Done Done false
27112 8370 Acked Read 3 Sent Done Done false
27113 8371 Acked Read 8 Sent Done Done false
27114 8372 Acked Read 10 Sent Done Done false
27115 8373 Acked Flush 0 Sent Done Done false
27116 8374 Acked Read 5 Sent Done Done false
27117 8375 Acked Write 10 Sent Done Done false
27118 8376 Acked Read 7 Sent Done Done false
27119 8377 Acked Write 9 Sent Done Done false
27120 8378 Acked Write 3 Sent Done Done false
27121 8379 Acked Read 7 Sent Done Done false
27122 8380 Acked Flush 0 Sent Done Done false
27123 8381 Acked Flush 0 Sent Done Done false
27124 8382 Acked Read 10 Sent Done Done false
27125 8383 Acked Read 6 Sent Done Done false
27126 8384 Acked Read 3 Sent Done Done false
27127 8385 Acked Write 2 Sent Done Done false
27128 8386 Acked Flush 0 Sent Done Done false
27129 8387 Acked Write 3 Sent Done Done false
27130 8388 Acked Write 6 Sent Done Done false
27131 8389 Acked Read 10 Sent Done Done false
27132 8390 Acked Read 5 Sent Done Done false
27133 8391 Acked Read 1 Sent Done Done false
27134 8392 Acked Flush 0 Sent Done Done false
27135 8393 Acked Read 8 Sent Done Done false
27136 8394 Acked Read 5 Sent Done Done false
27137 8395 Acked Flush 0 Sent Done Done false
27138 8396 Acked Read 2 Sent Done Done false
27139 8397 Acked Read 8 Sent Done Done false
27140 8398 Acked Read 8 Sent Done Done false
27141 8399 Acked Read 1 Sent Done Done false
27142 8400 Acked Write 2 Sent Done Done false
27143 8401 Acked Read 2 Sent Done Done false
27144 8402 Acked Read 10 Sent Done Done false
27145 8403 Acked Read 5 Sent Done Done false
27146 8404 Acked Read 8 Sent Done Done false
27147 8405 Acked Read 4 Sent Done Done false
27148 8406 Acked Flush 0 Sent Done Done false
27149 8407 Acked Read 9 Sent Done Done false
27150 8408 Acked Flush 0 Sent Done Done false
27151 8409 Acked Write 2 Sent Done Done false
27152 8410 Acked Write 10 Sent Done Done false
27153 8411 Acked Read 7 Sent Done Done false
27154 8412 Acked Read 5 Sent Done Done false
27155 8413 Acked Read 2 Sent Done Done false
27156 8414 Acked Read 5 Sent Done Done false
27157 8415 Acked Write 5 Sent Done Done false
27158 8416 Acked Read 5 Sent Done Done false
27159 8417 Acked Write 1 Sent Done Done false
27160 8418 Acked Flush 0 Sent Done Done false
27161 8419 Acked Flush 0 Sent Done Done false
27162 8420 Acked Flush 0 Sent Done Done false
27163 8421 Acked Read 3 Sent Done Done false
27164 8422 Acked Read 10 Sent Done Done false
27165 8423 Acked Read 10 Sent Done Done false
27166 8424 Acked Read 8 Sent Done Done false
27167 8425 Acked Write 8 Sent Done Done false
27168 8426 Acked Write 9 Sent Done Done false
27169 8427 Acked Write 6 Sent Done Done false
27170 8428 Acked Read 10 Sent Done Done false
27171 8429 Acked Write 10 Sent Done Done false
27172 8430 Acked Write 8 Sent Done Done false
27173 8431 Acked Read 6 Sent Done Done false
27174 8432 Acked Flush 0 Sent Done Done false
27175 8433 Acked Read 2 Sent Done Done false
27176 8434 Acked Read 6 Sent Done Done false
27177 8435 Acked Write 6 Sent Done Done false
27178 8436 Acked Write 5 Sent Done Done false
27179 8437 Acked Read 7 Sent Done Done false
27180 8438 Acked Write 7 Sent Done Done false
27181 8439 Acked Write 1 Sent Done Done false
27182 8440 Acked Write 3 Sent Done Done false
27183 8441 Acked Write 8 Sent Done Done false
27184 8442 Acked Write 10 Sent Done Done false
27185 8443 Acked Read 1 Sent Done Done false
27186 8444 Acked Read 3 Sent Done Done false
27187 8445 Acked Read 6 Sent Done Done false
27188 8446 Acked Read 2 Sent Done Done false
27189 8447 Acked Write 3 Sent Done Done false
27190 8448 Acked Read 2 Sent Done Done false
27191 8449 Acked Read 4 Sent Done Done false
27192 8450 Acked Write 5 Sent Done Done false
27193 8451 Acked Write 10 Sent Done Done false
27194 8452 Acked Read 9 Sent Done Done false
27195 8453 Acked Read 9 Sent Done Done false
27196 8454 Acked Read 7 Sent Done Done false
27197 8455 Acked Flush 0 Sent Done Done false
27198 8456 Acked Write 8 Sent Done Done false
27199 8457 Acked Write 10 Sent Done Done false
27200 8458 Acked Write 4 Sent Done Done false
27201 8459 Acked Read 1 Sent Done Done false
27202 8460 Acked Write 10 Sent Done Done false
27203 8461 Acked Read 3 Sent Done Done false
27204 8462 Acked Flush 0 Sent Done Done false
27205 8463 Acked Read 1 Sent Done Done false
27206 8464 Acked Write 3 Sent Done Done false
27207 8465 Acked Read 7 Sent Done Done false
27208 8466 Acked Read 8 Sent Done Done false
27209 8467 Acked Write 10 Sent Done Done false
27210 8468 Acked Write 10 Sent Done Done false
27211 8469 Acked Write 7 Sent Done Done false
27212 8470 Acked Write 5 Sent Done Done false
27213 8471 Acked Read 4 Sent Done Done false
27214 8472 Acked Read 3 Sent Done Done false
27215 8473 Acked Write 1 Sent Done Done false
27216 8474 Acked Write 5 Sent Done Done false
27217 8475 Acked Write 9 Sent Done Done false
27218 8476 Acked Read 3 Sent Done Done false
27219 8477 Acked Read 2 Sent Done Done false
27220 8478 Acked Write 6 Sent Done Done false
27221 8479 Acked Read 6 Sent Done Done false
27222 8480 Acked Read 8 Sent Done Done false
27223 8481 Acked Read 7 Sent Done Done false
27224 8482 Acked Flush 0 Sent Done Done false
27225 8483 Acked Write 10 Sent Done Done false
27226 8484 Acked Write 8 Sent Done Done false
27227 8485 Acked Write 5 Sent Done Done false
27228 8486 Acked Read 2 Sent Done Done false
27229 8487 Acked Write 3 Sent Done Done false
27230 8488 Acked Read 9 Sent Done Done false
27231 8489 Acked Read 8 Sent Done Done false
27232 8490 Acked Read 2 Sent Done Done false
27233 8491 Acked Read 5 Sent Done Done false
27234 8492 Acked Write 4 Sent Done Done false
27235 8493 Acked Read 1 Sent Done Done false
27236 8494 Acked Read 3 Sent Done Done false
27237 8495 Acked Write 1 Sent Done Done false
27238 8496 Acked Read 8 Sent Done Done false
27239 8497 Acked Flush 0 Sent Done Done false
27240 8498 Acked Read 10 Sent Done Done false
27241 8499 Acked Read 1 Sent Done Done false
27242 8500 Acked Read 1 Sent Done Done false
27243 8501 Acked Read 7 Sent Done Done false
27244 8502 Acked Read 8 Sent Done Done false
27245 8503 Acked Write 10 Sent Done Done false
27246 8504 Acked Write 3 Sent Done Done false
27247 8505 Acked Flush 0 Sent Done Done false
27248 8506 Acked Flush 0 Sent Done Done false
27249 8507 Acked Write 7 Sent Done Done false
27250 8508 Acked Write 9 Sent Done Done false
27251 8509 Acked Read 6 Sent Done Done false
27252 8510 Acked Write 10 Sent Done Done false
27253 8511 Acked Write 3 Sent Done Done false
27254 8512 Acked Write 4 Sent Done Done false
27255 8513 Acked Write 3 Sent Done Done false
27256 8514 Acked Read 9 Sent Done Done false
27257 8515 Acked Write 2 Sent Done Done false
27258 8516 Acked Read 1 Sent Done Done false
27259 8517 Acked Flush 0 Sent Done Done false
27260 8518 Acked Write 3 Sent Done Done false
27261 8519 Acked Read 4 Sent Done Done false
27262 8520 Acked Write 7 Sent Done Done false
27263 8521 Acked Read 7 Sent Done Done false
27264 8522 Acked Read 4 Sent Done Done false
27265 8523 Acked Read 3 Sent Done Done false
27266 8524 Acked Write 5 Sent Done Done false
27267 8525 Acked Write 8 Sent Done Done false
27268 8526 Acked Read 3 Sent Done Done false
27269 8527 Acked Read 1 Sent Done Done false
27270 8528 Acked Read 4 Sent Done Done false
27271 8529 Acked Write 4 Sent Done Done false
27272 8530 Acked Write 6 Sent Done Done false
27273 8531 Acked Read 5 Sent Done Done false
27274 8532 Acked Read 9 Sent Done Done false
27275 8533 Acked Read 8 Sent Done Done false
27276 8534 Acked Read 10 Sent Done Done false
27277 8535 Acked Read 9 Sent Done Done false
27278 8536 Acked Read 1 Sent Done Done false
27279 8537 Acked Read 1 Sent Done Done false
27280 8538 Acked Read 7 Sent Done Done false
27281 8539 Acked Write 6 Sent Done Done false
27282 8540 Acked Write 10 Sent Done Done false
27283 8541 Acked Read 5 Sent Done Done false
27284 8542 Acked Read 2 Sent Done Done false
27285 8543 Acked Flush 0 Sent Done Done false
27286 8544 Acked Write 8 Sent Done Done false
27287 8545 Acked Write 7 Sent Done Done false
27288 8546 Acked Read 6 Sent Done Done false
27289 8547 Acked Flush 0 Sent Done Done false
27290 STATES DS:0 DS:1 DS:2 TOTAL
27291 Sent 301 0 0 301
27292 Done 0 301 301 602
27293 Skipped 0 0 0 0
27294 Error 0 0 0 0
27295 Last Flush: 8246 8547 8547
27296 Downstairs last five completed: 8246 8245 8244 8243 8242
27297 Upstairs last five completed: 8547 8546 8545 8544 8543
27298 CLIENT: Up:0 ds:301 act:2
27299 [8810] got start action
27300 [8810] I am going to restart
27301 Starting downstairs at port 8810
27302 Make output file at "/tmp/dsc/downstairs-8810.txt"
27303 [8810][0] reports Starting
27304 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1225)
27305 [8810][0] reports Running
27306 ----------------------------------------------------------------
27307 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27308 States: Offline Active Active
27309 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27310 8247 Acked Read 10 Sent Done Done false
27311 8248 Acked Write 9 Sent Done Done false
27312 8249 Acked Read 9 Sent Done Done false
27313 8250 Acked Write 5 Sent Done Done false
27314 8251 Acked Write 2 Sent Done Done false
27315 8252 Acked Write 8 Sent Done Done false
27316 8253 Acked Flush 0 Sent Done Done false
27317 8254 Acked Read 1 Sent Done Done false
27318 8255 Acked Write 6 Sent Done Done false
27319 8256 Acked Write 5 Sent Done Done false
27320 8257 Acked Write 8 Sent Done Done false
27321 8258 Acked Read 5 Sent Done Done false
27322 8259 Acked Read 10 Sent Done Done false
27323 8260 Acked Write 4 Sent Done Done false
27324 8261 Acked Read 3 Sent Done Done false
27325 8262 Acked Write 5 Sent Done Done false
27326 8263 Acked Read 9 Sent Done Done false
27327 8264 Acked Read 9 Sent Done Done false
27328 8265 Acked Write 6 Sent Done Done false
27329 8266 Acked Write 4 Sent Done Done false
27330 8267 Acked Read 4 Sent Done Done false
27331 8268 Acked Read 2 Sent Done Done false
27332 8269 Acked Read 2 Sent Done Done false
27333 8270 Acked Write 3 Sent Done Done false
27334 8271 Acked Read 4 Sent Done Done false
27335 8272 Acked Write 1 Sent Done Done false
27336 8273 Acked Write 6 Sent Done Done false
27337 8274 Acked Read 7 Sent Done Done false
27338 8275 Acked Write 7 Sent Done Done false
27339 8276 Acked Write 1 Sent Done Done false
27340 8277 Acked Read 7 Sent Done Done false
27341 8278 Acked Read 5 Sent Done Done false
27342 8279 Acked Write 9 Sent Done Done false
27343 8280 Acked Read 1 Sent Done Done false
27344 8281 Acked Write 4 Sent Done Done false
27345 8282 Acked Read 1 Sent Done Done false
27346 8283 Acked Read 6 Sent Done Done false
27347 8284 Acked Read 7 Sent Done Done false
27348 8285 Acked Read 6 Sent Done Done false
27349 8286 Acked Read 8 Sent Done Done false
27350 8287 Acked Write 1 Sent Done Done false
27351 8288 Acked Read 10 Sent Done Done false
27352 8289 Acked Read 1 Sent Done Done false
27353 8290 Acked Read 2 Sent Done Done false
27354 8291 Acked Read 7 Sent Done Done false
27355 8292 Acked Read 1 Sent Done Done false
27356 8293 Acked Write 1 Sent Done Done false
27357 8294 Acked Read 6 Sent Done Done false
27358 8295 Acked Read 3 Sent Done Done false
27359 8296 Acked Read 10 Sent Done Done false
27360 8297 Acked Read 6 Sent Done Done false
27361 8298 Acked Write 4 Sent Done Done false
27362 8299 Acked Write 8 Sent Done Done false
27363 8300 Acked Read 8 Sent Done Done false
27364 8301 Acked Read 3 Sent Done Done false
27365 8302 Acked Flush 0 Sent Done Done false
27366 8303 Acked Read 9 Sent Done Done false
27367 8304 Acked Read 1 Sent Done Done false
27368 8305 Acked Read 2 Sent Done Done false
27369 8306 Acked Read 2 Sent Done Done false
27370 8307 Acked Write 2 Sent Done Done false
27371 8308 Acked Write 10 Sent Done Done false
27372 8309 Acked Read 6 Sent Done Done false
27373 8310 Acked Write 10 Sent Done Done false
27374 8311 Acked Write 10 Sent Done Done false
27375 8312 Acked Write 9 Sent Done Done false
27376 8313 Acked Flush 0 Sent Done Done false
27377 8314 Acked Write 1 Sent Done Done false
27378 8315 Acked Write 6 Sent Done Done false
27379 8316 Acked Write 8 Sent Done Done false
27380 8317 Acked Write 8 Sent Done Done false
27381 8318 Acked Flush 0 Sent Done Done false
27382 8319 Acked Write 3 Sent Done Done false
27383 8320 Acked Read 5 Sent Done Done false
27384 8321 Acked Read 5 Sent Done Done false
27385 8322 Acked Flush 0 Sent Done Done false
27386 8323 Acked Write 5 Sent Done Done false
27387 8324 Acked Write 7 Sent Done Done false
27388 8325 Acked Read 7 Sent Done Done false
27389 8326 Acked Read 2 Sent Done Done false
27390 8327 Acked Read 3 Sent Done Done false
27391 8328 Acked Read 5 Sent Done Done false
27392 8329 Acked Flush 0 Sent Done Done false
27393 8330 Acked Read 2 Sent Done Done false
27394 8331 Acked Read 10 Sent Done Done false
27395 8332 Acked Write 2 Sent Done Done false
27396 8333 Acked Read 3 Sent Done Done false
27397 8334 Acked Read 2 Sent Done Done false
27398 8335 Acked Read 5 Sent Done Done false
27399 8336 Acked Write 1 Sent Done Done false
27400 8337 Acked Flush 0 Sent Done Done false
27401 8338 Acked Read 4 Sent Done Done false
27402 8339 Acked Write 6 Sent Done Done false
27403 8340 Acked Write 1 Sent Done Done false
27404 8341 Acked Write 8 Sent Done Done false
27405 8342 Acked Read 8 Sent Done Done false
27406 8343 Acked Read 9 Sent Done Done false
27407 8344 Acked Read 1 Sent Done Done false
27408 8345 Acked Flush 0 Sent Done Done false
27409 8346 Acked Flush 0 Sent Done Done false
27410 8347 Acked Read 6 Sent Done Done false
27411 8348 Acked Read 7 Sent Done Done false
27412 8349 Acked Write 2 Sent Done Done false
27413 8350 Acked Read 4 Sent Done Done false
27414 8351 Acked Write 7 Sent Done Done false
27415 8352 Acked Flush 0 Sent Done Done false
27416 8353 Acked Read 10 Sent Done Done false
27417 8354 Acked Write 1 Sent Done Done false
27418 8355 Acked Read 7 Sent Done Done false
27419 8356 Acked Read 2 Sent Done Done false
27420 8357 Acked Read 6 Sent Done Done false
27421 8358 Acked Read 4 Sent Done Done false
27422 8359 Acked Write 9 Sent Done Done false
27423 8360 Acked Write 9 Sent Done Done false
27424 8361 Acked Write 3 Sent Done Done false
27425 8362 Acked Write 4 Sent Done Done false
27426 8363 Acked Read 8 Sent Done Done false
27427 8364 Acked Read 6 Sent Done Done false
27428 8365 Acked Read 9 Sent Done Done false
27429 8366 Acked Read 5 Sent Done Done false
27430 8367 Acked Write 1 Sent Done Done false
27431 8368 Acked Write 8 Sent Done Done false
27432 8369 Acked Write 5 Sent Done Done false
27433 8370 Acked Read 3 Sent Done Done false
27434 8371 Acked Read 8 Sent Done Done false
27435 8372 Acked Read 10 Sent Done Done false
27436 8373 Acked Flush 0 Sent Done Done false
27437 8374 Acked Read 5 Sent Done Done false
27438 8375 Acked Write 10 Sent Done Done false
27439 8376 Acked Read 7 Sent Done Done false
27440 8377 Acked Write 9 Sent Done Done false
27441 8378 Acked Write 3 Sent Done Done false
27442 8379 Acked Read 7 Sent Done Done false
27443 8380 Acked Flush 0 Sent Done Done false
27444 8381 Acked Flush 0 Sent Done Done false
27445 8382 Acked Read 10 Sent Done Done false
27446 8383 Acked Read 6 Sent Done Done false
27447 8384 Acked Read 3 Sent Done Done false
27448 8385 Acked Write 2 Sent Done Done false
27449 8386 Acked Flush 0 Sent Done Done false
27450 8387 Acked Write 3 Sent Done Done false
27451 8388 Acked Write 6 Sent Done Done false
27452 8389 Acked Read 10 Sent Done Done false
27453 8390 Acked Read 5 Sent Done Done false
27454 8391 Acked Read 1 Sent Done Done false
27455 8392 Acked Flush 0 Sent Done Done false
27456 8393 Acked Read 8 Sent Done Done false
27457 8394 Acked Read 5 Sent Done Done false
27458 8395 Acked Flush 0 Sent Done Done false
27459 8396 Acked Read 2 Sent Done Done false
27460 8397 Acked Read 8 Sent Done Done false
27461 8398 Acked Read 8 Sent Done Done false
27462 8399 Acked Read 1 Sent Done Done false
27463 8400 Acked Write 2 Sent Done Done false
27464 8401 Acked Read 2 Sent Done Done false
27465 8402 Acked Read 10 Sent Done Done false
27466 8403 Acked Read 5 Sent Done Done false
27467 8404 Acked Read 8 Sent Done Done false
27468 8405 Acked Read 4 Sent Done Done false
27469 8406 Acked Flush 0 Sent Done Done false
27470 8407 Acked Read 9 Sent Done Done false
27471 8408 Acked Flush 0 Sent Done Done false
27472 8409 Acked Write 2 Sent Done Done false
27473 8410 Acked Write 10 Sent Done Done false
27474 8411 Acked Read 7 Sent Done Done false
27475 8412 Acked Read 5 Sent Done Done false
27476 8413 Acked Read 2 Sent Done Done false
27477 8414 Acked Read 5 Sent Done Done false
27478 8415 Acked Write 5 Sent Done Done false
27479 8416 Acked Read 5 Sent Done Done false
27480 8417 Acked Write 1 Sent Done Done false
27481 8418 Acked Flush 0 Sent Done Done false
27482 8419 Acked Flush 0 Sent Done Done false
27483 8420 Acked Flush 0 Sent Done Done false
27484 8421 Acked Read 3 Sent Done Done false
27485 8422 Acked Read 10 Sent Done Done false
27486 8423 Acked Read 10 Sent Done Done false
27487 8424 Acked Read 8 Sent Done Done false
27488 8425 Acked Write 8 Sent Done Done false
27489 8426 Acked Write 9 Sent Done Done false
27490 8427 Acked Write 6 Sent Done Done false
27491 8428 Acked Read 10 Sent Done Done false
27492 8429 Acked Write 10 Sent Done Done false
27493 8430 Acked Write 8 Sent Done Done false
27494 8431 Acked Read 6 Sent Done Done false
27495 8432 Acked Flush 0 Sent Done Done false
27496 8433 Acked Read 2 Sent Done Done false
27497 8434 Acked Read 6 Sent Done Done false
27498 8435 Acked Write 6 Sent Done Done false
27499 8436 Acked Write 5 Sent Done Done false
27500 8437 Acked Read 7 Sent Done Done false
27501 8438 Acked Write 7 Sent Done Done false
27502 8439 Acked Write 1 Sent Done Done false
27503 8440 Acked Write 3 Sent Done Done false
27504 8441 Acked Write 8 Sent Done Done false
27505 8442 Acked Write 10 Sent Done Done false
27506 8443 Acked Read 1 Sent Done Done false
27507 8444 Acked Read 3 Sent Done Done false
27508 8445 Acked Read 6 Sent Done Done false
27509 8446 Acked Read 2 Sent Done Done false
27510 8447 Acked Write 3 Sent Done Done false
27511 8448 Acked Read 2 Sent Done Done false
27512 8449 Acked Read 4 Sent Done Done false
27513 8450 Acked Write 5 Sent Done Done false
27514 8451 Acked Write 10 Sent Done Done false
27515 8452 Acked Read 9 Sent Done Done false
27516 8453 Acked Read 9 Sent Done Done false
27517 8454 Acked Read 7 Sent Done Done false
27518 8455 Acked Flush 0 Sent Done Done false
27519 8456 Acked Write 8 Sent Done Done false
27520 8457 Acked Write 10 Sent Done Done false
27521 8458 Acked Write 4 Sent Done Done false
27522 8459 Acked Read 1 Sent Done Done false
27523 8460 Acked Write 10 Sent Done Done false
27524 8461 Acked Read 3 Sent Done Done false
27525 8462 Acked Flush 0 Sent Done Done false
27526 8463 Acked Read 1 Sent Done Done false
27527 8464 Acked Write 3 Sent Done Done false
27528 8465 Acked Read 7 Sent Done Done false
27529 8466 Acked Read 8 Sent Done Done false
27530 8467 Acked Write 10 Sent Done Done false
27531 8468 Acked Write 10 Sent Done Done false
27532 8469 Acked Write 7 Sent Done Done false
27533 8470 Acked Write 5 Sent Done Done false
27534 8471 Acked Read 4 Sent Done Done false
27535 8472 Acked Read 3 Sent Done Done false
27536 8473 Acked Write 1 Sent Done Done false
27537 8474 Acked Write 5 Sent Done Done false
27538 8475 Acked Write 9 Sent Done Done false
27539 8476 Acked Read 3 Sent Done Done false
27540 8477 Acked Read 2 Sent Done Done false
27541 8478 Acked Write 6 Sent Done Done false
27542 8479 Acked Read 6 Sent Done Done false
27543 8480 Acked Read 8 Sent Done Done false
27544 8481 Acked Read 7 Sent Done Done false
27545 8482 Acked Flush 0 Sent Done Done false
27546 8483 Acked Write 10 Sent Done Done false
27547 8484 Acked Write 8 Sent Done Done false
27548 8485 Acked Write 5 Sent Done Done false
27549 8486 Acked Read 2 Sent Done Done false
27550 8487 Acked Write 3 Sent Done Done false
27551 8488 Acked Read 9 Sent Done Done false
27552 8489 Acked Read 8 Sent Done Done false
27553 8490 Acked Read 2 Sent Done Done false
27554 8491 Acked Read 5 Sent Done Done false
27555 8492 Acked Write 4 Sent Done Done false
27556 8493 Acked Read 1 Sent Done Done false
27557 8494 Acked Read 3 Sent Done Done false
27558 8495 Acked Write 1 Sent Done Done false
27559 8496 Acked Read 8 Sent Done Done false
27560 8497 Acked Flush 0 Sent Done Done false
27561 8498 Acked Read 10 Sent Done Done false
27562 8499 Acked Read 1 Sent Done Done false
27563 8500 Acked Read 1 Sent Done Done false
27564 8501 Acked Read 7 Sent Done Done false
27565 8502 Acked Read 8 Sent Done Done false
27566 8503 Acked Write 10 Sent Done Done false
27567 8504 Acked Write 3 Sent Done Done false
27568 8505 Acked Flush 0 Sent Done Done false
27569 8506 Acked Flush 0 Sent Done Done false
27570 8507 Acked Write 7 Sent Done Done false
27571 8508 Acked Write 9 Sent Done Done false
27572 8509 Acked Read 6 Sent Done Done false
27573 8510 Acked Write 10 Sent Done Done false
27574 8511 Acked Write 3 Sent Done Done false
27575 8512 Acked Write 4 Sent Done Done false
27576 8513 Acked Write 3 Sent Done Done false
27577 8514 Acked Read 9 Sent Done Done false
27578 8515 Acked Write 2 Sent Done Done false
27579 8516 Acked Read 1 Sent Done Done false
27580 8517 Acked Flush 0 Sent Done Done false
27581 8518 Acked Write 3 Sent Done Done false
27582 8519 Acked Read 4 Sent Done Done false
27583 8520 Acked Write 7 Sent Done Done false
27584 8521 Acked Read 7 Sent Done Done false
27585 8522 Acked Read 4 Sent Done Done false
27586 8523 Acked Read 3 Sent Done Done false
27587 8524 Acked Write 5 Sent Done Done false
27588 8525 Acked Write 8 Sent Done Done false
27589 8526 Acked Read 3 Sent Done Done false
27590 8527 Acked Read 1 Sent Done Done false
27591 8528 Acked Read 4 Sent Done Done false
27592 8529 Acked Write 4 Sent Done Done false
27593 8530 Acked Write 6 Sent Done Done false
27594 8531 Acked Read 5 Sent Done Done false
27595 8532 Acked Read 9 Sent Done Done false
27596 8533 Acked Read 8 Sent Done Done false
27597 8534 Acked Read 10 Sent Done Done false
27598 8535 Acked Read 9 Sent Done Done false
27599 8536 Acked Read 1 Sent Done Done false
27600 8537 Acked Read 1 Sent Done Done false
27601 8538 Acked Read 7 Sent Done Done false
27602 8539 Acked Write 6 Sent Done Done false
27603 8540 Acked Write 10 Sent Done Done false
27604 8541 Acked Read 5 Sent Done Done false
27605 8542 Acked Read 2 Sent Done Done false
27606 8543 Acked Flush 0 Sent Done Done false
27607 8544 Acked Write 8 Sent Done Done false
27608 8545 Acked Write 7 Sent Done Done false
27609 8546 Acked Read 6 Sent Done Done false
27610 8547 Acked Flush 0 Sent Done Done false
27611 STATES DS:0 DS:1 DS:2 TOTAL
27612 Sent 301 0 0 301
27613 Done 0 301 301 602
27614 Skipped 0 0 0 0
27615 Error 0 0 0 0
27616 Last Flush: 8246 8547 8547
27617 Downstairs last five completed: 8246 8245 8244 8243 8242
27618 Upstairs last five completed: 8547 8546 8545 8544 8543
27619 CLIENT: Up:0 ds:301 act:2
276202025-01-21T15:52:57.294ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276212025-01-21T15:52:57.294ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:45134) = io task client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276222025-01-21T15:52:57.295ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276232025-01-21T15:52:57.295ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276242025-01-21T15:52:57.295ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276252025-01-21T15:52:57.295ZINFOcrucible: send last flush ID to this DS: 8246 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276262025-01-21T15:52:57.295ZINFOcrucible: Replied this last flush ID: 8246 = downstairs client = 0 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276272025-01-21T15:52:57.295ZINFOcrucible: [0] client re-new 301 jobs since flush 8246 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276282025-01-21T15:52:57.296ZINFOcrucible: [0] Replayed 301 jobs since flush: 8246 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27629 ----------------------------------------------------------------
27630 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27631 Downstairs last five completed: 8547 8546 8545 8544 8543
27632 Upstairs last five completed: 8547 8546 8545 8544 8543
27633 CLIENT: Up:0 ds:0 act:3
27634 Replay: All jobs finished
27635 Main task has work to do, go find it
27636 got dsc Stop(1)
27637 stop 1
27638 Jan 21 15:52:59.542 INFO request completed[8820] Got stop action so:false kr:false
27639 , latency_us: 331, response_code: 204, uri: /stop/cid/1, method: POST, req_id: e5cb52cd-ca95-41a4-8258-e50e284427c9, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
276402025-01-21T15:52:59.543ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276412025-01-21T15:52:59.543ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
276422025-01-21T15:52:59.543ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
27643 Jan 21 15:52:59.543 INFO request completed, latency_us: 127, response_code: 200, uri: /state/cid/1, method: GET, req_id: f2d359db-dba5-46fc-8323-f405e73d9473{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:52:59.543387918Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
27644 , remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
27645 [8820] Exited with: ExitStatus(unix_wait_status(9))
27646 [8820][1] reports Exit
27647 Jan 21 15:53:03.545 INFO request completed, latency_us: 219, response_code: 200, uri: /state/cid/1, method: GET, req_id: 888253a4-4522-42b2-81bd-dc47e2262f02, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
27648 001/300 Write block 258 len 28672 data: 4 5 5 6 7 7 4
27649 002/300 Read block 4117 len 24576
27650 003/300 Write block 2302 len 36864 data: 5 6 6 7 5 4 4 2 4
27651 004/300 Write block 24 len 40960 data: 4 5 4 3 3 3 4 5 5 3
27652 005/300 Write block 4005 len 24576 data: 6 7 6 5 5 4
27653 006/300 Write block 3556 len 20480 data: 6 7 8 7 5
27654 007/300 Write block 4421 len 32768 data: 6 7 7 7 7 7 4 5
27655 008/300 Write block 3720 len 8192 data: 8 10
27656 009/300 Write block 4791 len 40960 data: 4 3 3 4 6 5 5 6 7 8
27657 010/300 Write block 3427 len 16384 data: 2 2 2 3
27658 011/300 Write block 2822 len 28672 data: 7 7 6 6 5 7 6
27659 012/300 Write block 3645 len 12288 data: 4 3 3
27660 013/300 Read block 3898 len 32768
27661 014/300 Read block 2288 len 24576
27662 015/300 Read block 1363 len 36864
27663 016/300 Write block 2767 len 32768 data: 5 4 3 3 3 4 3 4
27664 017/300 Read block 3217 len 36864
27665 018/300 Read block 1944 len 32768
27666 019/300 Write block 413 len 16384 data: 6 6 5 5
27667 020/300 Write block 923 len 40960 data: 5 5 5 6 6 7 7 5 6 5
27668 021/300 Read block 777 len 24576
27669 022/300 Write block 4543 len 32768 data: 10 7 8 8 7 4 3 2
27670 023/300 Write block 1487 len 20480 data: 5 4 6 5 4
27671 024/300 Write block 4883 len 20480 data: 2 3 2 2 2
27672 025/300 Read block 791 len 28672
27673 026/300 Read block 3502 len 24576
27674 027/300 Write block 1787 len 40960 data: 6 6 6 5 6 6 6 7 8 8
27675 028/300 Write block 2710 len 24576 data: 8 8 9 8 8 6
27676 029/300 Write block 1037 len 28672 data: 8 9 9 7 7 8 8
27677 030/300 Write block 2563 len 32768 data: 7 5 6 6 5 5 6 5
27678 031/300 Flush
27679 032/300 Flush
27680 033/300 Read block 3434 len 24576
27681 034/300 Write block 3129 len 32768 data: 5 4 5 4 5 5 6 6
27682 035/300 Write block 3074 len 12288 data: 2 2 3
27683 036/300 Write block 1085 len 16384 data: 3 3 3 3
27684 037/300 Read block 1754 len 40960
27685 038/300 Read block 225 len 12288
27686 039/300 Read block 28 len 28672
27687 040/300 Write block 2081 len 4096 data: 8
27688 041/300 Read block 3041 len 24576
27689 042/300 Read block 3319 len 12288
27690 043/300 Read block 1713 len 28672
27691 044/300 Write block 230 len 36864 data: 8 7 6 7 8 8 7 7 6
27692 045/300 Flush
27693 046/300 Write block 1439 len 16384 data: 4 4 4 5
27694 047/300 Read block 2143 len 40960
27695 048/300 Write block 4395 len 24576 data: 8 7 9 9 8 8
27696 049/300 Write block 4819 len 32768 data: 3 3 3 3 3 4 4 5
27697 050/300 Write block 1205 len 20480 data: 4 4 3 3 3
27698 051/300 Write block 3952 len 12288 data: 8 7 6
27699 052/300 Read block 4475 len 4096
27700 053/300 Write block 3264 len 12288 data: 4 4 5
27701 054/300 Write block 3454 len 40960 data: 7 7 7 4 4 4 4 5 5 7
27702 055/300 Read block 4457 len 32768
27703 056/300 Read block 2310 len 20480
27704 057/300 Read block 3653 len 24576
27705 058/300 Read block 3184 len 32768
27706 059/300 Write block 3496 len 16384 data: 5 6 5 7
27707 060/300 Read block 1054 len 4096
27708 061/300 Read block 2435 len 12288
27709 062/300 Write block 1005 len 36864 data: 4 4 3 3 4 5 6 6 5
27710 063/300 Read block 1480 len 36864
27711 064/300 Read block 2899 len 28672
27712 065/300 Read block 2492 len 32768
27713 066/300 Write block 499 len 4096 data: 4
27714 067/300 Write block 841 len 12288 data: 6 7 7
27715 068/300 Read block 532 len 24576
27716 069/300 Read block 808 len 24576
27717 070/300 Write block 3051 len 16384 data: 5 4 3 4
27718 071/300 Read block 4652 len 16384
27719 072/300 Read block 2185 len 40960
27720 073/300 Write block 4975 len 28672 data: 3 4 4 3 3 3 4
27721 074/300 Write block 505 len 40960 data: 6 6 5 6 5 5 3 3 4 4
27722 075/300 Flush
27723 076/300 Read block 4116 len 16384
27724 077/300 Write block 3317 len 20480 data: 4 4 3 3 3
27725 078/300 Flush
27726 079/300 Read block 3397 len 36864
27727 080/300 Write block 1888 len 4096 data: 6
27728 081/300 Read block 3599 len 4096
27729 082/300 Write block 2856 len 4096 data: 9
27730 083/300 Write block 4222 len 16384 data: 5 3 3 3
27731 084/300 Flush
27732 085/300 Write block 1832 len 12288 data: 4 4 4
27733 086/300 Read block 1779 len 40960
27734 087/300 Flush
27735 088/300 Read block 4563 len 40960
27736 089/300 Write block 4479 len 4096 data: 5
27737 090/300 Read block 3944 len 28672
27738 091/300 Write block 4533 len 8192 data: 5 4
27739 092/300 Flush
27740 093/300 Read block 3199 len 36864
27741 094/300 Flush
27742 095/300 Write block 4911 len 28672 data: 8 8 8 9 9 7 6
27743 096/300 Read block 4509 len 36864
27744 097/300 Write block 333 len 32768 data: 3 3 3 4 4 5 6 7
27745 098/300 Read block 2878 len 20480
27746 099/300 Write block 3597 len 28672 data: 4 4 4 5 6 5 4
27747 100/300 Read block 1342 len 36864
27748 101/300 Write block 3485 len 28672 data: 7 7 7 5 4 5 6
27749 102/300 Read block 2980 len 12288
27750 103/300 Write block 817 len 36864 data: 4 5 6 7 7 6 5 6 4
27751 104/300 Flush
27752 105/300 Write block 366 len 24576 data: 3 5 6 5 6 6
27753 106/300 Flush
27754 107/300 Read block 3619 len 20480
27755 108/300 Write block 4283 len 8192 data: 3 2
27756 109/300 Write block 2996 len 8192 data: 5 4
27757 110/300 Read block 3194 len 8192
27758 111/300 Write block 3455 len 32768 data: 8 8 5 5 5 5 6 6
27759 112/300 Flush
27760 113/300 Read block 2053 len 32768
27761 114/300 Write block 268 len 4096 data: 5
27762 115/300 Write block 1827 len 36864 data: 3 3 3 4 4 5 5 5 5
27763 116/300 Write block 2121 len 28672 data: 6 5 5 6 6 5 5
27764 117/300 Flush
27765 118/300 Read block 2639 len 28672
27766 119/300 Write block 1246 len 4096 data: 7
27767 120/300 Read block 4538 len 16384
27768 121/300 Write block 2845 len 32768 data: 2 2 2 3 4 5 4 5
27769 122/300 Read block 2814 len 20480
27770 123/300 Read block 1514 len 12288
27771 124/300 Read block 3948 len 16384
27772 125/300 Write block 3100 len 40960 data: 3 3 3 5 5 6 6 6 5 5
27773 126/300 Read block 2791 len 12288
27774 127/300 Read block 1059 len 16384
27775 128/300 Read block 1833 len 12288
27776 129/300 Write block 474 len 36864 data: 4 4 4 4 4 5 6 5 5
27777 130/300 Read block 507 len 16384
27778 131/300 Write block 2288 len 28672 data: 5 6 6 6 6 5 5
27779 132/300 Read block 3080 len 12288
27780 133/300 Write block 47 len 40960 data: 7 6 7 6 6 6 6 5 4 4
27781 134/300 Write block 4739 len 24576 data: 4 4 4 3 3 4
27782 135/300 Write block 1422 len 12288 data: 4 3 3
27783 136/300 Read block 269 len 12288
27784 137/300 Flush
27785 138/300 Read block 2971 len 8192
27786 139/300 Write block 3004 len 40960 data: 3 3 3 3 3 2 2 2 2 3
27787 140/300 Flush
27788 141/300 Write block 3663 len 12288 data: 3 5 5
27789 142/300 Read block 1602 len 4096
27790 143/300 Write block 2717 len 20480 data: 6 6 5 4 5
27791 144/300 Read block 3408 len 4096
27792 145/300 Read block 707 len 40960
27793 146/300 Write block 2440 len 40960 data: 3 4 4 3 4 3 5 7 5 5
27794 147/300 Read block 283 len 16384
27795 148/300 Write block 2785 len 12288 data: 4 5 6
27796 149/300 Write block 2851 len 20480 data: 5 6 5 7 8
27797 150/300 Flush
27798 151/300 Write block 3002 len 20480 data: 3 3 4 4 4
27799 152/300 Read block 3013 len 32768
27800 153/300 Read block 3170 len 16384
27801 154/300 Write block 2583 len 28672 data: 3 3 3 3 4 5 5
27802 155/300 Flush
27803 156/300 Write block 4695 len 28672 data: 5 4 4 3 5 4 4
27804 157/300 Write block 556 len 32768 data: 8 8 7 4 4 5 5 4
27805 158/300 Read block 4447 len 40960
27806 159/300 Read block 1063 len 32768
27807 160/300 Write block 3636 len 8192 data: 2 2
27808 161/300 Read block 1792 len 4096
27809 162/300 Write block 3409 len 12288 data: 6 5 5
27810 163/300 Read block 3891 len 36864
27811 164/300 Read block 1095 len 12288
27812 165/300 Write block 4912 len 36864 data: 9 9 10 10 8 7 5 5 4
27813 166/300 Read block 3964 len 36864
27814 167/300 Write block 2222 len 32768 data: 4 5 7 6 6 9 9 7
27815 168/300 Read block 1529 len 24576
27816 169/300 Flush
27817 170/300 Flush
27818 171/300 Read block 1485 len 24576
27819 172/300 Read block 4676 len 4096
27820 173/300 Write block 2271 len 32768 data: 7 5 5 4 4 4 4 4
27821 174/300 Read block 3935 len 4096
27822 175/300 Write block 2319 len 40960 data: 5 5 5 5 6 5 5 5 5 4
27823 176/300 Write block 1829 len 4096 data: 4
27824 177/300 Write block 2440 len 40960 data: 4 5 5 4 5 4 6 8 6 6
27825 178/300 Read block 2557 len 8192
27826 179/300 Read block 4804 len 12288
27827 180/300 Read block 963 len 4096
27828 181/300 Write block 3509 len 36864 data: 4 4 4 5 7 7 6 6 6
27829 182/300 Flush
27830 183/300 Write block 535 len 32768 data: 5 5 5 6 6 4 5 3
27831 184/300 Write block 2866 len 28672 data: 4 4 3 3 4 4 4
27832 185/300 Read block 347 len 12288
27833 186/300 Write block 3627 len 24576 data: 7 9 10 6 5 3
27834 187/300 Write block 2692 len 8192 data: 2 2
27835 188/300 Read block 3711 len 20480
27836 189/300 Read block 2943 len 28672
27837 190/300 Read block 815 len 32768
27838 191/300 Read block 4983 len 12288
27839 192/300 Write block 1981 len 32768 data: 7 8 10 10 11 11 9 9
27840 193/300 Read block 2695 len 28672
27841 194/300 Write block 1474 len 40960 data: 5 4 2 6 5 5 3 3 3 5
27842 195/300 Read block 2782 len 32768
27843 196/300 Read block 1372 len 36864
27844 197/300 Flush
27845 198/300 Read block 154 len 20480
27846 199/300 Read block 787 len 36864
27847 200/300 Read block 3806 len 24576
27848 201/300 Read block 555 len 32768
27849 202/300 Write block 1963 len 32768 data: 2 3 3 3 3 2 2 2
27850 203/300 Read block 4669 len 8192
27851 204/300 Read block 1666 len 32768
27852 205/300 Read block 4772 len 4096
27853 206/300 Read block 2125 len 4096
27854 207/300 Write block 3528 len 20480 data: 9 7 6 5 4
27855 208/300 Read block 289 len 36864
27856 209/300 Read block 193 len 40960
27857 210/300 Write block 2283 len 8192 data: 4 2
27858 211/300 Flush
27859 212/300 Write block 1459 len 32768 data: 3 3 2 3 3 3 3 3
27860 213/300 Read block 1570 len 36864
27861 214/300 Read block 1738 len 24576
27862 215/300 Write block 2251 len 32768 data: 6 6 5 4 6 6 6 6
27863 216/300 Read block 87 len 12288
27864 217/300 Read block 3107 len 20480
27865 218/300 Flush
27866 219/300 Read block 2649 len 16384
27867 220/300 Read block 800 len 40960
27868 221/300 Read block 3723 len 12288
27869 222/300 Flush
27870 223/300 Write block 2381 len 24576 data: 2 2 3 3 4 4
27871 224/300 Read block 2783 len 8192
27872 225/300 Read block 3651 len 36864
27873 226/300 Read block 4828 len 16384
27874 227/300 Write block 884 len 40960 data: 5 4 4 5 5 7 8 7 8 9
27875 228/300 Write block 3594 len 32768 data: 4 4 4 5 5 5 6 7
27876 229/300 Read block 137 len 36864
27877 230/300 Flush
27878 231/300 Write block 2412 len 24576 data: 3 4 3 3 3 4
27879 232/300 Write block 2710 len 4096 data: 9
27880 233/300 Write block 3312 len 4096 data: 3
27881 234/300 Read block 3732 len 16384
27882 235/300 Read block 3189 len 32768
27883 236/300 Write block 2860 len 32768 data: 6 5 4 4 2 3 5 5
27884 237/300 Flush
27885 238/300 Read block 100 len 20480
27886 239/300 Write block 2726 len 16384 data: 3 3 3 3
27887 240/300 Read block 4045 len 12288
27888 241/300 Write block 4653 len 36864 data: 6 4 4 4 5 4 4 3 3
27889 242/300 Write block 2284 len 32768 data: 3 2 4 4 6 7 7 7
27890 243/300 Write block 2448 len 36864 data: 7 7 5 5 4 3 3 4 5
27891 244/300 Write block 3844 len 12288 data: 5 4 4
27892 245/300 Write block 4703 len 20480 data: 6 6 4 4 5
27893 246/300 Read block 429 len 36864
27894 247/300 Read block 4315 len 24576
27895 248/300 Read block 2723 len 16384
27896 249/300 Write block 2378 len 24576 data: 3 3 3 3 3 4
27897 250/300 Read block 1579 len 36864
27898 251/300 Read block 3980 len 24576
27899 252/300 Read block 1709 len 32768
27900 253/300 Read block 4377 len 36864
27901 254/300 Flush
27902 255/300 Read block 649 len 8192
27903 256/300 Write block 3151 len 24576 data: 7 7 6 6 5 4
27904 257/300 Read block 3996 len 4096
27905 258/300 Read block 2054 len 24576
27906 259/300 Read block 16 len 12288
27907 260/300 Read block 448 len 28672
27908 261/300 Read block 2776 len 4096
27909 262/300 Flush
27910 263/300 Read block 321 len 40960
27911 264/300 Read block 1224 len 32768
27912 265/300 Write block 315 len 24576 data: 6 5 4 5 4 3
27913 266/300 Write block 4317 len 20480 data: 6 6 5 6 4
27914 267/300 Flush
27915 268/300 Write block 481 len 32768 data: 6 6 7 7 8 7 7 6
27916 269/300 Read block 2774 len 36864
27917 270/300 Write block 722 len 16384 data: 7 6 6 6
27918 271/300 Write block 1361 len 8192 data: 5 8
27919 272/300 Write block 715 len 32768 data: 5 8 9 9 9 8 5 8
27920 273/300 Write block 4706 len 4096 data: 5
27921 274/300 Write block 1461 len 4096 data: 3
27922 275/300 Write block 4410 len 24576 data: 7 7 5 4 4 4
27923 276/300 Write block 989 len 4096 data: 5
27924 277/300 Write block 1303 len 20480 data: 4 6 5 4 4
27925 278/300 Read block 1881 len 24576
27926 279/300 Read block 3201 len 20480
27927 280/300 Read block 3 len 32768
27928 281/300 Flush
27929 282/300 Write block 374 len 24576 data: 6 5 4 4 3 3
27930 283/300 Read block 2527 len 16384
27931 284/300 Write block 2920 len 32768 data: 4 6 6 5 4 5 6 6
27932 285/300 Read block 86 len 16384
27933 286/300 Write block 4936 len 40960 data: 5 5 4 5 6 7 8 5 6 6
27934 287/300 Flush
27935 288/300 Write block 1554 len 4096 data: 4
27936 289/300 Read block 3644 len 16384
27937 290/300 Write block 140 len 36864 data: 5 6 5 5 6 6 5 4 5
27938 291/300 Write block 1302 len 40960 data: 5 5 7 6 5 5 4 4 5 6
27939 292/300 Read block 1092 len 12288
27940 293/300 Read block 1735 len 8192
27941 294/300 Read block 2251 len 28672
27942 295/300 Write block 1015 len 28672 data: 4 4 4 6 5 5 5
27943 296/300 Flush
27944 297/300 Write block 3099 len 16384 data: 3 4 4 4
27945 298/300 Write block 252 len 40960 data: 7 6 6 5 5 5 5 6 6 7
27946 299/300 Write block 3644 len 16384 data: 2 5 4 4
27947 300/300 Read block 3598 len 32768
27948 301/300 Read block 1103 len 8192
27949 Main task has work to do, go find it
27950 got dsc Start(1)
27951 start 1
27952 Jan 21 15:53:03.789 INFO request completed, latency_us: 209, response_code: 204, uri: /start/cid/1, method: POST, req_id: 4e29e85c-0838-4b53-9125-fb95b8ff6809, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
27953 Replay: started 1, returned:Ok(())
27954 ----------------------------------------------------------------
27955 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27956 States: Active Offline Active
27957 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27958 8548 Acked Write 7 Done Sent Done false
27959 8549 Acked Read 6 Done Sent Done false
27960 8550 Acked Write 9 Done Sent Done false
27961 8551 Acked Write 10 Done Sent Done false
27962 8552 Acked Write 6 Done Sent Done false
27963 8553 Acked Write 5 Done Sent Done false
27964 8554 Acked Write 8 Done Sent Done false
27965 8555 Acked Write 2 Done Sent Done false
27966 8556 Acked Write 10 Done Sent Done false
27967 8557 Acked Write 4 Done Sent Done false
27968 8558 Acked Write 7 Done Sent Done false
27969 8559 Acked Write 3 Done Sent Done false
27970 8560 Acked Read 8 Done Sent Done false
27971 8561 Acked Read 6 Done Sent Done false
27972 8562 Acked Read 9 Done Sent Done false
27973 8563 Acked Write 8 Done Sent Done false
27974 8564 Acked Read 9 Done Sent Done false
27975 8565 Acked Read 8 Done Sent Done false
27976 8566 Acked Write 4 Done Sent Done false
27977 8567 Acked Write 10 Done Sent Done false
27978 8568 Acked Read 6 Done Sent Done false
27979 8569 Acked Write 8 Done Sent Done false
27980 8570 Acked Write 5 Done Sent Done false
27981 8571 Acked Write 5 Done Sent Done false
27982 8572 Acked Read 7 Done Sent Done false
27983 8573 Acked Read 6 Done Sent Done false
27984 8574 Acked Write 10 Done Sent Done false
27985 8575 Acked Write 6 Done Sent Done false
27986 8576 Acked Write 7 Done Sent Done false
27987 8577 Acked Write 8 Done Sent Done false
27988 8578 Acked Flush 0 Done Sent Done false
27989 8579 Acked Flush 0 Done Sent Done false
27990 8580 Acked Read 6 Done Sent Done false
27991 8581 Acked Write 8 Done Sent Done false
27992 8582 Acked Write 3 Done Sent Done false
27993 8583 Acked Write 4 Done Sent Done false
27994 8584 Acked Read 10 Done Sent Done false
27995 8585 Acked Read 3 Done Sent Done false
27996 8586 Acked Read 7 Done Sent Done false
27997 8587 Acked Write 1 Done Sent Done false
27998 8588 Acked Read 6 Done Sent Done false
27999 8589 Acked Read 3 Done Sent Done false
28000 8590 Acked Read 7 Done Sent Done false
28001 8591 Acked Write 9 Done Sent Done false
28002 8592 Acked Flush 0 Done Sent Done false
28003 8593 Acked Write 4 Done Sent Done false
28004 8594 Acked Read 10 Done Sent Done false
28005 8595 Acked Write 6 Done Sent Done false
28006 8596 Acked Write 8 Done Sent Done false
28007 8597 Acked Write 5 Done Sent Done false
28008 8598 Acked Write 3 Done Sent Done false
28009 8599 Acked Read 1 Done Sent Done false
28010 8600 Acked Write 3 Done Sent Done false
28011 8601 Acked Write 10 Done Sent Done false
28012 8602 Acked Read 8 Done Sent Done false
28013 8603 Acked Read 5 Done Sent Done false
28014 8604 Acked Read 6 Done Sent Done false
28015 8605 Acked Read 8 Done Sent Done false
28016 8606 Acked Write 4 Done Sent Done false
28017 8607 Acked Read 1 Done Sent Done false
28018 8608 Acked Read 3 Done Sent Done false
28019 8609 Acked Write 9 Done Sent Done false
28020 8610 Acked Read 9 Done Sent Done false
28021 8611 Acked Read 7 Done Sent Done false
28022 8612 Acked Read 8 Done Sent Done false
28023 8613 Acked Write 1 Done Sent Done false
28024 8614 Acked Write 3 Done Sent Done false
28025 8615 Acked Read 6 Done Sent Done false
28026 8616 Acked Read 6 Done Sent Done false
28027 8617 Acked Write 4 Done Sent Done false
28028 8618 Acked Read 4 Done Sent Done false
28029 8619 Acked Read 10 Done Sent Done false
28030 8620 Acked Write 7 Done Sent Done false
28031 8621 Acked Write 10 Done Sent Done false
28032 8622 Acked Flush 0 Done Sent Done false
28033 8623 Acked Read 4 Done Sent Done false
28034 8624 Acked Write 5 Done Sent Done false
28035 8625 Acked Flush 0 Done Sent Done false
28036 8626 Acked Read 9 Done Sent Done false
28037 8627 Acked Write 1 Done Sent Done false
28038 8628 Acked Read 1 Done Sent Done false
28039 8629 Acked Write 1 Done Sent Done false
28040 8630 Acked Write 4 Done Sent Done false
28041 8631 Acked Flush 0 Done Sent Done false
28042 8632 Acked Write 3 Done Sent Done false
28043 8633 Acked Read 10 Done Sent Done false
28044 8634 Acked Flush 0 Done Sent Done false
28045 8635 Acked Read 10 Done Sent Done false
28046 8636 Acked Write 1 Done Sent Done false
28047 8637 Acked Read 7 Done Sent Done false
28048 8638 Acked Write 2 Done Sent Done false
28049 8639 Acked Flush 0 Done Sent Done false
28050 8640 Acked Read 9 Done Sent Done false
28051 8641 Acked Flush 0 Done Sent Done false
28052 8642 Acked Write 7 Done Sent Done false
28053 8643 Acked Read 9 Done Sent Done false
28054 8644 Acked Write 8 Done Sent Done false
28055 8645 Acked Read 5 Done Sent Done false
28056 8646 Acked Write 7 Done Sent Done false
28057 8647 Acked Read 9 Done Sent Done false
28058 8648 Acked Write 7 Done Sent Done false
28059 8649 Acked Read 3 Done Sent Done false
28060 8650 Acked Write 9 Done Sent Done false
28061 8651 Acked Flush 0 Done Sent Done false
28062 8652 Acked Write 6 Done Sent Done false
28063 8653 Acked Flush 0 Done Sent Done false
28064 8654 Acked Read 5 Done Sent Done false
28065 8655 Acked Write 2 Done Sent Done false
28066 8656 Acked Write 2 Done Sent Done false
28067 8657 Acked Read 2 Done Sent Done false
28068 8658 Acked Write 8 Done Sent Done false
28069 8659 Acked Flush 0 Done Sent Done false
28070 8660 Acked Read 8 Done Sent Done false
28071 8661 Acked Write 1 Done Sent Done false
28072 8662 Acked Write 9 Done Sent Done false
28073 8663 Acked Write 7 Done Sent Done false
28074 8664 Acked Flush 0 Done Sent Done false
28075 8665 Acked Read 7 Done Sent Done false
28076 8666 Acked Write 1 Done Sent Done false
28077 8667 Acked Read 4 Done Sent Done false
28078 8668 Acked Write 8 Done Sent Done false
28079 8669 Acked Read 5 Done Sent Done false
28080 8670 Acked Read 3 Done Sent Done false
28081 8671 Acked Read 4 Done Sent Done false
28082 8672 Acked Write 10 Done Sent Done false
28083 8673 Acked Read 3 Done Sent Done false
28084 8674 Acked Read 4 Done Sent Done false
28085 8675 Acked Read 3 Done Sent Done false
28086 8676 Acked Write 9 Done Sent Done false
28087 8677 Acked Read 4 Done Sent Done false
28088 8678 Acked Write 7 Done Sent Done false
28089 8679 Acked Read 3 Done Sent Done false
28090 8680 Acked Write 10 Done Sent Done false
28091 8681 Acked Write 6 Done Sent Done false
28092 8682 Acked Write 3 Done Sent Done false
28093 8683 Acked Read 3 Done Sent Done false
28094 8684 Acked Flush 0 Done Sent Done false
28095 8685 Acked Read 2 Done Sent Done false
28096 8686 Acked Write 10 Done Sent Done false
28097 8687 Acked Flush 0 Done Sent Done false
28098 8688 Acked Write 3 Done Sent Done false
28099 8689 Acked Read 1 Done Sent Done false
28100 8690 Acked Write 5 Done Sent Done false
28101 8691 Acked Read 1 Done Sent Done false
28102 8692 Acked Read 10 Done Sent Done false
28103 8693 Acked Write 10 Done Sent Done false
28104 8694 Acked Read 4 Done Sent Done false
28105 8695 Acked Write 3 Done Sent Done false
28106 8696 Acked Write 5 Done Sent Done false
28107 8697 Acked Flush 0 Done Sent Done false
28108 8698 Acked Write 5 Done Sent Done false
28109 8699 Acked Read 8 Done Sent Done false
28110 8700 Acked Read 4 Done Sent Done false
28111 8701 Acked Write 7 Done Sent Done false
28112 8702 Acked Flush 0 Done Sent Done false
28113 8703 Acked Write 7 Done Sent Done false
28114 8704 Acked Write 8 Done Sent Done false
28115 8705 Acked Read 10 Done Sent Done false
28116 8706 Acked Read 8 Done Sent Done false
28117 8707 Acked Write 2 Done Sent Done false
28118 8708 Acked Read 1 Done Sent Done false
28119 8709 Acked Write 3 Done Sent Done false
28120 8710 Acked Read 9 Done Sent Done false
28121 8711 Acked Read 3 Done Sent Done false
28122 8712 Acked Write 9 Done Sent Done false
28123 8713 Acked Read 9 Done Sent Done false
28124 8714 Acked Write 8 Done Sent Done false
28125 8715 Acked Read 6 Done Sent Done false
28126 8716 Acked Flush 0 Done Sent Done false
28127 8717 Acked Flush 0 Done Sent Done false
28128 8718 Acked Read 6 Done Sent Done false
28129 8719 Acked Read 1 Done Sent Done false
28130 8720 Acked Write 8 Done Sent Done false
28131 8721 Acked Read 1 Done Sent Done false
28132 8722 Acked Write 10 Done Sent Done false
28133 8723 Acked Write 1 Done Sent Done false
28134 8724 Acked Write 10 Done Sent Done false
28135 8725 Acked Read 2 Done Sent Done false
28136 8726 Acked Read 3 Done Sent Done false
28137 8727 Acked Read 1 Done Sent Done false
28138 8728 Acked Write 9 Done Sent Done false
28139 8729 Acked Flush 0 Done Sent Done false
28140 8730 Acked Write 8 Done Sent Done false
28141 8731 Acked Write 7 Done Sent Done false
28142 8732 Acked Read 3 Done Sent Done false
28143 8733 Acked Write 6 Done Sent Done false
28144 8734 Acked Write 2 Done Sent Done false
28145 8735 Acked Read 5 Done Sent Done false
28146 8736 Acked Read 7 Done Sent Done false
28147 8737 Acked Read 8 Done Sent Done false
28148 8738 Acked Read 3 Done Sent Done false
28149 8739 Acked Write 8 Done Sent Done false
28150 8740 Acked Read 7 Done Sent Done false
28151 8741 Acked Write 10 Done Sent Done false
28152 8742 Acked Read 8 Done Sent Done false
28153 8743 Acked Read 9 Done Sent Done false
28154 8744 Acked Flush 0 Done Sent Done false
28155 8745 Acked Read 5 Done Sent Done false
28156 8746 Acked Read 9 Done Sent Done false
28157 8747 Acked Read 6 Done Sent Done false
28158 8748 Acked Read 8 Done Sent Done false
28159 8749 Acked Write 8 Done Sent Done false
28160 8750 Acked Read 2 Done Sent Done false
28161 8751 Acked Read 8 Done Sent Done false
28162 8752 Acked Read 1 Done Sent Done false
28163 8753 Acked Read 1 Done Sent Done false
28164 8754 Acked Write 5 Done Sent Done false
28165 8755 Acked Read 9 Done Sent Done false
28166 8756 Acked Read 10 Done Sent Done false
28167 8757 Acked Write 2 Done Sent Done false
28168 8758 Acked Flush 0 Done Sent Done false
28169 8759 Acked Write 8 Done Sent Done false
28170 8760 Acked Read 9 Done Sent Done false
28171 8761 Acked Read 6 Done Sent Done false
28172 8762 Acked Write 8 Done Sent Done false
28173 8763 Acked Read 3 Done Sent Done false
28174 8764 Acked Read 5 Done Sent Done false
28175 8765 Acked Flush 0 Done Sent Done false
28176 8766 Acked Read 4 Done Sent Done false
28177 8767 Acked Read 10 Done Sent Done false
28178 8768 Acked Read 3 Done Sent Done false
28179 8769 Acked Flush 0 Done Sent Done false
28180 8770 Acked Write 6 Done Sent Done false
28181 8771 Acked Read 2 Done Sent Done false
28182 8772 Acked Read 9 Done Sent Done false
28183 8773 Acked Read 4 Done Sent Done false
28184 8774 Acked Write 10 Done Sent Done false
28185 8775 Acked Write 8 Done Sent Done false
28186 8776 Acked Read 9 Done Sent Done false
28187 8777 Acked Flush 0 Done Sent Done false
28188 8778 Acked Write 6 Done Sent Done false
28189 8779 Acked Write 1 Done Sent Done false
28190 8780 Acked Write 1 Done Sent Done false
28191 8781 Acked Read 4 Done Sent Done false
28192 8782 Acked Read 8 Done Sent Done false
28193 8783 Acked Write 8 Done Sent Done false
28194 8784 Acked Flush 0 Done Sent Done false
28195 8785 Acked Read 5 Done Sent Done false
28196 8786 Acked Write 4 Done Sent Done false
28197 8787 Acked Read 3 Done Sent Done false
28198 8788 Acked Write 9 Done Sent Done false
28199 8789 Acked Write 8 Done Sent Done false
28200 8790 Acked Write 9 Done Sent Done false
28201 8791 Acked Write 3 Done Sent Done false
28202 8792 Acked Write 5 Done Sent Done false
28203 8793 Acked Read 9 Done Sent Done false
28204 8794 Acked Read 6 Done Sent Done false
28205 8795 Acked Read 4 Done Sent Done false
28206 8796 Acked Write 6 Done Sent Done false
28207 8797 Acked Read 9 Done Sent Done false
28208 8798 Acked Read 6 Done Sent Done false
28209 8799 Acked Read 8 Done Sent Done false
28210 8800 Acked Read 9 Done Sent Done false
28211 8801 Acked Flush 0 Done Sent Done false
28212 8802 Acked Read 2 Done Sent Done false
28213 8803 Acked Write 6 Done Sent Done false
28214 8804 Acked Read 1 Done Sent Done false
28215 8805 Acked Read 6 Done Sent Done false
28216 8806 Acked Read 3 Done Sent Done false
28217 8807 Acked Read 7 Done Sent Done false
28218 8808 Acked Read 1 Done Sent Done false
28219 8809 Acked Flush 0 Done Sent Done false
28220 8810 Acked Read 10 Done Sent Done false
28221 8811 Acked Read 8 Done Sent Done false
28222 8812 Acked Write 6 Done Sent Done false
28223 8813 Acked Write 5 Done Sent Done false
28224 8814 Acked Flush 0 Done Sent Done false
28225 8815 Acked Write 8 Done Sent Done false
28226 8816 Acked Read 9 Done Sent Done false
28227 8817 Acked Write 4 Done Sent Done false
28228 8818 Acked Write 2 Done Sent Done false
28229 8819 Acked Write 8 Done Sent Done false
28230 8820 Acked Write 1 Done Sent Done false
28231 8821 Acked Write 1 Done Sent Done false
28232 8822 Acked Write 6 Done Sent Done false
28233 8823 Acked Write 1 Done Sent Done false
28234 8824 Acked Write 5 Done Sent Done false
28235 8825 Acked Read 6 Done Sent Done false
28236 8826 Acked Read 5 Done Sent Done false
28237 8827 Acked Read 8 Done Sent Done false
28238 8828 Acked Flush 0 Done Sent Done false
28239 8829 Acked Write 6 Done Sent Done false
28240 8830 Acked Read 4 Done Sent Done false
28241 8831 Acked Write 8 Done Sent Done false
28242 8832 Acked Read 4 Done Sent Done false
28243 8833 Acked Write 10 Done Sent Done false
28244 8834 Acked Flush 0 Done Sent Done false
28245 8835 Acked Write 1 Done Sent Done false
28246 8836 Acked Read 4 Done Sent Done false
28247 8837 Acked Write 9 Done Sent Done false
28248 8838 Acked Write 10 Done Sent Done false
28249 8839 Acked Read 3 Done Sent Done false
28250 8840 Acked Read 2 Done Sent Done false
28251 8841 Acked Read 7 Done Sent Done false
28252 8842 Acked Write 7 Done Sent Done false
28253 8843 Acked Flush 0 Done Sent Done false
28254 8844 Acked Write 4 Done Sent Done false
28255 8845 Acked Write 10 Done Sent Done false
28256 8846 Acked Write 4 Done Sent Done false
28257 8847 Acked Read 8 Done Sent Done false
28258 8848 Acked Read 2 Done Sent Done false
28259 STATES DS:0 DS:1 DS:2 TOTAL
28260 Sent 0 301 0 301
28261 Done 301 0 301 602
28262 Skipped 0 0 0 0
28263 Error 0 0 0 0
28264 Last Flush: 8843 8547 8843
28265 Downstairs last five completed: 8547 8546 8545 8544 8543
28266 Upstairs last five completed: 8848 8847 8846 8845 8844
28267 CLIENT: Up:0 ds:301 act:2
28268 [8820] got start action
28269 [8820] I am going to restart
28270 Starting downstairs at port 8820
28271 Make output file at "/tmp/dsc/downstairs-8820.txt"
28272 [8820][1] reports Starting
28273 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1226)
28274 [8820][1] reports Running
28275 ----------------------------------------------------------------
28276 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28277 States: Active Offline Active
28278 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28279 8548 Acked Write 7 Done Sent Done false
28280 8549 Acked Read 6 Done Sent Done false
28281 8550 Acked Write 9 Done Sent Done false
28282 8551 Acked Write 10 Done Sent Done false
28283 8552 Acked Write 6 Done Sent Done false
28284 8553 Acked Write 5 Done Sent Done false
28285 8554 Acked Write 8 Done Sent Done false
28286 8555 Acked Write 2 Done Sent Done false
28287 8556 Acked Write 10 Done Sent Done false
28288 8557 Acked Write 4 Done Sent Done false
28289 8558 Acked Write 7 Done Sent Done false
28290 8559 Acked Write 3 Done Sent Done false
28291 8560 Acked Read 8 Done Sent Done false
28292 8561 Acked Read 6 Done Sent Done false
28293 8562 Acked Read 9 Done Sent Done false
28294 8563 Acked Write 8 Done Sent Done false
28295 8564 Acked Read 9 Done Sent Done false
28296 8565 Acked Read 8 Done Sent Done false
28297 8566 Acked Write 4 Done Sent Done false
28298 8567 Acked Write 10 Done Sent Done false
28299 8568 Acked Read 6 Done Sent Done false
28300 8569 Acked Write 8 Done Sent Done false
28301 8570 Acked Write 5 Done Sent Done false
28302 8571 Acked Write 5 Done Sent Done false
28303 8572 Acked Read 7 Done Sent Done false
28304 8573 Acked Read 6 Done Sent Done false
28305 8574 Acked Write 10 Done Sent Done false
28306 8575 Acked Write 6 Done Sent Done false
28307 8576 Acked Write 7 Done Sent Done false
28308 8577 Acked Write 8 Done Sent Done false
28309 8578 Acked Flush 0 Done Sent Done false
28310 8579 Acked Flush 0 Done Sent Done false
28311 8580 Acked Read 6 Done Sent Done false
28312 8581 Acked Write 8 Done Sent Done false
28313 8582 Acked Write 3 Done Sent Done false
28314 8583 Acked Write 4 Done Sent Done false
28315 8584 Acked Read 10 Done Sent Done false
28316 8585 Acked Read 3 Done Sent Done false
28317 8586 Acked Read 7 Done Sent Done false
28318 8587 Acked Write 1 Done Sent Done false
28319 8588 Acked Read 6 Done Sent Done false
28320 8589 Acked Read 3 Done Sent Done false
28321 8590 Acked Read 7 Done Sent Done false
28322 8591 Acked Write 9 Done Sent Done false
28323 8592 Acked Flush 0 Done Sent Done false
28324 8593 Acked Write 4 Done Sent Done false
28325 8594 Acked Read 10 Done Sent Done false
28326 8595 Acked Write 6 Done Sent Done false
28327 8596 Acked Write 8 Done Sent Done false
28328 8597 Acked Write 5 Done Sent Done false
28329 8598 Acked Write 3 Done Sent Done false
28330 8599 Acked Read 1 Done Sent Done false
28331 8600 Acked Write 3 Done Sent Done false
28332 8601 Acked Write 10 Done Sent Done false
28333 8602 Acked Read 8 Done Sent Done false
28334 8603 Acked Read 5 Done Sent Done false
28335 8604 Acked Read 6 Done Sent Done false
28336 8605 Acked Read 8 Done Sent Done false
28337 8606 Acked Write 4 Done Sent Done false
28338 8607 Acked Read 1 Done Sent Done false
28339 8608 Acked Read 3 Done Sent Done false
28340 8609 Acked Write 9 Done Sent Done false
28341 8610 Acked Read 9 Done Sent Done false
28342 8611 Acked Read 7 Done Sent Done false
28343 8612 Acked Read 8 Done Sent Done false
28344 8613 Acked Write 1 Done Sent Done false
28345 8614 Acked Write 3 Done Sent Done false
28346 8615 Acked Read 6 Done Sent Done false
28347 8616 Acked Read 6 Done Sent Done false
28348 8617 Acked Write 4 Done Sent Done false
28349 8618 Acked Read 4 Done Sent Done false
28350 8619 Acked Read 10 Done Sent Done false
28351 8620 Acked Write 7 Done Sent Done false
28352 8621 Acked Write 10 Done Sent Done false
28353 8622 Acked Flush 0 Done Sent Done false
28354 8623 Acked Read 4 Done Sent Done false
28355 8624 Acked Write 5 Done Sent Done false
28356 8625 Acked Flush 0 Done Sent Done false
28357 8626 Acked Read 9 Done Sent Done false
28358 8627 Acked Write 1 Done Sent Done false
28359 8628 Acked Read 1 Done Sent Done false
28360 8629 Acked Write 1 Done Sent Done false
28361 8630 Acked Write 4 Done Sent Done false
28362 8631 Acked Flush 0 Done Sent Done false
28363 8632 Acked Write 3 Done Sent Done false
28364 8633 Acked Read 10 Done Sent Done false
28365 8634 Acked Flush 0 Done Sent Done false
28366 8635 Acked Read 10 Done Sent Done false
28367 8636 Acked Write 1 Done Sent Done false
28368 8637 Acked Read 7 Done Sent Done false
28369 8638 Acked Write 2 Done Sent Done false
28370 8639 Acked Flush 0 Done Sent Done false
28371 8640 Acked Read 9 Done Sent Done false
28372 8641 Acked Flush 0 Done Sent Done false
28373 8642 Acked Write 7 Done Sent Done false
28374 8643 Acked Read 9 Done Sent Done false
28375 8644 Acked Write 8 Done Sent Done false
28376 8645 Acked Read 5 Done Sent Done false
28377 8646 Acked Write 7 Done Sent Done false
28378 8647 Acked Read 9 Done Sent Done false
28379 8648 Acked Write 7 Done Sent Done false
28380 8649 Acked Read 3 Done Sent Done false
28381 8650 Acked Write 9 Done Sent Done false
28382 8651 Acked Flush 0 Done Sent Done false
28383 8652 Acked Write 6 Done Sent Done false
28384 8653 Acked Flush 0 Done Sent Done false
28385 8654 Acked Read 5 Done Sent Done false
28386 8655 Acked Write 2 Done Sent Done false
28387 8656 Acked Write 2 Done Sent Done false
28388 8657 Acked Read 2 Done Sent Done false
28389 8658 Acked Write 8 Done Sent Done false
28390 8659 Acked Flush 0 Done Sent Done false
28391 8660 Acked Read 8 Done Sent Done false
28392 8661 Acked Write 1 Done Sent Done false
28393 8662 Acked Write 9 Done Sent Done false
28394 8663 Acked Write 7 Done Sent Done false
28395 8664 Acked Flush 0 Done Sent Done false
28396 8665 Acked Read 7 Done Sent Done false
28397 8666 Acked Write 1 Done Sent Done false
28398 8667 Acked Read 4 Done Sent Done false
28399 8668 Acked Write 8 Done Sent Done false
28400 8669 Acked Read 5 Done Sent Done false
28401 8670 Acked Read 3 Done Sent Done false
28402 8671 Acked Read 4 Done Sent Done false
28403 8672 Acked Write 10 Done Sent Done false
28404 8673 Acked Read 3 Done Sent Done false
28405 8674 Acked Read 4 Done Sent Done false
28406 8675 Acked Read 3 Done Sent Done false
28407 8676 Acked Write 9 Done Sent Done false
28408 8677 Acked Read 4 Done Sent Done false
28409 8678 Acked Write 7 Done Sent Done false
28410 8679 Acked Read 3 Done Sent Done false
28411 8680 Acked Write 10 Done Sent Done false
28412 8681 Acked Write 6 Done Sent Done false
28413 8682 Acked Write 3 Done Sent Done false
28414 8683 Acked Read 3 Done Sent Done false
28415 8684 Acked Flush 0 Done Sent Done false
28416 8685 Acked Read 2 Done Sent Done false
28417 8686 Acked Write 10 Done Sent Done false
28418 8687 Acked Flush 0 Done Sent Done false
28419 8688 Acked Write 3 Done Sent Done false
28420 8689 Acked Read 1 Done Sent Done false
28421 8690 Acked Write 5 Done Sent Done false
28422 8691 Acked Read 1 Done Sent Done false
28423 8692 Acked Read 10 Done Sent Done false
28424 8693 Acked Write 10 Done Sent Done false
28425 8694 Acked Read 4 Done Sent Done false
28426 8695 Acked Write 3 Done Sent Done false
28427 8696 Acked Write 5 Done Sent Done false
28428 8697 Acked Flush 0 Done Sent Done false
28429 8698 Acked Write 5 Done Sent Done false
28430 8699 Acked Read 8 Done Sent Done false
28431 8700 Acked Read 4 Done Sent Done false
28432 8701 Acked Write 7 Done Sent Done false
28433 8702 Acked Flush 0 Done Sent Done false
28434 8703 Acked Write 7 Done Sent Done false
28435 8704 Acked Write 8 Done Sent Done false
28436 8705 Acked Read 10 Done Sent Done false
28437 8706 Acked Read 8 Done Sent Done false
28438 8707 Acked Write 2 Done Sent Done false
28439 8708 Acked Read 1 Done Sent Done false
28440 8709 Acked Write 3 Done Sent Done false
28441 8710 Acked Read 9 Done Sent Done false
28442 8711 Acked Read 3 Done Sent Done false
28443 8712 Acked Write 9 Done Sent Done false
28444 8713 Acked Read 9 Done Sent Done false
28445 8714 Acked Write 8 Done Sent Done false
28446 8715 Acked Read 6 Done Sent Done false
28447 8716 Acked Flush 0 Done Sent Done false
28448 8717 Acked Flush 0 Done Sent Done false
28449 8718 Acked Read 6 Done Sent Done false
28450 8719 Acked Read 1 Done Sent Done false
28451 8720 Acked Write 8 Done Sent Done false
28452 8721 Acked Read 1 Done Sent Done false
28453 8722 Acked Write 10 Done Sent Done false
28454 8723 Acked Write 1 Done Sent Done false
28455 8724 Acked Write 10 Done Sent Done false
28456 8725 Acked Read 2 Done Sent Done false
28457 8726 Acked Read 3 Done Sent Done false
28458 8727 Acked Read 1 Done Sent Done false
28459 8728 Acked Write 9 Done Sent Done false
28460 8729 Acked Flush 0 Done Sent Done false
28461 8730 Acked Write 8 Done Sent Done false
28462 8731 Acked Write 7 Done Sent Done false
28463 8732 Acked Read 3 Done Sent Done false
28464 8733 Acked Write 6 Done Sent Done false
28465 8734 Acked Write 2 Done Sent Done false
28466 8735 Acked Read 5 Done Sent Done false
28467 8736 Acked Read 7 Done Sent Done false
28468 8737 Acked Read 8 Done Sent Done false
28469 8738 Acked Read 3 Done Sent Done false
28470 8739 Acked Write 8 Done Sent Done false
28471 8740 Acked Read 7 Done Sent Done false
28472 8741 Acked Write 10 Done Sent Done false
28473 8742 Acked Read 8 Done Sent Done false
28474 8743 Acked Read 9 Done Sent Done false
28475 8744 Acked Flush 0 Done Sent Done false
28476 8745 Acked Read 5 Done Sent Done false
28477 8746 Acked Read 9 Done Sent Done false
28478 8747 Acked Read 6 Done Sent Done false
28479 8748 Acked Read 8 Done Sent Done false
28480 8749 Acked Write 8 Done Sent Done false
28481 8750 Acked Read 2 Done Sent Done false
28482 8751 Acked Read 8 Done Sent Done false
28483 8752 Acked Read 1 Done Sent Done false
28484 8753 Acked Read 1 Done Sent Done false
28485 8754 Acked Write 5 Done Sent Done false
28486 8755 Acked Read 9 Done Sent Done false
28487 8756 Acked Read 10 Done Sent Done false
28488 8757 Acked Write 2 Done Sent Done false
28489 8758 Acked Flush 0 Done Sent Done false
28490 8759 Acked Write 8 Done Sent Done false
28491 8760 Acked Read 9 Done Sent Done false
28492 8761 Acked Read 6 Done Sent Done false
28493 8762 Acked Write 8 Done Sent Done false
28494 8763 Acked Read 3 Done Sent Done false
28495 8764 Acked Read 5 Done Sent Done false
28496 8765 Acked Flush 0 Done Sent Done false
28497 8766 Acked Read 4 Done Sent Done false
28498 8767 Acked Read 10 Done Sent Done false
28499 8768 Acked Read 3 Done Sent Done false
28500 8769 Acked Flush 0 Done Sent Done false
28501 8770 Acked Write 6 Done Sent Done false
28502 8771 Acked Read 2 Done Sent Done false
28503 8772 Acked Read 9 Done Sent Done false
28504 8773 Acked Read 4 Done Sent Done false
28505 8774 Acked Write 10 Done Sent Done false
28506 8775 Acked Write 8 Done Sent Done false
28507 8776 Acked Read 9 Done Sent Done false
28508 8777 Acked Flush 0 Done Sent Done false
28509 8778 Acked Write 6 Done Sent Done false
28510 8779 Acked Write 1 Done Sent Done false
28511 8780 Acked Write 1 Done Sent Done false
28512 8781 Acked Read 4 Done Sent Done false
28513 8782 Acked Read 8 Done Sent Done false
28514 8783 Acked Write 8 Done Sent Done false
28515 8784 Acked Flush 0 Done Sent Done false
28516 8785 Acked Read 5 Done Sent Done false
28517 8786 Acked Write 4 Done Sent Done false
28518 8787 Acked Read 3 Done Sent Done false
28519 8788 Acked Write 9 Done Sent Done false
28520 8789 Acked Write 8 Done Sent Done false
28521 8790 Acked Write 9 Done Sent Done false
28522 8791 Acked Write 3 Done Sent Done false
28523 8792 Acked Write 5 Done Sent Done false
28524 8793 Acked Read 9 Done Sent Done false
28525 8794 Acked Read 6 Done Sent Done false
28526 8795 Acked Read 4 Done Sent Done false
28527 8796 Acked Write 6 Done Sent Done false
28528 8797 Acked Read 9 Done Sent Done false
28529 8798 Acked Read 6 Done Sent Done false
28530 8799 Acked Read 8 Done Sent Done false
28531 8800 Acked Read 9 Done Sent Done false
28532 8801 Acked Flush 0 Done Sent Done false
28533 8802 Acked Read 2 Done Sent Done false
28534 8803 Acked Write 6 Done Sent Done false
28535 8804 Acked Read 1 Done Sent Done false
28536 8805 Acked Read 6 Done Sent Done false
28537 8806 Acked Read 3 Done Sent Done false
28538 8807 Acked Read 7 Done Sent Done false
28539 8808 Acked Read 1 Done Sent Done false
28540 8809 Acked Flush 0 Done Sent Done false
28541 8810 Acked Read 10 Done Sent Done false
28542 8811 Acked Read 8 Done Sent Done false
28543 8812 Acked Write 6 Done Sent Done false
28544 8813 Acked Write 5 Done Sent Done false
28545 8814 Acked Flush 0 Done Sent Done false
28546 8815 Acked Write 8 Done Sent Done false
28547 8816 Acked Read 9 Done Sent Done false
28548 8817 Acked Write 4 Done Sent Done false
28549 8818 Acked Write 2 Done Sent Done false
28550 8819 Acked Write 8 Done Sent Done false
28551 8820 Acked Write 1 Done Sent Done false
28552 8821 Acked Write 1 Done Sent Done false
28553 8822 Acked Write 6 Done Sent Done false
28554 8823 Acked Write 1 Done Sent Done false
28555 8824 Acked Write 5 Done Sent Done false
28556 8825 Acked Read 6 Done Sent Done false
28557 8826 Acked Read 5 Done Sent Done false
28558 8827 Acked Read 8 Done Sent Done false
28559 8828 Acked Flush 0 Done Sent Done false
28560 8829 Acked Write 6 Done Sent Done false
28561 8830 Acked Read 4 Done Sent Done false
28562 8831 Acked Write 8 Done Sent Done false
28563 8832 Acked Read 4 Done Sent Done false
28564 8833 Acked Write 10 Done Sent Done false
28565 8834 Acked Flush 0 Done Sent Done false
28566 8835 Acked Write 1 Done Sent Done false
28567 8836 Acked Read 4 Done Sent Done false
28568 8837 Acked Write 9 Done Sent Done false
28569 8838 Acked Write 10 Done Sent Done false
28570 8839 Acked Read 3 Done Sent Done false
28571 8840 Acked Read 2 Done Sent Done false
28572 8841 Acked Read 7 Done Sent Done false
28573 8842 Acked Write 7 Done Sent Done false
28574 8843 Acked Flush 0 Done Sent Done false
28575 8844 Acked Write 4 Done Sent Done false
28576 8845 Acked Write 10 Done Sent Done false
28577 8846 Acked Write 4 Done Sent Done false
28578 8847 Acked Read 8 Done Sent Done false
28579 8848 Acked Read 2 Done Sent Done false
28580 8849 Acked Flush 0 Done Sent Done false
28581 STATES DS:0 DS:1 DS:2 TOTAL
28582 Sent 0 302 0 302
28583 Done 302 0 302 604
28584 Skipped 0 0 0 0
28585 Error 0 0 0 0
28586 Last Flush: 8849 8547 8849
28587 Downstairs last five completed: 8547 8546 8545 8544 8543
28588 Upstairs last five completed: 8849 8848 8847 8846 8845
28589 CLIENT: Up:0 ds:302 act:2
285902025-01-21T15:53:09.544ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
285912025-01-21T15:53:09.544ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55059) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
285922025-01-21T15:53:09.545ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
285932025-01-21T15:53:09.546ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
285942025-01-21T15:53:09.546ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
285952025-01-21T15:53:09.546ZINFOcrucible: send last flush ID to this DS: 8547 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
285962025-01-21T15:53:09.546ZINFOcrucible: Replied this last flush ID: 8547 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
285972025-01-21T15:53:09.546ZINFOcrucible: [1] client re-new 302 jobs since flush 8547 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
285982025-01-21T15:53:09.547ZINFOcrucible: [1] Replayed 302 jobs since flush: 8547 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
28599 ----------------------------------------------------------------
28600 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28601 Downstairs last five completed: 8849 8848 8847 8846 8845
28602 Upstairs last five completed: 8849 8848 8847 8846 8845
28603 CLIENT: Up:0 ds:0 act:3
28604 Replay: All jobs finished
28605 Main task has work to do, go find it
28606 got dsc Stop(2)
28607 stop 2
28608 [8830] Got stop action so:false kr:false
28609 Jan 21 15:53:11.795 INFO request completed, latency_us: 382, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 7e269814-f04e-4976-a537-4af3b60f3300, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
286102025-01-21T15:53:11.796ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
286112025-01-21T15:53:11.796ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
286122025-01-21T15:53:11.796ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
28613 Jan 21 15:53:11.796 INFO request completed, latency_us: 140, response_code: 200, uri: /state/cid/2, method: GET, req_id: 4f8837fa-c2fc-4230-ae30-435d17370fed, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
286142025-01-21T15:53:11.796ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
28615 [8830] Exited with: ExitStatus(unix_wait_status(9))
28616 [8830][2] reports Exit
28617 Jan 21 15:53:15.799 INFO request completed, latency_us: 278, response_code: 200, uri: /state/cid/2, method: GET, req_id: ec6ea25c-b9f9-4ec4-8c6f-6a723abba9b6, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
28618 001/300 Read block 2898 len 12288
28619 002/300 Read block 3291 len 24576
28620 003/300 Read block 2480 len 28672
28621 004/300 Write block 846 len 24576 data: 8 7 5 6 6 5
28622 005/300 Read block 2309 len 32768
28623 006/300 Read block 221 len 36864
28624 007/300 Write block 197 len 16384 data: 5 5 5 4
28625 008/300 Read block 2566 len 12288
28626 009/300 Write block 111 len 12288 data: 3 4 5
28627 010/300 Write block 1883 len 16384 data: 6 6 5 5
28628 011/300 Read block 569 len 4096
28629 012/300 Read block 3798 len 24576
28630 013/300 Write block 1658 len 28672 data: 8 7 8 10 8 9 8
28631 014/300 Write block 3858 len 28672 data: 9 7 6 6 3 4 5
28632 015/300 Read block 503 len 16384
28633 016/300 Write block 785 len 8192 data: 4 3
28634 017/300 Read block 496 len 28672
28635 018/300 Read block 1853 len 4096
28636 019/300 Write block 3065 len 8192 data: 5 5
28637 020/300 Read block 3779 len 28672
28638 021/300 Write block 1469 len 16384 data: 6 6 6 6
28639 022/300 Read block 1436 len 4096
28640 023/300 Read block 4428 len 40960
28641 024/300 Read block 3910 len 28672
28642 025/300 Read block 3882 len 28672
28643 026/300 Read block 4756 len 40960
28644 027/300 Read block 1371 len 16384
28645 028/300 Read block 2425 len 40960
28646 029/300 Write block 4476 len 8192 data: 4 7
28647 030/300 Write block 3794 len 28672 data: 8 7 7 5 5 5 5
28648 031/300 Read block 738 len 12288
28649 032/300 Write block 2026 len 12288 data: 9 11 10
28650 033/300 Read block 2051 len 12288
28651 034/300 Write block 4102 len 24576 data: 6 8 6 6 5 6
28652 035/300 Read block 2880 len 28672
28653 036/300 Read block 1527 len 28672
28654 037/300 Read block 1289 len 28672
28655 038/300 Write block 1235 len 8192 data: 3 3
28656 039/300 Read block 1977 len 24576
28657 040/300 Write block 824 len 20480 data: 7 5 4 3 3
28658 041/300 Write block 1948 len 24576 data: 3 3 4 3 4 3
28659 042/300 Read block 4302 len 24576
28660 043/300 Read block 584 len 32768
28661 044/300 Write block 1475 len 24576 data: 5 3 7 6 6 4
28662 045/300 Write block 3927 len 20480 data: 6 7 6 7 6
28663 046/300 Read block 2497 len 32768
28664 047/300 Read block 1244 len 12288
28665 048/300 Read block 3850 len 12288
28666 049/300 Read block 846 len 40960
28667 050/300 Write block 231 len 20480 data: 8 7 8 9 9
28668 051/300 Read block 3782 len 12288
28669 052/300 Write block 693 len 40960 data: 5 6 8 9 8 7 6 6 7 7
28670 053/300 Read block 4667 len 24576
28671 054/300 Write block 2123 len 40960 data: 6 7 7 6 6 5 5 5 5 7
28672 055/300 Write block 1844 len 16384 data: 10 9 8 9
28673 056/300 Write block 2495 len 4096 data: 6
28674 057/300 Write block 3822 len 24576 data: 6 5 4 4 3 4
28675 058/300 Read block 2218 len 16384
28676 059/300 Write block 865 len 40960 data: 9 9 8 6 4 3 2 2 2 2
28677 060/300 Write block 4392 len 12288 data: 8 6 6
28678 061/300 Read block 3034 len 8192
28679 062/300 Write block 1433 len 32768 data: 7 7 4 3 3 4 5 5
28680 063/300 Flush
28681 064/300 Read block 1031 len 32768
28682 065/300 Write block 4399 len 8192 data: 9 9
28683 066/300 Read block 4359 len 32768
28684 067/300 Read block 335 len 8192
28685 068/300 Read block 1948 len 8192
28686 069/300 Read block 615 len 20480
28687 070/300 Read block 4505 len 28672
28688 071/300 Write block 3876 len 32768 data: 7 10 11 9 8 9 7 5
28689 072/300 Write block 268 len 36864 data: 6 7 9 7 7 6 6 6 4
28690 073/300 Flush
28691 074/300 Read block 1003 len 32768
28692 075/300 Write block 841 len 16384 data: 7 8 8 8
28693 076/300 Write block 2594 len 12288 data: 4 4 5
28694 077/300 Write block 2647 len 16384 data: 9 7 6 6
28695 078/300 Flush
28696 079/300 Write block 404 len 16384 data: 8 9 9 10
28697 080/300 Read block 1243 len 36864
28698 081/300 Read block 3217 len 4096
28699 082/300 Write block 2293 len 4096 data: 6
28700 083/300 Read block 3592 len 20480
28701 084/300 Flush
28702 085/300 Write block 3465 len 32768 data: 6 7 6 9 9 8 8 8
28703 086/300 Read block 4447 len 4096
28704 087/300 Flush
28705 088/300 Write block 3853 len 24576 data: 8 9 10 10 9 10
28706 089/300 Write block 1608 len 40960 data: 3 3 3 4 4 4 4 4 5 4
28707 090/300 Flush
28708 091/300 Write block 3115 len 20480 data: 6 5 5 4 4
28709 092/300 Write block 1312 len 24576 data: 8 7 6 7 7 7
28710 093/300 Write block 522 len 20480 data: 4 5 5 5 6
28711 094/300 Write block 2309 len 28672 data: 3 5 5 5 2 5 5
28712 095/300 Write block 1945 len 40960 data: 4 4 3 4 4 5 4 5 4 3
28713 096/300 Write block 1900 len 24576 data: 7 7 8 8 8 7
28714 097/300 Write block 1505 len 28672 data: 6 5 7 7 7 9 11
28715 098/300 Write block 565 len 12288 data: 3 3 3
28716 099/300 Write block 3997 len 40960 data: 3 2 2 2 2 3 4 5 7 8
28717 100/300 Flush
28718 101/300 Read block 2394 len 40960
28719 102/300 Write block 3954 len 16384 data: 7 6 7 5
28720 103/300 Read block 112 len 40960
28721 104/300 Write block 1088 len 24576 data: 4 3 6 5 5 6
28722 105/300 Read block 526 len 32768
28723 106/300 Read block 3729 len 32768
28724 107/300 Write block 4272 len 20480 data: 7 5 5 3 4
28725 108/300 Write block 3995 len 24576 data: 4 3 4 3 3 3
28726 109/300 Write block 2976 len 36864 data: 4 6 5 6 4 4 5 5 6
28727 110/300 Write block 1769 len 32768 data: 8 6 6 6 5 4 3 5
28728 111/300 Write block 1403 len 8192 data: 3 2
28729 112/300 Write block 4557 len 12288 data: 5 4 3
28730 113/300 Read block 1435 len 4096
28731 114/300 Read block 4228 len 28672
28732 115/300 Read block 2521 len 8192
28733 116/300 Read block 2973 len 24576
28734 117/300 Read block 4021 len 24576
28735 118/300 Read block 4382 len 12288
28736 119/300 Read block 3134 len 4096
28737 120/300 Write block 1878 len 32768 data: 5 5 6 5 5 7 7 6
28738 121/300 Write block 817 len 32768 data: 5 6 7 8 8 7 6 8
28739 122/300 Write block 4765 len 12288 data: 3 4 4
28740 123/300 Read block 434 len 8192
28741 124/300 Read block 4050 len 32768
28742 125/300 Flush
28743 126/300 Flush
28744 127/300 Write block 4838 len 32768 data: 3 3 3 5 6 6 7 7
28745 128/300 Read block 1937 len 4096
28746 129/300 Write block 1311 len 12288 data: 7 9 8
28747 130/300 Write block 1940 len 12288 data: 6 7 6
28748 131/300 Write block 3571 len 24576 data: 6 5 3 4 5 4
28749 132/300 Flush
28750 133/300 Flush
28751 134/300 Read block 2655 len 32768
28752 135/300 Read block 3138 len 16384
28753 136/300 Write block 3454 len 40960 data: 8 9 9 6 6 6 6 7 7 8
28754 137/300 Read block 4510 len 12288
28755 138/300 Write block 2235 len 4096 data: 5
28756 139/300 Read block 2288 len 36864
28757 140/300 Write block 2047 len 4096 data: 7
28758 141/300 Read block 3911 len 36864
28759 142/300 Read block 1331 len 12288
28760 143/300 Read block 2380 len 4096
28761 144/300 Write block 1344 len 16384 data: 7 6 8 8
28762 145/300 Write block 2771 len 32768 data: 4 5 4 5 4 5 6 6
28763 146/300 Write block 818 len 36864 data: 7 8 9 9 8 7 9 6 5
28764 147/300 Read block 1826 len 8192
28765 148/300 Read block 2358 len 28672
28766 149/300 Read block 1601 len 8192
28767 150/300 Read block 517 len 16384
28768 151/300 Write block 1024 len 8192 data: 4 7
28769 152/300 Flush
28770 153/300 Flush
28771 154/300 Write block 2569 len 8192 data: 7 6
28772 155/300 Write block 3308 len 32768 data: 3 3 3 3 4 3 5 5
28773 156/300 Read block 2118 len 36864
28774 157/300 Read block 2199 len 32768
28775 158/300 Write block 357 len 4096 data: 5
28776 159/300 Read block 902 len 24576
28777 160/300 Read block 3396 len 28672
28778 161/300 Read block 2844 len 28672
28779 162/300 Read block 4882 len 4096
28780 163/300 Read block 3361 len 28672
28781 164/300 Write block 2308 len 12288 data: 5 4 6
28782 165/300 Read block 624 len 20480
28783 166/300 Write block 3945 len 24576 data: 5 6 6 8 6 6
28784 167/300 Write block 4560 len 36864 data: 3 3 4 3 3 3 3 5 4
28785 168/300 Write block 2315 len 36864 data: 6 5 5 4 6 6 6 6 7
28786 169/300 Read block 4883 len 20480
28787 170/300 Write block 1674 len 20480 data: 8 8 5 3 2
28788 171/300 Write block 628 len 8192 data: 7 7
28789 172/300 Write block 4166 len 32768 data: 4 3 5 5 6 7 7 7
28790 173/300 Read block 1715 len 32768
28791 174/300 Read block 193 len 16384
28792 175/300 Write block 316 len 16384 data: 6 5 6 5
28793 176/300 Read block 297 len 36864
28794 177/300 Write block 723 len 4096 data: 7
28795 178/300 Write block 3896 len 40960 data: 6 4 4 5 6 3 4 5 5 5
28796 179/300 Read block 119 len 16384
28797 180/300 Read block 809 len 4096
28798 181/300 Read block 3321 len 8192
28799 182/300 Write block 3596 len 32768 data: 5 6 6 6 7 8 6 5
28800 183/300 Write block 1000 len 32768 data: 8 7 5 5 4 5 5 4
28801 184/300 Read block 1647 len 32768
28802 185/300 Flush
28803 186/300 Write block 1169 len 4096 data: 5
28804 187/300 Read block 3788 len 36864
28805 188/300 Write block 2346 len 24576 data: 9 9 9 8 5 7
28806 189/300 Read block 1411 len 32768
28807 190/300 Read block 474 len 28672
28808 191/300 Read block 1025 len 20480
28809 192/300 Read block 1669 len 28672
28810 193/300 Read block 1607 len 4096
28811 194/300 Read block 383 len 28672
28812 195/300 Write block 3056 len 36864 data: 4 4 3 3 2 2 3 4 5
28813 196/300 Read block 1101 len 24576
28814 197/300 Flush
28815 198/300 Read block 3462 len 4096
28816 199/300 Read block 4400 len 16384
28817 200/300 Read block 572 len 32768
28818 201/300 Write block 4259 len 8192 data: 4 4
28819 202/300 Read block 1566 len 4096
28820 203/300 Read block 2641 len 24576
28821 204/300 Read block 1541 len 24576
28822 205/300 Flush
28823 206/300 Read block 2336 len 28672
28824 207/300 Read block 529 len 40960
28825 208/300 Read block 217 len 40960
28826 209/300 Write block 2283 len 8192 data: 5 4
28827 210/300 Read block 3080 len 36864
28828 211/300 Read block 2121 len 12288
28829 212/300 Flush
28830 213/300 Write block 1807 len 28672 data: 7 9 10 10 10 10 9
28831 214/300 Write block 4009 len 20480 data: 6 5 3 3 2
28832 215/300 Read block 2162 len 8192
28833 216/300 Write block 1662 len 12288 data: 9 10 9
28834 217/300 Read block 4628 len 40960
28835 218/300 Read block 1661 len 40960
28836 219/300 Read block 4463 len 40960
28837 220/300 Read block 1673 len 16384
28838 221/300 Read block 452 len 36864
28839 222/300 Read block 2285 len 28672
28840 223/300 Read block 2582 len 8192
28841 224/300 Read block 1334 len 36864
28842 225/300 Read block 3331 len 40960
28843 226/300 Write block 2730 len 20480 data: 4 4 4 5 4
28844 227/300 Read block 3887 len 32768
28845 228/300 Read block 1919 len 28672
28846 229/300 Write block 3762 len 4096 data: 4
28847 230/300 Read block 3124 len 40960
28848 231/300 Write block 625 len 28672 data: 7 7 8 8 8 7 6
28849 232/300 Read block 2888 len 16384
28850 233/300 Read block 2652 len 8192
28851 234/300 Write block 2581 len 24576 data: 5 4 4 4 4 4
28852 235/300 Write block 88 len 16384 data: 5 5 3 3
28853 236/300 Write block 2051 len 32768 data: 4 5 5 6 6 6 6 6
28854 237/300 Read block 3271 len 4096
28855 238/300 Read block 4655 len 36864
28856 239/300 Write block 4211 len 12288 data: 5 4 3
28857 240/300 Write block 100 len 32768 data: 3 3 3 5 6 6 6 5
28858 241/300 Read block 2772 len 28672
28859 242/300 Read block 605 len 40960
28860 243/300 Read block 4261 len 4096
28861 244/300 Flush
28862 245/300 Read block 3966 len 8192
28863 246/300 Write block 2078 len 4096 data: 8
28864 247/300 Read block 4748 len 16384
28865 248/300 Read block 4725 len 16384
28866 249/300 Write block 3585 len 12288 data: 3 3 3
28867 250/300 Write block 671 len 32768 data: 4 4 3 3 4 4 4 4
28868 251/300 Flush
28869 252/300 Read block 3357 len 32768
28870 253/300 Flush
28871 254/300 Write block 3620 len 32768 data: 3 4 5 5 7 7 6 8
28872 255/300 Write block 2783 len 20480 data: 5 7 5 6 7
28873 256/300 Write block 990 len 32768 data: 4 5 6 6 6 8 8 7
28874 257/300 Write block 4537 len 36864 data: 4 5 6 7 9 9 11 8 9
28875 258/300 Write block 37 len 4096 data: 6
28876 259/300 Write block 4638 len 32768 data: 6 5 4 3 3 3 3 3
28877 260/300 Flush
28878 261/300 Read block 584 len 40960
28879 262/300 Write block 2950 len 40960 data: 4 7 8 7 6 6 6 7 7 6
28880 263/300 Flush
28881 264/300 Write block 1624 len 4096 data: 5
28882 265/300 Write block 2194 len 32768 data: 2 2 3 4 6 5 5 4
28883 266/300 Write block 4349 len 4096 data: 8
28884 267/300 Write block 222 len 12288 data: 4 5 9
28885 268/300 Read block 4022 len 8192
28886 269/300 Read block 3856 len 36864
28887 270/300 Flush
28888 271/300 Read block 1544 len 20480
28889 272/300 Read block 61 len 40960
28890 273/300 Flush
28891 274/300 Read block 2456 len 36864
28892 275/300 Write block 2022 len 12288 data: 5 6 6
28893 276/300 Write block 2156 len 8192 data: 2 3
28894 277/300 Read block 2841 len 32768
28895 278/300 Read block 4387 len 24576
28896 279/300 Write block 1867 len 8192 data: 8 6
28897 280/300 Write block 4660 len 12288 data: 4 4 4
28898 281/300 Read block 1613 len 8192
28899 282/300 Flush
28900 283/300 Write block 2886 len 8192 data: 9 8
28901 284/300 Read block 1704 len 40960
28902 285/300 Write block 2684 len 20480 data: 3 3 3 4 4
28903 286/300 Write block 3398 len 8192 data: 9 6
28904 287/300 Write block 1784 len 20480 data: 6 6 6 7 7
28905 288/300 Read block 4605 len 28672
28906 289/300 Read block 1290 len 16384
28907 290/300 Read block 2783 len 24576
28908 291/300 Write block 1931 len 16384 data: 3 3 7 7
28909 292/300 Read block 1349 len 16384
28910 293/300 Read block 859 len 36864
28911 294/300 Read block 99 len 36864
28912 295/300 Write block 4751 len 12288 data: 6 5 5
28913 296/300 Read block 1132 len 32768
28914 297/300 Read block 1868 len 12288
28915 298/300 Write block 44 len 24576 data: 7 8 7 8 7 8
28916 299/300 Read block 493 len 8192
28917 300/300 Write block 2460 len 4096 data: 8
28918 301/300 Write block 1989 len 12288 data: 9 8 7
28919 Main task has work to do, go find it
28920 got dsc Start(2)
28921 start 2
28922 Jan 21 15:53:16.041 INFO request completed, latency_us: 242, response_code: 204, uri: /start/cid/2, method: POST, req_id: f1af652d-ba4a-4cf0-b344-11004c499dbc, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
28923 Replay: started 2, returned:Ok(())
28924 ----------------------------------------------------------------
28925 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28926 States: Active Active Offline
28927 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28928 8850 Acked Read 3 Done Done Sent false
28929 8851 Acked Read 6 Done Done Sent false
28930 8852 Acked Read 7 Done Done Sent false
28931 8853 Acked Write 6 Done Done Sent false
28932 8854 Acked Read 8 Done Done Sent false
28933 8855 Acked Read 9 Done Done Sent false
28934 8856 Acked Write 4 Done Done Sent false
28935 8857 Acked Read 3 Done Done Sent false
28936 8858 Acked Write 3 Done Done Sent false
28937 8859 Acked Write 4 Done Done Sent false
28938 8860 Acked Read 1 Done Done Sent false
28939 8861 Acked Read 6 Done Done Sent false
28940 8862 Acked Write 7 Done Done Sent false
28941 8863 Acked Write 7 Done Done Sent false
28942 8864 Acked Read 4 Done Done Sent false
28943 8865 Acked Write 2 Done Done Sent false
28944 8866 Acked Read 7 Done Done Sent false
28945 8867 Acked Read 1 Done Done Sent false
28946 8868 Acked Write 2 Done Done Sent false
28947 8869 Acked Read 7 Done Done Sent false
28948 8870 Acked Write 4 Done Done Sent false
28949 8871 Acked Read 1 Done Done Sent false
28950 8872 Acked Read 10 Done Done Sent false
28951 8873 Acked Read 7 Done Done Sent false
28952 8874 Acked Read 7 Done Done Sent false
28953 8875 Acked Read 10 Done Done Sent false
28954 8876 Acked Read 4 Done Done Sent false
28955 8877 Acked Read 10 Done Done Sent false
28956 8878 Acked Write 2 Done Done Sent false
28957 8879 Acked Write 7 Done Done Sent false
28958 8880 Acked Read 3 Done Done Sent false
28959 8881 Acked Write 3 Done Done Sent false
28960 8882 Acked Read 3 Done Done Sent false
28961 8883 Acked Write 6 Done Done Sent false
28962 8884 Acked Read 7 Done Done Sent false
28963 8885 Acked Read 7 Done Done Sent false
28964 8886 Acked Read 7 Done Done Sent false
28965 8887 Acked Write 2 Done Done Sent false
28966 8888 Acked Read 6 Done Done Sent false
28967 8889 Acked Write 5 Done Done Sent false
28968 8890 Acked Write 6 Done Done Sent false
28969 8891 Acked Read 6 Done Done Sent false
28970 8892 Acked Read 8 Done Done Sent false
28971 8893 Acked Write 6 Done Done Sent false
28972 8894 Acked Write 5 Done Done Sent false
28973 8895 Acked Read 8 Done Done Sent false
28974 8896 Acked Read 3 Done Done Sent false
28975 8897 Acked Read 3 Done Done Sent false
28976 8898 Acked Read 10 Done Done Sent false
28977 8899 Acked Write 5 Done Done Sent false
28978 8900 Acked Read 3 Done Done Sent false
28979 8901 Acked Write 10 Done Done Sent false
28980 8902 Acked Read 6 Done Done Sent false
28981 8903 Acked Write 10 Done Done Sent false
28982 8904 Acked Write 4 Done Done Sent false
28983 8905 Acked Write 1 Done Done Sent false
28984 8906 Acked Write 6 Done Done Sent false
28985 8907 Acked Read 4 Done Done Sent false
28986 8908 Acked Write 10 Done Done Sent false
28987 8909 Acked Write 3 Done Done Sent false
28988 8910 Acked Read 2 Done Done Sent false
28989 8911 Acked Write 8 Done Done Sent false
28990 8912 Acked Flush 0 Done Done Sent false
28991 8913 Acked Read 8 Done Done Sent false
28992 8914 Acked Write 2 Done Done Sent false
28993 8915 Acked Read 8 Done Done Sent false
28994 8916 Acked Read 2 Done Done Sent false
28995 8917 Acked Read 2 Done Done Sent false
28996 8918 Acked Read 5 Done Done Sent false
28997 8919 Acked Read 7 Done Done Sent false
28998 8920 Acked Write 8 Done Done Sent false
28999 8921 Acked Write 9 Done Done Sent false
29000 8922 Acked Flush 0 Done Done Sent false
29001 8923 Acked Read 8 Done Done Sent false
29002 8924 Acked Write 4 Done Done Sent false
29003 8925 Acked Write 3 Done Done Sent false
29004 8926 Acked Write 4 Done Done Sent false
29005 8927 Acked Flush 0 Done Done Sent false
29006 8928 Acked Write 4 Done Done Sent false
29007 8929 Acked Read 9 Done Done Sent false
29008 8930 Acked Read 1 Done Done Sent false
29009 8931 Acked Write 1 Done Done Sent false
29010 8932 Acked Read 5 Done Done Sent false
29011 8933 Acked Flush 0 Done Done Sent false
29012 8934 Acked Write 8 Done Done Sent false
29013 8935 Acked Read 1 Done Done Sent false
29014 8936 Acked Flush 0 Done Done Sent false
29015 8937 Acked Write 6 Done Done Sent false
29016 8938 Acked Write 10 Done Done Sent false
29017 8939 Acked Flush 0 Done Done Sent false
29018 8940 Acked Write 5 Done Done Sent false
29019 8941 Acked Write 6 Done Done Sent false
29020 8942 Acked Write 5 Done Done Sent false
29021 8943 Acked Write 7 Done Done Sent false
29022 8944 Acked Write 10 Done Done Sent false
29023 8945 Acked Write 6 Done Done Sent false
29024 8946 Acked Write 7 Done Done Sent false
29025 8947 Acked Write 3 Done Done Sent false
29026 8948 Acked Write 10 Done Done Sent false
29027 8949 Acked Flush 0 Done Done Sent false
29028 8950 Acked Read 10 Done Done Sent false
29029 8951 Acked Write 4 Done Done Sent false
29030 8952 Acked Read 10 Done Done Sent false
29031 8953 Acked Write 6 Done Done Sent false
29032 8954 Acked Read 8 Done Done Sent false
29033 8955 Acked Read 8 Done Done Sent false
29034 8956 Acked Write 5 Done Done Sent false
29035 8957 Acked Write 6 Done Done Sent false
29036 8958 Acked Write 9 Done Done Sent false
29037 8959 Acked Write 8 Done Done Sent false
29038 8960 Acked Write 2 Done Done Sent false
29039 8961 Acked Write 3 Done Done Sent false
29040 8962 Acked Read 1 Done Done Sent false
29041 8963 Acked Read 7 Done Done Sent false
29042 8964 Acked Read 2 Done Done Sent false
29043 8965 Acked Read 6 Done Done Sent false
29044 8966 Acked Read 6 Done Done Sent false
29045 8967 Acked Read 3 Done Done Sent false
29046 8968 Acked Read 1 Done Done Sent false
29047 8969 Acked Write 8 Done Done Sent false
29048 8970 Acked Write 8 Done Done Sent false
29049 8971 Acked Write 3 Done Done Sent false
29050 8972 Acked Read 2 Done Done Sent false
29051 8973 Acked Read 8 Done Done Sent false
29052 8974 Acked Flush 0 Done Done Sent false
29053 8975 Acked Flush 0 Done Done Sent false
29054 8976 Acked Write 8 Done Done Sent false
29055 8977 Acked Read 1 Done Done Sent false
29056 8978 Acked Write 3 Done Done Sent false
29057 8979 Acked Write 3 Done Done Sent false
29058 8980 Acked Write 6 Done Done Sent false
29059 8981 Acked Flush 0 Done Done Sent false
29060 8982 Acked Flush 0 Done Done Sent false
29061 8983 Acked Read 8 Done Done Sent false
29062 8984 Acked Read 4 Done Done Sent false
29063 8985 Acked Write 10 Done Done Sent false
29064 8986 Acked Read 3 Done Done Sent false
29065 8987 Acked Write 1 Done Done Sent false
29066 8988 Acked Read 9 Done Done Sent false
29067 8989 Acked Write 1 Done Done Sent false
29068 8990 Acked Read 9 Done Done Sent false
29069 8991 Acked Read 3 Done Done Sent false
29070 8992 Acked Read 1 Done Done Sent false
29071 8993 Acked Write 4 Done Done Sent false
29072 8994 Acked Write 8 Done Done Sent false
29073 8995 Acked Write 9 Done Done Sent false
29074 8996 Acked Read 2 Done Done Sent false
29075 8997 Acked Read 7 Done Done Sent false
29076 8998 Acked Read 2 Done Done Sent false
29077 8999 Acked Read 4 Done Done Sent false
29078 9000 Acked Write 2 Done Done Sent false
29079 9001 Acked Flush 0 Done Done Sent false
29080 9002 Acked Flush 0 Done Done Sent false
29081 9003 Acked Write 2 Done Done Sent false
29082 9004 Acked Write 8 Done Done Sent false
29083 9005 Acked Read 9 Done Done Sent false
29084 9006 Acked Read 8 Done Done Sent false
29085 9007 Acked Write 1 Done Done Sent false
29086 9008 Acked Read 6 Done Done Sent false
29087 9009 Acked Read 7 Done Done Sent false
29088 9010 Acked Read 7 Done Done Sent false
29089 9011 Acked Read 1 Done Done Sent false
29090 9012 Acked Read 7 Done Done Sent false
29091 9013 Acked Write 3 Done Done Sent false
29092 9014 Acked Read 5 Done Done Sent false
29093 9015 Acked Write 6 Done Done Sent false
29094 9016 Acked Write 9 Done Done Sent false
29095 9017 Acked Write 9 Done Done Sent false
29096 9018 Acked Read 5 Done Done Sent false
29097 9019 Acked Write 5 Done Done Sent false
29098 9020 Acked Write 2 Done Done Sent false
29099 9021 Acked Write 8 Done Done Sent false
29100 9022 Acked Read 8 Done Done Sent false
29101 9023 Acked Read 4 Done Done Sent false
29102 9024 Acked Write 4 Done Done Sent false
29103 9025 Acked Read 9 Done Done Sent false
29104 9026 Acked Write 1 Done Done Sent false
29105 9027 Acked Write 10 Done Done Sent false
29106 9028 Acked Read 4 Done Done Sent false
29107 9029 Acked Read 1 Done Done Sent false
29108 9030 Acked Read 2 Done Done Sent false
29109 9031 Acked Write 8 Done Done Sent false
29110 9032 Acked Write 8 Done Done Sent false
29111 9033 Acked Read 8 Done Done Sent false
29112 9034 Acked Flush 0 Done Done Sent false
29113 9035 Acked Write 1 Done Done Sent false
29114 9036 Acked Read 9 Done Done Sent false
29115 9037 Acked Write 6 Done Done Sent false
29116 9038 Acked Read 8 Done Done Sent false
29117 9039 Acked Read 7 Done Done Sent false
29118 9040 Acked Read 5 Done Done Sent false
29119 9041 Acked Read 7 Done Done Sent false
29120 9042 Acked Read 1 Done Done Sent false
29121 9043 Acked Read 7 Done Done Sent false
29122 9044 Acked Write 9 Done Done Sent false
29123 9045 Acked Read 6 Done Done Sent false
29124 9046 Acked Flush 0 Done Done Sent false
29125 9047 Acked Read 1 Done Done Sent false
29126 9048 Acked Read 4 Done Done Sent false
29127 9049 Acked Read 8 Done Done Sent false
29128 9050 Acked Write 2 Done Done Sent false
29129 9051 Acked Read 1 Done Done Sent false
29130 9052 Acked Read 6 Done Done Sent false
29131 9053 Acked Read 6 Done Done Sent false
29132 9054 Acked Flush 0 Done Done Sent false
29133 9055 Acked Read 7 Done Done Sent false
29134 9056 Acked Read 10 Done Done Sent false
29135 9057 Acked Read 10 Done Done Sent false
29136 9058 Acked Write 2 Done Done Sent false
29137 9059 Acked Read 9 Done Done Sent false
29138 9060 Acked Read 3 Done Done Sent false
29139 9061 Acked Flush 0 Done Done Sent false
29140 9062 Acked Write 7 Done Done Sent false
29141 9063 Acked Write 5 Done Done Sent false
29142 9064 Acked Read 2 Done Done Sent false
29143 9065 Acked Write 3 Done Done Sent false
29144 9066 Acked Read 10 Done Done Sent false
29145 9067 Acked Read 10 Done Done Sent false
29146 9068 Acked Read 10 Done Done Sent false
29147 9069 Acked Read 4 Done Done Sent false
29148 9070 Acked Read 9 Done Done Sent false
29149 9071 Acked Read 7 Done Done Sent false
29150 9072 Acked Read 2 Done Done Sent false
29151 9073 Acked Read 9 Done Done Sent false
29152 9074 Acked Read 10 Done Done Sent false
29153 9075 Acked Write 5 Done Done Sent false
29154 9076 Acked Read 8 Done Done Sent false
29155 9077 Acked Read 7 Done Done Sent false
29156 9078 Acked Write 1 Done Done Sent false
29157 9079 Acked Read 10 Done Done Sent false
29158 9080 Acked Write 7 Done Done Sent false
29159 9081 Acked Read 4 Done Done Sent false
29160 9082 Acked Read 2 Done Done Sent false
29161 9083 Acked Write 6 Done Done Sent false
29162 9084 Acked Write 4 Done Done Sent false
29163 9085 Acked Write 8 Done Done Sent false
29164 9086 Acked Read 1 Done Done Sent false
29165 9087 Acked Read 9 Done Done Sent false
29166 9088 Acked Write 3 Done Done Sent false
29167 9089 Acked Write 8 Done Done Sent false
29168 9090 Acked Read 7 Done Done Sent false
29169 9091 Acked Read 10 Done Done Sent false
29170 9092 Acked Read 1 Done Done Sent false
29171 9093 Acked Flush 0 Done Done Sent false
29172 9094 Acked Read 2 Done Done Sent false
29173 9095 Acked Write 1 Done Done Sent false
29174 9096 Acked Read 4 Done Done Sent false
29175 9097 Acked Read 4 Done Done Sent false
29176 9098 Acked Write 3 Done Done Sent false
29177 9099 Acked Write 8 Done Done Sent false
29178 9100 Acked Flush 0 Done Done Sent false
29179 9101 Acked Read 8 Done Done Sent false
29180 9102 Acked Flush 0 Done Done Sent false
29181 9103 Acked Write 8 Done Done Sent false
29182 9104 Acked Write 5 Done Done Sent false
29183 9105 Acked Write 8 Done Done Sent false
29184 9106 Acked Write 9 Done Done Sent false
29185 9107 Acked Write 1 Done Done Sent false
29186 9108 Acked Write 8 Done Done Sent false
29187 9109 Acked Flush 0 Done Done Sent false
29188 9110 Acked Read 10 Done Done Sent false
29189 9111 Acked Write 10 Done Done Sent false
29190 9112 Acked Flush 0 Done Done Sent false
29191 9113 Acked Write 1 Done Done Sent false
29192 9114 Acked Write 8 Done Done Sent false
29193 9115 Acked Write 1 Done Done Sent false
29194 9116 Acked Write 3 Done Done Sent false
29195 9117 Acked Read 2 Done Done Sent false
29196 9118 Acked Read 9 Done Done Sent false
29197 9119 Acked Flush 0 Done Done Sent false
29198 9120 Acked Read 5 Done Done Sent false
29199 9121 Acked Read 10 Done Done Sent false
29200 9122 Acked Flush 0 Done Done Sent false
29201 9123 Acked Read 9 Done Done Sent false
29202 9124 Acked Write 3 Done Done Sent false
29203 9125 Acked Write 2 Done Done Sent false
29204 9126 Acked Read 8 Done Done Sent false
29205 9127 Acked Read 6 Done Done Sent false
29206 9128 Acked Write 2 Done Done Sent false
29207 9129 Acked Write 3 Done Done Sent false
29208 9130 Acked Read 2 Done Done Sent false
29209 9131 Acked Flush 0 Done Done Sent false
29210 9132 Acked Write 2 Done Done Sent false
29211 9133 Acked Read 10 Done Done Sent false
29212 9134 Acked Write 5 Done Done Sent false
29213 9135 Acked Write 2 Done Done Sent false
29214 9136 Acked Write 5 Done Done Sent false
29215 9137 Acked Read 7 Done Done Sent false
29216 9138 Acked Read 4 Done Done Sent false
29217 9139 Acked Read 6 Done Done Sent false
29218 9140 Acked Write 4 Done Done Sent false
29219 9141 Acked Read 4 Done Done Sent false
29220 9142 Acked Read 9 Done Done Sent false
29221 9143 Acked Read 9 Done Done Sent false
29222 9144 Acked Write 3 Done Done Sent false
29223 9145 Acked Read 8 Done Done Sent false
29224 9146 Acked Read 3 Done Done Sent false
29225 9147 Acked Write 6 Done Done Sent false
29226 9148 Acked Read 2 Done Done Sent false
29227 9149 Acked Write 1 Done Done Sent false
29228 9150 Acked Write 3 Done Done Sent false
29229 STATES DS:0 DS:1 DS:2 TOTAL
29230 Sent 0 0 301 301
29231 Done 301 301 0 602
29232 Skipped 0 0 0 0
29233 Error 0 0 0 0
29234 Last Flush: 9131 9131 8849
29235 Downstairs last five completed: 8849 8848 8847 8846 8845
29236 Upstairs last five completed: 9150 9149 9148 9147 9146
29237 CLIENT: Up:0 ds:301 act:2
29238 [8830] got start action
29239 [8830] I am going to restart
29240 Starting downstairs at port 8830
29241 Make output file at "/tmp/dsc/downstairs-8830.txt"
29242 [8830][2] reports Starting
29243 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1227)
29244 [8830][2] reports Running
29245 ----------------------------------------------------------------
29246 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29247 States: Active Active Offline
29248 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29249 8850 Acked Read 3 Done Done Sent false
29250 8851 Acked Read 6 Done Done Sent false
29251 8852 Acked Read 7 Done Done Sent false
29252 8853 Acked Write 6 Done Done Sent false
29253 8854 Acked Read 8 Done Done Sent false
29254 8855 Acked Read 9 Done Done Sent false
29255 8856 Acked Write 4 Done Done Sent false
29256 8857 Acked Read 3 Done Done Sent false
29257 8858 Acked Write 3 Done Done Sent false
29258 8859 Acked Write 4 Done Done Sent false
29259 8860 Acked Read 1 Done Done Sent false
29260 8861 Acked Read 6 Done Done Sent false
29261 8862 Acked Write 7 Done Done Sent false
29262 8863 Acked Write 7 Done Done Sent false
29263 8864 Acked Read 4 Done Done Sent false
29264 8865 Acked Write 2 Done Done Sent false
29265 8866 Acked Read 7 Done Done Sent false
29266 8867 Acked Read 1 Done Done Sent false
29267 8868 Acked Write 2 Done Done Sent false
29268 8869 Acked Read 7 Done Done Sent false
29269 8870 Acked Write 4 Done Done Sent false
29270 8871 Acked Read 1 Done Done Sent false
29271 8872 Acked Read 10 Done Done Sent false
29272 8873 Acked Read 7 Done Done Sent false
29273 8874 Acked Read 7 Done Done Sent false
29274 8875 Acked Read 10 Done Done Sent false
29275 8876 Acked Read 4 Done Done Sent false
29276 8877 Acked Read 10 Done Done Sent false
29277 8878 Acked Write 2 Done Done Sent false
29278 8879 Acked Write 7 Done Done Sent false
29279 8880 Acked Read 3 Done Done Sent false
29280 8881 Acked Write 3 Done Done Sent false
29281 8882 Acked Read 3 Done Done Sent false
29282 8883 Acked Write 6 Done Done Sent false
29283 8884 Acked Read 7 Done Done Sent false
29284 8885 Acked Read 7 Done Done Sent false
29285 8886 Acked Read 7 Done Done Sent false
29286 8887 Acked Write 2 Done Done Sent false
29287 8888 Acked Read 6 Done Done Sent false
29288 8889 Acked Write 5 Done Done Sent false
29289 8890 Acked Write 6 Done Done Sent false
29290 8891 Acked Read 6 Done Done Sent false
29291 8892 Acked Read 8 Done Done Sent false
29292 8893 Acked Write 6 Done Done Sent false
29293 8894 Acked Write 5 Done Done Sent false
29294 8895 Acked Read 8 Done Done Sent false
29295 8896 Acked Read 3 Done Done Sent false
29296 8897 Acked Read 3 Done Done Sent false
29297 8898 Acked Read 10 Done Done Sent false
29298 8899 Acked Write 5 Done Done Sent false
29299 8900 Acked Read 3 Done Done Sent false
29300 8901 Acked Write 10 Done Done Sent false
29301 8902 Acked Read 6 Done Done Sent false
29302 8903 Acked Write 10 Done Done Sent false
29303 8904 Acked Write 4 Done Done Sent false
29304 8905 Acked Write 1 Done Done Sent false
29305 8906 Acked Write 6 Done Done Sent false
29306 8907 Acked Read 4 Done Done Sent false
29307 8908 Acked Write 10 Done Done Sent false
29308 8909 Acked Write 3 Done Done Sent false
29309 8910 Acked Read 2 Done Done Sent false
29310 8911 Acked Write 8 Done Done Sent false
29311 8912 Acked Flush 0 Done Done Sent false
29312 8913 Acked Read 8 Done Done Sent false
29313 8914 Acked Write 2 Done Done Sent false
29314 8915 Acked Read 8 Done Done Sent false
29315 8916 Acked Read 2 Done Done Sent false
29316 8917 Acked Read 2 Done Done Sent false
29317 8918 Acked Read 5 Done Done Sent false
29318 8919 Acked Read 7 Done Done Sent false
29319 8920 Acked Write 8 Done Done Sent false
29320 8921 Acked Write 9 Done Done Sent false
29321 8922 Acked Flush 0 Done Done Sent false
29322 8923 Acked Read 8 Done Done Sent false
29323 8924 Acked Write 4 Done Done Sent false
29324 8925 Acked Write 3 Done Done Sent false
29325 8926 Acked Write 4 Done Done Sent false
29326 8927 Acked Flush 0 Done Done Sent false
29327 8928 Acked Write 4 Done Done Sent false
29328 8929 Acked Read 9 Done Done Sent false
29329 8930 Acked Read 1 Done Done Sent false
29330 8931 Acked Write 1 Done Done Sent false
29331 8932 Acked Read 5 Done Done Sent false
29332 8933 Acked Flush 0 Done Done Sent false
29333 8934 Acked Write 8 Done Done Sent false
29334 8935 Acked Read 1 Done Done Sent false
29335 8936 Acked Flush 0 Done Done Sent false
29336 8937 Acked Write 6 Done Done Sent false
29337 8938 Acked Write 10 Done Done Sent false
29338 8939 Acked Flush 0 Done Done Sent false
29339 8940 Acked Write 5 Done Done Sent false
29340 8941 Acked Write 6 Done Done Sent false
29341 8942 Acked Write 5 Done Done Sent false
29342 8943 Acked Write 7 Done Done Sent false
29343 8944 Acked Write 10 Done Done Sent false
29344 8945 Acked Write 6 Done Done Sent false
29345 8946 Acked Write 7 Done Done Sent false
29346 8947 Acked Write 3 Done Done Sent false
29347 8948 Acked Write 10 Done Done Sent false
29348 8949 Acked Flush 0 Done Done Sent false
29349 8950 Acked Read 10 Done Done Sent false
29350 8951 Acked Write 4 Done Done Sent false
29351 8952 Acked Read 10 Done Done Sent false
29352 8953 Acked Write 6 Done Done Sent false
29353 8954 Acked Read 8 Done Done Sent false
29354 8955 Acked Read 8 Done Done Sent false
29355 8956 Acked Write 5 Done Done Sent false
29356 8957 Acked Write 6 Done Done Sent false
29357 8958 Acked Write 9 Done Done Sent false
29358 8959 Acked Write 8 Done Done Sent false
29359 8960 Acked Write 2 Done Done Sent false
29360 8961 Acked Write 3 Done Done Sent false
29361 8962 Acked Read 1 Done Done Sent false
29362 8963 Acked Read 7 Done Done Sent false
29363 8964 Acked Read 2 Done Done Sent false
29364 8965 Acked Read 6 Done Done Sent false
29365 8966 Acked Read 6 Done Done Sent false
29366 8967 Acked Read 3 Done Done Sent false
29367 8968 Acked Read 1 Done Done Sent false
29368 8969 Acked Write 8 Done Done Sent false
29369 8970 Acked Write 8 Done Done Sent false
29370 8971 Acked Write 3 Done Done Sent false
29371 8972 Acked Read 2 Done Done Sent false
29372 8973 Acked Read 8 Done Done Sent false
29373 8974 Acked Flush 0 Done Done Sent false
29374 8975 Acked Flush 0 Done Done Sent false
29375 8976 Acked Write 8 Done Done Sent false
29376 8977 Acked Read 1 Done Done Sent false
29377 8978 Acked Write 3 Done Done Sent false
29378 8979 Acked Write 3 Done Done Sent false
29379 8980 Acked Write 6 Done Done Sent false
29380 8981 Acked Flush 0 Done Done Sent false
29381 8982 Acked Flush 0 Done Done Sent false
29382 8983 Acked Read 8 Done Done Sent false
29383 8984 Acked Read 4 Done Done Sent false
29384 8985 Acked Write 10 Done Done Sent false
29385 8986 Acked Read 3 Done Done Sent false
29386 8987 Acked Write 1 Done Done Sent false
29387 8988 Acked Read 9 Done Done Sent false
29388 8989 Acked Write 1 Done Done Sent false
29389 8990 Acked Read 9 Done Done Sent false
29390 8991 Acked Read 3 Done Done Sent false
29391 8992 Acked Read 1 Done Done Sent false
29392 8993 Acked Write 4 Done Done Sent false
29393 8994 Acked Write 8 Done Done Sent false
29394 8995 Acked Write 9 Done Done Sent false
29395 8996 Acked Read 2 Done Done Sent false
29396 8997 Acked Read 7 Done Done Sent false
29397 8998 Acked Read 2 Done Done Sent false
29398 8999 Acked Read 4 Done Done Sent false
29399 9000 Acked Write 2 Done Done Sent false
29400 9001 Acked Flush 0 Done Done Sent false
29401 9002 Acked Flush 0 Done Done Sent false
29402 9003 Acked Write 2 Done Done Sent false
29403 9004 Acked Write 8 Done Done Sent false
29404 9005 Acked Read 9 Done Done Sent false
29405 9006 Acked Read 8 Done Done Sent false
29406 9007 Acked Write 1 Done Done Sent false
29407 9008 Acked Read 6 Done Done Sent false
29408 9009 Acked Read 7 Done Done Sent false
29409 9010 Acked Read 7 Done Done Sent false
29410 9011 Acked Read 1 Done Done Sent false
29411 9012 Acked Read 7 Done Done Sent false
29412 9013 Acked Write 3 Done Done Sent false
29413 9014 Acked Read 5 Done Done Sent false
29414 9015 Acked Write 6 Done Done Sent false
29415 9016 Acked Write 9 Done Done Sent false
29416 9017 Acked Write 9 Done Done Sent false
29417 9018 Acked Read 5 Done Done Sent false
29418 9019 Acked Write 5 Done Done Sent false
29419 9020 Acked Write 2 Done Done Sent false
29420 9021 Acked Write 8 Done Done Sent false
29421 9022 Acked Read 8 Done Done Sent false
29422 9023 Acked Read 4 Done Done Sent false
29423 9024 Acked Write 4 Done Done Sent false
29424 9025 Acked Read 9 Done Done Sent false
29425 9026 Acked Write 1 Done Done Sent false
29426 9027 Acked Write 10 Done Done Sent false
29427 9028 Acked Read 4 Done Done Sent false
29428 9029 Acked Read 1 Done Done Sent false
29429 9030 Acked Read 2 Done Done Sent false
29430 9031 Acked Write 8 Done Done Sent false
29431 9032 Acked Write 8 Done Done Sent false
29432 9033 Acked Read 8 Done Done Sent false
29433 9034 Acked Flush 0 Done Done Sent false
29434 9035 Acked Write 1 Done Done Sent false
29435 9036 Acked Read 9 Done Done Sent false
29436 9037 Acked Write 6 Done Done Sent false
29437 9038 Acked Read 8 Done Done Sent false
29438 9039 Acked Read 7 Done Done Sent false
29439 9040 Acked Read 5 Done Done Sent false
29440 9041 Acked Read 7 Done Done Sent false
29441 9042 Acked Read 1 Done Done Sent false
29442 9043 Acked Read 7 Done Done Sent false
29443 9044 Acked Write 9 Done Done Sent false
29444 9045 Acked Read 6 Done Done Sent false
29445 9046 Acked Flush 0 Done Done Sent false
29446 9047 Acked Read 1 Done Done Sent false
29447 9048 Acked Read 4 Done Done Sent false
29448 9049 Acked Read 8 Done Done Sent false
29449 9050 Acked Write 2 Done Done Sent false
29450 9051 Acked Read 1 Done Done Sent false
29451 9052 Acked Read 6 Done Done Sent false
29452 9053 Acked Read 6 Done Done Sent false
29453 9054 Acked Flush 0 Done Done Sent false
29454 9055 Acked Read 7 Done Done Sent false
29455 9056 Acked Read 10 Done Done Sent false
29456 9057 Acked Read 10 Done Done Sent false
29457 9058 Acked Write 2 Done Done Sent false
29458 9059 Acked Read 9 Done Done Sent false
29459 9060 Acked Read 3 Done Done Sent false
29460 9061 Acked Flush 0 Done Done Sent false
29461 9062 Acked Write 7 Done Done Sent false
29462 9063 Acked Write 5 Done Done Sent false
29463 9064 Acked Read 2 Done Done Sent false
29464 9065 Acked Write 3 Done Done Sent false
29465 9066 Acked Read 10 Done Done Sent false
29466 9067 Acked Read 10 Done Done Sent false
29467 9068 Acked Read 10 Done Done Sent false
29468 9069 Acked Read 4 Done Done Sent false
29469 9070 Acked Read 9 Done Done Sent false
29470 9071 Acked Read 7 Done Done Sent false
29471 9072 Acked Read 2 Done Done Sent false
29472 9073 Acked Read 9 Done Done Sent false
29473 9074 Acked Read 10 Done Done Sent false
29474 9075 Acked Write 5 Done Done Sent false
29475 9076 Acked Read 8 Done Done Sent false
29476 9077 Acked Read 7 Done Done Sent false
29477 9078 Acked Write 1 Done Done Sent false
29478 9079 Acked Read 10 Done Done Sent false
29479 9080 Acked Write 7 Done Done Sent false
29480 9081 Acked Read 4 Done Done Sent false
29481 9082 Acked Read 2 Done Done Sent false
29482 9083 Acked Write 6 Done Done Sent false
29483 9084 Acked Write 4 Done Done Sent false
29484 9085 Acked Write 8 Done Done Sent false
29485 9086 Acked Read 1 Done Done Sent false
29486 9087 Acked Read 9 Done Done Sent false
29487 9088 Acked Write 3 Done Done Sent false
29488 9089 Acked Write 8 Done Done Sent false
29489 9090 Acked Read 7 Done Done Sent false
29490 9091 Acked Read 10 Done Done Sent false
29491 9092 Acked Read 1 Done Done Sent false
29492 9093 Acked Flush 0 Done Done Sent false
29493 9094 Acked Read 2 Done Done Sent false
29494 9095 Acked Write 1 Done Done Sent false
29495 9096 Acked Read 4 Done Done Sent false
29496 9097 Acked Read 4 Done Done Sent false
29497 9098 Acked Write 3 Done Done Sent false
29498 9099 Acked Write 8 Done Done Sent false
29499 9100 Acked Flush 0 Done Done Sent false
29500 9101 Acked Read 8 Done Done Sent false
29501 9102 Acked Flush 0 Done Done Sent false
29502 9103 Acked Write 8 Done Done Sent false
29503 9104 Acked Write 5 Done Done Sent false
29504 9105 Acked Write 8 Done Done Sent false
29505 9106 Acked Write 9 Done Done Sent false
29506 9107 Acked Write 1 Done Done Sent false
29507 9108 Acked Write 8 Done Done Sent false
29508 9109 Acked Flush 0 Done Done Sent false
29509 9110 Acked Read 10 Done Done Sent false
29510 9111 Acked Write 10 Done Done Sent false
29511 9112 Acked Flush 0 Done Done Sent false
29512 9113 Acked Write 1 Done Done Sent false
29513 9114 Acked Write 8 Done Done Sent false
29514 9115 Acked Write 1 Done Done Sent false
29515 9116 Acked Write 3 Done Done Sent false
29516 9117 Acked Read 2 Done Done Sent false
29517 9118 Acked Read 9 Done Done Sent false
29518 9119 Acked Flush 0 Done Done Sent false
29519 9120 Acked Read 5 Done Done Sent false
29520 9121 Acked Read 10 Done Done Sent false
29521 9122 Acked Flush 0 Done Done Sent false
29522 9123 Acked Read 9 Done Done Sent false
29523 9124 Acked Write 3 Done Done Sent false
29524 9125 Acked Write 2 Done Done Sent false
29525 9126 Acked Read 8 Done Done Sent false
29526 9127 Acked Read 6 Done Done Sent false
29527 9128 Acked Write 2 Done Done Sent false
29528 9129 Acked Write 3 Done Done Sent false
29529 9130 Acked Read 2 Done Done Sent false
29530 9131 Acked Flush 0 Done Done Sent false
29531 9132 Acked Write 2 Done Done Sent false
29532 9133 Acked Read 10 Done Done Sent false
29533 9134 Acked Write 5 Done Done Sent false
29534 9135 Acked Write 2 Done Done Sent false
29535 9136 Acked Write 5 Done Done Sent false
29536 9137 Acked Read 7 Done Done Sent false
29537 9138 Acked Read 4 Done Done Sent false
29538 9139 Acked Read 6 Done Done Sent false
29539 9140 Acked Write 4 Done Done Sent false
29540 9141 Acked Read 4 Done Done Sent false
29541 9142 Acked Read 9 Done Done Sent false
29542 9143 Acked Read 9 Done Done Sent false
29543 9144 Acked Write 3 Done Done Sent false
29544 9145 Acked Read 8 Done Done Sent false
29545 9146 Acked Read 3 Done Done Sent false
29546 9147 Acked Write 6 Done Done Sent false
29547 9148 Acked Read 2 Done Done Sent false
29548 9149 Acked Write 1 Done Done Sent false
29549 9150 Acked Write 3 Done Done Sent false
29550 9151 Acked Flush 0 Done Done Sent false
29551 STATES DS:0 DS:1 DS:2 TOTAL
29552 Sent 0 0 302 302
29553 Done 302 302 0 604
29554 Skipped 0 0 0 0
29555 Error 0 0 0 0
29556 Last Flush: 9151 9151 8849
29557 Downstairs last five completed: 8849 8848 8847 8846 8845
29558 Upstairs last five completed: 9151 9150 9149 9148 9147
29559 CLIENT: Up:0 ds:302 act:2
295602025-01-21T15:53:21.797ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
295612025-01-21T15:53:21.798ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61698) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
295622025-01-21T15:53:21.799ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
295632025-01-21T15:53:21.799ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
295642025-01-21T15:53:21.799ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
295652025-01-21T15:53:21.799ZINFOcrucible: send last flush ID to this DS: 8849 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
295662025-01-21T15:53:21.799ZINFOcrucible: Replied this last flush ID: 8849 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
295672025-01-21T15:53:21.800ZINFOcrucible: [2] client re-new 302 jobs since flush 8849 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
295682025-01-21T15:53:21.800ZINFOcrucible: [2] Replayed 302 jobs since flush: 8849 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
29569 ----------------------------------------------------------------
29570 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29571 Downstairs last five completed: 9151 9150 9149 9148 9147
29572 Upstairs last five completed: 9151 9150 9149 9148 9147
29573 CLIENT: Up:0 ds:0 act:3
29574 Replay: All jobs finished
29575 Main task has work to do, go find it
29576 got dsc Stop(1)
29577 stop 1
29578 Jan 21 15:53:24.047 INFO request completed, latency_us: [8820] Got stop action so:false kr:false
29579 411, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 110f6569-9256-4345-8667-c98aca5d436f, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
295802025-01-21T15:53:24.048ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
29581 Jan 21 15:53:24.048 INFO request completed, latency_us: 114, response_code: 200, uri: /state/cid/1, method: GET, req_id: 91d586f6-ef03-42eb-8784-fe14aaee2e93, remote_addr: 127{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:53:24.048565707Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
29582 .0.0.1:52945{"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-21T15:53:24.048606245Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
29583 , local_addr: 127.0.0.1:9998
295842025-01-21T15:53:24.048ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
29585 [8820] Exited with: ExitStatus(unix_wait_status(9))
29586 [8820][1] reports Exit
29587 Jan 21 15:53:28.050 INFO request completed, latency_us: 212, response_code: 200, uri: /state/cid/1, method: GET, req_id: f03eb476-22fe-4c5c-93a4-3f43858b05af, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
29588 001/300 Flush
29589 002/300 Read block 2464 len 40960
29590 003/300 Write block 1865 len 4096 data: 10
29591 004/300 Write block 993 len 40960 data: 7 7 9 9 8 7 8 9 8 6
29592 005/300 Write block 2742 len 28672 data: 4 3 3 4 4 5 6
29593 006/300 Read block 2860 len 40960
29594 007/300 Write block 3355 len 40960 data: 8 7 5 4 4 5 4 5 5 4
29595 008/300 Read block 1523 len 20480
29596 009/300 Write block 2748 len 20480 data: 7 6 6 6 5
29597 010/300 Write block 393 len 20480 data: 3 4 4 4 4
29598 011/300 Write block 4634 len 4096 data: 4
29599 012/300 Read block 1796 len 28672
29600 013/300 Write block 1237 len 20480 data: 3 3 4 5 5
29601 014/300 Read block 3301 len 8192
29602 015/300 Read block 2946 len 24576
29603 016/300 Read block 3970 len 32768
29604 017/300 Write block 3364 len 32768 data: 5 4 5 5 6 8 7 7
29605 018/300 Read block 4290 len 32768
29606 019/300 Read block 3952 len 8192
29607 020/300 Read block 1540 len 8192
29608 021/300 Read block 1996 len 12288
29609 022/300 Read block 1675 len 8192
29610 023/300 Read block 503 len 32768
29611 024/300 Read block 2793 len 40960
29612 025/300 Write block 3425 len 28672 data: 6 3 3 3 3 4 3
29613 026/300 Read block 90 len 20480
29614 027/300 Read block 2700 len 20480
29615 028/300 Flush
29616 029/300 Read block 4845 len 28672
29617 030/300 Read block 1007 len 20480
29618 031/300 Read block 3764 len 28672
29619 032/300 Read block 2005 len 20480
29620 033/300 Write block 1977 len 40960 data: 4 6 7 7 8 9 11 11 12 12
29621 034/300 Write block 3224 len 16384 data: 5 6 7 7
29622 035/300 Flush
29623 036/300 Write block 594 len 12288 data: 8 8 9
29624 037/300 Read block 4902 len 16384
29625 038/300 Write block 2702 len 32768 data: 5 6 4 3 3 7 7 7
29626 039/300 Read block 2495 len 32768
29627 040/300 Read block 4386 len 28672
29628 041/300 Flush
29629 042/300 Read block 3028 len 32768
29630 043/300 Write block 2223 len 20480 data: 6 8 7 7 10
29631 044/300 Read block 4398 len 12288
29632 045/300 Write block 1383 len 8192 data: 8 6
29633 046/300 Write block 804 len 20480 data: 9 9 9 10 8
29634 047/300 Read block 3096 len 4096
29635 048/300 Write block 4050 len 4096 data: 7
29636 049/300 Read block 1584 len 20480
29637 050/300 Write block 2216 len 36864 data: 3 4 4 5 6 4 5 7 9
29638 051/300 Read block 3566 len 8192
29639 052/300 Write block 2551 len 4096 data: 2
29640 053/300 Write block 1114 len 24576 data: 4 4 6 5 7 8
29641 054/300 Flush
29642 055/300 Read block 2762 len 8192
29643 056/300 Write block 1716 len 12288 data: 6 5 5
29644 057/300 Write block 1855 len 40960 data: 4 5 5 3 4 5 5 7 9 9
29645 058/300 Read block 3798 len 16384
29646 059/300 Read block 4610 len 36864
29647 060/300 Write block 870 len 20480 data: 4 3 3 3 3
29648 061/300 Write block 2819 len 40960 data: 6 6 6 8 8 7 7 6 8 7
29649 062/300 Write block 1288 len 16384 data: 4 5 5 6
29650 063/300 Write block 1284 len 12288 data: 3 3 4
29651 064/300 Write block 4620 len 20480 data: 5 7 8 9 10
29652 065/300 Write block 3906 len 28672 data: 5 5 5 4 3 4 4
29653 066/300 Read block 1189 len 8192
29654 067/300 Read block 4512 len 20480
29655 068/300 Write block 2532 len 12288 data: 7 7 7
29656 069/300 Write block 4556 len 4096 data: 4
29657 070/300 Read block 3404 len 24576
29658 071/300 Read block 4832 len 24576
29659 072/300 Read block 3342 len 20480
29660 073/300 Read block 2008 len 32768
29661 074/300 Read block 4600 len 40960
29662 075/300 Write block 2727 len 40960 data: 4 4 4 5 5 5 6 5 5 6
29663 076/300 Read block 2051 len 12288
29664 077/300 Read block 1727 len 8192
29665 078/300 Write block 3337 len 24576 data: 4 4 4 4 4 4
29666 079/300 Read block 764 len 32768
29667 080/300 Read block 1142 len 20480
29668 081/300 Read block 2455 len 20480
29669 082/300 Read block 612 len 8192
29670 083/300 Write block 48 len 12288 data: 8 9 7
29671 084/300 Read block 3056 len 28672
29672 085/300 Write block 3277 len 12288 data: 4 4 4
29673 086/300 Read block 4463 len 24576
29674 087/300 Read block 1198 len 40960
29675 088/300 Write block 1554 len 24576 data: 5 5 6 6 6 5
29676 089/300 Read block 4327 len 16384
29677 090/300 Write block 677 len 20480 data: 5 5 4 4 4
29678 091/300 Flush
29679 092/300 Flush
29680 093/300 Write block 2781 len 20480 data: 7 6 6 8 6
29681 094/300 Read block 589 len 12288
29682 095/300 Write block 3793 len 8192 data: 10 9
29683 096/300 Flush
29684 097/300 Write block 85 len 8192 data: 5 4
29685 098/300 Read block 2609 len 12288
29686 099/300 Read block 1226 len 4096
29687 100/300 Write block 2658 len 16384 data: 5 6 6 7
29688 101/300 Write block 725 len 12288 data: 7 7 7
29689 102/300 Flush
29690 103/300 Flush
29691 104/300 Read block 4845 len 40960
29692 105/300 Read block 439 len 36864
29693 106/300 Read block 372 len 8192
29694 107/300 Write block 4564 len 4096 data: 4
29695 108/300 Read block 3350 len 24576
29696 109/300 Read block 3747 len 4096
29697 110/300 Write block 18 len 4096 data: 5
29698 111/300 Write block 94 len 20480 data: 4 3 2 2 2
29699 112/300 Read block 2843 len 8192
29700 113/300 Read block 1857 len 32768
29701 114/300 Read block 2655 len 28672
29702 115/300 Read block 3448 len 16384
29703 116/300 Flush
29704 117/300 Read block 1516 len 28672
29705 118/300 Write block 982 len 12288 data: 5 6 5
29706 119/300 Read block 3945 len 4096
29707 120/300 Write block 3188 len 20480 data: 2 3 3 3 3
29708 121/300 Read block 1464 len 4096
29709 122/300 Flush
29710 123/300 Read block 2176 len 32768
29711 124/300 Write block 4339 len 8192 data: 3 5
29712 125/300 Write block 498 len 40960 data: 4 5 4 5 7 9 8 7 7 6
29713 126/300 Write block 2421 len 8192 data: 7 8
29714 127/300 Write block 1038 len 4096 data: 10
29715 128/300 Read block 2472 len 8192
29716 129/300 Read block 3654 len 16384
29717 130/300 Read block 3142 len 20480
29718 131/300 Flush
29719 132/300 Read block 1654 len 28672
29720 133/300 Write block 3407 len 24576 data: 7 6 7 6 6 5
29721 134/300 Read block 203 len 4096
29722 135/300 Read block 4061 len 12288
29723 136/300 Read block 2453 len 28672
29724 137/300 Flush
29725 138/300 Read block 4370 len 20480
29726 139/300 Write block 4453 len 24576 data: 7 5 5 4 6 5
29727 140/300 Flush
29728 141/300 Read block 4900 len 36864
29729 142/300 Write block 1968 len 32768 data: 3 3 3 2 3 4 5 5
29730 143/300 Read block 1014 len 24576
29731 144/300 Read block 4180 len 24576
29732 145/300 Read block 1718 len 16384
29733 146/300 Read block 2206 len 24576
29734 147/300 Read block 3010 len 40960
29735 148/300 Read block 4769 len 8192
29736 149/300 Write block 4263 len 12288 data: 2 2 2
29737 150/300 Write block 4435 len 20480 data: 7 9 9 8 9
29738 151/300 Write block 1949 len 28672 data: 5 6 5 6 5 4 3
29739 152/300 Read block 415 len 32768
29740 153/300 Flush
29741 154/300 Flush
29742 155/300 Read block 4578 len 24576
29743 156/300 Write block 3703 len 16384 data: 13 12 8 6
29744 157/300 Write block 1967 len 24576 data: 4 4 4 4 3 4
29745 158/300 Read block 3794 len 12288
29746 159/300 Write block 2444 len 32768 data: 6 5 7 9 8 8 6 6
29747 160/300 Read block 1168 len 36864
29748 161/300 Read block 1116 len 16384
29749 162/300 Read block 4988 len 28672
29750 163/300 Read block 1939 len 24576
29751 164/300 Read block 941 len 40960
29752 165/300 Read block 3560 len 16384
29753 166/300 Write block 4901 len 20480 data: 4 3 5 5 5
29754 167/300 Write block 2503 len 4096 data: 3
29755 168/300 Read block 2858 len 36864
29756 169/300 Write block 4554 len 20480 data: 3 3 5 6 5
29757 170/300 Write block 443 len 28672 data: 4 4 4 5 6 6 5
29758 171/300 Write block 77 len 20480 data: 6 7 5 5 5
29759 172/300 Write block 30 len 20480 data: 5 6 6 4 2
29760 173/300 Write block 3739 len 24576 data: 4 4 3 5 5 4
29761 174/300 Flush
29762 175/300 Write block 3680 len 36864 data: 6 7 6 6 5 5 4 3 3
29763 176/300 Read block 3012 len 36864
29764 177/300 Write block 1862 len 4096 data: 8
29765 178/300 Read block 3590 len 16384
29766 179/300 Read block 275 len 16384
29767 180/300 Write block 4840 len 8192 data: 4 6
29768 181/300 Write block 509 len 40960 data: 6 6 4 4 5 5 4 3 4 4
29769 182/300 Write block 4922 len 16384 data: 4 5 5 5
29770 183/300 Read block 3796 len 36864
29771 184/300 Write block 2350 len 32768 data: 6 8 8 8 9 9 8 7
29772 185/300 Write block 1130 len 12288 data: 4 4 5
29773 186/300 Flush
29774 187/300 Flush
29775 188/300 Read block 3033 len 28672
29776 189/300 Read block 4623 len 20480
29777 190/300 Read block 4545 len 20480
29778 191/300 Flush
29779 192/300 Write block 852 len 16384 data: 7 5 6 8
29780 193/300 Write block 1910 len 28672 data: 7 8 8 9 8 8 8
29781 194/300 Write block 1663 len 28672 data: 11 10 8 8 8 9 9
29782 195/300 Write block 4651 len 28672 data: 4 5 7 5 5 5 6
29783 196/300 Write block 831 len 12288 data: 6 5 4
29784 197/300 Write block 4261 len 4096 data: 3
29785 198/300 Write block 2130 len 32768 data: 6 6 8 8 7 6 5 6
29786 199/300 Read block 1438 len 36864
29787 200/300 Write block 3759 len 40960 data: 7 5 4 5 5 6 6 5 5 8
29788 201/300 Read block 207 len 24576
29789 202/300 Write block 2829 len 28672 data: 5 5 5 6 4 4 4
29790 203/300 Read block 3402 len 12288
29791 204/300 Read block 4864 len 28672
29792 205/300 Read block 4333 len 36864
29793 206/300 Read block 4406 len 36864
29794 207/300 Write block 288 len 20480 data: 6 6 6 5 5
29795 208/300 Read block 1837 len 8192
29796 209/300 Write block 1209 len 40960 data: 4 2 2 2 2 3 4 4 4 4
29797 210/300 Write block 1894 len 24576 data: 4 4 6 7 5 5
29798 211/300 Read block 1997 len 40960
29799 212/300 Write block 4844 len 16384 data: 8 8 7 8
29800 213/300 Read block 3780 len 32768
29801 214/300 Write block 3773 len 4096 data: 6
29802 215/300 Write block 1535 len 20480 data: 6 5 5 5 5
29803 216/300 Read block 1998 len 16384
29804 217/300 Write block 731 len 20480 data: 8 7 6 5 6
29805 218/300 Write block 1307 len 40960 data: 6 5 5 6 8 10 9 7 8 8
29806 219/300 Read block 4635 len 40960
29807 220/300 Write block 4759 len 16384 data: 8 6 6 7
29808 221/300 Read block 4771 len 28672
29809 222/300 Flush
29810 223/300 Write block 1131 len 4096 data: 5
29811 224/300 Write block 2662 len 28672 data: 6 5 5 6 6 6 7
29812 225/300 Read block 3229 len 16384
29813 226/300 Write block 2157 len 32768 data: 4 3 3 4 3 4 4 5
29814 227/300 Read block 537 len 20480
29815 228/300 Read block 4647 len 4096
29816 229/300 Write block 3493 len 16384 data: 8 7 3 6
29817 230/300 Read block 4544 len 12288
29818 231/300 Read block 3546 len 36864
29819 232/300 Read block 3218 len 24576
29820 233/300 Write block 2258 len 36864 data: 7 6 6 5 6 7 6 5 6
29821 234/300 Write block 274 len 24576 data: 7 7 5 5 5 4
29822 235/300 Read block 40 len 4096
29823 236/300 Read block 40 len 16384
29824 237/300 Write block 3161 len 32768 data: 4 6 7 8 8 5 6 5
29825 238/300 Read block 1307 len 20480
29826 239/300 Write block 1712 len 16384 data: 7 6 5 6
29827 240/300 Read block 1397 len 20480
29828 241/300 Read block 1873 len 28672
29829 242/300 Write block 101 len 8192 data: 4 4
29830 243/300 Flush
29831 244/300 Read block 2329 len 12288
29832 245/300 Read block 2592 len 24576
29833 246/300 Read block 2216 len 8192
29834 247/300 Read block 3883 len 36864
29835 248/300 Read block 764 len 12288
29836 249/300 Read block 129 len 40960
29837 250/300 Read block 3522 len 28672
29838 251/300 Write block 4698 len 36864 data: 4 6 5 5 6 7 7 5 6
29839 252/300 Write block 1298 len 36864 data: 4 6 6 6 6 6 8 7 6
29840 253/300 Write block 1644 len 24576 data: 4 3 3 3 4 4
29841 254/300 Write block 823 len 24576 data: 8 10 7 6 4 4
29842 255/300 Read block 3486 len 24576
29843 256/300 Read block 2701 len 8192
29844 257/300 Read block 3325 len 20480
29845 258/300 Read block 2186 len 28672
29846 259/300 Flush
29847 260/300 Flush
29848 261/300 Write block 596 len 16384 data: 10 8 6 8
29849 262/300 Read block 1162 len 36864
29850 263/300 Write block 667 len 24576 data: 5 4 4 4 5 5
29851 264/300 Read block 4598 len 32768
29852 265/300 Write block 673 len 36864 data: 4 4 5 5 6 6 5 5 5
29853 266/300 Read block 1785 len 28672
29854 267/300 Write block 3013 len 4096 data: 4
29855 268/300 Write block 1921 len 8192 data: 5 5
29856 269/300 Read block 1970 len 4096
29857 270/300 Read block 3484 len 28672
29858 271/300 Read block 3137 len 16384
29859 272/300 Read block 3473 len 28672
29860 273/300 Read block 1969 len 16384
29861 274/300 Read block 3782 len 36864
29862 275/300 Write block 2513 len 40960 data: 4 3 3 4 4 5 7 6 7 8
29863 276/300 Read block 2163 len 16384
29864 277/300 Flush
29865 278/300 Write block 4157 len 16384 data: 4 4 5 5
29866 279/300 Read block 446 len 28672
29867 280/300 Read block 2894 len 4096
29868 281/300 Read block 4406 len 16384
29869 282/300 Write block 3523 len 24576 data: 8 8 7 8 9 10
29870 283/300 Read block 2797 len 36864
29871 284/300 Read block 3586 len 8192
29872 285/300 Write block 1331 len 16384 data: 8 7 7 6
29873 286/300 Write block 3568 len 32768 data: 8 8 7 7 6 4 5 6
29874 287/300 Read block 3704 len 4096
29875 288/300 Read block 3427 len 16384
29876 289/300 Read block 3848 len 36864
29877 290/300 Read block 4732 len 16384
29878 291/300 Read block 1011 len 32768
29879 292/300 Write block 735 len 28672 data: 7 6 7 8 11 10 7
29880 293/300 Read block 2114 len 28672
29881 294/300 Write block 1721 len 40960 data: 6 5 4 4 4 4 4 5 4 4
29882 295/300 Write block 263 len 12288 data: 8 5 4
29883 296/300 Read block 642 len 4096
29884 297/300 Read block 3305 len 16384
29885 298/300 Write block 3818 len 24576 data: 7 5 6 5 7 6
29886 299/300 Write block 2773 len 36864 data: 5 6 5 6 7 7 6 8 8
29887 300/300 Write block 573 len 36864 data: 6 5 5 4 4 6 4 5 5
29888 301/300 Read block 1287 len 12288
29889 Main task has work to do, go find it
29890 got dsc Start(1)
29891 start 1
29892 Jan 21 15:53:28.293 INFO request completed, latency_us: 257, response_code: 204, uri: /start/cid/1, method: POST, req_id: 7c2c3f89-40cc-4433-9804-9ed356cfa5bf, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
29893 Replay: started 1, returned:Ok(())
29894 ----------------------------------------------------------------
29895 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29896 States: Active Offline Active
29897 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29898 9152 Acked Flush 0 Done Sent Done false
29899 9153 Acked Read 10 Done Sent Done false
29900 9154 Acked Write 1 Done Sent Done false
29901 9155 Acked Write 10 Done Sent Done false
29902 9156 Acked Write 7 Done Sent Done false
29903 9157 Acked Read 10 Done Sent Done false
29904 9158 Acked Write 10 Done Sent Done false
29905 9159 Acked Read 5 Done Sent Done false
29906 9160 Acked Write 5 Done Sent Done false
29907 9161 Acked Write 5 Done Sent Done false
29908 9162 Acked Write 1 Done Sent Done false
29909 9163 Acked Read 7 Done Sent Done false
29910 9164 Acked Write 5 Done Sent Done false
29911 9165 Acked Read 2 Done Sent Done false
29912 9166 Acked Read 6 Done Sent Done false
29913 9167 Acked Read 8 Done Sent Done false
29914 9168 Acked Write 8 Done Sent Done false
29915 9169 Acked Read 8 Done Sent Done false
29916 9170 Acked Read 2 Done Sent Done false
29917 9171 Acked Read 2 Done Sent Done false
29918 9172 Acked Read 3 Done Sent Done false
29919 9173 Acked Read 2 Done Sent Done false
29920 9174 Acked Read 8 Done Sent Done false
29921 9175 Acked Read 10 Done Sent Done false
29922 9176 Acked Write 7 Done Sent Done false
29923 9177 Acked Read 5 Done Sent Done false
29924 9178 Acked Read 5 Done Sent Done false
29925 9179 Acked Flush 0 Done Sent Done false
29926 9180 Acked Read 7 Done Sent Done false
29927 9181 Acked Read 5 Done Sent Done false
29928 9182 Acked Read 7 Done Sent Done false
29929 9183 Acked Read 5 Done Sent Done false
29930 9184 Acked Write 10 Done Sent Done false
29931 9185 Acked Write 4 Done Sent Done false
29932 9186 Acked Flush 0 Done Sent Done false
29933 9187 Acked Write 3 Done Sent Done false
29934 9188 Acked Read 4 Done Sent Done false
29935 9189 Acked Write 8 Done Sent Done false
29936 9190 Acked Read 8 Done Sent Done false
29937 9191 Acked Read 7 Done Sent Done false
29938 9192 Acked Flush 0 Done Sent Done false
29939 9193 Acked Read 8 Done Sent Done false
29940 9194 Acked Write 5 Done Sent Done false
29941 9195 Acked Read 3 Done Sent Done false
29942 9196 Acked Write 2 Done Sent Done false
29943 9197 Acked Write 5 Done Sent Done false
29944 9198 Acked Read 1 Done Sent Done false
29945 9199 Acked Write 1 Done Sent Done false
29946 9200 Acked Read 5 Done Sent Done false
29947 9201 Acked Write 9 Done Sent Done false
29948 9202 Acked Read 2 Done Sent Done false
29949 9203 Acked Write 1 Done Sent Done false
29950 9204 Acked Write 6 Done Sent Done false
29951 9205 Acked Flush 0 Done Sent Done false
29952 9206 Acked Read 2 Done Sent Done false
29953 9207 Acked Write 3 Done Sent Done false
29954 9208 Acked Write 10 Done Sent Done false
29955 9209 Acked Read 4 Done Sent Done false
29956 9210 Acked Read 9 Done Sent Done false
29957 9211 Acked Write 5 Done Sent Done false
29958 9212 Acked Write 10 Done Sent Done false
29959 9213 Acked Write 4 Done Sent Done false
29960 9214 Acked Write 3 Done Sent Done false
29961 9215 Acked Write 5 Done Sent Done false
29962 9216 Acked Write 7 Done Sent Done false
29963 9217 Acked Read 2 Done Sent Done false
29964 9218 Acked Read 5 Done Sent Done false
29965 9219 Acked Write 3 Done Sent Done false
29966 9220 Acked Write 1 Done Sent Done false
29967 9221 Acked Read 6 Done Sent Done false
29968 9222 Acked Read 6 Done Sent Done false
29969 9223 Acked Read 5 Done Sent Done false
29970 9224 Acked Read 8 Done Sent Done false
29971 9225 Acked Read 10 Done Sent Done false
29972 9226 Acked Write 10 Done Sent Done false
29973 9227 Acked Read 3 Done Sent Done false
29974 9228 Acked Read 2 Done Sent Done false
29975 9229 Acked Write 6 Done Sent Done false
29976 9230 Acked Read 8 Done Sent Done false
29977 9231 Acked Read 5 Done Sent Done false
29978 9232 Acked Read 5 Done Sent Done false
29979 9233 Acked Read 2 Done Sent Done false
29980 9234 Acked Write 3 Done Sent Done false
29981 9235 Acked Read 7 Done Sent Done false
29982 9236 Acked Write 3 Done Sent Done false
29983 9237 Acked Read 6 Done Sent Done false
29984 9238 Acked Read 10 Done Sent Done false
29985 9239 Acked Write 6 Done Sent Done false
29986 9240 Acked Read 4 Done Sent Done false
29987 9241 Acked Write 5 Done Sent Done false
29988 9242 Acked Flush 0 Done Sent Done false
29989 9243 Acked Flush 0 Done Sent Done false
29990 9244 Acked Write 5 Done Sent Done false
29991 9245 Acked Read 3 Done Sent Done false
29992 9246 Acked Write 2 Done Sent Done false
29993 9247 Acked Flush 0 Done Sent Done false
29994 9248 Acked Write 2 Done Sent Done false
29995 9249 Acked Read 3 Done Sent Done false
29996 9250 Acked Read 1 Done Sent Done false
29997 9251 Acked Write 4 Done Sent Done false
29998 9252 Acked Write 3 Done Sent Done false
29999 9253 Acked Flush 0 Done Sent Done false
30000 9254 Acked Flush 0 Done Sent Done false
30001 9255 Acked Read 10 Done Sent Done false
30002 9256 Acked Read 9 Done Sent Done false
30003 9257 Acked Read 2 Done Sent Done false
30004 9258 Acked Write 1 Done Sent Done false
30005 9259 Acked Read 6 Done Sent Done false
30006 9260 Acked Read 1 Done Sent Done false
30007 9261 Acked Write 1 Done Sent Done false
30008 9262 Acked Write 5 Done Sent Done false
30009 9263 Acked Read 2 Done Sent Done false
30010 9264 Acked Read 8 Done Sent Done false
30011 9265 Acked Read 7 Done Sent Done false
30012 9266 Acked Read 4 Done Sent Done false
30013 9267 Acked Flush 0 Done Sent Done false
30014 9268 Acked Read 7 Done Sent Done false
30015 9269 Acked Write 3 Done Sent Done false
30016 9270 Acked Read 1 Done Sent Done false
30017 9271 Acked Write 5 Done Sent Done false
30018 9272 Acked Read 1 Done Sent Done false
30019 9273 Acked Flush 0 Done Sent Done false
30020 9274 Acked Read 8 Done Sent Done false
30021 9275 Acked Write 2 Done Sent Done false
30022 9276 Acked Write 10 Done Sent Done false
30023 9277 Acked Write 2 Done Sent Done false
30024 9278 Acked Write 1 Done Sent Done false
30025 9279 Acked Read 2 Done Sent Done false
30026 9280 Acked Read 4 Done Sent Done false
30027 9281 Acked Read 5 Done Sent Done false
30028 9282 Acked Flush 0 Done Sent Done false
30029 9283 Acked Read 7 Done Sent Done false
30030 9284 Acked Write 6 Done Sent Done false
30031 9285 Acked Read 1 Done Sent Done false
30032 9286 Acked Read 3 Done Sent Done false
30033 9287 Acked Read 7 Done Sent Done false
30034 9288 Acked Flush 0 Done Sent Done false
30035 9289 Acked Read 5 Done Sent Done false
30036 9290 Acked Write 6 Done Sent Done false
30037 9291 Acked Flush 0 Done Sent Done false
30038 9292 Acked Read 9 Done Sent Done false
30039 9293 Acked Write 8 Done Sent Done false
30040 9294 Acked Read 6 Done Sent Done false
30041 9295 Acked Read 6 Done Sent Done false
30042 9296 Acked Read 4 Done Sent Done false
30043 9297 Acked Read 6 Done Sent Done false
30044 9298 Acked Read 10 Done Sent Done false
30045 9299 Acked Read 2 Done Sent Done false
30046 9300 Acked Write 3 Done Sent Done false
30047 9301 Acked Write 5 Done Sent Done false
30048 9302 Acked Write 7 Done Sent Done false
30049 9303 Acked Read 8 Done Sent Done false
30050 9304 Acked Flush 0 Done Sent Done false
30051 9305 Acked Flush 0 Done Sent Done false
30052 9306 Acked Read 6 Done Sent Done false
30053 9307 Acked Write 4 Done Sent Done false
30054 9308 Acked Write 6 Done Sent Done false
30055 9309 Acked Read 3 Done Sent Done false
30056 9310 Acked Write 8 Done Sent Done false
30057 9311 Acked Read 9 Done Sent Done false
30058 9312 Acked Read 4 Done Sent Done false
30059 9313 Acked Read 7 Done Sent Done false
30060 9314 Acked Read 6 Done Sent Done false
30061 9315 Acked Read 10 Done Sent Done false
30062 9316 Acked Read 4 Done Sent Done false
30063 9317 Acked Write 5 Done Sent Done false
30064 9318 Acked Write 1 Done Sent Done false
30065 9319 Acked Read 9 Done Sent Done false
30066 9320 Acked Write 5 Done Sent Done false
30067 9321 Acked Write 7 Done Sent Done false
30068 9322 Acked Write 5 Done Sent Done false
30069 9323 Acked Write 5 Done Sent Done false
30070 9324 Acked Write 6 Done Sent Done false
30071 9325 Acked Flush 0 Done Sent Done false
30072 9326 Acked Write 9 Done Sent Done false
30073 9327 Acked Read 9 Done Sent Done false
30074 9328 Acked Write 1 Done Sent Done false
30075 9329 Acked Read 4 Done Sent Done false
30076 9330 Acked Read 4 Done Sent Done false
30077 9331 Acked Write 2 Done Sent Done false
30078 9332 Acked Write 10 Done Sent Done false
30079 9333 Acked Write 4 Done Sent Done false
30080 9334 Acked Read 9 Done Sent Done false
30081 9335 Acked Write 8 Done Sent Done false
30082 9336 Acked Write 3 Done Sent Done false
30083 9337 Acked Flush 0 Done Sent Done false
30084 9338 Acked Flush 0 Done Sent Done false
30085 9339 Acked Read 7 Done Sent Done false
30086 9340 Acked Read 5 Done Sent Done false
30087 9341 Acked Read 5 Done Sent Done false
30088 9342 Acked Flush 0 Done Sent Done false
30089 9343 Acked Write 4 Done Sent Done false
30090 9344 Acked Write 7 Done Sent Done false
30091 9345 Acked Write 7 Done Sent Done false
30092 9346 Acked Write 7 Done Sent Done false
30093 9347 Acked Write 3 Done Sent Done false
30094 9348 Acked Write 1 Done Sent Done false
30095 9349 Acked Write 8 Done Sent Done false
30096 9350 Acked Read 9 Done Sent Done false
30097 9351 Acked Write 10 Done Sent Done false
30098 9352 Acked Read 6 Done Sent Done false
30099 9353 Acked Write 7 Done Sent Done false
30100 9354 Acked Read 3 Done Sent Done false
30101 9355 Acked Read 7 Done Sent Done false
30102 9356 Acked Read 9 Done Sent Done false
30103 9357 Acked Read 9 Done Sent Done false
30104 9358 Acked Write 5 Done Sent Done false
30105 9359 Acked Read 2 Done Sent Done false
30106 9360 Acked Write 10 Done Sent Done false
30107 9361 Acked Write 6 Done Sent Done false
30108 9362 Acked Read 10 Done Sent Done false
30109 9363 Acked Write 4 Done Sent Done false
30110 9364 Acked Read 8 Done Sent Done false
30111 9365 Acked Write 1 Done Sent Done false
30112 9366 Acked Write 5 Done Sent Done false
30113 9367 Acked Read 4 Done Sent Done false
30114 9368 Acked Write 5 Done Sent Done false
30115 9369 Acked Write 10 Done Sent Done false
30116 9370 Acked Read 10 Done Sent Done false
30117 9371 Acked Write 4 Done Sent Done false
30118 9372 Acked Read 7 Done Sent Done false
30119 9373 Acked Flush 0 Done Sent Done false
30120 9374 Acked Write 1 Done Sent Done false
30121 9375 Acked Write 7 Done Sent Done false
30122 9376 Acked Read 4 Done Sent Done false
30123 9377 Acked Write 8 Done Sent Done false
30124 9378 Acked Read 5 Done Sent Done false
30125 9379 Acked Read 1 Done Sent Done false
30126 9380 Acked Write 4 Done Sent Done false
30127 9381 Acked Read 3 Done Sent Done false
30128 9382 Acked Read 9 Done Sent Done false
30129 9383 Acked Read 6 Done Sent Done false
30130 9384 Acked Write 9 Done Sent Done false
30131 9385 Acked Write 6 Done Sent Done false
30132 9386 Acked Read 1 Done Sent Done false
30133 9387 Acked Read 4 Done Sent Done false
30134 9388 Acked Write 8 Done Sent Done false
30135 9389 Acked Read 5 Done Sent Done false
30136 9390 Acked Write 4 Done Sent Done false
30137 9391 Acked Read 5 Done Sent Done false
30138 9392 Acked Read 7 Done Sent Done false
30139 9393 Acked Write 2 Done Sent Done false
30140 9394 Acked Flush 0 Done Sent Done false
30141 9395 Acked Read 3 Done Sent Done false
30142 9396 Acked Read 6 Done Sent Done false
30143 9397 Acked Read 2 Done Sent Done false
30144 9398 Acked Read 9 Done Sent Done false
30145 9399 Acked Read 3 Done Sent Done false
30146 9400 Acked Read 10 Done Sent Done false
30147 9401 Acked Read 7 Done Sent Done false
30148 9402 Acked Write 9 Done Sent Done false
30149 9403 Acked Write 9 Done Sent Done false
30150 9404 Acked Write 6 Done Sent Done false
30151 9405 Acked Write 6 Done Sent Done false
30152 9406 Acked Read 6 Done Sent Done false
30153 9407 Acked Read 2 Done Sent Done false
30154 9408 Acked Read 5 Done Sent Done false
30155 9409 Acked Read 7 Done Sent Done false
30156 9410 Acked Flush 0 Done Sent Done false
30157 9411 Acked Flush 0 Done Sent Done false
30158 9412 Acked Write 4 Done Sent Done false
30159 9413 Acked Read 9 Done Sent Done false
30160 9414 Acked Write 6 Done Sent Done false
30161 9415 Acked Read 8 Done Sent Done false
30162 9416 Acked Write 9 Done Sent Done false
30163 9417 Acked Read 7 Done Sent Done false
30164 9418 Acked Write 1 Done Sent Done false
30165 9419 Acked Write 2 Done Sent Done false
30166 9420 Acked Read 1 Done Sent Done false
30167 9421 Acked Read 7 Done Sent Done false
30168 9422 Acked Read 4 Done Sent Done false
30169 9423 Acked Read 7 Done Sent Done false
30170 9424 Acked Read 4 Done Sent Done false
30171 9425 Acked Read 9 Done Sent Done false
30172 9426 Acked Write 10 Done Sent Done false
30173 9427 Acked Read 4 Done Sent Done false
30174 9428 Acked Flush 0 Done Sent Done false
30175 9429 Acked Write 4 Done Sent Done false
30176 9430 Acked Read 7 Done Sent Done false
30177 9431 Acked Read 1 Done Sent Done false
30178 9432 Acked Read 4 Done Sent Done false
30179 9433 Acked Write 6 Done Sent Done false
30180 9434 Acked Read 9 Done Sent Done false
30181 9435 Acked Read 2 Done Sent Done false
30182 9436 Acked Write 4 Done Sent Done false
30183 9437 Acked Write 8 Done Sent Done false
30184 9438 Acked Read 1 Done Sent Done false
30185 9439 Acked Read 4 Done Sent Done false
30186 9440 Acked Read 9 Done Sent Done false
30187 9441 Acked Read 4 Done Sent Done false
30188 9442 Acked Read 8 Done Sent Done false
30189 9443 Acked Write 7 Done Sent Done false
30190 9444 Acked Read 7 Done Sent Done false
30191 9445 Acked Write 10 Done Sent Done false
30192 9446 Acked Write 3 Done Sent Done false
30193 9447 Acked Read 1 Done Sent Done false
30194 9448 Acked Read 4 Done Sent Done false
30195 9449 Acked Write 6 Done Sent Done false
30196 9450 Acked Write 9 Done Sent Done false
30197 9451 Acked Write 9 Done Sent Done false
30198 9452 Acked Read 3 Done Sent Done false
30199 STATES DS:0 DS:1 DS:2 TOTAL
30200 Sent 0 301 0 301
30201 Done 301 0 301 602
30202 Skipped 0 0 0 0
30203 Error 0 0 0 0
30204 Last Flush: 9428 9151 9428
30205 Downstairs last five completed: 9151 9150 9149 9148 9147
30206 Upstairs last five completed: 9452 9451 9450 9449 9448
30207 CLIENT: Up:0 ds:301 act:2
30208 [8820] got start action
30209 [8820] I am going to restart
30210 Starting downstairs at port 8820
30211 Make output file at "/tmp/dsc/downstairs-8820.txt"
30212 [8820][1] reports Starting
30213 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1228)
30214 [8820][1] reports Running
30215 ----------------------------------------------------------------
30216 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30217 States: Active Offline Active
30218 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30219 9152 Acked Flush 0 Done Sent Done false
30220 9153 Acked Read 10 Done Sent Done false
30221 9154 Acked Write 1 Done Sent Done false
30222 9155 Acked Write 10 Done Sent Done false
30223 9156 Acked Write 7 Done Sent Done false
30224 9157 Acked Read 10 Done Sent Done false
30225 9158 Acked Write 10 Done Sent Done false
30226 9159 Acked Read 5 Done Sent Done false
30227 9160 Acked Write 5 Done Sent Done false
30228 9161 Acked Write 5 Done Sent Done false
30229 9162 Acked Write 1 Done Sent Done false
30230 9163 Acked Read 7 Done Sent Done false
30231 9164 Acked Write 5 Done Sent Done false
30232 9165 Acked Read 2 Done Sent Done false
30233 9166 Acked Read 6 Done Sent Done false
30234 9167 Acked Read 8 Done Sent Done false
30235 9168 Acked Write 8 Done Sent Done false
30236 9169 Acked Read 8 Done Sent Done false
30237 9170 Acked Read 2 Done Sent Done false
30238 9171 Acked Read 2 Done Sent Done false
30239 9172 Acked Read 3 Done Sent Done false
30240 9173 Acked Read 2 Done Sent Done false
30241 9174 Acked Read 8 Done Sent Done false
30242 9175 Acked Read 10 Done Sent Done false
30243 9176 Acked Write 7 Done Sent Done false
30244 9177 Acked Read 5 Done Sent Done false
30245 9178 Acked Read 5 Done Sent Done false
30246 9179 Acked Flush 0 Done Sent Done false
30247 9180 Acked Read 7 Done Sent Done false
30248 9181 Acked Read 5 Done Sent Done false
30249 9182 Acked Read 7 Done Sent Done false
30250 9183 Acked Read 5 Done Sent Done false
30251 9184 Acked Write 10 Done Sent Done false
30252 9185 Acked Write 4 Done Sent Done false
30253 9186 Acked Flush 0 Done Sent Done false
30254 9187 Acked Write 3 Done Sent Done false
30255 9188 Acked Read 4 Done Sent Done false
30256 9189 Acked Write 8 Done Sent Done false
30257 9190 Acked Read 8 Done Sent Done false
30258 9191 Acked Read 7 Done Sent Done false
30259 9192 Acked Flush 0 Done Sent Done false
30260 9193 Acked Read 8 Done Sent Done false
30261 9194 Acked Write 5 Done Sent Done false
30262 9195 Acked Read 3 Done Sent Done false
30263 9196 Acked Write 2 Done Sent Done false
30264 9197 Acked Write 5 Done Sent Done false
30265 9198 Acked Read 1 Done Sent Done false
30266 9199 Acked Write 1 Done Sent Done false
30267 9200 Acked Read 5 Done Sent Done false
30268 9201 Acked Write 9 Done Sent Done false
30269 9202 Acked Read 2 Done Sent Done false
30270 9203 Acked Write 1 Done Sent Done false
30271 9204 Acked Write 6 Done Sent Done false
30272 9205 Acked Flush 0 Done Sent Done false
30273 9206 Acked Read 2 Done Sent Done false
30274 9207 Acked Write 3 Done Sent Done false
30275 9208 Acked Write 10 Done Sent Done false
30276 9209 Acked Read 4 Done Sent Done false
30277 9210 Acked Read 9 Done Sent Done false
30278 9211 Acked Write 5 Done Sent Done false
30279 9212 Acked Write 10 Done Sent Done false
30280 9213 Acked Write 4 Done Sent Done false
30281 9214 Acked Write 3 Done Sent Done false
30282 9215 Acked Write 5 Done Sent Done false
30283 9216 Acked Write 7 Done Sent Done false
30284 9217 Acked Read 2 Done Sent Done false
30285 9218 Acked Read 5 Done Sent Done false
30286 9219 Acked Write 3 Done Sent Done false
30287 9220 Acked Write 1 Done Sent Done false
30288 9221 Acked Read 6 Done Sent Done false
30289 9222 Acked Read 6 Done Sent Done false
30290 9223 Acked Read 5 Done Sent Done false
30291 9224 Acked Read 8 Done Sent Done false
30292 9225 Acked Read 10 Done Sent Done false
30293 9226 Acked Write 10 Done Sent Done false
30294 9227 Acked Read 3 Done Sent Done false
30295 9228 Acked Read 2 Done Sent Done false
30296 9229 Acked Write 6 Done Sent Done false
30297 9230 Acked Read 8 Done Sent Done false
30298 9231 Acked Read 5 Done Sent Done false
30299 9232 Acked Read 5 Done Sent Done false
30300 9233 Acked Read 2 Done Sent Done false
30301 9234 Acked Write 3 Done Sent Done false
30302 9235 Acked Read 7 Done Sent Done false
30303 9236 Acked Write 3 Done Sent Done false
30304 9237 Acked Read 6 Done Sent Done false
30305 9238 Acked Read 10 Done Sent Done false
30306 9239 Acked Write 6 Done Sent Done false
30307 9240 Acked Read 4 Done Sent Done false
30308 9241 Acked Write 5 Done Sent Done false
30309 9242 Acked Flush 0 Done Sent Done false
30310 9243 Acked Flush 0 Done Sent Done false
30311 9244 Acked Write 5 Done Sent Done false
30312 9245 Acked Read 3 Done Sent Done false
30313 9246 Acked Write 2 Done Sent Done false
30314 9247 Acked Flush 0 Done Sent Done false
30315 9248 Acked Write 2 Done Sent Done false
30316 9249 Acked Read 3 Done Sent Done false
30317 9250 Acked Read 1 Done Sent Done false
30318 9251 Acked Write 4 Done Sent Done false
30319 9252 Acked Write 3 Done Sent Done false
30320 9253 Acked Flush 0 Done Sent Done false
30321 9254 Acked Flush 0 Done Sent Done false
30322 9255 Acked Read 10 Done Sent Done false
30323 9256 Acked Read 9 Done Sent Done false
30324 9257 Acked Read 2 Done Sent Done false
30325 9258 Acked Write 1 Done Sent Done false
30326 9259 Acked Read 6 Done Sent Done false
30327 9260 Acked Read 1 Done Sent Done false
30328 9261 Acked Write 1 Done Sent Done false
30329 9262 Acked Write 5 Done Sent Done false
30330 9263 Acked Read 2 Done Sent Done false
30331 9264 Acked Read 8 Done Sent Done false
30332 9265 Acked Read 7 Done Sent Done false
30333 9266 Acked Read 4 Done Sent Done false
30334 9267 Acked Flush 0 Done Sent Done false
30335 9268 Acked Read 7 Done Sent Done false
30336 9269 Acked Write 3 Done Sent Done false
30337 9270 Acked Read 1 Done Sent Done false
30338 9271 Acked Write 5 Done Sent Done false
30339 9272 Acked Read 1 Done Sent Done false
30340 9273 Acked Flush 0 Done Sent Done false
30341 9274 Acked Read 8 Done Sent Done false
30342 9275 Acked Write 2 Done Sent Done false
30343 9276 Acked Write 10 Done Sent Done false
30344 9277 Acked Write 2 Done Sent Done false
30345 9278 Acked Write 1 Done Sent Done false
30346 9279 Acked Read 2 Done Sent Done false
30347 9280 Acked Read 4 Done Sent Done false
30348 9281 Acked Read 5 Done Sent Done false
30349 9282 Acked Flush 0 Done Sent Done false
30350 9283 Acked Read 7 Done Sent Done false
30351 9284 Acked Write 6 Done Sent Done false
30352 9285 Acked Read 1 Done Sent Done false
30353 9286 Acked Read 3 Done Sent Done false
30354 9287 Acked Read 7 Done Sent Done false
30355 9288 Acked Flush 0 Done Sent Done false
30356 9289 Acked Read 5 Done Sent Done false
30357 9290 Acked Write 6 Done Sent Done false
30358 9291 Acked Flush 0 Done Sent Done false
30359 9292 Acked Read 9 Done Sent Done false
30360 9293 Acked Write 8 Done Sent Done false
30361 9294 Acked Read 6 Done Sent Done false
30362 9295 Acked Read 6 Done Sent Done false
30363 9296 Acked Read 4 Done Sent Done false
30364 9297 Acked Read 6 Done Sent Done false
30365 9298 Acked Read 10 Done Sent Done false
30366 9299 Acked Read 2 Done Sent Done false
30367 9300 Acked Write 3 Done Sent Done false
30368 9301 Acked Write 5 Done Sent Done false
30369 9302 Acked Write 7 Done Sent Done false
30370 9303 Acked Read 8 Done Sent Done false
30371 9304 Acked Flush 0 Done Sent Done false
30372 9305 Acked Flush 0 Done Sent Done false
30373 9306 Acked Read 6 Done Sent Done false
30374 9307 Acked Write 4 Done Sent Done false
30375 9308 Acked Write 6 Done Sent Done false
30376 9309 Acked Read 3 Done Sent Done false
30377 9310 Acked Write 8 Done Sent Done false
30378 9311 Acked Read 9 Done Sent Done false
30379 9312 Acked Read 4 Done Sent Done false
30380 9313 Acked Read 7 Done Sent Done false
30381 9314 Acked Read 6 Done Sent Done false
30382 9315 Acked Read 10 Done Sent Done false
30383 9316 Acked Read 4 Done Sent Done false
30384 9317 Acked Write 5 Done Sent Done false
30385 9318 Acked Write 1 Done Sent Done false
30386 9319 Acked Read 9 Done Sent Done false
30387 9320 Acked Write 5 Done Sent Done false
30388 9321 Acked Write 7 Done Sent Done false
30389 9322 Acked Write 5 Done Sent Done false
30390 9323 Acked Write 5 Done Sent Done false
30391 9324 Acked Write 6 Done Sent Done false
30392 9325 Acked Flush 0 Done Sent Done false
30393 9326 Acked Write 9 Done Sent Done false
30394 9327 Acked Read 9 Done Sent Done false
30395 9328 Acked Write 1 Done Sent Done false
30396 9329 Acked Read 4 Done Sent Done false
30397 9330 Acked Read 4 Done Sent Done false
30398 9331 Acked Write 2 Done Sent Done false
30399 9332 Acked Write 10 Done Sent Done false
30400 9333 Acked Write 4 Done Sent Done false
30401 9334 Acked Read 9 Done Sent Done false
30402 9335 Acked Write 8 Done Sent Done false
30403 9336 Acked Write 3 Done Sent Done false
30404 9337 Acked Flush 0 Done Sent Done false
30405 9338 Acked Flush 0 Done Sent Done false
30406 9339 Acked Read 7 Done Sent Done false
30407 9340 Acked Read 5 Done Sent Done false
30408 9341 Acked Read 5 Done Sent Done false
30409 9342 Acked Flush 0 Done Sent Done false
30410 9343 Acked Write 4 Done Sent Done false
30411 9344 Acked Write 7 Done Sent Done false
30412 9345 Acked Write 7 Done Sent Done false
30413 9346 Acked Write 7 Done Sent Done false
30414 9347 Acked Write 3 Done Sent Done false
30415 9348 Acked Write 1 Done Sent Done false
30416 9349 Acked Write 8 Done Sent Done false
30417 9350 Acked Read 9 Done Sent Done false
30418 9351 Acked Write 10 Done Sent Done false
30419 9352 Acked Read 6 Done Sent Done false
30420 9353 Acked Write 7 Done Sent Done false
30421 9354 Acked Read 3 Done Sent Done false
30422 9355 Acked Read 7 Done Sent Done false
30423 9356 Acked Read 9 Done Sent Done false
30424 9357 Acked Read 9 Done Sent Done false
30425 9358 Acked Write 5 Done Sent Done false
30426 9359 Acked Read 2 Done Sent Done false
30427 9360 Acked Write 10 Done Sent Done false
30428 9361 Acked Write 6 Done Sent Done false
30429 9362 Acked Read 10 Done Sent Done false
30430 9363 Acked Write 4 Done Sent Done false
30431 9364 Acked Read 8 Done Sent Done false
30432 9365 Acked Write 1 Done Sent Done false
30433 9366 Acked Write 5 Done Sent Done false
30434 9367 Acked Read 4 Done Sent Done false
30435 9368 Acked Write 5 Done Sent Done false
30436 9369 Acked Write 10 Done Sent Done false
30437 9370 Acked Read 10 Done Sent Done false
30438 9371 Acked Write 4 Done Sent Done false
30439 9372 Acked Read 7 Done Sent Done false
30440 9373 Acked Flush 0 Done Sent Done false
30441 9374 Acked Write 1 Done Sent Done false
30442 9375 Acked Write 7 Done Sent Done false
30443 9376 Acked Read 4 Done Sent Done false
30444 9377 Acked Write 8 Done Sent Done false
30445 9378 Acked Read 5 Done Sent Done false
30446 9379 Acked Read 1 Done Sent Done false
30447 9380 Acked Write 4 Done Sent Done false
30448 9381 Acked Read 3 Done Sent Done false
30449 9382 Acked Read 9 Done Sent Done false
30450 9383 Acked Read 6 Done Sent Done false
30451 9384 Acked Write 9 Done Sent Done false
30452 9385 Acked Write 6 Done Sent Done false
30453 9386 Acked Read 1 Done Sent Done false
30454 9387 Acked Read 4 Done Sent Done false
30455 9388 Acked Write 8 Done Sent Done false
30456 9389 Acked Read 5 Done Sent Done false
30457 9390 Acked Write 4 Done Sent Done false
30458 9391 Acked Read 5 Done Sent Done false
30459 9392 Acked Read 7 Done Sent Done false
30460 9393 Acked Write 2 Done Sent Done false
30461 9394 Acked Flush 0 Done Sent Done false
30462 9395 Acked Read 3 Done Sent Done false
30463 9396 Acked Read 6 Done Sent Done false
30464 9397 Acked Read 2 Done Sent Done false
30465 9398 Acked Read 9 Done Sent Done false
30466 9399 Acked Read 3 Done Sent Done false
30467 9400 Acked Read 10 Done Sent Done false
30468 9401 Acked Read 7 Done Sent Done false
30469 9402 Acked Write 9 Done Sent Done false
30470 9403 Acked Write 9 Done Sent Done false
30471 9404 Acked Write 6 Done Sent Done false
30472 9405 Acked Write 6 Done Sent Done false
30473 9406 Acked Read 6 Done Sent Done false
30474 9407 Acked Read 2 Done Sent Done false
30475 9408 Acked Read 5 Done Sent Done false
30476 9409 Acked Read 7 Done Sent Done false
30477 9410 Acked Flush 0 Done Sent Done false
30478 9411 Acked Flush 0 Done Sent Done false
30479 9412 Acked Write 4 Done Sent Done false
30480 9413 Acked Read 9 Done Sent Done false
30481 9414 Acked Write 6 Done Sent Done false
30482 9415 Acked Read 8 Done Sent Done false
30483 9416 Acked Write 9 Done Sent Done false
30484 9417 Acked Read 7 Done Sent Done false
30485 9418 Acked Write 1 Done Sent Done false
30486 9419 Acked Write 2 Done Sent Done false
30487 9420 Acked Read 1 Done Sent Done false
30488 9421 Acked Read 7 Done Sent Done false
30489 9422 Acked Read 4 Done Sent Done false
30490 9423 Acked Read 7 Done Sent Done false
30491 9424 Acked Read 4 Done Sent Done false
30492 9425 Acked Read 9 Done Sent Done false
30493 9426 Acked Write 10 Done Sent Done false
30494 9427 Acked Read 4 Done Sent Done false
30495 9428 Acked Flush 0 Done Sent Done false
30496 9429 Acked Write 4 Done Sent Done false
30497 9430 Acked Read 7 Done Sent Done false
30498 9431 Acked Read 1 Done Sent Done false
30499 9432 Acked Read 4 Done Sent Done false
30500 9433 Acked Write 6 Done Sent Done false
30501 9434 Acked Read 9 Done Sent Done false
30502 9435 Acked Read 2 Done Sent Done false
30503 9436 Acked Write 4 Done Sent Done false
30504 9437 Acked Write 8 Done Sent Done false
30505 9438 Acked Read 1 Done Sent Done false
30506 9439 Acked Read 4 Done Sent Done false
30507 9440 Acked Read 9 Done Sent Done false
30508 9441 Acked Read 4 Done Sent Done false
30509 9442 Acked Read 8 Done Sent Done false
30510 9443 Acked Write 7 Done Sent Done false
30511 9444 Acked Read 7 Done Sent Done false
30512 9445 Acked Write 10 Done Sent Done false
30513 9446 Acked Write 3 Done Sent Done false
30514 9447 Acked Read 1 Done Sent Done false
30515 9448 Acked Read 4 Done Sent Done false
30516 9449 Acked Write 6 Done Sent Done false
30517 9450 Acked Write 9 Done Sent Done false
30518 9451 Acked Write 9 Done Sent Done false
30519 9452 Acked Read 3 Done Sent Done false
30520 9453 Acked Flush 0 Done Sent Done false
30521 STATES DS:0 DS:1 DS:2 TOTAL
30522 Sent 0 302 0 302
30523 Done 302 0 302 604
30524 Skipped 0 0 0 0
30525 Error 0 0 0 0
30526 Last Flush: 9453 9151 9453
30527 Downstairs last five completed: 9151 9150 9149 9148 9147
30528 Upstairs last five completed: 9453 9452 9451 9450 9449
30529 CLIENT: Up:0 ds:302 act:2
305302025-01-21T15:53:34.049ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305312025-01-21T15:53:34.050ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:60746) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305322025-01-21T15:53:34.051ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305332025-01-21T15:53:34.051ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305342025-01-21T15:53:34.051ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305352025-01-21T15:53:34.051ZINFOcrucible: send last flush ID to this DS: 9151 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305362025-01-21T15:53:34.051ZINFOcrucible: Replied this last flush ID: 9151 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305372025-01-21T15:53:34.051ZINFOcrucible: [1] client re-new 302 jobs since flush 9151 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305382025-01-21T15:53:34.052ZINFOcrucible: [1] Replayed 302 jobs since flush: 9151 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
30539 ----------------------------------------------------------------
30540 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30541 Downstairs last five completed: 9453 9452 9451 9450 9449
30542 Upstairs last five completed: 9453 9452 9451 9450 9449
30543 CLIENT: Up:0 ds:0 act:3
30544 Replay: All jobs finished
30545 Main task has work to do, go find it
30546 got dsc Stop(2)
30547 stop 2
30548 Jan 21 15:53:36.299 INFO request completed, latency_us: 382, response_code: 204, uri: /stop/cid/2[8830] Got stop action so:false kr:false
30549 , method: POST, req_id: bd99dffc-fce0-4282-98ef-b5db8594fd58, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
30550 Jan 21 15:53:36.300 INFO request completed, latency_us: 123, response_code: 200, uri: /state/cid/2, method: GET, req_id: 3cfb58f2-34cd-46d6-9c33-ffd1b1fa9ed4, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
305512025-01-21T15:53:36.300ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305522025-01-21T15:53:36.300ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305532025-01-21T15:53:36.300ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
305542025-01-21T15:53:36.300ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
30555 [8830] Exited with: ExitStatus(unix_wait_status(9))
30556 [8830][2] reports Exit
30557 Jan 21 15:53:40.303 INFO request completed, latency_us: 362, response_code: 200, uri: /state/cid/2, method: GET, req_id: 774ca405-31e7-42e1-a52f-5e03bf61a434, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
30558 001/300 Read block 1010 len 12288
30559 002/300 Write block 1652 len 32768 data: 5 6 6 5 6 5 9 8
30560 003/300 Read block 4881 len 12288
30561 004/300 Write block 2769 len 8192 data: 4 4
30562 005/300 Read block 1925 len 32768
30563 006/300 Read block 4139 len 4096
30564 007/300 Read block 2875 len 32768
30565 008/300 Flush
30566 009/300 Write block 223 len 40960 data: 6 10 10 10 8 8 10 9 9 8
30567 010/300 Write block 1006 len 12288 data: 6 5 4
30568 011/300 Read block 2550 len 16384
30569 012/300 Write block 2883 len 16384 data: 7 7 8 10
30570 013/300 Write block 1190 len 20480 data: 7 9 7 7 7
30571 014/300 Read block 2309 len 24576
30572 015/300 Read block 1834 len 28672
30573 016/300 Read block 360 len 16384
30574 017/300 Read block 4289 len 24576
30575 018/300 Flush
30576 019/300 Read block 1228 len 28672
30577 020/300 Read block 4950 len 28672
30578 021/300 Read block 4440 len 28672
30579 022/300 Read block 4576 len 32768
30580 023/300 Read block 4411 len 12288
30581 024/300 Flush
30582 025/300 Read block 1511 len 24576
30583 026/300 Read block 897 len 36864
30584 027/300 Write block 918 len 20480 data: 10 7 8 6 5
30585 028/300 Write block 4053 len 20480 data: 9 8 8 7 8
30586 029/300 Read block 3536 len 36864
30587 030/300 Flush
30588 031/300 Write block 3775 len 16384 data: 5 5 4 3
30589 032/300 Write block 1570 len 8192 data: 6 6
30590 033/300 Flush
30591 034/300 Write block 4552 len 40960 data: 2 3 4 4 6 7 6 4 4 4
30592 035/300 Write block 4707 len 40960 data: 6 5 6 4 5 4 5 5 5 6
30593 036/300 Write block 3818 len 32768 data: 8 6 7 6 8 7 5 5
30594 037/300 Read block 2906 len 16384
30595 038/300 Write block 815 len 20480 data: 7 5 6 8 9
30596 039/300 Read block 3763 len 8192
30597 040/300 Read block 1291 len 12288
30598 041/300 Flush
30599 042/300 Flush
30600 043/300 Write block 2004 len 24576 data: 4 4 4 6 6 6
30601 044/300 Read block 722 len 40960
30602 045/300 Read block 2890 len 8192
30603 046/300 Write block 1754 len 40960 data: 7 6 6 8 9 7 7 8 7 7
30604 047/300 Write block 4038 len 40960 data: 10 7 6 5 3 3 3 5 5 5
30605 048/300 Read block 699 len 8192
30606 049/300 Write block 3435 len 28672 data: 3 3 3 3 3 6 7
30607 050/300 Write block 3481 len 16384 data: 6 7 7 7
30608 051/300 Read block 362 len 4096
30609 052/300 Read block 4320 len 24576
30610 053/300 Write block 4200 len 32768 data: 4 4 4 5 5 7 6 6
30611 054/300 Write block 411 len 20480 data: 8 7 7 7 6
30612 055/300 Read block 2883 len 8192
30613 056/300 Flush
30614 057/300 Read block 2873 len 36864
30615 058/300 Write block 3761 len 12288 data: 5 6 6
30616 059/300 Flush
30617 060/300 Read block 3928 len 24576
30618 061/300 Write block 3047 len 36864 data: 3 4 5 5 6 5 4 5 4
30619 062/300 Write block 2627 len 4096 data: 6
30620 063/300 Write block 1643 len 8192 data: 4 5
30621 064/300 Read block 2860 len 36864
30622 065/300 Write block 4234 len 32768 data: 6 5 6 5 5 5 5 5
30623 066/300 Read block 1801 len 20480
30624 067/300 Write block 1885 len 36864 data: 7 6 5 7 6 3 4 4 4
30625 068/300 Write block 1250 len 16384 data: 5 5 3 4
30626 069/300 Write block 3188 len 12288 data: 3 4 4
30627 070/300 Read block 677 len 32768
30628 071/300 Read block 2434 len 8192
30629 072/300 Read block 2105 len 28672
30630 073/300 Write block 2004 len 8192 data: 5 5
30631 074/300 Write block 3987 len 32768 data: 6 6 3 3 2 3 3 4
30632 075/300 Read block 1738 len 36864
30633 076/300 Write block 672 len 20480 data: 6 5 5 6 6
30634 077/300 Write block 2638 len 20480 data: 5 5 4 5 6
30635 078/300 Write block 4445 len 4096 data: 4
30636 079/300 Write block 3907 len 32768 data: 6 6 5 4 5 5 6 7
30637 080/300 Write block 2483 len 12288 data: 4 3 2
30638 081/300 Write block 2655 len 36864 data: 4 5 6 6 7 7 8 7 6
30639 082/300 Write block 2649 len 8192 data: 7 7
30640 083/300 Write block 3640 len 16384 data: 2 2 2 2
30641 084/300 Read block 4615 len 36864
30642 085/300 Write block 883 len 12288 data: 5 6 5
30643 086/300 Write block 4545 len 4096 data: 10
30644 087/300 Write block 2785 len 36864 data: 7 7 8 6 5 6 7 7 6
30645 088/300 Read block 231 len 20480
30646 089/300 Flush
30647 090/300 Write block 1184 len 20480 data: 5 6 6 7 7
30648 091/300 Read block 2519 len 4096
30649 092/300 Write block 4123 len 16384 data: 6 6 6 6
30650 093/300 Write block 3929 len 4096 data: 7
30651 094/300 Write block 3648 len 24576 data: 5 6 7 6 5 4
30652 095/300 Write block 918 len 20480 data: 11 8 9 7 6
30653 096/300 Write block 583 len 8192 data: 4 4
30654 097/300 Write block 2422 len 28672 data: 9 8 9 9 8 7 7
30655 098/300 Write block 3820 len 12288 data: 8 7 9
30656 099/300 Read block 1221 len 16384
30657 100/300 Read block 4471 len 24576
30658 101/300 Read block 1370 len 32768
30659 102/300 Read block 2017 len 32768
30660 103/300 Write block 1820 len 12288 data: 5 5 5
30661 104/300 Read block 3044 len 4096
30662 105/300 Write block 3350 len 20480 data: 8 8 8 8 7
30663 106/300 Read block 566 len 36864
30664 107/300 Read block 3515 len 8192
30665 108/300 Read block 4804 len 12288
30666 109/300 Write block 2763 len 32768 data: 5 5 5 5 6 5 5 5
30667 110/300 Flush
30668 111/300 Read block 109 len 20480
30669 112/300 Read block 1072 len 12288
30670 113/300 Write block 1077 len 4096 data: 7
30671 114/300 Read block 548 len 20480
30672 115/300 Read block 3677 len 20480
30673 116/300 Flush
30674 117/300 Read block 836 len 40960
30675 118/300 Write block 4301 len 28672 data: 4 3 4 5 5 5 5
30676 119/300 Read block 4081 len 32768
30677 120/300 Read block 907 len 8192
30678 121/300 Write block 3702 len 36864 data: 13 14 13 9 7 8 8 7 6
30679 122/300 Read block 1498 len 28672
30680 123/300 Write block 972 len 8192 data: 5 5
30681 124/300 Read block 3432 len 36864
30682 125/300 Read block 4205 len 40960
30683 126/300 Read block 4739 len 12288
30684 127/300 Write block 2862 len 16384 data: 5 5 3 4
30685 128/300 Read block 3379 len 12288
30686 129/300 Read block 3931 len 20480
30687 130/300 Read block 2205 len 12288
30688 131/300 Flush
30689 132/300 Write block 1880 len 8192 data: 7 6
30690 133/300 Write block 1571 len 12288 data: 7 6 5
30691 134/300 Write block 4313 len 16384 data: 5 5 7 7
30692 135/300 Write block 794 len 8192 data: 5 4
30693 136/300 Read block 3050 len 20480
30694 137/300 Read block 4702 len 40960
30695 138/300 Write block 407 len 16384 data: 11 9 7 8
30696 139/300 Read block 3795 len 32768
30697 140/300 Write block 574 len 32768 data: 6 6 5 5 7 5 6 6
30698 141/300 Write block 3688 len 28672 data: 4 4 4 4 4 3 3
30699 142/300 Read block 3729 len 16384
30700 143/300 Flush
30701 144/300 Read block 4600 len 24576
30702 145/300 Read block 3076 len 12288
30703 146/300 Flush
30704 147/300 Write block 2047 len 40960 data: 8 6 5 5 5 6 6 7 7 7
30705 148/300 Read block 3342 len 32768
30706 149/300 Write block 4791 len 40960 data: 5 4 4 5 7 6 6 7 8 9
30707 150/300 Write block 1108 len 16384 data: 8 9 8 7
30708 151/300 Write block 4934 len 20480 data: 5 5 6 6 5
30709 152/300 Read block 3474 len 8192
30710 153/300 Write block 2639 len 36864 data: 6 5 6 7 6 7 7 8 10
30711 154/300 Read block 3212 len 36864
30712 155/300 Read block 3673 len 20480
30713 156/300 Read block 2875 len 28672
30714 157/300 Write block 1142 len 12288 data: 5 5 8
30715 158/300 Write block 2878 len 28672 data: 7 7 7 7 7 8 8
30716 159/300 Read block 3321 len 24576
30717 160/300 Read block 2894 len 12288
30718 161/300 Read block 28 len 8192
30719 162/300 Read block 2293 len 24576
30720 163/300 Read block 2219 len 12288
30721 164/300 Read block 3782 len 32768
30722 165/300 Write block 4750 len 16384 data: 6 7 6 6
30723 166/300 Read block 1560 len 20480
30724 167/300 Write block 1314 len 40960 data: 8 9 9 8 6 6 7 6 6 8
30725 168/300 Read block 3243 len 24576
30726 169/300 Write block 3032 len 4096 data: 8
30727 170/300 Write block 1491 len 4096 data: 5
30728 171/300 Write block 389 len 8192 data: 4 4
30729 172/300 Read block 3143 len 24576
30730 173/300 Write block 1414 len 16384 data: 4 5 5 4
30731 174/300 Read block 313 len 40960
30732 175/300 Write block 3151 len 36864 data: 8 8 7 7 6 5 4 3 3
30733 176/300 Write block 3095 len 36864 data: 4 4 4 4 4 5 5 5 6
30734 177/300 Read block 3127 len 12288
30735 178/300 Read block 3127 len 16384
30736 179/300 Read block 1792 len 4096
30737 180/300 Read block 3672 len 4096
30738 181/300 Write block 515 len 28672 data: 5 4 5 5 4 4 6
30739 182/300 Read block 938 len 24576
30740 183/300 Read block 3588 len 16384
30741 184/300 Read block 4282 len 36864
30742 185/300 Write block 1650 len 4096 data: 5
30743 186/300 Read block 4417 len 8192
30744 187/300 Read block 479 len 8192
30745 188/300 Flush
30746 189/300 Write block 3603 len 16384 data: 6 3 4 5
30747 190/300 Read block 2102 len 28672
30748 191/300 Read block 2627 len 20480
30749 192/300 Read block 2576 len 16384
30750 193/300 Read block 2316 len 12288
30751 194/300 Read block 706 len 32768
30752 195/300 Write block 1759 len 36864 data: 8 8 9 8 8 8 7 8 8
30753 196/300 Write block 2334 len 20480 data: 8 7 6 6 6
30754 197/300 Read block 1372 len 20480
30755 198/300 Read block 3567 len 12288
30756 199/300 Write block 4396 len 36864 data: 8 10 10 10 10 5 4 4 4
30757 200/300 Read block 4720 len 4096
30758 201/300 Read block 3637 len 32768
30759 202/300 Write block 1975 len 20480 data: 6 4 5 7 8
30760 203/300 Write block 522 len 40960 data: 5 6 6 6 7 9 10 12 11 8
30761 204/300 Write block 1702 len 20480 data: 6 6 6 7 7
30762 205/300 Write block 3584 len 28672 data: 3 4 4 4 3 4 4
30763 206/300 Read block 4752 len 28672
30764 207/300 Read block 4719 len 32768
30765 208/300 Read block 2829 len 16384
30766 209/300 Read block 996 len 32768
30767 210/300 Write block 2299 len 12288 data: 4 5 5
30768 211/300 Read block 1685 len 16384
30769 212/300 Read block 3457 len 24576
30770 213/300 Read block 3296 len 32768
30771 214/300 Read block 3966 len 16384
30772 215/300 Read block 2601 len 28672
30773 216/300 Read block 2397 len 8192
30774 217/300 Flush
30775 218/300 Read block 54 len 32768
30776 219/300 Write block 1938 len 32768 data: 6 7 7 8 7 4 5 5
30777 220/300 Write block 481 len 24576 data: 7 7 8 8 9 8
30778 221/300 Read block 4904 len 24576
30779 222/300 Read block 4587 len 36864
30780 223/300 Read block 4596 len 12288
30781 224/300 Write block 3709 len 36864 data: 8 7 7 7 8 8 7 6 5
30782 225/300 Write block 2698 len 16384 data: 6 6 6 6
30783 226/300 Write block 496 len 20480 data: 3 4 5 6 5
30784 227/300 Read block 927 len 28672
30785 228/300 Write block 784 len 4096 data: 4
30786 229/300 Read block 3769 len 16384
30787 230/300 Flush
30788 231/300 Write block 3778 len 8192 data: 4 3
30789 232/300 Read block 935 len 32768
30790 233/300 Write block 1299 len 20480 data: 7 7 7 7 7
30791 234/300 Write block 2468 len 28672 data: 6 6 6 7 6 4 4
30792 235/300 Write block 2968 len 16384 data: 5 4 4 3
30793 236/300 Read block 2278 len 12288
30794 237/300 Write block 2389 len 36864 data: 6 8 7 9 8 8 8 8 6
30795 238/300 Read block 1899 len 36864
30796 239/300 Read block 4011 len 12288
30797 240/300 Write block 1758 len 24576 data: 10 9 9 10 9 9
30798 241/300 Write block 1414 len 12288 data: 5 6 6
30799 242/300 Write block 304 len 20480 data: 7 6 6 4 4
30800 243/300 Write block 3488 len 20480 data: 6 5 6 7 6
30801 244/300 Read block 1748 len 32768
30802 245/300 Write block 356 len 12288 data: 4 6 7
30803 246/300 Write block 1939 len 40960 data: 8 8 9 8 5 6 6 5 4 5
30804 247/300 Write block 3384 len 12288 data: 6 5 5
30805 248/300 Write block 1489 len 36864 data: 7 6 6 3 4 5 4 5 6
30806 249/300 Read block 1790 len 16384
30807 250/300 Write block 3035 len 20480 data: 6 6 5 7 7
30808 251/300 Read block 1970 len 8192
30809 252/300 Write block 2998 len 24576 data: 3 2 2 2 4 4
30810 253/300 Write block 4930 len 12288 data: 6 6 4
30811 254/300 Read block 1874 len 12288
30812 255/300 Read block 2828 len 4096
30813 256/300 Write block 1091 len 4096 data: 6
30814 257/300 Write block 656 len 8192 data: 4 4
30815 258/300 Read block 1301 len 4096
30816 259/300 Read block 952 len 8192
30817 260/300 Read block 3505 len 20480
30818 261/300 Read block 1038 len 40960
30819 262/300 Read block 4102 len 28672
30820 263/300 Read block 1675 len 36864
30821 264/300 Read block 1070 len 36864
30822 265/300 Write block 2583 len 4096 data: 5
30823 266/300 Write block 1200 len 16384 data: 7 6 5 4
30824 267/300 Write block 2845 len 32768 data: 3 3 3 4 5 6 6 7
30825 268/300 Write block 2174 len 36864 data: 7 5 5 5 6 5 4 4 5
30826 269/300 Read block 4154 len 16384
30827 270/300 Flush
30828 271/300 Write block 4971 len 4096 data: 4
30829 272/300 Read block 1456 len 12288
30830 273/300 Read block 215 len 4096
30831 274/300 Read block 4847 len 20480
30832 275/300 Read block 1151 len 12288
30833 276/300 Flush
30834 277/300 Read block 251 len 8192
30835 278/300 Write block 646 len 16384 data: 6 6 8 9
30836 279/300 Read block 3038 len 24576
30837 280/300 Read block 328 len 24576
30838 281/300 Write block 3002 len 12288 data: 5 5 5
30839 282/300 Read block 2407 len 20480
30840 283/300 Write block 3799 len 4096 data: 6
30841 284/300 Read block 1364 len 36864
30842 285/300 Read block 1780 len 40960
30843 286/300 Flush
30844 287/300 Write block 94 len 40960 data: 5 4 3 3 3 2 4 5 5 6
30845 288/300 Read block 2950 len 36864
30846 289/300 Write block 355 len 32768 data: 4 5 7 8 6 7 8 7
30847 290/300 Write block 4508 len 20480 data: 4 3 3 2 2
30848 291/300 Read block 1109 len 12288
30849 292/300 Read block 602 len 12288
30850 293/300 Write block 3425 len 16384 data: 7 4 4 4
30851 294/300 Read block 213 len 24576
30852 295/300 Write block 1261 len 20480 data: 6 5 5 5 6
30853 296/300 Write block 4953 len 28672 data: 6 7 7 7 7 5 4
30854 297/300 Flush
30855 298/300 Write block 1659 len 12288 data: 9 9 11
30856 299/300 Read block 1553 len 4096
30857 300/300 Flush
30858 301/300 Write block 817 len 8192 data: 7 9
30859 Main task has work to do, go find it
30860 got dsc Start(2)
30861 start 2
30862 Jan 21 15:53:40.537 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/2, method: POST, req_id: 572a95e7-eca2-4135-8185-ded0a68b1c3f, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
30863 Replay: started 2, returned:Ok(())
30864 ----------------------------------------------------------------
30865 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30866 States: Active Active Offline
30867 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30868 9454 Acked Read 3 Done Done Sent false
30869 9455 Acked Write 8 Done Done Sent false
30870 9456 Acked Read 3 Done Done Sent false
30871 9457 Acked Write 2 Done Done Sent false
30872 9458 Acked Read 8 Done Done Sent false
30873 9459 Acked Read 1 Done Done Sent false
30874 9460 Acked Read 8 Done Done Sent false
30875 9461 Acked Flush 0 Done Done Sent false
30876 9462 Acked Write 10 Done Done Sent false
30877 9463 Acked Write 3 Done Done Sent false
30878 9464 Acked Read 4 Done Done Sent false
30879 9465 Acked Write 4 Done Done Sent false
30880 9466 Acked Write 5 Done Done Sent false
30881 9467 Acked Read 6 Done Done Sent false
30882 9468 Acked Read 7 Done Done Sent false
30883 9469 Acked Read 4 Done Done Sent false
30884 9470 Acked Read 6 Done Done Sent false
30885 9471 Acked Flush 0 Done Done Sent false
30886 9472 Acked Read 7 Done Done Sent false
30887 9473 Acked Read 7 Done Done Sent false
30888 9474 Acked Read 7 Done Done Sent false
30889 9475 Acked Read 8 Done Done Sent false
30890 9476 Acked Read 3 Done Done Sent false
30891 9477 Acked Flush 0 Done Done Sent false
30892 9478 Acked Read 6 Done Done Sent false
30893 9479 Acked Read 9 Done Done Sent false
30894 9480 Acked Write 5 Done Done Sent false
30895 9481 Acked Write 5 Done Done Sent false
30896 9482 Acked Read 9 Done Done Sent false
30897 9483 Acked Flush 0 Done Done Sent false
30898 9484 Acked Write 4 Done Done Sent false
30899 9485 Acked Write 2 Done Done Sent false
30900 9486 Acked Flush 0 Done Done Sent false
30901 9487 Acked Write 10 Done Done Sent false
30902 9488 Acked Write 10 Done Done Sent false
30903 9489 Acked Write 8 Done Done Sent false
30904 9490 Acked Read 4 Done Done Sent false
30905 9491 Acked Write 5 Done Done Sent false
30906 9492 Acked Read 2 Done Done Sent false
30907 9493 Acked Read 3 Done Done Sent false
30908 9494 Acked Flush 0 Done Done Sent false
30909 9495 Acked Flush 0 Done Done Sent false
30910 9496 Acked Write 6 Done Done Sent false
30911 9497 Acked Read 10 Done Done Sent false
30912 9498 Acked Read 2 Done Done Sent false
30913 9499 Acked Write 10 Done Done Sent false
30914 9500 Acked Write 10 Done Done Sent false
30915 9501 Acked Read 2 Done Done Sent false
30916 9502 Acked Write 7 Done Done Sent false
30917 9503 Acked Write 4 Done Done Sent false
30918 9504 Acked Read 1 Done Done Sent false
30919 9505 Acked Read 6 Done Done Sent false
30920 9506 Acked Write 8 Done Done Sent false
30921 9507 Acked Write 5 Done Done Sent false
30922 9508 Acked Read 2 Done Done Sent false
30923 9509 Acked Flush 0 Done Done Sent false
30924 9510 Acked Read 9 Done Done Sent false
30925 9511 Acked Write 3 Done Done Sent false
30926 9512 Acked Flush 0 Done Done Sent false
30927 9513 Acked Read 6 Done Done Sent false
30928 9514 Acked Write 9 Done Done Sent false
30929 9515 Acked Write 1 Done Done Sent false
30930 9516 Acked Write 2 Done Done Sent false
30931 9517 Acked Read 9 Done Done Sent false
30932 9518 Acked Write 8 Done Done Sent false
30933 9519 Acked Read 5 Done Done Sent false
30934 9520 Acked Write 9 Done Done Sent false
30935 9521 Acked Write 4 Done Done Sent false
30936 9522 Acked Write 3 Done Done Sent false
30937 9523 Acked Read 8 Done Done Sent false
30938 9524 Acked Read 2 Done Done Sent false
30939 9525 Acked Read 7 Done Done Sent false
30940 9526 Acked Write 2 Done Done Sent false
30941 9527 Acked Write 8 Done Done Sent false
30942 9528 Acked Read 9 Done Done Sent false
30943 9529 Acked Write 5 Done Done Sent false
30944 9530 Acked Write 5 Done Done Sent false
30945 9531 Acked Write 1 Done Done Sent false
30946 9532 Acked Write 8 Done Done Sent false
30947 9533 Acked Write 3 Done Done Sent false
30948 9534 Acked Write 9 Done Done Sent false
30949 9535 Acked Write 2 Done Done Sent false
30950 9536 Acked Write 4 Done Done Sent false
30951 9537 Acked Read 9 Done Done Sent false
30952 9538 Acked Write 3 Done Done Sent false
30953 9539 Acked Write 1 Done Done Sent false
30954 9540 Acked Write 9 Done Done Sent false
30955 9541 Acked Read 5 Done Done Sent false
30956 9542 Acked Flush 0 Done Done Sent false
30957 9543 Acked Write 5 Done Done Sent false
30958 9544 Acked Read 1 Done Done Sent false
30959 9545 Acked Write 4 Done Done Sent false
30960 9546 Acked Write 1 Done Done Sent false
30961 9547 Acked Write 6 Done Done Sent false
30962 9548 Acked Write 5 Done Done Sent false
30963 9549 Acked Write 2 Done Done Sent false
30964 9550 Acked Write 7 Done Done Sent false
30965 9551 Acked Write 3 Done Done Sent false
30966 9552 Acked Read 4 Done Done Sent false
30967 9553 Acked Read 6 Done Done Sent false
30968 9554 Acked Read 8 Done Done Sent false
30969 9555 Acked Read 8 Done Done Sent false
30970 9556 Acked Write 3 Done Done Sent false
30971 9557 Acked Read 1 Done Done Sent false
30972 9558 Acked Write 5 Done Done Sent false
30973 9559 Acked Read 9 Done Done Sent false
30974 9560 Acked Read 2 Done Done Sent false
30975 9561 Acked Read 3 Done Done Sent false
30976 9562 Acked Write 8 Done Done Sent false
30977 9563 Acked Flush 0 Done Done Sent false
30978 9564 Acked Read 5 Done Done Sent false
30979 9565 Acked Read 3 Done Done Sent false
30980 9566 Acked Write 1 Done Done Sent false
30981 9567 Acked Read 5 Done Done Sent false
30982 9568 Acked Read 5 Done Done Sent false
30983 9569 Acked Flush 0 Done Done Sent false
30984 9570 Acked Read 10 Done Done Sent false
30985 9571 Acked Write 7 Done Done Sent false
30986 9572 Acked Read 8 Done Done Sent false
30987 9573 Acked Read 2 Done Done Sent false
30988 9574 Acked Write 9 Done Done Sent false
30989 9575 Acked Read 7 Done Done Sent false
30990 9576 Acked Write 2 Done Done Sent false
30991 9577 Acked Read 9 Done Done Sent false
30992 9578 Acked Read 10 Done Done Sent false
30993 9579 Acked Read 3 Done Done Sent false
30994 9580 Acked Write 4 Done Done Sent false
30995 9581 Acked Read 3 Done Done Sent false
30996 9582 Acked Read 5 Done Done Sent false
30997 9583 Acked Read 3 Done Done Sent false
30998 9584 Acked Flush 0 Done Done Sent false
30999 9585 Acked Write 2 Done Done Sent false
31000 9586 Acked Write 3 Done Done Sent false
31001 9587 Acked Write 4 Done Done Sent false
31002 9588 Acked Write 2 Done Done Sent false
31003 9589 Acked Read 5 Done Done Sent false
31004 9590 Acked Read 10 Done Done Sent false
31005 9591 Acked Write 4 Done Done Sent false
31006 9592 Acked Read 8 Done Done Sent false
31007 9593 Acked Write 8 Done Done Sent false
31008 9594 Acked Write 7 Done Done Sent false
31009 9595 Acked Read 4 Done Done Sent false
31010 9596 Acked Flush 0 Done Done Sent false
31011 9597 Acked Read 6 Done Done Sent false
31012 9598 Acked Read 3 Done Done Sent false
31013 9599 Acked Flush 0 Done Done Sent false
31014 9600 Acked Write 10 Done Done Sent false
31015 9601 Acked Read 8 Done Done Sent false
31016 9602 Acked Write 10 Done Done Sent false
31017 9603 Acked Write 4 Done Done Sent false
31018 9604 Acked Write 5 Done Done Sent false
31019 9605 Acked Read 2 Done Done Sent false
31020 9606 Acked Write 9 Done Done Sent false
31021 9607 Acked Read 9 Done Done Sent false
31022 9608 Acked Read 5 Done Done Sent false
31023 9609 Acked Read 7 Done Done Sent false
31024 9610 Acked Write 3 Done Done Sent false
31025 9611 Acked Write 7 Done Done Sent false
31026 9612 Acked Read 6 Done Done Sent false
31027 9613 Acked Read 3 Done Done Sent false
31028 9614 Acked Read 2 Done Done Sent false
31029 9615 Acked Read 6 Done Done Sent false
31030 9616 Acked Read 3 Done Done Sent false
31031 9617 Acked Read 8 Done Done Sent false
31032 9618 Acked Write 4 Done Done Sent false
31033 9619 Acked Read 5 Done Done Sent false
31034 9620 Acked Write 10 Done Done Sent false
31035 9621 Acked Read 6 Done Done Sent false
31036 9622 Acked Write 1 Done Done Sent false
31037 9623 Acked Write 1 Done Done Sent false
31038 9624 Acked Write 2 Done Done Sent false
31039 9625 Acked Read 6 Done Done Sent false
31040 9626 Acked Write 4 Done Done Sent false
31041 9627 Acked Read 10 Done Done Sent false
31042 9628 Acked Write 9 Done Done Sent false
31043 9629 Acked Write 9 Done Done Sent false
31044 9630 Acked Read 3 Done Done Sent false
31045 9631 Acked Read 4 Done Done Sent false
31046 9632 Acked Read 1 Done Done Sent false
31047 9633 Acked Read 1 Done Done Sent false
31048 9634 Acked Write 7 Done Done Sent false
31049 9635 Acked Read 6 Done Done Sent false
31050 9636 Acked Read 4 Done Done Sent false
31051 9637 Acked Read 9 Done Done Sent false
31052 9638 Acked Write 1 Done Done Sent false
31053 9639 Acked Read 2 Done Done Sent false
31054 9640 Acked Read 2 Done Done Sent false
31055 9641 Acked Flush 0 Done Done Sent false
31056 9642 Acked Write 4 Done Done Sent false
31057 9643 Acked Read 7 Done Done Sent false
31058 9644 Acked Read 5 Done Done Sent false
31059 9645 Acked Read 4 Done Done Sent false
31060 9646 Acked Read 3 Done Done Sent false
31061 9647 Acked Read 8 Done Done Sent false
31062 9648 Acked Write 9 Done Done Sent false
31063 9649 Acked Write 5 Done Done Sent false
31064 9650 Acked Read 5 Done Done Sent false
31065 9651 Acked Read 3 Done Done Sent false
31066 9652 Acked Write 9 Done Done Sent false
31067 9653 Acked Read 1 Done Done Sent false
31068 9654 Acked Read 8 Done Done Sent false
31069 9655 Acked Write 5 Done Done Sent false
31070 9656 Acked Write 10 Done Done Sent false
31071 9657 Acked Write 5 Done Done Sent false
31072 9658 Acked Write 7 Done Done Sent false
31073 9659 Acked Read 7 Done Done Sent false
31074 9660 Acked Read 8 Done Done Sent false
31075 9661 Acked Read 4 Done Done Sent false
31076 9662 Acked Read 8 Done Done Sent false
31077 9663 Acked Write 3 Done Done Sent false
31078 9664 Acked Read 4 Done Done Sent false
31079 9665 Acked Read 6 Done Done Sent false
31080 9666 Acked Read 8 Done Done Sent false
31081 9667 Acked Read 4 Done Done Sent false
31082 9668 Acked Read 7 Done Done Sent false
31083 9669 Acked Read 2 Done Done Sent false
31084 9670 Acked Flush 0 Done Done Sent false
31085 9671 Acked Read 8 Done Done Sent false
31086 9672 Acked Write 8 Done Done Sent false
31087 9673 Acked Write 6 Done Done Sent false
31088 9674 Acked Read 6 Done Done Sent false
31089 9675 Acked Read 9 Done Done Sent false
31090 9676 Acked Read 3 Done Done Sent false
31091 9677 Acked Write 9 Done Done Sent false
31092 9678 Acked Write 4 Done Done Sent false
31093 9679 Acked Write 5 Done Done Sent false
31094 9680 Acked Read 7 Done Done Sent false
31095 9681 Acked Write 1 Done Done Sent false
31096 9682 Acked Read 4 Done Done Sent false
31097 9683 Acked Flush 0 Done Done Sent false
31098 9684 Acked Write 2 Done Done Sent false
31099 9685 Acked Read 8 Done Done Sent false
31100 9686 Acked Write 5 Done Done Sent false
31101 9687 Acked Write 7 Done Done Sent false
31102 9688 Acked Write 4 Done Done Sent false
31103 9689 Acked Read 3 Done Done Sent false
31104 9690 Acked Write 9 Done Done Sent false
31105 9691 Acked Read 9 Done Done Sent false
31106 9692 Acked Read 3 Done Done Sent false
31107 9693 Acked Write 6 Done Done Sent false
31108 9694 Acked Write 3 Done Done Sent false
31109 9695 Acked Write 5 Done Done Sent false
31110 9696 Acked Write 5 Done Done Sent false
31111 9697 Acked Read 8 Done Done Sent false
31112 9698 Acked Write 3 Done Done Sent false
31113 9699 Acked Write 10 Done Done Sent false
31114 9700 Acked Write 3 Done Done Sent false
31115 9701 Acked Write 9 Done Done Sent false
31116 9702 Acked Read 4 Done Done Sent false
31117 9703 Acked Write 5 Done Done Sent false
31118 9704 Acked Read 2 Done Done Sent false
31119 9705 Acked Write 6 Done Done Sent false
31120 9706 Acked Write 3 Done Done Sent false
31121 9707 Acked Read 3 Done Done Sent false
31122 9708 Acked Read 1 Done Done Sent false
31123 9709 Acked Write 1 Done Done Sent false
31124 9710 Acked Write 2 Done Done Sent false
31125 9711 Acked Read 1 Done Done Sent false
31126 9712 Acked Read 2 Done Done Sent false
31127 9713 Acked Read 5 Done Done Sent false
31128 9714 Acked Read 10 Done Done Sent false
31129 9715 Acked Read 7 Done Done Sent false
31130 9716 Acked Read 9 Done Done Sent false
31131 9717 Acked Read 9 Done Done Sent false
31132 9718 Acked Write 1 Done Done Sent false
31133 9719 Acked Write 4 Done Done Sent false
31134 9720 Acked Write 8 Done Done Sent false
31135 9721 Acked Write 9 Done Done Sent false
31136 9722 Acked Read 4 Done Done Sent false
31137 9723 Acked Flush 0 Done Done Sent false
31138 9724 Acked Write 1 Done Done Sent false
31139 9725 Acked Read 3 Done Done Sent false
31140 9726 Acked Read 1 Done Done Sent false
31141 9727 Acked Read 5 Done Done Sent false
31142 9728 Acked Read 3 Done Done Sent false
31143 9729 Acked Flush 0 Done Done Sent false
31144 9730 Acked Read 2 Done Done Sent false
31145 9731 Acked Write 4 Done Done Sent false
31146 9732 Acked Read 6 Done Done Sent false
31147 9733 Acked Read 6 Done Done Sent false
31148 9734 Acked Write 3 Done Done Sent false
31149 9735 Acked Read 5 Done Done Sent false
31150 9736 Acked Write 1 Done Done Sent false
31151 9737 Acked Read 9 Done Done Sent false
31152 9738 Acked Read 10 Done Done Sent false
31153 9739 Acked Flush 0 Done Done Sent false
31154 9740 Acked Write 10 Done Done Sent false
31155 9741 Acked Read 9 Done Done Sent false
31156 9742 Acked Write 8 Done Done Sent false
31157 9743 Acked Write 5 Done Done Sent false
31158 9744 Acked Read 3 Done Done Sent false
31159 9745 Acked Read 3 Done Done Sent false
31160 9746 Acked Write 4 Done Done Sent false
31161 9747 Acked Read 6 Done Done Sent false
31162 9748 Acked Write 5 Done Done Sent false
31163 9749 Acked Write 7 Done Done Sent false
31164 9750 Acked Flush 0 Done Done Sent false
31165 9751 Acked Write 3 Done Done Sent false
31166 9752 Acked Read 1 Done Done Sent false
31167 9753 Acked Flush 0 Done Done Sent false
31168 9754 Acked Write 2 Done Done Sent false
31169 STATES DS:0 DS:1 DS:2 TOTAL
31170 Sent 0 0 301 301
31171 Done 301 301 0 602
31172 Skipped 0 0 0 0
31173 Error 0 0 0 0
31174 Last Flush: 9753 9753 9453
31175 Downstairs last five completed: 9453 9452 9451 9450 9449
31176 Upstairs last five completed: 9754 9753 9752 9751 9750
31177 CLIENT: Up:0 ds:301 act:2
31178 [8830] got start action
31179 [8830] I am going to restart
31180 Starting downstairs at port 8830
31181 Make output file at "/tmp/dsc/downstairs-8830.txt"
31182 [8830][2] reports Starting
31183 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1229)
31184 [8830][2] reports Running
31185 ----------------------------------------------------------------
31186 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31187 States: Active Active Offline
31188 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31189 9454 Acked Read 3 Done Done Sent false
31190 9455 Acked Write 8 Done Done Sent false
31191 9456 Acked Read 3 Done Done Sent false
31192 9457 Acked Write 2 Done Done Sent false
31193 9458 Acked Read 8 Done Done Sent false
31194 9459 Acked Read 1 Done Done Sent false
31195 9460 Acked Read 8 Done Done Sent false
31196 9461 Acked Flush 0 Done Done Sent false
31197 9462 Acked Write 10 Done Done Sent false
31198 9463 Acked Write 3 Done Done Sent false
31199 9464 Acked Read 4 Done Done Sent false
31200 9465 Acked Write 4 Done Done Sent false
31201 9466 Acked Write 5 Done Done Sent false
31202 9467 Acked Read 6 Done Done Sent false
31203 9468 Acked Read 7 Done Done Sent false
31204 9469 Acked Read 4 Done Done Sent false
31205 9470 Acked Read 6 Done Done Sent false
31206 9471 Acked Flush 0 Done Done Sent false
31207 9472 Acked Read 7 Done Done Sent false
31208 9473 Acked Read 7 Done Done Sent false
31209 9474 Acked Read 7 Done Done Sent false
31210 9475 Acked Read 8 Done Done Sent false
31211 9476 Acked Read 3 Done Done Sent false
31212 9477 Acked Flush 0 Done Done Sent false
31213 9478 Acked Read 6 Done Done Sent false
31214 9479 Acked Read 9 Done Done Sent false
31215 9480 Acked Write 5 Done Done Sent false
31216 9481 Acked Write 5 Done Done Sent false
31217 9482 Acked Read 9 Done Done Sent false
31218 9483 Acked Flush 0 Done Done Sent false
31219 9484 Acked Write 4 Done Done Sent false
31220 9485 Acked Write 2 Done Done Sent false
31221 9486 Acked Flush 0 Done Done Sent false
31222 9487 Acked Write 10 Done Done Sent false
31223 9488 Acked Write 10 Done Done Sent false
31224 9489 Acked Write 8 Done Done Sent false
31225 9490 Acked Read 4 Done Done Sent false
31226 9491 Acked Write 5 Done Done Sent false
31227 9492 Acked Read 2 Done Done Sent false
31228 9493 Acked Read 3 Done Done Sent false
31229 9494 Acked Flush 0 Done Done Sent false
31230 9495 Acked Flush 0 Done Done Sent false
31231 9496 Acked Write 6 Done Done Sent false
31232 9497 Acked Read 10 Done Done Sent false
31233 9498 Acked Read 2 Done Done Sent false
31234 9499 Acked Write 10 Done Done Sent false
31235 9500 Acked Write 10 Done Done Sent false
31236 9501 Acked Read 2 Done Done Sent false
31237 9502 Acked Write 7 Done Done Sent false
31238 9503 Acked Write 4 Done Done Sent false
31239 9504 Acked Read 1 Done Done Sent false
31240 9505 Acked Read 6 Done Done Sent false
31241 9506 Acked Write 8 Done Done Sent false
31242 9507 Acked Write 5 Done Done Sent false
31243 9508 Acked Read 2 Done Done Sent false
31244 9509 Acked Flush 0 Done Done Sent false
31245 9510 Acked Read 9 Done Done Sent false
31246 9511 Acked Write 3 Done Done Sent false
31247 9512 Acked Flush 0 Done Done Sent false
31248 9513 Acked Read 6 Done Done Sent false
31249 9514 Acked Write 9 Done Done Sent false
31250 9515 Acked Write 1 Done Done Sent false
31251 9516 Acked Write 2 Done Done Sent false
31252 9517 Acked Read 9 Done Done Sent false
31253 9518 Acked Write 8 Done Done Sent false
31254 9519 Acked Read 5 Done Done Sent false
31255 9520 Acked Write 9 Done Done Sent false
31256 9521 Acked Write 4 Done Done Sent false
31257 9522 Acked Write 3 Done Done Sent false
31258 9523 Acked Read 8 Done Done Sent false
31259 9524 Acked Read 2 Done Done Sent false
31260 9525 Acked Read 7 Done Done Sent false
31261 9526 Acked Write 2 Done Done Sent false
31262 9527 Acked Write 8 Done Done Sent false
31263 9528 Acked Read 9 Done Done Sent false
31264 9529 Acked Write 5 Done Done Sent false
31265 9530 Acked Write 5 Done Done Sent false
31266 9531 Acked Write 1 Done Done Sent false
31267 9532 Acked Write 8 Done Done Sent false
31268 9533 Acked Write 3 Done Done Sent false
31269 9534 Acked Write 9 Done Done Sent false
31270 9535 Acked Write 2 Done Done Sent false
31271 9536 Acked Write 4 Done Done Sent false
31272 9537 Acked Read 9 Done Done Sent false
31273 9538 Acked Write 3 Done Done Sent false
31274 9539 Acked Write 1 Done Done Sent false
31275 9540 Acked Write 9 Done Done Sent false
31276 9541 Acked Read 5 Done Done Sent false
31277 9542 Acked Flush 0 Done Done Sent false
31278 9543 Acked Write 5 Done Done Sent false
31279 9544 Acked Read 1 Done Done Sent false
31280 9545 Acked Write 4 Done Done Sent false
31281 9546 Acked Write 1 Done Done Sent false
31282 9547 Acked Write 6 Done Done Sent false
31283 9548 Acked Write 5 Done Done Sent false
31284 9549 Acked Write 2 Done Done Sent false
31285 9550 Acked Write 7 Done Done Sent false
31286 9551 Acked Write 3 Done Done Sent false
31287 9552 Acked Read 4 Done Done Sent false
31288 9553 Acked Read 6 Done Done Sent false
31289 9554 Acked Read 8 Done Done Sent false
31290 9555 Acked Read 8 Done Done Sent false
31291 9556 Acked Write 3 Done Done Sent false
31292 9557 Acked Read 1 Done Done Sent false
31293 9558 Acked Write 5 Done Done Sent false
31294 9559 Acked Read 9 Done Done Sent false
31295 9560 Acked Read 2 Done Done Sent false
31296 9561 Acked Read 3 Done Done Sent false
31297 9562 Acked Write 8 Done Done Sent false
31298 9563 Acked Flush 0 Done Done Sent false
31299 9564 Acked Read 5 Done Done Sent false
31300 9565 Acked Read 3 Done Done Sent false
31301 9566 Acked Write 1 Done Done Sent false
31302 9567 Acked Read 5 Done Done Sent false
31303 9568 Acked Read 5 Done Done Sent false
31304 9569 Acked Flush 0 Done Done Sent false
31305 9570 Acked Read 10 Done Done Sent false
31306 9571 Acked Write 7 Done Done Sent false
31307 9572 Acked Read 8 Done Done Sent false
31308 9573 Acked Read 2 Done Done Sent false
31309 9574 Acked Write 9 Done Done Sent false
31310 9575 Acked Read 7 Done Done Sent false
31311 9576 Acked Write 2 Done Done Sent false
31312 9577 Acked Read 9 Done Done Sent false
31313 9578 Acked Read 10 Done Done Sent false
31314 9579 Acked Read 3 Done Done Sent false
31315 9580 Acked Write 4 Done Done Sent false
31316 9581 Acked Read 3 Done Done Sent false
31317 9582 Acked Read 5 Done Done Sent false
31318 9583 Acked Read 3 Done Done Sent false
31319 9584 Acked Flush 0 Done Done Sent false
31320 9585 Acked Write 2 Done Done Sent false
31321 9586 Acked Write 3 Done Done Sent false
31322 9587 Acked Write 4 Done Done Sent false
31323 9588 Acked Write 2 Done Done Sent false
31324 9589 Acked Read 5 Done Done Sent false
31325 9590 Acked Read 10 Done Done Sent false
31326 9591 Acked Write 4 Done Done Sent false
31327 9592 Acked Read 8 Done Done Sent false
31328 9593 Acked Write 8 Done Done Sent false
31329 9594 Acked Write 7 Done Done Sent false
31330 9595 Acked Read 4 Done Done Sent false
31331 9596 Acked Flush 0 Done Done Sent false
31332 9597 Acked Read 6 Done Done Sent false
31333 9598 Acked Read 3 Done Done Sent false
31334 9599 Acked Flush 0 Done Done Sent false
31335 9600 Acked Write 10 Done Done Sent false
31336 9601 Acked Read 8 Done Done Sent false
31337 9602 Acked Write 10 Done Done Sent false
31338 9603 Acked Write 4 Done Done Sent false
31339 9604 Acked Write 5 Done Done Sent false
31340 9605 Acked Read 2 Done Done Sent false
31341 9606 Acked Write 9 Done Done Sent false
31342 9607 Acked Read 9 Done Done Sent false
31343 9608 Acked Read 5 Done Done Sent false
31344 9609 Acked Read 7 Done Done Sent false
31345 9610 Acked Write 3 Done Done Sent false
31346 9611 Acked Write 7 Done Done Sent false
31347 9612 Acked Read 6 Done Done Sent false
31348 9613 Acked Read 3 Done Done Sent false
31349 9614 Acked Read 2 Done Done Sent false
31350 9615 Acked Read 6 Done Done Sent false
31351 9616 Acked Read 3 Done Done Sent false
31352 9617 Acked Read 8 Done Done Sent false
31353 9618 Acked Write 4 Done Done Sent false
31354 9619 Acked Read 5 Done Done Sent false
31355 9620 Acked Write 10 Done Done Sent false
31356 9621 Acked Read 6 Done Done Sent false
31357 9622 Acked Write 1 Done Done Sent false
31358 9623 Acked Write 1 Done Done Sent false
31359 9624 Acked Write 2 Done Done Sent false
31360 9625 Acked Read 6 Done Done Sent false
31361 9626 Acked Write 4 Done Done Sent false
31362 9627 Acked Read 10 Done Done Sent false
31363 9628 Acked Write 9 Done Done Sent false
31364 9629 Acked Write 9 Done Done Sent false
31365 9630 Acked Read 3 Done Done Sent false
31366 9631 Acked Read 4 Done Done Sent false
31367 9632 Acked Read 1 Done Done Sent false
31368 9633 Acked Read 1 Done Done Sent false
31369 9634 Acked Write 7 Done Done Sent false
31370 9635 Acked Read 6 Done Done Sent false
31371 9636 Acked Read 4 Done Done Sent false
31372 9637 Acked Read 9 Done Done Sent false
31373 9638 Acked Write 1 Done Done Sent false
31374 9639 Acked Read 2 Done Done Sent false
31375 9640 Acked Read 2 Done Done Sent false
31376 9641 Acked Flush 0 Done Done Sent false
31377 9642 Acked Write 4 Done Done Sent false
31378 9643 Acked Read 7 Done Done Sent false
31379 9644 Acked Read 5 Done Done Sent false
31380 9645 Acked Read 4 Done Done Sent false
31381 9646 Acked Read 3 Done Done Sent false
31382 9647 Acked Read 8 Done Done Sent false
31383 9648 Acked Write 9 Done Done Sent false
31384 9649 Acked Write 5 Done Done Sent false
31385 9650 Acked Read 5 Done Done Sent false
31386 9651 Acked Read 3 Done Done Sent false
31387 9652 Acked Write 9 Done Done Sent false
31388 9653 Acked Read 1 Done Done Sent false
31389 9654 Acked Read 8 Done Done Sent false
31390 9655 Acked Write 5 Done Done Sent false
31391 9656 Acked Write 10 Done Done Sent false
31392 9657 Acked Write 5 Done Done Sent false
31393 9658 Acked Write 7 Done Done Sent false
31394 9659 Acked Read 7 Done Done Sent false
31395 9660 Acked Read 8 Done Done Sent false
31396 9661 Acked Read 4 Done Done Sent false
31397 9662 Acked Read 8 Done Done Sent false
31398 9663 Acked Write 3 Done Done Sent false
31399 9664 Acked Read 4 Done Done Sent false
31400 9665 Acked Read 6 Done Done Sent false
31401 9666 Acked Read 8 Done Done Sent false
31402 9667 Acked Read 4 Done Done Sent false
31403 9668 Acked Read 7 Done Done Sent false
31404 9669 Acked Read 2 Done Done Sent false
31405 9670 Acked Flush 0 Done Done Sent false
31406 9671 Acked Read 8 Done Done Sent false
31407 9672 Acked Write 8 Done Done Sent false
31408 9673 Acked Write 6 Done Done Sent false
31409 9674 Acked Read 6 Done Done Sent false
31410 9675 Acked Read 9 Done Done Sent false
31411 9676 Acked Read 3 Done Done Sent false
31412 9677 Acked Write 9 Done Done Sent false
31413 9678 Acked Write 4 Done Done Sent false
31414 9679 Acked Write 5 Done Done Sent false
31415 9680 Acked Read 7 Done Done Sent false
31416 9681 Acked Write 1 Done Done Sent false
31417 9682 Acked Read 4 Done Done Sent false
31418 9683 Acked Flush 0 Done Done Sent false
31419 9684 Acked Write 2 Done Done Sent false
31420 9685 Acked Read 8 Done Done Sent false
31421 9686 Acked Write 5 Done Done Sent false
31422 9687 Acked Write 7 Done Done Sent false
31423 9688 Acked Write 4 Done Done Sent false
31424 9689 Acked Read 3 Done Done Sent false
31425 9690 Acked Write 9 Done Done Sent false
31426 9691 Acked Read 9 Done Done Sent false
31427 9692 Acked Read 3 Done Done Sent false
31428 9693 Acked Write 6 Done Done Sent false
31429 9694 Acked Write 3 Done Done Sent false
31430 9695 Acked Write 5 Done Done Sent false
31431 9696 Acked Write 5 Done Done Sent false
31432 9697 Acked Read 8 Done Done Sent false
31433 9698 Acked Write 3 Done Done Sent false
31434 9699 Acked Write 10 Done Done Sent false
31435 9700 Acked Write 3 Done Done Sent false
31436 9701 Acked Write 9 Done Done Sent false
31437 9702 Acked Read 4 Done Done Sent false
31438 9703 Acked Write 5 Done Done Sent false
31439 9704 Acked Read 2 Done Done Sent false
31440 9705 Acked Write 6 Done Done Sent false
31441 9706 Acked Write 3 Done Done Sent false
31442 9707 Acked Read 3 Done Done Sent false
31443 9708 Acked Read 1 Done Done Sent false
31444 9709 Acked Write 1 Done Done Sent false
31445 9710 Acked Write 2 Done Done Sent false
31446 9711 Acked Read 1 Done Done Sent false
31447 9712 Acked Read 2 Done Done Sent false
31448 9713 Acked Read 5 Done Done Sent false
31449 9714 Acked Read 10 Done Done Sent false
31450 9715 Acked Read 7 Done Done Sent false
31451 9716 Acked Read 9 Done Done Sent false
31452 9717 Acked Read 9 Done Done Sent false
31453 9718 Acked Write 1 Done Done Sent false
31454 9719 Acked Write 4 Done Done Sent false
31455 9720 Acked Write 8 Done Done Sent false
31456 9721 Acked Write 9 Done Done Sent false
31457 9722 Acked Read 4 Done Done Sent false
31458 9723 Acked Flush 0 Done Done Sent false
31459 9724 Acked Write 1 Done Done Sent false
31460 9725 Acked Read 3 Done Done Sent false
31461 9726 Acked Read 1 Done Done Sent false
31462 9727 Acked Read 5 Done Done Sent false
31463 9728 Acked Read 3 Done Done Sent false
31464 9729 Acked Flush 0 Done Done Sent false
31465 9730 Acked Read 2 Done Done Sent false
31466 9731 Acked Write 4 Done Done Sent false
31467 9732 Acked Read 6 Done Done Sent false
31468 9733 Acked Read 6 Done Done Sent false
31469 9734 Acked Write 3 Done Done Sent false
31470 9735 Acked Read 5 Done Done Sent false
31471 9736 Acked Write 1 Done Done Sent false
31472 9737 Acked Read 9 Done Done Sent false
31473 9738 Acked Read 10 Done Done Sent false
31474 9739 Acked Flush 0 Done Done Sent false
31475 9740 Acked Write 10 Done Done Sent false
31476 9741 Acked Read 9 Done Done Sent false
31477 9742 Acked Write 8 Done Done Sent false
31478 9743 Acked Write 5 Done Done Sent false
31479 9744 Acked Read 3 Done Done Sent false
31480 9745 Acked Read 3 Done Done Sent false
31481 9746 Acked Write 4 Done Done Sent false
31482 9747 Acked Read 6 Done Done Sent false
31483 9748 Acked Write 5 Done Done Sent false
31484 9749 Acked Write 7 Done Done Sent false
31485 9750 Acked Flush 0 Done Done Sent false
31486 9751 Acked Write 3 Done Done Sent false
31487 9752 Acked Read 1 Done Done Sent false
31488 9753 Acked Flush 0 Done Done Sent false
31489 9754 Acked Write 2 Done Done Sent false
31490 9755 Acked Flush 0 Done Done Sent false
31491 STATES DS:0 DS:1 DS:2 TOTAL
31492 Sent 0 0 302 302
31493 Done 302 302 0 604
31494 Skipped 0 0 0 0
31495 Error 0 0 0 0
31496 Last Flush: 9755 9755 9453
31497 Downstairs last five completed: 9453 9452 9451 9450 9449
31498 Upstairs last five completed: 9755 9754 9753 9752 9751
31499 CLIENT: Up:0 ds:302 act:2
315002025-01-21T15:53:46.302ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315012025-01-21T15:53:46.302ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41986) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315022025-01-21T15:53:46.303ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315032025-01-21T15:53:46.304ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315042025-01-21T15:53:46.304ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315052025-01-21T15:53:46.304ZINFOcrucible: send last flush ID to this DS: 9453 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315062025-01-21T15:53:46.304ZINFOcrucible: Replied this last flush ID: 9453 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315072025-01-21T15:53:46.304ZINFOcrucible: [2] client re-new 302 jobs since flush 9453 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315082025-01-21T15:53:46.305ZINFOcrucible: [2] Replayed 302 jobs since flush: 9453 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
31509 ----------------------------------------------------------------
31510 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31511 Downstairs last five completed: 9755 9754 9753 9752 9751
31512 Upstairs last five completed: 9755 9754 9753 9752 9751
31513 CLIENT: Up:0 ds:0 act:3
31514 Replay: All jobs finished
31515 Main task has work to do, go find it
31516 got dsc Stop(1)
31517 stop 1
31518 [8820] Got stop action so:false kr:false
31519 Jan 21 15:53:48.542 INFO request completed, latency_us: 367, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 29cb01d7-ced0-4065-a7fe-b69b965b6025, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
315202025-01-21T15:53:48.542ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315212025-01-21T15:53:48.542ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
315222025-01-21T15:53:48.542ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
31523 Jan 21 15:53:48.542 INFO request completed, latency_us: 135, response_code: 200, uri: /state/cid/1, method: GET, req_id: 7094d779-1a54-4d0d-aa2c-789e51b9a405, remote_addr: 127.0.{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-21T15:53:48.542949153Z","hostname":"w-01JJ4RN2GRXGQKDBHMN9H4FG7S","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"5d7111f5-ffb3-4e9c-baab-0950e41aedfd"}
31524 0.1:52945, local_addr: 127.0.0.1:9998
31525 [8820] Exited with: ExitStatus(unix_wait_status(9))
31526 [8820][1] reports Exit
31527 Jan 21 15:53:52.545 INFO request completed, latency_us: 325, response_code: 200, uri: /state/cid/1, method: GET, req_id: dfb9861d-9d67-425f-bc24-7c0f8b232353, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
31528 001/300 Write block 1105 len 40960 data: 10 7 7 9 10 9 8 8 6 5
31529 002/300 Write block 3177 len 24576 data: 8 7 5 5 4 3
31530 003/300 Write block 2645 len 36864 data: 8 9 11 8 8 8 7 6 5
31531 004/300 Read block 4293 len 20480
31532 005/300 Write block 572 len 4096 data: 5
31533 006/300 Write block 4952 len 24576 data: 6 7 8 8 8 8
31534 007/300 Write block 2968 len 4096 data: 6
31535 008/300 Flush
31536 009/300 Read block 210 len 4096
31537 010/300 Read block 2867 len 16384
31538 011/300 Write block 355 len 20480 data: 5 6 8 9 7
31539 012/300 Flush
31540 013/300 Read block 4477 len 32768
31541 014/300 Read block 1338 len 32768
31542 015/300 Read block 2235 len 20480
31543 016/300 Write block 160 len 8192 data: 6 6
31544 017/300 Read block 4811 len 40960
31545 018/300 Write block 1653 len 8192 data: 7 7
31546 019/300 Read block 3675 len 16384
31547 020/300 Write block 1646 len 16384 data: 4 4 5 5
31548 021/300 Read block 836 len 8192
31549 022/300 Read block 4903 len 36864
31550 023/300 Write block 3821 len 40960 data: 8 10 8 6 6 4 5 5 5 5
31551 024/300 Read block 4572 len 28672
31552 025/300 Write block 31 len 24576 data: 7 7 5 3 3 3
31553 026/300 Write block 1379 len 4096 data: 4
31554 027/300 Write block 3441 len 32768 data: 8 7 8 7 7 6 5 6
31555 028/300 Write block 4283 len 16384 data: 4 3 3 3
31556 029/300 Read block 2065 len 16384
31557 030/300 Write block 2809 len 28672 data: 5 5 6 6 7 6 5
31558 031/300 Read block 4546 len 28672
31559 032/300 Read block 4064 len 20480
31560 033/300 Read block 3101 len 32768
31561 034/300 Read block 3425 len 24576
31562 035/300 Read block 861 len 28672
31563 036/300 Read block 959 len 16384
31564 037/300 Read block 2166 len 16384
31565 038/300 Write block 2070 len 24576 data: 7 6 6 5 6 6
31566 039/300 Write block 2168 len 28672 data: 7 9 8 9 10 7 8
31567 040/300 Read block 3038 len 4096
31568 041/300 Flush
31569 042/300 Write block 3747 len 8192 data: 6 7
31570 043/300 Read block 2054 len 20480
31571 044/300 Flush
31572 045/300 Write block 860 len 16384 data: 8 8 6 8
31573 046/300 Write block 3935 len 4096 data: 4
31574 047/300 Read block 741 len 12288
31575 048/300 Read block 4093 len 36864
31576 049/300 Read block 875 len 24576
31577 050/300 Write block 2597 len 12288 data: 5 4 4
31578 051/300 Read block 4094 len 28672
31579 052/300 Read block 1807 len 36864
31580 053/300 Write block 1464 len 28672 data: 4 4 4 3 5 7 7
31581 054/300 Read block 4676 len 40960
31582 055/300 Read block 495 len 24576
31583 056/300 Read block 1011 len 16384
31584 057/300 Write block 2765 len 12288 data: 6 6 7
31585 058/300 Read block 304 len 24576
31586 059/300 Write block 3518 len 16384 data: 5 7 5 5
31587 060/300 Write block 3308 len 20480 data: 4 4 4 4 5
31588 061/300 Write block 2426 len 36864 data: 9 8 8 6 7 7 6 7 7
31589 062/300 Write block 2256 len 28672 data: 7 7 8 7 7 6 7
31590 063/300 Write block 3286 len 4096 data: 5
31591 064/300 Read block 2414 len 16384
31592 065/300 Write block 4550 len 12288 data: 3 2 3
31593 066/300 Flush
31594 067/300 Read block 888 len 32768
31595 068/300 Read block 1051 len 20480
31596 069/300 Read block 3552 len 24576
31597 070/300 Write block 2155 len 8192 data: 3 3
31598 071/300 Read block 167 len 36864
31599 072/300 Read block 3429 len 16384
31600 073/300 Read block 1449 len 20480
31601 074/300 Write block 2162 len 8192 data: 5 5
31602 075/300 Read block 3551 len 36864
31603 076/300 Read block 4134 len 24576
31604 077/300 Write block 1697 len 4096 data: 3
31605 078/300 Write block 1106 len 4096 data: 8
31606 079/300 Flush
31607 080/300 Write block 3517 len 24576 data: 7 6 8 6 6 6
31608 081/300 Write block 833 len 24576 data: 5 4 4 5 6 5
31609 082/300 Write block 46 len 16384 data: 8 9 9 10
31610 083/300 Read block 1237 len 32768
31611 084/300 Read block 1962 len 20480
31612 085/300 Read block 1761 len 20480
31613 086/300 Write block 4928 len 40960 data: 7 6 7 7 5 5 6 6 7 7
31614 087/300 Read block 4637 len 36864
31615 088/300 Write block 4161 len 20480 data: 3 3 2 2 3
31616 089/300 Write block 877 len 8192 data: 2 2
31617 090/300 Read block 2108 len 8192
31618 091/300 Read block 722 len 40960
31619 092/300 Flush
31620 093/300 Write block 2351 len 24576 data: 9 9 9 10 10 9
31621 094/300 Write block 46 len 8192 data: 9 10
31622 095/300 Write block 4245 len 28672 data: 2 3 3 2 2 2 4
31623 096/300 Read block 2111 len 32768
31624 097/300 Write block 2174 len 12288 data: 9 6 6
31625 098/300 Write block 3649 len 8192 data: 7 8
31626 099/300 Read block 1985 len 36864
31627 100/300 Read block 2343 len 40960
31628 101/300 Flush
31629 102/300 Read block 820 len 20480
31630 103/300 Read block 3606 len 28672
31631 104/300 Write block 2292 len 24576 data: 7 7 6 6 2 3
31632 105/300 Write block 820 len 40960 data: 10 10 9 9 11 8 7 5 5 4
31633 106/300 Write block 2294 len 40960 data: 7 7 3 4 4 5 6 6 6 7
31634 107/300 Read block 91 len 28672
31635 108/300 Read block 3128 len 4096
31636 109/300 Read block 3306 len 8192
31637 110/300 Write block 4526 len 36864 data: 5 5 5 5 3 3 4 6 5
31638 111/300 Read block 3500 len 24576
31639 112/300 Write block 32 len 24576 data: 8 6 4 4 4 7
31640 113/300 Read block 287 len 36864
31641 114/300 Read block 1096 len 36864
31642 115/300 Read block 3439 len 28672
31643 116/300 Write block 3207 len 40960 data: 6 5 5 4 4 5 3 5 4 4
31644 117/300 Flush
31645 118/300 Read block 785 len 28672
31646 119/300 Read block 1076 len 12288
31647 120/300 Read block 1185 len 36864
31648 121/300 Write block 4938 len 32768 data: 6 6 7 8 9 6 7 7
31649 122/300 Read block 2705 len 36864
31650 123/300 Read block 4981 len 40960
31651 124/300 Write block 4955 len 8192 data: 9 9
31652 125/300 Read block 4879 len 4096
31653 126/300 Read block 2803 len 8192
31654 127/300 Flush
31655 128/300 Read block 4470 len 16384
31656 129/300 Read block 3221 len 8192
31657 130/300 Flush
31658 131/300 Write block 4190 len 40960 data: 7 6 5 4 2 2 2 2 2 2
31659 132/300 Write block 2329 len 40960 data: 4 4 4 5 6 9 8 7 7 7
31660 133/300 Read block 4980 len 40960
31661 134/300 Read block 1018 len 36864
31662 135/300 Write block 796 len 8192 data: 4 4
31663 136/300 Read block 1520 len 8192
31664 137/300 Read block 2365 len 28672
31665 138/300 Read block 3318 len 40960
31666 139/300 Write block 1472 len 12288 data: 7 5 6
31667 140/300 Flush
31668 141/300 Read block 3879 len 28672
31669 142/300 Write block 4229 len 40960 data: 5 6 7 8 7 7 6 7 6 6
31670 143/300 Read block 4972 len 36864
31671 144/300 Write block 1432 len 40960 data: 5 8 8 5 4 4 5 6 6 5
31672 145/300 Write block 3330 len 36864 data: 4 4 3 3 3 4 4 5 5
31673 146/300 Read block 2740 len 36864
31674 147/300 Write block 3347 len 28672 data: 4 5 6 9 9 9 9
31675 148/300 Read block 560 len 20480
31676 149/300 Write block 4865 len 28672 data: 4 4 3 3 4 5 5
31677 150/300 Write block 4699 len 28672 data: 7 6 6 7 8 8 6
31678 151/300 Write block 1153 len 16384 data: 6 8 5 7
31679 152/300 Flush
31680 153/300 Flush
31681 154/300 Write block 3806 len 32768 data: 3 3 5 6 6 7 9 9
31682 155/300 Write block 2979 len 28672 data: 7 5 5 6 6 7 6
31683 156/300 Flush
31684 157/300 Read block 3868 len 20480
31685 158/300 Write block 3284 len 8192 data: 8 6
31686 159/300 Write block 4095 len 36864 data: 6 6 6 9 10 10 8 7 9
31687 160/300 Flush
31688 161/300 Read block 1276 len 36864
31689 162/300 Write block 1851 len 8192 data: 6 4
31690 163/300 Write block 364 len 40960 data: 3 3 4 6 7 6 7 7 6 7
31691 164/300 Read block 3882 len 12288
31692 165/300 Write block 2098 len 8192 data: 8 5
31693 166/300 Write block 2976 len 20480 data: 5 7 6 8 6
31694 167/300 Read block 3503 len 12288
31695 168/300 Write block 4396 len 12288 data: 9 11 11
31696 169/300 Flush
31697 170/300 Flush
31698 171/300 Read block 4426 len 8192
31699 172/300 Flush
31700 173/300 Read block 199 len 4096
31701 174/300 Read block 556 len 32768
31702 175/300 Read block 1651 len 28672
31703 176/300 Read block 2984 len 36864
31704 177/300 Read block 150 len 8192
31705 178/300 Read block 3470 len 32768
31706 179/300 Write block 2692 len 36864 data: 3 3 2 2 3 5 7 7 7
31707 180/300 Read block 4691 len 32768
31708 181/300 Write block 949 len 8192 data: 4 3
31709 182/300 Write block 3830 len 40960 data: 6 5 7 7 7 8 7 7 8 6
31710 183/300 Write block 3336 len 32768 data: 5 6 6 5 5 5 5 4
31711 184/300 Read block 4787 len 8192
31712 185/300 Write block 585 len 40960 data: 4 3 3 4 5 5 6 7 9 9
31713 186/300 Read block 4642 len 40960
31714 187/300 Read block 4373 len 4096
31715 188/300 Read block 1315 len 20480
31716 189/300 Read block 3910 len 28672
31717 190/300 Read block 26 len 40960
31718 191/300 Read block 1689 len 36864
31719 192/300 Write block 3029 len 36864 data: 8 8 9 9 7 8 7 7 6
31720 193/300 Write block 417 len 40960 data: 5 5 4 2 2 3 3 2 2 3
31721 194/300 Write block 2237 len 16384 data: 8 8 7 7
31722 195/300 Write block 4321 len 36864 data: 5 4 5 5 5 6 5 5 6
31723 196/300 Read block 453 len 32768
31724 197/300 Read block 3072 len 16384
31725 198/300 Read block 768 len 4096
31726 199/300 Read block 4474 len 36864
31727 200/300 Read block 519 len 8192
31728 201/300 Write block 2557 len 16384 data: 7 7 7 8
31729 202/300 Write block 4641 len 16384 data: 4 4 4 4
31730 203/300 Read block 1002 len 16384
31731 204/300 Write block 887 len 20480 data: 6 6 8 9 8
31732 205/300 Read block 2465 len 16384
31733 206/300 Read block 105 len 20480
31734 207/300 Flush
31735 208/300 Read block 4252 len 40960
31736 209/300 Read block 791 len 4096
31737 210/300 Flush
31738 211/300 Write block 3285 len 36864 data: 7 6 3 3 4 5 5 5 6
31739 212/300 Read block 4089 len 8192
31740 213/300 Write block 1863 len 40960 data: 10 10 11 8 9 7 6 7 7 5
31741 214/300 Read block 4147 len 20480
31742 215/300 Read block 2895 len 32768
31743 216/300 Write block 292 len 24576 data: 6 5 5 4 5 5
31744 217/300 Read block 3409 len 36864
31745 218/300 Write block 2193 len 24576 data: 3 3 3 4 5 7
31746 219/300 Write block 4587 len 4096 data: 6
31747 220/300 Read block 3412 len 24576
31748 221/300 Read block 677 len 20480
31749 222/300 Read block 2721 len 8192
31750 223/300 Read block 812 len 36864
31751 224/300 Read block 3175 len 12288
31752 225/300 Read block 4352 len 40960
31753 226/300 Read block 3412 len 8192
31754 227/300 Flush
31755 228/300 Read block 4057 len 4096
31756 229/300 Write block 4724 len 8192 data: 8 6
31757 230/300 Read block 584 len 16384
31758 231/300 Write block 4738 len 20480 data: 3 5 5 5 4
31759 232/300 Read block 4257 len 24576
31760 233/300 Read block 2445 len 24576
31761 234/300 Write block 1058 len 24576 data: 7 5 5 5 6 6
31762 235/300 Flush
31763 236/300 Read block 793 len 4096
31764 237/300 Read block 327 len 4096
31765 238/300 Write block 250 len 16384 data: 5 8 8 7
31766 239/300 Read block 2915 len 20480
31767 240/300 Read block 133 len 40960
31768 241/300 Write block 3604 len 16384 data: 4 5 6 6
31769 242/300 Write block 312 len 36864 data: 4 5 5 7 7 6 7 6 4
31770 243/300 Write block 1239 len 28672 data: 5 6 6 6 7 8 7
31771 244/300 Write block 140 len 24576 data: 6 7 6 6 7 7
31772 245/300 Write block 1061 len 4096 data: 6
31773 246/300 Read block 2275 len 36864
31774 247/300 Write block 906 len 16384 data: 5 3 2 2
31775 248/300 Write block 4259 len 28672 data: 5 5 4 3 3 3 3
31776 249/300 Read block 1332 len 20480
31777 250/300 Read block 2395 len 40960
31778 251/300 Write block 1816 len 20480 data: 6 6 6 5 6
31779 252/300 Write block 2183 len 16384 data: 6 6 6 6
31780 253/300 Write block 4513 len 20480 data: 3 3 5 6 6
31781 254/300 Read block 2436 len 8192
31782 255/300 Read block 2012 len 36864
31783 256/300 Write block 1157 len 36864 data: 8 8 8 8 9 6 5 4 3
31784 257/300 Write block 170 len 16384 data: 6 6 6 6
31785 258/300 Read block 3962 len 20480
31786 259/300 Read block 1476 len 4096
31787 260/300 Flush
31788 261/300 Write block 2760 len 36864 data: 4 4 5 6 6 7 7 8 6
31789 262/300 Read block 2528 len 8192
31790 263/300 Read block 2080 len 16384
31791 264/300 Write block 3342 len 20480 data: 6 5 6 4 4
31792 265/300 Flush
31793 266/300 Read block 1125 len 4096
31794 267/300 Write block 3881 len 12288 data: 10 8 6
31795 268/300 Write block 698 len 36864 data: 8 7 7 8 8 7 7 6 6
31796 269/300 Write block 4189 len 40960 data: 8 8 7 6 5 3 3 3 3 3
31797 270/300 Write block 3668 len 24576 data: 7 8 6 6 7 8
31798 271/300 Read block 2858 len 32768
31799 272/300 Write block 329 len 16384 data: 4 3 2 3
31800 273/300 Flush
31801 274/300 Flush
31802 275/300 Write block 338 len 12288 data: 6 7 8
31803 276/300 Read block 4135 len 24576
31804 277/300 Flush
31805 278/300 Read block 2404 len 4096
31806 279/300 Read block 1275 len 40960
31807 280/300 Write block 1520 len 12288 data: 3 2 2
31808 281/300 Read block 4169 len 28672
31809 282/300 Read block 4899 len 32768
31810 283/300 Read block 3484 len 28672
31811 284/300 Flush
31812 285/300 Read block 600 len 20480
31813 286/300 Write block 3114 len 28672 data: 6 7 6 6 5 5 4
31814 287/300 Write block 3497 len 36864 data: 7 6 8 8 8 8 9 9 8
31815 288/300 Read block 446 len 32768
31816 289/300 Read block 3747 len 32768
31817 290/300 Read block 4124 len 40960
31818 291/300 Flush
31819 292/300 Read block 2422 len 8192
31820 293/300 Read block 4349 len 12288
31821 294/300 Flush
31822 295/300 Write block 3353 len 32768 data: 10 8 9 8 6 5 5 6
31823 296/300 Write block 1603 len 16384 data: 3 2 2 2
31824 297/300 Write block 4378 len 4096 data: 2
31825 298/300 Read block 1241 len 24576
31826 299/300 Read block 3698 len 20480
31827 300/300 Read block 4829 len 36864
31828 301/300 Read block 1210 len 20480
31829 Main task has work to do, go find it
31830 got dsc Start(1)
31831 start 1
31832 Jan 21 15:53:52.790 INFO request completed, latency_us: 206, response_code: 204, uri: /start/cid/1, method: POST, req_id: b9c2f18b-8f7d-4c14-86f1-c079c85cef20, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
31833 Replay: started 1, returned:Ok(())
31834 ----------------------------------------------------------------
31835 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31836 States: Active Offline Active
31837 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31838 9756 Acked Write 10 Done Sent Done false
31839 9757 Acked Write 6 Done Sent Done false
31840 9758 Acked Write 9 Done Sent Done false
31841 9759 Acked Read 5 Done Sent Done false
31842 9760 Acked Write 1 Done Sent Done false
31843 9761 Acked Write 6 Done Sent Done false
31844 9762 Acked Write 1 Done Sent Done false
31845 9763 Acked Flush 0 Done Sent Done false
31846 9764 Acked Read 1 Done Sent Done false
31847 9765 Acked Read 4 Done Sent Done false
31848 9766 Acked Write 5 Done Sent Done false
31849 9767 Acked Flush 0 Done Sent Done false
31850 9768 Acked Read 8 Done Sent Done false
31851 9769 Acked Read 8 Done Sent Done false
31852 9770 Acked Read 5 Done Sent Done false
31853 9771 Acked Write 2 Done Sent Done false
31854 9772 Acked Read 10 Done Sent Done false
31855 9773 Acked Write 2 Done Sent Done false
31856 9774 Acked Read 4 Done Sent Done false
31857 9775 Acked Write 4 Done Sent Done false
31858 9776 Acked Read 2 Done Sent Done false
31859 9777 Acked Read 9 Done Sent Done false
31860 9778 Acked Write 10 Done Sent Done false
31861 9779 Acked Read 7 Done Sent Done false
31862 9780 Acked Write 6 Done Sent Done false
31863 9781 Acked Write 1 Done Sent Done false
31864 9782 Acked Write 8 Done Sent Done false
31865 9783 Acked Write 4 Done Sent Done false
31866 9784 Acked Read 4 Done Sent Done false
31867 9785 Acked Write 7 Done Sent Done false
31868 9786 Acked Read 7 Done Sent Done false
31869 9787 Acked Read 5 Done Sent Done false
31870 9788 Acked Read 8 Done Sent Done false
31871 9789 Acked Read 6 Done Sent Done false
31872 9790 Acked Read 7 Done Sent Done false
31873 9791 Acked Read 4 Done Sent Done false
31874 9792 Acked Read 4 Done Sent Done false
31875 9793 Acked Write 6 Done Sent Done false
31876 9794 Acked Write 7 Done Sent Done false
31877 9795 Acked Read 1 Done Sent Done false
31878 9796 Acked Flush 0 Done Sent Done false
31879 9797 Acked Write 2 Done Sent Done false
31880 9798 Acked Read 5 Done Sent Done false
31881 9799 Acked Flush 0 Done Sent Done false
31882 9800 Acked Write 4 Done Sent Done false
31883 9801 Acked Write 1 Done Sent Done false
31884 9802 Acked Read 3 Done Sent Done false
31885 9803 Acked Read 9 Done Sent Done false
31886 9804 Acked Read 6 Done Sent Done false
31887 9805 Acked Write 3 Done Sent Done false
31888 9806 Acked Read 7 Done Sent Done false
31889 9807 Acked Read 9 Done Sent Done false
31890 9808 Acked Write 7 Done Sent Done false
31891 9809 Acked Read 10 Done Sent Done false
31892 9810 Acked Read 6 Done Sent Done false
31893 9811 Acked Read 4 Done Sent Done false
31894 9812 Acked Write 3 Done Sent Done false
31895 9813 Acked Read 6 Done Sent Done false
31896 9814 Acked Write 4 Done Sent Done false
31897 9815 Acked Write 5 Done Sent Done false
31898 9816 Acked Write 9 Done Sent Done false
31899 9817 Acked Write 7 Done Sent Done false
31900 9818 Acked Write 1 Done Sent Done false
31901 9819 Acked Read 4 Done Sent Done false
31902 9820 Acked Write 3 Done Sent Done false
31903 9821 Acked Flush 0 Done Sent Done false
31904 9822 Acked Read 8 Done Sent Done false
31905 9823 Acked Read 5 Done Sent Done false
31906 9824 Acked Read 6 Done Sent Done false
31907 9825 Acked Write 2 Done Sent Done false
31908 9826 Acked Read 9 Done Sent Done false
31909 9827 Acked Read 4 Done Sent Done false
31910 9828 Acked Read 5 Done Sent Done false
31911 9829 Acked Write 2 Done Sent Done false
31912 9830 Acked Read 9 Done Sent Done false
31913 9831 Acked Read 6 Done Sent Done false
31914 9832 Acked Write 1 Done Sent Done false
31915 9833 Acked Write 1 Done Sent Done false
31916 9834 Acked Flush 0 Done Sent Done false
31917 9835 Acked Write 6 Done Sent Done false
31918 9836 Acked Write 6 Done Sent Done false
31919 9837 Acked Write 4 Done Sent Done false
31920 9838 Acked Read 8 Done Sent Done false
31921 9839 Acked Read 5 Done Sent Done false
31922 9840 Acked Read 5 Done Sent Done false
31923 9841 Acked Write 10 Done Sent Done false
31924 9842 Acked Read 9 Done Sent Done false
31925 9843 Acked Write 5 Done Sent Done false
31926 9844 Acked Write 2 Done Sent Done false
31927 9845 Acked Read 2 Done Sent Done false
31928 9846 Acked Read 10 Done Sent Done false
31929 9847 Acked Flush 0 Done Sent Done false
31930 9848 Acked Write 6 Done Sent Done false
31931 9849 Acked Write 2 Done Sent Done false
31932 9850 Acked Write 7 Done Sent Done false
31933 9851 Acked Read 8 Done Sent Done false
31934 9852 Acked Write 3 Done Sent Done false
31935 9853 Acked Write 2 Done Sent Done false
31936 9854 Acked Read 9 Done Sent Done false
31937 9855 Acked Read 10 Done Sent Done false
31938 9856 Acked Flush 0 Done Sent Done false
31939 9857 Acked Read 5 Done Sent Done false
31940 9858 Acked Read 7 Done Sent Done false
31941 9859 Acked Write 6 Done Sent Done false
31942 9860 Acked Write 10 Done Sent Done false
31943 9861 Acked Write 10 Done Sent Done false
31944 9862 Acked Read 7 Done Sent Done false
31945 9863 Acked Read 1 Done Sent Done false
31946 9864 Acked Read 2 Done Sent Done false
31947 9865 Acked Write 9 Done Sent Done false
31948 9866 Acked Read 6 Done Sent Done false
31949 9867 Acked Write 6 Done Sent Done false
31950 9868 Acked Read 9 Done Sent Done false
31951 9869 Acked Read 9 Done Sent Done false
31952 9870 Acked Read 7 Done Sent Done false
31953 9871 Acked Write 10 Done Sent Done false
31954 9872 Acked Flush 0 Done Sent Done false
31955 9873 Acked Read 7 Done Sent Done false
31956 9874 Acked Read 3 Done Sent Done false
31957 9875 Acked Read 9 Done Sent Done false
31958 9876 Acked Write 8 Done Sent Done false
31959 9877 Acked Read 9 Done Sent Done false
31960 9878 Acked Read 10 Done Sent Done false
31961 9879 Acked Write 2 Done Sent Done false
31962 9880 Acked Read 1 Done Sent Done false
31963 9881 Acked Read 2 Done Sent Done false
31964 9882 Acked Flush 0 Done Sent Done false
31965 9883 Acked Read 4 Done Sent Done false
31966 9884 Acked Read 2 Done Sent Done false
31967 9885 Acked Flush 0 Done Sent Done false
31968 9886 Acked Write 10 Done Sent Done false
31969 9887 Acked Write 10 Done Sent Done false
31970 9888 Acked Read 10 Done Sent Done false
31971 9889 Acked Read 9 Done Sent Done false
31972 9890 Acked Write 2 Done Sent Done false
31973 9891 Acked Read 2 Done Sent Done false
31974 9892 Acked Read 7 Done Sent Done false
31975 9893 Acked Read 10 Done Sent Done false
31976 9894 Acked Write 3 Done Sent Done false
31977 9895 Acked Flush 0 Done Sent Done false
31978 9896 Acked Read 7 Done Sent Done false
31979 9897 Acked Write 10 Done Sent Done false
31980 9898 Acked Read 9 Done Sent Done false
31981 9899 Acked Write 10 Done Sent Done false
31982 9900 Acked Write 9 Done Sent Done false
31983 9901 Acked Read 9 Done Sent Done false
31984 9902 Acked Write 7 Done Sent Done false
31985 9903 Acked Read 5 Done Sent Done false
31986 9904 Acked Write 7 Done Sent Done false
31987 9905 Acked Write 7 Done Sent Done false
31988 9906 Acked Write 4 Done Sent Done false
31989 9907 Acked Flush 0 Done Sent Done false
31990 9908 Acked Flush 0 Done Sent Done false
31991 9909 Acked Write 8 Done Sent Done false
31992 9910 Acked Write 7 Done Sent Done false
31993 9911 Acked Flush 0 Done Sent Done false
31994 9912 Acked Read 5 Done Sent Done false
31995 9913 Acked Write 2 Done Sent Done false
31996 9914 Acked Write 9 Done Sent Done false
31997 9915 Acked Flush 0 Done Sent Done false
31998 9916 Acked Read 9 Done Sent Done false
31999 9917 Acked Write 2 Done Sent Done false
32000 9918 Acked Write 10 Done Sent Done false
32001 9919 Acked Read 3 Done Sent Done false
32002 9920 Acked Write 2 Done Sent Done false
32003 9921 Acked Write 5 Done Sent Done false
32004 9922 Acked Read 3 Done Sent Done false
32005 9923 Acked Write 3 Done Sent Done false
32006 9924 Acked Flush 0 Done Sent Done false
32007 9925 Acked Flush 0 Done Sent Done false
32008 9926 Acked Read 2 Done Sent Done false
32009 9927 Acked Flush 0 Done Sent Done false
32010 9928 Acked Read 1 Done Sent Done false
32011 9929 Acked Read 8 Done Sent Done false
32012 9930 Acked Read 7 Done Sent Done false
32013 9931 Acked Read 9 Done Sent Done false
32014 9932 Acked Read 2 Done Sent Done false
32015 9933 Acked Read 8 Done Sent Done false
32016 9934 Acked Write 9 Done Sent Done false
32017 9935 Acked Read 8 Done Sent Done false
32018 9936 Acked Write 2 Done Sent Done false
32019 9937 Acked Write 10 Done Sent Done false
32020 9938 Acked Write 8 Done Sent Done false
32021 9939 Acked Read 2 Done Sent Done false
32022 9940 Acked Write 10 Done Sent Done false
32023 9941 Acked Read 10 Done Sent Done false
32024 9942 Acked Read 1 Done Sent Done false
32025 9943 Acked Read 5 Done Sent Done false
32026 9944 Acked Read 7 Done Sent Done false
32027 9945 Acked Read 10 Done Sent Done false
32028 9946 Acked Read 9 Done Sent Done false
32029 9947 Acked Write 9 Done Sent Done false
32030 9948 Acked Write 10 Done Sent Done false
32031 9949 Acked Write 4 Done Sent Done false
32032 9950 Acked Write 9 Done Sent Done false
32033 9951 Acked Read 8 Done Sent Done false
32034 9952 Acked Read 4 Done Sent Done false
32035 9953 Acked Read 1 Done Sent Done false
32036 9954 Acked Read 9 Done Sent Done false
32037 9955 Acked Read 2 Done Sent Done false
32038 9956 Acked Write 4 Done Sent Done false
32039 9957 Acked Write 4 Done Sent Done false
32040 9958 Acked Read 4 Done Sent Done false
32041 9959 Acked Write 5 Done Sent Done false
32042 9960 Acked Read 4 Done Sent Done false
32043 9961 Acked Read 5 Done Sent Done false
32044 9962 Acked Flush 0 Done Sent Done false
32045 9963 Acked Read 10 Done Sent Done false
32046 9964 Acked Read 1 Done Sent Done false
32047 9965 Acked Flush 0 Done Sent Done false
32048 9966 Acked Write 9 Done Sent Done false
32049 9967 Acked Read 2 Done Sent Done false
32050 9968 Acked Write 10 Done Sent Done false
32051 9969 Acked Read 5 Done Sent Done false
32052 9970 Acked Read 8 Done Sent Done false
32053 9971 Acked Write 6 Done Sent Done false
32054 9972 Acked Read 9 Done Sent Done false
32055 9973 Acked Write 6 Done Sent Done false
32056 9974 Acked Write 1 Done Sent Done false
32057 9975 Acked Read 6 Done Sent Done false
32058 9976 Acked Read 5 Done Sent Done false
32059 9977 Acked Read 2 Done Sent Done false
32060 9978 Acked Read 9 Done Sent Done false
32061 9979 Acked Read 3 Done Sent Done false
32062 9980 Acked Read 10 Done Sent Done false
32063 9981 Acked Read 2 Done Sent Done false
32064 9982 Acked Flush 0 Done Sent Done false
32065 9983 Acked Read 1 Done Sent Done false
32066 9984 Acked Write 2 Done Sent Done false
32067 9985 Acked Read 4 Done Sent Done false
32068 9986 Acked Write 5 Done Sent Done false
32069 9987 Acked Read 6 Done Sent Done false
32070 9988 Acked Read 6 Done Sent Done false
32071 9989 Acked Write 6 Done Sent Done false
32072 9990 Acked Flush 0 Done Sent Done false
32073 9991 Acked Read 1 Done Sent Done false
32074 9992 Acked Read 1 Done Sent Done false
32075 9993 Acked Write 4 Done Sent Done false
32076 9994 Acked Read 5 Done Sent Done false
32077 9995 Acked Read 10 Done Sent Done false
32078 9996 Acked Write 4 Done Sent Done false
32079 9997 Acked Write 9 Done Sent Done false
32080 9998 Acked Write 7 Done Sent Done false
32081 9999 Acked Write 6 Done Sent Done false
32082 10000 Acked Write 1 Done Sent Done false
32083 10001 Acked Read 9 Done Sent Done false
32084 10002 Acked Write 4 Done Sent Done false
32085 10003 Acked Write 7 Done Sent Done false
32086 10004 Acked Read 5 Done Sent Done false
32087 10005 Acked Read 10 Done Sent Done false
32088 10006 Acked Write 5 Done Sent Done false
32089 10007 Acked Write 4 Done Sent Done false
32090 10008 Acked Write 5 Done Sent Done false
32091 10009 Acked Read 2 Done Sent Done false
32092 10010 Acked Read 9 Done Sent Done false
32093 10011 Acked Write 9 Done Sent Done false
32094 10012 Acked Write 4 Done Sent Done false
32095 10013 Acked Read 5 Done Sent Done false
32096 10014 Acked Read 1 Done Sent Done false
32097 10015 Acked Flush 0 Done Sent Done false
32098 10016 Acked Write 9 Done Sent Done false
32099 10017 Acked Read 2 Done Sent Done false
32100 10018 Acked Read 4 Done Sent Done false
32101 10019 Acked Write 5 Done Sent Done false
32102 10020 Acked Flush 0 Done Sent Done false
32103 10021 Acked Read 1 Done Sent Done false
32104 10022 Acked Write 3 Done Sent Done false
32105 10023 Acked Write 9 Done Sent Done false
32106 10024 Acked Write 10 Done Sent Done false
32107 10025 Acked Write 6 Done Sent Done false
32108 10026 Acked Read 8 Done Sent Done false
32109 10027 Acked Write 4 Done Sent Done false
32110 10028 Acked Flush 0 Done Sent Done false
32111 10029 Acked Flush 0 Done Sent Done false
32112 10030 Acked Write 3 Done Sent Done false
32113 10031 Acked Read 6 Done Sent Done false
32114 10032 Acked Flush 0 Done Sent Done false
32115 10033 Acked Read 1 Done Sent Done false
32116 10034 Acked Read 10 Done Sent Done false
32117 10035 Acked Write 3 Done Sent Done false
32118 10036 Acked Read 7 Done Sent Done false
32119 10037 Acked Read 8 Done Sent Done false
32120 10038 Acked Read 7 Done Sent Done false
32121 10039 Acked Flush 0 Done Sent Done false
32122 10040 Acked Read 5 Done Sent Done false
32123 10041 Acked Write 7 Done Sent Done false
32124 10042 Acked Write 9 Done Sent Done false
32125 10043 Acked Read 8 Done Sent Done false
32126 10044 Acked Read 8 Done Sent Done false
32127 10045 Acked Read 10 Done Sent Done false
32128 10046 Acked Flush 0 Done Sent Done false
32129 10047 Acked Read 2 Done Sent Done false
32130 10048 Acked Read 3 Done Sent Done false
32131 10049 Acked Flush 0 Done Sent Done false
32132 10050 Acked Write 8 Done Sent Done false
32133 10051 Acked Write 4 Done Sent Done false
32134 10052 Acked Write 1 Done Sent Done false
32135 10053 Acked Read 6 Done Sent Done false
32136 10054 Acked Read 5 Done Sent Done false
32137 10055 Acked Read 9 Done Sent Done false
32138 10056 Acked Read 5 Done Sent Done false
32139 STATES DS:0 DS:1 DS:2 TOTAL
32140 Sent 0 301 0 301
32141 Done 301 0 301 602
32142 Skipped 0 0 0 0
32143 Error 0 0 0 0
32144 Last Flush: 10049 9755 10049
32145 Downstairs last five completed: 9755 9754 9753 9752 9751
32146 Upstairs last five completed: 10056 10055 10054 10053 10052
32147 CLIENT: Up:0 ds:301 act:2
32148 ----------------------------------------------------------------
32149 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32150 States: Active Offline Active
32151 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32152 9756 Acked Write 10 Done Sent Done false
32153 9757 Acked Write 6 Done Sent Done false
32154 9758 Acked Write 9 Done Sent Done false
32155 9759 Acked Read 5 Done Sent Done false
32156 9760 Acked Write 1 Done Sent Done false
32157 9761 Acked Write 6 Done Sent Done false
32158 9762 Acked Write 1 Done Sent Done false
32159 9763 Acked Flush 0 Done Sent Done false
32160 9764 Acked Read 1 Done Sent Done false
32161 9765 Acked Read 4 Done Sent Done false
32162 9766 Acked Write 5 Done Sent Done false
32163 9767 Acked Flush 0 Done Sent Done false
32164 9768 Acked Read 8 Done Sent Done false
32165 9769 Acked Read 8 Done Sent Done false
32166 9770 Acked Read 5 Done Sent Done false
32167 9771 Acked Write 2 Done Sent Done false
32168 9772 Acked Read 10 Done Sent Done false
32169 9773 Acked Write 2 Done Sent Done false
32170 9774 Acked Read 4 Done Sent Done false
32171 9775 Acked Write 4 Done Sent Done false
32172 9776 Acked Read 2 Done Sent Done false
32173 9777 Acked Read 9 Done Sent Done false
32174 9778 Acked Write 10 Done Sent Done false
32175 9779 Acked Read 7 Done Sent Done false
32176 9780 Acked Write 6 Done Sent Done false
32177 9781 Acked Write 1 Done Sent Done false
32178 9782 Acked Write 8 Done Sent Done false
32179 9783 Acked Write 4 Done Sent Done false
32180 9784 Acked Read 4 Done Sent Done false
32181 9785 Acked Write 7 Done Sent Done false
32182 9786 Acked Read 7 Done Sent Done false
32183 9787 Acked Read 5 Done Sent Done false
32184 9788 Acked Read 8 Done Sent Done false
32185 9789 Acked Read 6 Done Sent Done false
32186 9790 Acked Read 7 Done Sent Done false
32187 9791 Acked Read 4 Done Sent Done false
32188 9792 Acked Read 4 Done Sent Done false
32189 9793 Acked Write 6 Done Sent Done false
32190 9794 Acked Write 7 Done Sent Done false
32191 9795 Acked Read 1 Done Sent Done false
32192 9796 Acked Flush 0 Done Sent Done false
32193 9797 Acked Write 2 Done Sent Done false
32194 9798 Acked Read 5 Done Sent Done false
32195 9799 Acked Flush 0 Done Sent Done false
32196 9800 Acked Write 4 Done Sent Done false
32197 9801 Acked Write 1 Done Sent Done false
32198 9802 Acked Read 3 Done Sent Done false
32199 9803 Acked Read 9 Done Sent Done false
32200 9804 Acked Read 6 Done Sent Done false
32201 9805 Acked Write 3 Done Sent Done false
32202 9806 Acked Read 7 Done Sent Done false
32203 9807 Acked Read 9 Done Sent Done false
32204 9808 Acked Write 7 Done Sent Done false
32205 9809 Acked Read 10 Done Sent Done false
32206 9810 Acked Read 6 Done Sent Done false
32207 9811 Acked Read 4 Done Sent Done false
32208 9812 Acked Write 3 Done Sent Done false
32209 9813 Acked Read 6 Done Sent Done false
32210 9814 Acked Write 4 Done Sent Done false
32211 9815 Acked Write 5 Done Sent Done false
32212 9816 Acked Write 9 Done Sent Done false
32213 9817 Acked Write 7 Done Sent Done false
32214 9818 Acked Write 1 Done Sent Done false
32215 9819 Acked Read 4 Done Sent Done false
32216 9820 Acked Write 3 Done Sent Done false
32217 9821 Acked Flush 0 Done Sent Done false
32218 9822 Acked Read 8 Done Sent Done false
32219 9823 Acked Read 5 Done Sent Done false
32220 9824 Acked Read 6 Done Sent Done false
32221 9825 Acked Write 2 Done Sent Done false
32222 9826 Acked Read 9 Done Sent Done false
32223 9827 Acked Read 4 Done Sent Done false
32224 9828 Acked Read 5 Done Sent Done false
32225 9829 Acked Write 2 Done Sent Done false
32226 9830 Acked Read 9 Done Sent Done false
32227 9831 Acked Read 6 Done Sent Done false
32228 9832 Acked Write 1 Done Sent Done false
32229 9833 Acked Write 1 Done Sent Done false
32230 9834 Acked Flush 0 Done Sent Done false
32231 9835 Acked Write 6 Done Sent Done false
32232 9836 Acked Write 6 Done Sent Done false
32233 9837 Acked Write 4 Done Sent Done false
32234 9838 Acked Read 8 Done Sent Done false
32235 9839 Acked Read 5 Done Sent Done false
32236 9840 Acked Read 5 Done Sent Done false
32237 9841 Acked Write 10 Done Sent Done false
32238 9842 Acked Read 9 Done Sent Done false
32239 9843 Acked Write 5 Done Sent Done false
32240 9844 Acked Write 2 Done Sent Done false
32241 9845 Acked Read 2 Done Sent Done false
32242 9846 Acked Read 10 Done Sent Done false
32243 9847 Acked Flush 0 Done Sent Done false
32244 9848 Acked Write 6 Done Sent Done false
32245 9849 Acked Write 2 Done Sent Done false
32246 9850 Acked Write 7 Done Sent Done false
32247 9851 Acked Read 8 Done Sent Done false
32248 9852 Acked Write 3 Done Sent Done false
32249 9853 Acked Write 2 Done Sent Done false
32250 9854 Acked Read 9 Done Sent Done false
32251 9855 Acked Read 10 Done Sent Done false
32252 9856 Acked Flush 0 Done Sent Done false
32253 9857 Acked Read 5 Done Sent Done false
32254 9858 Acked Read 7 Done Sent Done false
32255 9859 Acked Write 6 Done Sent Done false
32256 9860 Acked Write 10 Done Sent Done false
32257 9861 Acked Write 10 Done Sent Done false
32258 9862 Acked Read 7 Done Sent Done false
32259 9863 Acked Read 1 Done Sent Done false
32260 9864 Acked Read 2 Done Sent Done false
32261 9865 Acked Write 9 Done Sent Done false
32262 9866 Acked Read 6 Done Sent Done false
32263 9867 Acked Write 6 Done Sent Done false
32264 9868 Acked Read 9 Done Sent Done false
32265 9869 Acked Read 9 Done Sent Done false
32266 9870 Acked Read 7 Done Sent Done false
32267 9871 Acked Write 10 Done Sent Done false
32268 9872 Acked Flush 0 Done Sent Done false
32269 9873 Acked Read 7 Done Sent Done false
32270 9874 Acked Read 3 Done Sent Done false
32271 9875 Acked Read 9 Done Sent Done false
32272 9876 Acked Write 8 Done Sent Done false
32273 9877 Acked Read 9 Done Sent Done false
32274 9878 Acked Read 10 Done Sent Done false
32275 9879 Acked Write 2 Done Sent Done false
32276 9880 Acked Read 1 Done Sent Done false
32277 9881 Acked Read 2 Done Sent Done false
32278 9882 Acked Flush 0 Done Sent Done false
32279 9883 Acked Read 4 Done Sent Done false
32280 9884 Acked Read 2 Done Sent Done false
32281 9885 Acked Flush 0 Done Sent Done false
32282 9886 Acked Write 10 Done Sent Done false
32283 9887 Acked Write 10 Done Sent Done false
32284 9888 Acked Read 10 Done Sent Done false
32285 9889 Acked Read 9 Done Sent Done false
32286 9890 Acked Write 2 Done Sent Done false
32287 9891 Acked Read 2 Done Sent Done false
32288 9892 Acked Read 7 Done Sent Done false
32289 9893 Acked Read 10 Done Sent Done false
32290 9894 Acked Write 3 Done Sent Done false
32291 9895 Acked Flush 0 Done Sent Done false
32292 9896 Acked Read 7 Done Sent Done false
32293 9897 Acked Write 10 Done Sent Done false
32294 9898 Acked Read 9 Done Sent Done false
32295 9899 Acked Write 10 Done Sent Done false
32296 9900 Acked Write 9 Done Sent Done false
32297 9901 Acked Read 9 Done Sent Done false
32298 9902 Acked Write 7 Done Sent Done false
32299 9903 Acked Read 5 Done Sent Done false
32300 9904 Acked Write 7 Done Sent Done false
32301 9905 Acked Write 7 Done Sent Done false
32302 9906 Acked Write 4 Done Sent Done false
32303 9907 Acked Flush 0 Done Sent Done false
32304 9908 Acked Flush 0 Done Sent Done false
32305 9909 Acked Write 8 Done Sent Done false
32306 9910 Acked Write 7 Done Sent Done false
32307 9911 Acked Flush 0 Done Sent Done false
32308 9912 Acked Read 5 Done Sent Done false
32309 9913 Acked Write 2 Done Sent Done false
32310 9914 Acked Write 9 Done Sent Done false
32311 9915 Acked Flush 0 Done Sent Done false
32312 9916 Acked Read 9 Done Sent Done false
32313 9917 Acked Write 2 Done Sent Done false
32314 9918 Acked Write 10 Done Sent Done false
32315 9919 Acked Read 3 Done Sent Done false
32316 9920 Acked Write 2 Done Sent Done false
32317 9921 Acked Write 5 Done Sent Done false
32318 9922 Acked Read 3 Done Sent Done false
32319 9923 Acked Write 3 Done Sent Done false
32320 9924 Acked Flush 0 Done Sent Done false
32321 9925 Acked Flush 0 Done Sent Done false
32322 9926 Acked Read 2 Done Sent Done false
32323 9927 Acked Flush 0 Done Sent Done false
32324 9928 Acked Read 1 Done Sent Done false
32325 9929 Acked Read 8 Done Sent Done false
32326 9930 Acked Read 7 Done Sent Done false
32327 9931 Acked Read 9 Done Sent Done false
32328 9932 Acked Read 2 Done Sent Done false
32329 9933 Acked Read 8 Done Sent Done false
32330 9934 Acked Write 9 Done Sent Done false
32331 9935 Acked Read 8 Done Sent Done false
32332 9936 Acked Write 2 Done Sent Done false
32333 9937 Acked Write 10 Done Sent Done false
32334 9938 Acked Write 8 Done Sent Done false
32335 9939 Acked Read 2 Done Sent Done false
32336 9940 Acked Write 10 Done Sent Done false
32337 9941 Acked Read 10 Done Sent Done false
32338 9942 Acked Read 1 Done Sent Done false
32339 9943 Acked Read 5 Done Sent Done false
32340 9944 Acked Read 7 Done Sent Done false
32341 9945 Acked Read 10 Done Sent Done false
32342 9946 Acked Read 9 Done Sent Done false
32343 9947 Acked Write 9 Done Sent Done false
32344 9948 Acked Write 10 Done Sent Done false
32345 9949 Acked Write 4 Done Sent Done false
32346 9950 Acked Write 9 Done Sent Done false
32347 9951 Acked Read 8 Done Sent Done false
32348 9952 Acked Read 4 Done Sent Done false
32349 9953 Acked Read 1 Done Sent Done false
32350 9954 Acked Read 9 Done Sent Done false
32351 9955 Acked Read 2 Done Sent Done false
32352 9956 Acked Write 4 Done Sent Done false
32353 9957 Acked Write 4 Done Sent Done false
32354 9958 Acked Read 4 Done Sent Done false
32355 9959 Acked Write 5 Done Sent Done false
32356 9960 Acked Read 4 Done Sent Done false
32357 9961 Acked Read 5 Done Sent Done false
32358 9962 Acked Flush 0 Done Sent Done false
32359 9963 Acked Read 10 Done Sent Done false
32360 9964 Acked Read 1 Done Sent Done false
32361 9965 Acked Flush 0 Done Sent Done false
32362 9966 Acked Write 9 Done Sent Done false
32363 9967 Acked Read 2 Done Sent Done false
32364 9968 Acked Write 10 Done Sent Done false
32365 9969 Acked Read 5 Done Sent Done false
32366 9970 Acked Read 8 Done Sent Done false
32367 9971 Acked Write 6 Done Sent Done false
32368 9972 Acked Read 9 Done Sent Done false
32369 9973 Acked Write 6 Done Sent Done false
32370 9974 Acked Write 1 Done Sent Done false
32371 9975 Acked Read 6 Done Sent Done false
32372 9976 Acked Read 5 Done Sent Done false
32373 9977 Acked Read 2 Done Sent Done false
32374 9978 Acked Read 9 Done Sent Done false
32375 9979 Acked Read 3 Done Sent Done false
32376 9980 Acked Read 10 Done Sent Done false
32377 9981 Acked Read 2 Done Sent Done false
32378 9982 Acked Flush 0 Done Sent Done false
32379 9983 Acked Read 1 Done Sent Done false
32380 9984 Acked Write 2 Done Sent Done false
32381 9985 Acked Read 4 Done Sent Done false
32382 9986 Acked Write 5 Done Sent Done false
32383 9987 Acked Read 6 Done Sent Done false
32384 9988 Acked Read 6 Done Sent Done false
32385 9989 Acked Write 6 Done Sent Done false
32386 9990 Acked Flush 0 Done Sent Done false
32387 9991 Acked Read 1 Done Sent Done false
32388 9992 Acked Read 1 Done Sent Done false
32389 9993 Acked Write 4 Done Sent Done false
32390 9994 Acked Read 5 Done Sent Done false
32391 9995 Acked Read 10 Done Sent Done false
32392 9996 Acked Write 4 Done Sent Done false
32393 9997 Acked Write 9 Done Sent Done false
32394 9998 Acked Write 7 Done Sent Done false
32395 9999 Acked Write 6 Done Sent Done false
32396 10000 Acked Write 1 Done Sent Done false
32397 10001 Acked Read 9 Done Sent Done false
32398 10002 Acked Write 4 Done Sent Done false
32399 10003 Acked Write 7 Done Sent Done false
32400 10004 Acked Read 5 Done Sent Done false
32401 10005 Acked Read 10 Done Sent Done false
32402 10006 Acked Write 5 Done Sent Done false
32403 10007 Acked Write 4 Done Sent Done false
32404 10008 Acked Write 5 Done Sent Done false
32405 10009 Acked Read 2 Done Sent Done false
32406 10010 Acked Read 9 Done Sent Done false
32407 10011 Acked Write 9 Done Sent Done false
32408 10012 Acked Write 4 Done Sent Done false
32409 10013 Acked Read 5 Done Sent Done false
32410 10014 Acked Read 1 Done Sent Done false
32411 10015 Acked Flush 0 Done Sent Done false
32412 10016 Acked Write 9 Done Sent Done false
32413 10017 Acked Read 2 Done Sent Done false
32414 10018 Acked Read 4 Done Sent Done false
32415 10019 Acked Write 5 Done Sent Done false
32416 10020 Acked Flush 0 Done Sent Done false
32417 10021 Acked Read 1 Done Sent Done false
32418 10022 Acked Write 3 Done Sent Done false
32419 10023 Acked Write 9 Done Sent Done false
32420 10024 Acked Write 10 Done Sent Done false
32421 10025 Acked Write 6 Done Sent Done false
32422 10026 Acked Read 8 Done Sent Done false
32423 10027 Acked Write 4 Done Sent Done false
32424 10028 Acked Flush 0 Done Sent Done false
32425 10029 Acked Flush 0 Done Sent Done false
32426 10030 Acked Write 3 Done Sent Done false
32427 10031 Acked Read 6 Done Sent Done false
32428 10032 Acked Flush 0 Done Sent Done false
32429 10033 Acked Read 1 Done Sent Done false
32430 10034 Acked Read 10 Done Sent Done false
32431 10035 Acked Write 3 Done Sent Done false
32432 10036 Acked Read 7 Done Sent Done false
32433 10037 Acked Read 8 Done Sent Done false
32434 10038 Acked Read 7 Done Sent Done false
32435 10039 Acked Flush 0 Done Sent Done false
32436 10040 Acked Read 5 Done Sent Done false
32437 10041 Acked Write 7 Done Sent Done false
32438 10042 Acked Write 9 Done Sent Done false
32439 10043 Acked Read 8 Done Sent Done false
32440 10044 Acked Read 8 Done Sent Done false
32441 10045 Acked Read 10 Done Sent Done false
32442 10046 Acked Flush 0 Done Sent Done false
32443 10047 Acked Read 2 Done Sent Done false
32444 10048 Acked Read 3 Done Sent Done false
32445 10049 Acked Flush 0 Done Sent Done false
32446 10050 Acked Write 8 Done Sent Done false
32447 10051 Acked Write 4 Done Sent Done false
32448 10052 Acked Write 1 Done Sent Done false
32449 10053 Acked Read 6 Done Sent Done false
32450 10054 Acked Read 5 Done Sent Done false
32451 10055 Acked Read 9 Done Sent Done false
32452 10056 Acked Read 5 Done Sent Done false
32453 10057 Acked Flush 0 Done Sent Done false
32454 STATES DS:0 DS:1 DS:2 TOTAL
32455 Sent 0 302 0 302
32456 Done 302 0 302 604
32457 Skipped 0 0 0 0
32458 Error 0 0 0 0
32459 Last Flush: 10057 9755 10057
32460 Downstairs last five completed: 9755 9754 9753 9752 9751
32461 Upstairs last five completed: 10057 10056 10055 10054 10053
32462 CLIENT: Up:0 ds:302 act:2
324632025-01-21T15:53:58.544ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
324642025-01-21T15:53:58.544ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
324652025-01-21T15:53:58.544ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
324662025-01-21T15:53:58.544ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
324672025-01-21T15:53:58.544ZINFOcrucible: client task is exiting = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
32468 ----------------------------------------------------------------
32469 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32470 States: Active Offline Active
32471 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32472 9756 Acked Write 10 Done Sent Done false
32473 9757 Acked Write 6 Done Sent Done false
32474 9758 Acked Write 9 Done Sent Done false
32475 9759 Acked Read 5 Done Sent Done false
32476 9760 Acked Write 1 Done Sent Done false
32477 9761 Acked Write 6 Done Sent Done false
32478 9762 Acked Write 1 Done Sent Done false
32479 9763 Acked Flush 0 Done Sent Done false
32480 9764 Acked Read 1 Done Sent Done false
32481 9765 Acked Read 4 Done Sent Done false
32482 9766 Acked Write 5 Done Sent Done false
32483 9767 Acked Flush 0 Done Sent Done false
32484 9768 Acked Read 8 Done Sent Done false
32485 9769 Acked Read 8 Done Sent Done false
32486 9770 Acked Read 5 Done Sent Done false
32487 9771 Acked Write 2 Done Sent Done false
32488 9772 Acked Read 10 Done Sent Done false
32489 9773 Acked Write 2 Done Sent Done false
32490 9774 Acked Read 4 Done Sent Done false
32491 9775 Acked Write 4 Done Sent Done false
32492 9776 Acked Read 2 Done Sent Done false
32493 9777 Acked Read 9 Done Sent Done false
32494 9778 Acked Write 10 Done Sent Done false
32495 9779 Acked Read 7 Done Sent Done false
32496 9780 Acked Write 6 Done Sent Done false
32497 9781 Acked Write 1 Done Sent Done false
32498 9782 Acked Write 8 Done Sent Done false
32499 9783 Acked Write 4 Done Sent Done false
32500 9784 Acked Read 4 Done Sent Done false
32501 9785 Acked Write 7 Done Sent Done false
32502 9786 Acked Read 7 Done Sent Done false
32503 9787 Acked Read 5 Done Sent Done false
32504 9788 Acked Read 8 Done Sent Done false
32505 9789 Acked Read 6 Done Sent Done false
32506 9790 Acked Read 7 Done Sent Done false
32507 9791 Acked Read 4 Done Sent Done false
32508 9792 Acked Read 4 Done Sent Done false
32509 9793 Acked Write 6 Done Sent Done false
32510 9794 Acked Write 7 Done Sent Done false
32511 9795 Acked Read 1 Done Sent Done false
32512 9796 Acked Flush 0 Done Sent Done false
32513 9797 Acked Write 2 Done Sent Done false
32514 9798 Acked Read 5 Done Sent Done false
32515 9799 Acked Flush 0 Done Sent Done false
32516 9800 Acked Write 4 Done Sent Done false
32517 9801 Acked Write 1 Done Sent Done false
32518 9802 Acked Read 3 Done Sent Done false
32519 9803 Acked Read 9 Done Sent Done false
32520 9804 Acked Read 6 Done Sent Done false
32521 9805 Acked Write 3 Done Sent Done false
32522 9806 Acked Read 7 Done Sent Done false
32523 9807 Acked Read 9 Done Sent Done false
32524 9808 Acked Write 7 Done Sent Done false
32525 9809 Acked Read 10 Done Sent Done false
32526 9810 Acked Read 6 Done Sent Done false
32527 9811 Acked Read 4 Done Sent Done false
32528 9812 Acked Write 3 Done Sent Done false
32529 9813 Acked Read 6 Done Sent Done false
32530 9814 Acked Write 4 Done Sent Done false
32531 9815 Acked Write 5 Done Sent Done false
32532 9816 Acked Write 9 Done Sent Done false
32533 9817 Acked Write 7 Done Sent Done false
32534 9818 Acked Write 1 Done Sent Done false
32535 9819 Acked Read 4 Done Sent Done false
32536 9820 Acked Write 3 Done Sent Done false
32537 9821 Acked Flush 0 Done Sent Done false
32538 9822 Acked Read 8 Done Sent Done false
32539 9823 Acked Read 5 Done Sent Done false
32540 9824 Acked Read 6 Done Sent Done false
32541 9825 Acked Write 2 Done Sent Done false
32542 9826 Acked Read 9 Done Sent Done false
32543 9827 Acked Read 4 Done Sent Done false
32544 9828 Acked Read 5 Done Sent Done false
32545 9829 Acked Write 2 Done Sent Done false
32546 9830 Acked Read 9 Done Sent Done false
32547 9831 Acked Read 6 Done Sent Done false
32548 9832 Acked Write 1 Done Sent Done false
32549 9833 Acked Write 1 Done Sent Done false
32550 9834 Acked Flush 0 Done Sent Done false
32551 9835 Acked Write 6 Done Sent Done false
32552 9836 Acked Write 6 Done Sent Done false
32553 9837 Acked Write 4 Done Sent Done false
32554 9838 Acked Read 8 Done Sent Done false
32555 9839 Acked Read 5 Done Sent Done false
32556 9840 Acked Read 5 Done Sent Done false
32557 9841 Acked Write 10 Done Sent Done false
32558 9842 Acked Read 9 Done Sent Done false
32559 9843 Acked Write 5 Done Sent Done false
32560 9844 Acked Write 2 Done Sent Done false
32561 9845 Acked Read 2 Done Sent Done false
32562 9846 Acked Read 10 Done Sent Done false
32563 9847 Acked Flush 0 Done Sent Done false
32564 9848 Acked Write 6 Done Sent Done false
32565 9849 Acked Write 2 Done Sent Done false
32566 9850 Acked Write 7 Done Sent Done false
32567 9851 Acked Read 8 Done Sent Done false
32568 9852 Acked Write 3 Done Sent Done false
32569 9853 Acked Write 2 Done Sent Done false
32570 9854 Acked Read 9 Done Sent Done false
32571 9855 Acked Read 10 Done Sent Done false
32572 9856 Acked Flush 0 Done Sent Done false
32573 9857 Acked Read 5 Done Sent Done false
32574 9858 Acked Read 7 Done Sent Done false
32575 9859 Acked Write 6 Done Sent Done false
32576 9860 Acked Write 10 Done Sent Done false
32577 9861 Acked Write 10 Done Sent Done false
32578 9862 Acked Read 7 Done Sent Done false
32579 9863 Acked Read 1 Done Sent Done false
32580 9864 Acked Read 2 Done Sent Done false
32581 9865 Acked Write 9 Done Sent Done false
32582 9866 Acked Read 6 Done Sent Done false
32583 9867 Acked Write 6 Done Sent Done false
32584 9868 Acked Read 9 Done Sent Done false
32585 9869 Acked Read 9 Done Sent Done false
32586 9870 Acked Read 7 Done Sent Done false
32587 9871 Acked Write 10 Done Sent Done false
32588 9872 Acked Flush 0 Done Sent Done false
32589 9873 Acked Read 7 Done Sent Done false
32590 9874 Acked Read 3 Done Sent Done false
32591 9875 Acked Read 9 Done Sent Done false
32592 9876 Acked Write 8 Done Sent Done false
32593 9877 Acked Read 9 Done Sent Done false
32594 9878 Acked Read 10 Done Sent Done false
32595 9879 Acked Write 2 Done Sent Done false
32596 9880 Acked Read 1 Done Sent Done false
32597 9881 Acked Read 2 Done Sent Done false
32598 9882 Acked Flush 0 Done Sent Done false
32599 9883 Acked Read 4 Done Sent Done false
32600 9884 Acked Read 2 Done Sent Done false
32601 9885 Acked Flush 0 Done Sent Done false
32602 9886 Acked Write 10 Done Sent Done false
32603 9887 Acked Write 10 Done Sent Done false
32604 9888 Acked Read 10 Done Sent Done false
32605 9889 Acked Read 9 Done Sent Done false
32606 9890 Acked Write 2 Done Sent Done false
32607 9891 Acked Read 2 Done Sent Done false
32608 9892 Acked Read 7 Done Sent Done false
32609 9893 Acked Read 10 Done Sent Done false
32610 9894 Acked Write 3 Done Sent Done false
32611 9895 Acked Flush 0 Done Sent Done false
32612 9896 Acked Read 7 Done Sent Done false
32613 9897 Acked Write 10 Done Sent Done false
32614 9898 Acked Read 9 Done Sent Done false
32615 9899 Acked Write 10 Done Sent Done false
32616 9900 Acked Write 9 Done Sent Done false
32617 9901 Acked Read 9 Done Sent Done false
32618 9902 Acked Write 7 Done Sent Done false
32619 9903 Acked Read 5 Done Sent Done false
32620 9904 Acked Write 7 Done Sent Done false
32621 9905 Acked Write 7 Done Sent Done false
32622 9906 Acked Write 4 Done Sent Done false
32623 9907 Acked Flush 0 Done Sent Done false
32624 9908 Acked Flush 0 Done Sent Done false
32625 9909 Acked Write 8 Done Sent Done false
32626 9910 Acked Write 7 Done Sent Done false
32627 9911 Acked Flush 0 Done Sent Done false
32628 9912 Acked Read 5 Done Sent Done false
32629 9913 Acked Write 2 Done Sent Done false
32630 9914 Acked Write 9 Done Sent Done false
32631 9915 Acked Flush 0 Done Sent Done false
32632 9916 Acked Read 9 Done Sent Done false
32633 9917 Acked Write 2 Done Sent Done false
32634 9918 Acked Write 10 Done Sent Done false
32635 9919 Acked Read 3 Done Sent Done false
32636 9920 Acked Write 2 Done Sent Done false
32637 9921 Acked Write 5 Done Sent Done false
32638 9922 Acked Read 3 Done Sent Done false
32639 9923 Acked Write 3 Done Sent Done false
32640 9924 Acked Flush 0 Done Sent Done false
32641 9925 Acked Flush 0 Done Sent Done false
32642 9926 Acked Read 2 Done Sent Done false
32643 9927 Acked Flush 0 Done Sent Done false
32644 9928 Acked Read 1 Done Sent Done false
32645 9929 Acked Read 8 Done Sent Done false
32646 9930 Acked Read 7 Done Sent Done false
32647 9931 Acked Read 9 Done Sent Done false
32648 9932 Acked Read 2 Done Sent Done false
32649 9933 Acked Read 8 Done Sent Done false
32650 9934 Acked Write 9 Done Sent Done false
32651 9935 Acked Read 8 Done Sent Done false
32652 9936 Acked Write 2 Done Sent Done false
32653 9937 Acked Write 10 Done Sent Done false
32654 9938 Acked Write 8 Done Sent Done false
32655 9939 Acked Read 2 Done Sent Done false
32656 9940 Acked Write 10 Done Sent Done false
32657 9941 Acked Read 10 Done Sent Done false
32658 9942 Acked Read 1 Done Sent Done false
32659 9943 Acked Read 5 Done Sent Done false
32660 9944 Acked Read 7 Done Sent Done false
32661 9945 Acked Read 10 Done Sent Done false
32662 9946 Acked Read 9 Done Sent Done false
32663 9947 Acked Write 9 Done Sent Done false
32664 9948 Acked Write 10 Done Sent Done false
32665 9949 Acked Write 4 Done Sent Done false
32666 9950 Acked Write 9 Done Sent Done false
32667 9951 Acked Read 8 Done Sent Done false
32668 9952 Acked Read 4 Done Sent Done false
32669 9953 Acked Read 1 Done Sent Done false
32670 9954 Acked Read 9 Done Sent Done false
32671 9955 Acked Read 2 Done Sent Done false
32672 9956 Acked Write 4 Done Sent Done false
32673 9957 Acked Write 4 Done Sent Done false
32674 9958 Acked Read 4 Done Sent Done false
32675 9959 Acked Write 5 Done Sent Done false
32676 9960 Acked Read 4 Done Sent Done false
32677 9961 Acked Read 5 Done Sent Done false
32678 9962 Acked Flush 0 Done Sent Done false
32679 9963 Acked Read 10 Done Sent Done false
32680 9964 Acked Read 1 Done Sent Done false
32681 9965 Acked Flush 0 Done Sent Done false
32682 9966 Acked Write 9 Done Sent Done false
32683 9967 Acked Read 2 Done Sent Done false
32684 9968 Acked Write 10 Done Sent Done false
32685 9969 Acked Read 5 Done Sent Done false
32686 9970 Acked Read 8 Done Sent Done false
32687 9971 Acked Write 6 Done Sent Done false
32688 9972 Acked Read 9 Done Sent Done false
32689 9973 Acked Write 6 Done Sent Done false
32690 9974 Acked Write 1 Done Sent Done false
32691 9975 Acked Read 6 Done Sent Done false
32692 9976 Acked Read 5 Done Sent Done false
32693 9977 Acked Read 2 Done Sent Done false
32694 9978 Acked Read 9 Done Sent Done false
32695 9979 Acked Read 3 Done Sent Done false
32696 9980 Acked Read 10 Done Sent Done false
32697 9981 Acked Read 2 Done Sent Done false
32698 9982 Acked Flush 0 Done Sent Done false
32699 9983 Acked Read 1 Done Sent Done false
32700 9984 Acked Write 2 Done Sent Done false
32701 9985 Acked Read 4 Done Sent Done false
32702 9986 Acked Write 5 Done Sent Done false
32703 9987 Acked Read 6 Done Sent Done false
32704 9988 Acked Read 6 Done Sent Done false
32705 9989 Acked Write 6 Done Sent Done false
32706 9990 Acked Flush 0 Done Sent Done false
32707 9991 Acked Read 1 Done Sent Done false
32708 9992 Acked Read 1 Done Sent Done false
32709 9993 Acked Write 4 Done Sent Done false
32710 9994 Acked Read 5 Done Sent Done false
32711 9995 Acked Read 10 Done Sent Done false
32712 9996 Acked Write 4 Done Sent Done false
32713 9997 Acked Write 9 Done Sent Done false
32714 9998 Acked Write 7 Done Sent Done false
32715 9999 Acked Write 6 Done Sent Done false
32716 10000 Acked Write 1 Done Sent Done false
32717 10001 Acked Read 9 Done Sent Done false
32718 10002 Acked Write 4 Done Sent Done false
32719 10003 Acked Write 7 Done Sent Done false
32720 10004 Acked Read 5 Done Sent Done false
32721 10005 Acked Read 10 Done Sent Done false
32722 10006 Acked Write 5 Done Sent Done false
32723 10007 Acked Write 4 Done Sent Done false
32724 10008 Acked Write 5 Done Sent Done false
32725 10009 Acked Read 2 Done Sent Done false
32726 10010 Acked Read 9 Done Sent Done false
32727 10011 Acked Write 9 Done Sent Done false
32728 10012 Acked Write 4 Done Sent Done false
32729 10013 Acked Read 5 Done Sent Done false
32730 10014 Acked Read 1 Done Sent Done false
32731 10015 Acked Flush 0 Done Sent Done false
32732 10016 Acked Write 9 Done Sent Done false
32733 10017 Acked Read 2 Done Sent Done false
32734 10018 Acked Read 4 Done Sent Done false
32735 10019 Acked Write 5 Done Sent Done false
32736 10020 Acked Flush 0 Done Sent Done false
32737 10021 Acked Read 1 Done Sent Done false
32738 10022 Acked Write 3 Done Sent Done false
32739 10023 Acked Write 9 Done Sent Done false
32740 10024 Acked Write 10 Done Sent Done false
32741 10025 Acked Write 6 Done Sent Done false
32742 10026 Acked Read 8 Done Sent Done false
32743 10027 Acked Write 4 Done Sent Done false
32744 10028 Acked Flush 0 Done Sent Done false
32745 10029 Acked Flush 0 Done Sent Done false
32746 10030 Acked Write 3 Done Sent Done false
32747 10031 Acked Read 6 Done Sent Done false
32748 10032 Acked Flush 0 Done Sent Done false
32749 10033 Acked Read 1 Done Sent Done false
32750 10034 Acked Read 10 Done Sent Done false
32751 10035 Acked Write 3 Done Sent Done false
32752 10036 Acked Read 7 Done Sent Done false
32753 10037 Acked Read 8 Done Sent Done false
32754 10038 Acked Read 7 Done Sent Done false
32755 10039 Acked Flush 0 Done Sent Done false
32756 10040 Acked Read 5 Done Sent Done false
32757 10041 Acked Write 7 Done Sent Done false
32758 10042 Acked Write 9 Done Sent Done false
32759 10043 Acked Read 8 Done Sent Done false
32760 10044 Acked Read 8 Done Sent Done false
32761 10045 Acked Read 10 Done Sent Done false
32762 10046 Acked Flush 0 Done Sent Done false
32763 10047 Acked Read 2 Done Sent Done false
32764 10048 Acked Read 3 Done Sent Done false
32765 10049 Acked Flush 0 Done Sent Done false
32766 10050 Acked Write 8 Done Sent Done false
32767 10051 Acked Write 4 Done Sent Done false
32768 10052 Acked Write 1 Done Sent Done false
32769 10053 Acked Read 6 Done Sent Done false
32770 10054 Acked Read 5 Done Sent Done false
32771 10055 Acked Read 9 Done Sent Done false
32772 10056 Acked Read 5 Done Sent Done false
32773 10057 Acked Flush 0 Done Sent Done false
32774 STATES DS:0 DS:1 DS:2 TOTAL
32775 Sent 0 302 0 302
32776 Done 302 0 302 604
32777 Skipped 0 0 0 0
32778 Error 0 0 0 0
32779 Last Flush: 10057 9755 10057
32780 Downstairs last five completed: 9755 9754 9753 9752 9751
32781 Upstairs last five completed: 10057 10056 10055 10054 10053
32782 CLIENT: Up:0 ds:302 act:2
32783 [8820] got start action
32784 [8820] I am going to restart
32785 Starting downstairs at port 8820
32786 Make output file at "/tmp/dsc/downstairs-8820.txt"
32787 [8820][1] reports Starting
32788 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1230)
32789 [8820][1] reports Running
32790 ----------------------------------------------------------------
32791 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32792 States: Active Offline Active
32793 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32794 9756 Acked Write 10 Done Sent Done false
32795 9757 Acked Write 6 Done Sent Done false
32796 9758 Acked Write 9 Done Sent Done false
32797 9759 Acked Read 5 Done Sent Done false
32798 9760 Acked Write 1 Done Sent Done false
32799 9761 Acked Write 6 Done Sent Done false
32800 9762 Acked Write 1 Done Sent Done false
32801 9763 Acked Flush 0 Done Sent Done false
32802 9764 Acked Read 1 Done Sent Done false
32803 9765 Acked Read 4 Done Sent Done false
32804 9766 Acked Write 5 Done Sent Done false
32805 9767 Acked Flush 0 Done Sent Done false
32806 9768 Acked Read 8 Done Sent Done false
32807 9769 Acked Read 8 Done Sent Done false
32808 9770 Acked Read 5 Done Sent Done false
32809 9771 Acked Write 2 Done Sent Done false
32810 9772 Acked Read 10 Done Sent Done false
32811 9773 Acked Write 2 Done Sent Done false
32812 9774 Acked Read 4 Done Sent Done false
32813 9775 Acked Write 4 Done Sent Done false
32814 9776 Acked Read 2 Done Sent Done false
32815 9777 Acked Read 9 Done Sent Done false
32816 9778 Acked Write 10 Done Sent Done false
32817 9779 Acked Read 7 Done Sent Done false
32818 9780 Acked Write 6 Done Sent Done false
32819 9781 Acked Write 1 Done Sent Done false
32820 9782 Acked Write 8 Done Sent Done false
32821 9783 Acked Write 4 Done Sent Done false
32822 9784 Acked Read 4 Done Sent Done false
32823 9785 Acked Write 7 Done Sent Done false
32824 9786 Acked Read 7 Done Sent Done false
32825 9787 Acked Read 5 Done Sent Done false
32826 9788 Acked Read 8 Done Sent Done false
32827 9789 Acked Read 6 Done Sent Done false
32828 9790 Acked Read 7 Done Sent Done false
32829 9791 Acked Read 4 Done Sent Done false
32830 9792 Acked Read 4 Done Sent Done false
32831 9793 Acked Write 6 Done Sent Done false
32832 9794 Acked Write 7 Done Sent Done false
32833 9795 Acked Read 1 Done Sent Done false
32834 9796 Acked Flush 0 Done Sent Done false
32835 9797 Acked Write 2 Done Sent Done false
32836 9798 Acked Read 5 Done Sent Done false
32837 9799 Acked Flush 0 Done Sent Done false
32838 9800 Acked Write 4 Done Sent Done false
32839 9801 Acked Write 1 Done Sent Done false
32840 9802 Acked Read 3 Done Sent Done false
32841 9803 Acked Read 9 Done Sent Done false
32842 9804 Acked Read 6 Done Sent Done false
32843 9805 Acked Write 3 Done Sent Done false
32844 9806 Acked Read 7 Done Sent Done false
32845 9807 Acked Read 9 Done Sent Done false
32846 9808 Acked Write 7 Done Sent Done false
32847 9809 Acked Read 10 Done Sent Done false
32848 9810 Acked Read 6 Done Sent Done false
32849 9811 Acked Read 4 Done Sent Done false
32850 9812 Acked Write 3 Done Sent Done false
32851 9813 Acked Read 6 Done Sent Done false
32852 9814 Acked Write 4 Done Sent Done false
32853 9815 Acked Write 5 Done Sent Done false
32854 9816 Acked Write 9 Done Sent Done false
32855 9817 Acked Write 7 Done Sent Done false
32856 9818 Acked Write 1 Done Sent Done false
32857 9819 Acked Read 4 Done Sent Done false
32858 9820 Acked Write 3 Done Sent Done false
32859 9821 Acked Flush 0 Done Sent Done false
32860 9822 Acked Read 8 Done Sent Done false
32861 9823 Acked Read 5 Done Sent Done false
32862 9824 Acked Read 6 Done Sent Done false
32863 9825 Acked Write 2 Done Sent Done false
32864 9826 Acked Read 9 Done Sent Done false
32865 9827 Acked Read 4 Done Sent Done false
32866 9828 Acked Read 5 Done Sent Done false
32867 9829 Acked Write 2 Done Sent Done false
32868 9830 Acked Read 9 Done Sent Done false
32869 9831 Acked Read 6 Done Sent Done false
32870 9832 Acked Write 1 Done Sent Done false
32871 9833 Acked Write 1 Done Sent Done false
32872 9834 Acked Flush 0 Done Sent Done false
32873 9835 Acked Write 6 Done Sent Done false
32874 9836 Acked Write 6 Done Sent Done false
32875 9837 Acked Write 4 Done Sent Done false
32876 9838 Acked Read 8 Done Sent Done false
32877 9839 Acked Read 5 Done Sent Done false
32878 9840 Acked Read 5 Done Sent Done false
32879 9841 Acked Write 10 Done Sent Done false
32880 9842 Acked Read 9 Done Sent Done false
32881 9843 Acked Write 5 Done Sent Done false
32882 9844 Acked Write 2 Done Sent Done false
32883 9845 Acked Read 2 Done Sent Done false
32884 9846 Acked Read 10 Done Sent Done false
32885 9847 Acked Flush 0 Done Sent Done false
32886 9848 Acked Write 6 Done Sent Done false
32887 9849 Acked Write 2 Done Sent Done false
32888 9850 Acked Write 7 Done Sent Done false
32889 9851 Acked Read 8 Done Sent Done false
32890 9852 Acked Write 3 Done Sent Done false
32891 9853 Acked Write 2 Done Sent Done false
32892 9854 Acked Read 9 Done Sent Done false
32893 9855 Acked Read 10 Done Sent Done false
32894 9856 Acked Flush 0 Done Sent Done false
32895 9857 Acked Read 5 Done Sent Done false
32896 9858 Acked Read 7 Done Sent Done false
32897 9859 Acked Write 6 Done Sent Done false
32898 9860 Acked Write 10 Done Sent Done false
32899 9861 Acked Write 10 Done Sent Done false
32900 9862 Acked Read 7 Done Sent Done false
32901 9863 Acked Read 1 Done Sent Done false
32902 9864 Acked Read 2 Done Sent Done false
32903 9865 Acked Write 9 Done Sent Done false
32904 9866 Acked Read 6 Done Sent Done false
32905 9867 Acked Write 6 Done Sent Done false
32906 9868 Acked Read 9 Done Sent Done false
32907 9869 Acked Read 9 Done Sent Done false
32908 9870 Acked Read 7 Done Sent Done false
32909 9871 Acked Write 10 Done Sent Done false
32910 9872 Acked Flush 0 Done Sent Done false
32911 9873 Acked Read 7 Done Sent Done false
32912 9874 Acked Read 3 Done Sent Done false
32913 9875 Acked Read 9 Done Sent Done false
32914 9876 Acked Write 8 Done Sent Done false
32915 9877 Acked Read 9 Done Sent Done false
32916 9878 Acked Read 10 Done Sent Done false
32917 9879 Acked Write 2 Done Sent Done false
32918 9880 Acked Read 1 Done Sent Done false
32919 9881 Acked Read 2 Done Sent Done false
32920 9882 Acked Flush 0 Done Sent Done false
32921 9883 Acked Read 4 Done Sent Done false
32922 9884 Acked Read 2 Done Sent Done false
32923 9885 Acked Flush 0 Done Sent Done false
32924 9886 Acked Write 10 Done Sent Done false
32925 9887 Acked Write 10 Done Sent Done false
32926 9888 Acked Read 10 Done Sent Done false
32927 9889 Acked Read 9 Done Sent Done false
32928 9890 Acked Write 2 Done Sent Done false
32929 9891 Acked Read 2 Done Sent Done false
32930 9892 Acked Read 7 Done Sent Done false
32931 9893 Acked Read 10 Done Sent Done false
32932 9894 Acked Write 3 Done Sent Done false
32933 9895 Acked Flush 0 Done Sent Done false
32934 9896 Acked Read 7 Done Sent Done false
32935 9897 Acked Write 10 Done Sent Done false
32936 9898 Acked Read 9 Done Sent Done false
32937 9899 Acked Write 10 Done Sent Done false
32938 9900 Acked Write 9 Done Sent Done false
32939 9901 Acked Read 9 Done Sent Done false
32940 9902 Acked Write 7 Done Sent Done false
32941 9903 Acked Read 5 Done Sent Done false
32942 9904 Acked Write 7 Done Sent Done false
32943 9905 Acked Write 7 Done Sent Done false
32944 9906 Acked Write 4 Done Sent Done false
32945 9907 Acked Flush 0 Done Sent Done false
32946 9908 Acked Flush 0 Done Sent Done false
32947 9909 Acked Write 8 Done Sent Done false
32948 9910 Acked Write 7 Done Sent Done false
32949 9911 Acked Flush 0 Done Sent Done false
32950 9912 Acked Read 5 Done Sent Done false
32951 9913 Acked Write 2 Done Sent Done false
32952 9914 Acked Write 9 Done Sent Done false
32953 9915 Acked Flush 0 Done Sent Done false
32954 9916 Acked Read 9 Done Sent Done false
32955 9917 Acked Write 2 Done Sent Done false
32956 9918 Acked Write 10 Done Sent Done false
32957 9919 Acked Read 3 Done Sent Done false
32958 9920 Acked Write 2 Done Sent Done false
32959 9921 Acked Write 5 Done Sent Done false
32960 9922 Acked Read 3 Done Sent Done false
32961 9923 Acked Write 3 Done Sent Done false
32962 9924 Acked Flush 0 Done Sent Done false
32963 9925 Acked Flush 0 Done Sent Done false
32964 9926 Acked Read 2 Done Sent Done false
32965 9927 Acked Flush 0 Done Sent Done false
32966 9928 Acked Read 1 Done Sent Done false
32967 9929 Acked Read 8 Done Sent Done false
32968 9930 Acked Read 7 Done Sent Done false
32969 9931 Acked Read 9 Done Sent Done false
32970 9932 Acked Read 2 Done Sent Done false
32971 9933 Acked Read 8 Done Sent Done false
32972 9934 Acked Write 9 Done Sent Done false
32973 9935 Acked Read 8 Done Sent Done false
32974 9936 Acked Write 2 Done Sent Done false
32975 9937 Acked Write 10 Done Sent Done false
32976 9938 Acked Write 8 Done Sent Done false
32977 9939 Acked Read 2 Done Sent Done false
32978 9940 Acked Write 10 Done Sent Done false
32979 9941 Acked Read 10 Done Sent Done false
32980 9942 Acked Read 1 Done Sent Done false
32981 9943 Acked Read 5 Done Sent Done false
32982 9944 Acked Read 7 Done Sent Done false
32983 9945 Acked Read 10 Done Sent Done false
32984 9946 Acked Read 9 Done Sent Done false
32985 9947 Acked Write 9 Done Sent Done false
32986 9948 Acked Write 10 Done Sent Done false
32987 9949 Acked Write 4 Done Sent Done false
32988 9950 Acked Write 9 Done Sent Done false
32989 9951 Acked Read 8 Done Sent Done false
32990 9952 Acked Read 4 Done Sent Done false
32991 9953 Acked Read 1 Done Sent Done false
32992 9954 Acked Read 9 Done Sent Done false
32993 9955 Acked Read 2 Done Sent Done false
32994 9956 Acked Write 4 Done Sent Done false
32995 9957 Acked Write 4 Done Sent Done false
32996 9958 Acked Read 4 Done Sent Done false
32997 9959 Acked Write 5 Done Sent Done false
32998 9960 Acked Read 4 Done Sent Done false
32999 9961 Acked Read 5 Done Sent Done false
33000 9962 Acked Flush 0 Done Sent Done false
33001 9963 Acked Read 10 Done Sent Done false
33002 9964 Acked Read 1 Done Sent Done false
33003 9965 Acked Flush 0 Done Sent Done false
33004 9966 Acked Write 9 Done Sent Done false
33005 9967 Acked Read 2 Done Sent Done false
33006 9968 Acked Write 10 Done Sent Done false
33007 9969 Acked Read 5 Done Sent Done false
33008 9970 Acked Read 8 Done Sent Done false
33009 9971 Acked Write 6 Done Sent Done false
33010 9972 Acked Read 9 Done Sent Done false
33011 9973 Acked Write 6 Done Sent Done false
33012 9974 Acked Write 1 Done Sent Done false
33013 9975 Acked Read 6 Done Sent Done false
33014 9976 Acked Read 5 Done Sent Done false
33015 9977 Acked Read 2 Done Sent Done false
33016 9978 Acked Read 9 Done Sent Done false
33017 9979 Acked Read 3 Done Sent Done false
33018 9980 Acked Read 10 Done Sent Done false
33019 9981 Acked Read 2 Done Sent Done false
33020 9982 Acked Flush 0 Done Sent Done false
33021 9983 Acked Read 1 Done Sent Done false
33022 9984 Acked Write 2 Done Sent Done false
33023 9985 Acked Read 4 Done Sent Done false
33024 9986 Acked Write 5 Done Sent Done false
33025 9987 Acked Read 6 Done Sent Done false
33026 9988 Acked Read 6 Done Sent Done false
33027 9989 Acked Write 6 Done Sent Done false
33028 9990 Acked Flush 0 Done Sent Done false
33029 9991 Acked Read 1 Done Sent Done false
33030 9992 Acked Read 1 Done Sent Done false
33031 9993 Acked Write 4 Done Sent Done false
33032 9994 Acked Read 5 Done Sent Done false
33033 9995 Acked Read 10 Done Sent Done false
33034 9996 Acked Write 4 Done Sent Done false
33035 9997 Acked Write 9 Done Sent Done false
33036 9998 Acked Write 7 Done Sent Done false
33037 9999 Acked Write 6 Done Sent Done false
33038 10000 Acked Write 1 Done Sent Done false
33039 10001 Acked Read 9 Done Sent Done false
33040 10002 Acked Write 4 Done Sent Done false
33041 10003 Acked Write 7 Done Sent Done false
33042 10004 Acked Read 5 Done Sent Done false
33043 10005 Acked Read 10 Done Sent Done false
33044 10006 Acked Write 5 Done Sent Done false
33045 10007 Acked Write 4 Done Sent Done false
33046 10008 Acked Write 5 Done Sent Done false
33047 10009 Acked Read 2 Done Sent Done false
33048 10010 Acked Read 9 Done Sent Done false
33049 10011 Acked Write 9 Done Sent Done false
33050 10012 Acked Write 4 Done Sent Done false
33051 10013 Acked Read 5 Done Sent Done false
33052 10014 Acked Read 1 Done Sent Done false
33053 10015 Acked Flush 0 Done Sent Done false
33054 10016 Acked Write 9 Done Sent Done false
33055 10017 Acked Read 2 Done Sent Done false
33056 10018 Acked Read 4 Done Sent Done false
33057 10019 Acked Write 5 Done Sent Done false
33058 10020 Acked Flush 0 Done Sent Done false
33059 10021 Acked Read 1 Done Sent Done false
33060 10022 Acked Write 3 Done Sent Done false
33061 10023 Acked Write 9 Done Sent Done false
33062 10024 Acked Write 10 Done Sent Done false
33063 10025 Acked Write 6 Done Sent Done false
33064 10026 Acked Read 8 Done Sent Done false
33065 10027 Acked Write 4 Done Sent Done false
33066 10028 Acked Flush 0 Done Sent Done false
33067 10029 Acked Flush 0 Done Sent Done false
33068 10030 Acked Write 3 Done Sent Done false
33069 10031 Acked Read 6 Done Sent Done false
33070 10032 Acked Flush 0 Done Sent Done false
33071 10033 Acked Read 1 Done Sent Done false
33072 10034 Acked Read 10 Done Sent Done false
33073 10035 Acked Write 3 Done Sent Done false
33074 10036 Acked Read 7 Done Sent Done false
33075 10037 Acked Read 8 Done Sent Done false
33076 10038 Acked Read 7 Done Sent Done false
33077 10039 Acked Flush 0 Done Sent Done false
33078 10040 Acked Read 5 Done Sent Done false
33079 10041 Acked Write 7 Done Sent Done false
33080 10042 Acked Write 9 Done Sent Done false
33081 10043 Acked Read 8 Done Sent Done false
33082 10044 Acked Read 8 Done Sent Done false
33083 10045 Acked Read 10 Done Sent Done false
33084 10046 Acked Flush 0 Done Sent Done false
33085 10047 Acked Read 2 Done Sent Done false
33086 10048 Acked Read 3 Done Sent Done false
33087 10049 Acked Flush 0 Done Sent Done false
33088 10050 Acked Write 8 Done Sent Done false
33089 10051 Acked Write 4 Done Sent Done false
33090 10052 Acked Write 1 Done Sent Done false
33091 10053 Acked Read 6 Done Sent Done false
33092 10054 Acked Read 5 Done Sent Done false
33093 10055 Acked Read 9 Done Sent Done false
33094 10056 Acked Read 5 Done Sent Done false
33095 10057 Acked Flush 0 Done Sent Done false
33096 STATES DS:0 DS:1 DS:2 TOTAL
33097 Sent 0 302 0 302
33098 Done 302 0 302 604
33099 Skipped 0 0 0 0
33100 Error 0 0 0 0
33101 Last Flush: 10057 9755 10057
33102 Downstairs last five completed: 9755 9754 9753 9752 9751
33103 Upstairs last five completed: 10057 10056 10055 10054 10053
33104 CLIENT: Up:0 ds:302 act:2
331052025-01-21T15:54:08.545ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331062025-01-21T15:54:08.545ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64703) = io task client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331072025-01-21T15:54:08.546ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331082025-01-21T15:54:08.547ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331092025-01-21T15:54:08.547ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331102025-01-21T15:54:08.547ZINFOcrucible: send last flush ID to this DS: 9755 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331112025-01-21T15:54:08.547ZINFOcrucible: Replied this last flush ID: 9755 = downstairs client = 1 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331122025-01-21T15:54:08.547ZINFOcrucible: [1] client re-new 302 jobs since flush 9755 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331132025-01-21T15:54:08.547ZINFOcrucible: [1] Replayed 302 jobs since flush: 9755 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
33114 ----------------------------------------------------------------
33115 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33116 Downstairs last five completed: 10057 10056 10055 10054 10053
33117 Upstairs last five completed: 10057 10056 10055 10054 10053
33118 CLIENT: Up:0 ds:0 act:3
33119 Replay: All jobs finished
33120 Main task has work to do, go find it
33121 got dsc Stop(2)
33122 stop 2
33123 [8830] Got stop action so:false kr:false
33124 Jan 21 15:54:08.799 INFO request completed, latency_us: 389, response_code: 204, uri: /stop/cid/2, method: POST, req_id: bb296080-3ed9-43ad-9f11-907c1b689db7, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
331252025-01-21T15:54:08.800ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331262025-01-21T15:54:08.800ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331272025-01-21T15:54:08.800ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
331282025-01-21T15:54:08.800ZINFOcrucible: client task is exiting = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
33129 Jan 21 15:54:08.800 INFO request completed, latency_us: 166, response_code: 200, uri: /state/cid/2, method: GET, req_id: 965611c9-62ec-4928-824b-898eaed6c021, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
33130 [8830] Exited with: ExitStatus(unix_wait_status(9))
33131 [8830][2] reports Exit
33132 Jan 21 15:54:12.802 INFO request completed, latency_us: 216, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8e5f118e-e37b-4809-b619-a364b4f4d245, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
33133 001/300 Write block 3531 len 40960 data: 6 5 4 3 3 6 6 6 6 7
33134 002/300 Write block 4560 len 4096 data: 5
33135 003/300 Write block 4963 len 16384 data: 3 2 3 3
33136 004/300 Write block 1748 len 40960 data: 5 6 6 7 8 8 8 7 7 9
33137 005/300 Read block 1926 len 28672
33138 006/300 Write block 4534 len 4096 data: 6
33139 007/300 Read block 2637 len 16384
33140 008/300 Write block 2859 len 24576 data: 7 7 6 6 6 4
33141 009/300 Write block 1838 len 40960 data: 6 7 7 7 7 9 11 10 9 10
33142 010/300 Write block 4318 len 36864 data: 7 6 7 6 5 6 6 6 7
33143 011/300 Read block 2640 len 20480
33144 012/300 Write block 3801 len 20480 data: 5 5 4 4 4
33145 013/300 Write block 2929 len 20480 data: 8 6 6 6 6
33146 014/300 Read block 45 len 12288
33147 015/300 Read block 319 len 20480
33148 016/300 Read block 4114 len 24576
33149 017/300 Read block 3032 len 36864
33150 018/300 Flush
33151 019/300 Write block 1628 len 40960 data: 5 5 4 4 3 3 3 3 4 4
33152 020/300 Write block 3213 len 28672 data: 4 6 5 5 4 4 5
33153 021/300 Read block 1929 len 28672
33154 022/300 Write block 326 len 32768 data: 5 4 4 5 4 3 4 4
33155 023/300 Read block 458 len 8192
33156 024/300 Write block 686 len 36864 data: 6 7 8 8 7 5 5 6 7
33157 025/300 Read block 1118 len 28672
33158 026/300 Write block 783 len 4096 data: 5
33159 027/300 Read block 3067 len 40960
33160 028/300 Read block 2582 len 8192
33161 029/300 Read block 3271 len 16384
33162 030/300 Read block 1589 len 20480
33163 031/300 Write block 3794 len 16384 data: 10 8 8 6
33164 032/300 Write block 1248 len 4096 data: 7
33165 033/300 Read block 1740 len 8192
33166 034/300 Read block 4296 len 12288
33167 035/300 Write block 3594 len 16384 data: 5 5 6 7
33168 036/300 Read block 562 len 28672
33169 037/300 Read block 1453 len 24576
33170 038/300 Write block 100 len 24576 data: 5 6 6 7 7 7
33171 039/300 Write block 229 len 32768 data: 11 10 10 9 9 10 10 8
33172 040/300 Read block 4482 len 32768
33173 041/300 Read block 4402 len 36864
33174 042/300 Read block 4780 len 40960
33175 043/300 Read block 3149 len 40960
33176 044/300 Write block 2543 len 40960 data: 5 6 5 6 6 7 4 3 3 3
33177 045/300 Write block 1984 len 32768 data: 12 13 13 10 10 10 9 8
33178 046/300 Read block 4753 len 4096
33179 047/300 Read block 2267 len 8192
33180 048/300 Write block 11 len 40960 data: 4 4 4 5 7 3 4 6 5 5
33181 049/300 Write block 992 len 40960 data: 7 8 8 10 10 9 8 9 10 9
33182 050/300 Read block 2497 len 20480
33183 051/300 Write block 1621 len 8192 data: 8 8
33184 052/300 Write block 3122 len 16384 data: 3 2 5 4
33185 053/300 Read block 4702 len 32768
33186 054/300 Read block 42 len 16384
33187 055/300 Write block 3332 len 36864 data: 4 4 4 5 6 7 7 6 6
33188 056/300 Read block 1098 len 36864
33189 057/300 Write block 744 len 20480 data: 6 4 7 7 7
33190 058/300 Write block 1923 len 36864 data: 6 5 5 4 4 4 3 3 4
33191 059/300 Flush
33192 060/300 Read block 3836 len 16384
33193 061/300 Read block 4476 len 32768
33194 062/300 Write block 3056 len 8192 data: 5 5
33195 063/300 Read block 3542 len 4096
33196 064/300 Read block 2430 len 20480
33197 065/300 Write block 670 len 24576 data: 5 6 7 6 6 7
33198 066/300 Read block 379 len 20480
33199 067/300 Read block 144 len 32768
33200 068/300 Write block 3314 len 36864 data: 6 6 5 5 5 4 4 4 3
33201 069/300 Read block 1176 len 12288
33202 070/300 Write block 522 len 12288 data: 6 7 7
33203 071/300 Read block 3962 len 16384
33204 072/300 Read block 512 len 40960
33205 073/300 Read block 3416 len 4096
33206 074/300 Write block 553 len 8192 data: 5 5
33207 075/300 Write block 4169 len 20480 data: 6 7 8 8 8
33208 076/300 Read block 774 len 8192
33209 077/300 Write block 1272 len 8192 data: 4 4
33210 078/300 Read block 3419 len 36864
33211 079/300 Write block 3902 len 20480 data: 5 6 6 6 6
33212 080/300 Read block 2263 len 4096
33213 081/300 Read block 2189 len 24576
33214 082/300 Read block 1950 len 4096
33215 083/300 Write block 4508 len 12288 data: 5 4 4
33216 084/300 Write block 3748 len 28672 data: 8 7 7 6 3 4 4
33217 085/300 Read block 2082 len 4096
33218 086/300 Flush
33219 087/300 Write block 3584 len 16384 data: 4 5 5 5
33220 088/300 Read block 1780 len 8192
33221 089/300 Write block 4682 len 20480 data: 6 6 4 4 2
33222 090/300 Write block 1538 len 24576 data: 6 6 9 7 10 10
33223 091/300 Flush
33224 092/300 Read block 4515 len 32768
33225 093/300 Read block 4373 len 12288
33226 094/300 Write block 4450 len 8192 data: 8 6
33227 095/300 Write block 3590 len 24576 data: 5 4 3 4 6 6
33228 096/300 Write block 3600 len 4096 data: 8
33229 097/300 Read block 4123 len 40960
33230 098/300 Read block 3322 len 16384
33231 099/300 Write block 2200 len 36864 data: 6 5 3 4 4 4 4 4 3
33232 100/300 Write block 3663 len 32768 data: 4 6 6 6 6 8 9 7
33233 101/300 Flush
33234 102/300 Read block 2131 len 8192
33235 103/300 Write block 1351 len 4096 data: 5
33236 104/300 Read block 3984 len 20480
33237 105/300 Write block 4091 len 4096 data: 7
33238 106/300 Write block 1851 len 32768 data: 7 5 3 3 5 6 6 4
33239 107/300 Write block 3592 len 28672 data: 4 5 7 7 7 8 7
33240 108/300 Write block 950 len 40960 data: 4 3 3 4 5 3 3 3 3 3
33241 109/300 Write block 4630 len 40960 data: 5 6 8 7 5 5 5 6 7 6
33242 110/300 Read block 2135 len 24576
33243 111/300 Read block 4232 len 16384
33244 112/300 Read block 4318 len 40960
33245 113/300 Read block 3840 len 12288
33246 114/300 Read block 2679 len 16384
33247 115/300 Write block 3038 len 8192 data: 8 8
33248 116/300 Read block 910 len 16384
33249 117/300 Read block 805 len 40960
33250 118/300 Write block 3853 len 32768 data: 9 10 11 11 10 11 8 7
33251 119/300 Read block 2581 len 16384
33252 120/300 Write block 571 len 40960 data: 4 6 7 7 7 6 6 8 6 7
33253 121/300 Read block 4162 len 4096
33254 122/300 Read block 682 len 24576
33255 123/300 Read block 1720 len 20480
33256 124/300 Read block 3727 len 40960
33257 125/300 Flush
33258 126/300 Read block 2549 len 40960
33259 127/300 Read block 891 len 32768
33260 128/300 Flush
33261 129/300 Read block 279 len 32768
33262 130/300 Flush
33263 131/300 Write block 3928 len 16384 data: 8 8 8 7
33264 132/300 Write block 2111 len 16384 data: 4 4 5 5
33265 133/300 Read block 3354 len 8192
33266 134/300 Read block 3540 len 24576
33267 135/300 Read block 2863 len 40960
33268 136/300 Read block 48 len 32768
33269 137/300 Read block 885 len 24576
33270 138/300 Write block 1142 len 8192 data: 6 6
33271 139/300 Write block 3822 len 20480 data: 11 9 7 7 5
33272 140/300 Read block 666 len 40960
33273 141/300 Write block 4398 len 4096 data: 12
33274 142/300 Write block 390 len 8192 data: 5 4
33275 143/300 Write block 1435 len 24576 data: 6 5 5 6 7 7
33276 144/300 Read block 3613 len 16384
33277 145/300 Read block 1711 len 20480
33278 146/300 Write block 1997 len 24576 data: 6 6 6 6 3 3
33279 147/300 Read block 2123 len 16384
33280 148/300 Write block 417 len 32768 data: 6 6 5 3 3 4 4 3
33281 149/300 Read block 2154 len 8192
33282 150/300 Read block 2491 len 8192
33283 151/300 Read block 2000 len 36864
33284 152/300 Flush
33285 153/300 Flush
33286 154/300 Write block 74 len 20480 data: 8 6 4 7 8
33287 155/300 Write block 4107 len 16384 data: 7 7 6 7
33288 156/300 Write block 1564 len 12288 data: 5 5 7
33289 157/300 Write block 3390 len 24576 data: 7 7 6 6 5 5
33290 158/300 Write block 3465 len 16384 data: 7 8 7 10
33291 159/300 Write block 3942 len 36864 data: 5 5 6 6 7 7 9 7 7
33292 160/300 Write block 507 len 32768 data: 7 7 7 7 5 5 6 6
33293 161/300 Read block 753 len 40960
33294 162/300 Read block 373 len 4096
33295 163/300 Write block 1935 len 12288 data: 7 7 8
33296 164/300 Read block 3314 len 16384
33297 165/300 Flush
33298 166/300 Read block 2368 len 32768
33299 167/300 Write block 4701 len 32768 data: 7 8 9 9 7 7 7 6
33300 168/300 Write block 2994 len 32768 data: 7 5 6 5 4 3 3 3
33301 169/300 Read block 2256 len 40960
33302 170/300 Write block 1543 len 8192 data: 11 8
33303 171/300 Read block 3415 len 36864
33304 172/300 Write block 1712 len 12288 data: 8 7 6
33305 173/300 Read block 4202 len 32768
33306 174/300 Write block 2631 len 20480 data: 9 9 8 8 8
33307 175/300 Read block 1389 len 36864
33308 176/300 Write block 401 len 12288 data: 5 5 6
33309 177/300 Read block 4258 len 12288
33310 178/300 Write block 3488 len 32768 data: 7 6 7 8 7 9 8 4
33311 179/300 Write block 283 len 20480 data: 5 7 6 6 5
33312 180/300 Read block 3689 len 20480
33313 181/300 Read block 1933 len 4096
33314 182/300 Read block 2419 len 28672
33315 183/300 Read block 2557 len 28672
33316 184/300 Read block 3302 len 40960
33317 185/300 Flush
33318 186/300 Write block 2055 len 4096 data: 8
33319 187/300 Read block 4334 len 36864
33320 188/300 Write block 3769 len 8192 data: 7 6
33321 189/300 Read block 2381 len 36864
33322 190/300 Read block 2690 len 32768
33323 191/300 Write block 4746 len 20480 data: 5 5 5 5 7
33324 192/300 Flush
33325 193/300 Read block 3216 len 12288
33326 194/300 Read block 1298 len 28672
33327 195/300 Read block 2154 len 32768
33328 196/300 Read block 4051 len 24576
33329 197/300 Read block 3620 len 40960
33330 198/300 Read block 3102 len 20480
33331 199/300 Read block 2904 len 8192
33332 200/300 Read block 214 len 36864
33333 201/300 Read block 464 len 32768
33334 202/300 Read block 3985 len 20480
33335 203/300 Flush
33336 204/300 Write block 438 len 4096 data: 6
33337 205/300 Read block 1584 len 32768
33338 206/300 Flush
33339 207/300 Write block 2740 len 20480 data: 3 3 5 4 4
33340 208/300 Read block 3795 len 28672
33341 209/300 Read block 3477 len 12288
33342 210/300 Write block 4599 len 16384 data: 5 4 4 5
33343 211/300 Read block 2472 len 24576
33344 212/300 Write block 692 len 8192 data: 6 7
33345 213/300 Write block 4393 len 36864 data: 7 7 9 10 12 13 11 11 6
33346 214/300 Write block 2113 len 28672 data: 6 6 5 6 6 5 5
33347 215/300 Read block 2256 len 24576
33348 216/300 Read block 2255 len 20480
33349 217/300 Write block 2541 len 12288 data: 5 5 6
33350 218/300 Read block 1266 len 4096
33351 219/300 Flush
33352 220/300 Write block 4099 len 16384 data: 11 11 9 8
33353 221/300 Flush
33354 222/300 Flush
33355 223/300 Read block 3987 len 36864
33356 224/300 Read block 1273 len 20480
33357 225/300 Read block 4636 len 32768
33358 226/300 Read block 1793 len 28672
33359 227/300 Write block 1492 len 24576 data: 4 5 6 5 6 7
33360 228/300 Write block 4166 len 40960 data: 5 4 6 7 8 9 9 9 7 6
33361 229/300 Read block 4373 len 12288
33362 230/300 Read block 1612 len 32768
33363 231/300 Write block 1652 len 32768 data: 6 8 8 6 7 6 10 10
33364 232/300 Read block 2425 len 24576
33365 233/300 Read block 3454 len 8192
33366 234/300 Write block 3571 len 8192 data: 8 7
33367 235/300 Write block 4201 len 8192 data: 5 5
33368 236/300 Read block 4113 len 8192
33369 237/300 Write block 2278 len 8192 data: 5 5
33370 238/300 Read block 137 len 8192
33371 239/300 Read block 2172 len 4096
33372 240/300 Write block 2548 len 8192 data: 8 5
33373 241/300 Read block 3156 len 40960
33374 242/300 Read block 151 len 32768
33375 243/300 Write block 2524 len 4096 data: 8
33376 244/300 Read block 1667 len 20480
33377 245/300 Write block 1944 len 24576 data: 7 7 6 5 6 6
33378 246/300 Read block 4118 len 36864
33379 247/300 Read block 859 len 36864
33380 248/300 Write block 1832 len 36864 data: 6 6 6 6 5 7 7 8 8
33381 249/300 Read block 1237 len 20480
33382 250/300 Write block 3034 len 28672 data: 9 8 8 7 9 9 7
33383 251/300 Write block 981 len 4096 data: 4
33384 252/300 Read block 792 len 8192
33385 253/300 Read block 2033 len 40960
33386 254/300 Write block 1269 len 4096 data: 5
33387 255/300 Write block 2568 len 16384 data: 6 8 7 4
33388 256/300 Flush
33389 257/300 Write block 3699 len 4096 data: 7
33390 258/300 Write block 2714 len 28672 data: 9 7 6 7 7 6 5
33391 259/300 Write block 428 len 28672 data: 5 5 6 6 6 5 6
33392 260/300 Write block 2083 len 4096 data: 7
33393 261/300 Read block 1085 len 40960
33394 262/300 Write block 98 len 16384 data: 4 3 6 7
33395 263/300 Read block 187 len 36864
33396 264/300 Read block 973 len 8192
33397 265/300 Read block 4657 len 32768
33398 266/300 Read block 1940 len 36864
33399 267/300 Read block 1109 len 40960
33400 268/300 Read block 4277 len 32768
33401 269/300 Read block 3583 len 8192
33402 270/300 Flush
33403 271/300 Read block 4802 len 4096
33404 272/300 Flush
33405 273/300 Flush
33406 274/300 Read block 3109 len 20480
33407 275/300 Write block 1646 len 24576 data: 5 5 6 6 6 5
33408 276/300 Read block 3168 len 20480
33409 277/300 Write block 1582 len 16384 data: 7 7 6 6
33410 278/300 Write block 879 len 8192 data: 3 3
33411 279/300 Flush
33412 280/300 Read block 469 len 16384
33413 281/300 Read block 4541 len 28672
33414 282/300 Read block 4006 len 4096
33415 283/300 Read block 907 len 32768
33416 284/300 Read block 1821 len 12288
33417 285/300 Read block 365 len 16384
33418 286/300 Read block 1450 len 36864
33419 287/300 Read block 3284 len 8192
33420 288/300 Flush
33421 289/300 Write block 5 len 32768 data: 4 4 4 4 4 5 5 5
33422 290/300 Write block 2431 len 36864 data: 8 7 8 8 6 6 6 5 3
33423 291/300 Read block 1652 len 4096
33424 292/300 Read block 2591 len 40960
33425 293/300 Flush
33426 294/300 Read block 1151 len 16384
33427 295/300 Read block 1173 len 16384
33428 296/300 Read block 1041 len 40960
33429 297/300 Write block 3154 len 4096 data: 8
33430 298/300 Read block 708 len 20480
33431 299/300 Read block 3569 len 4096
33432 300/300 Read block 126 len 8192
33433 301/300 Write block 3550 len 8192 data: 5 4
33434 Main task has work to do, go find it
33435 got dsc Start(2)
33436 start 2
33437 Jan 21 15:54:13.049 INFO request completed, latency_us: 242, response_code: 204, uri: /start/cid/2, method: POST, req_id: 47cae183-348a-4999-9f33-9c8b2c9a836b, remote_addr: 127.0.0.1:52945, local_addr: 127.0.0.1:9998
33438 Replay: started 2, returned:Ok(())
33439 ----------------------------------------------------------------
33440 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33441 States: Active Active Offline
33442 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33443 10058 Acked Write 10 Done Done Sent false
33444 10059 Acked Write 1 Done Done Sent false
33445 10060 Acked Write 4 Done Done Sent false
33446 10061 Acked Write 10 Done Done Sent false
33447 10062 Acked Read 7 Done Done Sent false
33448 10063 Acked Write 1 Done Done Sent false
33449 10064 Acked Read 4 Done Done Sent false
33450 10065 Acked Write 6 Done Done Sent false
33451 10066 Acked Write 10 Done Done Sent false
33452 10067 Acked Write 9 Done Done Sent false
33453 10068 Acked Read 5 Done Done Sent false
33454 10069 Acked Write 5 Done Done Sent false
33455 10070 Acked Write 5 Done Done Sent false
33456 10071 Acked Read 3 Done Done Sent false
33457 10072 Acked Read 5 Done Done Sent false
33458 10073 Acked Read 6 Done Done Sent false
33459 10074 Acked Read 9 Done Done Sent false
33460 10075 Acked Flush 0 Done Done Sent false
33461 10076 Acked Write 10 Done Done Sent false
33462 10077 Acked Write 7 Done Done Sent false
33463 10078 Acked Read 7 Done Done Sent false
33464 10079 Acked Write 8 Done Done Sent false
33465 10080 Acked Read 2 Done Done Sent false
33466 10081 Acked Write 9 Done Done Sent false
33467 10082 Acked Read 7 Done Done Sent false
33468 10083 Acked Write 1 Done Done Sent false
33469 10084 Acked Read 10 Done Done Sent false
33470 10085 Acked Read 2 Done Done Sent false
33471 10086 Acked Read 4 Done Done Sent false
33472 10087 Acked Read 5 Done Done Sent false
33473 10088 Acked Write 4 Done Done Sent false
33474 10089 Acked Write 1 Done Done Sent false
33475 10090 Acked Read 2 Done Done Sent false
33476 10091 Acked Read 3 Done Done Sent false
33477 10092 Acked Write 4 Done Done Sent false
33478 10093 Acked Read 7 Done Done Sent false
33479 10094 Acked Read 6 Done Done Sent false
33480 10095 Acked Write 6 Done Done Sent false
33481 10096 Acked Write 8 Done Done Sent false
33482 10097 Acked Read 8 Done Done Sent false
33483 10098 Acked Read 9 Done Done Sent false
33484 10099 Acked Read 10 Done Done Sent false
33485 10100 Acked Read 10 Done Done Sent false
33486 10101 Acked Write 10 Done Done Sent false
33487 10102 Acked Write 8 Done Done Sent false
33488 10103 Acked Read 1 Done Done Sent false
33489 10104 Acked Read 2 Done Done Sent false
33490 10105 Acked Write 10 Done Done Sent false
33491 10106 Acked Write 10 Done Done Sent false
33492 10107 Acked Read 5 Done Done Sent false
33493 10108 Acked Write 2 Done Done Sent false
33494 10109 Acked Write 4 Done Done Sent false
33495 10110 Acked Read 8 Done Done Sent false
33496 10111 Acked Read 4 Done Done Sent false
33497 10112 Acked Write 9 Done Done Sent false
33498 10113 Acked Read 9 Done Done Sent false
33499 10114 Acked Write 5 Done Done Sent false
33500 10115 Acked Write 9 Done Done Sent false
33501 10116 Acked Flush 0 Done Done Sent false
33502 10117 Acked Read 4 Done Done Sent false
33503 10118 Acked Read 8 Done Done Sent false
33504 10119 Acked Write 2 Done Done Sent false
33505 10120 Acked Read 1 Done Done Sent false
33506 10121 Acked Read 5 Done Done Sent false
33507 10122 Acked Write 6 Done Done Sent false
33508 10123 Acked Read 5 Done Done Sent false
33509 10124 Acked Read 8 Done Done Sent false
33510 10125 Acked Write 9 Done Done Sent false
33511 10126 Acked Read 3 Done Done Sent false
33512 10127 Acked Write 3 Done Done Sent false
33513 10128 Acked Read 4 Done Done Sent false
33514 10129 Acked Read 10 Done Done Sent false
33515 10130 Acked Read 1 Done Done Sent false
33516 10131 Acked Write 2 Done Done Sent false
33517 10132 Acked Write 5 Done Done Sent false
33518 10133 Acked Read 2 Done Done Sent false
33519 10134 Acked Write 2 Done Done Sent false
33520 10135 Acked Read 9 Done Done Sent false
33521 10136 Acked Write 5 Done Done Sent false
33522 10137 Acked Read 1 Done Done Sent false
33523 10138 Acked Read 6 Done Done Sent false
33524 10139 Acked Read 1 Done Done Sent false
33525 10140 Acked Write 3 Done Done Sent false
33526 10141 Acked Write 7 Done Done Sent false
33527 10142 Acked Read 1 Done Done Sent false
33528 10143 Acked Flush 0 Done Done Sent false
33529 10144 Acked Write 4 Done Done Sent false
33530 10145 Acked Read 2 Done Done Sent false
33531 10146 Acked Write 5 Done Done Sent false
33532 10147 Acked Write 6 Done Done Sent false
33533 10148 Acked Flush 0 Done Done Sent false
33534 10149 Acked Read 8 Done Done Sent false
33535 10150 Acked Read 3 Done Done Sent false
33536 10151 Acked Write 2 Done Done Sent false
33537 10152 Acked Write 6 Done Done Sent false
33538 10153 Acked Write 1 Done Done Sent false
33539 10154 Acked Read 10 Done Done Sent false
33540 10155 Acked Read 4 Done Done Sent false
33541 10156 Acked Write 9 Done Done Sent false
33542 10157 Acked Write 8 Done Done Sent false
33543 10158 Acked Flush 0 Done Done Sent false
33544 10159 Acked Read 2 Done Done Sent false
33545 10160 Acked Write 1 Done Done Sent false
33546 10161 Acked Read 5 Done Done Sent false
33547 10162 Acked Write 1 Done Done Sent false
33548 10163 Acked Write 8 Done Done Sent false
33549 10164 Acked Write 7 Done Done Sent false
33550 10165 Acked Write 10 Done Done Sent false
33551 10166 Acked Write 10 Done Done Sent false
33552 10167 Acked Read 6 Done Done Sent false
33553 10168 Acked Read 4 Done Done Sent false
33554 10169 Acked Read 10 Done Done Sent false
33555 10170 Acked Read 3 Done Done Sent false
33556 10171 Acked Read 4 Done Done Sent false
33557 10172 Acked Write 2 Done Done Sent false
33558 10173 Acked Read 4 Done Done Sent false
33559 10174 Acked Read 10 Done Done Sent false
33560 10175 Acked Write 8 Done Done Sent false
33561 10176 Acked Read 4 Done Done Sent false
33562 10177 Acked Write 10 Done Done Sent false
33563 10178 Acked Read 1 Done Done Sent false
33564 10179 Acked Read 6 Done Done Sent false
33565 10180 Acked Read 5 Done Done Sent false
33566 10181 Acked Read 10 Done Done Sent false
33567 10182 Acked Flush 0 Done Done Sent false
33568 10183 Acked Read 10 Done Done Sent false
33569 10184 Acked Read 8 Done Done Sent false
33570 10185 Acked Flush 0 Done Done Sent false
33571 10186 Acked Read 8 Done Done Sent false
33572 10187 Acked Flush 0 Done Done Sent false
33573 10188 Acked Write 4 Done Done Sent false
33574 10189 Acked Write 4 Done Done Sent false
33575 10190 Acked Read 2 Done Done Sent false
33576 10191 Acked Read 6 Done Done Sent false
33577 10192 Acked Read 10 Done Done Sent false
33578 10193 Acked Read 8 Done Done Sent false
33579 10194 Acked Read 6 Done Done Sent false
33580 10195 Acked Write 2 Done Done Sent false
33581 10196 Acked Write 5 Done Done Sent false
33582 10197 Acked Read 10 Done Done Sent false
33583 10198 Acked Write 1 Done Done Sent false
33584 10199 Acked Write 2 Done Done Sent false
33585 10200 Acked Write 6 Done Done Sent false
33586 10201 Acked Read 4 Done Done Sent false
33587 10202 Acked Read 5 Done Done Sent false
33588 10203 Acked Write 6 Done Done Sent false
33589 10204 Acked Read 4 Done Done Sent false
33590 10205 Acked Write 8 Done Done Sent false
33591 10206 Acked Read 2 Done Done Sent false
33592 10207 Acked Read 2 Done Done Sent false
33593 10208 Acked Read 9 Done Done Sent false
33594 10209 Acked Flush 0 Done Done Sent false
33595 10210 Acked Flush 0 Done Done Sent false
33596 10211 Acked Write 5 Done Done Sent false
33597 10212 Acked Write 4 Done Done Sent false
33598 10213 Acked Write 3 Done Done Sent false
33599 10214 Acked Write 6 Done Done Sent false
33600 10215 Acked Write 4 Done Done Sent false
33601 10216 Acked Write 9 Done Done Sent false
33602 10217 Acked Write 8 Done Done Sent false
33603 10218 Acked Read 10 Done Done Sent false
33604 10219 Acked Read 1 Done Done Sent false
33605 10220 Acked Write 3 Done Done Sent false
33606 10221 Acked Read 4 Done Done Sent false
33607 10222 Acked Flush 0 Done Done Sent false
33608 10223 Acked Read 8 Done Done Sent false
33609 10224 Acked Write 8 Done Done Sent false
33610 10225 Acked Write 8 Done Done Sent false
33611 10226 Acked Read 10 Done Done Sent false
33612 10227 Acked Write 2 Done Done Sent false
33613 10228 Acked Read 9 Done Done Sent false
33614 10229 Acked Write 3 Done Done Sent false
33615 10230 Acked Read 8 Done Done Sent false
33616 10231 Acked Write 5 Done Done Sent false
33617 10232 Acked Read 9 Done Done Sent false
33618 10233 Acked Write 3 Done Done Sent false
33619 10234 Acked Read 3 Done Done Sent false
33620 10235 Acked Write 8 Done Done Sent false
33621 10236 Acked Write 5 Done Done Sent false
33622 10237 Acked Read 5 Done Done Sent false
33623 10238 Acked Read 1 Done Done Sent false
33624 10239 Acked Read 7 Done Done Sent false
33625 10240 Acked Read 7 Done Done Sent false
33626 10241 Acked Read 10 Done Done Sent false
33627 10242 Acked Flush 0 Done Done Sent false
33628 10243 Acked Write 1 Done Done Sent false
33629 10244 Acked Read 9 Done Done Sent false
33630 10245 Acked Write 2 Done Done Sent false
33631 10246 Acked Read 9 Done Done Sent false
33632 10247 Acked Read 8 Done Done Sent false
33633 10248 Acked Write 5 Done Done Sent false
33634 10249 Acked Flush 0 Done Done Sent false
33635 10250 Acked Read 3 Done Done Sent false
33636 10251 Acked Read 7 Done Done Sent false
33637 10252 Acked Read 8 Done Done Sent false
33638 10253 Acked Read 6 Done Done Sent false
33639 10254 Acked Read 10 Done Done Sent false
33640 10255 Acked Read 5 Done Done Sent false
33641 10256 Acked Read 2 Done Done Sent false
33642 10257 Acked Read 9 Done Done Sent false
33643 10258 Acked Read 8 Done Done Sent false
33644 10259 Acked Read 5 Done Done Sent false
33645 10260 Acked Flush 0 Done Done Sent false
33646 10261 Acked Write 1 Done Done Sent false
33647 10262 Acked Read 8 Done Done Sent false
33648 10263 Acked Flush 0 Done Done Sent false
33649 10264 Acked Write 5 Done Done Sent false
33650 10265 Acked Read 7 Done Done Sent false
33651 10266 Acked Read 3 Done Done Sent false
33652 10267 Acked Write 4 Done Done Sent false
33653 10268 Acked Read 6 Done Done Sent false
33654 10269 Acked Write 2 Done Done Sent false
33655 10270 Acked Write 9 Done Done Sent false
33656 10271 Acked Write 7 Done Done Sent false
33657 10272 Acked Read 6 Done Done Sent false
33658 10273 Acked Read 5 Done Done Sent false
33659 10274 Acked Write 3 Done Done Sent false
33660 10275 Acked Read 1 Done Done Sent false
33661 10276 Acked Flush 0 Done Done Sent false
33662 10277 Acked Write 4 Done Done Sent false
33663 10278 Acked Flush 0 Done Done Sent false
33664 10279 Acked Flush 0 Done Done Sent false
33665 10280 Acked Read 9 Done Done Sent false
33666 10281 Acked Read 5 Done Done Sent false
33667 10282 Acked Read 8 Done Done Sent false
33668 10283 Acked Read 7 Done Done Sent false
33669 10284 Acked Write 6 Done Done Sent false
33670 10285 Acked Write 10 Done Done Sent false
33671 10286 Acked Read 3 Done Done Sent false
33672 10287 Acked Read 8 Done Done Sent false
33673 10288 Acked Write 8 Done Done Sent false
33674 10289 Acked Read 6 Done Done Sent false
33675 10290 Acked Read 2 Done Done Sent false
33676 10291 Acked Write 2 Done Done Sent false
33677 10292 Acked Write 2 Done Done Sent false
33678 10293 Acked Read 2 Done Done Sent false
33679 10294 Acked Write 2 Done Done Sent false
33680 10295 Acked Read 2 Done Done Sent false
33681 10296 Acked Read 1 Done Done Sent false
33682 10297 Acked Write 2 Done Done Sent false
33683 10298 Acked Read 10 Done Done Sent false
33684 10299 Acked Read 8 Done Done Sent false
33685 10300 Acked Write 1 Done Done Sent false
33686 10301 Acked Read 5 Done Done Sent false
33687 10302 Acked Write 6 Done Done Sent false
33688 10303 Acked Read 9 Done Done Sent false
33689 10304 Acked Read 9 Done Done Sent false
33690 10305 Acked Write 9 Done Done Sent false
33691 10306 Acked Read 5 Done Done Sent false
33692 10307 Acked Write 7 Done Done Sent false
33693 10308 Acked Write 1 Done Done Sent false
33694 10309 Acked Read 2 Done Done Sent false
33695 10310 Acked Read 10 Done Done Sent false
33696 10311 Acked Write 1 Done Done Sent false
33697 10312 Acked Write 4 Done Done Sent false
33698 10313 Acked Flush 0 Done Done Sent false
33699 10314 Acked Write 1 Done Done Sent false
33700 10315 Acked Write 7 Done Done Sent false
33701 10316 Acked Write 7 Done Done Sent false
33702 10317 Acked Write 1 Done Done Sent false
33703 10318 Acked Read 10 Done Done Sent false
33704 10319 Acked Write 4 Done Done Sent false
33705 10320 Acked Read 9 Done Done Sent false
33706 10321 Acked Read 2 Done Done Sent false
33707 10322 Acked Read 8 Done Done Sent false
33708 10323 Acked Read 9 Done Done Sent false
33709 10324 Acked Read 10 Done Done Sent false
33710 10325 Acked Read 8 Done Done Sent false
33711 10326 Acked Read 2 Done Done Sent false
33712 10327 Acked Flush 0 Done Done Sent false
33713 10328 Acked Read 1 Done Done Sent false
33714 10329 Acked Flush 0 Done Done Sent false
33715 10330 Acked Flush 0 Done Done Sent false
33716 10331 Acked Read 5 Done Done Sent false
33717 10332 Acked Write 6 Done Done Sent false
33718 10333 Acked Read 5 Done Done Sent false
33719 10334 Acked Write 4 Done Done Sent false
33720 10335 Acked Write 2 Done Done Sent false
33721 10336 Acked Flush 0 Done Done Sent false
33722 10337 Acked Read 4 Done Done Sent false
33723 10338 Acked Read 7 Done Done Sent false
33724 10339 Acked Read 1 Done Done Sent false
33725 10340 Acked Read 8 Done Done Sent false
33726 10341 Acked Read 3 Done Done Sent false
33727 10342 Acked Read 4 Done Done Sent false
33728 10343 Acked Read 9 Done Done Sent false
33729 10344 Acked Read 2 Done Done Sent false
33730 10345 Acked Flush 0 Done Done Sent false
33731 10346 Acked Write 8 Done Done Sent false
33732 10347 Acked Write 9 Done Done Sent false
33733 10348 Acked Read 1 Done Done Sent false
33734 10349 Acked Read 10 Done Done Sent false
33735 10350 Acked Flush 0 Done Done Sent false
33736 10351 Acked Read 4 Done Done Sent false
33737 10352 Acked Read 4 Done Done Sent false
33738 10353 Acked Read 10 Done Done Sent false
33739 10354 Acked Write 1 Done Done Sent false
33740 10355 Acked Read 5 Done Done Sent false
33741 10356 Acked Read 1 Done Done Sent false
33742 10357 Acked Read 2 Done Done Sent false
33743 10358 Acked Write 2 Done Done Sent false
33744 STATES DS:0 DS:1 DS:2 TOTAL
33745 Sent 0 0 301 301
33746 Done 301 301 0 602
33747 Skipped 0 0 0 0
33748 Error 0 0 0 0
33749 Last Flush: 10350 10350 10057
33750 Downstairs last five completed: 10057 10056 10055 10054 10053
33751 Upstairs last five completed: 10358 10357 10356 10355 10354
33752 CLIENT: Up:0 ds:301 act:2
33753 [8830] got start action
33754 [8830] I am going to restart
33755 Starting downstairs at port 8830
33756 Make output file at "/tmp/dsc/downstairs-8830.txt"
33757 [8830][2] reports Starting
33758 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1231)
33759 [8830][2] reports Running
33760 ----------------------------------------------------------------
33761 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33762 States: Active Active Offline
33763 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33764 10058 Acked Write 10 Done Done Sent false
33765 10059 Acked Write 1 Done Done Sent false
33766 10060 Acked Write 4 Done Done Sent false
33767 10061 Acked Write 10 Done Done Sent false
33768 10062 Acked Read 7 Done Done Sent false
33769 10063 Acked Write 1 Done Done Sent false
33770 10064 Acked Read 4 Done Done Sent false
33771 10065 Acked Write 6 Done Done Sent false
33772 10066 Acked Write 10 Done Done Sent false
33773 10067 Acked Write 9 Done Done Sent false
33774 10068 Acked Read 5 Done Done Sent false
33775 10069 Acked Write 5 Done Done Sent false
33776 10070 Acked Write 5 Done Done Sent false
33777 10071 Acked Read 3 Done Done Sent false
33778 10072 Acked Read 5 Done Done Sent false
33779 10073 Acked Read 6 Done Done Sent false
33780 10074 Acked Read 9 Done Done Sent false
33781 10075 Acked Flush 0 Done Done Sent false
33782 10076 Acked Write 10 Done Done Sent false
33783 10077 Acked Write 7 Done Done Sent false
33784 10078 Acked Read 7 Done Done Sent false
33785 10079 Acked Write 8 Done Done Sent false
33786 10080 Acked Read 2 Done Done Sent false
33787 10081 Acked Write 9 Done Done Sent false
33788 10082 Acked Read 7 Done Done Sent false
33789 10083 Acked Write 1 Done Done Sent false
33790 10084 Acked Read 10 Done Done Sent false
33791 10085 Acked Read 2 Done Done Sent false
33792 10086 Acked Read 4 Done Done Sent false
33793 10087 Acked Read 5 Done Done Sent false
33794 10088 Acked Write 4 Done Done Sent false
33795 10089 Acked Write 1 Done Done Sent false
33796 10090 Acked Read 2 Done Done Sent false
33797 10091 Acked Read 3 Done Done Sent false
33798 10092 Acked Write 4 Done Done Sent false
33799 10093 Acked Read 7 Done Done Sent false
33800 10094 Acked Read 6 Done Done Sent false
33801 10095 Acked Write 6 Done Done Sent false
33802 10096 Acked Write 8 Done Done Sent false
33803 10097 Acked Read 8 Done Done Sent false
33804 10098 Acked Read 9 Done Done Sent false
33805 10099 Acked Read 10 Done Done Sent false
33806 10100 Acked Read 10 Done Done Sent false
33807 10101 Acked Write 10 Done Done Sent false
33808 10102 Acked Write 8 Done Done Sent false
33809 10103 Acked Read 1 Done Done Sent false
33810 10104 Acked Read 2 Done Done Sent false
33811 10105 Acked Write 10 Done Done Sent false
33812 10106 Acked Write 10 Done Done Sent false
33813 10107 Acked Read 5 Done Done Sent false
33814 10108 Acked Write 2 Done Done Sent false
33815 10109 Acked Write 4 Done Done Sent false
33816 10110 Acked Read 8 Done Done Sent false
33817 10111 Acked Read 4 Done Done Sent false
33818 10112 Acked Write 9 Done Done Sent false
33819 10113 Acked Read 9 Done Done Sent false
33820 10114 Acked Write 5 Done Done Sent false
33821 10115 Acked Write 9 Done Done Sent false
33822 10116 Acked Flush 0 Done Done Sent false
33823 10117 Acked Read 4 Done Done Sent false
33824 10118 Acked Read 8 Done Done Sent false
33825 10119 Acked Write 2 Done Done Sent false
33826 10120 Acked Read 1 Done Done Sent false
33827 10121 Acked Read 5 Done Done Sent false
33828 10122 Acked Write 6 Done Done Sent false
33829 10123 Acked Read 5 Done Done Sent false
33830 10124 Acked Read 8 Done Done Sent false
33831 10125 Acked Write 9 Done Done Sent false
33832 10126 Acked Read 3 Done Done Sent false
33833 10127 Acked Write 3 Done Done Sent false
33834 10128 Acked Read 4 Done Done Sent false
33835 10129 Acked Read 10 Done Done Sent false
33836 10130 Acked Read 1 Done Done Sent false
33837 10131 Acked Write 2 Done Done Sent false
33838 10132 Acked Write 5 Done Done Sent false
33839 10133 Acked Read 2 Done Done Sent false
33840 10134 Acked Write 2 Done Done Sent false
33841 10135 Acked Read 9 Done Done Sent false
33842 10136 Acked Write 5 Done Done Sent false
33843 10137 Acked Read 1 Done Done Sent false
33844 10138 Acked Read 6 Done Done Sent false
33845 10139 Acked Read 1 Done Done Sent false
33846 10140 Acked Write 3 Done Done Sent false
33847 10141 Acked Write 7 Done Done Sent false
33848 10142 Acked Read 1 Done Done Sent false
33849 10143 Acked Flush 0 Done Done Sent false
33850 10144 Acked Write 4 Done Done Sent false
33851 10145 Acked Read 2 Done Done Sent false
33852 10146 Acked Write 5 Done Done Sent false
33853 10147 Acked Write 6 Done Done Sent false
33854 10148 Acked Flush 0 Done Done Sent false
33855 10149 Acked Read 8 Done Done Sent false
33856 10150 Acked Read 3 Done Done Sent false
33857 10151 Acked Write 2 Done Done Sent false
33858 10152 Acked Write 6 Done Done Sent false
33859 10153 Acked Write 1 Done Done Sent false
33860 10154 Acked Read 10 Done Done Sent false
33861 10155 Acked Read 4 Done Done Sent false
33862 10156 Acked Write 9 Done Done Sent false
33863 10157 Acked Write 8 Done Done Sent false
33864 10158 Acked Flush 0 Done Done Sent false
33865 10159 Acked Read 2 Done Done Sent false
33866 10160 Acked Write 1 Done Done Sent false
33867 10161 Acked Read 5 Done Done Sent false
33868 10162 Acked Write 1 Done Done Sent false
33869 10163 Acked Write 8 Done Done Sent false
33870 10164 Acked Write 7 Done Done Sent false
33871 10165 Acked Write 10 Done Done Sent false
33872 10166 Acked Write 10 Done Done Sent false
33873 10167 Acked Read 6 Done Done Sent false
33874 10168 Acked Read 4 Done Done Sent false
33875 10169 Acked Read 10 Done Done Sent false
33876 10170 Acked Read 3 Done Done Sent false
33877 10171 Acked Read 4 Done Done Sent false
33878 10172 Acked Write 2 Done Done Sent false
33879 10173 Acked Read 4 Done Done Sent false
33880 10174 Acked Read 10 Done Done Sent false
33881 10175 Acked Write 8 Done Done Sent false
33882 10176 Acked Read 4 Done Done Sent false
33883 10177 Acked Write 10 Done Done Sent false
33884 10178 Acked Read 1 Done Done Sent false
33885 10179 Acked Read 6 Done Done Sent false
33886 10180 Acked Read 5 Done Done Sent false
33887 10181 Acked Read 10 Done Done Sent false
33888 10182 Acked Flush 0 Done Done Sent false
33889 10183 Acked Read 10 Done Done Sent false
33890 10184 Acked Read 8 Done Done Sent false
33891 10185 Acked Flush 0 Done Done Sent false
33892 10186 Acked Read 8 Done Done Sent false
33893 10187 Acked Flush 0 Done Done Sent false
33894 10188 Acked Write 4 Done Done Sent false
33895 10189 Acked Write 4 Done Done Sent false
33896 10190 Acked Read 2 Done Done Sent false
33897 10191 Acked Read 6 Done Done Sent false
33898 10192 Acked Read 10 Done Done Sent false
33899 10193 Acked Read 8 Done Done Sent false
33900 10194 Acked Read 6 Done Done Sent false
33901 10195 Acked Write 2 Done Done Sent false
33902 10196 Acked Write 5 Done Done Sent false
33903 10197 Acked Read 10 Done Done Sent false
33904 10198 Acked Write 1 Done Done Sent false
33905 10199 Acked Write 2 Done Done Sent false
33906 10200 Acked Write 6 Done Done Sent false
33907 10201 Acked Read 4 Done Done Sent false
33908 10202 Acked Read 5 Done Done Sent false
33909 10203 Acked Write 6 Done Done Sent false
33910 10204 Acked Read 4 Done Done Sent false
33911 10205 Acked Write 8 Done Done Sent false
33912 10206 Acked Read 2 Done Done Sent false
33913 10207 Acked Read 2 Done Done Sent false
33914 10208 Acked Read 9 Done Done Sent false
33915 10209 Acked Flush 0 Done Done Sent false
33916 10210 Acked Flush 0 Done Done Sent false
33917 10211 Acked Write 5 Done Done Sent false
33918 10212 Acked Write 4 Done Done Sent false
33919 10213 Acked Write 3 Done Done Sent false
33920 10214 Acked Write 6 Done Done Sent false
33921 10215 Acked Write 4 Done Done Sent false
33922 10216 Acked Write 9 Done Done Sent false
33923 10217 Acked Write 8 Done Done Sent false
33924 10218 Acked Read 10 Done Done Sent false
33925 10219 Acked Read 1 Done Done Sent false
33926 10220 Acked Write 3 Done Done Sent false
33927 10221 Acked Read 4 Done Done Sent false
33928 10222 Acked Flush 0 Done Done Sent false
33929 10223 Acked Read 8 Done Done Sent false
33930 10224 Acked Write 8 Done Done Sent false
33931 10225 Acked Write 8 Done Done Sent false
33932 10226 Acked Read 10 Done Done Sent false
33933 10227 Acked Write 2 Done Done Sent false
33934 10228 Acked Read 9 Done Done Sent false
33935 10229 Acked Write 3 Done Done Sent false
33936 10230 Acked Read 8 Done Done Sent false
33937 10231 Acked Write 5 Done Done Sent false
33938 10232 Acked Read 9 Done Done Sent false
33939 10233 Acked Write 3 Done Done Sent false
33940 10234 Acked Read 3 Done Done Sent false
33941 10235 Acked Write 8 Done Done Sent false
33942 10236 Acked Write 5 Done Done Sent false
33943 10237 Acked Read 5 Done Done Sent false
33944 10238 Acked Read 1 Done Done Sent false
33945 10239 Acked Read 7 Done Done Sent false
33946 10240 Acked Read 7 Done Done Sent false
33947 10241 Acked Read 10 Done Done Sent false
33948 10242 Acked Flush 0 Done Done Sent false
33949 10243 Acked Write 1 Done Done Sent false
33950 10244 Acked Read 9 Done Done Sent false
33951 10245 Acked Write 2 Done Done Sent false
33952 10246 Acked Read 9 Done Done Sent false
33953 10247 Acked Read 8 Done Done Sent false
33954 10248 Acked Write 5 Done Done Sent false
33955 10249 Acked Flush 0 Done Done Sent false
33956 10250 Acked Read 3 Done Done Sent false
33957 10251 Acked Read 7 Done Done Sent false
33958 10252 Acked Read 8 Done Done Sent false
33959 10253 Acked Read 6 Done Done Sent false
33960 10254 Acked Read 10 Done Done Sent false
33961 10255 Acked Read 5 Done Done Sent false
33962 10256 Acked Read 2 Done Done Sent false
33963 10257 Acked Read 9 Done Done Sent false
33964 10258 Acked Read 8 Done Done Sent false
33965 10259 Acked Read 5 Done Done Sent false
33966 10260 Acked Flush 0 Done Done Sent false
33967 10261 Acked Write 1 Done Done Sent false
33968 10262 Acked Read 8 Done Done Sent false
33969 10263 Acked Flush 0 Done Done Sent false
33970 10264 Acked Write 5 Done Done Sent false
33971 10265 Acked Read 7 Done Done Sent false
33972 10266 Acked Read 3 Done Done Sent false
33973 10267 Acked Write 4 Done Done Sent false
33974 10268 Acked Read 6 Done Done Sent false
33975 10269 Acked Write 2 Done Done Sent false
33976 10270 Acked Write 9 Done Done Sent false
33977 10271 Acked Write 7 Done Done Sent false
33978 10272 Acked Read 6 Done Done Sent false
33979 10273 Acked Read 5 Done Done Sent false
33980 10274 Acked Write 3 Done Done Sent false
33981 10275 Acked Read 1 Done Done Sent false
33982 10276 Acked Flush 0 Done Done Sent false
33983 10277 Acked Write 4 Done Done Sent false
33984 10278 Acked Flush 0 Done Done Sent false
33985 10279 Acked Flush 0 Done Done Sent false
33986 10280 Acked Read 9 Done Done Sent false
33987 10281 Acked Read 5 Done Done Sent false
33988 10282 Acked Read 8 Done Done Sent false
33989 10283 Acked Read 7 Done Done Sent false
33990 10284 Acked Write 6 Done Done Sent false
33991 10285 Acked Write 10 Done Done Sent false
33992 10286 Acked Read 3 Done Done Sent false
33993 10287 Acked Read 8 Done Done Sent false
33994 10288 Acked Write 8 Done Done Sent false
33995 10289 Acked Read 6 Done Done Sent false
33996 10290 Acked Read 2 Done Done Sent false
33997 10291 Acked Write 2 Done Done Sent false
33998 10292 Acked Write 2 Done Done Sent false
33999 10293 Acked Read 2 Done Done Sent false
34000 10294 Acked Write 2 Done Done Sent false
34001 10295 Acked Read 2 Done Done Sent false
34002 10296 Acked Read 1 Done Done Sent false
34003 10297 Acked Write 2 Done Done Sent false
34004 10298 Acked Read 10 Done Done Sent false
34005 10299 Acked Read 8 Done Done Sent false
34006 10300 Acked Write 1 Done Done Sent false
34007 10301 Acked Read 5 Done Done Sent false
34008 10302 Acked Write 6 Done Done Sent false
34009 10303 Acked Read 9 Done Done Sent false
34010 10304 Acked Read 9 Done Done Sent false
34011 10305 Acked Write 9 Done Done Sent false
34012 10306 Acked Read 5 Done Done Sent false
34013 10307 Acked Write 7 Done Done Sent false
34014 10308 Acked Write 1 Done Done Sent false
34015 10309 Acked Read 2 Done Done Sent false
34016 10310 Acked Read 10 Done Done Sent false
34017 10311 Acked Write 1 Done Done Sent false
34018 10312 Acked Write 4 Done Done Sent false
34019 10313 Acked Flush 0 Done Done Sent false
34020 10314 Acked Write 1 Done Done Sent false
34021 10315 Acked Write 7 Done Done Sent false
34022 10316 Acked Write 7 Done Done Sent false
34023 10317 Acked Write 1 Done Done Sent false
34024 10318 Acked Read 10 Done Done Sent false
34025 10319 Acked Write 4 Done Done Sent false
34026 10320 Acked Read 9 Done Done Sent false
34027 10321 Acked Read 2 Done Done Sent false
34028 10322 Acked Read 8 Done Done Sent false
34029 10323 Acked Read 9 Done Done Sent false
34030 10324 Acked Read 10 Done Done Sent false
34031 10325 Acked Read 8 Done Done Sent false
34032 10326 Acked Read 2 Done Done Sent false
34033 10327 Acked Flush 0 Done Done Sent false
34034 10328 Acked Read 1 Done Done Sent false
34035 10329 Acked Flush 0 Done Done Sent false
34036 10330 Acked Flush 0 Done Done Sent false
34037 10331 Acked Read 5 Done Done Sent false
34038 10332 Acked Write 6 Done Done Sent false
34039 10333 Acked Read 5 Done Done Sent false
34040 10334 Acked Write 4 Done Done Sent false
34041 10335 Acked Write 2 Done Done Sent false
34042 10336 Acked Flush 0 Done Done Sent false
34043 10337 Acked Read 4 Done Done Sent false
34044 10338 Acked Read 7 Done Done Sent false
34045 10339 Acked Read 1 Done Done Sent false
34046 10340 Acked Read 8 Done Done Sent false
34047 10341 Acked Read 3 Done Done Sent false
34048 10342 Acked Read 4 Done Done Sent false
34049 10343 Acked Read 9 Done Done Sent false
34050 10344 Acked Read 2 Done Done Sent false
34051 10345 Acked Flush 0 Done Done Sent false
34052 10346 Acked Write 8 Done Done Sent false
34053 10347 Acked Write 9 Done Done Sent false
34054 10348 Acked Read 1 Done Done Sent false
34055 10349 Acked Read 10 Done Done Sent false
34056 10350 Acked Flush 0 Done Done Sent false
34057 10351 Acked Read 4 Done Done Sent false
34058 10352 Acked Read 4 Done Done Sent false
34059 10353 Acked Read 10 Done Done Sent false
34060 10354 Acked Write 1 Done Done Sent false
34061 10355 Acked Read 5 Done Done Sent false
34062 10356 Acked Read 1 Done Done Sent false
34063 10357 Acked Read 2 Done Done Sent false
34064 10358 Acked Write 2 Done Done Sent false
34065 10359 Acked Flush 0 Done Done Sent false
34066 STATES DS:0 DS:1 DS:2 TOTAL
34067 Sent 0 0 302 302
34068 Done 302 302 0 604
34069 Skipped 0 0 0 0
34070 Error 0 0 0 0
34071 Last Flush: 10359 10359 10057
34072 Downstairs last five completed: 10057 10056 10055 10054 10053
34073 Upstairs last five completed: 10359 10358 10357 10356 10355
34074 CLIENT: Up:0 ds:302 act:2
340752025-01-21T15:54:18.802ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
340762025-01-21T15:54:18.802ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38365) = io task client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
340772025-01-21T15:54:18.803ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
340782025-01-21T15:54:18.804ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
340792025-01-21T15:54:18.804ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
340802025-01-21T15:54:18.804ZINFOcrucible: send last flush ID to this DS: 10057 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
340812025-01-21T15:54:18.804ZINFOcrucible: Replied this last flush ID: 10057 = downstairs client = 2 session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
340822025-01-21T15:54:18.804ZINFOcrucible: [2] client re-new 302 jobs since flush 10057 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
340832025-01-21T15:54:18.805ZINFOcrucible: [2] Replayed 302 jobs since flush: 10057 = downstairs session_id = 5d7111f5-ffb3-4e9c-baab-0950e41aedfd
34084 ----------------------------------------------------------------
34085 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34086 Downstairs last five completed: 10359 10358 10357 10356 10355
34087 Upstairs last five completed: 10359 10358 10357 10356 10355
34088 CLIENT: Up:0 ds:0 act:3
34089 Replay: All jobs finished
34090 Test replay has completed
34091 Wrote out file "/tmp/test_replay_verify.log"
34092 CLIENT: Tests done. All submitted work has been ACK'd
34093 ----------------------------------------------------------------
34094 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34095 Downstairs last five completed: 10359 10358 10357 10356 10355
34096 Upstairs last five completed: 10359 10358 10357 10356 10355
34097 CLIENT: Up:0 ds:0 act:3
34098 CLIENT: All jobs finished, all DS active.
340992025-01-21T15:54:21.056ZWARNcrucible: Guest handle has been dropped
34100 Replays:30 time: 7:08
34101 Do final verify
34102 Jan 21 15:54:21.129 INFO accepted connection, remote_addr: 127.0.0.1:43602, local_addr: 127.0.0.1:9998
34103 Jan 21 15:54:21.130 INFO request completed, latency_us: 220, response_code: 200, uri: /regioninfo, method: GET, req_id: 8a65d33c-adb4-4d11-83b5-ebbca2fd6458, remote_addr: 127.0.0.1:43602, local_addr: 127.0.0.1:9998
341042025-01-21T15:54:21.130ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
34105 Jan 21 15:54:21.131 INFO request completed, latency_us: 129, response_code: 200, uri: /regioncount, method: GET, req_id: ff1451fd-31bd-4f96-8544-992d88ebaca0, remote_addr: 127.0.0.1:43602, local_addr: 127.0.0.1:9998
341062025-01-21T15:54:21.131ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
34107 Jan 21 15:54:21.131 INFO request completed, latency_us: 108, response_code: 200, uri: /port/cid/0, method: GET, req_id: a60fe652-289f-4335-bd61-ab2f7d9fe9c1, remote_addr: 127.0.0.1:43602, local_addr: 127.0.0.1:9998
34108 Jan 21 15:54:21.132 INFO request completed, latency_us: 93, response_code: 200, uri: /port/cid/1, method: GET, req_id: b4bf036a-45fb-4329-b0a9-9bf82a869da9, remote_addr: 127.0.0.1:43602, local_addr: 127.0.0.1:9998
34109 Jan 21 15:54:21.132 INFO request completed, latency_us: 132, response_code: 200, uri: /port/cid/2, method: GET, req_id: 294cdddc-9ddd-40ab-9897-7bc7dea84d21, remote_addr: 127.0.0.1:43602, local_addr: 127.0.0.1:9998
341102025-01-21T15:54:21.133ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
341112025-01-21T15:54:21.133ZINFOcrucible: Upstairs starts
341122025-01-21T15:54:21.133ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "5c2b227300b758c153f33da870809d67a5160dae", git_commit_timestamp: "2025-01-21T15:31:20.000000000Z", git_branch: "mkeeter/always-check-live-repair", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
341132025-01-21T15:54:21.133ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
341142025-01-21T15:54:21.133ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
341152025-01-21T15:54:21.134ZINFOcrucible: Crucible de417931-cfbb-4457-8843-579f68bdef35 has session id: 76d5cafc-4adc-4d93-9df4-a05277d1bcc6 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341162025-01-21T15:54:21.134ZINFOcrucible: Upstairs opts: Upstairs UUID: de417931-cfbb-4457-8843-579f68bdef35, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341172025-01-21T15:54:21.134ZINFOcrucible: Crucible stats registered with UUID: de417931-cfbb-4457-8843-579f68bdef35 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341182025-01-21T15:54:21.134ZWARNcrucible: could not find Downstairs address for Nexus session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341192025-01-21T15:54:21.134ZINFOcrucible: Downstairs in volume = 3 task = crutest
341202025-01-21T15:54:21.135ZINFOcrucible: The guest has requested activation with gen:3
341212025-01-21T15:54:21.135ZINFOcrucible: de417931-cfbb-4457-8843-579f68bdef35 active request set session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341222025-01-21T15:54:21.135ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341232025-01-21T15:54:21.135ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341242025-01-21T15:54:21.135ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341252025-01-21T15:54:21.135ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341262025-01-21T15:54:21.135ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341272025-01-21T15:54:21.135ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341282025-01-21T15:54:21.135ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341292025-01-21T15:54:21.135ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341302025-01-21T15:54:21.135ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341312025-01-21T15:54:21.135ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:55224) = io task client = 0 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341322025-01-21T15:54:21.135ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35210) = io task client = 2 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341332025-01-21T15:54:21.135ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:63438) = io task client = 1 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341342025-01-21T15:54:21.136ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341352025-01-21T15:54:21.136ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341362025-01-21T15:54:21.136ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341372025-01-21T15:54:21.136ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341382025-01-21T15:54:21.136ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341392025-01-21T15:54:21.136ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341402025-01-21T15:54:21.137ZINFOcrucible: Waiting for more clients to be ready session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341412025-01-21T15:54:21.137ZINFOcrucible: Waiting for more clients to be ready session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341422025-01-21T15:54:21.137ZINFOcrucible: [0]R flush_numbers[0..12]: [960, 955, 947, 944, 955, 946, 951, 937, 958, 954, 937, 944] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341432025-01-21T15:54:21.137ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341442025-01-21T15:54:21.137ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341452025-01-21T15:54:21.137ZINFOcrucible: [1]R flush_numbers[0..12]: [960, 955, 947, 944, 955, 946, 951, 937, 958, 954, 937, 944] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341462025-01-21T15:54:21.137ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341472025-01-21T15:54:21.137ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341482025-01-21T15:54:21.137ZINFOcrucible: [2]R flush_numbers[0..12]: [960, 955, 947, 944, 955, 946, 951, 937, 958, 954, 937, 944] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341492025-01-21T15:54:21.137ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341502025-01-21T15:54:21.137ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341512025-01-21T15:54:21.137ZINFOcrucible: Max found gen is 3 = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341522025-01-21T15:54:21.137ZINFOcrucible: Generation requested: 3 >= found:3 = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341532025-01-21T15:54:21.137ZINFOcrucible: Next flush: 962 = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
34154 Wait for a query_work_queue command to finish before sending IO
341552025-01-21T15:54:21.137ZINFOcrucible: All extents match = downstairs session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341562025-01-21T15:54:21.137ZINFOcrucible: No downstairs reconciliation required session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341572025-01-21T15:54:21.137ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341582025-01-21T15:54:21.137ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341592025-01-21T15:54:21.137ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341602025-01-21T15:54:21.137ZINFOcrucible: All required reconciliation work is completed session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341612025-01-21T15:54:21.137ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341622025-01-21T15:54:21.137ZINFOcrucible: de417931-cfbb-4457-8843-579f68bdef35 is now active with session: 76d5cafc-4adc-4d93-9df4-a05277d1bcc6 session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341632025-01-21T15:54:21.137ZINFOcrucible: Set Active after no reconciliation session_id = 76d5cafc-4adc-4d93-9df4-a05277d1bcc6
341642025-01-21T15:54:21.137ZINFOcrucible: The guest has finished waiting for activation with:3
34165 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
34166 Loading write count information from file "/tmp/test_replay_verify.log"
34167 Read and Verify all blocks (0..5000 range:false)
34168 Wrote out file "/tmp/test_replay_verify.log"
34169 Verify test completed
34170 Wrote out file "/tmp/test_replay_verify.log"
34171 CLIENT: Tests done. All submitted work has been ACK'd
34172 ----------------------------------------------------------------
34173 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
34174 States: Active Active Active
34175 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34176 1000 Acked Read 256 Done Done Done false
34177 1001 Acked Read 256 Done Done Done false
34178 1002 Acked Read 256 Done Done Done false
34179 1003 Acked Read 256 Done Done Done false
34180 1004 Acked Read 256 Done Done Done false
34181 1005 Acked Read 256 Done Done Done false
34182 1006 Acked Read 256 Done Done Done false
34183 1007 Acked Read 256 Done Done Done false
34184 1008 Acked Read 256 Done Done Done false
34185 1009 Acked Read 256 Done Done Done false
34186 1010 Acked Read 256 Done Done Done false
34187 1011 Acked Read 256 Done Done Done false
34188 1012 Acked Read 256 Done Done Done false
34189 1013 Acked Read 256 Done Done Done false
34190 1014 Acked Read 256 Done Done Done false
34191 1015 Acked Read 256 Done Done Done false
34192 1016 Acked Read 256 Done Done Done false
34193 1017 Acked Read 256 Done Done Done false
34194 1018 Acked Read 136 Done Done Done false
34195 1019 Acked Read 256 Done Done Done false
34196 STATES DS:0 DS:1 DS:2 TOTAL
34197 Sent 0 0 0 0
34198 Done 20 20 20 60
34199 Skipped 0 0 0 0
34200 Error 0 0 0 0
34201 Last Flush: 0 0 0
34202 Downstairs last five completed:
34203 Upstairs last five completed: 1019 1018 1017 1016 1015
34204 CLIENT: Up:0 ds:20 act:3
34205 ----------------------------------------------------------------
34206 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
34207 Downstairs last five completed: 1020 1019 1018 1017 1016
34208 Upstairs last five completed: 1020 1019 1018 1017 1016
34209 CLIENT: Up:0 ds:0 act:3
34210 CLIENT: All crucible jobs finished, exiting program
342112025-01-21T15:54:25.538ZWARNcrucible: Guest handle has been dropped
34212 Jan 21 15:54:25.603 INFO accepted connection, remote_addr: 127.0.0.1:60592, local_addr: 127.0.0.1:9998
34213 Main task has work to do, go find it
34214 got dsc Shutdown
34215 Shutdown
34216 Shut it down
34217 Jan 21 15:54:25.604 INFO request completed, latency_us: 164, response_code: 204, uri: [8810] Disable keep_running
34218 /shutdown[8820] Disable keep_running
34219 , method: POST, req_id: ede88dba-482d-4556-9d48-1fdf6c5b6f9e, remote_addr: 127.0[8810] Got stop action so:false kr:false
34220 .0.1:60592, local_addr: 127.0.0.[8820] Got stop action so:false kr:false
34221 1:9998
34222 [8830] Disable keep_running
34223 [8830] Got stop action so:false kr:false
34224 [8820] Exited with: ExitStatus(unix_wait_status(9))
34225 [8820][1] reports Exit
34226 [8830] Exited with: ExitStatus(unix_wait_status(9))
34227 [8810] Exited with: ExitStatus(unix_wait_status(9))
34228 [8830][2] reports Exit
34229 [8810][0] reports Exit
34230 January 21, 2025 at 03:54:30 PM UTC Test ends with 0