Max level shown:
1 starting January 27, 2025 at 07:36:14 PM UTC
2 Creating 3 downstairs regions
3 Creating region directory at: /var/tmp/dsc/region
4 Creating dsc directory at: /tmp/dsc
5 Downstairs region 0 created at /var/tmp/dsc/region/8810 in 0.33400872
6 Downstairs region 1 created at /var/tmp/dsc/region/8820 in 0.024296075
7 Downstairs region 2 created at /var/tmp/dsc/region/8830 in 0.021968076
8 Created 3 regions
9 Starting 3 downstairs
10 Using existing output directory "/tmp/dsc"
11 Update our region info with: Some(RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 })
12 start ds: 8810
13 start ds: 8820
14 start ds: 8830
15 start access at:127.0.0.1:9998
16 Starting downstairs at port 8820
17 Starting downstairs at port 8810
18 Make output file at "/tmp/dsc/downstairs-8810.txt"
19 Make output file at "/tmp/dsc/downstairs-8820.txt"
20 Starting downstairs at port 8830
21 Make output file at "/tmp/dsc/downstairs-8830.txt"
22 [8820][1] initial start wait reports Starting
23 [8810][0] initial start wait reports Starting
24 [8830][2] initial start wait reports Starting
25 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1190)
26 [8810][0] initial start wait reports Running
27 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1191)
28 [8820][1] initial start wait reports Running
29 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1192)
30 [8830][2] initial start wait reports Running
31 All downstairs are running
32 Jan 27 19:36:15.124 INFO listening, local_addr: 127.0.0.1:9998
33 Control access at:127.0.0.1:9998
34 Running initial fill
35 Jan 27 19:36:20.227 INFO accepted connection, remote_addr: 127.0.0.1:54371, local_addr: 127.0.0.1:9998
36 Jan 27 19:36:20.229 INFO request completed, latency_us: 428, response_code: 200, uri: /regioninfo, method: GET, req_id: 905a059f-6b56-4611-bb8e-058218864013, remote_addr: 127.0.0.1:54371, local_addr: 127.0.0.1:9998
37 Jan 27 19:36:20.231 INFO request completed, latency_us: 129, response_code: 200, uri: /regioncount, method: GET, req_id: cf8808cd-f9ed-41c6-ae34-8179e8a564d0, remote_addr: 127.0.0.1:54371, local_addr: 127.0.0.1:9998
382025-01-27T19:36:20.231ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
392025-01-27T19:36:20.232ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
40 Jan 27 19:36:20.232 INFO request completed, latency_us: 158, response_code: 200, uri: /port/cid/0, method: GET, req_id: 2dea7c18-c4d0-4dd2-be59-2b23e63d206a, remote_addr: 127.0.0.1:54371, local_addr: 127.0.0.1:9998
41 Jan 27 19:36:20.233 INFO request completed, latency_us: 102, response_code: 200, uri: /port/cid/1, method: GET, req_id: 69a3e2a4-a67a-4085-b330-8336d3b3e3bf, remote_addr: 127.0.0.1:54371, local_addr: 127.0.0.1:9998
42 Jan 27 19:36:20.233 INFO request completed, latency_us: 102, response_code: 200, uri: /port/cid/2, method: GET, req_id: c16841da-f17a-4844-86dc-010a18e1e8ed, remote_addr: 127.0.0.1:54371, local_addr: 127.0.0.1:9998
432025-01-27T19:36:20.234ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
442025-01-27T19:36:20.235ZINFOcrucible: Upstairs starts
452025-01-27T19:36:20.235ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "d77aa90b6afe7d7ddb1dc63211fa4216c8866a90", git_commit_timestamp: "2025-01-27T19:20:16.000000000Z", git_branch: "mkeeter/shrink-replay-buffer", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
462025-01-27T19:36:20.235ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
472025-01-27T19:36:20.235ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
482025-01-27T19:36:20.237ZINFOcrucible: Crucible 8ef76689-d9b6-498b-aa3a-f8fde761b2ed has session id: 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
492025-01-27T19:36:20.237ZINFOcrucible: Upstairs opts: Upstairs UUID: 8ef76689-d9b6-498b-aa3a-f8fde761b2ed, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
502025-01-27T19:36:20.237ZINFOcrucible: Crucible stats registered with UUID: 8ef76689-d9b6-498b-aa3a-f8fde761b2ed session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
512025-01-27T19:36:20.237ZWARNcrucible: could not find Downstairs address for Nexus session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
522025-01-27T19:36:20.239ZINFOcrucible: Downstairs in volume = 3 task = crutest
532025-01-27T19:36:20.239ZINFOcrucible: The guest has requested activation with gen:1
542025-01-27T19:36:20.239ZINFOcrucible: 8ef76689-d9b6-498b-aa3a-f8fde761b2ed active request set session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
552025-01-27T19:36:20.239ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
562025-01-27T19:36:20.239ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
572025-01-27T19:36:20.239ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
582025-01-27T19:36:20.239ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
592025-01-27T19:36:20.240ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
602025-01-27T19:36:20.240ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
612025-01-27T19:36:20.240ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
622025-01-27T19:36:20.240ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
632025-01-27T19:36:20.240ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
642025-01-27T19:36:20.240ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:64905) = io task client = 2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
652025-01-27T19:36:20.240ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37213) = io task client = 1 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
662025-01-27T19:36:20.240ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36100) = io task client = 0 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
672025-01-27T19:36:20.242ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
682025-01-27T19:36:20.242ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
692025-01-27T19:36:20.242ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
702025-01-27T19:36:20.243ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
712025-01-27T19:36:20.243ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
722025-01-27T19:36:20.243ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
732025-01-27T19:36:20.244ZINFOcrucible: Waiting for more clients to be ready session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
742025-01-27T19:36:20.244ZINFOcrucible: Waiting for more clients to be ready session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
752025-01-27T19:36:20.244ZINFOcrucible: [0]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
762025-01-27T19:36:20.244ZINFOcrucible: [0]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
772025-01-27T19:36:20.244ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
782025-01-27T19:36:20.244ZINFOcrucible: [1]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
792025-01-27T19:36:20.244ZINFOcrucible: [1]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
802025-01-27T19:36:20.244ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
812025-01-27T19:36:20.244ZINFOcrucible: [2]R flush_numbers[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
822025-01-27T19:36:20.244ZINFOcrucible: [2]R generation[0..12]: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
832025-01-27T19:36:20.244ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
842025-01-27T19:36:20.244ZINFOcrucible: Max found gen is 1 = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
852025-01-27T19:36:20.245ZINFOcrucible: Generation requested: 1 >= found:1 = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
862025-01-27T19:36:20.245ZINFOcrucible: Next flush: 1 = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
872025-01-27T19:36:20.245ZINFOcrucible: All extents match = downstairs session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
882025-01-27T19:36:20.245ZINFOcrucible: No downstairs reconciliation required session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
892025-01-27T19:36:20.245ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
902025-01-27T19:36:20.245ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
912025-01-27T19:36:20.245ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
922025-01-27T19:36:20.245ZINFOcrucible: All required reconciliation work is completed session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
932025-01-27T19:36:20.245ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
942025-01-27T19:36:20.245ZINFOcrucible: 8ef76689-d9b6-498b-aa3a-f8fde761b2ed is now active with session: 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2 session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
952025-01-27T19:36:20.245ZINFOcrucible: Set Active after no reconciliation session_id = 975f6c7c-1b94-4fd0-82f5-bf7e8ea9bbe2
962025-01-27T19:36:20.245ZINFOcrucible: The guest has finished waiting for activation with:1
97 Activate successful
98 Wait for a query_work_queue command to finish before sending IO
99 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
100 Fill test
101 Read and Verify all blocks (0..5000 range:false)
102 Wrote out file "/tmp/test_replay_verify.log"
103 CLIENT: Tests done. All submitted work has been ACK'd
104 ----------------------------------------------------------------
105 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:20
106 States: Active Active Active
107 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
108 1021 Acked Read 256 Done Done Done false
109 1022 Acked Read 256 Done Done Done false
110 1023 Acked Read 256 Done Done Done false
111 1024 Acked Read 256 Done Done Done false
112 1025 Acked Read 256 Done Done Done false
113 1026 Acked Read 256 Done Done Done false
114 1027 Acked Read 256 Done Done Done false
115 1028 Acked Read 256 Done Done Done false
116 1029 Acked Read 256 Done Done Done false
117 1030 Acked Read 256 Done Done Done false
118 1031 Acked Read 256 Done Done Done false
119 1032 Acked Read 256 Done Done Done false
120 1033 Acked Read 256 Done Done Done false
121 1034 Acked Read 256 Done Done Done false
122 1035 Acked Read 256 Done Done Done false
123 1036 Acked Read 256 Done Done Done false
124 1037 Acked Read 256 Done Done Done false
125 1038 Acked Read 136 Done Done Done false
126 1039 Acked Read 256 Done Done Done false
127 1040 Acked Read 256 Done Done Done false
128 STATES DS:0 DS:1 DS:2 TOTAL
129 Sent 0 0 0 0
130 Done 20 20 20 60
131 Skipped 0 0 0 0
132 Error 0 0 0 0
133 Last Flush: 1020 1020 1020
134 Downstairs last five completed: 1020 1019 1018 1017 1016
135 Upstairs last five completed: 1040 1039 1038 1037 1036
136 CLIENT: Up:0 ds:20 act:3
137 ----------------------------------------------------------------
138 Crucible gen:1 GIO:true work queues: Upstairs:0 downstairs:0
139 Downstairs last five completed: 1041 1040 1039 1038 1037
140 Upstairs last five completed: 1041 1040 1039 1038 1037
141 CLIENT: Up:0 ds:0 act:3
142 CLIENT: All crucible jobs finished, exiting program
1432025-01-27T19:36:24.812ZWARNcrucible: Guest handle has been dropped
144 Replay loop starts now January 27, 2025 at 07:36:24 PM UTC
145 Jan 27 19:36:24.882 INFO accepted connection, remote_addr: 127.0.0.1:51525, local_addr: 127.0.0.1:9998
146 Jan 27 19:36:24.882 INFO request completed, latency_us: 167, response_code: 200, uri: /regioninfo, method: GET, req_id: d0fcb2d0-25b5-4862-a408-d994c080d4c2, remote_addr: 127.0.0.1:51525, local_addr: 127.0.0.1:9998
1472025-01-27T19:36:24.883ZINFOcrucible: Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 } task = crutest
148 Jan 27 19:36:24.883 INFO request completed, latency_us: 105, response_code: 200, uri: /regioncount, method: GET, req_id: 4531ea96-5880-4a6b-a483-6ed5b74bc70b, remote_addr: 127.0.0.1:51525, local_addr: 127.0.0.1:9998
1492025-01-27T19:36:24.884ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
150 Jan 27 19:36:24.884 INFO request completed, latency_us: 127, response_code: 200, uri: /port/cid/0, method: GET, req_id: 08d8e7a9-0a5a-46d1-8e7c-549b34a7beac, remote_addr: 127.0.0.1:51525, local_addr: 127.0.0.1:9998
151 Jan 27 19:36:24.885 INFO request completed, latency_us: 108, response_code: 200, uri: /port/cid/1, method: GET, req_id: 9f3c4a17-24db-4044-bcf4-10bad5bbc127, remote_addr: 127.0.0.1:51525, local_addr: 127.0.0.1:9998
152 Jan 27 19:36:24.885 INFO request completed, latency_us: 88, response_code: 200, uri: /port/cid/2, method: GET, req_id: 301193ca-dc64-4e5a-9ea9-a720c17dfc6f, remote_addr: 127.0.0.1:51525, local_addr: 127.0.0.1:9998
1532025-01-27T19:36:24.885ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
1542025-01-27T19:36:24.886ZINFOcrucible: Upstairs starts
1552025-01-27T19:36:24.886ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "d77aa90b6afe7d7ddb1dc63211fa4216c8866a90", git_commit_timestamp: "2025-01-27T19:20:16.000000000Z", git_branch: "mkeeter/shrink-replay-buffer", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
1562025-01-27T19:36:24.886ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
1572025-01-27T19:36:24.886ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
1582025-01-27T19:36:24.886ZINFOcrucible: Crucible 0c5c0539-a805-490c-b9bd-b023a60aa5a2 has session id: edf4bf43-2393-40de-98dd-2a3c00d4e1dd session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1592025-01-27T19:36:24.886ZINFOcrucible: Upstairs opts: Upstairs UUID: 0c5c0539-a805-490c-b9bd-b023a60aa5a2, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1602025-01-27T19:36:24.886ZINFOcrucible: Crucible stats registered with UUID: 0c5c0539-a805-490c-b9bd-b023a60aa5a2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1612025-01-27T19:36:24.886ZWARNcrucible: could not find Downstairs address for Nexus session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1622025-01-27T19:36:24.887ZINFOcrucible: Downstairs in volume = 3 task = crutest
1632025-01-27T19:36:24.887ZINFOcrucible: The guest has requested activation with gen:2
1642025-01-27T19:36:24.887ZINFOcrucible: 0c5c0539-a805-490c-b9bd-b023a60aa5a2 active request set session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1652025-01-27T19:36:24.887ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1662025-01-27T19:36:24.887ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1672025-01-27T19:36:24.887ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1682025-01-27T19:36:24.887ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1692025-01-27T19:36:24.887ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1702025-01-27T19:36:24.887ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1712025-01-27T19:36:24.887ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1722025-01-27T19:36:24.887ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1732025-01-27T19:36:24.887ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1742025-01-27T19:36:24.887ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38964) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1752025-01-27T19:36:24.888ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59571) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1762025-01-27T19:36:24.888ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:51845) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1772025-01-27T19:36:24.888ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1782025-01-27T19:36:24.888ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1792025-01-27T19:36:24.888ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1802025-01-27T19:36:24.889ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1812025-01-27T19:36:24.889ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1822025-01-27T19:36:24.889ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1832025-01-27T19:36:24.889ZINFOcrucible: Waiting for more clients to be ready session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1842025-01-27T19:36:24.890ZINFOcrucible: Waiting for more clients to be ready session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1852025-01-27T19:36:24.890ZINFOcrucible: [0]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1862025-01-27T19:36:24.890ZINFOcrucible: [0]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1872025-01-27T19:36:24.890ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1882025-01-27T19:36:24.890ZINFOcrucible: [1]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1892025-01-27T19:36:24.890ZINFOcrucible: [1]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1902025-01-27T19:36:24.890ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1912025-01-27T19:36:24.890ZINFOcrucible: [2]R flush_numbers[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1922025-01-27T19:36:24.890ZINFOcrucible: [2]R generation[0..12]: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1932025-01-27T19:36:24.890ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1942025-01-27T19:36:24.890ZINFOcrucible: Max found gen is 2 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
195 {"msg":"Generation requested: 2 >= found:2","v":0,"name":"crucible","level":30,"time":"2025-01-27T19:36:24.890217776Z"Activate successful
196 ,"Wait for a query_work_queue command to finish before sending IO
197 hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"downstairs","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
1982025-01-27T19:36:24.890ZINFOcrucible: Next flush: 2 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1992025-01-27T19:36:24.890ZINFOcrucible: All extents match = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2002025-01-27T19:36:24.890ZINFOcrucible: No downstairs reconciliation required session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2012025-01-27T19:36:24.890ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2022025-01-27T19:36:24.890ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2032025-01-27T19:36:24.890ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2042025-01-27T19:36:24.890ZINFOcrucible: All required reconciliation work is completed session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2052025-01-27T19:36:24.890ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2062025-01-27T19:36:24.890ZINFOcrucible: 0c5c0539-a805-490c-b9bd-b023a60aa5a2 is now active with session: edf4bf43-2393-40de-98dd-2a3c00d4e1dd session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2072025-01-27T19:36:24.890ZINFOcrucible: Set Active after no reconciliation session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2082025-01-27T19:36:24.890ZINFOcrucible: The guest has finished waiting for activation with:2
209 Disk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
210 Loading write count information from file "/tmp/test_replay_verify.log"
211 Jan 27 19:36:24.924 INFO accepted connection, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
212 Main task has work to do, go find it
213 got dsc Stop(0)
214 stop 0
215 Jan 27 19:36:24.925 INFO request completed, latency_us: 177, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 7ccd7535-91bc-4a3e-aedf-0a62407e1b18, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
216 [8810] Got stop action so:false kr:false
2172025-01-27T19:36:24.926ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2182025-01-27T19:36:24.926ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2192025-01-27T19:36:24.926ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2202025-01-27T19:36:24.926ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
221 Jan 27 19:36:24.926 INFO request completed, latency_us: 246, response_code: 200, uri: /state/cid/0, method: GET, req_id: 4bec99ae-3fe3-4f01-87b1-80046ce199e3, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
222 [8810] Exited with: ExitStatus(unix_wait_status(9))
223 [8810][0] reports Exit
224 Jan 27 19:36:28.929 INFO request completed, latency_us: 239, response_code: 200, uri: /state/cid/0, method: GET, req_id: df0cefa2-6edb-477f-ad19-fabd9daf85e7, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
225 001/300 Read block 2729 len 32768
226 002/300 Flush
227 003/300 Read block 4517 len 28672
228 004/300 Read block 486 len 24576
229 005/300 Write block 2026 len 36864 data: 2 2 2 2 2 2 2 2 2
230 006/300 Read block 3891 len 32768
231 007/300 Write block 1543 len 12288 data: 2 2 2
232 008/300 Write block 820 len 4096 data: 2
233 009/300 Read block 4454 len 24576
234 010/300 Write block 4680 len 8192 data: 2 2
235 011/300 Write block 4250 len 32768 data: 2 2 2 2 2 2 2 2
236 012/300 Read block 1184 len 20480
237 013/300 Read block 4074 len 16384
238 014/300 Write block 393 len 20480 data: 2 2 2 2 2
239 015/300 Write block 1537 len 36864 data: 2 2 2 2 2 2 3 3 3
240 016/300 Read block 2224 len 16384
241 017/300 Write block 373 len 8192 data: 2 2
242 018/300 Read block 1675 len 36864
243 019/300 Write block 1189 len 32768 data: 2 2 2 2 2 2 2 2
244 020/300 Read block 81 len 40960
245 021/300 Read block 4473 len 8192
246 022/300 Write block 4118 len 20480 data: 2 2 2 2 2
247 023/300 Write block 3569 len 28672 data: 2 2 2 2 2 2 2
248 024/300 Read block 783 len 8192
249 025/300 Write block 1353 len 4096 data: 2
250 026/300 Write block 4017 len 28672 data: 2 2 2 2 2 2 2
251 027/300 Write block 2288 len 20480 data: 2 2 2 2 2
252 028/300 Write block 3285 len 12288 data: 2 2 2
253 029/300 Flush
254 030/300 Write block 2181 len 16384 data: 2 2 2 2
255 031/300 Read block 4067 len 16384
256 032/300 Flush
257 033/300 Read block 1835 len 32768
258 034/300 Write block 4771 len 4096 data: 2
259 035/300 Write block 4995 len 20480 data: 2 2 2 2 2
260 036/300 Write block 1207 len 36864 data: 2 2 2 2 2 2 2 2 2
261 037/300 Write block 3364 len 36864 data: 2 2 2 2 2 2 2 2 2
262 038/300 Flush
263 039/300 Flush
264 040/300 Write block 91 len 8192 data: 2 2
265 041/300 Read block 357 len 24576
266 042/300 Read block 1712 len 4096
267 043/300 Write block 3783 len 32768 data: 2 2 2 2 2 2 2 2
268 044/300 Read block 35 len 16384
269 045/300 Write block 1370 len 32768 data: 2 2 2 2 2 2 2 2
270 046/300 Flush
271 047/300 Read block 2118 len 16384
272 048/300 Flush
273 049/300 Write block 4391 len 24576 data: 2 2 2 2 2 2
274 050/300 Read block 3503 len 20480
275 051/300 Read block 3711 len 4096
276 052/300 Read block 1508 len 4096
277 053/300 Read block 1868 len 36864
278 054/300 Read block 686 len 20480
279 055/300 Read block 1767 len 12288
280 056/300 Read block 3597 len 40960
281 057/300 Read block 3333 len 40960
282 058/300 Read block 1658 len 24576
283 059/300 Flush
284 060/300 Write block 4153 len 12288 data: 2 2 2
285 061/300 Write block 1456 len 20480 data: 2 2 2 2 2
286 062/300 Write block 3904 len 40960 data: 2 2 2 2 2 2 2 2 2 2
287 063/300 Write block 4331 len 32768 data: 2 2 2 2 2 2 2 2
288 064/300 Write block 4730 len 28672 data: 2 2 2 2 2 2 2
289 065/300 Flush
290 066/300 Read block 2957 len 20480
291 067/300 Read block 3315 len 8192
292 068/300 Flush
293 069/300 Read block 1262 len 4096
294 070/300 Write block 2068 len 4096 data: 2
295 071/300 Write block 3514 len 24576 data: 2 2 2 2 2 2
296 072/300 Write block 602 len 16384 data: 2 2 2 2
297 073/300 Write block 207 len 36864 data: 2 2 2 2 2 2 2 2 2
298 074/300 Read block 323 len 36864
299 075/300 Write block 3158 len 12288 data: 2 2 2
300 076/300 Write block 3426 len 32768 data: 2 2 2 2 2 2 2 2
301 077/300 Read block 4011 len 4096
302 078/300 Read block 63 len 12288
303 079/300 Read block 1225 len 16384
304 080/300 Flush
305 081/300 Read block 4267 len 28672
306 082/300 Read block 4834 len 36864
307 083/300 Write block 4260 len 28672 data: 2 2 2 2 2 2 2
308 084/300 Read block 2493 len 8192
309 085/300 Write block 1811 len 8192 data: 2 2
310 086/300 Read block 3864 len 36864
311 087/300 Read block 3224 len 24576
312 088/300 Write block 1794 len 16384 data: 2 2 2 2
313 089/300 Write block 4718 len 36864 data: 2 2 2 2 2 2 2 2 2
314 090/300 Write block 630 len 4096 data: 2
315 091/300 Read block 4134 len 32768
316 092/300 Write block 4238 len 24576 data: 2 2 2 2 2 2
317 093/300 Write block 2767 len 36864 data: 2 2 2 2 2 2 2 2 2
318 094/300 Read block 2352 len 36864
319 095/300 Read block 976 len 28672
320 096/300 Write block 3230 len 24576 data: 2 2 2 2 2 2
321 097/300 Flush
322 098/300 Read block 2959 len 16384
323 099/300 Read block 4219 len 8192
324 100/300 Write block 3773 len 24576 data: 2 2 2 2 2 2
325 101/300 Write block 1924 len 12288 data: 2 2 2
326 102/300 Read block 620 len 16384
327 103/300 Read block 3443 len 20480
328 104/300 Flush
329 105/300 Read block 2181 len 20480
330 106/300 Write block 4896 len 16384 data: 2 2 2 2
331 107/300 Write block 3757 len 24576 data: 2 2 2 2 2 2
332 108/300 Write block 1580 len 28672 data: 2 2 2 2 2 2 2
333 109/300 Read block 143 len 20480
334 110/300 Read block 1419 len 8192
335 111/300 Read block 2048 len 24576
336 112/300 Write block 1820 len 12288 data: 2 2 2
337 113/300 Read block 3256 len 16384
338 114/300 Read block 1382 len 16384
339 115/300 Flush
340 116/300 Read block 4360 len 32768
341 117/300 Write block 4799 len 28672 data: 2 2 2 2 2 2 2
342 118/300 Write block 1848 len 32768 data: 2 2 2 2 2 2 2 2
343 119/300 Read block 626 len 20480
344 120/300 Write block 2283 len 32768 data: 2 2 2 2 2 3 3 3
345 121/300 Read block 1893 len 16384
346 122/300 Write block 1102 len 24576 data: 2 2 2 2 2 2
347 123/300 Write block 3826 len 20480 data: 2 2 2 2 2
348 124/300 Write block 1575 len 8192 data: 2 2
349 125/300 Read block 2825 len 4096
350 126/300 Write block 4922 len 32768 data: 2 2 2 2 2 2 2 2
351 127/300 Write block 738 len 24576 data: 2 2 2 2 2 2
352 128/300 Write block 1233 len 36864 data: 2 2 2 2 2 2 2 2 2
353 129/300 Write block 4942 len 24576 data: 2 2 2 2 2 2
354 130/300 Write block 1551 len 4096 data: 2
355 131/300 Write block 1290 len 8192 data: 2 2
356 132/300 Read block 4085 len 24576
357 133/300 Write block 720 len 28672 data: 2 2 2 2 2 2 2
358 134/300 Write block 1943 len 24576 data: 2 2 2 2 2 2
359 135/300 Read block 3428 len 40960
360 136/300 Write block 3000 len 36864 data: 2 2 2 2 2 2 2 2 2
361 137/300 Write block 4560 len 40960 data: 2 2 2 2 2 2 2 2 2 2
362 138/300 Write block 2658 len 16384 data: 2 2 2 2
363 139/300 Read block 3276 len 32768
364 140/300 Write block 4107 len 28672 data: 2 2 2 2 2 2 2
365 141/300 Write block 274 len 36864 data: 2 2 2 2 2 2 2 2 2
366 142/300 Flush
367 143/300 Read block 2900 len 8192
368 144/300 Write block 4170 len 36864 data: 2 2 2 2 2 2 2 2 2
369 145/300 Write block 2638 len 28672 data: 2 2 2 2 2 2 2
370 146/300 Read block 3641 len 8192
371 147/300 Read block 2198 len 20480
372 148/300 Flush
373 149/300 Write block 4173 len 4096 data: 3
374 150/300 Read block 4626 len 20480
375 151/300 Read block 2587 len 40960
376 152/300 Flush
377 153/300 Write block 4109 len 20480 data: 3 3 3 3 3
378 154/300 Flush
379 155/300 Read block 566 len 8192
380 156/300 Write block 4196 len 8192 data: 2 2
381 157/300 Read block 785 len 16384
382 158/300 Read block 2783 len 40960
383 159/300 Write block 2042 len 28672 data: 2 2 2 2 2 2 2
384 160/300 Write block 3215 len 8192 data: 2 2
385 161/300 Write block 1018 len 12288 data: 2 2 2
386 162/300 Write block 43 len 32768 data: 2 2 2 2 2 2 2 2
387 163/300 Read block 3002 len 28672
388 164/300 Write block 656 len 16384 data: 2 2 2 2
389 165/300 Read block 2679 len 4096
390 166/300 Read block 256 len 36864
391 167/300 Write block 3904 len 4096 data: 3
392 168/300 Write block 3129 len 32768 data: 2 2 2 2 2 2 2 2
393 169/300 Write block 2339 len 40960 data: 2 2 2 2 2 2 2 2 2 2
394 170/300 Read block 2873 len 28672
395 171/300 Read block 1526 len 16384
396 172/300 Write block 1581 len 24576 data: 3 3 3 3 3 3
397 173/300 Read block 4431 len 24576
398 174/300 Read block 2525 len 8192
399 175/300 Read block 470 len 24576
400 176/300 Read block 4104 len 32768
401 177/300 Read block 385 len 36864
402 178/300 Read block 97 len 36864
403 179/300 Write block 505 len 8192 data: 2 2
404 180/300 Write block 4714 len 20480 data: 2 2 2 2 3
405 181/300 Read block 3608 len 32768
406 182/300 Read block 1694 len 28672
407 183/300 Read block 3868 len 28672
408 184/300 Write block 498 len 8192 data: 2 2
409 185/300 Write block 3176 len 16384 data: 2 2 2 2
410 186/300 Flush
411 187/300 Write block 3896 len 16384 data: 2 2 2 2
412 188/300 Write block 904 len 4096 data: 2
413 189/300 Write block 4375 len 36864 data: 2 2 2 2 2 2 2 2 2
414 190/300 Write block 3037 len 20480 data: 2 2 2 2 2
415 191/300 Write block 1571 len 32768 data: 2 2 2 2 3 3 2 2
416 192/300 Read block 1344 len 4096
417 193/300 Flush
418 194/300 Read block 4626 len 32768
419 195/300 Read block 814 len 8192
420 196/300 Write block 4988 len 8192 data: 2 2
421 197/300 Read block 969 len 20480
422 198/300 Flush
423 199/300 Flush
424 200/300 Read block 3899 len 16384
425 201/300 Read block 4361 len 8192
426 202/300 Write block 2248 len 16384 data: 2 2 2 2
427 203/300 Read block 2884 len 40960
428 204/300 Read block 1486 len 4096
429 205/300 Write block 1375 len 40960 data: 3 3 3 2 2 2 2 2 2 2
430 206/300 Write block 3123 len 20480 data: 2 2 2 2 2
431 207/300 Write block 3161 len 12288 data: 2 2 2
432 208/300 Read block 1772 len 40960
433 209/300 Write block 3218 len 32768 data: 2 2 2 2 2 2 2 2
434 210/300 Read block 4960 len 16384
435 211/300 Write block 2239 len 40960 data: 2 2 2 2 2 2 2 2 2 3
436 212/300 Read block 3621 len 12288
437 213/300 Write block 2470 len 4096 data: 2
438 214/300 Read block 1372 len 28672
439 215/300 Write block 800 len 4096 data: 2
440 216/300 Read block 2101 len 8192
441 217/300 Write block 1029 len 8192 data: 2 2
442 218/300 Read block 109 len 16384
443 219/300 Read block 3343 len 32768
444 220/300 Write block 4693 len 36864 data: 2 2 2 2 2 2 2 2 2
445 221/300 Read block 4623 len 36864
446 222/300 Write block 131 len 4096 data: 2
447 223/300 Read block 2011 len 8192
448 224/300 Read block 1051 len 32768
449 225/300 Flush
450 226/300 Write block 4887 len 8192 data: 2 2
451 227/300 Read block 2260 len 36864
452 228/300 Write block 713 len 20480 data: 2 2 2 2 2
453 229/300 Read block 1172 len 24576
454 230/300 Read block 3350 len 36864
455 231/300 Read block 1834 len 24576
456 232/300 Flush
457 233/300 Write block 1129 len 28672 data: 2 2 2 2 2 2 2
458 234/300 Write block 575 len 40960 data: 2 2 2 2 2 2 2 2 2 2
459 235/300 Write block 2848 len 24576 data: 2 2 2 2 2 2
460 236/300 Write block 4277 len 8192 data: 2 2
461 237/300 Write block 1537 len 32768 data: 3 3 3 3 3 3 4 4
462 238/300 Flush
463 239/300 Read block 4565 len 12288
464 240/300 Write block 1249 len 4096 data: 2
465 241/300 Read block 3901 len 28672
466 242/300 Write block 1810 len 8192 data: 2 3
467 243/300 Write block 4325 len 28672 data: 2 2 2 2 2 2 3
468 244/300 Flush
469 245/300 Read block 4194 len 24576
470 246/300 Write block 2794 len 36864 data: 2 2 2 2 2 2 2 2 2
471 247/300 Read block 2286 len 20480
472 248/300 Write block 1952 len 40960 data: 2 2 2 2 2 2 2 2 2 2
473 249/300 Flush
474 250/300 Read block 2422 len 32768
475 251/300 Write block 3122 len 16384 data: 2 3 3 3
476 252/300 Write block 1417 len 24576 data: 2 2 2 2 2 2
477 253/300 Write block 4498 len 12288 data: 2 2 2
478 254/300 Write block 2473 len 20480 data: 2 2 2 2 2
479 255/300 Read block 4071 len 4096
480 256/300 Read block 953 len 36864
481 257/300 Read block 3474 len 28672
482 258/300 Read block 3454 len 32768
483 259/300 Read block 4991 len 8192
484 260/300 Write block 4958 len 20480 data: 2 2 2 2 2
485 261/300 Read block 1947 len 4096
486 262/300 Read block 2321 len 8192
487 263/300 Flush
488 264/300 Write block 2576 len 24576 data: 2 2 2 2 2 2
489 265/300 Read block 2474 len 36864
490 266/300 Write block 3565 len 4096 data: 2
491 267/300 Write block 1534 len 8192 data: 2 2
492 268/300 Write block 4134 len 40960 data: 2 2 2 2 2 2 2 2 2 2
493 269/300 Read block 4547 len 24576
494 270/300 Write block 3403 len 28672 data: 2 2 2 2 2 2 2
495 271/300 Read block 1627 len 36864
496 272/300 Read block 2818 len 28672
497 273/300 Write block 1410 len 36864 data: 2 2 2 2 2 2 2 3 3
498 274/300 Read block 1301 len 32768
499 275/300 Write block 4728 len 24576 data: 2 2 3 3 3 3
500 276/300 Read block 4760 len 28672
501 277/300 Read block 1070 len 4096
502 278/300 Read block 2709 len 16384
503 279/300 Read block 1590 len 12288
504 280/300 Write block 1049 len 12288 data: 2 2 2
505 281/300 Read block 4667 len 8192
506 282/300 Read block 3053 len 8192
507 283/300 Read block 579 len 24576
508 284/300 Write block 4790 len 28672 data: 2 2 2 2 2 2 2
509 285/300 Write block 1398 len 36864 data: 2 2 2 2 2 2 2 2 2
510 286/300 Read block 802 len 20480
511 287/300 Read block 2104 len 8192
512 288/300 Write block 4954 len 8192 data: 2 2
513 289/300 Read block 2288 len 36864
514 290/300 Read block 2166 len 20480
515 291/300 Write block 186 len 40960 data: 2 2 2 2 2 2 2 2 2 2
516 292/300 Write block 1641 len 24576 data: 2 2 2 2 2 2
517 293/300 Write block 2673 len 36864 data: 2 2 2 2 2 2 2 2 2
518 294/300 Read block 4729 len 8192
519 295/300 Read block 1371 len 12288
520 296/300 Write block 4492 len 20480 data: 2 2 2 2 2
521 297/300 Write block 856 len 28672 data: 2 2 2 2 2 2 2
522 298/300 Write block 2633 len 40960 data: 2 2 2 2 2 3 3 3 3 3
523 299/300 Write block 1391 len 20480 data: 2 2 2 2 2
524 300/300 Write block 1342 len 40960 data: 2 2 2 2 2 2 2 2 2 2
525 301/300 Write block 1785 len 28672 data: 2 2 2 2 2 2 2
526 Main task has work to do, go find it
527 got dsc Start(0)
528 start 0
529 Jan 27 19:36:29.154 INFO request completed, latency_us: 234, response_code: 204, uri: /start/cid/0, method: POST, req_id: 13fd5f6b-12bc-49c9-b874-0587d3d1a304, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
530 Replay: started 0, returned:Ok(())
531 ----------------------------------------------------------------
532 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
533 States: Offline Active Active
534 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
535 1000 Acked Read 8 Sent Done Done false
536 1001 Acked Flush 0 Sent Done Done false
537 1002 Acked Read 7 Sent Done Done false
538 1003 Acked Read 6 Sent Done Done false
539 1004 Acked Write 9 Sent Done Done false
540 1005 Acked Read 8 Sent Done Done false
541 1006 Acked Write 3 Sent Done Done false
542 1007 Acked Write 1 Sent Done Done false
543 1008 Acked Read 6 Sent Done Done false
544 1009 Acked Write 2 Sent Done Done false
545 1010 Acked Write 8 Sent Done Done false
546 1011 Acked Read 5 Sent Done Done false
547 1012 Acked Read 4 Sent Done Done false
548 1013 Acked Write 5 Sent Done Done false
549 1014 Acked Write 9 Sent Done Done false
550 1015 Acked Read 4 Sent Done Done false
551 1016 Acked Write 2 Sent Done Done false
552 1017 Acked Read 9 Sent Done Done false
553 1018 Acked Write 8 Sent Done Done false
554 1019 Acked Read 10 Sent Done Done false
555 1020 Acked Read 2 Sent Done Done false
556 1021 Acked Write 5 Sent Done Done false
557 1022 Acked Write 7 Sent Done Done false
558 1023 Acked Read 2 Sent Done Done false
559 1024 Acked Write 1 Sent Done Done false
560 1025 Acked Write 7 Sent Done Done false
561 1026 Acked Write 5 Sent Done Done false
562 1027 Acked Write 3 Sent Done Done false
563 1028 Acked Flush 0 Sent Done Done false
564 1029 Acked Write 4 Sent Done Done false
565 1030 Acked Read 4 Sent Done Done false
566 1031 Acked Flush 0 Sent Done Done false
567 1032 Acked Read 8 Sent Done Done false
568 1033 Acked Write 1 Sent Done Done false
569 1034 Acked Write 5 Sent Done Done false
570 1035 Acked Write 9 Sent Done Done false
571 1036 Acked Write 9 Sent Done Done false
572 1037 Acked Flush 0 Sent Done Done false
573 1038 Acked Flush 0 Sent Done Done false
574 1039 Acked Write 2 Sent Done Done false
575 1040 Acked Read 6 Sent Done Done false
576 1041 Acked Read 1 Sent Done Done false
577 1042 Acked Write 8 Sent Done Done false
578 1043 Acked Read 4 Sent Done Done false
579 1044 Acked Write 8 Sent Done Done false
580 1045 Acked Flush 0 Sent Done Done false
581 1046 Acked Read 4 Sent Done Done false
582 1047 Acked Flush 0 Sent Done Done false
583 1048 Acked Write 6 Sent Done Done false
584 1049 Acked Read 5 Sent Done Done false
585 1050 Acked Read 1 Sent Done Done false
586 1051 Acked Read 1 Sent Done Done false
587 1052 Acked Read 9 Sent Done Done false
588 1053 Acked Read 5 Sent Done Done false
589 1054 Acked Read 3 Sent Done Done false
590 1055 Acked Read 10 Sent Done Done false
591 1056 Acked Read 10 Sent Done Done false
592 1057 Acked Read 6 Sent Done Done false
593 1058 Acked Flush 0 Sent Done Done false
594 1059 Acked Write 3 Sent Done Done false
595 1060 Acked Write 5 Sent Done Done false
596 1061 Acked Write 10 Sent Done Done false
597 1062 Acked Write 8 Sent Done Done false
598 1063 Acked Write 7 Sent Done Done false
599 1064 Acked Flush 0 Sent Done Done false
600 1065 Acked Read 5 Sent Done Done false
601 1066 Acked Read 2 Sent Done Done false
602 1067 Acked Flush 0 Sent Done Done false
603 1068 Acked Read 1 Sent Done Done false
604 1069 Acked Write 1 Sent Done Done false
605 1070 Acked Write 6 Sent Done Done false
606 1071 Acked Write 4 Sent Done Done false
607 1072 Acked Write 9 Sent Done Done false
608 1073 Acked Read 9 Sent Done Done false
609 1074 Acked Write 3 Sent Done Done false
610 1075 Acked Write 8 Sent Done Done false
611 1076 Acked Read 1 Sent Done Done false
612 1077 Acked Read 3 Sent Done Done false
613 1078 Acked Read 4 Sent Done Done false
614 1079 Acked Flush 0 Sent Done Done false
615 1080 Acked Read 7 Sent Done Done false
616 1081 Acked Read 9 Sent Done Done false
617 1082 Acked Write 7 Sent Done Done false
618 1083 Acked Read 2 Sent Done Done false
619 1084 Acked Write 2 Sent Done Done false
620 1085 Acked Read 9 Sent Done Done false
621 1086 Acked Read 6 Sent Done Done false
622 1087 Acked Write 4 Sent Done Done false
623 1088 Acked Write 9 Sent Done Done false
624 1089 Acked Write 1 Sent Done Done false
625 1090 Acked Read 8 Sent Done Done false
626 1091 Acked Write 6 Sent Done Done false
627 1092 Acked Write 9 Sent Done Done false
628 1093 Acked Read 9 Sent Done Done false
629 1094 Acked Read 7 Sent Done Done false
630 1095 Acked Write 6 Sent Done Done false
631 1096 Acked Flush 0 Sent Done Done false
632 1097 Acked Read 4 Sent Done Done false
633 1098 Acked Read 2 Sent Done Done false
634 1099 Acked Write 6 Sent Done Done false
635 1100 Acked Write 3 Sent Done Done false
636 1101 Acked Read 4 Sent Done Done false
637 1102 Acked Read 5 Sent Done Done false
638 1103 Acked Flush 0 Sent Done Done false
639 1104 Acked Read 5 Sent Done Done false
640 1105 Acked Write 4 Sent Done Done false
641 1106 Acked Write 6 Sent Done Done false
642 1107 Acked Write 7 Sent Done Done false
643 1108 Acked Read 5 Sent Done Done false
644 1109 Acked Read 2 Sent Done Done false
645 1110 Acked Read 6 Sent Done Done false
646 1111 Acked Write 3 Sent Done Done false
647 1112 Acked Read 4 Sent Done Done false
648 1113 Acked Read 4 Sent Done Done false
649 1114 Acked Flush 0 Sent Done Done false
650 1115 Acked Read 8 Sent Done Done false
651 1116 Acked Write 7 Sent Done Done false
652 1117 Acked Write 8 Sent Done Done false
653 1118 Acked Read 5 Sent Done Done false
654 1119 Acked Write 8 Sent Done Done false
655 1120 Acked Read 4 Sent Done Done false
656 1121 Acked Write 6 Sent Done Done false
657 1122 Acked Write 5 Sent Done Done false
658 1123 Acked Write 2 Sent Done Done false
659 1124 Acked Read 1 Sent Done Done false
660 1125 Acked Write 8 Sent Done Done false
661 1126 Acked Write 6 Sent Done Done false
662 1127 Acked Write 9 Sent Done Done false
663 1128 Acked Write 6 Sent Done Done false
664 1129 Acked Write 1 Sent Done Done false
665 1130 Acked Write 2 Sent Done Done false
666 1131 Acked Read 6 Sent Done Done false
667 1132 Acked Write 7 Sent Done Done false
668 1133 Acked Write 6 Sent Done Done false
669 1134 Acked Read 10 Sent Done Done false
670 1135 Acked Write 9 Sent Done Done false
671 1136 Acked Write 10 Sent Done Done false
672 1137 Acked Write 4 Sent Done Done false
673 1138 Acked Read 8 Sent Done Done false
674 1139 Acked Write 7 Sent Done Done false
675 1140 Acked Write 9 Sent Done Done false
676 1141 Acked Flush 0 Sent Done Done false
677 1142 Acked Read 2 Sent Done Done false
678 1143 Acked Write 9 Sent Done Done false
679 1144 Acked Write 7 Sent Done Done false
680 1145 Acked Read 2 Sent Done Done false
681 1146 Acked Read 5 Sent Done Done false
682 1147 Acked Flush 0 Sent Done Done false
683 1148 Acked Write 1 Sent Done Done false
684 1149 Acked Read 5 Sent Done Done false
685 1150 Acked Read 10 Sent Done Done false
686 1151 Acked Flush 0 Sent Done Done false
687 1152 Acked Write 5 Sent Done Done false
688 1153 Acked Flush 0 Sent Done Done false
689 1154 Acked Read 2 Sent Done Done false
690 1155 Acked Write 2 Sent Done Done false
691 1156 Acked Read 4 Sent Done Done false
692 1157 Acked Read 10 Sent Done Done false
693 1158 Acked Write 7 Sent Done Done false
694 1159 Acked Write 2 Sent Done Done false
695 1160 Acked Write 3 Sent Done Done false
696 1161 Acked Write 8 Sent Done Done false
697 1162 Acked Read 7 Sent Done Done false
698 1163 Acked Write 4 Sent Done Done false
699 1164 Acked Read 1 Sent Done Done false
700 1165 Acked Read 9 Sent Done Done false
701 1166 Acked Write 1 Sent Done Done false
702 1167 Acked Write 8 Sent Done Done false
703 1168 Acked Write 10 Sent Done Done false
704 1169 Acked Read 7 Sent Done Done false
705 1170 Acked Read 4 Sent Done Done false
706 1171 Acked Write 6 Sent Done Done false
707 1172 Acked Read 6 Sent Done Done false
708 1173 Acked Read 2 Sent Done Done false
709 1174 Acked Read 6 Sent Done Done false
710 1175 Acked Read 8 Sent Done Done false
711 1176 Acked Read 9 Sent Done Done false
712 1177 Acked Read 9 Sent Done Done false
713 1178 Acked Write 2 Sent Done Done false
714 1179 Acked Write 5 Sent Done Done false
715 1180 Acked Read 8 Sent Done Done false
716 1181 Acked Read 7 Sent Done Done false
717 1182 Acked Read 7 Sent Done Done false
718 1183 Acked Write 2 Sent Done Done false
719 1184 Acked Write 4 Sent Done Done false
720 1185 Acked Flush 0 Sent Done Done false
721 1186 Acked Write 4 Sent Done Done false
722 1187 Acked Write 1 Sent Done Done false
723 1188 Acked Write 9 Sent Done Done false
724 1189 Acked Write 5 Sent Done Done false
725 1190 Acked Write 8 Sent Done Done false
726 1191 Acked Read 1 Sent Done Done false
727 1192 Acked Flush 0 Sent Done Done false
728 1193 Acked Read 8 Sent Done Done false
729 1194 Acked Read 2 Sent Done Done false
730 1195 Acked Write 2 Sent Done Done false
731 1196 Acked Read 5 Sent Done Done false
732 1197 Acked Flush 0 Sent Done Done false
733 1198 Acked Flush 0 Sent Done Done false
734 1199 Acked Read 4 Sent Done Done false
735 1200 Acked Read 2 Sent Done Done false
736 1201 Acked Write 4 Sent Done Done false
737 1202 Acked Read 10 Sent Done Done false
738 1203 Acked Read 1 Sent Done Done false
739 1204 Acked Write 10 Sent Done Done false
740 1205 Acked Write 5 Sent Done Done false
741 1206 Acked Write 3 Sent Done Done false
742 1207 Acked Read 10 Sent Done Done false
743 1208 Acked Write 8 Sent Done Done false
744 1209 Acked Read 4 Sent Done Done false
745 1210 Acked Write 10 Sent Done Done false
746 1211 Acked Read 3 Sent Done Done false
747 1212 Acked Write 1 Sent Done Done false
748 1213 Acked Read 7 Sent Done Done false
749 1214 Acked Write 1 Sent Done Done false
750 1215 Acked Read 2 Sent Done Done false
751 1216 Acked Write 2 Sent Done Done false
752 1217 Acked Read 4 Sent Done Done false
753 1218 Acked Read 8 Sent Done Done false
754 1219 Acked Write 9 Sent Done Done false
755 1220 Acked Read 9 Sent Done Done false
756 1221 Acked Write 1 Sent Done Done false
757 1222 Acked Read 2 Sent Done Done false
758 1223 Acked Read 8 Sent Done Done false
759 1224 Acked Flush 0 Sent Done Done false
760 1225 Acked Write 2 Sent Done Done false
761 1226 Acked Read 9 Sent Done Done false
762 1227 Acked Write 5 Sent Done Done false
763 1228 Acked Read 6 Sent Done Done false
764 1229 Acked Read 9 Sent Done Done false
765 1230 Acked Read 6 Sent Done Done false
766 1231 Acked Flush 0 Sent Done Done false
767 1232 Acked Write 7 Sent Done Done false
768 1233 Acked Write 10 Sent Done Done false
769 1234 Acked Write 6 Sent Done Done false
770 1235 Acked Write 2 Sent Done Done false
771 1236 Acked Write 8 Sent Done Done false
772 1237 Acked Flush 0 Sent Done Done false
773 1238 Acked Read 3 Sent Done Done false
774 1239 Acked Write 1 Sent Done Done false
775 1240 Acked Read 7 Sent Done Done false
776 1241 Acked Write 2 Sent Done Done false
777 1242 Acked Write 7 Sent Done Done false
778 1243 Acked Flush 0 Sent Done Done false
779 1244 Acked Read 6 Sent Done Done false
780 1245 Acked Write 9 Sent Done Done false
781 1246 Acked Read 5 Sent Done Done false
782 1247 Acked Write 10 Sent Done Done false
783 1248 Acked Flush 0 Sent Done Done false
784 1249 Acked Read 8 Sent Done Done false
785 1250 Acked Write 4 Sent Done Done false
786 1251 Acked Write 6 Sent Done Done false
787 1252 Acked Write 3 Sent Done Done false
788 1253 Acked Write 5 Sent Done Done false
789 1254 Acked Read 1 Sent Done Done false
790 1255 Acked Read 9 Sent Done Done false
791 1256 Acked Read 7 Sent Done Done false
792 1257 Acked Read 8 Sent Done Done false
793 1258 Acked Read 2 Sent Done Done false
794 1259 Acked Write 5 Sent Done Done false
795 1260 Acked Read 1 Sent Done Done false
796 1261 Acked Read 2 Sent Done Done false
797 1262 Acked Flush 0 Sent Done Done false
798 1263 Acked Write 6 Sent Done Done false
799 1264 Acked Read 9 Sent Done Done false
800 1265 Acked Write 1 Sent Done Done false
801 1266 Acked Write 2 Sent Done Done false
802 1267 Acked Write 10 Sent Done Done false
803 1268 Acked Read 6 Sent Done Done false
804 1269 Acked Write 7 Sent Done Done false
805 1270 Acked Read 9 Sent Done Done false
806 1271 Acked Read 7 Sent Done Done false
807 1272 Acked Write 9 Sent Done Done false
808 1273 Acked Read 8 Sent Done Done false
809 1274 Acked Write 6 Sent Done Done false
810 1275 Acked Read 7 Sent Done Done false
811 1276 Acked Read 1 Sent Done Done false
812 1277 Acked Read 4 Sent Done Done false
813 1278 Acked Read 3 Sent Done Done false
814 1279 Acked Write 3 Sent Done Done false
815 1280 Acked Read 2 Sent Done Done false
816 1281 Acked Read 2 Sent Done Done false
817 1282 Acked Read 6 Sent Done Done false
818 1283 Acked Write 7 Sent Done Done false
819 1284 Acked Write 9 Sent Done Done false
820 1285 Acked Read 5 Sent Done Done false
821 1286 Acked Read 2 Sent Done Done false
822 1287 Acked Write 2 Sent Done Done false
823 1288 Acked Read 9 Sent Done Done false
824 1289 Acked Read 5 Sent Done Done false
825 1290 Acked Write 10 Sent Done Done false
826 1291 Acked Write 6 Sent Done Done false
827 1292 Acked Write 9 Sent Done Done false
828 1293 Acked Read 2 Sent Done Done false
829 1294 Acked Read 3 Sent Done Done false
830 1295 Acked Write 5 Sent Done Done false
831 1296 Acked Write 7 Sent Done Done false
832 1297 Acked Write 10 Sent Done Done false
833 1298 Acked Write 5 Sent Done Done false
834 1299 Acked Write 10 Sent Done Done false
835 1300 Acked Write 7 Sent Done Done false
836 STATES DS:0 DS:1 DS:2 TOTAL
837 Sent 301 0 0 301
838 Done 0 301 301 602
839 Skipped 0 0 0 0
840 Error 0 0 0 0
841 Last Flush: 0 1262 1262
842 Downstairs last five completed:
843 Upstairs last five completed: 1300 1299 1298 1297 1296
844 CLIENT: Up:0 ds:301 act:2
845 [8810] got start action
846 [8810] I am going to restart
847 Starting downstairs at port 8810
848 Make output file at "/tmp/dsc/downstairs-8810.txt"
849 [8810][0] reports Starting
850 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1201)
851 [8810][0] reports Running
852 ----------------------------------------------------------------
853 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
854 States: Offline Active Active
855 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
856 1000 Acked Read 8 Sent Done Done false
857 1001 Acked Flush 0 Sent Done Done false
858 1002 Acked Read 7 Sent Done Done false
859 1003 Acked Read 6 Sent Done Done false
860 1004 Acked Write 9 Sent Done Done false
861 1005 Acked Read 8 Sent Done Done false
862 1006 Acked Write 3 Sent Done Done false
863 1007 Acked Write 1 Sent Done Done false
864 1008 Acked Read 6 Sent Done Done false
865 1009 Acked Write 2 Sent Done Done false
866 1010 Acked Write 8 Sent Done Done false
867 1011 Acked Read 5 Sent Done Done false
868 1012 Acked Read 4 Sent Done Done false
869 1013 Acked Write 5 Sent Done Done false
870 1014 Acked Write 9 Sent Done Done false
871 1015 Acked Read 4 Sent Done Done false
872 1016 Acked Write 2 Sent Done Done false
873 1017 Acked Read 9 Sent Done Done false
874 1018 Acked Write 8 Sent Done Done false
875 1019 Acked Read 10 Sent Done Done false
876 1020 Acked Read 2 Sent Done Done false
877 1021 Acked Write 5 Sent Done Done false
878 1022 Acked Write 7 Sent Done Done false
879 1023 Acked Read 2 Sent Done Done false
880 1024 Acked Write 1 Sent Done Done false
881 1025 Acked Write 7 Sent Done Done false
882 1026 Acked Write 5 Sent Done Done false
883 1027 Acked Write 3 Sent Done Done false
884 1028 Acked Flush 0 Sent Done Done false
885 1029 Acked Write 4 Sent Done Done false
886 1030 Acked Read 4 Sent Done Done false
887 1031 Acked Flush 0 Sent Done Done false
888 1032 Acked Read 8 Sent Done Done false
889 1033 Acked Write 1 Sent Done Done false
890 1034 Acked Write 5 Sent Done Done false
891 1035 Acked Write 9 Sent Done Done false
892 1036 Acked Write 9 Sent Done Done false
893 1037 Acked Flush 0 Sent Done Done false
894 1038 Acked Flush 0 Sent Done Done false
895 1039 Acked Write 2 Sent Done Done false
896 1040 Acked Read 6 Sent Done Done false
897 1041 Acked Read 1 Sent Done Done false
898 1042 Acked Write 8 Sent Done Done false
899 1043 Acked Read 4 Sent Done Done false
900 1044 Acked Write 8 Sent Done Done false
901 1045 Acked Flush 0 Sent Done Done false
902 1046 Acked Read 4 Sent Done Done false
903 1047 Acked Flush 0 Sent Done Done false
904 1048 Acked Write 6 Sent Done Done false
905 1049 Acked Read 5 Sent Done Done false
906 1050 Acked Read 1 Sent Done Done false
907 1051 Acked Read 1 Sent Done Done false
908 1052 Acked Read 9 Sent Done Done false
909 1053 Acked Read 5 Sent Done Done false
910 1054 Acked Read 3 Sent Done Done false
911 1055 Acked Read 10 Sent Done Done false
912 1056 Acked Read 10 Sent Done Done false
913 1057 Acked Read 6 Sent Done Done false
914 1058 Acked Flush 0 Sent Done Done false
915 1059 Acked Write 3 Sent Done Done false
916 1060 Acked Write 5 Sent Done Done false
917 1061 Acked Write 10 Sent Done Done false
918 1062 Acked Write 8 Sent Done Done false
919 1063 Acked Write 7 Sent Done Done false
920 1064 Acked Flush 0 Sent Done Done false
921 1065 Acked Read 5 Sent Done Done false
922 1066 Acked Read 2 Sent Done Done false
923 1067 Acked Flush 0 Sent Done Done false
924 1068 Acked Read 1 Sent Done Done false
925 1069 Acked Write 1 Sent Done Done false
926 1070 Acked Write 6 Sent Done Done false
927 1071 Acked Write 4 Sent Done Done false
928 1072 Acked Write 9 Sent Done Done false
929 1073 Acked Read 9 Sent Done Done false
930 1074 Acked Write 3 Sent Done Done false
931 1075 Acked Write 8 Sent Done Done false
932 1076 Acked Read 1 Sent Done Done false
933 1077 Acked Read 3 Sent Done Done false
934 1078 Acked Read 4 Sent Done Done false
935 1079 Acked Flush 0 Sent Done Done false
936 1080 Acked Read 7 Sent Done Done false
937 1081 Acked Read 9 Sent Done Done false
938 1082 Acked Write 7 Sent Done Done false
939 1083 Acked Read 2 Sent Done Done false
940 1084 Acked Write 2 Sent Done Done false
941 1085 Acked Read 9 Sent Done Done false
942 1086 Acked Read 6 Sent Done Done false
943 1087 Acked Write 4 Sent Done Done false
944 1088 Acked Write 9 Sent Done Done false
945 1089 Acked Write 1 Sent Done Done false
946 1090 Acked Read 8 Sent Done Done false
947 1091 Acked Write 6 Sent Done Done false
948 1092 Acked Write 9 Sent Done Done false
949 1093 Acked Read 9 Sent Done Done false
950 1094 Acked Read 7 Sent Done Done false
951 1095 Acked Write 6 Sent Done Done false
952 1096 Acked Flush 0 Sent Done Done false
953 1097 Acked Read 4 Sent Done Done false
954 1098 Acked Read 2 Sent Done Done false
955 1099 Acked Write 6 Sent Done Done false
956 1100 Acked Write 3 Sent Done Done false
957 1101 Acked Read 4 Sent Done Done false
958 1102 Acked Read 5 Sent Done Done false
959 1103 Acked Flush 0 Sent Done Done false
960 1104 Acked Read 5 Sent Done Done false
961 1105 Acked Write 4 Sent Done Done false
962 1106 Acked Write 6 Sent Done Done false
963 1107 Acked Write 7 Sent Done Done false
964 1108 Acked Read 5 Sent Done Done false
965 1109 Acked Read 2 Sent Done Done false
966 1110 Acked Read 6 Sent Done Done false
967 1111 Acked Write 3 Sent Done Done false
968 1112 Acked Read 4 Sent Done Done false
969 1113 Acked Read 4 Sent Done Done false
970 1114 Acked Flush 0 Sent Done Done false
971 1115 Acked Read 8 Sent Done Done false
972 1116 Acked Write 7 Sent Done Done false
973 1117 Acked Write 8 Sent Done Done false
974 1118 Acked Read 5 Sent Done Done false
975 1119 Acked Write 8 Sent Done Done false
976 1120 Acked Read 4 Sent Done Done false
977 1121 Acked Write 6 Sent Done Done false
978 1122 Acked Write 5 Sent Done Done false
979 1123 Acked Write 2 Sent Done Done false
980 1124 Acked Read 1 Sent Done Done false
981 1125 Acked Write 8 Sent Done Done false
982 1126 Acked Write 6 Sent Done Done false
983 1127 Acked Write 9 Sent Done Done false
984 1128 Acked Write 6 Sent Done Done false
985 1129 Acked Write 1 Sent Done Done false
986 1130 Acked Write 2 Sent Done Done false
987 1131 Acked Read 6 Sent Done Done false
988 1132 Acked Write 7 Sent Done Done false
989 1133 Acked Write 6 Sent Done Done false
990 1134 Acked Read 10 Sent Done Done false
991 1135 Acked Write 9 Sent Done Done false
992 1136 Acked Write 10 Sent Done Done false
993 1137 Acked Write 4 Sent Done Done false
994 1138 Acked Read 8 Sent Done Done false
995 1139 Acked Write 7 Sent Done Done false
996 1140 Acked Write 9 Sent Done Done false
997 1141 Acked Flush 0 Sent Done Done false
998 1142 Acked Read 2 Sent Done Done false
999 1143 Acked Write 9 Sent Done Done false
1000 1144 Acked Write 7 Sent Done Done false
1001 1145 Acked Read 2 Sent Done Done false
1002 1146 Acked Read 5 Sent Done Done false
1003 1147 Acked Flush 0 Sent Done Done false
1004 1148 Acked Write 1 Sent Done Done false
1005 1149 Acked Read 5 Sent Done Done false
1006 1150 Acked Read 10 Sent Done Done false
1007 1151 Acked Flush 0 Sent Done Done false
1008 1152 Acked Write 5 Sent Done Done false
1009 1153 Acked Flush 0 Sent Done Done false
1010 1154 Acked Read 2 Sent Done Done false
1011 1155 Acked Write 2 Sent Done Done false
1012 1156 Acked Read 4 Sent Done Done false
1013 1157 Acked Read 10 Sent Done Done false
1014 1158 Acked Write 7 Sent Done Done false
1015 1159 Acked Write 2 Sent Done Done false
1016 1160 Acked Write 3 Sent Done Done false
1017 1161 Acked Write 8 Sent Done Done false
1018 1162 Acked Read 7 Sent Done Done false
1019 1163 Acked Write 4 Sent Done Done false
1020 1164 Acked Read 1 Sent Done Done false
1021 1165 Acked Read 9 Sent Done Done false
1022 1166 Acked Write 1 Sent Done Done false
1023 1167 Acked Write 8 Sent Done Done false
1024 1168 Acked Write 10 Sent Done Done false
1025 1169 Acked Read 7 Sent Done Done false
1026 1170 Acked Read 4 Sent Done Done false
1027 1171 Acked Write 6 Sent Done Done false
1028 1172 Acked Read 6 Sent Done Done false
1029 1173 Acked Read 2 Sent Done Done false
1030 1174 Acked Read 6 Sent Done Done false
1031 1175 Acked Read 8 Sent Done Done false
1032 1176 Acked Read 9 Sent Done Done false
1033 1177 Acked Read 9 Sent Done Done false
1034 1178 Acked Write 2 Sent Done Done false
1035 1179 Acked Write 5 Sent Done Done false
1036 1180 Acked Read 8 Sent Done Done false
1037 1181 Acked Read 7 Sent Done Done false
1038 1182 Acked Read 7 Sent Done Done false
1039 1183 Acked Write 2 Sent Done Done false
1040 1184 Acked Write 4 Sent Done Done false
1041 1185 Acked Flush 0 Sent Done Done false
1042 1186 Acked Write 4 Sent Done Done false
1043 1187 Acked Write 1 Sent Done Done false
1044 1188 Acked Write 9 Sent Done Done false
1045 1189 Acked Write 5 Sent Done Done false
1046 1190 Acked Write 8 Sent Done Done false
1047 1191 Acked Read 1 Sent Done Done false
1048 1192 Acked Flush 0 Sent Done Done false
1049 1193 Acked Read 8 Sent Done Done false
1050 1194 Acked Read 2 Sent Done Done false
1051 1195 Acked Write 2 Sent Done Done false
1052 1196 Acked Read 5 Sent Done Done false
1053 1197 Acked Flush 0 Sent Done Done false
1054 1198 Acked Flush 0 Sent Done Done false
1055 1199 Acked Read 4 Sent Done Done false
1056 1200 Acked Read 2 Sent Done Done false
1057 1201 Acked Write 4 Sent Done Done false
1058 1202 Acked Read 10 Sent Done Done false
1059 1203 Acked Read 1 Sent Done Done false
1060 1204 Acked Write 10 Sent Done Done false
1061 1205 Acked Write 5 Sent Done Done false
1062 1206 Acked Write 3 Sent Done Done false
1063 1207 Acked Read 10 Sent Done Done false
1064 1208 Acked Write 8 Sent Done Done false
1065 1209 Acked Read 4 Sent Done Done false
1066 1210 Acked Write 10 Sent Done Done false
1067 1211 Acked Read 3 Sent Done Done false
1068 1212 Acked Write 1 Sent Done Done false
1069 1213 Acked Read 7 Sent Done Done false
1070 1214 Acked Write 1 Sent Done Done false
1071 1215 Acked Read 2 Sent Done Done false
1072 1216 Acked Write 2 Sent Done Done false
1073 1217 Acked Read 4 Sent Done Done false
1074 1218 Acked Read 8 Sent Done Done false
1075 1219 Acked Write 9 Sent Done Done false
1076 1220 Acked Read 9 Sent Done Done false
1077 1221 Acked Write 1 Sent Done Done false
1078 1222 Acked Read 2 Sent Done Done false
1079 1223 Acked Read 8 Sent Done Done false
1080 1224 Acked Flush 0 Sent Done Done false
1081 1225 Acked Write 2 Sent Done Done false
1082 1226 Acked Read 9 Sent Done Done false
1083 1227 Acked Write 5 Sent Done Done false
1084 1228 Acked Read 6 Sent Done Done false
1085 1229 Acked Read 9 Sent Done Done false
1086 1230 Acked Read 6 Sent Done Done false
1087 1231 Acked Flush 0 Sent Done Done false
1088 1232 Acked Write 7 Sent Done Done false
1089 1233 Acked Write 10 Sent Done Done false
1090 1234 Acked Write 6 Sent Done Done false
1091 1235 Acked Write 2 Sent Done Done false
1092 1236 Acked Write 8 Sent Done Done false
1093 1237 Acked Flush 0 Sent Done Done false
1094 1238 Acked Read 3 Sent Done Done false
1095 1239 Acked Write 1 Sent Done Done false
1096 1240 Acked Read 7 Sent Done Done false
1097 1241 Acked Write 2 Sent Done Done false
1098 1242 Acked Write 7 Sent Done Done false
1099 1243 Acked Flush 0 Sent Done Done false
1100 1244 Acked Read 6 Sent Done Done false
1101 1245 Acked Write 9 Sent Done Done false
1102 1246 Acked Read 5 Sent Done Done false
1103 1247 Acked Write 10 Sent Done Done false
1104 1248 Acked Flush 0 Sent Done Done false
1105 1249 Acked Read 8 Sent Done Done false
1106 1250 Acked Write 4 Sent Done Done false
1107 1251 Acked Write 6 Sent Done Done false
1108 1252 Acked Write 3 Sent Done Done false
1109 1253 Acked Write 5 Sent Done Done false
1110 1254 Acked Read 1 Sent Done Done false
1111 1255 Acked Read 9 Sent Done Done false
1112 1256 Acked Read 7 Sent Done Done false
1113 1257 Acked Read 8 Sent Done Done false
1114 1258 Acked Read 2 Sent Done Done false
1115 1259 Acked Write 5 Sent Done Done false
1116 1260 Acked Read 1 Sent Done Done false
1117 1261 Acked Read 2 Sent Done Done false
1118 1262 Acked Flush 0 Sent Done Done false
1119 1263 Acked Write 6 Sent Done Done false
1120 1264 Acked Read 9 Sent Done Done false
1121 1265 Acked Write 1 Sent Done Done false
1122 1266 Acked Write 2 Sent Done Done false
1123 1267 Acked Write 10 Sent Done Done false
1124 1268 Acked Read 6 Sent Done Done false
1125 1269 Acked Write 7 Sent Done Done false
1126 1270 Acked Read 9 Sent Done Done false
1127 1271 Acked Read 7 Sent Done Done false
1128 1272 Acked Write 9 Sent Done Done false
1129 1273 Acked Read 8 Sent Done Done false
1130 1274 Acked Write 6 Sent Done Done false
1131 1275 Acked Read 7 Sent Done Done false
1132 1276 Acked Read 1 Sent Done Done false
1133 1277 Acked Read 4 Sent Done Done false
1134 1278 Acked Read 3 Sent Done Done false
1135 1279 Acked Write 3 Sent Done Done false
1136 1280 Acked Read 2 Sent Done Done false
1137 1281 Acked Read 2 Sent Done Done false
1138 1282 Acked Read 6 Sent Done Done false
1139 1283 Acked Write 7 Sent Done Done false
1140 1284 Acked Write 9 Sent Done Done false
1141 1285 Acked Read 5 Sent Done Done false
1142 1286 Acked Read 2 Sent Done Done false
1143 1287 Acked Write 2 Sent Done Done false
1144 1288 Acked Read 9 Sent Done Done false
1145 1289 Acked Read 5 Sent Done Done false
1146 1290 Acked Write 10 Sent Done Done false
1147 1291 Acked Write 6 Sent Done Done false
1148 1292 Acked Write 9 Sent Done Done false
1149 1293 Acked Read 2 Sent Done Done false
1150 1294 Acked Read 3 Sent Done Done false
1151 1295 Acked Write 5 Sent Done Done false
1152 1296 Acked Write 7 Sent Done Done false
1153 1297 Acked Write 10 Sent Done Done false
1154 1298 Acked Write 5 Sent Done Done false
1155 1299 Acked Write 10 Sent Done Done false
1156 1300 Acked Write 7 Sent Done Done false
1157 1301 Acked Flush 0 Sent Done Done false
1158 STATES DS:0 DS:1 DS:2 TOTAL
1159 Sent 302 0 0 302
1160 Done 0 302 302 604
1161 Skipped 0 0 0 0
1162 Error 0 0 0 0
1163 Last Flush: 0 1301 1301
1164 Downstairs last five completed:
1165 Upstairs last five completed: 1301 1300 1299 1298 1297
1166 CLIENT: Up:0 ds:302 act:2
11672025-01-27T19:36:34.927ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11682025-01-27T19:36:34.927ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58544) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11692025-01-27T19:36:34.928ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11702025-01-27T19:36:34.929ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11712025-01-27T19:36:34.929ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11722025-01-27T19:36:34.929ZINFOcrucible: send last flush ID to this DS: 0 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11732025-01-27T19:36:34.929ZINFOcrucible: Replied this last flush ID: 0 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11742025-01-27T19:36:34.929ZINFOcrucible: [0] client re-new 302 jobs since flush 0 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11752025-01-27T19:36:34.930ZINFOcrucible: [0] Replayed 302 jobs since flush: 0 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1176 ----------------------------------------------------------------
1177 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
1178 Downstairs last five completed: 1301 1300 1299 1298 1297
1179 Upstairs last five completed: 1301 1300 1299 1298 1297
1180 CLIENT: Up:0 ds:0 act:3
1181 Replay: All jobs finished
1182 Main task has work to do, go find it
1183 got dsc Stop(0)
1184 stop 0
1185 Jan 27 19:36:37.160 INFO request completed, latency_us: 357[8810] Got stop action so:false kr:false
1186 , response_code: 204, uri: /stop/cid/0, method: POST, req_id: 6a7b3b1d-86a2-489e-9056-7fa604c3b0f7, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
11872025-01-27T19:36:37.161ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11882025-01-27T19:36:37.161ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11892025-01-27T19:36:37.161ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1190 Jan 27 19:36:37.161 INFO request completed, latency_us: 143, response_code: 200, uri: /state/cid/0, method: GET, req_id: 9a0b2a35-811c-4468-be91-47f1d91d1f40, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
11912025-01-27T19:36:37.161ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
1192 [8810] Exited with: ExitStatus(unix_wait_status(9))
1193 [8810][0] reports Exit
1194 Jan 27 19:36:41.164 INFO request completed, latency_us: 223, response_code: 200, uri: /state/cid/0, method: GET, req_id: 02bda8f9-355b-4ac9-999c-79b4a3aaeb66, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
1195 001/300 Write block 2423 len 32768 data: 2 2 2 2 2 2 2 2
1196 002/300 Read block 1088 len 36864
1197 003/300 Write block 2161 len 24576 data: 2 2 2 2 2 2
1198 004/300 Flush
1199 005/300 Write block 1535 len 16384 data: 3 2 4 4
1200 006/300 Read block 2819 len 12288
1201 007/300 Read block 955 len 32768
1202 008/300 Write block 4271 len 24576 data: 2 2 2 2 2 2
1203 009/300 Write block 2853 len 36864 data: 3 2 2 2 2 2 2 2 2
1204 010/300 Read block 3341 len 32768
1205 011/300 Write block 1553 len 8192 data: 2 2
1206 012/300 Write block 4520 len 12288 data: 2 2 2
1207 013/300 Read block 2824 len 20480
1208 014/300 Read block 2327 len 36864
1209 015/300 Read block 678 len 28672
1210 016/300 Write block 122 len 28672 data: 2 2 2 2 2 2 2
1211 017/300 Read block 4878 len 32768
1212 018/300 Flush
1213 019/300 Write block 1713 len 20480 data: 2 2 2 2 2
1214 020/300 Write block 2203 len 24576 data: 2 2 2 2 2 2
1215 021/300 Read block 1158 len 16384
1216 022/300 Write block 2522 len 12288 data: 2 2 2
1217 023/300 Flush
1218 024/300 Write block 4246 len 20480 data: 2 2 2 2 3
1219 025/300 Flush
1220 026/300 Write block 2378 len 20480 data: 2 2 2 2 2
1221 027/300 Write block 2836 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1222 028/300 Write block 919 len 32768 data: 2 2 2 2 2 2 2 2
1223 029/300 Write block 2136 len 4096 data: 2
1224 030/300 Read block 936 len 8192
1225 031/300 Read block 209 len 20480
1226 032/300 Write block 1029 len 36864 data: 3 3 2 2 2 2 2 2 2
1227 033/300 Flush
1228 034/300 Write block 3408 len 12288 data: 3 3 2
1229 035/300 Write block 4709 len 16384 data: 2 2 2 2
1230 036/300 Read block 602 len 16384
1231 037/300 Flush
1232 038/300 Read block 781 len 32768
1233 039/300 Write block 2201 len 12288 data: 2 2 3
1234 040/300 Write block 1373 len 32768 data: 3 3 4 4 4 3 3 3
1235 041/300 Write block 120 len 32768 data: 2 2 3 3 3 3 3 3
1236 042/300 Read block 4963 len 40960
1237 043/300 Write block 4567 len 8192 data: 3 3
1238 044/300 Flush
1239 045/300 Read block 2302 len 36864
1240 046/300 Write block 2102 len 16384 data: 2 2 2 2
1241 047/300 Read block 4686 len 12288
1242 048/300 Write block 1437 len 28672 data: 2 2 2 2 2 2 2
1243 049/300 Write block 300 len 12288 data: 2 2 2
1244 050/300 Read block 4500 len 16384
1245 051/300 Write block 2560 len 28672 data: 2 2 2 2 2 2 2
1246 052/300 Read block 4818 len 16384
1247 053/300 Read block 1558 len 4096
1248 054/300 Write block 3387 len 28672 data: 2 2 2 2 2 2 2
1249 055/300 Read block 3223 len 4096
1250 056/300 Read block 2545 len 24576
1251 057/300 Write block 485 len 28672 data: 2 2 2 2 2 2 2
1252 058/300 Read block 3664 len 20480
1253 059/300 Read block 3400 len 4096
1254 060/300 Read block 3973 len 40960
1255 061/300 Read block 1595 len 4096
1256 062/300 Write block 2323 len 8192 data: 2 2
1257 063/300 Flush
1258 064/300 Write block 3556 len 36864 data: 2 2 2 2 2 2 2 2 2
1259 065/300 Read block 3480 len 4096
1260 066/300 Read block 1402 len 36864
1261 067/300 Write block 2076 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1262 068/300 Write block 598 len 32768 data: 2 2 2 2 3 3 3 3
1263 069/300 Write block 648 len 8192 data: 2 2
1264 070/300 Flush
1265 071/300 Read block 2002 len 8192
1266 072/300 Read block 2023 len 4096
1267 073/300 Flush
1268 074/300 Flush
1269 075/300 Read block 3601 len 16384
1270 076/300 Write block 2846 len 4096 data: 2
1271 077/300 Flush
1272 078/300 Read block 4224 len 32768
1273 079/300 Read block 1850 len 20480
1274 080/300 Write block 866 len 8192 data: 2 2
1275 081/300 Read block 2131 len 32768
1276 082/300 Read block 3060 len 24576
1277 083/300 Write block 3231 len 20480 data: 3 3 3 3 3
1278 084/300 Read block 716 len 4096
1279 085/300 Write block 718 len 28672 data: 2 2 3 3 3 3 3
1280 086/300 Write block 4160 len 36864 data: 2 2 2 2 2 2 2 2 2
1281 087/300 Read block 18 len 4096
1282 088/300 Flush
1283 089/300 Read block 952 len 40960
1284 090/300 Read block 3454 len 28672
1285 091/300 Write block 3582 len 8192 data: 2 2
1286 092/300 Flush
1287 093/300 Read block 3047 len 24576
1288 094/300 Read block 3487 len 20480
1289 095/300 Write block 16 len 8192 data: 2 2
1290 096/300 Write block 3320 len 28672 data: 2 2 2 2 2 2 2
1291 097/300 Write block 2410 len 28672 data: 2 2 2 2 2 2 2
1292 098/300 Write block 557 len 36864 data: 2 2 2 2 2 2 2 2 2
1293 099/300 Write block 3908 len 24576 data: 3 3 3 3 3 3
1294 100/300 Read block 2088 len 16384
1295 101/300 Write block 3293 len 24576 data: 2 2 2 2 2 2
1296 102/300 Flush
1297 103/300 Read block 4506 len 40960
1298 104/300 Flush
1299 105/300 Flush
1300 106/300 Read block 1288 len 20480
1301 107/300 Write block 545 len 24576 data: 2 2 2 2 2 2
1302 108/300 Write block 1126 len 12288 data: 2 2 2
1303 109/300 Read block 3547 len 16384
1304 110/300 Write block 1042 len 24576 data: 2 2 2 2 2 2
1305 111/300 Read block 2757 len 12288
1306 112/300 Write block 1507 len 8192 data: 2 2
1307 113/300 Read block 1716 len 32768
1308 114/300 Read block 1327 len 32768
1309 115/300 Read block 3998 len 16384
1310 116/300 Read block 4414 len 24576
1311 117/300 Write block 3176 len 28672 data: 3 3 3 3 2 2 2
1312 118/300 Read block 2356 len 16384
1313 119/300 Write block 4502 len 24576 data: 2 2 2 2 2 2
1314 120/300 Read block 1696 len 20480
1315 121/300 Write block 1815 len 12288 data: 2 2 2
1316 122/300 Write block 3766 len 32768 data: 2 2 2 2 2 2 2 3
1317 123/300 Write block 1480 len 20480 data: 2 2 2 2 2
1318 124/300 Read block 671 len 24576
1319 125/300 Flush
1320 126/300 Read block 4500 len 28672
1321 127/300 Write block 372 len 36864 data: 2 3 3 2 2 2 2 2 2
1322 128/300 Read block 338 len 28672
1323 129/300 Write block 1983 len 24576 data: 2 2 2 2 2 2
1324 130/300 Read block 890 len 12288
1325 131/300 Read block 2900 len 12288
1326 132/300 Read block 1777 len 12288
1327 133/300 Flush
1328 134/300 Write block 3304 len 12288 data: 2 2 2
1329 135/300 Write block 2682 len 28672 data: 2 2 2 2 2 2 2
1330 136/300 Read block 3347 len 4096
1331 137/300 Read block 3162 len 24576
1332 138/300 Read block 4869 len 12288
1333 139/300 Write block 3455 len 12288 data: 2 2 2
1334 140/300 Read block 3594 len 24576
1335 141/300 Write block 4242 len 24576 data: 3 3 2 2 3 3
1336 142/300 Read block 494 len 16384
1337 143/300 Read block 4053 len 8192
1338 144/300 Write block 709 len 28672 data: 2 2 2 2 3 3 3
1339 145/300 Read block 4167 len 28672
1340 146/300 Write block 4439 len 32768 data: 2 2 2 2 2 2 2 2
1341 147/300 Read block 2284 len 20480
1342 148/300 Write block 2018 len 24576 data: 2 2 2 2 2 2
1343 149/300 Write block 4245 len 28672 data: 3 4 4 3 3 4 3
1344 150/300 Read block 2658 len 32768
1345 151/300 Write block 4013 len 36864 data: 2 2 2 2 3 3 3 3 3
1346 152/300 Read block 3972 len 4096
1347 153/300 Read block 1705 len 8192
1348 154/300 Flush
1349 155/300 Read block 4718 len 32768
1350 156/300 Write block 3904 len 36864 data: 4 3 3 3 4 4 4 4 4
1351 157/300 Write block 13 len 36864 data: 2 2 2 3 3 2 2 2 2
1352 158/300 Read block 3425 len 40960
1353 159/300 Read block 3159 len 24576
1354 160/300 Read block 3343 len 24576
1355 161/300 Read block 2318 len 36864
1356 162/300 Write block 1789 len 28672 data: 3 3 3 2 2 3 3
1357 163/300 Read block 1170 len 20480
1358 164/300 Write block 4788 len 40960 data: 2 2 3 3 3 3 3 3 3 2
1359 165/300 Read block 2406 len 32768
1360 166/300 Read block 3702 len 8192
1361 167/300 Read block 898 len 32768
1362 168/300 Write block 2790 len 24576 data: 2 2 2 2 3 3
1363 169/300 Read block 346 len 8192
1364 170/300 Write block 3872 len 28672 data: 2 2 2 2 2 2 2
1365 171/300 Read block 4494 len 8192
1366 172/300 Read block 1097 len 24576
1367 173/300 Read block 4877 len 36864
1368 174/300 Read block 977 len 36864
1369 175/300 Read block 285 len 16384
1370 176/300 Write block 4526 len 12288 data: 2 2 2
1371 177/300 Read block 2313 len 24576
1372 178/300 Read block 4239 len 4096
1373 179/300 Read block 3371 len 32768
1374 180/300 Read block 393 len 16384
1375 181/300 Read block 242 len 32768
1376 182/300 Read block 104 len 4096
1377 183/300 Write block 2437 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1378 184/300 Write block 2437 len 40960 data: 3 3 3 3 3 3 3 3 3 3
1379 185/300 Read block 2037 len 12288
1380 186/300 Flush
1381 187/300 Read block 2489 len 40960
1382 188/300 Flush
1383 189/300 Read block 2604 len 4096
1384 190/300 Write block 320 len 28672 data: 2 2 2 2 2 2 2
1385 191/300 Write block 1221 len 20480 data: 2 2 2 2 2
1386 192/300 Read block 1151 len 24576
1387 193/300 Write block 713 len 8192 data: 4 4
1388 194/300 Read block 2952 len 20480
1389 195/300 Write block 4985 len 12288 data: 2 2 2
1390 196/300 Read block 3337 len 16384
1391 197/300 Write block 238 len 24576 data: 2 2 2 2 2 2
1392 198/300 Write block 4824 len 8192 data: 2 2
1393 199/300 Write block 4938 len 28672 data: 2 2 2 2 3 3 3
1394 200/300 Write block 1616 len 16384 data: 2 2 2 2
1395 201/300 Read block 2951 len 40960
1396 202/300 Read block 4430 len 8192
1397 203/300 Write block 1303 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1398 204/300 Write block 3222 len 16384 data: 3 3 3 3
1399 205/300 Read block 3206 len 8192
1400 206/300 Write block 2493 len 8192 data: 2 2
1401 207/300 Read block 4966 len 36864
1402 208/300 Write block 3559 len 28672 data: 3 3 3 3 3 3 3
1403 209/300 Write block 2928 len 12288 data: 2 2 2
1404 210/300 Read block 2953 len 24576
1405 211/300 Read block 4608 len 24576
1406 212/300 Write block 4034 len 16384 data: 2 2 2 2
1407 213/300 Flush
1408 214/300 Flush
1409 215/300 Read block 1128 len 24576
1410 216/300 Write block 184 len 12288 data: 2 2 3
1411 217/300 Read block 296 len 40960
1412 218/300 Write block 3921 len 16384 data: 2 2 2 2
1413 219/300 Write block 1536 len 40960 data: 3 5 5 4 4 4 4 5 5 4
1414 220/300 Write block 587 len 32768 data: 2 2 2 2 2 2 2 2
1415 221/300 Read block 3984 len 4096
1416 222/300 Read block 3318 len 24576
1417 223/300 Write block 4304 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1418 224/300 Read block 193 len 8192
1419 225/300 Read block 3018 len 8192
1420 226/300 Write block 1812 len 28672 data: 3 2 2 3 3 3 2
1421 227/300 Write block 4330 len 40960 data: 3 4 3 3 3 3 3 3 3 2
1422 228/300 Read block 464 len 20480
1423 229/300 Read block 548 len 4096
1424 230/300 Read block 279 len 12288
1425 231/300 Write block 4189 len 40960 data: 2 2 2 2 2 2 2 3 3 2
1426 232/300 Flush
1427 233/300 Read block 1433 len 12288
1428 234/300 Write block 2255 len 16384 data: 2 2 2 2
1429 235/300 Read block 3159 len 28672
1430 236/300 Read block 1849 len 20480
1431 237/300 Read block 1727 len 16384
1432 238/300 Read block 1410 len 40960
1433 239/300 Write block 4810 len 8192 data: 2 2
1434 240/300 Read block 2362 len 32768
1435 241/300 Write block 4008 len 4096 data: 2
1436 242/300 Write block 3680 len 4096 data: 2
1437 243/300 Flush
1438 244/300 Read block 22 len 24576
1439 245/300 Read block 3306 len 24576
1440 246/300 Write block 2736 len 12288 data: 2 2 2
1441 247/300 Flush
1442 248/300 Read block 647 len 32768
1443 249/300 Read block 3616 len 32768
1444 250/300 Read block 1534 len 20480
1445 251/300 Read block 3702 len 20480
1446 252/300 Write block 1456 len 12288 data: 3 3 3
1447 253/300 Read block 3951 len 32768
1448 254/300 Read block 1628 len 40960
1449 255/300 Write block 4053 len 24576 data: 2 2 2 2 2 2
1450 256/300 Write block 1850 len 8192 data: 3 3
1451 257/300 Write block 2677 len 32768 data: 3 3 3 3 3 3 3 3
1452 258/300 Read block 3762 len 8192
1453 259/300 Read block 3522 len 20480
1454 260/300 Write block 4271 len 32768 data: 3 3 3 3 3 3 3 3
1455 261/300 Write block 3725 len 20480 data: 2 2 2 2 2
1456 262/300 Write block 2406 len 20480 data: 2 2 2 2 3
1457 263/300 Write block 2385 len 28672 data: 2 2 2 2 2 2 2
1458 264/300 Flush
1459 265/300 Write block 2193 len 36864 data: 2 2 2 2 2 2 2 2 3
1460 266/300 Write block 1306 len 20480 data: 3 3 3 3 3
1461 267/300 Read block 3958 len 20480
1462 268/300 Read block 662 len 32768
1463 269/300 Read block 2328 len 32768
1464 270/300 Write block 4853 len 12288 data: 2 2 2
1465 271/300 Write block 607 len 28672 data: 2 2 2 2 2 2 2
1466 272/300 Read block 3670 len 4096
1467 273/300 Write block 3314 len 28672 data: 2 2 2 2 2 2 3
1468 274/300 Flush
1469 275/300 Read block 2234 len 36864
1470 276/300 Flush
1471 277/300 Read block 4915 len 24576
1472 278/300 Read block 2166 len 32768
1473 279/300 Read block 1682 len 32768
1474 280/300 Read block 3235 len 36864
1475 281/300 Read block 1870 len 4096
1476 282/300 Write block 531 len 32768 data: 2 2 2 2 2 2 2 2
1477 283/300 Write block 1768 len 12288 data: 2 2 2
1478 284/300 Flush
1479 285/300 Write block 2947 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1480 286/300 Read block 3418 len 28672
1481 287/300 Write block 1719 len 36864 data: 2 2 2 2 2 2 2 2 2
1482 288/300 Write block 1234 len 24576 data: 3 3 3 3 3 3
1483 289/300 Read block 4149 len 20480
1484 290/300 Write block 930 len 16384 data: 2 2 2 2
1485 291/300 Read block 4770 len 20480
1486 292/300 Read block 950 len 4096
1487 293/300 Read block 2236 len 16384
1488 294/300 Write block 4509 len 40960 data: 2 2 2 2 2 2 2 2 2 2
1489 295/300 Read block 3168 len 20480
1490 296/300 Flush
1491 297/300 Read block 149 len 12288
1492 298/300 Read block 2831 len 24576
1493 299/300 Write block 4563 len 12288 data: 3 3 3
1494 300/300 Write block 2486 len 20480 data: 2 2 2 2 2
1495 301/300 Read block 4987 len 16384
1496 Main task has work to do, go find it
1497 got dsc Start(0)
1498 start 0
1499 Jan 27 19:36:41.398 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/0, method: POST, req_id: a0b80ea4-980c-47be-9e54-496b960c6e61, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
1500 Replay: started 0, returned:Ok(())
1501 ----------------------------------------------------------------
1502 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
1503 States: Offline Active Active
1504 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1505 1302 Acked Write 8 Sent Done Done false
1506 1303 Acked Read 9 Sent Done Done false
1507 1304 Acked Write 6 Sent Done Done false
1508 1305 Acked Flush 0 Sent Done Done false
1509 1306 Acked Write 4 Sent Done Done false
1510 1307 Acked Read 3 Sent Done Done false
1511 1308 Acked Read 8 Sent Done Done false
1512 1309 Acked Write 6 Sent Done Done false
1513 1310 Acked Write 9 Sent Done Done false
1514 1311 Acked Read 8 Sent Done Done false
1515 1312 Acked Write 2 Sent Done Done false
1516 1313 Acked Write 3 Sent Done Done false
1517 1314 Acked Read 5 Sent Done Done false
1518 1315 Acked Read 9 Sent Done Done false
1519 1316 Acked Read 7 Sent Done Done false
1520 1317 Acked Write 7 Sent Done Done false
1521 1318 Acked Read 8 Sent Done Done false
1522 1319 Acked Flush 0 Sent Done Done false
1523 1320 Acked Write 5 Sent Done Done false
1524 1321 Acked Write 6 Sent Done Done false
1525 1322 Acked Read 4 Sent Done Done false
1526 1323 Acked Write 3 Sent Done Done false
1527 1324 Acked Flush 0 Sent Done Done false
1528 1325 Acked Write 5 Sent Done Done false
1529 1326 Acked Flush 0 Sent Done Done false
1530 1327 Acked Write 5 Sent Done Done false
1531 1328 Acked Write 10 Sent Done Done false
1532 1329 Acked Write 8 Sent Done Done false
1533 1330 Acked Write 1 Sent Done Done false
1534 1331 Acked Read 2 Sent Done Done false
1535 1332 Acked Read 5 Sent Done Done false
1536 1333 Acked Write 9 Sent Done Done false
1537 1334 Acked Flush 0 Sent Done Done false
1538 1335 Acked Write 3 Sent Done Done false
1539 1336 Acked Write 4 Sent Done Done false
1540 1337 Acked Read 4 Sent Done Done false
1541 1338 Acked Flush 0 Sent Done Done false
1542 1339 Acked Read 8 Sent Done Done false
1543 1340 Acked Write 3 Sent Done Done false
1544 1341 Acked Write 8 Sent Done Done false
1545 1342 Acked Write 8 Sent Done Done false
1546 1343 Acked Read 10 Sent Done Done false
1547 1344 Acked Write 2 Sent Done Done false
1548 1345 Acked Flush 0 Sent Done Done false
1549 1346 Acked Read 9 Sent Done Done false
1550 1347 Acked Write 4 Sent Done Done false
1551 1348 Acked Read 3 Sent Done Done false
1552 1349 Acked Write 7 Sent Done Done false
1553 1350 Acked Write 3 Sent Done Done false
1554 1351 Acked Read 4 Sent Done Done false
1555 1352 Acked Write 7 Sent Done Done false
1556 1353 Acked Read 4 Sent Done Done false
1557 1354 Acked Read 1 Sent Done Done false
1558 1355 Acked Write 7 Sent Done Done false
1559 1356 Acked Read 1 Sent Done Done false
1560 1357 Acked Read 6 Sent Done Done false
1561 1358 Acked Write 7 Sent Done Done false
1562 1359 Acked Read 5 Sent Done Done false
1563 1360 Acked Read 1 Sent Done Done false
1564 1361 Acked Read 10 Sent Done Done false
1565 1362 Acked Read 1 Sent Done Done false
1566 1363 Acked Write 2 Sent Done Done false
1567 1364 Acked Flush 0 Sent Done Done false
1568 1365 Acked Write 9 Sent Done Done false
1569 1366 Acked Read 1 Sent Done Done false
1570 1367 Acked Read 9 Sent Done Done false
1571 1368 Acked Write 10 Sent Done Done false
1572 1369 Acked Write 8 Sent Done Done false
1573 1370 Acked Write 2 Sent Done Done false
1574 1371 Acked Flush 0 Sent Done Done false
1575 1372 Acked Read 2 Sent Done Done false
1576 1373 Acked Read 1 Sent Done Done false
1577 1374 Acked Flush 0 Sent Done Done false
1578 1375 Acked Flush 0 Sent Done Done false
1579 1376 Acked Read 4 Sent Done Done false
1580 1377 Acked Write 1 Sent Done Done false
1581 1378 Acked Flush 0 Sent Done Done false
1582 1379 Acked Read 8 Sent Done Done false
1583 1380 Acked Read 5 Sent Done Done false
1584 1381 Acked Write 2 Sent Done Done false
1585 1382 Acked Read 8 Sent Done Done false
1586 1383 Acked Read 6 Sent Done Done false
1587 1384 Acked Write 5 Sent Done Done false
1588 1385 Acked Read 1 Sent Done Done false
1589 1386 Acked Write 7 Sent Done Done false
1590 1387 Acked Write 9 Sent Done Done false
1591 1388 Acked Read 1 Sent Done Done false
1592 1389 Acked Flush 0 Sent Done Done false
1593 1390 Acked Read 10 Sent Done Done false
1594 1391 Acked Read 7 Sent Done Done false
1595 1392 Acked Write 2 Sent Done Done false
1596 1393 Acked Flush 0 Sent Done Done false
1597 1394 Acked Read 6 Sent Done Done false
1598 1395 Acked Read 5 Sent Done Done false
1599 1396 Acked Write 2 Sent Done Done false
1600 1397 Acked Write 7 Sent Done Done false
1601 1398 Acked Write 7 Sent Done Done false
1602 1399 Acked Write 9 Sent Done Done false
1603 1400 Acked Write 6 Sent Done Done false
1604 1401 Acked Read 4 Sent Done Done false
1605 1402 Acked Write 6 Sent Done Done false
1606 1403 Acked Flush 0 Sent Done Done false
1607 1404 Acked Read 10 Sent Done Done false
1608 1405 Acked Flush 0 Sent Done Done false
1609 1406 Acked Flush 0 Sent Done Done false
1610 1407 Acked Read 5 Sent Done Done false
1611 1408 Acked Write 6 Sent Done Done false
1612 1409 Acked Write 3 Sent Done Done false
1613 1410 Acked Read 4 Sent Done Done false
1614 1411 Acked Write 6 Sent Done Done false
1615 1412 Acked Read 3 Sent Done Done false
1616 1413 Acked Write 2 Sent Done Done false
1617 1414 Acked Read 8 Sent Done Done false
1618 1415 Acked Read 8 Sent Done Done false
1619 1416 Acked Read 4 Sent Done Done false
1620 1417 Acked Read 6 Sent Done Done false
1621 1418 Acked Write 7 Sent Done Done false
1622 1419 Acked Read 4 Sent Done Done false
1623 1420 Acked Write 6 Sent Done Done false
1624 1421 Acked Read 5 Sent Done Done false
1625 1422 Acked Write 3 Sent Done Done false
1626 1423 Acked Write 8 Sent Done Done false
1627 1424 Acked Write 5 Sent Done Done false
1628 1425 Acked Read 6 Sent Done Done false
1629 1426 Acked Flush 0 Sent Done Done false
1630 1427 Acked Read 7 Sent Done Done false
1631 1428 Acked Write 9 Sent Done Done false
1632 1429 Acked Read 7 Sent Done Done false
1633 1430 Acked Write 6 Sent Done Done false
1634 1431 Acked Read 3 Sent Done Done false
1635 1432 Acked Read 3 Sent Done Done false
1636 1433 Acked Read 3 Sent Done Done false
1637 1434 Acked Flush 0 Sent Done Done false
1638 1435 Acked Write 3 Sent Done Done false
1639 1436 Acked Write 7 Sent Done Done false
1640 1437 Acked Read 1 Sent Done Done false
1641 1438 Acked Read 6 Sent Done Done false
1642 1439 Acked Read 3 Sent Done Done false
1643 1440 Acked Write 3 Sent Done Done false
1644 1441 Acked Read 6 Sent Done Done false
1645 1442 Acked Write 6 Sent Done Done false
1646 1443 Acked Read 4 Sent Done Done false
1647 1444 Acked Read 2 Sent Done Done false
1648 1445 Acked Write 7 Sent Done Done false
1649 1446 Acked Read 7 Sent Done Done false
1650 1447 Acked Write 8 Sent Done Done false
1651 1448 Acked Read 5 Sent Done Done false
1652 1449 Acked Write 6 Sent Done Done false
1653 1450 Acked Write 7 Sent Done Done false
1654 1451 Acked Read 8 Sent Done Done false
1655 1452 Acked Write 9 Sent Done Done false
1656 1453 Acked Read 1 Sent Done Done false
1657 1454 Acked Read 2 Sent Done Done false
1658 1455 Acked Flush 0 Sent Done Done false
1659 1456 Acked Read 8 Sent Done Done false
1660 1457 Acked Write 9 Sent Done Done false
1661 1458 Acked Write 9 Sent Done Done false
1662 1459 Acked Read 10 Sent Done Done false
1663 1460 Acked Read 6 Sent Done Done false
1664 1461 Acked Read 6 Sent Done Done false
1665 1462 Acked Read 9 Sent Done Done false
1666 1463 Acked Write 7 Sent Done Done false
1667 1464 Acked Read 5 Sent Done Done false
1668 1465 Acked Write 10 Sent Done Done false
1669 1466 Acked Read 8 Sent Done Done false
1670 1467 Acked Read 2 Sent Done Done false
1671 1468 Acked Read 8 Sent Done Done false
1672 1469 Acked Write 6 Sent Done Done false
1673 1470 Acked Read 2 Sent Done Done false
1674 1471 Acked Write 7 Sent Done Done false
1675 1472 Acked Read 2 Sent Done Done false
1676 1473 Acked Read 6 Sent Done Done false
1677 1474 Acked Read 9 Sent Done Done false
1678 1475 Acked Read 9 Sent Done Done false
1679 1476 Acked Read 4 Sent Done Done false
1680 1477 Acked Write 3 Sent Done Done false
1681 1478 Acked Read 6 Sent Done Done false
1682 1479 Acked Read 1 Sent Done Done false
1683 1480 Acked Read 8 Sent Done Done false
1684 1481 Acked Read 4 Sent Done Done false
1685 1482 Acked Read 8 Sent Done Done false
1686 1483 Acked Read 1 Sent Done Done false
1687 1484 Acked Write 10 Sent Done Done false
1688 1485 Acked Write 10 Sent Done Done false
1689 1486 Acked Read 3 Sent Done Done false
1690 1487 Acked Flush 0 Sent Done Done false
1691 1488 Acked Read 10 Sent Done Done false
1692 1489 Acked Flush 0 Sent Done Done false
1693 1490 Acked Read 1 Sent Done Done false
1694 1491 Acked Write 7 Sent Done Done false
1695 1492 Acked Write 5 Sent Done Done false
1696 1493 Acked Read 6 Sent Done Done false
1697 1494 Acked Write 2 Sent Done Done false
1698 1495 Acked Read 5 Sent Done Done false
1699 1496 Acked Write 3 Sent Done Done false
1700 1497 Acked Read 4 Sent Done Done false
1701 1498 Acked Write 6 Sent Done Done false
1702 1499 Acked Write 2 Sent Done Done false
1703 1500 Acked Write 7 Sent Done Done false
1704 1501 Acked Write 4 Sent Done Done false
1705 1502 Acked Read 10 Sent Done Done false
1706 1503 Acked Read 2 Sent Done Done false
1707 1504 Acked Write 10 Sent Done Done false
1708 1505 Acked Write 4 Sent Done Done false
1709 1506 Acked Read 2 Sent Done Done false
1710 1507 Acked Write 2 Sent Done Done false
1711 1508 Acked Read 9 Sent Done Done false
1712 1509 Acked Write 7 Sent Done Done false
1713 1510 Acked Write 3 Sent Done Done false
1714 1511 Acked Read 6 Sent Done Done false
1715 1512 Acked Read 6 Sent Done Done false
1716 1513 Acked Write 4 Sent Done Done false
1717 1514 Acked Flush 0 Sent Done Done false
1718 1515 Acked Flush 0 Sent Done Done false
1719 1516 Acked Read 6 Sent Done Done false
1720 1517 Acked Write 3 Sent Done Done false
1721 1518 Acked Read 10 Sent Done Done false
1722 1519 Acked Write 4 Sent Done Done false
1723 1520 Acked Write 10 Sent Done Done false
1724 1521 Acked Write 8 Sent Done Done false
1725 1522 Acked Read 1 Sent Done Done false
1726 1523 Acked Read 6 Sent Done Done false
1727 1524 Acked Write 10 Sent Done Done false
1728 1525 Acked Read 2 Sent Done Done false
1729 1526 Acked Read 2 Sent Done Done false
1730 1527 Acked Write 7 Sent Done Done false
1731 1528 Acked Write 10 Sent Done Done false
1732 1529 Acked Read 5 Sent Done Done false
1733 1530 Acked Read 1 Sent Done Done false
1734 1531 Acked Read 3 Sent Done Done false
1735 1532 Acked Write 10 Sent Done Done false
1736 1533 Acked Flush 0 Sent Done Done false
1737 1534 Acked Read 3 Sent Done Done false
1738 1535 Acked Write 4 Sent Done Done false
1739 1536 Acked Read 7 Sent Done Done false
1740 1537 Acked Read 5 Sent Done Done false
1741 1538 Acked Read 4 Sent Done Done false
1742 1539 Acked Read 10 Sent Done Done false
1743 1540 Acked Write 2 Sent Done Done false
1744 1541 Acked Read 8 Sent Done Done false
1745 1542 Acked Write 1 Sent Done Done false
1746 1543 Acked Write 1 Sent Done Done false
1747 1544 Acked Flush 0 Sent Done Done false
1748 1545 Acked Read 6 Sent Done Done false
1749 1546 Acked Read 6 Sent Done Done false
1750 1547 Acked Write 3 Sent Done Done false
1751 1548 Acked Flush 0 Sent Done Done false
1752 1549 Acked Read 8 Sent Done Done false
1753 1550 Acked Read 8 Sent Done Done false
1754 1551 Acked Read 5 Sent Done Done false
1755 1552 Acked Read 5 Sent Done Done false
1756 1553 Acked Write 3 Sent Done Done false
1757 1554 Acked Read 8 Sent Done Done false
1758 1555 Acked Read 10 Sent Done Done false
1759 1556 Acked Write 6 Sent Done Done false
1760 1557 Acked Write 2 Sent Done Done false
1761 1558 Acked Write 8 Sent Done Done false
1762 1559 Acked Read 2 Sent Done Done false
1763 1560 Acked Read 5 Sent Done Done false
1764 1561 Acked Write 8 Sent Done Done false
1765 1562 Acked Write 5 Sent Done Done false
1766 1563 Acked Write 5 Sent Done Done false
1767 1564 Acked Write 7 Sent Done Done false
1768 1565 Acked Flush 0 Sent Done Done false
1769 1566 Acked Write 9 Sent Done Done false
1770 1567 Acked Write 5 Sent Done Done false
1771 1568 Acked Read 5 Sent Done Done false
1772 1569 Acked Read 8 Sent Done Done false
1773 1570 Acked Read 8 Sent Done Done false
1774 1571 Acked Write 3 Sent Done Done false
1775 1572 Acked Write 7 Sent Done Done false
1776 1573 Acked Read 1 Sent Done Done false
1777 1574 Acked Write 7 Sent Done Done false
1778 1575 Acked Flush 0 Sent Done Done false
1779 1576 Acked Read 9 Sent Done Done false
1780 1577 Acked Flush 0 Sent Done Done false
1781 1578 Acked Read 6 Sent Done Done false
1782 1579 Acked Read 8 Sent Done Done false
1783 1580 Acked Read 8 Sent Done Done false
1784 1581 Acked Read 9 Sent Done Done false
1785 1582 Acked Read 1 Sent Done Done false
1786 1583 Acked Write 8 Sent Done Done false
1787 1584 Acked Write 3 Sent Done Done false
1788 1585 Acked Flush 0 Sent Done Done false
1789 1586 Acked Write 10 Sent Done Done false
1790 1587 Acked Read 7 Sent Done Done false
1791 1588 Acked Write 9 Sent Done Done false
1792 1589 Acked Write 6 Sent Done Done false
1793 1590 Acked Read 5 Sent Done Done false
1794 1591 Acked Write 4 Sent Done Done false
1795 1592 Acked Read 5 Sent Done Done false
1796 1593 Acked Read 1 Sent Done Done false
1797 1594 Acked Read 4 Sent Done Done false
1798 1595 Acked Write 10 Sent Done Done false
1799 1596 Acked Read 5 Sent Done Done false
1800 1597 Acked Flush 0 Sent Done Done false
1801 1598 Acked Read 3 Sent Done Done false
1802 1599 Acked Read 6 Sent Done Done false
1803 1600 Acked Write 3 Sent Done Done false
1804 1601 Acked Write 5 Sent Done Done false
1805 1602 Acked Read 4 Sent Done Done false
1806 STATES DS:0 DS:1 DS:2 TOTAL
1807 Sent 301 0 0 301
1808 Done 0 301 301 602
1809 Skipped 0 0 0 0
1810 Error 0 0 0 0
1811 Last Flush: 1301 1597 1597
1812 Downstairs last five completed: 1301 1300 1299 1298 1297
1813 Upstairs last five completed: 1602 1601 1600 1599 1598
1814 CLIENT: Up:0 ds:301 act:2
1815 [8810] got start action
1816 [8810] I am going to restart
1817 Starting downstairs at port 8810
1818 Make output file at "/tmp/dsc/downstairs-8810.txt"
1819 [8810][0] reports Starting
1820 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1202)
1821 [8810][0] reports Running
1822 ----------------------------------------------------------------
1823 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
1824 States: Offline Active Active
1825 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
1826 1302 Acked Write 8 Sent Done Done false
1827 1303 Acked Read 9 Sent Done Done false
1828 1304 Acked Write 6 Sent Done Done false
1829 1305 Acked Flush 0 Sent Done Done false
1830 1306 Acked Write 4 Sent Done Done false
1831 1307 Acked Read 3 Sent Done Done false
1832 1308 Acked Read 8 Sent Done Done false
1833 1309 Acked Write 6 Sent Done Done false
1834 1310 Acked Write 9 Sent Done Done false
1835 1311 Acked Read 8 Sent Done Done false
1836 1312 Acked Write 2 Sent Done Done false
1837 1313 Acked Write 3 Sent Done Done false
1838 1314 Acked Read 5 Sent Done Done false
1839 1315 Acked Read 9 Sent Done Done false
1840 1316 Acked Read 7 Sent Done Done false
1841 1317 Acked Write 7 Sent Done Done false
1842 1318 Acked Read 8 Sent Done Done false
1843 1319 Acked Flush 0 Sent Done Done false
1844 1320 Acked Write 5 Sent Done Done false
1845 1321 Acked Write 6 Sent Done Done false
1846 1322 Acked Read 4 Sent Done Done false
1847 1323 Acked Write 3 Sent Done Done false
1848 1324 Acked Flush 0 Sent Done Done false
1849 1325 Acked Write 5 Sent Done Done false
1850 1326 Acked Flush 0 Sent Done Done false
1851 1327 Acked Write 5 Sent Done Done false
1852 1328 Acked Write 10 Sent Done Done false
1853 1329 Acked Write 8 Sent Done Done false
1854 1330 Acked Write 1 Sent Done Done false
1855 1331 Acked Read 2 Sent Done Done false
1856 1332 Acked Read 5 Sent Done Done false
1857 1333 Acked Write 9 Sent Done Done false
1858 1334 Acked Flush 0 Sent Done Done false
1859 1335 Acked Write 3 Sent Done Done false
1860 1336 Acked Write 4 Sent Done Done false
1861 1337 Acked Read 4 Sent Done Done false
1862 1338 Acked Flush 0 Sent Done Done false
1863 1339 Acked Read 8 Sent Done Done false
1864 1340 Acked Write 3 Sent Done Done false
1865 1341 Acked Write 8 Sent Done Done false
1866 1342 Acked Write 8 Sent Done Done false
1867 1343 Acked Read 10 Sent Done Done false
1868 1344 Acked Write 2 Sent Done Done false
1869 1345 Acked Flush 0 Sent Done Done false
1870 1346 Acked Read 9 Sent Done Done false
1871 1347 Acked Write 4 Sent Done Done false
1872 1348 Acked Read 3 Sent Done Done false
1873 1349 Acked Write 7 Sent Done Done false
1874 1350 Acked Write 3 Sent Done Done false
1875 1351 Acked Read 4 Sent Done Done false
1876 1352 Acked Write 7 Sent Done Done false
1877 1353 Acked Read 4 Sent Done Done false
1878 1354 Acked Read 1 Sent Done Done false
1879 1355 Acked Write 7 Sent Done Done false
1880 1356 Acked Read 1 Sent Done Done false
1881 1357 Acked Read 6 Sent Done Done false
1882 1358 Acked Write 7 Sent Done Done false
1883 1359 Acked Read 5 Sent Done Done false
1884 1360 Acked Read 1 Sent Done Done false
1885 1361 Acked Read 10 Sent Done Done false
1886 1362 Acked Read 1 Sent Done Done false
1887 1363 Acked Write 2 Sent Done Done false
1888 1364 Acked Flush 0 Sent Done Done false
1889 1365 Acked Write 9 Sent Done Done false
1890 1366 Acked Read 1 Sent Done Done false
1891 1367 Acked Read 9 Sent Done Done false
1892 1368 Acked Write 10 Sent Done Done false
1893 1369 Acked Write 8 Sent Done Done false
1894 1370 Acked Write 2 Sent Done Done false
1895 1371 Acked Flush 0 Sent Done Done false
1896 1372 Acked Read 2 Sent Done Done false
1897 1373 Acked Read 1 Sent Done Done false
1898 1374 Acked Flush 0 Sent Done Done false
1899 1375 Acked Flush 0 Sent Done Done false
1900 1376 Acked Read 4 Sent Done Done false
1901 1377 Acked Write 1 Sent Done Done false
1902 1378 Acked Flush 0 Sent Done Done false
1903 1379 Acked Read 8 Sent Done Done false
1904 1380 Acked Read 5 Sent Done Done false
1905 1381 Acked Write 2 Sent Done Done false
1906 1382 Acked Read 8 Sent Done Done false
1907 1383 Acked Read 6 Sent Done Done false
1908 1384 Acked Write 5 Sent Done Done false
1909 1385 Acked Read 1 Sent Done Done false
1910 1386 Acked Write 7 Sent Done Done false
1911 1387 Acked Write 9 Sent Done Done false
1912 1388 Acked Read 1 Sent Done Done false
1913 1389 Acked Flush 0 Sent Done Done false
1914 1390 Acked Read 10 Sent Done Done false
1915 1391 Acked Read 7 Sent Done Done false
1916 1392 Acked Write 2 Sent Done Done false
1917 1393 Acked Flush 0 Sent Done Done false
1918 1394 Acked Read 6 Sent Done Done false
1919 1395 Acked Read 5 Sent Done Done false
1920 1396 Acked Write 2 Sent Done Done false
1921 1397 Acked Write 7 Sent Done Done false
1922 1398 Acked Write 7 Sent Done Done false
1923 1399 Acked Write 9 Sent Done Done false
1924 1400 Acked Write 6 Sent Done Done false
1925 1401 Acked Read 4 Sent Done Done false
1926 1402 Acked Write 6 Sent Done Done false
1927 1403 Acked Flush 0 Sent Done Done false
1928 1404 Acked Read 10 Sent Done Done false
1929 1405 Acked Flush 0 Sent Done Done false
1930 1406 Acked Flush 0 Sent Done Done false
1931 1407 Acked Read 5 Sent Done Done false
1932 1408 Acked Write 6 Sent Done Done false
1933 1409 Acked Write 3 Sent Done Done false
1934 1410 Acked Read 4 Sent Done Done false
1935 1411 Acked Write 6 Sent Done Done false
1936 1412 Acked Read 3 Sent Done Done false
1937 1413 Acked Write 2 Sent Done Done false
1938 1414 Acked Read 8 Sent Done Done false
1939 1415 Acked Read 8 Sent Done Done false
1940 1416 Acked Read 4 Sent Done Done false
1941 1417 Acked Read 6 Sent Done Done false
1942 1418 Acked Write 7 Sent Done Done false
1943 1419 Acked Read 4 Sent Done Done false
1944 1420 Acked Write 6 Sent Done Done false
1945 1421 Acked Read 5 Sent Done Done false
1946 1422 Acked Write 3 Sent Done Done false
1947 1423 Acked Write 8 Sent Done Done false
1948 1424 Acked Write 5 Sent Done Done false
1949 1425 Acked Read 6 Sent Done Done false
1950 1426 Acked Flush 0 Sent Done Done false
1951 1427 Acked Read 7 Sent Done Done false
1952 1428 Acked Write 9 Sent Done Done false
1953 1429 Acked Read 7 Sent Done Done false
1954 1430 Acked Write 6 Sent Done Done false
1955 1431 Acked Read 3 Sent Done Done false
1956 1432 Acked Read 3 Sent Done Done false
1957 1433 Acked Read 3 Sent Done Done false
1958 1434 Acked Flush 0 Sent Done Done false
1959 1435 Acked Write 3 Sent Done Done false
1960 1436 Acked Write 7 Sent Done Done false
1961 1437 Acked Read 1 Sent Done Done false
1962 1438 Acked Read 6 Sent Done Done false
1963 1439 Acked Read 3 Sent Done Done false
1964 1440 Acked Write 3 Sent Done Done false
1965 1441 Acked Read 6 Sent Done Done false
1966 1442 Acked Write 6 Sent Done Done false
1967 1443 Acked Read 4 Sent Done Done false
1968 1444 Acked Read 2 Sent Done Done false
1969 1445 Acked Write 7 Sent Done Done false
1970 1446 Acked Read 7 Sent Done Done false
1971 1447 Acked Write 8 Sent Done Done false
1972 1448 Acked Read 5 Sent Done Done false
1973 1449 Acked Write 6 Sent Done Done false
1974 1450 Acked Write 7 Sent Done Done false
1975 1451 Acked Read 8 Sent Done Done false
1976 1452 Acked Write 9 Sent Done Done false
1977 1453 Acked Read 1 Sent Done Done false
1978 1454 Acked Read 2 Sent Done Done false
1979 1455 Acked Flush 0 Sent Done Done false
1980 1456 Acked Read 8 Sent Done Done false
1981 1457 Acked Write 9 Sent Done Done false
1982 1458 Acked Write 9 Sent Done Done false
1983 1459 Acked Read 10 Sent Done Done false
1984 1460 Acked Read 6 Sent Done Done false
1985 1461 Acked Read 6 Sent Done Done false
1986 1462 Acked Read 9 Sent Done Done false
1987 1463 Acked Write 7 Sent Done Done false
1988 1464 Acked Read 5 Sent Done Done false
1989 1465 Acked Write 10 Sent Done Done false
1990 1466 Acked Read 8 Sent Done Done false
1991 1467 Acked Read 2 Sent Done Done false
1992 1468 Acked Read 8 Sent Done Done false
1993 1469 Acked Write 6 Sent Done Done false
1994 1470 Acked Read 2 Sent Done Done false
1995 1471 Acked Write 7 Sent Done Done false
1996 1472 Acked Read 2 Sent Done Done false
1997 1473 Acked Read 6 Sent Done Done false
1998 1474 Acked Read 9 Sent Done Done false
1999 1475 Acked Read 9 Sent Done Done false
2000 1476 Acked Read 4 Sent Done Done false
2001 1477 Acked Write 3 Sent Done Done false
2002 1478 Acked Read 6 Sent Done Done false
2003 1479 Acked Read 1 Sent Done Done false
2004 1480 Acked Read 8 Sent Done Done false
2005 1481 Acked Read 4 Sent Done Done false
2006 1482 Acked Read 8 Sent Done Done false
2007 1483 Acked Read 1 Sent Done Done false
2008 1484 Acked Write 10 Sent Done Done false
2009 1485 Acked Write 10 Sent Done Done false
2010 1486 Acked Read 3 Sent Done Done false
2011 1487 Acked Flush 0 Sent Done Done false
2012 1488 Acked Read 10 Sent Done Done false
2013 1489 Acked Flush 0 Sent Done Done false
2014 1490 Acked Read 1 Sent Done Done false
2015 1491 Acked Write 7 Sent Done Done false
2016 1492 Acked Write 5 Sent Done Done false
2017 1493 Acked Read 6 Sent Done Done false
2018 1494 Acked Write 2 Sent Done Done false
2019 1495 Acked Read 5 Sent Done Done false
2020 1496 Acked Write 3 Sent Done Done false
2021 1497 Acked Read 4 Sent Done Done false
2022 1498 Acked Write 6 Sent Done Done false
2023 1499 Acked Write 2 Sent Done Done false
2024 1500 Acked Write 7 Sent Done Done false
2025 1501 Acked Write 4 Sent Done Done false
2026 1502 Acked Read 10 Sent Done Done false
2027 1503 Acked Read 2 Sent Done Done false
2028 1504 Acked Write 10 Sent Done Done false
2029 1505 Acked Write 4 Sent Done Done false
2030 1506 Acked Read 2 Sent Done Done false
2031 1507 Acked Write 2 Sent Done Done false
2032 1508 Acked Read 9 Sent Done Done false
2033 1509 Acked Write 7 Sent Done Done false
2034 1510 Acked Write 3 Sent Done Done false
2035 1511 Acked Read 6 Sent Done Done false
2036 1512 Acked Read 6 Sent Done Done false
2037 1513 Acked Write 4 Sent Done Done false
2038 1514 Acked Flush 0 Sent Done Done false
2039 1515 Acked Flush 0 Sent Done Done false
2040 1516 Acked Read 6 Sent Done Done false
2041 1517 Acked Write 3 Sent Done Done false
2042 1518 Acked Read 10 Sent Done Done false
2043 1519 Acked Write 4 Sent Done Done false
2044 1520 Acked Write 10 Sent Done Done false
2045 1521 Acked Write 8 Sent Done Done false
2046 1522 Acked Read 1 Sent Done Done false
2047 1523 Acked Read 6 Sent Done Done false
2048 1524 Acked Write 10 Sent Done Done false
2049 1525 Acked Read 2 Sent Done Done false
2050 1526 Acked Read 2 Sent Done Done false
2051 1527 Acked Write 7 Sent Done Done false
2052 1528 Acked Write 10 Sent Done Done false
2053 1529 Acked Read 5 Sent Done Done false
2054 1530 Acked Read 1 Sent Done Done false
2055 1531 Acked Read 3 Sent Done Done false
2056 1532 Acked Write 10 Sent Done Done false
2057 1533 Acked Flush 0 Sent Done Done false
2058 1534 Acked Read 3 Sent Done Done false
2059 1535 Acked Write 4 Sent Done Done false
2060 1536 Acked Read 7 Sent Done Done false
2061 1537 Acked Read 5 Sent Done Done false
2062 1538 Acked Read 4 Sent Done Done false
2063 1539 Acked Read 10 Sent Done Done false
2064 1540 Acked Write 2 Sent Done Done false
2065 1541 Acked Read 8 Sent Done Done false
2066 1542 Acked Write 1 Sent Done Done false
2067 1543 Acked Write 1 Sent Done Done false
2068 1544 Acked Flush 0 Sent Done Done false
2069 1545 Acked Read 6 Sent Done Done false
2070 1546 Acked Read 6 Sent Done Done false
2071 1547 Acked Write 3 Sent Done Done false
2072 1548 Acked Flush 0 Sent Done Done false
2073 1549 Acked Read 8 Sent Done Done false
2074 1550 Acked Read 8 Sent Done Done false
2075 1551 Acked Read 5 Sent Done Done false
2076 1552 Acked Read 5 Sent Done Done false
2077 1553 Acked Write 3 Sent Done Done false
2078 1554 Acked Read 8 Sent Done Done false
2079 1555 Acked Read 10 Sent Done Done false
2080 1556 Acked Write 6 Sent Done Done false
2081 1557 Acked Write 2 Sent Done Done false
2082 1558 Acked Write 8 Sent Done Done false
2083 1559 Acked Read 2 Sent Done Done false
2084 1560 Acked Read 5 Sent Done Done false
2085 1561 Acked Write 8 Sent Done Done false
2086 1562 Acked Write 5 Sent Done Done false
2087 1563 Acked Write 5 Sent Done Done false
2088 1564 Acked Write 7 Sent Done Done false
2089 1565 Acked Flush 0 Sent Done Done false
2090 1566 Acked Write 9 Sent Done Done false
2091 1567 Acked Write 5 Sent Done Done false
2092 1568 Acked Read 5 Sent Done Done false
2093 1569 Acked Read 8 Sent Done Done false
2094 1570 Acked Read 8 Sent Done Done false
2095 1571 Acked Write 3 Sent Done Done false
2096 1572 Acked Write 7 Sent Done Done false
2097 1573 Acked Read 1 Sent Done Done false
2098 1574 Acked Write 7 Sent Done Done false
2099 1575 Acked Flush 0 Sent Done Done false
2100 1576 Acked Read 9 Sent Done Done false
2101 1577 Acked Flush 0 Sent Done Done false
2102 1578 Acked Read 6 Sent Done Done false
2103 1579 Acked Read 8 Sent Done Done false
2104 1580 Acked Read 8 Sent Done Done false
2105 1581 Acked Read 9 Sent Done Done false
2106 1582 Acked Read 1 Sent Done Done false
2107 1583 Acked Write 8 Sent Done Done false
2108 1584 Acked Write 3 Sent Done Done false
2109 1585 Acked Flush 0 Sent Done Done false
2110 1586 Acked Write 10 Sent Done Done false
2111 1587 Acked Read 7 Sent Done Done false
2112 1588 Acked Write 9 Sent Done Done false
2113 1589 Acked Write 6 Sent Done Done false
2114 1590 Acked Read 5 Sent Done Done false
2115 1591 Acked Write 4 Sent Done Done false
2116 1592 Acked Read 5 Sent Done Done false
2117 1593 Acked Read 1 Sent Done Done false
2118 1594 Acked Read 4 Sent Done Done false
2119 1595 Acked Write 10 Sent Done Done false
2120 1596 Acked Read 5 Sent Done Done false
2121 1597 Acked Flush 0 Sent Done Done false
2122 1598 Acked Read 3 Sent Done Done false
2123 1599 Acked Read 6 Sent Done Done false
2124 1600 Acked Write 3 Sent Done Done false
2125 1601 Acked Write 5 Sent Done Done false
2126 1602 Acked Read 4 Sent Done Done false
2127 1603 Acked Flush 0 Sent Done Done false
2128 STATES DS:0 DS:1 DS:2 TOTAL
2129 Sent 302 0 0 302
2130 Done 0 302 302 604
2131 Skipped 0 0 0 0
2132 Error 0 0 0 0
2133 Last Flush: 1301 1603 1603
2134 Downstairs last five completed: 1301 1300 1299 1298 1297
2135 Upstairs last five completed: 1603 1602 1601 1600 1599
2136 CLIENT: Up:0 ds:302 act:2
21372025-01-27T19:36:47.162ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21382025-01-27T19:36:47.162ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33893) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21392025-01-27T19:36:47.163ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21402025-01-27T19:36:47.164ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21412025-01-27T19:36:47.164ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21422025-01-27T19:36:47.164ZINFOcrucible: send last flush ID to this DS: 1301 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21432025-01-27T19:36:47.164ZINFOcrucible: Replied this last flush ID: 1301 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21442025-01-27T19:36:47.164ZINFOcrucible: [0] client re-new 302 jobs since flush 1301 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21452025-01-27T19:36:47.165ZINFOcrucible: [0] Replayed 302 jobs since flush: 1301 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2146 ----------------------------------------------------------------
2147 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
2148 Downstairs last five completed: 1603 1602 1601 1600 1599
2149 Upstairs last five completed: 1603 1602 1601 1600 1599
2150 CLIENT: Up:0 ds:0 act:3
2151 Replay: All jobs finished
2152 Main task has work to do, go find it
2153 got dsc Stop(2)
2154 stop 2
2155 Jan 27 19:36:49.403 INFO request completed, latency_us: 315[8830] Got stop action so:false kr:false
2156 , response_code: 204, uri: /stop/cid/2, method: POST, req_id: d7d16f20-7f19-4cc3-8d0b-02182f0f4a84, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
2157 Jan 27 19:36:49.404 INFO request completed, latency_us: 117, response_code: 200, uri: /state/cid/2, method: GET, req_id: 319d1c4b-4c75-4ef3-8a8f-8c654d7db738, remote_addr: 127.0.0.{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:36:49.404146486Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
2158 1:55047, local_addr: 127.0.0.1:9998
21592025-01-27T19:36:49.404ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21602025-01-27T19:36:49.404ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21612025-01-27T19:36:49.404ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
2162 [8830] Exited with: ExitStatus(unix_wait_status(9))
2163 [8830][2] reports Exit
2164 Jan 27 19:36:53.406 INFO request completed, latency_us: 241, response_code: 200, uri: /state/cid/2, method: GET, req_id: 128e64e5-6436-4f1b-baab-06fe59abe7a4, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
2165 001/300 Write block 265 len 36864 data: 2 2 2 2 2 2 2 2 2
2166 002/300 Read block 4297 len 4096
2167 003/300 Flush
2168 004/300 Flush
2169 005/300 Read block 1619 len 36864
2170 006/300 Read block 2484 len 24576
2171 007/300 Read block 196 len 8192
2172 008/300 Read block 1366 len 16384
2173 009/300 Write block 4704 len 40960 data: 2 2 2 2 2 3 3 3 3 2
2174 010/300 Read block 631 len 4096
2175 011/300 Read block 776 len 20480
2176 012/300 Read block 1002 len 40960
2177 013/300 Write block 4621 len 24576 data: 2 2 2 2 2 2
2178 014/300 Write block 192 len 40960 data: 3 3 3 3 2 2 2 2 2 2
2179 015/300 Write block 4803 len 36864 data: 3 3 3 2 2 2 2 3 3
2180 016/300 Read block 3601 len 40960
2181 017/300 Read block 2044 len 12288
2182 018/300 Read block 2967 len 20480
2183 019/300 Flush
2184 020/300 Write block 3665 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2185 021/300 Write block 4970 len 8192 data: 2 2
2186 022/300 Read block 2206 len 20480
2187 023/300 Write block 2663 len 12288 data: 2 2 2
2188 024/300 Read block 3191 len 12288
2189 025/300 Write block 4189 len 32768 data: 3 3 3 3 3 3 3 4
2190 026/300 Read block 2748 len 12288
2191 027/300 Write block 2096 len 4096 data: 2
2192 028/300 Write block 3397 len 28672 data: 2 2 2 2 2 2 3
2193 029/300 Read block 4431 len 36864
2194 030/300 Write block 2186 len 8192 data: 2 2
2195 031/300 Write block 474 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2196 032/300 Read block 1147 len 16384
2197 033/300 Read block 2543 len 12288
2198 034/300 Flush
2199 035/300 Flush
2200 036/300 Write block 317 len 20480 data: 2 2 2 3 3
2201 037/300 Flush
2202 038/300 Read block 2405 len 24576
2203 039/300 Read block 1971 len 40960
2204 040/300 Read block 4881 len 8192
2205 041/300 Write block 1772 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2206 042/300 Read block 4413 len 28672
2207 043/300 Write block 3019 len 28672 data: 2 2 2 2 2 2 2
2208 044/300 Read block 656 len 12288
2209 045/300 Read block 1667 len 40960
2210 046/300 Read block 3338 len 4096
2211 047/300 Read block 3342 len 24576
2212 048/300 Read block 1202 len 36864
2213 049/300 Write block 48 len 28672 data: 3 3 3 2 2 2 2
2214 050/300 Write block 4229 len 40960 data: 2 2 2 2 2 2 2 2 2 3
2215 051/300 Write block 1597 len 28672 data: 2 2 2 2 2 2 2
2216 052/300 Read block 152 len 8192
2217 053/300 Read block 891 len 32768
2218 054/300 Read block 1851 len 36864
2219 055/300 Read block 3138 len 32768
2220 056/300 Read block 2809 len 8192
2221 057/300 Read block 2433 len 4096
2222 058/300 Read block 1164 len 8192
2223 059/300 Read block 1554 len 32768
2224 060/300 Read block 629 len 20480
2225 061/300 Flush
2226 062/300 Read block 1653 len 12288
2227 063/300 Read block 1894 len 20480
2228 064/300 Read block 2383 len 28672
2229 065/300 Flush
2230 066/300 Write block 3918 len 8192 data: 2 2
2231 067/300 Flush
2232 068/300 Read block 3264 len 20480
2233 069/300 Read block 117 len 40960
2234 070/300 Read block 4022 len 20480
2235 071/300 Read block 1719 len 24576
2236 072/300 Read block 4054 len 28672
2237 073/300 Read block 1008 len 24576
2238 074/300 Write block 974 len 28672 data: 2 2 2 2 2 2 2
2239 075/300 Flush
2240 076/300 Read block 4013 len 24576
2241 077/300 Write block 2616 len 36864 data: 2 2 2 2 2 2 2 2 2
2242 078/300 Read block 3531 len 28672
2243 079/300 Write block 1329 len 20480 data: 2 2 2 2 2
2244 080/300 Read block 1007 len 8192
2245 081/300 Read block 4423 len 32768
2246 082/300 Read block 833 len 32768
2247 083/300 Read block 1590 len 28672
2248 084/300 Write block 1018 len 16384 data: 3 3 3 2
2249 085/300 Write block 1923 len 28672 data: 2 3 3 3 2 2 2
2250 086/300 Write block 3325 len 8192 data: 3 3
2251 087/300 Read block 981 len 32768
2252 088/300 Read block 1618 len 4096
2253 089/300 Read block 3875 len 20480
2254 090/300 Read block 3961 len 16384
2255 091/300 Write block 1251 len 12288 data: 2 2 2
2256 092/300 Read block 3254 len 16384
2257 093/300 Read block 2677 len 40960
2258 094/300 Read block 2059 len 36864
2259 095/300 Write block 4100 len 32768 data: 2 2 2 2 2 2 2 3
2260 096/300 Write block 1070 len 24576 data: 2 2 2 2 2 2
2261 097/300 Flush
2262 098/300 Read block 3334 len 12288
2263 099/300 Read block 3019 len 8192
2264 100/300 Write block 3599 len 8192 data: 2 2
2265 101/300 Read block 1493 len 32768
2266 102/300 Write block 4521 len 40960 data: 3 3 2 2 2 3 3 3 2 2
2267 103/300 Read block 4951 len 12288
2268 104/300 Write block 3936 len 24576 data: 2 2 2 2 2 2
2269 105/300 Read block 2130 len 40960
2270 106/300 Write block 694 len 32768 data: 2 2 2 2 2 2 2 2
2271 107/300 Read block 409 len 40960
2272 108/300 Flush
2273 109/300 Read block 2484 len 4096
2274 110/300 Read block 518 len 32768
2275 111/300 Read block 2308 len 12288
2276 112/300 Read block 3904 len 4096
2277 113/300 Write block 1527 len 4096 data: 2
2278 114/300 Write block 3038 len 36864 data: 3 3 3 3 2 2 2 2 2
2279 115/300 Read block 2568 len 12288
2280 116/300 Write block 1095 len 28672 data: 2 2 2 2 2 2 2
2281 117/300 Write block 4797 len 36864 data: 3 2 3 3 3 3 4 4 4
2282 118/300 Write block 2057 len 16384 data: 2 2 2 2
2283 119/300 Write block 3773 len 28672 data: 4 3 3 3 3 3 2
2284 120/300 Write block 401 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2285 121/300 Read block 4091 len 28672
2286 122/300 Write block 2184 len 16384 data: 3 2 3 3
2287 123/300 Read block 2585 len 36864
2288 124/300 Flush
2289 125/300 Read block 3055 len 16384
2290 126/300 Flush
2291 127/300 Flush
2292 128/300 Write block 445 len 4096 data: 2
2293 129/300 Read block 4843 len 4096
2294 130/300 Read block 4414 len 28672
2295 131/300 Write block 1974 len 20480 data: 2 2 2 2 2
2296 132/300 Flush
2297 133/300 Write block 3499 len 8192 data: 2 2
2298 134/300 Write block 62 len 36864 data: 2 2 2 2 2 2 2 2 2
2299 135/300 Read block 962 len 16384
2300 136/300 Read block 840 len 32768
2301 137/300 Read block 1145 len 12288
2302 138/300 Flush
2303 139/300 Write block 4198 len 8192 data: 3 2
2304 140/300 Read block 1477 len 36864
2305 141/300 Read block 1417 len 40960
2306 142/300 Read block 469 len 20480
2307 143/300 Write block 603 len 28672 data: 4 4 4 2 3 3 3
2308 144/300 Read block 1750 len 20480
2309 145/300 Read block 2229 len 32768
2310 146/300 Read block 955 len 20480
2311 147/300 Write block 1855 len 20480 data: 3 2 2 2 2
2312 148/300 Write block 4601 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2313 149/300 Flush
2314 150/300 Read block 4298 len 8192
2315 151/300 Read block 4740 len 24576
2316 152/300 Write block 507 len 20480 data: 2 2 2 2 2
2317 153/300 Write block 4001 len 40960 data: 2 2 2 2 2 2 2 3 2 2
2318 154/300 Read block 3204 len 36864
2319 155/300 Write block 4169 len 40960 data: 2 3 3 3 4 3 3 3 3 3
2320 156/300 Write block 2718 len 20480 data: 2 2 2 2 2
2321 157/300 Write block 4713 len 20480 data: 3 3 3 3 3
2322 158/300 Write block 1852 len 40960 data: 3 3 3 4 3 3 3 3 2 2
2323 159/300 Read block 4228 len 40960
2324 160/300 Read block 2002 len 8192
2325 161/300 Write block 3449 len 36864 data: 2 2 2 2 2 2 3 3 3
2326 162/300 Read block 1862 len 16384
2327 163/300 Flush
2328 164/300 Read block 831 len 16384
2329 165/300 Flush
2330 166/300 Write block 2765 len 32768 data: 2 2 3 3 3 3 3 3
2331 167/300 Write block 2635 len 28672 data: 3 3 3 4 4 4 4
2332 168/300 Read block 3494 len 4096
2333 169/300 Write block 2059 len 16384 data: 3 3 2 2
2334 170/300 Read block 173 len 20480
2335 171/300 Write block 3504 len 8192 data: 2 2
2336 172/300 Read block 1285 len 12288
2337 173/300 Read block 3676 len 20480
2338 174/300 Flush
2339 175/300 Read block 4922 len 4096
2340 176/300 Flush
2341 177/300 Read block 181 len 24576
2342 178/300 Write block 3665 len 36864 data: 3 3 3 3 3 3 3 3 3
2343 179/300 Read block 2374 len 16384
2344 180/300 Read block 3486 len 4096
2345 181/300 Flush
2346 182/300 Read block 3516 len 20480
2347 183/300 Read block 591 len 4096
2348 184/300 Flush
2349 185/300 Write block 4361 len 36864 data: 2 2 2 2 2 2 2 2 2
2350 186/300 Read block 1187 len 8192
2351 187/300 Read block 2771 len 40960
2352 188/300 Flush
2353 189/300 Write block 1376 len 32768 data: 5 5 4 4 4 3 3 3
2354 190/300 Read block 1647 len 28672
2355 191/300 Write block 280 len 28672 data: 3 3 3 2 2 2 2
2356 192/300 Read block 2198 len 24576
2357 193/300 Write block 3952 len 32768 data: 2 2 2 2 2 2 2 2
2358 194/300 Read block 3637 len 32768
2359 195/300 Read block 775 len 24576
2360 196/300 Read block 2082 len 32768
2361 197/300 Read block 3409 len 12288
2362 198/300 Read block 758 len 24576
2363 199/300 Read block 2942 len 8192
2364 200/300 Read block 2838 len 32768
2365 201/300 Flush
2366 202/300 Write block 3332 len 4096 data: 2
2367 203/300 Read block 4271 len 12288
2368 204/300 Read block 2557 len 8192
2369 205/300 Write block 4353 len 4096 data: 2
2370 206/300 Write block 982 len 32768 data: 2 2 2 2 2 2 2 2
2371 207/300 Flush
2372 208/300 Write block 4908 len 16384 data: 2 2 2 2
2373 209/300 Read block 3683 len 8192
2374 210/300 Read block 3565 len 40960
2375 211/300 Read block 2068 len 40960
2376 212/300 Write block 3239 len 16384 data: 2 2 2 2
2377 213/300 Flush
2378 214/300 Read block 4018 len 28672
2379 215/300 Flush
2380 216/300 Read block 3992 len 4096
2381 217/300 Write block 351 len 32768 data: 2 2 2 2 2 2 2 2
2382 218/300 Read block 2250 len 12288
2383 219/300 Read block 2455 len 36864
2384 220/300 Write block 4118 len 24576 data: 3 3 3 3 3 2
2385 221/300 Read block 2421 len 32768
2386 222/300 Write block 653 len 4096 data: 2
2387 223/300 Read block 3080 len 4096
2388 224/300 Write block 4875 len 40960 data: 2 2 2 2 2 2 2 2 2 2
2389 225/300 Read block 1862 len 32768
2390 226/300 Read block 3333 len 24576
2391 227/300 Write block 1332 len 36864 data: 3 3 2 2 2 2 2 2 2
2392 228/300 Write block 3136 len 32768 data: 3 2 2 2 2 2 2 2
2393 229/300 Write block 3243 len 4096 data: 2
2394 230/300 Read block 2112 len 16384
2395 231/300 Write block 4681 len 32768 data: 3 2 2 2 2 2 2 2
2396 232/300 Read block 2371 len 20480
2397 233/300 Write block 871 len 36864 data: 2 2 2 2 2 2 2 2 2
2398 234/300 Read block 1139 len 32768
2399 235/300 Flush
2400 236/300 Write block 2064 len 28672 data: 2 2 2 2 3 2 2
2401 237/300 Read block 3988 len 8192
2402 238/300 Read block 3291 len 12288
2403 239/300 Read block 3561 len 28672
2404 240/300 Write block 2285 len 32768 data: 3 3 3 4 4 4 3 3
2405 241/300 Write block 1985 len 20480 data: 3 3 3 3 2
2406 242/300 Read block 2441 len 8192
2407 243/300 Write block 4165 len 4096 data: 3
2408 244/300 Write block 1184 len 8192 data: 2 2
2409 245/300 Write block 713 len 16384 data: 5 5 4 3
2410 246/300 Read block 243 len 28672
2411 247/300 Write block 1555 len 24576 data: 2 2 2 2 2 2
2412 248/300 Read block 2710 len 20480
2413 249/300 Write block 4413 len 28672 data: 2 2 2 2 2 2 2
2414 250/300 Flush
2415 251/300 Write block 2862 len 24576 data: 2 2 2 2 2 2
2416 252/300 Flush
2417 253/300 Read block 2054 len 8192
2418 254/300 Read block 4030 len 40960
2419 255/300 Read block 4293 len 12288
2420 256/300 Read block 237 len 8192
2421 257/300 Write block 1470 len 16384 data: 2 2 2 2
2422 258/300 Flush
2423 259/300 Flush
2424 260/300 Write block 1672 len 20480 data: 2 2 2 2 2
2425 261/300 Write block 3902 len 32768 data: 2 2 5 4 4 4 5 5
2426 262/300 Read block 4970 len 40960
2427 263/300 Write block 1057 len 20480 data: 2 2 2 2 2
2428 264/300 Write block 4141 len 40960 data: 3 3 3 2 2 2 2 2 2 2
2429 265/300 Write block 1473 len 16384 data: 3 2 2 2
2430 266/300 Flush
2431 267/300 Read block 4310 len 24576
2432 268/300 Read block 4852 len 28672
2433 269/300 Write block 3861 len 32768 data: 2 2 2 2 2 2 2 2
2434 270/300 Write block 2402 len 8192 data: 2 2
2435 271/300 Read block 2970 len 24576
2436 272/300 Write block 439 len 20480 data: 2 2 2 2 2
2437 273/300 Write block 4764 len 8192 data: 2 2
2438 274/300 Read block 3384 len 36864
2439 275/300 Read block 2705 len 4096
2440 276/300 Read block 3447 len 40960
2441 277/300 Read block 2854 len 24576
2442 278/300 Read block 56 len 40960
2443 279/300 Read block 3364 len 24576
2444 280/300 Flush
2445 281/300 Read block 3302 len 40960
2446 282/300 Read block 4386 len 8192
2447 283/300 Write block 3326 len 28672 data: 4 2 2 2 2 2 3
2448 284/300 Write block 3057 len 20480 data: 2 2 2 2 2
2449 285/300 Read block 4919 len 28672
2450 286/300 Write block 3453 len 40960 data: 3 3 4 4 4 2 2 2 2 2
2451 287/300 Write block 2048 len 16384 data: 3 2 2 2
2452 288/300 Write block 1223 len 16384 data: 3 3 3 2
2453 289/300 Read block 4307 len 28672
2454 290/300 Read block 3608 len 32768
2455 291/300 Read block 3010 len 8192
2456 292/300 Read block 3321 len 40960
2457 293/300 Read block 4930 len 28672
2458 294/300 Read block 801 len 16384
2459 295/300 Read block 3255 len 20480
2460 296/300 Write block 1201 len 28672 data: 2 2 2 2 2 2 3
2461 297/300 Read block 1736 len 12288
2462 298/300 Write block 2144 len 32768 data: 2 2 2 2 2 2 2 2
2463 299/300 Read block 1453 len 32768
2464 300/300 Read block 2099 len 16384
2465 301/300 Write block 2296 len 4096 data: 2
2466 Main task has work to do, go find it
2467 got dsc Start(2)
2468 start 2
2469 Jan 27 19:36:53.650 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/2, method: POST, req_id: f6dc7369-aef6-4af2-a4c1-a449a20809e9, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
2470 Replay: started 2, returned:Ok(())
2471 ----------------------------------------------------------------
2472 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
2473 States: Active Active Offline
2474 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2475 1604 Acked Write 9 Done Done Sent false
2476 1605 Acked Read 1 Done Done Sent false
2477 1606 Acked Flush 0 Done Done Sent false
2478 1607 Acked Flush 0 Done Done Sent false
2479 1608 Acked Read 9 Done Done Sent false
2480 1609 Acked Read 6 Done Done Sent false
2481 1610 Acked Read 2 Done Done Sent false
2482 1611 Acked Read 4 Done Done Sent false
2483 1612 Acked Write 10 Done Done Sent false
2484 1613 Acked Read 1 Done Done Sent false
2485 1614 Acked Read 5 Done Done Sent false
2486 1615 Acked Read 10 Done Done Sent false
2487 1616 Acked Write 6 Done Done Sent false
2488 1617 Acked Write 10 Done Done Sent false
2489 1618 Acked Write 9 Done Done Sent false
2490 1619 Acked Read 10 Done Done Sent false
2491 1620 Acked Read 3 Done Done Sent false
2492 1621 Acked Read 5 Done Done Sent false
2493 1622 Acked Flush 0 Done Done Sent false
2494 1623 Acked Write 10 Done Done Sent false
2495 1624 Acked Write 2 Done Done Sent false
2496 1625 Acked Read 5 Done Done Sent false
2497 1626 Acked Write 3 Done Done Sent false
2498 1627 Acked Read 3 Done Done Sent false
2499 1628 Acked Write 8 Done Done Sent false
2500 1629 Acked Read 3 Done Done Sent false
2501 1630 Acked Write 1 Done Done Sent false
2502 1631 Acked Write 7 Done Done Sent false
2503 1632 Acked Read 9 Done Done Sent false
2504 1633 Acked Write 2 Done Done Sent false
2505 1634 Acked Write 10 Done Done Sent false
2506 1635 Acked Read 4 Done Done Sent false
2507 1636 Acked Read 3 Done Done Sent false
2508 1637 Acked Flush 0 Done Done Sent false
2509 1638 Acked Flush 0 Done Done Sent false
2510 1639 Acked Write 5 Done Done Sent false
2511 1640 Acked Flush 0 Done Done Sent false
2512 1641 Acked Read 6 Done Done Sent false
2513 1642 Acked Read 10 Done Done Sent false
2514 1643 Acked Read 2 Done Done Sent false
2515 1644 Acked Write 10 Done Done Sent false
2516 1645 Acked Read 7 Done Done Sent false
2517 1646 Acked Write 7 Done Done Sent false
2518 1647 Acked Read 3 Done Done Sent false
2519 1648 Acked Read 10 Done Done Sent false
2520 1649 Acked Read 1 Done Done Sent false
2521 1650 Acked Read 6 Done Done Sent false
2522 1651 Acked Read 9 Done Done Sent false
2523 1652 Acked Write 7 Done Done Sent false
2524 1653 Acked Write 10 Done Done Sent false
2525 1654 Acked Write 7 Done Done Sent false
2526 1655 Acked Read 2 Done Done Sent false
2527 1656 Acked Read 8 Done Done Sent false
2528 1657 Acked Read 9 Done Done Sent false
2529 1658 Acked Read 8 Done Done Sent false
2530 1659 Acked Read 2 Done Done Sent false
2531 1660 Acked Read 1 Done Done Sent false
2532 1661 Acked Read 2 Done Done Sent false
2533 1662 Acked Read 8 Done Done Sent false
2534 1663 Acked Read 5 Done Done Sent false
2535 1664 Acked Flush 0 Done Done Sent false
2536 1665 Acked Read 3 Done Done Sent false
2537 1666 Acked Read 5 Done Done Sent false
2538 1667 Acked Read 7 Done Done Sent false
2539 1668 Acked Flush 0 Done Done Sent false
2540 1669 Acked Write 2 Done Done Sent false
2541 1670 Acked Flush 0 Done Done Sent false
2542 1671 Acked Read 5 Done Done Sent false
2543 1672 Acked Read 10 Done Done Sent false
2544 1673 Acked Read 5 Done Done Sent false
2545 1674 Acked Read 6 Done Done Sent false
2546 1675 Acked Read 7 Done Done Sent false
2547 1676 Acked Read 6 Done Done Sent false
2548 1677 Acked Write 7 Done Done Sent false
2549 1678 Acked Flush 0 Done Done Sent false
2550 1679 Acked Read 6 Done Done Sent false
2551 1680 Acked Write 9 Done Done Sent false
2552 1681 Acked Read 7 Done Done Sent false
2553 1682 Acked Write 5 Done Done Sent false
2554 1683 Acked Read 2 Done Done Sent false
2555 1684 Acked Read 8 Done Done Sent false
2556 1685 Acked Read 8 Done Done Sent false
2557 1686 Acked Read 7 Done Done Sent false
2558 1687 Acked Write 4 Done Done Sent false
2559 1688 Acked Write 7 Done Done Sent false
2560 1689 Acked Write 2 Done Done Sent false
2561 1690 Acked Read 8 Done Done Sent false
2562 1691 Acked Read 1 Done Done Sent false
2563 1692 Acked Read 5 Done Done Sent false
2564 1693 Acked Read 4 Done Done Sent false
2565 1694 Acked Write 3 Done Done Sent false
2566 1695 Acked Read 4 Done Done Sent false
2567 1696 Acked Read 10 Done Done Sent false
2568 1697 Acked Read 9 Done Done Sent false
2569 1698 Acked Write 8 Done Done Sent false
2570 1699 Acked Write 6 Done Done Sent false
2571 1700 Acked Flush 0 Done Done Sent false
2572 1701 Acked Read 3 Done Done Sent false
2573 1702 Acked Read 2 Done Done Sent false
2574 1703 Acked Write 2 Done Done Sent false
2575 1704 Acked Read 8 Done Done Sent false
2576 1705 Acked Write 10 Done Done Sent false
2577 1706 Acked Read 3 Done Done Sent false
2578 1707 Acked Write 6 Done Done Sent false
2579 1708 Acked Read 10 Done Done Sent false
2580 1709 Acked Write 8 Done Done Sent false
2581 1710 Acked Read 10 Done Done Sent false
2582 1711 Acked Flush 0 Done Done Sent false
2583 1712 Acked Read 1 Done Done Sent false
2584 1713 Acked Read 8 Done Done Sent false
2585 1714 Acked Read 3 Done Done Sent false
2586 1715 Acked Read 1 Done Done Sent false
2587 1716 Acked Write 1 Done Done Sent false
2588 1717 Acked Write 9 Done Done Sent false
2589 1718 Acked Read 3 Done Done Sent false
2590 1719 Acked Write 7 Done Done Sent false
2591 1720 Acked Write 9 Done Done Sent false
2592 1721 Acked Write 4 Done Done Sent false
2593 1722 Acked Write 7 Done Done Sent false
2594 1723 Acked Write 10 Done Done Sent false
2595 1724 Acked Read 7 Done Done Sent false
2596 1725 Acked Write 4 Done Done Sent false
2597 1726 Acked Read 9 Done Done Sent false
2598 1727 Acked Flush 0 Done Done Sent false
2599 1728 Acked Read 4 Done Done Sent false
2600 1729 Acked Flush 0 Done Done Sent false
2601 1730 Acked Flush 0 Done Done Sent false
2602 1731 Acked Write 1 Done Done Sent false
2603 1732 Acked Read 1 Done Done Sent false
2604 1733 Acked Read 7 Done Done Sent false
2605 1734 Acked Write 5 Done Done Sent false
2606 1735 Acked Flush 0 Done Done Sent false
2607 1736 Acked Write 2 Done Done Sent false
2608 1737 Acked Write 9 Done Done Sent false
2609 1738 Acked Read 4 Done Done Sent false
2610 1739 Acked Read 8 Done Done Sent false
2611 1740 Acked Read 3 Done Done Sent false
2612 1741 Acked Flush 0 Done Done Sent false
2613 1742 Acked Write 2 Done Done Sent false
2614 1743 Acked Read 9 Done Done Sent false
2615 1744 Acked Read 10 Done Done Sent false
2616 1745 Acked Read 5 Done Done Sent false
2617 1746 Acked Write 7 Done Done Sent false
2618 1747 Acked Read 5 Done Done Sent false
2619 1748 Acked Read 8 Done Done Sent false
2620 1749 Acked Read 5 Done Done Sent false
2621 1750 Acked Write 5 Done Done Sent false
2622 1751 Acked Write 10 Done Done Sent false
2623 1752 Acked Flush 0 Done Done Sent false
2624 1753 Acked Read 2 Done Done Sent false
2625 1754 Acked Read 6 Done Done Sent false
2626 1755 Acked Write 5 Done Done Sent false
2627 1756 Acked Write 10 Done Done Sent false
2628 1757 Acked Read 9 Done Done Sent false
2629 1758 Acked Write 10 Done Done Sent false
2630 1759 Acked Write 5 Done Done Sent false
2631 1760 Acked Write 5 Done Done Sent false
2632 1761 Acked Write 10 Done Done Sent false
2633 1762 Acked Read 10 Done Done Sent false
2634 1763 Acked Read 2 Done Done Sent false
2635 1764 Acked Write 9 Done Done Sent false
2636 1765 Acked Read 4 Done Done Sent false
2637 1766 Acked Flush 0 Done Done Sent false
2638 1767 Acked Read 4 Done Done Sent false
2639 1768 Acked Flush 0 Done Done Sent false
2640 1769 Acked Write 8 Done Done Sent false
2641 1770 Acked Write 7 Done Done Sent false
2642 1771 Acked Read 1 Done Done Sent false
2643 1772 Acked Write 4 Done Done Sent false
2644 1773 Acked Read 5 Done Done Sent false
2645 1774 Acked Write 2 Done Done Sent false
2646 1775 Acked Read 3 Done Done Sent false
2647 1776 Acked Read 5 Done Done Sent false
2648 1777 Acked Flush 0 Done Done Sent false
2649 1778 Acked Read 1 Done Done Sent false
2650 1779 Acked Flush 0 Done Done Sent false
2651 1780 Acked Read 6 Done Done Sent false
2652 1781 Acked Write 9 Done Done Sent false
2653 1782 Acked Read 4 Done Done Sent false
2654 1783 Acked Read 1 Done Done Sent false
2655 1784 Acked Flush 0 Done Done Sent false
2656 1785 Acked Read 5 Done Done Sent false
2657 1786 Acked Read 1 Done Done Sent false
2658 1787 Acked Flush 0 Done Done Sent false
2659 1788 Acked Write 9 Done Done Sent false
2660 1789 Acked Read 2 Done Done Sent false
2661 1790 Acked Read 10 Done Done Sent false
2662 1791 Acked Flush 0 Done Done Sent false
2663 1792 Acked Write 8 Done Done Sent false
2664 1793 Acked Read 7 Done Done Sent false
2665 1794 Acked Write 7 Done Done Sent false
2666 1795 Acked Read 6 Done Done Sent false
2667 1796 Acked Write 8 Done Done Sent false
2668 1797 Acked Read 8 Done Done Sent false
2669 1798 Acked Read 6 Done Done Sent false
2670 1799 Acked Read 8 Done Done Sent false
2671 1800 Acked Read 3 Done Done Sent false
2672 1801 Acked Read 6 Done Done Sent false
2673 1802 Acked Read 2 Done Done Sent false
2674 1803 Acked Read 8 Done Done Sent false
2675 1804 Acked Flush 0 Done Done Sent false
2676 1805 Acked Write 1 Done Done Sent false
2677 1806 Acked Read 3 Done Done Sent false
2678 1807 Acked Read 2 Done Done Sent false
2679 1808 Acked Write 1 Done Done Sent false
2680 1809 Acked Write 8 Done Done Sent false
2681 1810 Acked Flush 0 Done Done Sent false
2682 1811 Acked Write 4 Done Done Sent false
2683 1812 Acked Read 2 Done Done Sent false
2684 1813 Acked Read 10 Done Done Sent false
2685 1814 Acked Read 10 Done Done Sent false
2686 1815 Acked Write 4 Done Done Sent false
2687 1816 Acked Flush 0 Done Done Sent false
2688 1817 Acked Read 7 Done Done Sent false
2689 1818 Acked Flush 0 Done Done Sent false
2690 1819 Acked Read 1 Done Done Sent false
2691 1820 Acked Write 8 Done Done Sent false
2692 1821 Acked Read 3 Done Done Sent false
2693 1822 Acked Read 9 Done Done Sent false
2694 1823 Acked Write 6 Done Done Sent false
2695 1824 Acked Read 8 Done Done Sent false
2696 1825 Acked Write 1 Done Done Sent false
2697 1826 Acked Read 1 Done Done Sent false
2698 1827 Acked Write 10 Done Done Sent false
2699 1828 Acked Read 8 Done Done Sent false
2700 1829 Acked Read 6 Done Done Sent false
2701 1830 Acked Write 9 Done Done Sent false
2702 1831 Acked Write 8 Done Done Sent false
2703 1832 Acked Write 1 Done Done Sent false
2704 1833 Acked Read 4 Done Done Sent false
2705 1834 Acked Write 8 Done Done Sent false
2706 1835 Acked Read 5 Done Done Sent false
2707 1836 Acked Write 9 Done Done Sent false
2708 1837 Acked Read 8 Done Done Sent false
2709 1838 Acked Flush 0 Done Done Sent false
2710 1839 Acked Write 7 Done Done Sent false
2711 1840 Acked Read 2 Done Done Sent false
2712 1841 Acked Read 3 Done Done Sent false
2713 1842 Acked Read 7 Done Done Sent false
2714 1843 Acked Write 8 Done Done Sent false
2715 1844 Acked Write 5 Done Done Sent false
2716 1845 Acked Read 2 Done Done Sent false
2717 1846 Acked Write 1 Done Done Sent false
2718 1847 Acked Write 2 Done Done Sent false
2719 1848 Acked Write 4 Done Done Sent false
2720 1849 Acked Read 7 Done Done Sent false
2721 1850 Acked Write 6 Done Done Sent false
2722 1851 Acked Read 5 Done Done Sent false
2723 1852 Acked Write 7 Done Done Sent false
2724 1853 Acked Flush 0 Done Done Sent false
2725 1854 Acked Write 6 Done Done Sent false
2726 1855 Acked Flush 0 Done Done Sent false
2727 1856 Acked Read 2 Done Done Sent false
2728 1857 Acked Read 10 Done Done Sent false
2729 1858 Acked Read 3 Done Done Sent false
2730 1859 Acked Read 2 Done Done Sent false
2731 1860 Acked Write 4 Done Done Sent false
2732 1861 Acked Flush 0 Done Done Sent false
2733 1862 Acked Flush 0 Done Done Sent false
2734 1863 Acked Write 5 Done Done Sent false
2735 1864 Acked Write 8 Done Done Sent false
2736 1865 Acked Read 10 Done Done Sent false
2737 1866 Acked Write 5 Done Done Sent false
2738 1867 Acked Write 10 Done Done Sent false
2739 1868 Acked Write 4 Done Done Sent false
2740 1869 Acked Flush 0 Done Done Sent false
2741 1870 Acked Read 6 Done Done Sent false
2742 1871 Acked Read 7 Done Done Sent false
2743 1872 Acked Write 8 Done Done Sent false
2744 1873 Acked Write 2 Done Done Sent false
2745 1874 Acked Read 6 Done Done Sent false
2746 1875 Acked Write 5 Done Done Sent false
2747 1876 Acked Write 2 Done Done Sent false
2748 1877 Acked Read 9 Done Done Sent false
2749 1878 Acked Read 1 Done Done Sent false
2750 1879 Acked Read 10 Done Done Sent false
2751 1880 Acked Read 6 Done Done Sent false
2752 1881 Acked Read 10 Done Done Sent false
2753 1882 Acked Read 6 Done Done Sent false
2754 1883 Acked Flush 0 Done Done Sent false
2755 1884 Acked Read 10 Done Done Sent false
2756 1885 Acked Read 2 Done Done Sent false
2757 1886 Acked Write 7 Done Done Sent false
2758 1887 Acked Write 5 Done Done Sent false
2759 1888 Acked Read 7 Done Done Sent false
2760 1889 Acked Write 10 Done Done Sent false
2761 1890 Acked Write 4 Done Done Sent false
2762 1891 Acked Write 4 Done Done Sent false
2763 1892 Acked Read 7 Done Done Sent false
2764 1893 Acked Read 8 Done Done Sent false
2765 1894 Acked Read 2 Done Done Sent false
2766 1895 Acked Read 10 Done Done Sent false
2767 1896 Acked Read 7 Done Done Sent false
2768 1897 Acked Read 4 Done Done Sent false
2769 1898 Acked Read 5 Done Done Sent false
2770 1899 Acked Write 7 Done Done Sent false
2771 1900 Acked Read 3 Done Done Sent false
2772 1901 Acked Write 8 Done Done Sent false
2773 1902 Acked Read 8 Done Done Sent false
2774 1903 Acked Read 4 Done Done Sent false
2775 1904 Acked Write 1 Done Done Sent false
2776 STATES DS:0 DS:1 DS:2 TOTAL
2777 Sent 0 0 301 301
2778 Done 301 301 0 602
2779 Skipped 0 0 0 0
2780 Error 0 0 0 0
2781 Last Flush: 1883 1883 1603
2782 Downstairs last five completed: 1603 1602 1601 1600 1599
2783 Upstairs last five completed: 1904 1903 1902 1901 1900
2784 CLIENT: Up:0 ds:301 act:2
2785 [8830] got start action
2786 [8830] I am going to restart
2787 Starting downstairs at port 8830
2788 Make output file at "/tmp/dsc/downstairs-8830.txt"
2789 [8830][2] reports Starting
2790 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1203)
2791 [8830][2] reports Running
2792 ----------------------------------------------------------------
2793 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
2794 States: Active Active Offline
2795 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
2796 1604 Acked Write 9 Done Done Sent false
2797 1605 Acked Read 1 Done Done Sent false
2798 1606 Acked Flush 0 Done Done Sent false
2799 1607 Acked Flush 0 Done Done Sent false
2800 1608 Acked Read 9 Done Done Sent false
2801 1609 Acked Read 6 Done Done Sent false
2802 1610 Acked Read 2 Done Done Sent false
2803 1611 Acked Read 4 Done Done Sent false
2804 1612 Acked Write 10 Done Done Sent false
2805 1613 Acked Read 1 Done Done Sent false
2806 1614 Acked Read 5 Done Done Sent false
2807 1615 Acked Read 10 Done Done Sent false
2808 1616 Acked Write 6 Done Done Sent false
2809 1617 Acked Write 10 Done Done Sent false
2810 1618 Acked Write 9 Done Done Sent false
2811 1619 Acked Read 10 Done Done Sent false
2812 1620 Acked Read 3 Done Done Sent false
2813 1621 Acked Read 5 Done Done Sent false
2814 1622 Acked Flush 0 Done Done Sent false
2815 1623 Acked Write 10 Done Done Sent false
2816 1624 Acked Write 2 Done Done Sent false
2817 1625 Acked Read 5 Done Done Sent false
2818 1626 Acked Write 3 Done Done Sent false
2819 1627 Acked Read 3 Done Done Sent false
2820 1628 Acked Write 8 Done Done Sent false
2821 1629 Acked Read 3 Done Done Sent false
2822 1630 Acked Write 1 Done Done Sent false
2823 1631 Acked Write 7 Done Done Sent false
2824 1632 Acked Read 9 Done Done Sent false
2825 1633 Acked Write 2 Done Done Sent false
2826 1634 Acked Write 10 Done Done Sent false
2827 1635 Acked Read 4 Done Done Sent false
2828 1636 Acked Read 3 Done Done Sent false
2829 1637 Acked Flush 0 Done Done Sent false
2830 1638 Acked Flush 0 Done Done Sent false
2831 1639 Acked Write 5 Done Done Sent false
2832 1640 Acked Flush 0 Done Done Sent false
2833 1641 Acked Read 6 Done Done Sent false
2834 1642 Acked Read 10 Done Done Sent false
2835 1643 Acked Read 2 Done Done Sent false
2836 1644 Acked Write 10 Done Done Sent false
2837 1645 Acked Read 7 Done Done Sent false
2838 1646 Acked Write 7 Done Done Sent false
2839 1647 Acked Read 3 Done Done Sent false
2840 1648 Acked Read 10 Done Done Sent false
2841 1649 Acked Read 1 Done Done Sent false
2842 1650 Acked Read 6 Done Done Sent false
2843 1651 Acked Read 9 Done Done Sent false
2844 1652 Acked Write 7 Done Done Sent false
2845 1653 Acked Write 10 Done Done Sent false
2846 1654 Acked Write 7 Done Done Sent false
2847 1655 Acked Read 2 Done Done Sent false
2848 1656 Acked Read 8 Done Done Sent false
2849 1657 Acked Read 9 Done Done Sent false
2850 1658 Acked Read 8 Done Done Sent false
2851 1659 Acked Read 2 Done Done Sent false
2852 1660 Acked Read 1 Done Done Sent false
2853 1661 Acked Read 2 Done Done Sent false
2854 1662 Acked Read 8 Done Done Sent false
2855 1663 Acked Read 5 Done Done Sent false
2856 1664 Acked Flush 0 Done Done Sent false
2857 1665 Acked Read 3 Done Done Sent false
2858 1666 Acked Read 5 Done Done Sent false
2859 1667 Acked Read 7 Done Done Sent false
2860 1668 Acked Flush 0 Done Done Sent false
2861 1669 Acked Write 2 Done Done Sent false
2862 1670 Acked Flush 0 Done Done Sent false
2863 1671 Acked Read 5 Done Done Sent false
2864 1672 Acked Read 10 Done Done Sent false
2865 1673 Acked Read 5 Done Done Sent false
2866 1674 Acked Read 6 Done Done Sent false
2867 1675 Acked Read 7 Done Done Sent false
2868 1676 Acked Read 6 Done Done Sent false
2869 1677 Acked Write 7 Done Done Sent false
2870 1678 Acked Flush 0 Done Done Sent false
2871 1679 Acked Read 6 Done Done Sent false
2872 1680 Acked Write 9 Done Done Sent false
2873 1681 Acked Read 7 Done Done Sent false
2874 1682 Acked Write 5 Done Done Sent false
2875 1683 Acked Read 2 Done Done Sent false
2876 1684 Acked Read 8 Done Done Sent false
2877 1685 Acked Read 8 Done Done Sent false
2878 1686 Acked Read 7 Done Done Sent false
2879 1687 Acked Write 4 Done Done Sent false
2880 1688 Acked Write 7 Done Done Sent false
2881 1689 Acked Write 2 Done Done Sent false
2882 1690 Acked Read 8 Done Done Sent false
2883 1691 Acked Read 1 Done Done Sent false
2884 1692 Acked Read 5 Done Done Sent false
2885 1693 Acked Read 4 Done Done Sent false
2886 1694 Acked Write 3 Done Done Sent false
2887 1695 Acked Read 4 Done Done Sent false
2888 1696 Acked Read 10 Done Done Sent false
2889 1697 Acked Read 9 Done Done Sent false
2890 1698 Acked Write 8 Done Done Sent false
2891 1699 Acked Write 6 Done Done Sent false
2892 1700 Acked Flush 0 Done Done Sent false
2893 1701 Acked Read 3 Done Done Sent false
2894 1702 Acked Read 2 Done Done Sent false
2895 1703 Acked Write 2 Done Done Sent false
2896 1704 Acked Read 8 Done Done Sent false
2897 1705 Acked Write 10 Done Done Sent false
2898 1706 Acked Read 3 Done Done Sent false
2899 1707 Acked Write 6 Done Done Sent false
2900 1708 Acked Read 10 Done Done Sent false
2901 1709 Acked Write 8 Done Done Sent false
2902 1710 Acked Read 10 Done Done Sent false
2903 1711 Acked Flush 0 Done Done Sent false
2904 1712 Acked Read 1 Done Done Sent false
2905 1713 Acked Read 8 Done Done Sent false
2906 1714 Acked Read 3 Done Done Sent false
2907 1715 Acked Read 1 Done Done Sent false
2908 1716 Acked Write 1 Done Done Sent false
2909 1717 Acked Write 9 Done Done Sent false
2910 1718 Acked Read 3 Done Done Sent false
2911 1719 Acked Write 7 Done Done Sent false
2912 1720 Acked Write 9 Done Done Sent false
2913 1721 Acked Write 4 Done Done Sent false
2914 1722 Acked Write 7 Done Done Sent false
2915 1723 Acked Write 10 Done Done Sent false
2916 1724 Acked Read 7 Done Done Sent false
2917 1725 Acked Write 4 Done Done Sent false
2918 1726 Acked Read 9 Done Done Sent false
2919 1727 Acked Flush 0 Done Done Sent false
2920 1728 Acked Read 4 Done Done Sent false
2921 1729 Acked Flush 0 Done Done Sent false
2922 1730 Acked Flush 0 Done Done Sent false
2923 1731 Acked Write 1 Done Done Sent false
2924 1732 Acked Read 1 Done Done Sent false
2925 1733 Acked Read 7 Done Done Sent false
2926 1734 Acked Write 5 Done Done Sent false
2927 1735 Acked Flush 0 Done Done Sent false
2928 1736 Acked Write 2 Done Done Sent false
2929 1737 Acked Write 9 Done Done Sent false
2930 1738 Acked Read 4 Done Done Sent false
2931 1739 Acked Read 8 Done Done Sent false
2932 1740 Acked Read 3 Done Done Sent false
2933 1741 Acked Flush 0 Done Done Sent false
2934 1742 Acked Write 2 Done Done Sent false
2935 1743 Acked Read 9 Done Done Sent false
2936 1744 Acked Read 10 Done Done Sent false
2937 1745 Acked Read 5 Done Done Sent false
2938 1746 Acked Write 7 Done Done Sent false
2939 1747 Acked Read 5 Done Done Sent false
2940 1748 Acked Read 8 Done Done Sent false
2941 1749 Acked Read 5 Done Done Sent false
2942 1750 Acked Write 5 Done Done Sent false
2943 1751 Acked Write 10 Done Done Sent false
2944 1752 Acked Flush 0 Done Done Sent false
2945 1753 Acked Read 2 Done Done Sent false
2946 1754 Acked Read 6 Done Done Sent false
2947 1755 Acked Write 5 Done Done Sent false
2948 1756 Acked Write 10 Done Done Sent false
2949 1757 Acked Read 9 Done Done Sent false
2950 1758 Acked Write 10 Done Done Sent false
2951 1759 Acked Write 5 Done Done Sent false
2952 1760 Acked Write 5 Done Done Sent false
2953 1761 Acked Write 10 Done Done Sent false
2954 1762 Acked Read 10 Done Done Sent false
2955 1763 Acked Read 2 Done Done Sent false
2956 1764 Acked Write 9 Done Done Sent false
2957 1765 Acked Read 4 Done Done Sent false
2958 1766 Acked Flush 0 Done Done Sent false
2959 1767 Acked Read 4 Done Done Sent false
2960 1768 Acked Flush 0 Done Done Sent false
2961 1769 Acked Write 8 Done Done Sent false
2962 1770 Acked Write 7 Done Done Sent false
2963 1771 Acked Read 1 Done Done Sent false
2964 1772 Acked Write 4 Done Done Sent false
2965 1773 Acked Read 5 Done Done Sent false
2966 1774 Acked Write 2 Done Done Sent false
2967 1775 Acked Read 3 Done Done Sent false
2968 1776 Acked Read 5 Done Done Sent false
2969 1777 Acked Flush 0 Done Done Sent false
2970 1778 Acked Read 1 Done Done Sent false
2971 1779 Acked Flush 0 Done Done Sent false
2972 1780 Acked Read 6 Done Done Sent false
2973 1781 Acked Write 9 Done Done Sent false
2974 1782 Acked Read 4 Done Done Sent false
2975 1783 Acked Read 1 Done Done Sent false
2976 1784 Acked Flush 0 Done Done Sent false
2977 1785 Acked Read 5 Done Done Sent false
2978 1786 Acked Read 1 Done Done Sent false
2979 1787 Acked Flush 0 Done Done Sent false
2980 1788 Acked Write 9 Done Done Sent false
2981 1789 Acked Read 2 Done Done Sent false
2982 1790 Acked Read 10 Done Done Sent false
2983 1791 Acked Flush 0 Done Done Sent false
2984 1792 Acked Write 8 Done Done Sent false
2985 1793 Acked Read 7 Done Done Sent false
2986 1794 Acked Write 7 Done Done Sent false
2987 1795 Acked Read 6 Done Done Sent false
2988 1796 Acked Write 8 Done Done Sent false
2989 1797 Acked Read 8 Done Done Sent false
2990 1798 Acked Read 6 Done Done Sent false
2991 1799 Acked Read 8 Done Done Sent false
2992 1800 Acked Read 3 Done Done Sent false
2993 1801 Acked Read 6 Done Done Sent false
2994 1802 Acked Read 2 Done Done Sent false
2995 1803 Acked Read 8 Done Done Sent false
2996 1804 Acked Flush 0 Done Done Sent false
2997 1805 Acked Write 1 Done Done Sent false
2998 1806 Acked Read 3 Done Done Sent false
2999 1807 Acked Read 2 Done Done Sent false
3000 1808 Acked Write 1 Done Done Sent false
3001 1809 Acked Write 8 Done Done Sent false
3002 1810 Acked Flush 0 Done Done Sent false
3003 1811 Acked Write 4 Done Done Sent false
3004 1812 Acked Read 2 Done Done Sent false
3005 1813 Acked Read 10 Done Done Sent false
3006 1814 Acked Read 10 Done Done Sent false
3007 1815 Acked Write 4 Done Done Sent false
3008 1816 Acked Flush 0 Done Done Sent false
3009 1817 Acked Read 7 Done Done Sent false
3010 1818 Acked Flush 0 Done Done Sent false
3011 1819 Acked Read 1 Done Done Sent false
3012 1820 Acked Write 8 Done Done Sent false
3013 1821 Acked Read 3 Done Done Sent false
3014 1822 Acked Read 9 Done Done Sent false
3015 1823 Acked Write 6 Done Done Sent false
3016 1824 Acked Read 8 Done Done Sent false
3017 1825 Acked Write 1 Done Done Sent false
3018 1826 Acked Read 1 Done Done Sent false
3019 1827 Acked Write 10 Done Done Sent false
3020 1828 Acked Read 8 Done Done Sent false
3021 1829 Acked Read 6 Done Done Sent false
3022 1830 Acked Write 9 Done Done Sent false
3023 1831 Acked Write 8 Done Done Sent false
3024 1832 Acked Write 1 Done Done Sent false
3025 1833 Acked Read 4 Done Done Sent false
3026 1834 Acked Write 8 Done Done Sent false
3027 1835 Acked Read 5 Done Done Sent false
3028 1836 Acked Write 9 Done Done Sent false
3029 1837 Acked Read 8 Done Done Sent false
3030 1838 Acked Flush 0 Done Done Sent false
3031 1839 Acked Write 7 Done Done Sent false
3032 1840 Acked Read 2 Done Done Sent false
3033 1841 Acked Read 3 Done Done Sent false
3034 1842 Acked Read 7 Done Done Sent false
3035 1843 Acked Write 8 Done Done Sent false
3036 1844 Acked Write 5 Done Done Sent false
3037 1845 Acked Read 2 Done Done Sent false
3038 1846 Acked Write 1 Done Done Sent false
3039 1847 Acked Write 2 Done Done Sent false
3040 1848 Acked Write 4 Done Done Sent false
3041 1849 Acked Read 7 Done Done Sent false
3042 1850 Acked Write 6 Done Done Sent false
3043 1851 Acked Read 5 Done Done Sent false
3044 1852 Acked Write 7 Done Done Sent false
3045 1853 Acked Flush 0 Done Done Sent false
3046 1854 Acked Write 6 Done Done Sent false
3047 1855 Acked Flush 0 Done Done Sent false
3048 1856 Acked Read 2 Done Done Sent false
3049 1857 Acked Read 10 Done Done Sent false
3050 1858 Acked Read 3 Done Done Sent false
3051 1859 Acked Read 2 Done Done Sent false
3052 1860 Acked Write 4 Done Done Sent false
3053 1861 Acked Flush 0 Done Done Sent false
3054 1862 Acked Flush 0 Done Done Sent false
3055 1863 Acked Write 5 Done Done Sent false
3056 1864 Acked Write 8 Done Done Sent false
3057 1865 Acked Read 10 Done Done Sent false
3058 1866 Acked Write 5 Done Done Sent false
3059 1867 Acked Write 10 Done Done Sent false
3060 1868 Acked Write 4 Done Done Sent false
3061 1869 Acked Flush 0 Done Done Sent false
3062 1870 Acked Read 6 Done Done Sent false
3063 1871 Acked Read 7 Done Done Sent false
3064 1872 Acked Write 8 Done Done Sent false
3065 1873 Acked Write 2 Done Done Sent false
3066 1874 Acked Read 6 Done Done Sent false
3067 1875 Acked Write 5 Done Done Sent false
3068 1876 Acked Write 2 Done Done Sent false
3069 1877 Acked Read 9 Done Done Sent false
3070 1878 Acked Read 1 Done Done Sent false
3071 1879 Acked Read 10 Done Done Sent false
3072 1880 Acked Read 6 Done Done Sent false
3073 1881 Acked Read 10 Done Done Sent false
3074 1882 Acked Read 6 Done Done Sent false
3075 1883 Acked Flush 0 Done Done Sent false
3076 1884 Acked Read 10 Done Done Sent false
3077 1885 Acked Read 2 Done Done Sent false
3078 1886 Acked Write 7 Done Done Sent false
3079 1887 Acked Write 5 Done Done Sent false
3080 1888 Acked Read 7 Done Done Sent false
3081 1889 Acked Write 10 Done Done Sent false
3082 1890 Acked Write 4 Done Done Sent false
3083 1891 Acked Write 4 Done Done Sent false
3084 1892 Acked Read 7 Done Done Sent false
3085 1893 Acked Read 8 Done Done Sent false
3086 1894 Acked Read 2 Done Done Sent false
3087 1895 Acked Read 10 Done Done Sent false
3088 1896 Acked Read 7 Done Done Sent false
3089 1897 Acked Read 4 Done Done Sent false
3090 1898 Acked Read 5 Done Done Sent false
3091 1899 Acked Write 7 Done Done Sent false
3092 1900 Acked Read 3 Done Done Sent false
3093 1901 Acked Write 8 Done Done Sent false
3094 1902 Acked Read 8 Done Done Sent false
3095 1903 Acked Read 4 Done Done Sent false
3096 1904 Acked Write 1 Done Done Sent false
3097 1905 Acked Flush 0 Done Done Sent false
3098 STATES DS:0 DS:1 DS:2 TOTAL
3099 Sent 0 0 302 302
3100 Done 302 302 0 604
3101 Skipped 0 0 0 0
3102 Error 0 0 0 0
3103 Last Flush: 1905 1905 1603
3104 Downstairs last five completed: 1603 1602 1601 1600 1599
3105 Upstairs last five completed: 1905 1904 1903 1902 1901
3106 CLIENT: Up:0 ds:302 act:2
31072025-01-27T19:36:59.405ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31082025-01-27T19:36:59.405ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37391) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31092025-01-27T19:36:59.407ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31102025-01-27T19:36:59.407ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31112025-01-27T19:36:59.407ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31122025-01-27T19:36:59.407ZINFOcrucible: send last flush ID to this DS: 1603 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31132025-01-27T19:36:59.408ZINFOcrucible: Replied this last flush ID: 1603 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31142025-01-27T19:36:59.408ZINFOcrucible: [2] client re-new 302 jobs since flush 1603 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31152025-01-27T19:36:59.408ZINFOcrucible: [2] Replayed 302 jobs since flush: 1603 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
3116 ----------------------------------------------------------------
3117 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
3118 Downstairs last five completed: 1905 1904 1903 1902 1901
3119 Upstairs last five completed: 1905 1904 1903 1902 1901
3120 CLIENT: Up:0 ds:0 act:3
3121 Replay: All jobs finished
3122 Main task has work to do, go find it
3123 got dsc Stop(2)
3124 stop 2
3125 Jan 27 19:37:01.656 INFO request completed, latency_us: 359, [8830] Got stop action so:false kr:false
3126 response_code: 204, uri: /stop/cid/2, method: POST, req_id: 91c7e762-c6fb-4ffd-bbb3-6d6786754410, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
3127 Jan 27 19:37:01.656 INFO request completed, latency_us: 149, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2f09869b-b533-44b5-9d6f-42c4cc1b6370{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:37:01.656966138Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
3128 , remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
31292025-01-27T19:37:01.657ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31302025-01-27T19:37:01.657ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31312025-01-27T19:37:01.657ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
3132 [8830] Exited with: ExitStatus(unix_wait_status(9))
3133 [8830][2] reports Exit
3134 Jan 27 19:37:05.660 INFO request completed, latency_us: 330, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2b8880fd-dad6-45a8-ab94-b482b38256e7, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
3135 001/300 Write block 337 len 28672 data: 2 2 2 2 2 2 2
3136 002/300 Read block 3414 len 12288
3137 003/300 Write block 1361 len 28672 data: 2 2 2 2 2 2 2
3138 004/300 Write block 3250 len 20480 data: 2 2 2 2 2
3139 005/300 Write block 2978 len 24576 data: 2 2 2 2 2 2
3140 006/300 Read block 2115 len 24576
3141 007/300 Read block 3047 len 4096
3142 008/300 Read block 848 len 32768
3143 009/300 Read block 2382 len 20480
3144 010/300 Write block 4540 len 32768 data: 2 2 2 2 2 2 2 2
3145 011/300 Flush
3146 012/300 Read block 4060 len 28672
3147 013/300 Read block 3697 len 36864
3148 014/300 Read block 2949 len 4096
3149 015/300 Flush
3150 016/300 Flush
3151 017/300 Write block 1810 len 20480 data: 3 4 4 3 3
3152 018/300 Write block 2287 len 16384 data: 4 5 5 5
3153 019/300 Write block 1101 len 8192 data: 3 3
3154 020/300 Write block 2268 len 8192 data: 2 2
3155 021/300 Read block 3145 len 32768
3156 022/300 Read block 4615 len 12288
3157 023/300 Write block 4466 len 24576 data: 2 2 2 2 2 2
3158 024/300 Write block 222 len 32768 data: 2 2 2 2 2 2 2 2
3159 025/300 Read block 124 len 36864
3160 026/300 Read block 4528 len 20480
3161 027/300 Write block 3635 len 24576 data: 2 2 2 2 2 2
3162 028/300 Read block 2517 len 20480
3163 029/300 Read block 3033 len 40960
3164 030/300 Write block 2770 len 20480 data: 4 4 4 3 3
3165 031/300 Read block 3315 len 24576
3166 032/300 Read block 2594 len 8192
3167 033/300 Write block 3863 len 4096 data: 3
3168 034/300 Read block 4124 len 40960
3169 035/300 Write block 2393 len 4096 data: 2
3170 036/300 Write block 2997 len 28672 data: 2 2 2 3 3 3 3
3171 037/300 Read block 4733 len 4096
3172 038/300 Write block 3270 len 36864 data: 2 2 2 2 2 2 2 2 2
3173 039/300 Write block 1263 len 16384 data: 2 2 2 2
3174 040/300 Flush
3175 041/300 Write block 4633 len 28672 data: 2 2 2 2 2 2 2
3176 042/300 Read block 1479 len 16384
3177 043/300 Read block 4370 len 4096
3178 044/300 Read block 4600 len 28672
3179 045/300 Write block 2227 len 24576 data: 2 2 2 2 2 2
3180 046/300 Read block 1983 len 28672
3181 047/300 Write block 3130 len 16384 data: 3 3 3 3
3182 048/300 Read block 1762 len 24576
3183 049/300 Read block 343 len 16384
3184 050/300 Read block 1603 len 16384
3185 051/300 Write block 4248 len 12288 data: 4 4 5
3186 052/300 Write block 4383 len 40960 data: 3 2 2 2 2 2 2 2 3 3
3187 053/300 Read block 3561 len 12288
3188 054/300 Read block 3810 len 36864
3189 055/300 Flush
3190 056/300 Read block 777 len 24576
3191 057/300 Write block 1363 len 32768 data: 3 3 3 3 3 2 2 3
3192 058/300 Read block 3560 len 28672
3193 059/300 Read block 350 len 8192
3194 060/300 Read block 4442 len 12288
3195 061/300 Read block 1238 len 24576
3196 062/300 Read block 3203 len 32768
3197 063/300 Read block 1088 len 8192
3198 064/300 Read block 1477 len 32768
3199 065/300 Write block 371 len 8192 data: 2 3
3200 066/300 Read block 4755 len 20480
3201 067/300 Read block 313 len 40960
3202 068/300 Write block 28 len 12288 data: 2 2 2
3203 069/300 Read block 2792 len 32768
3204 070/300 Read block 4514 len 32768
3205 071/300 Read block 856 len 8192
3206 072/300 Write block 1297 len 4096 data: 2
3207 073/300 Read block 2482 len 24576
3208 074/300 Flush
3209 075/300 Write block 1412 len 32768 data: 3 3 3 3 3 4 4 3
3210 076/300 Write block 3235 len 16384 data: 4 2 2 2
3211 077/300 Write block 102 len 36864 data: 2 2 2 2 2 2 2 2 2
3212 078/300 Flush
3213 079/300 Write block 2933 len 4096 data: 2
3214 080/300 Write block 4116 len 8192 data: 2 2
3215 081/300 Read block 1300 len 16384
3216 082/300 Write block 4568 len 40960 data: 4 3 2 2 2 2 2 2 2 2
3217 083/300 Read block 971 len 28672
3218 084/300 Read block 2980 len 36864
3219 085/300 Read block 1460 len 32768
3220 086/300 Write block 2556 len 8192 data: 2 2
3221 087/300 Flush
3222 088/300 Write block 1543 len 40960 data: 6 6 5 2 2 2 2 2 3 2
3223 089/300 Write block 4912 len 20480 data: 2 2 2 2 2
3224 090/300 Read block 640 len 16384
3225 091/300 Write block 4468 len 28672 data: 3 3 3 3 2 2 2
3226 092/300 Read block 1686 len 8192
3227 093/300 Flush
3228 094/300 Read block 2016 len 32768
3229 095/300 Write block 669 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3230 096/300 Write block 120 len 20480 data: 3 3 4 4 4
3231 097/300 Write block 1358 len 16384 data: 2 2 2 3
3232 098/300 Write block 4100 len 4096 data: 3
3233 099/300 Read block 4705 len 16384
3234 100/300 Read block 3604 len 32768
3235 101/300 Write block 4963 len 36864 data: 2 2 2 2 2 2 2 3 3
3236 102/300 Write block 3335 len 12288 data: 2 2 2
3237 103/300 Read block 4768 len 8192
3238 104/300 Flush
3239 105/300 Write block 3046 len 36864 data: 3 2 2 2 2 2 2 2 2
3240 106/300 Read block 4095 len 36864
3241 107/300 Write block 866 len 36864 data: 3 3 2 2 2 3 3 3 3
3242 108/300 Read block 743 len 28672
3243 109/300 Write block 2563 len 40960 data: 3 3 3 3 2 2 2 2 2 2
3244 110/300 Write block 2851 len 8192 data: 3 3
3245 111/300 Read block 1966 len 4096
3246 112/300 Read block 1470 len 20480
3247 113/300 Read block 2570 len 32768
3248 114/300 Read block 3451 len 28672
3249 115/300 Read block 3954 len 32768
3250 116/300 Read block 3528 len 40960
3251 117/300 Read block 3968 len 24576
3252 118/300 Write block 1196 len 4096 data: 3
3253 119/300 Write block 3145 len 32768 data: 2 2 2 2 2 2 2 2
3254 120/300 Read block 3689 len 16384
3255 121/300 Flush
3256 122/300 Read block 2764 len 36864
3257 123/300 Write block 1674 len 24576 data: 3 3 3 2 2 2
3258 124/300 Read block 3884 len 12288
3259 125/300 Read block 525 len 4096
3260 126/300 Read block 3202 len 4096
3261 127/300 Read block 105 len 12288
3262 128/300 Read block 555 len 8192
3263 129/300 Read block 3544 len 40960
3264 130/300 Flush
3265 131/300 Write block 3752 len 20480 data: 2 2 2 2 2
3266 132/300 Write block 3727 len 20480 data: 3 3 3 2 2
3267 133/300 Flush
3268 134/300 Write block 3151 len 4096 data: 3
3269 135/300 Write block 397 len 40960 data: 3 2 2 2 3 3 3 3 3 3
3270 136/300 Read block 2531 len 24576
3271 137/300 Flush
3272 138/300 Read block 3390 len 36864
3273 139/300 Read block 12 len 36864
3274 140/300 Write block 1219 len 24576 data: 2 2 3 3 4 4
3275 141/300 Flush
3276 142/300 Flush
3277 143/300 Read block 322 len 36864
3278 144/300 Write block 3621 len 16384 data: 2 2 2 2
3279 145/300 Write block 4643 len 20480 data: 2 2 2 2 2
3280 146/300 Write block 4464 len 8192 data: 2 2
3281 147/300 Flush
3282 148/300 Write block 1047 len 8192 data: 3 2
3283 149/300 Read block 1271 len 16384
3284 150/300 Write block 2808 len 36864 data: 2 2 2 2 2 2 2 2 2
3285 151/300 Read block 4916 len 24576
3286 152/300 Read block 3213 len 8192
3287 153/300 Read block 1881 len 8192
3288 154/300 Write block 3153 len 20480 data: 2 2 2 2 2
3289 155/300 Write block 2302 len 12288 data: 2 2 2
3290 156/300 Write block 952 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3291 157/300 Flush
3292 158/300 Write block 3009 len 4096 data: 2
3293 159/300 Write block 648 len 32768 data: 3 3 2 2 2 3 2 2
3294 160/300 Read block 2504 len 32768
3295 161/300 Read block 4653 len 4096
3296 162/300 Write block 585 len 36864 data: 2 2 3 3 3 3 3 3 3
3297 163/300 Read block 368 len 24576
3298 164/300 Read block 468 len 28672
3299 165/300 Write block 2245 len 12288 data: 3 3 3
3300 166/300 Write block 148 len 12288 data: 2 2 2
3301 167/300 Write block 4144 len 8192 data: 3 3
3302 168/300 Read block 4969 len 36864
3303 169/300 Write block 3372 len 36864 data: 3 2 2 2 2 2 2 2 2
3304 170/300 Write block 1110 len 8192 data: 2 2
3305 171/300 Write block 604 len 12288 data: 5 5 3
3306 172/300 Read block 4002 len 32768
3307 173/300 Write block 1324 len 24576 data: 2 2 2 2 2 3
3308 174/300 Write block 2207 len 8192 data: 3 3
3309 175/300 Read block 2160 len 24576
3310 176/300 Write block 4735 len 32768 data: 3 3 2 2 2 2 2 2
3311 177/300 Write block 830 len 12288 data: 2 2 2
3312 178/300 Read block 107 len 24576
3313 179/300 Read block 2682 len 32768
3314 180/300 Read block 4892 len 40960
3315 181/300 Flush
3316 182/300 Read block 4112 len 4096
3317 183/300 Read block 101 len 40960
3318 184/300 Flush
3319 185/300 Flush
3320 186/300 Read block 3237 len 16384
3321 187/300 Read block 4788 len 32768
3322 188/300 Read block 1559 len 32768
3323 189/300 Read block 1465 len 24576
3324 190/300 Flush
3325 191/300 Read block 3577 len 28672
3326 192/300 Read block 3635 len 4096
3327 193/300 Write block 3135 len 40960 data: 3 4 3 3 3 3 3 3 3 2
3328 194/300 Read block 715 len 20480
3329 195/300 Read block 643 len 40960
3330 196/300 Read block 1252 len 40960
3331 197/300 Write block 3622 len 16384 data: 3 3 3 2
3332 198/300 Write block 1301 len 16384 data: 2 2 3 3
3333 199/300 Read block 978 len 28672
3334 200/300 Write block 275 len 28672 data: 3 3 3 3 3 4 4
3335 201/300 Read block 4538 len 8192
3336 202/300 Flush
3337 203/300 Read block 4391 len 4096
3338 204/300 Write block 4514 len 28672 data: 3 3 3 3 3 2 3
3339 205/300 Read block 1871 len 16384
3340 206/300 Read block 563 len 4096
3341 207/300 Read block 4430 len 8192
3342 208/300 Read block 4784 len 28672
3343 209/300 Write block 807 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3344 210/300 Flush
3345 211/300 Flush
3346 212/300 Read block 141 len 24576
3347 213/300 Read block 2503 len 32768
3348 214/300 Read block 1590 len 4096
3349 215/300 Write block 2371 len 32768 data: 2 2 2 2 2 2 2 3
3350 216/300 Read block 2801 len 28672
3351 217/300 Write block 1627 len 36864 data: 2 2 2 2 2 2 2 2 2
3352 218/300 Read block 1891 len 4096
3353 219/300 Read block 1508 len 12288
3354 220/300 Write block 295 len 36864 data: 2 2 2 2 2 3 3 3 2
3355 221/300 Read block 2557 len 36864
3356 222/300 Write block 3056 len 32768 data: 2 3 3 3 3 3 2 2
3357 223/300 Write block 3463 len 20480 data: 2 2 2 2 2
3358 224/300 Read block 2946 len 12288
3359 225/300 Read block 3853 len 8192
3360 226/300 Read block 1520 len 36864
3361 227/300 Write block 2011 len 4096 data: 2
3362 228/300 Read block 164 len 40960
3363 229/300 Flush
3364 230/300 Write block 738 len 32768 data: 3 3 3 3 3 3 2 2
3365 231/300 Read block 1716 len 36864
3366 232/300 Read block 3283 len 16384
3367 233/300 Write block 4118 len 24576 data: 4 4 4 4 4 3
3368 234/300 Flush
3369 235/300 Write block 2779 len 28672 data: 2 2 2 2 2 2 2
3370 236/300 Write block 4807 len 16384 data: 3 3 3 4
3371 237/300 Flush
3372 238/300 Write block 2353 len 8192 data: 2 2
3373 239/300 Write block 2837 len 40960 data: 3 3 3 3 3 3 3 3 3 3
3374 240/300 Flush
3375 241/300 Write block 4737 len 4096 data: 3
3376 242/300 Read block 3758 len 12288
3377 243/300 Read block 2976 len 16384
3378 244/300 Write block 2350 len 28672 data: 2 2 2 3 3 2 2
3379 245/300 Write block 1354 len 36864 data: 2 2 2 2 3 3 3 4 3
3380 246/300 Read block 1875 len 40960
3381 247/300 Read block 2501 len 32768
3382 248/300 Read block 292 len 8192
3383 249/300 Write block 37 len 32768 data: 2 2 2 2 2 2 3 3
3384 250/300 Flush
3385 251/300 Flush
3386 252/300 Read block 3691 len 32768
3387 253/300 Read block 3773 len 28672
3388 254/300 Read block 569 len 40960
3389 255/300 Write block 2400 len 24576 data: 2 2 3 3 2 2
3390 256/300 Read block 1350 len 24576
3391 257/300 Write block 868 len 4096 data: 3
3392 258/300 Write block 420 len 28672 data: 2 2 2 2 2 2 2
3393 259/300 Write block 564 len 4096 data: 3
3394 260/300 Read block 4196 len 8192
3395 261/300 Write block 4666 len 32768 data: 2 2 2 2 2 2 2 2
3396 262/300 Read block 4420 len 20480
3397 263/300 Read block 513 len 36864
3398 264/300 Write block 829 len 28672 data: 2 3 3 3 2 2 2
3399 265/300 Read block 377 len 8192
3400 266/300 Read block 1598 len 40960
3401 267/300 Write block 1 len 8192 data: 2 2
3402 268/300 Write block 713 len 16384 data: 6 6 5 4
3403 269/300 Flush
3404 270/300 Read block 3492 len 12288
3405 271/300 Read block 4225 len 4096
3406 272/300 Read block 2728 len 32768
3407 273/300 Flush
3408 274/300 Write block 166 len 24576 data: 2 2 2 2 2 2
3409 275/300 Write block 4190 len 12288 data: 4 4 4
3410 276/300 Read block 3079 len 20480
3411 277/300 Write block 3614 len 32768 data: 2 2 2 2 2 2 2 3
3412 278/300 Read block 907 len 32768
3413 279/300 Read block 4691 len 16384
3414 280/300 Read block 3091 len 20480
3415 281/300 Read block 3882 len 40960
3416 282/300 Write block 2463 len 4096 data: 2
3417 283/300 Write block 266 len 4096 data: 3
3418 284/300 Write block 2335 len 28672 data: 2 2 2 2 3 3 3
3419 285/300 Write block 364 len 24576 data: 2 2 2 2 2 2
3420 286/300 Write block 4646 len 16384 data: 3 3 2 2
3421 287/300 Read block 3172 len 40960
3422 288/300 Write block 3331 len 40960 data: 3 4 2 2 3 3 3 2 2 2
3423 289/300 Flush
3424 290/300 Write block 2801 len 8192 data: 3 3
3425 291/300 Flush
3426 292/300 Write block 1398 len 16384 data: 3 3 3 3
3427 293/300 Write block 1009 len 36864 data: 2 2 2 2 2 2 2 2 2
3428 294/300 Read block 590 len 36864
3429 295/300 Read block 245 len 12288
3430 296/300 Write block 959 len 24576 data: 3 3 3 2 2 2
3431 297/300 Read block 2560 len 12288
3432 298/300 Read block 4849 len 16384
3433 299/300 Write block 3688 len 40960 data: 2 2 2 2 2 2 2 2 2 2
3434 300/300 Write block 2483 len 12288 data: 2 2 2
3435 301/300 Read block 4236 len 24576
3436 Main task has work to do, go find it
3437 got dsc Start(2)
3438 start 2
3439 Jan 27 19:37:05.895 INFO request completed, latency_us: 176, response_code: 204, uri: /start/cid/2, method: POST, req_id: e21cf74d-f3d8-4596-8cc3-eecb39ef6728, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
3440 Replay: started 2, returned:Ok(())
3441 ----------------------------------------------------------------
3442 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
3443 States: Active Active Offline
3444 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3445 1906 Acked Write 7 Done Done Sent false
3446 1907 Acked Read 3 Done Done Sent false
3447 1908 Acked Write 7 Done Done Sent false
3448 1909 Acked Write 5 Done Done Sent false
3449 1910 Acked Write 6 Done Done Sent false
3450 1911 Acked Read 6 Done Done Sent false
3451 1912 Acked Read 1 Done Done Sent false
3452 1913 Acked Read 8 Done Done Sent false
3453 1914 Acked Read 5 Done Done Sent false
3454 1915 Acked Write 8 Done Done Sent false
3455 1916 Acked Flush 0 Done Done Sent false
3456 1917 Acked Read 7 Done Done Sent false
3457 1918 Acked Read 9 Done Done Sent false
3458 1919 Acked Read 1 Done Done Sent false
3459 1920 Acked Flush 0 Done Done Sent false
3460 1921 Acked Flush 0 Done Done Sent false
3461 1922 Acked Write 5 Done Done Sent false
3462 1923 Acked Write 4 Done Done Sent false
3463 1924 Acked Write 2 Done Done Sent false
3464 1925 Acked Write 2 Done Done Sent false
3465 1926 Acked Read 8 Done Done Sent false
3466 1927 Acked Read 3 Done Done Sent false
3467 1928 Acked Write 6 Done Done Sent false
3468 1929 Acked Write 8 Done Done Sent false
3469 1930 Acked Read 9 Done Done Sent false
3470 1931 Acked Read 5 Done Done Sent false
3471 1932 Acked Write 6 Done Done Sent false
3472 1933 Acked Read 5 Done Done Sent false
3473 1934 Acked Read 10 Done Done Sent false
3474 1935 Acked Write 5 Done Done Sent false
3475 1936 Acked Read 6 Done Done Sent false
3476 1937 Acked Read 2 Done Done Sent false
3477 1938 Acked Write 1 Done Done Sent false
3478 1939 Acked Read 10 Done Done Sent false
3479 1940 Acked Write 1 Done Done Sent false
3480 1941 Acked Write 7 Done Done Sent false
3481 1942 Acked Read 1 Done Done Sent false
3482 1943 Acked Write 9 Done Done Sent false
3483 1944 Acked Write 4 Done Done Sent false
3484 1945 Acked Flush 0 Done Done Sent false
3485 1946 Acked Write 7 Done Done Sent false
3486 1947 Acked Read 4 Done Done Sent false
3487 1948 Acked Read 1 Done Done Sent false
3488 1949 Acked Read 7 Done Done Sent false
3489 1950 Acked Write 6 Done Done Sent false
3490 1951 Acked Read 7 Done Done Sent false
3491 1952 Acked Write 4 Done Done Sent false
3492 1953 Acked Read 6 Done Done Sent false
3493 1954 Acked Read 4 Done Done Sent false
3494 1955 Acked Read 4 Done Done Sent false
3495 1956 Acked Write 3 Done Done Sent false
3496 1957 Acked Write 10 Done Done Sent false
3497 1958 Acked Read 3 Done Done Sent false
3498 1959 Acked Read 9 Done Done Sent false
3499 1960 Acked Flush 0 Done Done Sent false
3500 1961 Acked Read 6 Done Done Sent false
3501 1962 Acked Write 8 Done Done Sent false
3502 1963 Acked Read 7 Done Done Sent false
3503 1964 Acked Read 2 Done Done Sent false
3504 1965 Acked Read 3 Done Done Sent false
3505 1966 Acked Read 6 Done Done Sent false
3506 1967 Acked Read 8 Done Done Sent false
3507 1968 Acked Read 2 Done Done Sent false
3508 1969 Acked Read 8 Done Done Sent false
3509 1970 Acked Write 2 Done Done Sent false
3510 1971 Acked Read 5 Done Done Sent false
3511 1972 Acked Read 10 Done Done Sent false
3512 1973 Acked Write 3 Done Done Sent false
3513 1974 Acked Read 8 Done Done Sent false
3514 1975 Acked Read 8 Done Done Sent false
3515 1976 Acked Read 2 Done Done Sent false
3516 1977 Acked Write 1 Done Done Sent false
3517 1978 Acked Read 6 Done Done Sent false
3518 1979 Acked Flush 0 Done Done Sent false
3519 1980 Acked Write 8 Done Done Sent false
3520 1981 Acked Write 4 Done Done Sent false
3521 1982 Acked Write 9 Done Done Sent false
3522 1983 Acked Flush 0 Done Done Sent false
3523 1984 Acked Write 1 Done Done Sent false
3524 1985 Acked Write 2 Done Done Sent false
3525 1986 Acked Read 4 Done Done Sent false
3526 1987 Acked Write 10 Done Done Sent false
3527 1988 Acked Read 7 Done Done Sent false
3528 1989 Acked Read 9 Done Done Sent false
3529 1990 Acked Read 8 Done Done Sent false
3530 1991 Acked Write 2 Done Done Sent false
3531 1992 Acked Flush 0 Done Done Sent false
3532 1993 Acked Write 10 Done Done Sent false
3533 1994 Acked Write 5 Done Done Sent false
3534 1995 Acked Read 4 Done Done Sent false
3535 1996 Acked Write 7 Done Done Sent false
3536 1997 Acked Read 2 Done Done Sent false
3537 1998 Acked Flush 0 Done Done Sent false
3538 1999 Acked Read 8 Done Done Sent false
3539 2000 Acked Write 10 Done Done Sent false
3540 2001 Acked Write 5 Done Done Sent false
3541 2002 Acked Write 4 Done Done Sent false
3542 2003 Acked Write 1 Done Done Sent false
3543 2004 Acked Read 4 Done Done Sent false
3544 2005 Acked Read 8 Done Done Sent false
3545 2006 Acked Write 9 Done Done Sent false
3546 2007 Acked Write 3 Done Done Sent false
3547 2008 Acked Read 2 Done Done Sent false
3548 2009 Acked Flush 0 Done Done Sent false
3549 2010 Acked Write 9 Done Done Sent false
3550 2011 Acked Read 9 Done Done Sent false
3551 2012 Acked Write 9 Done Done Sent false
3552 2013 Acked Read 7 Done Done Sent false
3553 2014 Acked Write 10 Done Done Sent false
3554 2015 Acked Write 2 Done Done Sent false
3555 2016 Acked Read 1 Done Done Sent false
3556 2017 Acked Read 5 Done Done Sent false
3557 2018 Acked Read 8 Done Done Sent false
3558 2019 Acked Read 7 Done Done Sent false
3559 2020 Acked Read 8 Done Done Sent false
3560 2021 Acked Read 10 Done Done Sent false
3561 2022 Acked Read 6 Done Done Sent false
3562 2023 Acked Write 1 Done Done Sent false
3563 2024 Acked Write 8 Done Done Sent false
3564 2025 Acked Read 4 Done Done Sent false
3565 2026 Acked Flush 0 Done Done Sent false
3566 2027 Acked Read 9 Done Done Sent false
3567 2028 Acked Write 6 Done Done Sent false
3568 2029 Acked Read 3 Done Done Sent false
3569 2030 Acked Read 1 Done Done Sent false
3570 2031 Acked Read 1 Done Done Sent false
3571 2032 Acked Read 3 Done Done Sent false
3572 2033 Acked Read 2 Done Done Sent false
3573 2034 Acked Read 10 Done Done Sent false
3574 2035 Acked Flush 0 Done Done Sent false
3575 2036 Acked Write 5 Done Done Sent false
3576 2037 Acked Write 5 Done Done Sent false
3577 2038 Acked Flush 0 Done Done Sent false
3578 2039 Acked Write 1 Done Done Sent false
3579 2040 Acked Write 10 Done Done Sent false
3580 2041 Acked Read 6 Done Done Sent false
3581 2042 Acked Flush 0 Done Done Sent false
3582 2043 Acked Read 9 Done Done Sent false
3583 2044 Acked Read 9 Done Done Sent false
3584 2045 Acked Write 6 Done Done Sent false
3585 2046 Acked Flush 0 Done Done Sent false
3586 2047 Acked Flush 0 Done Done Sent false
3587 2048 Acked Read 9 Done Done Sent false
3588 2049 Acked Write 4 Done Done Sent false
3589 2050 Acked Write 5 Done Done Sent false
3590 2051 Acked Write 2 Done Done Sent false
3591 2052 Acked Flush 0 Done Done Sent false
3592 2053 Acked Write 2 Done Done Sent false
3593 2054 Acked Read 4 Done Done Sent false
3594 2055 Acked Write 9 Done Done Sent false
3595 2056 Acked Read 6 Done Done Sent false
3596 2057 Acked Read 2 Done Done Sent false
3597 2058 Acked Read 2 Done Done Sent false
3598 2059 Acked Write 5 Done Done Sent false
3599 2060 Acked Write 3 Done Done Sent false
3600 2061 Acked Write 10 Done Done Sent false
3601 2062 Acked Flush 0 Done Done Sent false
3602 2063 Acked Write 1 Done Done Sent false
3603 2064 Acked Write 8 Done Done Sent false
3604 2065 Acked Read 8 Done Done Sent false
3605 2066 Acked Read 1 Done Done Sent false
3606 2067 Acked Write 9 Done Done Sent false
3607 2068 Acked Read 6 Done Done Sent false
3608 2069 Acked Read 7 Done Done Sent false
3609 2070 Acked Write 3 Done Done Sent false
3610 2071 Acked Write 3 Done Done Sent false
3611 2072 Acked Write 2 Done Done Sent false
3612 2073 Acked Read 9 Done Done Sent false
3613 2074 Acked Write 9 Done Done Sent false
3614 2075 Acked Write 2 Done Done Sent false
3615 2076 Acked Write 3 Done Done Sent false
3616 2077 Acked Read 8 Done Done Sent false
3617 2078 Acked Write 6 Done Done Sent false
3618 2079 Acked Write 2 Done Done Sent false
3619 2080 Acked Read 6 Done Done Sent false
3620 2081 Acked Write 8 Done Done Sent false
3621 2082 Acked Write 3 Done Done Sent false
3622 2083 Acked Read 6 Done Done Sent false
3623 2084 Acked Read 8 Done Done Sent false
3624 2085 Acked Read 10 Done Done Sent false
3625 2086 Acked Flush 0 Done Done Sent false
3626 2087 Acked Read 1 Done Done Sent false
3627 2088 Acked Read 10 Done Done Sent false
3628 2089 Acked Flush 0 Done Done Sent false
3629 2090 Acked Flush 0 Done Done Sent false
3630 2091 Acked Read 4 Done Done Sent false
3631 2092 Acked Read 8 Done Done Sent false
3632 2093 Acked Read 8 Done Done Sent false
3633 2094 Acked Read 6 Done Done Sent false
3634 2095 Acked Flush 0 Done Done Sent false
3635 2096 Acked Read 7 Done Done Sent false
3636 2097 Acked Read 1 Done Done Sent false
3637 2098 Acked Write 10 Done Done Sent false
3638 2099 Acked Read 5 Done Done Sent false
3639 2100 Acked Read 10 Done Done Sent false
3640 2101 Acked Read 10 Done Done Sent false
3641 2102 Acked Write 4 Done Done Sent false
3642 2103 Acked Write 4 Done Done Sent false
3643 2104 Acked Read 7 Done Done Sent false
3644 2105 Acked Write 7 Done Done Sent false
3645 2106 Acked Read 2 Done Done Sent false
3646 2107 Acked Flush 0 Done Done Sent false
3647 2108 Acked Read 1 Done Done Sent false
3648 2109 Acked Write 7 Done Done Sent false
3649 2110 Acked Read 4 Done Done Sent false
3650 2111 Acked Read 1 Done Done Sent false
3651 2112 Acked Read 2 Done Done Sent false
3652 2113 Acked Read 7 Done Done Sent false
3653 2114 Acked Write 10 Done Done Sent false
3654 2115 Acked Flush 0 Done Done Sent false
3655 2116 Acked Flush 0 Done Done Sent false
3656 2117 Acked Read 6 Done Done Sent false
3657 2118 Acked Read 8 Done Done Sent false
3658 2119 Acked Read 1 Done Done Sent false
3659 2120 Acked Write 8 Done Done Sent false
3660 2121 Acked Read 7 Done Done Sent false
3661 2122 Acked Write 9 Done Done Sent false
3662 2123 Acked Read 1 Done Done Sent false
3663 2124 Acked Read 3 Done Done Sent false
3664 2125 Acked Write 9 Done Done Sent false
3665 2126 Acked Read 9 Done Done Sent false
3666 2127 Acked Write 8 Done Done Sent false
3667 2128 Acked Write 5 Done Done Sent false
3668 2129 Acked Read 3 Done Done Sent false
3669 2130 Acked Read 2 Done Done Sent false
3670 2131 Acked Read 9 Done Done Sent false
3671 2132 Acked Write 1 Done Done Sent false
3672 2133 Acked Read 10 Done Done Sent false
3673 2134 Acked Flush 0 Done Done Sent false
3674 2135 Acked Write 8 Done Done Sent false
3675 2136 Acked Read 9 Done Done Sent false
3676 2137 Acked Read 4 Done Done Sent false
3677 2138 Acked Write 6 Done Done Sent false
3678 2139 Acked Flush 0 Done Done Sent false
3679 2140 Acked Write 7 Done Done Sent false
3680 2141 Acked Write 4 Done Done Sent false
3681 2142 Acked Flush 0 Done Done Sent false
3682 2143 Acked Write 2 Done Done Sent false
3683 2144 Acked Write 10 Done Done Sent false
3684 2145 Acked Flush 0 Done Done Sent false
3685 2146 Acked Write 1 Done Done Sent false
3686 2147 Acked Read 3 Done Done Sent false
3687 2148 Acked Read 4 Done Done Sent false
3688 2149 Acked Write 7 Done Done Sent false
3689 2150 Acked Write 9 Done Done Sent false
3690 2151 Acked Read 10 Done Done Sent false
3691 2152 Acked Read 8 Done Done Sent false
3692 2153 Acked Read 2 Done Done Sent false
3693 2154 Acked Write 8 Done Done Sent false
3694 2155 Acked Flush 0 Done Done Sent false
3695 2156 Acked Flush 0 Done Done Sent false
3696 2157 Acked Read 8 Done Done Sent false
3697 2158 Acked Read 7 Done Done Sent false
3698 2159 Acked Read 10 Done Done Sent false
3699 2160 Acked Write 6 Done Done Sent false
3700 2161 Acked Read 6 Done Done Sent false
3701 2162 Acked Write 1 Done Done Sent false
3702 2163 Acked Write 7 Done Done Sent false
3703 2164 Acked Write 1 Done Done Sent false
3704 2165 Acked Read 2 Done Done Sent false
3705 2166 Acked Write 8 Done Done Sent false
3706 2167 Acked Read 5 Done Done Sent false
3707 2168 Acked Read 9 Done Done Sent false
3708 2169 Acked Write 7 Done Done Sent false
3709 2170 Acked Read 2 Done Done Sent false
3710 2171 Acked Read 10 Done Done Sent false
3711 2172 Acked Write 2 Done Done Sent false
3712 2173 Acked Write 4 Done Done Sent false
3713 2174 Acked Flush 0 Done Done Sent false
3714 2175 Acked Read 3 Done Done Sent false
3715 2176 Acked Read 1 Done Done Sent false
3716 2177 Acked Read 8 Done Done Sent false
3717 2178 Acked Flush 0 Done Done Sent false
3718 2179 Acked Write 6 Done Done Sent false
3719 2180 Acked Write 3 Done Done Sent false
3720 2181 Acked Read 5 Done Done Sent false
3721 2182 Acked Write 8 Done Done Sent false
3722 2183 Acked Read 8 Done Done Sent false
3723 2184 Acked Read 4 Done Done Sent false
3724 2185 Acked Read 5 Done Done Sent false
3725 2186 Acked Read 10 Done Done Sent false
3726 2187 Acked Write 1 Done Done Sent false
3727 2188 Acked Write 1 Done Done Sent false
3728 2189 Acked Write 7 Done Done Sent false
3729 2190 Acked Write 6 Done Done Sent false
3730 2191 Acked Write 4 Done Done Sent false
3731 2192 Acked Read 10 Done Done Sent false
3732 2193 Acked Write 10 Done Done Sent false
3733 2194 Acked Flush 0 Done Done Sent false
3734 2195 Acked Write 2 Done Done Sent false
3735 2196 Acked Flush 0 Done Done Sent false
3736 2197 Acked Write 4 Done Done Sent false
3737 2198 Acked Write 9 Done Done Sent false
3738 2199 Acked Read 9 Done Done Sent false
3739 2200 Acked Read 3 Done Done Sent false
3740 2201 Acked Write 6 Done Done Sent false
3741 2202 Acked Read 3 Done Done Sent false
3742 2203 Acked Read 4 Done Done Sent false
3743 2204 Acked Write 10 Done Done Sent false
3744 2205 Acked Write 3 Done Done Sent false
3745 2206 Acked Read 6 Done Done Sent false
3746 STATES DS:0 DS:1 DS:2 TOTAL
3747 Sent 0 0 301 301
3748 Done 301 301 0 602
3749 Skipped 0 0 0 0
3750 Error 0 0 0 0
3751 Last Flush: 2196 2196 1905
3752 Downstairs last five completed: 1905 1904 1903 1902 1901
3753 Upstairs last five completed: 2206 2205 2204 2203 2202
3754 CLIENT: Up:0 ds:301 act:2
3755 [8830] got start action
3756 [8830] I am going to restart
3757 Starting downstairs at port 8830
3758 Make output file at "/tmp/dsc/downstairs-8830.txt"
3759 [8830][2] reports Starting
3760 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1204)
3761 [8830][2] reports Running
3762 ----------------------------------------------------------------
3763 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
3764 States: Active Active Offline
3765 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
3766 1906 Acked Write 7 Done Done Sent false
3767 1907 Acked Read 3 Done Done Sent false
3768 1908 Acked Write 7 Done Done Sent false
3769 1909 Acked Write 5 Done Done Sent false
3770 1910 Acked Write 6 Done Done Sent false
3771 1911 Acked Read 6 Done Done Sent false
3772 1912 Acked Read 1 Done Done Sent false
3773 1913 Acked Read 8 Done Done Sent false
3774 1914 Acked Read 5 Done Done Sent false
3775 1915 Acked Write 8 Done Done Sent false
3776 1916 Acked Flush 0 Done Done Sent false
3777 1917 Acked Read 7 Done Done Sent false
3778 1918 Acked Read 9 Done Done Sent false
3779 1919 Acked Read 1 Done Done Sent false
3780 1920 Acked Flush 0 Done Done Sent false
3781 1921 Acked Flush 0 Done Done Sent false
3782 1922 Acked Write 5 Done Done Sent false
3783 1923 Acked Write 4 Done Done Sent false
3784 1924 Acked Write 2 Done Done Sent false
3785 1925 Acked Write 2 Done Done Sent false
3786 1926 Acked Read 8 Done Done Sent false
3787 1927 Acked Read 3 Done Done Sent false
3788 1928 Acked Write 6 Done Done Sent false
3789 1929 Acked Write 8 Done Done Sent false
3790 1930 Acked Read 9 Done Done Sent false
3791 1931 Acked Read 5 Done Done Sent false
3792 1932 Acked Write 6 Done Done Sent false
3793 1933 Acked Read 5 Done Done Sent false
3794 1934 Acked Read 10 Done Done Sent false
3795 1935 Acked Write 5 Done Done Sent false
3796 1936 Acked Read 6 Done Done Sent false
3797 1937 Acked Read 2 Done Done Sent false
3798 1938 Acked Write 1 Done Done Sent false
3799 1939 Acked Read 10 Done Done Sent false
3800 1940 Acked Write 1 Done Done Sent false
3801 1941 Acked Write 7 Done Done Sent false
3802 1942 Acked Read 1 Done Done Sent false
3803 1943 Acked Write 9 Done Done Sent false
3804 1944 Acked Write 4 Done Done Sent false
3805 1945 Acked Flush 0 Done Done Sent false
3806 1946 Acked Write 7 Done Done Sent false
3807 1947 Acked Read 4 Done Done Sent false
3808 1948 Acked Read 1 Done Done Sent false
3809 1949 Acked Read 7 Done Done Sent false
3810 1950 Acked Write 6 Done Done Sent false
3811 1951 Acked Read 7 Done Done Sent false
3812 1952 Acked Write 4 Done Done Sent false
3813 1953 Acked Read 6 Done Done Sent false
3814 1954 Acked Read 4 Done Done Sent false
3815 1955 Acked Read 4 Done Done Sent false
3816 1956 Acked Write 3 Done Done Sent false
3817 1957 Acked Write 10 Done Done Sent false
3818 1958 Acked Read 3 Done Done Sent false
3819 1959 Acked Read 9 Done Done Sent false
3820 1960 Acked Flush 0 Done Done Sent false
3821 1961 Acked Read 6 Done Done Sent false
3822 1962 Acked Write 8 Done Done Sent false
3823 1963 Acked Read 7 Done Done Sent false
3824 1964 Acked Read 2 Done Done Sent false
3825 1965 Acked Read 3 Done Done Sent false
3826 1966 Acked Read 6 Done Done Sent false
3827 1967 Acked Read 8 Done Done Sent false
3828 1968 Acked Read 2 Done Done Sent false
3829 1969 Acked Read 8 Done Done Sent false
3830 1970 Acked Write 2 Done Done Sent false
3831 1971 Acked Read 5 Done Done Sent false
3832 1972 Acked Read 10 Done Done Sent false
3833 1973 Acked Write 3 Done Done Sent false
3834 1974 Acked Read 8 Done Done Sent false
3835 1975 Acked Read 8 Done Done Sent false
3836 1976 Acked Read 2 Done Done Sent false
3837 1977 Acked Write 1 Done Done Sent false
3838 1978 Acked Read 6 Done Done Sent false
3839 1979 Acked Flush 0 Done Done Sent false
3840 1980 Acked Write 8 Done Done Sent false
3841 1981 Acked Write 4 Done Done Sent false
3842 1982 Acked Write 9 Done Done Sent false
3843 1983 Acked Flush 0 Done Done Sent false
3844 1984 Acked Write 1 Done Done Sent false
3845 1985 Acked Write 2 Done Done Sent false
3846 1986 Acked Read 4 Done Done Sent false
3847 1987 Acked Write 10 Done Done Sent false
3848 1988 Acked Read 7 Done Done Sent false
3849 1989 Acked Read 9 Done Done Sent false
3850 1990 Acked Read 8 Done Done Sent false
3851 1991 Acked Write 2 Done Done Sent false
3852 1992 Acked Flush 0 Done Done Sent false
3853 1993 Acked Write 10 Done Done Sent false
3854 1994 Acked Write 5 Done Done Sent false
3855 1995 Acked Read 4 Done Done Sent false
3856 1996 Acked Write 7 Done Done Sent false
3857 1997 Acked Read 2 Done Done Sent false
3858 1998 Acked Flush 0 Done Done Sent false
3859 1999 Acked Read 8 Done Done Sent false
3860 2000 Acked Write 10 Done Done Sent false
3861 2001 Acked Write 5 Done Done Sent false
3862 2002 Acked Write 4 Done Done Sent false
3863 2003 Acked Write 1 Done Done Sent false
3864 2004 Acked Read 4 Done Done Sent false
3865 2005 Acked Read 8 Done Done Sent false
3866 2006 Acked Write 9 Done Done Sent false
3867 2007 Acked Write 3 Done Done Sent false
3868 2008 Acked Read 2 Done Done Sent false
3869 2009 Acked Flush 0 Done Done Sent false
3870 2010 Acked Write 9 Done Done Sent false
3871 2011 Acked Read 9 Done Done Sent false
3872 2012 Acked Write 9 Done Done Sent false
3873 2013 Acked Read 7 Done Done Sent false
3874 2014 Acked Write 10 Done Done Sent false
3875 2015 Acked Write 2 Done Done Sent false
3876 2016 Acked Read 1 Done Done Sent false
3877 2017 Acked Read 5 Done Done Sent false
3878 2018 Acked Read 8 Done Done Sent false
3879 2019 Acked Read 7 Done Done Sent false
3880 2020 Acked Read 8 Done Done Sent false
3881 2021 Acked Read 10 Done Done Sent false
3882 2022 Acked Read 6 Done Done Sent false
3883 2023 Acked Write 1 Done Done Sent false
3884 2024 Acked Write 8 Done Done Sent false
3885 2025 Acked Read 4 Done Done Sent false
3886 2026 Acked Flush 0 Done Done Sent false
3887 2027 Acked Read 9 Done Done Sent false
3888 2028 Acked Write 6 Done Done Sent false
3889 2029 Acked Read 3 Done Done Sent false
3890 2030 Acked Read 1 Done Done Sent false
3891 2031 Acked Read 1 Done Done Sent false
3892 2032 Acked Read 3 Done Done Sent false
3893 2033 Acked Read 2 Done Done Sent false
3894 2034 Acked Read 10 Done Done Sent false
3895 2035 Acked Flush 0 Done Done Sent false
3896 2036 Acked Write 5 Done Done Sent false
3897 2037 Acked Write 5 Done Done Sent false
3898 2038 Acked Flush 0 Done Done Sent false
3899 2039 Acked Write 1 Done Done Sent false
3900 2040 Acked Write 10 Done Done Sent false
3901 2041 Acked Read 6 Done Done Sent false
3902 2042 Acked Flush 0 Done Done Sent false
3903 2043 Acked Read 9 Done Done Sent false
3904 2044 Acked Read 9 Done Done Sent false
3905 2045 Acked Write 6 Done Done Sent false
3906 2046 Acked Flush 0 Done Done Sent false
3907 2047 Acked Flush 0 Done Done Sent false
3908 2048 Acked Read 9 Done Done Sent false
3909 2049 Acked Write 4 Done Done Sent false
3910 2050 Acked Write 5 Done Done Sent false
3911 2051 Acked Write 2 Done Done Sent false
3912 2052 Acked Flush 0 Done Done Sent false
3913 2053 Acked Write 2 Done Done Sent false
3914 2054 Acked Read 4 Done Done Sent false
3915 2055 Acked Write 9 Done Done Sent false
3916 2056 Acked Read 6 Done Done Sent false
3917 2057 Acked Read 2 Done Done Sent false
3918 2058 Acked Read 2 Done Done Sent false
3919 2059 Acked Write 5 Done Done Sent false
3920 2060 Acked Write 3 Done Done Sent false
3921 2061 Acked Write 10 Done Done Sent false
3922 2062 Acked Flush 0 Done Done Sent false
3923 2063 Acked Write 1 Done Done Sent false
3924 2064 Acked Write 8 Done Done Sent false
3925 2065 Acked Read 8 Done Done Sent false
3926 2066 Acked Read 1 Done Done Sent false
3927 2067 Acked Write 9 Done Done Sent false
3928 2068 Acked Read 6 Done Done Sent false
3929 2069 Acked Read 7 Done Done Sent false
3930 2070 Acked Write 3 Done Done Sent false
3931 2071 Acked Write 3 Done Done Sent false
3932 2072 Acked Write 2 Done Done Sent false
3933 2073 Acked Read 9 Done Done Sent false
3934 2074 Acked Write 9 Done Done Sent false
3935 2075 Acked Write 2 Done Done Sent false
3936 2076 Acked Write 3 Done Done Sent false
3937 2077 Acked Read 8 Done Done Sent false
3938 2078 Acked Write 6 Done Done Sent false
3939 2079 Acked Write 2 Done Done Sent false
3940 2080 Acked Read 6 Done Done Sent false
3941 2081 Acked Write 8 Done Done Sent false
3942 2082 Acked Write 3 Done Done Sent false
3943 2083 Acked Read 6 Done Done Sent false
3944 2084 Acked Read 8 Done Done Sent false
3945 2085 Acked Read 10 Done Done Sent false
3946 2086 Acked Flush 0 Done Done Sent false
3947 2087 Acked Read 1 Done Done Sent false
3948 2088 Acked Read 10 Done Done Sent false
3949 2089 Acked Flush 0 Done Done Sent false
3950 2090 Acked Flush 0 Done Done Sent false
3951 2091 Acked Read 4 Done Done Sent false
3952 2092 Acked Read 8 Done Done Sent false
3953 2093 Acked Read 8 Done Done Sent false
3954 2094 Acked Read 6 Done Done Sent false
3955 2095 Acked Flush 0 Done Done Sent false
3956 2096 Acked Read 7 Done Done Sent false
3957 2097 Acked Read 1 Done Done Sent false
3958 2098 Acked Write 10 Done Done Sent false
3959 2099 Acked Read 5 Done Done Sent false
3960 2100 Acked Read 10 Done Done Sent false
3961 2101 Acked Read 10 Done Done Sent false
3962 2102 Acked Write 4 Done Done Sent false
3963 2103 Acked Write 4 Done Done Sent false
3964 2104 Acked Read 7 Done Done Sent false
3965 2105 Acked Write 7 Done Done Sent false
3966 2106 Acked Read 2 Done Done Sent false
3967 2107 Acked Flush 0 Done Done Sent false
3968 2108 Acked Read 1 Done Done Sent false
3969 2109 Acked Write 7 Done Done Sent false
3970 2110 Acked Read 4 Done Done Sent false
3971 2111 Acked Read 1 Done Done Sent false
3972 2112 Acked Read 2 Done Done Sent false
3973 2113 Acked Read 7 Done Done Sent false
3974 2114 Acked Write 10 Done Done Sent false
3975 2115 Acked Flush 0 Done Done Sent false
3976 2116 Acked Flush 0 Done Done Sent false
3977 2117 Acked Read 6 Done Done Sent false
3978 2118 Acked Read 8 Done Done Sent false
3979 2119 Acked Read 1 Done Done Sent false
3980 2120 Acked Write 8 Done Done Sent false
3981 2121 Acked Read 7 Done Done Sent false
3982 2122 Acked Write 9 Done Done Sent false
3983 2123 Acked Read 1 Done Done Sent false
3984 2124 Acked Read 3 Done Done Sent false
3985 2125 Acked Write 9 Done Done Sent false
3986 2126 Acked Read 9 Done Done Sent false
3987 2127 Acked Write 8 Done Done Sent false
3988 2128 Acked Write 5 Done Done Sent false
3989 2129 Acked Read 3 Done Done Sent false
3990 2130 Acked Read 2 Done Done Sent false
3991 2131 Acked Read 9 Done Done Sent false
3992 2132 Acked Write 1 Done Done Sent false
3993 2133 Acked Read 10 Done Done Sent false
3994 2134 Acked Flush 0 Done Done Sent false
3995 2135 Acked Write 8 Done Done Sent false
3996 2136 Acked Read 9 Done Done Sent false
3997 2137 Acked Read 4 Done Done Sent false
3998 2138 Acked Write 6 Done Done Sent false
3999 2139 Acked Flush 0 Done Done Sent false
4000 2140 Acked Write 7 Done Done Sent false
4001 2141 Acked Write 4 Done Done Sent false
4002 2142 Acked Flush 0 Done Done Sent false
4003 2143 Acked Write 2 Done Done Sent false
4004 2144 Acked Write 10 Done Done Sent false
4005 2145 Acked Flush 0 Done Done Sent false
4006 2146 Acked Write 1 Done Done Sent false
4007 2147 Acked Read 3 Done Done Sent false
4008 2148 Acked Read 4 Done Done Sent false
4009 2149 Acked Write 7 Done Done Sent false
4010 2150 Acked Write 9 Done Done Sent false
4011 2151 Acked Read 10 Done Done Sent false
4012 2152 Acked Read 8 Done Done Sent false
4013 2153 Acked Read 2 Done Done Sent false
4014 2154 Acked Write 8 Done Done Sent false
4015 2155 Acked Flush 0 Done Done Sent false
4016 2156 Acked Flush 0 Done Done Sent false
4017 2157 Acked Read 8 Done Done Sent false
4018 2158 Acked Read 7 Done Done Sent false
4019 2159 Acked Read 10 Done Done Sent false
4020 2160 Acked Write 6 Done Done Sent false
4021 2161 Acked Read 6 Done Done Sent false
4022 2162 Acked Write 1 Done Done Sent false
4023 2163 Acked Write 7 Done Done Sent false
4024 2164 Acked Write 1 Done Done Sent false
4025 2165 Acked Read 2 Done Done Sent false
4026 2166 Acked Write 8 Done Done Sent false
4027 2167 Acked Read 5 Done Done Sent false
4028 2168 Acked Read 9 Done Done Sent false
4029 2169 Acked Write 7 Done Done Sent false
4030 2170 Acked Read 2 Done Done Sent false
4031 2171 Acked Read 10 Done Done Sent false
4032 2172 Acked Write 2 Done Done Sent false
4033 2173 Acked Write 4 Done Done Sent false
4034 2174 Acked Flush 0 Done Done Sent false
4035 2175 Acked Read 3 Done Done Sent false
4036 2176 Acked Read 1 Done Done Sent false
4037 2177 Acked Read 8 Done Done Sent false
4038 2178 Acked Flush 0 Done Done Sent false
4039 2179 Acked Write 6 Done Done Sent false
4040 2180 Acked Write 3 Done Done Sent false
4041 2181 Acked Read 5 Done Done Sent false
4042 2182 Acked Write 8 Done Done Sent false
4043 2183 Acked Read 8 Done Done Sent false
4044 2184 Acked Read 4 Done Done Sent false
4045 2185 Acked Read 5 Done Done Sent false
4046 2186 Acked Read 10 Done Done Sent false
4047 2187 Acked Write 1 Done Done Sent false
4048 2188 Acked Write 1 Done Done Sent false
4049 2189 Acked Write 7 Done Done Sent false
4050 2190 Acked Write 6 Done Done Sent false
4051 2191 Acked Write 4 Done Done Sent false
4052 2192 Acked Read 10 Done Done Sent false
4053 2193 Acked Write 10 Done Done Sent false
4054 2194 Acked Flush 0 Done Done Sent false
4055 2195 Acked Write 2 Done Done Sent false
4056 2196 Acked Flush 0 Done Done Sent false
4057 2197 Acked Write 4 Done Done Sent false
4058 2198 Acked Write 9 Done Done Sent false
4059 2199 Acked Read 9 Done Done Sent false
4060 2200 Acked Read 3 Done Done Sent false
4061 2201 Acked Write 6 Done Done Sent false
4062 2202 Acked Read 3 Done Done Sent false
4063 2203 Acked Read 4 Done Done Sent false
4064 2204 Acked Write 10 Done Done Sent false
4065 2205 Acked Write 3 Done Done Sent false
4066 2206 Acked Read 6 Done Done Sent false
4067 2207 Acked Flush 0 Done Done Sent false
4068 STATES DS:0 DS:1 DS:2 TOTAL
4069 Sent 0 0 302 302
4070 Done 302 302 0 604
4071 Skipped 0 0 0 0
4072 Error 0 0 0 0
4073 Last Flush: 2207 2207 1905
4074 Downstairs last five completed: 1905 1904 1903 1902 1901
4075 Upstairs last five completed: 2207 2206 2205 2204 2203
4076 CLIENT: Up:0 ds:302 act:2
40772025-01-27T19:37:11.658ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
40782025-01-27T19:37:11.658ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:49560) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
40792025-01-27T19:37:11.659ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
40802025-01-27T19:37:11.660ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
40812025-01-27T19:37:11.660ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
40822025-01-27T19:37:11.660ZINFOcrucible: send last flush ID to this DS: 1905 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
40832025-01-27T19:37:11.660ZINFOcrucible: Replied this last flush ID: 1905 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
40842025-01-27T19:37:11.660ZINFOcrucible: [2] client re-new 302 jobs since flush 1905 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
40852025-01-27T19:37:11.660ZINFOcrucible: [2] Replayed 302 jobs since flush: 1905 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
4086 ----------------------------------------------------------------
4087 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
4088 Downstairs last five completed: 2207 2206 2205 2204 2203
4089 Upstairs last five completed: 2207 2206 2205 2204 2203
4090 CLIENT: Up:0 ds:0 act:3
4091 Replay: All jobs finished
4092 Main task has work to do, go find it
4093 got dsc Stop(0)
4094 stop 0
4095 Jan 27 19:37:13.899 INFO [8810] Got stop action so:false kr:false
4096 request completed, latency_us: 367, response_code: 204, uri: /stop/cid/0, method: POST, req_id: e3202517-f3d4-4353-9906-0b6cd855b192, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
40972025-01-27T19:37:13.900ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
4098 Jan 27 19:37:13.900 INFO request completed, latency_us: 141, response_code: 200, uri: /state/cid/0, method: GET, req_id: d1fa1e2c-b3c6-48c4-9136-52912ddca85d, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
40992025-01-27T19:37:13.900ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
41002025-01-27T19:37:13.900ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
41012025-01-27T19:37:13.900ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
4102 [8810] Exited with: ExitStatus(unix_wait_status(9))
4103 [8810][0] reports Exit
4104 Jan 27 19:37:17.903 INFO request completed, latency_us: 364, response_code: 200, uri: /state/cid/0, method: GET, req_id: f35e5b79-5f1b-4228-a79d-dfb24713d66c, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
4105 001/300 Read block 2972 len 16384
4106 002/300 Flush
4107 003/300 Write block 194 len 28672 data: 4 4 3 3 3 3 3
4108 004/300 Write block 4190 len 32768 data: 5 5 5 4 4 4 5 4
4109 005/300 Write block 568 len 24576 data: 2 2 2 2 2 2
4110 006/300 Read block 673 len 16384
4111 007/300 Write block 2977 len 20480 data: 2 3 3 3 3
4112 008/300 Write block 3926 len 32768 data: 2 2 2 2 2 2 2 2
4113 009/300 Read block 4221 len 28672
4114 010/300 Write block 76 len 32768 data: 2 2 2 2 2 2 2 2
4115 011/300 Read block 2573 len 20480
4116 012/300 Read block 4906 len 36864
4117 013/300 Write block 4922 len 8192 data: 3 3
4118 014/300 Read block 3179 len 20480
4119 015/300 Write block 2663 len 16384 data: 3 3 3 2
4120 016/300 Read block 2032 len 36864
4121 017/300 Read block 1777 len 36864
4122 018/300 Write block 3720 len 4096 data: 2
4123 019/300 Read block 1982 len 4096
4124 020/300 Read block 4401 len 12288
4125 021/300 Read block 3264 len 36864
4126 022/300 Read block 3882 len 36864
4127 023/300 Read block 296 len 8192
4128 024/300 Write block 1028 len 36864 data: 2 4 4 3 3 3 3 3 3
4129 025/300 Read block 2078 len 32768
4130 026/300 Write block 3735 len 12288 data: 2 2 2
4131 027/300 Write block 4405 len 16384 data: 2 2 2 2
4132 028/300 Read block 3473 len 16384
4133 029/300 Flush
4134 030/300 Read block 1388 len 8192
4135 031/300 Read block 2933 len 28672
4136 032/300 Read block 1808 len 32768
4137 033/300 Read block 4707 len 8192
4138 034/300 Read block 1414 len 20480
4139 035/300 Write block 2359 len 8192 data: 2 2
4140 036/300 Write block 3931 len 28672 data: 3 3 3 2 2 3 3
4141 037/300 Write block 910 len 20480 data: 2 2 2 2 2
4142 038/300 Flush
4143 039/300 Write block 88 len 4096 data: 2
4144 040/300 Read block 1435 len 12288
4145 041/300 Read block 2694 len 16384
4146 042/300 Read block 2811 len 36864
4147 043/300 Read block 3500 len 20480
4148 044/300 Read block 1550 len 4096
4149 045/300 Read block 4114 len 36864
4150 046/300 Write block 3178 len 28672 data: 4 4 3 3 3 2 2
4151 047/300 Read block 439 len 36864
4152 048/300 Write block 4243 len 28672 data: 4 3 4 5 5 5 5
4153 049/300 Write block 502 len 20480 data: 2 2 2 3 3
4154 050/300 Read block 853 len 20480
4155 051/300 Write block 941 len 28672 data: 2 2 2 2 2 2 2
4156 052/300 Read block 4729 len 32768
4157 053/300 Write block 4284 len 12288 data: 2 2 2
4158 054/300 Read block 714 len 8192
4159 055/300 Read block 4035 len 20480
4160 056/300 Read block 16 len 28672
4161 057/300 Read block 135 len 16384
4162 058/300 Read block 4442 len 8192
4163 059/300 Write block 2364 len 32768 data: 2 2 2 2 2 2 2 3
4164 060/300 Write block 3658 len 28672 data: 2 2 2 2 2 2 2
4165 061/300 Read block 2634 len 12288
4166 062/300 Flush
4167 063/300 Write block 346 len 12288 data: 2 2 2
4168 064/300 Write block 2391 len 16384 data: 3 2 3 2
4169 065/300 Read block 4994 len 12288
4170 066/300 Write block 3292 len 20480 data: 2 3 3 3 3
4171 067/300 Write block 4042 len 32768 data: 2 2 2 2 2 2 2 2
4172 068/300 Write block 206 len 4096 data: 2
4173 069/300 Read block 3541 len 20480
4174 070/300 Read block 2185 len 40960
4175 071/300 Write block 2222 len 24576 data: 2 2 2 2 2 3
4176 072/300 Read block 3305 len 32768
4177 073/300 Write block 96 len 36864 data: 2 2 2 2 2 2 3 3 3
4178 074/300 Write block 4336 len 24576 data: 4 4 4 3 2 2
4179 075/300 Write block 1509 len 4096 data: 2
4180 076/300 Flush
4181 077/300 Read block 2981 len 24576
4182 078/300 Write block 1425 len 16384 data: 2 2 2 2
4183 079/300 Write block 4531 len 24576 data: 2 2 2 2 2 2
4184 080/300 Read block 2661 len 12288
4185 081/300 Read block 889 len 16384
4186 082/300 Read block 3016 len 12288
4187 083/300 Flush
4188 084/300 Read block 1016 len 12288
4189 085/300 Read block 1913 len 24576
4190 086/300 Write block 4193 len 40960 data: 5 5 5 6 5 4 3 2 2 2
4191 087/300 Read block 4997 len 8192
4192 088/300 Read block 2067 len 12288
4193 089/300 Read block 1548 len 24576
4194 090/300 Read block 4766 len 24576
4195 091/300 Write block 1428 len 8192 data: 3 2
4196 092/300 Write block 451 len 32768 data: 2 2 2 2 2 2 2 2
4197 093/300 Write block 3880 len 36864 data: 2 2 2 2 2 2 2 2 2
4198 094/300 Read block 1712 len 40960
4199 095/300 Read block 1830 len 24576
4200 096/300 Read block 4531 len 20480
4201 097/300 Read block 2577 len 36864
4202 098/300 Read block 1643 len 28672
4203 099/300 Read block 928 len 36864
4204 100/300 Flush
4205 101/300 Write block 1076 len 20480 data: 2 2 2 2 2
4206 102/300 Flush
4207 103/300 Read block 3328 len 8192
4208 104/300 Read block 1445 len 4096
4209 105/300 Read block 773 len 12288
4210 106/300 Write block 3094 len 8192 data: 2 2
4211 107/300 Read block 251 len 12288
4212 108/300 Read block 1381 len 8192
4213 109/300 Read block 3008 len 24576
4214 110/300 Flush
4215 111/300 Write block 1995 len 20480 data: 2 2 2 2 2
4216 112/300 Write block 1031 len 16384 data: 4 4 4 4
4217 113/300 Write block 492 len 40960 data: 2 2 2 2 2 2 3 3 2 2
4218 114/300 Write block 4103 len 4096 data: 3
4219 115/300 Read block 4867 len 8192
4220 116/300 Write block 2162 len 32768 data: 3 3 3 3 3 2 2 2
4221 117/300 Write block 2979 len 36864 data: 4 4 4 3 3 2 2 2 2
4222 118/300 Flush
4223 119/300 Read block 3419 len 4096
4224 120/300 Read block 3402 len 4096
4225 121/300 Write block 1845 len 32768 data: 2 2 2 3 3 4 4 4
4226 122/300 Read block 1170 len 8192
4227 123/300 Write block 76 len 16384 data: 3 3 3 3
4228 124/300 Read block 231 len 40960
4229 125/300 Read block 1900 len 32768
4230 126/300 Read block 1781 len 36864
4231 127/300 Read block 1322 len 24576
4232 128/300 Write block 2557 len 40960 data: 3 2 2 3 3 3 4 4 4 4
4233 129/300 Write block 3774 len 36864 data: 4 4 4 4 4 3 2 2 2
4234 130/300 Flush
4235 131/300 Read block 1687 len 16384
4236 132/300 Read block 3053 len 40960
4237 133/300 Write block 4594 len 40960 data: 2 2 2 2 2 2 2 3 3 3
4238 134/300 Write block 4708 len 12288 data: 3 4 4
4239 135/300 Write block 2598 len 12288 data: 2 2 2
4240 136/300 Write block 4077 len 8192 data: 2 2
4241 137/300 Write block 1019 len 12288 data: 4 4 3
4242 138/300 Flush
4243 139/300 Read block 786 len 16384
4244 140/300 Read block 764 len 28672
4245 141/300 Read block 3603 len 12288
4246 142/300 Flush
4247 143/300 Write block 742 len 32768 data: 4 4 3 3 2 2 2 2
4248 144/300 Read block 1802 len 20480
4249 145/300 Read block 4158 len 12288
4250 146/300 Write block 2274 len 8192 data: 2 2
4251 147/300 Read block 4741 len 24576
4252 148/300 Read block 3899 len 28672
4253 149/300 Read block 2096 len 8192
4254 150/300 Read block 3465 len 4096
4255 151/300 Read block 4937 len 24576
4256 152/300 Write block 3091 len 20480 data: 2 2 2 3 3
4257 153/300 Write block 3919 len 16384 data: 3 2 3 3
4258 154/300 Read block 4487 len 28672
4259 155/300 Read block 4679 len 16384
4260 156/300 Write block 4355 len 20480 data: 2 2 2 2 2
4261 157/300 Write block 4733 len 36864 data: 4 3 4 4 4 3 3 3 3
4262 158/300 Read block 4335 len 16384
4263 159/300 Read block 3797 len 4096
4264 160/300 Read block 3265 len 20480
4265 161/300 Read block 3394 len 4096
4266 162/300 Read block 1987 len 40960
4267 163/300 Read block 813 len 16384
4268 164/300 Read block 1475 len 20480
4269 165/300 Read block 3492 len 12288
4270 166/300 Flush
4271 167/300 Read block 4321 len 32768
4272 168/300 Read block 788 len 32768
4273 169/300 Read block 4104 len 4096
4274 170/300 Read block 459 len 24576
4275 171/300 Read block 3992 len 28672
4276 172/300 Write block 1478 len 20480 data: 2 2 3 3 3
4277 173/300 Flush
4278 174/300 Read block 4246 len 28672
4279 175/300 Flush
4280 176/300 Read block 1325 len 16384
4281 177/300 Read block 3465 len 4096
4282 178/300 Write block 1420 len 40960 data: 3 3 3 2 2 3 3 3 4 3
4283 179/300 Write block 3397 len 8192 data: 3 3
4284 180/300 Flush
4285 181/300 Flush
4286 182/300 Read block 2932 len 12288
4287 183/300 Flush
4288 184/300 Read block 3129 len 20480
4289 185/300 Read block 1752 len 40960
4290 186/300 Write block 552 len 16384 data: 2 2 2 2
4291 187/300 Write block 4411 len 12288 data: 2 2 3
4292 188/300 Read block 1224 len 4096
4293 189/300 Read block 3635 len 24576
4294 190/300 Read block 2885 len 36864
4295 191/300 Write block 1098 len 20480 data: 3 3 3 4 4
4296 192/300 Read block 3438 len 32768
4297 193/300 Read block 1443 len 4096
4298 194/300 Read block 1378 len 4096
4299 195/300 Read block 264 len 28672
4300 196/300 Read block 1840 len 24576
4301 197/300 Read block 4744 len 16384
4302 198/300 Read block 3218 len 20480
4303 199/300 Write block 2286 len 12288 data: 4 5 6
4304 200/300 Write block 312 len 28672 data: 2 2 2 2 2 3 3
4305 201/300 Read block 4718 len 24576
4306 202/300 Write block 2026 len 28672 data: 3 3 3 3 3 3 3
4307 203/300 Read block 4351 len 40960
4308 204/300 Read block 4798 len 12288
4309 205/300 Read block 3528 len 20480
4310 206/300 Write block 3388 len 16384 data: 3 3 3 3
4311 207/300 Write block 2830 len 20480 data: 2 2 2 2 2
4312 208/300 Write block 2448 len 40960 data: 2 2 2 2 2 2 2 2 2 2
4313 209/300 Write block 3520 len 28672 data: 2 2 2 2 2 2 2
4314 210/300 Read block 347 len 8192
4315 211/300 Read block 2563 len 8192
4316 212/300 Write block 4437 len 24576 data: 2 2 3 3 3 3
4317 213/300 Read block 919 len 12288
4318 214/300 Read block 4208 len 24576
4319 215/300 Write block 1314 len 32768 data: 2 2 2 2 2 2 2 2
4320 216/300 Read block 2358 len 40960
4321 217/300 Write block 3747 len 16384 data: 2 2 2 2
4322 218/300 Write block 623 len 12288 data: 2 2 2
4323 219/300 Read block 3029 len 16384
4324 220/300 Write block 418 len 8192 data: 2 2
4325 221/300 Write block 3271 len 16384 data: 3 3 3 3
4326 222/300 Write block 2253 len 12288 data: 2 2 3
4327 223/300 Read block 528 len 40960
4328 224/300 Read block 1733 len 36864
4329 225/300 Write block 2969 len 32768 data: 2 2 2 2 2 2 2 2
4330 226/300 Write block 2205 len 4096 data: 3
4331 227/300 Read block 2403 len 24576
4332 228/300 Write block 3826 len 36864 data: 3 3 3 3 3 2 2 2 2
4333 229/300 Write block 3150 len 12288 data: 3 4 3
4334 230/300 Write block 1164 len 36864 data: 2 2 2 2 2 2 2 2 2
4335 231/300 Write block 2709 len 8192 data: 2 2
4336 232/300 Write block 3023 len 36864 data: 3 3 3 2 2 2 2 2 2
4337 233/300 Write block 1930 len 4096 data: 2
4338 234/300 Write block 4967 len 24576 data: 3 3 3 4 4 2
4339 235/300 Read block 4365 len 16384
4340 236/300 Flush
4341 237/300 Write block 2718 len 36864 data: 3 3 3 3 3 2 2 2 2
4342 238/300 Read block 3912 len 40960
4343 239/300 Write block 2124 len 8192 data: 2 2
4344 240/300 Read block 3055 len 32768
4345 241/300 Write block 3576 len 4096 data: 2
4346 242/300 Read block 1950 len 4096
4347 243/300 Write block 4034 len 16384 data: 3 3 3 3
4348 244/300 Read block 316 len 36864
4349 245/300 Read block 3476 len 40960
4350 246/300 Write block 288 len 28672 data: 2 2 2 2 2 2 2
4351 247/300 Read block 4746 len 16384
4352 248/300 Flush
4353 249/300 Read block 298 len 36864
4354 250/300 Write block 255 len 16384 data: 2 2 2 2
4355 251/300 Read block 3557 len 32768
4356 252/300 Write block 4829 len 20480 data: 2 2 2 2 2
4357 253/300 Flush
4358 254/300 Read block 4851 len 36864
4359 255/300 Flush
4360 256/300 Write block 2255 len 8192 data: 4 3
4361 257/300 Write block 2164 len 28672 data: 4 4 4 3 3 3 2
4362 258/300 Write block 225 len 12288 data: 3 3 3
4363 259/300 Read block 521 len 28672
4364 260/300 Flush
4365 261/300 Flush
4366 262/300 Write block 3845 len 20480 data: 2 2 2 2 2
4367 263/300 Write block 4902 len 4096 data: 2
4368 264/300 Write block 4863 len 8192 data: 2 2
4369 265/300 Read block 3102 len 28672
4370 266/300 Read block 582 len 28672
4371 267/300 Write block 280 len 40960 data: 5 5 4 3 3 3 3 2 3 3
4372 268/300 Read block 1045 len 32768
4373 269/300 Write block 4882 len 4096 data: 3
4374 270/300 Flush
4375 271/300 Write block 789 len 36864 data: 2 2 2 2 2 2 2 2 2
4376 272/300 Read block 1615 len 28672
4377 273/300 Read block 4497 len 32768
4378 274/300 Write block 4907 len 8192 data: 2 3
4379 275/300 Read block 2427 len 36864
4380 276/300 Write block 1464 len 36864 data: 2 2 2 2 2 2 3 3 3
4381 277/300 Read block 293 len 24576
4382 278/300 Read block 414 len 24576
4383 279/300 Write block 4957 len 32768 data: 2 3 3 3 3 3 3 3
4384 280/300 Write block 2751 len 24576 data: 2 2 2 2 2 2
4385 281/300 Read block 2452 len 8192
4386 282/300 Flush
4387 283/300 Read block 3547 len 16384
4388 284/300 Write block 4830 len 4096 data: 3
4389 285/300 Flush
4390 286/300 Read block 1152 len 16384
4391 287/300 Write block 1843 len 24576 data: 2 2 3 3 3 4
4392 288/300 Read block 40 len 36864
4393 289/300 Read block 1823 len 40960
4394 290/300 Write block 794 len 16384 data: 3 3 3 3
4395 291/300 Read block 156 len 28672
4396 292/300 Write block 3708 len 32768 data: 2 2 2 2 2 2 2 2
4397 293/300 Write block 2619 len 28672 data: 3 3 3 3 3 3 2
4398 294/300 Read block 1724 len 24576
4399 295/300 Write block 4943 len 36864 data: 4 4 3 3 3 2 2 2 2
4400 296/300 Read block 4265 len 32768
4401 297/300 Write block 1854 len 36864 data: 4 5 4 4 4 4 3 3 2
4402 298/300 Write block 735 len 12288 data: 2 2 2
4403 299/300 Read block 3418 len 20480
4404 300/300 Flush
4405 301/300 Write block 588 len 36864 data: 4 4 4 4 4 4 3 2 2
4406 Main task has work to do, go find it
4407 got dsc Start(0)
4408 start 0
4409 Jan 27 19:37:18.143 INFO request completed, latency_us: 205, response_code: 204, uri: /start/cid/0, method: POST, req_id: e1249de8-2e63-44ad-884a-9b010a6a24be, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
4410 Replay: started 0, returned:Ok(())
4411 ----------------------------------------------------------------
4412 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
4413 States: Offline Active Active
4414 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4415 2208 Acked Read 4 Sent Done Done false
4416 2209 Acked Flush 0 Sent Done Done false
4417 2210 Acked Write 7 Sent Done Done false
4418 2211 Acked Write 8 Sent Done Done false
4419 2212 Acked Write 6 Sent Done Done false
4420 2213 Acked Read 4 Sent Done Done false
4421 2214 Acked Write 5 Sent Done Done false
4422 2215 Acked Write 8 Sent Done Done false
4423 2216 Acked Read 7 Sent Done Done false
4424 2217 Acked Write 8 Sent Done Done false
4425 2218 Acked Read 5 Sent Done Done false
4426 2219 Acked Read 9 Sent Done Done false
4427 2220 Acked Write 2 Sent Done Done false
4428 2221 Acked Read 5 Sent Done Done false
4429 2222 Acked Write 4 Sent Done Done false
4430 2223 Acked Read 9 Sent Done Done false
4431 2224 Acked Read 9 Sent Done Done false
4432 2225 Acked Write 1 Sent Done Done false
4433 2226 Acked Read 1 Sent Done Done false
4434 2227 Acked Read 3 Sent Done Done false
4435 2228 Acked Read 9 Sent Done Done false
4436 2229 Acked Read 9 Sent Done Done false
4437 2230 Acked Read 2 Sent Done Done false
4438 2231 Acked Write 9 Sent Done Done false
4439 2232 Acked Read 8 Sent Done Done false
4440 2233 Acked Write 3 Sent Done Done false
4441 2234 Acked Write 4 Sent Done Done false
4442 2235 Acked Read 4 Sent Done Done false
4443 2236 Acked Flush 0 Sent Done Done false
4444 2237 Acked Read 2 Sent Done Done false
4445 2238 Acked Read 7 Sent Done Done false
4446 2239 Acked Read 8 Sent Done Done false
4447 2240 Acked Read 2 Sent Done Done false
4448 2241 Acked Read 5 Sent Done Done false
4449 2242 Acked Write 2 Sent Done Done false
4450 2243 Acked Write 7 Sent Done Done false
4451 2244 Acked Write 5 Sent Done Done false
4452 2245 Acked Flush 0 Sent Done Done false
4453 2246 Acked Write 1 Sent Done Done false
4454 2247 Acked Read 3 Sent Done Done false
4455 2248 Acked Read 4 Sent Done Done false
4456 2249 Acked Read 9 Sent Done Done false
4457 2250 Acked Read 5 Sent Done Done false
4458 2251 Acked Read 1 Sent Done Done false
4459 2252 Acked Read 9 Sent Done Done false
4460 2253 Acked Write 7 Sent Done Done false
4461 2254 Acked Read 9 Sent Done Done false
4462 2255 Acked Write 7 Sent Done Done false
4463 2256 Acked Write 5 Sent Done Done false
4464 2257 Acked Read 5 Sent Done Done false
4465 2258 Acked Write 7 Sent Done Done false
4466 2259 Acked Read 8 Sent Done Done false
4467 2260 Acked Write 3 Sent Done Done false
4468 2261 Acked Read 2 Sent Done Done false
4469 2262 Acked Read 5 Sent Done Done false
4470 2263 Acked Read 7 Sent Done Done false
4471 2264 Acked Read 4 Sent Done Done false
4472 2265 Acked Read 2 Sent Done Done false
4473 2266 Acked Write 8 Sent Done Done false
4474 2267 Acked Write 7 Sent Done Done false
4475 2268 Acked Read 3 Sent Done Done false
4476 2269 Acked Flush 0 Sent Done Done false
4477 2270 Acked Write 3 Sent Done Done false
4478 2271 Acked Write 4 Sent Done Done false
4479 2272 Acked Read 3 Sent Done Done false
4480 2273 Acked Write 5 Sent Done Done false
4481 2274 Acked Write 8 Sent Done Done false
4482 2275 Acked Write 1 Sent Done Done false
4483 2276 Acked Read 5 Sent Done Done false
4484 2277 Acked Read 10 Sent Done Done false
4485 2278 Acked Write 6 Sent Done Done false
4486 2279 Acked Read 8 Sent Done Done false
4487 2280 Acked Write 9 Sent Done Done false
4488 2281 Acked Write 6 Sent Done Done false
4489 2282 Acked Write 1 Sent Done Done false
4490 2283 Acked Flush 0 Sent Done Done false
4491 2284 Acked Read 6 Sent Done Done false
4492 2285 Acked Write 4 Sent Done Done false
4493 2286 Acked Write 6 Sent Done Done false
4494 2287 Acked Read 3 Sent Done Done false
4495 2288 Acked Read 4 Sent Done Done false
4496 2289 Acked Read 3 Sent Done Done false
4497 2290 Acked Flush 0 Sent Done Done false
4498 2291 Acked Read 3 Sent Done Done false
4499 2292 Acked Read 6 Sent Done Done false
4500 2293 Acked Write 10 Sent Done Done false
4501 2294 Acked Read 2 Sent Done Done false
4502 2295 Acked Read 3 Sent Done Done false
4503 2296 Acked Read 6 Sent Done Done false
4504 2297 Acked Read 6 Sent Done Done false
4505 2298 Acked Write 2 Sent Done Done false
4506 2299 Acked Write 8 Sent Done Done false
4507 2300 Acked Write 9 Sent Done Done false
4508 2301 Acked Read 10 Sent Done Done false
4509 2302 Acked Read 6 Sent Done Done false
4510 2303 Acked Read 5 Sent Done Done false
4511 2304 Acked Read 9 Sent Done Done false
4512 2305 Acked Read 7 Sent Done Done false
4513 2306 Acked Read 9 Sent Done Done false
4514 2307 Acked Flush 0 Sent Done Done false
4515 2308 Acked Write 5 Sent Done Done false
4516 2309 Acked Flush 0 Sent Done Done false
4517 2310 Acked Read 2 Sent Done Done false
4518 2311 Acked Read 1 Sent Done Done false
4519 2312 Acked Read 3 Sent Done Done false
4520 2313 Acked Write 2 Sent Done Done false
4521 2314 Acked Read 3 Sent Done Done false
4522 2315 Acked Read 2 Sent Done Done false
4523 2316 Acked Read 6 Sent Done Done false
4524 2317 Acked Flush 0 Sent Done Done false
4525 2318 Acked Write 5 Sent Done Done false
4526 2319 Acked Write 4 Sent Done Done false
4527 2320 Acked Write 10 Sent Done Done false
4528 2321 Acked Write 1 Sent Done Done false
4529 2322 Acked Read 2 Sent Done Done false
4530 2323 Acked Write 8 Sent Done Done false
4531 2324 Acked Write 9 Sent Done Done false
4532 2325 Acked Flush 0 Sent Done Done false
4533 2326 Acked Read 1 Sent Done Done false
4534 2327 Acked Read 1 Sent Done Done false
4535 2328 Acked Write 8 Sent Done Done false
4536 2329 Acked Read 2 Sent Done Done false
4537 2330 Acked Write 4 Sent Done Done false
4538 2331 Acked Read 10 Sent Done Done false
4539 2332 Acked Read 8 Sent Done Done false
4540 2333 Acked Read 9 Sent Done Done false
4541 2334 Acked Read 6 Sent Done Done false
4542 2335 Acked Write 10 Sent Done Done false
4543 2336 Acked Write 9 Sent Done Done false
4544 2337 Acked Flush 0 Sent Done Done false
4545 2338 Acked Read 4 Sent Done Done false
4546 2339 Acked Read 10 Sent Done Done false
4547 2340 Acked Write 10 Sent Done Done false
4548 2341 Acked Write 3 Sent Done Done false
4549 2342 Acked Write 3 Sent Done Done false
4550 2343 Acked Write 2 Sent Done Done false
4551 2344 Acked Write 3 Sent Done Done false
4552 2345 Acked Flush 0 Sent Done Done false
4553 2346 Acked Read 4 Sent Done Done false
4554 2347 Acked Read 7 Sent Done Done false
4555 2348 Acked Read 3 Sent Done Done false
4556 2349 Acked Flush 0 Sent Done Done false
4557 2350 Acked Write 8 Sent Done Done false
4558 2351 Acked Read 5 Sent Done Done false
4559 2352 Acked Read 3 Sent Done Done false
4560 2353 Acked Write 2 Sent Done Done false
4561 2354 Acked Read 6 Sent Done Done false
4562 2355 Acked Read 7 Sent Done Done false
4563 2356 Acked Read 2 Sent Done Done false
4564 2357 Acked Read 1 Sent Done Done false
4565 2358 Acked Read 6 Sent Done Done false
4566 2359 Acked Write 5 Sent Done Done false
4567 2360 Acked Write 4 Sent Done Done false
4568 2361 Acked Read 7 Sent Done Done false
4569 2362 Acked Read 4 Sent Done Done false
4570 2363 Acked Write 5 Sent Done Done false
4571 2364 Acked Write 9 Sent Done Done false
4572 2365 Acked Read 4 Sent Done Done false
4573 2366 Acked Read 1 Sent Done Done false
4574 2367 Acked Read 5 Sent Done Done false
4575 2368 Acked Read 1 Sent Done Done false
4576 2369 Acked Read 10 Sent Done Done false
4577 2370 Acked Read 4 Sent Done Done false
4578 2371 Acked Read 5 Sent Done Done false
4579 2372 Acked Read 3 Sent Done Done false
4580 2373 Acked Flush 0 Sent Done Done false
4581 2374 Acked Read 8 Sent Done Done false
4582 2375 Acked Read 8 Sent Done Done false
4583 2376 Acked Read 1 Sent Done Done false
4584 2377 Acked Read 6 Sent Done Done false
4585 2378 Acked Read 7 Sent Done Done false
4586 2379 Acked Write 5 Sent Done Done false
4587 2380 Acked Flush 0 Sent Done Done false
4588 2381 Acked Read 7 Sent Done Done false
4589 2382 Acked Flush 0 Sent Done Done false
4590 2383 Acked Read 4 Sent Done Done false
4591 2384 Acked Read 1 Sent Done Done false
4592 2385 Acked Write 10 Sent Done Done false
4593 2386 Acked Write 2 Sent Done Done false
4594 2387 Acked Flush 0 Sent Done Done false
4595 2388 Acked Flush 0 Sent Done Done false
4596 2389 Acked Read 3 Sent Done Done false
4597 2390 Acked Flush 0 Sent Done Done false
4598 2391 Acked Read 5 Sent Done Done false
4599 2392 Acked Read 10 Sent Done Done false
4600 2393 Acked Write 4 Sent Done Done false
4601 2394 Acked Write 3 Sent Done Done false
4602 2395 Acked Read 1 Sent Done Done false
4603 2396 Acked Read 6 Sent Done Done false
4604 2397 Acked Read 9 Sent Done Done false
4605 2398 Acked Write 5 Sent Done Done false
4606 2399 Acked Read 8 Sent Done Done false
4607 2400 Acked Read 1 Sent Done Done false
4608 2401 Acked Read 1 Sent Done Done false
4609 2402 Acked Read 7 Sent Done Done false
4610 2403 Acked Read 6 Sent Done Done false
4611 2404 Acked Read 4 Sent Done Done false
4612 2405 Acked Read 5 Sent Done Done false
4613 2406 Acked Write 3 Sent Done Done false
4614 2407 Acked Write 7 Sent Done Done false
4615 2408 Acked Read 6 Sent Done Done false
4616 2409 Acked Write 7 Sent Done Done false
4617 2410 Acked Read 10 Sent Done Done false
4618 2411 Acked Read 3 Sent Done Done false
4619 2412 Acked Read 5 Sent Done Done false
4620 2413 Acked Write 4 Sent Done Done false
4621 2414 Acked Write 5 Sent Done Done false
4622 2415 Acked Write 10 Sent Done Done false
4623 2416 Acked Write 7 Sent Done Done false
4624 2417 Acked Read 2 Sent Done Done false
4625 2418 Acked Read 2 Sent Done Done false
4626 2419 Acked Write 6 Sent Done Done false
4627 2420 Acked Read 3 Sent Done Done false
4628 2421 Acked Read 6 Sent Done Done false
4629 2422 Acked Write 8 Sent Done Done false
4630 2423 Acked Read 10 Sent Done Done false
4631 2424 Acked Write 4 Sent Done Done false
4632 2425 Acked Write 3 Sent Done Done false
4633 2426 Acked Read 4 Sent Done Done false
4634 2427 Acked Write 2 Sent Done Done false
4635 2428 Acked Write 4 Sent Done Done false
4636 2429 Acked Write 3 Sent Done Done false
4637 2430 Acked Read 10 Sent Done Done false
4638 2431 Acked Read 9 Sent Done Done false
4639 2432 Acked Write 8 Sent Done Done false
4640 2433 Acked Write 1 Sent Done Done false
4641 2434 Acked Read 6 Sent Done Done false
4642 2435 Acked Write 9 Sent Done Done false
4643 2436 Acked Write 3 Sent Done Done false
4644 2437 Acked Write 9 Sent Done Done false
4645 2438 Acked Write 2 Sent Done Done false
4646 2439 Acked Write 9 Sent Done Done false
4647 2440 Acked Write 1 Sent Done Done false
4648 2441 Acked Write 6 Sent Done Done false
4649 2442 Acked Read 4 Sent Done Done false
4650 2443 Acked Flush 0 Sent Done Done false
4651 2444 Acked Write 9 Sent Done Done false
4652 2445 Acked Read 10 Sent Done Done false
4653 2446 Acked Write 2 Sent Done Done false
4654 2447 Acked Read 8 Sent Done Done false
4655 2448 Acked Write 1 Sent Done Done false
4656 2449 Acked Read 1 Sent Done Done false
4657 2450 Acked Write 4 Sent Done Done false
4658 2451 Acked Read 9 Sent Done Done false
4659 2452 Acked Read 10 Sent Done Done false
4660 2453 Acked Write 7 Sent Done Done false
4661 2454 Acked Read 4 Sent Done Done false
4662 2455 Acked Flush 0 Sent Done Done false
4663 2456 Acked Read 9 Sent Done Done false
4664 2457 Acked Write 4 Sent Done Done false
4665 2458 Acked Read 8 Sent Done Done false
4666 2459 Acked Write 5 Sent Done Done false
4667 2460 Acked Flush 0 Sent Done Done false
4668 2461 Acked Read 9 Sent Done Done false
4669 2462 Acked Flush 0 Sent Done Done false
4670 2463 Acked Write 2 Sent Done Done false
4671 2464 Acked Write 7 Sent Done Done false
4672 2465 Acked Write 3 Sent Done Done false
4673 2466 Acked Read 7 Sent Done Done false
4674 2467 Acked Flush 0 Sent Done Done false
4675 2468 Acked Flush 0 Sent Done Done false
4676 2469 Acked Write 5 Sent Done Done false
4677 2470 Acked Write 1 Sent Done Done false
4678 2471 Acked Write 2 Sent Done Done false
4679 2472 Acked Read 7 Sent Done Done false
4680 2473 Acked Read 7 Sent Done Done false
4681 2474 Acked Write 10 Sent Done Done false
4682 2475 Acked Read 8 Sent Done Done false
4683 2476 Acked Write 1 Sent Done Done false
4684 2477 Acked Flush 0 Sent Done Done false
4685 2478 Acked Write 9 Sent Done Done false
4686 2479 Acked Read 7 Sent Done Done false
4687 2480 Acked Read 8 Sent Done Done false
4688 2481 Acked Write 2 Sent Done Done false
4689 2482 Acked Read 9 Sent Done Done false
4690 2483 Acked Write 9 Sent Done Done false
4691 2484 Acked Read 6 Sent Done Done false
4692 2485 Acked Read 6 Sent Done Done false
4693 2486 Acked Write 8 Sent Done Done false
4694 2487 Acked Write 6 Sent Done Done false
4695 2488 Acked Read 2 Sent Done Done false
4696 2489 Acked Flush 0 Sent Done Done false
4697 2490 Acked Read 4 Sent Done Done false
4698 2491 Acked Write 1 Sent Done Done false
4699 2492 Acked Flush 0 Sent Done Done false
4700 2493 Acked Read 4 Sent Done Done false
4701 2494 Acked Write 6 Sent Done Done false
4702 2495 Acked Read 9 Sent Done Done false
4703 2496 Acked Read 10 Sent Done Done false
4704 2497 Acked Write 4 Sent Done Done false
4705 2498 Acked Read 7 Sent Done Done false
4706 2499 Acked Write 8 Sent Done Done false
4707 2500 Acked Write 7 Sent Done Done false
4708 2501 Acked Read 6 Sent Done Done false
4709 2502 Acked Write 9 Sent Done Done false
4710 2503 Acked Read 8 Sent Done Done false
4711 2504 Acked Write 9 Sent Done Done false
4712 2505 Acked Write 3 Sent Done Done false
4713 2506 Acked Read 5 Sent Done Done false
4714 2507 Acked Flush 0 Sent Done Done false
4715 2508 Acked Write 9 Sent Done Done false
4716 STATES DS:0 DS:1 DS:2 TOTAL
4717 Sent 301 0 0 301
4718 Done 0 301 301 602
4719 Skipped 0 0 0 0
4720 Error 0 0 0 0
4721 Last Flush: 2207 2507 2507
4722 Downstairs last five completed: 2207 2206 2205 2204 2203
4723 Upstairs last five completed: 2508 2507 2506 2505 2504
4724 CLIENT: Up:0 ds:301 act:2
4725 ----------------------------------------------------------------
4726 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
4727 States: Offline Active Active
4728 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
4729 2208 Acked Read 4 Sent Done Done false
4730 2209 Acked Flush 0 Sent Done Done false
4731 2210 Acked Write 7 Sent Done Done false
4732 2211 Acked Write 8 Sent Done Done false
4733 2212 Acked Write 6 Sent Done Done false
4734 2213 Acked Read 4 Sent Done Done false
4735 2214 Acked Write 5 Sent Done Done false
4736 2215 Acked Write 8 Sent Done Done false
4737 2216 Acked Read 7 Sent Done Done false
4738 2217 Acked Write 8 Sent Done Done false
4739 2218 Acked Read 5 Sent Done Done false
4740 2219 Acked Read 9 Sent Done Done false
4741 2220 Acked Write 2 Sent Done Done false
4742 2221 Acked Read 5 Sent Done Done false
4743 2222 Acked Write 4 Sent Done Done false
4744 2223 Acked Read 9 Sent Done Done false
4745 2224 Acked Read 9 Sent Done Done false
4746 2225 Acked Write 1 Sent Done Done false
4747 2226 Acked Read 1 Sent Done Done false
4748 2227 Acked Read 3 Sent Done Done false
4749 2228 Acked Read 9 Sent Done Done false
4750 2229 Acked Read 9 Sent Done Done false
4751 2230 Acked Read 2 Sent Done Done false
4752 2231 Acked Write 9 Sent Done Done false
4753 2232 Acked Read 8 Sent Done Done false
4754 2233 Acked Write 3 Sent Done Done false
4755 2234 Acked Write 4 Sent Done Done false
4756 2235 Acked Read 4 Sent Done Done false
4757 2236 Acked Flush 0 Sent Done Done false
4758 2237 Acked Read 2 Sent Done Done false
4759 2238 Acked Read 7 Sent Done Done false
4760 2239 Acked Read 8 Sent Done Done false
4761 2240 Acked Read 2 Sent Done Done false
4762 2241 Acked Read 5 Sent Done Done false
4763 2242 Acked Write 2 Sent Done Done false
4764 2243 Acked Write 7 Sent Done Done false
4765 2244 Acked Write 5 Sent Done Done false
4766 2245 Acked Flush 0 Sent Done Done false
4767 2246 Acked Write 1 Sent Done Done false
4768 2247 Acked Read 3 Sent Done Done false
4769 2248 Acked Read 4 Sent Done Done false
4770 2249 Acked Read 9 Sent Done Done false
4771 2250 Acked Read 5 Sent Done Done false
4772 2251 Acked Read 1 Sent Done Done false
4773 2252 Acked Read 9 Sent Done Done false
4774 2253 Acked Write 7 Sent Done Done false
4775 2254 Acked Read 9 Sent Done Done false
4776 2255 Acked Write 7 Sent Done Done false
4777 2256 Acked Write 5 Sent Done Done false
4778 2257 Acked Read 5 Sent Done Done false
4779 2258 Acked Write 7 Sent Done Done false
4780 2259 Acked Read 8 Sent Done Done false
4781 2260 Acked Write 3 Sent Done Done false
4782 2261 Acked Read 2 Sent Done Done false
4783 2262 Acked Read 5 Sent Done Done false
4784 2263 Acked Read 7 Sent Done Done false
4785 2264 Acked Read 4 Sent Done Done false
4786 2265 Acked Read 2 Sent Done Done false
4787 2266 Acked Write 8 Sent Done Done false
4788 2267 Acked Write 7 Sent Done Done false
4789 2268 Acked Read 3 Sent Done Done false
4790 2269 Acked Flush 0 Sent Done Done false
4791 2270 Acked Write 3 Sent Done Done false
4792 2271 Acked Write 4 Sent Done Done false
4793 2272 Acked Read 3 Sent Done Done false
4794 2273 Acked Write 5 Sent Done Done false
4795 2274 Acked Write 8 Sent Done Done false
4796 2275 Acked Write 1 Sent Done Done false
4797 2276 Acked Read 5 Sent Done Done false
4798 2277 Acked Read 10 Sent Done Done false
4799 2278 Acked Write 6 Sent Done Done false
4800 2279 Acked Read 8 Sent Done Done false
4801 2280 Acked Write 9 Sent Done Done false
4802 2281 Acked Write 6 Sent Done Done false
4803 2282 Acked Write 1 Sent Done Done false
4804 2283 Acked Flush 0 Sent Done Done false
4805 2284 Acked Read 6 Sent Done Done false
4806 2285 Acked Write 4 Sent Done Done false
4807 2286 Acked Write 6 Sent Done Done false
4808 2287 Acked Read 3 Sent Done Done false
4809 2288 Acked Read 4 Sent Done Done false
4810 2289 Acked Read 3 Sent Done Done false
4811 2290 Acked Flush 0 Sent Done Done false
4812 2291 Acked Read 3 Sent Done Done false
4813 2292 Acked Read 6 Sent Done Done false
4814 2293 Acked Write 10 Sent Done Done false
4815 2294 Acked Read 2 Sent Done Done false
4816 2295 Acked Read 3 Sent Done Done false
4817 2296 Acked Read 6 Sent Done Done false
4818 2297 Acked Read 6 Sent Done Done false
4819 2298 Acked Write 2 Sent Done Done false
4820 2299 Acked Write 8 Sent Done Done false
4821 2300 Acked Write 9 Sent Done Done false
4822 2301 Acked Read 10 Sent Done Done false
4823 2302 Acked Read 6 Sent Done Done false
4824 2303 Acked Read 5 Sent Done Done false
4825 2304 Acked Read 9 Sent Done Done false
4826 2305 Acked Read 7 Sent Done Done false
4827 2306 Acked Read 9 Sent Done Done false
4828 2307 Acked Flush 0 Sent Done Done false
4829 2308 Acked Write 5 Sent Done Done false
4830 2309 Acked Flush 0 Sent Done Done false
4831 2310 Acked Read 2 Sent Done Done false
4832 2311 Acked Read 1 Sent Done Done false
4833 2312 Acked Read 3 Sent Done Done false
4834 2313 Acked Write 2 Sent Done Done false
4835 2314 Acked Read 3 Sent Done Done false
4836 2315 Acked Read 2 Sent Done Done false
4837 2316 Acked Read 6 Sent Done Done false
4838 2317 Acked Flush 0 Sent Done Done false
4839 2318 Acked Write 5 Sent Done Done false
4840 2319 Acked Write 4 Sent Done Done false
4841 2320 Acked Write 10 Sent Done Done false
4842 2321 Acked Write 1 Sent Done Done false
4843 2322 Acked Read 2 Sent Done Done false
4844 2323 Acked Write 8 Sent Done Done false
4845 2324 Acked Write 9 Sent Done Done false
4846 2325 Acked Flush 0 Sent Done Done false
4847 2326 Acked Read 1 Sent Done Done false
4848 2327 Acked Read 1 Sent Done Done false
4849 2328 Acked Write 8 Sent Done Done false
4850 2329 Acked Read 2 Sent Done Done false
4851 2330 Acked Write 4 Sent Done Done false
4852 2331 Acked Read 10 Sent Done Done false
4853 2332 Acked Read 8 Sent Done Done false
4854 2333 Acked Read 9 Sent Done Done false
4855 2334 Acked Read 6 Sent Done Done false
4856 2335 Acked Write 10 Sent Done Done false
4857 2336 Acked Write 9 Sent Done Done false
4858 2337 Acked Flush 0 Sent Done Done false
4859 2338 Acked Read 4 Sent Done Done false
4860 2339 Acked Read 10 Sent Done Done false
4861 2340 Acked Write 10 Sent Done Done false
4862 2341 Acked Write 3 Sent Done Done false
4863 2342 Acked Write 3 Sent Done Done false
4864 2343 Acked Write 2 Sent Done Done false
4865 2344 Acked Write 3 Sent Done Done false
4866 2345 Acked Flush 0 Sent Done Done false
4867 2346 Acked Read 4 Sent Done Done false
4868 2347 Acked Read 7 Sent Done Done false
4869 2348 Acked Read 3 Sent Done Done false
4870 2349 Acked Flush 0 Sent Done Done false
4871 2350 Acked Write 8 Sent Done Done false
4872 2351 Acked Read 5 Sent Done Done false
4873 2352 Acked Read 3 Sent Done Done false
4874 2353 Acked Write 2 Sent Done Done false
4875 2354 Acked Read 6 Sent Done Done false
4876 2355 Acked Read 7 Sent Done Done false
4877 2356 Acked Read 2 Sent Done Done false
4878 2357 Acked Read 1 Sent Done Done false
4879 2358 Acked Read 6 Sent Done Done false
4880 2359 Acked Write 5 Sent Done Done false
4881 2360 Acked Write 4 Sent Done Done false
4882 2361 Acked Read 7 Sent Done Done false
4883 2362 Acked Read 4 Sent Done Done false
4884 2363 Acked Write 5 Sent Done Done false
4885 2364 Acked Write 9 Sent Done Done false
4886 2365 Acked Read 4 Sent Done Done false
4887 2366 Acked Read 1 Sent Done Done false
4888 2367 Acked Read 5 Sent Done Done false
4889 2368 Acked Read 1 Sent Done Done false
4890 2369 Acked Read 10 Sent Done Done false
4891 2370 Acked Read 4 Sent Done Done false
4892 2371 Acked Read 5 Sent Done Done false
4893 2372 Acked Read 3 Sent Done Done false
4894 2373 Acked Flush 0 Sent Done Done false
4895 2374 Acked Read 8 Sent Done Done false
4896 2375 Acked Read 8 Sent Done Done false
4897 2376 Acked Read 1 Sent Done Done false
4898 2377 Acked Read 6 Sent Done Done false
4899 2378 Acked Read 7 Sent Done Done false
4900 2379 Acked Write 5 Sent Done Done false
4901 2380 Acked Flush 0 Sent Done Done false
4902 2381 Acked Read 7 Sent Done Done false
4903 2382 Acked Flush 0 Sent Done Done false
4904 2383 Acked Read 4 Sent Done Done false
4905 2384 Acked Read 1 Sent Done Done false
4906 2385 Acked Write 10 Sent Done Done false
4907 2386 Acked Write 2 Sent Done Done false
4908 2387 Acked Flush 0 Sent Done Done false
4909 2388 Acked Flush 0 Sent Done Done false
4910 2389 Acked Read 3 Sent Done Done false
4911 2390 Acked Flush 0 Sent Done Done false
4912 2391 Acked Read 5 Sent Done Done false
4913 2392 Acked Read 10 Sent Done Done false
4914 2393 Acked Write 4 Sent Done Done false
4915 2394 Acked Write 3 Sent Done Done false
4916 2395 Acked Read 1 Sent Done Done false
4917 2396 Acked Read 6 Sent Done Done false
4918 2397 Acked Read 9 Sent Done Done false
4919 2398 Acked Write 5 Sent Done Done false
4920 2399 Acked Read 8 Sent Done Done false
4921 2400 Acked Read 1 Sent Done Done false
4922 2401 Acked Read 1 Sent Done Done false
4923 2402 Acked Read 7 Sent Done Done false
4924 2403 Acked Read 6 Sent Done Done false
4925 2404 Acked Read 4 Sent Done Done false
4926 2405 Acked Read 5 Sent Done Done false
4927 2406 Acked Write 3 Sent Done Done false
4928 2407 Acked Write 7 Sent Done Done false
4929 2408 Acked Read 6 Sent Done Done false
4930 2409 Acked Write 7 Sent Done Done false
4931 2410 Acked Read 10 Sent Done Done false
4932 2411 Acked Read 3 Sent Done Done false
4933 2412 Acked Read 5 Sent Done Done false
4934 2413 Acked Write 4 Sent Done Done false
4935 2414 Acked Write 5 Sent Done Done false
4936 2415 Acked Write 10 Sent Done Done false
4937 2416 Acked Write 7 Sent Done Done false
4938 2417 Acked Read 2 Sent Done Done false
4939 2418 Acked Read 2 Sent Done Done false
4940 2419 Acked Write 6 Sent Done Done false
4941 2420 Acked Read 3 Sent Done Done false
4942 2421 Acked Read 6 Sent Done Done false
4943 2422 Acked Write 8 Sent Done Done false
4944 2423 Acked Read 10 Sent Done Done false
4945 2424 Acked Write 4 Sent Done Done false
4946 2425 Acked Write 3 Sent Done Done false
4947 2426 Acked Read 4 Sent Done Done false
4948 2427 Acked Write 2 Sent Done Done false
4949 2428 Acked Write 4 Sent Done Done false
4950 2429 Acked Write 3 Sent Done Done false
4951 2430 Acked Read 10 Sent Done Done false
4952 2431 Acked Read 9 Sent Done Done false
4953 2432 Acked Write 8 Sent Done Done false
4954 2433 Acked Write 1 Sent Done Done false
4955 2434 Acked Read 6 Sent Done Done false
4956 2435 Acked Write 9 Sent Done Done false
4957 2436 Acked Write 3 Sent Done Done false
4958 2437 Acked Write 9 Sent Done Done false
4959 2438 Acked Write 2 Sent Done Done false
4960 2439 Acked Write 9 Sent Done Done false
4961 2440 Acked Write 1 Sent Done Done false
4962 2441 Acked Write 6 Sent Done Done false
4963 2442 Acked Read 4 Sent Done Done false
4964 2443 Acked Flush 0 Sent Done Done false
4965 2444 Acked Write 9 Sent Done Done false
4966 2445 Acked Read 10 Sent Done Done false
4967 2446 Acked Write 2 Sent Done Done false
4968 2447 Acked Read 8 Sent Done Done false
4969 2448 Acked Write 1 Sent Done Done false
4970 2449 Acked Read 1 Sent Done Done false
4971 2450 Acked Write 4 Sent Done Done false
4972 2451 Acked Read 9 Sent Done Done false
4973 2452 Acked Read 10 Sent Done Done false
4974 2453 Acked Write 7 Sent Done Done false
4975 2454 Acked Read 4 Sent Done Done false
4976 2455 Acked Flush 0 Sent Done Done false
4977 2456 Acked Read 9 Sent Done Done false
4978 2457 Acked Write 4 Sent Done Done false
4979 2458 Acked Read 8 Sent Done Done false
4980 2459 Acked Write 5 Sent Done Done false
4981 2460 Acked Flush 0 Sent Done Done false
4982 2461 Acked Read 9 Sent Done Done false
4983 2462 Acked Flush 0 Sent Done Done false
4984 2463 Acked Write 2 Sent Done Done false
4985 2464 Acked Write 7 Sent Done Done false
4986 2465 Acked Write 3 Sent Done Done false
4987 2466 Acked Read 7 Sent Done Done false
4988 2467 Acked Flush 0 Sent Done Done false
4989 2468 Acked Flush 0 Sent Done Done false
4990 2469 Acked Write 5 Sent Done Done false
4991 2470 Acked Write 1 Sent Done Done false
4992 2471 Acked Write 2 Sent Done Done false
4993 2472 Acked Read 7 Sent Done Done false
4994 2473 Acked Read 7 Sent Done Done false
4995 2474 Acked Write 10 Sent Done Done false
4996 2475 Acked Read 8 Sent Done Done false
4997 2476 Acked Write 1 Sent Done Done false
4998 2477 Acked Flush 0 Sent Done Done false
4999 2478 Acked Write 9 Sent Done Done false
5000 2479 Acked Read 7 Sent Done Done false
5001 2480 Acked Read 8 Sent Done Done false
5002 2481 Acked Write 2 Sent Done Done false
5003 2482 Acked Read 9 Sent Done Done false
5004 2483 Acked Write 9 Sent Done Done false
5005 2484 Acked Read 6 Sent Done Done false
5006 2485 Acked Read 6 Sent Done Done false
5007 2486 Acked Write 8 Sent Done Done false
5008 2487 Acked Write 6 Sent Done Done false
5009 2488 Acked Read 2 Sent Done Done false
5010 2489 Acked Flush 0 Sent Done Done false
5011 2490 Acked Read 4 Sent Done Done false
5012 2491 Acked Write 1 Sent Done Done false
5013 2492 Acked Flush 0 Sent Done Done false
5014 2493 Acked Read 4 Sent Done Done false
5015 2494 Acked Write 6 Sent Done Done false
5016 2495 Acked Read 9 Sent Done Done false
5017 2496 Acked Read 10 Sent Done Done false
5018 2497 Acked Write 4 Sent Done Done false
5019 2498 Acked Read 7 Sent Done Done false
5020 2499 Acked Write 8 Sent Done Done false
5021 2500 Acked Write 7 Sent Done Done false
5022 2501 Acked Read 6 Sent Done Done false
5023 2502 Acked Write 9 Sent Done Done false
5024 2503 Acked Read 8 Sent Done Done false
5025 2504 Acked Write 9 Sent Done Done false
5026 2505 Acked Write 3 Sent Done Done false
5027 2506 Acked Read 5 Sent Done Done false
5028 2507 Acked Flush 0 Sent Done Done false
5029 2508 Acked Write 9 Sent Done Done false
5030 2509 Acked Flush 0 Sent Done Done false
5031 STATES DS:0 DS:1 DS:2 TOTAL
5032 Sent 302 0 0 302
5033 Done 0 302 302 604
5034 Skipped 0 0 0 0
5035 Error 0 0 0 0
5036 Last Flush: 2207 2509 2509
5037 Downstairs last five completed: 2207 2206 2205 2204 2203
5038 Upstairs last five completed: 2509 2508 2507 2506 2505
5039 CLIENT: Up:0 ds:302 act:2
50402025-01-27T19:37:23.901ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
50412025-01-27T19:37:23.901ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
50422025-01-27T19:37:23.901ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
50432025-01-27T19:37:23.901ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
50442025-01-27T19:37:23.901ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
5045 [8810] got start action
5046 [8810] I am going to restart
5047 Starting downstairs at port 8810
5048 Make output file at "/tmp/dsc/downstairs-8810.txt"
5049 [8810][0] reports Starting
5050 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1205)
5051 [8810][0] reports Running
5052 ----------------------------------------------------------------
5053 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5054 States: Offline Active Active
5055 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5056 2208 Acked Read 4 Sent Done Done false
5057 2209 Acked Flush 0 Sent Done Done false
5058 2210 Acked Write 7 Sent Done Done false
5059 2211 Acked Write 8 Sent Done Done false
5060 2212 Acked Write 6 Sent Done Done false
5061 2213 Acked Read 4 Sent Done Done false
5062 2214 Acked Write 5 Sent Done Done false
5063 2215 Acked Write 8 Sent Done Done false
5064 2216 Acked Read 7 Sent Done Done false
5065 2217 Acked Write 8 Sent Done Done false
5066 2218 Acked Read 5 Sent Done Done false
5067 2219 Acked Read 9 Sent Done Done false
5068 2220 Acked Write 2 Sent Done Done false
5069 2221 Acked Read 5 Sent Done Done false
5070 2222 Acked Write 4 Sent Done Done false
5071 2223 Acked Read 9 Sent Done Done false
5072 2224 Acked Read 9 Sent Done Done false
5073 2225 Acked Write 1 Sent Done Done false
5074 2226 Acked Read 1 Sent Done Done false
5075 2227 Acked Read 3 Sent Done Done false
5076 2228 Acked Read 9 Sent Done Done false
5077 2229 Acked Read 9 Sent Done Done false
5078 2230 Acked Read 2 Sent Done Done false
5079 2231 Acked Write 9 Sent Done Done false
5080 2232 Acked Read 8 Sent Done Done false
5081 2233 Acked Write 3 Sent Done Done false
5082 2234 Acked Write 4 Sent Done Done false
5083 2235 Acked Read 4 Sent Done Done false
5084 2236 Acked Flush 0 Sent Done Done false
5085 2237 Acked Read 2 Sent Done Done false
5086 2238 Acked Read 7 Sent Done Done false
5087 2239 Acked Read 8 Sent Done Done false
5088 2240 Acked Read 2 Sent Done Done false
5089 2241 Acked Read 5 Sent Done Done false
5090 2242 Acked Write 2 Sent Done Done false
5091 2243 Acked Write 7 Sent Done Done false
5092 2244 Acked Write 5 Sent Done Done false
5093 2245 Acked Flush 0 Sent Done Done false
5094 2246 Acked Write 1 Sent Done Done false
5095 2247 Acked Read 3 Sent Done Done false
5096 2248 Acked Read 4 Sent Done Done false
5097 2249 Acked Read 9 Sent Done Done false
5098 2250 Acked Read 5 Sent Done Done false
5099 2251 Acked Read 1 Sent Done Done false
5100 2252 Acked Read 9 Sent Done Done false
5101 2253 Acked Write 7 Sent Done Done false
5102 2254 Acked Read 9 Sent Done Done false
5103 2255 Acked Write 7 Sent Done Done false
5104 2256 Acked Write 5 Sent Done Done false
5105 2257 Acked Read 5 Sent Done Done false
5106 2258 Acked Write 7 Sent Done Done false
5107 2259 Acked Read 8 Sent Done Done false
5108 2260 Acked Write 3 Sent Done Done false
5109 2261 Acked Read 2 Sent Done Done false
5110 2262 Acked Read 5 Sent Done Done false
5111 2263 Acked Read 7 Sent Done Done false
5112 2264 Acked Read 4 Sent Done Done false
5113 2265 Acked Read 2 Sent Done Done false
5114 2266 Acked Write 8 Sent Done Done false
5115 2267 Acked Write 7 Sent Done Done false
5116 2268 Acked Read 3 Sent Done Done false
5117 2269 Acked Flush 0 Sent Done Done false
5118 2270 Acked Write 3 Sent Done Done false
5119 2271 Acked Write 4 Sent Done Done false
5120 2272 Acked Read 3 Sent Done Done false
5121 2273 Acked Write 5 Sent Done Done false
5122 2274 Acked Write 8 Sent Done Done false
5123 2275 Acked Write 1 Sent Done Done false
5124 2276 Acked Read 5 Sent Done Done false
5125 2277 Acked Read 10 Sent Done Done false
5126 2278 Acked Write 6 Sent Done Done false
5127 2279 Acked Read 8 Sent Done Done false
5128 2280 Acked Write 9 Sent Done Done false
5129 2281 Acked Write 6 Sent Done Done false
5130 2282 Acked Write 1 Sent Done Done false
5131 2283 Acked Flush 0 Sent Done Done false
5132 2284 Acked Read 6 Sent Done Done false
5133 2285 Acked Write 4 Sent Done Done false
5134 2286 Acked Write 6 Sent Done Done false
5135 2287 Acked Read 3 Sent Done Done false
5136 2288 Acked Read 4 Sent Done Done false
5137 2289 Acked Read 3 Sent Done Done false
5138 2290 Acked Flush 0 Sent Done Done false
5139 2291 Acked Read 3 Sent Done Done false
5140 2292 Acked Read 6 Sent Done Done false
5141 2293 Acked Write 10 Sent Done Done false
5142 2294 Acked Read 2 Sent Done Done false
5143 2295 Acked Read 3 Sent Done Done false
5144 2296 Acked Read 6 Sent Done Done false
5145 2297 Acked Read 6 Sent Done Done false
5146 2298 Acked Write 2 Sent Done Done false
5147 2299 Acked Write 8 Sent Done Done false
5148 2300 Acked Write 9 Sent Done Done false
5149 2301 Acked Read 10 Sent Done Done false
5150 2302 Acked Read 6 Sent Done Done false
5151 2303 Acked Read 5 Sent Done Done false
5152 2304 Acked Read 9 Sent Done Done false
5153 2305 Acked Read 7 Sent Done Done false
5154 2306 Acked Read 9 Sent Done Done false
5155 2307 Acked Flush 0 Sent Done Done false
5156 2308 Acked Write 5 Sent Done Done false
5157 2309 Acked Flush 0 Sent Done Done false
5158 2310 Acked Read 2 Sent Done Done false
5159 2311 Acked Read 1 Sent Done Done false
5160 2312 Acked Read 3 Sent Done Done false
5161 2313 Acked Write 2 Sent Done Done false
5162 2314 Acked Read 3 Sent Done Done false
5163 2315 Acked Read 2 Sent Done Done false
5164 2316 Acked Read 6 Sent Done Done false
5165 2317 Acked Flush 0 Sent Done Done false
5166 2318 Acked Write 5 Sent Done Done false
5167 2319 Acked Write 4 Sent Done Done false
5168 2320 Acked Write 10 Sent Done Done false
5169 2321 Acked Write 1 Sent Done Done false
5170 2322 Acked Read 2 Sent Done Done false
5171 2323 Acked Write 8 Sent Done Done false
5172 2324 Acked Write 9 Sent Done Done false
5173 2325 Acked Flush 0 Sent Done Done false
5174 2326 Acked Read 1 Sent Done Done false
5175 2327 Acked Read 1 Sent Done Done false
5176 2328 Acked Write 8 Sent Done Done false
5177 2329 Acked Read 2 Sent Done Done false
5178 2330 Acked Write 4 Sent Done Done false
5179 2331 Acked Read 10 Sent Done Done false
5180 2332 Acked Read 8 Sent Done Done false
5181 2333 Acked Read 9 Sent Done Done false
5182 2334 Acked Read 6 Sent Done Done false
5183 2335 Acked Write 10 Sent Done Done false
5184 2336 Acked Write 9 Sent Done Done false
5185 2337 Acked Flush 0 Sent Done Done false
5186 2338 Acked Read 4 Sent Done Done false
5187 2339 Acked Read 10 Sent Done Done false
5188 2340 Acked Write 10 Sent Done Done false
5189 2341 Acked Write 3 Sent Done Done false
5190 2342 Acked Write 3 Sent Done Done false
5191 2343 Acked Write 2 Sent Done Done false
5192 2344 Acked Write 3 Sent Done Done false
5193 2345 Acked Flush 0 Sent Done Done false
5194 2346 Acked Read 4 Sent Done Done false
5195 2347 Acked Read 7 Sent Done Done false
5196 2348 Acked Read 3 Sent Done Done false
5197 2349 Acked Flush 0 Sent Done Done false
5198 2350 Acked Write 8 Sent Done Done false
5199 2351 Acked Read 5 Sent Done Done false
5200 2352 Acked Read 3 Sent Done Done false
5201 2353 Acked Write 2 Sent Done Done false
5202 2354 Acked Read 6 Sent Done Done false
5203 2355 Acked Read 7 Sent Done Done false
5204 2356 Acked Read 2 Sent Done Done false
5205 2357 Acked Read 1 Sent Done Done false
5206 2358 Acked Read 6 Sent Done Done false
5207 2359 Acked Write 5 Sent Done Done false
5208 2360 Acked Write 4 Sent Done Done false
5209 2361 Acked Read 7 Sent Done Done false
5210 2362 Acked Read 4 Sent Done Done false
5211 2363 Acked Write 5 Sent Done Done false
5212 2364 Acked Write 9 Sent Done Done false
5213 2365 Acked Read 4 Sent Done Done false
5214 2366 Acked Read 1 Sent Done Done false
5215 2367 Acked Read 5 Sent Done Done false
5216 2368 Acked Read 1 Sent Done Done false
5217 2369 Acked Read 10 Sent Done Done false
5218 2370 Acked Read 4 Sent Done Done false
5219 2371 Acked Read 5 Sent Done Done false
5220 2372 Acked Read 3 Sent Done Done false
5221 2373 Acked Flush 0 Sent Done Done false
5222 2374 Acked Read 8 Sent Done Done false
5223 2375 Acked Read 8 Sent Done Done false
5224 2376 Acked Read 1 Sent Done Done false
5225 2377 Acked Read 6 Sent Done Done false
5226 2378 Acked Read 7 Sent Done Done false
5227 2379 Acked Write 5 Sent Done Done false
5228 2380 Acked Flush 0 Sent Done Done false
5229 2381 Acked Read 7 Sent Done Done false
5230 2382 Acked Flush 0 Sent Done Done false
5231 2383 Acked Read 4 Sent Done Done false
5232 2384 Acked Read 1 Sent Done Done false
5233 2385 Acked Write 10 Sent Done Done false
5234 2386 Acked Write 2 Sent Done Done false
5235 2387 Acked Flush 0 Sent Done Done false
5236 2388 Acked Flush 0 Sent Done Done false
5237 2389 Acked Read 3 Sent Done Done false
5238 2390 Acked Flush 0 Sent Done Done false
5239 2391 Acked Read 5 Sent Done Done false
5240 2392 Acked Read 10 Sent Done Done false
5241 2393 Acked Write 4 Sent Done Done false
5242 2394 Acked Write 3 Sent Done Done false
5243 2395 Acked Read 1 Sent Done Done false
5244 2396 Acked Read 6 Sent Done Done false
5245 2397 Acked Read 9 Sent Done Done false
5246 2398 Acked Write 5 Sent Done Done false
5247 2399 Acked Read 8 Sent Done Done false
5248 2400 Acked Read 1 Sent Done Done false
5249 2401 Acked Read 1 Sent Done Done false
5250 2402 Acked Read 7 Sent Done Done false
5251 2403 Acked Read 6 Sent Done Done false
5252 2404 Acked Read 4 Sent Done Done false
5253 2405 Acked Read 5 Sent Done Done false
5254 2406 Acked Write 3 Sent Done Done false
5255 2407 Acked Write 7 Sent Done Done false
5256 2408 Acked Read 6 Sent Done Done false
5257 2409 Acked Write 7 Sent Done Done false
5258 2410 Acked Read 10 Sent Done Done false
5259 2411 Acked Read 3 Sent Done Done false
5260 2412 Acked Read 5 Sent Done Done false
5261 2413 Acked Write 4 Sent Done Done false
5262 2414 Acked Write 5 Sent Done Done false
5263 2415 Acked Write 10 Sent Done Done false
5264 2416 Acked Write 7 Sent Done Done false
5265 2417 Acked Read 2 Sent Done Done false
5266 2418 Acked Read 2 Sent Done Done false
5267 2419 Acked Write 6 Sent Done Done false
5268 2420 Acked Read 3 Sent Done Done false
5269 2421 Acked Read 6 Sent Done Done false
5270 2422 Acked Write 8 Sent Done Done false
5271 2423 Acked Read 10 Sent Done Done false
5272 2424 Acked Write 4 Sent Done Done false
5273 2425 Acked Write 3 Sent Done Done false
5274 2426 Acked Read 4 Sent Done Done false
5275 2427 Acked Write 2 Sent Done Done false
5276 2428 Acked Write 4 Sent Done Done false
5277 2429 Acked Write 3 Sent Done Done false
5278 2430 Acked Read 10 Sent Done Done false
5279 2431 Acked Read 9 Sent Done Done false
5280 2432 Acked Write 8 Sent Done Done false
5281 2433 Acked Write 1 Sent Done Done false
5282 2434 Acked Read 6 Sent Done Done false
5283 2435 Acked Write 9 Sent Done Done false
5284 2436 Acked Write 3 Sent Done Done false
5285 2437 Acked Write 9 Sent Done Done false
5286 2438 Acked Write 2 Sent Done Done false
5287 2439 Acked Write 9 Sent Done Done false
5288 2440 Acked Write 1 Sent Done Done false
5289 2441 Acked Write 6 Sent Done Done false
5290 2442 Acked Read 4 Sent Done Done false
5291 2443 Acked Flush 0 Sent Done Done false
5292 2444 Acked Write 9 Sent Done Done false
5293 2445 Acked Read 10 Sent Done Done false
5294 2446 Acked Write 2 Sent Done Done false
5295 2447 Acked Read 8 Sent Done Done false
5296 2448 Acked Write 1 Sent Done Done false
5297 2449 Acked Read 1 Sent Done Done false
5298 2450 Acked Write 4 Sent Done Done false
5299 2451 Acked Read 9 Sent Done Done false
5300 2452 Acked Read 10 Sent Done Done false
5301 2453 Acked Write 7 Sent Done Done false
5302 2454 Acked Read 4 Sent Done Done false
5303 2455 Acked Flush 0 Sent Done Done false
5304 2456 Acked Read 9 Sent Done Done false
5305 2457 Acked Write 4 Sent Done Done false
5306 2458 Acked Read 8 Sent Done Done false
5307 2459 Acked Write 5 Sent Done Done false
5308 2460 Acked Flush 0 Sent Done Done false
5309 2461 Acked Read 9 Sent Done Done false
5310 2462 Acked Flush 0 Sent Done Done false
5311 2463 Acked Write 2 Sent Done Done false
5312 2464 Acked Write 7 Sent Done Done false
5313 2465 Acked Write 3 Sent Done Done false
5314 2466 Acked Read 7 Sent Done Done false
5315 2467 Acked Flush 0 Sent Done Done false
5316 2468 Acked Flush 0 Sent Done Done false
5317 2469 Acked Write 5 Sent Done Done false
5318 2470 Acked Write 1 Sent Done Done false
5319 2471 Acked Write 2 Sent Done Done false
5320 2472 Acked Read 7 Sent Done Done false
5321 2473 Acked Read 7 Sent Done Done false
5322 2474 Acked Write 10 Sent Done Done false
5323 2475 Acked Read 8 Sent Done Done false
5324 2476 Acked Write 1 Sent Done Done false
5325 2477 Acked Flush 0 Sent Done Done false
5326 2478 Acked Write 9 Sent Done Done false
5327 2479 Acked Read 7 Sent Done Done false
5328 2480 Acked Read 8 Sent Done Done false
5329 2481 Acked Write 2 Sent Done Done false
5330 2482 Acked Read 9 Sent Done Done false
5331 2483 Acked Write 9 Sent Done Done false
5332 2484 Acked Read 6 Sent Done Done false
5333 2485 Acked Read 6 Sent Done Done false
5334 2486 Acked Write 8 Sent Done Done false
5335 2487 Acked Write 6 Sent Done Done false
5336 2488 Acked Read 2 Sent Done Done false
5337 2489 Acked Flush 0 Sent Done Done false
5338 2490 Acked Read 4 Sent Done Done false
5339 2491 Acked Write 1 Sent Done Done false
5340 2492 Acked Flush 0 Sent Done Done false
5341 2493 Acked Read 4 Sent Done Done false
5342 2494 Acked Write 6 Sent Done Done false
5343 2495 Acked Read 9 Sent Done Done false
5344 2496 Acked Read 10 Sent Done Done false
5345 2497 Acked Write 4 Sent Done Done false
5346 2498 Acked Read 7 Sent Done Done false
5347 2499 Acked Write 8 Sent Done Done false
5348 2500 Acked Write 7 Sent Done Done false
5349 2501 Acked Read 6 Sent Done Done false
5350 2502 Acked Write 9 Sent Done Done false
5351 2503 Acked Read 8 Sent Done Done false
5352 2504 Acked Write 9 Sent Done Done false
5353 2505 Acked Write 3 Sent Done Done false
5354 2506 Acked Read 5 Sent Done Done false
5355 2507 Acked Flush 0 Sent Done Done false
5356 2508 Acked Write 9 Sent Done Done false
5357 2509 Acked Flush 0 Sent Done Done false
5358 STATES DS:0 DS:1 DS:2 TOTAL
5359 Sent 302 0 0 302
5360 Done 0 302 302 604
5361 Skipped 0 0 0 0
5362 Error 0 0 0 0
5363 Last Flush: 2207 2509 2509
5364 Downstairs last five completed: 2207 2206 2205 2204 2203
5365 Upstairs last five completed: 2509 2508 2507 2506 2505
5366 CLIENT: Up:0 ds:302 act:2
5367 ----------------------------------------------------------------
5368 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
5369 States: Offline Active Active
5370 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
5371 2208 Acked Read 4 Sent Done Done false
5372 2209 Acked Flush 0 Sent Done Done false
5373 2210 Acked Write 7 Sent Done Done false
5374 2211 Acked Write 8 Sent Done Done false
5375 2212 Acked Write 6 Sent Done Done false
5376 2213 Acked Read 4 Sent Done Done false
5377 2214 Acked Write 5 Sent Done Done false
5378 2215 Acked Write 8 Sent Done Done false
5379 2216 Acked Read 7 Sent Done Done false
5380 2217 Acked Write 8 Sent Done Done false
5381 2218 Acked Read 5 Sent Done Done false
5382 2219 Acked Read 9 Sent Done Done false
5383 2220 Acked Write 2 Sent Done Done false
5384 2221 Acked Read 5 Sent Done Done false
5385 2222 Acked Write 4 Sent Done Done false
5386 2223 Acked Read 9 Sent Done Done false
5387 2224 Acked Read 9 Sent Done Done false
5388 2225 Acked Write 1 Sent Done Done false
5389 2226 Acked Read 1 Sent Done Done false
5390 2227 Acked Read 3 Sent Done Done false
5391 2228 Acked Read 9 Sent Done Done false
5392 2229 Acked Read 9 Sent Done Done false
5393 2230 Acked Read 2 Sent Done Done false
5394 2231 Acked Write 9 Sent Done Done false
5395 2232 Acked Read 8 Sent Done Done false
5396 2233 Acked Write 3 Sent Done Done false
5397 2234 Acked Write 4 Sent Done Done false
5398 2235 Acked Read 4 Sent Done Done false
5399 2236 Acked Flush 0 Sent Done Done false
5400 2237 Acked Read 2 Sent Done Done false
5401 2238 Acked Read 7 Sent Done Done false
5402 2239 Acked Read 8 Sent Done Done false
5403 2240 Acked Read 2 Sent Done Done false
5404 2241 Acked Read 5 Sent Done Done false
5405 2242 Acked Write 2 Sent Done Done false
5406 2243 Acked Write 7 Sent Done Done false
5407 2244 Acked Write 5 Sent Done Done false
5408 2245 Acked Flush 0 Sent Done Done false
5409 2246 Acked Write 1 Sent Done Done false
5410 2247 Acked Read 3 Sent Done Done false
5411 2248 Acked Read 4 Sent Done Done false
5412 2249 Acked Read 9 Sent Done Done false
5413 2250 Acked Read 5 Sent Done Done false
5414 2251 Acked Read 1 Sent Done Done false
5415 2252 Acked Read 9 Sent Done Done false
5416 2253 Acked Write 7 Sent Done Done false
5417 2254 Acked Read 9 Sent Done Done false
5418 2255 Acked Write 7 Sent Done Done false
5419 2256 Acked Write 5 Sent Done Done false
5420 2257 Acked Read 5 Sent Done Done false
5421 2258 Acked Write 7 Sent Done Done false
5422 2259 Acked Read 8 Sent Done Done false
5423 2260 Acked Write 3 Sent Done Done false
5424 2261 Acked Read 2 Sent Done Done false
5425 2262 Acked Read 5 Sent Done Done false
5426 2263 Acked Read 7 Sent Done Done false
5427 2264 Acked Read 4 Sent Done Done false
5428 2265 Acked Read 2 Sent Done Done false
5429 2266 Acked Write 8 Sent Done Done false
5430 2267 Acked Write 7 Sent Done Done false
5431 2268 Acked Read 3 Sent Done Done false
5432 2269 Acked Flush 0 Sent Done Done false
5433 2270 Acked Write 3 Sent Done Done false
5434 2271 Acked Write 4 Sent Done Done false
5435 2272 Acked Read 3 Sent Done Done false
5436 2273 Acked Write 5 Sent Done Done false
5437 2274 Acked Write 8 Sent Done Done false
5438 2275 Acked Write 1 Sent Done Done false
5439 2276 Acked Read 5 Sent Done Done false
5440 2277 Acked Read 10 Sent Done Done false
5441 2278 Acked Write 6 Sent Done Done false
5442 2279 Acked Read 8 Sent Done Done false
5443 2280 Acked Write 9 Sent Done Done false
5444 2281 Acked Write 6 Sent Done Done false
5445 2282 Acked Write 1 Sent Done Done false
5446 2283 Acked Flush 0 Sent Done Done false
5447 2284 Acked Read 6 Sent Done Done false
5448 2285 Acked Write 4 Sent Done Done false
5449 2286 Acked Write 6 Sent Done Done false
5450 2287 Acked Read 3 Sent Done Done false
5451 2288 Acked Read 4 Sent Done Done false
5452 2289 Acked Read 3 Sent Done Done false
5453 2290 Acked Flush 0 Sent Done Done false
5454 2291 Acked Read 3 Sent Done Done false
5455 2292 Acked Read 6 Sent Done Done false
5456 2293 Acked Write 10 Sent Done Done false
5457 2294 Acked Read 2 Sent Done Done false
5458 2295 Acked Read 3 Sent Done Done false
5459 2296 Acked Read 6 Sent Done Done false
5460 2297 Acked Read 6 Sent Done Done false
5461 2298 Acked Write 2 Sent Done Done false
5462 2299 Acked Write 8 Sent Done Done false
5463 2300 Acked Write 9 Sent Done Done false
5464 2301 Acked Read 10 Sent Done Done false
5465 2302 Acked Read 6 Sent Done Done false
5466 2303 Acked Read 5 Sent Done Done false
5467 2304 Acked Read 9 Sent Done Done false
5468 2305 Acked Read 7 Sent Done Done false
5469 2306 Acked Read 9 Sent Done Done false
5470 2307 Acked Flush 0 Sent Done Done false
5471 2308 Acked Write 5 Sent Done Done false
5472 2309 Acked Flush 0 Sent Done Done false
5473 2310 Acked Read 2 Sent Done Done false
5474 2311 Acked Read 1 Sent Done Done false
5475 2312 Acked Read 3 Sent Done Done false
5476 2313 Acked Write 2 Sent Done Done false
5477 2314 Acked Read 3 Sent Done Done false
5478 2315 Acked Read 2 Sent Done Done false
5479 2316 Acked Read 6 Sent Done Done false
5480 2317 Acked Flush 0 Sent Done Done false
5481 2318 Acked Write 5 Sent Done Done false
5482 2319 Acked Write 4 Sent Done Done false
5483 2320 Acked Write 10 Sent Done Done false
5484 2321 Acked Write 1 Sent Done Done false
5485 2322 Acked Read 2 Sent Done Done false
5486 2323 Acked Write 8 Sent Done Done false
5487 2324 Acked Write 9 Sent Done Done false
5488 2325 Acked Flush 0 Sent Done Done false
5489 2326 Acked Read 1 Sent Done Done false
5490 2327 Acked Read 1 Sent Done Done false
5491 2328 Acked Write 8 Sent Done Done false
5492 2329 Acked Read 2 Sent Done Done false
5493 2330 Acked Write 4 Sent Done Done false
5494 2331 Acked Read 10 Sent Done Done false
5495 2332 Acked Read 8 Sent Done Done false
5496 2333 Acked Read 9 Sent Done Done false
5497 2334 Acked Read 6 Sent Done Done false
5498 2335 Acked Write 10 Sent Done Done false
5499 2336 Acked Write 9 Sent Done Done false
5500 2337 Acked Flush 0 Sent Done Done false
5501 2338 Acked Read 4 Sent Done Done false
5502 2339 Acked Read 10 Sent Done Done false
5503 2340 Acked Write 10 Sent Done Done false
5504 2341 Acked Write 3 Sent Done Done false
5505 2342 Acked Write 3 Sent Done Done false
5506 2343 Acked Write 2 Sent Done Done false
5507 2344 Acked Write 3 Sent Done Done false
5508 2345 Acked Flush 0 Sent Done Done false
5509 2346 Acked Read 4 Sent Done Done false
5510 2347 Acked Read 7 Sent Done Done false
5511 2348 Acked Read 3 Sent Done Done false
5512 2349 Acked Flush 0 Sent Done Done false
5513 2350 Acked Write 8 Sent Done Done false
5514 2351 Acked Read 5 Sent Done Done false
5515 2352 Acked Read 3 Sent Done Done false
5516 2353 Acked Write 2 Sent Done Done false
5517 2354 Acked Read 6 Sent Done Done false
5518 2355 Acked Read 7 Sent Done Done false
5519 2356 Acked Read 2 Sent Done Done false
5520 2357 Acked Read 1 Sent Done Done false
5521 2358 Acked Read 6 Sent Done Done false
5522 2359 Acked Write 5 Sent Done Done false
5523 2360 Acked Write 4 Sent Done Done false
5524 2361 Acked Read 7 Sent Done Done false
5525 2362 Acked Read 4 Sent Done Done false
5526 2363 Acked Write 5 Sent Done Done false
5527 2364 Acked Write 9 Sent Done Done false
5528 2365 Acked Read 4 Sent Done Done false
5529 2366 Acked Read 1 Sent Done Done false
5530 2367 Acked Read 5 Sent Done Done false
5531 2368 Acked Read 1 Sent Done Done false
5532 2369 Acked Read 10 Sent Done Done false
5533 2370 Acked Read 4 Sent Done Done false
5534 2371 Acked Read 5 Sent Done Done false
5535 2372 Acked Read 3 Sent Done Done false
5536 2373 Acked Flush 0 Sent Done Done false
5537 2374 Acked Read 8 Sent Done Done false
5538 2375 Acked Read 8 Sent Done Done false
5539 2376 Acked Read 1 Sent Done Done false
5540 2377 Acked Read 6 Sent Done Done false
5541 2378 Acked Read 7 Sent Done Done false
5542 2379 Acked Write 5 Sent Done Done false
5543 2380 Acked Flush 0 Sent Done Done false
5544 2381 Acked Read 7 Sent Done Done false
5545 2382 Acked Flush 0 Sent Done Done false
5546 2383 Acked Read 4 Sent Done Done false
5547 2384 Acked Read 1 Sent Done Done false
5548 2385 Acked Write 10 Sent Done Done false
5549 2386 Acked Write 2 Sent Done Done false
5550 2387 Acked Flush 0 Sent Done Done false
5551 2388 Acked Flush 0 Sent Done Done false
5552 2389 Acked Read 3 Sent Done Done false
5553 2390 Acked Flush 0 Sent Done Done false
5554 2391 Acked Read 5 Sent Done Done false
5555 2392 Acked Read 10 Sent Done Done false
5556 2393 Acked Write 4 Sent Done Done false
5557 2394 Acked Write 3 Sent Done Done false
5558 2395 Acked Read 1 Sent Done Done false
5559 2396 Acked Read 6 Sent Done Done false
5560 2397 Acked Read 9 Sent Done Done false
5561 2398 Acked Write 5 Sent Done Done false
5562 2399 Acked Read 8 Sent Done Done false
5563 2400 Acked Read 1 Sent Done Done false
5564 2401 Acked Read 1 Sent Done Done false
5565 2402 Acked Read 7 Sent Done Done false
5566 2403 Acked Read 6 Sent Done Done false
5567 2404 Acked Read 4 Sent Done Done false
5568 2405 Acked Read 5 Sent Done Done false
5569 2406 Acked Write 3 Sent Done Done false
5570 2407 Acked Write 7 Sent Done Done false
5571 2408 Acked Read 6 Sent Done Done false
5572 2409 Acked Write 7 Sent Done Done false
5573 2410 Acked Read 10 Sent Done Done false
5574 2411 Acked Read 3 Sent Done Done false
5575 2412 Acked Read 5 Sent Done Done false
5576 2413 Acked Write 4 Sent Done Done false
5577 2414 Acked Write 5 Sent Done Done false
5578 2415 Acked Write 10 Sent Done Done false
5579 2416 Acked Write 7 Sent Done Done false
5580 2417 Acked Read 2 Sent Done Done false
5581 2418 Acked Read 2 Sent Done Done false
5582 2419 Acked Write 6 Sent Done Done false
5583 2420 Acked Read 3 Sent Done Done false
5584 2421 Acked Read 6 Sent Done Done false
5585 2422 Acked Write 8 Sent Done Done false
5586 2423 Acked Read 10 Sent Done Done false
5587 2424 Acked Write 4 Sent Done Done false
5588 2425 Acked Write 3 Sent Done Done false
5589 2426 Acked Read 4 Sent Done Done false
5590 2427 Acked Write 2 Sent Done Done false
5591 2428 Acked Write 4 Sent Done Done false
5592 2429 Acked Write 3 Sent Done Done false
5593 2430 Acked Read 10 Sent Done Done false
5594 2431 Acked Read 9 Sent Done Done false
5595 2432 Acked Write 8 Sent Done Done false
5596 2433 Acked Write 1 Sent Done Done false
5597 2434 Acked Read 6 Sent Done Done false
5598 2435 Acked Write 9 Sent Done Done false
5599 2436 Acked Write 3 Sent Done Done false
5600 2437 Acked Write 9 Sent Done Done false
5601 2438 Acked Write 2 Sent Done Done false
5602 2439 Acked Write 9 Sent Done Done false
5603 2440 Acked Write 1 Sent Done Done false
5604 2441 Acked Write 6 Sent Done Done false
5605 2442 Acked Read 4 Sent Done Done false
5606 2443 Acked Flush 0 Sent Done Done false
5607 2444 Acked Write 9 Sent Done Done false
5608 2445 Acked Read 10 Sent Done Done false
5609 2446 Acked Write 2 Sent Done Done false
5610 2447 Acked Read 8 Sent Done Done false
5611 2448 Acked Write 1 Sent Done Done false
5612 2449 Acked Read 1 Sent Done Done false
5613 2450 Acked Write 4 Sent Done Done false
5614 2451 Acked Read 9 Sent Done Done false
5615 2452 Acked Read 10 Sent Done Done false
5616 2453 Acked Write 7 Sent Done Done false
5617 2454 Acked Read 4 Sent Done Done false
5618 2455 Acked Flush 0 Sent Done Done false
5619 2456 Acked Read 9 Sent Done Done false
5620 2457 Acked Write 4 Sent Done Done false
5621 2458 Acked Read 8 Sent Done Done false
5622 2459 Acked Write 5 Sent Done Done false
5623 2460 Acked Flush 0 Sent Done Done false
5624 2461 Acked Read 9 Sent Done Done false
5625 2462 Acked Flush 0 Sent Done Done false
5626 2463 Acked Write 2 Sent Done Done false
5627 2464 Acked Write 7 Sent Done Done false
5628 2465 Acked Write 3 Sent Done Done false
5629 2466 Acked Read 7 Sent Done Done false
5630 2467 Acked Flush 0 Sent Done Done false
5631 2468 Acked Flush 0 Sent Done Done false
5632 2469 Acked Write 5 Sent Done Done false
5633 2470 Acked Write 1 Sent Done Done false
5634 2471 Acked Write 2 Sent Done Done false
5635 2472 Acked Read 7 Sent Done Done false
5636 2473 Acked Read 7 Sent Done Done false
5637 2474 Acked Write 10 Sent Done Done false
5638 2475 Acked Read 8 Sent Done Done false
5639 2476 Acked Write 1 Sent Done Done false
5640 2477 Acked Flush 0 Sent Done Done false
5641 2478 Acked Write 9 Sent Done Done false
5642 2479 Acked Read 7 Sent Done Done false
5643 2480 Acked Read 8 Sent Done Done false
5644 2481 Acked Write 2 Sent Done Done false
5645 2482 Acked Read 9 Sent Done Done false
5646 2483 Acked Write 9 Sent Done Done false
5647 2484 Acked Read 6 Sent Done Done false
5648 2485 Acked Read 6 Sent Done Done false
5649 2486 Acked Write 8 Sent Done Done false
5650 2487 Acked Write 6 Sent Done Done false
5651 2488 Acked Read 2 Sent Done Done false
5652 2489 Acked Flush 0 Sent Done Done false
5653 2490 Acked Read 4 Sent Done Done false
5654 2491 Acked Write 1 Sent Done Done false
5655 2492 Acked Flush 0 Sent Done Done false
5656 2493 Acked Read 4 Sent Done Done false
5657 2494 Acked Write 6 Sent Done Done false
5658 2495 Acked Read 9 Sent Done Done false
5659 2496 Acked Read 10 Sent Done Done false
5660 2497 Acked Write 4 Sent Done Done false
5661 2498 Acked Read 7 Sent Done Done false
5662 2499 Acked Write 8 Sent Done Done false
5663 2500 Acked Write 7 Sent Done Done false
5664 2501 Acked Read 6 Sent Done Done false
5665 2502 Acked Write 9 Sent Done Done false
5666 2503 Acked Read 8 Sent Done Done false
5667 2504 Acked Write 9 Sent Done Done false
5668 2505 Acked Write 3 Sent Done Done false
5669 2506 Acked Read 5 Sent Done Done false
5670 2507 Acked Flush 0 Sent Done Done false
5671 2508 Acked Write 9 Sent Done Done false
5672 2509 Acked Flush 0 Sent Done Done false
5673 STATES DS:0 DS:1 DS:2 TOTAL
5674 Sent 302 0 0 302
5675 Done 0 302 302 604
5676 Skipped 0 0 0 0
5677 Error 0 0 0 0
5678 Last Flush: 2207 2509 2509
5679 Downstairs last five completed: 2207 2206 2205 2204 2203
5680 Upstairs last five completed: 2509 2508 2507 2506 2505
5681 CLIENT: Up:0 ds:302 act:2
56822025-01-27T19:37:33.903ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
56832025-01-27T19:37:33.903ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43186) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
56842025-01-27T19:37:33.904ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
56852025-01-27T19:37:33.904ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
56862025-01-27T19:37:33.904ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
56872025-01-27T19:37:33.904ZINFOcrucible: send last flush ID to this DS: 2207 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
56882025-01-27T19:37:33.905ZINFOcrucible: Replied this last flush ID: 2207 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
56892025-01-27T19:37:33.905ZINFOcrucible: [0] client re-new 302 jobs since flush 2207 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
56902025-01-27T19:37:33.905ZINFOcrucible: [0] Replayed 302 jobs since flush: 2207 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
5691 ----------------------------------------------------------------
5692 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
5693 Downstairs last five completed: 2509 2508 2507 2506 2505
5694 Upstairs last five completed: 2509 2508 2507 2506 2505
5695 CLIENT: Up:0 ds:0 act:3
5696 Replay: All jobs finished
5697 Main task has work to do, go find it
5698 got dsc Stop(0)
5699 stop 0
5700 Jan 27 19:37:34.152 INFO [8810] Got stop action so:false kr:false
5701 request completed, latency_us: 354, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 909c229c-967c-46e0-b904-2fb5fdfd1032, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
57022025-01-27T19:37:34.153ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
57032025-01-27T19:37:34.153ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
57042025-01-27T19:37:34.153ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
57052025-01-27T19:37:34.153ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
5706 Jan 27 19:37:34.153 INFO request completed, latency_us: 180, response_code: 200, uri: /state/cid/0, method: GET, req_id: de9fe33f-328d-45bc-b2cb-6edfad43c2d2, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
5707 [8810] Exited with: ExitStatus(unix_wait_status(9))
5708 [8810][0] reports Exit
5709 Jan 27 19:37:38.156 INFO request completed, latency_us: 310, response_code: 200, uri: /state/cid/0, method: GET, req_id: 9b102583-1124-4240-a2db-594a0272657b, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
5710 001/300 Read block 4566 len 24576
5711 002/300 Read block 1011 len 12288
5712 003/300 Read block 357 len 16384
5713 004/300 Write block 4790 len 20480 data: 4 4 4 4 4
5714 005/300 Read block 609 len 36864
5715 006/300 Write block 3036 len 4096 data: 2
5716 007/300 Read block 4686 len 20480
5717 008/300 Read block 3167 len 20480
5718 009/300 Read block 251 len 28672
5719 010/300 Read block 1654 len 32768
5720 011/300 Read block 4784 len 36864
5721 012/300 Flush
5722 013/300 Write block 4523 len 8192 data: 3 3
5723 014/300 Read block 4292 len 28672
5724 015/300 Read block 2876 len 40960
5725 016/300 Write block 4993 len 24576 data: 2 2 3 3 3 3
5726 017/300 Read block 4375 len 20480
5727 018/300 Read block 3263 len 24576
5728 019/300 Read block 741 len 24576
5729 020/300 Read block 3609 len 24576
5730 021/300 Read block 3833 len 40960
5731 022/300 Read block 3383 len 24576
5732 023/300 Flush
5733 024/300 Read block 1393 len 28672
5734 025/300 Read block 1885 len 4096
5735 026/300 Write block 4512 len 12288 data: 3 3 4
5736 027/300 Read block 1933 len 12288
5737 028/300 Write block 4207 len 16384 data: 2 2 2 2
5738 029/300 Read block 1688 len 20480
5739 030/300 Read block 3401 len 40960
5740 031/300 Read block 43 len 40960
5741 032/300 Write block 443 len 4096 data: 3
5742 033/300 Write block 2574 len 28672 data: 2 2 3 3 3 3 3
5743 034/300 Read block 4270 len 40960
5744 035/300 Read block 273 len 12288
5745 036/300 Write block 3046 len 28672 data: 4 3 3 3 3 3 3
5746 037/300 Read block 4105 len 28672
5747 038/300 Write block 2353 len 16384 data: 4 4 3 3
5748 039/300 Read block 833 len 8192
5749 040/300 Read block 249 len 4096
5750 041/300 Write block 167 len 24576 data: 3 3 3 3 3 2
5751 042/300 Read block 3649 len 36864
5752 043/300 Write block 2470 len 28672 data: 3 2 2 3 3 3 3
5753 044/300 Read block 2906 len 36864
5754 045/300 Read block 422 len 28672
5755 046/300 Read block 3343 len 20480
5756 047/300 Flush
5757 048/300 Write block 3359 len 24576 data: 2 2 2 2 2 3
5758 049/300 Write block 351 len 24576 data: 3 3 3 3 3 3
5759 050/300 Write block 3367 len 32768 data: 3 3 3 3 3 4 3 3
5760 051/300 Write block 327 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5761 052/300 Write block 1597 len 40960 data: 3 3 3 3 3 3 3 2 2 2
5762 053/300 Read block 3625 len 16384
5763 054/300 Flush
5764 055/300 Write block 4453 len 4096 data: 2
5765 056/300 Write block 4960 len 12288 data: 4 4 4
5766 057/300 Flush
5767 058/300 Flush
5768 059/300 Write block 4224 len 12288 data: 2 2 2
5769 060/300 Write block 4009 len 20480 data: 3 3 2 2 3
5770 061/300 Read block 1215 len 36864
5771 062/300 Read block 4930 len 36864
5772 063/300 Read block 4817 len 24576
5773 064/300 Write block 2832 len 8192 data: 3 3
5774 065/300 Write block 2400 len 8192 data: 3 3
5775 066/300 Read block 2223 len 16384
5776 067/300 Read block 4039 len 28672
5777 068/300 Read block 3448 len 20480
5778 069/300 Write block 3735 len 24576 data: 3 3 3 2 2 2
5779 070/300 Write block 2751 len 4096 data: 3
5780 071/300 Write block 68 len 24576 data: 3 3 3 2 2 2
5781 072/300 Read block 3140 len 20480
5782 073/300 Write block 4427 len 32768 data: 2 2 2 2 2 2 2 2
5783 074/300 Flush
5784 075/300 Read block 1537 len 40960
5785 076/300 Write block 2885 len 36864 data: 2 2 2 2 2 2 2 2 2
5786 077/300 Flush
5787 078/300 Read block 2809 len 36864
5788 079/300 Read block 2862 len 4096
5789 080/300 Write block 432 len 4096 data: 2
5790 081/300 Flush
5791 082/300 Read block 308 len 20480
5792 083/300 Write block 2988 len 36864 data: 2 2 2 2 2 2 2 2 2
5793 084/300 Write block 638 len 20480 data: 2 2 2 2 2
5794 085/300 Read block 4683 len 16384
5795 086/300 Flush
5796 087/300 Write block 3974 len 4096 data: 2
5797 088/300 Flush
5798 089/300 Read block 4530 len 32768
5799 090/300 Write block 3519 len 36864 data: 3 3 3 3 3 3 3 3 2
5800 091/300 Write block 1470 len 4096 data: 4
5801 092/300 Write block 1886 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5802 093/300 Write block 2936 len 28672 data: 2 2 2 2 2 2 2
5803 094/300 Read block 3766 len 20480
5804 095/300 Read block 2066 len 16384
5805 096/300 Flush
5806 097/300 Flush
5807 098/300 Write block 873 len 4096 data: 4
5808 099/300 Write block 2173 len 32768 data: 2 2 2 2 2 2 2 2
5809 100/300 Read block 1977 len 4096
5810 101/300 Read block 2127 len 8192
5811 102/300 Read block 2267 len 8192
5812 103/300 Write block 3488 len 32768 data: 2 2 2 2 2 2 2 2
5813 104/300 Flush
5814 105/300 Read block 349 len 32768
5815 106/300 Write block 1471 len 4096 data: 4
5816 107/300 Write block 2369 len 8192 data: 3 3
5817 108/300 Write block 655 len 4096 data: 3
5818 109/300 Read block 1225 len 24576
5819 110/300 Write block 2412 len 36864 data: 3 3 3 3 3 2 2 2 2
5820 111/300 Write block 1862 len 20480 data: 3 2 2 2 2
5821 112/300 Read block 2355 len 4096
5822 113/300 Read block 3368 len 8192
5823 114/300 Read block 1926 len 4096
5824 115/300 Flush
5825 116/300 Write block 4215 len 8192 data: 2 2
5826 117/300 Write block 4156 len 24576 data: 2 2 2 2 3 3
5827 118/300 Read block 4178 len 16384
5828 119/300 Write block 3527 len 8192 data: 3 2
5829 120/300 Flush
5830 121/300 Read block 2289 len 16384
5831 122/300 Read block 1719 len 24576
5832 123/300 Read block 1638 len 20480
5833 124/300 Flush
5834 125/300 Write block 314 len 32768 data: 3 3 3 4 4 3 4 4
5835 126/300 Read block 2610 len 32768
5836 127/300 Read block 4175 len 28672
5837 128/300 Read block 4382 len 16384
5838 129/300 Read block 1875 len 36864
5839 130/300 Write block 497 len 4096 data: 3
5840 131/300 Write block 1591 len 32768 data: 2 2 2 2 2 2 4 4
5841 132/300 Write block 1129 len 16384 data: 3 3 3 3
5842 133/300 Read block 4185 len 4096
5843 134/300 Read block 757 len 24576
5844 135/300 Write block 4755 len 4096 data: 2
5845 136/300 Read block 87 len 16384
5846 137/300 Write block 574 len 40960 data: 2 3 3 3 3 3 3 3 3 3
5847 138/300 Read block 3352 len 16384
5848 139/300 Read block 3963 len 24576
5849 140/300 Flush
5850 141/300 Read block 453 len 8192
5851 142/300 Read block 4231 len 4096
5852 143/300 Flush
5853 144/300 Read block 2019 len 32768
5854 145/300 Read block 1274 len 12288
5855 146/300 Read block 3763 len 16384
5856 147/300 Read block 970 len 16384
5857 148/300 Read block 822 len 12288
5858 149/300 Read block 2574 len 24576
5859 150/300 Read block 3302 len 28672
5860 151/300 Read block 4812 len 40960
5861 152/300 Read block 124 len 40960
5862 153/300 Read block 3434 len 40960
5863 154/300 Write block 1090 len 8192 data: 2 2
5864 155/300 Write block 1708 len 4096 data: 2
5865 156/300 Read block 1484 len 4096
5866 157/300 Write block 3836 len 28672 data: 2 2 2 2 2 2 2
5867 158/300 Flush
5868 159/300 Read block 1724 len 16384
5869 160/300 Read block 4472 len 20480
5870 161/300 Write block 3197 len 24576 data: 2 2 2 2 2 2
5871 162/300 Read block 2306 len 12288
5872 163/300 Flush
5873 164/300 Read block 1188 len 32768
5874 165/300 Read block 346 len 20480
5875 166/300 Write block 559 len 20480 data: 3 3 3 3 3
5876 167/300 Write block 2190 len 40960 data: 2 2 2 3 3 3 3 3 3 3
5877 168/300 Write block 3242 len 8192 data: 3 3
5878 169/300 Flush
5879 170/300 Flush
5880 171/300 Write block 2175 len 32768 data: 3 3 3 3 3 3 3 3
5881 172/300 Read block 3265 len 4096
5882 173/300 Read block 987 len 28672
5883 174/300 Flush
5884 175/300 Write block 3841 len 12288 data: 3 3 2
5885 176/300 Read block 4509 len 8192
5886 177/300 Read block 1442 len 8192
5887 178/300 Write block 1111 len 32768 data: 3 2 2 2 2 2 2 2
5888 179/300 Write block 1627 len 12288 data: 3 3 3
5889 180/300 Read block 3298 len 16384
5890 181/300 Read block 115 len 36864
5891 182/300 Read block 3063 len 32768
5892 183/300 Write block 2512 len 24576 data: 2 2 2 2 2 2
5893 184/300 Write block 1222 len 12288 data: 4 5 5
5894 185/300 Read block 47 len 12288
5895 186/300 Read block 3936 len 24576
5896 187/300 Write block 2099 len 16384 data: 2 2 2 3
5897 188/300 Flush
5898 189/300 Read block 4997 len 12288
5899 190/300 Read block 1454 len 36864
5900 191/300 Read block 221 len 32768
5901 192/300 Read block 3283 len 24576
5902 193/300 Write block 128 len 24576 data: 3 2 2 3 2 2
5903 194/300 Flush
5904 195/300 Write block 928 len 28672 data: 2 2 3 3 3 3 2
5905 196/300 Write block 4027 len 4096 data: 2
5906 197/300 Write block 4486 len 12288 data: 2 2 2
5907 198/300 Read block 2891 len 8192
5908 199/300 Read block 797 len 24576
5909 200/300 Flush
5910 201/300 Write block 1870 len 40960 data: 2 2 2 2 2 2 2 2 2 2
5911 202/300 Flush
5912 203/300 Flush
5913 204/300 Flush
5914 205/300 Read block 1341 len 12288
5915 206/300 Write block 2710 len 12288 data: 3 2 2
5916 207/300 Read block 2126 len 12288
5917 208/300 Read block 1070 len 28672
5918 209/300 Read block 1267 len 36864
5919 210/300 Read block 576 len 12288
5920 211/300 Read block 2695 len 4096
5921 212/300 Read block 1087 len 24576
5922 213/300 Read block 1700 len 32768
5923 214/300 Write block 1475 len 32768 data: 3 3 2 3 3 4 4 4
5924 215/300 Read block 2154 len 28672
5925 216/300 Write block 4010 len 16384 data: 4 3 3 4
5926 217/300 Read block 4264 len 8192
5927 218/300 Write block 3893 len 40960 data: 2 2 2 3 3 3 3 2 2 3
5928 219/300 Read block 4672 len 12288
5929 220/300 Read block 1006 len 24576
5930 221/300 Read block 279 len 32768
5931 222/300 Write block 4059 len 4096 data: 2
5932 223/300 Write block 689 len 4096 data: 2
5933 224/300 Read block 820 len 28672
5934 225/300 Write block 4922 len 28672 data: 4 4 3 3 3 3 3
5935 226/300 Read block 2132 len 24576
5936 227/300 Write block 2506 len 16384 data: 2 2 2 2
5937 228/300 Read block 4119 len 40960
5938 229/300 Read block 742 len 4096
5939 230/300 Write block 3956 len 32768 data: 3 3 3 3 2 2 2 2
5940 231/300 Read block 2981 len 20480
5941 232/300 Write block 1803 len 32768 data: 2 2 2 2 2 2 2 4
5942 233/300 Write block 3708 len 16384 data: 3 3 3 3
5943 234/300 Write block 2331 len 40960 data: 2 2 2 2 3 3 3 3 4 4
5944 235/300 Write block 86 len 20480 data: 2 2 3 2 2
5945 236/300 Flush
5946 237/300 Write block 2375 len 24576 data: 3 3 3 4 3 3
5947 238/300 Read block 4392 len 28672
5948 239/300 Write block 1298 len 40960 data: 2 2 2 3 3 4 4 3 4 4
5949 240/300 Read block 1160 len 20480
5950 241/300 Read block 2652 len 24576
5951 242/300 Write block 2934 len 16384 data: 2 2 3 3
5952 243/300 Write block 2154 len 40960 data: 2 2 2 2 2 2 2 3 4 4
5953 244/300 Write block 1557 len 12288 data: 3 3 3
5954 245/300 Read block 2442 len 4096
5955 246/300 Read block 3714 len 40960
5956 247/300 Write block 817 len 36864 data: 2 2 2 3 2 2 2 2 2
5957 248/300 Flush
5958 249/300 Read block 292 len 36864
5959 250/300 Write block 3518 len 20480 data: 3 4 4 4 4
5960 251/300 Read block 2584 len 32768
5961 252/300 Read block 418 len 8192
5962 253/300 Write block 583 len 4096 data: 4
5963 254/300 Write block 4615 len 20480 data: 2 2 2 2 2
5964 255/300 Read block 2497 len 4096
5965 256/300 Read block 1140 len 40960
5966 257/300 Read block 825 len 8192
5967 258/300 Read block 2274 len 40960
5968 259/300 Write block 2414 len 4096 data: 4
5969 260/300 Write block 4867 len 24576 data: 2 2 2 2 2 2
5970 261/300 Read block 3559 len 8192
5971 262/300 Write block 2629 len 4096 data: 2
5972 263/300 Write block 1569 len 20480 data: 2 2 3 3 3
5973 264/300 Read block 2498 len 36864
5974 265/300 Write block 3633 len 16384 data: 2 2 3 3
5975 266/300 Write block 3129 len 28672 data: 3 4 4 4 4 3 4
5976 267/300 Write block 708 len 4096 data: 2
5977 268/300 Read block 2805 len 20480
5978 269/300 Write block 3568 len 16384 data: 2 3 3 3
5979 270/300 Write block 46 len 8192 data: 3 3
5980 271/300 Read block 2478 len 16384
5981 272/300 Read block 3669 len 12288
5982 273/300 Read block 2989 len 24576
5983 274/300 Write block 681 len 16384 data: 2 2 2 2
5984 275/300 Read block 1484 len 32768
5985 276/300 Read block 680 len 32768
5986 277/300 Read block 4111 len 24576
5987 278/300 Read block 3508 len 36864
5988 279/300 Read block 2447 len 28672
5989 280/300 Read block 1041 len 32768
5990 281/300 Flush
5991 282/300 Write block 173 len 20480 data: 2 2 2 2 2
5992 283/300 Write block 2944 len 12288 data: 2 2 2
5993 284/300 Read block 2913 len 16384
5994 285/300 Write block 2979 len 40960 data: 5 5 5 4 4 3 3 3 3 3
5995 286/300 Read block 2607 len 8192
5996 287/300 Write block 1754 len 36864 data: 2 2 2 2 2 2 2 2 2
5997 288/300 Read block 2064 len 24576
5998 289/300 Read block 4079 len 36864
5999 290/300 Write block 3836 len 20480 data: 3 3 3 3 3
6000 291/300 Read block 2321 len 20480
6001 292/300 Write block 4838 len 36864 data: 2 2 2 2 2 2 2 2 2
6002 293/300 Write block 2679 len 24576 data: 4 4 4 4 4 4
6003 294/300 Read block 932 len 16384
6004 295/300 Flush
6005 296/300 Write block 4065 len 36864 data: 2 2 2 2 2 2 2 2 2
6006 297/300 Read block 4503 len 40960
6007 298/300 Write block 3280 len 16384 data: 2 2 2 2
6008 299/300 Read block 317 len 36864
6009 300/300 Write block 4265 len 28672 data: 3 3 2 2 2 2 4
6010 301/300 Flush
6011 Main task has work to do, go find it
6012 got dsc Start(0)
6013 start 0
6014 Jan 27 19:37:38.394 INFO request completed, latency_us: 207, response_code: 204, uri: /start/cid/0, method: POST, req_id: 781ce18e-daaf-485b-96cf-83f7856313c6, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
6015 Replay: started 0, returned:Ok(())
6016 ----------------------------------------------------------------
6017 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6018 States: Offline Active Active
6019 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6020 2510 Acked Read 6 Sent Done Done false
6021 2511 Acked Read 3 Sent Done Done false
6022 2512 Acked Read 4 Sent Done Done false
6023 2513 Acked Write 5 Sent Done Done false
6024 2514 Acked Read 9 Sent Done Done false
6025 2515 Acked Write 1 Sent Done Done false
6026 2516 Acked Read 5 Sent Done Done false
6027 2517 Acked Read 5 Sent Done Done false
6028 2518 Acked Read 7 Sent Done Done false
6029 2519 Acked Read 8 Sent Done Done false
6030 2520 Acked Read 9 Sent Done Done false
6031 2521 Acked Flush 0 Sent Done Done false
6032 2522 Acked Write 2 Sent Done Done false
6033 2523 Acked Read 7 Sent Done Done false
6034 2524 Acked Read 10 Sent Done Done false
6035 2525 Acked Write 6 Sent Done Done false
6036 2526 Acked Read 5 Sent Done Done false
6037 2527 Acked Read 6 Sent Done Done false
6038 2528 Acked Read 6 Sent Done Done false
6039 2529 Acked Read 6 Sent Done Done false
6040 2530 Acked Read 10 Sent Done Done false
6041 2531 Acked Read 6 Sent Done Done false
6042 2532 Acked Flush 0 Sent Done Done false
6043 2533 Acked Read 7 Sent Done Done false
6044 2534 Acked Read 1 Sent Done Done false
6045 2535 Acked Write 3 Sent Done Done false
6046 2536 Acked Read 3 Sent Done Done false
6047 2537 Acked Write 4 Sent Done Done false
6048 2538 Acked Read 5 Sent Done Done false
6049 2539 Acked Read 10 Sent Done Done false
6050 2540 Acked Read 10 Sent Done Done false
6051 2541 Acked Write 1 Sent Done Done false
6052 2542 Acked Write 7 Sent Done Done false
6053 2543 Acked Read 10 Sent Done Done false
6054 2544 Acked Read 3 Sent Done Done false
6055 2545 Acked Write 7 Sent Done Done false
6056 2546 Acked Read 7 Sent Done Done false
6057 2547 Acked Write 4 Sent Done Done false
6058 2548 Acked Read 2 Sent Done Done false
6059 2549 Acked Read 1 Sent Done Done false
6060 2550 Acked Write 6 Sent Done Done false
6061 2551 Acked Read 9 Sent Done Done false
6062 2552 Acked Write 7 Sent Done Done false
6063 2553 Acked Read 9 Sent Done Done false
6064 2554 Acked Read 7 Sent Done Done false
6065 2555 Acked Read 5 Sent Done Done false
6066 2556 Acked Flush 0 Sent Done Done false
6067 2557 Acked Write 6 Sent Done Done false
6068 2558 Acked Write 6 Sent Done Done false
6069 2559 Acked Write 8 Sent Done Done false
6070 2560 Acked Write 10 Sent Done Done false
6071 2561 Acked Write 10 Sent Done Done false
6072 2562 Acked Read 4 Sent Done Done false
6073 2563 Acked Flush 0 Sent Done Done false
6074 2564 Acked Write 1 Sent Done Done false
6075 2565 Acked Write 3 Sent Done Done false
6076 2566 Acked Flush 0 Sent Done Done false
6077 2567 Acked Flush 0 Sent Done Done false
6078 2568 Acked Write 3 Sent Done Done false
6079 2569 Acked Write 5 Sent Done Done false
6080 2570 Acked Read 9 Sent Done Done false
6081 2571 Acked Read 9 Sent Done Done false
6082 2572 Acked Read 6 Sent Done Done false
6083 2573 Acked Write 2 Sent Done Done false
6084 2574 Acked Write 2 Sent Done Done false
6085 2575 Acked Read 4 Sent Done Done false
6086 2576 Acked Read 7 Sent Done Done false
6087 2577 Acked Read 5 Sent Done Done false
6088 2578 Acked Write 6 Sent Done Done false
6089 2579 Acked Write 1 Sent Done Done false
6090 2580 Acked Write 6 Sent Done Done false
6091 2581 Acked Read 5 Sent Done Done false
6092 2582 Acked Write 8 Sent Done Done false
6093 2583 Acked Flush 0 Sent Done Done false
6094 2584 Acked Read 10 Sent Done Done false
6095 2585 Acked Write 9 Sent Done Done false
6096 2586 Acked Flush 0 Sent Done Done false
6097 2587 Acked Read 9 Sent Done Done false
6098 2588 Acked Read 1 Sent Done Done false
6099 2589 Acked Write 1 Sent Done Done false
6100 2590 Acked Flush 0 Sent Done Done false
6101 2591 Acked Read 5 Sent Done Done false
6102 2592 Acked Write 9 Sent Done Done false
6103 2593 Acked Write 5 Sent Done Done false
6104 2594 Acked Read 4 Sent Done Done false
6105 2595 Acked Flush 0 Sent Done Done false
6106 2596 Acked Write 1 Sent Done Done false
6107 2597 Acked Flush 0 Sent Done Done false
6108 2598 Acked Read 8 Sent Done Done false
6109 2599 Acked Write 9 Sent Done Done false
6110 2600 Acked Write 1 Sent Done Done false
6111 2601 Acked Write 10 Sent Done Done false
6112 2602 Acked Write 7 Sent Done Done false
6113 2603 Acked Read 5 Sent Done Done false
6114 2604 Acked Read 4 Sent Done Done false
6115 2605 Acked Flush 0 Sent Done Done false
6116 2606 Acked Flush 0 Sent Done Done false
6117 2607 Acked Write 1 Sent Done Done false
6118 2608 Acked Write 8 Sent Done Done false
6119 2609 Acked Read 1 Sent Done Done false
6120 2610 Acked Read 2 Sent Done Done false
6121 2611 Acked Read 2 Sent Done Done false
6122 2612 Acked Write 8 Sent Done Done false
6123 2613 Acked Flush 0 Sent Done Done false
6124 2614 Acked Read 8 Sent Done Done false
6125 2615 Acked Write 1 Sent Done Done false
6126 2616 Acked Write 2 Sent Done Done false
6127 2617 Acked Write 1 Sent Done Done false
6128 2618 Acked Read 6 Sent Done Done false
6129 2619 Acked Write 9 Sent Done Done false
6130 2620 Acked Write 5 Sent Done Done false
6131 2621 Acked Read 1 Sent Done Done false
6132 2622 Acked Read 2 Sent Done Done false
6133 2623 Acked Read 1 Sent Done Done false
6134 2624 Acked Flush 0 Sent Done Done false
6135 2625 Acked Write 2 Sent Done Done false
6136 2626 Acked Write 6 Sent Done Done false
6137 2627 Acked Read 4 Sent Done Done false
6138 2628 Acked Write 2 Sent Done Done false
6139 2629 Acked Flush 0 Sent Done Done false
6140 2630 Acked Read 4 Sent Done Done false
6141 2631 Acked Read 6 Sent Done Done false
6142 2632 Acked Read 5 Sent Done Done false
6143 2633 Acked Flush 0 Sent Done Done false
6144 2634 Acked Write 8 Sent Done Done false
6145 2635 Acked Read 8 Sent Done Done false
6146 2636 Acked Read 7 Sent Done Done false
6147 2637 Acked Read 4 Sent Done Done false
6148 2638 Acked Read 9 Sent Done Done false
6149 2639 Acked Write 1 Sent Done Done false
6150 2640 Acked Write 8 Sent Done Done false
6151 2641 Acked Write 4 Sent Done Done false
6152 2642 Acked Read 1 Sent Done Done false
6153 2643 Acked Read 6 Sent Done Done false
6154 2644 Acked Write 1 Sent Done Done false
6155 2645 Acked Read 4 Sent Done Done false
6156 2646 Acked Write 10 Sent Done Done false
6157 2647 Acked Read 4 Sent Done Done false
6158 2648 Acked Read 6 Sent Done Done false
6159 2649 Acked Flush 0 Sent Done Done false
6160 2650 Acked Read 2 Sent Done Done false
6161 2651 Acked Read 1 Sent Done Done false
6162 2652 Acked Flush 0 Sent Done Done false
6163 2653 Acked Read 8 Sent Done Done false
6164 2654 Acked Read 3 Sent Done Done false
6165 2655 Acked Read 4 Sent Done Done false
6166 2656 Acked Read 4 Sent Done Done false
6167 2657 Acked Read 3 Sent Done Done false
6168 2658 Acked Read 6 Sent Done Done false
6169 2659 Acked Read 7 Sent Done Done false
6170 2660 Acked Read 10 Sent Done Done false
6171 2661 Acked Read 10 Sent Done Done false
6172 2662 Acked Read 10 Sent Done Done false
6173 2663 Acked Write 2 Sent Done Done false
6174 2664 Acked Write 1 Sent Done Done false
6175 2665 Acked Read 1 Sent Done Done false
6176 2666 Acked Write 7 Sent Done Done false
6177 2667 Acked Flush 0 Sent Done Done false
6178 2668 Acked Read 4 Sent Done Done false
6179 2669 Acked Read 5 Sent Done Done false
6180 2670 Acked Write 6 Sent Done Done false
6181 2671 Acked Read 3 Sent Done Done false
6182 2672 Acked Flush 0 Sent Done Done false
6183 2673 Acked Read 8 Sent Done Done false
6184 2674 Acked Read 5 Sent Done Done false
6185 2675 Acked Write 5 Sent Done Done false
6186 2676 Acked Write 10 Sent Done Done false
6187 2677 Acked Write 2 Sent Done Done false
6188 2678 Acked Flush 0 Sent Done Done false
6189 2679 Acked Flush 0 Sent Done Done false
6190 2680 Acked Write 8 Sent Done Done false
6191 2681 Acked Read 1 Sent Done Done false
6192 2682 Acked Read 7 Sent Done Done false
6193 2683 Acked Flush 0 Sent Done Done false
6194 2684 Acked Write 3 Sent Done Done false
6195 2685 Acked Read 2 Sent Done Done false
6196 2686 Acked Read 2 Sent Done Done false
6197 2687 Acked Write 8 Sent Done Done false
6198 2688 Acked Write 3 Sent Done Done false
6199 2689 Acked Read 4 Sent Done Done false
6200 2690 Acked Read 9 Sent Done Done false
6201 2691 Acked Read 8 Sent Done Done false
6202 2692 Acked Write 6 Sent Done Done false
6203 2693 Acked Write 3 Sent Done Done false
6204 2694 Acked Read 3 Sent Done Done false
6205 2695 Acked Read 6 Sent Done Done false
6206 2696 Acked Write 4 Sent Done Done false
6207 2697 Acked Flush 0 Sent Done Done false
6208 2698 Acked Read 3 Sent Done Done false
6209 2699 Acked Read 9 Sent Done Done false
6210 2700 Acked Read 8 Sent Done Done false
6211 2701 Acked Read 6 Sent Done Done false
6212 2702 Acked Write 6 Sent Done Done false
6213 2703 Acked Flush 0 Sent Done Done false
6214 2704 Acked Write 7 Sent Done Done false
6215 2705 Acked Write 1 Sent Done Done false
6216 2706 Acked Write 3 Sent Done Done false
6217 2707 Acked Read 2 Sent Done Done false
6218 2708 Acked Read 6 Sent Done Done false
6219 2709 Acked Flush 0 Sent Done Done false
6220 2710 Acked Write 10 Sent Done Done false
6221 2711 Acked Flush 0 Sent Done Done false
6222 2712 Acked Flush 0 Sent Done Done false
6223 2713 Acked Flush 0 Sent Done Done false
6224 2714 Acked Read 3 Sent Done Done false
6225 2715 Acked Write 3 Sent Done Done false
6226 2716 Acked Read 3 Sent Done Done false
6227 2717 Acked Read 7 Sent Done Done false
6228 2718 Acked Read 9 Sent Done Done false
6229 2719 Acked Read 3 Sent Done Done false
6230 2720 Acked Read 1 Sent Done Done false
6231 2721 Acked Read 6 Sent Done Done false
6232 2722 Acked Read 8 Sent Done Done false
6233 2723 Acked Write 8 Sent Done Done false
6234 2724 Acked Read 7 Sent Done Done false
6235 2725 Acked Write 4 Sent Done Done false
6236 2726 Acked Read 2 Sent Done Done false
6237 2727 Acked Write 10 Sent Done Done false
6238 2728 Acked Read 3 Sent Done Done false
6239 2729 Acked Read 6 Sent Done Done false
6240 2730 Acked Read 8 Sent Done Done false
6241 2731 Acked Write 1 Sent Done Done false
6242 2732 Acked Write 1 Sent Done Done false
6243 2733 Acked Read 7 Sent Done Done false
6244 2734 Acked Write 7 Sent Done Done false
6245 2735 Acked Read 6 Sent Done Done false
6246 2736 Acked Write 4 Sent Done Done false
6247 2737 Acked Read 10 Sent Done Done false
6248 2738 Acked Read 1 Sent Done Done false
6249 2739 Acked Write 8 Sent Done Done false
6250 2740 Acked Read 5 Sent Done Done false
6251 2741 Acked Write 8 Sent Done Done false
6252 2742 Acked Write 4 Sent Done Done false
6253 2743 Acked Write 10 Sent Done Done false
6254 2744 Acked Write 5 Sent Done Done false
6255 2745 Acked Flush 0 Sent Done Done false
6256 2746 Acked Write 6 Sent Done Done false
6257 2747 Acked Read 7 Sent Done Done false
6258 2748 Acked Write 10 Sent Done Done false
6259 2749 Acked Read 5 Sent Done Done false
6260 2750 Acked Read 6 Sent Done Done false
6261 2751 Acked Write 4 Sent Done Done false
6262 2752 Acked Write 10 Sent Done Done false
6263 2753 Acked Write 3 Sent Done Done false
6264 2754 Acked Read 1 Sent Done Done false
6265 2755 Acked Read 10 Sent Done Done false
6266 2756 Acked Write 9 Sent Done Done false
6267 2757 Acked Flush 0 Sent Done Done false
6268 2758 Acked Read 9 Sent Done Done false
6269 2759 Acked Write 5 Sent Done Done false
6270 2760 Acked Read 8 Sent Done Done false
6271 2761 Acked Read 2 Sent Done Done false
6272 2762 Acked Write 1 Sent Done Done false
6273 2763 Acked Write 5 Sent Done Done false
6274 2764 Acked Read 1 Sent Done Done false
6275 2765 Acked Read 10 Sent Done Done false
6276 2766 Acked Read 2 Sent Done Done false
6277 2767 Acked Read 10 Sent Done Done false
6278 2768 Acked Write 1 Sent Done Done false
6279 2769 Acked Write 6 Sent Done Done false
6280 2770 Acked Read 2 Sent Done Done false
6281 2771 Acked Write 1 Sent Done Done false
6282 2772 Acked Write 5 Sent Done Done false
6283 2773 Acked Read 9 Sent Done Done false
6284 2774 Acked Write 4 Sent Done Done false
6285 2775 Acked Write 7 Sent Done Done false
6286 2776 Acked Write 1 Sent Done Done false
6287 2777 Acked Read 5 Sent Done Done false
6288 2778 Acked Write 4 Sent Done Done false
6289 2779 Acked Write 2 Sent Done Done false
6290 2780 Acked Read 4 Sent Done Done false
6291 2781 Acked Read 3 Sent Done Done false
6292 2782 Acked Read 6 Sent Done Done false
6293 2783 Acked Write 4 Sent Done Done false
6294 2784 Acked Read 8 Sent Done Done false
6295 2785 Acked Read 8 Sent Done Done false
6296 2786 Acked Read 6 Sent Done Done false
6297 2787 Acked Read 9 Sent Done Done false
6298 2788 Acked Read 7 Sent Done Done false
6299 2789 Acked Read 8 Sent Done Done false
6300 2790 Acked Flush 0 Sent Done Done false
6301 2791 Acked Write 5 Sent Done Done false
6302 2792 Acked Write 3 Sent Done Done false
6303 2793 Acked Read 4 Sent Done Done false
6304 2794 Acked Write 10 Sent Done Done false
6305 2795 Acked Read 2 Sent Done Done false
6306 2796 Acked Write 9 Sent Done Done false
6307 2797 Acked Read 6 Sent Done Done false
6308 2798 Acked Read 9 Sent Done Done false
6309 2799 Acked Write 5 Sent Done Done false
6310 2800 Acked Read 5 Sent Done Done false
6311 2801 Acked Write 9 Sent Done Done false
6312 2802 Acked Write 6 Sent Done Done false
6313 2803 Acked Read 4 Sent Done Done false
6314 2804 Acked Flush 0 Sent Done Done false
6315 2805 Acked Write 9 Sent Done Done false
6316 2806 Acked Read 10 Sent Done Done false
6317 2807 Acked Write 4 Sent Done Done false
6318 2808 Acked Read 9 Sent Done Done false
6319 2809 Acked Write 7 Sent Done Done false
6320 2810 Acked Flush 0 Sent Done Done false
6321 STATES DS:0 DS:1 DS:2 TOTAL
6322 Sent 301 0 0 301
6323 Done 0 301 301 602
6324 Skipped 0 0 0 0
6325 Error 0 0 0 0
6326 Last Flush: 2509 2810 2810
6327 Downstairs last five completed: 2509 2508 2507 2506 2505
6328 Upstairs last five completed: 2810 2809 2808 2807 2806
6329 CLIENT: Up:0 ds:301 act:2
6330 [8810] got start action
6331 [8810] I am going to restart
6332 Starting downstairs at port 8810
6333 Make output file at "/tmp/dsc/downstairs-8810.txt"
6334 [8810][0] reports Starting
6335 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1206)
6336 [8810][0] reports Running
6337 ----------------------------------------------------------------
6338 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6339 States: Offline Active Active
6340 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6341 2510 Acked Read 6 Sent Done Done false
6342 2511 Acked Read 3 Sent Done Done false
6343 2512 Acked Read 4 Sent Done Done false
6344 2513 Acked Write 5 Sent Done Done false
6345 2514 Acked Read 9 Sent Done Done false
6346 2515 Acked Write 1 Sent Done Done false
6347 2516 Acked Read 5 Sent Done Done false
6348 2517 Acked Read 5 Sent Done Done false
6349 2518 Acked Read 7 Sent Done Done false
6350 2519 Acked Read 8 Sent Done Done false
6351 2520 Acked Read 9 Sent Done Done false
6352 2521 Acked Flush 0 Sent Done Done false
6353 2522 Acked Write 2 Sent Done Done false
6354 2523 Acked Read 7 Sent Done Done false
6355 2524 Acked Read 10 Sent Done Done false
6356 2525 Acked Write 6 Sent Done Done false
6357 2526 Acked Read 5 Sent Done Done false
6358 2527 Acked Read 6 Sent Done Done false
6359 2528 Acked Read 6 Sent Done Done false
6360 2529 Acked Read 6 Sent Done Done false
6361 2530 Acked Read 10 Sent Done Done false
6362 2531 Acked Read 6 Sent Done Done false
6363 2532 Acked Flush 0 Sent Done Done false
6364 2533 Acked Read 7 Sent Done Done false
6365 2534 Acked Read 1 Sent Done Done false
6366 2535 Acked Write 3 Sent Done Done false
6367 2536 Acked Read 3 Sent Done Done false
6368 2537 Acked Write 4 Sent Done Done false
6369 2538 Acked Read 5 Sent Done Done false
6370 2539 Acked Read 10 Sent Done Done false
6371 2540 Acked Read 10 Sent Done Done false
6372 2541 Acked Write 1 Sent Done Done false
6373 2542 Acked Write 7 Sent Done Done false
6374 2543 Acked Read 10 Sent Done Done false
6375 2544 Acked Read 3 Sent Done Done false
6376 2545 Acked Write 7 Sent Done Done false
6377 2546 Acked Read 7 Sent Done Done false
6378 2547 Acked Write 4 Sent Done Done false
6379 2548 Acked Read 2 Sent Done Done false
6380 2549 Acked Read 1 Sent Done Done false
6381 2550 Acked Write 6 Sent Done Done false
6382 2551 Acked Read 9 Sent Done Done false
6383 2552 Acked Write 7 Sent Done Done false
6384 2553 Acked Read 9 Sent Done Done false
6385 2554 Acked Read 7 Sent Done Done false
6386 2555 Acked Read 5 Sent Done Done false
6387 2556 Acked Flush 0 Sent Done Done false
6388 2557 Acked Write 6 Sent Done Done false
6389 2558 Acked Write 6 Sent Done Done false
6390 2559 Acked Write 8 Sent Done Done false
6391 2560 Acked Write 10 Sent Done Done false
6392 2561 Acked Write 10 Sent Done Done false
6393 2562 Acked Read 4 Sent Done Done false
6394 2563 Acked Flush 0 Sent Done Done false
6395 2564 Acked Write 1 Sent Done Done false
6396 2565 Acked Write 3 Sent Done Done false
6397 2566 Acked Flush 0 Sent Done Done false
6398 2567 Acked Flush 0 Sent Done Done false
6399 2568 Acked Write 3 Sent Done Done false
6400 2569 Acked Write 5 Sent Done Done false
6401 2570 Acked Read 9 Sent Done Done false
6402 2571 Acked Read 9 Sent Done Done false
6403 2572 Acked Read 6 Sent Done Done false
6404 2573 Acked Write 2 Sent Done Done false
6405 2574 Acked Write 2 Sent Done Done false
6406 2575 Acked Read 4 Sent Done Done false
6407 2576 Acked Read 7 Sent Done Done false
6408 2577 Acked Read 5 Sent Done Done false
6409 2578 Acked Write 6 Sent Done Done false
6410 2579 Acked Write 1 Sent Done Done false
6411 2580 Acked Write 6 Sent Done Done false
6412 2581 Acked Read 5 Sent Done Done false
6413 2582 Acked Write 8 Sent Done Done false
6414 2583 Acked Flush 0 Sent Done Done false
6415 2584 Acked Read 10 Sent Done Done false
6416 2585 Acked Write 9 Sent Done Done false
6417 2586 Acked Flush 0 Sent Done Done false
6418 2587 Acked Read 9 Sent Done Done false
6419 2588 Acked Read 1 Sent Done Done false
6420 2589 Acked Write 1 Sent Done Done false
6421 2590 Acked Flush 0 Sent Done Done false
6422 2591 Acked Read 5 Sent Done Done false
6423 2592 Acked Write 9 Sent Done Done false
6424 2593 Acked Write 5 Sent Done Done false
6425 2594 Acked Read 4 Sent Done Done false
6426 2595 Acked Flush 0 Sent Done Done false
6427 2596 Acked Write 1 Sent Done Done false
6428 2597 Acked Flush 0 Sent Done Done false
6429 2598 Acked Read 8 Sent Done Done false
6430 2599 Acked Write 9 Sent Done Done false
6431 2600 Acked Write 1 Sent Done Done false
6432 2601 Acked Write 10 Sent Done Done false
6433 2602 Acked Write 7 Sent Done Done false
6434 2603 Acked Read 5 Sent Done Done false
6435 2604 Acked Read 4 Sent Done Done false
6436 2605 Acked Flush 0 Sent Done Done false
6437 2606 Acked Flush 0 Sent Done Done false
6438 2607 Acked Write 1 Sent Done Done false
6439 2608 Acked Write 8 Sent Done Done false
6440 2609 Acked Read 1 Sent Done Done false
6441 2610 Acked Read 2 Sent Done Done false
6442 2611 Acked Read 2 Sent Done Done false
6443 2612 Acked Write 8 Sent Done Done false
6444 2613 Acked Flush 0 Sent Done Done false
6445 2614 Acked Read 8 Sent Done Done false
6446 2615 Acked Write 1 Sent Done Done false
6447 2616 Acked Write 2 Sent Done Done false
6448 2617 Acked Write 1 Sent Done Done false
6449 2618 Acked Read 6 Sent Done Done false
6450 2619 Acked Write 9 Sent Done Done false
6451 2620 Acked Write 5 Sent Done Done false
6452 2621 Acked Read 1 Sent Done Done false
6453 2622 Acked Read 2 Sent Done Done false
6454 2623 Acked Read 1 Sent Done Done false
6455 2624 Acked Flush 0 Sent Done Done false
6456 2625 Acked Write 2 Sent Done Done false
6457 2626 Acked Write 6 Sent Done Done false
6458 2627 Acked Read 4 Sent Done Done false
6459 2628 Acked Write 2 Sent Done Done false
6460 2629 Acked Flush 0 Sent Done Done false
6461 2630 Acked Read 4 Sent Done Done false
6462 2631 Acked Read 6 Sent Done Done false
6463 2632 Acked Read 5 Sent Done Done false
6464 2633 Acked Flush 0 Sent Done Done false
6465 2634 Acked Write 8 Sent Done Done false
6466 2635 Acked Read 8 Sent Done Done false
6467 2636 Acked Read 7 Sent Done Done false
6468 2637 Acked Read 4 Sent Done Done false
6469 2638 Acked Read 9 Sent Done Done false
6470 2639 Acked Write 1 Sent Done Done false
6471 2640 Acked Write 8 Sent Done Done false
6472 2641 Acked Write 4 Sent Done Done false
6473 2642 Acked Read 1 Sent Done Done false
6474 2643 Acked Read 6 Sent Done Done false
6475 2644 Acked Write 1 Sent Done Done false
6476 2645 Acked Read 4 Sent Done Done false
6477 2646 Acked Write 10 Sent Done Done false
6478 2647 Acked Read 4 Sent Done Done false
6479 2648 Acked Read 6 Sent Done Done false
6480 2649 Acked Flush 0 Sent Done Done false
6481 2650 Acked Read 2 Sent Done Done false
6482 2651 Acked Read 1 Sent Done Done false
6483 2652 Acked Flush 0 Sent Done Done false
6484 2653 Acked Read 8 Sent Done Done false
6485 2654 Acked Read 3 Sent Done Done false
6486 2655 Acked Read 4 Sent Done Done false
6487 2656 Acked Read 4 Sent Done Done false
6488 2657 Acked Read 3 Sent Done Done false
6489 2658 Acked Read 6 Sent Done Done false
6490 2659 Acked Read 7 Sent Done Done false
6491 2660 Acked Read 10 Sent Done Done false
6492 2661 Acked Read 10 Sent Done Done false
6493 2662 Acked Read 10 Sent Done Done false
6494 2663 Acked Write 2 Sent Done Done false
6495 2664 Acked Write 1 Sent Done Done false
6496 2665 Acked Read 1 Sent Done Done false
6497 2666 Acked Write 7 Sent Done Done false
6498 2667 Acked Flush 0 Sent Done Done false
6499 2668 Acked Read 4 Sent Done Done false
6500 2669 Acked Read 5 Sent Done Done false
6501 2670 Acked Write 6 Sent Done Done false
6502 2671 Acked Read 3 Sent Done Done false
6503 2672 Acked Flush 0 Sent Done Done false
6504 2673 Acked Read 8 Sent Done Done false
6505 2674 Acked Read 5 Sent Done Done false
6506 2675 Acked Write 5 Sent Done Done false
6507 2676 Acked Write 10 Sent Done Done false
6508 2677 Acked Write 2 Sent Done Done false
6509 2678 Acked Flush 0 Sent Done Done false
6510 2679 Acked Flush 0 Sent Done Done false
6511 2680 Acked Write 8 Sent Done Done false
6512 2681 Acked Read 1 Sent Done Done false
6513 2682 Acked Read 7 Sent Done Done false
6514 2683 Acked Flush 0 Sent Done Done false
6515 2684 Acked Write 3 Sent Done Done false
6516 2685 Acked Read 2 Sent Done Done false
6517 2686 Acked Read 2 Sent Done Done false
6518 2687 Acked Write 8 Sent Done Done false
6519 2688 Acked Write 3 Sent Done Done false
6520 2689 Acked Read 4 Sent Done Done false
6521 2690 Acked Read 9 Sent Done Done false
6522 2691 Acked Read 8 Sent Done Done false
6523 2692 Acked Write 6 Sent Done Done false
6524 2693 Acked Write 3 Sent Done Done false
6525 2694 Acked Read 3 Sent Done Done false
6526 2695 Acked Read 6 Sent Done Done false
6527 2696 Acked Write 4 Sent Done Done false
6528 2697 Acked Flush 0 Sent Done Done false
6529 2698 Acked Read 3 Sent Done Done false
6530 2699 Acked Read 9 Sent Done Done false
6531 2700 Acked Read 8 Sent Done Done false
6532 2701 Acked Read 6 Sent Done Done false
6533 2702 Acked Write 6 Sent Done Done false
6534 2703 Acked Flush 0 Sent Done Done false
6535 2704 Acked Write 7 Sent Done Done false
6536 2705 Acked Write 1 Sent Done Done false
6537 2706 Acked Write 3 Sent Done Done false
6538 2707 Acked Read 2 Sent Done Done false
6539 2708 Acked Read 6 Sent Done Done false
6540 2709 Acked Flush 0 Sent Done Done false
6541 2710 Acked Write 10 Sent Done Done false
6542 2711 Acked Flush 0 Sent Done Done false
6543 2712 Acked Flush 0 Sent Done Done false
6544 2713 Acked Flush 0 Sent Done Done false
6545 2714 Acked Read 3 Sent Done Done false
6546 2715 Acked Write 3 Sent Done Done false
6547 2716 Acked Read 3 Sent Done Done false
6548 2717 Acked Read 7 Sent Done Done false
6549 2718 Acked Read 9 Sent Done Done false
6550 2719 Acked Read 3 Sent Done Done false
6551 2720 Acked Read 1 Sent Done Done false
6552 2721 Acked Read 6 Sent Done Done false
6553 2722 Acked Read 8 Sent Done Done false
6554 2723 Acked Write 8 Sent Done Done false
6555 2724 Acked Read 7 Sent Done Done false
6556 2725 Acked Write 4 Sent Done Done false
6557 2726 Acked Read 2 Sent Done Done false
6558 2727 Acked Write 10 Sent Done Done false
6559 2728 Acked Read 3 Sent Done Done false
6560 2729 Acked Read 6 Sent Done Done false
6561 2730 Acked Read 8 Sent Done Done false
6562 2731 Acked Write 1 Sent Done Done false
6563 2732 Acked Write 1 Sent Done Done false
6564 2733 Acked Read 7 Sent Done Done false
6565 2734 Acked Write 7 Sent Done Done false
6566 2735 Acked Read 6 Sent Done Done false
6567 2736 Acked Write 4 Sent Done Done false
6568 2737 Acked Read 10 Sent Done Done false
6569 2738 Acked Read 1 Sent Done Done false
6570 2739 Acked Write 8 Sent Done Done false
6571 2740 Acked Read 5 Sent Done Done false
6572 2741 Acked Write 8 Sent Done Done false
6573 2742 Acked Write 4 Sent Done Done false
6574 2743 Acked Write 10 Sent Done Done false
6575 2744 Acked Write 5 Sent Done Done false
6576 2745 Acked Flush 0 Sent Done Done false
6577 2746 Acked Write 6 Sent Done Done false
6578 2747 Acked Read 7 Sent Done Done false
6579 2748 Acked Write 10 Sent Done Done false
6580 2749 Acked Read 5 Sent Done Done false
6581 2750 Acked Read 6 Sent Done Done false
6582 2751 Acked Write 4 Sent Done Done false
6583 2752 Acked Write 10 Sent Done Done false
6584 2753 Acked Write 3 Sent Done Done false
6585 2754 Acked Read 1 Sent Done Done false
6586 2755 Acked Read 10 Sent Done Done false
6587 2756 Acked Write 9 Sent Done Done false
6588 2757 Acked Flush 0 Sent Done Done false
6589 2758 Acked Read 9 Sent Done Done false
6590 2759 Acked Write 5 Sent Done Done false
6591 2760 Acked Read 8 Sent Done Done false
6592 2761 Acked Read 2 Sent Done Done false
6593 2762 Acked Write 1 Sent Done Done false
6594 2763 Acked Write 5 Sent Done Done false
6595 2764 Acked Read 1 Sent Done Done false
6596 2765 Acked Read 10 Sent Done Done false
6597 2766 Acked Read 2 Sent Done Done false
6598 2767 Acked Read 10 Sent Done Done false
6599 2768 Acked Write 1 Sent Done Done false
6600 2769 Acked Write 6 Sent Done Done false
6601 2770 Acked Read 2 Sent Done Done false
6602 2771 Acked Write 1 Sent Done Done false
6603 2772 Acked Write 5 Sent Done Done false
6604 2773 Acked Read 9 Sent Done Done false
6605 2774 Acked Write 4 Sent Done Done false
6606 2775 Acked Write 7 Sent Done Done false
6607 2776 Acked Write 1 Sent Done Done false
6608 2777 Acked Read 5 Sent Done Done false
6609 2778 Acked Write 4 Sent Done Done false
6610 2779 Acked Write 2 Sent Done Done false
6611 2780 Acked Read 4 Sent Done Done false
6612 2781 Acked Read 3 Sent Done Done false
6613 2782 Acked Read 6 Sent Done Done false
6614 2783 Acked Write 4 Sent Done Done false
6615 2784 Acked Read 8 Sent Done Done false
6616 2785 Acked Read 8 Sent Done Done false
6617 2786 Acked Read 6 Sent Done Done false
6618 2787 Acked Read 9 Sent Done Done false
6619 2788 Acked Read 7 Sent Done Done false
6620 2789 Acked Read 8 Sent Done Done false
6621 2790 Acked Flush 0 Sent Done Done false
6622 2791 Acked Write 5 Sent Done Done false
6623 2792 Acked Write 3 Sent Done Done false
6624 2793 Acked Read 4 Sent Done Done false
6625 2794 Acked Write 10 Sent Done Done false
6626 2795 Acked Read 2 Sent Done Done false
6627 2796 Acked Write 9 Sent Done Done false
6628 2797 Acked Read 6 Sent Done Done false
6629 2798 Acked Read 9 Sent Done Done false
6630 2799 Acked Write 5 Sent Done Done false
6631 2800 Acked Read 5 Sent Done Done false
6632 2801 Acked Write 9 Sent Done Done false
6633 2802 Acked Write 6 Sent Done Done false
6634 2803 Acked Read 4 Sent Done Done false
6635 2804 Acked Flush 0 Sent Done Done false
6636 2805 Acked Write 9 Sent Done Done false
6637 2806 Acked Read 10 Sent Done Done false
6638 2807 Acked Write 4 Sent Done Done false
6639 2808 Acked Read 9 Sent Done Done false
6640 2809 Acked Write 7 Sent Done Done false
6641 2810 Acked Flush 0 Sent Done Done false
6642 STATES DS:0 DS:1 DS:2 TOTAL
6643 Sent 301 0 0 301
6644 Done 0 301 301 602
6645 Skipped 0 0 0 0
6646 Error 0 0 0 0
6647 Last Flush: 2509 2810 2810
6648 Downstairs last five completed: 2509 2508 2507 2506 2505
6649 Upstairs last five completed: 2810 2809 2808 2807 2806
6650 CLIENT: Up:0 ds:301 act:2
66512025-01-27T19:37:44.154ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66522025-01-27T19:37:44.154ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43241) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66532025-01-27T19:37:44.155ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66542025-01-27T19:37:44.156ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66552025-01-27T19:37:44.156ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66562025-01-27T19:37:44.156ZINFOcrucible: send last flush ID to this DS: 2509 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66572025-01-27T19:37:44.156ZINFOcrucible: Replied this last flush ID: 2509 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66582025-01-27T19:37:44.156ZINFOcrucible: [0] client re-new 301 jobs since flush 2509 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66592025-01-27T19:37:44.156ZINFOcrucible: [0] Replayed 301 jobs since flush: 2509 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
6660 ----------------------------------------------------------------
6661 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
6662 Downstairs last five completed: 2810 2809 2808 2807 2806
6663 Upstairs last five completed: 2810 2809 2808 2807 2806
6664 CLIENT: Up:0 ds:0 act:3
6665 Replay: All jobs finished
6666 Main task has work to do, go find it
6667 got dsc Stop(0)
6668 stop 0
6669 Jan 27 19:37:46.400 INFO request completed[8810] Got stop action so:false kr:false
6670 , latency_us: 317, response_code: 204, uri: /stop/cid/0, method: POST, req_id: dc48ee5a-bc75-43ad-96ac-026b8780f1dd, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
66712025-01-27T19:37:46.400ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66722025-01-27T19:37:46.400ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
66732025-01-27T19:37:46.400ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
6674 Jan 27 19:37:46.400 INFO request completed, latency_us: 131, response_code: 200, uri: /state/cid/0, method: GET, req_id: 9cd0afa2-f6ea-459f-827c-e293df9ee632, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
66752025-01-27T19:37:46.400ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
6676 [8810] Exited with: ExitStatus(unix_wait_status(9))
6677 [8810][0] reports Exit
6678 Jan 27 19:37:50.402 INFO request completed, latency_us: 189, response_code: 200, uri: /state/cid/0, method: GET, req_id: 970aa308-e7f3-421d-aba5-f8b4dc4ae539, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
6679 001/300 Write block 2437 len 40960 data: 4 4 4 4 4 4 4 4 4 4
6680 002/300 Read block 4351 len 20480
6681 003/300 Write block 4785 len 28672 data: 2 2 2 3 3 5 5
6682 004/300 Read block 2364 len 20480
6683 005/300 Write block 1764 len 8192 data: 2 2
6684 006/300 Read block 515 len 8192
6685 007/300 Read block 3804 len 8192
6686 008/300 Read block 2697 len 24576
6687 009/300 Read block 2699 len 40960
6688 010/300 Read block 3418 len 20480
6689 011/300 Write block 4561 len 24576 data: 3 3 4 4 4 3
6690 012/300 Read block 3408 len 12288
6691 013/300 Write block 3742 len 36864 data: 2 2 2 2 2 3 3 3 3
6692 014/300 Read block 4944 len 24576
6693 015/300 Write block 1950 len 40960 data: 2 2 3 3 3 3 3 3 3 3
6694 016/300 Write block 2796 len 4096 data: 3
6695 017/300 Flush
6696 018/300 Write block 2472 len 24576 data: 3 4 4 4 4 3
6697 019/300 Write block 4480 len 16384 data: 2 2 2 2
6698 020/300 Write block 1591 len 12288 data: 3 3 3
6699 021/300 Read block 1310 len 40960
6700 022/300 Flush
6701 023/300 Write block 3137 len 28672 data: 4 4 4 4 4 4 4
6702 024/300 Write block 4036 len 32768 data: 4 4 2 2 2 2 3 3
6703 025/300 Read block 721 len 24576
6704 026/300 Read block 4356 len 36864
6705 027/300 Write block 4168 len 8192 data: 3 3
6706 028/300 Write block 868 len 24576 data: 4 3 3 4 4 5
6707 029/300 Read block 2128 len 8192
6708 030/300 Write block 1805 len 28672 data: 3 3 3 3 3 5 5
6709 031/300 Flush
6710 032/300 Read block 2725 len 32768
6711 033/300 Write block 3277 len 32768 data: 3 3 2 3 3 3 3 2
6712 034/300 Write block 2428 len 8192 data: 3 3
6713 035/300 Write block 226 len 4096 data: 4
6714 036/300 Flush
6715 037/300 Read block 4231 len 40960
6716 038/300 Read block 953 len 16384
6717 039/300 Read block 976 len 40960
6718 040/300 Write block 802 len 8192 data: 2 2
6719 041/300 Read block 3961 len 32768
6720 042/300 Write block 2692 len 8192 data: 2 2
6721 043/300 Read block 3369 len 28672
6722 044/300 Write block 1531 len 12288 data: 2 2 2
6723 045/300 Read block 4003 len 36864
6724 046/300 Read block 1278 len 8192
6725 047/300 Write block 2565 len 8192 data: 5 5
6726 048/300 Read block 4563 len 16384
6727 049/300 Write block 2757 len 20480 data: 2 2 2 2 2
6728 050/300 Read block 4212 len 32768
6729 051/300 Read block 2473 len 20480
6730 052/300 Flush
6731 053/300 Write block 1904 len 36864 data: 2 2 2 2 2 2 2 2 2
6732 054/300 Read block 1005 len 24576
6733 055/300 Write block 2254 len 40960 data: 3 5 4 3 3 2 2 2 2 2
6734 056/300 Read block 1868 len 40960
6735 057/300 Read block 2899 len 4096
6736 058/300 Write block 956 len 12288 data: 3 3 3
6737 059/300 Read block 1844 len 40960
6738 060/300 Read block 3935 len 28672
6739 061/300 Read block 1887 len 40960
6740 062/300 Read block 2820 len 24576
6741 063/300 Write block 2013 len 12288 data: 2 2 2
6742 064/300 Read block 1157 len 32768
6743 065/300 Read block 3843 len 20480
6744 066/300 Read block 4107 len 16384
6745 067/300 Write block 1754 len 4096 data: 3
6746 068/300 Read block 4204 len 24576
6747 069/300 Read block 175 len 20480
6748 070/300 Write block 4656 len 16384 data: 2 2 2 2
6749 071/300 Read block 806 len 12288
6750 072/300 Read block 1425 len 40960
6751 073/300 Write block 4706 len 12288 data: 3 3 4
6752 074/300 Read block 2458 len 12288
6753 075/300 Read block 3624 len 24576
6754 076/300 Write block 4279 len 40960 data: 2 2 2 2 2 3 3 3 2 2
6755 077/300 Read block 4900 len 24576
6756 078/300 Read block 4231 len 12288
6757 079/300 Read block 1989 len 28672
6758 080/300 Write block 542 len 32768 data: 2 2 2 3 3 3 3 3
6759 081/300 Read block 1517 len 8192
6760 082/300 Flush
6761 083/300 Read block 4892 len 36864
6762 084/300 Write block 154 len 28672 data: 2 2 2 2 2 2 2
6763 085/300 Read block 4041 len 20480
6764 086/300 Write block 798 len 4096 data: 2
6765 087/300 Write block 3751 len 40960 data: 2 3 3 3 3 3 3 3 3 3
6766 088/300 Write block 3140 len 8192 data: 5 5
6767 089/300 Read block 2449 len 12288
6768 090/300 Read block 833 len 32768
6769 091/300 Write block 2719 len 4096 data: 4
6770 092/300 Write block 3187 len 40960 data: 2 2 2 2 2 2 2 2 2 2
6771 093/300 Flush
6772 094/300 Flush
6773 095/300 Read block 4528 len 8192
6774 096/300 Read block 617 len 8192
6775 097/300 Write block 3842 len 40960 data: 4 3 2 3 3 3 3 3 2 2
6776 098/300 Write block 4372 len 28672 data: 2 2 2 3 3 3 3
6777 099/300 Read block 3673 len 28672
6778 100/300 Read block 2316 len 40960
6779 101/300 Flush
6780 102/300 Read block 4886 len 36864
6781 103/300 Read block 2172 len 8192
6782 104/300 Read block 2807 len 20480
6783 105/300 Write block 205 len 28672 data: 2 3 3 3 3 3 3
6784 106/300 Read block 1925 len 36864
6785 107/300 Read block 3618 len 12288
6786 108/300 Read block 4240 len 8192
6787 109/300 Read block 2134 len 16384
6788 110/300 Read block 171 len 4096
6789 111/300 Write block 3965 len 40960 data: 2 2 2 2 2 2 2 2 2 3
6790 112/300 Write block 3605 len 4096 data: 2
6791 113/300 Read block 3486 len 12288
6792 114/300 Write block 64 len 12288 data: 3 3 3
6793 115/300 Read block 2361 len 8192
6794 116/300 Read block 1441 len 4096
6795 117/300 Read block 3621 len 32768
6796 118/300 Write block 4188 len 40960 data: 2 4 6 6 6 6 6 6 7 6
6797 119/300 Read block 4018 len 12288
6798 120/300 Write block 1145 len 20480 data: 2 2 2 2 2
6799 121/300 Write block 2524 len 32768 data: 3 2 2 2 2 2 2 2
6800 122/300 Write block 1314 len 16384 data: 3 3 3 3
6801 123/300 Read block 2567 len 8192
6802 124/300 Read block 2235 len 20480
6803 125/300 Read block 799 len 4096
6804 126/300 Write block 2534 len 24576 data: 2 2 2 2 2 2
6805 127/300 Write block 3449 len 12288 data: 3 3 3
6806 128/300 Read block 3553 len 12288
6807 129/300 Read block 2787 len 40960
6808 130/300 Read block 3631 len 12288
6809 131/300 Flush
6810 132/300 Read block 1618 len 36864
6811 133/300 Read block 4410 len 24576
6812 134/300 Read block 1998 len 40960
6813 135/300 Read block 932 len 16384
6814 136/300 Flush
6815 137/300 Flush
6816 138/300 Read block 2885 len 32768
6817 139/300 Read block 4967 len 36864
6818 140/300 Read block 938 len 8192
6819 141/300 Read block 3103 len 20480
6820 142/300 Write block 1032 len 24576 data: 5 5 5 4 4 3
6821 143/300 Write block 4579 len 12288 data: 2 2 2
6822 144/300 Write block 3559 len 40960 data: 4 4 4 4 4 4 4 2 2 3
6823 145/300 Write block 498 len 24576 data: 4 4 3 3 3 3
6824 146/300 Read block 4241 len 40960
6825 147/300 Read block 4718 len 24576
6826 148/300 Write block 129 len 16384 data: 3 3 4 3
6827 149/300 Read block 553 len 40960
6828 150/300 Flush
6829 151/300 Write block 278 len 24576 data: 4 4 6 6 5 4
6830 152/300 Write block 1105 len 4096 data: 3
6831 153/300 Write block 1411 len 28672 data: 3 4 4 4 4 4 5
6832 154/300 Write block 3464 len 28672 data: 3 3 3 3 2 2 2
6833 155/300 Write block 4435 len 32768 data: 2 2 3 3 4 4 4 4
6834 156/300 Read block 3956 len 28672
6835 157/300 Read block 1511 len 20480
6836 158/300 Read block 1204 len 36864
6837 159/300 Read block 4554 len 16384
6838 160/300 Write block 2444 len 32768 data: 5 5 5 2 3 3 3 3
6839 161/300 Read block 2711 len 16384
6840 162/300 Read block 2344 len 28672
6841 163/300 Read block 3016 len 12288
6842 164/300 Write block 4093 len 16384 data: 2 2 2 2
6843 165/300 Read block 2916 len 12288
6844 166/300 Read block 502 len 20480
6845 167/300 Flush
6846 168/300 Write block 4492 len 16384 data: 3 3 3 3
6847 169/300 Read block 548 len 24576
6848 170/300 Read block 1292 len 40960
6849 171/300 Write block 234 len 16384 data: 2 2 2 2
6850 172/300 Read block 304 len 4096
6851 173/300 Write block 3107 len 8192 data: 2 2
6852 174/300 Write block 3635 len 16384 data: 4 4 3 3
6853 175/300 Read block 1667 len 4096
6854 176/300 Write block 1196 len 12288 data: 4 2 2
6855 177/300 Read block 4382 len 24576
6856 178/300 Write block 4654 len 24576 data: 2 2 3 3 3 3
6857 179/300 Read block 3331 len 16384
6858 180/300 Write block 1231 len 40960 data: 2 2 3 4 4 4 4 4 4 3
6859 181/300 Read block 4277 len 8192
6860 182/300 Write block 3841 len 36864 data: 4 5 4 3 4 4 4 4 4
6861 183/300 Read block 4168 len 12288
6862 184/300 Read block 2784 len 8192
6863 185/300 Read block 1327 len 28672
6864 186/300 Write block 2529 len 24576 data: 3 3 3 2 2 3
6865 187/300 Read block 4295 len 24576
6866 188/300 Write block 746 len 12288 data: 3 3 3
6867 189/300 Flush
6868 190/300 Write block 3217 len 20480 data: 2 3 3 3 3
6869 191/300 Read block 1002 len 28672
6870 192/300 Read block 1417 len 36864
6871 193/300 Write block 3937 len 20480 data: 4 3 3 3 3
6872 194/300 Read block 2860 len 20480
6873 195/300 Read block 1420 len 12288
6874 196/300 Read block 132 len 12288
6875 197/300 Write block 4897 len 16384 data: 3 3 3 2
6876 198/300 Write block 2398 len 12288 data: 2 2 4
6877 199/300 Write block 2621 len 16384 data: 4 4 4 4
6878 200/300 Flush
6879 201/300 Flush
6880 202/300 Read block 3101 len 32768
6881 203/300 Read block 725 len 36864
6882 204/300 Write block 4683 len 4096 data: 3
6883 205/300 Write block 4982 len 36864 data: 2 2 2 3 3 3 3 3 2
6884 206/300 Read block 2419 len 40960
6885 207/300 Read block 425 len 40960
6886 208/300 Read block 1498 len 24576
6887 209/300 Flush
6888 210/300 Read block 4688 len 8192
6889 211/300 Read block 111 len 12288
6890 212/300 Read block 57 len 4096
6891 213/300 Read block 1196 len 24576
6892 214/300 Write block 3791 len 12288 data: 2 2 2
6893 215/300 Read block 3896 len 16384
6894 216/300 Write block 4446 len 40960 data: 3 2 2 2 2 2 2 3 2 2
6895 217/300 Flush
6896 218/300 Read block 1691 len 24576
6897 219/300 Read block 2790 len 20480
6898 220/300 Write block 3397 len 36864 data: 4 4 3 3 3 3 4 3 3
6899 221/300 Write block 4900 len 12288 data: 3 2 3
6900 222/300 Write block 2600 len 40960 data: 3 2 2 2 2 2 2 2 2 2
6901 223/300 Read block 882 len 16384
6902 224/300 Read block 3580 len 36864
6903 225/300 Write block 2035 len 4096 data: 2
6904 226/300 Flush
6905 227/300 Read block 1159 len 16384
6906 228/300 Write block 223 len 8192 data: 3 3
6907 229/300 Read block 951 len 20480
6908 230/300 Write block 719 len 24576 data: 3 4 4 4 4 4
6909 231/300 Read block 1930 len 4096
6910 232/300 Write block 711 len 16384 data: 3 3 7 7
6911 233/300 Write block 705 len 16384 data: 2 2 2 3
6912 234/300 Flush
6913 235/300 Write block 437 len 16384 data: 2 2 3 3
6914 236/300 Write block 4883 len 32768 data: 3 3 2 2 3 3 2 2
6915 237/300 Read block 4982 len 16384
6916 238/300 Read block 4118 len 20480
6917 239/300 Read block 127 len 40960
6918 240/300 Read block 2390 len 28672
6919 241/300 Read block 3881 len 8192
6920 242/300 Read block 2268 len 8192
6921 243/300 Read block 4643 len 28672
6922 244/300 Flush
6923 245/300 Read block 4224 len 8192
6924 246/300 Read block 3756 len 28672
6925 247/300 Read block 4337 len 8192
6926 248/300 Read block 4190 len 4096
6927 249/300 Read block 4595 len 32768
6928 250/300 Write block 239 len 16384 data: 3 3 3 3
6929 251/300 Read block 3145 len 40960
6930 252/300 Flush
6931 253/300 Write block 400 len 20480 data: 3 4 4 4 4
6932 254/300 Write block 956 len 40960 data: 4 4 4 4 4 4 3 3 3 2
6933 255/300 Flush
6934 256/300 Read block 432 len 8192
6935 257/300 Read block 3282 len 32768
6936 258/300 Write block 1458 len 8192 data: 4 3
6937 259/300 Flush
6938 260/300 Read block 4497 len 40960
6939 261/300 Read block 3874 len 12288
6940 262/300 Read block 1700 len 12288
6941 263/300 Write block 4857 len 20480 data: 2 2 2 2 2
6942 264/300 Write block 2830 len 20480 data: 3 3 4 4 3
6943 265/300 Read block 4374 len 20480
6944 266/300 Read block 3293 len 16384
6945 267/300 Write block 3519 len 32768 data: 5 5 5 5 4 4 4 4
6946 268/300 Read block 2054 len 12288
6947 269/300 Read block 2738 len 24576
6948 270/300 Read block 1021 len 8192
6949 271/300 Read block 789 len 28672
6950 272/300 Write block 1623 len 20480 data: 2 2 2 2 4
6951 273/300 Read block 1077 len 32768
6952 274/300 Read block 3003 len 4096
6953 275/300 Write block 3721 len 40960 data: 2 2 2 2 3 3 4 4 4 3
6954 276/300 Flush
6955 277/300 Write block 3492 len 40960 data: 3 3 3 3 2 2 2 3 3 2
6956 278/300 Write block 4628 len 28672 data: 2 2 2 2 2 3 3
6957 279/300 Read block 387 len 40960
6958 280/300 Read block 3240 len 24576
6959 281/300 Write block 942 len 4096 data: 3
6960 282/300 Write block 994 len 8192 data: 2 2
6961 283/300 Read block 2493 len 16384
6962 284/300 Flush
6963 285/300 Read block 551 len 4096
6964 286/300 Read block 2670 len 32768
6965 287/300 Write block 2010 len 36864 data: 2 3 2 3 3 3 2 2 3
6966 288/300 Read block 3235 len 24576
6967 289/300 Write block 2482 len 12288 data: 2 3 3
6968 290/300 Write block 1333 len 16384 data: 4 3 3 3
6969 291/300 Read block 4897 len 20480
6970 292/300 Read block 2124 len 24576
6971 293/300 Read block 2397 len 20480
6972 294/300 Write block 3604 len 32768 data: 2 3 2 2 2 2 2 2
6973 295/300 Write block 4562 len 12288 data: 4 5 5
6974 296/300 Write block 4651 len 16384 data: 2 2 2 3
6975 297/300 Read block 1888 len 24576
6976 298/300 Write block 3309 len 36864 data: 2 2 2 2 2 3 3 3 3
6977 299/300 Write block 3603 len 24576 data: 2 3 4 3 3 3
6978 300/300 Read block 1928 len 24576
6979 301/300 Flush
6980 Main task has work to do, go find it
6981 got dsc Start(0)
6982 start 0
6983 Jan 27 19:37:50.644 INFO request completed, latency_us: 167, response_code: 204, uri: /start/cid/0, method: POST, req_id: 84d9516a-2cd9-4253-b134-ea60c908dbeb, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
6984 Replay: started 0, returned:Ok(())
6985 ----------------------------------------------------------------
6986 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
6987 States: Offline Active Active
6988 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
6989 2811 Acked Write 10 Sent Done Done false
6990 2812 Acked Read 5 Sent Done Done false
6991 2813 Acked Write 7 Sent Done Done false
6992 2814 Acked Read 5 Sent Done Done false
6993 2815 Acked Write 2 Sent Done Done false
6994 2816 Acked Read 2 Sent Done Done false
6995 2817 Acked Read 2 Sent Done Done false
6996 2818 Acked Read 6 Sent Done Done false
6997 2819 Acked Read 10 Sent Done Done false
6998 2820 Acked Read 5 Sent Done Done false
6999 2821 Acked Write 6 Sent Done Done false
7000 2822 Acked Read 3 Sent Done Done false
7001 2823 Acked Write 9 Sent Done Done false
7002 2824 Acked Read 6 Sent Done Done false
7003 2825 Acked Write 10 Sent Done Done false
7004 2826 Acked Write 1 Sent Done Done false
7005 2827 Acked Flush 0 Sent Done Done false
7006 2828 Acked Write 6 Sent Done Done false
7007 2829 Acked Write 4 Sent Done Done false
7008 2830 Acked Write 3 Sent Done Done false
7009 2831 Acked Read 10 Sent Done Done false
7010 2832 Acked Flush 0 Sent Done Done false
7011 2833 Acked Write 7 Sent Done Done false
7012 2834 Acked Write 8 Sent Done Done false
7013 2835 Acked Read 6 Sent Done Done false
7014 2836 Acked Read 9 Sent Done Done false
7015 2837 Acked Write 2 Sent Done Done false
7016 2838 Acked Write 6 Sent Done Done false
7017 2839 Acked Read 2 Sent Done Done false
7018 2840 Acked Write 7 Sent Done Done false
7019 2841 Acked Flush 0 Sent Done Done false
7020 2842 Acked Read 8 Sent Done Done false
7021 2843 Acked Write 8 Sent Done Done false
7022 2844 Acked Write 2 Sent Done Done false
7023 2845 Acked Write 1 Sent Done Done false
7024 2846 Acked Flush 0 Sent Done Done false
7025 2847 Acked Read 10 Sent Done Done false
7026 2848 Acked Read 4 Sent Done Done false
7027 2849 Acked Read 10 Sent Done Done false
7028 2850 Acked Write 2 Sent Done Done false
7029 2851 Acked Read 8 Sent Done Done false
7030 2852 Acked Write 2 Sent Done Done false
7031 2853 Acked Read 7 Sent Done Done false
7032 2854 Acked Write 3 Sent Done Done false
7033 2855 Acked Read 9 Sent Done Done false
7034 2856 Acked Read 2 Sent Done Done false
7035 2857 Acked Write 2 Sent Done Done false
7036 2858 Acked Read 4 Sent Done Done false
7037 2859 Acked Write 5 Sent Done Done false
7038 2860 Acked Read 8 Sent Done Done false
7039 2861 Acked Read 5 Sent Done Done false
7040 2862 Acked Flush 0 Sent Done Done false
7041 2863 Acked Write 9 Sent Done Done false
7042 2864 Acked Read 6 Sent Done Done false
7043 2865 Acked Write 10 Sent Done Done false
7044 2866 Acked Read 10 Sent Done Done false
7045 2867 Acked Read 1 Sent Done Done false
7046 2868 Acked Write 3 Sent Done Done false
7047 2869 Acked Read 10 Sent Done Done false
7048 2870 Acked Read 7 Sent Done Done false
7049 2871 Acked Read 10 Sent Done Done false
7050 2872 Acked Read 6 Sent Done Done false
7051 2873 Acked Write 3 Sent Done Done false
7052 2874 Acked Read 8 Sent Done Done false
7053 2875 Acked Read 5 Sent Done Done false
7054 2876 Acked Read 4 Sent Done Done false
7055 2877 Acked Write 1 Sent Done Done false
7056 2878 Acked Read 6 Sent Done Done false
7057 2879 Acked Read 5 Sent Done Done false
7058 2880 Acked Write 4 Sent Done Done false
7059 2881 Acked Read 3 Sent Done Done false
7060 2882 Acked Read 10 Sent Done Done false
7061 2883 Acked Write 3 Sent Done Done false
7062 2884 Acked Read 3 Sent Done Done false
7063 2885 Acked Read 6 Sent Done Done false
7064 2886 Acked Write 10 Sent Done Done false
7065 2887 Acked Read 6 Sent Done Done false
7066 2888 Acked Read 3 Sent Done Done false
7067 2889 Acked Read 7 Sent Done Done false
7068 2890 Acked Write 8 Sent Done Done false
7069 2891 Acked Read 2 Sent Done Done false
7070 2892 Acked Flush 0 Sent Done Done false
7071 2893 Acked Read 9 Sent Done Done false
7072 2894 Acked Write 7 Sent Done Done false
7073 2895 Acked Read 5 Sent Done Done false
7074 2896 Acked Write 1 Sent Done Done false
7075 2897 Acked Write 10 Sent Done Done false
7076 2898 Acked Write 2 Sent Done Done false
7077 2899 Acked Read 3 Sent Done Done false
7078 2900 Acked Read 8 Sent Done Done false
7079 2901 Acked Write 1 Sent Done Done false
7080 2902 Acked Write 10 Sent Done Done false
7081 2903 Acked Flush 0 Sent Done Done false
7082 2904 Acked Flush 0 Sent Done Done false
7083 2905 Acked Read 2 Sent Done Done false
7084 2906 Acked Read 2 Sent Done Done false
7085 2907 Acked Write 10 Sent Done Done false
7086 2908 Acked Write 7 Sent Done Done false
7087 2909 Acked Read 7 Sent Done Done false
7088 2910 Acked Read 10 Sent Done Done false
7089 2911 Acked Flush 0 Sent Done Done false
7090 2912 Acked Read 9 Sent Done Done false
7091 2913 Acked Read 2 Sent Done Done false
7092 2914 Acked Read 5 Sent Done Done false
7093 2915 Acked Write 7 Sent Done Done false
7094 2916 Acked Read 9 Sent Done Done false
7095 2917 Acked Read 3 Sent Done Done false
7096 2918 Acked Read 2 Sent Done Done false
7097 2919 Acked Read 4 Sent Done Done false
7098 2920 Acked Read 1 Sent Done Done false
7099 2921 Acked Write 10 Sent Done Done false
7100 2922 Acked Write 1 Sent Done Done false
7101 2923 Acked Read 3 Sent Done Done false
7102 2924 Acked Write 3 Sent Done Done false
7103 2925 Acked Read 2 Sent Done Done false
7104 2926 Acked Read 1 Sent Done Done false
7105 2927 Acked Read 8 Sent Done Done false
7106 2928 Acked Write 10 Sent Done Done false
7107 2929 Acked Read 3 Sent Done Done false
7108 2930 Acked Write 5 Sent Done Done false
7109 2931 Acked Write 8 Sent Done Done false
7110 2932 Acked Write 4 Sent Done Done false
7111 2933 Acked Read 2 Sent Done Done false
7112 2934 Acked Read 5 Sent Done Done false
7113 2935 Acked Read 1 Sent Done Done false
7114 2936 Acked Write 6 Sent Done Done false
7115 2937 Acked Write 3 Sent Done Done false
7116 2938 Acked Read 3 Sent Done Done false
7117 2939 Acked Read 10 Sent Done Done false
7118 2940 Acked Read 3 Sent Done Done false
7119 2941 Acked Flush 0 Sent Done Done false
7120 2942 Acked Read 9 Sent Done Done false
7121 2943 Acked Read 6 Sent Done Done false
7122 2944 Acked Read 10 Sent Done Done false
7123 2945 Acked Read 4 Sent Done Done false
7124 2946 Acked Flush 0 Sent Done Done false
7125 2947 Acked Flush 0 Sent Done Done false
7126 2948 Acked Read 8 Sent Done Done false
7127 2949 Acked Read 9 Sent Done Done false
7128 2950 Acked Read 2 Sent Done Done false
7129 2951 Acked Read 5 Sent Done Done false
7130 2952 Acked Write 6 Sent Done Done false
7131 2953 Acked Write 3 Sent Done Done false
7132 2954 Acked Write 10 Sent Done Done false
7133 2955 Acked Write 6 Sent Done Done false
7134 2956 Acked Read 10 Sent Done Done false
7135 2957 Acked Read 6 Sent Done Done false
7136 2958 Acked Write 4 Sent Done Done false
7137 2959 Acked Read 10 Sent Done Done false
7138 2960 Acked Flush 0 Sent Done Done false
7139 2961 Acked Write 6 Sent Done Done false
7140 2962 Acked Write 1 Sent Done Done false
7141 2963 Acked Write 7 Sent Done Done false
7142 2964 Acked Write 7 Sent Done Done false
7143 2965 Acked Write 8 Sent Done Done false
7144 2966 Acked Read 7 Sent Done Done false
7145 2967 Acked Read 5 Sent Done Done false
7146 2968 Acked Read 9 Sent Done Done false
7147 2969 Acked Read 4 Sent Done Done false
7148 2970 Acked Write 8 Sent Done Done false
7149 2971 Acked Read 4 Sent Done Done false
7150 2972 Acked Read 7 Sent Done Done false
7151 2973 Acked Read 3 Sent Done Done false
7152 2974 Acked Write 4 Sent Done Done false
7153 2975 Acked Read 3 Sent Done Done false
7154 2976 Acked Read 5 Sent Done Done false
7155 2977 Acked Flush 0 Sent Done Done false
7156 2978 Acked Write 4 Sent Done Done false
7157 2979 Acked Read 6 Sent Done Done false
7158 2980 Acked Read 10 Sent Done Done false
7159 2981 Acked Write 4 Sent Done Done false
7160 2982 Acked Read 1 Sent Done Done false
7161 2983 Acked Write 2 Sent Done Done false
7162 2984 Acked Write 4 Sent Done Done false
7163 2985 Acked Read 1 Sent Done Done false
7164 2986 Acked Write 3 Sent Done Done false
7165 2987 Acked Read 6 Sent Done Done false
7166 2988 Acked Write 6 Sent Done Done false
7167 2989 Acked Read 4 Sent Done Done false
7168 2990 Acked Write 10 Sent Done Done false
7169 2991 Acked Read 2 Sent Done Done false
7170 2992 Acked Write 9 Sent Done Done false
7171 2993 Acked Read 3 Sent Done Done false
7172 2994 Acked Read 2 Sent Done Done false
7173 2995 Acked Read 7 Sent Done Done false
7174 2996 Acked Write 6 Sent Done Done false
7175 2997 Acked Read 6 Sent Done Done false
7176 2998 Acked Write 3 Sent Done Done false
7177 2999 Acked Flush 0 Sent Done Done false
7178 3000 Acked Write 5 Sent Done Done false
7179 3001 Acked Read 7 Sent Done Done false
7180 3002 Acked Read 9 Sent Done Done false
7181 3003 Acked Write 5 Sent Done Done false
7182 3004 Acked Read 5 Sent Done Done false
7183 3005 Acked Read 3 Sent Done Done false
7184 3006 Acked Read 3 Sent Done Done false
7185 3007 Acked Write 4 Sent Done Done false
7186 3008 Acked Write 3 Sent Done Done false
7187 3009 Acked Write 4 Sent Done Done false
7188 3010 Acked Flush 0 Sent Done Done false
7189 3011 Acked Flush 0 Sent Done Done false
7190 3012 Acked Read 8 Sent Done Done false
7191 3013 Acked Read 9 Sent Done Done false
7192 3014 Acked Write 1 Sent Done Done false
7193 3015 Acked Write 9 Sent Done Done false
7194 3016 Acked Read 10 Sent Done Done false
7195 3017 Acked Read 10 Sent Done Done false
7196 3018 Acked Read 6 Sent Done Done false
7197 3019 Acked Flush 0 Sent Done Done false
7198 3020 Acked Read 2 Sent Done Done false
7199 3021 Acked Read 3 Sent Done Done false
7200 3022 Acked Read 1 Sent Done Done false
7201 3023 Acked Read 6 Sent Done Done false
7202 3024 Acked Write 3 Sent Done Done false
7203 3025 Acked Read 4 Sent Done Done false
7204 3026 Acked Write 10 Sent Done Done false
7205 3027 Acked Flush 0 Sent Done Done false
7206 3028 Acked Read 6 Sent Done Done false
7207 3029 Acked Read 5 Sent Done Done false
7208 3030 Acked Write 9 Sent Done Done false
7209 3031 Acked Write 3 Sent Done Done false
7210 3032 Acked Write 10 Sent Done Done false
7211 3033 Acked Read 4 Sent Done Done false
7212 3034 Acked Read 9 Sent Done Done false
7213 3035 Acked Write 1 Sent Done Done false
7214 3036 Acked Flush 0 Sent Done Done false
7215 3037 Acked Read 4 Sent Done Done false
7216 3038 Acked Write 2 Sent Done Done false
7217 3039 Acked Read 5 Sent Done Done false
7218 3040 Acked Write 6 Sent Done Done false
7219 3041 Acked Read 1 Sent Done Done false
7220 3042 Acked Write 4 Sent Done Done false
7221 3043 Acked Write 4 Sent Done Done false
7222 3044 Acked Flush 0 Sent Done Done false
7223 3045 Acked Write 4 Sent Done Done false
7224 3046 Acked Write 8 Sent Done Done false
7225 3047 Acked Read 4 Sent Done Done false
7226 3048 Acked Read 5 Sent Done Done false
7227 3049 Acked Read 10 Sent Done Done false
7228 3050 Acked Read 7 Sent Done Done false
7229 3051 Acked Read 2 Sent Done Done false
7230 3052 Acked Read 2 Sent Done Done false
7231 3053 Acked Read 7 Sent Done Done false
7232 3054 Acked Flush 0 Sent Done Done false
7233 3055 Acked Read 2 Sent Done Done false
7234 3056 Acked Read 7 Sent Done Done false
7235 3057 Acked Read 2 Sent Done Done false
7236 3058 Acked Read 1 Sent Done Done false
7237 3059 Acked Read 8 Sent Done Done false
7238 3060 Acked Write 4 Sent Done Done false
7239 3061 Acked Read 10 Sent Done Done false
7240 3062 Acked Flush 0 Sent Done Done false
7241 3063 Acked Write 5 Sent Done Done false
7242 3064 Acked Write 10 Sent Done Done false
7243 3065 Acked Flush 0 Sent Done Done false
7244 3066 Acked Read 2 Sent Done Done false
7245 3067 Acked Read 8 Sent Done Done false
7246 3068 Acked Write 2 Sent Done Done false
7247 3069 Acked Flush 0 Sent Done Done false
7248 3070 Acked Read 10 Sent Done Done false
7249 3071 Acked Read 3 Sent Done Done false
7250 3072 Acked Read 3 Sent Done Done false
7251 3073 Acked Write 5 Sent Done Done false
7252 3074 Acked Write 5 Sent Done Done false
7253 3075 Acked Read 5 Sent Done Done false
7254 3076 Acked Read 4 Sent Done Done false
7255 3077 Acked Write 8 Sent Done Done false
7256 3078 Acked Read 3 Sent Done Done false
7257 3079 Acked Read 6 Sent Done Done false
7258 3080 Acked Read 2 Sent Done Done false
7259 3081 Acked Read 7 Sent Done Done false
7260 3082 Acked Write 5 Sent Done Done false
7261 3083 Acked Read 8 Sent Done Done false
7262 3084 Acked Read 1 Sent Done Done false
7263 3085 Acked Write 10 Sent Done Done false
7264 3086 Acked Flush 0 Sent Done Done false
7265 3087 Acked Write 10 Sent Done Done false
7266 3088 Acked Write 7 Sent Done Done false
7267 3089 Acked Read 10 Sent Done Done false
7268 3090 Acked Read 6 Sent Done Done false
7269 3091 Acked Write 1 Sent Done Done false
7270 3092 Acked Write 2 Sent Done Done false
7271 3093 Acked Read 4 Sent Done Done false
7272 3094 Acked Flush 0 Sent Done Done false
7273 3095 Acked Read 1 Sent Done Done false
7274 3096 Acked Read 8 Sent Done Done false
7275 3097 Acked Write 9 Sent Done Done false
7276 3098 Acked Read 6 Sent Done Done false
7277 3099 Acked Write 3 Sent Done Done false
7278 3100 Acked Write 4 Sent Done Done false
7279 3101 Acked Read 5 Sent Done Done false
7280 3102 Acked Read 6 Sent Done Done false
7281 3103 Acked Read 5 Sent Done Done false
7282 3104 Acked Write 8 Sent Done Done false
7283 3105 Acked Write 3 Sent Done Done false
7284 3106 Acked Write 4 Sent Done Done false
7285 3107 Acked Read 6 Sent Done Done false
7286 3108 Acked Write 9 Sent Done Done false
7287 3109 Acked Write 6 Sent Done Done false
7288 3110 Acked Read 6 Sent Done Done false
7289 3111 Acked Flush 0 Sent Done Done false
7290 STATES DS:0 DS:1 DS:2 TOTAL
7291 Sent 301 0 0 301
7292 Done 0 301 301 602
7293 Skipped 0 0 0 0
7294 Error 0 0 0 0
7295 Last Flush: 2810 3111 3111
7296 Downstairs last five completed: 2810 2809 2808 2807 2806
7297 Upstairs last five completed: 3111 3110 3109 3108 3107
7298 CLIENT: Up:0 ds:301 act:2
7299 [8810] got start action
7300 [8810] I am going to restart
7301 Starting downstairs at port 8810
7302 Make output file at "/tmp/dsc/downstairs-8810.txt"
7303 [8810][0] reports Starting
7304 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1207)
7305 [8810][0] reports Running
7306 ----------------------------------------------------------------
7307 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7308 States: Offline Active Active
7309 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7310 2811 Acked Write 10 Sent Done Done false
7311 2812 Acked Read 5 Sent Done Done false
7312 2813 Acked Write 7 Sent Done Done false
7313 2814 Acked Read 5 Sent Done Done false
7314 2815 Acked Write 2 Sent Done Done false
7315 2816 Acked Read 2 Sent Done Done false
7316 2817 Acked Read 2 Sent Done Done false
7317 2818 Acked Read 6 Sent Done Done false
7318 2819 Acked Read 10 Sent Done Done false
7319 2820 Acked Read 5 Sent Done Done false
7320 2821 Acked Write 6 Sent Done Done false
7321 2822 Acked Read 3 Sent Done Done false
7322 2823 Acked Write 9 Sent Done Done false
7323 2824 Acked Read 6 Sent Done Done false
7324 2825 Acked Write 10 Sent Done Done false
7325 2826 Acked Write 1 Sent Done Done false
7326 2827 Acked Flush 0 Sent Done Done false
7327 2828 Acked Write 6 Sent Done Done false
7328 2829 Acked Write 4 Sent Done Done false
7329 2830 Acked Write 3 Sent Done Done false
7330 2831 Acked Read 10 Sent Done Done false
7331 2832 Acked Flush 0 Sent Done Done false
7332 2833 Acked Write 7 Sent Done Done false
7333 2834 Acked Write 8 Sent Done Done false
7334 2835 Acked Read 6 Sent Done Done false
7335 2836 Acked Read 9 Sent Done Done false
7336 2837 Acked Write 2 Sent Done Done false
7337 2838 Acked Write 6 Sent Done Done false
7338 2839 Acked Read 2 Sent Done Done false
7339 2840 Acked Write 7 Sent Done Done false
7340 2841 Acked Flush 0 Sent Done Done false
7341 2842 Acked Read 8 Sent Done Done false
7342 2843 Acked Write 8 Sent Done Done false
7343 2844 Acked Write 2 Sent Done Done false
7344 2845 Acked Write 1 Sent Done Done false
7345 2846 Acked Flush 0 Sent Done Done false
7346 2847 Acked Read 10 Sent Done Done false
7347 2848 Acked Read 4 Sent Done Done false
7348 2849 Acked Read 10 Sent Done Done false
7349 2850 Acked Write 2 Sent Done Done false
7350 2851 Acked Read 8 Sent Done Done false
7351 2852 Acked Write 2 Sent Done Done false
7352 2853 Acked Read 7 Sent Done Done false
7353 2854 Acked Write 3 Sent Done Done false
7354 2855 Acked Read 9 Sent Done Done false
7355 2856 Acked Read 2 Sent Done Done false
7356 2857 Acked Write 2 Sent Done Done false
7357 2858 Acked Read 4 Sent Done Done false
7358 2859 Acked Write 5 Sent Done Done false
7359 2860 Acked Read 8 Sent Done Done false
7360 2861 Acked Read 5 Sent Done Done false
7361 2862 Acked Flush 0 Sent Done Done false
7362 2863 Acked Write 9 Sent Done Done false
7363 2864 Acked Read 6 Sent Done Done false
7364 2865 Acked Write 10 Sent Done Done false
7365 2866 Acked Read 10 Sent Done Done false
7366 2867 Acked Read 1 Sent Done Done false
7367 2868 Acked Write 3 Sent Done Done false
7368 2869 Acked Read 10 Sent Done Done false
7369 2870 Acked Read 7 Sent Done Done false
7370 2871 Acked Read 10 Sent Done Done false
7371 2872 Acked Read 6 Sent Done Done false
7372 2873 Acked Write 3 Sent Done Done false
7373 2874 Acked Read 8 Sent Done Done false
7374 2875 Acked Read 5 Sent Done Done false
7375 2876 Acked Read 4 Sent Done Done false
7376 2877 Acked Write 1 Sent Done Done false
7377 2878 Acked Read 6 Sent Done Done false
7378 2879 Acked Read 5 Sent Done Done false
7379 2880 Acked Write 4 Sent Done Done false
7380 2881 Acked Read 3 Sent Done Done false
7381 2882 Acked Read 10 Sent Done Done false
7382 2883 Acked Write 3 Sent Done Done false
7383 2884 Acked Read 3 Sent Done Done false
7384 2885 Acked Read 6 Sent Done Done false
7385 2886 Acked Write 10 Sent Done Done false
7386 2887 Acked Read 6 Sent Done Done false
7387 2888 Acked Read 3 Sent Done Done false
7388 2889 Acked Read 7 Sent Done Done false
7389 2890 Acked Write 8 Sent Done Done false
7390 2891 Acked Read 2 Sent Done Done false
7391 2892 Acked Flush 0 Sent Done Done false
7392 2893 Acked Read 9 Sent Done Done false
7393 2894 Acked Write 7 Sent Done Done false
7394 2895 Acked Read 5 Sent Done Done false
7395 2896 Acked Write 1 Sent Done Done false
7396 2897 Acked Write 10 Sent Done Done false
7397 2898 Acked Write 2 Sent Done Done false
7398 2899 Acked Read 3 Sent Done Done false
7399 2900 Acked Read 8 Sent Done Done false
7400 2901 Acked Write 1 Sent Done Done false
7401 2902 Acked Write 10 Sent Done Done false
7402 2903 Acked Flush 0 Sent Done Done false
7403 2904 Acked Flush 0 Sent Done Done false
7404 2905 Acked Read 2 Sent Done Done false
7405 2906 Acked Read 2 Sent Done Done false
7406 2907 Acked Write 10 Sent Done Done false
7407 2908 Acked Write 7 Sent Done Done false
7408 2909 Acked Read 7 Sent Done Done false
7409 2910 Acked Read 10 Sent Done Done false
7410 2911 Acked Flush 0 Sent Done Done false
7411 2912 Acked Read 9 Sent Done Done false
7412 2913 Acked Read 2 Sent Done Done false
7413 2914 Acked Read 5 Sent Done Done false
7414 2915 Acked Write 7 Sent Done Done false
7415 2916 Acked Read 9 Sent Done Done false
7416 2917 Acked Read 3 Sent Done Done false
7417 2918 Acked Read 2 Sent Done Done false
7418 2919 Acked Read 4 Sent Done Done false
7419 2920 Acked Read 1 Sent Done Done false
7420 2921 Acked Write 10 Sent Done Done false
7421 2922 Acked Write 1 Sent Done Done false
7422 2923 Acked Read 3 Sent Done Done false
7423 2924 Acked Write 3 Sent Done Done false
7424 2925 Acked Read 2 Sent Done Done false
7425 2926 Acked Read 1 Sent Done Done false
7426 2927 Acked Read 8 Sent Done Done false
7427 2928 Acked Write 10 Sent Done Done false
7428 2929 Acked Read 3 Sent Done Done false
7429 2930 Acked Write 5 Sent Done Done false
7430 2931 Acked Write 8 Sent Done Done false
7431 2932 Acked Write 4 Sent Done Done false
7432 2933 Acked Read 2 Sent Done Done false
7433 2934 Acked Read 5 Sent Done Done false
7434 2935 Acked Read 1 Sent Done Done false
7435 2936 Acked Write 6 Sent Done Done false
7436 2937 Acked Write 3 Sent Done Done false
7437 2938 Acked Read 3 Sent Done Done false
7438 2939 Acked Read 10 Sent Done Done false
7439 2940 Acked Read 3 Sent Done Done false
7440 2941 Acked Flush 0 Sent Done Done false
7441 2942 Acked Read 9 Sent Done Done false
7442 2943 Acked Read 6 Sent Done Done false
7443 2944 Acked Read 10 Sent Done Done false
7444 2945 Acked Read 4 Sent Done Done false
7445 2946 Acked Flush 0 Sent Done Done false
7446 2947 Acked Flush 0 Sent Done Done false
7447 2948 Acked Read 8 Sent Done Done false
7448 2949 Acked Read 9 Sent Done Done false
7449 2950 Acked Read 2 Sent Done Done false
7450 2951 Acked Read 5 Sent Done Done false
7451 2952 Acked Write 6 Sent Done Done false
7452 2953 Acked Write 3 Sent Done Done false
7453 2954 Acked Write 10 Sent Done Done false
7454 2955 Acked Write 6 Sent Done Done false
7455 2956 Acked Read 10 Sent Done Done false
7456 2957 Acked Read 6 Sent Done Done false
7457 2958 Acked Write 4 Sent Done Done false
7458 2959 Acked Read 10 Sent Done Done false
7459 2960 Acked Flush 0 Sent Done Done false
7460 2961 Acked Write 6 Sent Done Done false
7461 2962 Acked Write 1 Sent Done Done false
7462 2963 Acked Write 7 Sent Done Done false
7463 2964 Acked Write 7 Sent Done Done false
7464 2965 Acked Write 8 Sent Done Done false
7465 2966 Acked Read 7 Sent Done Done false
7466 2967 Acked Read 5 Sent Done Done false
7467 2968 Acked Read 9 Sent Done Done false
7468 2969 Acked Read 4 Sent Done Done false
7469 2970 Acked Write 8 Sent Done Done false
7470 2971 Acked Read 4 Sent Done Done false
7471 2972 Acked Read 7 Sent Done Done false
7472 2973 Acked Read 3 Sent Done Done false
7473 2974 Acked Write 4 Sent Done Done false
7474 2975 Acked Read 3 Sent Done Done false
7475 2976 Acked Read 5 Sent Done Done false
7476 2977 Acked Flush 0 Sent Done Done false
7477 2978 Acked Write 4 Sent Done Done false
7478 2979 Acked Read 6 Sent Done Done false
7479 2980 Acked Read 10 Sent Done Done false
7480 2981 Acked Write 4 Sent Done Done false
7481 2982 Acked Read 1 Sent Done Done false
7482 2983 Acked Write 2 Sent Done Done false
7483 2984 Acked Write 4 Sent Done Done false
7484 2985 Acked Read 1 Sent Done Done false
7485 2986 Acked Write 3 Sent Done Done false
7486 2987 Acked Read 6 Sent Done Done false
7487 2988 Acked Write 6 Sent Done Done false
7488 2989 Acked Read 4 Sent Done Done false
7489 2990 Acked Write 10 Sent Done Done false
7490 2991 Acked Read 2 Sent Done Done false
7491 2992 Acked Write 9 Sent Done Done false
7492 2993 Acked Read 3 Sent Done Done false
7493 2994 Acked Read 2 Sent Done Done false
7494 2995 Acked Read 7 Sent Done Done false
7495 2996 Acked Write 6 Sent Done Done false
7496 2997 Acked Read 6 Sent Done Done false
7497 2998 Acked Write 3 Sent Done Done false
7498 2999 Acked Flush 0 Sent Done Done false
7499 3000 Acked Write 5 Sent Done Done false
7500 3001 Acked Read 7 Sent Done Done false
7501 3002 Acked Read 9 Sent Done Done false
7502 3003 Acked Write 5 Sent Done Done false
7503 3004 Acked Read 5 Sent Done Done false
7504 3005 Acked Read 3 Sent Done Done false
7505 3006 Acked Read 3 Sent Done Done false
7506 3007 Acked Write 4 Sent Done Done false
7507 3008 Acked Write 3 Sent Done Done false
7508 3009 Acked Write 4 Sent Done Done false
7509 3010 Acked Flush 0 Sent Done Done false
7510 3011 Acked Flush 0 Sent Done Done false
7511 3012 Acked Read 8 Sent Done Done false
7512 3013 Acked Read 9 Sent Done Done false
7513 3014 Acked Write 1 Sent Done Done false
7514 3015 Acked Write 9 Sent Done Done false
7515 3016 Acked Read 10 Sent Done Done false
7516 3017 Acked Read 10 Sent Done Done false
7517 3018 Acked Read 6 Sent Done Done false
7518 3019 Acked Flush 0 Sent Done Done false
7519 3020 Acked Read 2 Sent Done Done false
7520 3021 Acked Read 3 Sent Done Done false
7521 3022 Acked Read 1 Sent Done Done false
7522 3023 Acked Read 6 Sent Done Done false
7523 3024 Acked Write 3 Sent Done Done false
7524 3025 Acked Read 4 Sent Done Done false
7525 3026 Acked Write 10 Sent Done Done false
7526 3027 Acked Flush 0 Sent Done Done false
7527 3028 Acked Read 6 Sent Done Done false
7528 3029 Acked Read 5 Sent Done Done false
7529 3030 Acked Write 9 Sent Done Done false
7530 3031 Acked Write 3 Sent Done Done false
7531 3032 Acked Write 10 Sent Done Done false
7532 3033 Acked Read 4 Sent Done Done false
7533 3034 Acked Read 9 Sent Done Done false
7534 3035 Acked Write 1 Sent Done Done false
7535 3036 Acked Flush 0 Sent Done Done false
7536 3037 Acked Read 4 Sent Done Done false
7537 3038 Acked Write 2 Sent Done Done false
7538 3039 Acked Read 5 Sent Done Done false
7539 3040 Acked Write 6 Sent Done Done false
7540 3041 Acked Read 1 Sent Done Done false
7541 3042 Acked Write 4 Sent Done Done false
7542 3043 Acked Write 4 Sent Done Done false
7543 3044 Acked Flush 0 Sent Done Done false
7544 3045 Acked Write 4 Sent Done Done false
7545 3046 Acked Write 8 Sent Done Done false
7546 3047 Acked Read 4 Sent Done Done false
7547 3048 Acked Read 5 Sent Done Done false
7548 3049 Acked Read 10 Sent Done Done false
7549 3050 Acked Read 7 Sent Done Done false
7550 3051 Acked Read 2 Sent Done Done false
7551 3052 Acked Read 2 Sent Done Done false
7552 3053 Acked Read 7 Sent Done Done false
7553 3054 Acked Flush 0 Sent Done Done false
7554 3055 Acked Read 2 Sent Done Done false
7555 3056 Acked Read 7 Sent Done Done false
7556 3057 Acked Read 2 Sent Done Done false
7557 3058 Acked Read 1 Sent Done Done false
7558 3059 Acked Read 8 Sent Done Done false
7559 3060 Acked Write 4 Sent Done Done false
7560 3061 Acked Read 10 Sent Done Done false
7561 3062 Acked Flush 0 Sent Done Done false
7562 3063 Acked Write 5 Sent Done Done false
7563 3064 Acked Write 10 Sent Done Done false
7564 3065 Acked Flush 0 Sent Done Done false
7565 3066 Acked Read 2 Sent Done Done false
7566 3067 Acked Read 8 Sent Done Done false
7567 3068 Acked Write 2 Sent Done Done false
7568 3069 Acked Flush 0 Sent Done Done false
7569 3070 Acked Read 10 Sent Done Done false
7570 3071 Acked Read 3 Sent Done Done false
7571 3072 Acked Read 3 Sent Done Done false
7572 3073 Acked Write 5 Sent Done Done false
7573 3074 Acked Write 5 Sent Done Done false
7574 3075 Acked Read 5 Sent Done Done false
7575 3076 Acked Read 4 Sent Done Done false
7576 3077 Acked Write 8 Sent Done Done false
7577 3078 Acked Read 3 Sent Done Done false
7578 3079 Acked Read 6 Sent Done Done false
7579 3080 Acked Read 2 Sent Done Done false
7580 3081 Acked Read 7 Sent Done Done false
7581 3082 Acked Write 5 Sent Done Done false
7582 3083 Acked Read 8 Sent Done Done false
7583 3084 Acked Read 1 Sent Done Done false
7584 3085 Acked Write 10 Sent Done Done false
7585 3086 Acked Flush 0 Sent Done Done false
7586 3087 Acked Write 10 Sent Done Done false
7587 3088 Acked Write 7 Sent Done Done false
7588 3089 Acked Read 10 Sent Done Done false
7589 3090 Acked Read 6 Sent Done Done false
7590 3091 Acked Write 1 Sent Done Done false
7591 3092 Acked Write 2 Sent Done Done false
7592 3093 Acked Read 4 Sent Done Done false
7593 3094 Acked Flush 0 Sent Done Done false
7594 3095 Acked Read 1 Sent Done Done false
7595 3096 Acked Read 8 Sent Done Done false
7596 3097 Acked Write 9 Sent Done Done false
7597 3098 Acked Read 6 Sent Done Done false
7598 3099 Acked Write 3 Sent Done Done false
7599 3100 Acked Write 4 Sent Done Done false
7600 3101 Acked Read 5 Sent Done Done false
7601 3102 Acked Read 6 Sent Done Done false
7602 3103 Acked Read 5 Sent Done Done false
7603 3104 Acked Write 8 Sent Done Done false
7604 3105 Acked Write 3 Sent Done Done false
7605 3106 Acked Write 4 Sent Done Done false
7606 3107 Acked Read 6 Sent Done Done false
7607 3108 Acked Write 9 Sent Done Done false
7608 3109 Acked Write 6 Sent Done Done false
7609 3110 Acked Read 6 Sent Done Done false
7610 3111 Acked Flush 0 Sent Done Done false
7611 STATES DS:0 DS:1 DS:2 TOTAL
7612 Sent 301 0 0 301
7613 Done 0 301 301 602
7614 Skipped 0 0 0 0
7615 Error 0 0 0 0
7616 Last Flush: 2810 3111 3111
7617 Downstairs last five completed: 2810 2809 2808 2807 2806
7618 Upstairs last five completed: 3111 3110 3109 3108 3107
7619 CLIENT: Up:0 ds:301 act:2
76202025-01-27T19:37:56.402ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76212025-01-27T19:37:56.402ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:36058) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76222025-01-27T19:37:56.403ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76232025-01-27T19:37:56.403ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76242025-01-27T19:37:56.403ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76252025-01-27T19:37:56.404ZINFOcrucible: send last flush ID to this DS: 2810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76262025-01-27T19:37:56.404ZINFOcrucible: Replied this last flush ID: 2810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76272025-01-27T19:37:56.404ZINFOcrucible: [0] client re-new 301 jobs since flush 2810 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76282025-01-27T19:37:56.404ZINFOcrucible: [0] Replayed 301 jobs since flush: 2810 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
7629 ----------------------------------------------------------------
7630 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
7631 Downstairs last five completed: 3111 3110 3109 3108 3107
7632 Upstairs last five completed: 3111 3110 3109 3108 3107
7633 CLIENT: Up:0 ds:0 act:3
7634 Replay: All jobs finished
7635 Main task has work to do, go find it
7636 got dsc Stop(2)
7637 stop 2
7638 Jan 27 19:37:58.650[8830] Got stop action so:false kr:false
7639 INFO request completed, latency_us: 358, response_code: 204, uri: /stop/cid/2, method: POST, req_id: a1769050-cc4e-45d7-8611-7203ddc2db3f, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
76402025-01-27T19:37:58.651ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76412025-01-27T19:37:58.651ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
76422025-01-27T19:37:58.651ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
7643 Jan 27 19:37:58.651 INFO request completed, latency_us: 170, response_code: 200, uri: /state/cid/2, method: GET, req_id: b7c7087b-a422-4035-9ab5-218bdfb28bd7, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
76442025-01-27T19:37:58.651ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
7645 [8830] Exited with: ExitStatus(unix_wait_status(9))
7646 [8830][2] reports Exit
7647 Jan 27 19:38:02.653 INFO request completed, latency_us: 189, response_code: 200, uri: /state/cid/2, method: GET, req_id: cf50052b-bc81-40c2-ba21-b3be849d8f51, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
7648 001/300 Flush
7649 002/300 Read block 4628 len 16384
7650 003/300 Write block 3211 len 12288 data: 2 2 2
7651 004/300 Write block 2390 len 8192 data: 3 4
7652 005/300 Read block 3152 len 4096
7653 006/300 Write block 1669 len 24576 data: 2 2 2 3 3 4
7654 007/300 Read block 2496 len 32768
7655 008/300 Read block 2204 len 20480
7656 009/300 Write block 3687 len 32768 data: 2 3 3 3 3 3 3 3
7657 010/300 Write block 4475 len 16384 data: 2 2 2 2
7658 011/300 Flush
7659 012/300 Flush
7660 013/300 Flush
7661 014/300 Read block 3150 len 24576
7662 015/300 Write block 2951 len 28672 data: 3 3 3 3 3 3 2
7663 016/300 Read block 350 len 4096
7664 017/300 Write block 1522 len 16384 data: 2 2 2 2
7665 018/300 Read block 1356 len 12288
7666 019/300 Read block 4521 len 12288
7667 020/300 Read block 1467 len 40960
7668 021/300 Read block 216 len 4096
7669 022/300 Read block 1786 len 8192
7670 023/300 Flush
7671 024/300 Flush
7672 025/300 Read block 4131 len 36864
7673 026/300 Read block 3629 len 8192
7674 027/300 Read block 3488 len 12288
7675 028/300 Write block 2276 len 12288 data: 2 2 2
7676 029/300 Write block 4344 len 36864 data: 2 2 2 2 2 2 2 2 2
7677 030/300 Write block 2231 len 4096 data: 3
7678 031/300 Read block 723 len 40960
7679 032/300 Write block 352 len 28672 data: 4 4 4 4 4 3 3
7680 033/300 Read block 338 len 8192
7681 034/300 Write block 3434 len 32768 data: 2 2 2 2 2 2 2 2
7682 035/300 Write block 4658 len 4096 data: 4
7683 036/300 Write block 1384 len 20480 data: 3 2 2 2 2
7684 037/300 Write block 980 len 4096 data: 3
7685 038/300 Read block 970 len 36864
7686 039/300 Read block 3776 len 4096
7687 040/300 Read block 4737 len 36864
7688 041/300 Write block 3290 len 28672 data: 2 2 3 4 4 4 4
7689 042/300 Read block 1075 len 4096
7690 043/300 Write block 2167 len 20480 data: 4 4 4 3 2
7691 044/300 Read block 1240 len 36864
7692 045/300 Write block 4139 len 24576 data: 3 3 4 4 4 4
7693 046/300 Read block 3991 len 4096
7694 047/300 Read block 320 len 8192
7695 048/300 Read block 731 len 8192
7696 049/300 Read block 369 len 36864
7697 050/300 Write block 3950 len 12288 data: 2 2 3
7698 051/300 Write block 861 len 32768 data: 3 3 2 2 2 4 4 5
7699 052/300 Write block 1389 len 24576 data: 2 2 3 3 3 3
7700 053/300 Read block 4739 len 8192
7701 054/300 Read block 4310 len 24576
7702 055/300 Read block 564 len 12288
7703 056/300 Write block 58 len 12288 data: 2 2 2
7704 057/300 Read block 475 len 12288
7705 058/300 Write block 2340 len 8192 data: 5 4
7706 059/300 Write block 2234 len 32768 data: 2 2 2 2 2 3 3 3
7707 060/300 Read block 12 len 32768
7708 061/300 Write block 4810 len 28672 data: 5 4 2 2 2 2 2
7709 062/300 Read block 3604 len 12288
7710 063/300 Write block 2983 len 20480 data: 5 4 4 4 4
7711 064/300 Read block 706 len 8192
7712 065/300 Write block 4245 len 36864 data: 5 6 6 6 6 6 4 3 3
7713 066/300 Write block 3759 len 28672 data: 4 4 3 3 2 2 2
7714 067/300 Flush
7715 068/300 Write block 279 len 16384 data: 5 7 7 6
7716 069/300 Flush
7717 070/300 Read block 2436 len 24576
7718 071/300 Read block 1931 len 28672
7719 072/300 Write block 1148 len 8192 data: 3 3
7720 073/300 Read block 4492 len 16384
7721 074/300 Read block 2954 len 4096
7722 075/300 Read block 3821 len 8192
7723 076/300 Read block 1193 len 36864
7724 077/300 Write block 2152 len 40960 data: 2 2 3 3 3 3 3 3 3 4
7725 078/300 Write block 259 len 24576 data: 2 2 2 2 2 2
7726 079/300 Read block 4055 len 24576
7727 080/300 Read block 2752 len 28672
7728 081/300 Read block 448 len 16384
7729 082/300 Read block 2151 len 16384
7730 083/300 Read block 3785 len 16384
7731 084/300 Write block 2097 len 12288 data: 2 2 3
7732 085/300 Write block 1318 len 24576 data: 3 3 3 3 2 2
7733 086/300 Read block 2724 len 40960
7734 087/300 Read block 2884 len 36864
7735 088/300 Write block 2491 len 36864 data: 2 2 3 3 2 2 2 2 2
7736 089/300 Read block 4157 len 20480
7737 090/300 Read block 3779 len 24576
7738 091/300 Write block 4406 len 32768 data: 3 3 3 2 2 3 3 4
7739 092/300 Write block 1463 len 20480 data: 2 3 3 3 3
7740 093/300 Read block 3786 len 16384
7741 094/300 Write block 1167 len 28672 data: 3 3 3 3 3 3 2
7742 095/300 Write block 564 len 32768 data: 4 3 2 2 3 3 3 3
7743 096/300 Read block 3717 len 24576
7744 097/300 Flush
7745 098/300 Write block 297 len 16384 data: 3 3 3 4
7746 099/300 Write block 1636 len 28672 data: 2 2 2 2 2 3 3
7747 100/300 Write block 3346 len 20480 data: 2 2 2 2 2
7748 101/300 Read block 3775 len 40960
7749 102/300 Read block 4277 len 4096
7750 103/300 Read block 4449 len 40960
7751 104/300 Flush
7752 105/300 Write block 1128 len 28672 data: 3 4 4 4 4 3 3
7753 106/300 Write block 3707 len 36864 data: 2 4 4 4 4 3 3 3 3
7754 107/300 Write block 22 len 16384 data: 2 2 2 2
7755 108/300 Flush
7756 109/300 Write block 2844 len 20480 data: 4 4 4 2 3
7757 110/300 Read block 1979 len 28672
7758 111/300 Read block 3610 len 36864
7759 112/300 Write block 4164 len 28672 data: 3 4 3 3 4 4 4
7760 113/300 Write block 967 len 40960 data: 2 2 2 2 2 2 2 3 3 3
7761 114/300 Write block 1221 len 4096 data: 4
7762 115/300 Read block 1751 len 24576
7763 116/300 Write block 781 len 16384 data: 2 2 2 2
7764 117/300 Read block 3922 len 24576
7765 118/300 Write block 1726 len 16384 data: 3 3 2 2
7766 119/300 Write block 3096 len 36864 data: 2 2 2 2 2 2 2 2 2
7767 120/300 Write block 2962 len 32768 data: 2 2 2 2 2 2 2 3
7768 121/300 Write block 3091 len 12288 data: 3 3 3
7769 122/300 Read block 1625 len 24576
7770 123/300 Read block 2572 len 28672
7771 124/300 Write block 4534 len 8192 data: 3 3
7772 125/300 Write block 3761 len 32768 data: 4 4 3 3 3 3 3 3
7773 126/300 Write block 715 len 8192 data: 6 5
7774 127/300 Read block 4018 len 8192
7775 128/300 Write block 4842 len 20480 data: 3 3 3 3 3
7776 129/300 Read block 1531 len 28672
7777 130/300 Write block 3479 len 16384 data: 2 2 2 2
7778 131/300 Write block 2171 len 36864 data: 3 2 3 3 4 4 4 4 4
7779 132/300 Write block 3873 len 20480 data: 3 3 3 3 3
7780 133/300 Flush
7781 134/300 Read block 4432 len 36864
7782 135/300 Read block 2921 len 4096
7783 136/300 Read block 2923 len 16384
7784 137/300 Read block 2768 len 20480
7785 138/300 Read block 4689 len 24576
7786 139/300 Write block 1438 len 20480 data: 3 3 3 3 3
7787 140/300 Read block 79 len 20480
7788 141/300 Read block 1266 len 28672
7789 142/300 Write block 1004 len 24576 data: 2 2 2 2 2 3
7790 143/300 Write block 1654 len 40960 data: 2 2 2 2 2 2 2 2 2 2
7791 144/300 Write block 1394 len 40960 data: 4 3 2 2 4 4 4 4 3 3
7792 145/300 Write block 9 len 36864 data: 2 2 2 2 3 3 3 4 4
7793 146/300 Read block 4750 len 28672
7794 147/300 Read block 47 len 16384
7795 148/300 Write block 2441 len 12288 data: 5 5 5
7796 149/300 Write block 143 len 24576 data: 2 2 2 2 2 3
7797 150/300 Write block 138 len 8192 data: 2 2
7798 151/300 Write block 3721 len 28672 data: 3 3 3 3 4 4 5
7799 152/300 Flush
7800 153/300 Read block 1187 len 20480
7801 154/300 Read block 3744 len 4096
7802 155/300 Read block 293 len 40960
7803 156/300 Read block 3424 len 8192
7804 157/300 Write block 4671 len 24576 data: 3 3 3 2 2 2
7805 158/300 Read block 1355 len 16384
7806 159/300 Read block 4847 len 32768
7807 160/300 Flush
7808 161/300 Flush
7809 162/300 Write block 1079 len 16384 data: 3 3 2 2
7810 163/300 Read block 719 len 12288
7811 164/300 Read block 4944 len 12288
7812 165/300 Read block 4036 len 36864
7813 166/300 Flush
7814 167/300 Read block 1790 len 20480
7815 168/300 Read block 4934 len 36864
7816 169/300 Write block 1666 len 4096 data: 2
7817 170/300 Write block 1136 len 8192 data: 2 2
7818 171/300 Read block 233 len 20480
7819 172/300 Read block 3521 len 16384
7820 173/300 Write block 3239 len 36864 data: 3 3 3 4 4 2 2 2 2
7821 174/300 Write block 40 len 4096 data: 3
7822 175/300 Write block 3414 len 12288 data: 2 2 2
7823 176/300 Read block 3048 len 32768
7824 177/300 Write block 713 len 4096 data: 8
7825 178/300 Flush
7826 179/300 Read block 2389 len 20480
7827 180/300 Read block 2422 len 28672
7828 181/300 Read block 1679 len 4096
7829 182/300 Write block 2857 len 40960 data: 3 3 3 3 3 3 3 3 3 3
7830 183/300 Read block 574 len 8192
7831 184/300 Write block 2383 len 40960 data: 2 2 3 3 3 3 3 4 5 3
7832 185/300 Flush
7833 186/300 Write block 4009 len 8192 data: 4 5
7834 187/300 Read block 431 len 8192
7835 188/300 Write block 3671 len 4096 data: 4
7836 189/300 Read block 1278 len 20480
7837 190/300 Read block 4418 len 32768
7838 191/300 Write block 2141 len 16384 data: 2 2 2 3
7839 192/300 Read block 1927 len 36864
7840 193/300 Read block 1205 len 28672
7841 194/300 Read block 1206 len 28672
7842 195/300 Read block 4478 len 28672
7843 196/300 Write block 3867 len 16384 data: 3 3 2 2
7844 197/300 Write block 3172 len 8192 data: 2 2
7845 198/300 Write block 460 len 24576 data: 2 2 2 2 2 2
7846 199/300 Read block 2637 len 28672
7847 200/300 Read block 409 len 4096
7848 201/300 Write block 4962 len 4096 data: 5
7849 202/300 Write block 3138 len 36864 data: 5 5 6 6 5 5 3 3 3
7850 203/300 Write block 371 len 28672 data: 3 4 4 4 3 3 3
7851 204/300 Read block 798 len 4096
7852 205/300 Read block 2175 len 8192
7853 206/300 Read block 4632 len 36864
7854 207/300 Write block 1292 len 40960 data: 2 2 2 2 2 3 3 3 3 4
7855 208/300 Write block 478 len 4096 data: 3
7856 209/300 Read block 3085 len 8192
7857 210/300 Read block 4592 len 28672
7858 211/300 Write block 2651 len 36864 data: 2 2 2 2 2 2 2 3 3
7859 212/300 Read block 4083 len 32768
7860 213/300 Read block 1152 len 8192
7861 214/300 Write block 4869 len 4096 data: 3
7862 215/300 Write block 3329 len 8192 data: 3 3
7863 216/300 Read block 4438 len 36864
7864 217/300 Flush
7865 218/300 Write block 2849 len 40960 data: 3 3 4 4 4 3 3 3 4 4
7866 219/300 Write block 4784 len 16384 data: 2 3 3 3
7867 220/300 Read block 3990 len 8192
7868 221/300 Write block 176 len 40960 data: 3 3 2 2 2 2 2 2 3 3
7869 222/300 Read block 3779 len 36864
7870 223/300 Read block 169 len 20480
7871 224/300 Read block 368 len 12288
7872 225/300 Write block 4424 len 24576 data: 2 2 2 3 3 3
7873 226/300 Read block 4798 len 8192
7874 227/300 Write block 846 len 24576 data: 2 2 2 2 2 2
7875 228/300 Read block 117 len 28672
7876 229/300 Write block 4959 len 24576 data: 4 5 5 6 4 4
7877 230/300 Write block 463 len 36864 data: 3 3 3 2 2 2 2 2 2
7878 231/300 Read block 3241 len 20480
7879 232/300 Flush
7880 233/300 Flush
7881 234/300 Read block 3140 len 16384
7882 235/300 Read block 2830 len 20480
7883 236/300 Write block 4435 len 4096 data: 3
7884 237/300 Write block 956 len 8192 data: 5 5
7885 238/300 Write block 199 len 24576 data: 4 4 3 2 2 2
7886 239/300 Read block 4451 len 20480
7887 240/300 Write block 205 len 20480 data: 3 4 4 4 4
7888 241/300 Write block 601 len 4096 data: 3
7889 242/300 Read block 3471 len 32768
7890 243/300 Read block 3126 len 24576
7891 244/300 Write block 3878 len 40960 data: 3 2 3 3 3 3 3 3 3 3
7892 245/300 Read block 409 len 36864
7893 246/300 Read block 938 len 4096
7894 247/300 Read block 1984 len 16384
7895 248/300 Read block 197 len 36864
7896 249/300 Read block 931 len 4096
7897 250/300 Write block 1138 len 24576 data: 2 2 2 2 2 2
7898 251/300 Write block 2649 len 40960 data: 2 2 3 3 3 3 3 3 3 4
7899 252/300 Write block 4613 len 32768 data: 2 2 3 3 3 3 3 2
7900 253/300 Read block 632 len 12288
7901 254/300 Flush
7902 255/300 Write block 3772 len 24576 data: 3 5 5 5 5 5
7903 256/300 Write block 1358 len 40960 data: 4 4 4 5 4 4 4 4 4 4
7904 257/300 Write block 279 len 12288 data: 6 8 8
7905 258/300 Write block 294 len 28672 data: 3 3 3 4 4 4 5
7906 259/300 Flush
7907 260/300 Write block 4655 len 12288 data: 3 4 4
7908 261/300 Read block 4144 len 20480
7909 262/300 Read block 1945 len 32768
7910 263/300 Read block 2930 len 32768
7911 264/300 Read block 1974 len 16384
7912 265/300 Write block 537 len 12288 data: 3 3 2
7913 266/300 Flush
7914 267/300 Write block 4335 len 20480 data: 4 5 5 5 4
7915 268/300 Read block 899 len 20480
7916 269/300 Flush
7917 270/300 Read block 4042 len 4096
7918 271/300 Write block 475 len 32768 data: 3 3 3 4 3 3 3 3
7919 272/300 Write block 138 len 16384 data: 3 3 2 2
7920 273/300 Read block 21 len 20480
7921 274/300 Write block 3105 len 8192 data: 2 2
7922 275/300 Write block 1879 len 32768 data: 3 2 2 2 2 2 2 3
7923 276/300 Read block 3325 len 20480
7924 277/300 Read block 774 len 12288
7925 278/300 Flush
7926 279/300 Read block 4874 len 24576
7927 280/300 Read block 2431 len 12288
7928 281/300 Read block 1105 len 40960
7929 282/300 Read block 1752 len 20480
7930 283/300 Read block 1659 len 8192
7931 284/300 Write block 3354 len 36864 data: 2 2 2 2 2 3 3 3 3
7932 285/300 Write block 1662 len 12288 data: 3 3 2
7933 286/300 Write block 3246 len 4096 data: 3
7934 287/300 Read block 1050 len 20480
7935 288/300 Read block 3493 len 32768
7936 289/300 Read block 1652 len 12288
7937 290/300 Write block 1405 len 12288 data: 3 3 2
7938 291/300 Write block 1811 len 8192 data: 6 5
7939 292/300 Write block 924 len 32768 data: 3 3 3 2 3 3 4 4
7940 293/300 Write block 832 len 20480 data: 4 3 3 3 2
7941 294/300 Read block 810 len 32768
7942 295/300 Write block 1059 len 8192 data: 3 3
7943 296/300 Write block 2608 len 4096 data: 3
7944 297/300 Read block 1446 len 32768
7945 298/300 Read block 2396 len 20480
7946 299/300 Write block 1353 len 36864 data: 3 3 3 3 3 5 5 5 6
7947 300/300 Read block 903 len 40960
7948 301/300 Read block 4299 len 36864
7949 Main task has work to do, go find it
7950 got dsc Start(2)
7951 start 2
7952 Jan 27 19:38:02.881 INFO request completed, latency_us: 187, response_code: 204, uri: /start/cid/2, method: POST, req_id: 71bf678c-629e-4503-85e5-7859c18af32d, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
7953 Replay: started 2, returned:Ok(())
7954 ----------------------------------------------------------------
7955 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
7956 States: Active Active Offline
7957 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
7958 3112 Acked Flush 0 Done Done Sent false
7959 3113 Acked Read 4 Done Done Sent false
7960 3114 Acked Write 3 Done Done Sent false
7961 3115 Acked Write 2 Done Done Sent false
7962 3116 Acked Read 1 Done Done Sent false
7963 3117 Acked Write 6 Done Done Sent false
7964 3118 Acked Read 8 Done Done Sent false
7965 3119 Acked Read 5 Done Done Sent false
7966 3120 Acked Write 8 Done Done Sent false
7967 3121 Acked Write 4 Done Done Sent false
7968 3122 Acked Flush 0 Done Done Sent false
7969 3123 Acked Flush 0 Done Done Sent false
7970 3124 Acked Flush 0 Done Done Sent false
7971 3125 Acked Read 6 Done Done Sent false
7972 3126 Acked Write 7 Done Done Sent false
7973 3127 Acked Read 1 Done Done Sent false
7974 3128 Acked Write 4 Done Done Sent false
7975 3129 Acked Read 3 Done Done Sent false
7976 3130 Acked Read 3 Done Done Sent false
7977 3131 Acked Read 10 Done Done Sent false
7978 3132 Acked Read 1 Done Done Sent false
7979 3133 Acked Read 2 Done Done Sent false
7980 3134 Acked Flush 0 Done Done Sent false
7981 3135 Acked Flush 0 Done Done Sent false
7982 3136 Acked Read 9 Done Done Sent false
7983 3137 Acked Read 2 Done Done Sent false
7984 3138 Acked Read 3 Done Done Sent false
7985 3139 Acked Write 3 Done Done Sent false
7986 3140 Acked Write 9 Done Done Sent false
7987 3141 Acked Write 1 Done Done Sent false
7988 3142 Acked Read 10 Done Done Sent false
7989 3143 Acked Write 7 Done Done Sent false
7990 3144 Acked Read 2 Done Done Sent false
7991 3145 Acked Write 8 Done Done Sent false
7992 3146 Acked Write 1 Done Done Sent false
7993 3147 Acked Write 5 Done Done Sent false
7994 3148 Acked Write 1 Done Done Sent false
7995 3149 Acked Read 9 Done Done Sent false
7996 3150 Acked Read 1 Done Done Sent false
7997 3151 Acked Read 9 Done Done Sent false
7998 3152 Acked Write 7 Done Done Sent false
7999 3153 Acked Read 1 Done Done Sent false
8000 3154 Acked Write 5 Done Done Sent false
8001 3155 Acked Read 9 Done Done Sent false
8002 3156 Acked Write 6 Done Done Sent false
8003 3157 Acked Read 1 Done Done Sent false
8004 3158 Acked Read 2 Done Done Sent false
8005 3159 Acked Read 2 Done Done Sent false
8006 3160 Acked Read 9 Done Done Sent false
8007 3161 Acked Write 3 Done Done Sent false
8008 3162 Acked Write 8 Done Done Sent false
8009 3163 Acked Write 6 Done Done Sent false
8010 3164 Acked Read 2 Done Done Sent false
8011 3165 Acked Read 6 Done Done Sent false
8012 3166 Acked Read 3 Done Done Sent false
8013 3167 Acked Write 3 Done Done Sent false
8014 3168 Acked Read 3 Done Done Sent false
8015 3169 Acked Write 2 Done Done Sent false
8016 3170 Acked Write 8 Done Done Sent false
8017 3171 Acked Read 8 Done Done Sent false
8018 3172 Acked Write 7 Done Done Sent false
8019 3173 Acked Read 3 Done Done Sent false
8020 3174 Acked Write 5 Done Done Sent false
8021 3175 Acked Read 2 Done Done Sent false
8022 3176 Acked Write 9 Done Done Sent false
8023 3177 Acked Write 7 Done Done Sent false
8024 3178 Acked Flush 0 Done Done Sent false
8025 3179 Acked Write 4 Done Done Sent false
8026 3180 Acked Flush 0 Done Done Sent false
8027 3181 Acked Read 6 Done Done Sent false
8028 3182 Acked Read 7 Done Done Sent false
8029 3183 Acked Write 2 Done Done Sent false
8030 3184 Acked Read 4 Done Done Sent false
8031 3185 Acked Read 1 Done Done Sent false
8032 3186 Acked Read 2 Done Done Sent false
8033 3187 Acked Read 9 Done Done Sent false
8034 3188 Acked Write 10 Done Done Sent false
8035 3189 Acked Write 6 Done Done Sent false
8036 3190 Acked Read 6 Done Done Sent false
8037 3191 Acked Read 7 Done Done Sent false
8038 3192 Acked Read 4 Done Done Sent false
8039 3193 Acked Read 4 Done Done Sent false
8040 3194 Acked Read 4 Done Done Sent false
8041 3195 Acked Write 3 Done Done Sent false
8042 3196 Acked Write 6 Done Done Sent false
8043 3197 Acked Read 10 Done Done Sent false
8044 3198 Acked Read 9 Done Done Sent false
8045 3199 Acked Write 9 Done Done Sent false
8046 3200 Acked Read 5 Done Done Sent false
8047 3201 Acked Read 6 Done Done Sent false
8048 3202 Acked Write 8 Done Done Sent false
8049 3203 Acked Write 5 Done Done Sent false
8050 3204 Acked Read 4 Done Done Sent false
8051 3205 Acked Write 7 Done Done Sent false
8052 3206 Acked Write 8 Done Done Sent false
8053 3207 Acked Read 6 Done Done Sent false
8054 3208 Acked Flush 0 Done Done Sent false
8055 3209 Acked Write 4 Done Done Sent false
8056 3210 Acked Write 7 Done Done Sent false
8057 3211 Acked Write 5 Done Done Sent false
8058 3212 Acked Read 10 Done Done Sent false
8059 3213 Acked Read 1 Done Done Sent false
8060 3214 Acked Read 10 Done Done Sent false
8061 3215 Acked Flush 0 Done Done Sent false
8062 3216 Acked Write 7 Done Done Sent false
8063 3217 Acked Write 9 Done Done Sent false
8064 3218 Acked Write 4 Done Done Sent false
8065 3219 Acked Flush 0 Done Done Sent false
8066 3220 Acked Write 5 Done Done Sent false
8067 3221 Acked Read 7 Done Done Sent false
8068 3222 Acked Read 9 Done Done Sent false
8069 3223 Acked Write 7 Done Done Sent false
8070 3224 Acked Write 10 Done Done Sent false
8071 3225 Acked Write 1 Done Done Sent false
8072 3226 Acked Read 6 Done Done Sent false
8073 3227 Acked Write 4 Done Done Sent false
8074 3228 Acked Read 6 Done Done Sent false
8075 3229 Acked Write 4 Done Done Sent false
8076 3230 Acked Write 9 Done Done Sent false
8077 3231 Acked Write 8 Done Done Sent false
8078 3232 Acked Write 3 Done Done Sent false
8079 3233 Acked Read 6 Done Done Sent false
8080 3234 Acked Read 7 Done Done Sent false
8081 3235 Acked Write 2 Done Done Sent false
8082 3236 Acked Write 8 Done Done Sent false
8083 3237 Acked Write 2 Done Done Sent false
8084 3238 Acked Read 2 Done Done Sent false
8085 3239 Acked Write 5 Done Done Sent false
8086 3240 Acked Read 7 Done Done Sent false
8087 3241 Acked Write 4 Done Done Sent false
8088 3242 Acked Write 9 Done Done Sent false
8089 3243 Acked Write 5 Done Done Sent false
8090 3244 Acked Flush 0 Done Done Sent false
8091 3245 Acked Read 9 Done Done Sent false
8092 3246 Acked Read 1 Done Done Sent false
8093 3247 Acked Read 4 Done Done Sent false
8094 3248 Acked Read 5 Done Done Sent false
8095 3249 Acked Read 6 Done Done Sent false
8096 3250 Acked Write 5 Done Done Sent false
8097 3251 Acked Read 5 Done Done Sent false
8098 3252 Acked Read 7 Done Done Sent false
8099 3253 Acked Write 6 Done Done Sent false
8100 3254 Acked Write 10 Done Done Sent false
8101 3255 Acked Write 10 Done Done Sent false
8102 3256 Acked Write 9 Done Done Sent false
8103 3257 Acked Read 7 Done Done Sent false
8104 3258 Acked Read 4 Done Done Sent false
8105 3259 Acked Write 3 Done Done Sent false
8106 3260 Acked Write 6 Done Done Sent false
8107 3261 Acked Write 2 Done Done Sent false
8108 3262 Acked Write 7 Done Done Sent false
8109 3263 Acked Flush 0 Done Done Sent false
8110 3264 Acked Read 5 Done Done Sent false
8111 3265 Acked Read 1 Done Done Sent false
8112 3266 Acked Read 10 Done Done Sent false
8113 3267 Acked Read 2 Done Done Sent false
8114 3268 Acked Write 6 Done Done Sent false
8115 3269 Acked Read 4 Done Done Sent false
8116 3270 Acked Read 8 Done Done Sent false
8117 3271 Acked Flush 0 Done Done Sent false
8118 3272 Acked Flush 0 Done Done Sent false
8119 3273 Acked Write 4 Done Done Sent false
8120 3274 Acked Read 3 Done Done Sent false
8121 3275 Acked Read 3 Done Done Sent false
8122 3276 Acked Read 9 Done Done Sent false
8123 3277 Acked Flush 0 Done Done Sent false
8124 3278 Acked Read 5 Done Done Sent false
8125 3279 Acked Read 9 Done Done Sent false
8126 3280 Acked Write 1 Done Done Sent false
8127 3281 Acked Write 2 Done Done Sent false
8128 3282 Acked Read 5 Done Done Sent false
8129 3283 Acked Read 4 Done Done Sent false
8130 3284 Acked Write 9 Done Done Sent false
8131 3285 Acked Write 1 Done Done Sent false
8132 3286 Acked Write 3 Done Done Sent false
8133 3287 Acked Read 8 Done Done Sent false
8134 3288 Acked Write 1 Done Done Sent false
8135 3289 Acked Flush 0 Done Done Sent false
8136 3290 Acked Read 5 Done Done Sent false
8137 3291 Acked Read 7 Done Done Sent false
8138 3292 Acked Read 1 Done Done Sent false
8139 3293 Acked Write 10 Done Done Sent false
8140 3294 Acked Read 2 Done Done Sent false
8141 3295 Acked Write 10 Done Done Sent false
8142 3296 Acked Flush 0 Done Done Sent false
8143 3297 Acked Write 2 Done Done Sent false
8144 3298 Acked Read 2 Done Done Sent false
8145 3299 Acked Write 1 Done Done Sent false
8146 3300 Acked Read 5 Done Done Sent false
8147 3301 Acked Read 8 Done Done Sent false
8148 3302 Acked Write 4 Done Done Sent false
8149 3303 Acked Read 9 Done Done Sent false
8150 3304 Acked Read 7 Done Done Sent false
8151 3305 Acked Read 7 Done Done Sent false
8152 3306 Acked Read 7 Done Done Sent false
8153 3307 Acked Write 4 Done Done Sent false
8154 3308 Acked Write 2 Done Done Sent false
8155 3309 Acked Write 6 Done Done Sent false
8156 3310 Acked Read 7 Done Done Sent false
8157 3311 Acked Read 1 Done Done Sent false
8158 3312 Acked Write 1 Done Done Sent false
8159 3313 Acked Write 9 Done Done Sent false
8160 3314 Acked Write 7 Done Done Sent false
8161 3315 Acked Read 1 Done Done Sent false
8162 3316 Acked Read 2 Done Done Sent false
8163 3317 Acked Read 9 Done Done Sent false
8164 3318 Acked Write 10 Done Done Sent false
8165 3319 Acked Write 1 Done Done Sent false
8166 3320 Acked Read 2 Done Done Sent false
8167 3321 Acked Read 7 Done Done Sent false
8168 3322 Acked Write 9 Done Done Sent false
8169 3323 Acked Read 8 Done Done Sent false
8170 3324 Acked Read 2 Done Done Sent false
8171 3325 Acked Write 1 Done Done Sent false
8172 3326 Acked Write 2 Done Done Sent false
8173 3327 Acked Read 9 Done Done Sent false
8174 3328 Acked Flush 0 Done Done Sent false
8175 3329 Acked Write 10 Done Done Sent false
8176 3330 Acked Write 4 Done Done Sent false
8177 3331 Acked Read 2 Done Done Sent false
8178 3332 Acked Write 10 Done Done Sent false
8179 3333 Acked Read 9 Done Done Sent false
8180 3334 Acked Read 5 Done Done Sent false
8181 3335 Acked Read 3 Done Done Sent false
8182 3336 Acked Write 6 Done Done Sent false
8183 3337 Acked Read 2 Done Done Sent false
8184 3338 Acked Write 6 Done Done Sent false
8185 3339 Acked Read 7 Done Done Sent false
8186 3340 Acked Write 6 Done Done Sent false
8187 3341 Acked Write 9 Done Done Sent false
8188 3342 Acked Read 5 Done Done Sent false
8189 3343 Acked Flush 0 Done Done Sent false
8190 3344 Acked Flush 0 Done Done Sent false
8191 3345 Acked Read 4 Done Done Sent false
8192 3346 Acked Read 5 Done Done Sent false
8193 3347 Acked Write 1 Done Done Sent false
8194 3348 Acked Write 2 Done Done Sent false
8195 3349 Acked Write 6 Done Done Sent false
8196 3350 Acked Read 5 Done Done Sent false
8197 3351 Acked Write 5 Done Done Sent false
8198 3352 Acked Write 1 Done Done Sent false
8199 3353 Acked Read 8 Done Done Sent false
8200 3354 Acked Read 6 Done Done Sent false
8201 3355 Acked Write 10 Done Done Sent false
8202 3356 Acked Read 9 Done Done Sent false
8203 3357 Acked Read 1 Done Done Sent false
8204 3358 Acked Read 4 Done Done Sent false
8205 3359 Acked Read 9 Done Done Sent false
8206 3360 Acked Read 1 Done Done Sent false
8207 3361 Acked Write 6 Done Done Sent false
8208 3362 Acked Write 10 Done Done Sent false
8209 3363 Acked Write 8 Done Done Sent false
8210 3364 Acked Read 3 Done Done Sent false
8211 3365 Acked Flush 0 Done Done Sent false
8212 3366 Acked Write 6 Done Done Sent false
8213 3367 Acked Write 10 Done Done Sent false
8214 3368 Acked Write 3 Done Done Sent false
8215 3369 Acked Write 7 Done Done Sent false
8216 3370 Acked Flush 0 Done Done Sent false
8217 3371 Acked Write 3 Done Done Sent false
8218 3372 Acked Read 5 Done Done Sent false
8219 3373 Acked Read 8 Done Done Sent false
8220 3374 Acked Read 8 Done Done Sent false
8221 3375 Acked Read 4 Done Done Sent false
8222 3376 Acked Write 3 Done Done Sent false
8223 3377 Acked Flush 0 Done Done Sent false
8224 3378 Acked Write 5 Done Done Sent false
8225 3379 Acked Read 5 Done Done Sent false
8226 3380 Acked Flush 0 Done Done Sent false
8227 3381 Acked Read 1 Done Done Sent false
8228 3382 Acked Write 8 Done Done Sent false
8229 3383 Acked Write 4 Done Done Sent false
8230 3384 Acked Read 5 Done Done Sent false
8231 3385 Acked Write 2 Done Done Sent false
8232 3386 Acked Write 8 Done Done Sent false
8233 3387 Acked Read 5 Done Done Sent false
8234 3388 Acked Read 3 Done Done Sent false
8235 3389 Acked Flush 0 Done Done Sent false
8236 3390 Acked Read 6 Done Done Sent false
8237 3391 Acked Read 3 Done Done Sent false
8238 3392 Acked Read 10 Done Done Sent false
8239 3393 Acked Read 5 Done Done Sent false
8240 3394 Acked Read 2 Done Done Sent false
8241 3395 Acked Write 9 Done Done Sent false
8242 3396 Acked Write 3 Done Done Sent false
8243 3397 Acked Write 1 Done Done Sent false
8244 3398 Acked Read 5 Done Done Sent false
8245 3399 Acked Read 8 Done Done Sent false
8246 3400 Acked Read 3 Done Done Sent false
8247 3401 Acked Write 3 Done Done Sent false
8248 3402 Acked Write 2 Done Done Sent false
8249 3403 Acked Write 8 Done Done Sent false
8250 3404 Acked Write 5 Done Done Sent false
8251 3405 Acked Read 8 Done Done Sent false
8252 3406 Acked Write 2 Done Done Sent false
8253 3407 Acked Write 1 Done Done Sent false
8254 3408 Acked Read 8 Done Done Sent false
8255 3409 Acked Read 5 Done Done Sent false
8256 3410 Acked Write 9 Done Done Sent false
8257 3411 Acked Read 10 Done Done Sent false
8258 3412 Acked Read 9 Done Done Sent false
8259 STATES DS:0 DS:1 DS:2 TOTAL
8260 Sent 0 0 301 301
8261 Done 301 301 0 602
8262 Skipped 0 0 0 0
8263 Error 0 0 0 0
8264 Last Flush: 3389 3389 3111
8265 Downstairs last five completed: 3111 3110 3109 3108 3107
8266 Upstairs last five completed: 3412 3411 3410 3409 3408
8267 CLIENT: Up:0 ds:301 act:2
8268 [8830] got start action
8269 [8830] I am going to restart
8270 Starting downstairs at port 8830
8271 Make output file at "/tmp/dsc/downstairs-8830.txt"
8272 [8830][2] reports Starting
8273 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1208)
8274 [8830][2] reports Running
8275 ----------------------------------------------------------------
8276 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
8277 States: Active Active Offline
8278 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8279 3112 Acked Flush 0 Done Done Sent false
8280 3113 Acked Read 4 Done Done Sent false
8281 3114 Acked Write 3 Done Done Sent false
8282 3115 Acked Write 2 Done Done Sent false
8283 3116 Acked Read 1 Done Done Sent false
8284 3117 Acked Write 6 Done Done Sent false
8285 3118 Acked Read 8 Done Done Sent false
8286 3119 Acked Read 5 Done Done Sent false
8287 3120 Acked Write 8 Done Done Sent false
8288 3121 Acked Write 4 Done Done Sent false
8289 3122 Acked Flush 0 Done Done Sent false
8290 3123 Acked Flush 0 Done Done Sent false
8291 3124 Acked Flush 0 Done Done Sent false
8292 3125 Acked Read 6 Done Done Sent false
8293 3126 Acked Write 7 Done Done Sent false
8294 3127 Acked Read 1 Done Done Sent false
8295 3128 Acked Write 4 Done Done Sent false
8296 3129 Acked Read 3 Done Done Sent false
8297 3130 Acked Read 3 Done Done Sent false
8298 3131 Acked Read 10 Done Done Sent false
8299 3132 Acked Read 1 Done Done Sent false
8300 3133 Acked Read 2 Done Done Sent false
8301 3134 Acked Flush 0 Done Done Sent false
8302 3135 Acked Flush 0 Done Done Sent false
8303 3136 Acked Read 9 Done Done Sent false
8304 3137 Acked Read 2 Done Done Sent false
8305 3138 Acked Read 3 Done Done Sent false
8306 3139 Acked Write 3 Done Done Sent false
8307 3140 Acked Write 9 Done Done Sent false
8308 3141 Acked Write 1 Done Done Sent false
8309 3142 Acked Read 10 Done Done Sent false
8310 3143 Acked Write 7 Done Done Sent false
8311 3144 Acked Read 2 Done Done Sent false
8312 3145 Acked Write 8 Done Done Sent false
8313 3146 Acked Write 1 Done Done Sent false
8314 3147 Acked Write 5 Done Done Sent false
8315 3148 Acked Write 1 Done Done Sent false
8316 3149 Acked Read 9 Done Done Sent false
8317 3150 Acked Read 1 Done Done Sent false
8318 3151 Acked Read 9 Done Done Sent false
8319 3152 Acked Write 7 Done Done Sent false
8320 3153 Acked Read 1 Done Done Sent false
8321 3154 Acked Write 5 Done Done Sent false
8322 3155 Acked Read 9 Done Done Sent false
8323 3156 Acked Write 6 Done Done Sent false
8324 3157 Acked Read 1 Done Done Sent false
8325 3158 Acked Read 2 Done Done Sent false
8326 3159 Acked Read 2 Done Done Sent false
8327 3160 Acked Read 9 Done Done Sent false
8328 3161 Acked Write 3 Done Done Sent false
8329 3162 Acked Write 8 Done Done Sent false
8330 3163 Acked Write 6 Done Done Sent false
8331 3164 Acked Read 2 Done Done Sent false
8332 3165 Acked Read 6 Done Done Sent false
8333 3166 Acked Read 3 Done Done Sent false
8334 3167 Acked Write 3 Done Done Sent false
8335 3168 Acked Read 3 Done Done Sent false
8336 3169 Acked Write 2 Done Done Sent false
8337 3170 Acked Write 8 Done Done Sent false
8338 3171 Acked Read 8 Done Done Sent false
8339 3172 Acked Write 7 Done Done Sent false
8340 3173 Acked Read 3 Done Done Sent false
8341 3174 Acked Write 5 Done Done Sent false
8342 3175 Acked Read 2 Done Done Sent false
8343 3176 Acked Write 9 Done Done Sent false
8344 3177 Acked Write 7 Done Done Sent false
8345 3178 Acked Flush 0 Done Done Sent false
8346 3179 Acked Write 4 Done Done Sent false
8347 3180 Acked Flush 0 Done Done Sent false
8348 3181 Acked Read 6 Done Done Sent false
8349 3182 Acked Read 7 Done Done Sent false
8350 3183 Acked Write 2 Done Done Sent false
8351 3184 Acked Read 4 Done Done Sent false
8352 3185 Acked Read 1 Done Done Sent false
8353 3186 Acked Read 2 Done Done Sent false
8354 3187 Acked Read 9 Done Done Sent false
8355 3188 Acked Write 10 Done Done Sent false
8356 3189 Acked Write 6 Done Done Sent false
8357 3190 Acked Read 6 Done Done Sent false
8358 3191 Acked Read 7 Done Done Sent false
8359 3192 Acked Read 4 Done Done Sent false
8360 3193 Acked Read 4 Done Done Sent false
8361 3194 Acked Read 4 Done Done Sent false
8362 3195 Acked Write 3 Done Done Sent false
8363 3196 Acked Write 6 Done Done Sent false
8364 3197 Acked Read 10 Done Done Sent false
8365 3198 Acked Read 9 Done Done Sent false
8366 3199 Acked Write 9 Done Done Sent false
8367 3200 Acked Read 5 Done Done Sent false
8368 3201 Acked Read 6 Done Done Sent false
8369 3202 Acked Write 8 Done Done Sent false
8370 3203 Acked Write 5 Done Done Sent false
8371 3204 Acked Read 4 Done Done Sent false
8372 3205 Acked Write 7 Done Done Sent false
8373 3206 Acked Write 8 Done Done Sent false
8374 3207 Acked Read 6 Done Done Sent false
8375 3208 Acked Flush 0 Done Done Sent false
8376 3209 Acked Write 4 Done Done Sent false
8377 3210 Acked Write 7 Done Done Sent false
8378 3211 Acked Write 5 Done Done Sent false
8379 3212 Acked Read 10 Done Done Sent false
8380 3213 Acked Read 1 Done Done Sent false
8381 3214 Acked Read 10 Done Done Sent false
8382 3215 Acked Flush 0 Done Done Sent false
8383 3216 Acked Write 7 Done Done Sent false
8384 3217 Acked Write 9 Done Done Sent false
8385 3218 Acked Write 4 Done Done Sent false
8386 3219 Acked Flush 0 Done Done Sent false
8387 3220 Acked Write 5 Done Done Sent false
8388 3221 Acked Read 7 Done Done Sent false
8389 3222 Acked Read 9 Done Done Sent false
8390 3223 Acked Write 7 Done Done Sent false
8391 3224 Acked Write 10 Done Done Sent false
8392 3225 Acked Write 1 Done Done Sent false
8393 3226 Acked Read 6 Done Done Sent false
8394 3227 Acked Write 4 Done Done Sent false
8395 3228 Acked Read 6 Done Done Sent false
8396 3229 Acked Write 4 Done Done Sent false
8397 3230 Acked Write 9 Done Done Sent false
8398 3231 Acked Write 8 Done Done Sent false
8399 3232 Acked Write 3 Done Done Sent false
8400 3233 Acked Read 6 Done Done Sent false
8401 3234 Acked Read 7 Done Done Sent false
8402 3235 Acked Write 2 Done Done Sent false
8403 3236 Acked Write 8 Done Done Sent false
8404 3237 Acked Write 2 Done Done Sent false
8405 3238 Acked Read 2 Done Done Sent false
8406 3239 Acked Write 5 Done Done Sent false
8407 3240 Acked Read 7 Done Done Sent false
8408 3241 Acked Write 4 Done Done Sent false
8409 3242 Acked Write 9 Done Done Sent false
8410 3243 Acked Write 5 Done Done Sent false
8411 3244 Acked Flush 0 Done Done Sent false
8412 3245 Acked Read 9 Done Done Sent false
8413 3246 Acked Read 1 Done Done Sent false
8414 3247 Acked Read 4 Done Done Sent false
8415 3248 Acked Read 5 Done Done Sent false
8416 3249 Acked Read 6 Done Done Sent false
8417 3250 Acked Write 5 Done Done Sent false
8418 3251 Acked Read 5 Done Done Sent false
8419 3252 Acked Read 7 Done Done Sent false
8420 3253 Acked Write 6 Done Done Sent false
8421 3254 Acked Write 10 Done Done Sent false
8422 3255 Acked Write 10 Done Done Sent false
8423 3256 Acked Write 9 Done Done Sent false
8424 3257 Acked Read 7 Done Done Sent false
8425 3258 Acked Read 4 Done Done Sent false
8426 3259 Acked Write 3 Done Done Sent false
8427 3260 Acked Write 6 Done Done Sent false
8428 3261 Acked Write 2 Done Done Sent false
8429 3262 Acked Write 7 Done Done Sent false
8430 3263 Acked Flush 0 Done Done Sent false
8431 3264 Acked Read 5 Done Done Sent false
8432 3265 Acked Read 1 Done Done Sent false
8433 3266 Acked Read 10 Done Done Sent false
8434 3267 Acked Read 2 Done Done Sent false
8435 3268 Acked Write 6 Done Done Sent false
8436 3269 Acked Read 4 Done Done Sent false
8437 3270 Acked Read 8 Done Done Sent false
8438 3271 Acked Flush 0 Done Done Sent false
8439 3272 Acked Flush 0 Done Done Sent false
8440 3273 Acked Write 4 Done Done Sent false
8441 3274 Acked Read 3 Done Done Sent false
8442 3275 Acked Read 3 Done Done Sent false
8443 3276 Acked Read 9 Done Done Sent false
8444 3277 Acked Flush 0 Done Done Sent false
8445 3278 Acked Read 5 Done Done Sent false
8446 3279 Acked Read 9 Done Done Sent false
8447 3280 Acked Write 1 Done Done Sent false
8448 3281 Acked Write 2 Done Done Sent false
8449 3282 Acked Read 5 Done Done Sent false
8450 3283 Acked Read 4 Done Done Sent false
8451 3284 Acked Write 9 Done Done Sent false
8452 3285 Acked Write 1 Done Done Sent false
8453 3286 Acked Write 3 Done Done Sent false
8454 3287 Acked Read 8 Done Done Sent false
8455 3288 Acked Write 1 Done Done Sent false
8456 3289 Acked Flush 0 Done Done Sent false
8457 3290 Acked Read 5 Done Done Sent false
8458 3291 Acked Read 7 Done Done Sent false
8459 3292 Acked Read 1 Done Done Sent false
8460 3293 Acked Write 10 Done Done Sent false
8461 3294 Acked Read 2 Done Done Sent false
8462 3295 Acked Write 10 Done Done Sent false
8463 3296 Acked Flush 0 Done Done Sent false
8464 3297 Acked Write 2 Done Done Sent false
8465 3298 Acked Read 2 Done Done Sent false
8466 3299 Acked Write 1 Done Done Sent false
8467 3300 Acked Read 5 Done Done Sent false
8468 3301 Acked Read 8 Done Done Sent false
8469 3302 Acked Write 4 Done Done Sent false
8470 3303 Acked Read 9 Done Done Sent false
8471 3304 Acked Read 7 Done Done Sent false
8472 3305 Acked Read 7 Done Done Sent false
8473 3306 Acked Read 7 Done Done Sent false
8474 3307 Acked Write 4 Done Done Sent false
8475 3308 Acked Write 2 Done Done Sent false
8476 3309 Acked Write 6 Done Done Sent false
8477 3310 Acked Read 7 Done Done Sent false
8478 3311 Acked Read 1 Done Done Sent false
8479 3312 Acked Write 1 Done Done Sent false
8480 3313 Acked Write 9 Done Done Sent false
8481 3314 Acked Write 7 Done Done Sent false
8482 3315 Acked Read 1 Done Done Sent false
8483 3316 Acked Read 2 Done Done Sent false
8484 3317 Acked Read 9 Done Done Sent false
8485 3318 Acked Write 10 Done Done Sent false
8486 3319 Acked Write 1 Done Done Sent false
8487 3320 Acked Read 2 Done Done Sent false
8488 3321 Acked Read 7 Done Done Sent false
8489 3322 Acked Write 9 Done Done Sent false
8490 3323 Acked Read 8 Done Done Sent false
8491 3324 Acked Read 2 Done Done Sent false
8492 3325 Acked Write 1 Done Done Sent false
8493 3326 Acked Write 2 Done Done Sent false
8494 3327 Acked Read 9 Done Done Sent false
8495 3328 Acked Flush 0 Done Done Sent false
8496 3329 Acked Write 10 Done Done Sent false
8497 3330 Acked Write 4 Done Done Sent false
8498 3331 Acked Read 2 Done Done Sent false
8499 3332 Acked Write 10 Done Done Sent false
8500 3333 Acked Read 9 Done Done Sent false
8501 3334 Acked Read 5 Done Done Sent false
8502 3335 Acked Read 3 Done Done Sent false
8503 3336 Acked Write 6 Done Done Sent false
8504 3337 Acked Read 2 Done Done Sent false
8505 3338 Acked Write 6 Done Done Sent false
8506 3339 Acked Read 7 Done Done Sent false
8507 3340 Acked Write 6 Done Done Sent false
8508 3341 Acked Write 9 Done Done Sent false
8509 3342 Acked Read 5 Done Done Sent false
8510 3343 Acked Flush 0 Done Done Sent false
8511 3344 Acked Flush 0 Done Done Sent false
8512 3345 Acked Read 4 Done Done Sent false
8513 3346 Acked Read 5 Done Done Sent false
8514 3347 Acked Write 1 Done Done Sent false
8515 3348 Acked Write 2 Done Done Sent false
8516 3349 Acked Write 6 Done Done Sent false
8517 3350 Acked Read 5 Done Done Sent false
8518 3351 Acked Write 5 Done Done Sent false
8519 3352 Acked Write 1 Done Done Sent false
8520 3353 Acked Read 8 Done Done Sent false
8521 3354 Acked Read 6 Done Done Sent false
8522 3355 Acked Write 10 Done Done Sent false
8523 3356 Acked Read 9 Done Done Sent false
8524 3357 Acked Read 1 Done Done Sent false
8525 3358 Acked Read 4 Done Done Sent false
8526 3359 Acked Read 9 Done Done Sent false
8527 3360 Acked Read 1 Done Done Sent false
8528 3361 Acked Write 6 Done Done Sent false
8529 3362 Acked Write 10 Done Done Sent false
8530 3363 Acked Write 8 Done Done Sent false
8531 3364 Acked Read 3 Done Done Sent false
8532 3365 Acked Flush 0 Done Done Sent false
8533 3366 Acked Write 6 Done Done Sent false
8534 3367 Acked Write 10 Done Done Sent false
8535 3368 Acked Write 3 Done Done Sent false
8536 3369 Acked Write 7 Done Done Sent false
8537 3370 Acked Flush 0 Done Done Sent false
8538 3371 Acked Write 3 Done Done Sent false
8539 3372 Acked Read 5 Done Done Sent false
8540 3373 Acked Read 8 Done Done Sent false
8541 3374 Acked Read 8 Done Done Sent false
8542 3375 Acked Read 4 Done Done Sent false
8543 3376 Acked Write 3 Done Done Sent false
8544 3377 Acked Flush 0 Done Done Sent false
8545 3378 Acked Write 5 Done Done Sent false
8546 3379 Acked Read 5 Done Done Sent false
8547 3380 Acked Flush 0 Done Done Sent false
8548 3381 Acked Read 1 Done Done Sent false
8549 3382 Acked Write 8 Done Done Sent false
8550 3383 Acked Write 4 Done Done Sent false
8551 3384 Acked Read 5 Done Done Sent false
8552 3385 Acked Write 2 Done Done Sent false
8553 3386 Acked Write 8 Done Done Sent false
8554 3387 Acked Read 5 Done Done Sent false
8555 3388 Acked Read 3 Done Done Sent false
8556 3389 Acked Flush 0 Done Done Sent false
8557 3390 Acked Read 6 Done Done Sent false
8558 3391 Acked Read 3 Done Done Sent false
8559 3392 Acked Read 10 Done Done Sent false
8560 3393 Acked Read 5 Done Done Sent false
8561 3394 Acked Read 2 Done Done Sent false
8562 3395 Acked Write 9 Done Done Sent false
8563 3396 Acked Write 3 Done Done Sent false
8564 3397 Acked Write 1 Done Done Sent false
8565 3398 Acked Read 5 Done Done Sent false
8566 3399 Acked Read 8 Done Done Sent false
8567 3400 Acked Read 3 Done Done Sent false
8568 3401 Acked Write 3 Done Done Sent false
8569 3402 Acked Write 2 Done Done Sent false
8570 3403 Acked Write 8 Done Done Sent false
8571 3404 Acked Write 5 Done Done Sent false
8572 3405 Acked Read 8 Done Done Sent false
8573 3406 Acked Write 2 Done Done Sent false
8574 3407 Acked Write 1 Done Done Sent false
8575 3408 Acked Read 8 Done Done Sent false
8576 3409 Acked Read 5 Done Done Sent false
8577 3410 Acked Write 9 Done Done Sent false
8578 3411 Acked Read 10 Done Done Sent false
8579 3412 Acked Read 9 Done Done Sent false
8580 3413 Acked Flush 0 Done Done Sent false
8581 STATES DS:0 DS:1 DS:2 TOTAL
8582 Sent 0 0 302 302
8583 Done 302 302 0 604
8584 Skipped 0 0 0 0
8585 Error 0 0 0 0
8586 Last Flush: 3413 3413 3111
8587 Downstairs last five completed: 3111 3110 3109 3108 3107
8588 Upstairs last five completed: 3413 3412 3411 3410 3409
8589 CLIENT: Up:0 ds:302 act:2
85902025-01-27T19:38:08.652ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
85912025-01-27T19:38:08.652ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42755) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
85922025-01-27T19:38:08.653ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
85932025-01-27T19:38:08.654ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
85942025-01-27T19:38:08.654ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
85952025-01-27T19:38:08.654ZINFOcrucible: send last flush ID to this DS: 3111 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
85962025-01-27T19:38:08.654ZINFOcrucible: Replied this last flush ID: 3111 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
85972025-01-27T19:38:08.654ZINFOcrucible: [2] client re-new 302 jobs since flush 3111 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
85982025-01-27T19:38:08.654ZINFOcrucible: [2] Replayed 302 jobs since flush: 3111 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
8599 ----------------------------------------------------------------
8600 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
8601 Downstairs last five completed: 3413 3412 3411 3410 3409
8602 Upstairs last five completed: 3413 3412 3411 3410 3409
8603 CLIENT: Up:0 ds:0 act:3
8604 Replay: All jobs finished
8605 Main task has work to do, go find it
8606 got dsc Stop(1)
8607 stop 1
8608 [8820] Got stop action so:false kr:false
8609 Jan 27 19:38:10.887 INFO request completed, latency_us: 372, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 156584cb-fbb9-44e3-aae0-b03361d9276c, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
86102025-01-27T19:38:10.887ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
86112025-01-27T19:38:10.887ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
86122025-01-27T19:38:10.887ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
8613 Jan 27 19:38:10.887 INFO request completed, latency_us: 155, response_code: 200, uri: /state/cid/1{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-27T19:38:10.887898722Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
8614 , method: GET, req_id: 2ff668b2-bf55-4138-9c13-47dba7ddf290, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
8615 [8820] Exited with: ExitStatus(unix_wait_status(9))
8616 [8820][1] reports Exit
8617 Jan 27 19:38:14.890 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/1, method: GET, req_id: 52509c1e-e90e-4c93-928d-4044d689f964, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
8618 001/300 Write block 4321 len 24576 data: 2 2 2 2 3 3
8619 002/300 Read block 442 len 20480
8620 003/300 Write block 4294 len 8192 data: 2 2
8621 004/300 Read block 3971 len 4096
8622 005/300 Write block 2972 len 12288 data: 3 3 3
8623 006/300 Read block 490 len 12288
8624 007/300 Read block 3679 len 12288
8625 008/300 Write block 1122 len 32768 data: 2 2 2 2 3 3 4 5
8626 009/300 Read block 542 len 4096
8627 010/300 Read block 2470 len 16384
8628 011/300 Read block 378 len 16384
8629 012/300 Write block 2156 len 40960 data: 4 4 4 4 4 5 5 5 5 5
8630 013/300 Read block 781 len 20480
8631 014/300 Read block 3459 len 12288
8632 015/300 Read block 1915 len 12288
8633 016/300 Write block 544 len 12288 data: 3 4 4
8634 017/300 Read block 2709 len 28672
8635 018/300 Write block 3215 len 40960 data: 3 3 3 4 4 4 4 4 4 4
8636 019/300 Write block 3685 len 36864 data: 2 2 3 4 4 4 4 4 4
8637 020/300 Flush
8638 021/300 Flush
8639 022/300 Write block 3765 len 28672 data: 4 4 4 4 3 3 3
8640 023/300 Flush
8641 024/300 Write block 3272 len 32768 data: 4 4 4 3 3 4 4 3
8642 025/300 Read block 1385 len 8192
8643 026/300 Write block 3539 len 16384 data: 2 2 2 2
8644 027/300 Write block 3301 len 12288 data: 2 2 2
8645 028/300 Write block 3732 len 20480 data: 2 2 2 4 4
8646 029/300 Read block 2199 len 12288
8647 030/300 Read block 4885 len 16384
8648 031/300 Read block 1165 len 4096
8649 032/300 Write block 4898 len 28672 data: 4 4 4 3 4 2 2
8650 033/300 Write block 2875 len 16384 data: 2 2 2 2
8651 034/300 Read block 2348 len 8192
8652 035/300 Write block 3190 len 28672 data: 3 3 3 3 3 3 3
8653 036/300 Read block 386 len 32768
8654 037/300 Read block 3301 len 32768
8655 038/300 Write block 689 len 24576 data: 3 2 2 2 2 3
8656 039/300 Read block 4671 len 4096
8657 040/300 Read block 4832 len 24576
8658 041/300 Write block 3917 len 8192 data: 2 3
8659 042/300 Read block 4128 len 8192
8660 043/300 Read block 2823 len 32768
8661 044/300 Read block 4104 len 20480
8662 045/300 Read block 1606 len 24576
8663 046/300 Write block 4236 len 28672 data: 3 3 4 3 3 3 4
8664 047/300 Write block 1753 len 24576 data: 2 4 3 3 3 3
8665 048/300 Write block 1958 len 8192 data: 4 4
8666 049/300 Read block 4615 len 28672
8667 050/300 Write block 409 len 36864 data: 3 3 2 2 2 2 2 2 2
8668 051/300 Read block 395 len 32768
8669 052/300 Write block 719 len 16384 data: 4 5 5 5
8670 053/300 Read block 2395 len 36864
8671 054/300 Read block 3873 len 28672
8672 055/300 Read block 374 len 28672
8673 056/300 Read block 4348 len 20480
8674 057/300 Write block 4968 len 28672 data: 4 4 5 5 3 2 2
8675 058/300 Read block 4368 len 40960
8676 059/300 Read block 4405 len 28672
8677 060/300 Write block 2182 len 24576 data: 4 3 4 3 4 4
8678 061/300 Read block 2249 len 28672
8679 062/300 Write block 2267 len 12288 data: 2 3 3
8680 063/300 Flush
8681 064/300 Write block 364 len 20480 data: 3 3 3 3 3
8682 065/300 Write block 1524 len 16384 data: 3 3 2 3
8683 066/300 Flush
8684 067/300 Read block 2175 len 40960
8685 068/300 Write block 2045 len 12288 data: 3 3 3
8686 069/300 Write block 1337 len 12288 data: 3 3 3
8687 070/300 Read block 4490 len 20480
8688 071/300 Flush
8689 072/300 Write block 3703 len 28672 data: 2 2 2 2 3 5 5
8690 073/300 Read block 4866 len 32768
8691 074/300 Write block 417 len 36864 data: 3 3 3 3 3 3 3 3 3
8692 075/300 Read block 3791 len 24576
8693 076/300 Read block 987 len 12288
8694 077/300 Write block 1256 len 12288 data: 2 2 2
8695 078/300 Read block 4367 len 32768
8696 079/300 Write block 2728 len 40960 data: 2 2 2 2 2 2 2 2 3 3
8697 080/300 Flush
8698 081/300 Read block 4945 len 36864
8699 082/300 Read block 533 len 36864
8700 083/300 Read block 1942 len 20480
8701 084/300 Read block 4063 len 12288
8702 085/300 Read block 3489 len 28672
8703 086/300 Read block 4382 len 8192
8704 087/300 Write block 598 len 12288 data: 3 3 3
8705 088/300 Read block 3629 len 40960
8706 089/300 Read block 4451 len 12288
8707 090/300 Flush
8708 091/300 Read block 372 len 20480
8709 092/300 Read block 1501 len 32768
8710 093/300 Write block 1448 len 28672 data: 2 2 2 2 2 2 2
8711 094/300 Write block 4746 len 20480 data: 2 2 2 2 2
8712 095/300 Flush
8713 096/300 Read block 2239 len 36864
8714 097/300 Read block 934 len 28672
8715 098/300 Write block 3963 len 12288 data: 3 2 3
8716 099/300 Read block 2224 len 4096
8717 100/300 Read block 2774 len 32768
8718 101/300 Write block 1859 len 16384 data: 5 4 4 4
8719 102/300 Write block 916 len 24576 data: 2 2 2 3 3 3
8720 103/300 Read block 1769 len 4096
8721 104/300 Write block 3267 len 28672 data: 2 2 2 3 4 5 5
8722 105/300 Write block 4769 len 40960 data: 2 2 3 2 2 2 2 2 2 2
8723 106/300 Write block 1771 len 12288 data: 2 3 3
8724 107/300 Read block 2822 len 4096
8725 108/300 Write block 4905 len 4096 data: 2
8726 109/300 Write block 4043 len 16384 data: 4 3 3 3
8727 110/300 Read block 2717 len 4096
8728 111/300 Write block 1373 len 28672 data: 4 4 5 6 6 5 5
8729 112/300 Read block 4914 len 12288
8730 113/300 Write block 1407 len 12288 data: 3 2 2
8731 114/300 Write block 1352 len 36864 data: 2 4 4 4 4 4 6 6 6
8732 115/300 Read block 2895 len 20480
8733 116/300 Write block 4302 len 12288 data: 2 2 3
8734 117/300 Write block 1619 len 40960 data: 3 2 2 2 3 3 3 3 5 4
8735 118/300 Read block 3961 len 40960
8736 119/300 Read block 4685 len 36864
8737 120/300 Write block 666 len 12288 data: 2 2 2
8738 121/300 Read block 3381 len 40960
8739 122/300 Write block 164 len 32768 data: 2 2 3 4 4 4 4 4
8740 123/300 Write block 3424 len 12288 data: 2 2 3
8741 124/300 Read block 934 len 36864
8742 125/300 Read block 4741 len 8192
8743 126/300 Read block 910 len 4096
8744 127/300 Write block 3142 len 8192 data: 6 6
8745 128/300 Write block 1294 len 24576 data: 3 3 3 4 4 4
8746 129/300 Write block 3072 len 8192 data: 2 2
8747 130/300 Read block 4094 len 12288
8748 131/300 Write block 2342 len 12288 data: 3 3 3
8749 132/300 Write block 4275 len 40960 data: 4 4 4 4 3 3 3 3 3 4
8750 133/300 Read block 1561 len 36864
8751 134/300 Write block 4505 len 40960 data: 3 3 3 2 3 3 3 4 4 5
8752 135/300 Read block 2797 len 36864
8753 136/300 Read block 1511 len 12288
8754 137/300 Read block 4977 len 36864
8755 138/300 Write block 4770 len 32768 data: 3 4 3 3 3 3 3 3
8756 139/300 Read block 1366 len 28672
8757 140/300 Write block 4050 len 4096 data: 2
8758 141/300 Read block 3997 len 4096
8759 142/300 Flush
8760 143/300 Write block 2959 len 20480 data: 2 2 2 3 3
8761 144/300 Read block 2844 len 8192
8762 145/300 Flush
8763 146/300 Read block 3728 len 40960
8764 147/300 Write block 794 len 36864 data: 4 4 4 4 3 2 3 2 3
8765 148/300 Read block 2074 len 20480
8766 149/300 Write block 114 len 24576 data: 2 2 2 2 2 2
8767 150/300 Read block 3217 len 16384
8768 151/300 Flush
8769 152/300 Read block 1701 len 40960
8770 153/300 Write block 552 len 12288 data: 3 3 3
8771 154/300 Read block 4541 len 24576
8772 155/300 Read block 4100 len 4096
8773 156/300 Read block 1970 len 24576
8774 157/300 Write block 4941 len 16384 data: 3 4 5 5
8775 158/300 Read block 840 len 16384
8776 159/300 Write block 1728 len 12288 data: 3 3 2
8777 160/300 Flush
8778 161/300 Read block 2331 len 36864
8779 162/300 Read block 1110 len 32768
8780 163/300 Read block 3876 len 36864
8781 164/300 Write block 1254 len 12288 data: 2 2 3
8782 165/300 Flush
8783 166/300 Flush
8784 167/300 Read block 1724 len 4096
8785 168/300 Read block 80 len 16384
8786 169/300 Write block 3382 len 16384 data: 2 2 2 2
8787 170/300 Read block 3102 len 24576
8788 171/300 Read block 3414 len 12288
8789 172/300 Flush
8790 173/300 Read block 2731 len 4096
8791 174/300 Write block 1530 len 12288 data: 2 3 3
8792 175/300 Flush
8793 176/300 Write block 1010 len 24576 data: 3 3 3 3 3 3
8794 177/300 Read block 442 len 36864
8795 178/300 Read block 1972 len 36864
8796 179/300 Read block 2863 len 24576
8797 180/300 Read block 1354 len 24576
8798 181/300 Read block 4097 len 36864
8799 182/300 Write block 1405 len 16384 data: 4 4 4 3
8800 183/300 Write block 2236 len 16384 data: 3 3 3 4
8801 184/300 Read block 2697 len 12288
8802 185/300 Write block 2935 len 8192 data: 3 4
8803 186/300 Flush
8804 187/300 Write block 3758 len 20480 data: 4 5 5 5 5
8805 188/300 Flush
8806 189/300 Read block 254 len 8192
8807 190/300 Read block 4536 len 40960
8808 191/300 Read block 1698 len 40960
8809 192/300 Write block 3050 len 8192 data: 4 4
8810 193/300 Read block 1718 len 24576
8811 194/300 Write block 2305 len 32768 data: 2 2 2 2 2 2 2 2
8812 195/300 Write block 1338 len 4096 data: 4
8813 196/300 Write block 2907 len 32768 data: 2 2 2 2 2 2 2 2
8814 197/300 Write block 83 len 32768 data: 3 2 2 3 3 4 3 3
8815 198/300 Read block 1527 len 28672
8816 199/300 Read block 3153 len 36864
8817 200/300 Read block 4319 len 32768
8818 201/300 Write block 2788 len 32768 data: 2 2 3 3 3 3 4 4
8819 202/300 Write block 60 len 36864 data: 3 2 3 3 4 4 4 3 4
8820 203/300 Write block 1479 len 4096 data: 4
8821 204/300 Flush
8822 205/300 Write block 4829 len 4096 data: 3
8823 206/300 Read block 228 len 40960
8824 207/300 Write block 2425 len 4096 data: 3
8825 208/300 Write block 1460 len 8192 data: 3 2
8826 209/300 Flush
8827 210/300 Read block 1412 len 32768
8828 211/300 Flush
8829 212/300 Read block 3148 len 36864
8830 213/300 Read block 650 len 20480
8831 214/300 Flush
8832 215/300 Read block 1153 len 36864
8833 216/300 Read block 2667 len 16384
8834 217/300 Read block 1624 len 32768
8835 218/300 Write block 3140 len 24576 data: 7 7 7 7 4 4
8836 219/300 Flush
8837 220/300 Write block 2719 len 8192 data: 5 4
8838 221/300 Write block 3020 len 40960 data: 3 3 3 4 4 4 3 3 3 3
8839 222/300 Write block 3893 len 24576 data: 3 3 3 4 4 4
8840 223/300 Read block 1930 len 28672
8841 224/300 Write block 4162 len 8192 data: 3 3
8842 225/300 Write block 1853 len 16384 data: 4 5 6 5
8843 226/300 Read block 2956 len 36864
8844 227/300 Write block 4789 len 28672 data: 4 6 6 5 5 5 4
8845 228/300 Write block 464 len 12288 data: 4 4 3
8846 229/300 Write block 246 len 4096 data: 2
8847 230/300 Write block 1734 len 12288 data: 2 2 2
8848 231/300 Write block 196 len 20480 data: 4 4 4 5 5
8849 232/300 Write block 658 len 32768 data: 3 3 2 2 2 2 2 2
8850 233/300 Write block 4297 len 32768 data: 2 2 2 2 2 3 3 4
8851 234/300 Flush
8852 235/300 Read block 2486 len 32768
8853 236/300 Read block 2896 len 20480
8854 237/300 Read block 4350 len 4096
8855 238/300 Read block 4141 len 20480
8856 239/300 Flush
8857 240/300 Write block 2934 len 12288 data: 3 4 5
8858 241/300 Write block 4240 len 12288 data: 4 4 5
8859 242/300 Flush
8860 243/300 Read block 2117 len 28672
8861 244/300 Flush
8862 245/300 Read block 685 len 20480
8863 246/300 Read block 3558 len 20480
8864 247/300 Write block 3809 len 24576 data: 2 2 2 2 2 2
8865 248/300 Flush
8866 249/300 Write block 3984 len 16384 data: 2 2 2 2
8867 250/300 Read block 1536 len 8192
8868 251/300 Write block 773 len 24576 data: 2 2 2 2 2 2
8869 252/300 Read block 269 len 12288
8870 253/300 Write block 4218 len 20480 data: 2 2 2 2 2
8871 254/300 Read block 370 len 28672
8872 255/300 Read block 38 len 12288
8873 256/300 Read block 3631 len 36864
8874 257/300 Read block 247 len 28672
8875 258/300 Read block 1195 len 40960
8876 259/300 Write block 2440 len 8192 data: 5 6
8877 260/300 Write block 1158 len 8192 data: 2 2
8878 261/300 Flush
8879 262/300 Flush
8880 263/300 Flush
8881 264/300 Read block 3489 len 40960
8882 265/300 Write block 4615 len 24576 data: 4 4 4 4 4 3
8883 266/300 Read block 3049 len 40960
8884 267/300 Read block 2467 len 16384
8885 268/300 Write block 1784 len 24576 data: 2 3 3 3 3 4
8886 269/300 Write block 809 len 16384 data: 3 3 3 3
8887 270/300 Read block 363 len 4096
8888 271/300 Write block 3383 len 28672 data: 3 3 3 2 3 4 4
8889 272/300 Write block 2299 len 28672 data: 2 2 2 3 3 3 3
8890 273/300 Read block 2469 len 4096
8891 274/300 Read block 4629 len 16384
8892 275/300 Read block 4776 len 4096
8893 276/300 Flush
8894 277/300 Read block 1410 len 32768
8895 278/300 Write block 2811 len 20480 data: 3 3 3 3 3
8896 279/300 Write block 3020 len 40960 data: 4 4 4 5 5 5 4 4 4 4
8897 280/300 Read block 2385 len 40960
8898 281/300 Read block 4246 len 24576
8899 282/300 Write block 3117 len 8192 data: 2 2
8900 283/300 Read block 109 len 24576
8901 284/300 Read block 4252 len 28672
8902 285/300 Write block 545 len 40960 data: 5 5 4 4 4 3 2 4 4 4
8903 286/300 Read block 1692 len 24576
8904 287/300 Flush
8905 288/300 Read block 1840 len 8192
8906 289/300 Write block 1606 len 24576 data: 3 2 2 2 2 2
8907 290/300 Write block 3169 len 36864 data: 2 2 2 3 3 2 2 4 4
8908 291/300 Write block 3880 len 4096 data: 4
8909 292/300 Read block 3589 len 28672
8910 293/300 Write block 3422 len 40960 data: 2 2 3 3 4 3 3 3 3 3
8911 294/300 Read block 1824 len 4096
8912 295/300 Read block 2164 len 40960
8913 296/300 Read block 979 len 12288
8914 297/300 Read block 565 len 32768
8915 298/300 Read block 3005 len 8192
8916 299/300 Read block 2105 len 20480
8917 300/300 Read block 1450 len 32768
8918 301/300 Flush
8919 Main task has work to do, go find it
8920 got dsc Start(1)
8921 start 1
8922 Jan 27 19:38:15.125 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/1, method: POST, req_id: f4c56b86-01ec-48a3-b83e-7ca4b7633510, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
8923 Replay: started 1, returned:Ok(())
8924 ----------------------------------------------------------------
8925 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
8926 States: Active Offline Active
8927 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
8928 3414 Acked Write 6 Done Sent Done false
8929 3415 Acked Read 5 Done Sent Done false
8930 3416 Acked Write 2 Done Sent Done false
8931 3417 Acked Read 1 Done Sent Done false
8932 3418 Acked Write 3 Done Sent Done false
8933 3419 Acked Read 3 Done Sent Done false
8934 3420 Acked Read 3 Done Sent Done false
8935 3421 Acked Write 8 Done Sent Done false
8936 3422 Acked Read 1 Done Sent Done false
8937 3423 Acked Read 4 Done Sent Done false
8938 3424 Acked Read 4 Done Sent Done false
8939 3425 Acked Write 10 Done Sent Done false
8940 3426 Acked Read 5 Done Sent Done false
8941 3427 Acked Read 3 Done Sent Done false
8942 3428 Acked Read 3 Done Sent Done false
8943 3429 Acked Write 3 Done Sent Done false
8944 3430 Acked Read 7 Done Sent Done false
8945 3431 Acked Write 10 Done Sent Done false
8946 3432 Acked Write 9 Done Sent Done false
8947 3433 Acked Flush 0 Done Sent Done false
8948 3434 Acked Flush 0 Done Sent Done false
8949 3435 Acked Write 7 Done Sent Done false
8950 3436 Acked Flush 0 Done Sent Done false
8951 3437 Acked Write 8 Done Sent Done false
8952 3438 Acked Read 2 Done Sent Done false
8953 3439 Acked Write 4 Done Sent Done false
8954 3440 Acked Write 3 Done Sent Done false
8955 3441 Acked Write 5 Done Sent Done false
8956 3442 Acked Read 3 Done Sent Done false
8957 3443 Acked Read 4 Done Sent Done false
8958 3444 Acked Read 1 Done Sent Done false
8959 3445 Acked Write 7 Done Sent Done false
8960 3446 Acked Write 4 Done Sent Done false
8961 3447 Acked Read 2 Done Sent Done false
8962 3448 Acked Write 7 Done Sent Done false
8963 3449 Acked Read 8 Done Sent Done false
8964 3450 Acked Read 8 Done Sent Done false
8965 3451 Acked Write 6 Done Sent Done false
8966 3452 Acked Read 1 Done Sent Done false
8967 3453 Acked Read 6 Done Sent Done false
8968 3454 Acked Write 2 Done Sent Done false
8969 3455 Acked Read 2 Done Sent Done false
8970 3456 Acked Read 8 Done Sent Done false
8971 3457 Acked Read 5 Done Sent Done false
8972 3458 Acked Read 6 Done Sent Done false
8973 3459 Acked Write 7 Done Sent Done false
8974 3460 Acked Write 6 Done Sent Done false
8975 3461 Acked Write 2 Done Sent Done false
8976 3462 Acked Read 7 Done Sent Done false
8977 3463 Acked Write 9 Done Sent Done false
8978 3464 Acked Read 8 Done Sent Done false
8979 3465 Acked Write 4 Done Sent Done false
8980 3466 Acked Read 9 Done Sent Done false
8981 3467 Acked Read 7 Done Sent Done false
8982 3468 Acked Read 7 Done Sent Done false
8983 3469 Acked Read 5 Done Sent Done false
8984 3470 Acked Write 7 Done Sent Done false
8985 3471 Acked Read 10 Done Sent Done false
8986 3472 Acked Read 7 Done Sent Done false
8987 3473 Acked Write 6 Done Sent Done false
8988 3474 Acked Read 7 Done Sent Done false
8989 3475 Acked Write 3 Done Sent Done false
8990 3476 Acked Flush 0 Done Sent Done false
8991 3477 Acked Write 5 Done Sent Done false
8992 3478 Acked Write 4 Done Sent Done false
8993 3479 Acked Flush 0 Done Sent Done false
8994 3480 Acked Read 10 Done Sent Done false
8995 3481 Acked Write 3 Done Sent Done false
8996 3482 Acked Write 3 Done Sent Done false
8997 3483 Acked Read 5 Done Sent Done false
8998 3484 Acked Flush 0 Done Sent Done false
8999 3485 Acked Write 7 Done Sent Done false
9000 3486 Acked Read 8 Done Sent Done false
9001 3487 Acked Write 9 Done Sent Done false
9002 3488 Acked Read 6 Done Sent Done false
9003 3489 Acked Read 3 Done Sent Done false
9004 3490 Acked Write 3 Done Sent Done false
9005 3491 Acked Read 8 Done Sent Done false
9006 3492 Acked Write 10 Done Sent Done false
9007 3493 Acked Flush 0 Done Sent Done false
9008 3494 Acked Read 9 Done Sent Done false
9009 3495 Acked Read 9 Done Sent Done false
9010 3496 Acked Read 5 Done Sent Done false
9011 3497 Acked Read 3 Done Sent Done false
9012 3498 Acked Read 7 Done Sent Done false
9013 3499 Acked Read 2 Done Sent Done false
9014 3500 Acked Write 3 Done Sent Done false
9015 3501 Acked Read 10 Done Sent Done false
9016 3502 Acked Read 3 Done Sent Done false
9017 3503 Acked Flush 0 Done Sent Done false
9018 3504 Acked Read 5 Done Sent Done false
9019 3505 Acked Read 8 Done Sent Done false
9020 3506 Acked Write 7 Done Sent Done false
9021 3507 Acked Write 5 Done Sent Done false
9022 3508 Acked Flush 0 Done Sent Done false
9023 3509 Acked Read 9 Done Sent Done false
9024 3510 Acked Read 7 Done Sent Done false
9025 3511 Acked Write 3 Done Sent Done false
9026 3512 Acked Read 1 Done Sent Done false
9027 3513 Acked Read 8 Done Sent Done false
9028 3514 Acked Write 4 Done Sent Done false
9029 3515 Acked Write 6 Done Sent Done false
9030 3516 Acked Read 1 Done Sent Done false
9031 3517 Acked Write 7 Done Sent Done false
9032 3518 Acked Write 10 Done Sent Done false
9033 3519 Acked Write 3 Done Sent Done false
9034 3520 Acked Read 1 Done Sent Done false
9035 3521 Acked Write 1 Done Sent Done false
9036 3522 Acked Write 4 Done Sent Done false
9037 3523 Acked Read 1 Done Sent Done false
9038 3524 Acked Write 7 Done Sent Done false
9039 3525 Acked Read 3 Done Sent Done false
9040 3526 Acked Write 3 Done Sent Done false
9041 3527 Acked Write 9 Done Sent Done false
9042 3528 Acked Read 5 Done Sent Done false
9043 3529 Acked Write 3 Done Sent Done false
9044 3530 Acked Write 10 Done Sent Done false
9045 3531 Acked Read 10 Done Sent Done false
9046 3532 Acked Read 9 Done Sent Done false
9047 3533 Acked Write 3 Done Sent Done false
9048 3534 Acked Read 10 Done Sent Done false
9049 3535 Acked Write 8 Done Sent Done false
9050 3536 Acked Write 3 Done Sent Done false
9051 3537 Acked Read 9 Done Sent Done false
9052 3538 Acked Read 2 Done Sent Done false
9053 3539 Acked Read 1 Done Sent Done false
9054 3540 Acked Write 2 Done Sent Done false
9055 3541 Acked Write 6 Done Sent Done false
9056 3542 Acked Write 2 Done Sent Done false
9057 3543 Acked Read 3 Done Sent Done false
9058 3544 Acked Write 3 Done Sent Done false
9059 3545 Acked Write 10 Done Sent Done false
9060 3546 Acked Read 9 Done Sent Done false
9061 3547 Acked Write 10 Done Sent Done false
9062 3548 Acked Read 9 Done Sent Done false
9063 3549 Acked Read 3 Done Sent Done false
9064 3550 Acked Read 9 Done Sent Done false
9065 3551 Acked Write 8 Done Sent Done false
9066 3552 Acked Read 7 Done Sent Done false
9067 3553 Acked Write 1 Done Sent Done false
9068 3554 Acked Read 1 Done Sent Done false
9069 3555 Acked Flush 0 Done Sent Done false
9070 3556 Acked Write 5 Done Sent Done false
9071 3557 Acked Read 2 Done Sent Done false
9072 3558 Acked Flush 0 Done Sent Done false
9073 3559 Acked Read 10 Done Sent Done false
9074 3560 Acked Write 9 Done Sent Done false
9075 3561 Acked Read 5 Done Sent Done false
9076 3562 Acked Write 6 Done Sent Done false
9077 3563 Acked Read 4 Done Sent Done false
9078 3564 Acked Flush 0 Done Sent Done false
9079 3565 Acked Read 10 Done Sent Done false
9080 3566 Acked Write 3 Done Sent Done false
9081 3567 Acked Read 6 Done Sent Done false
9082 3568 Acked Read 1 Done Sent Done false
9083 3569 Acked Read 6 Done Sent Done false
9084 3570 Acked Write 4 Done Sent Done false
9085 3571 Acked Read 4 Done Sent Done false
9086 3572 Acked Write 3 Done Sent Done false
9087 3573 Acked Flush 0 Done Sent Done false
9088 3574 Acked Read 9 Done Sent Done false
9089 3575 Acked Read 8 Done Sent Done false
9090 3576 Acked Read 9 Done Sent Done false
9091 3577 Acked Write 3 Done Sent Done false
9092 3578 Acked Flush 0 Done Sent Done false
9093 3579 Acked Flush 0 Done Sent Done false
9094 3580 Acked Read 1 Done Sent Done false
9095 3581 Acked Read 4 Done Sent Done false
9096 3582 Acked Write 4 Done Sent Done false
9097 3583 Acked Read 6 Done Sent Done false
9098 3584 Acked Read 3 Done Sent Done false
9099 3585 Acked Flush 0 Done Sent Done false
9100 3586 Acked Read 1 Done Sent Done false
9101 3587 Acked Write 3 Done Sent Done false
9102 3588 Acked Flush 0 Done Sent Done false
9103 3589 Acked Write 6 Done Sent Done false
9104 3590 Acked Read 9 Done Sent Done false
9105 3591 Acked Read 9 Done Sent Done false
9106 3592 Acked Read 6 Done Sent Done false
9107 3593 Acked Read 6 Done Sent Done false
9108 3594 Acked Read 9 Done Sent Done false
9109 3595 Acked Write 4 Done Sent Done false
9110 3596 Acked Write 4 Done Sent Done false
9111 3597 Acked Read 3 Done Sent Done false
9112 3598 Acked Write 2 Done Sent Done false
9113 3599 Acked Flush 0 Done Sent Done false
9114 3600 Acked Write 5 Done Sent Done false
9115 3601 Acked Flush 0 Done Sent Done false
9116 3602 Acked Read 2 Done Sent Done false
9117 3603 Acked Read 10 Done Sent Done false
9118 3604 Acked Read 10 Done Sent Done false
9119 3605 Acked Write 2 Done Sent Done false
9120 3606 Acked Read 6 Done Sent Done false
9121 3607 Acked Write 8 Done Sent Done false
9122 3608 Acked Write 1 Done Sent Done false
9123 3609 Acked Write 8 Done Sent Done false
9124 3610 Acked Write 8 Done Sent Done false
9125 3611 Acked Read 7 Done Sent Done false
9126 3612 Acked Read 9 Done Sent Done false
9127 3613 Acked Read 8 Done Sent Done false
9128 3614 Acked Write 8 Done Sent Done false
9129 3615 Acked Write 9 Done Sent Done false
9130 3616 Acked Write 1 Done Sent Done false
9131 3617 Acked Flush 0 Done Sent Done false
9132 3618 Acked Write 1 Done Sent Done false
9133 3619 Acked Read 10 Done Sent Done false
9134 3620 Acked Write 1 Done Sent Done false
9135 3621 Acked Write 2 Done Sent Done false
9136 3622 Acked Flush 0 Done Sent Done false
9137 3623 Acked Read 8 Done Sent Done false
9138 3624 Acked Flush 0 Done Sent Done false
9139 3625 Acked Read 9 Done Sent Done false
9140 3626 Acked Read 5 Done Sent Done false
9141 3627 Acked Flush 0 Done Sent Done false
9142 3628 Acked Read 9 Done Sent Done false
9143 3629 Acked Read 4 Done Sent Done false
9144 3630 Acked Read 8 Done Sent Done false
9145 3631 Acked Write 6 Done Sent Done false
9146 3632 Acked Flush 0 Done Sent Done false
9147 3633 Acked Write 2 Done Sent Done false
9148 3634 Acked Write 10 Done Sent Done false
9149 3635 Acked Write 6 Done Sent Done false
9150 3636 Acked Read 7 Done Sent Done false
9151 3637 Acked Write 2 Done Sent Done false
9152 3638 Acked Write 4 Done Sent Done false
9153 3639 Acked Read 9 Done Sent Done false
9154 3640 Acked Write 7 Done Sent Done false
9155 3641 Acked Write 3 Done Sent Done false
9156 3642 Acked Write 1 Done Sent Done false
9157 3643 Acked Write 3 Done Sent Done false
9158 3644 Acked Write 5 Done Sent Done false
9159 3645 Acked Write 8 Done Sent Done false
9160 3646 Acked Write 8 Done Sent Done false
9161 3647 Acked Flush 0 Done Sent Done false
9162 3648 Acked Read 8 Done Sent Done false
9163 3649 Acked Read 5 Done Sent Done false
9164 3650 Acked Read 1 Done Sent Done false
9165 3651 Acked Read 5 Done Sent Done false
9166 3652 Acked Flush 0 Done Sent Done false
9167 3653 Acked Write 3 Done Sent Done false
9168 3654 Acked Write 3 Done Sent Done false
9169 3655 Acked Flush 0 Done Sent Done false
9170 3656 Acked Read 7 Done Sent Done false
9171 3657 Acked Flush 0 Done Sent Done false
9172 3658 Acked Read 5 Done Sent Done false
9173 3659 Acked Read 5 Done Sent Done false
9174 3660 Acked Write 6 Done Sent Done false
9175 3661 Acked Flush 0 Done Sent Done false
9176 3662 Acked Write 4 Done Sent Done false
9177 3663 Acked Read 2 Done Sent Done false
9178 3664 Acked Write 6 Done Sent Done false
9179 3665 Acked Read 3 Done Sent Done false
9180 3666 Acked Write 5 Done Sent Done false
9181 3667 Acked Read 7 Done Sent Done false
9182 3668 Acked Read 3 Done Sent Done false
9183 3669 Acked Read 9 Done Sent Done false
9184 3670 Acked Read 7 Done Sent Done false
9185 3671 Acked Read 10 Done Sent Done false
9186 3672 Acked Write 2 Done Sent Done false
9187 3673 Acked Write 2 Done Sent Done false
9188 3674 Acked Flush 0 Done Sent Done false
9189 3675 Acked Flush 0 Done Sent Done false
9190 3676 Acked Flush 0 Done Sent Done false
9191 3677 Acked Read 10 Done Sent Done false
9192 3678 Acked Write 6 Done Sent Done false
9193 3679 Acked Read 10 Done Sent Done false
9194 3680 Acked Read 4 Done Sent Done false
9195 3681 Acked Write 6 Done Sent Done false
9196 3682 Acked Write 4 Done Sent Done false
9197 3683 Acked Read 1 Done Sent Done false
9198 3684 Acked Write 7 Done Sent Done false
9199 3685 Acked Write 7 Done Sent Done false
9200 3686 Acked Read 1 Done Sent Done false
9201 3687 Acked Read 4 Done Sent Done false
9202 3688 Acked Read 1 Done Sent Done false
9203 3689 Acked Flush 0 Done Sent Done false
9204 3690 Acked Read 8 Done Sent Done false
9205 3691 Acked Write 5 Done Sent Done false
9206 3692 Acked Write 10 Done Sent Done false
9207 3693 Acked Read 10 Done Sent Done false
9208 3694 Acked Read 6 Done Sent Done false
9209 3695 Acked Write 2 Done Sent Done false
9210 3696 Acked Read 6 Done Sent Done false
9211 3697 Acked Read 7 Done Sent Done false
9212 3698 Acked Write 10 Done Sent Done false
9213 3699 Acked Read 6 Done Sent Done false
9214 3700 Acked Flush 0 Done Sent Done false
9215 3701 Acked Read 2 Done Sent Done false
9216 3702 Acked Write 6 Done Sent Done false
9217 3703 Acked Write 9 Done Sent Done false
9218 3704 Acked Write 1 Done Sent Done false
9219 3705 Acked Read 7 Done Sent Done false
9220 3706 Acked Write 10 Done Sent Done false
9221 3707 Acked Read 1 Done Sent Done false
9222 3708 Acked Read 10 Done Sent Done false
9223 3709 Acked Read 3 Done Sent Done false
9224 3710 Acked Read 8 Done Sent Done false
9225 3711 Acked Read 2 Done Sent Done false
9226 3712 Acked Read 5 Done Sent Done false
9227 3713 Acked Read 8 Done Sent Done false
9228 3714 Acked Flush 0 Done Sent Done false
9229 STATES DS:0 DS:1 DS:2 TOTAL
9230 Sent 0 301 0 301
9231 Done 301 0 301 602
9232 Skipped 0 0 0 0
9233 Error 0 0 0 0
9234 Last Flush: 3714 3413 3714
9235 Downstairs last five completed: 3413 3412 3411 3410 3409
9236 Upstairs last five completed: 3714 3713 3712 3711 3710
9237 CLIENT: Up:0 ds:301 act:2
9238 [8820] got start action
9239 [8820] I am going to restart
9240 Starting downstairs at port 8820
9241 Make output file at "/tmp/dsc/downstairs-8820.txt"
9242 [8820][1] reports Starting
9243 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1209)
9244 [8820][1] reports Running
9245 ----------------------------------------------------------------
9246 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9247 States: Active Offline Active
9248 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9249 3414 Acked Write 6 Done Sent Done false
9250 3415 Acked Read 5 Done Sent Done false
9251 3416 Acked Write 2 Done Sent Done false
9252 3417 Acked Read 1 Done Sent Done false
9253 3418 Acked Write 3 Done Sent Done false
9254 3419 Acked Read 3 Done Sent Done false
9255 3420 Acked Read 3 Done Sent Done false
9256 3421 Acked Write 8 Done Sent Done false
9257 3422 Acked Read 1 Done Sent Done false
9258 3423 Acked Read 4 Done Sent Done false
9259 3424 Acked Read 4 Done Sent Done false
9260 3425 Acked Write 10 Done Sent Done false
9261 3426 Acked Read 5 Done Sent Done false
9262 3427 Acked Read 3 Done Sent Done false
9263 3428 Acked Read 3 Done Sent Done false
9264 3429 Acked Write 3 Done Sent Done false
9265 3430 Acked Read 7 Done Sent Done false
9266 3431 Acked Write 10 Done Sent Done false
9267 3432 Acked Write 9 Done Sent Done false
9268 3433 Acked Flush 0 Done Sent Done false
9269 3434 Acked Flush 0 Done Sent Done false
9270 3435 Acked Write 7 Done Sent Done false
9271 3436 Acked Flush 0 Done Sent Done false
9272 3437 Acked Write 8 Done Sent Done false
9273 3438 Acked Read 2 Done Sent Done false
9274 3439 Acked Write 4 Done Sent Done false
9275 3440 Acked Write 3 Done Sent Done false
9276 3441 Acked Write 5 Done Sent Done false
9277 3442 Acked Read 3 Done Sent Done false
9278 3443 Acked Read 4 Done Sent Done false
9279 3444 Acked Read 1 Done Sent Done false
9280 3445 Acked Write 7 Done Sent Done false
9281 3446 Acked Write 4 Done Sent Done false
9282 3447 Acked Read 2 Done Sent Done false
9283 3448 Acked Write 7 Done Sent Done false
9284 3449 Acked Read 8 Done Sent Done false
9285 3450 Acked Read 8 Done Sent Done false
9286 3451 Acked Write 6 Done Sent Done false
9287 3452 Acked Read 1 Done Sent Done false
9288 3453 Acked Read 6 Done Sent Done false
9289 3454 Acked Write 2 Done Sent Done false
9290 3455 Acked Read 2 Done Sent Done false
9291 3456 Acked Read 8 Done Sent Done false
9292 3457 Acked Read 5 Done Sent Done false
9293 3458 Acked Read 6 Done Sent Done false
9294 3459 Acked Write 7 Done Sent Done false
9295 3460 Acked Write 6 Done Sent Done false
9296 3461 Acked Write 2 Done Sent Done false
9297 3462 Acked Read 7 Done Sent Done false
9298 3463 Acked Write 9 Done Sent Done false
9299 3464 Acked Read 8 Done Sent Done false
9300 3465 Acked Write 4 Done Sent Done false
9301 3466 Acked Read 9 Done Sent Done false
9302 3467 Acked Read 7 Done Sent Done false
9303 3468 Acked Read 7 Done Sent Done false
9304 3469 Acked Read 5 Done Sent Done false
9305 3470 Acked Write 7 Done Sent Done false
9306 3471 Acked Read 10 Done Sent Done false
9307 3472 Acked Read 7 Done Sent Done false
9308 3473 Acked Write 6 Done Sent Done false
9309 3474 Acked Read 7 Done Sent Done false
9310 3475 Acked Write 3 Done Sent Done false
9311 3476 Acked Flush 0 Done Sent Done false
9312 3477 Acked Write 5 Done Sent Done false
9313 3478 Acked Write 4 Done Sent Done false
9314 3479 Acked Flush 0 Done Sent Done false
9315 3480 Acked Read 10 Done Sent Done false
9316 3481 Acked Write 3 Done Sent Done false
9317 3482 Acked Write 3 Done Sent Done false
9318 3483 Acked Read 5 Done Sent Done false
9319 3484 Acked Flush 0 Done Sent Done false
9320 3485 Acked Write 7 Done Sent Done false
9321 3486 Acked Read 8 Done Sent Done false
9322 3487 Acked Write 9 Done Sent Done false
9323 3488 Acked Read 6 Done Sent Done false
9324 3489 Acked Read 3 Done Sent Done false
9325 3490 Acked Write 3 Done Sent Done false
9326 3491 Acked Read 8 Done Sent Done false
9327 3492 Acked Write 10 Done Sent Done false
9328 3493 Acked Flush 0 Done Sent Done false
9329 3494 Acked Read 9 Done Sent Done false
9330 3495 Acked Read 9 Done Sent Done false
9331 3496 Acked Read 5 Done Sent Done false
9332 3497 Acked Read 3 Done Sent Done false
9333 3498 Acked Read 7 Done Sent Done false
9334 3499 Acked Read 2 Done Sent Done false
9335 3500 Acked Write 3 Done Sent Done false
9336 3501 Acked Read 10 Done Sent Done false
9337 3502 Acked Read 3 Done Sent Done false
9338 3503 Acked Flush 0 Done Sent Done false
9339 3504 Acked Read 5 Done Sent Done false
9340 3505 Acked Read 8 Done Sent Done false
9341 3506 Acked Write 7 Done Sent Done false
9342 3507 Acked Write 5 Done Sent Done false
9343 3508 Acked Flush 0 Done Sent Done false
9344 3509 Acked Read 9 Done Sent Done false
9345 3510 Acked Read 7 Done Sent Done false
9346 3511 Acked Write 3 Done Sent Done false
9347 3512 Acked Read 1 Done Sent Done false
9348 3513 Acked Read 8 Done Sent Done false
9349 3514 Acked Write 4 Done Sent Done false
9350 3515 Acked Write 6 Done Sent Done false
9351 3516 Acked Read 1 Done Sent Done false
9352 3517 Acked Write 7 Done Sent Done false
9353 3518 Acked Write 10 Done Sent Done false
9354 3519 Acked Write 3 Done Sent Done false
9355 3520 Acked Read 1 Done Sent Done false
9356 3521 Acked Write 1 Done Sent Done false
9357 3522 Acked Write 4 Done Sent Done false
9358 3523 Acked Read 1 Done Sent Done false
9359 3524 Acked Write 7 Done Sent Done false
9360 3525 Acked Read 3 Done Sent Done false
9361 3526 Acked Write 3 Done Sent Done false
9362 3527 Acked Write 9 Done Sent Done false
9363 3528 Acked Read 5 Done Sent Done false
9364 3529 Acked Write 3 Done Sent Done false
9365 3530 Acked Write 10 Done Sent Done false
9366 3531 Acked Read 10 Done Sent Done false
9367 3532 Acked Read 9 Done Sent Done false
9368 3533 Acked Write 3 Done Sent Done false
9369 3534 Acked Read 10 Done Sent Done false
9370 3535 Acked Write 8 Done Sent Done false
9371 3536 Acked Write 3 Done Sent Done false
9372 3537 Acked Read 9 Done Sent Done false
9373 3538 Acked Read 2 Done Sent Done false
9374 3539 Acked Read 1 Done Sent Done false
9375 3540 Acked Write 2 Done Sent Done false
9376 3541 Acked Write 6 Done Sent Done false
9377 3542 Acked Write 2 Done Sent Done false
9378 3543 Acked Read 3 Done Sent Done false
9379 3544 Acked Write 3 Done Sent Done false
9380 3545 Acked Write 10 Done Sent Done false
9381 3546 Acked Read 9 Done Sent Done false
9382 3547 Acked Write 10 Done Sent Done false
9383 3548 Acked Read 9 Done Sent Done false
9384 3549 Acked Read 3 Done Sent Done false
9385 3550 Acked Read 9 Done Sent Done false
9386 3551 Acked Write 8 Done Sent Done false
9387 3552 Acked Read 7 Done Sent Done false
9388 3553 Acked Write 1 Done Sent Done false
9389 3554 Acked Read 1 Done Sent Done false
9390 3555 Acked Flush 0 Done Sent Done false
9391 3556 Acked Write 5 Done Sent Done false
9392 3557 Acked Read 2 Done Sent Done false
9393 3558 Acked Flush 0 Done Sent Done false
9394 3559 Acked Read 10 Done Sent Done false
9395 3560 Acked Write 9 Done Sent Done false
9396 3561 Acked Read 5 Done Sent Done false
9397 3562 Acked Write 6 Done Sent Done false
9398 3563 Acked Read 4 Done Sent Done false
9399 3564 Acked Flush 0 Done Sent Done false
9400 3565 Acked Read 10 Done Sent Done false
9401 3566 Acked Write 3 Done Sent Done false
9402 3567 Acked Read 6 Done Sent Done false
9403 3568 Acked Read 1 Done Sent Done false
9404 3569 Acked Read 6 Done Sent Done false
9405 3570 Acked Write 4 Done Sent Done false
9406 3571 Acked Read 4 Done Sent Done false
9407 3572 Acked Write 3 Done Sent Done false
9408 3573 Acked Flush 0 Done Sent Done false
9409 3574 Acked Read 9 Done Sent Done false
9410 3575 Acked Read 8 Done Sent Done false
9411 3576 Acked Read 9 Done Sent Done false
9412 3577 Acked Write 3 Done Sent Done false
9413 3578 Acked Flush 0 Done Sent Done false
9414 3579 Acked Flush 0 Done Sent Done false
9415 3580 Acked Read 1 Done Sent Done false
9416 3581 Acked Read 4 Done Sent Done false
9417 3582 Acked Write 4 Done Sent Done false
9418 3583 Acked Read 6 Done Sent Done false
9419 3584 Acked Read 3 Done Sent Done false
9420 3585 Acked Flush 0 Done Sent Done false
9421 3586 Acked Read 1 Done Sent Done false
9422 3587 Acked Write 3 Done Sent Done false
9423 3588 Acked Flush 0 Done Sent Done false
9424 3589 Acked Write 6 Done Sent Done false
9425 3590 Acked Read 9 Done Sent Done false
9426 3591 Acked Read 9 Done Sent Done false
9427 3592 Acked Read 6 Done Sent Done false
9428 3593 Acked Read 6 Done Sent Done false
9429 3594 Acked Read 9 Done Sent Done false
9430 3595 Acked Write 4 Done Sent Done false
9431 3596 Acked Write 4 Done Sent Done false
9432 3597 Acked Read 3 Done Sent Done false
9433 3598 Acked Write 2 Done Sent Done false
9434 3599 Acked Flush 0 Done Sent Done false
9435 3600 Acked Write 5 Done Sent Done false
9436 3601 Acked Flush 0 Done Sent Done false
9437 3602 Acked Read 2 Done Sent Done false
9438 3603 Acked Read 10 Done Sent Done false
9439 3604 Acked Read 10 Done Sent Done false
9440 3605 Acked Write 2 Done Sent Done false
9441 3606 Acked Read 6 Done Sent Done false
9442 3607 Acked Write 8 Done Sent Done false
9443 3608 Acked Write 1 Done Sent Done false
9444 3609 Acked Write 8 Done Sent Done false
9445 3610 Acked Write 8 Done Sent Done false
9446 3611 Acked Read 7 Done Sent Done false
9447 3612 Acked Read 9 Done Sent Done false
9448 3613 Acked Read 8 Done Sent Done false
9449 3614 Acked Write 8 Done Sent Done false
9450 3615 Acked Write 9 Done Sent Done false
9451 3616 Acked Write 1 Done Sent Done false
9452 3617 Acked Flush 0 Done Sent Done false
9453 3618 Acked Write 1 Done Sent Done false
9454 3619 Acked Read 10 Done Sent Done false
9455 3620 Acked Write 1 Done Sent Done false
9456 3621 Acked Write 2 Done Sent Done false
9457 3622 Acked Flush 0 Done Sent Done false
9458 3623 Acked Read 8 Done Sent Done false
9459 3624 Acked Flush 0 Done Sent Done false
9460 3625 Acked Read 9 Done Sent Done false
9461 3626 Acked Read 5 Done Sent Done false
9462 3627 Acked Flush 0 Done Sent Done false
9463 3628 Acked Read 9 Done Sent Done false
9464 3629 Acked Read 4 Done Sent Done false
9465 3630 Acked Read 8 Done Sent Done false
9466 3631 Acked Write 6 Done Sent Done false
9467 3632 Acked Flush 0 Done Sent Done false
9468 3633 Acked Write 2 Done Sent Done false
9469 3634 Acked Write 10 Done Sent Done false
9470 3635 Acked Write 6 Done Sent Done false
9471 3636 Acked Read 7 Done Sent Done false
9472 3637 Acked Write 2 Done Sent Done false
9473 3638 Acked Write 4 Done Sent Done false
9474 3639 Acked Read 9 Done Sent Done false
9475 3640 Acked Write 7 Done Sent Done false
9476 3641 Acked Write 3 Done Sent Done false
9477 3642 Acked Write 1 Done Sent Done false
9478 3643 Acked Write 3 Done Sent Done false
9479 3644 Acked Write 5 Done Sent Done false
9480 3645 Acked Write 8 Done Sent Done false
9481 3646 Acked Write 8 Done Sent Done false
9482 3647 Acked Flush 0 Done Sent Done false
9483 3648 Acked Read 8 Done Sent Done false
9484 3649 Acked Read 5 Done Sent Done false
9485 3650 Acked Read 1 Done Sent Done false
9486 3651 Acked Read 5 Done Sent Done false
9487 3652 Acked Flush 0 Done Sent Done false
9488 3653 Acked Write 3 Done Sent Done false
9489 3654 Acked Write 3 Done Sent Done false
9490 3655 Acked Flush 0 Done Sent Done false
9491 3656 Acked Read 7 Done Sent Done false
9492 3657 Acked Flush 0 Done Sent Done false
9493 3658 Acked Read 5 Done Sent Done false
9494 3659 Acked Read 5 Done Sent Done false
9495 3660 Acked Write 6 Done Sent Done false
9496 3661 Acked Flush 0 Done Sent Done false
9497 3662 Acked Write 4 Done Sent Done false
9498 3663 Acked Read 2 Done Sent Done false
9499 3664 Acked Write 6 Done Sent Done false
9500 3665 Acked Read 3 Done Sent Done false
9501 3666 Acked Write 5 Done Sent Done false
9502 3667 Acked Read 7 Done Sent Done false
9503 3668 Acked Read 3 Done Sent Done false
9504 3669 Acked Read 9 Done Sent Done false
9505 3670 Acked Read 7 Done Sent Done false
9506 3671 Acked Read 10 Done Sent Done false
9507 3672 Acked Write 2 Done Sent Done false
9508 3673 Acked Write 2 Done Sent Done false
9509 3674 Acked Flush 0 Done Sent Done false
9510 3675 Acked Flush 0 Done Sent Done false
9511 3676 Acked Flush 0 Done Sent Done false
9512 3677 Acked Read 10 Done Sent Done false
9513 3678 Acked Write 6 Done Sent Done false
9514 3679 Acked Read 10 Done Sent Done false
9515 3680 Acked Read 4 Done Sent Done false
9516 3681 Acked Write 6 Done Sent Done false
9517 3682 Acked Write 4 Done Sent Done false
9518 3683 Acked Read 1 Done Sent Done false
9519 3684 Acked Write 7 Done Sent Done false
9520 3685 Acked Write 7 Done Sent Done false
9521 3686 Acked Read 1 Done Sent Done false
9522 3687 Acked Read 4 Done Sent Done false
9523 3688 Acked Read 1 Done Sent Done false
9524 3689 Acked Flush 0 Done Sent Done false
9525 3690 Acked Read 8 Done Sent Done false
9526 3691 Acked Write 5 Done Sent Done false
9527 3692 Acked Write 10 Done Sent Done false
9528 3693 Acked Read 10 Done Sent Done false
9529 3694 Acked Read 6 Done Sent Done false
9530 3695 Acked Write 2 Done Sent Done false
9531 3696 Acked Read 6 Done Sent Done false
9532 3697 Acked Read 7 Done Sent Done false
9533 3698 Acked Write 10 Done Sent Done false
9534 3699 Acked Read 6 Done Sent Done false
9535 3700 Acked Flush 0 Done Sent Done false
9536 3701 Acked Read 2 Done Sent Done false
9537 3702 Acked Write 6 Done Sent Done false
9538 3703 Acked Write 9 Done Sent Done false
9539 3704 Acked Write 1 Done Sent Done false
9540 3705 Acked Read 7 Done Sent Done false
9541 3706 Acked Write 10 Done Sent Done false
9542 3707 Acked Read 1 Done Sent Done false
9543 3708 Acked Read 10 Done Sent Done false
9544 3709 Acked Read 3 Done Sent Done false
9545 3710 Acked Read 8 Done Sent Done false
9546 3711 Acked Read 2 Done Sent Done false
9547 3712 Acked Read 5 Done Sent Done false
9548 3713 Acked Read 8 Done Sent Done false
9549 3714 Acked Flush 0 Done Sent Done false
9550 STATES DS:0 DS:1 DS:2 TOTAL
9551 Sent 0 301 0 301
9552 Done 301 0 301 602
9553 Skipped 0 0 0 0
9554 Error 0 0 0 0
9555 Last Flush: 3714 3413 3714
9556 Downstairs last five completed: 3413 3412 3411 3410 3409
9557 Upstairs last five completed: 3714 3713 3712 3711 3710
9558 CLIENT: Up:0 ds:301 act:2
95592025-01-27T19:38:20.888ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95602025-01-27T19:38:20.888ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42395) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95612025-01-27T19:38:20.889ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95622025-01-27T19:38:20.890ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95632025-01-27T19:38:20.890ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95642025-01-27T19:38:20.890ZINFOcrucible: send last flush ID to this DS: 3413 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95652025-01-27T19:38:20.890ZINFOcrucible: Replied this last flush ID: 3413 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95662025-01-27T19:38:20.890ZINFOcrucible: [1] client re-new 301 jobs since flush 3413 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95672025-01-27T19:38:20.891ZINFOcrucible: [1] Replayed 301 jobs since flush: 3413 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
9568 ----------------------------------------------------------------
9569 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
9570 Downstairs last five completed: 3714 3713 3712 3711 3710
9571 Upstairs last five completed: 3714 3713 3712 3711 3710
9572 CLIENT: Up:0 ds:0 act:3
9573 Replay: All jobs finished
9574 Main task has work to do, go find it
9575 got dsc Stop(1)
9576 stop 1
9577 Jan 27 19:38:23.130 INFO request completed, latency_us[8820] Got stop action so:false kr:false
9578 : 366, response_code: 204, uri: /stop/cid/1, method: POST, req_id: b5291f64-d6a5-4bfd-b344-4af562e6ab2c, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
9579 Jan 27 19:38:23.130 INFO request completed, latency_us: 113, response_code: 200, uri: /state/cid/1, method: GET, req_id: f1526735-a161-48ea-9133-fd70e5f89fad, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
95802025-01-27T19:38:23.130ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95812025-01-27T19:38:23.131ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95822025-01-27T19:38:23.131ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
95832025-01-27T19:38:23.131ZINFOcrucible: client task is exiting = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
9584 [8820] Exited with: ExitStatus(unix_wait_status(9))
9585 [8820][1] reports Exit
9586 Jan 27 19:38:27.132 INFO request completed, latency_us: 189, response_code: 200, uri: /state/cid/1, method: GET, req_id: fbf6302c-421d-4026-a24f-bf8ea784a049, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
9587 001/300 Write block 2651 len 24576 data: 4 4 4 4 4 4
9588 002/300 Write block 42 len 16384 data: 3 4 4 3
9589 003/300 Flush
9590 004/300 Read block 104 len 4096
9591 005/300 Read block 1019 len 16384
9592 006/300 Flush
9593 007/300 Read block 85 len 36864
9594 008/300 Read block 3102 len 28672
9595 009/300 Write block 4863 len 40960 data: 3 3 2 2 3 3 4 3 3 3
9596 010/300 Flush
9597 011/300 Read block 913 len 4096
9598 012/300 Flush
9599 013/300 Write block 2847 len 40960 data: 3 4 4 4 5 5 5 4 4 4
9600 014/300 Read block 3879 len 40960
9601 015/300 Write block 2870 len 32768 data: 2 2 2 2 2 3 3 3
9602 016/300 Write block 658 len 16384 data: 4 4 3 3
9603 017/300 Write block 3261 len 20480 data: 2 2 2 2 2
9604 018/300 Read block 3353 len 20480
9605 019/300 Read block 4167 len 20480
9606 020/300 Write block 3748 len 8192 data: 4 4
9607 021/300 Flush
9608 022/300 Write block 1298 len 20480 data: 5 5 4 5 4
9609 023/300 Read block 1958 len 28672
9610 024/300 Read block 763 len 12288
9611 025/300 Read block 4867 len 4096
9612 026/300 Read block 2795 len 8192
9613 027/300 Write block 2034 len 20480 data: 3 3 2 2 2
9614 028/300 Write block 1729 len 12288 data: 4 3 2
9615 029/300 Flush
9616 030/300 Read block 2343 len 36864
9617 031/300 Flush
9618 032/300 Flush
9619 033/300 Write block 2724 len 36864 data: 3 3 3 2 3 3 3 3 3
9620 034/300 Read block 4494 len 8192
9621 035/300 Write block 1674 len 12288 data: 5 4 4
9622 036/300 Write block 4556 len 4096 data: 2
9623 037/300 Write block 1694 len 24576 data: 2 2 2 2 2 2
9624 038/300 Read block 1405 len 20480
9625 039/300 Read block 397 len 28672
9626 040/300 Read block 4817 len 40960
9627 041/300 Write block 3648 len 8192 data: 2 2
9628 042/300 Write block 919 len 16384 data: 4 4 4 3
9629 043/300 Read block 2762 len 28672
9630 044/300 Read block 4465 len 12288
9631 045/300 Read block 1102 len 40960
9632 046/300 Flush
9633 047/300 Write block 4774 len 12288 data: 4 4 4
9634 048/300 Read block 2472 len 24576
9635 049/300 Flush
9636 050/300 Read block 2293 len 12288
9637 051/300 Read block 3060 len 12288
9638 052/300 Write block 1888 len 24576 data: 3 3 3 3 3 3
9639 053/300 Flush
9640 054/300 Read block 1044 len 12288
9641 055/300 Flush
9642 056/300 Read block 1898 len 4096
9643 057/300 Write block 2798 len 36864 data: 3 3 3 4 4 2 2 2 2
9644 058/300 Write block 2395 len 12288 data: 2 2 2
9645 059/300 Write block 1751 len 28672 data: 2 2 3 5 4 4 4
9646 060/300 Read block 4329 len 36864
9647 061/300 Read block 925 len 16384
9648 062/300 Write block 4475 len 32768 data: 3 3 3 3 2 3 3 3
9649 063/300 Write block 1991 len 36864 data: 2 2 2 2 3 3 3 3 3
9650 064/300 Read block 3971 len 12288
9651 065/300 Flush
9652 066/300 Read block 991 len 28672
9653 067/300 Write block 1017 len 8192 data: 3 4
9654 068/300 Read block 3690 len 40960
9655 069/300 Write block 3452 len 28672 data: 3 4 4 5 5 5 3
9656 070/300 Read block 4791 len 8192
9657 071/300 Write block 717 len 12288 data: 3 3 5
9658 072/300 Read block 2458 len 4096
9659 073/300 Read block 3171 len 8192
9660 074/300 Read block 4564 len 36864
9661 075/300 Write block 3853 len 12288 data: 2 2 2
9662 076/300 Write block 2672 len 40960 data: 2 3 3 3 3 4 4 5 5 5
9663 077/300 Read block 4043 len 12288
9664 078/300 Write block 4393 len 8192 data: 3 3
9665 079/300 Read block 906 len 12288
9666 080/300 Read block 1567 len 40960
9667 081/300 Write block 249 len 24576 data: 2 2 2 2 2 2
9668 082/300 Read block 265 len 20480
9669 083/300 Read block 2347 len 24576
9670 084/300 Read block 2238 len 40960
9671 085/300 Read block 3568 len 16384
9672 086/300 Read block 2768 len 32768
9673 087/300 Read block 3105 len 40960
9674 088/300 Read block 2357 len 4096
9675 089/300 Write block 1215 len 20480 data: 3 2 2 2 3
9676 090/300 Read block 3819 len 16384
9677 091/300 Write block 2680 len 28672 data: 6 6 5 5 5 3 3
9678 092/300 Write block 2154 len 24576 data: 4 4 5 5 5 5
9679 093/300 Read block 1934 len 24576
9680 094/300 Read block 3521 len 40960
9681 095/300 Write block 2658 len 8192 data: 5 4
9682 096/300 Read block 2585 len 8192
9683 097/300 Flush
9684 098/300 Read block 1567 len 16384
9685 099/300 Read block 4093 len 32768
9686 100/300 Write block 4989 len 16384 data: 4 3 2 2
9687 101/300 Write block 4231 len 8192 data: 3 3
9688 102/300 Write block 3649 len 16384 data: 3 2 2 2
9689 103/300 Flush
9690 104/300 Read block 903 len 32768
9691 105/300 Read block 1448 len 36864
9692 106/300 Write block 2137 len 40960 data: 2 2 2 2 3 3 3 4 3 3
9693 107/300 Write block 2611 len 20480 data: 2 2 2 2 2
9694 108/300 Write block 3054 len 12288 data: 3 2 3
9695 109/300 Read block 2301 len 32768
9696 110/300 Read block 731 len 24576
9697 111/300 Read block 124 len 40960
9698 112/300 Read block 3999 len 16384
9699 113/300 Read block 1920 len 8192
9700 114/300 Read block 3926 len 40960
9701 115/300 Write block 4250 len 12288 data: 7 5 4
9702 116/300 Flush
9703 117/300 Write block 1199 len 4096 data: 2
9704 118/300 Read block 2787 len 28672
9705 119/300 Read block 3333 len 32768
9706 120/300 Write block 2744 len 40960 data: 2 2 2 2 2 2 2 4 3 3
9707 121/300 Read block 2595 len 24576
9708 122/300 Write block 1068 len 20480 data: 2 2 3 3 3
9709 123/300 Read block 71 len 32768
9710 124/300 Read block 1920 len 8192
9711 125/300 Write block 187 len 4096 data: 3
9712 126/300 Read block 3858 len 16384
9713 127/300 Read block 4773 len 24576
9714 128/300 Flush
9715 129/300 Flush
9716 130/300 Write block 1104 len 12288 data: 3 4 3
9717 131/300 Read block 4884 len 36864
9718 132/300 Read block 3279 len 8192
9719 133/300 Write block 1120 len 40960 data: 2 2 3 3 3 3 4 4 5 6
9720 134/300 Read block 3266 len 40960
9721 135/300 Flush
9722 136/300 Read block 1227 len 24576
9723 137/300 Read block 3138 len 12288
9724 138/300 Flush
9725 139/300 Read block 4548 len 40960
9726 140/300 Flush
9727 141/300 Flush
9728 142/300 Write block 2197 len 28672 data: 4 4 4 3 4 3 4
9729 143/300 Write block 3721 len 4096 data: 4
9730 144/300 Write block 1784 len 8192 data: 3 4
9731 145/300 Read block 1021 len 12288
9732 146/300 Read block 3497 len 16384
9733 147/300 Read block 3681 len 20480
9734 148/300 Write block 4967 len 36864 data: 4 5 5 6 6 4 3 3 2
9735 149/300 Write block 3878 len 16384 data: 4 3 5 4
9736 150/300 Read block 4122 len 8192
9737 151/300 Read block 1640 len 28672
9738 152/300 Write block 1234 len 28672 data: 5 5 5 5 5 5 4
9739 153/300 Write block 3517 len 24576 data: 3 4 6 6 6 6
9740 154/300 Write block 3334 len 28672 data: 3 4 4 4 3 3 3
9741 155/300 Write block 2638 len 32768 data: 5 5 5 5 4 3 3 2
9742 156/300 Read block 1041 len 20480
9743 157/300 Read block 4187 len 24576
9744 158/300 Write block 2209 len 24576 data: 2 2 2 2 2 2
9745 159/300 Read block 2352 len 4096
9746 160/300 Write block 4275 len 4096 data: 5
9747 161/300 Read block 3359 len 40960
9748 162/300 Write block 4610 len 16384 data: 3 2 2 3
9749 163/300 Flush
9750 164/300 Read block 1813 len 24576
9751 165/300 Write block 2345 len 32768 data: 3 3 3 3 2 3 3 3
9752 166/300 Read block 4825 len 36864
9753 167/300 Read block 211 len 20480
9754 168/300 Write block 3442 len 4096 data: 2
9755 169/300 Flush
9756 170/300 Read block 307 len 40960
9757 171/300 Read block 248 len 36864
9758 172/300 Read block 4129 len 4096
9759 173/300 Read block 1027 len 28672
9760 174/300 Flush
9761 175/300 Write block 2995 len 24576 data: 3 3 3 3 3 4
9762 176/300 Read block 1283 len 16384
9763 177/300 Read block 3962 len 8192
9764 178/300 Write block 3491 len 12288 data: 3 4 4
9765 179/300 Read block 2304 len 32768
9766 180/300 Write block 3889 len 4096 data: 2
9767 181/300 Write block 4416 len 12288 data: 3 3 3
9768 182/300 Read block 1866 len 28672
9769 183/300 Read block 4510 len 8192
9770 184/300 Write block 2490 len 24576 data: 3 3 3 4 4 3
9771 185/300 Write block 767 len 28672 data: 2 2 2 2 2 2 3
9772 186/300 Read block 1556 len 8192
9773 187/300 Read block 2247 len 8192
9774 188/300 Flush
9775 189/300 Write block 3653 len 12288 data: 2 2 2
9776 190/300 Read block 2185 len 16384
9777 191/300 Write block 648 len 12288 data: 4 4 3
9778 192/300 Write block 3658 len 8192 data: 3 3
9779 193/300 Write block 4855 len 32768 data: 3 2 3 3 3 3 3 2
9780 194/300 Write block 4667 len 4096 data: 3
9781 195/300 Write block 4127 len 32768 data: 2 2 2 2 2 2 2 3
9782 196/300 Read block 641 len 20480
9783 197/300 Write block 1449 len 40960 data: 3 3 3 3 3 3 2 4 4 5
9784 198/300 Write block 3020 len 4096 data: 5
9785 199/300 Read block 2079 len 20480
9786 200/300 Write block 1229 len 4096 data: 2
9787 201/300 Read block 4884 len 36864
9788 202/300 Read block 954 len 40960
9789 203/300 Write block 4056 len 16384 data: 3 3 3 3
9790 204/300 Write block 2971 len 12288 data: 3 4 4
9791 205/300 Read block 213 len 20480
9792 206/300 Write block 972 len 28672 data: 3 3 4 4 4 3 3
9793 207/300 Read block 1815 len 16384
9794 208/300 Read block 4406 len 32768
9795 209/300 Write block 4111 len 36864 data: 4 4 4 2 2 3 3 5 5
9796 210/300 Write block 1941 len 32768 data: 2 2 3 3 3 3 3 3
9797 211/300 Read block 1828 len 4096
9798 212/300 Write block 4640 len 8192 data: 2 2
9799 213/300 Read block 2900 len 16384
9800 214/300 Write block 235 len 16384 data: 3 3 3 3
9801 215/300 Read block 275 len 32768
9802 216/300 Write block 4713 len 24576 data: 4 4 4 4 4 4
9803 217/300 Flush
9804 218/300 Write block 4549 len 40960 data: 2 2 2 2 2 2 2 3 2 2
9805 219/300 Write block 2182 len 20480 data: 5 4 5 4 5
9806 220/300 Read block 4679 len 4096
9807 221/300 Write block 1838 len 16384 data: 2 2 2 2
9808 222/300 Read block 3051 len 20480
9809 223/300 Read block 3995 len 4096
9810 224/300 Read block 1455 len 4096
9811 225/300 Flush
9812 226/300 Read block 2686 len 40960
9813 227/300 Write block 3323 len 20480 data: 3 3 4 5 3
9814 228/300 Write block 1339 len 40960 data: 4 3 2 3 3 3 3 3 3 3
9815 229/300 Read block 223 len 24576
9816 230/300 Read block 3053 len 40960
9817 231/300 Write block 1966 len 20480 data: 2 2 2 2 2
9818 232/300 Read block 4533 len 20480
9819 233/300 Write block 4055 len 36864 data: 3 4 4 4 4 2 2 2 2
9820 234/300 Read block 529 len 40960
9821 235/300 Read block 2446 len 20480
9822 236/300 Read block 3386 len 36864
9823 237/300 Read block 1565 len 8192
9824 238/300 Write block 709 len 40960 data: 3 3 4 4 9 8 7 6 4 4
9825 239/300 Write block 3660 len 36864 data: 3 3 3 3 3 4 4 4 4
9826 240/300 Write block 337 len 28672 data: 3 3 3 3 3 3 3
9827 241/300 Read block 2409 len 28672
9828 242/300 Write block 4927 len 24576 data: 4 4 3 2 2 2
9829 243/300 Write block 3894 len 40960 data: 4 4 5 5 5 4 3 3 4 3
9830 244/300 Read block 3000 len 28672
9831 245/300 Write block 2111 len 8192 data: 2 2
9832 246/300 Write block 2683 len 24576 data: 6 6 4 4 3 3
9833 247/300 Read block 229 len 24576
9834 248/300 Write block 3389 len 40960 data: 5 4 4 3 3 2 2 2 5 5
9835 249/300 Read block 4576 len 4096
9836 250/300 Read block 1278 len 32768
9837 251/300 Write block 168 len 8192 data: 5 5
9838 252/300 Read block 2579 len 24576
9839 253/300 Write block 4023 len 28672 data: 3 2 2 2 3 2 2
9840 254/300 Write block 3664 len 8192 data: 4 5
9841 255/300 Read block 2132 len 40960
9842 256/300 Write block 288 len 36864 data: 4 4 3 3 3 3 4 4 4
9843 257/300 Write block 393 len 28672 data: 3 3 3 3 4 3 3
9844 258/300 Read block 2342 len 32768
9845 259/300 Flush
9846 260/300 Write block 866 len 24576 data: 5 5 6 4 4 5
9847 261/300 Read block 1028 len 36864
9848 262/300 Read block 80 len 28672
9849 263/300 Read block 926 len 16384
9850 264/300 Read block 1733 len 4096
9851 265/300 Read block 3461 len 20480
9852 266/300 Read block 1884 len 24576
9853 267/300 Read block 4677 len 28672
9854 268/300 Read block 2017 len 28672
9855 269/300 Read block 1544 len 28672
9856 270/300 Read block 3208 len 16384
9857 271/300 Write block 1444 len 4096 data: 2
9858 272/300 Read block 2859 len 4096
9859 273/300 Read block 4178 len 20480
9860 274/300 Read block 1994 len 8192
9861 275/300 Read block 4035 len 4096
9862 276/300 Read block 3576 len 24576
9863 277/300 Read block 4589 len 12288
9864 278/300 Read block 205 len 28672
9865 279/300 Write block 1554 len 36864 data: 3 3 3 4 4 4 3 2 2
9866 280/300 Write block 1556 len 20480 data: 4 5 5 5 4
9867 281/300 Write block 1434 len 12288 data: 2 2 2
9868 282/300 Write block 860 len 40960 data: 3 4 4 3 3 3 6 6 7 5
9869 283/300 Write block 830 len 40960 data: 4 4 5 4 4 4 3 2 2 2
9870 284/300 Write block 4457 len 8192 data: 2 2
9871 285/300 Write block 1529 len 24576 data: 2 3 4 4 3 3
9872 286/300 Write block 1866 len 40960 data: 3 2 2 2 3 3 3 3 3 3
9873 287/300 Read block 4713 len 8192
9874 288/300 Write block 3287 len 8192 data: 3 2
9875 289/300 Read block 3725 len 12288
9876 290/300 Read block 3860 len 16384
9877 291/300 Flush
9878 292/300 Write block 1978 len 4096 data: 3
9879 293/300 Flush
9880 294/300 Read block 3797 len 16384
9881 295/300 Read block 1401 len 32768
9882 296/300 Write block 2773 len 4096 data: 4
9883 297/300 Write block 4638 len 24576 data: 3 3 3 3 2 3
9884 298/300 Write block 3090 len 32768 data: 2 4 4 4 4 4 3 3
9885 299/300 Flush
9886 300/300 Flush
9887 301/300 Read block 3479 len 8192
9888 Main task has work to do, go find it
9889 got dsc Start(1)
9890 start 1
9891 Jan 27 19:38:27.364 INFO request completed, latency_us: 200, response_code: 204, uri: /start/cid/1, method: POST, req_id: fd56c5bd-ba34-4c12-9e20-b3a9856d9550, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
9892 Replay: started 1, returned:Ok(())
9893 ----------------------------------------------------------------
9894 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
9895 States: Active Offline Active
9896 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
9897 3715 Acked Write 6 Done Sent Done false
9898 3716 Acked Write 4 Done Sent Done false
9899 3717 Acked Flush 0 Done Sent Done false
9900 3718 Acked Read 1 Done Sent Done false
9901 3719 Acked Read 4 Done Sent Done false
9902 3720 Acked Flush 0 Done Sent Done false
9903 3721 Acked Read 9 Done Sent Done false
9904 3722 Acked Read 7 Done Sent Done false
9905 3723 Acked Write 10 Done Sent Done false
9906 3724 Acked Flush 0 Done Sent Done false
9907 3725 Acked Read 1 Done Sent Done false
9908 3726 Acked Flush 0 Done Sent Done false
9909 3727 Acked Write 10 Done Sent Done false
9910 3728 Acked Read 10 Done Sent Done false
9911 3729 Acked Write 8 Done Sent Done false
9912 3730 Acked Write 4 Done Sent Done false
9913 3731 Acked Write 5 Done Sent Done false
9914 3732 Acked Read 5 Done Sent Done false
9915 3733 Acked Read 5 Done Sent Done false
9916 3734 Acked Write 2 Done Sent Done false
9917 3735 Acked Flush 0 Done Sent Done false
9918 3736 Acked Write 5 Done Sent Done false
9919 3737 Acked Read 7 Done Sent Done false
9920 3738 Acked Read 3 Done Sent Done false
9921 3739 Acked Read 1 Done Sent Done false
9922 3740 Acked Read 2 Done Sent Done false
9923 3741 Acked Write 5 Done Sent Done false
9924 3742 Acked Write 3 Done Sent Done false
9925 3743 Acked Flush 0 Done Sent Done false
9926 3744 Acked Read 9 Done Sent Done false
9927 3745 Acked Flush 0 Done Sent Done false
9928 3746 Acked Flush 0 Done Sent Done false
9929 3747 Acked Write 9 Done Sent Done false
9930 3748 Acked Read 2 Done Sent Done false
9931 3749 Acked Write 3 Done Sent Done false
9932 3750 Acked Write 1 Done Sent Done false
9933 3751 Acked Write 6 Done Sent Done false
9934 3752 Acked Read 5 Done Sent Done false
9935 3753 Acked Read 7 Done Sent Done false
9936 3754 Acked Read 10 Done Sent Done false
9937 3755 Acked Write 2 Done Sent Done false
9938 3756 Acked Write 4 Done Sent Done false
9939 3757 Acked Read 7 Done Sent Done false
9940 3758 Acked Read 3 Done Sent Done false
9941 3759 Acked Read 10 Done Sent Done false
9942 3760 Acked Flush 0 Done Sent Done false
9943 3761 Acked Write 3 Done Sent Done false
9944 3762 Acked Read 6 Done Sent Done false
9945 3763 Acked Flush 0 Done Sent Done false
9946 3764 Acked Read 3 Done Sent Done false
9947 3765 Acked Read 3 Done Sent Done false
9948 3766 Acked Write 6 Done Sent Done false
9949 3767 Acked Flush 0 Done Sent Done false
9950 3768 Acked Read 3 Done Sent Done false
9951 3769 Acked Flush 0 Done Sent Done false
9952 3770 Acked Read 1 Done Sent Done false
9953 3771 Acked Write 9 Done Sent Done false
9954 3772 Acked Write 3 Done Sent Done false
9955 3773 Acked Write 7 Done Sent Done false
9956 3774 Acked Read 9 Done Sent Done false
9957 3775 Acked Read 4 Done Sent Done false
9958 3776 Acked Write 8 Done Sent Done false
9959 3777 Acked Write 9 Done Sent Done false
9960 3778 Acked Read 3 Done Sent Done false
9961 3779 Acked Flush 0 Done Sent Done false
9962 3780 Acked Read 7 Done Sent Done false
9963 3781 Acked Write 2 Done Sent Done false
9964 3782 Acked Read 10 Done Sent Done false
9965 3783 Acked Write 7 Done Sent Done false
9966 3784 Acked Read 2 Done Sent Done false
9967 3785 Acked Write 3 Done Sent Done false
9968 3786 Acked Read 1 Done Sent Done false
9969 3787 Acked Read 2 Done Sent Done false
9970 3788 Acked Read 9 Done Sent Done false
9971 3789 Acked Write 3 Done Sent Done false
9972 3790 Acked Write 10 Done Sent Done false
9973 3791 Acked Read 3 Done Sent Done false
9974 3792 Acked Write 2 Done Sent Done false
9975 3793 Acked Read 3 Done Sent Done false
9976 3794 Acked Read 10 Done Sent Done false
9977 3795 Acked Write 6 Done Sent Done false
9978 3796 Acked Read 5 Done Sent Done false
9979 3797 Acked Read 6 Done Sent Done false
9980 3798 Acked Read 10 Done Sent Done false
9981 3799 Acked Read 4 Done Sent Done false
9982 3800 Acked Read 8 Done Sent Done false
9983 3801 Acked Read 10 Done Sent Done false
9984 3802 Acked Read 1 Done Sent Done false
9985 3803 Acked Write 5 Done Sent Done false
9986 3804 Acked Read 4 Done Sent Done false
9987 3805 Acked Write 7 Done Sent Done false
9988 3806 Acked Write 6 Done Sent Done false
9989 3807 Acked Read 6 Done Sent Done false
9990 3808 Acked Read 10 Done Sent Done false
9991 3809 Acked Write 2 Done Sent Done false
9992 3810 Acked Read 2 Done Sent Done false
9993 3811 Acked Flush 0 Done Sent Done false
9994 3812 Acked Read 4 Done Sent Done false
9995 3813 Acked Read 8 Done Sent Done false
9996 3814 Acked Write 4 Done Sent Done false
9997 3815 Acked Write 2 Done Sent Done false
9998 3816 Acked Write 4 Done Sent Done false
9999 3817 Acked Flush 0 Done Sent Done false
10000 3818 Acked Read 8 Done Sent Done false
10001 3819 Acked Read 9 Done Sent Done false
10002 3820 Acked Write 10 Done Sent Done false
10003 3821 Acked Write 5 Done Sent Done false
10004 3822 Acked Write 3 Done Sent Done false
10005 3823 Acked Read 8 Done Sent Done false
10006 3824 Acked Read 6 Done Sent Done false
10007 3825 Acked Read 10 Done Sent Done false
10008 3826 Acked Read 4 Done Sent Done false
10009 3827 Acked Read 2 Done Sent Done false
10010 3828 Acked Read 10 Done Sent Done false
10011 3829 Acked Write 3 Done Sent Done false
10012 3830 Acked Flush 0 Done Sent Done false
10013 3831 Acked Write 1 Done Sent Done false
10014 3832 Acked Read 7 Done Sent Done false
10015 3833 Acked Read 8 Done Sent Done false
10016 3834 Acked Write 10 Done Sent Done false
10017 3835 Acked Read 6 Done Sent Done false
10018 3836 Acked Write 5 Done Sent Done false
10019 3837 Acked Read 8 Done Sent Done false
10020 3838 Acked Read 2 Done Sent Done false
10021 3839 Acked Write 1 Done Sent Done false
10022 3840 Acked Read 4 Done Sent Done false
10023 3841 Acked Read 6 Done Sent Done false
10024 3842 Acked Flush 0 Done Sent Done false
10025 3843 Acked Flush 0 Done Sent Done false
10026 3844 Acked Write 3 Done Sent Done false
10027 3845 Acked Read 9 Done Sent Done false
10028 3846 Acked Read 2 Done Sent Done false
10029 3847 Acked Write 10 Done Sent Done false
10030 3848 Acked Read 10 Done Sent Done false
10031 3849 Acked Flush 0 Done Sent Done false
10032 3850 Acked Read 6 Done Sent Done false
10033 3851 Acked Read 3 Done Sent Done false
10034 3852 Acked Flush 0 Done Sent Done false
10035 3853 Acked Read 10 Done Sent Done false
10036 3854 Acked Flush 0 Done Sent Done false
10037 3855 Acked Flush 0 Done Sent Done false
10038 3856 Acked Write 7 Done Sent Done false
10039 3857 Acked Write 1 Done Sent Done false
10040 3858 Acked Write 2 Done Sent Done false
10041 3859 Acked Read 3 Done Sent Done false
10042 3860 Acked Read 4 Done Sent Done false
10043 3861 Acked Read 5 Done Sent Done false
10044 3862 Acked Write 9 Done Sent Done false
10045 3863 Acked Write 4 Done Sent Done false
10046 3864 Acked Read 2 Done Sent Done false
10047 3865 Acked Read 7 Done Sent Done false
10048 3866 Acked Write 7 Done Sent Done false
10049 3867 Acked Write 6 Done Sent Done false
10050 3868 Acked Write 7 Done Sent Done false
10051 3869 Acked Write 8 Done Sent Done false
10052 3870 Acked Read 5 Done Sent Done false
10053 3871 Acked Read 6 Done Sent Done false
10054 3872 Acked Write 6 Done Sent Done false
10055 3873 Acked Read 1 Done Sent Done false
10056 3874 Acked Write 1 Done Sent Done false
10057 3875 Acked Read 10 Done Sent Done false
10058 3876 Acked Write 4 Done Sent Done false
10059 3877 Acked Flush 0 Done Sent Done false
10060 3878 Acked Read 6 Done Sent Done false
10061 3879 Acked Write 8 Done Sent Done false
10062 3880 Acked Read 9 Done Sent Done false
10063 3881 Acked Read 5 Done Sent Done false
10064 3882 Acked Write 1 Done Sent Done false
10065 3883 Acked Flush 0 Done Sent Done false
10066 3884 Acked Read 10 Done Sent Done false
10067 3885 Acked Read 9 Done Sent Done false
10068 3886 Acked Read 1 Done Sent Done false
10069 3887 Acked Read 7 Done Sent Done false
10070 3888 Acked Flush 0 Done Sent Done false
10071 3889 Acked Write 6 Done Sent Done false
10072 3890 Acked Read 4 Done Sent Done false
10073 3891 Acked Read 2 Done Sent Done false
10074 3892 Acked Write 3 Done Sent Done false
10075 3893 Acked Read 8 Done Sent Done false
10076 3894 Acked Write 1 Done Sent Done false
10077 3895 Acked Write 3 Done Sent Done false
10078 3896 Acked Read 7 Done Sent Done false
10079 3897 Acked Read 2 Done Sent Done false
10080 3898 Acked Write 6 Done Sent Done false
10081 3899 Acked Write 7 Done Sent Done false
10082 3900 Acked Read 2 Done Sent Done false
10083 3901 Acked Read 2 Done Sent Done false
10084 3902 Acked Flush 0 Done Sent Done false
10085 3903 Acked Write 3 Done Sent Done false
10086 3904 Acked Read 4 Done Sent Done false
10087 3905 Acked Write 3 Done Sent Done false
10088 3906 Acked Write 2 Done Sent Done false
10089 3907 Acked Write 8 Done Sent Done false
10090 3908 Acked Write 1 Done Sent Done false
10091 3909 Acked Write 8 Done Sent Done false
10092 3910 Acked Read 5 Done Sent Done false
10093 3911 Acked Write 10 Done Sent Done false
10094 3912 Acked Write 1 Done Sent Done false
10095 3913 Acked Read 5 Done Sent Done false
10096 3914 Acked Write 1 Done Sent Done false
10097 3915 Acked Read 9 Done Sent Done false
10098 3916 Acked Read 10 Done Sent Done false
10099 3917 Acked Write 4 Done Sent Done false
10100 3918 Acked Write 3 Done Sent Done false
10101 3919 Acked Read 5 Done Sent Done false
10102 3920 Acked Write 7 Done Sent Done false
10103 3921 Acked Read 4 Done Sent Done false
10104 3922 Acked Read 8 Done Sent Done false
10105 3923 Acked Write 9 Done Sent Done false
10106 3924 Acked Write 8 Done Sent Done false
10107 3925 Acked Read 1 Done Sent Done false
10108 3926 Acked Write 2 Done Sent Done false
10109 3927 Acked Read 4 Done Sent Done false
10110 3928 Acked Write 4 Done Sent Done false
10111 3929 Acked Read 8 Done Sent Done false
10112 3930 Acked Write 6 Done Sent Done false
10113 3931 Acked Flush 0 Done Sent Done false
10114 3932 Acked Write 10 Done Sent Done false
10115 3933 Acked Write 5 Done Sent Done false
10116 3934 Acked Read 1 Done Sent Done false
10117 3935 Acked Write 4 Done Sent Done false
10118 3936 Acked Read 5 Done Sent Done false
10119 3937 Acked Read 1 Done Sent Done false
10120 3938 Acked Read 1 Done Sent Done false
10121 3939 Acked Flush 0 Done Sent Done false
10122 3940 Acked Read 10 Done Sent Done false
10123 3941 Acked Write 5 Done Sent Done false
10124 3942 Acked Write 10 Done Sent Done false
10125 3943 Acked Read 6 Done Sent Done false
10126 3944 Acked Read 10 Done Sent Done false
10127 3945 Acked Write 5 Done Sent Done false
10128 3946 Acked Read 5 Done Sent Done false
10129 3947 Acked Write 9 Done Sent Done false
10130 3948 Acked Read 10 Done Sent Done false
10131 3949 Acked Read 5 Done Sent Done false
10132 3950 Acked Read 9 Done Sent Done false
10133 3951 Acked Read 2 Done Sent Done false
10134 3952 Acked Write 10 Done Sent Done false
10135 3953 Acked Write 9 Done Sent Done false
10136 3954 Acked Write 7 Done Sent Done false
10137 3955 Acked Read 7 Done Sent Done false
10138 3956 Acked Write 6 Done Sent Done false
10139 3957 Acked Write 10 Done Sent Done false
10140 3958 Acked Read 7 Done Sent Done false
10141 3959 Acked Write 2 Done Sent Done false
10142 3960 Acked Write 6 Done Sent Done false
10143 3961 Acked Read 6 Done Sent Done false
10144 3962 Acked Write 10 Done Sent Done false
10145 3963 Acked Read 1 Done Sent Done false
10146 3964 Acked Read 8 Done Sent Done false
10147 3965 Acked Write 2 Done Sent Done false
10148 3966 Acked Read 6 Done Sent Done false
10149 3967 Acked Write 7 Done Sent Done false
10150 3968 Acked Write 2 Done Sent Done false
10151 3969 Acked Read 10 Done Sent Done false
10152 3970 Acked Write 9 Done Sent Done false
10153 3971 Acked Write 7 Done Sent Done false
10154 3972 Acked Read 8 Done Sent Done false
10155 3973 Acked Flush 0 Done Sent Done false
10156 3974 Acked Write 6 Done Sent Done false
10157 3975 Acked Read 9 Done Sent Done false
10158 3976 Acked Read 7 Done Sent Done false
10159 3977 Acked Read 4 Done Sent Done false
10160 3978 Acked Read 1 Done Sent Done false
10161 3979 Acked Read 5 Done Sent Done false
10162 3980 Acked Read 6 Done Sent Done false
10163 3981 Acked Read 7 Done Sent Done false
10164 3982 Acked Read 7 Done Sent Done false
10165 3983 Acked Read 7 Done Sent Done false
10166 3984 Acked Read 4 Done Sent Done false
10167 3985 Acked Write 1 Done Sent Done false
10168 3986 Acked Read 1 Done Sent Done false
10169 3987 Acked Read 5 Done Sent Done false
10170 3988 Acked Read 2 Done Sent Done false
10171 3989 Acked Read 1 Done Sent Done false
10172 3990 Acked Read 6 Done Sent Done false
10173 3991 Acked Read 3 Done Sent Done false
10174 3992 Acked Read 7 Done Sent Done false
10175 3993 Acked Write 9 Done Sent Done false
10176 3994 Acked Write 5 Done Sent Done false
10177 3995 Acked Write 3 Done Sent Done false
10178 3996 Acked Write 10 Done Sent Done false
10179 3997 Acked Write 10 Done Sent Done false
10180 3998 Acked Write 2 Done Sent Done false
10181 3999 Acked Write 6 Done Sent Done false
10182 4000 Acked Write 10 Done Sent Done false
10183 4001 Acked Read 2 Done Sent Done false
10184 4002 Acked Write 2 Done Sent Done false
10185 4003 Acked Read 3 Done Sent Done false
10186 4004 Acked Read 4 Done Sent Done false
10187 4005 Acked Flush 0 Done Sent Done false
10188 4006 Acked Write 1 Done Sent Done false
10189 4007 Acked Flush 0 Done Sent Done false
10190 4008 Acked Read 4 Done Sent Done false
10191 4009 Acked Read 8 Done Sent Done false
10192 4010 Acked Write 1 Done Sent Done false
10193 4011 Acked Write 6 Done Sent Done false
10194 4012 Acked Write 8 Done Sent Done false
10195 4013 Acked Flush 0 Done Sent Done false
10196 4014 Acked Flush 0 Done Sent Done false
10197 4015 Acked Read 2 Done Sent Done false
10198 STATES DS:0 DS:1 DS:2 TOTAL
10199 Sent 0 301 0 301
10200 Done 301 0 301 602
10201 Skipped 0 0 0 0
10202 Error 0 0 0 0
10203 Last Flush: 4014 3714 4014
10204 Downstairs last five completed: 3714 3713 3712 3711 3710
10205 Upstairs last five completed: 4015 4014 4013 4012 4011
10206 CLIENT: Up:0 ds:301 act:2
10207 [8820] got start action
10208 [8820] I am going to restart
10209 Starting downstairs at port 8820
10210 Make output file at "/tmp/dsc/downstairs-8820.txt"
10211 [8820][1] reports Starting
10212 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1210)
10213 [8820][1] reports Running
10214 ----------------------------------------------------------------
10215 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
10216 States: Active Offline Active
10217 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10218 3715 Acked Write 6 Done Sent Done false
10219 3716 Acked Write 4 Done Sent Done false
10220 3717 Acked Flush 0 Done Sent Done false
10221 3718 Acked Read 1 Done Sent Done false
10222 3719 Acked Read 4 Done Sent Done false
10223 3720 Acked Flush 0 Done Sent Done false
10224 3721 Acked Read 9 Done Sent Done false
10225 3722 Acked Read 7 Done Sent Done false
10226 3723 Acked Write 10 Done Sent Done false
10227 3724 Acked Flush 0 Done Sent Done false
10228 3725 Acked Read 1 Done Sent Done false
10229 3726 Acked Flush 0 Done Sent Done false
10230 3727 Acked Write 10 Done Sent Done false
10231 3728 Acked Read 10 Done Sent Done false
10232 3729 Acked Write 8 Done Sent Done false
10233 3730 Acked Write 4 Done Sent Done false
10234 3731 Acked Write 5 Done Sent Done false
10235 3732 Acked Read 5 Done Sent Done false
10236 3733 Acked Read 5 Done Sent Done false
10237 3734 Acked Write 2 Done Sent Done false
10238 3735 Acked Flush 0 Done Sent Done false
10239 3736 Acked Write 5 Done Sent Done false
10240 3737 Acked Read 7 Done Sent Done false
10241 3738 Acked Read 3 Done Sent Done false
10242 3739 Acked Read 1 Done Sent Done false
10243 3740 Acked Read 2 Done Sent Done false
10244 3741 Acked Write 5 Done Sent Done false
10245 3742 Acked Write 3 Done Sent Done false
10246 3743 Acked Flush 0 Done Sent Done false
10247 3744 Acked Read 9 Done Sent Done false
10248 3745 Acked Flush 0 Done Sent Done false
10249 3746 Acked Flush 0 Done Sent Done false
10250 3747 Acked Write 9 Done Sent Done false
10251 3748 Acked Read 2 Done Sent Done false
10252 3749 Acked Write 3 Done Sent Done false
10253 3750 Acked Write 1 Done Sent Done false
10254 3751 Acked Write 6 Done Sent Done false
10255 3752 Acked Read 5 Done Sent Done false
10256 3753 Acked Read 7 Done Sent Done false
10257 3754 Acked Read 10 Done Sent Done false
10258 3755 Acked Write 2 Done Sent Done false
10259 3756 Acked Write 4 Done Sent Done false
10260 3757 Acked Read 7 Done Sent Done false
10261 3758 Acked Read 3 Done Sent Done false
10262 3759 Acked Read 10 Done Sent Done false
10263 3760 Acked Flush 0 Done Sent Done false
10264 3761 Acked Write 3 Done Sent Done false
10265 3762 Acked Read 6 Done Sent Done false
10266 3763 Acked Flush 0 Done Sent Done false
10267 3764 Acked Read 3 Done Sent Done false
10268 3765 Acked Read 3 Done Sent Done false
10269 3766 Acked Write 6 Done Sent Done false
10270 3767 Acked Flush 0 Done Sent Done false
10271 3768 Acked Read 3 Done Sent Done false
10272 3769 Acked Flush 0 Done Sent Done false
10273 3770 Acked Read 1 Done Sent Done false
10274 3771 Acked Write 9 Done Sent Done false
10275 3772 Acked Write 3 Done Sent Done false
10276 3773 Acked Write 7 Done Sent Done false
10277 3774 Acked Read 9 Done Sent Done false
10278 3775 Acked Read 4 Done Sent Done false
10279 3776 Acked Write 8 Done Sent Done false
10280 3777 Acked Write 9 Done Sent Done false
10281 3778 Acked Read 3 Done Sent Done false
10282 3779 Acked Flush 0 Done Sent Done false
10283 3780 Acked Read 7 Done Sent Done false
10284 3781 Acked Write 2 Done Sent Done false
10285 3782 Acked Read 10 Done Sent Done false
10286 3783 Acked Write 7 Done Sent Done false
10287 3784 Acked Read 2 Done Sent Done false
10288 3785 Acked Write 3 Done Sent Done false
10289 3786 Acked Read 1 Done Sent Done false
10290 3787 Acked Read 2 Done Sent Done false
10291 3788 Acked Read 9 Done Sent Done false
10292 3789 Acked Write 3 Done Sent Done false
10293 3790 Acked Write 10 Done Sent Done false
10294 3791 Acked Read 3 Done Sent Done false
10295 3792 Acked Write 2 Done Sent Done false
10296 3793 Acked Read 3 Done Sent Done false
10297 3794 Acked Read 10 Done Sent Done false
10298 3795 Acked Write 6 Done Sent Done false
10299 3796 Acked Read 5 Done Sent Done false
10300 3797 Acked Read 6 Done Sent Done false
10301 3798 Acked Read 10 Done Sent Done false
10302 3799 Acked Read 4 Done Sent Done false
10303 3800 Acked Read 8 Done Sent Done false
10304 3801 Acked Read 10 Done Sent Done false
10305 3802 Acked Read 1 Done Sent Done false
10306 3803 Acked Write 5 Done Sent Done false
10307 3804 Acked Read 4 Done Sent Done false
10308 3805 Acked Write 7 Done Sent Done false
10309 3806 Acked Write 6 Done Sent Done false
10310 3807 Acked Read 6 Done Sent Done false
10311 3808 Acked Read 10 Done Sent Done false
10312 3809 Acked Write 2 Done Sent Done false
10313 3810 Acked Read 2 Done Sent Done false
10314 3811 Acked Flush 0 Done Sent Done false
10315 3812 Acked Read 4 Done Sent Done false
10316 3813 Acked Read 8 Done Sent Done false
10317 3814 Acked Write 4 Done Sent Done false
10318 3815 Acked Write 2 Done Sent Done false
10319 3816 Acked Write 4 Done Sent Done false
10320 3817 Acked Flush 0 Done Sent Done false
10321 3818 Acked Read 8 Done Sent Done false
10322 3819 Acked Read 9 Done Sent Done false
10323 3820 Acked Write 10 Done Sent Done false
10324 3821 Acked Write 5 Done Sent Done false
10325 3822 Acked Write 3 Done Sent Done false
10326 3823 Acked Read 8 Done Sent Done false
10327 3824 Acked Read 6 Done Sent Done false
10328 3825 Acked Read 10 Done Sent Done false
10329 3826 Acked Read 4 Done Sent Done false
10330 3827 Acked Read 2 Done Sent Done false
10331 3828 Acked Read 10 Done Sent Done false
10332 3829 Acked Write 3 Done Sent Done false
10333 3830 Acked Flush 0 Done Sent Done false
10334 3831 Acked Write 1 Done Sent Done false
10335 3832 Acked Read 7 Done Sent Done false
10336 3833 Acked Read 8 Done Sent Done false
10337 3834 Acked Write 10 Done Sent Done false
10338 3835 Acked Read 6 Done Sent Done false
10339 3836 Acked Write 5 Done Sent Done false
10340 3837 Acked Read 8 Done Sent Done false
10341 3838 Acked Read 2 Done Sent Done false
10342 3839 Acked Write 1 Done Sent Done false
10343 3840 Acked Read 4 Done Sent Done false
10344 3841 Acked Read 6 Done Sent Done false
10345 3842 Acked Flush 0 Done Sent Done false
10346 3843 Acked Flush 0 Done Sent Done false
10347 3844 Acked Write 3 Done Sent Done false
10348 3845 Acked Read 9 Done Sent Done false
10349 3846 Acked Read 2 Done Sent Done false
10350 3847 Acked Write 10 Done Sent Done false
10351 3848 Acked Read 10 Done Sent Done false
10352 3849 Acked Flush 0 Done Sent Done false
10353 3850 Acked Read 6 Done Sent Done false
10354 3851 Acked Read 3 Done Sent Done false
10355 3852 Acked Flush 0 Done Sent Done false
10356 3853 Acked Read 10 Done Sent Done false
10357 3854 Acked Flush 0 Done Sent Done false
10358 3855 Acked Flush 0 Done Sent Done false
10359 3856 Acked Write 7 Done Sent Done false
10360 3857 Acked Write 1 Done Sent Done false
10361 3858 Acked Write 2 Done Sent Done false
10362 3859 Acked Read 3 Done Sent Done false
10363 3860 Acked Read 4 Done Sent Done false
10364 3861 Acked Read 5 Done Sent Done false
10365 3862 Acked Write 9 Done Sent Done false
10366 3863 Acked Write 4 Done Sent Done false
10367 3864 Acked Read 2 Done Sent Done false
10368 3865 Acked Read 7 Done Sent Done false
10369 3866 Acked Write 7 Done Sent Done false
10370 3867 Acked Write 6 Done Sent Done false
10371 3868 Acked Write 7 Done Sent Done false
10372 3869 Acked Write 8 Done Sent Done false
10373 3870 Acked Read 5 Done Sent Done false
10374 3871 Acked Read 6 Done Sent Done false
10375 3872 Acked Write 6 Done Sent Done false
10376 3873 Acked Read 1 Done Sent Done false
10377 3874 Acked Write 1 Done Sent Done false
10378 3875 Acked Read 10 Done Sent Done false
10379 3876 Acked Write 4 Done Sent Done false
10380 3877 Acked Flush 0 Done Sent Done false
10381 3878 Acked Read 6 Done Sent Done false
10382 3879 Acked Write 8 Done Sent Done false
10383 3880 Acked Read 9 Done Sent Done false
10384 3881 Acked Read 5 Done Sent Done false
10385 3882 Acked Write 1 Done Sent Done false
10386 3883 Acked Flush 0 Done Sent Done false
10387 3884 Acked Read 10 Done Sent Done false
10388 3885 Acked Read 9 Done Sent Done false
10389 3886 Acked Read 1 Done Sent Done false
10390 3887 Acked Read 7 Done Sent Done false
10391 3888 Acked Flush 0 Done Sent Done false
10392 3889 Acked Write 6 Done Sent Done false
10393 3890 Acked Read 4 Done Sent Done false
10394 3891 Acked Read 2 Done Sent Done false
10395 3892 Acked Write 3 Done Sent Done false
10396 3893 Acked Read 8 Done Sent Done false
10397 3894 Acked Write 1 Done Sent Done false
10398 3895 Acked Write 3 Done Sent Done false
10399 3896 Acked Read 7 Done Sent Done false
10400 3897 Acked Read 2 Done Sent Done false
10401 3898 Acked Write 6 Done Sent Done false
10402 3899 Acked Write 7 Done Sent Done false
10403 3900 Acked Read 2 Done Sent Done false
10404 3901 Acked Read 2 Done Sent Done false
10405 3902 Acked Flush 0 Done Sent Done false
10406 3903 Acked Write 3 Done Sent Done false
10407 3904 Acked Read 4 Done Sent Done false
10408 3905 Acked Write 3 Done Sent Done false
10409 3906 Acked Write 2 Done Sent Done false
10410 3907 Acked Write 8 Done Sent Done false
10411 3908 Acked Write 1 Done Sent Done false
10412 3909 Acked Write 8 Done Sent Done false
10413 3910 Acked Read 5 Done Sent Done false
10414 3911 Acked Write 10 Done Sent Done false
10415 3912 Acked Write 1 Done Sent Done false
10416 3913 Acked Read 5 Done Sent Done false
10417 3914 Acked Write 1 Done Sent Done false
10418 3915 Acked Read 9 Done Sent Done false
10419 3916 Acked Read 10 Done Sent Done false
10420 3917 Acked Write 4 Done Sent Done false
10421 3918 Acked Write 3 Done Sent Done false
10422 3919 Acked Read 5 Done Sent Done false
10423 3920 Acked Write 7 Done Sent Done false
10424 3921 Acked Read 4 Done Sent Done false
10425 3922 Acked Read 8 Done Sent Done false
10426 3923 Acked Write 9 Done Sent Done false
10427 3924 Acked Write 8 Done Sent Done false
10428 3925 Acked Read 1 Done Sent Done false
10429 3926 Acked Write 2 Done Sent Done false
10430 3927 Acked Read 4 Done Sent Done false
10431 3928 Acked Write 4 Done Sent Done false
10432 3929 Acked Read 8 Done Sent Done false
10433 3930 Acked Write 6 Done Sent Done false
10434 3931 Acked Flush 0 Done Sent Done false
10435 3932 Acked Write 10 Done Sent Done false
10436 3933 Acked Write 5 Done Sent Done false
10437 3934 Acked Read 1 Done Sent Done false
10438 3935 Acked Write 4 Done Sent Done false
10439 3936 Acked Read 5 Done Sent Done false
10440 3937 Acked Read 1 Done Sent Done false
10441 3938 Acked Read 1 Done Sent Done false
10442 3939 Acked Flush 0 Done Sent Done false
10443 3940 Acked Read 10 Done Sent Done false
10444 3941 Acked Write 5 Done Sent Done false
10445 3942 Acked Write 10 Done Sent Done false
10446 3943 Acked Read 6 Done Sent Done false
10447 3944 Acked Read 10 Done Sent Done false
10448 3945 Acked Write 5 Done Sent Done false
10449 3946 Acked Read 5 Done Sent Done false
10450 3947 Acked Write 9 Done Sent Done false
10451 3948 Acked Read 10 Done Sent Done false
10452 3949 Acked Read 5 Done Sent Done false
10453 3950 Acked Read 9 Done Sent Done false
10454 3951 Acked Read 2 Done Sent Done false
10455 3952 Acked Write 10 Done Sent Done false
10456 3953 Acked Write 9 Done Sent Done false
10457 3954 Acked Write 7 Done Sent Done false
10458 3955 Acked Read 7 Done Sent Done false
10459 3956 Acked Write 6 Done Sent Done false
10460 3957 Acked Write 10 Done Sent Done false
10461 3958 Acked Read 7 Done Sent Done false
10462 3959 Acked Write 2 Done Sent Done false
10463 3960 Acked Write 6 Done Sent Done false
10464 3961 Acked Read 6 Done Sent Done false
10465 3962 Acked Write 10 Done Sent Done false
10466 3963 Acked Read 1 Done Sent Done false
10467 3964 Acked Read 8 Done Sent Done false
10468 3965 Acked Write 2 Done Sent Done false
10469 3966 Acked Read 6 Done Sent Done false
10470 3967 Acked Write 7 Done Sent Done false
10471 3968 Acked Write 2 Done Sent Done false
10472 3969 Acked Read 10 Done Sent Done false
10473 3970 Acked Write 9 Done Sent Done false
10474 3971 Acked Write 7 Done Sent Done false
10475 3972 Acked Read 8 Done Sent Done false
10476 3973 Acked Flush 0 Done Sent Done false
10477 3974 Acked Write 6 Done Sent Done false
10478 3975 Acked Read 9 Done Sent Done false
10479 3976 Acked Read 7 Done Sent Done false
10480 3977 Acked Read 4 Done Sent Done false
10481 3978 Acked Read 1 Done Sent Done false
10482 3979 Acked Read 5 Done Sent Done false
10483 3980 Acked Read 6 Done Sent Done false
10484 3981 Acked Read 7 Done Sent Done false
10485 3982 Acked Read 7 Done Sent Done false
10486 3983 Acked Read 7 Done Sent Done false
10487 3984 Acked Read 4 Done Sent Done false
10488 3985 Acked Write 1 Done Sent Done false
10489 3986 Acked Read 1 Done Sent Done false
10490 3987 Acked Read 5 Done Sent Done false
10491 3988 Acked Read 2 Done Sent Done false
10492 3989 Acked Read 1 Done Sent Done false
10493 3990 Acked Read 6 Done Sent Done false
10494 3991 Acked Read 3 Done Sent Done false
10495 3992 Acked Read 7 Done Sent Done false
10496 3993 Acked Write 9 Done Sent Done false
10497 3994 Acked Write 5 Done Sent Done false
10498 3995 Acked Write 3 Done Sent Done false
10499 3996 Acked Write 10 Done Sent Done false
10500 3997 Acked Write 10 Done Sent Done false
10501 3998 Acked Write 2 Done Sent Done false
10502 3999 Acked Write 6 Done Sent Done false
10503 4000 Acked Write 10 Done Sent Done false
10504 4001 Acked Read 2 Done Sent Done false
10505 4002 Acked Write 2 Done Sent Done false
10506 4003 Acked Read 3 Done Sent Done false
10507 4004 Acked Read 4 Done Sent Done false
10508 4005 Acked Flush 0 Done Sent Done false
10509 4006 Acked Write 1 Done Sent Done false
10510 4007 Acked Flush 0 Done Sent Done false
10511 4008 Acked Read 4 Done Sent Done false
10512 4009 Acked Read 8 Done Sent Done false
10513 4010 Acked Write 1 Done Sent Done false
10514 4011 Acked Write 6 Done Sent Done false
10515 4012 Acked Write 8 Done Sent Done false
10516 4013 Acked Flush 0 Done Sent Done false
10517 4014 Acked Flush 0 Done Sent Done false
10518 4015 Acked Read 2 Done Sent Done false
10519 4016 Acked Flush 0 Done Sent Done false
10520 STATES DS:0 DS:1 DS:2 TOTAL
10521 Sent 0 302 0 302
10522 Done 302 0 302 604
10523 Skipped 0 0 0 0
10524 Error 0 0 0 0
10525 Last Flush: 4016 3714 4016
10526 Downstairs last five completed: 3714 3713 3712 3711 3710
10527 Upstairs last five completed: 4016 4015 4014 4013 4012
10528 CLIENT: Up:0 ds:302 act:2
105292025-01-27T19:38:33.132ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105302025-01-27T19:38:33.132ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35527) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105312025-01-27T19:38:33.133ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105322025-01-27T19:38:33.134ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105332025-01-27T19:38:33.134ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105342025-01-27T19:38:33.134ZINFOcrucible: send last flush ID to this DS: 3714 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105352025-01-27T19:38:33.134ZINFOcrucible: Replied this last flush ID: 3714 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105362025-01-27T19:38:33.134ZINFOcrucible: [1] client re-new 302 jobs since flush 3714 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105372025-01-27T19:38:33.135ZINFOcrucible: [1] Replayed 302 jobs since flush: 3714 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
10538 ----------------------------------------------------------------
10539 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
10540 Downstairs last five completed: 4016 4015 4014 4013 4012
10541 Upstairs last five completed: 4016 4015 4014 4013 4012
10542 CLIENT: Up:0 ds:0 act:3
10543 Replay: All jobs finished
10544 Main task has work to do, go find it
10545 got dsc Stop(2)
10546 stop 2
10547 Jan 27 19:38:35.369 [8830] Got stop action so:false kr:false
10548 INFO request completed, latency_us: 428, response_code: 204, uri: /stop/cid/2, method: POST, req_id: f40ac462-28d4-4bca-a934-e9b5c95475ba, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
10549 Jan 27 19:38:35.370 INFO request completed, latency_us: 133, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7eb26a11-abda-4c83-9f62-ce86232cb9f5, remote_addr: 127.0.0.1:55047, local_addr: 127{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:38:35.370435952Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
10550 .0.0.1:9998
105512025-01-27T19:38:35.370ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105522025-01-27T19:38:35.370ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
105532025-01-27T19:38:35.370ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
10554 [8830] Exited with: ExitStatus(unix_wait_status(9))
10555 [8830][2] reports Exit
10556 Jan 27 19:38:39.372 INFO request completed, latency_us: 310, response_code: 200, uri: /state/cid/2, method: GET, req_id: 6714c97f-9527-4f5e-9bbc-f600b6b72a76, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
10557 001/300 Write block 2041 len 12288 data: 2 3 3
10558 002/300 Write block 2299 len 28672 data: 3 3 3 4 4 4 4
10559 003/300 Read block 2457 len 16384
10560 004/300 Write block 3959 len 40960 data: 4 3 3 3 4 3 4 3 3 3
10561 005/300 Read block 1493 len 8192
10562 006/300 Write block 3103 len 28672 data: 3 3 3 3 3 3 2
10563 007/300 Read block 1019 len 36864
10564 008/300 Read block 416 len 28672
10565 009/300 Read block 4022 len 32768
10566 010/300 Write block 1949 len 20480 data: 2 3 3 4 4
10567 011/300 Flush
10568 012/300 Read block 3237 len 20480
10569 013/300 Write block 1225 len 16384 data: 4 3 2 2
10570 014/300 Read block 1717 len 8192
10571 015/300 Flush
10572 016/300 Read block 136 len 4096
10573 017/300 Write block 1873 len 16384 data: 4 4 4 3
10574 018/300 Read block 148 len 40960
10575 019/300 Read block 3739 len 16384
10576 020/300 Read block 3812 len 12288
10577 021/300 Write block 1920 len 24576 data: 2 2 2 3 4 4
10578 022/300 Write block 1815 len 4096 data: 4
10579 023/300 Write block 281 len 32768 data: 9 7 5 4 4 4 3 5
10580 024/300 Read block 4209 len 20480
10581 025/300 Read block 1045 len 12288
10582 026/300 Write block 842 len 4096 data: 2
10583 027/300 Read block 2091 len 36864
10584 028/300 Write block 3696 len 32768 data: 3 3 2 2 2 2 2 3
10585 029/300 Read block 1254 len 40960
10586 030/300 Read block 2690 len 32768
10587 031/300 Read block 4134 len 20480
10588 032/300 Read block 4150 len 12288
10589 033/300 Read block 331 len 4096
10590 034/300 Write block 1735 len 28672 data: 3 3 2 2 2 2 2
10591 035/300 Flush
10592 036/300 Read block 1119 len 4096
10593 037/300 Write block 1996 len 32768 data: 4 4 4 4 2 2 2 2
10594 038/300 Write block 843 len 16384 data: 2 2 2 3
10595 039/300 Read block 3491 len 4096
10596 040/300 Read block 3553 len 24576
10597 041/300 Write block 54 len 28672 data: 3 2 2 2 3 3 4
10598 042/300 Read block 2601 len 16384
10599 043/300 Read block 2162 len 16384
10600 044/300 Read block 3036 len 16384
10601 045/300 Write block 3474 len 40960 data: 2 2 2 2 2 3 3 3 3 2
10602 046/300 Write block 2234 len 4096 data: 3
10603 047/300 Write block 4347 len 36864 data: 3 3 3 3 3 3 3 2 3
10604 048/300 Read block 2867 len 40960
10605 049/300 Read block 1148 len 40960
10606 050/300 Read block 3476 len 4096
10607 051/300 Read block 4845 len 40960
10608 052/300 Write block 1203 len 28672 data: 3 3 3 3 4 3 3
10609 053/300 Read block 3738 len 40960
10610 054/300 Read block 4115 len 36864
10611 055/300 Read block 2893 len 12288
10612 056/300 Read block 1123 len 36864
10613 057/300 Read block 2774 len 12288
10614 058/300 Read block 3859 len 32768
10615 059/300 Write block 3869 len 24576 data: 3 3 2 3 4 4
10616 060/300 Read block 3330 len 12288
10617 061/300 Write block 2288 len 20480 data: 7 6 6 4 4
10618 062/300 Read block 2956 len 40960
10619 063/300 Write block 3711 len 16384 data: 5 4 4 4
10620 064/300 Write block 1033 len 28672 data: 6 6 5 5 4 2 2
10621 065/300 Read block 3176 len 32768
10622 066/300 Read block 2526 len 24576
10623 067/300 Write block 2087 len 4096 data: 2
10624 068/300 Write block 179 len 32768 data: 3 3 3 3 3 4 4 4
10625 069/300 Write block 2308 len 12288 data: 3 3 3
10626 070/300 Read block 2820 len 28672
10627 071/300 Read block 558 len 20480
10628 072/300 Read block 952 len 28672
10629 073/300 Read block 698 len 20480
10630 074/300 Write block 2659 len 36864 data: 5 3 3 2 4 4 4 3 2
10631 075/300 Write block 2633 len 4096 data: 3
10632 076/300 Read block 724 len 32768
10633 077/300 Read block 2563 len 24576
10634 078/300 Flush
10635 079/300 Read block 1226 len 20480
10636 080/300 Flush
10637 081/300 Read block 497 len 16384
10638 082/300 Read block 1501 len 8192
10639 083/300 Read block 1292 len 8192
10640 084/300 Flush
10641 085/300 Write block 259 len 20480 data: 3 3 3 3 3
10642 086/300 Read block 1978 len 16384
10643 087/300 Write block 4223 len 12288 data: 2 3 3
10644 088/300 Write block 749 len 16384 data: 3 2 2 2
10645 089/300 Read block 3709 len 16384
10646 090/300 Read block 3432 len 40960
10647 091/300 Read block 2024 len 4096
10648 092/300 Write block 3292 len 32768 data: 4 5 5 5 5 3 3 2
10649 093/300 Write block 1451 len 36864 data: 4 4 4 4 3 5 5 6 4
10650 094/300 Read block 2056 len 4096
10651 095/300 Read block 4607 len 8192
10652 096/300 Write block 2307 len 36864 data: 3 4 4 4 3 3 2 2 2
10653 097/300 Read block 3846 len 20480
10654 098/300 Write block 1049 len 8192 data: 3 3
10655 099/300 Write block 4700 len 16384 data: 3 3 2 2
10656 100/300 Read block 3013 len 40960
10657 101/300 Write block 1849 len 4096 data: 4
10658 102/300 Write block 3405 len 28672 data: 4 3 3 4 4 3 2
10659 103/300 Flush
10660 104/300 Flush
10661 105/300 Read block 902 len 24576
10662 106/300 Write block 4497 len 32768 data: 2 3 3 3 2 3 3 3
10663 107/300 Read block 4803 len 36864
10664 108/300 Read block 779 len 36864
10665 109/300 Write block 2538 len 8192 data: 3 3
10666 110/300 Flush
10667 111/300 Read block 2776 len 24576
10668 112/300 Read block 1568 len 8192
10669 113/300 Read block 1355 len 16384
10670 114/300 Write block 3328 len 4096 data: 3
10671 115/300 Read block 2086 len 16384
10672 116/300 Flush
10673 117/300 Write block 3965 len 28672 data: 5 4 4 4 3 3 3
10674 118/300 Read block 353 len 4096
10675 119/300 Write block 2171 len 32768 data: 4 3 4 4 5 5 5 5
10676 120/300 Write block 819 len 4096 data: 3
10677 121/300 Read block 851 len 40960
10678 122/300 Read block 968 len 8192
10679 123/300 Flush
10680 124/300 Write block 1965 len 36864 data: 2 3 3 3 3 3 2 2 2
10681 125/300 Write block 1553 len 28672 data: 3 4 4 5 6 6 6
10682 126/300 Read block 4964 len 36864
10683 127/300 Write block 320 len 40960 data: 5 5 3 3 3 3 3 3 3 3
10684 128/300 Write block 2298 len 12288 data: 2 4 4
10685 129/300 Write block 3330 len 12288 data: 4 4 5
10686 130/300 Write block 1890 len 4096 data: 4
10687 131/300 Flush
10688 132/300 Write block 3479 len 32768 data: 4 4 4 4 3 2 2 2
10689 133/300 Read block 3874 len 20480
10690 134/300 Write block 124 len 4096 data: 5
10691 135/300 Flush
10692 136/300 Read block 4441 len 32768
10693 137/300 Write block 3768 len 36864 data: 5 4 4 4 4 6 6 6 6
10694 138/300 Read block 3736 len 24576
10695 139/300 Read block 2633 len 28672
10696 140/300 Read block 664 len 32768
10697 141/300 Write block 2498 len 20480 data: 3 3 2 2 2
10698 142/300 Read block 642 len 40960
10699 143/300 Read block 65 len 40960
10700 144/300 Read block 841 len 8192
10701 145/300 Write block 494 len 24576 data: 3 3 3 4 5 5
10702 146/300 Write block 2353 len 4096 data: 5
10703 147/300 Read block 3097 len 28672
10704 148/300 Flush
10705 149/300 Write block 1019 len 16384 data: 5 5 4 2
10706 150/300 Read block 3588 len 16384
10707 151/300 Write block 2604 len 16384 data: 3 3 3 3
10708 152/300 Write block 3817 len 12288 data: 2 2 2
10709 153/300 Write block 1515 len 12288 data: 2 2 2
10710 154/300 Read block 3051 len 36864
10711 155/300 Flush
10712 156/300 Write block 1197 len 40960 data: 3 3 3 2 3 3 4 4 4 4
10713 157/300 Write block 48 len 20480 data: 4 4 4 3 3
10714 158/300 Read block 1060 len 8192
10715 159/300 Read block 2481 len 40960
10716 160/300 Read block 3636 len 24576
10717 161/300 Flush
10718 162/300 Write block 1216 len 20480 data: 3 3 3 4 3
10719 163/300 Flush
10720 164/300 Write block 4680 len 12288 data: 3 4 3
10721 165/300 Read block 3802 len 24576
10722 166/300 Read block 4928 len 24576
10723 167/300 Read block 3168 len 24576
10724 168/300 Read block 3035 len 32768
10725 169/300 Read block 3232 len 40960
10726 170/300 Read block 100 len 12288
10727 171/300 Read block 2772 len 16384
10728 172/300 Write block 1068 len 4096 data: 3
10729 173/300 Write block 2918 len 12288 data: 2 2 2
10730 174/300 Read block 4057 len 20480
10731 175/300 Read block 1329 len 36864
10732 176/300 Read block 1473 len 12288
10733 177/300 Read block 1489 len 32768
10734 178/300 Write block 3222 len 36864 data: 5 5 5 4 2 2 2 2 3
10735 179/300 Write block 4101 len 28672 data: 3 3 4 3 3 3 4
10736 180/300 Write block 3829 len 20480 data: 4 4 3 3 3
10737 181/300 Read block 657 len 28672
10738 182/300 Read block 560 len 40960
10739 183/300 Write block 1454 len 32768 data: 5 4 6 6 7 5 4 3
10740 184/300 Read block 3439 len 36864
10741 185/300 Write block 2388 len 20480 data: 4 4 5 6 4
10742 186/300 Write block 602 len 8192 data: 4 5
10743 187/300 Read block 3605 len 4096
10744 188/300 Read block 729 len 36864
10745 189/300 Read block 3420 len 28672
10746 190/300 Read block 4960 len 40960
10747 191/300 Read block 4973 len 32768
10748 192/300 Write block 4080 len 16384 data: 2 2 2 2
10749 193/300 Read block 3707 len 36864
10750 194/300 Read block 1546 len 8192
10751 195/300 Write block 4730 len 32768 data: 4 4 4 5 4 5 5 5
10752 196/300 Write block 2586 len 32768 data: 2 2 2 2 2 2 2 2
10753 197/300 Flush
10754 198/300 Write block 4979 len 4096 data: 2
10755 199/300 Flush
10756 200/300 Read block 1208 len 24576
10757 201/300 Flush
10758 202/300 Flush
10759 203/300 Write block 4829 len 40960 data: 4 4 3 3 3 2 2 2 2 3
10760 204/300 Write block 2793 len 8192 data: 4 5
10761 205/300 Read block 4295 len 12288
10762 206/300 Write block 578 len 24576 data: 4 4 4 4 4 5
10763 207/300 Read block 4556 len 40960
10764 208/300 Read block 1952 len 32768
10765 209/300 Read block 2018 len 36864
10766 210/300 Read block 2880 len 20480
10767 211/300 Read block 3240 len 36864
10768 212/300 Write block 228 len 16384 data: 3 3 2 2
10769 213/300 Read block 4840 len 12288
10770 214/300 Read block 2021 len 24576
10771 215/300 Write block 1614 len 20480 data: 2 2 3 3 3
10772 216/300 Read block 2641 len 12288
10773 217/300 Read block 4979 len 32768
10774 218/300 Write block 901 len 16384 data: 2 2 2 3
10775 219/300 Write block 4380 len 32768 data: 3 3 3 4 3 3 3 3
10776 220/300 Read block 3886 len 20480
10777 221/300 Flush
10778 222/300 Write block 860 len 20480 data: 4 5 5 4 4
10779 223/300 Read block 2932 len 36864
10780 224/300 Read block 2991 len 40960
10781 225/300 Read block 3604 len 12288
10782 226/300 Write block 170 len 40960 data: 5 5 3 3 3 3 4 4 3 4
10783 227/300 Read block 2232 len 8192
10784 228/300 Read block 4174 len 40960
10785 229/300 Read block 2957 len 16384
10786 230/300 Write block 2642 len 40960 data: 5 4 4 3 2 2 2 3 3 5
10787 231/300 Write block 1404 len 24576 data: 3 5 5 5 4 3
10788 232/300 Read block 2944 len 8192
10789 233/300 Write block 1193 len 32768 data: 3 3 3 5 4 4 4 3
10790 234/300 Read block 344 len 20480
10791 235/300 Read block 861 len 36864
10792 236/300 Read block 4946 len 16384
10793 237/300 Read block 2624 len 4096
10794 238/300 Write block 1962 len 32768 data: 2 2 2 3 4 4 4 4
10795 239/300 Write block 776 len 28672 data: 3 3 3 2 2 3 3
10796 240/300 Read block 1920 len 16384
10797 241/300 Write block 2846 len 4096 data: 5
10798 242/300 Flush
10799 243/300 Read block 851 len 36864
10800 244/300 Write block 1903 len 20480 data: 2 3 3 3 3
10801 245/300 Read block 1846 len 24576
10802 246/300 Read block 959 len 36864
10803 247/300 Write block 2669 len 28672 data: 2 2 2 3 4 4 4
10804 248/300 Write block 4718 len 4096 data: 5
10805 249/300 Read block 3058 len 32768
10806 250/300 Read block 453 len 24576
10807 251/300 Flush
10808 252/300 Read block 3421 len 8192
10809 253/300 Read block 2462 len 36864
10810 254/300 Write block 3140 len 4096 data: 8
10811 255/300 Write block 683 len 16384 data: 3 3 2 2
10812 256/300 Write block 3616 len 36864 data: 3 3 3 3 3 4 4 4 4
10813 257/300 Write block 1312 len 40960 data: 3 2 4 4 4 4 4 4 4 4
10814 258/300 Read block 4246 len 24576
10815 259/300 Write block 4072 len 8192 data: 3 3
10816 260/300 Read block 3955 len 8192
10817 261/300 Write block 1548 len 16384 data: 3 3 3 4
10818 262/300 Write block 626 len 28672 data: 2 2 2 2 3 2 2
10819 263/300 Write block 3915 len 36864 data: 2 2 3 4 4 3 4 4 3
10820 264/300 Write block 870 len 32768 data: 5 6 5 6 4 3 3 3
10821 265/300 Read block 3583 len 36864
10822 266/300 Write block 4475 len 16384 data: 4 4 4 4
10823 267/300 Write block 3425 len 12288 data: 4 5 4
10824 268/300 Write block 2862 len 36864 data: 4 4 4 4 4 3 2 2 3
10825 269/300 Read block 666 len 8192
10826 270/300 Read block 4002 len 16384
10827 271/300 Read block 696 len 4096
10828 272/300 Write block 2346 len 28672 data: 4 4 4 3 4 4 4
10829 273/300 Read block 1198 len 24576
10830 274/300 Write block 1106 len 28672 data: 4 3 2 2 3 4 3
10831 275/300 Read block 3564 len 40960
10832 276/300 Write block 183 len 4096 data: 4
10833 277/300 Read block 3075 len 8192
10834 278/300 Read block 170 len 36864
10835 279/300 Read block 3410 len 40960
10836 280/300 Read block 3250 len 32768
10837 281/300 Read block 1619 len 28672
10838 282/300 Write block 4923 len 12288 data: 5 4 4
10839 283/300 Write block 3426 len 36864 data: 6 5 4 4 4 4 3 3 3
10840 284/300 Flush
10841 285/300 Flush
10842 286/300 Write block 2011 len 12288 data: 4 3 4
10843 287/300 Write block 1627 len 4096 data: 6
10844 288/300 Read block 760 len 20480
10845 289/300 Write block 704 len 20480 data: 2 3 3 3 4
10846 290/300 Write block 2050 len 28672 data: 3 3 2 2 2 2 2
10847 291/300 Flush
10848 292/300 Write block 3171 len 36864 data: 3 4 4 3 3 5 5 5 5
10849 293/300 Flush
10850 294/300 Read block 2133 len 16384
10851 295/300 Read block 4974 len 8192
10852 296/300 Write block 1364 len 32768 data: 5 5 5 5 3 3 4 3
10853 297/300 Read block 3596 len 20480
10854 298/300 Read block 3302 len 24576
10855 299/300 Flush
10856 300/300 Write block 2751 len 24576 data: 5 4 4 3 3 3
10857 301/300 Write block 2281 len 8192 data: 2 2
10858 Main task has work to do, go find it
10859 got dsc Start(2)
10860 start 2
10861 Jan 27 19:38:39.615 INFO request completed, latency_us: 185, response_code: 204, uri: /start/cid/2, method: POST, req_id: 00eee15a-2609-4943-be28-6e70ad9a4454, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
10862 Replay: started 2, returned:Ok(())
10863 ----------------------------------------------------------------
10864 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
10865 States: Active Active Offline
10866 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
10867 4017 Acked Write 3 Done Done Sent false
10868 4018 Acked Write 7 Done Done Sent false
10869 4019 Acked Read 4 Done Done Sent false
10870 4020 Acked Write 10 Done Done Sent false
10871 4021 Acked Read 2 Done Done Sent false
10872 4022 Acked Write 7 Done Done Sent false
10873 4023 Acked Read 9 Done Done Sent false
10874 4024 Acked Read 7 Done Done Sent false
10875 4025 Acked Read 8 Done Done Sent false
10876 4026 Acked Write 5 Done Done Sent false
10877 4027 Acked Flush 0 Done Done Sent false
10878 4028 Acked Read 5 Done Done Sent false
10879 4029 Acked Write 4 Done Done Sent false
10880 4030 Acked Read 2 Done Done Sent false
10881 4031 Acked Flush 0 Done Done Sent false
10882 4032 Acked Read 1 Done Done Sent false
10883 4033 Acked Write 4 Done Done Sent false
10884 4034 Acked Read 10 Done Done Sent false
10885 4035 Acked Read 4 Done Done Sent false
10886 4036 Acked Read 3 Done Done Sent false
10887 4037 Acked Write 6 Done Done Sent false
10888 4038 Acked Write 1 Done Done Sent false
10889 4039 Acked Write 8 Done Done Sent false
10890 4040 Acked Read 5 Done Done Sent false
10891 4041 Acked Read 3 Done Done Sent false
10892 4042 Acked Write 1 Done Done Sent false
10893 4043 Acked Read 9 Done Done Sent false
10894 4044 Acked Write 8 Done Done Sent false
10895 4045 Acked Read 10 Done Done Sent false
10896 4046 Acked Read 8 Done Done Sent false
10897 4047 Acked Read 5 Done Done Sent false
10898 4048 Acked Read 3 Done Done Sent false
10899 4049 Acked Read 1 Done Done Sent false
10900 4050 Acked Write 7 Done Done Sent false
10901 4051 Acked Flush 0 Done Done Sent false
10902 4052 Acked Read 1 Done Done Sent false
10903 4053 Acked Write 8 Done Done Sent false
10904 4054 Acked Write 4 Done Done Sent false
10905 4055 Acked Read 1 Done Done Sent false
10906 4056 Acked Read 6 Done Done Sent false
10907 4057 Acked Write 7 Done Done Sent false
10908 4058 Acked Read 4 Done Done Sent false
10909 4059 Acked Read 4 Done Done Sent false
10910 4060 Acked Read 4 Done Done Sent false
10911 4061 Acked Write 10 Done Done Sent false
10912 4062 Acked Write 1 Done Done Sent false
10913 4063 Acked Write 9 Done Done Sent false
10914 4064 Acked Read 10 Done Done Sent false
10915 4065 Acked Read 10 Done Done Sent false
10916 4066 Acked Read 1 Done Done Sent false
10917 4067 Acked Read 10 Done Done Sent false
10918 4068 Acked Write 7 Done Done Sent false
10919 4069 Acked Read 10 Done Done Sent false
10920 4070 Acked Read 9 Done Done Sent false
10921 4071 Acked Read 3 Done Done Sent false
10922 4072 Acked Read 9 Done Done Sent false
10923 4073 Acked Read 3 Done Done Sent false
10924 4074 Acked Read 8 Done Done Sent false
10925 4075 Acked Write 6 Done Done Sent false
10926 4076 Acked Read 3 Done Done Sent false
10927 4077 Acked Write 5 Done Done Sent false
10928 4078 Acked Read 10 Done Done Sent false
10929 4079 Acked Write 4 Done Done Sent false
10930 4080 Acked Write 7 Done Done Sent false
10931 4081 Acked Read 8 Done Done Sent false
10932 4082 Acked Read 6 Done Done Sent false
10933 4083 Acked Write 1 Done Done Sent false
10934 4084 Acked Write 8 Done Done Sent false
10935 4085 Acked Write 3 Done Done Sent false
10936 4086 Acked Read 7 Done Done Sent false
10937 4087 Acked Read 5 Done Done Sent false
10938 4088 Acked Read 7 Done Done Sent false
10939 4089 Acked Read 5 Done Done Sent false
10940 4090 Acked Write 9 Done Done Sent false
10941 4091 Acked Write 1 Done Done Sent false
10942 4092 Acked Read 8 Done Done Sent false
10943 4093 Acked Read 6 Done Done Sent false
10944 4094 Acked Flush 0 Done Done Sent false
10945 4095 Acked Read 5 Done Done Sent false
10946 4096 Acked Flush 0 Done Done Sent false
10947 4097 Acked Read 4 Done Done Sent false
10948 4098 Acked Read 2 Done Done Sent false
10949 4099 Acked Read 2 Done Done Sent false
10950 4100 Acked Flush 0 Done Done Sent false
10951 4101 Acked Write 5 Done Done Sent false
10952 4102 Acked Read 4 Done Done Sent false
10953 4103 Acked Write 3 Done Done Sent false
10954 4104 Acked Write 4 Done Done Sent false
10955 4105 Acked Read 4 Done Done Sent false
10956 4106 Acked Read 10 Done Done Sent false
10957 4107 Acked Read 1 Done Done Sent false
10958 4108 Acked Write 8 Done Done Sent false
10959 4109 Acked Write 9 Done Done Sent false
10960 4110 Acked Read 1 Done Done Sent false
10961 4111 Acked Read 2 Done Done Sent false
10962 4112 Acked Write 9 Done Done Sent false
10963 4113 Acked Read 5 Done Done Sent false
10964 4114 Acked Write 2 Done Done Sent false
10965 4115 Acked Write 4 Done Done Sent false
10966 4116 Acked Read 10 Done Done Sent false
10967 4117 Acked Write 1 Done Done Sent false
10968 4118 Acked Write 7 Done Done Sent false
10969 4119 Acked Flush 0 Done Done Sent false
10970 4120 Acked Flush 0 Done Done Sent false
10971 4121 Acked Read 6 Done Done Sent false
10972 4122 Acked Write 8 Done Done Sent false
10973 4123 Acked Read 9 Done Done Sent false
10974 4124 Acked Read 9 Done Done Sent false
10975 4125 Acked Write 2 Done Done Sent false
10976 4126 Acked Flush 0 Done Done Sent false
10977 4127 Acked Read 6 Done Done Sent false
10978 4128 Acked Read 2 Done Done Sent false
10979 4129 Acked Read 4 Done Done Sent false
10980 4130 Acked Write 1 Done Done Sent false
10981 4131 Acked Read 4 Done Done Sent false
10982 4132 Acked Flush 0 Done Done Sent false
10983 4133 Acked Write 7 Done Done Sent false
10984 4134 Acked Read 1 Done Done Sent false
10985 4135 Acked Write 8 Done Done Sent false
10986 4136 Acked Write 1 Done Done Sent false
10987 4137 Acked Read 10 Done Done Sent false
10988 4138 Acked Read 2 Done Done Sent false
10989 4139 Acked Flush 0 Done Done Sent false
10990 4140 Acked Write 9 Done Done Sent false
10991 4141 Acked Write 7 Done Done Sent false
10992 4142 Acked Read 9 Done Done Sent false
10993 4143 Acked Write 10 Done Done Sent false
10994 4144 Acked Write 3 Done Done Sent false
10995 4145 Acked Write 3 Done Done Sent false
10996 4146 Acked Write 1 Done Done Sent false
10997 4147 Acked Flush 0 Done Done Sent false
10998 4148 Acked Write 8 Done Done Sent false
10999 4149 Acked Read 5 Done Done Sent false
11000 4150 Acked Write 1 Done Done Sent false
11001 4151 Acked Flush 0 Done Done Sent false
11002 4152 Acked Read 8 Done Done Sent false
11003 4153 Acked Write 9 Done Done Sent false
11004 4154 Acked Read 6 Done Done Sent false
11005 4155 Acked Read 7 Done Done Sent false
11006 4156 Acked Read 8 Done Done Sent false
11007 4157 Acked Write 5 Done Done Sent false
11008 4158 Acked Read 10 Done Done Sent false
11009 4159 Acked Read 10 Done Done Sent false
11010 4160 Acked Read 2 Done Done Sent false
11011 4161 Acked Write 6 Done Done Sent false
11012 4162 Acked Write 1 Done Done Sent false
11013 4163 Acked Read 7 Done Done Sent false
11014 4164 Acked Flush 0 Done Done Sent false
11015 4165 Acked Write 4 Done Done Sent false
11016 4166 Acked Read 4 Done Done Sent false
11017 4167 Acked Write 4 Done Done Sent false
11018 4168 Acked Write 3 Done Done Sent false
11019 4169 Acked Write 3 Done Done Sent false
11020 4170 Acked Read 9 Done Done Sent false
11021 4171 Acked Flush 0 Done Done Sent false
11022 4172 Acked Write 10 Done Done Sent false
11023 4173 Acked Write 5 Done Done Sent false
11024 4174 Acked Read 2 Done Done Sent false
11025 4175 Acked Read 10 Done Done Sent false
11026 4176 Acked Read 6 Done Done Sent false
11027 4177 Acked Flush 0 Done Done Sent false
11028 4178 Acked Write 5 Done Done Sent false
11029 4179 Acked Flush 0 Done Done Sent false
11030 4180 Acked Write 3 Done Done Sent false
11031 4181 Acked Read 6 Done Done Sent false
11032 4182 Acked Read 6 Done Done Sent false
11033 4183 Acked Read 6 Done Done Sent false
11034 4184 Acked Read 8 Done Done Sent false
11035 4185 Acked Read 10 Done Done Sent false
11036 4186 Acked Read 3 Done Done Sent false
11037 4187 Acked Read 4 Done Done Sent false
11038 4188 Acked Write 1 Done Done Sent false
11039 4189 Acked Write 3 Done Done Sent false
11040 4190 Acked Read 5 Done Done Sent false
11041 4191 Acked Read 9 Done Done Sent false
11042 4192 Acked Read 3 Done Done Sent false
11043 4193 Acked Read 8 Done Done Sent false
11044 4194 Acked Write 9 Done Done Sent false
11045 4195 Acked Write 7 Done Done Sent false
11046 4196 Acked Write 5 Done Done Sent false
11047 4197 Acked Read 7 Done Done Sent false
11048 4198 Acked Read 10 Done Done Sent false
11049 4199 Acked Write 8 Done Done Sent false
11050 4200 Acked Read 9 Done Done Sent false
11051 4201 Acked Write 5 Done Done Sent false
11052 4202 Acked Write 2 Done Done Sent false
11053 4203 Acked Read 1 Done Done Sent false
11054 4204 Acked Read 9 Done Done Sent false
11055 4205 Acked Read 7 Done Done Sent false
11056 4206 Acked Read 10 Done Done Sent false
11057 4207 Acked Read 8 Done Done Sent false
11058 4208 Acked Write 4 Done Done Sent false
11059 4209 Acked Read 9 Done Done Sent false
11060 4210 Acked Read 2 Done Done Sent false
11061 4211 Acked Write 8 Done Done Sent false
11062 4212 Acked Write 8 Done Done Sent false
11063 4213 Acked Flush 0 Done Done Sent false
11064 4214 Acked Write 1 Done Done Sent false
11065 4215 Acked Flush 0 Done Done Sent false
11066 4216 Acked Read 6 Done Done Sent false
11067 4217 Acked Flush 0 Done Done Sent false
11068 4218 Acked Flush 0 Done Done Sent false
11069 4219 Acked Write 10 Done Done Sent false
11070 4220 Acked Write 2 Done Done Sent false
11071 4221 Acked Read 3 Done Done Sent false
11072 4222 Acked Write 6 Done Done Sent false
11073 4223 Acked Read 10 Done Done Sent false
11074 4224 Acked Read 8 Done Done Sent false
11075 4225 Acked Read 9 Done Done Sent false
11076 4226 Acked Read 5 Done Done Sent false
11077 4227 Acked Read 9 Done Done Sent false
11078 4228 Acked Write 4 Done Done Sent false
11079 4229 Acked Read 3 Done Done Sent false
11080 4230 Acked Read 6 Done Done Sent false
11081 4231 Acked Write 5 Done Done Sent false
11082 4232 Acked Read 3 Done Done Sent false
11083 4233 Acked Read 8 Done Done Sent false
11084 4234 Acked Write 4 Done Done Sent false
11085 4235 Acked Write 8 Done Done Sent false
11086 4236 Acked Read 5 Done Done Sent false
11087 4237 Acked Flush 0 Done Done Sent false
11088 4238 Acked Write 5 Done Done Sent false
11089 4239 Acked Read 9 Done Done Sent false
11090 4240 Acked Read 10 Done Done Sent false
11091 4241 Acked Read 3 Done Done Sent false
11092 4242 Acked Write 10 Done Done Sent false
11093 4243 Acked Read 2 Done Done Sent false
11094 4244 Acked Read 10 Done Done Sent false
11095 4245 Acked Read 4 Done Done Sent false
11096 4246 Acked Write 10 Done Done Sent false
11097 4247 Acked Write 6 Done Done Sent false
11098 4248 Acked Read 2 Done Done Sent false
11099 4249 Acked Write 8 Done Done Sent false
11100 4250 Acked Read 5 Done Done Sent false
11101 4251 Acked Read 9 Done Done Sent false
11102 4252 Acked Read 4 Done Done Sent false
11103 4253 Acked Read 1 Done Done Sent false
11104 4254 Acked Write 8 Done Done Sent false
11105 4255 Acked Write 7 Done Done Sent false
11106 4256 Acked Read 4 Done Done Sent false
11107 4257 Acked Write 1 Done Done Sent false
11108 4258 Acked Flush 0 Done Done Sent false
11109 4259 Acked Read 9 Done Done Sent false
11110 4260 Acked Write 5 Done Done Sent false
11111 4261 Acked Read 6 Done Done Sent false
11112 4262 Acked Read 9 Done Done Sent false
11113 4263 Acked Write 7 Done Done Sent false
11114 4264 Acked Write 1 Done Done Sent false
11115 4265 Acked Read 8 Done Done Sent false
11116 4266 Acked Read 6 Done Done Sent false
11117 4267 Acked Flush 0 Done Done Sent false
11118 4268 Acked Read 2 Done Done Sent false
11119 4269 Acked Read 9 Done Done Sent false
11120 4270 Acked Write 1 Done Done Sent false
11121 4271 Acked Write 4 Done Done Sent false
11122 4272 Acked Write 9 Done Done Sent false
11123 4273 Acked Write 10 Done Done Sent false
11124 4274 Acked Read 6 Done Done Sent false
11125 4275 Acked Write 2 Done Done Sent false
11126 4276 Acked Read 2 Done Done Sent false
11127 4277 Acked Write 4 Done Done Sent false
11128 4278 Acked Write 7 Done Done Sent false
11129 4279 Acked Write 9 Done Done Sent false
11130 4280 Acked Write 8 Done Done Sent false
11131 4281 Acked Read 9 Done Done Sent false
11132 4282 Acked Write 4 Done Done Sent false
11133 4283 Acked Write 3 Done Done Sent false
11134 4284 Acked Write 9 Done Done Sent false
11135 4285 Acked Read 2 Done Done Sent false
11136 4286 Acked Read 4 Done Done Sent false
11137 4287 Acked Read 1 Done Done Sent false
11138 4288 Acked Write 7 Done Done Sent false
11139 4289 Acked Read 6 Done Done Sent false
11140 4290 Acked Write 7 Done Done Sent false
11141 4291 Acked Read 10 Done Done Sent false
11142 4292 Acked Write 1 Done Done Sent false
11143 4293 Acked Read 2 Done Done Sent false
11144 4294 Acked Read 9 Done Done Sent false
11145 4295 Acked Read 10 Done Done Sent false
11146 4296 Acked Read 8 Done Done Sent false
11147 4297 Acked Read 7 Done Done Sent false
11148 4298 Acked Write 3 Done Done Sent false
11149 4299 Acked Write 9 Done Done Sent false
11150 4300 Acked Flush 0 Done Done Sent false
11151 4301 Acked Flush 0 Done Done Sent false
11152 4302 Acked Write 3 Done Done Sent false
11153 4303 Acked Write 1 Done Done Sent false
11154 4304 Acked Read 5 Done Done Sent false
11155 4305 Acked Write 5 Done Done Sent false
11156 4306 Acked Write 7 Done Done Sent false
11157 4307 Acked Flush 0 Done Done Sent false
11158 4308 Acked Write 9 Done Done Sent false
11159 4309 Acked Flush 0 Done Done Sent false
11160 4310 Acked Read 4 Done Done Sent false
11161 4311 Acked Read 2 Done Done Sent false
11162 4312 Acked Write 8 Done Done Sent false
11163 4313 Acked Read 5 Done Done Sent false
11164 4314 Acked Read 6 Done Done Sent false
11165 4315 Acked Flush 0 Done Done Sent false
11166 4316 Acked Write 6 Done Done Sent false
11167 4317 Acked Write 2 Done Done Sent false
11168 STATES DS:0 DS:1 DS:2 TOTAL
11169 Sent 0 0 301 301
11170 Done 301 301 0 602
11171 Skipped 0 0 0 0
11172 Error 0 0 0 0
11173 Last Flush: 4315 4315 4016
11174 Downstairs last five completed: 4016 4015 4014 4013 4012
11175 Upstairs last five completed: 4317 4316 4315 4314 4313
11176 CLIENT: Up:0 ds:301 act:2
11177 ----------------------------------------------------------------
11178 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11179 States: Active Active Offline
11180 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11181 4017 Acked Write 3 Done Done Sent false
11182 4018 Acked Write 7 Done Done Sent false
11183 4019 Acked Read 4 Done Done Sent false
11184 4020 Acked Write 10 Done Done Sent false
11185 4021 Acked Read 2 Done Done Sent false
11186 4022 Acked Write 7 Done Done Sent false
11187 4023 Acked Read 9 Done Done Sent false
11188 4024 Acked Read 7 Done Done Sent false
11189 4025 Acked Read 8 Done Done Sent false
11190 4026 Acked Write 5 Done Done Sent false
11191 4027 Acked Flush 0 Done Done Sent false
11192 4028 Acked Read 5 Done Done Sent false
11193 4029 Acked Write 4 Done Done Sent false
11194 4030 Acked Read 2 Done Done Sent false
11195 4031 Acked Flush 0 Done Done Sent false
11196 4032 Acked Read 1 Done Done Sent false
11197 4033 Acked Write 4 Done Done Sent false
11198 4034 Acked Read 10 Done Done Sent false
11199 4035 Acked Read 4 Done Done Sent false
11200 4036 Acked Read 3 Done Done Sent false
11201 4037 Acked Write 6 Done Done Sent false
11202 4038 Acked Write 1 Done Done Sent false
11203 4039 Acked Write 8 Done Done Sent false
11204 4040 Acked Read 5 Done Done Sent false
11205 4041 Acked Read 3 Done Done Sent false
11206 4042 Acked Write 1 Done Done Sent false
11207 4043 Acked Read 9 Done Done Sent false
11208 4044 Acked Write 8 Done Done Sent false
11209 4045 Acked Read 10 Done Done Sent false
11210 4046 Acked Read 8 Done Done Sent false
11211 4047 Acked Read 5 Done Done Sent false
11212 4048 Acked Read 3 Done Done Sent false
11213 4049 Acked Read 1 Done Done Sent false
11214 4050 Acked Write 7 Done Done Sent false
11215 4051 Acked Flush 0 Done Done Sent false
11216 4052 Acked Read 1 Done Done Sent false
11217 4053 Acked Write 8 Done Done Sent false
11218 4054 Acked Write 4 Done Done Sent false
11219 4055 Acked Read 1 Done Done Sent false
11220 4056 Acked Read 6 Done Done Sent false
11221 4057 Acked Write 7 Done Done Sent false
11222 4058 Acked Read 4 Done Done Sent false
11223 4059 Acked Read 4 Done Done Sent false
11224 4060 Acked Read 4 Done Done Sent false
11225 4061 Acked Write 10 Done Done Sent false
11226 4062 Acked Write 1 Done Done Sent false
11227 4063 Acked Write 9 Done Done Sent false
11228 4064 Acked Read 10 Done Done Sent false
11229 4065 Acked Read 10 Done Done Sent false
11230 4066 Acked Read 1 Done Done Sent false
11231 4067 Acked Read 10 Done Done Sent false
11232 4068 Acked Write 7 Done Done Sent false
11233 4069 Acked Read 10 Done Done Sent false
11234 4070 Acked Read 9 Done Done Sent false
11235 4071 Acked Read 3 Done Done Sent false
11236 4072 Acked Read 9 Done Done Sent false
11237 4073 Acked Read 3 Done Done Sent false
11238 4074 Acked Read 8 Done Done Sent false
11239 4075 Acked Write 6 Done Done Sent false
11240 4076 Acked Read 3 Done Done Sent false
11241 4077 Acked Write 5 Done Done Sent false
11242 4078 Acked Read 10 Done Done Sent false
11243 4079 Acked Write 4 Done Done Sent false
11244 4080 Acked Write 7 Done Done Sent false
11245 4081 Acked Read 8 Done Done Sent false
11246 4082 Acked Read 6 Done Done Sent false
11247 4083 Acked Write 1 Done Done Sent false
11248 4084 Acked Write 8 Done Done Sent false
11249 4085 Acked Write 3 Done Done Sent false
11250 4086 Acked Read 7 Done Done Sent false
11251 4087 Acked Read 5 Done Done Sent false
11252 4088 Acked Read 7 Done Done Sent false
11253 4089 Acked Read 5 Done Done Sent false
11254 4090 Acked Write 9 Done Done Sent false
11255 4091 Acked Write 1 Done Done Sent false
11256 4092 Acked Read 8 Done Done Sent false
11257 4093 Acked Read 6 Done Done Sent false
11258 4094 Acked Flush 0 Done Done Sent false
11259 4095 Acked Read 5 Done Done Sent false
11260 4096 Acked Flush 0 Done Done Sent false
11261 4097 Acked Read 4 Done Done Sent false
11262 4098 Acked Read 2 Done Done Sent false
11263 4099 Acked Read 2 Done Done Sent false
11264 4100 Acked Flush 0 Done Done Sent false
11265 4101 Acked Write 5 Done Done Sent false
11266 4102 Acked Read 4 Done Done Sent false
11267 4103 Acked Write 3 Done Done Sent false
11268 4104 Acked Write 4 Done Done Sent false
11269 4105 Acked Read 4 Done Done Sent false
11270 4106 Acked Read 10 Done Done Sent false
11271 4107 Acked Read 1 Done Done Sent false
11272 4108 Acked Write 8 Done Done Sent false
11273 4109 Acked Write 9 Done Done Sent false
11274 4110 Acked Read 1 Done Done Sent false
11275 4111 Acked Read 2 Done Done Sent false
11276 4112 Acked Write 9 Done Done Sent false
11277 4113 Acked Read 5 Done Done Sent false
11278 4114 Acked Write 2 Done Done Sent false
11279 4115 Acked Write 4 Done Done Sent false
11280 4116 Acked Read 10 Done Done Sent false
11281 4117 Acked Write 1 Done Done Sent false
11282 4118 Acked Write 7 Done Done Sent false
11283 4119 Acked Flush 0 Done Done Sent false
11284 4120 Acked Flush 0 Done Done Sent false
11285 4121 Acked Read 6 Done Done Sent false
11286 4122 Acked Write 8 Done Done Sent false
11287 4123 Acked Read 9 Done Done Sent false
11288 4124 Acked Read 9 Done Done Sent false
11289 4125 Acked Write 2 Done Done Sent false
11290 4126 Acked Flush 0 Done Done Sent false
11291 4127 Acked Read 6 Done Done Sent false
11292 4128 Acked Read 2 Done Done Sent false
11293 4129 Acked Read 4 Done Done Sent false
11294 4130 Acked Write 1 Done Done Sent false
11295 4131 Acked Read 4 Done Done Sent false
11296 4132 Acked Flush 0 Done Done Sent false
11297 4133 Acked Write 7 Done Done Sent false
11298 4134 Acked Read 1 Done Done Sent false
11299 4135 Acked Write 8 Done Done Sent false
11300 4136 Acked Write 1 Done Done Sent false
11301 4137 Acked Read 10 Done Done Sent false
11302 4138 Acked Read 2 Done Done Sent false
11303 4139 Acked Flush 0 Done Done Sent false
11304 4140 Acked Write 9 Done Done Sent false
11305 4141 Acked Write 7 Done Done Sent false
11306 4142 Acked Read 9 Done Done Sent false
11307 4143 Acked Write 10 Done Done Sent false
11308 4144 Acked Write 3 Done Done Sent false
11309 4145 Acked Write 3 Done Done Sent false
11310 4146 Acked Write 1 Done Done Sent false
11311 4147 Acked Flush 0 Done Done Sent false
11312 4148 Acked Write 8 Done Done Sent false
11313 4149 Acked Read 5 Done Done Sent false
11314 4150 Acked Write 1 Done Done Sent false
11315 4151 Acked Flush 0 Done Done Sent false
11316 4152 Acked Read 8 Done Done Sent false
11317 4153 Acked Write 9 Done Done Sent false
11318 4154 Acked Read 6 Done Done Sent false
11319 4155 Acked Read 7 Done Done Sent false
11320 4156 Acked Read 8 Done Done Sent false
11321 4157 Acked Write 5 Done Done Sent false
11322 4158 Acked Read 10 Done Done Sent false
11323 4159 Acked Read 10 Done Done Sent false
11324 4160 Acked Read 2 Done Done Sent false
11325 4161 Acked Write 6 Done Done Sent false
11326 4162 Acked Write 1 Done Done Sent false
11327 4163 Acked Read 7 Done Done Sent false
11328 4164 Acked Flush 0 Done Done Sent false
11329 4165 Acked Write 4 Done Done Sent false
11330 4166 Acked Read 4 Done Done Sent false
11331 4167 Acked Write 4 Done Done Sent false
11332 4168 Acked Write 3 Done Done Sent false
11333 4169 Acked Write 3 Done Done Sent false
11334 4170 Acked Read 9 Done Done Sent false
11335 4171 Acked Flush 0 Done Done Sent false
11336 4172 Acked Write 10 Done Done Sent false
11337 4173 Acked Write 5 Done Done Sent false
11338 4174 Acked Read 2 Done Done Sent false
11339 4175 Acked Read 10 Done Done Sent false
11340 4176 Acked Read 6 Done Done Sent false
11341 4177 Acked Flush 0 Done Done Sent false
11342 4178 Acked Write 5 Done Done Sent false
11343 4179 Acked Flush 0 Done Done Sent false
11344 4180 Acked Write 3 Done Done Sent false
11345 4181 Acked Read 6 Done Done Sent false
11346 4182 Acked Read 6 Done Done Sent false
11347 4183 Acked Read 6 Done Done Sent false
11348 4184 Acked Read 8 Done Done Sent false
11349 4185 Acked Read 10 Done Done Sent false
11350 4186 Acked Read 3 Done Done Sent false
11351 4187 Acked Read 4 Done Done Sent false
11352 4188 Acked Write 1 Done Done Sent false
11353 4189 Acked Write 3 Done Done Sent false
11354 4190 Acked Read 5 Done Done Sent false
11355 4191 Acked Read 9 Done Done Sent false
11356 4192 Acked Read 3 Done Done Sent false
11357 4193 Acked Read 8 Done Done Sent false
11358 4194 Acked Write 9 Done Done Sent false
11359 4195 Acked Write 7 Done Done Sent false
11360 4196 Acked Write 5 Done Done Sent false
11361 4197 Acked Read 7 Done Done Sent false
11362 4198 Acked Read 10 Done Done Sent false
11363 4199 Acked Write 8 Done Done Sent false
11364 4200 Acked Read 9 Done Done Sent false
11365 4201 Acked Write 5 Done Done Sent false
11366 4202 Acked Write 2 Done Done Sent false
11367 4203 Acked Read 1 Done Done Sent false
11368 4204 Acked Read 9 Done Done Sent false
11369 4205 Acked Read 7 Done Done Sent false
11370 4206 Acked Read 10 Done Done Sent false
11371 4207 Acked Read 8 Done Done Sent false
11372 4208 Acked Write 4 Done Done Sent false
11373 4209 Acked Read 9 Done Done Sent false
11374 4210 Acked Read 2 Done Done Sent false
11375 4211 Acked Write 8 Done Done Sent false
11376 4212 Acked Write 8 Done Done Sent false
11377 4213 Acked Flush 0 Done Done Sent false
11378 4214 Acked Write 1 Done Done Sent false
11379 4215 Acked Flush 0 Done Done Sent false
11380 4216 Acked Read 6 Done Done Sent false
11381 4217 Acked Flush 0 Done Done Sent false
11382 4218 Acked Flush 0 Done Done Sent false
11383 4219 Acked Write 10 Done Done Sent false
11384 4220 Acked Write 2 Done Done Sent false
11385 4221 Acked Read 3 Done Done Sent false
11386 4222 Acked Write 6 Done Done Sent false
11387 4223 Acked Read 10 Done Done Sent false
11388 4224 Acked Read 8 Done Done Sent false
11389 4225 Acked Read 9 Done Done Sent false
11390 4226 Acked Read 5 Done Done Sent false
11391 4227 Acked Read 9 Done Done Sent false
11392 4228 Acked Write 4 Done Done Sent false
11393 4229 Acked Read 3 Done Done Sent false
11394 4230 Acked Read 6 Done Done Sent false
11395 4231 Acked Write 5 Done Done Sent false
11396 4232 Acked Read 3 Done Done Sent false
11397 4233 Acked Read 8 Done Done Sent false
11398 4234 Acked Write 4 Done Done Sent false
11399 4235 Acked Write 8 Done Done Sent false
11400 4236 Acked Read 5 Done Done Sent false
11401 4237 Acked Flush 0 Done Done Sent false
11402 4238 Acked Write 5 Done Done Sent false
11403 4239 Acked Read 9 Done Done Sent false
11404 4240 Acked Read 10 Done Done Sent false
11405 4241 Acked Read 3 Done Done Sent false
11406 4242 Acked Write 10 Done Done Sent false
11407 4243 Acked Read 2 Done Done Sent false
11408 4244 Acked Read 10 Done Done Sent false
11409 4245 Acked Read 4 Done Done Sent false
11410 4246 Acked Write 10 Done Done Sent false
11411 4247 Acked Write 6 Done Done Sent false
11412 4248 Acked Read 2 Done Done Sent false
11413 4249 Acked Write 8 Done Done Sent false
11414 4250 Acked Read 5 Done Done Sent false
11415 4251 Acked Read 9 Done Done Sent false
11416 4252 Acked Read 4 Done Done Sent false
11417 4253 Acked Read 1 Done Done Sent false
11418 4254 Acked Write 8 Done Done Sent false
11419 4255 Acked Write 7 Done Done Sent false
11420 4256 Acked Read 4 Done Done Sent false
11421 4257 Acked Write 1 Done Done Sent false
11422 4258 Acked Flush 0 Done Done Sent false
11423 4259 Acked Read 9 Done Done Sent false
11424 4260 Acked Write 5 Done Done Sent false
11425 4261 Acked Read 6 Done Done Sent false
11426 4262 Acked Read 9 Done Done Sent false
11427 4263 Acked Write 7 Done Done Sent false
11428 4264 Acked Write 1 Done Done Sent false
11429 4265 Acked Read 8 Done Done Sent false
11430 4266 Acked Read 6 Done Done Sent false
11431 4267 Acked Flush 0 Done Done Sent false
11432 4268 Acked Read 2 Done Done Sent false
11433 4269 Acked Read 9 Done Done Sent false
11434 4270 Acked Write 1 Done Done Sent false
11435 4271 Acked Write 4 Done Done Sent false
11436 4272 Acked Write 9 Done Done Sent false
11437 4273 Acked Write 10 Done Done Sent false
11438 4274 Acked Read 6 Done Done Sent false
11439 4275 Acked Write 2 Done Done Sent false
11440 4276 Acked Read 2 Done Done Sent false
11441 4277 Acked Write 4 Done Done Sent false
11442 4278 Acked Write 7 Done Done Sent false
11443 4279 Acked Write 9 Done Done Sent false
11444 4280 Acked Write 8 Done Done Sent false
11445 4281 Acked Read 9 Done Done Sent false
11446 4282 Acked Write 4 Done Done Sent false
11447 4283 Acked Write 3 Done Done Sent false
11448 4284 Acked Write 9 Done Done Sent false
11449 4285 Acked Read 2 Done Done Sent false
11450 4286 Acked Read 4 Done Done Sent false
11451 4287 Acked Read 1 Done Done Sent false
11452 4288 Acked Write 7 Done Done Sent false
11453 4289 Acked Read 6 Done Done Sent false
11454 4290 Acked Write 7 Done Done Sent false
11455 4291 Acked Read 10 Done Done Sent false
11456 4292 Acked Write 1 Done Done Sent false
11457 4293 Acked Read 2 Done Done Sent false
11458 4294 Acked Read 9 Done Done Sent false
11459 4295 Acked Read 10 Done Done Sent false
11460 4296 Acked Read 8 Done Done Sent false
11461 4297 Acked Read 7 Done Done Sent false
11462 4298 Acked Write 3 Done Done Sent false
11463 4299 Acked Write 9 Done Done Sent false
11464 4300 Acked Flush 0 Done Done Sent false
11465 4301 Acked Flush 0 Done Done Sent false
11466 4302 Acked Write 3 Done Done Sent false
11467 4303 Acked Write 1 Done Done Sent false
11468 4304 Acked Read 5 Done Done Sent false
11469 4305 Acked Write 5 Done Done Sent false
11470 4306 Acked Write 7 Done Done Sent false
11471 4307 Acked Flush 0 Done Done Sent false
11472 4308 Acked Write 9 Done Done Sent false
11473 4309 Acked Flush 0 Done Done Sent false
11474 4310 Acked Read 4 Done Done Sent false
11475 4311 Acked Read 2 Done Done Sent false
11476 4312 Acked Write 8 Done Done Sent false
11477 4313 Acked Read 5 Done Done Sent false
11478 4314 Acked Read 6 Done Done Sent false
11479 4315 Acked Flush 0 Done Done Sent false
11480 4316 Acked Write 6 Done Done Sent false
11481 4317 Acked Write 2 Done Done Sent false
11482 4318 Acked Flush 0 Done Done Sent false
11483 STATES DS:0 DS:1 DS:2 TOTAL
11484 Sent 0 0 302 302
11485 Done 302 302 0 604
11486 Skipped 0 0 0 0
11487 Error 0 0 0 0
11488 Last Flush: 4318 4318 4016
11489 Downstairs last five completed: 4016 4015 4014 4013 4012
11490 Upstairs last five completed: 4318 4317 4316 4315 4314
11491 CLIENT: Up:0 ds:302 act:2
114922025-01-27T19:38:45.371ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
114932025-01-27T19:38:45.371ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
114942025-01-27T19:38:45.371ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
114952025-01-27T19:38:45.371ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
114962025-01-27T19:38:45.372ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
11497 [8830] got start action
11498 [8830] I am going to restart
11499 Starting downstairs at port 8830
11500 Make output file at "/tmp/dsc/downstairs-8830.txt"
11501 [8830][2] reports Starting
11502 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1211)
11503 [8830][2] reports Running
11504 ----------------------------------------------------------------
11505 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11506 States: Active Active Offline
11507 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11508 4017 Acked Write 3 Done Done Sent false
11509 4018 Acked Write 7 Done Done Sent false
11510 4019 Acked Read 4 Done Done Sent false
11511 4020 Acked Write 10 Done Done Sent false
11512 4021 Acked Read 2 Done Done Sent false
11513 4022 Acked Write 7 Done Done Sent false
11514 4023 Acked Read 9 Done Done Sent false
11515 4024 Acked Read 7 Done Done Sent false
11516 4025 Acked Read 8 Done Done Sent false
11517 4026 Acked Write 5 Done Done Sent false
11518 4027 Acked Flush 0 Done Done Sent false
11519 4028 Acked Read 5 Done Done Sent false
11520 4029 Acked Write 4 Done Done Sent false
11521 4030 Acked Read 2 Done Done Sent false
11522 4031 Acked Flush 0 Done Done Sent false
11523 4032 Acked Read 1 Done Done Sent false
11524 4033 Acked Write 4 Done Done Sent false
11525 4034 Acked Read 10 Done Done Sent false
11526 4035 Acked Read 4 Done Done Sent false
11527 4036 Acked Read 3 Done Done Sent false
11528 4037 Acked Write 6 Done Done Sent false
11529 4038 Acked Write 1 Done Done Sent false
11530 4039 Acked Write 8 Done Done Sent false
11531 4040 Acked Read 5 Done Done Sent false
11532 4041 Acked Read 3 Done Done Sent false
11533 4042 Acked Write 1 Done Done Sent false
11534 4043 Acked Read 9 Done Done Sent false
11535 4044 Acked Write 8 Done Done Sent false
11536 4045 Acked Read 10 Done Done Sent false
11537 4046 Acked Read 8 Done Done Sent false
11538 4047 Acked Read 5 Done Done Sent false
11539 4048 Acked Read 3 Done Done Sent false
11540 4049 Acked Read 1 Done Done Sent false
11541 4050 Acked Write 7 Done Done Sent false
11542 4051 Acked Flush 0 Done Done Sent false
11543 4052 Acked Read 1 Done Done Sent false
11544 4053 Acked Write 8 Done Done Sent false
11545 4054 Acked Write 4 Done Done Sent false
11546 4055 Acked Read 1 Done Done Sent false
11547 4056 Acked Read 6 Done Done Sent false
11548 4057 Acked Write 7 Done Done Sent false
11549 4058 Acked Read 4 Done Done Sent false
11550 4059 Acked Read 4 Done Done Sent false
11551 4060 Acked Read 4 Done Done Sent false
11552 4061 Acked Write 10 Done Done Sent false
11553 4062 Acked Write 1 Done Done Sent false
11554 4063 Acked Write 9 Done Done Sent false
11555 4064 Acked Read 10 Done Done Sent false
11556 4065 Acked Read 10 Done Done Sent false
11557 4066 Acked Read 1 Done Done Sent false
11558 4067 Acked Read 10 Done Done Sent false
11559 4068 Acked Write 7 Done Done Sent false
11560 4069 Acked Read 10 Done Done Sent false
11561 4070 Acked Read 9 Done Done Sent false
11562 4071 Acked Read 3 Done Done Sent false
11563 4072 Acked Read 9 Done Done Sent false
11564 4073 Acked Read 3 Done Done Sent false
11565 4074 Acked Read 8 Done Done Sent false
11566 4075 Acked Write 6 Done Done Sent false
11567 4076 Acked Read 3 Done Done Sent false
11568 4077 Acked Write 5 Done Done Sent false
11569 4078 Acked Read 10 Done Done Sent false
11570 4079 Acked Write 4 Done Done Sent false
11571 4080 Acked Write 7 Done Done Sent false
11572 4081 Acked Read 8 Done Done Sent false
11573 4082 Acked Read 6 Done Done Sent false
11574 4083 Acked Write 1 Done Done Sent false
11575 4084 Acked Write 8 Done Done Sent false
11576 4085 Acked Write 3 Done Done Sent false
11577 4086 Acked Read 7 Done Done Sent false
11578 4087 Acked Read 5 Done Done Sent false
11579 4088 Acked Read 7 Done Done Sent false
11580 4089 Acked Read 5 Done Done Sent false
11581 4090 Acked Write 9 Done Done Sent false
11582 4091 Acked Write 1 Done Done Sent false
11583 4092 Acked Read 8 Done Done Sent false
11584 4093 Acked Read 6 Done Done Sent false
11585 4094 Acked Flush 0 Done Done Sent false
11586 4095 Acked Read 5 Done Done Sent false
11587 4096 Acked Flush 0 Done Done Sent false
11588 4097 Acked Read 4 Done Done Sent false
11589 4098 Acked Read 2 Done Done Sent false
11590 4099 Acked Read 2 Done Done Sent false
11591 4100 Acked Flush 0 Done Done Sent false
11592 4101 Acked Write 5 Done Done Sent false
11593 4102 Acked Read 4 Done Done Sent false
11594 4103 Acked Write 3 Done Done Sent false
11595 4104 Acked Write 4 Done Done Sent false
11596 4105 Acked Read 4 Done Done Sent false
11597 4106 Acked Read 10 Done Done Sent false
11598 4107 Acked Read 1 Done Done Sent false
11599 4108 Acked Write 8 Done Done Sent false
11600 4109 Acked Write 9 Done Done Sent false
11601 4110 Acked Read 1 Done Done Sent false
11602 4111 Acked Read 2 Done Done Sent false
11603 4112 Acked Write 9 Done Done Sent false
11604 4113 Acked Read 5 Done Done Sent false
11605 4114 Acked Write 2 Done Done Sent false
11606 4115 Acked Write 4 Done Done Sent false
11607 4116 Acked Read 10 Done Done Sent false
11608 4117 Acked Write 1 Done Done Sent false
11609 4118 Acked Write 7 Done Done Sent false
11610 4119 Acked Flush 0 Done Done Sent false
11611 4120 Acked Flush 0 Done Done Sent false
11612 4121 Acked Read 6 Done Done Sent false
11613 4122 Acked Write 8 Done Done Sent false
11614 4123 Acked Read 9 Done Done Sent false
11615 4124 Acked Read 9 Done Done Sent false
11616 4125 Acked Write 2 Done Done Sent false
11617 4126 Acked Flush 0 Done Done Sent false
11618 4127 Acked Read 6 Done Done Sent false
11619 4128 Acked Read 2 Done Done Sent false
11620 4129 Acked Read 4 Done Done Sent false
11621 4130 Acked Write 1 Done Done Sent false
11622 4131 Acked Read 4 Done Done Sent false
11623 4132 Acked Flush 0 Done Done Sent false
11624 4133 Acked Write 7 Done Done Sent false
11625 4134 Acked Read 1 Done Done Sent false
11626 4135 Acked Write 8 Done Done Sent false
11627 4136 Acked Write 1 Done Done Sent false
11628 4137 Acked Read 10 Done Done Sent false
11629 4138 Acked Read 2 Done Done Sent false
11630 4139 Acked Flush 0 Done Done Sent false
11631 4140 Acked Write 9 Done Done Sent false
11632 4141 Acked Write 7 Done Done Sent false
11633 4142 Acked Read 9 Done Done Sent false
11634 4143 Acked Write 10 Done Done Sent false
11635 4144 Acked Write 3 Done Done Sent false
11636 4145 Acked Write 3 Done Done Sent false
11637 4146 Acked Write 1 Done Done Sent false
11638 4147 Acked Flush 0 Done Done Sent false
11639 4148 Acked Write 8 Done Done Sent false
11640 4149 Acked Read 5 Done Done Sent false
11641 4150 Acked Write 1 Done Done Sent false
11642 4151 Acked Flush 0 Done Done Sent false
11643 4152 Acked Read 8 Done Done Sent false
11644 4153 Acked Write 9 Done Done Sent false
11645 4154 Acked Read 6 Done Done Sent false
11646 4155 Acked Read 7 Done Done Sent false
11647 4156 Acked Read 8 Done Done Sent false
11648 4157 Acked Write 5 Done Done Sent false
11649 4158 Acked Read 10 Done Done Sent false
11650 4159 Acked Read 10 Done Done Sent false
11651 4160 Acked Read 2 Done Done Sent false
11652 4161 Acked Write 6 Done Done Sent false
11653 4162 Acked Write 1 Done Done Sent false
11654 4163 Acked Read 7 Done Done Sent false
11655 4164 Acked Flush 0 Done Done Sent false
11656 4165 Acked Write 4 Done Done Sent false
11657 4166 Acked Read 4 Done Done Sent false
11658 4167 Acked Write 4 Done Done Sent false
11659 4168 Acked Write 3 Done Done Sent false
11660 4169 Acked Write 3 Done Done Sent false
11661 4170 Acked Read 9 Done Done Sent false
11662 4171 Acked Flush 0 Done Done Sent false
11663 4172 Acked Write 10 Done Done Sent false
11664 4173 Acked Write 5 Done Done Sent false
11665 4174 Acked Read 2 Done Done Sent false
11666 4175 Acked Read 10 Done Done Sent false
11667 4176 Acked Read 6 Done Done Sent false
11668 4177 Acked Flush 0 Done Done Sent false
11669 4178 Acked Write 5 Done Done Sent false
11670 4179 Acked Flush 0 Done Done Sent false
11671 4180 Acked Write 3 Done Done Sent false
11672 4181 Acked Read 6 Done Done Sent false
11673 4182 Acked Read 6 Done Done Sent false
11674 4183 Acked Read 6 Done Done Sent false
11675 4184 Acked Read 8 Done Done Sent false
11676 4185 Acked Read 10 Done Done Sent false
11677 4186 Acked Read 3 Done Done Sent false
11678 4187 Acked Read 4 Done Done Sent false
11679 4188 Acked Write 1 Done Done Sent false
11680 4189 Acked Write 3 Done Done Sent false
11681 4190 Acked Read 5 Done Done Sent false
11682 4191 Acked Read 9 Done Done Sent false
11683 4192 Acked Read 3 Done Done Sent false
11684 4193 Acked Read 8 Done Done Sent false
11685 4194 Acked Write 9 Done Done Sent false
11686 4195 Acked Write 7 Done Done Sent false
11687 4196 Acked Write 5 Done Done Sent false
11688 4197 Acked Read 7 Done Done Sent false
11689 4198 Acked Read 10 Done Done Sent false
11690 4199 Acked Write 8 Done Done Sent false
11691 4200 Acked Read 9 Done Done Sent false
11692 4201 Acked Write 5 Done Done Sent false
11693 4202 Acked Write 2 Done Done Sent false
11694 4203 Acked Read 1 Done Done Sent false
11695 4204 Acked Read 9 Done Done Sent false
11696 4205 Acked Read 7 Done Done Sent false
11697 4206 Acked Read 10 Done Done Sent false
11698 4207 Acked Read 8 Done Done Sent false
11699 4208 Acked Write 4 Done Done Sent false
11700 4209 Acked Read 9 Done Done Sent false
11701 4210 Acked Read 2 Done Done Sent false
11702 4211 Acked Write 8 Done Done Sent false
11703 4212 Acked Write 8 Done Done Sent false
11704 4213 Acked Flush 0 Done Done Sent false
11705 4214 Acked Write 1 Done Done Sent false
11706 4215 Acked Flush 0 Done Done Sent false
11707 4216 Acked Read 6 Done Done Sent false
11708 4217 Acked Flush 0 Done Done Sent false
11709 4218 Acked Flush 0 Done Done Sent false
11710 4219 Acked Write 10 Done Done Sent false
11711 4220 Acked Write 2 Done Done Sent false
11712 4221 Acked Read 3 Done Done Sent false
11713 4222 Acked Write 6 Done Done Sent false
11714 4223 Acked Read 10 Done Done Sent false
11715 4224 Acked Read 8 Done Done Sent false
11716 4225 Acked Read 9 Done Done Sent false
11717 4226 Acked Read 5 Done Done Sent false
11718 4227 Acked Read 9 Done Done Sent false
11719 4228 Acked Write 4 Done Done Sent false
11720 4229 Acked Read 3 Done Done Sent false
11721 4230 Acked Read 6 Done Done Sent false
11722 4231 Acked Write 5 Done Done Sent false
11723 4232 Acked Read 3 Done Done Sent false
11724 4233 Acked Read 8 Done Done Sent false
11725 4234 Acked Write 4 Done Done Sent false
11726 4235 Acked Write 8 Done Done Sent false
11727 4236 Acked Read 5 Done Done Sent false
11728 4237 Acked Flush 0 Done Done Sent false
11729 4238 Acked Write 5 Done Done Sent false
11730 4239 Acked Read 9 Done Done Sent false
11731 4240 Acked Read 10 Done Done Sent false
11732 4241 Acked Read 3 Done Done Sent false
11733 4242 Acked Write 10 Done Done Sent false
11734 4243 Acked Read 2 Done Done Sent false
11735 4244 Acked Read 10 Done Done Sent false
11736 4245 Acked Read 4 Done Done Sent false
11737 4246 Acked Write 10 Done Done Sent false
11738 4247 Acked Write 6 Done Done Sent false
11739 4248 Acked Read 2 Done Done Sent false
11740 4249 Acked Write 8 Done Done Sent false
11741 4250 Acked Read 5 Done Done Sent false
11742 4251 Acked Read 9 Done Done Sent false
11743 4252 Acked Read 4 Done Done Sent false
11744 4253 Acked Read 1 Done Done Sent false
11745 4254 Acked Write 8 Done Done Sent false
11746 4255 Acked Write 7 Done Done Sent false
11747 4256 Acked Read 4 Done Done Sent false
11748 4257 Acked Write 1 Done Done Sent false
11749 4258 Acked Flush 0 Done Done Sent false
11750 4259 Acked Read 9 Done Done Sent false
11751 4260 Acked Write 5 Done Done Sent false
11752 4261 Acked Read 6 Done Done Sent false
11753 4262 Acked Read 9 Done Done Sent false
11754 4263 Acked Write 7 Done Done Sent false
11755 4264 Acked Write 1 Done Done Sent false
11756 4265 Acked Read 8 Done Done Sent false
11757 4266 Acked Read 6 Done Done Sent false
11758 4267 Acked Flush 0 Done Done Sent false
11759 4268 Acked Read 2 Done Done Sent false
11760 4269 Acked Read 9 Done Done Sent false
11761 4270 Acked Write 1 Done Done Sent false
11762 4271 Acked Write 4 Done Done Sent false
11763 4272 Acked Write 9 Done Done Sent false
11764 4273 Acked Write 10 Done Done Sent false
11765 4274 Acked Read 6 Done Done Sent false
11766 4275 Acked Write 2 Done Done Sent false
11767 4276 Acked Read 2 Done Done Sent false
11768 4277 Acked Write 4 Done Done Sent false
11769 4278 Acked Write 7 Done Done Sent false
11770 4279 Acked Write 9 Done Done Sent false
11771 4280 Acked Write 8 Done Done Sent false
11772 4281 Acked Read 9 Done Done Sent false
11773 4282 Acked Write 4 Done Done Sent false
11774 4283 Acked Write 3 Done Done Sent false
11775 4284 Acked Write 9 Done Done Sent false
11776 4285 Acked Read 2 Done Done Sent false
11777 4286 Acked Read 4 Done Done Sent false
11778 4287 Acked Read 1 Done Done Sent false
11779 4288 Acked Write 7 Done Done Sent false
11780 4289 Acked Read 6 Done Done Sent false
11781 4290 Acked Write 7 Done Done Sent false
11782 4291 Acked Read 10 Done Done Sent false
11783 4292 Acked Write 1 Done Done Sent false
11784 4293 Acked Read 2 Done Done Sent false
11785 4294 Acked Read 9 Done Done Sent false
11786 4295 Acked Read 10 Done Done Sent false
11787 4296 Acked Read 8 Done Done Sent false
11788 4297 Acked Read 7 Done Done Sent false
11789 4298 Acked Write 3 Done Done Sent false
11790 4299 Acked Write 9 Done Done Sent false
11791 4300 Acked Flush 0 Done Done Sent false
11792 4301 Acked Flush 0 Done Done Sent false
11793 4302 Acked Write 3 Done Done Sent false
11794 4303 Acked Write 1 Done Done Sent false
11795 4304 Acked Read 5 Done Done Sent false
11796 4305 Acked Write 5 Done Done Sent false
11797 4306 Acked Write 7 Done Done Sent false
11798 4307 Acked Flush 0 Done Done Sent false
11799 4308 Acked Write 9 Done Done Sent false
11800 4309 Acked Flush 0 Done Done Sent false
11801 4310 Acked Read 4 Done Done Sent false
11802 4311 Acked Read 2 Done Done Sent false
11803 4312 Acked Write 8 Done Done Sent false
11804 4313 Acked Read 5 Done Done Sent false
11805 4314 Acked Read 6 Done Done Sent false
11806 4315 Acked Flush 0 Done Done Sent false
11807 4316 Acked Write 6 Done Done Sent false
11808 4317 Acked Write 2 Done Done Sent false
11809 4318 Acked Flush 0 Done Done Sent false
11810 STATES DS:0 DS:1 DS:2 TOTAL
11811 Sent 0 0 302 302
11812 Done 302 302 0 604
11813 Skipped 0 0 0 0
11814 Error 0 0 0 0
11815 Last Flush: 4318 4318 4016
11816 Downstairs last five completed: 4016 4015 4014 4013 4012
11817 Upstairs last five completed: 4318 4317 4316 4315 4314
11818 CLIENT: Up:0 ds:302 act:2
11819 ----------------------------------------------------------------
11820 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
11821 States: Active Active Offline
11822 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
11823 4017 Acked Write 3 Done Done Sent false
11824 4018 Acked Write 7 Done Done Sent false
11825 4019 Acked Read 4 Done Done Sent false
11826 4020 Acked Write 10 Done Done Sent false
11827 4021 Acked Read 2 Done Done Sent false
11828 4022 Acked Write 7 Done Done Sent false
11829 4023 Acked Read 9 Done Done Sent false
11830 4024 Acked Read 7 Done Done Sent false
11831 4025 Acked Read 8 Done Done Sent false
11832 4026 Acked Write 5 Done Done Sent false
11833 4027 Acked Flush 0 Done Done Sent false
11834 4028 Acked Read 5 Done Done Sent false
11835 4029 Acked Write 4 Done Done Sent false
11836 4030 Acked Read 2 Done Done Sent false
11837 4031 Acked Flush 0 Done Done Sent false
11838 4032 Acked Read 1 Done Done Sent false
11839 4033 Acked Write 4 Done Done Sent false
11840 4034 Acked Read 10 Done Done Sent false
11841 4035 Acked Read 4 Done Done Sent false
11842 4036 Acked Read 3 Done Done Sent false
11843 4037 Acked Write 6 Done Done Sent false
11844 4038 Acked Write 1 Done Done Sent false
11845 4039 Acked Write 8 Done Done Sent false
11846 4040 Acked Read 5 Done Done Sent false
11847 4041 Acked Read 3 Done Done Sent false
11848 4042 Acked Write 1 Done Done Sent false
11849 4043 Acked Read 9 Done Done Sent false
11850 4044 Acked Write 8 Done Done Sent false
11851 4045 Acked Read 10 Done Done Sent false
11852 4046 Acked Read 8 Done Done Sent false
11853 4047 Acked Read 5 Done Done Sent false
11854 4048 Acked Read 3 Done Done Sent false
11855 4049 Acked Read 1 Done Done Sent false
11856 4050 Acked Write 7 Done Done Sent false
11857 4051 Acked Flush 0 Done Done Sent false
11858 4052 Acked Read 1 Done Done Sent false
11859 4053 Acked Write 8 Done Done Sent false
11860 4054 Acked Write 4 Done Done Sent false
11861 4055 Acked Read 1 Done Done Sent false
11862 4056 Acked Read 6 Done Done Sent false
11863 4057 Acked Write 7 Done Done Sent false
11864 4058 Acked Read 4 Done Done Sent false
11865 4059 Acked Read 4 Done Done Sent false
11866 4060 Acked Read 4 Done Done Sent false
11867 4061 Acked Write 10 Done Done Sent false
11868 4062 Acked Write 1 Done Done Sent false
11869 4063 Acked Write 9 Done Done Sent false
11870 4064 Acked Read 10 Done Done Sent false
11871 4065 Acked Read 10 Done Done Sent false
11872 4066 Acked Read 1 Done Done Sent false
11873 4067 Acked Read 10 Done Done Sent false
11874 4068 Acked Write 7 Done Done Sent false
11875 4069 Acked Read 10 Done Done Sent false
11876 4070 Acked Read 9 Done Done Sent false
11877 4071 Acked Read 3 Done Done Sent false
11878 4072 Acked Read 9 Done Done Sent false
11879 4073 Acked Read 3 Done Done Sent false
11880 4074 Acked Read 8 Done Done Sent false
11881 4075 Acked Write 6 Done Done Sent false
11882 4076 Acked Read 3 Done Done Sent false
11883 4077 Acked Write 5 Done Done Sent false
11884 4078 Acked Read 10 Done Done Sent false
11885 4079 Acked Write 4 Done Done Sent false
11886 4080 Acked Write 7 Done Done Sent false
11887 4081 Acked Read 8 Done Done Sent false
11888 4082 Acked Read 6 Done Done Sent false
11889 4083 Acked Write 1 Done Done Sent false
11890 4084 Acked Write 8 Done Done Sent false
11891 4085 Acked Write 3 Done Done Sent false
11892 4086 Acked Read 7 Done Done Sent false
11893 4087 Acked Read 5 Done Done Sent false
11894 4088 Acked Read 7 Done Done Sent false
11895 4089 Acked Read 5 Done Done Sent false
11896 4090 Acked Write 9 Done Done Sent false
11897 4091 Acked Write 1 Done Done Sent false
11898 4092 Acked Read 8 Done Done Sent false
11899 4093 Acked Read 6 Done Done Sent false
11900 4094 Acked Flush 0 Done Done Sent false
11901 4095 Acked Read 5 Done Done Sent false
11902 4096 Acked Flush 0 Done Done Sent false
11903 4097 Acked Read 4 Done Done Sent false
11904 4098 Acked Read 2 Done Done Sent false
11905 4099 Acked Read 2 Done Done Sent false
11906 4100 Acked Flush 0 Done Done Sent false
11907 4101 Acked Write 5 Done Done Sent false
11908 4102 Acked Read 4 Done Done Sent false
11909 4103 Acked Write 3 Done Done Sent false
11910 4104 Acked Write 4 Done Done Sent false
11911 4105 Acked Read 4 Done Done Sent false
11912 4106 Acked Read 10 Done Done Sent false
11913 4107 Acked Read 1 Done Done Sent false
11914 4108 Acked Write 8 Done Done Sent false
11915 4109 Acked Write 9 Done Done Sent false
11916 4110 Acked Read 1 Done Done Sent false
11917 4111 Acked Read 2 Done Done Sent false
11918 4112 Acked Write 9 Done Done Sent false
11919 4113 Acked Read 5 Done Done Sent false
11920 4114 Acked Write 2 Done Done Sent false
11921 4115 Acked Write 4 Done Done Sent false
11922 4116 Acked Read 10 Done Done Sent false
11923 4117 Acked Write 1 Done Done Sent false
11924 4118 Acked Write 7 Done Done Sent false
11925 4119 Acked Flush 0 Done Done Sent false
11926 4120 Acked Flush 0 Done Done Sent false
11927 4121 Acked Read 6 Done Done Sent false
11928 4122 Acked Write 8 Done Done Sent false
11929 4123 Acked Read 9 Done Done Sent false
11930 4124 Acked Read 9 Done Done Sent false
11931 4125 Acked Write 2 Done Done Sent false
11932 4126 Acked Flush 0 Done Done Sent false
11933 4127 Acked Read 6 Done Done Sent false
11934 4128 Acked Read 2 Done Done Sent false
11935 4129 Acked Read 4 Done Done Sent false
11936 4130 Acked Write 1 Done Done Sent false
11937 4131 Acked Read 4 Done Done Sent false
11938 4132 Acked Flush 0 Done Done Sent false
11939 4133 Acked Write 7 Done Done Sent false
11940 4134 Acked Read 1 Done Done Sent false
11941 4135 Acked Write 8 Done Done Sent false
11942 4136 Acked Write 1 Done Done Sent false
11943 4137 Acked Read 10 Done Done Sent false
11944 4138 Acked Read 2 Done Done Sent false
11945 4139 Acked Flush 0 Done Done Sent false
11946 4140 Acked Write 9 Done Done Sent false
11947 4141 Acked Write 7 Done Done Sent false
11948 4142 Acked Read 9 Done Done Sent false
11949 4143 Acked Write 10 Done Done Sent false
11950 4144 Acked Write 3 Done Done Sent false
11951 4145 Acked Write 3 Done Done Sent false
11952 4146 Acked Write 1 Done Done Sent false
11953 4147 Acked Flush 0 Done Done Sent false
11954 4148 Acked Write 8 Done Done Sent false
11955 4149 Acked Read 5 Done Done Sent false
11956 4150 Acked Write 1 Done Done Sent false
11957 4151 Acked Flush 0 Done Done Sent false
11958 4152 Acked Read 8 Done Done Sent false
11959 4153 Acked Write 9 Done Done Sent false
11960 4154 Acked Read 6 Done Done Sent false
11961 4155 Acked Read 7 Done Done Sent false
11962 4156 Acked Read 8 Done Done Sent false
11963 4157 Acked Write 5 Done Done Sent false
11964 4158 Acked Read 10 Done Done Sent false
11965 4159 Acked Read 10 Done Done Sent false
11966 4160 Acked Read 2 Done Done Sent false
11967 4161 Acked Write 6 Done Done Sent false
11968 4162 Acked Write 1 Done Done Sent false
11969 4163 Acked Read 7 Done Done Sent false
11970 4164 Acked Flush 0 Done Done Sent false
11971 4165 Acked Write 4 Done Done Sent false
11972 4166 Acked Read 4 Done Done Sent false
11973 4167 Acked Write 4 Done Done Sent false
11974 4168 Acked Write 3 Done Done Sent false
11975 4169 Acked Write 3 Done Done Sent false
11976 4170 Acked Read 9 Done Done Sent false
11977 4171 Acked Flush 0 Done Done Sent false
11978 4172 Acked Write 10 Done Done Sent false
11979 4173 Acked Write 5 Done Done Sent false
11980 4174 Acked Read 2 Done Done Sent false
11981 4175 Acked Read 10 Done Done Sent false
11982 4176 Acked Read 6 Done Done Sent false
11983 4177 Acked Flush 0 Done Done Sent false
11984 4178 Acked Write 5 Done Done Sent false
11985 4179 Acked Flush 0 Done Done Sent false
11986 4180 Acked Write 3 Done Done Sent false
11987 4181 Acked Read 6 Done Done Sent false
11988 4182 Acked Read 6 Done Done Sent false
11989 4183 Acked Read 6 Done Done Sent false
11990 4184 Acked Read 8 Done Done Sent false
11991 4185 Acked Read 10 Done Done Sent false
11992 4186 Acked Read 3 Done Done Sent false
11993 4187 Acked Read 4 Done Done Sent false
11994 4188 Acked Write 1 Done Done Sent false
11995 4189 Acked Write 3 Done Done Sent false
11996 4190 Acked Read 5 Done Done Sent false
11997 4191 Acked Read 9 Done Done Sent false
11998 4192 Acked Read 3 Done Done Sent false
11999 4193 Acked Read 8 Done Done Sent false
12000 4194 Acked Write 9 Done Done Sent false
12001 4195 Acked Write 7 Done Done Sent false
12002 4196 Acked Write 5 Done Done Sent false
12003 4197 Acked Read 7 Done Done Sent false
12004 4198 Acked Read 10 Done Done Sent false
12005 4199 Acked Write 8 Done Done Sent false
12006 4200 Acked Read 9 Done Done Sent false
12007 4201 Acked Write 5 Done Done Sent false
12008 4202 Acked Write 2 Done Done Sent false
12009 4203 Acked Read 1 Done Done Sent false
12010 4204 Acked Read 9 Done Done Sent false
12011 4205 Acked Read 7 Done Done Sent false
12012 4206 Acked Read 10 Done Done Sent false
12013 4207 Acked Read 8 Done Done Sent false
12014 4208 Acked Write 4 Done Done Sent false
12015 4209 Acked Read 9 Done Done Sent false
12016 4210 Acked Read 2 Done Done Sent false
12017 4211 Acked Write 8 Done Done Sent false
12018 4212 Acked Write 8 Done Done Sent false
12019 4213 Acked Flush 0 Done Done Sent false
12020 4214 Acked Write 1 Done Done Sent false
12021 4215 Acked Flush 0 Done Done Sent false
12022 4216 Acked Read 6 Done Done Sent false
12023 4217 Acked Flush 0 Done Done Sent false
12024 4218 Acked Flush 0 Done Done Sent false
12025 4219 Acked Write 10 Done Done Sent false
12026 4220 Acked Write 2 Done Done Sent false
12027 4221 Acked Read 3 Done Done Sent false
12028 4222 Acked Write 6 Done Done Sent false
12029 4223 Acked Read 10 Done Done Sent false
12030 4224 Acked Read 8 Done Done Sent false
12031 4225 Acked Read 9 Done Done Sent false
12032 4226 Acked Read 5 Done Done Sent false
12033 4227 Acked Read 9 Done Done Sent false
12034 4228 Acked Write 4 Done Done Sent false
12035 4229 Acked Read 3 Done Done Sent false
12036 4230 Acked Read 6 Done Done Sent false
12037 4231 Acked Write 5 Done Done Sent false
12038 4232 Acked Read 3 Done Done Sent false
12039 4233 Acked Read 8 Done Done Sent false
12040 4234 Acked Write 4 Done Done Sent false
12041 4235 Acked Write 8 Done Done Sent false
12042 4236 Acked Read 5 Done Done Sent false
12043 4237 Acked Flush 0 Done Done Sent false
12044 4238 Acked Write 5 Done Done Sent false
12045 4239 Acked Read 9 Done Done Sent false
12046 4240 Acked Read 10 Done Done Sent false
12047 4241 Acked Read 3 Done Done Sent false
12048 4242 Acked Write 10 Done Done Sent false
12049 4243 Acked Read 2 Done Done Sent false
12050 4244 Acked Read 10 Done Done Sent false
12051 4245 Acked Read 4 Done Done Sent false
12052 4246 Acked Write 10 Done Done Sent false
12053 4247 Acked Write 6 Done Done Sent false
12054 4248 Acked Read 2 Done Done Sent false
12055 4249 Acked Write 8 Done Done Sent false
12056 4250 Acked Read 5 Done Done Sent false
12057 4251 Acked Read 9 Done Done Sent false
12058 4252 Acked Read 4 Done Done Sent false
12059 4253 Acked Read 1 Done Done Sent false
12060 4254 Acked Write 8 Done Done Sent false
12061 4255 Acked Write 7 Done Done Sent false
12062 4256 Acked Read 4 Done Done Sent false
12063 4257 Acked Write 1 Done Done Sent false
12064 4258 Acked Flush 0 Done Done Sent false
12065 4259 Acked Read 9 Done Done Sent false
12066 4260 Acked Write 5 Done Done Sent false
12067 4261 Acked Read 6 Done Done Sent false
12068 4262 Acked Read 9 Done Done Sent false
12069 4263 Acked Write 7 Done Done Sent false
12070 4264 Acked Write 1 Done Done Sent false
12071 4265 Acked Read 8 Done Done Sent false
12072 4266 Acked Read 6 Done Done Sent false
12073 4267 Acked Flush 0 Done Done Sent false
12074 4268 Acked Read 2 Done Done Sent false
12075 4269 Acked Read 9 Done Done Sent false
12076 4270 Acked Write 1 Done Done Sent false
12077 4271 Acked Write 4 Done Done Sent false
12078 4272 Acked Write 9 Done Done Sent false
12079 4273 Acked Write 10 Done Done Sent false
12080 4274 Acked Read 6 Done Done Sent false
12081 4275 Acked Write 2 Done Done Sent false
12082 4276 Acked Read 2 Done Done Sent false
12083 4277 Acked Write 4 Done Done Sent false
12084 4278 Acked Write 7 Done Done Sent false
12085 4279 Acked Write 9 Done Done Sent false
12086 4280 Acked Write 8 Done Done Sent false
12087 4281 Acked Read 9 Done Done Sent false
12088 4282 Acked Write 4 Done Done Sent false
12089 4283 Acked Write 3 Done Done Sent false
12090 4284 Acked Write 9 Done Done Sent false
12091 4285 Acked Read 2 Done Done Sent false
12092 4286 Acked Read 4 Done Done Sent false
12093 4287 Acked Read 1 Done Done Sent false
12094 4288 Acked Write 7 Done Done Sent false
12095 4289 Acked Read 6 Done Done Sent false
12096 4290 Acked Write 7 Done Done Sent false
12097 4291 Acked Read 10 Done Done Sent false
12098 4292 Acked Write 1 Done Done Sent false
12099 4293 Acked Read 2 Done Done Sent false
12100 4294 Acked Read 9 Done Done Sent false
12101 4295 Acked Read 10 Done Done Sent false
12102 4296 Acked Read 8 Done Done Sent false
12103 4297 Acked Read 7 Done Done Sent false
12104 4298 Acked Write 3 Done Done Sent false
12105 4299 Acked Write 9 Done Done Sent false
12106 4300 Acked Flush 0 Done Done Sent false
12107 4301 Acked Flush 0 Done Done Sent false
12108 4302 Acked Write 3 Done Done Sent false
12109 4303 Acked Write 1 Done Done Sent false
12110 4304 Acked Read 5 Done Done Sent false
12111 4305 Acked Write 5 Done Done Sent false
12112 4306 Acked Write 7 Done Done Sent false
12113 4307 Acked Flush 0 Done Done Sent false
12114 4308 Acked Write 9 Done Done Sent false
12115 4309 Acked Flush 0 Done Done Sent false
12116 4310 Acked Read 4 Done Done Sent false
12117 4311 Acked Read 2 Done Done Sent false
12118 4312 Acked Write 8 Done Done Sent false
12119 4313 Acked Read 5 Done Done Sent false
12120 4314 Acked Read 6 Done Done Sent false
12121 4315 Acked Flush 0 Done Done Sent false
12122 4316 Acked Write 6 Done Done Sent false
12123 4317 Acked Write 2 Done Done Sent false
12124 4318 Acked Flush 0 Done Done Sent false
12125 STATES DS:0 DS:1 DS:2 TOTAL
12126 Sent 0 0 302 302
12127 Done 302 302 0 604
12128 Skipped 0 0 0 0
12129 Error 0 0 0 0
12130 Last Flush: 4318 4318 4016
12131 Downstairs last five completed: 4016 4015 4014 4013 4012
12132 Upstairs last five completed: 4318 4317 4316 4315 4314
12133 CLIENT: Up:0 ds:302 act:2
121342025-01-27T19:38:55.374ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121352025-01-27T19:38:55.374ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57441) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121362025-01-27T19:38:55.375ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121372025-01-27T19:38:55.375ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121382025-01-27T19:38:55.375ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121392025-01-27T19:38:55.375ZINFOcrucible: send last flush ID to this DS: 4016 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121402025-01-27T19:38:55.376ZINFOcrucible: Replied this last flush ID: 4016 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121412025-01-27T19:38:55.376ZINFOcrucible: [2] client re-new 302 jobs since flush 4016 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121422025-01-27T19:38:55.376ZINFOcrucible: [2] Replayed 302 jobs since flush: 4016 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
12143 ----------------------------------------------------------------
12144 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
12145 Downstairs last five completed: 4318 4317 4316 4315 4314
12146 Upstairs last five completed: 4318 4317 4316 4315 4314
12147 CLIENT: Up:0 ds:0 act:3
12148 Replay: All jobs finished
12149 Main task has work to do, go find it
12150 got dsc Stop(1)
12151 stop 1
12152 Jan 27 19:38:55.626 INFO request completed, latency_us: 362, response_code:[8820] Got stop action so:false kr:false
12153 204, uri: /stop/cid/1, method: POST, req_id: 41c367a2-4aae-4c76-8413-378f226f9180, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
121542025-01-27T19:38:55.626ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121552025-01-27T19:38:55.626ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
121562025-01-27T19:38:55.626ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
12157 Jan 27 19:38:55.626 INFO request completed, latency_us: 120, response_code: 200, uri: /state/cid/1, method: GET, req_id: 81bf71fe-f088-4a0c-9922-f1031751c5f3, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
121582025-01-27T19:38:55.626ZINFOcrucible: client task is exiting = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
12159 [8820] Exited with: ExitStatus(unix_wait_status(9))
12160 [8820][1] reports Exit
12161 Jan 27 19:38:59.629 INFO request completed, latency_us: 209, response_code: 200, uri: /state/cid/1, method: GET, req_id: 0e1005a3-0a20-49c2-b295-984e50526c6d, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
12162 001/300 Write block 2715 len 32768 data: 2 2 2 4 6 5 4 4
12163 002/300 Read block 4247 len 40960
12164 003/300 Flush
12165 004/300 Write block 552 len 24576 data: 5 5 5 3 2 3
12166 005/300 Read block 405 len 36864
12167 006/300 Read block 1158 len 40960
12168 007/300 Read block 3800 len 24576
12169 008/300 Read block 2726 len 20480
12170 009/300 Write block 4893 len 8192 data: 2 2
12171 010/300 Write block 2773 len 28672 data: 5 4 3 2 2 2 3
12172 011/300 Write block 828 len 32768 data: 2 3 5 5 6 5 5 5
12173 012/300 Read block 3703 len 20480
12174 013/300 Read block 4092 len 20480
12175 014/300 Write block 4480 len 28672 data: 4 4 4 3 2 2 3
12176 015/300 Read block 1777 len 16384
12177 016/300 Read block 4231 len 12288
12178 017/300 Write block 3465 len 20480 data: 4 4 4 3 3
12179 018/300 Read block 1402 len 24576
12180 019/300 Write block 1342 len 36864 data: 4 4 4 4 4 4 4 3 3
12181 020/300 Read block 2565 len 4096
12182 021/300 Read block 649 len 40960
12183 022/300 Read block 4590 len 24576
12184 023/300 Write block 2004 len 12288 data: 2 2 2
12185 024/300 Write block 2949 len 24576 data: 3 3 4 4 4 4
12186 025/300 Read block 2814 len 16384
12187 026/300 Read block 389 len 12288
12188 027/300 Read block 3827 len 16384
12189 028/300 Write block 1399 len 20480 data: 5 5 5 4 4
12190 029/300 Read block 2606 len 40960
12191 030/300 Read block 4670 len 8192
12192 031/300 Read block 475 len 8192
12193 032/300 Read block 2121 len 12288
12194 033/300 Read block 1618 len 36864
12195 034/300 Read block 1124 len 36864
12196 035/300 Write block 3350 len 12288 data: 3 2 2
12197 036/300 Write block 2209 len 16384 data: 3 3 3 3
12198 037/300 Write block 3528 len 12288 data: 3 2 2
12199 038/300 Flush
12200 039/300 Write block 1510 len 4096 data: 2
12201 040/300 Write block 1249 len 24576 data: 3 2 3 3 3 3
12202 041/300 Write block 2610 len 16384 data: 2 3 3 3
12203 042/300 Read block 149 len 28672
12204 043/300 Write block 3367 len 24576 data: 4 4 4 4 4 5
12205 044/300 Read block 4543 len 12288
12206 045/300 Read block 2773 len 40960
12207 046/300 Read block 1255 len 8192
12208 047/300 Read block 997 len 20480
12209 048/300 Read block 3390 len 8192
12210 049/300 Read block 3943 len 8192
12211 050/300 Read block 189 len 16384
12212 051/300 Write block 4759 len 32768 data: 2 2 2 2 2 3 3 2
12213 052/300 Flush
12214 053/300 Read block 334 len 40960
12215 054/300 Flush
12216 055/300 Read block 3164 len 40960
12217 056/300 Write block 768 len 24576 data: 3 3 3 3 3 4
12218 057/300 Write block 478 len 24576 data: 5 4 4 4 4 3
12219 058/300 Read block 4167 len 20480
12220 059/300 Write block 4256 len 36864 data: 3 3 2 2 3 3 3 3 3
12221 060/300 Read block 2853 len 24576
12222 061/300 Read block 1502 len 12288
12223 062/300 Read block 4134 len 8192
12224 063/300 Read block 2912 len 12288
12225 064/300 Read block 2743 len 4096
12226 065/300 Write block 2287 len 36864 data: 6 8 7 7 5 5 2 2 2
12227 066/300 Write block 2285 len 8192 data: 4 5
12228 067/300 Read block 1492 len 20480
12229 068/300 Write block 2256 len 20480 data: 5 4 4 3 3
12230 069/300 Write block 3031 len 16384 data: 3 2 2 2
12231 070/300 Read block 3226 len 40960
12232 071/300 Read block 2388 len 16384
12233 072/300 Write block 4335 len 12288 data: 5 6 6
12234 073/300 Read block 347 len 20480
12235 074/300 Read block 4319 len 8192
12236 075/300 Write block 2259 len 24576 data: 4 4 3 3 3 2
12237 076/300 Read block 4150 len 28672
12238 077/300 Flush
12239 078/300 Read block 1800 len 24576
12240 079/300 Read block 4562 len 40960
12241 080/300 Write block 1697 len 40960 data: 3 3 3 2 2 2 2 2 2 2
12242 081/300 Write block 2441 len 4096 data: 7
12243 082/300 Write block 529 len 28672 data: 2 2 3 3 3 3 3
12244 083/300 Write block 1998 len 32768 data: 5 5 3 3 3 3 3 3
12245 084/300 Flush
12246 085/300 Read block 1422 len 20480
12247 086/300 Read block 1560 len 8192
12248 087/300 Write block 3975 len 36864 data: 2 2 2 2 2 2 2 2 2
12249 088/300 Write block 4762 len 40960 data: 3 3 4 4 3 2 2 3 4 5
12250 089/300 Read block 4120 len 36864
12251 090/300 Read block 2406 len 32768
12252 091/300 Write block 4821 len 28672 data: 2 2 2 3 3 2 2
12253 092/300 Write block 2409 len 8192 data: 3 4
12254 093/300 Read block 4365 len 4096
12255 094/300 Read block 3056 len 16384
12256 095/300 Read block 4950 len 12288
12257 096/300 Write block 3001 len 8192 data: 4 4
12258 097/300 Write block 3143 len 40960 data: 8 5 5 4 3 3 3 4 5 4
12259 098/300 Flush
12260 099/300 Read block 2305 len 40960
12261 100/300 Read block 2436 len 28672
12262 101/300 Write block 1751 len 4096 data: 3
12263 102/300 Write block 1409 len 8192 data: 4 3
12264 103/300 Write block 2392 len 4096 data: 5
12265 104/300 Write block 2872 len 28672 data: 3 3 3 4 4 4 3
12266 105/300 Write block 2264 len 32768 data: 3 2 2 3 4 4 2 2
12267 106/300 Write block 3074 len 32768 data: 2 2 2 2 2 2 2 2
12268 107/300 Read block 1819 len 8192
12269 108/300 Read block 3265 len 40960
12270 109/300 Flush
12271 110/300 Write block 1264 len 36864 data: 3 3 3 2 2 2 2 2 2
12272 111/300 Read block 1803 len 4096
12273 112/300 Write block 4056 len 36864 data: 5 5 5 5 3 3 3 3 2
12274 113/300 Read block 2316 len 20480
12275 114/300 Write block 2508 len 4096 data: 3
12276 115/300 Write block 2172 len 20480 data: 4 5 5 6 6
12277 116/300 Read block 866 len 24576
12278 117/300 Read block 4496 len 40960
12279 118/300 Flush
12280 119/300 Read block 3621 len 36864
12281 120/300 Read block 1391 len 40960
12282 121/300 Read block 1612 len 24576
12283 122/300 Write block 2927 len 12288 data: 2 3 3
12284 123/300 Read block 509 len 12288
12285 124/300 Read block 4173 len 4096
12286 125/300 Write block 222 len 4096 data: 3
12287 126/300 Read block 1304 len 12288
12288 127/300 Write block 4805 len 4096 data: 5
12289 128/300 Read block 3873 len 20480
12290 129/300 Read block 780 len 40960
12291 130/300 Write block 1981 len 32768 data: 2 2 3 3 4 4 4 4
12292 131/300 Flush
12293 132/300 Read block 705 len 12288
12294 133/300 Read block 4362 len 28672
12295 134/300 Read block 1368 len 40960
12296 135/300 Write block 3346 len 4096 data: 3
12297 136/300 Flush
12298 137/300 Write block 3606 len 36864 data: 4 4 4 3 3 3 2 2 3
12299 138/300 Write block 1615 len 24576 data: 3 4 4 4 4 3
12300 139/300 Read block 2533 len 16384
12301 140/300 Read block 4933 len 40960
12302 141/300 Read block 1218 len 28672
12303 142/300 Write block 658 len 32768 data: 5 5 4 4 3 3 3 3
12304 143/300 Flush
12305 144/300 Write block 676 len 12288 data: 3 3 3
12306 145/300 Read block 69 len 32768
12307 146/300 Read block 2946 len 28672
12308 147/300 Flush
12309 148/300 Read block 3463 len 12288
12310 149/300 Read block 756 len 12288
12311 150/300 Write block 4736 len 8192 data: 6 6
12312 151/300 Write block 1814 len 36864 data: 4 5 4 4 3 2 3 3 3
12313 152/300 Write block 3509 len 16384 data: 2 2 2 2
12314 153/300 Read block 3773 len 32768
12315 154/300 Write block 2375 len 16384 data: 4 4 4 5
12316 155/300 Write block 2716 len 16384 data: 3 3 5 7
12317 156/300 Read block 3878 len 32768
12318 157/300 Write block 2029 len 8192 data: 4 4
12319 158/300 Read block 2221 len 8192
12320 159/300 Read block 4293 len 16384
12321 160/300 Write block 409 len 20480 data: 4 4 3 3 3
12322 161/300 Write block 3153 len 20480 data: 3 3 3 3 3
12323 162/300 Write block 1626 len 40960 data: 4 7 5 4 3 3 3 3 3 3
12324 163/300 Read block 4676 len 16384
12325 164/300 Read block 3468 len 32768
12326 165/300 Read block 1801 len 16384
12327 166/300 Write block 1891 len 8192 data: 4 4
12328 167/300 Write block 3567 len 32768 data: 3 4 4 4 4 3 3 3
12329 168/300 Write block 3931 len 28672 data: 4 4 4 3 3 4 5
12330 169/300 Read block 1249 len 8192
12331 170/300 Read block 3710 len 16384
12332 171/300 Read block 1675 len 36864
12333 172/300 Write block 3338 len 16384 data: 4 4 4 2
12334 173/300 Read block 2757 len 32768
12335 174/300 Read block 2278 len 32768
12336 175/300 Write block 2611 len 16384 data: 4 4 4 3
12337 176/300 Write block 2201 len 12288 data: 5 4 5
12338 177/300 Read block 1425 len 12288
12339 178/300 Read block 3625 len 36864
12340 179/300 Write block 3606 len 16384 data: 5 5 5 4
12341 180/300 Write block 2974 len 28672 data: 4 3 3 3 4 6 6
12342 181/300 Read block 3711 len 24576
12343 182/300 Write block 3850 len 32768 data: 3 3 2 3 3 3 2 2
12344 183/300 Write block 4336 len 16384 data: 7 7 6 5
12345 184/300 Write block 1127 len 36864 data: 5 6 7 5 5 5 4 4 3
12346 185/300 Write block 3062 len 28672 data: 3 3 2 2 2 2 2
12347 186/300 Read block 291 len 40960
12348 187/300 Read block 1903 len 40960
12349 188/300 Write block 4740 len 28672 data: 4 4 3 2 2 2 3
12350 189/300 Read block 2581 len 32768
12351 190/300 Read block 305 len 8192
12352 191/300 Read block 2034 len 28672
12353 192/300 Write block 4861 len 12288 data: 4 3 4
12354 193/300 Write block 517 len 28672 data: 2 2 2 2 2 2 2
12355 194/300 Read block 4269 len 24576
12356 195/300 Write block 2076 len 36864 data: 3 3 3 3 3 3 3 3 3
12357 196/300 Write block 1388 len 16384 data: 3 3 3 4
12358 197/300 Read block 4407 len 24576
12359 198/300 Flush
12360 199/300 Write block 3537 len 16384 data: 2 2 3 3
12361 200/300 Read block 1114 len 28672
12362 201/300 Read block 3685 len 32768
12363 202/300 Read block 834 len 36864
12364 203/300 Flush
12365 204/300 Write block 3115 len 20480 data: 2 2 3 3 2
12366 205/300 Write block 3462 len 4096 data: 3
12367 206/300 Read block 3955 len 36864
12368 207/300 Read block 3728 len 8192
12369 208/300 Read block 3057 len 28672
12370 209/300 Read block 3374 len 16384
12371 210/300 Write block 934 len 20480 data: 3 2 2 2 2
12372 211/300 Read block 4425 len 36864
12373 212/300 Read block 142 len 40960
12374 213/300 Read block 397 len 28672
12375 214/300 Write block 3051 len 24576 data: 5 4 3 4 3 4
12376 215/300 Read block 4865 len 20480
12377 216/300 Read block 428 len 28672
12378 217/300 Write block 3767 len 4096 data: 5
12379 218/300 Read block 1927 len 20480
12380 219/300 Read block 912 len 8192
12381 220/300 Write block 2679 len 36864 data: 6 7 7 6 7 7 5 5 4
12382 221/300 Read block 3690 len 32768
12383 222/300 Read block 2440 len 4096
12384 223/300 Write block 4211 len 12288 data: 2 2 2
12385 224/300 Read block 2304 len 20480
12386 225/300 Read block 980 len 40960
12387 226/300 Read block 287 len 28672
12388 227/300 Write block 272 len 28672 data: 3 3 3 4 4 4 5
12389 228/300 Read block 797 len 8192
12390 229/300 Read block 4470 len 36864
12391 230/300 Read block 1933 len 40960
12392 231/300 Write block 115 len 40960 data: 3 3 3 3 3 4 4 5 5 6
12393 232/300 Write block 497 len 24576 data: 5 6 6 4 4 4
12394 233/300 Read block 2741 len 24576
12395 234/300 Flush
12396 235/300 Read block 523 len 16384
12397 236/300 Write block 4224 len 4096 data: 4
12398 237/300 Read block 1936 len 16384
12399 238/300 Write block 3241 len 16384 data: 4 5 5 3
12400 239/300 Write block 3754 len 36864 data: 4 4 4 4 5 6 6 6 6
12401 240/300 Read block 2507 len 40960
12402 241/300 Read block 2179 len 36864
12403 242/300 Read block 4213 len 28672
12404 243/300 Flush
12405 244/300 Read block 3316 len 36864
12406 245/300 Read block 3627 len 12288
12407 246/300 Write block 1715 len 36864 data: 3 3 3 2 3 3 3 3 3
12408 247/300 Flush
12409 248/300 Read block 2721 len 4096
12410 249/300 Read block 4116 len 32768
12411 250/300 Read block 2001 len 12288
12412 251/300 Read block 1370 len 8192
12413 252/300 Write block 1147 len 16384 data: 3 4 4 2
12414 253/300 Read block 826 len 12288
12415 254/300 Read block 2873 len 32768
12416 255/300 Read block 2546 len 20480
12417 256/300 Read block 4405 len 40960
12418 257/300 Read block 4473 len 4096
12419 258/300 Read block 3782 len 28672
12420 259/300 Read block 4384 len 32768
12421 260/300 Read block 4471 len 28672
12422 261/300 Write block 2624 len 40960 data: 5 3 2 2 2 3 2 2 2 4
12423 262/300 Read block 1920 len 24576
12424 263/300 Read block 2423 len 20480
12425 264/300 Write block 2157 len 24576 data: 6 6 6 5 6 6
12426 265/300 Flush
12427 266/300 Read block 2974 len 36864
12428 267/300 Read block 4788 len 36864
12429 268/300 Flush
12430 269/300 Write block 114 len 12288 data: 3 4 4
12431 270/300 Read block 3719 len 24576
12432 271/300 Write block 2386 len 20480 data: 4 4 5 5 6
12433 272/300 Flush
12434 273/300 Flush
12435 274/300 Read block 3166 len 4096
12436 275/300 Write block 4797 len 20480 data: 4 3 4 4 4
12437 276/300 Write block 4864 len 28672 data: 4 3 3 4 4 5 4
12438 277/300 Read block 1754 len 20480
12439 278/300 Read block 4323 len 28672
12440 279/300 Read block 1725 len 20480
12441 280/300 Write block 3455 len 20480 data: 6 6 6 4 3
12442 281/300 Write block 2820 len 40960 data: 2 2 2 2 2 2 2 2 2 2
12443 282/300 Read block 1312 len 16384
12444 283/300 Read block 2617 len 20480
12445 284/300 Write block 4363 len 16384 data: 3 3 3 3
12446 285/300 Read block 3716 len 8192
12447 286/300 Read block 414 len 28672
12448 287/300 Read block 4608 len 12288
12449 288/300 Read block 859 len 20480
12450 289/300 Read block 949 len 32768
12451 290/300 Write block 4102 len 40960 data: 4 5 4 4 4 5 3 4 4 5
12452 291/300 Read block 1606 len 8192
12453 292/300 Read block 4336 len 12288
12454 293/300 Write block 2266 len 16384 data: 3 4 5 5
12455 294/300 Write block 4225 len 24576 data: 4 3 2 2 3 3
12456 295/300 Write block 1889 len 4096 data: 4
12457 296/300 Write block 2216 len 16384 data: 2 2 2 2
12458 297/300 Write block 4041 len 28672 data: 3 4 5 4 4 4 3
12459 298/300 Read block 707 len 36864
12460 299/300 Write block 1572 len 36864 data: 4 4 3 4 4 3 3 2 3
12461 300/300 Read block 3643 len 36864
12462 301/300 Read block 1542 len 12288
12463 Main task has work to do, go find it
12464 got dsc Start(1)
12465 start 1
12466 Jan 27 19:38:59.879 INFO request completed, latency_us: 161, response_code: 204, uri: /start/cid/1, method: POST, req_id: 2f0548c5-4a27-4305-945c-01644facebf5, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
12467 Replay: started 1, returned:Ok(())
12468 ----------------------------------------------------------------
12469 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
12470 States: Active Offline Active
12471 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12472 4319 Acked Write 8 Done Sent Done false
12473 4320 Acked Read 10 Done Sent Done false
12474 4321 Acked Flush 0 Done Sent Done false
12475 4322 Acked Write 6 Done Sent Done false
12476 4323 Acked Read 9 Done Sent Done false
12477 4324 Acked Read 10 Done Sent Done false
12478 4325 Acked Read 6 Done Sent Done false
12479 4326 Acked Read 5 Done Sent Done false
12480 4327 Acked Write 2 Done Sent Done false
12481 4328 Acked Write 7 Done Sent Done false
12482 4329 Acked Write 8 Done Sent Done false
12483 4330 Acked Read 5 Done Sent Done false
12484 4331 Acked Read 5 Done Sent Done false
12485 4332 Acked Write 7 Done Sent Done false
12486 4333 Acked Read 4 Done Sent Done false
12487 4334 Acked Read 3 Done Sent Done false
12488 4335 Acked Write 5 Done Sent Done false
12489 4336 Acked Read 6 Done Sent Done false
12490 4337 Acked Write 9 Done Sent Done false
12491 4338 Acked Read 1 Done Sent Done false
12492 4339 Acked Read 10 Done Sent Done false
12493 4340 Acked Read 6 Done Sent Done false
12494 4341 Acked Write 3 Done Sent Done false
12495 4342 Acked Write 6 Done Sent Done false
12496 4343 Acked Read 4 Done Sent Done false
12497 4344 Acked Read 3 Done Sent Done false
12498 4345 Acked Read 4 Done Sent Done false
12499 4346 Acked Write 5 Done Sent Done false
12500 4347 Acked Read 10 Done Sent Done false
12501 4348 Acked Read 2 Done Sent Done false
12502 4349 Acked Read 2 Done Sent Done false
12503 4350 Acked Read 3 Done Sent Done false
12504 4351 Acked Read 9 Done Sent Done false
12505 4352 Acked Read 9 Done Sent Done false
12506 4353 Acked Write 3 Done Sent Done false
12507 4354 Acked Write 4 Done Sent Done false
12508 4355 Acked Write 3 Done Sent Done false
12509 4356 Acked Flush 0 Done Sent Done false
12510 4357 Acked Write 1 Done Sent Done false
12511 4358 Acked Write 6 Done Sent Done false
12512 4359 Acked Write 4 Done Sent Done false
12513 4360 Acked Read 7 Done Sent Done false
12514 4361 Acked Write 6 Done Sent Done false
12515 4362 Acked Read 3 Done Sent Done false
12516 4363 Acked Read 10 Done Sent Done false
12517 4364 Acked Read 2 Done Sent Done false
12518 4365 Acked Read 5 Done Sent Done false
12519 4366 Acked Read 2 Done Sent Done false
12520 4367 Acked Read 2 Done Sent Done false
12521 4368 Acked Read 4 Done Sent Done false
12522 4369 Acked Write 8 Done Sent Done false
12523 4370 Acked Flush 0 Done Sent Done false
12524 4371 Acked Read 10 Done Sent Done false
12525 4372 Acked Flush 0 Done Sent Done false
12526 4373 Acked Read 10 Done Sent Done false
12527 4374 Acked Write 6 Done Sent Done false
12528 4375 Acked Write 6 Done Sent Done false
12529 4376 Acked Read 5 Done Sent Done false
12530 4377 Acked Write 9 Done Sent Done false
12531 4378 Acked Read 6 Done Sent Done false
12532 4379 Acked Read 3 Done Sent Done false
12533 4380 Acked Read 2 Done Sent Done false
12534 4381 Acked Read 3 Done Sent Done false
12535 4382 Acked Read 1 Done Sent Done false
12536 4383 Acked Write 9 Done Sent Done false
12537 4384 Acked Write 2 Done Sent Done false
12538 4385 Acked Read 5 Done Sent Done false
12539 4386 Acked Write 5 Done Sent Done false
12540 4387 Acked Write 4 Done Sent Done false
12541 4388 Acked Read 10 Done Sent Done false
12542 4389 Acked Read 4 Done Sent Done false
12543 4390 Acked Write 3 Done Sent Done false
12544 4391 Acked Read 5 Done Sent Done false
12545 4392 Acked Read 2 Done Sent Done false
12546 4393 Acked Write 6 Done Sent Done false
12547 4394 Acked Read 7 Done Sent Done false
12548 4395 Acked Flush 0 Done Sent Done false
12549 4396 Acked Read 6 Done Sent Done false
12550 4397 Acked Read 10 Done Sent Done false
12551 4398 Acked Write 10 Done Sent Done false
12552 4399 Acked Write 1 Done Sent Done false
12553 4400 Acked Write 7 Done Sent Done false
12554 4401 Acked Write 8 Done Sent Done false
12555 4402 Acked Flush 0 Done Sent Done false
12556 4403 Acked Read 5 Done Sent Done false
12557 4404 Acked Read 2 Done Sent Done false
12558 4405 Acked Write 9 Done Sent Done false
12559 4406 Acked Write 10 Done Sent Done false
12560 4407 Acked Read 9 Done Sent Done false
12561 4408 Acked Read 8 Done Sent Done false
12562 4409 Acked Write 7 Done Sent Done false
12563 4410 Acked Write 2 Done Sent Done false
12564 4411 Acked Read 1 Done Sent Done false
12565 4412 Acked Read 4 Done Sent Done false
12566 4413 Acked Read 3 Done Sent Done false
12567 4414 Acked Write 2 Done Sent Done false
12568 4415 Acked Write 10 Done Sent Done false
12569 4416 Acked Flush 0 Done Sent Done false
12570 4417 Acked Read 10 Done Sent Done false
12571 4418 Acked Read 7 Done Sent Done false
12572 4419 Acked Write 1 Done Sent Done false
12573 4420 Acked Write 2 Done Sent Done false
12574 4421 Acked Write 1 Done Sent Done false
12575 4422 Acked Write 7 Done Sent Done false
12576 4423 Acked Write 8 Done Sent Done false
12577 4424 Acked Write 8 Done Sent Done false
12578 4425 Acked Read 2 Done Sent Done false
12579 4426 Acked Read 10 Done Sent Done false
12580 4427 Acked Flush 0 Done Sent Done false
12581 4428 Acked Write 9 Done Sent Done false
12582 4429 Acked Read 1 Done Sent Done false
12583 4430 Acked Write 9 Done Sent Done false
12584 4431 Acked Read 5 Done Sent Done false
12585 4432 Acked Write 1 Done Sent Done false
12586 4433 Acked Write 5 Done Sent Done false
12587 4434 Acked Read 6 Done Sent Done false
12588 4435 Acked Read 10 Done Sent Done false
12589 4436 Acked Flush 0 Done Sent Done false
12590 4437 Acked Read 9 Done Sent Done false
12591 4438 Acked Read 10 Done Sent Done false
12592 4439 Acked Read 6 Done Sent Done false
12593 4440 Acked Write 3 Done Sent Done false
12594 4441 Acked Read 3 Done Sent Done false
12595 4442 Acked Read 1 Done Sent Done false
12596 4443 Acked Write 1 Done Sent Done false
12597 4444 Acked Read 3 Done Sent Done false
12598 4445 Acked Write 1 Done Sent Done false
12599 4446 Acked Read 5 Done Sent Done false
12600 4447 Acked Read 10 Done Sent Done false
12601 4448 Acked Write 8 Done Sent Done false
12602 4449 Acked Flush 0 Done Sent Done false
12603 4450 Acked Read 3 Done Sent Done false
12604 4451 Acked Read 7 Done Sent Done false
12605 4452 Acked Read 10 Done Sent Done false
12606 4453 Acked Write 1 Done Sent Done false
12607 4454 Acked Flush 0 Done Sent Done false
12608 4455 Acked Write 9 Done Sent Done false
12609 4456 Acked Write 6 Done Sent Done false
12610 4457 Acked Read 4 Done Sent Done false
12611 4458 Acked Read 10 Done Sent Done false
12612 4459 Acked Read 7 Done Sent Done false
12613 4460 Acked Write 8 Done Sent Done false
12614 4461 Acked Flush 0 Done Sent Done false
12615 4462 Acked Write 3 Done Sent Done false
12616 4463 Acked Read 8 Done Sent Done false
12617 4464 Acked Read 7 Done Sent Done false
12618 4465 Acked Flush 0 Done Sent Done false
12619 4466 Acked Read 3 Done Sent Done false
12620 4467 Acked Read 3 Done Sent Done false
12621 4468 Acked Write 2 Done Sent Done false
12622 4469 Acked Write 9 Done Sent Done false
12623 4470 Acked Write 4 Done Sent Done false
12624 4471 Acked Read 8 Done Sent Done false
12625 4472 Acked Write 4 Done Sent Done false
12626 4473 Acked Write 4 Done Sent Done false
12627 4474 Acked Read 8 Done Sent Done false
12628 4475 Acked Write 2 Done Sent Done false
12629 4476 Acked Read 2 Done Sent Done false
12630 4477 Acked Read 4 Done Sent Done false
12631 4478 Acked Write 5 Done Sent Done false
12632 4479 Acked Write 5 Done Sent Done false
12633 4480 Acked Write 10 Done Sent Done false
12634 4481 Acked Read 4 Done Sent Done false
12635 4482 Acked Read 8 Done Sent Done false
12636 4483 Acked Read 4 Done Sent Done false
12637 4484 Acked Write 2 Done Sent Done false
12638 4485 Acked Write 8 Done Sent Done false
12639 4486 Acked Write 7 Done Sent Done false
12640 4487 Acked Read 2 Done Sent Done false
12641 4488 Acked Read 4 Done Sent Done false
12642 4489 Acked Read 9 Done Sent Done false
12643 4490 Acked Write 4 Done Sent Done false
12644 4491 Acked Read 8 Done Sent Done false
12645 4492 Acked Read 8 Done Sent Done false
12646 4493 Acked Write 4 Done Sent Done false
12647 4494 Acked Write 3 Done Sent Done false
12648 4495 Acked Read 3 Done Sent Done false
12649 4496 Acked Read 9 Done Sent Done false
12650 4497 Acked Write 4 Done Sent Done false
12651 4498 Acked Write 7 Done Sent Done false
12652 4499 Acked Read 6 Done Sent Done false
12653 4500 Acked Write 8 Done Sent Done false
12654 4501 Acked Write 4 Done Sent Done false
12655 4502 Acked Write 9 Done Sent Done false
12656 4503 Acked Write 7 Done Sent Done false
12657 4504 Acked Read 10 Done Sent Done false
12658 4505 Acked Read 10 Done Sent Done false
12659 4506 Acked Write 7 Done Sent Done false
12660 4507 Acked Read 8 Done Sent Done false
12661 4508 Acked Read 2 Done Sent Done false
12662 4509 Acked Read 7 Done Sent Done false
12663 4510 Acked Write 3 Done Sent Done false
12664 4511 Acked Write 7 Done Sent Done false
12665 4512 Acked Read 6 Done Sent Done false
12666 4513 Acked Write 9 Done Sent Done false
12667 4514 Acked Write 4 Done Sent Done false
12668 4515 Acked Read 6 Done Sent Done false
12669 4516 Acked Flush 0 Done Sent Done false
12670 4517 Acked Write 4 Done Sent Done false
12671 4518 Acked Read 7 Done Sent Done false
12672 4519 Acked Read 8 Done Sent Done false
12673 4520 Acked Read 9 Done Sent Done false
12674 4521 Acked Flush 0 Done Sent Done false
12675 4522 Acked Write 5 Done Sent Done false
12676 4523 Acked Write 1 Done Sent Done false
12677 4524 Acked Read 9 Done Sent Done false
12678 4525 Acked Read 2 Done Sent Done false
12679 4526 Acked Read 7 Done Sent Done false
12680 4527 Acked Read 4 Done Sent Done false
12681 4528 Acked Write 5 Done Sent Done false
12682 4529 Acked Read 9 Done Sent Done false
12683 4530 Acked Read 10 Done Sent Done false
12684 4531 Acked Read 7 Done Sent Done false
12685 4532 Acked Write 6 Done Sent Done false
12686 4533 Acked Read 5 Done Sent Done false
12687 4534 Acked Read 7 Done Sent Done false
12688 4535 Acked Write 1 Done Sent Done false
12689 4536 Acked Read 5 Done Sent Done false
12690 4537 Acked Read 2 Done Sent Done false
12691 4538 Acked Write 9 Done Sent Done false
12692 4539 Acked Read 8 Done Sent Done false
12693 4540 Acked Read 1 Done Sent Done false
12694 4541 Acked Write 3 Done Sent Done false
12695 4542 Acked Read 5 Done Sent Done false
12696 4543 Acked Read 10 Done Sent Done false
12697 4544 Acked Read 7 Done Sent Done false
12698 4545 Acked Write 7 Done Sent Done false
12699 4546 Acked Read 2 Done Sent Done false
12700 4547 Acked Read 9 Done Sent Done false
12701 4548 Acked Read 10 Done Sent Done false
12702 4549 Acked Write 10 Done Sent Done false
12703 4550 Acked Write 6 Done Sent Done false
12704 4551 Acked Read 6 Done Sent Done false
12705 4552 Acked Flush 0 Done Sent Done false
12706 4553 Acked Read 4 Done Sent Done false
12707 4554 Acked Write 1 Done Sent Done false
12708 4555 Acked Read 4 Done Sent Done false
12709 4556 Acked Write 4 Done Sent Done false
12710 4557 Acked Write 9 Done Sent Done false
12711 4558 Acked Read 10 Done Sent Done false
12712 4559 Acked Read 9 Done Sent Done false
12713 4560 Acked Read 7 Done Sent Done false
12714 4561 Acked Flush 0 Done Sent Done false
12715 4562 Acked Read 9 Done Sent Done false
12716 4563 Acked Read 3 Done Sent Done false
12717 4564 Acked Write 9 Done Sent Done false
12718 4565 Acked Flush 0 Done Sent Done false
12719 4566 Acked Read 1 Done Sent Done false
12720 4567 Acked Read 8 Done Sent Done false
12721 4568 Acked Read 3 Done Sent Done false
12722 4569 Acked Read 2 Done Sent Done false
12723 4570 Acked Write 4 Done Sent Done false
12724 4571 Acked Read 3 Done Sent Done false
12725 4572 Acked Read 8 Done Sent Done false
12726 4573 Acked Read 5 Done Sent Done false
12727 4574 Acked Read 10 Done Sent Done false
12728 4575 Acked Read 1 Done Sent Done false
12729 4576 Acked Read 7 Done Sent Done false
12730 4577 Acked Read 8 Done Sent Done false
12731 4578 Acked Read 7 Done Sent Done false
12732 4579 Acked Write 10 Done Sent Done false
12733 4580 Acked Read 6 Done Sent Done false
12734 4581 Acked Read 5 Done Sent Done false
12735 4582 Acked Write 6 Done Sent Done false
12736 4583 Acked Flush 0 Done Sent Done false
12737 4584 Acked Read 9 Done Sent Done false
12738 4585 Acked Read 9 Done Sent Done false
12739 4586 Acked Flush 0 Done Sent Done false
12740 4587 Acked Write 3 Done Sent Done false
12741 4588 Acked Read 6 Done Sent Done false
12742 4589 Acked Write 5 Done Sent Done false
12743 4590 Acked Flush 0 Done Sent Done false
12744 4591 Acked Flush 0 Done Sent Done false
12745 4592 Acked Read 1 Done Sent Done false
12746 4593 Acked Write 5 Done Sent Done false
12747 4594 Acked Write 7 Done Sent Done false
12748 4595 Acked Read 5 Done Sent Done false
12749 4596 Acked Read 7 Done Sent Done false
12750 4597 Acked Read 5 Done Sent Done false
12751 4598 Acked Write 5 Done Sent Done false
12752 4599 Acked Write 10 Done Sent Done false
12753 4600 Acked Read 4 Done Sent Done false
12754 4601 Acked Read 5 Done Sent Done false
12755 4602 Acked Write 4 Done Sent Done false
12756 4603 Acked Read 2 Done Sent Done false
12757 4604 Acked Read 7 Done Sent Done false
12758 4605 Acked Read 3 Done Sent Done false
12759 4606 Acked Read 5 Done Sent Done false
12760 4607 Acked Read 8 Done Sent Done false
12761 4608 Acked Write 10 Done Sent Done false
12762 4609 Acked Read 2 Done Sent Done false
12763 4610 Acked Read 3 Done Sent Done false
12764 4611 Acked Write 4 Done Sent Done false
12765 4612 Acked Write 6 Done Sent Done false
12766 4613 Acked Write 1 Done Sent Done false
12767 4614 Acked Write 4 Done Sent Done false
12768 4615 Acked Write 7 Done Sent Done false
12769 4616 Acked Read 9 Done Sent Done false
12770 4617 Acked Write 9 Done Sent Done false
12771 4618 Acked Read 9 Done Sent Done false
12772 4619 Acked Read 3 Done Sent Done false
12773 STATES DS:0 DS:1 DS:2 TOTAL
12774 Sent 0 301 0 301
12775 Done 301 0 301 602
12776 Skipped 0 0 0 0
12777 Error 0 0 0 0
12778 Last Flush: 4591 4318 4591
12779 Downstairs last five completed: 4318 4317 4316 4315 4314
12780 Upstairs last five completed: 4619 4618 4617 4616 4615
12781 CLIENT: Up:0 ds:301 act:2
12782 [8820] got start action
12783 [8820] I am going to restart
12784 Starting downstairs at port 8820
12785 Make output file at "/tmp/dsc/downstairs-8820.txt"
12786 [8820][1] reports Starting
12787 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1212)
12788 [8820][1] reports Running
12789 ----------------------------------------------------------------
12790 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
12791 States: Active Offline Active
12792 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
12793 4319 Acked Write 8 Done Sent Done false
12794 4320 Acked Read 10 Done Sent Done false
12795 4321 Acked Flush 0 Done Sent Done false
12796 4322 Acked Write 6 Done Sent Done false
12797 4323 Acked Read 9 Done Sent Done false
12798 4324 Acked Read 10 Done Sent Done false
12799 4325 Acked Read 6 Done Sent Done false
12800 4326 Acked Read 5 Done Sent Done false
12801 4327 Acked Write 2 Done Sent Done false
12802 4328 Acked Write 7 Done Sent Done false
12803 4329 Acked Write 8 Done Sent Done false
12804 4330 Acked Read 5 Done Sent Done false
12805 4331 Acked Read 5 Done Sent Done false
12806 4332 Acked Write 7 Done Sent Done false
12807 4333 Acked Read 4 Done Sent Done false
12808 4334 Acked Read 3 Done Sent Done false
12809 4335 Acked Write 5 Done Sent Done false
12810 4336 Acked Read 6 Done Sent Done false
12811 4337 Acked Write 9 Done Sent Done false
12812 4338 Acked Read 1 Done Sent Done false
12813 4339 Acked Read 10 Done Sent Done false
12814 4340 Acked Read 6 Done Sent Done false
12815 4341 Acked Write 3 Done Sent Done false
12816 4342 Acked Write 6 Done Sent Done false
12817 4343 Acked Read 4 Done Sent Done false
12818 4344 Acked Read 3 Done Sent Done false
12819 4345 Acked Read 4 Done Sent Done false
12820 4346 Acked Write 5 Done Sent Done false
12821 4347 Acked Read 10 Done Sent Done false
12822 4348 Acked Read 2 Done Sent Done false
12823 4349 Acked Read 2 Done Sent Done false
12824 4350 Acked Read 3 Done Sent Done false
12825 4351 Acked Read 9 Done Sent Done false
12826 4352 Acked Read 9 Done Sent Done false
12827 4353 Acked Write 3 Done Sent Done false
12828 4354 Acked Write 4 Done Sent Done false
12829 4355 Acked Write 3 Done Sent Done false
12830 4356 Acked Flush 0 Done Sent Done false
12831 4357 Acked Write 1 Done Sent Done false
12832 4358 Acked Write 6 Done Sent Done false
12833 4359 Acked Write 4 Done Sent Done false
12834 4360 Acked Read 7 Done Sent Done false
12835 4361 Acked Write 6 Done Sent Done false
12836 4362 Acked Read 3 Done Sent Done false
12837 4363 Acked Read 10 Done Sent Done false
12838 4364 Acked Read 2 Done Sent Done false
12839 4365 Acked Read 5 Done Sent Done false
12840 4366 Acked Read 2 Done Sent Done false
12841 4367 Acked Read 2 Done Sent Done false
12842 4368 Acked Read 4 Done Sent Done false
12843 4369 Acked Write 8 Done Sent Done false
12844 4370 Acked Flush 0 Done Sent Done false
12845 4371 Acked Read 10 Done Sent Done false
12846 4372 Acked Flush 0 Done Sent Done false
12847 4373 Acked Read 10 Done Sent Done false
12848 4374 Acked Write 6 Done Sent Done false
12849 4375 Acked Write 6 Done Sent Done false
12850 4376 Acked Read 5 Done Sent Done false
12851 4377 Acked Write 9 Done Sent Done false
12852 4378 Acked Read 6 Done Sent Done false
12853 4379 Acked Read 3 Done Sent Done false
12854 4380 Acked Read 2 Done Sent Done false
12855 4381 Acked Read 3 Done Sent Done false
12856 4382 Acked Read 1 Done Sent Done false
12857 4383 Acked Write 9 Done Sent Done false
12858 4384 Acked Write 2 Done Sent Done false
12859 4385 Acked Read 5 Done Sent Done false
12860 4386 Acked Write 5 Done Sent Done false
12861 4387 Acked Write 4 Done Sent Done false
12862 4388 Acked Read 10 Done Sent Done false
12863 4389 Acked Read 4 Done Sent Done false
12864 4390 Acked Write 3 Done Sent Done false
12865 4391 Acked Read 5 Done Sent Done false
12866 4392 Acked Read 2 Done Sent Done false
12867 4393 Acked Write 6 Done Sent Done false
12868 4394 Acked Read 7 Done Sent Done false
12869 4395 Acked Flush 0 Done Sent Done false
12870 4396 Acked Read 6 Done Sent Done false
12871 4397 Acked Read 10 Done Sent Done false
12872 4398 Acked Write 10 Done Sent Done false
12873 4399 Acked Write 1 Done Sent Done false
12874 4400 Acked Write 7 Done Sent Done false
12875 4401 Acked Write 8 Done Sent Done false
12876 4402 Acked Flush 0 Done Sent Done false
12877 4403 Acked Read 5 Done Sent Done false
12878 4404 Acked Read 2 Done Sent Done false
12879 4405 Acked Write 9 Done Sent Done false
12880 4406 Acked Write 10 Done Sent Done false
12881 4407 Acked Read 9 Done Sent Done false
12882 4408 Acked Read 8 Done Sent Done false
12883 4409 Acked Write 7 Done Sent Done false
12884 4410 Acked Write 2 Done Sent Done false
12885 4411 Acked Read 1 Done Sent Done false
12886 4412 Acked Read 4 Done Sent Done false
12887 4413 Acked Read 3 Done Sent Done false
12888 4414 Acked Write 2 Done Sent Done false
12889 4415 Acked Write 10 Done Sent Done false
12890 4416 Acked Flush 0 Done Sent Done false
12891 4417 Acked Read 10 Done Sent Done false
12892 4418 Acked Read 7 Done Sent Done false
12893 4419 Acked Write 1 Done Sent Done false
12894 4420 Acked Write 2 Done Sent Done false
12895 4421 Acked Write 1 Done Sent Done false
12896 4422 Acked Write 7 Done Sent Done false
12897 4423 Acked Write 8 Done Sent Done false
12898 4424 Acked Write 8 Done Sent Done false
12899 4425 Acked Read 2 Done Sent Done false
12900 4426 Acked Read 10 Done Sent Done false
12901 4427 Acked Flush 0 Done Sent Done false
12902 4428 Acked Write 9 Done Sent Done false
12903 4429 Acked Read 1 Done Sent Done false
12904 4430 Acked Write 9 Done Sent Done false
12905 4431 Acked Read 5 Done Sent Done false
12906 4432 Acked Write 1 Done Sent Done false
12907 4433 Acked Write 5 Done Sent Done false
12908 4434 Acked Read 6 Done Sent Done false
12909 4435 Acked Read 10 Done Sent Done false
12910 4436 Acked Flush 0 Done Sent Done false
12911 4437 Acked Read 9 Done Sent Done false
12912 4438 Acked Read 10 Done Sent Done false
12913 4439 Acked Read 6 Done Sent Done false
12914 4440 Acked Write 3 Done Sent Done false
12915 4441 Acked Read 3 Done Sent Done false
12916 4442 Acked Read 1 Done Sent Done false
12917 4443 Acked Write 1 Done Sent Done false
12918 4444 Acked Read 3 Done Sent Done false
12919 4445 Acked Write 1 Done Sent Done false
12920 4446 Acked Read 5 Done Sent Done false
12921 4447 Acked Read 10 Done Sent Done false
12922 4448 Acked Write 8 Done Sent Done false
12923 4449 Acked Flush 0 Done Sent Done false
12924 4450 Acked Read 3 Done Sent Done false
12925 4451 Acked Read 7 Done Sent Done false
12926 4452 Acked Read 10 Done Sent Done false
12927 4453 Acked Write 1 Done Sent Done false
12928 4454 Acked Flush 0 Done Sent Done false
12929 4455 Acked Write 9 Done Sent Done false
12930 4456 Acked Write 6 Done Sent Done false
12931 4457 Acked Read 4 Done Sent Done false
12932 4458 Acked Read 10 Done Sent Done false
12933 4459 Acked Read 7 Done Sent Done false
12934 4460 Acked Write 8 Done Sent Done false
12935 4461 Acked Flush 0 Done Sent Done false
12936 4462 Acked Write 3 Done Sent Done false
12937 4463 Acked Read 8 Done Sent Done false
12938 4464 Acked Read 7 Done Sent Done false
12939 4465 Acked Flush 0 Done Sent Done false
12940 4466 Acked Read 3 Done Sent Done false
12941 4467 Acked Read 3 Done Sent Done false
12942 4468 Acked Write 2 Done Sent Done false
12943 4469 Acked Write 9 Done Sent Done false
12944 4470 Acked Write 4 Done Sent Done false
12945 4471 Acked Read 8 Done Sent Done false
12946 4472 Acked Write 4 Done Sent Done false
12947 4473 Acked Write 4 Done Sent Done false
12948 4474 Acked Read 8 Done Sent Done false
12949 4475 Acked Write 2 Done Sent Done false
12950 4476 Acked Read 2 Done Sent Done false
12951 4477 Acked Read 4 Done Sent Done false
12952 4478 Acked Write 5 Done Sent Done false
12953 4479 Acked Write 5 Done Sent Done false
12954 4480 Acked Write 10 Done Sent Done false
12955 4481 Acked Read 4 Done Sent Done false
12956 4482 Acked Read 8 Done Sent Done false
12957 4483 Acked Read 4 Done Sent Done false
12958 4484 Acked Write 2 Done Sent Done false
12959 4485 Acked Write 8 Done Sent Done false
12960 4486 Acked Write 7 Done Sent Done false
12961 4487 Acked Read 2 Done Sent Done false
12962 4488 Acked Read 4 Done Sent Done false
12963 4489 Acked Read 9 Done Sent Done false
12964 4490 Acked Write 4 Done Sent Done false
12965 4491 Acked Read 8 Done Sent Done false
12966 4492 Acked Read 8 Done Sent Done false
12967 4493 Acked Write 4 Done Sent Done false
12968 4494 Acked Write 3 Done Sent Done false
12969 4495 Acked Read 3 Done Sent Done false
12970 4496 Acked Read 9 Done Sent Done false
12971 4497 Acked Write 4 Done Sent Done false
12972 4498 Acked Write 7 Done Sent Done false
12973 4499 Acked Read 6 Done Sent Done false
12974 4500 Acked Write 8 Done Sent Done false
12975 4501 Acked Write 4 Done Sent Done false
12976 4502 Acked Write 9 Done Sent Done false
12977 4503 Acked Write 7 Done Sent Done false
12978 4504 Acked Read 10 Done Sent Done false
12979 4505 Acked Read 10 Done Sent Done false
12980 4506 Acked Write 7 Done Sent Done false
12981 4507 Acked Read 8 Done Sent Done false
12982 4508 Acked Read 2 Done Sent Done false
12983 4509 Acked Read 7 Done Sent Done false
12984 4510 Acked Write 3 Done Sent Done false
12985 4511 Acked Write 7 Done Sent Done false
12986 4512 Acked Read 6 Done Sent Done false
12987 4513 Acked Write 9 Done Sent Done false
12988 4514 Acked Write 4 Done Sent Done false
12989 4515 Acked Read 6 Done Sent Done false
12990 4516 Acked Flush 0 Done Sent Done false
12991 4517 Acked Write 4 Done Sent Done false
12992 4518 Acked Read 7 Done Sent Done false
12993 4519 Acked Read 8 Done Sent Done false
12994 4520 Acked Read 9 Done Sent Done false
12995 4521 Acked Flush 0 Done Sent Done false
12996 4522 Acked Write 5 Done Sent Done false
12997 4523 Acked Write 1 Done Sent Done false
12998 4524 Acked Read 9 Done Sent Done false
12999 4525 Acked Read 2 Done Sent Done false
13000 4526 Acked Read 7 Done Sent Done false
13001 4527 Acked Read 4 Done Sent Done false
13002 4528 Acked Write 5 Done Sent Done false
13003 4529 Acked Read 9 Done Sent Done false
13004 4530 Acked Read 10 Done Sent Done false
13005 4531 Acked Read 7 Done Sent Done false
13006 4532 Acked Write 6 Done Sent Done false
13007 4533 Acked Read 5 Done Sent Done false
13008 4534 Acked Read 7 Done Sent Done false
13009 4535 Acked Write 1 Done Sent Done false
13010 4536 Acked Read 5 Done Sent Done false
13011 4537 Acked Read 2 Done Sent Done false
13012 4538 Acked Write 9 Done Sent Done false
13013 4539 Acked Read 8 Done Sent Done false
13014 4540 Acked Read 1 Done Sent Done false
13015 4541 Acked Write 3 Done Sent Done false
13016 4542 Acked Read 5 Done Sent Done false
13017 4543 Acked Read 10 Done Sent Done false
13018 4544 Acked Read 7 Done Sent Done false
13019 4545 Acked Write 7 Done Sent Done false
13020 4546 Acked Read 2 Done Sent Done false
13021 4547 Acked Read 9 Done Sent Done false
13022 4548 Acked Read 10 Done Sent Done false
13023 4549 Acked Write 10 Done Sent Done false
13024 4550 Acked Write 6 Done Sent Done false
13025 4551 Acked Read 6 Done Sent Done false
13026 4552 Acked Flush 0 Done Sent Done false
13027 4553 Acked Read 4 Done Sent Done false
13028 4554 Acked Write 1 Done Sent Done false
13029 4555 Acked Read 4 Done Sent Done false
13030 4556 Acked Write 4 Done Sent Done false
13031 4557 Acked Write 9 Done Sent Done false
13032 4558 Acked Read 10 Done Sent Done false
13033 4559 Acked Read 9 Done Sent Done false
13034 4560 Acked Read 7 Done Sent Done false
13035 4561 Acked Flush 0 Done Sent Done false
13036 4562 Acked Read 9 Done Sent Done false
13037 4563 Acked Read 3 Done Sent Done false
13038 4564 Acked Write 9 Done Sent Done false
13039 4565 Acked Flush 0 Done Sent Done false
13040 4566 Acked Read 1 Done Sent Done false
13041 4567 Acked Read 8 Done Sent Done false
13042 4568 Acked Read 3 Done Sent Done false
13043 4569 Acked Read 2 Done Sent Done false
13044 4570 Acked Write 4 Done Sent Done false
13045 4571 Acked Read 3 Done Sent Done false
13046 4572 Acked Read 8 Done Sent Done false
13047 4573 Acked Read 5 Done Sent Done false
13048 4574 Acked Read 10 Done Sent Done false
13049 4575 Acked Read 1 Done Sent Done false
13050 4576 Acked Read 7 Done Sent Done false
13051 4577 Acked Read 8 Done Sent Done false
13052 4578 Acked Read 7 Done Sent Done false
13053 4579 Acked Write 10 Done Sent Done false
13054 4580 Acked Read 6 Done Sent Done false
13055 4581 Acked Read 5 Done Sent Done false
13056 4582 Acked Write 6 Done Sent Done false
13057 4583 Acked Flush 0 Done Sent Done false
13058 4584 Acked Read 9 Done Sent Done false
13059 4585 Acked Read 9 Done Sent Done false
13060 4586 Acked Flush 0 Done Sent Done false
13061 4587 Acked Write 3 Done Sent Done false
13062 4588 Acked Read 6 Done Sent Done false
13063 4589 Acked Write 5 Done Sent Done false
13064 4590 Acked Flush 0 Done Sent Done false
13065 4591 Acked Flush 0 Done Sent Done false
13066 4592 Acked Read 1 Done Sent Done false
13067 4593 Acked Write 5 Done Sent Done false
13068 4594 Acked Write 7 Done Sent Done false
13069 4595 Acked Read 5 Done Sent Done false
13070 4596 Acked Read 7 Done Sent Done false
13071 4597 Acked Read 5 Done Sent Done false
13072 4598 Acked Write 5 Done Sent Done false
13073 4599 Acked Write 10 Done Sent Done false
13074 4600 Acked Read 4 Done Sent Done false
13075 4601 Acked Read 5 Done Sent Done false
13076 4602 Acked Write 4 Done Sent Done false
13077 4603 Acked Read 2 Done Sent Done false
13078 4604 Acked Read 7 Done Sent Done false
13079 4605 Acked Read 3 Done Sent Done false
13080 4606 Acked Read 5 Done Sent Done false
13081 4607 Acked Read 8 Done Sent Done false
13082 4608 Acked Write 10 Done Sent Done false
13083 4609 Acked Read 2 Done Sent Done false
13084 4610 Acked Read 3 Done Sent Done false
13085 4611 Acked Write 4 Done Sent Done false
13086 4612 Acked Write 6 Done Sent Done false
13087 4613 Acked Write 1 Done Sent Done false
13088 4614 Acked Write 4 Done Sent Done false
13089 4615 Acked Write 7 Done Sent Done false
13090 4616 Acked Read 9 Done Sent Done false
13091 4617 Acked Write 9 Done Sent Done false
13092 4618 Acked Read 9 Done Sent Done false
13093 4619 Acked Read 3 Done Sent Done false
13094 4620 Acked Flush 0 Done Sent Done false
13095 STATES DS:0 DS:1 DS:2 TOTAL
13096 Sent 0 302 0 302
13097 Done 302 0 302 604
13098 Skipped 0 0 0 0
13099 Error 0 0 0 0
13100 Last Flush: 4620 4318 4620
13101 Downstairs last five completed: 4318 4317 4316 4315 4314
13102 Upstairs last five completed: 4620 4619 4618 4617 4616
13103 CLIENT: Up:0 ds:302 act:2
131042025-01-27T19:39:05.627ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
131052025-01-27T19:39:05.627ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:53696) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
131062025-01-27T19:39:05.628ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
131072025-01-27T19:39:05.629ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
131082025-01-27T19:39:05.629ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
131092025-01-27T19:39:05.629ZINFOcrucible: send last flush ID to this DS: 4318 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
131102025-01-27T19:39:05.629ZINFOcrucible: Replied this last flush ID: 4318 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
131112025-01-27T19:39:05.629ZINFOcrucible: [1] client re-new 302 jobs since flush 4318 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
131122025-01-27T19:39:05.629ZINFOcrucible: [1] Replayed 302 jobs since flush: 4318 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
13113 ----------------------------------------------------------------
13114 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
13115 Downstairs last five completed: 4620 4619 4618 4617 4616
13116 Upstairs last five completed: 4620 4619 4618 4617 4616
13117 CLIENT: Up:0 ds:0 act:3
13118 Replay: All jobs finished
13119 Main task has work to do, go find it
13120 got dsc Stop(0)
13121 stop 0
13122 [8810] Got stop action so:false kr:false
13123 Jan 27 19:39:07.885 INFO request completed, latency_us: 338, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 84401d90-f99c-402c-9426-c7b1c60dd0f3, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
131242025-01-27T19:39:07.886ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
13125 Jan 27 19:39:07.886 INFO request completed, latency_us{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:39:07.886260058Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
13126 : 144, response_code: 200, uri: /state/cid/0, method: GET, {"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:39:07.886295454Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
13127 req_id: 46f3d705-0d89-4e92-ac73-7b425e36205a, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
131282025-01-27T19:39:07.886ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
13129 [8810] Exited with: ExitStatus(unix_wait_status(9))
13130 [8810][0] reports Exit
13131 Jan 27 19:39:11.888 INFO request completed, latency_us: 306, response_code: 200, uri: /state/cid/0, method: GET, req_id: 108bfa34-f9ab-48eb-921d-f07cf5a77813, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
13132 001/300 Write block 2558 len 24576 data: 3 3 4 4 4 5
13133 002/300 Read block 1394 len 12288
13134 003/300 Read block 182 len 4096
13135 004/300 Write block 4348 len 40960 data: 4 4 4 4 4 4 3 4 3 3
13136 005/300 Write block 1311 len 12288 data: 3 4 3
13137 006/300 Read block 2736 len 8192
13138 007/300 Read block 684 len 24576
13139 008/300 Write block 279 len 24576 data: 7 9 10 8 6 5
13140 009/300 Read block 4962 len 32768
13141 010/300 Read block 2213 len 36864
13142 011/300 Read block 3854 len 40960
13143 012/300 Read block 1499 len 28672
13144 013/300 Read block 3655 len 36864
13145 014/300 Write block 511 len 24576 data: 3 2 2 2 2 2
13146 015/300 Write block 558 len 24576 data: 3 4 4 4 4 4
13147 016/300 Read block 201 len 12288
13148 017/300 Read block 4500 len 8192
13149 018/300 Read block 4114 len 28672
13150 019/300 Write block 1739 len 40960 data: 3 3 3 2 2 2 2 2 2 2
13151 020/300 Write block 3638 len 36864 data: 4 3 3 2 2 2 2 2 2
13152 021/300 Write block 3275 len 32768 data: 4 4 5 5 4 4 4 4
13153 022/300 Write block 3329 len 36864 data: 4 5 5 6 3 4 5 5 5
13154 023/300 Read block 2348 len 16384
13155 024/300 Write block 2247 len 4096 data: 4
13156 025/300 Write block 3471 len 40960 data: 2 2 2 3 3 3 3 3 5 5
13157 026/300 Read block 4718 len 4096
13158 027/300 Write block 1818 len 40960 data: 4 3 4 4 4 2 2 2 2 2
13159 028/300 Read block 4473 len 24576
13160 029/300 Write block 3924 len 20480 data: 3 2 3 3 3
13161 030/300 Read block 3881 len 36864
13162 031/300 Flush
13163 032/300 Write block 2140 len 32768 data: 3 4 4 4 5 4 4 3
13164 033/300 Write block 4254 len 12288 data: 3 3 4
13165 034/300 Read block 81 len 8192
13166 035/300 Read block 3712 len 36864
13167 036/300 Write block 1303 len 32768 data: 5 5 4 5 5 4 4 4
13168 037/300 Write block 2952 len 20480 data: 5 5 5 4 4
13169 038/300 Read block 487 len 32768
13170 039/300 Write block 890 len 12288 data: 2 2 2
13171 040/300 Read block 4583 len 24576
13172 041/300 Write block 3678 len 4096 data: 2
13173 042/300 Flush
13174 043/300 Write block 1854 len 28672 data: 6 7 6 5 5 6 5
13175 044/300 Read block 4176 len 40960
13176 045/300 Write block 3777 len 20480 data: 6 5 4 3 3
13177 046/300 Read block 263 len 8192
13178 047/300 Read block 4070 len 20480
13179 048/300 Read block 1171 len 40960
13180 049/300 Read block 1201 len 12288
13181 050/300 Read block 2420 len 36864
13182 051/300 Read block 530 len 20480
13183 052/300 Read block 2305 len 16384
13184 053/300 Read block 45 len 16384
13185 054/300 Write block 24 len 28672 data: 3 3 2 2 3 3 3
13186 055/300 Read block 875 len 28672
13187 056/300 Write block 501 len 8192 data: 5 5
13188 057/300 Read block 2101 len 36864
13189 058/300 Write block 1445 len 36864 data: 2 2 2 3 4 4 5 5 5
13190 059/300 Read block 2288 len 40960
13191 060/300 Read block 3570 len 16384
13192 061/300 Read block 1138 len 16384
13193 062/300 Read block 4311 len 24576
13194 063/300 Read block 2179 len 16384
13195 064/300 Flush
13196 065/300 Write block 765 len 32768 data: 2 2 3 4 4 4 4 4
13197 066/300 Read block 3577 len 16384
13198 067/300 Flush
13199 068/300 Flush
13200 069/300 Read block 1031 len 40960
13201 070/300 Flush
13202 071/300 Write block 3104 len 28672 data: 4 4 4 4 4 3 2
13203 072/300 Write block 4086 len 28672 data: 2 2 2 2 2 2 2
13204 073/300 Flush
13205 074/300 Read block 971 len 32768
13206 075/300 Read block 4351 len 8192
13207 076/300 Read block 534 len 28672
13208 077/300 Read block 821 len 16384
13209 078/300 Write block 3174 len 8192 data: 4 4
13210 079/300 Write block 137 len 16384 data: 2 4 4 3
13211 080/300 Read block 228 len 4096
13212 081/300 Write block 1463 len 36864 data: 3 4 4 4 4 3 3 5 5
13213 082/300 Read block 1506 len 8192
13214 083/300 Write block 3774 len 8192 data: 7 7
13215 084/300 Flush
13216 085/300 Read block 4921 len 12288
13217 086/300 Write block 1807 len 32768 data: 4 4 4 6 7 6 4 5
13218 087/300 Write block 2828 len 24576 data: 3 3 4 4 5 5
13219 088/300 Read block 1322 len 8192
13220 089/300 Write block 35 len 28672 data: 2 2 3 3 3 4 3
13221 090/300 Read block 2982 len 32768
13222 091/300 Read block 1030 len 24576
13223 092/300 Write block 2604 len 4096 data: 4
13224 093/300 Read block 717 len 16384
13225 094/300 Read block 2326 len 36864
13226 095/300 Write block 1819 len 16384 data: 4 5 5 5
13227 096/300 Read block 3206 len 40960
13228 097/300 Write block 3436 len 16384 data: 3 3 3 3
13229 098/300 Read block 238 len 28672
13230 099/300 Read block 817 len 4096
13231 100/300 Flush
13232 101/300 Read block 1008 len 16384
13233 102/300 Read block 2813 len 8192
13234 103/300 Read block 3274 len 24576
13235 104/300 Flush
13236 105/300 Read block 2718 len 16384
13237 106/300 Write block 2389 len 4096 data: 6
13238 107/300 Read block 663 len 16384
13239 108/300 Write block 983 len 20480 data: 3 3 3 3 3
13240 109/300 Write block 2766 len 8192 data: 3 4
13241 110/300 Read block 1140 len 36864
13242 111/300 Read block 2779 len 40960
13243 112/300 Write block 638 len 16384 data: 3 3 3 3
13244 113/300 Read block 2749 len 28672
13245 114/300 Write block 528 len 12288 data: 2 3 3
13246 115/300 Write block 4427 len 8192 data: 4 4
13247 116/300 Read block 3116 len 40960
13248 117/300 Read block 4461 len 32768
13249 118/300 Flush
13250 119/300 Flush
13251 120/300 Write block 4925 len 36864 data: 5 4 5 5 4 3 3 3 2
13252 121/300 Write block 2718 len 4096 data: 6
13253 122/300 Write block 426 len 16384 data: 3 2 2 2
13254 123/300 Read block 2357 len 20480
13255 124/300 Read block 2297 len 36864
13256 125/300 Write block 2605 len 24576 data: 4 4 4 4 3 3
13257 126/300 Read block 1160 len 12288
13258 127/300 Write block 1395 len 12288 data: 4 3 3
13259 128/300 Read block 1635 len 40960
13260 129/300 Write block 4161 len 28672 data: 4 4 4 4 5 4 4
13261 130/300 Read block 4898 len 16384
13262 131/300 Read block 1464 len 20480
13263 132/300 Write block 2669 len 40960 data: 3 3 3 4 5 5 5 4 5 5
13264 133/300 Write block 4035 len 20480 data: 4 5 5 3 3
13265 134/300 Read block 2645 len 28672
13266 135/300 Read block 4487 len 16384
13267 136/300 Flush
13268 137/300 Flush
13269 138/300 Write block 2841 len 36864 data: 4 4 4 5 5 6 4 5 5
13270 139/300 Write block 2353 len 8192 data: 6 5
13271 140/300 Write block 4008 len 16384 data: 4 5 6 4
13272 141/300 Write block 1376 len 36864 data: 7 7 6 6 5 4 4 4 4
13273 142/300 Write block 1690 len 28672 data: 2 2 2 2 3 3 3
13274 143/300 Write block 1190 len 16384 data: 3 3 3 4
13275 144/300 Read block 3868 len 4096
13276 145/300 Read block 3357 len 40960
13277 146/300 Read block 4408 len 28672
13278 147/300 Read block 2653 len 32768
13279 148/300 Write block 1122 len 24576 data: 4 4 4 4 5 6
13280 149/300 Flush
13281 150/300 Write block 3502 len 36864 data: 2 2 3 3 2 2 2 3 3
13282 151/300 Write block 2194 len 12288 data: 4 4 4
13283 152/300 Read block 971 len 4096
13284 153/300 Read block 1824 len 12288
13285 154/300 Write block 3215 len 32768 data: 4 4 4 5 5 5 5 6
13286 155/300 Read block 1905 len 32768
13287 156/300 Read block 1759 len 24576
13288 157/300 Read block 138 len 16384
13289 158/300 Read block 2372 len 36864
13290 159/300 Read block 710 len 4096
13291 160/300 Write block 2204 len 12288 data: 3 4 3
13292 161/300 Write block 2016 len 36864 data: 3 3 4 3 3 3 3 3 2
13293 162/300 Read block 3815 len 28672
13294 163/300 Write block 3988 len 28672 data: 2 2 2 2 2 2 2
13295 164/300 Read block 4048 len 36864
13296 165/300 Read block 4669 len 4096
13297 166/300 Write block 3426 len 40960 data: 7 6 5 5 5 5 4 4 4 3
13298 167/300 Read block 3044 len 28672
13299 168/300 Flush
13300 169/300 Read block 286 len 32768
13301 170/300 Read block 1959 len 12288
13302 171/300 Write block 2242 len 36864 data: 3 3 3 4 4 5 4 3 3
13303 172/300 Write block 3710 len 36864 data: 5 6 5 5 5 4 2 2 2
13304 173/300 Read block 4742 len 4096
13305 174/300 Write block 1635 len 20480 data: 4 3 3 3 3
13306 175/300 Read block 4969 len 20480
13307 176/300 Write block 2361 len 4096 data: 2
13308 177/300 Flush
13309 178/300 Read block 3110 len 4096
13310 179/300 Write block 4112 len 8192 data: 5 5
13311 180/300 Read block 3079 len 24576
13312 181/300 Read block 1058 len 40960
13313 182/300 Write block 4487 len 36864 data: 3 3 2 2 2 4 4 4 4
13314 183/300 Read block 3745 len 40960
13315 184/300 Write block 2378 len 4096 data: 6
13316 185/300 Read block 3174 len 8192
13317 186/300 Read block 4549 len 12288
13318 187/300 Read block 4584 len 4096
13319 188/300 Read block 3309 len 32768
13320 189/300 Read block 784 len 40960
13321 190/300 Read block 395 len 16384
13322 191/300 Read block 2560 len 24576
13323 192/300 Write block 2459 len 8192 data: 2 2
13324 193/300 Read block 3759 len 16384
13325 194/300 Write block 3191 len 32768 data: 4 4 4 4 4 4 3 3
13326 195/300 Read block 1583 len 8192
13327 196/300 Read block 969 len 28672
13328 197/300 Read block 4424 len 16384
13329 198/300 Write block 4133 len 20480 data: 3 4 3 3 3
13330 199/300 Write block 1072 len 36864 data: 4 3 3 3 3 3 3 4 4
13331 200/300 Write block 4503 len 4096 data: 4
13332 201/300 Write block 1240 len 8192 data: 5 3
13333 202/300 Flush
13334 203/300 Read block 799 len 32768
13335 204/300 Flush
13336 205/300 Read block 3063 len 20480
13337 206/300 Write block 3854 len 16384 data: 4 4 3 3
13338 207/300 Write block 3838 len 28672 data: 4 4 4 5 6 5 4
13339 208/300 Read block 1612 len 32768
13340 209/300 Read block 2191 len 40960
13341 210/300 Flush
13342 211/300 Read block 1361 len 4096
13343 212/300 Read block 2726 len 28672
13344 213/300 Read block 552 len 20480
13345 214/300 Read block 93 len 28672
13346 215/300 Read block 978 len 16384
13347 216/300 Write block 2793 len 12288 data: 5 6 5
13348 217/300 Write block 1123 len 8192 data: 5 5
13349 218/300 Read block 883 len 8192
13350 219/300 Write block 2592 len 40960 data: 3 3 2 2 2 2 3 3 4 3
13351 220/300 Write block 3562 len 8192 data: 5 5
13352 221/300 Read block 477 len 28672
13353 222/300 Write block 2807 len 4096 data: 2
13354 223/300 Flush
13355 224/300 Read block 4343 len 32768
13356 225/300 Write block 2590 len 28672 data: 3 3 4 4 3 3 3
13357 226/300 Write block 300 len 36864 data: 6 4 4 3 2 2 2 2 2
13358 227/300 Write block 4495 len 40960 data: 5 3 3 4 4 4 3 4 5 4
13359 228/300 Read block 4913 len 40960
13360 229/300 Read block 4792 len 24576
13361 230/300 Read block 1392 len 24576
13362 231/300 Read block 1892 len 28672
13363 232/300 Read block 2914 len 4096
13364 233/300 Write block 2372 len 40960 data: 3 3 3 5 5 5 7 4 4 3
13365 234/300 Flush
13366 235/300 Read block 1251 len 36864
13367 236/300 Read block 1489 len 8192
13368 237/300 Write block 2550 len 24576 data: 2 2 2 2 2 2
13369 238/300 Write block 3684 len 32768 data: 2 3 3 4 5 5 5 5
13370 239/300 Read block 3357 len 36864
13371 240/300 Write block 217 len 16384 data: 2 2 2 2
13372 241/300 Write block 2675 len 12288 data: 6 5 6
13373 242/300 Read block 2905 len 32768
13374 243/300 Write block 3048 len 32768 data: 4 4 5 6 5 4 5 4
13375 244/300 Write block 439 len 24576 data: 4 4 3 3 4 2
13376 245/300 Write block 1665 len 12288 data: 2 3 2
13377 246/300 Write block 4911 len 28672 data: 3 3 3 3 3 3 2
13378 247/300 Write block 3051 len 20480 data: 7 6 5 6 5
13379 248/300 Write block 1003 len 28672 data: 2 3 3 3 3 3 4
13380 249/300 Read block 3330 len 36864
13381 250/300 Read block 4194 len 8192
13382 251/300 Write block 356 len 28672 data: 5 4 4 2 2 2 2
13383 252/300 Write block 2928 len 24576 data: 4 4 3 2 2 3
13384 253/300 Read block 1951 len 28672
13385 254/300 Write block 2341 len 20480 data: 5 4 4 4 4
13386 255/300 Write block 1009 len 8192 data: 5 4
13387 256/300 Write block 4949 len 16384 data: 3 3 3 2
13388 257/300 Read block 1844 len 16384
13389 258/300 Read block 247 len 20480
13390 259/300 Write block 4089 len 20480 data: 3 3 3 3 3
13391 260/300 Read block 2874 len 8192
13392 261/300 Write block 2024 len 20480 data: 3 2 4 4 4
13393 262/300 Flush
13394 263/300 Read block 695 len 16384
13395 264/300 Write block 2682 len 8192 data: 7 8
13396 265/300 Read block 2721 len 24576
13397 266/300 Read block 3894 len 4096
13398 267/300 Read block 3848 len 12288
13399 268/300 Read block 4147 len 24576
13400 269/300 Write block 1137 len 20480 data: 3 3 3 3 3
13401 270/300 Read block 4397 len 28672
13402 271/300 Write block 997 len 32768 data: 2 2 2 2 2 2 3 4
13403 272/300 Read block 4755 len 20480
13404 273/300 Write block 3888 len 8192 data: 3 3
13405 274/300 Read block 2291 len 12288
13406 275/300 Flush
13407 276/300 Flush
13408 277/300 Read block 994 len 32768
13409 278/300 Flush
13410 279/300 Flush
13411 280/300 Write block 4305 len 4096 data: 3
13412 281/300 Write block 4630 len 16384 data: 3 3 3 4
13413 282/300 Read block 422 len 28672
13414 283/300 Read block 2373 len 20480
13415 284/300 Read block 2590 len 40960
13416 285/300 Flush
13417 286/300 Read block 3643 len 12288
13418 287/300 Write block 982 len 8192 data: 3 4
13419 288/300 Write block 4993 len 16384 data: 3 3 4 4
13420 289/300 Write block 4858 len 40960 data: 4 4 4 5 4 5 5 4 4 5
13421 290/300 Write block 3805 len 28672 data: 2 2 2 2 3 3 3
13422 291/300 Write block 4329 len 40960 data: 3 4 5 4 4 4 6 8 8 7
13423 292/300 Write block 1072 len 24576 data: 5 4 4 4 4 4
13424 293/300 Read block 4654 len 8192
13425 294/300 Read block 4064 len 24576
13426 295/300 Write block 429 len 12288 data: 3 2 2
13427 296/300 Read block 3291 len 4096
13428 297/300 Read block 4972 len 4096
13429 298/300 Write block 3715 len 20480 data: 5 3 3 3 2
13430 299/300 Write block 4181 len 8192 data: 2 2
13431 300/300 Write block 1895 len 4096 data: 3
13432 301/300 Write block 3060 len 40960 data: 4 4 4 4 3 3 3 3 3 2
13433 Main task has work to do, go find it
13434 got dsc Start(0)
13435 start 0
13436 Jan 27 19:39:12.124 INFO request completed, latency_us: 195, response_code: 204, uri: /start/cid/0, method: POST, req_id: 6275d77b-902d-4c57-864b-588bdc66c430, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
13437 Replay: started 0, returned:Ok(())
13438 ----------------------------------------------------------------
13439 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
13440 States: Offline Active Active
13441 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13442 4621 Acked Write 6 Sent Done Done false
13443 4622 Acked Read 3 Sent Done Done false
13444 4623 Acked Read 1 Sent Done Done false
13445 4624 Acked Write 10 Sent Done Done false
13446 4625 Acked Write 3 Sent Done Done false
13447 4626 Acked Read 2 Sent Done Done false
13448 4627 Acked Read 6 Sent Done Done false
13449 4628 Acked Write 6 Sent Done Done false
13450 4629 Acked Read 8 Sent Done Done false
13451 4630 Acked Read 9 Sent Done Done false
13452 4631 Acked Read 10 Sent Done Done false
13453 4632 Acked Read 7 Sent Done Done false
13454 4633 Acked Read 9 Sent Done Done false
13455 4634 Acked Write 6 Sent Done Done false
13456 4635 Acked Write 6 Sent Done Done false
13457 4636 Acked Read 3 Sent Done Done false
13458 4637 Acked Read 2 Sent Done Done false
13459 4638 Acked Read 7 Sent Done Done false
13460 4639 Acked Write 10 Sent Done Done false
13461 4640 Acked Write 9 Sent Done Done false
13462 4641 Acked Write 8 Sent Done Done false
13463 4642 Acked Write 9 Sent Done Done false
13464 4643 Acked Read 4 Sent Done Done false
13465 4644 Acked Write 1 Sent Done Done false
13466 4645 Acked Write 10 Sent Done Done false
13467 4646 Acked Read 1 Sent Done Done false
13468 4647 Acked Write 10 Sent Done Done false
13469 4648 Acked Read 6 Sent Done Done false
13470 4649 Acked Write 5 Sent Done Done false
13471 4650 Acked Read 9 Sent Done Done false
13472 4651 Acked Flush 0 Sent Done Done false
13473 4652 Acked Write 8 Sent Done Done false
13474 4653 Acked Write 3 Sent Done Done false
13475 4654 Acked Read 2 Sent Done Done false
13476 4655 Acked Read 9 Sent Done Done false
13477 4656 Acked Write 8 Sent Done Done false
13478 4657 Acked Write 5 Sent Done Done false
13479 4658 Acked Read 8 Sent Done Done false
13480 4659 Acked Write 3 Sent Done Done false
13481 4660 Acked Read 6 Sent Done Done false
13482 4661 Acked Write 1 Sent Done Done false
13483 4662 Acked Flush 0 Sent Done Done false
13484 4663 Acked Write 7 Sent Done Done false
13485 4664 Acked Read 10 Sent Done Done false
13486 4665 Acked Write 5 Sent Done Done false
13487 4666 Acked Read 2 Sent Done Done false
13488 4667 Acked Read 5 Sent Done Done false
13489 4668 Acked Read 10 Sent Done Done false
13490 4669 Acked Read 3 Sent Done Done false
13491 4670 Acked Read 9 Sent Done Done false
13492 4671 Acked Read 5 Sent Done Done false
13493 4672 Acked Read 4 Sent Done Done false
13494 4673 Acked Read 4 Sent Done Done false
13495 4674 Acked Write 7 Sent Done Done false
13496 4675 Acked Read 7 Sent Done Done false
13497 4676 Acked Write 2 Sent Done Done false
13498 4677 Acked Read 9 Sent Done Done false
13499 4678 Acked Write 9 Sent Done Done false
13500 4679 Acked Read 10 Sent Done Done false
13501 4680 Acked Read 4 Sent Done Done false
13502 4681 Acked Read 4 Sent Done Done false
13503 4682 Acked Read 6 Sent Done Done false
13504 4683 Acked Read 4 Sent Done Done false
13505 4684 Acked Flush 0 Sent Done Done false
13506 4685 Acked Write 8 Sent Done Done false
13507 4686 Acked Read 4 Sent Done Done false
13508 4687 Acked Flush 0 Sent Done Done false
13509 4688 Acked Flush 0 Sent Done Done false
13510 4689 Acked Read 10 Sent Done Done false
13511 4690 Acked Flush 0 Sent Done Done false
13512 4691 Acked Write 7 Sent Done Done false
13513 4692 Acked Write 7 Sent Done Done false
13514 4693 Acked Flush 0 Sent Done Done false
13515 4694 Acked Read 8 Sent Done Done false
13516 4695 Acked Read 2 Sent Done Done false
13517 4696 Acked Read 7 Sent Done Done false
13518 4697 Acked Read 4 Sent Done Done false
13519 4698 Acked Write 2 Sent Done Done false
13520 4699 Acked Write 4 Sent Done Done false
13521 4700 Acked Read 1 Sent Done Done false
13522 4701 Acked Write 9 Sent Done Done false
13523 4702 Acked Read 2 Sent Done Done false
13524 4703 Acked Write 2 Sent Done Done false
13525 4704 Acked Flush 0 Sent Done Done false
13526 4705 Acked Read 3 Sent Done Done false
13527 4706 Acked Write 8 Sent Done Done false
13528 4707 Acked Write 6 Sent Done Done false
13529 4708 Acked Read 2 Sent Done Done false
13530 4709 Acked Write 7 Sent Done Done false
13531 4710 Acked Read 8 Sent Done Done false
13532 4711 Acked Read 6 Sent Done Done false
13533 4712 Acked Write 1 Sent Done Done false
13534 4713 Acked Read 4 Sent Done Done false
13535 4714 Acked Read 9 Sent Done Done false
13536 4715 Acked Write 4 Sent Done Done false
13537 4716 Acked Read 10 Sent Done Done false
13538 4717 Acked Write 4 Sent Done Done false
13539 4718 Acked Read 7 Sent Done Done false
13540 4719 Acked Read 1 Sent Done Done false
13541 4720 Acked Flush 0 Sent Done Done false
13542 4721 Acked Read 4 Sent Done Done false
13543 4722 Acked Read 2 Sent Done Done false
13544 4723 Acked Read 6 Sent Done Done false
13545 4724 Acked Flush 0 Sent Done Done false
13546 4725 Acked Read 4 Sent Done Done false
13547 4726 Acked Write 1 Sent Done Done false
13548 4727 Acked Read 4 Sent Done Done false
13549 4728 Acked Write 5 Sent Done Done false
13550 4729 Acked Write 2 Sent Done Done false
13551 4730 Acked Read 9 Sent Done Done false
13552 4731 Acked Read 10 Sent Done Done false
13553 4732 Acked Write 4 Sent Done Done false
13554 4733 Acked Read 7 Sent Done Done false
13555 4734 Acked Write 3 Sent Done Done false
13556 4735 Acked Write 2 Sent Done Done false
13557 4736 Acked Read 10 Sent Done Done false
13558 4737 Acked Read 8 Sent Done Done false
13559 4738 Acked Flush 0 Sent Done Done false
13560 4739 Acked Flush 0 Sent Done Done false
13561 4740 Acked Write 9 Sent Done Done false
13562 4741 Acked Write 1 Sent Done Done false
13563 4742 Acked Write 4 Sent Done Done false
13564 4743 Acked Read 5 Sent Done Done false
13565 4744 Acked Read 9 Sent Done Done false
13566 4745 Acked Write 6 Sent Done Done false
13567 4746 Acked Read 3 Sent Done Done false
13568 4747 Acked Write 3 Sent Done Done false
13569 4748 Acked Read 10 Sent Done Done false
13570 4749 Acked Write 7 Sent Done Done false
13571 4750 Acked Read 4 Sent Done Done false
13572 4751 Acked Read 5 Sent Done Done false
13573 4752 Acked Write 10 Sent Done Done false
13574 4753 Acked Write 5 Sent Done Done false
13575 4754 Acked Read 7 Sent Done Done false
13576 4755 Acked Read 4 Sent Done Done false
13577 4756 Acked Flush 0 Sent Done Done false
13578 4757 Acked Flush 0 Sent Done Done false
13579 4758 Acked Write 9 Sent Done Done false
13580 4759 Acked Write 2 Sent Done Done false
13581 4760 Acked Write 4 Sent Done Done false
13582 4761 Acked Write 9 Sent Done Done false
13583 4762 Acked Write 7 Sent Done Done false
13584 4763 Acked Write 4 Sent Done Done false
13585 4764 Acked Read 1 Sent Done Done false
13586 4765 Acked Read 10 Sent Done Done false
13587 4766 Acked Read 7 Sent Done Done false
13588 4767 Acked Read 8 Sent Done Done false
13589 4768 Acked Write 6 Sent Done Done false
13590 4769 Acked Flush 0 Sent Done Done false
13591 4770 Acked Write 9 Sent Done Done false
13592 4771 Acked Write 3 Sent Done Done false
13593 4772 Acked Read 1 Sent Done Done false
13594 4773 Acked Read 3 Sent Done Done false
13595 4774 Acked Write 8 Sent Done Done false
13596 4775 Acked Read 8 Sent Done Done false
13597 4776 Acked Read 6 Sent Done Done false
13598 4777 Acked Read 4 Sent Done Done false
13599 4778 Acked Read 9 Sent Done Done false
13600 4779 Acked Read 1 Sent Done Done false
13601 4780 Acked Write 3 Sent Done Done false
13602 4781 Acked Write 9 Sent Done Done false
13603 4782 Acked Read 7 Sent Done Done false
13604 4783 Acked Write 7 Sent Done Done false
13605 4784 Acked Read 9 Sent Done Done false
13606 4785 Acked Read 1 Sent Done Done false
13607 4786 Acked Write 10 Sent Done Done false
13608 4787 Acked Read 7 Sent Done Done false
13609 4788 Acked Flush 0 Sent Done Done false
13610 4789 Acked Read 8 Sent Done Done false
13611 4790 Acked Read 3 Sent Done Done false
13612 4791 Acked Write 9 Sent Done Done false
13613 4792 Acked Write 9 Sent Done Done false
13614 4793 Acked Read 1 Sent Done Done false
13615 4794 Acked Write 5 Sent Done Done false
13616 4795 Acked Read 5 Sent Done Done false
13617 4796 Acked Write 1 Sent Done Done false
13618 4797 Acked Flush 0 Sent Done Done false
13619 4798 Acked Read 1 Sent Done Done false
13620 4799 Acked Write 2 Sent Done Done false
13621 4800 Acked Read 6 Sent Done Done false
13622 4801 Acked Read 10 Sent Done Done false
13623 4802 Acked Write 9 Sent Done Done false
13624 4803 Acked Read 10 Sent Done Done false
13625 4804 Acked Write 1 Sent Done Done false
13626 4805 Acked Read 2 Sent Done Done false
13627 4806 Acked Read 3 Sent Done Done false
13628 4807 Acked Read 1 Sent Done Done false
13629 4808 Acked Read 8 Sent Done Done false
13630 4809 Acked Read 10 Sent Done Done false
13631 4810 Acked Read 4 Sent Done Done false
13632 4811 Acked Read 6 Sent Done Done false
13633 4812 Acked Write 2 Sent Done Done false
13634 4813 Acked Read 4 Sent Done Done false
13635 4814 Acked Write 8 Sent Done Done false
13636 4815 Acked Read 2 Sent Done Done false
13637 4816 Acked Read 7 Sent Done Done false
13638 4817 Acked Read 4 Sent Done Done false
13639 4818 Acked Write 5 Sent Done Done false
13640 4819 Acked Write 9 Sent Done Done false
13641 4820 Acked Write 1 Sent Done Done false
13642 4821 Acked Write 2 Sent Done Done false
13643 4822 Acked Flush 0 Sent Done Done false
13644 4823 Acked Read 8 Sent Done Done false
13645 4824 Acked Flush 0 Sent Done Done false
13646 4825 Acked Read 5 Sent Done Done false
13647 4826 Acked Write 4 Sent Done Done false
13648 4827 Acked Write 7 Sent Done Done false
13649 4828 Acked Read 8 Sent Done Done false
13650 4829 Acked Read 10 Sent Done Done false
13651 4830 Acked Flush 0 Sent Done Done false
13652 4831 Acked Read 1 Sent Done Done false
13653 4832 Acked Read 7 Sent Done Done false
13654 4833 Acked Read 5 Sent Done Done false
13655 4834 Acked Read 7 Sent Done Done false
13656 4835 Acked Read 4 Sent Done Done false
13657 4836 Acked Write 3 Sent Done Done false
13658 4837 Acked Write 2 Sent Done Done false
13659 4838 Acked Read 2 Sent Done Done false
13660 4839 Acked Write 10 Sent Done Done false
13661 4840 Acked Write 2 Sent Done Done false
13662 4841 Acked Read 7 Sent Done Done false
13663 4842 Acked Write 1 Sent Done Done false
13664 4843 Acked Flush 0 Sent Done Done false
13665 4844 Acked Read 8 Sent Done Done false
13666 4845 Acked Write 7 Sent Done Done false
13667 4846 Acked Write 9 Sent Done Done false
13668 4847 Acked Write 10 Sent Done Done false
13669 4848 Acked Read 10 Sent Done Done false
13670 4849 Acked Read 6 Sent Done Done false
13671 4850 Acked Read 6 Sent Done Done false
13672 4851 Acked Read 7 Sent Done Done false
13673 4852 Acked Read 1 Sent Done Done false
13674 4853 Acked Write 10 Sent Done Done false
13675 4854 Acked Flush 0 Sent Done Done false
13676 4855 Acked Read 9 Sent Done Done false
13677 4856 Acked Read 2 Sent Done Done false
13678 4857 Acked Write 6 Sent Done Done false
13679 4858 Acked Write 8 Sent Done Done false
13680 4859 Acked Read 9 Sent Done Done false
13681 4860 Acked Write 4 Sent Done Done false
13682 4861 Acked Write 3 Sent Done Done false
13683 4862 Acked Read 8 Sent Done Done false
13684 4863 Acked Write 8 Sent Done Done false
13685 4864 Acked Write 6 Sent Done Done false
13686 4865 Acked Write 3 Sent Done Done false
13687 4866 Acked Write 7 Sent Done Done false
13688 4867 Acked Write 5 Sent Done Done false
13689 4868 Acked Write 7 Sent Done Done false
13690 4869 Acked Read 9 Sent Done Done false
13691 4870 Acked Read 2 Sent Done Done false
13692 4871 Acked Write 7 Sent Done Done false
13693 4872 Acked Write 6 Sent Done Done false
13694 4873 Acked Read 7 Sent Done Done false
13695 4874 Acked Write 5 Sent Done Done false
13696 4875 Acked Write 2 Sent Done Done false
13697 4876 Acked Write 4 Sent Done Done false
13698 4877 Acked Read 4 Sent Done Done false
13699 4878 Acked Read 5 Sent Done Done false
13700 4879 Acked Write 5 Sent Done Done false
13701 4880 Acked Read 2 Sent Done Done false
13702 4881 Acked Write 5 Sent Done Done false
13703 4882 Acked Flush 0 Sent Done Done false
13704 4883 Acked Read 4 Sent Done Done false
13705 4884 Acked Write 2 Sent Done Done false
13706 4885 Acked Read 6 Sent Done Done false
13707 4886 Acked Read 1 Sent Done Done false
13708 4887 Acked Read 3 Sent Done Done false
13709 4888 Acked Read 6 Sent Done Done false
13710 4889 Acked Write 5 Sent Done Done false
13711 4890 Acked Read 7 Sent Done Done false
13712 4891 Acked Write 8 Sent Done Done false
13713 4892 Acked Read 5 Sent Done Done false
13714 4893 Acked Write 2 Sent Done Done false
13715 4894 Acked Read 3 Sent Done Done false
13716 4895 Acked Flush 0 Sent Done Done false
13717 4896 Acked Flush 0 Sent Done Done false
13718 4897 Acked Read 8 Sent Done Done false
13719 4898 Acked Flush 0 Sent Done Done false
13720 4899 Acked Flush 0 Sent Done Done false
13721 4900 Acked Write 1 Sent Done Done false
13722 4901 Acked Write 4 Sent Done Done false
13723 4902 Acked Read 7 Sent Done Done false
13724 4903 Acked Read 5 Sent Done Done false
13725 4904 Acked Read 10 Sent Done Done false
13726 4905 Acked Flush 0 Sent Done Done false
13727 4906 Acked Read 3 Sent Done Done false
13728 4907 Acked Write 2 Sent Done Done false
13729 4908 Acked Write 4 Sent Done Done false
13730 4909 Acked Write 10 Sent Done Done false
13731 4910 Acked Write 7 Sent Done Done false
13732 4911 Acked Write 10 Sent Done Done false
13733 4912 Acked Write 6 Sent Done Done false
13734 4913 Acked Read 2 Sent Done Done false
13735 4914 Acked Read 6 Sent Done Done false
13736 4915 Acked Write 3 Sent Done Done false
13737 4916 Acked Read 1 Sent Done Done false
13738 4917 Acked Read 1 Sent Done Done false
13739 4918 Acked Write 5 Sent Done Done false
13740 4919 Acked Write 2 Sent Done Done false
13741 4920 Acked Write 1 Sent Done Done false
13742 4921 Acked Write 10 Sent Done Done false
13743 STATES DS:0 DS:1 DS:2 TOTAL
13744 Sent 301 0 0 301
13745 Done 0 301 301 602
13746 Skipped 0 0 0 0
13747 Error 0 0 0 0
13748 Last Flush: 4620 4905 4905
13749 Downstairs last five completed: 4620 4619 4618 4617 4616
13750 Upstairs last five completed: 4921 4920 4919 4918 4917
13751 CLIENT: Up:0 ds:301 act:2
13752 [8810] got start action
13753 [8810] I am going to restart
13754 Starting downstairs at port 8810
13755 Make output file at "/tmp/dsc/downstairs-8810.txt"
13756 [8810][0] reports Starting
13757 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1213)
13758 [8810][0] reports Running
13759 ----------------------------------------------------------------
13760 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
13761 States: Offline Active Active
13762 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
13763 4621 Acked Write 6 Sent Done Done false
13764 4622 Acked Read 3 Sent Done Done false
13765 4623 Acked Read 1 Sent Done Done false
13766 4624 Acked Write 10 Sent Done Done false
13767 4625 Acked Write 3 Sent Done Done false
13768 4626 Acked Read 2 Sent Done Done false
13769 4627 Acked Read 6 Sent Done Done false
13770 4628 Acked Write 6 Sent Done Done false
13771 4629 Acked Read 8 Sent Done Done false
13772 4630 Acked Read 9 Sent Done Done false
13773 4631 Acked Read 10 Sent Done Done false
13774 4632 Acked Read 7 Sent Done Done false
13775 4633 Acked Read 9 Sent Done Done false
13776 4634 Acked Write 6 Sent Done Done false
13777 4635 Acked Write 6 Sent Done Done false
13778 4636 Acked Read 3 Sent Done Done false
13779 4637 Acked Read 2 Sent Done Done false
13780 4638 Acked Read 7 Sent Done Done false
13781 4639 Acked Write 10 Sent Done Done false
13782 4640 Acked Write 9 Sent Done Done false
13783 4641 Acked Write 8 Sent Done Done false
13784 4642 Acked Write 9 Sent Done Done false
13785 4643 Acked Read 4 Sent Done Done false
13786 4644 Acked Write 1 Sent Done Done false
13787 4645 Acked Write 10 Sent Done Done false
13788 4646 Acked Read 1 Sent Done Done false
13789 4647 Acked Write 10 Sent Done Done false
13790 4648 Acked Read 6 Sent Done Done false
13791 4649 Acked Write 5 Sent Done Done false
13792 4650 Acked Read 9 Sent Done Done false
13793 4651 Acked Flush 0 Sent Done Done false
13794 4652 Acked Write 8 Sent Done Done false
13795 4653 Acked Write 3 Sent Done Done false
13796 4654 Acked Read 2 Sent Done Done false
13797 4655 Acked Read 9 Sent Done Done false
13798 4656 Acked Write 8 Sent Done Done false
13799 4657 Acked Write 5 Sent Done Done false
13800 4658 Acked Read 8 Sent Done Done false
13801 4659 Acked Write 3 Sent Done Done false
13802 4660 Acked Read 6 Sent Done Done false
13803 4661 Acked Write 1 Sent Done Done false
13804 4662 Acked Flush 0 Sent Done Done false
13805 4663 Acked Write 7 Sent Done Done false
13806 4664 Acked Read 10 Sent Done Done false
13807 4665 Acked Write 5 Sent Done Done false
13808 4666 Acked Read 2 Sent Done Done false
13809 4667 Acked Read 5 Sent Done Done false
13810 4668 Acked Read 10 Sent Done Done false
13811 4669 Acked Read 3 Sent Done Done false
13812 4670 Acked Read 9 Sent Done Done false
13813 4671 Acked Read 5 Sent Done Done false
13814 4672 Acked Read 4 Sent Done Done false
13815 4673 Acked Read 4 Sent Done Done false
13816 4674 Acked Write 7 Sent Done Done false
13817 4675 Acked Read 7 Sent Done Done false
13818 4676 Acked Write 2 Sent Done Done false
13819 4677 Acked Read 9 Sent Done Done false
13820 4678 Acked Write 9 Sent Done Done false
13821 4679 Acked Read 10 Sent Done Done false
13822 4680 Acked Read 4 Sent Done Done false
13823 4681 Acked Read 4 Sent Done Done false
13824 4682 Acked Read 6 Sent Done Done false
13825 4683 Acked Read 4 Sent Done Done false
13826 4684 Acked Flush 0 Sent Done Done false
13827 4685 Acked Write 8 Sent Done Done false
13828 4686 Acked Read 4 Sent Done Done false
13829 4687 Acked Flush 0 Sent Done Done false
13830 4688 Acked Flush 0 Sent Done Done false
13831 4689 Acked Read 10 Sent Done Done false
13832 4690 Acked Flush 0 Sent Done Done false
13833 4691 Acked Write 7 Sent Done Done false
13834 4692 Acked Write 7 Sent Done Done false
13835 4693 Acked Flush 0 Sent Done Done false
13836 4694 Acked Read 8 Sent Done Done false
13837 4695 Acked Read 2 Sent Done Done false
13838 4696 Acked Read 7 Sent Done Done false
13839 4697 Acked Read 4 Sent Done Done false
13840 4698 Acked Write 2 Sent Done Done false
13841 4699 Acked Write 4 Sent Done Done false
13842 4700 Acked Read 1 Sent Done Done false
13843 4701 Acked Write 9 Sent Done Done false
13844 4702 Acked Read 2 Sent Done Done false
13845 4703 Acked Write 2 Sent Done Done false
13846 4704 Acked Flush 0 Sent Done Done false
13847 4705 Acked Read 3 Sent Done Done false
13848 4706 Acked Write 8 Sent Done Done false
13849 4707 Acked Write 6 Sent Done Done false
13850 4708 Acked Read 2 Sent Done Done false
13851 4709 Acked Write 7 Sent Done Done false
13852 4710 Acked Read 8 Sent Done Done false
13853 4711 Acked Read 6 Sent Done Done false
13854 4712 Acked Write 1 Sent Done Done false
13855 4713 Acked Read 4 Sent Done Done false
13856 4714 Acked Read 9 Sent Done Done false
13857 4715 Acked Write 4 Sent Done Done false
13858 4716 Acked Read 10 Sent Done Done false
13859 4717 Acked Write 4 Sent Done Done false
13860 4718 Acked Read 7 Sent Done Done false
13861 4719 Acked Read 1 Sent Done Done false
13862 4720 Acked Flush 0 Sent Done Done false
13863 4721 Acked Read 4 Sent Done Done false
13864 4722 Acked Read 2 Sent Done Done false
13865 4723 Acked Read 6 Sent Done Done false
13866 4724 Acked Flush 0 Sent Done Done false
13867 4725 Acked Read 4 Sent Done Done false
13868 4726 Acked Write 1 Sent Done Done false
13869 4727 Acked Read 4 Sent Done Done false
13870 4728 Acked Write 5 Sent Done Done false
13871 4729 Acked Write 2 Sent Done Done false
13872 4730 Acked Read 9 Sent Done Done false
13873 4731 Acked Read 10 Sent Done Done false
13874 4732 Acked Write 4 Sent Done Done false
13875 4733 Acked Read 7 Sent Done Done false
13876 4734 Acked Write 3 Sent Done Done false
13877 4735 Acked Write 2 Sent Done Done false
13878 4736 Acked Read 10 Sent Done Done false
13879 4737 Acked Read 8 Sent Done Done false
13880 4738 Acked Flush 0 Sent Done Done false
13881 4739 Acked Flush 0 Sent Done Done false
13882 4740 Acked Write 9 Sent Done Done false
13883 4741 Acked Write 1 Sent Done Done false
13884 4742 Acked Write 4 Sent Done Done false
13885 4743 Acked Read 5 Sent Done Done false
13886 4744 Acked Read 9 Sent Done Done false
13887 4745 Acked Write 6 Sent Done Done false
13888 4746 Acked Read 3 Sent Done Done false
13889 4747 Acked Write 3 Sent Done Done false
13890 4748 Acked Read 10 Sent Done Done false
13891 4749 Acked Write 7 Sent Done Done false
13892 4750 Acked Read 4 Sent Done Done false
13893 4751 Acked Read 5 Sent Done Done false
13894 4752 Acked Write 10 Sent Done Done false
13895 4753 Acked Write 5 Sent Done Done false
13896 4754 Acked Read 7 Sent Done Done false
13897 4755 Acked Read 4 Sent Done Done false
13898 4756 Acked Flush 0 Sent Done Done false
13899 4757 Acked Flush 0 Sent Done Done false
13900 4758 Acked Write 9 Sent Done Done false
13901 4759 Acked Write 2 Sent Done Done false
13902 4760 Acked Write 4 Sent Done Done false
13903 4761 Acked Write 9 Sent Done Done false
13904 4762 Acked Write 7 Sent Done Done false
13905 4763 Acked Write 4 Sent Done Done false
13906 4764 Acked Read 1 Sent Done Done false
13907 4765 Acked Read 10 Sent Done Done false
13908 4766 Acked Read 7 Sent Done Done false
13909 4767 Acked Read 8 Sent Done Done false
13910 4768 Acked Write 6 Sent Done Done false
13911 4769 Acked Flush 0 Sent Done Done false
13912 4770 Acked Write 9 Sent Done Done false
13913 4771 Acked Write 3 Sent Done Done false
13914 4772 Acked Read 1 Sent Done Done false
13915 4773 Acked Read 3 Sent Done Done false
13916 4774 Acked Write 8 Sent Done Done false
13917 4775 Acked Read 8 Sent Done Done false
13918 4776 Acked Read 6 Sent Done Done false
13919 4777 Acked Read 4 Sent Done Done false
13920 4778 Acked Read 9 Sent Done Done false
13921 4779 Acked Read 1 Sent Done Done false
13922 4780 Acked Write 3 Sent Done Done false
13923 4781 Acked Write 9 Sent Done Done false
13924 4782 Acked Read 7 Sent Done Done false
13925 4783 Acked Write 7 Sent Done Done false
13926 4784 Acked Read 9 Sent Done Done false
13927 4785 Acked Read 1 Sent Done Done false
13928 4786 Acked Write 10 Sent Done Done false
13929 4787 Acked Read 7 Sent Done Done false
13930 4788 Acked Flush 0 Sent Done Done false
13931 4789 Acked Read 8 Sent Done Done false
13932 4790 Acked Read 3 Sent Done Done false
13933 4791 Acked Write 9 Sent Done Done false
13934 4792 Acked Write 9 Sent Done Done false
13935 4793 Acked Read 1 Sent Done Done false
13936 4794 Acked Write 5 Sent Done Done false
13937 4795 Acked Read 5 Sent Done Done false
13938 4796 Acked Write 1 Sent Done Done false
13939 4797 Acked Flush 0 Sent Done Done false
13940 4798 Acked Read 1 Sent Done Done false
13941 4799 Acked Write 2 Sent Done Done false
13942 4800 Acked Read 6 Sent Done Done false
13943 4801 Acked Read 10 Sent Done Done false
13944 4802 Acked Write 9 Sent Done Done false
13945 4803 Acked Read 10 Sent Done Done false
13946 4804 Acked Write 1 Sent Done Done false
13947 4805 Acked Read 2 Sent Done Done false
13948 4806 Acked Read 3 Sent Done Done false
13949 4807 Acked Read 1 Sent Done Done false
13950 4808 Acked Read 8 Sent Done Done false
13951 4809 Acked Read 10 Sent Done Done false
13952 4810 Acked Read 4 Sent Done Done false
13953 4811 Acked Read 6 Sent Done Done false
13954 4812 Acked Write 2 Sent Done Done false
13955 4813 Acked Read 4 Sent Done Done false
13956 4814 Acked Write 8 Sent Done Done false
13957 4815 Acked Read 2 Sent Done Done false
13958 4816 Acked Read 7 Sent Done Done false
13959 4817 Acked Read 4 Sent Done Done false
13960 4818 Acked Write 5 Sent Done Done false
13961 4819 Acked Write 9 Sent Done Done false
13962 4820 Acked Write 1 Sent Done Done false
13963 4821 Acked Write 2 Sent Done Done false
13964 4822 Acked Flush 0 Sent Done Done false
13965 4823 Acked Read 8 Sent Done Done false
13966 4824 Acked Flush 0 Sent Done Done false
13967 4825 Acked Read 5 Sent Done Done false
13968 4826 Acked Write 4 Sent Done Done false
13969 4827 Acked Write 7 Sent Done Done false
13970 4828 Acked Read 8 Sent Done Done false
13971 4829 Acked Read 10 Sent Done Done false
13972 4830 Acked Flush 0 Sent Done Done false
13973 4831 Acked Read 1 Sent Done Done false
13974 4832 Acked Read 7 Sent Done Done false
13975 4833 Acked Read 5 Sent Done Done false
13976 4834 Acked Read 7 Sent Done Done false
13977 4835 Acked Read 4 Sent Done Done false
13978 4836 Acked Write 3 Sent Done Done false
13979 4837 Acked Write 2 Sent Done Done false
13980 4838 Acked Read 2 Sent Done Done false
13981 4839 Acked Write 10 Sent Done Done false
13982 4840 Acked Write 2 Sent Done Done false
13983 4841 Acked Read 7 Sent Done Done false
13984 4842 Acked Write 1 Sent Done Done false
13985 4843 Acked Flush 0 Sent Done Done false
13986 4844 Acked Read 8 Sent Done Done false
13987 4845 Acked Write 7 Sent Done Done false
13988 4846 Acked Write 9 Sent Done Done false
13989 4847 Acked Write 10 Sent Done Done false
13990 4848 Acked Read 10 Sent Done Done false
13991 4849 Acked Read 6 Sent Done Done false
13992 4850 Acked Read 6 Sent Done Done false
13993 4851 Acked Read 7 Sent Done Done false
13994 4852 Acked Read 1 Sent Done Done false
13995 4853 Acked Write 10 Sent Done Done false
13996 4854 Acked Flush 0 Sent Done Done false
13997 4855 Acked Read 9 Sent Done Done false
13998 4856 Acked Read 2 Sent Done Done false
13999 4857 Acked Write 6 Sent Done Done false
14000 4858 Acked Write 8 Sent Done Done false
14001 4859 Acked Read 9 Sent Done Done false
14002 4860 Acked Write 4 Sent Done Done false
14003 4861 Acked Write 3 Sent Done Done false
14004 4862 Acked Read 8 Sent Done Done false
14005 4863 Acked Write 8 Sent Done Done false
14006 4864 Acked Write 6 Sent Done Done false
14007 4865 Acked Write 3 Sent Done Done false
14008 4866 Acked Write 7 Sent Done Done false
14009 4867 Acked Write 5 Sent Done Done false
14010 4868 Acked Write 7 Sent Done Done false
14011 4869 Acked Read 9 Sent Done Done false
14012 4870 Acked Read 2 Sent Done Done false
14013 4871 Acked Write 7 Sent Done Done false
14014 4872 Acked Write 6 Sent Done Done false
14015 4873 Acked Read 7 Sent Done Done false
14016 4874 Acked Write 5 Sent Done Done false
14017 4875 Acked Write 2 Sent Done Done false
14018 4876 Acked Write 4 Sent Done Done false
14019 4877 Acked Read 4 Sent Done Done false
14020 4878 Acked Read 5 Sent Done Done false
14021 4879 Acked Write 5 Sent Done Done false
14022 4880 Acked Read 2 Sent Done Done false
14023 4881 Acked Write 5 Sent Done Done false
14024 4882 Acked Flush 0 Sent Done Done false
14025 4883 Acked Read 4 Sent Done Done false
14026 4884 Acked Write 2 Sent Done Done false
14027 4885 Acked Read 6 Sent Done Done false
14028 4886 Acked Read 1 Sent Done Done false
14029 4887 Acked Read 3 Sent Done Done false
14030 4888 Acked Read 6 Sent Done Done false
14031 4889 Acked Write 5 Sent Done Done false
14032 4890 Acked Read 7 Sent Done Done false
14033 4891 Acked Write 8 Sent Done Done false
14034 4892 Acked Read 5 Sent Done Done false
14035 4893 Acked Write 2 Sent Done Done false
14036 4894 Acked Read 3 Sent Done Done false
14037 4895 Acked Flush 0 Sent Done Done false
14038 4896 Acked Flush 0 Sent Done Done false
14039 4897 Acked Read 8 Sent Done Done false
14040 4898 Acked Flush 0 Sent Done Done false
14041 4899 Acked Flush 0 Sent Done Done false
14042 4900 Acked Write 1 Sent Done Done false
14043 4901 Acked Write 4 Sent Done Done false
14044 4902 Acked Read 7 Sent Done Done false
14045 4903 Acked Read 5 Sent Done Done false
14046 4904 Acked Read 10 Sent Done Done false
14047 4905 Acked Flush 0 Sent Done Done false
14048 4906 Acked Read 3 Sent Done Done false
14049 4907 Acked Write 2 Sent Done Done false
14050 4908 Acked Write 4 Sent Done Done false
14051 4909 Acked Write 10 Sent Done Done false
14052 4910 Acked Write 7 Sent Done Done false
14053 4911 Acked Write 10 Sent Done Done false
14054 4912 Acked Write 6 Sent Done Done false
14055 4913 Acked Read 2 Sent Done Done false
14056 4914 Acked Read 6 Sent Done Done false
14057 4915 Acked Write 3 Sent Done Done false
14058 4916 Acked Read 1 Sent Done Done false
14059 4917 Acked Read 1 Sent Done Done false
14060 4918 Acked Write 5 Sent Done Done false
14061 4919 Acked Write 2 Sent Done Done false
14062 4920 Acked Write 1 Sent Done Done false
14063 4921 Acked Write 10 Sent Done Done false
14064 4922 Acked Flush 0 Sent Done Done false
14065 STATES DS:0 DS:1 DS:2 TOTAL
14066 Sent 302 0 0 302
14067 Done 0 302 302 604
14068 Skipped 0 0 0 0
14069 Error 0 0 0 0
14070 Last Flush: 4620 4922 4922
14071 Downstairs last five completed: 4620 4619 4618 4617 4616
14072 Upstairs last five completed: 4922 4921 4920 4919 4918
14073 CLIENT: Up:0 ds:302 act:2
140742025-01-27T19:39:17.887ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140752025-01-27T19:39:17.887ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58328) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140762025-01-27T19:39:17.888ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140772025-01-27T19:39:17.889ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140782025-01-27T19:39:17.889ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140792025-01-27T19:39:17.889ZINFOcrucible: send last flush ID to this DS: 4620 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140802025-01-27T19:39:17.889ZINFOcrucible: Replied this last flush ID: 4620 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140812025-01-27T19:39:17.889ZINFOcrucible: [0] client re-new 302 jobs since flush 4620 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140822025-01-27T19:39:17.890ZINFOcrucible: [0] Replayed 302 jobs since flush: 4620 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
14083 ----------------------------------------------------------------
14084 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
14085 Downstairs last five completed: 4922 4921 4920 4919 4918
14086 Upstairs last five completed: 4922 4921 4920 4919 4918
14087 CLIENT: Up:0 ds:0 act:3
14088 Replay: All jobs finished
14089 Main task has work to do, go find it
14090 got dsc Stop(0)
14091 stop 0
14092 Jan 27 19:39:20.129 INFO request completed[8810] Got stop action so:false kr:false
14093 , latency_us: 317, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 48abf409-74ce-4465-9a05-7aaacf83661d, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
140942025-01-27T19:39:20.130ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140952025-01-27T19:39:20.130ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
140962025-01-27T19:39:20.130ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
14097 Jan 27 19:39:20.130 INFO request completed, latency_us: 141, response_code: 200, uri: /state/cid/0, method: GET, req_id: 754bed8b-5c38-49d3-b6f9-06f619e9878e, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
140982025-01-27T19:39:20.130ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
14099 [8810] Exited with: ExitStatus(unix_wait_status(9))
14100 [8810][0] reports Exit
14101 Jan 27 19:39:24.132 INFO request completed, latency_us: 213, response_code: 200, uri: /state/cid/0, method: GET, req_id: bc870c15-9322-441b-9e92-e80023ea7b78, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
14102 001/300 Write block 397 len 12288 data: 5 4 4
14103 002/300 Write block 1763 len 32768 data: 2 3 3 2 2 3 3 3
14104 003/300 Write block 2702 len 8192 data: 2 2
14105 004/300 Read block 941 len 32768
14106 005/300 Read block 2027 len 24576
14107 006/300 Read block 1015 len 32768
14108 007/300 Write block 196 len 40960 data: 5 5 5 6 6 4 3 3 3 4
14109 008/300 Read block 1555 len 20480
14110 009/300 Read block 3680 len 28672
14111 010/300 Write block 3380 len 8192 data: 3 2
14112 011/300 Write block 114 len 24576 data: 4 5 5 4 4 4
14113 012/300 Write block 1244 len 20480 data: 2 2 2 2 2
14114 013/300 Read block 2780 len 16384
14115 014/300 Read block 495 len 8192
14116 015/300 Read block 2507 len 32768
14117 016/300 Write block 2120 len 4096 data: 2
14118 017/300 Read block 4053 len 8192
14119 018/300 Write block 375 len 24576 data: 4 4 4 3 3 3
14120 019/300 Write block 3365 len 36864 data: 3 3 5 5 5 5 5 6 4
14121 020/300 Read block 4537 len 4096
14122 021/300 Write block 4275 len 24576 data: 6 5 5 5 4 4
14123 022/300 Read block 4409 len 4096
14124 023/300 Read block 865 len 16384
14125 024/300 Write block 3553 len 40960 data: 2 2 2 3 3 3 5 5 5 6
14126 025/300 Write block 3262 len 12288 data: 3 3 3
14127 026/300 Write block 4161 len 24576 data: 5 5 5 5 6 5
14128 027/300 Read block 2869 len 24576
14129 028/300 Read block 4055 len 24576
14130 029/300 Read block 1999 len 28672
14131 030/300 Read block 389 len 28672
14132 031/300 Write block 2007 len 20480 data: 2 2 2 3 5
14133 032/300 Write block 157 len 40960 data: 3 3 3 3 2 2 2 3 3 4
14134 033/300 Read block 2950 len 36864
14135 034/300 Write block 2508 len 36864 data: 4 3 2 2 3 3 3 3 3
14136 035/300 Write block 1883 len 32768 data: 3 3 3 4 3 4 5 5
14137 036/300 Read block 455 len 16384
14138 037/300 Read block 837 len 12288
14139 038/300 Write block 1034 len 40960 data: 7 6 6 5 3 3 2 2 3 3
14140 039/300 Write block 2549 len 36864 data: 2 3 3 3 3 3 3 3 4
14141 040/300 Read block 1272 len 36864
14142 041/300 Read block 2642 len 12288
14143 042/300 Write block 2072 len 32768 data: 2 2 2 2 4 4 4 4
14144 043/300 Read block 4079 len 24576
14145 044/300 Write block 2002 len 4096 data: 4
14146 045/300 Read block 2460 len 28672
14147 046/300 Read block 1003 len 4096
14148 047/300 Read block 3422 len 4096
14149 048/300 Write block 3644 len 4096 data: 3
14150 049/300 Read block 2105 len 40960
14151 050/300 Read block 2215 len 16384
14152 051/300 Write block 3256 len 36864 data: 2 2 2 2 2 3 4 4 4
14153 052/300 Read block 3035 len 24576
14154 053/300 Write block 1065 len 40960 data: 2 2 2 4 3 4 4 6 5 5
14155 054/300 Read block 1254 len 12288
14156 055/300 Read block 2256 len 8192
14157 056/300 Flush
14158 057/300 Write block 1355 len 20480 data: 5 5 5 7 7
14159 058/300 Read block 4667 len 40960
14160 059/300 Read block 1923 len 16384
14161 060/300 Write block 2806 len 36864 data: 3 3 3 3 3 4 4 4 4
14162 061/300 Read block 3925 len 36864
14163 062/300 Read block 1404 len 12288
14164 063/300 Write block 3705 len 32768 data: 3 3 4 6 6 6 7 6
14165 064/300 Flush
14166 065/300 Write block 4207 len 20480 data: 3 3 3 3 3
14167 066/300 Write block 278 len 4096 data: 6
14168 067/300 Write block 460 len 8192 data: 3 3
14169 068/300 Write block 627 len 32768 data: 3 3 3 4 3 3 2 2
14170 069/300 Write block 2049 len 12288 data: 3 4 4
14171 070/300 Read block 4110 len 20480
14172 071/300 Read block 2900 len 8192
14173 072/300 Read block 330 len 12288
14174 073/300 Read block 195 len 8192
14175 074/300 Read block 2090 len 20480
14176 075/300 Read block 4228 len 4096
14177 076/300 Read block 3218 len 8192
14178 077/300 Read block 564 len 24576
14179 078/300 Write block 4349 len 28672 data: 5 5 5 5 5 4 5
14180 079/300 Read block 3588 len 36864
14181 080/300 Read block 4648 len 4096
14182 081/300 Write block 2125 len 40960 data: 3 2 2 2 2 2 2 2 2 2
14183 082/300 Flush
14184 083/300 Read block 2539 len 28672
14185 084/300 Read block 1127 len 32768
14186 085/300 Read block 3611 len 20480
14187 086/300 Read block 4918 len 16384
14188 087/300 Write block 2885 len 20480 data: 3 3 3 3 3
14189 088/300 Write block 3618 len 28672 data: 4 4 4 5 5 5 5
14190 089/300 Read block 1751 len 8192
14191 090/300 Read block 2000 len 36864
14192 091/300 Read block 1560 len 28672
14193 092/300 Flush
14194 093/300 Read block 3574 len 24576
14195 094/300 Read block 1523 len 36864
14196 095/300 Write block 3962 len 32768 data: 4 5 4 6 5 5 5 4
14197 096/300 Read block 2326 len 8192
14198 097/300 Write block 776 len 28672 data: 4 4 4 3 3 4 4
14199 098/300 Flush
14200 099/300 Read block 4636 len 24576
14201 100/300 Read block 22 len 20480
14202 101/300 Write block 2088 len 12288 data: 2 2 2
14203 102/300 Write block 130 len 40960 data: 4 5 4 3 2 2 2 3 5 5
14204 103/300 Write block 2685 len 16384 data: 6 6 5 4
14205 104/300 Write block 4345 len 32768 data: 3 3 4 5 6 6 6 6
14206 105/300 Read block 2491 len 20480
14207 106/300 Read block 1577 len 24576
14208 107/300 Read block 2834 len 16384
14209 108/300 Write block 94 len 36864 data: 2 2 3 3 3 3 3 3 4
14210 109/300 Write block 4119 len 36864 data: 6 5 5 5 4 2 2 2 3
14211 110/300 Write block 564 len 4096 data: 5
14212 111/300 Read block 580 len 16384
14213 112/300 Read block 3984 len 36864
14214 113/300 Read block 1519 len 36864
14215 114/300 Read block 4178 len 12288
14216 115/300 Write block 4886 len 28672 data: 3 4 4 3 3 2 2
14217 116/300 Read block 3879 len 16384
14218 117/300 Write block 1383 len 40960 data: 5 5 3 3 3 4 4 4 5 4
14219 118/300 Write block 1865 len 4096 data: 3
14220 119/300 Write block 309 len 32768 data: 2 2 2 3 3 4 4 4
14221 120/300 Read block 2143 len 40960
14222 121/300 Read block 3541 len 8192
14223 122/300 Read block 4598 len 28672
14224 123/300 Write block 3724 len 40960 data: 4 5 5 6 5 5 4 3 3 3
14225 124/300 Write block 1673 len 36864 data: 4 6 5 5 3 3 3 2 2
14226 125/300 Flush
14227 126/300 Read block 3417 len 12288
14228 127/300 Read block 1044 len 32768
14229 128/300 Read block 3891 len 4096
14230 129/300 Write block 891 len 20480 data: 3 3 2 2 2
14231 130/300 Write block 3176 len 36864 data: 6 6 6 6 4 4 4 3 3
14232 131/300 Read block 1448 len 4096
14233 132/300 Read block 2551 len 16384
14234 133/300 Write block 743 len 32768 data: 5 4 4 4 4 4 4 3
14235 134/300 Write block 4513 len 24576 data: 5 6 4 4 4 4
14236 135/300 Write block 78 len 12288 data: 4 4 3
14237 136/300 Read block 3696 len 4096
14238 137/300 Read block 92 len 4096
14239 138/300 Write block 946 len 32768 data: 3 3 2 2 2 2 3 3
14240 139/300 Write block 3860 len 8192 data: 2 3
14241 140/300 Write block 519 len 8192 data: 3 3
14242 141/300 Write block 4613 len 8192 data: 4 3
14243 142/300 Read block 4301 len 20480
14244 143/300 Write block 3469 len 4096 data: 4
14245 144/300 Write block 495 len 28672 data: 4 4 6 7 7 5 6
14246 145/300 Read block 2835 len 4096
14247 146/300 Write block 1335 len 16384 data: 4 4 4 5
14248 147/300 Read block 3330 len 40960
14249 148/300 Read block 2837 len 36864
14250 149/300 Read block 2169 len 8192
14251 150/300 Flush
14252 151/300 Write block 1185 len 28672 data: 3 2 2 2 3 4 4
14253 152/300 Write block 4712 len 12288 data: 4 5 5
14254 153/300 Write block 79 len 20480 data: 5 4 3 3 4
14255 154/300 Read block 1418 len 36864
14256 155/300 Read block 3701 len 20480
14257 156/300 Write block 4040 len 36864 data: 3 4 5 6 5 5 5 4 3
14258 157/300 Flush
14259 158/300 Read block 4103 len 12288
14260 159/300 Write block 2133 len 8192 data: 3 3
14261 160/300 Read block 1402 len 36864
14262 161/300 Read block 4251 len 28672
14263 162/300 Read block 363 len 28672
14264 163/300 Write block 3606 len 12288 data: 6 6 6
14265 164/300 Read block 665 len 4096
14266 165/300 Read block 4310 len 20480
14267 166/300 Flush
14268 167/300 Read block 3532 len 4096
14269 168/300 Read block 1967 len 20480
14270 169/300 Read block 2260 len 12288
14271 170/300 Read block 3433 len 28672
14272 171/300 Read block 1908 len 32768
14273 172/300 Read block 187 len 16384
14274 173/300 Read block 559 len 32768
14275 174/300 Write block 3461 len 32768 data: 3 4 3 4 5 5 5 4
14276 175/300 Write block 4916 len 36864 data: 4 3 2 2 2 2 5 6 5
14277 176/300 Write block 1751 len 4096 data: 4
14278 177/300 Read block 1816 len 8192
14279 178/300 Read block 3488 len 4096
14280 179/300 Read block 1448 len 32768
14281 180/300 Flush
14282 181/300 Read block 2948 len 12288
14283 182/300 Write block 192 len 8192 data: 4 4
14284 183/300 Write block 1234 len 12288 data: 6 6 6
14285 184/300 Read block 683 len 12288
14286 185/300 Read block 2305 len 32768
14287 186/300 Write block 4750 len 28672 data: 3 2 2 2 2 3 2
14288 187/300 Read block 1948 len 8192
14289 188/300 Write block 1738 len 28672 data: 3 4 4 4 3 3 3
14290 189/300 Read block 1711 len 36864
14291 190/300 Flush
14292 191/300 Write block 3004 len 36864 data: 3 3 3 3 3 3 2 2 2
14293 192/300 Write block 3623 len 16384 data: 6 6 3 2
14294 193/300 Read block 1240 len 12288
14295 194/300 Flush
14296 195/300 Read block 1733 len 28672
14297 196/300 Write block 1276 len 36864 data: 2 2 2 2 2 2 2 2 2
14298 197/300 Read block 1372 len 28672
14299 198/300 Read block 2388 len 20480
14300 199/300 Read block 3316 len 4096
14301 200/300 Flush
14302 201/300 Read block 4745 len 20480
14303 202/300 Read block 4060 len 4096
14304 203/300 Read block 3405 len 4096
14305 204/300 Write block 420 len 28672 data: 4 4 4 4 4 4 4
14306 205/300 Read block 3524 len 4096
14307 206/300 Read block 2458 len 16384
14308 207/300 Read block 1597 len 36864
14309 208/300 Write block 2014 len 16384 data: 4 4 4 4
14310 209/300 Read block 4609 len 20480
14311 210/300 Read block 3042 len 36864
14312 211/300 Read block 4045 len 40960
14313 212/300 Flush
14314 213/300 Read block 2537 len 12288
14315 214/300 Flush
14316 215/300 Read block 4255 len 32768
14317 216/300 Read block 4878 len 28672
14318 217/300 Read block 2915 len 24576
14319 218/300 Read block 2708 len 12288
14320 219/300 Read block 2111 len 28672
14321 220/300 Read block 4096 len 12288
14322 221/300 Write block 4383 len 24576 data: 5 4 4 4 4 3
14323 222/300 Read block 3028 len 28672
14324 223/300 Write block 2269 len 20480 data: 6 3 3 2 2
14325 224/300 Read block 1401 len 40960
14326 225/300 Write block 1117 len 28672 data: 3 3 2 3 3 5 6
14327 226/300 Read block 4496 len 32768
14328 227/300 Write block 3451 len 20480 data: 4 4 5 5 7
14329 228/300 Write block 815 len 28672 data: 3 3 3 3 4 4 3
14330 229/300 Write block 2320 len 12288 data: 2 2 2
14331 230/300 Write block 1368 len 32768 data: 4 4 5 4 3 5 5 6
14332 231/300 Write block 1815 len 8192 data: 6 5
14333 232/300 Read block 4094 len 36864
14334 233/300 Read block 1670 len 28672
14335 234/300 Write block 4835 len 32768 data: 3 3 3 4 3 3 3 4
14336 235/300 Read block 2857 len 20480
14337 236/300 Read block 2218 len 40960
14338 237/300 Read block 2157 len 20480
14339 238/300 Read block 2772 len 20480
14340 239/300 Read block 2239 len 32768
14341 240/300 Write block 4173 len 32768 data: 5 4 4 4 4 4 2 2
14342 241/300 Read block 2566 len 24576
14343 242/300 Write block 2032 len 12288 data: 4 3 4
14344 243/300 Write block 453 len 8192 data: 3 3
14345 244/300 Read block 2561 len 20480
14346 245/300 Write block 168 len 20480 data: 6 6 6 6 4
14347 246/300 Write block 4348 len 8192 data: 6 7
14348 247/300 Write block 505 len 12288 data: 4 4 3
14349 248/300 Read block 1485 len 16384
14350 249/300 Read block 4509 len 12288
14351 250/300 Read block 1402 len 4096
14352 251/300 Read block 3015 len 32768
14353 252/300 Write block 4834 len 24576 data: 3 4 4 4 5 4
14354 253/300 Write block 3428 len 16384 data: 6 6 6 6
14355 254/300 Write block 1596 len 4096 data: 3
14356 255/300 Write block 4870 len 12288 data: 5 4 4
14357 256/300 Write block 707 len 20480 data: 4 5 4 4 5
14358 257/300 Read block 392 len 28672
14359 258/300 Read block 2250 len 40960
14360 259/300 Read block 361 len 12288
14361 260/300 Read block 2603 len 8192
14362 261/300 Read block 4743 len 4096
14363 262/300 Read block 755 len 28672
14364 263/300 Write block 4480 len 4096 data: 5
14365 264/300 Write block 2201 len 36864 data: 6 5 6 4 5 4 4 4 4
14366 265/300 Read block 1549 len 8192
14367 266/300 Write block 2106 len 20480 data: 2 2 2 2 2
14368 267/300 Write block 1815 len 12288 data: 7 6 5
14369 268/300 Read block 1138 len 40960
14370 269/300 Read block 365 len 28672
14371 270/300 Write block 4935 len 24576 data: 2 2 2 3 3 3
14372 271/300 Read block 3163 len 4096
14373 272/300 Write block 1204 len 28672 data: 5 5 5 5 4 4 3
14374 273/300 Read block 1266 len 28672
14375 274/300 Write block 4489 len 24576 data: 3 3 3 5 5 5
14376 275/300 Read block 2636 len 4096
14377 276/300 Write block 451 len 8192 data: 3 3
14378 277/300 Read block 1270 len 4096
14379 278/300 Read block 1534 len 40960
14380 279/300 Write block 2036 len 36864 data: 3 3 3 2 2 3 4 4 3
14381 280/300 Read block 4921 len 32768
14382 281/300 Read block 1687 len 32768
14383 282/300 Write block 46 len 36864 data: 4 4 5 5 5 4 4 3 4
14384 283/300 Write block 3628 len 36864 data: 2 2 2 2 2 3 3 5 5
14385 284/300 Read block 4148 len 28672
14386 285/300 Write block 4690 len 36864 data: 2 2 2 3 3 3 3 3 3
14387 286/300 Write block 1344 len 24576 data: 5 5 5 5 5 4
14388 287/300 Read block 2972 len 20480
14389 288/300 Read block 3249 len 16384
14390 289/300 Read block 1545 len 12288
14391 290/300 Read block 2504 len 36864
14392 291/300 Flush
14393 292/300 Read block 1062 len 36864
14394 293/300 Read block 2952 len 28672
14395 294/300 Write block 4805 len 16384 data: 6 3 4 4
14396 295/300 Flush
14397 296/300 Read block 1608 len 24576
14398 297/300 Read block 3804 len 4096
14399 298/300 Read block 117 len 12288
14400 299/300 Write block 2925 len 40960 data: 2 2 3 5 5 4 3 3 4 4
14401 300/300 Write block 2936 len 32768 data: 6 4 3 3 3 3 3 2
14402 301/300 Read block 1686 len 28672
14403 Main task has work to do, go find it
14404 got dsc Start(0)
14405 start 0
14406 Jan 27 19:39:24.372 INFO request completed, latency_us: 185, response_code: 204, uri: /start/cid/0, method: POST, req_id: ad283adf-1532-4024-bdd0-9bdf6e832c02, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
14407 Replay: started 0, returned:Ok(())
14408 ----------------------------------------------------------------
14409 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
14410 States: Offline Active Active
14411 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14412 4923 Acked Write 3 Sent Done Done false
14413 4924 Acked Write 8 Sent Done Done false
14414 4925 Acked Write 2 Sent Done Done false
14415 4926 Acked Read 8 Sent Done Done false
14416 4927 Acked Read 6 Sent Done Done false
14417 4928 Acked Read 8 Sent Done Done false
14418 4929 Acked Write 10 Sent Done Done false
14419 4930 Acked Read 5 Sent Done Done false
14420 4931 Acked Read 7 Sent Done Done false
14421 4932 Acked Write 2 Sent Done Done false
14422 4933 Acked Write 6 Sent Done Done false
14423 4934 Acked Write 5 Sent Done Done false
14424 4935 Acked Read 4 Sent Done Done false
14425 4936 Acked Read 2 Sent Done Done false
14426 4937 Acked Read 8 Sent Done Done false
14427 4938 Acked Write 1 Sent Done Done false
14428 4939 Acked Read 2 Sent Done Done false
14429 4940 Acked Write 6 Sent Done Done false
14430 4941 Acked Write 9 Sent Done Done false
14431 4942 Acked Read 1 Sent Done Done false
14432 4943 Acked Write 6 Sent Done Done false
14433 4944 Acked Read 1 Sent Done Done false
14434 4945 Acked Read 4 Sent Done Done false
14435 4946 Acked Write 10 Sent Done Done false
14436 4947 Acked Write 3 Sent Done Done false
14437 4948 Acked Write 6 Sent Done Done false
14438 4949 Acked Read 6 Sent Done Done false
14439 4950 Acked Read 6 Sent Done Done false
14440 4951 Acked Read 7 Sent Done Done false
14441 4952 Acked Read 7 Sent Done Done false
14442 4953 Acked Write 5 Sent Done Done false
14443 4954 Acked Write 10 Sent Done Done false
14444 4955 Acked Read 9 Sent Done Done false
14445 4956 Acked Write 9 Sent Done Done false
14446 4957 Acked Write 8 Sent Done Done false
14447 4958 Acked Read 4 Sent Done Done false
14448 4959 Acked Read 3 Sent Done Done false
14449 4960 Acked Write 10 Sent Done Done false
14450 4961 Acked Write 9 Sent Done Done false
14451 4962 Acked Read 9 Sent Done Done false
14452 4963 Acked Read 3 Sent Done Done false
14453 4964 Acked Write 8 Sent Done Done false
14454 4965 Acked Read 6 Sent Done Done false
14455 4966 Acked Write 1 Sent Done Done false
14456 4967 Acked Read 7 Sent Done Done false
14457 4968 Acked Read 1 Sent Done Done false
14458 4969 Acked Read 1 Sent Done Done false
14459 4970 Acked Write 1 Sent Done Done false
14460 4971 Acked Read 10 Sent Done Done false
14461 4972 Acked Read 4 Sent Done Done false
14462 4973 Acked Write 9 Sent Done Done false
14463 4974 Acked Read 6 Sent Done Done false
14464 4975 Acked Write 10 Sent Done Done false
14465 4976 Acked Read 3 Sent Done Done false
14466 4977 Acked Read 2 Sent Done Done false
14467 4978 Acked Flush 0 Sent Done Done false
14468 4979 Acked Write 5 Sent Done Done false
14469 4980 Acked Read 10 Sent Done Done false
14470 4981 Acked Read 4 Sent Done Done false
14471 4982 Acked Write 9 Sent Done Done false
14472 4983 Acked Read 9 Sent Done Done false
14473 4984 Acked Read 3 Sent Done Done false
14474 4985 Acked Write 8 Sent Done Done false
14475 4986 Acked Flush 0 Sent Done Done false
14476 4987 Acked Write 5 Sent Done Done false
14477 4988 Acked Write 1 Sent Done Done false
14478 4989 Acked Write 2 Sent Done Done false
14479 4990 Acked Write 8 Sent Done Done false
14480 4991 Acked Write 3 Sent Done Done false
14481 4992 Acked Read 5 Sent Done Done false
14482 4993 Acked Read 2 Sent Done Done false
14483 4994 Acked Read 3 Sent Done Done false
14484 4995 Acked Read 2 Sent Done Done false
14485 4996 Acked Read 5 Sent Done Done false
14486 4997 Acked Read 1 Sent Done Done false
14487 4998 Acked Read 2 Sent Done Done false
14488 4999 Acked Read 6 Sent Done Done false
14489 5000 Acked Write 7 Sent Done Done false
14490 5001 Acked Read 9 Sent Done Done false
14491 5002 Acked Read 1 Sent Done Done false
14492 5003 Acked Write 10 Sent Done Done false
14493 5004 Acked Flush 0 Sent Done Done false
14494 5005 Acked Read 7 Sent Done Done false
14495 5006 Acked Read 8 Sent Done Done false
14496 5007 Acked Read 5 Sent Done Done false
14497 5008 Acked Read 4 Sent Done Done false
14498 5009 Acked Write 5 Sent Done Done false
14499 5010 Acked Write 7 Sent Done Done false
14500 5011 Acked Read 2 Sent Done Done false
14501 5012 Acked Read 9 Sent Done Done false
14502 5013 Acked Read 7 Sent Done Done false
14503 5014 Acked Flush 0 Sent Done Done false
14504 5015 Acked Read 6 Sent Done Done false
14505 5016 Acked Read 9 Sent Done Done false
14506 5017 Acked Write 8 Sent Done Done false
14507 5018 Acked Read 2 Sent Done Done false
14508 5019 Acked Write 7 Sent Done Done false
14509 5020 Acked Flush 0 Sent Done Done false
14510 5021 Acked Read 6 Sent Done Done false
14511 5022 Acked Read 5 Sent Done Done false
14512 5023 Acked Write 3 Sent Done Done false
14513 5024 Acked Write 10 Sent Done Done false
14514 5025 Acked Write 4 Sent Done Done false
14515 5026 Acked Write 8 Sent Done Done false
14516 5027 Acked Read 5 Sent Done Done false
14517 5028 Acked Read 6 Sent Done Done false
14518 5029 Acked Read 4 Sent Done Done false
14519 5030 Acked Write 9 Sent Done Done false
14520 5031 Acked Write 9 Sent Done Done false
14521 5032 Acked Write 1 Sent Done Done false
14522 5033 Acked Read 4 Sent Done Done false
14523 5034 Acked Read 9 Sent Done Done false
14524 5035 Acked Read 9 Sent Done Done false
14525 5036 Acked Read 3 Sent Done Done false
14526 5037 Acked Write 7 Sent Done Done false
14527 5038 Acked Read 4 Sent Done Done false
14528 5039 Acked Write 10 Sent Done Done false
14529 5040 Acked Write 1 Sent Done Done false
14530 5041 Acked Write 8 Sent Done Done false
14531 5042 Acked Read 10 Sent Done Done false
14532 5043 Acked Read 2 Sent Done Done false
14533 5044 Acked Read 7 Sent Done Done false
14534 5045 Acked Write 10 Sent Done Done false
14535 5046 Acked Write 9 Sent Done Done false
14536 5047 Acked Flush 0 Sent Done Done false
14537 5048 Acked Read 3 Sent Done Done false
14538 5049 Acked Read 8 Sent Done Done false
14539 5050 Acked Read 1 Sent Done Done false
14540 5051 Acked Write 5 Sent Done Done false
14541 5052 Acked Write 9 Sent Done Done false
14542 5053 Acked Read 1 Sent Done Done false
14543 5054 Acked Read 4 Sent Done Done false
14544 5055 Acked Write 8 Sent Done Done false
14545 5056 Acked Write 6 Sent Done Done false
14546 5057 Acked Write 3 Sent Done Done false
14547 5058 Acked Read 1 Sent Done Done false
14548 5059 Acked Read 1 Sent Done Done false
14549 5060 Acked Write 8 Sent Done Done false
14550 5061 Acked Write 2 Sent Done Done false
14551 5062 Acked Write 2 Sent Done Done false
14552 5063 Acked Write 2 Sent Done Done false
14553 5064 Acked Read 5 Sent Done Done false
14554 5065 Acked Write 1 Sent Done Done false
14555 5066 Acked Write 7 Sent Done Done false
14556 5067 Acked Read 1 Sent Done Done false
14557 5068 Acked Write 4 Sent Done Done false
14558 5069 Acked Read 10 Sent Done Done false
14559 5070 Acked Read 9 Sent Done Done false
14560 5071 Acked Read 2 Sent Done Done false
14561 5072 Acked Flush 0 Sent Done Done false
14562 5073 Acked Write 7 Sent Done Done false
14563 5074 Acked Write 3 Sent Done Done false
14564 5075 Acked Write 5 Sent Done Done false
14565 5076 Acked Read 9 Sent Done Done false
14566 5077 Acked Read 5 Sent Done Done false
14567 5078 Acked Write 9 Sent Done Done false
14568 5079 Acked Flush 0 Sent Done Done false
14569 5080 Acked Read 3 Sent Done Done false
14570 5081 Acked Write 2 Sent Done Done false
14571 5082 Acked Read 9 Sent Done Done false
14572 5083 Acked Read 7 Sent Done Done false
14573 5084 Acked Read 7 Sent Done Done false
14574 5085 Acked Write 3 Sent Done Done false
14575 5086 Acked Read 1 Sent Done Done false
14576 5087 Acked Read 5 Sent Done Done false
14577 5088 Acked Flush 0 Sent Done Done false
14578 5089 Acked Read 1 Sent Done Done false
14579 5090 Acked Read 5 Sent Done Done false
14580 5091 Acked Read 3 Sent Done Done false
14581 5092 Acked Read 7 Sent Done Done false
14582 5093 Acked Read 8 Sent Done Done false
14583 5094 Acked Read 4 Sent Done Done false
14584 5095 Acked Read 8 Sent Done Done false
14585 5096 Acked Write 8 Sent Done Done false
14586 5097 Acked Write 9 Sent Done Done false
14587 5098 Acked Write 1 Sent Done Done false
14588 5099 Acked Read 2 Sent Done Done false
14589 5100 Acked Read 1 Sent Done Done false
14590 5101 Acked Read 8 Sent Done Done false
14591 5102 Acked Flush 0 Sent Done Done false
14592 5103 Acked Read 3 Sent Done Done false
14593 5104 Acked Write 2 Sent Done Done false
14594 5105 Acked Write 3 Sent Done Done false
14595 5106 Acked Read 3 Sent Done Done false
14596 5107 Acked Read 8 Sent Done Done false
14597 5108 Acked Write 7 Sent Done Done false
14598 5109 Acked Read 2 Sent Done Done false
14599 5110 Acked Write 7 Sent Done Done false
14600 5111 Acked Read 9 Sent Done Done false
14601 5112 Acked Flush 0 Sent Done Done false
14602 5113 Acked Write 9 Sent Done Done false
14603 5114 Acked Write 4 Sent Done Done false
14604 5115 Acked Read 3 Sent Done Done false
14605 5116 Acked Flush 0 Sent Done Done false
14606 5117 Acked Read 7 Sent Done Done false
14607 5118 Acked Write 9 Sent Done Done false
14608 5119 Acked Read 7 Sent Done Done false
14609 5120 Acked Read 5 Sent Done Done false
14610 5121 Acked Read 1 Sent Done Done false
14611 5122 Acked Flush 0 Sent Done Done false
14612 5123 Acked Read 5 Sent Done Done false
14613 5124 Acked Read 1 Sent Done Done false
14614 5125 Acked Read 1 Sent Done Done false
14615 5126 Acked Write 7 Sent Done Done false
14616 5127 Acked Read 1 Sent Done Done false
14617 5128 Acked Read 4 Sent Done Done false
14618 5129 Acked Read 9 Sent Done Done false
14619 5130 Acked Write 4 Sent Done Done false
14620 5131 Acked Read 5 Sent Done Done false
14621 5132 Acked Read 9 Sent Done Done false
14622 5133 Acked Read 10 Sent Done Done false
14623 5134 Acked Flush 0 Sent Done Done false
14624 5135 Acked Read 3 Sent Done Done false
14625 5136 Acked Flush 0 Sent Done Done false
14626 5137 Acked Read 8 Sent Done Done false
14627 5138 Acked Read 7 Sent Done Done false
14628 5139 Acked Read 6 Sent Done Done false
14629 5140 Acked Read 3 Sent Done Done false
14630 5141 Acked Read 7 Sent Done Done false
14631 5142 Acked Read 3 Sent Done Done false
14632 5143 Acked Write 6 Sent Done Done false
14633 5144 Acked Read 7 Sent Done Done false
14634 5145 Acked Write 5 Sent Done Done false
14635 5146 Acked Read 10 Sent Done Done false
14636 5147 Acked Write 7 Sent Done Done false
14637 5148 Acked Read 8 Sent Done Done false
14638 5149 Acked Write 5 Sent Done Done false
14639 5150 Acked Write 7 Sent Done Done false
14640 5151 Acked Write 3 Sent Done Done false
14641 5152 Acked Write 8 Sent Done Done false
14642 5153 Acked Write 2 Sent Done Done false
14643 5154 Acked Read 9 Sent Done Done false
14644 5155 Acked Read 7 Sent Done Done false
14645 5156 Acked Write 8 Sent Done Done false
14646 5157 Acked Read 5 Sent Done Done false
14647 5158 Acked Read 10 Sent Done Done false
14648 5159 Acked Read 5 Sent Done Done false
14649 5160 Acked Read 5 Sent Done Done false
14650 5161 Acked Read 8 Sent Done Done false
14651 5162 Acked Write 8 Sent Done Done false
14652 5163 Acked Read 6 Sent Done Done false
14653 5164 Acked Write 3 Sent Done Done false
14654 5165 Acked Write 2 Sent Done Done false
14655 5166 Acked Read 5 Sent Done Done false
14656 5167 Acked Write 5 Sent Done Done false
14657 5168 Acked Write 2 Sent Done Done false
14658 5169 Acked Write 3 Sent Done Done false
14659 5170 Acked Read 4 Sent Done Done false
14660 5171 Acked Read 3 Sent Done Done false
14661 5172 Acked Read 1 Sent Done Done false
14662 5173 Acked Read 8 Sent Done Done false
14663 5174 Acked Write 6 Sent Done Done false
14664 5175 Acked Write 4 Sent Done Done false
14665 5176 Acked Write 1 Sent Done Done false
14666 5177 Acked Write 3 Sent Done Done false
14667 5178 Acked Write 5 Sent Done Done false
14668 5179 Acked Read 7 Sent Done Done false
14669 5180 Acked Read 10 Sent Done Done false
14670 5181 Acked Read 3 Sent Done Done false
14671 5182 Acked Read 2 Sent Done Done false
14672 5183 Acked Read 1 Sent Done Done false
14673 5184 Acked Read 7 Sent Done Done false
14674 5185 Acked Write 1 Sent Done Done false
14675 5186 Acked Write 9 Sent Done Done false
14676 5187 Acked Read 2 Sent Done Done false
14677 5188 Acked Write 5 Sent Done Done false
14678 5189 Acked Write 3 Sent Done Done false
14679 5190 Acked Read 10 Sent Done Done false
14680 5191 Acked Read 7 Sent Done Done false
14681 5192 Acked Write 6 Sent Done Done false
14682 5193 Acked Read 1 Sent Done Done false
14683 5194 Acked Write 7 Sent Done Done false
14684 5195 Acked Read 7 Sent Done Done false
14685 5196 Acked Write 6 Sent Done Done false
14686 5197 Acked Read 1 Sent Done Done false
14687 5198 Acked Write 2 Sent Done Done false
14688 5199 Acked Read 1 Sent Done Done false
14689 5200 Acked Read 10 Sent Done Done false
14690 5201 Acked Write 9 Sent Done Done false
14691 5202 Acked Read 8 Sent Done Done false
14692 5203 Acked Read 8 Sent Done Done false
14693 5204 Acked Write 9 Sent Done Done false
14694 5205 Acked Write 9 Sent Done Done false
14695 5206 Acked Read 7 Sent Done Done false
14696 5207 Acked Write 9 Sent Done Done false
14697 5208 Acked Write 6 Sent Done Done false
14698 5209 Acked Read 5 Sent Done Done false
14699 5210 Acked Read 4 Sent Done Done false
14700 5211 Acked Read 3 Sent Done Done false
14701 5212 Acked Read 9 Sent Done Done false
14702 5213 Acked Flush 0 Sent Done Done false
14703 5214 Acked Read 9 Sent Done Done false
14704 5215 Acked Read 7 Sent Done Done false
14705 5216 Acked Write 4 Sent Done Done false
14706 5217 Acked Flush 0 Sent Done Done false
14707 5218 Acked Read 6 Sent Done Done false
14708 5219 Acked Read 1 Sent Done Done false
14709 5220 Acked Read 3 Sent Done Done false
14710 5221 Acked Write 10 Sent Done Done false
14711 5222 Acked Write 8 Sent Done Done false
14712 5223 Acked Read 7 Sent Done Done false
14713 STATES DS:0 DS:1 DS:2 TOTAL
14714 Sent 301 0 0 301
14715 Done 0 301 301 602
14716 Skipped 0 0 0 0
14717 Error 0 0 0 0
14718 Last Flush: 4922 5217 5217
14719 Downstairs last five completed: 4922 4921 4920 4919 4918
14720 Upstairs last five completed: 5223 5222 5221 5220 5219
14721 CLIENT: Up:0 ds:301 act:2
14722 ----------------------------------------------------------------
14723 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
14724 States: Offline Active Active
14725 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
14726 4923 Acked Write 3 Sent Done Done false
14727 4924 Acked Write 8 Sent Done Done false
14728 4925 Acked Write 2 Sent Done Done false
14729 4926 Acked Read 8 Sent Done Done false
14730 4927 Acked Read 6 Sent Done Done false
14731 4928 Acked Read 8 Sent Done Done false
14732 4929 Acked Write 10 Sent Done Done false
14733 4930 Acked Read 5 Sent Done Done false
14734 4931 Acked Read 7 Sent Done Done false
14735 4932 Acked Write 2 Sent Done Done false
14736 4933 Acked Write 6 Sent Done Done false
14737 4934 Acked Write 5 Sent Done Done false
14738 4935 Acked Read 4 Sent Done Done false
14739 4936 Acked Read 2 Sent Done Done false
14740 4937 Acked Read 8 Sent Done Done false
14741 4938 Acked Write 1 Sent Done Done false
14742 4939 Acked Read 2 Sent Done Done false
14743 4940 Acked Write 6 Sent Done Done false
14744 4941 Acked Write 9 Sent Done Done false
14745 4942 Acked Read 1 Sent Done Done false
14746 4943 Acked Write 6 Sent Done Done false
14747 4944 Acked Read 1 Sent Done Done false
14748 4945 Acked Read 4 Sent Done Done false
14749 4946 Acked Write 10 Sent Done Done false
14750 4947 Acked Write 3 Sent Done Done false
14751 4948 Acked Write 6 Sent Done Done false
14752 4949 Acked Read 6 Sent Done Done false
14753 4950 Acked Read 6 Sent Done Done false
14754 4951 Acked Read 7 Sent Done Done false
14755 4952 Acked Read 7 Sent Done Done false
14756 4953 Acked Write 5 Sent Done Done false
14757 4954 Acked Write 10 Sent Done Done false
14758 4955 Acked Read 9 Sent Done Done false
14759 4956 Acked Write 9 Sent Done Done false
14760 4957 Acked Write 8 Sent Done Done false
14761 4958 Acked Read 4 Sent Done Done false
14762 4959 Acked Read 3 Sent Done Done false
14763 4960 Acked Write 10 Sent Done Done false
14764 4961 Acked Write 9 Sent Done Done false
14765 4962 Acked Read 9 Sent Done Done false
14766 4963 Acked Read 3 Sent Done Done false
14767 4964 Acked Write 8 Sent Done Done false
14768 4965 Acked Read 6 Sent Done Done false
14769 4966 Acked Write 1 Sent Done Done false
14770 4967 Acked Read 7 Sent Done Done false
14771 4968 Acked Read 1 Sent Done Done false
14772 4969 Acked Read 1 Sent Done Done false
14773 4970 Acked Write 1 Sent Done Done false
14774 4971 Acked Read 10 Sent Done Done false
14775 4972 Acked Read 4 Sent Done Done false
14776 4973 Acked Write 9 Sent Done Done false
14777 4974 Acked Read 6 Sent Done Done false
14778 4975 Acked Write 10 Sent Done Done false
14779 4976 Acked Read 3 Sent Done Done false
14780 4977 Acked Read 2 Sent Done Done false
14781 4978 Acked Flush 0 Sent Done Done false
14782 4979 Acked Write 5 Sent Done Done false
14783 4980 Acked Read 10 Sent Done Done false
14784 4981 Acked Read 4 Sent Done Done false
14785 4982 Acked Write 9 Sent Done Done false
14786 4983 Acked Read 9 Sent Done Done false
14787 4984 Acked Read 3 Sent Done Done false
14788 4985 Acked Write 8 Sent Done Done false
14789 4986 Acked Flush 0 Sent Done Done false
14790 4987 Acked Write 5 Sent Done Done false
14791 4988 Acked Write 1 Sent Done Done false
14792 4989 Acked Write 2 Sent Done Done false
14793 4990 Acked Write 8 Sent Done Done false
14794 4991 Acked Write 3 Sent Done Done false
14795 4992 Acked Read 5 Sent Done Done false
14796 4993 Acked Read 2 Sent Done Done false
14797 4994 Acked Read 3 Sent Done Done false
14798 4995 Acked Read 2 Sent Done Done false
14799 4996 Acked Read 5 Sent Done Done false
14800 4997 Acked Read 1 Sent Done Done false
14801 4998 Acked Read 2 Sent Done Done false
14802 4999 Acked Read 6 Sent Done Done false
14803 5000 Acked Write 7 Sent Done Done false
14804 5001 Acked Read 9 Sent Done Done false
14805 5002 Acked Read 1 Sent Done Done false
14806 5003 Acked Write 10 Sent Done Done false
14807 5004 Acked Flush 0 Sent Done Done false
14808 5005 Acked Read 7 Sent Done Done false
14809 5006 Acked Read 8 Sent Done Done false
14810 5007 Acked Read 5 Sent Done Done false
14811 5008 Acked Read 4 Sent Done Done false
14812 5009 Acked Write 5 Sent Done Done false
14813 5010 Acked Write 7 Sent Done Done false
14814 5011 Acked Read 2 Sent Done Done false
14815 5012 Acked Read 9 Sent Done Done false
14816 5013 Acked Read 7 Sent Done Done false
14817 5014 Acked Flush 0 Sent Done Done false
14818 5015 Acked Read 6 Sent Done Done false
14819 5016 Acked Read 9 Sent Done Done false
14820 5017 Acked Write 8 Sent Done Done false
14821 5018 Acked Read 2 Sent Done Done false
14822 5019 Acked Write 7 Sent Done Done false
14823 5020 Acked Flush 0 Sent Done Done false
14824 5021 Acked Read 6 Sent Done Done false
14825 5022 Acked Read 5 Sent Done Done false
14826 5023 Acked Write 3 Sent Done Done false
14827 5024 Acked Write 10 Sent Done Done false
14828 5025 Acked Write 4 Sent Done Done false
14829 5026 Acked Write 8 Sent Done Done false
14830 5027 Acked Read 5 Sent Done Done false
14831 5028 Acked Read 6 Sent Done Done false
14832 5029 Acked Read 4 Sent Done Done false
14833 5030 Acked Write 9 Sent Done Done false
14834 5031 Acked Write 9 Sent Done Done false
14835 5032 Acked Write 1 Sent Done Done false
14836 5033 Acked Read 4 Sent Done Done false
14837 5034 Acked Read 9 Sent Done Done false
14838 5035 Acked Read 9 Sent Done Done false
14839 5036 Acked Read 3 Sent Done Done false
14840 5037 Acked Write 7 Sent Done Done false
14841 5038 Acked Read 4 Sent Done Done false
14842 5039 Acked Write 10 Sent Done Done false
14843 5040 Acked Write 1 Sent Done Done false
14844 5041 Acked Write 8 Sent Done Done false
14845 5042 Acked Read 10 Sent Done Done false
14846 5043 Acked Read 2 Sent Done Done false
14847 5044 Acked Read 7 Sent Done Done false
14848 5045 Acked Write 10 Sent Done Done false
14849 5046 Acked Write 9 Sent Done Done false
14850 5047 Acked Flush 0 Sent Done Done false
14851 5048 Acked Read 3 Sent Done Done false
14852 5049 Acked Read 8 Sent Done Done false
14853 5050 Acked Read 1 Sent Done Done false
14854 5051 Acked Write 5 Sent Done Done false
14855 5052 Acked Write 9 Sent Done Done false
14856 5053 Acked Read 1 Sent Done Done false
14857 5054 Acked Read 4 Sent Done Done false
14858 5055 Acked Write 8 Sent Done Done false
14859 5056 Acked Write 6 Sent Done Done false
14860 5057 Acked Write 3 Sent Done Done false
14861 5058 Acked Read 1 Sent Done Done false
14862 5059 Acked Read 1 Sent Done Done false
14863 5060 Acked Write 8 Sent Done Done false
14864 5061 Acked Write 2 Sent Done Done false
14865 5062 Acked Write 2 Sent Done Done false
14866 5063 Acked Write 2 Sent Done Done false
14867 5064 Acked Read 5 Sent Done Done false
14868 5065 Acked Write 1 Sent Done Done false
14869 5066 Acked Write 7 Sent Done Done false
14870 5067 Acked Read 1 Sent Done Done false
14871 5068 Acked Write 4 Sent Done Done false
14872 5069 Acked Read 10 Sent Done Done false
14873 5070 Acked Read 9 Sent Done Done false
14874 5071 Acked Read 2 Sent Done Done false
14875 5072 Acked Flush 0 Sent Done Done false
14876 5073 Acked Write 7 Sent Done Done false
14877 5074 Acked Write 3 Sent Done Done false
14878 5075 Acked Write 5 Sent Done Done false
14879 5076 Acked Read 9 Sent Done Done false
14880 5077 Acked Read 5 Sent Done Done false
14881 5078 Acked Write 9 Sent Done Done false
14882 5079 Acked Flush 0 Sent Done Done false
14883 5080 Acked Read 3 Sent Done Done false
14884 5081 Acked Write 2 Sent Done Done false
14885 5082 Acked Read 9 Sent Done Done false
14886 5083 Acked Read 7 Sent Done Done false
14887 5084 Acked Read 7 Sent Done Done false
14888 5085 Acked Write 3 Sent Done Done false
14889 5086 Acked Read 1 Sent Done Done false
14890 5087 Acked Read 5 Sent Done Done false
14891 5088 Acked Flush 0 Sent Done Done false
14892 5089 Acked Read 1 Sent Done Done false
14893 5090 Acked Read 5 Sent Done Done false
14894 5091 Acked Read 3 Sent Done Done false
14895 5092 Acked Read 7 Sent Done Done false
14896 5093 Acked Read 8 Sent Done Done false
14897 5094 Acked Read 4 Sent Done Done false
14898 5095 Acked Read 8 Sent Done Done false
14899 5096 Acked Write 8 Sent Done Done false
14900 5097 Acked Write 9 Sent Done Done false
14901 5098 Acked Write 1 Sent Done Done false
14902 5099 Acked Read 2 Sent Done Done false
14903 5100 Acked Read 1 Sent Done Done false
14904 5101 Acked Read 8 Sent Done Done false
14905 5102 Acked Flush 0 Sent Done Done false
14906 5103 Acked Read 3 Sent Done Done false
14907 5104 Acked Write 2 Sent Done Done false
14908 5105 Acked Write 3 Sent Done Done false
14909 5106 Acked Read 3 Sent Done Done false
14910 5107 Acked Read 8 Sent Done Done false
14911 5108 Acked Write 7 Sent Done Done false
14912 5109 Acked Read 2 Sent Done Done false
14913 5110 Acked Write 7 Sent Done Done false
14914 5111 Acked Read 9 Sent Done Done false
14915 5112 Acked Flush 0 Sent Done Done false
14916 5113 Acked Write 9 Sent Done Done false
14917 5114 Acked Write 4 Sent Done Done false
14918 5115 Acked Read 3 Sent Done Done false
14919 5116 Acked Flush 0 Sent Done Done false
14920 5117 Acked Read 7 Sent Done Done false
14921 5118 Acked Write 9 Sent Done Done false
14922 5119 Acked Read 7 Sent Done Done false
14923 5120 Acked Read 5 Sent Done Done false
14924 5121 Acked Read 1 Sent Done Done false
14925 5122 Acked Flush 0 Sent Done Done false
14926 5123 Acked Read 5 Sent Done Done false
14927 5124 Acked Read 1 Sent Done Done false
14928 5125 Acked Read 1 Sent Done Done false
14929 5126 Acked Write 7 Sent Done Done false
14930 5127 Acked Read 1 Sent Done Done false
14931 5128 Acked Read 4 Sent Done Done false
14932 5129 Acked Read 9 Sent Done Done false
14933 5130 Acked Write 4 Sent Done Done false
14934 5131 Acked Read 5 Sent Done Done false
14935 5132 Acked Read 9 Sent Done Done false
14936 5133 Acked Read 10 Sent Done Done false
14937 5134 Acked Flush 0 Sent Done Done false
14938 5135 Acked Read 3 Sent Done Done false
14939 5136 Acked Flush 0 Sent Done Done false
14940 5137 Acked Read 8 Sent Done Done false
14941 5138 Acked Read 7 Sent Done Done false
14942 5139 Acked Read 6 Sent Done Done false
14943 5140 Acked Read 3 Sent Done Done false
14944 5141 Acked Read 7 Sent Done Done false
14945 5142 Acked Read 3 Sent Done Done false
14946 5143 Acked Write 6 Sent Done Done false
14947 5144 Acked Read 7 Sent Done Done false
14948 5145 Acked Write 5 Sent Done Done false
14949 5146 Acked Read 10 Sent Done Done false
14950 5147 Acked Write 7 Sent Done Done false
14951 5148 Acked Read 8 Sent Done Done false
14952 5149 Acked Write 5 Sent Done Done false
14953 5150 Acked Write 7 Sent Done Done false
14954 5151 Acked Write 3 Sent Done Done false
14955 5152 Acked Write 8 Sent Done Done false
14956 5153 Acked Write 2 Sent Done Done false
14957 5154 Acked Read 9 Sent Done Done false
14958 5155 Acked Read 7 Sent Done Done false
14959 5156 Acked Write 8 Sent Done Done false
14960 5157 Acked Read 5 Sent Done Done false
14961 5158 Acked Read 10 Sent Done Done false
14962 5159 Acked Read 5 Sent Done Done false
14963 5160 Acked Read 5 Sent Done Done false
14964 5161 Acked Read 8 Sent Done Done false
14965 5162 Acked Write 8 Sent Done Done false
14966 5163 Acked Read 6 Sent Done Done false
14967 5164 Acked Write 3 Sent Done Done false
14968 5165 Acked Write 2 Sent Done Done false
14969 5166 Acked Read 5 Sent Done Done false
14970 5167 Acked Write 5 Sent Done Done false
14971 5168 Acked Write 2 Sent Done Done false
14972 5169 Acked Write 3 Sent Done Done false
14973 5170 Acked Read 4 Sent Done Done false
14974 5171 Acked Read 3 Sent Done Done false
14975 5172 Acked Read 1 Sent Done Done false
14976 5173 Acked Read 8 Sent Done Done false
14977 5174 Acked Write 6 Sent Done Done false
14978 5175 Acked Write 4 Sent Done Done false
14979 5176 Acked Write 1 Sent Done Done false
14980 5177 Acked Write 3 Sent Done Done false
14981 5178 Acked Write 5 Sent Done Done false
14982 5179 Acked Read 7 Sent Done Done false
14983 5180 Acked Read 10 Sent Done Done false
14984 5181 Acked Read 3 Sent Done Done false
14985 5182 Acked Read 2 Sent Done Done false
14986 5183 Acked Read 1 Sent Done Done false
14987 5184 Acked Read 7 Sent Done Done false
14988 5185 Acked Write 1 Sent Done Done false
14989 5186 Acked Write 9 Sent Done Done false
14990 5187 Acked Read 2 Sent Done Done false
14991 5188 Acked Write 5 Sent Done Done false
14992 5189 Acked Write 3 Sent Done Done false
14993 5190 Acked Read 10 Sent Done Done false
14994 5191 Acked Read 7 Sent Done Done false
14995 5192 Acked Write 6 Sent Done Done false
14996 5193 Acked Read 1 Sent Done Done false
14997 5194 Acked Write 7 Sent Done Done false
14998 5195 Acked Read 7 Sent Done Done false
14999 5196 Acked Write 6 Sent Done Done false
15000 5197 Acked Read 1 Sent Done Done false
15001 5198 Acked Write 2 Sent Done Done false
15002 5199 Acked Read 1 Sent Done Done false
15003 5200 Acked Read 10 Sent Done Done false
15004 5201 Acked Write 9 Sent Done Done false
15005 5202 Acked Read 8 Sent Done Done false
15006 5203 Acked Read 8 Sent Done Done false
15007 5204 Acked Write 9 Sent Done Done false
15008 5205 Acked Write 9 Sent Done Done false
15009 5206 Acked Read 7 Sent Done Done false
15010 5207 Acked Write 9 Sent Done Done false
15011 5208 Acked Write 6 Sent Done Done false
15012 5209 Acked Read 5 Sent Done Done false
15013 5210 Acked Read 4 Sent Done Done false
15014 5211 Acked Read 3 Sent Done Done false
15015 5212 Acked Read 9 Sent Done Done false
15016 5213 Acked Flush 0 Sent Done Done false
15017 5214 Acked Read 9 Sent Done Done false
15018 5215 Acked Read 7 Sent Done Done false
15019 5216 Acked Write 4 Sent Done Done false
15020 5217 Acked Flush 0 Sent Done Done false
15021 5218 Acked Read 6 Sent Done Done false
15022 5219 Acked Read 1 Sent Done Done false
15023 5220 Acked Read 3 Sent Done Done false
15024 5221 Acked Write 10 Sent Done Done false
15025 5222 Acked Write 8 Sent Done Done false
15026 5223 Acked Read 7 Sent Done Done false
15027 5224 Acked Flush 0 Sent Done Done false
15028 STATES DS:0 DS:1 DS:2 TOTAL
15029 Sent 302 0 0 302
15030 Done 0 302 302 604
15031 Skipped 0 0 0 0
15032 Error 0 0 0 0
15033 Last Flush: 4922 5224 5224
15034 Downstairs last five completed: 4922 4921 4920 4919 4918
15035 Upstairs last five completed: 5224 5223 5222 5221 5220
15036 CLIENT: Up:0 ds:302 act:2
150372025-01-27T19:39:30.131ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
150382025-01-27T19:39:30.131ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
150392025-01-27T19:39:30.131ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
150402025-01-27T19:39:30.131ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
150412025-01-27T19:39:30.131ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
15042 [8810] got start action
15043 [8810] I am going to restart
15044 Starting downstairs at port 8810
15045 Make output file at "/tmp/dsc/downstairs-8810.txt"
15046 [8810][0] reports Starting
15047 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1214)
15048 [8810][0] reports Running
15049 ----------------------------------------------------------------
15050 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15051 States: Offline Active Active
15052 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15053 4923 Acked Write 3 Sent Done Done false
15054 4924 Acked Write 8 Sent Done Done false
15055 4925 Acked Write 2 Sent Done Done false
15056 4926 Acked Read 8 Sent Done Done false
15057 4927 Acked Read 6 Sent Done Done false
15058 4928 Acked Read 8 Sent Done Done false
15059 4929 Acked Write 10 Sent Done Done false
15060 4930 Acked Read 5 Sent Done Done false
15061 4931 Acked Read 7 Sent Done Done false
15062 4932 Acked Write 2 Sent Done Done false
15063 4933 Acked Write 6 Sent Done Done false
15064 4934 Acked Write 5 Sent Done Done false
15065 4935 Acked Read 4 Sent Done Done false
15066 4936 Acked Read 2 Sent Done Done false
15067 4937 Acked Read 8 Sent Done Done false
15068 4938 Acked Write 1 Sent Done Done false
15069 4939 Acked Read 2 Sent Done Done false
15070 4940 Acked Write 6 Sent Done Done false
15071 4941 Acked Write 9 Sent Done Done false
15072 4942 Acked Read 1 Sent Done Done false
15073 4943 Acked Write 6 Sent Done Done false
15074 4944 Acked Read 1 Sent Done Done false
15075 4945 Acked Read 4 Sent Done Done false
15076 4946 Acked Write 10 Sent Done Done false
15077 4947 Acked Write 3 Sent Done Done false
15078 4948 Acked Write 6 Sent Done Done false
15079 4949 Acked Read 6 Sent Done Done false
15080 4950 Acked Read 6 Sent Done Done false
15081 4951 Acked Read 7 Sent Done Done false
15082 4952 Acked Read 7 Sent Done Done false
15083 4953 Acked Write 5 Sent Done Done false
15084 4954 Acked Write 10 Sent Done Done false
15085 4955 Acked Read 9 Sent Done Done false
15086 4956 Acked Write 9 Sent Done Done false
15087 4957 Acked Write 8 Sent Done Done false
15088 4958 Acked Read 4 Sent Done Done false
15089 4959 Acked Read 3 Sent Done Done false
15090 4960 Acked Write 10 Sent Done Done false
15091 4961 Acked Write 9 Sent Done Done false
15092 4962 Acked Read 9 Sent Done Done false
15093 4963 Acked Read 3 Sent Done Done false
15094 4964 Acked Write 8 Sent Done Done false
15095 4965 Acked Read 6 Sent Done Done false
15096 4966 Acked Write 1 Sent Done Done false
15097 4967 Acked Read 7 Sent Done Done false
15098 4968 Acked Read 1 Sent Done Done false
15099 4969 Acked Read 1 Sent Done Done false
15100 4970 Acked Write 1 Sent Done Done false
15101 4971 Acked Read 10 Sent Done Done false
15102 4972 Acked Read 4 Sent Done Done false
15103 4973 Acked Write 9 Sent Done Done false
15104 4974 Acked Read 6 Sent Done Done false
15105 4975 Acked Write 10 Sent Done Done false
15106 4976 Acked Read 3 Sent Done Done false
15107 4977 Acked Read 2 Sent Done Done false
15108 4978 Acked Flush 0 Sent Done Done false
15109 4979 Acked Write 5 Sent Done Done false
15110 4980 Acked Read 10 Sent Done Done false
15111 4981 Acked Read 4 Sent Done Done false
15112 4982 Acked Write 9 Sent Done Done false
15113 4983 Acked Read 9 Sent Done Done false
15114 4984 Acked Read 3 Sent Done Done false
15115 4985 Acked Write 8 Sent Done Done false
15116 4986 Acked Flush 0 Sent Done Done false
15117 4987 Acked Write 5 Sent Done Done false
15118 4988 Acked Write 1 Sent Done Done false
15119 4989 Acked Write 2 Sent Done Done false
15120 4990 Acked Write 8 Sent Done Done false
15121 4991 Acked Write 3 Sent Done Done false
15122 4992 Acked Read 5 Sent Done Done false
15123 4993 Acked Read 2 Sent Done Done false
15124 4994 Acked Read 3 Sent Done Done false
15125 4995 Acked Read 2 Sent Done Done false
15126 4996 Acked Read 5 Sent Done Done false
15127 4997 Acked Read 1 Sent Done Done false
15128 4998 Acked Read 2 Sent Done Done false
15129 4999 Acked Read 6 Sent Done Done false
15130 5000 Acked Write 7 Sent Done Done false
15131 5001 Acked Read 9 Sent Done Done false
15132 5002 Acked Read 1 Sent Done Done false
15133 5003 Acked Write 10 Sent Done Done false
15134 5004 Acked Flush 0 Sent Done Done false
15135 5005 Acked Read 7 Sent Done Done false
15136 5006 Acked Read 8 Sent Done Done false
15137 5007 Acked Read 5 Sent Done Done false
15138 5008 Acked Read 4 Sent Done Done false
15139 5009 Acked Write 5 Sent Done Done false
15140 5010 Acked Write 7 Sent Done Done false
15141 5011 Acked Read 2 Sent Done Done false
15142 5012 Acked Read 9 Sent Done Done false
15143 5013 Acked Read 7 Sent Done Done false
15144 5014 Acked Flush 0 Sent Done Done false
15145 5015 Acked Read 6 Sent Done Done false
15146 5016 Acked Read 9 Sent Done Done false
15147 5017 Acked Write 8 Sent Done Done false
15148 5018 Acked Read 2 Sent Done Done false
15149 5019 Acked Write 7 Sent Done Done false
15150 5020 Acked Flush 0 Sent Done Done false
15151 5021 Acked Read 6 Sent Done Done false
15152 5022 Acked Read 5 Sent Done Done false
15153 5023 Acked Write 3 Sent Done Done false
15154 5024 Acked Write 10 Sent Done Done false
15155 5025 Acked Write 4 Sent Done Done false
15156 5026 Acked Write 8 Sent Done Done false
15157 5027 Acked Read 5 Sent Done Done false
15158 5028 Acked Read 6 Sent Done Done false
15159 5029 Acked Read 4 Sent Done Done false
15160 5030 Acked Write 9 Sent Done Done false
15161 5031 Acked Write 9 Sent Done Done false
15162 5032 Acked Write 1 Sent Done Done false
15163 5033 Acked Read 4 Sent Done Done false
15164 5034 Acked Read 9 Sent Done Done false
15165 5035 Acked Read 9 Sent Done Done false
15166 5036 Acked Read 3 Sent Done Done false
15167 5037 Acked Write 7 Sent Done Done false
15168 5038 Acked Read 4 Sent Done Done false
15169 5039 Acked Write 10 Sent Done Done false
15170 5040 Acked Write 1 Sent Done Done false
15171 5041 Acked Write 8 Sent Done Done false
15172 5042 Acked Read 10 Sent Done Done false
15173 5043 Acked Read 2 Sent Done Done false
15174 5044 Acked Read 7 Sent Done Done false
15175 5045 Acked Write 10 Sent Done Done false
15176 5046 Acked Write 9 Sent Done Done false
15177 5047 Acked Flush 0 Sent Done Done false
15178 5048 Acked Read 3 Sent Done Done false
15179 5049 Acked Read 8 Sent Done Done false
15180 5050 Acked Read 1 Sent Done Done false
15181 5051 Acked Write 5 Sent Done Done false
15182 5052 Acked Write 9 Sent Done Done false
15183 5053 Acked Read 1 Sent Done Done false
15184 5054 Acked Read 4 Sent Done Done false
15185 5055 Acked Write 8 Sent Done Done false
15186 5056 Acked Write 6 Sent Done Done false
15187 5057 Acked Write 3 Sent Done Done false
15188 5058 Acked Read 1 Sent Done Done false
15189 5059 Acked Read 1 Sent Done Done false
15190 5060 Acked Write 8 Sent Done Done false
15191 5061 Acked Write 2 Sent Done Done false
15192 5062 Acked Write 2 Sent Done Done false
15193 5063 Acked Write 2 Sent Done Done false
15194 5064 Acked Read 5 Sent Done Done false
15195 5065 Acked Write 1 Sent Done Done false
15196 5066 Acked Write 7 Sent Done Done false
15197 5067 Acked Read 1 Sent Done Done false
15198 5068 Acked Write 4 Sent Done Done false
15199 5069 Acked Read 10 Sent Done Done false
15200 5070 Acked Read 9 Sent Done Done false
15201 5071 Acked Read 2 Sent Done Done false
15202 5072 Acked Flush 0 Sent Done Done false
15203 5073 Acked Write 7 Sent Done Done false
15204 5074 Acked Write 3 Sent Done Done false
15205 5075 Acked Write 5 Sent Done Done false
15206 5076 Acked Read 9 Sent Done Done false
15207 5077 Acked Read 5 Sent Done Done false
15208 5078 Acked Write 9 Sent Done Done false
15209 5079 Acked Flush 0 Sent Done Done false
15210 5080 Acked Read 3 Sent Done Done false
15211 5081 Acked Write 2 Sent Done Done false
15212 5082 Acked Read 9 Sent Done Done false
15213 5083 Acked Read 7 Sent Done Done false
15214 5084 Acked Read 7 Sent Done Done false
15215 5085 Acked Write 3 Sent Done Done false
15216 5086 Acked Read 1 Sent Done Done false
15217 5087 Acked Read 5 Sent Done Done false
15218 5088 Acked Flush 0 Sent Done Done false
15219 5089 Acked Read 1 Sent Done Done false
15220 5090 Acked Read 5 Sent Done Done false
15221 5091 Acked Read 3 Sent Done Done false
15222 5092 Acked Read 7 Sent Done Done false
15223 5093 Acked Read 8 Sent Done Done false
15224 5094 Acked Read 4 Sent Done Done false
15225 5095 Acked Read 8 Sent Done Done false
15226 5096 Acked Write 8 Sent Done Done false
15227 5097 Acked Write 9 Sent Done Done false
15228 5098 Acked Write 1 Sent Done Done false
15229 5099 Acked Read 2 Sent Done Done false
15230 5100 Acked Read 1 Sent Done Done false
15231 5101 Acked Read 8 Sent Done Done false
15232 5102 Acked Flush 0 Sent Done Done false
15233 5103 Acked Read 3 Sent Done Done false
15234 5104 Acked Write 2 Sent Done Done false
15235 5105 Acked Write 3 Sent Done Done false
15236 5106 Acked Read 3 Sent Done Done false
15237 5107 Acked Read 8 Sent Done Done false
15238 5108 Acked Write 7 Sent Done Done false
15239 5109 Acked Read 2 Sent Done Done false
15240 5110 Acked Write 7 Sent Done Done false
15241 5111 Acked Read 9 Sent Done Done false
15242 5112 Acked Flush 0 Sent Done Done false
15243 5113 Acked Write 9 Sent Done Done false
15244 5114 Acked Write 4 Sent Done Done false
15245 5115 Acked Read 3 Sent Done Done false
15246 5116 Acked Flush 0 Sent Done Done false
15247 5117 Acked Read 7 Sent Done Done false
15248 5118 Acked Write 9 Sent Done Done false
15249 5119 Acked Read 7 Sent Done Done false
15250 5120 Acked Read 5 Sent Done Done false
15251 5121 Acked Read 1 Sent Done Done false
15252 5122 Acked Flush 0 Sent Done Done false
15253 5123 Acked Read 5 Sent Done Done false
15254 5124 Acked Read 1 Sent Done Done false
15255 5125 Acked Read 1 Sent Done Done false
15256 5126 Acked Write 7 Sent Done Done false
15257 5127 Acked Read 1 Sent Done Done false
15258 5128 Acked Read 4 Sent Done Done false
15259 5129 Acked Read 9 Sent Done Done false
15260 5130 Acked Write 4 Sent Done Done false
15261 5131 Acked Read 5 Sent Done Done false
15262 5132 Acked Read 9 Sent Done Done false
15263 5133 Acked Read 10 Sent Done Done false
15264 5134 Acked Flush 0 Sent Done Done false
15265 5135 Acked Read 3 Sent Done Done false
15266 5136 Acked Flush 0 Sent Done Done false
15267 5137 Acked Read 8 Sent Done Done false
15268 5138 Acked Read 7 Sent Done Done false
15269 5139 Acked Read 6 Sent Done Done false
15270 5140 Acked Read 3 Sent Done Done false
15271 5141 Acked Read 7 Sent Done Done false
15272 5142 Acked Read 3 Sent Done Done false
15273 5143 Acked Write 6 Sent Done Done false
15274 5144 Acked Read 7 Sent Done Done false
15275 5145 Acked Write 5 Sent Done Done false
15276 5146 Acked Read 10 Sent Done Done false
15277 5147 Acked Write 7 Sent Done Done false
15278 5148 Acked Read 8 Sent Done Done false
15279 5149 Acked Write 5 Sent Done Done false
15280 5150 Acked Write 7 Sent Done Done false
15281 5151 Acked Write 3 Sent Done Done false
15282 5152 Acked Write 8 Sent Done Done false
15283 5153 Acked Write 2 Sent Done Done false
15284 5154 Acked Read 9 Sent Done Done false
15285 5155 Acked Read 7 Sent Done Done false
15286 5156 Acked Write 8 Sent Done Done false
15287 5157 Acked Read 5 Sent Done Done false
15288 5158 Acked Read 10 Sent Done Done false
15289 5159 Acked Read 5 Sent Done Done false
15290 5160 Acked Read 5 Sent Done Done false
15291 5161 Acked Read 8 Sent Done Done false
15292 5162 Acked Write 8 Sent Done Done false
15293 5163 Acked Read 6 Sent Done Done false
15294 5164 Acked Write 3 Sent Done Done false
15295 5165 Acked Write 2 Sent Done Done false
15296 5166 Acked Read 5 Sent Done Done false
15297 5167 Acked Write 5 Sent Done Done false
15298 5168 Acked Write 2 Sent Done Done false
15299 5169 Acked Write 3 Sent Done Done false
15300 5170 Acked Read 4 Sent Done Done false
15301 5171 Acked Read 3 Sent Done Done false
15302 5172 Acked Read 1 Sent Done Done false
15303 5173 Acked Read 8 Sent Done Done false
15304 5174 Acked Write 6 Sent Done Done false
15305 5175 Acked Write 4 Sent Done Done false
15306 5176 Acked Write 1 Sent Done Done false
15307 5177 Acked Write 3 Sent Done Done false
15308 5178 Acked Write 5 Sent Done Done false
15309 5179 Acked Read 7 Sent Done Done false
15310 5180 Acked Read 10 Sent Done Done false
15311 5181 Acked Read 3 Sent Done Done false
15312 5182 Acked Read 2 Sent Done Done false
15313 5183 Acked Read 1 Sent Done Done false
15314 5184 Acked Read 7 Sent Done Done false
15315 5185 Acked Write 1 Sent Done Done false
15316 5186 Acked Write 9 Sent Done Done false
15317 5187 Acked Read 2 Sent Done Done false
15318 5188 Acked Write 5 Sent Done Done false
15319 5189 Acked Write 3 Sent Done Done false
15320 5190 Acked Read 10 Sent Done Done false
15321 5191 Acked Read 7 Sent Done Done false
15322 5192 Acked Write 6 Sent Done Done false
15323 5193 Acked Read 1 Sent Done Done false
15324 5194 Acked Write 7 Sent Done Done false
15325 5195 Acked Read 7 Sent Done Done false
15326 5196 Acked Write 6 Sent Done Done false
15327 5197 Acked Read 1 Sent Done Done false
15328 5198 Acked Write 2 Sent Done Done false
15329 5199 Acked Read 1 Sent Done Done false
15330 5200 Acked Read 10 Sent Done Done false
15331 5201 Acked Write 9 Sent Done Done false
15332 5202 Acked Read 8 Sent Done Done false
15333 5203 Acked Read 8 Sent Done Done false
15334 5204 Acked Write 9 Sent Done Done false
15335 5205 Acked Write 9 Sent Done Done false
15336 5206 Acked Read 7 Sent Done Done false
15337 5207 Acked Write 9 Sent Done Done false
15338 5208 Acked Write 6 Sent Done Done false
15339 5209 Acked Read 5 Sent Done Done false
15340 5210 Acked Read 4 Sent Done Done false
15341 5211 Acked Read 3 Sent Done Done false
15342 5212 Acked Read 9 Sent Done Done false
15343 5213 Acked Flush 0 Sent Done Done false
15344 5214 Acked Read 9 Sent Done Done false
15345 5215 Acked Read 7 Sent Done Done false
15346 5216 Acked Write 4 Sent Done Done false
15347 5217 Acked Flush 0 Sent Done Done false
15348 5218 Acked Read 6 Sent Done Done false
15349 5219 Acked Read 1 Sent Done Done false
15350 5220 Acked Read 3 Sent Done Done false
15351 5221 Acked Write 10 Sent Done Done false
15352 5222 Acked Write 8 Sent Done Done false
15353 5223 Acked Read 7 Sent Done Done false
15354 5224 Acked Flush 0 Sent Done Done false
15355 STATES DS:0 DS:1 DS:2 TOTAL
15356 Sent 302 0 0 302
15357 Done 0 302 302 604
15358 Skipped 0 0 0 0
15359 Error 0 0 0 0
15360 Last Flush: 4922 5224 5224
15361 Downstairs last five completed: 4922 4921 4920 4919 4918
15362 Upstairs last five completed: 5224 5223 5222 5221 5220
15363 CLIENT: Up:0 ds:302 act:2
15364 ----------------------------------------------------------------
15365 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
15366 States: Offline Active Active
15367 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
15368 4923 Acked Write 3 Sent Done Done false
15369 4924 Acked Write 8 Sent Done Done false
15370 4925 Acked Write 2 Sent Done Done false
15371 4926 Acked Read 8 Sent Done Done false
15372 4927 Acked Read 6 Sent Done Done false
15373 4928 Acked Read 8 Sent Done Done false
15374 4929 Acked Write 10 Sent Done Done false
15375 4930 Acked Read 5 Sent Done Done false
15376 4931 Acked Read 7 Sent Done Done false
15377 4932 Acked Write 2 Sent Done Done false
15378 4933 Acked Write 6 Sent Done Done false
15379 4934 Acked Write 5 Sent Done Done false
15380 4935 Acked Read 4 Sent Done Done false
15381 4936 Acked Read 2 Sent Done Done false
15382 4937 Acked Read 8 Sent Done Done false
15383 4938 Acked Write 1 Sent Done Done false
15384 4939 Acked Read 2 Sent Done Done false
15385 4940 Acked Write 6 Sent Done Done false
15386 4941 Acked Write 9 Sent Done Done false
15387 4942 Acked Read 1 Sent Done Done false
15388 4943 Acked Write 6 Sent Done Done false
15389 4944 Acked Read 1 Sent Done Done false
15390 4945 Acked Read 4 Sent Done Done false
15391 4946 Acked Write 10 Sent Done Done false
15392 4947 Acked Write 3 Sent Done Done false
15393 4948 Acked Write 6 Sent Done Done false
15394 4949 Acked Read 6 Sent Done Done false
15395 4950 Acked Read 6 Sent Done Done false
15396 4951 Acked Read 7 Sent Done Done false
15397 4952 Acked Read 7 Sent Done Done false
15398 4953 Acked Write 5 Sent Done Done false
15399 4954 Acked Write 10 Sent Done Done false
15400 4955 Acked Read 9 Sent Done Done false
15401 4956 Acked Write 9 Sent Done Done false
15402 4957 Acked Write 8 Sent Done Done false
15403 4958 Acked Read 4 Sent Done Done false
15404 4959 Acked Read 3 Sent Done Done false
15405 4960 Acked Write 10 Sent Done Done false
15406 4961 Acked Write 9 Sent Done Done false
15407 4962 Acked Read 9 Sent Done Done false
15408 4963 Acked Read 3 Sent Done Done false
15409 4964 Acked Write 8 Sent Done Done false
15410 4965 Acked Read 6 Sent Done Done false
15411 4966 Acked Write 1 Sent Done Done false
15412 4967 Acked Read 7 Sent Done Done false
15413 4968 Acked Read 1 Sent Done Done false
15414 4969 Acked Read 1 Sent Done Done false
15415 4970 Acked Write 1 Sent Done Done false
15416 4971 Acked Read 10 Sent Done Done false
15417 4972 Acked Read 4 Sent Done Done false
15418 4973 Acked Write 9 Sent Done Done false
15419 4974 Acked Read 6 Sent Done Done false
15420 4975 Acked Write 10 Sent Done Done false
15421 4976 Acked Read 3 Sent Done Done false
15422 4977 Acked Read 2 Sent Done Done false
15423 4978 Acked Flush 0 Sent Done Done false
15424 4979 Acked Write 5 Sent Done Done false
15425 4980 Acked Read 10 Sent Done Done false
15426 4981 Acked Read 4 Sent Done Done false
15427 4982 Acked Write 9 Sent Done Done false
15428 4983 Acked Read 9 Sent Done Done false
15429 4984 Acked Read 3 Sent Done Done false
15430 4985 Acked Write 8 Sent Done Done false
15431 4986 Acked Flush 0 Sent Done Done false
15432 4987 Acked Write 5 Sent Done Done false
15433 4988 Acked Write 1 Sent Done Done false
15434 4989 Acked Write 2 Sent Done Done false
15435 4990 Acked Write 8 Sent Done Done false
15436 4991 Acked Write 3 Sent Done Done false
15437 4992 Acked Read 5 Sent Done Done false
15438 4993 Acked Read 2 Sent Done Done false
15439 4994 Acked Read 3 Sent Done Done false
15440 4995 Acked Read 2 Sent Done Done false
15441 4996 Acked Read 5 Sent Done Done false
15442 4997 Acked Read 1 Sent Done Done false
15443 4998 Acked Read 2 Sent Done Done false
15444 4999 Acked Read 6 Sent Done Done false
15445 5000 Acked Write 7 Sent Done Done false
15446 5001 Acked Read 9 Sent Done Done false
15447 5002 Acked Read 1 Sent Done Done false
15448 5003 Acked Write 10 Sent Done Done false
15449 5004 Acked Flush 0 Sent Done Done false
15450 5005 Acked Read 7 Sent Done Done false
15451 5006 Acked Read 8 Sent Done Done false
15452 5007 Acked Read 5 Sent Done Done false
15453 5008 Acked Read 4 Sent Done Done false
15454 5009 Acked Write 5 Sent Done Done false
15455 5010 Acked Write 7 Sent Done Done false
15456 5011 Acked Read 2 Sent Done Done false
15457 5012 Acked Read 9 Sent Done Done false
15458 5013 Acked Read 7 Sent Done Done false
15459 5014 Acked Flush 0 Sent Done Done false
15460 5015 Acked Read 6 Sent Done Done false
15461 5016 Acked Read 9 Sent Done Done false
15462 5017 Acked Write 8 Sent Done Done false
15463 5018 Acked Read 2 Sent Done Done false
15464 5019 Acked Write 7 Sent Done Done false
15465 5020 Acked Flush 0 Sent Done Done false
15466 5021 Acked Read 6 Sent Done Done false
15467 5022 Acked Read 5 Sent Done Done false
15468 5023 Acked Write 3 Sent Done Done false
15469 5024 Acked Write 10 Sent Done Done false
15470 5025 Acked Write 4 Sent Done Done false
15471 5026 Acked Write 8 Sent Done Done false
15472 5027 Acked Read 5 Sent Done Done false
15473 5028 Acked Read 6 Sent Done Done false
15474 5029 Acked Read 4 Sent Done Done false
15475 5030 Acked Write 9 Sent Done Done false
15476 5031 Acked Write 9 Sent Done Done false
15477 5032 Acked Write 1 Sent Done Done false
15478 5033 Acked Read 4 Sent Done Done false
15479 5034 Acked Read 9 Sent Done Done false
15480 5035 Acked Read 9 Sent Done Done false
15481 5036 Acked Read 3 Sent Done Done false
15482 5037 Acked Write 7 Sent Done Done false
15483 5038 Acked Read 4 Sent Done Done false
15484 5039 Acked Write 10 Sent Done Done false
15485 5040 Acked Write 1 Sent Done Done false
15486 5041 Acked Write 8 Sent Done Done false
15487 5042 Acked Read 10 Sent Done Done false
15488 5043 Acked Read 2 Sent Done Done false
15489 5044 Acked Read 7 Sent Done Done false
15490 5045 Acked Write 10 Sent Done Done false
15491 5046 Acked Write 9 Sent Done Done false
15492 5047 Acked Flush 0 Sent Done Done false
15493 5048 Acked Read 3 Sent Done Done false
15494 5049 Acked Read 8 Sent Done Done false
15495 5050 Acked Read 1 Sent Done Done false
15496 5051 Acked Write 5 Sent Done Done false
15497 5052 Acked Write 9 Sent Done Done false
15498 5053 Acked Read 1 Sent Done Done false
15499 5054 Acked Read 4 Sent Done Done false
15500 5055 Acked Write 8 Sent Done Done false
15501 5056 Acked Write 6 Sent Done Done false
15502 5057 Acked Write 3 Sent Done Done false
15503 5058 Acked Read 1 Sent Done Done false
15504 5059 Acked Read 1 Sent Done Done false
15505 5060 Acked Write 8 Sent Done Done false
15506 5061 Acked Write 2 Sent Done Done false
15507 5062 Acked Write 2 Sent Done Done false
15508 5063 Acked Write 2 Sent Done Done false
15509 5064 Acked Read 5 Sent Done Done false
15510 5065 Acked Write 1 Sent Done Done false
15511 5066 Acked Write 7 Sent Done Done false
15512 5067 Acked Read 1 Sent Done Done false
15513 5068 Acked Write 4 Sent Done Done false
15514 5069 Acked Read 10 Sent Done Done false
15515 5070 Acked Read 9 Sent Done Done false
15516 5071 Acked Read 2 Sent Done Done false
15517 5072 Acked Flush 0 Sent Done Done false
15518 5073 Acked Write 7 Sent Done Done false
15519 5074 Acked Write 3 Sent Done Done false
15520 5075 Acked Write 5 Sent Done Done false
15521 5076 Acked Read 9 Sent Done Done false
15522 5077 Acked Read 5 Sent Done Done false
15523 5078 Acked Write 9 Sent Done Done false
15524 5079 Acked Flush 0 Sent Done Done false
15525 5080 Acked Read 3 Sent Done Done false
15526 5081 Acked Write 2 Sent Done Done false
15527 5082 Acked Read 9 Sent Done Done false
15528 5083 Acked Read 7 Sent Done Done false
15529 5084 Acked Read 7 Sent Done Done false
15530 5085 Acked Write 3 Sent Done Done false
15531 5086 Acked Read 1 Sent Done Done false
15532 5087 Acked Read 5 Sent Done Done false
15533 5088 Acked Flush 0 Sent Done Done false
15534 5089 Acked Read 1 Sent Done Done false
15535 5090 Acked Read 5 Sent Done Done false
15536 5091 Acked Read 3 Sent Done Done false
15537 5092 Acked Read 7 Sent Done Done false
15538 5093 Acked Read 8 Sent Done Done false
15539 5094 Acked Read 4 Sent Done Done false
15540 5095 Acked Read 8 Sent Done Done false
15541 5096 Acked Write 8 Sent Done Done false
15542 5097 Acked Write 9 Sent Done Done false
15543 5098 Acked Write 1 Sent Done Done false
15544 5099 Acked Read 2 Sent Done Done false
15545 5100 Acked Read 1 Sent Done Done false
15546 5101 Acked Read 8 Sent Done Done false
15547 5102 Acked Flush 0 Sent Done Done false
15548 5103 Acked Read 3 Sent Done Done false
15549 5104 Acked Write 2 Sent Done Done false
15550 5105 Acked Write 3 Sent Done Done false
15551 5106 Acked Read 3 Sent Done Done false
15552 5107 Acked Read 8 Sent Done Done false
15553 5108 Acked Write 7 Sent Done Done false
15554 5109 Acked Read 2 Sent Done Done false
15555 5110 Acked Write 7 Sent Done Done false
15556 5111 Acked Read 9 Sent Done Done false
15557 5112 Acked Flush 0 Sent Done Done false
15558 5113 Acked Write 9 Sent Done Done false
15559 5114 Acked Write 4 Sent Done Done false
15560 5115 Acked Read 3 Sent Done Done false
15561 5116 Acked Flush 0 Sent Done Done false
15562 5117 Acked Read 7 Sent Done Done false
15563 5118 Acked Write 9 Sent Done Done false
15564 5119 Acked Read 7 Sent Done Done false
15565 5120 Acked Read 5 Sent Done Done false
15566 5121 Acked Read 1 Sent Done Done false
15567 5122 Acked Flush 0 Sent Done Done false
15568 5123 Acked Read 5 Sent Done Done false
15569 5124 Acked Read 1 Sent Done Done false
15570 5125 Acked Read 1 Sent Done Done false
15571 5126 Acked Write 7 Sent Done Done false
15572 5127 Acked Read 1 Sent Done Done false
15573 5128 Acked Read 4 Sent Done Done false
15574 5129 Acked Read 9 Sent Done Done false
15575 5130 Acked Write 4 Sent Done Done false
15576 5131 Acked Read 5 Sent Done Done false
15577 5132 Acked Read 9 Sent Done Done false
15578 5133 Acked Read 10 Sent Done Done false
15579 5134 Acked Flush 0 Sent Done Done false
15580 5135 Acked Read 3 Sent Done Done false
15581 5136 Acked Flush 0 Sent Done Done false
15582 5137 Acked Read 8 Sent Done Done false
15583 5138 Acked Read 7 Sent Done Done false
15584 5139 Acked Read 6 Sent Done Done false
15585 5140 Acked Read 3 Sent Done Done false
15586 5141 Acked Read 7 Sent Done Done false
15587 5142 Acked Read 3 Sent Done Done false
15588 5143 Acked Write 6 Sent Done Done false
15589 5144 Acked Read 7 Sent Done Done false
15590 5145 Acked Write 5 Sent Done Done false
15591 5146 Acked Read 10 Sent Done Done false
15592 5147 Acked Write 7 Sent Done Done false
15593 5148 Acked Read 8 Sent Done Done false
15594 5149 Acked Write 5 Sent Done Done false
15595 5150 Acked Write 7 Sent Done Done false
15596 5151 Acked Write 3 Sent Done Done false
15597 5152 Acked Write 8 Sent Done Done false
15598 5153 Acked Write 2 Sent Done Done false
15599 5154 Acked Read 9 Sent Done Done false
15600 5155 Acked Read 7 Sent Done Done false
15601 5156 Acked Write 8 Sent Done Done false
15602 5157 Acked Read 5 Sent Done Done false
15603 5158 Acked Read 10 Sent Done Done false
15604 5159 Acked Read 5 Sent Done Done false
15605 5160 Acked Read 5 Sent Done Done false
15606 5161 Acked Read 8 Sent Done Done false
15607 5162 Acked Write 8 Sent Done Done false
15608 5163 Acked Read 6 Sent Done Done false
15609 5164 Acked Write 3 Sent Done Done false
15610 5165 Acked Write 2 Sent Done Done false
15611 5166 Acked Read 5 Sent Done Done false
15612 5167 Acked Write 5 Sent Done Done false
15613 5168 Acked Write 2 Sent Done Done false
15614 5169 Acked Write 3 Sent Done Done false
15615 5170 Acked Read 4 Sent Done Done false
15616 5171 Acked Read 3 Sent Done Done false
15617 5172 Acked Read 1 Sent Done Done false
15618 5173 Acked Read 8 Sent Done Done false
15619 5174 Acked Write 6 Sent Done Done false
15620 5175 Acked Write 4 Sent Done Done false
15621 5176 Acked Write 1 Sent Done Done false
15622 5177 Acked Write 3 Sent Done Done false
15623 5178 Acked Write 5 Sent Done Done false
15624 5179 Acked Read 7 Sent Done Done false
15625 5180 Acked Read 10 Sent Done Done false
15626 5181 Acked Read 3 Sent Done Done false
15627 5182 Acked Read 2 Sent Done Done false
15628 5183 Acked Read 1 Sent Done Done false
15629 5184 Acked Read 7 Sent Done Done false
15630 5185 Acked Write 1 Sent Done Done false
15631 5186 Acked Write 9 Sent Done Done false
15632 5187 Acked Read 2 Sent Done Done false
15633 5188 Acked Write 5 Sent Done Done false
15634 5189 Acked Write 3 Sent Done Done false
15635 5190 Acked Read 10 Sent Done Done false
15636 5191 Acked Read 7 Sent Done Done false
15637 5192 Acked Write 6 Sent Done Done false
15638 5193 Acked Read 1 Sent Done Done false
15639 5194 Acked Write 7 Sent Done Done false
15640 5195 Acked Read 7 Sent Done Done false
15641 5196 Acked Write 6 Sent Done Done false
15642 5197 Acked Read 1 Sent Done Done false
15643 5198 Acked Write 2 Sent Done Done false
15644 5199 Acked Read 1 Sent Done Done false
15645 5200 Acked Read 10 Sent Done Done false
15646 5201 Acked Write 9 Sent Done Done false
15647 5202 Acked Read 8 Sent Done Done false
15648 5203 Acked Read 8 Sent Done Done false
15649 5204 Acked Write 9 Sent Done Done false
15650 5205 Acked Write 9 Sent Done Done false
15651 5206 Acked Read 7 Sent Done Done false
15652 5207 Acked Write 9 Sent Done Done false
15653 5208 Acked Write 6 Sent Done Done false
15654 5209 Acked Read 5 Sent Done Done false
15655 5210 Acked Read 4 Sent Done Done false
15656 5211 Acked Read 3 Sent Done Done false
15657 5212 Acked Read 9 Sent Done Done false
15658 5213 Acked Flush 0 Sent Done Done false
15659 5214 Acked Read 9 Sent Done Done false
15660 5215 Acked Read 7 Sent Done Done false
15661 5216 Acked Write 4 Sent Done Done false
15662 5217 Acked Flush 0 Sent Done Done false
15663 5218 Acked Read 6 Sent Done Done false
15664 5219 Acked Read 1 Sent Done Done false
15665 5220 Acked Read 3 Sent Done Done false
15666 5221 Acked Write 10 Sent Done Done false
15667 5222 Acked Write 8 Sent Done Done false
15668 5223 Acked Read 7 Sent Done Done false
15669 5224 Acked Flush 0 Sent Done Done false
15670 STATES DS:0 DS:1 DS:2 TOTAL
15671 Sent 302 0 0 302
15672 Done 0 302 302 604
15673 Skipped 0 0 0 0
15674 Error 0 0 0 0
15675 Last Flush: 4922 5224 5224
15676 Downstairs last five completed: 4922 4921 4920 4919 4918
15677 Upstairs last five completed: 5224 5223 5222 5221 5220
15678 CLIENT: Up:0 ds:302 act:2
156792025-01-27T19:39:40.132ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
156802025-01-27T19:39:40.132ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:62129) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
156812025-01-27T19:39:40.134ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
156822025-01-27T19:39:40.134ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
156832025-01-27T19:39:40.134ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
156842025-01-27T19:39:40.134ZINFOcrucible: send last flush ID to this DS: 4922 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
156852025-01-27T19:39:40.134ZINFOcrucible: Replied this last flush ID: 4922 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
156862025-01-27T19:39:40.134ZINFOcrucible: [0] client re-new 302 jobs since flush 4922 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
156872025-01-27T19:39:40.135ZINFOcrucible: [0] Replayed 302 jobs since flush: 4922 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
15688 ----------------------------------------------------------------
15689 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
15690 Downstairs last five completed: 5224 5223 5222 5221 5220
15691 Upstairs last five completed: 5224 5223 5222 5221 5220
15692 CLIENT: Up:0 ds:0 act:3
15693 Replay: All jobs finished
15694 Main task has work to do, go find it
15695 got dsc Stop(2)
15696 stop 2
15697 Jan 27 19:39:40.382 INFO request completed, latency_us: 358, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 8097fdf7-71e9-4f55-809f-4db79a52658a, remote_addr: 127.0[8830] Got stop action so:false kr:false
15698 .0.1:55047, local_addr: 127.0.0.1:9998
15699 Jan 27 19:39:40.383 INFO request completed, latency_us: 131, response_code: 200, uri: /state/cid/2, method: GET, req_id: fdf06a36-0477-4bfa-b783-b55311cb23b1, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
157002025-01-27T19:39:40.384ZWARNcrucible: downstairs client error Connection reset by peer (os error 131) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
15701 [8830] Exited with: ExitStatus(unix_wait_status(9))
15702 [8830][2] reports Exit
157032025-01-27T19:39:40.552ZWARNcrucible: client task is sending Done(ReadFailed(Connection reset by peer (os error 131) Stack backtrace: 0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from at /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.92/src/backtrace.rs:27:14 1: <core::task::poll::Poll<core::option::Option<core::result::Result<T,F>>> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869/library/core/src/task/poll.rs:290:44)) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
157042025-01-27T19:39:40.552ZWARNcrucible: downstairs task for 2 stopped due to ReadFailed(Connection reset by peer (os error 131) Stack backtrace: 0: anyhow::error::<impl core::convert::From<E> for anyhow::Error>::from at /home/build/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.92/src/backtrace.rs:27:14 1: <core::task::poll::Poll<core::option::Option<core::result::Result<T,F>>> as core::ops::try_trait::FromResidual<core::result::Result<core::convert::Infallible,E>>>::from_residual at /rustc/9fc6b43126469e3858e2fe86cafb4f0fd5068869/library/core/src/task/poll.rs:290:44) session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
157052025-01-27T19:39:40.552ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
15706 Jan 27 19:39:44.385 INFO request completed, latency_us: 264, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8a9b823c-e628-4133-8eed-077729e8be94, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
15707 001/300 Write block 2347 len 28672 data: 5 5 4 5 5 5 7
15708 002/300 Write block 3561 len 24576 data: 6 7 6 5 5 3
15709 003/300 Read block 2922 len 20480
15710 004/300 Read block 4560 len 12288
15711 005/300 Read block 3744 len 28672
15712 006/300 Write block 2736 len 20480 data: 4 4 3 2 2
15713 007/300 Write block 4442 len 8192 data: 5 3
15714 008/300 Read block 622 len 28672
15715 009/300 Write block 2686 len 8192 data: 7 6
15716 010/300 Write block 470 len 16384 data: 3 3 2 2
15717 011/300 Read block 1762 len 12288
15718 012/300 Flush
15719 013/300 Flush
15720 014/300 Read block 1664 len 24576
15721 015/300 Flush
15722 016/300 Read block 2169 len 16384
15723 017/300 Write block 546 len 36864 data: 6 5 5 5 4 3 6 6 6
15724 018/300 Read block 1824 len 8192
15725 019/300 Read block 4243 len 12288
15726 020/300 Write block 1552 len 16384 data: 3 4 5 5
15727 021/300 Read block 2446 len 4096
15728 022/300 Read block 1435 len 36864
15729 023/300 Write block 1117 len 32768 data: 4 4 3 4 4 6 7 6
15730 024/300 Read block 2343 len 36864
15731 025/300 Read block 3487 len 40960
15732 026/300 Write block 3674 len 24576 data: 3 2 2 2 3 2
15733 027/300 Write block 2256 len 28672 data: 6 5 5 5 5 4 4
15734 028/300 Read block 4183 len 4096
15735 029/300 Read block 2234 len 8192
15736 030/300 Write block 3568 len 28672 data: 5 5 5 5 4 4 4
15737 031/300 Read block 4519 len 4096
15738 032/300 Write block 795 len 40960 data: 5 5 5 4 3 4 3 4 3 2
15739 033/300 Read block 4423 len 40960
15740 034/300 Write block 4279 len 12288 data: 5 5 4
15741 035/300 Read block 3739 len 32768
15742 036/300 Read block 4190 len 20480
15743 037/300 Flush
15744 038/300 Read block 677 len 12288
15745 039/300 Read block 4463 len 32768
15746 040/300 Read block 3189 len 36864
15747 041/300 Write block 16 len 24576 data: 5 5 3 3 3 3
15748 042/300 Read block 1322 len 36864
15749 043/300 Flush
15750 044/300 Read block 3679 len 20480
15751 045/300 Read block 576 len 20480
15752 046/300 Flush
15753 047/300 Read block 4498 len 4096
15754 048/300 Write block 4115 len 24576 data: 3 4 4 6 7 6
15755 049/300 Read block 672 len 28672
15756 050/300 Read block 1819 len 20480
15757 051/300 Write block 619 len 24576 data: 2 2 2 2 3 3
15758 052/300 Read block 96 len 12288
15759 053/300 Write block 4200 len 24576 data: 3 3 3 2 2 2
15760 054/300 Write block 2425 len 28672 data: 4 3 3 4 4 3 2
15761 055/300 Read block 3819 len 24576
15762 056/300 Read block 1299 len 8192
15763 057/300 Read block 3735 len 36864
15764 058/300 Read block 4995 len 8192
15765 059/300 Read block 3168 len 12288
15766 060/300 Flush
15767 061/300 Write block 4773 len 28672 data: 4 5 5 5 4 3 2
15768 062/300 Read block 4139 len 32768
15769 063/300 Write block 4749 len 32768 data: 3 4 3 3 3 3 4 3
15770 064/300 Read block 2839 len 40960
15771 065/300 Write block 4680 len 32768 data: 4 5 4 4 3 3 3 3
15772 066/300 Write block 1301 len 20480 data: 6 5 6 6 5
15773 067/300 Write block 3088 len 4096 data: 2
15774 068/300 Write block 178 len 40960 data: 4 5 4 4 4 5 5 5 5 4
15775 069/300 Write block 3253 len 8192 data: 3 3
15776 070/300 Write block 3725 len 8192 data: 6 6
15777 071/300 Write block 4114 len 36864 data: 3 4 5 5 7 8 7 6 6
15778 072/300 Read block 1243 len 40960
15779 073/300 Read block 1910 len 8192
15780 074/300 Write block 1934 len 8192 data: 2 2
15781 075/300 Read block 4622 len 8192
15782 076/300 Write block 1624 len 40960 data: 4 4 5 8 6 5 4 4 4 4
15783 077/300 Write block 129 len 28672 data: 4 5 6 5 4 3 3
15784 078/300 Read block 4157 len 4096
15785 079/300 Read block 3094 len 12288
15786 080/300 Read block 2387 len 12288
15787 081/300 Read block 243 len 40960
15788 082/300 Read block 2743 len 16384
15789 083/300 Read block 3348 len 12288
15790 084/300 Write block 1114 len 20480 data: 3 3 3 5 5
15791 085/300 Read block 2376 len 40960
15792 086/300 Read block 500 len 24576
15793 087/300 Read block 2127 len 4096
15794 088/300 Write block 3103 len 28672 data: 4 5 5 5 5 5 4
15795 089/300 Read block 423 len 16384
15796 090/300 Write block 583 len 12288 data: 6 3 3
15797 091/300 Read block 776 len 16384
15798 092/300 Read block 2316 len 8192
15799 093/300 Write block 2488 len 8192 data: 3 3
15800 094/300 Write block 4105 len 16384 data: 5 5 6 4
15801 095/300 Write block 758 len 40960 data: 2 2 2 2 2 2 2 3 3 4
15802 096/300 Read block 1862 len 4096
15803 097/300 Write block 3396 len 24576 data: 3 6 6 4 4 4
15804 098/300 Write block 440 len 4096 data: 5
15805 099/300 Write block 2511 len 32768 data: 3 4 4 4 4 4 3 2
15806 100/300 Flush
15807 101/300 Write block 1382 len 24576 data: 5 6 6 4 4 4
15808 102/300 Write block 3426 len 20480 data: 8 7 7 7 7
15809 103/300 Flush
15810 104/300 Write block 115 len 16384 data: 6 6 5 5
15811 105/300 Read block 3159 len 4096
15812 106/300 Read block 405 len 24576
15813 107/300 Read block 2605 len 32768
15814 108/300 Write block 2686 len 24576 data: 8 7 5 2 2 2
15815 109/300 Read block 2876 len 12288
15816 110/300 Read block 510 len 12288
15817 111/300 Write block 1346 len 32768 data: 6 6 6 5 4 3 3 5
15818 112/300 Flush
15819 113/300 Write block 3162 len 12288 data: 3 3 2
15820 114/300 Read block 3409 len 12288
15821 115/300 Read block 45 len 8192
15822 116/300 Write block 4981 len 12288 data: 2 3 3
15823 117/300 Read block 4784 len 24576
15824 118/300 Read block 2584 len 4096
15825 119/300 Read block 3449 len 28672
15826 120/300 Read block 1026 len 8192
15827 121/300 Write block 1405 len 24576 data: 6 6 6 5 5 4
15828 122/300 Read block 3300 len 24576
15829 123/300 Read block 3958 len 8192
15830 124/300 Write block 1251 len 8192 data: 4 4
15831 125/300 Read block 172 len 40960
15832 126/300 Write block 4481 len 20480 data: 5 5 4 3 3
15833 127/300 Read block 3103 len 20480
15834 128/300 Read block 4949 len 36864
15835 129/300 Write block 1233 len 40960 data: 4 7 7 7 6 6 6 6 4 2
15836 130/300 Write block 510 len 24576 data: 3 4 3 3 3 3
15837 131/300 Read block 4575 len 16384
15838 132/300 Read block 3064 len 28672
15839 133/300 Read block 1836 len 28672
15840 134/300 Write block 2450 len 4096 data: 4
15841 135/300 Write block 3002 len 8192 data: 5 4
15842 136/300 Flush
15843 137/300 Write block 302 len 32768 data: 5 4 3 3 3 3 3 3
15844 138/300 Read block 4268 len 24576
15845 139/300 Flush
15846 140/300 Read block 683 len 36864
15847 141/300 Write block 4288 len 32768 data: 3 2 2 2 2 2 3 3
15848 142/300 Read block 1966 len 20480
15849 143/300 Read block 4852 len 20480
15850 144/300 Write block 2283 len 32768 data: 3 3 5 6 7 9 8 8
15851 145/300 Write block 4923 len 36864 data: 7 6 6 5 6 6 5 4 4
15852 146/300 Write block 4094 len 28672 data: 3 3 3 2 2 2 4
15853 147/300 Read block 877 len 24576
15854 148/300 Read block 14 len 32768
15855 149/300 Write block 3432 len 28672 data: 5 5 5 4 4 4 4
15856 150/300 Write block 1605 len 36864 data: 3 4 3 3 3 3 3 2 2
15857 151/300 Write block 4949 len 32768 data: 4 4 4 3 2 3 3 2
15858 152/300 Read block 2511 len 36864
15859 153/300 Read block 1551 len 28672
15860 154/300 Write block 3108 len 16384 data: 6 5 3 2
15861 155/300 Write block 4551 len 40960 data: 3 3 3 3 3 4 3 3 2 3
15862 156/300 Write block 1422 len 24576 data: 4 3 3 4 4 4
15863 157/300 Read block 489 len 20480
15864 158/300 Write block 3098 len 28672 data: 3 3 3 3 3 5 6
15865 159/300 Read block 1895 len 24576
15866 160/300 Read block 14 len 12288
15867 161/300 Write block 4689 len 28672 data: 2 3 3 3 4 4 4
15868 162/300 Read block 104 len 12288
15869 163/300 Write block 1275 len 12288 data: 2 3 3
15870 164/300 Read block 2363 len 32768
15871 165/300 Read block 1796 len 16384
15872 166/300 Write block 2639 len 4096 data: 6
15873 167/300 Write block 1479 len 20480 data: 5 5 5 5 3
15874 168/300 Read block 899 len 4096
15875 169/300 Read block 3799 len 28672
15876 170/300 Read block 1864 len 16384
15877 171/300 Write block 4782 len 8192 data: 2 2
15878 172/300 Read block 2309 len 12288
15879 173/300 Read block 1699 len 40960
15880 174/300 Read block 4137 len 12288
15881 175/300 Write block 2042 len 20480 data: 5 5 4 4 4
15882 176/300 Flush
15883 177/300 Read block 2817 len 8192
15884 178/300 Read block 4347 len 4096
15885 179/300 Flush
15886 180/300 Read block 3576 len 16384
15887 181/300 Read block 185 len 16384
15888 182/300 Read block 2733 len 8192
15889 183/300 Read block 1841 len 32768
15890 184/300 Read block 1118 len 16384
15891 185/300 Read block 2150 len 36864
15892 186/300 Read block 3306 len 28672
15893 187/300 Write block 2164 len 4096 data: 6
15894 188/300 Write block 2574 len 4096 data: 3
15895 189/300 Read block 2875 len 36864
15896 190/300 Read block 2122 len 16384
15897 191/300 Write block 2449 len 24576 data: 4 5 4 3 3 3
15898 192/300 Flush
15899 193/300 Write block 709 len 36864 data: 5 5 6 5 10 9 8 7 5
15900 194/300 Read block 2815 len 24576
15901 195/300 Read block 465 len 28672
15902 196/300 Write block 1525 len 16384 data: 4 3 4 2
15903 197/300 Write block 1558 len 16384 data: 7 7 5 3
15904 198/300 Read block 4765 len 4096
15905 199/300 Read block 4696 len 28672
15906 200/300 Write block 819 len 40960 data: 5 5 4 3 3 3 3 2 2 3
15907 201/300 Write block 621 len 12288 data: 3 3 4
15908 202/300 Write block 502 len 4096 data: 6
15909 203/300 Read block 2243 len 16384
15910 204/300 Write block 3618 len 16384 data: 5 5 5 6
15911 205/300 Write block 2331 len 12288 data: 3 3 3
15912 206/300 Read block 1478 len 8192
15913 207/300 Write block 2176 len 36864 data: 7 6 6 5 4 4 6 5 6
15914 208/300 Read block 3218 len 12288
15915 209/300 Flush
15916 210/300 Write block 4745 len 8192 data: 3 4
15917 211/300 Write block 3958 len 32768 data: 4 5 4 4 5 6 5 7
15918 212/300 Write block 600 len 20480 data: 4 4 5 6 6
15919 213/300 Read block 154 len 8192
15920 214/300 Write block 4263 len 40960 data: 4 4 4 4 3 3 3 3 5 4
15921 215/300 Write block 3032 len 32768 data: 3 3 3 2 3 3 4 4
15922 216/300 Write block 1000 len 8192 data: 3 3
15923 217/300 Read block 3655 len 4096
15924 218/300 Read block 2046 len 16384
15925 219/300 Read block 2288 len 32768
15926 220/300 Write block 2800 len 20480 data: 4 5 5 3 3
15927 221/300 Write block 858 len 32768 data: 3 3 5 6 6 5 5 4
15928 222/300 Read block 649 len 36864
15929 223/300 Write block 4933 len 24576 data: 3 2 3 3 3 4
15930 224/300 Read block 4051 len 24576
15931 225/300 Write block 828 len 32768 data: 4 4 6 6 7 6 6 6
15932 226/300 Write block 321 len 20480 data: 6 4 4 4 4
15933 227/300 Write block 3314 len 4096 data: 4
15934 228/300 Write block 1394 len 4096 data: 5
15935 229/300 Write block 3880 len 12288 data: 6 5 4
15936 230/300 Read block 4944 len 12288
15937 231/300 Write block 2782 len 20480 data: 3 3 3 3 2
15938 232/300 Read block 276 len 36864
15939 233/300 Write block 515 len 36864 data: 4 3 3 3 4 4 3 3 3
15940 234/300 Read block 3551 len 24576
15941 235/300 Read block 730 len 24576
15942 236/300 Read block 874 len 20480
15943 237/300 Read block 2390 len 12288
15944 238/300 Write block 1612 len 12288 data: 3 3 3
15945 239/300 Read block 112 len 24576
15946 240/300 Read block 1386 len 32768
15947 241/300 Read block 1832 len 36864
15948 242/300 Read block 2464 len 40960
15949 243/300 Flush
15950 244/300 Write block 492 len 40960 data: 3 3 4 5 5 7 8 8 6 7
15951 245/300 Read block 4931 len 24576
15952 246/300 Write block 1866 len 8192 data: 4 3
15953 247/300 Write block 4692 len 28672 data: 4 5 5 5 4 4 4
15954 248/300 Read block 4577 len 4096
15955 249/300 Write block 3301 len 36864 data: 3 3 3 3 3 3 2 2 3
15956 250/300 Read block 3705 len 32768
15957 251/300 Read block 2557 len 16384
15958 252/300 Flush
15959 253/300 Write block 2376 len 12288 data: 6 6 8
15960 254/300 Read block 2420 len 8192
15961 255/300 Write block 1397 len 12288 data: 4 5 6
15962 256/300 Read block 3564 len 12288
15963 257/300 Write block 2834 len 36864 data: 4 2 3 4 4 4 4 5 5
15964 258/300 Write block 396 len 20480 data: 4 6 5 5 4
15965 259/300 Write block 87 len 36864 data: 4 5 4 4 3 3 2 3 3
15966 260/300 Write block 1658 len 28672 data: 3 3 3 3 4 4 3
15967 261/300 Read block 3358 len 40960
15968 262/300 Read block 927 len 28672
15969 263/300 Write block 4158 len 28672 data: 3 3 4 6 6 6 6
15970 264/300 Read block 1839 len 40960
15971 265/300 Read block 3532 len 32768
15972 266/300 Read block 3776 len 24576
15973 267/300 Write block 2411 len 40960 data: 3 4 4 5 4 4 3 3 3 3
15974 268/300 Read block 1463 len 12288
15975 269/300 Write block 2962 len 20480 data: 4 4 3 3 3
15976 270/300 Read block 1164 len 8192
15977 271/300 Write block 3597 len 32768 data: 2 2 3 3 2 2 3 4
15978 272/300 Read block 1517 len 40960
15979 273/300 Read block 4053 len 24576
15980 274/300 Flush
15981 275/300 Read block 3506 len 28672
15982 276/300 Read block 744 len 24576
15983 277/300 Write block 3840 len 28672 data: 5 6 7 6 5 5 5
15984 278/300 Write block 1771 len 24576 data: 3 4 4 3 3 3
15985 279/300 Read block 3507 len 20480
15986 280/300 Read block 4878 len 24576
15987 281/300 Write block 3607 len 16384 data: 7 7 5 4
15988 282/300 Read block 2805 len 40960
15989 283/300 Read block 376 len 4096
15990 284/300 Read block 3454 len 16384
15991 285/300 Read block 570 len 24576
15992 286/300 Flush
15993 287/300 Write block 4989 len 20480 data: 5 4 3 3 4
15994 288/300 Write block 3211 len 24576 data: 3 3 3 2 5 5
15995 289/300 Write block 293 len 8192 data: 4 5
15996 290/300 Write block 2713 len 12288 data: 2 2 3
15997 291/300 Write block 1089 len 12288 data: 2 3 3
15998 292/300 Write block 1079 len 24576 data: 5 5 3 3 2 2
15999 293/300 Write block 2316 len 12288 data: 2 2 2
16000 294/300 Write block 4407 len 20480 data: 4 4 3 3 4
16001 295/300 Read block 1957 len 8192
16002 296/300 Write block 1121 len 40960 data: 5 7 8 7 5 6 7 7 8 6
16003 297/300 Write block 2124 len 36864 data: 3 4 3 3 3 3 3 3 3
16004 298/300 Flush
16005 299/300 Write block 1022 len 8192 data: 3 2
16006 300/300 Read block 4205 len 20480
16007 301/300 Read block 2841 len 28672
16008 Main task has work to do, go find it
16009 got dsc Start(2)
16010 start 2
16011 Jan 27 19:39:44.617 INFO request completed, latency_us: 172, response_code: 204, uri: /start/cid/2, method: POST, req_id: 5bbab708-7902-448b-a1a4-29aca33376ef, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
16012 Replay: started 2, returned:Ok(())
16013 ----------------------------------------------------------------
16014 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16015 States: Active Active Offline
16016 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16017 5225 Acked Write 7 Done Done Sent false
16018 5226 Acked Write 6 Done Done Sent false
16019 5227 Acked Read 5 Done Done Sent false
16020 5228 Acked Read 3 Done Done Sent false
16021 5229 Acked Read 7 Done Done Sent false
16022 5230 Acked Write 5 Done Done Sent false
16023 5231 Acked Write 2 Done Done Sent false
16024 5232 Acked Read 7 Done Done Sent false
16025 5233 Acked Write 2 Done Done Sent false
16026 5234 Acked Write 4 Done Done Sent false
16027 5235 Acked Read 3 Done Done Sent false
16028 5236 Acked Flush 0 Done Done Sent false
16029 5237 Acked Flush 0 Done Done Sent false
16030 5238 Acked Read 6 Done Done Sent false
16031 5239 Acked Flush 0 Done Done Sent false
16032 5240 Acked Read 4 Done Done Sent false
16033 5241 Acked Write 9 Done Done Sent false
16034 5242 Acked Read 2 Done Done Sent false
16035 5243 Acked Read 3 Done Done Sent false
16036 5244 Acked Write 4 Done Done Sent false
16037 5245 Acked Read 1 Done Done Sent false
16038 5246 Acked Read 9 Done Done Sent false
16039 5247 Acked Write 8 Done Done Sent false
16040 5248 Acked Read 9 Done Done Sent false
16041 5249 Acked Read 10 Done Done Sent false
16042 5250 Acked Write 6 Done Done Sent false
16043 5251 Acked Write 7 Done Done Sent false
16044 5252 Acked Read 1 Done Done Sent false
16045 5253 Acked Read 2 Done Done Sent false
16046 5254 Acked Write 7 Done Done Sent false
16047 5255 Acked Read 1 Done Done Sent false
16048 5256 Acked Write 10 Done Done Sent false
16049 5257 Acked Read 10 Done Done Sent false
16050 5258 Acked Write 3 Done Done Sent false
16051 5259 Acked Read 8 Done Done Sent false
16052 5260 Acked Read 5 Done Done Sent false
16053 5261 Acked Flush 0 Done Done Sent false
16054 5262 Acked Read 3 Done Done Sent false
16055 5263 Acked Read 8 Done Done Sent false
16056 5264 Acked Read 9 Done Done Sent false
16057 5265 Acked Write 6 Done Done Sent false
16058 5266 Acked Read 9 Done Done Sent false
16059 5267 Acked Flush 0 Done Done Sent false
16060 5268 Acked Read 5 Done Done Sent false
16061 5269 Acked Read 5 Done Done Sent false
16062 5270 Acked Flush 0 Done Done Sent false
16063 5271 Acked Read 1 Done Done Sent false
16064 5272 Acked Write 6 Done Done Sent false
16065 5273 Acked Read 7 Done Done Sent false
16066 5274 Acked Read 5 Done Done Sent false
16067 5275 Acked Write 6 Done Done Sent false
16068 5276 Acked Read 3 Done Done Sent false
16069 5277 Acked Write 6 Done Done Sent false
16070 5278 Acked Write 7 Done Done Sent false
16071 5279 Acked Read 6 Done Done Sent false
16072 5280 Acked Read 2 Done Done Sent false
16073 5281 Acked Read 9 Done Done Sent false
16074 5282 Acked Read 2 Done Done Sent false
16075 5283 Acked Read 3 Done Done Sent false
16076 5284 Acked Flush 0 Done Done Sent false
16077 5285 Acked Write 7 Done Done Sent false
16078 5286 Acked Read 8 Done Done Sent false
16079 5287 Acked Write 8 Done Done Sent false
16080 5288 Acked Read 10 Done Done Sent false
16081 5289 Acked Write 8 Done Done Sent false
16082 5290 Acked Write 5 Done Done Sent false
16083 5291 Acked Write 1 Done Done Sent false
16084 5292 Acked Write 10 Done Done Sent false
16085 5293 Acked Write 2 Done Done Sent false
16086 5294 Acked Write 2 Done Done Sent false
16087 5295 Acked Write 9 Done Done Sent false
16088 5296 Acked Read 10 Done Done Sent false
16089 5297 Acked Read 2 Done Done Sent false
16090 5298 Acked Write 2 Done Done Sent false
16091 5299 Acked Read 2 Done Done Sent false
16092 5300 Acked Write 10 Done Done Sent false
16093 5301 Acked Write 7 Done Done Sent false
16094 5302 Acked Read 1 Done Done Sent false
16095 5303 Acked Read 3 Done Done Sent false
16096 5304 Acked Read 3 Done Done Sent false
16097 5305 Acked Read 10 Done Done Sent false
16098 5306 Acked Read 4 Done Done Sent false
16099 5307 Acked Read 3 Done Done Sent false
16100 5308 Acked Write 5 Done Done Sent false
16101 5309 Acked Read 10 Done Done Sent false
16102 5310 Acked Read 6 Done Done Sent false
16103 5311 Acked Read 1 Done Done Sent false
16104 5312 Acked Write 7 Done Done Sent false
16105 5313 Acked Read 4 Done Done Sent false
16106 5314 Acked Write 3 Done Done Sent false
16107 5315 Acked Read 4 Done Done Sent false
16108 5316 Acked Read 2 Done Done Sent false
16109 5317 Acked Write 2 Done Done Sent false
16110 5318 Acked Write 4 Done Done Sent false
16111 5319 Acked Write 10 Done Done Sent false
16112 5320 Acked Read 1 Done Done Sent false
16113 5321 Acked Write 6 Done Done Sent false
16114 5322 Acked Write 1 Done Done Sent false
16115 5323 Acked Write 8 Done Done Sent false
16116 5324 Acked Flush 0 Done Done Sent false
16117 5325 Acked Write 6 Done Done Sent false
16118 5326 Acked Write 5 Done Done Sent false
16119 5327 Acked Flush 0 Done Done Sent false
16120 5328 Acked Write 4 Done Done Sent false
16121 5329 Acked Read 1 Done Done Sent false
16122 5330 Acked Read 6 Done Done Sent false
16123 5331 Acked Read 8 Done Done Sent false
16124 5332 Acked Write 6 Done Done Sent false
16125 5333 Acked Read 3 Done Done Sent false
16126 5334 Acked Read 3 Done Done Sent false
16127 5335 Acked Write 8 Done Done Sent false
16128 5336 Acked Flush 0 Done Done Sent false
16129 5337 Acked Write 3 Done Done Sent false
16130 5338 Acked Read 3 Done Done Sent false
16131 5339 Acked Read 2 Done Done Sent false
16132 5340 Acked Write 3 Done Done Sent false
16133 5341 Acked Read 6 Done Done Sent false
16134 5342 Acked Read 1 Done Done Sent false
16135 5343 Acked Read 7 Done Done Sent false
16136 5344 Acked Read 2 Done Done Sent false
16137 5345 Acked Write 6 Done Done Sent false
16138 5346 Acked Read 6 Done Done Sent false
16139 5347 Acked Read 2 Done Done Sent false
16140 5348 Acked Write 2 Done Done Sent false
16141 5349 Acked Read 10 Done Done Sent false
16142 5350 Acked Write 5 Done Done Sent false
16143 5351 Acked Read 5 Done Done Sent false
16144 5352 Acked Read 9 Done Done Sent false
16145 5353 Acked Write 10 Done Done Sent false
16146 5354 Acked Write 6 Done Done Sent false
16147 5355 Acked Read 4 Done Done Sent false
16148 5356 Acked Read 7 Done Done Sent false
16149 5357 Acked Read 7 Done Done Sent false
16150 5358 Acked Write 1 Done Done Sent false
16151 5359 Acked Write 2 Done Done Sent false
16152 5360 Acked Flush 0 Done Done Sent false
16153 5361 Acked Write 8 Done Done Sent false
16154 5362 Acked Read 6 Done Done Sent false
16155 5363 Acked Flush 0 Done Done Sent false
16156 5364 Acked Read 9 Done Done Sent false
16157 5365 Acked Write 8 Done Done Sent false
16158 5366 Acked Read 5 Done Done Sent false
16159 5367 Acked Read 5 Done Done Sent false
16160 5368 Acked Write 8 Done Done Sent false
16161 5369 Acked Write 9 Done Done Sent false
16162 5370 Acked Write 7 Done Done Sent false
16163 5371 Acked Read 6 Done Done Sent false
16164 5372 Acked Read 8 Done Done Sent false
16165 5373 Acked Write 7 Done Done Sent false
16166 5374 Acked Write 9 Done Done Sent false
16167 5375 Acked Write 8 Done Done Sent false
16168 5376 Acked Read 9 Done Done Sent false
16169 5377 Acked Read 7 Done Done Sent false
16170 5378 Acked Write 4 Done Done Sent false
16171 5379 Acked Write 10 Done Done Sent false
16172 5380 Acked Write 6 Done Done Sent false
16173 5381 Acked Read 5 Done Done Sent false
16174 5382 Acked Write 7 Done Done Sent false
16175 5383 Acked Read 6 Done Done Sent false
16176 5384 Acked Read 3 Done Done Sent false
16177 5385 Acked Write 7 Done Done Sent false
16178 5386 Acked Read 3 Done Done Sent false
16179 5387 Acked Write 3 Done Done Sent false
16180 5388 Acked Read 8 Done Done Sent false
16181 5389 Acked Read 4 Done Done Sent false
16182 5390 Acked Write 1 Done Done Sent false
16183 5391 Acked Write 5 Done Done Sent false
16184 5392 Acked Read 1 Done Done Sent false
16185 5393 Acked Read 7 Done Done Sent false
16186 5394 Acked Read 4 Done Done Sent false
16187 5395 Acked Write 2 Done Done Sent false
16188 5396 Acked Read 3 Done Done Sent false
16189 5397 Acked Read 10 Done Done Sent false
16190 5398 Acked Read 3 Done Done Sent false
16191 5399 Acked Write 5 Done Done Sent false
16192 5400 Acked Flush 0 Done Done Sent false
16193 5401 Acked Read 2 Done Done Sent false
16194 5402 Acked Read 1 Done Done Sent false
16195 5403 Acked Flush 0 Done Done Sent false
16196 5404 Acked Read 4 Done Done Sent false
16197 5405 Acked Read 4 Done Done Sent false
16198 5406 Acked Read 2 Done Done Sent false
16199 5407 Acked Read 8 Done Done Sent false
16200 5408 Acked Read 4 Done Done Sent false
16201 5409 Acked Read 9 Done Done Sent false
16202 5410 Acked Read 7 Done Done Sent false
16203 5411 Acked Write 1 Done Done Sent false
16204 5412 Acked Write 1 Done Done Sent false
16205 5413 Acked Read 9 Done Done Sent false
16206 5414 Acked Read 4 Done Done Sent false
16207 5415 Acked Write 6 Done Done Sent false
16208 5416 Acked Flush 0 Done Done Sent false
16209 5417 Acked Write 9 Done Done Sent false
16210 5418 Acked Read 6 Done Done Sent false
16211 5419 Acked Read 7 Done Done Sent false
16212 5420 Acked Write 4 Done Done Sent false
16213 5421 Acked Write 4 Done Done Sent false
16214 5422 Acked Read 1 Done Done Sent false
16215 5423 Acked Read 7 Done Done Sent false
16216 5424 Acked Write 10 Done Done Sent false
16217 5425 Acked Write 3 Done Done Sent false
16218 5426 Acked Write 1 Done Done Sent false
16219 5427 Acked Read 4 Done Done Sent false
16220 5428 Acked Write 4 Done Done Sent false
16221 5429 Acked Write 3 Done Done Sent false
16222 5430 Acked Read 2 Done Done Sent false
16223 5431 Acked Write 9 Done Done Sent false
16224 5432 Acked Read 3 Done Done Sent false
16225 5433 Acked Flush 0 Done Done Sent false
16226 5434 Acked Write 2 Done Done Sent false
16227 5435 Acked Write 8 Done Done Sent false
16228 5436 Acked Write 5 Done Done Sent false
16229 5437 Acked Read 2 Done Done Sent false
16230 5438 Acked Write 10 Done Done Sent false
16231 5439 Acked Write 8 Done Done Sent false
16232 5440 Acked Write 2 Done Done Sent false
16233 5441 Acked Read 1 Done Done Sent false
16234 5442 Acked Read 4 Done Done Sent false
16235 5443 Acked Read 8 Done Done Sent false
16236 5444 Acked Write 5 Done Done Sent false
16237 5445 Acked Write 8 Done Done Sent false
16238 5446 Acked Read 9 Done Done Sent false
16239 5447 Acked Write 6 Done Done Sent false
16240 5448 Acked Read 6 Done Done Sent false
16241 5449 Acked Write 8 Done Done Sent false
16242 5450 Acked Write 5 Done Done Sent false
16243 5451 Acked Write 1 Done Done Sent false
16244 5452 Acked Write 1 Done Done Sent false
16245 5453 Acked Write 3 Done Done Sent false
16246 5454 Acked Read 3 Done Done Sent false
16247 5455 Acked Write 5 Done Done Sent false
16248 5456 Acked Read 9 Done Done Sent false
16249 5457 Acked Write 9 Done Done Sent false
16250 5458 Acked Read 6 Done Done Sent false
16251 5459 Acked Read 6 Done Done Sent false
16252 5460 Acked Read 5 Done Done Sent false
16253 5461 Acked Read 3 Done Done Sent false
16254 5462 Acked Write 3 Done Done Sent false
16255 5463 Acked Read 6 Done Done Sent false
16256 5464 Acked Read 8 Done Done Sent false
16257 5465 Acked Read 9 Done Done Sent false
16258 5466 Acked Read 10 Done Done Sent false
16259 5467 Acked Flush 0 Done Done Sent false
16260 5468 Acked Write 10 Done Done Sent false
16261 5469 Acked Read 6 Done Done Sent false
16262 5470 Acked Write 2 Done Done Sent false
16263 5471 Acked Write 7 Done Done Sent false
16264 5472 Acked Read 1 Done Done Sent false
16265 5473 Acked Write 9 Done Done Sent false
16266 5474 Acked Read 8 Done Done Sent false
16267 5475 Acked Read 4 Done Done Sent false
16268 5476 Acked Flush 0 Done Done Sent false
16269 5477 Acked Write 3 Done Done Sent false
16270 5478 Acked Read 2 Done Done Sent false
16271 5479 Acked Write 3 Done Done Sent false
16272 5480 Acked Read 3 Done Done Sent false
16273 5481 Acked Write 9 Done Done Sent false
16274 5482 Acked Write 5 Done Done Sent false
16275 5483 Acked Write 9 Done Done Sent false
16276 5484 Acked Write 7 Done Done Sent false
16277 5485 Acked Read 10 Done Done Sent false
16278 5486 Acked Read 7 Done Done Sent false
16279 5487 Acked Write 7 Done Done Sent false
16280 5488 Acked Read 10 Done Done Sent false
16281 5489 Acked Read 8 Done Done Sent false
16282 5490 Acked Read 6 Done Done Sent false
16283 5491 Acked Write 10 Done Done Sent false
16284 5492 Acked Read 3 Done Done Sent false
16285 5493 Acked Write 5 Done Done Sent false
16286 5494 Acked Read 2 Done Done Sent false
16287 5495 Acked Write 8 Done Done Sent false
16288 5496 Acked Read 10 Done Done Sent false
16289 5497 Acked Read 6 Done Done Sent false
16290 5498 Acked Flush 0 Done Done Sent false
16291 5499 Acked Read 7 Done Done Sent false
16292 5500 Acked Read 6 Done Done Sent false
16293 5501 Acked Write 7 Done Done Sent false
16294 5502 Acked Write 6 Done Done Sent false
16295 5503 Acked Read 5 Done Done Sent false
16296 5504 Acked Read 6 Done Done Sent false
16297 5505 Acked Write 4 Done Done Sent false
16298 5506 Acked Read 10 Done Done Sent false
16299 5507 Acked Read 1 Done Done Sent false
16300 5508 Acked Read 4 Done Done Sent false
16301 5509 Acked Read 6 Done Done Sent false
16302 5510 Acked Flush 0 Done Done Sent false
16303 5511 Acked Write 5 Done Done Sent false
16304 5512 Acked Write 6 Done Done Sent false
16305 5513 Acked Write 2 Done Done Sent false
16306 5514 Acked Write 3 Done Done Sent false
16307 5515 Acked Write 3 Done Done Sent false
16308 5516 Acked Write 6 Done Done Sent false
16309 5517 Acked Write 3 Done Done Sent false
16310 5518 Acked Write 5 Done Done Sent false
16311 5519 Acked Read 2 Done Done Sent false
16312 5520 Acked Write 10 Done Done Sent false
16313 5521 Acked Write 9 Done Done Sent false
16314 5522 Acked Flush 0 Done Done Sent false
16315 5523 Acked Write 2 Done Done Sent false
16316 5524 Acked Read 5 Done Done Sent false
16317 5525 Acked Read 7 Done Done Sent false
16318 STATES DS:0 DS:1 DS:2 TOTAL
16319 Sent 0 0 301 301
16320 Done 301 301 0 602
16321 Skipped 0 0 0 0
16322 Error 0 0 0 0
16323 Last Flush: 5522 5522 5224
16324 Downstairs last five completed: 5224 5223 5222 5221 5220
16325 Upstairs last five completed: 5525 5524 5523 5522 5521
16326 CLIENT: Up:0 ds:301 act:2
16327 [8830] got start action
16328 [8830] I am going to restart
16329 Starting downstairs at port 8830
16330 Make output file at "/tmp/dsc/downstairs-8830.txt"
16331 [8830][2] reports Starting
16332 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1215)
16333 [8830][2] reports Running
16334 ----------------------------------------------------------------
16335 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
16336 States: Active Active Offline
16337 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16338 5225 Acked Write 7 Done Done Sent false
16339 5226 Acked Write 6 Done Done Sent false
16340 5227 Acked Read 5 Done Done Sent false
16341 5228 Acked Read 3 Done Done Sent false
16342 5229 Acked Read 7 Done Done Sent false
16343 5230 Acked Write 5 Done Done Sent false
16344 5231 Acked Write 2 Done Done Sent false
16345 5232 Acked Read 7 Done Done Sent false
16346 5233 Acked Write 2 Done Done Sent false
16347 5234 Acked Write 4 Done Done Sent false
16348 5235 Acked Read 3 Done Done Sent false
16349 5236 Acked Flush 0 Done Done Sent false
16350 5237 Acked Flush 0 Done Done Sent false
16351 5238 Acked Read 6 Done Done Sent false
16352 5239 Acked Flush 0 Done Done Sent false
16353 5240 Acked Read 4 Done Done Sent false
16354 5241 Acked Write 9 Done Done Sent false
16355 5242 Acked Read 2 Done Done Sent false
16356 5243 Acked Read 3 Done Done Sent false
16357 5244 Acked Write 4 Done Done Sent false
16358 5245 Acked Read 1 Done Done Sent false
16359 5246 Acked Read 9 Done Done Sent false
16360 5247 Acked Write 8 Done Done Sent false
16361 5248 Acked Read 9 Done Done Sent false
16362 5249 Acked Read 10 Done Done Sent false
16363 5250 Acked Write 6 Done Done Sent false
16364 5251 Acked Write 7 Done Done Sent false
16365 5252 Acked Read 1 Done Done Sent false
16366 5253 Acked Read 2 Done Done Sent false
16367 5254 Acked Write 7 Done Done Sent false
16368 5255 Acked Read 1 Done Done Sent false
16369 5256 Acked Write 10 Done Done Sent false
16370 5257 Acked Read 10 Done Done Sent false
16371 5258 Acked Write 3 Done Done Sent false
16372 5259 Acked Read 8 Done Done Sent false
16373 5260 Acked Read 5 Done Done Sent false
16374 5261 Acked Flush 0 Done Done Sent false
16375 5262 Acked Read 3 Done Done Sent false
16376 5263 Acked Read 8 Done Done Sent false
16377 5264 Acked Read 9 Done Done Sent false
16378 5265 Acked Write 6 Done Done Sent false
16379 5266 Acked Read 9 Done Done Sent false
16380 5267 Acked Flush 0 Done Done Sent false
16381 5268 Acked Read 5 Done Done Sent false
16382 5269 Acked Read 5 Done Done Sent false
16383 5270 Acked Flush 0 Done Done Sent false
16384 5271 Acked Read 1 Done Done Sent false
16385 5272 Acked Write 6 Done Done Sent false
16386 5273 Acked Read 7 Done Done Sent false
16387 5274 Acked Read 5 Done Done Sent false
16388 5275 Acked Write 6 Done Done Sent false
16389 5276 Acked Read 3 Done Done Sent false
16390 5277 Acked Write 6 Done Done Sent false
16391 5278 Acked Write 7 Done Done Sent false
16392 5279 Acked Read 6 Done Done Sent false
16393 5280 Acked Read 2 Done Done Sent false
16394 5281 Acked Read 9 Done Done Sent false
16395 5282 Acked Read 2 Done Done Sent false
16396 5283 Acked Read 3 Done Done Sent false
16397 5284 Acked Flush 0 Done Done Sent false
16398 5285 Acked Write 7 Done Done Sent false
16399 5286 Acked Read 8 Done Done Sent false
16400 5287 Acked Write 8 Done Done Sent false
16401 5288 Acked Read 10 Done Done Sent false
16402 5289 Acked Write 8 Done Done Sent false
16403 5290 Acked Write 5 Done Done Sent false
16404 5291 Acked Write 1 Done Done Sent false
16405 5292 Acked Write 10 Done Done Sent false
16406 5293 Acked Write 2 Done Done Sent false
16407 5294 Acked Write 2 Done Done Sent false
16408 5295 Acked Write 9 Done Done Sent false
16409 5296 Acked Read 10 Done Done Sent false
16410 5297 Acked Read 2 Done Done Sent false
16411 5298 Acked Write 2 Done Done Sent false
16412 5299 Acked Read 2 Done Done Sent false
16413 5300 Acked Write 10 Done Done Sent false
16414 5301 Acked Write 7 Done Done Sent false
16415 5302 Acked Read 1 Done Done Sent false
16416 5303 Acked Read 3 Done Done Sent false
16417 5304 Acked Read 3 Done Done Sent false
16418 5305 Acked Read 10 Done Done Sent false
16419 5306 Acked Read 4 Done Done Sent false
16420 5307 Acked Read 3 Done Done Sent false
16421 5308 Acked Write 5 Done Done Sent false
16422 5309 Acked Read 10 Done Done Sent false
16423 5310 Acked Read 6 Done Done Sent false
16424 5311 Acked Read 1 Done Done Sent false
16425 5312 Acked Write 7 Done Done Sent false
16426 5313 Acked Read 4 Done Done Sent false
16427 5314 Acked Write 3 Done Done Sent false
16428 5315 Acked Read 4 Done Done Sent false
16429 5316 Acked Read 2 Done Done Sent false
16430 5317 Acked Write 2 Done Done Sent false
16431 5318 Acked Write 4 Done Done Sent false
16432 5319 Acked Write 10 Done Done Sent false
16433 5320 Acked Read 1 Done Done Sent false
16434 5321 Acked Write 6 Done Done Sent false
16435 5322 Acked Write 1 Done Done Sent false
16436 5323 Acked Write 8 Done Done Sent false
16437 5324 Acked Flush 0 Done Done Sent false
16438 5325 Acked Write 6 Done Done Sent false
16439 5326 Acked Write 5 Done Done Sent false
16440 5327 Acked Flush 0 Done Done Sent false
16441 5328 Acked Write 4 Done Done Sent false
16442 5329 Acked Read 1 Done Done Sent false
16443 5330 Acked Read 6 Done Done Sent false
16444 5331 Acked Read 8 Done Done Sent false
16445 5332 Acked Write 6 Done Done Sent false
16446 5333 Acked Read 3 Done Done Sent false
16447 5334 Acked Read 3 Done Done Sent false
16448 5335 Acked Write 8 Done Done Sent false
16449 5336 Acked Flush 0 Done Done Sent false
16450 5337 Acked Write 3 Done Done Sent false
16451 5338 Acked Read 3 Done Done Sent false
16452 5339 Acked Read 2 Done Done Sent false
16453 5340 Acked Write 3 Done Done Sent false
16454 5341 Acked Read 6 Done Done Sent false
16455 5342 Acked Read 1 Done Done Sent false
16456 5343 Acked Read 7 Done Done Sent false
16457 5344 Acked Read 2 Done Done Sent false
16458 5345 Acked Write 6 Done Done Sent false
16459 5346 Acked Read 6 Done Done Sent false
16460 5347 Acked Read 2 Done Done Sent false
16461 5348 Acked Write 2 Done Done Sent false
16462 5349 Acked Read 10 Done Done Sent false
16463 5350 Acked Write 5 Done Done Sent false
16464 5351 Acked Read 5 Done Done Sent false
16465 5352 Acked Read 9 Done Done Sent false
16466 5353 Acked Write 10 Done Done Sent false
16467 5354 Acked Write 6 Done Done Sent false
16468 5355 Acked Read 4 Done Done Sent false
16469 5356 Acked Read 7 Done Done Sent false
16470 5357 Acked Read 7 Done Done Sent false
16471 5358 Acked Write 1 Done Done Sent false
16472 5359 Acked Write 2 Done Done Sent false
16473 5360 Acked Flush 0 Done Done Sent false
16474 5361 Acked Write 8 Done Done Sent false
16475 5362 Acked Read 6 Done Done Sent false
16476 5363 Acked Flush 0 Done Done Sent false
16477 5364 Acked Read 9 Done Done Sent false
16478 5365 Acked Write 8 Done Done Sent false
16479 5366 Acked Read 5 Done Done Sent false
16480 5367 Acked Read 5 Done Done Sent false
16481 5368 Acked Write 8 Done Done Sent false
16482 5369 Acked Write 9 Done Done Sent false
16483 5370 Acked Write 7 Done Done Sent false
16484 5371 Acked Read 6 Done Done Sent false
16485 5372 Acked Read 8 Done Done Sent false
16486 5373 Acked Write 7 Done Done Sent false
16487 5374 Acked Write 9 Done Done Sent false
16488 5375 Acked Write 8 Done Done Sent false
16489 5376 Acked Read 9 Done Done Sent false
16490 5377 Acked Read 7 Done Done Sent false
16491 5378 Acked Write 4 Done Done Sent false
16492 5379 Acked Write 10 Done Done Sent false
16493 5380 Acked Write 6 Done Done Sent false
16494 5381 Acked Read 5 Done Done Sent false
16495 5382 Acked Write 7 Done Done Sent false
16496 5383 Acked Read 6 Done Done Sent false
16497 5384 Acked Read 3 Done Done Sent false
16498 5385 Acked Write 7 Done Done Sent false
16499 5386 Acked Read 3 Done Done Sent false
16500 5387 Acked Write 3 Done Done Sent false
16501 5388 Acked Read 8 Done Done Sent false
16502 5389 Acked Read 4 Done Done Sent false
16503 5390 Acked Write 1 Done Done Sent false
16504 5391 Acked Write 5 Done Done Sent false
16505 5392 Acked Read 1 Done Done Sent false
16506 5393 Acked Read 7 Done Done Sent false
16507 5394 Acked Read 4 Done Done Sent false
16508 5395 Acked Write 2 Done Done Sent false
16509 5396 Acked Read 3 Done Done Sent false
16510 5397 Acked Read 10 Done Done Sent false
16511 5398 Acked Read 3 Done Done Sent false
16512 5399 Acked Write 5 Done Done Sent false
16513 5400 Acked Flush 0 Done Done Sent false
16514 5401 Acked Read 2 Done Done Sent false
16515 5402 Acked Read 1 Done Done Sent false
16516 5403 Acked Flush 0 Done Done Sent false
16517 5404 Acked Read 4 Done Done Sent false
16518 5405 Acked Read 4 Done Done Sent false
16519 5406 Acked Read 2 Done Done Sent false
16520 5407 Acked Read 8 Done Done Sent false
16521 5408 Acked Read 4 Done Done Sent false
16522 5409 Acked Read 9 Done Done Sent false
16523 5410 Acked Read 7 Done Done Sent false
16524 5411 Acked Write 1 Done Done Sent false
16525 5412 Acked Write 1 Done Done Sent false
16526 5413 Acked Read 9 Done Done Sent false
16527 5414 Acked Read 4 Done Done Sent false
16528 5415 Acked Write 6 Done Done Sent false
16529 5416 Acked Flush 0 Done Done Sent false
16530 5417 Acked Write 9 Done Done Sent false
16531 5418 Acked Read 6 Done Done Sent false
16532 5419 Acked Read 7 Done Done Sent false
16533 5420 Acked Write 4 Done Done Sent false
16534 5421 Acked Write 4 Done Done Sent false
16535 5422 Acked Read 1 Done Done Sent false
16536 5423 Acked Read 7 Done Done Sent false
16537 5424 Acked Write 10 Done Done Sent false
16538 5425 Acked Write 3 Done Done Sent false
16539 5426 Acked Write 1 Done Done Sent false
16540 5427 Acked Read 4 Done Done Sent false
16541 5428 Acked Write 4 Done Done Sent false
16542 5429 Acked Write 3 Done Done Sent false
16543 5430 Acked Read 2 Done Done Sent false
16544 5431 Acked Write 9 Done Done Sent false
16545 5432 Acked Read 3 Done Done Sent false
16546 5433 Acked Flush 0 Done Done Sent false
16547 5434 Acked Write 2 Done Done Sent false
16548 5435 Acked Write 8 Done Done Sent false
16549 5436 Acked Write 5 Done Done Sent false
16550 5437 Acked Read 2 Done Done Sent false
16551 5438 Acked Write 10 Done Done Sent false
16552 5439 Acked Write 8 Done Done Sent false
16553 5440 Acked Write 2 Done Done Sent false
16554 5441 Acked Read 1 Done Done Sent false
16555 5442 Acked Read 4 Done Done Sent false
16556 5443 Acked Read 8 Done Done Sent false
16557 5444 Acked Write 5 Done Done Sent false
16558 5445 Acked Write 8 Done Done Sent false
16559 5446 Acked Read 9 Done Done Sent false
16560 5447 Acked Write 6 Done Done Sent false
16561 5448 Acked Read 6 Done Done Sent false
16562 5449 Acked Write 8 Done Done Sent false
16563 5450 Acked Write 5 Done Done Sent false
16564 5451 Acked Write 1 Done Done Sent false
16565 5452 Acked Write 1 Done Done Sent false
16566 5453 Acked Write 3 Done Done Sent false
16567 5454 Acked Read 3 Done Done Sent false
16568 5455 Acked Write 5 Done Done Sent false
16569 5456 Acked Read 9 Done Done Sent false
16570 5457 Acked Write 9 Done Done Sent false
16571 5458 Acked Read 6 Done Done Sent false
16572 5459 Acked Read 6 Done Done Sent false
16573 5460 Acked Read 5 Done Done Sent false
16574 5461 Acked Read 3 Done Done Sent false
16575 5462 Acked Write 3 Done Done Sent false
16576 5463 Acked Read 6 Done Done Sent false
16577 5464 Acked Read 8 Done Done Sent false
16578 5465 Acked Read 9 Done Done Sent false
16579 5466 Acked Read 10 Done Done Sent false
16580 5467 Acked Flush 0 Done Done Sent false
16581 5468 Acked Write 10 Done Done Sent false
16582 5469 Acked Read 6 Done Done Sent false
16583 5470 Acked Write 2 Done Done Sent false
16584 5471 Acked Write 7 Done Done Sent false
16585 5472 Acked Read 1 Done Done Sent false
16586 5473 Acked Write 9 Done Done Sent false
16587 5474 Acked Read 8 Done Done Sent false
16588 5475 Acked Read 4 Done Done Sent false
16589 5476 Acked Flush 0 Done Done Sent false
16590 5477 Acked Write 3 Done Done Sent false
16591 5478 Acked Read 2 Done Done Sent false
16592 5479 Acked Write 3 Done Done Sent false
16593 5480 Acked Read 3 Done Done Sent false
16594 5481 Acked Write 9 Done Done Sent false
16595 5482 Acked Write 5 Done Done Sent false
16596 5483 Acked Write 9 Done Done Sent false
16597 5484 Acked Write 7 Done Done Sent false
16598 5485 Acked Read 10 Done Done Sent false
16599 5486 Acked Read 7 Done Done Sent false
16600 5487 Acked Write 7 Done Done Sent false
16601 5488 Acked Read 10 Done Done Sent false
16602 5489 Acked Read 8 Done Done Sent false
16603 5490 Acked Read 6 Done Done Sent false
16604 5491 Acked Write 10 Done Done Sent false
16605 5492 Acked Read 3 Done Done Sent false
16606 5493 Acked Write 5 Done Done Sent false
16607 5494 Acked Read 2 Done Done Sent false
16608 5495 Acked Write 8 Done Done Sent false
16609 5496 Acked Read 10 Done Done Sent false
16610 5497 Acked Read 6 Done Done Sent false
16611 5498 Acked Flush 0 Done Done Sent false
16612 5499 Acked Read 7 Done Done Sent false
16613 5500 Acked Read 6 Done Done Sent false
16614 5501 Acked Write 7 Done Done Sent false
16615 5502 Acked Write 6 Done Done Sent false
16616 5503 Acked Read 5 Done Done Sent false
16617 5504 Acked Read 6 Done Done Sent false
16618 5505 Acked Write 4 Done Done Sent false
16619 5506 Acked Read 10 Done Done Sent false
16620 5507 Acked Read 1 Done Done Sent false
16621 5508 Acked Read 4 Done Done Sent false
16622 5509 Acked Read 6 Done Done Sent false
16623 5510 Acked Flush 0 Done Done Sent false
16624 5511 Acked Write 5 Done Done Sent false
16625 5512 Acked Write 6 Done Done Sent false
16626 5513 Acked Write 2 Done Done Sent false
16627 5514 Acked Write 3 Done Done Sent false
16628 5515 Acked Write 3 Done Done Sent false
16629 5516 Acked Write 6 Done Done Sent false
16630 5517 Acked Write 3 Done Done Sent false
16631 5518 Acked Write 5 Done Done Sent false
16632 5519 Acked Read 2 Done Done Sent false
16633 5520 Acked Write 10 Done Done Sent false
16634 5521 Acked Write 9 Done Done Sent false
16635 5522 Acked Flush 0 Done Done Sent false
16636 5523 Acked Write 2 Done Done Sent false
16637 5524 Acked Read 5 Done Done Sent false
16638 5525 Acked Read 7 Done Done Sent false
16639 5526 Acked Flush 0 Done Done Sent false
16640 STATES DS:0 DS:1 DS:2 TOTAL
16641 Sent 0 0 302 302
16642 Done 302 302 0 604
16643 Skipped 0 0 0 0
16644 Error 0 0 0 0
16645 Last Flush: 5526 5526 5224
16646 Downstairs last five completed: 5224 5223 5222 5221 5220
16647 Upstairs last five completed: 5526 5525 5524 5523 5522
16648 CLIENT: Up:0 ds:302 act:2
166492025-01-27T19:39:50.553ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166502025-01-27T19:39:50.553ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:61193) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166512025-01-27T19:39:50.554ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166522025-01-27T19:39:50.555ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166532025-01-27T19:39:50.555ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166542025-01-27T19:39:50.555ZINFOcrucible: send last flush ID to this DS: 5224 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166552025-01-27T19:39:50.555ZINFOcrucible: Replied this last flush ID: 5224 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166562025-01-27T19:39:50.555ZINFOcrucible: [2] client re-new 302 jobs since flush 5224 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166572025-01-27T19:39:50.555ZINFOcrucible: [2] Replayed 302 jobs since flush: 5224 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
16658 ----------------------------------------------------------------
16659 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
16660 Downstairs last five completed: 5526 5525 5524 5523 5522
16661 Upstairs last five completed: 5526 5525 5524 5523 5522
16662 CLIENT: Up:0 ds:0 act:3
16663 Replay: All jobs finished
16664 Main task has work to do, go find it
16665 got dsc Stop(2)
16666 stop 2
16667 Jan 27 19:39:52.622 [8830] Got stop action so:false kr:false
16668 INFO request completed, latency_us: 334, response_code: 204, uri: /stop/cid/2, method: POST, req_id: a9a37ef1-36a9-4c1c-b5e7-b895f135150a, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
166692025-01-27T19:39:52.623ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166702025-01-27T19:39:52.623ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
166712025-01-27T19:39:52.623ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
16672 Jan 27 19:39:52.623 INFO request completed, latency_us: 133, response_code: 200, uri: /state/cid/2, method: GET, req_id: 21c3b959-7133-469a-890a-7b913a4497b7, remote_addr: 127.0.0.1:55047{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-27T19:39:52.623394776Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
16673 , local_addr: 127.0.0.1:9998
16674 [8830] Exited with: ExitStatus(unix_wait_status(9))
16675 [8830][2] reports Exit
16676 Jan 27 19:39:56.625 INFO request completed, latency_us: 218, response_code: 200, uri: /state/cid/2, method: GET, req_id: 97dc4fef-f831-48f5-8c3c-41dc0d792140, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
16677 001/300 Read block 2676 len 16384
16678 002/300 Read block 470 len 16384
16679 003/300 Write block 2035 len 36864 data: 4 4 4 4 3 3 4 6 6
16680 004/300 Read block 1493 len 32768
16681 005/300 Read block 4233 len 28672
16682 006/300 Read block 4486 len 12288
16683 007/300 Flush
16684 008/300 Read block 3565 len 16384
16685 009/300 Read block 4957 len 32768
16686 010/300 Write block 1294 len 20480 data: 4 4 4 5 6
16687 011/300 Write block 4411 len 16384 data: 5 4 5 3
16688 012/300 Read block 2504 len 4096
16689 013/300 Write block 593 len 28672 data: 5 4 3 3 2 4 4
16690 014/300 Read block 799 len 32768
16691 015/300 Read block 3325 len 16384
16692 016/300 Read block 2778 len 28672
16693 017/300 Read block 2716 len 12288
16694 018/300 Read block 2922 len 36864
16695 019/300 Write block 2611 len 4096 data: 5
16696 020/300 Write block 1809 len 36864 data: 5 7 8 7 5 6 8 7 6
16697 021/300 Write block 3104 len 40960 data: 7 6 6 6 7 6 4 3 2 2
16698 022/300 Write block 4649 len 20480 data: 3 2 3 3 3
16699 023/300 Write block 1785 len 4096 data: 5
16700 024/300 Read block 2294 len 16384
16701 025/300 Read block 1675 len 12288
16702 026/300 Read block 1420 len 16384
16703 027/300 Read block 4646 len 20480
16704 028/300 Read block 132 len 24576
16705 029/300 Read block 710 len 16384
16706 030/300 Read block 1020 len 4096
16707 031/300 Read block 776 len 20480
16708 032/300 Read block 723 len 20480
16709 033/300 Flush
16710 034/300 Write block 4030 len 4096 data: 2
16711 035/300 Read block 484 len 4096
16712 036/300 Read block 2386 len 8192
16713 037/300 Read block 3030 len 20480
16714 038/300 Flush
16715 039/300 Write block 2727 len 4096 data: 3
16716 040/300 Read block 3738 len 32768
16717 041/300 Read block 2652 len 12288
16718 042/300 Write block 2867 len 28672 data: 4 3 3 4 3 4 4
16719 043/300 Write block 1374 len 36864 data: 6 7 8 8 7 7 6 5 6
16720 044/300 Read block 2378 len 24576
16721 045/300 Write block 4075 len 4096 data: 2
16722 046/300 Flush
16723 047/300 Read block 2066 len 8192
16724 048/300 Write block 2279 len 12288 data: 2 2 3
16725 049/300 Write block 4738 len 24576 data: 4 4 5 5 4 3
16726 050/300 Read block 706 len 28672
16727 051/300 Flush
16728 052/300 Flush
16729 053/300 Write block 2006 len 16384 data: 3 3 3 3
16730 054/300 Read block 4019 len 4096
16731 055/300 Flush
16732 056/300 Read block 1681 len 12288
16733 057/300 Write block 435 len 24576 data: 2 2 3 3 5 6
16734 058/300 Write block 2678 len 4096 data: 6
16735 059/300 Read block 4974 len 12288
16736 060/300 Write block 4258 len 12288 data: 3 3 4
16737 061/300 Write block 2236 len 40960 data: 4 4 4 5 4 4 4 4 4 5
16738 062/300 Write block 4264 len 32768 data: 5 5 5 4 4 4 4 6
16739 063/300 Write block 4214 len 40960 data: 2 3 3 2 3 3 3 3 3 3
16740 064/300 Read block 2439 len 4096
16741 065/300 Write block 2072 len 36864 data: 3 3 3 3 5 5 5 5 4
16742 066/300 Write block 3061 len 12288 data: 5 5 5
16743 067/300 Read block 3976 len 36864
16744 068/300 Flush
16745 069/300 Write block 3713 len 20480 data: 6 6 6 4 4
16746 070/300 Read block 697 len 28672
16747 071/300 Read block 390 len 40960
16748 072/300 Read block 2377 len 24576
16749 073/300 Read block 1749 len 36864
16750 074/300 Write block 3253 len 36864 data: 4 4 2 3 3 3 3 3 4
16751 075/300 Flush
16752 076/300 Read block 54 len 4096
16753 077/300 Read block 517 len 20480
16754 078/300 Flush
16755 079/300 Write block 1307 len 8192 data: 6 5
16756 080/300 Flush
16757 081/300 Read block 575 len 12288
16758 082/300 Read block 1328 len 20480
16759 083/300 Read block 1261 len 28672
16760 084/300 Read block 1457 len 20480
16761 085/300 Read block 4920 len 16384
16762 086/300 Write block 2713 len 12288 data: 3 3 4
16763 087/300 Write block 3314 len 32768 data: 5 4 4 4 3 3 4 3
16764 088/300 Read block 4624 len 36864
16765 089/300 Write block 642 len 32768 data: 3 2 2 2 2 2 5 5
16766 090/300 Read block 4977 len 24576
16767 091/300 Read block 1053 len 36864
16768 092/300 Read block 971 len 40960
16769 093/300 Write block 361 len 16384 data: 3 3 2 4
16770 094/300 Write block 2303 len 24576 data: 5 5 5 3 4 5
16771 095/300 Write block 1652 len 36864 data: 2 2 3 3 3 3 4 4 4
16772 096/300 Flush
16773 097/300 Write block 4473 len 24576 data: 3 3 5 5 5 5
16774 098/300 Read block 2604 len 40960
16775 099/300 Read block 2068 len 8192
16776 100/300 Write block 2394 len 36864 data: 3 3 3 3 3 3 5 4 4
16777 101/300 Write block 2246 len 28672 data: 5 6 5 4 4 3 2
16778 102/300 Write block 1938 len 24576 data: 2 2 2 3 3 4
16779 103/300 Write block 2407 len 40960 data: 3 3 4 5 4 5 5 6 5 5
16780 104/300 Read block 3769 len 16384
16781 105/300 Read block 1254 len 4096
16782 106/300 Read block 924 len 4096
16783 107/300 Read block 4227 len 32768
16784 108/300 Read block 1561 len 24576
16785 109/300 Write block 3877 len 20480 data: 4 5 4 7 6
16786 110/300 Flush
16787 111/300 Read block 2914 len 4096
16788 112/300 Read block 2984 len 36864
16789 113/300 Write block 4185 len 32768 data: 2 2 2 3 5 7 7 7
16790 114/300 Write block 3707 len 20480 data: 5 7 7 7 8
16791 115/300 Read block 2569 len 4096
16792 116/300 Read block 1918 len 4096
16793 117/300 Write block 422 len 40960 data: 5 5 5 5 5 3 3 4 3 3
16794 118/300 Read block 4694 len 36864
16795 119/300 Read block 390 len 24576
16796 120/300 Write block 4835 len 40960 data: 5 5 5 6 5 4 4 5 4 4
16797 121/300 Read block 2935 len 40960
16798 122/300 Write block 3443 len 4096 data: 2
16799 123/300 Read block 3796 len 12288
16800 124/300 Read block 1435 len 36864
16801 125/300 Read block 1390 len 20480
16802 126/300 Write block 54 len 36864 data: 5 3 3 3 4 4 5 3 4
16803 127/300 Write block 1590 len 36864 data: 2 4 4 4 3 3 4 5 5
16804 128/300 Read block 3629 len 12288
16805 129/300 Write block 1976 len 40960 data: 3 3 4 2 2 3 3 4 4 5
16806 130/300 Write block 613 len 16384 data: 3 2 2 2
16807 131/300 Write block 3757 len 8192 data: 5 6
16808 132/300 Read block 3702 len 36864
16809 133/300 Write block 58 len 32768 data: 5 5 6 4 5 4 5 5
16810 134/300 Read block 3735 len 12288
16811 135/300 Write block 1208 len 4096 data: 5
16812 136/300 Read block 2873 len 32768
16813 137/300 Read block 815 len 40960
16814 138/300 Write block 232 len 36864 data: 2 2 3 4 4 4 4 4 4
16815 139/300 Write block 1957 len 4096 data: 4
16816 140/300 Read block 1654 len 28672
16817 141/300 Flush
16818 142/300 Read block 4618 len 16384
16819 143/300 Write block 608 len 12288 data: 4 4 3
16820 144/300 Flush
16821 145/300 Flush
16822 146/300 Write block 1565 len 40960 data: 2 2 2 2 3 3 4 5 5 4
16823 147/300 Read block 3862 len 24576
16824 148/300 Flush
16825 149/300 Read block 886 len 20480
16826 150/300 Flush
16827 151/300 Read block 1838 len 40960
16828 152/300 Read block 935 len 28672
16829 153/300 Read block 4666 len 12288
16830 154/300 Read block 4432 len 24576
16831 155/300 Write block 3919 len 12288 data: 5 4 5
16832 156/300 Write block 3693 len 20480 data: 5 4 3 4 4
16833 157/300 Flush
16834 158/300 Write block 1378 len 8192 data: 8 8
16835 159/300 Write block 2558 len 28672 data: 4 4 5 5 5 6 5
16836 160/300 Write block 3594 len 16384 data: 2 2 2 3
16837 161/300 Read block 1227 len 20480
16838 162/300 Write block 1370 len 4096 data: 6
16839 163/300 Write block 2711 len 24576 data: 3 3 4 4 5 4
16840 164/300 Flush
16841 165/300 Read block 2815 len 12288
16842 166/300 Write block 2288 len 20480 data: 10 9 9 6 6
16843 167/300 Write block 4564 len 20480 data: 6 5 4 4 5
16844 168/300 Write block 221 len 36864 data: 2 4 4 4 4 5 4 4 4
16845 169/300 Read block 2991 len 4096
16846 170/300 Write block 4524 len 20480 data: 4 3 4 4 4
16847 171/300 Read block 4402 len 36864
16848 172/300 Read block 294 len 36864
16849 173/300 Read block 4107 len 28672
16850 174/300 Write block 402 len 20480 data: 5 5 5 4 4
16851 175/300 Write block 3946 len 12288 data: 2 2 2
16852 176/300 Read block 2601 len 24576
16853 177/300 Read block 1153 len 20480
16854 178/300 Flush
16855 179/300 Read block 557 len 36864
16856 180/300 Read block 2978 len 36864
16857 181/300 Write block 2606 len 8192 data: 5 5
16858 182/300 Write block 1740 len 8192 data: 5 5
16859 183/300 Read block 967 len 16384
16860 184/300 Read block 1158 len 12288
16861 185/300 Write block 2104 len 24576 data: 3 3 3 3 3 3
16862 186/300 Read block 2922 len 12288
16863 187/300 Write block 2982 len 12288 data: 5 6 5
16864 188/300 Read block 155 len 24576
16865 189/300 Write block 588 len 16384 data: 5 5 5 5
16866 190/300 Read block 2535 len 32768
16867 191/300 Write block 1234 len 36864 data: 8 8 8 7 7 7 7 5 3
16868 192/300 Write block 352 len 36864 data: 5 5 5 5 6 5 5 3 3
16869 193/300 Read block 3173 len 40960
16870 194/300 Flush
16871 195/300 Write block 918 len 20480 data: 3 5 5 5 4
16872 196/300 Write block 2941 len 24576 data: 4 4 3 3 3 3
16873 197/300 Write block 23 len 24576 data: 3 4 4 3 3 4
16874 198/300 Read block 2445 len 40960
16875 199/300 Read block 321 len 16384
16876 200/300 Write block 401 len 32768 data: 5 6 6 6 5 5 3 3
16877 201/300 Write block 1235 len 8192 data: 9 9
16878 202/300 Read block 3243 len 40960
16879 203/300 Write block 3003 len 8192 data: 5 4
16880 204/300 Write block 1766 len 8192 data: 3 3
16881 205/300 Write block 2557 len 8192 data: 5 5
16882 206/300 Write block 4320 len 24576 data: 2 3 3 3 3 4
16883 207/300 Read block 999 len 36864
16884 208/300 Write block 153 len 28672 data: 2 3 3 3 4 4 4
16885 209/300 Write block 2532 len 40960 data: 3 3 4 3 3 3 4 4 2 2
16886 210/300 Write block 1640 len 24576 data: 3 4 4 3 3 3
16887 211/300 Read block 3282 len 32768
16888 212/300 Read block 4734 len 4096
16889 213/300 Read block 2132 len 40960
16890 214/300 Write block 2385 len 12288 data: 4 5 5
16891 215/300 Write block 2839 len 12288 data: 5 5 6
16892 216/300 Read block 1656 len 32768
16893 217/300 Read block 4037 len 4096
16894 218/300 Write block 4756 len 4096 data: 4
16895 219/300 Write block 3148 len 16384 data: 4 4 5 6
16896 220/300 Read block 2323 len 24576
16897 221/300 Read block 2531 len 40960
16898 222/300 Read block 2579 len 24576
16899 223/300 Write block 3854 len 32768 data: 5 5 4 4 2 2 3 4
16900 224/300 Read block 858 len 36864
16901 225/300 Read block 330 len 24576
16902 226/300 Write block 1017 len 4096 data: 4
16903 227/300 Read block 3726 len 28672
16904 228/300 Read block 4313 len 36864
16905 229/300 Read block 4367 len 16384
16906 230/300 Write block 4525 len 36864 data: 4 5 5 5 3 3 3 3 3
16907 231/300 Write block 3876 len 4096 data: 4
16908 232/300 Read block 102 len 32768
16909 233/300 Flush
16910 234/300 Read block 4453 len 12288
16911 235/300 Read block 3499 len 32768
16912 236/300 Read block 4295 len 20480
16913 237/300 Write block 699 len 40960 data: 3 3 3 2 2 3 4 4 5 6
16914 238/300 Read block 1203 len 32768
16915 239/300 Write block 3209 len 20480 data: 2 2 4 4 4
16916 240/300 Flush
16917 241/300 Read block 291 len 32768
16918 242/300 Write block 2764 len 20480 data: 2 3 4 5 4
16919 243/300 Read block 1487 len 20480
16920 244/300 Write block 3062 len 8192 data: 6 6
16921 245/300 Write block 136 len 20480 data: 3 4 6 6 4
16922 246/300 Read block 511 len 40960
16923 247/300 Write block 3410 len 8192 data: 4 3
16924 248/300 Read block 2191 len 28672
16925 249/300 Read block 4337 len 8192
16926 250/300 Write block 4937 len 20480 data: 4 5 4 4 4
16927 251/300 Flush
16928 252/300 Write block 1408 len 40960 data: 6 6 5 4 5 5 5 5 5 6
16929 253/300 Read block 1733 len 4096
16930 254/300 Write block 336 len 16384 data: 3 4 4 4
16931 255/300 Flush
16932 256/300 Read block 4805 len 12288
16933 257/300 Read block 2314 len 12288
16934 258/300 Write block 4602 len 8192 data: 4 4
16935 259/300 Flush
16936 260/300 Write block 3208 len 8192 data: 2 3
16937 261/300 Write block 679 len 40960 data: 2 2 3 3 4 4 3 3 2 2
16938 262/300 Read block 3290 len 24576
16939 263/300 Read block 3365 len 4096
16940 264/300 Write block 3321 len 40960 data: 4 3 4 4 5 6 4 4 5 6
16941 265/300 Read block 1656 len 20480
16942 266/300 Write block 2810 len 12288 data: 4 5 5
16943 267/300 Read block 4501 len 12288
16944 268/300 Write block 1591 len 32768 data: 5 5 5 4 4 5 6 6
16945 269/300 Read block 2591 len 24576
16946 270/300 Read block 271 len 28672
16947 271/300 Read block 3987 len 36864
16948 272/300 Write block 462 len 40960 data: 3 4 5 5 4 3 3 3 4 4
16949 273/300 Write block 1781 len 20480 data: 3 2 2 4 6
16950 274/300 Read block 1943 len 40960
16951 275/300 Write block 1904 len 24576 data: 4 4 4 4 3 3
16952 276/300 Write block 704 len 40960 data: 4 5 5 6 7 6 6 7 6 11
16953 277/300 Read block 3376 len 8192
16954 278/300 Read block 2629 len 24576
16955 279/300 Read block 258 len 28672
16956 280/300 Write block 2651 len 24576 data: 6 5 5 5 5 5
16957 281/300 Read block 256 len 24576
16958 282/300 Write block 2599 len 32768 data: 4 5 4 3 3 5 5 6
16959 283/300 Read block 845 len 24576
16960 284/300 Read block 4561 len 8192
16961 285/300 Read block 1527 len 4096
16962 286/300 Read block 3737 len 28672
16963 287/300 Read block 2488 len 12288
16964 288/300 Read block 1234 len 12288
16965 289/300 Read block 4140 len 28672
16966 290/300 Write block 1396 len 20480 data: 4 5 6 7 6
16967 291/300 Read block 1435 len 40960
16968 292/300 Read block 323 len 36864
16969 293/300 Write block 271 len 28672 data: 3 4 4 4 5 5 5
16970 294/300 Read block 2235 len 16384
16971 295/300 Write block 4329 len 12288 data: 4 5 6
16972 296/300 Write block 107 len 12288 data: 3 3 3
16973 297/300 Write block 218 len 36864 data: 3 3 3 3 5 5 5 5 6
16974 298/300 Write block 2828 len 40960 data: 4 4 5 5 6 6 5 3 4 5
16975 299/300 Flush
16976 300/300 Write block 4762 len 40960 data: 4 4 5 5 4 3 3 4 5 6
16977 301/300 Read block 1779 len 32768
16978 Main task has work to do, go find it
16979 got dsc Start(2)
16980 start 2
16981 Jan 27 19:39:56.864 INFO request completed, latency_us: 199, response_code: 204, uri: /start/cid/2, method: POST, req_id: 0ba6721d-707a-4421-84fd-8573f09ac7ff, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
16982 Replay: started 2, returned:Ok(())
16983 ----------------------------------------------------------------
16984 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
16985 States: Active Active Offline
16986 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
16987 5527 Acked Read 4 Done Done Sent false
16988 5528 Acked Read 4 Done Done Sent false
16989 5529 Acked Write 9 Done Done Sent false
16990 5530 Acked Read 8 Done Done Sent false
16991 5531 Acked Read 7 Done Done Sent false
16992 5532 Acked Read 3 Done Done Sent false
16993 5533 Acked Flush 0 Done Done Sent false
16994 5534 Acked Read 4 Done Done Sent false
16995 5535 Acked Read 8 Done Done Sent false
16996 5536 Acked Write 5 Done Done Sent false
16997 5537 Acked Write 4 Done Done Sent false
16998 5538 Acked Read 1 Done Done Sent false
16999 5539 Acked Write 7 Done Done Sent false
17000 5540 Acked Read 8 Done Done Sent false
17001 5541 Acked Read 4 Done Done Sent false
17002 5542 Acked Read 7 Done Done Sent false
17003 5543 Acked Read 3 Done Done Sent false
17004 5544 Acked Read 9 Done Done Sent false
17005 5545 Acked Write 1 Done Done Sent false
17006 5546 Acked Write 9 Done Done Sent false
17007 5547 Acked Write 10 Done Done Sent false
17008 5548 Acked Write 5 Done Done Sent false
17009 5549 Acked Write 1 Done Done Sent false
17010 5550 Acked Read 4 Done Done Sent false
17011 5551 Acked Read 3 Done Done Sent false
17012 5552 Acked Read 4 Done Done Sent false
17013 5553 Acked Read 5 Done Done Sent false
17014 5554 Acked Read 6 Done Done Sent false
17015 5555 Acked Read 4 Done Done Sent false
17016 5556 Acked Read 1 Done Done Sent false
17017 5557 Acked Read 5 Done Done Sent false
17018 5558 Acked Read 5 Done Done Sent false
17019 5559 Acked Flush 0 Done Done Sent false
17020 5560 Acked Write 1 Done Done Sent false
17021 5561 Acked Read 1 Done Done Sent false
17022 5562 Acked Read 2 Done Done Sent false
17023 5563 Acked Read 5 Done Done Sent false
17024 5564 Acked Flush 0 Done Done Sent false
17025 5565 Acked Write 1 Done Done Sent false
17026 5566 Acked Read 8 Done Done Sent false
17027 5567 Acked Read 3 Done Done Sent false
17028 5568 Acked Write 7 Done Done Sent false
17029 5569 Acked Write 9 Done Done Sent false
17030 5570 Acked Read 6 Done Done Sent false
17031 5571 Acked Write 1 Done Done Sent false
17032 5572 Acked Flush 0 Done Done Sent false
17033 5573 Acked Read 2 Done Done Sent false
17034 5574 Acked Write 3 Done Done Sent false
17035 5575 Acked Write 6 Done Done Sent false
17036 5576 Acked Read 7 Done Done Sent false
17037 5577 Acked Flush 0 Done Done Sent false
17038 5578 Acked Flush 0 Done Done Sent false
17039 5579 Acked Write 4 Done Done Sent false
17040 5580 Acked Read 1 Done Done Sent false
17041 5581 Acked Flush 0 Done Done Sent false
17042 5582 Acked Read 3 Done Done Sent false
17043 5583 Acked Write 6 Done Done Sent false
17044 5584 Acked Write 1 Done Done Sent false
17045 5585 Acked Read 3 Done Done Sent false
17046 5586 Acked Write 3 Done Done Sent false
17047 5587 Acked Write 10 Done Done Sent false
17048 5588 Acked Write 8 Done Done Sent false
17049 5589 Acked Write 10 Done Done Sent false
17050 5590 Acked Read 1 Done Done Sent false
17051 5591 Acked Write 9 Done Done Sent false
17052 5592 Acked Write 3 Done Done Sent false
17053 5593 Acked Read 9 Done Done Sent false
17054 5594 Acked Flush 0 Done Done Sent false
17055 5595 Acked Write 5 Done Done Sent false
17056 5596 Acked Read 7 Done Done Sent false
17057 5597 Acked Read 10 Done Done Sent false
17058 5598 Acked Read 6 Done Done Sent false
17059 5599 Acked Read 9 Done Done Sent false
17060 5600 Acked Write 9 Done Done Sent false
17061 5601 Acked Flush 0 Done Done Sent false
17062 5602 Acked Read 1 Done Done Sent false
17063 5603 Acked Read 5 Done Done Sent false
17064 5604 Acked Flush 0 Done Done Sent false
17065 5605 Acked Write 2 Done Done Sent false
17066 5606 Acked Flush 0 Done Done Sent false
17067 5607 Acked Read 3 Done Done Sent false
17068 5608 Acked Read 5 Done Done Sent false
17069 5609 Acked Read 7 Done Done Sent false
17070 5610 Acked Read 5 Done Done Sent false
17071 5611 Acked Read 4 Done Done Sent false
17072 5612 Acked Write 3 Done Done Sent false
17073 5613 Acked Write 8 Done Done Sent false
17074 5614 Acked Read 9 Done Done Sent false
17075 5615 Acked Write 8 Done Done Sent false
17076 5616 Acked Read 6 Done Done Sent false
17077 5617 Acked Read 9 Done Done Sent false
17078 5618 Acked Read 10 Done Done Sent false
17079 5619 Acked Write 4 Done Done Sent false
17080 5620 Acked Write 6 Done Done Sent false
17081 5621 Acked Write 9 Done Done Sent false
17082 5622 Acked Flush 0 Done Done Sent false
17083 5623 Acked Write 6 Done Done Sent false
17084 5624 Acked Read 10 Done Done Sent false
17085 5625 Acked Read 2 Done Done Sent false
17086 5626 Acked Write 9 Done Done Sent false
17087 5627 Acked Write 7 Done Done Sent false
17088 5628 Acked Write 6 Done Done Sent false
17089 5629 Acked Write 10 Done Done Sent false
17090 5630 Acked Read 4 Done Done Sent false
17091 5631 Acked Read 1 Done Done Sent false
17092 5632 Acked Read 1 Done Done Sent false
17093 5633 Acked Read 8 Done Done Sent false
17094 5634 Acked Read 6 Done Done Sent false
17095 5635 Acked Write 5 Done Done Sent false
17096 5636 Acked Flush 0 Done Done Sent false
17097 5637 Acked Read 1 Done Done Sent false
17098 5638 Acked Read 9 Done Done Sent false
17099 5639 Acked Write 8 Done Done Sent false
17100 5640 Acked Write 5 Done Done Sent false
17101 5641 Acked Read 1 Done Done Sent false
17102 5642 Acked Read 1 Done Done Sent false
17103 5643 Acked Write 10 Done Done Sent false
17104 5644 Acked Read 9 Done Done Sent false
17105 5645 Acked Read 6 Done Done Sent false
17106 5646 Acked Write 10 Done Done Sent false
17107 5647 Acked Read 10 Done Done Sent false
17108 5648 Acked Write 1 Done Done Sent false
17109 5649 Acked Read 3 Done Done Sent false
17110 5650 Acked Read 9 Done Done Sent false
17111 5651 Acked Read 5 Done Done Sent false
17112 5652 Acked Write 9 Done Done Sent false
17113 5653 Acked Write 9 Done Done Sent false
17114 5654 Acked Read 3 Done Done Sent false
17115 5655 Acked Write 10 Done Done Sent false
17116 5656 Acked Write 4 Done Done Sent false
17117 5657 Acked Write 2 Done Done Sent false
17118 5658 Acked Read 9 Done Done Sent false
17119 5659 Acked Write 8 Done Done Sent false
17120 5660 Acked Read 3 Done Done Sent false
17121 5661 Acked Write 1 Done Done Sent false
17122 5662 Acked Read 8 Done Done Sent false
17123 5663 Acked Read 10 Done Done Sent false
17124 5664 Acked Write 9 Done Done Sent false
17125 5665 Acked Write 1 Done Done Sent false
17126 5666 Acked Read 7 Done Done Sent false
17127 5667 Acked Flush 0 Done Done Sent false
17128 5668 Acked Read 4 Done Done Sent false
17129 5669 Acked Write 3 Done Done Sent false
17130 5670 Acked Flush 0 Done Done Sent false
17131 5671 Acked Flush 0 Done Done Sent false
17132 5672 Acked Write 10 Done Done Sent false
17133 5673 Acked Read 6 Done Done Sent false
17134 5674 Acked Flush 0 Done Done Sent false
17135 5675 Acked Read 5 Done Done Sent false
17136 5676 Acked Flush 0 Done Done Sent false
17137 5677 Acked Read 10 Done Done Sent false
17138 5678 Acked Read 7 Done Done Sent false
17139 5679 Acked Read 3 Done Done Sent false
17140 5680 Acked Read 6 Done Done Sent false
17141 5681 Acked Write 3 Done Done Sent false
17142 5682 Acked Write 5 Done Done Sent false
17143 5683 Acked Flush 0 Done Done Sent false
17144 5684 Acked Write 2 Done Done Sent false
17145 5685 Acked Write 7 Done Done Sent false
17146 5686 Acked Write 4 Done Done Sent false
17147 5687 Acked Read 5 Done Done Sent false
17148 5688 Acked Write 1 Done Done Sent false
17149 5689 Acked Write 6 Done Done Sent false
17150 5690 Acked Flush 0 Done Done Sent false
17151 5691 Acked Read 3 Done Done Sent false
17152 5692 Acked Write 5 Done Done Sent false
17153 5693 Acked Write 5 Done Done Sent false
17154 5694 Acked Write 9 Done Done Sent false
17155 5695 Acked Read 1 Done Done Sent false
17156 5696 Acked Write 5 Done Done Sent false
17157 5697 Acked Read 9 Done Done Sent false
17158 5698 Acked Read 9 Done Done Sent false
17159 5699 Acked Read 7 Done Done Sent false
17160 5700 Acked Write 5 Done Done Sent false
17161 5701 Acked Write 3 Done Done Sent false
17162 5702 Acked Read 6 Done Done Sent false
17163 5703 Acked Read 5 Done Done Sent false
17164 5704 Acked Flush 0 Done Done Sent false
17165 5705 Acked Read 9 Done Done Sent false
17166 5706 Acked Read 9 Done Done Sent false
17167 5707 Acked Write 2 Done Done Sent false
17168 5708 Acked Write 2 Done Done Sent false
17169 5709 Acked Read 4 Done Done Sent false
17170 5710 Acked Read 3 Done Done Sent false
17171 5711 Acked Write 6 Done Done Sent false
17172 5712 Acked Read 3 Done Done Sent false
17173 5713 Acked Write 3 Done Done Sent false
17174 5714 Acked Read 6 Done Done Sent false
17175 5715 Acked Write 4 Done Done Sent false
17176 5716 Acked Read 8 Done Done Sent false
17177 5717 Acked Write 9 Done Done Sent false
17178 5718 Acked Write 9 Done Done Sent false
17179 5719 Acked Read 10 Done Done Sent false
17180 5720 Acked Flush 0 Done Done Sent false
17181 5721 Acked Write 5 Done Done Sent false
17182 5722 Acked Write 6 Done Done Sent false
17183 5723 Acked Write 6 Done Done Sent false
17184 5724 Acked Read 10 Done Done Sent false
17185 5725 Acked Read 4 Done Done Sent false
17186 5726 Acked Write 8 Done Done Sent false
17187 5727 Acked Write 2 Done Done Sent false
17188 5728 Acked Read 10 Done Done Sent false
17189 5729 Acked Write 2 Done Done Sent false
17190 5730 Acked Write 2 Done Done Sent false
17191 5731 Acked Write 2 Done Done Sent false
17192 5732 Acked Write 6 Done Done Sent false
17193 5733 Acked Read 9 Done Done Sent false
17194 5734 Acked Write 7 Done Done Sent false
17195 5735 Acked Write 10 Done Done Sent false
17196 5736 Acked Write 6 Done Done Sent false
17197 5737 Acked Read 8 Done Done Sent false
17198 5738 Acked Read 1 Done Done Sent false
17199 5739 Acked Read 10 Done Done Sent false
17200 5740 Acked Write 3 Done Done Sent false
17201 5741 Acked Write 3 Done Done Sent false
17202 5742 Acked Read 8 Done Done Sent false
17203 5743 Acked Read 1 Done Done Sent false
17204 5744 Acked Write 1 Done Done Sent false
17205 5745 Acked Write 4 Done Done Sent false
17206 5746 Acked Read 6 Done Done Sent false
17207 5747 Acked Read 10 Done Done Sent false
17208 5748 Acked Read 6 Done Done Sent false
17209 5749 Acked Write 8 Done Done Sent false
17210 5750 Acked Read 9 Done Done Sent false
17211 5751 Acked Read 6 Done Done Sent false
17212 5752 Acked Write 1 Done Done Sent false
17213 5753 Acked Read 7 Done Done Sent false
17214 5754 Acked Read 9 Done Done Sent false
17215 5755 Acked Read 4 Done Done Sent false
17216 5756 Acked Write 9 Done Done Sent false
17217 5757 Acked Write 1 Done Done Sent false
17218 5758 Acked Read 8 Done Done Sent false
17219 5759 Acked Flush 0 Done Done Sent false
17220 5760 Acked Read 3 Done Done Sent false
17221 5761 Acked Read 8 Done Done Sent false
17222 5762 Acked Read 5 Done Done Sent false
17223 5763 Acked Write 10 Done Done Sent false
17224 5764 Acked Read 8 Done Done Sent false
17225 5765 Acked Write 5 Done Done Sent false
17226 5766 Acked Flush 0 Done Done Sent false
17227 5767 Acked Read 8 Done Done Sent false
17228 5768 Acked Write 5 Done Done Sent false
17229 5769 Acked Read 5 Done Done Sent false
17230 5770 Acked Write 2 Done Done Sent false
17231 5771 Acked Write 5 Done Done Sent false
17232 5772 Acked Read 10 Done Done Sent false
17233 5773 Acked Write 2 Done Done Sent false
17234 5774 Acked Read 7 Done Done Sent false
17235 5775 Acked Read 2 Done Done Sent false
17236 5776 Acked Write 5 Done Done Sent false
17237 5777 Acked Flush 0 Done Done Sent false
17238 5778 Acked Write 10 Done Done Sent false
17239 5779 Acked Read 1 Done Done Sent false
17240 5780 Acked Write 4 Done Done Sent false
17241 5781 Acked Flush 0 Done Done Sent false
17242 5782 Acked Read 3 Done Done Sent false
17243 5783 Acked Read 3 Done Done Sent false
17244 5784 Acked Write 2 Done Done Sent false
17245 5785 Acked Flush 0 Done Done Sent false
17246 5786 Acked Write 2 Done Done Sent false
17247 5787 Acked Write 10 Done Done Sent false
17248 5788 Acked Read 6 Done Done Sent false
17249 5789 Acked Read 1 Done Done Sent false
17250 5790 Acked Write 10 Done Done Sent false
17251 5791 Acked Read 5 Done Done Sent false
17252 5792 Acked Write 3 Done Done Sent false
17253 5793 Acked Read 3 Done Done Sent false
17254 5794 Acked Write 8 Done Done Sent false
17255 5795 Acked Read 6 Done Done Sent false
17256 5796 Acked Read 7 Done Done Sent false
17257 5797 Acked Read 9 Done Done Sent false
17258 5798 Acked Write 10 Done Done Sent false
17259 5799 Acked Write 5 Done Done Sent false
17260 5800 Acked Read 10 Done Done Sent false
17261 5801 Acked Write 6 Done Done Sent false
17262 5802 Acked Write 10 Done Done Sent false
17263 5803 Acked Read 2 Done Done Sent false
17264 5804 Acked Read 6 Done Done Sent false
17265 5805 Acked Read 7 Done Done Sent false
17266 5806 Acked Write 6 Done Done Sent false
17267 5807 Acked Read 6 Done Done Sent false
17268 5808 Acked Write 8 Done Done Sent false
17269 5809 Acked Read 6 Done Done Sent false
17270 5810 Acked Read 2 Done Done Sent false
17271 5811 Acked Read 1 Done Done Sent false
17272 5812 Acked Read 7 Done Done Sent false
17273 5813 Acked Read 3 Done Done Sent false
17274 5814 Acked Read 3 Done Done Sent false
17275 5815 Acked Read 7 Done Done Sent false
17276 5816 Acked Write 5 Done Done Sent false
17277 5817 Acked Read 10 Done Done Sent false
17278 5818 Acked Read 9 Done Done Sent false
17279 5819 Acked Write 7 Done Done Sent false
17280 5820 Acked Read 4 Done Done Sent false
17281 5821 Acked Write 3 Done Done Sent false
17282 5822 Acked Write 3 Done Done Sent false
17283 5823 Acked Write 9 Done Done Sent false
17284 5824 Acked Write 10 Done Done Sent false
17285 5825 Acked Flush 0 Done Done Sent false
17286 5826 Acked Write 10 Done Done Sent false
17287 5827 Acked Read 8 Done Done Sent false
17288 STATES DS:0 DS:1 DS:2 TOTAL
17289 Sent 0 0 301 301
17290 Done 301 301 0 602
17291 Skipped 0 0 0 0
17292 Error 0 0 0 0
17293 Last Flush: 5825 5825 5526
17294 Downstairs last five completed: 5526 5525 5524 5523 5522
17295 Upstairs last five completed: 5827 5826 5825 5824 5823
17296 CLIENT: Up:0 ds:301 act:2
17297 [8830] got start action
17298 [8830] I am going to restart
17299 Starting downstairs at port 8830
17300 Make output file at "/tmp/dsc/downstairs-8830.txt"
17301 [8830][2] reports Starting
17302 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1216)
17303 [8830][2] reports Running
17304 ----------------------------------------------------------------
17305 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
17306 States: Active Active Offline
17307 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17308 5527 Acked Read 4 Done Done Sent false
17309 5528 Acked Read 4 Done Done Sent false
17310 5529 Acked Write 9 Done Done Sent false
17311 5530 Acked Read 8 Done Done Sent false
17312 5531 Acked Read 7 Done Done Sent false
17313 5532 Acked Read 3 Done Done Sent false
17314 5533 Acked Flush 0 Done Done Sent false
17315 5534 Acked Read 4 Done Done Sent false
17316 5535 Acked Read 8 Done Done Sent false
17317 5536 Acked Write 5 Done Done Sent false
17318 5537 Acked Write 4 Done Done Sent false
17319 5538 Acked Read 1 Done Done Sent false
17320 5539 Acked Write 7 Done Done Sent false
17321 5540 Acked Read 8 Done Done Sent false
17322 5541 Acked Read 4 Done Done Sent false
17323 5542 Acked Read 7 Done Done Sent false
17324 5543 Acked Read 3 Done Done Sent false
17325 5544 Acked Read 9 Done Done Sent false
17326 5545 Acked Write 1 Done Done Sent false
17327 5546 Acked Write 9 Done Done Sent false
17328 5547 Acked Write 10 Done Done Sent false
17329 5548 Acked Write 5 Done Done Sent false
17330 5549 Acked Write 1 Done Done Sent false
17331 5550 Acked Read 4 Done Done Sent false
17332 5551 Acked Read 3 Done Done Sent false
17333 5552 Acked Read 4 Done Done Sent false
17334 5553 Acked Read 5 Done Done Sent false
17335 5554 Acked Read 6 Done Done Sent false
17336 5555 Acked Read 4 Done Done Sent false
17337 5556 Acked Read 1 Done Done Sent false
17338 5557 Acked Read 5 Done Done Sent false
17339 5558 Acked Read 5 Done Done Sent false
17340 5559 Acked Flush 0 Done Done Sent false
17341 5560 Acked Write 1 Done Done Sent false
17342 5561 Acked Read 1 Done Done Sent false
17343 5562 Acked Read 2 Done Done Sent false
17344 5563 Acked Read 5 Done Done Sent false
17345 5564 Acked Flush 0 Done Done Sent false
17346 5565 Acked Write 1 Done Done Sent false
17347 5566 Acked Read 8 Done Done Sent false
17348 5567 Acked Read 3 Done Done Sent false
17349 5568 Acked Write 7 Done Done Sent false
17350 5569 Acked Write 9 Done Done Sent false
17351 5570 Acked Read 6 Done Done Sent false
17352 5571 Acked Write 1 Done Done Sent false
17353 5572 Acked Flush 0 Done Done Sent false
17354 5573 Acked Read 2 Done Done Sent false
17355 5574 Acked Write 3 Done Done Sent false
17356 5575 Acked Write 6 Done Done Sent false
17357 5576 Acked Read 7 Done Done Sent false
17358 5577 Acked Flush 0 Done Done Sent false
17359 5578 Acked Flush 0 Done Done Sent false
17360 5579 Acked Write 4 Done Done Sent false
17361 5580 Acked Read 1 Done Done Sent false
17362 5581 Acked Flush 0 Done Done Sent false
17363 5582 Acked Read 3 Done Done Sent false
17364 5583 Acked Write 6 Done Done Sent false
17365 5584 Acked Write 1 Done Done Sent false
17366 5585 Acked Read 3 Done Done Sent false
17367 5586 Acked Write 3 Done Done Sent false
17368 5587 Acked Write 10 Done Done Sent false
17369 5588 Acked Write 8 Done Done Sent false
17370 5589 Acked Write 10 Done Done Sent false
17371 5590 Acked Read 1 Done Done Sent false
17372 5591 Acked Write 9 Done Done Sent false
17373 5592 Acked Write 3 Done Done Sent false
17374 5593 Acked Read 9 Done Done Sent false
17375 5594 Acked Flush 0 Done Done Sent false
17376 5595 Acked Write 5 Done Done Sent false
17377 5596 Acked Read 7 Done Done Sent false
17378 5597 Acked Read 10 Done Done Sent false
17379 5598 Acked Read 6 Done Done Sent false
17380 5599 Acked Read 9 Done Done Sent false
17381 5600 Acked Write 9 Done Done Sent false
17382 5601 Acked Flush 0 Done Done Sent false
17383 5602 Acked Read 1 Done Done Sent false
17384 5603 Acked Read 5 Done Done Sent false
17385 5604 Acked Flush 0 Done Done Sent false
17386 5605 Acked Write 2 Done Done Sent false
17387 5606 Acked Flush 0 Done Done Sent false
17388 5607 Acked Read 3 Done Done Sent false
17389 5608 Acked Read 5 Done Done Sent false
17390 5609 Acked Read 7 Done Done Sent false
17391 5610 Acked Read 5 Done Done Sent false
17392 5611 Acked Read 4 Done Done Sent false
17393 5612 Acked Write 3 Done Done Sent false
17394 5613 Acked Write 8 Done Done Sent false
17395 5614 Acked Read 9 Done Done Sent false
17396 5615 Acked Write 8 Done Done Sent false
17397 5616 Acked Read 6 Done Done Sent false
17398 5617 Acked Read 9 Done Done Sent false
17399 5618 Acked Read 10 Done Done Sent false
17400 5619 Acked Write 4 Done Done Sent false
17401 5620 Acked Write 6 Done Done Sent false
17402 5621 Acked Write 9 Done Done Sent false
17403 5622 Acked Flush 0 Done Done Sent false
17404 5623 Acked Write 6 Done Done Sent false
17405 5624 Acked Read 10 Done Done Sent false
17406 5625 Acked Read 2 Done Done Sent false
17407 5626 Acked Write 9 Done Done Sent false
17408 5627 Acked Write 7 Done Done Sent false
17409 5628 Acked Write 6 Done Done Sent false
17410 5629 Acked Write 10 Done Done Sent false
17411 5630 Acked Read 4 Done Done Sent false
17412 5631 Acked Read 1 Done Done Sent false
17413 5632 Acked Read 1 Done Done Sent false
17414 5633 Acked Read 8 Done Done Sent false
17415 5634 Acked Read 6 Done Done Sent false
17416 5635 Acked Write 5 Done Done Sent false
17417 5636 Acked Flush 0 Done Done Sent false
17418 5637 Acked Read 1 Done Done Sent false
17419 5638 Acked Read 9 Done Done Sent false
17420 5639 Acked Write 8 Done Done Sent false
17421 5640 Acked Write 5 Done Done Sent false
17422 5641 Acked Read 1 Done Done Sent false
17423 5642 Acked Read 1 Done Done Sent false
17424 5643 Acked Write 10 Done Done Sent false
17425 5644 Acked Read 9 Done Done Sent false
17426 5645 Acked Read 6 Done Done Sent false
17427 5646 Acked Write 10 Done Done Sent false
17428 5647 Acked Read 10 Done Done Sent false
17429 5648 Acked Write 1 Done Done Sent false
17430 5649 Acked Read 3 Done Done Sent false
17431 5650 Acked Read 9 Done Done Sent false
17432 5651 Acked Read 5 Done Done Sent false
17433 5652 Acked Write 9 Done Done Sent false
17434 5653 Acked Write 9 Done Done Sent false
17435 5654 Acked Read 3 Done Done Sent false
17436 5655 Acked Write 10 Done Done Sent false
17437 5656 Acked Write 4 Done Done Sent false
17438 5657 Acked Write 2 Done Done Sent false
17439 5658 Acked Read 9 Done Done Sent false
17440 5659 Acked Write 8 Done Done Sent false
17441 5660 Acked Read 3 Done Done Sent false
17442 5661 Acked Write 1 Done Done Sent false
17443 5662 Acked Read 8 Done Done Sent false
17444 5663 Acked Read 10 Done Done Sent false
17445 5664 Acked Write 9 Done Done Sent false
17446 5665 Acked Write 1 Done Done Sent false
17447 5666 Acked Read 7 Done Done Sent false
17448 5667 Acked Flush 0 Done Done Sent false
17449 5668 Acked Read 4 Done Done Sent false
17450 5669 Acked Write 3 Done Done Sent false
17451 5670 Acked Flush 0 Done Done Sent false
17452 5671 Acked Flush 0 Done Done Sent false
17453 5672 Acked Write 10 Done Done Sent false
17454 5673 Acked Read 6 Done Done Sent false
17455 5674 Acked Flush 0 Done Done Sent false
17456 5675 Acked Read 5 Done Done Sent false
17457 5676 Acked Flush 0 Done Done Sent false
17458 5677 Acked Read 10 Done Done Sent false
17459 5678 Acked Read 7 Done Done Sent false
17460 5679 Acked Read 3 Done Done Sent false
17461 5680 Acked Read 6 Done Done Sent false
17462 5681 Acked Write 3 Done Done Sent false
17463 5682 Acked Write 5 Done Done Sent false
17464 5683 Acked Flush 0 Done Done Sent false
17465 5684 Acked Write 2 Done Done Sent false
17466 5685 Acked Write 7 Done Done Sent false
17467 5686 Acked Write 4 Done Done Sent false
17468 5687 Acked Read 5 Done Done Sent false
17469 5688 Acked Write 1 Done Done Sent false
17470 5689 Acked Write 6 Done Done Sent false
17471 5690 Acked Flush 0 Done Done Sent false
17472 5691 Acked Read 3 Done Done Sent false
17473 5692 Acked Write 5 Done Done Sent false
17474 5693 Acked Write 5 Done Done Sent false
17475 5694 Acked Write 9 Done Done Sent false
17476 5695 Acked Read 1 Done Done Sent false
17477 5696 Acked Write 5 Done Done Sent false
17478 5697 Acked Read 9 Done Done Sent false
17479 5698 Acked Read 9 Done Done Sent false
17480 5699 Acked Read 7 Done Done Sent false
17481 5700 Acked Write 5 Done Done Sent false
17482 5701 Acked Write 3 Done Done Sent false
17483 5702 Acked Read 6 Done Done Sent false
17484 5703 Acked Read 5 Done Done Sent false
17485 5704 Acked Flush 0 Done Done Sent false
17486 5705 Acked Read 9 Done Done Sent false
17487 5706 Acked Read 9 Done Done Sent false
17488 5707 Acked Write 2 Done Done Sent false
17489 5708 Acked Write 2 Done Done Sent false
17490 5709 Acked Read 4 Done Done Sent false
17491 5710 Acked Read 3 Done Done Sent false
17492 5711 Acked Write 6 Done Done Sent false
17493 5712 Acked Read 3 Done Done Sent false
17494 5713 Acked Write 3 Done Done Sent false
17495 5714 Acked Read 6 Done Done Sent false
17496 5715 Acked Write 4 Done Done Sent false
17497 5716 Acked Read 8 Done Done Sent false
17498 5717 Acked Write 9 Done Done Sent false
17499 5718 Acked Write 9 Done Done Sent false
17500 5719 Acked Read 10 Done Done Sent false
17501 5720 Acked Flush 0 Done Done Sent false
17502 5721 Acked Write 5 Done Done Sent false
17503 5722 Acked Write 6 Done Done Sent false
17504 5723 Acked Write 6 Done Done Sent false
17505 5724 Acked Read 10 Done Done Sent false
17506 5725 Acked Read 4 Done Done Sent false
17507 5726 Acked Write 8 Done Done Sent false
17508 5727 Acked Write 2 Done Done Sent false
17509 5728 Acked Read 10 Done Done Sent false
17510 5729 Acked Write 2 Done Done Sent false
17511 5730 Acked Write 2 Done Done Sent false
17512 5731 Acked Write 2 Done Done Sent false
17513 5732 Acked Write 6 Done Done Sent false
17514 5733 Acked Read 9 Done Done Sent false
17515 5734 Acked Write 7 Done Done Sent false
17516 5735 Acked Write 10 Done Done Sent false
17517 5736 Acked Write 6 Done Done Sent false
17518 5737 Acked Read 8 Done Done Sent false
17519 5738 Acked Read 1 Done Done Sent false
17520 5739 Acked Read 10 Done Done Sent false
17521 5740 Acked Write 3 Done Done Sent false
17522 5741 Acked Write 3 Done Done Sent false
17523 5742 Acked Read 8 Done Done Sent false
17524 5743 Acked Read 1 Done Done Sent false
17525 5744 Acked Write 1 Done Done Sent false
17526 5745 Acked Write 4 Done Done Sent false
17527 5746 Acked Read 6 Done Done Sent false
17528 5747 Acked Read 10 Done Done Sent false
17529 5748 Acked Read 6 Done Done Sent false
17530 5749 Acked Write 8 Done Done Sent false
17531 5750 Acked Read 9 Done Done Sent false
17532 5751 Acked Read 6 Done Done Sent false
17533 5752 Acked Write 1 Done Done Sent false
17534 5753 Acked Read 7 Done Done Sent false
17535 5754 Acked Read 9 Done Done Sent false
17536 5755 Acked Read 4 Done Done Sent false
17537 5756 Acked Write 9 Done Done Sent false
17538 5757 Acked Write 1 Done Done Sent false
17539 5758 Acked Read 8 Done Done Sent false
17540 5759 Acked Flush 0 Done Done Sent false
17541 5760 Acked Read 3 Done Done Sent false
17542 5761 Acked Read 8 Done Done Sent false
17543 5762 Acked Read 5 Done Done Sent false
17544 5763 Acked Write 10 Done Done Sent false
17545 5764 Acked Read 8 Done Done Sent false
17546 5765 Acked Write 5 Done Done Sent false
17547 5766 Acked Flush 0 Done Done Sent false
17548 5767 Acked Read 8 Done Done Sent false
17549 5768 Acked Write 5 Done Done Sent false
17550 5769 Acked Read 5 Done Done Sent false
17551 5770 Acked Write 2 Done Done Sent false
17552 5771 Acked Write 5 Done Done Sent false
17553 5772 Acked Read 10 Done Done Sent false
17554 5773 Acked Write 2 Done Done Sent false
17555 5774 Acked Read 7 Done Done Sent false
17556 5775 Acked Read 2 Done Done Sent false
17557 5776 Acked Write 5 Done Done Sent false
17558 5777 Acked Flush 0 Done Done Sent false
17559 5778 Acked Write 10 Done Done Sent false
17560 5779 Acked Read 1 Done Done Sent false
17561 5780 Acked Write 4 Done Done Sent false
17562 5781 Acked Flush 0 Done Done Sent false
17563 5782 Acked Read 3 Done Done Sent false
17564 5783 Acked Read 3 Done Done Sent false
17565 5784 Acked Write 2 Done Done Sent false
17566 5785 Acked Flush 0 Done Done Sent false
17567 5786 Acked Write 2 Done Done Sent false
17568 5787 Acked Write 10 Done Done Sent false
17569 5788 Acked Read 6 Done Done Sent false
17570 5789 Acked Read 1 Done Done Sent false
17571 5790 Acked Write 10 Done Done Sent false
17572 5791 Acked Read 5 Done Done Sent false
17573 5792 Acked Write 3 Done Done Sent false
17574 5793 Acked Read 3 Done Done Sent false
17575 5794 Acked Write 8 Done Done Sent false
17576 5795 Acked Read 6 Done Done Sent false
17577 5796 Acked Read 7 Done Done Sent false
17578 5797 Acked Read 9 Done Done Sent false
17579 5798 Acked Write 10 Done Done Sent false
17580 5799 Acked Write 5 Done Done Sent false
17581 5800 Acked Read 10 Done Done Sent false
17582 5801 Acked Write 6 Done Done Sent false
17583 5802 Acked Write 10 Done Done Sent false
17584 5803 Acked Read 2 Done Done Sent false
17585 5804 Acked Read 6 Done Done Sent false
17586 5805 Acked Read 7 Done Done Sent false
17587 5806 Acked Write 6 Done Done Sent false
17588 5807 Acked Read 6 Done Done Sent false
17589 5808 Acked Write 8 Done Done Sent false
17590 5809 Acked Read 6 Done Done Sent false
17591 5810 Acked Read 2 Done Done Sent false
17592 5811 Acked Read 1 Done Done Sent false
17593 5812 Acked Read 7 Done Done Sent false
17594 5813 Acked Read 3 Done Done Sent false
17595 5814 Acked Read 3 Done Done Sent false
17596 5815 Acked Read 7 Done Done Sent false
17597 5816 Acked Write 5 Done Done Sent false
17598 5817 Acked Read 10 Done Done Sent false
17599 5818 Acked Read 9 Done Done Sent false
17600 5819 Acked Write 7 Done Done Sent false
17601 5820 Acked Read 4 Done Done Sent false
17602 5821 Acked Write 3 Done Done Sent false
17603 5822 Acked Write 3 Done Done Sent false
17604 5823 Acked Write 9 Done Done Sent false
17605 5824 Acked Write 10 Done Done Sent false
17606 5825 Acked Flush 0 Done Done Sent false
17607 5826 Acked Write 10 Done Done Sent false
17608 5827 Acked Read 8 Done Done Sent false
17609 5828 Acked Flush 0 Done Done Sent false
17610 STATES DS:0 DS:1 DS:2 TOTAL
17611 Sent 0 0 302 302
17612 Done 302 302 0 604
17613 Skipped 0 0 0 0
17614 Error 0 0 0 0
17615 Last Flush: 5828 5828 5526
17616 Downstairs last five completed: 5526 5525 5524 5523 5522
17617 Upstairs last five completed: 5828 5827 5826 5825 5824
17618 CLIENT: Up:0 ds:302 act:2
176192025-01-27T19:40:02.624ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176202025-01-27T19:40:02.624ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57141) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176212025-01-27T19:40:02.625ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176222025-01-27T19:40:02.626ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176232025-01-27T19:40:02.626ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176242025-01-27T19:40:02.626ZINFOcrucible: send last flush ID to this DS: 5526 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176252025-01-27T19:40:02.626ZINFOcrucible: Replied this last flush ID: 5526 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176262025-01-27T19:40:02.626ZINFOcrucible: [2] client re-new 302 jobs since flush 5526 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176272025-01-27T19:40:02.626ZINFOcrucible: [2] Replayed 302 jobs since flush: 5526 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
17628 ----------------------------------------------------------------
17629 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
17630 Downstairs last five completed: 5828 5827 5826 5825 5824
17631 Upstairs last five completed: 5828 5827 5826 5825 5824
17632 CLIENT: Up:0 ds:0 act:3
17633 Replay: All jobs finished
17634 Main task has work to do, go find it
17635 got dsc Stop(0)
17636 stop 0
17637 Jan 27 19:40:04.869 INFO request completed, latency_us: 345, response_code: [8810] Got stop action so:false kr:false
17638 204, uri: /stop/cid/0, method: POST, req_id: d4f4cb10-48c8-466c-8e4c-62f5f763f167, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
176392025-01-27T19:40:04.870ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176402025-01-27T19:40:04.870ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
176412025-01-27T19:40:04.870ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
17642 Jan 27 19:40:04.870 INFO request completed, latency_us: 142, response_code: 200, uri: /state/cid/0, method: GET, req_id: f26b93fc-3966-4a21-987c-2d40c09065db, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
176432025-01-27T19:40:04.870ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
17644 [8810] Exited with: ExitStatus(unix_wait_status(9))
17645 [8810][0] reports Exit
17646 Jan 27 19:40:08.871 INFO request completed, latency_us: 220, response_code: 200, uri: /state/cid/0, method: GET, req_id: 47a63636-649f-428d-81f2-6b1470ee38b3, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
17647 001/300 Read block 3677 len 40960
17648 002/300 Write block 488 len 8192 data: 3 3
17649 003/300 Read block 837 len 16384
17650 004/300 Read block 290 len 12288
17651 005/300 Read block 1797 len 32768
17652 006/300 Write block 2261 len 8192 data: 5 5
17653 007/300 Read block 1831 len 16384
17654 008/300 Write block 42 len 24576 data: 4 5 5 4 5 5
17655 009/300 Read block 4387 len 32768
17656 010/300 Write block 3957 len 12288 data: 4 5 6
17657 011/300 Flush
17658 012/300 Read block 1742 len 24576
17659 013/300 Flush
17660 014/300 Read block 3204 len 16384
17661 015/300 Read block 2426 len 12288
17662 016/300 Read block 4546 len 28672
17663 017/300 Write block 1886 len 16384 data: 5 4 5 6
17664 018/300 Read block 84 len 24576
17665 019/300 Read block 3842 len 32768
17666 020/300 Read block 3350 len 24576
17667 021/300 Read block 2560 len 12288
17668 022/300 Read block 3777 len 20480
17669 023/300 Flush
17670 024/300 Read block 2526 len 12288
17671 025/300 Read block 645 len 32768
17672 026/300 Write block 4177 len 8192 data: 5 5
17673 027/300 Write block 1638 len 8192 data: 4 4
17674 028/300 Flush
17675 029/300 Write block 1683 len 20480 data: 2 2 2 2 2
17676 030/300 Read block 484 len 28672
17677 031/300 Flush
17678 032/300 Write block 1966 len 12288 data: 5 5 5
17679 033/300 Read block 1519 len 24576
17680 034/300 Read block 3824 len 24576
17681 035/300 Write block 3068 len 16384 data: 4 3 2 2
17682 036/300 Read block 3410 len 8192
17683 037/300 Read block 3755 len 32768
17684 038/300 Read block 1468 len 16384
17685 039/300 Write block 2308 len 32768 data: 6 5 5 4 4 3 3 3
17686 040/300 Read block 1116 len 32768
17687 041/300 Write block 3251 len 8192 data: 3 3
17688 042/300 Write block 3597 len 16384 data: 4 3 4 4
17689 043/300 Read block 4271 len 16384
17690 044/300 Write block 1870 len 40960 data: 4 4 4 5 5 5 4 3 3 4
17691 045/300 Read block 2531 len 32768
17692 046/300 Write block 4575 len 12288 data: 3 3 3
17693 047/300 Write block 2700 len 20480 data: 2 2 3 3 2
17694 048/300 Read block 4080 len 8192
17695 049/300 Write block 2684 len 24576 data: 8 7 9 8 6 3
17696 050/300 Write block 1990 len 20480 data: 2 3 3 3 3
17697 051/300 Read block 3657 len 40960
17698 052/300 Read block 4178 len 32768
17699 053/300 Flush
17700 054/300 Read block 3684 len 32768
17701 055/300 Write block 1687 len 32768 data: 3 2 2 3 3 3 3 4
17702 056/300 Read block 1498 len 40960
17703 057/300 Read block 4889 len 20480
17704 058/300 Write block 3642 len 40960 data: 3 3 4 3 3 2 3 4 3 3
17705 059/300 Read block 577 len 16384
17706 060/300 Read block 2340 len 24576
17707 061/300 Write block 1058 len 20480 data: 3 4 4 3 2
17708 062/300 Read block 2024 len 8192
17709 063/300 Read block 4565 len 16384
17710 064/300 Read block 2567 len 20480
17711 065/300 Write block 1206 len 4096 data: 6
17712 066/300 Flush
17713 067/300 Write block 2042 len 28672 data: 7 7 5 5 5 4 4
17714 068/300 Write block 3542 len 40960 data: 3 2 2 2 2 2 2 2 2 2
17715 069/300 Write block 653 len 8192 data: 4 3
17716 070/300 Read block 2288 len 20480
17717 071/300 Read block 4794 len 20480
17718 072/300 Flush
17719 073/300 Read block 105 len 32768
17720 074/300 Write block 3241 len 24576 data: 5 6 6 4 3 4
17721 075/300 Write block 4124 len 28672 data: 3 3 3 4 3 3 3
17722 076/300 Read block 4518 len 16384
17723 077/300 Read block 493 len 36864
17724 078/300 Write block 1933 len 40960 data: 2 3 3 2 2 3 3 3 4 4
17725 079/300 Read block 4676 len 20480
17726 080/300 Read block 1899 len 12288
17727 081/300 Write block 523 len 8192 data: 4 2
17728 082/300 Write block 1163 len 40960 data: 2 3 3 3 4 4 4 4 4 4
17729 083/300 Write block 479 len 20480 data: 5 5 5 5 4
17730 084/300 Write block 1098 len 20480 data: 4 4 4 5 5
17731 085/300 Write block 968 len 36864 data: 3 3 3 3 4 4 5 5 5
17732 086/300 Read block 1936 len 24576
17733 087/300 Write block 3921 len 40960 data: 6 5 4 4 3 4 4 4 3 3
17734 088/300 Read block 441 len 12288
17735 089/300 Write block 2611 len 4096 data: 6
17736 090/300 Write block 4607 len 28672 data: 3 3 3 4 3 3 5
17737 091/300 Write block 2931 len 20480 data: 4 4 5 5 5
17738 092/300 Write block 724 len 28672 data: 5 3 3 2 2 2 2
17739 093/300 Write block 597 len 28672 data: 3 5 5 5 5 6 7
17740 094/300 Read block 3076 len 24576
17741 095/300 Read block 2566 len 32768
17742 096/300 Flush
17743 097/300 Flush
17744 098/300 Read block 3572 len 28672
17745 099/300 Write block 1052 len 12288 data: 2 2 2
17746 100/300 Write block 4292 len 28672 data: 3 3 4 4 2 3 3
17747 101/300 Write block 3178 len 20480 data: 7 7 5 5 5
17748 102/300 Read block 3513 len 32768
17749 103/300 Read block 4348 len 40960
17750 104/300 Write block 4844 len 12288 data: 5 4 4
17751 105/300 Read block 2524 len 4096
17752 106/300 Read block 626 len 12288
17753 107/300 Read block 3280 len 24576
17754 108/300 Read block 4567 len 36864
17755 109/300 Read block 3738 len 8192
17756 110/300 Read block 2222 len 36864
17757 111/300 Write block 44 len 20480 data: 6 5 6 6 6
17758 112/300 Write block 2985 len 32768 data: 5 5 5 4 3 3 3 3
17759 113/300 Read block 4453 len 40960
17760 114/300 Write block 74 len 8192 data: 2 2
17761 115/300 Read block 4917 len 12288
17762 116/300 Read block 4420 len 20480
17763 117/300 Read block 756 len 8192
17764 118/300 Read block 3749 len 24576
17765 119/300 Read block 2127 len 12288
17766 120/300 Read block 1756 len 36864
17767 121/300 Read block 3316 len 16384
17768 122/300 Write block 1110 len 28672 data: 4 5 4 3 4 4 4
17769 123/300 Read block 984 len 40960
17770 124/300 Flush
17771 125/300 Write block 696 len 28672 data: 3 3 3 4 4 4 3
17772 126/300 Read block 4534 len 32768
17773 127/300 Read block 303 len 40960
17774 128/300 Write block 4588 len 40960 data: 2 2 2 2 2 2 3 3 3 3
17775 129/300 Write block 4830 len 16384 data: 5 4 4 4
17776 130/300 Read block 2207 len 16384
17777 131/300 Write block 1982 len 28672 data: 4 5 5 6 5 5 5
17778 132/300 Read block 2180 len 24576
17779 133/300 Read block 2200 len 8192
17780 134/300 Read block 4072 len 4096
17781 135/300 Write block 2632 len 20480 data: 3 5 3 4 4
17782 136/300 Write block 2393 len 12288 data: 4 4 4
17783 137/300 Read block 1855 len 16384
17784 138/300 Flush
17785 139/300 Write block 4613 len 36864 data: 6 4 5 5 5 5 5 4 3
17786 140/300 Write block 4819 len 20480 data: 2 2 3 3 3
17787 141/300 Read block 3081 len 12288
17788 142/300 Read block 4240 len 4096
17789 143/300 Write block 3119 len 24576 data: 3 2 2 3 4 4
17790 144/300 Read block 2355 len 40960
17791 145/300 Read block 395 len 36864
17792 146/300 Write block 1326 len 36864 data: 3 3 3 4 3 3 4 5 4
17793 147/300 Read block 940 len 12288
17794 148/300 Write block 3074 len 36864 data: 3 3 3 3 3 3 3 3 2
17795 149/300 Flush
17796 150/300 Write block 450 len 24576 data: 2 4 4 4 4 3
17797 151/300 Read block 3106 len 20480
17798 152/300 Flush
17799 153/300 Read block 1635 len 16384
17800 154/300 Read block 2072 len 24576
17801 155/300 Write block 2845 len 32768 data: 6 7 5 6 6 5 6 6
17802 156/300 Write block 3650 len 8192 data: 4 4
17803 157/300 Write block 580 len 20480 data: 5 5 5 7 4
17804 158/300 Write block 2507 len 24576 data: 3 5 4 3 4 5
17805 159/300 Flush
17806 160/300 Write block 806 len 28672 data: 2 3 3 4 4 4 4
17807 161/300 Read block 4654 len 28672
17808 162/300 Read block 1866 len 12288
17809 163/300 Write block 3370 len 28672 data: 6 6 7 5 4 3 3
17810 164/300 Write block 2352 len 24576 data: 6 8 6 4 4 2
17811 165/300 Write block 285 len 12288 data: 5 5 4
17812 166/300 Write block 3888 len 12288 data: 4 4 2
17813 167/300 Write block 591 len 36864 data: 6 5 6 5 4 4 4 6 6
17814 168/300 Flush
17815 169/300 Read block 2750 len 12288
17816 170/300 Flush
17817 171/300 Read block 946 len 36864
17818 172/300 Write block 4147 len 40960 data: 3 3 3 3 2 2 3 3 3 3
17819 173/300 Write block 494 len 28672 data: 5 6 6 8 9 9 7
17820 174/300 Read block 1632 len 36864
17821 175/300 Read block 1666 len 40960
17822 176/300 Read block 742 len 16384
17823 177/300 Read block 2889 len 4096
17824 178/300 Write block 4706 len 12288 data: 4 4 5
17825 179/300 Read block 3223 len 4096
17826 180/300 Write block 908 len 28672 data: 2 2 3 3 3 3 3
17827 181/300 Flush
17828 182/300 Write block 4474 len 40960 data: 4 6 6 6 6 3 6 6 6 5
17829 183/300 Read block 3438 len 16384
17830 184/300 Read block 3347 len 20480
17831 185/300 Write block 1737 len 40960 data: 3 4 5 6 6 4 4 4 3 3
17832 186/300 Read block 4570 len 16384
17833 187/300 Flush
17834 188/300 Write block 3606 len 8192 data: 7 8
17835 189/300 Write block 745 len 28672 data: 5 5 5 5 5 4 3
17836 190/300 Read block 3674 len 20480
17837 191/300 Write block 4629 len 12288 data: 3 4 4
17838 192/300 Write block 2860 len 32768 data: 4 4 5 5 5 5 5 5
17839 193/300 Write block 988 len 20480 data: 3 3 2 2 2
17840 194/300 Read block 4712 len 36864
17841 195/300 Read block 1263 len 32768
17842 196/300 Write block 3128 len 12288 data: 2 4 5
17843 197/300 Read block 4338 len 12288
17844 198/300 Read block 1986 len 40960
17845 199/300 Read block 4361 len 40960
17846 200/300 Read block 3976 len 24576
17847 201/300 Flush
17848 202/300 Write block 897 len 40960 data: 2 2 2 2 3 3 3 4 2 2
17849 203/300 Flush
17850 204/300 Read block 1033 len 4096
17851 205/300 Read block 4024 len 4096
17852 206/300 Read block 4725 len 24576
17853 207/300 Flush
17854 208/300 Read block 470 len 28672
17855 209/300 Read block 2776 len 36864
17856 210/300 Read block 2007 len 32768
17857 211/300 Write block 4558 len 28672 data: 4 3 4 4 5 6 7
17858 212/300 Read block 4708 len 4096
17859 213/300 Write block 182 len 20480 data: 5 6 6 6 6
17860 214/300 Read block 4043 len 16384
17861 215/300 Write block 2967 len 36864 data: 3 3 4 3 4 5 5 5 4
17862 216/300 Write block 1661 len 20480 data: 4 5 5 4 3
17863 217/300 Read block 4596 len 36864
17864 218/300 Write block 4744 len 28672 data: 3 4 5 3 3 4 5
17865 219/300 Write block 2631 len 24576 data: 3 4 6 4 5 5
17866 220/300 Read block 441 len 16384
17867 221/300 Read block 3646 len 12288
17868 222/300 Read block 2206 len 8192
17869 223/300 Read block 2376 len 28672
17870 224/300 Write block 459 len 24576 data: 2 4 4 4 5 6
17871 225/300 Read block 2117 len 12288
17872 226/300 Read block 3679 len 28672
17873 227/300 Write block 2960 len 36864 data: 3 3 5 5 4 4 4 4 4
17874 228/300 Write block 784 len 40960 data: 3 2 2 2 2 3 3 3 3 3
17875 229/300 Read block 2016 len 36864
17876 230/300 Read block 2354 len 28672
17877 231/300 Read block 664 len 4096
17878 232/300 Read block 2279 len 20480
17879 233/300 Read block 1288 len 40960
17880 234/300 Read block 97 len 36864
17881 235/300 Write block 3098 len 8192 data: 4 4
17882 236/300 Write block 3971 len 8192 data: 4 3
17883 237/300 Read block 3789 len 16384
17884 238/300 Read block 4741 len 4096
17885 239/300 Read block 4858 len 20480
17886 240/300 Flush
17887 241/300 Read block 122 len 8192
17888 242/300 Read block 3458 len 28672
17889 243/300 Read block 1613 len 4096
17890 244/300 Write block 4836 len 32768 data: 6 6 7 6 5 5 6 5
17891 245/300 Write block 2231 len 4096 data: 4
17892 246/300 Write block 1858 len 36864 data: 6 7 6 5 5 3 3 4 5
17893 247/300 Flush
17894 248/300 Read block 688 len 12288
17895 249/300 Read block 3955 len 24576
17896 250/300 Write block 4985 len 16384 data: 4 4 4 4
17897 251/300 Read block 1126 len 12288
17898 252/300 Flush
17899 253/300 Write block 3936 len 36864 data: 5 6 4 4 4 4 2 2 2
17900 254/300 Read block 24 len 36864
17901 255/300 Write block 3449 len 20480 data: 4 4 5 5 6
17902 256/300 Read block 529 len 20480
17903 257/300 Write block 510 len 8192 data: 4 5
17904 258/300 Write block 3279 len 4096 data: 5
17905 259/300 Read block 4890 len 12288
17906 260/300 Read block 2652 len 20480
17907 261/300 Read block 364 len 20480
17908 262/300 Write block 4445 len 40960 data: 3 4 3 3 3 3 3 3 4 3
17909 263/300 Write block 2371 len 12288 data: 4 4 4
17910 264/300 Flush
17911 265/300 Read block 440 len 32768
17912 266/300 Read block 659 len 32768
17913 267/300 Read block 4384 len 40960
17914 268/300 Read block 1332 len 16384
17915 269/300 Read block 4127 len 40960
17916 270/300 Write block 2763 len 28672 data: 2 3 4 5 6 5 4
17917 271/300 Read block 1178 len 40960
17918 272/300 Write block 809 len 16384 data: 5 5 5 5
17919 273/300 Write block 1446 len 20480 data: 3 3 4 5 5
17920 274/300 Read block 3897 len 36864
17921 275/300 Write block 4805 len 32768 data: 7 4 5 5 4 6 5 3
17922 276/300 Read block 566 len 28672
17923 277/300 Read block 4266 len 16384
17924 278/300 Flush
17925 279/300 Read block 2571 len 24576
17926 280/300 Read block 2266 len 28672
17927 281/300 Read block 1722 len 32768
17928 282/300 Read block 1101 len 36864
17929 283/300 Read block 1652 len 4096
17930 284/300 Write block 1627 len 32768 data: 9 7 6 5 5 5 5 4
17931 285/300 Write block 3131 len 20480 data: 5 5 5 4 5
17932 286/300 Read block 1408 len 24576
17933 287/300 Write block 3142 len 16384 data: 8 9 6 6
17934 288/300 Read block 3766 len 36864
17935 289/300 Read block 1512 len 24576
17936 290/300 Flush
17937 291/300 Write block 573 len 20480 data: 3 3 4 4 4
17938 292/300 Write block 3556 len 12288 data: 4 4 4
17939 293/300 Read block 4206 len 32768
17940 294/300 Read block 4641 len 28672
17941 295/300 Read block 1715 len 28672
17942 296/300 Read block 2231 len 4096
17943 297/300 Write block 1770 len 28672 data: 4 4 5 5 4 4 4
17944 298/300 Read block 4543 len 24576
17945 299/300 Write block 352 len 12288 data: 6 6 6
17946 300/300 Read block 364 len 32768
17947 301/300 Write block 3080 len 20480 data: 4 4 3 2 2
17948 Main task has work to do, go find it
17949 got dsc Start(0)
17950 start 0
17951 Jan 27 19:40:09.119 INFO request completed, latency_us: 252, response_code: 204, uri: /start/cid/0, method: POST, req_id: 22238fbf-e189-4b55-9b96-04d9bf5278e9, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
17952 Replay: started 0, returned:Ok(())
17953 ----------------------------------------------------------------
17954 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
17955 States: Offline Active Active
17956 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
17957 5829 Acked Read 10 Sent Done Done false
17958 5830 Acked Write 2 Sent Done Done false
17959 5831 Acked Read 4 Sent Done Done false
17960 5832 Acked Read 3 Sent Done Done false
17961 5833 Acked Read 8 Sent Done Done false
17962 5834 Acked Write 2 Sent Done Done false
17963 5835 Acked Read 4 Sent Done Done false
17964 5836 Acked Write 6 Sent Done Done false
17965 5837 Acked Read 8 Sent Done Done false
17966 5838 Acked Write 3 Sent Done Done false
17967 5839 Acked Flush 0 Sent Done Done false
17968 5840 Acked Read 6 Sent Done Done false
17969 5841 Acked Flush 0 Sent Done Done false
17970 5842 Acked Read 4 Sent Done Done false
17971 5843 Acked Read 3 Sent Done Done false
17972 5844 Acked Read 7 Sent Done Done false
17973 5845 Acked Write 4 Sent Done Done false
17974 5846 Acked Read 6 Sent Done Done false
17975 5847 Acked Read 8 Sent Done Done false
17976 5848 Acked Read 6 Sent Done Done false
17977 5849 Acked Read 3 Sent Done Done false
17978 5850 Acked Read 5 Sent Done Done false
17979 5851 Acked Flush 0 Sent Done Done false
17980 5852 Acked Read 3 Sent Done Done false
17981 5853 Acked Read 8 Sent Done Done false
17982 5854 Acked Write 2 Sent Done Done false
17983 5855 Acked Write 2 Sent Done Done false
17984 5856 Acked Flush 0 Sent Done Done false
17985 5857 Acked Write 5 Sent Done Done false
17986 5858 Acked Read 7 Sent Done Done false
17987 5859 Acked Flush 0 Sent Done Done false
17988 5860 Acked Write 3 Sent Done Done false
17989 5861 Acked Read 6 Sent Done Done false
17990 5862 Acked Read 6 Sent Done Done false
17991 5863 Acked Write 4 Sent Done Done false
17992 5864 Acked Read 2 Sent Done Done false
17993 5865 Acked Read 8 Sent Done Done false
17994 5866 Acked Read 4 Sent Done Done false
17995 5867 Acked Write 8 Sent Done Done false
17996 5868 Acked Read 8 Sent Done Done false
17997 5869 Acked Write 2 Sent Done Done false
17998 5870 Acked Write 4 Sent Done Done false
17999 5871 Acked Read 4 Sent Done Done false
18000 5872 Acked Write 10 Sent Done Done false
18001 5873 Acked Read 8 Sent Done Done false
18002 5874 Acked Write 3 Sent Done Done false
18003 5875 Acked Write 5 Sent Done Done false
18004 5876 Acked Read 2 Sent Done Done false
18005 5877 Acked Write 6 Sent Done Done false
18006 5878 Acked Write 5 Sent Done Done false
18007 5879 Acked Read 10 Sent Done Done false
18008 5880 Acked Read 8 Sent Done Done false
18009 5881 Acked Flush 0 Sent Done Done false
18010 5882 Acked Read 8 Sent Done Done false
18011 5883 Acked Write 8 Sent Done Done false
18012 5884 Acked Read 10 Sent Done Done false
18013 5885 Acked Read 5 Sent Done Done false
18014 5886 Acked Write 10 Sent Done Done false
18015 5887 Acked Read 4 Sent Done Done false
18016 5888 Acked Read 6 Sent Done Done false
18017 5889 Acked Write 5 Sent Done Done false
18018 5890 Acked Read 2 Sent Done Done false
18019 5891 Acked Read 4 Sent Done Done false
18020 5892 Acked Read 5 Sent Done Done false
18021 5893 Acked Write 1 Sent Done Done false
18022 5894 Acked Flush 0 Sent Done Done false
18023 5895 Acked Write 7 Sent Done Done false
18024 5896 Acked Write 10 Sent Done Done false
18025 5897 Acked Write 2 Sent Done Done false
18026 5898 Acked Read 5 Sent Done Done false
18027 5899 Acked Read 5 Sent Done Done false
18028 5900 Acked Flush 0 Sent Done Done false
18029 5901 Acked Read 8 Sent Done Done false
18030 5902 Acked Write 6 Sent Done Done false
18031 5903 Acked Write 7 Sent Done Done false
18032 5904 Acked Read 4 Sent Done Done false
18033 5905 Acked Read 9 Sent Done Done false
18034 5906 Acked Write 10 Sent Done Done false
18035 5907 Acked Read 5 Sent Done Done false
18036 5908 Acked Read 3 Sent Done Done false
18037 5909 Acked Write 2 Sent Done Done false
18038 5910 Acked Write 10 Sent Done Done false
18039 5911 Acked Write 5 Sent Done Done false
18040 5912 Acked Write 5 Sent Done Done false
18041 5913 Acked Write 9 Sent Done Done false
18042 5914 Acked Read 6 Sent Done Done false
18043 5915 Acked Write 10 Sent Done Done false
18044 5916 Acked Read 3 Sent Done Done false
18045 5917 Acked Write 1 Sent Done Done false
18046 5918 Acked Write 7 Sent Done Done false
18047 5919 Acked Write 5 Sent Done Done false
18048 5920 Acked Write 7 Sent Done Done false
18049 5921 Acked Write 7 Sent Done Done false
18050 5922 Acked Read 6 Sent Done Done false
18051 5923 Acked Read 8 Sent Done Done false
18052 5924 Acked Flush 0 Sent Done Done false
18053 5925 Acked Flush 0 Sent Done Done false
18054 5926 Acked Read 7 Sent Done Done false
18055 5927 Acked Write 3 Sent Done Done false
18056 5928 Acked Write 7 Sent Done Done false
18057 5929 Acked Write 5 Sent Done Done false
18058 5930 Acked Read 8 Sent Done Done false
18059 5931 Acked Read 10 Sent Done Done false
18060 5932 Acked Write 3 Sent Done Done false
18061 5933 Acked Read 1 Sent Done Done false
18062 5934 Acked Read 3 Sent Done Done false
18063 5935 Acked Read 6 Sent Done Done false
18064 5936 Acked Read 9 Sent Done Done false
18065 5937 Acked Read 2 Sent Done Done false
18066 5938 Acked Read 9 Sent Done Done false
18067 5939 Acked Write 5 Sent Done Done false
18068 5940 Acked Write 8 Sent Done Done false
18069 5941 Acked Read 10 Sent Done Done false
18070 5942 Acked Write 2 Sent Done Done false
18071 5943 Acked Read 3 Sent Done Done false
18072 5944 Acked Read 5 Sent Done Done false
18073 5945 Acked Read 2 Sent Done Done false
18074 5946 Acked Read 6 Sent Done Done false
18075 5947 Acked Read 3 Sent Done Done false
18076 5948 Acked Read 9 Sent Done Done false
18077 5949 Acked Read 4 Sent Done Done false
18078 5950 Acked Write 7 Sent Done Done false
18079 5951 Acked Read 10 Sent Done Done false
18080 5952 Acked Flush 0 Sent Done Done false
18081 5953 Acked Write 7 Sent Done Done false
18082 5954 Acked Read 8 Sent Done Done false
18083 5955 Acked Read 10 Sent Done Done false
18084 5956 Acked Write 10 Sent Done Done false
18085 5957 Acked Write 4 Sent Done Done false
18086 5958 Acked Read 4 Sent Done Done false
18087 5959 Acked Write 7 Sent Done Done false
18088 5960 Acked Read 6 Sent Done Done false
18089 5961 Acked Read 2 Sent Done Done false
18090 5962 Acked Read 1 Sent Done Done false
18091 5963 Acked Write 5 Sent Done Done false
18092 5964 Acked Write 3 Sent Done Done false
18093 5965 Acked Read 4 Sent Done Done false
18094 5966 Acked Flush 0 Sent Done Done false
18095 5967 Acked Write 9 Sent Done Done false
18096 5968 Acked Write 5 Sent Done Done false
18097 5969 Acked Read 3 Sent Done Done false
18098 5970 Acked Read 1 Sent Done Done false
18099 5971 Acked Write 6 Sent Done Done false
18100 5972 Acked Read 10 Sent Done Done false
18101 5973 Acked Read 9 Sent Done Done false
18102 5974 Acked Write 9 Sent Done Done false
18103 5975 Acked Read 3 Sent Done Done false
18104 5976 Acked Write 9 Sent Done Done false
18105 5977 Acked Flush 0 Sent Done Done false
18106 5978 Acked Write 6 Sent Done Done false
18107 5979 Acked Read 5 Sent Done Done false
18108 5980 Acked Flush 0 Sent Done Done false
18109 5981 Acked Read 4 Sent Done Done false
18110 5982 Acked Read 6 Sent Done Done false
18111 5983 Acked Write 8 Sent Done Done false
18112 5984 Acked Write 2 Sent Done Done false
18113 5985 Acked Write 5 Sent Done Done false
18114 5986 Acked Write 6 Sent Done Done false
18115 5987 Acked Flush 0 Sent Done Done false
18116 5988 Acked Write 7 Sent Done Done false
18117 5989 Acked Read 7 Sent Done Done false
18118 5990 Acked Read 3 Sent Done Done false
18119 5991 Acked Write 7 Sent Done Done false
18120 5992 Acked Write 6 Sent Done Done false
18121 5993 Acked Write 3 Sent Done Done false
18122 5994 Acked Write 3 Sent Done Done false
18123 5995 Acked Write 9 Sent Done Done false
18124 5996 Acked Flush 0 Sent Done Done false
18125 5997 Acked Read 3 Sent Done Done false
18126 5998 Acked Flush 0 Sent Done Done false
18127 5999 Acked Read 9 Sent Done Done false
18128 6000 Acked Write 10 Sent Done Done false
18129 6001 Acked Write 7 Sent Done Done false
18130 6002 Acked Read 9 Sent Done Done false
18131 6003 Acked Read 10 Sent Done Done false
18132 6004 Acked Read 4 Sent Done Done false
18133 6005 Acked Read 1 Sent Done Done false
18134 6006 Acked Write 3 Sent Done Done false
18135 6007 Acked Read 1 Sent Done Done false
18136 6008 Acked Write 7 Sent Done Done false
18137 6009 Acked Flush 0 Sent Done Done false
18138 6010 Acked Write 10 Sent Done Done false
18139 6011 Acked Read 4 Sent Done Done false
18140 6012 Acked Read 5 Sent Done Done false
18141 6013 Acked Write 10 Sent Done Done false
18142 6014 Acked Read 4 Sent Done Done false
18143 6015 Acked Flush 0 Sent Done Done false
18144 6016 Acked Write 2 Sent Done Done false
18145 6017 Acked Write 7 Sent Done Done false
18146 6018 Acked Read 5 Sent Done Done false
18147 6019 Acked Write 3 Sent Done Done false
18148 6020 Acked Write 8 Sent Done Done false
18149 6021 Acked Write 5 Sent Done Done false
18150 6022 Acked Read 9 Sent Done Done false
18151 6023 Acked Read 8 Sent Done Done false
18152 6024 Acked Write 3 Sent Done Done false
18153 6025 Acked Read 3 Sent Done Done false
18154 6026 Acked Read 10 Sent Done Done false
18155 6027 Acked Read 10 Sent Done Done false
18156 6028 Acked Read 6 Sent Done Done false
18157 6029 Acked Flush 0 Sent Done Done false
18158 6030 Acked Write 10 Sent Done Done false
18159 6031 Acked Flush 0 Sent Done Done false
18160 6032 Acked Read 1 Sent Done Done false
18161 6033 Acked Read 1 Sent Done Done false
18162 6034 Acked Read 6 Sent Done Done false
18163 6035 Acked Flush 0 Sent Done Done false
18164 6036 Acked Read 7 Sent Done Done false
18165 6037 Acked Read 9 Sent Done Done false
18166 6038 Acked Read 8 Sent Done Done false
18167 6039 Acked Write 7 Sent Done Done false
18168 6040 Acked Read 1 Sent Done Done false
18169 6041 Acked Write 5 Sent Done Done false
18170 6042 Acked Read 4 Sent Done Done false
18171 6043 Acked Write 9 Sent Done Done false
18172 6044 Acked Write 5 Sent Done Done false
18173 6045 Acked Read 9 Sent Done Done false
18174 6046 Acked Write 7 Sent Done Done false
18175 6047 Acked Write 6 Sent Done Done false
18176 6048 Acked Read 4 Sent Done Done false
18177 6049 Acked Read 3 Sent Done Done false
18178 6050 Acked Read 2 Sent Done Done false
18179 6051 Acked Read 7 Sent Done Done false
18180 6052 Acked Write 6 Sent Done Done false
18181 6053 Acked Read 3 Sent Done Done false
18182 6054 Acked Read 7 Sent Done Done false
18183 6055 Acked Write 9 Sent Done Done false
18184 6056 Acked Write 10 Sent Done Done false
18185 6057 Acked Read 9 Sent Done Done false
18186 6058 Acked Read 7 Sent Done Done false
18187 6059 Acked Read 1 Sent Done Done false
18188 6060 Acked Read 5 Sent Done Done false
18189 6061 Acked Read 10 Sent Done Done false
18190 6062 Acked Read 9 Sent Done Done false
18191 6063 Acked Write 2 Sent Done Done false
18192 6064 Acked Write 2 Sent Done Done false
18193 6065 Acked Read 4 Sent Done Done false
18194 6066 Acked Read 1 Sent Done Done false
18195 6067 Acked Read 5 Sent Done Done false
18196 6068 Acked Flush 0 Sent Done Done false
18197 6069 Acked Read 2 Sent Done Done false
18198 6070 Acked Read 7 Sent Done Done false
18199 6071 Acked Read 1 Sent Done Done false
18200 6072 Acked Write 8 Sent Done Done false
18201 6073 Acked Write 1 Sent Done Done false
18202 6074 Acked Write 9 Sent Done Done false
18203 6075 Acked Flush 0 Sent Done Done false
18204 6076 Acked Read 3 Sent Done Done false
18205 6077 Acked Read 6 Sent Done Done false
18206 6078 Acked Write 4 Sent Done Done false
18207 6079 Acked Read 3 Sent Done Done false
18208 6080 Acked Flush 0 Sent Done Done false
18209 6081 Acked Write 9 Sent Done Done false
18210 6082 Acked Read 9 Sent Done Done false
18211 6083 Acked Write 5 Sent Done Done false
18212 6084 Acked Read 5 Sent Done Done false
18213 6085 Acked Write 2 Sent Done Done false
18214 6086 Acked Write 1 Sent Done Done false
18215 6087 Acked Read 3 Sent Done Done false
18216 6088 Acked Read 5 Sent Done Done false
18217 6089 Acked Read 5 Sent Done Done false
18218 6090 Acked Write 10 Sent Done Done false
18219 6091 Acked Write 3 Sent Done Done false
18220 6092 Acked Flush 0 Sent Done Done false
18221 6093 Acked Read 8 Sent Done Done false
18222 6094 Acked Read 8 Sent Done Done false
18223 6095 Acked Read 10 Sent Done Done false
18224 6096 Acked Read 4 Sent Done Done false
18225 6097 Acked Read 10 Sent Done Done false
18226 6098 Acked Write 7 Sent Done Done false
18227 6099 Acked Read 10 Sent Done Done false
18228 6100 Acked Write 4 Sent Done Done false
18229 6101 Acked Write 5 Sent Done Done false
18230 6102 Acked Read 9 Sent Done Done false
18231 6103 Acked Write 8 Sent Done Done false
18232 6104 Acked Read 7 Sent Done Done false
18233 6105 Acked Read 4 Sent Done Done false
18234 6106 Acked Flush 0 Sent Done Done false
18235 6107 Acked Read 6 Sent Done Done false
18236 6108 Acked Read 7 Sent Done Done false
18237 6109 Acked Read 8 Sent Done Done false
18238 6110 Acked Read 9 Sent Done Done false
18239 6111 Acked Read 1 Sent Done Done false
18240 6112 Acked Write 8 Sent Done Done false
18241 6113 Acked Write 5 Sent Done Done false
18242 6114 Acked Read 6 Sent Done Done false
18243 6115 Acked Write 4 Sent Done Done false
18244 6116 Acked Read 9 Sent Done Done false
18245 6117 Acked Read 6 Sent Done Done false
18246 6118 Acked Flush 0 Sent Done Done false
18247 6119 Acked Write 5 Sent Done Done false
18248 6120 Acked Write 3 Sent Done Done false
18249 6121 Acked Read 8 Sent Done Done false
18250 6122 Acked Read 7 Sent Done Done false
18251 6123 Acked Read 7 Sent Done Done false
18252 6124 Acked Read 1 Sent Done Done false
18253 6125 Acked Write 7 Sent Done Done false
18254 6126 Acked Read 6 Sent Done Done false
18255 6127 Acked Write 3 Sent Done Done false
18256 6128 Acked Read 8 Sent Done Done false
18257 6129 Acked Write 5 Sent Done Done false
18258 STATES DS:0 DS:1 DS:2 TOTAL
18259 Sent 301 0 0 301
18260 Done 0 301 301 602
18261 Skipped 0 0 0 0
18262 Error 0 0 0 0
18263 Last Flush: 5828 6118 6118
18264 Downstairs last five completed: 5828 5827 5826 5825 5824
18265 Upstairs last five completed: 6129 6128 6127 6126 6125
18266 CLIENT: Up:0 ds:301 act:2
18267 [8810] got start action
18268 [8810] I am going to restart
18269 Starting downstairs at port 8810
18270 Make output file at "/tmp/dsc/downstairs-8810.txt"
18271 [8810][0] reports Starting
18272 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1217)
18273 [8810][0] reports Running
18274 ----------------------------------------------------------------
18275 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
18276 States: Offline Active Active
18277 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18278 5829 Acked Read 10 Sent Done Done false
18279 5830 Acked Write 2 Sent Done Done false
18280 5831 Acked Read 4 Sent Done Done false
18281 5832 Acked Read 3 Sent Done Done false
18282 5833 Acked Read 8 Sent Done Done false
18283 5834 Acked Write 2 Sent Done Done false
18284 5835 Acked Read 4 Sent Done Done false
18285 5836 Acked Write 6 Sent Done Done false
18286 5837 Acked Read 8 Sent Done Done false
18287 5838 Acked Write 3 Sent Done Done false
18288 5839 Acked Flush 0 Sent Done Done false
18289 5840 Acked Read 6 Sent Done Done false
18290 5841 Acked Flush 0 Sent Done Done false
18291 5842 Acked Read 4 Sent Done Done false
18292 5843 Acked Read 3 Sent Done Done false
18293 5844 Acked Read 7 Sent Done Done false
18294 5845 Acked Write 4 Sent Done Done false
18295 5846 Acked Read 6 Sent Done Done false
18296 5847 Acked Read 8 Sent Done Done false
18297 5848 Acked Read 6 Sent Done Done false
18298 5849 Acked Read 3 Sent Done Done false
18299 5850 Acked Read 5 Sent Done Done false
18300 5851 Acked Flush 0 Sent Done Done false
18301 5852 Acked Read 3 Sent Done Done false
18302 5853 Acked Read 8 Sent Done Done false
18303 5854 Acked Write 2 Sent Done Done false
18304 5855 Acked Write 2 Sent Done Done false
18305 5856 Acked Flush 0 Sent Done Done false
18306 5857 Acked Write 5 Sent Done Done false
18307 5858 Acked Read 7 Sent Done Done false
18308 5859 Acked Flush 0 Sent Done Done false
18309 5860 Acked Write 3 Sent Done Done false
18310 5861 Acked Read 6 Sent Done Done false
18311 5862 Acked Read 6 Sent Done Done false
18312 5863 Acked Write 4 Sent Done Done false
18313 5864 Acked Read 2 Sent Done Done false
18314 5865 Acked Read 8 Sent Done Done false
18315 5866 Acked Read 4 Sent Done Done false
18316 5867 Acked Write 8 Sent Done Done false
18317 5868 Acked Read 8 Sent Done Done false
18318 5869 Acked Write 2 Sent Done Done false
18319 5870 Acked Write 4 Sent Done Done false
18320 5871 Acked Read 4 Sent Done Done false
18321 5872 Acked Write 10 Sent Done Done false
18322 5873 Acked Read 8 Sent Done Done false
18323 5874 Acked Write 3 Sent Done Done false
18324 5875 Acked Write 5 Sent Done Done false
18325 5876 Acked Read 2 Sent Done Done false
18326 5877 Acked Write 6 Sent Done Done false
18327 5878 Acked Write 5 Sent Done Done false
18328 5879 Acked Read 10 Sent Done Done false
18329 5880 Acked Read 8 Sent Done Done false
18330 5881 Acked Flush 0 Sent Done Done false
18331 5882 Acked Read 8 Sent Done Done false
18332 5883 Acked Write 8 Sent Done Done false
18333 5884 Acked Read 10 Sent Done Done false
18334 5885 Acked Read 5 Sent Done Done false
18335 5886 Acked Write 10 Sent Done Done false
18336 5887 Acked Read 4 Sent Done Done false
18337 5888 Acked Read 6 Sent Done Done false
18338 5889 Acked Write 5 Sent Done Done false
18339 5890 Acked Read 2 Sent Done Done false
18340 5891 Acked Read 4 Sent Done Done false
18341 5892 Acked Read 5 Sent Done Done false
18342 5893 Acked Write 1 Sent Done Done false
18343 5894 Acked Flush 0 Sent Done Done false
18344 5895 Acked Write 7 Sent Done Done false
18345 5896 Acked Write 10 Sent Done Done false
18346 5897 Acked Write 2 Sent Done Done false
18347 5898 Acked Read 5 Sent Done Done false
18348 5899 Acked Read 5 Sent Done Done false
18349 5900 Acked Flush 0 Sent Done Done false
18350 5901 Acked Read 8 Sent Done Done false
18351 5902 Acked Write 6 Sent Done Done false
18352 5903 Acked Write 7 Sent Done Done false
18353 5904 Acked Read 4 Sent Done Done false
18354 5905 Acked Read 9 Sent Done Done false
18355 5906 Acked Write 10 Sent Done Done false
18356 5907 Acked Read 5 Sent Done Done false
18357 5908 Acked Read 3 Sent Done Done false
18358 5909 Acked Write 2 Sent Done Done false
18359 5910 Acked Write 10 Sent Done Done false
18360 5911 Acked Write 5 Sent Done Done false
18361 5912 Acked Write 5 Sent Done Done false
18362 5913 Acked Write 9 Sent Done Done false
18363 5914 Acked Read 6 Sent Done Done false
18364 5915 Acked Write 10 Sent Done Done false
18365 5916 Acked Read 3 Sent Done Done false
18366 5917 Acked Write 1 Sent Done Done false
18367 5918 Acked Write 7 Sent Done Done false
18368 5919 Acked Write 5 Sent Done Done false
18369 5920 Acked Write 7 Sent Done Done false
18370 5921 Acked Write 7 Sent Done Done false
18371 5922 Acked Read 6 Sent Done Done false
18372 5923 Acked Read 8 Sent Done Done false
18373 5924 Acked Flush 0 Sent Done Done false
18374 5925 Acked Flush 0 Sent Done Done false
18375 5926 Acked Read 7 Sent Done Done false
18376 5927 Acked Write 3 Sent Done Done false
18377 5928 Acked Write 7 Sent Done Done false
18378 5929 Acked Write 5 Sent Done Done false
18379 5930 Acked Read 8 Sent Done Done false
18380 5931 Acked Read 10 Sent Done Done false
18381 5932 Acked Write 3 Sent Done Done false
18382 5933 Acked Read 1 Sent Done Done false
18383 5934 Acked Read 3 Sent Done Done false
18384 5935 Acked Read 6 Sent Done Done false
18385 5936 Acked Read 9 Sent Done Done false
18386 5937 Acked Read 2 Sent Done Done false
18387 5938 Acked Read 9 Sent Done Done false
18388 5939 Acked Write 5 Sent Done Done false
18389 5940 Acked Write 8 Sent Done Done false
18390 5941 Acked Read 10 Sent Done Done false
18391 5942 Acked Write 2 Sent Done Done false
18392 5943 Acked Read 3 Sent Done Done false
18393 5944 Acked Read 5 Sent Done Done false
18394 5945 Acked Read 2 Sent Done Done false
18395 5946 Acked Read 6 Sent Done Done false
18396 5947 Acked Read 3 Sent Done Done false
18397 5948 Acked Read 9 Sent Done Done false
18398 5949 Acked Read 4 Sent Done Done false
18399 5950 Acked Write 7 Sent Done Done false
18400 5951 Acked Read 10 Sent Done Done false
18401 5952 Acked Flush 0 Sent Done Done false
18402 5953 Acked Write 7 Sent Done Done false
18403 5954 Acked Read 8 Sent Done Done false
18404 5955 Acked Read 10 Sent Done Done false
18405 5956 Acked Write 10 Sent Done Done false
18406 5957 Acked Write 4 Sent Done Done false
18407 5958 Acked Read 4 Sent Done Done false
18408 5959 Acked Write 7 Sent Done Done false
18409 5960 Acked Read 6 Sent Done Done false
18410 5961 Acked Read 2 Sent Done Done false
18411 5962 Acked Read 1 Sent Done Done false
18412 5963 Acked Write 5 Sent Done Done false
18413 5964 Acked Write 3 Sent Done Done false
18414 5965 Acked Read 4 Sent Done Done false
18415 5966 Acked Flush 0 Sent Done Done false
18416 5967 Acked Write 9 Sent Done Done false
18417 5968 Acked Write 5 Sent Done Done false
18418 5969 Acked Read 3 Sent Done Done false
18419 5970 Acked Read 1 Sent Done Done false
18420 5971 Acked Write 6 Sent Done Done false
18421 5972 Acked Read 10 Sent Done Done false
18422 5973 Acked Read 9 Sent Done Done false
18423 5974 Acked Write 9 Sent Done Done false
18424 5975 Acked Read 3 Sent Done Done false
18425 5976 Acked Write 9 Sent Done Done false
18426 5977 Acked Flush 0 Sent Done Done false
18427 5978 Acked Write 6 Sent Done Done false
18428 5979 Acked Read 5 Sent Done Done false
18429 5980 Acked Flush 0 Sent Done Done false
18430 5981 Acked Read 4 Sent Done Done false
18431 5982 Acked Read 6 Sent Done Done false
18432 5983 Acked Write 8 Sent Done Done false
18433 5984 Acked Write 2 Sent Done Done false
18434 5985 Acked Write 5 Sent Done Done false
18435 5986 Acked Write 6 Sent Done Done false
18436 5987 Acked Flush 0 Sent Done Done false
18437 5988 Acked Write 7 Sent Done Done false
18438 5989 Acked Read 7 Sent Done Done false
18439 5990 Acked Read 3 Sent Done Done false
18440 5991 Acked Write 7 Sent Done Done false
18441 5992 Acked Write 6 Sent Done Done false
18442 5993 Acked Write 3 Sent Done Done false
18443 5994 Acked Write 3 Sent Done Done false
18444 5995 Acked Write 9 Sent Done Done false
18445 5996 Acked Flush 0 Sent Done Done false
18446 5997 Acked Read 3 Sent Done Done false
18447 5998 Acked Flush 0 Sent Done Done false
18448 5999 Acked Read 9 Sent Done Done false
18449 6000 Acked Write 10 Sent Done Done false
18450 6001 Acked Write 7 Sent Done Done false
18451 6002 Acked Read 9 Sent Done Done false
18452 6003 Acked Read 10 Sent Done Done false
18453 6004 Acked Read 4 Sent Done Done false
18454 6005 Acked Read 1 Sent Done Done false
18455 6006 Acked Write 3 Sent Done Done false
18456 6007 Acked Read 1 Sent Done Done false
18457 6008 Acked Write 7 Sent Done Done false
18458 6009 Acked Flush 0 Sent Done Done false
18459 6010 Acked Write 10 Sent Done Done false
18460 6011 Acked Read 4 Sent Done Done false
18461 6012 Acked Read 5 Sent Done Done false
18462 6013 Acked Write 10 Sent Done Done false
18463 6014 Acked Read 4 Sent Done Done false
18464 6015 Acked Flush 0 Sent Done Done false
18465 6016 Acked Write 2 Sent Done Done false
18466 6017 Acked Write 7 Sent Done Done false
18467 6018 Acked Read 5 Sent Done Done false
18468 6019 Acked Write 3 Sent Done Done false
18469 6020 Acked Write 8 Sent Done Done false
18470 6021 Acked Write 5 Sent Done Done false
18471 6022 Acked Read 9 Sent Done Done false
18472 6023 Acked Read 8 Sent Done Done false
18473 6024 Acked Write 3 Sent Done Done false
18474 6025 Acked Read 3 Sent Done Done false
18475 6026 Acked Read 10 Sent Done Done false
18476 6027 Acked Read 10 Sent Done Done false
18477 6028 Acked Read 6 Sent Done Done false
18478 6029 Acked Flush 0 Sent Done Done false
18479 6030 Acked Write 10 Sent Done Done false
18480 6031 Acked Flush 0 Sent Done Done false
18481 6032 Acked Read 1 Sent Done Done false
18482 6033 Acked Read 1 Sent Done Done false
18483 6034 Acked Read 6 Sent Done Done false
18484 6035 Acked Flush 0 Sent Done Done false
18485 6036 Acked Read 7 Sent Done Done false
18486 6037 Acked Read 9 Sent Done Done false
18487 6038 Acked Read 8 Sent Done Done false
18488 6039 Acked Write 7 Sent Done Done false
18489 6040 Acked Read 1 Sent Done Done false
18490 6041 Acked Write 5 Sent Done Done false
18491 6042 Acked Read 4 Sent Done Done false
18492 6043 Acked Write 9 Sent Done Done false
18493 6044 Acked Write 5 Sent Done Done false
18494 6045 Acked Read 9 Sent Done Done false
18495 6046 Acked Write 7 Sent Done Done false
18496 6047 Acked Write 6 Sent Done Done false
18497 6048 Acked Read 4 Sent Done Done false
18498 6049 Acked Read 3 Sent Done Done false
18499 6050 Acked Read 2 Sent Done Done false
18500 6051 Acked Read 7 Sent Done Done false
18501 6052 Acked Write 6 Sent Done Done false
18502 6053 Acked Read 3 Sent Done Done false
18503 6054 Acked Read 7 Sent Done Done false
18504 6055 Acked Write 9 Sent Done Done false
18505 6056 Acked Write 10 Sent Done Done false
18506 6057 Acked Read 9 Sent Done Done false
18507 6058 Acked Read 7 Sent Done Done false
18508 6059 Acked Read 1 Sent Done Done false
18509 6060 Acked Read 5 Sent Done Done false
18510 6061 Acked Read 10 Sent Done Done false
18511 6062 Acked Read 9 Sent Done Done false
18512 6063 Acked Write 2 Sent Done Done false
18513 6064 Acked Write 2 Sent Done Done false
18514 6065 Acked Read 4 Sent Done Done false
18515 6066 Acked Read 1 Sent Done Done false
18516 6067 Acked Read 5 Sent Done Done false
18517 6068 Acked Flush 0 Sent Done Done false
18518 6069 Acked Read 2 Sent Done Done false
18519 6070 Acked Read 7 Sent Done Done false
18520 6071 Acked Read 1 Sent Done Done false
18521 6072 Acked Write 8 Sent Done Done false
18522 6073 Acked Write 1 Sent Done Done false
18523 6074 Acked Write 9 Sent Done Done false
18524 6075 Acked Flush 0 Sent Done Done false
18525 6076 Acked Read 3 Sent Done Done false
18526 6077 Acked Read 6 Sent Done Done false
18527 6078 Acked Write 4 Sent Done Done false
18528 6079 Acked Read 3 Sent Done Done false
18529 6080 Acked Flush 0 Sent Done Done false
18530 6081 Acked Write 9 Sent Done Done false
18531 6082 Acked Read 9 Sent Done Done false
18532 6083 Acked Write 5 Sent Done Done false
18533 6084 Acked Read 5 Sent Done Done false
18534 6085 Acked Write 2 Sent Done Done false
18535 6086 Acked Write 1 Sent Done Done false
18536 6087 Acked Read 3 Sent Done Done false
18537 6088 Acked Read 5 Sent Done Done false
18538 6089 Acked Read 5 Sent Done Done false
18539 6090 Acked Write 10 Sent Done Done false
18540 6091 Acked Write 3 Sent Done Done false
18541 6092 Acked Flush 0 Sent Done Done false
18542 6093 Acked Read 8 Sent Done Done false
18543 6094 Acked Read 8 Sent Done Done false
18544 6095 Acked Read 10 Sent Done Done false
18545 6096 Acked Read 4 Sent Done Done false
18546 6097 Acked Read 10 Sent Done Done false
18547 6098 Acked Write 7 Sent Done Done false
18548 6099 Acked Read 10 Sent Done Done false
18549 6100 Acked Write 4 Sent Done Done false
18550 6101 Acked Write 5 Sent Done Done false
18551 6102 Acked Read 9 Sent Done Done false
18552 6103 Acked Write 8 Sent Done Done false
18553 6104 Acked Read 7 Sent Done Done false
18554 6105 Acked Read 4 Sent Done Done false
18555 6106 Acked Flush 0 Sent Done Done false
18556 6107 Acked Read 6 Sent Done Done false
18557 6108 Acked Read 7 Sent Done Done false
18558 6109 Acked Read 8 Sent Done Done false
18559 6110 Acked Read 9 Sent Done Done false
18560 6111 Acked Read 1 Sent Done Done false
18561 6112 Acked Write 8 Sent Done Done false
18562 6113 Acked Write 5 Sent Done Done false
18563 6114 Acked Read 6 Sent Done Done false
18564 6115 Acked Write 4 Sent Done Done false
18565 6116 Acked Read 9 Sent Done Done false
18566 6117 Acked Read 6 Sent Done Done false
18567 6118 Acked Flush 0 Sent Done Done false
18568 6119 Acked Write 5 Sent Done Done false
18569 6120 Acked Write 3 Sent Done Done false
18570 6121 Acked Read 8 Sent Done Done false
18571 6122 Acked Read 7 Sent Done Done false
18572 6123 Acked Read 7 Sent Done Done false
18573 6124 Acked Read 1 Sent Done Done false
18574 6125 Acked Write 7 Sent Done Done false
18575 6126 Acked Read 6 Sent Done Done false
18576 6127 Acked Write 3 Sent Done Done false
18577 6128 Acked Read 8 Sent Done Done false
18578 6129 Acked Write 5 Sent Done Done false
18579 6130 Acked Flush 0 Sent Done Done false
18580 STATES DS:0 DS:1 DS:2 TOTAL
18581 Sent 302 0 0 302
18582 Done 0 302 302 604
18583 Skipped 0 0 0 0
18584 Error 0 0 0 0
18585 Last Flush: 5828 6130 6130
18586 Downstairs last five completed: 5828 5827 5826 5825 5824
18587 Upstairs last five completed: 6130 6129 6128 6127 6126
18588 CLIENT: Up:0 ds:302 act:2
185892025-01-27T19:40:14.871ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
185902025-01-27T19:40:14.872ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:43221) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
185912025-01-27T19:40:14.873ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
185922025-01-27T19:40:14.874ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
185932025-01-27T19:40:14.874ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
185942025-01-27T19:40:14.874ZINFOcrucible: send last flush ID to this DS: 5828 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
185952025-01-27T19:40:14.874ZINFOcrucible: Replied this last flush ID: 5828 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
185962025-01-27T19:40:14.874ZINFOcrucible: [0] client re-new 302 jobs since flush 5828 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
185972025-01-27T19:40:14.874ZINFOcrucible: [0] Replayed 302 jobs since flush: 5828 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
18598 ----------------------------------------------------------------
18599 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
18600 Downstairs last five completed: 6130 6129 6128 6127 6126
18601 Upstairs last five completed: 6130 6129 6128 6127 6126
18602 CLIENT: Up:0 ds:0 act:3
18603 Replay: All jobs finished
18604 Main task has work to do, go find it
18605 got dsc Stop(2)
18606 stop 2
18607 Jan 27 19:40:17.124 [8830] Got stop action so:false kr:false
18608 INFO request completed, latency_us: 351, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 38fb13ac-8180-4a7d-b677-e6b7b14fb26b, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
186092025-01-27T19:40:17.125ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
18610 Jan 27 19:40:17.125 INFO request completed, {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:40:17.125397294Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
18611 latency_us: 123, {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:40:17.125443838Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
18612 response_code: 200, uri: /state/cid/2, method{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-27T19:40:17.125486983Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
18613 : GET, req_id: eedd0820-c5ca-478b-a451-edaca50510ea, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
18614 [8830] Exited with: ExitStatus(unix_wait_status(9))
18615 [8830][2] reports Exit
18616 Jan 27 19:40:21.127 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/2, method: GET, req_id: 7b74da59-e323-45f1-8f0e-609b6fdb7b7c, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
18617 001/300 Write block 1122 len 16384 data: 8 9 8 6
18618 002/300 Write block 2016 len 32768 data: 5 5 5 4 4 4 4 4
18619 003/300 Write block 2484 len 20480 data: 4 3 3 3 4
18620 004/300 Read block 1218 len 12288
18621 005/300 Write block 2660 len 12288 data: 4 4 3
18622 006/300 Read block 4581 len 32768
18623 007/300 Write block 3430 len 40960 data: 8 7 6 6 6 5 5 5 5 4
18624 008/300 Write block 4280 len 24576 data: 6 5 4 4 5 4
18625 009/300 Write block 1580 len 4096 data: 4
18626 010/300 Write block 3052 len 40960 data: 7 6 7 6 5 4 4 4 5 6
18627 011/300 Flush
18628 012/300 Read block 2153 len 32768
18629 013/300 Write block 3275 len 12288 data: 5 5 6
18630 014/300 Write block 1761 len 40960 data: 3 3 3 4 4 4 4 4 4 5
18631 015/300 Write block 2856 len 8192 data: 5 5
18632 016/300 Write block 2156 len 24576 data: 6 7 7 7 6 7
18633 017/300 Write block 1145 len 24576 data: 3 3 4 5 5 3
18634 018/300 Write block 2182 len 28672 data: 7 6 7 5 6 5 2
18635 019/300 Read block 4644 len 4096
18636 020/300 Flush
18637 021/300 Write block 3775 len 24576 data: 8 7 7 6 5 4
18638 022/300 Write block 3809 len 8192 data: 4 4
18639 023/300 Read block 3920 len 28672
18640 024/300 Read block 490 len 4096
18641 025/300 Write block 2252 len 20480 data: 3 3 4 6 7
18642 026/300 Write block 1736 len 40960 data: 4 4 5 6 7 7 5 5 5 4
18643 027/300 Write block 4062 len 40960 data: 4 4 3 3 3 3 3 3 3 3
18644 028/300 Read block 908 len 40960
18645 029/300 Flush
18646 030/300 Read block 2581 len 12288
18647 031/300 Flush
18648 032/300 Read block 3448 len 40960
18649 033/300 Write block 3442 len 24576 data: 3 3 2 2 2 2
18650 034/300 Read block 2065 len 28672
18651 035/300 Read block 4272 len 4096
18652 036/300 Read block 4578 len 16384
18653 037/300 Flush
18654 038/300 Read block 3450 len 36864
18655 039/300 Write block 3553 len 20480 data: 3 3 3 5 5
18656 040/300 Write block 3120 len 40960 data: 3 3 4 5 5 4 3 3 3 5
18657 041/300 Write block 4775 len 16384 data: 6 6 5 4
18658 042/300 Write block 1860 len 16384 data: 7 6 6 4
18659 043/300 Write block 3617 len 12288 data: 4 6 6
18660 044/300 Write block 4505 len 16384 data: 4 4 4 3
18661 045/300 Write block 4110 len 16384 data: 5 6 6 6
18662 046/300 Read block 1472 len 12288
18663 047/300 Write block 214 len 24576 data: 3 3 2 3 4 4
18664 048/300 Read block 767 len 20480
18665 049/300 Read block 3736 len 32768
18666 050/300 Write block 349 len 36864 data: 2 2 4 7 7 7 6 7 6
18667 051/300 Read block 2635 len 32768
18668 052/300 Write block 445 len 16384 data: 3 2 2 2
18669 053/300 Flush
18670 054/300 Flush
18671 055/300 Write block 4797 len 36864 data: 5 4 5 5 5 4 5 5 8
18672 056/300 Read block 598 len 8192
18673 057/300 Read block 2734 len 24576
18674 058/300 Read block 345 len 12288
18675 059/300 Write block 1258 len 24576 data: 3 2 2 2 2 3
18676 060/300 Flush
18677 061/300 Write block 199 len 4096 data: 7
18678 062/300 Write block 2405 len 16384 data: 3 3 4 4
18679 063/300 Flush
18680 064/300 Read block 3338 len 16384
18681 065/300 Write block 848 len 32768 data: 3 3 3 3 2 2 2 2
18682 066/300 Write block 3019 len 4096 data: 3
18683 067/300 Read block 2264 len 28672
18684 068/300 Read block 2276 len 24576
18685 069/300 Write block 51 len 4096 data: 5
18686 070/300 Write block 878 len 20480 data: 3 3 2 2 2
18687 071/300 Flush
18688 072/300 Read block 1331 len 12288
18689 073/300 Write block 670 len 40960 data: 3 3 3 3 3 3 4 4 4 3
18690 074/300 Read block 772 len 40960
18691 075/300 Write block 3600 len 12288 data: 5 3 3
18692 076/300 Read block 2003 len 32768
18693 077/300 Read block 1752 len 36864
18694 078/300 Write block 1529 len 12288 data: 3 4 5
18695 079/300 Write block 859 len 4096 data: 4
18696 080/300 Read block 2972 len 36864
18697 081/300 Write block 2597 len 28672 data: 3 4 5 6 5 4 4
18698 082/300 Flush
18699 083/300 Write block 4879 len 36864 data: 3 3 3 4 4 4 3 4 5
18700 084/300 Read block 2534 len 20480
18701 085/300 Read block 535 len 32768
18702 086/300 Write block 2701 len 28672 data: 3 4 4 3 2 2 2
18703 087/300 Read block 1471 len 40960
18704 088/300 Read block 495 len 40960
18705 089/300 Read block 2214 len 36864
18706 090/300 Read block 4310 len 8192
18707 091/300 Read block 2598 len 36864
18708 092/300 Write block 3184 len 4096 data: 4
18709 093/300 Write block 1063 len 16384 data: 2 2 3 3
18710 094/300 Read block 2311 len 32768
18711 095/300 Read block 1902 len 40960
18712 096/300 Read block 2117 len 4096
18713 097/300 Flush
18714 098/300 Read block 313 len 16384
18715 099/300 Write block 3046 len 32768 data: 5 4 5 5 6 8 8 7
18716 100/300 Flush
18717 101/300 Write block 4826 len 8192 data: 3 3
18718 102/300 Read block 1115 len 32768
18719 103/300 Read block 4032 len 40960
18720 104/300 Flush
18721 105/300 Read block 4829 len 8192
18722 106/300 Write block 2410 len 28672 data: 6 5 6 6 7 6 6
18723 107/300 Read block 4910 len 20480
18724 108/300 Write block 2411 len 20480 data: 6 7 7 8 7
18725 109/300 Read block 4072 len 28672
18726 110/300 Read block 588 len 24576
18727 111/300 Write block 3186 len 28672 data: 2 3 3 3 4 5 5
18728 112/300 Flush
18729 113/300 Read block 408 len 16384
18730 114/300 Read block 3286 len 28672
18731 115/300 Flush
18732 116/300 Write block 687 len 28672 data: 3 3 4 3 3 3 3
18733 117/300 Write block 1444 len 20480 data: 3 3 4 4 5
18734 118/300 Write block 3228 len 40960 data: 3 3 4 4 4 4 4 5 3 3
18735 119/300 Write block 2368 len 28672 data: 3 4 4 5 5 5 4
18736 120/300 Write block 1592 len 36864 data: 6 6 5 5 6 7 7 4 4
18737 121/300 Write block 1015 len 12288 data: 4 3 5
18738 122/300 Read block 1335 len 40960
18739 123/300 Read block 2812 len 36864
18740 124/300 Write block 682 len 36864 data: 4 5 5 4 4 4 4 5 4
18741 125/300 Read block 172 len 16384
18742 126/300 Write block 2053 len 20480 data: 3 3 3 3 3
18743 127/300 Read block 1568 len 28672
18744 128/300 Read block 1488 len 40960
18745 129/300 Read block 2356 len 32768
18746 130/300 Flush
18747 131/300 Write block 2925 len 12288 data: 3 3 4
18748 132/300 Read block 341 len 16384
18749 133/300 Write block 1323 len 32768 data: 3 3 3 4 4 4 5 4
18750 134/300 Write block 4475 len 12288 data: 7 7 7
18751 135/300 Read block 3052 len 32768
18752 136/300 Flush
18753 137/300 Read block 2520 len 20480
18754 138/300 Read block 2438 len 28672
18755 139/300 Read block 3247 len 32768
18756 140/300 Flush
18757 141/300 Read block 381 len 36864
18758 142/300 Read block 4084 len 4096
18759 143/300 Write block 2298 len 12288 data: 3 5 5
18760 144/300 Read block 2168 len 8192
18761 145/300 Write block 462 len 4096 data: 5
18762 146/300 Write block 3331 len 8192 data: 6 7
18763 147/300 Read block 3915 len 8192
18764 148/300 Flush
18765 149/300 Flush
18766 150/300 Read block 3630 len 8192
18767 151/300 Read block 355 len 4096
18768 152/300 Write block 1353 len 12288 data: 6 5 6
18769 153/300 Read block 4623 len 20480
18770 154/300 Write block 226 len 16384 data: 7 5 5 5
18771 155/300 Flush
18772 156/300 Write block 3439 len 28672 data: 5 3 3 4 4 3 3
18773 157/300 Write block 685 len 16384 data: 5 5 5 5
18774 158/300 Write block 3535 len 36864 data: 2 2 3 3 4 4 3 4 3
18775 159/300 Write block 4639 len 24576 data: 4 4 4 3 4 3
18776 160/300 Write block 4335 len 8192 data: 7 9
18777 161/300 Read block 764 len 40960
18778 162/300 Write block 1700 len 32768 data: 3 3 3 3 3 3 3 2
18779 163/300 Read block 27 len 28672
18780 164/300 Read block 3785 len 36864
18781 165/300 Write block 633 len 8192 data: 3 3
18782 166/300 Write block 2780 len 40960 data: 3 3 4 4 4 4 3 2 3 3
18783 167/300 Read block 4954 len 4096
18784 168/300 Read block 1456 len 40960
18785 169/300 Read block 3311 len 8192
18786 170/300 Read block 1994 len 36864
18787 171/300 Read block 1628 len 36864
18788 172/300 Write block 2973 len 8192 data: 6 6
18789 173/300 Write block 1513 len 20480 data: 2 2 3 3 3
18790 174/300 Write block 2339 len 24576 data: 5 6 6 5 5 5
18791 175/300 Flush
18792 176/300 Write block 305 len 12288 data: 4 4 4
18793 177/300 Read block 1227 len 8192
18794 178/300 Read block 4261 len 20480
18795 179/300 Read block 350 len 12288
18796 180/300 Write block 2627 len 16384 data: 3 3 4 3
18797 181/300 Write block 2765 len 32768 data: 5 6 7 6 5 5 5 5
18798 182/300 Write block 1719 len 40960 data: 4 4 4 4 4 3 3 4 4 4
18799 183/300 Read block 1214 len 32768
18800 184/300 Read block 3576 len 12288
18801 185/300 Write block 4797 len 28672 data: 6 5 6 6 6 5 6
18802 186/300 Read block 4366 len 32768
18803 187/300 Write block 578 len 40960 data: 5 5 6 6 6 8 5 4 3 4
18804 188/300 Flush
18805 189/300 Read block 2623 len 28672
18806 190/300 Read block 2196 len 36864
18807 191/300 Write block 4939 len 4096 data: 5
18808 192/300 Read block 4886 len 20480
18809 193/300 Write block 3372 len 32768 data: 8 6 5 4 4 3 3 3
18810 194/300 Read block 4635 len 4096
18811 195/300 Read block 2413 len 24576
18812 196/300 Read block 1092 len 32768
18813 197/300 Write block 2772 len 32768 data: 6 6 5 4 3 3 3 4
18814 198/300 Read block 151 len 16384
18815 199/300 Read block 1149 len 24576
18816 200/300 Read block 1696 len 24576
18817 201/300 Read block 299 len 16384
18818 202/300 Read block 3003 len 20480
18819 203/300 Write block 2782 len 40960 data: 5 5 5 5 4 3 4 4 4 4
18820 204/300 Write block 3744 len 24576 data: 3 3 3 4 5 5
18821 205/300 Flush
18822 206/300 Read block 2528 len 36864
18823 207/300 Write block 361 len 28672 data: 4 4 3 5 4 4 4
18824 208/300 Write block 2678 len 28672 data: 7 7 8 8 8 9 9
18825 209/300 Read block 2126 len 20480
18826 210/300 Read block 3568 len 4096
18827 211/300 Write block 1093 len 12288 data: 2 2 3
18828 212/300 Read block 2837 len 16384
18829 213/300 Write block 866 len 8192 data: 7 7
18830 214/300 Write block 4433 len 16384 data: 3 3 4 3
18831 215/300 Read block 3385 len 28672
18832 216/300 Write block 131 len 24576 data: 7 6 5 4 4 4
18833 217/300 Read block 3235 len 40960
18834 218/300 Flush
18835 219/300 Flush
18836 220/300 Read block 2128 len 8192
18837 221/300 Read block 1412 len 36864
18838 222/300 Read block 1550 len 4096
18839 223/300 Read block 4105 len 24576
18840 224/300 Write block 4260 len 36864 data: 5 4 4 5 6 6 6 5 5
18841 225/300 Write block 1924 len 28672 data: 5 5 4 3 3 3 3
18842 226/300 Read block 1582 len 40960
18843 227/300 Read block 1697 len 20480
18844 228/300 Read block 2866 len 36864
18845 229/300 Write block 4738 len 28672 data: 5 5 6 6 5 4 4
18846 230/300 Read block 2146 len 16384
18847 231/300 Write block 414 len 12288 data: 3 3 3
18848 232/300 Write block 511 len 16384 data: 6 4 4 4
18849 233/300 Write block 2653 len 16384 data: 6 6 6 6
18850 234/300 Write block 1669 len 32768 data: 3 3 3 4 5 7 6 6
18851 235/300 Read block 3484 len 8192
18852 236/300 Read block 1262 len 24576
18853 237/300 Read block 945 len 20480
18854 238/300 Read block 4984 len 36864
18855 239/300 Write block 2404 len 20480 data: 3 4 4 5 5
18856 240/300 Write block 4158 len 40960 data: 4 4 5 7 7 7 7 7 6 5
18857 241/300 Read block 1203 len 24576
18858 242/300 Read block 4958 len 28672
18859 243/300 Read block 3229 len 12288
18860 244/300 Write block 4708 len 16384 data: 6 5 5 4
18861 245/300 Read block 3813 len 32768
18862 246/300 Write block 2282 len 8192 data: 3 4
18863 247/300 Write block 590 len 4096 data: 6
18864 248/300 Flush
18865 249/300 Write block 4119 len 36864 data: 9 8 7 7 5 4 4 4 5
18866 250/300 Read block 3740 len 28672
18867 251/300 Write block 3360 len 32768 data: 4 4 4 3 4 4 4 6
18868 252/300 Read block 3598 len 36864
18869 253/300 Write block 1726 len 8192 data: 5 5
18870 254/300 Read block 1112 len 12288
18871 255/300 Read block 2988 len 20480
18872 256/300 Read block 1197 len 28672
18873 257/300 Read block 501 len 24576
18874 258/300 Read block 4897 len 40960
18875 259/300 Read block 4970 len 28672
18876 260/300 Read block 4513 len 4096
18877 261/300 Write block 1792 len 40960 data: 3 3 4 4 3 3 2 2 2 2
18878 262/300 Read block 2400 len 36864
18879 263/300 Read block 1029 len 24576
18880 264/300 Read block 3466 len 36864
18881 265/300 Write block 653 len 24576 data: 5 4 4 3 3 6
18882 266/300 Flush
18883 267/300 Read block 3227 len 12288
18884 268/300 Read block 147 len 20480
18885 269/300 Write block 4562 len 32768 data: 6 7 8 6 5 5 6 4
18886 270/300 Read block 4440 len 32768
18887 271/300 Write block 1883 len 32768 data: 4 4 4 6 5 6 7 6
18888 272/300 Read block 3469 len 8192
18889 273/300 Write block 2762 len 28672 data: 2 3 4 6 7 8 7
18890 274/300 Write block 4603 len 36864 data: 5 3 3 3 4 4 4 5 4
18891 275/300 Read block 420 len 12288
18892 276/300 Read block 756 len 12288
18893 277/300 Read block 1095 len 16384
18894 278/300 Read block 1001 len 8192
18895 279/300 Write block 4764 len 12288 data: 6 6 5
18896 280/300 Write block 3200 len 24576 data: 3 3 3 2 2 2
18897 281/300 Write block 1075 len 4096 data: 5
18898 282/300 Write block 221 len 28672 data: 4 6 6 6 6 8 6
18899 283/300 Read block 1455 len 20480
18900 284/300 Read block 742 len 16384
18901 285/300 Write block 4706 len 28672 data: 5 5 7 6 6 5 5
18902 286/300 Write block 4786 len 28672 data: 4 4 4 5 7 7 6
18903 287/300 Flush
18904 288/300 Flush
18905 289/300 Write block 1851 len 40960 data: 5 5 5 7 8 7 6 7 8 8
18906 290/300 Write block 4803 len 24576 data: 7 6 9 5 6 6
18907 291/300 Read block 4044 len 32768
18908 292/300 Read block 3309 len 4096
18909 293/300 Read block 1403 len 32768
18910 294/300 Write block 2171 len 12288 data: 5 5 6
18911 295/300 Write block 2370 len 8192 data: 5 6
18912 296/300 Write block 2240 len 36864 data: 5 5 5 5 5 6 6 7 6
18913 297/300 Flush
18914 298/300 Read block 749 len 32768
18915 299/300 Read block 1548 len 12288
18916 300/300 Write block 3653 len 12288 data: 3 3 3
18917 301/300 Read block 1950 len 24576
18918 Main task has work to do, go find it
18919 got dsc Start(2)
18920 start 2
18921 Jan 27 19:40:21.362 INFO request completed, latency_us: 154, response_code: 204, uri: /start/cid/2, method: POST, req_id: a97192fc-13d6-4f64-8383-5d4d98761464, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
18922 Replay: started 2, returned:Ok(())
18923 ----------------------------------------------------------------
18924 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
18925 States: Active Active Offline
18926 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
18927 6131 Acked Write 4 Done Done Sent false
18928 6132 Acked Write 8 Done Done Sent false
18929 6133 Acked Write 5 Done Done Sent false
18930 6134 Acked Read 3 Done Done Sent false
18931 6135 Acked Write 3 Done Done Sent false
18932 6136 Acked Read 8 Done Done Sent false
18933 6137 Acked Write 10 Done Done Sent false
18934 6138 Acked Write 6 Done Done Sent false
18935 6139 Acked Write 1 Done Done Sent false
18936 6140 Acked Write 10 Done Done Sent false
18937 6141 Acked Flush 0 Done Done Sent false
18938 6142 Acked Read 8 Done Done Sent false
18939 6143 Acked Write 3 Done Done Sent false
18940 6144 Acked Write 10 Done Done Sent false
18941 6145 Acked Write 2 Done Done Sent false
18942 6146 Acked Write 6 Done Done Sent false
18943 6147 Acked Write 6 Done Done Sent false
18944 6148 Acked Write 7 Done Done Sent false
18945 6149 Acked Read 1 Done Done Sent false
18946 6150 Acked Flush 0 Done Done Sent false
18947 6151 Acked Write 6 Done Done Sent false
18948 6152 Acked Write 2 Done Done Sent false
18949 6153 Acked Read 7 Done Done Sent false
18950 6154 Acked Read 1 Done Done Sent false
18951 6155 Acked Write 5 Done Done Sent false
18952 6156 Acked Write 10 Done Done Sent false
18953 6157 Acked Write 10 Done Done Sent false
18954 6158 Acked Read 10 Done Done Sent false
18955 6159 Acked Flush 0 Done Done Sent false
18956 6160 Acked Read 3 Done Done Sent false
18957 6161 Acked Flush 0 Done Done Sent false
18958 6162 Acked Read 10 Done Done Sent false
18959 6163 Acked Write 6 Done Done Sent false
18960 6164 Acked Read 7 Done Done Sent false
18961 6165 Acked Read 1 Done Done Sent false
18962 6166 Acked Read 4 Done Done Sent false
18963 6167 Acked Flush 0 Done Done Sent false
18964 6168 Acked Read 9 Done Done Sent false
18965 6169 Acked Write 5 Done Done Sent false
18966 6170 Acked Write 10 Done Done Sent false
18967 6171 Acked Write 4 Done Done Sent false
18968 6172 Acked Write 4 Done Done Sent false
18969 6173 Acked Write 3 Done Done Sent false
18970 6174 Acked Write 4 Done Done Sent false
18971 6175 Acked Write 4 Done Done Sent false
18972 6176 Acked Read 3 Done Done Sent false
18973 6177 Acked Write 6 Done Done Sent false
18974 6178 Acked Read 5 Done Done Sent false
18975 6179 Acked Read 8 Done Done Sent false
18976 6180 Acked Write 9 Done Done Sent false
18977 6181 Acked Read 8 Done Done Sent false
18978 6182 Acked Write 4 Done Done Sent false
18979 6183 Acked Flush 0 Done Done Sent false
18980 6184 Acked Flush 0 Done Done Sent false
18981 6185 Acked Write 9 Done Done Sent false
18982 6186 Acked Read 2 Done Done Sent false
18983 6187 Acked Read 6 Done Done Sent false
18984 6188 Acked Read 3 Done Done Sent false
18985 6189 Acked Write 6 Done Done Sent false
18986 6190 Acked Flush 0 Done Done Sent false
18987 6191 Acked Write 1 Done Done Sent false
18988 6192 Acked Write 4 Done Done Sent false
18989 6193 Acked Flush 0 Done Done Sent false
18990 6194 Acked Read 4 Done Done Sent false
18991 6195 Acked Write 8 Done Done Sent false
18992 6196 Acked Write 1 Done Done Sent false
18993 6197 Acked Read 7 Done Done Sent false
18994 6198 Acked Read 6 Done Done Sent false
18995 6199 Acked Write 1 Done Done Sent false
18996 6200 Acked Write 5 Done Done Sent false
18997 6201 Acked Flush 0 Done Done Sent false
18998 6202 Acked Read 3 Done Done Sent false
18999 6203 Acked Write 10 Done Done Sent false
19000 6204 Acked Read 10 Done Done Sent false
19001 6205 Acked Write 3 Done Done Sent false
19002 6206 Acked Read 8 Done Done Sent false
19003 6207 Acked Read 9 Done Done Sent false
19004 6208 Acked Write 3 Done Done Sent false
19005 6209 Acked Write 1 Done Done Sent false
19006 6210 Acked Read 9 Done Done Sent false
19007 6211 Acked Write 7 Done Done Sent false
19008 6212 Acked Flush 0 Done Done Sent false
19009 6213 Acked Write 9 Done Done Sent false
19010 6214 Acked Read 5 Done Done Sent false
19011 6215 Acked Read 8 Done Done Sent false
19012 6216 Acked Write 7 Done Done Sent false
19013 6217 Acked Read 10 Done Done Sent false
19014 6218 Acked Read 10 Done Done Sent false
19015 6219 Acked Read 9 Done Done Sent false
19016 6220 Acked Read 2 Done Done Sent false
19017 6221 Acked Read 9 Done Done Sent false
19018 6222 Acked Write 1 Done Done Sent false
19019 6223 Acked Write 4 Done Done Sent false
19020 6224 Acked Read 8 Done Done Sent false
19021 6225 Acked Read 10 Done Done Sent false
19022 6226 Acked Read 1 Done Done Sent false
19023 6227 Acked Flush 0 Done Done Sent false
19024 6228 Acked Read 4 Done Done Sent false
19025 6229 Acked Write 8 Done Done Sent false
19026 6230 Acked Flush 0 Done Done Sent false
19027 6231 Acked Write 2 Done Done Sent false
19028 6232 Acked Read 8 Done Done Sent false
19029 6233 Acked Read 10 Done Done Sent false
19030 6234 Acked Flush 0 Done Done Sent false
19031 6235 Acked Read 2 Done Done Sent false
19032 6236 Acked Write 7 Done Done Sent false
19033 6237 Acked Read 5 Done Done Sent false
19034 6238 Acked Write 5 Done Done Sent false
19035 6239 Acked Read 7 Done Done Sent false
19036 6240 Acked Read 6 Done Done Sent false
19037 6241 Acked Write 7 Done Done Sent false
19038 6242 Acked Flush 0 Done Done Sent false
19039 6243 Acked Read 4 Done Done Sent false
19040 6244 Acked Read 7 Done Done Sent false
19041 6245 Acked Flush 0 Done Done Sent false
19042 6246 Acked Write 7 Done Done Sent false
19043 6247 Acked Write 5 Done Done Sent false
19044 6248 Acked Write 10 Done Done Sent false
19045 6249 Acked Write 7 Done Done Sent false
19046 6250 Acked Write 9 Done Done Sent false
19047 6251 Acked Write 3 Done Done Sent false
19048 6252 Acked Read 10 Done Done Sent false
19049 6253 Acked Read 9 Done Done Sent false
19050 6254 Acked Write 9 Done Done Sent false
19051 6255 Acked Read 4 Done Done Sent false
19052 6256 Acked Write 5 Done Done Sent false
19053 6257 Acked Read 7 Done Done Sent false
19054 6258 Acked Read 10 Done Done Sent false
19055 6259 Acked Read 8 Done Done Sent false
19056 6260 Acked Flush 0 Done Done Sent false
19057 6261 Acked Write 3 Done Done Sent false
19058 6262 Acked Read 4 Done Done Sent false
19059 6263 Acked Write 8 Done Done Sent false
19060 6264 Acked Write 3 Done Done Sent false
19061 6265 Acked Read 8 Done Done Sent false
19062 6266 Acked Flush 0 Done Done Sent false
19063 6267 Acked Read 5 Done Done Sent false
19064 6268 Acked Read 7 Done Done Sent false
19065 6269 Acked Read 8 Done Done Sent false
19066 6270 Acked Flush 0 Done Done Sent false
19067 6271 Acked Read 9 Done Done Sent false
19068 6272 Acked Read 1 Done Done Sent false
19069 6273 Acked Write 3 Done Done Sent false
19070 6274 Acked Read 2 Done Done Sent false
19071 6275 Acked Write 1 Done Done Sent false
19072 6276 Acked Write 2 Done Done Sent false
19073 6277 Acked Read 2 Done Done Sent false
19074 6278 Acked Flush 0 Done Done Sent false
19075 6279 Acked Flush 0 Done Done Sent false
19076 6280 Acked Read 2 Done Done Sent false
19077 6281 Acked Read 1 Done Done Sent false
19078 6282 Acked Write 3 Done Done Sent false
19079 6283 Acked Read 5 Done Done Sent false
19080 6284 Acked Write 4 Done Done Sent false
19081 6285 Acked Flush 0 Done Done Sent false
19082 6286 Acked Write 7 Done Done Sent false
19083 6287 Acked Write 4 Done Done Sent false
19084 6288 Acked Write 9 Done Done Sent false
19085 6289 Acked Write 6 Done Done Sent false
19086 6290 Acked Write 2 Done Done Sent false
19087 6291 Acked Read 10 Done Done Sent false
19088 6292 Acked Write 8 Done Done Sent false
19089 6293 Acked Read 7 Done Done Sent false
19090 6294 Acked Read 9 Done Done Sent false
19091 6295 Acked Write 2 Done Done Sent false
19092 6296 Acked Write 10 Done Done Sent false
19093 6297 Acked Read 1 Done Done Sent false
19094 6298 Acked Read 10 Done Done Sent false
19095 6299 Acked Read 2 Done Done Sent false
19096 6300 Acked Read 9 Done Done Sent false
19097 6301 Acked Read 9 Done Done Sent false
19098 6302 Acked Write 2 Done Done Sent false
19099 6303 Acked Write 5 Done Done Sent false
19100 6304 Acked Write 6 Done Done Sent false
19101 6305 Acked Flush 0 Done Done Sent false
19102 6306 Acked Write 3 Done Done Sent false
19103 6307 Acked Read 2 Done Done Sent false
19104 6308 Acked Read 5 Done Done Sent false
19105 6309 Acked Read 3 Done Done Sent false
19106 6310 Acked Write 4 Done Done Sent false
19107 6311 Acked Write 8 Done Done Sent false
19108 6312 Acked Write 10 Done Done Sent false
19109 6313 Acked Read 8 Done Done Sent false
19110 6314 Acked Read 3 Done Done Sent false
19111 6315 Acked Write 7 Done Done Sent false
19112 6316 Acked Read 8 Done Done Sent false
19113 6317 Acked Write 10 Done Done Sent false
19114 6318 Acked Flush 0 Done Done Sent false
19115 6319 Acked Read 7 Done Done Sent false
19116 6320 Acked Read 9 Done Done Sent false
19117 6321 Acked Write 1 Done Done Sent false
19118 6322 Acked Read 5 Done Done Sent false
19119 6323 Acked Write 8 Done Done Sent false
19120 6324 Acked Read 1 Done Done Sent false
19121 6325 Acked Read 6 Done Done Sent false
19122 6326 Acked Read 8 Done Done Sent false
19123 6327 Acked Write 8 Done Done Sent false
19124 6328 Acked Read 4 Done Done Sent false
19125 6329 Acked Read 6 Done Done Sent false
19126 6330 Acked Read 6 Done Done Sent false
19127 6331 Acked Read 4 Done Done Sent false
19128 6332 Acked Read 5 Done Done Sent false
19129 6333 Acked Write 10 Done Done Sent false
19130 6334 Acked Write 6 Done Done Sent false
19131 6335 Acked Flush 0 Done Done Sent false
19132 6336 Acked Read 9 Done Done Sent false
19133 6337 Acked Write 7 Done Done Sent false
19134 6338 Acked Write 7 Done Done Sent false
19135 6339 Acked Read 5 Done Done Sent false
19136 6340 Acked Read 1 Done Done Sent false
19137 6341 Acked Write 3 Done Done Sent false
19138 6342 Acked Read 4 Done Done Sent false
19139 6343 Acked Write 2 Done Done Sent false
19140 6344 Acked Write 4 Done Done Sent false
19141 6345 Acked Read 7 Done Done Sent false
19142 6346 Acked Write 6 Done Done Sent false
19143 6347 Acked Read 10 Done Done Sent false
19144 6348 Acked Flush 0 Done Done Sent false
19145 6349 Acked Flush 0 Done Done Sent false
19146 6350 Acked Read 2 Done Done Sent false
19147 6351 Acked Read 9 Done Done Sent false
19148 6352 Acked Read 1 Done Done Sent false
19149 6353 Acked Read 6 Done Done Sent false
19150 6354 Acked Write 9 Done Done Sent false
19151 6355 Acked Write 7 Done Done Sent false
19152 6356 Acked Read 10 Done Done Sent false
19153 6357 Acked Read 5 Done Done Sent false
19154 6358 Acked Read 9 Done Done Sent false
19155 6359 Acked Write 7 Done Done Sent false
19156 6360 Acked Read 4 Done Done Sent false
19157 6361 Acked Write 3 Done Done Sent false
19158 6362 Acked Write 4 Done Done Sent false
19159 6363 Acked Write 4 Done Done Sent false
19160 6364 Acked Write 8 Done Done Sent false
19161 6365 Acked Read 2 Done Done Sent false
19162 6366 Acked Read 6 Done Done Sent false
19163 6367 Acked Read 5 Done Done Sent false
19164 6368 Acked Read 9 Done Done Sent false
19165 6369 Acked Write 5 Done Done Sent false
19166 6370 Acked Write 10 Done Done Sent false
19167 6371 Acked Read 6 Done Done Sent false
19168 6372 Acked Read 7 Done Done Sent false
19169 6373 Acked Read 3 Done Done Sent false
19170 6374 Acked Write 4 Done Done Sent false
19171 6375 Acked Read 8 Done Done Sent false
19172 6376 Acked Write 2 Done Done Sent false
19173 6377 Acked Write 1 Done Done Sent false
19174 6378 Acked Flush 0 Done Done Sent false
19175 6379 Acked Write 9 Done Done Sent false
19176 6380 Acked Read 7 Done Done Sent false
19177 6381 Acked Write 8 Done Done Sent false
19178 6382 Acked Read 9 Done Done Sent false
19179 6383 Acked Write 2 Done Done Sent false
19180 6384 Acked Read 3 Done Done Sent false
19181 6385 Acked Read 5 Done Done Sent false
19182 6386 Acked Read 7 Done Done Sent false
19183 6387 Acked Read 6 Done Done Sent false
19184 6388 Acked Read 10 Done Done Sent false
19185 6389 Acked Read 7 Done Done Sent false
19186 6390 Acked Read 1 Done Done Sent false
19187 6391 Acked Write 10 Done Done Sent false
19188 6392 Acked Read 9 Done Done Sent false
19189 6393 Acked Read 6 Done Done Sent false
19190 6394 Acked Read 9 Done Done Sent false
19191 6395 Acked Write 6 Done Done Sent false
19192 6396 Acked Flush 0 Done Done Sent false
19193 6397 Acked Read 3 Done Done Sent false
19194 6398 Acked Read 5 Done Done Sent false
19195 6399 Acked Write 8 Done Done Sent false
19196 6400 Acked Read 8 Done Done Sent false
19197 6401 Acked Write 8 Done Done Sent false
19198 6402 Acked Read 2 Done Done Sent false
19199 6403 Acked Write 7 Done Done Sent false
19200 6404 Acked Write 9 Done Done Sent false
19201 6405 Acked Read 3 Done Done Sent false
19202 6406 Acked Read 3 Done Done Sent false
19203 6407 Acked Read 4 Done Done Sent false
19204 6408 Acked Read 2 Done Done Sent false
19205 6409 Acked Write 3 Done Done Sent false
19206 6410 Acked Write 6 Done Done Sent false
19207 6411 Acked Write 1 Done Done Sent false
19208 6412 Acked Write 7 Done Done Sent false
19209 6413 Acked Read 5 Done Done Sent false
19210 6414 Acked Read 4 Done Done Sent false
19211 6415 Acked Write 7 Done Done Sent false
19212 6416 Acked Write 7 Done Done Sent false
19213 6417 Acked Flush 0 Done Done Sent false
19214 6418 Acked Flush 0 Done Done Sent false
19215 6419 Acked Write 10 Done Done Sent false
19216 6420 Acked Write 6 Done Done Sent false
19217 6421 Acked Read 8 Done Done Sent false
19218 6422 Acked Read 1 Done Done Sent false
19219 6423 Acked Read 8 Done Done Sent false
19220 6424 Acked Write 3 Done Done Sent false
19221 6425 Acked Write 2 Done Done Sent false
19222 6426 Acked Write 9 Done Done Sent false
19223 6427 Acked Flush 0 Done Done Sent false
19224 6428 Acked Read 8 Done Done Sent false
19225 6429 Acked Read 3 Done Done Sent false
19226 6430 Acked Write 3 Done Done Sent false
19227 6431 Acked Read 6 Done Done Sent false
19228 STATES DS:0 DS:1 DS:2 TOTAL
19229 Sent 0 0 301 301
19230 Done 301 301 0 602
19231 Skipped 0 0 0 0
19232 Error 0 0 0 0
19233 Last Flush: 6427 6427 6130
19234 Downstairs last five completed: 6130 6129 6128 6127 6126
19235 Upstairs last five completed: 6431 6430 6429 6428 6427
19236 CLIENT: Up:0 ds:301 act:2
19237 ----------------------------------------------------------------
19238 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19239 States: Active Active Offline
19240 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19241 6131 Acked Write 4 Done Done Sent false
19242 6132 Acked Write 8 Done Done Sent false
19243 6133 Acked Write 5 Done Done Sent false
19244 6134 Acked Read 3 Done Done Sent false
19245 6135 Acked Write 3 Done Done Sent false
19246 6136 Acked Read 8 Done Done Sent false
19247 6137 Acked Write 10 Done Done Sent false
19248 6138 Acked Write 6 Done Done Sent false
19249 6139 Acked Write 1 Done Done Sent false
19250 6140 Acked Write 10 Done Done Sent false
19251 6141 Acked Flush 0 Done Done Sent false
19252 6142 Acked Read 8 Done Done Sent false
19253 6143 Acked Write 3 Done Done Sent false
19254 6144 Acked Write 10 Done Done Sent false
19255 6145 Acked Write 2 Done Done Sent false
19256 6146 Acked Write 6 Done Done Sent false
19257 6147 Acked Write 6 Done Done Sent false
19258 6148 Acked Write 7 Done Done Sent false
19259 6149 Acked Read 1 Done Done Sent false
19260 6150 Acked Flush 0 Done Done Sent false
19261 6151 Acked Write 6 Done Done Sent false
19262 6152 Acked Write 2 Done Done Sent false
19263 6153 Acked Read 7 Done Done Sent false
19264 6154 Acked Read 1 Done Done Sent false
19265 6155 Acked Write 5 Done Done Sent false
19266 6156 Acked Write 10 Done Done Sent false
19267 6157 Acked Write 10 Done Done Sent false
19268 6158 Acked Read 10 Done Done Sent false
19269 6159 Acked Flush 0 Done Done Sent false
19270 6160 Acked Read 3 Done Done Sent false
19271 6161 Acked Flush 0 Done Done Sent false
19272 6162 Acked Read 10 Done Done Sent false
19273 6163 Acked Write 6 Done Done Sent false
19274 6164 Acked Read 7 Done Done Sent false
19275 6165 Acked Read 1 Done Done Sent false
19276 6166 Acked Read 4 Done Done Sent false
19277 6167 Acked Flush 0 Done Done Sent false
19278 6168 Acked Read 9 Done Done Sent false
19279 6169 Acked Write 5 Done Done Sent false
19280 6170 Acked Write 10 Done Done Sent false
19281 6171 Acked Write 4 Done Done Sent false
19282 6172 Acked Write 4 Done Done Sent false
19283 6173 Acked Write 3 Done Done Sent false
19284 6174 Acked Write 4 Done Done Sent false
19285 6175 Acked Write 4 Done Done Sent false
19286 6176 Acked Read 3 Done Done Sent false
19287 6177 Acked Write 6 Done Done Sent false
19288 6178 Acked Read 5 Done Done Sent false
19289 6179 Acked Read 8 Done Done Sent false
19290 6180 Acked Write 9 Done Done Sent false
19291 6181 Acked Read 8 Done Done Sent false
19292 6182 Acked Write 4 Done Done Sent false
19293 6183 Acked Flush 0 Done Done Sent false
19294 6184 Acked Flush 0 Done Done Sent false
19295 6185 Acked Write 9 Done Done Sent false
19296 6186 Acked Read 2 Done Done Sent false
19297 6187 Acked Read 6 Done Done Sent false
19298 6188 Acked Read 3 Done Done Sent false
19299 6189 Acked Write 6 Done Done Sent false
19300 6190 Acked Flush 0 Done Done Sent false
19301 6191 Acked Write 1 Done Done Sent false
19302 6192 Acked Write 4 Done Done Sent false
19303 6193 Acked Flush 0 Done Done Sent false
19304 6194 Acked Read 4 Done Done Sent false
19305 6195 Acked Write 8 Done Done Sent false
19306 6196 Acked Write 1 Done Done Sent false
19307 6197 Acked Read 7 Done Done Sent false
19308 6198 Acked Read 6 Done Done Sent false
19309 6199 Acked Write 1 Done Done Sent false
19310 6200 Acked Write 5 Done Done Sent false
19311 6201 Acked Flush 0 Done Done Sent false
19312 6202 Acked Read 3 Done Done Sent false
19313 6203 Acked Write 10 Done Done Sent false
19314 6204 Acked Read 10 Done Done Sent false
19315 6205 Acked Write 3 Done Done Sent false
19316 6206 Acked Read 8 Done Done Sent false
19317 6207 Acked Read 9 Done Done Sent false
19318 6208 Acked Write 3 Done Done Sent false
19319 6209 Acked Write 1 Done Done Sent false
19320 6210 Acked Read 9 Done Done Sent false
19321 6211 Acked Write 7 Done Done Sent false
19322 6212 Acked Flush 0 Done Done Sent false
19323 6213 Acked Write 9 Done Done Sent false
19324 6214 Acked Read 5 Done Done Sent false
19325 6215 Acked Read 8 Done Done Sent false
19326 6216 Acked Write 7 Done Done Sent false
19327 6217 Acked Read 10 Done Done Sent false
19328 6218 Acked Read 10 Done Done Sent false
19329 6219 Acked Read 9 Done Done Sent false
19330 6220 Acked Read 2 Done Done Sent false
19331 6221 Acked Read 9 Done Done Sent false
19332 6222 Acked Write 1 Done Done Sent false
19333 6223 Acked Write 4 Done Done Sent false
19334 6224 Acked Read 8 Done Done Sent false
19335 6225 Acked Read 10 Done Done Sent false
19336 6226 Acked Read 1 Done Done Sent false
19337 6227 Acked Flush 0 Done Done Sent false
19338 6228 Acked Read 4 Done Done Sent false
19339 6229 Acked Write 8 Done Done Sent false
19340 6230 Acked Flush 0 Done Done Sent false
19341 6231 Acked Write 2 Done Done Sent false
19342 6232 Acked Read 8 Done Done Sent false
19343 6233 Acked Read 10 Done Done Sent false
19344 6234 Acked Flush 0 Done Done Sent false
19345 6235 Acked Read 2 Done Done Sent false
19346 6236 Acked Write 7 Done Done Sent false
19347 6237 Acked Read 5 Done Done Sent false
19348 6238 Acked Write 5 Done Done Sent false
19349 6239 Acked Read 7 Done Done Sent false
19350 6240 Acked Read 6 Done Done Sent false
19351 6241 Acked Write 7 Done Done Sent false
19352 6242 Acked Flush 0 Done Done Sent false
19353 6243 Acked Read 4 Done Done Sent false
19354 6244 Acked Read 7 Done Done Sent false
19355 6245 Acked Flush 0 Done Done Sent false
19356 6246 Acked Write 7 Done Done Sent false
19357 6247 Acked Write 5 Done Done Sent false
19358 6248 Acked Write 10 Done Done Sent false
19359 6249 Acked Write 7 Done Done Sent false
19360 6250 Acked Write 9 Done Done Sent false
19361 6251 Acked Write 3 Done Done Sent false
19362 6252 Acked Read 10 Done Done Sent false
19363 6253 Acked Read 9 Done Done Sent false
19364 6254 Acked Write 9 Done Done Sent false
19365 6255 Acked Read 4 Done Done Sent false
19366 6256 Acked Write 5 Done Done Sent false
19367 6257 Acked Read 7 Done Done Sent false
19368 6258 Acked Read 10 Done Done Sent false
19369 6259 Acked Read 8 Done Done Sent false
19370 6260 Acked Flush 0 Done Done Sent false
19371 6261 Acked Write 3 Done Done Sent false
19372 6262 Acked Read 4 Done Done Sent false
19373 6263 Acked Write 8 Done Done Sent false
19374 6264 Acked Write 3 Done Done Sent false
19375 6265 Acked Read 8 Done Done Sent false
19376 6266 Acked Flush 0 Done Done Sent false
19377 6267 Acked Read 5 Done Done Sent false
19378 6268 Acked Read 7 Done Done Sent false
19379 6269 Acked Read 8 Done Done Sent false
19380 6270 Acked Flush 0 Done Done Sent false
19381 6271 Acked Read 9 Done Done Sent false
19382 6272 Acked Read 1 Done Done Sent false
19383 6273 Acked Write 3 Done Done Sent false
19384 6274 Acked Read 2 Done Done Sent false
19385 6275 Acked Write 1 Done Done Sent false
19386 6276 Acked Write 2 Done Done Sent false
19387 6277 Acked Read 2 Done Done Sent false
19388 6278 Acked Flush 0 Done Done Sent false
19389 6279 Acked Flush 0 Done Done Sent false
19390 6280 Acked Read 2 Done Done Sent false
19391 6281 Acked Read 1 Done Done Sent false
19392 6282 Acked Write 3 Done Done Sent false
19393 6283 Acked Read 5 Done Done Sent false
19394 6284 Acked Write 4 Done Done Sent false
19395 6285 Acked Flush 0 Done Done Sent false
19396 6286 Acked Write 7 Done Done Sent false
19397 6287 Acked Write 4 Done Done Sent false
19398 6288 Acked Write 9 Done Done Sent false
19399 6289 Acked Write 6 Done Done Sent false
19400 6290 Acked Write 2 Done Done Sent false
19401 6291 Acked Read 10 Done Done Sent false
19402 6292 Acked Write 8 Done Done Sent false
19403 6293 Acked Read 7 Done Done Sent false
19404 6294 Acked Read 9 Done Done Sent false
19405 6295 Acked Write 2 Done Done Sent false
19406 6296 Acked Write 10 Done Done Sent false
19407 6297 Acked Read 1 Done Done Sent false
19408 6298 Acked Read 10 Done Done Sent false
19409 6299 Acked Read 2 Done Done Sent false
19410 6300 Acked Read 9 Done Done Sent false
19411 6301 Acked Read 9 Done Done Sent false
19412 6302 Acked Write 2 Done Done Sent false
19413 6303 Acked Write 5 Done Done Sent false
19414 6304 Acked Write 6 Done Done Sent false
19415 6305 Acked Flush 0 Done Done Sent false
19416 6306 Acked Write 3 Done Done Sent false
19417 6307 Acked Read 2 Done Done Sent false
19418 6308 Acked Read 5 Done Done Sent false
19419 6309 Acked Read 3 Done Done Sent false
19420 6310 Acked Write 4 Done Done Sent false
19421 6311 Acked Write 8 Done Done Sent false
19422 6312 Acked Write 10 Done Done Sent false
19423 6313 Acked Read 8 Done Done Sent false
19424 6314 Acked Read 3 Done Done Sent false
19425 6315 Acked Write 7 Done Done Sent false
19426 6316 Acked Read 8 Done Done Sent false
19427 6317 Acked Write 10 Done Done Sent false
19428 6318 Acked Flush 0 Done Done Sent false
19429 6319 Acked Read 7 Done Done Sent false
19430 6320 Acked Read 9 Done Done Sent false
19431 6321 Acked Write 1 Done Done Sent false
19432 6322 Acked Read 5 Done Done Sent false
19433 6323 Acked Write 8 Done Done Sent false
19434 6324 Acked Read 1 Done Done Sent false
19435 6325 Acked Read 6 Done Done Sent false
19436 6326 Acked Read 8 Done Done Sent false
19437 6327 Acked Write 8 Done Done Sent false
19438 6328 Acked Read 4 Done Done Sent false
19439 6329 Acked Read 6 Done Done Sent false
19440 6330 Acked Read 6 Done Done Sent false
19441 6331 Acked Read 4 Done Done Sent false
19442 6332 Acked Read 5 Done Done Sent false
19443 6333 Acked Write 10 Done Done Sent false
19444 6334 Acked Write 6 Done Done Sent false
19445 6335 Acked Flush 0 Done Done Sent false
19446 6336 Acked Read 9 Done Done Sent false
19447 6337 Acked Write 7 Done Done Sent false
19448 6338 Acked Write 7 Done Done Sent false
19449 6339 Acked Read 5 Done Done Sent false
19450 6340 Acked Read 1 Done Done Sent false
19451 6341 Acked Write 3 Done Done Sent false
19452 6342 Acked Read 4 Done Done Sent false
19453 6343 Acked Write 2 Done Done Sent false
19454 6344 Acked Write 4 Done Done Sent false
19455 6345 Acked Read 7 Done Done Sent false
19456 6346 Acked Write 6 Done Done Sent false
19457 6347 Acked Read 10 Done Done Sent false
19458 6348 Acked Flush 0 Done Done Sent false
19459 6349 Acked Flush 0 Done Done Sent false
19460 6350 Acked Read 2 Done Done Sent false
19461 6351 Acked Read 9 Done Done Sent false
19462 6352 Acked Read 1 Done Done Sent false
19463 6353 Acked Read 6 Done Done Sent false
19464 6354 Acked Write 9 Done Done Sent false
19465 6355 Acked Write 7 Done Done Sent false
19466 6356 Acked Read 10 Done Done Sent false
19467 6357 Acked Read 5 Done Done Sent false
19468 6358 Acked Read 9 Done Done Sent false
19469 6359 Acked Write 7 Done Done Sent false
19470 6360 Acked Read 4 Done Done Sent false
19471 6361 Acked Write 3 Done Done Sent false
19472 6362 Acked Write 4 Done Done Sent false
19473 6363 Acked Write 4 Done Done Sent false
19474 6364 Acked Write 8 Done Done Sent false
19475 6365 Acked Read 2 Done Done Sent false
19476 6366 Acked Read 6 Done Done Sent false
19477 6367 Acked Read 5 Done Done Sent false
19478 6368 Acked Read 9 Done Done Sent false
19479 6369 Acked Write 5 Done Done Sent false
19480 6370 Acked Write 10 Done Done Sent false
19481 6371 Acked Read 6 Done Done Sent false
19482 6372 Acked Read 7 Done Done Sent false
19483 6373 Acked Read 3 Done Done Sent false
19484 6374 Acked Write 4 Done Done Sent false
19485 6375 Acked Read 8 Done Done Sent false
19486 6376 Acked Write 2 Done Done Sent false
19487 6377 Acked Write 1 Done Done Sent false
19488 6378 Acked Flush 0 Done Done Sent false
19489 6379 Acked Write 9 Done Done Sent false
19490 6380 Acked Read 7 Done Done Sent false
19491 6381 Acked Write 8 Done Done Sent false
19492 6382 Acked Read 9 Done Done Sent false
19493 6383 Acked Write 2 Done Done Sent false
19494 6384 Acked Read 3 Done Done Sent false
19495 6385 Acked Read 5 Done Done Sent false
19496 6386 Acked Read 7 Done Done Sent false
19497 6387 Acked Read 6 Done Done Sent false
19498 6388 Acked Read 10 Done Done Sent false
19499 6389 Acked Read 7 Done Done Sent false
19500 6390 Acked Read 1 Done Done Sent false
19501 6391 Acked Write 10 Done Done Sent false
19502 6392 Acked Read 9 Done Done Sent false
19503 6393 Acked Read 6 Done Done Sent false
19504 6394 Acked Read 9 Done Done Sent false
19505 6395 Acked Write 6 Done Done Sent false
19506 6396 Acked Flush 0 Done Done Sent false
19507 6397 Acked Read 3 Done Done Sent false
19508 6398 Acked Read 5 Done Done Sent false
19509 6399 Acked Write 8 Done Done Sent false
19510 6400 Acked Read 8 Done Done Sent false
19511 6401 Acked Write 8 Done Done Sent false
19512 6402 Acked Read 2 Done Done Sent false
19513 6403 Acked Write 7 Done Done Sent false
19514 6404 Acked Write 9 Done Done Sent false
19515 6405 Acked Read 3 Done Done Sent false
19516 6406 Acked Read 3 Done Done Sent false
19517 6407 Acked Read 4 Done Done Sent false
19518 6408 Acked Read 2 Done Done Sent false
19519 6409 Acked Write 3 Done Done Sent false
19520 6410 Acked Write 6 Done Done Sent false
19521 6411 Acked Write 1 Done Done Sent false
19522 6412 Acked Write 7 Done Done Sent false
19523 6413 Acked Read 5 Done Done Sent false
19524 6414 Acked Read 4 Done Done Sent false
19525 6415 Acked Write 7 Done Done Sent false
19526 6416 Acked Write 7 Done Done Sent false
19527 6417 Acked Flush 0 Done Done Sent false
19528 6418 Acked Flush 0 Done Done Sent false
19529 6419 Acked Write 10 Done Done Sent false
19530 6420 Acked Write 6 Done Done Sent false
19531 6421 Acked Read 8 Done Done Sent false
19532 6422 Acked Read 1 Done Done Sent false
19533 6423 Acked Read 8 Done Done Sent false
19534 6424 Acked Write 3 Done Done Sent false
19535 6425 Acked Write 2 Done Done Sent false
19536 6426 Acked Write 9 Done Done Sent false
19537 6427 Acked Flush 0 Done Done Sent false
19538 6428 Acked Read 8 Done Done Sent false
19539 6429 Acked Read 3 Done Done Sent false
19540 6430 Acked Write 3 Done Done Sent false
19541 6431 Acked Read 6 Done Done Sent false
19542 6432 Acked Flush 0 Done Done Sent false
19543 STATES DS:0 DS:1 DS:2 TOTAL
19544 Sent 0 0 302 302
19545 Done 302 302 0 604
19546 Skipped 0 0 0 0
19547 Error 0 0 0 0
19548 Last Flush: 6432 6432 6130
19549 Downstairs last five completed: 6130 6129 6128 6127 6126
19550 Upstairs last five completed: 6432 6431 6430 6429 6428
19551 CLIENT: Up:0 ds:302 act:2
195522025-01-27T19:40:27.126ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
195532025-01-27T19:40:27.126ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
195542025-01-27T19:40:27.126ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
195552025-01-27T19:40:27.126ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
195562025-01-27T19:40:27.126ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
19557 ----------------------------------------------------------------
19558 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19559 States: Active Active Offline
19560 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19561 6131 Acked Write 4 Done Done Sent false
19562 6132 Acked Write 8 Done Done Sent false
19563 6133 Acked Write 5 Done Done Sent false
19564 6134 Acked Read 3 Done Done Sent false
19565 6135 Acked Write 3 Done Done Sent false
19566 6136 Acked Read 8 Done Done Sent false
19567 6137 Acked Write 10 Done Done Sent false
19568 6138 Acked Write 6 Done Done Sent false
19569 6139 Acked Write 1 Done Done Sent false
19570 6140 Acked Write 10 Done Done Sent false
19571 6141 Acked Flush 0 Done Done Sent false
19572 6142 Acked Read 8 Done Done Sent false
19573 6143 Acked Write 3 Done Done Sent false
19574 6144 Acked Write 10 Done Done Sent false
19575 6145 Acked Write 2 Done Done Sent false
19576 6146 Acked Write 6 Done Done Sent false
19577 6147 Acked Write 6 Done Done Sent false
19578 6148 Acked Write 7 Done Done Sent false
19579 6149 Acked Read 1 Done Done Sent false
19580 6150 Acked Flush 0 Done Done Sent false
19581 6151 Acked Write 6 Done Done Sent false
19582 6152 Acked Write 2 Done Done Sent false
19583 6153 Acked Read 7 Done Done Sent false
19584 6154 Acked Read 1 Done Done Sent false
19585 6155 Acked Write 5 Done Done Sent false
19586 6156 Acked Write 10 Done Done Sent false
19587 6157 Acked Write 10 Done Done Sent false
19588 6158 Acked Read 10 Done Done Sent false
19589 6159 Acked Flush 0 Done Done Sent false
19590 6160 Acked Read 3 Done Done Sent false
19591 6161 Acked Flush 0 Done Done Sent false
19592 6162 Acked Read 10 Done Done Sent false
19593 6163 Acked Write 6 Done Done Sent false
19594 6164 Acked Read 7 Done Done Sent false
19595 6165 Acked Read 1 Done Done Sent false
19596 6166 Acked Read 4 Done Done Sent false
19597 6167 Acked Flush 0 Done Done Sent false
19598 6168 Acked Read 9 Done Done Sent false
19599 6169 Acked Write 5 Done Done Sent false
19600 6170 Acked Write 10 Done Done Sent false
19601 6171 Acked Write 4 Done Done Sent false
19602 6172 Acked Write 4 Done Done Sent false
19603 6173 Acked Write 3 Done Done Sent false
19604 6174 Acked Write 4 Done Done Sent false
19605 6175 Acked Write 4 Done Done Sent false
19606 6176 Acked Read 3 Done Done Sent false
19607 6177 Acked Write 6 Done Done Sent false
19608 6178 Acked Read 5 Done Done Sent false
19609 6179 Acked Read 8 Done Done Sent false
19610 6180 Acked Write 9 Done Done Sent false
19611 6181 Acked Read 8 Done Done Sent false
19612 6182 Acked Write 4 Done Done Sent false
19613 6183 Acked Flush 0 Done Done Sent false
19614 6184 Acked Flush 0 Done Done Sent false
19615 6185 Acked Write 9 Done Done Sent false
19616 6186 Acked Read 2 Done Done Sent false
19617 6187 Acked Read 6 Done Done Sent false
19618 6188 Acked Read 3 Done Done Sent false
19619 6189 Acked Write 6 Done Done Sent false
19620 6190 Acked Flush 0 Done Done Sent false
19621 6191 Acked Write 1 Done Done Sent false
19622 6192 Acked Write 4 Done Done Sent false
19623 6193 Acked Flush 0 Done Done Sent false
19624 6194 Acked Read 4 Done Done Sent false
19625 6195 Acked Write 8 Done Done Sent false
19626 6196 Acked Write 1 Done Done Sent false
19627 6197 Acked Read 7 Done Done Sent false
19628 6198 Acked Read 6 Done Done Sent false
19629 6199 Acked Write 1 Done Done Sent false
19630 6200 Acked Write 5 Done Done Sent false
19631 6201 Acked Flush 0 Done Done Sent false
19632 6202 Acked Read 3 Done Done Sent false
19633 6203 Acked Write 10 Done Done Sent false
19634 6204 Acked Read 10 Done Done Sent false
19635 6205 Acked Write 3 Done Done Sent false
19636 6206 Acked Read 8 Done Done Sent false
19637 6207 Acked Read 9 Done Done Sent false
19638 6208 Acked Write 3 Done Done Sent false
19639 6209 Acked Write 1 Done Done Sent false
19640 6210 Acked Read 9 Done Done Sent false
19641 6211 Acked Write 7 Done Done Sent false
19642 6212 Acked Flush 0 Done Done Sent false
19643 6213 Acked Write 9 Done Done Sent false
19644 6214 Acked Read 5 Done Done Sent false
19645 6215 Acked Read 8 Done Done Sent false
19646 6216 Acked Write 7 Done Done Sent false
19647 6217 Acked Read 10 Done Done Sent false
19648 6218 Acked Read 10 Done Done Sent false
19649 6219 Acked Read 9 Done Done Sent false
19650 6220 Acked Read 2 Done Done Sent false
19651 6221 Acked Read 9 Done Done Sent false
19652 6222 Acked Write 1 Done Done Sent false
19653 6223 Acked Write 4 Done Done Sent false
19654 6224 Acked Read 8 Done Done Sent false
19655 6225 Acked Read 10 Done Done Sent false
19656 6226 Acked Read 1 Done Done Sent false
19657 6227 Acked Flush 0 Done Done Sent false
19658 6228 Acked Read 4 Done Done Sent false
19659 6229 Acked Write 8 Done Done Sent false
19660 6230 Acked Flush 0 Done Done Sent false
19661 6231 Acked Write 2 Done Done Sent false
19662 6232 Acked Read 8 Done Done Sent false
19663 6233 Acked Read 10 Done Done Sent false
19664 6234 Acked Flush 0 Done Done Sent false
19665 6235 Acked Read 2 Done Done Sent false
19666 6236 Acked Write 7 Done Done Sent false
19667 6237 Acked Read 5 Done Done Sent false
19668 6238 Acked Write 5 Done Done Sent false
19669 6239 Acked Read 7 Done Done Sent false
19670 6240 Acked Read 6 Done Done Sent false
19671 6241 Acked Write 7 Done Done Sent false
19672 6242 Acked Flush 0 Done Done Sent false
19673 6243 Acked Read 4 Done Done Sent false
19674 6244 Acked Read 7 Done Done Sent false
19675 6245 Acked Flush 0 Done Done Sent false
19676 6246 Acked Write 7 Done Done Sent false
19677 6247 Acked Write 5 Done Done Sent false
19678 6248 Acked Write 10 Done Done Sent false
19679 6249 Acked Write 7 Done Done Sent false
19680 6250 Acked Write 9 Done Done Sent false
19681 6251 Acked Write 3 Done Done Sent false
19682 6252 Acked Read 10 Done Done Sent false
19683 6253 Acked Read 9 Done Done Sent false
19684 6254 Acked Write 9 Done Done Sent false
19685 6255 Acked Read 4 Done Done Sent false
19686 6256 Acked Write 5 Done Done Sent false
19687 6257 Acked Read 7 Done Done Sent false
19688 6258 Acked Read 10 Done Done Sent false
19689 6259 Acked Read 8 Done Done Sent false
19690 6260 Acked Flush 0 Done Done Sent false
19691 6261 Acked Write 3 Done Done Sent false
19692 6262 Acked Read 4 Done Done Sent false
19693 6263 Acked Write 8 Done Done Sent false
19694 6264 Acked Write 3 Done Done Sent false
19695 6265 Acked Read 8 Done Done Sent false
19696 6266 Acked Flush 0 Done Done Sent false
19697 6267 Acked Read 5 Done Done Sent false
19698 6268 Acked Read 7 Done Done Sent false
19699 6269 Acked Read 8 Done Done Sent false
19700 6270 Acked Flush 0 Done Done Sent false
19701 6271 Acked Read 9 Done Done Sent false
19702 6272 Acked Read 1 Done Done Sent false
19703 6273 Acked Write 3 Done Done Sent false
19704 6274 Acked Read 2 Done Done Sent false
19705 6275 Acked Write 1 Done Done Sent false
19706 6276 Acked Write 2 Done Done Sent false
19707 6277 Acked Read 2 Done Done Sent false
19708 6278 Acked Flush 0 Done Done Sent false
19709 6279 Acked Flush 0 Done Done Sent false
19710 6280 Acked Read 2 Done Done Sent false
19711 6281 Acked Read 1 Done Done Sent false
19712 6282 Acked Write 3 Done Done Sent false
19713 6283 Acked Read 5 Done Done Sent false
19714 6284 Acked Write 4 Done Done Sent false
19715 6285 Acked Flush 0 Done Done Sent false
19716 6286 Acked Write 7 Done Done Sent false
19717 6287 Acked Write 4 Done Done Sent false
19718 6288 Acked Write 9 Done Done Sent false
19719 6289 Acked Write 6 Done Done Sent false
19720 6290 Acked Write 2 Done Done Sent false
19721 6291 Acked Read 10 Done Done Sent false
19722 6292 Acked Write 8 Done Done Sent false
19723 6293 Acked Read 7 Done Done Sent false
19724 6294 Acked Read 9 Done Done Sent false
19725 6295 Acked Write 2 Done Done Sent false
19726 6296 Acked Write 10 Done Done Sent false
19727 6297 Acked Read 1 Done Done Sent false
19728 6298 Acked Read 10 Done Done Sent false
19729 6299 Acked Read 2 Done Done Sent false
19730 6300 Acked Read 9 Done Done Sent false
19731 6301 Acked Read 9 Done Done Sent false
19732 6302 Acked Write 2 Done Done Sent false
19733 6303 Acked Write 5 Done Done Sent false
19734 6304 Acked Write 6 Done Done Sent false
19735 6305 Acked Flush 0 Done Done Sent false
19736 6306 Acked Write 3 Done Done Sent false
19737 6307 Acked Read 2 Done Done Sent false
19738 6308 Acked Read 5 Done Done Sent false
19739 6309 Acked Read 3 Done Done Sent false
19740 6310 Acked Write 4 Done Done Sent false
19741 6311 Acked Write 8 Done Done Sent false
19742 6312 Acked Write 10 Done Done Sent false
19743 6313 Acked Read 8 Done Done Sent false
19744 6314 Acked Read 3 Done Done Sent false
19745 6315 Acked Write 7 Done Done Sent false
19746 6316 Acked Read 8 Done Done Sent false
19747 6317 Acked Write 10 Done Done Sent false
19748 6318 Acked Flush 0 Done Done Sent false
19749 6319 Acked Read 7 Done Done Sent false
19750 6320 Acked Read 9 Done Done Sent false
19751 6321 Acked Write 1 Done Done Sent false
19752 6322 Acked Read 5 Done Done Sent false
19753 6323 Acked Write 8 Done Done Sent false
19754 6324 Acked Read 1 Done Done Sent false
19755 6325 Acked Read 6 Done Done Sent false
19756 6326 Acked Read 8 Done Done Sent false
19757 6327 Acked Write 8 Done Done Sent false
19758 6328 Acked Read 4 Done Done Sent false
19759 6329 Acked Read 6 Done Done Sent false
19760 6330 Acked Read 6 Done Done Sent false
19761 6331 Acked Read 4 Done Done Sent false
19762 6332 Acked Read 5 Done Done Sent false
19763 6333 Acked Write 10 Done Done Sent false
19764 6334 Acked Write 6 Done Done Sent false
19765 6335 Acked Flush 0 Done Done Sent false
19766 6336 Acked Read 9 Done Done Sent false
19767 6337 Acked Write 7 Done Done Sent false
19768 6338 Acked Write 7 Done Done Sent false
19769 6339 Acked Read 5 Done Done Sent false
19770 6340 Acked Read 1 Done Done Sent false
19771 6341 Acked Write 3 Done Done Sent false
19772 6342 Acked Read 4 Done Done Sent false
19773 6343 Acked Write 2 Done Done Sent false
19774 6344 Acked Write 4 Done Done Sent false
19775 6345 Acked Read 7 Done Done Sent false
19776 6346 Acked Write 6 Done Done Sent false
19777 6347 Acked Read 10 Done Done Sent false
19778 6348 Acked Flush 0 Done Done Sent false
19779 6349 Acked Flush 0 Done Done Sent false
19780 6350 Acked Read 2 Done Done Sent false
19781 6351 Acked Read 9 Done Done Sent false
19782 6352 Acked Read 1 Done Done Sent false
19783 6353 Acked Read 6 Done Done Sent false
19784 6354 Acked Write 9 Done Done Sent false
19785 6355 Acked Write 7 Done Done Sent false
19786 6356 Acked Read 10 Done Done Sent false
19787 6357 Acked Read 5 Done Done Sent false
19788 6358 Acked Read 9 Done Done Sent false
19789 6359 Acked Write 7 Done Done Sent false
19790 6360 Acked Read 4 Done Done Sent false
19791 6361 Acked Write 3 Done Done Sent false
19792 6362 Acked Write 4 Done Done Sent false
19793 6363 Acked Write 4 Done Done Sent false
19794 6364 Acked Write 8 Done Done Sent false
19795 6365 Acked Read 2 Done Done Sent false
19796 6366 Acked Read 6 Done Done Sent false
19797 6367 Acked Read 5 Done Done Sent false
19798 6368 Acked Read 9 Done Done Sent false
19799 6369 Acked Write 5 Done Done Sent false
19800 6370 Acked Write 10 Done Done Sent false
19801 6371 Acked Read 6 Done Done Sent false
19802 6372 Acked Read 7 Done Done Sent false
19803 6373 Acked Read 3 Done Done Sent false
19804 6374 Acked Write 4 Done Done Sent false
19805 6375 Acked Read 8 Done Done Sent false
19806 6376 Acked Write 2 Done Done Sent false
19807 6377 Acked Write 1 Done Done Sent false
19808 6378 Acked Flush 0 Done Done Sent false
19809 6379 Acked Write 9 Done Done Sent false
19810 6380 Acked Read 7 Done Done Sent false
19811 6381 Acked Write 8 Done Done Sent false
19812 6382 Acked Read 9 Done Done Sent false
19813 6383 Acked Write 2 Done Done Sent false
19814 6384 Acked Read 3 Done Done Sent false
19815 6385 Acked Read 5 Done Done Sent false
19816 6386 Acked Read 7 Done Done Sent false
19817 6387 Acked Read 6 Done Done Sent false
19818 6388 Acked Read 10 Done Done Sent false
19819 6389 Acked Read 7 Done Done Sent false
19820 6390 Acked Read 1 Done Done Sent false
19821 6391 Acked Write 10 Done Done Sent false
19822 6392 Acked Read 9 Done Done Sent false
19823 6393 Acked Read 6 Done Done Sent false
19824 6394 Acked Read 9 Done Done Sent false
19825 6395 Acked Write 6 Done Done Sent false
19826 6396 Acked Flush 0 Done Done Sent false
19827 6397 Acked Read 3 Done Done Sent false
19828 6398 Acked Read 5 Done Done Sent false
19829 6399 Acked Write 8 Done Done Sent false
19830 6400 Acked Read 8 Done Done Sent false
19831 6401 Acked Write 8 Done Done Sent false
19832 6402 Acked Read 2 Done Done Sent false
19833 6403 Acked Write 7 Done Done Sent false
19834 6404 Acked Write 9 Done Done Sent false
19835 6405 Acked Read 3 Done Done Sent false
19836 6406 Acked Read 3 Done Done Sent false
19837 6407 Acked Read 4 Done Done Sent false
19838 6408 Acked Read 2 Done Done Sent false
19839 6409 Acked Write 3 Done Done Sent false
19840 6410 Acked Write 6 Done Done Sent false
19841 6411 Acked Write 1 Done Done Sent false
19842 6412 Acked Write 7 Done Done Sent false
19843 6413 Acked Read 5 Done Done Sent false
19844 6414 Acked Read 4 Done Done Sent false
19845 6415 Acked Write 7 Done Done Sent false
19846 6416 Acked Write 7 Done Done Sent false
19847 6417 Acked Flush 0 Done Done Sent false
19848 6418 Acked Flush 0 Done Done Sent false
19849 6419 Acked Write 10 Done Done Sent false
19850 6420 Acked Write 6 Done Done Sent false
19851 6421 Acked Read 8 Done Done Sent false
19852 6422 Acked Read 1 Done Done Sent false
19853 6423 Acked Read 8 Done Done Sent false
19854 6424 Acked Write 3 Done Done Sent false
19855 6425 Acked Write 2 Done Done Sent false
19856 6426 Acked Write 9 Done Done Sent false
19857 6427 Acked Flush 0 Done Done Sent false
19858 6428 Acked Read 8 Done Done Sent false
19859 6429 Acked Read 3 Done Done Sent false
19860 6430 Acked Write 3 Done Done Sent false
19861 6431 Acked Read 6 Done Done Sent false
19862 6432 Acked Flush 0 Done Done Sent false
19863 STATES DS:0 DS:1 DS:2 TOTAL
19864 Sent 0 0 302 302
19865 Done 302 302 0 604
19866 Skipped 0 0 0 0
19867 Error 0 0 0 0
19868 Last Flush: 6432 6432 6130
19869 Downstairs last five completed: 6130 6129 6128 6127 6126
19870 Upstairs last five completed: 6432 6431 6430 6429 6428
19871 CLIENT: Up:0 ds:302 act:2
19872 [8830] got start action
19873 [8830] I am going to restart
19874 Starting downstairs at port 8830
19875 Make output file at "/tmp/dsc/downstairs-8830.txt"
19876 [8830][2] reports Starting
19877 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1218)
19878 [8830][2] reports Running
19879 ----------------------------------------------------------------
19880 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
19881 States: Active Active Offline
19882 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
19883 6131 Acked Write 4 Done Done Sent false
19884 6132 Acked Write 8 Done Done Sent false
19885 6133 Acked Write 5 Done Done Sent false
19886 6134 Acked Read 3 Done Done Sent false
19887 6135 Acked Write 3 Done Done Sent false
19888 6136 Acked Read 8 Done Done Sent false
19889 6137 Acked Write 10 Done Done Sent false
19890 6138 Acked Write 6 Done Done Sent false
19891 6139 Acked Write 1 Done Done Sent false
19892 6140 Acked Write 10 Done Done Sent false
19893 6141 Acked Flush 0 Done Done Sent false
19894 6142 Acked Read 8 Done Done Sent false
19895 6143 Acked Write 3 Done Done Sent false
19896 6144 Acked Write 10 Done Done Sent false
19897 6145 Acked Write 2 Done Done Sent false
19898 6146 Acked Write 6 Done Done Sent false
19899 6147 Acked Write 6 Done Done Sent false
19900 6148 Acked Write 7 Done Done Sent false
19901 6149 Acked Read 1 Done Done Sent false
19902 6150 Acked Flush 0 Done Done Sent false
19903 6151 Acked Write 6 Done Done Sent false
19904 6152 Acked Write 2 Done Done Sent false
19905 6153 Acked Read 7 Done Done Sent false
19906 6154 Acked Read 1 Done Done Sent false
19907 6155 Acked Write 5 Done Done Sent false
19908 6156 Acked Write 10 Done Done Sent false
19909 6157 Acked Write 10 Done Done Sent false
19910 6158 Acked Read 10 Done Done Sent false
19911 6159 Acked Flush 0 Done Done Sent false
19912 6160 Acked Read 3 Done Done Sent false
19913 6161 Acked Flush 0 Done Done Sent false
19914 6162 Acked Read 10 Done Done Sent false
19915 6163 Acked Write 6 Done Done Sent false
19916 6164 Acked Read 7 Done Done Sent false
19917 6165 Acked Read 1 Done Done Sent false
19918 6166 Acked Read 4 Done Done Sent false
19919 6167 Acked Flush 0 Done Done Sent false
19920 6168 Acked Read 9 Done Done Sent false
19921 6169 Acked Write 5 Done Done Sent false
19922 6170 Acked Write 10 Done Done Sent false
19923 6171 Acked Write 4 Done Done Sent false
19924 6172 Acked Write 4 Done Done Sent false
19925 6173 Acked Write 3 Done Done Sent false
19926 6174 Acked Write 4 Done Done Sent false
19927 6175 Acked Write 4 Done Done Sent false
19928 6176 Acked Read 3 Done Done Sent false
19929 6177 Acked Write 6 Done Done Sent false
19930 6178 Acked Read 5 Done Done Sent false
19931 6179 Acked Read 8 Done Done Sent false
19932 6180 Acked Write 9 Done Done Sent false
19933 6181 Acked Read 8 Done Done Sent false
19934 6182 Acked Write 4 Done Done Sent false
19935 6183 Acked Flush 0 Done Done Sent false
19936 6184 Acked Flush 0 Done Done Sent false
19937 6185 Acked Write 9 Done Done Sent false
19938 6186 Acked Read 2 Done Done Sent false
19939 6187 Acked Read 6 Done Done Sent false
19940 6188 Acked Read 3 Done Done Sent false
19941 6189 Acked Write 6 Done Done Sent false
19942 6190 Acked Flush 0 Done Done Sent false
19943 6191 Acked Write 1 Done Done Sent false
19944 6192 Acked Write 4 Done Done Sent false
19945 6193 Acked Flush 0 Done Done Sent false
19946 6194 Acked Read 4 Done Done Sent false
19947 6195 Acked Write 8 Done Done Sent false
19948 6196 Acked Write 1 Done Done Sent false
19949 6197 Acked Read 7 Done Done Sent false
19950 6198 Acked Read 6 Done Done Sent false
19951 6199 Acked Write 1 Done Done Sent false
19952 6200 Acked Write 5 Done Done Sent false
19953 6201 Acked Flush 0 Done Done Sent false
19954 6202 Acked Read 3 Done Done Sent false
19955 6203 Acked Write 10 Done Done Sent false
19956 6204 Acked Read 10 Done Done Sent false
19957 6205 Acked Write 3 Done Done Sent false
19958 6206 Acked Read 8 Done Done Sent false
19959 6207 Acked Read 9 Done Done Sent false
19960 6208 Acked Write 3 Done Done Sent false
19961 6209 Acked Write 1 Done Done Sent false
19962 6210 Acked Read 9 Done Done Sent false
19963 6211 Acked Write 7 Done Done Sent false
19964 6212 Acked Flush 0 Done Done Sent false
19965 6213 Acked Write 9 Done Done Sent false
19966 6214 Acked Read 5 Done Done Sent false
19967 6215 Acked Read 8 Done Done Sent false
19968 6216 Acked Write 7 Done Done Sent false
19969 6217 Acked Read 10 Done Done Sent false
19970 6218 Acked Read 10 Done Done Sent false
19971 6219 Acked Read 9 Done Done Sent false
19972 6220 Acked Read 2 Done Done Sent false
19973 6221 Acked Read 9 Done Done Sent false
19974 6222 Acked Write 1 Done Done Sent false
19975 6223 Acked Write 4 Done Done Sent false
19976 6224 Acked Read 8 Done Done Sent false
19977 6225 Acked Read 10 Done Done Sent false
19978 6226 Acked Read 1 Done Done Sent false
19979 6227 Acked Flush 0 Done Done Sent false
19980 6228 Acked Read 4 Done Done Sent false
19981 6229 Acked Write 8 Done Done Sent false
19982 6230 Acked Flush 0 Done Done Sent false
19983 6231 Acked Write 2 Done Done Sent false
19984 6232 Acked Read 8 Done Done Sent false
19985 6233 Acked Read 10 Done Done Sent false
19986 6234 Acked Flush 0 Done Done Sent false
19987 6235 Acked Read 2 Done Done Sent false
19988 6236 Acked Write 7 Done Done Sent false
19989 6237 Acked Read 5 Done Done Sent false
19990 6238 Acked Write 5 Done Done Sent false
19991 6239 Acked Read 7 Done Done Sent false
19992 6240 Acked Read 6 Done Done Sent false
19993 6241 Acked Write 7 Done Done Sent false
19994 6242 Acked Flush 0 Done Done Sent false
19995 6243 Acked Read 4 Done Done Sent false
19996 6244 Acked Read 7 Done Done Sent false
19997 6245 Acked Flush 0 Done Done Sent false
19998 6246 Acked Write 7 Done Done Sent false
19999 6247 Acked Write 5 Done Done Sent false
20000 6248 Acked Write 10 Done Done Sent false
20001 6249 Acked Write 7 Done Done Sent false
20002 6250 Acked Write 9 Done Done Sent false
20003 6251 Acked Write 3 Done Done Sent false
20004 6252 Acked Read 10 Done Done Sent false
20005 6253 Acked Read 9 Done Done Sent false
20006 6254 Acked Write 9 Done Done Sent false
20007 6255 Acked Read 4 Done Done Sent false
20008 6256 Acked Write 5 Done Done Sent false
20009 6257 Acked Read 7 Done Done Sent false
20010 6258 Acked Read 10 Done Done Sent false
20011 6259 Acked Read 8 Done Done Sent false
20012 6260 Acked Flush 0 Done Done Sent false
20013 6261 Acked Write 3 Done Done Sent false
20014 6262 Acked Read 4 Done Done Sent false
20015 6263 Acked Write 8 Done Done Sent false
20016 6264 Acked Write 3 Done Done Sent false
20017 6265 Acked Read 8 Done Done Sent false
20018 6266 Acked Flush 0 Done Done Sent false
20019 6267 Acked Read 5 Done Done Sent false
20020 6268 Acked Read 7 Done Done Sent false
20021 6269 Acked Read 8 Done Done Sent false
20022 6270 Acked Flush 0 Done Done Sent false
20023 6271 Acked Read 9 Done Done Sent false
20024 6272 Acked Read 1 Done Done Sent false
20025 6273 Acked Write 3 Done Done Sent false
20026 6274 Acked Read 2 Done Done Sent false
20027 6275 Acked Write 1 Done Done Sent false
20028 6276 Acked Write 2 Done Done Sent false
20029 6277 Acked Read 2 Done Done Sent false
20030 6278 Acked Flush 0 Done Done Sent false
20031 6279 Acked Flush 0 Done Done Sent false
20032 6280 Acked Read 2 Done Done Sent false
20033 6281 Acked Read 1 Done Done Sent false
20034 6282 Acked Write 3 Done Done Sent false
20035 6283 Acked Read 5 Done Done Sent false
20036 6284 Acked Write 4 Done Done Sent false
20037 6285 Acked Flush 0 Done Done Sent false
20038 6286 Acked Write 7 Done Done Sent false
20039 6287 Acked Write 4 Done Done Sent false
20040 6288 Acked Write 9 Done Done Sent false
20041 6289 Acked Write 6 Done Done Sent false
20042 6290 Acked Write 2 Done Done Sent false
20043 6291 Acked Read 10 Done Done Sent false
20044 6292 Acked Write 8 Done Done Sent false
20045 6293 Acked Read 7 Done Done Sent false
20046 6294 Acked Read 9 Done Done Sent false
20047 6295 Acked Write 2 Done Done Sent false
20048 6296 Acked Write 10 Done Done Sent false
20049 6297 Acked Read 1 Done Done Sent false
20050 6298 Acked Read 10 Done Done Sent false
20051 6299 Acked Read 2 Done Done Sent false
20052 6300 Acked Read 9 Done Done Sent false
20053 6301 Acked Read 9 Done Done Sent false
20054 6302 Acked Write 2 Done Done Sent false
20055 6303 Acked Write 5 Done Done Sent false
20056 6304 Acked Write 6 Done Done Sent false
20057 6305 Acked Flush 0 Done Done Sent false
20058 6306 Acked Write 3 Done Done Sent false
20059 6307 Acked Read 2 Done Done Sent false
20060 6308 Acked Read 5 Done Done Sent false
20061 6309 Acked Read 3 Done Done Sent false
20062 6310 Acked Write 4 Done Done Sent false
20063 6311 Acked Write 8 Done Done Sent false
20064 6312 Acked Write 10 Done Done Sent false
20065 6313 Acked Read 8 Done Done Sent false
20066 6314 Acked Read 3 Done Done Sent false
20067 6315 Acked Write 7 Done Done Sent false
20068 6316 Acked Read 8 Done Done Sent false
20069 6317 Acked Write 10 Done Done Sent false
20070 6318 Acked Flush 0 Done Done Sent false
20071 6319 Acked Read 7 Done Done Sent false
20072 6320 Acked Read 9 Done Done Sent false
20073 6321 Acked Write 1 Done Done Sent false
20074 6322 Acked Read 5 Done Done Sent false
20075 6323 Acked Write 8 Done Done Sent false
20076 6324 Acked Read 1 Done Done Sent false
20077 6325 Acked Read 6 Done Done Sent false
20078 6326 Acked Read 8 Done Done Sent false
20079 6327 Acked Write 8 Done Done Sent false
20080 6328 Acked Read 4 Done Done Sent false
20081 6329 Acked Read 6 Done Done Sent false
20082 6330 Acked Read 6 Done Done Sent false
20083 6331 Acked Read 4 Done Done Sent false
20084 6332 Acked Read 5 Done Done Sent false
20085 6333 Acked Write 10 Done Done Sent false
20086 6334 Acked Write 6 Done Done Sent false
20087 6335 Acked Flush 0 Done Done Sent false
20088 6336 Acked Read 9 Done Done Sent false
20089 6337 Acked Write 7 Done Done Sent false
20090 6338 Acked Write 7 Done Done Sent false
20091 6339 Acked Read 5 Done Done Sent false
20092 6340 Acked Read 1 Done Done Sent false
20093 6341 Acked Write 3 Done Done Sent false
20094 6342 Acked Read 4 Done Done Sent false
20095 6343 Acked Write 2 Done Done Sent false
20096 6344 Acked Write 4 Done Done Sent false
20097 6345 Acked Read 7 Done Done Sent false
20098 6346 Acked Write 6 Done Done Sent false
20099 6347 Acked Read 10 Done Done Sent false
20100 6348 Acked Flush 0 Done Done Sent false
20101 6349 Acked Flush 0 Done Done Sent false
20102 6350 Acked Read 2 Done Done Sent false
20103 6351 Acked Read 9 Done Done Sent false
20104 6352 Acked Read 1 Done Done Sent false
20105 6353 Acked Read 6 Done Done Sent false
20106 6354 Acked Write 9 Done Done Sent false
20107 6355 Acked Write 7 Done Done Sent false
20108 6356 Acked Read 10 Done Done Sent false
20109 6357 Acked Read 5 Done Done Sent false
20110 6358 Acked Read 9 Done Done Sent false
20111 6359 Acked Write 7 Done Done Sent false
20112 6360 Acked Read 4 Done Done Sent false
20113 6361 Acked Write 3 Done Done Sent false
20114 6362 Acked Write 4 Done Done Sent false
20115 6363 Acked Write 4 Done Done Sent false
20116 6364 Acked Write 8 Done Done Sent false
20117 6365 Acked Read 2 Done Done Sent false
20118 6366 Acked Read 6 Done Done Sent false
20119 6367 Acked Read 5 Done Done Sent false
20120 6368 Acked Read 9 Done Done Sent false
20121 6369 Acked Write 5 Done Done Sent false
20122 6370 Acked Write 10 Done Done Sent false
20123 6371 Acked Read 6 Done Done Sent false
20124 6372 Acked Read 7 Done Done Sent false
20125 6373 Acked Read 3 Done Done Sent false
20126 6374 Acked Write 4 Done Done Sent false
20127 6375 Acked Read 8 Done Done Sent false
20128 6376 Acked Write 2 Done Done Sent false
20129 6377 Acked Write 1 Done Done Sent false
20130 6378 Acked Flush 0 Done Done Sent false
20131 6379 Acked Write 9 Done Done Sent false
20132 6380 Acked Read 7 Done Done Sent false
20133 6381 Acked Write 8 Done Done Sent false
20134 6382 Acked Read 9 Done Done Sent false
20135 6383 Acked Write 2 Done Done Sent false
20136 6384 Acked Read 3 Done Done Sent false
20137 6385 Acked Read 5 Done Done Sent false
20138 6386 Acked Read 7 Done Done Sent false
20139 6387 Acked Read 6 Done Done Sent false
20140 6388 Acked Read 10 Done Done Sent false
20141 6389 Acked Read 7 Done Done Sent false
20142 6390 Acked Read 1 Done Done Sent false
20143 6391 Acked Write 10 Done Done Sent false
20144 6392 Acked Read 9 Done Done Sent false
20145 6393 Acked Read 6 Done Done Sent false
20146 6394 Acked Read 9 Done Done Sent false
20147 6395 Acked Write 6 Done Done Sent false
20148 6396 Acked Flush 0 Done Done Sent false
20149 6397 Acked Read 3 Done Done Sent false
20150 6398 Acked Read 5 Done Done Sent false
20151 6399 Acked Write 8 Done Done Sent false
20152 6400 Acked Read 8 Done Done Sent false
20153 6401 Acked Write 8 Done Done Sent false
20154 6402 Acked Read 2 Done Done Sent false
20155 6403 Acked Write 7 Done Done Sent false
20156 6404 Acked Write 9 Done Done Sent false
20157 6405 Acked Read 3 Done Done Sent false
20158 6406 Acked Read 3 Done Done Sent false
20159 6407 Acked Read 4 Done Done Sent false
20160 6408 Acked Read 2 Done Done Sent false
20161 6409 Acked Write 3 Done Done Sent false
20162 6410 Acked Write 6 Done Done Sent false
20163 6411 Acked Write 1 Done Done Sent false
20164 6412 Acked Write 7 Done Done Sent false
20165 6413 Acked Read 5 Done Done Sent false
20166 6414 Acked Read 4 Done Done Sent false
20167 6415 Acked Write 7 Done Done Sent false
20168 6416 Acked Write 7 Done Done Sent false
20169 6417 Acked Flush 0 Done Done Sent false
20170 6418 Acked Flush 0 Done Done Sent false
20171 6419 Acked Write 10 Done Done Sent false
20172 6420 Acked Write 6 Done Done Sent false
20173 6421 Acked Read 8 Done Done Sent false
20174 6422 Acked Read 1 Done Done Sent false
20175 6423 Acked Read 8 Done Done Sent false
20176 6424 Acked Write 3 Done Done Sent false
20177 6425 Acked Write 2 Done Done Sent false
20178 6426 Acked Write 9 Done Done Sent false
20179 6427 Acked Flush 0 Done Done Sent false
20180 6428 Acked Read 8 Done Done Sent false
20181 6429 Acked Read 3 Done Done Sent false
20182 6430 Acked Write 3 Done Done Sent false
20183 6431 Acked Read 6 Done Done Sent false
20184 6432 Acked Flush 0 Done Done Sent false
20185 STATES DS:0 DS:1 DS:2 TOTAL
20186 Sent 0 0 302 302
20187 Done 302 302 0 604
20188 Skipped 0 0 0 0
20189 Error 0 0 0 0
20190 Last Flush: 6432 6432 6130
20191 Downstairs last five completed: 6130 6129 6128 6127 6126
20192 Upstairs last five completed: 6432 6431 6430 6429 6428
20193 CLIENT: Up:0 ds:302 act:2
201942025-01-27T19:40:37.128ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
201952025-01-27T19:40:37.128ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:42397) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
201962025-01-27T19:40:37.129ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
201972025-01-27T19:40:37.130ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
201982025-01-27T19:40:37.130ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
201992025-01-27T19:40:37.130ZINFOcrucible: send last flush ID to this DS: 6130 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
202002025-01-27T19:40:37.130ZINFOcrucible: Replied this last flush ID: 6130 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
202012025-01-27T19:40:37.130ZINFOcrucible: [2] client re-new 302 jobs since flush 6130 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
202022025-01-27T19:40:37.130ZINFOcrucible: [2] Replayed 302 jobs since flush: 6130 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
20203 ----------------------------------------------------------------
20204 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
20205 Downstairs last five completed: 6432 6431 6430 6429 6428
20206 Upstairs last five completed: 6432 6431 6430 6429 6428
20207 CLIENT: Up:0 ds:0 act:3
20208 Replay: All jobs finished
20209 Main task has work to do, go find it
20210 got dsc Stop(0)
20211 stop 0
20212 Jan 27 19:40:37.372 INFO request completed, latency_us: 374, response_code: 204[8810] Got stop action so:false kr:false
20213 , uri: /stop/cid/0, method: POST, req_id: 8af69870-9e89-4f71-bedc-a4a08527a99a, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
202142025-01-27T19:40:37.373ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
20215 Jan 27 19:40:37.373 INFO request completed, latency_us: 160, response_code: 200, uri: /state/cid/0, method: GET, req_id: abb59965-ffda-42d4-a17f-7286920ffd84, remote_addr: 127.{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:40:37.373644727Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
20216 0.0.1:55047, local_addr: 127.0.0.1:9998
202172025-01-27T19:40:37.373ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
202182025-01-27T19:40:37.373ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
20219 [8810] Exited with: ExitStatus(unix_wait_status(9))
20220 [8810][0] reports Exit
20221 Jan 27 19:40:41.375 INFO request completed, latency_us: 204, response_code: 200, uri: /state/cid/0, method: GET, req_id: 8d4949b7-4a94-45b0-b178-1bff3e1c7b46, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
20222 001/300 Write block 1771 len 12288 data: 5 6 6
20223 002/300 Read block 4302 len 28672
20224 003/300 Read block 1818 len 24576
20225 004/300 Read block 1731 len 12288
20226 005/300 Flush
20227 006/300 Read block 1428 len 40960
20228 007/300 Read block 2680 len 20480
20229 008/300 Flush
20230 009/300 Read block 1950 len 20480
20231 010/300 Read block 2933 len 20480
20232 011/300 Write block 233 len 12288 data: 3 4 5
20233 012/300 Read block 3932 len 36864
20234 013/300 Read block 917 len 40960
20235 014/300 Write block 3175 len 4096 data: 5
20236 015/300 Read block 501 len 36864
20237 016/300 Write block 1323 len 12288 data: 4 4 4
20238 017/300 Read block 898 len 24576
20239 018/300 Read block 2536 len 40960
20240 019/300 Write block 4910 len 4096 data: 3
20241 020/300 Read block 2505 len 8192
20242 021/300 Read block 1061 len 16384
20243 022/300 Flush
20244 023/300 Write block 4244 len 4096 data: 4
20245 024/300 Read block 2853 len 32768
20246 025/300 Read block 2842 len 32768
20247 026/300 Read block 3059 len 36864
20248 027/300 Write block 2418 len 32768 data: 4 4 4 2 2 3 3 5
20249 028/300 Read block 1758 len 8192
20250 029/300 Read block 2656 len 36864
20251 030/300 Write block 4159 len 40960 data: 5 6 8 8 8 8 8 7 6 5
20252 031/300 Write block 4973 len 40960 data: 4 4 3 2 2 2 3 2 3 4
20253 032/300 Flush
20254 033/300 Read block 2366 len 40960
20255 034/300 Write block 4954 len 12288 data: 4 4 3
20256 035/300 Read block 4766 len 24576
20257 036/300 Read block 1691 len 40960
20258 037/300 Write block 338 len 24576 data: 5 5 4 4 4 4
20259 038/300 Read block 2476 len 40960
20260 039/300 Read block 4497 len 4096
20261 040/300 Read block 2443 len 4096
20262 041/300 Read block 608 len 36864
20263 042/300 Read block 2536 len 20480
20264 043/300 Write block 369 len 28672 data: 3 2 4 5 5 5 5
20265 044/300 Read block 380 len 8192
20266 045/300 Read block 529 len 32768
20267 046/300 Read block 1164 len 28672
20268 047/300 Read block 3756 len 16384
20269 048/300 Read block 1142 len 32768
20270 049/300 Write block 3433 len 24576 data: 7 7 6 6 6 6
20271 050/300 Read block 831 len 4096
20272 051/300 Read block 396 len 40960
20273 052/300 Read block 1344 len 16384
20274 053/300 Read block 3385 len 28672
20275 054/300 Read block 4106 len 24576
20276 055/300 Flush
20277 056/300 Read block 1280 len 8192
20278 057/300 Flush
20279 058/300 Read block 4634 len 8192
20280 059/300 Write block 205 len 28672 data: 5 5 5 5 5 4 4
20281 060/300 Write block 1218 len 20480 data: 4 5 4 5 5
20282 061/300 Write block 4074 len 40960 data: 2 3 2 3 3 2 3 3 3 3
20283 062/300 Read block 3543 len 28672
20284 063/300 Write block 1956 len 12288 data: 4 5 5
20285 064/300 Read block 482 len 24576
20286 065/300 Read block 3444 len 16384
20287 066/300 Write block 3863 len 28672 data: 4 3 3 3 4 4 4
20288 067/300 Read block 4257 len 20480
20289 068/300 Read block 109 len 24576
20290 069/300 Write block 4753 len 12288 data: 4 4 5
20291 070/300 Write block 4401 len 4096 data: 2
20292 071/300 Read block 1448 len 40960
20293 072/300 Read block 333 len 8192
20294 073/300 Write block 2291 len 8192 data: 7 7
20295 074/300 Read block 1157 len 40960
20296 075/300 Write block 1197 len 36864 data: 5 5 5 4 4 4 5 6 6
20297 076/300 Read block 4982 len 4096
20298 077/300 Write block 253 len 4096 data: 3
20299 078/300 Flush
20300 079/300 Read block 1953 len 28672
20301 080/300 Read block 4885 len 16384
20302 081/300 Flush
20303 082/300 Read block 4963 len 32768
20304 083/300 Write block 2235 len 12288 data: 3 5 5
20305 084/300 Read block 1833 len 24576
20306 085/300 Flush
20307 086/300 Read block 1260 len 8192
20308 087/300 Read block 1717 len 12288
20309 088/300 Read block 3109 len 8192
20310 089/300 Flush
20311 090/300 Write block 1600 len 16384 data: 5 4 4 4
20312 091/300 Read block 2014 len 28672
20313 092/300 Write block 4498 len 16384 data: 5 5 5 4
20314 093/300 Read block 4633 len 24576
20315 094/300 Read block 3940 len 4096
20316 095/300 Write block 2144 len 12288 data: 6 5 5
20317 096/300 Read block 4642 len 8192
20318 097/300 Read block 1949 len 32768
20319 098/300 Read block 4241 len 36864
20320 099/300 Write block 4255 len 28672 data: 4 5 4 4 4 6 5
20321 100/300 Read block 4302 len 4096
20322 101/300 Write block 742 len 32768 data: 5 6 5 6 6 6 6 6
20323 102/300 Flush
20324 103/300 Flush
20325 104/300 Write block 1312 len 36864 data: 5 4 5 5 5 5 5 5 5
20326 105/300 Read block 3309 len 40960
20327 106/300 Read block 3109 len 8192
20328 107/300 Read block 3374 len 12288
20329 108/300 Read block 1382 len 36864
20330 109/300 Read block 3715 len 36864
20331 110/300 Flush
20332 111/300 Write block 2722 len 8192 data: 5 3
20333 112/300 Read block 3156 len 20480
20334 113/300 Read block 2783 len 28672
20335 114/300 Read block 4659 len 24576
20336 115/300 Write block 843 len 24576 data: 3 3 3 4 3 4
20337 116/300 Write block 3162 len 36864 data: 4 4 3 2 2 2 2 3 3
20338 117/300 Write block 4677 len 20480 data: 2 2 2 5 6
20339 118/300 Flush
20340 119/300 Flush
20341 120/300 Read block 1012 len 32768
20342 121/300 Read block 4042 len 28672
20343 122/300 Read block 2628 len 24576
20344 123/300 Read block 3393 len 20480
20345 124/300 Write block 1619 len 16384 data: 5 4 3 3
20346 125/300 Flush
20347 126/300 Read block 803 len 28672
20348 127/300 Write block 2562 len 8192 data: 6 7
20349 128/300 Read block 3523 len 4096
20350 129/300 Read block 674 len 32768
20351 130/300 Read block 3063 len 36864
20352 131/300 Write block 512 len 40960 data: 5 5 5 5 4 4 4 5 5 4
20353 132/300 Write block 1598 len 12288 data: 8 5 6
20354 133/300 Read block 1090 len 20480
20355 134/300 Flush
20356 135/300 Read block 1384 len 8192
20357 136/300 Write block 3128 len 4096 data: 4
20358 137/300 Read block 60 len 12288
20359 138/300 Write block 4777 len 40960 data: 6 5 3 2 2 3 3 3 4 5
20360 139/300 Write block 2188 len 24576 data: 3 2 3 3 3 4
20361 140/300 Read block 290 len 40960
20362 141/300 Read block 2507 len 20480
20363 142/300 Read block 4416 len 20480
20364 143/300 Read block 3075 len 12288
20365 144/300 Write block 4339 len 12288 data: 6 3 3
20366 145/300 Read block 1025 len 40960
20367 146/300 Read block 1450 len 32768
20368 147/300 Flush
20369 148/300 Write block 496 len 16384 data: 7 9 10 10
20370 149/300 Write block 885 len 36864 data: 2 2 2 2 2 3 4 4 3
20371 150/300 Read block 4735 len 12288
20372 151/300 Flush
20373 152/300 Read block 3869 len 32768
20374 153/300 Read block 4000 len 8192
20375 154/300 Read block 389 len 36864
20376 155/300 Write block 3525 len 20480 data: 5 5 4 4 3
20377 156/300 Read block 4981 len 4096
20378 157/300 Read block 224 len 16384
20379 158/300 Read block 346 len 4096
20380 159/300 Read block 3563 len 16384
20381 160/300 Read block 3771 len 16384
20382 161/300 Read block 3539 len 28672
20383 162/300 Read block 187 len 24576
20384 163/300 Write block 2008 len 36864 data: 4 4 4 6 4 5 5 5 6
20385 164/300 Write block 3450 len 32768 data: 5 6 6 7 6 8 7 7
20386 165/300 Write block 166 len 28672 data: 5 5 7 7 7 7 5
20387 166/300 Read block 3867 len 40960
20388 167/300 Read block 3676 len 24576
20389 168/300 Read block 749 len 28672
20390 169/300 Read block 3431 len 28672
20391 170/300 Flush
20392 171/300 Write block 4 len 12288 data: 2 2 2
20393 172/300 Write block 2824 len 40960 data: 3 3 3 3 5 5 6 6 7 7
20394 173/300 Write block 3580 len 8192 data: 2 2
20395 174/300 Write block 168 len 16384 data: 8 8 8 8
20396 175/300 Read block 2572 len 4096
20397 176/300 Write block 2620 len 36864 data: 4 5 5 5 6 4 3 4 4
20398 177/300 Flush
20399 178/300 Write block 2255 len 24576 data: 7 8 6 6 6 6
20400 179/300 Flush
20401 180/300 Read block 2458 len 20480
20402 181/300 Read block 1666 len 8192
20403 182/300 Read block 2929 len 20480
20404 183/300 Write block 252 len 40960 data: 3 4 3 3 3 3 3 4 4 4
20405 184/300 Read block 2518 len 36864
20406 185/300 Write block 406 len 36864 data: 6 4 4 5 5 4 4 4 4
20407 186/300 Flush
20408 187/300 Read block 4140 len 12288
20409 188/300 Read block 4443 len 24576
20410 189/300 Read block 4528 len 12288
20411 190/300 Write block 4368 len 16384 data: 3 3 2 2
20412 191/300 Read block 1 len 20480
20413 192/300 Write block 3549 len 40960 data: 3 3 3 2 4 4 4 6 6 5
20414 193/300 Write block 1371 len 40960 data: 5 4 6 7 8 9 9 9 9 7
20415 194/300 Write block 134 len 20480 data: 5 5 5 5 7
20416 195/300 Read block 1814 len 24576
20417 196/300 Flush
20418 197/300 Read block 397 len 4096
20419 198/300 Read block 1247 len 4096
20420 199/300 Read block 4117 len 8192
20421 200/300 Write block 2608 len 8192 data: 5 4
20422 201/300 Flush
20423 202/300 Read block 4949 len 8192
20424 203/300 Write block 1400 len 28672 data: 7 6 5 5 4 7 7
20425 204/300 Write block 4835 len 4096 data: 6
20426 205/300 Write block 4197 len 12288 data: 7 5 4
20427 206/300 Read block 3683 len 40960
20428 207/300 Flush
20429 208/300 Write block 2617 len 36864 data: 3 3 4 5 6 6 6 7 5
20430 209/300 Read block 10 len 4096
20431 210/300 Read block 4630 len 32768
20432 211/300 Write block 4632 len 32768 data: 4 5 4 3 3 3 4 5
20433 212/300 Write block 256 len 32768 data: 4 4 4 5 5 5 4 4
20434 213/300 Flush
20435 214/300 Read block 1982 len 8192
20436 215/300 Read block 695 len 32768
20437 216/300 Read block 1110 len 4096
20438 217/300 Read block 2839 len 12288
20439 218/300 Write block 4860 len 12288 data: 5 6 5
20440 219/300 Write block 3748 len 16384 data: 6 6 4 3
20441 220/300 Write block 1113 len 36864 data: 4 5 5 5 6 6 4 5 6
20442 221/300 Read block 1410 len 12288
20443 222/300 Read block 2779 len 20480
20444 223/300 Read block 570 len 4096
20445 224/300 Write block 1019 len 12288 data: 6 6 5
20446 225/300 Read block 3354 len 4096
20447 226/300 Read block 1635 len 36864
20448 227/300 Read block 3405 len 16384
20449 228/300 Write block 702 len 8192 data: 4 3
20450 229/300 Read block 626 len 8192
20451 230/300 Write block 2377 len 12288 data: 7 9 5
20452 231/300 Write block 4273 len 32768 data: 4 4 7 6 6 6 6 7
20453 232/300 Read block 4998 len 8192
20454 233/300 Write block 2408 len 16384 data: 6 5 7 7
20455 234/300 Read block 2049 len 20480
20456 235/300 Read block 4762 len 28672
20457 236/300 Read block 2218 len 40960
20458 237/300 Read block 2397 len 32768
20459 238/300 Write block 3646 len 28672 data: 4 3 4 5 5 5 3
20460 239/300 Write block 4686 len 4096 data: 4
20461 240/300 Read block 1606 len 8192
20462 241/300 Read block 2533 len 32768
20463 242/300 Read block 2551 len 24576
20464 243/300 Read block 1938 len 8192
20465 244/300 Read block 1766 len 8192
20466 245/300 Read block 4703 len 28672
20467 246/300 Read block 2235 len 8192
20468 247/300 Write block 2133 len 24576 data: 4 4 2 3 3 3
20469 248/300 Write block 1710 len 40960 data: 2 2 2 3 3 4 4 4 3 5
20470 249/300 Read block 3553 len 40960
20471 250/300 Write block 2278 len 28672 data: 3 3 3 4 4 5 4
20472 251/300 Write block 3987 len 8192 data: 3 3
20473 252/300 Write block 1878 len 4096 data: 4
20474 253/300 Read block 470 len 32768
20475 254/300 Read block 3806 len 4096
20476 255/300 Write block 3444 len 24576 data: 4 4 3 3 2 5
20477 256/300 Read block 4061 len 24576
20478 257/300 Write block 3557 len 12288 data: 7 6 6
20479 258/300 Flush
20480 259/300 Read block 3653 len 40960
20481 260/300 Write block 4777 len 20480 data: 7 6 4 3 3
20482 261/300 Read block 3244 len 8192
20483 262/300 Flush
20484 263/300 Write block 2252 len 28672 data: 4 4 5 8 9 7 7
20485 264/300 Flush
20486 265/300 Read block 3718 len 40960
20487 266/300 Read block 2049 len 28672
20488 267/300 Read block 227 len 24576
20489 268/300 Read block 3715 len 24576
20490 269/300 Read block 133 len 32768
20491 270/300 Write block 3830 len 24576 data: 5 4 4 4 3 2
20492 271/300 Read block 4915 len 24576
20493 272/300 Read block 1723 len 40960
20494 273/300 Read block 3027 len 28672
20495 274/300 Write block 1620 len 24576 data: 5 4 4 4 5 5
20496 275/300 Read block 3586 len 20480
20497 276/300 Write block 257 len 24576 data: 5 5 6 6 6 5
20498 277/300 Write block 937 len 20480 data: 3 3 2 2 3
20499 278/300 Write block 4358 len 28672 data: 3 3 2 3 3 4 4
20500 279/300 Read block 1566 len 36864
20501 280/300 Read block 1657 len 32768
20502 281/300 Read block 1921 len 36864
20503 282/300 Read block 4962 len 28672
20504 283/300 Read block 4603 len 40960
20505 284/300 Write block 2152 len 28672 data: 3 3 5 5 7 8 8
20506 285/300 Write block 252 len 28672 data: 4 5 4 4 5 6 6
20507 286/300 Read block 1289 len 40960
20508 287/300 Flush
20509 288/300 Flush
20510 289/300 Write block 279 len 32768 data: 8 10 11 9 7 6 6 6
20511 290/300 Read block 2339 len 28672
20512 291/300 Flush
20513 292/300 Read block 3729 len 4096
20514 293/300 Read block 4371 len 12288
20515 294/300 Write block 602 len 4096 data: 7
20516 295/300 Write block 4718 len 8192 data: 6 3
20517 296/300 Write block 4867 len 12288 data: 6 5 6
20518 297/300 Read block 1337 len 32768
20519 298/300 Read block 1734 len 4096
20520 299/300 Flush
20521 300/300 Read block 4266 len 12288
20522 301/300 Read block 4599 len 28672
20523 Main task has work to do, go find it
20524 got dsc Start(0)
20525 start 0
20526 Jan 27 19:40:41.622 INFO request completed, latency_us: 191, response_code: 204, uri: /start/cid/0, method: POST, req_id: 00035e63-56ef-4e12-b691-405374461798, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
20527 Replay: started 0, returned:Ok(())
20528 ----------------------------------------------------------------
20529 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
20530 States: Offline Active Active
20531 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20532 6433 Acked Write 3 Sent Done Done false
20533 6434 Acked Read 7 Sent Done Done false
20534 6435 Acked Read 6 Sent Done Done false
20535 6436 Acked Read 3 Sent Done Done false
20536 6437 Acked Flush 0 Sent Done Done false
20537 6438 Acked Read 10 Sent Done Done false
20538 6439 Acked Read 5 Sent Done Done false
20539 6440 Acked Flush 0 Sent Done Done false
20540 6441 Acked Read 5 Sent Done Done false
20541 6442 Acked Read 5 Sent Done Done false
20542 6443 Acked Write 3 Sent Done Done false
20543 6444 Acked Read 9 Sent Done Done false
20544 6445 Acked Read 10 Sent Done Done false
20545 6446 Acked Write 1 Sent Done Done false
20546 6447 Acked Read 9 Sent Done Done false
20547 6448 Acked Write 3 Sent Done Done false
20548 6449 Acked Read 6 Sent Done Done false
20549 6450 Acked Read 10 Sent Done Done false
20550 6451 Acked Write 1 Sent Done Done false
20551 6452 Acked Read 2 Sent Done Done false
20552 6453 Acked Read 4 Sent Done Done false
20553 6454 Acked Flush 0 Sent Done Done false
20554 6455 Acked Write 1 Sent Done Done false
20555 6456 Acked Read 8 Sent Done Done false
20556 6457 Acked Read 8 Sent Done Done false
20557 6458 Acked Read 9 Sent Done Done false
20558 6459 Acked Write 8 Sent Done Done false
20559 6460 Acked Read 2 Sent Done Done false
20560 6461 Acked Read 9 Sent Done Done false
20561 6462 Acked Write 10 Sent Done Done false
20562 6463 Acked Write 10 Sent Done Done false
20563 6464 Acked Flush 0 Sent Done Done false
20564 6465 Acked Read 10 Sent Done Done false
20565 6466 Acked Write 3 Sent Done Done false
20566 6467 Acked Read 6 Sent Done Done false
20567 6468 Acked Read 10 Sent Done Done false
20568 6469 Acked Write 6 Sent Done Done false
20569 6470 Acked Read 10 Sent Done Done false
20570 6471 Acked Read 1 Sent Done Done false
20571 6472 Acked Read 1 Sent Done Done false
20572 6473 Acked Read 9 Sent Done Done false
20573 6474 Acked Read 5 Sent Done Done false
20574 6475 Acked Write 7 Sent Done Done false
20575 6476 Acked Read 2 Sent Done Done false
20576 6477 Acked Read 8 Sent Done Done false
20577 6478 Acked Read 7 Sent Done Done false
20578 6479 Acked Read 4 Sent Done Done false
20579 6480 Acked Read 8 Sent Done Done false
20580 6481 Acked Write 6 Sent Done Done false
20581 6482 Acked Read 1 Sent Done Done false
20582 6483 Acked Read 10 Sent Done Done false
20583 6484 Acked Read 4 Sent Done Done false
20584 6485 Acked Read 7 Sent Done Done false
20585 6486 Acked Read 6 Sent Done Done false
20586 6487 Acked Flush 0 Sent Done Done false
20587 6488 Acked Read 2 Sent Done Done false
20588 6489 Acked Flush 0 Sent Done Done false
20589 6490 Acked Read 2 Sent Done Done false
20590 6491 Acked Write 7 Sent Done Done false
20591 6492 Acked Write 5 Sent Done Done false
20592 6493 Acked Write 10 Sent Done Done false
20593 6494 Acked Read 7 Sent Done Done false
20594 6495 Acked Write 3 Sent Done Done false
20595 6496 Acked Read 6 Sent Done Done false
20596 6497 Acked Read 4 Sent Done Done false
20597 6498 Acked Write 7 Sent Done Done false
20598 6499 Acked Read 5 Sent Done Done false
20599 6500 Acked Read 6 Sent Done Done false
20600 6501 Acked Write 3 Sent Done Done false
20601 6502 Acked Write 1 Sent Done Done false
20602 6503 Acked Read 10 Sent Done Done false
20603 6504 Acked Read 2 Sent Done Done false
20604 6505 Acked Write 2 Sent Done Done false
20605 6506 Acked Read 10 Sent Done Done false
20606 6507 Acked Write 9 Sent Done Done false
20607 6508 Acked Read 1 Sent Done Done false
20608 6509 Acked Write 1 Sent Done Done false
20609 6510 Acked Flush 0 Sent Done Done false
20610 6511 Acked Read 7 Sent Done Done false
20611 6512 Acked Read 4 Sent Done Done false
20612 6513 Acked Flush 0 Sent Done Done false
20613 6514 Acked Read 8 Sent Done Done false
20614 6515 Acked Write 3 Sent Done Done false
20615 6516 Acked Read 6 Sent Done Done false
20616 6517 Acked Flush 0 Sent Done Done false
20617 6518 Acked Read 2 Sent Done Done false
20618 6519 Acked Read 3 Sent Done Done false
20619 6520 Acked Read 2 Sent Done Done false
20620 6521 Acked Flush 0 Sent Done Done false
20621 6522 Acked Write 4 Sent Done Done false
20622 6523 Acked Read 7 Sent Done Done false
20623 6524 Acked Write 4 Sent Done Done false
20624 6525 Acked Read 6 Sent Done Done false
20625 6526 Acked Read 1 Sent Done Done false
20626 6527 Acked Write 3 Sent Done Done false
20627 6528 Acked Read 2 Sent Done Done false
20628 6529 Acked Read 8 Sent Done Done false
20629 6530 Acked Read 9 Sent Done Done false
20630 6531 Acked Write 7 Sent Done Done false
20631 6532 Acked Read 1 Sent Done Done false
20632 6533 Acked Write 8 Sent Done Done false
20633 6534 Acked Flush 0 Sent Done Done false
20634 6535 Acked Flush 0 Sent Done Done false
20635 6536 Acked Write 9 Sent Done Done false
20636 6537 Acked Read 10 Sent Done Done false
20637 6538 Acked Read 2 Sent Done Done false
20638 6539 Acked Read 3 Sent Done Done false
20639 6540 Acked Read 9 Sent Done Done false
20640 6541 Acked Read 9 Sent Done Done false
20641 6542 Acked Flush 0 Sent Done Done false
20642 6543 Acked Write 2 Sent Done Done false
20643 6544 Acked Read 5 Sent Done Done false
20644 6545 Acked Read 7 Sent Done Done false
20645 6546 Acked Read 6 Sent Done Done false
20646 6547 Acked Write 6 Sent Done Done false
20647 6548 Acked Write 9 Sent Done Done false
20648 6549 Acked Write 5 Sent Done Done false
20649 6550 Acked Flush 0 Sent Done Done false
20650 6551 Acked Flush 0 Sent Done Done false
20651 6552 Acked Read 8 Sent Done Done false
20652 6553 Acked Read 7 Sent Done Done false
20653 6554 Acked Read 6 Sent Done Done false
20654 6555 Acked Read 5 Sent Done Done false
20655 6556 Acked Write 4 Sent Done Done false
20656 6557 Acked Flush 0 Sent Done Done false
20657 6558 Acked Read 7 Sent Done Done false
20658 6559 Acked Write 2 Sent Done Done false
20659 6560 Acked Read 1 Sent Done Done false
20660 6561 Acked Read 8 Sent Done Done false
20661 6562 Acked Read 9 Sent Done Done false
20662 6563 Acked Write 10 Sent Done Done false
20663 6564 Acked Write 3 Sent Done Done false
20664 6565 Acked Read 5 Sent Done Done false
20665 6566 Acked Flush 0 Sent Done Done false
20666 6567 Acked Read 2 Sent Done Done false
20667 6568 Acked Write 1 Sent Done Done false
20668 6569 Acked Read 3 Sent Done Done false
20669 6570 Acked Write 10 Sent Done Done false
20670 6571 Acked Write 6 Sent Done Done false
20671 6572 Acked Read 10 Sent Done Done false
20672 6573 Acked Read 5 Sent Done Done false
20673 6574 Acked Read 5 Sent Done Done false
20674 6575 Acked Read 3 Sent Done Done false
20675 6576 Acked Write 3 Sent Done Done false
20676 6577 Acked Read 10 Sent Done Done false
20677 6578 Acked Read 8 Sent Done Done false
20678 6579 Acked Flush 0 Sent Done Done false
20679 6580 Acked Write 4 Sent Done Done false
20680 6581 Acked Write 9 Sent Done Done false
20681 6582 Acked Read 3 Sent Done Done false
20682 6583 Acked Flush 0 Sent Done Done false
20683 6584 Acked Read 8 Sent Done Done false
20684 6585 Acked Read 2 Sent Done Done false
20685 6586 Acked Read 9 Sent Done Done false
20686 6587 Acked Write 5 Sent Done Done false
20687 6588 Acked Read 1 Sent Done Done false
20688 6589 Acked Read 4 Sent Done Done false
20689 6590 Acked Read 1 Sent Done Done false
20690 6591 Acked Read 4 Sent Done Done false
20691 6592 Acked Read 4 Sent Done Done false
20692 6593 Acked Read 7 Sent Done Done false
20693 6594 Acked Read 6 Sent Done Done false
20694 6595 Acked Write 9 Sent Done Done false
20695 6596 Acked Write 8 Sent Done Done false
20696 6597 Acked Write 7 Sent Done Done false
20697 6598 Acked Read 10 Sent Done Done false
20698 6599 Acked Read 6 Sent Done Done false
20699 6600 Acked Read 7 Sent Done Done false
20700 6601 Acked Read 7 Sent Done Done false
20701 6602 Acked Flush 0 Sent Done Done false
20702 6603 Acked Write 3 Sent Done Done false
20703 6604 Acked Write 10 Sent Done Done false
20704 6605 Acked Write 2 Sent Done Done false
20705 6606 Acked Write 4 Sent Done Done false
20706 6607 Acked Read 1 Sent Done Done false
20707 6608 Acked Write 9 Sent Done Done false
20708 6609 Acked Flush 0 Sent Done Done false
20709 6610 Acked Write 6 Sent Done Done false
20710 6611 Acked Flush 0 Sent Done Done false
20711 6612 Acked Read 5 Sent Done Done false
20712 6613 Acked Read 2 Sent Done Done false
20713 6614 Acked Read 5 Sent Done Done false
20714 6615 Acked Write 10 Sent Done Done false
20715 6616 Acked Read 9 Sent Done Done false
20716 6617 Acked Write 9 Sent Done Done false
20717 6618 Acked Flush 0 Sent Done Done false
20718 6619 Acked Read 3 Sent Done Done false
20719 6620 Acked Read 6 Sent Done Done false
20720 6621 Acked Read 3 Sent Done Done false
20721 6622 Acked Write 4 Sent Done Done false
20722 6623 Acked Read 5 Sent Done Done false
20723 6624 Acked Write 10 Sent Done Done false
20724 6625 Acked Write 10 Sent Done Done false
20725 6626 Acked Write 5 Sent Done Done false
20726 6627 Acked Read 6 Sent Done Done false
20727 6628 Acked Flush 0 Sent Done Done false
20728 6629 Acked Read 1 Sent Done Done false
20729 6630 Acked Read 1 Sent Done Done false
20730 6631 Acked Read 2 Sent Done Done false
20731 6632 Acked Write 2 Sent Done Done false
20732 6633 Acked Flush 0 Sent Done Done false
20733 6634 Acked Read 2 Sent Done Done false
20734 6635 Acked Write 7 Sent Done Done false
20735 6636 Acked Write 1 Sent Done Done false
20736 6637 Acked Write 3 Sent Done Done false
20737 6638 Acked Read 10 Sent Done Done false
20738 6639 Acked Flush 0 Sent Done Done false
20739 6640 Acked Write 9 Sent Done Done false
20740 6641 Acked Read 1 Sent Done Done false
20741 6642 Acked Read 8 Sent Done Done false
20742 6643 Acked Write 8 Sent Done Done false
20743 6644 Acked Write 8 Sent Done Done false
20744 6645 Acked Flush 0 Sent Done Done false
20745 6646 Acked Read 2 Sent Done Done false
20746 6647 Acked Read 8 Sent Done Done false
20747 6648 Acked Read 1 Sent Done Done false
20748 6649 Acked Read 3 Sent Done Done false
20749 6650 Acked Write 3 Sent Done Done false
20750 6651 Acked Write 4 Sent Done Done false
20751 6652 Acked Write 9 Sent Done Done false
20752 6653 Acked Read 3 Sent Done Done false
20753 6654 Acked Read 5 Sent Done Done false
20754 6655 Acked Read 1 Sent Done Done false
20755 6656 Acked Write 3 Sent Done Done false
20756 6657 Acked Read 1 Sent Done Done false
20757 6658 Acked Read 9 Sent Done Done false
20758 6659 Acked Read 4 Sent Done Done false
20759 6660 Acked Write 2 Sent Done Done false
20760 6661 Acked Read 2 Sent Done Done false
20761 6662 Acked Write 3 Sent Done Done false
20762 6663 Acked Write 8 Sent Done Done false
20763 6664 Acked Read 2 Sent Done Done false
20764 6665 Acked Write 4 Sent Done Done false
20765 6666 Acked Read 5 Sent Done Done false
20766 6667 Acked Read 7 Sent Done Done false
20767 6668 Acked Read 10 Sent Done Done false
20768 6669 Acked Read 8 Sent Done Done false
20769 6670 Acked Write 7 Sent Done Done false
20770 6671 Acked Write 1 Sent Done Done false
20771 6672 Acked Read 2 Sent Done Done false
20772 6673 Acked Read 8 Sent Done Done false
20773 6674 Acked Read 6 Sent Done Done false
20774 6675 Acked Read 2 Sent Done Done false
20775 6676 Acked Read 2 Sent Done Done false
20776 6677 Acked Read 7 Sent Done Done false
20777 6678 Acked Read 2 Sent Done Done false
20778 6679 Acked Write 6 Sent Done Done false
20779 6680 Acked Write 10 Sent Done Done false
20780 6681 Acked Read 10 Sent Done Done false
20781 6682 Acked Write 7 Sent Done Done false
20782 6683 Acked Write 2 Sent Done Done false
20783 6684 Acked Write 1 Sent Done Done false
20784 6685 Acked Read 8 Sent Done Done false
20785 6686 Acked Read 1 Sent Done Done false
20786 6687 Acked Write 6 Sent Done Done false
20787 6688 Acked Read 6 Sent Done Done false
20788 6689 Acked Write 3 Sent Done Done false
20789 6690 Acked Flush 0 Sent Done Done false
20790 6691 Acked Read 10 Sent Done Done false
20791 6692 Acked Write 5 Sent Done Done false
20792 6693 Acked Read 2 Sent Done Done false
20793 6694 Acked Flush 0 Sent Done Done false
20794 6695 Acked Write 7 Sent Done Done false
20795 6696 Acked Flush 0 Sent Done Done false
20796 6697 Acked Read 10 Sent Done Done false
20797 6698 Acked Read 7 Sent Done Done false
20798 6699 Acked Read 6 Sent Done Done false
20799 6700 Acked Read 6 Sent Done Done false
20800 6701 Acked Read 8 Sent Done Done false
20801 6702 Acked Write 6 Sent Done Done false
20802 6703 Acked Read 6 Sent Done Done false
20803 6704 Acked Read 10 Sent Done Done false
20804 6705 Acked Read 7 Sent Done Done false
20805 6706 Acked Write 6 Sent Done Done false
20806 6707 Acked Read 5 Sent Done Done false
20807 6708 Acked Write 6 Sent Done Done false
20808 6709 Acked Write 5 Sent Done Done false
20809 6710 Acked Write 7 Sent Done Done false
20810 6711 Acked Read 9 Sent Done Done false
20811 6712 Acked Read 8 Sent Done Done false
20812 6713 Acked Read 9 Sent Done Done false
20813 6714 Acked Read 7 Sent Done Done false
20814 6715 Acked Read 10 Sent Done Done false
20815 6716 Acked Write 7 Sent Done Done false
20816 6717 Acked Write 7 Sent Done Done false
20817 6718 Acked Read 10 Sent Done Done false
20818 6719 Acked Flush 0 Sent Done Done false
20819 6720 Acked Flush 0 Sent Done Done false
20820 6721 Acked Write 8 Sent Done Done false
20821 6722 Acked Read 7 Sent Done Done false
20822 6723 Acked Flush 0 Sent Done Done false
20823 6724 Acked Read 1 Sent Done Done false
20824 6725 Acked Read 3 Sent Done Done false
20825 6726 Acked Write 1 Sent Done Done false
20826 6727 Acked Write 2 Sent Done Done false
20827 6728 Acked Write 3 Sent Done Done false
20828 6729 Acked Read 8 Sent Done Done false
20829 6730 Acked Read 1 Sent Done Done false
20830 6731 Acked Flush 0 Sent Done Done false
20831 6732 Acked Read 3 Sent Done Done false
20832 6733 Acked Read 7 Sent Done Done false
20833 STATES DS:0 DS:1 DS:2 TOTAL
20834 Sent 301 0 0 301
20835 Done 0 301 301 602
20836 Skipped 0 0 0 0
20837 Error 0 0 0 0
20838 Last Flush: 6432 6731 6731
20839 Downstairs last five completed: 6432 6431 6430 6429 6428
20840 Upstairs last five completed: 6733 6732 6731 6730 6729
20841 CLIENT: Up:0 ds:301 act:2
20842 ----------------------------------------------------------------
20843 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
20844 States: Offline Active Active
20845 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
20846 6433 Acked Write 3 Sent Done Done false
20847 6434 Acked Read 7 Sent Done Done false
20848 6435 Acked Read 6 Sent Done Done false
20849 6436 Acked Read 3 Sent Done Done false
20850 6437 Acked Flush 0 Sent Done Done false
20851 6438 Acked Read 10 Sent Done Done false
20852 6439 Acked Read 5 Sent Done Done false
20853 6440 Acked Flush 0 Sent Done Done false
20854 6441 Acked Read 5 Sent Done Done false
20855 6442 Acked Read 5 Sent Done Done false
20856 6443 Acked Write 3 Sent Done Done false
20857 6444 Acked Read 9 Sent Done Done false
20858 6445 Acked Read 10 Sent Done Done false
20859 6446 Acked Write 1 Sent Done Done false
20860 6447 Acked Read 9 Sent Done Done false
20861 6448 Acked Write 3 Sent Done Done false
20862 6449 Acked Read 6 Sent Done Done false
20863 6450 Acked Read 10 Sent Done Done false
20864 6451 Acked Write 1 Sent Done Done false
20865 6452 Acked Read 2 Sent Done Done false
20866 6453 Acked Read 4 Sent Done Done false
20867 6454 Acked Flush 0 Sent Done Done false
20868 6455 Acked Write 1 Sent Done Done false
20869 6456 Acked Read 8 Sent Done Done false
20870 6457 Acked Read 8 Sent Done Done false
20871 6458 Acked Read 9 Sent Done Done false
20872 6459 Acked Write 8 Sent Done Done false
20873 6460 Acked Read 2 Sent Done Done false
20874 6461 Acked Read 9 Sent Done Done false
20875 6462 Acked Write 10 Sent Done Done false
20876 6463 Acked Write 10 Sent Done Done false
20877 6464 Acked Flush 0 Sent Done Done false
20878 6465 Acked Read 10 Sent Done Done false
20879 6466 Acked Write 3 Sent Done Done false
20880 6467 Acked Read 6 Sent Done Done false
20881 6468 Acked Read 10 Sent Done Done false
20882 6469 Acked Write 6 Sent Done Done false
20883 6470 Acked Read 10 Sent Done Done false
20884 6471 Acked Read 1 Sent Done Done false
20885 6472 Acked Read 1 Sent Done Done false
20886 6473 Acked Read 9 Sent Done Done false
20887 6474 Acked Read 5 Sent Done Done false
20888 6475 Acked Write 7 Sent Done Done false
20889 6476 Acked Read 2 Sent Done Done false
20890 6477 Acked Read 8 Sent Done Done false
20891 6478 Acked Read 7 Sent Done Done false
20892 6479 Acked Read 4 Sent Done Done false
20893 6480 Acked Read 8 Sent Done Done false
20894 6481 Acked Write 6 Sent Done Done false
20895 6482 Acked Read 1 Sent Done Done false
20896 6483 Acked Read 10 Sent Done Done false
20897 6484 Acked Read 4 Sent Done Done false
20898 6485 Acked Read 7 Sent Done Done false
20899 6486 Acked Read 6 Sent Done Done false
20900 6487 Acked Flush 0 Sent Done Done false
20901 6488 Acked Read 2 Sent Done Done false
20902 6489 Acked Flush 0 Sent Done Done false
20903 6490 Acked Read 2 Sent Done Done false
20904 6491 Acked Write 7 Sent Done Done false
20905 6492 Acked Write 5 Sent Done Done false
20906 6493 Acked Write 10 Sent Done Done false
20907 6494 Acked Read 7 Sent Done Done false
20908 6495 Acked Write 3 Sent Done Done false
20909 6496 Acked Read 6 Sent Done Done false
20910 6497 Acked Read 4 Sent Done Done false
20911 6498 Acked Write 7 Sent Done Done false
20912 6499 Acked Read 5 Sent Done Done false
20913 6500 Acked Read 6 Sent Done Done false
20914 6501 Acked Write 3 Sent Done Done false
20915 6502 Acked Write 1 Sent Done Done false
20916 6503 Acked Read 10 Sent Done Done false
20917 6504 Acked Read 2 Sent Done Done false
20918 6505 Acked Write 2 Sent Done Done false
20919 6506 Acked Read 10 Sent Done Done false
20920 6507 Acked Write 9 Sent Done Done false
20921 6508 Acked Read 1 Sent Done Done false
20922 6509 Acked Write 1 Sent Done Done false
20923 6510 Acked Flush 0 Sent Done Done false
20924 6511 Acked Read 7 Sent Done Done false
20925 6512 Acked Read 4 Sent Done Done false
20926 6513 Acked Flush 0 Sent Done Done false
20927 6514 Acked Read 8 Sent Done Done false
20928 6515 Acked Write 3 Sent Done Done false
20929 6516 Acked Read 6 Sent Done Done false
20930 6517 Acked Flush 0 Sent Done Done false
20931 6518 Acked Read 2 Sent Done Done false
20932 6519 Acked Read 3 Sent Done Done false
20933 6520 Acked Read 2 Sent Done Done false
20934 6521 Acked Flush 0 Sent Done Done false
20935 6522 Acked Write 4 Sent Done Done false
20936 6523 Acked Read 7 Sent Done Done false
20937 6524 Acked Write 4 Sent Done Done false
20938 6525 Acked Read 6 Sent Done Done false
20939 6526 Acked Read 1 Sent Done Done false
20940 6527 Acked Write 3 Sent Done Done false
20941 6528 Acked Read 2 Sent Done Done false
20942 6529 Acked Read 8 Sent Done Done false
20943 6530 Acked Read 9 Sent Done Done false
20944 6531 Acked Write 7 Sent Done Done false
20945 6532 Acked Read 1 Sent Done Done false
20946 6533 Acked Write 8 Sent Done Done false
20947 6534 Acked Flush 0 Sent Done Done false
20948 6535 Acked Flush 0 Sent Done Done false
20949 6536 Acked Write 9 Sent Done Done false
20950 6537 Acked Read 10 Sent Done Done false
20951 6538 Acked Read 2 Sent Done Done false
20952 6539 Acked Read 3 Sent Done Done false
20953 6540 Acked Read 9 Sent Done Done false
20954 6541 Acked Read 9 Sent Done Done false
20955 6542 Acked Flush 0 Sent Done Done false
20956 6543 Acked Write 2 Sent Done Done false
20957 6544 Acked Read 5 Sent Done Done false
20958 6545 Acked Read 7 Sent Done Done false
20959 6546 Acked Read 6 Sent Done Done false
20960 6547 Acked Write 6 Sent Done Done false
20961 6548 Acked Write 9 Sent Done Done false
20962 6549 Acked Write 5 Sent Done Done false
20963 6550 Acked Flush 0 Sent Done Done false
20964 6551 Acked Flush 0 Sent Done Done false
20965 6552 Acked Read 8 Sent Done Done false
20966 6553 Acked Read 7 Sent Done Done false
20967 6554 Acked Read 6 Sent Done Done false
20968 6555 Acked Read 5 Sent Done Done false
20969 6556 Acked Write 4 Sent Done Done false
20970 6557 Acked Flush 0 Sent Done Done false
20971 6558 Acked Read 7 Sent Done Done false
20972 6559 Acked Write 2 Sent Done Done false
20973 6560 Acked Read 1 Sent Done Done false
20974 6561 Acked Read 8 Sent Done Done false
20975 6562 Acked Read 9 Sent Done Done false
20976 6563 Acked Write 10 Sent Done Done false
20977 6564 Acked Write 3 Sent Done Done false
20978 6565 Acked Read 5 Sent Done Done false
20979 6566 Acked Flush 0 Sent Done Done false
20980 6567 Acked Read 2 Sent Done Done false
20981 6568 Acked Write 1 Sent Done Done false
20982 6569 Acked Read 3 Sent Done Done false
20983 6570 Acked Write 10 Sent Done Done false
20984 6571 Acked Write 6 Sent Done Done false
20985 6572 Acked Read 10 Sent Done Done false
20986 6573 Acked Read 5 Sent Done Done false
20987 6574 Acked Read 5 Sent Done Done false
20988 6575 Acked Read 3 Sent Done Done false
20989 6576 Acked Write 3 Sent Done Done false
20990 6577 Acked Read 10 Sent Done Done false
20991 6578 Acked Read 8 Sent Done Done false
20992 6579 Acked Flush 0 Sent Done Done false
20993 6580 Acked Write 4 Sent Done Done false
20994 6581 Acked Write 9 Sent Done Done false
20995 6582 Acked Read 3 Sent Done Done false
20996 6583 Acked Flush 0 Sent Done Done false
20997 6584 Acked Read 8 Sent Done Done false
20998 6585 Acked Read 2 Sent Done Done false
20999 6586 Acked Read 9 Sent Done Done false
21000 6587 Acked Write 5 Sent Done Done false
21001 6588 Acked Read 1 Sent Done Done false
21002 6589 Acked Read 4 Sent Done Done false
21003 6590 Acked Read 1 Sent Done Done false
21004 6591 Acked Read 4 Sent Done Done false
21005 6592 Acked Read 4 Sent Done Done false
21006 6593 Acked Read 7 Sent Done Done false
21007 6594 Acked Read 6 Sent Done Done false
21008 6595 Acked Write 9 Sent Done Done false
21009 6596 Acked Write 8 Sent Done Done false
21010 6597 Acked Write 7 Sent Done Done false
21011 6598 Acked Read 10 Sent Done Done false
21012 6599 Acked Read 6 Sent Done Done false
21013 6600 Acked Read 7 Sent Done Done false
21014 6601 Acked Read 7 Sent Done Done false
21015 6602 Acked Flush 0 Sent Done Done false
21016 6603 Acked Write 3 Sent Done Done false
21017 6604 Acked Write 10 Sent Done Done false
21018 6605 Acked Write 2 Sent Done Done false
21019 6606 Acked Write 4 Sent Done Done false
21020 6607 Acked Read 1 Sent Done Done false
21021 6608 Acked Write 9 Sent Done Done false
21022 6609 Acked Flush 0 Sent Done Done false
21023 6610 Acked Write 6 Sent Done Done false
21024 6611 Acked Flush 0 Sent Done Done false
21025 6612 Acked Read 5 Sent Done Done false
21026 6613 Acked Read 2 Sent Done Done false
21027 6614 Acked Read 5 Sent Done Done false
21028 6615 Acked Write 10 Sent Done Done false
21029 6616 Acked Read 9 Sent Done Done false
21030 6617 Acked Write 9 Sent Done Done false
21031 6618 Acked Flush 0 Sent Done Done false
21032 6619 Acked Read 3 Sent Done Done false
21033 6620 Acked Read 6 Sent Done Done false
21034 6621 Acked Read 3 Sent Done Done false
21035 6622 Acked Write 4 Sent Done Done false
21036 6623 Acked Read 5 Sent Done Done false
21037 6624 Acked Write 10 Sent Done Done false
21038 6625 Acked Write 10 Sent Done Done false
21039 6626 Acked Write 5 Sent Done Done false
21040 6627 Acked Read 6 Sent Done Done false
21041 6628 Acked Flush 0 Sent Done Done false
21042 6629 Acked Read 1 Sent Done Done false
21043 6630 Acked Read 1 Sent Done Done false
21044 6631 Acked Read 2 Sent Done Done false
21045 6632 Acked Write 2 Sent Done Done false
21046 6633 Acked Flush 0 Sent Done Done false
21047 6634 Acked Read 2 Sent Done Done false
21048 6635 Acked Write 7 Sent Done Done false
21049 6636 Acked Write 1 Sent Done Done false
21050 6637 Acked Write 3 Sent Done Done false
21051 6638 Acked Read 10 Sent Done Done false
21052 6639 Acked Flush 0 Sent Done Done false
21053 6640 Acked Write 9 Sent Done Done false
21054 6641 Acked Read 1 Sent Done Done false
21055 6642 Acked Read 8 Sent Done Done false
21056 6643 Acked Write 8 Sent Done Done false
21057 6644 Acked Write 8 Sent Done Done false
21058 6645 Acked Flush 0 Sent Done Done false
21059 6646 Acked Read 2 Sent Done Done false
21060 6647 Acked Read 8 Sent Done Done false
21061 6648 Acked Read 1 Sent Done Done false
21062 6649 Acked Read 3 Sent Done Done false
21063 6650 Acked Write 3 Sent Done Done false
21064 6651 Acked Write 4 Sent Done Done false
21065 6652 Acked Write 9 Sent Done Done false
21066 6653 Acked Read 3 Sent Done Done false
21067 6654 Acked Read 5 Sent Done Done false
21068 6655 Acked Read 1 Sent Done Done false
21069 6656 Acked Write 3 Sent Done Done false
21070 6657 Acked Read 1 Sent Done Done false
21071 6658 Acked Read 9 Sent Done Done false
21072 6659 Acked Read 4 Sent Done Done false
21073 6660 Acked Write 2 Sent Done Done false
21074 6661 Acked Read 2 Sent Done Done false
21075 6662 Acked Write 3 Sent Done Done false
21076 6663 Acked Write 8 Sent Done Done false
21077 6664 Acked Read 2 Sent Done Done false
21078 6665 Acked Write 4 Sent Done Done false
21079 6666 Acked Read 5 Sent Done Done false
21080 6667 Acked Read 7 Sent Done Done false
21081 6668 Acked Read 10 Sent Done Done false
21082 6669 Acked Read 8 Sent Done Done false
21083 6670 Acked Write 7 Sent Done Done false
21084 6671 Acked Write 1 Sent Done Done false
21085 6672 Acked Read 2 Sent Done Done false
21086 6673 Acked Read 8 Sent Done Done false
21087 6674 Acked Read 6 Sent Done Done false
21088 6675 Acked Read 2 Sent Done Done false
21089 6676 Acked Read 2 Sent Done Done false
21090 6677 Acked Read 7 Sent Done Done false
21091 6678 Acked Read 2 Sent Done Done false
21092 6679 Acked Write 6 Sent Done Done false
21093 6680 Acked Write 10 Sent Done Done false
21094 6681 Acked Read 10 Sent Done Done false
21095 6682 Acked Write 7 Sent Done Done false
21096 6683 Acked Write 2 Sent Done Done false
21097 6684 Acked Write 1 Sent Done Done false
21098 6685 Acked Read 8 Sent Done Done false
21099 6686 Acked Read 1 Sent Done Done false
21100 6687 Acked Write 6 Sent Done Done false
21101 6688 Acked Read 6 Sent Done Done false
21102 6689 Acked Write 3 Sent Done Done false
21103 6690 Acked Flush 0 Sent Done Done false
21104 6691 Acked Read 10 Sent Done Done false
21105 6692 Acked Write 5 Sent Done Done false
21106 6693 Acked Read 2 Sent Done Done false
21107 6694 Acked Flush 0 Sent Done Done false
21108 6695 Acked Write 7 Sent Done Done false
21109 6696 Acked Flush 0 Sent Done Done false
21110 6697 Acked Read 10 Sent Done Done false
21111 6698 Acked Read 7 Sent Done Done false
21112 6699 Acked Read 6 Sent Done Done false
21113 6700 Acked Read 6 Sent Done Done false
21114 6701 Acked Read 8 Sent Done Done false
21115 6702 Acked Write 6 Sent Done Done false
21116 6703 Acked Read 6 Sent Done Done false
21117 6704 Acked Read 10 Sent Done Done false
21118 6705 Acked Read 7 Sent Done Done false
21119 6706 Acked Write 6 Sent Done Done false
21120 6707 Acked Read 5 Sent Done Done false
21121 6708 Acked Write 6 Sent Done Done false
21122 6709 Acked Write 5 Sent Done Done false
21123 6710 Acked Write 7 Sent Done Done false
21124 6711 Acked Read 9 Sent Done Done false
21125 6712 Acked Read 8 Sent Done Done false
21126 6713 Acked Read 9 Sent Done Done false
21127 6714 Acked Read 7 Sent Done Done false
21128 6715 Acked Read 10 Sent Done Done false
21129 6716 Acked Write 7 Sent Done Done false
21130 6717 Acked Write 7 Sent Done Done false
21131 6718 Acked Read 10 Sent Done Done false
21132 6719 Acked Flush 0 Sent Done Done false
21133 6720 Acked Flush 0 Sent Done Done false
21134 6721 Acked Write 8 Sent Done Done false
21135 6722 Acked Read 7 Sent Done Done false
21136 6723 Acked Flush 0 Sent Done Done false
21137 6724 Acked Read 1 Sent Done Done false
21138 6725 Acked Read 3 Sent Done Done false
21139 6726 Acked Write 1 Sent Done Done false
21140 6727 Acked Write 2 Sent Done Done false
21141 6728 Acked Write 3 Sent Done Done false
21142 6729 Acked Read 8 Sent Done Done false
21143 6730 Acked Read 1 Sent Done Done false
21144 6731 Acked Flush 0 Sent Done Done false
21145 6732 Acked Read 3 Sent Done Done false
21146 6733 Acked Read 7 Sent Done Done false
21147 6734 Acked Flush 0 Sent Done Done false
21148 STATES DS:0 DS:1 DS:2 TOTAL
21149 Sent 302 0 0 302
21150 Done 0 302 302 604
21151 Skipped 0 0 0 0
21152 Error 0 0 0 0
21153 Last Flush: 6432 6734 6734
21154 Downstairs last five completed: 6432 6431 6430 6429 6428
21155 Upstairs last five completed: 6734 6733 6732 6731 6730
21156 CLIENT: Up:0 ds:302 act:2
211572025-01-27T19:40:47.374ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
211582025-01-27T19:40:47.374ZWARNcrucible: ds_connection connect to 127.0.0.1:8810 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
211592025-01-27T19:40:47.374ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
211602025-01-27T19:40:47.374ZWARNcrucible: downstairs task for 0 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
211612025-01-27T19:40:47.374ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21162 [8810] got start action
21163 [8810] I am going to restart
21164 Starting downstairs at port 8810
21165 Make output file at "/tmp/dsc/downstairs-8810.txt"
21166 [8810][0] reports Starting
21167 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1219)
21168 [8810][0] reports Running
21169 ----------------------------------------------------------------
21170 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21171 States: Offline Active Active
21172 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21173 6433 Acked Write 3 Sent Done Done false
21174 6434 Acked Read 7 Sent Done Done false
21175 6435 Acked Read 6 Sent Done Done false
21176 6436 Acked Read 3 Sent Done Done false
21177 6437 Acked Flush 0 Sent Done Done false
21178 6438 Acked Read 10 Sent Done Done false
21179 6439 Acked Read 5 Sent Done Done false
21180 6440 Acked Flush 0 Sent Done Done false
21181 6441 Acked Read 5 Sent Done Done false
21182 6442 Acked Read 5 Sent Done Done false
21183 6443 Acked Write 3 Sent Done Done false
21184 6444 Acked Read 9 Sent Done Done false
21185 6445 Acked Read 10 Sent Done Done false
21186 6446 Acked Write 1 Sent Done Done false
21187 6447 Acked Read 9 Sent Done Done false
21188 6448 Acked Write 3 Sent Done Done false
21189 6449 Acked Read 6 Sent Done Done false
21190 6450 Acked Read 10 Sent Done Done false
21191 6451 Acked Write 1 Sent Done Done false
21192 6452 Acked Read 2 Sent Done Done false
21193 6453 Acked Read 4 Sent Done Done false
21194 6454 Acked Flush 0 Sent Done Done false
21195 6455 Acked Write 1 Sent Done Done false
21196 6456 Acked Read 8 Sent Done Done false
21197 6457 Acked Read 8 Sent Done Done false
21198 6458 Acked Read 9 Sent Done Done false
21199 6459 Acked Write 8 Sent Done Done false
21200 6460 Acked Read 2 Sent Done Done false
21201 6461 Acked Read 9 Sent Done Done false
21202 6462 Acked Write 10 Sent Done Done false
21203 6463 Acked Write 10 Sent Done Done false
21204 6464 Acked Flush 0 Sent Done Done false
21205 6465 Acked Read 10 Sent Done Done false
21206 6466 Acked Write 3 Sent Done Done false
21207 6467 Acked Read 6 Sent Done Done false
21208 6468 Acked Read 10 Sent Done Done false
21209 6469 Acked Write 6 Sent Done Done false
21210 6470 Acked Read 10 Sent Done Done false
21211 6471 Acked Read 1 Sent Done Done false
21212 6472 Acked Read 1 Sent Done Done false
21213 6473 Acked Read 9 Sent Done Done false
21214 6474 Acked Read 5 Sent Done Done false
21215 6475 Acked Write 7 Sent Done Done false
21216 6476 Acked Read 2 Sent Done Done false
21217 6477 Acked Read 8 Sent Done Done false
21218 6478 Acked Read 7 Sent Done Done false
21219 6479 Acked Read 4 Sent Done Done false
21220 6480 Acked Read 8 Sent Done Done false
21221 6481 Acked Write 6 Sent Done Done false
21222 6482 Acked Read 1 Sent Done Done false
21223 6483 Acked Read 10 Sent Done Done false
21224 6484 Acked Read 4 Sent Done Done false
21225 6485 Acked Read 7 Sent Done Done false
21226 6486 Acked Read 6 Sent Done Done false
21227 6487 Acked Flush 0 Sent Done Done false
21228 6488 Acked Read 2 Sent Done Done false
21229 6489 Acked Flush 0 Sent Done Done false
21230 6490 Acked Read 2 Sent Done Done false
21231 6491 Acked Write 7 Sent Done Done false
21232 6492 Acked Write 5 Sent Done Done false
21233 6493 Acked Write 10 Sent Done Done false
21234 6494 Acked Read 7 Sent Done Done false
21235 6495 Acked Write 3 Sent Done Done false
21236 6496 Acked Read 6 Sent Done Done false
21237 6497 Acked Read 4 Sent Done Done false
21238 6498 Acked Write 7 Sent Done Done false
21239 6499 Acked Read 5 Sent Done Done false
21240 6500 Acked Read 6 Sent Done Done false
21241 6501 Acked Write 3 Sent Done Done false
21242 6502 Acked Write 1 Sent Done Done false
21243 6503 Acked Read 10 Sent Done Done false
21244 6504 Acked Read 2 Sent Done Done false
21245 6505 Acked Write 2 Sent Done Done false
21246 6506 Acked Read 10 Sent Done Done false
21247 6507 Acked Write 9 Sent Done Done false
21248 6508 Acked Read 1 Sent Done Done false
21249 6509 Acked Write 1 Sent Done Done false
21250 6510 Acked Flush 0 Sent Done Done false
21251 6511 Acked Read 7 Sent Done Done false
21252 6512 Acked Read 4 Sent Done Done false
21253 6513 Acked Flush 0 Sent Done Done false
21254 6514 Acked Read 8 Sent Done Done false
21255 6515 Acked Write 3 Sent Done Done false
21256 6516 Acked Read 6 Sent Done Done false
21257 6517 Acked Flush 0 Sent Done Done false
21258 6518 Acked Read 2 Sent Done Done false
21259 6519 Acked Read 3 Sent Done Done false
21260 6520 Acked Read 2 Sent Done Done false
21261 6521 Acked Flush 0 Sent Done Done false
21262 6522 Acked Write 4 Sent Done Done false
21263 6523 Acked Read 7 Sent Done Done false
21264 6524 Acked Write 4 Sent Done Done false
21265 6525 Acked Read 6 Sent Done Done false
21266 6526 Acked Read 1 Sent Done Done false
21267 6527 Acked Write 3 Sent Done Done false
21268 6528 Acked Read 2 Sent Done Done false
21269 6529 Acked Read 8 Sent Done Done false
21270 6530 Acked Read 9 Sent Done Done false
21271 6531 Acked Write 7 Sent Done Done false
21272 6532 Acked Read 1 Sent Done Done false
21273 6533 Acked Write 8 Sent Done Done false
21274 6534 Acked Flush 0 Sent Done Done false
21275 6535 Acked Flush 0 Sent Done Done false
21276 6536 Acked Write 9 Sent Done Done false
21277 6537 Acked Read 10 Sent Done Done false
21278 6538 Acked Read 2 Sent Done Done false
21279 6539 Acked Read 3 Sent Done Done false
21280 6540 Acked Read 9 Sent Done Done false
21281 6541 Acked Read 9 Sent Done Done false
21282 6542 Acked Flush 0 Sent Done Done false
21283 6543 Acked Write 2 Sent Done Done false
21284 6544 Acked Read 5 Sent Done Done false
21285 6545 Acked Read 7 Sent Done Done false
21286 6546 Acked Read 6 Sent Done Done false
21287 6547 Acked Write 6 Sent Done Done false
21288 6548 Acked Write 9 Sent Done Done false
21289 6549 Acked Write 5 Sent Done Done false
21290 6550 Acked Flush 0 Sent Done Done false
21291 6551 Acked Flush 0 Sent Done Done false
21292 6552 Acked Read 8 Sent Done Done false
21293 6553 Acked Read 7 Sent Done Done false
21294 6554 Acked Read 6 Sent Done Done false
21295 6555 Acked Read 5 Sent Done Done false
21296 6556 Acked Write 4 Sent Done Done false
21297 6557 Acked Flush 0 Sent Done Done false
21298 6558 Acked Read 7 Sent Done Done false
21299 6559 Acked Write 2 Sent Done Done false
21300 6560 Acked Read 1 Sent Done Done false
21301 6561 Acked Read 8 Sent Done Done false
21302 6562 Acked Read 9 Sent Done Done false
21303 6563 Acked Write 10 Sent Done Done false
21304 6564 Acked Write 3 Sent Done Done false
21305 6565 Acked Read 5 Sent Done Done false
21306 6566 Acked Flush 0 Sent Done Done false
21307 6567 Acked Read 2 Sent Done Done false
21308 6568 Acked Write 1 Sent Done Done false
21309 6569 Acked Read 3 Sent Done Done false
21310 6570 Acked Write 10 Sent Done Done false
21311 6571 Acked Write 6 Sent Done Done false
21312 6572 Acked Read 10 Sent Done Done false
21313 6573 Acked Read 5 Sent Done Done false
21314 6574 Acked Read 5 Sent Done Done false
21315 6575 Acked Read 3 Sent Done Done false
21316 6576 Acked Write 3 Sent Done Done false
21317 6577 Acked Read 10 Sent Done Done false
21318 6578 Acked Read 8 Sent Done Done false
21319 6579 Acked Flush 0 Sent Done Done false
21320 6580 Acked Write 4 Sent Done Done false
21321 6581 Acked Write 9 Sent Done Done false
21322 6582 Acked Read 3 Sent Done Done false
21323 6583 Acked Flush 0 Sent Done Done false
21324 6584 Acked Read 8 Sent Done Done false
21325 6585 Acked Read 2 Sent Done Done false
21326 6586 Acked Read 9 Sent Done Done false
21327 6587 Acked Write 5 Sent Done Done false
21328 6588 Acked Read 1 Sent Done Done false
21329 6589 Acked Read 4 Sent Done Done false
21330 6590 Acked Read 1 Sent Done Done false
21331 6591 Acked Read 4 Sent Done Done false
21332 6592 Acked Read 4 Sent Done Done false
21333 6593 Acked Read 7 Sent Done Done false
21334 6594 Acked Read 6 Sent Done Done false
21335 6595 Acked Write 9 Sent Done Done false
21336 6596 Acked Write 8 Sent Done Done false
21337 6597 Acked Write 7 Sent Done Done false
21338 6598 Acked Read 10 Sent Done Done false
21339 6599 Acked Read 6 Sent Done Done false
21340 6600 Acked Read 7 Sent Done Done false
21341 6601 Acked Read 7 Sent Done Done false
21342 6602 Acked Flush 0 Sent Done Done false
21343 6603 Acked Write 3 Sent Done Done false
21344 6604 Acked Write 10 Sent Done Done false
21345 6605 Acked Write 2 Sent Done Done false
21346 6606 Acked Write 4 Sent Done Done false
21347 6607 Acked Read 1 Sent Done Done false
21348 6608 Acked Write 9 Sent Done Done false
21349 6609 Acked Flush 0 Sent Done Done false
21350 6610 Acked Write 6 Sent Done Done false
21351 6611 Acked Flush 0 Sent Done Done false
21352 6612 Acked Read 5 Sent Done Done false
21353 6613 Acked Read 2 Sent Done Done false
21354 6614 Acked Read 5 Sent Done Done false
21355 6615 Acked Write 10 Sent Done Done false
21356 6616 Acked Read 9 Sent Done Done false
21357 6617 Acked Write 9 Sent Done Done false
21358 6618 Acked Flush 0 Sent Done Done false
21359 6619 Acked Read 3 Sent Done Done false
21360 6620 Acked Read 6 Sent Done Done false
21361 6621 Acked Read 3 Sent Done Done false
21362 6622 Acked Write 4 Sent Done Done false
21363 6623 Acked Read 5 Sent Done Done false
21364 6624 Acked Write 10 Sent Done Done false
21365 6625 Acked Write 10 Sent Done Done false
21366 6626 Acked Write 5 Sent Done Done false
21367 6627 Acked Read 6 Sent Done Done false
21368 6628 Acked Flush 0 Sent Done Done false
21369 6629 Acked Read 1 Sent Done Done false
21370 6630 Acked Read 1 Sent Done Done false
21371 6631 Acked Read 2 Sent Done Done false
21372 6632 Acked Write 2 Sent Done Done false
21373 6633 Acked Flush 0 Sent Done Done false
21374 6634 Acked Read 2 Sent Done Done false
21375 6635 Acked Write 7 Sent Done Done false
21376 6636 Acked Write 1 Sent Done Done false
21377 6637 Acked Write 3 Sent Done Done false
21378 6638 Acked Read 10 Sent Done Done false
21379 6639 Acked Flush 0 Sent Done Done false
21380 6640 Acked Write 9 Sent Done Done false
21381 6641 Acked Read 1 Sent Done Done false
21382 6642 Acked Read 8 Sent Done Done false
21383 6643 Acked Write 8 Sent Done Done false
21384 6644 Acked Write 8 Sent Done Done false
21385 6645 Acked Flush 0 Sent Done Done false
21386 6646 Acked Read 2 Sent Done Done false
21387 6647 Acked Read 8 Sent Done Done false
21388 6648 Acked Read 1 Sent Done Done false
21389 6649 Acked Read 3 Sent Done Done false
21390 6650 Acked Write 3 Sent Done Done false
21391 6651 Acked Write 4 Sent Done Done false
21392 6652 Acked Write 9 Sent Done Done false
21393 6653 Acked Read 3 Sent Done Done false
21394 6654 Acked Read 5 Sent Done Done false
21395 6655 Acked Read 1 Sent Done Done false
21396 6656 Acked Write 3 Sent Done Done false
21397 6657 Acked Read 1 Sent Done Done false
21398 6658 Acked Read 9 Sent Done Done false
21399 6659 Acked Read 4 Sent Done Done false
21400 6660 Acked Write 2 Sent Done Done false
21401 6661 Acked Read 2 Sent Done Done false
21402 6662 Acked Write 3 Sent Done Done false
21403 6663 Acked Write 8 Sent Done Done false
21404 6664 Acked Read 2 Sent Done Done false
21405 6665 Acked Write 4 Sent Done Done false
21406 6666 Acked Read 5 Sent Done Done false
21407 6667 Acked Read 7 Sent Done Done false
21408 6668 Acked Read 10 Sent Done Done false
21409 6669 Acked Read 8 Sent Done Done false
21410 6670 Acked Write 7 Sent Done Done false
21411 6671 Acked Write 1 Sent Done Done false
21412 6672 Acked Read 2 Sent Done Done false
21413 6673 Acked Read 8 Sent Done Done false
21414 6674 Acked Read 6 Sent Done Done false
21415 6675 Acked Read 2 Sent Done Done false
21416 6676 Acked Read 2 Sent Done Done false
21417 6677 Acked Read 7 Sent Done Done false
21418 6678 Acked Read 2 Sent Done Done false
21419 6679 Acked Write 6 Sent Done Done false
21420 6680 Acked Write 10 Sent Done Done false
21421 6681 Acked Read 10 Sent Done Done false
21422 6682 Acked Write 7 Sent Done Done false
21423 6683 Acked Write 2 Sent Done Done false
21424 6684 Acked Write 1 Sent Done Done false
21425 6685 Acked Read 8 Sent Done Done false
21426 6686 Acked Read 1 Sent Done Done false
21427 6687 Acked Write 6 Sent Done Done false
21428 6688 Acked Read 6 Sent Done Done false
21429 6689 Acked Write 3 Sent Done Done false
21430 6690 Acked Flush 0 Sent Done Done false
21431 6691 Acked Read 10 Sent Done Done false
21432 6692 Acked Write 5 Sent Done Done false
21433 6693 Acked Read 2 Sent Done Done false
21434 6694 Acked Flush 0 Sent Done Done false
21435 6695 Acked Write 7 Sent Done Done false
21436 6696 Acked Flush 0 Sent Done Done false
21437 6697 Acked Read 10 Sent Done Done false
21438 6698 Acked Read 7 Sent Done Done false
21439 6699 Acked Read 6 Sent Done Done false
21440 6700 Acked Read 6 Sent Done Done false
21441 6701 Acked Read 8 Sent Done Done false
21442 6702 Acked Write 6 Sent Done Done false
21443 6703 Acked Read 6 Sent Done Done false
21444 6704 Acked Read 10 Sent Done Done false
21445 6705 Acked Read 7 Sent Done Done false
21446 6706 Acked Write 6 Sent Done Done false
21447 6707 Acked Read 5 Sent Done Done false
21448 6708 Acked Write 6 Sent Done Done false
21449 6709 Acked Write 5 Sent Done Done false
21450 6710 Acked Write 7 Sent Done Done false
21451 6711 Acked Read 9 Sent Done Done false
21452 6712 Acked Read 8 Sent Done Done false
21453 6713 Acked Read 9 Sent Done Done false
21454 6714 Acked Read 7 Sent Done Done false
21455 6715 Acked Read 10 Sent Done Done false
21456 6716 Acked Write 7 Sent Done Done false
21457 6717 Acked Write 7 Sent Done Done false
21458 6718 Acked Read 10 Sent Done Done false
21459 6719 Acked Flush 0 Sent Done Done false
21460 6720 Acked Flush 0 Sent Done Done false
21461 6721 Acked Write 8 Sent Done Done false
21462 6722 Acked Read 7 Sent Done Done false
21463 6723 Acked Flush 0 Sent Done Done false
21464 6724 Acked Read 1 Sent Done Done false
21465 6725 Acked Read 3 Sent Done Done false
21466 6726 Acked Write 1 Sent Done Done false
21467 6727 Acked Write 2 Sent Done Done false
21468 6728 Acked Write 3 Sent Done Done false
21469 6729 Acked Read 8 Sent Done Done false
21470 6730 Acked Read 1 Sent Done Done false
21471 6731 Acked Flush 0 Sent Done Done false
21472 6732 Acked Read 3 Sent Done Done false
21473 6733 Acked Read 7 Sent Done Done false
21474 6734 Acked Flush 0 Sent Done Done false
21475 STATES DS:0 DS:1 DS:2 TOTAL
21476 Sent 302 0 0 302
21477 Done 0 302 302 604
21478 Skipped 0 0 0 0
21479 Error 0 0 0 0
21480 Last Flush: 6432 6734 6734
21481 Downstairs last five completed: 6432 6431 6430 6429 6428
21482 Upstairs last five completed: 6734 6733 6732 6731 6730
21483 CLIENT: Up:0 ds:302 act:2
21484 ----------------------------------------------------------------
21485 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
21486 States: Offline Active Active
21487 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
21488 6433 Acked Write 3 Sent Done Done false
21489 6434 Acked Read 7 Sent Done Done false
21490 6435 Acked Read 6 Sent Done Done false
21491 6436 Acked Read 3 Sent Done Done false
21492 6437 Acked Flush 0 Sent Done Done false
21493 6438 Acked Read 10 Sent Done Done false
21494 6439 Acked Read 5 Sent Done Done false
21495 6440 Acked Flush 0 Sent Done Done false
21496 6441 Acked Read 5 Sent Done Done false
21497 6442 Acked Read 5 Sent Done Done false
21498 6443 Acked Write 3 Sent Done Done false
21499 6444 Acked Read 9 Sent Done Done false
21500 6445 Acked Read 10 Sent Done Done false
21501 6446 Acked Write 1 Sent Done Done false
21502 6447 Acked Read 9 Sent Done Done false
21503 6448 Acked Write 3 Sent Done Done false
21504 6449 Acked Read 6 Sent Done Done false
21505 6450 Acked Read 10 Sent Done Done false
21506 6451 Acked Write 1 Sent Done Done false
21507 6452 Acked Read 2 Sent Done Done false
21508 6453 Acked Read 4 Sent Done Done false
21509 6454 Acked Flush 0 Sent Done Done false
21510 6455 Acked Write 1 Sent Done Done false
21511 6456 Acked Read 8 Sent Done Done false
21512 6457 Acked Read 8 Sent Done Done false
21513 6458 Acked Read 9 Sent Done Done false
21514 6459 Acked Write 8 Sent Done Done false
21515 6460 Acked Read 2 Sent Done Done false
21516 6461 Acked Read 9 Sent Done Done false
21517 6462 Acked Write 10 Sent Done Done false
21518 6463 Acked Write 10 Sent Done Done false
21519 6464 Acked Flush 0 Sent Done Done false
21520 6465 Acked Read 10 Sent Done Done false
21521 6466 Acked Write 3 Sent Done Done false
21522 6467 Acked Read 6 Sent Done Done false
21523 6468 Acked Read 10 Sent Done Done false
21524 6469 Acked Write 6 Sent Done Done false
21525 6470 Acked Read 10 Sent Done Done false
21526 6471 Acked Read 1 Sent Done Done false
21527 6472 Acked Read 1 Sent Done Done false
21528 6473 Acked Read 9 Sent Done Done false
21529 6474 Acked Read 5 Sent Done Done false
21530 6475 Acked Write 7 Sent Done Done false
21531 6476 Acked Read 2 Sent Done Done false
21532 6477 Acked Read 8 Sent Done Done false
21533 6478 Acked Read 7 Sent Done Done false
21534 6479 Acked Read 4 Sent Done Done false
21535 6480 Acked Read 8 Sent Done Done false
21536 6481 Acked Write 6 Sent Done Done false
21537 6482 Acked Read 1 Sent Done Done false
21538 6483 Acked Read 10 Sent Done Done false
21539 6484 Acked Read 4 Sent Done Done false
21540 6485 Acked Read 7 Sent Done Done false
21541 6486 Acked Read 6 Sent Done Done false
21542 6487 Acked Flush 0 Sent Done Done false
21543 6488 Acked Read 2 Sent Done Done false
21544 6489 Acked Flush 0 Sent Done Done false
21545 6490 Acked Read 2 Sent Done Done false
21546 6491 Acked Write 7 Sent Done Done false
21547 6492 Acked Write 5 Sent Done Done false
21548 6493 Acked Write 10 Sent Done Done false
21549 6494 Acked Read 7 Sent Done Done false
21550 6495 Acked Write 3 Sent Done Done false
21551 6496 Acked Read 6 Sent Done Done false
21552 6497 Acked Read 4 Sent Done Done false
21553 6498 Acked Write 7 Sent Done Done false
21554 6499 Acked Read 5 Sent Done Done false
21555 6500 Acked Read 6 Sent Done Done false
21556 6501 Acked Write 3 Sent Done Done false
21557 6502 Acked Write 1 Sent Done Done false
21558 6503 Acked Read 10 Sent Done Done false
21559 6504 Acked Read 2 Sent Done Done false
21560 6505 Acked Write 2 Sent Done Done false
21561 6506 Acked Read 10 Sent Done Done false
21562 6507 Acked Write 9 Sent Done Done false
21563 6508 Acked Read 1 Sent Done Done false
21564 6509 Acked Write 1 Sent Done Done false
21565 6510 Acked Flush 0 Sent Done Done false
21566 6511 Acked Read 7 Sent Done Done false
21567 6512 Acked Read 4 Sent Done Done false
21568 6513 Acked Flush 0 Sent Done Done false
21569 6514 Acked Read 8 Sent Done Done false
21570 6515 Acked Write 3 Sent Done Done false
21571 6516 Acked Read 6 Sent Done Done false
21572 6517 Acked Flush 0 Sent Done Done false
21573 6518 Acked Read 2 Sent Done Done false
21574 6519 Acked Read 3 Sent Done Done false
21575 6520 Acked Read 2 Sent Done Done false
21576 6521 Acked Flush 0 Sent Done Done false
21577 6522 Acked Write 4 Sent Done Done false
21578 6523 Acked Read 7 Sent Done Done false
21579 6524 Acked Write 4 Sent Done Done false
21580 6525 Acked Read 6 Sent Done Done false
21581 6526 Acked Read 1 Sent Done Done false
21582 6527 Acked Write 3 Sent Done Done false
21583 6528 Acked Read 2 Sent Done Done false
21584 6529 Acked Read 8 Sent Done Done false
21585 6530 Acked Read 9 Sent Done Done false
21586 6531 Acked Write 7 Sent Done Done false
21587 6532 Acked Read 1 Sent Done Done false
21588 6533 Acked Write 8 Sent Done Done false
21589 6534 Acked Flush 0 Sent Done Done false
21590 6535 Acked Flush 0 Sent Done Done false
21591 6536 Acked Write 9 Sent Done Done false
21592 6537 Acked Read 10 Sent Done Done false
21593 6538 Acked Read 2 Sent Done Done false
21594 6539 Acked Read 3 Sent Done Done false
21595 6540 Acked Read 9 Sent Done Done false
21596 6541 Acked Read 9 Sent Done Done false
21597 6542 Acked Flush 0 Sent Done Done false
21598 6543 Acked Write 2 Sent Done Done false
21599 6544 Acked Read 5 Sent Done Done false
21600 6545 Acked Read 7 Sent Done Done false
21601 6546 Acked Read 6 Sent Done Done false
21602 6547 Acked Write 6 Sent Done Done false
21603 6548 Acked Write 9 Sent Done Done false
21604 6549 Acked Write 5 Sent Done Done false
21605 6550 Acked Flush 0 Sent Done Done false
21606 6551 Acked Flush 0 Sent Done Done false
21607 6552 Acked Read 8 Sent Done Done false
21608 6553 Acked Read 7 Sent Done Done false
21609 6554 Acked Read 6 Sent Done Done false
21610 6555 Acked Read 5 Sent Done Done false
21611 6556 Acked Write 4 Sent Done Done false
21612 6557 Acked Flush 0 Sent Done Done false
21613 6558 Acked Read 7 Sent Done Done false
21614 6559 Acked Write 2 Sent Done Done false
21615 6560 Acked Read 1 Sent Done Done false
21616 6561 Acked Read 8 Sent Done Done false
21617 6562 Acked Read 9 Sent Done Done false
21618 6563 Acked Write 10 Sent Done Done false
21619 6564 Acked Write 3 Sent Done Done false
21620 6565 Acked Read 5 Sent Done Done false
21621 6566 Acked Flush 0 Sent Done Done false
21622 6567 Acked Read 2 Sent Done Done false
21623 6568 Acked Write 1 Sent Done Done false
21624 6569 Acked Read 3 Sent Done Done false
21625 6570 Acked Write 10 Sent Done Done false
21626 6571 Acked Write 6 Sent Done Done false
21627 6572 Acked Read 10 Sent Done Done false
21628 6573 Acked Read 5 Sent Done Done false
21629 6574 Acked Read 5 Sent Done Done false
21630 6575 Acked Read 3 Sent Done Done false
21631 6576 Acked Write 3 Sent Done Done false
21632 6577 Acked Read 10 Sent Done Done false
21633 6578 Acked Read 8 Sent Done Done false
21634 6579 Acked Flush 0 Sent Done Done false
21635 6580 Acked Write 4 Sent Done Done false
21636 6581 Acked Write 9 Sent Done Done false
21637 6582 Acked Read 3 Sent Done Done false
21638 6583 Acked Flush 0 Sent Done Done false
21639 6584 Acked Read 8 Sent Done Done false
21640 6585 Acked Read 2 Sent Done Done false
21641 6586 Acked Read 9 Sent Done Done false
21642 6587 Acked Write 5 Sent Done Done false
21643 6588 Acked Read 1 Sent Done Done false
21644 6589 Acked Read 4 Sent Done Done false
21645 6590 Acked Read 1 Sent Done Done false
21646 6591 Acked Read 4 Sent Done Done false
21647 6592 Acked Read 4 Sent Done Done false
21648 6593 Acked Read 7 Sent Done Done false
21649 6594 Acked Read 6 Sent Done Done false
21650 6595 Acked Write 9 Sent Done Done false
21651 6596 Acked Write 8 Sent Done Done false
21652 6597 Acked Write 7 Sent Done Done false
21653 6598 Acked Read 10 Sent Done Done false
21654 6599 Acked Read 6 Sent Done Done false
21655 6600 Acked Read 7 Sent Done Done false
21656 6601 Acked Read 7 Sent Done Done false
21657 6602 Acked Flush 0 Sent Done Done false
21658 6603 Acked Write 3 Sent Done Done false
21659 6604 Acked Write 10 Sent Done Done false
21660 6605 Acked Write 2 Sent Done Done false
21661 6606 Acked Write 4 Sent Done Done false
21662 6607 Acked Read 1 Sent Done Done false
21663 6608 Acked Write 9 Sent Done Done false
21664 6609 Acked Flush 0 Sent Done Done false
21665 6610 Acked Write 6 Sent Done Done false
21666 6611 Acked Flush 0 Sent Done Done false
21667 6612 Acked Read 5 Sent Done Done false
21668 6613 Acked Read 2 Sent Done Done false
21669 6614 Acked Read 5 Sent Done Done false
21670 6615 Acked Write 10 Sent Done Done false
21671 6616 Acked Read 9 Sent Done Done false
21672 6617 Acked Write 9 Sent Done Done false
21673 6618 Acked Flush 0 Sent Done Done false
21674 6619 Acked Read 3 Sent Done Done false
21675 6620 Acked Read 6 Sent Done Done false
21676 6621 Acked Read 3 Sent Done Done false
21677 6622 Acked Write 4 Sent Done Done false
21678 6623 Acked Read 5 Sent Done Done false
21679 6624 Acked Write 10 Sent Done Done false
21680 6625 Acked Write 10 Sent Done Done false
21681 6626 Acked Write 5 Sent Done Done false
21682 6627 Acked Read 6 Sent Done Done false
21683 6628 Acked Flush 0 Sent Done Done false
21684 6629 Acked Read 1 Sent Done Done false
21685 6630 Acked Read 1 Sent Done Done false
21686 6631 Acked Read 2 Sent Done Done false
21687 6632 Acked Write 2 Sent Done Done false
21688 6633 Acked Flush 0 Sent Done Done false
21689 6634 Acked Read 2 Sent Done Done false
21690 6635 Acked Write 7 Sent Done Done false
21691 6636 Acked Write 1 Sent Done Done false
21692 6637 Acked Write 3 Sent Done Done false
21693 6638 Acked Read 10 Sent Done Done false
21694 6639 Acked Flush 0 Sent Done Done false
21695 6640 Acked Write 9 Sent Done Done false
21696 6641 Acked Read 1 Sent Done Done false
21697 6642 Acked Read 8 Sent Done Done false
21698 6643 Acked Write 8 Sent Done Done false
21699 6644 Acked Write 8 Sent Done Done false
21700 6645 Acked Flush 0 Sent Done Done false
21701 6646 Acked Read 2 Sent Done Done false
21702 6647 Acked Read 8 Sent Done Done false
21703 6648 Acked Read 1 Sent Done Done false
21704 6649 Acked Read 3 Sent Done Done false
21705 6650 Acked Write 3 Sent Done Done false
21706 6651 Acked Write 4 Sent Done Done false
21707 6652 Acked Write 9 Sent Done Done false
21708 6653 Acked Read 3 Sent Done Done false
21709 6654 Acked Read 5 Sent Done Done false
21710 6655 Acked Read 1 Sent Done Done false
21711 6656 Acked Write 3 Sent Done Done false
21712 6657 Acked Read 1 Sent Done Done false
21713 6658 Acked Read 9 Sent Done Done false
21714 6659 Acked Read 4 Sent Done Done false
21715 6660 Acked Write 2 Sent Done Done false
21716 6661 Acked Read 2 Sent Done Done false
21717 6662 Acked Write 3 Sent Done Done false
21718 6663 Acked Write 8 Sent Done Done false
21719 6664 Acked Read 2 Sent Done Done false
21720 6665 Acked Write 4 Sent Done Done false
21721 6666 Acked Read 5 Sent Done Done false
21722 6667 Acked Read 7 Sent Done Done false
21723 6668 Acked Read 10 Sent Done Done false
21724 6669 Acked Read 8 Sent Done Done false
21725 6670 Acked Write 7 Sent Done Done false
21726 6671 Acked Write 1 Sent Done Done false
21727 6672 Acked Read 2 Sent Done Done false
21728 6673 Acked Read 8 Sent Done Done false
21729 6674 Acked Read 6 Sent Done Done false
21730 6675 Acked Read 2 Sent Done Done false
21731 6676 Acked Read 2 Sent Done Done false
21732 6677 Acked Read 7 Sent Done Done false
21733 6678 Acked Read 2 Sent Done Done false
21734 6679 Acked Write 6 Sent Done Done false
21735 6680 Acked Write 10 Sent Done Done false
21736 6681 Acked Read 10 Sent Done Done false
21737 6682 Acked Write 7 Sent Done Done false
21738 6683 Acked Write 2 Sent Done Done false
21739 6684 Acked Write 1 Sent Done Done false
21740 6685 Acked Read 8 Sent Done Done false
21741 6686 Acked Read 1 Sent Done Done false
21742 6687 Acked Write 6 Sent Done Done false
21743 6688 Acked Read 6 Sent Done Done false
21744 6689 Acked Write 3 Sent Done Done false
21745 6690 Acked Flush 0 Sent Done Done false
21746 6691 Acked Read 10 Sent Done Done false
21747 6692 Acked Write 5 Sent Done Done false
21748 6693 Acked Read 2 Sent Done Done false
21749 6694 Acked Flush 0 Sent Done Done false
21750 6695 Acked Write 7 Sent Done Done false
21751 6696 Acked Flush 0 Sent Done Done false
21752 6697 Acked Read 10 Sent Done Done false
21753 6698 Acked Read 7 Sent Done Done false
21754 6699 Acked Read 6 Sent Done Done false
21755 6700 Acked Read 6 Sent Done Done false
21756 6701 Acked Read 8 Sent Done Done false
21757 6702 Acked Write 6 Sent Done Done false
21758 6703 Acked Read 6 Sent Done Done false
21759 6704 Acked Read 10 Sent Done Done false
21760 6705 Acked Read 7 Sent Done Done false
21761 6706 Acked Write 6 Sent Done Done false
21762 6707 Acked Read 5 Sent Done Done false
21763 6708 Acked Write 6 Sent Done Done false
21764 6709 Acked Write 5 Sent Done Done false
21765 6710 Acked Write 7 Sent Done Done false
21766 6711 Acked Read 9 Sent Done Done false
21767 6712 Acked Read 8 Sent Done Done false
21768 6713 Acked Read 9 Sent Done Done false
21769 6714 Acked Read 7 Sent Done Done false
21770 6715 Acked Read 10 Sent Done Done false
21771 6716 Acked Write 7 Sent Done Done false
21772 6717 Acked Write 7 Sent Done Done false
21773 6718 Acked Read 10 Sent Done Done false
21774 6719 Acked Flush 0 Sent Done Done false
21775 6720 Acked Flush 0 Sent Done Done false
21776 6721 Acked Write 8 Sent Done Done false
21777 6722 Acked Read 7 Sent Done Done false
21778 6723 Acked Flush 0 Sent Done Done false
21779 6724 Acked Read 1 Sent Done Done false
21780 6725 Acked Read 3 Sent Done Done false
21781 6726 Acked Write 1 Sent Done Done false
21782 6727 Acked Write 2 Sent Done Done false
21783 6728 Acked Write 3 Sent Done Done false
21784 6729 Acked Read 8 Sent Done Done false
21785 6730 Acked Read 1 Sent Done Done false
21786 6731 Acked Flush 0 Sent Done Done false
21787 6732 Acked Read 3 Sent Done Done false
21788 6733 Acked Read 7 Sent Done Done false
21789 6734 Acked Flush 0 Sent Done Done false
21790 STATES DS:0 DS:1 DS:2 TOTAL
21791 Sent 302 0 0 302
21792 Done 0 302 302 604
21793 Skipped 0 0 0 0
21794 Error 0 0 0 0
21795 Last Flush: 6432 6734 6734
21796 Downstairs last five completed: 6432 6431 6430 6429 6428
21797 Upstairs last five completed: 6734 6733 6732 6731 6730
21798 CLIENT: Up:0 ds:302 act:2
217992025-01-27T19:40:57.375ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218002025-01-27T19:40:57.375ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:65232) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218012025-01-27T19:40:57.376ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218022025-01-27T19:40:57.377ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218032025-01-27T19:40:57.377ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218042025-01-27T19:40:57.377ZINFOcrucible: send last flush ID to this DS: 6432 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218052025-01-27T19:40:57.377ZINFOcrucible: Replied this last flush ID: 6432 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218062025-01-27T19:40:57.377ZINFOcrucible: [0] client re-new 302 jobs since flush 6432 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218072025-01-27T19:40:57.377ZINFOcrucible: [0] Replayed 302 jobs since flush: 6432 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21808 ----------------------------------------------------------------
21809 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
21810 Downstairs last five completed: 6734 6733 6732 6731 6730
21811 Upstairs last five completed: 6734 6733 6732 6731 6730
21812 CLIENT: Up:0 ds:0 act:3
21813 Replay: All jobs finished
21814 Main task has work to do, go find it
21815 got dsc Stop(1)
21816 stop 1
21817 [8820] Got stop action so:false kr:false
21818 Jan 27 19:40:57.632 INFO request completed, latency_us: 579, response_code: 204, uri: /stop/cid/1, method: POST, req_id: e886db29-46ca-45f9-ac40-321efaa74c34, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
218192025-01-27T19:40:57.632ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218202025-01-27T19:40:57.633ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218212025-01-27T19:40:57.633ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
218222025-01-27T19:40:57.633ZINFOcrucible: client task is exiting = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
21823 Jan 27 19:40:57.633 INFO request completed, latency_us: 129, response_code: 200, uri: /state/cid/1, method: GET, req_id: 365e158b-774d-4aee-9f4e-bde42ca4abec, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
21824 [8820] Exited with: ExitStatus(unix_wait_status(9))
21825 [8820][1] reports Exit
21826 Jan 27 19:41:01.635 INFO request completed, latency_us: 252, response_code: 200, uri: /state/cid/1, method: GET, req_id: 97c49dd6-cbce-4c7a-a188-7d41318c02ae, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
21827 001/300 Read block 967 len 32768
21828 002/300 Write block 4822 len 20480 data: 4 4 4 4 4
21829 003/300 Read block 1459 len 36864
21830 004/300 Read block 1393 len 8192
21831 005/300 Read block 1681 len 24576
21832 006/300 Read block 2225 len 16384
21833 007/300 Read block 143 len 4096
21834 008/300 Read block 105 len 28672
21835 009/300 Flush
21836 010/300 Write block 878 len 40960 data: 4 4 3 3 3 2 2 3 3 3
21837 011/300 Read block 4800 len 4096
21838 012/300 Read block 3137 len 24576
21839 013/300 Read block 4826 len 36864
21840 014/300 Write block 2773 len 8192 data: 7 6
21841 015/300 Read block 4810 len 24576
21842 016/300 Flush
21843 017/300 Write block 3873 len 32768 data: 5 5 4 5 5 6 5 8
21844 018/300 Read block 879 len 16384
21845 019/300 Flush
21846 020/300 Flush
21847 021/300 Write block 1305 len 16384 data: 6 6 7 6
21848 022/300 Read block 1475 len 16384
21849 023/300 Write block 3581 len 28672 data: 3 3 3 2 2 2 2
21850 024/300 Write block 997 len 36864 data: 3 3 3 4 4 3 4 5 4
21851 025/300 Write block 2071 len 16384 data: 2 4 4 4
21852 026/300 Read block 1365 len 24576
21853 027/300 Write block 2246 len 20480 data: 7 8 7 5 5
21854 028/300 Write block 287 len 4096 data: 5
21855 029/300 Write block 4403 len 12288 data: 2 2 3
21856 030/300 Flush
21857 031/300 Read block 3186 len 28672
21858 032/300 Read block 1828 len 32768
21859 033/300 Write block 3131 len 20480 data: 6 6 6 5 6
21860 034/300 Read block 4474 len 16384
21861 035/300 Read block 2751 len 8192
21862 036/300 Read block 1172 len 40960
21863 037/300 Read block 1374 len 8192
21864 038/300 Write block 597 len 4096 data: 5
21865 039/300 Read block 2557 len 36864
21866 040/300 Write block 4299 len 4096 data: 3
21867 041/300 Write block 2964 len 20480 data: 5 5 5 5 5
21868 042/300 Flush
21869 043/300 Read block 2560 len 24576
21870 044/300 Write block 4644 len 24576 data: 4 3 4 4 3 4
21871 045/300 Read block 4485 len 16384
21872 046/300 Read block 3662 len 36864
21873 047/300 Flush
21874 048/300 Write block 4503 len 12288 data: 6 5 5
21875 049/300 Read block 4848 len 12288
21876 050/300 Write block 4588 len 28672 data: 3 3 3 3 3 3 4
21877 051/300 Read block 794 len 24576
21878 052/300 Write block 35 len 16384 data: 3 3 4 4
21879 053/300 Write block 351 len 4096 data: 5
21880 054/300 Write block 1879 len 40960 data: 5 3 3 3 5 5 5 7 6 7
21881 055/300 Read block 649 len 28672
21882 056/300 Write block 1915 len 28672 data: 2 2 2 2 2 3 3
21883 057/300 Read block 664 len 12288
21884 058/300 Write block 2023 len 40960 data: 5 4 3 5 5 5 5 5 4 5
21885 059/300 Read block 3958 len 12288
21886 060/300 Write block 3208 len 36864 data: 3 4 3 5 5 5 3 6 6
21887 061/300 Flush
21888 062/300 Write block 49 len 16384 data: 6 6 6 5
21889 063/300 Write block 2429 len 20480 data: 5 4 3 2 2
21890 064/300 Read block 1255 len 28672
21891 065/300 Read block 3971 len 4096
21892 066/300 Flush
21893 067/300 Read block 939 len 12288
21894 068/300 Write block 303 len 32768 data: 5 4 5 5 5 4 4 3
21895 069/300 Read block 3468 len 28672
21896 070/300 Read block 4537 len 24576
21897 071/300 Read block 3777 len 12288
21898 072/300 Flush
21899 073/300 Read block 478 len 40960
21900 074/300 Read block 3937 len 24576
21901 075/300 Read block 1251 len 8192
21902 076/300 Read block 3942 len 16384
21903 077/300 Write block 4591 len 28672 data: 4 4 4 5 4 4 4
21904 078/300 Read block 649 len 24576
21905 079/300 Flush
21906 080/300 Write block 675 len 28672 data: 4 5 5 5 4 3 4
21907 081/300 Read block 1120 len 24576
21908 082/300 Read block 494 len 16384
21909 083/300 Read block 4548 len 16384
21910 084/300 Read block 2555 len 16384
21911 085/300 Write block 145 len 4096 data: 3
21912 086/300 Write block 2436 len 36864 data: 2 5 5 5 6 8 6 6 6
21913 087/300 Read block 3430 len 16384
21914 088/300 Write block 2424 len 8192 data: 4 6
21915 089/300 Read block 692 len 20480
21916 090/300 Write block 2457 len 32768 data: 3 2 3 3 2 2 3 2
21917 091/300 Write block 786 len 24576 data: 3 3 3 4 4 4
21918 092/300 Read block 1201 len 40960
21919 093/300 Read block 3935 len 4096
21920 094/300 Write block 2825 len 8192 data: 4 4
21921 095/300 Read block 2283 len 20480
21922 096/300 Write block 4302 len 4096 data: 4
21923 097/300 Write block 2692 len 8192 data: 3 3
21924 098/300 Flush
21925 099/300 Read block 163 len 16384
21926 100/300 Read block 1454 len 40960
21927 101/300 Write block 1899 len 32768 data: 2 2 2 2 3 5 5 5
21928 102/300 Read block 3638 len 4096
21929 103/300 Flush
21930 104/300 Read block 3242 len 8192
21931 105/300 Flush
21932 106/300 Read block 3023 len 20480
21933 107/300 Write block 2788 len 12288 data: 5 5 5
21934 108/300 Write block 4446 len 24576 data: 5 4 4 4 4 4
21935 109/300 Read block 2063 len 12288
21936 110/300 Write block 501 len 32768 data: 8 7 4 3 5 5 4 3
21937 111/300 Read block 390 len 12288
21938 112/300 Read block 2085 len 4096
21939 113/300 Write block 1632 len 20480 data: 6 6 5 5 4
21940 114/300 Read block 101 len 32768
21941 115/300 Flush
21942 116/300 Read block 4054 len 28672
21943 117/300 Write block 2768 len 40960 data: 8 6 6 6 7 8 7 5 4 4
21944 118/300 Read block 2400 len 4096
21945 119/300 Write block 677 len 40960 data: 6 6 5 4 5 5 6 6 6 6
21946 120/300 Write block 1355 len 20480 data: 7 6 6 8 8
21947 121/300 Write block 4892 len 24576 data: 3 3 3 2 3 4
21948 122/300 Read block 2839 len 32768
21949 123/300 Flush
21950 124/300 Read block 589 len 36864
21951 125/300 Write block 2907 len 28672 data: 3 3 3 3 3 3 3
21952 126/300 Write block 1389 len 12288 data: 5 5 6
21953 127/300 Read block 4851 len 32768
21954 128/300 Read block 3933 len 20480
21955 129/300 Write block 3011 len 32768 data: 3 3 2 2 2 2 2 2
21956 130/300 Write block 903 len 16384 data: 4 5 3 3
21957 131/300 Write block 3319 len 36864 data: 4 5 5 4 5 5 6 7 5
21958 132/300 Write block 2596 len 4096 data: 4
21959 133/300 Write block 1457 len 16384 data: 7 8 6 5
21960 134/300 Read block 429 len 12288
21961 135/300 Read block 2568 len 12288
21962 136/300 Write block 4531 len 32768 data: 4 4 4 4 4 3 2 2
21963 137/300 Read block 4485 len 36864
21964 138/300 Read block 2575 len 4096
21965 139/300 Read block 506 len 12288
21966 140/300 Read block 108 len 24576
21967 141/300 Read block 241 len 28672
21968 142/300 Read block 3734 len 28672
21969 143/300 Write block 713 len 40960 data: 12 10 9 8 6 5 6 6 6 6
21970 144/300 Read block 928 len 40960
21971 145/300 Write block 4586 len 4096 data: 2
21972 146/300 Read block 4800 len 16384
21973 147/300 Write block 4904 len 24576 data: 3 3 2 3 4 3
21974 148/300 Write block 892 len 4096 data: 5
21975 149/300 Read block 3205 len 12288
21976 150/300 Read block 184 len 40960
21977 151/300 Write block 3011 len 12288 data: 4 4 3
21978 152/300 Write block 2708 len 24576 data: 2 3 4 4 4 5
21979 153/300 Read block 3961 len 32768
21980 154/300 Read block 3733 len 28672
21981 155/300 Read block 3875 len 8192
21982 156/300 Flush
21983 157/300 Read block 4629 len 8192
21984 158/300 Read block 2865 len 12288
21985 159/300 Read block 1190 len 20480
21986 160/300 Read block 2075 len 16384
21987 161/300 Read block 1118 len 24576
21988 162/300 Read block 1625 len 28672
21989 163/300 Read block 315 len 12288
21990 164/300 Flush
21991 165/300 Read block 3885 len 16384
21992 166/300 Read block 2320 len 4096
21993 167/300 Read block 3225 len 24576
21994 168/300 Write block 1096 len 28672 data: 3 3 5 5 5 6 6
21995 169/300 Write block 162 len 12288 data: 3 3 4
21996 170/300 Write block 2607 len 16384 data: 6 6 5 4
21997 171/300 Flush
21998 172/300 Write block 4933 len 4096 data: 4
21999 173/300 Read block 4881 len 40960
22000 174/300 Read block 4722 len 40960
22001 175/300 Write block 116 len 32768 data: 7 6 6 5 5 5 6 6
22002 176/300 Read block 1552 len 36864
22003 177/300 Write block 1019 len 28672 data: 7 7 6 4 3 2 2
22004 178/300 Read block 3495 len 4096
22005 179/300 Write block 2587 len 20480 data: 3 3 3 4 4
22006 180/300 Write block 3047 len 16384 data: 5 6 6 7
22007 181/300 Write block 930 len 12288 data: 5 5 4
22008 182/300 Write block 255 len 12288 data: 5 6 7
22009 183/300 Write block 205 len 32768 data: 6 6 6 6 6 5 5 3
22010 184/300 Read block 4906 len 12288
22011 185/300 Read block 362 len 12288
22012 186/300 Read block 3033 len 28672
22013 187/300 Read block 4019 len 28672
22014 188/300 Write block 4772 len 12288 data: 4 5 6
22015 189/300 Flush
22016 190/300 Read block 321 len 12288
22017 191/300 Write block 3174 len 16384 data: 5 6 7 7
22018 192/300 Read block 2446 len 36864
22019 193/300 Read block 1759 len 36864
22020 194/300 Read block 3871 len 20480
22021 195/300 Read block 4468 len 4096
22022 196/300 Flush
22023 197/300 Read block 2309 len 28672
22024 198/300 Read block 1788 len 40960
22025 199/300 Read block 1411 len 8192
22026 200/300 Read block 4948 len 24576
22027 201/300 Read block 2542 len 16384
22028 202/300 Write block 1934 len 16384 data: 4 4 3 3
22029 203/300 Read block 3476 len 24576
22030 204/300 Write block 2671 len 16384 data: 4 5 6 6
22031 205/300 Write block 813 len 28672 data: 3 3 4 4 4 4 6
22032 206/300 Flush
22033 207/300 Write block 2564 len 24576 data: 6 6 6 3 3 3
22034 208/300 Read block 1798 len 20480
22035 209/300 Write block 2157 len 4096 data: 9
22036 210/300 Write block 286 len 16384 data: 7 6 6 5
22037 211/300 Read block 906 len 36864
22038 212/300 Write block 2750 len 28672 data: 3 6 5 5 4 4 4
22039 213/300 Read block 2585 len 12288
22040 214/300 Write block 2065 len 16384 data: 3 3 3 4
22041 215/300 Read block 3259 len 32768
22042 216/300 Write block 4658 len 12288 data: 5 4 2
22043 217/300 Write block 720 len 4096 data: 7
22044 218/300 Flush
22045 219/300 Read block 1406 len 12288
22046 220/300 Write block 2518 len 16384 data: 3 2 2 2
22047 221/300 Write block 3608 len 12288 data: 8 6 5
22048 222/300 Write block 184 len 32768 data: 7 7 7 5 3 3 3 3
22049 223/300 Read block 1712 len 24576
22050 224/300 Write block 2407 len 16384 data: 6 7 6 8
22051 225/300 Read block 4583 len 16384
22052 226/300 Write block 1464 len 20480 data: 5 5 5 5 4
22053 227/300 Write block 3123 len 28672 data: 6 6 5 4 4 5 6
22054 228/300 Read block 272 len 40960
22055 229/300 Read block 3322 len 24576
22056 230/300 Read block 2007 len 12288
22057 231/300 Read block 2903 len 28672
22058 232/300 Flush
22059 233/300 Write block 2929 len 24576 data: 6 5 5 5 6 6
22060 234/300 Read block 771 len 40960
22061 235/300 Write block 4735 len 20480 data: 6 7 7 6 6
22062 236/300 Read block 3731 len 12288
22063 237/300 Read block 575 len 28672
22064 238/300 Read block 1175 len 8192
22065 239/300 Read block 857 len 8192
22066 240/300 Read block 516 len 36864
22067 241/300 Read block 4049 len 28672
22068 242/300 Read block 2849 len 12288
22069 243/300 Read block 4498 len 40960
22070 244/300 Read block 1631 len 24576
22071 245/300 Write block 757 len 8192 data: 2 3
22072 246/300 Flush
22073 247/300 Write block 644 len 20480 data: 3 3 3 3 6
22074 248/300 Write block 3712 len 36864 data: 7 7 7 7 5 5 4 3 3
22075 249/300 Write block 228 len 8192 data: 6 6
22076 250/300 Read block 3045 len 20480
22077 251/300 Flush
22078 252/300 Flush
22079 253/300 Read block 4762 len 8192
22080 254/300 Read block 1443 len 36864
22081 255/300 Write block 3004 len 28672 data: 5 4 4 4 4 4 3
22082 256/300 Read block 1730 len 12288
22083 257/300 Read block 1019 len 28672
22084 258/300 Write block 3226 len 24576 data: 3 3 4 4 5 5
22085 259/300 Write block 3810 len 32768 data: 5 4 3 3 3 2 2 3
22086 260/300 Flush
22087 261/300 Write block 2490 len 16384 data: 4 4 4 5
22088 262/300 Read block 789 len 24576
22089 263/300 Read block 32 len 40960
22090 264/300 Read block 3395 len 16384
22091 265/300 Read block 1617 len 36864
22092 266/300 Write block 2803 len 4096 data: 4
22093 267/300 Read block 250 len 28672
22094 268/300 Flush
22095 269/300 Read block 3620 len 40960
22096 270/300 Read block 3918 len 32768
22097 271/300 Read block 4781 len 16384
22098 272/300 Flush
22099 273/300 Read block 2346 len 4096
22100 274/300 Write block 3541 len 12288 data: 4 5 4
22101 275/300 Read block 4683 len 8192
22102 276/300 Write block 3106 len 8192 data: 7 7
22103 277/300 Write block 2349 len 40960 data: 5 6 6 7 9 7 5 5 3 2
22104 278/300 Write block 4865 len 24576 data: 5 5 7 6 7 6
22105 279/300 Write block 645 len 16384 data: 4 4 4 7
22106 280/300 Read block 3626 len 4096
22107 281/300 Read block 1515 len 4096
22108 282/300 Write block 4190 len 40960 data: 8 8 8 7 7 7 8 8 6 5
22109 283/300 Read block 2824 len 36864
22110 284/300 Read block 4716 len 12288
22111 285/300 Flush
22112 286/300 Write block 1683 len 16384 data: 3 3 3 3
22113 287/300 Read block 500 len 4096
22114 288/300 Flush
22115 289/300 Flush
22116 290/300 Flush
22117 291/300 Read block 2504 len 4096
22118 292/300 Read block 1080 len 32768
22119 293/300 Write block 1651 len 24576 data: 2 3 3 4 4 4
22120 294/300 Write block 2870 len 20480 data: 5 4 5 5 4
22121 295/300 Flush
22122 296/300 Flush
22123 297/300 Write block 1197 len 8192 data: 6 6
22124 298/300 Read block 4435 len 20480
22125 299/300 Read block 274 len 16384
22126 300/300 Read block 1763 len 16384
22127 301/300 Write block 1596 len 32768 data: 7 8 9 6 7 5 5 5
22128 Main task has work to do, go find it
22129 got dsc Start(1)
22130 start 1
22131 Jan 27 19:41:01.867 INFO request completed, latency_us: 226, response_code: 204, uri: /start/cid/1, method: POST, req_id: e742cd8a-f9af-45c0-99c1-e6739e7cc972, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
22132 Replay: started 1, returned:Ok(())
22133 ----------------------------------------------------------------
22134 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
22135 States: Active Offline Active
22136 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22137 6735 Acked Read 8 Done Sent Done false
22138 6736 Acked Write 5 Done Sent Done false
22139 6737 Acked Read 9 Done Sent Done false
22140 6738 Acked Read 2 Done Sent Done false
22141 6739 Acked Read 6 Done Sent Done false
22142 6740 Acked Read 4 Done Sent Done false
22143 6741 Acked Read 1 Done Sent Done false
22144 6742 Acked Read 7 Done Sent Done false
22145 6743 Acked Flush 0 Done Sent Done false
22146 6744 Acked Write 10 Done Sent Done false
22147 6745 Acked Read 1 Done Sent Done false
22148 6746 Acked Read 6 Done Sent Done false
22149 6747 Acked Read 9 Done Sent Done false
22150 6748 Acked Write 2 Done Sent Done false
22151 6749 Acked Read 6 Done Sent Done false
22152 6750 Acked Flush 0 Done Sent Done false
22153 6751 Acked Write 8 Done Sent Done false
22154 6752 Acked Read 4 Done Sent Done false
22155 6753 Acked Flush 0 Done Sent Done false
22156 6754 Acked Flush 0 Done Sent Done false
22157 6755 Acked Write 4 Done Sent Done false
22158 6756 Acked Read 4 Done Sent Done false
22159 6757 Acked Write 7 Done Sent Done false
22160 6758 Acked Write 9 Done Sent Done false
22161 6759 Acked Write 4 Done Sent Done false
22162 6760 Acked Read 6 Done Sent Done false
22163 6761 Acked Write 5 Done Sent Done false
22164 6762 Acked Write 1 Done Sent Done false
22165 6763 Acked Write 3 Done Sent Done false
22166 6764 Acked Flush 0 Done Sent Done false
22167 6765 Acked Read 7 Done Sent Done false
22168 6766 Acked Read 8 Done Sent Done false
22169 6767 Acked Write 5 Done Sent Done false
22170 6768 Acked Read 4 Done Sent Done false
22171 6769 Acked Read 2 Done Sent Done false
22172 6770 Acked Read 10 Done Sent Done false
22173 6771 Acked Read 2 Done Sent Done false
22174 6772 Acked Write 1 Done Sent Done false
22175 6773 Acked Read 9 Done Sent Done false
22176 6774 Acked Write 1 Done Sent Done false
22177 6775 Acked Write 5 Done Sent Done false
22178 6776 Acked Flush 0 Done Sent Done false
22179 6777 Acked Read 6 Done Sent Done false
22180 6778 Acked Write 6 Done Sent Done false
22181 6779 Acked Read 4 Done Sent Done false
22182 6780 Acked Read 9 Done Sent Done false
22183 6781 Acked Flush 0 Done Sent Done false
22184 6782 Acked Write 3 Done Sent Done false
22185 6783 Acked Read 3 Done Sent Done false
22186 6784 Acked Write 7 Done Sent Done false
22187 6785 Acked Read 6 Done Sent Done false
22188 6786 Acked Write 4 Done Sent Done false
22189 6787 Acked Write 1 Done Sent Done false
22190 6788 Acked Write 10 Done Sent Done false
22191 6789 Acked Read 7 Done Sent Done false
22192 6790 Acked Write 7 Done Sent Done false
22193 6791 Acked Read 3 Done Sent Done false
22194 6792 Acked Write 10 Done Sent Done false
22195 6793 Acked Read 3 Done Sent Done false
22196 6794 Acked Write 9 Done Sent Done false
22197 6795 Acked Flush 0 Done Sent Done false
22198 6796 Acked Write 4 Done Sent Done false
22199 6797 Acked Write 5 Done Sent Done false
22200 6798 Acked Read 7 Done Sent Done false
22201 6799 Acked Read 1 Done Sent Done false
22202 6800 Acked Flush 0 Done Sent Done false
22203 6801 Acked Read 3 Done Sent Done false
22204 6802 Acked Write 8 Done Sent Done false
22205 6803 Acked Read 7 Done Sent Done false
22206 6804 Acked Read 6 Done Sent Done false
22207 6805 Acked Read 3 Done Sent Done false
22208 6806 Acked Flush 0 Done Sent Done false
22209 6807 Acked Read 10 Done Sent Done false
22210 6808 Acked Read 6 Done Sent Done false
22211 6809 Acked Read 2 Done Sent Done false
22212 6810 Acked Read 4 Done Sent Done false
22213 6811 Acked Write 7 Done Sent Done false
22214 6812 Acked Read 6 Done Sent Done false
22215 6813 Acked Flush 0 Done Sent Done false
22216 6814 Acked Write 7 Done Sent Done false
22217 6815 Acked Read 6 Done Sent Done false
22218 6816 Acked Read 4 Done Sent Done false
22219 6817 Acked Read 4 Done Sent Done false
22220 6818 Acked Read 4 Done Sent Done false
22221 6819 Acked Write 1 Done Sent Done false
22222 6820 Acked Write 9 Done Sent Done false
22223 6821 Acked Read 4 Done Sent Done false
22224 6822 Acked Write 2 Done Sent Done false
22225 6823 Acked Read 5 Done Sent Done false
22226 6824 Acked Write 8 Done Sent Done false
22227 6825 Acked Write 6 Done Sent Done false
22228 6826 Acked Read 10 Done Sent Done false
22229 6827 Acked Read 1 Done Sent Done false
22230 6828 Acked Write 2 Done Sent Done false
22231 6829 Acked Read 5 Done Sent Done false
22232 6830 Acked Write 1 Done Sent Done false
22233 6831 Acked Write 2 Done Sent Done false
22234 6832 Acked Flush 0 Done Sent Done false
22235 6833 Acked Read 4 Done Sent Done false
22236 6834 Acked Read 10 Done Sent Done false
22237 6835 Acked Write 8 Done Sent Done false
22238 6836 Acked Read 1 Done Sent Done false
22239 6837 Acked Flush 0 Done Sent Done false
22240 6838 Acked Read 2 Done Sent Done false
22241 6839 Acked Flush 0 Done Sent Done false
22242 6840 Acked Read 5 Done Sent Done false
22243 6841 Acked Write 3 Done Sent Done false
22244 6842 Acked Write 6 Done Sent Done false
22245 6843 Acked Read 3 Done Sent Done false
22246 6844 Acked Write 8 Done Sent Done false
22247 6845 Acked Read 3 Done Sent Done false
22248 6846 Acked Read 1 Done Sent Done false
22249 6847 Acked Write 5 Done Sent Done false
22250 6848 Acked Read 8 Done Sent Done false
22251 6849 Acked Flush 0 Done Sent Done false
22252 6850 Acked Read 7 Done Sent Done false
22253 6851 Acked Write 10 Done Sent Done false
22254 6852 Acked Read 1 Done Sent Done false
22255 6853 Acked Write 10 Done Sent Done false
22256 6854 Acked Write 5 Done Sent Done false
22257 6855 Acked Write 6 Done Sent Done false
22258 6856 Acked Read 8 Done Sent Done false
22259 6857 Acked Flush 0 Done Sent Done false
22260 6858 Acked Read 9 Done Sent Done false
22261 6859 Acked Write 7 Done Sent Done false
22262 6860 Acked Write 3 Done Sent Done false
22263 6861 Acked Read 8 Done Sent Done false
22264 6862 Acked Read 5 Done Sent Done false
22265 6863 Acked Write 8 Done Sent Done false
22266 6864 Acked Write 4 Done Sent Done false
22267 6865 Acked Write 9 Done Sent Done false
22268 6866 Acked Write 1 Done Sent Done false
22269 6867 Acked Write 4 Done Sent Done false
22270 6868 Acked Read 3 Done Sent Done false
22271 6869 Acked Read 3 Done Sent Done false
22272 6870 Acked Write 8 Done Sent Done false
22273 6871 Acked Read 9 Done Sent Done false
22274 6872 Acked Read 1 Done Sent Done false
22275 6873 Acked Read 3 Done Sent Done false
22276 6874 Acked Read 6 Done Sent Done false
22277 6875 Acked Read 7 Done Sent Done false
22278 6876 Acked Read 7 Done Sent Done false
22279 6877 Acked Write 10 Done Sent Done false
22280 6878 Acked Read 10 Done Sent Done false
22281 6879 Acked Write 1 Done Sent Done false
22282 6880 Acked Read 4 Done Sent Done false
22283 6881 Acked Write 6 Done Sent Done false
22284 6882 Acked Write 1 Done Sent Done false
22285 6883 Acked Read 3 Done Sent Done false
22286 6884 Acked Read 10 Done Sent Done false
22287 6885 Acked Write 3 Done Sent Done false
22288 6886 Acked Write 6 Done Sent Done false
22289 6887 Acked Read 8 Done Sent Done false
22290 6888 Acked Read 7 Done Sent Done false
22291 6889 Acked Read 2 Done Sent Done false
22292 6890 Acked Flush 0 Done Sent Done false
22293 6891 Acked Read 2 Done Sent Done false
22294 6892 Acked Read 3 Done Sent Done false
22295 6893 Acked Read 5 Done Sent Done false
22296 6894 Acked Read 4 Done Sent Done false
22297 6895 Acked Read 6 Done Sent Done false
22298 6896 Acked Read 7 Done Sent Done false
22299 6897 Acked Read 3 Done Sent Done false
22300 6898 Acked Flush 0 Done Sent Done false
22301 6899 Acked Read 4 Done Sent Done false
22302 6900 Acked Read 1 Done Sent Done false
22303 6901 Acked Read 6 Done Sent Done false
22304 6902 Acked Write 7 Done Sent Done false
22305 6903 Acked Write 3 Done Sent Done false
22306 6904 Acked Write 4 Done Sent Done false
22307 6905 Acked Flush 0 Done Sent Done false
22308 6906 Acked Write 1 Done Sent Done false
22309 6907 Acked Read 10 Done Sent Done false
22310 6908 Acked Read 10 Done Sent Done false
22311 6909 Acked Write 8 Done Sent Done false
22312 6910 Acked Read 9 Done Sent Done false
22313 6911 Acked Write 7 Done Sent Done false
22314 6912 Acked Read 1 Done Sent Done false
22315 6913 Acked Write 5 Done Sent Done false
22316 6914 Acked Write 4 Done Sent Done false
22317 6915 Acked Write 3 Done Sent Done false
22318 6916 Acked Write 3 Done Sent Done false
22319 6917 Acked Write 8 Done Sent Done false
22320 6918 Acked Read 3 Done Sent Done false
22321 6919 Acked Read 3 Done Sent Done false
22322 6920 Acked Read 7 Done Sent Done false
22323 6921 Acked Read 7 Done Sent Done false
22324 6922 Acked Write 3 Done Sent Done false
22325 6923 Acked Flush 0 Done Sent Done false
22326 6924 Acked Read 3 Done Sent Done false
22327 6925 Acked Write 4 Done Sent Done false
22328 6926 Acked Read 9 Done Sent Done false
22329 6927 Acked Read 9 Done Sent Done false
22330 6928 Acked Read 5 Done Sent Done false
22331 6929 Acked Read 1 Done Sent Done false
22332 6930 Acked Flush 0 Done Sent Done false
22333 6931 Acked Read 7 Done Sent Done false
22334 6932 Acked Read 10 Done Sent Done false
22335 6933 Acked Read 2 Done Sent Done false
22336 6934 Acked Read 6 Done Sent Done false
22337 6935 Acked Read 4 Done Sent Done false
22338 6936 Acked Write 4 Done Sent Done false
22339 6937 Acked Read 6 Done Sent Done false
22340 6938 Acked Write 4 Done Sent Done false
22341 6939 Acked Write 7 Done Sent Done false
22342 6940 Acked Flush 0 Done Sent Done false
22343 6941 Acked Write 6 Done Sent Done false
22344 6942 Acked Read 5 Done Sent Done false
22345 6943 Acked Write 1 Done Sent Done false
22346 6944 Acked Write 4 Done Sent Done false
22347 6945 Acked Read 9 Done Sent Done false
22348 6946 Acked Write 7 Done Sent Done false
22349 6947 Acked Read 3 Done Sent Done false
22350 6948 Acked Write 4 Done Sent Done false
22351 6949 Acked Read 8 Done Sent Done false
22352 6950 Acked Write 3 Done Sent Done false
22353 6951 Acked Write 1 Done Sent Done false
22354 6952 Acked Flush 0 Done Sent Done false
22355 6953 Acked Read 3 Done Sent Done false
22356 6954 Acked Write 4 Done Sent Done false
22357 6955 Acked Write 3 Done Sent Done false
22358 6956 Acked Write 8 Done Sent Done false
22359 6957 Acked Read 6 Done Sent Done false
22360 6958 Acked Write 4 Done Sent Done false
22361 6959 Acked Read 4 Done Sent Done false
22362 6960 Acked Write 5 Done Sent Done false
22363 6961 Acked Write 7 Done Sent Done false
22364 6962 Acked Read 10 Done Sent Done false
22365 6963 Acked Read 6 Done Sent Done false
22366 6964 Acked Read 3 Done Sent Done false
22367 6965 Acked Read 7 Done Sent Done false
22368 6966 Acked Flush 0 Done Sent Done false
22369 6967 Acked Write 6 Done Sent Done false
22370 6968 Acked Read 10 Done Sent Done false
22371 6969 Acked Write 5 Done Sent Done false
22372 6970 Acked Read 3 Done Sent Done false
22373 6971 Acked Read 7 Done Sent Done false
22374 6972 Acked Read 2 Done Sent Done false
22375 6973 Acked Read 2 Done Sent Done false
22376 6974 Acked Read 9 Done Sent Done false
22377 6975 Acked Read 7 Done Sent Done false
22378 6976 Acked Read 3 Done Sent Done false
22379 6977 Acked Read 10 Done Sent Done false
22380 6978 Acked Read 6 Done Sent Done false
22381 6979 Acked Write 2 Done Sent Done false
22382 6980 Acked Flush 0 Done Sent Done false
22383 6981 Acked Write 5 Done Sent Done false
22384 6982 Acked Write 9 Done Sent Done false
22385 6983 Acked Write 2 Done Sent Done false
22386 6984 Acked Read 5 Done Sent Done false
22387 6985 Acked Flush 0 Done Sent Done false
22388 6986 Acked Flush 0 Done Sent Done false
22389 6987 Acked Read 2 Done Sent Done false
22390 6988 Acked Read 9 Done Sent Done false
22391 6989 Acked Write 7 Done Sent Done false
22392 6990 Acked Read 3 Done Sent Done false
22393 6991 Acked Read 7 Done Sent Done false
22394 6992 Acked Write 6 Done Sent Done false
22395 6993 Acked Write 8 Done Sent Done false
22396 6994 Acked Flush 0 Done Sent Done false
22397 6995 Acked Write 4 Done Sent Done false
22398 6996 Acked Read 6 Done Sent Done false
22399 6997 Acked Read 10 Done Sent Done false
22400 6998 Acked Read 4 Done Sent Done false
22401 6999 Acked Read 9 Done Sent Done false
22402 7000 Acked Write 1 Done Sent Done false
22403 7001 Acked Read 7 Done Sent Done false
22404 7002 Acked Flush 0 Done Sent Done false
22405 7003 Acked Read 10 Done Sent Done false
22406 7004 Acked Read 8 Done Sent Done false
22407 7005 Acked Read 4 Done Sent Done false
22408 7006 Acked Flush 0 Done Sent Done false
22409 7007 Acked Read 1 Done Sent Done false
22410 7008 Acked Write 3 Done Sent Done false
22411 7009 Acked Read 2 Done Sent Done false
22412 7010 Acked Write 2 Done Sent Done false
22413 7011 Acked Write 10 Done Sent Done false
22414 7012 Acked Write 6 Done Sent Done false
22415 7013 Acked Write 4 Done Sent Done false
22416 7014 Acked Read 1 Done Sent Done false
22417 7015 Acked Read 1 Done Sent Done false
22418 7016 Acked Write 10 Done Sent Done false
22419 7017 Acked Read 9 Done Sent Done false
22420 7018 Acked Read 3 Done Sent Done false
22421 7019 Acked Flush 0 Done Sent Done false
22422 7020 Acked Write 4 Done Sent Done false
22423 7021 Acked Read 1 Done Sent Done false
22424 7022 Acked Flush 0 Done Sent Done false
22425 7023 Acked Flush 0 Done Sent Done false
22426 7024 Acked Flush 0 Done Sent Done false
22427 7025 Acked Read 1 Done Sent Done false
22428 7026 Acked Read 8 Done Sent Done false
22429 7027 Acked Write 6 Done Sent Done false
22430 7028 Acked Write 5 Done Sent Done false
22431 7029 Acked Flush 0 Done Sent Done false
22432 7030 Acked Flush 0 Done Sent Done false
22433 7031 Acked Write 2 Done Sent Done false
22434 7032 Acked Read 5 Done Sent Done false
22435 7033 Acked Read 4 Done Sent Done false
22436 7034 Acked Read 4 Done Sent Done false
22437 7035 Acked Write 8 Done Sent Done false
22438 STATES DS:0 DS:1 DS:2 TOTAL
22439 Sent 0 301 0 301
22440 Done 301 0 301 602
22441 Skipped 0 0 0 0
22442 Error 0 0 0 0
22443 Last Flush: 7030 6734 7030
22444 Downstairs last five completed: 6734 6733 6732 6731 6730
22445 Upstairs last five completed: 7035 7034 7033 7032 7031
22446 CLIENT: Up:0 ds:301 act:2
22447 [8820] got start action
22448 [8820] I am going to restart
22449 Starting downstairs at port 8820
22450 Make output file at "/tmp/dsc/downstairs-8820.txt"
22451 [8820][1] reports Starting
22452 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1220)
22453 [8820][1] reports Running
22454 ----------------------------------------------------------------
22455 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
22456 States: Active Offline Active
22457 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
22458 6735 Acked Read 8 Done Sent Done false
22459 6736 Acked Write 5 Done Sent Done false
22460 6737 Acked Read 9 Done Sent Done false
22461 6738 Acked Read 2 Done Sent Done false
22462 6739 Acked Read 6 Done Sent Done false
22463 6740 Acked Read 4 Done Sent Done false
22464 6741 Acked Read 1 Done Sent Done false
22465 6742 Acked Read 7 Done Sent Done false
22466 6743 Acked Flush 0 Done Sent Done false
22467 6744 Acked Write 10 Done Sent Done false
22468 6745 Acked Read 1 Done Sent Done false
22469 6746 Acked Read 6 Done Sent Done false
22470 6747 Acked Read 9 Done Sent Done false
22471 6748 Acked Write 2 Done Sent Done false
22472 6749 Acked Read 6 Done Sent Done false
22473 6750 Acked Flush 0 Done Sent Done false
22474 6751 Acked Write 8 Done Sent Done false
22475 6752 Acked Read 4 Done Sent Done false
22476 6753 Acked Flush 0 Done Sent Done false
22477 6754 Acked Flush 0 Done Sent Done false
22478 6755 Acked Write 4 Done Sent Done false
22479 6756 Acked Read 4 Done Sent Done false
22480 6757 Acked Write 7 Done Sent Done false
22481 6758 Acked Write 9 Done Sent Done false
22482 6759 Acked Write 4 Done Sent Done false
22483 6760 Acked Read 6 Done Sent Done false
22484 6761 Acked Write 5 Done Sent Done false
22485 6762 Acked Write 1 Done Sent Done false
22486 6763 Acked Write 3 Done Sent Done false
22487 6764 Acked Flush 0 Done Sent Done false
22488 6765 Acked Read 7 Done Sent Done false
22489 6766 Acked Read 8 Done Sent Done false
22490 6767 Acked Write 5 Done Sent Done false
22491 6768 Acked Read 4 Done Sent Done false
22492 6769 Acked Read 2 Done Sent Done false
22493 6770 Acked Read 10 Done Sent Done false
22494 6771 Acked Read 2 Done Sent Done false
22495 6772 Acked Write 1 Done Sent Done false
22496 6773 Acked Read 9 Done Sent Done false
22497 6774 Acked Write 1 Done Sent Done false
22498 6775 Acked Write 5 Done Sent Done false
22499 6776 Acked Flush 0 Done Sent Done false
22500 6777 Acked Read 6 Done Sent Done false
22501 6778 Acked Write 6 Done Sent Done false
22502 6779 Acked Read 4 Done Sent Done false
22503 6780 Acked Read 9 Done Sent Done false
22504 6781 Acked Flush 0 Done Sent Done false
22505 6782 Acked Write 3 Done Sent Done false
22506 6783 Acked Read 3 Done Sent Done false
22507 6784 Acked Write 7 Done Sent Done false
22508 6785 Acked Read 6 Done Sent Done false
22509 6786 Acked Write 4 Done Sent Done false
22510 6787 Acked Write 1 Done Sent Done false
22511 6788 Acked Write 10 Done Sent Done false
22512 6789 Acked Read 7 Done Sent Done false
22513 6790 Acked Write 7 Done Sent Done false
22514 6791 Acked Read 3 Done Sent Done false
22515 6792 Acked Write 10 Done Sent Done false
22516 6793 Acked Read 3 Done Sent Done false
22517 6794 Acked Write 9 Done Sent Done false
22518 6795 Acked Flush 0 Done Sent Done false
22519 6796 Acked Write 4 Done Sent Done false
22520 6797 Acked Write 5 Done Sent Done false
22521 6798 Acked Read 7 Done Sent Done false
22522 6799 Acked Read 1 Done Sent Done false
22523 6800 Acked Flush 0 Done Sent Done false
22524 6801 Acked Read 3 Done Sent Done false
22525 6802 Acked Write 8 Done Sent Done false
22526 6803 Acked Read 7 Done Sent Done false
22527 6804 Acked Read 6 Done Sent Done false
22528 6805 Acked Read 3 Done Sent Done false
22529 6806 Acked Flush 0 Done Sent Done false
22530 6807 Acked Read 10 Done Sent Done false
22531 6808 Acked Read 6 Done Sent Done false
22532 6809 Acked Read 2 Done Sent Done false
22533 6810 Acked Read 4 Done Sent Done false
22534 6811 Acked Write 7 Done Sent Done false
22535 6812 Acked Read 6 Done Sent Done false
22536 6813 Acked Flush 0 Done Sent Done false
22537 6814 Acked Write 7 Done Sent Done false
22538 6815 Acked Read 6 Done Sent Done false
22539 6816 Acked Read 4 Done Sent Done false
22540 6817 Acked Read 4 Done Sent Done false
22541 6818 Acked Read 4 Done Sent Done false
22542 6819 Acked Write 1 Done Sent Done false
22543 6820 Acked Write 9 Done Sent Done false
22544 6821 Acked Read 4 Done Sent Done false
22545 6822 Acked Write 2 Done Sent Done false
22546 6823 Acked Read 5 Done Sent Done false
22547 6824 Acked Write 8 Done Sent Done false
22548 6825 Acked Write 6 Done Sent Done false
22549 6826 Acked Read 10 Done Sent Done false
22550 6827 Acked Read 1 Done Sent Done false
22551 6828 Acked Write 2 Done Sent Done false
22552 6829 Acked Read 5 Done Sent Done false
22553 6830 Acked Write 1 Done Sent Done false
22554 6831 Acked Write 2 Done Sent Done false
22555 6832 Acked Flush 0 Done Sent Done false
22556 6833 Acked Read 4 Done Sent Done false
22557 6834 Acked Read 10 Done Sent Done false
22558 6835 Acked Write 8 Done Sent Done false
22559 6836 Acked Read 1 Done Sent Done false
22560 6837 Acked Flush 0 Done Sent Done false
22561 6838 Acked Read 2 Done Sent Done false
22562 6839 Acked Flush 0 Done Sent Done false
22563 6840 Acked Read 5 Done Sent Done false
22564 6841 Acked Write 3 Done Sent Done false
22565 6842 Acked Write 6 Done Sent Done false
22566 6843 Acked Read 3 Done Sent Done false
22567 6844 Acked Write 8 Done Sent Done false
22568 6845 Acked Read 3 Done Sent Done false
22569 6846 Acked Read 1 Done Sent Done false
22570 6847 Acked Write 5 Done Sent Done false
22571 6848 Acked Read 8 Done Sent Done false
22572 6849 Acked Flush 0 Done Sent Done false
22573 6850 Acked Read 7 Done Sent Done false
22574 6851 Acked Write 10 Done Sent Done false
22575 6852 Acked Read 1 Done Sent Done false
22576 6853 Acked Write 10 Done Sent Done false
22577 6854 Acked Write 5 Done Sent Done false
22578 6855 Acked Write 6 Done Sent Done false
22579 6856 Acked Read 8 Done Sent Done false
22580 6857 Acked Flush 0 Done Sent Done false
22581 6858 Acked Read 9 Done Sent Done false
22582 6859 Acked Write 7 Done Sent Done false
22583 6860 Acked Write 3 Done Sent Done false
22584 6861 Acked Read 8 Done Sent Done false
22585 6862 Acked Read 5 Done Sent Done false
22586 6863 Acked Write 8 Done Sent Done false
22587 6864 Acked Write 4 Done Sent Done false
22588 6865 Acked Write 9 Done Sent Done false
22589 6866 Acked Write 1 Done Sent Done false
22590 6867 Acked Write 4 Done Sent Done false
22591 6868 Acked Read 3 Done Sent Done false
22592 6869 Acked Read 3 Done Sent Done false
22593 6870 Acked Write 8 Done Sent Done false
22594 6871 Acked Read 9 Done Sent Done false
22595 6872 Acked Read 1 Done Sent Done false
22596 6873 Acked Read 3 Done Sent Done false
22597 6874 Acked Read 6 Done Sent Done false
22598 6875 Acked Read 7 Done Sent Done false
22599 6876 Acked Read 7 Done Sent Done false
22600 6877 Acked Write 10 Done Sent Done false
22601 6878 Acked Read 10 Done Sent Done false
22602 6879 Acked Write 1 Done Sent Done false
22603 6880 Acked Read 4 Done Sent Done false
22604 6881 Acked Write 6 Done Sent Done false
22605 6882 Acked Write 1 Done Sent Done false
22606 6883 Acked Read 3 Done Sent Done false
22607 6884 Acked Read 10 Done Sent Done false
22608 6885 Acked Write 3 Done Sent Done false
22609 6886 Acked Write 6 Done Sent Done false
22610 6887 Acked Read 8 Done Sent Done false
22611 6888 Acked Read 7 Done Sent Done false
22612 6889 Acked Read 2 Done Sent Done false
22613 6890 Acked Flush 0 Done Sent Done false
22614 6891 Acked Read 2 Done Sent Done false
22615 6892 Acked Read 3 Done Sent Done false
22616 6893 Acked Read 5 Done Sent Done false
22617 6894 Acked Read 4 Done Sent Done false
22618 6895 Acked Read 6 Done Sent Done false
22619 6896 Acked Read 7 Done Sent Done false
22620 6897 Acked Read 3 Done Sent Done false
22621 6898 Acked Flush 0 Done Sent Done false
22622 6899 Acked Read 4 Done Sent Done false
22623 6900 Acked Read 1 Done Sent Done false
22624 6901 Acked Read 6 Done Sent Done false
22625 6902 Acked Write 7 Done Sent Done false
22626 6903 Acked Write 3 Done Sent Done false
22627 6904 Acked Write 4 Done Sent Done false
22628 6905 Acked Flush 0 Done Sent Done false
22629 6906 Acked Write 1 Done Sent Done false
22630 6907 Acked Read 10 Done Sent Done false
22631 6908 Acked Read 10 Done Sent Done false
22632 6909 Acked Write 8 Done Sent Done false
22633 6910 Acked Read 9 Done Sent Done false
22634 6911 Acked Write 7 Done Sent Done false
22635 6912 Acked Read 1 Done Sent Done false
22636 6913 Acked Write 5 Done Sent Done false
22637 6914 Acked Write 4 Done Sent Done false
22638 6915 Acked Write 3 Done Sent Done false
22639 6916 Acked Write 3 Done Sent Done false
22640 6917 Acked Write 8 Done Sent Done false
22641 6918 Acked Read 3 Done Sent Done false
22642 6919 Acked Read 3 Done Sent Done false
22643 6920 Acked Read 7 Done Sent Done false
22644 6921 Acked Read 7 Done Sent Done false
22645 6922 Acked Write 3 Done Sent Done false
22646 6923 Acked Flush 0 Done Sent Done false
22647 6924 Acked Read 3 Done Sent Done false
22648 6925 Acked Write 4 Done Sent Done false
22649 6926 Acked Read 9 Done Sent Done false
22650 6927 Acked Read 9 Done Sent Done false
22651 6928 Acked Read 5 Done Sent Done false
22652 6929 Acked Read 1 Done Sent Done false
22653 6930 Acked Flush 0 Done Sent Done false
22654 6931 Acked Read 7 Done Sent Done false
22655 6932 Acked Read 10 Done Sent Done false
22656 6933 Acked Read 2 Done Sent Done false
22657 6934 Acked Read 6 Done Sent Done false
22658 6935 Acked Read 4 Done Sent Done false
22659 6936 Acked Write 4 Done Sent Done false
22660 6937 Acked Read 6 Done Sent Done false
22661 6938 Acked Write 4 Done Sent Done false
22662 6939 Acked Write 7 Done Sent Done false
22663 6940 Acked Flush 0 Done Sent Done false
22664 6941 Acked Write 6 Done Sent Done false
22665 6942 Acked Read 5 Done Sent Done false
22666 6943 Acked Write 1 Done Sent Done false
22667 6944 Acked Write 4 Done Sent Done false
22668 6945 Acked Read 9 Done Sent Done false
22669 6946 Acked Write 7 Done Sent Done false
22670 6947 Acked Read 3 Done Sent Done false
22671 6948 Acked Write 4 Done Sent Done false
22672 6949 Acked Read 8 Done Sent Done false
22673 6950 Acked Write 3 Done Sent Done false
22674 6951 Acked Write 1 Done Sent Done false
22675 6952 Acked Flush 0 Done Sent Done false
22676 6953 Acked Read 3 Done Sent Done false
22677 6954 Acked Write 4 Done Sent Done false
22678 6955 Acked Write 3 Done Sent Done false
22679 6956 Acked Write 8 Done Sent Done false
22680 6957 Acked Read 6 Done Sent Done false
22681 6958 Acked Write 4 Done Sent Done false
22682 6959 Acked Read 4 Done Sent Done false
22683 6960 Acked Write 5 Done Sent Done false
22684 6961 Acked Write 7 Done Sent Done false
22685 6962 Acked Read 10 Done Sent Done false
22686 6963 Acked Read 6 Done Sent Done false
22687 6964 Acked Read 3 Done Sent Done false
22688 6965 Acked Read 7 Done Sent Done false
22689 6966 Acked Flush 0 Done Sent Done false
22690 6967 Acked Write 6 Done Sent Done false
22691 6968 Acked Read 10 Done Sent Done false
22692 6969 Acked Write 5 Done Sent Done false
22693 6970 Acked Read 3 Done Sent Done false
22694 6971 Acked Read 7 Done Sent Done false
22695 6972 Acked Read 2 Done Sent Done false
22696 6973 Acked Read 2 Done Sent Done false
22697 6974 Acked Read 9 Done Sent Done false
22698 6975 Acked Read 7 Done Sent Done false
22699 6976 Acked Read 3 Done Sent Done false
22700 6977 Acked Read 10 Done Sent Done false
22701 6978 Acked Read 6 Done Sent Done false
22702 6979 Acked Write 2 Done Sent Done false
22703 6980 Acked Flush 0 Done Sent Done false
22704 6981 Acked Write 5 Done Sent Done false
22705 6982 Acked Write 9 Done Sent Done false
22706 6983 Acked Write 2 Done Sent Done false
22707 6984 Acked Read 5 Done Sent Done false
22708 6985 Acked Flush 0 Done Sent Done false
22709 6986 Acked Flush 0 Done Sent Done false
22710 6987 Acked Read 2 Done Sent Done false
22711 6988 Acked Read 9 Done Sent Done false
22712 6989 Acked Write 7 Done Sent Done false
22713 6990 Acked Read 3 Done Sent Done false
22714 6991 Acked Read 7 Done Sent Done false
22715 6992 Acked Write 6 Done Sent Done false
22716 6993 Acked Write 8 Done Sent Done false
22717 6994 Acked Flush 0 Done Sent Done false
22718 6995 Acked Write 4 Done Sent Done false
22719 6996 Acked Read 6 Done Sent Done false
22720 6997 Acked Read 10 Done Sent Done false
22721 6998 Acked Read 4 Done Sent Done false
22722 6999 Acked Read 9 Done Sent Done false
22723 7000 Acked Write 1 Done Sent Done false
22724 7001 Acked Read 7 Done Sent Done false
22725 7002 Acked Flush 0 Done Sent Done false
22726 7003 Acked Read 10 Done Sent Done false
22727 7004 Acked Read 8 Done Sent Done false
22728 7005 Acked Read 4 Done Sent Done false
22729 7006 Acked Flush 0 Done Sent Done false
22730 7007 Acked Read 1 Done Sent Done false
22731 7008 Acked Write 3 Done Sent Done false
22732 7009 Acked Read 2 Done Sent Done false
22733 7010 Acked Write 2 Done Sent Done false
22734 7011 Acked Write 10 Done Sent Done false
22735 7012 Acked Write 6 Done Sent Done false
22736 7013 Acked Write 4 Done Sent Done false
22737 7014 Acked Read 1 Done Sent Done false
22738 7015 Acked Read 1 Done Sent Done false
22739 7016 Acked Write 10 Done Sent Done false
22740 7017 Acked Read 9 Done Sent Done false
22741 7018 Acked Read 3 Done Sent Done false
22742 7019 Acked Flush 0 Done Sent Done false
22743 7020 Acked Write 4 Done Sent Done false
22744 7021 Acked Read 1 Done Sent Done false
22745 7022 Acked Flush 0 Done Sent Done false
22746 7023 Acked Flush 0 Done Sent Done false
22747 7024 Acked Flush 0 Done Sent Done false
22748 7025 Acked Read 1 Done Sent Done false
22749 7026 Acked Read 8 Done Sent Done false
22750 7027 Acked Write 6 Done Sent Done false
22751 7028 Acked Write 5 Done Sent Done false
22752 7029 Acked Flush 0 Done Sent Done false
22753 7030 Acked Flush 0 Done Sent Done false
22754 7031 Acked Write 2 Done Sent Done false
22755 7032 Acked Read 5 Done Sent Done false
22756 7033 Acked Read 4 Done Sent Done false
22757 7034 Acked Read 4 Done Sent Done false
22758 7035 Acked Write 8 Done Sent Done false
22759 7036 Acked Flush 0 Done Sent Done false
22760 STATES DS:0 DS:1 DS:2 TOTAL
22761 Sent 0 302 0 302
22762 Done 302 0 302 604
22763 Skipped 0 0 0 0
22764 Error 0 0 0 0
22765 Last Flush: 7036 6734 7036
22766 Downstairs last five completed: 6734 6733 6732 6731 6730
22767 Upstairs last five completed: 7036 7035 7034 7033 7032
22768 CLIENT: Up:0 ds:302 act:2
227692025-01-27T19:41:07.634ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227702025-01-27T19:41:07.634ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:52788) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227712025-01-27T19:41:07.635ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227722025-01-27T19:41:07.636ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227732025-01-27T19:41:07.636ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227742025-01-27T19:41:07.636ZINFOcrucible: send last flush ID to this DS: 6734 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227752025-01-27T19:41:07.636ZINFOcrucible: Replied this last flush ID: 6734 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227762025-01-27T19:41:07.636ZINFOcrucible: [1] client re-new 302 jobs since flush 6734 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227772025-01-27T19:41:07.636ZINFOcrucible: [1] Replayed 302 jobs since flush: 6734 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
22778 ----------------------------------------------------------------
22779 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
22780 Downstairs last five completed: 7036 7035 7034 7033 7032
22781 Upstairs last five completed: 7036 7035 7034 7033 7032
22782 CLIENT: Up:0 ds:0 act:3
22783 Replay: All jobs finished
22784 Main task has work to do, go find it
22785 got dsc Stop(2)
22786 stop 2
22787 Jan 27 19:41:09.872 INFO request completed, latency_us: 365, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 055c4d6a-2a9d-4816-8c32-82d4dbc5b4cc, [8830] Got stop action so:false kr:false
22788 remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
22789 Jan 27 19:41:09.873 INFO request completed, latency_us: 139, response_code: 200, uri: /state/cid/2, method: GET, req_id: 5317fb54-546f-4339-b615-db0c98957eb3, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
227902025-01-27T19:41:09.873ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227912025-01-27T19:41:09.873ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227922025-01-27T19:41:09.873ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
227932025-01-27T19:41:09.873ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
22794 [8830] Exited with: ExitStatus(unix_wait_status(9))
22795 [8830][2] reports Exit
22796 Jan 27 19:41:13.876 INFO request completed, latency_us: 319, response_code: 200, uri: /state/cid/2, method: GET, req_id: b5df2c67-f99e-4c87-b67a-c1ae0ae3e606, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
22797 001/300 Read block 4880 len 16384
22798 002/300 Read block 3042 len 20480
22799 003/300 Read block 2134 len 4096
22800 004/300 Write block 4268 len 24576 data: 6 5 5 7 5 5
22801 005/300 Read block 91 len 4096
22802 006/300 Read block 4825 len 4096
22803 007/300 Read block 2538 len 16384
22804 008/300 Read block 1158 len 28672
22805 009/300 Read block 3517 len 40960
22806 010/300 Write block 1807 len 8192 data: 5 5
22807 011/300 Flush
22808 012/300 Write block 2198 len 4096 data: 5
22809 013/300 Read block 3901 len 4096
22810 014/300 Read block 2075 len 12288
22811 015/300 Write block 3960 len 36864 data: 5 5 6 7 6 8 6 6 6
22812 016/300 Write block 4135 len 16384 data: 4 4 4 3
22813 017/300 Flush
22814 018/300 Write block 1339 len 24576 data: 5 4 3 5 5 6
22815 019/300 Read block 2151 len 4096
22816 020/300 Flush
22817 021/300 Read block 4898 len 12288
22818 022/300 Read block 4959 len 16384
22819 023/300 Read block 3856 len 28672
22820 024/300 Read block 634 len 8192
22821 025/300 Read block 4774 len 24576
22822 026/300 Flush
22823 027/300 Write block 4924 len 32768 data: 7 7 6 7 7 6 5 5
22824 028/300 Write block 1214 len 20480 data: 3 4 4 4 5
22825 029/300 Write block 2489 len 16384 data: 4 5 5 5
22826 030/300 Write block 2506 len 16384 data: 3 4 6 5
22827 031/300 Read block 972 len 4096
22828 032/300 Read block 1367 len 24576
22829 033/300 Write block 2555 len 8192 data: 4 4
22830 034/300 Read block 1074 len 20480
22831 035/300 Write block 3866 len 4096 data: 4
22832 036/300 Read block 554 len 20480
22833 037/300 Read block 4530 len 40960
22834 038/300 Write block 503 len 40960 data: 5 4 6 6 5 4 3 5 7 6
22835 039/300 Read block 4341 len 4096
22836 040/300 Write block 2251 len 24576 data: 4 5 5 6 9 10
22837 041/300 Read block 2153 len 12288
22838 042/300 Read block 3254 len 12288
22839 043/300 Write block 4655 len 28672 data: 4 5 5 6 5 3 2
22840 044/300 Write block 1820 len 4096 data: 6
22841 045/300 Write block 1028 len 12288 data: 3 5 5
22842 046/300 Read block 2609 len 4096
22843 047/300 Read block 404 len 32768
22844 048/300 Write block 3174 len 32768 data: 6 7 8 8 8 8 6 6
22845 049/300 Read block 138 len 4096
22846 050/300 Read block 1216 len 40960
22847 051/300 Read block 2315 len 8192
22848 052/300 Write block 3715 len 16384 data: 8 6 6 5
22849 053/300 Read block 3143 len 24576
22850 054/300 Write block 3122 len 28672 data: 5 7 7 6 5 5 6
22851 055/300 Flush
22852 056/300 Read block 3877 len 24576
22853 057/300 Read block 979 len 36864
22854 058/300 Read block 1090 len 16384
22855 059/300 Flush
22856 060/300 Read block 1178 len 16384
22857 061/300 Write block 3068 len 20480 data: 5 4 3 3 3
22858 062/300 Read block 1947 len 24576
22859 063/300 Write block 2884 len 12288 data: 2 4 4
22860 064/300 Read block 4392 len 4096
22861 065/300 Read block 4328 len 40960
22862 066/300 Write block 2105 len 24576 data: 4 4 4 4 4 3
22863 067/300 Write block 3993 len 28672 data: 3 3 2 2 2 2 2
22864 068/300 Read block 2618 len 24576
22865 069/300 Read block 474 len 12288
22866 070/300 Flush
22867 071/300 Read block 2886 len 32768
22868 072/300 Write block 4130 len 32768 data: 4 3 3 4 5 5 5 5
22869 073/300 Read block 1878 len 28672
22870 074/300 Write block 4529 len 36864 data: 4 4 5 5 5 5 5 4 3
22871 075/300 Read block 2370 len 20480
22872 076/300 Read block 3536 len 16384
22873 077/300 Write block 920 len 24576 data: 6 6 5 3 4 4
22874 078/300 Read block 3131 len 24576
22875 079/300 Write block 1573 len 40960 data: 6 5 5 5 4 4 3 5 4 4
22876 080/300 Write block 1223 len 24576 data: 6 6 5 4 3 3
22877 081/300 Write block 864 len 20480 data: 6 5 8 8 8
22878 082/300 Read block 1165 len 20480
22879 083/300 Write block 4007 len 40960 data: 3 5 6 7 5 4 5 3 3 3
22880 084/300 Read block 3988 len 4096
22881 085/300 Write block 2380 len 24576 data: 5 4 3 3 3 5
22882 086/300 Read block 3646 len 40960
22883 087/300 Read block 4315 len 32768
22884 088/300 Write block 3576 len 24576 data: 3 2 2 2 3 4
22885 089/300 Read block 3049 len 28672
22886 090/300 Read block 464 len 4096
22887 091/300 Write block 3627 len 32768 data: 2 3 3 3 3 3 4 4
22888 092/300 Write block 4905 len 36864 data: 4 3 4 5 4 4 4 4 4
22889 093/300 Write block 31 len 8192 data: 2 2
22890 094/300 Write block 3506 len 20480 data: 3 3 3 4 4
22891 095/300 Read block 386 len 4096
22892 096/300 Write block 3786 len 36864 data: 3 3 3 3 3 3 3 3 2
22893 097/300 Write block 2950 len 28672 data: 4 5 6 6 6 5 5
22894 098/300 Read block 3883 len 24576
22895 099/300 Write block 3550 len 16384 data: 4 4 3 5
22896 100/300 Write block 2479 len 8192 data: 2 2
22897 101/300 Read block 2660 len 12288
22898 102/300 Read block 4066 len 16384
22899 103/300 Read block 795 len 40960
22900 104/300 Read block 145 len 4096
22901 105/300 Read block 4022 len 28672
22902 106/300 Write block 3750 len 12288 data: 5 4 4
22903 107/300 Write block 2610 len 40960 data: 5 7 5 5 4 3 3 4 4 5
22904 108/300 Write block 1594 len 28672 data: 6 6 8 9 10 7 8
22905 109/300 Write block 644 len 4096 data: 4
22906 110/300 Read block 1550 len 12288
22907 111/300 Write block 3101 len 32768 data: 4 4 6 8 7 8 8 8
22908 112/300 Read block 2607 len 36864
22909 113/300 Write block 4000 len 16384 data: 2 3 3 3
22910 114/300 Write block 4031 len 28672 data: 2 2 2 4 5 6 6
22911 115/300 Read block 3400 len 24576
22912 116/300 Read block 2564 len 20480
22913 117/300 Write block 3614 len 4096 data: 4
22914 118/300 Read block 2865 len 32768
22915 119/300 Write block 1918 len 36864 data: 3 3 4 4 3 4 6 6 5
22916 120/300 Read block 4552 len 36864
22917 121/300 Write block 960 len 24576 data: 5 5 4 4 4 3
22918 122/300 Write block 4485 len 4096 data: 4
22919 123/300 Read block 1013 len 12288
22920 124/300 Read block 4241 len 8192
22921 125/300 Write block 1167 len 16384 data: 5 5 5 5
22922 126/300 Flush
22923 127/300 Write block 2050 len 32768 data: 5 5 3 4 4 4 4 4
22924 128/300 Read block 3568 len 8192
22925 129/300 Write block 400 len 24576 data: 5 6 7 7 7 6
22926 130/300 Write block 116 len 8192 data: 8 7
22927 131/300 Read block 4759 len 28672
22928 132/300 Read block 326 len 36864
22929 133/300 Flush
22930 134/300 Write block 4636 len 12288 data: 4 4 5
22931 135/300 Read block 1096 len 40960
22932 136/300 Read block 4681 len 8192
22933 137/300 Write block 738 len 32768 data: 4 4 4 4 6 7 6 7
22934 138/300 Read block 3362 len 40960
22935 139/300 Write block 1814 len 32768 data: 7 9 8 7 5 5 7 6
22936 140/300 Flush
22937 141/300 Write block 1709 len 12288 data: 2 3 3
22938 142/300 Write block 4987 len 24576 data: 5 5 6 5 4 4
22939 143/300 Write block 954 len 36864 data: 3 3 6 6 5 5 6 6 5
22940 144/300 Flush
22941 145/300 Write block 1362 len 32768 data: 5 5 6 6 6 6 5 5
22942 146/300 Read block 4579 len 28672
22943 147/300 Read block 2094 len 12288
22944 148/300 Read block 195 len 16384
22945 149/300 Write block 3098 len 20480 data: 5 5 4 5 5
22946 150/300 Write block 3836 len 36864 data: 4 4 5 5 6 7 8 7 6
22947 151/300 Read block 392 len 4096
22948 152/300 Write block 2275 len 32768 data: 3 3 3 4 4 4 5 5
22949 153/300 Read block 3203 len 4096
22950 154/300 Write block 2011 len 12288 data: 7 5 6
22951 155/300 Write block 3337 len 16384 data: 6 5 5 5
22952 156/300 Write block 4825 len 28672 data: 5 5 4 2 5 6 5
22953 157/300 Read block 830 len 8192
22954 158/300 Read block 1049 len 4096
22955 159/300 Read block 3823 len 16384
22956 160/300 Flush
22957 161/300 Write block 1176 len 12288 data: 2 2 2
22958 162/300 Read block 4099 len 16384
22959 163/300 Read block 1019 len 36864
22960 164/300 Read block 1558 len 16384
22961 165/300 Write block 509 len 16384 data: 4 6 8 7
22962 166/300 Flush
22963 167/300 Write block 2302 len 24576 data: 5 6 6 6 4 5
22964 168/300 Read block 1195 len 8192
22965 169/300 Read block 787 len 12288
22966 170/300 Read block 1068 len 32768
22967 171/300 Read block 3546 len 4096
22968 172/300 Read block 1657 len 12288
22969 173/300 Read block 3007 len 40960
22970 174/300 Write block 325 len 32768 data: 5 4 4 4 4 3 3 3
22971 175/300 Read block 1532 len 16384
22972 176/300 Read block 1590 len 24576
22973 177/300 Write block 858 len 32768 data: 4 5 6 7 7 6 7 6
22974 178/300 Write block 1504 len 20480 data: 2 2 2 3 3
22975 179/300 Write block 2391 len 12288 data: 7 6 5
22976 180/300 Read block 400 len 12288
22977 181/300 Write block 182 len 12288 data: 6 7 8
22978 182/300 Write block 381 len 28672 data: 2 2 2 2 2 2 2
22979 183/300 Read block 1527 len 16384
22980 184/300 Read block 2540 len 28672
22981 185/300 Read block 255 len 12288
22982 186/300 Read block 1703 len 28672
22983 187/300 Read block 1099 len 40960
22984 188/300 Write block 2839 len 8192 data: 6 6
22985 189/300 Read block 2287 len 4096
22986 190/300 Write block 1729 len 20480 data: 5 4 3 2 2
22987 191/300 Flush
22988 192/300 Read block 3083 len 36864
22989 193/300 Write block 3802 len 4096 data: 2
22990 194/300 Write block 4247 len 16384 data: 7 7 7 8
22991 195/300 Flush
22992 196/300 Read block 369 len 8192
22993 197/300 Read block 4468 len 40960
22994 198/300 Write block 4576 len 4096 data: 4
22995 199/300 Read block 2519 len 24576
22996 200/300 Read block 123 len 8192
22997 201/300 Read block 3400 len 4096
22998 202/300 Read block 306 len 40960
22999 203/300 Read block 3634 len 28672
23000 204/300 Read block 4123 len 28672
23001 205/300 Read block 3408 len 4096
23002 206/300 Write block 1219 len 8192 data: 6 5
23003 207/300 Write block 91 len 20480 data: 4 4 3 4 4
23004 208/300 Read block 4591 len 12288
23005 209/300 Read block 1437 len 32768
23006 210/300 Write block 4526 len 36864 data: 6 6 6 5 5 6 6 6 6
23007 211/300 Read block 4194 len 20480
23008 212/300 Read block 2589 len 8192
23009 213/300 Flush
23010 214/300 Write block 3679 len 12288 data: 3 3 2
23011 215/300 Read block 2458 len 28672
23012 216/300 Read block 1068 len 24576
23013 217/300 Write block 138 len 4096 data: 8
23014 218/300 Read block 2474 len 20480
23015 219/300 Read block 3132 len 12288
23016 220/300 Read block 3867 len 36864
23017 221/300 Flush
23018 222/300 Read block 3323 len 4096
23019 223/300 Read block 3297 len 36864
23020 224/300 Write block 4909 len 32768 data: 5 5 5 5 5 4 4 5
23021 225/300 Write block 1581 len 12288 data: 5 5 4
23022 226/300 Write block 796 len 36864 data: 6 6 5 4 5 4 5 4 3
23023 227/300 Write block 942 len 8192 data: 4 3
23024 228/300 Read block 1438 len 4096
23025 229/300 Flush
23026 230/300 Read block 2011 len 40960
23027 231/300 Read block 4491 len 32768
23028 232/300 Read block 720 len 40960
23029 233/300 Read block 1657 len 24576
23030 234/300 Read block 2516 len 32768
23031 235/300 Read block 4267 len 8192
23032 236/300 Write block 554 len 28672 data: 7 4 3 4 4 5 5
23033 237/300 Read block 1643 len 40960
23034 238/300 Read block 4091 len 40960
23035 239/300 Read block 1712 len 12288
23036 240/300 Flush
23037 241/300 Read block 3317 len 4096
23038 242/300 Read block 2730 len 40960
23039 243/300 Read block 907 len 16384
23040 244/300 Flush
23041 245/300 Read block 1396 len 32768
23042 246/300 Flush
23043 247/300 Read block 1254 len 20480
23044 248/300 Write block 4614 len 40960 data: 5 6 6 6 6 6 5 4 3 3
23045 249/300 Write block 180 len 4096 data: 5
23046 250/300 Read block 4264 len 8192
23047 251/300 Write block 3621 len 24576 data: 7 6 7 7 4 3
23048 252/300 Read block 2988 len 8192
23049 253/300 Read block 2180 len 36864
23050 254/300 Read block 1987 len 36864
23051 255/300 Read block 3093 len 24576
23052 256/300 Write block 1863 len 36864 data: 5 4 5 6 4 3 3 5 5
23053 257/300 Write block 2586 len 28672 data: 3 4 4 4 5 5 5
23054 258/300 Read block 4728 len 16384
23055 259/300 Read block 1423 len 24576
23056 260/300 Flush
23057 261/300 Flush
23058 262/300 Write block 2811 len 12288 data: 6 6 5
23059 263/300 Flush
23060 264/300 Write block 1973 len 8192 data: 3 3
23061 265/300 Read block 1285 len 24576
23062 266/300 Write block 2729 len 28672 data: 4 4 4 4 3 3 3
23063 267/300 Write block 3645 len 20480 data: 4 5 4 5 6
23064 268/300 Write block 936 len 12288 data: 3 4 4
23065 269/300 Read block 4607 len 16384
23066 270/300 Write block 454 len 24576 data: 5 4 3 3 3 3
23067 271/300 Write block 3207 len 4096 data: 2
23068 272/300 Write block 2748 len 12288 data: 3 3 4
23069 273/300 Read block 1877 len 20480
23070 274/300 Flush
23071 275/300 Write block 684 len 32768 data: 7 7 7 6 6 6 5 4
23072 276/300 Read block 3557 len 24576
23073 277/300 Flush
23074 278/300 Write block 4736 len 8192 data: 8 8
23075 279/300 Read block 683 len 24576
23076 280/300 Read block 3022 len 4096
23077 281/300 Read block 3608 len 32768
23078 282/300 Write block 666 len 36864 data: 3 3 3 3 4 4 4 4 4
23079 283/300 Write block 2094 len 12288 data: 2 2 3
23080 284/300 Write block 1172 len 8192 data: 5 3
23081 285/300 Write block 4327 len 12288 data: 3 3 5
23082 286/300 Write block 2153 len 16384 data: 4 6 6 8
23083 287/300 Read block 4642 len 16384
23084 288/300 Read block 3838 len 8192
23085 289/300 Write block 272 len 28672 data: 5 5 5 6 6 6 7
23086 290/300 Read block 4656 len 20480
23087 291/300 Write block 2370 len 24576 data: 6 7 6 6 5 6
23088 292/300 Write block 4599 len 36864 data: 3 3 4 5 6 4 4 4 5
23089 293/300 Write block 4415 len 12288 data: 3 4 4
23090 294/300 Flush
23091 295/300 Read block 214 len 4096
23092 296/300 Read block 2870 len 32768
23093 297/300 Flush
23094 298/300 Read block 2661 len 8192
23095 299/300 Read block 4607 len 12288
23096 300/300 Read block 6 len 40960
23097 301/300 Flush
23098 Main task has work to do, go find it
23099 got dsc Start(2)
23100 start 2
23101 Jan 27 19:41:14.109 INFO request completed, latency_us: 162, response_code: 204, uri: /start/cid/2, method: POST, req_id: 8463033e-a51b-4464-b5dd-7215b10c4a63, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
23102 Replay: started 2, returned:Ok(())
23103 ----------------------------------------------------------------
23104 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23105 States: Active Active Offline
23106 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23107 7037 Acked Read 4 Done Done Sent false
23108 7038 Acked Read 5 Done Done Sent false
23109 7039 Acked Read 1 Done Done Sent false
23110 7040 Acked Write 6 Done Done Sent false
23111 7041 Acked Read 1 Done Done Sent false
23112 7042 Acked Read 1 Done Done Sent false
23113 7043 Acked Read 4 Done Done Sent false
23114 7044 Acked Read 7 Done Done Sent false
23115 7045 Acked Read 10 Done Done Sent false
23116 7046 Acked Write 2 Done Done Sent false
23117 7047 Acked Flush 0 Done Done Sent false
23118 7048 Acked Write 1 Done Done Sent false
23119 7049 Acked Read 1 Done Done Sent false
23120 7050 Acked Read 3 Done Done Sent false
23121 7051 Acked Write 9 Done Done Sent false
23122 7052 Acked Write 4 Done Done Sent false
23123 7053 Acked Flush 0 Done Done Sent false
23124 7054 Acked Write 6 Done Done Sent false
23125 7055 Acked Read 1 Done Done Sent false
23126 7056 Acked Flush 0 Done Done Sent false
23127 7057 Acked Read 3 Done Done Sent false
23128 7058 Acked Read 4 Done Done Sent false
23129 7059 Acked Read 7 Done Done Sent false
23130 7060 Acked Read 2 Done Done Sent false
23131 7061 Acked Read 6 Done Done Sent false
23132 7062 Acked Flush 0 Done Done Sent false
23133 7063 Acked Write 8 Done Done Sent false
23134 7064 Acked Write 5 Done Done Sent false
23135 7065 Acked Write 4 Done Done Sent false
23136 7066 Acked Write 4 Done Done Sent false
23137 7067 Acked Read 1 Done Done Sent false
23138 7068 Acked Read 6 Done Done Sent false
23139 7069 Acked Write 2 Done Done Sent false
23140 7070 Acked Read 5 Done Done Sent false
23141 7071 Acked Write 1 Done Done Sent false
23142 7072 Acked Read 5 Done Done Sent false
23143 7073 Acked Read 10 Done Done Sent false
23144 7074 Acked Write 10 Done Done Sent false
23145 7075 Acked Read 1 Done Done Sent false
23146 7076 Acked Write 6 Done Done Sent false
23147 7077 Acked Read 3 Done Done Sent false
23148 7078 Acked Read 3 Done Done Sent false
23149 7079 Acked Write 7 Done Done Sent false
23150 7080 Acked Write 1 Done Done Sent false
23151 7081 Acked Write 3 Done Done Sent false
23152 7082 Acked Read 1 Done Done Sent false
23153 7083 Acked Read 8 Done Done Sent false
23154 7084 Acked Write 8 Done Done Sent false
23155 7085 Acked Read 1 Done Done Sent false
23156 7086 Acked Read 10 Done Done Sent false
23157 7087 Acked Read 2 Done Done Sent false
23158 7088 Acked Write 4 Done Done Sent false
23159 7089 Acked Read 6 Done Done Sent false
23160 7090 Acked Write 7 Done Done Sent false
23161 7091 Acked Flush 0 Done Done Sent false
23162 7092 Acked Read 6 Done Done Sent false
23163 7093 Acked Read 9 Done Done Sent false
23164 7094 Acked Read 4 Done Done Sent false
23165 7095 Acked Flush 0 Done Done Sent false
23166 7096 Acked Read 4 Done Done Sent false
23167 7097 Acked Write 5 Done Done Sent false
23168 7098 Acked Read 6 Done Done Sent false
23169 7099 Acked Write 3 Done Done Sent false
23170 7100 Acked Read 1 Done Done Sent false
23171 7101 Acked Read 10 Done Done Sent false
23172 7102 Acked Write 6 Done Done Sent false
23173 7103 Acked Write 7 Done Done Sent false
23174 7104 Acked Read 6 Done Done Sent false
23175 7105 Acked Read 3 Done Done Sent false
23176 7106 Acked Flush 0 Done Done Sent false
23177 7107 Acked Read 8 Done Done Sent false
23178 7108 Acked Write 8 Done Done Sent false
23179 7109 Acked Read 7 Done Done Sent false
23180 7110 Acked Write 9 Done Done Sent false
23181 7111 Acked Read 5 Done Done Sent false
23182 7112 Acked Read 4 Done Done Sent false
23183 7113 Acked Write 6 Done Done Sent false
23184 7114 Acked Read 6 Done Done Sent false
23185 7115 Acked Write 10 Done Done Sent false
23186 7116 Acked Write 6 Done Done Sent false
23187 7117 Acked Write 5 Done Done Sent false
23188 7118 Acked Read 5 Done Done Sent false
23189 7119 Acked Write 10 Done Done Sent false
23190 7120 Acked Read 1 Done Done Sent false
23191 7121 Acked Write 6 Done Done Sent false
23192 7122 Acked Read 10 Done Done Sent false
23193 7123 Acked Read 8 Done Done Sent false
23194 7124 Acked Write 6 Done Done Sent false
23195 7125 Acked Read 7 Done Done Sent false
23196 7126 Acked Read 1 Done Done Sent false
23197 7127 Acked Write 8 Done Done Sent false
23198 7128 Acked Write 9 Done Done Sent false
23199 7129 Acked Write 2 Done Done Sent false
23200 7130 Acked Write 5 Done Done Sent false
23201 7131 Acked Read 1 Done Done Sent false
23202 7132 Acked Write 9 Done Done Sent false
23203 7133 Acked Write 7 Done Done Sent false
23204 7134 Acked Read 6 Done Done Sent false
23205 7135 Acked Write 4 Done Done Sent false
23206 7136 Acked Write 2 Done Done Sent false
23207 7137 Acked Read 3 Done Done Sent false
23208 7138 Acked Read 4 Done Done Sent false
23209 7139 Acked Read 10 Done Done Sent false
23210 7140 Acked Read 1 Done Done Sent false
23211 7141 Acked Read 7 Done Done Sent false
23212 7142 Acked Write 3 Done Done Sent false
23213 7143 Acked Write 10 Done Done Sent false
23214 7144 Acked Write 7 Done Done Sent false
23215 7145 Acked Write 1 Done Done Sent false
23216 7146 Acked Read 3 Done Done Sent false
23217 7147 Acked Write 8 Done Done Sent false
23218 7148 Acked Read 9 Done Done Sent false
23219 7149 Acked Write 4 Done Done Sent false
23220 7150 Acked Write 7 Done Done Sent false
23221 7151 Acked Read 6 Done Done Sent false
23222 7152 Acked Read 5 Done Done Sent false
23223 7153 Acked Write 1 Done Done Sent false
23224 7154 Acked Read 8 Done Done Sent false
23225 7155 Acked Write 9 Done Done Sent false
23226 7156 Acked Read 9 Done Done Sent false
23227 7157 Acked Write 6 Done Done Sent false
23228 7158 Acked Write 1 Done Done Sent false
23229 7159 Acked Read 3 Done Done Sent false
23230 7160 Acked Read 2 Done Done Sent false
23231 7161 Acked Write 4 Done Done Sent false
23232 7162 Acked Flush 0 Done Done Sent false
23233 7163 Acked Write 8 Done Done Sent false
23234 7164 Acked Read 2 Done Done Sent false
23235 7165 Acked Write 6 Done Done Sent false
23236 7166 Acked Write 2 Done Done Sent false
23237 7167 Acked Read 7 Done Done Sent false
23238 7168 Acked Read 9 Done Done Sent false
23239 7169 Acked Flush 0 Done Done Sent false
23240 7170 Acked Write 3 Done Done Sent false
23241 7171 Acked Read 10 Done Done Sent false
23242 7172 Acked Read 2 Done Done Sent false
23243 7173 Acked Write 8 Done Done Sent false
23244 7174 Acked Read 10 Done Done Sent false
23245 7175 Acked Write 8 Done Done Sent false
23246 7176 Acked Flush 0 Done Done Sent false
23247 7177 Acked Write 3 Done Done Sent false
23248 7178 Acked Write 6 Done Done Sent false
23249 7179 Acked Write 9 Done Done Sent false
23250 7180 Acked Flush 0 Done Done Sent false
23251 7181 Acked Write 8 Done Done Sent false
23252 7182 Acked Read 7 Done Done Sent false
23253 7183 Acked Read 3 Done Done Sent false
23254 7184 Acked Read 4 Done Done Sent false
23255 7185 Acked Write 5 Done Done Sent false
23256 7186 Acked Write 9 Done Done Sent false
23257 7187 Acked Read 1 Done Done Sent false
23258 7188 Acked Write 8 Done Done Sent false
23259 7189 Acked Read 1 Done Done Sent false
23260 7190 Acked Write 3 Done Done Sent false
23261 7191 Acked Write 4 Done Done Sent false
23262 7192 Acked Write 7 Done Done Sent false
23263 7193 Acked Read 2 Done Done Sent false
23264 7194 Acked Read 1 Done Done Sent false
23265 7195 Acked Read 4 Done Done Sent false
23266 7196 Acked Flush 0 Done Done Sent false
23267 7197 Acked Write 3 Done Done Sent false
23268 7198 Acked Read 4 Done Done Sent false
23269 7199 Acked Read 9 Done Done Sent false
23270 7200 Acked Read 4 Done Done Sent false
23271 7201 Acked Write 4 Done Done Sent false
23272 7202 Acked Flush 0 Done Done Sent false
23273 7203 Acked Write 6 Done Done Sent false
23274 7204 Acked Read 2 Done Done Sent false
23275 7205 Acked Read 3 Done Done Sent false
23276 7206 Acked Read 8 Done Done Sent false
23277 7207 Acked Read 1 Done Done Sent false
23278 7208 Acked Read 3 Done Done Sent false
23279 7209 Acked Read 10 Done Done Sent false
23280 7210 Acked Write 8 Done Done Sent false
23281 7211 Acked Read 4 Done Done Sent false
23282 7212 Acked Read 6 Done Done Sent false
23283 7213 Acked Write 8 Done Done Sent false
23284 7214 Acked Write 5 Done Done Sent false
23285 7215 Acked Write 3 Done Done Sent false
23286 7216 Acked Read 3 Done Done Sent false
23287 7217 Acked Write 3 Done Done Sent false
23288 7218 Acked Write 7 Done Done Sent false
23289 7219 Acked Read 4 Done Done Sent false
23290 7220 Acked Read 7 Done Done Sent false
23291 7221 Acked Read 3 Done Done Sent false
23292 7222 Acked Read 7 Done Done Sent false
23293 7223 Acked Read 10 Done Done Sent false
23294 7224 Acked Write 2 Done Done Sent false
23295 7225 Acked Read 1 Done Done Sent false
23296 7226 Acked Write 5 Done Done Sent false
23297 7227 Acked Flush 0 Done Done Sent false
23298 7228 Acked Read 9 Done Done Sent false
23299 7229 Acked Write 1 Done Done Sent false
23300 7230 Acked Write 4 Done Done Sent false
23301 7231 Acked Flush 0 Done Done Sent false
23302 7232 Acked Read 2 Done Done Sent false
23303 7233 Acked Read 10 Done Done Sent false
23304 7234 Acked Write 1 Done Done Sent false
23305 7235 Acked Read 6 Done Done Sent false
23306 7236 Acked Read 2 Done Done Sent false
23307 7237 Acked Read 1 Done Done Sent false
23308 7238 Acked Read 10 Done Done Sent false
23309 7239 Acked Read 7 Done Done Sent false
23310 7240 Acked Read 7 Done Done Sent false
23311 7241 Acked Read 1 Done Done Sent false
23312 7242 Acked Write 2 Done Done Sent false
23313 7243 Acked Write 5 Done Done Sent false
23314 7244 Acked Read 3 Done Done Sent false
23315 7245 Acked Read 8 Done Done Sent false
23316 7246 Acked Write 9 Done Done Sent false
23317 7247 Acked Read 5 Done Done Sent false
23318 7248 Acked Read 2 Done Done Sent false
23319 7249 Acked Flush 0 Done Done Sent false
23320 7250 Acked Write 3 Done Done Sent false
23321 7251 Acked Read 7 Done Done Sent false
23322 7252 Acked Read 6 Done Done Sent false
23323 7253 Acked Write 1 Done Done Sent false
23324 7254 Acked Read 5 Done Done Sent false
23325 7255 Acked Read 3 Done Done Sent false
23326 7256 Acked Read 9 Done Done Sent false
23327 7257 Acked Flush 0 Done Done Sent false
23328 7258 Acked Read 1 Done Done Sent false
23329 7259 Acked Read 9 Done Done Sent false
23330 7260 Acked Write 8 Done Done Sent false
23331 7261 Acked Write 3 Done Done Sent false
23332 7262 Acked Write 9 Done Done Sent false
23333 7263 Acked Write 2 Done Done Sent false
23334 7264 Acked Read 1 Done Done Sent false
23335 7265 Acked Flush 0 Done Done Sent false
23336 7266 Acked Read 10 Done Done Sent false
23337 7267 Acked Read 8 Done Done Sent false
23338 7268 Acked Read 10 Done Done Sent false
23339 7269 Acked Read 6 Done Done Sent false
23340 7270 Acked Read 8 Done Done Sent false
23341 7271 Acked Read 2 Done Done Sent false
23342 7272 Acked Write 7 Done Done Sent false
23343 7273 Acked Read 10 Done Done Sent false
23344 7274 Acked Read 10 Done Done Sent false
23345 7275 Acked Read 3 Done Done Sent false
23346 7276 Acked Flush 0 Done Done Sent false
23347 7277 Acked Read 1 Done Done Sent false
23348 7278 Acked Read 10 Done Done Sent false
23349 7279 Acked Read 4 Done Done Sent false
23350 7280 Acked Flush 0 Done Done Sent false
23351 7281 Acked Read 8 Done Done Sent false
23352 7282 Acked Flush 0 Done Done Sent false
23353 7283 Acked Read 5 Done Done Sent false
23354 7284 Acked Write 10 Done Done Sent false
23355 7285 Acked Write 1 Done Done Sent false
23356 7286 Acked Read 2 Done Done Sent false
23357 7287 Acked Write 6 Done Done Sent false
23358 7288 Acked Read 2 Done Done Sent false
23359 7289 Acked Read 9 Done Done Sent false
23360 7290 Acked Read 9 Done Done Sent false
23361 7291 Acked Read 6 Done Done Sent false
23362 7292 Acked Write 9 Done Done Sent false
23363 7293 Acked Write 7 Done Done Sent false
23364 7294 Acked Read 4 Done Done Sent false
23365 7295 Acked Read 6 Done Done Sent false
23366 7296 Acked Flush 0 Done Done Sent false
23367 7297 Acked Flush 0 Done Done Sent false
23368 7298 Acked Write 3 Done Done Sent false
23369 7299 Acked Flush 0 Done Done Sent false
23370 7300 Acked Write 2 Done Done Sent false
23371 7301 Acked Read 6 Done Done Sent false
23372 7302 Acked Write 7 Done Done Sent false
23373 7303 Acked Write 5 Done Done Sent false
23374 7304 Acked Write 3 Done Done Sent false
23375 7305 Acked Read 4 Done Done Sent false
23376 7306 Acked Write 6 Done Done Sent false
23377 7307 Acked Write 1 Done Done Sent false
23378 7308 Acked Write 3 Done Done Sent false
23379 7309 Acked Read 5 Done Done Sent false
23380 7310 Acked Flush 0 Done Done Sent false
23381 7311 Acked Write 8 Done Done Sent false
23382 7312 Acked Read 6 Done Done Sent false
23383 7313 Acked Flush 0 Done Done Sent false
23384 7314 Acked Write 2 Done Done Sent false
23385 7315 Acked Read 6 Done Done Sent false
23386 7316 Acked Read 1 Done Done Sent false
23387 7317 Acked Read 8 Done Done Sent false
23388 7318 Acked Write 9 Done Done Sent false
23389 7319 Acked Write 3 Done Done Sent false
23390 7320 Acked Write 2 Done Done Sent false
23391 7321 Acked Write 3 Done Done Sent false
23392 7322 Acked Write 4 Done Done Sent false
23393 7323 Acked Read 4 Done Done Sent false
23394 7324 Acked Read 2 Done Done Sent false
23395 7325 Acked Write 7 Done Done Sent false
23396 7326 Acked Read 5 Done Done Sent false
23397 7327 Acked Write 6 Done Done Sent false
23398 7328 Acked Write 9 Done Done Sent false
23399 7329 Acked Write 3 Done Done Sent false
23400 7330 Acked Flush 0 Done Done Sent false
23401 7331 Acked Read 1 Done Done Sent false
23402 7332 Acked Read 8 Done Done Sent false
23403 7333 Acked Flush 0 Done Done Sent false
23404 7334 Acked Read 2 Done Done Sent false
23405 7335 Acked Read 3 Done Done Sent false
23406 7336 Acked Read 10 Done Done Sent false
23407 7337 Acked Flush 0 Done Done Sent false
23408 STATES DS:0 DS:1 DS:2 TOTAL
23409 Sent 0 0 301 301
23410 Done 301 301 0 602
23411 Skipped 0 0 0 0
23412 Error 0 0 0 0
23413 Last Flush: 7337 7337 7036
23414 Downstairs last five completed: 7036 7035 7034 7033 7032
23415 Upstairs last five completed: 7337 7336 7335 7334 7333
23416 CLIENT: Up:0 ds:301 act:2
23417 [8830] got start action
23418 [8830] I am going to restart
23419 Starting downstairs at port 8830
23420 Make output file at "/tmp/dsc/downstairs-8830.txt"
23421 [8830][2] reports Starting
23422 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1221)
23423 [8830][2] reports Running
23424 ----------------------------------------------------------------
23425 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
23426 States: Active Active Offline
23427 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
23428 7037 Acked Read 4 Done Done Sent false
23429 7038 Acked Read 5 Done Done Sent false
23430 7039 Acked Read 1 Done Done Sent false
23431 7040 Acked Write 6 Done Done Sent false
23432 7041 Acked Read 1 Done Done Sent false
23433 7042 Acked Read 1 Done Done Sent false
23434 7043 Acked Read 4 Done Done Sent false
23435 7044 Acked Read 7 Done Done Sent false
23436 7045 Acked Read 10 Done Done Sent false
23437 7046 Acked Write 2 Done Done Sent false
23438 7047 Acked Flush 0 Done Done Sent false
23439 7048 Acked Write 1 Done Done Sent false
23440 7049 Acked Read 1 Done Done Sent false
23441 7050 Acked Read 3 Done Done Sent false
23442 7051 Acked Write 9 Done Done Sent false
23443 7052 Acked Write 4 Done Done Sent false
23444 7053 Acked Flush 0 Done Done Sent false
23445 7054 Acked Write 6 Done Done Sent false
23446 7055 Acked Read 1 Done Done Sent false
23447 7056 Acked Flush 0 Done Done Sent false
23448 7057 Acked Read 3 Done Done Sent false
23449 7058 Acked Read 4 Done Done Sent false
23450 7059 Acked Read 7 Done Done Sent false
23451 7060 Acked Read 2 Done Done Sent false
23452 7061 Acked Read 6 Done Done Sent false
23453 7062 Acked Flush 0 Done Done Sent false
23454 7063 Acked Write 8 Done Done Sent false
23455 7064 Acked Write 5 Done Done Sent false
23456 7065 Acked Write 4 Done Done Sent false
23457 7066 Acked Write 4 Done Done Sent false
23458 7067 Acked Read 1 Done Done Sent false
23459 7068 Acked Read 6 Done Done Sent false
23460 7069 Acked Write 2 Done Done Sent false
23461 7070 Acked Read 5 Done Done Sent false
23462 7071 Acked Write 1 Done Done Sent false
23463 7072 Acked Read 5 Done Done Sent false
23464 7073 Acked Read 10 Done Done Sent false
23465 7074 Acked Write 10 Done Done Sent false
23466 7075 Acked Read 1 Done Done Sent false
23467 7076 Acked Write 6 Done Done Sent false
23468 7077 Acked Read 3 Done Done Sent false
23469 7078 Acked Read 3 Done Done Sent false
23470 7079 Acked Write 7 Done Done Sent false
23471 7080 Acked Write 1 Done Done Sent false
23472 7081 Acked Write 3 Done Done Sent false
23473 7082 Acked Read 1 Done Done Sent false
23474 7083 Acked Read 8 Done Done Sent false
23475 7084 Acked Write 8 Done Done Sent false
23476 7085 Acked Read 1 Done Done Sent false
23477 7086 Acked Read 10 Done Done Sent false
23478 7087 Acked Read 2 Done Done Sent false
23479 7088 Acked Write 4 Done Done Sent false
23480 7089 Acked Read 6 Done Done Sent false
23481 7090 Acked Write 7 Done Done Sent false
23482 7091 Acked Flush 0 Done Done Sent false
23483 7092 Acked Read 6 Done Done Sent false
23484 7093 Acked Read 9 Done Done Sent false
23485 7094 Acked Read 4 Done Done Sent false
23486 7095 Acked Flush 0 Done Done Sent false
23487 7096 Acked Read 4 Done Done Sent false
23488 7097 Acked Write 5 Done Done Sent false
23489 7098 Acked Read 6 Done Done Sent false
23490 7099 Acked Write 3 Done Done Sent false
23491 7100 Acked Read 1 Done Done Sent false
23492 7101 Acked Read 10 Done Done Sent false
23493 7102 Acked Write 6 Done Done Sent false
23494 7103 Acked Write 7 Done Done Sent false
23495 7104 Acked Read 6 Done Done Sent false
23496 7105 Acked Read 3 Done Done Sent false
23497 7106 Acked Flush 0 Done Done Sent false
23498 7107 Acked Read 8 Done Done Sent false
23499 7108 Acked Write 8 Done Done Sent false
23500 7109 Acked Read 7 Done Done Sent false
23501 7110 Acked Write 9 Done Done Sent false
23502 7111 Acked Read 5 Done Done Sent false
23503 7112 Acked Read 4 Done Done Sent false
23504 7113 Acked Write 6 Done Done Sent false
23505 7114 Acked Read 6 Done Done Sent false
23506 7115 Acked Write 10 Done Done Sent false
23507 7116 Acked Write 6 Done Done Sent false
23508 7117 Acked Write 5 Done Done Sent false
23509 7118 Acked Read 5 Done Done Sent false
23510 7119 Acked Write 10 Done Done Sent false
23511 7120 Acked Read 1 Done Done Sent false
23512 7121 Acked Write 6 Done Done Sent false
23513 7122 Acked Read 10 Done Done Sent false
23514 7123 Acked Read 8 Done Done Sent false
23515 7124 Acked Write 6 Done Done Sent false
23516 7125 Acked Read 7 Done Done Sent false
23517 7126 Acked Read 1 Done Done Sent false
23518 7127 Acked Write 8 Done Done Sent false
23519 7128 Acked Write 9 Done Done Sent false
23520 7129 Acked Write 2 Done Done Sent false
23521 7130 Acked Write 5 Done Done Sent false
23522 7131 Acked Read 1 Done Done Sent false
23523 7132 Acked Write 9 Done Done Sent false
23524 7133 Acked Write 7 Done Done Sent false
23525 7134 Acked Read 6 Done Done Sent false
23526 7135 Acked Write 4 Done Done Sent false
23527 7136 Acked Write 2 Done Done Sent false
23528 7137 Acked Read 3 Done Done Sent false
23529 7138 Acked Read 4 Done Done Sent false
23530 7139 Acked Read 10 Done Done Sent false
23531 7140 Acked Read 1 Done Done Sent false
23532 7141 Acked Read 7 Done Done Sent false
23533 7142 Acked Write 3 Done Done Sent false
23534 7143 Acked Write 10 Done Done Sent false
23535 7144 Acked Write 7 Done Done Sent false
23536 7145 Acked Write 1 Done Done Sent false
23537 7146 Acked Read 3 Done Done Sent false
23538 7147 Acked Write 8 Done Done Sent false
23539 7148 Acked Read 9 Done Done Sent false
23540 7149 Acked Write 4 Done Done Sent false
23541 7150 Acked Write 7 Done Done Sent false
23542 7151 Acked Read 6 Done Done Sent false
23543 7152 Acked Read 5 Done Done Sent false
23544 7153 Acked Write 1 Done Done Sent false
23545 7154 Acked Read 8 Done Done Sent false
23546 7155 Acked Write 9 Done Done Sent false
23547 7156 Acked Read 9 Done Done Sent false
23548 7157 Acked Write 6 Done Done Sent false
23549 7158 Acked Write 1 Done Done Sent false
23550 7159 Acked Read 3 Done Done Sent false
23551 7160 Acked Read 2 Done Done Sent false
23552 7161 Acked Write 4 Done Done Sent false
23553 7162 Acked Flush 0 Done Done Sent false
23554 7163 Acked Write 8 Done Done Sent false
23555 7164 Acked Read 2 Done Done Sent false
23556 7165 Acked Write 6 Done Done Sent false
23557 7166 Acked Write 2 Done Done Sent false
23558 7167 Acked Read 7 Done Done Sent false
23559 7168 Acked Read 9 Done Done Sent false
23560 7169 Acked Flush 0 Done Done Sent false
23561 7170 Acked Write 3 Done Done Sent false
23562 7171 Acked Read 10 Done Done Sent false
23563 7172 Acked Read 2 Done Done Sent false
23564 7173 Acked Write 8 Done Done Sent false
23565 7174 Acked Read 10 Done Done Sent false
23566 7175 Acked Write 8 Done Done Sent false
23567 7176 Acked Flush 0 Done Done Sent false
23568 7177 Acked Write 3 Done Done Sent false
23569 7178 Acked Write 6 Done Done Sent false
23570 7179 Acked Write 9 Done Done Sent false
23571 7180 Acked Flush 0 Done Done Sent false
23572 7181 Acked Write 8 Done Done Sent false
23573 7182 Acked Read 7 Done Done Sent false
23574 7183 Acked Read 3 Done Done Sent false
23575 7184 Acked Read 4 Done Done Sent false
23576 7185 Acked Write 5 Done Done Sent false
23577 7186 Acked Write 9 Done Done Sent false
23578 7187 Acked Read 1 Done Done Sent false
23579 7188 Acked Write 8 Done Done Sent false
23580 7189 Acked Read 1 Done Done Sent false
23581 7190 Acked Write 3 Done Done Sent false
23582 7191 Acked Write 4 Done Done Sent false
23583 7192 Acked Write 7 Done Done Sent false
23584 7193 Acked Read 2 Done Done Sent false
23585 7194 Acked Read 1 Done Done Sent false
23586 7195 Acked Read 4 Done Done Sent false
23587 7196 Acked Flush 0 Done Done Sent false
23588 7197 Acked Write 3 Done Done Sent false
23589 7198 Acked Read 4 Done Done Sent false
23590 7199 Acked Read 9 Done Done Sent false
23591 7200 Acked Read 4 Done Done Sent false
23592 7201 Acked Write 4 Done Done Sent false
23593 7202 Acked Flush 0 Done Done Sent false
23594 7203 Acked Write 6 Done Done Sent false
23595 7204 Acked Read 2 Done Done Sent false
23596 7205 Acked Read 3 Done Done Sent false
23597 7206 Acked Read 8 Done Done Sent false
23598 7207 Acked Read 1 Done Done Sent false
23599 7208 Acked Read 3 Done Done Sent false
23600 7209 Acked Read 10 Done Done Sent false
23601 7210 Acked Write 8 Done Done Sent false
23602 7211 Acked Read 4 Done Done Sent false
23603 7212 Acked Read 6 Done Done Sent false
23604 7213 Acked Write 8 Done Done Sent false
23605 7214 Acked Write 5 Done Done Sent false
23606 7215 Acked Write 3 Done Done Sent false
23607 7216 Acked Read 3 Done Done Sent false
23608 7217 Acked Write 3 Done Done Sent false
23609 7218 Acked Write 7 Done Done Sent false
23610 7219 Acked Read 4 Done Done Sent false
23611 7220 Acked Read 7 Done Done Sent false
23612 7221 Acked Read 3 Done Done Sent false
23613 7222 Acked Read 7 Done Done Sent false
23614 7223 Acked Read 10 Done Done Sent false
23615 7224 Acked Write 2 Done Done Sent false
23616 7225 Acked Read 1 Done Done Sent false
23617 7226 Acked Write 5 Done Done Sent false
23618 7227 Acked Flush 0 Done Done Sent false
23619 7228 Acked Read 9 Done Done Sent false
23620 7229 Acked Write 1 Done Done Sent false
23621 7230 Acked Write 4 Done Done Sent false
23622 7231 Acked Flush 0 Done Done Sent false
23623 7232 Acked Read 2 Done Done Sent false
23624 7233 Acked Read 10 Done Done Sent false
23625 7234 Acked Write 1 Done Done Sent false
23626 7235 Acked Read 6 Done Done Sent false
23627 7236 Acked Read 2 Done Done Sent false
23628 7237 Acked Read 1 Done Done Sent false
23629 7238 Acked Read 10 Done Done Sent false
23630 7239 Acked Read 7 Done Done Sent false
23631 7240 Acked Read 7 Done Done Sent false
23632 7241 Acked Read 1 Done Done Sent false
23633 7242 Acked Write 2 Done Done Sent false
23634 7243 Acked Write 5 Done Done Sent false
23635 7244 Acked Read 3 Done Done Sent false
23636 7245 Acked Read 8 Done Done Sent false
23637 7246 Acked Write 9 Done Done Sent false
23638 7247 Acked Read 5 Done Done Sent false
23639 7248 Acked Read 2 Done Done Sent false
23640 7249 Acked Flush 0 Done Done Sent false
23641 7250 Acked Write 3 Done Done Sent false
23642 7251 Acked Read 7 Done Done Sent false
23643 7252 Acked Read 6 Done Done Sent false
23644 7253 Acked Write 1 Done Done Sent false
23645 7254 Acked Read 5 Done Done Sent false
23646 7255 Acked Read 3 Done Done Sent false
23647 7256 Acked Read 9 Done Done Sent false
23648 7257 Acked Flush 0 Done Done Sent false
23649 7258 Acked Read 1 Done Done Sent false
23650 7259 Acked Read 9 Done Done Sent false
23651 7260 Acked Write 8 Done Done Sent false
23652 7261 Acked Write 3 Done Done Sent false
23653 7262 Acked Write 9 Done Done Sent false
23654 7263 Acked Write 2 Done Done Sent false
23655 7264 Acked Read 1 Done Done Sent false
23656 7265 Acked Flush 0 Done Done Sent false
23657 7266 Acked Read 10 Done Done Sent false
23658 7267 Acked Read 8 Done Done Sent false
23659 7268 Acked Read 10 Done Done Sent false
23660 7269 Acked Read 6 Done Done Sent false
23661 7270 Acked Read 8 Done Done Sent false
23662 7271 Acked Read 2 Done Done Sent false
23663 7272 Acked Write 7 Done Done Sent false
23664 7273 Acked Read 10 Done Done Sent false
23665 7274 Acked Read 10 Done Done Sent false
23666 7275 Acked Read 3 Done Done Sent false
23667 7276 Acked Flush 0 Done Done Sent false
23668 7277 Acked Read 1 Done Done Sent false
23669 7278 Acked Read 10 Done Done Sent false
23670 7279 Acked Read 4 Done Done Sent false
23671 7280 Acked Flush 0 Done Done Sent false
23672 7281 Acked Read 8 Done Done Sent false
23673 7282 Acked Flush 0 Done Done Sent false
23674 7283 Acked Read 5 Done Done Sent false
23675 7284 Acked Write 10 Done Done Sent false
23676 7285 Acked Write 1 Done Done Sent false
23677 7286 Acked Read 2 Done Done Sent false
23678 7287 Acked Write 6 Done Done Sent false
23679 7288 Acked Read 2 Done Done Sent false
23680 7289 Acked Read 9 Done Done Sent false
23681 7290 Acked Read 9 Done Done Sent false
23682 7291 Acked Read 6 Done Done Sent false
23683 7292 Acked Write 9 Done Done Sent false
23684 7293 Acked Write 7 Done Done Sent false
23685 7294 Acked Read 4 Done Done Sent false
23686 7295 Acked Read 6 Done Done Sent false
23687 7296 Acked Flush 0 Done Done Sent false
23688 7297 Acked Flush 0 Done Done Sent false
23689 7298 Acked Write 3 Done Done Sent false
23690 7299 Acked Flush 0 Done Done Sent false
23691 7300 Acked Write 2 Done Done Sent false
23692 7301 Acked Read 6 Done Done Sent false
23693 7302 Acked Write 7 Done Done Sent false
23694 7303 Acked Write 5 Done Done Sent false
23695 7304 Acked Write 3 Done Done Sent false
23696 7305 Acked Read 4 Done Done Sent false
23697 7306 Acked Write 6 Done Done Sent false
23698 7307 Acked Write 1 Done Done Sent false
23699 7308 Acked Write 3 Done Done Sent false
23700 7309 Acked Read 5 Done Done Sent false
23701 7310 Acked Flush 0 Done Done Sent false
23702 7311 Acked Write 8 Done Done Sent false
23703 7312 Acked Read 6 Done Done Sent false
23704 7313 Acked Flush 0 Done Done Sent false
23705 7314 Acked Write 2 Done Done Sent false
23706 7315 Acked Read 6 Done Done Sent false
23707 7316 Acked Read 1 Done Done Sent false
23708 7317 Acked Read 8 Done Done Sent false
23709 7318 Acked Write 9 Done Done Sent false
23710 7319 Acked Write 3 Done Done Sent false
23711 7320 Acked Write 2 Done Done Sent false
23712 7321 Acked Write 3 Done Done Sent false
23713 7322 Acked Write 4 Done Done Sent false
23714 7323 Acked Read 4 Done Done Sent false
23715 7324 Acked Read 2 Done Done Sent false
23716 7325 Acked Write 7 Done Done Sent false
23717 7326 Acked Read 5 Done Done Sent false
23718 7327 Acked Write 6 Done Done Sent false
23719 7328 Acked Write 9 Done Done Sent false
23720 7329 Acked Write 3 Done Done Sent false
23721 7330 Acked Flush 0 Done Done Sent false
23722 7331 Acked Read 1 Done Done Sent false
23723 7332 Acked Read 8 Done Done Sent false
23724 7333 Acked Flush 0 Done Done Sent false
23725 7334 Acked Read 2 Done Done Sent false
23726 7335 Acked Read 3 Done Done Sent false
23727 7336 Acked Read 10 Done Done Sent false
23728 7337 Acked Flush 0 Done Done Sent false
23729 STATES DS:0 DS:1 DS:2 TOTAL
23730 Sent 0 0 301 301
23731 Done 301 301 0 602
23732 Skipped 0 0 0 0
23733 Error 0 0 0 0
23734 Last Flush: 7337 7337 7036
23735 Downstairs last five completed: 7036 7035 7034 7033 7032
23736 Upstairs last five completed: 7337 7336 7335 7334 7333
23737 CLIENT: Up:0 ds:301 act:2
237382025-01-27T19:41:19.874ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
237392025-01-27T19:41:19.874ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:35952) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
237402025-01-27T19:41:19.875ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
237412025-01-27T19:41:19.876ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
237422025-01-27T19:41:19.876ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
237432025-01-27T19:41:19.876ZINFOcrucible: send last flush ID to this DS: 7036 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
237442025-01-27T19:41:19.876ZINFOcrucible: Replied this last flush ID: 7036 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
237452025-01-27T19:41:19.876ZINFOcrucible: [2] client re-new 301 jobs since flush 7036 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
237462025-01-27T19:41:19.876ZINFOcrucible: [2] Replayed 301 jobs since flush: 7036 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
23747 ----------------------------------------------------------------
23748 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
23749 Downstairs last five completed: 7337 7336 7335 7334 7333
23750 Upstairs last five completed: 7337 7336 7335 7334 7333
23751 CLIENT: Up:0 ds:0 act:3
23752 Replay: All jobs finished
23753 Main task has work to do, go find it
23754 got dsc Stop(1)
23755 stop 1
23756 [8820] Got stop action so:false kr:false
23757 Jan 27 19:41:22.116 INFO request completed, latency_us: 360, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 3bd20715-5187-4381-af2c-d8bc343479ff, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
237582025-01-27T19:41:22.116ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
23759 Jan 27 19:41:22.116 INFO request completed, latency_us: 133, response_code{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:41:22.116863333Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"1","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
23760 : 200, uri: /state/cid/1, method: GET, req_id: {"msg":"downstairs task for 1 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:41:22.116896319Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
23761 274f2289-c467-4fca-933a-623153329301, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
237622025-01-27T19:41:22.116ZINFOcrucible: client task is exiting = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
23763 [8820] Exited with: ExitStatus(unix_wait_status(9))
23764 [8820][1] reports Exit
23765 Jan 27 19:41:26.119 INFO request completed, latency_us: 328, response_code: 200, uri: /state/cid/1, method: GET, req_id: bf907f22-54e0-4b40-b839-e29ae35b5399, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
23766 001/300 Flush
23767 002/300 Read block 390 len 4096
23768 003/300 Read block 1495 len 32768
23769 004/300 Write block 778 len 8192 data: 5 4
23770 005/300 Flush
23771 006/300 Read block 2278 len 4096
23772 007/300 Write block 4925 len 16384 data: 8 7 8 8
23773 008/300 Read block 1034 len 8192
23774 009/300 Flush
23775 010/300 Read block 1728 len 24576
23776 011/300 Write block 1358 len 28672 data: 9 9 7 7 6 6 7
23777 012/300 Write block 608 len 28672 data: 5 5 4 3 3 4 3
23778 013/300 Read block 4628 len 8192
23779 014/300 Read block 4523 len 4096
23780 015/300 Write block 48 len 36864 data: 7 7 7 7 6 4 6 4 4
23781 016/300 Write block 1612 len 16384 data: 4 4 4 4
23782 017/300 Write block 1951 len 4096 data: 4
23783 018/300 Write block 327 len 24576 data: 5 5 5 4 4 4
23784 019/300 Write block 4672 len 24576 data: 4 4 3 3 3 3
23785 020/300 Write block 906 len 12288 data: 4 2 3
23786 021/300 Read block 2488 len 36864
23787 022/300 Read block 1005 len 12288
23788 023/300 Write block 4680 len 12288 data: 6 7 5
23789 024/300 Write block 3000 len 32768 data: 5 5 6 6 6 5 5 5
23790 025/300 Write block 122 len 40960 data: 7 7 7 4 4 4 4 5 6 8
23791 026/300 Read block 775 len 16384
23792 027/300 Read block 1706 len 20480
23793 028/300 Read block 283 len 12288
23794 029/300 Read block 1785 len 16384
23795 030/300 Read block 1989 len 8192
23796 031/300 Write block 1183 len 20480 data: 2 3 4 3 3
23797 032/300 Write block 1847 len 28672 data: 4 5 5 5 6 6 6
23798 033/300 Write block 1254 len 40960 data: 4 3 4 3 4 3 3 3 3 4
23799 034/300 Flush
23800 035/300 Read block 4882 len 40960
23801 036/300 Write block 2514 len 8192 data: 5 5
23802 037/300 Write block 1403 len 24576 data: 6 5 8 8 7 7
23803 038/300 Flush
23804 039/300 Read block 4922 len 32768
23805 040/300 Read block 1074 len 12288
23806 041/300 Read block 1767 len 16384
23807 042/300 Read block 4211 len 40960
23808 043/300 Read block 3520 len 4096
23809 044/300 Write block 4713 len 28672 data: 6 6 5 5 5 7 4
23810 045/300 Flush
23811 046/300 Write block 1941 len 36864 data: 5 5 5 4 4 4 4 4 3
23812 047/300 Write block 4867 len 20480 data: 8 7 8 7 5
23813 048/300 Read block 3187 len 28672
23814 049/300 Write block 955 len 32768 data: 4 7 7 6 6 7 7 6
23815 050/300 Read block 2121 len 16384
23816 051/300 Write block 1143 len 32768 data: 3 2 4 4 5 6 6 4
23817 052/300 Write block 1093 len 32768 data: 3 3 4 4 4 6 6 6
23818 053/300 Read block 2124 len 36864
23819 054/300 Read block 3769 len 40960
23820 055/300 Write block 396 len 8192 data: 5 7
23821 056/300 Write block 1516 len 12288 data: 4 4 2
23822 057/300 Write block 627 len 32768 data: 4 4 4 5 4 4 4 4
23823 058/300 Read block 3750 len 4096
23824 059/300 Read block 3337 len 32768
23825 060/300 Read block 1770 len 28672
23826 061/300 Read block 311 len 12288
23827 062/300 Read block 4440 len 12288
23828 063/300 Read block 3719 len 16384
23829 064/300 Write block 2144 len 20480 data: 7 6 6 4 3
23830 065/300 Read block 2952 len 12288
23831 066/300 Flush
23832 067/300 Read block 2814 len 28672
23833 068/300 Read block 1158 len 40960
23834 069/300 Read block 3525 len 4096
23835 070/300 Write block 856 len 24576 data: 3 3 5 6 7 8
23836 071/300 Read block 1134 len 36864
23837 072/300 Write block 918 len 40960 data: 4 6 7 7 6 4 5 5 4 3
23838 073/300 Read block 2575 len 16384
23839 074/300 Write block 2282 len 36864 data: 6 6 5 6 7 8 11 10 10
23840 075/300 Read block 811 len 16384
23841 076/300 Read block 3006 len 40960
23842 077/300 Write block 3355 len 12288 data: 3 3 3
23843 078/300 Write block 2628 len 28672 data: 5 5 4 4 5 7 5
23844 079/300 Write block 4683 len 20480 data: 5 4 4 5 4
23845 080/300 Read block 1387 len 32768
23846 081/300 Write block 1303 len 12288 data: 7 7 7
23847 082/300 Read block 1178 len 4096
23848 083/300 Write block 2516 len 20480 data: 5 4 4 3 3
23849 084/300 Write block 3924 len 36864 data: 5 4 5 5 5 4 4 5 5
23850 085/300 Read block 3228 len 36864
23851 086/300 Write block 914 len 32768 data: 4 2 3 3 5 7 8 8
23852 087/300 Read block 2712 len 32768
23853 088/300 Read block 38 len 4096
23854 089/300 Read block 194 len 16384
23855 090/300 Write block 4369 len 12288 data: 4 3 3
23856 091/300 Read block 1037 len 32768
23857 092/300 Read block 1 len 20480
23858 093/300 Flush
23859 094/300 Write block 1088 len 28672 data: 2 3 4 4 2 4 4
23860 095/300 Write block 470 len 28672 data: 5 5 3 3 3 4 4
23861 096/300 Read block 1042 len 20480
23862 097/300 Read block 756 len 20480
23863 098/300 Read block 234 len 24576
23864 099/300 Read block 1882 len 28672
23865 100/300 Write block 539 len 20480 data: 3 2 2 3 3
23866 101/300 Write block 4297 len 36864 data: 4 4 4 3 3 5 4 5 4
23867 102/300 Write block 1966 len 8192 data: 6 6
23868 103/300 Flush
23869 104/300 Flush
23870 105/300 Read block 144 len 40960
23871 106/300 Write block 1496 len 8192 data: 2 2
23872 107/300 Read block 4280 len 32768
23873 108/300 Read block 4959 len 4096
23874 109/300 Write block 3798 len 12288 data: 2 2 2
23875 110/300 Write block 1840 len 8192 data: 3 3
23876 111/300 Read block 1854 len 36864
23877 112/300 Read block 1283 len 20480
23878 113/300 Read block 2798 len 40960
23879 114/300 Read block 0 len 16384
23880 115/300 Read block 2745 len 4096
23881 116/300 Read block 4889 len 32768
23882 117/300 Write block 799 len 20480 data: 5 6 5 6 5
23883 118/300 Write block 1139 len 4096 data: 4
23884 119/300 Read block 4386 len 12288
23885 120/300 Write block 2813 len 16384 data: 6 5 4 3
23886 121/300 Write block 1231 len 4096 data: 3
23887 122/300 Read block 4034 len 20480
23888 123/300 Read block 4336 len 40960
23889 124/300 Read block 2714 len 20480
23890 125/300 Write block 1577 len 24576 data: 5 5 4 6 6 6
23891 126/300 Write block 2826 len 32768 data: 5 4 6 6 7 7 8 8
23892 127/300 Read block 114 len 12288
23893 128/300 Read block 628 len 16384
23894 129/300 Flush
23895 130/300 Read block 2509 len 20480
23896 131/300 Read block 193 len 4096
23897 132/300 Read block 2392 len 8192
23898 133/300 Read block 3147 len 36864
23899 134/300 Read block 2409 len 36864
23900 135/300 Read block 4971 len 8192
23901 136/300 Read block 4422 len 8192
23902 137/300 Write block 1621 len 8192 data: 5 5
23903 138/300 Read block 1305 len 12288
23904 139/300 Read block 3872 len 24576
23905 140/300 Write block 1629 len 8192 data: 7 6
23906 141/300 Write block 4314 len 32768 data: 2 2 2 2 2 2 3 4
23907 142/300 Write block 2520 len 24576 data: 4 3 3 3 4 3
23908 143/300 Read block 554 len 28672
23909 144/300 Write block 614 len 24576 data: 4 3 3 2 2 3
23910 145/300 Flush
23911 146/300 Write block 1120 len 36864 data: 6 7 9 10 9 7 7 8 8
23912 147/300 Flush
23913 148/300 Write block 3692 len 4096 data: 5
23914 149/300 Read block 257 len 4096
23915 150/300 Write block 1475 len 40960 data: 4 4 3 4 6 6 6 6 4 3
23916 151/300 Write block 181 len 16384 data: 5 7 8 9
23917 152/300 Read block 231 len 40960
23918 153/300 Read block 626 len 28672
23919 154/300 Write block 1135 len 12288 data: 4 3 4
23920 155/300 Write block 165 len 40960 data: 4 6 6 9 9 9 9 6 4 4
23921 156/300 Write block 1978 len 8192 data: 5 3
23922 157/300 Write block 4526 len 28672 data: 7 7 7 6 6 7 7
23923 158/300 Read block 2438 len 32768
23924 159/300 Read block 2941 len 8192
23925 160/300 Read block 597 len 4096
23926 161/300 Write block 3672 len 24576 data: 4 4 4 3 3 3
23927 162/300 Write block 502 len 28672 data: 8 6 5 7 7 6 5
23928 163/300 Write block 2550 len 40960 data: 4 4 4 4 4 5 5 6 6 5
23929 164/300 Flush
23930 165/300 Read block 1252 len 16384
23931 166/300 Read block 2590 len 40960
23932 167/300 Flush
23933 168/300 Read block 3915 len 32768
23934 169/300 Read block 345 len 4096
23935 170/300 Read block 915 len 12288
23936 171/300 Read block 400 len 36864
23937 172/300 Write block 2660 len 28672 data: 5 5 4 5 5 5 4
23938 173/300 Read block 4657 len 36864
23939 174/300 Write block 111 len 8192 data: 2 2
23940 175/300 Read block 3492 len 4096
23941 176/300 Write block 2365 len 40960 data: 3 3 3 4 5 7 8 7 7 6
23942 177/300 Read block 4309 len 4096
23943 178/300 Write block 1297 len 28672 data: 6 7 6 5 7 6 8
23944 179/300 Read block 1711 len 32768
23945 180/300 Write block 4473 len 36864 data: 4 5 8 8 8 7 4 7 7
23946 181/300 Read block 4273 len 40960
23947 182/300 Read block 1601 len 32768
23948 183/300 Write block 2619 len 24576 data: 6 6 7 7 7 8
23949 184/300 Read block 2909 len 16384
23950 185/300 Read block 3384 len 24576
23951 186/300 Flush
23952 187/300 Read block 2193 len 32768
23953 188/300 Write block 1531 len 24576 data: 6 5 4 4 4 4
23954 189/300 Write block 2010 len 8192 data: 5 8
23955 190/300 Flush
23956 191/300 Read block 78 len 36864
23957 192/300 Write block 2959 len 28672 data: 3 4 4 6 6 6 6
23958 193/300 Read block 4164 len 28672
23959 194/300 Write block 3084 len 28672 data: 3 2 2 2 3 2 3
23960 195/300 Read block 4499 len 8192
23961 196/300 Flush
23962 197/300 Read block 2026 len 12288
23963 198/300 Read block 2692 len 32768
23964 199/300 Flush
23965 200/300 Read block 102 len 24576
23966 201/300 Read block 265 len 4096
23967 202/300 Read block 2641 len 20480
23968 203/300 Read block 3274 len 12288
23969 204/300 Read block 2767 len 20480
23970 205/300 Read block 3805 len 24576
23971 206/300 Read block 2148 len 8192
23972 207/300 Read block 4519 len 8192
23973 208/300 Read block 1451 len 24576
23974 209/300 Read block 4267 len 24576
23975 210/300 Flush
23976 211/300 Write block 2758 len 24576 data: 3 3 3 3 3 4
23977 212/300 Write block 4347 len 16384 data: 5 7 8 7
23978 213/300 Read block 4316 len 32768
23979 214/300 Read block 1733 len 12288
23980 215/300 Read block 778 len 28672
23981 216/300 Write block 1498 len 40960 data: 2 2 2 2 2 2 3 3 3 4
23982 217/300 Read block 4367 len 28672
23983 218/300 Flush
23984 219/300 Flush
23985 220/300 Read block 343 len 8192
23986 221/300 Write block 3720 len 20480 data: 4 5 4 4 5
23987 222/300 Write block 3804 len 36864 data: 2 3 3 3 3 5 6 5 4
23988 223/300 Read block 3749 len 4096
23989 224/300 Read block 1468 len 12288
23990 225/300 Write block 3175 len 32768 data: 8 9 9 9 9 7 7 6
23991 226/300 Read block 4508 len 4096
23992 227/300 Read block 418 len 16384
23993 228/300 Read block 4948 len 28672
23994 229/300 Read block 282 len 36864
23995 230/300 Read block 2640 len 20480
23996 231/300 Write block 3762 len 28672 data: 7 4 4 5 5 6 6
23997 232/300 Write block 4494 len 28672 data: 6 6 4 4 6 6 6
23998 233/300 Write block 2401 len 36864 data: 5 5 4 4 5 5 7 8 7
23999 234/300 Read block 1464 len 12288
24000 235/300 Read block 2633 len 16384
24001 236/300 Read block 2250 len 4096
24002 237/300 Write block 1390 len 40960 data: 6 7 5 4 6 5 5 6 7 8
24003 238/300 Write block 2180 len 8192 data: 5 5
24004 239/300 Flush
24005 240/300 Read block 3488 len 32768
24006 241/300 Write block 2042 len 28672 data: 8 8 6 6 6 5 5
24007 242/300 Write block 3888 len 32768 data: 5 5 3 2 2 4 5 5
24008 243/300 Write block 4658 len 8192 data: 7 6
24009 244/300 Write block 500 len 12288 data: 8 9 9
24010 245/300 Read block 493 len 8192
24011 246/300 Read block 3455 len 16384
24012 247/300 Read block 2337 len 28672
24013 248/300 Read block 3587 len 16384
24014 249/300 Write block 1725 len 36864 data: 4 6 6 5 6 5 4 3 3
24015 250/300 Read block 2754 len 20480
24016 251/300 Read block 2984 len 40960
24017 252/300 Write block 1357 len 12288 data: 7 10 10
24018 253/300 Read block 1596 len 40960
24019 254/300 Write block 2709 len 40960 data: 4 5 5 5 6 5 6 5 4 7
24020 255/300 Flush
24021 256/300 Read block 4497 len 32768
24022 257/300 Flush
24023 258/300 Read block 2398 len 20480
24024 259/300 Write block 4991 len 28672 data: 5 5 5 4 5 5 4
24025 260/300 Read block 4847 len 36864
24026 261/300 Read block 3964 len 16384
24027 262/300 Write block 4397 len 32768 data: 2 2 2 2 3 2 3 3
24028 263/300 Read block 66 len 36864
24029 264/300 Read block 3760 len 36864
24030 265/300 Flush
24031 266/300 Read block 3312 len 24576
24032 267/300 Write block 2475 len 28672 data: 5 5 4 2 3 3 2
24033 268/300 Read block 4365 len 20480
24034 269/300 Read block 592 len 40960
24035 270/300 Read block 1183 len 28672
24036 271/300 Flush
24037 272/300 Read block 6 len 4096
24038 273/300 Read block 3475 len 16384
24039 274/300 Flush
24040 275/300 Read block 2828 len 20480
24041 276/300 Read block 3206 len 32768
24042 277/300 Write block 1066 len 32768 data: 4 3 5 4 5 5 7 6
24043 278/300 Write block 4348 len 20480 data: 8 9 8 7 7
24044 279/300 Read block 4058 len 20480
24045 280/300 Write block 4167 len 28672 data: 7 6 5 5 4 4 6
24046 281/300 Read block 292 len 20480
24047 282/300 Write block 3221 len 28672 data: 6 7 6 6 5 4 4
24048 283/300 Read block 4948 len 32768
24049 284/300 Read block 3785 len 8192
24050 285/300 Write block 599 len 28672 data: 7 6 6 8 8 7 6
24051 286/300 Write block 1542 len 12288 data: 5 7 7
24052 287/300 Write block 1539 len 12288 data: 5 5 5
24053 288/300 Read block 2178 len 32768
24054 289/300 Read block 3057 len 20480
24055 290/300 Read block 160 len 28672
24056 291/300 Flush
24057 292/300 Flush
24058 293/300 Write block 1195 len 16384 data: 4 6 7 7
24059 294/300 Write block 2191 len 8192 data: 4 4
24060 295/300 Read block 4767 len 40960
24061 296/300 Write block 3386 len 28672 data: 3 4 5 6 5 5 4
24062 297/300 Write block 900 len 36864 data: 3 4 4 5 6 4 5 3 4
24063 298/300 Flush
24064 299/300 Read block 3599 len 20480
24065 300/300 Write block 295 len 20480 data: 5 5 5 5 5
24066 301/300 Read block 1023 len 40960
24067 Main task has work to do, go find it
24068 got dsc Start(1)
24069 start 1
24070 Jan 27 19:41:26.360 INFO request completed, latency_us: 169, response_code: 204, uri: /start/cid/1, method: POST, req_id: 2944104e-a640-4f71-9c7f-02530711e025, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
24071 Replay: started 1, returned:Ok(())
24072 ----------------------------------------------------------------
24073 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
24074 States: Active Offline Active
24075 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24076 7338 Acked Flush 0 Done Sent Done false
24077 7339 Acked Read 1 Done Sent Done false
24078 7340 Acked Read 8 Done Sent Done false
24079 7341 Acked Write 2 Done Sent Done false
24080 7342 Acked Flush 0 Done Sent Done false
24081 7343 Acked Read 1 Done Sent Done false
24082 7344 Acked Write 4 Done Sent Done false
24083 7345 Acked Read 2 Done Sent Done false
24084 7346 Acked Flush 0 Done Sent Done false
24085 7347 Acked Read 6 Done Sent Done false
24086 7348 Acked Write 7 Done Sent Done false
24087 7349 Acked Write 7 Done Sent Done false
24088 7350 Acked Read 2 Done Sent Done false
24089 7351 Acked Read 1 Done Sent Done false
24090 7352 Acked Write 9 Done Sent Done false
24091 7353 Acked Write 4 Done Sent Done false
24092 7354 Acked Write 1 Done Sent Done false
24093 7355 Acked Write 6 Done Sent Done false
24094 7356 Acked Write 6 Done Sent Done false
24095 7357 Acked Write 3 Done Sent Done false
24096 7358 Acked Read 9 Done Sent Done false
24097 7359 Acked Read 3 Done Sent Done false
24098 7360 Acked Write 3 Done Sent Done false
24099 7361 Acked Write 8 Done Sent Done false
24100 7362 Acked Write 10 Done Sent Done false
24101 7363 Acked Read 4 Done Sent Done false
24102 7364 Acked Read 5 Done Sent Done false
24103 7365 Acked Read 3 Done Sent Done false
24104 7366 Acked Read 4 Done Sent Done false
24105 7367 Acked Read 2 Done Sent Done false
24106 7368 Acked Write 5 Done Sent Done false
24107 7369 Acked Write 7 Done Sent Done false
24108 7370 Acked Write 10 Done Sent Done false
24109 7371 Acked Flush 0 Done Sent Done false
24110 7372 Acked Read 10 Done Sent Done false
24111 7373 Acked Write 2 Done Sent Done false
24112 7374 Acked Write 6 Done Sent Done false
24113 7375 Acked Flush 0 Done Sent Done false
24114 7376 Acked Read 8 Done Sent Done false
24115 7377 Acked Read 3 Done Sent Done false
24116 7378 Acked Read 4 Done Sent Done false
24117 7379 Acked Read 10 Done Sent Done false
24118 7380 Acked Read 1 Done Sent Done false
24119 7381 Acked Write 7 Done Sent Done false
24120 7382 Acked Flush 0 Done Sent Done false
24121 7383 Acked Write 9 Done Sent Done false
24122 7384 Acked Write 5 Done Sent Done false
24123 7385 Acked Read 7 Done Sent Done false
24124 7386 Acked Write 8 Done Sent Done false
24125 7387 Acked Read 4 Done Sent Done false
24126 7388 Acked Write 8 Done Sent Done false
24127 7389 Acked Write 8 Done Sent Done false
24128 7390 Acked Read 9 Done Sent Done false
24129 7391 Acked Read 10 Done Sent Done false
24130 7392 Acked Write 2 Done Sent Done false
24131 7393 Acked Write 3 Done Sent Done false
24132 7394 Acked Write 8 Done Sent Done false
24133 7395 Acked Read 1 Done Sent Done false
24134 7396 Acked Read 8 Done Sent Done false
24135 7397 Acked Read 7 Done Sent Done false
24136 7398 Acked Read 3 Done Sent Done false
24137 7399 Acked Read 3 Done Sent Done false
24138 7400 Acked Read 4 Done Sent Done false
24139 7401 Acked Write 5 Done Sent Done false
24140 7402 Acked Read 3 Done Sent Done false
24141 7403 Acked Flush 0 Done Sent Done false
24142 7404 Acked Read 7 Done Sent Done false
24143 7405 Acked Read 10 Done Sent Done false
24144 7406 Acked Read 1 Done Sent Done false
24145 7407 Acked Write 6 Done Sent Done false
24146 7408 Acked Read 9 Done Sent Done false
24147 7409 Acked Write 10 Done Sent Done false
24148 7410 Acked Read 4 Done Sent Done false
24149 7411 Acked Write 9 Done Sent Done false
24150 7412 Acked Read 4 Done Sent Done false
24151 7413 Acked Read 10 Done Sent Done false
24152 7414 Acked Write 3 Done Sent Done false
24153 7415 Acked Write 7 Done Sent Done false
24154 7416 Acked Write 5 Done Sent Done false
24155 7417 Acked Read 8 Done Sent Done false
24156 7418 Acked Write 3 Done Sent Done false
24157 7419 Acked Read 1 Done Sent Done false
24158 7420 Acked Write 5 Done Sent Done false
24159 7421 Acked Write 9 Done Sent Done false
24160 7422 Acked Read 9 Done Sent Done false
24161 7423 Acked Write 8 Done Sent Done false
24162 7424 Acked Read 8 Done Sent Done false
24163 7425 Acked Read 1 Done Sent Done false
24164 7426 Acked Read 4 Done Sent Done false
24165 7427 Acked Write 3 Done Sent Done false
24166 7428 Acked Read 8 Done Sent Done false
24167 7429 Acked Read 5 Done Sent Done false
24168 7430 Acked Flush 0 Done Sent Done false
24169 7431 Acked Write 7 Done Sent Done false
24170 7432 Acked Write 7 Done Sent Done false
24171 7433 Acked Read 5 Done Sent Done false
24172 7434 Acked Read 5 Done Sent Done false
24173 7435 Acked Read 6 Done Sent Done false
24174 7436 Acked Read 7 Done Sent Done false
24175 7437 Acked Write 5 Done Sent Done false
24176 7438 Acked Write 9 Done Sent Done false
24177 7439 Acked Write 2 Done Sent Done false
24178 7440 Acked Flush 0 Done Sent Done false
24179 7441 Acked Flush 0 Done Sent Done false
24180 7442 Acked Read 10 Done Sent Done false
24181 7443 Acked Write 2 Done Sent Done false
24182 7444 Acked Read 8 Done Sent Done false
24183 7445 Acked Read 1 Done Sent Done false
24184 7446 Acked Write 3 Done Sent Done false
24185 7447 Acked Write 2 Done Sent Done false
24186 7448 Acked Read 9 Done Sent Done false
24187 7449 Acked Read 5 Done Sent Done false
24188 7450 Acked Read 10 Done Sent Done false
24189 7451 Acked Read 4 Done Sent Done false
24190 7452 Acked Read 1 Done Sent Done false
24191 7453 Acked Read 8 Done Sent Done false
24192 7454 Acked Write 5 Done Sent Done false
24193 7455 Acked Write 1 Done Sent Done false
24194 7456 Acked Read 3 Done Sent Done false
24195 7457 Acked Write 4 Done Sent Done false
24196 7458 Acked Write 1 Done Sent Done false
24197 7459 Acked Read 5 Done Sent Done false
24198 7460 Acked Read 10 Done Sent Done false
24199 7461 Acked Read 5 Done Sent Done false
24200 7462 Acked Write 6 Done Sent Done false
24201 7463 Acked Write 8 Done Sent Done false
24202 7464 Acked Read 3 Done Sent Done false
24203 7465 Acked Read 4 Done Sent Done false
24204 7466 Acked Flush 0 Done Sent Done false
24205 7467 Acked Read 5 Done Sent Done false
24206 7468 Acked Read 1 Done Sent Done false
24207 7469 Acked Read 2 Done Sent Done false
24208 7470 Acked Read 9 Done Sent Done false
24209 7471 Acked Read 9 Done Sent Done false
24210 7472 Acked Read 2 Done Sent Done false
24211 7473 Acked Read 2 Done Sent Done false
24212 7474 Acked Write 2 Done Sent Done false
24213 7475 Acked Read 3 Done Sent Done false
24214 7476 Acked Read 6 Done Sent Done false
24215 7477 Acked Write 2 Done Sent Done false
24216 7478 Acked Write 8 Done Sent Done false
24217 7479 Acked Write 6 Done Sent Done false
24218 7480 Acked Read 7 Done Sent Done false
24219 7481 Acked Write 6 Done Sent Done false
24220 7482 Acked Flush 0 Done Sent Done false
24221 7483 Acked Write 9 Done Sent Done false
24222 7484 Acked Flush 0 Done Sent Done false
24223 7485 Acked Write 1 Done Sent Done false
24224 7486 Acked Read 1 Done Sent Done false
24225 7487 Acked Write 10 Done Sent Done false
24226 7488 Acked Write 4 Done Sent Done false
24227 7489 Acked Read 10 Done Sent Done false
24228 7490 Acked Read 7 Done Sent Done false
24229 7491 Acked Write 3 Done Sent Done false
24230 7492 Acked Write 10 Done Sent Done false
24231 7493 Acked Write 2 Done Sent Done false
24232 7494 Acked Write 7 Done Sent Done false
24233 7495 Acked Read 8 Done Sent Done false
24234 7496 Acked Read 2 Done Sent Done false
24235 7497 Acked Read 1 Done Sent Done false
24236 7498 Acked Write 6 Done Sent Done false
24237 7499 Acked Write 7 Done Sent Done false
24238 7500 Acked Write 10 Done Sent Done false
24239 7501 Acked Flush 0 Done Sent Done false
24240 7502 Acked Read 4 Done Sent Done false
24241 7503 Acked Read 10 Done Sent Done false
24242 7504 Acked Flush 0 Done Sent Done false
24243 7505 Acked Read 8 Done Sent Done false
24244 7506 Acked Read 1 Done Sent Done false
24245 7507 Acked Read 3 Done Sent Done false
24246 7508 Acked Read 9 Done Sent Done false
24247 7509 Acked Write 7 Done Sent Done false
24248 7510 Acked Read 9 Done Sent Done false
24249 7511 Acked Write 2 Done Sent Done false
24250 7512 Acked Read 1 Done Sent Done false
24251 7513 Acked Write 10 Done Sent Done false
24252 7514 Acked Read 1 Done Sent Done false
24253 7515 Acked Write 7 Done Sent Done false
24254 7516 Acked Read 8 Done Sent Done false
24255 7517 Acked Write 9 Done Sent Done false
24256 7518 Acked Read 10 Done Sent Done false
24257 7519 Acked Read 8 Done Sent Done false
24258 7520 Acked Write 6 Done Sent Done false
24259 7521 Acked Read 4 Done Sent Done false
24260 7522 Acked Read 6 Done Sent Done false
24261 7523 Acked Flush 0 Done Sent Done false
24262 7524 Acked Read 8 Done Sent Done false
24263 7525 Acked Write 6 Done Sent Done false
24264 7526 Acked Write 2 Done Sent Done false
24265 7527 Acked Flush 0 Done Sent Done false
24266 7528 Acked Read 9 Done Sent Done false
24267 7529 Acked Write 7 Done Sent Done false
24268 7530 Acked Read 7 Done Sent Done false
24269 7531 Acked Write 7 Done Sent Done false
24270 7532 Acked Read 2 Done Sent Done false
24271 7533 Acked Flush 0 Done Sent Done false
24272 7534 Acked Read 3 Done Sent Done false
24273 7535 Acked Read 8 Done Sent Done false
24274 7536 Acked Flush 0 Done Sent Done false
24275 7537 Acked Read 6 Done Sent Done false
24276 7538 Acked Read 1 Done Sent Done false
24277 7539 Acked Read 5 Done Sent Done false
24278 7540 Acked Read 3 Done Sent Done false
24279 7541 Acked Read 5 Done Sent Done false
24280 7542 Acked Read 6 Done Sent Done false
24281 7543 Acked Read 2 Done Sent Done false
24282 7544 Acked Read 2 Done Sent Done false
24283 7545 Acked Read 6 Done Sent Done false
24284 7546 Acked Read 6 Done Sent Done false
24285 7547 Acked Flush 0 Done Sent Done false
24286 7548 Acked Write 6 Done Sent Done false
24287 7549 Acked Write 4 Done Sent Done false
24288 7550 Acked Read 8 Done Sent Done false
24289 7551 Acked Read 3 Done Sent Done false
24290 7552 Acked Read 7 Done Sent Done false
24291 7553 Acked Write 10 Done Sent Done false
24292 7554 Acked Read 7 Done Sent Done false
24293 7555 Acked Flush 0 Done Sent Done false
24294 7556 Acked Flush 0 Done Sent Done false
24295 7557 Acked Read 2 Done Sent Done false
24296 7558 Acked Write 5 Done Sent Done false
24297 7559 Acked Write 9 Done Sent Done false
24298 7560 Acked Read 1 Done Sent Done false
24299 7561 Acked Read 3 Done Sent Done false
24300 7562 Acked Write 8 Done Sent Done false
24301 7563 Acked Read 1 Done Sent Done false
24302 7564 Acked Read 4 Done Sent Done false
24303 7565 Acked Read 7 Done Sent Done false
24304 7566 Acked Read 9 Done Sent Done false
24305 7567 Acked Read 5 Done Sent Done false
24306 7568 Acked Write 7 Done Sent Done false
24307 7569 Acked Write 7 Done Sent Done false
24308 7570 Acked Write 9 Done Sent Done false
24309 7571 Acked Read 3 Done Sent Done false
24310 7572 Acked Read 4 Done Sent Done false
24311 7573 Acked Read 1 Done Sent Done false
24312 7574 Acked Write 10 Done Sent Done false
24313 7575 Acked Write 2 Done Sent Done false
24314 7576 Acked Flush 0 Done Sent Done false
24315 7577 Acked Read 8 Done Sent Done false
24316 7578 Acked Write 7 Done Sent Done false
24317 7579 Acked Write 8 Done Sent Done false
24318 7580 Acked Write 2 Done Sent Done false
24319 7581 Acked Write 3 Done Sent Done false
24320 7582 Acked Read 2 Done Sent Done false
24321 7583 Acked Read 4 Done Sent Done false
24322 7584 Acked Read 7 Done Sent Done false
24323 7585 Acked Read 4 Done Sent Done false
24324 7586 Acked Write 9 Done Sent Done false
24325 7587 Acked Read 5 Done Sent Done false
24326 7588 Acked Read 10 Done Sent Done false
24327 7589 Acked Write 3 Done Sent Done false
24328 7590 Acked Read 10 Done Sent Done false
24329 7591 Acked Write 10 Done Sent Done false
24330 7592 Acked Flush 0 Done Sent Done false
24331 7593 Acked Read 8 Done Sent Done false
24332 7594 Acked Flush 0 Done Sent Done false
24333 7595 Acked Read 5 Done Sent Done false
24334 7596 Acked Write 7 Done Sent Done false
24335 7597 Acked Read 9 Done Sent Done false
24336 7598 Acked Read 4 Done Sent Done false
24337 7599 Acked Write 8 Done Sent Done false
24338 7600 Acked Read 9 Done Sent Done false
24339 7601 Acked Read 9 Done Sent Done false
24340 7602 Acked Flush 0 Done Sent Done false
24341 7603 Acked Read 6 Done Sent Done false
24342 7604 Acked Write 7 Done Sent Done false
24343 7605 Acked Read 5 Done Sent Done false
24344 7606 Acked Read 10 Done Sent Done false
24345 7607 Acked Read 7 Done Sent Done false
24346 7608 Acked Flush 0 Done Sent Done false
24347 7609 Acked Read 1 Done Sent Done false
24348 7610 Acked Read 4 Done Sent Done false
24349 7611 Acked Flush 0 Done Sent Done false
24350 7612 Acked Read 5 Done Sent Done false
24351 7613 Acked Read 8 Done Sent Done false
24352 7614 Acked Write 8 Done Sent Done false
24353 7615 Acked Write 5 Done Sent Done false
24354 7616 Acked Read 5 Done Sent Done false
24355 7617 Acked Write 7 Done Sent Done false
24356 7618 Acked Read 5 Done Sent Done false
24357 7619 Acked Write 7 Done Sent Done false
24358 7620 Acked Read 8 Done Sent Done false
24359 7621 Acked Read 2 Done Sent Done false
24360 7622 Acked Write 7 Done Sent Done false
24361 7623 Acked Write 3 Done Sent Done false
24362 7624 Acked Write 3 Done Sent Done false
24363 7625 Acked Read 8 Done Sent Done false
24364 7626 Acked Read 5 Done Sent Done false
24365 7627 Acked Read 7 Done Sent Done false
24366 7628 Acked Flush 0 Done Sent Done false
24367 7629 Acked Flush 0 Done Sent Done false
24368 7630 Acked Write 4 Done Sent Done false
24369 7631 Acked Write 2 Done Sent Done false
24370 7632 Acked Read 10 Done Sent Done false
24371 7633 Acked Write 7 Done Sent Done false
24372 7634 Acked Write 9 Done Sent Done false
24373 7635 Acked Flush 0 Done Sent Done false
24374 7636 Acked Read 5 Done Sent Done false
24375 7637 Acked Write 5 Done Sent Done false
24376 7638 Acked Read 10 Done Sent Done false
24377 STATES DS:0 DS:1 DS:2 TOTAL
24378 Sent 0 301 0 301
24379 Done 301 0 301 602
24380 Skipped 0 0 0 0
24381 Error 0 0 0 0
24382 Last Flush: 7635 7337 7635
24383 Downstairs last five completed: 7337 7336 7335 7334 7333
24384 Upstairs last five completed: 7638 7637 7636 7635 7634
24385 CLIENT: Up:0 ds:301 act:2
24386 ----------------------------------------------------------------
24387 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24388 States: Active Offline Active
24389 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24390 7338 Acked Flush 0 Done Sent Done false
24391 7339 Acked Read 1 Done Sent Done false
24392 7340 Acked Read 8 Done Sent Done false
24393 7341 Acked Write 2 Done Sent Done false
24394 7342 Acked Flush 0 Done Sent Done false
24395 7343 Acked Read 1 Done Sent Done false
24396 7344 Acked Write 4 Done Sent Done false
24397 7345 Acked Read 2 Done Sent Done false
24398 7346 Acked Flush 0 Done Sent Done false
24399 7347 Acked Read 6 Done Sent Done false
24400 7348 Acked Write 7 Done Sent Done false
24401 7349 Acked Write 7 Done Sent Done false
24402 7350 Acked Read 2 Done Sent Done false
24403 7351 Acked Read 1 Done Sent Done false
24404 7352 Acked Write 9 Done Sent Done false
24405 7353 Acked Write 4 Done Sent Done false
24406 7354 Acked Write 1 Done Sent Done false
24407 7355 Acked Write 6 Done Sent Done false
24408 7356 Acked Write 6 Done Sent Done false
24409 7357 Acked Write 3 Done Sent Done false
24410 7358 Acked Read 9 Done Sent Done false
24411 7359 Acked Read 3 Done Sent Done false
24412 7360 Acked Write 3 Done Sent Done false
24413 7361 Acked Write 8 Done Sent Done false
24414 7362 Acked Write 10 Done Sent Done false
24415 7363 Acked Read 4 Done Sent Done false
24416 7364 Acked Read 5 Done Sent Done false
24417 7365 Acked Read 3 Done Sent Done false
24418 7366 Acked Read 4 Done Sent Done false
24419 7367 Acked Read 2 Done Sent Done false
24420 7368 Acked Write 5 Done Sent Done false
24421 7369 Acked Write 7 Done Sent Done false
24422 7370 Acked Write 10 Done Sent Done false
24423 7371 Acked Flush 0 Done Sent Done false
24424 7372 Acked Read 10 Done Sent Done false
24425 7373 Acked Write 2 Done Sent Done false
24426 7374 Acked Write 6 Done Sent Done false
24427 7375 Acked Flush 0 Done Sent Done false
24428 7376 Acked Read 8 Done Sent Done false
24429 7377 Acked Read 3 Done Sent Done false
24430 7378 Acked Read 4 Done Sent Done false
24431 7379 Acked Read 10 Done Sent Done false
24432 7380 Acked Read 1 Done Sent Done false
24433 7381 Acked Write 7 Done Sent Done false
24434 7382 Acked Flush 0 Done Sent Done false
24435 7383 Acked Write 9 Done Sent Done false
24436 7384 Acked Write 5 Done Sent Done false
24437 7385 Acked Read 7 Done Sent Done false
24438 7386 Acked Write 8 Done Sent Done false
24439 7387 Acked Read 4 Done Sent Done false
24440 7388 Acked Write 8 Done Sent Done false
24441 7389 Acked Write 8 Done Sent Done false
24442 7390 Acked Read 9 Done Sent Done false
24443 7391 Acked Read 10 Done Sent Done false
24444 7392 Acked Write 2 Done Sent Done false
24445 7393 Acked Write 3 Done Sent Done false
24446 7394 Acked Write 8 Done Sent Done false
24447 7395 Acked Read 1 Done Sent Done false
24448 7396 Acked Read 8 Done Sent Done false
24449 7397 Acked Read 7 Done Sent Done false
24450 7398 Acked Read 3 Done Sent Done false
24451 7399 Acked Read 3 Done Sent Done false
24452 7400 Acked Read 4 Done Sent Done false
24453 7401 Acked Write 5 Done Sent Done false
24454 7402 Acked Read 3 Done Sent Done false
24455 7403 Acked Flush 0 Done Sent Done false
24456 7404 Acked Read 7 Done Sent Done false
24457 7405 Acked Read 10 Done Sent Done false
24458 7406 Acked Read 1 Done Sent Done false
24459 7407 Acked Write 6 Done Sent Done false
24460 7408 Acked Read 9 Done Sent Done false
24461 7409 Acked Write 10 Done Sent Done false
24462 7410 Acked Read 4 Done Sent Done false
24463 7411 Acked Write 9 Done Sent Done false
24464 7412 Acked Read 4 Done Sent Done false
24465 7413 Acked Read 10 Done Sent Done false
24466 7414 Acked Write 3 Done Sent Done false
24467 7415 Acked Write 7 Done Sent Done false
24468 7416 Acked Write 5 Done Sent Done false
24469 7417 Acked Read 8 Done Sent Done false
24470 7418 Acked Write 3 Done Sent Done false
24471 7419 Acked Read 1 Done Sent Done false
24472 7420 Acked Write 5 Done Sent Done false
24473 7421 Acked Write 9 Done Sent Done false
24474 7422 Acked Read 9 Done Sent Done false
24475 7423 Acked Write 8 Done Sent Done false
24476 7424 Acked Read 8 Done Sent Done false
24477 7425 Acked Read 1 Done Sent Done false
24478 7426 Acked Read 4 Done Sent Done false
24479 7427 Acked Write 3 Done Sent Done false
24480 7428 Acked Read 8 Done Sent Done false
24481 7429 Acked Read 5 Done Sent Done false
24482 7430 Acked Flush 0 Done Sent Done false
24483 7431 Acked Write 7 Done Sent Done false
24484 7432 Acked Write 7 Done Sent Done false
24485 7433 Acked Read 5 Done Sent Done false
24486 7434 Acked Read 5 Done Sent Done false
24487 7435 Acked Read 6 Done Sent Done false
24488 7436 Acked Read 7 Done Sent Done false
24489 7437 Acked Write 5 Done Sent Done false
24490 7438 Acked Write 9 Done Sent Done false
24491 7439 Acked Write 2 Done Sent Done false
24492 7440 Acked Flush 0 Done Sent Done false
24493 7441 Acked Flush 0 Done Sent Done false
24494 7442 Acked Read 10 Done Sent Done false
24495 7443 Acked Write 2 Done Sent Done false
24496 7444 Acked Read 8 Done Sent Done false
24497 7445 Acked Read 1 Done Sent Done false
24498 7446 Acked Write 3 Done Sent Done false
24499 7447 Acked Write 2 Done Sent Done false
24500 7448 Acked Read 9 Done Sent Done false
24501 7449 Acked Read 5 Done Sent Done false
24502 7450 Acked Read 10 Done Sent Done false
24503 7451 Acked Read 4 Done Sent Done false
24504 7452 Acked Read 1 Done Sent Done false
24505 7453 Acked Read 8 Done Sent Done false
24506 7454 Acked Write 5 Done Sent Done false
24507 7455 Acked Write 1 Done Sent Done false
24508 7456 Acked Read 3 Done Sent Done false
24509 7457 Acked Write 4 Done Sent Done false
24510 7458 Acked Write 1 Done Sent Done false
24511 7459 Acked Read 5 Done Sent Done false
24512 7460 Acked Read 10 Done Sent Done false
24513 7461 Acked Read 5 Done Sent Done false
24514 7462 Acked Write 6 Done Sent Done false
24515 7463 Acked Write 8 Done Sent Done false
24516 7464 Acked Read 3 Done Sent Done false
24517 7465 Acked Read 4 Done Sent Done false
24518 7466 Acked Flush 0 Done Sent Done false
24519 7467 Acked Read 5 Done Sent Done false
24520 7468 Acked Read 1 Done Sent Done false
24521 7469 Acked Read 2 Done Sent Done false
24522 7470 Acked Read 9 Done Sent Done false
24523 7471 Acked Read 9 Done Sent Done false
24524 7472 Acked Read 2 Done Sent Done false
24525 7473 Acked Read 2 Done Sent Done false
24526 7474 Acked Write 2 Done Sent Done false
24527 7475 Acked Read 3 Done Sent Done false
24528 7476 Acked Read 6 Done Sent Done false
24529 7477 Acked Write 2 Done Sent Done false
24530 7478 Acked Write 8 Done Sent Done false
24531 7479 Acked Write 6 Done Sent Done false
24532 7480 Acked Read 7 Done Sent Done false
24533 7481 Acked Write 6 Done Sent Done false
24534 7482 Acked Flush 0 Done Sent Done false
24535 7483 Acked Write 9 Done Sent Done false
24536 7484 Acked Flush 0 Done Sent Done false
24537 7485 Acked Write 1 Done Sent Done false
24538 7486 Acked Read 1 Done Sent Done false
24539 7487 Acked Write 10 Done Sent Done false
24540 7488 Acked Write 4 Done Sent Done false
24541 7489 Acked Read 10 Done Sent Done false
24542 7490 Acked Read 7 Done Sent Done false
24543 7491 Acked Write 3 Done Sent Done false
24544 7492 Acked Write 10 Done Sent Done false
24545 7493 Acked Write 2 Done Sent Done false
24546 7494 Acked Write 7 Done Sent Done false
24547 7495 Acked Read 8 Done Sent Done false
24548 7496 Acked Read 2 Done Sent Done false
24549 7497 Acked Read 1 Done Sent Done false
24550 7498 Acked Write 6 Done Sent Done false
24551 7499 Acked Write 7 Done Sent Done false
24552 7500 Acked Write 10 Done Sent Done false
24553 7501 Acked Flush 0 Done Sent Done false
24554 7502 Acked Read 4 Done Sent Done false
24555 7503 Acked Read 10 Done Sent Done false
24556 7504 Acked Flush 0 Done Sent Done false
24557 7505 Acked Read 8 Done Sent Done false
24558 7506 Acked Read 1 Done Sent Done false
24559 7507 Acked Read 3 Done Sent Done false
24560 7508 Acked Read 9 Done Sent Done false
24561 7509 Acked Write 7 Done Sent Done false
24562 7510 Acked Read 9 Done Sent Done false
24563 7511 Acked Write 2 Done Sent Done false
24564 7512 Acked Read 1 Done Sent Done false
24565 7513 Acked Write 10 Done Sent Done false
24566 7514 Acked Read 1 Done Sent Done false
24567 7515 Acked Write 7 Done Sent Done false
24568 7516 Acked Read 8 Done Sent Done false
24569 7517 Acked Write 9 Done Sent Done false
24570 7518 Acked Read 10 Done Sent Done false
24571 7519 Acked Read 8 Done Sent Done false
24572 7520 Acked Write 6 Done Sent Done false
24573 7521 Acked Read 4 Done Sent Done false
24574 7522 Acked Read 6 Done Sent Done false
24575 7523 Acked Flush 0 Done Sent Done false
24576 7524 Acked Read 8 Done Sent Done false
24577 7525 Acked Write 6 Done Sent Done false
24578 7526 Acked Write 2 Done Sent Done false
24579 7527 Acked Flush 0 Done Sent Done false
24580 7528 Acked Read 9 Done Sent Done false
24581 7529 Acked Write 7 Done Sent Done false
24582 7530 Acked Read 7 Done Sent Done false
24583 7531 Acked Write 7 Done Sent Done false
24584 7532 Acked Read 2 Done Sent Done false
24585 7533 Acked Flush 0 Done Sent Done false
24586 7534 Acked Read 3 Done Sent Done false
24587 7535 Acked Read 8 Done Sent Done false
24588 7536 Acked Flush 0 Done Sent Done false
24589 7537 Acked Read 6 Done Sent Done false
24590 7538 Acked Read 1 Done Sent Done false
24591 7539 Acked Read 5 Done Sent Done false
24592 7540 Acked Read 3 Done Sent Done false
24593 7541 Acked Read 5 Done Sent Done false
24594 7542 Acked Read 6 Done Sent Done false
24595 7543 Acked Read 2 Done Sent Done false
24596 7544 Acked Read 2 Done Sent Done false
24597 7545 Acked Read 6 Done Sent Done false
24598 7546 Acked Read 6 Done Sent Done false
24599 7547 Acked Flush 0 Done Sent Done false
24600 7548 Acked Write 6 Done Sent Done false
24601 7549 Acked Write 4 Done Sent Done false
24602 7550 Acked Read 8 Done Sent Done false
24603 7551 Acked Read 3 Done Sent Done false
24604 7552 Acked Read 7 Done Sent Done false
24605 7553 Acked Write 10 Done Sent Done false
24606 7554 Acked Read 7 Done Sent Done false
24607 7555 Acked Flush 0 Done Sent Done false
24608 7556 Acked Flush 0 Done Sent Done false
24609 7557 Acked Read 2 Done Sent Done false
24610 7558 Acked Write 5 Done Sent Done false
24611 7559 Acked Write 9 Done Sent Done false
24612 7560 Acked Read 1 Done Sent Done false
24613 7561 Acked Read 3 Done Sent Done false
24614 7562 Acked Write 8 Done Sent Done false
24615 7563 Acked Read 1 Done Sent Done false
24616 7564 Acked Read 4 Done Sent Done false
24617 7565 Acked Read 7 Done Sent Done false
24618 7566 Acked Read 9 Done Sent Done false
24619 7567 Acked Read 5 Done Sent Done false
24620 7568 Acked Write 7 Done Sent Done false
24621 7569 Acked Write 7 Done Sent Done false
24622 7570 Acked Write 9 Done Sent Done false
24623 7571 Acked Read 3 Done Sent Done false
24624 7572 Acked Read 4 Done Sent Done false
24625 7573 Acked Read 1 Done Sent Done false
24626 7574 Acked Write 10 Done Sent Done false
24627 7575 Acked Write 2 Done Sent Done false
24628 7576 Acked Flush 0 Done Sent Done false
24629 7577 Acked Read 8 Done Sent Done false
24630 7578 Acked Write 7 Done Sent Done false
24631 7579 Acked Write 8 Done Sent Done false
24632 7580 Acked Write 2 Done Sent Done false
24633 7581 Acked Write 3 Done Sent Done false
24634 7582 Acked Read 2 Done Sent Done false
24635 7583 Acked Read 4 Done Sent Done false
24636 7584 Acked Read 7 Done Sent Done false
24637 7585 Acked Read 4 Done Sent Done false
24638 7586 Acked Write 9 Done Sent Done false
24639 7587 Acked Read 5 Done Sent Done false
24640 7588 Acked Read 10 Done Sent Done false
24641 7589 Acked Write 3 Done Sent Done false
24642 7590 Acked Read 10 Done Sent Done false
24643 7591 Acked Write 10 Done Sent Done false
24644 7592 Acked Flush 0 Done Sent Done false
24645 7593 Acked Read 8 Done Sent Done false
24646 7594 Acked Flush 0 Done Sent Done false
24647 7595 Acked Read 5 Done Sent Done false
24648 7596 Acked Write 7 Done Sent Done false
24649 7597 Acked Read 9 Done Sent Done false
24650 7598 Acked Read 4 Done Sent Done false
24651 7599 Acked Write 8 Done Sent Done false
24652 7600 Acked Read 9 Done Sent Done false
24653 7601 Acked Read 9 Done Sent Done false
24654 7602 Acked Flush 0 Done Sent Done false
24655 7603 Acked Read 6 Done Sent Done false
24656 7604 Acked Write 7 Done Sent Done false
24657 7605 Acked Read 5 Done Sent Done false
24658 7606 Acked Read 10 Done Sent Done false
24659 7607 Acked Read 7 Done Sent Done false
24660 7608 Acked Flush 0 Done Sent Done false
24661 7609 Acked Read 1 Done Sent Done false
24662 7610 Acked Read 4 Done Sent Done false
24663 7611 Acked Flush 0 Done Sent Done false
24664 7612 Acked Read 5 Done Sent Done false
24665 7613 Acked Read 8 Done Sent Done false
24666 7614 Acked Write 8 Done Sent Done false
24667 7615 Acked Write 5 Done Sent Done false
24668 7616 Acked Read 5 Done Sent Done false
24669 7617 Acked Write 7 Done Sent Done false
24670 7618 Acked Read 5 Done Sent Done false
24671 7619 Acked Write 7 Done Sent Done false
24672 7620 Acked Read 8 Done Sent Done false
24673 7621 Acked Read 2 Done Sent Done false
24674 7622 Acked Write 7 Done Sent Done false
24675 7623 Acked Write 3 Done Sent Done false
24676 7624 Acked Write 3 Done Sent Done false
24677 7625 Acked Read 8 Done Sent Done false
24678 7626 Acked Read 5 Done Sent Done false
24679 7627 Acked Read 7 Done Sent Done false
24680 7628 Acked Flush 0 Done Sent Done false
24681 7629 Acked Flush 0 Done Sent Done false
24682 7630 Acked Write 4 Done Sent Done false
24683 7631 Acked Write 2 Done Sent Done false
24684 7632 Acked Read 10 Done Sent Done false
24685 7633 Acked Write 7 Done Sent Done false
24686 7634 Acked Write 9 Done Sent Done false
24687 7635 Acked Flush 0 Done Sent Done false
24688 7636 Acked Read 5 Done Sent Done false
24689 7637 Acked Write 5 Done Sent Done false
24690 7638 Acked Read 10 Done Sent Done false
24691 7639 Acked Flush 0 Done Sent Done false
24692 STATES DS:0 DS:1 DS:2 TOTAL
24693 Sent 0 302 0 302
24694 Done 302 0 302 604
24695 Skipped 0 0 0 0
24696 Error 0 0 0 0
24697 Last Flush: 7639 7337 7639
24698 Downstairs last five completed: 7337 7336 7335 7334 7333
24699 Upstairs last five completed: 7639 7638 7637 7636 7635
24700 CLIENT: Up:0 ds:302 act:2
247012025-01-27T19:41:32.118ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
247022025-01-27T19:41:32.118ZWARNcrucible: ds_connection connect to 127.0.0.1:8820 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
247032025-01-27T19:41:32.118ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
247042025-01-27T19:41:32.118ZWARNcrucible: downstairs task for 1 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
247052025-01-27T19:41:32.118ZINFOcrucible: client task is exiting = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
24706 ----------------------------------------------------------------
24707 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
24708 States: Active Offline Active
24709 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
24710 7338 Acked Flush 0 Done Sent Done false
24711 7339 Acked Read 1 Done Sent Done false
24712 7340 Acked Read 8 Done Sent Done false
24713 7341 Acked Write 2 Done Sent Done false
24714 7342 Acked Flush 0 Done Sent Done false
24715 7343 Acked Read 1 Done Sent Done false
24716 7344 Acked Write 4 Done Sent Done false
24717 7345 Acked Read 2 Done Sent Done false
24718 7346 Acked Flush 0 Done Sent Done false
24719 7347 Acked Read 6 Done Sent Done false
24720 7348 Acked Write 7 Done Sent Done false
24721 7349 Acked Write 7 Done Sent Done false
24722 7350 Acked Read 2 Done Sent Done false
24723 7351 Acked Read 1 Done Sent Done false
24724 7352 Acked Write 9 Done Sent Done false
24725 7353 Acked Write 4 Done Sent Done false
24726 7354 Acked Write 1 Done Sent Done false
24727 7355 Acked Write 6 Done Sent Done false
24728 7356 Acked Write 6 Done Sent Done false
24729 7357 Acked Write 3 Done Sent Done false
24730 7358 Acked Read 9 Done Sent Done false
24731 7359 Acked Read 3 Done Sent Done false
24732 7360 Acked Write 3 Done Sent Done false
24733 7361 Acked Write 8 Done Sent Done false
24734 7362 Acked Write 10 Done Sent Done false
24735 7363 Acked Read 4 Done Sent Done false
24736 7364 Acked Read 5 Done Sent Done false
24737 7365 Acked Read 3 Done Sent Done false
24738 7366 Acked Read 4 Done Sent Done false
24739 7367 Acked Read 2 Done Sent Done false
24740 7368 Acked Write 5 Done Sent Done false
24741 7369 Acked Write 7 Done Sent Done false
24742 7370 Acked Write 10 Done Sent Done false
24743 7371 Acked Flush 0 Done Sent Done false
24744 7372 Acked Read 10 Done Sent Done false
24745 7373 Acked Write 2 Done Sent Done false
24746 7374 Acked Write 6 Done Sent Done false
24747 7375 Acked Flush 0 Done Sent Done false
24748 7376 Acked Read 8 Done Sent Done false
24749 7377 Acked Read 3 Done Sent Done false
24750 7378 Acked Read 4 Done Sent Done false
24751 7379 Acked Read 10 Done Sent Done false
24752 7380 Acked Read 1 Done Sent Done false
24753 7381 Acked Write 7 Done Sent Done false
24754 7382 Acked Flush 0 Done Sent Done false
24755 7383 Acked Write 9 Done Sent Done false
24756 7384 Acked Write 5 Done Sent Done false
24757 7385 Acked Read 7 Done Sent Done false
24758 7386 Acked Write 8 Done Sent Done false
24759 7387 Acked Read 4 Done Sent Done false
24760 7388 Acked Write 8 Done Sent Done false
24761 7389 Acked Write 8 Done Sent Done false
24762 7390 Acked Read 9 Done Sent Done false
24763 7391 Acked Read 10 Done Sent Done false
24764 7392 Acked Write 2 Done Sent Done false
24765 7393 Acked Write 3 Done Sent Done false
24766 7394 Acked Write 8 Done Sent Done false
24767 7395 Acked Read 1 Done Sent Done false
24768 7396 Acked Read 8 Done Sent Done false
24769 7397 Acked Read 7 Done Sent Done false
24770 7398 Acked Read 3 Done Sent Done false
24771 7399 Acked Read 3 Done Sent Done false
24772 7400 Acked Read 4 Done Sent Done false
24773 7401 Acked Write 5 Done Sent Done false
24774 7402 Acked Read 3 Done Sent Done false
24775 7403 Acked Flush 0 Done Sent Done false
24776 7404 Acked Read 7 Done Sent Done false
24777 7405 Acked Read 10 Done Sent Done false
24778 7406 Acked Read 1 Done Sent Done false
24779 7407 Acked Write 6 Done Sent Done false
24780 7408 Acked Read 9 Done Sent Done false
24781 7409 Acked Write 10 Done Sent Done false
24782 7410 Acked Read 4 Done Sent Done false
24783 7411 Acked Write 9 Done Sent Done false
24784 7412 Acked Read 4 Done Sent Done false
24785 7413 Acked Read 10 Done Sent Done false
24786 7414 Acked Write 3 Done Sent Done false
24787 7415 Acked Write 7 Done Sent Done false
24788 7416 Acked Write 5 Done Sent Done false
24789 7417 Acked Read 8 Done Sent Done false
24790 7418 Acked Write 3 Done Sent Done false
24791 7419 Acked Read 1 Done Sent Done false
24792 7420 Acked Write 5 Done Sent Done false
24793 7421 Acked Write 9 Done Sent Done false
24794 7422 Acked Read 9 Done Sent Done false
24795 7423 Acked Write 8 Done Sent Done false
24796 7424 Acked Read 8 Done Sent Done false
24797 7425 Acked Read 1 Done Sent Done false
24798 7426 Acked Read 4 Done Sent Done false
24799 7427 Acked Write 3 Done Sent Done false
24800 7428 Acked Read 8 Done Sent Done false
24801 7429 Acked Read 5 Done Sent Done false
24802 7430 Acked Flush 0 Done Sent Done false
24803 7431 Acked Write 7 Done Sent Done false
24804 7432 Acked Write 7 Done Sent Done false
24805 7433 Acked Read 5 Done Sent Done false
24806 7434 Acked Read 5 Done Sent Done false
24807 7435 Acked Read 6 Done Sent Done false
24808 7436 Acked Read 7 Done Sent Done false
24809 7437 Acked Write 5 Done Sent Done false
24810 7438 Acked Write 9 Done Sent Done false
24811 7439 Acked Write 2 Done Sent Done false
24812 7440 Acked Flush 0 Done Sent Done false
24813 7441 Acked Flush 0 Done Sent Done false
24814 7442 Acked Read 10 Done Sent Done false
24815 7443 Acked Write 2 Done Sent Done false
24816 7444 Acked Read 8 Done Sent Done false
24817 7445 Acked Read 1 Done Sent Done false
24818 7446 Acked Write 3 Done Sent Done false
24819 7447 Acked Write 2 Done Sent Done false
24820 7448 Acked Read 9 Done Sent Done false
24821 7449 Acked Read 5 Done Sent Done false
24822 7450 Acked Read 10 Done Sent Done false
24823 7451 Acked Read 4 Done Sent Done false
24824 7452 Acked Read 1 Done Sent Done false
24825 7453 Acked Read 8 Done Sent Done false
24826 7454 Acked Write 5 Done Sent Done false
24827 7455 Acked Write 1 Done Sent Done false
24828 7456 Acked Read 3 Done Sent Done false
24829 7457 Acked Write 4 Done Sent Done false
24830 7458 Acked Write 1 Done Sent Done false
24831 7459 Acked Read 5 Done Sent Done false
24832 7460 Acked Read 10 Done Sent Done false
24833 7461 Acked Read 5 Done Sent Done false
24834 7462 Acked Write 6 Done Sent Done false
24835 7463 Acked Write 8 Done Sent Done false
24836 7464 Acked Read 3 Done Sent Done false
24837 7465 Acked Read 4 Done Sent Done false
24838 7466 Acked Flush 0 Done Sent Done false
24839 7467 Acked Read 5 Done Sent Done false
24840 7468 Acked Read 1 Done Sent Done false
24841 7469 Acked Read 2 Done Sent Done false
24842 7470 Acked Read 9 Done Sent Done false
24843 7471 Acked Read 9 Done Sent Done false
24844 7472 Acked Read 2 Done Sent Done false
24845 7473 Acked Read 2 Done Sent Done false
24846 7474 Acked Write 2 Done Sent Done false
24847 7475 Acked Read 3 Done Sent Done false
24848 7476 Acked Read 6 Done Sent Done false
24849 7477 Acked Write 2 Done Sent Done false
24850 7478 Acked Write 8 Done Sent Done false
24851 7479 Acked Write 6 Done Sent Done false
24852 7480 Acked Read 7 Done Sent Done false
24853 7481 Acked Write 6 Done Sent Done false
24854 7482 Acked Flush 0 Done Sent Done false
24855 7483 Acked Write 9 Done Sent Done false
24856 7484 Acked Flush 0 Done Sent Done false
24857 7485 Acked Write 1 Done Sent Done false
24858 7486 Acked Read 1 Done Sent Done false
24859 7487 Acked Write 10 Done Sent Done false
24860 7488 Acked Write 4 Done Sent Done false
24861 7489 Acked Read 10 Done Sent Done false
24862 7490 Acked Read 7 Done Sent Done false
24863 7491 Acked Write 3 Done Sent Done false
24864 7492 Acked Write 10 Done Sent Done false
24865 7493 Acked Write 2 Done Sent Done false
24866 7494 Acked Write 7 Done Sent Done false
24867 7495 Acked Read 8 Done Sent Done false
24868 7496 Acked Read 2 Done Sent Done false
24869 7497 Acked Read 1 Done Sent Done false
24870 7498 Acked Write 6 Done Sent Done false
24871 7499 Acked Write 7 Done Sent Done false
24872 7500 Acked Write 10 Done Sent Done false
24873 7501 Acked Flush 0 Done Sent Done false
24874 7502 Acked Read 4 Done Sent Done false
24875 7503 Acked Read 10 Done Sent Done false
24876 7504 Acked Flush 0 Done Sent Done false
24877 7505 Acked Read 8 Done Sent Done false
24878 7506 Acked Read 1 Done Sent Done false
24879 7507 Acked Read 3 Done Sent Done false
24880 7508 Acked Read 9 Done Sent Done false
24881 7509 Acked Write 7 Done Sent Done false
24882 7510 Acked Read 9 Done Sent Done false
24883 7511 Acked Write 2 Done Sent Done false
24884 7512 Acked Read 1 Done Sent Done false
24885 7513 Acked Write 10 Done Sent Done false
24886 7514 Acked Read 1 Done Sent Done false
24887 7515 Acked Write 7 Done Sent Done false
24888 7516 Acked Read 8 Done Sent Done false
24889 7517 Acked Write 9 Done Sent Done false
24890 7518 Acked Read 10 Done Sent Done false
24891 7519 Acked Read 8 Done Sent Done false
24892 7520 Acked Write 6 Done Sent Done false
24893 7521 Acked Read 4 Done Sent Done false
24894 7522 Acked Read 6 Done Sent Done false
24895 7523 Acked Flush 0 Done Sent Done false
24896 7524 Acked Read 8 Done Sent Done false
24897 7525 Acked Write 6 Done Sent Done false
24898 7526 Acked Write 2 Done Sent Done false
24899 7527 Acked Flush 0 Done Sent Done false
24900 7528 Acked Read 9 Done Sent Done false
24901 7529 Acked Write 7 Done Sent Done false
24902 7530 Acked Read 7 Done Sent Done false
24903 7531 Acked Write 7 Done Sent Done false
24904 7532 Acked Read 2 Done Sent Done false
24905 7533 Acked Flush 0 Done Sent Done false
24906 7534 Acked Read 3 Done Sent Done false
24907 7535 Acked Read 8 Done Sent Done false
24908 7536 Acked Flush 0 Done Sent Done false
24909 7537 Acked Read 6 Done Sent Done false
24910 7538 Acked Read 1 Done Sent Done false
24911 7539 Acked Read 5 Done Sent Done false
24912 7540 Acked Read 3 Done Sent Done false
24913 7541 Acked Read 5 Done Sent Done false
24914 7542 Acked Read 6 Done Sent Done false
24915 7543 Acked Read 2 Done Sent Done false
24916 7544 Acked Read 2 Done Sent Done false
24917 7545 Acked Read 6 Done Sent Done false
24918 7546 Acked Read 6 Done Sent Done false
24919 7547 Acked Flush 0 Done Sent Done false
24920 7548 Acked Write 6 Done Sent Done false
24921 7549 Acked Write 4 Done Sent Done false
24922 7550 Acked Read 8 Done Sent Done false
24923 7551 Acked Read 3 Done Sent Done false
24924 7552 Acked Read 7 Done Sent Done false
24925 7553 Acked Write 10 Done Sent Done false
24926 7554 Acked Read 7 Done Sent Done false
24927 7555 Acked Flush 0 Done Sent Done false
24928 7556 Acked Flush 0 Done Sent Done false
24929 7557 Acked Read 2 Done Sent Done false
24930 7558 Acked Write 5 Done Sent Done false
24931 7559 Acked Write 9 Done Sent Done false
24932 7560 Acked Read 1 Done Sent Done false
24933 7561 Acked Read 3 Done Sent Done false
24934 7562 Acked Write 8 Done Sent Done false
24935 7563 Acked Read 1 Done Sent Done false
24936 7564 Acked Read 4 Done Sent Done false
24937 7565 Acked Read 7 Done Sent Done false
24938 7566 Acked Read 9 Done Sent Done false
24939 7567 Acked Read 5 Done Sent Done false
24940 7568 Acked Write 7 Done Sent Done false
24941 7569 Acked Write 7 Done Sent Done false
24942 7570 Acked Write 9 Done Sent Done false
24943 7571 Acked Read 3 Done Sent Done false
24944 7572 Acked Read 4 Done Sent Done false
24945 7573 Acked Read 1 Done Sent Done false
24946 7574 Acked Write 10 Done Sent Done false
24947 7575 Acked Write 2 Done Sent Done false
24948 7576 Acked Flush 0 Done Sent Done false
24949 7577 Acked Read 8 Done Sent Done false
24950 7578 Acked Write 7 Done Sent Done false
24951 7579 Acked Write 8 Done Sent Done false
24952 7580 Acked Write 2 Done Sent Done false
24953 7581 Acked Write 3 Done Sent Done false
24954 7582 Acked Read 2 Done Sent Done false
24955 7583 Acked Read 4 Done Sent Done false
24956 7584 Acked Read 7 Done Sent Done false
24957 7585 Acked Read 4 Done Sent Done false
24958 7586 Acked Write 9 Done Sent Done false
24959 7587 Acked Read 5 Done Sent Done false
24960 7588 Acked Read 10 Done Sent Done false
24961 7589 Acked Write 3 Done Sent Done false
24962 7590 Acked Read 10 Done Sent Done false
24963 7591 Acked Write 10 Done Sent Done false
24964 7592 Acked Flush 0 Done Sent Done false
24965 7593 Acked Read 8 Done Sent Done false
24966 7594 Acked Flush 0 Done Sent Done false
24967 7595 Acked Read 5 Done Sent Done false
24968 7596 Acked Write 7 Done Sent Done false
24969 7597 Acked Read 9 Done Sent Done false
24970 7598 Acked Read 4 Done Sent Done false
24971 7599 Acked Write 8 Done Sent Done false
24972 7600 Acked Read 9 Done Sent Done false
24973 7601 Acked Read 9 Done Sent Done false
24974 7602 Acked Flush 0 Done Sent Done false
24975 7603 Acked Read 6 Done Sent Done false
24976 7604 Acked Write 7 Done Sent Done false
24977 7605 Acked Read 5 Done Sent Done false
24978 7606 Acked Read 10 Done Sent Done false
24979 7607 Acked Read 7 Done Sent Done false
24980 7608 Acked Flush 0 Done Sent Done false
24981 7609 Acked Read 1 Done Sent Done false
24982 7610 Acked Read 4 Done Sent Done false
24983 7611 Acked Flush 0 Done Sent Done false
24984 7612 Acked Read 5 Done Sent Done false
24985 7613 Acked Read 8 Done Sent Done false
24986 7614 Acked Write 8 Done Sent Done false
24987 7615 Acked Write 5 Done Sent Done false
24988 7616 Acked Read 5 Done Sent Done false
24989 7617 Acked Write 7 Done Sent Done false
24990 7618 Acked Read 5 Done Sent Done false
24991 7619 Acked Write 7 Done Sent Done false
24992 7620 Acked Read 8 Done Sent Done false
24993 7621 Acked Read 2 Done Sent Done false
24994 7622 Acked Write 7 Done Sent Done false
24995 7623 Acked Write 3 Done Sent Done false
24996 7624 Acked Write 3 Done Sent Done false
24997 7625 Acked Read 8 Done Sent Done false
24998 7626 Acked Read 5 Done Sent Done false
24999 7627 Acked Read 7 Done Sent Done false
25000 7628 Acked Flush 0 Done Sent Done false
25001 7629 Acked Flush 0 Done Sent Done false
25002 7630 Acked Write 4 Done Sent Done false
25003 7631 Acked Write 2 Done Sent Done false
25004 7632 Acked Read 10 Done Sent Done false
25005 7633 Acked Write 7 Done Sent Done false
25006 7634 Acked Write 9 Done Sent Done false
25007 7635 Acked Flush 0 Done Sent Done false
25008 7636 Acked Read 5 Done Sent Done false
25009 7637 Acked Write 5 Done Sent Done false
25010 7638 Acked Read 10 Done Sent Done false
25011 7639 Acked Flush 0 Done Sent Done false
25012 STATES DS:0 DS:1 DS:2 TOTAL
25013 Sent 0 302 0 302
25014 Done 302 0 302 604
25015 Skipped 0 0 0 0
25016 Error 0 0 0 0
25017 Last Flush: 7639 7337 7639
25018 Downstairs last five completed: 7337 7336 7335 7334 7333
25019 Upstairs last five completed: 7639 7638 7637 7636 7635
25020 CLIENT: Up:0 ds:302 act:2
25021 [8820] got start action
25022 [8820] I am going to restart
25023 Starting downstairs at port 8820
25024 Make output file at "/tmp/dsc/downstairs-8820.txt"
25025 [8820][1] reports Starting
25026 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1222)
25027 [8820][1] reports Running
25028 ----------------------------------------------------------------
25029 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
25030 States: Active Offline Active
25031 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25032 7338 Acked Flush 0 Done Sent Done false
25033 7339 Acked Read 1 Done Sent Done false
25034 7340 Acked Read 8 Done Sent Done false
25035 7341 Acked Write 2 Done Sent Done false
25036 7342 Acked Flush 0 Done Sent Done false
25037 7343 Acked Read 1 Done Sent Done false
25038 7344 Acked Write 4 Done Sent Done false
25039 7345 Acked Read 2 Done Sent Done false
25040 7346 Acked Flush 0 Done Sent Done false
25041 7347 Acked Read 6 Done Sent Done false
25042 7348 Acked Write 7 Done Sent Done false
25043 7349 Acked Write 7 Done Sent Done false
25044 7350 Acked Read 2 Done Sent Done false
25045 7351 Acked Read 1 Done Sent Done false
25046 7352 Acked Write 9 Done Sent Done false
25047 7353 Acked Write 4 Done Sent Done false
25048 7354 Acked Write 1 Done Sent Done false
25049 7355 Acked Write 6 Done Sent Done false
25050 7356 Acked Write 6 Done Sent Done false
25051 7357 Acked Write 3 Done Sent Done false
25052 7358 Acked Read 9 Done Sent Done false
25053 7359 Acked Read 3 Done Sent Done false
25054 7360 Acked Write 3 Done Sent Done false
25055 7361 Acked Write 8 Done Sent Done false
25056 7362 Acked Write 10 Done Sent Done false
25057 7363 Acked Read 4 Done Sent Done false
25058 7364 Acked Read 5 Done Sent Done false
25059 7365 Acked Read 3 Done Sent Done false
25060 7366 Acked Read 4 Done Sent Done false
25061 7367 Acked Read 2 Done Sent Done false
25062 7368 Acked Write 5 Done Sent Done false
25063 7369 Acked Write 7 Done Sent Done false
25064 7370 Acked Write 10 Done Sent Done false
25065 7371 Acked Flush 0 Done Sent Done false
25066 7372 Acked Read 10 Done Sent Done false
25067 7373 Acked Write 2 Done Sent Done false
25068 7374 Acked Write 6 Done Sent Done false
25069 7375 Acked Flush 0 Done Sent Done false
25070 7376 Acked Read 8 Done Sent Done false
25071 7377 Acked Read 3 Done Sent Done false
25072 7378 Acked Read 4 Done Sent Done false
25073 7379 Acked Read 10 Done Sent Done false
25074 7380 Acked Read 1 Done Sent Done false
25075 7381 Acked Write 7 Done Sent Done false
25076 7382 Acked Flush 0 Done Sent Done false
25077 7383 Acked Write 9 Done Sent Done false
25078 7384 Acked Write 5 Done Sent Done false
25079 7385 Acked Read 7 Done Sent Done false
25080 7386 Acked Write 8 Done Sent Done false
25081 7387 Acked Read 4 Done Sent Done false
25082 7388 Acked Write 8 Done Sent Done false
25083 7389 Acked Write 8 Done Sent Done false
25084 7390 Acked Read 9 Done Sent Done false
25085 7391 Acked Read 10 Done Sent Done false
25086 7392 Acked Write 2 Done Sent Done false
25087 7393 Acked Write 3 Done Sent Done false
25088 7394 Acked Write 8 Done Sent Done false
25089 7395 Acked Read 1 Done Sent Done false
25090 7396 Acked Read 8 Done Sent Done false
25091 7397 Acked Read 7 Done Sent Done false
25092 7398 Acked Read 3 Done Sent Done false
25093 7399 Acked Read 3 Done Sent Done false
25094 7400 Acked Read 4 Done Sent Done false
25095 7401 Acked Write 5 Done Sent Done false
25096 7402 Acked Read 3 Done Sent Done false
25097 7403 Acked Flush 0 Done Sent Done false
25098 7404 Acked Read 7 Done Sent Done false
25099 7405 Acked Read 10 Done Sent Done false
25100 7406 Acked Read 1 Done Sent Done false
25101 7407 Acked Write 6 Done Sent Done false
25102 7408 Acked Read 9 Done Sent Done false
25103 7409 Acked Write 10 Done Sent Done false
25104 7410 Acked Read 4 Done Sent Done false
25105 7411 Acked Write 9 Done Sent Done false
25106 7412 Acked Read 4 Done Sent Done false
25107 7413 Acked Read 10 Done Sent Done false
25108 7414 Acked Write 3 Done Sent Done false
25109 7415 Acked Write 7 Done Sent Done false
25110 7416 Acked Write 5 Done Sent Done false
25111 7417 Acked Read 8 Done Sent Done false
25112 7418 Acked Write 3 Done Sent Done false
25113 7419 Acked Read 1 Done Sent Done false
25114 7420 Acked Write 5 Done Sent Done false
25115 7421 Acked Write 9 Done Sent Done false
25116 7422 Acked Read 9 Done Sent Done false
25117 7423 Acked Write 8 Done Sent Done false
25118 7424 Acked Read 8 Done Sent Done false
25119 7425 Acked Read 1 Done Sent Done false
25120 7426 Acked Read 4 Done Sent Done false
25121 7427 Acked Write 3 Done Sent Done false
25122 7428 Acked Read 8 Done Sent Done false
25123 7429 Acked Read 5 Done Sent Done false
25124 7430 Acked Flush 0 Done Sent Done false
25125 7431 Acked Write 7 Done Sent Done false
25126 7432 Acked Write 7 Done Sent Done false
25127 7433 Acked Read 5 Done Sent Done false
25128 7434 Acked Read 5 Done Sent Done false
25129 7435 Acked Read 6 Done Sent Done false
25130 7436 Acked Read 7 Done Sent Done false
25131 7437 Acked Write 5 Done Sent Done false
25132 7438 Acked Write 9 Done Sent Done false
25133 7439 Acked Write 2 Done Sent Done false
25134 7440 Acked Flush 0 Done Sent Done false
25135 7441 Acked Flush 0 Done Sent Done false
25136 7442 Acked Read 10 Done Sent Done false
25137 7443 Acked Write 2 Done Sent Done false
25138 7444 Acked Read 8 Done Sent Done false
25139 7445 Acked Read 1 Done Sent Done false
25140 7446 Acked Write 3 Done Sent Done false
25141 7447 Acked Write 2 Done Sent Done false
25142 7448 Acked Read 9 Done Sent Done false
25143 7449 Acked Read 5 Done Sent Done false
25144 7450 Acked Read 10 Done Sent Done false
25145 7451 Acked Read 4 Done Sent Done false
25146 7452 Acked Read 1 Done Sent Done false
25147 7453 Acked Read 8 Done Sent Done false
25148 7454 Acked Write 5 Done Sent Done false
25149 7455 Acked Write 1 Done Sent Done false
25150 7456 Acked Read 3 Done Sent Done false
25151 7457 Acked Write 4 Done Sent Done false
25152 7458 Acked Write 1 Done Sent Done false
25153 7459 Acked Read 5 Done Sent Done false
25154 7460 Acked Read 10 Done Sent Done false
25155 7461 Acked Read 5 Done Sent Done false
25156 7462 Acked Write 6 Done Sent Done false
25157 7463 Acked Write 8 Done Sent Done false
25158 7464 Acked Read 3 Done Sent Done false
25159 7465 Acked Read 4 Done Sent Done false
25160 7466 Acked Flush 0 Done Sent Done false
25161 7467 Acked Read 5 Done Sent Done false
25162 7468 Acked Read 1 Done Sent Done false
25163 7469 Acked Read 2 Done Sent Done false
25164 7470 Acked Read 9 Done Sent Done false
25165 7471 Acked Read 9 Done Sent Done false
25166 7472 Acked Read 2 Done Sent Done false
25167 7473 Acked Read 2 Done Sent Done false
25168 7474 Acked Write 2 Done Sent Done false
25169 7475 Acked Read 3 Done Sent Done false
25170 7476 Acked Read 6 Done Sent Done false
25171 7477 Acked Write 2 Done Sent Done false
25172 7478 Acked Write 8 Done Sent Done false
25173 7479 Acked Write 6 Done Sent Done false
25174 7480 Acked Read 7 Done Sent Done false
25175 7481 Acked Write 6 Done Sent Done false
25176 7482 Acked Flush 0 Done Sent Done false
25177 7483 Acked Write 9 Done Sent Done false
25178 7484 Acked Flush 0 Done Sent Done false
25179 7485 Acked Write 1 Done Sent Done false
25180 7486 Acked Read 1 Done Sent Done false
25181 7487 Acked Write 10 Done Sent Done false
25182 7488 Acked Write 4 Done Sent Done false
25183 7489 Acked Read 10 Done Sent Done false
25184 7490 Acked Read 7 Done Sent Done false
25185 7491 Acked Write 3 Done Sent Done false
25186 7492 Acked Write 10 Done Sent Done false
25187 7493 Acked Write 2 Done Sent Done false
25188 7494 Acked Write 7 Done Sent Done false
25189 7495 Acked Read 8 Done Sent Done false
25190 7496 Acked Read 2 Done Sent Done false
25191 7497 Acked Read 1 Done Sent Done false
25192 7498 Acked Write 6 Done Sent Done false
25193 7499 Acked Write 7 Done Sent Done false
25194 7500 Acked Write 10 Done Sent Done false
25195 7501 Acked Flush 0 Done Sent Done false
25196 7502 Acked Read 4 Done Sent Done false
25197 7503 Acked Read 10 Done Sent Done false
25198 7504 Acked Flush 0 Done Sent Done false
25199 7505 Acked Read 8 Done Sent Done false
25200 7506 Acked Read 1 Done Sent Done false
25201 7507 Acked Read 3 Done Sent Done false
25202 7508 Acked Read 9 Done Sent Done false
25203 7509 Acked Write 7 Done Sent Done false
25204 7510 Acked Read 9 Done Sent Done false
25205 7511 Acked Write 2 Done Sent Done false
25206 7512 Acked Read 1 Done Sent Done false
25207 7513 Acked Write 10 Done Sent Done false
25208 7514 Acked Read 1 Done Sent Done false
25209 7515 Acked Write 7 Done Sent Done false
25210 7516 Acked Read 8 Done Sent Done false
25211 7517 Acked Write 9 Done Sent Done false
25212 7518 Acked Read 10 Done Sent Done false
25213 7519 Acked Read 8 Done Sent Done false
25214 7520 Acked Write 6 Done Sent Done false
25215 7521 Acked Read 4 Done Sent Done false
25216 7522 Acked Read 6 Done Sent Done false
25217 7523 Acked Flush 0 Done Sent Done false
25218 7524 Acked Read 8 Done Sent Done false
25219 7525 Acked Write 6 Done Sent Done false
25220 7526 Acked Write 2 Done Sent Done false
25221 7527 Acked Flush 0 Done Sent Done false
25222 7528 Acked Read 9 Done Sent Done false
25223 7529 Acked Write 7 Done Sent Done false
25224 7530 Acked Read 7 Done Sent Done false
25225 7531 Acked Write 7 Done Sent Done false
25226 7532 Acked Read 2 Done Sent Done false
25227 7533 Acked Flush 0 Done Sent Done false
25228 7534 Acked Read 3 Done Sent Done false
25229 7535 Acked Read 8 Done Sent Done false
25230 7536 Acked Flush 0 Done Sent Done false
25231 7537 Acked Read 6 Done Sent Done false
25232 7538 Acked Read 1 Done Sent Done false
25233 7539 Acked Read 5 Done Sent Done false
25234 7540 Acked Read 3 Done Sent Done false
25235 7541 Acked Read 5 Done Sent Done false
25236 7542 Acked Read 6 Done Sent Done false
25237 7543 Acked Read 2 Done Sent Done false
25238 7544 Acked Read 2 Done Sent Done false
25239 7545 Acked Read 6 Done Sent Done false
25240 7546 Acked Read 6 Done Sent Done false
25241 7547 Acked Flush 0 Done Sent Done false
25242 7548 Acked Write 6 Done Sent Done false
25243 7549 Acked Write 4 Done Sent Done false
25244 7550 Acked Read 8 Done Sent Done false
25245 7551 Acked Read 3 Done Sent Done false
25246 7552 Acked Read 7 Done Sent Done false
25247 7553 Acked Write 10 Done Sent Done false
25248 7554 Acked Read 7 Done Sent Done false
25249 7555 Acked Flush 0 Done Sent Done false
25250 7556 Acked Flush 0 Done Sent Done false
25251 7557 Acked Read 2 Done Sent Done false
25252 7558 Acked Write 5 Done Sent Done false
25253 7559 Acked Write 9 Done Sent Done false
25254 7560 Acked Read 1 Done Sent Done false
25255 7561 Acked Read 3 Done Sent Done false
25256 7562 Acked Write 8 Done Sent Done false
25257 7563 Acked Read 1 Done Sent Done false
25258 7564 Acked Read 4 Done Sent Done false
25259 7565 Acked Read 7 Done Sent Done false
25260 7566 Acked Read 9 Done Sent Done false
25261 7567 Acked Read 5 Done Sent Done false
25262 7568 Acked Write 7 Done Sent Done false
25263 7569 Acked Write 7 Done Sent Done false
25264 7570 Acked Write 9 Done Sent Done false
25265 7571 Acked Read 3 Done Sent Done false
25266 7572 Acked Read 4 Done Sent Done false
25267 7573 Acked Read 1 Done Sent Done false
25268 7574 Acked Write 10 Done Sent Done false
25269 7575 Acked Write 2 Done Sent Done false
25270 7576 Acked Flush 0 Done Sent Done false
25271 7577 Acked Read 8 Done Sent Done false
25272 7578 Acked Write 7 Done Sent Done false
25273 7579 Acked Write 8 Done Sent Done false
25274 7580 Acked Write 2 Done Sent Done false
25275 7581 Acked Write 3 Done Sent Done false
25276 7582 Acked Read 2 Done Sent Done false
25277 7583 Acked Read 4 Done Sent Done false
25278 7584 Acked Read 7 Done Sent Done false
25279 7585 Acked Read 4 Done Sent Done false
25280 7586 Acked Write 9 Done Sent Done false
25281 7587 Acked Read 5 Done Sent Done false
25282 7588 Acked Read 10 Done Sent Done false
25283 7589 Acked Write 3 Done Sent Done false
25284 7590 Acked Read 10 Done Sent Done false
25285 7591 Acked Write 10 Done Sent Done false
25286 7592 Acked Flush 0 Done Sent Done false
25287 7593 Acked Read 8 Done Sent Done false
25288 7594 Acked Flush 0 Done Sent Done false
25289 7595 Acked Read 5 Done Sent Done false
25290 7596 Acked Write 7 Done Sent Done false
25291 7597 Acked Read 9 Done Sent Done false
25292 7598 Acked Read 4 Done Sent Done false
25293 7599 Acked Write 8 Done Sent Done false
25294 7600 Acked Read 9 Done Sent Done false
25295 7601 Acked Read 9 Done Sent Done false
25296 7602 Acked Flush 0 Done Sent Done false
25297 7603 Acked Read 6 Done Sent Done false
25298 7604 Acked Write 7 Done Sent Done false
25299 7605 Acked Read 5 Done Sent Done false
25300 7606 Acked Read 10 Done Sent Done false
25301 7607 Acked Read 7 Done Sent Done false
25302 7608 Acked Flush 0 Done Sent Done false
25303 7609 Acked Read 1 Done Sent Done false
25304 7610 Acked Read 4 Done Sent Done false
25305 7611 Acked Flush 0 Done Sent Done false
25306 7612 Acked Read 5 Done Sent Done false
25307 7613 Acked Read 8 Done Sent Done false
25308 7614 Acked Write 8 Done Sent Done false
25309 7615 Acked Write 5 Done Sent Done false
25310 7616 Acked Read 5 Done Sent Done false
25311 7617 Acked Write 7 Done Sent Done false
25312 7618 Acked Read 5 Done Sent Done false
25313 7619 Acked Write 7 Done Sent Done false
25314 7620 Acked Read 8 Done Sent Done false
25315 7621 Acked Read 2 Done Sent Done false
25316 7622 Acked Write 7 Done Sent Done false
25317 7623 Acked Write 3 Done Sent Done false
25318 7624 Acked Write 3 Done Sent Done false
25319 7625 Acked Read 8 Done Sent Done false
25320 7626 Acked Read 5 Done Sent Done false
25321 7627 Acked Read 7 Done Sent Done false
25322 7628 Acked Flush 0 Done Sent Done false
25323 7629 Acked Flush 0 Done Sent Done false
25324 7630 Acked Write 4 Done Sent Done false
25325 7631 Acked Write 2 Done Sent Done false
25326 7632 Acked Read 10 Done Sent Done false
25327 7633 Acked Write 7 Done Sent Done false
25328 7634 Acked Write 9 Done Sent Done false
25329 7635 Acked Flush 0 Done Sent Done false
25330 7636 Acked Read 5 Done Sent Done false
25331 7637 Acked Write 5 Done Sent Done false
25332 7638 Acked Read 10 Done Sent Done false
25333 7639 Acked Flush 0 Done Sent Done false
25334 STATES DS:0 DS:1 DS:2 TOTAL
25335 Sent 0 302 0 302
25336 Done 302 0 302 604
25337 Skipped 0 0 0 0
25338 Error 0 0 0 0
25339 Last Flush: 7639 7337 7639
25340 Downstairs last five completed: 7337 7336 7335 7334 7333
25341 Upstairs last five completed: 7639 7638 7637 7636 7635
25342 CLIENT: Up:0 ds:302 act:2
253432025-01-27T19:41:42.120ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253442025-01-27T19:41:42.120ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:33987) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253452025-01-27T19:41:42.122ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253462025-01-27T19:41:42.122ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253472025-01-27T19:41:42.122ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253482025-01-27T19:41:42.122ZINFOcrucible: send last flush ID to this DS: 7337 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253492025-01-27T19:41:42.122ZINFOcrucible: Replied this last flush ID: 7337 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253502025-01-27T19:41:42.122ZINFOcrucible: [1] client re-new 302 jobs since flush 7337 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253512025-01-27T19:41:42.123ZINFOcrucible: [1] Replayed 302 jobs since flush: 7337 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
25352 ----------------------------------------------------------------
25353 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
25354 Downstairs last five completed: 7639 7638 7637 7636 7635
25355 Upstairs last five completed: 7639 7638 7637 7636 7635
25356 CLIENT: Up:0 ds:0 act:3
25357 Replay: All jobs finished
25358 Main task has work to do, go find it
25359 got dsc Stop(2)
25360 stop 2
25361 [8830] Got stop action so:false kr:false
25362 Jan 27 19:41:42.369 INFO request completed, latency_us: 321, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 205b0e3d-6fb2-4683-a876-e934a730ba04, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
253632025-01-27T19:41:42.369ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
253642025-01-27T19:41:42.369ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
25365 Jan 27 19:41:42.369 INFO request completed, {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:41:42.36987047Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
25366 latency_us: 129, response_code: 200, uri: /state/cid/2, method: GET, req_id: 625089a1-cddc-46bd-8001-2eb77fe3d7fe, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
253672025-01-27T19:41:42.369ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
25368 [8830] Exited with: ExitStatus(unix_wait_status(9))
25369 [8830][2] reports Exit
25370 Jan 27 19:41:46.373 INFO request completed, latency_us: 335, response_code: 200, uri: /state/cid/2, method: GET, req_id: 9afaddf4-4d77-4f63-8dcc-e2d46d45b9b2, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
25371 001/300 Read block 212 len 28672
25372 002/300 Read block 600 len 12288
25373 003/300 Read block 246 len 12288
25374 004/300 Read block 4945 len 28672
25375 005/300 Read block 738 len 12288
25376 006/300 Write block 4466 len 24576 data: 3 3 4 4 4 4
25377 007/300 Read block 4179 len 12288
25378 008/300 Write block 2376 len 12288 data: 7 8 10
25379 009/300 Write block 3509 len 24576 data: 5 5 3 3 2 3
25380 010/300 Write block 1024 len 28672 data: 3 3 2 2 4 6 6
25381 011/300 Read block 50 len 20480
25382 012/300 Read block 2211 len 36864
25383 013/300 Write block 2786 len 32768 data: 5 4 6 6 6 5 4 6
25384 014/300 Write block 1597 len 8192 data: 10 11
25385 015/300 Read block 4773 len 40960
25386 016/300 Write block 1795 len 16384 data: 5 4 4 3
25387 017/300 Read block 639 len 16384
25388 018/300 Read block 329 len 32768
25389 019/300 Write block 467 len 24576 data: 4 4 4 6 6 4
25390 020/300 Read block 1655 len 4096
25391 021/300 Read block 4993 len 28672
25392 022/300 Read block 407 len 4096
25393 023/300 Write block 738 len 16384 data: 5 5 5 5
25394 024/300 Read block 4463 len 24576
25395 025/300 Write block 3509 len 4096 data: 6
25396 026/300 Read block 1841 len 40960
25397 027/300 Flush
25398 028/300 Write block 4168 len 8192 data: 7 6
25399 029/300 Write block 2188 len 4096 data: 4
25400 030/300 Read block 4708 len 16384
25401 031/300 Flush
25402 032/300 Read block 3409 len 20480
25403 033/300 Read block 813 len 40960
25404 034/300 Read block 763 len 24576
25405 035/300 Write block 1704 len 20480 data: 4 4 4 3 3
25406 036/300 Write block 2908 len 4096 data: 4
25407 037/300 Read block 4331 len 36864
25408 038/300 Write block 4966 len 32768 data: 3 5 6 6 7 7 5 5
25409 039/300 Read block 4235 len 4096
25410 040/300 Read block 4662 len 32768
25411 041/300 Read block 4001 len 8192
25412 042/300 Flush
25413 043/300 Read block 3088 len 20480
25414 044/300 Write block 4008 len 8192 data: 6 7
25415 045/300 Write block 4582 len 20480 data: 2 2 2 2 3
25416 046/300 Write block 3686 len 28672 data: 4 5 6 6 6 6 6
25417 047/300 Read block 3885 len 40960
25418 048/300 Write block 2151 len 12288 data: 3 4 5
25419 049/300 Write block 344 len 32768 data: 2 2 3 3 3 3 3 6
25420 050/300 Read block 4215 len 32768
25421 051/300 Write block 3763 len 20480 data: 5 5 6 6 7
25422 052/300 Read block 4223 len 36864
25423 053/300 Read block 2969 len 8192
25424 054/300 Read block 4923 len 24576
25425 055/300 Read block 1529 len 36864
25426 056/300 Read block 1279 len 28672
25427 057/300 Write block 4776 len 16384 data: 7 8 7 5
25428 058/300 Read block 4344 len 32768
25429 059/300 Read block 2436 len 36864
25430 060/300 Write block 990 len 16384 data: 3 3 3 2
25431 061/300 Flush
25432 062/300 Flush
25433 063/300 Read block 2379 len 12288
25434 064/300 Read block 775 len 4096
25435 065/300 Read block 556 len 8192
25436 066/300 Read block 1052 len 20480
25437 067/300 Read block 3902 len 8192
25438 068/300 Write block 2297 len 8192 data: 2 4
25439 069/300 Read block 4035 len 4096
25440 070/300 Write block 3019 len 16384 data: 4 6 5 5
25441 071/300 Write block 1207 len 36864 data: 6 6 5 4 3 3 3 4 5
25442 072/300 Read block 1144 len 20480
25443 073/300 Write block 1160 len 4096 data: 2
25444 074/300 Read block 4719 len 8192
25445 075/300 Flush
25446 076/300 Write block 2436 len 20480 data: 3 6 6 6 7
25447 077/300 Write block 3432 len 16384 data: 7 8 8 7
25448 078/300 Write block 4264 len 12288 data: 7 7 7
25449 079/300 Read block 4593 len 8192
25450 080/300 Read block 4860 len 12288
25451 081/300 Write block 802 len 32768 data: 7 6 4 2 3 4 4 6
25452 082/300 Write block 3169 len 32768 data: 4 4 4 5 5 7 9 10
25453 083/300 Write block 2135 len 4096 data: 3
25454 084/300 Read block 4021 len 40960
25455 085/300 Read block 1508 len 32768
25456 086/300 Read block 842 len 4096
25457 087/300 Read block 4328 len 4096
25458 088/300 Write block 2210 len 8192 data: 4 4
25459 089/300 Read block 1180 len 8192
25460 090/300 Write block 1057 len 32768 data: 3 4 5 5 4 3 3 3
25461 091/300 Flush
25462 092/300 Write block 623 len 28672 data: 5 4 3 3 5 5 5
25463 093/300 Flush
25464 094/300 Read block 1392 len 12288
25465 095/300 Write block 1491 len 20480 data: 2 2 2 2 2
25466 096/300 Flush
25467 097/300 Read block 4877 len 20480
25468 098/300 Write block 2199 len 24576 data: 5 4 7 6 7 5
25469 099/300 Read block 4832 len 4096
25470 100/300 Write block 3992 len 32768 data: 3 4 4 3 3 3 3 3
25471 101/300 Write block 2935 len 32768 data: 6 7 5 4 4 4 5 5
25472 102/300 Write block 946 len 20480 data: 4 4 3 3 3
25473 103/300 Write block 3422 len 40960 data: 3 3 4 5 9 8 8 8 9 8
25474 104/300 Read block 4873 len 28672
25475 105/300 Read block 3173 len 8192
25476 106/300 Write block 4349 len 28672 data: 10 9 8 8 6 5 6
25477 107/300 Read block 4161 len 4096
25478 108/300 Write block 519 len 40960 data: 6 6 5 4 5 3 2 2 2 3
25479 109/300 Write block 987 len 24576 data: 4 4 4 4 4 4
25480 110/300 Read block 2389 len 40960
25481 111/300 Write block 1294 len 40960 data: 5 5 5 7 8 7 6 8 7 9
25482 112/300 Write block 4035 len 24576 data: 6 7 7 4 4 4
25483 113/300 Write block 422 len 4096 data: 6
25484 114/300 Write block 1526 len 16384 data: 4 5 3 4
25485 115/300 Read block 1059 len 24576
25486 116/300 Read block 4 len 32768
25487 117/300 Read block 4361 len 20480
25488 118/300 Write block 2579 len 40960 data: 4 4 3 2 2 2 2 4 5 5
25489 119/300 Write block 665 len 16384 data: 4 4 4 4
25490 120/300 Write block 245 len 40960 data: 2 3 2 2 3 3 3 5 6 5
25491 121/300 Flush
25492 122/300 Write block 4405 len 32768 data: 4 4 5 5 4 4 6 5
25493 123/300 Write block 2426 len 12288 data: 4 4 5
25494 124/300 Read block 3235 len 32768
25495 125/300 Write block 2559 len 36864 data: 6 6 6 7 8 7 7 7 4
25496 126/300 Read block 3031 len 28672
25497 127/300 Write block 1942 len 8192 data: 6 6
25498 128/300 Read block 162 len 24576
25499 129/300 Write block 591 len 28672 data: 7 6 7 6 5 5 6
25500 130/300 Read block 2296 len 24576
25501 131/300 Write block 1425 len 4096 data: 5
25502 132/300 Write block 1423 len 20480 data: 4 4 6 5 5
25503 133/300 Write block 1971 len 24576 data: 3 3 4 4 3 4
25504 134/300 Read block 3007 len 16384
25505 135/300 Write block 1040 len 36864 data: 3 3 4 4 3 3 3 4 3
25506 136/300 Flush
25507 137/300 Write block 2585 len 12288 data: 3 5 6
25508 138/300 Read block 4004 len 40960
25509 139/300 Flush
25510 140/300 Read block 1584 len 12288
25511 141/300 Read block 2283 len 40960
25512 142/300 Flush
25513 143/300 Read block 4664 len 16384
25514 144/300 Flush
25515 145/300 Write block 34 len 4096 data: 2
25516 146/300 Read block 1850 len 20480
25517 147/300 Read block 1130 len 8192
25518 148/300 Write block 3271 len 12288 data: 5 6 6
25519 149/300 Read block 1718 len 20480
25520 150/300 Write block 1563 len 12288 data: 2 2 3
25521 151/300 Write block 1887 len 24576 data: 7 8 8 7 5 5
25522 152/300 Write block 1095 len 12288 data: 5 5 5
25523 153/300 Flush
25524 154/300 Write block 4841 len 40960 data: 6 7 6 6 5 5 2 2 2 2
25525 155/300 Read block 2119 len 4096
25526 156/300 Read block 728 len 28672
25527 157/300 Read block 209 len 28672
25528 158/300 Read block 1301 len 16384
25529 159/300 Write block 4490 len 28672 data: 4 4 6 6 7 7 5
25530 160/300 Read block 3523 len 4096
25531 161/300 Read block 3559 len 12288
25532 162/300 Read block 3409 len 20480
25533 163/300 Write block 2614 len 36864 data: 5 4 4 5 5 7 7 8 8
25534 164/300 Read block 3077 len 4096
25535 165/300 Write block 959 len 40960 data: 7 8 8 7 5 5 4 2 3 4
25536 166/300 Read block 1675 len 40960
25537 167/300 Read block 2626 len 36864
25538 168/300 Read block 4852 len 40960
25539 169/300 Read block 2161 len 24576
25540 170/300 Write block 2154 len 20480 data: 7 7 9 10 9
25541 171/300 Read block 1218 len 4096
25542 172/300 Flush
25543 173/300 Write block 4807 len 24576 data: 7 7 5 7 6 4
25544 174/300 Write block 3779 len 24576 data: 6 5 4 3 3 3
25545 175/300 Read block 2848 len 28672
25546 176/300 Flush
25547 177/300 Write block 1220 len 36864 data: 6 6 6 7 7 6 5 4 4
25548 178/300 Read block 2742 len 16384
25549 179/300 Write block 891 len 24576 data: 5 6 4 3 3 2
25550 180/300 Read block 1945 len 28672
25551 181/300 Read block 1679 len 24576
25552 182/300 Read block 348 len 28672
25553 183/300 Read block 1499 len 28672
25554 184/300 Read block 1177 len 40960
25555 185/300 Write block 1480 len 12288 data: 7 7 7
25556 186/300 Read block 1177 len 20480
25557 187/300 Read block 481 len 36864
25558 188/300 Write block 4229 len 12288 data: 4 4 4
25559 189/300 Read block 3486 len 8192
25560 190/300 Write block 3739 len 16384 data: 3 3 2 3
25561 191/300 Read block 2713 len 16384
25562 192/300 Read block 1896 len 16384
25563 193/300 Read block 44 len 4096
25564 194/300 Read block 2411 len 4096
25565 195/300 Write block 654 len 36864 data: 5 5 4 4 7 6 5 5 4
25566 196/300 Write block 1676 len 28672 data: 7 4 4 4 3 3 2
25567 197/300 Write block 766 len 4096 data: 4
25568 198/300 Flush
25569 199/300 Write block 3361 len 24576 data: 5 5 4 5 5 5
25570 200/300 Read block 4173 len 16384
25571 201/300 Write block 2529 len 8192 data: 4 4
25572 202/300 Write block 803 len 12288 data: 7 5 3
25573 203/300 Flush
25574 204/300 Read block 2134 len 24576
25575 205/300 Read block 1874 len 32768
25576 206/300 Write block 2739 len 12288 data: 3 3 2
25577 207/300 Read block 4546 len 24576
25578 208/300 Write block 2295 len 20480 data: 3 3 3 5 6
25579 209/300 Read block 3524 len 28672
25580 210/300 Flush
25581 211/300 Write block 1983 len 20480 data: 6 6 7 6 6
25582 212/300 Read block 52 len 12288
25583 213/300 Read block 2713 len 28672
25584 214/300 Read block 2399 len 12288
25585 215/300 Flush
25586 216/300 Read block 4068 len 16384
25587 217/300 Write block 3604 len 40960 data: 5 5 8 9 9 7 6 4 3 3
25588 218/300 Read block 1477 len 32768
25589 219/300 Read block 2563 len 12288
25590 220/300 Read block 3221 len 20480
25591 221/300 Write block 826 len 16384 data: 3 3 5 5
25592 222/300 Read block 1603 len 28672
25593 223/300 Read block 710 len 8192
25594 224/300 Write block 1130 len 40960 data: 7 6 6 5 5 5 4 5 4 5
25595 225/300 Write block 363 len 24576 data: 4 6 5 5 5 4
25596 226/300 Read block 4565 len 24576
25597 227/300 Read block 2061 len 20480
25598 228/300 Write block 3359 len 32768 data: 4 5 6 6 5 6 6 6
25599 229/300 Read block 1441 len 20480
25600 230/300 Read block 2414 len 32768
25601 231/300 Write block 4264 len 32768 data: 8 8 8 6 7 6 6 8
25602 232/300 Read block 2849 len 32768
25603 233/300 Read block 2093 len 36864
25604 234/300 Write block 1283 len 28672 data: 3 3 2 2 2 2 2
25605 235/300 Read block 716 len 20480
25606 236/300 Read block 939 len 28672
25607 237/300 Write block 523 len 8192 data: 6 4
25608 238/300 Write block 2929 len 36864 data: 7 6 6 6 7 7 7 8 6
25609 239/300 Write block 60 len 12288 data: 7 5 6
25610 240/300 Write block 3812 len 4096 data: 5
25611 241/300 Write block 4466 len 20480 data: 4 4 5 5 5
25612 242/300 Write block 3588 len 12288 data: 2 2 2
25613 243/300 Read block 2478 len 36864
25614 244/300 Read block 4260 len 12288
25615 245/300 Flush
25616 246/300 Read block 1047 len 8192
25617 247/300 Read block 3790 len 12288
25618 248/300 Read block 1089 len 8192
25619 249/300 Read block 145 len 28672
25620 250/300 Read block 1042 len 28672
25621 251/300 Write block 4295 len 20480 data: 5 3 5 5 5
25622 252/300 Flush
25623 253/300 Flush
25624 254/300 Read block 3198 len 16384
25625 255/300 Write block 928 len 12288 data: 4 4 6
25626 256/300 Read block 657 len 32768
25627 257/300 Read block 2645 len 8192
25628 258/300 Read block 4033 len 4096
25629 259/300 Write block 2918 len 16384 data: 3 3 3 2
25630 260/300 Write block 2990 len 8192 data: 4 4
25631 261/300 Read block 2088 len 4096
25632 262/300 Read block 1620 len 28672
25633 263/300 Read block 4704 len 12288
25634 264/300 Write block 4800 len 28672 data: 7 7 6 8 7 10 6
25635 265/300 Flush
25636 266/300 Read block 3810 len 16384
25637 267/300 Write block 3208 len 28672 data: 4 5 4 6 6 6 4
25638 268/300 Read block 2518 len 40960
25639 269/300 Read block 1969 len 24576
25640 270/300 Read block 4311 len 32768
25641 271/300 Read block 4109 len 32768
25642 272/300 Write block 3720 len 36864 data: 5 6 5 5 6 7 7 7 6
25643 273/300 Read block 4104 len 12288
25644 274/300 Write block 4453 len 12288 data: 5 4 3
25645 275/300 Read block 4262 len 28672
25646 276/300 Read block 2436 len 28672
25647 277/300 Read block 2873 len 12288
25648 278/300 Write block 4271 len 36864 data: 9 6 6 5 8 7 7 7 7
25649 279/300 Read block 2533 len 36864
25650 280/300 Read block 821 len 36864
25651 281/300 Read block 4092 len 16384
25652 282/300 Write block 1842 len 24576 data: 2 3 3 4 4 5
25653 283/300 Flush
25654 284/300 Flush
25655 285/300 Read block 4562 len 36864
25656 286/300 Write block 2231 len 16384 data: 5 3 2 4
25657 287/300 Write block 1825 len 12288 data: 3 3 3
25658 288/300 Write block 368 len 20480 data: 5 4 3 5 6
25659 289/300 Read block 2584 len 8192
25660 290/300 Write block 759 len 32768 data: 3 3 3 3 3 3 4 5
25661 291/300 Write block 3103 len 40960 data: 7 9 8 9 9 9 7 5 4 3
25662 292/300 Read block 2307 len 16384
25663 293/300 Write block 875 len 8192 data: 4 4
25664 294/300 Write block 4593 len 32768 data: 5 6 5 5 5 3 4 4
25665 295/300 Read block 3325 len 4096
25666 296/300 Write block 1367 len 20480 data: 7 6 6 7 6
25667 297/300 Write block 444 len 24576 data: 3 4 3 3 3 2
25668 298/300 Read block 4409 len 16384
25669 299/300 Write block 3002 len 40960 data: 7 7 7 6 6 6 5 5 4 5
25670 300/300 Write block 3644 len 36864 data: 5 5 6 5 6 7 6 6 4
25671 301/300 Write block 1476 len 24576 data: 5 4 5 7 8 8
25672 Main task has work to do, go find it
25673 got dsc Start(2)
25674 start 2
25675 Jan 27 19:41:46.606 INFO request completed, latency_us: 212, response_code: 204, uri: /start/cid/2, method: POST, req_id: d9043ca4-d777-4264-b9c7-856c6ed4042a, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
25676 Replay: started 2, returned:Ok(())
25677 ----------------------------------------------------------------
25678 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
25679 States: Active Active Offline
25680 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
25681 7640 Acked Read 7 Done Done Sent false
25682 7641 Acked Read 3 Done Done Sent false
25683 7642 Acked Read 3 Done Done Sent false
25684 7643 Acked Read 7 Done Done Sent false
25685 7644 Acked Read 3 Done Done Sent false
25686 7645 Acked Write 6 Done Done Sent false
25687 7646 Acked Read 3 Done Done Sent false
25688 7647 Acked Write 3 Done Done Sent false
25689 7648 Acked Write 6 Done Done Sent false
25690 7649 Acked Write 7 Done Done Sent false
25691 7650 Acked Read 5 Done Done Sent false
25692 7651 Acked Read 9 Done Done Sent false
25693 7652 Acked Write 8 Done Done Sent false
25694 7653 Acked Write 2 Done Done Sent false
25695 7654 Acked Read 10 Done Done Sent false
25696 7655 Acked Write 4 Done Done Sent false
25697 7656 Acked Read 4 Done Done Sent false
25698 7657 Acked Read 8 Done Done Sent false
25699 7658 Acked Write 6 Done Done Sent false
25700 7659 Acked Read 1 Done Done Sent false
25701 7660 Acked Read 7 Done Done Sent false
25702 7661 Acked Read 1 Done Done Sent false
25703 7662 Acked Write 4 Done Done Sent false
25704 7663 Acked Read 6 Done Done Sent false
25705 7664 Acked Write 1 Done Done Sent false
25706 7665 Acked Read 10 Done Done Sent false
25707 7666 Acked Flush 0 Done Done Sent false
25708 7667 Acked Write 2 Done Done Sent false
25709 7668 Acked Write 1 Done Done Sent false
25710 7669 Acked Read 4 Done Done Sent false
25711 7670 Acked Flush 0 Done Done Sent false
25712 7671 Acked Read 5 Done Done Sent false
25713 7672 Acked Read 10 Done Done Sent false
25714 7673 Acked Read 6 Done Done Sent false
25715 7674 Acked Write 5 Done Done Sent false
25716 7675 Acked Write 1 Done Done Sent false
25717 7676 Acked Read 9 Done Done Sent false
25718 7677 Acked Write 8 Done Done Sent false
25719 7678 Acked Read 1 Done Done Sent false
25720 7679 Acked Read 8 Done Done Sent false
25721 7680 Acked Read 2 Done Done Sent false
25722 7681 Acked Flush 0 Done Done Sent false
25723 7682 Acked Read 5 Done Done Sent false
25724 7683 Acked Write 2 Done Done Sent false
25725 7684 Acked Write 5 Done Done Sent false
25726 7685 Acked Write 7 Done Done Sent false
25727 7686 Acked Read 10 Done Done Sent false
25728 7687 Acked Write 3 Done Done Sent false
25729 7688 Acked Write 8 Done Done Sent false
25730 7689 Acked Read 8 Done Done Sent false
25731 7690 Acked Write 5 Done Done Sent false
25732 7691 Acked Read 9 Done Done Sent false
25733 7692 Acked Read 2 Done Done Sent false
25734 7693 Acked Read 6 Done Done Sent false
25735 7694 Acked Read 9 Done Done Sent false
25736 7695 Acked Read 7 Done Done Sent false
25737 7696 Acked Write 4 Done Done Sent false
25738 7697 Acked Read 8 Done Done Sent false
25739 7698 Acked Read 9 Done Done Sent false
25740 7699 Acked Write 4 Done Done Sent false
25741 7700 Acked Flush 0 Done Done Sent false
25742 7701 Acked Flush 0 Done Done Sent false
25743 7702 Acked Read 3 Done Done Sent false
25744 7703 Acked Read 1 Done Done Sent false
25745 7704 Acked Read 2 Done Done Sent false
25746 7705 Acked Read 5 Done Done Sent false
25747 7706 Acked Read 2 Done Done Sent false
25748 7707 Acked Write 2 Done Done Sent false
25749 7708 Acked Read 1 Done Done Sent false
25750 7709 Acked Write 4 Done Done Sent false
25751 7710 Acked Write 9 Done Done Sent false
25752 7711 Acked Read 5 Done Done Sent false
25753 7712 Acked Write 1 Done Done Sent false
25754 7713 Acked Read 2 Done Done Sent false
25755 7714 Acked Flush 0 Done Done Sent false
25756 7715 Acked Write 5 Done Done Sent false
25757 7716 Acked Write 4 Done Done Sent false
25758 7717 Acked Write 3 Done Done Sent false
25759 7718 Acked Read 2 Done Done Sent false
25760 7719 Acked Read 3 Done Done Sent false
25761 7720 Acked Write 8 Done Done Sent false
25762 7721 Acked Write 8 Done Done Sent false
25763 7722 Acked Write 1 Done Done Sent false
25764 7723 Acked Read 10 Done Done Sent false
25765 7724 Acked Read 8 Done Done Sent false
25766 7725 Acked Read 1 Done Done Sent false
25767 7726 Acked Read 1 Done Done Sent false
25768 7727 Acked Write 2 Done Done Sent false
25769 7728 Acked Read 2 Done Done Sent false
25770 7729 Acked Write 8 Done Done Sent false
25771 7730 Acked Flush 0 Done Done Sent false
25772 7731 Acked Write 7 Done Done Sent false
25773 7732 Acked Flush 0 Done Done Sent false
25774 7733 Acked Read 3 Done Done Sent false
25775 7734 Acked Write 5 Done Done Sent false
25776 7735 Acked Flush 0 Done Done Sent false
25777 7736 Acked Read 5 Done Done Sent false
25778 7737 Acked Write 6 Done Done Sent false
25779 7738 Acked Read 1 Done Done Sent false
25780 7739 Acked Write 8 Done Done Sent false
25781 7740 Acked Write 8 Done Done Sent false
25782 7741 Acked Write 5 Done Done Sent false
25783 7742 Acked Write 10 Done Done Sent false
25784 7743 Acked Read 7 Done Done Sent false
25785 7744 Acked Read 2 Done Done Sent false
25786 7745 Acked Write 7 Done Done Sent false
25787 7746 Acked Read 1 Done Done Sent false
25788 7747 Acked Write 10 Done Done Sent false
25789 7748 Acked Write 6 Done Done Sent false
25790 7749 Acked Read 10 Done Done Sent false
25791 7750 Acked Write 10 Done Done Sent false
25792 7751 Acked Write 6 Done Done Sent false
25793 7752 Acked Write 1 Done Done Sent false
25794 7753 Acked Write 4 Done Done Sent false
25795 7754 Acked Read 6 Done Done Sent false
25796 7755 Acked Read 8 Done Done Sent false
25797 7756 Acked Read 5 Done Done Sent false
25798 7757 Acked Write 10 Done Done Sent false
25799 7758 Acked Write 4 Done Done Sent false
25800 7759 Acked Write 10 Done Done Sent false
25801 7760 Acked Flush 0 Done Done Sent false
25802 7761 Acked Write 8 Done Done Sent false
25803 7762 Acked Write 3 Done Done Sent false
25804 7763 Acked Read 8 Done Done Sent false
25805 7764 Acked Write 9 Done Done Sent false
25806 7765 Acked Read 7 Done Done Sent false
25807 7766 Acked Write 2 Done Done Sent false
25808 7767 Acked Read 6 Done Done Sent false
25809 7768 Acked Write 7 Done Done Sent false
25810 7769 Acked Read 6 Done Done Sent false
25811 7770 Acked Write 1 Done Done Sent false
25812 7771 Acked Write 5 Done Done Sent false
25813 7772 Acked Write 6 Done Done Sent false
25814 7773 Acked Read 4 Done Done Sent false
25815 7774 Acked Write 9 Done Done Sent false
25816 7775 Acked Flush 0 Done Done Sent false
25817 7776 Acked Write 3 Done Done Sent false
25818 7777 Acked Read 10 Done Done Sent false
25819 7778 Acked Flush 0 Done Done Sent false
25820 7779 Acked Read 3 Done Done Sent false
25821 7780 Acked Read 10 Done Done Sent false
25822 7781 Acked Flush 0 Done Done Sent false
25823 7782 Acked Read 4 Done Done Sent false
25824 7783 Acked Flush 0 Done Done Sent false
25825 7784 Acked Write 1 Done Done Sent false
25826 7785 Acked Read 5 Done Done Sent false
25827 7786 Acked Read 2 Done Done Sent false
25828 7787 Acked Write 3 Done Done Sent false
25829 7788 Acked Read 5 Done Done Sent false
25830 7789 Acked Write 3 Done Done Sent false
25831 7790 Acked Write 6 Done Done Sent false
25832 7791 Acked Write 3 Done Done Sent false
25833 7792 Acked Flush 0 Done Done Sent false
25834 7793 Acked Write 10 Done Done Sent false
25835 7794 Acked Read 1 Done Done Sent false
25836 7795 Acked Read 7 Done Done Sent false
25837 7796 Acked Read 7 Done Done Sent false
25838 7797 Acked Read 4 Done Done Sent false
25839 7798 Acked Write 7 Done Done Sent false
25840 7799 Acked Read 1 Done Done Sent false
25841 7800 Acked Read 3 Done Done Sent false
25842 7801 Acked Read 5 Done Done Sent false
25843 7802 Acked Write 9 Done Done Sent false
25844 7803 Acked Read 1 Done Done Sent false
25845 7804 Acked Write 10 Done Done Sent false
25846 7805 Acked Read 10 Done Done Sent false
25847 7806 Acked Read 9 Done Done Sent false
25848 7807 Acked Read 10 Done Done Sent false
25849 7808 Acked Read 6 Done Done Sent false
25850 7809 Acked Write 5 Done Done Sent false
25851 7810 Acked Read 1 Done Done Sent false
25852 7811 Acked Flush 0 Done Done Sent false
25853 7812 Acked Write 6 Done Done Sent false
25854 7813 Acked Write 6 Done Done Sent false
25855 7814 Acked Read 7 Done Done Sent false
25856 7815 Acked Flush 0 Done Done Sent false
25857 7816 Acked Write 9 Done Done Sent false
25858 7817 Acked Read 4 Done Done Sent false
25859 7818 Acked Write 6 Done Done Sent false
25860 7819 Acked Read 7 Done Done Sent false
25861 7820 Acked Read 6 Done Done Sent false
25862 7821 Acked Read 7 Done Done Sent false
25863 7822 Acked Read 7 Done Done Sent false
25864 7823 Acked Read 10 Done Done Sent false
25865 7824 Acked Write 3 Done Done Sent false
25866 7825 Acked Read 5 Done Done Sent false
25867 7826 Acked Read 9 Done Done Sent false
25868 7827 Acked Write 3 Done Done Sent false
25869 7828 Acked Read 2 Done Done Sent false
25870 7829 Acked Write 4 Done Done Sent false
25871 7830 Acked Read 4 Done Done Sent false
25872 7831 Acked Read 4 Done Done Sent false
25873 7832 Acked Read 1 Done Done Sent false
25874 7833 Acked Read 1 Done Done Sent false
25875 7834 Acked Write 9 Done Done Sent false
25876 7835 Acked Write 7 Done Done Sent false
25877 7836 Acked Write 1 Done Done Sent false
25878 7837 Acked Flush 0 Done Done Sent false
25879 7838 Acked Write 6 Done Done Sent false
25880 7839 Acked Read 4 Done Done Sent false
25881 7840 Acked Write 2 Done Done Sent false
25882 7841 Acked Write 3 Done Done Sent false
25883 7842 Acked Flush 0 Done Done Sent false
25884 7843 Acked Read 6 Done Done Sent false
25885 7844 Acked Read 8 Done Done Sent false
25886 7845 Acked Write 3 Done Done Sent false
25887 7846 Acked Read 6 Done Done Sent false
25888 7847 Acked Write 5 Done Done Sent false
25889 7848 Acked Read 7 Done Done Sent false
25890 7849 Acked Flush 0 Done Done Sent false
25891 7850 Acked Write 5 Done Done Sent false
25892 7851 Acked Read 3 Done Done Sent false
25893 7852 Acked Read 7 Done Done Sent false
25894 7853 Acked Read 3 Done Done Sent false
25895 7854 Acked Flush 0 Done Done Sent false
25896 7855 Acked Read 4 Done Done Sent false
25897 7856 Acked Write 10 Done Done Sent false
25898 7857 Acked Read 8 Done Done Sent false
25899 7858 Acked Read 3 Done Done Sent false
25900 7859 Acked Read 5 Done Done Sent false
25901 7860 Acked Write 4 Done Done Sent false
25902 7861 Acked Read 7 Done Done Sent false
25903 7862 Acked Read 2 Done Done Sent false
25904 7863 Acked Write 10 Done Done Sent false
25905 7864 Acked Write 6 Done Done Sent false
25906 7865 Acked Read 6 Done Done Sent false
25907 7866 Acked Read 5 Done Done Sent false
25908 7867 Acked Write 8 Done Done Sent false
25909 7868 Acked Read 5 Done Done Sent false
25910 7869 Acked Read 8 Done Done Sent false
25911 7870 Acked Write 8 Done Done Sent false
25912 7871 Acked Read 8 Done Done Sent false
25913 7872 Acked Read 9 Done Done Sent false
25914 7873 Acked Write 7 Done Done Sent false
25915 7874 Acked Read 5 Done Done Sent false
25916 7875 Acked Read 7 Done Done Sent false
25917 7876 Acked Write 2 Done Done Sent false
25918 7877 Acked Write 9 Done Done Sent false
25919 7878 Acked Write 3 Done Done Sent false
25920 7879 Acked Write 1 Done Done Sent false
25921 7880 Acked Write 5 Done Done Sent false
25922 7881 Acked Write 3 Done Done Sent false
25923 7882 Acked Read 9 Done Done Sent false
25924 7883 Acked Read 3 Done Done Sent false
25925 7884 Acked Flush 0 Done Done Sent false
25926 7885 Acked Read 2 Done Done Sent false
25927 7886 Acked Read 3 Done Done Sent false
25928 7887 Acked Read 2 Done Done Sent false
25929 7888 Acked Read 7 Done Done Sent false
25930 7889 Acked Read 7 Done Done Sent false
25931 7890 Acked Write 5 Done Done Sent false
25932 7891 Acked Flush 0 Done Done Sent false
25933 7892 Acked Flush 0 Done Done Sent false
25934 7893 Acked Read 4 Done Done Sent false
25935 7894 Acked Write 3 Done Done Sent false
25936 7895 Acked Read 8 Done Done Sent false
25937 7896 Acked Read 2 Done Done Sent false
25938 7897 Acked Read 1 Done Done Sent false
25939 7898 Acked Write 4 Done Done Sent false
25940 7899 Acked Write 2 Done Done Sent false
25941 7900 Acked Read 1 Done Done Sent false
25942 7901 Acked Read 7 Done Done Sent false
25943 7902 Acked Read 3 Done Done Sent false
25944 7903 Acked Write 7 Done Done Sent false
25945 7904 Acked Flush 0 Done Done Sent false
25946 7905 Acked Read 4 Done Done Sent false
25947 7906 Acked Write 7 Done Done Sent false
25948 7907 Acked Read 10 Done Done Sent false
25949 7908 Acked Read 6 Done Done Sent false
25950 7909 Acked Read 8 Done Done Sent false
25951 7910 Acked Read 8 Done Done Sent false
25952 7911 Acked Write 9 Done Done Sent false
25953 7912 Acked Read 3 Done Done Sent false
25954 7913 Acked Write 3 Done Done Sent false
25955 7914 Acked Read 7 Done Done Sent false
25956 7915 Acked Read 7 Done Done Sent false
25957 7916 Acked Read 3 Done Done Sent false
25958 7917 Acked Write 9 Done Done Sent false
25959 7918 Acked Read 9 Done Done Sent false
25960 7919 Acked Read 9 Done Done Sent false
25961 7920 Acked Read 4 Done Done Sent false
25962 7921 Acked Write 6 Done Done Sent false
25963 7922 Acked Flush 0 Done Done Sent false
25964 7923 Acked Flush 0 Done Done Sent false
25965 7924 Acked Read 9 Done Done Sent false
25966 7925 Acked Write 4 Done Done Sent false
25967 7926 Acked Write 3 Done Done Sent false
25968 7927 Acked Write 5 Done Done Sent false
25969 7928 Acked Read 2 Done Done Sent false
25970 7929 Acked Write 8 Done Done Sent false
25971 7930 Acked Write 10 Done Done Sent false
25972 7931 Acked Read 4 Done Done Sent false
25973 7932 Acked Write 2 Done Done Sent false
25974 7933 Acked Write 8 Done Done Sent false
25975 7934 Acked Read 1 Done Done Sent false
25976 7935 Acked Write 5 Done Done Sent false
25977 7936 Acked Write 6 Done Done Sent false
25978 7937 Acked Read 4 Done Done Sent false
25979 7938 Acked Write 10 Done Done Sent false
25980 7939 Acked Write 9 Done Done Sent false
25981 7940 Acked Write 6 Sent Sent Sent false
25982 STATES DS:0 DS:1 DS:2 TOTAL
25983 Sent 1 1 301 303
25984 Done 300 300 0 600
25985 Skipped 0 0 0 0
25986 Error 0 0 0 0
25987 Last Flush: 7923 7923 7639
25988 Downstairs last five completed: 7639 7638 7637 7636 7635
25989 Upstairs last five completed: 7940 7939 7938 7937 7936
25990 CLIENT: Up:0 ds:301 act:2
25991 [8830] got start action
25992 [8830] I am going to restart
25993 Starting downstairs at port 8830
25994 Make output file at "/tmp/dsc/downstairs-8830.txt"
25995 [8830][2] reports Starting
25996 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1223)
25997 [8830][2] reports Running
25998 ----------------------------------------------------------------
25999 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26000 States: Active Active Offline
26001 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26002 7640 Acked Read 7 Done Done Sent false
26003 7641 Acked Read 3 Done Done Sent false
26004 7642 Acked Read 3 Done Done Sent false
26005 7643 Acked Read 7 Done Done Sent false
26006 7644 Acked Read 3 Done Done Sent false
26007 7645 Acked Write 6 Done Done Sent false
26008 7646 Acked Read 3 Done Done Sent false
26009 7647 Acked Write 3 Done Done Sent false
26010 7648 Acked Write 6 Done Done Sent false
26011 7649 Acked Write 7 Done Done Sent false
26012 7650 Acked Read 5 Done Done Sent false
26013 7651 Acked Read 9 Done Done Sent false
26014 7652 Acked Write 8 Done Done Sent false
26015 7653 Acked Write 2 Done Done Sent false
26016 7654 Acked Read 10 Done Done Sent false
26017 7655 Acked Write 4 Done Done Sent false
26018 7656 Acked Read 4 Done Done Sent false
26019 7657 Acked Read 8 Done Done Sent false
26020 7658 Acked Write 6 Done Done Sent false
26021 7659 Acked Read 1 Done Done Sent false
26022 7660 Acked Read 7 Done Done Sent false
26023 7661 Acked Read 1 Done Done Sent false
26024 7662 Acked Write 4 Done Done Sent false
26025 7663 Acked Read 6 Done Done Sent false
26026 7664 Acked Write 1 Done Done Sent false
26027 7665 Acked Read 10 Done Done Sent false
26028 7666 Acked Flush 0 Done Done Sent false
26029 7667 Acked Write 2 Done Done Sent false
26030 7668 Acked Write 1 Done Done Sent false
26031 7669 Acked Read 4 Done Done Sent false
26032 7670 Acked Flush 0 Done Done Sent false
26033 7671 Acked Read 5 Done Done Sent false
26034 7672 Acked Read 10 Done Done Sent false
26035 7673 Acked Read 6 Done Done Sent false
26036 7674 Acked Write 5 Done Done Sent false
26037 7675 Acked Write 1 Done Done Sent false
26038 7676 Acked Read 9 Done Done Sent false
26039 7677 Acked Write 8 Done Done Sent false
26040 7678 Acked Read 1 Done Done Sent false
26041 7679 Acked Read 8 Done Done Sent false
26042 7680 Acked Read 2 Done Done Sent false
26043 7681 Acked Flush 0 Done Done Sent false
26044 7682 Acked Read 5 Done Done Sent false
26045 7683 Acked Write 2 Done Done Sent false
26046 7684 Acked Write 5 Done Done Sent false
26047 7685 Acked Write 7 Done Done Sent false
26048 7686 Acked Read 10 Done Done Sent false
26049 7687 Acked Write 3 Done Done Sent false
26050 7688 Acked Write 8 Done Done Sent false
26051 7689 Acked Read 8 Done Done Sent false
26052 7690 Acked Write 5 Done Done Sent false
26053 7691 Acked Read 9 Done Done Sent false
26054 7692 Acked Read 2 Done Done Sent false
26055 7693 Acked Read 6 Done Done Sent false
26056 7694 Acked Read 9 Done Done Sent false
26057 7695 Acked Read 7 Done Done Sent false
26058 7696 Acked Write 4 Done Done Sent false
26059 7697 Acked Read 8 Done Done Sent false
26060 7698 Acked Read 9 Done Done Sent false
26061 7699 Acked Write 4 Done Done Sent false
26062 7700 Acked Flush 0 Done Done Sent false
26063 7701 Acked Flush 0 Done Done Sent false
26064 7702 Acked Read 3 Done Done Sent false
26065 7703 Acked Read 1 Done Done Sent false
26066 7704 Acked Read 2 Done Done Sent false
26067 7705 Acked Read 5 Done Done Sent false
26068 7706 Acked Read 2 Done Done Sent false
26069 7707 Acked Write 2 Done Done Sent false
26070 7708 Acked Read 1 Done Done Sent false
26071 7709 Acked Write 4 Done Done Sent false
26072 7710 Acked Write 9 Done Done Sent false
26073 7711 Acked Read 5 Done Done Sent false
26074 7712 Acked Write 1 Done Done Sent false
26075 7713 Acked Read 2 Done Done Sent false
26076 7714 Acked Flush 0 Done Done Sent false
26077 7715 Acked Write 5 Done Done Sent false
26078 7716 Acked Write 4 Done Done Sent false
26079 7717 Acked Write 3 Done Done Sent false
26080 7718 Acked Read 2 Done Done Sent false
26081 7719 Acked Read 3 Done Done Sent false
26082 7720 Acked Write 8 Done Done Sent false
26083 7721 Acked Write 8 Done Done Sent false
26084 7722 Acked Write 1 Done Done Sent false
26085 7723 Acked Read 10 Done Done Sent false
26086 7724 Acked Read 8 Done Done Sent false
26087 7725 Acked Read 1 Done Done Sent false
26088 7726 Acked Read 1 Done Done Sent false
26089 7727 Acked Write 2 Done Done Sent false
26090 7728 Acked Read 2 Done Done Sent false
26091 7729 Acked Write 8 Done Done Sent false
26092 7730 Acked Flush 0 Done Done Sent false
26093 7731 Acked Write 7 Done Done Sent false
26094 7732 Acked Flush 0 Done Done Sent false
26095 7733 Acked Read 3 Done Done Sent false
26096 7734 Acked Write 5 Done Done Sent false
26097 7735 Acked Flush 0 Done Done Sent false
26098 7736 Acked Read 5 Done Done Sent false
26099 7737 Acked Write 6 Done Done Sent false
26100 7738 Acked Read 1 Done Done Sent false
26101 7739 Acked Write 8 Done Done Sent false
26102 7740 Acked Write 8 Done Done Sent false
26103 7741 Acked Write 5 Done Done Sent false
26104 7742 Acked Write 10 Done Done Sent false
26105 7743 Acked Read 7 Done Done Sent false
26106 7744 Acked Read 2 Done Done Sent false
26107 7745 Acked Write 7 Done Done Sent false
26108 7746 Acked Read 1 Done Done Sent false
26109 7747 Acked Write 10 Done Done Sent false
26110 7748 Acked Write 6 Done Done Sent false
26111 7749 Acked Read 10 Done Done Sent false
26112 7750 Acked Write 10 Done Done Sent false
26113 7751 Acked Write 6 Done Done Sent false
26114 7752 Acked Write 1 Done Done Sent false
26115 7753 Acked Write 4 Done Done Sent false
26116 7754 Acked Read 6 Done Done Sent false
26117 7755 Acked Read 8 Done Done Sent false
26118 7756 Acked Read 5 Done Done Sent false
26119 7757 Acked Write 10 Done Done Sent false
26120 7758 Acked Write 4 Done Done Sent false
26121 7759 Acked Write 10 Done Done Sent false
26122 7760 Acked Flush 0 Done Done Sent false
26123 7761 Acked Write 8 Done Done Sent false
26124 7762 Acked Write 3 Done Done Sent false
26125 7763 Acked Read 8 Done Done Sent false
26126 7764 Acked Write 9 Done Done Sent false
26127 7765 Acked Read 7 Done Done Sent false
26128 7766 Acked Write 2 Done Done Sent false
26129 7767 Acked Read 6 Done Done Sent false
26130 7768 Acked Write 7 Done Done Sent false
26131 7769 Acked Read 6 Done Done Sent false
26132 7770 Acked Write 1 Done Done Sent false
26133 7771 Acked Write 5 Done Done Sent false
26134 7772 Acked Write 6 Done Done Sent false
26135 7773 Acked Read 4 Done Done Sent false
26136 7774 Acked Write 9 Done Done Sent false
26137 7775 Acked Flush 0 Done Done Sent false
26138 7776 Acked Write 3 Done Done Sent false
26139 7777 Acked Read 10 Done Done Sent false
26140 7778 Acked Flush 0 Done Done Sent false
26141 7779 Acked Read 3 Done Done Sent false
26142 7780 Acked Read 10 Done Done Sent false
26143 7781 Acked Flush 0 Done Done Sent false
26144 7782 Acked Read 4 Done Done Sent false
26145 7783 Acked Flush 0 Done Done Sent false
26146 7784 Acked Write 1 Done Done Sent false
26147 7785 Acked Read 5 Done Done Sent false
26148 7786 Acked Read 2 Done Done Sent false
26149 7787 Acked Write 3 Done Done Sent false
26150 7788 Acked Read 5 Done Done Sent false
26151 7789 Acked Write 3 Done Done Sent false
26152 7790 Acked Write 6 Done Done Sent false
26153 7791 Acked Write 3 Done Done Sent false
26154 7792 Acked Flush 0 Done Done Sent false
26155 7793 Acked Write 10 Done Done Sent false
26156 7794 Acked Read 1 Done Done Sent false
26157 7795 Acked Read 7 Done Done Sent false
26158 7796 Acked Read 7 Done Done Sent false
26159 7797 Acked Read 4 Done Done Sent false
26160 7798 Acked Write 7 Done Done Sent false
26161 7799 Acked Read 1 Done Done Sent false
26162 7800 Acked Read 3 Done Done Sent false
26163 7801 Acked Read 5 Done Done Sent false
26164 7802 Acked Write 9 Done Done Sent false
26165 7803 Acked Read 1 Done Done Sent false
26166 7804 Acked Write 10 Done Done Sent false
26167 7805 Acked Read 10 Done Done Sent false
26168 7806 Acked Read 9 Done Done Sent false
26169 7807 Acked Read 10 Done Done Sent false
26170 7808 Acked Read 6 Done Done Sent false
26171 7809 Acked Write 5 Done Done Sent false
26172 7810 Acked Read 1 Done Done Sent false
26173 7811 Acked Flush 0 Done Done Sent false
26174 7812 Acked Write 6 Done Done Sent false
26175 7813 Acked Write 6 Done Done Sent false
26176 7814 Acked Read 7 Done Done Sent false
26177 7815 Acked Flush 0 Done Done Sent false
26178 7816 Acked Write 9 Done Done Sent false
26179 7817 Acked Read 4 Done Done Sent false
26180 7818 Acked Write 6 Done Done Sent false
26181 7819 Acked Read 7 Done Done Sent false
26182 7820 Acked Read 6 Done Done Sent false
26183 7821 Acked Read 7 Done Done Sent false
26184 7822 Acked Read 7 Done Done Sent false
26185 7823 Acked Read 10 Done Done Sent false
26186 7824 Acked Write 3 Done Done Sent false
26187 7825 Acked Read 5 Done Done Sent false
26188 7826 Acked Read 9 Done Done Sent false
26189 7827 Acked Write 3 Done Done Sent false
26190 7828 Acked Read 2 Done Done Sent false
26191 7829 Acked Write 4 Done Done Sent false
26192 7830 Acked Read 4 Done Done Sent false
26193 7831 Acked Read 4 Done Done Sent false
26194 7832 Acked Read 1 Done Done Sent false
26195 7833 Acked Read 1 Done Done Sent false
26196 7834 Acked Write 9 Done Done Sent false
26197 7835 Acked Write 7 Done Done Sent false
26198 7836 Acked Write 1 Done Done Sent false
26199 7837 Acked Flush 0 Done Done Sent false
26200 7838 Acked Write 6 Done Done Sent false
26201 7839 Acked Read 4 Done Done Sent false
26202 7840 Acked Write 2 Done Done Sent false
26203 7841 Acked Write 3 Done Done Sent false
26204 7842 Acked Flush 0 Done Done Sent false
26205 7843 Acked Read 6 Done Done Sent false
26206 7844 Acked Read 8 Done Done Sent false
26207 7845 Acked Write 3 Done Done Sent false
26208 7846 Acked Read 6 Done Done Sent false
26209 7847 Acked Write 5 Done Done Sent false
26210 7848 Acked Read 7 Done Done Sent false
26211 7849 Acked Flush 0 Done Done Sent false
26212 7850 Acked Write 5 Done Done Sent false
26213 7851 Acked Read 3 Done Done Sent false
26214 7852 Acked Read 7 Done Done Sent false
26215 7853 Acked Read 3 Done Done Sent false
26216 7854 Acked Flush 0 Done Done Sent false
26217 7855 Acked Read 4 Done Done Sent false
26218 7856 Acked Write 10 Done Done Sent false
26219 7857 Acked Read 8 Done Done Sent false
26220 7858 Acked Read 3 Done Done Sent false
26221 7859 Acked Read 5 Done Done Sent false
26222 7860 Acked Write 4 Done Done Sent false
26223 7861 Acked Read 7 Done Done Sent false
26224 7862 Acked Read 2 Done Done Sent false
26225 7863 Acked Write 10 Done Done Sent false
26226 7864 Acked Write 6 Done Done Sent false
26227 7865 Acked Read 6 Done Done Sent false
26228 7866 Acked Read 5 Done Done Sent false
26229 7867 Acked Write 8 Done Done Sent false
26230 7868 Acked Read 5 Done Done Sent false
26231 7869 Acked Read 8 Done Done Sent false
26232 7870 Acked Write 8 Done Done Sent false
26233 7871 Acked Read 8 Done Done Sent false
26234 7872 Acked Read 9 Done Done Sent false
26235 7873 Acked Write 7 Done Done Sent false
26236 7874 Acked Read 5 Done Done Sent false
26237 7875 Acked Read 7 Done Done Sent false
26238 7876 Acked Write 2 Done Done Sent false
26239 7877 Acked Write 9 Done Done Sent false
26240 7878 Acked Write 3 Done Done Sent false
26241 7879 Acked Write 1 Done Done Sent false
26242 7880 Acked Write 5 Done Done Sent false
26243 7881 Acked Write 3 Done Done Sent false
26244 7882 Acked Read 9 Done Done Sent false
26245 7883 Acked Read 3 Done Done Sent false
26246 7884 Acked Flush 0 Done Done Sent false
26247 7885 Acked Read 2 Done Done Sent false
26248 7886 Acked Read 3 Done Done Sent false
26249 7887 Acked Read 2 Done Done Sent false
26250 7888 Acked Read 7 Done Done Sent false
26251 7889 Acked Read 7 Done Done Sent false
26252 7890 Acked Write 5 Done Done Sent false
26253 7891 Acked Flush 0 Done Done Sent false
26254 7892 Acked Flush 0 Done Done Sent false
26255 7893 Acked Read 4 Done Done Sent false
26256 7894 Acked Write 3 Done Done Sent false
26257 7895 Acked Read 8 Done Done Sent false
26258 7896 Acked Read 2 Done Done Sent false
26259 7897 Acked Read 1 Done Done Sent false
26260 7898 Acked Write 4 Done Done Sent false
26261 7899 Acked Write 2 Done Done Sent false
26262 7900 Acked Read 1 Done Done Sent false
26263 7901 Acked Read 7 Done Done Sent false
26264 7902 Acked Read 3 Done Done Sent false
26265 7903 Acked Write 7 Done Done Sent false
26266 7904 Acked Flush 0 Done Done Sent false
26267 7905 Acked Read 4 Done Done Sent false
26268 7906 Acked Write 7 Done Done Sent false
26269 7907 Acked Read 10 Done Done Sent false
26270 7908 Acked Read 6 Done Done Sent false
26271 7909 Acked Read 8 Done Done Sent false
26272 7910 Acked Read 8 Done Done Sent false
26273 7911 Acked Write 9 Done Done Sent false
26274 7912 Acked Read 3 Done Done Sent false
26275 7913 Acked Write 3 Done Done Sent false
26276 7914 Acked Read 7 Done Done Sent false
26277 7915 Acked Read 7 Done Done Sent false
26278 7916 Acked Read 3 Done Done Sent false
26279 7917 Acked Write 9 Done Done Sent false
26280 7918 Acked Read 9 Done Done Sent false
26281 7919 Acked Read 9 Done Done Sent false
26282 7920 Acked Read 4 Done Done Sent false
26283 7921 Acked Write 6 Done Done Sent false
26284 7922 Acked Flush 0 Done Done Sent false
26285 7923 Acked Flush 0 Done Done Sent false
26286 7924 Acked Read 9 Done Done Sent false
26287 7925 Acked Write 4 Done Done Sent false
26288 7926 Acked Write 3 Done Done Sent false
26289 7927 Acked Write 5 Done Done Sent false
26290 7928 Acked Read 2 Done Done Sent false
26291 7929 Acked Write 8 Done Done Sent false
26292 7930 Acked Write 10 Done Done Sent false
26293 7931 Acked Read 4 Done Done Sent false
26294 7932 Acked Write 2 Done Done Sent false
26295 7933 Acked Write 8 Done Done Sent false
26296 7934 Acked Read 1 Done Done Sent false
26297 7935 Acked Write 5 Done Done Sent false
26298 7936 Acked Write 6 Done Done Sent false
26299 7937 Acked Read 4 Done Done Sent false
26300 7938 Acked Write 10 Done Done Sent false
26301 7939 Acked Write 9 Done Done Sent false
26302 7940 Acked Write 6 Done Done Sent false
26303 7941 Acked Flush 0 Done Done Sent false
26304 STATES DS:0 DS:1 DS:2 TOTAL
26305 Sent 0 0 302 302
26306 Done 302 302 0 604
26307 Skipped 0 0 0 0
26308 Error 0 0 0 0
26309 Last Flush: 7941 7941 7639
26310 Downstairs last five completed: 7639 7638 7637 7636 7635
26311 Upstairs last five completed: 7941 7940 7939 7938 7937
26312 CLIENT: Up:0 ds:302 act:2
263132025-01-27T19:41:52.371ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263142025-01-27T19:41:52.371ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58585) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263152025-01-27T19:41:52.372ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263162025-01-27T19:41:52.372ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263172025-01-27T19:41:52.372ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263182025-01-27T19:41:52.372ZINFOcrucible: send last flush ID to this DS: 7639 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263192025-01-27T19:41:52.373ZINFOcrucible: Replied this last flush ID: 7639 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263202025-01-27T19:41:52.373ZINFOcrucible: [2] client re-new 302 jobs since flush 7639 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263212025-01-27T19:41:52.373ZINFOcrucible: [2] Replayed 302 jobs since flush: 7639 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
26322 ----------------------------------------------------------------
26323 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
26324 Downstairs last five completed: 7941 7940 7939 7938 7937
26325 Upstairs last five completed: 7941 7940 7939 7938 7937
26326 CLIENT: Up:0 ds:0 act:3
26327 Replay: All jobs finished
26328 Main task has work to do, go find it
26329 got dsc Stop(0)
26330 stop 0
26331 Jan 27 19:41:54.612 [8810] Got stop action so:false kr:false
26332 INFO request completed, latency_us: 551, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 77e30fea-9383-40fc-a9c1-86a3ffbd5e36, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
263332025-01-27T19:41:54.613ZWARNcrucible: downstairs disconnected = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263342025-01-27T19:41:54.613ZWARNcrucible: client task is sending Done(Finished) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
263352025-01-27T19:41:54.613ZWARNcrucible: downstairs task for 0 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
26336 Jan 27 19:41:54.613 INFO request completed, latency_us: 148, response_code: 200, uri: /state/cid/0, method: GET, req_id: 1c945dd5-008e-45ef-a253-751e96d179a6{"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-27T19:41:54.613820438Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
26337 , remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
26338 [8810] Exited with: ExitStatus(unix_wait_status(9))
26339 [8810][0] reports Exit
26340 Jan 27 19:41:58.616 INFO request completed, latency_us: 206, response_code: 200, uri: /state/cid/0, method: GET, req_id: a8cde49d-cc86-446b-b859-f9879c2d016a, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
26341 001/300 Flush
26342 002/300 Read block 4199 len 32768
26343 003/300 Write block 1242 len 8192 data: 4 2
26344 004/300 Read block 3262 len 24576
26345 005/300 Read block 2485 len 32768
26346 006/300 Write block 2195 len 24576 data: 5 5 5 6 6 5
26347 007/300 Read block 2719 len 16384
26348 008/300 Write block 4447 len 12288 data: 5 5 5
26349 009/300 Read block 2424 len 8192
26350 010/300 Write block 4956 len 28672 data: 4 3 4 5 6 6 7
26351 011/300 Read block 4591 len 16384
26352 012/300 Read block 1596 len 24576
26353 013/300 Read block 3684 len 16384
26354 014/300 Write block 2926 len 4096 data: 4
26355 015/300 Read block 3951 len 20480
26356 016/300 Read block 3880 len 40960
26357 017/300 Read block 1679 len 36864
26358 018/300 Write block 4268 len 4096 data: 8
26359 019/300 Write block 4231 len 16384 data: 5 4 3 3
26360 020/300 Flush
26361 021/300 Write block 1003 len 8192 data: 5 6
26362 022/300 Flush
26363 023/300 Write block 1934 len 8192 data: 5 5
26364 024/300 Read block 749 len 36864
26365 025/300 Write block 2745 len 4096 data: 3
26366 026/300 Write block 4115 len 12288 data: 5 6 6
26367 027/300 Write block 3998 len 20480 data: 4 4 3 4 4
26368 028/300 Write block 2453 len 20480 data: 4 4 3 3 4
26369 029/300 Flush
26370 030/300 Write block 2648 len 4096 data: 3
26371 031/300 Read block 2816 len 28672
26372 032/300 Write block 3336 len 36864 data: 6 7 6 6 6 3 2 2 2
26373 033/300 Write block 2944 len 28672 data: 4 4 4 3 3 4 5
26374 034/300 Read block 1752 len 12288
26375 035/300 Write block 1183 len 12288 data: 3 4 5
26376 036/300 Write block 4719 len 36864 data: 5 3 3 3 3 3 3 3 2
26377 037/300 Read block 1133 len 40960
26378 038/300 Write block 1186 len 28672 data: 4 4 3 4 5 5 4
26379 039/300 Write block 1897 len 16384 data: 2 2 3 3
26380 040/300 Write block 2578 len 40960 data: 4 5 5 4 3 3 3 4 6 7
26381 041/300 Read block 3684 len 8192
26382 042/300 Read block 4556 len 32768
26383 043/300 Read block 416 len 36864
26384 044/300 Write block 4154 len 4096 data: 4
26385 045/300 Read block 4859 len 16384
26386 046/300 Write block 4643 len 24576 data: 5 5 4 5 5 4
26387 047/300 Read block 2674 len 8192
26388 048/300 Read block 761 len 40960
26389 049/300 Write block 945 len 40960 data: 3 5 5 4 4 4 3 4 4 4
26390 050/300 Write block 499 len 28672 data: 11 9 10 10 7 6 8
26391 051/300 Flush
26392 052/300 Read block 3010 len 12288
26393 053/300 Flush
26394 054/300 Read block 2711 len 20480
26395 055/300 Write block 1483 len 36864 data: 5 4 2 2 2 2 2 2 3
26396 056/300 Read block 2600 len 4096
26397 057/300 Read block 417 len 36864
26398 058/300 Read block 3496 len 16384
26399 059/300 Read block 3242 len 8192
26400 060/300 Read block 811 len 28672
26401 061/300 Read block 1985 len 28672
26402 062/300 Write block 2028 len 16384 data: 6 6 6 5
26403 063/300 Write block 4531 len 12288 data: 8 8 7
26404 064/300 Write block 2029 len 4096 data: 7
26405 065/300 Flush
26406 066/300 Read block 4198 len 20480
26407 067/300 Write block 1670 len 8192 data: 4 4
26408 068/300 Read block 2521 len 36864
26409 069/300 Read block 2867 len 4096
26410 070/300 Read block 4228 len 40960
26411 071/300 Read block 957 len 32768
26412 072/300 Read block 1214 len 4096
26413 073/300 Write block 3881 len 8192 data: 7 5
26414 074/300 Read block 1827 len 12288
26415 075/300 Flush
26416 076/300 Write block 3716 len 20480 data: 7 7 6 4 6
26417 077/300 Flush
26418 078/300 Read block 2129 len 28672
26419 079/300 Read block 2550 len 20480
26420 080/300 Flush
26421 081/300 Write block 836 len 24576 data: 4 3 3 3 2 2
26422 082/300 Read block 958 len 4096
26423 083/300 Write block 265 len 4096 data: 3
26424 084/300 Write block 2395 len 4096 data: 5
26425 085/300 Read block 2088 len 4096
26426 086/300 Read block 1327 len 12288
26427 087/300 Read block 182 len 28672
26428 088/300 Read block 103 len 12288
26429 089/300 Read block 3588 len 4096
26430 090/300 Read block 988 len 36864
26431 091/300 Flush
26432 092/300 Read block 308 len 36864
26433 093/300 Write block 1616 len 40960 data: 5 5 5 6 6 6 6 5 6 6
26434 094/300 Write block 1937 len 4096 data: 4
26435 095/300 Write block 2596 len 12288 data: 5 4 5
26436 096/300 Write block 3118 len 40960 data: 4 4 4 4 6 8 8 7 6 6
26437 097/300 Read block 1209 len 20480
26438 098/300 Read block 2515 len 28672
26439 099/300 Flush
26440 100/300 Write block 717 len 16384 data: 7 6 7 8
26441 101/300 Read block 4165 len 36864
26442 102/300 Read block 1458 len 4096
26443 103/300 Read block 1368 len 16384
26444 104/300 Flush
26445 105/300 Write block 2804 len 24576 data: 4 3 4 4 4 4
26446 106/300 Write block 1001 len 28672 data: 5 4 6 7 5 4 4
26447 107/300 Read block 1928 len 12288
26448 108/300 Write block 1689 len 12288 data: 3 4 4
26449 109/300 Write block 490 len 4096 data: 3
26450 110/300 Write block 1995 len 24576 data: 4 5 5 6 6 4
26451 111/300 Write block 2692 len 36864 data: 4 4 2 2 2 2 2 2 3
26452 112/300 Flush
26453 113/300 Read block 1928 len 24576
26454 114/300 Read block 1411 len 24576
26455 115/300 Read block 3134 len 20480
26456 116/300 Write block 1274 len 24576 data: 2 3 4 4 3 3
26457 117/300 Write block 3430 len 16384 data: 10 9 8 9
26458 118/300 Read block 250 len 24576
26459 119/300 Flush
26460 120/300 Read block 829 len 20480
26461 121/300 Write block 964 len 32768 data: 6 5 3 4 5 4 4 4
26462 122/300 Flush
26463 123/300 Read block 3138 len 40960
26464 124/300 Write block 1568 len 36864 data: 3 4 4 5 6 7 6 6 6
26465 125/300 Read block 3538 len 12288
26466 126/300 Read block 4125 len 8192
26467 127/300 Read block 3552 len 8192
26468 128/300 Write block 3081 len 12288 data: 5 4 3
26469 129/300 Write block 2750 len 36864 data: 5 7 6 6 5 5 5 3 4
26470 130/300 Read block 3217 len 20480
26471 131/300 Read block 1223 len 20480
26472 132/300 Write block 4340 len 20480 data: 4 4 2 2 3
26473 133/300 Write block 3836 len 28672 data: 5 5 6 6 7 8 9
26474 134/300 Read block 4504 len 16384
26475 135/300 Flush
26476 136/300 Write block 4833 len 28672 data: 5 4 7 7 7 8 7
26477 137/300 Read block 485 len 12288
26478 138/300 Read block 540 len 8192
26479 139/300 Read block 2463 len 8192
26480 140/300 Read block 16 len 4096
26481 141/300 Read block 3474 len 40960
26482 142/300 Write block 4901 len 4096 data: 4
26483 143/300 Read block 3668 len 36864
26484 144/300 Write block 4907 len 32768 data: 5 6 6 6 6 6 6 5
26485 145/300 Write block 1441 len 24576 data: 4 4 3 4 4 5
26486 146/300 Write block 4882 len 24576 data: 5 5 5 4 5 6
26487 147/300 Read block 4722 len 4096
26488 148/300 Read block 3631 len 16384
26489 149/300 Write block 1485 len 12288 data: 3 3 3
26490 150/300 Write block 1365 len 32768 data: 7 7 8 7 7 8 7 5
26491 151/300 Write block 1483 len 24576 data: 6 5 4 4 4 3
26492 152/300 Write block 4131 len 16384 data: 4 4 5 6
26493 153/300 Flush
26494 154/300 Read block 393 len 20480
26495 155/300 Read block 2955 len 16384
26496 156/300 Write block 1038 len 20480 data: 4 4 4 4 5
26497 157/300 Write block 1415 len 40960 data: 6 6 7 5 4 4 4 5 5 5
26498 158/300 Flush
26499 159/300 Write block 3026 len 4096 data: 5
26500 160/300 Read block 4977 len 28672
26501 161/300 Write block 3461 len 16384 data: 4 5 4 5
26502 162/300 Write block 691 len 12288 data: 5 4 4
26503 163/300 Write block 1815 len 4096 data: 10
26504 164/300 Read block 955 len 4096
26505 165/300 Read block 1028 len 24576
26506 166/300 Read block 838 len 32768
26507 167/300 Write block 2103 len 32768 data: 3 4 5 5 5 5 5 4
26508 168/300 Read block 1533 len 16384
26509 169/300 Read block 4679 len 24576
26510 170/300 Flush
26511 171/300 Read block 1267 len 8192
26512 172/300 Write block 3087 len 36864 data: 3 4 3 4 5 5 5 5 5
26513 173/300 Read block 1971 len 20480
26514 174/300 Write block 91 len 20480 data: 5 5 4 5 5
26515 175/300 Read block 1325 len 12288
26516 176/300 Read block 3667 len 28672
26517 177/300 Write block 4647 len 32768 data: 6 5 5 3 4 4 4 4
26518 178/300 Read block 1888 len 36864
26519 179/300 Write block 4186 len 40960 data: 3 3 4 6 9 9 9 8 8 8
26520 180/300 Write block 1593 len 16384 data: 7 7 7 9
26521 181/300 Read block 4540 len 32768
26522 182/300 Read block 837 len 20480
26523 183/300 Read block 4455 len 4096
26524 184/300 Write block 4751 len 36864 data: 4 4 5 5 6 5 2 2 3
26525 185/300 Read block 226 len 28672
26526 186/300 Read block 946 len 28672
26527 187/300 Read block 79 len 4096
26528 188/300 Read block 2730 len 36864
26529 189/300 Write block 1097 len 8192 data: 6 7
26530 190/300 Flush
26531 191/300 Write block 2471 len 40960 data: 3 4 5 5 6 6 5 3 4 4
26532 192/300 Write block 4560 len 4096 data: 5
26533 193/300 Write block 4957 len 32768 data: 4 5 6 7 7 8 5 5
26534 194/300 Read block 3540 len 36864
26535 195/300 Read block 2025 len 16384
26536 196/300 Read block 1330 len 24576
26537 197/300 Read block 806 len 32768
26538 198/300 Write block 2880 len 12288 data: 2 2 2
26539 199/300 Write block 49 len 28672 data: 8 8 8 7 5 7 5
26540 200/300 Write block 372 len 4096 data: 7
26541 201/300 Read block 2556 len 32768
26542 202/300 Read block 1596 len 24576
26543 203/300 Read block 2504 len 28672
26544 204/300 Write block 3978 len 28672 data: 3 3 3 3 3 3 3
26545 205/300 Flush
26546 206/300 Write block 4220 len 4096 data: 4
26547 207/300 Read block 3304 len 32768
26548 208/300 Read block 516 len 28672
26549 209/300 Read block 4969 len 24576
26550 210/300 Write block 2037 len 20480 data: 5 5 4 4 5
26551 211/300 Flush
26552 212/300 Read block 4409 len 8192
26553 213/300 Flush
26554 214/300 Read block 1601 len 28672
26555 215/300 Write block 4455 len 12288 data: 4 2 3
26556 216/300 Flush
26557 217/300 Read block 14 len 8192
26558 218/300 Read block 830 len 12288
26559 219/300 Read block 4905 len 4096
26560 220/300 Read block 3107 len 4096
26561 221/300 Write block 2321 len 28672 data: 3 3 3 3 2 2 2
26562 222/300 Read block 1363 len 40960
26563 223/300 Write block 1329 len 40960 data: 6 5 4 5 6 5 5 5 5 6
26564 224/300 Read block 1693 len 16384
26565 225/300 Read block 4388 len 8192
26566 226/300 Write block 288 len 24576 data: 7 6 4 4 4 5
26567 227/300 Read block 3246 len 40960
26568 228/300 Read block 83 len 40960
26569 229/300 Read block 4730 len 8192
26570 230/300 Flush
26571 231/300 Write block 3056 len 20480 data: 6 5 5 5 6
26572 232/300 Read block 1088 len 24576
26573 233/300 Read block 3423 len 24576
26574 234/300 Read block 1976 len 36864
26575 235/300 Write block 3845 len 32768 data: 6 6 5 5 5 4 4 3
26576 236/300 Read block 3111 len 36864
26577 237/300 Flush
26578 238/300 Write block 3000 len 28672 data: 6 6 8 8 8 7 7
26579 239/300 Read block 2506 len 20480
26580 240/300 Read block 762 len 8192
26581 241/300 Read block 4347 len 24576
26582 242/300 Read block 1131 len 16384
26583 243/300 Read block 4039 len 28672
26584 244/300 Write block 569 len 4096 data: 4
26585 245/300 Write block 279 len 40960 data: 9 11 12 10 8 7 7 8 7 8
26586 246/300 Write block 1154 len 24576 data: 2 2 2 2 3 3
26587 247/300 Read block 1621 len 24576
26588 248/300 Write block 1636 len 4096 data: 5
26589 249/300 Read block 924 len 20480
26590 250/300 Read block 1277 len 32768
26591 251/300 Write block 3692 len 28672 data: 7 6 5 4 5 5 3
26592 252/300 Read block 4691 len 36864
26593 253/300 Write block 721 len 4096 data: 7
26594 254/300 Write block 2700 len 40960 data: 4 4 5 5 4 3 3 3 3 5
26595 255/300 Flush
26596 256/300 Write block 2688 len 36864 data: 7 4 3 3 5 5 3 3 3
26597 257/300 Read block 3175 len 36864
26598 258/300 Read block 3729 len 32768
26599 259/300 Write block 2920 len 28672 data: 4 3 2 2 2 4 5
26600 260/300 Write block 567 len 40960 data: 3 4 5 4 4 3 4 4 5 5
26601 261/300 Write block 2430 len 32768 data: 5 4 3 3 2 2 4 7
26602 262/300 Read block 2915 len 20480
26603 263/300 Read block 1117 len 40960
26604 264/300 Read block 1513 len 8192
26605 265/300 Read block 2013 len 36864
26606 266/300 Write block 2816 len 8192 data: 4 2
26607 267/300 Read block 4544 len 32768
26608 268/300 Read block 3380 len 36864
26609 269/300 Read block 2444 len 16384
26610 270/300 Read block 1116 len 4096
26611 271/300 Write block 4018 len 24576 data: 4 4 4 4 3 4
26612 272/300 Write block 4958 len 4096 data: 6
26613 273/300 Write block 4404 len 28672 data: 4 5 5 6 6 5 5
26614 274/300 Read block 4294 len 28672
26615 275/300 Write block 3184 len 28672 data: 5 2 3 4 4 4 5
26616 276/300 Flush
26617 277/300 Write block 786 len 8192 data: 4 4
26618 278/300 Write block 726 len 36864 data: 4 3 3 3 3 2 2 2 2
26619 279/300 Read block 2161 len 8192
26620 280/300 Read block 448 len 20480
26621 281/300 Write block 2075 len 36864 data: 4 6 6 6 6 5 4 4 4
26622 282/300 Write block 1614 len 28672 data: 5 5 6 6 6 7 7
26623 283/300 Read block 3150 len 12288
26624 284/300 Read block 4408 len 32768
26625 285/300 Write block 3394 len 16384 data: 3 3 4 7
26626 286/300 Read block 246 len 4096
26627 287/300 Read block 559 len 24576
26628 288/300 Read block 1994 len 36864
26629 289/300 Read block 4104 len 40960
26630 290/300 Write block 2509 len 36864 data: 6 4 5 6 5 6 6 6 5
26631 291/300 Read block 2258 len 28672
26632 292/300 Write block 3360 len 8192 data: 6 7
26633 293/300 Read block 4107 len 28672
26634 294/300 Write block 4989 len 20480 data: 7 6 6 6 6
26635 295/300 Read block 2879 len 20480
26636 296/300 Flush
26637 297/300 Read block 1338 len 32768
26638 298/300 Flush
26639 299/300 Write block 4388 len 40960 data: 4 3 3 4 4 4 4 3 3 3
26640 300/300 Read block 4840 len 20480
26641 301/300 Read block 2713 len 36864
26642 Main task has work to do, go find it
26643 got dsc Start(0)
26644 start 0
26645 Jan 27 19:41:58.855 INFO request completed, latency_us: 258, response_code: 204, uri: /start/cid/0, method: POST, req_id: 080a6df0-0671-4c32-ab95-6d70d89e594b, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
26646 Replay: started 0, returned:Ok(())
26647 ----------------------------------------------------------------
26648 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
26649 States: Offline Active Active
26650 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26651 7942 Acked Flush 0 Sent Done Done false
26652 7943 Acked Read 8 Sent Done Done false
26653 7944 Acked Write 2 Sent Done Done false
26654 7945 Acked Read 6 Sent Done Done false
26655 7946 Acked Read 8 Sent Done Done false
26656 7947 Acked Write 6 Sent Done Done false
26657 7948 Acked Read 4 Sent Done Done false
26658 7949 Acked Write 3 Sent Done Done false
26659 7950 Acked Read 2 Sent Done Done false
26660 7951 Acked Write 7 Sent Done Done false
26661 7952 Acked Read 4 Sent Done Done false
26662 7953 Acked Read 6 Sent Done Done false
26663 7954 Acked Read 4 Sent Done Done false
26664 7955 Acked Write 1 Sent Done Done false
26665 7956 Acked Read 5 Sent Done Done false
26666 7957 Acked Read 10 Sent Done Done false
26667 7958 Acked Read 9 Sent Done Done false
26668 7959 Acked Write 1 Sent Done Done false
26669 7960 Acked Write 4 Sent Done Done false
26670 7961 Acked Flush 0 Sent Done Done false
26671 7962 Acked Write 2 Sent Done Done false
26672 7963 Acked Flush 0 Sent Done Done false
26673 7964 Acked Write 2 Sent Done Done false
26674 7965 Acked Read 9 Sent Done Done false
26675 7966 Acked Write 1 Sent Done Done false
26676 7967 Acked Write 3 Sent Done Done false
26677 7968 Acked Write 5 Sent Done Done false
26678 7969 Acked Write 5 Sent Done Done false
26679 7970 Acked Flush 0 Sent Done Done false
26680 7971 Acked Write 1 Sent Done Done false
26681 7972 Acked Read 7 Sent Done Done false
26682 7973 Acked Write 9 Sent Done Done false
26683 7974 Acked Write 7 Sent Done Done false
26684 7975 Acked Read 3 Sent Done Done false
26685 7976 Acked Write 3 Sent Done Done false
26686 7977 Acked Write 9 Sent Done Done false
26687 7978 Acked Read 10 Sent Done Done false
26688 7979 Acked Write 7 Sent Done Done false
26689 7980 Acked Write 4 Sent Done Done false
26690 7981 Acked Write 10 Sent Done Done false
26691 7982 Acked Read 2 Sent Done Done false
26692 7983 Acked Read 8 Sent Done Done false
26693 7984 Acked Read 9 Sent Done Done false
26694 7985 Acked Write 1 Sent Done Done false
26695 7986 Acked Read 4 Sent Done Done false
26696 7987 Acked Write 6 Sent Done Done false
26697 7988 Acked Read 2 Sent Done Done false
26698 7989 Acked Read 10 Sent Done Done false
26699 7990 Acked Write 10 Sent Done Done false
26700 7991 Acked Write 7 Sent Done Done false
26701 7992 Acked Flush 0 Sent Done Done false
26702 7993 Acked Read 3 Sent Done Done false
26703 7994 Acked Flush 0 Sent Done Done false
26704 7995 Acked Read 5 Sent Done Done false
26705 7996 Acked Write 9 Sent Done Done false
26706 7997 Acked Read 1 Sent Done Done false
26707 7998 Acked Read 9 Sent Done Done false
26708 7999 Acked Read 4 Sent Done Done false
26709 8000 Acked Read 2 Sent Done Done false
26710 8001 Acked Read 7 Sent Done Done false
26711 8002 Acked Read 7 Sent Done Done false
26712 8003 Acked Write 4 Sent Done Done false
26713 8004 Acked Write 3 Sent Done Done false
26714 8005 Acked Write 1 Sent Done Done false
26715 8006 Acked Flush 0 Sent Done Done false
26716 8007 Acked Read 5 Sent Done Done false
26717 8008 Acked Write 2 Sent Done Done false
26718 8009 Acked Read 9 Sent Done Done false
26719 8010 Acked Read 1 Sent Done Done false
26720 8011 Acked Read 10 Sent Done Done false
26721 8012 Acked Read 8 Sent Done Done false
26722 8013 Acked Read 1 Sent Done Done false
26723 8014 Acked Write 2 Sent Done Done false
26724 8015 Acked Read 3 Sent Done Done false
26725 8016 Acked Flush 0 Sent Done Done false
26726 8017 Acked Write 5 Sent Done Done false
26727 8018 Acked Flush 0 Sent Done Done false
26728 8019 Acked Read 7 Sent Done Done false
26729 8020 Acked Read 5 Sent Done Done false
26730 8021 Acked Flush 0 Sent Done Done false
26731 8022 Acked Write 6 Sent Done Done false
26732 8023 Acked Read 1 Sent Done Done false
26733 8024 Acked Write 1 Sent Done Done false
26734 8025 Acked Write 1 Sent Done Done false
26735 8026 Acked Read 1 Sent Done Done false
26736 8027 Acked Read 3 Sent Done Done false
26737 8028 Acked Read 7 Sent Done Done false
26738 8029 Acked Read 3 Sent Done Done false
26739 8030 Acked Read 1 Sent Done Done false
26740 8031 Acked Read 9 Sent Done Done false
26741 8032 Acked Flush 0 Sent Done Done false
26742 8033 Acked Read 9 Sent Done Done false
26743 8034 Acked Write 10 Sent Done Done false
26744 8035 Acked Write 1 Sent Done Done false
26745 8036 Acked Write 3 Sent Done Done false
26746 8037 Acked Write 10 Sent Done Done false
26747 8038 Acked Read 5 Sent Done Done false
26748 8039 Acked Read 7 Sent Done Done false
26749 8040 Acked Flush 0 Sent Done Done false
26750 8041 Acked Write 4 Sent Done Done false
26751 8042 Acked Read 9 Sent Done Done false
26752 8043 Acked Read 1 Sent Done Done false
26753 8044 Acked Read 4 Sent Done Done false
26754 8045 Acked Flush 0 Sent Done Done false
26755 8046 Acked Write 6 Sent Done Done false
26756 8047 Acked Write 7 Sent Done Done false
26757 8048 Acked Read 3 Sent Done Done false
26758 8049 Acked Write 3 Sent Done Done false
26759 8050 Acked Write 1 Sent Done Done false
26760 8051 Acked Write 6 Sent Done Done false
26761 8052 Acked Write 9 Sent Done Done false
26762 8053 Acked Flush 0 Sent Done Done false
26763 8054 Acked Read 6 Sent Done Done false
26764 8055 Acked Read 6 Sent Done Done false
26765 8056 Acked Read 5 Sent Done Done false
26766 8057 Acked Write 6 Sent Done Done false
26767 8058 Acked Write 4 Sent Done Done false
26768 8059 Acked Read 6 Sent Done Done false
26769 8060 Acked Flush 0 Sent Done Done false
26770 8061 Acked Read 5 Sent Done Done false
26771 8062 Acked Write 8 Sent Done Done false
26772 8063 Acked Flush 0 Sent Done Done false
26773 8064 Acked Read 10 Sent Done Done false
26774 8065 Acked Write 9 Sent Done Done false
26775 8066 Acked Read 3 Sent Done Done false
26776 8067 Acked Read 2 Sent Done Done false
26777 8068 Acked Read 2 Sent Done Done false
26778 8069 Acked Write 3 Sent Done Done false
26779 8070 Acked Write 9 Sent Done Done false
26780 8071 Acked Read 5 Sent Done Done false
26781 8072 Acked Read 5 Sent Done Done false
26782 8073 Acked Write 5 Sent Done Done false
26783 8074 Acked Write 7 Sent Done Done false
26784 8075 Acked Read 4 Sent Done Done false
26785 8076 Acked Flush 0 Sent Done Done false
26786 8077 Acked Write 7 Sent Done Done false
26787 8078 Acked Read 3 Sent Done Done false
26788 8079 Acked Read 2 Sent Done Done false
26789 8080 Acked Read 2 Sent Done Done false
26790 8081 Acked Read 1 Sent Done Done false
26791 8082 Acked Read 10 Sent Done Done false
26792 8083 Acked Write 1 Sent Done Done false
26793 8084 Acked Read 9 Sent Done Done false
26794 8085 Acked Write 8 Sent Done Done false
26795 8086 Acked Write 6 Sent Done Done false
26796 8087 Acked Write 6 Sent Done Done false
26797 8088 Acked Read 1 Sent Done Done false
26798 8089 Acked Read 4 Sent Done Done false
26799 8090 Acked Write 3 Sent Done Done false
26800 8091 Acked Write 8 Sent Done Done false
26801 8092 Acked Write 6 Sent Done Done false
26802 8093 Acked Write 4 Sent Done Done false
26803 8094 Acked Flush 0 Sent Done Done false
26804 8095 Acked Read 5 Sent Done Done false
26805 8096 Acked Read 4 Sent Done Done false
26806 8097 Acked Write 5 Sent Done Done false
26807 8098 Acked Write 10 Sent Done Done false
26808 8099 Acked Flush 0 Sent Done Done false
26809 8100 Acked Write 1 Sent Done Done false
26810 8101 Acked Read 7 Sent Done Done false
26811 8102 Acked Write 4 Sent Done Done false
26812 8103 Acked Write 3 Sent Done Done false
26813 8104 Acked Write 1 Sent Done Done false
26814 8105 Acked Read 1 Sent Done Done false
26815 8106 Acked Read 6 Sent Done Done false
26816 8107 Acked Read 8 Sent Done Done false
26817 8108 Acked Write 8 Sent Done Done false
26818 8109 Acked Read 4 Sent Done Done false
26819 8110 Acked Read 6 Sent Done Done false
26820 8111 Acked Flush 0 Sent Done Done false
26821 8112 Acked Read 2 Sent Done Done false
26822 8113 Acked Write 9 Sent Done Done false
26823 8114 Acked Read 5 Sent Done Done false
26824 8115 Acked Write 5 Sent Done Done false
26825 8116 Acked Read 3 Sent Done Done false
26826 8117 Acked Read 7 Sent Done Done false
26827 8118 Acked Write 8 Sent Done Done false
26828 8119 Acked Read 9 Sent Done Done false
26829 8120 Acked Write 10 Sent Done Done false
26830 8121 Acked Write 4 Sent Done Done false
26831 8122 Acked Read 8 Sent Done Done false
26832 8123 Acked Read 5 Sent Done Done false
26833 8124 Acked Read 1 Sent Done Done false
26834 8125 Acked Write 9 Sent Done Done false
26835 8126 Acked Read 7 Sent Done Done false
26836 8127 Acked Read 7 Sent Done Done false
26837 8128 Acked Read 1 Sent Done Done false
26838 8129 Acked Read 9 Sent Done Done false
26839 8130 Acked Write 2 Sent Done Done false
26840 8131 Acked Flush 0 Sent Done Done false
26841 8132 Acked Write 10 Sent Done Done false
26842 8133 Acked Write 1 Sent Done Done false
26843 8134 Acked Write 8 Sent Done Done false
26844 8135 Acked Read 9 Sent Done Done false
26845 8136 Acked Read 4 Sent Done Done false
26846 8137 Acked Read 6 Sent Done Done false
26847 8138 Acked Read 8 Sent Done Done false
26848 8139 Acked Write 3 Sent Done Done false
26849 8140 Acked Write 7 Sent Done Done false
26850 8141 Acked Write 1 Sent Done Done false
26851 8142 Acked Read 8 Sent Done Done false
26852 8143 Acked Read 6 Sent Done Done false
26853 8144 Acked Read 7 Sent Done Done false
26854 8145 Acked Write 7 Sent Done Done false
26855 8146 Acked Flush 0 Sent Done Done false
26856 8147 Acked Write 1 Sent Done Done false
26857 8148 Acked Read 8 Sent Done Done false
26858 8149 Acked Read 7 Sent Done Done false
26859 8150 Acked Read 6 Sent Done Done false
26860 8151 Acked Write 5 Sent Done Done false
26861 8152 Acked Flush 0 Sent Done Done false
26862 8153 Acked Read 2 Sent Done Done false
26863 8154 Acked Flush 0 Sent Done Done false
26864 8155 Acked Read 7 Sent Done Done false
26865 8156 Acked Write 3 Sent Done Done false
26866 8157 Acked Flush 0 Sent Done Done false
26867 8158 Acked Read 2 Sent Done Done false
26868 8159 Acked Read 3 Sent Done Done false
26869 8160 Acked Read 1 Sent Done Done false
26870 8161 Acked Read 1 Sent Done Done false
26871 8162 Acked Write 7 Sent Done Done false
26872 8163 Acked Read 10 Sent Done Done false
26873 8164 Acked Write 10 Sent Done Done false
26874 8165 Acked Read 4 Sent Done Done false
26875 8166 Acked Read 2 Sent Done Done false
26876 8167 Acked Write 6 Sent Done Done false
26877 8168 Acked Read 10 Sent Done Done false
26878 8169 Acked Read 10 Sent Done Done false
26879 8170 Acked Read 2 Sent Done Done false
26880 8171 Acked Flush 0 Sent Done Done false
26881 8172 Acked Write 5 Sent Done Done false
26882 8173 Acked Read 6 Sent Done Done false
26883 8174 Acked Read 6 Sent Done Done false
26884 8175 Acked Read 9 Sent Done Done false
26885 8176 Acked Write 8 Sent Done Done false
26886 8177 Acked Read 9 Sent Done Done false
26887 8178 Acked Flush 0 Sent Done Done false
26888 8179 Acked Write 7 Sent Done Done false
26889 8180 Acked Read 5 Sent Done Done false
26890 8181 Acked Read 2 Sent Done Done false
26891 8182 Acked Read 6 Sent Done Done false
26892 8183 Acked Read 4 Sent Done Done false
26893 8184 Acked Read 7 Sent Done Done false
26894 8185 Acked Write 1 Sent Done Done false
26895 8186 Acked Write 10 Sent Done Done false
26896 8187 Acked Write 6 Sent Done Done false
26897 8188 Acked Read 6 Sent Done Done false
26898 8189 Acked Write 1 Sent Done Done false
26899 8190 Acked Read 5 Sent Done Done false
26900 8191 Acked Read 8 Sent Done Done false
26901 8192 Acked Write 7 Sent Done Done false
26902 8193 Acked Read 9 Sent Done Done false
26903 8194 Acked Write 1 Sent Done Done false
26904 8195 Acked Write 10 Sent Done Done false
26905 8196 Acked Flush 0 Sent Done Done false
26906 8197 Acked Write 9 Sent Done Done false
26907 8198 Acked Read 9 Sent Done Done false
26908 8199 Acked Read 8 Sent Done Done false
26909 8200 Acked Write 7 Sent Done Done false
26910 8201 Acked Write 10 Sent Done Done false
26911 8202 Acked Write 8 Sent Done Done false
26912 8203 Acked Read 5 Sent Done Done false
26913 8204 Acked Read 10 Sent Done Done false
26914 8205 Acked Read 2 Sent Done Done false
26915 8206 Acked Read 9 Sent Done Done false
26916 8207 Acked Write 2 Sent Done Done false
26917 8208 Acked Read 8 Sent Done Done false
26918 8209 Acked Read 9 Sent Done Done false
26919 8210 Acked Read 4 Sent Done Done false
26920 8211 Acked Read 1 Sent Done Done false
26921 8212 Acked Write 6 Sent Done Done false
26922 8213 Acked Write 1 Sent Done Done false
26923 8214 Acked Write 7 Sent Done Done false
26924 8215 Acked Read 7 Sent Done Done false
26925 8216 Acked Write 7 Sent Done Done false
26926 8217 Acked Flush 0 Sent Done Done false
26927 8218 Acked Write 2 Sent Done Done false
26928 8219 Acked Write 9 Sent Done Done false
26929 8220 Acked Read 2 Sent Done Done false
26930 8221 Acked Read 5 Sent Done Done false
26931 8222 Acked Write 9 Sent Done Done false
26932 8223 Acked Write 7 Sent Done Done false
26933 8224 Acked Read 3 Sent Done Done false
26934 8225 Acked Read 8 Sent Done Done false
26935 8226 Acked Write 4 Sent Done Done false
26936 8227 Acked Read 1 Sent Done Done false
26937 8228 Acked Read 6 Sent Done Done false
26938 8229 Acked Read 9 Sent Done Done false
26939 8230 Acked Read 10 Sent Done Done false
26940 8231 Acked Write 9 Sent Done Done false
26941 8232 Acked Read 7 Sent Done Done false
26942 8233 Acked Write 2 Sent Done Done false
26943 8234 Acked Read 7 Sent Done Done false
26944 8235 Acked Write 5 Sent Done Done false
26945 8236 Acked Read 5 Sent Done Done false
26946 8237 Acked Flush 0 Sent Done Done false
26947 8238 Acked Read 8 Sent Done Done false
26948 8239 Acked Flush 0 Sent Done Done false
26949 8240 Acked Write 10 Sent Done Done false
26950 8241 Acked Read 5 Sent Done Done false
26951 8242 Acked Read 9 Sent Done Done false
26952 STATES DS:0 DS:1 DS:2 TOTAL
26953 Sent 301 0 0 301
26954 Done 0 301 301 602
26955 Skipped 0 0 0 0
26956 Error 0 0 0 0
26957 Last Flush: 7941 8239 8239
26958 Downstairs last five completed: 7941 7940 7939 7938 7937
26959 Upstairs last five completed: 8242 8241 8240 8239 8238
26960 CLIENT: Up:0 ds:301 act:2
26961 [8810] got start action
26962 [8810] I am going to restart
26963 Starting downstairs at port 8810
26964 Make output file at "/tmp/dsc/downstairs-8810.txt"
26965 [8810][0] reports Starting
26966 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1224)
26967 [8810][0] reports Running
26968 ----------------------------------------------------------------
26969 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
26970 States: Offline Active Active
26971 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
26972 7942 Acked Flush 0 Sent Done Done false
26973 7943 Acked Read 8 Sent Done Done false
26974 7944 Acked Write 2 Sent Done Done false
26975 7945 Acked Read 6 Sent Done Done false
26976 7946 Acked Read 8 Sent Done Done false
26977 7947 Acked Write 6 Sent Done Done false
26978 7948 Acked Read 4 Sent Done Done false
26979 7949 Acked Write 3 Sent Done Done false
26980 7950 Acked Read 2 Sent Done Done false
26981 7951 Acked Write 7 Sent Done Done false
26982 7952 Acked Read 4 Sent Done Done false
26983 7953 Acked Read 6 Sent Done Done false
26984 7954 Acked Read 4 Sent Done Done false
26985 7955 Acked Write 1 Sent Done Done false
26986 7956 Acked Read 5 Sent Done Done false
26987 7957 Acked Read 10 Sent Done Done false
26988 7958 Acked Read 9 Sent Done Done false
26989 7959 Acked Write 1 Sent Done Done false
26990 7960 Acked Write 4 Sent Done Done false
26991 7961 Acked Flush 0 Sent Done Done false
26992 7962 Acked Write 2 Sent Done Done false
26993 7963 Acked Flush 0 Sent Done Done false
26994 7964 Acked Write 2 Sent Done Done false
26995 7965 Acked Read 9 Sent Done Done false
26996 7966 Acked Write 1 Sent Done Done false
26997 7967 Acked Write 3 Sent Done Done false
26998 7968 Acked Write 5 Sent Done Done false
26999 7969 Acked Write 5 Sent Done Done false
27000 7970 Acked Flush 0 Sent Done Done false
27001 7971 Acked Write 1 Sent Done Done false
27002 7972 Acked Read 7 Sent Done Done false
27003 7973 Acked Write 9 Sent Done Done false
27004 7974 Acked Write 7 Sent Done Done false
27005 7975 Acked Read 3 Sent Done Done false
27006 7976 Acked Write 3 Sent Done Done false
27007 7977 Acked Write 9 Sent Done Done false
27008 7978 Acked Read 10 Sent Done Done false
27009 7979 Acked Write 7 Sent Done Done false
27010 7980 Acked Write 4 Sent Done Done false
27011 7981 Acked Write 10 Sent Done Done false
27012 7982 Acked Read 2 Sent Done Done false
27013 7983 Acked Read 8 Sent Done Done false
27014 7984 Acked Read 9 Sent Done Done false
27015 7985 Acked Write 1 Sent Done Done false
27016 7986 Acked Read 4 Sent Done Done false
27017 7987 Acked Write 6 Sent Done Done false
27018 7988 Acked Read 2 Sent Done Done false
27019 7989 Acked Read 10 Sent Done Done false
27020 7990 Acked Write 10 Sent Done Done false
27021 7991 Acked Write 7 Sent Done Done false
27022 7992 Acked Flush 0 Sent Done Done false
27023 7993 Acked Read 3 Sent Done Done false
27024 7994 Acked Flush 0 Sent Done Done false
27025 7995 Acked Read 5 Sent Done Done false
27026 7996 Acked Write 9 Sent Done Done false
27027 7997 Acked Read 1 Sent Done Done false
27028 7998 Acked Read 9 Sent Done Done false
27029 7999 Acked Read 4 Sent Done Done false
27030 8000 Acked Read 2 Sent Done Done false
27031 8001 Acked Read 7 Sent Done Done false
27032 8002 Acked Read 7 Sent Done Done false
27033 8003 Acked Write 4 Sent Done Done false
27034 8004 Acked Write 3 Sent Done Done false
27035 8005 Acked Write 1 Sent Done Done false
27036 8006 Acked Flush 0 Sent Done Done false
27037 8007 Acked Read 5 Sent Done Done false
27038 8008 Acked Write 2 Sent Done Done false
27039 8009 Acked Read 9 Sent Done Done false
27040 8010 Acked Read 1 Sent Done Done false
27041 8011 Acked Read 10 Sent Done Done false
27042 8012 Acked Read 8 Sent Done Done false
27043 8013 Acked Read 1 Sent Done Done false
27044 8014 Acked Write 2 Sent Done Done false
27045 8015 Acked Read 3 Sent Done Done false
27046 8016 Acked Flush 0 Sent Done Done false
27047 8017 Acked Write 5 Sent Done Done false
27048 8018 Acked Flush 0 Sent Done Done false
27049 8019 Acked Read 7 Sent Done Done false
27050 8020 Acked Read 5 Sent Done Done false
27051 8021 Acked Flush 0 Sent Done Done false
27052 8022 Acked Write 6 Sent Done Done false
27053 8023 Acked Read 1 Sent Done Done false
27054 8024 Acked Write 1 Sent Done Done false
27055 8025 Acked Write 1 Sent Done Done false
27056 8026 Acked Read 1 Sent Done Done false
27057 8027 Acked Read 3 Sent Done Done false
27058 8028 Acked Read 7 Sent Done Done false
27059 8029 Acked Read 3 Sent Done Done false
27060 8030 Acked Read 1 Sent Done Done false
27061 8031 Acked Read 9 Sent Done Done false
27062 8032 Acked Flush 0 Sent Done Done false
27063 8033 Acked Read 9 Sent Done Done false
27064 8034 Acked Write 10 Sent Done Done false
27065 8035 Acked Write 1 Sent Done Done false
27066 8036 Acked Write 3 Sent Done Done false
27067 8037 Acked Write 10 Sent Done Done false
27068 8038 Acked Read 5 Sent Done Done false
27069 8039 Acked Read 7 Sent Done Done false
27070 8040 Acked Flush 0 Sent Done Done false
27071 8041 Acked Write 4 Sent Done Done false
27072 8042 Acked Read 9 Sent Done Done false
27073 8043 Acked Read 1 Sent Done Done false
27074 8044 Acked Read 4 Sent Done Done false
27075 8045 Acked Flush 0 Sent Done Done false
27076 8046 Acked Write 6 Sent Done Done false
27077 8047 Acked Write 7 Sent Done Done false
27078 8048 Acked Read 3 Sent Done Done false
27079 8049 Acked Write 3 Sent Done Done false
27080 8050 Acked Write 1 Sent Done Done false
27081 8051 Acked Write 6 Sent Done Done false
27082 8052 Acked Write 9 Sent Done Done false
27083 8053 Acked Flush 0 Sent Done Done false
27084 8054 Acked Read 6 Sent Done Done false
27085 8055 Acked Read 6 Sent Done Done false
27086 8056 Acked Read 5 Sent Done Done false
27087 8057 Acked Write 6 Sent Done Done false
27088 8058 Acked Write 4 Sent Done Done false
27089 8059 Acked Read 6 Sent Done Done false
27090 8060 Acked Flush 0 Sent Done Done false
27091 8061 Acked Read 5 Sent Done Done false
27092 8062 Acked Write 8 Sent Done Done false
27093 8063 Acked Flush 0 Sent Done Done false
27094 8064 Acked Read 10 Sent Done Done false
27095 8065 Acked Write 9 Sent Done Done false
27096 8066 Acked Read 3 Sent Done Done false
27097 8067 Acked Read 2 Sent Done Done false
27098 8068 Acked Read 2 Sent Done Done false
27099 8069 Acked Write 3 Sent Done Done false
27100 8070 Acked Write 9 Sent Done Done false
27101 8071 Acked Read 5 Sent Done Done false
27102 8072 Acked Read 5 Sent Done Done false
27103 8073 Acked Write 5 Sent Done Done false
27104 8074 Acked Write 7 Sent Done Done false
27105 8075 Acked Read 4 Sent Done Done false
27106 8076 Acked Flush 0 Sent Done Done false
27107 8077 Acked Write 7 Sent Done Done false
27108 8078 Acked Read 3 Sent Done Done false
27109 8079 Acked Read 2 Sent Done Done false
27110 8080 Acked Read 2 Sent Done Done false
27111 8081 Acked Read 1 Sent Done Done false
27112 8082 Acked Read 10 Sent Done Done false
27113 8083 Acked Write 1 Sent Done Done false
27114 8084 Acked Read 9 Sent Done Done false
27115 8085 Acked Write 8 Sent Done Done false
27116 8086 Acked Write 6 Sent Done Done false
27117 8087 Acked Write 6 Sent Done Done false
27118 8088 Acked Read 1 Sent Done Done false
27119 8089 Acked Read 4 Sent Done Done false
27120 8090 Acked Write 3 Sent Done Done false
27121 8091 Acked Write 8 Sent Done Done false
27122 8092 Acked Write 6 Sent Done Done false
27123 8093 Acked Write 4 Sent Done Done false
27124 8094 Acked Flush 0 Sent Done Done false
27125 8095 Acked Read 5 Sent Done Done false
27126 8096 Acked Read 4 Sent Done Done false
27127 8097 Acked Write 5 Sent Done Done false
27128 8098 Acked Write 10 Sent Done Done false
27129 8099 Acked Flush 0 Sent Done Done false
27130 8100 Acked Write 1 Sent Done Done false
27131 8101 Acked Read 7 Sent Done Done false
27132 8102 Acked Write 4 Sent Done Done false
27133 8103 Acked Write 3 Sent Done Done false
27134 8104 Acked Write 1 Sent Done Done false
27135 8105 Acked Read 1 Sent Done Done false
27136 8106 Acked Read 6 Sent Done Done false
27137 8107 Acked Read 8 Sent Done Done false
27138 8108 Acked Write 8 Sent Done Done false
27139 8109 Acked Read 4 Sent Done Done false
27140 8110 Acked Read 6 Sent Done Done false
27141 8111 Acked Flush 0 Sent Done Done false
27142 8112 Acked Read 2 Sent Done Done false
27143 8113 Acked Write 9 Sent Done Done false
27144 8114 Acked Read 5 Sent Done Done false
27145 8115 Acked Write 5 Sent Done Done false
27146 8116 Acked Read 3 Sent Done Done false
27147 8117 Acked Read 7 Sent Done Done false
27148 8118 Acked Write 8 Sent Done Done false
27149 8119 Acked Read 9 Sent Done Done false
27150 8120 Acked Write 10 Sent Done Done false
27151 8121 Acked Write 4 Sent Done Done false
27152 8122 Acked Read 8 Sent Done Done false
27153 8123 Acked Read 5 Sent Done Done false
27154 8124 Acked Read 1 Sent Done Done false
27155 8125 Acked Write 9 Sent Done Done false
27156 8126 Acked Read 7 Sent Done Done false
27157 8127 Acked Read 7 Sent Done Done false
27158 8128 Acked Read 1 Sent Done Done false
27159 8129 Acked Read 9 Sent Done Done false
27160 8130 Acked Write 2 Sent Done Done false
27161 8131 Acked Flush 0 Sent Done Done false
27162 8132 Acked Write 10 Sent Done Done false
27163 8133 Acked Write 1 Sent Done Done false
27164 8134 Acked Write 8 Sent Done Done false
27165 8135 Acked Read 9 Sent Done Done false
27166 8136 Acked Read 4 Sent Done Done false
27167 8137 Acked Read 6 Sent Done Done false
27168 8138 Acked Read 8 Sent Done Done false
27169 8139 Acked Write 3 Sent Done Done false
27170 8140 Acked Write 7 Sent Done Done false
27171 8141 Acked Write 1 Sent Done Done false
27172 8142 Acked Read 8 Sent Done Done false
27173 8143 Acked Read 6 Sent Done Done false
27174 8144 Acked Read 7 Sent Done Done false
27175 8145 Acked Write 7 Sent Done Done false
27176 8146 Acked Flush 0 Sent Done Done false
27177 8147 Acked Write 1 Sent Done Done false
27178 8148 Acked Read 8 Sent Done Done false
27179 8149 Acked Read 7 Sent Done Done false
27180 8150 Acked Read 6 Sent Done Done false
27181 8151 Acked Write 5 Sent Done Done false
27182 8152 Acked Flush 0 Sent Done Done false
27183 8153 Acked Read 2 Sent Done Done false
27184 8154 Acked Flush 0 Sent Done Done false
27185 8155 Acked Read 7 Sent Done Done false
27186 8156 Acked Write 3 Sent Done Done false
27187 8157 Acked Flush 0 Sent Done Done false
27188 8158 Acked Read 2 Sent Done Done false
27189 8159 Acked Read 3 Sent Done Done false
27190 8160 Acked Read 1 Sent Done Done false
27191 8161 Acked Read 1 Sent Done Done false
27192 8162 Acked Write 7 Sent Done Done false
27193 8163 Acked Read 10 Sent Done Done false
27194 8164 Acked Write 10 Sent Done Done false
27195 8165 Acked Read 4 Sent Done Done false
27196 8166 Acked Read 2 Sent Done Done false
27197 8167 Acked Write 6 Sent Done Done false
27198 8168 Acked Read 10 Sent Done Done false
27199 8169 Acked Read 10 Sent Done Done false
27200 8170 Acked Read 2 Sent Done Done false
27201 8171 Acked Flush 0 Sent Done Done false
27202 8172 Acked Write 5 Sent Done Done false
27203 8173 Acked Read 6 Sent Done Done false
27204 8174 Acked Read 6 Sent Done Done false
27205 8175 Acked Read 9 Sent Done Done false
27206 8176 Acked Write 8 Sent Done Done false
27207 8177 Acked Read 9 Sent Done Done false
27208 8178 Acked Flush 0 Sent Done Done false
27209 8179 Acked Write 7 Sent Done Done false
27210 8180 Acked Read 5 Sent Done Done false
27211 8181 Acked Read 2 Sent Done Done false
27212 8182 Acked Read 6 Sent Done Done false
27213 8183 Acked Read 4 Sent Done Done false
27214 8184 Acked Read 7 Sent Done Done false
27215 8185 Acked Write 1 Sent Done Done false
27216 8186 Acked Write 10 Sent Done Done false
27217 8187 Acked Write 6 Sent Done Done false
27218 8188 Acked Read 6 Sent Done Done false
27219 8189 Acked Write 1 Sent Done Done false
27220 8190 Acked Read 5 Sent Done Done false
27221 8191 Acked Read 8 Sent Done Done false
27222 8192 Acked Write 7 Sent Done Done false
27223 8193 Acked Read 9 Sent Done Done false
27224 8194 Acked Write 1 Sent Done Done false
27225 8195 Acked Write 10 Sent Done Done false
27226 8196 Acked Flush 0 Sent Done Done false
27227 8197 Acked Write 9 Sent Done Done false
27228 8198 Acked Read 9 Sent Done Done false
27229 8199 Acked Read 8 Sent Done Done false
27230 8200 Acked Write 7 Sent Done Done false
27231 8201 Acked Write 10 Sent Done Done false
27232 8202 Acked Write 8 Sent Done Done false
27233 8203 Acked Read 5 Sent Done Done false
27234 8204 Acked Read 10 Sent Done Done false
27235 8205 Acked Read 2 Sent Done Done false
27236 8206 Acked Read 9 Sent Done Done false
27237 8207 Acked Write 2 Sent Done Done false
27238 8208 Acked Read 8 Sent Done Done false
27239 8209 Acked Read 9 Sent Done Done false
27240 8210 Acked Read 4 Sent Done Done false
27241 8211 Acked Read 1 Sent Done Done false
27242 8212 Acked Write 6 Sent Done Done false
27243 8213 Acked Write 1 Sent Done Done false
27244 8214 Acked Write 7 Sent Done Done false
27245 8215 Acked Read 7 Sent Done Done false
27246 8216 Acked Write 7 Sent Done Done false
27247 8217 Acked Flush 0 Sent Done Done false
27248 8218 Acked Write 2 Sent Done Done false
27249 8219 Acked Write 9 Sent Done Done false
27250 8220 Acked Read 2 Sent Done Done false
27251 8221 Acked Read 5 Sent Done Done false
27252 8222 Acked Write 9 Sent Done Done false
27253 8223 Acked Write 7 Sent Done Done false
27254 8224 Acked Read 3 Sent Done Done false
27255 8225 Acked Read 8 Sent Done Done false
27256 8226 Acked Write 4 Sent Done Done false
27257 8227 Acked Read 1 Sent Done Done false
27258 8228 Acked Read 6 Sent Done Done false
27259 8229 Acked Read 9 Sent Done Done false
27260 8230 Acked Read 10 Sent Done Done false
27261 8231 Acked Write 9 Sent Done Done false
27262 8232 Acked Read 7 Sent Done Done false
27263 8233 Acked Write 2 Sent Done Done false
27264 8234 Acked Read 7 Sent Done Done false
27265 8235 Acked Write 5 Sent Done Done false
27266 8236 Acked Read 5 Sent Done Done false
27267 8237 Acked Flush 0 Sent Done Done false
27268 8238 Acked Read 8 Sent Done Done false
27269 8239 Acked Flush 0 Sent Done Done false
27270 8240 Acked Write 10 Sent Done Done false
27271 8241 Acked Read 5 Sent Done Done false
27272 8242 Acked Read 9 Sent Done Done false
27273 8243 Acked Flush 0 Sent Done Done false
27274 STATES DS:0 DS:1 DS:2 TOTAL
27275 Sent 302 0 0 302
27276 Done 0 302 302 604
27277 Skipped 0 0 0 0
27278 Error 0 0 0 0
27279 Last Flush: 7941 8243 8243
27280 Downstairs last five completed: 7941 7940 7939 7938 7937
27281 Upstairs last five completed: 8243 8242 8241 8240 8239
27282 CLIENT: Up:0 ds:302 act:2
272832025-01-27T19:42:04.614ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
272842025-01-27T19:42:04.614ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:57799) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
272852025-01-27T19:42:04.615ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
272862025-01-27T19:42:04.616ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
272872025-01-27T19:42:04.616ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
272882025-01-27T19:42:04.616ZINFOcrucible: send last flush ID to this DS: 7941 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
272892025-01-27T19:42:04.616ZINFOcrucible: Replied this last flush ID: 7941 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
272902025-01-27T19:42:04.617ZINFOcrucible: [0] client re-new 302 jobs since flush 7941 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
272912025-01-27T19:42:04.617ZINFOcrucible: [0] Replayed 302 jobs since flush: 7941 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
27292 ----------------------------------------------------------------
27293 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
27294 Downstairs last five completed: 8243 8242 8241 8240 8239
27295 Upstairs last five completed: 8243 8242 8241 8240 8239
27296 CLIENT: Up:0 ds:0 act:3
27297 Replay: All jobs finished
27298 Main task has work to do, go find it
27299 got dsc Stop(1)
27300 stop 1
27301 Jan 27 19:42:06.860 INFO request completed, latency_us: 355, response_code: 204, uri:[8820] Got stop action so:false kr:false
27302 /stop/cid/1, method: POST, req_id: c0d1a9da-a30b-4c0c-b513-e30c4e201253, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
273032025-01-27T19:42:06.861ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
273042025-01-27T19:42:06.861ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
273052025-01-27T19:42:06.861ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
27306 Jan 27 19:42:06.861 INFO request completed, latency_us: 156, response_code: 200, uri: /state/cid/1, method: GET, req_id: 17f1db11-25e3-4515-84c1-d3bc0db27315, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
273072025-01-27T19:42:06.861ZINFOcrucible: client task is exiting = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
27308 [8820] Exited with: ExitStatus(unix_wait_status(9))
27309 [8820][1] reports Exit
27310 Jan 27 19:42:10.863 INFO request completed, latency_us: 246, response_code: 200, uri: /state/cid/1, method: GET, req_id: 3763f3c0-4108-4c4a-882d-bc2596a448d3, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
27311 001/300 Write block 1490 len 20480 data: 3 4 3 3 3
27312 002/300 Read block 4265 len 32768
27313 003/300 Write block 98 len 36864 data: 4 4 4 4 5 4 4 3 3
27314 004/300 Read block 1960 len 24576
27315 005/300 Write block 892 len 20480 data: 7 5 4 4 3
27316 006/300 Write block 1860 len 28672 data: 9 7 7 6 5 6 7
27317 007/300 Read block 68 len 16384
27318 008/300 Write block 378 len 32768 data: 4 4 4 3 3 3 3 3
27319 009/300 Write block 246 len 16384 data: 4 3 3 4
27320 010/300 Read block 2249 len 36864
27321 011/300 Read block 3160 len 28672
27322 012/300 Write block 2123 len 8192 data: 2 4
27323 013/300 Write block 3688 len 8192 data: 7 7
27324 014/300 Read block 1173 len 28672
27325 015/300 Read block 4543 len 4096
27326 016/300 Write block 831 len 24576 data: 7 8 7 7 7 5
27327 017/300 Read block 479 len 20480
27328 018/300 Read block 3917 len 24576
27329 019/300 Read block 1432 len 32768
27330 020/300 Write block 62 len 4096 data: 7
27331 021/300 Write block 473 len 28672 data: 4 4 5 5 4 6 6
27332 022/300 Read block 3636 len 16384
27333 023/300 Read block 3309 len 40960
27334 024/300 Read block 4885 len 40960
27335 025/300 Write block 3613 len 24576 data: 4 5 3 4 5 7
27336 026/300 Write block 3883 len 20480 data: 4 4 4 4 4
27337 027/300 Read block 325 len 8192
27338 028/300 Read block 2425 len 32768
27339 029/300 Write block 858 len 20480 data: 6 7 8 9 8
27340 030/300 Write block 545 len 20480 data: 6 7 6 6 6
27341 031/300 Write block 1493 len 20480 data: 4 4 3 3 3
27342 032/300 Write block 4928 len 24576 data: 9 7 6 6 4 5
27343 033/300 Read block 4033 len 12288
27344 034/300 Write block 657 len 4096 data: 5
27345 035/300 Read block 3902 len 16384
27346 036/300 Read block 540 len 16384
27347 037/300 Read block 4353 len 16384
27348 038/300 Write block 4289 len 16384 data: 3 3 3 4
27349 039/300 Write block 3462 len 12288 data: 6 5 6
27350 040/300 Read block 4449 len 16384
27351 041/300 Flush
27352 042/300 Write block 2992 len 32768 data: 4 3 3 4 4 4 4 4
27353 043/300 Read block 3349 len 12288
27354 044/300 Write block 3682 len 36864 data: 2 2 3 4 5 6 8 8 7
27355 045/300 Write block 2917 len 16384 data: 2 4 4 5
27356 046/300 Write block 2187 len 12288 data: 6 5 3
27357 047/300 Write block 2593 len 32768 data: 5 4 4 6 5 6 6 7
27358 048/300 Read block 3497 len 16384
27359 049/300 Read block 495 len 20480
27360 050/300 Write block 4499 len 24576 data: 7 7 5 5 7 6
27361 051/300 Write block 2573 len 36864 data: 2 4 3 4 4 5 6 6 5
27362 052/300 Write block 1058 len 40960 data: 5 6 6 5 4 4 4 4 5 4
27363 053/300 Flush
27364 054/300 Read block 4473 len 28672
27365 055/300 Write block 658 len 16384 data: 8 7 6 6
27366 056/300 Write block 4248 len 40960 data: 8 8 9 6 5 4 4 5 6 5
27367 057/300 Flush
27368 058/300 Write block 214 len 16384 data: 4 4 3 4
27369 059/300 Read block 4637 len 40960
27370 060/300 Read block 3706 len 28672
27371 061/300 Write block 2008 len 8192 data: 5 5
27372 062/300 Read block 4888 len 4096
27373 063/300 Write block 1351 len 40960 data: 4 4 7 6 8 7 8 11 11 8
27374 064/300 Read block 2289 len 12288
27375 065/300 Read block 719 len 36864
27376 066/300 Write block 4245 len 32768 data: 6 7 8 9 9 10 7 6
27377 067/300 Write block 154 len 12288 data: 4 4 4
27378 068/300 Read block 2264 len 32768
27379 069/300 Write block 2258 len 4096 data: 8
27380 070/300 Write block 4688 len 20480 data: 3 3 4 4 5
27381 071/300 Read block 3818 len 36864
27382 072/300 Write block 3141 len 36864 data: 8 9 10 7 7 5 4 5 5
27383 073/300 Write block 4755 len 20480 data: 7 6 3 3 4
27384 074/300 Read block 3232 len 8192
27385 075/300 Read block 1766 len 40960
27386 076/300 Write block 2250 len 28672 data: 6 5 6 6 7 10 11
27387 077/300 Write block 1177 len 12288 data: 3 3 2
27388 078/300 Flush
27389 079/300 Write block 349 len 32768 data: 4 4 7 8 8 8 7 8
27390 080/300 Write block 527 len 4096 data: 3
27391 081/300 Write block 3131 len 16384 data: 7 7 7 6
27392 082/300 Read block 1937 len 40960
27393 083/300 Write block 2144 len 4096 data: 8
27394 084/300 Read block 3435 len 4096
27395 085/300 Write block 4253 len 24576 data: 5 5 6 7 6 5
27396 086/300 Write block 735 len 24576 data: 3 3 3 6 6 6
27397 087/300 Read block 2080 len 12288
27398 088/300 Flush
27399 089/300 Write block 729 len 20480 data: 4 4 3 3 3
27400 090/300 Read block 706 len 24576
27401 091/300 Read block 2476 len 20480
27402 092/300 Read block 4231 len 8192
27403 093/300 Write block 3703 len 28672 data: 4 3 4 4 6 8 8
27404 094/300 Read block 4042 len 36864
27405 095/300 Read block 2764 len 8192
27406 096/300 Read block 3760 len 24576
27407 097/300 Read block 3350 len 16384
27408 098/300 Flush
27409 099/300 Read block 681 len 12288
27410 100/300 Read block 2166 len 36864
27411 101/300 Write block 1657 len 40960 data: 4 5 5 5 5 6 6 5 4 4
27412 102/300 Write block 3822 len 12288 data: 2 2 2
27413 103/300 Read block 3104 len 16384
27414 104/300 Read block 2232 len 4096
27415 105/300 Flush
27416 106/300 Read block 3036 len 24576
27417 107/300 Read block 3197 len 8192
27418 108/300 Read block 4306 len 36864
27419 109/300 Read block 1046 len 8192
27420 110/300 Flush
27421 111/300 Read block 2330 len 12288
27422 112/300 Flush
27423 113/300 Write block 1648 len 28672 data: 2 2 2 3 4 4 5
27424 114/300 Write block 2314 len 28672 data: 4 4 3 3 3 2 3
27425 115/300 Read block 1199 len 40960
27426 116/300 Write block 2712 len 24576 data: 6 7 6 7 6 5
27427 117/300 Flush
27428 118/300 Write block 3479 len 40960 data: 6 6 5 5 4 3 3 3 2 3
27429 119/300 Read block 607 len 24576
27430 120/300 Write block 3488 len 36864 data: 4 3 3 4 5 5 4 4 3
27431 121/300 Flush
27432 122/300 Write block 2585 len 24576 data: 5 7 8 6 5 6
27433 123/300 Read block 1981 len 36864
27434 124/300 Read block 2402 len 16384
27435 125/300 Write block 4153 len 28672 data: 4 5 4 4 3 5 6
27436 126/300 Read block 611 len 8192
27437 127/300 Read block 2641 len 8192
27438 128/300 Write block 3159 len 8192 data: 3 3
27439 129/300 Read block 913 len 20480
27440 130/300 Write block 1666 len 24576 data: 5 3 2 4 5 5
27441 131/300 Flush
27442 132/300 Read block 1391 len 16384
27443 133/300 Read block 1408 len 4096
27444 134/300 Read block 4223 len 20480
27445 135/300 Write block 1412 len 24576 data: 6 6 6 7 7 8
27446 136/300 Write block 3437 len 8192 data: 7 7
27447 137/300 Read block 1916 len 20480
27448 138/300 Write block 3599 len 16384 data: 5 6 4 4
27449 139/300 Write block 3872 len 40960 data: 4 6 6 5 6 6 7 6 9 8
27450 140/300 Read block 1053 len 24576
27451 141/300 Write block 4074 len 4096 data: 3
27452 142/300 Read block 2103 len 8192
27453 143/300 Read block 3562 len 8192
27454 144/300 Read block 4751 len 8192
27455 145/300 Flush
27456 146/300 Read block 2425 len 40960
27457 147/300 Write block 580 len 4096 data: 7
27458 148/300 Read block 856 len 16384
27459 149/300 Read block 4063 len 28672
27460 150/300 Write block 4372 len 24576 data: 3 3 3 4 4 4
27461 151/300 Read block 3779 len 28672
27462 152/300 Flush
27463 153/300 Write block 1851 len 12288 data: 7 7 7
27464 154/300 Write block 680 len 12288 data: 5 6 6
27465 155/300 Write block 2708 len 32768 data: 4 6 6 6 7 8 7 8
27466 156/300 Write block 1749 len 8192 data: 2 2
27467 157/300 Read block 4718 len 36864
27468 158/300 Read block 1802 len 28672
27469 159/300 Flush
27470 160/300 Read block 4232 len 8192
27471 161/300 Read block 4181 len 20480
27472 162/300 Write block 382 len 4096 data: 4
27473 163/300 Write block 2824 len 36864 data: 4 5 6 5 7 7 8 8 9
27474 164/300 Read block 568 len 12288
27475 165/300 Read block 970 len 12288
27476 166/300 Read block 2128 len 12288
27477 167/300 Read block 3077 len 24576
27478 168/300 Write block 2683 len 8192 data: 10 10
27479 169/300 Read block 784 len 20480
27480 170/300 Read block 2752 len 32768
27481 171/300 Read block 103 len 28672
27482 172/300 Read block 2853 len 32768
27483 173/300 Write block 3099 len 24576 data: 6 5 6 6 8 10
27484 174/300 Write block 4502 len 24576 data: 6 8 7 6 5 5
27485 175/300 Read block 3999 len 24576
27486 176/300 Read block 4956 len 40960
27487 177/300 Write block 1292 len 12288 data: 3 3 6
27488 178/300 Read block 1056 len 28672
27489 179/300 Flush
27490 180/300 Read block 3921 len 8192
27491 181/300 Write block 3781 len 32768 data: 5 4 4 4 3 4 4 4
27492 182/300 Write block 2369 len 4096 data: 6
27493 183/300 Flush
27494 184/300 Write block 4771 len 12288 data: 7 5 6
27495 185/300 Read block 4540 len 16384
27496 186/300 Read block 4659 len 28672
27497 187/300 Read block 1577 len 12288
27498 188/300 Read block 1065 len 32768
27499 189/300 Write block 4375 len 28672 data: 5 5 5 4 3 4 4
27500 190/300 Read block 3770 len 28672
27501 191/300 Read block 4333 len 24576
27502 192/300 Read block 5 len 28672
27503 193/300 Write block 1569 len 24576 data: 5 5 6 7 8 7
27504 194/300 Write block 1589 len 4096 data: 2
27505 195/300 Read block 884 len 28672
27506 196/300 Read block 4434 len 4096
27507 197/300 Read block 994 len 36864
27508 198/300 Write block 2071 len 36864 data: 3 5 5 5 5 7 7 7 7
27509 199/300 Read block 2669 len 12288
27510 200/300 Read block 789 len 28672
27511 201/300 Flush
27512 202/300 Read block 2805 len 16384
27513 203/300 Write block 3478 len 12288 data: 4 7 7
27514 204/300 Flush
27515 205/300 Read block 2572 len 20480
27516 206/300 Flush
27517 207/300 Write block 4306 len 20480 data: 3 3 3 3 3
27518 208/300 Read block 677 len 24576
27519 209/300 Write block 345 len 28672 data: 3 4 4 4 5 5 8
27520 210/300 Write block 2590 len 32768 data: 7 6 6 6 5 5 7 6
27521 211/300 Flush
27522 212/300 Write block 1464 len 20480 data: 6 6 6 6 5
27523 213/300 Read block 4189 len 36864
27524 214/300 Read block 1410 len 24576
27525 215/300 Read block 2601 len 32768
27526 216/300 Read block 1646 len 20480
27527 217/300 Write block 1243 len 36864 data: 3 3 3 3 3 3 4 3 5
27528 218/300 Write block 2250 len 12288 data: 7 6 7
27529 219/300 Write block 4026 len 28672 data: 3 4 3 3 3 3 3
27530 220/300 Write block 4481 len 28672 data: 8 7 6 4 5 4 4
27531 221/300 Flush
27532 222/300 Write block 4072 len 12288 data: 4 4 4
27533 223/300 Write block 1925 len 36864 data: 7 6 4 4 4 4 2 2 3
27534 224/300 Read block 4875 len 36864
27535 225/300 Write block 339 len 12288 data: 6 5 5
27536 226/300 Write block 3670 len 20480 data: 4 5 5 5 5
27537 227/300 Read block 3126 len 40960
27538 228/300 Write block 4409 len 12288 data: 6 6 7
27539 229/300 Read block 4042 len 4096
27540 230/300 Read block 3903 len 28672
27541 231/300 Write block 514 len 36864 data: 6 6 5 5 5 7 7 6 5
27542 232/300 Write block 3456 len 20480 data: 8 8 5 4 3
27543 233/300 Write block 1619 len 12288 data: 8 8 7
27544 234/300 Read block 338 len 28672
27545 235/300 Read block 1604 len 28672
27546 236/300 Flush
27547 237/300 Read block 3242 len 24576
27548 238/300 Read block 397 len 40960
27549 239/300 Read block 4007 len 4096
27550 240/300 Write block 1575 len 36864 data: 7 7 6 6 5 7 7 7 5
27551 241/300 Read block 2531 len 8192
27552 242/300 Flush
27553 243/300 Write block 2117 len 24576 data: 2 2 2 3 2 2
27554 244/300 Write block 3726 len 12288 data: 8 8 7
27555 245/300 Read block 346 len 40960
27556 246/300 Write block 1726 len 32768 data: 7 7 6 7 6 5 4 4
27557 247/300 Write block 3854 len 36864 data: 6 6 5 5 3 3 4 5 3
27558 248/300 Read block 2657 len 20480
27559 249/300 Read block 2032 len 16384
27560 250/300 Write block 4323 len 12288 data: 4 4 5
27561 251/300 Flush
27562 252/300 Flush
27563 253/300 Write block 204 len 12288 data: 4 7 7
27564 254/300 Write block 4806 len 40960 data: 7 8 8 6 8 7 5 3 3 3
27565 255/300 Write block 2358 len 20480 data: 3 3 3 3 2
27566 256/300 Read block 269 len 28672
27567 257/300 Write block 2172 len 20480 data: 6 7 6 7 8
27568 258/300 Read block 2444 len 4096
27569 259/300 Write block 4678 len 20480 data: 3 3 7 8 6
27570 260/300 Write block 4597 len 4096 data: 6
27571 261/300 Read block 683 len 32768
27572 262/300 Write block 3279 len 40960 data: 6 5 5 5 4 3 3 3 4 3
27573 263/300 Read block 1521 len 24576
27574 264/300 Read block 3355 len 24576
27575 265/300 Read block 4264 len 36864
27576 266/300 Read block 4196 len 36864
27577 267/300 Flush
27578 268/300 Write block 1722 len 12288 data: 5 5 4
27579 269/300 Write block 720 len 28672 data: 9 8 7 5 6 4 5
27580 270/300 Flush
27581 271/300 Read block 2720 len 12288
27582 272/300 Read block 3042 len 20480
27583 273/300 Write block 489 len 36864 data: 4 4 3 4 4 6 7 8 10
27584 274/300 Write block 2749 len 16384 data: 4 6 8 7
27585 275/300 Write block 2818 len 32768 data: 2 2 3 3 3 3 5 6
27586 276/300 Read block 746 len 36864
27587 277/300 Read block 267 len 40960
27588 278/300 Write block 319 len 36864 data: 4 6 7 5 5 5 6 5 6
27589 279/300 Read block 4440 len 16384
27590 280/300 Read block 392 len 20480
27591 281/300 Read block 1694 len 40960
27592 282/300 Read block 1556 len 12288
27593 283/300 Write block 3098 len 4096 data: 6
27594 284/300 Read block 2673 len 32768
27595 285/300 Read block 4105 len 12288
27596 286/300 Write block 4687 len 40960 data: 5 4 4 5 5 6 6 6 6 5
27597 287/300 Write block 2743 len 32768 data: 2 3 4 3 3 4 5 7
27598 288/300 Write block 3730 len 36864 data: 5 4 4 4 3 5 5 4 3
27599 289/300 Read block 1910 len 24576
27600 290/300 Read block 234 len 24576
27601 291/300 Flush
27602 292/300 Read block 3405 len 16384
27603 293/300 Write block 4220 len 8192 data: 5 4
27604 294/300 Write block 839 len 32768 data: 4 3 3 3 4 4 4 5
27605 295/300 Write block 12 len 4096 data: 3
27606 296/300 Write block 2445 len 12288 data: 6 6 3
27607 297/300 Write block 610 len 8192 data: 5 4
27608 298/300 Read block 2406 len 12288
27609 299/300 Write block 887 len 20480 data: 4 3 3 4 6
27610 300/300 Write block 2927 len 28672 data: 5 6 8 7 7 7 8
27611 301/300 Write block 235 len 4096 data: 6
27612 Main task has work to do, go find it
27613 got dsc Start(1)
27614 start 1
27615 Jan 27 19:42:11.097 INFO request completed, latency_us: 220, response_code: 204, uri: /start/cid/1, method: POST, req_id: 168749a6-92f1-4205-9539-169d38059a2c, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
27616 Replay: started 1, returned:Ok(())
27617 ----------------------------------------------------------------
27618 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
27619 States: Active Offline Active
27620 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27621 8244 Acked Write 5 Done Sent Done false
27622 8245 Acked Read 8 Done Sent Done false
27623 8246 Acked Write 9 Done Sent Done false
27624 8247 Acked Read 6 Done Sent Done false
27625 8248 Acked Write 5 Done Sent Done false
27626 8249 Acked Write 7 Done Sent Done false
27627 8250 Acked Read 4 Done Sent Done false
27628 8251 Acked Write 8 Done Sent Done false
27629 8252 Acked Write 4 Done Sent Done false
27630 8253 Acked Read 9 Done Sent Done false
27631 8254 Acked Read 7 Done Sent Done false
27632 8255 Acked Write 2 Done Sent Done false
27633 8256 Acked Write 2 Done Sent Done false
27634 8257 Acked Read 7 Done Sent Done false
27635 8258 Acked Read 1 Done Sent Done false
27636 8259 Acked Write 6 Done Sent Done false
27637 8260 Acked Read 5 Done Sent Done false
27638 8261 Acked Read 6 Done Sent Done false
27639 8262 Acked Read 8 Done Sent Done false
27640 8263 Acked Write 1 Done Sent Done false
27641 8264 Acked Write 7 Done Sent Done false
27642 8265 Acked Read 4 Done Sent Done false
27643 8266 Acked Read 10 Done Sent Done false
27644 8267 Acked Read 10 Done Sent Done false
27645 8268 Acked Write 6 Done Sent Done false
27646 8269 Acked Write 5 Done Sent Done false
27647 8270 Acked Read 2 Done Sent Done false
27648 8271 Acked Read 8 Done Sent Done false
27649 8272 Acked Write 5 Done Sent Done false
27650 8273 Acked Write 5 Done Sent Done false
27651 8274 Acked Write 5 Done Sent Done false
27652 8275 Acked Write 6 Done Sent Done false
27653 8276 Acked Read 3 Done Sent Done false
27654 8277 Acked Write 1 Done Sent Done false
27655 8278 Acked Read 4 Done Sent Done false
27656 8279 Acked Read 4 Done Sent Done false
27657 8280 Acked Read 4 Done Sent Done false
27658 8281 Acked Write 4 Done Sent Done false
27659 8282 Acked Write 3 Done Sent Done false
27660 8283 Acked Read 4 Done Sent Done false
27661 8284 Acked Flush 0 Done Sent Done false
27662 8285 Acked Write 8 Done Sent Done false
27663 8286 Acked Read 3 Done Sent Done false
27664 8287 Acked Write 9 Done Sent Done false
27665 8288 Acked Write 4 Done Sent Done false
27666 8289 Acked Write 3 Done Sent Done false
27667 8290 Acked Write 8 Done Sent Done false
27668 8291 Acked Read 4 Done Sent Done false
27669 8292 Acked Read 5 Done Sent Done false
27670 8293 Acked Write 6 Done Sent Done false
27671 8294 Acked Write 9 Done Sent Done false
27672 8295 Acked Write 10 Done Sent Done false
27673 8296 Acked Flush 0 Done Sent Done false
27674 8297 Acked Read 7 Done Sent Done false
27675 8298 Acked Write 4 Done Sent Done false
27676 8299 Acked Write 10 Done Sent Done false
27677 8300 Acked Flush 0 Done Sent Done false
27678 8301 Acked Write 4 Done Sent Done false
27679 8302 Acked Read 10 Done Sent Done false
27680 8303 Acked Read 7 Done Sent Done false
27681 8304 Acked Write 2 Done Sent Done false
27682 8305 Acked Read 1 Done Sent Done false
27683 8306 Acked Write 10 Done Sent Done false
27684 8307 Acked Read 3 Done Sent Done false
27685 8308 Acked Read 9 Done Sent Done false
27686 8309 Acked Write 8 Done Sent Done false
27687 8310 Acked Write 3 Done Sent Done false
27688 8311 Acked Read 8 Done Sent Done false
27689 8312 Acked Write 1 Done Sent Done false
27690 8313 Acked Write 5 Done Sent Done false
27691 8314 Acked Read 9 Done Sent Done false
27692 8315 Acked Write 9 Done Sent Done false
27693 8316 Acked Write 5 Done Sent Done false
27694 8317 Acked Read 2 Done Sent Done false
27695 8318 Acked Read 10 Done Sent Done false
27696 8319 Acked Write 7 Done Sent Done false
27697 8320 Acked Write 3 Done Sent Done false
27698 8321 Acked Flush 0 Done Sent Done false
27699 8322 Acked Write 8 Done Sent Done false
27700 8323 Acked Write 1 Done Sent Done false
27701 8324 Acked Write 4 Done Sent Done false
27702 8325 Acked Read 10 Done Sent Done false
27703 8326 Acked Write 1 Done Sent Done false
27704 8327 Acked Read 1 Done Sent Done false
27705 8328 Acked Write 6 Done Sent Done false
27706 8329 Acked Write 6 Done Sent Done false
27707 8330 Acked Read 3 Done Sent Done false
27708 8331 Acked Flush 0 Done Sent Done false
27709 8332 Acked Write 5 Done Sent Done false
27710 8333 Acked Read 6 Done Sent Done false
27711 8334 Acked Read 5 Done Sent Done false
27712 8335 Acked Read 2 Done Sent Done false
27713 8336 Acked Write 7 Done Sent Done false
27714 8337 Acked Read 9 Done Sent Done false
27715 8338 Acked Read 2 Done Sent Done false
27716 8339 Acked Read 6 Done Sent Done false
27717 8340 Acked Read 4 Done Sent Done false
27718 8341 Acked Flush 0 Done Sent Done false
27719 8342 Acked Read 3 Done Sent Done false
27720 8343 Acked Read 9 Done Sent Done false
27721 8344 Acked Write 10 Done Sent Done false
27722 8345 Acked Write 3 Done Sent Done false
27723 8346 Acked Read 4 Done Sent Done false
27724 8347 Acked Read 1 Done Sent Done false
27725 8348 Acked Flush 0 Done Sent Done false
27726 8349 Acked Read 6 Done Sent Done false
27727 8350 Acked Read 2 Done Sent Done false
27728 8351 Acked Read 9 Done Sent Done false
27729 8352 Acked Read 2 Done Sent Done false
27730 8353 Acked Flush 0 Done Sent Done false
27731 8354 Acked Read 3 Done Sent Done false
27732 8355 Acked Flush 0 Done Sent Done false
27733 8356 Acked Write 7 Done Sent Done false
27734 8357 Acked Write 7 Done Sent Done false
27735 8358 Acked Read 10 Done Sent Done false
27736 8359 Acked Write 6 Done Sent Done false
27737 8360 Acked Flush 0 Done Sent Done false
27738 8361 Acked Write 10 Done Sent Done false
27739 8362 Acked Read 6 Done Sent Done false
27740 8363 Acked Write 9 Done Sent Done false
27741 8364 Acked Flush 0 Done Sent Done false
27742 8365 Acked Write 6 Done Sent Done false
27743 8366 Acked Read 9 Done Sent Done false
27744 8367 Acked Read 4 Done Sent Done false
27745 8368 Acked Write 7 Done Sent Done false
27746 8369 Acked Read 2 Done Sent Done false
27747 8370 Acked Read 2 Done Sent Done false
27748 8371 Acked Write 2 Done Sent Done false
27749 8372 Acked Read 5 Done Sent Done false
27750 8373 Acked Write 6 Done Sent Done false
27751 8374 Acked Flush 0 Done Sent Done false
27752 8375 Acked Read 4 Done Sent Done false
27753 8376 Acked Read 1 Done Sent Done false
27754 8377 Acked Read 5 Done Sent Done false
27755 8378 Acked Write 6 Done Sent Done false
27756 8379 Acked Write 2 Done Sent Done false
27757 8380 Acked Read 5 Done Sent Done false
27758 8381 Acked Write 4 Done Sent Done false
27759 8382 Acked Write 10 Done Sent Done false
27760 8383 Acked Read 6 Done Sent Done false
27761 8384 Acked Write 1 Done Sent Done false
27762 8385 Acked Read 2 Done Sent Done false
27763 8386 Acked Read 2 Done Sent Done false
27764 8387 Acked Read 2 Done Sent Done false
27765 8388 Acked Flush 0 Done Sent Done false
27766 8389 Acked Read 10 Done Sent Done false
27767 8390 Acked Write 1 Done Sent Done false
27768 8391 Acked Read 4 Done Sent Done false
27769 8392 Acked Read 7 Done Sent Done false
27770 8393 Acked Write 6 Done Sent Done false
27771 8394 Acked Read 7 Done Sent Done false
27772 8395 Acked Flush 0 Done Sent Done false
27773 8396 Acked Write 3 Done Sent Done false
27774 8397 Acked Write 3 Done Sent Done false
27775 8398 Acked Write 8 Done Sent Done false
27776 8399 Acked Write 2 Done Sent Done false
27777 8400 Acked Read 9 Done Sent Done false
27778 8401 Acked Read 7 Done Sent Done false
27779 8402 Acked Flush 0 Done Sent Done false
27780 8403 Acked Read 2 Done Sent Done false
27781 8404 Acked Read 5 Done Sent Done false
27782 8405 Acked Write 1 Done Sent Done false
27783 8406 Acked Write 9 Done Sent Done false
27784 8407 Acked Read 3 Done Sent Done false
27785 8408 Acked Read 3 Done Sent Done false
27786 8409 Acked Read 3 Done Sent Done false
27787 8410 Acked Read 6 Done Sent Done false
27788 8411 Acked Write 2 Done Sent Done false
27789 8412 Acked Read 5 Done Sent Done false
27790 8413 Acked Read 8 Done Sent Done false
27791 8414 Acked Read 7 Done Sent Done false
27792 8415 Acked Read 8 Done Sent Done false
27793 8416 Acked Write 6 Done Sent Done false
27794 8417 Acked Write 6 Done Sent Done false
27795 8418 Acked Read 6 Done Sent Done false
27796 8419 Acked Read 10 Done Sent Done false
27797 8420 Acked Write 3 Done Sent Done false
27798 8421 Acked Read 7 Done Sent Done false
27799 8422 Acked Flush 0 Done Sent Done false
27800 8423 Acked Read 2 Done Sent Done false
27801 8424 Acked Write 8 Done Sent Done false
27802 8425 Acked Write 1 Done Sent Done false
27803 8426 Acked Flush 0 Done Sent Done false
27804 8427 Acked Write 3 Done Sent Done false
27805 8428 Acked Read 4 Done Sent Done false
27806 8429 Acked Read 7 Done Sent Done false
27807 8430 Acked Read 3 Done Sent Done false
27808 8431 Acked Read 8 Done Sent Done false
27809 8432 Acked Write 7 Done Sent Done false
27810 8433 Acked Read 7 Done Sent Done false
27811 8434 Acked Read 6 Done Sent Done false
27812 8435 Acked Read 7 Done Sent Done false
27813 8436 Acked Write 6 Done Sent Done false
27814 8437 Acked Write 1 Done Sent Done false
27815 8438 Acked Read 7 Done Sent Done false
27816 8439 Acked Read 1 Done Sent Done false
27817 8440 Acked Read 9 Done Sent Done false
27818 8441 Acked Write 9 Done Sent Done false
27819 8442 Acked Read 3 Done Sent Done false
27820 8443 Acked Read 7 Done Sent Done false
27821 8444 Acked Flush 0 Done Sent Done false
27822 8445 Acked Read 4 Done Sent Done false
27823 8446 Acked Write 3 Done Sent Done false
27824 8447 Acked Flush 0 Done Sent Done false
27825 8448 Acked Read 5 Done Sent Done false
27826 8449 Acked Flush 0 Done Sent Done false
27827 8450 Acked Write 5 Done Sent Done false
27828 8451 Acked Read 6 Done Sent Done false
27829 8452 Acked Write 7 Done Sent Done false
27830 8453 Acked Write 8 Done Sent Done false
27831 8454 Acked Flush 0 Done Sent Done false
27832 8455 Acked Write 5 Done Sent Done false
27833 8456 Acked Read 9 Done Sent Done false
27834 8457 Acked Read 6 Done Sent Done false
27835 8458 Acked Read 8 Done Sent Done false
27836 8459 Acked Read 5 Done Sent Done false
27837 8460 Acked Write 9 Done Sent Done false
27838 8461 Acked Write 3 Done Sent Done false
27839 8462 Acked Write 7 Done Sent Done false
27840 8463 Acked Write 7 Done Sent Done false
27841 8464 Acked Flush 0 Done Sent Done false
27842 8465 Acked Write 3 Done Sent Done false
27843 8466 Acked Write 9 Done Sent Done false
27844 8467 Acked Read 9 Done Sent Done false
27845 8468 Acked Write 3 Done Sent Done false
27846 8469 Acked Write 5 Done Sent Done false
27847 8470 Acked Read 10 Done Sent Done false
27848 8471 Acked Write 3 Done Sent Done false
27849 8472 Acked Read 1 Done Sent Done false
27850 8473 Acked Read 7 Done Sent Done false
27851 8474 Acked Write 9 Done Sent Done false
27852 8475 Acked Write 5 Done Sent Done false
27853 8476 Acked Write 3 Done Sent Done false
27854 8477 Acked Read 7 Done Sent Done false
27855 8478 Acked Read 7 Done Sent Done false
27856 8479 Acked Flush 0 Done Sent Done false
27857 8480 Acked Read 6 Done Sent Done false
27858 8481 Acked Read 10 Done Sent Done false
27859 8482 Acked Read 1 Done Sent Done false
27860 8483 Acked Write 9 Done Sent Done false
27861 8484 Acked Read 2 Done Sent Done false
27862 8485 Acked Flush 0 Done Sent Done false
27863 8486 Acked Write 6 Done Sent Done false
27864 8487 Acked Write 3 Done Sent Done false
27865 8488 Acked Read 10 Done Sent Done false
27866 8489 Acked Write 8 Done Sent Done false
27867 8490 Acked Write 9 Done Sent Done false
27868 8491 Acked Read 5 Done Sent Done false
27869 8492 Acked Read 4 Done Sent Done false
27870 8493 Acked Write 3 Done Sent Done false
27871 8494 Acked Flush 0 Done Sent Done false
27872 8495 Acked Flush 0 Done Sent Done false
27873 8496 Acked Write 3 Done Sent Done false
27874 8497 Acked Write 10 Done Sent Done false
27875 8498 Acked Write 5 Done Sent Done false
27876 8499 Acked Read 7 Done Sent Done false
27877 8500 Acked Write 5 Done Sent Done false
27878 8501 Acked Read 1 Done Sent Done false
27879 8502 Acked Write 5 Done Sent Done false
27880 8503 Acked Write 1 Done Sent Done false
27881 8504 Acked Read 8 Done Sent Done false
27882 8505 Acked Write 10 Done Sent Done false
27883 8506 Acked Read 6 Done Sent Done false
27884 8507 Acked Read 6 Done Sent Done false
27885 8508 Acked Read 9 Done Sent Done false
27886 8509 Acked Read 9 Done Sent Done false
27887 8510 Acked Flush 0 Done Sent Done false
27888 8511 Acked Write 3 Done Sent Done false
27889 8512 Acked Write 7 Done Sent Done false
27890 8513 Acked Flush 0 Done Sent Done false
27891 8514 Acked Read 3 Done Sent Done false
27892 8515 Acked Read 5 Done Sent Done false
27893 8516 Acked Write 9 Done Sent Done false
27894 8517 Acked Write 4 Done Sent Done false
27895 8518 Acked Write 8 Done Sent Done false
27896 8519 Acked Read 9 Done Sent Done false
27897 8520 Acked Read 10 Done Sent Done false
27898 8521 Acked Write 9 Done Sent Done false
27899 8522 Acked Read 4 Done Sent Done false
27900 8523 Acked Read 5 Done Sent Done false
27901 8524 Acked Read 10 Done Sent Done false
27902 8525 Acked Read 3 Done Sent Done false
27903 8526 Acked Write 1 Done Sent Done false
27904 8527 Acked Read 8 Done Sent Done false
27905 8528 Acked Read 3 Done Sent Done false
27906 8529 Acked Write 10 Done Sent Done false
27907 8530 Acked Write 8 Done Sent Done false
27908 8531 Acked Write 9 Done Sent Done false
27909 8532 Acked Read 6 Done Sent Done false
27910 8533 Acked Read 6 Done Sent Done false
27911 8534 Acked Flush 0 Done Sent Done false
27912 8535 Acked Read 4 Done Sent Done false
27913 8536 Acked Write 2 Done Sent Done false
27914 8537 Acked Write 8 Done Sent Done false
27915 8538 Acked Write 1 Done Sent Done false
27916 8539 Acked Write 3 Done Sent Done false
27917 8540 Acked Write 2 Done Sent Done false
27918 8541 Acked Read 3 Done Sent Done false
27919 8542 Acked Write 5 Done Sent Done false
27920 8543 Acked Write 7 Done Sent Done false
27921 8544 Acked Write 1 Done Sent Done false
27922 STATES DS:0 DS:1 DS:2 TOTAL
27923 Sent 0 301 0 301
27924 Done 301 0 301 602
27925 Skipped 0 0 0 0
27926 Error 0 0 0 0
27927 Last Flush: 8534 8243 8534
27928 Downstairs last five completed: 8243 8242 8241 8240 8239
27929 Upstairs last five completed: 8544 8543 8542 8541 8540
27930 CLIENT: Up:0 ds:301 act:2
27931 [8820] got start action
27932 [8820] I am going to restart
27933 Starting downstairs at port 8820
27934 Make output file at "/tmp/dsc/downstairs-8820.txt"
27935 [8820][1] reports Starting
27936 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1225)
27937 [8820][1] reports Running
27938 ----------------------------------------------------------------
27939 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
27940 States: Active Offline Active
27941 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
27942 8244 Acked Write 5 Done Sent Done false
27943 8245 Acked Read 8 Done Sent Done false
27944 8246 Acked Write 9 Done Sent Done false
27945 8247 Acked Read 6 Done Sent Done false
27946 8248 Acked Write 5 Done Sent Done false
27947 8249 Acked Write 7 Done Sent Done false
27948 8250 Acked Read 4 Done Sent Done false
27949 8251 Acked Write 8 Done Sent Done false
27950 8252 Acked Write 4 Done Sent Done false
27951 8253 Acked Read 9 Done Sent Done false
27952 8254 Acked Read 7 Done Sent Done false
27953 8255 Acked Write 2 Done Sent Done false
27954 8256 Acked Write 2 Done Sent Done false
27955 8257 Acked Read 7 Done Sent Done false
27956 8258 Acked Read 1 Done Sent Done false
27957 8259 Acked Write 6 Done Sent Done false
27958 8260 Acked Read 5 Done Sent Done false
27959 8261 Acked Read 6 Done Sent Done false
27960 8262 Acked Read 8 Done Sent Done false
27961 8263 Acked Write 1 Done Sent Done false
27962 8264 Acked Write 7 Done Sent Done false
27963 8265 Acked Read 4 Done Sent Done false
27964 8266 Acked Read 10 Done Sent Done false
27965 8267 Acked Read 10 Done Sent Done false
27966 8268 Acked Write 6 Done Sent Done false
27967 8269 Acked Write 5 Done Sent Done false
27968 8270 Acked Read 2 Done Sent Done false
27969 8271 Acked Read 8 Done Sent Done false
27970 8272 Acked Write 5 Done Sent Done false
27971 8273 Acked Write 5 Done Sent Done false
27972 8274 Acked Write 5 Done Sent Done false
27973 8275 Acked Write 6 Done Sent Done false
27974 8276 Acked Read 3 Done Sent Done false
27975 8277 Acked Write 1 Done Sent Done false
27976 8278 Acked Read 4 Done Sent Done false
27977 8279 Acked Read 4 Done Sent Done false
27978 8280 Acked Read 4 Done Sent Done false
27979 8281 Acked Write 4 Done Sent Done false
27980 8282 Acked Write 3 Done Sent Done false
27981 8283 Acked Read 4 Done Sent Done false
27982 8284 Acked Flush 0 Done Sent Done false
27983 8285 Acked Write 8 Done Sent Done false
27984 8286 Acked Read 3 Done Sent Done false
27985 8287 Acked Write 9 Done Sent Done false
27986 8288 Acked Write 4 Done Sent Done false
27987 8289 Acked Write 3 Done Sent Done false
27988 8290 Acked Write 8 Done Sent Done false
27989 8291 Acked Read 4 Done Sent Done false
27990 8292 Acked Read 5 Done Sent Done false
27991 8293 Acked Write 6 Done Sent Done false
27992 8294 Acked Write 9 Done Sent Done false
27993 8295 Acked Write 10 Done Sent Done false
27994 8296 Acked Flush 0 Done Sent Done false
27995 8297 Acked Read 7 Done Sent Done false
27996 8298 Acked Write 4 Done Sent Done false
27997 8299 Acked Write 10 Done Sent Done false
27998 8300 Acked Flush 0 Done Sent Done false
27999 8301 Acked Write 4 Done Sent Done false
28000 8302 Acked Read 10 Done Sent Done false
28001 8303 Acked Read 7 Done Sent Done false
28002 8304 Acked Write 2 Done Sent Done false
28003 8305 Acked Read 1 Done Sent Done false
28004 8306 Acked Write 10 Done Sent Done false
28005 8307 Acked Read 3 Done Sent Done false
28006 8308 Acked Read 9 Done Sent Done false
28007 8309 Acked Write 8 Done Sent Done false
28008 8310 Acked Write 3 Done Sent Done false
28009 8311 Acked Read 8 Done Sent Done false
28010 8312 Acked Write 1 Done Sent Done false
28011 8313 Acked Write 5 Done Sent Done false
28012 8314 Acked Read 9 Done Sent Done false
28013 8315 Acked Write 9 Done Sent Done false
28014 8316 Acked Write 5 Done Sent Done false
28015 8317 Acked Read 2 Done Sent Done false
28016 8318 Acked Read 10 Done Sent Done false
28017 8319 Acked Write 7 Done Sent Done false
28018 8320 Acked Write 3 Done Sent Done false
28019 8321 Acked Flush 0 Done Sent Done false
28020 8322 Acked Write 8 Done Sent Done false
28021 8323 Acked Write 1 Done Sent Done false
28022 8324 Acked Write 4 Done Sent Done false
28023 8325 Acked Read 10 Done Sent Done false
28024 8326 Acked Write 1 Done Sent Done false
28025 8327 Acked Read 1 Done Sent Done false
28026 8328 Acked Write 6 Done Sent Done false
28027 8329 Acked Write 6 Done Sent Done false
28028 8330 Acked Read 3 Done Sent Done false
28029 8331 Acked Flush 0 Done Sent Done false
28030 8332 Acked Write 5 Done Sent Done false
28031 8333 Acked Read 6 Done Sent Done false
28032 8334 Acked Read 5 Done Sent Done false
28033 8335 Acked Read 2 Done Sent Done false
28034 8336 Acked Write 7 Done Sent Done false
28035 8337 Acked Read 9 Done Sent Done false
28036 8338 Acked Read 2 Done Sent Done false
28037 8339 Acked Read 6 Done Sent Done false
28038 8340 Acked Read 4 Done Sent Done false
28039 8341 Acked Flush 0 Done Sent Done false
28040 8342 Acked Read 3 Done Sent Done false
28041 8343 Acked Read 9 Done Sent Done false
28042 8344 Acked Write 10 Done Sent Done false
28043 8345 Acked Write 3 Done Sent Done false
28044 8346 Acked Read 4 Done Sent Done false
28045 8347 Acked Read 1 Done Sent Done false
28046 8348 Acked Flush 0 Done Sent Done false
28047 8349 Acked Read 6 Done Sent Done false
28048 8350 Acked Read 2 Done Sent Done false
28049 8351 Acked Read 9 Done Sent Done false
28050 8352 Acked Read 2 Done Sent Done false
28051 8353 Acked Flush 0 Done Sent Done false
28052 8354 Acked Read 3 Done Sent Done false
28053 8355 Acked Flush 0 Done Sent Done false
28054 8356 Acked Write 7 Done Sent Done false
28055 8357 Acked Write 7 Done Sent Done false
28056 8358 Acked Read 10 Done Sent Done false
28057 8359 Acked Write 6 Done Sent Done false
28058 8360 Acked Flush 0 Done Sent Done false
28059 8361 Acked Write 10 Done Sent Done false
28060 8362 Acked Read 6 Done Sent Done false
28061 8363 Acked Write 9 Done Sent Done false
28062 8364 Acked Flush 0 Done Sent Done false
28063 8365 Acked Write 6 Done Sent Done false
28064 8366 Acked Read 9 Done Sent Done false
28065 8367 Acked Read 4 Done Sent Done false
28066 8368 Acked Write 7 Done Sent Done false
28067 8369 Acked Read 2 Done Sent Done false
28068 8370 Acked Read 2 Done Sent Done false
28069 8371 Acked Write 2 Done Sent Done false
28070 8372 Acked Read 5 Done Sent Done false
28071 8373 Acked Write 6 Done Sent Done false
28072 8374 Acked Flush 0 Done Sent Done false
28073 8375 Acked Read 4 Done Sent Done false
28074 8376 Acked Read 1 Done Sent Done false
28075 8377 Acked Read 5 Done Sent Done false
28076 8378 Acked Write 6 Done Sent Done false
28077 8379 Acked Write 2 Done Sent Done false
28078 8380 Acked Read 5 Done Sent Done false
28079 8381 Acked Write 4 Done Sent Done false
28080 8382 Acked Write 10 Done Sent Done false
28081 8383 Acked Read 6 Done Sent Done false
28082 8384 Acked Write 1 Done Sent Done false
28083 8385 Acked Read 2 Done Sent Done false
28084 8386 Acked Read 2 Done Sent Done false
28085 8387 Acked Read 2 Done Sent Done false
28086 8388 Acked Flush 0 Done Sent Done false
28087 8389 Acked Read 10 Done Sent Done false
28088 8390 Acked Write 1 Done Sent Done false
28089 8391 Acked Read 4 Done Sent Done false
28090 8392 Acked Read 7 Done Sent Done false
28091 8393 Acked Write 6 Done Sent Done false
28092 8394 Acked Read 7 Done Sent Done false
28093 8395 Acked Flush 0 Done Sent Done false
28094 8396 Acked Write 3 Done Sent Done false
28095 8397 Acked Write 3 Done Sent Done false
28096 8398 Acked Write 8 Done Sent Done false
28097 8399 Acked Write 2 Done Sent Done false
28098 8400 Acked Read 9 Done Sent Done false
28099 8401 Acked Read 7 Done Sent Done false
28100 8402 Acked Flush 0 Done Sent Done false
28101 8403 Acked Read 2 Done Sent Done false
28102 8404 Acked Read 5 Done Sent Done false
28103 8405 Acked Write 1 Done Sent Done false
28104 8406 Acked Write 9 Done Sent Done false
28105 8407 Acked Read 3 Done Sent Done false
28106 8408 Acked Read 3 Done Sent Done false
28107 8409 Acked Read 3 Done Sent Done false
28108 8410 Acked Read 6 Done Sent Done false
28109 8411 Acked Write 2 Done Sent Done false
28110 8412 Acked Read 5 Done Sent Done false
28111 8413 Acked Read 8 Done Sent Done false
28112 8414 Acked Read 7 Done Sent Done false
28113 8415 Acked Read 8 Done Sent Done false
28114 8416 Acked Write 6 Done Sent Done false
28115 8417 Acked Write 6 Done Sent Done false
28116 8418 Acked Read 6 Done Sent Done false
28117 8419 Acked Read 10 Done Sent Done false
28118 8420 Acked Write 3 Done Sent Done false
28119 8421 Acked Read 7 Done Sent Done false
28120 8422 Acked Flush 0 Done Sent Done false
28121 8423 Acked Read 2 Done Sent Done false
28122 8424 Acked Write 8 Done Sent Done false
28123 8425 Acked Write 1 Done Sent Done false
28124 8426 Acked Flush 0 Done Sent Done false
28125 8427 Acked Write 3 Done Sent Done false
28126 8428 Acked Read 4 Done Sent Done false
28127 8429 Acked Read 7 Done Sent Done false
28128 8430 Acked Read 3 Done Sent Done false
28129 8431 Acked Read 8 Done Sent Done false
28130 8432 Acked Write 7 Done Sent Done false
28131 8433 Acked Read 7 Done Sent Done false
28132 8434 Acked Read 6 Done Sent Done false
28133 8435 Acked Read 7 Done Sent Done false
28134 8436 Acked Write 6 Done Sent Done false
28135 8437 Acked Write 1 Done Sent Done false
28136 8438 Acked Read 7 Done Sent Done false
28137 8439 Acked Read 1 Done Sent Done false
28138 8440 Acked Read 9 Done Sent Done false
28139 8441 Acked Write 9 Done Sent Done false
28140 8442 Acked Read 3 Done Sent Done false
28141 8443 Acked Read 7 Done Sent Done false
28142 8444 Acked Flush 0 Done Sent Done false
28143 8445 Acked Read 4 Done Sent Done false
28144 8446 Acked Write 3 Done Sent Done false
28145 8447 Acked Flush 0 Done Sent Done false
28146 8448 Acked Read 5 Done Sent Done false
28147 8449 Acked Flush 0 Done Sent Done false
28148 8450 Acked Write 5 Done Sent Done false
28149 8451 Acked Read 6 Done Sent Done false
28150 8452 Acked Write 7 Done Sent Done false
28151 8453 Acked Write 8 Done Sent Done false
28152 8454 Acked Flush 0 Done Sent Done false
28153 8455 Acked Write 5 Done Sent Done false
28154 8456 Acked Read 9 Done Sent Done false
28155 8457 Acked Read 6 Done Sent Done false
28156 8458 Acked Read 8 Done Sent Done false
28157 8459 Acked Read 5 Done Sent Done false
28158 8460 Acked Write 9 Done Sent Done false
28159 8461 Acked Write 3 Done Sent Done false
28160 8462 Acked Write 7 Done Sent Done false
28161 8463 Acked Write 7 Done Sent Done false
28162 8464 Acked Flush 0 Done Sent Done false
28163 8465 Acked Write 3 Done Sent Done false
28164 8466 Acked Write 9 Done Sent Done false
28165 8467 Acked Read 9 Done Sent Done false
28166 8468 Acked Write 3 Done Sent Done false
28167 8469 Acked Write 5 Done Sent Done false
28168 8470 Acked Read 10 Done Sent Done false
28169 8471 Acked Write 3 Done Sent Done false
28170 8472 Acked Read 1 Done Sent Done false
28171 8473 Acked Read 7 Done Sent Done false
28172 8474 Acked Write 9 Done Sent Done false
28173 8475 Acked Write 5 Done Sent Done false
28174 8476 Acked Write 3 Done Sent Done false
28175 8477 Acked Read 7 Done Sent Done false
28176 8478 Acked Read 7 Done Sent Done false
28177 8479 Acked Flush 0 Done Sent Done false
28178 8480 Acked Read 6 Done Sent Done false
28179 8481 Acked Read 10 Done Sent Done false
28180 8482 Acked Read 1 Done Sent Done false
28181 8483 Acked Write 9 Done Sent Done false
28182 8484 Acked Read 2 Done Sent Done false
28183 8485 Acked Flush 0 Done Sent Done false
28184 8486 Acked Write 6 Done Sent Done false
28185 8487 Acked Write 3 Done Sent Done false
28186 8488 Acked Read 10 Done Sent Done false
28187 8489 Acked Write 8 Done Sent Done false
28188 8490 Acked Write 9 Done Sent Done false
28189 8491 Acked Read 5 Done Sent Done false
28190 8492 Acked Read 4 Done Sent Done false
28191 8493 Acked Write 3 Done Sent Done false
28192 8494 Acked Flush 0 Done Sent Done false
28193 8495 Acked Flush 0 Done Sent Done false
28194 8496 Acked Write 3 Done Sent Done false
28195 8497 Acked Write 10 Done Sent Done false
28196 8498 Acked Write 5 Done Sent Done false
28197 8499 Acked Read 7 Done Sent Done false
28198 8500 Acked Write 5 Done Sent Done false
28199 8501 Acked Read 1 Done Sent Done false
28200 8502 Acked Write 5 Done Sent Done false
28201 8503 Acked Write 1 Done Sent Done false
28202 8504 Acked Read 8 Done Sent Done false
28203 8505 Acked Write 10 Done Sent Done false
28204 8506 Acked Read 6 Done Sent Done false
28205 8507 Acked Read 6 Done Sent Done false
28206 8508 Acked Read 9 Done Sent Done false
28207 8509 Acked Read 9 Done Sent Done false
28208 8510 Acked Flush 0 Done Sent Done false
28209 8511 Acked Write 3 Done Sent Done false
28210 8512 Acked Write 7 Done Sent Done false
28211 8513 Acked Flush 0 Done Sent Done false
28212 8514 Acked Read 3 Done Sent Done false
28213 8515 Acked Read 5 Done Sent Done false
28214 8516 Acked Write 9 Done Sent Done false
28215 8517 Acked Write 4 Done Sent Done false
28216 8518 Acked Write 8 Done Sent Done false
28217 8519 Acked Read 9 Done Sent Done false
28218 8520 Acked Read 10 Done Sent Done false
28219 8521 Acked Write 9 Done Sent Done false
28220 8522 Acked Read 4 Done Sent Done false
28221 8523 Acked Read 5 Done Sent Done false
28222 8524 Acked Read 10 Done Sent Done false
28223 8525 Acked Read 3 Done Sent Done false
28224 8526 Acked Write 1 Done Sent Done false
28225 8527 Acked Read 8 Done Sent Done false
28226 8528 Acked Read 3 Done Sent Done false
28227 8529 Acked Write 10 Done Sent Done false
28228 8530 Acked Write 8 Done Sent Done false
28229 8531 Acked Write 9 Done Sent Done false
28230 8532 Acked Read 6 Done Sent Done false
28231 8533 Acked Read 6 Done Sent Done false
28232 8534 Acked Flush 0 Done Sent Done false
28233 8535 Acked Read 4 Done Sent Done false
28234 8536 Acked Write 2 Done Sent Done false
28235 8537 Acked Write 8 Done Sent Done false
28236 8538 Acked Write 1 Done Sent Done false
28237 8539 Acked Write 3 Done Sent Done false
28238 8540 Acked Write 2 Done Sent Done false
28239 8541 Acked Read 3 Done Sent Done false
28240 8542 Acked Write 5 Done Sent Done false
28241 8543 Acked Write 7 Done Sent Done false
28242 8544 Acked Write 1 Done Sent Done false
28243 8545 Acked Flush 0 Done Sent Done false
28244 STATES DS:0 DS:1 DS:2 TOTAL
28245 Sent 0 302 0 302
28246 Done 302 0 302 604
28247 Skipped 0 0 0 0
28248 Error 0 0 0 0
28249 Last Flush: 8545 8243 8545
28250 Downstairs last five completed: 8243 8242 8241 8240 8239
28251 Upstairs last five completed: 8545 8544 8543 8542 8541
28252 CLIENT: Up:0 ds:302 act:2
282532025-01-27T19:42:16.862ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
282542025-01-27T19:42:16.862ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:59311) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
282552025-01-27T19:42:16.863ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
282562025-01-27T19:42:16.864ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
282572025-01-27T19:42:16.864ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
282582025-01-27T19:42:16.864ZINFOcrucible: send last flush ID to this DS: 8243 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
282592025-01-27T19:42:16.864ZINFOcrucible: Replied this last flush ID: 8243 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
282602025-01-27T19:42:16.864ZINFOcrucible: [1] client re-new 302 jobs since flush 8243 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
282612025-01-27T19:42:16.864ZINFOcrucible: [1] Replayed 302 jobs since flush: 8243 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
28262 ----------------------------------------------------------------
28263 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
28264 Downstairs last five completed: 8545 8544 8543 8542 8541
28265 Upstairs last five completed: 8545 8544 8543 8542 8541
28266 CLIENT: Up:0 ds:0 act:3
28267 Replay: All jobs finished
28268 Main task has work to do, go find it
28269 got dsc Stop(2)
28270 stop 2
28271 [8830] Got stop action so:false kr:false
28272 Jan 27 19:42:19.102 INFO request completed, latency_us: 326, response_code: 204, uri: /stop/cid/2, method: POST, req_id: 4c6004c8-3205-43a8-b8b2-f7bf5a792cd4, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
282732025-01-27T19:42:19.103ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
28274 Jan 27 19:42:19.103 INFO request completed, latency_us: {"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:42:19.103762573Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
28275 115, response_code: 200, uri: /state/cid/2, method: {"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:42:19.103826955Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
28276 GET, req_id: f2b1f0ad-594f-4eda-900c-78ee667c86c5, remote_addr: 127.0.0.1:55047, local_addr: {"msg":"client task is exiting","v":0,"name":"crucible","level":30,"time":"2025-01-27T19:42:19.103857632Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"2","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
28277 127.0.0.1:9998
28278 [8830] Exited with: ExitStatus(unix_wait_status(9))
28279 [8830][2] reports Exit
28280 Jan 27 19:42:23.106 INFO request completed, latency_us: 217, response_code: 200, uri: /state/cid/2, method: GET, req_id: 2559416e-2fa3-4fba-99f2-2dd3132525ba, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
28281 001/300 Write block 48 len 20480 data: 8 9 9 9 8
28282 002/300 Read block 182 len 28672
28283 003/300 Read block 362 len 40960
28284 004/300 Read block 4930 len 32768
28285 005/300 Read block 479 len 16384
28286 006/300 Read block 4806 len 20480
28287 007/300 Flush
28288 008/300 Flush
28289 009/300 Write block 4008 len 32768 data: 7 8 8 6 5 6 4 4
28290 010/300 Write block 3315 len 8192 data: 5 5
28291 011/300 Write block 916 len 28672 data: 4 4 6 8 9 9 7
28292 012/300 Read block 2879 len 40960
28293 013/300 Read block 2600 len 4096
28294 014/300 Read block 1518 len 32768
28295 015/300 Write block 1991 len 36864 data: 4 4 4 4 5 6 6 7 7
28296 016/300 Read block 4013 len 20480
28297 017/300 Write block 760 len 40960 data: 4 4 4 4 4 5 6 5 5 5
28298 018/300 Read block 869 len 28672
28299 019/300 Read block 1410 len 36864
28300 020/300 Read block 4248 len 8192
28301 021/300 Read block 2209 len 40960
28302 022/300 Write block 2378 len 28672 data: 11 6 6 5 4 4 4
28303 023/300 Read block 1791 len 24576
28304 024/300 Write block 2555 len 24576 data: 6 6 7 7 7 7
28305 025/300 Read block 2560 len 40960
28306 026/300 Write block 1400 len 40960 data: 8 7 6 7 6 9 9 8 8 7
28307 027/300 Flush
28308 028/300 Write block 2182 len 12288 data: 8 7 8
28309 029/300 Read block 323 len 8192
28310 030/300 Write block 4534 len 32768 data: 7 6 5 4 3 2 3 3
28311 031/300 Read block 4281 len 28672
28312 032/300 Write block 4785 len 20480 data: 5 6 5 5 6
28313 033/300 Write block 3090 len 16384 data: 5 6 6 6
28314 034/300 Read block 4721 len 28672
28315 035/300 Read block 1148 len 16384
28316 036/300 Read block 1133 len 24576
28317 037/300 Read block 4018 len 28672
28318 038/300 Read block 660 len 4096
28319 039/300 Read block 3934 len 16384
28320 040/300 Read block 3656 len 32768
28321 041/300 Write block 2303 len 20480 data: 7 7 7 5 6
28322 042/300 Read block 3677 len 32768
28323 043/300 Write block 3817 len 20480 data: 4 3 3 2 2
28324 044/300 Write block 861 len 36864 data: 10 9 7 8 7 9 9 9 6
28325 045/300 Flush
28326 046/300 Read block 1527 len 20480
28327 047/300 Read block 2835 len 32768
28328 048/300 Read block 2106 len 16384
28329 049/300 Read block 1300 len 20480
28330 050/300 Write block 4682 len 8192 data: 7 6
28331 051/300 Flush
28332 052/300 Flush
28333 053/300 Read block 4500 len 16384
28334 054/300 Write block 425 len 32768 data: 6 6 4 4 5 4 4 3
28335 055/300 Read block 554 len 16384
28336 056/300 Read block 4436 len 8192
28337 057/300 Flush
28338 058/300 Flush
28339 059/300 Write block 3617 len 28672 data: 6 8 7 6 8 7 8
28340 060/300 Write block 2346 len 8192 data: 5 6
28341 061/300 Write block 3046 len 40960 data: 6 6 7 7 8 9 9 8 8 7
28342 062/300 Read block 2703 len 36864
28343 063/300 Read block 752 len 4096
28344 064/300 Write block 4820 len 16384 data: 3 4 5 5
28345 065/300 Read block 4663 len 12288
28346 066/300 Write block 4938 len 40960 data: 6 6 5 5 5 6 6 4 4 4
28347 067/300 Read block 1029 len 8192
28348 068/300 Flush
28349 069/300 Read block 2117 len 4096
28350 070/300 Read block 4074 len 16384
28351 071/300 Flush
28352 072/300 Write block 3660 len 12288 data: 4 4 4
28353 073/300 Read block 440 len 32768
28354 074/300 Write block 4589 len 16384 data: 4 4 5 5
28355 075/300 Read block 4882 len 8192
28356 076/300 Read block 3285 len 24576
28357 077/300 Flush
28358 078/300 Flush
28359 079/300 Read block 3452 len 20480
28360 080/300 Flush
28361 081/300 Read block 2536 len 12288
28362 082/300 Write block 1395 len 28672 data: 6 6 7 8 9 9 8
28363 083/300 Write block 2501 len 20480 data: 3 3 2 2 2
28364 084/300 Read block 185 len 16384
28365 085/300 Read block 1801 len 24576
28366 086/300 Write block 1411 len 12288 data: 5 7 7
28367 087/300 Write block 4164 len 40960 data: 9 9 8 8 8 7 6 5 5 7
28368 088/300 Write block 2447 len 36864 data: 4 4 5 6 5 4 5 5 4
28369 089/300 Write block 4866 len 36864 data: 6 9 8 9 8 6 5 2 2
28370 090/300 Write block 2359 len 4096 data: 4
28371 091/300 Flush
28372 092/300 Read block 3133 len 32768
28373 093/300 Write block 3804 len 36864 data: 3 4 4 4 4 6 7 6 6
28374 094/300 Read block 4805 len 36864
28375 095/300 Write block 2329 len 32768 data: 2 2 4 4 4 3 4 4
28376 096/300 Write block 4764 len 8192 data: 7 7
28377 097/300 Flush
28378 098/300 Read block 2911 len 36864
28379 099/300 Write block 4829 len 24576 data: 6 7 6 5 6 5
28380 100/300 Read block 1365 len 24576
28381 101/300 Write block 126 len 4096 data: 5
28382 102/300 Read block 3473 len 28672
28383 103/300 Flush
28384 104/300 Flush
28385 105/300 Read block 3371 len 8192
28386 106/300 Write block 2139 len 36864 data: 3 4 5 5 5 9 7 7 5
28387 107/300 Read block 3054 len 24576
28388 108/300 Flush
28389 109/300 Write block 2546 len 8192 data: 2 2
28390 110/300 Read block 2040 len 20480
28391 111/300 Write block 1821 len 24576 data: 7 6 3 3 4 4
28392 112/300 Flush
28393 113/300 Read block 1846 len 20480
28394 114/300 Read block 4277 len 8192
28395 115/300 Read block 1789 len 36864
28396 116/300 Read block 4158 len 40960
28397 117/300 Read block 1193 len 40960
28398 118/300 Read block 1844 len 36864
28399 119/300 Read block 2623 len 16384
28400 120/300 Read block 4528 len 20480
28401 121/300 Write block 2754 len 24576 data: 6 6 6 4 5 4
28402 122/300 Read block 1518 len 4096
28403 123/300 Write block 1290 len 32768 data: 3 3 4 4 7 6 6 8
28404 124/300 Flush
28405 125/300 Read block 3523 len 32768
28406 126/300 Read block 536 len 32768
28407 127/300 Read block 2933 len 40960
28408 128/300 Flush
28409 129/300 Write block 835 len 36864 data: 8 6 4 4 5 4 4 4 5
28410 130/300 Read block 4888 len 4096
28411 131/300 Write block 4790 len 16384 data: 8 8 7 6
28412 132/300 Read block 3870 len 8192
28413 133/300 Write block 4564 len 36864 data: 9 7 6 6 7 5 3 3 3
28414 134/300 Write block 1488 len 32768 data: 4 3 4 5 4 5 5 4
28415 135/300 Write block 4131 len 36864 data: 5 5 6 7 6 6 6 4 4
28416 136/300 Read block 3614 len 8192
28417 137/300 Write block 873 len 36864 data: 7 5 5 5 4 5 5 4 4
28418 138/300 Read block 1569 len 40960
28419 139/300 Write block 4237 len 20480 data: 4 5 4 5 5
28420 140/300 Read block 1364 len 24576
28421 141/300 Write block 1739 len 4096 data: 7
28422 142/300 Read block 1379 len 20480
28423 143/300 Read block 864 len 40960
28424 144/300 Write block 2942 len 16384 data: 6 4 5 5
28425 145/300 Write block 3716 len 40960 data: 8 8 7 5 7 7 6 6 7 8
28426 146/300 Read block 2974 len 4096
28427 147/300 Read block 612 len 40960
28428 148/300 Read block 2846 len 12288
28429 149/300 Read block 2356 len 4096
28430 150/300 Read block 190 len 8192
28431 151/300 Write block 2614 len 28672 data: 6 5 5 6 6 8 8
28432 152/300 Write block 2372 len 8192 data: 8 8
28433 153/300 Read block 255 len 28672
28434 154/300 Write block 223 len 20480 data: 7 7 7 9 7
28435 155/300 Flush
28436 156/300 Read block 2025 len 40960
28437 157/300 Write block 726 len 32768 data: 6 4 4 5 5 4 4 4
28438 158/300 Read block 3453 len 12288
28439 159/300 Read block 3948 len 28672
28440 160/300 Read block 3933 len 20480
28441 161/300 Flush
28442 162/300 Read block 3937 len 20480
28443 163/300 Write block 3691 len 12288 data: 7 8 7
28444 164/300 Write block 1819 len 20480 data: 6 8 8 7 4
28445 165/300 Write block 705 len 8192 data: 6 6
28446 166/300 Write block 1815 len 32768 data: 11 9 8 6 7 9 9 8
28447 167/300 Flush
28448 168/300 Write block 3560 len 4096 data: 6
28449 169/300 Write block 1056 len 20480 data: 2 4 6 7 7
28450 170/300 Write block 3673 len 28672 data: 6 6 4 4 4 4 4
28451 171/300 Write block 3688 len 16384 data: 9 9 8 8
28452 172/300 Write block 852 len 12288 data: 3 3 3
28453 173/300 Write block 4592 len 8192 data: 6 6
28454 174/300 Write block 1744 len 8192 data: 6 5
28455 175/300 Write block 3653 len 36864 data: 4 4 4 2 2 4 4 5 5
28456 176/300 Write block 2018 len 20480 data: 6 5 5 5 5
28457 177/300 Read block 1265 len 4096
28458 178/300 Read block 1022 len 40960
28459 179/300 Read block 2304 len 16384
28460 180/300 Read block 2806 len 32768
28461 181/300 Read block 3420 len 8192
28462 182/300 Read block 2566 len 8192
28463 183/300 Flush
28464 184/300 Write block 1438 len 40960 data: 4 4 4 5 5 4 5 5 6 5
28465 185/300 Flush
28466 186/300 Read block 1779 len 40960
28467 187/300 Read block 2267 len 8192
28468 188/300 Write block 401 len 4096 data: 7
28469 189/300 Read block 2655 len 16384
28470 190/300 Write block 325 len 24576 data: 7 6 7 6 6 5
28471 191/300 Write block 556 len 40960 data: 4 5 5 6 6 5 5 5 6 4
28472 192/300 Write block 434 len 36864 data: 2 3 3 4 4 6 7 4 4
28473 193/300 Flush
28474 194/300 Read block 2584 len 12288
28475 195/300 Write block 3699 len 36864 data: 3 3 3 3 5 4 5 5 7
28476 196/300 Read block 3105 len 16384
28477 197/300 Write block 4160 len 24576 data: 7 9 9 9 10 10
28478 198/300 Flush
28479 199/300 Write block 2920 len 12288 data: 6 4 3
28480 200/300 Flush
28481 201/300 Flush
28482 202/300 Write block 790 len 8192 data: 5 5
28483 203/300 Write block 1997 len 40960 data: 7 8 8 5 4 5 4 4 4 4
28484 204/300 Read block 1560 len 40960
28485 205/300 Read block 1348 len 8192
28486 206/300 Write block 1227 len 36864 data: 5 5 3 2 4 3 5 9 10
28487 207/300 Read block 1040 len 20480
28488 208/300 Read block 2659 len 24576
28489 209/300 Write block 1387 len 4096 data: 5
28490 210/300 Read block 1893 len 12288
28491 211/300 Read block 3077 len 16384
28492 212/300 Write block 2612 len 12288 data: 6 6 7
28493 213/300 Read block 2409 len 36864
28494 214/300 Write block 377 len 12288 data: 5 5 5
28495 215/300 Write block 4255 len 32768 data: 7 8 7 6 5 7 6 5
28496 216/300 Read block 4069 len 4096
28497 217/300 Read block 3822 len 4096
28498 218/300 Read block 2456 len 20480
28499 219/300 Write block 758 len 8192 data: 4 4
28500 220/300 Write block 3187 len 24576 data: 5 5 5 6 6 6
28501 221/300 Flush
28502 222/300 Read block 2264 len 40960
28503 223/300 Read block 1647 len 8192
28504 224/300 Read block 258 len 12288
28505 225/300 Read block 3957 len 36864
28506 226/300 Read block 2216 len 40960
28507 227/300 Write block 1108 len 8192 data: 3 3
28508 228/300 Read block 1102 len 40960
28509 229/300 Read block 4776 len 16384
28510 230/300 Read block 1308 len 28672
28511 231/300 Read block 884 len 8192
28512 232/300 Write block 495 len 36864 data: 8 9 11 11 12 10 11 11 8
28513 233/300 Write block 3404 len 16384 data: 4 5 4 4
28514 234/300 Read block 4159 len 16384
28515 235/300 Read block 4538 len 32768
28516 236/300 Write block 2337 len 16384 data: 4 4 6 7
28517 237/300 Read block 562 len 4096
28518 238/300 Read block 1952 len 20480
28519 239/300 Write block 3757 len 4096 data: 6
28520 240/300 Write block 918 len 36864 data: 7 9 10 10 8 5 6 6 5
28521 241/300 Read block 489 len 36864
28522 242/300 Flush
28523 243/300 Read block 970 len 36864
28524 244/300 Write block 4244 len 32768 data: 5 7 8 9 10 10 11 8
28525 245/300 Write block 4061 len 4096 data: 4
28526 246/300 Read block 4323 len 36864
28527 247/300 Write block 602 len 28672 data: 9 9 8 7 4 4 6
28528 248/300 Write block 1829 len 12288 data: 2 2 2
28529 249/300 Read block 2121 len 4096
28530 250/300 Read block 103 len 40960
28531 251/300 Write block 4181 len 24576 data: 3 3 2 2 3 4
28532 252/300 Write block 2242 len 28672 data: 6 6 6 7 8 9 8
28533 253/300 Flush
28534 254/300 Read block 821 len 16384
28535 255/300 Read block 4209 len 32768
28536 256/300 Read block 4722 len 16384
28537 257/300 Read block 2700 len 28672
28538 258/300 Write block 2665 len 16384 data: 6 5 3 2
28539 259/300 Read block 2053 len 8192
28540 260/300 Write block 3703 len 16384 data: 6 5 6 6
28541 261/300 Read block 4656 len 20480
28542 262/300 Write block 37 len 4096 data: 5
28543 263/300 Read block 1684 len 36864
28544 264/300 Write block 1957 len 4096 data: 6
28545 265/300 Read block 3290 len 12288
28546 266/300 Read block 2446 len 4096
28547 267/300 Read block 4684 len 40960
28548 268/300 Write block 4193 len 20480 data: 9 9 9 9 9
28549 269/300 Read block 2070 len 24576
28550 270/300 Write block 2909 len 8192 data: 4 4
28551 271/300 Read block 2139 len 28672
28552 272/300 Flush
28553 273/300 Read block 2432 len 40960
28554 274/300 Write block 2615 len 32768 data: 6 6 7 7 9 9 9 9
28555 275/300 Read block 4867 len 20480
28556 276/300 Read block 3329 len 4096
28557 277/300 Write block 61 len 8192 data: 6 8
28558 278/300 Read block 2857 len 4096
28559 279/300 Flush
28560 280/300 Write block 3894 len 4096 data: 6
28561 281/300 Write block 2106 len 4096 data: 6
28562 282/300 Read block 1800 len 36864
28563 283/300 Read block 3444 len 24576
28564 284/300 Read block 2973 len 32768
28565 285/300 Read block 4957 len 32768
28566 286/300 Read block 1227 len 16384
28567 287/300 Write block 4892 len 28672 data: 4 4 4 3 4 5 5
28568 288/300 Write block 2912 len 20480 data: 4 4 3 2 2
28569 289/300 Write block 785 len 36864 data: 3 5 5 4 5 6 6 4 4
28570 290/300 Read block 791 len 36864
28571 291/300 Read block 4458 len 24576
28572 292/300 Write block 2282 len 4096 data: 7
28573 293/300 Write block 572 len 12288 data: 4 5 5
28574 294/300 Write block 2524 len 16384 data: 5 4 3 3
28575 295/300 Flush
28576 296/300 Read block 4028 len 28672
28577 297/300 Read block 3799 len 28672
28578 298/300 Read block 2740 len 12288
28579 299/300 Flush
28580 300/300 Read block 4046 len 8192
28581 301/300 Write block 3442 len 12288 data: 5 5 5
28582 Main task has work to do, go find it
28583 got dsc Start(2)
28584 start 2
28585 Jan 27 19:42:23.344 INFO request completed, latency_us: 164, response_code: 204, uri: /start/cid/2, method: POST, req_id: f67f2bc2-32ae-4db5-b772-65ea82b7b2fc, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
28586 Replay: started 2, returned:Ok(())
28587 ----------------------------------------------------------------
28588 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
28589 States: Active Active Offline
28590 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28591 8546 Acked Write 5 Done Done Sent false
28592 8547 Acked Read 7 Done Done Sent false
28593 8548 Acked Read 10 Done Done Sent false
28594 8549 Acked Read 8 Done Done Sent false
28595 8550 Acked Read 4 Done Done Sent false
28596 8551 Acked Read 5 Done Done Sent false
28597 8552 Acked Flush 0 Done Done Sent false
28598 8553 Acked Flush 0 Done Done Sent false
28599 8554 Acked Write 8 Done Done Sent false
28600 8555 Acked Write 2 Done Done Sent false
28601 8556 Acked Write 7 Done Done Sent false
28602 8557 Acked Read 10 Done Done Sent false
28603 8558 Acked Read 1 Done Done Sent false
28604 8559 Acked Read 8 Done Done Sent false
28605 8560 Acked Write 9 Done Done Sent false
28606 8561 Acked Read 5 Done Done Sent false
28607 8562 Acked Write 10 Done Done Sent false
28608 8563 Acked Read 7 Done Done Sent false
28609 8564 Acked Read 9 Done Done Sent false
28610 8565 Acked Read 2 Done Done Sent false
28611 8566 Acked Read 10 Done Done Sent false
28612 8567 Acked Write 7 Done Done Sent false
28613 8568 Acked Read 6 Done Done Sent false
28614 8569 Acked Write 6 Done Done Sent false
28615 8570 Acked Read 10 Done Done Sent false
28616 8571 Acked Write 10 Done Done Sent false
28617 8572 Acked Flush 0 Done Done Sent false
28618 8573 Acked Write 3 Done Done Sent false
28619 8574 Acked Read 2 Done Done Sent false
28620 8575 Acked Write 8 Done Done Sent false
28621 8576 Acked Read 7 Done Done Sent false
28622 8577 Acked Write 5 Done Done Sent false
28623 8578 Acked Write 4 Done Done Sent false
28624 8579 Acked Read 7 Done Done Sent false
28625 8580 Acked Read 4 Done Done Sent false
28626 8581 Acked Read 6 Done Done Sent false
28627 8582 Acked Read 7 Done Done Sent false
28628 8583 Acked Read 1 Done Done Sent false
28629 8584 Acked Read 4 Done Done Sent false
28630 8585 Acked Read 8 Done Done Sent false
28631 8586 Acked Write 5 Done Done Sent false
28632 8587 Acked Read 8 Done Done Sent false
28633 8588 Acked Write 5 Done Done Sent false
28634 8589 Acked Write 9 Done Done Sent false
28635 8590 Acked Flush 0 Done Done Sent false
28636 8591 Acked Read 5 Done Done Sent false
28637 8592 Acked Read 8 Done Done Sent false
28638 8593 Acked Read 4 Done Done Sent false
28639 8594 Acked Read 5 Done Done Sent false
28640 8595 Acked Write 2 Done Done Sent false
28641 8596 Acked Flush 0 Done Done Sent false
28642 8597 Acked Flush 0 Done Done Sent false
28643 8598 Acked Read 4 Done Done Sent false
28644 8599 Acked Write 8 Done Done Sent false
28645 8600 Acked Read 4 Done Done Sent false
28646 8601 Acked Read 2 Done Done Sent false
28647 8602 Acked Flush 0 Done Done Sent false
28648 8603 Acked Flush 0 Done Done Sent false
28649 8604 Acked Write 7 Done Done Sent false
28650 8605 Acked Write 2 Done Done Sent false
28651 8606 Acked Write 10 Done Done Sent false
28652 8607 Acked Read 9 Done Done Sent false
28653 8608 Acked Read 1 Done Done Sent false
28654 8609 Acked Write 4 Done Done Sent false
28655 8610 Acked Read 3 Done Done Sent false
28656 8611 Acked Write 10 Done Done Sent false
28657 8612 Acked Read 2 Done Done Sent false
28658 8613 Acked Flush 0 Done Done Sent false
28659 8614 Acked Read 1 Done Done Sent false
28660 8615 Acked Read 4 Done Done Sent false
28661 8616 Acked Flush 0 Done Done Sent false
28662 8617 Acked Write 3 Done Done Sent false
28663 8618 Acked Read 8 Done Done Sent false
28664 8619 Acked Write 4 Done Done Sent false
28665 8620 Acked Read 2 Done Done Sent false
28666 8621 Acked Read 6 Done Done Sent false
28667 8622 Acked Flush 0 Done Done Sent false
28668 8623 Acked Flush 0 Done Done Sent false
28669 8624 Acked Read 5 Done Done Sent false
28670 8625 Acked Flush 0 Done Done Sent false
28671 8626 Acked Read 3 Done Done Sent false
28672 8627 Acked Write 7 Done Done Sent false
28673 8628 Acked Write 5 Done Done Sent false
28674 8629 Acked Read 4 Done Done Sent false
28675 8630 Acked Read 6 Done Done Sent false
28676 8631 Acked Write 3 Done Done Sent false
28677 8632 Acked Write 10 Done Done Sent false
28678 8633 Acked Write 9 Done Done Sent false
28679 8634 Acked Write 9 Done Done Sent false
28680 8635 Acked Write 1 Done Done Sent false
28681 8636 Acked Flush 0 Done Done Sent false
28682 8637 Acked Read 8 Done Done Sent false
28683 8638 Acked Write 9 Done Done Sent false
28684 8639 Acked Read 9 Done Done Sent false
28685 8640 Acked Write 8 Done Done Sent false
28686 8641 Acked Write 2 Done Done Sent false
28687 8642 Acked Flush 0 Done Done Sent false
28688 8643 Acked Read 9 Done Done Sent false
28689 8644 Acked Write 6 Done Done Sent false
28690 8645 Acked Read 6 Done Done Sent false
28691 8646 Acked Write 1 Done Done Sent false
28692 8647 Acked Read 7 Done Done Sent false
28693 8648 Acked Flush 0 Done Done Sent false
28694 8649 Acked Flush 0 Done Done Sent false
28695 8650 Acked Read 2 Done Done Sent false
28696 8651 Acked Write 9 Done Done Sent false
28697 8652 Acked Read 6 Done Done Sent false
28698 8653 Acked Flush 0 Done Done Sent false
28699 8654 Acked Write 2 Done Done Sent false
28700 8655 Acked Read 5 Done Done Sent false
28701 8656 Acked Write 6 Done Done Sent false
28702 8657 Acked Flush 0 Done Done Sent false
28703 8658 Acked Read 5 Done Done Sent false
28704 8659 Acked Read 2 Done Done Sent false
28705 8660 Acked Read 9 Done Done Sent false
28706 8661 Acked Read 10 Done Done Sent false
28707 8662 Acked Read 10 Done Done Sent false
28708 8663 Acked Read 9 Done Done Sent false
28709 8664 Acked Read 4 Done Done Sent false
28710 8665 Acked Read 5 Done Done Sent false
28711 8666 Acked Write 6 Done Done Sent false
28712 8667 Acked Read 1 Done Done Sent false
28713 8668 Acked Write 8 Done Done Sent false
28714 8669 Acked Flush 0 Done Done Sent false
28715 8670 Acked Read 8 Done Done Sent false
28716 8671 Acked Read 8 Done Done Sent false
28717 8672 Acked Read 10 Done Done Sent false
28718 8673 Acked Flush 0 Done Done Sent false
28719 8674 Acked Write 9 Done Done Sent false
28720 8675 Acked Read 1 Done Done Sent false
28721 8676 Acked Write 4 Done Done Sent false
28722 8677 Acked Read 2 Done Done Sent false
28723 8678 Acked Write 9 Done Done Sent false
28724 8679 Acked Write 8 Done Done Sent false
28725 8680 Acked Write 9 Done Done Sent false
28726 8681 Acked Read 2 Done Done Sent false
28727 8682 Acked Write 9 Done Done Sent false
28728 8683 Acked Read 10 Done Done Sent false
28729 8684 Acked Write 5 Done Done Sent false
28730 8685 Acked Read 6 Done Done Sent false
28731 8686 Acked Write 1 Done Done Sent false
28732 8687 Acked Read 5 Done Done Sent false
28733 8688 Acked Read 10 Done Done Sent false
28734 8689 Acked Write 4 Done Done Sent false
28735 8690 Acked Write 10 Done Done Sent false
28736 8691 Acked Read 1 Done Done Sent false
28737 8692 Acked Read 10 Done Done Sent false
28738 8693 Acked Read 3 Done Done Sent false
28739 8694 Acked Read 1 Done Done Sent false
28740 8695 Acked Read 2 Done Done Sent false
28741 8696 Acked Write 7 Done Done Sent false
28742 8697 Acked Write 2 Done Done Sent false
28743 8698 Acked Read 7 Done Done Sent false
28744 8699 Acked Write 5 Done Done Sent false
28745 8700 Acked Flush 0 Done Done Sent false
28746 8701 Acked Read 10 Done Done Sent false
28747 8702 Acked Write 8 Done Done Sent false
28748 8703 Acked Read 3 Done Done Sent false
28749 8704 Acked Read 7 Done Done Sent false
28750 8705 Acked Read 5 Done Done Sent false
28751 8706 Acked Flush 0 Done Done Sent false
28752 8707 Acked Read 5 Done Done Sent false
28753 8708 Acked Write 3 Done Done Sent false
28754 8709 Acked Write 5 Done Done Sent false
28755 8710 Acked Write 2 Done Done Sent false
28756 8711 Acked Write 8 Done Done Sent false
28757 8712 Acked Flush 0 Done Done Sent false
28758 8713 Acked Write 1 Done Done Sent false
28759 8714 Acked Write 5 Done Done Sent false
28760 8715 Acked Write 7 Done Done Sent false
28761 8716 Acked Write 4 Done Done Sent false
28762 8717 Acked Write 3 Done Done Sent false
28763 8718 Acked Write 2 Done Done Sent false
28764 8719 Acked Write 2 Done Done Sent false
28765 8720 Acked Write 9 Done Done Sent false
28766 8721 Acked Write 5 Done Done Sent false
28767 8722 Acked Read 1 Done Done Sent false
28768 8723 Acked Read 10 Done Done Sent false
28769 8724 Acked Read 4 Done Done Sent false
28770 8725 Acked Read 8 Done Done Sent false
28771 8726 Acked Read 2 Done Done Sent false
28772 8727 Acked Read 2 Done Done Sent false
28773 8728 Acked Flush 0 Done Done Sent false
28774 8729 Acked Write 10 Done Done Sent false
28775 8730 Acked Flush 0 Done Done Sent false
28776 8731 Acked Read 10 Done Done Sent false
28777 8732 Acked Read 2 Done Done Sent false
28778 8733 Acked Write 1 Done Done Sent false
28779 8734 Acked Read 4 Done Done Sent false
28780 8735 Acked Write 6 Done Done Sent false
28781 8736 Acked Write 10 Done Done Sent false
28782 8737 Acked Write 9 Done Done Sent false
28783 8738 Acked Flush 0 Done Done Sent false
28784 8739 Acked Read 3 Done Done Sent false
28785 8740 Acked Write 9 Done Done Sent false
28786 8741 Acked Read 4 Done Done Sent false
28787 8742 Acked Write 6 Done Done Sent false
28788 8743 Acked Flush 0 Done Done Sent false
28789 8744 Acked Write 3 Done Done Sent false
28790 8745 Acked Flush 0 Done Done Sent false
28791 8746 Acked Flush 0 Done Done Sent false
28792 8747 Acked Write 2 Done Done Sent false
28793 8748 Acked Write 10 Done Done Sent false
28794 8749 Acked Read 10 Done Done Sent false
28795 8750 Acked Read 2 Done Done Sent false
28796 8751 Acked Write 9 Done Done Sent false
28797 8752 Acked Read 5 Done Done Sent false
28798 8753 Acked Read 6 Done Done Sent false
28799 8754 Acked Write 1 Done Done Sent false
28800 8755 Acked Read 3 Done Done Sent false
28801 8756 Acked Read 4 Done Done Sent false
28802 8757 Acked Write 3 Done Done Sent false
28803 8758 Acked Read 9 Done Done Sent false
28804 8759 Acked Write 3 Done Done Sent false
28805 8760 Acked Write 8 Done Done Sent false
28806 8761 Acked Read 1 Done Done Sent false
28807 8762 Acked Read 1 Done Done Sent false
28808 8763 Acked Read 5 Done Done Sent false
28809 8764 Acked Write 2 Done Done Sent false
28810 8765 Acked Write 6 Done Done Sent false
28811 8766 Acked Flush 0 Done Done Sent false
28812 8767 Acked Read 10 Done Done Sent false
28813 8768 Acked Read 2 Done Done Sent false
28814 8769 Acked Read 3 Done Done Sent false
28815 8770 Acked Read 9 Done Done Sent false
28816 8771 Acked Read 10 Done Done Sent false
28817 8772 Acked Write 2 Done Done Sent false
28818 8773 Acked Read 10 Done Done Sent false
28819 8774 Acked Read 4 Done Done Sent false
28820 8775 Acked Read 7 Done Done Sent false
28821 8776 Acked Read 2 Done Done Sent false
28822 8777 Acked Write 9 Done Done Sent false
28823 8778 Acked Write 4 Done Done Sent false
28824 8779 Acked Read 4 Done Done Sent false
28825 8780 Acked Read 8 Done Done Sent false
28826 8781 Acked Write 4 Done Done Sent false
28827 8782 Acked Read 1 Done Done Sent false
28828 8783 Acked Read 5 Done Done Sent false
28829 8784 Acked Write 1 Done Done Sent false
28830 8785 Acked Write 9 Done Done Sent false
28831 8786 Acked Read 9 Done Done Sent false
28832 8787 Acked Flush 0 Done Done Sent false
28833 8788 Acked Read 9 Done Done Sent false
28834 8789 Acked Write 8 Done Done Sent false
28835 8790 Acked Write 1 Done Done Sent false
28836 8791 Acked Read 9 Done Done Sent false
28837 8792 Acked Write 7 Done Done Sent false
28838 8793 Acked Write 3 Done Done Sent false
28839 8794 Acked Read 1 Done Done Sent false
28840 8795 Acked Read 10 Done Done Sent false
28841 8796 Acked Write 6 Done Done Sent false
28842 8797 Acked Write 7 Done Done Sent false
28843 8798 Acked Flush 0 Done Done Sent false
28844 8799 Acked Read 4 Done Done Sent false
28845 8800 Acked Read 8 Done Done Sent false
28846 8801 Acked Read 4 Done Done Sent false
28847 8802 Acked Read 7 Done Done Sent false
28848 8803 Acked Write 4 Done Done Sent false
28849 8804 Acked Read 2 Done Done Sent false
28850 8805 Acked Write 4 Done Done Sent false
28851 8806 Acked Read 5 Done Done Sent false
28852 8807 Acked Write 1 Done Done Sent false
28853 8808 Acked Read 9 Done Done Sent false
28854 8809 Acked Write 1 Done Done Sent false
28855 8810 Acked Read 3 Done Done Sent false
28856 8811 Acked Read 1 Done Done Sent false
28857 8812 Acked Read 10 Done Done Sent false
28858 8813 Acked Write 5 Done Done Sent false
28859 8814 Acked Read 6 Done Done Sent false
28860 8815 Acked Write 2 Done Done Sent false
28861 8816 Acked Read 7 Done Done Sent false
28862 8817 Acked Flush 0 Done Done Sent false
28863 8818 Acked Read 10 Done Done Sent false
28864 8819 Acked Write 8 Done Done Sent false
28865 8820 Acked Read 5 Done Done Sent false
28866 8821 Acked Read 1 Done Done Sent false
28867 8822 Acked Write 2 Done Done Sent false
28868 8823 Acked Read 1 Done Done Sent false
28869 8824 Acked Flush 0 Done Done Sent false
28870 8825 Acked Write 1 Done Done Sent false
28871 8826 Acked Write 1 Done Done Sent false
28872 8827 Acked Read 9 Done Done Sent false
28873 8828 Acked Read 6 Done Done Sent false
28874 8829 Acked Read 8 Done Done Sent false
28875 8830 Acked Read 8 Done Done Sent false
28876 8831 Acked Read 4 Done Done Sent false
28877 8832 Acked Write 7 Done Done Sent false
28878 8833 Acked Write 5 Done Done Sent false
28879 8834 Acked Write 9 Done Done Sent false
28880 8835 Acked Read 9 Done Done Sent false
28881 8836 Acked Read 6 Done Done Sent false
28882 8837 Acked Write 1 Done Done Sent false
28883 8838 Acked Write 3 Done Done Sent false
28884 8839 Acked Write 4 Done Done Sent false
28885 8840 Acked Flush 0 Done Done Sent false
28886 8841 Acked Read 7 Done Done Sent false
28887 8842 Acked Read 7 Done Done Sent false
28888 8843 Acked Read 3 Done Done Sent false
28889 8844 Acked Flush 0 Done Done Sent false
28890 8845 Acked Read 2 Done Done Sent false
28891 8846 Acked Write 3 Done Done Sent false
28892 STATES DS:0 DS:1 DS:2 TOTAL
28893 Sent 0 0 301 301
28894 Done 301 301 0 602
28895 Skipped 0 0 0 0
28896 Error 0 0 0 0
28897 Last Flush: 8844 8844 8545
28898 Downstairs last five completed: 8545 8544 8543 8542 8541
28899 Upstairs last five completed: 8846 8845 8844 8843 8842
28900 CLIENT: Up:0 ds:301 act:2
28901 [8830] got start action
28902 [8830] I am going to restart
28903 Starting downstairs at port 8830
28904 Make output file at "/tmp/dsc/downstairs-8830.txt"
28905 [8830][2] reports Starting
28906 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1226)
28907 [8830][2] reports Running
28908 ----------------------------------------------------------------
28909 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
28910 States: Active Active Offline
28911 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
28912 8546 Acked Write 5 Done Done Sent false
28913 8547 Acked Read 7 Done Done Sent false
28914 8548 Acked Read 10 Done Done Sent false
28915 8549 Acked Read 8 Done Done Sent false
28916 8550 Acked Read 4 Done Done Sent false
28917 8551 Acked Read 5 Done Done Sent false
28918 8552 Acked Flush 0 Done Done Sent false
28919 8553 Acked Flush 0 Done Done Sent false
28920 8554 Acked Write 8 Done Done Sent false
28921 8555 Acked Write 2 Done Done Sent false
28922 8556 Acked Write 7 Done Done Sent false
28923 8557 Acked Read 10 Done Done Sent false
28924 8558 Acked Read 1 Done Done Sent false
28925 8559 Acked Read 8 Done Done Sent false
28926 8560 Acked Write 9 Done Done Sent false
28927 8561 Acked Read 5 Done Done Sent false
28928 8562 Acked Write 10 Done Done Sent false
28929 8563 Acked Read 7 Done Done Sent false
28930 8564 Acked Read 9 Done Done Sent false
28931 8565 Acked Read 2 Done Done Sent false
28932 8566 Acked Read 10 Done Done Sent false
28933 8567 Acked Write 7 Done Done Sent false
28934 8568 Acked Read 6 Done Done Sent false
28935 8569 Acked Write 6 Done Done Sent false
28936 8570 Acked Read 10 Done Done Sent false
28937 8571 Acked Write 10 Done Done Sent false
28938 8572 Acked Flush 0 Done Done Sent false
28939 8573 Acked Write 3 Done Done Sent false
28940 8574 Acked Read 2 Done Done Sent false
28941 8575 Acked Write 8 Done Done Sent false
28942 8576 Acked Read 7 Done Done Sent false
28943 8577 Acked Write 5 Done Done Sent false
28944 8578 Acked Write 4 Done Done Sent false
28945 8579 Acked Read 7 Done Done Sent false
28946 8580 Acked Read 4 Done Done Sent false
28947 8581 Acked Read 6 Done Done Sent false
28948 8582 Acked Read 7 Done Done Sent false
28949 8583 Acked Read 1 Done Done Sent false
28950 8584 Acked Read 4 Done Done Sent false
28951 8585 Acked Read 8 Done Done Sent false
28952 8586 Acked Write 5 Done Done Sent false
28953 8587 Acked Read 8 Done Done Sent false
28954 8588 Acked Write 5 Done Done Sent false
28955 8589 Acked Write 9 Done Done Sent false
28956 8590 Acked Flush 0 Done Done Sent false
28957 8591 Acked Read 5 Done Done Sent false
28958 8592 Acked Read 8 Done Done Sent false
28959 8593 Acked Read 4 Done Done Sent false
28960 8594 Acked Read 5 Done Done Sent false
28961 8595 Acked Write 2 Done Done Sent false
28962 8596 Acked Flush 0 Done Done Sent false
28963 8597 Acked Flush 0 Done Done Sent false
28964 8598 Acked Read 4 Done Done Sent false
28965 8599 Acked Write 8 Done Done Sent false
28966 8600 Acked Read 4 Done Done Sent false
28967 8601 Acked Read 2 Done Done Sent false
28968 8602 Acked Flush 0 Done Done Sent false
28969 8603 Acked Flush 0 Done Done Sent false
28970 8604 Acked Write 7 Done Done Sent false
28971 8605 Acked Write 2 Done Done Sent false
28972 8606 Acked Write 10 Done Done Sent false
28973 8607 Acked Read 9 Done Done Sent false
28974 8608 Acked Read 1 Done Done Sent false
28975 8609 Acked Write 4 Done Done Sent false
28976 8610 Acked Read 3 Done Done Sent false
28977 8611 Acked Write 10 Done Done Sent false
28978 8612 Acked Read 2 Done Done Sent false
28979 8613 Acked Flush 0 Done Done Sent false
28980 8614 Acked Read 1 Done Done Sent false
28981 8615 Acked Read 4 Done Done Sent false
28982 8616 Acked Flush 0 Done Done Sent false
28983 8617 Acked Write 3 Done Done Sent false
28984 8618 Acked Read 8 Done Done Sent false
28985 8619 Acked Write 4 Done Done Sent false
28986 8620 Acked Read 2 Done Done Sent false
28987 8621 Acked Read 6 Done Done Sent false
28988 8622 Acked Flush 0 Done Done Sent false
28989 8623 Acked Flush 0 Done Done Sent false
28990 8624 Acked Read 5 Done Done Sent false
28991 8625 Acked Flush 0 Done Done Sent false
28992 8626 Acked Read 3 Done Done Sent false
28993 8627 Acked Write 7 Done Done Sent false
28994 8628 Acked Write 5 Done Done Sent false
28995 8629 Acked Read 4 Done Done Sent false
28996 8630 Acked Read 6 Done Done Sent false
28997 8631 Acked Write 3 Done Done Sent false
28998 8632 Acked Write 10 Done Done Sent false
28999 8633 Acked Write 9 Done Done Sent false
29000 8634 Acked Write 9 Done Done Sent false
29001 8635 Acked Write 1 Done Done Sent false
29002 8636 Acked Flush 0 Done Done Sent false
29003 8637 Acked Read 8 Done Done Sent false
29004 8638 Acked Write 9 Done Done Sent false
29005 8639 Acked Read 9 Done Done Sent false
29006 8640 Acked Write 8 Done Done Sent false
29007 8641 Acked Write 2 Done Done Sent false
29008 8642 Acked Flush 0 Done Done Sent false
29009 8643 Acked Read 9 Done Done Sent false
29010 8644 Acked Write 6 Done Done Sent false
29011 8645 Acked Read 6 Done Done Sent false
29012 8646 Acked Write 1 Done Done Sent false
29013 8647 Acked Read 7 Done Done Sent false
29014 8648 Acked Flush 0 Done Done Sent false
29015 8649 Acked Flush 0 Done Done Sent false
29016 8650 Acked Read 2 Done Done Sent false
29017 8651 Acked Write 9 Done Done Sent false
29018 8652 Acked Read 6 Done Done Sent false
29019 8653 Acked Flush 0 Done Done Sent false
29020 8654 Acked Write 2 Done Done Sent false
29021 8655 Acked Read 5 Done Done Sent false
29022 8656 Acked Write 6 Done Done Sent false
29023 8657 Acked Flush 0 Done Done Sent false
29024 8658 Acked Read 5 Done Done Sent false
29025 8659 Acked Read 2 Done Done Sent false
29026 8660 Acked Read 9 Done Done Sent false
29027 8661 Acked Read 10 Done Done Sent false
29028 8662 Acked Read 10 Done Done Sent false
29029 8663 Acked Read 9 Done Done Sent false
29030 8664 Acked Read 4 Done Done Sent false
29031 8665 Acked Read 5 Done Done Sent false
29032 8666 Acked Write 6 Done Done Sent false
29033 8667 Acked Read 1 Done Done Sent false
29034 8668 Acked Write 8 Done Done Sent false
29035 8669 Acked Flush 0 Done Done Sent false
29036 8670 Acked Read 8 Done Done Sent false
29037 8671 Acked Read 8 Done Done Sent false
29038 8672 Acked Read 10 Done Done Sent false
29039 8673 Acked Flush 0 Done Done Sent false
29040 8674 Acked Write 9 Done Done Sent false
29041 8675 Acked Read 1 Done Done Sent false
29042 8676 Acked Write 4 Done Done Sent false
29043 8677 Acked Read 2 Done Done Sent false
29044 8678 Acked Write 9 Done Done Sent false
29045 8679 Acked Write 8 Done Done Sent false
29046 8680 Acked Write 9 Done Done Sent false
29047 8681 Acked Read 2 Done Done Sent false
29048 8682 Acked Write 9 Done Done Sent false
29049 8683 Acked Read 10 Done Done Sent false
29050 8684 Acked Write 5 Done Done Sent false
29051 8685 Acked Read 6 Done Done Sent false
29052 8686 Acked Write 1 Done Done Sent false
29053 8687 Acked Read 5 Done Done Sent false
29054 8688 Acked Read 10 Done Done Sent false
29055 8689 Acked Write 4 Done Done Sent false
29056 8690 Acked Write 10 Done Done Sent false
29057 8691 Acked Read 1 Done Done Sent false
29058 8692 Acked Read 10 Done Done Sent false
29059 8693 Acked Read 3 Done Done Sent false
29060 8694 Acked Read 1 Done Done Sent false
29061 8695 Acked Read 2 Done Done Sent false
29062 8696 Acked Write 7 Done Done Sent false
29063 8697 Acked Write 2 Done Done Sent false
29064 8698 Acked Read 7 Done Done Sent false
29065 8699 Acked Write 5 Done Done Sent false
29066 8700 Acked Flush 0 Done Done Sent false
29067 8701 Acked Read 10 Done Done Sent false
29068 8702 Acked Write 8 Done Done Sent false
29069 8703 Acked Read 3 Done Done Sent false
29070 8704 Acked Read 7 Done Done Sent false
29071 8705 Acked Read 5 Done Done Sent false
29072 8706 Acked Flush 0 Done Done Sent false
29073 8707 Acked Read 5 Done Done Sent false
29074 8708 Acked Write 3 Done Done Sent false
29075 8709 Acked Write 5 Done Done Sent false
29076 8710 Acked Write 2 Done Done Sent false
29077 8711 Acked Write 8 Done Done Sent false
29078 8712 Acked Flush 0 Done Done Sent false
29079 8713 Acked Write 1 Done Done Sent false
29080 8714 Acked Write 5 Done Done Sent false
29081 8715 Acked Write 7 Done Done Sent false
29082 8716 Acked Write 4 Done Done Sent false
29083 8717 Acked Write 3 Done Done Sent false
29084 8718 Acked Write 2 Done Done Sent false
29085 8719 Acked Write 2 Done Done Sent false
29086 8720 Acked Write 9 Done Done Sent false
29087 8721 Acked Write 5 Done Done Sent false
29088 8722 Acked Read 1 Done Done Sent false
29089 8723 Acked Read 10 Done Done Sent false
29090 8724 Acked Read 4 Done Done Sent false
29091 8725 Acked Read 8 Done Done Sent false
29092 8726 Acked Read 2 Done Done Sent false
29093 8727 Acked Read 2 Done Done Sent false
29094 8728 Acked Flush 0 Done Done Sent false
29095 8729 Acked Write 10 Done Done Sent false
29096 8730 Acked Flush 0 Done Done Sent false
29097 8731 Acked Read 10 Done Done Sent false
29098 8732 Acked Read 2 Done Done Sent false
29099 8733 Acked Write 1 Done Done Sent false
29100 8734 Acked Read 4 Done Done Sent false
29101 8735 Acked Write 6 Done Done Sent false
29102 8736 Acked Write 10 Done Done Sent false
29103 8737 Acked Write 9 Done Done Sent false
29104 8738 Acked Flush 0 Done Done Sent false
29105 8739 Acked Read 3 Done Done Sent false
29106 8740 Acked Write 9 Done Done Sent false
29107 8741 Acked Read 4 Done Done Sent false
29108 8742 Acked Write 6 Done Done Sent false
29109 8743 Acked Flush 0 Done Done Sent false
29110 8744 Acked Write 3 Done Done Sent false
29111 8745 Acked Flush 0 Done Done Sent false
29112 8746 Acked Flush 0 Done Done Sent false
29113 8747 Acked Write 2 Done Done Sent false
29114 8748 Acked Write 10 Done Done Sent false
29115 8749 Acked Read 10 Done Done Sent false
29116 8750 Acked Read 2 Done Done Sent false
29117 8751 Acked Write 9 Done Done Sent false
29118 8752 Acked Read 5 Done Done Sent false
29119 8753 Acked Read 6 Done Done Sent false
29120 8754 Acked Write 1 Done Done Sent false
29121 8755 Acked Read 3 Done Done Sent false
29122 8756 Acked Read 4 Done Done Sent false
29123 8757 Acked Write 3 Done Done Sent false
29124 8758 Acked Read 9 Done Done Sent false
29125 8759 Acked Write 3 Done Done Sent false
29126 8760 Acked Write 8 Done Done Sent false
29127 8761 Acked Read 1 Done Done Sent false
29128 8762 Acked Read 1 Done Done Sent false
29129 8763 Acked Read 5 Done Done Sent false
29130 8764 Acked Write 2 Done Done Sent false
29131 8765 Acked Write 6 Done Done Sent false
29132 8766 Acked Flush 0 Done Done Sent false
29133 8767 Acked Read 10 Done Done Sent false
29134 8768 Acked Read 2 Done Done Sent false
29135 8769 Acked Read 3 Done Done Sent false
29136 8770 Acked Read 9 Done Done Sent false
29137 8771 Acked Read 10 Done Done Sent false
29138 8772 Acked Write 2 Done Done Sent false
29139 8773 Acked Read 10 Done Done Sent false
29140 8774 Acked Read 4 Done Done Sent false
29141 8775 Acked Read 7 Done Done Sent false
29142 8776 Acked Read 2 Done Done Sent false
29143 8777 Acked Write 9 Done Done Sent false
29144 8778 Acked Write 4 Done Done Sent false
29145 8779 Acked Read 4 Done Done Sent false
29146 8780 Acked Read 8 Done Done Sent false
29147 8781 Acked Write 4 Done Done Sent false
29148 8782 Acked Read 1 Done Done Sent false
29149 8783 Acked Read 5 Done Done Sent false
29150 8784 Acked Write 1 Done Done Sent false
29151 8785 Acked Write 9 Done Done Sent false
29152 8786 Acked Read 9 Done Done Sent false
29153 8787 Acked Flush 0 Done Done Sent false
29154 8788 Acked Read 9 Done Done Sent false
29155 8789 Acked Write 8 Done Done Sent false
29156 8790 Acked Write 1 Done Done Sent false
29157 8791 Acked Read 9 Done Done Sent false
29158 8792 Acked Write 7 Done Done Sent false
29159 8793 Acked Write 3 Done Done Sent false
29160 8794 Acked Read 1 Done Done Sent false
29161 8795 Acked Read 10 Done Done Sent false
29162 8796 Acked Write 6 Done Done Sent false
29163 8797 Acked Write 7 Done Done Sent false
29164 8798 Acked Flush 0 Done Done Sent false
29165 8799 Acked Read 4 Done Done Sent false
29166 8800 Acked Read 8 Done Done Sent false
29167 8801 Acked Read 4 Done Done Sent false
29168 8802 Acked Read 7 Done Done Sent false
29169 8803 Acked Write 4 Done Done Sent false
29170 8804 Acked Read 2 Done Done Sent false
29171 8805 Acked Write 4 Done Done Sent false
29172 8806 Acked Read 5 Done Done Sent false
29173 8807 Acked Write 1 Done Done Sent false
29174 8808 Acked Read 9 Done Done Sent false
29175 8809 Acked Write 1 Done Done Sent false
29176 8810 Acked Read 3 Done Done Sent false
29177 8811 Acked Read 1 Done Done Sent false
29178 8812 Acked Read 10 Done Done Sent false
29179 8813 Acked Write 5 Done Done Sent false
29180 8814 Acked Read 6 Done Done Sent false
29181 8815 Acked Write 2 Done Done Sent false
29182 8816 Acked Read 7 Done Done Sent false
29183 8817 Acked Flush 0 Done Done Sent false
29184 8818 Acked Read 10 Done Done Sent false
29185 8819 Acked Write 8 Done Done Sent false
29186 8820 Acked Read 5 Done Done Sent false
29187 8821 Acked Read 1 Done Done Sent false
29188 8822 Acked Write 2 Done Done Sent false
29189 8823 Acked Read 1 Done Done Sent false
29190 8824 Acked Flush 0 Done Done Sent false
29191 8825 Acked Write 1 Done Done Sent false
29192 8826 Acked Write 1 Done Done Sent false
29193 8827 Acked Read 9 Done Done Sent false
29194 8828 Acked Read 6 Done Done Sent false
29195 8829 Acked Read 8 Done Done Sent false
29196 8830 Acked Read 8 Done Done Sent false
29197 8831 Acked Read 4 Done Done Sent false
29198 8832 Acked Write 7 Done Done Sent false
29199 8833 Acked Write 5 Done Done Sent false
29200 8834 Acked Write 9 Done Done Sent false
29201 8835 Acked Read 9 Done Done Sent false
29202 8836 Acked Read 6 Done Done Sent false
29203 8837 Acked Write 1 Done Done Sent false
29204 8838 Acked Write 3 Done Done Sent false
29205 8839 Acked Write 4 Done Done Sent false
29206 8840 Acked Flush 0 Done Done Sent false
29207 8841 Acked Read 7 Done Done Sent false
29208 8842 Acked Read 7 Done Done Sent false
29209 8843 Acked Read 3 Done Done Sent false
29210 8844 Acked Flush 0 Done Done Sent false
29211 8845 Acked Read 2 Done Done Sent false
29212 8846 Acked Write 3 Done Done Sent false
29213 8847 Acked Flush 0 Done Done Sent false
29214 STATES DS:0 DS:1 DS:2 TOTAL
29215 Sent 0 0 302 302
29216 Done 302 302 0 604
29217 Skipped 0 0 0 0
29218 Error 0 0 0 0
29219 Last Flush: 8847 8847 8545
29220 Downstairs last five completed: 8545 8544 8543 8542 8541
29221 Upstairs last five completed: 8847 8846 8845 8844 8843
29222 CLIENT: Up:0 ds:302 act:2
292232025-01-27T19:42:29.104ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292242025-01-27T19:42:29.105ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:41890) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292252025-01-27T19:42:29.106ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292262025-01-27T19:42:29.106ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292272025-01-27T19:42:29.106ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292282025-01-27T19:42:29.106ZINFOcrucible: send last flush ID to this DS: 8545 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292292025-01-27T19:42:29.107ZINFOcrucible: Replied this last flush ID: 8545 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292302025-01-27T19:42:29.107ZINFOcrucible: [2] client re-new 302 jobs since flush 8545 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292312025-01-27T19:42:29.107ZINFOcrucible: [2] Replayed 302 jobs since flush: 8545 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
29232 ----------------------------------------------------------------
29233 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
29234 Downstairs last five completed: 8847 8846 8845 8844 8843
29235 Upstairs last five completed: 8847 8846 8845 8844 8843
29236 CLIENT: Up:0 ds:0 act:3
29237 Replay: All jobs finished
29238 Main task has work to do, go find it
29239 got dsc Stop(2)
29240 stop 2
29241 Jan 27 19:42:31.349 INFO request completed, latency_us: 349, response_code: 204, uri: /stop/cid/2, method: POST[8830] Got stop action so:false kr:false
29242 , req_id: ff72eda5-63dc-4484-b46b-b159014a8a84, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
292432025-01-27T19:42:31.350ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292442025-01-27T19:42:31.350ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292452025-01-27T19:42:31.350ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
292462025-01-27T19:42:31.350ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
29247 Jan 27 19:42:31.350 INFO request completed, latency_us: 149, response_code: 200, uri: /state/cid/2, method: GET, req_id: 76737829-05e0-4df0-b1ca-f79d502e691f, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
29248 [8830] Exited with: ExitStatus(unix_wait_status(9))
29249 [8830][2] reports Exit
29250 Jan 27 19:42:35.354 INFO request completed, latency_us: 248, response_code: 200, uri: /state/cid/2, method: GET, req_id: 45cd4c02-4d92-4801-9cc1-f98dd2e5b2f0, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
29251 001/300 Read block 3055 len 36864
29252 002/300 Write block 1467 len 24576 data: 7 6 4 6 6 4
29253 003/300 Read block 1295 len 40960
29254 004/300 Read block 2859 len 16384
29255 005/300 Write block 3120 len 4096 data: 5
29256 006/300 Write block 3984 len 20480 data: 4 3 3 4 4
29257 007/300 Write block 75 len 28672 data: 3 4 4 5 6 5 4
29258 008/300 Read block 4727 len 20480
29259 009/300 Read block 2175 len 20480
29260 010/300 Read block 349 len 24576
29261 011/300 Read block 3951 len 24576
29262 012/300 Read block 2995 len 24576
29263 013/300 Write block 4136 len 36864 data: 7 7 5 5 4 5 5 5 5
29264 014/300 Write block 3945 len 28672 data: 2 3 3 3 2 3 3
29265 015/300 Read block 2820 len 40960
29266 016/300 Read block 582 len 32768
29267 017/300 Flush
29268 018/300 Write block 3556 len 28672 data: 7 8 7 7 7 7 8
29269 019/300 Write block 4048 len 36864 data: 4 3 3 2 2 3 3 4 6
29270 020/300 Write block 3096 len 24576 data: 4 4 7 7 6 7
29271 021/300 Read block 2775 len 28672
29272 022/300 Write block 1198 len 20480 data: 8 6 5 5 5
29273 023/300 Write block 2506 len 36864 data: 4 5 7 7 5 6 7 6 7
29274 024/300 Write block 2170 len 36864 data: 4 6 7 8 7 8 9 7 7
29275 025/300 Read block 2258 len 12288
29276 026/300 Write block 1597 len 12288 data: 11 12 8
29277 027/300 Read block 1482 len 24576
29278 028/300 Read block 1841 len 24576
29279 029/300 Read block 3914 len 32768
29280 030/300 Write block 2008 len 32768 data: 6 6 6 9 6 7 6 6
29281 031/300 Write block 3216 len 8192 data: 7 5
29282 032/300 Write block 1480 len 36864 data: 9 9 8 7 6 5 5 5 5
29283 033/300 Write block 4639 len 28672 data: 6 5 5 4 6 6 5
29284 034/300 Read block 370 len 20480
29285 035/300 Read block 652 len 12288
29286 036/300 Write block 2590 len 8192 data: 8 7
29287 037/300 Read block 1184 len 24576
29288 038/300 Read block 2691 len 32768
29289 039/300 Read block 3277 len 36864
29290 040/300 Read block 446 len 4096
29291 041/300 Flush
29292 042/300 Flush
29293 043/300 Read block 3878 len 8192
29294 044/300 Write block 650 len 20480 data: 4 3 3 6 6
29295 045/300 Write block 4739 len 20480 data: 7 7 7 6 5
29296 046/300 Write block 4447 len 36864 data: 6 6 6 5 5 4 6 5 5
29297 047/300 Write block 1721 len 8192 data: 5 6
29298 048/300 Read block 4399 len 32768
29299 049/300 Read block 965 len 20480
29300 050/300 Read block 202 len 8192
29301 051/300 Read block 175 len 40960
29302 052/300 Write block 1136 len 12288 data: 5 6 5
29303 053/300 Flush
29304 054/300 Write block 4062 len 20480 data: 5 5 4 4 4
29305 055/300 Write block 4536 len 12288 data: 6 5 4
29306 056/300 Read block 1085 len 32768
29307 057/300 Write block 1827 len 4096 data: 4
29308 058/300 Read block 413 len 4096
29309 059/300 Read block 4938 len 28672
29310 060/300 Read block 3651 len 32768
29311 061/300 Read block 480 len 20480
29312 062/300 Write block 4603 len 12288 data: 7 5 5
29313 063/300 Write block 4646 len 40960 data: 6 7 6 6 4 5 5 5 5 5
29314 064/300 Read block 3045 len 20480
29315 065/300 Read block 309 len 40960
29316 066/300 Read block 4625 len 40960
29317 067/300 Read block 4535 len 36864
29318 068/300 Read block 709 len 4096
29319 069/300 Flush
29320 070/300 Write block 2627 len 36864 data: 5 6 6 5 5 6 8 6 6
29321 071/300 Flush
29322 072/300 Read block 1498 len 20480
29323 073/300 Read block 1020 len 24576
29324 074/300 Write block 4292 len 36864 data: 5 4 5 6 4 6 6 6 4
29325 075/300 Read block 4217 len 40960
29326 076/300 Read block 585 len 28672
29327 077/300 Read block 4697 len 8192
29328 078/300 Read block 1182 len 20480
29329 079/300 Read block 566 len 12288
29330 080/300 Read block 3112 len 32768
29331 081/300 Read block 2485 len 16384
29332 082/300 Read block 325 len 28672
29333 083/300 Write block 3629 len 12288 data: 4 4 4
29334 084/300 Write block 1854 len 4096 data: 8
29335 085/300 Read block 3440 len 36864
29336 086/300 Read block 445 len 28672
29337 087/300 Read block 1077 len 28672
29338 088/300 Write block 4619 len 4096 data: 7
29339 089/300 Read block 2314 len 36864
29340 090/300 Read block 235 len 20480
29341 091/300 Write block 2722 len 16384 data: 6 4 4 4
29342 092/300 Write block 358 len 36864 data: 6 4 4 5 5 5 7 6 6
29343 093/300 Read block 1547 len 16384
29344 094/300 Write block 2179 len 4096 data: 6
29345 095/300 Read block 3087 len 32768
29346 096/300 Write block 3485 len 12288 data: 4 4 3
29347 097/300 Flush
29348 098/300 Read block 1207 len 20480
29349 099/300 Read block 3216 len 20480
29350 100/300 Read block 2249 len 24576
29351 101/300 Read block 4113 len 24576
29352 102/300 Read block 73 len 24576
29353 103/300 Flush
29354 104/300 Read block 1119 len 12288
29355 105/300 Read block 2181 len 28672
29356 106/300 Read block 4978 len 16384
29357 107/300 Read block 2453 len 8192
29358 108/300 Read block 697 len 40960
29359 109/300 Read block 4614 len 32768
29360 110/300 Write block 3864 len 20480 data: 4 4 5 5 5
29361 111/300 Write block 1073 len 28672 data: 7 6 6 5 5 4 6
29362 112/300 Write block 2014 len 4096 data: 7
29363 113/300 Read block 1435 len 28672
29364 114/300 Read block 2992 len 4096
29365 115/300 Read block 4168 len 4096
29366 116/300 Read block 2255 len 32768
29367 117/300 Read block 1651 len 16384
29368 118/300 Read block 1811 len 12288
29369 119/300 Read block 1775 len 24576
29370 120/300 Read block 1281 len 12288
29371 121/300 Read block 3673 len 4096
29372 122/300 Write block 4440 len 4096 data: 5
29373 123/300 Write block 517 len 24576 data: 6 6 8 8 7 6
29374 124/300 Write block 3082 len 40960 data: 5 4 4 3 3 4 5 4 6 7
29375 125/300 Flush
29376 126/300 Read block 2932 len 8192
29377 127/300 Read block 3100 len 20480
29378 128/300 Write block 783 len 32768 data: 3 4 4 6 6 5 6 7
29379 129/300 Read block 979 len 32768
29380 130/300 Read block 1081 len 24576
29381 131/300 Write block 3292 len 16384 data: 5 6 6 6
29382 132/300 Read block 4556 len 12288
29383 133/300 Read block 1273 len 32768
29384 134/300 Write block 3302 len 28672 data: 4 4 4 4 4 3 3
29385 135/300 Write block 4366 len 28672 data: 4 3 4 5 4 4 4
29386 136/300 Write block 2952 len 4096 data: 7
29387 137/300 Flush
29388 138/300 Read block 2683 len 12288
29389 139/300 Write block 703 len 28672 data: 4 5 7 7 7 8 7
29390 140/300 Flush
29391 141/300 Write block 4885 len 36864 data: 5 6 7 5 4 4 3 5 5
29392 142/300 Read block 1957 len 40960
29393 143/300 Write block 2169 len 32768 data: 5 5 7 8 9 8 9 10
29394 144/300 Read block 2414 len 40960
29395 145/300 Write block 4160 len 12288 data: 8 10 10
29396 146/300 Flush
29397 147/300 Write block 2498 len 24576 data: 4 4 3 4 4 3
29398 148/300 Read block 2444 len 8192
29399 149/300 Write block 1368 len 32768 data: 8 8 9 8 6 7 8 9
29400 150/300 Read block 1273 len 36864
29401 151/300 Write block 1461 len 16384 data: 4 2 4 7
29402 152/300 Read block 2266 len 36864
29403 153/300 Read block 3847 len 12288
29404 154/300 Read block 4234 len 40960
29405 155/300 Write block 1453 len 20480 data: 6 6 5 7 8
29406 156/300 Read block 4079 len 16384
29407 157/300 Read block 2363 len 24576
29408 158/300 Read block 164 len 28672
29409 159/300 Read block 2520 len 20480
29410 160/300 Write block 298 len 28672 data: 6 6 7 5 6 6 5
29411 161/300 Write block 127 len 20480 data: 5 5 6 7 9
29412 162/300 Flush
29413 163/300 Flush
29414 164/300 Write block 126 len 40960 data: 6 6 6 7 8 10 7 6 6 6
29415 165/300 Write block 406 len 12288 data: 7 5 5
29416 166/300 Write block 4575 len 8192 data: 4 5
29417 167/300 Read block 2931 len 4096
29418 168/300 Write block 185 len 8192 data: 8 8
29419 169/300 Write block 555 len 8192 data: 5 5
29420 170/300 Flush
29421 171/300 Write block 1146 len 24576 data: 5 6 7 7 5 2
29422 172/300 Read block 3512 len 24576
29423 173/300 Read block 1309 len 12288
29424 174/300 Read block 1265 len 28672
29425 175/300 Write block 1670 len 8192 data: 6 6
29426 176/300 Write block 2494 len 40960 data: 5 4 3 3 5 5 4 5 5 4
29427 177/300 Write block 818 len 36864 data: 5 7 6 5 4 4 4 4 4
29428 178/300 Write block 4355 len 12288 data: 7 4 4
29429 179/300 Read block 1931 len 20480
29430 180/300 Flush
29431 181/300 Read block 2565 len 16384
29432 182/300 Read block 581 len 16384
29433 183/300 Read block 1734 len 36864
29434 184/300 Write block 219 len 20480 data: 5 4 5 7 8
29435 185/300 Read block 2608 len 16384
29436 186/300 Read block 1113 len 12288
29437 187/300 Write block 3888 len 4096 data: 6
29438 188/300 Read block 4983 len 36864
29439 189/300 Write block 1407 len 4096 data: 9
29440 190/300 Read block 861 len 32768
29441 191/300 Read block 3024 len 40960
29442 192/300 Write block 317 len 20480 data: 5 5 5 7 8
29443 193/300 Read block 3140 len 8192
29444 194/300 Write block 102 len 12288 data: 6 5 5
29445 195/300 Read block 4221 len 28672
29446 196/300 Flush
29447 197/300 Write block 768 len 24576 data: 6 6 5 5 5 5
29448 198/300 Write block 1298 len 40960 data: 9 8 7 9 8 10 8 8 7 8
29449 199/300 Read block 258 len 8192
29450 200/300 Write block 562 len 8192 data: 6 6
29451 201/300 Read block 1 len 12288
29452 202/300 Read block 2175 len 28672
29453 203/300 Flush
29454 204/300 Write block 1400 len 4096 data: 10
29455 205/300 Write block 2136 len 8192 data: 4 4
29456 206/300 Write block 2818 len 28672 data: 3 3 4 4 4 4 6
29457 207/300 Write block 3303 len 40960 data: 5 5 5 5 4 4 4 3 3 3
29458 208/300 Read block 111 len 24576
29459 209/300 Read block 3164 len 12288
29460 210/300 Read block 1252 len 40960
29461 211/300 Read block 4341 len 40960
29462 212/300 Write block 3578 len 16384 data: 3 3 4 5
29463 213/300 Read block 1294 len 24576
29464 214/300 Read block 332 len 20480
29465 215/300 Read block 4558 len 16384
29466 216/300 Write block 1361 len 4096 data: 8
29467 217/300 Write block 300 len 24576 data: 8 6 7 7 6 6
29468 218/300 Write block 4698 len 4096 data: 5
29469 219/300 Write block 1262 len 28672 data: 4 5 4 4 4 3 3
29470 220/300 Read block 3667 len 36864
29471 221/300 Read block 4565 len 8192
29472 222/300 Write block 96 len 16384 data: 4 4 5 5
29473 223/300 Read block 1309 len 40960
29474 224/300 Write block 1160 len 28672 data: 3 2 2 3 4 4 4
29475 225/300 Flush
29476 226/300 Write block 4401 len 4096 data: 4
29477 227/300 Write block 2365 len 24576 data: 4 4 4 5 7 8
29478 228/300 Read block 296 len 4096
29479 229/300 Read block 2707 len 16384
29480 230/300 Write block 1780 len 8192 data: 3 4
29481 231/300 Write block 178 len 4096 data: 5
29482 232/300 Flush
29483 233/300 Write block 207 len 12288 data: 7 7 7
29484 234/300 Read block 177 len 8192
29485 235/300 Write block 732 len 4096 data: 5
29486 236/300 Read block 3517 len 16384
29487 237/300 Read block 726 len 36864
29488 238/300 Read block 1669 len 40960
29489 239/300 Read block 2921 len 32768
29490 240/300 Read block 3100 len 28672
29491 241/300 Write block 2215 len 12288 data: 2 3 3
29492 242/300 Read block 9 len 16384
29493 243/300 Read block 2333 len 24576
29494 244/300 Write block 1 len 24576 data: 3 3 2 3 3 3
29495 245/300 Write block 3810 len 32768 data: 8 7 7 4 4 3 3 5
29496 246/300 Write block 2301 len 20480 data: 4 6 8 8 8
29497 247/300 Read block 4019 len 40960
29498 248/300 Write block 642 len 4096 data: 4
29499 249/300 Write block 1185 len 40960 data: 6 5 5 4 5 6 6 5 5 4
29500 250/300 Write block 3038 len 28672 data: 5 5 4 4 3 3 3
29501 251/300 Read block 4103 len 12288
29502 252/300 Read block 4389 len 24576
29503 253/300 Read block 862 len 40960
29504 254/300 Read block 3823 len 20480
29505 255/300 Write block 3388 len 28672 data: 6 7 6 6 5 4 4
29506 256/300 Write block 554 len 8192 data: 8 6
29507 257/300 Write block 4232 len 24576 data: 5 4 4 3 4 5
29508 258/300 Write block 772 len 8192 data: 6 6
29509 259/300 Read block 4232 len 4096
29510 260/300 Write block 1635 len 4096 data: 6
29511 261/300 Read block 1403 len 20480
29512 262/300 Write block 4652 len 16384 data: 6 6 6 6
29513 263/300 Flush
29514 264/300 Flush
29515 265/300 Read block 2253 len 20480
29516 266/300 Read block 3431 len 28672
29517 267/300 Write block 2030 len 16384 data: 7 6 6 4
29518 268/300 Write block 2079 len 32768 data: 8 6 5 5 5 4 3 2
29519 269/300 Write block 3534 len 36864 data: 2 3 3 4 4 5 5 5 6
29520 270/300 Read block 1523 len 8192
29521 271/300 Write block 2624 len 32768 data: 9 6 4 6 7 7 6 6
29522 272/300 Read block 585 len 36864
29523 273/300 Read block 1445 len 32768
29524 274/300 Write block 4233 len 24576 data: 5 5 4 5 6 6
29525 275/300 Read block 3292 len 20480
29526 276/300 Read block 4022 len 16384
29527 277/300 Read block 3127 len 36864
29528 278/300 Read block 337 len 40960
29529 279/300 Write block 4221 len 8192 data: 5 4
29530 280/300 Flush
29531 281/300 Read block 1893 len 16384
29532 282/300 Write block 2844 len 24576 data: 6 7 8 6 7 7
29533 283/300 Flush
29534 284/300 Read block 4045 len 32768
29535 285/300 Flush
29536 286/300 Write block 2826 len 36864 data: 7 6 8 8 9 9 10 9 6
29537 287/300 Write block 3694 len 16384 data: 6 5 6 6
29538 288/300 Read block 485 len 12288
29539 289/300 Write block 1149 len 40960 data: 8 6 3 2 2 3 3 3 3 4
29540 290/300 Write block 2122 len 20480 data: 3 3 5 5 4
29541 291/300 Read block 4043 len 20480
29542 292/300 Read block 4081 len 4096
29543 293/300 Write block 3378 len 40960 data: 4 4 4 3 3 4 4 4 4 5
29544 294/300 Read block 4363 len 20480
29545 295/300 Read block 1274 len 28672
29546 296/300 Write block 126 len 4096 data: 7
29547 297/300 Write block 1093 len 16384 data: 5 5 6 6
29548 298/300 Read block 3070 len 32768
29549 299/300 Write block 1435 len 36864 data: 3 3 3 5 5 5 6 6 5
29550 300/300 Flush
29551 301/300 Write block 4689 len 4096 data: 5
29552 Main task has work to do, go find it
29553 got dsc Start(2)
29554 start 2
29555 Jan 27 19:42:35.593 INFO request completed, latency_us: 253, response_code: 204, uri: /start/cid/2, method: POST, req_id: a45320f6-6c30-4a61-a1a3-5e57a492f63b, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
29556 Replay: started 2, returned:Ok(())
29557 ----------------------------------------------------------------
29558 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
29559 States: Active Active Offline
29560 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29561 8848 Acked Read 9 Done Done Sent false
29562 8849 Acked Write 6 Done Done Sent false
29563 8850 Acked Read 10 Done Done Sent false
29564 8851 Acked Read 4 Done Done Sent false
29565 8852 Acked Write 1 Done Done Sent false
29566 8853 Acked Write 5 Done Done Sent false
29567 8854 Acked Write 7 Done Done Sent false
29568 8855 Acked Read 5 Done Done Sent false
29569 8856 Acked Read 5 Done Done Sent false
29570 8857 Acked Read 6 Done Done Sent false
29571 8858 Acked Read 6 Done Done Sent false
29572 8859 Acked Read 6 Done Done Sent false
29573 8860 Acked Write 9 Done Done Sent false
29574 8861 Acked Write 7 Done Done Sent false
29575 8862 Acked Read 10 Done Done Sent false
29576 8863 Acked Read 8 Done Done Sent false
29577 8864 Acked Flush 0 Done Done Sent false
29578 8865 Acked Write 7 Done Done Sent false
29579 8866 Acked Write 9 Done Done Sent false
29580 8867 Acked Write 6 Done Done Sent false
29581 8868 Acked Read 7 Done Done Sent false
29582 8869 Acked Write 5 Done Done Sent false
29583 8870 Acked Write 9 Done Done Sent false
29584 8871 Acked Write 9 Done Done Sent false
29585 8872 Acked Read 3 Done Done Sent false
29586 8873 Acked Write 3 Done Done Sent false
29587 8874 Acked Read 6 Done Done Sent false
29588 8875 Acked Read 6 Done Done Sent false
29589 8876 Acked Read 8 Done Done Sent false
29590 8877 Acked Write 8 Done Done Sent false
29591 8878 Acked Write 2 Done Done Sent false
29592 8879 Acked Write 9 Done Done Sent false
29593 8880 Acked Write 7 Done Done Sent false
29594 8881 Acked Read 5 Done Done Sent false
29595 8882 Acked Read 3 Done Done Sent false
29596 8883 Acked Write 2 Done Done Sent false
29597 8884 Acked Read 6 Done Done Sent false
29598 8885 Acked Read 8 Done Done Sent false
29599 8886 Acked Read 9 Done Done Sent false
29600 8887 Acked Read 1 Done Done Sent false
29601 8888 Acked Flush 0 Done Done Sent false
29602 8889 Acked Flush 0 Done Done Sent false
29603 8890 Acked Read 2 Done Done Sent false
29604 8891 Acked Write 5 Done Done Sent false
29605 8892 Acked Write 5 Done Done Sent false
29606 8893 Acked Write 9 Done Done Sent false
29607 8894 Acked Write 2 Done Done Sent false
29608 8895 Acked Read 8 Done Done Sent false
29609 8896 Acked Read 5 Done Done Sent false
29610 8897 Acked Read 2 Done Done Sent false
29611 8898 Acked Read 10 Done Done Sent false
29612 8899 Acked Write 3 Done Done Sent false
29613 8900 Acked Flush 0 Done Done Sent false
29614 8901 Acked Write 5 Done Done Sent false
29615 8902 Acked Write 3 Done Done Sent false
29616 8903 Acked Read 8 Done Done Sent false
29617 8904 Acked Write 1 Done Done Sent false
29618 8905 Acked Read 1 Done Done Sent false
29619 8906 Acked Read 7 Done Done Sent false
29620 8907 Acked Read 8 Done Done Sent false
29621 8908 Acked Read 5 Done Done Sent false
29622 8909 Acked Write 3 Done Done Sent false
29623 8910 Acked Write 10 Done Done Sent false
29624 8911 Acked Read 5 Done Done Sent false
29625 8912 Acked Read 10 Done Done Sent false
29626 8913 Acked Read 10 Done Done Sent false
29627 8914 Acked Read 9 Done Done Sent false
29628 8915 Acked Read 1 Done Done Sent false
29629 8916 Acked Flush 0 Done Done Sent false
29630 8917 Acked Write 9 Done Done Sent false
29631 8918 Acked Flush 0 Done Done Sent false
29632 8919 Acked Read 5 Done Done Sent false
29633 8920 Acked Read 6 Done Done Sent false
29634 8921 Acked Write 9 Done Done Sent false
29635 8922 Acked Read 10 Done Done Sent false
29636 8923 Acked Read 7 Done Done Sent false
29637 8924 Acked Read 2 Done Done Sent false
29638 8925 Acked Read 5 Done Done Sent false
29639 8926 Acked Read 3 Done Done Sent false
29640 8927 Acked Read 8 Done Done Sent false
29641 8928 Acked Read 4 Done Done Sent false
29642 8929 Acked Read 7 Done Done Sent false
29643 8930 Acked Write 3 Done Done Sent false
29644 8931 Acked Write 1 Done Done Sent false
29645 8932 Acked Read 9 Done Done Sent false
29646 8933 Acked Read 7 Done Done Sent false
29647 8934 Acked Read 7 Done Done Sent false
29648 8935 Acked Write 1 Done Done Sent false
29649 8936 Acked Read 9 Done Done Sent false
29650 8937 Acked Read 5 Done Done Sent false
29651 8938 Acked Write 4 Done Done Sent false
29652 8939 Acked Write 9 Done Done Sent false
29653 8940 Acked Read 4 Done Done Sent false
29654 8941 Acked Write 1 Done Done Sent false
29655 8942 Acked Read 8 Done Done Sent false
29656 8943 Acked Write 3 Done Done Sent false
29657 8944 Acked Flush 0 Done Done Sent false
29658 8945 Acked Read 5 Done Done Sent false
29659 8946 Acked Read 5 Done Done Sent false
29660 8947 Acked Read 6 Done Done Sent false
29661 8948 Acked Read 6 Done Done Sent false
29662 8949 Acked Read 6 Done Done Sent false
29663 8950 Acked Flush 0 Done Done Sent false
29664 8951 Acked Read 3 Done Done Sent false
29665 8952 Acked Read 7 Done Done Sent false
29666 8953 Acked Read 4 Done Done Sent false
29667 8954 Acked Read 2 Done Done Sent false
29668 8955 Acked Read 10 Done Done Sent false
29669 8956 Acked Read 8 Done Done Sent false
29670 8957 Acked Write 5 Done Done Sent false
29671 8958 Acked Write 7 Done Done Sent false
29672 8959 Acked Write 1 Done Done Sent false
29673 8960 Acked Read 7 Done Done Sent false
29674 8961 Acked Read 1 Done Done Sent false
29675 8962 Acked Read 1 Done Done Sent false
29676 8963 Acked Read 8 Done Done Sent false
29677 8964 Acked Read 4 Done Done Sent false
29678 8965 Acked Read 3 Done Done Sent false
29679 8966 Acked Read 6 Done Done Sent false
29680 8967 Acked Read 3 Done Done Sent false
29681 8968 Acked Read 1 Done Done Sent false
29682 8969 Acked Write 1 Done Done Sent false
29683 8970 Acked Write 6 Done Done Sent false
29684 8971 Acked Write 10 Done Done Sent false
29685 8972 Acked Flush 0 Done Done Sent false
29686 8973 Acked Read 2 Done Done Sent false
29687 8974 Acked Read 5 Done Done Sent false
29688 8975 Acked Write 8 Done Done Sent false
29689 8976 Acked Read 8 Done Done Sent false
29690 8977 Acked Read 6 Done Done Sent false
29691 8978 Acked Write 4 Done Done Sent false
29692 8979 Acked Read 3 Done Done Sent false
29693 8980 Acked Read 8 Done Done Sent false
29694 8981 Acked Write 7 Done Done Sent false
29695 8982 Acked Write 7 Done Done Sent false
29696 8983 Acked Write 1 Done Done Sent false
29697 8984 Acked Flush 0 Done Done Sent false
29698 8985 Acked Read 3 Done Done Sent false
29699 8986 Acked Write 7 Done Done Sent false
29700 8987 Acked Flush 0 Done Done Sent false
29701 8988 Acked Write 9 Done Done Sent false
29702 8989 Acked Read 10 Done Done Sent false
29703 8990 Acked Write 8 Done Done Sent false
29704 8991 Acked Read 10 Done Done Sent false
29705 8992 Acked Write 3 Done Done Sent false
29706 8993 Acked Flush 0 Done Done Sent false
29707 8994 Acked Write 6 Done Done Sent false
29708 8995 Acked Read 2 Done Done Sent false
29709 8996 Acked Write 8 Done Done Sent false
29710 8997 Acked Read 9 Done Done Sent false
29711 8998 Acked Write 4 Done Done Sent false
29712 8999 Acked Read 9 Done Done Sent false
29713 9000 Acked Read 3 Done Done Sent false
29714 9001 Acked Read 10 Done Done Sent false
29715 9002 Acked Write 5 Done Done Sent false
29716 9003 Acked Read 4 Done Done Sent false
29717 9004 Acked Read 6 Done Done Sent false
29718 9005 Acked Read 7 Done Done Sent false
29719 9006 Acked Read 5 Done Done Sent false
29720 9007 Acked Write 7 Done Done Sent false
29721 9008 Acked Write 5 Done Done Sent false
29722 9009 Acked Flush 0 Done Done Sent false
29723 9010 Acked Flush 0 Done Done Sent false
29724 9011 Acked Write 10 Done Done Sent false
29725 9012 Acked Write 3 Done Done Sent false
29726 9013 Acked Write 2 Done Done Sent false
29727 9014 Acked Read 1 Done Done Sent false
29728 9015 Acked Write 2 Done Done Sent false
29729 9016 Acked Write 2 Done Done Sent false
29730 9017 Acked Flush 0 Done Done Sent false
29731 9018 Acked Write 6 Done Done Sent false
29732 9019 Acked Read 6 Done Done Sent false
29733 9020 Acked Read 3 Done Done Sent false
29734 9021 Acked Read 7 Done Done Sent false
29735 9022 Acked Write 2 Done Done Sent false
29736 9023 Acked Write 10 Done Done Sent false
29737 9024 Acked Write 9 Done Done Sent false
29738 9025 Acked Write 3 Done Done Sent false
29739 9026 Acked Read 5 Done Done Sent false
29740 9027 Acked Flush 0 Done Done Sent false
29741 9028 Acked Read 4 Done Done Sent false
29742 9029 Acked Read 4 Done Done Sent false
29743 9030 Acked Read 9 Done Done Sent false
29744 9031 Acked Write 5 Done Done Sent false
29745 9032 Acked Read 4 Done Done Sent false
29746 9033 Acked Read 3 Done Done Sent false
29747 9034 Acked Write 1 Done Done Sent false
29748 9035 Acked Read 9 Done Done Sent false
29749 9036 Acked Write 1 Done Done Sent false
29750 9037 Acked Read 8 Done Done Sent false
29751 9038 Acked Read 10 Done Done Sent false
29752 9039 Acked Write 5 Done Done Sent false
29753 9040 Acked Read 2 Done Done Sent false
29754 9041 Acked Write 3 Done Done Sent false
29755 9042 Acked Read 7 Done Done Sent false
29756 9043 Acked Flush 0 Done Done Sent false
29757 9044 Acked Write 6 Done Done Sent false
29758 9045 Acked Write 10 Done Done Sent false
29759 9046 Acked Read 2 Done Done Sent false
29760 9047 Acked Write 2 Done Done Sent false
29761 9048 Acked Read 3 Done Done Sent false
29762 9049 Acked Read 7 Done Done Sent false
29763 9050 Acked Flush 0 Done Done Sent false
29764 9051 Acked Write 1 Done Done Sent false
29765 9052 Acked Write 2 Done Done Sent false
29766 9053 Acked Write 7 Done Done Sent false
29767 9054 Acked Write 10 Done Done Sent false
29768 9055 Acked Read 6 Done Done Sent false
29769 9056 Acked Read 3 Done Done Sent false
29770 9057 Acked Read 10 Done Done Sent false
29771 9058 Acked Read 10 Done Done Sent false
29772 9059 Acked Write 4 Done Done Sent false
29773 9060 Acked Read 6 Done Done Sent false
29774 9061 Acked Read 5 Done Done Sent false
29775 9062 Acked Read 4 Done Done Sent false
29776 9063 Acked Write 1 Done Done Sent false
29777 9064 Acked Write 6 Done Done Sent false
29778 9065 Acked Write 1 Done Done Sent false
29779 9066 Acked Write 7 Done Done Sent false
29780 9067 Acked Read 9 Done Done Sent false
29781 9068 Acked Read 2 Done Done Sent false
29782 9069 Acked Write 4 Done Done Sent false
29783 9070 Acked Read 10 Done Done Sent false
29784 9071 Acked Write 7 Done Done Sent false
29785 9072 Acked Flush 0 Done Done Sent false
29786 9073 Acked Write 1 Done Done Sent false
29787 9074 Acked Write 6 Done Done Sent false
29788 9075 Acked Read 1 Done Done Sent false
29789 9076 Acked Read 4 Done Done Sent false
29790 9077 Acked Write 2 Done Done Sent false
29791 9078 Acked Write 1 Done Done Sent false
29792 9079 Acked Flush 0 Done Done Sent false
29793 9080 Acked Write 3 Done Done Sent false
29794 9081 Acked Read 2 Done Done Sent false
29795 9082 Acked Write 1 Done Done Sent false
29796 9083 Acked Read 4 Done Done Sent false
29797 9084 Acked Read 9 Done Done Sent false
29798 9085 Acked Read 10 Done Done Sent false
29799 9086 Acked Read 8 Done Done Sent false
29800 9087 Acked Read 7 Done Done Sent false
29801 9088 Acked Write 3 Done Done Sent false
29802 9089 Acked Read 4 Done Done Sent false
29803 9090 Acked Read 6 Done Done Sent false
29804 9091 Acked Write 6 Done Done Sent false
29805 9092 Acked Write 8 Done Done Sent false
29806 9093 Acked Write 5 Done Done Sent false
29807 9094 Acked Read 10 Done Done Sent false
29808 9095 Acked Write 1 Done Done Sent false
29809 9096 Acked Write 10 Done Done Sent false
29810 9097 Acked Write 7 Done Done Sent false
29811 9098 Acked Read 3 Done Done Sent false
29812 9099 Acked Read 6 Done Done Sent false
29813 9100 Acked Read 10 Done Done Sent false
29814 9101 Acked Read 5 Done Done Sent false
29815 9102 Acked Write 7 Done Done Sent false
29816 9103 Acked Write 2 Done Done Sent false
29817 9104 Acked Write 6 Done Done Sent false
29818 9105 Acked Write 2 Done Done Sent false
29819 9106 Acked Read 1 Done Done Sent false
29820 9107 Acked Write 1 Done Done Sent false
29821 9108 Acked Read 5 Done Done Sent false
29822 9109 Acked Write 4 Done Done Sent false
29823 9110 Acked Flush 0 Done Done Sent false
29824 9111 Acked Flush 0 Done Done Sent false
29825 9112 Acked Read 5 Done Done Sent false
29826 9113 Acked Read 7 Done Done Sent false
29827 9114 Acked Write 4 Done Done Sent false
29828 9115 Acked Write 8 Done Done Sent false
29829 9116 Acked Write 9 Done Done Sent false
29830 9117 Acked Read 2 Done Done Sent false
29831 9118 Acked Write 8 Done Done Sent false
29832 9119 Acked Read 9 Done Done Sent false
29833 9120 Acked Read 8 Done Done Sent false
29834 9121 Acked Write 6 Done Done Sent false
29835 9122 Acked Read 5 Done Done Sent false
29836 9123 Acked Read 4 Done Done Sent false
29837 9124 Acked Read 9 Done Done Sent false
29838 9125 Acked Read 10 Done Done Sent false
29839 9126 Acked Write 2 Done Done Sent false
29840 9127 Acked Flush 0 Done Done Sent false
29841 9128 Acked Read 4 Done Done Sent false
29842 9129 Acked Write 6 Done Done Sent false
29843 9130 Acked Flush 0 Done Done Sent false
29844 9131 Acked Read 8 Done Done Sent false
29845 9132 Acked Flush 0 Done Done Sent false
29846 9133 Acked Write 9 Done Done Sent false
29847 9134 Acked Write 4 Done Done Sent false
29848 9135 Acked Read 3 Done Done Sent false
29849 9136 Acked Write 10 Done Done Sent false
29850 9137 Acked Write 5 Done Done Sent false
29851 9138 Acked Read 5 Done Done Sent false
29852 9139 Acked Read 1 Done Done Sent false
29853 9140 Acked Write 10 Done Done Sent false
29854 9141 Acked Read 5 Done Done Sent false
29855 9142 Acked Read 7 Done Done Sent false
29856 9143 Acked Write 1 Done Done Sent false
29857 9144 Acked Write 4 Done Done Sent false
29858 9145 Acked Read 8 Done Done Sent false
29859 9146 Acked Write 9 Done Done Sent false
29860 9147 Acked Flush 0 Done Done Sent false
29861 9148 Acked Write 1 Done Done Sent false
29862 STATES DS:0 DS:1 DS:2 TOTAL
29863 Sent 0 0 301 301
29864 Done 301 301 0 602
29865 Skipped 0 0 0 0
29866 Error 0 0 0 0
29867 Last Flush: 9147 9147 8847
29868 Downstairs last five completed: 8847 8846 8845 8844 8843
29869 Upstairs last five completed: 9148 9147 9146 9145 9144
29870 CLIENT: Up:0 ds:301 act:2
29871 [8830] got start action
29872 [8830] I am going to restart
29873 Starting downstairs at port 8830
29874 Make output file at "/tmp/dsc/downstairs-8830.txt"
29875 [8830][2] reports Starting
29876 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1227)
29877 [8830][2] reports Running
29878 ----------------------------------------------------------------
29879 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
29880 States: Active Active Offline
29881 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
29882 8848 Acked Read 9 Done Done Sent false
29883 8849 Acked Write 6 Done Done Sent false
29884 8850 Acked Read 10 Done Done Sent false
29885 8851 Acked Read 4 Done Done Sent false
29886 8852 Acked Write 1 Done Done Sent false
29887 8853 Acked Write 5 Done Done Sent false
29888 8854 Acked Write 7 Done Done Sent false
29889 8855 Acked Read 5 Done Done Sent false
29890 8856 Acked Read 5 Done Done Sent false
29891 8857 Acked Read 6 Done Done Sent false
29892 8858 Acked Read 6 Done Done Sent false
29893 8859 Acked Read 6 Done Done Sent false
29894 8860 Acked Write 9 Done Done Sent false
29895 8861 Acked Write 7 Done Done Sent false
29896 8862 Acked Read 10 Done Done Sent false
29897 8863 Acked Read 8 Done Done Sent false
29898 8864 Acked Flush 0 Done Done Sent false
29899 8865 Acked Write 7 Done Done Sent false
29900 8866 Acked Write 9 Done Done Sent false
29901 8867 Acked Write 6 Done Done Sent false
29902 8868 Acked Read 7 Done Done Sent false
29903 8869 Acked Write 5 Done Done Sent false
29904 8870 Acked Write 9 Done Done Sent false
29905 8871 Acked Write 9 Done Done Sent false
29906 8872 Acked Read 3 Done Done Sent false
29907 8873 Acked Write 3 Done Done Sent false
29908 8874 Acked Read 6 Done Done Sent false
29909 8875 Acked Read 6 Done Done Sent false
29910 8876 Acked Read 8 Done Done Sent false
29911 8877 Acked Write 8 Done Done Sent false
29912 8878 Acked Write 2 Done Done Sent false
29913 8879 Acked Write 9 Done Done Sent false
29914 8880 Acked Write 7 Done Done Sent false
29915 8881 Acked Read 5 Done Done Sent false
29916 8882 Acked Read 3 Done Done Sent false
29917 8883 Acked Write 2 Done Done Sent false
29918 8884 Acked Read 6 Done Done Sent false
29919 8885 Acked Read 8 Done Done Sent false
29920 8886 Acked Read 9 Done Done Sent false
29921 8887 Acked Read 1 Done Done Sent false
29922 8888 Acked Flush 0 Done Done Sent false
29923 8889 Acked Flush 0 Done Done Sent false
29924 8890 Acked Read 2 Done Done Sent false
29925 8891 Acked Write 5 Done Done Sent false
29926 8892 Acked Write 5 Done Done Sent false
29927 8893 Acked Write 9 Done Done Sent false
29928 8894 Acked Write 2 Done Done Sent false
29929 8895 Acked Read 8 Done Done Sent false
29930 8896 Acked Read 5 Done Done Sent false
29931 8897 Acked Read 2 Done Done Sent false
29932 8898 Acked Read 10 Done Done Sent false
29933 8899 Acked Write 3 Done Done Sent false
29934 8900 Acked Flush 0 Done Done Sent false
29935 8901 Acked Write 5 Done Done Sent false
29936 8902 Acked Write 3 Done Done Sent false
29937 8903 Acked Read 8 Done Done Sent false
29938 8904 Acked Write 1 Done Done Sent false
29939 8905 Acked Read 1 Done Done Sent false
29940 8906 Acked Read 7 Done Done Sent false
29941 8907 Acked Read 8 Done Done Sent false
29942 8908 Acked Read 5 Done Done Sent false
29943 8909 Acked Write 3 Done Done Sent false
29944 8910 Acked Write 10 Done Done Sent false
29945 8911 Acked Read 5 Done Done Sent false
29946 8912 Acked Read 10 Done Done Sent false
29947 8913 Acked Read 10 Done Done Sent false
29948 8914 Acked Read 9 Done Done Sent false
29949 8915 Acked Read 1 Done Done Sent false
29950 8916 Acked Flush 0 Done Done Sent false
29951 8917 Acked Write 9 Done Done Sent false
29952 8918 Acked Flush 0 Done Done Sent false
29953 8919 Acked Read 5 Done Done Sent false
29954 8920 Acked Read 6 Done Done Sent false
29955 8921 Acked Write 9 Done Done Sent false
29956 8922 Acked Read 10 Done Done Sent false
29957 8923 Acked Read 7 Done Done Sent false
29958 8924 Acked Read 2 Done Done Sent false
29959 8925 Acked Read 5 Done Done Sent false
29960 8926 Acked Read 3 Done Done Sent false
29961 8927 Acked Read 8 Done Done Sent false
29962 8928 Acked Read 4 Done Done Sent false
29963 8929 Acked Read 7 Done Done Sent false
29964 8930 Acked Write 3 Done Done Sent false
29965 8931 Acked Write 1 Done Done Sent false
29966 8932 Acked Read 9 Done Done Sent false
29967 8933 Acked Read 7 Done Done Sent false
29968 8934 Acked Read 7 Done Done Sent false
29969 8935 Acked Write 1 Done Done Sent false
29970 8936 Acked Read 9 Done Done Sent false
29971 8937 Acked Read 5 Done Done Sent false
29972 8938 Acked Write 4 Done Done Sent false
29973 8939 Acked Write 9 Done Done Sent false
29974 8940 Acked Read 4 Done Done Sent false
29975 8941 Acked Write 1 Done Done Sent false
29976 8942 Acked Read 8 Done Done Sent false
29977 8943 Acked Write 3 Done Done Sent false
29978 8944 Acked Flush 0 Done Done Sent false
29979 8945 Acked Read 5 Done Done Sent false
29980 8946 Acked Read 5 Done Done Sent false
29981 8947 Acked Read 6 Done Done Sent false
29982 8948 Acked Read 6 Done Done Sent false
29983 8949 Acked Read 6 Done Done Sent false
29984 8950 Acked Flush 0 Done Done Sent false
29985 8951 Acked Read 3 Done Done Sent false
29986 8952 Acked Read 7 Done Done Sent false
29987 8953 Acked Read 4 Done Done Sent false
29988 8954 Acked Read 2 Done Done Sent false
29989 8955 Acked Read 10 Done Done Sent false
29990 8956 Acked Read 8 Done Done Sent false
29991 8957 Acked Write 5 Done Done Sent false
29992 8958 Acked Write 7 Done Done Sent false
29993 8959 Acked Write 1 Done Done Sent false
29994 8960 Acked Read 7 Done Done Sent false
29995 8961 Acked Read 1 Done Done Sent false
29996 8962 Acked Read 1 Done Done Sent false
29997 8963 Acked Read 8 Done Done Sent false
29998 8964 Acked Read 4 Done Done Sent false
29999 8965 Acked Read 3 Done Done Sent false
30000 8966 Acked Read 6 Done Done Sent false
30001 8967 Acked Read 3 Done Done Sent false
30002 8968 Acked Read 1 Done Done Sent false
30003 8969 Acked Write 1 Done Done Sent false
30004 8970 Acked Write 6 Done Done Sent false
30005 8971 Acked Write 10 Done Done Sent false
30006 8972 Acked Flush 0 Done Done Sent false
30007 8973 Acked Read 2 Done Done Sent false
30008 8974 Acked Read 5 Done Done Sent false
30009 8975 Acked Write 8 Done Done Sent false
30010 8976 Acked Read 8 Done Done Sent false
30011 8977 Acked Read 6 Done Done Sent false
30012 8978 Acked Write 4 Done Done Sent false
30013 8979 Acked Read 3 Done Done Sent false
30014 8980 Acked Read 8 Done Done Sent false
30015 8981 Acked Write 7 Done Done Sent false
30016 8982 Acked Write 7 Done Done Sent false
30017 8983 Acked Write 1 Done Done Sent false
30018 8984 Acked Flush 0 Done Done Sent false
30019 8985 Acked Read 3 Done Done Sent false
30020 8986 Acked Write 7 Done Done Sent false
30021 8987 Acked Flush 0 Done Done Sent false
30022 8988 Acked Write 9 Done Done Sent false
30023 8989 Acked Read 10 Done Done Sent false
30024 8990 Acked Write 8 Done Done Sent false
30025 8991 Acked Read 10 Done Done Sent false
30026 8992 Acked Write 3 Done Done Sent false
30027 8993 Acked Flush 0 Done Done Sent false
30028 8994 Acked Write 6 Done Done Sent false
30029 8995 Acked Read 2 Done Done Sent false
30030 8996 Acked Write 8 Done Done Sent false
30031 8997 Acked Read 9 Done Done Sent false
30032 8998 Acked Write 4 Done Done Sent false
30033 8999 Acked Read 9 Done Done Sent false
30034 9000 Acked Read 3 Done Done Sent false
30035 9001 Acked Read 10 Done Done Sent false
30036 9002 Acked Write 5 Done Done Sent false
30037 9003 Acked Read 4 Done Done Sent false
30038 9004 Acked Read 6 Done Done Sent false
30039 9005 Acked Read 7 Done Done Sent false
30040 9006 Acked Read 5 Done Done Sent false
30041 9007 Acked Write 7 Done Done Sent false
30042 9008 Acked Write 5 Done Done Sent false
30043 9009 Acked Flush 0 Done Done Sent false
30044 9010 Acked Flush 0 Done Done Sent false
30045 9011 Acked Write 10 Done Done Sent false
30046 9012 Acked Write 3 Done Done Sent false
30047 9013 Acked Write 2 Done Done Sent false
30048 9014 Acked Read 1 Done Done Sent false
30049 9015 Acked Write 2 Done Done Sent false
30050 9016 Acked Write 2 Done Done Sent false
30051 9017 Acked Flush 0 Done Done Sent false
30052 9018 Acked Write 6 Done Done Sent false
30053 9019 Acked Read 6 Done Done Sent false
30054 9020 Acked Read 3 Done Done Sent false
30055 9021 Acked Read 7 Done Done Sent false
30056 9022 Acked Write 2 Done Done Sent false
30057 9023 Acked Write 10 Done Done Sent false
30058 9024 Acked Write 9 Done Done Sent false
30059 9025 Acked Write 3 Done Done Sent false
30060 9026 Acked Read 5 Done Done Sent false
30061 9027 Acked Flush 0 Done Done Sent false
30062 9028 Acked Read 4 Done Done Sent false
30063 9029 Acked Read 4 Done Done Sent false
30064 9030 Acked Read 9 Done Done Sent false
30065 9031 Acked Write 5 Done Done Sent false
30066 9032 Acked Read 4 Done Done Sent false
30067 9033 Acked Read 3 Done Done Sent false
30068 9034 Acked Write 1 Done Done Sent false
30069 9035 Acked Read 9 Done Done Sent false
30070 9036 Acked Write 1 Done Done Sent false
30071 9037 Acked Read 8 Done Done Sent false
30072 9038 Acked Read 10 Done Done Sent false
30073 9039 Acked Write 5 Done Done Sent false
30074 9040 Acked Read 2 Done Done Sent false
30075 9041 Acked Write 3 Done Done Sent false
30076 9042 Acked Read 7 Done Done Sent false
30077 9043 Acked Flush 0 Done Done Sent false
30078 9044 Acked Write 6 Done Done Sent false
30079 9045 Acked Write 10 Done Done Sent false
30080 9046 Acked Read 2 Done Done Sent false
30081 9047 Acked Write 2 Done Done Sent false
30082 9048 Acked Read 3 Done Done Sent false
30083 9049 Acked Read 7 Done Done Sent false
30084 9050 Acked Flush 0 Done Done Sent false
30085 9051 Acked Write 1 Done Done Sent false
30086 9052 Acked Write 2 Done Done Sent false
30087 9053 Acked Write 7 Done Done Sent false
30088 9054 Acked Write 10 Done Done Sent false
30089 9055 Acked Read 6 Done Done Sent false
30090 9056 Acked Read 3 Done Done Sent false
30091 9057 Acked Read 10 Done Done Sent false
30092 9058 Acked Read 10 Done Done Sent false
30093 9059 Acked Write 4 Done Done Sent false
30094 9060 Acked Read 6 Done Done Sent false
30095 9061 Acked Read 5 Done Done Sent false
30096 9062 Acked Read 4 Done Done Sent false
30097 9063 Acked Write 1 Done Done Sent false
30098 9064 Acked Write 6 Done Done Sent false
30099 9065 Acked Write 1 Done Done Sent false
30100 9066 Acked Write 7 Done Done Sent false
30101 9067 Acked Read 9 Done Done Sent false
30102 9068 Acked Read 2 Done Done Sent false
30103 9069 Acked Write 4 Done Done Sent false
30104 9070 Acked Read 10 Done Done Sent false
30105 9071 Acked Write 7 Done Done Sent false
30106 9072 Acked Flush 0 Done Done Sent false
30107 9073 Acked Write 1 Done Done Sent false
30108 9074 Acked Write 6 Done Done Sent false
30109 9075 Acked Read 1 Done Done Sent false
30110 9076 Acked Read 4 Done Done Sent false
30111 9077 Acked Write 2 Done Done Sent false
30112 9078 Acked Write 1 Done Done Sent false
30113 9079 Acked Flush 0 Done Done Sent false
30114 9080 Acked Write 3 Done Done Sent false
30115 9081 Acked Read 2 Done Done Sent false
30116 9082 Acked Write 1 Done Done Sent false
30117 9083 Acked Read 4 Done Done Sent false
30118 9084 Acked Read 9 Done Done Sent false
30119 9085 Acked Read 10 Done Done Sent false
30120 9086 Acked Read 8 Done Done Sent false
30121 9087 Acked Read 7 Done Done Sent false
30122 9088 Acked Write 3 Done Done Sent false
30123 9089 Acked Read 4 Done Done Sent false
30124 9090 Acked Read 6 Done Done Sent false
30125 9091 Acked Write 6 Done Done Sent false
30126 9092 Acked Write 8 Done Done Sent false
30127 9093 Acked Write 5 Done Done Sent false
30128 9094 Acked Read 10 Done Done Sent false
30129 9095 Acked Write 1 Done Done Sent false
30130 9096 Acked Write 10 Done Done Sent false
30131 9097 Acked Write 7 Done Done Sent false
30132 9098 Acked Read 3 Done Done Sent false
30133 9099 Acked Read 6 Done Done Sent false
30134 9100 Acked Read 10 Done Done Sent false
30135 9101 Acked Read 5 Done Done Sent false
30136 9102 Acked Write 7 Done Done Sent false
30137 9103 Acked Write 2 Done Done Sent false
30138 9104 Acked Write 6 Done Done Sent false
30139 9105 Acked Write 2 Done Done Sent false
30140 9106 Acked Read 1 Done Done Sent false
30141 9107 Acked Write 1 Done Done Sent false
30142 9108 Acked Read 5 Done Done Sent false
30143 9109 Acked Write 4 Done Done Sent false
30144 9110 Acked Flush 0 Done Done Sent false
30145 9111 Acked Flush 0 Done Done Sent false
30146 9112 Acked Read 5 Done Done Sent false
30147 9113 Acked Read 7 Done Done Sent false
30148 9114 Acked Write 4 Done Done Sent false
30149 9115 Acked Write 8 Done Done Sent false
30150 9116 Acked Write 9 Done Done Sent false
30151 9117 Acked Read 2 Done Done Sent false
30152 9118 Acked Write 8 Done Done Sent false
30153 9119 Acked Read 9 Done Done Sent false
30154 9120 Acked Read 8 Done Done Sent false
30155 9121 Acked Write 6 Done Done Sent false
30156 9122 Acked Read 5 Done Done Sent false
30157 9123 Acked Read 4 Done Done Sent false
30158 9124 Acked Read 9 Done Done Sent false
30159 9125 Acked Read 10 Done Done Sent false
30160 9126 Acked Write 2 Done Done Sent false
30161 9127 Acked Flush 0 Done Done Sent false
30162 9128 Acked Read 4 Done Done Sent false
30163 9129 Acked Write 6 Done Done Sent false
30164 9130 Acked Flush 0 Done Done Sent false
30165 9131 Acked Read 8 Done Done Sent false
30166 9132 Acked Flush 0 Done Done Sent false
30167 9133 Acked Write 9 Done Done Sent false
30168 9134 Acked Write 4 Done Done Sent false
30169 9135 Acked Read 3 Done Done Sent false
30170 9136 Acked Write 10 Done Done Sent false
30171 9137 Acked Write 5 Done Done Sent false
30172 9138 Acked Read 5 Done Done Sent false
30173 9139 Acked Read 1 Done Done Sent false
30174 9140 Acked Write 10 Done Done Sent false
30175 9141 Acked Read 5 Done Done Sent false
30176 9142 Acked Read 7 Done Done Sent false
30177 9143 Acked Write 1 Done Done Sent false
30178 9144 Acked Write 4 Done Done Sent false
30179 9145 Acked Read 8 Done Done Sent false
30180 9146 Acked Write 9 Done Done Sent false
30181 9147 Acked Flush 0 Done Done Sent false
30182 9148 Acked Write 1 Done Done Sent false
30183 9149 Acked Flush 0 Done Done Sent false
30184 STATES DS:0 DS:1 DS:2 TOTAL
30185 Sent 0 0 302 302
30186 Done 302 302 0 604
30187 Skipped 0 0 0 0
30188 Error 0 0 0 0
30189 Last Flush: 9149 9149 8847
30190 Downstairs last five completed: 8847 8846 8845 8844 8843
30191 Upstairs last five completed: 9149 9148 9147 9146 9145
30192 CLIENT: Up:0 ds:302 act:2
301932025-01-27T19:42:41.351ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
301942025-01-27T19:42:41.352ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:54580) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
301952025-01-27T19:42:41.353ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
301962025-01-27T19:42:41.353ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
301972025-01-27T19:42:41.353ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
301982025-01-27T19:42:41.353ZINFOcrucible: send last flush ID to this DS: 8847 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
301992025-01-27T19:42:41.353ZINFOcrucible: Replied this last flush ID: 8847 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
302002025-01-27T19:42:41.353ZINFOcrucible: [2] client re-new 302 jobs since flush 8847 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
302012025-01-27T19:42:41.354ZINFOcrucible: [2] Replayed 302 jobs since flush: 8847 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
30202 ----------------------------------------------------------------
30203 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
30204 Downstairs last five completed: 9149 9148 9147 9146 9145
30205 Upstairs last five completed: 9149 9148 9147 9146 9145
30206 CLIENT: Up:0 ds:0 act:3
30207 Replay: All jobs finished
30208 Main task has work to do, go find it
30209 got dsc Stop(1)
30210 stop 1
30211 Jan 27 19:42:43.598[8820] Got stop action so:false kr:false
30212 INFO request completed, latency_us: 381, response_code: 204, uri: /stop/cid/1, method: POST, req_id: 54296742-2bfb-45d9-9390-1c1ae63b3c0b, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
302132025-01-27T19:42:43.599ZWARNcrucible: downstairs disconnected = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
302142025-01-27T19:42:43.599ZWARNcrucible: client task is sending Done(Finished) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
302152025-01-27T19:42:43.599ZWARNcrucible: downstairs task for 1 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
302162025-01-27T19:42:43.599ZINFOcrucible: client task is exiting = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
30217 Jan 27 19:42:43.599 INFO request completed, latency_us: 145, response_code: 200, uri: /state/cid/1, method: GET, req_id: e28d64ab-13a6-4daf-b2a7-061b37505070, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
30218 [8820] Exited with: ExitStatus(unix_wait_status(9))
30219 [8820][1] reports Exit
30220 Jan 27 19:42:47.601 INFO request completed, latency_us: 203, response_code: 200, uri: /state/cid/1, method: GET, req_id: b31129bd-c4fd-48d8-b14b-f32b4ad69282, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
30221 001/300 Read block 1327 len 8192
30222 002/300 Read block 2046 len 28672
30223 003/300 Read block 3968 len 4096
30224 004/300 Read block 4955 len 16384
30225 005/300 Flush
30226 006/300 Write block 2737 len 4096 data: 5
30227 007/300 Write block 2776 len 40960 data: 5 5 4 5 4 4 6 6 6 6
30228 008/300 Read block 4330 len 20480
30229 009/300 Write block 3907 len 8192 data: 5 6
30230 010/300 Read block 492 len 20480
30231 011/300 Read block 358 len 16384
30232 012/300 Read block 2588 len 24576
30233 013/300 Write block 1646 len 20480 data: 3 2 3 3 3
30234 014/300 Read block 777 len 12288
30235 015/300 Read block 4695 len 20480
30236 016/300 Write block 2420 len 4096 data: 5
30237 017/300 Read block 2067 len 20480
30238 018/300 Write block 3229 len 8192 data: 5 6
30239 019/300 Write block 153 len 32768 data: 3 5 5 5 5 5 5 4
30240 020/300 Write block 3234 len 40960 data: 5 6 4 4 3 4 4 6 7 7
30241 021/300 Write block 2981 len 16384 data: 6 6 7 6
30242 022/300 Write block 2612 len 36864 data: 7 7 8 7 7 8 8 10 10
30243 023/300 Write block 2984 len 20480 data: 7 6 6 6 5
30244 024/300 Read block 2842 len 4096
30245 025/300 Write block 1332 len 36864 data: 6 7 6 6 6 6 7 6 5
30246 026/300 Write block 1078 len 24576 data: 5 7 6 4 4 3
30247 027/300 Read block 2377 len 12288
30248 028/300 Read block 2517 len 40960
30249 029/300 Read block 3379 len 4096
30250 030/300 Write block 4758 len 36864 data: 4 5 3 3 5 5 8 8 6
30251 031/300 Read block 1981 len 16384
30252 032/300 Write block 1227 len 4096 data: 6
30253 033/300 Write block 4826 len 40960 data: 6 5 3 7 8 7 6 7 6 8
30254 034/300 Read block 4741 len 40960
30255 035/300 Read block 104 len 12288
30256 036/300 Flush
30257 037/300 Read block 1442 len 24576
30258 038/300 Write block 811 len 40960 data: 6 6 4 4 5 5 5 6 8 7
30259 039/300 Read block 4874 len 8192
30260 040/300 Read block 52 len 16384
30261 041/300 Write block 218 len 32768 data: 5 6 5 6 8 9 8 8
30262 042/300 Write block 4722 len 20480 data: 4 4 4 4 4
30263 043/300 Read block 3156 len 28672
30264 044/300 Write block 3644 len 12288 data: 6 6 7
30265 045/300 Write block 2622 len 28672 data: 10 8 10 7 5 7 8
30266 046/300 Write block 4493 len 16384 data: 7 8 8 6
30267 047/300 Write block 4308 len 36864 data: 4 4 4 3 3 3 3 3 3
30268 048/300 Read block 3842 len 40960
30269 049/300 Write block 1315 len 4096 data: 6
30270 050/300 Read block 3118 len 4096
30271 051/300 Read block 1817 len 40960
30272 052/300 Read block 119 len 20480
30273 053/300 Read block 4567 len 32768
30274 054/300 Write block 4680 len 40960 data: 8 9 8 7 5 5 6 6 5 6
30275 055/300 Read block 3654 len 28672
30276 056/300 Read block 3167 len 36864
30277 057/300 Write block 4589 len 28672 data: 5 5 6 7 7 7 6
30278 058/300 Write block 3464 len 24576 data: 7 6 6 6 5 5
30279 059/300 Write block 2924 len 36864 data: 3 5 6 6 7 9 8 8 8
30280 060/300 Read block 4563 len 40960
30281 061/300 Read block 1354 len 8192
30282 062/300 Write block 2223 len 28672 data: 3 3 3 3 4 3 3
30283 063/300 Write block 2766 len 12288 data: 8 9 9
30284 064/300 Flush
30285 065/300 Write block 3060 len 40960 data: 7 7 7 7 4 4 4 4 6 5
30286 066/300 Write block 1740 len 40960 data: 8 8 6 6 7 6 4 3 3 3
30287 067/300 Read block 2326 len 16384
30288 068/300 Read block 303 len 20480
30289 069/300 Read block 4990 len 12288
30290 070/300 Write block 1079 len 28672 data: 8 7 5 5 4 3 2
30291 071/300 Read block 960 len 40960
30292 072/300 Read block 4825 len 36864
30293 073/300 Write block 1114 len 28672 data: 6 6 6 7 7 5 7
30294 074/300 Write block 2658 len 12288 data: 6 6 6
30295 075/300 Flush
30296 076/300 Write block 2408 len 16384 data: 9 8 9 8
30297 077/300 Read block 4472 len 36864
30298 078/300 Write block 3334 len 8192 data: 5 6
30299 079/300 Flush
30300 080/300 Read block 742 len 20480
30301 081/300 Read block 507 len 12288
30302 082/300 Write block 1533 len 16384 data: 5 5 5 5
30303 083/300 Flush
30304 084/300 Read block 4868 len 24576
30305 085/300 Read block 4094 len 36864
30306 086/300 Read block 190 len 8192
30307 087/300 Write block 1905 len 28672 data: 6 6 5 4 4 3 3
30308 088/300 Read block 1107 len 8192
30309 089/300 Read block 1520 len 12288
30310 090/300 Read block 3540 len 12288
30311 091/300 Read block 672 len 28672
30312 092/300 Write block 1423 len 20480 data: 6 6 7 6 6
30313 093/300 Read block 1336 len 32768
30314 094/300 Read block 1997 len 32768
30315 095/300 Flush
30316 096/300 Flush
30317 097/300 Read block 1421 len 24576
30318 098/300 Read block 4352 len 32768
30319 099/300 Write block 2644 len 4096 data: 5
30320 100/300 Flush
30321 101/300 Write block 1902 len 20480 data: 3 4 6 7 7
30322 102/300 Write block 3021 len 32768 data: 6 6 6 6 6 6 5 5
30323 103/300 Write block 2303 len 28672 data: 9 9 9 6 7 7 6
30324 104/300 Write block 2515 len 4096 data: 7
30325 105/300 Read block 2663 len 20480
30326 106/300 Read block 3810 len 20480
30327 107/300 Write block 821 len 36864 data: 6 5 5 5 5 5 4 6 6
30328 108/300 Flush
30329 109/300 Write block 382 len 8192 data: 5 4
30330 110/300 Read block 2801 len 36864
30331 111/300 Read block 2308 len 12288
30332 112/300 Read block 60 len 20480
30333 113/300 Read block 2919 len 24576
30334 114/300 Read block 2128 len 8192
30335 115/300 Write block 3016 len 12288 data: 3 3 3
30336 116/300 Write block 3782 len 8192 data: 5 5
30337 117/300 Read block 2888 len 40960
30338 118/300 Write block 4336 len 20480 data: 10 9 8 7 5
30339 119/300 Read block 3982 len 32768
30340 120/300 Read block 102 len 28672
30341 121/300 Read block 2856 len 8192
30342 122/300 Write block 2028 len 8192 data: 7 8
30343 123/300 Write block 4386 len 8192 data: 5 5
30344 124/300 Read block 1474 len 40960
30345 125/300 Read block 3859 len 28672
30346 126/300 Write block 2481 len 16384 data: 3 3 4 5
30347 127/300 Read block 3415 len 40960
30348 128/300 Write block 2132 len 40960 data: 4 5 5 4 5 5 4 4 5 6
30349 129/300 Read block 4689 len 40960
30350 130/300 Write block 3273 len 28672 data: 7 5 6 6 7 6 7
30351 131/300 Write block 3858 len 20480 data: 4 4 5 6 4
30352 132/300 Write block 4262 len 4096 data: 6
30353 133/300 Read block 4677 len 12288
30354 134/300 Write block 4256 len 4096 data: 9
30355 135/300 Read block 4030 len 40960
30356 136/300 Write block 791 len 4096 data: 7
30357 137/300 Write block 2557 len 24576 data: 8 8 8 8 7 8
30358 138/300 Write block 4363 len 32768 data: 5 5 4 5 4 5 6 5
30359 139/300 Read block 2003 len 36864
30360 140/300 Read block 4391 len 4096
30361 141/300 Write block 2589 len 12288 data: 6 9 8
30362 142/300 Read block 2471 len 4096
30363 143/300 Read block 4653 len 4096
30364 144/300 Read block 3908 len 24576
30365 145/300 Write block 347 len 32768 data: 5 5 6 6 9 9 9 9
30366 146/300 Read block 658 len 36864
30367 147/300 Read block 747 len 40960
30368 148/300 Write block 3804 len 20480 data: 4 5 5 5 5
30369 149/300 Write block 2706 len 4096 data: 4
30370 150/300 Read block 3802 len 36864
30371 151/300 Write block 2835 len 12288 data: 4 5 6
30372 152/300 Flush
30373 153/300 Read block 4146 len 32768
30374 154/300 Read block 477 len 36864
30375 155/300 Read block 3275 len 8192
30376 156/300 Read block 1796 len 40960
30377 157/300 Write block 504 len 4096 data: 7
30378 158/300 Write block 1112 len 8192 data: 5 5
30379 159/300 Read block 2261 len 16384
30380 160/300 Read block 1680 len 4096
30381 161/300 Read block 2729 len 32768
30382 162/300 Write block 3720 len 28672 data: 8 8 7 7 8 9 9
30383 163/300 Write block 1654 len 28672 data: 6 5 5 5 6 6 6
30384 164/300 Write block 4752 len 32768 data: 5 6 6 8 7 4 5 6
30385 165/300 Write block 3550 len 16384 data: 5 5 4 6
30386 166/300 Write block 3824 len 36864 data: 3 2 4 4 4 5 6 5 5
30387 167/300 Read block 1259 len 12288
30388 168/300 Flush
30389 169/300 Read block 847 len 4096
30390 170/300 Flush
30391 171/300 Write block 3266 len 12288 data: 2 3 3
30392 172/300 Read block 1802 len 16384
30393 173/300 Flush
30394 174/300 Write block 4268 len 28672 data: 9 7 7 10 7 7 6
30395 175/300 Write block 1511 len 12288 data: 2 2 3
30396 176/300 Write block 210 len 24576 data: 6 6 4 3 5 5
30397 177/300 Read block 1638 len 12288
30398 178/300 Write block 4662 len 40960 data: 2 2 2 2 3 4 3 3 3 4
30399 179/300 Write block 3767 len 36864 data: 8 7 5 5 5 5 7 8 9
30400 180/300 Read block 4762 len 4096
30401 181/300 Write block 3220 len 12288 data: 6 7 8
30402 182/300 Read block 1262 len 32768
30403 183/300 Read block 4248 len 40960
30404 184/300 Read block 4826 len 4096
30405 185/300 Flush
30406 186/300 Flush
30407 187/300 Write block 81 len 36864 data: 5 4 5 3 3 4 5 6 5
30408 188/300 Read block 1211 len 28672
30409 189/300 Read block 778 len 12288
30410 190/300 Write block 3190 len 12288 data: 7 7 7
30411 191/300 Read block 165 len 32768
30412 192/300 Read block 885 len 28672
30413 193/300 Read block 242 len 4096
30414 194/300 Write block 3110 len 16384 data: 6 5 4 3
30415 195/300 Write block 2240 len 20480 data: 6 6 7 7 7
30416 196/300 Write block 1682 len 36864 data: 3 4 4 4 4 4 3 4 5
30417 197/300 Flush
30418 198/300 Read block 1729 len 28672
30419 199/300 Read block 275 len 4096
30420 200/300 Write block 3682 len 20480 data: 3 3 4 5 6
30421 201/300 Write block 3258 len 16384 data: 4 4 4 5
30422 202/300 Write block 2599 len 36864 data: 7 8 6 5 5 6 6 7 7
30423 203/300 Write block 3770 len 4096 data: 6
30424 204/300 Read block 322 len 16384
30425 205/300 Read block 4026 len 8192
30426 206/300 Write block 4464 len 12288 data: 3 3 5
30427 207/300 Write block 3024 len 24576 data: 7 7 7 6 6 5
30428 208/300 Write block 4292 len 4096 data: 6
30429 209/300 Write block 4787 len 20480 data: 6 6 7 9 9
30430 210/300 Read block 4172 len 36864
30431 211/300 Write block 4163 len 40960 data: 10 11 11 9 9 9 8 7 6 6
30432 212/300 Write block 3861 len 12288 data: 7 5 5
30433 213/300 Read block 2322 len 4096
30434 214/300 Read block 2836 len 4096
30435 215/300 Write block 233 len 8192 data: 4 5
30436 216/300 Read block 3824 len 8192
30437 217/300 Write block 312 len 36864 data: 4 4 5 5 5 6 6 6 8
30438 218/300 Read block 3674 len 40960
30439 219/300 Read block 4192 len 20480
30440 220/300 Flush
30441 221/300 Write block 2794 len 16384 data: 7 6 4 3
30442 222/300 Read block 140 len 36864
30443 223/300 Read block 4220 len 12288
30444 224/300 Write block 4523 len 16384 data: 4 5 5 8
30445 225/300 Read block 3572 len 32768
30446 226/300 Write block 4355 len 16384 data: 8 5 5 4
30447 227/300 Write block 3007 len 40960 data: 7 6 6 5 6 5 4 3 3 4
30448 228/300 Read block 1108 len 12288
30449 229/300 Read block 3276 len 8192
30450 230/300 Read block 2337 len 40960
30451 231/300 Read block 2322 len 16384
30452 232/300 Write block 248 len 28672 data: 4 5 4 4 6 7 6
30453 233/300 Read block 4144 len 40960
30454 234/300 Read block 178 len 4096
30455 235/300 Read block 1518 len 12288
30456 236/300 Read block 807 len 8192
30457 237/300 Read block 576 len 12288
30458 238/300 Read block 4370 len 24576
30459 239/300 Flush
30460 240/300 Write block 3617 len 12288 data: 7 9 8
30461 241/300 Write block 1497 len 32768 data: 4 3 3 3 3 3 3 4
30462 242/300 Read block 2137 len 20480
30463 243/300 Write block 746 len 12288 data: 7 7 7
30464 244/300 Flush
30465 245/300 Write block 2621 len 8192 data: 10 11
30466 246/300 Read block 3083 len 8192
30467 247/300 Write block 3192 len 40960 data: 8 5 5 5 5 4 4 3 4 4
30468 248/300 Write block 339 len 16384 data: 7 6 6 5
30469 249/300 Write block 3382 len 36864 data: 4 5 5 5 5 6 7 8 7
30470 250/300 Read block 1122 len 4096
30471 251/300 Write block 846 len 40960 data: 6 4 5 4 4 4 4 4 4 3
30472 252/300 Write block 848 len 40960 data: 6 5 5 5 5 5 5 4 4 4
30473 253/300 Write block 1699 len 20480 data: 4 4 4 4 4
30474 254/300 Read block 2304 len 28672
30475 255/300 Read block 1860 len 4096
30476 256/300 Read block 2914 len 16384
30477 257/300 Flush
30478 258/300 Flush
30479 259/300 Read block 2960 len 24576
30480 260/300 Read block 4024 len 16384
30481 261/300 Write block 4395 len 12288 data: 4 4 4
30482 262/300 Read block 1156 len 24576
30483 263/300 Read block 2636 len 28672
30484 264/300 Read block 360 len 36864
30485 265/300 Read block 1930 len 24576
30486 266/300 Read block 4467 len 8192
30487 267/300 Read block 1943 len 40960
30488 268/300 Read block 2315 len 24576
30489 269/300 Flush
30490 270/300 Read block 923 len 12288
30491 271/300 Write block 4730 len 4096 data: 5
30492 272/300 Read block 3229 len 12288
30493 273/300 Read block 3391 len 20480
30494 274/300 Read block 4081 len 16384
30495 275/300 Write block 1298 len 32768 data: 10 9 8 10 9 11 9 9
30496 276/300 Flush
30497 277/300 Write block 3194 len 8192 data: 6 6
30498 278/300 Write block 474 len 36864 data: 5 6 6 5 7 7 6 6 6
30499 279/300 Write block 1531 len 24576 data: 7 6 6 6 6 6
30500 280/300 Read block 2432 len 24576
30501 281/300 Write block 1886 len 24576 data: 8 8 9 9 8 6
30502 282/300 Write block 4402 len 28672 data: 3 4 5 6 6 7 7
30503 283/300 Read block 2933 len 40960
30504 284/300 Read block 1844 len 32768
30505 285/300 Read block 4938 len 36864
30506 286/300 Write block 1718 len 12288 data: 4 6 5
30507 287/300 Read block 2510 len 24576
30508 288/300 Write block 4707 len 32768 data: 6 8 7 7 6 6 7 7
30509 289/300 Read block 890 len 40960
30510 290/300 Read block 2745 len 16384
30511 291/300 Write block 2282 len 4096 data: 8
30512 292/300 Read block 3681 len 16384
30513 293/300 Read block 1412 len 4096
30514 294/300 Flush
30515 295/300 Read block 4729 len 12288
30516 296/300 Flush
30517 297/300 Read block 4383 len 4096
30518 298/300 Write block 3669 len 40960 data: 4 5 6 6 7 7 5 5 5 5
30519 299/300 Read block 3699 len 32768
30520 300/300 Write block 1324 len 20480 data: 5 5 5 5 5
30521 301/300 Read block 3006 len 16384
30522 Main task has work to do, go find it
30523 got dsc Start(1)
30524 start 1
30525 Jan 27 19:42:47.835 INFO request completed, latency_us: 216, response_code: 204, uri: /start/cid/1, method: POST, req_id: 5012df3b-d860-4d41-bbc5-b8ed10022a85, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
30526 Replay: started 1, returned:Ok(())
30527 ----------------------------------------------------------------
30528 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
30529 States: Active Offline Active
30530 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30531 9150 Acked Read 2 Done Sent Done false
30532 9151 Acked Read 7 Done Sent Done false
30533 9152 Acked Read 1 Done Sent Done false
30534 9153 Acked Read 4 Done Sent Done false
30535 9154 Acked Flush 0 Done Sent Done false
30536 9155 Acked Write 1 Done Sent Done false
30537 9156 Acked Write 10 Done Sent Done false
30538 9157 Acked Read 5 Done Sent Done false
30539 9158 Acked Write 2 Done Sent Done false
30540 9159 Acked Read 5 Done Sent Done false
30541 9160 Acked Read 4 Done Sent Done false
30542 9161 Acked Read 6 Done Sent Done false
30543 9162 Acked Write 5 Done Sent Done false
30544 9163 Acked Read 3 Done Sent Done false
30545 9164 Acked Read 5 Done Sent Done false
30546 9165 Acked Write 1 Done Sent Done false
30547 9166 Acked Read 5 Done Sent Done false
30548 9167 Acked Write 2 Done Sent Done false
30549 9168 Acked Write 8 Done Sent Done false
30550 9169 Acked Write 10 Done Sent Done false
30551 9170 Acked Write 4 Done Sent Done false
30552 9171 Acked Write 9 Done Sent Done false
30553 9172 Acked Write 5 Done Sent Done false
30554 9173 Acked Read 1 Done Sent Done false
30555 9174 Acked Write 9 Done Sent Done false
30556 9175 Acked Write 6 Done Sent Done false
30557 9176 Acked Read 3 Done Sent Done false
30558 9177 Acked Read 10 Done Sent Done false
30559 9178 Acked Read 1 Done Sent Done false
30560 9179 Acked Write 9 Done Sent Done false
30561 9180 Acked Read 4 Done Sent Done false
30562 9181 Acked Write 1 Done Sent Done false
30563 9182 Acked Write 10 Done Sent Done false
30564 9183 Acked Read 10 Done Sent Done false
30565 9184 Acked Read 3 Done Sent Done false
30566 9185 Acked Flush 0 Done Sent Done false
30567 9186 Acked Read 6 Done Sent Done false
30568 9187 Acked Write 10 Done Sent Done false
30569 9188 Acked Read 2 Done Sent Done false
30570 9189 Acked Read 4 Done Sent Done false
30571 9190 Acked Write 8 Done Sent Done false
30572 9191 Acked Write 5 Done Sent Done false
30573 9192 Acked Read 7 Done Sent Done false
30574 9193 Acked Write 3 Done Sent Done false
30575 9194 Acked Write 7 Done Sent Done false
30576 9195 Acked Write 4 Done Sent Done false
30577 9196 Acked Write 9 Done Sent Done false
30578 9197 Acked Read 10 Done Sent Done false
30579 9198 Acked Write 1 Done Sent Done false
30580 9199 Acked Read 1 Done Sent Done false
30581 9200 Acked Read 10 Done Sent Done false
30582 9201 Acked Read 5 Done Sent Done false
30583 9202 Acked Read 8 Done Sent Done false
30584 9203 Acked Write 10 Done Sent Done false
30585 9204 Acked Read 7 Done Sent Done false
30586 9205 Acked Read 9 Done Sent Done false
30587 9206 Acked Write 7 Done Sent Done false
30588 9207 Acked Write 6 Done Sent Done false
30589 9208 Acked Write 9 Done Sent Done false
30590 9209 Acked Read 10 Done Sent Done false
30591 9210 Acked Read 2 Done Sent Done false
30592 9211 Acked Write 7 Done Sent Done false
30593 9212 Acked Write 3 Done Sent Done false
30594 9213 Acked Flush 0 Done Sent Done false
30595 9214 Acked Write 10 Done Sent Done false
30596 9215 Acked Write 10 Done Sent Done false
30597 9216 Acked Read 4 Done Sent Done false
30598 9217 Acked Read 5 Done Sent Done false
30599 9218 Acked Read 3 Done Sent Done false
30600 9219 Acked Write 7 Done Sent Done false
30601 9220 Acked Read 10 Done Sent Done false
30602 9221 Acked Read 9 Done Sent Done false
30603 9222 Acked Write 7 Done Sent Done false
30604 9223 Acked Write 3 Done Sent Done false
30605 9224 Acked Flush 0 Done Sent Done false
30606 9225 Acked Write 4 Done Sent Done false
30607 9226 Acked Read 9 Done Sent Done false
30608 9227 Acked Write 2 Done Sent Done false
30609 9228 Acked Flush 0 Done Sent Done false
30610 9229 Acked Read 5 Done Sent Done false
30611 9230 Acked Read 3 Done Sent Done false
30612 9231 Acked Write 4 Done Sent Done false
30613 9232 Acked Flush 0 Done Sent Done false
30614 9233 Acked Read 6 Done Sent Done false
30615 9234 Acked Read 9 Done Sent Done false
30616 9235 Acked Read 2 Done Sent Done false
30617 9236 Acked Write 7 Done Sent Done false
30618 9237 Acked Read 2 Done Sent Done false
30619 9238 Acked Read 3 Done Sent Done false
30620 9239 Acked Read 3 Done Sent Done false
30621 9240 Acked Read 7 Done Sent Done false
30622 9241 Acked Write 5 Done Sent Done false
30623 9242 Acked Read 8 Done Sent Done false
30624 9243 Acked Read 8 Done Sent Done false
30625 9244 Acked Flush 0 Done Sent Done false
30626 9245 Acked Flush 0 Done Sent Done false
30627 9246 Acked Read 6 Done Sent Done false
30628 9247 Acked Read 8 Done Sent Done false
30629 9248 Acked Write 1 Done Sent Done false
30630 9249 Acked Flush 0 Done Sent Done false
30631 9250 Acked Write 5 Done Sent Done false
30632 9251 Acked Write 8 Done Sent Done false
30633 9252 Acked Write 7 Done Sent Done false
30634 9253 Acked Write 1 Done Sent Done false
30635 9254 Acked Read 5 Done Sent Done false
30636 9255 Acked Read 5 Done Sent Done false
30637 9256 Acked Write 9 Done Sent Done false
30638 9257 Acked Flush 0 Done Sent Done false
30639 9258 Acked Write 2 Done Sent Done false
30640 9259 Acked Read 9 Done Sent Done false
30641 9260 Acked Read 3 Done Sent Done false
30642 9261 Acked Read 5 Done Sent Done false
30643 9262 Acked Read 6 Done Sent Done false
30644 9263 Acked Read 2 Done Sent Done false
30645 9264 Acked Write 3 Done Sent Done false
30646 9265 Acked Write 2 Done Sent Done false
30647 9266 Acked Read 10 Done Sent Done false
30648 9267 Acked Write 5 Done Sent Done false
30649 9268 Acked Read 8 Done Sent Done false
30650 9269 Acked Read 7 Done Sent Done false
30651 9270 Acked Read 2 Done Sent Done false
30652 9271 Acked Write 2 Done Sent Done false
30653 9272 Acked Write 2 Done Sent Done false
30654 9273 Acked Read 10 Done Sent Done false
30655 9274 Acked Read 7 Done Sent Done false
30656 9275 Acked Write 4 Done Sent Done false
30657 9276 Acked Read 10 Done Sent Done false
30658 9277 Acked Write 10 Done Sent Done false
30659 9278 Acked Read 10 Done Sent Done false
30660 9279 Acked Write 7 Done Sent Done false
30661 9280 Acked Write 5 Done Sent Done false
30662 9281 Acked Write 1 Done Sent Done false
30663 9282 Acked Read 3 Done Sent Done false
30664 9283 Acked Write 1 Done Sent Done false
30665 9284 Acked Read 10 Done Sent Done false
30666 9285 Acked Write 1 Done Sent Done false
30667 9286 Acked Write 6 Done Sent Done false
30668 9287 Acked Write 8 Done Sent Done false
30669 9288 Acked Read 9 Done Sent Done false
30670 9289 Acked Read 1 Done Sent Done false
30671 9290 Acked Write 3 Done Sent Done false
30672 9291 Acked Read 1 Done Sent Done false
30673 9292 Acked Read 1 Done Sent Done false
30674 9293 Acked Read 6 Done Sent Done false
30675 9294 Acked Write 8 Done Sent Done false
30676 9295 Acked Read 9 Done Sent Done false
30677 9296 Acked Read 10 Done Sent Done false
30678 9297 Acked Write 5 Done Sent Done false
30679 9298 Acked Write 1 Done Sent Done false
30680 9299 Acked Read 9 Done Sent Done false
30681 9300 Acked Write 3 Done Sent Done false
30682 9301 Acked Flush 0 Done Sent Done false
30683 9302 Acked Read 8 Done Sent Done false
30684 9303 Acked Read 9 Done Sent Done false
30685 9304 Acked Read 2 Done Sent Done false
30686 9305 Acked Read 10 Done Sent Done false
30687 9306 Acked Write 1 Done Sent Done false
30688 9307 Acked Write 2 Done Sent Done false
30689 9308 Acked Read 4 Done Sent Done false
30690 9309 Acked Read 1 Done Sent Done false
30691 9310 Acked Read 8 Done Sent Done false
30692 9311 Acked Write 7 Done Sent Done false
30693 9312 Acked Write 7 Done Sent Done false
30694 9313 Acked Write 8 Done Sent Done false
30695 9314 Acked Write 4 Done Sent Done false
30696 9315 Acked Write 9 Done Sent Done false
30697 9316 Acked Read 3 Done Sent Done false
30698 9317 Acked Flush 0 Done Sent Done false
30699 9318 Acked Read 1 Done Sent Done false
30700 9319 Acked Flush 0 Done Sent Done false
30701 9320 Acked Write 3 Done Sent Done false
30702 9321 Acked Read 4 Done Sent Done false
30703 9322 Acked Flush 0 Done Sent Done false
30704 9323 Acked Write 7 Done Sent Done false
30705 9324 Acked Write 3 Done Sent Done false
30706 9325 Acked Write 6 Done Sent Done false
30707 9326 Acked Read 3 Done Sent Done false
30708 9327 Acked Write 10 Done Sent Done false
30709 9328 Acked Write 9 Done Sent Done false
30710 9329 Acked Read 1 Done Sent Done false
30711 9330 Acked Write 3 Done Sent Done false
30712 9331 Acked Read 8 Done Sent Done false
30713 9332 Acked Read 10 Done Sent Done false
30714 9333 Acked Read 1 Done Sent Done false
30715 9334 Acked Flush 0 Done Sent Done false
30716 9335 Acked Flush 0 Done Sent Done false
30717 9336 Acked Write 9 Done Sent Done false
30718 9337 Acked Read 7 Done Sent Done false
30719 9338 Acked Read 3 Done Sent Done false
30720 9339 Acked Write 3 Done Sent Done false
30721 9340 Acked Read 8 Done Sent Done false
30722 9341 Acked Read 7 Done Sent Done false
30723 9342 Acked Read 1 Done Sent Done false
30724 9343 Acked Write 4 Done Sent Done false
30725 9344 Acked Write 5 Done Sent Done false
30726 9345 Acked Write 9 Done Sent Done false
30727 9346 Acked Flush 0 Done Sent Done false
30728 9347 Acked Read 7 Done Sent Done false
30729 9348 Acked Read 1 Done Sent Done false
30730 9349 Acked Write 5 Done Sent Done false
30731 9350 Acked Write 4 Done Sent Done false
30732 9351 Acked Write 9 Done Sent Done false
30733 9352 Acked Write 1 Done Sent Done false
30734 9353 Acked Read 4 Done Sent Done false
30735 9354 Acked Read 2 Done Sent Done false
30736 9355 Acked Write 3 Done Sent Done false
30737 9356 Acked Write 6 Done Sent Done false
30738 9357 Acked Write 1 Done Sent Done false
30739 9358 Acked Write 5 Done Sent Done false
30740 9359 Acked Read 9 Done Sent Done false
30741 9360 Acked Write 10 Done Sent Done false
30742 9361 Acked Write 3 Done Sent Done false
30743 9362 Acked Read 1 Done Sent Done false
30744 9363 Acked Read 1 Done Sent Done false
30745 9364 Acked Write 2 Done Sent Done false
30746 9365 Acked Read 2 Done Sent Done false
30747 9366 Acked Write 9 Done Sent Done false
30748 9367 Acked Read 10 Done Sent Done false
30749 9368 Acked Read 5 Done Sent Done false
30750 9369 Acked Flush 0 Done Sent Done false
30751 9370 Acked Write 4 Done Sent Done false
30752 9371 Acked Read 9 Done Sent Done false
30753 9372 Acked Read 3 Done Sent Done false
30754 9373 Acked Write 4 Done Sent Done false
30755 9374 Acked Read 8 Done Sent Done false
30756 9375 Acked Write 4 Done Sent Done false
30757 9376 Acked Write 10 Done Sent Done false
30758 9377 Acked Read 3 Done Sent Done false
30759 9378 Acked Read 2 Done Sent Done false
30760 9379 Acked Read 10 Done Sent Done false
30761 9380 Acked Read 4 Done Sent Done false
30762 9381 Acked Write 7 Done Sent Done false
30763 9382 Acked Read 10 Done Sent Done false
30764 9383 Acked Read 1 Done Sent Done false
30765 9384 Acked Read 3 Done Sent Done false
30766 9385 Acked Read 2 Done Sent Done false
30767 9386 Acked Read 3 Done Sent Done false
30768 9387 Acked Read 6 Done Sent Done false
30769 9388 Acked Flush 0 Done Sent Done false
30770 9389 Acked Write 3 Done Sent Done false
30771 9390 Acked Write 8 Done Sent Done false
30772 9391 Acked Read 5 Done Sent Done false
30773 9392 Acked Write 3 Done Sent Done false
30774 9393 Acked Flush 0 Done Sent Done false
30775 9394 Acked Write 2 Done Sent Done false
30776 9395 Acked Read 2 Done Sent Done false
30777 9396 Acked Write 10 Done Sent Done false
30778 9397 Acked Write 4 Done Sent Done false
30779 9398 Acked Write 9 Done Sent Done false
30780 9399 Acked Read 1 Done Sent Done false
30781 9400 Acked Write 10 Done Sent Done false
30782 9401 Acked Write 10 Done Sent Done false
30783 9402 Acked Write 5 Done Sent Done false
30784 9403 Acked Read 7 Done Sent Done false
30785 9404 Acked Read 1 Done Sent Done false
30786 9405 Acked Read 4 Done Sent Done false
30787 9406 Acked Flush 0 Done Sent Done false
30788 9407 Acked Flush 0 Done Sent Done false
30789 9408 Acked Read 6 Done Sent Done false
30790 9409 Acked Read 4 Done Sent Done false
30791 9410 Acked Write 3 Done Sent Done false
30792 9411 Acked Read 6 Done Sent Done false
30793 9412 Acked Read 7 Done Sent Done false
30794 9413 Acked Read 9 Done Sent Done false
30795 9414 Acked Read 6 Done Sent Done false
30796 9415 Acked Read 2 Done Sent Done false
30797 9416 Acked Read 10 Done Sent Done false
30798 9417 Acked Read 6 Done Sent Done false
30799 9418 Acked Flush 0 Done Sent Done false
30800 9419 Acked Read 3 Done Sent Done false
30801 9420 Acked Write 1 Done Sent Done false
30802 9421 Acked Read 3 Done Sent Done false
30803 9422 Acked Read 5 Done Sent Done false
30804 9423 Acked Read 4 Done Sent Done false
30805 9424 Acked Write 8 Done Sent Done false
30806 9425 Acked Flush 0 Done Sent Done false
30807 9426 Acked Write 2 Done Sent Done false
30808 9427 Acked Write 9 Done Sent Done false
30809 9428 Acked Write 6 Done Sent Done false
30810 9429 Acked Read 6 Done Sent Done false
30811 9430 Acked Write 6 Done Sent Done false
30812 9431 Acked Write 7 Done Sent Done false
30813 9432 Acked Read 10 Done Sent Done false
30814 9433 Acked Read 8 Done Sent Done false
30815 9434 Acked Read 9 Done Sent Done false
30816 9435 Acked Write 3 Done Sent Done false
30817 9436 Acked Read 6 Done Sent Done false
30818 9437 Acked Write 8 Done Sent Done false
30819 9438 Acked Read 10 Done Sent Done false
30820 9439 Acked Read 4 Done Sent Done false
30821 9440 Acked Write 1 Done Sent Done false
30822 9441 Acked Read 4 Done Sent Done false
30823 9442 Acked Read 1 Done Sent Done false
30824 9443 Acked Flush 0 Done Sent Done false
30825 9444 Acked Read 3 Done Sent Done false
30826 9445 Acked Flush 0 Done Sent Done false
30827 9446 Acked Read 1 Done Sent Done false
30828 9447 Acked Write 10 Done Sent Done false
30829 9448 Acked Read 8 Done Sent Done false
30830 9449 Acked Write 5 Done Sent Done false
30831 9450 Acked Read 4 Done Sent Done false
30832 STATES DS:0 DS:1 DS:2 TOTAL
30833 Sent 0 301 0 301
30834 Done 301 0 301 602
30835 Skipped 0 0 0 0
30836 Error 0 0 0 0
30837 Last Flush: 9445 9149 9445
30838 Downstairs last five completed: 9149 9148 9147 9146 9145
30839 Upstairs last five completed: 9450 9449 9448 9447 9446
30840 CLIENT: Up:0 ds:301 act:2
30841 [8820] got start action
30842 [8820] I am going to restart
30843 Starting downstairs at port 8820
30844 Make output file at "/tmp/dsc/downstairs-8820.txt"
30845 [8820][1] reports Starting
30846 Downstairs /var/tmp/dsc/region/8820 port 8820 PID:Some(1228)
30847 [8820][1] reports Running
30848 ----------------------------------------------------------------
30849 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
30850 States: Active Offline Active
30851 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
30852 9150 Acked Read 2 Done Sent Done false
30853 9151 Acked Read 7 Done Sent Done false
30854 9152 Acked Read 1 Done Sent Done false
30855 9153 Acked Read 4 Done Sent Done false
30856 9154 Acked Flush 0 Done Sent Done false
30857 9155 Acked Write 1 Done Sent Done false
30858 9156 Acked Write 10 Done Sent Done false
30859 9157 Acked Read 5 Done Sent Done false
30860 9158 Acked Write 2 Done Sent Done false
30861 9159 Acked Read 5 Done Sent Done false
30862 9160 Acked Read 4 Done Sent Done false
30863 9161 Acked Read 6 Done Sent Done false
30864 9162 Acked Write 5 Done Sent Done false
30865 9163 Acked Read 3 Done Sent Done false
30866 9164 Acked Read 5 Done Sent Done false
30867 9165 Acked Write 1 Done Sent Done false
30868 9166 Acked Read 5 Done Sent Done false
30869 9167 Acked Write 2 Done Sent Done false
30870 9168 Acked Write 8 Done Sent Done false
30871 9169 Acked Write 10 Done Sent Done false
30872 9170 Acked Write 4 Done Sent Done false
30873 9171 Acked Write 9 Done Sent Done false
30874 9172 Acked Write 5 Done Sent Done false
30875 9173 Acked Read 1 Done Sent Done false
30876 9174 Acked Write 9 Done Sent Done false
30877 9175 Acked Write 6 Done Sent Done false
30878 9176 Acked Read 3 Done Sent Done false
30879 9177 Acked Read 10 Done Sent Done false
30880 9178 Acked Read 1 Done Sent Done false
30881 9179 Acked Write 9 Done Sent Done false
30882 9180 Acked Read 4 Done Sent Done false
30883 9181 Acked Write 1 Done Sent Done false
30884 9182 Acked Write 10 Done Sent Done false
30885 9183 Acked Read 10 Done Sent Done false
30886 9184 Acked Read 3 Done Sent Done false
30887 9185 Acked Flush 0 Done Sent Done false
30888 9186 Acked Read 6 Done Sent Done false
30889 9187 Acked Write 10 Done Sent Done false
30890 9188 Acked Read 2 Done Sent Done false
30891 9189 Acked Read 4 Done Sent Done false
30892 9190 Acked Write 8 Done Sent Done false
30893 9191 Acked Write 5 Done Sent Done false
30894 9192 Acked Read 7 Done Sent Done false
30895 9193 Acked Write 3 Done Sent Done false
30896 9194 Acked Write 7 Done Sent Done false
30897 9195 Acked Write 4 Done Sent Done false
30898 9196 Acked Write 9 Done Sent Done false
30899 9197 Acked Read 10 Done Sent Done false
30900 9198 Acked Write 1 Done Sent Done false
30901 9199 Acked Read 1 Done Sent Done false
30902 9200 Acked Read 10 Done Sent Done false
30903 9201 Acked Read 5 Done Sent Done false
30904 9202 Acked Read 8 Done Sent Done false
30905 9203 Acked Write 10 Done Sent Done false
30906 9204 Acked Read 7 Done Sent Done false
30907 9205 Acked Read 9 Done Sent Done false
30908 9206 Acked Write 7 Done Sent Done false
30909 9207 Acked Write 6 Done Sent Done false
30910 9208 Acked Write 9 Done Sent Done false
30911 9209 Acked Read 10 Done Sent Done false
30912 9210 Acked Read 2 Done Sent Done false
30913 9211 Acked Write 7 Done Sent Done false
30914 9212 Acked Write 3 Done Sent Done false
30915 9213 Acked Flush 0 Done Sent Done false
30916 9214 Acked Write 10 Done Sent Done false
30917 9215 Acked Write 10 Done Sent Done false
30918 9216 Acked Read 4 Done Sent Done false
30919 9217 Acked Read 5 Done Sent Done false
30920 9218 Acked Read 3 Done Sent Done false
30921 9219 Acked Write 7 Done Sent Done false
30922 9220 Acked Read 10 Done Sent Done false
30923 9221 Acked Read 9 Done Sent Done false
30924 9222 Acked Write 7 Done Sent Done false
30925 9223 Acked Write 3 Done Sent Done false
30926 9224 Acked Flush 0 Done Sent Done false
30927 9225 Acked Write 4 Done Sent Done false
30928 9226 Acked Read 9 Done Sent Done false
30929 9227 Acked Write 2 Done Sent Done false
30930 9228 Acked Flush 0 Done Sent Done false
30931 9229 Acked Read 5 Done Sent Done false
30932 9230 Acked Read 3 Done Sent Done false
30933 9231 Acked Write 4 Done Sent Done false
30934 9232 Acked Flush 0 Done Sent Done false
30935 9233 Acked Read 6 Done Sent Done false
30936 9234 Acked Read 9 Done Sent Done false
30937 9235 Acked Read 2 Done Sent Done false
30938 9236 Acked Write 7 Done Sent Done false
30939 9237 Acked Read 2 Done Sent Done false
30940 9238 Acked Read 3 Done Sent Done false
30941 9239 Acked Read 3 Done Sent Done false
30942 9240 Acked Read 7 Done Sent Done false
30943 9241 Acked Write 5 Done Sent Done false
30944 9242 Acked Read 8 Done Sent Done false
30945 9243 Acked Read 8 Done Sent Done false
30946 9244 Acked Flush 0 Done Sent Done false
30947 9245 Acked Flush 0 Done Sent Done false
30948 9246 Acked Read 6 Done Sent Done false
30949 9247 Acked Read 8 Done Sent Done false
30950 9248 Acked Write 1 Done Sent Done false
30951 9249 Acked Flush 0 Done Sent Done false
30952 9250 Acked Write 5 Done Sent Done false
30953 9251 Acked Write 8 Done Sent Done false
30954 9252 Acked Write 7 Done Sent Done false
30955 9253 Acked Write 1 Done Sent Done false
30956 9254 Acked Read 5 Done Sent Done false
30957 9255 Acked Read 5 Done Sent Done false
30958 9256 Acked Write 9 Done Sent Done false
30959 9257 Acked Flush 0 Done Sent Done false
30960 9258 Acked Write 2 Done Sent Done false
30961 9259 Acked Read 9 Done Sent Done false
30962 9260 Acked Read 3 Done Sent Done false
30963 9261 Acked Read 5 Done Sent Done false
30964 9262 Acked Read 6 Done Sent Done false
30965 9263 Acked Read 2 Done Sent Done false
30966 9264 Acked Write 3 Done Sent Done false
30967 9265 Acked Write 2 Done Sent Done false
30968 9266 Acked Read 10 Done Sent Done false
30969 9267 Acked Write 5 Done Sent Done false
30970 9268 Acked Read 8 Done Sent Done false
30971 9269 Acked Read 7 Done Sent Done false
30972 9270 Acked Read 2 Done Sent Done false
30973 9271 Acked Write 2 Done Sent Done false
30974 9272 Acked Write 2 Done Sent Done false
30975 9273 Acked Read 10 Done Sent Done false
30976 9274 Acked Read 7 Done Sent Done false
30977 9275 Acked Write 4 Done Sent Done false
30978 9276 Acked Read 10 Done Sent Done false
30979 9277 Acked Write 10 Done Sent Done false
30980 9278 Acked Read 10 Done Sent Done false
30981 9279 Acked Write 7 Done Sent Done false
30982 9280 Acked Write 5 Done Sent Done false
30983 9281 Acked Write 1 Done Sent Done false
30984 9282 Acked Read 3 Done Sent Done false
30985 9283 Acked Write 1 Done Sent Done false
30986 9284 Acked Read 10 Done Sent Done false
30987 9285 Acked Write 1 Done Sent Done false
30988 9286 Acked Write 6 Done Sent Done false
30989 9287 Acked Write 8 Done Sent Done false
30990 9288 Acked Read 9 Done Sent Done false
30991 9289 Acked Read 1 Done Sent Done false
30992 9290 Acked Write 3 Done Sent Done false
30993 9291 Acked Read 1 Done Sent Done false
30994 9292 Acked Read 1 Done Sent Done false
30995 9293 Acked Read 6 Done Sent Done false
30996 9294 Acked Write 8 Done Sent Done false
30997 9295 Acked Read 9 Done Sent Done false
30998 9296 Acked Read 10 Done Sent Done false
30999 9297 Acked Write 5 Done Sent Done false
31000 9298 Acked Write 1 Done Sent Done false
31001 9299 Acked Read 9 Done Sent Done false
31002 9300 Acked Write 3 Done Sent Done false
31003 9301 Acked Flush 0 Done Sent Done false
31004 9302 Acked Read 8 Done Sent Done false
31005 9303 Acked Read 9 Done Sent Done false
31006 9304 Acked Read 2 Done Sent Done false
31007 9305 Acked Read 10 Done Sent Done false
31008 9306 Acked Write 1 Done Sent Done false
31009 9307 Acked Write 2 Done Sent Done false
31010 9308 Acked Read 4 Done Sent Done false
31011 9309 Acked Read 1 Done Sent Done false
31012 9310 Acked Read 8 Done Sent Done false
31013 9311 Acked Write 7 Done Sent Done false
31014 9312 Acked Write 7 Done Sent Done false
31015 9313 Acked Write 8 Done Sent Done false
31016 9314 Acked Write 4 Done Sent Done false
31017 9315 Acked Write 9 Done Sent Done false
31018 9316 Acked Read 3 Done Sent Done false
31019 9317 Acked Flush 0 Done Sent Done false
31020 9318 Acked Read 1 Done Sent Done false
31021 9319 Acked Flush 0 Done Sent Done false
31022 9320 Acked Write 3 Done Sent Done false
31023 9321 Acked Read 4 Done Sent Done false
31024 9322 Acked Flush 0 Done Sent Done false
31025 9323 Acked Write 7 Done Sent Done false
31026 9324 Acked Write 3 Done Sent Done false
31027 9325 Acked Write 6 Done Sent Done false
31028 9326 Acked Read 3 Done Sent Done false
31029 9327 Acked Write 10 Done Sent Done false
31030 9328 Acked Write 9 Done Sent Done false
31031 9329 Acked Read 1 Done Sent Done false
31032 9330 Acked Write 3 Done Sent Done false
31033 9331 Acked Read 8 Done Sent Done false
31034 9332 Acked Read 10 Done Sent Done false
31035 9333 Acked Read 1 Done Sent Done false
31036 9334 Acked Flush 0 Done Sent Done false
31037 9335 Acked Flush 0 Done Sent Done false
31038 9336 Acked Write 9 Done Sent Done false
31039 9337 Acked Read 7 Done Sent Done false
31040 9338 Acked Read 3 Done Sent Done false
31041 9339 Acked Write 3 Done Sent Done false
31042 9340 Acked Read 8 Done Sent Done false
31043 9341 Acked Read 7 Done Sent Done false
31044 9342 Acked Read 1 Done Sent Done false
31045 9343 Acked Write 4 Done Sent Done false
31046 9344 Acked Write 5 Done Sent Done false
31047 9345 Acked Write 9 Done Sent Done false
31048 9346 Acked Flush 0 Done Sent Done false
31049 9347 Acked Read 7 Done Sent Done false
31050 9348 Acked Read 1 Done Sent Done false
31051 9349 Acked Write 5 Done Sent Done false
31052 9350 Acked Write 4 Done Sent Done false
31053 9351 Acked Write 9 Done Sent Done false
31054 9352 Acked Write 1 Done Sent Done false
31055 9353 Acked Read 4 Done Sent Done false
31056 9354 Acked Read 2 Done Sent Done false
31057 9355 Acked Write 3 Done Sent Done false
31058 9356 Acked Write 6 Done Sent Done false
31059 9357 Acked Write 1 Done Sent Done false
31060 9358 Acked Write 5 Done Sent Done false
31061 9359 Acked Read 9 Done Sent Done false
31062 9360 Acked Write 10 Done Sent Done false
31063 9361 Acked Write 3 Done Sent Done false
31064 9362 Acked Read 1 Done Sent Done false
31065 9363 Acked Read 1 Done Sent Done false
31066 9364 Acked Write 2 Done Sent Done false
31067 9365 Acked Read 2 Done Sent Done false
31068 9366 Acked Write 9 Done Sent Done false
31069 9367 Acked Read 10 Done Sent Done false
31070 9368 Acked Read 5 Done Sent Done false
31071 9369 Acked Flush 0 Done Sent Done false
31072 9370 Acked Write 4 Done Sent Done false
31073 9371 Acked Read 9 Done Sent Done false
31074 9372 Acked Read 3 Done Sent Done false
31075 9373 Acked Write 4 Done Sent Done false
31076 9374 Acked Read 8 Done Sent Done false
31077 9375 Acked Write 4 Done Sent Done false
31078 9376 Acked Write 10 Done Sent Done false
31079 9377 Acked Read 3 Done Sent Done false
31080 9378 Acked Read 2 Done Sent Done false
31081 9379 Acked Read 10 Done Sent Done false
31082 9380 Acked Read 4 Done Sent Done false
31083 9381 Acked Write 7 Done Sent Done false
31084 9382 Acked Read 10 Done Sent Done false
31085 9383 Acked Read 1 Done Sent Done false
31086 9384 Acked Read 3 Done Sent Done false
31087 9385 Acked Read 2 Done Sent Done false
31088 9386 Acked Read 3 Done Sent Done false
31089 9387 Acked Read 6 Done Sent Done false
31090 9388 Acked Flush 0 Done Sent Done false
31091 9389 Acked Write 3 Done Sent Done false
31092 9390 Acked Write 8 Done Sent Done false
31093 9391 Acked Read 5 Done Sent Done false
31094 9392 Acked Write 3 Done Sent Done false
31095 9393 Acked Flush 0 Done Sent Done false
31096 9394 Acked Write 2 Done Sent Done false
31097 9395 Acked Read 2 Done Sent Done false
31098 9396 Acked Write 10 Done Sent Done false
31099 9397 Acked Write 4 Done Sent Done false
31100 9398 Acked Write 9 Done Sent Done false
31101 9399 Acked Read 1 Done Sent Done false
31102 9400 Acked Write 10 Done Sent Done false
31103 9401 Acked Write 10 Done Sent Done false
31104 9402 Acked Write 5 Done Sent Done false
31105 9403 Acked Read 7 Done Sent Done false
31106 9404 Acked Read 1 Done Sent Done false
31107 9405 Acked Read 4 Done Sent Done false
31108 9406 Acked Flush 0 Done Sent Done false
31109 9407 Acked Flush 0 Done Sent Done false
31110 9408 Acked Read 6 Done Sent Done false
31111 9409 Acked Read 4 Done Sent Done false
31112 9410 Acked Write 3 Done Sent Done false
31113 9411 Acked Read 6 Done Sent Done false
31114 9412 Acked Read 7 Done Sent Done false
31115 9413 Acked Read 9 Done Sent Done false
31116 9414 Acked Read 6 Done Sent Done false
31117 9415 Acked Read 2 Done Sent Done false
31118 9416 Acked Read 10 Done Sent Done false
31119 9417 Acked Read 6 Done Sent Done false
31120 9418 Acked Flush 0 Done Sent Done false
31121 9419 Acked Read 3 Done Sent Done false
31122 9420 Acked Write 1 Done Sent Done false
31123 9421 Acked Read 3 Done Sent Done false
31124 9422 Acked Read 5 Done Sent Done false
31125 9423 Acked Read 4 Done Sent Done false
31126 9424 Acked Write 8 Done Sent Done false
31127 9425 Acked Flush 0 Done Sent Done false
31128 9426 Acked Write 2 Done Sent Done false
31129 9427 Acked Write 9 Done Sent Done false
31130 9428 Acked Write 6 Done Sent Done false
31131 9429 Acked Read 6 Done Sent Done false
31132 9430 Acked Write 6 Done Sent Done false
31133 9431 Acked Write 7 Done Sent Done false
31134 9432 Acked Read 10 Done Sent Done false
31135 9433 Acked Read 8 Done Sent Done false
31136 9434 Acked Read 9 Done Sent Done false
31137 9435 Acked Write 3 Done Sent Done false
31138 9436 Acked Read 6 Done Sent Done false
31139 9437 Acked Write 8 Done Sent Done false
31140 9438 Acked Read 10 Done Sent Done false
31141 9439 Acked Read 4 Done Sent Done false
31142 9440 Acked Write 1 Done Sent Done false
31143 9441 Acked Read 4 Done Sent Done false
31144 9442 Acked Read 1 Done Sent Done false
31145 9443 Acked Flush 0 Done Sent Done false
31146 9444 Acked Read 3 Done Sent Done false
31147 9445 Acked Flush 0 Done Sent Done false
31148 9446 Acked Read 1 Done Sent Done false
31149 9447 Acked Write 10 Done Sent Done false
31150 9448 Acked Read 8 Done Sent Done false
31151 9449 Acked Write 5 Done Sent Done false
31152 9450 Acked Read 4 Done Sent Done false
31153 9451 Acked Flush 0 Done Sent Done false
31154 STATES DS:0 DS:1 DS:2 TOTAL
31155 Sent 0 302 0 302
31156 Done 302 0 302 604
31157 Skipped 0 0 0 0
31158 Error 0 0 0 0
31159 Last Flush: 9451 9149 9451
31160 Downstairs last five completed: 9149 9148 9147 9146 9145
31161 Upstairs last five completed: 9451 9450 9449 9448 9447
31162 CLIENT: Up:0 ds:302 act:2
311632025-01-27T19:42:53.600ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311642025-01-27T19:42:53.600ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:47001) = io task client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311652025-01-27T19:42:53.601ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311662025-01-27T19:42:53.602ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311672025-01-27T19:42:53.602ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008820 matches = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311682025-01-27T19:42:53.602ZINFOcrucible: send last flush ID to this DS: 9149 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311692025-01-27T19:42:53.602ZINFOcrucible: Replied this last flush ID: 9149 = downstairs client = 1 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311702025-01-27T19:42:53.602ZINFOcrucible: [1] client re-new 302 jobs since flush 9149 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311712025-01-27T19:42:53.602ZINFOcrucible: [1] Replayed 302 jobs since flush: 9149 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31172 ----------------------------------------------------------------
31173 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
31174 Downstairs last five completed: 9451 9450 9449 9448 9447
31175 Upstairs last five completed: 9451 9450 9449 9448 9447
31176 CLIENT: Up:0 ds:0 act:3
31177 Replay: All jobs finished
31178 Main task has work to do, go find it
31179 got dsc Stop(2)
31180 stop 2
31181 Jan 27 19:42:55.841 INFO request completed[8830] Got stop action so:false kr:false
31182 , latency_us: 412, response_code: 204, uri: /stop/cid/2, method: POST, req_id: a9e5d28d-3f73-4f90-9f57-d0bfc936378a, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
311832025-01-27T19:42:55.842ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
311842025-01-27T19:42:55.842ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31185 Jan 27 19:42:55.842 INFO request completed, latency_us: 127{"msg":"downstairs task for 2 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:42:55.842279476Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
31186 , response_code: 200, uri: /state/cid/2, method: GET, req_id: faa5f01c-b08a-49ad-80f1-51741722a80d, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
311872025-01-27T19:42:55.842ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
31188 [8830] Exited with: ExitStatus(unix_wait_status(9))
31189 [8830][2] reports Exit
31190 Jan 27 19:42:59.845 INFO request completed, latency_us: 330, response_code: 200, uri: /state/cid/2, method: GET, req_id: 8df8dcda-bef0-468e-88c0-a4d6ccdd3003, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
31191 001/300 Read block 2243 len 4096
31192 002/300 Read block 900 len 4096
31193 003/300 Read block 4414 len 28672
31194 004/300 Read block 632 len 20480
31195 005/300 Write block 3269 len 36864 data: 3 4 6 7 8 6 7 7 8
31196 006/300 Write block 3966 len 20480 data: 7 7 7 5 4
31197 007/300 Write block 4545 len 40960 data: 3 3 3 2 3 3 4 4 4 4
31198 008/300 Read block 2540 len 24576
31199 009/300 Write block 2564 len 8192 data: 8 8
31200 010/300 Write block 4281 len 40960 data: 6 5 5 6 5 4 3 4 4 4
31201 011/300 Write block 2274 len 36864 data: 3 4 4 4 5 5 5 6 9
31202 012/300 Read block 4941 len 40960
31203 013/300 Read block 4465 len 36864
31204 014/300 Write block 1445 len 16384 data: 6 7 6 6
31205 015/300 Write block 1413 len 16384 data: 8 7 8 8
31206 016/300 Read block 1218 len 8192
31207 017/300 Write block 2970 len 36864 data: 4 5 6 7 7 5 4 4 5
31208 018/300 Read block 1090 len 32768
31209 019/300 Read block 569 len 8192
31210 020/300 Write block 3230 len 8192 data: 7 6
31211 021/300 Write block 4811 len 8192 data: 8 6
31212 022/300 Write block 28 len 40960 data: 5 4 4 3 3 2 3 4 4 6
31213 023/300 Read block 3089 len 20480
31214 024/300 Flush
31215 025/300 Read block 1957 len 36864
31216 026/300 Write block 246 len 40960 data: 5 4 5 6 5 5 7 8 7 6
31217 027/300 Read block 1287 len 12288
31218 028/300 Write block 1682 len 12288 data: 4 5 5
31219 029/300 Read block 3292 len 20480
31220 030/300 Write block 4092 len 16384 data: 4 4 4 4
31221 031/300 Write block 3186 len 16384 data: 4 6 6 6
31222 032/300 Read block 4218 len 20480
31223 033/300 Write block 1716 len 12288 data: 5 5 5
31224 034/300 Read block 4534 len 12288
31225 035/300 Read block 2239 len 16384
31226 036/300 Write block 947 len 36864 data: 6 5 5 5 4 5 5 5 5
31227 037/300 Read block 2322 len 16384
31228 038/300 Write block 1451 len 16384 data: 6 6 7 7
31229 039/300 Read block 1863 len 16384
31230 040/300 Read block 2488 len 36864
31231 041/300 Write block 1690 len 16384 data: 6 5 4 4
31232 042/300 Flush
31233 043/300 Read block 3183 len 4096
31234 044/300 Flush
31235 045/300 Read block 3459 len 4096
31236 046/300 Flush
31237 047/300 Write block 1855 len 8192 data: 9 8
31238 048/300 Read block 4580 len 8192
31239 049/300 Read block 4657 len 4096
31240 050/300 Write block 3382 len 4096 data: 5
31241 051/300 Read block 4731 len 4096
31242 052/300 Read block 4376 len 20480
31243 053/300 Read block 1223 len 12288
31244 054/300 Read block 4099 len 8192
31245 055/300 Flush
31246 056/300 Flush
31247 057/300 Write block 3097 len 28672 data: 5 8 8 7 8 7 9
31248 058/300 Read block 1262 len 8192
31249 059/300 Read block 642 len 24576
31250 060/300 Read block 3961 len 12288
31251 061/300 Read block 3210 len 40960
31252 062/300 Write block 347 len 16384 data: 6 6 7 7
31253 063/300 Flush
31254 064/300 Read block 4269 len 8192
31255 065/300 Write block 4833 len 40960 data: 8 7 9 8 8 9 8 6 7 8
31256 066/300 Write block 1674 len 36864 data: 8 7 8 5 5 5 4 4 5
31257 067/300 Read block 3595 len 24576
31258 068/300 Read block 1508 len 8192
31259 069/300 Write block 4317 len 32768 data: 3 3 3 4 5 4 5 5
31260 070/300 Write block 3576 len 20480 data: 4 3 4 4 5
31261 071/300 Read block 2196 len 40960
31262 072/300 Read block 1704 len 16384
31263 073/300 Read block 3288 len 24576
31264 074/300 Write block 2911 len 12288 data: 4 5 5
31265 075/300 Write block 1723 len 12288 data: 6 5 5
31266 076/300 Read block 1535 len 40960
31267 077/300 Read block 3941 len 32768
31268 078/300 Write block 2271 len 24576 data: 4 3 3 4 5 5
31269 079/300 Write block 2602 len 40960 data: 6 6 7 7 8 8 7 6 6 8
31270 080/300 Read block 3622 len 20480
31271 081/300 Flush
31272 082/300 Read block 3637 len 12288
31273 083/300 Read block 4297 len 24576
31274 084/300 Read block 1842 len 36864
31275 085/300 Read block 74 len 4096
31276 086/300 Read block 1426 len 8192
31277 087/300 Write block 151 len 8192 data: 2 2
31278 088/300 Flush
31279 089/300 Read block 3506 len 4096
31280 090/300 Read block 3727 len 40960
31281 091/300 Write block 4959 len 16384 data: 7 8 8 9
31282 092/300 Read block 493 len 4096
31283 093/300 Read block 4478 len 32768
31284 094/300 Write block 3494 len 24576 data: 5 5 4 3 3 4
31285 095/300 Write block 3317 len 8192 data: 5 4
31286 096/300 Write block 2095 len 40960 data: 3 4 3 3 4 3 3 4 4 5
31287 097/300 Write block 1413 len 12288 data: 9 8 9
31288 098/300 Write block 3397 len 16384 data: 8 7 5 5
31289 099/300 Read block 1352 len 16384
31290 100/300 Write block 484 len 40960 data: 2 3 3 3 4 5 5 4 5 5
31291 101/300 Read block 3273 len 20480
31292 102/300 Write block 4981 len 24576 data: 4 5 4 3 5 5
31293 103/300 Read block 322 len 32768
31294 104/300 Flush
31295 105/300 Read block 4334 len 36864
31296 106/300 Read block 4259 len 36864
31297 107/300 Read block 3004 len 40960
31298 108/300 Write block 2778 len 40960 data: 5 6 5 5 7 7 7 7 6 5
31299 109/300 Flush
31300 110/300 Write block 3047 len 4096 data: 7
31301 111/300 Flush
31302 112/300 Read block 2883 len 40960
31303 113/300 Flush
31304 114/300 Write block 3342 len 4096 data: 3
31305 115/300 Read block 3370 len 32768
31306 116/300 Read block 3178 len 32768
31307 117/300 Flush
31308 118/300 Write block 3014 len 16384 data: 4 4 5 4
31309 119/300 Read block 3060 len 20480
31310 120/300 Write block 595 len 16384 data: 6 6 7 7
31311 121/300 Read block 4994 len 20480
31312 122/300 Read block 1656 len 40960
31313 123/300 Write block 4652 len 8192 data: 7 7
31314 124/300 Write block 1180 len 32768 data: 2 2 2 4 5 7 6 6
31315 125/300 Flush
31316 126/300 Read block 1400 len 36864
31317 127/300 Read block 4085 len 8192
31318 128/300 Write block 3495 len 12288 data: 6 5 4
31319 129/300 Read block 4311 len 40960
31320 130/300 Read block 4238 len 12288
31321 131/300 Read block 1822 len 40960
31322 132/300 Write block 819 len 20480 data: 9 8 7 6 6
31323 133/300 Write block 4019 len 12288 data: 5 5 5
31324 134/300 Read block 4138 len 24576
31325 135/300 Write block 2038 len 8192 data: 6 5
31326 136/300 Write block 4967 len 4096 data: 6
31327 137/300 Write block 2331 len 12288 data: 5 5 5
31328 138/300 Read block 3945 len 36864
31329 139/300 Read block 3565 len 12288
31330 140/300 Write block 1916 len 12288 data: 3 3 4
31331 141/300 Write block 4981 len 36864 data: 5 6 5 4 6 6 6 6 8
31332 142/300 Read block 2925 len 8192
31333 143/300 Write block 3364 len 36864 data: 7 7 7 7 6 6 7 7 9
31334 144/300 Write block 4486 len 24576 data: 5 5 4 4 5 5
31335 145/300 Read block 3659 len 36864
31336 146/300 Write block 4173 len 28672 data: 8 5 5 5 6 6 3
31337 147/300 Write block 1427 len 8192 data: 7 5
31338 148/300 Read block 4138 len 40960
31339 149/300 Write block 4138 len 8192 data: 6 6
31340 150/300 Write block 4513 len 12288 data: 6 7 5
31341 151/300 Read block 2825 len 8192
31342 152/300 Write block 1997 len 28672 data: 8 9 9 6 5 6 5
31343 153/300 Write block 1756 len 28672 data: 5 5 4 3 3 4 4
31344 154/300 Write block 6 len 32768 data: 4 2 2 3 3 3 4 4
31345 155/300 Read block 1970 len 36864
31346 156/300 Write block 3647 len 40960 data: 6 7 8 7 7 5 5 5 5 3
31347 157/300 Read block 1325 len 20480
31348 158/300 Read block 3491 len 24576
31349 159/300 Write block 3285 len 16384 data: 4 4 5 4
31350 160/300 Write block 326 len 16384 data: 7 8 7 7
31351 161/300 Read block 537 len 16384
31352 162/300 Write block 662 len 8192 data: 5 4
31353 163/300 Write block 1284 len 20480 data: 4 3 3 3 3
31354 164/300 Flush
31355 165/300 Read block 1772 len 40960
31356 166/300 Read block 2821 len 36864
31357 167/300 Read block 1201 len 28672
31358 168/300 Write block 878 len 20480 data: 6 6 5 5 4
31359 169/300 Flush
31360 170/300 Read block 3919 len 16384
31361 171/300 Read block 3135 len 36864
31362 172/300 Read block 589 len 40960
31363 173/300 Read block 3968 len 16384
31364 174/300 Write block 2343 len 36864 data: 6 6 5 6 7 6 6 7 7
31365 175/300 Flush
31366 176/300 Read block 221 len 20480
31367 177/300 Write block 3298 len 40960 data: 4 3 2 4 5 6 6 6 6 5
31368 178/300 Flush
31369 179/300 Write block 4292 len 8192 data: 7 5
31370 180/300 Read block 177 len 32768
31371 181/300 Read block 4033 len 40960
31372 182/300 Write block 1274 len 32768 data: 3 4 5 5 4 4 3 3
31373 183/300 Read block 263 len 28672
31374 184/300 Write block 1688 len 24576 data: 4 5 7 6 5 5
31375 185/300 Write block 4957 len 36864 data: 5 7 8 9 9 10 6 6 3
31376 186/300 Write block 2213 len 4096 data: 3
31377 187/300 Flush
31378 188/300 Read block 3801 len 20480
31379 189/300 Read block 4667 len 4096
31380 190/300 Write block 2162 len 4096 data: 7
31381 191/300 Read block 1268 len 12288
31382 192/300 Read block 2058 len 28672
31383 193/300 Read block 3028 len 12288
31384 194/300 Read block 1606 len 32768
31385 195/300 Write block 209 len 36864 data: 8 7 7 5 4 6 6 4 5
31386 196/300 Flush
31387 197/300 Read block 3281 len 36864
31388 198/300 Write block 4267 len 28672 data: 7 10 8 8 11 8 8
31389 199/300 Write block 2095 len 16384 data: 4 5 4 4
31390 200/300 Read block 4848 len 4096
31391 201/300 Flush
31392 202/300 Write block 3408 len 4096 data: 5
31393 203/300 Write block 4072 len 8192 data: 5 5
31394 204/300 Read block 294 len 24576
31395 205/300 Read block 4478 len 20480
31396 206/300 Read block 1746 len 36864
31397 207/300 Write block 330 len 36864 data: 6 5 5 3 3 3 4 5 6
31398 208/300 Read block 3331 len 8192
31399 209/300 Write block 2858 len 28672 data: 5 4 5 5 6 6 6
31400 210/300 Read block 2771 len 12288
31401 211/300 Read block 1922 len 32768
31402 212/300 Flush
31403 213/300 Write block 611 len 40960 data: 5 4 5 5 4 4 3 3 4 3
31404 214/300 Read block 303 len 20480
31405 215/300 Write block 1330 len 16384 data: 6 5 7 8
31406 216/300 Read block 2599 len 12288
31407 217/300 Write block 1947 len 16384 data: 5 5 4 4
31408 218/300 Read block 3871 len 36864
31409 219/300 Write block 4052 len 36864 data: 3 4 4 5 7 6 6 6 4
31410 220/300 Read block 2929 len 4096
31411 221/300 Read block 1980 len 12288
31412 222/300 Write block 4939 len 28672 data: 7 6 6 6 7 7 5
31413 223/300 Read block 3215 len 12288
31414 224/300 Write block 146 len 32768 data: 3 3 4 3 3 3 3 4
31415 225/300 Read block 1671 len 32768
31416 226/300 Write block 4069 len 28672 data: 4 4 4 6 6 5 4
31417 227/300 Read block 204 len 32768
31418 228/300 Read block 219 len 8192
31419 229/300 Read block 3364 len 8192
31420 230/300 Read block 104 len 12288
31421 231/300 Read block 109 len 36864
31422 232/300 Read block 1338 len 32768
31423 233/300 Read block 1264 len 4096
31424 234/300 Write block 1327 len 32768 data: 6 6 7 7 6 8 9 7
31425 235/300 Write block 2393 len 8192 data: 6 5
31426 236/300 Write block 4234 len 20480 data: 6 5 6 7 7
31427 237/300 Read block 2439 len 40960
31428 238/300 Read block 3859 len 40960
31429 239/300 Write block 4661 len 8192 data: 3 3
31430 240/300 Flush
31431 241/300 Write block 2454 len 32768 data: 6 5 4 5 3 4 4 3
31432 242/300 Read block 1010 len 24576
31433 243/300 Write block 1825 len 8192 data: 5 5
31434 244/300 Write block 3102 len 32768 data: 8 10 11 9 10 10 10 8
31435 245/300 Write block 2490 len 28672 data: 6 6 6 6 6 5 4
31436 246/300 Read block 1367 len 40960
31437 247/300 Read block 3433 len 24576
31438 248/300 Read block 1170 len 16384
31439 249/300 Read block 3156 len 16384
31440 250/300 Read block 4705 len 16384
31441 251/300 Write block 2549 len 36864 data: 3 5 5 5 5 5 7 7 9
31442 252/300 Read block 1112 len 36864
31443 253/300 Read block 4713 len 4096
31444 254/300 Read block 447 len 8192
31445 255/300 Write block 3169 len 12288 data: 5 5 5
31446 256/300 Read block 1399 len 40960
31447 257/300 Write block 1542 len 32768 data: 6 8 8 6 3 3 4 4
31448 258/300 Write block 1254 len 40960 data: 5 4 5 4 5 4 4 4 5 6
31449 259/300 Read block 2279 len 12288
31450 260/300 Read block 3098 len 24576
31451 261/300 Write block 2341 len 16384 data: 7 6 7 7
31452 262/300 Read block 3993 len 20480
31453 263/300 Flush
31454 264/300 Read block 4838 len 4096
31455 265/300 Read block 851 len 20480
31456 266/300 Read block 332 len 24576
31457 267/300 Write block 292 len 32768 data: 5 6 6 6 6 6 7 7
31458 268/300 Read block 429 len 36864
31459 269/300 Write block 824 len 20480 data: 6 6 6 5 7
31460 270/300 Write block 578 len 36864 data: 6 6 8 7 7 9 6 5 4
31461 271/300 Read block 2770 len 28672
31462 272/300 Read block 3384 len 4096
31463 273/300 Read block 3004 len 36864
31464 274/300 Read block 3926 len 8192
31465 275/300 Read block 2266 len 16384
31466 276/300 Write block 2155 len 36864 data: 8 10 11 10 8 7 8 8 6
31467 277/300 Read block 3559 len 32768
31468 278/300 Read block 692 len 20480
31469 279/300 Read block 443 len 8192
31470 280/300 Read block 3682 len 12288
31471 281/300 Read block 4871 len 40960
31472 282/300 Read block 4346 len 4096
31473 283/300 Write block 1651 len 12288 data: 4 5 5
31474 284/300 Read block 4848 len 20480
31475 285/300 Read block 3911 len 8192
31476 286/300 Write block 3706 len 36864 data: 7 8 9 9 8 9 8 8 8
31477 287/300 Read block 2243 len 4096
31478 288/300 Flush
31479 289/300 Write block 2803 len 8192 data: 5 5
31480 290/300 Write block 235 len 24576 data: 7 5 5 5 5 5
31481 291/300 Write block 1721 len 8192 data: 6 7
31482 292/300 Read block 422 len 12288
31483 293/300 Read block 4589 len 24576
31484 294/300 Read block 2466 len 28672
31485 295/300 Read block 1646 len 16384
31486 296/300 Write block 353 len 28672 data: 10 10 8 9 7 7 5
31487 297/300 Read block 2315 len 12288
31488 298/300 Read block 838 len 16384
31489 299/300 Flush
31490 300/300 Read block 3805 len 12288
31491 301/300 Read block 4505 len 40960
31492 Main task has work to do, go find it
31493 got dsc Start(2)
31494 start 2
31495 Jan 27 19:43:00.086 INFO request completed, latency_us: 164, response_code: 204, uri: /start/cid/2, method: POST, req_id: 7faf77d4-2acf-4e55-9cd3-4d22394ffc48, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
31496 Replay: started 2, returned:Ok(())
31497 ----------------------------------------------------------------
31498 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
31499 States: Active Active Offline
31500 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31501 9452 Acked Read 1 Done Done Sent false
31502 9453 Acked Read 1 Done Done Sent false
31503 9454 Acked Read 7 Done Done Sent false
31504 9455 Acked Read 5 Done Done Sent false
31505 9456 Acked Write 9 Done Done Sent false
31506 9457 Acked Write 5 Done Done Sent false
31507 9458 Acked Write 10 Done Done Sent false
31508 9459 Acked Read 6 Done Done Sent false
31509 9460 Acked Write 2 Done Done Sent false
31510 9461 Acked Write 10 Done Done Sent false
31511 9462 Acked Write 9 Done Done Sent false
31512 9463 Acked Read 10 Done Done Sent false
31513 9464 Acked Read 9 Done Done Sent false
31514 9465 Acked Write 4 Done Done Sent false
31515 9466 Acked Write 4 Done Done Sent false
31516 9467 Acked Read 2 Done Done Sent false
31517 9468 Acked Write 9 Done Done Sent false
31518 9469 Acked Read 8 Done Done Sent false
31519 9470 Acked Read 2 Done Done Sent false
31520 9471 Acked Write 2 Done Done Sent false
31521 9472 Acked Write 2 Done Done Sent false
31522 9473 Acked Write 10 Done Done Sent false
31523 9474 Acked Read 5 Done Done Sent false
31524 9475 Acked Flush 0 Done Done Sent false
31525 9476 Acked Read 9 Done Done Sent false
31526 9477 Acked Write 10 Done Done Sent false
31527 9478 Acked Read 3 Done Done Sent false
31528 9479 Acked Write 3 Done Done Sent false
31529 9480 Acked Read 5 Done Done Sent false
31530 9481 Acked Write 4 Done Done Sent false
31531 9482 Acked Write 4 Done Done Sent false
31532 9483 Acked Read 5 Done Done Sent false
31533 9484 Acked Write 3 Done Done Sent false
31534 9485 Acked Read 3 Done Done Sent false
31535 9486 Acked Read 4 Done Done Sent false
31536 9487 Acked Write 9 Done Done Sent false
31537 9488 Acked Read 4 Done Done Sent false
31538 9489 Acked Write 4 Done Done Sent false
31539 9490 Acked Read 4 Done Done Sent false
31540 9491 Acked Read 9 Done Done Sent false
31541 9492 Acked Write 4 Done Done Sent false
31542 9493 Acked Flush 0 Done Done Sent false
31543 9494 Acked Read 1 Done Done Sent false
31544 9495 Acked Flush 0 Done Done Sent false
31545 9496 Acked Read 1 Done Done Sent false
31546 9497 Acked Flush 0 Done Done Sent false
31547 9498 Acked Write 2 Done Done Sent false
31548 9499 Acked Read 2 Done Done Sent false
31549 9500 Acked Read 1 Done Done Sent false
31550 9501 Acked Write 1 Done Done Sent false
31551 9502 Acked Read 1 Done Done Sent false
31552 9503 Acked Read 5 Done Done Sent false
31553 9504 Acked Read 3 Done Done Sent false
31554 9505 Acked Read 2 Done Done Sent false
31555 9506 Acked Flush 0 Done Done Sent false
31556 9507 Acked Flush 0 Done Done Sent false
31557 9508 Acked Write 7 Done Done Sent false
31558 9509 Acked Read 2 Done Done Sent false
31559 9510 Acked Read 6 Done Done Sent false
31560 9511 Acked Read 3 Done Done Sent false
31561 9512 Acked Read 10 Done Done Sent false
31562 9513 Acked Write 4 Done Done Sent false
31563 9514 Acked Flush 0 Done Done Sent false
31564 9515 Acked Read 2 Done Done Sent false
31565 9516 Acked Write 10 Done Done Sent false
31566 9517 Acked Write 9 Done Done Sent false
31567 9518 Acked Read 6 Done Done Sent false
31568 9519 Acked Read 2 Done Done Sent false
31569 9520 Acked Write 8 Done Done Sent false
31570 9521 Acked Write 5 Done Done Sent false
31571 9522 Acked Read 10 Done Done Sent false
31572 9523 Acked Read 4 Done Done Sent false
31573 9524 Acked Read 6 Done Done Sent false
31574 9525 Acked Write 3 Done Done Sent false
31575 9526 Acked Write 3 Done Done Sent false
31576 9527 Acked Read 10 Done Done Sent false
31577 9528 Acked Read 8 Done Done Sent false
31578 9529 Acked Write 6 Done Done Sent false
31579 9530 Acked Write 10 Done Done Sent false
31580 9531 Acked Read 5 Done Done Sent false
31581 9532 Acked Flush 0 Done Done Sent false
31582 9533 Acked Read 3 Done Done Sent false
31583 9534 Acked Read 6 Done Done Sent false
31584 9535 Acked Read 9 Done Done Sent false
31585 9536 Acked Read 1 Done Done Sent false
31586 9537 Acked Read 2 Done Done Sent false
31587 9538 Acked Write 2 Done Done Sent false
31588 9539 Acked Flush 0 Done Done Sent false
31589 9540 Acked Read 1 Done Done Sent false
31590 9541 Acked Read 10 Done Done Sent false
31591 9542 Acked Write 4 Done Done Sent false
31592 9543 Acked Read 1 Done Done Sent false
31593 9544 Acked Read 8 Done Done Sent false
31594 9545 Acked Write 6 Done Done Sent false
31595 9546 Acked Write 2 Done Done Sent false
31596 9547 Acked Write 10 Done Done Sent false
31597 9548 Acked Write 3 Done Done Sent false
31598 9549 Acked Write 4 Done Done Sent false
31599 9550 Acked Read 4 Done Done Sent false
31600 9551 Acked Write 10 Done Done Sent false
31601 9552 Acked Read 5 Done Done Sent false
31602 9553 Acked Write 6 Done Done Sent false
31603 9554 Acked Read 8 Done Done Sent false
31604 9555 Acked Flush 0 Done Done Sent false
31605 9556 Acked Read 9 Done Done Sent false
31606 9557 Acked Read 9 Done Done Sent false
31607 9558 Acked Read 10 Done Done Sent false
31608 9559 Acked Write 10 Done Done Sent false
31609 9560 Acked Flush 0 Done Done Sent false
31610 9561 Acked Write 1 Done Done Sent false
31611 9562 Acked Flush 0 Done Done Sent false
31612 9563 Acked Read 10 Done Done Sent false
31613 9564 Acked Flush 0 Done Done Sent false
31614 9565 Acked Write 1 Done Done Sent false
31615 9566 Acked Read 8 Done Done Sent false
31616 9567 Acked Read 8 Done Done Sent false
31617 9568 Acked Flush 0 Done Done Sent false
31618 9569 Acked Write 4 Done Done Sent false
31619 9570 Acked Read 5 Done Done Sent false
31620 9571 Acked Write 4 Done Done Sent false
31621 9572 Acked Read 5 Done Done Sent false
31622 9573 Acked Read 10 Done Done Sent false
31623 9574 Acked Write 2 Done Done Sent false
31624 9575 Acked Write 8 Done Done Sent false
31625 9576 Acked Flush 0 Done Done Sent false
31626 9577 Acked Read 9 Done Done Sent false
31627 9578 Acked Read 2 Done Done Sent false
31628 9579 Acked Write 3 Done Done Sent false
31629 9580 Acked Read 10 Done Done Sent false
31630 9581 Acked Read 3 Done Done Sent false
31631 9582 Acked Read 10 Done Done Sent false
31632 9583 Acked Write 5 Done Done Sent false
31633 9584 Acked Write 3 Done Done Sent false
31634 9585 Acked Read 6 Done Done Sent false
31635 9586 Acked Write 2 Done Done Sent false
31636 9587 Acked Write 1 Done Done Sent false
31637 9588 Acked Write 3 Done Done Sent false
31638 9589 Acked Read 9 Done Done Sent false
31639 9590 Acked Read 3 Done Done Sent false
31640 9591 Acked Write 3 Done Done Sent false
31641 9592 Acked Write 9 Done Done Sent false
31642 9593 Acked Read 2 Done Done Sent false
31643 9594 Acked Write 9 Done Done Sent false
31644 9595 Acked Write 6 Done Done Sent false
31645 9596 Acked Read 9 Done Done Sent false
31646 9597 Acked Write 7 Done Done Sent false
31647 9598 Acked Write 2 Done Done Sent false
31648 9599 Acked Read 10 Done Done Sent false
31649 9600 Acked Write 2 Done Done Sent false
31650 9601 Acked Write 3 Done Done Sent false
31651 9602 Acked Read 2 Done Done Sent false
31652 9603 Acked Write 7 Done Done Sent false
31653 9604 Acked Write 7 Done Done Sent false
31654 9605 Acked Write 8 Done Done Sent false
31655 9606 Acked Read 9 Done Done Sent false
31656 9607 Acked Write 10 Done Done Sent false
31657 9608 Acked Read 5 Done Done Sent false
31658 9609 Acked Read 6 Done Done Sent false
31659 9610 Acked Write 4 Done Done Sent false
31660 9611 Acked Write 4 Done Done Sent false
31661 9612 Acked Read 4 Done Done Sent false
31662 9613 Acked Write 2 Done Done Sent false
31663 9614 Acked Write 5 Done Done Sent false
31664 9615 Acked Flush 0 Done Done Sent false
31665 9616 Acked Read 10 Done Done Sent false
31666 9617 Acked Read 9 Done Done Sent false
31667 9618 Acked Read 7 Done Done Sent false
31668 9619 Acked Write 5 Done Done Sent false
31669 9620 Acked Flush 0 Done Done Sent false
31670 9621 Acked Read 4 Done Done Sent false
31671 9622 Acked Read 9 Done Done Sent false
31672 9623 Acked Read 10 Done Done Sent false
31673 9624 Acked Read 4 Done Done Sent false
31674 9625 Acked Write 9 Done Done Sent false
31675 9626 Acked Flush 0 Done Done Sent false
31676 9627 Acked Read 5 Done Done Sent false
31677 9628 Acked Write 10 Done Done Sent false
31678 9629 Acked Flush 0 Done Done Sent false
31679 9630 Acked Write 2 Done Done Sent false
31680 9631 Acked Read 8 Done Done Sent false
31681 9632 Acked Read 10 Done Done Sent false
31682 9633 Acked Write 8 Done Done Sent false
31683 9634 Acked Read 7 Done Done Sent false
31684 9635 Acked Write 6 Done Done Sent false
31685 9636 Acked Write 9 Done Done Sent false
31686 9637 Acked Write 1 Done Done Sent false
31687 9638 Acked Flush 0 Done Done Sent false
31688 9639 Acked Read 5 Done Done Sent false
31689 9640 Acked Read 1 Done Done Sent false
31690 9641 Acked Write 1 Done Done Sent false
31691 9642 Acked Read 3 Done Done Sent false
31692 9643 Acked Read 7 Done Done Sent false
31693 9644 Acked Read 3 Done Done Sent false
31694 9645 Acked Read 8 Done Done Sent false
31695 9646 Acked Write 9 Done Done Sent false
31696 9647 Acked Flush 0 Done Done Sent false
31697 9648 Acked Read 9 Done Done Sent false
31698 9649 Acked Write 7 Done Done Sent false
31699 9650 Acked Write 4 Done Done Sent false
31700 9651 Acked Read 1 Done Done Sent false
31701 9652 Acked Flush 0 Done Done Sent false
31702 9653 Acked Write 1 Done Done Sent false
31703 9654 Acked Write 2 Done Done Sent false
31704 9655 Acked Read 6 Done Done Sent false
31705 9656 Acked Read 5 Done Done Sent false
31706 9657 Acked Read 9 Done Done Sent false
31707 9658 Acked Write 9 Done Done Sent false
31708 9659 Acked Read 2 Done Done Sent false
31709 9660 Acked Write 7 Done Done Sent false
31710 9661 Acked Read 3 Done Done Sent false
31711 9662 Acked Read 8 Done Done Sent false
31712 9663 Acked Flush 0 Done Done Sent false
31713 9664 Acked Write 10 Done Done Sent false
31714 9665 Acked Read 5 Done Done Sent false
31715 9666 Acked Write 4 Done Done Sent false
31716 9667 Acked Read 3 Done Done Sent false
31717 9668 Acked Write 4 Done Done Sent false
31718 9669 Acked Read 9 Done Done Sent false
31719 9670 Acked Write 9 Done Done Sent false
31720 9671 Acked Read 1 Done Done Sent false
31721 9672 Acked Read 3 Done Done Sent false
31722 9673 Acked Write 7 Done Done Sent false
31723 9674 Acked Read 3 Done Done Sent false
31724 9675 Acked Write 8 Done Done Sent false
31725 9676 Acked Read 8 Done Done Sent false
31726 9677 Acked Write 7 Done Done Sent false
31727 9678 Acked Read 8 Done Done Sent false
31728 9679 Acked Read 2 Done Done Sent false
31729 9680 Acked Read 2 Done Done Sent false
31730 9681 Acked Read 3 Done Done Sent false
31731 9682 Acked Read 9 Done Done Sent false
31732 9683 Acked Read 8 Done Done Sent false
31733 9684 Acked Read 1 Done Done Sent false
31734 9685 Acked Write 8 Done Done Sent false
31735 9686 Acked Write 2 Done Done Sent false
31736 9687 Acked Write 5 Done Done Sent false
31737 9688 Acked Read 10 Done Done Sent false
31738 9689 Acked Read 10 Done Done Sent false
31739 9690 Acked Write 2 Done Done Sent false
31740 9691 Acked Flush 0 Done Done Sent false
31741 9692 Acked Write 8 Done Done Sent false
31742 9693 Acked Read 6 Done Done Sent false
31743 9694 Acked Write 2 Done Done Sent false
31744 9695 Acked Write 8 Done Done Sent false
31745 9696 Acked Write 7 Done Done Sent false
31746 9697 Acked Read 10 Done Done Sent false
31747 9698 Acked Read 6 Done Done Sent false
31748 9699 Acked Read 4 Done Done Sent false
31749 9700 Acked Read 4 Done Done Sent false
31750 9701 Acked Read 4 Done Done Sent false
31751 9702 Acked Write 9 Done Done Sent false
31752 9703 Acked Read 9 Done Done Sent false
31753 9704 Acked Read 1 Done Done Sent false
31754 9705 Acked Read 2 Done Done Sent false
31755 9706 Acked Write 3 Done Done Sent false
31756 9707 Acked Read 10 Done Done Sent false
31757 9708 Acked Write 8 Done Done Sent false
31758 9709 Acked Write 10 Done Done Sent false
31759 9710 Acked Read 3 Done Done Sent false
31760 9711 Acked Read 6 Done Done Sent false
31761 9712 Acked Write 4 Done Done Sent false
31762 9713 Acked Read 5 Done Done Sent false
31763 9714 Acked Flush 0 Done Done Sent false
31764 9715 Acked Read 1 Done Done Sent false
31765 9716 Acked Read 5 Done Done Sent false
31766 9717 Acked Read 6 Done Done Sent false
31767 9718 Acked Write 8 Done Done Sent false
31768 9719 Acked Read 9 Done Done Sent false
31769 9720 Acked Write 5 Done Done Sent false
31770 9721 Acked Write 9 Done Done Sent false
31771 9722 Acked Read 7 Done Done Sent false
31772 9723 Acked Read 1 Done Done Sent false
31773 9724 Acked Read 9 Done Done Sent false
31774 9725 Acked Read 2 Done Done Sent false
31775 9726 Acked Read 4 Done Done Sent false
31776 9727 Acked Write 9 Done Done Sent false
31777 9728 Acked Read 8 Done Done Sent false
31778 9729 Acked Read 5 Done Done Sent false
31779 9730 Acked Read 2 Done Done Sent false
31780 9731 Acked Read 3 Done Done Sent false
31781 9732 Acked Read 10 Done Done Sent false
31782 9733 Acked Read 1 Done Done Sent false
31783 9734 Acked Write 3 Done Done Sent false
31784 9735 Acked Read 5 Done Done Sent false
31785 9736 Acked Read 2 Done Done Sent false
31786 9737 Acked Write 9 Done Done Sent false
31787 9738 Acked Read 1 Done Done Sent false
31788 9739 Acked Flush 0 Done Done Sent false
31789 9740 Acked Write 2 Done Done Sent false
31790 9741 Acked Write 6 Done Done Sent false
31791 9742 Acked Write 2 Done Done Sent false
31792 9743 Acked Read 3 Done Done Sent false
31793 9744 Acked Read 6 Done Done Sent false
31794 9745 Acked Read 7 Done Done Sent false
31795 9746 Acked Read 4 Done Done Sent false
31796 9747 Acked Write 7 Done Done Sent false
31797 9748 Acked Read 3 Done Done Sent false
31798 9749 Acked Read 4 Done Done Sent false
31799 9750 Acked Flush 0 Done Done Sent false
31800 9751 Acked Read 3 Done Done Sent false
31801 9752 Acked Read 10 Done Done Sent false
31802 STATES DS:0 DS:1 DS:2 TOTAL
31803 Sent 0 0 301 301
31804 Done 301 301 0 602
31805 Skipped 0 0 0 0
31806 Error 0 0 0 0
31807 Last Flush: 9750 9750 9451
31808 Downstairs last five completed: 9451 9450 9449 9448 9447
31809 Upstairs last five completed: 9752 9751 9750 9749 9748
31810 CLIENT: Up:0 ds:301 act:2
31811 ----------------------------------------------------------------
31812 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
31813 States: Active Active Offline
31814 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
31815 9452 Acked Read 1 Done Done Sent false
31816 9453 Acked Read 1 Done Done Sent false
31817 9454 Acked Read 7 Done Done Sent false
31818 9455 Acked Read 5 Done Done Sent false
31819 9456 Acked Write 9 Done Done Sent false
31820 9457 Acked Write 5 Done Done Sent false
31821 9458 Acked Write 10 Done Done Sent false
31822 9459 Acked Read 6 Done Done Sent false
31823 9460 Acked Write 2 Done Done Sent false
31824 9461 Acked Write 10 Done Done Sent false
31825 9462 Acked Write 9 Done Done Sent false
31826 9463 Acked Read 10 Done Done Sent false
31827 9464 Acked Read 9 Done Done Sent false
31828 9465 Acked Write 4 Done Done Sent false
31829 9466 Acked Write 4 Done Done Sent false
31830 9467 Acked Read 2 Done Done Sent false
31831 9468 Acked Write 9 Done Done Sent false
31832 9469 Acked Read 8 Done Done Sent false
31833 9470 Acked Read 2 Done Done Sent false
31834 9471 Acked Write 2 Done Done Sent false
31835 9472 Acked Write 2 Done Done Sent false
31836 9473 Acked Write 10 Done Done Sent false
31837 9474 Acked Read 5 Done Done Sent false
31838 9475 Acked Flush 0 Done Done Sent false
31839 9476 Acked Read 9 Done Done Sent false
31840 9477 Acked Write 10 Done Done Sent false
31841 9478 Acked Read 3 Done Done Sent false
31842 9479 Acked Write 3 Done Done Sent false
31843 9480 Acked Read 5 Done Done Sent false
31844 9481 Acked Write 4 Done Done Sent false
31845 9482 Acked Write 4 Done Done Sent false
31846 9483 Acked Read 5 Done Done Sent false
31847 9484 Acked Write 3 Done Done Sent false
31848 9485 Acked Read 3 Done Done Sent false
31849 9486 Acked Read 4 Done Done Sent false
31850 9487 Acked Write 9 Done Done Sent false
31851 9488 Acked Read 4 Done Done Sent false
31852 9489 Acked Write 4 Done Done Sent false
31853 9490 Acked Read 4 Done Done Sent false
31854 9491 Acked Read 9 Done Done Sent false
31855 9492 Acked Write 4 Done Done Sent false
31856 9493 Acked Flush 0 Done Done Sent false
31857 9494 Acked Read 1 Done Done Sent false
31858 9495 Acked Flush 0 Done Done Sent false
31859 9496 Acked Read 1 Done Done Sent false
31860 9497 Acked Flush 0 Done Done Sent false
31861 9498 Acked Write 2 Done Done Sent false
31862 9499 Acked Read 2 Done Done Sent false
31863 9500 Acked Read 1 Done Done Sent false
31864 9501 Acked Write 1 Done Done Sent false
31865 9502 Acked Read 1 Done Done Sent false
31866 9503 Acked Read 5 Done Done Sent false
31867 9504 Acked Read 3 Done Done Sent false
31868 9505 Acked Read 2 Done Done Sent false
31869 9506 Acked Flush 0 Done Done Sent false
31870 9507 Acked Flush 0 Done Done Sent false
31871 9508 Acked Write 7 Done Done Sent false
31872 9509 Acked Read 2 Done Done Sent false
31873 9510 Acked Read 6 Done Done Sent false
31874 9511 Acked Read 3 Done Done Sent false
31875 9512 Acked Read 10 Done Done Sent false
31876 9513 Acked Write 4 Done Done Sent false
31877 9514 Acked Flush 0 Done Done Sent false
31878 9515 Acked Read 2 Done Done Sent false
31879 9516 Acked Write 10 Done Done Sent false
31880 9517 Acked Write 9 Done Done Sent false
31881 9518 Acked Read 6 Done Done Sent false
31882 9519 Acked Read 2 Done Done Sent false
31883 9520 Acked Write 8 Done Done Sent false
31884 9521 Acked Write 5 Done Done Sent false
31885 9522 Acked Read 10 Done Done Sent false
31886 9523 Acked Read 4 Done Done Sent false
31887 9524 Acked Read 6 Done Done Sent false
31888 9525 Acked Write 3 Done Done Sent false
31889 9526 Acked Write 3 Done Done Sent false
31890 9527 Acked Read 10 Done Done Sent false
31891 9528 Acked Read 8 Done Done Sent false
31892 9529 Acked Write 6 Done Done Sent false
31893 9530 Acked Write 10 Done Done Sent false
31894 9531 Acked Read 5 Done Done Sent false
31895 9532 Acked Flush 0 Done Done Sent false
31896 9533 Acked Read 3 Done Done Sent false
31897 9534 Acked Read 6 Done Done Sent false
31898 9535 Acked Read 9 Done Done Sent false
31899 9536 Acked Read 1 Done Done Sent false
31900 9537 Acked Read 2 Done Done Sent false
31901 9538 Acked Write 2 Done Done Sent false
31902 9539 Acked Flush 0 Done Done Sent false
31903 9540 Acked Read 1 Done Done Sent false
31904 9541 Acked Read 10 Done Done Sent false
31905 9542 Acked Write 4 Done Done Sent false
31906 9543 Acked Read 1 Done Done Sent false
31907 9544 Acked Read 8 Done Done Sent false
31908 9545 Acked Write 6 Done Done Sent false
31909 9546 Acked Write 2 Done Done Sent false
31910 9547 Acked Write 10 Done Done Sent false
31911 9548 Acked Write 3 Done Done Sent false
31912 9549 Acked Write 4 Done Done Sent false
31913 9550 Acked Read 4 Done Done Sent false
31914 9551 Acked Write 10 Done Done Sent false
31915 9552 Acked Read 5 Done Done Sent false
31916 9553 Acked Write 6 Done Done Sent false
31917 9554 Acked Read 8 Done Done Sent false
31918 9555 Acked Flush 0 Done Done Sent false
31919 9556 Acked Read 9 Done Done Sent false
31920 9557 Acked Read 9 Done Done Sent false
31921 9558 Acked Read 10 Done Done Sent false
31922 9559 Acked Write 10 Done Done Sent false
31923 9560 Acked Flush 0 Done Done Sent false
31924 9561 Acked Write 1 Done Done Sent false
31925 9562 Acked Flush 0 Done Done Sent false
31926 9563 Acked Read 10 Done Done Sent false
31927 9564 Acked Flush 0 Done Done Sent false
31928 9565 Acked Write 1 Done Done Sent false
31929 9566 Acked Read 8 Done Done Sent false
31930 9567 Acked Read 8 Done Done Sent false
31931 9568 Acked Flush 0 Done Done Sent false
31932 9569 Acked Write 4 Done Done Sent false
31933 9570 Acked Read 5 Done Done Sent false
31934 9571 Acked Write 4 Done Done Sent false
31935 9572 Acked Read 5 Done Done Sent false
31936 9573 Acked Read 10 Done Done Sent false
31937 9574 Acked Write 2 Done Done Sent false
31938 9575 Acked Write 8 Done Done Sent false
31939 9576 Acked Flush 0 Done Done Sent false
31940 9577 Acked Read 9 Done Done Sent false
31941 9578 Acked Read 2 Done Done Sent false
31942 9579 Acked Write 3 Done Done Sent false
31943 9580 Acked Read 10 Done Done Sent false
31944 9581 Acked Read 3 Done Done Sent false
31945 9582 Acked Read 10 Done Done Sent false
31946 9583 Acked Write 5 Done Done Sent false
31947 9584 Acked Write 3 Done Done Sent false
31948 9585 Acked Read 6 Done Done Sent false
31949 9586 Acked Write 2 Done Done Sent false
31950 9587 Acked Write 1 Done Done Sent false
31951 9588 Acked Write 3 Done Done Sent false
31952 9589 Acked Read 9 Done Done Sent false
31953 9590 Acked Read 3 Done Done Sent false
31954 9591 Acked Write 3 Done Done Sent false
31955 9592 Acked Write 9 Done Done Sent false
31956 9593 Acked Read 2 Done Done Sent false
31957 9594 Acked Write 9 Done Done Sent false
31958 9595 Acked Write 6 Done Done Sent false
31959 9596 Acked Read 9 Done Done Sent false
31960 9597 Acked Write 7 Done Done Sent false
31961 9598 Acked Write 2 Done Done Sent false
31962 9599 Acked Read 10 Done Done Sent false
31963 9600 Acked Write 2 Done Done Sent false
31964 9601 Acked Write 3 Done Done Sent false
31965 9602 Acked Read 2 Done Done Sent false
31966 9603 Acked Write 7 Done Done Sent false
31967 9604 Acked Write 7 Done Done Sent false
31968 9605 Acked Write 8 Done Done Sent false
31969 9606 Acked Read 9 Done Done Sent false
31970 9607 Acked Write 10 Done Done Sent false
31971 9608 Acked Read 5 Done Done Sent false
31972 9609 Acked Read 6 Done Done Sent false
31973 9610 Acked Write 4 Done Done Sent false
31974 9611 Acked Write 4 Done Done Sent false
31975 9612 Acked Read 4 Done Done Sent false
31976 9613 Acked Write 2 Done Done Sent false
31977 9614 Acked Write 5 Done Done Sent false
31978 9615 Acked Flush 0 Done Done Sent false
31979 9616 Acked Read 10 Done Done Sent false
31980 9617 Acked Read 9 Done Done Sent false
31981 9618 Acked Read 7 Done Done Sent false
31982 9619 Acked Write 5 Done Done Sent false
31983 9620 Acked Flush 0 Done Done Sent false
31984 9621 Acked Read 4 Done Done Sent false
31985 9622 Acked Read 9 Done Done Sent false
31986 9623 Acked Read 10 Done Done Sent false
31987 9624 Acked Read 4 Done Done Sent false
31988 9625 Acked Write 9 Done Done Sent false
31989 9626 Acked Flush 0 Done Done Sent false
31990 9627 Acked Read 5 Done Done Sent false
31991 9628 Acked Write 10 Done Done Sent false
31992 9629 Acked Flush 0 Done Done Sent false
31993 9630 Acked Write 2 Done Done Sent false
31994 9631 Acked Read 8 Done Done Sent false
31995 9632 Acked Read 10 Done Done Sent false
31996 9633 Acked Write 8 Done Done Sent false
31997 9634 Acked Read 7 Done Done Sent false
31998 9635 Acked Write 6 Done Done Sent false
31999 9636 Acked Write 9 Done Done Sent false
32000 9637 Acked Write 1 Done Done Sent false
32001 9638 Acked Flush 0 Done Done Sent false
32002 9639 Acked Read 5 Done Done Sent false
32003 9640 Acked Read 1 Done Done Sent false
32004 9641 Acked Write 1 Done Done Sent false
32005 9642 Acked Read 3 Done Done Sent false
32006 9643 Acked Read 7 Done Done Sent false
32007 9644 Acked Read 3 Done Done Sent false
32008 9645 Acked Read 8 Done Done Sent false
32009 9646 Acked Write 9 Done Done Sent false
32010 9647 Acked Flush 0 Done Done Sent false
32011 9648 Acked Read 9 Done Done Sent false
32012 9649 Acked Write 7 Done Done Sent false
32013 9650 Acked Write 4 Done Done Sent false
32014 9651 Acked Read 1 Done Done Sent false
32015 9652 Acked Flush 0 Done Done Sent false
32016 9653 Acked Write 1 Done Done Sent false
32017 9654 Acked Write 2 Done Done Sent false
32018 9655 Acked Read 6 Done Done Sent false
32019 9656 Acked Read 5 Done Done Sent false
32020 9657 Acked Read 9 Done Done Sent false
32021 9658 Acked Write 9 Done Done Sent false
32022 9659 Acked Read 2 Done Done Sent false
32023 9660 Acked Write 7 Done Done Sent false
32024 9661 Acked Read 3 Done Done Sent false
32025 9662 Acked Read 8 Done Done Sent false
32026 9663 Acked Flush 0 Done Done Sent false
32027 9664 Acked Write 10 Done Done Sent false
32028 9665 Acked Read 5 Done Done Sent false
32029 9666 Acked Write 4 Done Done Sent false
32030 9667 Acked Read 3 Done Done Sent false
32031 9668 Acked Write 4 Done Done Sent false
32032 9669 Acked Read 9 Done Done Sent false
32033 9670 Acked Write 9 Done Done Sent false
32034 9671 Acked Read 1 Done Done Sent false
32035 9672 Acked Read 3 Done Done Sent false
32036 9673 Acked Write 7 Done Done Sent false
32037 9674 Acked Read 3 Done Done Sent false
32038 9675 Acked Write 8 Done Done Sent false
32039 9676 Acked Read 8 Done Done Sent false
32040 9677 Acked Write 7 Done Done Sent false
32041 9678 Acked Read 8 Done Done Sent false
32042 9679 Acked Read 2 Done Done Sent false
32043 9680 Acked Read 2 Done Done Sent false
32044 9681 Acked Read 3 Done Done Sent false
32045 9682 Acked Read 9 Done Done Sent false
32046 9683 Acked Read 8 Done Done Sent false
32047 9684 Acked Read 1 Done Done Sent false
32048 9685 Acked Write 8 Done Done Sent false
32049 9686 Acked Write 2 Done Done Sent false
32050 9687 Acked Write 5 Done Done Sent false
32051 9688 Acked Read 10 Done Done Sent false
32052 9689 Acked Read 10 Done Done Sent false
32053 9690 Acked Write 2 Done Done Sent false
32054 9691 Acked Flush 0 Done Done Sent false
32055 9692 Acked Write 8 Done Done Sent false
32056 9693 Acked Read 6 Done Done Sent false
32057 9694 Acked Write 2 Done Done Sent false
32058 9695 Acked Write 8 Done Done Sent false
32059 9696 Acked Write 7 Done Done Sent false
32060 9697 Acked Read 10 Done Done Sent false
32061 9698 Acked Read 6 Done Done Sent false
32062 9699 Acked Read 4 Done Done Sent false
32063 9700 Acked Read 4 Done Done Sent false
32064 9701 Acked Read 4 Done Done Sent false
32065 9702 Acked Write 9 Done Done Sent false
32066 9703 Acked Read 9 Done Done Sent false
32067 9704 Acked Read 1 Done Done Sent false
32068 9705 Acked Read 2 Done Done Sent false
32069 9706 Acked Write 3 Done Done Sent false
32070 9707 Acked Read 10 Done Done Sent false
32071 9708 Acked Write 8 Done Done Sent false
32072 9709 Acked Write 10 Done Done Sent false
32073 9710 Acked Read 3 Done Done Sent false
32074 9711 Acked Read 6 Done Done Sent false
32075 9712 Acked Write 4 Done Done Sent false
32076 9713 Acked Read 5 Done Done Sent false
32077 9714 Acked Flush 0 Done Done Sent false
32078 9715 Acked Read 1 Done Done Sent false
32079 9716 Acked Read 5 Done Done Sent false
32080 9717 Acked Read 6 Done Done Sent false
32081 9718 Acked Write 8 Done Done Sent false
32082 9719 Acked Read 9 Done Done Sent false
32083 9720 Acked Write 5 Done Done Sent false
32084 9721 Acked Write 9 Done Done Sent false
32085 9722 Acked Read 7 Done Done Sent false
32086 9723 Acked Read 1 Done Done Sent false
32087 9724 Acked Read 9 Done Done Sent false
32088 9725 Acked Read 2 Done Done Sent false
32089 9726 Acked Read 4 Done Done Sent false
32090 9727 Acked Write 9 Done Done Sent false
32091 9728 Acked Read 8 Done Done Sent false
32092 9729 Acked Read 5 Done Done Sent false
32093 9730 Acked Read 2 Done Done Sent false
32094 9731 Acked Read 3 Done Done Sent false
32095 9732 Acked Read 10 Done Done Sent false
32096 9733 Acked Read 1 Done Done Sent false
32097 9734 Acked Write 3 Done Done Sent false
32098 9735 Acked Read 5 Done Done Sent false
32099 9736 Acked Read 2 Done Done Sent false
32100 9737 Acked Write 9 Done Done Sent false
32101 9738 Acked Read 1 Done Done Sent false
32102 9739 Acked Flush 0 Done Done Sent false
32103 9740 Acked Write 2 Done Done Sent false
32104 9741 Acked Write 6 Done Done Sent false
32105 9742 Acked Write 2 Done Done Sent false
32106 9743 Acked Read 3 Done Done Sent false
32107 9744 Acked Read 6 Done Done Sent false
32108 9745 Acked Read 7 Done Done Sent false
32109 9746 Acked Read 4 Done Done Sent false
32110 9747 Acked Write 7 Done Done Sent false
32111 9748 Acked Read 3 Done Done Sent false
32112 9749 Acked Read 4 Done Done Sent false
32113 9750 Acked Flush 0 Done Done Sent false
32114 9751 Acked Read 3 Done Done Sent false
32115 9752 Acked Read 10 Done Done Sent false
32116 9753 Acked Flush 0 Done Done Sent false
32117 STATES DS:0 DS:1 DS:2 TOTAL
32118 Sent 0 0 302 302
32119 Done 302 302 0 604
32120 Skipped 0 0 0 0
32121 Error 0 0 0 0
32122 Last Flush: 9753 9753 9451
32123 Downstairs last five completed: 9451 9450 9449 9448 9447
32124 Upstairs last five completed: 9753 9752 9751 9750 9749
32125 CLIENT: Up:0 ds:302 act:2
321262025-01-27T19:43:05.844ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
321272025-01-27T19:43:05.844ZWARNcrucible: ds_connection connect to 127.0.0.1:8830 failure: Os { code: 146, kind: ConnectionRefused, message: "Connection refused" } = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
321282025-01-27T19:43:05.844ZWARNcrucible: client task is sending Done(ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
321292025-01-27T19:43:05.844ZWARNcrucible: downstairs task for 2 stopped due to ConnectionFailed(Os { code: 146, kind: ConnectionRefused, message: "Connection refused" }) session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
321302025-01-27T19:43:05.844ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
32131 [8830] got start action
32132 [8830] I am going to restart
32133 Starting downstairs at port 8830
32134 Make output file at "/tmp/dsc/downstairs-8830.txt"
32135 [8830][2] reports Starting
32136 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1229)
32137 [8830][2] reports Running
32138 ----------------------------------------------------------------
32139 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32140 States: Active Active Offline
32141 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32142 9452 Acked Read 1 Done Done Sent false
32143 9453 Acked Read 1 Done Done Sent false
32144 9454 Acked Read 7 Done Done Sent false
32145 9455 Acked Read 5 Done Done Sent false
32146 9456 Acked Write 9 Done Done Sent false
32147 9457 Acked Write 5 Done Done Sent false
32148 9458 Acked Write 10 Done Done Sent false
32149 9459 Acked Read 6 Done Done Sent false
32150 9460 Acked Write 2 Done Done Sent false
32151 9461 Acked Write 10 Done Done Sent false
32152 9462 Acked Write 9 Done Done Sent false
32153 9463 Acked Read 10 Done Done Sent false
32154 9464 Acked Read 9 Done Done Sent false
32155 9465 Acked Write 4 Done Done Sent false
32156 9466 Acked Write 4 Done Done Sent false
32157 9467 Acked Read 2 Done Done Sent false
32158 9468 Acked Write 9 Done Done Sent false
32159 9469 Acked Read 8 Done Done Sent false
32160 9470 Acked Read 2 Done Done Sent false
32161 9471 Acked Write 2 Done Done Sent false
32162 9472 Acked Write 2 Done Done Sent false
32163 9473 Acked Write 10 Done Done Sent false
32164 9474 Acked Read 5 Done Done Sent false
32165 9475 Acked Flush 0 Done Done Sent false
32166 9476 Acked Read 9 Done Done Sent false
32167 9477 Acked Write 10 Done Done Sent false
32168 9478 Acked Read 3 Done Done Sent false
32169 9479 Acked Write 3 Done Done Sent false
32170 9480 Acked Read 5 Done Done Sent false
32171 9481 Acked Write 4 Done Done Sent false
32172 9482 Acked Write 4 Done Done Sent false
32173 9483 Acked Read 5 Done Done Sent false
32174 9484 Acked Write 3 Done Done Sent false
32175 9485 Acked Read 3 Done Done Sent false
32176 9486 Acked Read 4 Done Done Sent false
32177 9487 Acked Write 9 Done Done Sent false
32178 9488 Acked Read 4 Done Done Sent false
32179 9489 Acked Write 4 Done Done Sent false
32180 9490 Acked Read 4 Done Done Sent false
32181 9491 Acked Read 9 Done Done Sent false
32182 9492 Acked Write 4 Done Done Sent false
32183 9493 Acked Flush 0 Done Done Sent false
32184 9494 Acked Read 1 Done Done Sent false
32185 9495 Acked Flush 0 Done Done Sent false
32186 9496 Acked Read 1 Done Done Sent false
32187 9497 Acked Flush 0 Done Done Sent false
32188 9498 Acked Write 2 Done Done Sent false
32189 9499 Acked Read 2 Done Done Sent false
32190 9500 Acked Read 1 Done Done Sent false
32191 9501 Acked Write 1 Done Done Sent false
32192 9502 Acked Read 1 Done Done Sent false
32193 9503 Acked Read 5 Done Done Sent false
32194 9504 Acked Read 3 Done Done Sent false
32195 9505 Acked Read 2 Done Done Sent false
32196 9506 Acked Flush 0 Done Done Sent false
32197 9507 Acked Flush 0 Done Done Sent false
32198 9508 Acked Write 7 Done Done Sent false
32199 9509 Acked Read 2 Done Done Sent false
32200 9510 Acked Read 6 Done Done Sent false
32201 9511 Acked Read 3 Done Done Sent false
32202 9512 Acked Read 10 Done Done Sent false
32203 9513 Acked Write 4 Done Done Sent false
32204 9514 Acked Flush 0 Done Done Sent false
32205 9515 Acked Read 2 Done Done Sent false
32206 9516 Acked Write 10 Done Done Sent false
32207 9517 Acked Write 9 Done Done Sent false
32208 9518 Acked Read 6 Done Done Sent false
32209 9519 Acked Read 2 Done Done Sent false
32210 9520 Acked Write 8 Done Done Sent false
32211 9521 Acked Write 5 Done Done Sent false
32212 9522 Acked Read 10 Done Done Sent false
32213 9523 Acked Read 4 Done Done Sent false
32214 9524 Acked Read 6 Done Done Sent false
32215 9525 Acked Write 3 Done Done Sent false
32216 9526 Acked Write 3 Done Done Sent false
32217 9527 Acked Read 10 Done Done Sent false
32218 9528 Acked Read 8 Done Done Sent false
32219 9529 Acked Write 6 Done Done Sent false
32220 9530 Acked Write 10 Done Done Sent false
32221 9531 Acked Read 5 Done Done Sent false
32222 9532 Acked Flush 0 Done Done Sent false
32223 9533 Acked Read 3 Done Done Sent false
32224 9534 Acked Read 6 Done Done Sent false
32225 9535 Acked Read 9 Done Done Sent false
32226 9536 Acked Read 1 Done Done Sent false
32227 9537 Acked Read 2 Done Done Sent false
32228 9538 Acked Write 2 Done Done Sent false
32229 9539 Acked Flush 0 Done Done Sent false
32230 9540 Acked Read 1 Done Done Sent false
32231 9541 Acked Read 10 Done Done Sent false
32232 9542 Acked Write 4 Done Done Sent false
32233 9543 Acked Read 1 Done Done Sent false
32234 9544 Acked Read 8 Done Done Sent false
32235 9545 Acked Write 6 Done Done Sent false
32236 9546 Acked Write 2 Done Done Sent false
32237 9547 Acked Write 10 Done Done Sent false
32238 9548 Acked Write 3 Done Done Sent false
32239 9549 Acked Write 4 Done Done Sent false
32240 9550 Acked Read 4 Done Done Sent false
32241 9551 Acked Write 10 Done Done Sent false
32242 9552 Acked Read 5 Done Done Sent false
32243 9553 Acked Write 6 Done Done Sent false
32244 9554 Acked Read 8 Done Done Sent false
32245 9555 Acked Flush 0 Done Done Sent false
32246 9556 Acked Read 9 Done Done Sent false
32247 9557 Acked Read 9 Done Done Sent false
32248 9558 Acked Read 10 Done Done Sent false
32249 9559 Acked Write 10 Done Done Sent false
32250 9560 Acked Flush 0 Done Done Sent false
32251 9561 Acked Write 1 Done Done Sent false
32252 9562 Acked Flush 0 Done Done Sent false
32253 9563 Acked Read 10 Done Done Sent false
32254 9564 Acked Flush 0 Done Done Sent false
32255 9565 Acked Write 1 Done Done Sent false
32256 9566 Acked Read 8 Done Done Sent false
32257 9567 Acked Read 8 Done Done Sent false
32258 9568 Acked Flush 0 Done Done Sent false
32259 9569 Acked Write 4 Done Done Sent false
32260 9570 Acked Read 5 Done Done Sent false
32261 9571 Acked Write 4 Done Done Sent false
32262 9572 Acked Read 5 Done Done Sent false
32263 9573 Acked Read 10 Done Done Sent false
32264 9574 Acked Write 2 Done Done Sent false
32265 9575 Acked Write 8 Done Done Sent false
32266 9576 Acked Flush 0 Done Done Sent false
32267 9577 Acked Read 9 Done Done Sent false
32268 9578 Acked Read 2 Done Done Sent false
32269 9579 Acked Write 3 Done Done Sent false
32270 9580 Acked Read 10 Done Done Sent false
32271 9581 Acked Read 3 Done Done Sent false
32272 9582 Acked Read 10 Done Done Sent false
32273 9583 Acked Write 5 Done Done Sent false
32274 9584 Acked Write 3 Done Done Sent false
32275 9585 Acked Read 6 Done Done Sent false
32276 9586 Acked Write 2 Done Done Sent false
32277 9587 Acked Write 1 Done Done Sent false
32278 9588 Acked Write 3 Done Done Sent false
32279 9589 Acked Read 9 Done Done Sent false
32280 9590 Acked Read 3 Done Done Sent false
32281 9591 Acked Write 3 Done Done Sent false
32282 9592 Acked Write 9 Done Done Sent false
32283 9593 Acked Read 2 Done Done Sent false
32284 9594 Acked Write 9 Done Done Sent false
32285 9595 Acked Write 6 Done Done Sent false
32286 9596 Acked Read 9 Done Done Sent false
32287 9597 Acked Write 7 Done Done Sent false
32288 9598 Acked Write 2 Done Done Sent false
32289 9599 Acked Read 10 Done Done Sent false
32290 9600 Acked Write 2 Done Done Sent false
32291 9601 Acked Write 3 Done Done Sent false
32292 9602 Acked Read 2 Done Done Sent false
32293 9603 Acked Write 7 Done Done Sent false
32294 9604 Acked Write 7 Done Done Sent false
32295 9605 Acked Write 8 Done Done Sent false
32296 9606 Acked Read 9 Done Done Sent false
32297 9607 Acked Write 10 Done Done Sent false
32298 9608 Acked Read 5 Done Done Sent false
32299 9609 Acked Read 6 Done Done Sent false
32300 9610 Acked Write 4 Done Done Sent false
32301 9611 Acked Write 4 Done Done Sent false
32302 9612 Acked Read 4 Done Done Sent false
32303 9613 Acked Write 2 Done Done Sent false
32304 9614 Acked Write 5 Done Done Sent false
32305 9615 Acked Flush 0 Done Done Sent false
32306 9616 Acked Read 10 Done Done Sent false
32307 9617 Acked Read 9 Done Done Sent false
32308 9618 Acked Read 7 Done Done Sent false
32309 9619 Acked Write 5 Done Done Sent false
32310 9620 Acked Flush 0 Done Done Sent false
32311 9621 Acked Read 4 Done Done Sent false
32312 9622 Acked Read 9 Done Done Sent false
32313 9623 Acked Read 10 Done Done Sent false
32314 9624 Acked Read 4 Done Done Sent false
32315 9625 Acked Write 9 Done Done Sent false
32316 9626 Acked Flush 0 Done Done Sent false
32317 9627 Acked Read 5 Done Done Sent false
32318 9628 Acked Write 10 Done Done Sent false
32319 9629 Acked Flush 0 Done Done Sent false
32320 9630 Acked Write 2 Done Done Sent false
32321 9631 Acked Read 8 Done Done Sent false
32322 9632 Acked Read 10 Done Done Sent false
32323 9633 Acked Write 8 Done Done Sent false
32324 9634 Acked Read 7 Done Done Sent false
32325 9635 Acked Write 6 Done Done Sent false
32326 9636 Acked Write 9 Done Done Sent false
32327 9637 Acked Write 1 Done Done Sent false
32328 9638 Acked Flush 0 Done Done Sent false
32329 9639 Acked Read 5 Done Done Sent false
32330 9640 Acked Read 1 Done Done Sent false
32331 9641 Acked Write 1 Done Done Sent false
32332 9642 Acked Read 3 Done Done Sent false
32333 9643 Acked Read 7 Done Done Sent false
32334 9644 Acked Read 3 Done Done Sent false
32335 9645 Acked Read 8 Done Done Sent false
32336 9646 Acked Write 9 Done Done Sent false
32337 9647 Acked Flush 0 Done Done Sent false
32338 9648 Acked Read 9 Done Done Sent false
32339 9649 Acked Write 7 Done Done Sent false
32340 9650 Acked Write 4 Done Done Sent false
32341 9651 Acked Read 1 Done Done Sent false
32342 9652 Acked Flush 0 Done Done Sent false
32343 9653 Acked Write 1 Done Done Sent false
32344 9654 Acked Write 2 Done Done Sent false
32345 9655 Acked Read 6 Done Done Sent false
32346 9656 Acked Read 5 Done Done Sent false
32347 9657 Acked Read 9 Done Done Sent false
32348 9658 Acked Write 9 Done Done Sent false
32349 9659 Acked Read 2 Done Done Sent false
32350 9660 Acked Write 7 Done Done Sent false
32351 9661 Acked Read 3 Done Done Sent false
32352 9662 Acked Read 8 Done Done Sent false
32353 9663 Acked Flush 0 Done Done Sent false
32354 9664 Acked Write 10 Done Done Sent false
32355 9665 Acked Read 5 Done Done Sent false
32356 9666 Acked Write 4 Done Done Sent false
32357 9667 Acked Read 3 Done Done Sent false
32358 9668 Acked Write 4 Done Done Sent false
32359 9669 Acked Read 9 Done Done Sent false
32360 9670 Acked Write 9 Done Done Sent false
32361 9671 Acked Read 1 Done Done Sent false
32362 9672 Acked Read 3 Done Done Sent false
32363 9673 Acked Write 7 Done Done Sent false
32364 9674 Acked Read 3 Done Done Sent false
32365 9675 Acked Write 8 Done Done Sent false
32366 9676 Acked Read 8 Done Done Sent false
32367 9677 Acked Write 7 Done Done Sent false
32368 9678 Acked Read 8 Done Done Sent false
32369 9679 Acked Read 2 Done Done Sent false
32370 9680 Acked Read 2 Done Done Sent false
32371 9681 Acked Read 3 Done Done Sent false
32372 9682 Acked Read 9 Done Done Sent false
32373 9683 Acked Read 8 Done Done Sent false
32374 9684 Acked Read 1 Done Done Sent false
32375 9685 Acked Write 8 Done Done Sent false
32376 9686 Acked Write 2 Done Done Sent false
32377 9687 Acked Write 5 Done Done Sent false
32378 9688 Acked Read 10 Done Done Sent false
32379 9689 Acked Read 10 Done Done Sent false
32380 9690 Acked Write 2 Done Done Sent false
32381 9691 Acked Flush 0 Done Done Sent false
32382 9692 Acked Write 8 Done Done Sent false
32383 9693 Acked Read 6 Done Done Sent false
32384 9694 Acked Write 2 Done Done Sent false
32385 9695 Acked Write 8 Done Done Sent false
32386 9696 Acked Write 7 Done Done Sent false
32387 9697 Acked Read 10 Done Done Sent false
32388 9698 Acked Read 6 Done Done Sent false
32389 9699 Acked Read 4 Done Done Sent false
32390 9700 Acked Read 4 Done Done Sent false
32391 9701 Acked Read 4 Done Done Sent false
32392 9702 Acked Write 9 Done Done Sent false
32393 9703 Acked Read 9 Done Done Sent false
32394 9704 Acked Read 1 Done Done Sent false
32395 9705 Acked Read 2 Done Done Sent false
32396 9706 Acked Write 3 Done Done Sent false
32397 9707 Acked Read 10 Done Done Sent false
32398 9708 Acked Write 8 Done Done Sent false
32399 9709 Acked Write 10 Done Done Sent false
32400 9710 Acked Read 3 Done Done Sent false
32401 9711 Acked Read 6 Done Done Sent false
32402 9712 Acked Write 4 Done Done Sent false
32403 9713 Acked Read 5 Done Done Sent false
32404 9714 Acked Flush 0 Done Done Sent false
32405 9715 Acked Read 1 Done Done Sent false
32406 9716 Acked Read 5 Done Done Sent false
32407 9717 Acked Read 6 Done Done Sent false
32408 9718 Acked Write 8 Done Done Sent false
32409 9719 Acked Read 9 Done Done Sent false
32410 9720 Acked Write 5 Done Done Sent false
32411 9721 Acked Write 9 Done Done Sent false
32412 9722 Acked Read 7 Done Done Sent false
32413 9723 Acked Read 1 Done Done Sent false
32414 9724 Acked Read 9 Done Done Sent false
32415 9725 Acked Read 2 Done Done Sent false
32416 9726 Acked Read 4 Done Done Sent false
32417 9727 Acked Write 9 Done Done Sent false
32418 9728 Acked Read 8 Done Done Sent false
32419 9729 Acked Read 5 Done Done Sent false
32420 9730 Acked Read 2 Done Done Sent false
32421 9731 Acked Read 3 Done Done Sent false
32422 9732 Acked Read 10 Done Done Sent false
32423 9733 Acked Read 1 Done Done Sent false
32424 9734 Acked Write 3 Done Done Sent false
32425 9735 Acked Read 5 Done Done Sent false
32426 9736 Acked Read 2 Done Done Sent false
32427 9737 Acked Write 9 Done Done Sent false
32428 9738 Acked Read 1 Done Done Sent false
32429 9739 Acked Flush 0 Done Done Sent false
32430 9740 Acked Write 2 Done Done Sent false
32431 9741 Acked Write 6 Done Done Sent false
32432 9742 Acked Write 2 Done Done Sent false
32433 9743 Acked Read 3 Done Done Sent false
32434 9744 Acked Read 6 Done Done Sent false
32435 9745 Acked Read 7 Done Done Sent false
32436 9746 Acked Read 4 Done Done Sent false
32437 9747 Acked Write 7 Done Done Sent false
32438 9748 Acked Read 3 Done Done Sent false
32439 9749 Acked Read 4 Done Done Sent false
32440 9750 Acked Flush 0 Done Done Sent false
32441 9751 Acked Read 3 Done Done Sent false
32442 9752 Acked Read 10 Done Done Sent false
32443 9753 Acked Flush 0 Done Done Sent false
32444 STATES DS:0 DS:1 DS:2 TOTAL
32445 Sent 0 0 302 302
32446 Done 302 302 0 604
32447 Skipped 0 0 0 0
32448 Error 0 0 0 0
32449 Last Flush: 9753 9753 9451
32450 Downstairs last five completed: 9451 9450 9449 9448 9447
32451 Upstairs last five completed: 9753 9752 9751 9750 9749
32452 CLIENT: Up:0 ds:302 act:2
32453 ----------------------------------------------------------------
32454 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
32455 States: Active Active Offline
32456 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
32457 9452 Acked Read 1 Done Done Sent false
32458 9453 Acked Read 1 Done Done Sent false
32459 9454 Acked Read 7 Done Done Sent false
32460 9455 Acked Read 5 Done Done Sent false
32461 9456 Acked Write 9 Done Done Sent false
32462 9457 Acked Write 5 Done Done Sent false
32463 9458 Acked Write 10 Done Done Sent false
32464 9459 Acked Read 6 Done Done Sent false
32465 9460 Acked Write 2 Done Done Sent false
32466 9461 Acked Write 10 Done Done Sent false
32467 9462 Acked Write 9 Done Done Sent false
32468 9463 Acked Read 10 Done Done Sent false
32469 9464 Acked Read 9 Done Done Sent false
32470 9465 Acked Write 4 Done Done Sent false
32471 9466 Acked Write 4 Done Done Sent false
32472 9467 Acked Read 2 Done Done Sent false
32473 9468 Acked Write 9 Done Done Sent false
32474 9469 Acked Read 8 Done Done Sent false
32475 9470 Acked Read 2 Done Done Sent false
32476 9471 Acked Write 2 Done Done Sent false
32477 9472 Acked Write 2 Done Done Sent false
32478 9473 Acked Write 10 Done Done Sent false
32479 9474 Acked Read 5 Done Done Sent false
32480 9475 Acked Flush 0 Done Done Sent false
32481 9476 Acked Read 9 Done Done Sent false
32482 9477 Acked Write 10 Done Done Sent false
32483 9478 Acked Read 3 Done Done Sent false
32484 9479 Acked Write 3 Done Done Sent false
32485 9480 Acked Read 5 Done Done Sent false
32486 9481 Acked Write 4 Done Done Sent false
32487 9482 Acked Write 4 Done Done Sent false
32488 9483 Acked Read 5 Done Done Sent false
32489 9484 Acked Write 3 Done Done Sent false
32490 9485 Acked Read 3 Done Done Sent false
32491 9486 Acked Read 4 Done Done Sent false
32492 9487 Acked Write 9 Done Done Sent false
32493 9488 Acked Read 4 Done Done Sent false
32494 9489 Acked Write 4 Done Done Sent false
32495 9490 Acked Read 4 Done Done Sent false
32496 9491 Acked Read 9 Done Done Sent false
32497 9492 Acked Write 4 Done Done Sent false
32498 9493 Acked Flush 0 Done Done Sent false
32499 9494 Acked Read 1 Done Done Sent false
32500 9495 Acked Flush 0 Done Done Sent false
32501 9496 Acked Read 1 Done Done Sent false
32502 9497 Acked Flush 0 Done Done Sent false
32503 9498 Acked Write 2 Done Done Sent false
32504 9499 Acked Read 2 Done Done Sent false
32505 9500 Acked Read 1 Done Done Sent false
32506 9501 Acked Write 1 Done Done Sent false
32507 9502 Acked Read 1 Done Done Sent false
32508 9503 Acked Read 5 Done Done Sent false
32509 9504 Acked Read 3 Done Done Sent false
32510 9505 Acked Read 2 Done Done Sent false
32511 9506 Acked Flush 0 Done Done Sent false
32512 9507 Acked Flush 0 Done Done Sent false
32513 9508 Acked Write 7 Done Done Sent false
32514 9509 Acked Read 2 Done Done Sent false
32515 9510 Acked Read 6 Done Done Sent false
32516 9511 Acked Read 3 Done Done Sent false
32517 9512 Acked Read 10 Done Done Sent false
32518 9513 Acked Write 4 Done Done Sent false
32519 9514 Acked Flush 0 Done Done Sent false
32520 9515 Acked Read 2 Done Done Sent false
32521 9516 Acked Write 10 Done Done Sent false
32522 9517 Acked Write 9 Done Done Sent false
32523 9518 Acked Read 6 Done Done Sent false
32524 9519 Acked Read 2 Done Done Sent false
32525 9520 Acked Write 8 Done Done Sent false
32526 9521 Acked Write 5 Done Done Sent false
32527 9522 Acked Read 10 Done Done Sent false
32528 9523 Acked Read 4 Done Done Sent false
32529 9524 Acked Read 6 Done Done Sent false
32530 9525 Acked Write 3 Done Done Sent false
32531 9526 Acked Write 3 Done Done Sent false
32532 9527 Acked Read 10 Done Done Sent false
32533 9528 Acked Read 8 Done Done Sent false
32534 9529 Acked Write 6 Done Done Sent false
32535 9530 Acked Write 10 Done Done Sent false
32536 9531 Acked Read 5 Done Done Sent false
32537 9532 Acked Flush 0 Done Done Sent false
32538 9533 Acked Read 3 Done Done Sent false
32539 9534 Acked Read 6 Done Done Sent false
32540 9535 Acked Read 9 Done Done Sent false
32541 9536 Acked Read 1 Done Done Sent false
32542 9537 Acked Read 2 Done Done Sent false
32543 9538 Acked Write 2 Done Done Sent false
32544 9539 Acked Flush 0 Done Done Sent false
32545 9540 Acked Read 1 Done Done Sent false
32546 9541 Acked Read 10 Done Done Sent false
32547 9542 Acked Write 4 Done Done Sent false
32548 9543 Acked Read 1 Done Done Sent false
32549 9544 Acked Read 8 Done Done Sent false
32550 9545 Acked Write 6 Done Done Sent false
32551 9546 Acked Write 2 Done Done Sent false
32552 9547 Acked Write 10 Done Done Sent false
32553 9548 Acked Write 3 Done Done Sent false
32554 9549 Acked Write 4 Done Done Sent false
32555 9550 Acked Read 4 Done Done Sent false
32556 9551 Acked Write 10 Done Done Sent false
32557 9552 Acked Read 5 Done Done Sent false
32558 9553 Acked Write 6 Done Done Sent false
32559 9554 Acked Read 8 Done Done Sent false
32560 9555 Acked Flush 0 Done Done Sent false
32561 9556 Acked Read 9 Done Done Sent false
32562 9557 Acked Read 9 Done Done Sent false
32563 9558 Acked Read 10 Done Done Sent false
32564 9559 Acked Write 10 Done Done Sent false
32565 9560 Acked Flush 0 Done Done Sent false
32566 9561 Acked Write 1 Done Done Sent false
32567 9562 Acked Flush 0 Done Done Sent false
32568 9563 Acked Read 10 Done Done Sent false
32569 9564 Acked Flush 0 Done Done Sent false
32570 9565 Acked Write 1 Done Done Sent false
32571 9566 Acked Read 8 Done Done Sent false
32572 9567 Acked Read 8 Done Done Sent false
32573 9568 Acked Flush 0 Done Done Sent false
32574 9569 Acked Write 4 Done Done Sent false
32575 9570 Acked Read 5 Done Done Sent false
32576 9571 Acked Write 4 Done Done Sent false
32577 9572 Acked Read 5 Done Done Sent false
32578 9573 Acked Read 10 Done Done Sent false
32579 9574 Acked Write 2 Done Done Sent false
32580 9575 Acked Write 8 Done Done Sent false
32581 9576 Acked Flush 0 Done Done Sent false
32582 9577 Acked Read 9 Done Done Sent false
32583 9578 Acked Read 2 Done Done Sent false
32584 9579 Acked Write 3 Done Done Sent false
32585 9580 Acked Read 10 Done Done Sent false
32586 9581 Acked Read 3 Done Done Sent false
32587 9582 Acked Read 10 Done Done Sent false
32588 9583 Acked Write 5 Done Done Sent false
32589 9584 Acked Write 3 Done Done Sent false
32590 9585 Acked Read 6 Done Done Sent false
32591 9586 Acked Write 2 Done Done Sent false
32592 9587 Acked Write 1 Done Done Sent false
32593 9588 Acked Write 3 Done Done Sent false
32594 9589 Acked Read 9 Done Done Sent false
32595 9590 Acked Read 3 Done Done Sent false
32596 9591 Acked Write 3 Done Done Sent false
32597 9592 Acked Write 9 Done Done Sent false
32598 9593 Acked Read 2 Done Done Sent false
32599 9594 Acked Write 9 Done Done Sent false
32600 9595 Acked Write 6 Done Done Sent false
32601 9596 Acked Read 9 Done Done Sent false
32602 9597 Acked Write 7 Done Done Sent false
32603 9598 Acked Write 2 Done Done Sent false
32604 9599 Acked Read 10 Done Done Sent false
32605 9600 Acked Write 2 Done Done Sent false
32606 9601 Acked Write 3 Done Done Sent false
32607 9602 Acked Read 2 Done Done Sent false
32608 9603 Acked Write 7 Done Done Sent false
32609 9604 Acked Write 7 Done Done Sent false
32610 9605 Acked Write 8 Done Done Sent false
32611 9606 Acked Read 9 Done Done Sent false
32612 9607 Acked Write 10 Done Done Sent false
32613 9608 Acked Read 5 Done Done Sent false
32614 9609 Acked Read 6 Done Done Sent false
32615 9610 Acked Write 4 Done Done Sent false
32616 9611 Acked Write 4 Done Done Sent false
32617 9612 Acked Read 4 Done Done Sent false
32618 9613 Acked Write 2 Done Done Sent false
32619 9614 Acked Write 5 Done Done Sent false
32620 9615 Acked Flush 0 Done Done Sent false
32621 9616 Acked Read 10 Done Done Sent false
32622 9617 Acked Read 9 Done Done Sent false
32623 9618 Acked Read 7 Done Done Sent false
32624 9619 Acked Write 5 Done Done Sent false
32625 9620 Acked Flush 0 Done Done Sent false
32626 9621 Acked Read 4 Done Done Sent false
32627 9622 Acked Read 9 Done Done Sent false
32628 9623 Acked Read 10 Done Done Sent false
32629 9624 Acked Read 4 Done Done Sent false
32630 9625 Acked Write 9 Done Done Sent false
32631 9626 Acked Flush 0 Done Done Sent false
32632 9627 Acked Read 5 Done Done Sent false
32633 9628 Acked Write 10 Done Done Sent false
32634 9629 Acked Flush 0 Done Done Sent false
32635 9630 Acked Write 2 Done Done Sent false
32636 9631 Acked Read 8 Done Done Sent false
32637 9632 Acked Read 10 Done Done Sent false
32638 9633 Acked Write 8 Done Done Sent false
32639 9634 Acked Read 7 Done Done Sent false
32640 9635 Acked Write 6 Done Done Sent false
32641 9636 Acked Write 9 Done Done Sent false
32642 9637 Acked Write 1 Done Done Sent false
32643 9638 Acked Flush 0 Done Done Sent false
32644 9639 Acked Read 5 Done Done Sent false
32645 9640 Acked Read 1 Done Done Sent false
32646 9641 Acked Write 1 Done Done Sent false
32647 9642 Acked Read 3 Done Done Sent false
32648 9643 Acked Read 7 Done Done Sent false
32649 9644 Acked Read 3 Done Done Sent false
32650 9645 Acked Read 8 Done Done Sent false
32651 9646 Acked Write 9 Done Done Sent false
32652 9647 Acked Flush 0 Done Done Sent false
32653 9648 Acked Read 9 Done Done Sent false
32654 9649 Acked Write 7 Done Done Sent false
32655 9650 Acked Write 4 Done Done Sent false
32656 9651 Acked Read 1 Done Done Sent false
32657 9652 Acked Flush 0 Done Done Sent false
32658 9653 Acked Write 1 Done Done Sent false
32659 9654 Acked Write 2 Done Done Sent false
32660 9655 Acked Read 6 Done Done Sent false
32661 9656 Acked Read 5 Done Done Sent false
32662 9657 Acked Read 9 Done Done Sent false
32663 9658 Acked Write 9 Done Done Sent false
32664 9659 Acked Read 2 Done Done Sent false
32665 9660 Acked Write 7 Done Done Sent false
32666 9661 Acked Read 3 Done Done Sent false
32667 9662 Acked Read 8 Done Done Sent false
32668 9663 Acked Flush 0 Done Done Sent false
32669 9664 Acked Write 10 Done Done Sent false
32670 9665 Acked Read 5 Done Done Sent false
32671 9666 Acked Write 4 Done Done Sent false
32672 9667 Acked Read 3 Done Done Sent false
32673 9668 Acked Write 4 Done Done Sent false
32674 9669 Acked Read 9 Done Done Sent false
32675 9670 Acked Write 9 Done Done Sent false
32676 9671 Acked Read 1 Done Done Sent false
32677 9672 Acked Read 3 Done Done Sent false
32678 9673 Acked Write 7 Done Done Sent false
32679 9674 Acked Read 3 Done Done Sent false
32680 9675 Acked Write 8 Done Done Sent false
32681 9676 Acked Read 8 Done Done Sent false
32682 9677 Acked Write 7 Done Done Sent false
32683 9678 Acked Read 8 Done Done Sent false
32684 9679 Acked Read 2 Done Done Sent false
32685 9680 Acked Read 2 Done Done Sent false
32686 9681 Acked Read 3 Done Done Sent false
32687 9682 Acked Read 9 Done Done Sent false
32688 9683 Acked Read 8 Done Done Sent false
32689 9684 Acked Read 1 Done Done Sent false
32690 9685 Acked Write 8 Done Done Sent false
32691 9686 Acked Write 2 Done Done Sent false
32692 9687 Acked Write 5 Done Done Sent false
32693 9688 Acked Read 10 Done Done Sent false
32694 9689 Acked Read 10 Done Done Sent false
32695 9690 Acked Write 2 Done Done Sent false
32696 9691 Acked Flush 0 Done Done Sent false
32697 9692 Acked Write 8 Done Done Sent false
32698 9693 Acked Read 6 Done Done Sent false
32699 9694 Acked Write 2 Done Done Sent false
32700 9695 Acked Write 8 Done Done Sent false
32701 9696 Acked Write 7 Done Done Sent false
32702 9697 Acked Read 10 Done Done Sent false
32703 9698 Acked Read 6 Done Done Sent false
32704 9699 Acked Read 4 Done Done Sent false
32705 9700 Acked Read 4 Done Done Sent false
32706 9701 Acked Read 4 Done Done Sent false
32707 9702 Acked Write 9 Done Done Sent false
32708 9703 Acked Read 9 Done Done Sent false
32709 9704 Acked Read 1 Done Done Sent false
32710 9705 Acked Read 2 Done Done Sent false
32711 9706 Acked Write 3 Done Done Sent false
32712 9707 Acked Read 10 Done Done Sent false
32713 9708 Acked Write 8 Done Done Sent false
32714 9709 Acked Write 10 Done Done Sent false
32715 9710 Acked Read 3 Done Done Sent false
32716 9711 Acked Read 6 Done Done Sent false
32717 9712 Acked Write 4 Done Done Sent false
32718 9713 Acked Read 5 Done Done Sent false
32719 9714 Acked Flush 0 Done Done Sent false
32720 9715 Acked Read 1 Done Done Sent false
32721 9716 Acked Read 5 Done Done Sent false
32722 9717 Acked Read 6 Done Done Sent false
32723 9718 Acked Write 8 Done Done Sent false
32724 9719 Acked Read 9 Done Done Sent false
32725 9720 Acked Write 5 Done Done Sent false
32726 9721 Acked Write 9 Done Done Sent false
32727 9722 Acked Read 7 Done Done Sent false
32728 9723 Acked Read 1 Done Done Sent false
32729 9724 Acked Read 9 Done Done Sent false
32730 9725 Acked Read 2 Done Done Sent false
32731 9726 Acked Read 4 Done Done Sent false
32732 9727 Acked Write 9 Done Done Sent false
32733 9728 Acked Read 8 Done Done Sent false
32734 9729 Acked Read 5 Done Done Sent false
32735 9730 Acked Read 2 Done Done Sent false
32736 9731 Acked Read 3 Done Done Sent false
32737 9732 Acked Read 10 Done Done Sent false
32738 9733 Acked Read 1 Done Done Sent false
32739 9734 Acked Write 3 Done Done Sent false
32740 9735 Acked Read 5 Done Done Sent false
32741 9736 Acked Read 2 Done Done Sent false
32742 9737 Acked Write 9 Done Done Sent false
32743 9738 Acked Read 1 Done Done Sent false
32744 9739 Acked Flush 0 Done Done Sent false
32745 9740 Acked Write 2 Done Done Sent false
32746 9741 Acked Write 6 Done Done Sent false
32747 9742 Acked Write 2 Done Done Sent false
32748 9743 Acked Read 3 Done Done Sent false
32749 9744 Acked Read 6 Done Done Sent false
32750 9745 Acked Read 7 Done Done Sent false
32751 9746 Acked Read 4 Done Done Sent false
32752 9747 Acked Write 7 Done Done Sent false
32753 9748 Acked Read 3 Done Done Sent false
32754 9749 Acked Read 4 Done Done Sent false
32755 9750 Acked Flush 0 Done Done Sent false
32756 9751 Acked Read 3 Done Done Sent false
32757 9752 Acked Read 10 Done Done Sent false
32758 9753 Acked Flush 0 Done Done Sent false
32759 STATES DS:0 DS:1 DS:2 TOTAL
32760 Sent 0 0 302 302
32761 Done 302 302 0 604
32762 Skipped 0 0 0 0
32763 Error 0 0 0 0
32764 Last Flush: 9753 9753 9451
32765 Downstairs last five completed: 9451 9450 9449 9448 9447
32766 Upstairs last five completed: 9753 9752 9751 9750 9749
32767 CLIENT: Up:0 ds:302 act:2
327682025-01-27T19:43:15.846ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
327692025-01-27T19:43:15.846ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:50756) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
327702025-01-27T19:43:15.847ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
327712025-01-27T19:43:15.847ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
327722025-01-27T19:43:15.847ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
327732025-01-27T19:43:15.847ZINFOcrucible: send last flush ID to this DS: 9451 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
327742025-01-27T19:43:15.848ZINFOcrucible: Replied this last flush ID: 9451 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
327752025-01-27T19:43:15.848ZINFOcrucible: [2] client re-new 302 jobs since flush 9451 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
327762025-01-27T19:43:15.848ZINFOcrucible: [2] Replayed 302 jobs since flush: 9451 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
32777 ----------------------------------------------------------------
32778 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
32779 Downstairs last five completed: 9753 9752 9751 9750 9749
32780 Upstairs last five completed: 9753 9752 9751 9750 9749
32781 CLIENT: Up:0 ds:0 act:3
32782 Replay: All jobs finished
32783 Main task has work to do, go find it
32784 got dsc Stop(0)
32785 stop 0
32786 [8810] Got stop action so:false kr:false
32787 Jan 27 19:43:16.096 INFO request completed, latency_us: 461, response_code: 204, uri: /stop/cid/0, method: POST, req_id: 934d509f-61e7-4071-a169-9a2251016960, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
32788 Jan 27 19:43:16.097 INFO request completed, latency_us: 127{"msg":"downstairs disconnected","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:43:16.097615413Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
32789 , response_code: 200, uri: /state/cid/0, method: GET, req_id: 95391008-11be-456e-b818-8a1426ed50a6, remote_addr: 127.0.0.{"msg":"client task is sending Done(Finished)","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:43:16.097679595Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"":"io task","client":"0","":"downstairs","session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
32790 1:55047, local_addr: 127.0.0.1:{"msg":"downstairs task for 0 stopped due to Finished","v":0,"name":"crucible","level":40,"time":"2025-01-27T19:43:16.097717411Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1200,"session_id":"edf4bf43-2393-40de-98dd-2a3c00d4e1dd"}
32791 9998
327922025-01-27T19:43:16.097ZINFOcrucible: client task is exiting = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
32793 [8810] Exited with: ExitStatus(unix_wait_status(9))
32794 [8810][0] reports Exit
32795 Jan 27 19:43:20.099 INFO request completed, latency_us: 209, response_code: 200, uri: /state/cid/0, method: GET, req_id: ea210c26-c502-4b2c-b553-9e684603ba09, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
32796 001/300 Read block 2468 len 8192
32797 002/300 Write block 3132 len 24576 data: 8 8 7 7 5 5
32798 003/300 Read block 3367 len 28672
32799 004/300 Read block 2128 len 28672
32800 005/300 Write block 4691 len 28672 data: 6 7 7 7 7 6 5
32801 006/300 Write block 1101 len 4096 data: 7
32802 007/300 Write block 1129 len 20480 data: 9 8 7 7 6
32803 008/300 Write block 2071 len 24576 data: 4 6 6 6 6 8
32804 009/300 Read block 938 len 4096
32805 010/300 Read block 850 len 20480
32806 011/300 Write block 2136 len 12288 data: 6 6 5
32807 012/300 Write block 4544 len 12288 data: 3 4 4
32808 013/300 Write block 4080 len 16384 data: 4 4 4 4
32809 014/300 Write block 4968 len 36864 data: 7 7 8 8 6 6 5 4 3
32810 015/300 Read block 4928 len 4096
32811 016/300 Write block 1390 len 28672 data: 7 8 6 5 7 7 7
32812 017/300 Write block 614 len 20480 data: 6 5 5 4 4
32813 018/300 Read block 297 len 28672
32814 019/300 Write block 2318 len 12288 data: 4 3 4
32815 020/300 Flush
32816 021/300 Read block 2354 len 16384
32817 022/300 Write block 686 len 8192 data: 8 7
32818 023/300 Flush
32819 024/300 Read block 1983 len 4096
32820 025/300 Write block 3262 len 32768 data: 5 5 5 3 3 4 4 4
32821 026/300 Read block 4900 len 28672
32822 027/300 Read block 3322 len 12288
32823 028/300 Read block 3476 len 12288
32824 029/300 Read block 1411 len 16384
32825 030/300 Read block 2756 len 24576
32826 031/300 Read block 4365 len 28672
32827 032/300 Flush
32828 033/300 Read block 1665 len 12288
32829 034/300 Read block 1750 len 8192
32830 035/300 Write block 2800 len 20480 data: 5 6 6 6 6
32831 036/300 Read block 2581 len 4096
32832 037/300 Write block 2198 len 20480 data: 7 7 6 8 7
32833 038/300 Read block 1763 len 36864
32834 039/300 Read block 899 len 28672
32835 040/300 Read block 1674 len 12288
32836 041/300 Read block 802 len 24576
32837 042/300 Read block 950 len 16384
32838 043/300 Write block 3975 len 8192 data: 3 3
32839 044/300 Write block 3285 len 32768 data: 5 5 6 5 2 3 3 6
32840 045/300 Read block 2998 len 20480
32841 046/300 Write block 2801 len 4096 data: 7
32842 047/300 Write block 3059 len 8192 data: 6 8
32843 048/300 Read block 4769 len 24576
32844 049/300 Flush
32845 050/300 Read block 2948 len 20480
32846 051/300 Write block 3572 len 12288 data: 5 5 5
32847 052/300 Write block 1076 len 16384 data: 6 6 6 9
32848 053/300 Write block 3125 len 32768 data: 8 7 7 7 7 6 8 9
32849 054/300 Write block 2973 len 28672 data: 8 8 6 5 5 6 7
32850 055/300 Read block 4802 len 12288
32851 056/300 Read block 2873 len 40960
32852 057/300 Flush
32853 058/300 Write block 2001 len 32768 data: 6 7 6 5 5 5 4 7
32854 059/300 Flush
32855 060/300 Read block 4375 len 8192
32856 061/300 Read block 862 len 8192
32857 062/300 Write block 855 len 36864 data: 5 5 5 7 8 9 11 10 8
32858 063/300 Read block 2608 len 20480
32859 064/300 Write block 2372 len 36864 data: 9 9 7 7 8 9 12 7 7
32860 065/300 Read block 4675 len 40960
32861 066/300 Flush
32862 067/300 Read block 2142 len 4096
32863 068/300 Write block 751 len 4096 data: 4
32864 069/300 Read block 1602 len 12288
32865 070/300 Flush
32866 071/300 Write block 3023 len 4096 data: 7
32867 072/300 Write block 2772 len 20480 data: 8 9 8 6 6
32868 073/300 Write block 779 len 16384 data: 5 4 5 5
32869 074/300 Flush
32870 075/300 Read block 3179 len 36864
32871 076/300 Write block 3780 len 4096 data: 6
32872 077/300 Write block 3072 len 8192 data: 4 3
32873 078/300 Read block 407 len 36864
32874 079/300 Read block 4211 len 24576
32875 080/300 Read block 2208 len 16384
32876 081/300 Read block 1674 len 16384
32877 082/300 Write block 14 len 8192 data: 4 4
32878 083/300 Write block 1666 len 16384 data: 6 4 3 5
32879 084/300 Read block 2936 len 12288
32880 085/300 Read block 2005 len 36864
32881 086/300 Read block 2977 len 20480
32882 087/300 Write block 2920 len 40960 data: 7 5 4 3 4 6 7 7 8 10
32883 088/300 Write block 422 len 24576 data: 7 6 6 7 7 5
32884 089/300 Write block 314 len 28672 data: 6 6 6 7 7 7 9
32885 090/300 Flush
32886 091/300 Read block 3419 len 4096
32887 092/300 Write block 1738 len 12288 data: 6 8 9
32888 093/300 Read block 2927 len 32768
32889 094/300 Read block 3485 len 32768
32890 095/300 Flush
32891 096/300 Write block 1746 len 20480 data: 5 4 4 4 3
32892 097/300 Write block 4645 len 32768 data: 6 7 8 7 7 5 6 8
32893 098/300 Read block 2278 len 36864
32894 099/300 Write block 3617 len 32768 data: 8 10 9 7 9 8 9 8
32895 100/300 Write block 3002 len 24576 data: 9 9 9 8 8 8
32896 101/300 Write block 3144 len 24576 data: 8 8 6 5 6 6
32897 102/300 Read block 580 len 24576
32898 103/300 Flush
32899 104/300 Write block 2720 len 12288 data: 6 5 7
32900 105/300 Read block 2247 len 12288
32901 106/300 Write block 4252 len 12288 data: 7 6 6
32902 107/300 Write block 3547 len 24576 data: 3 3 4 6 6 5
32903 108/300 Read block 2664 len 40960
32904 109/300 Write block 2473 len 8192 data: 6 6
32905 110/300 Read block 1293 len 40960
32906 111/300 Write block 2986 len 36864 data: 7 7 6 4 5 5 5 4 4
32907 112/300 Read block 4926 len 28672
32908 113/300 Write block 1925 len 36864 data: 8 7 5 5 5 5 3 3 4
32909 114/300 Write block 1052 len 36864 data: 3 3 3 2 3 5 7 8 8
32910 115/300 Read block 1395 len 40960
32911 116/300 Write block 4091 len 16384 data: 4 5 5 5
32912 117/300 Read block 3902 len 24576
32913 118/300 Read block 2154 len 40960
32914 119/300 Read block 1676 len 28672
32915 120/300 Read block 4404 len 36864
32916 121/300 Read block 2211 len 40960
32917 122/300 Read block 3661 len 24576
32918 123/300 Write block 2939 len 24576 data: 5 5 6 7 5 6
32919 124/300 Read block 4392 len 8192
32920 125/300 Read block 4979 len 24576
32921 126/300 Write block 3177 len 20480 data: 10 10 10 8 8
32922 127/300 Write block 4207 len 28672 data: 4 4 4 4 4 3 3
32923 128/300 Write block 4278 len 12288 data: 8 8 8
32924 129/300 Read block 36 len 28672
32925 130/300 Read block 793 len 40960
32926 131/300 Read block 4995 len 4096
32927 132/300 Flush
32928 133/300 Write block 908 len 4096 data: 5
32929 134/300 Read block 2419 len 20480
32930 135/300 Read block 2525 len 28672
32931 136/300 Read block 3810 len 24576
32932 137/300 Flush
32933 138/300 Read block 3401 len 20480
32934 139/300 Read block 4052 len 8192
32935 140/300 Read block 1901 len 16384
32936 141/300 Write block 1355 len 40960 data: 9 8 9 12 12 9 9 7 7 8
32937 142/300 Write block 2682 len 12288 data: 9 11 11
32938 143/300 Flush
32939 144/300 Read block 505 len 32768
32940 145/300 Flush
32941 146/300 Write block 3991 len 8192 data: 3 4
32942 147/300 Read block 3005 len 36864
32943 148/300 Write block 4710 len 40960 data: 8 7 7 8 8 6 6 6 8 6
32944 149/300 Write block 1154 len 36864 data: 4 4 4 4 5 4 4 3 3
32945 150/300 Flush
32946 151/300 Read block 3465 len 28672
32947 152/300 Read block 4189 len 24576
32948 153/300 Read block 586 len 24576
32949 154/300 Write block 2986 len 36864 data: 8 8 7 5 6 6 6 5 5
32950 155/300 Write block 871 len 8192 data: 7 6
32951 156/300 Write block 1493 len 32768 data: 6 6 5 4 5 4 4 4
32952 157/300 Read block 715 len 12288
32953 158/300 Write block 1333 len 40960 data: 10 8 7 7 7 8 7 6 4 6
32954 159/300 Read block 4105 len 12288
32955 160/300 Write block 3231 len 40960 data: 7 5 5 6 7 5 5 4 5 5
32956 161/300 Flush
32957 162/300 Read block 4775 len 40960
32958 163/300 Write block 3269 len 4096 data: 5
32959 164/300 Read block 1892 len 8192
32960 165/300 Write block 3795 len 28672 data: 2 2 2 3 3 3 2
32961 166/300 Write block 1483 len 4096 data: 8
32962 167/300 Flush
32963 168/300 Read block 3456 len 20480
32964 169/300 Write block 1438 len 24576 data: 6 6 6 7 7 6
32965 170/300 Read block 4440 len 24576
32966 171/300 Read block 3823 len 36864
32967 172/300 Read block 466 len 20480
32968 173/300 Read block 1812 len 20480
32969 174/300 Write block 485 len 20480 data: 4 4 4 5 6
32970 175/300 Write block 3038 len 20480 data: 6 6 5 5 4
32971 176/300 Write block 1725 len 12288 data: 6 8 8
32972 177/300 Flush
32973 178/300 Write block 3630 len 36864 data: 5 5 4 5 5 6 6 4 5
32974 179/300 Read block 2911 len 36864
32975 180/300 Read block 2518 len 24576
32976 181/300 Write block 4073 len 8192 data: 7 6
32977 182/300 Write block 774 len 12288 data: 3 3 5
32978 183/300 Write block 2515 len 36864 data: 8 7 6 5 4 5 4 4 4
32979 184/300 Read block 2246 len 32768
32980 185/300 Read block 3311 len 32768
32981 186/300 Write block 898 len 28672 data: 3 3 4 5 5 6 7
32982 187/300 Write block 1371 len 32768 data: 9 7 8 9 10 10 10 10
32983 188/300 Read block 4865 len 20480
32984 189/300 Read block 341 len 4096
32985 190/300 Write block 733 len 32768 data: 5 3 4 4 4 7 7 7
32986 191/300 Read block 356 len 8192
32987 192/300 Read block 3506 len 40960
32988 193/300 Read block 2583 len 28672
32989 194/300 Read block 3950 len 12288
32990 195/300 Write block 4642 len 8192 data: 5 7
32991 196/300 Read block 1623 len 36864
32992 197/300 Read block 2876 len 4096
32993 198/300 Write block 3035 len 36864 data: 3 4 4 7 7 6 6 5 4
32994 199/300 Write block 4537 len 32768 data: 6 5 3 4 4 3 3 4
32995 200/300 Read block 1125 len 36864
32996 201/300 Read block 3045 len 32768
32997 202/300 Write block 1725 len 36864 data: 7 9 9 7 8 7 6 5 5
32998 203/300 Write block 3656 len 20480 data: 4 3 5 5 6
32999 204/300 Read block 4379 len 36864
33000 205/300 Read block 2847 len 36864
33001 206/300 Write block 3432 len 8192 data: 9 10
33002 207/300 Flush
33003 208/300 Read block 3719 len 28672
33004 209/300 Read block 2195 len 28672
33005 210/300 Read block 4153 len 4096
33006 211/300 Write block 4224 len 16384 data: 5 5 4 3
33007 212/300 Write block 2319 len 12288 data: 4 5 4
33008 213/300 Read block 4679 len 40960
33009 214/300 Write block 1662 len 20480 data: 7 7 6 5 7
33010 215/300 Read block 4657 len 28672
33011 216/300 Write block 4416 len 16384 data: 5 5 4 3
33012 217/300 Read block 3841 len 12288
33013 218/300 Flush
33014 219/300 Read block 1526 len 32768
33015 220/300 Write block 4906 len 28672 data: 4 6 7 7 7 7 7
33016 221/300 Write block 2813 len 32768 data: 7 6 5 5 3 4 4 5
33017 222/300 Read block 4146 len 4096
33018 223/300 Write block 4744 len 32768 data: 5 5 6 4 4 5 6 5
33019 224/300 Read block 499 len 8192
33020 225/300 Read block 3059 len 40960
33021 226/300 Read block 3250 len 24576
33022 227/300 Write block 432 len 40960 data: 4 2 3 4 4 5 5 7 8 5
33023 228/300 Read block 3836 len 24576
33024 229/300 Write block 4813 len 40960 data: 4 4 4 3 2 2 3 4 5 6
33025 230/300 Read block 1553 len 8192
33026 231/300 Read block 571 len 40960
33027 232/300 Read block 1537 len 40960
33028 233/300 Read block 4919 len 24576
33029 234/300 Read block 4309 len 36864
33030 235/300 Read block 3277 len 4096
33031 236/300 Read block 3710 len 36864
33032 237/300 Write block 4663 len 4096 data: 3
33033 238/300 Read block 211 len 32768
33034 239/300 Read block 1548 len 16384
33035 240/300 Write block 69 len 4096 data: 4
33036 241/300 Write block 2808 len 40960 data: 5 5 5 7 7 8 7 6 6 4
33037 242/300 Read block 4227 len 40960
33038 243/300 Read block 391 len 40960
33039 244/300 Write block 4094 len 32768 data: 6 5 4 3 3 3 5 4
33040 245/300 Read block 2924 len 24576
33041 246/300 Flush
33042 247/300 Read block 4186 len 24576
33043 248/300 Read block 747 len 20480
33044 249/300 Read block 3487 len 36864
33045 250/300 Flush
33046 251/300 Write block 1587 len 16384 data: 2 2 3 3
33047 252/300 Write block 4425 len 16384 data: 3 3 5 5
33048 253/300 Flush
33049 254/300 Write block 4042 len 20480 data: 6 7 6 6 6
33050 255/300 Read block 2278 len 16384
33051 256/300 Write block 861 len 28672 data: 12 11 9 9 8 10 10
33052 257/300 Read block 3262 len 12288
33053 258/300 Write block 949 len 40960 data: 6 6 5 6 6 6 6 8 8 7
33054 259/300 Write block 4077 len 32768 data: 4 4 3 5 5 5 5 2
33055 260/300 Write block 574 len 4096 data: 6
33056 261/300 Write block 2413 len 16384 data: 8 9 8 7
33057 262/300 Read block 4520 len 8192
33058 263/300 Read block 4412 len 16384
33059 264/300 Flush
33060 265/300 Read block 2931 len 4096
33061 266/300 Read block 4946 len 12288
33062 267/300 Write block 3034 len 24576 data: 4 4 5 5 8 8
33063 268/300 Read block 3752 len 4096
33064 269/300 Read block 4070 len 32768
33065 270/300 Read block 548 len 8192
33066 271/300 Read block 4360 len 8192
33067 272/300 Read block 901 len 24576
33068 273/300 Write block 3345 len 36864 data: 2 4 3 3 3 4 3 3 2
33069 274/300 Flush
33070 275/300 Read block 99 len 40960
33071 276/300 Flush
33072 277/300 Read block 1712 len 8192
33073 278/300 Read block 2492 len 28672
33074 279/300 Read block 350 len 24576
33075 280/300 Write block 875 len 24576 data: 6 6 5 7 7 6
33076 281/300 Read block 655 len 16384
33077 282/300 Read block 1084 len 40960
33078 283/300 Read block 1791 len 16384
33079 284/300 Write block 3024 len 16384 data: 8 8 8 7
33080 285/300 Read block 4760 len 12288
33081 286/300 Flush
33082 287/300 Write block 770 len 16384 data: 6 6 7 7
33083 288/300 Read block 2847 len 4096
33084 289/300 Read block 4395 len 24576
33085 290/300 Write block 2133 len 12288 data: 6 6 5
33086 291/300 Read block 1993 len 20480
33087 292/300 Write block 1542 len 12288 data: 7 9 9
33088 293/300 Write block 4809 len 4096 data: 7
33089 294/300 Write block 3889 len 24576 data: 6 4 3 3 5 7
33090 295/300 Read block 4549 len 8192
33091 296/300 Write block 2097 len 32768 data: 5 5 5 4 4 5 5 6
33092 297/300 Read block 1531 len 32768
33093 298/300 Read block 3692 len 8192
33094 299/300 Read block 4636 len 4096
33095 300/300 Read block 101 len 16384
33096 301/300 Read block 4601 len 8192
33097 Main task has work to do, go find it
33098 got dsc Start(0)
33099 start 0
33100 Jan 27 19:43:20.336 INFO request completed, latency_us: 194, response_code: 204, uri: /start/cid/0, method: POST, req_id: 12ac5137-617f-41e3-a8b1-dbd33bdfe205, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
33101 Replay: started 0, returned:Ok(())
33102 ----------------------------------------------------------------
33103 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
33104 States: Offline Active Active
33105 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33106 9754 Acked Read 2 Sent Done Done false
33107 9755 Acked Write 6 Sent Done Done false
33108 9756 Acked Read 7 Sent Done Done false
33109 9757 Acked Read 7 Sent Done Done false
33110 9758 Acked Write 7 Sent Done Done false
33111 9759 Acked Write 1 Sent Done Done false
33112 9760 Acked Write 5 Sent Done Done false
33113 9761 Acked Write 6 Sent Done Done false
33114 9762 Acked Read 1 Sent Done Done false
33115 9763 Acked Read 5 Sent Done Done false
33116 9764 Acked Write 3 Sent Done Done false
33117 9765 Acked Write 3 Sent Done Done false
33118 9766 Acked Write 4 Sent Done Done false
33119 9767 Acked Write 9 Sent Done Done false
33120 9768 Acked Read 1 Sent Done Done false
33121 9769 Acked Write 7 Sent Done Done false
33122 9770 Acked Write 5 Sent Done Done false
33123 9771 Acked Read 7 Sent Done Done false
33124 9772 Acked Write 3 Sent Done Done false
33125 9773 Acked Flush 0 Sent Done Done false
33126 9774 Acked Read 4 Sent Done Done false
33127 9775 Acked Write 2 Sent Done Done false
33128 9776 Acked Flush 0 Sent Done Done false
33129 9777 Acked Read 1 Sent Done Done false
33130 9778 Acked Write 8 Sent Done Done false
33131 9779 Acked Read 7 Sent Done Done false
33132 9780 Acked Read 3 Sent Done Done false
33133 9781 Acked Read 3 Sent Done Done false
33134 9782 Acked Read 4 Sent Done Done false
33135 9783 Acked Read 6 Sent Done Done false
33136 9784 Acked Read 7 Sent Done Done false
33137 9785 Acked Flush 0 Sent Done Done false
33138 9786 Acked Read 3 Sent Done Done false
33139 9787 Acked Read 2 Sent Done Done false
33140 9788 Acked Write 5 Sent Done Done false
33141 9789 Acked Read 1 Sent Done Done false
33142 9790 Acked Write 5 Sent Done Done false
33143 9791 Acked Read 9 Sent Done Done false
33144 9792 Acked Read 7 Sent Done Done false
33145 9793 Acked Read 3 Sent Done Done false
33146 9794 Acked Read 6 Sent Done Done false
33147 9795 Acked Read 4 Sent Done Done false
33148 9796 Acked Write 2 Sent Done Done false
33149 9797 Acked Write 8 Sent Done Done false
33150 9798 Acked Read 5 Sent Done Done false
33151 9799 Acked Write 1 Sent Done Done false
33152 9800 Acked Write 2 Sent Done Done false
33153 9801 Acked Read 6 Sent Done Done false
33154 9802 Acked Flush 0 Sent Done Done false
33155 9803 Acked Read 5 Sent Done Done false
33156 9804 Acked Write 3 Sent Done Done false
33157 9805 Acked Write 4 Sent Done Done false
33158 9806 Acked Write 8 Sent Done Done false
33159 9807 Acked Write 7 Sent Done Done false
33160 9808 Acked Read 3 Sent Done Done false
33161 9809 Acked Read 10 Sent Done Done false
33162 9810 Acked Flush 0 Sent Done Done false
33163 9811 Acked Write 8 Sent Done Done false
33164 9812 Acked Flush 0 Sent Done Done false
33165 9813 Acked Read 2 Sent Done Done false
33166 9814 Acked Read 2 Sent Done Done false
33167 9815 Acked Write 9 Sent Done Done false
33168 9816 Acked Read 5 Sent Done Done false
33169 9817 Acked Write 9 Sent Done Done false
33170 9818 Acked Read 10 Sent Done Done false
33171 9819 Acked Flush 0 Sent Done Done false
33172 9820 Acked Read 1 Sent Done Done false
33173 9821 Acked Write 1 Sent Done Done false
33174 9822 Acked Read 3 Sent Done Done false
33175 9823 Acked Flush 0 Sent Done Done false
33176 9824 Acked Write 1 Sent Done Done false
33177 9825 Acked Write 5 Sent Done Done false
33178 9826 Acked Write 4 Sent Done Done false
33179 9827 Acked Flush 0 Sent Done Done false
33180 9828 Acked Read 9 Sent Done Done false
33181 9829 Acked Write 1 Sent Done Done false
33182 9830 Acked Write 2 Sent Done Done false
33183 9831 Acked Read 9 Sent Done Done false
33184 9832 Acked Read 6 Sent Done Done false
33185 9833 Acked Read 4 Sent Done Done false
33186 9834 Acked Read 4 Sent Done Done false
33187 9835 Acked Write 2 Sent Done Done false
33188 9836 Acked Write 4 Sent Done Done false
33189 9837 Acked Read 3 Sent Done Done false
33190 9838 Acked Read 9 Sent Done Done false
33191 9839 Acked Read 5 Sent Done Done false
33192 9840 Acked Write 10 Sent Done Done false
33193 9841 Acked Write 6 Sent Done Done false
33194 9842 Acked Write 7 Sent Done Done false
33195 9843 Acked Flush 0 Sent Done Done false
33196 9844 Acked Read 1 Sent Done Done false
33197 9845 Acked Write 3 Sent Done Done false
33198 9846 Acked Read 8 Sent Done Done false
33199 9847 Acked Read 8 Sent Done Done false
33200 9848 Acked Flush 0 Sent Done Done false
33201 9849 Acked Write 5 Sent Done Done false
33202 9850 Acked Write 8 Sent Done Done false
33203 9851 Acked Read 9 Sent Done Done false
33204 9852 Acked Write 8 Sent Done Done false
33205 9853 Acked Write 6 Sent Done Done false
33206 9854 Acked Write 6 Sent Done Done false
33207 9855 Acked Read 6 Sent Done Done false
33208 9856 Acked Flush 0 Sent Done Done false
33209 9857 Acked Write 3 Sent Done Done false
33210 9858 Acked Read 3 Sent Done Done false
33211 9859 Acked Write 3 Sent Done Done false
33212 9860 Acked Write 6 Sent Done Done false
33213 9861 Acked Read 10 Sent Done Done false
33214 9862 Acked Write 2 Sent Done Done false
33215 9863 Acked Read 10 Sent Done Done false
33216 9864 Acked Write 9 Sent Done Done false
33217 9865 Acked Read 7 Sent Done Done false
33218 9866 Acked Write 9 Sent Done Done false
33219 9867 Acked Write 9 Sent Done Done false
33220 9868 Acked Read 10 Sent Done Done false
33221 9869 Acked Write 4 Sent Done Done false
33222 9870 Acked Read 6 Sent Done Done false
33223 9871 Acked Read 10 Sent Done Done false
33224 9872 Acked Read 7 Sent Done Done false
33225 9873 Acked Read 9 Sent Done Done false
33226 9874 Acked Read 10 Sent Done Done false
33227 9875 Acked Read 6 Sent Done Done false
33228 9876 Acked Write 6 Sent Done Done false
33229 9877 Acked Read 2 Sent Done Done false
33230 9878 Acked Read 6 Sent Done Done false
33231 9879 Acked Write 5 Sent Done Done false
33232 9880 Acked Write 7 Sent Done Done false
33233 9881 Acked Write 3 Sent Done Done false
33234 9882 Acked Read 7 Sent Done Done false
33235 9883 Acked Read 10 Sent Done Done false
33236 9884 Acked Read 1 Sent Done Done false
33237 9885 Acked Flush 0 Sent Done Done false
33238 9886 Acked Write 1 Sent Done Done false
33239 9887 Acked Read 5 Sent Done Done false
33240 9888 Acked Read 7 Sent Done Done false
33241 9889 Acked Read 6 Sent Done Done false
33242 9890 Acked Flush 0 Sent Done Done false
33243 9891 Acked Read 5 Sent Done Done false
33244 9892 Acked Read 2 Sent Done Done false
33245 9893 Acked Read 4 Sent Done Done false
33246 9894 Acked Write 10 Sent Done Done false
33247 9895 Acked Write 3 Sent Done Done false
33248 9896 Acked Flush 0 Sent Done Done false
33249 9897 Acked Read 8 Sent Done Done false
33250 9898 Acked Flush 0 Sent Done Done false
33251 9899 Acked Write 2 Sent Done Done false
33252 9900 Acked Read 9 Sent Done Done false
33253 9901 Acked Write 10 Sent Done Done false
33254 9902 Acked Write 9 Sent Done Done false
33255 9903 Acked Flush 0 Sent Done Done false
33256 9904 Acked Read 7 Sent Done Done false
33257 9905 Acked Read 6 Sent Done Done false
33258 9906 Acked Read 6 Sent Done Done false
33259 9907 Acked Write 9 Sent Done Done false
33260 9908 Acked Write 2 Sent Done Done false
33261 9909 Acked Write 8 Sent Done Done false
33262 9910 Acked Read 3 Sent Done Done false
33263 9911 Acked Write 10 Sent Done Done false
33264 9912 Acked Read 3 Sent Done Done false
33265 9913 Acked Write 10 Sent Done Done false
33266 9914 Acked Flush 0 Sent Done Done false
33267 9915 Acked Read 10 Sent Done Done false
33268 9916 Acked Write 1 Sent Done Done false
33269 9917 Acked Read 2 Sent Done Done false
33270 9918 Acked Write 7 Sent Done Done false
33271 9919 Acked Write 1 Sent Done Done false
33272 9920 Acked Flush 0 Sent Done Done false
33273 9921 Acked Read 5 Sent Done Done false
33274 9922 Acked Write 6 Sent Done Done false
33275 9923 Acked Read 6 Sent Done Done false
33276 9924 Acked Read 9 Sent Done Done false
33277 9925 Acked Read 5 Sent Done Done false
33278 9926 Acked Read 5 Sent Done Done false
33279 9927 Acked Write 5 Sent Done Done false
33280 9928 Acked Write 5 Sent Done Done false
33281 9929 Acked Write 3 Sent Done Done false
33282 9930 Acked Flush 0 Sent Done Done false
33283 9931 Acked Write 9 Sent Done Done false
33284 9932 Acked Read 9 Sent Done Done false
33285 9933 Acked Read 6 Sent Done Done false
33286 9934 Acked Write 2 Sent Done Done false
33287 9935 Acked Write 3 Sent Done Done false
33288 9936 Acked Write 9 Sent Done Done false
33289 9937 Acked Read 8 Sent Done Done false
33290 9938 Acked Read 8 Sent Done Done false
33291 9939 Acked Write 7 Sent Done Done false
33292 9940 Acked Write 8 Sent Done Done false
33293 9941 Acked Read 5 Sent Done Done false
33294 9942 Acked Read 1 Sent Done Done false
33295 9943 Acked Write 8 Sent Done Done false
33296 9944 Acked Read 2 Sent Done Done false
33297 9945 Acked Read 10 Sent Done Done false
33298 9946 Acked Read 7 Sent Done Done false
33299 9947 Acked Read 3 Sent Done Done false
33300 9948 Acked Write 2 Sent Done Done false
33301 9949 Acked Read 9 Sent Done Done false
33302 9950 Acked Read 1 Sent Done Done false
33303 9951 Acked Write 9 Sent Done Done false
33304 9952 Acked Write 8 Sent Done Done false
33305 9953 Acked Read 9 Sent Done Done false
33306 9954 Acked Read 8 Sent Done Done false
33307 9955 Acked Write 9 Sent Done Done false
33308 9956 Acked Write 5 Sent Done Done false
33309 9957 Acked Read 9 Sent Done Done false
33310 9958 Acked Read 9 Sent Done Done false
33311 9959 Acked Write 2 Sent Done Done false
33312 9960 Acked Flush 0 Sent Done Done false
33313 9961 Acked Read 7 Sent Done Done false
33314 9962 Acked Read 7 Sent Done Done false
33315 9963 Acked Read 1 Sent Done Done false
33316 9964 Acked Write 4 Sent Done Done false
33317 9965 Acked Write 3 Sent Done Done false
33318 9966 Acked Read 10 Sent Done Done false
33319 9967 Acked Write 5 Sent Done Done false
33320 9968 Acked Read 7 Sent Done Done false
33321 9969 Acked Write 4 Sent Done Done false
33322 9970 Acked Read 3 Sent Done Done false
33323 9971 Acked Flush 0 Sent Done Done false
33324 9972 Acked Read 8 Sent Done Done false
33325 9973 Acked Write 7 Sent Done Done false
33326 9974 Acked Write 8 Sent Done Done false
33327 9975 Acked Read 1 Sent Done Done false
33328 9976 Acked Write 8 Sent Done Done false
33329 9977 Acked Read 2 Sent Done Done false
33330 9978 Acked Read 10 Sent Done Done false
33331 9979 Acked Read 6 Sent Done Done false
33332 9980 Acked Write 10 Sent Done Done false
33333 9981 Acked Read 6 Sent Done Done false
33334 9982 Acked Write 10 Sent Done Done false
33335 9983 Acked Read 2 Sent Done Done false
33336 9984 Acked Read 10 Sent Done Done false
33337 9985 Acked Read 10 Sent Done Done false
33338 9986 Acked Read 6 Sent Done Done false
33339 9987 Acked Read 9 Sent Done Done false
33340 9988 Acked Read 1 Sent Done Done false
33341 9989 Acked Read 9 Sent Done Done false
33342 9990 Acked Write 1 Sent Done Done false
33343 9991 Acked Read 8 Sent Done Done false
33344 9992 Acked Read 4 Sent Done Done false
33345 9993 Acked Write 1 Sent Done Done false
33346 9994 Acked Write 10 Sent Done Done false
33347 9995 Acked Read 10 Sent Done Done false
33348 9996 Acked Read 10 Sent Done Done false
33349 9997 Acked Write 8 Sent Done Done false
33350 9998 Acked Read 6 Sent Done Done false
33351 9999 Acked Flush 0 Sent Done Done false
33352 10000 Acked Read 6 Sent Done Done false
33353 10001 Acked Read 5 Sent Done Done false
33354 10002 Acked Read 9 Sent Done Done false
33355 10003 Acked Flush 0 Sent Done Done false
33356 10004 Acked Write 4 Sent Done Done false
33357 10005 Acked Write 4 Sent Done Done false
33358 10006 Acked Flush 0 Sent Done Done false
33359 10007 Acked Write 5 Sent Done Done false
33360 10008 Acked Read 4 Sent Done Done false
33361 10009 Acked Write 7 Sent Done Done false
33362 10010 Acked Read 3 Sent Done Done false
33363 10011 Acked Write 10 Sent Done Done false
33364 10012 Acked Write 8 Sent Done Done false
33365 10013 Acked Write 1 Sent Done Done false
33366 10014 Acked Write 4 Sent Done Done false
33367 10015 Acked Read 2 Sent Done Done false
33368 10016 Acked Read 4 Sent Done Done false
33369 10017 Acked Flush 0 Sent Done Done false
33370 10018 Acked Read 1 Sent Done Done false
33371 10019 Acked Read 3 Sent Done Done false
33372 10020 Acked Write 6 Sent Done Done false
33373 10021 Acked Read 1 Sent Done Done false
33374 10022 Acked Read 8 Sent Done Done false
33375 10023 Acked Read 2 Sent Done Done false
33376 10024 Acked Read 2 Sent Done Done false
33377 10025 Acked Read 6 Sent Done Done false
33378 10026 Acked Write 9 Sent Done Done false
33379 10027 Acked Flush 0 Sent Done Done false
33380 10028 Acked Read 10 Sent Done Done false
33381 10029 Acked Flush 0 Sent Done Done false
33382 10030 Acked Read 2 Sent Done Done false
33383 10031 Acked Read 7 Sent Done Done false
33384 10032 Acked Read 6 Sent Done Done false
33385 10033 Acked Write 6 Sent Done Done false
33386 10034 Acked Read 4 Sent Done Done false
33387 10035 Acked Read 10 Sent Done Done false
33388 10036 Acked Read 4 Sent Done Done false
33389 10037 Acked Write 4 Sent Done Done false
33390 10038 Acked Read 3 Sent Done Done false
33391 10039 Acked Flush 0 Sent Done Done false
33392 10040 Acked Write 4 Sent Done Done false
33393 10041 Acked Read 1 Sent Done Done false
33394 10042 Acked Read 6 Sent Done Done false
33395 10043 Acked Write 3 Sent Done Done false
33396 10044 Acked Read 5 Sent Done Done false
33397 10045 Acked Write 3 Sent Done Done false
33398 10046 Acked Write 1 Sent Done Done false
33399 10047 Acked Write 6 Sent Done Done false
33400 10048 Acked Read 2 Sent Done Done false
33401 10049 Acked Write 8 Sent Done Done false
33402 10050 Acked Read 8 Sent Done Done false
33403 10051 Acked Read 2 Sent Done Done false
33404 10052 Acked Read 1 Sent Done Done false
33405 10053 Acked Read 4 Sent Done Done false
33406 10054 Acked Read 2 Sent Done Done false
33407 STATES DS:0 DS:1 DS:2 TOTAL
33408 Sent 301 0 0 301
33409 Done 0 301 301 602
33410 Skipped 0 0 0 0
33411 Error 0 0 0 0
33412 Last Flush: 9753 10039 10039
33413 Downstairs last five completed: 9753 9752 9751 9750 9749
33414 Upstairs last five completed: 10054 10053 10052 10051 10050
33415 CLIENT: Up:0 ds:301 act:2
33416 [8810] got start action
33417 [8810] I am going to restart
33418 Starting downstairs at port 8810
33419 Make output file at "/tmp/dsc/downstairs-8810.txt"
33420 [8810][0] reports Starting
33421 Downstairs /var/tmp/dsc/region/8810 port 8810 PID:Some(1230)
33422 [8810][0] reports Running
33423 ----------------------------------------------------------------
33424 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
33425 States: Offline Active Active
33426 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
33427 9754 Acked Read 2 Sent Done Done false
33428 9755 Acked Write 6 Sent Done Done false
33429 9756 Acked Read 7 Sent Done Done false
33430 9757 Acked Read 7 Sent Done Done false
33431 9758 Acked Write 7 Sent Done Done false
33432 9759 Acked Write 1 Sent Done Done false
33433 9760 Acked Write 5 Sent Done Done false
33434 9761 Acked Write 6 Sent Done Done false
33435 9762 Acked Read 1 Sent Done Done false
33436 9763 Acked Read 5 Sent Done Done false
33437 9764 Acked Write 3 Sent Done Done false
33438 9765 Acked Write 3 Sent Done Done false
33439 9766 Acked Write 4 Sent Done Done false
33440 9767 Acked Write 9 Sent Done Done false
33441 9768 Acked Read 1 Sent Done Done false
33442 9769 Acked Write 7 Sent Done Done false
33443 9770 Acked Write 5 Sent Done Done false
33444 9771 Acked Read 7 Sent Done Done false
33445 9772 Acked Write 3 Sent Done Done false
33446 9773 Acked Flush 0 Sent Done Done false
33447 9774 Acked Read 4 Sent Done Done false
33448 9775 Acked Write 2 Sent Done Done false
33449 9776 Acked Flush 0 Sent Done Done false
33450 9777 Acked Read 1 Sent Done Done false
33451 9778 Acked Write 8 Sent Done Done false
33452 9779 Acked Read 7 Sent Done Done false
33453 9780 Acked Read 3 Sent Done Done false
33454 9781 Acked Read 3 Sent Done Done false
33455 9782 Acked Read 4 Sent Done Done false
33456 9783 Acked Read 6 Sent Done Done false
33457 9784 Acked Read 7 Sent Done Done false
33458 9785 Acked Flush 0 Sent Done Done false
33459 9786 Acked Read 3 Sent Done Done false
33460 9787 Acked Read 2 Sent Done Done false
33461 9788 Acked Write 5 Sent Done Done false
33462 9789 Acked Read 1 Sent Done Done false
33463 9790 Acked Write 5 Sent Done Done false
33464 9791 Acked Read 9 Sent Done Done false
33465 9792 Acked Read 7 Sent Done Done false
33466 9793 Acked Read 3 Sent Done Done false
33467 9794 Acked Read 6 Sent Done Done false
33468 9795 Acked Read 4 Sent Done Done false
33469 9796 Acked Write 2 Sent Done Done false
33470 9797 Acked Write 8 Sent Done Done false
33471 9798 Acked Read 5 Sent Done Done false
33472 9799 Acked Write 1 Sent Done Done false
33473 9800 Acked Write 2 Sent Done Done false
33474 9801 Acked Read 6 Sent Done Done false
33475 9802 Acked Flush 0 Sent Done Done false
33476 9803 Acked Read 5 Sent Done Done false
33477 9804 Acked Write 3 Sent Done Done false
33478 9805 Acked Write 4 Sent Done Done false
33479 9806 Acked Write 8 Sent Done Done false
33480 9807 Acked Write 7 Sent Done Done false
33481 9808 Acked Read 3 Sent Done Done false
33482 9809 Acked Read 10 Sent Done Done false
33483 9810 Acked Flush 0 Sent Done Done false
33484 9811 Acked Write 8 Sent Done Done false
33485 9812 Acked Flush 0 Sent Done Done false
33486 9813 Acked Read 2 Sent Done Done false
33487 9814 Acked Read 2 Sent Done Done false
33488 9815 Acked Write 9 Sent Done Done false
33489 9816 Acked Read 5 Sent Done Done false
33490 9817 Acked Write 9 Sent Done Done false
33491 9818 Acked Read 10 Sent Done Done false
33492 9819 Acked Flush 0 Sent Done Done false
33493 9820 Acked Read 1 Sent Done Done false
33494 9821 Acked Write 1 Sent Done Done false
33495 9822 Acked Read 3 Sent Done Done false
33496 9823 Acked Flush 0 Sent Done Done false
33497 9824 Acked Write 1 Sent Done Done false
33498 9825 Acked Write 5 Sent Done Done false
33499 9826 Acked Write 4 Sent Done Done false
33500 9827 Acked Flush 0 Sent Done Done false
33501 9828 Acked Read 9 Sent Done Done false
33502 9829 Acked Write 1 Sent Done Done false
33503 9830 Acked Write 2 Sent Done Done false
33504 9831 Acked Read 9 Sent Done Done false
33505 9832 Acked Read 6 Sent Done Done false
33506 9833 Acked Read 4 Sent Done Done false
33507 9834 Acked Read 4 Sent Done Done false
33508 9835 Acked Write 2 Sent Done Done false
33509 9836 Acked Write 4 Sent Done Done false
33510 9837 Acked Read 3 Sent Done Done false
33511 9838 Acked Read 9 Sent Done Done false
33512 9839 Acked Read 5 Sent Done Done false
33513 9840 Acked Write 10 Sent Done Done false
33514 9841 Acked Write 6 Sent Done Done false
33515 9842 Acked Write 7 Sent Done Done false
33516 9843 Acked Flush 0 Sent Done Done false
33517 9844 Acked Read 1 Sent Done Done false
33518 9845 Acked Write 3 Sent Done Done false
33519 9846 Acked Read 8 Sent Done Done false
33520 9847 Acked Read 8 Sent Done Done false
33521 9848 Acked Flush 0 Sent Done Done false
33522 9849 Acked Write 5 Sent Done Done false
33523 9850 Acked Write 8 Sent Done Done false
33524 9851 Acked Read 9 Sent Done Done false
33525 9852 Acked Write 8 Sent Done Done false
33526 9853 Acked Write 6 Sent Done Done false
33527 9854 Acked Write 6 Sent Done Done false
33528 9855 Acked Read 6 Sent Done Done false
33529 9856 Acked Flush 0 Sent Done Done false
33530 9857 Acked Write 3 Sent Done Done false
33531 9858 Acked Read 3 Sent Done Done false
33532 9859 Acked Write 3 Sent Done Done false
33533 9860 Acked Write 6 Sent Done Done false
33534 9861 Acked Read 10 Sent Done Done false
33535 9862 Acked Write 2 Sent Done Done false
33536 9863 Acked Read 10 Sent Done Done false
33537 9864 Acked Write 9 Sent Done Done false
33538 9865 Acked Read 7 Sent Done Done false
33539 9866 Acked Write 9 Sent Done Done false
33540 9867 Acked Write 9 Sent Done Done false
33541 9868 Acked Read 10 Sent Done Done false
33542 9869 Acked Write 4 Sent Done Done false
33543 9870 Acked Read 6 Sent Done Done false
33544 9871 Acked Read 10 Sent Done Done false
33545 9872 Acked Read 7 Sent Done Done false
33546 9873 Acked Read 9 Sent Done Done false
33547 9874 Acked Read 10 Sent Done Done false
33548 9875 Acked Read 6 Sent Done Done false
33549 9876 Acked Write 6 Sent Done Done false
33550 9877 Acked Read 2 Sent Done Done false
33551 9878 Acked Read 6 Sent Done Done false
33552 9879 Acked Write 5 Sent Done Done false
33553 9880 Acked Write 7 Sent Done Done false
33554 9881 Acked Write 3 Sent Done Done false
33555 9882 Acked Read 7 Sent Done Done false
33556 9883 Acked Read 10 Sent Done Done false
33557 9884 Acked Read 1 Sent Done Done false
33558 9885 Acked Flush 0 Sent Done Done false
33559 9886 Acked Write 1 Sent Done Done false
33560 9887 Acked Read 5 Sent Done Done false
33561 9888 Acked Read 7 Sent Done Done false
33562 9889 Acked Read 6 Sent Done Done false
33563 9890 Acked Flush 0 Sent Done Done false
33564 9891 Acked Read 5 Sent Done Done false
33565 9892 Acked Read 2 Sent Done Done false
33566 9893 Acked Read 4 Sent Done Done false
33567 9894 Acked Write 10 Sent Done Done false
33568 9895 Acked Write 3 Sent Done Done false
33569 9896 Acked Flush 0 Sent Done Done false
33570 9897 Acked Read 8 Sent Done Done false
33571 9898 Acked Flush 0 Sent Done Done false
33572 9899 Acked Write 2 Sent Done Done false
33573 9900 Acked Read 9 Sent Done Done false
33574 9901 Acked Write 10 Sent Done Done false
33575 9902 Acked Write 9 Sent Done Done false
33576 9903 Acked Flush 0 Sent Done Done false
33577 9904 Acked Read 7 Sent Done Done false
33578 9905 Acked Read 6 Sent Done Done false
33579 9906 Acked Read 6 Sent Done Done false
33580 9907 Acked Write 9 Sent Done Done false
33581 9908 Acked Write 2 Sent Done Done false
33582 9909 Acked Write 8 Sent Done Done false
33583 9910 Acked Read 3 Sent Done Done false
33584 9911 Acked Write 10 Sent Done Done false
33585 9912 Acked Read 3 Sent Done Done false
33586 9913 Acked Write 10 Sent Done Done false
33587 9914 Acked Flush 0 Sent Done Done false
33588 9915 Acked Read 10 Sent Done Done false
33589 9916 Acked Write 1 Sent Done Done false
33590 9917 Acked Read 2 Sent Done Done false
33591 9918 Acked Write 7 Sent Done Done false
33592 9919 Acked Write 1 Sent Done Done false
33593 9920 Acked Flush 0 Sent Done Done false
33594 9921 Acked Read 5 Sent Done Done false
33595 9922 Acked Write 6 Sent Done Done false
33596 9923 Acked Read 6 Sent Done Done false
33597 9924 Acked Read 9 Sent Done Done false
33598 9925 Acked Read 5 Sent Done Done false
33599 9926 Acked Read 5 Sent Done Done false
33600 9927 Acked Write 5 Sent Done Done false
33601 9928 Acked Write 5 Sent Done Done false
33602 9929 Acked Write 3 Sent Done Done false
33603 9930 Acked Flush 0 Sent Done Done false
33604 9931 Acked Write 9 Sent Done Done false
33605 9932 Acked Read 9 Sent Done Done false
33606 9933 Acked Read 6 Sent Done Done false
33607 9934 Acked Write 2 Sent Done Done false
33608 9935 Acked Write 3 Sent Done Done false
33609 9936 Acked Write 9 Sent Done Done false
33610 9937 Acked Read 8 Sent Done Done false
33611 9938 Acked Read 8 Sent Done Done false
33612 9939 Acked Write 7 Sent Done Done false
33613 9940 Acked Write 8 Sent Done Done false
33614 9941 Acked Read 5 Sent Done Done false
33615 9942 Acked Read 1 Sent Done Done false
33616 9943 Acked Write 8 Sent Done Done false
33617 9944 Acked Read 2 Sent Done Done false
33618 9945 Acked Read 10 Sent Done Done false
33619 9946 Acked Read 7 Sent Done Done false
33620 9947 Acked Read 3 Sent Done Done false
33621 9948 Acked Write 2 Sent Done Done false
33622 9949 Acked Read 9 Sent Done Done false
33623 9950 Acked Read 1 Sent Done Done false
33624 9951 Acked Write 9 Sent Done Done false
33625 9952 Acked Write 8 Sent Done Done false
33626 9953 Acked Read 9 Sent Done Done false
33627 9954 Acked Read 8 Sent Done Done false
33628 9955 Acked Write 9 Sent Done Done false
33629 9956 Acked Write 5 Sent Done Done false
33630 9957 Acked Read 9 Sent Done Done false
33631 9958 Acked Read 9 Sent Done Done false
33632 9959 Acked Write 2 Sent Done Done false
33633 9960 Acked Flush 0 Sent Done Done false
33634 9961 Acked Read 7 Sent Done Done false
33635 9962 Acked Read 7 Sent Done Done false
33636 9963 Acked Read 1 Sent Done Done false
33637 9964 Acked Write 4 Sent Done Done false
33638 9965 Acked Write 3 Sent Done Done false
33639 9966 Acked Read 10 Sent Done Done false
33640 9967 Acked Write 5 Sent Done Done false
33641 9968 Acked Read 7 Sent Done Done false
33642 9969 Acked Write 4 Sent Done Done false
33643 9970 Acked Read 3 Sent Done Done false
33644 9971 Acked Flush 0 Sent Done Done false
33645 9972 Acked Read 8 Sent Done Done false
33646 9973 Acked Write 7 Sent Done Done false
33647 9974 Acked Write 8 Sent Done Done false
33648 9975 Acked Read 1 Sent Done Done false
33649 9976 Acked Write 8 Sent Done Done false
33650 9977 Acked Read 2 Sent Done Done false
33651 9978 Acked Read 10 Sent Done Done false
33652 9979 Acked Read 6 Sent Done Done false
33653 9980 Acked Write 10 Sent Done Done false
33654 9981 Acked Read 6 Sent Done Done false
33655 9982 Acked Write 10 Sent Done Done false
33656 9983 Acked Read 2 Sent Done Done false
33657 9984 Acked Read 10 Sent Done Done false
33658 9985 Acked Read 10 Sent Done Done false
33659 9986 Acked Read 6 Sent Done Done false
33660 9987 Acked Read 9 Sent Done Done false
33661 9988 Acked Read 1 Sent Done Done false
33662 9989 Acked Read 9 Sent Done Done false
33663 9990 Acked Write 1 Sent Done Done false
33664 9991 Acked Read 8 Sent Done Done false
33665 9992 Acked Read 4 Sent Done Done false
33666 9993 Acked Write 1 Sent Done Done false
33667 9994 Acked Write 10 Sent Done Done false
33668 9995 Acked Read 10 Sent Done Done false
33669 9996 Acked Read 10 Sent Done Done false
33670 9997 Acked Write 8 Sent Done Done false
33671 9998 Acked Read 6 Sent Done Done false
33672 9999 Acked Flush 0 Sent Done Done false
33673 10000 Acked Read 6 Sent Done Done false
33674 10001 Acked Read 5 Sent Done Done false
33675 10002 Acked Read 9 Sent Done Done false
33676 10003 Acked Flush 0 Sent Done Done false
33677 10004 Acked Write 4 Sent Done Done false
33678 10005 Acked Write 4 Sent Done Done false
33679 10006 Acked Flush 0 Sent Done Done false
33680 10007 Acked Write 5 Sent Done Done false
33681 10008 Acked Read 4 Sent Done Done false
33682 10009 Acked Write 7 Sent Done Done false
33683 10010 Acked Read 3 Sent Done Done false
33684 10011 Acked Write 10 Sent Done Done false
33685 10012 Acked Write 8 Sent Done Done false
33686 10013 Acked Write 1 Sent Done Done false
33687 10014 Acked Write 4 Sent Done Done false
33688 10015 Acked Read 2 Sent Done Done false
33689 10016 Acked Read 4 Sent Done Done false
33690 10017 Acked Flush 0 Sent Done Done false
33691 10018 Acked Read 1 Sent Done Done false
33692 10019 Acked Read 3 Sent Done Done false
33693 10020 Acked Write 6 Sent Done Done false
33694 10021 Acked Read 1 Sent Done Done false
33695 10022 Acked Read 8 Sent Done Done false
33696 10023 Acked Read 2 Sent Done Done false
33697 10024 Acked Read 2 Sent Done Done false
33698 10025 Acked Read 6 Sent Done Done false
33699 10026 Acked Write 9 Sent Done Done false
33700 10027 Acked Flush 0 Sent Done Done false
33701 10028 Acked Read 10 Sent Done Done false
33702 10029 Acked Flush 0 Sent Done Done false
33703 10030 Acked Read 2 Sent Done Done false
33704 10031 Acked Read 7 Sent Done Done false
33705 10032 Acked Read 6 Sent Done Done false
33706 10033 Acked Write 6 Sent Done Done false
33707 10034 Acked Read 4 Sent Done Done false
33708 10035 Acked Read 10 Sent Done Done false
33709 10036 Acked Read 4 Sent Done Done false
33710 10037 Acked Write 4 Sent Done Done false
33711 10038 Acked Read 3 Sent Done Done false
33712 10039 Acked Flush 0 Sent Done Done false
33713 10040 Acked Write 4 Sent Done Done false
33714 10041 Acked Read 1 Sent Done Done false
33715 10042 Acked Read 6 Sent Done Done false
33716 10043 Acked Write 3 Sent Done Done false
33717 10044 Acked Read 5 Sent Done Done false
33718 10045 Acked Write 3 Sent Done Done false
33719 10046 Acked Write 1 Sent Done Done false
33720 10047 Acked Write 6 Sent Done Done false
33721 10048 Acked Read 2 Sent Done Done false
33722 10049 Acked Write 8 Sent Done Done false
33723 10050 Acked Read 8 Sent Done Done false
33724 10051 Acked Read 2 Sent Done Done false
33725 10052 Acked Read 1 Sent Done Done false
33726 10053 Acked Read 4 Sent Done Done false
33727 10054 Acked Read 2 Sent Done Done false
33728 10055 Acked Flush 0 Sent Done Done false
33729 STATES DS:0 DS:1 DS:2 TOTAL
33730 Sent 302 0 0 302
33731 Done 0 302 302 604
33732 Skipped 0 0 0 0
33733 Error 0 0 0 0
33734 Last Flush: 9753 10055 10055
33735 Downstairs last five completed: 9753 9752 9751 9750 9749
33736 Upstairs last five completed: 10055 10054 10053 10052 10051
33737 CLIENT: Up:0 ds:302 act:2
337382025-01-27T19:43:26.098ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337392025-01-27T19:43:26.098ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58381) = io task client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337402025-01-27T19:43:26.099ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337412025-01-27T19:43:26.100ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337422025-01-27T19:43:26.100ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008810 matches = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337432025-01-27T19:43:26.100ZINFOcrucible: send last flush ID to this DS: 9753 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337442025-01-27T19:43:26.100ZINFOcrucible: Replied this last flush ID: 9753 = downstairs client = 0 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337452025-01-27T19:43:26.100ZINFOcrucible: [0] client re-new 302 jobs since flush 9753 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337462025-01-27T19:43:26.101ZINFOcrucible: [0] Replayed 302 jobs since flush: 9753 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
33747 ----------------------------------------------------------------
33748 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
33749 Downstairs last five completed: 10055 10054 10053 10052 10051
33750 Upstairs last five completed: 10055 10054 10053 10052 10051
33751 CLIENT: Up:0 ds:0 act:3
33752 Replay: All jobs finished
33753 Main task has work to do, go find it
33754 got dsc Stop(2)
33755 stop 2
33756 Jan 27 19:43:28.343 INFO request completed, latency_us: 387, response_code:[8830] Got stop action so:false kr:false
33757 204, uri: /stop/cid/2, method: POST, req_id: c3062292-3136-44b9-acfa-a9f551aa0004, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
337582025-01-27T19:43:28.343ZWARNcrucible: downstairs disconnected = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337592025-01-27T19:43:28.343ZWARNcrucible: client task is sending Done(Finished) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337602025-01-27T19:43:28.343ZWARNcrucible: downstairs task for 2 stopped due to Finished session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
337612025-01-27T19:43:28.343ZINFOcrucible: client task is exiting = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
33762 Jan 27 19:43:28.343 INFO request completed, latency_us: 162, response_code: 200, uri: /state/cid/2, method: GET, req_id: b7f504cb-cb5a-4099-ac22-a9f219068f3a, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
33763 [8830] Exited with: ExitStatus(unix_wait_status(9))
33764 [8830][2] reports Exit
33765 Jan 27 19:43:32.346 INFO request completed, latency_us: 226, response_code: 200, uri: /state/cid/2, method: GET, req_id: 3e402c90-a33b-4cec-8551-f0e1f3cdbaf6, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
33766 001/300 Write block 1251 len 12288 data: 6 5 4
33767 002/300 Write block 2214 len 12288 data: 3 3 4
33768 003/300 Read block 1227 len 28672
33769 004/300 Read block 2621 len 36864
33770 005/300 Read block 1676 len 20480
33771 006/300 Write block 2257 len 36864 data: 8 9 7 7 6 6 4 4 3
33772 007/300 Write block 2860 len 20480 data: 6 6 7 7 7
33773 008/300 Write block 1166 len 12288 data: 5 6 6
33774 009/300 Flush
33775 010/300 Read block 3564 len 20480
33776 011/300 Write block 2287 len 32768 data: 9 12 11 11 8 8 3 3
33777 012/300 Write block 2133 len 36864 data: 7 7 6 7 7 6 5 6 7
33778 013/300 Write block 2411 len 8192 data: 9 8
33779 014/300 Read block 2508 len 4096
33780 015/300 Write block 2867 len 4096 data: 6
33781 016/300 Flush
33782 017/300 Read block 1224 len 36864
33783 018/300 Write block 1307 len 16384 data: 9 7 5 5
33784 019/300 Read block 2496 len 16384
33785 020/300 Read block 2270 len 28672
33786 021/300 Flush
33787 022/300 Read block 1254 len 24576
33788 023/300 Write block 1493 len 36864 data: 7 7 6 5 6 5 5 5 4
33789 024/300 Write block 254 len 28672 data: 8 7 7 8 7 7 7
33790 025/300 Read block 3754 len 40960
33791 026/300 Read block 1832 len 12288
33792 027/300 Write block 4964 len 16384 data: 7 4 4 7
33793 028/300 Write block 3512 len 40960 data: 4 3 4 3 3 4 5 7 7 7
33794 029/300 Write block 4740 len 28672 data: 8 8 7 6 6 6 7
33795 030/300 Flush
33796 031/300 Write block 3967 len 8192 data: 8 8
33797 032/300 Write block 3234 len 32768 data: 7 8 6 6 5 6 6 7
33798 033/300 Read block 3071 len 8192
33799 034/300 Read block 4941 len 40960
33800 035/300 Read block 191 len 20480
33801 036/300 Read block 2176 len 8192
33802 037/300 Write block 3088 len 16384 data: 6 5 7 8
33803 038/300 Write block 4323 len 20480 data: 6 6 6 4 4
33804 039/300 Read block 55 len 12288
33805 040/300 Write block 2800 len 36864 data: 6 8 7 7 7 4 5 5 6
33806 041/300 Read block 1176 len 28672
33807 042/300 Read block 1219 len 28672
33808 043/300 Read block 853 len 32768
33809 044/300 Write block 3960 len 8192 data: 6 6
33810 045/300 Read block 2090 len 36864
33811 046/300 Write block 1934 len 20480 data: 6 6 4 5 4
33812 047/300 Read block 1085 len 8192
33813 048/300 Read block 2688 len 12288
33814 049/300 Flush
33815 050/300 Read block 3842 len 20480
33816 051/300 Write block 2929 len 36864 data: 11 9 9 9 9 8 8 9 7
33817 052/300 Read block 2242 len 4096
33818 053/300 Read block 3908 len 32768
33819 054/300 Read block 1960 len 32768
33820 055/300 Write block 2518 len 36864 data: 6 5 6 5 5 5 6 5 4
33821 056/300 Write block 231 len 12288 data: 3 3 5
33822 057/300 Read block 4345 len 32768
33823 058/300 Read block 3385 len 20480
33824 059/300 Write block 1251 len 20480 data: 7 6 5 6 5
33825 060/300 Read block 3800 len 32768
33826 061/300 Flush
33827 062/300 Flush
33828 063/300 Flush
33829 064/300 Write block 2899 len 12288 data: 2 2 2
33830 065/300 Flush
33831 066/300 Read block 4727 len 28672
33832 067/300 Read block 144 len 16384
33833 068/300 Write block 1560 len 12288 data: 6 4 3
33834 069/300 Read block 1336 len 4096
33835 070/300 Write block 3165 len 16384 data: 3 3 3 3
33836 071/300 Read block 2859 len 4096
33837 072/300 Read block 2159 len 28672
33838 073/300 Flush
33839 074/300 Write block 4800 len 4096 data: 8
33840 075/300 Write block 2020 len 20480 data: 6 6 6 6 5
33841 076/300 Read block 3879 len 24576
33842 077/300 Read block 1142 len 24576
33843 078/300 Read block 611 len 36864
33844 079/300 Flush
33845 080/300 Read block 1500 len 32768
33846 081/300 Read block 4365 len 28672
33847 082/300 Write block 2257 len 4096 data: 9
33848 083/300 Write block 123 len 4096 data: 8
33849 084/300 Write block 3361 len 12288 data: 8 7 6
33850 085/300 Read block 2134 len 28672
33851 086/300 Read block 2999 len 24576
33852 087/300 Flush
33853 088/300 Read block 3197 len 12288
33854 089/300 Read block 1138 len 12288
33855 090/300 Read block 3338 len 12288
33856 091/300 Read block 4820 len 20480
33857 092/300 Write block 1438 len 20480 data: 7 7 7 8 8
33858 093/300 Read block 1337 len 24576
33859 094/300 Flush
33860 095/300 Write block 3547 len 40960 data: 4 4 5 7 7 6 7 5 5 8
33861 096/300 Read block 4599 len 28672
33862 097/300 Read block 4635 len 36864
33863 098/300 Write block 643 len 28672 data: 3 5 5 5 5 8 6
33864 099/300 Read block 2825 len 4096
33865 100/300 Read block 4165 len 28672
33866 101/300 Read block 275 len 12288
33867 102/300 Write block 2951 len 20480 data: 6 8 7 7 6
33868 103/300 Read block 2431 len 20480
33869 104/300 Flush
33870 105/300 Read block 3568 len 32768
33871 106/300 Write block 1752 len 16384 data: 3 4 6 5
33872 107/300 Read block 4873 len 32768
33873 108/300 Write block 2550 len 36864 data: 6 6 6 6 6 8 8 10 9
33874 109/300 Write block 3974 len 12288 data: 4 4 4
33875 110/300 Read block 2673 len 12288
33876 111/300 Read block 4280 len 24576
33877 112/300 Read block 1375 len 8192
33878 113/300 Read block 2199 len 40960
33879 114/300 Write block 1068 len 40960 data: 6 5 6 6 8 8 7 7 7 7
33880 115/300 Write block 4716 len 40960 data: 7 7 9 7 4 4 5 5 5 5
33881 116/300 Write block 566 len 4096 data: 3
33882 117/300 Read block 3988 len 36864
33883 118/300 Write block 4825 len 24576 data: 6 7 6 4 8 9
33884 119/300 Read block 606 len 24576
33885 120/300 Write block 4906 len 32768 data: 5 7 8 8 8 8 8 7
33886 121/300 Read block 2733 len 40960
33887 122/300 Read block 726 len 8192
33888 123/300 Read block 3885 len 28672
33889 124/300 Read block 3645 len 12288
33890 125/300 Read block 3935 len 24576
33891 126/300 Read block 3908 len 16384
33892 127/300 Read block 3584 len 36864
33893 128/300 Write block 1457 len 4096 data: 9
33894 129/300 Write block 1913 len 8192 data: 2 2
33895 130/300 Read block 1626 len 28672
33896 131/300 Write block 682 len 20480 data: 7 7 8 8 9
33897 132/300 Write block 1584 len 8192 data: 4 4
33898 133/300 Write block 4942 len 32768 data: 7 8 8 6 5 5 3 5
33899 134/300 Read block 1654 len 32768
33900 135/300 Flush
33901 136/300 Read block 4874 len 28672
33902 137/300 Flush
33903 138/300 Read block 797 len 8192
33904 139/300 Write block 846 len 16384 data: 7 5 7 6
33905 140/300 Read block 1674 len 28672
33906 141/300 Write block 1927 len 32768 data: 6 6 6 6 4 4 5 7
33907 142/300 Write block 4168 len 32768 data: 10 9 8 7 7 9 6 6
33908 143/300 Read block 2405 len 32768
33909 144/300 Write block 3886 len 32768 data: 5 5 7 7 5 4 4 6
33910 145/300 Read block 4777 len 24576
33911 146/300 Write block 330 len 8192 data: 7 6
33912 147/300 Write block 3638 len 12288 data: 6 4 4
33913 148/300 Write block 1672 len 12288 data: 5 6 9
33914 149/300 Write block 1700 len 12288 data: 5 5 5
33915 150/300 Read block 3691 len 16384
33916 151/300 Read block 3514 len 40960
33917 152/300 Write block 3846 len 8192 data: 7 6
33918 153/300 Read block 3685 len 20480
33919 154/300 Read block 870 len 24576
33920 155/300 Read block 2009 len 12288
33921 156/300 Read block 1303 len 8192
33922 157/300 Write block 1506 len 8192 data: 4 5
33923 158/300 Write block 4567 len 36864 data: 7 8 6 4 4 4 3 3 5
33924 159/300 Read block 930 len 16384
33925 160/300 Read block 27 len 24576
33926 161/300 Write block 2235 len 16384 data: 4 6 6 5
33927 162/300 Write block 4418 len 12288 data: 5 4 2
33928 163/300 Write block 4502 len 8192 data: 7 9
33929 164/300 Read block 133 len 12288
33930 165/300 Read block 2933 len 8192
33931 166/300 Write block 2779 len 20480 data: 7 6 6 8 8
33932 167/300 Read block 527 len 20480
33933 168/300 Read block 3791 len 32768
33934 169/300 Write block 2134 len 12288 data: 8 7 8
33935 170/300 Write block 2024 len 12288 data: 6 4 6
33936 171/300 Read block 3432 len 40960
33937 172/300 Write block 3597 len 16384 data: 5 4 6 7
33938 173/300 Flush
33939 174/300 Write block 3415 len 40960 data: 3 3 2 2 2 2 2 4 4 5
33940 175/300 Read block 49 len 40960
33941 176/300 Read block 3258 len 40960
33942 177/300 Flush
33943 178/300 Write block 2488 len 32768 data: 5 5 7 7 7 7 7 6
33944 179/300 Write block 2435 len 4096 data: 3
33945 180/300 Read block 3490 len 12288
33946 181/300 Read block 2324 len 40960
33947 182/300 Write block 609 len 24576 data: 6 6 6 5 6 7
33948 183/300 Write block 2496 len 36864 data: 5 4 6 6 5 6 6 5 3
33949 184/300 Write block 2505 len 16384 data: 3 5 6 8
33950 185/300 Read block 1152 len 40960
33951 186/300 Write block 4950 len 36864 data: 5 5 4 3 5 5 5 6 8
33952 187/300 Read block 1318 len 36864
33953 188/300 Write block 4113 len 4096 data: 7
33954 189/300 Write block 4074 len 32768 data: 7 5 3 5 5 4 6 6
33955 190/300 Read block 2725 len 24576
33956 191/300 Read block 1258 len 40960
33957 192/300 Read block 1114 len 40960
33958 193/300 Write block 4662 len 36864 data: 4 4 3 3 4 5 4 4 4
33959 194/300 Read block 858 len 28672
33960 195/300 Write block 4303 len 36864 data: 5 6 5 4 4 5 5 5 4
33961 196/300 Write block 2345 len 8192 data: 6 7
33962 197/300 Write block 4370 len 20480 data: 6 5 5 4 4
33963 198/300 Write block 826 len 16384 data: 7 6 8 7
33964 199/300 Read block 4021 len 32768
33965 200/300 Write block 16 len 24576 data: 6 6 4 4 4 4
33966 201/300 Read block 2233 len 4096
33967 202/300 Write block 4018 len 32768 data: 5 6 6 6 4 5 3 3
33968 203/300 Write block 1595 len 28672 data: 8 10 12 13 9 9 6
33969 204/300 Read block 3871 len 12288
33970 205/300 Write block 195 len 40960 data: 5 6 6 6 8 7 5 4 4 5
33971 206/300 Write block 2873 len 20480 data: 6 5 5 5 5
33972 207/300 Read block 754 len 28672
33973 208/300 Read block 1662 len 28672
33974 209/300 Read block 260 len 12288
33975 210/300 Flush
33976 211/300 Write block 2974 len 16384 data: 9 7 6 6
33977 212/300 Read block 1180 len 12288
33978 213/300 Write block 2818 len 20480 data: 5 5 6 5 5
33979 214/300 Flush
33980 215/300 Write block 3099 len 4096 data: 9
33981 216/300 Write block 75 len 32768 data: 4 5 5 6 7 6 6 5
33982 217/300 Write block 1292 len 8192 data: 5 5
33983 218/300 Flush
33984 219/300 Read block 382 len 8192
33985 220/300 Write block 2838 len 40960 data: 5 7 7 7 6 5 7 8 9 7
33986 221/300 Write block 4684 len 32768 data: 6 6 7 7 6 7 6 7
33987 222/300 Read block 3302 len 24576
33988 223/300 Write block 979 len 12288 data: 3 4 2
33989 224/300 Read block 3855 len 8192
33990 225/300 Read block 3648 len 40960
33991 226/300 Read block 4787 len 36864
33992 227/300 Write block 1360 len 4096 data: 10
33993 228/300 Read block 4101 len 12288
33994 229/300 Read block 4799 len 8192
33995 230/300 Write block 4913 len 12288 data: 8 6 5
33996 231/300 Write block 2791 len 20480 data: 6 5 7 8 7
33997 232/300 Write block 4066 len 8192 data: 5 4
33998 233/300 Read block 1607 len 4096
33999 234/300 Flush
34000 235/300 Read block 1389 len 40960
34001 236/300 Read block 4442 len 40960
34002 237/300 Read block 1467 len 32768
34003 238/300 Read block 2718 len 36864
34004 239/300 Write block 3827 len 4096 data: 5
34005 240/300 Flush
34006 241/300 Read block 2538 len 16384
34007 242/300 Read block 2356 len 20480
34008 243/300 Flush
34009 244/300 Write block 4466 len 8192 data: 6 5
34010 245/300 Write block 3386 len 28672 data: 6 7 8 9 8 7 6
34011 246/300 Write block 3778 len 20480 data: 7 7 7 6 6
34012 247/300 Read block 3023 len 32768
34013 248/300 Write block 1687 len 8192 data: 5 5
34014 249/300 Read block 4834 len 20480
34015 250/300 Write block 3021 len 16384 data: 7 7 8 9
34016 251/300 Flush
34017 252/300 Write block 2146 len 8192 data: 8 6
34018 253/300 Write block 2960 len 12288 data: 5 5 7
34019 254/300 Flush
34020 255/300 Read block 2383 len 32768
34021 256/300 Read block 4579 len 16384
34022 257/300 Read block 420 len 32768
34023 258/300 Read block 2812 len 36864
34024 259/300 Write block 1240 len 4096 data: 8
34025 260/300 Read block 3282 len 36864
34026 261/300 Write block 3621 len 4096 data: 10
34027 262/300 Read block 4935 len 8192
34028 263/300 Read block 2788 len 12288
34029 264/300 Read block 3085 len 36864
34030 265/300 Flush
34031 266/300 Read block 4286 len 32768
34032 267/300 Write block 2137 len 12288 data: 8 7 6
34033 268/300 Read block 3681 len 28672
34034 269/300 Write block 870 len 32768 data: 6 8 7 8 6 7 7 6
34035 270/300 Flush
34036 271/300 Read block 2075 len 40960
34037 272/300 Write block 4876 len 28672 data: 3 3 3 4 4 4 6
34038 273/300 Write block 2158 len 32768 data: 11 9 8 9 9 7 7 6
34039 274/300 Read block 447 len 4096
34040 275/300 Read block 3813 len 12288
34041 276/300 Read block 42 len 16384
34042 277/300 Read block 2076 len 12288
34043 278/300 Read block 2433 len 36864
34044 279/300 Write block 2174 len 24576 data: 9 10 11 8 8 7
34045 280/300 Read block 1950 len 28672
34046 281/300 Read block 648 len 4096
34047 282/300 Flush
34048 283/300 Write block 1565 len 40960 data: 4 3 3 4 6 6 7 8 9 8
34049 284/300 Read block 4414 len 24576
34050 285/300 Write block 508 len 24576 data: 6 5 7 9 8 6
34051 286/300 Read block 1686 len 32768
34052 287/300 Write block 3154 len 12288 data: 4 4 4
34053 288/300 Write block 1249 len 20480 data: 5 4 8 7 6
34054 289/300 Read block 1586 len 8192
34055 290/300 Write block 4961 len 4096 data: 10
34056 291/300 Write block 380 len 36864 data: 5 4 6 5 4 4 3 3 2
34057 292/300 Write block 22 len 40960 data: 3 4 5 5 4 4 6 5 5 4
34058 293/300 Read block 1757 len 36864
34059 294/300 Write block 3891 len 36864 data: 5 5 7 8 6 6 6 6 5
34060 295/300 Write block 4544 len 36864 data: 5 5 5 4 3 4 4 5 5
34061 296/300 Write block 3496 len 12288 data: 6 5 4
34062 297/300 Read block 2164 len 8192
34063 298/300 Write block 2294 len 28672 data: 4 4 4 4 6 7 6
34064 299/300 Write block 1198 len 16384 data: 9 7 6 6
34065 300/300 Write block 4538 len 32768 data: 6 4 5 5 4 4 6 6
34066 301/300 Read block 2520 len 40960
34067 Main task has work to do, go find it
34068 got dsc Start(2)
34069 start 2
34070 Jan 27 19:43:32.581 INFO request completed, latency_us: 191, response_code: 204, uri: /start/cid/2, method: POST, req_id: bf9b5e89-a792-44e7-9d64-3d2c07686bd9, remote_addr: 127.0.0.1:55047, local_addr: 127.0.0.1:9998
34071 Replay: started 2, returned:Ok(())
34072 ----------------------------------------------------------------
34073 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:301
34074 States: Active Active Offline
34075 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34076 10056 Acked Write 3 Done Done Sent false
34077 10057 Acked Write 3 Done Done Sent false
34078 10058 Acked Read 7 Done Done Sent false
34079 10059 Acked Read 9 Done Done Sent false
34080 10060 Acked Read 5 Done Done Sent false
34081 10061 Acked Write 9 Done Done Sent false
34082 10062 Acked Write 5 Done Done Sent false
34083 10063 Acked Write 3 Done Done Sent false
34084 10064 Acked Flush 0 Done Done Sent false
34085 10065 Acked Read 5 Done Done Sent false
34086 10066 Acked Write 8 Done Done Sent false
34087 10067 Acked Write 9 Done Done Sent false
34088 10068 Acked Write 2 Done Done Sent false
34089 10069 Acked Read 1 Done Done Sent false
34090 10070 Acked Write 1 Done Done Sent false
34091 10071 Acked Flush 0 Done Done Sent false
34092 10072 Acked Read 9 Done Done Sent false
34093 10073 Acked Write 4 Done Done Sent false
34094 10074 Acked Read 4 Done Done Sent false
34095 10075 Acked Read 7 Done Done Sent false
34096 10076 Acked Flush 0 Done Done Sent false
34097 10077 Acked Read 6 Done Done Sent false
34098 10078 Acked Write 9 Done Done Sent false
34099 10079 Acked Write 7 Done Done Sent false
34100 10080 Acked Read 10 Done Done Sent false
34101 10081 Acked Read 3 Done Done Sent false
34102 10082 Acked Write 4 Done Done Sent false
34103 10083 Acked Write 10 Done Done Sent false
34104 10084 Acked Write 7 Done Done Sent false
34105 10085 Acked Flush 0 Done Done Sent false
34106 10086 Acked Write 2 Done Done Sent false
34107 10087 Acked Write 8 Done Done Sent false
34108 10088 Acked Read 2 Done Done Sent false
34109 10089 Acked Read 10 Done Done Sent false
34110 10090 Acked Read 5 Done Done Sent false
34111 10091 Acked Read 2 Done Done Sent false
34112 10092 Acked Write 4 Done Done Sent false
34113 10093 Acked Write 5 Done Done Sent false
34114 10094 Acked Read 3 Done Done Sent false
34115 10095 Acked Write 9 Done Done Sent false
34116 10096 Acked Read 7 Done Done Sent false
34117 10097 Acked Read 7 Done Done Sent false
34118 10098 Acked Read 8 Done Done Sent false
34119 10099 Acked Write 2 Done Done Sent false
34120 10100 Acked Read 9 Done Done Sent false
34121 10101 Acked Write 5 Done Done Sent false
34122 10102 Acked Read 2 Done Done Sent false
34123 10103 Acked Read 3 Done Done Sent false
34124 10104 Acked Flush 0 Done Done Sent false
34125 10105 Acked Read 5 Done Done Sent false
34126 10106 Acked Write 9 Done Done Sent false
34127 10107 Acked Read 1 Done Done Sent false
34128 10108 Acked Read 8 Done Done Sent false
34129 10109 Acked Read 8 Done Done Sent false
34130 10110 Acked Write 9 Done Done Sent false
34131 10111 Acked Write 3 Done Done Sent false
34132 10112 Acked Read 8 Done Done Sent false
34133 10113 Acked Read 5 Done Done Sent false
34134 10114 Acked Write 5 Done Done Sent false
34135 10115 Acked Read 8 Done Done Sent false
34136 10116 Acked Flush 0 Done Done Sent false
34137 10117 Acked Flush 0 Done Done Sent false
34138 10118 Acked Flush 0 Done Done Sent false
34139 10119 Acked Write 3 Done Done Sent false
34140 10120 Acked Flush 0 Done Done Sent false
34141 10121 Acked Read 7 Done Done Sent false
34142 10122 Acked Read 4 Done Done Sent false
34143 10123 Acked Write 3 Done Done Sent false
34144 10124 Acked Read 1 Done Done Sent false
34145 10125 Acked Write 4 Done Done Sent false
34146 10126 Acked Read 1 Done Done Sent false
34147 10127 Acked Read 7 Done Done Sent false
34148 10128 Acked Flush 0 Done Done Sent false
34149 10129 Acked Write 1 Done Done Sent false
34150 10130 Acked Write 5 Done Done Sent false
34151 10131 Acked Read 6 Done Done Sent false
34152 10132 Acked Read 6 Done Done Sent false
34153 10133 Acked Read 9 Done Done Sent false
34154 10134 Acked Flush 0 Done Done Sent false
34155 10135 Acked Read 8 Done Done Sent false
34156 10136 Acked Read 7 Done Done Sent false
34157 10137 Acked Write 1 Done Done Sent false
34158 10138 Acked Write 1 Done Done Sent false
34159 10139 Acked Write 3 Done Done Sent false
34160 10140 Acked Read 7 Done Done Sent false
34161 10141 Acked Read 6 Done Done Sent false
34162 10142 Acked Flush 0 Done Done Sent false
34163 10143 Acked Read 3 Done Done Sent false
34164 10144 Acked Read 3 Done Done Sent false
34165 10145 Acked Read 3 Done Done Sent false
34166 10146 Acked Read 5 Done Done Sent false
34167 10147 Acked Write 5 Done Done Sent false
34168 10148 Acked Read 6 Done Done Sent false
34169 10149 Acked Flush 0 Done Done Sent false
34170 10150 Acked Write 10 Done Done Sent false
34171 10151 Acked Read 7 Done Done Sent false
34172 10152 Acked Read 9 Done Done Sent false
34173 10153 Acked Write 7 Done Done Sent false
34174 10154 Acked Read 1 Done Done Sent false
34175 10155 Acked Read 7 Done Done Sent false
34176 10156 Acked Read 3 Done Done Sent false
34177 10157 Acked Write 5 Done Done Sent false
34178 10158 Acked Read 5 Done Done Sent false
34179 10159 Acked Flush 0 Done Done Sent false
34180 10160 Acked Read 8 Done Done Sent false
34181 10161 Acked Write 4 Done Done Sent false
34182 10162 Acked Read 8 Done Done Sent false
34183 10163 Acked Write 9 Done Done Sent false
34184 10164 Acked Write 3 Done Done Sent false
34185 10165 Acked Read 3 Done Done Sent false
34186 10166 Acked Read 6 Done Done Sent false
34187 10167 Acked Read 2 Done Done Sent false
34188 10168 Acked Read 10 Done Done Sent false
34189 10169 Acked Write 10 Done Done Sent false
34190 10170 Acked Write 10 Done Done Sent false
34191 10171 Acked Write 1 Done Done Sent false
34192 10172 Acked Read 9 Done Done Sent false
34193 10173 Acked Write 6 Done Done Sent false
34194 10174 Acked Read 6 Done Done Sent false
34195 10175 Acked Write 8 Done Done Sent false
34196 10176 Acked Read 10 Done Done Sent false
34197 10177 Acked Read 2 Done Done Sent false
34198 10178 Acked Read 7 Done Done Sent false
34199 10179 Acked Read 3 Done Done Sent false
34200 10180 Acked Read 6 Done Done Sent false
34201 10181 Acked Read 4 Done Done Sent false
34202 10182 Acked Read 9 Done Done Sent false
34203 10183 Acked Write 1 Done Done Sent false
34204 10184 Acked Write 2 Done Done Sent false
34205 10185 Acked Read 7 Done Done Sent false
34206 10186 Acked Write 5 Done Done Sent false
34207 10187 Acked Write 2 Done Done Sent false
34208 10188 Acked Write 8 Done Done Sent false
34209 10189 Acked Read 8 Done Done Sent false
34210 10190 Acked Flush 0 Done Done Sent false
34211 10191 Acked Read 7 Done Done Sent false
34212 10192 Acked Flush 0 Done Done Sent false
34213 10193 Acked Read 2 Done Done Sent false
34214 10194 Acked Write 4 Done Done Sent false
34215 10195 Acked Read 7 Done Done Sent false
34216 10196 Acked Write 8 Done Done Sent false
34217 10197 Acked Write 8 Done Done Sent false
34218 10198 Acked Read 8 Done Done Sent false
34219 10199 Acked Write 8 Done Done Sent false
34220 10200 Acked Read 6 Done Done Sent false
34221 10201 Acked Write 2 Done Done Sent false
34222 10202 Acked Write 3 Done Done Sent false
34223 10203 Acked Write 3 Done Done Sent false
34224 10204 Acked Write 3 Done Done Sent false
34225 10205 Acked Read 4 Done Done Sent false
34226 10206 Acked Read 10 Done Done Sent false
34227 10207 Acked Write 2 Done Done Sent false
34228 10208 Acked Read 5 Done Done Sent false
34229 10209 Acked Read 6 Done Done Sent false
34230 10210 Acked Read 3 Done Done Sent false
34231 10211 Acked Read 2 Done Done Sent false
34232 10212 Acked Write 2 Done Done Sent false
34233 10213 Acked Write 9 Done Done Sent false
34234 10214 Acked Read 4 Done Done Sent false
34235 10215 Acked Read 6 Done Done Sent false
34236 10216 Acked Write 4 Done Done Sent false
34237 10217 Acked Write 3 Done Done Sent false
34238 10218 Acked Write 2 Done Done Sent false
34239 10219 Acked Read 3 Done Done Sent false
34240 10220 Acked Read 2 Done Done Sent false
34241 10221 Acked Write 5 Done Done Sent false
34242 10222 Acked Read 5 Done Done Sent false
34243 10223 Acked Read 8 Done Done Sent false
34244 10224 Acked Write 3 Done Done Sent false
34245 10225 Acked Write 3 Done Done Sent false
34246 10226 Acked Read 10 Done Done Sent false
34247 10227 Acked Write 4 Done Done Sent false
34248 10228 Acked Flush 0 Done Done Sent false
34249 10229 Acked Write 10 Done Done Sent false
34250 10230 Acked Read 10 Done Done Sent false
34251 10231 Acked Read 10 Done Done Sent false
34252 10232 Acked Flush 0 Done Done Sent false
34253 10233 Acked Write 8 Done Done Sent false
34254 10234 Acked Write 1 Done Done Sent false
34255 10235 Acked Read 3 Done Done Sent false
34256 10236 Acked Read 10 Done Done Sent false
34257 10237 Acked Write 6 Done Done Sent false
34258 10238 Acked Write 9 Done Done Sent false
34259 10239 Acked Write 4 Done Done Sent false
34260 10240 Acked Read 10 Done Done Sent false
34261 10241 Acked Write 9 Done Done Sent false
34262 10242 Acked Read 9 Done Done Sent false
34263 10243 Acked Write 1 Done Done Sent false
34264 10244 Acked Write 8 Done Done Sent false
34265 10245 Acked Read 6 Done Done Sent false
34266 10246 Acked Read 10 Done Done Sent false
34267 10247 Acked Read 10 Done Done Sent false
34268 10248 Acked Write 9 Done Done Sent false
34269 10249 Acked Read 7 Done Done Sent false
34270 10250 Acked Write 9 Done Done Sent false
34271 10251 Acked Write 2 Done Done Sent false
34272 10252 Acked Write 5 Done Done Sent false
34273 10253 Acked Write 4 Done Done Sent false
34274 10254 Acked Read 8 Done Done Sent false
34275 10255 Acked Write 6 Done Done Sent false
34276 10256 Acked Read 1 Done Done Sent false
34277 10257 Acked Write 8 Done Done Sent false
34278 10258 Acked Write 7 Done Done Sent false
34279 10259 Acked Read 3 Done Done Sent false
34280 10260 Acked Write 10 Done Done Sent false
34281 10261 Acked Write 5 Done Done Sent false
34282 10262 Acked Read 7 Done Done Sent false
34283 10263 Acked Read 7 Done Done Sent false
34284 10264 Acked Read 3 Done Done Sent false
34285 10265 Acked Flush 0 Done Done Sent false
34286 10266 Acked Write 4 Done Done Sent false
34287 10267 Acked Read 3 Done Done Sent false
34288 10268 Acked Write 5 Done Done Sent false
34289 10269 Acked Flush 0 Done Done Sent false
34290 10270 Acked Write 1 Done Done Sent false
34291 10271 Acked Write 8 Done Done Sent false
34292 10272 Acked Write 2 Done Done Sent false
34293 10273 Acked Flush 0 Done Done Sent false
34294 10274 Acked Read 2 Done Done Sent false
34295 10275 Acked Write 10 Done Done Sent false
34296 10276 Acked Write 8 Done Done Sent false
34297 10277 Acked Read 6 Done Done Sent false
34298 10278 Acked Write 3 Done Done Sent false
34299 10279 Acked Read 2 Done Done Sent false
34300 10280 Acked Read 10 Done Done Sent false
34301 10281 Acked Read 9 Done Done Sent false
34302 10282 Acked Write 1 Done Done Sent false
34303 10283 Acked Read 3 Done Done Sent false
34304 10284 Acked Read 2 Done Done Sent false
34305 10285 Acked Write 3 Done Done Sent false
34306 10286 Acked Write 5 Done Done Sent false
34307 10287 Acked Write 2 Done Done Sent false
34308 10288 Acked Read 1 Done Done Sent false
34309 10289 Acked Flush 0 Done Done Sent false
34310 10290 Acked Read 10 Done Done Sent false
34311 10291 Acked Read 10 Done Done Sent false
34312 10292 Acked Read 8 Done Done Sent false
34313 10293 Acked Read 9 Done Done Sent false
34314 10294 Acked Write 1 Done Done Sent false
34315 10295 Acked Flush 0 Done Done Sent false
34316 10296 Acked Read 4 Done Done Sent false
34317 10297 Acked Read 5 Done Done Sent false
34318 10298 Acked Flush 0 Done Done Sent false
34319 10299 Acked Write 2 Done Done Sent false
34320 10300 Acked Write 7 Done Done Sent false
34321 10301 Acked Write 5 Done Done Sent false
34322 10302 Acked Read 8 Done Done Sent false
34323 10303 Acked Write 2 Done Done Sent false
34324 10304 Acked Read 5 Done Done Sent false
34325 10305 Acked Write 4 Done Done Sent false
34326 10306 Acked Flush 0 Done Done Sent false
34327 10307 Acked Write 2 Done Done Sent false
34328 10308 Acked Write 3 Done Done Sent false
34329 10309 Acked Flush 0 Done Done Sent false
34330 10310 Acked Read 8 Done Done Sent false
34331 10311 Acked Read 4 Done Done Sent false
34332 10312 Acked Read 8 Done Done Sent false
34333 10313 Acked Read 9 Done Done Sent false
34334 10314 Acked Write 1 Done Done Sent false
34335 10315 Acked Read 9 Done Done Sent false
34336 10316 Acked Write 1 Done Done Sent false
34337 10317 Acked Read 2 Done Done Sent false
34338 10318 Acked Read 3 Done Done Sent false
34339 10319 Acked Read 9 Done Done Sent false
34340 10320 Acked Flush 0 Done Done Sent false
34341 10321 Acked Read 8 Done Done Sent false
34342 10322 Acked Write 3 Done Done Sent false
34343 10323 Acked Read 7 Done Done Sent false
34344 10324 Acked Write 8 Done Done Sent false
34345 10325 Acked Flush 0 Done Done Sent false
34346 10326 Acked Read 10 Done Done Sent false
34347 10327 Acked Write 7 Done Done Sent false
34348 10328 Acked Write 8 Done Done Sent false
34349 10329 Acked Read 1 Done Done Sent false
34350 10330 Acked Read 3 Done Done Sent false
34351 10331 Acked Read 4 Done Done Sent false
34352 10332 Acked Read 3 Done Done Sent false
34353 10333 Acked Read 9 Done Done Sent false
34354 10334 Acked Write 6 Done Done Sent false
34355 10335 Acked Read 7 Done Done Sent false
34356 10336 Acked Read 1 Done Done Sent false
34357 10337 Acked Flush 0 Done Done Sent false
34358 10338 Acked Write 10 Done Done Sent false
34359 10339 Acked Read 6 Done Done Sent false
34360 10340 Acked Write 6 Done Done Sent false
34361 10341 Acked Read 8 Done Done Sent false
34362 10342 Acked Write 3 Done Done Sent false
34363 10343 Acked Write 5 Done Done Sent false
34364 10344 Acked Read 2 Done Done Sent false
34365 10345 Acked Write 1 Done Done Sent false
34366 10346 Acked Write 9 Done Done Sent false
34367 10347 Acked Write 10 Done Done Sent false
34368 10348 Acked Read 9 Done Done Sent false
34369 10349 Acked Write 9 Done Done Sent false
34370 10350 Acked Write 9 Done Done Sent false
34371 10351 Acked Write 3 Done Done Sent false
34372 10352 Acked Read 2 Done Done Sent false
34373 10353 Acked Write 7 Done Done Sent false
34374 10354 Acked Write 4 Done Done Sent false
34375 10355 Acked Write 8 Done Done Sent false
34376 10356 Acked Read 10 Done Done Sent false
34377 STATES DS:0 DS:1 DS:2 TOTAL
34378 Sent 0 0 301 301
34379 Done 301 301 0 602
34380 Skipped 0 0 0 0
34381 Error 0 0 0 0
34382 Last Flush: 10337 10337 10055
34383 Downstairs last five completed: 10055 10054 10053 10052 10051
34384 Upstairs last five completed: 10356 10355 10354 10353 10352
34385 CLIENT: Up:0 ds:301 act:2
34386 [8830] got start action
34387 [8830] I am going to restart
34388 Starting downstairs at port 8830
34389 Make output file at "/tmp/dsc/downstairs-8830.txt"
34390 [8830][2] reports Starting
34391 Downstairs /var/tmp/dsc/region/8830 port 8830 PID:Some(1231)
34392 [8830][2] reports Running
34393 ----------------------------------------------------------------
34394 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:302
34395 States: Active Active Offline
34396 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34397 10056 Acked Write 3 Done Done Sent false
34398 10057 Acked Write 3 Done Done Sent false
34399 10058 Acked Read 7 Done Done Sent false
34400 10059 Acked Read 9 Done Done Sent false
34401 10060 Acked Read 5 Done Done Sent false
34402 10061 Acked Write 9 Done Done Sent false
34403 10062 Acked Write 5 Done Done Sent false
34404 10063 Acked Write 3 Done Done Sent false
34405 10064 Acked Flush 0 Done Done Sent false
34406 10065 Acked Read 5 Done Done Sent false
34407 10066 Acked Write 8 Done Done Sent false
34408 10067 Acked Write 9 Done Done Sent false
34409 10068 Acked Write 2 Done Done Sent false
34410 10069 Acked Read 1 Done Done Sent false
34411 10070 Acked Write 1 Done Done Sent false
34412 10071 Acked Flush 0 Done Done Sent false
34413 10072 Acked Read 9 Done Done Sent false
34414 10073 Acked Write 4 Done Done Sent false
34415 10074 Acked Read 4 Done Done Sent false
34416 10075 Acked Read 7 Done Done Sent false
34417 10076 Acked Flush 0 Done Done Sent false
34418 10077 Acked Read 6 Done Done Sent false
34419 10078 Acked Write 9 Done Done Sent false
34420 10079 Acked Write 7 Done Done Sent false
34421 10080 Acked Read 10 Done Done Sent false
34422 10081 Acked Read 3 Done Done Sent false
34423 10082 Acked Write 4 Done Done Sent false
34424 10083 Acked Write 10 Done Done Sent false
34425 10084 Acked Write 7 Done Done Sent false
34426 10085 Acked Flush 0 Done Done Sent false
34427 10086 Acked Write 2 Done Done Sent false
34428 10087 Acked Write 8 Done Done Sent false
34429 10088 Acked Read 2 Done Done Sent false
34430 10089 Acked Read 10 Done Done Sent false
34431 10090 Acked Read 5 Done Done Sent false
34432 10091 Acked Read 2 Done Done Sent false
34433 10092 Acked Write 4 Done Done Sent false
34434 10093 Acked Write 5 Done Done Sent false
34435 10094 Acked Read 3 Done Done Sent false
34436 10095 Acked Write 9 Done Done Sent false
34437 10096 Acked Read 7 Done Done Sent false
34438 10097 Acked Read 7 Done Done Sent false
34439 10098 Acked Read 8 Done Done Sent false
34440 10099 Acked Write 2 Done Done Sent false
34441 10100 Acked Read 9 Done Done Sent false
34442 10101 Acked Write 5 Done Done Sent false
34443 10102 Acked Read 2 Done Done Sent false
34444 10103 Acked Read 3 Done Done Sent false
34445 10104 Acked Flush 0 Done Done Sent false
34446 10105 Acked Read 5 Done Done Sent false
34447 10106 Acked Write 9 Done Done Sent false
34448 10107 Acked Read 1 Done Done Sent false
34449 10108 Acked Read 8 Done Done Sent false
34450 10109 Acked Read 8 Done Done Sent false
34451 10110 Acked Write 9 Done Done Sent false
34452 10111 Acked Write 3 Done Done Sent false
34453 10112 Acked Read 8 Done Done Sent false
34454 10113 Acked Read 5 Done Done Sent false
34455 10114 Acked Write 5 Done Done Sent false
34456 10115 Acked Read 8 Done Done Sent false
34457 10116 Acked Flush 0 Done Done Sent false
34458 10117 Acked Flush 0 Done Done Sent false
34459 10118 Acked Flush 0 Done Done Sent false
34460 10119 Acked Write 3 Done Done Sent false
34461 10120 Acked Flush 0 Done Done Sent false
34462 10121 Acked Read 7 Done Done Sent false
34463 10122 Acked Read 4 Done Done Sent false
34464 10123 Acked Write 3 Done Done Sent false
34465 10124 Acked Read 1 Done Done Sent false
34466 10125 Acked Write 4 Done Done Sent false
34467 10126 Acked Read 1 Done Done Sent false
34468 10127 Acked Read 7 Done Done Sent false
34469 10128 Acked Flush 0 Done Done Sent false
34470 10129 Acked Write 1 Done Done Sent false
34471 10130 Acked Write 5 Done Done Sent false
34472 10131 Acked Read 6 Done Done Sent false
34473 10132 Acked Read 6 Done Done Sent false
34474 10133 Acked Read 9 Done Done Sent false
34475 10134 Acked Flush 0 Done Done Sent false
34476 10135 Acked Read 8 Done Done Sent false
34477 10136 Acked Read 7 Done Done Sent false
34478 10137 Acked Write 1 Done Done Sent false
34479 10138 Acked Write 1 Done Done Sent false
34480 10139 Acked Write 3 Done Done Sent false
34481 10140 Acked Read 7 Done Done Sent false
34482 10141 Acked Read 6 Done Done Sent false
34483 10142 Acked Flush 0 Done Done Sent false
34484 10143 Acked Read 3 Done Done Sent false
34485 10144 Acked Read 3 Done Done Sent false
34486 10145 Acked Read 3 Done Done Sent false
34487 10146 Acked Read 5 Done Done Sent false
34488 10147 Acked Write 5 Done Done Sent false
34489 10148 Acked Read 6 Done Done Sent false
34490 10149 Acked Flush 0 Done Done Sent false
34491 10150 Acked Write 10 Done Done Sent false
34492 10151 Acked Read 7 Done Done Sent false
34493 10152 Acked Read 9 Done Done Sent false
34494 10153 Acked Write 7 Done Done Sent false
34495 10154 Acked Read 1 Done Done Sent false
34496 10155 Acked Read 7 Done Done Sent false
34497 10156 Acked Read 3 Done Done Sent false
34498 10157 Acked Write 5 Done Done Sent false
34499 10158 Acked Read 5 Done Done Sent false
34500 10159 Acked Flush 0 Done Done Sent false
34501 10160 Acked Read 8 Done Done Sent false
34502 10161 Acked Write 4 Done Done Sent false
34503 10162 Acked Read 8 Done Done Sent false
34504 10163 Acked Write 9 Done Done Sent false
34505 10164 Acked Write 3 Done Done Sent false
34506 10165 Acked Read 3 Done Done Sent false
34507 10166 Acked Read 6 Done Done Sent false
34508 10167 Acked Read 2 Done Done Sent false
34509 10168 Acked Read 10 Done Done Sent false
34510 10169 Acked Write 10 Done Done Sent false
34511 10170 Acked Write 10 Done Done Sent false
34512 10171 Acked Write 1 Done Done Sent false
34513 10172 Acked Read 9 Done Done Sent false
34514 10173 Acked Write 6 Done Done Sent false
34515 10174 Acked Read 6 Done Done Sent false
34516 10175 Acked Write 8 Done Done Sent false
34517 10176 Acked Read 10 Done Done Sent false
34518 10177 Acked Read 2 Done Done Sent false
34519 10178 Acked Read 7 Done Done Sent false
34520 10179 Acked Read 3 Done Done Sent false
34521 10180 Acked Read 6 Done Done Sent false
34522 10181 Acked Read 4 Done Done Sent false
34523 10182 Acked Read 9 Done Done Sent false
34524 10183 Acked Write 1 Done Done Sent false
34525 10184 Acked Write 2 Done Done Sent false
34526 10185 Acked Read 7 Done Done Sent false
34527 10186 Acked Write 5 Done Done Sent false
34528 10187 Acked Write 2 Done Done Sent false
34529 10188 Acked Write 8 Done Done Sent false
34530 10189 Acked Read 8 Done Done Sent false
34531 10190 Acked Flush 0 Done Done Sent false
34532 10191 Acked Read 7 Done Done Sent false
34533 10192 Acked Flush 0 Done Done Sent false
34534 10193 Acked Read 2 Done Done Sent false
34535 10194 Acked Write 4 Done Done Sent false
34536 10195 Acked Read 7 Done Done Sent false
34537 10196 Acked Write 8 Done Done Sent false
34538 10197 Acked Write 8 Done Done Sent false
34539 10198 Acked Read 8 Done Done Sent false
34540 10199 Acked Write 8 Done Done Sent false
34541 10200 Acked Read 6 Done Done Sent false
34542 10201 Acked Write 2 Done Done Sent false
34543 10202 Acked Write 3 Done Done Sent false
34544 10203 Acked Write 3 Done Done Sent false
34545 10204 Acked Write 3 Done Done Sent false
34546 10205 Acked Read 4 Done Done Sent false
34547 10206 Acked Read 10 Done Done Sent false
34548 10207 Acked Write 2 Done Done Sent false
34549 10208 Acked Read 5 Done Done Sent false
34550 10209 Acked Read 6 Done Done Sent false
34551 10210 Acked Read 3 Done Done Sent false
34552 10211 Acked Read 2 Done Done Sent false
34553 10212 Acked Write 2 Done Done Sent false
34554 10213 Acked Write 9 Done Done Sent false
34555 10214 Acked Read 4 Done Done Sent false
34556 10215 Acked Read 6 Done Done Sent false
34557 10216 Acked Write 4 Done Done Sent false
34558 10217 Acked Write 3 Done Done Sent false
34559 10218 Acked Write 2 Done Done Sent false
34560 10219 Acked Read 3 Done Done Sent false
34561 10220 Acked Read 2 Done Done Sent false
34562 10221 Acked Write 5 Done Done Sent false
34563 10222 Acked Read 5 Done Done Sent false
34564 10223 Acked Read 8 Done Done Sent false
34565 10224 Acked Write 3 Done Done Sent false
34566 10225 Acked Write 3 Done Done Sent false
34567 10226 Acked Read 10 Done Done Sent false
34568 10227 Acked Write 4 Done Done Sent false
34569 10228 Acked Flush 0 Done Done Sent false
34570 10229 Acked Write 10 Done Done Sent false
34571 10230 Acked Read 10 Done Done Sent false
34572 10231 Acked Read 10 Done Done Sent false
34573 10232 Acked Flush 0 Done Done Sent false
34574 10233 Acked Write 8 Done Done Sent false
34575 10234 Acked Write 1 Done Done Sent false
34576 10235 Acked Read 3 Done Done Sent false
34577 10236 Acked Read 10 Done Done Sent false
34578 10237 Acked Write 6 Done Done Sent false
34579 10238 Acked Write 9 Done Done Sent false
34580 10239 Acked Write 4 Done Done Sent false
34581 10240 Acked Read 10 Done Done Sent false
34582 10241 Acked Write 9 Done Done Sent false
34583 10242 Acked Read 9 Done Done Sent false
34584 10243 Acked Write 1 Done Done Sent false
34585 10244 Acked Write 8 Done Done Sent false
34586 10245 Acked Read 6 Done Done Sent false
34587 10246 Acked Read 10 Done Done Sent false
34588 10247 Acked Read 10 Done Done Sent false
34589 10248 Acked Write 9 Done Done Sent false
34590 10249 Acked Read 7 Done Done Sent false
34591 10250 Acked Write 9 Done Done Sent false
34592 10251 Acked Write 2 Done Done Sent false
34593 10252 Acked Write 5 Done Done Sent false
34594 10253 Acked Write 4 Done Done Sent false
34595 10254 Acked Read 8 Done Done Sent false
34596 10255 Acked Write 6 Done Done Sent false
34597 10256 Acked Read 1 Done Done Sent false
34598 10257 Acked Write 8 Done Done Sent false
34599 10258 Acked Write 7 Done Done Sent false
34600 10259 Acked Read 3 Done Done Sent false
34601 10260 Acked Write 10 Done Done Sent false
34602 10261 Acked Write 5 Done Done Sent false
34603 10262 Acked Read 7 Done Done Sent false
34604 10263 Acked Read 7 Done Done Sent false
34605 10264 Acked Read 3 Done Done Sent false
34606 10265 Acked Flush 0 Done Done Sent false
34607 10266 Acked Write 4 Done Done Sent false
34608 10267 Acked Read 3 Done Done Sent false
34609 10268 Acked Write 5 Done Done Sent false
34610 10269 Acked Flush 0 Done Done Sent false
34611 10270 Acked Write 1 Done Done Sent false
34612 10271 Acked Write 8 Done Done Sent false
34613 10272 Acked Write 2 Done Done Sent false
34614 10273 Acked Flush 0 Done Done Sent false
34615 10274 Acked Read 2 Done Done Sent false
34616 10275 Acked Write 10 Done Done Sent false
34617 10276 Acked Write 8 Done Done Sent false
34618 10277 Acked Read 6 Done Done Sent false
34619 10278 Acked Write 3 Done Done Sent false
34620 10279 Acked Read 2 Done Done Sent false
34621 10280 Acked Read 10 Done Done Sent false
34622 10281 Acked Read 9 Done Done Sent false
34623 10282 Acked Write 1 Done Done Sent false
34624 10283 Acked Read 3 Done Done Sent false
34625 10284 Acked Read 2 Done Done Sent false
34626 10285 Acked Write 3 Done Done Sent false
34627 10286 Acked Write 5 Done Done Sent false
34628 10287 Acked Write 2 Done Done Sent false
34629 10288 Acked Read 1 Done Done Sent false
34630 10289 Acked Flush 0 Done Done Sent false
34631 10290 Acked Read 10 Done Done Sent false
34632 10291 Acked Read 10 Done Done Sent false
34633 10292 Acked Read 8 Done Done Sent false
34634 10293 Acked Read 9 Done Done Sent false
34635 10294 Acked Write 1 Done Done Sent false
34636 10295 Acked Flush 0 Done Done Sent false
34637 10296 Acked Read 4 Done Done Sent false
34638 10297 Acked Read 5 Done Done Sent false
34639 10298 Acked Flush 0 Done Done Sent false
34640 10299 Acked Write 2 Done Done Sent false
34641 10300 Acked Write 7 Done Done Sent false
34642 10301 Acked Write 5 Done Done Sent false
34643 10302 Acked Read 8 Done Done Sent false
34644 10303 Acked Write 2 Done Done Sent false
34645 10304 Acked Read 5 Done Done Sent false
34646 10305 Acked Write 4 Done Done Sent false
34647 10306 Acked Flush 0 Done Done Sent false
34648 10307 Acked Write 2 Done Done Sent false
34649 10308 Acked Write 3 Done Done Sent false
34650 10309 Acked Flush 0 Done Done Sent false
34651 10310 Acked Read 8 Done Done Sent false
34652 10311 Acked Read 4 Done Done Sent false
34653 10312 Acked Read 8 Done Done Sent false
34654 10313 Acked Read 9 Done Done Sent false
34655 10314 Acked Write 1 Done Done Sent false
34656 10315 Acked Read 9 Done Done Sent false
34657 10316 Acked Write 1 Done Done Sent false
34658 10317 Acked Read 2 Done Done Sent false
34659 10318 Acked Read 3 Done Done Sent false
34660 10319 Acked Read 9 Done Done Sent false
34661 10320 Acked Flush 0 Done Done Sent false
34662 10321 Acked Read 8 Done Done Sent false
34663 10322 Acked Write 3 Done Done Sent false
34664 10323 Acked Read 7 Done Done Sent false
34665 10324 Acked Write 8 Done Done Sent false
34666 10325 Acked Flush 0 Done Done Sent false
34667 10326 Acked Read 10 Done Done Sent false
34668 10327 Acked Write 7 Done Done Sent false
34669 10328 Acked Write 8 Done Done Sent false
34670 10329 Acked Read 1 Done Done Sent false
34671 10330 Acked Read 3 Done Done Sent false
34672 10331 Acked Read 4 Done Done Sent false
34673 10332 Acked Read 3 Done Done Sent false
34674 10333 Acked Read 9 Done Done Sent false
34675 10334 Acked Write 6 Done Done Sent false
34676 10335 Acked Read 7 Done Done Sent false
34677 10336 Acked Read 1 Done Done Sent false
34678 10337 Acked Flush 0 Done Done Sent false
34679 10338 Acked Write 10 Done Done Sent false
34680 10339 Acked Read 6 Done Done Sent false
34681 10340 Acked Write 6 Done Done Sent false
34682 10341 Acked Read 8 Done Done Sent false
34683 10342 Acked Write 3 Done Done Sent false
34684 10343 Acked Write 5 Done Done Sent false
34685 10344 Acked Read 2 Done Done Sent false
34686 10345 Acked Write 1 Done Done Sent false
34687 10346 Acked Write 9 Done Done Sent false
34688 10347 Acked Write 10 Done Done Sent false
34689 10348 Acked Read 9 Done Done Sent false
34690 10349 Acked Write 9 Done Done Sent false
34691 10350 Acked Write 9 Done Done Sent false
34692 10351 Acked Write 3 Done Done Sent false
34693 10352 Acked Read 2 Done Done Sent false
34694 10353 Acked Write 7 Done Done Sent false
34695 10354 Acked Write 4 Done Done Sent false
34696 10355 Acked Write 8 Done Done Sent false
34697 10356 Acked Read 10 Done Done Sent false
34698 10357 Acked Flush 0 Done Done Sent false
34699 STATES DS:0 DS:1 DS:2 TOTAL
34700 Sent 0 0 302 302
34701 Done 302 302 0 604
34702 Skipped 0 0 0 0
34703 Error 0 0 0 0
34704 Last Flush: 10357 10357 10055
34705 Downstairs last five completed: 10055 10054 10053 10052 10051
34706 Upstairs last five completed: 10357 10356 10355 10354 10353
34707 CLIENT: Up:0 ds:302 act:2
347082025-01-27T19:43:38.345ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
347092025-01-27T19:43:38.346ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:38327) = io task client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
347102025-01-27T19:43:38.347ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: Offline } = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
347112025-01-27T19:43:38.347ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
347122025-01-27T19:43:38.347ZINFOcrucible: Returning UUID:12345678-0000-0000-0000-000000008830 matches = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
347132025-01-27T19:43:38.347ZINFOcrucible: send last flush ID to this DS: 10055 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
347142025-01-27T19:43:38.347ZINFOcrucible: Replied this last flush ID: 10055 = downstairs client = 2 session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
347152025-01-27T19:43:38.347ZINFOcrucible: [2] client re-new 302 jobs since flush 10055 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
347162025-01-27T19:43:38.348ZINFOcrucible: [2] Replayed 302 jobs since flush: 10055 = downstairs session_id = edf4bf43-2393-40de-98dd-2a3c00d4e1dd
34717 ----------------------------------------------------------------
34718 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34719 Downstairs last five completed: 10357 10356 10355 10354 10353
34720 Upstairs last five completed: 10357 10356 10355 10354 10353
34721 CLIENT: Up:0 ds:0 act:3
34722 Replay: All jobs finished
34723 Test replay has completed
34724 Wrote out file "/tmp/test_replay_verify.log"
34725 CLIENT: Tests done. All submitted work has been ACK'd
34726 ----------------------------------------------------------------
34727 Crucible gen:2 GIO:true work queues: Upstairs:0 downstairs:0
34728 Downstairs last five completed: 10357 10356 10355 10354 10353
34729 Upstairs last five completed: 10357 10356 10355 10354 10353
34730 CLIENT: Up:0 ds:0 act:3
34731 CLIENT: All jobs finished, all DS active.
347322025-01-27T19:43:40.589ZWARNcrucible: Guest handle has been dropped
34733 Replays:30 time: 7:16
34734 Do final verify
34735 Jan 27 19:43:40.677 INFO accepted connection, remote_addr: 127.0.0.1:39659, local_addr: 127.0.0.1:9998
34736 Jan 27 19:43:40.677 INFO request completed, latency_us: 209, response_code: 200, uri: /regioninfo, method: GET, req_id: 7d8b5577-b640-42d6-8b56-be4688cb1480, remote_addr: 127.0.0.1:39659, local_addr: 127.0.0.1:9998
34737 Jan 27 19:43:40.678 INFO request completed, latency_us:{"msg":"Use this region info from dsc: RegionExtentInfo { block_size: 4096, blocks_per_extent: 100, extent_count: 50 }","v":0,"name":"crucible","level":30,"time":"2025-01-27T19:43:40.678510355Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1236,"task":"crutest"}
34738 90, response_code: 200, uri: /regioncount, method: GET, req_id: 1a83dc71-810f-425f-811c-701907a30946, remote_addr: 127.0.0.1:39659, local_addr: 127.0.0.1:9998
347392025-01-27T19:43:40.679ZINFOcrucible: dsc has 3 regions. This means 1 sub_volumes task = crutest
34740 Jan 27 19:43:40.679 INFO request completed, latency_us: 101, response_code: 200, uri: /port/cid/0, method: GET, req_id: 03b188d9-997e-4c55-b1ff-8766c8cdfdb5, remote_addr: 127.0.0.1:39659, local_addr: 127.0.0.1:9998
34741 Jan 27 19:43:40.679 INFO request completed, latency_us: 106, response_code: 200, uri: /port/cid/1, method: GET, req_id: a0b08fe8-f673-4155-b1a0-2c2ee2a7b1d1, remote_addr: 127.0.0.1:39659, local_addr: 127.0.0.1:9998
34742 Jan 27 19:43:40.680 INFO request completed, latency_us: 129, response_code: 200, uri: /port/cid/2, method: GET, req_id: 813ef9da-9ccc-4455-ac10-2986f590cff6, remote_addr: 127.0.0.1:39659, local_addr: 127.0.0.1:9998
347432025-01-27T19:43:40.680ZINFOcrucible: SV 0 has targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830] task = crutest
347442025-01-27T19:43:40.680ZINFOcrucible: Upstairs starts
347452025-01-27T19:43:40.680ZINFOcrucible: Crucible Version: BuildInfo { version: "0.0.1", git_sha: "d77aa90b6afe7d7ddb1dc63211fa4216c8866a90", git_commit_timestamp: "2025-01-27T19:20:16.000000000Z", git_branch: "mkeeter/shrink-replay-buffer", rustc_semver: "1.84.0", rustc_channel: "stable", rustc_host_triple: "x86_64-unknown-illumos", rustc_commit_sha: "9fc6b43126469e3858e2fe86cafb4f0fd5068869", cargo_triple: "x86_64-unknown-illumos", debug: true, opt_level: 0, }
347462025-01-27T19:43:40.680ZINFOcrucible: Upstairs <-> Downstairs Message Version: 12
347472025-01-27T19:43:40.680ZINFOcrucible: Using region definition RegionDefinition { block_size: 4096, extent_size: Block { value: 100, shift: 12 }, extent_count: 50, uuid: 00000000-0000-0000-0000-000000000000, encrypted: false, database_read_version: 1, database_write_version: 1 }
347482025-01-27T19:43:40.681ZINFOcrucible: Crucible f799725d-62e8-4ece-8332-dabeb84162bf has session id: 066847eb-0730-4bf3-a1b4-47ac637a7749 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347492025-01-27T19:43:40.681ZINFOcrucible: Upstairs opts: Upstairs UUID: f799725d-62e8-4ece-8332-dabeb84162bf, Targets: [127.0.0.1:8810, 127.0.0.1:8820, 127.0.0.1:8830], lossy: false, flush_timeout: None, key populated: false, cert_pem populated: false, key_pem populated: false, root_cert_pem populated: false, Control: None, read_only: false session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347502025-01-27T19:43:40.681ZINFOcrucible: Crucible stats registered with UUID: f799725d-62e8-4ece-8332-dabeb84162bf session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347512025-01-27T19:43:40.681ZWARNcrucible: could not find Downstairs address for Nexus session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347522025-01-27T19:43:40.682ZINFOcrucible: Downstairs in volume = 3 task = crutest
347532025-01-27T19:43:40.682ZINFOcrucible: The guest has requested activation with gen:3
347542025-01-27T19:43:40.682ZINFOcrucible: f799725d-62e8-4ece-8332-dabeb84162bf active request set session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347552025-01-27T19:43:40.682ZINFOcrucible: sending connect oneshot to client = downstairs client = 0 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347562025-01-27T19:43:40.682ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 0 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347572025-01-27T19:43:40.682ZINFOcrucible: sending connect oneshot to client = downstairs client = 1 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347582025-01-27T19:43:40.682ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 1 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347592025-01-27T19:43:40.682ZINFOcrucible: sending connect oneshot to client = downstairs client = 2 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347602025-01-27T19:43:40.682ZINFOcrucible: client set_active_request while in Connecting { state: Start { auto_promote: true }, mode: New }; waiting... = downstairs client = 2 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347612025-01-27T19:43:40.682ZINFOcrucible: connecting to 127.0.0.1:8830 = io task client = 2 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347622025-01-27T19:43:40.682ZINFOcrucible: connecting to 127.0.0.1:8810 = io task client = 0 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347632025-01-27T19:43:40.682ZINFOcrucible: connecting to 127.0.0.1:8820 = io task client = 1 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347642025-01-27T19:43:40.682ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:40944) = io task client = 2 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347652025-01-27T19:43:40.682ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:37755) = io task client = 0 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347662025-01-27T19:43:40.683ZINFOcrucible: ds_connection connected from Ok(127.0.0.1:58030) = io task client = 1 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347672025-01-27T19:43:40.683ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 2 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347682025-01-27T19:43:40.683ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 0 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347692025-01-27T19:43:40.683ZINFOcrucible: version negotiation from state Connecting { state: WaitForPromote, mode: New } = downstairs client = 1 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347702025-01-27T19:43:40.684ZINFOcrucible: downstairs client at Some(127.0.0.1:8830) has region UUID 12345678-0000-0000-0000-000000008830 = downstairs client = 2 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347712025-01-27T19:43:40.684ZINFOcrucible: downstairs client at Some(127.0.0.1:8810) has region UUID 12345678-0000-0000-0000-000000008810 = downstairs client = 0 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347722025-01-27T19:43:40.684ZINFOcrucible: downstairs client at Some(127.0.0.1:8820) has region UUID 12345678-0000-0000-0000-000000008820 = downstairs client = 1 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347732025-01-27T19:43:40.684ZINFOcrucible: Waiting for more clients to be ready session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347742025-01-27T19:43:40.685ZINFOcrucible: Waiting for more clients to be ready session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347752025-01-27T19:43:40.685ZINFOcrucible: [0]R flush_numbers[0..12]: [941, 930, 930, 941, 904, 941, 930, 911, 939, 933, 926, 941] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347762025-01-27T19:43:40.685ZINFOcrucible: [0]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347772025-01-27T19:43:40.685ZINFOcrucible: [0]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347782025-01-27T19:43:40.685ZINFOcrucible: [1]R flush_numbers[0..12]: [941, 930, 930, 941, 904, 941, 930, 911, 939, 933, 926, 941] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347792025-01-27T19:43:40.685ZINFOcrucible: [1]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347802025-01-27T19:43:40.685ZINFOcrucible: [1]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347812025-01-27T19:43:40.685ZINFOcrucible: [2]R flush_numbers[0..12]: [941, 930, 930, 941, 904, 941, 930, 911, 939, 933, 926, 941] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347822025-01-27T19:43:40.685ZINFOcrucible: [2]R generation[0..12]: [2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347832025-01-27T19:43:40.685ZINFOcrucible: [2]R dirty[0..12]: [false, false, false, false, false, false, false, false, false, false, false, false] = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347842025-01-27T19:43:40.685ZINFOcrucible: Max found gen is 3 = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
34785 {"msg":"Generation requested: 3 >= found:3","v":0,"name":"crucible","level":30Wait for a query_work_queue command to finish before sending IO
34786 ,"time":"2025-01-27T19:43:40.685236881Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1236,"":"downstairs","":"downstairs","session_id":"066847eb-0730-4bf3-a1b4-47ac637a7749"}
347872025-01-27T19:43:40.685ZINFOcrucible: Next flush: 942 = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347882025-01-27T19:43:40.685ZINFOcrucible: All extents match = downstairs session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347892025-01-27T19:43:40.685ZINFOcrucible: No downstairs reconciliation required session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347902025-01-27T19:43:40.685ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 0 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347912025-01-27T19:43:40.685ZINFOcrucible: Transition from WaitQuorum to Active = downstairs client = 1 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
34792 {"msgDisk: sv:1 bs:4096 ts:20480000 tb:5000 max_io:256 or 1048576
34793 ":"Transition from WaitQuorum to Active","v":0,"name":"crucible","level":30,"time":"2025-01-27T19:43:40.685377454Z","hostname":"w-01JJMM51R1R6GNKMPF161F4D5M","pid":1236,"client":"2","":"downstairs","session_id":"066847eb-0730-4bf3-a1b4-47ac637a7749"}
347942025-01-27T19:43:40.685ZINFOcrucible: All required reconciliation work is completed session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347952025-01-27T19:43:40.685ZINFOcrucible: Set Downstairs and Upstairs active after reconciliation session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347962025-01-27T19:43:40.685ZINFOcrucible: f799725d-62e8-4ece-8332-dabeb84162bf is now active with session: 066847eb-0730-4bf3-a1b4-47ac637a7749 session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347972025-01-27T19:43:40.685ZINFOcrucible: Set Active after no reconciliation session_id = 066847eb-0730-4bf3-a1b4-47ac637a7749
347982025-01-27T19:43:40.685ZINFOcrucible: The guest has finished waiting for activation with:3
34799 Loading write count information from file "/tmp/test_replay_verify.log"
34800 Read and Verify all blocks (0..5000 range:false)
34801 Wrote out file "/tmp/test_replay_verify.log"
34802 Verify test completed
34803 Wrote out file "/tmp/test_replay_verify.log"
34804 CLIENT: Tests done. All submitted work has been ACK'd
34805 ----------------------------------------------------------------
34806 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:20
34807 States: Active Active Active
34808 JOBID ACK TYPE BKS/EXT DS:0 DS:1 DS:2 REPLAY
34809 1000 Acked Read 256 Done Done Done false
34810 1001 Acked Read 256 Done Done Done false
34811 1002 Acked Read 256 Done Done Done false
34812 1003 Acked Read 256 Done Done Done false
34813 1004 Acked Read 256 Done Done Done false
34814 1005 Acked Read 256 Done Done Done false
34815 1006 Acked Read 256 Done Done Done false
34816 1007 Acked Read 256 Done Done Done false
34817 1008 Acked Read 256 Done Done Done false
34818 1009 Acked Read 256 Done Done Done false
34819 1010 Acked Read 256 Done Done Done false
34820 1011 Acked Read 256 Done Done Done false
34821 1012 Acked Read 256 Done Done Done false
34822 1013 Acked Read 256 Done Done Done false
34823 1014 Acked Read 256 Done Done Done false
34824 1015 Acked Read 256 Done Done Done false
34825 1016 Acked Read 256 Done Done Done false
34826 1017 Acked Read 256 Done Done Done false
34827 1018 Acked Read 136 Done Done Done false
34828 1019 Acked Read 256 Done Done Done false
34829 STATES DS:0 DS:1 DS:2 TOTAL
34830 Sent 0 0 0 0
34831 Done 20 20 20 60
34832 Skipped 0 0 0 0
34833 Error 0 0 0 0
34834 Last Flush: 0 0 0
34835 Downstairs last five completed:
34836 Upstairs last five completed: 1019 1018 1017 1016 1015
34837 CLIENT: Up:0 ds:20 act:3
34838 ----------------------------------------------------------------
34839 Crucible gen:3 GIO:true work queues: Upstairs:0 downstairs:0
34840 Downstairs last five completed: 1020 1019 1018 1017 1016
34841 Upstairs last five completed: 1020 1019 1018 1017 1016
34842 CLIENT: Up:0 ds:0 act:3
34843 CLIENT: All crucible jobs finished, exiting program
348442025-01-27T19:43:45.093ZWARNcrucible: Guest handle has been dropped
34845 Jan 27 19:43:45.157 INFO accepted connection, remote_addr: 127.0.0.1:51249, local_addr: 127.0.0.1:9998
34846 Main task has work to do, go find it
34847 got dsc Shutdown
34848 Shutdown
34849 Jan 27 19:43:45.158 INFO request completed, latency_us: 161Shut it down
34850 , [8810] Disable keep_running
34851 response_code: 204, uri:[8810] Got stop action so:false kr:false
34852 /shutdown, method: POST, req_id: 390aa63e-68c3-4fa3-b9c2-1de197ac803b, remote_addr: 127.0.0.1:51249, local_addr: 127.0.0.1:9998
34853 [8820] Disable keep_running
34854 [8820] Got stop action so:false kr:false
34855 [8830] Disable keep_running
34856 [8830] Got stop action so:false kr:false
34857 [8830] Exited with: ExitStatus(unix_wait_status(9))
34858 [8830][2] reports Exit
34859 [8820] Exited with: ExitStatus(unix_wait_status(9))
34860 [8810] Exited with: ExitStatus(unix_wait_status(9))
34861 [8820][1] reports Exit
34862 [8810][0] reports Exit
34863 January 27, 2025 at 07:43:50 PM UTC Test ends with 0